What Is a Token, Really? On Turkish, AI, and the Shape of Meaning
- Seda
- 29 minutes ago
- 7 min read
Before you learn the grammar of a language, you learn what it feels like to stand outside it. Every word you meet at first arrives as something borrowed, a shape someone else built long before you existed. You do not invent meaning when you learn a new language. You inherit it, the way a guest inherits the layout of a house that already had its walls up before they arrived.
There is a difference between visiting a language and living inside it as its host. A visitor moves carefully, unsure where anything belongs. A host stops noticing the room at all, because its layout has become instinct rather than information. Learning Turkish is largely the slow work of turning a visitor into a host.
I wrote kitaplarımızdan on the whiteboard during a lesson. One of my students looked at it for a moment and asked, "Is that really one word?" It is one word, and it holds four ideas at once: book, plural, our, and from. To me it arrives as a single thought. To someone still visiting the language, it arrives as four separate rooms with no obvious doors between them.
An AI language model runs into a version of the same problem before it can respond to anything you write, and the gap between how a native speaker reads that word and how a model processes it turns out to explain more than it first appears to.
What a Token Actually Is
Before a system like ChatGPT or Claude can respond to anything you write, it first breaks your sentence into small pieces called tokens. A token is simply the unit of text the model works with internally. Sometimes a token is a whole word. Sometimes it is only part of a word, a couple of letters, or a single character. The model does not read language the way you do, in full words with obvious edges. It reads whatever pieces its tokenizer produces.
This step happens before any higher-level processing takes place, and it shapes everything that follows. If a word is divided in a way that respects its internal structure, the model has an easier time connecting those pieces into useful representations. If the word is divided across meaningful suffix boundaries, that process becomes less efficient.
Why Turkish Breaks the Usual Rules
Turkish is agglutinative. Meaning is built by stacking suffixes onto a root, one after another, and each suffix adds a specific, predictable layer of information. English does a small version of this with endings like -ed or -ing, but Turkish does it constantly and at scale. A single root can carry six, eight, or even ten suffixes and still sound completely natural.
Most widely used tokenizers were not designed with this structure in mind. They were built around statistical frequency. A tokenizer examines enormous amounts of text, finds recurring character sequences, and treats those sequences as useful building blocks. This works well for languages where grammatical information is spread across separate words. In Turkish, the same strategy often cuts directly through suffix boundaries, leaving the model with fragments that no longer correspond to meaningful grammatical units.
A New Attempt: Teaching a Tokenizer to Read Suffixes as Suffixes
A study published in 2025 and revised in March 2026 by M. Ali Bayram and colleagues explored a different approach. Instead of relying only on statistical frequency, the researchers built a tokenizer designed around Turkish morphology. It uses a dictionary of roughly 20,000 root identifiers and 72 affix categories, while recognizing that endings such as -dan and -tan represent the same grammatical function even though their spelling changes through consonant harmony.
The results were measurable. On Turkish-language benchmarks, this tokenizer produced a much higher proportion of linguistically meaningful tokens than the tokenizers used by models such as Gemma or GPT. When the researchers trained identical model architectures from scratch, the morphology-aware version performed better on Turkish sentence similarity tasks and on benchmarks designed to detect subtle grammatical distinctions.
The Trade-off Nobody Mentions First
This was the finding that stayed with me the longest.
The tokenizer actually produces more tokens per word, not fewer. Because every suffix is treated as its own meaningful unit, a Turkish sentence is divided into more pieces than it would be under a simpler frequency-based tokenizer. The researchers considered that a worthwhile trade-off because those additional pieces correspond to real grammatical information instead of arbitrary fragments.
Understanding Turkish more precisely comes at the cost of processing more units.
What This Has to Do With Your Own Mind
Cognitive psychologists have a name for a closely related idea: chunking. Working memory can hold only a limited number of units at once, and what counts as a unit changes with experience. A chess master looks at a board and recognizes familiar patterns. A beginner sees individual pieces. The expert is not remembering more. The expert is remembering larger chunks.
Turkish presents a similar challenge.
To me, kitaplarımızdan is one chunk. To a learner whose first language is English, French, or German, where meaning is usually distributed across separate words and prepositions instead of suffixes, the same word often arrives as four or five separate pieces. Each demands attention. Each has to be interpreted in the correct order. Even vowel harmony changes the form of each ending.
This is not simply a matter of vocabulary. The U.S. Foreign Service Institute estimates that native English speakers generally need roughly twice as much classroom time to reach professional proficiency in Turkish as they do in languages such as French or Spanish. Many linguists and experienced teachers point to Turkish morphology and sentence structure as major reasons for that difference. Speakers of Finnish or Hungarian, whose languages are also agglutinative, often report adapting to Turkish suffixation more easily because the underlying structural pattern already feels familiar.
The comparison between a tokenizer and a human learner deserves caution. A tokenizer is an engineering design. It has no awareness and no experience. A person gradually reorganizes expectations through repeated exposure until separate suffixes stop demanding conscious attention. The mechanisms are different. Yet the shape of the challenge, encountering a language whose structure differs from what you already expect, may be more similar than it first appears.
I have written before about how Turkish word order, evidentiality, and the absence of grammatical gender influence the way learners gradually interpret meaning. The same underlying idea appears here as well. Structure changes what becomes easy to notice.
A language model gradually builds statistical expectations from its training data. A person gradually builds expectations about a language through years of hearing it spoken or, in the case of a second language, through sustained exposure and practice. Neither begins with Turkish already organized in memory. Over time, both adapt to the structure they encounter, though through fundamentally different processes.
The visitor becomes a host by allowing those expectations to reorganize around the house they are learning to live in.
A Question I Am Not Ready to Answer
So this is the question I keep returning to.
When a student tells me a Turkish sentence feels like too many pieces at once, is that experience structurally similar to what an undertrained tokenizer encounters when it processes a language it was never designed around? Or is the comparison only a useful metaphor?
I do not think the two processes should be treated as equivalent. One belongs to engineering, the other to human cognition. Even so, both involve learning to recognize larger patterns where, at first, only disconnected fragments seem to exist.
Perhaps that shared experience is meaningful. Or perhaps it simply gives us a better language for describing what learning Turkish feels like during the earliest stages.
Frequently Asked Questions (FAQ)
Q: What is a token, in simple terms?
A: A token is the small unit of text an AI language model processes internally. A token may be a whole word, part of a word, or even a single character. Every prompt is divided into tokens before the model begins generating a response.
Q: Why is Turkish difficult for AI tokenizers?
A: Turkish builds meaning by attaching multiple suffixes to a single root. Many widely used tokenizers were designed around statistical frequency rather than Turkish morphology, so they may split words at positions that do not align with meaningful grammatical boundaries.
Q: Does a morphology-aware tokenizer solve Turkish language understanding?
A: No. The study shows that using Turkish morphology during tokenization improves performance on several Turkish-language benchmarks. It also increases the number of tokens that must be processed. It is an important improvement, although it does not solve every challenge of understanding Turkish.
Q: Is learning Turkish similar to the way an AI model processes language?
A: Only as a metaphor. Human language learning and AI tokenization rely on very different mechanisms. The comparison in this article highlights a similar structural challenge: both encounter difficulty when meaningful units are divided into unfamiliar pieces.
Q: Why does one Turkish word sometimes feel like several words?
A: Turkish often combines information that English expresses with separate words into a single word through suffixes. New learners usually process each suffix separately at first. With experience, those separate pieces gradually become larger
mental chunks.
Sources
Bayram, M. Ali, Ali Arda Fincan, Ahmet Semih Gümüş, Sercan Karakaş, Banu Diri, Savaş Yıldırım, and Demircan Çelik. Tokens with Meaning: A Hybrid Tokenization Approach for Turkish. arXiv:2508.14292. Submitted August 2025. Revised March 2026.
U.S. Department of State, Foreign Service Institute. Foreign Language Training Categories.
Miller, George A. "The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information." Psychological Review 63, no. 2 (1956): 81-97.
Gobet, Fernand, William G. Chase, and Herbert A. Simon. "Chunking Mechanisms in Human Learning." Trends in Cognitive Sciences 5, no. 6 (2001): 236-243.
Newell, Allen, and Herbert A. Simon. Human Problem Solving. Englewood Cliffs, NJ: Prentice-Hall, 1972.