MTLD: Measure of Textual Lexical Diversity
- MTLD is a lexical diversity metric that computes the average factor length of token sequences where the type-token ratio remains above 0.72.
- It uses a sequential algorithm to count tokens until lexical repetition forces the ratio below a fixed threshold, reducing text-length bias.
- MTLD is widely applied in computational linguistics for corpus profiling, tracking writing development, and evaluating machine translation quality.
Measure of Textual Lexical Diversity (MTLD) is a lexical diversity index designed to quantify “the range and variety of vocabulary deployed in a text” while reducing the strong text-length bias of the simple type–token ratio (TTR) (Bestgen, 2023). In standard descriptions, MTLD is defined as the mean length of sequential token strings, or “factors,” that maintain a running TTR above a fixed threshold, usually $0.72$ (Bestgen, 2023). Because it operationalizes how long a text can sustain lexical novelty before repetition lowers TTR below that threshold, MTLD has become a standard tool in applied linguistics and computational text analysis, and it is often used as a measure of lexical diversity, lexical complexity, or lexical sophistication (Lin et al., 10 Apr 2025).
1. Conceptual basis
MTLD emerged as a response to the limitations of TTR, the foundational lexical diversity measure
where types are distinct words and tokens are running words (Dang et al., 28 Feb 2025). TTR is simple, but it is heavily biased by text length: shorter texts artificially inflate TTR, whereas longer texts depress it (Lin et al., 10 Apr 2025). Bestgen’s methodological review treats this as the classic length problem in lexical diversity research and places MTLD among the indices designed to address it by reducing texts to a common effective length through an algorithmic procedure rather than by using a single global ratio (Bestgen, 2023).
In this framework, MTLD is not merely another type–token formula. It is a sequential measure of how quickly lexical repetition accumulates as a text unfolds. The core intuition, stated across several papers, is that higher MTLD corresponds to longer stretches of text before the running TTR falls below the chosen threshold, and therefore to greater lexical diversity; lower MTLD indicates that repetition drives TTR downward more quickly (Dang et al., 28 Feb 2025). Applied studies often extend this interpretation from lexical diversity to lexical complexity or sophistication, especially when comparing writing across authors, domains, or time periods (Lin et al., 10 Apr 2025).
2. Computation and parameterization
Standard descriptions compute MTLD by traversing a token sequence in order, maintaining the number of tokens and distinct types in the current segment, and recalculating the segment TTR after each new token (Frens et al., 2018). When the running TTR falls to or below the threshold, conventionally $0.72$, the segment is counted as a completed factor, the counters are reset, and the procedure begins again on the next token (Lin et al., 10 Apr 2025). At the end of the text, any unfinished segment contributes a partial or fractional factor, computed in proportion to how close its final TTR is to the threshold (Frens et al., 2018).
The resulting score is conventionally summarized as
or, equivalently, as the average factor length in tokens (Lin et al., 10 Apr 2025). Some standard descriptions also compute MTLD in both forward and reverse directions and average the two values,
to reduce sensitivity to where rare words happen to occur in the text (Kambhatla et al., 23 May 2025).
The main control parameter is the factor threshold. The standard value is $0.72$, and Bestgen’s review treats this threshold as the MTLD parameter whose variation generates the second part of the “twofold length problem” (Bestgen, 2023). That review examines ten threshold values between $0.66$ and $0.75$, showing that changing the threshold affects both absolute MTLD values and, in some cases, the rank ordering of texts (Bestgen, 2023). This parameter sensitivity is not incidental; it is part of the metric’s behavior.
Implementation choices also matter. Large-scale English studies have computed MTLD on surface word forms without lemmatization (Frens et al., 2018), whereas broader lexical diversity literature notes that non-English applications sometimes use lemmatization to reduce morphological noise (Bestgen, 2023). Some studies compute one MTLD value per abstract or per chapter (Lin et al., 10 Apr 2025), while others compute it on book-length texts (Ploeger et al., 2024). This suggests that MTLD is best understood as a parameterized family of procedures rather than a single fully standardized number.
3. Length robustness and the “twofold length problem”
MTLD is widely described as length-robust, and comparative work supports that characterization relative to raw TTR and several classic alternatives. Bestgen groups MTLD with the top-performing indices for controlling the classic dependency on text length, alongside HD-D, MATTR, MSTTR, and MTTRSS, and clearly separates it from poorer performers such as TTR, Guiraud’s , Herdan’s , Maas’ 0, and MTTRRS (Bestgen, 2023). In that sense, MTLD succeeds at the problem it was designed to solve.
That success is nonetheless partial. Bestgen’s central claim is that lexical diversity measurement faces not one but two length problems: the classic dependence on overall text length, and a second dependence on the parameter that determines the effective length to which texts are reduced (Bestgen, 2023). For MTLD, that parameter is the factor threshold. Different texts can respond differently as the threshold changes, which means that rankings may cross when one moves from, for example, 1 to 2 (Bestgen, 2023). Bestgen also notes that highly diverse texts are particularly susceptible to distortions because they contain fewer factors and are therefore more affected by how the final incomplete factor is estimated (Bestgen, 2023).
Recent NLP work reaches a compatible conclusion. An autoencoder-based study treats MTLD as one of the better traditional metrics but reports a slight positive correlation between MTLD and work length across multiple languages, together with slope differences across languages that reflect sensitivity to segmentation configuration and language-specific characteristics (Dang et al., 28 Feb 2025). In other words, MTLD is more stable than TTR and VOCD, but it is not fully length-invariant.
Text length at the low end is also consequential. One general description states that MTLD is intended for texts of at least about 3–4 tokens because very short texts can yield unstable results (Kambhatla et al., 23 May 2025). A large fanfiction study operationalized that concern by removing 5 chapters with fewer than 6 words and excluding extreme MTLD values below 7 or above 8, many of which reflected non-narrative or pathological content such as emoticon strings, random words, and glossaries (Frens et al., 2018). The methodological implication is clear: MTLD is robust relative to naive ratios, but it still requires length-aware study design and data cleaning.
4. Position among lexical diversity measures
MTLD occupies a specific position within the broader ecology of lexical diversity metrics. Bestgen classifies it as a local index: unlike global “bag of words” measures, it respects textual sequence and reflects how repetitions accumulate within contiguous stretches of discourse (Bestgen, 2023). This distinguishes it from probabilistic global measures such as HD-D, which estimate expected type counts for fixed sample sizes using the hypergeometric distribution and, in Bestgen’s evaluation, are even more stable across text lengths (Bestgen, 2023).
Recent work in NLP has broadened this perspective. One study explicitly treats TTR, VOCD, and MTLD as traditional baseline metrics but argues that they “overlook sentence-level variations, contextual relationships, and duplication impact,” “treat all tokens equally, disregarding semantic and syntactic roles,” and are sensitive to dataset size and domain-specific characteristics (Dang et al., 28 Feb 2025). Another study frames lexical diversity as multi-dimensional and places repetition-based measures such as MATTR, and by implication MTLD-type indices, within only one dimension, “variety-repetition,” alongside other dimensions such as evenness, disparity, and dispersion (Kendro et al., 31 Jul 2025). A related study on persona-driven synthetic data likewise cites MTLD as part of the established lexical diversity literature but uses compression ratio, POS-based compression, n-gram diversity, self-repetition, and homogenization with BERTScore instead (Kambhatla et al., 23 May 2025).
Alternative frameworks can also redefine the unit of diversity. A study of LLM-generated news text does not use MTLD at all, instead computing Shannon entropy and the Simpson diversity index over Head-Driven Phrase Structure Grammar lexical types rather than word forms or lemmas (Gude et al., 7 May 2026). This suggests a broader methodological point: MTLD measures one specific aspect of lexical behavior—the rate at which new surface forms stop offsetting repetition—and not diversity at every representational level.
5. Empirical applications
MTLD has been used extensively in large-scale observational studies of writing development. In a study of 9 million authors on Fanfiction.net, lexical diversity was operationalized solely as MTLD, and the final analysis covered $0.72$0 English chapters with normally distributed MTLD scores, mean $0.72$1 and standard deviation $0.72$2 (Frens et al., 2018). Within that corpus, MTLD increased by about $0.72$3 points per year in late adolescence, and cumulative reviews had a positive association with MTLD; the authors note that receiving roughly $0.72$4 reviews predicted the same increase in lexical diversity as one year of maturation (Frens et al., 2018). In that setting, MTLD functions as a longitudinal proxy for vocabulary breadth in narrative writing.
A recent causal analysis of academic writing uses MTLD even more centrally. Across $0.72$5 million English-language OpenAlex abstracts from $0.72$6 to $0.72$7, MTLD serves as the primary dependent variable in difference-in-differences and event-study models designed to estimate the effect of ChatGPT on the lexical complexity of non-native English scholarly writing (Lin et al., 10 Apr 2025). The estimated post-ChatGPT increase in MTLD for non-native English authors relative to native-English controls ranges from about $0.72$8 to $0.72$9 MTLD units across specifications, rises to about 0 in preprints, and is largest in technology fields (1) and substantial in biology (2) (Lin et al., 10 Apr 2025). Here MTLD is treated ոչ only as a descriptive index but as a scalable outcome variable for causal inference.
MTLD has also been used to diagnose and partially optimize machine translation. In literary English-to-Dutch translation across 3 books, human translations averaged MTLD 4, whereas vanilla machine translation averaged 5 (Ploeger et al., 2024). A tagging approach increased MTLD to 6, and tailored reranking with top-7 sampling reached 8, overshooting the human-translation average while trading off BLEU and COMET (Ploeger et al., 2024). In that study, MTLD is not only evaluative; it is part of a book-level lexical diversity target used to steer generation toward the diversity levels associated with individual novels.
6. Interpretation, limitations, and current status
MTLD remains attractive because it is interpretable, computationally cheap, and relatively robust to text length compared with raw TTR (Dang et al., 28 Feb 2025). Higher values can be read straightforwardly as longer spans of text before lexical repetition depresses TTR below the factor threshold. This makes MTLD useful for descriptive corpus profiling, within-language comparisons, and large-scale automatic analysis (Dang et al., 28 Feb 2025).
Its limitations are equally well documented. Recent critiques emphasize that MTLD is still a count-based measure: it uses types and tokens, not syntax or semantics; it cannot capture sentence structure, long-range dependencies, or contextual usage of words; and it may miss losses in structural variety when duplication changes discourse patterns without drastically changing type counts (Dang et al., 28 Feb 2025). Multi-dimensional work on lexical diversity further implies that MTLD is only one component of a richer construct that also includes evenness, disparity, dispersion, and, in some taxonomies, specialness (Kendro et al., 31 Jul 2025).
Several common misconceptions follow from these limitations. MTLD is not a general measure of “vocabulary quality,” and it is not a direct measure of semantic richness or communicative adequacy (Dang et al., 28 Feb 2025). It is also not length-invariant in any absolute sense: threshold choice, preprocessing, language-specific morphology, and the treatment of short or pathological texts all affect the resulting score (Bestgen, 2023). This suggests that MTLD is most reliable when used as an explicitly parameterized measure of variety-repetition, ideally alongside complementary indices that capture dimensions of lexical, structural, or semantic diversity that MTLD was never designed to represent.
