---
title: 'MTLD: Measure of Textual Lexical Diversity'
url: https://www.emergentmind.com/topics/measure-of-textual-lexical-diversity-mtld
type: topic
---

# MTLD: Measure of Textual Lexical Diversity

Measure of Textual Lexical Diversity (MTLD) is a lexical diversity index designed to quantify “the range and variety of vocabulary deployed in a text” while reducing the strong text-length bias of the simple type–token ratio (TTR) [2307.04626]. In standard descriptions, MTLD is defined as the mean length of sequential token strings, or “factors,” that maintain a running TTR above a fixed threshold, usually \(0.72\) [2307.04626]. Because it operationalizes how long a text can sustain lexical novelty before repetition lowers TTR below that threshold, MTLD has become a standard tool in applied linguistics and computational text analysis, and it is often used as a measure of lexical diversity, lexical complexity, or lexical sophistication [2504.12317].

## 1. Conceptual basis

MTLD emerged as a response to the limitations of TTR, the foundational lexical diversity measure

$$
\text{TTR} = \frac{\text{number of types}}{\text{number of tokens}},
$$

where types are distinct words and tokens are running words [2503.00209]. TTR is simple, but it is heavily biased by text length: shorter texts artificially inflate TTR, whereas longer texts depress it [2504.12317]. Bestgen’s methodological review treats this as the classic length problem in lexical diversity research and places MTLD among the indices designed to address it by reducing texts to a common effective length through an algorithmic procedure rather than by using a single global ratio [2307.04626].

In this framework, MTLD is not merely another type–token formula. It is a sequential measure of how quickly lexical repetition accumulates as a text unfolds. The core intuition, stated across several recent papers, is that higher MTLD corresponds to longer stretches of text before the running TTR falls below the chosen threshold, and therefore to greater lexical diversity; lower MTLD indicates that repetition drives TTR downward more quickly [2503.00209]. Applied studies often extend this interpretation from lexical diversity to lexical complexity or sophistication, especially when comparing writing across authors, domains, or time periods [2504.12317].

## 2. Computation and parameterization

Standard descriptions compute MTLD by traversing a token sequence in order, maintaining the number of tokens and distinct types in the current segment, and recalculating the segment TTR after each new token [1809.10268]. When the running TTR falls to or below the threshold, conventionally \(0.72\), the segment is counted as a completed factor, the counters are reset, and the procedure begins again on the next token [2504.12317]. At the end of the text, any unfinished segment contributes a partial or fractional factor, computed in proportion to how close its final TTR is to the threshold [1809.10268].

The resulting score is conventionally summarized as

$$
\text{MTLD} = \frac{\text{Total number of tokens}}{\text{Number of (possibly fractional) factors}},
$$

or, equivalently, as the average factor length in tokens [2504.12317]. Some standard descriptions also compute MTLD in both forward and reverse directions and average the two values,

$$
\text{MTLD} = \frac{\text{MTLD}_{\text{forward}} + \text{MTLD}_{\text{reverse}}}{2},
$$

to reduce sensitivity to where rare words happen to occur in the text [2505.17390].

The main control parameter is the factor threshold. The standard value is \(0.72\), and Bestgen’s review treats this threshold as the MTLD parameter whose variation generates the second part of the “twofold length problem” [2307.04626]. That review examines ten threshold values between \(0.66\) and \(0.75\), showing that changing the threshold affects both absolute MTLD values and, in some cases, the rank ordering of texts [2307.04626]. This parameter sensitivity is not incidental; it is part of the metric’s behavior.

Implementation choices also matter. Large-scale English studies have computed MTLD on surface word forms without lemmatization [1809.10268], whereas broader lexical diversity literature notes that non-English applications sometimes use lemmatization to reduce morphological noise [2307.04626]. Some studies compute one MTLD value per abstract or per chapter [2504.12317], while others compute it on book-length texts [2408.17308]. This suggests that MTLD is best understood as a parameterized family of procedures rather than a single fully standardized number.

## 3. Length robustness and the “twofold length problem”

MTLD is widely described as length-robust, and comparative work supports that characterization relative to raw TTR and several classic alternatives. Bestgen groups MTLD with the top-performing indices for controlling the classic dependency on text length, alongside HD-D, MATTR, MSTTR, and MTTRSS, and clearly separates it from poorer performers such as TTR, Guiraud’s \(R\), Herdan’s \(C\), Maas’ \(a\), and MTTRRS [2307.04626]. In that sense, MTLD succeeds at the problem it was designed to solve.

That success is nonetheless partial. Bestgen’s central claim is that lexical diversity measurement faces not one but two length problems: the classic dependence on overall text length, and a second dependence on the parameter that determines the effective length to which texts are reduced [2307.04626]. For MTLD, that parameter is the factor threshold. Different texts can respond differently as the threshold changes, which means that rankings may cross when one moves from, for example, \(0.66\) to \(0.75\) [2307.04626]. Bestgen also notes that highly diverse texts are particularly susceptible to distortions because they contain fewer factors and are therefore more affected by how the final incomplete factor is estimated [2307.04626].

Recent NLP work reaches a compatible conclusion. An autoencoder-based study treats MTLD as one of the better traditional metrics but reports a slight positive correlation between MTLD and work length across multiple languages, together with slope differences across languages that reflect sensitivity to segmentation configuration and language-specific characteristics [2503.00209]. In other words, MTLD is more stable than TTR and VOCD, but it is not fully length-invariant.

Text length at the low end is also consequential. One general description states that MTLD is intended for texts of at least about \(100\)–\(150\) tokens because very short texts can yield unstable results [2505.17390]. A large fanfiction study operationalized that concern by removing \(427{,}662\) chapters with fewer than \(100\) words and excluding extreme MTLD values below \(5\) or above \(300\), many of which reflected non-narrative or pathological content such as emoticon strings, random words, and glossaries [1809.10268]. The methodological implication is clear: MTLD is robust relative to naive ratios, but it still requires length-aware study design and data cleaning.

## 4. Position among lexical diversity measures

MTLD occupies a specific position within the broader ecology of lexical diversity metrics. Bestgen classifies it as a local index: unlike global “bag of words” measures, it respects textual sequence and reflects how repetitions accumulate within contiguous stretches of discourse [2307.04626]. This distinguishes it from probabilistic global measures such as HD-D, which estimate expected type counts for fixed sample sizes using the hypergeometric distribution and, in Bestgen’s evaluation, are even more stable across text lengths [2307.04626].

Recent work in NLP has broadened this perspective. One study explicitly treats TTR, VOCD, and MTLD as traditional baseline metrics but argues that they “overlook sentence-level variations, contextual relationships, and duplication impact,” “treat all tokens equally, disregarding semantic and syntactic roles,” and are sensitive to dataset size and domain-specific characteristics [2503.00209]. Another study frames lexical diversity as multi-dimensional and places repetition-based measures such as MATTR, and by implication MTLD-type indices, within only one dimension, “variety-repetition,” alongside other dimensions such as evenness, disparity, and dispersion [2508.00086]. A related study on persona-driven synthetic data likewise cites MTLD as part of the established lexical diversity literature but uses compression ratio, POS-based compression, n-gram diversity, self-repetition, and homogenization with BERTScore instead [2505.17390].

Alternative frameworks can also redefine the unit of diversity. A study of LLM-generated news text does not use MTLD at all, instead computing Shannon entropy and the Simpson diversity index over Head-Driven Phrase Structure Grammar lexical types rather than word forms or lemmas [2605.06030]. This suggests a broader methodological point: MTLD measures one specific aspect of lexical behavior—the rate at which new surface forms stop offsetting repetition—and not diversity at every representational level.

## 5. Empirical applications

MTLD has been used extensively in large-scale observational studies of writing development. In a study of \(1.5\) million authors on Fanfiction.net, lexical diversity was operationalized solely as MTLD, and the final analysis covered \(24{,}835{,}868\) English chapters with normally distributed MTLD scores, mean \(97.35\) and standard deviation \(21.96\) [1809.10268]. Within that corpus, MTLD increased by about \(1.66\) points per year in late adolescence, and cumulative reviews had a positive association with MTLD; the authors note that receiving roughly \(650\) reviews predicted the same increase in lexical diversity as one year of maturation [1809.10268]. In that setting, MTLD functions as a longitudinal proxy for vocabulary breadth in narrative writing.

A recent causal analysis of academic writing uses MTLD even more centrally. Across \(2.8\) million English-language OpenAlex abstracts from \(2020\) to \(2024\), MTLD serves as the primary dependent variable in difference-in-differences and event-study models designed to estimate the effect of ChatGPT on the lexical complexity of non-native English scholarly writing [2504.12317]. The estimated post-ChatGPT increase in MTLD for non-native English authors relative to native-English controls ranges from about \(0.05\) to \(0.08\) MTLD units across specifications, rises to about \(0.38\) in preprints, and is largest in technology fields (\(0.185\)) and substantial in biology (\(0.080\)) [2504.12317]. Here MTLD is treated ոչ only as a descriptive index but as a scalable outcome variable for causal inference.

MTLD has also been used to diagnose and partially optimize machine translation. In literary English-to-Dutch translation across \(31\) books, human translations averaged MTLD \(96.05\), whereas vanilla machine translation averaged \(90.21\) [2408.17308]. A tagging approach increased MTLD to \(94.08\), and tailored reranking with top-\(k\) sampling reached \(104.25\), overshooting the human-translation average while trading off BLEU and COMET [2408.17308]. In that study, MTLD is not only evaluative; it is part of a book-level lexical diversity target used to steer generation toward the diversity levels associated with individual novels.

## 6. Interpretation, limitations, and current status

MTLD remains attractive because it is interpretable, computationally cheap, and relatively robust to text length compared with raw TTR [2503.00209]. Higher values can be read straightforwardly as longer spans of text before lexical repetition depresses TTR below the factor threshold. This makes MTLD useful for descriptive corpus profiling, within-language comparisons, and large-scale automatic analysis [2503.00209].

Its limitations are equally well documented. Recent critiques emphasize that MTLD is still a count-based measure: it uses types and tokens, not syntax or semantics; it cannot capture sentence structure, long-range dependencies, or contextual usage of words; and it may miss losses in structural variety when duplication changes discourse patterns without drastically changing type counts [2503.00209]. Multi-dimensional work on lexical diversity further implies that MTLD is only one component of a richer construct that also includes evenness, disparity, dispersion, and, in some taxonomies, specialness [2508.00086].

Several common misconceptions follow from these limitations. MTLD is not a general measure of “vocabulary quality,” and it is not a direct measure of semantic richness or communicative adequacy [2503.00209]. It is also not length-invariant in any absolute sense: threshold choice, preprocessing, language-specific morphology, and the treatment of short or pathological texts all affect the resulting score [2307.04626]. This suggests that MTLD is most reliable when used as an explicitly parameterized measure of variety-repetition, ideally alongside complementary indices that capture dimensions of lexical, structural, or semantic diversity that MTLD was never designed to represent.

Source: https://www.emergentmind.com/topics/measure-of-textual-lexical-diversity-mtld