Language AI Readiness Index (EQUATE)

• The Language AI Readiness Index (EQUATE) is a composite measure that evaluates the preparedness of attested languages and their communities for equitable AI development.
• It incorporates multidimensional factors—including AI resources, digital infrastructure, and socioeconomic conditions—to reveal disparities and growth potential in language AI.
• The index employs rigorous methods like PCA, normalization, and geometric aggregation to expose linguistic inequality and highlight underutilized language AI opportunities.

Searching arXiv for papers on EQUATE and related language AI readiness frameworks. The Language AI Readiness Index (EQUATE) is a language-centered composite index introduced to assess “the urgency and feasibility of AI development for all attested languages” and to measure the readiness of languages and their speaker communities for language AI across multiple geographic scales. It was proposed in response to the claim that language AI is diffusing through a new global linguistic hierarchy in which a small number of dominant languages accumulate models, datasets, and institutional attention, while most of the world’s linguistic communities remain digitally marginalized. EQUATE therefore shifts the unit of analysis from the nation-state to the language, and from simple model-counting to a broader assessment of AI resources, digital infrastructure, and socioeconomic conditions (Occhini et al., 12 Feb 2026).

1. Origins in linguistic inequality and the move from countries to languages

EQUATE emerged from a broader argument that language AI is not diffusing like earlier information technologies. Using Hugging Face data from 2020–2024, validated against language representation in the ACL Anthology, the underlying study reports that models and datasets per language follow a power-law distribution and are becoming more concentrated over time, a process described as “Zipfianisation.” English is treated as an outlier, with model growth exceeding what a simple Zipfian distribution would predict. The paper contrasts this pattern with the S-shaped diffusion curves associated with mobile phones, broadband, personal computers, and electric vehicles, and instead characterizes language AI as exhibiting a “hype-driven pattern of spread” and a “two-stage dynamic (hyper-growth followed by lock-in)” (Occhini et al., 12 Feb 2026).

This framing matters because national AI readiness indices can obscure linguistic inequality within multilingual states. EQUATE addresses that blind spot by treating the language as the primary unit of analysis while still attaching languages to speaker communities and country-specific contexts. The index covers 6,003 attested languages across 217 countries and territories. Its data architecture operates at three geographic levels, depending on availability: the language population level, the first subnational unit, and the country level. The source inventory and language centroids derive from Glottolog, with documentation of vocabulary additionally checked through PanLex and scripture resources (Occhini et al., 12 Feb 2026).

The motivating critique is not only that low-resource languages have fewer models, but that current coverage statistics may overstate meaningful inclusion. The paper explicitly warns against “representation washing”: multilingual model inventories can suggest broad linguistic inclusion even when actual capability and benefit remain concentrated in dominant languages. EQUATE is intended to distinguish such shallow representation from conditions under which language AI can actually be developed, deployed, and used.

2. Readiness as a multidimensional sociotechnical construct

EQUATE defines readiness through three domains: AI resources, digital infrastructure, and socioeconomic conditions. The central claim is that language AI readiness is irreducibly multidimensional: a language may have some data but weak infrastructure, or strong infrastructure and institutions but little existing AI investment. One of the index’s key conceptual contributions is the distinction between under-resourced and underutilized languages. The former lack core prerequisites for immediate development; the latter possess substantial enabling conditions but remain overlooked by AI developers (Occhini et al., 12 Feb 2026).

Domain	Examples of variables	Readiness function
AI resources	datasets, models, Common Crawl, OPUS, Wikipedia, archival resources, Bible translations	Captures existing technical artifacts and digital language resources
Digital infrastructure	household access to computers, phones, home internet, internet use, download/upload speed, latency	Captures whether communities can develop, deploy, or benefit from AI
Socioeconomic conditions	number of speakers, HDI, GDP per capita, education, literacy, R&D expenditure, cybersecurity legislation, distance to university	Captures human capital, institutions, and enabling social capacity

Within the AI resources domain, the paper lists variables such as the number of speakers, Bible translations, number of models, data volume from Common Crawl, OPUS, and Wikipedia, archival resources from repositories including AILLA, ELAR, and PARADISEC, and XEUS. Survey results reported in the same study also mention candidate indicators such as monolingual and multilingual LLMs, existence of a dictionary, typological similarity to well-resourced language families, audio data volume, and online active Wikipedia users. Not all of these are clearly confirmed as final included variables in the published formula, but they define the intended scope of the resource dimension (Occhini et al., 12 Feb 2026).

The digital infrastructure domain includes percentages of households with computers, phones, and home internet; percentages of individuals using the internet; and network-quality measures such as average download speed, average upload speed, and average latency. The socioeconomic domain includes subnational HDI, subnational GDP per capita, educational attainment, literacy, distance to the nearest university, cybersecurity legislation, R&D expenditure as a share of GDP, and the percentage of STEM graduates. In the paper’s interpretation, these variables matter because language technology is not only a corpus problem: it also depends on institutions, educational systems, connectivity, and innovation ecosystems (Occhini et al., 12 Feb 2026).

3. Data architecture, preprocessing, and composite-score construction

The paper describes EQUATE as being built from 25 linguistic and subnational features, while its PCA and correlation discussion refers to 24 variables; the text does not fully reconcile this discrepancy. The score construction is nonetheless specified as a multi-stage pipeline. Highly correlated features within a group are merged when Pearson correlation exceeds 0.85. All non-binary numeric features are shifted upward by 1 to avoid zero terms in later geometric aggregation. Features suspected of exponential behavior are tested against the exponential distribution using the Kolmogorov–Smirnov test, with optional log transformation when appropriate. Continuous features are then min–max normalized to $[0,1]$, with a small constant imposed to avoid zeros (Occhini et al., 12 Feb 2026).

Within each feature group, normalized variables are aggregated by weighted geometric mean. Across groups, the grouped scores are again combined geometrically. Binary variables enter multiplicatively as penalties when absent. In the paper’s notation, if $G_{i,g}$ is the grouped score for language $i$ in group $g$, and $P_{i,b}$ is the penalty for binary variable $b$, the final readiness score is:

$$S_i = \left( \prod_{g=1}^{G} G_{i,g} \right)^{1/G} \cdot \prod_{b \in B} \left( 1 - w_b (1 - x_{i,b}) \right)$$

This design encodes a substantive assumption: readiness is weakest where bottlenecks are most severe. The use of geometric aggregation penalizes imbalance more strongly than an arithmetic mean would. A language cannot compensate indefinitely for weak infrastructure with strong data resources, or for strong socioeconomic conditions with absent digital access (Occhini et al., 12 Feb 2026).

The weighting strategy combines empirical and expert components. The paper states that EQUATE was developed in two stages: first, correlational and dimensionality analysis to determine empirical structure and non-redundancy; second, a global expert survey to assign relative importance to factors. The reported participant tables list 28 NLP/multilingual AI respondents and 20 non-NLP sociotechnical respondents. The survey yields ranked priorities rather than explicit final numerical weights. For example, top-ranked AI-resource items include Common Crawl bytes, Wikipedia bytes, and already available multilingual LLMs; top-ranked digital infrastructure items include households with phones, home internet, and network speed; top-ranked socioeconomic items include institutional status, number of speakers, and vitality status. The paper does not disclose the exact final weights used in the released index (Occhini et al., 12 Feb 2026).

Missing data are handled by geographical interpolation. If a subnational value is missing but other regions in the same country have data, the missing value is replaced by the average of observed regions in that country. If all regional values are missing, the paper states that a value is borrowed from a country with a similar development level, determined by a composite development index such as HDI or GDP per capita rather than by geographic proximity. The exact similarity rule is not specified (Occhini et al., 12 Feb 2026).

4. Validation, score interpretation, and empirical findings

EQUATE is validated structurally rather than against a gold-standard external readiness benchmark. The correlation matrix reportedly separates into two broad clusters: one centered on AI resources, the other on socioeconomic plus digital infrastructure. PCA retains two components explaining 58.4% of total variance. After varimax rotation, PC1 is dominated by social and digital infrastructure variables, with strongest loadings from individuals using the internet $(0.35)$, HDI $(0.35)$, and education level $(0.33)$, while latency loads negatively $G_{i,g}$0. PC2 captures AI resources, with strongest loadings from Wikipedia active users $G_{i,g}$1, Common Crawl GB $G_{i,g}$2, and number of Hugging Face datasets $G_{i,g}$3 (Occhini et al., 12 Feb 2026).

The study also models the number of available AI models as an outcome in mixed-effects regression. In the stepwise model, significant positive predictors include Bible existence $G_{i,g}$4, number of speakers $G_{i,g}$5, OPUS GB $G_{i,g}$6, and XEUS GB $G_{i,g}$7. A PCA-based mixed-effects model finds both principal components significant predictors of model counts: the AI-resources PC $G_{i,g}$8 and the socioeconomic/digital-infrastructure PC $G_{i,g}$9. The PCA-based model is reported as having substantially better fit, with $i$0 versus $i$1 for the fuller model. The paper treats this as evidence that language AI readiness is genuinely multidimensional rather than reducible to resource counts alone (Occhini et al., 12 Feb 2026).

In substantive interpretation, a high EQUATE score means that a language community has stronger structural readiness for language AI: more technical resources, more enabling infrastructure, and more favorable socioeconomic conditions. A low score means that key prerequisites for sustainable development and deployment are weak or missing. Crucially, the authors stress that readiness is not the same as current AI coverage. A language can have low current model counts but still score relatively high if surrounding conditions suggest that targeted investment would be feasible and effective. This is the paper’s category of “underutilized potential” (Occhini et al., 12 Feb 2026).

The paper’s most prominent empirical application is the identification of languages that appear more ready than their current AI coverage suggests. It reports that more than a thousand languages have substantial data coverage, even when Bible translations are excluded, and that 597 languages, mainly in Africa and Asia, have fewer than three models but nonetheless exhibit characteristics consistent with AI readiness. It highlights Hakka Chinese and Darija (Moroccan Arabic) as examples of communities whose socioeconomic and infrastructural readiness may be stronger than their AI coverage would imply. By contrast, Esperanto is presented as a case of disproportionate AI attention relative to social utility. At the broader regional level, many under-resourced languages are concentrated in Sub-Saharan Africa, South Asia, and the Middle East, while overrepresented languages are overwhelmingly European, including dead or historical languages such as Latin, Ancient Greek, and Old English (Occhini et al., 12 Feb 2026).

The index is operationalized as an open-access and open-source tool at https://www.equate-index.ai/. The interface allows languages to be ranked globally or country-specifically, inspected by overall score or by individual dimension, and explored through an interactive map with filters by ranking and minimum speaker count. The authors state that the index will be updated annually (Occhini et al., 12 Feb 2026).

5. Relation to adjacent frameworks and domain-specific evidence

Although EQUATE is defined as a language-centered readiness index, later and adjacent work shows that readiness can be decomposed further into capability, institutional, pedagogical, and governance layers. A qualitative study of Bangladesh argues that AI readiness “extends beyond infrastructure” and includes “curriculum design, workforce development, and cross-sector collaboration.” That study does not provide a formal scoring model, but it identifies readiness as distributed across government vision, academic capacity, industry absorption, computing access, inclusion, and ethics, thereby offering a sociotechnical template for interpreting EQUATE scores in Global South settings (Sultana et al., 19 Jan 2026).

At the capability layer, the AI Language Proficiency Monitor provides a benchmark-based measure of multilingual model performance across up to 200 languages, with a Language Proficiency Score defined as the mean of min–max-normalized task metrics across translation, classification, QA, reasoning, truthfulness, and math. It is explicitly presented as a continuously updated benchmarking system rather than a full readiness index, but it operationalizes one important EQUATE-adjacent component: comparable language-level capability measurement across many models and tasks (Pomerenke et al., 11 Jul 2025). A more localized example is Alvorada-Bench, a 4,515-question benchmark based on five Brazilian university entrance examinations. Its main value for readiness analysis is to show that educational capability in Portuguese depends jointly on language, culture, and reasoning, and that high aggregate scores can coexist with persistent weaknesses in mathematics and engineering-style exams (Godoy, 19 Aug 2025).

At the diffusion layer, AI Diffusion in Low Resource Language Countries argues that linguistic accessibility functions as an independent barrier to adoption. Using a weighted regression framework, it reports that in 2025 Low-Resource Language Countries had a share of AI users approximately 20% lower relative to their baseline, corresponding to a 2.1 percentage point shortfall in AI user share after adjusting for GDP per capita, electricity access, internet access, and age structure. This result is directly compatible with EQUATE’s premise that language should be modeled as a distinct readiness dimension rather than absorbed into generic development indicators (Misra et al., 4 Nov 2025).

Domain-specific work further shows that “readiness” depends on context of use. In language education, L2-Bench proposes a construct-based evaluation methodology built around “learning experience design” and a two-level taxonomy of 12 competencies and 30 sub-competencies, rather than around narrow task accuracy. In language testing, interview and survey data from English test-takers indicate that AI may improve perceived fairness, consistency, and availability while also generating mistrust about reliability and interactivity, with consequences for behavior and well-being. In Indigenous-language assessment, a Hawaiian workflow for the KĀʻEO program demonstrates strong governance readiness, expert-in-the-loop capacity, and document-grounded AI within a high-stakes, culturally governed setting, while also showing that current models remain weak at autonomous interpretation of Hawaiian nuance (Edgell et al., 20 Mar 2026, Zhang et al., 2023, Kūkea-Shultz et al., 19 Dec 2025).

Taken together, these works suggest that EQUATE is best understood as a high-level infrastructural and socio-technical index whose interpretation can be enriched by capability benchmarks, adoption studies, educational evaluation frameworks, and domain-governance case studies.

6. Limitations, interpretive cautions, and distinction from other uses of “EQUATE”

EQUATE’s principal limitations are methodological transparency and epistemic scope. The paper does not provide exact final feature weights, exact readiness-tier thresholds, a complete account of how negatively oriented variables are transformed, the precise rule for selecting a “similar” country in missing-data imputation, or a full uncertainty or sensitivity analysis for final scores. The text also contains unresolved inconsistencies, notably the reference to 25 features in one place and 24 variables in another. The authors cite methodological literature on robustness for composite indicators, but those procedures are not fully reported in the paper text summarized here (Occhini et al., 12 Feb 2026).

There are also conceptual caveats. Speaker coverage is measured by summing speaker populations across covered languages, which can double-count multilingual individuals. Subnational and national proxies are used where language-community-level data do not exist. The index is cross-sectional in its current released form, even though the broader paper includes longitudinal analysis of diffusion from 2020–2024. Most importantly, the authors explicitly note that languages should not be understood in isolation: future work must consider multilingual ecosystems, varying use domains, and community-specific priorities. A readiness score therefore does not imply that AI should be built for every language in the same way or for the same applications (Occhini et al., 12 Feb 2026).

A further source of confusion is terminological. The acronym EQUATE was previously used for “Evaluating Quantity Understanding Aptitude in Textual Entailment,” a 2019 benchmark evaluation framework for quantitative reasoning in natural language inference. That earlier EQUATE is a sentence-pair reasoning benchmark, not a language-centered readiness index. The two are unrelated except for the acronym and should not be conflated in citation or interpretation (Ravichander et al., 2019).

In that restricted but important sense, the Language AI Readiness Index marks a methodological shift. It reframes language AI inequality as a problem of sociotechnical readiness at the level of linguistic communities, not merely as a count of datasets, models, or state-level AI strategies. Its central intervention is to ask not only which languages already have AI, but which languages are structurally prepared for equitable, sustainable, and meaningful language-AI development (Occhini et al., 12 Feb 2026).