Multilingual Date Fragmentation Ratio (mDFR)

• The paper introduces mDFR as an extension of DFR for multilingual and multi-calendar contexts, accurately quantifying subword fragmentation using four calibrated error signals.
• mDFR is defined as a weighted sum of errors from token splitting, delimiter loss, token count inflation, and cosine distance relative to an ideal segmentation.
• Empirical evaluations on MultiTempBench show that higher mDFR scores predict lower temporal reasoning accuracy, notably in low-resource languages and non-Gregorian calendars.

Searching arXiv for the cited papers and closely related work on date fragmentation metrics and temporal reasoning benchmarks Multilingual Date Fragmentation Ratio (mDFR) is a metric for quantifying how subword tokenisation degrades structured date inputs across languages and calendar conventions It was introduced in the context of MultiTempBench^^^^^^^^34. a multilingual temporal reasoning benchmark spanning date arithmetic^^^^^^^^34. time zone conversion^^^^^^^^34. and temporal relation extraction across English^^^^^^^^34. German^^^^^^^^34. Chinese^^^^^^^^34. Arabic^^^^^^^^34. and Hausa^^^^^^^^34. and across Gregorian^^^^^^^^34. Hijri^^^^^^^^34. and Chinese Lunar conventions (Bhatia et al., 19 Mar 2026) mDFR extends the Date Fragmentation Ratio (DFR) of Bhatia et al by measuring whether semantic date units are split^^^^^^^^34. whether delimiters are lost^^^^^^^^34. how much token count is inflated relative to an ideal semantic baseline^^^^^^^^34. and how far the model’s token-count profile diverges from that baseline (&&& In the reported experiments^^^^^^^^34. mDFR functions as an input-level predictor of temporal reasoning difficulty: fragmentation is a resource-dependent bottleneck^^^^^^^^34. especially in low-resource languages and rarer calendar formats^^^^^^^^34. whereas in high-resource settings internal temporal representation becomes the stronger predictor of performance (Bhatia et al., 19 Mar 2026)

Formal definition

mDFR extends the Date Fragmentation Ratio (DFR) of Bhatia et al (^^^^^^^^^^^^36. to the multilingual setting (Bhatia et al., 19 Mar 2026) For any tokeniser’s segmentation of a date string^^^^^^^^34. mDFR is a weighted sum of four error signals:

• $\mathbbm{1}_{\mathrm{split}} = 1$ if any semantic root is split into multiple tokens^^^^^^^^34. else PRESERVED_PLACEHOLDER_
• PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36. if any delimiter is lost^^^^^^^^34. else PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.
• PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.3 the relative inflation in token count compared to an ideal semantic baseline
• $\theta$^^^^^^^^34. the cosine-distance between the model’s token-count vector and the ideal segmentation vector

Formally^^^^^^^^34. if $\alpha_1 \ldots \alpha_4$ are non-negative coefficients summing to $1$^^^^^^^^34. then

$\text{mDFR} \;=\; F \;=\; \alpha_1\,\mathbbm{1}_{\mathrm{split}} \;+\; \alpha_2\,\mathbbm{1}_{\mathrm{delimiter}} \;+\; \alpha_3\,\Delta N \;+\; \alpha_4\,\theta \quad \in [0,1].$

In the reported instantiation^^^^^^^^34. after normalising^^^^^^^^34. the learned weights were

$$(\alpha_1,\alpha_2,\alpha_3,\alpha_4) = (0.20,\,0.20,\,0.10,\,0.50).$$

This formulation differs from the original DFR in both scope and parameterisation DFR was defined as a per-date fragmentation score by comparing a model’s subword output against a rule-based ideal baseline^^^^^^^^34. with components for split fields^^^^^^^^34. delimiter loss^^^^^^^^34. token-count inflation^^^^^^^^34. and cosine-distance over year^^^^^^^^34. month^^^^^^^^34. and day field counts (&&& A plausible implication is that mDFR preserves the interpretability of DFR while adapting it to multilingual and multi-calendar settings where date strings may contain script-specific markers^^^^^^^^34. different field orders^^^^^^^^34. and calendar-specific tokens

^^^^^^^^^^^^36. Calibration against human severity judgments

To ensure that mDFR aligns with human perception of how destructive a fragmentation is^^^^^^^^34. the metric was calibrated in a controlled annotation study (Bhatia et al., 19 Mar 2026) The stimulus set consisted of ^^^^0^^^^0 tokenised date strings covering multiple languages and formats Five graduate-level NLP readers^^^^^^^^34. blind to the metric’s definition^^^^^^^^34. rated fragmentation severity on a Likert scale^^^^^^^^34. where corresponded to perfect preservation of year/month/day and 5 to date structure being completely obscured^^^^^^^^34. including cases such as single-digit splits and spurious tokens

The reported inter-annotator reliability was Krippendorff’s PRESERVED_PLACEHOLDER_^^^^0 A linear regression was then fit to predict the average human rating from the four components PRESERVED_PLACEHOLDER_ The resulting coefficients closely matched the intuitive weights^^^^^^^^34. and the final mDFR score correlated with human ratings at Spearman PRESERVED_PLACEHOLDER_ compared with BLEU at PRESERVED_PLACEHOLDER_ (Bhatia et al., 19 Mar 2026)

The calibration procedure gives mDFR a psychometric grounding not present in a purely heuristic token-count ratio This suggests that the metric is not merely a tokenizer-descriptive statistic but an estimate of perceived structural damage to date compositionality In the broader line of work initiated by DFR^^^^^^^^34. this emphasis on preserving year^^^^^^^^34. month^^^^^^^^34. and day as meaningful units is central: DFR was introduced precisely to measure how faithfully a tokenizer preserves multi-digit date components needed for robust temporal reasoning (&&&

^^^^^^^^^^^^36. Multilingual and multi-calendar computation

In MultiTempBench^^^^^^^^34. mDFR is computed for each language/calendar variant by comparing the model’s native subword segmentation^^^^^^^^34. via its TikToken or HuggingFace tokenizer^^^^^^^^34. with a semantic baseline segmentation that isolates year^^^^^^^^34. month^^^^^^^^34. day^^^^^^^^34. delimiters^^^^^^^^34. and calendar markers as atomic tokens (Bhatia et al., 19 Mar 2026) The benchmark spans five languages and six date formats

The paper reports concrete fragmentations for Gemma ^^^^^^^^^^^^36. tokenizer For German^^^^^^^^34. the date string “^^^^0 has the baseline segmentation ["^^^^0^^^^"^^^^^^^^34. "^^^^^^^^^^^^^^^^^^^^36.3^^^^^^^^^^^^"^^^^^^^^34. "^^^^^^^^^^^^^^^^36.^^^^^^^^^^^^36.^^^^^^^^36.^^^^^^^^^^^^"^^^^^^^^34. "^^^^^^^^^^^^^^^^36.^^^^34^^^^^^^^^^^^"]^^^^^^^^34. whereas Gemma ^^^^^^^^^^^^36. yields ["^^^^^^^^^^^^^^^^36.3^^^^^^^^"|"0"|"^^^^^^^^^^^^^^^^^^^^36.3^^^^^^^^^^^^"|"O"|"kt"|"ober"|"^^^^^^^^^^^^36.^^^^^^^^^^^^36.^^^^^^^^36.^^^^^^^^"|"0"|"^^^^^^^^^^^^36.^^^^34^^^^^^^^"|"^^^^^^^^^^^^36.3^^^^"]^^^^^^^^34. giving approximately mDFR PRESERVED_PLACEHOLDER_ (Bhatia et al., 19 Mar 2026) For Chinese Lunar^^^^^^^^34. “^^^^^^^^^^^^36.4 has baseline segmentation ["^^^^^^^^^^^^36.4^^^^"^^^^^^^^34. "^^^^^^^^^^^^36.5^^^^"^^^^^^^^34. "^^^^^^^^^^^^36.6^^^^"]^^^^^^^^34. but Gemma ^^^^^^^^^^^^36. yields ["^^^^^^^^^^^^36.7^^^^"|"^^^^^^^^^^^^36.8^^^^"|"^^^^^^^^^^^^36.9^^^^"|"^^^^^^^^34^^^^^^^^3^^^^34^^^^^^^^^^^^"|"^^^^^^^^34^^^^^^^^^^^^36.^^^^"|"^^^^^^^^34^^^^^^^^34^^^^^^^^"|"^^^^^^^^34^^^^3^^^^"]^^^^^^^^34. with mDFR PRESERVED_PLACEHOLDER_ For Hausa Gregorian^^^^^^^^34. “Okt^^^^3^^^^3^^^^34 ^^^^0 has baseline segmentation ["Okt^^^^3^^^^3^^^^34^^^^^^^^^^^^"^^^^^^^^34. "^^^^0^^^^"^^^^^^^^34. "^^^^^^^^34."^^^^^^^^34. "^^^^^^^^^^^^^^^^36.^^^^34^^^^^^^^^^^^"]^^^^^^^^34. while Gemma ^^^^^^^^^^^^36. yields ["O"|"kt"|"^^^^3^^^^3^^^^34^^^^^^^^^^^^"|"^^^^^^^^^^^^^^^^36.3^^^^^^^^"|"0"|"^^^^^^^^34."|"^^^^^^^^^^^^36.^^^^^^^^^^^^36.^^^^^^^^36.^^^^^^^^"|"0"|"^^^^^^^^^^^^36.^^^^34^^^^^^^^"|"^^^^^^^^^^^^36.3^^^^"]^^^^^^^^34. with mDFR PRESERVED_PLACEHOLDER_ (Bhatia et al., 19 Mar 2026)

The aggregate ranges reported in Table 5 show strong variation by language and calendar system (Bhatia et al., 19 Mar 2026)

Setting	Reported mDFR range
High-resource/Gregorian (en^^^^^^^^34. de^^^^^^^^34. zh)	PRESERVED_PLACEHOLDER_
Arabic Hijri & English Hijri	PRESERVED_PLACEHOLDER_
Chinese Lunar	PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.
Hausa (low-resource)	PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.

These values instantiate the multilingual extension anticipated in the earlier DFR framework^^^^^^^^34. which noted challenges such as different string-order conventions^^^^^^^^34. native digits versus Arabic “0–9”^^^^^^^^34. vocabulary mismatch across scripts^^^^^^^^34. and format explosion in language-specific date serialisations (&&& In that earlier formulation^^^^^^^^34. multilingual evaluation required a language-specific baseline splitter and per-language aggregation before computing an overall mDFR (&&& The MultiTempBench implementation operationalises this general idea in a concrete multilingual benchmark

^^^^^^^^^^^^36.3 Function within MultiTempBench evaluation

MultiTempBench contains examples built by translating 750 curated English questions and expanding each into controlled date-format variants (Bhatia et al., 19 Mar 2026) The benchmark evaluates ^^^^^^^^^^^^36. LLMs on zero-shot temporal reasoning across three tasks: date arithmetic^^^^^^^^34. time zone conversion^^^^^^^^34. and temporal relation extraction

For each question^^^^^^^^34. the evaluation workflow is defined in three steps (Bhatia et al., 19 Mar 2026):

Compute mDFR of the date strings in the prompt ^^^^^^^^^^^^36. Query the LLM and judge correctness with GPT-^^^^^^^^^^^^36.3 as a judge ^^^^^^^^^^^^36. Analyze the relationship between mDFR and per-task accuracy

Within this pipeline^^^^^^^^34. mDFR serves as an input-level predictor^^^^^^^^34. capturing how tokenisation obscures the compositional elements—year^^^^^^^^34. month^^^^^^^^34. and day—the model needs before any internal reasoning can occur (Bhatia et al., 19 Mar 2026) That role is conceptually continuous with the earlier DFR paper^^^^^^^^34. which argued that modern BPE tokenizers often split calendar dates into meaningless fragments^^^^^^^^34. inflating token counts and obscuring the inherent structure needed for temporal reasoning (&&&

A plausible implication is that mDFR separates front-end representational damage from downstream reasoning capacity The benchmark’s design makes it possible to examine whether errors arise because the model never receives a recoverable decomposition of the date string^^^^^^^^34. or because it receives one but lacks an adequate temporal representation for arithmetic or comparison

5 Empirical relationship to temporal reasoning accuracy

Across the three tasks in MultiTempBench^^^^^^^^34. mDFR is reported as a strong negative correlate of accuracy in low-resource settings but weak or near-zero in high-resource ones (Bhatia et al., 19 Mar 2026) In the Date Arithmetic task^^^^^^^^34. the reported correlations are:

• Hausa^^^^^^^^34. mDFR range PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36. PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.
• Arabic^^^^^^^^34. mDFR range PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36. PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.
• English^^^^^^^^34. mDFR range PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36. PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.
• German: PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.
• Chinese: PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.

Time-zone conversion and temporal relation extraction show the same regime-dependent pattern (Bhatia et al., 19 Mar 2026) The paper further states that models with mDFR PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36. in Hausa collapse to PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36. accuracy^^^^^^^^34. whereas the same models in English^^^^^^^^34. with mDFR PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36. reliably exceed PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36. accuracy (Bhatia et al., 19 Mar 2026)

These findings extend the earlier DFR results^^^^^^^^34. which reported that higher fragmentation correlates with lower accuracy across temporal regimes and date formats^^^^^^^^34. and that excessive fragmentation can impose an approximately ^^^^0^^^^-point accuracy penalty on uncommon dates such as historical and futuristic dates (&&& In the multilingual case^^^^^^^^34. the dependence on resource level is made explicit This suggests that fragmentation is not uniformly harmful: its observed impact depends on whether a model has sufficient language-specific exposure to learn compensatory mechanisms

6 Resource dependence^^^^^^^^34. temporal linearity^^^^^^^^34. and regime shift

The reported analysis goes beyond bivariate correlation by fitting a crossed mixed-effects logistic regression over ^^^^^^^^^^^^36. predictions (Bhatia et al., 19 Mar 2026):

PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.3

Here^^^^^^^^34. PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.3 is z-scored fragmentation^^^^^^^^34. PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.3 is z-scored temporal linearity PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.3 probe of hidden states^^^^^^^^34. and PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.3 (Bhatia et al., 19 Mar 2026)

The key fixed-effect results are reported as follows:

• PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.3 PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.3. PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.3
• PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.3 PRESERVED_PLACEHOLDER_^^^^^^^^^^^^36.3ober $\theta$0
• Three-way interaction $\theta$ $\theta$^^^^^^^^^^^^36. $\theta$^^^^^^^^^^^^36.

The interpretation given in the paper is that^^^^^^^^34. in low-resource settings such as Arabic and Hausa^^^^^^^^34. tokenisation as measured by mDFR is the bottleneck^^^^^^^^34. because poorly segmented dates never surface the year^^^^^^^^34. month^^^^^^^^34. and day for computation (Bhatia et al., 19 Mar 2026) In high-resource settings such as English^^^^^^^^34. German^^^^^^^^34. and Chinese^^^^^^^^34. models can often stitch fragments from abundant training data^^^^^^^^34. and success instead hinges on having a clean internal temporal axis^^^^^^^^34. reflected in high temporal linearity (Bhatia et al., 19 Mar 2026)

Complementary PCA visualisations show that by mid-layers^^^^^^^^34. English^^^^^^^^34. German^^^^^^^^34. and Chinese date-year embeddings form near-straight trajectories^^^^^^^^34. whereas Hausa remains a tangled cluster (Bhatia et al., 19 Mar 2026) This supports the claim that mDFR constrains the entry point to any internal temporal representation The earlier DFR work had already described an emergent date-abstraction mechanism whereby LLMs stitch together fragments of month^^^^^^^^34. day^^^^^^^^34. and year components^^^^^^^^34. and observed that the larger the model^^^^^^^^34. the faster this emergent date abstraction is accomplished (&&& MultiTempBench refines that picture by showing that the relative importance of front-end fragmentation versus internal geometry shifts systematically with resource availability

7 Interpretation^^^^^^^^34. scope^^^^^^^^34. and related formulations

mDFR operationalises how subword tokenisation degrades structured date inputs in multilingual^^^^^^^^34. multi-calendar contexts (Bhatia et al., 19 Mar 2026) Its practical significance lies in making tokenizer failure on temporal artefacts measurable before downstream inference It is therefore not a temporal reasoning metric in itself^^^^^^^^34. but a predictor of how much date structure reaches the model in an interpretable form

The metric should be distinguished from simpler multilingual averages proposed in earlier extensions of DFR The earlier framework described a per-language average $\theta$^^^^^^^^^^^^36.3^^^^ and then an aggregate $\theta$5^^^^^^^^34. and also mentioned an alternative simpler token-count version (&&& By contrast^^^^^^^^34. MultiTempBench defines mDFR directly as a weighted sum of four calibrated error signals at the date-string level^^^^^^^^34. with learned weights aligned to human severity judgments (Bhatia et al., 19 Mar 2026) A plausible implication is that the later formulation privileges interpretability and psychometric calibration over purely aggregate token-count summaries

A common misconception is that tokenisation alone determines temporal reasoning performance The MultiTempBench results explicitly reject that uniform view: tokenisation quality of temporal artefacts is a resource-dependent bottleneck^^^^^^^^34. and crossed mixed-effects regression identifies temporal linearity as the strongest predictor in high-resource languages^^^^^^^^34. whereas fragmentation is the stronger predictor in low-resource languages (Bhatia et al., 19 Mar 2026) Another possible misconception is that digit-level splitting is always fatal The reported results state instead that high-resource settings are often robust to digit-level splitting^^^^^^^^34. while fragmentation causes accuracy collapse in low-resource languages and rarer calendar formats (Bhatia et al., 19 Mar 2026)

Within this research line^^^^^^^^34. mDFR occupies a specific analytical niche DFR identified date fragmentation as a hidden bottleneck of tokenization for temporal reasoning (&&& mDFR generalises that bottleneck to multilingual and multi-calendar evaluation^^^^^^^^34. quantifies it with calibrated weights^^^^^^^^34. and situates it within a broader account in which tokenisation and internal temporal representation jointly control LLM performance on temporal tasks (Bhatia et al., 19 Mar 2026)