Donut-MINT: Mechanistic Interpretability Trimming

• Donut-MINT is a mechanistic interpretability-guided network trimming method that prunes Donut models for document VQA by preserving essential functional circuits.
• It employs targeted sub-layer and attention head pruning combined with knowledge distillation, achieving significant parameter and FLOP reduction while maintaining competitive performance.
• The term also extends to Donut-centric indexing workflows for construction documents, where table-of-contents extraction underpins efficient multi-modal navigation.

Donut-MINT most specifically denotes “Mechanistic Interpretability-based Network Trimming”, a family of pruned Donut variants for document visual question answering that uses mechanistic interpretability to identify essential decoder circuits, remove functionally redundant structure, and distill a compact student model from a larger teacher. In adjacent usage, the name has also been applied to a Donut-centric, multi-modal indexing and navigation workflow for construction specification documents in which table-of-contents extraction provides the backbone of document structure. The primary technical sense, however, is the compression framework introduced for DocVQA, where Donut-MINT targets lower inference latency and memory usage while preserving strong OCR-free document understanding performance (Mansour et al., 30 Sep 2025, Feyisa et al., 2024).

1. Terminology and scope

In its formal sense, Donut-MINT is a pruned Donut variant obtained by mechanistic interpretability-guided sub-layer pruning, head pruning, and knowledge distillation. The method is designed for document VQA and is evaluated on DocVQA, where it is compared against coarse-grained decoder pruning, unstructured pruning, and structured pruning baselines. Its central claim is methodological rather than merely architectural: compression is treated as circuit discovery, not as black-box parameter removal (Mansour et al., 30 Sep 2025).

A second usage appears in work on automatic document indexing via table-of-contents extraction from long construction specification PDFs. There, a Donut-centric pipeline for ToC detection, extraction, and JSON structuring is described as directly aligned with what one might call “Donut-MINT”, namely a Donut-centric, multi-modal indexing and navigation toolkit in which ToCs are the explicit semantic backbone for indexing, retrieval, and navigation. That usage is application-oriented and not the same as the formal compression method, but it is relevant because it situates Donut within a broader ecosystem of document-structure automation (Feyisa et al., 2024).

The acronym MINT also appears independently in a separate line of work on a dynamic-precision CNN inference accelerator based on left-to-right arithmetic for FPGA deployment. That accelerator is unrelated to Donut and concerns CNN hardware, not document VLM compression; the overlap is terminological rather than methodological (Usman et al., 30 Jun 2026).

2. Donut substrate and interpretability basis

The Donut model underlying Donut-MINT is an OCR-free document VQA VLM composed of a Swin-B Transformer vision encoder and a multimodal decoder using the first four layers of mBART. Each decoder layer $\ell \in \{0,1,2,3\}$ contains three sub-layers: self-attention $S\ell$, cross-attention $C\ell$, and feed-forward network $M\ell$. The paper names these sub-layers explicitly as S0–S3, C0–C3, and M0–M3. The full model has 257M parameters in total, with 75M in the encoder, 59M in the embedding, 64M in the decoder, and 59M in the unembedding; for a typical DocVQA example generating $m=35$ answer tokens from $n=42$ prompt tokens, total compute is 2.99T FLOPs, of which 1.57T FLOPs come from the decoder (Mansour et al., 30 Sep 2025).

Donut-MINT begins from a mechanistic analysis of these decoder components. The analysis is performed in a teacher forcing regime and combines activation patching, logit lens, token reprojection, and attention-map statistics. Activation patching measures the effect of skipping or modifying individual components using perplexity as the diagnostic signal. Token reprojection tests whether an intermediate activation behaves essentially like a token embedding by mapping the most likely token back through the embedding layer and resuming the forward pass. For head ranking, self-attention heads are scored by the entropy of their attention distributions, while cross-attention heads are scored by the variance of their attention maps over visual patches (Mansour et al., 30 Sep 2025).

Several concrete functional findings drive the pruning design. First, C3 is identified as the main direct text transcription circuit: the composed map $$\text{lm\_head} \circ \text{C3.out\_proj} \circ \text{C3.v\_proj}$$ can decode patch-wise tokens directly from Swin encoder features, and heads C3.H4, C3.H7, C3.H8, C3.H9, C3.H10, C3.H11 are singled out as the heads that retrieve tokens at attended positions. Second, M3 is characterized as a residual cleaner rather than a semantic transformer, because skipping it or cutting C3’s residual stream yields nearly identical perplexity. Third, M0 is described as mapping tokens to the representation of their lowercase form, effectively neutralizing case distinctions in the embedding space. Fourth, for keyword-style questions such as “date” or “title,” the self-attention path propagates the keyword signal to the special <s_answer> token, whose representation then functions as a semantic query for the final cross-attention layer. For contextualized key–value questions, the analysis indicates a strong positional bias in C3 queries, suggesting a circuit that first locates the key and then reads a nearby value (Mansour et al., 30 Sep 2025).

3. MINT pruning and distillation procedure

The MINT procedure is organized as interpret, then prune, then distill. In the first stage, interpretability results determine which entire decoder sub-layers appear functionally redundant or approximable. In the second stage, remaining attention layers undergo head pruning based on entropy or variance scores, while heads identified as critical for reading or query formation are preserved. In the third stage, the resulting student decoder is retrained with knowledge distillation from a Donut teacher fine-tuned on DocVQA (Mansour et al., 30 Sep 2025).

A key design choice is that the encoder, embedding, and unembedding layers are kept unchanged and frozen during student training. The teacher is donut-base-finetuned-docvqa, while the students inherit teacher weights for all preserved components. Only the student decoder is updated. The training objective combines ground-truth supervision with soft teacher targets:

$$\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{CE}} + \alpha \mathcal{L}_{\text{KD}}.$$

Here $\mathcal{L}_{\text{CE}}$ is the standard seq2seq cross-entropy loss, and $\mathcal{L}_{\text{KD}}$ is a softened KL-divergence term between teacher and student output distributions. Training uses the DocVQA training split and is performed on two NVIDIA L40S GPUs (Mansour et al., 30 Sep 2025).

Two principal Donut-MINT variants are reported. Donut-MINT$S\ell$0 retains 31% of decoder parameters, corresponding to 20M decoder parameters and 757G decoder FLOPs. This model results from MI-guided sub-layer pruning that preserves the minimal circuits needed for lowercasing and normalization, prompt-query construction, and cross-attentive reading. Donut-MINT$S\ell$1 pushes compression further through head pruning, yielding 4.48M decoder parameters and 211G decoder FLOPs, or 7% of original decoder parameters and 13% of original decoder FLOPs. In that setting, C3 keeps only the heads responsible for accurate transcription, while other heads are removed (Mansour et al., 30 Sep 2025).

The broader methodological claim is that mechanistic interpretability-based network trimming can identify not only which components matter, but what function they perform. That functional understanding justifies structured pruning decisions that would be difficult to derive from magnitude alone. In this sense, Donut-MINT redefines compression as the preservation of a minimal reading-and-retrieval circuit rather than the maintenance of a nominal layer budget (Mansour et al., 30 Sep 2025).

4. Quantitative performance on DocVQA

Evaluation is conducted on DocVQA, described as a benchmark with ~50,000 question–answer pairs over more than 12,000 document images spanning forms, tables, invoices, and letters. Questions include both keyword queries and contextualized key–value questions. The original validation set is partitioned into 20% for mechanistic interpretability analysis, 20% for student model selection, and 60% for post-pruning evaluation, while final test results are obtained from the official DocVQA evaluation server. The main task metric is ANLS (Average Normalized Levenshtein Similarity), with additional reporting of decoder parameter counts and decoder FLOPs (Mansour et al., 30 Sep 2025).

The reported results show that MI-guided pruning substantially outperforms naive structured pruning under similar budgets, and remains competitive with strong fine-grained pruning baselines at very high compression rates.

Model	Decoder size / compute	DocVQA ANLS
Donut-base teacher	64M params, 1.57T FLOPs	66%
Donut-MINT$S\ell$2	20M params, 757G FLOPs	50%
Donut-MINT$S\ell$3	4.48M params, 211G FLOPs	51%

Among coarse-grained students near the 30–40% parameter and ~50% FLOP regime, the enumeration baselines Donut-T1, Donut-T2, and Donut-T3 achieve only 23%, 22%, and 28% ANLS, respectively, despite each using 24M parameters and 766G FLOPs. By contrast, Donut-MINT$S\ell$4 reaches 50% ANLS with fewer parameters and slightly lower FLOPs. Among fine-grained students at the 7% decoder parameter budget, Wanda attains 46% ANLS but with 1.57T FLOPs, because unstructured sparsity does not reduce dense-hardware compute; FinerCut achieves 50% ANLS at 188G FLOPs; Donut-MINT$S\ell$5 reaches 51% ANLS at 211G FLOPs. The paper summarizes this as retaining roughly 77% of teacher performance at about an 8× decoder FLOP reduction (Mansour et al., 30 Sep 2025).

These results support a specific interpretation of Donut-MINT’s contribution. The performance gap between teacher and student is not eliminated, but the method demonstrates that function-preserving structural compression is materially better than pruning strategies that do not distinguish between transcription heads, query-construction paths, and residual-cleaning components. A plausible implication is that the main advantage of Donut-MINT lies less in raw sparsity and more in task-aligned preservation of multimodal circuits.

5. Application-oriented usage in document indexing

In construction document processing, Donut-MINT has been used in a looser sense to denote a Donut-centric, multi-modal indexing and navigation toolkit whose core operation is table-of-contents extraction from long industrial PDFs. That workflow targets construction specification documents, where the ToC is the explicit locus of semantic structure across divisions, sections, and subsections. The pipeline addresses three tasks: detecting which pages are ToC pages, extracting heading numbers, heading titles, subheading numbers, and subheading titles, and structuring the result into a machine-readable JSON hierarchy for indexing, navigation, and downstream retrieval (Feyisa et al., 2024).

The pipeline contains two complementary branches. The Donut-only branch operates on page images: each page is rasterized and resized to 1260 × 960 pixels, a donut-base model pre-trained on RVL-CDIP is fine-tuned as a page classifier for “ToC” versus “other”, and a second Donut model pre-trained on ICDAR-SROIE is fine-tuned as a sequence-to-JSON extractor. The target schema uses the fields hn, ht, shn, and sht, corresponding to heading number, heading title, subheading number, and subheading title. The GPT branch starts from raw PDF text, uses GPT-3.5 Turbo to extract the ToC text segment, and then uses few-shot prompting to format that text into the same JSON schema (Feyisa et al., 2024).

The annotated extraction dataset consists of 200 ToC images, split 90% train, 10% test, spanning at least “old format” and “MasterFormat” construction specifications. For the extraction model, evaluation is reported on 20 test documents. Donut directly outputs a JSON-formatted string, after which the JSON is parsed, stored in a database, and displayed through a React/Next.js dashboard served through a Flask API. The final JSON supports section-level indexing, UI navigation, and project-level linkage to downstream artifacts such as drawings, RFIs, or contracts (Feyisa et al., 2024).

Reported extraction results differentiate numeric and textual fields. On the 20 test documents, Donut achieves 82.2% overall ToC extraction accuracy; field-level results show 0.92 for heading numbers, 0.78 for heading titles, 0.91 for subheading numbers, 0.76 for subheading titles, and 0.85 average. GPT-3.5 Turbo reaches 0.92 for heading numbers, 0.83 for heading titles, 0.88 for subheading numbers, 0.84 for subheading titles, and 0.89 average. The contrast is explicit: both methods perform strongly on numeric fields, while GPT is stronger on the free-form textual fields. In this application-oriented sense, “Donut-MINT” designates a Donut-centered indexing stack rather than a compressed VQA model, but the common thread remains Donut’s role as an OCR-free document-structure engine (Feyisa et al., 2024).

6. Limitations, generalization, and adjacent Donut-derived directions

The formal Donut-MINT line identifies several limitations. There remains a 15-point ANLS gap between the teacher and Donut-MINT$S\ell$6, which may be prohibitive in high-stakes deployments. The interpretability study is task- and domain-specific, centered on Donut’s DocVQA behavior and especially on its tendency to copy text from images. The mechanistic analysis is also described as largely manual, relying on designed patching experiments and visual inspection. In addition, the distillation stage matches only final logits, not internal activations or circuits, leaving open the possibility of more circuit-aware training objectives (Mansour et al., 30 Sep 2025).

At the same time, the method is presented as a candidate general strategy for other VLMs with explicit cross-attention between vision encoder and language decoder. The paper names BLIP, LLaVA-style VLMs, and Pix2Struct as architectures that could potentially benefit from the same pattern of interpret, then prune, then distill. Proposed extensions include automated circuit discovery, attribution patching, and intermediate-activation distillation targeted at the specific subcircuits identified during analysis (Mansour et al., 30 Sep 2025).

A related but distinct Donut-derived direction appears in trajectory forecasting. Work on DONUT-NLL does not use the name Donut-MINT, but it develops a distribution-first framework in which a Donut-based forecaster is trained with metric-agnostic NLL objectives and benchmark-specific outputs are generated only afterward through Trajectory Distribution Evaluation (TraDiE) policies. That paper treats metric optimization as a downstream task applied to the predictive distribution, rather than as something built directly into the training loss. This suggests a broader conceptual family of Donut variants in which the core model learns a calibrated latent or predictive structure, and downstream modules specialize it for compression, indexing, or metric-conditioned evaluation (Knoche et al., 1 Jul 2026).

Taken together, these usages show that Donut-MINT is not merely a single model checkpoint name. In its strict sense it is an interpretability-guided compression methodology for Donut; in looser applied usage it refers to Donut-centered document indexing workflows; and in adjacent Donut research it resonates with a wider move toward modular specialization, where the Donut substrate is preserved and higher-level capabilities are supplied by pruning rules, JSON schemas, or metric-specific policies.