---
title: 'Donut-MINT: Mechanistic Interpretability Trimming'
url: https://www.emergentmind.com/topics/donut-mint
type: topic
---

# Donut-MINT: Mechanistic Interpretability Trimming

Donut-MINT most specifically denotes **“Mechanistic Interpretability-based Network Trimming”**, a family of pruned Donut variants for document visual question answering that uses mechanistic interpretability to identify essential decoder circuits, remove functionally redundant structure, and distill a compact student model from a larger teacher. In adjacent usage, the name has also been applied to a Donut-centric, multi-modal indexing and navigation workflow for construction specification documents in which table-of-contents extraction provides the backbone of document structure. The primary technical sense, however, is the compression framework introduced for DocVQA, where Donut-MINT targets lower inference latency and memory usage while preserving strong OCR-free document understanding performance [2509.26235], [2403.07553].

## 1. Terminology and scope

In its formal sense, Donut-MINT is a **pruned Donut variant** obtained by **mechanistic interpretability-guided sub-layer pruning, head pruning, and knowledge distillation**. The method is designed for **document VQA** and is evaluated on **DocVQA**, where it is compared against coarse-grained decoder pruning, unstructured pruning, and structured pruning baselines. Its central claim is methodological rather than merely architectural: compression is treated as **circuit discovery**, not as black-box parameter removal [2509.26235].

A second usage appears in work on **automatic document indexing via table-of-contents extraction** from long construction specification PDFs. There, a Donut-centric pipeline for ToC detection, extraction, and JSON structuring is described as directly aligned with what one might call **“Donut-MINT”**, namely a **Donut-centric, multi-modal indexing and navigation toolkit** in which ToCs are the explicit semantic backbone for indexing, retrieval, and navigation. That usage is application-oriented and not the same as the formal compression method, but it is relevant because it situates Donut within a broader ecosystem of document-structure automation [2403.07553].

The acronym **MINT** also appears independently in a separate line of work on a **dynamic-precision CNN inference accelerator** based on left-to-right arithmetic for FPGA deployment. That accelerator is unrelated to Donut and concerns CNN hardware, not document VLM compression; the overlap is terminological rather than methodological [2606.31514].

## 2. Donut substrate and interpretability basis

The Donut model underlying Donut-MINT is an **OCR-free document VQA VLM** composed of a **Swin-B Transformer vision encoder** and a **multimodal decoder using the first four layers of mBART**. Each decoder layer $\ell \in \{0,1,2,3\}$ contains three sub-layers: self-attention $S\ell$, cross-attention $C\ell$, and feed-forward network $M\ell$. The paper names these sub-layers explicitly as **S0–S3**, **C0–C3**, and **M0–M3**. The full model has **257M parameters** in total, with **75M** in the encoder, **59M** in the embedding, **64M** in the decoder, and **59M** in the unembedding; for a typical DocVQA example generating $m=35$ answer tokens from $n=42$ prompt tokens, total compute is **2.99T FLOPs**, of which **1.57T FLOPs** come from the decoder [2509.26235].

Donut-MINT begins from a mechanistic analysis of these decoder components. The analysis is performed in a **teacher forcing** regime and combines **activation patching**, **logit lens**, **token reprojection**, and **attention-map statistics**. Activation patching measures the effect of skipping or modifying individual components using **perplexity** as the diagnostic signal. Token reprojection tests whether an intermediate activation behaves essentially like a token embedding by mapping the most likely token back through the embedding layer and resuming the forward pass. For head ranking, **self-attention heads** are scored by the **entropy** of their attention distributions, while **cross-attention heads** are scored by the **variance** of their attention maps over visual patches [2509.26235].

Several concrete functional findings drive the pruning design. First, **C3** is identified as the main **direct text transcription circuit**: the composed map $\text{lm\_head} \circ \text{C3.out\_proj} \circ \text{C3.v\_proj}$ can decode patch-wise tokens directly from Swin encoder features, and heads **C3.H4, C3.H7, C3.H8, C3.H9, C3.H10, C3.H11** are singled out as the heads that retrieve tokens at attended positions. Second, **M3** is characterized as a **residual cleaner** rather than a semantic transformer, because skipping it or cutting C3’s residual stream yields nearly identical perplexity. Third, **M0** is described as mapping tokens to the representation of their **lowercase form**, effectively neutralizing case distinctions in the embedding space. Fourth, for keyword-style questions such as “date” or “title,” the self-attention path propagates the keyword signal to the special `<s_answer>` token, whose representation then functions as a **semantic query** for the final cross-attention layer. For contextualized key–value questions, the analysis indicates a **strong positional bias** in C3 queries, suggesting a circuit that first locates the key and then reads a nearby value [2509.26235].

## 3. MINT pruning and distillation procedure

The MINT procedure is organized as **interpret, then prune, then distill**. In the first stage, interpretability results determine which entire decoder sub-layers appear functionally redundant or approximable. In the second stage, remaining attention layers undergo **head pruning** based on entropy or variance scores, while heads identified as critical for reading or query formation are preserved. In the third stage, the resulting student decoder is retrained with **knowledge distillation** from a Donut teacher fine-tuned on DocVQA [2509.26235].

A key design choice is that the **encoder, embedding, and unembedding layers are kept unchanged and frozen** during student training. The teacher is **`donut-base-finetuned-docvqa`**, while the students inherit teacher weights for all preserved components. Only the student decoder is updated. The training objective combines ground-truth supervision with soft teacher targets:
$$
\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{CE}} + \alpha \mathcal{L}_{\text{KD}}.
$$
Here $\mathcal{L}_{\text{CE}}$ is the standard seq2seq cross-entropy loss, and $\mathcal{L}_{\text{KD}}$ is a softened KL-divergence term between teacher and student output distributions. Training uses the **DocVQA training split** and is performed on **two NVIDIA L40S GPUs** [2509.26235].

Two principal Donut-MINT variants are reported. **Donut-MINT\(_{31\%}\)** retains **31% of decoder parameters**, corresponding to **20M** decoder parameters and **757G** decoder FLOPs. This model results from MI-guided sub-layer pruning that preserves the minimal circuits needed for lowercasing and normalization, prompt-query construction, and cross-attentive reading. **Donut-MINT\(_{7\%}\)** pushes compression further through head pruning, yielding **4.48M** decoder parameters and **211G** decoder FLOPs, or **7%** of original decoder parameters and **13%** of original decoder FLOPs. In that setting, C3 keeps only the heads responsible for accurate transcription, while other heads are removed [2509.26235].

The broader methodological claim is that **mechanistic interpretability-based network trimming** can identify not only which components matter, but **what function they perform**. That functional understanding justifies structured pruning decisions that would be difficult to derive from magnitude alone. In this sense, Donut-MINT redefines compression as the preservation of a minimal reading-and-retrieval circuit rather than the maintenance of a nominal layer budget [2509.26235].

## 4. Quantitative performance on DocVQA

Evaluation is conducted on **DocVQA**, described as a benchmark with **~50,000 question–answer pairs** over **more than 12,000 document images** spanning forms, tables, invoices, and letters. Questions include both **keyword queries** and **contextualized key–value questions**. The original validation set is partitioned into **20% for mechanistic interpretability analysis**, **20% for student model selection**, and **60% for post-pruning evaluation**, while final test results are obtained from the official DocVQA evaluation server. The main task metric is **ANLS** (Average Normalized Levenshtein Similarity), with additional reporting of decoder parameter counts and decoder FLOPs [2509.26235].

The reported results show that MI-guided pruning substantially outperforms naive structured pruning under similar budgets, and remains competitive with strong fine-grained pruning baselines at very high compression rates.

| Model | Decoder size / compute | DocVQA ANLS |
|---|---|---|
| Donut-base teacher | 64M params, 1.57T FLOPs | 66% |
| Donut-MINT\(_{31\%}\) | 20M params, 757G FLOPs | 50% |
| Donut-MINT\(_{7\%}\) | 4.48M params, 211G FLOPs | 51% |

Among **coarse-grained students** near the **30–40% parameter** and **~50% FLOP** regime, the enumeration baselines **Donut-T1**, **Donut-T2**, and **Donut-T3** achieve only **23%**, **22%**, and **28%** ANLS, respectively, despite each using **24M parameters** and **766G FLOPs**. By contrast, **Donut-MINT\(_{31\%}\)** reaches **50% ANLS** with fewer parameters and slightly lower FLOPs. Among **fine-grained students** at the **7% decoder parameter** budget, **Wanda** attains **46% ANLS** but with **1.57T FLOPs**, because unstructured sparsity does not reduce dense-hardware compute; **FinerCut** achieves **50% ANLS** at **188G FLOPs**; **Donut-MINT\(_{7\%}\)** reaches **51% ANLS** at **211G FLOPs**. The paper summarizes this as retaining roughly **77% of teacher performance** at about an **8× decoder FLOP reduction** [2509.26235].

These results support a specific interpretation of Donut-MINT’s contribution. The performance gap between teacher and student is not eliminated, but the method demonstrates that **function-preserving structural compression** is materially better than pruning strategies that do not distinguish between transcription heads, query-construction paths, and residual-cleaning components. A plausible implication is that the main advantage of Donut-MINT lies less in raw sparsity and more in **task-aligned preservation of multimodal circuits**.

## 5. Application-oriented usage in document indexing

In construction document processing, Donut-MINT has been used in a looser sense to denote a **Donut-centric, multi-modal indexing and navigation toolkit** whose core operation is **table-of-contents extraction** from long industrial PDFs. That workflow targets **construction specification documents**, where the ToC is the explicit locus of semantic structure across divisions, sections, and subsections. The pipeline addresses three tasks: detecting which pages are ToC pages, extracting **heading numbers**, **heading titles**, **subheading numbers**, and **subheading titles**, and structuring the result into a machine-readable **JSON hierarchy** for indexing, navigation, and downstream retrieval [2403.07553].

The pipeline contains two complementary branches. The **Donut-only branch** operates on page images: each page is rasterized and resized to **1260 × 960** pixels, a **`donut-base`** model pre-trained on **RVL-CDIP** is fine-tuned as a **page classifier** for **“ToC”** versus **“other”**, and a second Donut model pre-trained on **ICDAR-SROIE** is fine-tuned as a **sequence-to-JSON extractor**. The target schema uses the fields **`hn`**, **`ht`**, **`shn`**, and **`sht`**, corresponding to heading number, heading title, subheading number, and subheading title. The **GPT branch** starts from raw PDF text, uses **GPT-3.5 Turbo** to extract the ToC text segment, and then uses few-shot prompting to format that text into the same JSON schema [2403.07553].

The annotated extraction dataset consists of **200 ToC images**, split **90% train, 10% test**, spanning at least **“old format”** and **“MasterFormat”** construction specifications. For the extraction model, evaluation is reported on **20 test documents**. Donut directly outputs a JSON-formatted string, after which the JSON is parsed, stored in a database, and displayed through a **React/Next.js dashboard** served through a **Flask API**. The final JSON supports section-level indexing, UI navigation, and project-level linkage to downstream artifacts such as drawings, RFIs, or contracts [2403.07553].

Reported extraction results differentiate numeric and textual fields. On the 20 test documents, Donut achieves **82.2% overall ToC extraction accuracy**; field-level results show **0.92** for **heading numbers**, **0.78** for **heading titles**, **0.91** for **subheading numbers**, **0.76** for **subheading titles**, and **0.85 average**. GPT-3.5 Turbo reaches **0.92** for heading numbers, **0.83** for heading titles, **0.88** for subheading numbers, **0.84** for subheading titles, and **0.89 average**. The contrast is explicit: both methods perform strongly on numeric fields, while GPT is stronger on the free-form textual fields. In this application-oriented sense, “Donut-MINT” designates a **Donut-centered indexing stack** rather than a compressed VQA model, but the common thread remains Donut’s role as an OCR-free document-structure engine [2403.07553].

## 6. Limitations, generalization, and adjacent Donut-derived directions

The formal Donut-MINT line identifies several limitations. There remains a **15-point ANLS gap** between the teacher and **Donut-MINT\(_{7\%}\)**, which may be prohibitive in high-stakes deployments. The interpretability study is **task- and domain-specific**, centered on Donut’s DocVQA behavior and especially on its tendency to **copy text from images**. The mechanistic analysis is also described as largely **manual**, relying on designed patching experiments and visual inspection. In addition, the distillation stage matches only **final logits**, not internal activations or circuits, leaving open the possibility of more circuit-aware training objectives [2509.26235].

At the same time, the method is presented as a candidate **general strategy** for other VLMs with explicit cross-attention between vision encoder and language decoder. The paper names **BLIP**, **LLaVA-style VLMs**, and **Pix2Struct** as architectures that could potentially benefit from the same pattern of **interpret, then prune, then distill**. Proposed extensions include **automated circuit discovery**, **attribution patching**, and **intermediate-activation distillation** targeted at the specific subcircuits identified during analysis [2509.26235].

A related but distinct Donut-derived direction appears in trajectory forecasting. Work on **DONUT-NLL** does **not** use the name Donut-MINT, but it develops a distribution-first framework in which a Donut-based forecaster is trained with **metric-agnostic NLL objectives** and benchmark-specific outputs are generated only afterward through **Trajectory Distribution Evaluation (TraDiE) policies**. That paper treats **metric optimization as a downstream task applied to the predictive distribution**, rather than as something built directly into the training loss. This suggests a broader conceptual family of Donut variants in which the core model learns a calibrated latent or predictive structure, and downstream modules specialize it for compression, indexing, or metric-conditioned evaluation [2607.01133].

Taken together, these usages show that Donut-MINT is not merely a single model checkpoint name. In its strict sense it is an **interpretability-guided compression methodology for Donut**; in looser applied usage it refers to **Donut-centered document indexing workflows**; and in adjacent Donut research it resonates with a wider move toward **modular specialization**, where the Donut substrate is preserved and higher-level capabilities are supplied by pruning rules, JSON schemas, or metric-specific policies.

Source: https://www.emergentmind.com/topics/donut-mint