LLMs Corrupt Your Documents When You Delegate

Abstract: LLMs are poised to disrupt knowledge work, with the emergence of delegated work as a new interaction paradigm (e.g., vibe coding). Delegation requires trust - the expectation that the LLM will faithfully execute the task without introducing errors into documents. We introduce DELEGATE-52 to study the readiness of AI systems in delegated workflows. DELEGATE-52 simulates long delegated workflows that require in-depth document editing across 52 professional domains, such as coding, crystallography, and music notation. Our large-scale experiment with 19 LLMs reveals that current models degrade documents during delegation: even frontier models (Gemini 3.1 Pro, Claude 4.6 Opus, GPT 5.4) corrupt an average of 25% of document content by the end of long workflows, with other models failing more severely. Additional experiments reveal that agentic tool use does not improve performance on DELEGATE-52, and that degradation severity is exacerbated by document size, length of interaction, or presence of distractor files. Our analysis shows that current LLMs are unreliable delegates: they introduce sparse but severe errors that silently corrupt documents, compounding over long interaction.

• The paper introduces the DELEGATE-52 benchmark to evaluate LLM reliability in long-horizon document editing tasks.
• It demonstrates that iterative delegated interactions lead to over 20% loss or corruption of semantic content, varying notably by domain.
• Experimental results highlight multiplicative degradation factors and expose limitations in both text and image-based delegation workflows.

Document Corruption in Delegated LLM Workflows: A Technical Analysis of "LLMs Corrupt Your Documents When You Delegate" (2604.15597)

Introduction

The study introduces the DELEGATE-52 benchmark to systematically evaluate LLM reliability for long-horizon document editing workflows across 52 professional domains. It addresses the emerging paradigm of delegated work, where LLMs are entrusted with end-to-end task execution on user documents. The central finding is that contemporary LLMs—across scale and architecture—induce substantial, compounding corruption in documents over iterative delegated interactions, undermining the premise of trust required for effective delegation.

Benchmark Design: DELEGATE-52

DELEGATE-52 simulates extensive knowledge work across five domain categories (Science & Engineering, Code & Configuration, Creative & Media, Structured Records, and Everyday tasks).

Figure 1: DELEGATE-52 spans 52 professional domains, ensuring broad professional coverage for document-centric workflows.

Each domain contains six distinct work environments built from real-world seed documents (~2-5k tokens). Every environment specifies 5–10 complex, realistic editing tasks defined via reversible instruction pairs (forward/backward), operationalized in the round-trip relay setting. The benchmark departs from prior domain-limited editing benchmarks by scaling semantic evaluation through custom domain-specific parsing and structured similarity metrics for each domain.

Figure 2: Example from the accounting domain: a ledger document, split and merged via reversible editing instructions.

The round-trip relay methodology measures LLM fidelity by comparing the original document $s$ with the reconstruction $\hat{s}$ after an $n$-step composition of reversible edits. Perfect LLM behavior yields a reconstruction score (RS) of 1.0 at every step, enabling reference-free, compositional evaluation.

Figure 3: Domain-level evaluation leverages specialized parsing and similarity functions for precise, semantically calibrated scoring.

The evaluation framework ensures that semantic equivalence—not superficial textual overlap—determines score outcomes; ablations demonstrate that generic similarity metrics (Levenshtein, ROUGE, BERTScore, LLM-as-judge) explain only a fraction of human-calibrated semantic variance.

Experimental Results

A large-scale simulation with 19 cutting-edge LLMs (OpenAI, Anthropic, Google, Moonshot, xAI, Mistral) exposes severe limitations in delegated document workflows.

Aggregate Degradation

After 20 interactions (10 round-trip relay cycles), the mean score across all models is approximately 50%, indicating that half of semantic document content is lost or corrupted. Even frontier models (Gemini 3.1 Pro, Claude 4.6 Opus, GPT-5.4) average only 71.5–80.9% reconstruction under the same conditions.

Figure 4: LLMs introduce silent, accumulating document corruption in long-horizon workflows—examples span diagrams, textile patterns, and 3D objects.

Performance is notably domain-dependent: Python code manipulation is the sole area where most models are delegation-ready (RS@20 ≥ 98%), while the majority of domains (80% of all model-domain pairs) exhibit catastrophic (>20%) degradation (Table 2, domain breakdown, omitted for brevity).

Degradation Dynamics

Degradation is neither gradual nor evenly distributed. A small number of critical rounds (≥10% drop in RS) account for ≥80% of total degradation. Weaker models primarily induce deletions, while stronger models shift toward corruption of existing content (hallucination, misclassification, or structural disturbance).

Compounding Factors

Empirical analysis demonstrates multiplicative compounding from several factors:

• Longer Documents: Each 1k-token increment progressively increases degradation, with the effect amplifying over multiple iterations.
• Longer Relays: Degradation accelerates with horizon length; no plateau is observed up to 100 interactions.
• Distractors: Presence of irrelevant files exacerbates errors; impact grows disproportionately with interaction length.
• Single- vs. Multi-task Scheduling: Task diversity sharply increases error accumulation versus repeating a single (overfit) editing instruction.
• Agentic Tool Use: Contrary to expectations, agentic LLMs with file and code tool access often perform worse than direct models, likely due to token processing overhead and failure to leverage code execution for complex, non-trivial edits.

Modality Extension

The methodology generalizes to image editing. Nine image generation models subjected to similar relay evaluation degrade visual documents even more rapidly than text: no model retains >30% reconstruction after 20 interactions. This signals severe reliability gaps for multimodal delegated workflows.

Figure 5: Progressive visual degradation across models in an image relay—fidelity rapidly collapses versus textual relays.

Analysis of Difficulty and Failure Modes

• Domains: LLMs excel in highly structured, repetitive domains (code, chemicals, chess), but perform poorly with natural language prose, low redundancy, and high vocabulary diversity.
• Semantic Operations: Tasks involving global document restructuring (split/merge, classification) are most error-prone; local operations (string manipulation, referencing) are less so.
• Content Loss Taxonomy: Weak models primarily suffer from deletion; frontier models tend to corrupt retained content instead.

Figure 6: Semantic operation frequencies and point-biserial correlations with relay performance highlight the categorical sources of failure.

Implications

For Developers

DELEGATE-52 is positioned as both an evaluative and generative tool: its reversible edit constructs create natural “mini-gym” reinforcement learning environments. Cycle consistency can be exploited for reward signal specification, but care must be taken to avoid reward hacking through spurious invariance or identity transformations.

For Benchmarks

The findings demonstrate that short-horizon metrics grossly overestimate model reliability and that wide, multi-domain, and long-horizon benchmarks are critical to predeployment validation for knowledge work automation.

For Users and Product Managers

Current delegation workflows are not robustly trustworthy except in isolated, highly structured domains. Users must carefully supervise outputs, and cannot extrapolate domain-specific LLM performance to arbitrary professional workflows.

Theoretical and Practical Limitations

• Single-turn sessions: Multi-turn conversational edits are not addressed; in practice, real delegation may be even more fragile.
• Domain specificity: The framework favors structured domains with clear semantic parsing; open-ended creative tasks remain challenging for automatic evaluation.
• Edit reversibility: Benchmark is constrained to lossless/invertible tasks, although book-keeping extensions (auxiliary files) partly alleviate this.
• Resource scaling: Real-world scenarios may involve even larger contexts and longer horizons, which would likely amplify observed failures.

  Figure 7: The creation of DELEGATE-52 was itself a semi-agentic workflow, highlighting both the necessity and challenges of reliable LLM-based task delegation.

Conclusion

This research demonstrates that even state-of-the-art LLMs, under careful evaluation, are unreliable for long-horizon delegated document editing: sparse but severe errors accrue, compounding silently, and systematically corrupt professional artifacts. The DELEGATE-52 benchmark establishes a new standard for evaluating real-world LLM deployment in knowledge work and provides a controlled platform for future developments in robust, agentic, and multi-modal document manipulation (2604.15597).