Formal Error Taxonomy

• Formal Error Taxonomy is a rigorously defined hierarchical framework that categorizes and analyzes error phenomena across diverse technical fields.
• It employs structured layers, formal definitions, and logical notations to ensure accurate classification and reproducibility.
• Practical applications include AI model evaluation, benchmark design, and error diagnosis in domains like EFL writing, spreadsheets, and quantum processes.

A formal error taxonomy is a rigorously defined hierarchical framework for categorizing, distinguishing, and analyzing errors and error-like phenomena within a given technical domain. Example implementations span domains including EFL writing analysis (Heywood et al., 29 Nov 2025), infrastructure-as-code LLM code generation (Nekrasov et al., 16 Dec 2025), spreadsheet faults (Kulesz et al., 2018, Przasnyski et al., 2011), retrieval-augmented generation systems (Leung et al., 15 Oct 2025), medical QA (Roy et al., 2024), text simplification (Vendeville et al., 22 May 2025), formal-language pedagogy (Schmellenkamp et al., 2024), error generators in quantum processes (Blume-Kohout et al., 2021), and aggregated Markov chain error bounds (Michel et al., 2024). These taxonomies are often organized by dimensions reflecting validation stages, underlying failure mechanics, linguistic or semantic level, or operational impact. Modern approaches formalize both taxonomy structure and classification procedures, employing set-theoretic, functional, or logical notation, and support application in AI-assisted evaluation pipelines or benchmark design.

1. Theoretical Foundations and Structuring Principles

Formal error taxonomies are anchored in domain-specific theories, classical frameworks, and standardized anomaly models. For EFL error analysis, Heywood et al. synthesize Corder’s distinction between systematic errors and mistakes (G_L ≠ G_T), Richards’ source-based partition (interlingual, intralingual, developmental), and James’ pragmatic/communicative effect paradigm (Heywood et al., 29 Nov 2025). Spreadsheet error taxonomy, “Asheetoxy,” eschews the ambiguous “error” label, instead organizing observable phenomena (wrong actions, defects, failures, problems) and causal chains (X ⇒ Y) (Kulesz et al., 2018). ML I/O error models (Isakov et al., 2022) decompose global prediction error e(j) into components reflecting application, system, dataset, contention, and noise mechanisms.

Hierarchical organization typically encompasses multiple layers:

• Top-level: distinct domains or validation stages (e.g., syntax, schema, runtime, intent (Nekrasov et al., 16 Dec 2025); accessibility, bias, clinical reasoning, communication, privacy (Chen et al., 26 Sep 2025); fluency, alignment, information, simplification (Vendeville et al., 22 May 2025)).
• Mid-level: subdomains by error source, linguistic level, pipeline stage, or semantic effect.
• Leaf-level: atomic, mutually exclusive error codes, phenomena, or predicates (e.g., “Hard-coding,” “MisinterpretationOfClinicalQuery,” “Unsupported argument”).

Formal representations include rooted trees, set partitions, functional mappings (f_word, f_sent), and category-specific indicator functions (e.g., I_{eᵢ}(M) for spreadsheet atomic errors (Przasnyski et al., 2011)). Taxonomy completeness, exclusivity, and causal linkages are enforced by design constraints.

2. Taxonomy Hierarchy and Formal Definitions

Each taxonomy defines categories and classification functions with precise mathematical or logical notation. Representative examples:

2.1 Example: EFL Error Hierarchy (Heywood et al., 29 Nov 2025)

$$E = \{\mathrm{SP}\} \cup \{\mathrm{GW}_1,\dots,\mathrm{GW}_{13}\} \cup \{\mathrm{GS}_1,\dots,\mathrm{GS}_6\}$$

Mappings: $$f_\mathrm{word}: \text{tokens}(s) \rightarrow 2^{E_\mathrm{word} \cup \{\mathrm{SP}\}} \qquad f_\mathrm{sent}: \text{sentence }s \rightarrow 2^{E_\mathrm{sent}}$$ With rules (e.g., (R1) spelling overrides, (R2) sentence-level suppression of word overlaps).

2.2 Example: Asheetoxy (Kulesz et al., 2018)

$$P_\text{total} = P_\mathrm{WA} \cup P_\mathrm{A} \cup P_\mathrm{PR}$$

Defects: $D = I \cup F,\quad F = L \cup M$ Cause-effect: $$P_\mathrm{WA} \Rightarrow D \Rightarrow U \Rightarrow P_\mathrm{PR}$$ Formal predicates define each phenomenon; for instance, a “Wrong Action” is any observed human operation that introduces a negative artifact.

2.3 Example: ML I/O Model Error Decomposition (Isakov et al., 2022)

$$e(j) = e_\text{app} + e_\text{system} + e_\text{OoD} + e_\text{contention} + e_\text{noise}$$

Each term is formally characterized (e.g., e_\text{app}(j) ≡ f_a(j) - m(j_o, ζo); e\text{noise} irreducible via stochastic system term ω).

3. Classification Methodologies and Decision Rules

Taxonomy development proceeds via literature-driven aggregation, empirical review, inductive coding, and expert adjudication. Procedures include:

• Open and axial coding of empirical error logs or model outputs, grouping by manifestations and underlying causes (Nekrasov et al., 16 Dec 2025, Chen et al., 26 Sep 2025).
• Litmus tests for error-source detection (duplicate error bound for application modeling, system-feature injection for global system error, uncertainty quantification for OoD coverage (Isakov et al., 2022)).
• Formal indicator functions and labeling schemas (e.g., span-level multi-label annotation (Roy et al., 2024, Vendeville et al., 22 May 2025); instance-wise error indicators (Przasnyski et al., 2011)).
• Rule-based or probabilistic assignment procedures for atomic error types; enforcement of mutual exclusivity within subdomains and free co-occurrence across domains/subdomains (Chen et al., 26 Sep 2025).

Tables and diagrams may summarize category incidence, error flow, or Sankey transitions (e.g., IaC script error cascades (Nekrasov et al., 16 Dec 2025)).

4. Domains of Application and Representative Taxonomies

Formal error taxonomies span a broad spectrum:

Domain	Example Taxonomy	Key Dimensions
EFL Writing	Heywood et al. (Heywood et al., 29 Nov 2025)	Word/sentence level, spelling, grammar, punctuation
Spreadsheet Modelling	Asheetoxy (Kulesz et al., 2018), Przasnyski (Przasnyski et al., 2011)	Phenomenon hierarchy, qualitative flaws, atomic indicators
IaC Generation (Terraform)	Peng et al. (Nekrasov et al., 16 Dec 2025)	Validation stage, LLM failure pattern
RAG Systems	Zhang et al. (Leung et al., 15 Oct 2025)	Pipeline stage, error type
Medical QA	Roy et al. (Roy et al., 2024)	Reasoning, knowledge, comprehension, non-error categories
Text Simplification	Vendeville et al. (Vendeville et al., 22 May 2025)	Fluency, alignment, info distortion, simplification
HPC I/O ML Models	Liu et al. (Isakov et al., 2022)	Application, system, dataset, contention, noise
Quantum Process Generators	Greenbaum (Blume-Kohout et al., 2021)	Coherent/incoherent/non-unital, Pauli support
Markov Aggregation	Meyer et al. (Michel et al., 2024)	Dynamic error bounds, lumpability, aggregation

5. Limitations, Inter-category Constraints, and Extension Plans

Formal taxonomies are subject to known constraints and evolving capabilities:

• Many frameworks do not differentiate intent (deliberate vs accidental) or attach risk/severity natively; intent and severity tagging are considered future extensions (Kulesz et al., 2018, Heywood et al., 29 Nov 2025, Chen et al., 26 Sep 2025).
• Inter-category exclusivity is managed by design (e.g., one code per subdomain) with freedom to co-apply codes across domains/subdomains.
• Contextual and cross-sentence errors remain challenging for sentence-isolated taxonomies; sliding-window or document-level context is a planned feature (Heywood et al., 29 Nov 2025).
• Some domains lack coverage of advanced features (e.g., macros, external links in spreadsheets (Przasnyski et al., 2011)), stylistic/discourse issues in NLP, or deep semantic errors in code synthesis.
• Refinement and validation cycles incorporate human panel adjudication, reliability studies, and automated annotation system development (Chen et al., 26 Sep 2025, Leung et al., 15 Oct 2025).
• Extension methods involve domain expert input, pilot annotation, and iterative metric-based category revision (Roy et al., 2024).

6. Significance and Practical Impact

Evaluation frameworks for robust AI and human-centric systems now routinely rely on formal error taxonomies for:

• Benchmark design and evaluation, yielding fine-grained diagnosis and targeted mitigation (Nekrasov et al., 16 Dec 2025, Leung et al., 15 Oct 2025, Isakov et al., 2022).
• Automated annotation pipelines, guardrails for sensitive domains (e.g., clinical messaging, medical QA (Chen et al., 26 Sep 2025, Roy et al., 2024)).
• Improved interpretability and actionable feedback for educators, developers, and system managers.
• Theoretical development including proof of error bounds, extension to probabilistic and dynamic systems (Michel et al., 2024, Blume-Kohout et al., 2021).
• Support for extension to new domains by adaptation criteria (multi-labeling, completeness, disjointness, concept coverage (Roy et al., 2024)).

Formal error taxonomies thus provide a foundational apparatus for rigorous, reproducible, and domain-adaptable error analysis across scientific, engineering, and educational contexts.