Synthesizing Backward Error Bounds, Backward

Abstract: Backward stability is a desirable property for a well-designed numerical algorithm: given an input, a backward stable floating-point program produces the exact output for a nearby input. While automated tools for bounding the forward error of a numerical program are well-established, few existing tools target backward error analysis. We present a formal framework that enables sound, automated backward error analysis for a broad class of numerical programs. First, we propose a novel generalization of the definition of backward stability that is both compositional and flexible, satisfied by a wide range of floating-point operations. Second, based on this generalization, we develop the category Shel where morphisms model stable numerical programs, and show that structures in Shel support a rich variety of backward error analyses. Third, we implement a tool, eggshel, that automatically searches within a syntactic subcategory of Shel to prove backward stability for a given program. Our algorithm handles many programs with variable reuse, a known challenge in backward error analysis. We prove soundness of our algorithm and use our tool to synthesize backward error bounds for a suite of programs that were previously beyond the reach of automated analysis.

• The paper presents a novel categorical framework using relational backward error lenses to extend backward error analysis for floating-point programs.
• It introduces the Shel category, enabling compositional reasoning and automated symbolic search via the eggshel tool, achieving tight per-variable error bounds.
• Numerical evaluations demonstrate the framework’s effectiveness in handling variable reuse, complex dependencies, and challenging floating-point computations.

Synthesizing Backward Error Bounds via Relational Lenses and Category Theory

Background and Motivation

Backward error analysis is foundational in the evaluation of numerical algorithms, particularly those operating under floating-point semantics. While forward error analysis quantifies deviation at the output, backward error asks: To what perturbed input does the computed result correspond if interpreted as the exact outcome of a slightly different problem? This distinction is critical, as backward stability decouples implementation quality from the inherent conditioning of the numerical problem, providing a more algorithm-centric robustness measure.

Despite the prominence of backward stability in numerical mathematics, existing automated tools for floating-point programs overwhelmingly focus on forward error bounds (e.g., FPTaylor, PRECiSA, Gappa). Bean previously pioneered a sound, automated approach to backward error analysis using affine type systems and categorical semantics, but it fundamentally limited the compositionality and failed to support many programs involving variable reuse or more complex primitive operations.

The paper "Synthesizing Backward Error Bounds, Backward" (2604.15633) introduces a categorical approach built on relational backward error lenses and the category $Shel$, yielding sound compositional backward error analysis beyond the capabilities of prior systems.

Relational Backward Error Lenses

The classical definition of backward stability does not preserve composition: composing two individually backward stable program fragments does not, in general, guarantee backward stability of the composition, especially in the presence of variable reuse. To resolve this, the authors generalize Bean's lens framework, introducing relational backward error lenses, which:

• Remove the non-expansiveness constraint of Bean's original lenses, allowing the distance between perturbed input $x'$ and actual input $x$ to be controlled more flexibly as a function of output shifts.
• Support variable reuse by representing dependencies between shifts in variables via group actions—enabling compositional reasoning in more expressive program fragments.

A relational lens $(f, \tilde{f}, b)$ (where $f$ is the ideal function, $\tilde{f}$ is the floating-point implementation, and $b$ is the backward map) ensures that for every floating-point input $x$ and every output $y$ close to $\tilde{f}(x)$, there is some input $x'$0 close to $x'$1 such that $x'$2. This relaxation retains soundness for backward error guarantees while strictly enlarging the class of composable proofs.

Relational lenses compose naturally, generalizing the compositionality of non-expansive lenses and capturing non-obvious backward stability instances such as square roots, logarithms, and complex polynomial structures.

The $x'$3 Category: Morphisms and Structure

To systematize and automate the compositional analysis, the authors formalize $x'$4—the category of shifted error lenses:

• Objects: Triples $x'$5 where $x'$6 is the set of variable values, $x'$7 is an abelian group of shifts with a group action and a componentwise norm, and $x'$8 is a per-variable backward error budget.
• Morphisms: Triples $x'$9 with compatibility conditions on shifts, ensuring that shifted computations in the output space can be mapped to appropriate shifted inputs.
• Tensor Product: Allows parallel composition of variables (contexts) and morphisms.
• Push Product: Generalizes the tensor, enabling correlated perturbations when a variable's shift must be propagated to a dependent variable (essential in variable reuse patterns).
• Share Product: Provides bounded duplication; variables can be duplicated but must receive identical perturbations.

This categorical infrastructure is rich: $x$0 is symmetric monoidal (enabling arbitrary reassociation and permutation of contexts), supports push products for generalized dependency management, and is expressive enough to encode subtle variable sharing patterns typical in numerically stable routines (e.g., dot products, Cholesky factorization).

The construction of morphisms for complex programs boils down to contextual application of primitive and structural lenses, ensuring all composed shifts preserve the required backward stability constraints.

Automation: The Eggshel Tool

Automating this reasoning, the paper introduces eggshel, implemented in the $x$1 reasoning system. The tool operates by symbolic backward search:

• It starts from a goal (e.g., proving a program is backward stable with certain per-variable budgets) and composes lenses in reverse-syntax order, leveraging both computational (e.g., $x$2, $x$3, $x$4) and structural (e.g., tensor, push, share) lenses.
• Contexts are represented syntactically with variable-shift associations, and a logic programming engine (augmented by equality saturation) resolves lens application, context manipulation, and variable-shift reparameterization.
• Given a program, eggshel synthesizes a compositional proof of backward stability, providing explicit per-variable backward error bounds.

Strong numerical performance results are detailed for classical floating-point routines including sum, norm, dot products, polynomial evaluation, and even matrix factorizations such as Cholesky for small matrices. The framework robustly handles intricate patterns of variable reuse and output dependencies, delivering analyses previously unattainable by other tools—often with backward error bounds matching or improving on previous analyses.

Numerical Results and Evaluation

The system is evaluated on a spectrum of numerical kernels:

• Variable-size programs: For e.g., sums of $x$5 inputs, $x$6-variate polynomials, and dot products, the tool reports backward error bounds per input that scale as expected ($x$7 operations yield max backward error bound $x$8 for unit roundoff $x$9), matching theoretical predictions from numerical analysis.
• Challenging variable-reuse programs: Programs such as $(f, \tilde{f}, b)$0, $(f, \tilde{f}, b)$1, and polynomials with repeated subexpressions are analyzed automatically, synthesizing backward error witnesses not supported by Bean or previous type-based methods.
• Structural limitations: For very large program sizes, search and rewrite space management become the main bottlenecks, suggesting future work on optimized lens composition strategies.

Theoretical and Practical Implications

Theoretical Contribution: The definition and formalization of relational backward error lenses and the $(f, \tilde{f}, b)$2 category unify and extend previous attempts at categorical backward analysis in numerical software. The framework brings new proof-theoretic tools for describing algorithmic stability, offers a broader compositional calculus, and bridges category-theoretic abstractions with concrete floating-point semantics.

Practical Impact: eggshel and its underlying method enable rigorous, automated backward error analyses for a much broader class of floating-point programs, especially those that reuse variables or require floating-point primitives beyond simple arithmetic. This opens the door for wider adoption of backward stability checking in scientific software, numerical libraries, and could inform compiler or optimizer techniques that preserve numerical soundness.

Future Directions

• Optimization of Search and Rewriting: Current bottlenecks can likely be alleviated with more aggressive context canonicalization, lens indexing, or even cost-guided search for minimal error bounds.
• Conditional and Structural Properties: Encoding domain-specific knowledge (e.g., SPD criteria for Cholesky) could enable even more powerful analyses for structured numerical algorithms.
• Extending to Probabilistic and Other Non-deterministic Rounding Models: Extending $(f, \tilde{f}, b)$3 to model stochastic or hardware-induced rounding error variations would broaden applicability to emerging hardware and probabilistic numerical algorithms.

Conclusion

The work fundamentally advances the state of automated backward error analysis by providing a sound, compositional, and programmable approach with the flexibility to handle complex variable dependencies and richer sets of floating-point operations. The combination of categorical semantics, lens-based abstraction, and symbolic search offers a robust foundation for future research in certified numerical computing, optimization of floating-point software, and potentially the formal verification of numerical methods in critical scientific applications.

For a detailed formalization and implementation, see "Synthesizing Backward Error Bounds, Backward" (2604.15633).