Inferring Equivalence Classes from Legacy Undocumented Embedded Binaries for ISO 26262-Compliant Testing

Published 24 Apr 2026 in cs.SE and cs.SC | (2604.22673v1)

Abstract: Equivalence class partitioning is a well-established test design technique mandated by safety standards such as ISO~26262 for systematic testing of safety software. In industrial practice, however, its application to legacy undocumented embedded firmware is often hindered by incomplete or outdated functional specifications. This paper proposes a binary-level methodology for inferring output-oriented equivalence classes directly from compiled firmware, without relying on source-level annotations or external documentation. The approach combines control-flow reconstruction and guided symbolic execution to analyze individual functions and group execution paths according to indistinguishable observable behavior, including return values and output parameters. An optional post-processing step produces human-readable representations to support comprehension and documentation. The methodology is evaluated in an industrial automotive context through a practitioner-based study assessing correctness and interpretability. Results indicate strong alignment with expert expectations and a positive perception of readability and usefulness for supporting function understanding and test design. These findings demonstrate the feasibility and practical relevance of binary-level equivalence class inference for systematic testing of legacy undocumented safety-embedded software.

Abstract PDF Upgrade to Chat

Authors (5)

Summary

The paper introduces a binary-level methodology that infers output-oriented equivalence classes directly from compiled firmware, bypassing undocumented source information.
It employs control- and data-flow extraction combined with guided symbolic execution to cluster function behaviors based on observable outputs.
Industrial validation confirmed an 83% alignment between tool-generated equivalence classes and expert expectations, supporting ISO 26262 compliance.

Inferring Equivalence Classes from Legacy Undocumented Embedded Binaries

Introduction and Motivation

Equivalence class partitioning (ECP) is a mandatory test design method prescribed by ISO 26262 for systematic software verification in automotive and safety-critical domains. Yet, practical ECP adoption is severely impeded in industrial environments by large-scale legacy firmware deployments that lack up-to-date documentation and reliable specifications. This paper presents a binary-level methodology which supports ISO 26262-compliant ECP by inferring output-oriented equivalence classes directly from compiled (DWARF-annotated) firmware binaries, without relying on source code annotations or external artifacts (2604.22673).

The approach integrates control- and data-flow extraction, call graph analysis, and guided symbolic execution to partition the function input domain according to indistinguishable external behaviors (covering return values and output/interface effects). Output-oriented ECP enables systematic input-domain coverage aligned with safety certification requirements, even in the absence of explicit specifications or incomplete source-level documentation. The methodology has been validated in a real industrial context via a practitioner study, with strong alignment between inferred ECs and expert expectations.

Methodology Overview

The methodology executes in two main phases: (1) binary structural analysis and function classification, and (2) symbolic execution with equivalence class inference.

Structural Analysis and Function Clustering

The process operates strictly on the final binary, exploiting DWARF debug symbols (and map files) to recover fine-grained function boundaries, signatures, and type information. This guarantees fidelity to the actual deployed firmware configuration—a non-trivial gain over source-level methods, given extensive configuration-dependent variability in industrial embedded software.

The control flow graph (CFG) and call graph are reconstructed for all functions:

Figure 1: Phase 1 overview, depicting binary structural analysis and function dependency clustering.

Dependency-aware clustering leverages call depth and accessed global variables, structuring the input for symbolic analysis and summary-based compositional reasoning. This mitigates path explosion and scales the analysis to firmware-scale binaries.

Symbolic Execution and EC Extraction

The second phase processes functions per cluster, instantiating parameters as constrained symbolic bit-vectors, leveraging type domains explicitly parsed from DWARF data.

Figure 3: Phase 2 overview, showing symbolic execution, constraint collection, path merging, and generation of equivalence classes.

Symbolic execution (using angr as backend) is guided and bounded (with practical industrial limits such as finite loop unrolling). Each unique observable behavior (over output parameters/returns/globals) identifies an output-oriented equivalence class, grouping symbolic paths by output. Path conditions are normalized, simplified, and optionally post-processed into natural language via a constrained LLM component, optimized strictly for human readability and traceability. The entire process is deterministic and serializer output is available in JSON for toolchain integration.

Illustrative Examples

Conditional Branch Partitioning

Functions that structurally encode domain-partitioning via conditional branches result in well-formed, output-oriented equivalence classes as the symbolic execution groups paths by return value/effect. Control-flow recovery, path exploration, and constraint extraction collectively enable high-fidelity ECP fully aligned with actual implementation logic.

Figure 2: Source code, CFG, and symbolic path conditions for a typical conditional-partitioned function.

Loop-induced Partitioning

Output equivalence classes are not sensitive to syntactic path counts, but rather to semantically distinct observable behaviors. As shown below, multi-path exploration (e.g., induced by loops) is collapsed into compact ECs whenever postconditions and outputs are identical.

Figure 4: Code, CFG, and symbolic path conditions for a function with loop-based partitioning.

Bitwise and Arithmetic Path Constraints

Equivalence class boundaries are preserved across arithmetic transformations and nontrivial bitwise manipulation, and LLM-guided post-processing improves human readability for review and audit, further enabling adoption in safety-critical workflows.

Figure 5: Source code, CFG, and symbolic constraints for a function involving bitwise/arithmetic branching.

Industrial Evaluation and Empirical Results

Validation is based on an in-situ practitioner study (n=12), with participants independently reverse engineering and designing ECPs for 27 pure computational functions (median 31 LOC, representative of industrial firmware). Generated equivalence classes were then compared to manual baselines, and assessment performed via Likert-scale feedback and categorical metrics.

Correctness and Fidelity

All participants reported that tool-generated ECs matched or exceeded their own expectations for function-level behavioral partitioning. 83% of responses confirmed alignment between tool results and domain knowledge for functional behavior mapping. Perceived missing information was mostly related to secondary details (e.g., explicit edge-case boundaries, path feasibility), not primary output partitioning.

Figure 6: Likert-scale rating distribution for functional correctness and information adequacy (Q1–Q3).

Readability and Usability

Readability and interpretability received high ratings (median of "Agree"). Human-friendliness was further enhanced by structured formatting and replacement of low-level literals by macro-names where possible. Machine-readable output (JSON) and concise human summaries were both seen as necessary for downstream integration and certification documentation. Participants prioritized concise EC sets and simplified constraints for test design and review, rather than branch-specific verbose path listings.

Figure 7: Likert-scale rating distribution for readability, interpretability, and usability (Q5, Q8, Q9).

Lessons Learned and Implications

Direct binary analysis ensures ECs accurately reflect deployed firmware, unaffected by build variability and configuration drift.
Automation bridges the documentation gap identified in industry, directly supporting ISO 26262 ECP mandates without access to formal specifications.
LLM post-processing is valuable solely for readability, and industry adoption is strongly modulated by output format and traceability rather than only technical correctness.
Compositional analysis with symbolic execution summaries (recursively updated per call graph cluster) scales to firmware-sized binaries.
Output-oriented definitions (i.e., equivalence by observable effect) are suitable and practically relevant for safety cases and audit preparation.

Prior approaches (manual partitioning, model-based test generation, static/dynamic specification mining, symbolic execution, constraint logic programming) have addressed specification extraction, test input generation, and path-based invariant inference, but have not delivered industrially viable, binary-level automated ECP aligned with safety standards. Notably, existing tools focus chiefly on API usage, invariants, or API-centric automated test generation, lacking direct support for equivalence partitioning as a first-class artifact in audit-driven testing of embedded safety software [Huang2016, Udeshi2024, weideman2021perfume, Albert2014, Winkelmann2023].

Conclusion

Binary-level ECP via symbolic execution and constraint merging enables reliable, scalable equivalence class extraction aligned with safety standard mandates for legacy, undocumented embedded software. The approach addresses a critical industrial gap, as confirmed by practitioner validation in an automotive setting. Future work should address richer interface models (e.g., hardware-induced side effects, concurrency), more generalized path-merging strategies (loop/count abstraction), and larger cross-domain empirical studies.

References

"Inferring Equivalence Classes from Legacy Undocumented Embedded Binaries for ISO 26262-Compliant Testing" (2604.22673)
[Huang2016]
[Udeshi2024], [weideman2021perfume]
[Albert2014], [Winkelmann2023]