Bit-Exact Conformance Packs Overview

• Bit-Exact Conformance Packs are self-contained artifacts that record execution metadata, arithmetic semantics, and expected outputs to guarantee bit-level reproducibility.
• They enable strict validation in domains like AI inference, numeric encoding, random number generation, and simulation through deterministic workflows.
• They mitigate risks from floating-point non-associativity and platform variability by fixing execution environment details and reducing unverifiable degrees of freedom.

Bit-Exact Conformance Packs are self-contained artifacts that make reproducibility and verification a byte-level or bit-level property rather than an approximate one. Across recent work, the term denotes packages that fix enough metadata, state, arithmetic semantics, and expected outputs to support exact replay or exact encode/decode validation: in AI inference, $\hat{y}=\mathrm{Recompute}(x;\mathrm{Pack})$ and $$\mathrm{Pass}\Leftrightarrow \mathrm{bytes}(\hat{y})=\mathrm{bytes}(y)$$; in numeric-format catalogs, each row fixes an input value, the exact target-format bit pattern, the decoded value back in double precision, and the absolute error; in reproducible random-number generation, acceptance is exact byte equality for raw words and double encodings; and in Simulink conformance, bit-exactness requires $$\mathrm{bitpattern}(x_{\mathrm{Tool}})=\mathrm{bitpattern}(x_{\mathrm{Simulink}})$$ for all outputs and updated states (Cankaya, 29 May 2026, Vasilev, 8 Jun 2026, Jónasson, 6 May 2026, Ishii et al., 23 Jun 2026).

1. Formal semantics of bit-exactness

Bit-exactness is stricter than numerical closeness. In AI inference verification, it requires that the verifier can recompute the prover’s model outputs, byte for byte, without access to the prover’s hardware, and that for any intermediate tensor or final output $y$, $\hat{y}=\mathrm{Recompute}(x;\mathrm{Pack})$ passes only when $\mathrm{bytes}(\hat{y})=\mathrm{bytes}(y)$. The same paper distinguishes true non-determinism, such as floating-point atomic operations, from determinism with non-invariance, where $f(x;\mathrm{cfg})$ is deterministic while not invariant across $\mathrm{cfg}$ because reduction order, kernel choice, or batch size changes rounded sums (Cankaya, 29 May 2026).

Equivalent ideas appear in other domains, but the equality object changes. In Simulink model checking, equality is defined on raw bit patterns rather than on mathematical reals or integers, using IEEE-754 encodings for floating-point and two’s-complement bit vectors for integers and fixed-point. In blockchain conformance testing, the object of comparison is canonical serialization at the protocol boundary, with conformance requiring $\sigma(s_k)=\sigma_I(s'_k)$ and, for commitments, $H_k=H'_k$. In Randompack, bit-exact outputs are bit-identical across programming languages, CPU architectures, compilers, and operating systems for raw bitstreams, discrete distributions, $$\mathrm{Pass}\Leftrightarrow \mathrm{bytes}(\hat{y})=\mathrm{bytes}(y)$$0, $$\mathrm{Pass}\Leftrightarrow \mathrm{bytes}(\hat{y})=\mathrm{bytes}(y)$$1, $$\mathrm{Pass}\Leftrightarrow \mathrm{bytes}(\hat{y})=\mathrm{bytes}(y)$$2, and $$\mathrm{Pass}\Leftrightarrow \mathrm{bytes}(\hat{y})=\mathrm{bytes}(y)$$3 with $$\mathrm{Pass}\Leftrightarrow \mathrm{bytes}(\hat{y})=\mathrm{bytes}(y)$$4 when bitexact mode or OpenLibm is used for the relevant transcendental evaluations in the rejection tests (Ishii et al., 23 Jun 2026, Drobnjakovic et al., 15 Jan 2025, Jónasson, 6 May 2026).

A recurring misconception is that bit-exactness is impossible whenever floating-point arithmetic is involved. The AI inference work argues instead that auditors need not accept approximation: modern inference stacks produce deterministic outputs that are bitwise re-computable once the right environmental metadata is captured and, critically, provided no floating-point atomic operations are used in reductions. This suggests that the central difficulty is usually not floating-point arithmetic per se, but incomplete specification of the execution environment (Cankaya, 29 May 2026).

2. Motivations, threat models, and failure modes

The immediate motivation for bit-exact conformance packs is that approximate equality leaves unverifiable degrees of freedom. In AI governance, the threat model is a covert adversary that complies with monitoring only if detection likelihood is high and exploits any unverifiable degrees of freedom. The listed attack vectors are steganography in outputs, unreported inference software modification, and covert computation via hidden batch elements. The underlying numerical mechanism is non-associativity: floating-point addition is non-associative, and reductions executed in parallel depend on the reduction tree, so “close enough” checks admit plausible deniability and covert bandwidth (Cankaya, 29 May 2026).

In numeric-format standardization, the problem is silent divergence across accelerators and libraries. The conformance-pack paper states that modern ML hardware and libraries implement FP8, BF16, mixed FP4 elements, and microscaling block formats with permitted differences in overflow/NaN handling, block-scale quantization, and rounding details. Without a vendor-neutral, bit-exact reference, the same model run on two accelerators can produce numerically different results, with the discrepancy remaining hidden until it affects accuracy or stability. The packs therefore function as a shared ruler at the representation layer (Vasilev, 8 Jun 2026).

In protocol engineering, semantic mismatch between multiple clients that implement one blockchain protocol can permanently split the blockchain and introduce new attack vectors. The formal-model-guided framework therefore uses a deterministic blockchain simulator and two complementary workflows because only one workflow cannot detect both over-acceptance and under-acceptance. In tool qualification for cyber-physical systems, the motivation is analogous: model checkers may analyze Simulink blocks differently from the simulator because there is no semantics of Simulink that covers every element of models and the details of its numerical behavior (Drobnjakovic et al., 15 Jan 2025, Ishii et al., 23 Jun 2026).

A contrast case is independent software builds. “Levels of Binary Equivalence for the Comparison of Binaries from Alternative Builds” argues that bitwise equality has significant shortcomings in practice. In a dataset of Java artifacts built from equivalent sources by multiple providers, 3,750 out of 14,156 provider pairs (26.49%) had at least one non-identical class file, even though many were practically equivalent. This does not reject bit-exact packs; it marks a boundary condition. Where benign build variability is endemic, weaker notions such as metadata-normalized or structural equivalence may be required (Dietrich et al., 2024).

3. Pack structure and recorded information

Across domains, bit-exact conformance packs are notable for their explicitness. They do not merely store outputs; they record whatever fixes serialization, arithmetic, scheduling, or encode/decode policy.

Domain	Pack contents	Primary pass condition
AI inference	signed, append-only JSON (or CBOR); hardware, software, model, execution_policy, dispatch_catalogs, batches, attestations	$$\mathrm{Pass}\Leftrightarrow \mathrm{bytes}(\hat{y})=\mathrm{bytes}(y)$$5 and $$\mathrm{Pass}\Leftrightarrow \mathrm{bytes}(\hat{y})=\mathrm{bytes}(y)$$6
Numeric formats	self-contained JSON; format parameters, saturation policy, max-finite, SHA-256 fingerprint, cross-walk notes, per-row encode/decode vectors	target-format bit pattern and decoded value match the row
Randompack	engine identifier, seed, spawn key, stream selector, serialized state blob, requested outputs	exact byte equality of raw words and double encodings
Blockchain	metadata, initial state, canonical inputs, expected outputs, commitments	$$\mathrm{Pass}\Leftrightarrow \mathrm{bytes}(\hat{y})=\mathrm{bytes}(y)$$7 and $$\mathrm{Pass}\Leftrightarrow \mathrm{bytes}(\hat{y})=\mathrm{bytes}(y)$$8
Simulink	feature model, parameterized test models, input signals, oracle bit patterns, verdicts, automation scripts	$$\mathrm{Pass}\Leftrightarrow \mathrm{bytes}(\hat{y})=\mathrm{bytes}(y)$$9

For AI inference, the minimum metadata is unusually extensive because it must capture everything that fixes reduction order, SFU behavior, epilogue casting, and covert batch manipulation. The required fields include hardware SKU, exact model weights in deployed quantization format, parallelism topology, software versions and attention backend, and batch size per forward pass. The proposed pack schema includes identity, hardware, software, model, execution_policy, dispatch_catalogs, batches, attestations, and signatures, and it is explicitly signed and append-only (Cankaya, 29 May 2026).

For numeric formats, the structure is row-centric. Each row stores name, input_f64, input_f64_hex, target-format bits in hex and integer form, decoded_f64, decoded_f64_hex, abs_error, and category. Pack headers store format parameters, saturation_policy, max_finite, sha256, ml_dtypes_anchor, p3109_crosswalk, anchor_identity_ref, and optional block metadata such as block_size or scale_bias. Every pack also carries an anchor vector that encodes 3.0 as a cross-pack sanity check (Vasilev, 8 Jun 2026).

Randompack conformance packs center on RNG state and stream assignment. The paper specifies engine identifiers, seed, spawn key, explicit stream selectors, opaque serialized-state blob bytes, raw 64-bit words as hex, $$\mathrm{bitpattern}(x_{\mathrm{Tool}})=\mathrm{bitpattern}(x_{\mathrm{Simulink}})$$0 doubles, and distribution samples for standard parameter sets. State serialization is portable across platforms, and stream setters such as pcg64_set_inc, squares_set_key, philox_set_key, sfc64_set_abc, cwg128_set_weyl, and chacha20_set_nonce operate only on integer state words and are bit-exact across architectures (Jónasson, 6 May 2026).

Blockchain and Simulink packs are more executional than tabular. Blockchain vectors encode protocol version, serialization version, initial state commitment, participant keys and stake weights, RNG seed, deterministic scheduler, canonical byte inputs, per-step canonical outputs, and final and intermediate commitments. Simulink packs bundle feature models, test models, concrete or abstract input signals, an oracle from Simulink or a peer checker, and scripts that compute pass rates (Drobnjakovic et al., 15 Jan 2025, Ishii et al., 23 Jun 2026).

4. Verification workflows and oracle construction

Bit-exact conformance packs are useful only if the verification workflow is itself exact. In AI inference, the auditor workflow begins with secure input capture and pack validation, including signature verification and an explicit “no floating-point atomics” attestation. Environment reconstruction then proceeds either by loading the emulator with the pack’s hardware profile and dispatch catalogs or by provisioning identical hardware and software versions. Emulated recomputation must reproduce kernel order, streams, split-K and tile sizes, K-walk mainloops, fused FFMA, BF16 epilogue boundaries, and RoPE/libdevice math before byte-level comparison is applied to outputs and recorded intermediate hashes (Cankaya, 29 May 2026).

In blockchain protocols, the central methodological feature is bidirectionality. Workflow II treats the model as trace generator and the implementation as checker; Workflow I treats the implementation as trace generator and the model as checker. The paper states that both workflows are jointly necessary to detect both over-acceptance and under-acceptance. A pack-worthy trace is therefore one that has already passed both deterministic execution and model checking, after which the canonical inputs $$\mathrm{bitpattern}(x_{\mathrm{Tool}})=\mathrm{bitpattern}(x_{\mathrm{Simulink}})$$1, outputs $$\mathrm{bitpattern}(x_{\mathrm{Tool}})=\mathrm{bitpattern}(x_{\mathrm{Simulink}})$$2 or $$\mathrm{bitpattern}(x_{\mathrm{Tool}})=\mathrm{bitpattern}(x_{\mathrm{Simulink}})$$3, and commitments $$\mathrm{bitpattern}(x_{\mathrm{Tool}})=\mathrm{bitpattern}(x_{\mathrm{Simulink}})$$4 can be distributed as golden vectors (Drobnjakovic et al., 15 Jan 2025).

Randompack uses a more library-centric oracle strategy. Validators regenerate outputs through the C API or language bindings after setting seeds, spawn keys, and stream selectors, and compare raw words and double bytes for exact equality. For normal generation specifically, the guidance is that validators should call the bound library function in each language interface for bit-exact comparisons rather than reimplementing the ziggurat algorithm. This is an example of a conformance philosophy in which the pack fixes state and expected bytes, but the reference implementation remains the preferred arithmetic oracle (Jónasson, 6 May 2026).

Simulink testing distinguishes conformance to Simulink from conformance between model checkers. For ConfSL, Simulink produces the expected output signal and verdict subsystems assert bit-exact equality. For ConfMCs, the harness is abstracted by mutating point equalities into equality, $$\mathrm{bitpattern}(x_{\mathrm{Tool}})=\mathrm{bitpattern}(x_{\mathrm{Simulink}})$$5, or $$\mathrm{bitpattern}(x_{\mathrm{Tool}})=\mathrm{bitpattern}(x_{\mathrm{Simulink}})$$6, SmtMC evaluates the bounded property, and the resulting Boolean verdict is checked by SLDV. The pass rate metric is then the proportion of tests whose bit patterns match the reference (Ishii et al., 23 Jun 2026).

5. Empirical evidence across domains

The recent literature provides several strong empirical results. In AI inference, repeated runs yielded bitwise identical outputs $$\mathrm{bitpattern}(x_{\mathrm{Tool}})=\mathrm{bitpattern}(x_{\mathrm{Simulink}})$$7 when hardware SKU, software stack, quantization format and kernel variant, and tensor-parallel rank were fixed; different physical cards of the same SKU also produced identical outputs. The only observed source of non-determinism was floating-point atomic kernels, notably vLLM’s exllama GPTQ path. The emulator reproduced FFN and FlashAttention-2 numerics with 0 BF16 diffs and 0 FP32 accumulator diffs across A100, L40/L40S, and H100, and for attention this held across approximately 71M elements at 4,000 tokens. Cross-hardware behavior showed zeros within SKU and within architecture, and non-zero L2 distances across architecture boundaries, which the paper interprets as rounding signatures (Cankaya, 29 May 2026).

In Simulink model checking, the reported pass rates were not uniform across tools. SmtMC achieved 100% pass across all blocks and tests for conformance to Simulink. SLDV achieved average pass rates of 94% (R2020b) and 96% (R2025b) for conformance to Simulink, and 80% (R2020b) and 90% (R2025b) for conformance between SmtMC and SLDV. The dominant failure mode was non-finite FP handling, with additional timeouts and internal errors concentrated in Product2, Delay, Unit Delay, and Discrete Integrator (Ishii et al., 23 Jun 2026).

The numeric-format packs provide a different kind of empirical evidence: exact vector agreement against a public oracle. BF16 achieved 21/21 exact matches with ml_dtypes.bfloat16 (0.5.4), FP8 E5M2 achieved 17/17 exact matches with ml_dtypes.float8_e5m2, and FP8 E4M3 achieved 15/16 exact matches with one documented divergence. The divergence is explicit: input 1000.0 maps to 0x7E and 448.0 under the pack’s saturation-to-max-finite convention, versus 0x7F and NaN under ml_dtypes’ overflow-to-NaN convention, with both choices stated to be permitted by OCP MX v1.0 (Vasilev, 8 Jun 2026).

Randompack contributes cross-platform reproducibility evidence rather than conformance percentages. Benchmarks show speedup factors ranging from about 1 to 15 depending on engine, distribution, interface, and platform, while the library gives compatible results across programming languages, computers, CPU architectures, and compilers when seeds are the same. For distributions beyond the bit-exact default set, the paper states that outputs are tolerance-reproducible by default and bit-exact if the bitexact setting is enabled (Jónasson, 6 May 2026).

6. Scope, limitations, and relation to weaker equivalence

Bit-exact conformance packs are strongest when the semantics to be fixed are controllable and inspectable. The AI inference paper is explicit that its results cover NVIDIA GPUs only, across Lovelace/Ada, Ampere, and Hopper; inference only rather than training; and require either cataloging or avoidance for Hopper nvjet short-sequence dispatch. Training and backward passes introduce atomics in some paths, and MoE primitives are covered numerically but not yet by a shipped MoE-specific end-to-end emulator (Cankaya, 29 May 2026).

Other domains exhibit analogous boundaries. The numeric-format packs verify representation-layer encode/decode only; operation-layer semantics such as Add, Mul, and FMA are Track-2 work. Cross-validation uses ml_dtypes 0.5.4 where types exist, so GF16 and MXFP4 element rely on round-trip checks and OCP MX tables. Randompack’s bit-exact guarantees apply immediately to raw bitstreams, discrete distributions, $$\mathrm{bitpattern}(x_{\mathrm{Tool}})=\mathrm{bitpattern}(x_{\mathrm{Simulink}})$$8, $$\mathrm{bitpattern}(x_{\mathrm{Tool}})=\mathrm{bitpattern}(x_{\mathrm{Simulink}})$$9, $y$0, and $y$1 with $y$2, while multivariate normal is explicitly not guaranteed bit-exact because it is matrix-factorization dependent. In blockchain conformance, the model is foundational: packs are only as good as the specification and the refinement mapping (Vasilev, 8 Jun 2026, Jónasson, 6 May 2026, Drobnjakovic et al., 15 Jan 2025).

A second limitation is normative rather than technical: exact equality is not always the right acceptance criterion. The binary-equivalence study introduces four increasingly permissive relations, from bit-exact equivalence $y$3 to metadata-normalized $y$4, structural bytecode equivalence $y$5, and semantic equivalence $y$6. Its empirical finding that 26.49% of independent Java build pairs differ at the class-file level despite equivalent sources shows that bitwise equality can be too strict for supply-chain comparisons (Dietrich et al., 2024). This suggests that “bit-exact conformance pack” is best understood as the highest-assurance point in a broader design space of conformance artifacts, not as a universally appropriate requirement.

The standardization trajectory is nonetheless clear where exactness is feasible. In AI inference, proposed pathways include field definitions, compliance tests, interoperability with MLOps, and remote attestation; in numeric formats, the packs are aligned to an IEEE P3109 v3.2.0 cross-walk and carry SHA-256 fingerprints; in Randompack, exact state serialization and platform-agnostic stream setters make cross-language fixtures practical. Across these settings, the unifying principle is that non-associativity, rounding, serialization, and state are not treated as ambient noise but as auditable structure (Cankaya, 29 May 2026, Vasilev, 8 Jun 2026, Jónasson, 6 May 2026).