Implementation of the attention reduction-order analysis

Implement the theoretical reduction-order analysis for attention kernels as a concrete bitwise-equivalence and descriptor mechanism.

Background

The paper extends its reduction-order framework theoretically to attention, whose recurrence introduces rescaling and multiple nested reductions in addition to matrix multiplications. It identifies how block boundaries, causal masking, row-sum trees, probability narrowing, and final normalization affect numerical results.

The implementation is expressly unresolved: the paper states that it provides a theoretical description while leaving implementation open. This is distinct from the broader discussion item concerning the incomplete treatment of attention and GEMM fusions.

References

Readers may refer to Appendix~\ref{app:attention} for the reduction order of a complex kernel such as flash attention, where this work offers a theoretical description and leaves the implementation open.

— Taming Bitwise Behavior in GPU Kernels with Tensor Core: Black-Box Reconstruction, Compiler Enforcement, and Static Verification  (2609.11356 - Yang et al., 10 Sep 2026) in Section 4, subsection Attention

LUCE we could not reduce at all: its matrix multiplies use the Hopper warpgroup instructions, whose floor is 128 threads, and our reducer handles nothing above a single warp. Raising that floor means extending the reducer, and extending Kuiper with those semantics as well.

— RESOLVE: Language-Agnostic Validation of GPU Kernels Through Testing, Reduction, and Proof  (2610.05683 - Gargary et al., 5 Oct 2026) in Section 4.3, "Fixing the bugs"