Orbit compression of singular branching states

Determine whether the branching states in the first-pivot recursion for Cholesky fiber cardinalities over finite fields can be compressed by finitely many flag invariants, thereby improving the worst-case exponential algorithm to a parameterized polynomial algorithm.

Background

The paper derives an exact first-pivot recursion for the number of upper triangular matrices U over a finite field satisfying U^T U=A for a prescribed symmetric target A. When the leading pivot and corresponding leading row vanish, the recursion branches over rank-one perturbations indexed by vectors x, and memoization can reduce repeated subproblems but does not establish polynomial worst-case complexity.

The authors explicitly identify compression of these singular branching states as unresolved. The later conclusion formulates the same issue more specifically in terms of finitely many flag invariants and parameterized polynomial complexity.