Measure the effective rank of the trained readout Jacobian

Measure the effective rank of the trained readout Jacobian on test inputs to determine whether the nonlinear readouts develop an effectively rank-one active subspace in the matrix latent Z.

Background

The paper finds that bilinear, bilinear-plus-GELU, SVD-augmented, and quadratic readouts all produce nearly flat rank-k ablation curves, despite several readouts having non-constant Jacobians. The authors propose that the trained readouts may nevertheless use an effectively rank-one active subspace in Z, which could explain why rank truncation has little effect.

Testing this hypothesis requires evaluating the effective rank of the Jacobian J(Z) at the relevant checkpoints and test inputs. The paper explicitly identifies this measurement as not yet completed.

References

We have not yet measured $\mathrm{erank}(J(Z))$ on these checkpoints to test this directly.

— The Gradient Does Not See Rank: Rank-Indifference in Matrix-CODI on ProsQA  (2609.03090 - Larson, 2 Sep 2026) in Section 5, subsection “The mechanism is not just readout linearity”

An $n!=!10$ replication is pending.

— The Gradient Does Not See Rank: Rank-Indifference in Matrix-CODI on ProsQA  (2609.03090 - Larson, 2 Sep 2026) in Section 7, paragraph “Seed-dependent Z rank”