Identify the operative accessibility channel behind model-side scaling differences

Determine whether the model-side rate dependence observed in resource-matched architecture comparisons is caused by architectural support, completed-prefix acquisition, or dispersed off-prefix gain.

Background

The paper develops Coupled Scaling to explain why systems trained on the same data can exhibit different scaling exponents. It distinguishes architectural support, the order in which task-relevant directions are acquired, and gains from acquiring directions outside a completed high-value prefix.

The cited resource-matched architecture comparison provides evidence of model-side rate dependence but does not isolate which of these accessibility mechanisms is responsible. The paper therefore identifies an unresolved empirical question requiring independently measured, task-conditioned geometry trajectories.

References

In Coupled Scaling terms, it supplies unusually controlled evidence of model-side rate dependence while leaving open whether the operative accessibility channel is support, completed-prefix acquisition, or dispersed off-prefix gain.

— Coupled Scaling: A Representational Accessibility Framework for Neural Scaling Laws  (2609.03533 - Wang, 3 Sep 2026) in Section 2.2, “Same-Data Evidence for Model-Side Dependence”

However, how to fairly compare this redistribution of computation with the uniform computation of standard Transformers remains an open question.

— Orthros: Phase-Aware Heterogeneous Attention for Efficient Transformers  (2610.13735 - Chou et al., 9 Oct 2026) in Section 5, Discussion, “Prefill-Dominated Workloads and Compute Allocation,” p. 12