Scaling the shared topology predictor

Determine whether the shared topology predictor in DAGFormer retains its performance lead over per-layer routing architectures at larger model scales, particularly through routing plans that no individual layer can observe.

Background

The paper finds that a shared predictor reading the input tokens outperforms independent per-layer predictors and several architectures whose routers read each layer’s hidden state. The experiments, however, cover models only up to 1B parameters, and the authors note that the comparison between global planning and local routing is not fully settled by the reported results.

The unresolved question concerns whether the architectural advantage of globally planning the cross-layer topology persists as model scale increases, especially if the shared predictor can coordinate routing decisions that are unavailable to any single backbone layer.

References

Whether the predictor keeps its lead at larger scale, for instance by planning routing that no single layer can see, is untested.

— DAGFormer: Global Topology Prediction for Transformer Cross-Layer Connections  (2610.19625 - Yang et al., 14 Oct 2026) in Section 6, paragraph “Shared predictor and per-layer routers”