Validation on new layer types and accelerators

Validate Primus Projection for distributed-training workloads containing layer types beyond those evaluated in the paper, including subquadratic layers such as Gated DeltaNet and sparse-attention layers, and for additional accelerators.

Background

The evaluated workloads focus on dense and mixture-of-experts transformer architectures whose compute can be represented through profiled layer parameters and GEMM or attention models. The paper notes that extending the framework to other layer types is technically possible through parameterizable layer models, such as profiled FLOP estimates for subquadratic layers and density-weighted FLOP counts for sparse attention.

However, the accuracy of these extensions has not been established experimentally. The authors explicitly identify validation of such new layer types and accelerators as future work, leaving the reliability of Primus Projection for these workloads unresolved.