SoftQ: Scaling Laws and Software-Defined Systems

Updated 1 July 2026

SoftQ is a framework that introduces a soft-coupled, five-parameter scaling law for ML pretraining, outperforming classical additive models by capturing dynamic data bottlenecks.
It derives a smooth interpolation between parameter-limited and data-limited regimes, enabling resource allocation insights and a data-equivalent view of regularization strategies.
Beyond ML, SoftQ extends to software-defined quantum communications and HPC QoS, promoting adaptive, real-time control over resource management and system performance.

SoftQ refers to a range of paradigms and scaling laws in both machine learning and computer systems, typically distinguished by a software-defined approach to resource management, communication, or modeling. Across diverse scientific domains, "SoftQ" characterizes frameworks or analytic formulations that (1) soft-couple key system parameters through non-hard constraints or bottlenecks and (2) enable dynamic, programmable, or empirical adaptation in settings where resource or data constraints are fundamental.

1. SoftQ Scaling Law for Data-Constrained LLM Pretraining

In the context of LLM pretraining under data-constrained, compute-rich regimes, SoftQ denotes a non-additive, five-parameter scaling law for validation loss that smoothly couples model size $N$ (in parameters) and unique data budget $U$ (in tokens) (Xu et al., 5 Jun 2026). Classical scaling laws—such as the "Chinchilla" law—presume additive separability of model and data effects, i.e.,

$L_\mathrm{Ch}(N, U) = E + A N^{-\alpha} + B U^{-\beta}$

where $E$ is the irreducible loss, $A$ , $B$ , $\alpha$ , $\beta > 0$ , and the terms for $N$ and $U$ are independent. However, in repeated-epoch, data-limited settings, empirical evidence shows these laws underestimate the performance degradation of large models under strict data bottlenecks, motivating the SoftQ alternative.

The SoftQ scaling law takes the form:

$U$ 0

Where:

$U$ 1, $U$ 2: model size and unique data, measured in billions,
$U$ 3: minimal achievable loss,
$U$ 4, $U$ 5: scaling strengths for $U$ 6- and $U$ 7-limited regimes,
$U$ 8: high-compute asymptotic exponent,
$U$ 9: transition softening parameter.

This formulation embodies a "soft bottleneck," interpolating smoothly between parameter-limited and data-limited asymptotes. The law recovers known verticals:

As $L_\mathrm{Ch}(N, U) = E + A N^{-\alpha} + B U^{-\beta}$ 0: $L_\mathrm{Ch}(N, U) = E + A N^{-\alpha} + B U^{-\beta}$ 1
As $L_\mathrm{Ch}(N, U) = E + A N^{-\alpha} + B U^{-\beta}$ 2: $L_\mathrm{Ch}(N, U) = E + A N^{-\alpha} + B U^{-\beta}$ 3

These limits align precisely with the "skill-learning" or "Quanta" view, where effective loss scales with the number of acquired "skills" as $L_\mathrm{Ch}(N, U) = E + A N^{-\alpha} + B U^{-\beta}$ 4 (Xu et al., 5 Jun 2026).

2. Derivation, Fitting Protocol, and Empirical Assessment

SoftQ emerges from analysis of empirical loss curves in the multi-epoch, finite-data regime. Rather than a hard $L_\mathrm{Ch}(N, U) = E + A N^{-\alpha} + B U^{-\beta}$ 5 bottleneck, SoftQ replaces this with a soft sum in "inverse skill capacity" space—motivated by both theoretical considerations and observable model behavior.

Empirical fitting in (Xu et al., 5 Jun 2026) involved a grid of 20 training runs (model sizes 0.072–1.44B; unique tokens 0.1–0.4B) with strong regularization. Parameters were chosen to minimize Huber loss on log-residuals between the predicted and observed validation losses using nonlinear least-squares.

Fit quality is benchmarked by RMSE / MAE:

Scaling Law	RMSE	MAE
Chinchilla	0.02653	0.01802
Quanta	0.01252	0.00889
Muennighoff	0.02335	0.01713
SoftQ	0.00801	0.00520

SoftQ outperformed all alternatives both on-fit and in extrapolation (e.g., held-out $L_\mathrm{Ch}(N, U) = E + A N^{-\alpha} + B U^{-\beta}$ 6B), capturing the "fan-out" effect—i.e., the increased penalty that large $L_\mathrm{Ch}(N, U) = E + A N^{-\alpha} + B U^{-\beta}$ 7 incurs under limited $L_\mathrm{Ch}(N, U) = E + A N^{-\alpha} + B U^{-\beta}$ 8 (Xu et al., 5 Jun 2026).

3. Interpretation, Marginal Exponents, and Data Efficiency

Parameter exponents, as fitted in (Xu et al., 5 Jun 2026) (for $L_\mathrm{Ch}(N, U) = E + A N^{-\alpha} + B U^{-\beta}$ 9, $E$ 0 in billions), are:

Fitted form: $E$ 1
$E$ 2
$E$ 3
$E$ 4

Key practical insight: in this regime, doubling $E$ 5 yields diminishing marginal returns ( $E$ 6), as does doubling $E$ 7 ( $E$ 8). Plotting iso-loss contours of $E$ 9 directly informs resource allocation between model scaling and data acquisition.

SoftQ also enables a data-equivalent interpretation of regularization strategies: for instance, masked-input regularization (MIR) was found to confer a model improvement commensurate to $A$ 0 more unique data, as extracted from iso-loss curves (Xu et al., 5 Jun 2026).

4. Software-Defined Quantum Communication and Networking

Distinct from ML scaling, SoftQ designates a set of software-defined paradigms for quantum communications systems and networks (Chistyakov et al., 2017, Humble et al., 2014). Here, SoftQ encapsulates architectures in which:

Classical SDN controllers interface with quantum hardware (sources, modulators, detectors) via programmable protocols (e.g., OpenFlow, REST APIs) (Chistyakov et al., 2017).
Key link parameters (channel loss $A$ 1, mean photon number $A$ 2, QBER) and cryptographic functions are monitored and controlled in real time, with all higher protocol logic (e.g., basis selection, error correction) implemented as software (Humble et al., 2014).
Dynamic adaptation is possible: routing, choice of encryption (quantum vs classical), and quality-of-service enforcement (bandwidth allocation, security level) can be adapted on sub-second timescales using programmable flow-table rules and token-based policies.

Testbed deployments achieve transparent quantum-classical switching with reprogramming delays $A$ 3 ms, sifted key rates of $A$ 4 Mbit/s at $A$ 5 km fiber with QBER $A$ 6, and robust fallback-to-classical operation at QBER thresholds (Chistyakov et al., 2017).

5. Software-Defined QoS Frameworks in HPC

In high-performance computing, SoftQ refers to software-defined QoS provisioning (SDQPro) in large-scale I/O scheduling (Tavakoli et al., 2018). SDQPro soft-couples per-application and per-server resources by:

Centralizing global bandwidth benchmarks into software-controlled token bucket abstractions,
Allowing token borrowing across idle and hot-spot storage servers to mitigate unbalanced I/O bursts,
Employing extended M-LWDF scheduling to prioritize applications both by fairness and urgency, with explicit per-server bandwidth limits.

Simulation demonstrates substantial performance gains: with token-borrowing enabled, applications realize $A$ 7 of their desired bandwidth (vs $A$ 8 baseline), and small I/O microbenchmarks (4-8 KB) see $A$ 9– $B$ 0 improvement over legacy scheduling (Tavakoli et al., 2018).

6. Significance and Theoretical Unification

Across fields, SoftQ formulations serve to:

Replace hard constraints (e.g., $B$ 1-based bottlenecks, fixed hardware functions) with smooth, soft, or programmable couplings—enabling more precise, empirically accurate models and flexible system control.
Bridge theoretical analysis (e.g., skill-learning scaling, queueing theory, information theory) with practical deployment in resource-constrained settings.
Facilitate rapid prototyping and easy adaptation to dynamically changing environments—for instance, by decoupling quantum communication protocol logic from hardware drivers, or by enabling allocation policies to respond to workload-induced temporal variation.

A plausible implication is that SoftQ-like methodologies will become increasingly central in both analytic modeling (where non-separable, softly interpolated laws are needed) and systems (where flexibility and adaptation are critical under constraints). The approach also supports principled resource allocation, fair scheduling, and robust system operation as empirical data diverge from classical, additive model assumptions.