OutControl: Quantized LLM & Quantum Protocols

• OutControl is a dual-purpose protocol that enables high-fidelity 4-bit LLM training and precise quantum tracking control by managing outlier channels and enforcing strict operational constraints.
• In LLM training, it employs a backward-pass Random Hadamard Transformation and static precision retention on outliers to mitigate variance and improve model performance.
• In quantum control, it ensures robust manipulation of observables by upholding unitarity and mobility constraints, thereby preventing nonunique solution branches.

OutControl denotes two distinct but conceptually related protocol classes in contemporary computational research: (1) an algorithm for high-fidelity fully-quantized training in LLMs via precision-retaining subroutines and stochastic-variance control (Chen et al., 31 Oct 2025), and (2) a tracking-control protocol for nonlinear, field-free manipulation of observables in strongly correlated many-body quantum systems (McCaul et al., 2019). Both instances, although arising from different disciplines, share an emphasis on outlier management and exact constraint satisfaction in high-dimensional spaces.

1. OutControl in Fully Quantized LLM Training

OutControl is a critical component of the TetraJet-v2 framework for efficient 4-bit fully-quantized training (FQT) of LLMs using the NVFP4 numerical format (Chen et al., 31 Oct 2025). It is designed to address pathological sensitivity to outlier activation and gradient channels under the limited dynamic range of 4-bit quantization. Such outlier channels—whose magnitudes dwarf the bulk—collapse the precision available for ordinary components and degrade model quality. OutControl mitigates this through two coupled mechanisms:

• Backward-Pass Random Hadamard Transformation (RHT): Applies a randomized, block-wise Hadamard transform in the backward pass, dispersing large-magnitude gradients over a larger subspace and bounding the variance incurred by stochastic rounding.
• Static Precision Retention for Outliers: Statistically profiles activations at the start of training to identify a fixed subset C (top‐norm $p\%$ channels) for which both forward and backward passes are computed in higher precision (FP8 or BF16), bypassing NVFP4 quantization only for these channels.

This dual approach preserves the majority of computation in low-precision format while selectively safeguarding accuracy for the critical outlier subset.

2. Mathematical Formulation

Given input activations $X\in\mathbb{R}^{N\times D}$ and weights $W\in\mathbb{R}^{C\times D}$, the OutControl protocol partitions the channel dimension into outliers $C$ and complement $\bar C$, with $|C|=pD$ (typically $p=10\%$ for FP8 retention). The quantization operators $Q_D(\cdot)$ (deterministic) and $Q_S(\cdot)$ (stochastic) are defined for NVFP4, and RHT is constructed from a block-Hadamard matrix $H_n$ and sign-flip diagonal $X\in\mathbb{R}^{N\times D}$0 as $X\in\mathbb{R}^{N\times D}$1.

Forward Pass:

$X\in\mathbb{R}^{N\times D}$2

Here, $X\in\mathbb{R}^{N\times D}$3 is quantized (FP4$X\in\mathbb{R}^{N\times D}$4FP4), while $X\in\mathbb{R}^{N\times D}$5 is retained at higher precision.

Backward Pass (gradients $X\in\mathbb{R}^{N\times D}$6, $X\in\mathbb{R}^{N\times D}$7):

$X\in\mathbb{R}^{N\times D}$8

Channel selection $X\in\mathbb{R}^{N\times D}$9 is fixed once via offline profiling, removing the need for dynamic updates.

3. Pseudocode and Operational Workflow

Implementation is succinctly captured by the following, which assumes a precomputed outlier set $W\in\mathbb{R}^{C\times D}$0:

$C$6 This structure ensures precision retention and stochastic-variance mitigation at minimal computational surplus.

4. Hyperparameterization and Tuning

• Outlier ratio $W\in\mathbb{R}^{C\times D}$1: $W\in\mathbb{R}^{C\times D}$2 (BF16), $W\in\mathbb{R}^{C\times D}$3 (FP8). TetraJet-v2 uses $W\in\mathbb{R}^{C\times D}$4 with FP8 throughout.
• RHT block size: 32 channels.
• Retained-precision format: FP8 on selected $W\in\mathbb{R}^{C\times D}$5 channels.
• Outlier index computation: Single offline profiling at training start.
• Compute cost overhead: Approximately $W\in\mathbb{R}^{C\times D}$6 relative to full NVFP4 pipeline, attributed to additional high-precision matrix multiplications on outlier channels.

5. Ablations and Empirical Performance

Ablative studies on a 150M-parameter LLM trained for 100B tokens reveal the quantitative impact of OutControl [(Chen et al., 31 Oct 2025), Table 8]:

Variant	Validation PPL (avg, last 50 steps)
TetraJet-v2‐base (no OutControl)	31.49
OutControl (forward only)	31.41
OutControl (forward + backward)	31.28
Top-norm $W\in\mathbb{R}^{C\times D}$7 channels (forward+back)	31.28
Random $W\in\mathbb{R}^{C\times D}$8 channels	31.47
No selection (all FP4)	31.49

Figure 1 (Chen et al., 31 Oct 2025) demonstrates that OutControl consistently narrows the gap to BF16 reference training on OLMo2-370M across 200B tokens, confirming ~0.2–0.3 PPL improvement with minimal precision upscaling.

6. Practical Considerations and Limitations

• Offline static selection: Channel indices for outlier mitigation are computed once, leveraging the constancy of outlier patterns during training and mitigating runtime overhead.
• Limited overhead: Only $W\in\mathbb{R}^{C\times D}$9 columns require high-precision computation, with the rest fully in NVFP4.
• Algorithmic demonstration: All results are algorithmic; hardware-level speedups are currently unattainable due to lack of ultra-low-precision device support.
• Scalability: Experiments are currently capped at 370M parameters and 200B tokens; hardware efficacy and performance at greater scales are untested.
• Future directions: Adaptive re-profiling of $C$0, layer-specific $C$1, and decay of $C$2 during training remain open for further study. OutControl is complementary to OsciReset (oscillation suppression) and unbiased quantization schemes.

7. OutControl in Quantum Tracking-Control

In the context of correlated many-body quantum systems, OutControl refers to a tracking protocol designed for non-perturbative, field-free manipulation of arbitrary observables, notably demonstrated for total current in the 1D Fermi–Hubbard model (McCaul et al., 2019). It replaces explicit field-driven Hamiltonian evolution with a self-consistent, state-dependent nonlinear Schrödinger equation. The protocol ensures perfect tracking of the target observable provided two constraints are respected:

1. Unitarity Constraint: $C$3 with $C$4. Prevents loss of Hermiticity and nonuniqueness in the solution.
2. Mobility Constraint: $C$5. Guarantees kinetic terms do not vanish, preserving unique solution flow.

Violation of these constraints leads to nonunique solution branches and control pathologies. The protocol is generalizable to arbitrary Hermitian observables, with explicit construction of control fields via inversion of their differential relations. Validity is a posteriori checked via exact Ehrenfest theorems. Preliminary studies indicate implementation feasibility in engineered quantum and photonic systems, subject to field amplitude and bandwidth constraints (McCaul et al., 2019).

OutControl, across both domains, exemplifies a principled approach to outlier and constraint management where precision, variance control, and exact physical constraints play a decisive role in achieving fidelity and robustness within extreme parameter regimes.