HC Newton (HCN) in mHC-Style Transformers

• HC Newton (HCN) is a structured Newton method for mHC-style residual Transformers that replaces costly Jacobian-vector products with learned surrogate mixing matrices.
• It reformulates the hidden-state trace as a nonlinear residual system and employs small-matrix corrections to enable efficient parallel computation.
• SNLP-aware regularization improves both sequential perplexity and inference speed, as demonstrated by up to 20-30% speedups on 0.5B Nanochat models.

HC Newton (HCN) is the Structured Newton Layer Parallelism (SNLP) instantiation for mHC-style residual Transformers, where each block maintains a small learned residual-stream mixture that serves as a surrogate Jacobian in place of the full hidden-state Jacobian. In this formulation, the layerwise hidden-state trace is recast as a nonlinear residual equation, and inference is performed by Newton-style corrections that exploit the model’s residual mixing matrix rather than exact Jacobian-vector products. Within the broader SNLP framework, HCN is the mHC counterpart of Identity Newton (IDN), which is used for residual Transformers; the paper studies HCN as both an inference procedure and a training target via SNLP-aware regularization (Han et al., 18 May 2026).

1. Architectural setting and conceptual role

HCN is defined for mHC-style residual Transformers. In this setting, an $L$-layer decoder has hidden states $h_0,\dots,h_L \in \mathbb{R}^d$, and each block uses a small residual mixture over $M$ streams. The block is written in two stages: $$x'_l = H_{l}^{res,attn}\,h_{l-1} + H_{l}^{post,attn}\cdot \mathrm{Attn}_l(H_{l}^{pre,attn}\,h_{l-1}),$$

$$h_l = H_{l}^{res,mlp}\,x'_l + H_{l}^{post,mlp}\cdot \mathrm{MLP}_l(H_{l}^{pre,mlp}\,x'_l).$$

Each $H^{\cdot}$ is an $M\times M$ stream-mixing matrix, with $M \ll d$ (Han et al., 18 May 2026).

Within SNLP, HCN addresses the latency bottleneck created by strictly sequential Transformer-layer execution. The central idea is not to remove the forward dependency exactly, but to relax it through a structured Newton correction whose surrogate dynamics are induced by the architecture itself. The paper places HCN in a broader program that asks whether the hidden-state trace across layers can be treated as the solution of a nonlinear residual equation and then approximated by parallel Newton-style updates. In that sense, HCN is a solver-based layer-parallel approximation specialized to mHC blocks rather than a generic parallelization heuristic.

A common misunderstanding is to treat HCN as exact Newton inference. The paper explicitly distinguishes the two: exact Newton would require expensive Jacobian-vector products, whereas HCN replaces the exact layer Jacobians with a cheap surrogate built from residual mixing matrices. This makes HCN a structured Newton method rather than a full Newton solve.

2. Residual-system formulation

The mHC forward pass is rewritten as a root-finding problem. Writing $H=(h_1,\dots,h_L)$ and defining

$G_l(H) = h_l - f_l(h_{l-1}), \qquad l=1,\dots,L,$

the sequential forward satisfies $h_0,\dots,h_L \in \mathbb{R}^d$0, where $h_0,\dots,h_L \in \mathbb{R}^d$1 is a block-lower-bidiagonal system (Han et al., 18 May 2026).

This formulation is the mathematical basis for HCN. Rather than viewing the model purely as a composition of layers, the method views the full layer trace as the solution of a coupled nonlinear system. In exact Newton form, the Jacobian of $h_0,\dots,h_L \in \mathbb{R}^d$2 would be

$h_0,\dots,h_L \in \mathbb{R}^d$3

with $h_0,\dots,h_L \in \mathbb{R}^d$4.

The significance of this reformulation is methodological. It turns the serial layer dependency into a structured nonlinear solve, which in turn makes it possible to ask whether a small number of approximate Newton corrections can recover most of the sequential computation. The paper’s broader claim is that this perspective is principled, but that naïve fixed-point iterations are unstable on trained Transformers and exact Newton is too expensive. HCN occupies the middle ground created by that observation.

3. Surrogate Jacobian and HC Newton update

The defining approximation in HCN is the replacement of the exact layer Jacobian $h_0,\dots,h_L \in \mathbb{R}^d$5 by a small surrogate matrix

$h_0,\dots,h_L \in \mathbb{R}^d$6

which is only $h_0,\dots,h_L \in \mathbb{R}^d$7 (Han et al., 18 May 2026). The paper motivates this by decomposing

$h_0,\dots,h_L \in \mathbb{R}^d$8

and then training the model so that these branch terms become small, making $h_0,\dots,h_L \in \mathbb{R}^d$9.

Newton’s method for $M$0 would take the form

$M$1

HCN replaces $M$2 with a surrogate block matrix whose diagonal blocks are $M$3 and whose off-diagonal blocks are $M$4: $M$5

$M$6

Because this surrogate system is block-lower-bidiagonal, applying the inverse reduces to a recurrence: $M$7 and for $M$8,

$M$9

This update is the operational core of HCN. It preserves the lower-bidiagonal structure of the original residual system while replacing the expensive $$x'_l = H_{l}^{res,attn}\,h_{l-1} + H_{l}^{post,attn}\cdot \mathrm{Attn}_l(H_{l}^{pre,attn}\,h_{l-1}),$$0 Jacobians with the model’s learned stream-mixing matrices. A plausible implication is that HCN is best understood as a structured preconditioned Newton step whose preconditioner is supplied by the architecture.

4. Inference procedure and computational profile

The paper describes HCN inference in a prefix–suffix decomposition. A prefix of $$x'_l = H_{l}^{res,attn}\,h_{l-1} + H_{l}^{post,attn}\cdot \mathrm{Attn}_l(H_{l}^{pre,attn}\,h_{l-1}),$$1 layers is run sequentially, and a suffix of $$x'_l = H_{l}^{res,attn}\,h_{l-1} + H_{l}^{post,attn}\cdot \mathrm{Attn}_l(H_{l}^{pre,attn}\,h_{l-1}),$$2 layers is then processed by SNLP. With $$x'_l = H_{l}^{res,attn}\,h_{l-1} + H_{l}^{post,attn}\cdot \mathrm{Attn}_l(H_{l}^{pre,attn}\,h_{l-1}),$$3, the prefix is computed by the ordinary sequential recurrence $$x'_l = H_{l}^{res,attn}\,h_{l-1} + H_{l}^{post,attn}\cdot \mathrm{Attn}_l(H_{l}^{pre,attn}\,h_{l-1}),$$4 for $$x'_l = H_{l}^{res,attn}\,h_{l-1} + H_{l}^{post,attn}\cdot \mathrm{Attn}_l(H_{l}^{pre,attn}\,h_{l-1}),$$5. The suffix is initialized by setting $$x'_l = H_{l}^{res,attn}\,h_{l-1} + H_{l}^{post,attn}\cdot \mathrm{Attn}_l(H_{l}^{pre,attn}\,h_{l-1}),$$6 for $$x'_l = H_{l}^{res,attn}\,h_{l-1} + H_{l}^{post,attn}\cdot \mathrm{Attn}_l(H_{l}^{pre,attn}\,h_{l-1}),$$7 (Han et al., 18 May 2026).

Each HCN iteration then has two stages. First, the nonlinear blocks in the suffix are evaluated in parallel: $$x'_l = H_{l}^{res,attn}\,h_{l-1} + H_{l}^{post,attn}\cdot \mathrm{Attn}_l(H_{l}^{pre,attn}\,h_{l-1}),$$8 Second, a sequential small-matrix correction is applied: $$x'_l = H_{l}^{res,attn}\,h_{l-1} + H_{l}^{post,attn}\cdot \mathrm{Attn}_l(H_{l}^{pre,attn}\,h_{l-1}),$$9 The paper states that, in practice, $$h_l = H_{l}^{res,mlp}\,x'_l + H_{l}^{post,mlp}\cdot \mathrm{MLP}_l(H_{l}^{pre,mlp}\,x'_l).$$0 or $$h_l = H_{l}^{res,mlp}\,x'_l + H_{l}^{post,mlp}\cdot \mathrm{MLP}_l(H_{l}^{pre,mlp}\,x'_l).$$1 is enough on mHC models trained with the SNLP-aware loss.

The computational argument for HCN is tied to this separation between expensive nonlinear block evaluations and cheap stream-mixing corrections. If $$h_l = H_{l}^{res,mlp}\,x'_l + H_{l}^{post,mlp}\cdot \mathrm{MLP}_l(H_{l}^{pre,mlp}\,x'_l).$$2 denotes the cost of one Transformer block forward, sequential execution has cost approximately $$h_l = H_{l}^{res,mlp}\,x'_l + H_{l}^{post,mlp}\cdot \mathrm{MLP}_l(H_{l}^{pre,mlp}\,x'_l).$$3. HCN with $$h_l = H_{l}^{res,mlp}\,x'_l + H_{l}^{post,mlp}\cdot \mathrm{MLP}_l(H_{l}^{pre,mlp}\,x'_l).$$4 iterations evaluates the $$h_l = H_{l}^{res,mlp}\,x'_l + H_{l}^{post,mlp}\cdot \mathrm{MLP}_l(H_{l}^{pre,mlp}\,x'_l).$$5 suffix blocks in parallel, giving cost approximately $$h_l = H_{l}^{res,mlp}\,x'_l + H_{l}^{post,mlp}\cdot \mathrm{MLP}_l(H_{l}^{pre,mlp}\,x'_l).$$6 when the layers are batched into one giant fused kernel, plus a correction loop of cost $$h_l = H_{l}^{res,mlp}\,x'_l + H_{l}^{post,mlp}\cdot \mathrm{MLP}_l(H_{l}^{pre,mlp}\,x'_l).$$7. Since $$h_l = H_{l}^{res,mlp}\,x'_l + H_{l}^{post,mlp}\cdot \mathrm{MLP}_l(H_{l}^{pre,mlp}\,x'_l).$$8, the paper characterizes this correction cost as negligible versus $$h_l = H_{l}^{res,mlp}\,x'_l + H_{l}^{post,mlp}\cdot \mathrm{MLP}_l(H_{l}^{pre,mlp}\,x'_l).$$9. The memory footprint requires two copies of the suffix states, giving roughly $H^{\cdot}$0 floats versus $H^{\cdot}$1 for sequential execution, described as an $H^{\cdot}$2 overhead (Han et al., 18 May 2026).

5. SNLP-aware regularization

HCN is not presented only as an inference-time approximation. The paper also introduces SNLP-aware regularization to make the surrogate matrices $H^{\cdot}$3 better approximations to the true layer Jacobians. The training objective augments the standard language-model cross-entropy with a term that penalizes divergence between SNLP-produced hidden states and sequential hidden states: $H^{\cdot}$4 In training, the paper uses $H^{\cdot}$5, while $H^{\cdot}$6 selects a possibly strided subset of layers including the final layer (Han et al., 18 May 2026).

The stated purpose of this loss is twofold. First, it forces one HCN iteration to reproduce the sequential trace. Second, it pushes $H^{\cdot}$7 toward $H^{\cdot}$8. This means that HCN is not merely an inference wrapper placed on top of a frozen architecture; rather, it is part of a co-design in which the model is trained to admit accurate structured Newton corrections.

For the reported 0.5B mHC Nanochat experiment, the paper uses $H^{\cdot}$9 streams, $M\times M$0 layers, $M\times M$1, and stride $M\times M$2, with the regularizer matching only the final hidden state. The paper also states more broadly that SNLP regularization improves layer-parallel compatibility and can improve standard sequential perplexity, reducing baseline perplexity by $M\times M$3 on nanochat-scale Transformers.

6. Empirical results, scope, and limitations

The paper reports HCN-specific results on a 0.5B Nanochat-mHC model. The baseline sequential model has validation perplexity $M\times M$4. After SNLP-aware regularization and one HCN iteration with prefix-state initialization, the sequential perplexity becomes $M\times M$5, a reported improvement of $M\times M$6 (Han et al., 18 May 2026).

The detailed mHC Nanochat results are summarized below.

Configuration	PPL	Speed
Seq PPL (No Reg)	73.24	—
Seq PPL (HCN Reg)	67.23	—
20×F1-h0, $M\times M$7	66.56	$M\times M$8
8×F1-h0, $M\times M$9	65.91	$M \ll d$0

These results show two distinct regimes. A chunkwise layer-fusion configuration labeled $M \ll d$1-h0 with $M \ll d$2 yields perplexity $M \ll d$3 and wall-clock speedup of approximately $M \ll d$4. A more quality-oriented configuration labeled $M \ll d$5-h0 with $M \ll d$6 yields perplexity $M \ll d$7 with speed near parity, approximately $M \ll d$8. The paper interprets these results as evidence that HCN can both improve sequential perplexity by “cleaning up the layer trace” and unlock approximately $M \ll d$9 wall-clock speedups on mHC models when chunked carefully (Han et al., 18 May 2026).

At the framework level, the paper reports that SNLP combined with layer fusion and chunkwise decomposition reaches $H=(h_1,\dots,h_L)$0 speedup on a 0.5B Nanochat model while still improving perplexity by $H=(h_1,\dots,h_L)$1. The same paper also characterizes several limitations. Off-the-shelf pretrained models are described as less amenable to the procedure. Exact convergence recovers the sequential computation rather than providing monotonic inference-time scaling. The paper therefore argues that layer-parallel inference should not be viewed solely as a numerical approximation to sequential execution, but can also act as a useful solver-induced inference bias. This suggests that the practical value of HCN lies not only in asymptotic parallelism, but in the joint interaction among architecture, training regularization, and truncated structured Newton inference.