Magnitude–Direction Decoupling

• Magnitude–Direction Decoupling is a technique that separates vector signals into scalar magnitudes and unit-norm directions, clarifying and isolating key signal properties.
• It facilitates precise adjustments in neural networks, signal processing, and reinforcement learning by independently optimizing directional and magnitude components.
• This decoupling enhances robustness, stability, and scalability in models while providing interpretable insights into geometric and error dynamics.

Magnitude–Direction Decoupling refers to the systematic separation of a vector-valued signal or parameter—whether in the context of neural activations, physical forces, model updates, or policy actions—into its scalar magnitude (norm, gain) and its unit-norm direction (angular, geometric, or phase component), with downstream methods or optimizers treating these components as independent objects for analysis, learning, or control. Across a wide array of application domains, this paradigm enables more targeted learning objectives, geometric monitoring, robust parameterizations, and sometimes exact control over critical properties such as signal norm, phase, or angular displacement.

1. Mathematical Foundations of Magnitude–Direction Decoupling

Let $v \in \mathbb{R}^d$ denote a generic vector (e.g., a hidden-state activation, model weight, STFT spectrum, control input, update). The essential decomposition is

$$v = r \cdot u,\qquad \text{where}\quad r = \|v\|_2,\;\; u = v/\|v\|_2,\;\; \|u\|_2 = 1.$$

Magnitude–direction (or magnitude–phase in the complex-valued case) decoupling leverages this separation so that algorithms or analyses treat $r$ and $u$ as distinct entities, each possibly parameterized, optimized, regularized, or quantized independently. In deep learning, this may be implemented per row, column, or per full matrix, by reparameterizing parameters or activations as a product of scalar gains and a constrained direction matrix or vector (Hägele et al., 24 Jun 2026).

Angular relationships (e.g., via cosine similarity) then measure how two directions relate: $$\mathrm{cos}\;\theta = \frac{u_1 \cdot u_2}{\|u_1\|_2\|u_2\|_2}.$$ Empirical and theoretical findings across domains demonstrate that changes in direction (rotations, phase shifts) are often more consequential for system behavior or accuracy than changes in norm, and thus are monitored or controlled separately.

2. Core Mechanisms in Deep Neural Models

2.1 In-Context Learning: LLMs and Geometric Displacement

Recent work in in-context conflict resolution for LLMs (Zhang et al., 4 Feb 2026) provides a canonical example:

• Residual-stream updates are decomposed at each layer into radial (norm-based) and angular (cosine-based) components.
• When LLMs exhibit sycophancy or hallucination, they do not merely dilute learned knowledge via reduced signal magnitude (“Manifold Dilution”); instead, the injected update is quasi-orthogonal to the ground-truth basis vector (“Orthogonal Interference”), rotating rather than shrinking the representation.
• Empirical analysis shows near-zero cosine similarity between the interference vector and the correct direction (cosine ≈ 0 over several models), confirming that performance can degrade sharply with little or no change in signal norm.
• Vectorial monitoring—tracking layer-wise cosine alignment to known truth axes—is required to distinguish genuine knowledge integration (directional shift towards ground truth over multiple steps) from superficial compliance (sudden orthogonal rotation) (Zhang et al., 4 Feb 2026).

2.2 Optimizer Decoupling for Neural Net Weights

“Magnitude–Direction (MD) Decoupling” as an optimizer modification (Hägele et al., 24 Jun 2026) explicitly factorizes each weight matrix: $W = D_r \cdot U \cdot D_c,$ where $U$ has a fixed Frobenius norm (direction on a hypersphere), and $D_r$, $D_c$ are learned per-row/per-column non-negative gains (magnitudes). The direction and magnitude receive independent gradient updates at distinct learning rates; $U$ is projected back to its initial norm after each step. This:

• Removes the need for classical weight decay and learning-rate warmup, as kept norms and rate-separation yield stable training without auxiliary heuristics.
• Achieves robust learning-rate transfer across model widths and depths, as the angular learning rate governs the true step size.
• Outperforms well-tuned Adam/Muon baselines on large-scale LMs and Mixture-of-Experts models in terms of loss, hyperparameter robustness, and compute efficiency (Hägele et al., 24 Jun 2026).

3. Signal Processing, Source Separation, and Masking

Magnitude–direction decoupling is fundamental in signal separation problems where phase (direction) recovery is critical.

3.1 Speech Enhancement: Two-Stage Decoupling

In the two-stage Deep Noise Suppression framework (Li et al., 2021):

• Stage 1 estimates only the magnitude $$v = r \cdot u,\qquad \text{where}\quad r = \|v\|_2,\;\; u = v/\|v\|_2,\;\; \|u\|_2 = 1.$$0 from $$v = r \cdot u,\qquad \text{where}\quad r = \|v\|_2,\;\; u = v/\|v\|_2,\;\; \|u\|_2 = 1.$$1.
• Stage 2 uses the coarse estimated magnitude and the noisy complex phase to refine both, predicting a small complex residual to improve phase and magnitude.
• The joint loss is a weighted sum of real-imaginary MSE and magnitude MSE.
• This explicit decoupling eases learning, improves convergence and results in state-of-the-art subjective and objective scores.

3.2 Music Source Separation: cIRM Decomposition

For complex ideal ratio masks (cIRMs) in source separation (Kong et al., 2021):

• The complex mask $$v = r \cdot u,\qquad \text{where}\quad r = \|v\|_2,\;\; u = v/\|v\|_2,\;\; \|u\|_2 = 1.$$2 is split into a magnitude branch ($$v = r \cdot u,\qquad \text{where}\quad r = \|v\|_2,\;\; u = v/\|v\|_2,\;\; \|u\|_2 = 1.$$3, possibly $$v = r \cdot u,\qquad \text{where}\quad r = \|v\|_2,\;\; u = v/\|v\|_2,\;\; \|u\|_2 = 1.$$4) and a phase-direction branch ($$v = r \cdot u,\qquad \text{where}\quad r = \|v\|_2,\;\; u = v/\|v\|_2,\;\; \|u\|_2 = 1.$$5, normalized to the unit circle).
• Explicit decoupling allows the architecture to bypass the “$$v = r \cdot u,\qquad \text{where}\quad r = \|v\|_2,\;\; u = v/\|v\|_2,\;\; \|u\|_2 = 1.$$6” magnitude ceiling and to recover out-of-phase sources, yielding a significant SDR increase (e.g., vocals SDR 8.98 dB, vs previous best 7.24 dB).
• The direction branch is always normalized, while the magnitude can be specialized for bins with $$v = r \cdot u,\qquad \text{where}\quad r = \|v\|_2,\;\; u = v/\|v\|_2,\;\; \|u\|_2 = 1.$$7.
• This approach generalizes to speech enhancement, dereverberation, and complex-valued data domains.

4. Robust Parameterizations and Policy Design

4.1 Reinforcement Learning: MAD Policy Parameterization

The MAD (“Magnitude And Direction”) policy framework (Furieri et al., 3 Apr 2025) for RL decomposes actions as $$v = r \cdot u,\qquad \text{where}\quad r = \|v\|_2,\;\; u = v/\|v\|_2,\;\; \|u\|_2 = 1.$$8, with $$v = r \cdot u,\qquad \text{where}\quad r = \|v\|_2,\;\; u = v/\|v\|_2,\;\; \|u\|_2 = 1.$$9.

• The magnitude operator $r$0 is constrained to be $r$1-stable (disturbance feedback), ensuring closed-loop stability regardless of the direction.
• The direction term is a state-feedback unit vector, parameterized by a normalized neural net.
• Separate parameterizations enable explicit stability guarantees across general nonlinear plants, and robust transfer under model mismatch.
• MAD policies generalize as well as unconstrained neural net actors, but never destabilize or produce unbounded control signals (Furieri et al., 3 Apr 2025).

4.2 Low-Rank Adaptation: Dual LoRA

Dual LoRA (Xu et al., 3 Dec 2025) improves on conventional LoRA by decoupling update matrices into a non-negative “magnitude” group (ReLU(B A)) and a {$r$2} “direction” group (Sign(D C)). The update: $r$3 reflects true step size (how much) and update polarity (which way), more closely mimicking full gradient-based optimization. Across NLP, NLU, and reasoning tasks Dual LoRA delivers consistent accuracy and BLEU gains with unchanged parameter budget.

5. Model Compression, Quantization, and Geometric Editing

5.1 Vector Quantization: Polar Coordinate Decoupling

PCDVQ (“Polar Coordinate Decoupled Vector Quantization”) (Yue et al., 5 Jun 2025) applies separate quantization to the magnitude and direction of weight vectors in LLMs:

• Direction (angular component) sensitivity: At 2-bit clustering, quantizing only magnitude incurs ~2.3% accuracy drop, but direction-only quantization leads to ~46.5% drop.
• Algorithm: Decompose $r$4, codebooks for both, separately cluster, then reconstruct via $r$5.
• Distribution-aligned codebooks further optimize fidelity.
• This decoupling paradigm enables extreme compression with minimal degradation, providing superiority over Euclidean-quantized baselines at ultra-low bitrates.

5.2 Activation Editing: Householder Pseudo-Rotation

Householder Pseudo-Rotation (Pham et al., 2024) edits internal LLM activations by rotating directions on the sphere while preserving magnitude:

• Each vector is reflected then planar-rotated (norm-preserving) to change its “direction” relative to undesirable/targeted probes.
• Steering-vector edits (additive) unavoidably perturb norms, destabilizing internal scaling.
• HPR yields substantial gains on truthfulness, bias, and safety benchmarks over steering methods, without changing perplexity or output fluency.

6. Specialized Applications and Theoretical Decoupling

6.1 Physics: Spin-Orbit Torque Engineering

In spintronics, decoupling of magnitude and direction is exemplified in bilayer SOT devices (He et al., 2017):

• Independent control of current-induced effective field magnitude (via tuning Pt layer thickness) and torque orientation (fixed by interfacial symmetry).
• The ratio of longitudinal to transverse SOT components is locked by interface properties, while overall magnitude and sign can be swept by geometric design.
• The DMI is decoupled from SOT magnitude, enabling device-level modulation of switching thresholds without altering chiral properties.

6.2 Causal Inference: Collider Bias in Binary Variables

Bias induced by conditioning on a collider is analytically decomposed into independently computable “magnitude” and “direction” (sign) factors (Nguyen et al., 2016): $r$6 This decoupling allows practitioners to predict over/underestimation from the sign term alone, and separately assess magnitude via marginal prevalences, guiding informed confounder adjustment decisions.

6.3 Computer Vision: Motion Direction Decoupling in Tracking

In fast object tracking (e.g. sports), TrackNetV5 (Haonan et al., 2 Dec 2025) demonstrates pixel-level motion direction decoupling. Difference images are split into positive and negative polarity channels (brightening and darkening regions), conveying both trajectory occurrence and vectorial route, not just intensity of change. This explicit modeling markedly improves occlusion and blur recovery, yielding precise high-F1 results with minimal compute cost.

7. Broader Implications and Future Directions

Magnitude–direction decoupling universally exposes underlying geometric, probabilistic, or algorithmic structure in signal processing, machine learning, physics, and statistical inference. Across domains:

• It enables more interpretable and robust learning and control by isolating components most sensitive to accuracy, error, or stability.
• It forms the basis for geometric detectors of hallucination, sycophancy, or spurious bias in high-dimensional models.
• Separate parameterization and optimization yield hyperparameter robustness, predictable scaling behavior, and the removal of major training heuristics (weight decay, warmup).
• In model merging, compression, and adaptation, it allows selective injection of complementary properties (e.g., multilinguality vs multimodality) by layer or parameter subset.

Systematically disentangling magnitude and direction is now established as a foundational strategy for controlling, understanding, and improving the behavior of complex vector-valued systems in contemporary research and engineering practice.