Iterative Noise Alignment (iterNA)

• iterNA is a framework where iterative updates incorporate noise modeling to distinguish signal from spurious data while improving alignment quality.
• It is applied in diverse settings such as vector-space mapping, policy refinement under noisy verifiers, multi-reference alignment, and diffusion-based imaging.
• The approach highlights that without proper constraints like mini-batching or guidance, more iterations or data can inadvertently amplify noise-induced biases.

Iterative Noise Alignment (iterNA) denotes a family of iterative alignment procedures in which noise is not treated solely as nuisance variance, but as a central factor in how correspondences, transformations, preferences, or latent states are updated. Across recent arXiv literature, the term is used in several technically distinct but structurally related senses: EM-based de-noising of supervised lexicons in vector-space mapping, iterative policy alignment under noisy verifiers in mathematical reasoning, low-SNR alignment-and-averaging pathologies in multi-reference alignment, and score-based stochastic alignment in cross-modal medical imaging and diffusion generation (Lubin et al., 2019, Rao et al., 5 Feb 2026, Balanov et al., 27 May 2025, Lee et al., 11 Apr 2026, Guo et al., 2024). This suggests that iterNA is best understood as an umbrella concept for iterative alignment under noisy or noise-mediated updates, rather than a single canonical algorithm.

1. Conceptual scope and usage

The literature assigns iterNA to at least three recurring technical patterns. In one pattern, the objective is to identify and suppress corrupted supervision. This is the role played by the EM-based noise-aware alignment model for supervised lexicons, where each training pair is treated as either an inlier or an outlier and the mapping is updated using posterior inlier weights (Lubin et al., 2019). In a second pattern, the objective is to avoid iterative amplification of noise-induced bias. This is explicit in multi-reference alignment (MRA), where repeated alignment to the current estimate can cause the algorithm to align the noise itself, producing biased reconstructions such as “Einstein from Noise” and “Ghost of Newton” (Balanov et al., 27 May 2025). In a third pattern, noise is used constructively as a search mechanism. Score-based methods for retinal image registration and diffusion-based image generation inject or optimize noise in order to regularize updates, improve exploration, and escape poor local optima (Lee et al., 11 Apr 2026, Guo et al., 2024).

A related but distinct formulation appears in iterative alignment for LLMs. There, iterNA is framed around how exploration budgets, verifier error, and distribution shift interact across training iterations. The PACE framework treats iterative policy updates under noisy feedback as a signal-purity problem: larger exploration breadth can amplify false positives and drive trust-region violations, whereas small-budget corrective exploration can preserve pair quality and long-term stability (Rao et al., 5 Feb 2026).

Setting	Role of noise	Canonical mechanism
Vector-space alignment	Corrupted supervision to be inferred	EM mixture over inliers and outliers
Mathematical reasoning alignment	Verifier noise and label corruption	DPO-style iterative preference optimization
Multi-reference alignment	Noise aligned to the current template	EM over latent group actions
Cross-modal retinal alignment	Injected stochasticity for exploration	Coupled score-based diffusion
Text-to-image generation	Initial latent noise quality	Attention-guided noise optimization

A common misconception is that iterNA always denotes a deliberate robustness method. The cited work shows both constructive and pathological forms. In some settings, iterative procedures de-noise supervision or use stochasticity to improve search; in others, the same iterative structure can lock onto spurious correlations and strengthen them over time.

2. Noise-aware supervised alignment in vector spaces

In supervised vector-space alignment, iterNA appears as an EM procedure that jointly estimates a linear map and the noise structure of the supervision set. The setting assumes source embeddings $X=\{x_i\}$, target embeddings $Y=\{y_i\}$, and a lexicon of aligned pairs $(x_i,y_i)$, some of which may be incorrect. The goal is to recover a linear mapping $W \in \mathbb{R}^{d \times d}$ such that $W x_i \approx y_i$ for correct pairs, despite lexicon noise (Lubin et al., 2019).

The generative model introduces a latent binary variable $z_i \in \{0,1\}$ indicating whether pair $i$ is an inlier. With mixing proportion $\pi=P(z_i=1)$, the pairwise likelihood is

$$p(y_i \mid x_i,\theta)=\pi\,\mathcal{N}(y_i;Wx_i,\sigma^2 I)+(1-\pi)\,\mathcal{N}(y_i;\mu_y,\sigma_y^2 I),$$

where the first Gaussian models aligned pairs and the second models outliers using a global target-space mean $\mu_y$ and isotropic variance $Y=\{y_i\}$0. The E-step computes responsibilities

$Y=\{y_i\}$1

which act as posterior inlier probabilities. The M-step updates $Y=\{y_i\}$2, $Y=\{y_i\}$3, $Y=\{y_i\}$4, $Y=\{y_i\}$5, and $Y=\{y_i\}$6.

For orthogonal alignment, the update for $Y=\{y_i\}$7 is a weighted orthogonal Procrustes problem. With $Y=\{y_i\}$8 and weighted cross-covariance

$Y=\{y_i\}$9

the SVD $(x_i,y_i)$0 yields the update $(x_i,y_i)$1. An unconstrained weighted least-squares variant is also available:

$(x_i,y_i)$2

with optional Tikhonov regularization when needed. The paper also reports a hard-EM variant using $(x_i,y_i)$3, which converged faster and typically produced posteriors close to $(x_i,y_i)$4.

Empirically, the approach improves bilingual word translation accuracy over a strong orthogonal Procrustes baseline while often requiring fewer iterations. Reported best and average precision@1 values include En→De best $(x_i,y_i)$5 vs $(x_i,y_i)$6 and average $(x_i,y_i)$7 vs $(x_i,y_i)$8, En→Fi best $(x_i,y_i)$9 vs $W \in \mathbb{R}^{d \times d}$0 and average $W \in \mathbb{R}^{d \times d}$1 vs $W \in \mathbb{R}^{d \times d}$2, En→Es best $W \in \mathbb{R}^{d \times d}$3 vs $W \in \mathbb{R}^{d \times d}$4 and average $W \in \mathbb{R}^{d \times d}$5 vs $W \in \mathbb{R}^{d \times d}$6, and En→It equal best $W \in \mathbb{R}^{d \times d}$7 with slightly improved average $W \in \mathbb{R}^{d \times d}$8 vs $W \in \mathbb{R}^{d \times d}$9; the soft-EM variant attains similar accuracy but requires about $W x_i \approx y_i$0 more iterations (Lubin et al., 2019). In diachronic alignment, $W x_i \approx y_i$1 of self-lexicon pairs were identified as noise, reflecting semantic shift and other artifacts.

This formulation represents the most classical sense of iterNA: iterative estimation alternates between alignment and explicit noise attribution. Noise is modeled, quantified, and downweighted rather than passively tolerated.

3. Iterative policy alignment under noisy verifiers

In iterative alignment for mathematical reasoning, iterNA is formulated around a noise–shift trade-off induced by exploration. Iterative Direct Preference Optimization, exemplified by DPO-R1, constructs preference pairs $W x_i \approx y_i$2 by sampling candidate trajectories, scoring them with a verifier $W x_i \approx y_i$3, and updating the policy to prefer winners over losers. The dominant heuristic is Best-of-$W x_i \approx y_i$4 (BoN) sampling: draw $W x_i \approx y_i$5 candidate completions, use the highest-scoring completion as $W x_i \approx y_i$6, and select a failing sample as $W x_i \approx y_i$7. The working assumption has been that larger $W x_i \approx y_i$8 reveals more correct solutions in the tail (Rao et al., 5 Feb 2026).

The PACE analysis challenges that assumption for mathematical reasoning. Let $W x_i \approx y_i$9 denote intrinsic capability under a ground-truth oracle $z_i \in \{0,1\}$0, and let $z_i \in \{0,1\}$1 denote the verifier’s conditional defect rate, with perfect recall on correct solutions. The verifier pass rate is

$z_i \in \{0,1\}$2

and the false-positive probability among verifier-passing trajectories is

$z_i \in \{0,1\}$3

Since

$z_i \in \{0,1\}$4

false-positive prevalence decreases with capability. However, BoN changes which prompts contribute positives. The probability of obtaining at least one verifier pass under budget $z_i \in \{0,1\}$5 is

$z_i \in \{0,1\}$6

As $z_i \in \{0,1\}$7 grows, prompts with low empirical pass rate begin to produce at least one passing sample; these marginal positives are increasingly likely to be spurious. Using the MLE posterior correction

$z_i \in \{0,1\}$8

projected to $z_i \in \{0,1\}$9, the analysis shows that for marginal tasks passing only when the budget increases from $i$0 to $i$1, one has $i$2 and $i$3 as $i$4, implying $i$5. A Bayesian refinement further states that for a marginal task producing exactly one pass at attempt $i$6, the posterior expected false-positive rate increases strictly with $i$7.

The second pathology is distribution shift. Even with a perfect verifier, the KL divergence lower bound between a target policy with success at least $i$8 and a reference policy with success rate $i$9 satisfies

$\pi=P(z_i=1)$0

Under BoN with marginal $\pi=P(z_i=1)$1, the lower bound becomes

$\pi=P(z_i=1)$2

where $\pi=P(z_i=1)$3. The implication is that increasing $\pi=P(z_i=1)$4 drives updates toward tail trajectories and inflates the minimum KL shift roughly as $\pi=P(z_i=1)$5, violating a proximal trust region and destabilizing optimization.

PACE replaces brute-force mining with corrective exploration at a minimal effective budget $\pi=P(z_i=1)$6, implemented as $\pi=P(z_i=1)$7 plus one corrective refinement when both samples fail. The training loss remains a standard DPO objective,

$\pi=P(z_i=1)$8

but PACE changes how $\pi=P(z_i=1)$9 is constructed. When one of two trajectories passes verification, a natural pair is added. When both fail, the method generates a corrected path

$$p(y_i \mid x_i,\theta)=\pi\,\mathcal{N}(y_i;Wx_i,\sigma^2 I)+(1-\pi)\,\mathcal{N}(y_i;\mu_y,\sigma_y^2 I),$$0

applies a consistency filter that strips the final answer and checks whether the reasoning entails the ground-truth answer, then forms a synthetic proximal pair $$p(y_i \mid x_i,\theta)=\pi\,\mathcal{N}(y_i;Wx_i,\sigma^2 I)+(1-\pi)\,\mathcal{N}(y_i;\mu_y,\sigma_y^2 I),$$1 using the higher-probability failure as an on-policy hard negative. The expected generation cost per prompt is

$$p(y_i \mid x_i,\theta)=\pi\,\mathcal{N}(y_i;Wx_i,\sigma^2 I)+(1-\pi)\,\mathcal{N}(y_i;\mu_y,\sigma_y^2 I),$$2

with $$p(y_i \mid x_i,\theta)=\pi\,\mathcal{N}(y_i;Wx_i,\sigma^2 I)+(1-\pi)\,\mathcal{N}(y_i;\mu_y,\sigma_y^2 I),$$3.

Empirically, PACE outperforms DPO-R1 at $$p(y_i \mid x_i,\theta)=\pi\,\mathcal{N}(y_i;Wx_i,\sigma^2 I)+(1-\pi)\,\mathcal{N}(y_i;\mu_y,\sigma_y^2 I),$$4 while using about one-fifth of the compute. On Qwen3-4B, per-sample processing time is $$p(y_i \mid x_i,\theta)=\pi\,\mathcal{N}(y_i;Wx_i,\sigma^2 I)+(1-\pi)\,\mathcal{N}(y_i;\mu_y,\sigma_y^2 I),$$5 seconds for DPO-R1 $$p(y_i \mid x_i,\theta)=\pi\,\mathcal{N}(y_i;Wx_i,\sigma^2 I)+(1-\pi)\,\mathcal{N}(y_i;\mu_y,\sigma_y^2 I),$$6 versus $$p(y_i \mid x_i,\theta)=\pi\,\mathcal{N}(y_i;Wx_i,\sigma^2 I)+(1-\pi)\,\mathcal{N}(y_i;\mu_y,\sigma_y^2 I),$$7 second for PACE. On Llama-3.1-8B, average accuracy across math benchmarks is $$p(y_i \mid x_i,\theta)=\pi\,\mathcal{N}(y_i;Wx_i,\sigma^2 I)+(1-\pi)\,\mathcal{N}(y_i;\mu_y,\sigma_y^2 I),$$8 for PACE versus $$p(y_i \mid x_i,\theta)=\pi\,\mathcal{N}(y_i;Wx_i,\sigma^2 I)+(1-\pi)\,\mathcal{N}(y_i;\mu_y,\sigma_y^2 I),$$9 for DPO-R1 $\mu_y$0; on Qwen3-8B, the corresponding figures are $\mu_y$1 versus $\mu_y$2. Under $\mu_y$3 label corruption, PACE maintains Noise Hit Rate around $\mu_y$4 and monotonic improvement across four iterations up to about $\mu_y$5 accuracy, whereas DPO-R1 $\mu_y$6 reaches Noise Hit Rate around $\mu_y$7 and exhibits late-iteration degradation. Topology analysis further reports higher winner–loser cosine similarity for PACE, about $\mu_y$8 versus about $\mu_y$9 for BoN, indicating more proximal hard negatives (Rao et al., 5 Feb 2026).

Within this usage, iterNA emphasizes that iterative alignment quality depends less on the raw volume of explored trajectories than on how exploration modulates exposure to noisy feedback.

4. Alignment-induced noise amplification in multi-reference alignment

In multi-reference alignment, iterNA refers to the mechanism by which repeated alignment-and-averaging steps lock onto spurious correlations in noisy observations and reinforce them across iterations. The model assumes an unknown signal $Y=\{y_i\}$00, a group $Y=\{y_i\}$01 acting orthogonally on $Y=\{y_i\}$02, and observations

$Y=\{y_i\}$03

The marginal likelihood is a group mixture, and EM alternates between posterior alignment weights over group elements and an average of aligned observations (Balanov et al., 27 May 2025).

With current estimate $Y=\{y_i\}$04, the E-step computes

$Y=\{y_i\}$05

and, for uniform $Y=\{y_i\}$06,

$Y=\{y_i\}$07

The M-step updates

$Y=\{y_i\}$08

Although EM increases the log-likelihood monotonically, this does not imply decreasing error with respect to the true signal.

The pure-noise regime isolates the pathology. In “Einstein from Noise,” observations are $Y=\{y_i\}$09, but EM is initialized from a nontrivial template such as Einstein. Even in this case, the algorithm aligns each noisy sample to the template and averages the aligned noise. Theoretical results show that, as $Y=\{y_i\}$10, Fourier phases of successive iterates are preserved and reinforced; the phase error decreases as $Y=\{y_i\}$11, and successive iterates satisfy $Y=\{y_i\}$12 almost surely for large $Y=\{y_i\}$13. In high dimension, under vanishing long-shift autocorrelation, $Y=\{y_i\}$14 in probability as $Y=\{y_i\}$15, so the procedure converges to the initialization. The data therefore identify iterNA as a positive-feedback loop in which aligned noise becomes correlated with the current estimate.

At moderately low SNR, the paper describes “Ghost of Newton.” The update decomposes into aligned signal and aligned noise,

$Y=\{y_i\}$16

Early iterations are dominated by the aligned signal and move toward the true object, but later iterations increasingly align noise to the estimate that already resembles the signal. The result is a structured noise image that looks like a noisy version of the target. When the norms of the aligned-noise and aligned-signal components become comparable, reconstruction fidelity degrades even though the log-likelihood keeps rising. The paper reports a representative case at $Y=\{y_i\}$17, where EM improves for about $Y=\{y_i\}$18 iterations and then diverges in MSE.

The proposed remedy is mini-batching. Instead of full-batch EM, the method computes stochastic gradients or EM-like batch averages on small subsets:

$Y=\{y_i\}$19

followed by an adaptive optimizer such as ADAM or Ranger. The rationale is that the per-iteration bias of aligned noise scales with the amount of data used in each update. Since Einstein-from-Noise bias scales like $Y=\{y_i\}$20, reducing the effective $Y=\{y_i\}$21 per iteration weakens phase-locking and breaks the feedback loop. In experiments with $Y=\{y_i\}$22 and $Y=\{y_i\}$23, full-batch EM over $Y=\{y_i\}$24 iterations took about $Y=\{y_i\}$25 minutes, whereas mini-batch SGD with batch size $Y=\{y_i\}$26 over $Y=\{y_i\}$27 iterations took about $Y=\{y_i\}$28 minutes, matched EM’s accuracy at moderate and high SNR, and avoided Ghost-of-Newton drift (Balanov et al., 27 May 2025).

This line of work establishes a central iterNA lesson: more averaging and more iterations can intensify bias if the alignment operator is itself noise-sensitive.

5. Score-based and diffusion formulations

A different usage of iterNA appears in score-based alignment and diffusion generation, where noise is deliberately injected or optimized to regularize inference and improve search. In Active Diffusion Matching (ADM), iterNA is defined as image alignment through a stochastic iterative process that progressively refines transformation parameters under injected noise, guided by learned score functions. The method addresses alignment between Standard Fundus Images and Ultra-Widefield Fundus Images by jointly estimating a global homography $Y=\{y_i\}$29 and a dense local deformation field $Y=\{y_i\}$30 through two interdependent score-based diffusion models (Lee et al., 11 Apr 2026).

The warped source image is

$Y=\{y_i\}$31

and inference is guided by an appearance-plus-regularization energy

$Y=\{y_i\}$32

where $Y=\{y_i\}$33 is $Y=\{y_i\}$34 between vessel-enhanced warped source and target images, $Y=\{y_i\}$35 penalizes homography deviation from identity, and $Y=\{y_i\}$36 enforces local smoothness. ADM trains global and local score networks using denoising score matching with Gaussian perturbations,

$Y=\{y_i\}$37

$Y=\{y_i\}$38

and performs reverse updates

$Y=\{y_i\}$39

$Y=\{y_i\}$40

The global score is modified by input-adaptive guidance,

$Y=\{y_i\}$41

which couples the global and local chains.

The reported results show state-of-the-art accuracy. On the private KBSMC SFI–UWFI dataset, ADM achieves Acceptable $Y=\{y_i\}$42 and mAUC $Y=\{y_i\}$43, improving mAUC by $Y=\{y_i\}$44 points over GeoFormer’s $Y=\{y_i\}$45 and Acceptable rate by $Y=\{y_i\}$46 percentage points over $Y=\{y_i\}$47. On FIRE, ADM reaches Acceptable $Y=\{y_i\}$48 and mAUC $Y=\{y_i\}$49, which is $Y=\{y_i\}$50 above GeoFormer’s $Y=\{y_i\}$51. Inference cost is about $Y=\{y_i\}$52 seconds per pair and $Y=\{y_i\}$53 GB memory on a single RTX 4090 for $Y=\{y_i\}$54 images (Lee et al., 11 Apr 2026).

The same broad logic appears in text-to-image diffusion, but here the target of alignment is the initial latent noise rather than a geometric transform. InitNO formalizes the observation that not all initial noise samples are equally compatible with a text prompt. It defines a cross-attention response score

$Y=\{y_i\}$55

and a self-attention conflict score

$Y=\{y_i\}$56

with thresholds $Y=\{y_i\}$57 and $Y=\{y_i\}$58 to classify an initial noise as valid or invalid. Rather than directly perturbing the latent, InitNO optimizes parameters $Y=\{y_i\}$59 of a Gaussian

$Y=\{y_i\}$60

using the joint objective

$Y=\{y_i\}$61

with $Y=\{y_i\}$62, $Y=\{y_i\}$63, and $Y=\{y_i\}$64. The KL term

$Y=\{y_i\}$65

keeps the optimized noise close to the diffusion prior (Guo et al., 2024).

InitNO is described as an initial-stage variant of iterative noise alignment: optimization iterates over the initial noise parameters and uses first-step attention maps as a proxy for later semantic fidelity, avoiding per-step trajectory interventions. It is training-free and plug-and-play. The paper reports superior CLIP-based alignment metrics and a user study in which InitNO received $Y=\{y_i\}$66 preference versus $Y=\{y_i\}$67 for Attend-and-Excite, $Y=\{y_i\}$68 for Divide-and-Bind, and smaller shares for the other baselines; runtime increases from $Y=\{y_i\}$69 seconds per image for baseline Stable Diffusion to $Y=\{y_i\}$70 seconds per image on a Tesla V100 32 GB at $Y=\{y_i\}$71 (Guo et al., 2024).

These diffusion-based formulations invert the pathology observed in low-SNR EM. Instead of accidentally aligning noise to the current estimate, they design stochasticity so that noise aids controlled exploration, regularization, or prompt-faithful initialization.

6. Cross-cutting principles, misconceptions, and open problems

Across these literatures, several shared principles recur. First, iterative alignment is highly sensitive to how noise enters the update. When noise is explicitly modeled as a latent mixture component, as in noisy lexicon alignment, it can be estimated and discounted. When stochasticity is injected through a score-based sampler and coupled to anatomy-aware or attention-aware guidance, it can improve search robustness and escape poor local optima. By contrast, when iterative updates repeatedly select trajectories that exploit verifier defects, or repeatedly align observations to a biased template, the procedure amplifies noise rather than attenuating it (Lubin et al., 2019, Lee et al., 11 Apr 2026, Guo et al., 2024, Rao et al., 5 Feb 2026, Balanov et al., 27 May 2025).

Second, the literature repeatedly rejects the heuristic that more exploration, more data per iteration, or higher likelihood necessarily improves alignment. In PACE, increasing BoN budget raises false-positive exposure and the KL lower bound roughly as $Y=\{y_i\}$72, so aggressive exploration can trigger policy collapse rather than better reasoning (Rao et al., 5 Feb 2026). In MRA, EM increases log-likelihood monotonically while MSE can worsen, and larger full-batch updates strengthen template bias instead of correcting it (Balanov et al., 27 May 2025). A plausible implication is that iterative alignment should be evaluated in terms of signal purity and update geometry, not only final objective values.

Third, successful iterNA designs tend to impose proximality or filtering constraints. PACE uses $Y=\{y_i\}$73, consistency gating, and on-policy hard negatives; noisy lexicon alignment uses posterior responsibilities or hard thresholds; ADM uses guidance on vessel-enhanced images with regularization on $Y=\{y_i\}$74 and $Y=\{y_i\}$75; InitNO constrains optimized noise through a KL penalty. These mechanisms differ, but they all limit the influence of evidence that is distant from the current model, weakly justified, or likely to be spurious.

The open questions are correspondingly structural. For iterative reasoning alignment, the noise analysis assumes perfect recall on correct solutions and constant $Y=\{y_i\}$76; extending the theory to instance-dependent $Y=\{y_i\}$77 is explicitly identified as important (Rao et al., 5 Feb 2026). For score-based samplers such as ADM, formal convergence guarantees are not provided, and pathological image pairs can still fail (Lee et al., 11 Apr 2026). For open-ended tasks without precise ground truth, robust consistency checks are harder to design, which complicates transfer of PACE-like corrective alignment beyond mathematics (Rao et al., 5 Feb 2026). In vector-space alignment, EM can struggle when the inlier proportion is very small or when the orthogonality assumption breaks down (Lubin et al., 2019). In MRA, initialization sensitivity remains fundamental, even though mini-batching mitigates the worst artifacts (Balanov et al., 27 May 2025).

Taken together, these results position iterNA as a unifying language for a broad class of iterative procedures in which alignment quality depends on whether noise is inferred, constrained, corrected, or inadvertently reinforced. The central technical divide is not between deterministic and stochastic methods, but between update rules that preserve signal fidelity under repeated alignment and update rules that progressively align the model to its own noise-corrupted evidence.