--- title: Iterative Noise Alignment (iterNA) url: https://www.emergentmind.com/topics/iterative-noise-alignment-iterna type: topic --- # Iterative Noise Alignment (iterNA) Iterative Noise Alignment (iterNA) denotes a family of iterative alignment procedures in which noise is not treated solely as nuisance variance, but as a central factor in how correspondences, transformations, preferences, or latent states are updated. Across recent arXiv literature, the term is used in several technically distinct but structurally related senses: EM-based de-noising of supervised lexicons in vector-space mapping, iterative policy alignment under noisy verifiers in mathematical reasoning, low-SNR alignment-and-averaging pathologies in multi-reference alignment, and score-based stochastic alignment in cross-modal medical imaging and diffusion generation [1903.10238; 2602.05370; 2505.21435; 2604.10084; 2404.04650]. This suggests that iterNA is best understood as an umbrella concept for iterative alignment under noisy or noise-mediated updates, rather than a single canonical algorithm. ## 1. Conceptual scope and usage The literature assigns iterNA to at least three recurring technical patterns. In one pattern, the objective is to **identify and suppress corrupted supervision**. This is the role played by the EM-based noise-aware alignment model for supervised lexicons, where each training pair is treated as either an inlier or an outlier and the mapping is updated using posterior inlier weights [1903.10238]. In a second pattern, the objective is to **avoid iterative amplification of noise-induced bias**. This is explicit in multi-reference alignment (MRA), where repeated alignment to the current estimate can cause the algorithm to align the noise itself, producing biased reconstructions such as “Einstein from Noise” and “Ghost of Newton” [2505.21435]. In a third pattern, noise is used **constructively as a search mechanism**. Score-based methods for retinal image registration and diffusion-based image generation inject or optimize noise in order to regularize updates, improve exploration, and escape poor local optima [2604.10084; 2404.04650]. A related but distinct formulation appears in iterative alignment for large language models. There, iterNA is framed around how exploration budgets, verifier error, and distribution shift interact across training iterations. The PACE framework treats iterative policy updates under noisy feedback as a signal-purity problem: larger exploration breadth can amplify false positives and drive trust-region violations, whereas small-budget corrective exploration can preserve pair quality and long-term stability [2602.05370]. | Setting | Role of noise | Canonical mechanism | |---|---|---| | Vector-space alignment | Corrupted supervision to be inferred | EM mixture over inliers and outliers | | Mathematical reasoning alignment | Verifier noise and label corruption | DPO-style iterative preference optimization | | Multi-reference alignment | Noise aligned to the current template | EM over latent group actions | | Cross-modal retinal alignment | Injected stochasticity for exploration | Coupled score-based diffusion | | Text-to-image generation | Initial latent noise quality | Attention-guided noise optimization | A common misconception is that iterNA always denotes a deliberate robustness method. The cited work shows both constructive and pathological forms. In some settings, iterative procedures de-noise supervision or use stochasticity to improve search; in others, the same iterative structure can lock onto spurious correlations and strengthen them over time. ## 2. Noise-aware supervised alignment in vector spaces In supervised vector-space alignment, iterNA appears as an EM procedure that jointly estimates a linear map and the noise structure of the supervision set. The setting assumes source embeddings $X=\{x_i\}$, target embeddings $Y=\{y_i\}$, and a lexicon of aligned pairs $(x_i,y_i)$, some of which may be incorrect. The goal is to recover a linear mapping $W \in \mathbb{R}^{d \times d}$ such that $W x_i \approx y_i$ for correct pairs, despite lexicon noise [1903.10238]. The generative model introduces a latent binary variable $z_i \in \{0,1\}$ indicating whether pair $i$ is an inlier. With mixing proportion $\pi=P(z_i=1)$, the pairwise likelihood is $$ p(y_i \mid x_i,\theta)=\pi\,\mathcal{N}(y_i;Wx_i,\sigma^2 I)+(1-\pi)\,\mathcal{N}(y_i;\mu_y,\sigma_y^2 I), $$ where the first Gaussian models aligned pairs and the second models outliers using a global target-space mean $\mu_y$ and isotropic variance $\sigma_y^2$. The E-step computes responsibilities $$ r_i=P(z_i=1 \mid x_i,y_i,\theta) =\frac{\pi\,\mathcal{N}(y_i;Wx_i,\sigma^2 I)} {\pi\,\mathcal{N}(y_i;Wx_i,\sigma^2 I)+(1-\pi)\,\mathcal{N}(y_i;\mu_y,\sigma_y^2 I)}, $$ which act as posterior inlier probabilities. The M-step updates $\pi$, $\mu_y$, $\sigma_y^2$, $\sigma^2$, and $W$. For orthogonal alignment, the update for $W$ is a weighted orthogonal Procrustes problem. With $R=\mathrm{diag}(r_1,\dots,r_n)$ and weighted cross-covariance $$ A=YRX^\top=\sum_{i=1}^n r_i\, y_i x_i^\top, $$ the SVD $A=U\Sigma V^\top$ yields the update $W \leftarrow UV^\top$. An unconstrained weighted least-squares variant is also available: $$ W \leftarrow (YRX^\top)(XRX^\top)^{-1}, $$ with optional Tikhonov regularization when needed. The paper also reports a hard-EM variant using $h_i=\mathbf{1}(r_i>0.5)$, which converged faster and typically produced posteriors close to $0/1$. Empirically, the approach improves bilingual word translation accuracy over a strong orthogonal Procrustes baseline while often requiring fewer iterations. Reported best and average precision@1 values include En→De best $49.67$ vs $48.47$ and average $48.89$ vs $48.19$, En→Fi best $33.98$ vs $33.50$ and average $33.68$ vs $32.63$, En→Es best $38.40$ vs $37.60$ and average $37.79$ vs $37.33$, and En→It equal best $48.53$ with slightly improved average $48.20$ vs $48.13$; the soft-EM variant attains similar accuracy but requires about $15\%$ more iterations [1903.10238]. In diachronic alignment, $45.5\%$ of self-lexicon pairs were identified as noise, reflecting semantic shift and other artifacts. This formulation represents the most classical sense of iterNA: iterative estimation alternates between alignment and explicit noise attribution. Noise is modeled, quantified, and downweighted rather than passively tolerated. ## 3. Iterative policy alignment under noisy verifiers In iterative alignment for mathematical reasoning, iterNA is formulated around a noise–shift trade-off induced by exploration. Iterative Direct Preference Optimization, exemplified by DPO-R1, constructs preference pairs $(x,y_w,y_l)$ by sampling candidate trajectories, scoring them with a verifier $V$, and updating the policy to prefer winners over losers. The dominant heuristic is Best-of-$N$ (BoN) sampling: draw $N$ candidate completions, use the highest-scoring completion as $y_w$, and select a failing sample as $y_l$. The working assumption has been that larger $N$ reveals more correct solutions in the tail [2602.05370]. The PACE analysis challenges that assumption for mathematical reasoning. Let $a=P(R(y)=1)$ denote intrinsic capability under a ground-truth oracle $R$, and let $\epsilon=P(V(y)=1 \mid R(y)=0)$ denote the verifier’s conditional defect rate, with perfect recall on correct solutions. The verifier pass rate is $$ p=P(V(y)=1)=a+(1-a)\epsilon=(1-\epsilon)a+\epsilon, $$ and the false-positive probability among verifier-passing trajectories is $$ \Phi(a)=P(R(y)=0 \mid V(y)=1)=\frac{(1-a)\epsilon}{a+(1-a)\epsilon}. $$ Since $$ \frac{d\Phi}{da}=-\frac{\epsilon}{(a+(1-a)\epsilon)^2}<0, $$ false-positive prevalence decreases with capability. However, BoN changes which prompts contribute positives. The probability of obtaining at least one verifier pass under budget $N$ is $$ P_N=1-(1-p)^N. $$ As $N$ grows, prompts with low empirical pass rate begin to produce at least one passing sample; these marginal positives are increasingly likely to be spurious. Using the MLE posterior correction $$ \hat a=\frac{p-\epsilon}{1-\epsilon}, $$ projected to $[0,1]$, the analysis shows that for marginal tasks passing only when the budget increases from $N-1$ to $N$, one has $p \approx 1/N$ and $\hat a \to 0$ as $N \to \infty$, implying $\lim_{N\to\infty}\Phi(\hat a)=1$. A Bayesian refinement further states that for a marginal task producing exactly one pass at attempt $N$, the posterior expected false-positive rate increases strictly with $N$. The second pathology is distribution shift. Even with a perfect verifier, the KL divergence lower bound between a target policy with success at least $n$ and a reference policy with success rate $a$ satisfies $$ D_{\mathrm{KL}}(T_{\mathrm{new}} \Vert T_{\mathrm{ref}}) \ge n \log(n/a)+(1-n)\log((1-n)/(1-a)). $$ Under BoN with marginal $\hat a \approx 1/N$, the lower bound becomes $$ \mathrm{LB} \approx n\log N - H(n) + O(1/N), $$ where $H(n)=-n\log n-(1-n)\log(1-n)$. The implication is that increasing $N$ drives updates toward tail trajectories and inflates the minimum KL shift roughly as $n\log N$, violating a proximal trust region and destabilizing optimization. PACE replaces brute-force mining with corrective exploration at a minimal effective budget $20$ almost surely for large $M$. In high dimension, under vanishing long-shift autocorrelation, $\hat x^{(t+1)}-\hat x^{(t)} \to 0$ in probability as $M,d \to \infty$, so the procedure converges to the initialization. The data therefore identify iterNA as a positive-feedback loop in which aligned noise becomes correlated with the current estimate. At moderately low SNR, the paper describes “Ghost of Newton.” The update decomposes into aligned signal and aligned noise, $$ \hat x^{(t+1)}= \hat x_{\mathrm{signal}}^{(t+1)}+\hat x_{\mathrm{noise}}^{(t+1)}. $$ Early iterations are dominated by the aligned signal and move toward the true object, but later iterations increasingly align noise to the estimate that already resembles the signal. The result is a structured noise image that looks like a noisy version of the target. When the norms of the aligned-noise and aligned-signal components become comparable, reconstruction fidelity degrades even though the log-likelihood keeps rising. The paper reports a representative case at $\mathrm{SNR}\approx 5\times 10^{-3}$, where EM improves for about $50$ iterations and then diverges in MSE. The proposed remedy is mini-batching. Instead of full-batch EM, the method computes stochastic gradients or EM-like batch averages on small subsets: $$ g^{(t)}=\hat x^{(t)}-\frac{1}{|\mathcal{B}_t|}\sum_{i\in \mathcal{B}_t}\sum_{\ell=0}^{d-1}p_{i,t}^{(\ell)}\,\mathcal{T}_\ell^{-1} y_i, $$ followed by an adaptive optimizer such as ADAM or Ranger. The rationale is that the per-iteration bias of aligned noise scales with the amount of data used in each update. Since Einstein-from-Noise bias scales like $t^2/M$, reducing the effective $M$ per iteration weakens phase-locking and breaks the feedback loop. In experiments with $d=64\times 64$ and $M=2\times 10^4$, full-batch EM over $T=200$ iterations took about $15$ minutes, whereas mini-batch SGD with batch size $256$ over $T=800$ iterations took about $0.5$ minutes, matched EM’s accuracy at moderate and high SNR, and avoided Ghost-of-Newton drift [2505.21435]. This line of work establishes a central iterNA lesson: more averaging and more iterations can intensify bias if the alignment operator is itself noise-sensitive. ## 5. Score-based and diffusion formulations A different usage of iterNA appears in score-based alignment and diffusion generation, where noise is deliberately injected or optimized to regularize inference and improve search. In Active Diffusion Matching (ADM), iterNA is defined as image alignment through a stochastic iterative process that progressively refines transformation parameters under injected noise, guided by learned score functions. The method addresses alignment between Standard Fundus Images and Ultra-Widefield Fundus Images by jointly estimating a global homography $H$ and a dense local deformation field $v$ through two interdependent score-based diffusion models [2604.10084]. The warped source image is $$ \hat I_s(x;H,v)=I_s(W(x;H)+v(x)), $$ and inference is guided by an appearance-plus-regularization energy $$ E(H,v;S,T)=L_x^a(H,v;S,T)+\gamma_H L_R^H(H)+\gamma_v L_R^v(v), $$ where $L_x^a$ is $-\mathrm{NCC}$ between vessel-enhanced warped source and target images, $L_R^H$ penalizes homography deviation from identity, and $L_R^v(v)=\sum_x \|\nabla v(x)\|^2$ enforces local smoothness. ADM trains global and local score networks using denoising score matching with Gaussian perturbations, $$ L_{s_\theta}=\mathbb{E}\!\left[\left\|s_\theta(H_t,t\mid S,T)+\frac{z^H}{\sqrt{1-\bar\alpha_t^H}}\right\|^2\right], $$ $$ L_{s_\phi}=\mathbb{E}\!\left[\left\|s_\phi(v_t,t\mid S,T,H_t)+\frac{z^v}{\sqrt{1-\bar\alpha_t^v}}\right\|^2\right], $$ and performs reverse updates $$ H_{t-1}=\frac{1}{\sqrt{1-\beta_t^H}}H_t+\beta_t^H \hat s_\theta(H_t,t\mid S,T)+\sqrt{\beta_t^H}\,z_t^H, $$ $$ v_{t-1}=\frac{1}{\sqrt{1-\beta_t^v}}v_t+\beta_t^v s_\phi(v_t,t\mid S,T,H_t)+\sqrt{\beta_t^v}\,z_t^v. $$ The global score is modified by input-adaptive guidance, $$ \hat s_\theta(H_t,t\mid S,T)=s_\theta(H_t,t\mid S,T)-g_L \nabla_{H_t}L_x^a(H_t,v_t;S,T), $$ which couples the global and local chains. The reported results show state-of-the-art accuracy. On the private KBSMC SFI–UWFI dataset, ADM achieves Acceptable $41.98\%$ and mAUC $29.3$, improving mAUC by $5.2$ points over GeoFormer’s $24.1$ and Acceptable rate by $5.88$ percentage points over $36.10\%$. On FIRE, ADM reaches Acceptable $98.51\%$ and mAUC $76.0$, which is $0.4$ above GeoFormer’s $75.6$. Inference cost is about $47.12$ seconds per pair and $1.2$ GB memory on a single RTX 4090 for $768\times 768$ images [2604.10084]. The same broad logic appears in text-to-image diffusion, but here the target of alignment is the initial latent noise rather than a geometric transform. InitNO formalizes the observation that not all initial noise samples are equally compatible with a text prompt. It defines a cross-attention response score $$ S_{\mathrm{CrossAttn}}(z_T)=1-\min_{y_i\in Y} r_i(z_T), \qquad r_i(z_T)=\max_{x,y} A^\mathrm{cross}_{y_i}[x,y], $$ and a self-attention conflict score $$ S_{\mathrm{SelfAttn}}(z_T)=\frac{1}{N}\sum_{i