TM-ARC: Diffusion Hallucination Controller

• TM-ARC is a closed-loop latent-space controller that mitigates hallucinations by monitoring and correcting trajectory drift in a three-axis latent space.
• It defines hallucination as misregulated tension across Semantic Coherence, Structural Alignment, and Knowledge Grounding via the Alignment Risk Code (ARC).
• Empirical results show that TM-ARC improves image quality metrics while retaining diversity, demonstrating its practical significance in prompt-faithful diffusion sampling.

Searching arXiv for the TM-ARC paper and closely related text-to-image hallucination control work. TensionModulator, abbreviated TM-ARC, is a closed-loop, latent-space controller for reducing hallucinations in text-to-image diffusion models. It is introduced together with the Hallucination Tri-Space and the Alignment Risk Code (ARC), which recast hallucination not as a purely random artifact or an isolated attention failure, but as trajectory drift in a prompt-conditioned latent alignment process (Yang et al., 7 Jul 2025). In this formulation, generation evolves through a three-axis latent space spanning Semantic Coherence (SC), Structural Alignment (SA), and Knowledge Grounding (KG), while TM-ARC monitors the evolving alignment tension and applies axis-specific interventions during sampling so that the latent trajectory remains near the prompt-aligned manifold (Yang et al., 7 Jul 2025).

1. Conceptual definition and modeling assumptions

TM-ARC is defined in the source paper as a lightweight controller that operates entirely in latent space and is intended to mitigate hallucinations during diffusion sampling without retraining the diffusion backbone (Yang et al., 7 Jul 2025). The motivating premise is that text-to-image hallucination reflects misregulated tension across multiple alignment objectives rather than a single failure mode. This leads to a dynamic systems view in which denoising is a traversal

$$z_0 \rightarrow z_1 \rightarrow \dots \rightarrow z_T,$$

and hallucination occurs when that trajectory drifts away from the ideal prompt-aligned manifold $\mathcal{M}_{\text{ideal}}$ under multiaxial tension (Yang et al., 7 Jul 2025).

The paper formalizes the latent environment as the Hallucination Tri-Space $\mathcal{T}^3$, whose three orthogonal axes are:

• Semantic Coherence (SC): whether the generated content preserves intended object identity and meaning.
• Structural Alignment (SA): whether spatial layout, compositional relations, and object positions are preserved.
• Knowledge Grounding (KG): whether content is commonsensically or factually plausible (Yang et al., 7 Jul 2025).

This tri-axial decomposition is central to TM-ARC because the controller does not treat hallucination as a scalar error. Instead, it treats hallucination risk as a directional and anisotropic state in a latent alignment space. A plausible implication is that the framework is designed to distinguish failures that may look similar in the final image but arise from different internal instability patterns.

2. Alignment Risk Code and the tri-space geometry

The diagnostic core of TM-ARC is the Alignment Risk Code (ARC), a dynamic three-dimensional vector: $$\vec{\tau}(p, t) = \left[ \tau_{SC}(p, t),\; \tau_{SA}(p, t),\; \tau_{KG}(p, t) \right]^\mathsf{T} \in \mathbb{R}^3.$$ The manuscript also presents an earlier broken-format rendering of the same object, but the intended meaning is the same: at denoising step $t$ for prompt $p$, each component quantifies the instantaneous alignment tension along one of the three axes (Yang et al., 7 Jul 2025).

The axis-specific tension is defined as a gradient norm: $$\tau_i(p, t) = \left\| \nabla_{z_t} \mathcal{A}_i(z_t, p) \right\|,$$ where $\mathcal{A}_i(z_t, p)$ is a scalar potential measuring alignment with objective $i$ (Yang et al., 7 Jul 2025). In the paper’s interpretation, larger $\tau_i$ indicates stronger restorative pressure in that subspace.

ARC is explicitly described as having three interpretable properties. First, its magnitude

$\mathcal{M}_{\text{ideal}}$0

measures overall alignment stress. Second, its direction identifies the dominant failure axis; the paper also summarizes directionality through

$\mathcal{M}_{\text{ideal}}$1

described as a tension skew or probability-like attribution of hallucination directionality. Third, its imbalance or anisotropy is measured by

$\mathcal{M}_{\text{ideal}}$2

with high variance interpreted as strong axis-specific hallucination risk (Yang et al., 7 Jul 2025).

The paper further gives a threshold-style risk criterion: $\mathcal{M}_{\text{ideal}}$3 Under this condition, hallucination risk is considered high either because total tension is large or because the tension distribution is excessively skewed (Yang et al., 7 Jul 2025). This suggests that the framework separates two distinct failure regimes: globally unstable sampling and axis-dominated drift.

3. Controller architecture and latent-space intervention

TM-ARC converts ARC from a descriptive state variable into a feedback controller. The control loop in the paper has four stages: monitoring, risk assessment, axis identification, and correction (Yang et al., 7 Jul 2025).

At each denoising step $\mathcal{M}_{\text{ideal}}$4, TM-ARC estimates the three ARC components $\mathcal{M}_{\text{ideal}}$5, $\mathcal{M}_{\text{ideal}}$6, and $\mathcal{M}_{\text{ideal}}$7. It then evaluates the total magnitude $\mathcal{M}_{\text{ideal}}$8, the imbalance $\mathcal{M}_{\text{ideal}}$9, and the relative dominance of components. The dominant component is interpreted as the active drift mode: high $\mathcal{T}^3$0 for semantic drift, high $\mathcal{T}^3$1 for structural distortion, and high $\mathcal{T}^3$2 for factual or commonsense grounding failure (Yang et al., 7 Jul 2025).

The intervention layer consists of three specialized submodules:

• SC-Gate: mitigates semantic drift by reactivating attention to prompt-critical entities.
• SA-Tuner: corrects spatial encodings and positional relations to restore structure.
• KG-Augment: injects or reweights grounding information to improve factual plausibility (Yang et al., 7 Jul 2025).

For KG-Augment, the paper describes two modes. In static injection, KG-related prompt embeddings are prepended to the text encoder input. In dynamic modulation, cross-attention layers are reweighted according to $\mathcal{T}^3$3 (Yang et al., 7 Jul 2025).

The latent update is given as a gated composite correction: $\mathcal{T}^3$4 The paper states that $\mathcal{T}^3$5 is a tension-adaptive scaling function and $\mathcal{T}^3$6 is the correction operator for axis $\mathcal{T}^3$7. The intervention therefore grows stronger when total tension is high (Yang et al., 7 Jul 2025). Conceptually, the controller acts as a damping mechanism: low-tension trajectories are perturbed minimally, whereas high-tension or highly anisotropic trajectories are nudged more strongly toward $\mathcal{T}^3$8.

The drift model behind this design is written as

$\mathcal{T}^3$9

with

$$\vec{\tau}(p, t) = \left[ \tau_{SC}(p, t),\; \tau_{SA}(p, t),\; \tau_{KG}(p, t) \right]^\mathsf{T} \in \mathbb{R}^3.$$0

Here, $$\vec{\tau}(p, t) = \left[ \tau_{SC}(p, t),\; \tau_{SA}(p, t),\; \tau_{KG}(p, t) \right]^\mathsf{T} \in \mathbb{R}^3.$$1 is a learned mapping from ARC to perturbation strength, $$\vec{\tau}(p, t) = \left[ \tau_{SC}(p, t),\; \tau_{SA}(p, t),\; \tau_{KG}(p, t) \right]^\mathsf{T} \in \mathbb{R}^3.$$2 and $$\vec{\tau}(p, t) = \left[ \tau_{SC}(p, t),\; \tau_{SA}(p, t),\; \tau_{KG}(p, t) \right]^\mathsf{T} \in \mathbb{R}^3.$$3 are sensitivity coefficients, and $$\vec{\tau}(p, t) = \left[ \tau_{SC}(p, t),\; \tau_{SA}(p, t),\; \tau_{KG}(p, t) \right]^\mathsf{T} \in \mathbb{R}^3.$$4 is a directional deviation term (Yang et al., 7 Jul 2025). Within the paper’s framing, hallucinations are therefore cumulative biased drifts produced by sustained multiaxial tension rather than abrupt isolated glitches.

4. Empirical evidence and benchmark behavior

The paper motivates TM-ARC with a synthetic experiment intended to show that hallucinations have structured geometry even in a highly controlled setting. It constructs 1000 prompt-image pairs with deterministic mappings, where prompts describe two geometric objects with fixed shape, color, and position. A standard unconditional DDPM is trained on this dataset, yet 27.8\% of generated samples still violate the prompt (Yang et al., 7 Jul 2025).

These violations are reported to cluster into three types: shape/color/identity errors, position/layout swaps, and factual/commonsense-like mismatch analogues. Using a 3D deviation representation, the authors report clustering results that outperform PCA and random 3D mapping:

Representation	Silhouette / Calinski-Harabasz / Davies-Bouldin
Alignment-feature representation	0.63 / 1480.5 / 0.84
PCA	0.41 / 752.3 / 1.37
Random 3D mapping	0.21 / 389.1 / 2.10

These results are presented as evidence that hallucinations organize into stable axis-dependent patterns rather than arising as unstructured noise (Yang et al., 7 Jul 2025).

The main evaluation uses DrawBench and Pick-a-Pic with four diffusion backbones: Stable Diffusion XL (SDXL), Stable Diffusion 1.5 (SD1.5), PixArt-sigma, and Hunyuan-DiT. Baselines are Vanilla sampling, Prompt-to-Prompt, Attend-and-Excite, and Zigzag Diffusion Sampling. The reported metrics are CLIPScore $$\vec{\tau}(p, t) = \left[ \tau_{SC}(p, t),\; \tau_{SA}(p, t),\; \tau_{KG}(p, t) \right]^\mathsf{T} \in \mathbb{R}^3.$$5, PickScore $$\vec{\tau}(p, t) = \left[ \tau_{SC}(p, t),\; \tau_{SA}(p, t),\; \tau_{KG}(p, t) \right]^\mathsf{T} \in \mathbb{R}^3.$$6, ImageReward $$\vec{\tau}(p, t) = \left[ \tau_{SC}(p, t),\; \tau_{SA}(p, t),\; \tau_{KG}(p, t) \right]^\mathsf{T} \in \mathbb{R}^3.$$7, and FID $$\vec{\tau}(p, t) = \left[ \tau_{SC}(p, t),\; \tau_{SA}(p, t),\; \tau_{KG}(p, t) \right]^\mathsf{T} \in \mathbb{R}^3.$$8 (Yang et al., 7 Jul 2025).

A compact summary of representative results is as follows:

Setting	Vanilla	ARC
DrawBench – SDXL	28.3 / 20.9 / 0.77 / 47.5	29.3 / 21.6 / 0.85 / 20.0
DrawBench – Hunyuan-DiT	29.1 / 21.7 / 0.80 / 57.0	29.4 / 22.6 / 0.81 / 18.2
Pick-a-Pic – SDXL	29.4 / 21.0 / 0.71 / 41.5	29.4 / 22.0 / 0.90 / 21.5
Pick-a-Pic – Hunyuan-DiT	27.8 / 22.9 / 0.78 / 27.0	29.1 / 22.7 / 0.83 / 18.8

The paper further states that ARC delivers the best PickScore in 6 of 8 settings and the lowest FID in 7 of 8 settings (Yang et al., 7 Jul 2025). The authors interpret these results as showing that semantic alignment can be improved while maintaining strong image quality. Their stated conclusion is that TM-ARC reduces hallucination without sacrificing diversity or perceptual quality (Yang et al., 7 Jul 2025).

5. Controllability, interpretability, and operational significance

A notable feature of TM-ARC is that the controller is presented as interpretable at the level of its state variables. Because ARC is a three-component code rather than a scalar score, the system can report whether an observed risk pattern is predominantly semantic, structural, or grounding-related (Yang et al., 7 Jul 2025). This is operationally important because the correction module is not monolithic: each axis is associated with a dedicated intervention.

The paper includes prompt-level examples to illustrate this control behavior. For “Flying Elephant”, ARC changes from $$\vec{\tau}(p, t) = \left[ \tau_{SC}(p, t),\; \tau_{SA}(p, t),\; \tau_{KG}(p, t) \right]^\mathsf{T} \in \mathbb{R}^3.$$9 to $t$0, with a reported faithfulness gain of +14.3. For “Red Triangle”, ARC changes from $t$1 to $t$2, with a faithfulness gain of +9.7 (Yang et al., 7 Jul 2025). In both examples, the dominant component is suppressed while the other components remain comparatively balanced. This supports the intended interpretation of TM-ARC as a targeted stabilizer rather than a uniform perturbation mechanism.

The paper also positions TM-ARC against several broad classes of hallucination mitigation methods, namely attention refocusing, post-hoc correction, retrieval augmentation, and isolated condition-misalignment fixes (Yang et al., 7 Jul 2025). Its claim is not that these methods are invalid, but that they often address symptoms rather than the underlying generative dynamics. TM-ARC is proposed as an alternative centered on online detection of multiaxial tension and preemptive intervention during generation.

A common misconception would be to treat TM-ARC as merely another prompt-conditioning heuristic. The paper’s own presentation argues for a narrower and more technical characterization: it is a plug-and-play latent-space modulation mechanism whose control variable is a real-time estimate of axis-specific alignment stress (Yang et al., 7 Jul 2025). Another plausible implication is that its interpretability derives less from explicit symbolic reasoning than from the low-dimensional organization imposed by the tri-space representation.

6. Scope, limitations, and open questions

The paper explicitly describes TM-ARC as plug-and-play and architecture-agnostic: it requires no retraining, no architecture modification, and performs latent-space only intervention, with intended generalization across pretrained diffusion backbones (Yang et al., 7 Jul 2025). This suggests a deployment model in which the controller sits outside the backbone and modulates inference rather than training.

At the same time, several limitations are identified in the source. First, the Hallucination Tri-Space and ARC are described as interpretive modeling constructs, and the mapping from diffusion internals to $t$3, $t$4, and $t$5 is not fully mechanistically derived (Yang et al., 7 Jul 2025). Second, TM-ARC is described as lightweight but heuristic: although modular and differentiable, the manuscript does not present a fully rigorous optimization derivation for every correction operator (Yang et al., 7 Jul 2025).

Third, the framework depends on reliable inference-time estimation of latent tension signals, and the paper states that the scalability and robustness of this estimation in more complex or high-resolution settings are not fully proven (Yang et al., 7 Jul 2025). Fourth, the reported experimental scope is limited to standard benchmarks and several popular backbones, without exhaustive exploration of all diffusion families or multimodal settings (Yang et al., 7 Jul 2025). Fifth, while the framework is highly interpretable at the conceptual level, some mathematical notation in the manuscript is approximate or inconsistently formatted, which the source itself suggests may reflect an implementation that is more practically motivated than theoretically formalized (Yang et al., 7 Jul 2025).

Within those constraints, the paper’s stated contribution is a unified view of hallucination as trajectory drift in a three-axis latent alignment space, with TM-ARC serving as the corresponding tension-aware feedback controller (Yang et al., 7 Jul 2025). A plausible implication is that the significance of TM-ARC lies not only in benchmark gains but also in the introduction of a control-theoretic vocabulary for prompt-faithful diffusion sampling: magnitude, direction, imbalance, drift, and axis-specific correction all become explicit objects of analysis.