VisAligner: Visual Signal Alignment

• VisAligner is a design pattern that constrains visual backbones with task-specific alignment signals to bridge visual representations and external structures.
• It employs dual supervision by combining primary and auxiliary streams to ensure both sequence-level correctness and local feature fidelity.
• Empirical results demonstrate significant performance gains across applications such as sign language recognition, photorealistic translation, and vision-language modeling.

VisAligner denotes a family of mechanisms that align visual signals with external structure. In the cited literature, the name appears both as the explicit designation of a Sim2Real appearance module in EmbodieDreamer and as a descriptive label for methods that supervise visual features against sequence labels, language embeddings, geometric relations, clinical treatment plans, or human perceptual judgments (Wang et al., 7 Jul 2025, Min et al., 2021, Masry et al., 3 Feb 2025, Gatenyo et al., 20 Jan 2026, Shen et al., 2022, Lee et al., 2023). Across these settings, the shared objective is explicit correspondence rather than generic representation learning: between frames and glosses, simulator states and photorealistic videos, visual patches and token embeddings, meshes and language-described poses, rendered teeth and orthodontic plans, or model outputs and human abstention behavior.

1. Terminological scope and recurrent formulation

In the cited work, VisAligner is not a single standardized architecture. It instead names, or plausibly names, a recurring design pattern: a visual backbone or renderer is constrained by an auxiliary alignment signal that is closer to the task semantics than unconstrained end-to-end supervision alone. This suggests a unifying interpretation in which a VisAligner is a module that narrows the gap between a visual intermediate representation and a target structure that is otherwise only weakly supervised.

Context	Alignment target	Representative formulation
Continuous sign language recognition	Visual features, gloss sequence, and alignment module predictions	Visual Alignment Constraint with $$\mathcal{L}=\mathcal{L}_{CTC}+\mathcal{L}_{VE}+\alpha \mathcal{L}_{VA}$$ (Min et al., 2021)
Embodied Real2Sim2Real transfer	Low-fidelity simulator renderings and real camera appearance	Conditional latent video diffusion for Sim2Real appearance translation (Wang et al., 7 Jul 2025)
Vision-language modeling	Visual patch features and the LLM text embedding manifold	Weighted average of LLM token embeddings via vocabulary softmax (Masry et al., 3 Feb 2025)
Zero-shot 3D object alignment	Relative pose of meshes and language-described spatial relations	CLIP-guided test-time optimization with soft-ICP and penetration losses (Gatenyo et al., 20 Jan 2026)
Orthodontic visualization	3D staged tooth geometry and frontal photographic appearance	Differentiable rendering plus multi-modal StyleGAN2+UNet generation (Shen et al., 2022)
Alignment evaluation	Model output distributions and human perceptual judgments	Hellinger distance and abstention-aware reliability scoring (Lee et al., 2023)

A recurrent technical motif is dual supervision. One stream models the full task with strong context or generative capacity; another stream exposes a more local, interpretable, or human-referenced signal. The alignment mechanism then constrains the visual representation so that the high-capacity component does not dominate in ways that weaken robustness or fidelity.

2. Visual Alignment Constraint in continuous sign language recognition

In continuous sign language recognition, the most explicit early formulation of a VisAligner-like mechanism is the Visual Alignment Constraint, introduced for vision-based CSLR under CTC training (Min et al., 2021). The setting uses an unsegmented frame sequence

$$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$

and predicts a gloss sequence

$$\boldsymbol{l} = (l_1,\dots,l_N),\quad l_i \in \mathbb{G}.$$

A standard pipeline comprises a feature extractor producing

$$\boldsymbol{V} = (\boldsymbol{v}_1,\dots,\boldsymbol{v}_{T'})$$

and an alignment module, typically a two-layer BiLSTM with hidden size 512 per direction, producing logits

$$\boldsymbol{Z} = (\boldsymbol{z}_1,\dots,\boldsymbol{z}_{T'}),\quad \boldsymbol{z}_t \in \mathbb{R}^{|\mathbb{G}'|}.$$

The core diagnosis is that CTC induces spiky posteriors dominated by blank predictions outside a small set of discriminative frames. In low-data CSLR regimes, the BiLSTM alignment module can overfit quickly: it learns correct spikes from partial visual cues plus context, gradients to the visual backbone become weak and sparse, and the feature extractor remains undertrained. Earlier CSLR systems mitigated this with iterative training, pseudo labels, freezing or down-weighting temporal modules, and repeated fine-tuning of the CNN. VAC aims to replicate the beneficial effect of that procedure within a single end-to-end optimization.

VAC adds an auxiliary classifier $F_a$ directly on $\boldsymbol{V}$, producing

$$\tilde{\boldsymbol{Z}} = (\tilde{\boldsymbol{z}}_1,\dots,\tilde{\boldsymbol{z}}_{T'}) = F_a(\boldsymbol{V}),$$

alongside the primary BiLSTM-based classifier $F_p$. The total objective is

$$\mathcal{L} = \mathcal{L}_{CTC} + \mathcal{L}_{VE} + \alpha \mathcal{L}_{VA},$$

with default $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$0. The Visual Enhancement loss $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$1 is a CTC loss on $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$2, forcing the feature extractor and auxiliary head to support sequence recognition using the same sentence-level gloss supervision. The Visual Alignment loss $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$3 is a KL divergence between softened primary and auxiliary predictions,

$$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$4

where $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$5, $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$6, and a typical choice is $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$7. The intended effect is twofold: sequence-level correctness from $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$8 and framewise alignment to the context-aware spike structure learned by the BiLSTM from $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$9.

The paper also introduces overfitting diagnostics based on prediction inconsistency between the primary and auxiliary streams. Word Deterioration Rate measures cases in which the auxiliary classifier is correct but the primary stream overrides it incorrectly. Word Amelioration Rate measures the reverse, when the primary stream corrects an auxiliary error. Their modified WER relation is

$$\boldsymbol{l} = (l_1,\dots,l_N),\quad l_i \in \mathbb{G}.$$0

This reframes overfitting as a measurable conflict between visual evidence and temporal context.

Empirically, VAC makes CSLR networks end-to-end trainable and substantially improves performance. On RWTH-PHOENIX-Weather 2014, a baseline end-to-end model with ResNet18, BiLSTM, CTC, and BatchNorm reports Dev WER $$\boldsymbol{l} = (l_1,\dots,l_N),\quad l_i \in \mathbb{G}.$$1 and Test WER $$\boldsymbol{l} = (l_1,\dots,l_N),\quad l_i \in \mathbb{G}.$$2; iterative training improves this to Dev $$\boldsymbol{l} = (l_1,\dots,l_N),\quad l_i \in \mathbb{G}.$$3, Test $$\boldsymbol{l} = (l_1,\dots,l_N),\quad l_i \in \mathbb{G}.$$4 after 3 iterations; Baseline + VE reaches Dev $$\boldsymbol{l} = (l_1,\dots,l_N),\quad l_i \in \mathbb{G}.$$5, Test $$\boldsymbol{l} = (l_1,\dots,l_N),\quad l_i \in \mathbb{G}.$$6; Baseline + VA reaches Dev $$\boldsymbol{l} = (l_1,\dots,l_N),\quad l_i \in \mathbb{G}.$$7, Test $$\boldsymbol{l} = (l_1,\dots,l_N),\quad l_i \in \mathbb{G}.$$8; and Baseline + VE + VA reaches Dev $$\boldsymbol{l} = (l_1,\dots,l_N),\quad l_i \in \mathbb{G}.$$9, Test $$\boldsymbol{V} = (\boldsymbol{v}_1,\dots,\boldsymbol{v}_{T'})$$0. On CSL, the baseline reports WER $$\boldsymbol{V} = (\boldsymbol{v}_1,\dots,\boldsymbol{v}_{T'})$$1 and Baseline + VAC reports $$\boldsymbol{V} = (\boldsymbol{v}_1,\dots,\boldsymbol{v}_{T'})$$2. The ablation data show complementarity: VE alone gives $$\boldsymbol{V} = (\boldsymbol{v}_1,\dots,\boldsymbol{v}_{T'})$$3 absolute Dev improvement, VA alone $$\boldsymbol{V} = (\boldsymbol{v}_1,\dots,\boldsymbol{v}_{T'})$$4, and both together $$\boldsymbol{V} = (\boldsymbol{v}_1,\dots,\boldsymbol{v}_{T'})$$5. BatchNorm after temporal convolutions is reported to help significantly, revealing that the feature extractor had otherwise been undertrained.

3. VisAligner as Sim2Real appearance translator in embodied world models

In EmbodieDreamer, VisAligner is an explicit module for reducing the Sim2Real appearance gap within a Real2Sim2Real pipeline (Wang et al., 7 Jul 2025). PhysAligner addresses the Real2Sim physical gap by fitting robot-specific parameters such as control gains and friction coefficients, whereas VisAligner attacks the Sim2Real appearance gap by translating low-fidelity simulated renderings into photorealistic RGB videos conditioned on the true simulation state.

The training data are paired. For each manually collected real episode $$\boldsymbol{V} = (\boldsymbol{v}_1,\dots,\boldsymbol{v}_{T'})$$6 of length $$\boldsymbol{V} = (\boldsymbol{v}_1,\dots,\boldsymbol{v}_{T'})$$7, joint positions and actions satisfy

$$\boldsymbol{V} = (\boldsymbol{v}_1,\dots,\boldsymbol{v}_{T'})$$8

and the ground-truth video is

$$\boldsymbol{V} = (\boldsymbol{v}_1,\dots,\boldsymbol{v}_{T'})$$9

with $$\boldsymbol{Z} = (\boldsymbol{z}_1,\dots,\boldsymbol{z}_{T'}),\quad \boldsymbol{z}_t \in \mathbb{R}^{|\mathbb{G}'|}.$$0, $$\boldsymbol{Z} = (\boldsymbol{z}_1,\dots,\boldsymbol{z}_{T'}),\quad \boldsymbol{z}_t \in \mathbb{R}^{|\mathbb{G}'|}.$$1, and $$\boldsymbol{Z} = (\boldsymbol{z}_1,\dots,\boldsymbol{z}_{T'}),\quad \boldsymbol{z}_t \in \mathbb{R}^{|\mathbb{G}'|}.$$2. VisAligner conditions on three explicit scene components. The robot is rendered from URDF $$\boldsymbol{Z} = (\boldsymbol{z}_1,\dots,\boldsymbol{z}_{T'}),\quad \boldsymbol{z}_t \in \mathbb{R}^{|\mathbb{G}'|}.$$3 under a manually calibrated virtual camera by replaying $$\boldsymbol{Z} = (\boldsymbol{z}_1,\dots,\boldsymbol{z}_{T'}),\quad \boldsymbol{z}_t \in \mathbb{R}^{|\mathbb{G}'|}.$$4,

$$\boldsymbol{Z} = (\boldsymbol{z}_1,\dots,\boldsymbol{z}_{T'}),\quad \boldsymbol{z}_t \in \mathbb{R}^{|\mathbb{G}'|}.$$5

The foreground object is obtained from Grounded-SAM, yielding

$$\boldsymbol{Z} = (\boldsymbol{z}_1,\dots,\boldsymbol{z}_{T'}),\quad \boldsymbol{z}_t \in \mathbb{R}^{|\mathbb{G}'|}.$$6

A simulation condition signal is then formed by pixel-wise addition,

$$\boldsymbol{Z} = (\boldsymbol{z}_1,\dots,\boldsymbol{z}_{T'}),\quad \boldsymbol{z}_t \in \mathbb{R}^{|\mathbb{G}'|}.$$7

Because the camera is static, a single reference real image

$$\boldsymbol{Z} = (\boldsymbol{z}_1,\dots,\boldsymbol{z}_{T'}),\quad \boldsymbol{z}_t \in \mathbb{R}^{|\mathbb{G}'|}.$$8

is used for the background. The condition sequence is

$$\boldsymbol{Z} = (\boldsymbol{z}_1,\dots,\boldsymbol{z}_{T'}),\quad \boldsymbol{z}_t \in \mathbb{R}^{|\mathbb{G}'|}.$$9

VisAligner is implemented as a conditional latent video diffusion model with a Stable Video Diffusion–style backbone. A video VAE encoder $F_a$0 maps the conditioning video and target real video into latent sequences,

$F_a$1

where the latent channel dimension is explicitly $F_a$2. The forward diffusion process is

$F_a$3

and the denoiser $F_a$4 is a video U-Net with spatiotemporal attention. It is trained with

$F_a$5

Conditioning latents are concatenated along the channel dimension with noisy latents, and temporal consistency is induced by operating on full latent video volumes and applying spatiotemporal attention across frames.

This appearance model is not merely an offline renderer. During reinforcement learning, the policy acts in the aligned physics simulator, raw simulator observations are translated by VisAligner, and the last frame of the generated photorealistic video is fed back as the next observation. For imitation learning and data augmentation, edited reference images can be replayed in simulation and translated into new photorealistic demonstrations.

On the RT-1 test set, the paper compares an SVD-style baseline without object segmentation masks in the conditioning against full VisAligner. The baseline reports FVD $F_a$6, PSNR $F_a$7, SSIM $F_a$8, and LPIPS $F_a$9; VisAligner reports FVD $\boldsymbol{V}$0, PSNR $\boldsymbol{V}$1, SSIM $\boldsymbol{V}$2, and LPIPS $\boldsymbol{V}$3. The qualitative interpretation given in the paper is that explicit segmentation conditioning prevents manipulated objects from deforming or disappearing. In downstream policy learning, training in the EmbodieDreamer world yields an average real-world task success of $\boldsymbol{V}$4, compared with $\boldsymbol{V}$5 for ACT SFT with 3 views and $\boldsymbol{V}$6 for ACT SFT with 1 view, corresponding to the reported $\boldsymbol{V}$7 average improvement after reinforcement learning. The principal limitation is computational: video diffusion generation is reported to take about 2 minutes per sequence.

4. Latent-space alignment between vision and language

A second major sense of VisAligner is conceptual rather than nominal: a module that aligns visual features with a pretrained language embedding manifold. AlignVLM provides a clear exemplar of this pattern (Masry et al., 3 Feb 2025). The central problem is that standard shallow connectors, such as MLPs or Perceiver-style modules, may project vision features into out-of-distribution regions of the LLM input space. AlignVLM instead maps each visual feature to a weighted average of existing LLM text embeddings.

The architecture consists of a SigLip-400M vision encoder, an alignment connector, and an LLM. To support high-resolution arbitrary-aspect-ratio documents, images are split into tiles with predefined aspect ratios $\boldsymbol{V}$8, with at most 9 tiles. Each tile is decomposed into $\boldsymbol{V}$9 patches, and the encoder outputs contextual patch features $$\tilde{\boldsymbol{Z}} = (\tilde{\boldsymbol{z}}_1,\dots,\tilde{\boldsymbol{z}}_{T'}) = F_a(\boldsymbol{V}),$$0. Across tiles, these features are concatenated into

$$\tilde{\boldsymbol{Z}} = (\tilde{\boldsymbol{z}}_1,\dots,\tilde{\boldsymbol{z}}_{T'}) = F_a(\boldsymbol{V}),$$1

Let $$\tilde{\boldsymbol{Z}} = (\tilde{\boldsymbol{z}}_1,\dots,\tilde{\boldsymbol{z}}_{T'}) = F_a(\boldsymbol{V}),$$2 denote the LLM input embedding matrix. The connector first applies a linear projection $$\tilde{\boldsymbol{Z}} = (\tilde{\boldsymbol{z}}_1,\dots,\tilde{\boldsymbol{z}}_{T'}) = F_a(\boldsymbol{V}),$$3, then a vocabulary projection $$\tilde{\boldsymbol{Z}} = (\tilde{\boldsymbol{z}}_1,\dots,\tilde{\boldsymbol{z}}_{T'}) = F_a(\boldsymbol{V}),$$4, initialized from the LM head, to produce a per-patch distribution over vocabulary items: $$\tilde{\boldsymbol{Z}} = (\tilde{\boldsymbol{z}}_1,\dots,\tilde{\boldsymbol{z}}_{T'}) = F_a(\boldsymbol{V}),$$5 The aligned visual representation is then

$$\tilde{\boldsymbol{Z}} = (\tilde{\boldsymbol{z}}_1,\dots,\tilde{\boldsymbol{z}}_{T'}) = F_a(\boldsymbol{V}),$$6

so each patch embedding becomes a convex combination of rows of $$\tilde{\boldsymbol{Z}} = (\tilde{\boldsymbol{z}}_1,\dots,\tilde{\boldsymbol{z}}_{T'}) = F_a(\boldsymbol{V}),$$7. This yields an explicit in-distribution constraint: the visual token lies in the convex hull of the LLM vocabulary embedding matrix rather than in an arbitrary region of $$\tilde{\boldsymbol{Z}} = (\tilde{\boldsymbol{z}}_1,\dots,\tilde{\boldsymbol{z}}_{T'}) = F_a(\boldsymbol{V}),$$8.

Training proceeds in three stages: generic multimodal alignment on CC-12M, document understanding pretraining on BigDocs-7.5M, and instruction tuning on DocDownstream. In Stages 1 and 2 the full model is trainable; in Stage 3 the vision encoder is frozen while the connector and LLM remain trainable. The optimization objective is standard autoregressive cross-entropy on target text.

The connector ablation on a Llama-3.2-3B backbone reports average scores over 9 document benchmarks of $$\tilde{\boldsymbol{Z}} = (\tilde{\boldsymbol{z}}_1,\dots,\tilde{\boldsymbol{z}}_{T'}) = F_a(\boldsymbol{V}),$$9 for MLP, $F_p$0 for Perceiver Resampler, $F_p$1 for Ovis, and $F_p$2 for the proposed connector. The full benchmark reports averages of $F_p$3 for the 1B model, $F_p$4 for the 3B model, and $F_p$5 for the 8B model. The robustness analysis is especially revealing. After adding Gaussian noise to visual features,

$F_p$6

the Llama-3.2-3B-MLP model drops from $F_p$7 to $F_p$8, whereas the proposed connector drops from $F_p$9 to $$\mathcal{L} = \mathcal{L}_{CTC} + \mathcal{L}_{VE} + \alpha \mathcal{L}_{VA},$$0. The paper interprets this as a regularization effect of constraining outputs to probability distributions over known token embeddings. A noted trade-off is that dense distributions over tokens can bias outputs toward frequent tokens; failure cases include rare proper names being mapped to more common alternatives.

5. Geometric and clinical alignment

Visual alignment also appears in forms where the primary object is not a latent feature but a spatial configuration. In zero-shot 3D object-object alignment, the method of "Copy-Trasform-Paste: Zero-Shot Object-Object Alignment Guided by Vision-Language and Geometric Constraints" directly optimizes the relative pose

$$\mathcal{L} = \mathcal{L}_{CTC} + \mathcal{L}_{VE} + \alpha \mathcal{L}_{VA},$$1

of a source mesh with respect to a target mesh under a text prompt describing the desired relation (Gatenyo et al., 20 Jan 2026). The transformed source vertices satisfy

$$\mathcal{L} = \mathcal{L}_{CTC} + \mathcal{L}_{VE} + \alpha \mathcal{L}_{VA},$$2

Optimization is driven by a differentiable renderer, CLIP image-text similarity,

$$\mathcal{L} = \mathcal{L}_{CTC} + \mathcal{L}_{VE} + \alpha \mathcal{L}_{VA},$$3

a fractional soft-ICP attachment term $$\mathcal{L} = \mathcal{L}_{CTC} + \mathcal{L}_{VE} + \alpha \mathcal{L}_{VA},$$4, and a penetration loss $$\mathcal{L} = \mathcal{L}_{CTC} + \mathcal{L}_{VE} + \alpha \mathcal{L}_{VA},$$5, combined as

$$\mathcal{L} = \mathcal{L}_{CTC} + \mathcal{L}_{VE} + \alpha \mathcal{L}_{VA},$$6

The method uses a phased schedule, 8 camera views per iteration, and a 50-instance benchmark with rigid and scale-enabled variants. In the rigid setting, it reports CLIP $$\mathcal{L} = \mathcal{L}_{CTC} + \mathcal{L}_{VE} + \alpha \mathcal{L}_{VA},$$7, ALIGN $$\mathcal{L} = \mathcal{L}_{CTC} + \mathcal{L}_{VE} + \alpha \mathcal{L}_{VA},$$8, SigLIP $$\mathcal{L} = \mathcal{L}_{CTC} + \mathcal{L}_{VE} + \alpha \mathcal{L}_{VA},$$9, and intersection $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$00, outperforming geometry-only and diffusion-based baselines on semantic measures. In a user study with 47 participants and 15 scenes, the method is selected in $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$01 of trials for best match to the description and $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$02 of trials for physical plausibility. The cited limitations include residual penetration, viewpoint sensitivity for relations such as “left of,” and susceptibility to local minima.

A clinically distinct but structurally related instance appears in orthodontic visualization through OrthoGAN, which synthesizes identity-preserving frontal facial images with aligned teeth from a frontal face image and the patient’s 3D scanned teeth model (Shen et al., 2022). The system takes a frontal facial photograph $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$03, patient-specific 3D staged tooth models $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$04, and the orthodontic planning steps encoded by tooth-wise translations and rotations. It first extracts a mouth crop using YOLOv5 and predicts a teeth silhouette $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$05 and mouth label $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$06 with a UNet-based multi-head network. A custom differentiable renderer then estimates camera parameters $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$07 by minimizing

$$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$08

For each stage $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$09, it renders a silhouette $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$10 and a depth-derived mask $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$11, clips both by $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$12, and feeds them together with the original mouth image $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$13 into OrthoGAN: $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$14 The generator combines StyleGAN2-style synthesis with UNet-like geometry injection. A constant latent in $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$15 models teeth material and illumination, while multi-scale geometry features encode the exact rendered plan. A fusion layer preserves all pixels outside the mouth region, and a post-hoc color transfer in perceptual $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$16 space matches generated tooth color to the original image.

On 263 cases, the differentiable-rendering pose fitting reports movement errors in 10 cases and scaling issues in 40 cases, compared with 30 and 201 under the earlier EM-based method. The generation silhouette error is $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$17 for OrthoGAN and $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$18 for TSynNet. In a pilot study, patients rate simulation authenticity at $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$19 for OrthoGAN, $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$20 for TSynNet, and $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$21 for pSp. Orthodontists rate authenticity at $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$22 for OrthoGAN, $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$23 for TSynNet, and $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$24 for pSp. The method is frontal-only, depends on 3D teeth scans and staged treatment models, and does not model facial change beyond the mouth region.

6. Human-centered evaluation and recurring limitations

The VisAlign benchmark reframes visual alignment as agreement with human perception in image classification with abstention (Lee et al., 2023). It evaluates 10 mammal classes plus abstention across three groups: Must-Act, Must-Abstain, and Uncertain. The open test set contains 900 images: 100 each in Categories 1–7 and 200 in Category 8. Category 8, which represents cropped and corrupted in-class images, is annotated with 134 MTurk responses per image, yielding an empirical human distribution over 11 outcomes. The paper reports intra-annotator Fleiss’ $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$25, inter-annotator $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$26, and Cronbach’s alpha $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$27.

Visual alignment is measured by Hellinger distance between the human distribution $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$28 and the model distribution $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$29,

$$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$30

where $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$31 includes 10 softmax probabilities and an abstention probability from an abstention method. Reliability is summarized by an abstention-aware score $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$32, where correct action on Must-Act or Must-Abstain yields $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$33, abstention on Must-Act yields $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$34, and incorrect action yields $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$35. The main empirical conclusion is that no current method performs well across all categories, abstention method matters more than architecture, and lower mean Hellinger distance correlates strongly with higher $$\boldsymbol{X} = (\boldsymbol{x}_1,\dots,\boldsymbol{x}_T)$$36. Distance-based methods such as KNN, Mahalanobis detector, and TAPUDD are stronger on Must-Act but weak on Must-Abstain; probability-based methods such as SP, ASP, MC-Dropout, and OpenMax show the reverse trade-off.

Viewed across the broader VisAligner literature, several recurring limitations appear. VAC leverages the spike behavior of CTC, so transfer to non-CTC aligners remains to be validated (Min et al., 2021). Embodied video diffusion is slow and assumes accurate simulator geometry and camera calibration, with static-camera scenes and limited object complexity (Wang et al., 7 Jul 2025). Convex-hull connectors in vision-language modeling can bias outputs toward frequent tokens (Masry et al., 3 Feb 2025). CLIP-guided 3D pose optimization remains viewpoint-sensitive and susceptible to local minima (Gatenyo et al., 20 Jan 2026). Orthodontic visualization currently requires 3D scans and supports frontal smiles only (Shen et al., 2022). The VisAlign benchmark itself is restricted to 10 mammal classes and synthetic uncertainty regimes (Lee et al., 2023).

These constraints suggest a common research direction rather than a single mature solution. A plausible implication is that future VisAligner systems will increasingly combine explicit structural conditioning, auxiliary or distillation-style supervision, calibrated abstention, and human-referenced evaluation. The literature surveyed here already exhibits that trajectory: visual alignment is treated not as a by-product of end-to-end optimization, but as a first-class design objective.