Visibility Transformer Overview

• Visibility Transformer is a family of methods that explicitly models unseen or partial data by elevating visibility as a primary objective.
• These approaches integrate specialized masking, recurrent memory, and decoupled spatial-temporal attention to reconstruct beyond-field-of-view content and optimize outputs.
• Applications span autonomous driving, LLM interpretability, adversary-aware navigation, and action recognition, showing significant empirical performance gains.

Searching arXiv for papers related to "Visibility Transformer" and the provided identifiers. Visibility Transformer denotes a set of transformer-based formulations in which visibility is treated as a primary modeling target rather than a secondary by-product of representation learning. In recent arXiv usage, the term has been applied to online beyond-field-of-view scene reconstruction for autonomous vehicles, Generative Engine Optimization in LLM-mediated search, visualization of hidden-state and information flow inside decoder-only LLMs, adversary-aware navigation under partial observability, and skeleton action recognition under constrained field of view (Shi et al., 2022, Lüttgenau et al., 3 Jul 2025, Brunello et al., 18 Jul 2025, Chauhan et al., 29 Nov 2025, Dai et al., 1 Jul 2026). Across these settings, “visibility” refers respectively to unseen scene content, citation-grounded prominence in generated answers, interpretability of internal computation, line-of-sight exposure and cover, and joint-level observability under FoV dropout. This suggests an umbrella usage: transformers are being specialized to propagate, quantify, or expose information that is only partially available at inference time.

1. Scope and conceptual unification

The term is not used for a single canonical architecture. Instead, the literature associates it with distinct technical programs that elevate visibility into the objective, masking structure, memory mechanism, or evaluation protocol.

Domain	Visibility target	Representative system
Autonomous driving video	Beyond-FoV scene content from past frames	FlowLens (Shi et al., 2022)
Generative search	How much and how early a source is cited in LLM output	BART-based GEO model (Lüttgenau et al., 3 Jul 2025)
LLM interpretability	Visibility of layerwise token predictions and information flow	InTraVisTo (Brunello et al., 18 Jul 2025)
Adversary-aware navigation	Exposure, cover, and LoS under partial observability	HAVEN with DTQN (Chauhan et al., 29 Nov 2025)
Skeleton action recognition	Joint observability under constrained FoV	PartialVisGraph (Dai et al., 1 Jul 2026)

A plausible implication is that “visibility transformer” functions more as a design orientation than as a fixed macro-architecture. In some papers, visibility is reconstructed from past observations; in others, it is optimized as salience in generated text, or injected directly as a prior into attention logits. The shared principle is explicit treatment of incomplete, hidden, delayed, or selectively observable information.

2. Beyond-field-of-view scene reconstruction

In autonomous driving, FlowLens operationalizes a visibility transformer by reconstructing regions outside the current camera FoV using only past video, under an online constraint that prohibits access to future frames (Shi et al., 2022). The method frames the problem as online video inpainting for autonomous vehicles and couples explicit local motion propagation with implicit global clip-recurrent attention. Its architecture has four stages: a convolutional stem over Local Frames and Past Reference Frames, explicit flow-guided feature propagation, a Clip-Recurrent Transformer, and an output convolutional decoder.

The explicit branch completes optical flow in masked FoV regions and warps features across adjacent frames. With downsampling by $d_4$, the completed flow is

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$

and warped features follow

$\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$

Residual misalignment is compensated with modulated DCN, and forward and backward features are fused with a $1 \times 1$ convolution. This branch is intended to preserve pixel-accurate local motion compensation and temporal coherence.

The implicit branch is the Clip-Recurrent Transformer. It soft-splits concatenated local and past-reference features into overlapping tokens, projects them to $Q$, $K$, and $V$, and caches keys and values from the previous clip with stop-gradient. Its Clip-Recurrent Hub uses 3D-Decoupled Cross Attention (DDCA): temporal attention is applied at fixed spatial positions, while spatial attention is factorized into horizontal and vertical strip attentions with non-local strip pooling. The temporal term is

$$Z_t = \mathrm{Attn}(Q_t, K_t, V_t) = \mathrm{Softmax}(Q_t K_t^\top / \sqrt{d})V_t,$$

and cross-clip fusion is applied through

$$\bar{Z}'_{i+1} = \mathrm{DDCA}(Q_{i+1}, P_{kv}(\bar{K}_i), P_{kv}(\bar{V}_i)).$$

MixF3N replaces the standard FFN with multi-branch depth-wise convolutions over overlapping tokens to strengthen fine-grained local detail flow.

Training uses a composite objective

$$L = \lambda_{rec}L_{rec} + \lambda_{adv}L_{adv} + \lambda_{flow}L_{flow},$$

with $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$0, $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$1, and $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$2. The streaming setup uses $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$3 local frames and $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$4 past reference frames, with memory length $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$5 reported as best. FlowLens runs at $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$6 s/frame with $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$7 GFLOPs, while FlowLens-s achieves $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$8 s/frame at $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$9 GFLOPs on RTX 3090; the Clip-Recurrent Hub adds $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$0 FLOPs and $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$1 parameters.

Evaluation is performed on a derived KITTI360 dataset with $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$2, $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$3, and $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$4 FoV expansion masks, covering both outer-FoV and inner-FoV scenarios. On outer-FoV reconstruction, FlowLens reports PSNR $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$5 dB, SSIM $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$6, VFID $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$7, and Ewarp $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$8; FlowLens+ reaches $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$9. On inner-FoV reconstruction, FlowLens reaches PSNR $1 \times 1$0 dB, SSIM $1 \times 1$1, VFID $1 \times 1$2, and Ewarp $1 \times 1$3; FlowLens+ reaches $1 \times 1$4. Beyond reconstruction, semantic segmentation on completed frames improves markedly: in pinhole outer-FoV settings, mIoU is reported as seen $1 \times 1$5 $1 \times 1$6, unseen $1 \times 1$7 $1 \times 1$8, and overall $1 \times 1$9 $Q$0; in spherical inner-FoV settings, seen $Q$1 $Q$2, unseen $Q$3 $Q$4, and overall $Q$5 $Q$6.

The paper explicitly states that FlowLens “embodies this concept concretely”: it transforms past visibility into current beyond-FoV reconstructions and context, and the quantitative gains indicate that visibility enhancement can improve both unseen-region completion and within-FoV perception. Its limitations are equally explicit: outward expansion is harder than inward expansion, fast ego or object motion and flow errors can degrade alignment, longer clip memories did not help due to mixed temporal distributions, and embedded deployment remains compute-constrained.

3. Generative Engine Optimization and content prominence

In Generative Engine Optimization, visibility is defined not by physical observability but by how much, how early, and how prominently a source’s content is surfaced inside an LLM-generated answer (Lüttgenau et al., 3 Jul 2025). The task is to rewrite a source document $Q$7 into $Q$8 so that, when the optimized document is included among the sources presented to an LLM, the resulting response contains more text grounded in that source and mentions it earlier.

The formulation introduces two citation-grounded extrinsic metrics. For a source $Q$9 and model response $K$0, the absolute word count is

$K$1

where $K$2 is the set of sentences in $K$3 that explicitly cite $K$4. The position-adjusted variant gives earlier citations more weight:

$K$5

Aggregated query-level improvements are computed as percentage changes after replacing $K$6 with $K$7.

The proposed model is a fine-tuned BART-base, described as a $K$8-layer encoder and $K$9-layer decoder transformer with $V$0 attention heads, hidden size $V$1, FFN size $V$2, dropout $V$3, shared embeddings, tied LM head, BPE tokenization, and support for up to $V$4 tokens. Relative to the intrinsic baseline, the proposed configuration uses a larger encoder context window, with $V$5 versus $V$6, and a capped decoder length, with $V$7 versus $V$8. The objective is token-level cross-entropy,

$V$9

optimized with AdamW at learning rate $$Z_t = \mathrm{Attn}(Q_t, K_t, V_t) = \mathrm{Softmax}(Q_t K_t^\top / \sqrt{d})V_t,$$0, weight decay $$Z_t = \mathrm{Attn}(Q_t, K_t, V_t) = \mathrm{Softmax}(Q_t K_t^\top / \sqrt{d})V_t,$$1, linear warmup for $$Z_t = \mathrm{Attn}(Q_t, K_t, V_t) = \mathrm{Softmax}(Q_t K_t^\top / \sqrt{d})V_t,$$2 updates, and cosine decay. Decoding uses length penalty $$Z_t = \mathrm{Attn}(Q_t, K_t, V_t) = \mathrm{Softmax}(Q_t K_t^\top / \sqrt{d})V_t,$$3 and $$Z_t = \mathrm{Attn}(Q_t, K_t, V_t) = \mathrm{Softmax}(Q_t K_t^\top / \sqrt{d})V_t,$$4.

Training data are synthetically constructed travel-domain pairs. The final dataset contains $$Z_t = \mathrm{Attn}(Q_t, K_t, V_t) = \mathrm{Softmax}(Q_t K_t^\top / \sqrt{d})V_t,$$5 cleaned $$Z_t = \mathrm{Attn}(Q_t, K_t, V_t) = \mathrm{Softmax}(Q_t K_t^\top / \sqrt{d})V_t,$$6 pairs. Query generation uses Meta-Llama-3-8B-Instruct at temperature $$Z_t = \mathrm{Attn}(Q_t, K_t, V_t) = \mathrm{Softmax}(Q_t K_t^\top / \sqrt{d})V_t,$$7 across $$Z_t = \mathrm{Attn}(Q_t, K_t, V_t) = \mathrm{Softmax}(Q_t K_t^\top / \sqrt{d})V_t,$$8 subcategories; optimized targets are produced by Llama-3.3-70B-Instruct-Turbo-Free in a three-stage pipeline that inserts credible-sounding citations, improves linguistic fluency while preserving key facts, and adds strategically placed statistics. The paper states that no hand-crafted citation insertion rules or explicit auxiliary labels are used during fine-tuning.

On the $$Z_t = \mathrm{Attn}(Q_t, K_t, V_t) = \mathrm{Softmax}(Q_t K_t^\top / \sqrt{d})V_t,$$9-instance test set, baseline BART reports ROUGE-L $$\bar{Z}'_{i+1} = \mathrm{DDCA}(Q_{i+1}, P_{kv}(\bar{K}_i), P_{kv}(\bar{V}_i)).$$0, BLEU $$\bar{Z}'_{i+1} = \mathrm{DDCA}(Q_{i+1}, P_{kv}(\bar{K}_i), P_{kv}(\bar{V}_i)).$$1, PPL $$\bar{Z}'_{i+1} = \mathrm{DDCA}(Q_{i+1}, P_{kv}(\bar{K}_i), P_{kv}(\bar{V}_i)).$$2, and length ratio $$\bar{Z}'_{i+1} = \mathrm{DDCA}(Q_{i+1}, P_{kv}(\bar{K}_i), P_{kv}(\bar{V}_i)).$$3, whereas the proposed “Visibility Transformer” reports ROUGE-L $$\bar{Z}'_{i+1} = \mathrm{DDCA}(Q_{i+1}, P_{kv}(\bar{K}_i), P_{kv}(\bar{V}_i)).$$4, BLEU $$\bar{Z}'_{i+1} = \mathrm{DDCA}(Q_{i+1}, P_{kv}(\bar{K}_i), P_{kv}(\bar{V}_i)).$$5, PPL $$\bar{Z}'_{i+1} = \mathrm{DDCA}(Q_{i+1}, P_{kv}(\bar{K}_i), P_{kv}(\bar{V}_i)).$$6, and length ratio $$\bar{Z}'_{i+1} = \mathrm{DDCA}(Q_{i+1}, P_{kv}(\bar{K}_i), P_{kv}(\bar{V}_i)).$$7. In controlled experiments with Llama-3.3-70B, the optimized pages yield $$\bar{Z}'_{i+1} = \mathrm{DDCA}(Q_{i+1}, P_{kv}(\bar{K}_i), P_{kv}(\bar{V}_i)).$$8 average improvement in absolute word count and $$\bar{Z}'_{i+1} = \mathrm{DDCA}(Q_{i+1}, P_{kv}(\bar{K}_i), P_{kv}(\bar{V}_i)).$$9 average improvement in position-adjusted word count. The paper interprets these gains as higher salience in generative answers.

The work is explicit about its limits. The study normalizes results and excludes outliers, but the normalization rule is not specified in detail. The sample size is $$L = \lambda_{rec}L_{rec} + \lambda_{adv}L_{adv} + \lambda_{flow}L_{flow},$$0 queries, confidence intervals and $$L = \lambda_{rec}L_{rec} + \lambda_{adv}L_{adv} + \lambda_{flow}L_{flow},$$1-values are not reported, and no ablation isolates the contributions of citations versus statistics versus style changes. The training data are travel-specific, and the paper advises domain adaptation for healthcare, finance, or technical domains.

4. Making transformer computation itself visible

InTraVisTo redefines visibility as interpretability of the computational process that generates each token in a decoder-only LLM (Brunello et al., 18 Jul 2025). Rather than optimizing external observability, it exposes the internal state of the Transformer by decoding token embeddings at each layer, visualizing component-wise information flow with a Sankey diagram, and allowing interactive embedding injection to probe causal effects.

The tool assumes a Pre-LN causal Transformer. For residual stream $$L = \lambda_{rec}L_{rec} + \lambda_{adv}L_{adv} + \lambda_{flow}L_{flow},$$2, attention and MLP updates are

$$L = \lambda_{rec}L_{rec} + \lambda_{adv}L_{adv} + \lambda_{flow}L_{flow},$$3

with

$$L = \lambda_{rec}L_{rec} + \lambda_{adv}L_{adv} + \lambda_{flow}L_{flow},$$4

Standard unembedding decodes a hidden vector through

$$L = \lambda_{rec}L_{rec} + \lambda_{adv}L_{adv} + \lambda_{flow}L_{flow},$$5

but InTraVisTo modifies this with RMS normalization and depth-aware decoder choice:

$$L = \lambda_{rec}L_{rec} + \lambda_{adv}L_{adv} + \lambda_{flow}L_{flow},$$6

The decoder matrix is interpolated across depth as

$$L = \lambda_{rec}L_{rec} + \lambda_{adv}L_{adv} + \lambda_{flow}L_{flow},$$7

For untied models, dual decoding with both $$L = \lambda_{rec}L_{rec} + \lambda_{adv}L_{adv} + \lambda_{flow}L_{flow},$$8 and $$L = \lambda_{rec}L_{rec} + \lambda_{adv}L_{adv} + \lambda_{flow}L_{flow},$$9 is available.

The Sankey view decomposes each layer into attention, residual, and feed-forward branches. With

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$00

relative branch contributions are computed using vector norms:

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$01

Backward flow across layers uses averaged attention weights

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$02

The tool has been demonstrated on GPT-2, LLaMA/Llama 2, Mistral, and Gemma families. In the reverse-digits prompt “Write numbers in reverse order. Number: 13843234 Reverse:”, mistralai/Mistral-7B-Instruct-v0.2 produces “43234381” instead of “43234831”. The paper reports that $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$03 decoding shows an “8” emerging in the fourth-last layer, and that in the Sankey flow, the FF block at layer $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$04 contributes $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$05 flow, compared with other local contributions $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$06. Embedding injection is defined by

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$07

and injecting an “8” flips the output to the correct digit sequence, though with the side effect of appending an extra digit rather than a newline. In a factual example, “Q: What is the capital of Italy? A:”, the heatmap shows early layers decoding generic continuations and higher layers converging to “Rome”.

The paper is explicit that these visualizations are heuristic rather than definitive causal proofs. It notes an “attention-as-explanation caveat,” sensitivity to normalization and decoder choice, ambiguity in mapping hidden states to tokens, and scalability issues for full-path Sankey diagrams. The stated role of the tool is lightweight, real-time visualization and hypothesis generation rather than replacement of formal mechanistic experiments.

5. Visibility-aware decision making under partial observability

In HAVEN, visibility is a control variable tied to cover utilization, enemy FoV, and line-of-sight exposure in a partially observable environment (Chauhan et al., 29 Nov 2025). The problem is formulated as a POMDP

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$08

where observation aliasing arises because threats and cover may be outside the current FoV or temporarily occluded. The transformer therefore serves as a temporal memory and inference module that attends over short histories to infer latent visibility and select low-exposure subgoals.

The high-level policy is a Deep Transformer Q-Network. Each candidate subgoal $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$09 is encoded by a tactical $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$10-D feature vector containing agent state, goal-relative quantities, candidate geometry, and visibility cues: number of enemies $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$11, minimum candidate-to-enemy distance $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$12, number of enemies with LoS to the candidate $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$13, and current exposure indicator $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$14. The short history length is $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$15 with causal attention; for each candidate, the input sequence $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$16 is formed by tiling $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$17 across $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$18 positions. The scalar Q-value is read from the final token,

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$19

Visibility enters the policy both structurally and through reward shaping. Candidates are masked by

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$20

with

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$21

The reported reward is

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$22

plus bonuses for subgoal and goal completion. Low-level execution is handled by a potential-field controller combining attraction to the selected subgoal, obstacle repulsion, enemy-FoV avoidance, anti-trap, and escape terms:

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$23

followed by smoothing and saturation,

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$24

Training uses temporal-difference learning with multi-step backup over $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$25 low-level steps:

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$26

optimized with Adam at learning rate $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$27 and $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$28-greedy exploration with $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$29. In $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$30 randomized $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$31D environments, averaged over $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$32 episodes per environment, HAVEN reports Success $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$33, Collision $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$34, Exposure $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$35, Path $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$36, and Time $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$37. The memory-less ablation reports Success $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$38, Collision $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$39, Exposure $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$40, Path $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$41, and Time $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$42, while several classical and recurrent baselines perform worse on at least one of safety, exposure, or success.

The empirical claim is that short-horizon memory directly increases stealth and safety under partial observability. The paper also states that exact transformer hyperparameters are not disclosed, that feature tiling across the $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$43 positions limits raw temporal richness, and that latency between perception and control in $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$44D can yield unsmooth motion. These caveats matter because the method’s “visibility transformer” character lies less in an exotic attention block than in the integration of masking, visibility cues, and memory into value estimation.

6. Visibility priors in structured attention for action recognition

PartialVisGraph addresses constrained FoV in skeleton-based action recognition by making visibility an explicit prior inside a hypergraph-transformer architecture (Dai et al., 1 Jul 2026). The input is a skeleton sequence

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$45

which is linearly projected with positional embedding to $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$46 and processed by a Hybrid Graph–Hypergraph block followed by an MS-TCN stack and a classifier.

The hypergraph branch constructs a learnable soft incidence matrix from temporally pooled joint features and a bank of virtual hyperedges. With temporal summary $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$47 and learnable hyperedge tokens $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$48, where experiments use $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$49, similarity is

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$50

and row-wise sparse normalization yields

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$51

Sparsemax produces exact zeros, allowing inactive memberships and identically zero columns.

The Single-Head Sample-Adaptive Transformer (SHSAT) concatenates replicated hyperedge tokens and joint tokens per frame,

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$52

and derives a sample-adaptive mask $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$53 from $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$54 so that joints attend only to hyperedges with nonzero incidence. Visibility is injected as an additive logit bias. With $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$55,

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$56

broadcast onto joint-to-hyperedge attention entries. The per-frame attention is

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$57

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$58

and the updated hyperedge tokens are the first $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$59 rows of $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$60. Joint features are reconstructed through

$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$61

The paper emphasizes that visibility is applied at the attention-logit level rather than multiplied into $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$62.

Training combines cross-entropy classification with three regularizers: a hyperedge diversity loss $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$63 based on the Gram matrix of normalized hyperedges, an assignment regularizer $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$64 combining balance and max-hinge terms, and a class-centre clustering loss $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$65. The schedule uses curriculum learning over visibility, starting with $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$66–$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$67 visible joints and gradually reducing to $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$68–$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$69 visible by epoch $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$70, plus Temporal CutMix with probability $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$71. Optimization uses SGD with Nesterov momentum $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$72, weight decay $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$73, batch size $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$74, total $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$75 epochs, and learning rate $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$76 reduced by $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$77 at epochs $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$78 and $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$79.

FoV-constrained benchmarks are defined on NTU RGB+D 60 and 120 with Easy, Medium, and Hard splits corresponding to approximately $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$80, $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$81, and $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$82 visible joints. Under severe FoV restriction, base-of-spine centre, the reported gains are large: on NTU 60 X-View, Hyperformer $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$83 PartialVisGraph $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$84 $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$85 pp$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$86; on NTU 60 X-Sub, $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$87 $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$88 pp$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$89; on NTU 120 X-Set, $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$90 $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$91 pp$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$92; and on NTU 120 X-Sub, $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$93 $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$94 pp$\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$95. The paper states that gains are “up to $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$96” on severely restricted subsets. Under full FoV with a $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$97-stream ensemble, the method reports NTU 60 accuracies of $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$98 (X-Sub) and $\hat{V}_{i \to j} = F(d_4(X_i), d_4(X_j)),$99 (X-View), and NTU 120 accuracies of $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$00 (X-Sub) and $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$01 (X-Set).

Ablations show that removing the visibility bias $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$02 reduces constrained-FoV accuracy from $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$03 to $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$04, and removing curriculum learning yields the same drop. Removing $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$05 in full-FoV single-stream experiments reduces accuracy from $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$06 to $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$07 and causes $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$08 to collapse to a single active hyperedge. The result is a clear example of a visibility-aware transformer in which observability is encoded as a first-class logit prior.

7. Terminological boundaries and related uses

A recurring source of ambiguity is the proximity between “visibility transformer” and “vision transformer.” The quadrotor paper on end-to-end obstacle avoidance explicitly states that the term is “likely a misnomer for Vision Transformer in this context” (Bhattacharya et al., 2024). There, the model is a SegFormer-inspired hierarchical ViT for depth image-to-control, optionally combined with an LSTM head, rather than a system whose objective is to reconstruct, quantify, or expose visibility itself.

That work maps depth image $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$09, attitude quaternion $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$10, and forward speed $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$11 to world-frame velocity commands $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$12. It reports that ViT models are more effective than convolutional, U-Net, and recurrent baselines as speed increases, and that ViT+LSTM further improves performance while reducing energy cost. The hardware and simulation evaluations reach speeds up to $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$13 m/s, with model sizes near $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$14M parameters and real-time inference; for example, ViT+LSTM reports $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$15 parameters and $\tilde{f}_i(x) = f_j(x + \hat{V}_{i \to j}(x)).$16 ms/frame on an RTX 3060 GPU. Attention visualizations show obstacle edges and surrounding context rather than explicit visibility masks or visibility-grounded rewards.

This contrast is useful for delimiting the term. In FlowLens, HAVEN, PartialVisGraph, GEO, and InTraVisTo, visibility is the quantity being reconstructed, optimized, masked, or displayed. In the quadrotor ViT paper, the transformer processes visual input effectively, but visibility is not formalized as a separate state variable or evaluation objective. A reasonable conclusion is that current usage distinguishes between transformers that merely operate on visible input and transformers that explicitly reason about visibility as a problem-specific signal.

Across the literature, the strongest common thread is explicit handling of partial information. In one branch, recurrent memory and motion alignment transform past observations into current beyond-FoV reconstructions; in another, citation-grounded metrics reshape documents for prominence in generated answers; in another, internal hidden-state dynamics are made visible to the analyst; and in robotics and action recognition, masks, priors, and temporal context convert partial observability into structured attention. The term therefore denotes a broad but technically coherent family of transformer adaptations centered on visibility as an operational variable rather than a generic synonym for transformer-based vision.