Glance then Scrutinize (GtS) Pattern

• GtS is a two-stage design pattern where a broad, lightweight global pass gathers non-semantic context followed by a focused local pass for detailed recovery in tasks like image reconstruction and video QA.
• The methodology leverages techniques such as global statistics, sparse localization, and depth-wise convolution to reduce computational cost while improving performance.
• Empirical studies across images, videos, and graphs demonstrate that combining glance and scrutinize phases yields improved accuracy and efficient resource allocation.

Searching arXiv for the specified GtS-related papers to ground the article in the cited preprints. Searching arXiv for (Gao et al., 2024). Glance then Scrutinize (GtS) denotes a recurrent two-stage design pattern in which a system first performs a coarse, global, or lightweight pass to obtain broad context, and then performs a focused, local, or computationally heavier pass to recover task-critical detail. In the arXiv literature, GtS appears as a human-inspired principle for image reconstruction, vision Transformers, video question answering, multimodal search, anomaly grounding, edge object detection, and graph learning. Across these settings, the first stage is variously implemented as global statistics, sparse localization, episodic memory formation, routing, or prompt-guided candidate generation, while the second stage performs local matching, dense convolution, crop-level reasoning, temporal refinement, or selective invocation of a larger model (Gao et al., 2024, Yu et al., 2021, Bai et al., 2024, Bai et al., 20 Jan 2026, Gao et al., 29 Jul 2025, Solanki et al., 16 Mar 2026, Xing et al., 8 Apr 2026, Loveland et al., 12 Oct 2025).

1. Conceptual basis and recurring rationale

Several GtS formulations explicitly derive their motivation from biological or human perceptual analogies. In "MS-Glance: Bio-Insipred Non-semantic Context Vectors and their Applications in Supervising Image Reconstruction," non-semantic context information is treated as crucial because the human visual perception system first uses global statistics to process scenes rapidly before identifying specific objects (Gao et al., 2024). "Glance-and-Gaze Vision Transformer" states that its design is motivated by the glance and gaze behavior of human beings when recognizing objects in natural scenes, pairing a global branch with a local compensatory branch (Yu et al., 2021). "Insights from Visual Cognition: Understanding Human Action Dynamics with Overall Glance and Refined Gaze Transformer" argues that the importance of temporal and spatial information varies across different time scales and that attention is allocated sparsely over time through glance and gaze behavior (Xing et al., 8 Apr 2026). "Glance and Focus: Memory Prompting for Multi-Event Video Question Answering" frames glancing as the formation of episodic memories and focusing as the use of those memories as anchors for later reasoning (Bai et al., 2024).

The same logic extends beyond visual backbones. "Glance-or-Gaze: Incentivizing LMMs to Adaptively Focus Search via Reinforcement Learning" shifts a large multimodal model from passive perception to active visual planning through a policy that decides whether to glance at global context or gaze into high-value regions (Bai et al., 20 Jan 2026). "VAGU & GtS: LLM-Based Benchmark and Framework for Joint Video Anomaly Grounding and Understanding" uses coarse localization to bootstrap semantic interpretation and then feeds semantic context back into temporal boundary refinement (Gao et al., 29 Jul 2025). "Glance for Context: Learning When to Leverage LLMs for Node-Aware GNN-LLM Fusion" generalizes the pattern further: a fast GNN handles most nodes, and an LLM is invoked only where scrutiny is likely to help (Loveland et al., 12 Oct 2025).

Taken together, these works indicate that GtS is not a single architecture but a family of coarse-to-fine allocation strategies. This suggests that the defining property is not modality, but the deliberate separation between inexpensive context acquisition and expensive detail recovery.

2. Canonical structure across tasks

Despite major differences in modality and objective, the surveyed systems share a common decomposition into a broad first pass and a selective second pass.

Work	Glance stage	Scrutinize stage
MS-Glance	Global Glance vector from randomly retrieved pixels	Local Glance vector from sliding windows
GG-Transformer	Self-attention on adaptively-dilated partitions	Depth-wise convolution for local context
Glance-Focus	Dynamic event-memory generation	Multi-level cross-attention over memory and frames
GoG	Global image decision over search mode	Region selection and iterative crop-based search
OG-ReG	SoDA coarse overall spatiotemporal path	MDConv local detail path
VAGU & GtS	Prompt-guided coarse anomaly localization	VQA interpretation and VTG boundary refinement
GLANCE for AR/VR	DWN gaze estimation	Attention-guided ROI detection
GLANCE for graphs	GNN prediction plus router	LLM refinement on routed nodes

In MS-Glance, the glance step gathers non-semantic global statistics by randomly selecting a subset of pixels and grouping them into small patches, whereas the scrutinize step applies a conventional sliding-window extraction directly to the full image (Gao et al., 2024). In GG-Transformer, the glance branch performs self-attention over adaptively dilated partitions of the full feature map, while the gaze branch uses a depth-wise convolution to recover local image context (Yu et al., 2021). In OG-ReG, the Glance path is Spatial-only Downsampling Attention and the Gaze path is Masked Dynamic Convolution mixing 2D and 3D kernels (Xing et al., 8 Apr 2026). In Glance-Focus, the first phase produces a set of dynamic event memories, and the second phase first focuses on those memories and then on the most relevant moments through a multi-level cross-attention pipeline (Bai et al., 2024).

Search-oriented systems preserve the same decomposition in policy form. GoG’s Glance policy decides among “no search,” text search, whole-image search, or region cropping, while its Gaze policy scores candidate regions after grounding and may iterate by self-correction (Bai et al., 20 Jan 2026). VAGU’s Stage I uses prompt-guided cross-modal scoring to obtain coarse candidate intervals, and Stage II non-uniformly samples frames, obtains anomaly category and semantic description via VQA, and refines timestamps via a VTG model (Gao et al., 29 Jul 2025). In the AR/VR system titled "GLANCE: Gaze-Led Attention Network for Compressed Edge-inference," gaze prediction defines attended regions and object detection is run only on a union-of-ROIs mosaic (Solanki et al., 16 Mar 2026). In graph learning, GLANCE routes only the top-$k$ scoring nodes in a batch to an LLM, while non-routed nodes retain the base GNN prediction (Loveland et al., 12 Oct 2025).

3. Representation-level realizations in images and video backbones

MS-Glance provides a formalized GtS mechanism for image similarity and supervision. For an image $I \in \mathbb{R}^{h \times w}$, a coordinate set $$\Omega \subseteq \{1,\dots,h\} \times \{1,\dots,w\}$$ with $|\Omega| = n \cdot m \ll h \cdot w$ is sampled, optionally excluding air regions in MRI by thresholding with $\delta = 0.01$. The sampled pixels are reshaped into $\mathbf{S} \in \mathbb{R}^{n \times m}$, windowed into patches, flattened, and standardized to zero mean and unit variance to form the Global Glance vectors. The Local Glance vectors are produced by applying an $n_g \times m_g$ window directly to the full image and flattening the resulting patches. Their union constitutes the multi-scale Glance vectors, and the Glance Index Measure is defined through the inner product of standardized vectors; the corresponding loss is $$L_{\mathrm{Glance}}(I_0,I_1)=1-\mathrm{GlanceIM}(I_0,I_1)$$ (Gao et al., 2024). The paper’s interpretation is explicit: the global vectors encode long-range, non-semantic spatial configuration, while the local vectors capture fine structural details.

GG-Transformer realizes GtS inside each Transformer block. Its Glance branch performs partition-wise multi-head self-attention on adaptively dilated groups of tokens, replacing the quadratic term of full self-attention with a term linear in the total token count when $M \ll N$. Its Gaze branch applies a $k \times k$ depth-wise convolution to the merged values from the Glance branch, then fuses both paths before the MLP and residual update. The reported complexity is

$I \in \mathbb{R}^{h \times w}$0

contrasted with the standard full-attention term $I \in \mathbb{R}^{h \times w}$1 (Yu et al., 2021). The associated ablation is central to the GtS interpretation: Glance-only and Gaze-only variants underperform the combined design, and partitioning alone removes local adjacency that the depth-wise convolution restores.

OG-ReG transfers the same principle to video Transformers. The model uses a dual-path block repeated in four stages of a hierarchical backbone. The Glance path, SoDA, spatially downsamples tokens before self-attention to capture coarse, low-frequency, long-range spatiotemporal cues. The Gaze path, MDConv, adaptively mixes 2D and 3D kernels using coefficients $I \in \mathbb{R}^{h \times w}$2, where $I \in \mathbb{R}^{h \times w}$3 is derived from frame similarity. The Glance and Gaze outputs are summed and fed to the FFN (Xing et al., 8 Apr 2026). The complexity reduction is explicit: vanilla global attention scales as $I \in \mathbb{R}^{h \times w}$4, while SoDA scales as $I \in \mathbb{R}^{h \times w}$5 with $I \in \mathbb{R}^{h \times w}$6, reducing quadratic token cost by a factor of $I \in \mathbb{R}^{h \times w}$7 and trading it for a linear-cost local convolution.

These backbone-level realizations show that GtS can be embedded directly into the feature extractor rather than treated only as a downstream inference policy. A plausible implication is that the paradigm is as much about factorizing representational burden as it is about reducing inference cost.

4. Memory, search, and semantic refinement

In VideoQA, GtS appears as memory prompting. Glance-Focus maintains $I \in \mathbb{R}^{h \times w}$8 learnable memory queries $I \in \mathbb{R}^{h \times w}$9, uses a 2-layer, 8-head Transformer encoder-decoder to produce updated memories $$\Omega \subseteq \{1,\dots,h\} \times \{1,\dots,w\}$$0, and then trains those memories either by unsupervised diversity losses or by supervised bipartite matching with the Hungarian algorithm (Bai et al., 2024). After sorting the memories by predicted temporal center and adding temporal positional encoding, the focusing stage concatenates encoded video features, memory prompts, and question embeddings, then performs three successive cross-attention operations: focus on memory, focus on frame, and answer prediction. The underlying GtS interpretation is explicit in the paper: glancing builds a compact episodic summary, and focusing uses those summaries to home in on relevant moments.

GoG turns GtS into an autonomous search policy for large multimodal models. Its Selective Gaze mechanism defines a region-scoring function

$$\Omega \subseteq \{1,\dots,h\} \times \{1,\dots,w\}$$1

and a soft selection distribution

$$\Omega \subseteq \{1,\dots,h\} \times \{1,\dots,w\}$$2

At inference, the top-$$\Omega \subseteq \{1,\dots,h\} \times \{1,\dots,w\}$$3 regions are cropped for search or direct analysis. The training recipe has two phases: Reflective GoG Behavior Alignment by supervised fine-tuning on “GoG-Instruct” with 5,750 human-verified multimodal trajectories, and Complexity-Adaptive Reinforcement Learning using GRPO with a KL penalty against the SFT reference policy (Bai et al., 20 Jan 2026). The action space includes $$\Omega \subseteq \{1,\dots,h\} \times \{1,\dots,w\}$$4no-search, whole-image search, crop-search, text-search, direct-answer$$\Omega \subseteq \{1,\dots,h\} \times \{1,\dots,w\}$$5, and rewards combine semantic correctness and structural compliance.

VAGU & GtS makes the same two-stage logic explicit in joint anomaly grounding and understanding. Prompt lists are generated from a video caption, an anomaly catalog, and a phrase bank; static prompts and dynamic prompts are scored against frames or segments by image-CLIP and video-CLIP encoders, fused and smoothed into a global curve $$\Omega \subseteq \{1,\dots,h\} \times \{1,\dots,w\}$$6, and converted into coarse intervals by peak detection, thresholding, and windowing (Gao et al., 29 Jul 2025). For each interval, non-uniform frame sampling is driven by the cumulative integral of $$\Omega \subseteq \{1,\dots,h\} \times \{1,\dots,w\}$$7, a VQA model produces anomaly category and semantic description, and a VTG model refines the start and end times conditioned on the narrative description. The paper also defines JeAUG, a joint metric that multiplies a grounding term derived from IoU by a semantic understanding score and a video-length compensation term.

Across these works, scrutiny is not merely local feature extraction. It may instead be question-conditioned reasoning, segment-level semantic interpretation, or search-loop refinement. This suggests that GtS is compatible with symbolic, textual, and tool-augmented decision layers in addition to neural feature hierarchies.

5. Efficiency, routing, and selective computation

Some GtS systems are designed primarily around compute allocation. In the AR/VR work "GLANCE: Gaze-Led Attention Network for Compressed Edge-inference," the Glance stage is a Differentiable Weightless Neural Network for gaze estimation. It uses a single-channel eye image, $$\Omega \subseteq \{1,\dots,h\} \times \{1,\dots,w\}$$8 normalization, average pooling, thermometer encoding, $$\Omega \subseteq \{1,\dots,h\} \times \{1,\dots,w\}$$9 small LUTs addressed by fixed subsets of bits, and a linear head followed by $|\Omega| = n \cdot m \ll h \cdot w$0 normalization to output a 3D gaze unit vector (Solanki et al., 16 Mar 2026). The reported cost per frame is 131 memory lookups and 393 MACs; total parameters are approximately 9.6 K stored in 2.2 KiB plus 131 bits LUT; mean angular error on MPIIGaze is $|\Omega| = n \cdot m \ll h \cdot w$1; and runtime on Arduino Nano 33 BLE is approximately $|\Omega| = n \cdot m \ll h \cdot w$2 per frame. The Scrutinize stage projects gaze to an ROI, forms a union-of-ROIs mosaic, optionally accumulates regions over time with decay $|\Omega| = n \cdot m \ll h \cdot w$3, and runs YOLOv12n only on the attended mosaic. When $|\Omega| = n \cdot m \ll h \cdot w$4, compute saved is approximately $|\Omega| = n \cdot m \ll h \cdot w$5, and measured energy saved is approximately $|\Omega| = n \cdot m \ll h \cdot w$6.

GLANCE for text-attributed graphs adopts the same principle through a learned router. A pre-trained GNN computes $|\Omega| = n \cdot m \ll h \cdot w$7 and class probabilities $|\Omega| = n \cdot m \ll h \cdot w$8; a router $|\Omega| = n \cdot m \ll h \cdot w$9 inspects inexpensive per-node signals $\delta = 0.01$0 and outputs $\delta = 0.01$1; the top-$\delta = 0.01$2 nodes are then sent to the LLM, which produces three prompt-conditioned embeddings for ego-only, ego + 1-hop neighbors, and ego + 2-hop neighbors (Loveland et al., 12 Oct 2025). The routing objective uses an advantage

$\delta = 0.01$3

and a REINFORCE-style router loss with entropy regularization. The dominant cost is the $\delta = 0.01$4 LLM calls, while the router and refiner add approximately $\delta = 0.01$5–$\delta = 0.01$6 of total time. On OGB-Products with 2.4 M nodes, the method routes only approximately $\delta = 0.01$7 of the nodes per batch.

These systems clarify a central property of GtS: the first stage need not be semantically complete. Its role is to estimate where or when the second stage should spend budget. In one case the budget is CNN inference on a host GPU; in the other it is LLM invocation over serialized graph neighborhoods.

6. Empirical profile, misconceptions, and open directions

Reported results across the literature are consistently framed around the complementarity of the two stages. In MS-Glance, on COCO objects for INR fitting, baseline SIREN achieves $\delta = 0.01$8 SSIM and adding MS-Glance raises performance to $\delta = 0.01$9 and $\mathbf{S} \in \mathbb{R}^{n \times m}$0 SSIM; in undersampled MRI on IXI brain at $\mathbf{S} \in \mathbb{R}^{n \times m}$1 acceleration, the $\mathbf{S} \in \mathbb{R}^{n \times m}$2 baseline yields $\mathbf{S} \in \mathbb{R}^{n \times m}$3 and $\mathbf{S} \in \mathbb{R}^{n \times m}$4MS-Glance reaches $\mathbf{S} \in \mathbb{R}^{n \times m}$5; ablations show that Local and Global Glance each help, their union is strongest, and an MRI-air prior further boosts performance by approximately $\mathbf{S} \in \mathbb{R}^{n \times m}$6 (Gao et al., 2024). In GG-Transformer, GG-T reaches $\mathbf{S} \in \mathbb{R}^{n \times m}$7 top-1 on ImageNet-1K versus $\mathbf{S} \in \mathbb{R}^{n \times m}$8 for Swin-T, $\mathbf{S} \in \mathbb{R}^{n \times m}$9 mIoU on ADE20K versus $n_g \times m_g$0, and $n_g \times m_g$1 box AP / $n_g \times m_g$2 mask AP on COCO versus $n_g \times m_g$3; the ablation further reports $n_g \times m_g$4 for Glance only, $n_g \times m_g$5 for Gaze only, $n_g \times m_g$6 for Glance + Gaze with self-attention style, $n_g \times m_g$7 for Glance + Gaze with depth-wise convolution, and $n_g \times m_g$8 for full MSA in those stages (Yu et al., 2021).

In Glance-Focus, STAR mean accuracy improves from $n_g \times m_g$9 for MIST to $$L_{\mathrm{Glance}}(I_0,I_1)=1-\mathrm{GlanceIM}(I_0,I_1)$$0 with unsupervised glancing and $$L_{\mathrm{Glance}}(I_0,I_1)=1-\mathrm{GlanceIM}(I_0,I_1)$$1 with supervised glancing; EgoTaskQA improves from $$L_{\mathrm{Glance}}(I_0,I_1)=1-\mathrm{GlanceIM}(I_0,I_1)$$2 to $$L_{\mathrm{Glance}}(I_0,I_1)=1-\mathrm{GlanceIM}(I_0,I_1)$$3 and $$L_{\mathrm{Glance}}(I_0,I_1)=1-\mathrm{GlanceIM}(I_0,I_1)$$4; AGQA v2 “all” reaches $$L_{\mathrm{Glance}}(I_0,I_1)=1-\mathrm{GlanceIM}(I_0,I_1)$$5 with supervised glancing and Faster-R-CNN features; and NExT-QA ATP-hard “all” reaches $$L_{\mathrm{Glance}}(I_0,I_1)=1-\mathrm{GlanceIM}(I_0,I_1)$$6 with large gains on Descriptive and Causal subsets (Bai et al., 2024). In GoG, average accuracy over six tasks rises from $$L_{\mathrm{Glance}}(I_0,I_1)=1-\mathrm{GlanceIM}(I_0,I_1)$$7 for MMSearch-R1* to $$L_{\mathrm{Glance}}(I_0,I_1)=1-\mathrm{GlanceIM}(I_0,I_1)$$8 for GoG-2.5-7B-SFT, $$L_{\mathrm{Glance}}(I_0,I_1)=1-\mathrm{GlanceIM}(I_0,I_1)$$9 for GoG-3-8B-Think-SFT, $M \ll N$0 for GoG-2.5-7B-RL, and $M \ll N$1 for GoG-3-8B-Think-RL; removing Selective Gaze decreases the average from $M \ll N$2 to $M \ll N$3 for Qwen2.5-VL-7B-SFT and from $M \ll N$4 to $M \ll N$5 for Qwen3-VL-8B-Think-SFT (Bai et al., 20 Jan 2026).

In OG-ReG, OG-ReG-B reports $M \ll N$6 top-1 on Kinetics-400 versus $M \ll N$7 for Video-Swin-B, $M \ll N$8 on Something-Something v2 versus $M \ll N$9, and $k \times k$0 on Diving-48 v2 versus $k \times k$1; the ablation on OG-ReG-T reports $k \times k$2 for SoDA only, $k \times k$3 for MDConv only, and $k \times k$4 for the combined design on Kinetics-400, with analogous behavior on Something-Something v2 (Xing et al., 8 Apr 2026). In the AR/VR GLANCE system, end-to-end latency is below $k \times k$5, overall COCO mAP is $k \times k$6, mAP on attended objects is $k \times k$7, and the ROI method improves small, MEDium, and LARGE object accuracy from $k \times k$8, $k \times k$9, and $I \in \mathbb{R}^{h \times w}$00 to $I \in \mathbb{R}^{h \times w}$01, $I \in \mathbb{R}^{h \times w}$02, and $I \in \mathbb{R}^{h \times w}$03, respectively (Solanki et al., 16 Mar 2026). In graph GLANCE, overall accuracy gains are up to $I \in \mathbb{R}^{h \times w}$04, gains on the most heterophilous nodes reach $I \in \mathbb{R}^{h \times w}$05 on Cora and $I \in \mathbb{R}^{h \times w}$06 on Pubmed compared to the next best method, and removing estimated local homophily drops accuracy on $I \in \mathbb{R}^{h \times w}$07 by $I \in \mathbb{R}^{h \times w}$08, $I \in \mathbb{R}^{h \times w}$09, and $I \in \mathbb{R}^{h \times w}$10 on Cora, Pubmed, and Arxiv23, respectively (Loveland et al., 12 Oct 2025).

A common misconception is to treat GtS as synonymous with simple cropping or low-resolution preprocessing. The surveyed literature shows otherwise. In some cases, “glance” is a non-semantic context descriptor over randomly retrieved pixels (Gao et al., 2024); in others it is adaptively dilated self-attention (Yu et al., 2021), dynamic event-memory generation (Bai et al., 2024), sparse spatiotemporal attention (Xing et al., 8 Apr 2026), prompt-guided temporal scoring (Gao et al., 29 Jul 2025), or a learned routing policy over nodes (Loveland et al., 12 Oct 2025). Likewise, “scrutinize” may mean local convolution, crop-level search, VQA-conditioned temporal refinement, or selective LLM fusion. The corpus therefore suggests that GtS is better understood as a resource-structured inference principle than as a fixed module type.

Open directions are stated explicitly in several papers. OG-ReG proposes learnable DR/UR, cross-attention from glance tokens into gaze feature maps, multi-scale glimpse, self-supervised pretraining such as masked MAE, and modality fusion in the glance stream (Xing et al., 8 Apr 2026). GoG identifies infrastructure reliability, multilingual visual knowledge, and fine-grained uncertainty estimation as future work (Bai et al., 20 Jan 2026). VAGU & GtS introduces JeAUG to jointly evaluate semantic interpretability and temporal precision, indicating that evaluation itself is becoming GtS-aware rather than purely localization-centric (Gao et al., 29 Jul 2025). GLANCE for graphs suggests multimodal graphs, hierarchical routers over multiple expert LLMs, and online adaptation under distribution shift (Loveland et al., 12 Oct 2025). These directions collectively indicate that ongoing work is moving from hand-designed coarse-to-fine decompositions toward learned, uncertainty-aware, and modality-spanning GtS systems.