ToggleIn2D: 2D Video Comparison & GUI Control

• ToggleIn2D is a dual-context method that applies a toggling paradigm both for immersive-video comparison and for state-aware binary GUI control.
• In immersive video research, it uses a single 2D desktop viewport to alternately display synchronized 360° videos, facilitating fine-grained visual comparisons.
• In GUI-agent applications, ToggleIn2D benchmarks models on identifying and operating binary controls under state-dependent natural-language instructions.

Searching arXiv for the cited papers and topic context. arXiv search: "(Wang et al., 22 Feb 2026) ToggleIn2D immersive video comparison"; "(Wu et al., 17 Sep 2025) ToggleIn2D GUI toggle state-aware reasoning" ToggleIn2D is a term used in two distinct research contexts. In immersive-video research, it denotes a 2D desktop technique for comparing synchronized 360° immersive videos by showing only one video at a time in a single shared field of view and letting the viewer toggle between Video A and Video B. In multimodal GUI-agent research, it denotes the capability of an agent to correctly identify and operate binary GUI controls in 2D screenshots under state-dependent natural-language instructions. The shared label reflects a common toggling metaphor, but the underlying objects of comparison, interaction primitives, and evaluation criteria are different (Wang et al., 22 Feb 2026, Wu et al., 17 Sep 2025).

1. Terminological scope and disambiguation

“ToggleIn2D” is not a single standardized method across the literature. The term has been used for both immersive-video comparison and 2D GUI state control, with different formal objects, interfaces, and task definitions.

Research context	Meaning of ToggleIn2D	Core operation
Immersive video comparison	A browser-based 2D desktop comparison technique for 360° immersive videos	Toggle which synchronized IV is visible in one shared viewport
Multimodal GUI control	The capability to correctly operate binary toggle controls in 2D screenshots	Decide whether to CLICK or output COMPLETED

In the immersive-video setting, ToggleIn2D is explicitly introduced as a baseline 2D toggle method alongside SideBySideIn2D and ToggleInVR, and is contrasted with the more flexible sliding techniques SlideInVR and SlideIn2D. In the GUI-agent setting, the term refers to a state-aware decision problem in which an agent must perceive the current state of a 2D widget, infer the desired state from language, and determine whether to act (Wang et al., 22 Feb 2026, Wu et al., 17 Sep 2025).

A common misconception is that the two uses differ only by application domain. The available evidence suggests a stronger distinction: in one case ToggleIn2D is a concrete interaction technique with a fixed UI and rendering stack, whereas in the other it is a capability benchmark and training target for multimodal agents.

2. ToggleIn2D as an immersive-video comparison technique

In immersive-video comparison, ToggleIn2D is an implementation of the toggle / superposition paradigm for 360° immersive videos, adapted from 2D visual comparison literature. It is designed so that only one immersive video is visible at a time in a single shared FoV, with the user toggling between Video A and Video B. Conceptually, it corresponds to juxtaposition in time rather than in space: instead of showing both IVs side-by-side, the system alternates which one is shown in the same viewport. The paper also places it within Gleicher’s taxonomy as a superposition / overlay technique, because both videos occupy the same spatial region and the user toggles which is visible, sometimes with peeking (Wang et al., 22 Feb 2026).

The technique is deliberately minimal. The layout uses a common 2D structure shared with SlideIn2D and SideBySideIn2D: a top row of controls, a central canvas, and a bottom row for temporal navigation. The central canvas contains a single IV viewport that shows Video A or Video B, but never both fully at once. The viewport is fixed-size, and users drag to change where they look in the 360° sphere. Unlike SlideIn2D and SideBySideIn2D, ToggleIn2D has no split view, no adjustable slider, and no per-video workspace. Spatial navigation is shared between the two videos, and a separate minimap for each video provides global context and ROI trajectory using an azimuthal equidistant projection (Wang et al., 22 Feb 2026).

Its design rationale follows established toggling metaphors from superposition / overlay methods, quick toggle / flip strategies in 2D video comparison, and “before–after” photography widgets such as Juxtapose. The paper emphasizes that traditional toggling is effective for fine-grained comparison when content is spatially aligned, but imposes a burden on short-term visual memory, especially when content is dynamic or appears in different locations. ToggleIn2D therefore serves as a compact 2D baseline for 360° video comparison rather than a fully general solution (Wang et al., 22 Feb 2026).

3. Rendering, interaction, and constraints in the immersive-video implementation

ToggleIn2D is implemented as a web application built with Vite + React for the UI and Three.js for WebGL-based rendering of 360° IVs. Each IV is a monoscopic 360° video stored and displayed as an equirectangular projection. The system renders a textured sphere or skybox in Three.js and places a camera at its center. The visible region is determined by the camera’s vertical FoV and the canvas aspect ratio, with horizontal FoV computed as

$$hFoV = 2 \cdot \arctan\!\left(\tan\!\left(\tfrac{vFoV}{2}\right)\cdot\tfrac{W}{H}\right).$$

For ToggleIn2D, the paper specifies $vFoV = 40^\circ$ and an $hFoV \approx 98^\circ$ in the experimental monitor setup, so the technique shows roughly a $40^\circ \times 98^\circ$ window into the 360° sphere (Wang et al., 22 Feb 2026).

Spatial navigation is performed by dragging the viewport with the mouse. Horizontal drag changes yaw and vertical drag changes pitch. A Reset button returns the shared view to the default front-facing direction for both videos. A crucial property is that ToggleIn2D uses a single shared view for both IVs: when the user drags the view, the underlying camera orientation is applied to both videos equally, and on toggle the new video appears in exactly the same orientation. This shared-orientation constraint is what gives the method its spatial consistency across toggles (Wang et al., 22 Feb 2026).

The toggling mechanism exposes a Toggle button, and optionally keyboard shortcuts, that instantly switches which IV is shown in the viewport without changing camera orientation or zoom. The technique also supports peek: pressing “P” temporarily reveals the other video in a small circular region centered at the mouse cursor. For the 2D techniques, the peek region is $300 \text{ px} \times 300 \text{ px}$, corresponding to roughly 21° × 21° of view, and a “Peeking” label appears in the top-right corner while active. This gives ToggleIn2D a local simultaneous-comparison capability even though the global view remains single-source (Wang et al., 22 Feb 2026).

Temporal control is provided by a progress bar with a seek handle, Play / Pause, Jump −5s / +5s, current and total time labels, and Replay Videos. In ToggleIn2D, only the currently visible video responds to seek, play, and pause; Replay Videos restarts both IVs from the beginning and keeps them temporally synchronized. The stimuli are designed as comparable 30-second segments with synchronized clips. The technique also inherits the minimap design from SlideIn2D: each IV has a small azimuthal equidistant projection minimap showing the whole 360° sphere, the current ROI location, and a trajectory curve color-coded from red to green. However, ToggleIn2D does not support the ROI-based display manipulations ROIs SxS and ROIs Overlay, because a single shared camera orientation cannot independently reframe each video within one viewport (Wang et al., 22 Feb 2026).

4. User study findings for immersive-video ToggleIn2D

ToggleIn2D was one of five within-subject conditions in the immersive-video study: SlideInVR, ToggleInVR, SlideIn2D, ToggleIn2D, and SideBySideIn2D. Ordering was balanced with a Latin square. Participants performed 4 tasks per technique, covering T1 temporal occurrence, T2 spatial–temporal motion, T3 local visual differences, and T4 global visual differences. Measured variables included Accuracy, NASA-TLX subscales, UMUX-Lite, custom 7-point Likert ratings, free-text comments, and a post-study semi-structured interview (Wang et al., 22 Feb 2026).

The quantitative results place ToggleIn2D in an unusual position. On UMUX-Lite, it had the lowest mean score: 66.23, compared with 76.53 for SideBySideIn2D, 75.98 for SlideIn2D, 72.19 for ToggleInVR, and 69.48 for SlideInVR. A Wilcoxon test showed SideBySideIn2D > ToggleIn2D with $p = 0.038^*$, while no other pairwise differences remained significant after correction. On accuracy, however, ToggleIn2D achieved a mean proportion correct of 0.69, and a mixed-effects logistic regression found no significant main effect of technique on accuracy $(all\ p \ge 0.313)$. For NASA-TLX, there was no significant difference in overall workload across techniques $(p = 0.381)$; only physical demand differed significantly $(p < 0.001)$, with SlideInVR tending to be higher than the 2D techniques. For helpfulness ratings, only T1 showed a significant overall effect of technique $(p = 0.017)$, and post-hoc differences involving ToggleIn2D did not remain significant after correction (Wang et al., 22 Feb 2026).

The qualitative findings clarify why preference and usability lagged behind correctness. Participants often reported that toggle-style techniques were effective for subtle, local visual differences when the ROI was in approximately the same location in both videos and the framing was similar. The paper quotes one participant on a posture task:

“…for the lion posture task clearly, I used the overlay (toggle) very quickly, because it gave me very clear differences between how [the lion posture in] the top layer is different from [the lion posture in] the second layer.”

At the same time, ToggleIn2D became difficult when ROIs moved differently or appeared in different directions. Another participant described toggling in both VR and 2D as

“somewhat horrible for tracking motion… When objects are moving in a very different way or show in drastically different locations, toggling back and forth doesn't let you track both.”

A recurring theme was reliance on memory: for event counting, long sequences, or dispersed comparisons, users had to remember exactly what had been seen in the previous video, which increased cognitive burden. Peek mitigated this difficulty by enabling localized simultaneous comparison, but did not remove the underlying sequential nature of the technique (Wang et al., 22 Feb 2026).

Preference data reinforce this interpretation. ToggleIn2D was ranked last (5th) by 50% of participants. A Friedman test on overall preference showed a significant effect $vFoV = 40^\circ$0, and post-hoc analysis indicated that ToggleIn2D was significantly less preferred than SlideIn2D $vFoV = 40^\circ$1, SlideInVR $vFoV = 40^\circ$2, and SideBySideIn2D $vFoV = 40^\circ$3. This counters the assumption that a simpler interface necessarily produces a better comparison experience: the study suggests that simplicity in ToggleIn2D came at the cost of flexibility, especially for motion-heavy or spatially dispersed tasks (Wang et al., 22 Feb 2026).

5. ToggleIn2D as a 2D GUI state-control capability

In multimodal GUI-agent research, ToggleIn2D refers to the capability of a model to correctly identify and operate binary GUI controls in 2D screenshots under state-dependent natural-language instructions. The paper formalizes a toggle as a GUI element

$vFoV = 40^\circ$4

where $vFoV = 40^\circ$5 is the screenshot, $vFoV = 40^\circ$6 is the bounding box, $vFoV = 40^\circ$7 is the current binary state, and $vFoV = 40^\circ$8 is a natural-language description of the feature or semantics. Typical targets include ON/OFF switches in mobile settings, checkbox-like controls, and other visually toggle-like widgets. Interaction is a single CLICK at a 2D coordinate inside $vFoV = 40^\circ$9 (Wu et al., 17 Sep 2025).

The task is fully state-dependent. Given a screenshot $hFoV \approx 98^\circ$0 and instruction $hFoV \approx 98^\circ$1, the model must determine which GUI element corresponds to the instruction, infer its current visual state, infer the desired state from language, and decide whether a state change is needed. The paper identifies two systematic failure modes in existing agents. The first is false negative toggles, where current state and desired state differ but the agent predicts COMPLETED or another non-toggling action. The second is false positive toggles, where the current state already matches the desired state but the agent still clicks, switching the toggle to the wrong state. The central difficulty is therefore not just grounding a widget, but determining when not to toggle (Wu et al., 17 Sep 2025).

To isolate this problem, the authors construct a state-control benchmark from public Android and GUI datasets: AMEX, RICOSCA, GUIAct, AndroidWorld, AITW, and the OS-Atlas grounding dataset. Screenshots are paired with original widget bounding boxes $hFoV \approx 98^\circ$2 and automatically detected clickable regions $hFoV \approx 98^\circ$3 from OminiParser, yielding a unified set $hFoV \approx 98^\circ$4. Toggle identification and state–feature annotation are performed by two independent commercial multimodal models, GLM-4V and Qwen-2-VL-72B, with inter-annotator agreement used as a filter. The result is 40,918 quadruplets $hFoV \approx 98^\circ$5, after which the box-highlighted screenshot $hFoV \approx 98^\circ$6 is replaced with the original full screenshot $hFoV \approx 98^\circ$7 so that the agent sees the full GUI (Wu et al., 17 Sep 2025).

Each toggle annotation generates two instructions. A positive instruction requires a state change, so the ground-truth action is CLICK. A negative instruction corresponds to a state already matching the instruction, so the ground-truth action is COMPLETED. This yields 81,836 synthetic samples, with 73,652 for training and 8,184 for test, balanced between positive and negative cases. The formal task is

$hFoV \approx 98^\circ$8

where $hFoV \approx 98^\circ$9 is a structured action with a type $40^\circ \times 98^\circ$0, and for CLICK the parameters are normalized 2D coordinates in $40^\circ \times 98^\circ$1. The evaluation distinguishes action-type correctness from full action correctness, including a 4% screen distance threshold in the state-control benchmark (Wu et al., 17 Sep 2025).

6. State-aware Reasoning, performance, and limitations in GUI ToggleIn2D

The training method proposed for GUI ToggleIn2D is State-aware Reasoning (StaR). StaR is not a new architecture; it is a training method applied to existing multimodal agents. Its core decomposition has three steps: Perceiving the current toggle state $40^\circ \times 98^\circ$2 from the screenshot, Analyzing the desired state $40^\circ \times 98^\circ$3 from the instruction, and Deciding whether to CLICK or return COMPLETED by comparing the two. The reasoning chain is explicitly supervised in the model’s generated “Thought” section, and standard next-token supervised learning with cross-entropy is used over the concatenated reasoning and action output. StaR is applied to agents based on Qwen-2-VL-7B and MiniCPM-V, including OS-Atlas-7B, UI-TARS-7B, and AgentCPM-GUI-8B (Wu et al., 17 Sep 2025).

The benchmark results show that this state-aware decomposition materially changes performance. On the state-control benchmark, proprietary models such as GPT-5, GPT-4o, and Gemini 2.5 Pro are reported to have O-AMR < 40%, P-TMR close to 100% but P-AMR ~ 6–22%, and high N-FPTR and N-FPR, indicating a strong tendency to click without sufficient state awareness. Open-source agents perform somewhat better but still exhibit substantial false positives. With StaR, OS-Atlas-7B improves from O-AMR = 43.95% to 79.72% $40^\circ \times 98^\circ$4, N-AMR from 35.80% to 96.48% $40^\circ \times 98^\circ$5, N-FPTR from 64.10% to 3.52%, and N-FPR from 28.67% to 1.52%. UI-TARS-7B improves from O-AMR = 45.53% to 74.52% $40^\circ \times 98^\circ$6, and AgentCPM-GUI-8B from 64.08% to 79.00% $40^\circ \times 98^\circ$7. These numbers support the paper’s claim that StaR can improve toggle instruction execution accuracy by over 30\% for some agents (Wu et al., 17 Sep 2025).

The gains are not restricted to the isolated toggle benchmark. On AndroidControl-H/L, AITZ, and GUI-Odyssey, the paper reports that StaR does not degrade performance and often improves it. For example, on AndroidControl-L, OS-Atlas-7B improves TSR from 58.45% to 64.55%, and on GUI-Odyssey the paper reports that UI-TARS-7B improves all four metrics—TMR, AMR, TSR, and GMR—by roughly 7–20 percentage points. In a dynamic evaluation on 20 real-world Android tasks, OS-Atlas-7B improves from 10% (2/20) to 55% (11/20), UI-TARS-7B from 35% to 40%, and AgentCPM-GUI-8B from 20% to 42.5%. The paper interprets this as evidence that 2D toggle competence learned in static offline data transfers to real devices and dynamic GUIs (Wu et al., 17 Sep 2025).

The limitations are explicit. The benchmark and training setup focus on binary toggles, with state labels $40^\circ \times 98^\circ$8 and an action space centered on CLICK and COMPLETED. The work does not cover sliders, tri-state controls, long press, or continuous control. The benchmark is mostly Android/mobile, so generalization to desktop and web GUIs is not yet validated. Remaining failures are associated with non-standard themes, visually ambiguous toggles, overlapping elements, and very small widgets. A plausible implication is that “ToggleIn2D” in this literature should be understood as a robust solution to the binary state-aware toggle problem rather than a complete account of 2D GUI control (Wu et al., 17 Sep 2025).

Across both literatures, ToggleIn2D denotes a toggle-centered approach to comparison or control in a 2D environment. In immersive-video research it is a single-view, memory-intensive, locally precise comparison technique whose main strengths appear when the compared content is tightly aligned. In multimodal GUI-agent research it is a state-aware act-versus-do-not-act problem whose difficulty lies in correct state perception and action suppression. The terminological overlap is real, but the two uses occupy different technical strata: one is a human-facing comparison interface, the other an agentic decision capability.