---
title: 'ToggleIn2D: 2D Video Comparison & GUI Control'
url: https://www.emergentmind.com/topics/togglein2d
type: topic
---

# ToggleIn2D: 2D Video Comparison & GUI Control

Searching arXiv for the cited papers and topic context.
arXiv search: "2602.19048 ToggleIn2D immersive video comparison"; "2509.13615 ToggleIn2D GUI toggle state-aware reasoning"
ToggleIn2D is a term used in two distinct research contexts. In immersive-video research, it denotes a **2D desktop technique** for comparing synchronized **360° immersive videos** by showing only one video at a time in a single shared field of view and letting the viewer **toggle** between Video A and Video B. In multimodal GUI-agent research, it denotes the capability of an agent to correctly identify and operate **binary GUI controls** in **2D screenshots** under state-dependent natural-language instructions. The shared label reflects a common toggling metaphor, but the underlying objects of comparison, interaction primitives, and evaluation criteria are different [2602.19048], [2509.13615].

## 1. Terminological scope and disambiguation

“ToggleIn2D” is not a single standardized method across the literature. The term has been used for both immersive-video comparison and 2D GUI state control, with different formal objects, interfaces, and task definitions.

| Research context | Meaning of ToggleIn2D | Core operation |
|---|---|---|
| Immersive video comparison | A browser-based **2D desktop** comparison technique for **360° immersive videos** | Toggle which synchronized IV is visible in one shared viewport |
| Multimodal GUI control | The capability to correctly operate **binary toggle controls** in **2D screenshots** | Decide whether to `CLICK` or output `COMPLETED` |

In the immersive-video setting, ToggleIn2D is explicitly introduced as a **baseline 2D toggle method** alongside SideBySideIn2D and ToggleInVR, and is contrasted with the more flexible sliding techniques SlideInVR and SlideIn2D. In the GUI-agent setting, the term refers to a state-aware decision problem in which an agent must perceive the current state of a 2D widget, infer the desired state from language, and determine whether to act [2602.19048], [2509.13615].

A common misconception is that the two uses differ only by application domain. The available evidence suggests a stronger distinction: in one case ToggleIn2D is a concrete interaction technique with a fixed UI and rendering stack, whereas in the other it is a capability benchmark and training target for multimodal agents.

## 2. ToggleIn2D as an immersive-video comparison technique

In immersive-video comparison, ToggleIn2D is an implementation of the **toggle / superposition** paradigm for **360° immersive videos**, adapted from 2D visual comparison literature. It is designed so that **only one immersive video is visible at a time** in a single shared FoV, with the user **toggling** between Video A and Video B. Conceptually, it corresponds to **juxtaposition in time rather than in space**: instead of showing both IVs side-by-side, the system alternates which one is shown in the same viewport. The paper also places it within Gleicher’s taxonomy as a **superposition / overlay** technique, because both videos occupy the same spatial region and the user toggles which is visible, sometimes with peeking [2602.19048].

The technique is deliberately minimal. The layout uses a common 2D structure shared with SlideIn2D and SideBySideIn2D: a **top row** of controls, a **central canvas**, and a **bottom row** for temporal navigation. The central canvas contains a **single IV viewport** that shows Video A or Video B, but never both fully at once. The viewport is fixed-size, and users drag to change where they look in the 360° sphere. Unlike SlideIn2D and SideBySideIn2D, ToggleIn2D has **no split view**, **no adjustable slider**, and **no per-video workspace**. Spatial navigation is shared between the two videos, and a separate **minimap** for each video provides global context and ROI trajectory using an **azimuthal equidistant projection** [2602.19048].

Its design rationale follows established toggling metaphors from **superposition / overlay** methods, **quick toggle / flip** strategies in 2D video comparison, and “before–after” photography widgets such as **Juxtapose**. The paper emphasizes that traditional toggling is effective for **fine-grained comparison** when content is **spatially aligned**, but imposes a burden on **short-term visual memory**, especially when content is dynamic or appears in different locations. ToggleIn2D therefore serves as a compact 2D baseline for 360° video comparison rather than a fully general solution [2602.19048].

## 3. Rendering, interaction, and constraints in the immersive-video implementation

ToggleIn2D is implemented as a **web application** built with **Vite + React** for the UI and **Three.js** for WebGL-based rendering of 360° IVs. Each IV is a monoscopic 360° video stored and displayed as an **equirectangular projection**. The system renders a textured sphere or skybox in Three.js and places a camera at its center. The visible region is determined by the camera’s **vertical FoV** and the canvas aspect ratio, with horizontal FoV computed as

\[
hFoV = 2 \cdot \arctan\!\left(\tan\!\left(\tfrac{vFoV}{2}\right)\cdot\tfrac{W}{H}\right).
\]

For ToggleIn2D, the paper specifies \(vFoV = 40^\circ\) and an \(hFoV \approx 98^\circ\) in the experimental monitor setup, so the technique shows roughly a \(40^\circ \times 98^\circ\) window into the 360° sphere [2602.19048].

Spatial navigation is performed by **dragging the viewport** with the mouse. Horizontal drag changes yaw and vertical drag changes pitch. A **Reset** button returns the shared view to the default **front-facing** direction for both videos. A crucial property is that ToggleIn2D uses a **single shared view** for both IVs: when the user drags the view, the underlying camera orientation is applied to both videos equally, and on toggle the new video appears in exactly the same orientation. This shared-orientation constraint is what gives the method its spatial consistency across toggles [2602.19048].

The toggling mechanism exposes a **Toggle** button, and optionally keyboard shortcuts, that instantly switches which IV is shown in the viewport without changing camera orientation or zoom. The technique also supports **peek**: pressing **“P”** temporarily reveals the other video in a small circular region centered at the mouse cursor. For the 2D techniques, the peek region is \(300 \text{ px} \times 300 \text{ px}\), corresponding to roughly **21° × 21°** of view, and a “Peeking” label appears in the top-right corner while active. This gives ToggleIn2D a local simultaneous-comparison capability even though the global view remains single-source [2602.19048].

Temporal control is provided by a **progress bar** with a seek handle, **Play / Pause**, **Jump −5s / +5s**, current and total time labels, and **Replay Videos**. In ToggleIn2D, only the **currently visible video** responds to seek, play, and pause; **Replay Videos** restarts **both** IVs from the beginning and keeps them temporally synchronized. The stimuli are designed as comparable 30-second segments with synchronized clips. The technique also inherits the minimap design from SlideIn2D: each IV has a small **azimuthal equidistant projection** minimap showing the whole 360° sphere, the current ROI location, and a trajectory curve color-coded from red to green. However, ToggleIn2D **does not support** the ROI-based display manipulations **ROIs SxS** and **ROIs Overlay**, because a single shared camera orientation cannot independently reframe each video within one viewport [2602.19048].

## 4. User study findings for immersive-video ToggleIn2D

ToggleIn2D was one of five **within-subject conditions** in the immersive-video study: SlideInVR, ToggleInVR, SlideIn2D, ToggleIn2D, and SideBySideIn2D. Ordering was balanced with a **Latin square**. Participants performed **4 tasks per technique**, covering **T1** temporal occurrence, **T2** spatial–temporal motion, **T3** local visual differences, and **T4** global visual differences. Measured variables included **Accuracy**, **NASA-TLX** subscales, **UMUX-Lite**, custom 7-point Likert ratings, free-text comments, and a post-study semi-structured interview [2602.19048].

The quantitative results place ToggleIn2D in an unusual position. On **UMUX-Lite**, it had the **lowest mean** score: **66.23**, compared with **76.53** for SideBySideIn2D, **75.98** for SlideIn2D, **72.19** for ToggleInVR, and **69.48** for SlideInVR. A Wilcoxon test showed **SideBySideIn2D > ToggleIn2D** with \(p = 0.038^*\), while no other pairwise differences remained significant after correction. On **accuracy**, however, ToggleIn2D achieved a mean proportion correct of **0.69**, and a mixed-effects logistic regression found **no significant main effect of technique** on accuracy \((all\ p \ge 0.313)\). For **NASA-TLX**, there was **no significant difference in overall workload across techniques** \((p = 0.381)\); only **physical demand** differed significantly \((p < 0.001)\), with SlideInVR tending to be higher than the 2D techniques. For helpfulness ratings, only **T1** showed a significant overall effect of technique \((p = 0.017)\), and post-hoc differences involving ToggleIn2D did not remain significant after correction [2602.19048].

The qualitative findings clarify why preference and usability lagged behind correctness. Participants often reported that toggle-style techniques were effective for **subtle, local visual differences** when the ROI was in approximately the **same location** in both videos and the framing was similar. The paper quotes one participant on a posture task:

> “…for the lion posture task clearly, I used the overlay (toggle) very quickly, because it gave me very clear differences between how [the lion posture in] the top layer is different from [the lion posture in] the second layer.”

At the same time, ToggleIn2D became difficult when ROIs moved differently or appeared in different directions. Another participant described toggling in both VR and 2D as

> “somewhat horrible for tracking motion… When objects are moving in a very different way or show in drastically different locations, toggling back and forth doesn't let you track both.”

A recurring theme was reliance on **memory**: for event counting, long sequences, or dispersed comparisons, users had to remember exactly what had been seen in the previous video, which increased cognitive burden. **Peek** mitigated this difficulty by enabling localized simultaneous comparison, but did not remove the underlying sequential nature of the technique [2602.19048].

Preference data reinforce this interpretation. ToggleIn2D was ranked **last (5th)** by **50%** of participants. A Friedman test on overall preference showed a significant effect \((p < 0.001)\), and post-hoc analysis indicated that ToggleIn2D was significantly less preferred than **SlideIn2D** \((p = 0.006^{**})\), **SlideInVR** \((p = 0.04^{*})\), and **SideBySideIn2D** \((p = 0.04^{*})\). This counters the assumption that a simpler interface necessarily produces a better comparison experience: the study suggests that simplicity in ToggleIn2D came at the cost of flexibility, especially for motion-heavy or spatially dispersed tasks [2602.19048].

## 5. ToggleIn2D as a 2D GUI state-control capability

In multimodal GUI-agent research, ToggleIn2D refers to the capability of a model to correctly identify and operate **binary GUI controls** in **2D screenshots** under state-dependent natural-language instructions. The paper formalizes a toggle as a GUI element

\[
\langle s, b, \sigma, f \rangle
\]

where \(s\) is the screenshot, \(b\) is the bounding box, \(\sigma \in \{0,1\}\) is the current binary state, and \(f\) is a natural-language description of the feature or semantics. Typical targets include ON/OFF **switches** in mobile settings, **checkbox-like** controls, and other visually toggle-like widgets. Interaction is a single `CLICK` at a 2D coordinate inside \(b\) [2509.13615].

The task is fully state-dependent. Given a screenshot \(s\) and instruction \(u\), the model must determine which GUI element corresponds to the instruction, infer its current visual state, infer the desired state from language, and decide whether a state change is needed. The paper identifies two systematic failure modes in existing agents. The first is **false negative toggles**, where current state and desired state differ but the agent predicts `COMPLETED` or another non-toggling action. The second is **false positive toggles**, where the current state already matches the desired state but the agent still clicks, switching the toggle to the wrong state. The central difficulty is therefore not just grounding a widget, but determining **when not to toggle** [2509.13615].

To isolate this problem, the authors construct a **state-control benchmark** from public Android and GUI datasets: **AMEX**, **RICOSCA**, **GUIAct**, **AndroidWorld**, **AITW**, and the **OS-Atlas grounding dataset**. Screenshots are paired with original widget bounding boxes \(b_o\) and automatically detected clickable regions \(b_p\) from **OminiParser**, yielding a unified set \(\{b\} = \{b_o\} \cup \{b_p\}\). Toggle identification and state–feature annotation are performed by two independent commercial multimodal models, **GLM-4V** and **Qwen-2-VL-72B**, with inter-annotator agreement used as a filter. The result is **40,918 quadruplets** \(\langle s_b, b, \sigma, f \rangle\), after which the box-highlighted screenshot \(s_b\) is replaced with the original full screenshot \(s\) so that the agent sees the full GUI [2509.13615].

Each toggle annotation generates two instructions. A **positive instruction** requires a state change, so the ground-truth action is `CLICK`. A **negative instruction** corresponds to a state already matching the instruction, so the ground-truth action is `COMPLETED`. This yields **81,836 synthetic samples**, with **73,652** for training and **8,184** for test, balanced between positive and negative cases. The formal task is

\[
\pi_\theta: (s, u) \to a
\]

where \(a\) is a structured action with a type \(t \in \{\text{CLICK}, \text{COMPLETED}, \dots\}\), and for `CLICK` the parameters are normalized 2D coordinates in \([0,1000]^2\). The evaluation distinguishes action-type correctness from full action correctness, including a **4% screen distance threshold** in the state-control benchmark [2509.13615].

## 6. State-aware Reasoning, performance, and limitations in GUI ToggleIn2D

The training method proposed for GUI ToggleIn2D is **State-aware Reasoning (StaR)**. StaR is not a new architecture; it is a **training method** applied to existing multimodal agents. Its core decomposition has three steps: **Perceiving** the current toggle state \(\sigma\) from the screenshot, **Analyzing** the desired state \(\sigma_u\) from the instruction, and **Deciding** whether to `CLICK` or return `COMPLETED` by comparing the two. The reasoning chain is explicitly supervised in the model’s generated “Thought” section, and standard next-token supervised learning with cross-entropy is used over the concatenated reasoning and action output. StaR is applied to agents based on **Qwen-2-VL-7B** and **MiniCPM-V**, including **OS-Atlas-7B**, **UI-TARS-7B**, and **AgentCPM-GUI-8B** [2509.13615].

The benchmark results show that this state-aware decomposition materially changes performance. On the state-control benchmark, proprietary models such as **GPT-5**, **GPT-4o**, and **Gemini 2.5 Pro** are reported to have **O-AMR < 40%**, **P-TMR close to 100%** but **P-AMR ~ 6–22%**, and high **N-FPTR** and **N-FPR**, indicating a strong tendency to click without sufficient state awareness. Open-source agents perform somewhat better but still exhibit substantial false positives. With StaR, **OS-Atlas-7B** improves from **O-AMR = 43.95%** to **79.72%** \((+35.77\%)\), **N-AMR** from **35.80%** to **96.48%** \((+60.68\%)\), **N-FPTR** from **64.10%** to **3.52%**, and **N-FPR** from **28.67%** to **1.52%**. **UI-TARS-7B** improves from **O-AMR = 45.53%** to **74.52%** \((+28.99\%)\), and **AgentCPM-GUI-8B** from **64.08%** to **79.00%** \((+14.92\%)\). These numbers support the paper’s claim that StaR can improve toggle instruction execution accuracy by **over 30\%** for some agents [2509.13615].

The gains are not restricted to the isolated toggle benchmark. On **AndroidControl-H/L**, **AITZ**, and **GUI-Odyssey**, the paper reports that StaR does **not degrade** performance and often improves it. For example, on **AndroidControl-L**, **OS-Atlas-7B** improves **TSR** from **58.45%** to **64.55%**, and on GUI-Odyssey the paper reports that **UI-TARS-7B** improves **all four metrics**—**TMR**, **AMR**, **TSR**, and **GMR**—by roughly **7–20 percentage points**. In a dynamic evaluation on **20 real-world Android tasks**, **OS-Atlas-7B** improves from **10% (2/20)** to **55% (11/20)**, **UI-TARS-7B** from **35%** to **40%**, and **AgentCPM-GUI-8B** from **20%** to **42.5%**. The paper interprets this as evidence that 2D toggle competence learned in static offline data transfers to real devices and dynamic GUIs [2509.13615].

The limitations are explicit. The benchmark and training setup focus on **binary** toggles, with state labels \(\sigma \in \{0,1\}\) and an action space centered on `CLICK` and `COMPLETED`. The work does not cover sliders, tri-state controls, long press, or continuous control. The benchmark is mostly **Android/mobile**, so generalization to desktop and web GUIs is not yet validated. Remaining failures are associated with non-standard themes, visually ambiguous toggles, overlapping elements, and very small widgets. A plausible implication is that “ToggleIn2D” in this literature should be understood as a robust solution to the **binary** state-aware toggle problem rather than a complete account of 2D GUI control [2509.13615].

Across both literatures, ToggleIn2D denotes a toggle-centered approach to comparison or control in a 2D environment. In immersive-video research it is a **single-view, memory-intensive, locally precise** comparison technique whose main strengths appear when the compared content is tightly aligned. In multimodal GUI-agent research it is a **state-aware act-versus-do-not-act** problem whose difficulty lies in correct state perception and action suppression. The terminological overlap is real, but the two uses occupy different technical strata: one is a human-facing comparison interface, the other an agentic decision capability.

Source: https://www.emergentmind.com/topics/togglein2d