Visual Boundary Actions Overview

• Visual Boundary Actions are a multifaceted research motif where boundaries act as interfaces for both geometric group dynamics and operational tool use.
• They demonstrate key properties such as C⁰-stability, semi-conjugacy, and rigidity in settings from negatively curved manifolds to higher-rank flag varieties.
• Applications span from geometric group theory and visual computing to vision-language agents, guiding analysis of stability, classification, and risk propagation at operational boundaries.

Searching arXiv for recent and foundational uses of “visual boundary actions” across relevant literatures. In the cited literature, Visual Boundary Actions names several technically distinct constructions centered on a boundary that is simultaneously geometric and operational. In geometric group theory and dynamics, it refers to actions on a visual boundary or on closely related boundary quotients such as generalized flag varieties $G/Q$; in this setting the main questions are $C^0$-stability, semi-rigidity, classification, freeness, kernel, and orbit-equivalence properties (Bowden et al., 2019). In vision-language agents, the phrase is used for agent actions whose arguments are derived from visual inputs and that cross a system boundary into memory, messaging, search, or other tools, so that visible text becomes operational rather than merely observed (Wang et al., 29 May 2026). Adjacent visual-computing, control, and field-theoretic literatures use boundary-centered actions more broadly for future boundary prediction, completion switching, segmentation-by-boundaries, boundary-only control, and edge dynamics (Bhattacharyya et al., 2016).

1. Terminological scope and research domains

In the current literature, the phrase does not denote a single unified theory. Rather, it organizes several families of problems in which a boundary is the locus where asymptotic geometry, system action, or local dynamics becomes explicit. In the rank-one negatively curved setting, the boundary is $\partial_\infty \widetilde M\cong S^{n-1}$, and the standard action is the natural action of $\pi_1(M)$ by homeomorphisms of that sphere (Bowden et al., 2019). In higher-rank semisimple settings, the relevant homogeneous boundaries are generalized flag varieties $G/Q$, including Furstenberg boundaries, and the standard action is left multiplication by a lattice (Connell et al., 2023). In CAT(0) and small-cancellation settings, the boundary object may be the visual boundary, contracting boundary, horofunction boundary, or the Gromov boundary of a coned-off Cayley graph, depending on which hyperbolic or asymptotic model is geometrically natural (Baik et al., 2024, Karpinski et al., 6 Sep 2025).

In contemporary agentic vision, the boundary is not geometric at infinity but a tool boundary. The relevant question is whether text visible in screenshots, documents, forms, or dashboards is copied into downstream tool-call arguments such as notes, email bodies, or search queries (Wang et al., 29 May 2026). In visual computing, closely related notions appear in future boundary forecasting, completion-aware switching, and multi-view “boundary blending,” where boundaries are representation-, data-, or semantics-level interfaces rather than only spatial delimiters (Bhattacharyya et al., 2016, Sano et al., 29 May 2026, Sun et al., 2023).

2. Rank-one visual boundaries and local $C^0$ stability

For a closed negatively curved manifold $M$, the universal cover $\widetilde M$ is a Hadamard manifold of pinched negative curvature and the boundary at infinity $\partial_\infty \widetilde M$ is topologically an $(n-1)$-sphere. The deck action of $C^0$0 extends continuously to the visual boundary, giving the standard boundary action

$C^0$1

The fundamental local theorem is that this action is $C^0$2-stable up to factor: every sufficiently small perturbation $C^0$3 in $C^0$4 is a topological factor of $C^0$5, and for any neighborhood $C^0$6 of the identity in the space of continuous self-maps of $C^0$7, there is a neighborhood $C^0$8 of $C^0$9 such that every $\partial_\infty \widetilde M\cong S^{n-1}$0 is semi-conjugate to $\partial_\infty \widetilde M\cong S^{n-1}$1 by some map in $\partial_\infty \widetilde M\cong S^{n-1}$2 (Bowden et al., 2019).

Here the factor map is a surjective continuous map

$\partial_\infty \widetilde M\cong S^{n-1}$3

satisfying

$\partial_\infty \widetilde M\cong S^{n-1}$4

The paper constructs $\partial_\infty \widetilde M\cong S^{n-1}$5 as a positive endpoint map $\partial_\infty \widetilde M\cong S^{n-1}$6, proves that it is continuous, surjective, and close to the identity when $\partial_\infty \widetilde M\cong S^{n-1}$7 is close to $\partial_\infty \widetilde M\cong S^{n-1}$8, and shows that $\partial_\infty \widetilde M\cong S^{n-1}$9 is constant on each horizontal leaf of the suspension foliation (Bowden et al., 2019). The proof replaces unavailable smooth boundary hyperbolicity with a combination of suspension geometry, weak stable and unstable foliations of geodesic flow, quasi-geodesic stability in $\pi_1(M)$0-hyperbolic spaces, and the convergence-group “conical limit point” property.

The theorem is sharp. It does not upgrade from semi-conjugacy to conjugacy. The cited examples include Cannon–Thurston maps for hyperbolic 3-manifolds fibering over the circle and a blow-up construction for $\pi_1(M)$1 actions on spheres, both producing nearby actions that are semi-conjugate but not conjugate to the standard action (Bowden et al., 2019). This identifies the correct local rigidity notion for visual boundary actions in the $\pi_1(M)$2 category: stability up to factor, not local conjugacy rigidity.

3. Higher-rank flag boundaries, classification, and rigidity limits

For cocompact lattices $\pi_1(M)$3 in semisimple Lie groups $\pi_1(M)$4, the higher-rank analogue of a rank-one boundary action is the standard action on a generalized flag variety $\pi_1(M)$5,

$\pi_1(M)$6

In this setting, sufficiently small $\pi_1(M)$7-perturbations of $\pi_1(M)$8 continuously factor onto the original action by a semiconjugacy $\pi_1(M)$9 close to the identity, and the semiconjugacy is unique in a small neighborhood of the identity (Connell et al., 2023). The proof replaces rank-one quasi-geodesic arguments by the geometry of Weyl chamber $G/Q$0-face bundles, center-stable and center-unstable foliations, barycenter constructions in compact fibers, and quasiflat rigidity for higher-rank symmetric spaces.

The positive result is confined to the homogeneous boundary strata $G/Q$1. The same paper proves a sharp negative result for the full geodesic/visual boundary $G/Q$2 of a higher-rank symmetric space $G/Q$3: the standard action of a lattice on $G/Q$4 is not locally semi-rigid among $G/Q$5-actions for any $G/Q$6 (Connell et al., 2023). Thus higher-rank rigidity survives on parabolic or flag boundaries but fails on the entire visual boundary.

A complementary classification theorem shows that in the smallest boundary dimension, higher-rank boundary actions are not merely models but essentially the only infinite-image actions. For higher-rank lattices, if $G/Q$7, where $G/Q$8 is the smallest dimension of a generalized flag variety $G/Q$9, then sufficiently smooth infinite-image actions are classified by finite covers of standard actions on $C^0$0; in the $C^0$1 case with $C^0$2, the action is $C^0$3-conjugate to the standard projective action on $C^0$4 or $C^0$5, up to the explicit double-cover ambiguity (Brown et al., 2024). The same work proves that standard projective actions on $C^0$6 are locally rigid and that smooth factors of the minimal boundary action $C^0$7 are again homogeneous flag varieties $C^0$8 (Brown et al., 2024). This suggests that, in higher-rank low dimensions, visual boundary actions become a classification principle rather than merely a source of examples.

4. Dynamical invariants, freeness, kernels, and operator-algebraic consequences

Several recent works study structural invariants of visual boundary actions rather than local stability. For a finitely generated group $C^0$9 acting geometrically on a proper CAT(0) space $M$0, if $M$1 contains a rank-one isometry and is not virtually cyclic, then

$M$2

where $M$3 is the unique maximal finite normal subgroup; the same subgroup is also the kernel of every asymptotic-cone action and of the contracting and sublinearly Morse boundary actions under the same hypotheses (Baik et al., 2024). In this regime the visual boundary detects every element outside finite normal noise.

A second line of work proves that many CAT(0) visual boundary actions are not only minimal but topologically free strong boundary actions. For a proper isometric action of a non-elementary group on a proper CAT(0) space with a rank-one element and trivial elliptic radical, if the action is cocompact or the space is geodesically complete, then the action on the limit set $M$4 is a topologically free strong boundary action; in the cocompact case this is the full visual boundary action (Ma et al., 7 Oct 2025). The proof uses Myrberg points, recurrence to rank-one axes, Tits-diameter bounds for parabolic fixed sets, and rank-one north-south dynamics.

These dynamical properties feed directly into crossed-product $M$5-algebras. For irreducible RACGs and RAAGs, the visual boundary actions on the boundaries of the Davis and Salvetti-type CAT(0) cube complexes yield simple purely infinite crossed products $M$6 and $M$7 under the irreducibility hypotheses stated in the paper (Ma et al., 2022). For actions on Bass–Serre tree boundaries, the existence of a repeatable path forces a strong boundary action and leads to large classes of unital Kirchberg algebras (Ma et al., 2022). In a different descriptive-set-theoretic direction, graphical $M$8 small-cancellation groups acting on the Gromov boundaries of their coned-off Cayley graphs have hyperfinite orbit-equivalence relations on the boundary under the “extremely fine” hypothesis (Karpinski et al., 6 Sep 2025). Together these results move visual boundary actions from rigidity questions to kernels, freeness, hyperfiniteness, and $M$9-algebraic classification.

5. Vision-language agents and action-boundary propagation

In vision-language agents, the paper introducing VisualLeakBench defines the core safety primitive as action-boundary propagation: a trace is a failure if the input image contains a target sensitive or unsafe string, the model emits a downstream tool call, and the normalized target string appears in one or more tool-call arguments (Wang et al., 29 May 2026). The relevant boundary is the tool-call argument surface, not the visible assistant response. This yields a paper-grounded definition of visual boundary actions as agent tool invocations whose arguments are derived from visible text in images and cross into memory, messaging, search, or other tools (Wang et al., 29 May 2026).

The benchmark contains 500 images over five scene families—UI, chat, document, form, and dashboard—and the agent experiments use a deterministic stratified 100-image subset, with note capture via save_note(content) and external handoff via send_email(to, body) as the two main workflows (Wang et al., 29 May 2026). The primary metric is the tool propagation rate, defined as “the fraction of non-error traces where the normalized target string appears in emitted tool arguments.” The benchmark measures visual-to-tool propagation rather than downstream instruction execution, so the failure is containment failure at the action boundary, not necessarily completed downstream harm (Wang et al., 29 May 2026).

Track	Baseline	Mitigated
PII tool propagation	78.8% = 309/392	2.0% = 8/399
Rendered unsafe-text propagation	85.5% = 335/392	52.6% = 204/388

The mitigation is strongly asymmetric. Under a defensive system prompt, PII propagation falls to 2.0%, but rendered unsafe-text propagation remains 52.6%; the paper interprets this as a difference between content that is recognizable as a sensitive format and content that is often treated as screenshot text worth preserving or forwarding (Wang et al., 29 May 2026). The paper also shows that the defense works largely by suppressing tool use rather than by preserving utility with sanitized arguments, and that failures are overwhelmingly tool-only rather than response-only. In the baseline, PII traces are 78.1% tool only and 0.0% response only, while unsafe traces are 83.4% tool only and 1.5% response only (Wang et al., 29 May 2026). This supports the paper’s central claim that response inspection alone substantially underestimates risk.

The tool surface matters. On the additional web_research surface, PII propagation is 1.0% at baseline and 0.0% under mitigation, whereas unsafe-text propagation is 54.5% at baseline and 28.0% under mitigation (Wang et al., 29 May 2026). The authors attribute this to tool semantics: search queries are short and keyword-oriented, while note and handoff tools naturally invite content preservation. The oracle study further localizes the failure at the tool boundary by blocking a tool call whenever the known benchmark target string appears in tool arguments; most unsafe traces disappear under this intervention, leaving response-side leakage as a residual risk (Wang et al., 29 May 2026).

6. Boundary-centered visual computation and adjacent theories

Several neighboring literatures use boundary actions in a more literal visual-computing sense. Future boundary prediction treats the evolving contour field, rather than RGB, as the forecasting target. One work introduced “spatio-temporal image boundary extrapolation” and showed that a multi-scale architecture performs best for future boundary maps in VSB100 and synthetic billiard settings (Bhattacharyya et al., 2016). A later long-term study proposed the Convolutional Multi-Scale Context (CMSC) model and reported, for synthetic single-ball worlds, best F-measure improvements from 0.141 to 0.987 at $\widetilde M$0, from 0.038 to 0.900 at $\widetilde M$1, and from 0.002 to 0.632 at $\widetilde M$2, while arguing that boundary forecasting captures motion patterns, object interactions, and a notion of intuitive physics (Bhattacharyya et al., 2016). In visual-language-action control, Completion at the Boundary (CaB) represents completion as a posterior over Boundary-Phase Tokens—Before, Hit, and After bins around first success—and uses a fixed global switching rule; under matched deployability constraints, CaB-When reaches E1 F1 $\widetilde M$3, full CaB reaches $\widetilde M$4, and Composite-SR rises from $\widetilde M$5 to $\widetilde M$6 when the same completion object is reused for control conditioning (Sano et al., 29 May 2026).

Boundary-centered representation also appears in segmentation and visualization design. The VHBS segmentation method defines boundary confidence by

$\widetilde M$7

encoding the rules that global-scale boundaries tend to be real object boundaries and that adjacent regions with different colors or textures tend to yield real boundaries (Su et al., 2010). In multi-view visualization, “Boundary Blending” distinguishes data boundary, representation boundary, and semantic boundary, and studies four boundary-manipulation strategies—highlighting, linking, embedding, and extending—as progressively stronger ways of moving from separated views to connected, overlaid, or blended visual spaces (Sun et al., 2023).

A more abstract but structurally related use arises in systems and field theory. For one-dimensional Boolean cellular automata, boundary reachability asks whether a target region can be driven to an arbitrary state by acting only on the boundary cells; peripherally linear rules are controllable for reachability, and double peripherally linear rules are optimally controllable (Bagnoli et al., 2016). In topological field theory, gauge theory, gravity, and supersymmetry, boundary actions arise because total derivative terms or gauge redundancies become physical edge dynamics in the presence of a boundary. The TFT boundary-reduction program derives the 2D Luttinger liquid and 4D Maxwell theory as boundary theories of higher-dimensional topological bulk models (Amoretti et al., 2014); covariant phase space methods systematize the extraction of edge-mode actions in gauge theory and gravity (Kim et al., 2023); and in 4d $\widetilde M$8 supersymmetry, universal A-type and B-type compensating boundary actions preserve different subalgebras and correspond to domain-wall and string charges, respectively (Pietro et al., 2015).

Taken together, these literatures show that “Visual Boundary Actions” is best understood as a boundary-centered research motif rather than a single doctrine. In some contexts the boundary is the sphere at infinity of a negatively curved or CAT(0) space; in others it is the tool-call interface of a multimodal agent, the completion interface of a VLA policy, the structural contour of a visual scene, or the edge where a topological gauge redundancy becomes a local degree of freedom. This suggests a common conceptual pattern: the boundary is the place where latent structure becomes observable, classifiable, or actionable.