CoGuide: Explicit Guidance Mechanisms

• CoGuide is a framework of explicit guidance mechanisms that supplement baseline processes with learned vectors, strategies, or vector fields to bias trajectories.
• It has been applied to diverse domains—image captioning, visual analytics, multi-robot navigation, language reasoning, and spatial inverse problems—each with tailored mathematical formulations.
• By decoupling guidance from core solvers, CoGuide enables enhanced performance, safety, and richer outputs through persistent semantic conditioning and coordinated interactions.

CoGuide denotes, across several research literatures, a family of explicit guidance mechanisms that steer decoding, analysis, control, navigation, or reasoning by adding a structured guiding component to a baseline process. In the cited work, that component appears as a learned guiding vector for image captioning, a strategy/action/orchestrator stack for visual analytics, a cooperative guiding vector field for multi-agent path and surface following, a dual-branch cross-view arbiter for guide dog robots, an automated guide agent for LLMs, a contrastive likelihood surrogate for diffusion-based spatial inverse problems, and an intent-sketch pipeline for zero-shot omni-modal reasoning (Jiang et al., 2018, Sperrle et al., 2022, Chen et al., 2023, Hu et al., 2024, Liu et al., 20 Mar 2026, Sharma et al., 3 Apr 2025, Basu et al., 30 Sep 2025, Shou et al., 8 Sep 2025).

1. Scope and definitional landscape

A common misconception is to treat CoGuide as a single standardized architecture. In the cited literature, it is instead a recurring guidance pattern whose mathematical form depends on the host problem: persistent semantic conditioning in sequence generation, co-adaptive support in visual analytics, nonsingular vector-field construction in robotics, user-centric hazard arbitration in assistive navigation, guide–reasoner separation in LLM systems, and surrogate likelihood guidance in diffusion samplers.

Research area	Guidance object	Representative paper
Image captioning	guiding vector $\mathbf{v}$ injected into decoder input	(Jiang et al., 2018)
Visual analytics	guidance strategies, actions, and orchestrator	(Sperrle et al., 2022)
Cooperative path following	non-singular cooperative guiding vector field	(Chen et al., 2023)
Multi-robot surface navigation	coordinated guiding vector field (CGVF)	(Hu et al., 2024)
Guide dog robotics	cross-view fusion between robot and human branches	(Liu et al., 20 Mar 2026)
LLM collaboration	automated guide interacting with a frozen reasoner	(Sharma et al., 3 Apr 2025)
Spatial inverse problems	contrastive diffusion guidance in embedding space	(Basu et al., 30 Sep 2025)
Zero-shot omni-modal reasoning	intent sketch with understand–plan–select pipeline	(Shou et al., 8 Sep 2025)

Despite the heterogeneity, the shared motif is stable. A baseline solver or controller remains intact, while an auxiliary guidance object biases its trajectory: the decoder receives $W_v\mathbf{v}$ at every time step, the VA system filters and pushes suggestions through a guidance loop, robots evolve under added virtual-coordinate dynamics, the guide dog executes human-view overrides, and diffusion sampling adds a learned surrogate likelihood gradient. This suggests that CoGuide is best understood as an architectural principle of explicit steering rather than as a single algorithmic family.

2. Learned decoder guidance and persistent semantic conditioning

In image captioning, the closest antecedent to CoGuide is the guiding network proposed in “Learning to Guide Decoding for Image Captioning” (Jiang et al., 2018). The baseline is an attention-based encoder–decoder model with an Inception-V3 encoder, global image feature $\mathbf{a}_0 \in \mathbb{R}^{2048}$, annotation vectors $\mathcal{A}=\{\mathbf{a}_1,\dots,\mathbf{a}_k\}$, and an LSTM decoder with visual attention. The key modification is the insertion of a per-image guiding vector $\mathbf{v}$ into the decoder input at every time step,

$$\mathbf{x}_t = \mathbf{E}\mathbf{y}_t + W_v \mathbf{v},$$

so the decoder is no longer driven only by the previous word embedding. The guiding network takes the annotation vectors and, optionally, attribute features $\mathbf{e}$ obtained from a Multiple Instance Learning model predicting frequent word occurrences in captions. It forms augmented annotations $\mathbf{b}_i=[\mathbf{a}_i;\mathbf{e}]$, applies a learned linear transform, and then uses element-wise max pooling across spatial positions to produce $\mathbf{v}$. Because $\mathbf{v}$ enters $W_v\mathbf{v}$0 at all decoding steps, captioning-loss gradients backpropagate to the guiding network through the full sequence, so the guidance is jointly shaped by visual evidence and linguistic supervision.

The training objective combines the standard negative log-likelihood of the caption with an optional discriminative multi-label loss. The discriminative term treats the guiding-vector components as scores over a fixed vocabulary of frequent words and uses a margin-based ranking loss,

$W_v\mathbf{v}$1

yielding

$W_v\mathbf{v}$2

The same idea is extended to Review Net with two levels of guidance: one guiding the review phase and another guiding the decoder phase, each with its own discriminative loss.

On MS COCO, the paper reports that adding MIL guidance improves over vanilla Soft Attention and Review Net, and replacing direct MIL injection with the learned guiding vector improves further. For single models, CIDEr rises from 0.978 for Soft Attention and 1.009 for Review Net to 1.023 for LTG-Soft-Attention and 1.039 for LTG-Review-Net. For 5-model ensembles, CIDEr rises from 1.015 for Soft Attention and 1.046 for Review Net to 1.070 for LTG-Soft-Attention and 1.078 for LTG-Review-Net. Ablations show that keeping both MIL and annotations gives the best performance, some discriminative supervision helps, and values around $W_v\mathbf{v}$3 work well in the reported experiments. The models with guiding networks also generate more unique words than their counterparts, which the paper interprets as richer vocabulary use.

The significance of this formulation lies in the distinction between attention and guidance. Attention answers where to look at each time step; the guiding vector supplies a global semantic bias about what high-level concepts the caption should mention. Its main limitation is equally explicit: $W_v\mathbf{v}$4 is static over time, so it does not encode a dynamic mention order or temporal plan.

3. System-provided guidance in visual analytics

In visual analytics, “Lotse: A Practical Framework for Guidance in Visual Analytics” operationalizes co-adaptive guidance as a practical, retrofit-friendly architecture (Sperrle et al., 2022). Lotse synthesizes prior guidance theory into an implementation model with three layers: templates as design artifacts, strategies and actions as system artifacts, and an orchestrator over a central analysis state. It maps the outer inference loop to strategy applicability and the inner guidance loop to action generation, suggestion delivery, retraction, and feedback-driven adaptation. Rather than assuming perfect user-intent inference, it assumes imperfect knowledge of intent and emphasizes what the system can implement today: define potentially helpful strategies, trigger them under simple conditions, and adapt over time through relevance feedback.

At runtime, the core types are explicit. A Guidance Strategy contains determine_applicability(state) -> {True, False} and a reference to an action. A Guidance Action provides is_applicable(state) -> {True, False}, generate_suggestion_content(state) -> (title, description, content), should_retract(state, suggestion) -> {True, False}, and interaction hooks such as accept, reject, preview_start, and preview_end. A Guidance Orchestrator implements the meta-strategy via filter_suggestions(candidate_suggestions). Suggestions are structured objects with id, strategy, action, title, description, and arbitrary JSON content, and are pushed to the frontend through WebSockets.

Lotse’s implementation separates specification, execution, and integration. Analysis state, strategies, actions, and the orchestrator are defined in YAML with embedded Python callbacks. The runtime engine is implemented in Python, runs a REST server for state updates and interaction endpoints, a WebSocket server for suggestion pushes and retractions, a periodic guidance loop with default period 1s, and an inference loop with default period 30s. Because the guidance loop is also triggered immediately on state update, the framework supports continuous, event-responsive system initiative. Frontend adapters for d3, Vega, Angular, and React are provided, and the design is intentionally UI-agnostic.

The paper’s use cases and case studies show the range of this guidance abstraction. Behavior-driven visualization recommendation uses interaction logs such as “slider moved” and “point hovered” to trigger suggestions like switching from a scatterplot to a line chart. The vacation example learns preference weights over weather dimensions from accept/reject feedback and uses a meta-strategy that prioritizes month switching over similar-city highlighting. In iNNspector, two VA researchers implemented YAML-defined guidance that recommends which neural-network debugging tool to use next; they reported that Lotse was “not difficult”, “efficient” and “intuitive”. In CorpusVis, two VA researchers implemented six strategies such as duplicate removal and data aggregation, maintaining per-strategy relevance scores that rise on acceptance and fall on rejection. The cognitive-dimensions analysis characterizes the notation as modular and consistent but still dependent on imperative callbacks, and the paper identifies a fully declarative grammar of guidance as an open challenge.

4. Cooperative guiding vector fields and coordinated surface or path navigation

In robotics and control, CoGuide is formalized as a vector-field construction. “Non-singular Cooperative Guiding Vector Field Under a Homotopy Equivalence Transformation” develops a guiding vector field for cooperative moving path following by lifting the path into an extended space with a virtual parameter $W_v\mathbf{v}$5 (Chen et al., 2023). A path in frame $W_v\mathbf{v}$6 is written as

$W_v\mathbf{v}$7

with extended state

$W_v\mathbf{v}$8

The resulting GVF in $W_v\mathbf{v}$9 has the Yao form

$\mathbf{a}_0 \in \mathbb{R}^{2048}$0

and the wedge term is nonzero, which removes the singularities that are topologically unavoidable for many classical GVFs on closed curves. The paper then embeds a time-varying homotopy equivalence transformation

$\mathbf{a}_0 \in \mathbb{R}^{2048}$1

with invertible Jacobian, so the GVF can be pulled back into the inertial frame while compensating for frame motion. Cooperation is introduced by assigning each agent its own $\mathbf{a}_0 \in \mathbb{R}^{2048}$2 and adding a consensus protocol

$\mathbf{a}_0 \in \mathbb{R}^{2048}$3

thereby enforcing phase offsets over a communication graph. The combined vector field yields global convergence under the reported condition $\mathbf{a}_0 \in \mathbb{R}^{2048}$4. Simulations include two robots tracking a moving ellipse with $\mathbf{a}_0 \in \mathbb{R}^{2048}$5 and four agents tracking a 3-D Lissajous curve with successive phase offsets $\mathbf{a}_0 \in \mathbb{R}^{2048}$6.

“Coordinated Guiding Vector Field Design for Ordering-Flexible Multi-Robot Surface Navigation” generalizes the construction to 2D surfaces embedded in $\mathbf{a}_0 \in \mathbb{R}^{2048}$7 and names the field explicitly as a coordinated guiding vector field, or CGVF (Hu et al., 2024). Each robot has physical position $\mathbf{a}_0 \in \mathbb{R}^{2048}$8 and virtual parameters $\mathbf{a}_0 \in \mathbb{R}^{2048}$9, with surface parameterization

$\mathcal{A}=\{\mathbf{a}_1,\dots,\mathbf{a}_k\}$0

The CGVF has three terms: a convergence term $\mathcal{A}=\{\mathbf{a}_1,\dots,\mathbf{a}_k\}$1 driving the robot to the surface, a propagation term $\mathcal{A}=\{\mathbf{a}_1,\dots,\mathbf{a}_k\}$2 providing a traversing direction on the surface, and a coordinated term in the virtual-coordinate dynamics,

$\mathcal{A}=\{\mathbf{a}_1,\dots,\mathbf{a}_k\}$3

where $\mathcal{A}=\{\mathbf{a}_1,\dots,\mathbf{a}_k\}$4 estimates a virtual target and $\mathcal{A}=\{\mathbf{a}_1,\dots,\mathbf{a}_k\}$5 is a repulsive term defined from neighboring parameter distances. The repulsion uses $\mathcal{A}=\{\mathbf{a}_1,\dots,\mathbf{a}_k\}$6 with $\mathcal{A}=\{\mathbf{a}_1,\dots,\mathbf{a}_k\}$7 for $\mathcal{A}=\{\mathbf{a}_1,\dots,\mathbf{a}_k\}$8 and $\mathcal{A}=\{\mathbf{a}_1,\dots,\mathbf{a}_k\}$9, so pairwise parameter distances remain above $\mathbf{v}$0. The theoretical result is ordering-flexible coordination: the average virtual coordinate tracks the target, local distances satisfy

$\mathbf{v}$1

and no fixed ordering of robot identities along the surface is prescribed. The paper emphasizes reduced communication and computation because each robot shares only two virtual coordinates. Its numerical simulations use 22 robots navigating a torus in $\mathbf{v}$2 with $\mathbf{v}$3, $\mathbf{v}$4, $\mathbf{v}$5, and $\mathbf{v}$6.

The two papers occupy different points in the same design space. The former focuses on transformed path following with homotopy-equivalent lifting and consensus on phase offsets; the latter focuses on ordering-flexible surface navigation with repulsive coordination in parameter space. In both, additional virtual coordinates simultaneously desingularize the guidance field and create a coordination channel that is simpler than direct geometric control on the original manifold.

5. Assistive co-ego guidance and viewpoint complementarity

In assistive navigation, “Not an Obstacle for Dog, but a Hazard for Human: A Co-Ego Navigation System for Guide Dog Robots” introduces a guidance architecture centered on viewpoint asymmetry (Liu et al., 20 Mar 2026). Quadruped guide robots typically mount sensors around 30–50 cm above ground, while a Blind and Low-Vision user has a body-height exposure zone roughly 1–1.8 m above ground. The system identifies a class of human-specific obstacles—bent branches, hanging lamps, queue barriers, low signs, and overhanging shelves—that are above the robot’s sensor horizon or filtered out as overhead clutter but remain dangerous at human head or torso height.

Co-Ego addresses this gap with a dual-branch obstacle-avoidance framework. The robot-centric branch uses an Intel RealSense D435i to generate a point cloud, applies a Z-axis pass-through filter, projects retained points to a 2D local costmap, inflates occupied cells by radius $\mathbf{v}$7, and computes an Artificial Potential Field velocity $\mathbf{v}$8. The human-centric branch uses a chest/neck-worn smartphone with LiDAR, operates in its own egocentric optical frame, detects hazards in the user’s critical safety region, and emits a reactive avoidance command $\mathbf{v}$9. Arbitration is priority-based: $$\mathbf{x}_t = \mathbf{E}\mathbf{y}_t + W_v \mathbf{v},$$0 When the human branch detects an imminent hazard, it overrides the robot branch. The system also adds a speech layer: a geometric hazard trigger on the robot side prompts an onboard Vision-LLM to produce a concise semantic description that is conveyed through TTS.

The user study uses a Unitree Go2, six sighted blindfolded participants, a hallway containing three ground-level obstacles and two hanging obstacles, and three conditions: unassisted, robot-only, and Co-Ego cross-view fusion. Reported collisions are $$\mathbf{x}_t = \mathbf{E}\mathbf{y}_t + W_v \mathbf{v},$$1 for unassisted, $$\mathbf{x}_t = \mathbf{E}\mathbf{y}_t + W_v \mathbf{v},$$2 for robot-only, and $$\mathbf{x}_t = \mathbf{E}\mathbf{y}_t + W_v \mathbf{v},$$3 for Co-Ego. Task completion time is $$\mathbf{x}_t = \mathbf{E}\mathbf{y}_t + W_v \mathbf{v},$$4 s for unassisted, $$\mathbf{x}_t = \mathbf{E}\mathbf{y}_t + W_v \mathbf{v},$$5 s for robot-only, and $$\mathbf{x}_t = \mathbf{E}\mathbf{y}_t + W_v \mathbf{v},$$6 s for Co-Ego. NASA-TLX scores are highest for unassisted, medium for robot-only, and lowest for Co-Ego; perceived safety is lowest for unassisted and highest for Co-Ego. The trade-off is explicit: cross-view fusion improves safety and lowers cognitive load, but participants move more cautiously and pauses induced by hazard reactions lengthen completion time. The paper also notes limitations: six sighted participants under blindfold, a single hallway scenario, forward-facing sensors with limited horizontal FoV, and an APF planner more suitable for structured corridors than for open spaces with local-minimum pathologies.

6. Guide models for language and omni-modal reasoning

In language reasoning, “CoLa -- Learning to Interactively Collaborate with Large LMs” casts CoGuide as an explicit guide–reasoner architecture (Sharma et al., 3 Apr 2025). A frozen reasoner $$\mathbf{x}_t = \mathbf{E}\mathbf{y}_t + W_v \mathbf{v},$$7 and a trainable guide $$\mathbf{x}_t = \mathbf{E}\mathbf{y}_t + W_v \mathbf{v},$$8 interact through a conversation

$$\mathbf{x}_t = \mathbf{E}\mathbf{y}_t + W_v \mathbf{v},$$9

where the guide begins, structures the problem, provides hints, corrects intermediate mistakes, and decides when to stop. CoLa’s learning pipeline has three stages. In Interacting, humans act as guides on a small seed set and only successful conversations are kept. In Expanding, GPT-4 is prompted as a simulated guide to generate larger numbers of guide–reasoner traces; again only successful conversations are retained. In Learning, a small guide model is first trained by supervised fine-tuning on prefixes of successful interactions, then refined by PPO using a Critique model that predicts whether a partial interaction is “successful” or “unsuccessful”, with reward

$\mathbf{e}$0

A further CoLa-RL ens stage forms beam-level ensembles and pseudo-labels via reward-weighted voting. On SocialIQA and CSQA2 with GPT as reasoner, Llama⊗GPT CoLa-RL ens reaches 86.8 and 88.3, outperforming zero-CoT, CoT, Self-Refine, MAD, GPT⊗GPT CoLa-Prompt, and CoLa-SFT. With Llama as reasoner, CoLa-SFT, CoLa-RL, and CoLa-RL ens also outperform a fully supervised baseline trained on the same amount of labeled data. On NYT Connections, Llama⊗GPT CoLa-RL reaches Acc 32.4, G-Acc 2.7, and Purity 89.8, while on CommonGen Llama⊗GPT CoLa-RL reaches Win+Tie 64.7 and Coverage 99.9. The human study shows that CoLa-RL ens exceeds human guides on both SocialIQA and CSQA2, while strategy analysis reveals a difference rather than a simple imitation: automated guides rely heavily on “correcting step in a reasoning chain” and “propagating reasoner’s response leading to contradiction”, whereas one human-only pattern, “asking the original question based on new context”, remains absent from the automated guides.

In zero-shot multimodal reasoning, “CogGuide: Human-Like Guidance for Zero-Shot Omni-Modal Reasoning” defines guidance as a plug-and-play understand–plan–select pipeline centered on an “intent sketch” (Shou et al., 8 Sep 2025). The three modules are the Intent Perceiver, which produces an intent representation $\mathbf{e}$1 from multimodal input $\mathbf{e}$2; the Strategy Generator, which samples multiple candidate strategies $\mathbf{e}$3 conditioned on $\mathbf{e}$4; and the Strategy Selector, which chooses the strategy minimizing answer uncertainty. The paper formalizes the informational role of intent as

$\mathbf{e}$5

and the selector’s target as

$\mathbf{e}$6

Unlike CoLa, CogGuide is parameter-free: no fine-tuning is performed, and cross-model transfer is obtained entirely through in-context engineering. Across IntentBench, WorldSense, and Daily-Omni, the complete three-module scheme consistently improves over the corresponding baselines for HumanOmniV2, Qwen2.5-Omni, and Qwen2.5-VL. The largest reported gain is on Daily-Omni, where Qwen2.5-Omni rises from 47.45% to 56.96%, a gain of approximately 9.51 percentage points. Ablations show that removing the Intent Perceiver or collapsing to a single strategy degrades accuracy, especially on benchmarks with strong audio–visual and temporal-alignment demands.

Taken together, CoLa and CogGuide establish two distinct but related forms of linguistic guidance. CoLa learns a guide policy specialized to a frozen reasoner and a task family; CogGuide leaves all parameters frozen and instead structures reasoning through a human-like, prompt-based strategy bottleneck. Both separate guidance from answer generation, but one does so through SFT, Critique training, and PPO, while the other does so through modular in-context control.

7. Contrastive diffusion guidance for spatial inverse problems

In “Contrastive Diffusion Guidance for Spatial Inverse Problems”, CoGuide names a diffusion-based generative inverse solver for reconstructing floorplans from movement trajectories (Basu et al., 30 Sep 2025). The latent variable is a binary occupancy grid $\mathbf{e}$7, the observation is a trajectory image $\mathbf{e}$8, and the forward process is

$\mathbf{e}$9

where $\mathbf{b}_i=[\mathbf{a}_i;\mathbf{e}]$0 is a path-planning operator based on shortest-path planning via A* and $\mathbf{b}_i=[\mathbf{a}_i;\mathbf{e}]$1 is additive Gaussian noise. The central difficulty is that the planner is non-invertible, non-differentiable, and highly non-smooth: small spatial edits such as adding doors can cause discontinuous rerouting. This destabilizes conventional guided diffusion based on the likelihood score $\mathbf{b}_i=[\mathbf{a}_i;\mathbf{e}]$2.

CoGuide breaks from differentiable-planner guidance by learning a joint embedding space with floorplan encoder $\mathbf{b}_i=[\mathbf{a}_i;\mathbf{e}]$3 and trajectory encoder $\mathbf{b}_i=[\mathbf{a}_i;\mathbf{e}]$4, both producing unit-norm vectors. A supervised contrastive loss brings compatible floorplan–trajectory pairs close and pushes mismatched pairs apart. The surrogate likelihood is then defined through the embedding inner product,

$\mathbf{b}_i=[\mathbf{a}_i;\mathbf{e}]$5

with the InfoNCE-based approximation

$\mathbf{b}_i=[\mathbf{a}_i;\mathbf{e}]$6

Accordingly,

$\mathbf{b}_i=[\mathbf{a}_i;\mathbf{e}]$7

is used as a surrogate likelihood score inside a DDIM posterior sampler. The paper also adds an intersection penalty

$\mathbf{b}_i=[\mathbf{a}_i;\mathbf{e}]$8

to discourage trajectories crossing walls, and uses Adam with cosine-annealed learning rate inside the reverse diffusion loop. This yields a guided sampler that combines the unconditional diffusion prior with a learned contrastive guidance term.

On HouseExpo, CoGuide outperforms DPS combined with Neural A*, TransPath, or DiPPeR, and also outperforms DiffPIR and DMPlug across sparse and moderate trajectory densities. In the sparse regime it achieves F1/IoU of $\mathbf{b}_i=[\mathbf{a}_i;\mathbf{e}]$9, compared with 0.79/0.67 for DPS+Neural A*, 0.76/0.64 for DPS+TransPath, 0.77/0.64 for DPS+DiPPeR, 0.31/0.19 for DMPlug, 0.63/0.47 for DiffPIR, and 0.86/0.76 for CFG. In the moderate regime it reaches 0.94/0.89, slightly above CFG at 0.93/0.88. In the dense regime, CFG is best at 0.97/0.95, while CoGuide remains close at 0.95/0.90. The paper therefore positions CoGuide as especially advantageous in the more ill-posed sparse and moderate settings, where a conditional diffusion model has less direct observational support.

Across these domains, CoGuide consistently names an explicit steering mechanism inserted between raw inputs and final behavior. What varies is the carrier of guidance: a vector, a strategy set, a vector field, a user-centric override, a guide policy, or an embedding-space score. The literature therefore does not define CoGuide by a single formalism; it defines it by the decision to make guidance first-class.