Human-on-the-Bridge (HOB) Paradigm

• HOB is a paradigm that encodes persistent human expertise to serve as a live, strategic guidance layer for automated system execution.
• It is applied in robotic navigation, teleoperation, and agentic AI evaluation, leveraging continuous human input to shape system decisions.
• By systematically reusing expert knowledge, HOB enhances scalability, safety, and error diagnosis across diverse autonomous applications.

Human-on-the-Bridge (HOB) denotes a family of paradigms that position the human operator or domain expert "on the bridge"—that is, at the upstream interface between strategic intent or expert knowledge and automated system execution. The HOB construct spans navigation and control of physical robots, cooperative decision-making, teleoperation, and scalable evaluation of agentic artificial intelligence. Common to all instantiations is the encoding and reuse of human expertise as a continuous, live reference or guidance signal for the automated system, in contrast to episodic human-in-the-loop intervention or purely post hoc review.

1. Foundational Principles of the HOB Paradigm

The central tenet of Human-on-the-Bridge is the upfront encoding and persistent use of human knowledge, intent, or evaluation within an operational loop. This model is realized in distinct domains as follows:

• In multi-turn robotic navigation, the human's trajectory becomes a "virtual bridge" of waypoints, providing a live, sparse but authoritative reference that the robot tracks—each subsystem hands off increasingly refined representations from raw pixel-to-bounding box tracking, through fused multi-modal state estimation, to kinematically feasible trajectories and final motor execution (Antonucci et al., 2021).
• In agent evaluation, HOB shifts expert involvement from ad hoc per-task or per-turn judging to a process where evaluation intelligence (domain context, traps, juror personas, scoring, audit rules, and fallback policies) is curated once and then systematically reused and replayed in adversarial, multi-turn harness trials, with multi-juror scoring and audit tracing (Bousetouane, 15 Jun 2026).
• In cooperative decision tasks and teleoperation, HOB emerges as rigorous re-use of human conventions (bridge bidding systems, robot end effector kinematics) as the persistent substrate upon which agent policy improvement or robot control layers are built (Lockhart et al., 2020, Poignant et al., 2024).

2. Architectural Instantiations Across Domains

Robotic Navigation: Waypoint-Bridge Model

In "Humans as Path-Finders for Safe Navigation" (Antonucci et al., 2021), HOB is concretized as a sequence of three tightly integrated modules:

1. Appearance-based tracking: Lightweight SSD/MobileNet or YOLOv3 detectors identify team members every $m$ frames. Leader re-identification leverages per-box appearance vectors (GoogLeNet, ResNet50), KNN classification, and drift rejection logic, with CSRT tracking smoothing out short-term occlusion gaps.
2. Spatiotemporal fusion and global tracking: LIDAR clustering localizes humans in 2D space. Sensor fusion synchronizes visual and LIDAR detections in a global frame using odometry-propagated priors and a Multiple-Model Kalman Filter/EKF mixture, maintaining a unimodal Gaussian track that supports both continuous leader tracking and recovery from occlusion.
3. Waypoint interpolation and execution: Filtered leader positions are LOESS-smoothed and resampled. A $G^2$-continuous clothoid spline (variable curvature) interpolates the human-marked path, which is then supplied to a non-velocity-dependent steering controller for the unicycle robot.

Agentic AI Evaluation: Evaluation Intelligence Bridge

HOB evaluation moves all expert knowledge acquisition upstream—the "bridge" is the domain-specific evaluation intelligence $I_{\rm HOB}$, which includes domain context $D$, Red-Team Traps $T$, Juror Personas $J$, Scoring Rubrics $S$, Audit Rules $R$, and Fallback Policies $F$. Automated harnesses (such as ProofAgent Harness) then instantiate adversarial session drivers, trace capture, multi-juror scoring, rule-based policy drift detection, and reporting (Bousetouane, 15 Jun 2026).

Cooperative Bidding and Teleoperation

In bridge bidding, the HOB structure emerges as imitation learning anchored to a hand-coded human-convention bot (WBridge5) which defines the communication substrate. This enables subsequent in-training search and policy iteration to selectively improve performance without sacrificing human compatibility or introducing language drift (Lockhart et al., 2020).

In teleoperation, HOB is realized by a virtual, reconfigurable kinematic link between a tracked human body frame and robot end-effector frame, rendered in augmented reality, with hybrid control modes exploiting both embodied position-based control and joystick velocity for extended reach (Poignant et al., 2024).

3. Algorithmic and Evaluation Frameworks

Evaluation Harness Pseudocode and Score Computation

Let $A$ be the Agent Under Test, $G^2$0 the Harness LLM, and $G^2$1 the evaluation intelligence. The core routine is:

$G^2$2

where $G^2$3 consists of the trajectory $G^2$4, a score vector $G^2$5, detected failures $G^2$6, and a final report $G^2$7.

Key pseudocode modules include:

• Input Manager (domain/trap loading)
• Adversary Engine (multi-turn user-agent protocol)
• Trace Recorder (full turn and metadata capture)
• Multi-Juror Evaluator (scoring and audit rule enforcement)
• Fallback Manager (pipeline error recovery)

Formal scoring aggregates objective audit failures: $G^2$8 and subjective, persona-dependent scores: $G^2$9 with total score vector $I_{\rm HOB}$0.

Specialized rules detect trace-verifiable errors such as phantom tool-call claims: $I_{\rm HOB}$1 and policy-drift metrics quantify the fraction of agent turns violating mandatory guidance: $I_{\rm HOB}$2

Teleoperation Mapping and Control Laws

For body-to-robot position mapping in AR teleoperation (Poignant et al., 2024): $I_{\rm HOB}$3 with end-effector servoed to the moving virtual target by

$I_{\rm HOB}$4

Hybrid control merges direct body mapping and joystick-provided displacements via time-integrated velocity additions to the virtual link geometry.

4. Empirical Results, Comparison, and Scalability

Robotic Navigation

• In hallway navigation (58 trials, dynamic obstacles), success rate was 95% (55/58 runs without re-initialization), 2D position RMSE ≈ 0.12 m, lateral path-following errors <0.15 m in straightaways.
• Appearance-based tracking achieved ≈92% recall and ≈90% re-ID; occlusion bridging up to $I_{\rm HOB}$5 frames.
• Failure rates under crowd interference were low (<3% false re-assignments) (Antonucci et al., 2021).

Evaluation Harness and Agentic AI

• HOB supports both symmetric (evaluator ≈ agent) and asymmetric (evaluator ≪ agent) configurations. Across 23,500 agent turns in three domains, smaller evaluators found all objective, trace-verifiable failures (phantom tool calls, missing compliance) also found by larger models, though subjective scores varied.
• Cost break-even for HOB (compared to traditional human-in-the-loop) is achieved once $I_{\rm HOB}$6 with $I_{\rm HOB}$7 one-time curation, $I_{\rm HOB}$8 per-run execution, $I_{\rm HOB}$9 per-run human review, and $D$0 fraction flagged for review (Bousetouane, 15 Jun 2026).
• Key agent behavior defects diagnosed by HOB included phantom claims (16% of code-gen turns), safe-but-non-resolving refusals, and missing mandatory tool calls.

Teleoperation

• HOB body-only mode was ≈10% faster to target than joystick/hybrid (median $D$1 = 5.4s, $D$2). Dual mode usage analysis: body contributed ≈80% of displacement, joystick ≈20%.
• Body mode was more physically demanding (3–6× total displacement), while hybrid mitigated this for large/depth-dominant tasks (Poignant et al., 2024).
• Assistive tests: users with severe upper-body restriction could perform pick-and-place via head linkage within minutes of practice.

Cooperative Bidding

• Soft policy improvement, anchored to human convention, outperformed both pure imitation and pure search. Compatible PI + test-time search yielded IMP/deal gains (+0.48 to +0.56 for compatible, +0.12 to +0.85 for partnership; state of the art) against baseline (Lockhart et al., 2020).
• Compatibility with human partners was preserved, with expert human play showing +0.97 IMP/deal over WBridge5 baseline.

5. Advantages, Limitations, Failure Modes, and Best Practices

Advantages

• Scalability: HOB amortizes the up-front curation cost of human expertise across large numbers of agent runs or robot trajectories.
• Multi-modal extensibility: The paradigm generalizes across navigation, teleoperation, decision support, agentic evaluation, and mixed-initiative tasks.
• Evidence-linking: Objective, trace-verifiable failures can be consistently diagnosed independent of evaluator size or human-in-the-loop intensity.

Limitations

• Coverage Limitations: HOB requires robust initial curation (domain/traps/audit/rubrics), which may not generalize across divergent agent protocols or robotic tasks.
• Calibration Needs: Juror-score alignment is needed for subjective assessment; non-factorial experimental sweeps may limit statistical confidence.
• Policy Drift: In co-learning settings, deviation from human anchors can undermine compatibility; soft policy improvement is preferable to hard switches (Lockhart et al., 2020).

Best Practices

• Store $D$3 as versioned, composable artifacts (JSON/YAML), with explicit changelogs for traceability.
• Version both evaluation harness and domain-specific rules/scenarios to support reproducibility and continuous challenge adaptation.
• In teleoperation, hybrid position/velocity channels should be provided to balance intuitiveness and physical effort, especially for depth-intense or large-range tasks.

6. Synthesis and Theoretical Significance

HOB establishes a middle ground between entirely human-in-the-loop and fully autonomous-agent paradigms. In HOB, humans construct the "bridge"—a manifold spanning intent, knowledge, or evaluative logic—over which the system's behavioral trace is projected, refined, and ultimately enacted. Each subsystem or module (vision, sensor fusion, control, agent policy, evaluator) successively hands off a more abstract, lossless, or kinematically/practically feasible representation, maximizing both autonomy and safety/compatibility. This mediation can be represented as a pipeline:

Subsystem	Bridge Artefact	Domain Example
Perception/Tracking	Discrete 2D/3D pose estimates	Navigation, Teleoperation
Sensor Fusion	Global, filtered state tracks	Navigation
Policy/Evaluator Design	Evaluation intelligence ($D$4)	AI Agent Evaluation
Path/Policy Optimization	$D$5-continuous paths, soft-updated policies	Navigation, Bidding
Controller/Juror Engine	Real-time actuation/scoring	All domains

This suggests that the HOB paradigm, in its various instantiations, systematically shifts human expertise from ongoing reactive oversight to proactive, reusable substrate. A plausible implication is improved coverage, fewer missed failures, and scalable integration with continuous deployment or operationalizing systems with embedded human values and judgment.

7. Applications and Future Directions

• Agentic AI: Introduction of domain-specific jurors, automated adversarial scenario discovery, and CI/CD integration for continuous evaluation harness operation (Bousetouane, 15 Jun 2026).
• Navigation/Teleoperation: Extension to head- or gaze-based control for accessibility, additive hybrid channels, and ergonomic optimization (Poignant et al., 2024).
• Cooperative Decision Tasks: Application to domains with multi-partner, multi-style conventions, and adaptation to online style detection (Lockhart et al., 2020).
• Evaluation Science: Systematic comparison across evaluator tier, domain, and run configuration for tighter confidence intervals in agent assessment.
• Persistent Reference Pathways: Application in autonomous vehicles, assistive robots, and complex workflow agents where human-in-the-bridge roles remain critical for safety and overspecification deterrence.

Taken together, HOB defines a rigorous, reusable mediation of human strategic knowledge and automated system execution, supporting comprehensive, evidence-linked, and context-aware evaluation or control across the spectrum from physical machines to agentic software systems (Antonucci et al., 2021, Lockhart et al., 2020, Poignant et al., 2024, Bousetouane, 15 Jun 2026).