IMCopilot: Autonomous Systems Frameworks

• IMCopilot is a family of frameworks and system architectures that integrate formal runtime verification, RL-driven manipulation, and distributed optimization to advance cyber-physical autonomy.
• Key components include NASA’s ICAROUS-based flight monitoring, in-hand manipulation via PPO-trained controllers, and multi-agent DCOP solvers for proactive emergency response.
• The system achieves measurable improvements in safety assurance, robotic dexterity, and resource allocation efficiency with minimal latency and robust performance.

IMCopilot is a family of frameworks and system architectures, each addressing distinct challenges at the intersection of autonomy, safety, monitoring, manipulation, and situational awareness in cyber-physical and robotic systems. The term spans at least three distinct research lines: monitoring and runtime verification for autonomous aircraft (Monitoring ICAROUS), reinforcement learning–augmented shared-autonomy for dexterous in-hand manipulation (In-hand Manipulation Copilot), and multi-agent optimization for proactive tasking and incident response. This entry summarizes these major variants, unified by their focus on embedded intelligence, formal guarantees, and human-in-the-loop or distributed operation.

1. Monitoring ICAROUS: Formal Runtime Verification for Autonomous Flight

IMCopilot for Monitoring ICAROUS implements tool-chain–driven runtime monitoring for autonomous unmanned aerial systems (UAS) within NASA’s ICAROUS framework. Built atop cFS middleware, it provides closed-loop assurance for safety-critical requirements through code generation from structured natural language specifications (Dutle et al., 2020).

Key system components include:

• FRET (Formal Requirements Elicitation Tool): Used to specify flight rules in a restricted natural language (fretish), automatically translated into metric temporal logic (MTL) or past-time LTL.
• Copilot: An embedded runtime verification (RV) DSL which ingests these logic formulas, compiling them to static-memory, predictable-latency C99 monitors.
• Integration Workflow: Operators define requirements in FRET, auto-generate monitors via Copilot, and deploy them as cFS applications that subscribe to live telemetry (e.g., flight_mode, intruder distance). Upon violation, monitors issue alerts usable by mission logic, operators, or resolution modules.

Example requirement formalization:

• NL: "While flying, the aircraft shall remain separated from an intruder aircraft, always: horizontal_intruder_dist > 250 ft OR vertical_intruder_dist > 50 ft"
• Temporal logic (MTL): $$G\left(\mathrm{flight\_mode} = \mathrm{in\_flight} \implies \left(\mathrm{horizontal\_intruder\_distance} > 250 \lor \mathrm{vertical\_intruder\_distance} > 50\right)\right)$$

Copilot’s statically compiled monitors run per-cycle (e.g., 10 Hz), with sub-millisecond decision latency and minimal memory. Violations trigger cFS events, enabling autonomous resolution or human alerts. The approach ensures traceable translation from requirements to executable code, facilitating formal verification by construction.

2. IMCopilot as RL-Augmented Shared-Autonomy for Dexerous Manipulation

In dexterous robotic manipulation, IMCopilot denotes an "In-hand Manipulation Copilot"—a hierarchy of RL-trained atomic controllers supplying both low-level assistance during shared teleoperation and callable closed-loop primitives for vision-language-action (VLA) policies (Tang et al., 9 Mar 2026).

Framework Components

• Skill Library: Includes stable grasp maintenance and in-hand object rotation, trained via PPO (Proximal Policy Optimization) with asymmetric actor-critic architecture and teacher-student distillation. Observation vectors combine multi-step hand kinematics, fingertip force/tactile, and task parameters.
• Shared-Autonomy Teleoperation: Human arm/hand positions are mapped via exoskeleton and VR hardware; teleoperator uses foot pedal switches to dynamically defer in-hand control to IMCopilot’s RL policies. This arbitration sharply reduces operator burden, enabling high-fidelity demonstration of contact-rich skills.
• Primitives for VLA: High-level VLA policies output action vectors and a skill gating signal; when the latter is high (e.g., language specifies “rotate apple 90°”), IMCopilot is invoked to generate the detailed dexterous motion.

Quantitative experiments show near-tripling of in-hand rotation success rates (e.g., 10% to 83% on ping-pong balls), and ablation studies indicate significant performance drops upon removing IMCopilot or multimodal inputs.

3. Distributed Resource Allocation and Emergency Response

IMCopilot frameworks also extend to distributed multi-agent optimization, notably for traffic and incident management (TIM) problems. Here, IMCopilot leverages Distributed Constraint Optimization Problem (DCOP) solvers to proactively allocate resources (emergency vehicles, UAVs) in the presence of stochastic incident evolution and interdependent constraints (Darko et al., 2022).

Technical Structure

• Agent and Variable Spaces: ERV agents ($A = \{A_i\}$) and UAV agents ($U = \{U_j\}$) optimize location assignments $X_i, Y_j$ over grid cells, modeled with incident occurrence probabilities and predicted durations.
• Cost Modeling: Binary ERV-ERV and UAV-UAV costs/utility functions incorporate both immediate and look-ahead (horizon $h=2$) incident impact. Primary and secondary incident risks are explicitly modeled using Poisson-like and cross-triggered probability matrices.
• UAV Integration: UAV assignments reduce uncertainty and hazard indices for ERV routing/costs via direct feedback into the DCOP structure. Posterior delay and uncertainty are updated using Bayesian fusion of traffic sensor and UAV data.
• Solvers: Local search heuristics (Maximum-Gain Method, Distributed Stochastic Algorithm) yield robust and scalable real-time solutions, outperforming greedy baselines by 3–15% in total delay.
• Scalability: The combined DCOP approach is robust across a range of incidents and resource densities, terminating within practical iterations and computation.

4. Mutual-Information Pilot Design and ISAC Trade-Offs

Within the context of integrated sensing and communications (ISAC), IMCopilot is invoked as a parameter-selection and pilot-design methodology optimizing the trade-off between radar sensing MI and multi-user communication MI (Bazzi et al., 2023).

Formal Optimization Problem

1. Objective Metrics:
   • Communication MI for user $k$, $\mathcal{M}_k^{\rm comm}(\Phi)$, under Gaussian-mixture priors and MIMO channel models.
   • Sensing MI, $\mathcal{M}^{\rm sense}(\Phi)$, as derived from MIMO-radar detection likelihoods.
2. MOOP to Scalar Reduction:
   • Weighted-sum objective over communication ($w_k$) and sensing ($1-\rho$) trade-off parameter,

   $$\mathcal{M}^{\rm ISAC}(\Phi) = \rho\,\mathcal{M}^{\rm comm} + (1-\rho)\,\mathcal{M}^{\rm sense}$$

• Constraint: orthogonal pilots, $\Phi\Phi^H = I_L$, i.e., Stiefel manifold.

1. Projected Gradient Descent:

   • Closed-form gradients for both MI metrics.
   • Projection via skinny SVD per step, global convergence to stationary orthogonal point is established under restricted strong convexity.
2. Performance and Guidelines:
   • Optimized pilots yield up to 6 dB NMSE gain for channel estimation and double-digit detection probability improvements (e.g., $P_d$ from 0.475 to 0.7 at $P_{fa}=10^{-4}$).
   • Trade-off parameter selection ($\rho$) scales by SNR regime and pilot length; information-overlap phenomenon enables near-simultaneous optima when target and user AoAs align.

5. Resilient State Estimation for Compromised Sensors

IMCopilot methodologies also address resilience in embedded state estimation, notably for MAVs with compromised IMUs (Tu et al., 2018):

• State-Space Models: Full 12D dynamics and measurement models for micro-aerial vehicles are discretized and estimated.
• Robust Filtering: Smooth Variable-Structure Filter (SVSF) computes innovations and “chattering” vectors to detect and isolate sensor faults, exploiting the differing fault observability of IMUs and position sensors.
• Geometric Recovery: On IMU failure, attitude (roll, pitch) is reconstructed using only position double-differences and heading, leveraging the geometric linkage between net thrust vectors and the body frame. Closed-form reconstructions enable sub-5° hover even with persistent adversarial IMU spoofing.
• Computational Impact: The approach introduces negligible CPU or SWaP overhead; full recovery is initiated within 25 ms of anomaly detection.

6. Distributed Positioning for V2X Systems

The Implicit Cooperative Positioning (ICP) method proposed within IMCopilot exploits V2V and V2F (Vehicle-to-Feature) links in vehicular networks for joint probabilistic localization (Soatti et al., 2017).

• Factor-Graph Formulation: System dynamics, GNSS, and V2F measurements are captured in a Gaussian graphical model, enabling distributed belief propagation.
• Consensus-Aided Message Passing: Feature localization and associated vehicles’ beliefs are updated via embedded average-consensus protocols, ensuring correct covariance weighting without fusion centers.
• No Ranging Hardware Needed: The implementation requires only broadcast GNSS and V2F position data; no explicit V2V ranging.
• Empirical Results: Distributed ICP achieves submeter positioning in urban canyons and robustly matches centralized Kalman performance.

7. Practical Implications and Limitations

Across applications, the IMCopilot approaches leverage formal requirement-to-code pipelines, robust shared-autonomy controllers, distributed optimization, and closed-loop resilience to deliver measurable performance improvements and verifiable guarantees. Current limitations—such as manual variable binding, deployment overhead, constraints on expressiveness of temporal logics, and reliance on skilled operators for human-in-the-loop systems—are under active investigation, with proposed solutions including parametric monitor instantiation, dynamic module updates, expanded logic expressivity, and active learning for further human effort reduction.

IMCopilot thereby functions as a transdisciplinary conceptual and technical infrastructure, blending runtime monitoring, cooperative autonomy, robust estimation, and optimization to advance the reliability, assurance, and efficiency of next-generation intelligent systems.