InSight: Advances in Robotics, Sensing & Space

• InSight is a multidisciplinary suite encompassing robotics introspection, advanced tactile sensing, high-energy astrophysics instrumentation, and continual skill acquisition.
• It introduces novel methods like token-level uncertainty in VLA models and integrates photometric stereo with machine learning for precise force mapping.
• The framework enhances real-time robotic control and astrophysical analyses by pioneering scalable, self-guided techniques across diverse applications.

InSight refers to a set of distinct research efforts spanning robotics, haptics, space astrophysics instrumentation, and vision-language-action learning. The term encompasses: (1) learning frameworks for introspection in vision-language-action (VLA) models; (2) vision-based robotic tactile sensors; (3) China’s first X-ray astronomy satellite (Insight-HXMT); and (4) continual, self-guided VLA skill acquisition. Each of these systems embodies different fundamental methodologies and technical objectives and has made notable contributions to its respective field.

1. Vision-Language-Action Model Introspection: The INSIGHT Framework

Recent developments in VLA models—neural policies mapping natural language and raw sensory input to robot control signals—have demonstrated strong generalization but lack inference-time introspection. The INSIGHT framework introduces a systematic approach for leveraging token-level uncertainty measures to predict, at each decoding step, whether a VLA model should request human intervention (Karli et al., 1 Oct 2025).

INSIGHT operates as an auxiliary module alongside autoregressive VLA policies such as π₀-FAST. At each step, for the sequence of predicted tokens, INSIGHT extracts a four-dimensional uncertainty vector per token: entropy, negative log-probability, and Dirichlet-based estimates of aleatoric and epistemic uncertainty. Sequence modeling using a compact transformer encoder (1 layer, 4 heads, 64-dimensional embedding, ≈300k parameters) aggregates these features to output a per-step probability of needing help.

Two supervision regimes are examined: (1) step-level, strongly-supervised binary labels, and (2) weakly-supervised episode-level labels indicating task success/failure. While strong supervision yields superior F1 and earlier help-triggering, weak labels offer scalable data annotation with competitive performance when label granularity is matched at evaluation. Temporal modeling of token-level uncertainty provides significant gains over static sequence-level metrics such as entropy or perplexity conformal prediction.

Experiments on an xArm7 kitchen task suite and simulated LIBERO tasks establish token-level INSIGHT as state-of-the-art for real-time failure anticipation in VLAs, showcasing robust transfer to both in-distribution and out-of-distribution conditions.

2. Vision-Based Haptic Sensing: The Insight Sensor

Insight, as described by Yuan et al., is a vision-based, thumb-sized, soft haptic sensor for robotics that achieves accurate, all-round force perception using a monocular camera, photometric stereo, and structured light (Sun et al., 2021). Its mechanical structure comprises a conical elastomer shell (40 mm base, 70 mm height) with a high-sensitivity “tactile fovea” (13×11 mm²) near the tip and an internal aluminum skeleton to guarantee robustness.

Illumination is provided by an eight-LED WS2812 ring, projected through a custom collimator, simultaneously supporting photometric stereo (for tangent surface normals) and structured light (for depth/deformation). The system is calibrated by offline modeling of intensity/depth relationships for each RGB channel.

A customized ResNet-18 variant maps input image differences and a skeleton mask directly to a dense 3D force map over a mesh of 3,800 nodes. The sensor achieves median spatial localization error of 0.4 mm, force magnitude accuracy of ~0.03 N, and directional accuracy of ~5°, with generalization across a range of indenter sizes and under multi-contact conditions. The hardware and inference pipeline are modular and transferable to various robot geometries.

3. High-Energy Astrophysics Instrumentation: Insight-HXMT

Insight-HXMT (Hard X-ray Modulation Telescope) is China’s first X-ray astronomy satellite, utilizing a unique configuration of 18 NaI(Tl)/CsI(Na) phoswich detectors for both focused X-ray and all-sky gamma-ray monitoring from 0.2 to 3 MeV (Song et al., 2022). The thick CsI layers enable high effective area (~5100 cm² total) for gamma-ray burst (GRB) detection above 500 keV, filling a spectral detection gap between Fermi/GBM and Swift/BAT.

The satellite does not employ on-board GRB triggering; all GRB candidates are detected offline using blind searches and targeted, coherent analyses based on external alerts. The first four-year GRB catalog contains 322 events, categorized into GOLDEN (well-calibrated, multi-mission), SILVER (insufficient HXMT spectral coverage), BRONZE (HXMT-only), and IRON (data saturation).

Joint spectral analysis leverages simultaneous HXMT, Fermi/GBM, and Swift/BAT measurements, significantly improving constraints on Band function parameters (α, β, E_peak) at high energies and enabling population studies of prompt GRB spectra in the MeV regime. The catalog provides detailed measurements of T₅₀, T₉₀, hardness ratios, fluence, and peak photon fluxes. The data enable cross-calibration and facilitate transient multi-messenger astrophysics.

4. Continual Skill Acquisition for VLA Models: InSight (Self-Guided Skill Acquisition)

InSight extends the VLA paradigm by enabling autonomous, continual skill acquisition without the need for new human-provided demonstration trajectories (Wang et al., 23 Jun 2026). The method renders VLAs steerable at the primitive-action level by decomposing high-level tasks into sequences of labeled primitives using a vision-LLM (VLM).

The process consists of two primary pipelines:

1. Automated Segmentation: Demonstration trajectories are partitioned into primitive segments using VLM-based plan decomposition, detection of gripper state transitions, and end-effector pose clustering. Each segment is automatically matched to a semantic primitive label, and fine-tuning is performed with LoRA adapters.
2. VLM-Guided Data Flywheel: For novel tasks requiring unknown primitives (not in the current policy vocabulary), the VLM identifies gaps, proposes low-level control parameters, and orchestrates autonomous robot attempts. Success is evaluated by the VLM through before/after image comparison. Successful executions are labeled and integrated into the training set, expanding the VLA’s primitive skill repertoire.

Experimental evaluations on both simulated and real hardware environments demonstrate that InSight attains up to 96% hardware success for complex skills (e.g., bottle twisting and pouring) with zero task-specific human demonstration, and 75% block-flip success in simulation within 500 attempts, outperforming RL baselines under the same budget. The policy efficiently composes newly acquired primitives for long-horizon tasks with high end-to-end success rates. Selective constraints (e.g., single-axis primitive proposals) are critical to stability.

5. Technical and Methodological Synthesis

The projects under the “InSight” umbrella probe inference, perception, control, and data efficiency at multiple levels:

• The INSIGHT framework for VLA introspection (Karli et al., 1 Oct 2025) establishes that sequential modeling of granular uncertainty signals (entropy, log-probability, Dirichlet AU/EU) with transformers is superior to global heuristics for real-time error forecasting in robotic control.
• The tactile Insight sensor (Sun et al., 2021) achieves high spatial and force accuracy by integrating photometric stereo and structured light with learned mappings, representing a cohesive application of physical modeling and self-calibrating machine learning.
• Insight-HXMT (Song et al., 2022) pioneers MeV-scale all-sky monitoring and collaborative transient astrophysics, utilizing data-driven detector response modeling with Monte Carlo calibration and robust offline search architectures.
• InSight for skill acquisition (Wang et al., 23 Jun 2026) demonstrates a closed feedback loop where VLMs serve as both planners and data-collection oracles, thus efficiently expanding the policy’s manipulation vocabulary with minimal human oversight.

6. Open Challenges and Future Directions

Shared across these InSight efforts are themes of introspection, scalability, and modularity. Future research directions include:

• For introspection frameworks: Integration of active learning, real-time human-in-the-loop control, domain adaptation to hybrid action decoders, and policy-agnostic modules for generalized uncertainty introspection (Karli et al., 1 Oct 2025).
• For haptic sensing: Improvements in mechanical homogeneity, dynamic compensation, real-time slip detection, and adaptation to complex robot morphologies (Sun et al., 2021).
• For astrophysical instrumentation: Enhanced background modeling, joint low/high-energy detector analysis, rapid low-latency target-of-opportunity triggering, and application of machine learning for false positive suppression (Song et al., 2022).
• For VLA skill acquisition: Expansion to higher-DOF robot embodiments, multi-axis primitive acquisition, sim-to-real transfer, and leveraging VLM-based feedback for increased sample efficiency (Wang et al., 23 Jun 2026).

A plausible implication is that introspective, modular design patterns, unifying physics-driven modeling, machine learning, and uncertainty estimation, will continue to accelerate progress across embodied AI, sensing, and scientific observation domains.