Goal-oriented Communication for Fast and Robust Robotic Fault Detection and Recovery

Abstract: Autonomous robotic systems are widely deployed in smart factories and operate in dynamic, uncertain, and human-involved environments that require low-latency and robust fault detection and recovery (FDR). However, existing FDR frameworks exhibit various limitations, such as significant delays in communication and computation, and unreliability in robot motion/trajectory generation, mainly because the communication-computation-control (3C) loop is designed without considering the downstream FDR goal. To address this, we propose a novel Goal-oriented Communication (GoC) framework that jointly designs the 3C loop tailored for fast and robust robotic FDR, with the goal of minimising the FDR time while maximising the robotic task (e.g., workpiece sorting) success rate. For fault detection, our GoC framework innovatively defines and extracts the 3D scene graph (3D-SG) as the semantic representation via our designed representation extractor, and detects faults by monitoring spatial relationship changes in the 3D-SG. For fault recovery, we fine-tune a small LLM (SLM) via Low-Rank Adaptation (LoRA) and enhance its reasoning and generalization capabilities via knowledge distillation to generate recovery motions for robots. We also design a lightweight goal-oriented digital twin reconstruction module to refine the recovery motions generated by the SLM when fine-grained robotic control is required, using only task-relevant object contours for digital twin reconstruction. Extensive simulations demonstrate that our GoC framework reduces the FDR time by up to 82.6% and improves the task success rate by up to 76%, compared to the state-of-the-art frameworks that rely on vision LLMs for fault detection and LLMs for fault recovery.

• The paper introduces a novel goal-oriented framework that jointly optimizes semantic data transmission, lightweight inference, and precision control, reducing fault detection and recovery time by up to 82.6%.
• It leverages dual-branch semantic extraction from RGB-D imagery and point clouds to rapidly detect task-level and motion-level anomalies with event-driven communication.
• The framework integrates adaptive computation offloading and a fine-tuned small language model to achieve 76% improvement in task success and 92% lower inference latency.

Goal-Oriented Communication for Fast and Robust Robotic Fault Detection and Recovery

Overview and Motivation

The study "Goal-oriented Communication for Fast and Robust Robotic Fault Detection and Recovery" (2601.18765) presents a comprehensive framework for agile robotic fault detection and recovery (FDR) in industrial autonomy. Autonomous robots in smart factories are increasingly expected to function in uncertain, dynamic environments, demanding FDR systems that are both low-latency and resilient to disturbances. Prevailing FDR solutions either rely on frequent transmission of high-volume sensor data for remote inference or employ computationally intensive LLMs for recovery, both incurring substantial communication and inference delays. Critically, existing frameworks optimize components of the communication-computation-control (3C) loop in isolation, rather than in service of downstream FDR objectives.

This paper introduces a novel Goal-oriented Communication (GoC) architecture that jointly designs the communication, reasoning, and control pipeline—explicitly optimizing semantic data representations, lightweight inference models, and precision control modalities for FDR tasks. By tailoring both what is communicated and when, and by leveraging expressive representations and adaptive control strategies, the GoC approach fundamentally reduces FDR time and enhances task completion reliability.

Technical Contributions

Semantic Representation and Scene Understanding

The GoC framework posits dual-branch semantic extraction, operationalizing distinct pathways for task-level and motion-level faults:

• Task-level branch: Utilizes real-time RGB-D imagery to generate a 3D scene graph (3D-SG), where nodes denote objects and edges encode spatial relationships. Extraction leverages YOLO-seg for object segmentation, followed by graph construction via a Triplet Graph Convolutional Network (TripletGCN) [triplet]. Only fault-triggered, compact 3D-SG data are transmitted, drastically reducing uplink bandwidth compared to continuous full sensor streams.
• Motion-level branch: On imminent collision or fine-grained uncertainty, edge points are selected from local point clouds via attention-based sampling [10204840], followed by B-spline curve fitting for contour reconstruction. Only these critical boundary points are uploaded for lightweight digital twin synthesis.

This strict semantic bottleneck ensures communication efficiency, supporting rapid remote reasoning while maintaining the fidelity required for both symbolic and geometric fault analysis.

Fault Detection Paradigm

Faults are categorized according to their propagation in the 3D-SG:

• Task-level faults: Failures where scene transitions (e.g., "parcel grasped by robot") do not materialize post-action, detected by absence or mismatch of expected graph relations.
• Motion-level faults: Emergent threats (e.g., "robot next to human") signaled by anomalous proximity relations, detected via geometric deviations in the evolving 3D-SG and further validated by edge point transmission.

Transmission is strictly event-driven; semantic updates are communicated only upon detected anomalies, mitigating latency inherent to periodic data uploads.

Adaptive Computation Offloading

To address fluctuating wireless conditions and device compute heterogeneity, GoC incorporates an adaptive offloading scheduler. By explicitly modeling wireless channel parameters (Nakagami-$m$ fading, path-loss, bandwidth thresholds), the system dynamically chooses between local and remote processing of semantic representation extraction, minimizing detection latency as a function of available resources. A bandwidth-thresholded strategy ensures local inference for low-bandwidth regimes, hybrid processing when intermediate, and full edge offload at high bandwidth.

Recovery: Small LLM Pipeline

Rejecting the latency overhead of LLMs, GoC adopts a tailored Small LLM (SLM) for fault recovery:

• SLM selection and fine-tuning: Pretrained Llama-3.2-1B/11B models [llama] are fine-tuned via Low-Rank Adaptation (LoRA) [lora] for structured recovery motion generation, optimizing only adapter layers within the attention mechanisms; this enables high inference throughput and rapid deployment.
• Knowledge distillation: SLM capabilities are augmented via online distillation from GPT-4o, leveraging teacher-forced cross-entropy supervision on failed SLM outputs to better generalize recovery strategies for complex, unforeseen scenarios.
• Two-way control: For motion-level corrections, SLM-generated waypoint sequences are iteratively verified in a digital twin reconstructed from edge points, allowing for prompt refinement before execution.

Control and Digital Twin

Unlike full-fidelity digital twins [8477101], the GoC paradigm reconstructs only the task-relevant contours necessary for local path planning and collision avoidance, leveraging the semantic representation pipeline to minimize redundancy and transmission volume. This enables fast simulation and iterative motion validation for high-precision recovery, without incurring significant overhead.

Empirical Results

Extensive robotic simulations (MuJoCo [mujoco]) spanning workpiece sorting, grocery packing, and parcel palletizing validate the GoC framework against key contemporary baselines:

• Textual constraint-based SOTA [10802284, 10990233]: Periodic image transmission, VLM verification, LLM recovery.
• Spatial constraint-based SOTA [huang2024rekep]: Keypoint extraction, spatial reasoning, LLM recovery.

Key findings:

• GoC achieves up to 82.6% reduction in FDR time and up to 76% absolute improvement in task success rate over SOTA frameworks.
• For task-level faults, the 3D-SG enables rapid, interpretable detection and SLM-based replanning.
• For motion-level faults, edge point transmission and digital twin verification yield significant geometric precision and mitigate failure propagation.
• SLM fine-tuning and distillation provide LLM-comparable reasoning with 92% lower inference latency; ablation confirms necessity of both LoRA and teacher-driven distillation for high reliability without increased computational cost.
• The adaptive offloading scheme robustly balances local and remote computing modalities, maintaining low FDR latency across bandwidth regimes.

Implications and Future Outlook

The GoC framework exemplifies the efficacy of holistic, task-tailored system design—jointly optimizing semantic communication, cognitive inference, and actuator control for mission-critical FDR. The shift from raw data transmission to compact, expressive semantic representations substantially reduces bandwidth and enables real-time responsiveness, a crucial requirement for safe human-robot collaboration and deployment in bandwidth-constrained industrial settings.

The adoption of SLMs for real-time robotic reasoning demonstrates the growing viability of model distillation and parameter-efficient learning in domain-specific contexts, challenging prevailing reliance on large, slow LLMs. Furthermore, the utility-focused digital twin, reconstructing only task-relevant geometries, offers a paradigm for scalable simulation and control that could extend across broader robotic manipulation and navigation tasks.

Going forward, extending the GoC approach to more generalizable semantic abstraction, integrating multimodal sensory fusion, and leveraging continual online distillation for evolving operational domains represent compelling directions, as does the investigation of distributed, decentralized reasoning architectures for multi-robot teams [roco, 10610676]. Industrial robotics stands to benefit from architectures that explicitly align data, inference, and control with downstream utility, promising safer, more efficient autonomy.

Conclusion

This work rigorously demonstrates that goal-oriented semantic communication, model-efficient reasoning, and utility-focused control provide a superior platform for fast, robust robotic fault detection and recovery. The GoC framework, advancing far beyond periodic raw data transmission and monolithic inference, delivers significant gains in latency and reliability through joint system optimization, establishing a principled blueprint for next-generation industrial autonomy (2601.18765).