Robot Awareness Service (RAS) Overview

• Robot Awareness Service (RAS) is a service abstraction that externalizes latent perception variables into actionable cues for robot, human, and environmental state.
• RAS spans diverse applications including collaborative telepresence, dynamic situational awareness in human–robot teams, metacognitive service routing, V2X systems, and surveillance.
• RAS architectures integrate multimodal sensing, advanced perception algorithms, and coordinated interfaces to reduce cognitive load and improve operational safety and efficiency.

Robot Awareness Service (RAS) denotes a family of service-layer constructs that operationalize awareness in robotics by maintaining, exposing, and acting on state about robots, humans, environments, tasks, and coordination context. Across the literature, the term is used for awareness-augmented collaborative telepresence, dynamic situational awareness in human–robot teams, metacognitive service-robot middleware, V2X facility-layer robot awareness via the Robot Awareness Message (RAM), surveillance-oriented remote monitoring and speech control, real-time awareness of human behaviors from 3D skeletons, and robot-aware visual control policies (Li et al., 2024, Senaratne et al., 15 Jan 2025, Behnke, 25 Jan 2025, Arockiasamy et al., 7 May 2026, Kavimandan et al., 18 Aug 2025, Han et al., 2017, Hu et al., 2021).

1. Scope and terminological range

The literature does not use RAS as a single canonical protocol. Instead, it appears as a recurring service abstraction whose purpose is to make awareness computable, communicable, and actionable. In telepresence robotics, RAS exposes awareness cues for collaborative locomotion; in human–robot teaming, it estimates gaps between required and actual situational awareness; in service robotics, it adds self-monitoring, uncertainty, and risk assessment above habitual control; in V2X systems, it is a facility-layer service realized by RAM; and in surveillance or perception-driven systems, it exposes video, detections, commands, or human-action labels as awareness outputs (Li et al., 2024, Senaratne et al., 15 Jan 2025, Behnke, 25 Jan 2025, Arockiasamy et al., 7 May 2026, Kavimandan et al., 18 Aug 2025, Han et al., 2017, Hu et al., 2021).

Context	RAS meaning	Representative paper
Collaborative telepresence	Awareness services for User–Partner–Environment state and cues	(Li et al., 2024)
Human–robot teams	Dynamic situational awareness estimation and intervention	(Senaratne et al., 15 Jan 2025)
Service robotics	Metacognitive layer with uncertainty, risk, OOD, and planning	(Behnke, 25 Jan 2025)
V2X environments	Facility-layer Robot Awareness Service realized by RAM	(Arockiasamy et al., 7 May 2026)
Surveillance robot	Remote awareness and control via video, detections, speech, and depth	(Kavimandan et al., 18 Aug 2025)
Human behavior recognition	Real-time action-awareness pipeline based on FABL	(Han et al., 2017)
Visual control transfer	Reusable service for robot-aware prediction and planning	(Hu et al., 2021)

A common denominator is that RAS externalizes latent state that would otherwise remain embedded in perception or control loops. This suggests that RAS is best understood as an architectural pattern: a layer that turns awareness variables into services, messages, APIs, cues, or policies suitable for coordination and decision support.

2. Awareness models and state representations

One major formulation is the User–Partner–Environment (U–P–E) awareness model for collaborative locomotion. TeleAware derives four goals from exhibition co-visits: enhance environmental visibility for remote users, support remote users’ perception of the local partner’s location and status, enhance embodied interaction between local and remote users, and support joint referencing of environmental information. The observational basis is explicit: visual perception dominates environmental understanding, with approximately $16.1\%$ of time spent scanning and $70.1\%$ viewing exhibits or people; partner checks occurred at approximately $1.6/\text{min}$; light touch was used by $9/16$ dyads, with an average of $3.3$ touches per $10$ min; gaze following occurred at approximately $1.5/\text{min}$ and gesture indication at approximately $2.5/\text{min}$ (Li et al., 2024).

A second formulation treats RAS as dynamic situational awareness management. In that view, situational awareness follows Endsley’s three-level model—perception, comprehension, and projection—but is divided into two time-varying constructs: required SA and actual SA. Required SA is not “knowing everything”; it is dynamic, context-sensitive, and admits a tolerance band. The formalization introduces $SA_{\min}^{req}(t)$, $SA_{\max}^{req}(t)$, the center level $70.1\%$0, and the gap

$70.1\%$1

Misalignment yields five inefficiencies: SA latency, SA loss, SA inaccuracy, incomplete SA, and excess SA. Their consequences are delayed, missed, faulty, unnecessary actions, confusion, and overload (Senaratne et al., 15 Jan 2025).

A third formulation places RAS inside a cognitive architecture for “conscious” service robots. Here the service continuously maintains a belief over the world and the robot’s internal state, computes uncertainty and risk, detects anomalies and out-of-distribution inputs, reasons causally in a working memory, plans and revises actions, and adapts models or policies online. The mapping is explicitly cognitive: System 1/C0 provides reactive perception and skills, while System 2 adds C1 global availability through working memory and C2 metacognition through confidence, error detection, and self-knowledge (Behnke, 25 Jan 2025).

A fourth formulation appears in robot-aware visual control. There, awareness is the explicit distinction between robot and world in visual observations. The observation model is

$70.1\%$2

with a robot mask

$70.1\%$3

and a world-only image

$70.1\%$4

This factorization supports transferable planning because robot appearance and motion are handled separately from world dynamics (Hu et al., 2021).

A fifth formulation is FABL, where awareness of human behavior is learned directly from 3D skeletal data. FABL defines a regression-like objective with modality-based and joint-based structured sparsity,

$70.1\%$5

so that discriminative feature modalities and discriminative body parts are selected simultaneously (Han et al., 2017).

3. Service decomposition, messaging, and interfaces

In awareness-augmented telepresence, RAS is decomposed into four cooperating subsystems: sensing and data acquisition, perception and context modeling, awareness cue generation and orchestration, and user interfaces and feedback. The data flow is explicit: sensors publish frames, tracker poses, force events, and robot state; perception services detect people and extract keypoints; context modeling fuses these into a U–P–E state; cue orchestration selects and renders overlays, projector rays, and robot turns; operators then drive and indicate through the UI, feeding back into the loop. The proposed services include RAS.VideoService, RAS.TrackingService, RAS.PoseService, RAS.ContextModel, RAS.CueOrchestrator, RAS.OverlayService, RAS.ProjectorService, RAS.EmbodiedInputService, and RAS.MotionService, with concrete interfaces such as GET /api/partner/state, POST /api/projector/ray, and POST /api/robot/yaw (Li et al., 2024).

In surveillance-oriented robotics, RAS is a service layer exposed to phone or browser clients. The available endpoints include /video/live, /detections, /events, /speech/command, /tts/response, and /control/drive. The architecture is split across two Raspberry Pi 4 units: a front server on a differential-drive base connected to camera, microphone, and speaker, and a central server that serves the live feed and runs perception. The end-to-end pipeline is camera/Kinect to FFmpeg video stream to central server to YOLOv3 detection to awareness events, together with microphone to speech recognition to translation to action mapping to GPIO motor control (Kavimandan et al., 18 Aug 2025).

In V2X environments, RAS is a facility-layer service aligned with ETSI ITS-G5 and realized by RAM. RAM is containerized and CAM-aligned, with ItsPduHeader, GenerationDeltaTime, a mandatory BasicContainer, a mandatory RobotHighFrequencyContainer, an optional RobotLowFrequencyContainer, and a mandatory RobotStatusContainer. Optional coordination-related containers include RobotLeaderFollowerOperationContainer, VruClusterInformationContainer, and VruMotionPredictionContainer. RMCS complements RAS through RMCM, whose LeaderManeuverContainer and FollowerManeuverContainer support event-driven maneuver negotiation under explicitly established roles (Arockiasamy et al., 7 May 2026).

In metacognitive service robotics, the reference architecture is organized around perception, a world/causal model, working and episodic memory, a planner, a metacognitive monitor, a safety/risk manager, and an adaptation/meta-learner. The ROS/ROS 2 mapping includes nodes such as perception_node, scene_graph_node, awareness_manager, ood_detector, wm_service, planner_node, risk_manager, adaptation_node, safety_interlock, and hri_gateway; topics such as /awareness/belief, /awareness/uncertainty, and /awareness/anomaly; and services such as /awareness/reset_belief, /planner/replan, /risk/approve_action, /hri/escalate, and /adaptation/update_model (Behnke, 25 Jan 2025).

4. Perception, estimation, and awareness computation

Telepresence-oriented RAS combines multimodal sensing with context inference. TeleAware uses a $70.1\%$6 wide-angle main camera, a top-mounted binocular surveillance camera pair with one PTZ unit of approximately $70.1\%$7 yaw and $70.1\%$8 pitch, HTC Vive trackers on both robot and local user, force sensors mounted as “shoulders” on the robot’s display frame, a floor projector, and bi-directional A/V. Person detection and pose estimation are performed with YOLOX and MediaPipe Pose. Pointing is interpreted as a “visual-touch” line derived from human pose keypoints. Partner state tracking computes distance and movement, for example

$70.1\%$9

with movement states $1.6/\text{min}$0. Smoothed proximity is formalized as

$1.6/\text{min}$1

Cue generation then renders directional arrows, distance bubbles, movement-state indicators, guidance lines, projected floor rays, and automatic yaw motions after shoulder taps (Li et al., 2024).

Dynamic-SA RAS turns awareness estimation into an online inference problem. Required SA is parameterized as $1.6/\text{min}$2, driven by task criticality, safety criticality, time criticality, number of robots per human, time on mission, autonomy capability, robustness, status severity, and role. Actual SA is parameterized as $1.6/\text{min}$3, driven by expertise, mental models, trust calibration, workload, willingness to delegate, communication quality, distractions, distance, information availability, information format quality, and attention allocation. A derived operationalization uses

$1.6/\text{min}$4

and event-level latency

$1.6/\text{min}$5

The paper emphasizes gaze as a useful process index and proposes attention coverage, fixation entropy, revisit latency, alert handling, panel switches, query rates, override frequency, task queue length, manual control time fraction, and communication QoS as measurable proxies (Senaratne et al., 15 Jan 2025).

Metacognitive RAS extends estimation to belief, uncertainty, and risk. The decision process is modeled as a POMDP,

$1.6/\text{min}$6

with belief update

$1.6/\text{min}$7

Risk-sensitive reasoning includes $1.6/\text{min}$8, chance constraints $1.6/\text{min}$9, and interventional queries $9/16$0. Uncertainty and OOD detection are computed through predictive entropy $9/16$1, mutual information $9/16$2, Mahalanobis distance $9/16$3, and change-point detection via CUSUM,

$9/16$4

with an alarm if $9/16$5 (Behnke, 25 Jan 2025).

In surveillance robotics, awareness computation is simpler but still serviceable. YOLOv3 is trained on the COCO detection dataset and the ImageNet classification dataset, with confidence defined as

$9/16$6

and class-specific confidence at test time as

$9/16$7

The standard IoU is

$9/16$8

Voice interaction uses speech_recognition, googletrans, and pyttsx3, while Kinect RGB-D provides obstacle cues for indoor navigation (Kavimandan et al., 18 Aug 2025).

FABL provides a different awareness computation path: from 3D skeleton streams to action labels. It uses four simple feature modalities—spatial joint displacement relative to the torso, temporal joint displacement, long-term temporal joint displacement, and spatial joint distance to torso center—and solves a convex optimization problem with an iterative reweighted scheme. The test-time decision rule is

$9/16$9

The reported processing speeds for feature computation and classification are $3.3$0 Hz on MSR Action3D, $3.3$1 Hz on CAD-60, and $3.3$2 Hz on Baxter (Han et al., 2017).

5. Interaction, control, and coordination regimes

Telepresence RAS is explicitly designed for collaborative locomotion under dynamic roles. The remote workstation UI combines main camera video with overlays, a secondary PTZ panel, click-to-project for shared reference, and keyboard WASD driving. The local side provides physical tapping affordances, projected rays on the floor, and natural co-locomotion with the robot. The control paradigm is “Human-in-the-loop” manual drive by the remote user, with local embodied interrupts through tap-to-turn and negotiated reference establishment. The system accommodates dynamic leadership; in the experiment, remote leaders completed guided routes faster on average ($3.3$3 s) than local leaders ($3.3$4 s) (Li et al., 2024).

Dynamic-SA RAS generalizes interaction beyond telepresence by linking gap types to interventions. Its policy manager selects alerts, explanations, summaries, confirmations, and autonomy advice as a function of $3.3$5, severity, communication quality, attention state, and workload. The derived decision objective is

$3.3$6

with $3.3$7 subject to gap state and context. Triggering examples include priority alerts under likely SA loss or latency, short explanations under incomplete SA or confusion, counterfactual explanations when inaccuracy is suspected, big-picture refresh prompts under excess SA or tunnel vision, and conservative autonomy takeover suggestions when workload is high and robot confidence supports it (Senaratne et al., 15 Jan 2025).

V2X RAS couples awareness to explicit coordination. Roles are established through helpStatus handshakes in RAM, and maneuvers are executed through RMCM under a formally specified finite-state model with states

$3.3$8

and events

$3.3$9

The proof-of-concept scenario uses a humanoid robot as leader and a quadruped as follower to assist a pedestrian during a road crossing, without centralized infrastructure or prior pairing (Arockiasamy et al., 7 May 2026).

Surveillance RAS supports interaction through spoken commands mapped to differential-drive control. The control path is speech to translation or normalization to action mapping to GPIO motor-driver signals, and the system is reported to “translate them to actions without manual control.” Responses are synthesized through pyttsx3 and played back through speakers (Kavimandan et al., 18 Aug 2025).

Robot-aware visual-control RAS links awareness directly to model-predictive control. The robot dynamics module predicts

$10$0

the world dynamics module predicts

$10$1

and planning minimizes a decomposed cost

$10$2

over action sequences using CEM. This prevents robot pixels from dominating pixel costs and enables plug-and-play transfer across robots (Hu et al., 2021).

6. Empirical outcomes, misconceptions, and open issues

The most directly evaluated human-facing RAS is TeleAware. Its controlled experiment used a $10$3 mixed design with robot system (TeleAware vs Standard) and task role (Leader vs Follower), $10$4 participants in $10$5 dyads, and four rounds per dyad. Measures included task completion time, tracker-based trajectories and proximity, follower memory questionnaire, Social Presence scales, NASA-TLX with five seven-point subscales, and IOS. Relative to the standard robot, TeleAware yielded significantly lower cognitive demand ($10$6), lower frustration ($10$7), higher IOS for remote users ($10$8) and local users ($10$9), and a smaller mean inter-partner distance in the subset with available data: $1.5/\text{min}$0 m versus $1.5/\text{min}$1 m ($1.5/\text{min}$2). It also improved multiple mutual-awareness and social-presence items, while showing no significant differences in overall task time or follower memory accuracy across systems (Li et al., 2024).

V2X-oriented RAS was evaluated both in simulation and in a real-world proof of concept. The ITS-G5 stack used IEEE 802.11p/EDCA at $1.5/\text{min}$3 GHz, a $1.5/\text{min}$4 MHz channel, and $1.5/\text{min}$5 mW transmit power. In a $1.5/\text{min}$6 Manhattan-grid simulation with observation radii $1.5/\text{min}$7, non-V2X VRU coverage increased from $1.5/\text{min}$8 with no robots to approximately $1.5/\text{min}$9–$2.5/\text{min}$0 with $2.5/\text{min}$1 robot at $2.5/\text{min}$2 m and approximately $2.5/\text{min}$3–$2.5/\text{min}$4 with $2.5/\text{min}$5 robots at $2.5/\text{min}$6 m. Mean channel-busy-ratio reduction reached up to $2.5/\text{min}$7 at $2.5/\text{min}$8 m with $2.5/\text{min}$9 robots and $SA_{\min}^{req}(t)$0 pedestrians. In the pedestrian-assistance proof of concept, the initialPos maneuver had $SA_{\min}^{req}(t)$1 s and $SA_{\min}^{req}(t)$2 s, while the Move maneuver had $SA_{\min}^{req}(t)$3 s, $SA_{\min}^{req}(t)$4 s, and $SA_{\min}^{req}(t)$5 s (Arockiasamy et al., 7 May 2026).

FABL was evaluated on MSR Action3D, CAD-60, and a Baxter assistive-living scenario. Reported accuracies were $SA_{\min}^{req}(t)$6 on MSR Action3D, $SA_{\min}^{req}(t)$7 on CAD-60, and $SA_{\min}^{req}(t)$8 on Baxter. The corresponding baselines—feature-learning-only, body-part-only, or no-regularization variants—were lower in each benchmark. The optimization algorithm is stated to have a theoretical guarantee to find the optimal solution, and the reported speeds place recognition in the $SA_{\min}^{req}(t)$9 Hz regime for feature computation plus classification (Han et al., 2017).

Robot-aware visual control reports large transfer gains. In simulation, zero-shot Fetch pushing reached $SA_{\max}^{req}(t)$0 success for RA/RA versus $SA_{\max}^{req}(t)$1 for VF+State/Pixel and $SA_{\max}^{req}(t)$2 for CycleGAN+VF+State/Pixel; Fetch pick-and-place reached $SA_{\max}^{req}(t)$3 for RA/RA versus $SA_{\max}^{req}(t)$4 for VF+State/Pixel. In real pushing on Franka, RA/RA achieved $SA_{\max}^{req}(t)$5 success when trained on a single robot and $SA_{\max}^{req}(t)$6 with multi-robot pretraining, whereas VF+State/Pixel ranged from $SA_{\max}^{req}(t)$7 to $SA_{\max}^{req}(t)$8. A cost-decomposition ablation reported $SA_{\max}^{req}(t)$9 success for RA cost versus $70.1\%$00 for pixel cost, and two-view RA/RA improved pick-and-place from $70.1\%$01 to $70.1\%$02 (Hu et al., 2021).

Several recurrent misconceptions are explicitly challenged by the literature. One is that the best situational awareness is “knowing everything at all times”; the interview study rejects this and argues that the right SA is dynamic and context-dependent (Senaratne et al., 15 Jan 2025). Another is that awareness cues necessarily improve task throughput; TeleAware improved workload, social proximity, and social presence, but not overall time or follower memory accuracy (Li et al., 2024). A third is that RAS is already a stable standard across domains; only the V2X formulation is standards-aligned at the facility layer, whereas other uses remain architectural proposals or system designs (Arockiasamy et al., 7 May 2026).

Open issues are equally consistent. Generalization beyond dyads and crowded venues remains open in collaborative telepresence, and the contribution of individual TeleAware features was not isolated (Li et al., 2024). Dynamic-SA estimators, including the precise $70.1\%$03, $70.1\%$04, weights, and thresholds, require empirical tuning and validation in controlled and field experiments (Senaratne et al., 15 Jan 2025). Metacognitive RAS adds compute and latency overhead, and LLM oracles can hallucinate unless kept inside a risk-gated loop (Behnke, 25 Jan 2025). Surveillance-oriented RAS does not specify quantitative latency figures, mAP, NMS, or access-control policy (Kavimandan et al., 18 Aug 2025). V2X RAS leaves encoder details, exact RAM/RMCM byte sizes, and richer uncertainty fields for future work (Arockiasamy et al., 7 May 2026). Taken together, these works indicate that RAS is not a single closed specification but a design space centered on exposing awareness in forms that are actionable under role, workload, uncertainty, and safety constraints.