PhysiAgent: Adaptive AI for Robotics & Therapy

• PhysiAgent is defined as a set of agentic AI systems that decompose tasks into specialized modules, enabling persistent state, feedback, and adaptive control.
• In robotics, it enhances task execution by integrating a high-level VLM planner with a low-level VLA executor, supported by Monitor, Memory, and Self-Reflection modules for real-time error correction.
• In physiotherapy, it operates as a closed-loop multi-agent system that translates clinical prescriptions into personalized video exercises with real-time pose feedback and corrective guidance.

Searching arXiv for “PhysiAgent” and closely related entries to ground the article in current literature. PhysiAgent is a name used in recent arXiv literature for distinct agentic AI systems that operate in physically grounded environments. In one usage, it denotes an embodied agent framework for real-world robotics that inserts Monitor, Memory, and Self-Reflection modules between a high-level Vision-LLM (VLM) Planner and a low-level Vision-Language-Action (VLA) executor (Wang et al., 29 Sep 2025). In another, it denotes a closed-loop Multi-Agent System (MAS) for personalized physiotherapy that links clinical text parsing, personalized video generation, pose estimation, and deterministic corrective feedback through a shared patient state (Dharmaratnakar et al., 22 Apr 2026). The shared naming reflects a common architectural theme—explicit decomposition into specialized components, persistent state, and feedback-driven adaptation—while the target domains, evaluation protocols, and operational constraints differ substantially.

1. Disambiguation and scope

In current usage, “PhysiAgent” does not identify a single canonical framework. At least two separate systems have appeared under that name.

Usage of “PhysiAgent”	Paper	Core setting
Embodied agent framework in physical world	(Wang et al., 29 Sep 2025)	Real-world robotic manipulation with VLM–VLA coordination
Personalized physiotherapy framework	(Dharmaratnakar et al., 22 Apr 2026)	Tele-rehabilitation with clinical NLP, video synthesis, pose tracking, and feedback

The robotic PhysiAgent is framed as an embodied agent scaffold that addresses ineffective collaboration and grounding challenges in existing VLM–VLA combinations by using monitor, memory, self-reflection mechanisms, and lightweight off-the-shelf toolboxes (Wang et al., 29 Sep 2025). The physiotherapy PhysiAgent is framed as a four-agent closed loop intended to address low at-home compliance by generating personalized exercise videos and issuing real-time pose corrections (Dharmaratnakar et al., 22 Apr 2026).

A recurrent misconception is to treat the name as denoting a single research lineage. The literature instead uses it for two domain-specific architectures whose overlap is primarily organizational: both rely on specialized submodules, intermediate structured representations, and recurrent feedback, but they solve different classes of problems.

2. Embodied PhysiAgent for robotics

The embodied PhysiAgent treats an embodied VLM–VLA stack as a single, self-regulating agent by inserting three scaffold modules—Monitor, Memory, and Self-Reflection (Reflector)—between a high-level VLM “Planner” and a low-level VLA executor, plus a small set of lightweight, off-the-shelf toolboxes (Wang et al., 29 Sep 2025). The component definitions are explicit:

• Planner $(F_p)$ decomposes a language instruction $l \in L$ and the history of past subgoals $\{l^1,\dots,l^k\}$ into the next executable subinstruction $l^t$:

$l^t = F_p(o_t, \{l^i\}_{i<t}, l)$

• Monitor $(F_m)$ ingests a short image buffer $(o_{t-h}, o_t)$, the current subinstruction $l^t$, and any accumulated visual constraints $C$, and returns a discrete progression flag $$p_t \in \{\mathrm{Hinder}, \mathrm{Ongoing}, \mathrm{Failure}, \mathrm{Done}\}$$:

$l \in L$0

• Reflector $l \in L$1 checks $l \in L$2 against the visual transition and, if inconsistent, emits a corrective visual constraint $l \in L$3:

$l \in L$4

• Short Memory $l \in L$5 stores step-level tuples $l \in L$6.
• Long Memory $l \in L$7 stores episodic summaries $l \in L$8 where $l \in L$9.
• Embodied Toolbox includes rule-based or lightweight neural tools such as get_wrist_view, backtrack, and replan.

The information flow is bidirectional. The forward path is user instruction $\{l^1,\dots,l^k\}$0 Planner $\{l^1,\dots,l^k\}$1 sub-instruction $\{l^1,\dots,l^k\}$2 $\{l^1,\dots,l^k\}$3 VLA action $\{l^1,\dots,l^k\}$4. The backward path is VLA rollout $\{l^1,\dots,l^k\}$5 Monitor $\{l^1,\dots,l^k\}$6 progression flag $\{l^1,\dots,l^k\}$7 $\{l^1,\dots,l^k\}$8 Reflector $\{l^1,\dots,l^k\}$9 new constraint $l^t$0 $l^t$1 updated constraint set $l^t$2, which then conditions subsequent Monitor calls. Memory modules continuously summarize prior transitions for later Planner and Reflector calls.

This design is notable because it does not reduce the VLA to a purely reactive controller or the VLM to a one-shot planner. Instead, the scaffold forces repeated grounding of symbolic subgoals against visual evidence, with explicit correction of monitoring errors. A plausible implication is that the framework is less a conventional multi-agent system than a modular control loop in which different VLM calls are assigned persistent epistemic roles.

3. Feedback, prompt scaffolding, and self-regulation

Prompt scaffolding is central to the robotic PhysiAgent. Each VLM call is dynamically conditioned by Memory and recent feedback, rather than being driven by a static prompt (Wang et al., 29 Sep 2025). For the Monitor, the generic template is:

$l^t$3

After a new reflection $l^t$4, the next prompt is reconfigured to include it:

$l^t$5

The paper’s concrete example concerns the subtask “move the shrimp to plate.” Before adaptation, the Monitor asks whether the transition is DONE, FAILURE, ONGOING, or HINDER. After adaptation, the prompt is augmented with the visual constraint “Shrimp must be on the plate, not table.” This is a direct instance of feedback-conditioned grounding rather than static instruction following.

The technical summary further formalizes “real-time proficiency feedback” by mapping discrete flags and contextual features to a continuous score $l^t$6, with one simple instantiation given as

$l^t$7

Self-reflection is then written as an adaptation in the Monitor’s internal VLM parameters through a reflection loss

$l^t$8

with gradient updates

$l^t$9

The qualitative case studies clarify what this machinery is meant to repair. In a false-positive case, the Monitor labels “Done” even though the shrimp remains on the table; the Reflector then adds the constraint “Shrimp must be placed on the plate rather than the table,” which prevents future misclassification. In a false-negative case, a correct reach motion is flagged “Failure,” and the Reflector adds “Movement towards shrimp is positive progression,” allowing subsequent monitoring to recognize partial progress. The operational point is that the framework attempts to improve not only planning and acting, but the accuracy of its own internal evaluators.

4. Empirical performance in real-world manipulation

The robotic PhysiAgent is evaluated in real-world tabletop manipulation with a 6-DOF AIRBOT arm and three RGB cameras, using two VLAs—RDT-1B and Diffusion Policy—each fine-tuned on 150 human demonstrations per primitive task (Wang et al., 29 Sep 2025). The tasks are grouped by complexity:

• Level 1 (2–3 subtasks): “Grab foods that contain dietary fiber”; “Grab foods that contain protein and fat”.
• Level 2 (5 subtasks): “Cook a meal (prepare breakfast)”.

The reported metrics are success rate, completion time in VLA steps, and error rate. Baselines are a Vanilla VLA end-to-end policy without high-level reasoning and a hierarchical method with a static VLM planner and a human-in-loop variant for re-planning.

Performance gains are substantial. On the “Fiber” task, success rate is 94% ± 2.1 for PhysiAgent, versus 62% ± 4.2 for Vanilla VLA and 81% ± 3.5 for the hierarchical baseline; completion time is 29.2 ± 2.4 steps versus 48.3 ± 5.1 and 37.8 ± 3.8. On “Protein+Fat”, success rate is 91% ± 3.0 versus 55% ± 6.0 and 78% ± 4.1, with completion time 42.7 ± 3.6 versus 72.6 ± 6.8 and 58.4 ± 5.0. On “Cook brunch”, success rate is 88% ± 3.2 versus 40% ± 5.5 and 69% ± 4.7, with completion time 86.5 ± 7.1 versus 140.2 ± 10.4 and 112.3 ± 8.9. Paired $l^t = F_p(o_t, \{l^i\}_{i<t}, l)$0-tests are reported to confirm that PhysiAgent’s success rates exceed both baselines with $l^t = F_p(o_t, \{l^i\}_{i<t}, l)$1.

The paper’s limitations are equally explicit. The tested domains are limited to tabletop manipulation; extension to navigation or multi-agent settings remains unverified. Self-reflection updates add approximately 1–2 s latency per step. The current VLMs are proprietary, specifically Gemini 2.0 and GPT-4o. These constraints matter because the framework’s gains are coupled to extra inference overhead and to dependencies that may constrain reproducibility and fine-tuning.

5. PhysiAgent for personalized physiotherapy

A second system using the name PhysiAgent is a closed-loop MAS for tele-rehabilitation, organized around four specialized micro-agents interacting via a shared state object called PatientState $l^t = F_p(o_t, \{l^i\}_{i<t}, l)$2 (Dharmaratnakar et al., 22 Apr 2026). The agents are:

• Clinical Extraction Agent $l^t = F_p(o_t, \{l^i\}_{i<t}, l)$3: input unstructured clinical prescription text $l^t = F_p(o_t, \{l^i\}_{i<t}, l)$4; output structured kinematic constraints $l^t = F_p(o_t, \{l^i\}_{i<t}, l)$5 as JSON.
• Video Synthesis Agent $l^t = F_p(o_t, \{l^i\}_{i<t}, l)$6: input constraints $l^t = F_p(o_t, \{l^i\}_{i<t}, l)$7; output a personalized demonstration video URL video_url.
• Vision Processing Agent $l^t = F_p(o_t, \{l^i\}_{i<t}, l)$8: input continuous RGB frames $l^t = F_p(o_t, \{l^i\}_{i<t}, l)$9; output a real-time pose vector $(F_m)$0 as 3D keypoints.
• Diagnostic Feedback Agent $(F_m)$1: input live joint angles $(F_m)$2 and target constraints $(F_m)$3; output feedback message $(F_m)$4.

The Clinical Extraction Agent is formalized by

$(F_m)$5

where each constraint is

$(F_m)$6

with joint, axis, angular bounds, and urgency. The implementation uses Named-Entity Recognition and Dependency Parsing, template-based numeric extraction with regex for “$(F_m)$7” patterns, and rule-based disambiguation. The output JSON example is: $l^t$1

The physiotherapy framework defines explicit safety constraints. For each joint-axis pair $(F_m)$8,

$(F_m)$9

A range-of-motion velocity constraint is also given:

$(o_{t-h}, o_t)$0

where $(o_{t-h}, o_t)$1 is a maximum safe joint velocity, with the example value 0.5 deg/s.

The Video Synthesis Agent is described as a conditional video diffusion model inspired by Ho et al. 2022, with a U-Net backbone, temporal-attention layers, cross-attention conditioning on joint-angle sequences and a background image embedding, and learned embeddings for each constraint key. Home-environment customization proceeds by uploading a panoramic room image, extracting a floor/obstacle map $(o_{t-h}, o_t)$2 with a segmentation model, and conditioning the diffusion model on $(o_{t-h}, o_t)$3 so the avatar never clips furniture or exits the field of view.

The Vision Processing Agent uses MediaPipe Pose. Frames are resized to 256×256, normalized to [–1,1], and mapped to 33 landmarks $(o_{t-h}, o_t)$4. Joint angles are then computed from keypoints, for example elbow flexion:

$(o_{t-h}, o_t)$5

The Diagnostic Feedback Agent applies deterministic threshold rules with tolerance $(o_{t-h}, o_t)$6. If $(o_{t-h}, o_t)$7, the message is “Warning: Too far. Reduce your angle.” If $(o_{t-h}, o_t)$8, it is “Raise slightly to reach target.” If $(o_{t-h}, o_t)$9, it is “Perfect form. Hold.” The formal decision function is

$l^t$0

The prototype implementation initializes a shared Python dictionary $l^t$2 then uses GPT-4 plus regex for constraint extraction, a locally hosted diffusion server for video generation, and MediaPipe-based pose tracking in a web UI loop. Reported prototype results are Pose Estimation Latency: 28 ms (target < 50 ms), Joint-Angle Error: ±3.2° (target < 5°), Clinical Parsing Accuracy: 96.5% (target > 95%), and Video Synthesis Time: 45 s (target < 60 s). A proposed clinical study is a crossover trial versus a standard pre-recorded library with n=60 post-op rotator-cuff patients, measuring execution accuracy, compliance rate, and patient satisfaction, with the stated hypothesis of at least 15% improvement in execution accuracy and at least 20% increase in compliance over static libraries.

6. Relationship to adjacent agentic systems

The PhysiAgent name sits within a broader cluster of similarly named systems, but the surrounding literature spans several distinct research programs. “Physio” is a retrieval-augmented, chat-based physiotherapy assistant built around GPT-4 and a curated medical knowledge base, using BM25 over curated webpages, exercise collections, and DrugBank over-the-counter entries; it is not presented as an embodied closed-loop control scaffold (Almeida et al., 2024). “Vivaldi” is a role-structured multi-agent system for explaining multivariate physiological time series in emergency medicine, with TriageAgent, DoctorAgent, ConsultantAgent, CoderAgent, and SynthesizerAgent coordinated through a shared memory buffer (Gabrielli et al., 4 Mar 2026). “PhysAgent” is a simulator-in-the-loop multi-agent framework for automated physics-based 4D synthesis, built around a Semantic Agent, a Force Field Skill Library, and trajectory-grounded Refine Agents (Lv et al., 7 Jun 2026). “PhysicalAgent” is a robotic manipulation framework that combines VLM planning, diffusion-based video generation, a video-to-control adapter, and iterative failure-driven replanning (Lykov et al., 17 Sep 2025).

These neighboring systems clarify what is and is not distinctive about PhysiAgent. The robotic PhysiAgent shares with PhysicalAgent an interest in grounded execution and recovery, but it emphasizes VLM–VLA self-regulation through Monitor, Memory, and Reflector modules rather than diffusion-video reasoning (Wang et al., 29 Sep 2025). The physiotherapy PhysiAgent shares with Physio and Vivaldi a healthcare setting, but it is organized as a four-agent loop that combines clinical NLP, generative video, pose estimation, and deterministic thresholded feedback rather than retrieval-augmented dialogue or time-series explanation (Dharmaratnakar et al., 22 Apr 2026).

Taken together, the literature suggests that “PhysiAgent” is best understood not as a single benchmarked architecture, but as a label applied to agentic systems that emphasize physically grounded adaptation. In robotics, that adaptation is directed toward improving collaboration between generalized VLM reasoning and specialized VLA execution. In physiotherapy, it is directed toward safe personalization, real-time form assessment, and structured translation of clinical prescriptions into executable kinematic constraints.