Human-Aware Dynamic Control

• Human-aware dynamic control modules are systems that integrate dynamic strategies with explicit human cues such as motion, intent, and safety constraints.
• They employ modular architectures that decompose perception, human-state estimation, and control synthesis to enable adaptive, real-time behavior in varied applications.
• These modules leverage techniques like MPC, QP, and barrier functions to facilitate human-robot co-manipulation, adaptive assistance, and shared autonomy with robust safety guarantees.

Human-aware dynamic control modules are control components that couple dynamic control with explicit representations of human motion, intent, preferences, cognitive availability, or appearance, so that the controlled system adapts its behavior in real time while preserving safety, feasibility, or identity. In published work, the term is instantiated in diffusion-based human image animation, visual navigation among humans, adaptive assistance, mixed-initiative teleoperation, human–robot co-manipulation, legged shared autonomy, and human-centred cyber-physical systems (Chang et al., 17 Jan 2025, Tolani et al., 2020, Qin et al., 2024, Petousakis et al., 2021, Shao et al., 2024, Sambhus et al., 26 Sep 2025).

1. Definition and cross-domain scope

The phrase denotes a family of modules rather than a single algorithm. In diffusion-based animation, it refers to unified control over full-body pose, identity-disentangled facial expressions and head motion, and environment dynamics consistent with subject appearance and scene context. In robotics, it denotes modules that predict human-aware waypoints, infer human utilities, estimate human intent, adjust the level of automation, or enforce safety around humans through optimization, barrier functions, or shared-control arbitration. In human-centred cyber-physical systems, it can denote supervisory shared-control components that infer a finite-state abstraction of human behavior, synthesize correct-by-construction safety strategies, and dynamically adjust the level of automation (Chang et al., 17 Jan 2025, Tolani et al., 2020, Qin et al., 2024, Hajnorouzi et al., 18 Nov 2025).

The historical trajectory visible in the literature moves from explicit model-based HRI formulations toward richer multimodal and probabilistic designs. Early work cast HRI scenarios as linear dynamical systems with mixed-integer constraints, using MPC to regulate productivity, workload, connection, or awkwardness (Jorgensen et al., 2017). Subsequent systems learned human-aware waypoints from MPC supervision for navigation among humans (Tolani et al., 2020), introduced modular DEC-based posture control in which one DEC module controls one DOF and modules of neighboring joints are synergistically interconnected (Lippi et al., 2021), and added cognitive-availability-aware mixed initiative for LOA switching (Petousakis et al., 2021). More recent work integrates probabilistic human forecasting with Control Barrier Functions (Busellato et al., 28 Aug 2025), online parameter adaptation inside nonlinear MPC for quadrupedal shared autonomy (Sambhus et al., 26 Sep 2025), context-aware adaptive shared control with multimodal authority prediction and bidirectional haptics (Wang et al., 15 Mar 2026), and uncertainty-aware or utility-aware assistive control (Qin et al., 2024, Shao et al., 2024).

A common misconception is that human-aware dynamic control always requires explicit human trajectory prediction. Several systems do use explicit prediction or forecasting, but others instead map current visual cues, cognitive cues, or assistance signals directly into control-relevant variables, or encode human-awareness through costs, constraints, and supervision (Tolani et al., 2020, Petousakis et al., 2021, Busellato et al., 28 Aug 2025).

2. Recurrent architectural pattern

Across domains, these modules typically decompose into perception or estimation, intermediate human-aware representation, control synthesis, and execution. In visual navigation among humans, the perception module takes a monocular, first-person RGB image, robot proprioception, and an egocentric goal, and outputs a waypoint $\hat{w}_t = (\hat{x}_t,\hat{y}_t,\hat{\theta}_t)$ that is then tracked by spline planning and LQR. In HARMONIOUS, multimodal perception converts RGB-D, proximity, and tactile signals into projected collision points on the robot surface, which become linear inequality constraints in a QP motion controller. In the cognitive-availability-aware mixed-initiative controller, webcam-based head yaw is mapped into fuzzy cognitive-availability inputs for LOA arbitration (Tolani et al., 2020, Rozlivek et al., 2023, Petousakis et al., 2021).

The intermediate representation is often the decisive design choice. Some systems use waypoints, some use finite-state abstractions, some use predicted future trajectories, some use utility parameters, and some use learned authority weights. Examples include the anticipation module and utility module in adaptive assistance, deterministic Mealy abstractions learned from ACT-R simulations for shared-control analysis, DS parameter particles for object co-manipulation, and per-manipulator authority chunks $\hat{\alpha}_{t,i,k}$ for bimanual magnetic micromanipulation (Qin et al., 2024, Hajnorouzi et al., 18 Nov 2025, Shao et al., 2024, Wang et al., 15 Mar 2026).

The control layer then embeds that representation into a dynamics-aware optimizer or arbitration mechanism. The literature includes spline planning with LQR tracking, DWA/DWB objective augmentation, fuzzy bang-bang switching, MPC and nonlinear MPC, QP whole-body control, control barrier functions, impedance control, and diffusion denoisers with residual attention injection. In X-Dyna, the human-aware module modifies spatial self-attention in a frozen SD 1.5 UNet while preserving the capacity of motion modules to synthesize fluid and intricate dynamic details. In robotics, the corresponding role is usually played by a low-level controller that tracks a human-aware reference while preserving feasibility and safety (Chang et al., 17 Jan 2025, Kalliokoski et al., 2022, Busellato et al., 28 Aug 2025, Sambhus et al., 26 Sep 2025).

3. Control primitives and mathematical formulations

A defining feature of this literature is that human-awareness is not treated as an external annotation but as an explicit control variable, constraint, or residual stream. In shared control, the canonical form is convex blending. Bi-CAST uses

$$\mathbf{u}_i(t) = \alpha_i(t)\,\mathbf{u}^{\mathrm{auto}}_i(t) + \big(1-\alpha_i(t)\big)\,\mathbf{u}^{\mathrm{human}}_i(t), \quad i \in \{\mathrm{L},\mathrm{R}\},$$

with $\alpha_i(t)\in[0,0.9]$, and the teleoperator-aware quadruped controller uses

$u(t)=\lambda\,u^{R}(t)+(1-\lambda)\,u^{H}(t),$

with a fixed arbitration weight $\lambda=0.35$ in experiments (Wang et al., 15 Mar 2026, Sambhus et al., 26 Sep 2025).

In local navigation and telepresence, human influence is often inserted directly into the optimization objective. HI-DWA augments the standard DWA objective with

$$J_{sh}(u_t)= \begin{bmatrix} |v_h-v_t|\ |\omega_h-\omega_t| \end{bmatrix},$$

and selects

$$u_t^*=\underset{u_t \in U_{free}^\Delta(t)}{\arg \min}\;J(u_t),$$

so the selected control remains in the collision-free admissible set while being biased toward user-indicated motion (Kalliokoski et al., 2022).

Safety-critical formulations frequently rely on barrier functions or formal game solutions. UA-PCBFs define the uncertainty-aware barrier

$$h_{ua}(\tau,x)=d_{min}+\overline{\sigma}(\tau,x)-d(\tau,x),$$

and enforce predictive barrier constraints inside a QP, with uncertainty also modulating the predictive slack penalty. The legged shared-autonomy controller instead embeds a discrete-time CBF condition,

$$\Delta h\big(x(t),u(t)\big)\ge -\gamma\big(h(x(t))\big),$$

inside a high-level ANMPC with a human-optimality equality constraint $\hat{\alpha}_{t,i,k}$0 (Busellato et al., 28 Aug 2025, Sambhus et al., 26 Sep 2025).

Generative systems use a different control substrate but a structurally similar idea. X-Dyna keeps the base spatial and temporal attentions intact and adds appearance guidance as a residual:

$\hat{\alpha}_{t,i,k}$1

$\hat{\alpha}_{t,i,k}$2

with $\hat{\alpha}_{t,i,k}$3 zero-initialized. This preserves motion priors while supplying detailed, spatially correlated appearance guidance (Chang et al., 17 Jan 2025).

The older HRI MPC literature already contains the same structural elements: explicit state dynamics, objective shaping, and mixed-integer logical constraints. In the assistive and gaze scenarios, the core plant is

$\hat{\alpha}_{t,i,k}$4

and the controller minimizes a quadratic objective in predicted observed states while enforcing if-then logic through binary variables (Jorgensen et al., 2017).

4. Human-state estimation, intent inference, and prediction

Human-aware control depends on the chosen human representation. One line of work estimates motion directly. Adaptive-assistance systems formulate the interaction as a Dec-POMDP and introduce an anticipation module

$\hat{\alpha}_{t,i,k}$5

trained with an L2 prediction loss, alongside a utility module that infers feature-based preference rewards from demonstrations and injects $\hat{\alpha}_{t,i,k}$6 into the robot reward (Qin et al., 2024). Constraint-aware co-manipulation instead represents intent as a pair of DS models in position and orientation, and estimates $\hat{\alpha}_{t,i,k}$7 and $\hat{\alpha}_{t,i,k}$8 with particle filters whose process noise is shaped by human manipulability and whose particles are trimmed by GPU-accelerated IK feasibility checks (Shao et al., 2024).

Another line of work abstracts human behavior to make formal synthesis tractable. The HCPS framework learns a deterministic, finite-state Mealy abstraction

$\hat{\alpha}_{t,i,k}$9

from ACT-R simulations using Angluin’s L* and then composes it with the automation and environment into a timed game automaton. Safety specifications are expressed as $$\mathbf{u}_i(t) = \alpha_i(t)\,\mathbf{u}^{\mathrm{auto}}_i(t) + \big(1-\alpha_i(t)\big)\,\mathbf{u}^{\mathrm{human}}_i(t), \quad i \in \{\mathrm{L},\mathrm{R}\},$$0 or $$\mathbf{u}_i(t) = \alpha_i(t)\,\mathbf{u}^{\mathrm{auto}}_i(t) + \big(1-\alpha_i(t)\big)\,\mathbf{u}^{\mathrm{human}}_i(t), \quad i \in \{\mathrm{L},\mathrm{R}\},$$1, and a memoryless strategy is extracted if the initial state lies in the winning set (Hajnorouzi et al., 18 Nov 2025).

Human state can also be cognitive rather than kinematic. The cognitive-availability-aware mixed-initiative controller uses Deepgaze head pose estimation from an off-the-shelf webcam, applies EMA smoothing to head yaw, compares the smoothed yaw against baseline thresholds for attending versus not attending the GUI, and maps the result into fuzzy membership values that drive LOA arbitration. The crucial design choice is that cognitive availability and active LOA have priority over other factors in the hierarchical fuzzy rule base (Petousakis et al., 2021).

A further distinction concerns whether anticipation is explicit. Visual navigation among humans demonstrates that a policy can anticipate and react without explicitly predicting future human motion: the CNN maps monocular RGB cues such as leg spread and toe direction into a waypoint that already encodes human-aware behavior (Tolani et al., 2020). This does not eliminate prediction; it changes where prediction is represented.

5. Shared autonomy, assistance, and authority negotiation

Shared-autonomy systems differ mainly in how they allocate authority between human and automation. HI-DWA preserves autonomous collision avoidance and efficient goal-directed navigation, while allowing the user to bias local decisions through a human-influence term. Its design goal is to inject “preferences” at the decision step rather than replace the autonomous controller, and its collision-free admissible set remains the same as standard DWA (Kalliokoski et al., 2022).

Bi-CAST pushes this logic further by learning continuous, per-arm authority from multimodal context. A ResNet-50 and an eight-layer Transformer fuse visual features, safety signals $$\mathbf{u}_i(t) = \alpha_i(t)\,\mathbf{u}^{\mathrm{auto}}_i(t) + \big(1-\alpha_i(t)\big)\,\mathbf{u}^{\mathrm{human}}_i(t), \quad i \in \{\mathrm{L},\mathrm{R}\},$$2, and haptic intent features $$\mathbf{u}_i(t) = \alpha_i(t)\,\mathbf{u}^{\mathrm{auto}}_i(t) + \big(1-\alpha_i(t)\big)\,\mathbf{u}^{\mathrm{human}}_i(t), \quad i \in \{\mathrm{L},\mathrm{R}\},$$3, producing left and right authority chunks over $$\mathbf{u}_i(t) = \alpha_i(t)\,\mathbf{u}^{\mathrm{auto}}_i(t) + \big(1-\alpha_i(t)\big)\,\mathbf{u}^{\mathrm{human}}_i(t), \quad i \in \{\mathrm{L},\mathrm{R}\},$$4 future steps. Haptic rendering is likewise authority-aware:

$$\mathbf{u}_i(t) = \alpha_i(t)\,\mathbf{u}^{\mathrm{auto}}_i(t) + \big(1-\alpha_i(t)\big)\,\mathbf{u}^{\mathrm{human}}_i(t), \quad i \in \{\mathrm{L},\mathrm{R}\},$$5

The imposed bound $$\mathbf{u}_i(t) = \alpha_i(t)\,\mathbf{u}^{\mathrm{auto}}_i(t) + \big(1-\alpha_i(t)\big)\,\mathbf{u}^{\mathrm{human}}_i(t), \quad i \in \{\mathrm{L},\mathrm{R}\},$$6 preserves at least 10% human authority, and guidance cues fade as autonomy increases (Wang et al., 15 Mar 2026).

Mixed-initiative systems often use mode switching rather than continuous blending. In the cognitive-availability-aware controller, both human-initiated and AI-initiated switches are allowed, but the hierarchical fuzzy rule base ensures that when the operator is not attending the screen, the robot operates in autonomy. The HCPS shared-control framework formalizes a related three-level pattern—Nominal, Advisory, Intervention—where automation authority rises only when risk predicates or winning-set boundaries warrant it (Petousakis et al., 2021, Hajnorouzi et al., 18 Nov 2025).

Assistance-aware physical collaboration adds another notion of authority: assistance may be exploited only when it reduces task error or advances the desired trajectory. In partner-aware control for humanoids, helpful interaction is defined by the projection of the external agent’s contribution along the task error or desired-motion direction, and the Lyapunov derivative becomes more negative when helpful assistance is present. In tethered aerial guidance, the control law

$$\mathbf{u}_i(t) = \alpha_i(t)\,\mathbf{u}^{\mathrm{auto}}_i(t) + \big(1-\alpha_i(t)\big)\,\mathbf{u}^{\mathrm{human}}_i(t), \quad i \in \{\mathrm{L},\mathrm{R}\},$$7

adds human velocity feedback specifically to preserve the desired guidance force irrespective of walking speed, provided $$\mathbf{u}_i(t) = \alpha_i(t)\,\mathbf{u}^{\mathrm{auto}}_i(t) + \big(1-\alpha_i(t)\big)\,\mathbf{u}^{\mathrm{human}}_i(t), \quad i \in \{\mathrm{L},\mathrm{R}\},$$8 (Tirupachuri, 2020, Allenspach et al., 2022).

A recurring design trade-off is whether authority should be fixed, scheduled, or inferred online. Fixed blending can be predictable but inflexible; adaptive authority can be responsive but harder to interpret; mode switching can be transparent but may create discontinuities. The literature contains all three choices.

6. Evaluation, limitations, and open problems

Evaluation protocols reflect domain-specific priorities, but several representative results illustrate the state of the art. X-Dyna reports foreground DTFVD $$\mathbf{u}_i(t) = \alpha_i(t)\,\mathbf{u}^{\mathrm{auto}}_i(t) + \big(1-\alpha_i(t)\big)\,\mathbf{u}^{\mathrm{human}}_i(t), \quad i \in \{\mathrm{L},\mathrm{R}\},$$9, background $\alpha_i(t)\in[0,0.9]$0, overall $\alpha_i(t)\in[0,0.9]$1, and a user study in which it scored highest across FG-Dyn $\alpha_i(t)\in[0,0.9]$2, BG-Dyn $\alpha_i(t)\in[0,0.9]$3, ID $\alpha_i(t)\in[0,0.9]$4, Overall $\alpha_i(t)\in[0,0.9]$5 (Chang et al., 17 Jan 2025). In visual navigation among humans, LB-WayPtNav-DH achieved Success $\alpha_i(t)\in[0,0.9]$6, Time $\alpha_i(t)\in[0,0.9]$7 s, Acc $\alpha_i(t)\in[0,0.9]$8, Jerk $\alpha_i(t)\in[0,0.9]$9 in simulation, and 20/20 successes in hardware across 4 scenarios in 2 unseen buildings (Tolani et al., 2020). HI-DWA yielded faster task completion than switching between autonomy and manual control, with Mean $u(t)=\lambda\,u^{R}(t)+(1-\lambda)\,u^{H}(t),$0 s versus $u(t)=\lambda\,u^{R}(t)+(1-\lambda)\,u^{H}(t),$1 s, and 22/32 participants found shared control with joystick easier to use (Kalliokoski et al., 2022).

In adaptive assistance, the proposed framework achieved Human Reward $u(t)=\lambda\,u^{R}(t)+(1-\lambda)\,u^{H}(t),$2 versus PPO $u(t)=\lambda\,u^{R}(t)+(1-\lambda)\,u^{H}(t),$3 and TD3 $u(t)=\lambda\,u^{R}(t)+(1-\lambda)\,u^{H}(t),$4 in feeding, and maintained $u(t)=\lambda\,u^{R}(t)+(1-\lambda)\,u^{H}(t),$5 success in several assistive settings (Qin et al., 2024). UA-PCBFs reduced safe-space violations in both repeatable and human-interaction experiments; in the mock-hand setup with $u(t)=\lambda\,u^{R}(t)+(1-\lambda)\,u^{H}(t),$6, they reached $u(t)=\lambda\,u^{R}(t)+(1-\lambda)\,u^{H}(t),$7 violations with magnitude $u(t)=\lambda\,u^{R}(t)+(1-\lambda)\,u^{H}(t),$8 m, compared with $u(t)=\lambda\,u^{R}(t)+(1-\lambda)\,u^{H}(t),$9 violations and magnitude $\lambda=0.35$0 m for a reactive CBF baseline (Busellato et al., 28 Aug 2025). Bi-CAST reported up to $\lambda=0.35$1 reduction in collisions, $\lambda=0.35$2 improvement in trajectory smoothness, and $\lambda=0.35$3 lower NASA-TLX workload than fixed-authority or discrete-switching baselines (Wang et al., 15 Mar 2026).

The principal limitations are also consistent across domains. Dense crowds, multi-human interactions, occlusions, narrow fields of view, and reactive memoryless policies remain difficult in navigation (Tolani et al., 2020). In adaptive assistance and co-manipulation, safety is often penalty-based rather than certified, and preference identifiability depends on whether the chosen features actually capture human comfort and intent (Qin et al., 2024, Shao et al., 2024). In UA-PCBFs, over- or under-conservative uncertainty inflation, sensing occlusions, and rapid intent changes remain problematic (Busellato et al., 28 Aug 2025). In legged shared autonomy, solver load, perception drift, and the need to balance fixed arbitration with online intent adaptation remain central issues (Sambhus et al., 26 Sep 2025). In X-Dyna, extreme scale changes, fine hand articulations, large occlusions, and rapid camera motion can reduce temporal fidelity or identity preservation (Chang et al., 17 Jan 2025).

Two broader points follow. First, “human-aware” does not by itself imply hard guarantees: some systems use soft penalties or supervised priors, while others use barrier functions, QP constraints, or timed-game synthesis. Second, “dynamic” does not always mean explicit temporal forecasting: it may instead denote online authority negotiation, adaptive impedance, state-dependent LOA switching, or residual attention injection that preserves a generative backbone’s motion capacity. This suggests that the enduring research problem is not merely to detect humans, but to decide which human variable should enter the control loop, in what representation, and with what guarantees.