Dynamic Grounded Re-planning

• Dynamic grounded re-planning is a method that adapts plans in real-time by integrating high-fidelity sensor inputs with modern AI techniques to ensure robust execution in unpredictable environments.
• It combines classical planning, deep learning, model-predictive control, and symbolic reasoning to address challenges like dynamic obstacles, partial observability, and uncertain environments.
• Applications include mobile navigation, robotic manipulation, and autonomous systems, achieving notable improvements in planning speed, trajectory efficiency, and task success rates.

Dynamic grounded re-planning is a family of methodologies for online, feedback-driven adjustment of plans in dynamic environments, characterized by continual or event-triggered plan updates that leverage high-fidelity environment grounding—whether geometric, semantic, visual, or physical. The objective is to ensure optimal, safe, and robust task or motion execution despite unpredictably evolving obstacles, environmental changes, or incomplete prior knowledge. This paradigm integrates classical motion planning, modern deep learning, model-predictive/reactive control, and high-level symbolic (potentially language- or vision-driven) reasoning to allow agents—robots, virtual assistants, or embodied LLM-driven systems—to act reliably “in the real,” responding rapidly and adaptively as the state of the world deviates from initial assumptions.

1. Foundational Principles and Approaches

Dynamic grounded re-planning targets settings where static plans are insufficient due to non-stationarity, partial observability, or adversarial uncertainty. Core principles, as shown in the adaptive dimensionality work (Vemula et al., 2016), are as follows:

• Selective High-dimensional Reasoning: Many methods avoid computation over the entire space-time or geometric-semantic manifold. Instead, expensive dimensions (e.g., time in dynamic obstacle avoidance) are only added locally where required (e.g., near predicted collisions, or in regions where observed scene deviates from the prior plan).
• Feedback-driven Adaptation: Agents explicitly monitor for execution failures, unexpected environmental events, or plan performance degradations, triggering local or global re-planning loops tethered to real-time perception, semantic labeling, or physical simulation state (Ota et al., 2020, Finean et al., 2021, Skreta et al., 8 Jan 2024).
• Grounding of Planning: Dynamic re-planning is “grounded” by integrating real-world sensor input—such as RGB-D or point clouds in mapping/planning systems (Finean et al., 2021), explicit object and status detection in embodied agents (Song et al., 2022, Kim et al., 23 Dec 2024), or detailed physical parameterization inferred via vision-LLMs (Elhafsi et al., 20 May 2025).
• Hierarchical and Modular Architectures: Many systems are architected hierarchically, with high-level symbolic or task planners generating sequences of subtasks, and low-level reactive modules (controllers, learned policies, or behaviors) responsible for physical execution and immediate reactivity. Discrepancies or failures at any layer can percolate up or down to trigger re-planning (Zhao et al., 2018, Pan et al., 5 Jun 2024).

2. Algorithmic Methodologies

Dynamic grounded re-planning subsumes a wide array of algorithmic approaches:

Paradigm	Representative Algorithms/Frameworks	Key Features/Mechanisms
Adaptive Dimensionality	(Vemula et al., 2016)	Locally increase plan fidelity (e.g., add time dim) only near dynamic events; projection/inverse-projection per spatial location; tracking tunnels; cost-bound guarantees
Sampling-based Tree Adaptation	Informed RRT*, branch-and-bound, prior-trees reuse (Vu et al., 2022, Connell et al., 2017, Khanal et al., 2023)	Sample only meaningful regions, prune invalid nodes, locally rewire tree at runtime, exploit previous valid paths for rapid updates
Continuous Trajectory Optimization	GPMP2 with live factor graphs and receding horizon replanning (Finean et al., 2021)	Real-time EDT update, collision-checking at 10Hz, warm-start from latest plan, use of signed/unsigned distance fields
Deep Visuo-Motor and RL Approaches	CNN+RL fixed-horizon policies, Transformer-based motion policies (Ota et al., 2020, Yang et al., 8 Sep 2025)	End-to-end reactive policies; direct sensor-to-actuation; path-conditioned reward shaping, curriculum-based adaptation
Task and Motion Planning with Execution Gaps	TAMP with behavior-based repair (Pan et al., 5 Jun 2024), RC loops with VLM diagnosis (Skreta et al., 8 Jan 2024)	Plan up to known gaps, fill with sensor-driven behaviors; execution failures trigger symbolic re-planning with updated world state
Memory-Augmented LLM Planning	Coarse-to-fine memory, skill memory, visual-grounded plan correction (Yang et al., 21 Aug 2025, Kagaya et al., 29 Sep 2025, Kim et al., 23 Dec 2024)	Episodic trajectory reflection, context-based example retrieval, skill re-use to stabilize future execution, self-QA for anomaly handling
LLM/VLM-Driven Symbolic Planning	Few-shot/in-context planning with perception loop (Song et al., 2022, Kim et al., 23 Dec 2024)	Immediate incorporation of observed objects, subgoal re-generation, feedback coupling between low-level failures and task planner, plan representation as (action, object, relation) triplets
Grid-based/Symbolic Navigation	Dynamic scene graphs, D*-replanning, semantic anchor chaining (Alanazi et al., 13 Oct 2025, Yang et al., 8 Oct 2025)	Open-vocabulary object/scene graphs, real-time occupancy grid/failure detection, LLM-based introspection, path relay with multi-hop co-occurrence chains

3. Grounding Modalities and Environmental Feedback

Grounding in dynamic re-planning is achieved via:

• Geometric Grounding: Continuous updates to signed or unsigned distance fields from live RGB-D input enable geometric collision checking, as in GPU-Voxels + GPMP2 pipelines (Finean et al., 2021).
• Semantic and Visual Grounding: Foundation object detection and segmentation models (YOLOv8, OWLv2, SAM), dynamic scene graphs, or language-vision models (CLIP, GPT-4o) provide pixel-to-symbol mapping for object-aware planning or correction (Elhafsi et al., 20 May 2025, Alanazi et al., 13 Oct 2025).
• Physical/Material Grounding: Vision-LLMs infer material properties per object (e.g., friction, restitution), with outputs directly inserted into physics simulations for physically-informed plan rollouts (Elhafsi et al., 20 May 2025).
• Skill Memory and Experience Retrieval: Successful plans, once grounded and verified, are cached for replay in future executions, minimizing re-computation and improving consistency under observation-to-action variability (Kagaya et al., 29 Sep 2025, Yang et al., 21 Aug 2025).

4. Theoretical Guarantees and Performance Tradeoffs

Several frameworks offer formal guarantees:

• Completeness and Bounded Suboptimality: The adaptive dimensionality algorithm provides guaranteed solution discovery (completeness) and path cost within a configurable bound of optimal in dynamic environments (Vemula et al., 2016).
• Correctness and Robustness by Construction: Reactive task and motion planning architectures with LTL synthesis (temporal logic games) guarantee safe execution under specified model and disturbance assumptions (Zhao et al., 2018).
• Empirical and Practical Metrics: Most empirical work evaluates dynamic grounded re-planning using metrics such as planning time, trajectory cost, success rate, action efficiency (SPL, SAE), robustness against unmodeled disturbances, and computational cost (such as network function evaluations for diffusion-based planners (Punyamoorty et al., 25 Sep 2024)).

Empirical results often highlight:

• Major speedups compared to full-dimensional planning (up to an order of magnitude (Vemula et al., 2016)).
• Substantial jump in task success rate in out-of-distribution or unseen environments, especially when re-planning is informed by perception (Song et al., 2022, Kim et al., 23 Dec 2024).
• Improved efficiency in step count (up to 14.7 fewer navigation steps (Yang et al., 8 Oct 2025)), reduced failures under dynamic adversarial scenarios, and robust sim-to-real performance on physical robot platforms (Finean et al., 2021, Alanazi et al., 13 Oct 2025).

5. Comparison of Strategies and System Variants

System/Paradigm	Core Re-planning Trigger	Grounding Modalities	Target Use Cases
Adaptive Dimensionality (Vemula et al., 2016)	Local cost/consistency failures	Spatial-temporal reasoning, time-augmented planning	Mobile navigation in dynamic scenes
Sampling-based Tree Methods	Obstacle encounters along path	Occupancy/scene geometry, sensor-detected events	Manipulator path replanning
Deep End-to-End/Visuo-Motor (Ota et al., 2020, Yang et al., 8 Sep 2025)	Sensor input deviation	Depth/point cloud, visual waypoints, RL reward shaping	Manipulation, reaching, navigation
Skill/Memory-Augmented LLMs (Yang et al., 21 Aug 2025, Kagaya et al., 29 Sep 2025)	Action anomaly, low task confidence	Trajectory and subgoal retrieval, example-based tips	Embodied task planning, web agents
Physics-Driven Simulators (Elhafsi et al., 20 May 2025)	Object state change, scene update	Physically parameterized scenes, 3D splatting, VLM-inferred materials	Robot manipulation, quadrotor landing
Symbolic/Hybrid Planners (Alanazi et al., 13 Oct 2025, Pan et al., 5 Jun 2024)	Unreachable subgoals, sensor failure	Dynamic scene graphs, open-vocab object detection	Semantic navigation, cluttered environments

6. Applications and Benchmark Tasks

Dynamic grounded re-planning has been demonstrated and benchmarked across diverse domains:

• Mobile Robot and Vehicle Navigation: Path planning with static and dynamic obstacles (maze, hallway, outdoor, urban, adversarial scenarios), receding-horizon trajectory optimization with live distance field updates (Vemula et al., 2016, Connell et al., 2017, Finean et al., 2021, Ajeleye, 18 Nov 2024).
• Manipulation and Pick-and-Place: Reactive arm motion, peg insertion, object rearrangement, obstacle avoidance, robust handling of goal or environmental changes (Ota et al., 2020, Yang et al., 8 Sep 2025).
• Closed-Loop Locomotion: Whole-body and legged locomotion with formal safety and progress guarantees, dynamically selective contact switching (Zhao et al., 2018).
• Task Planning under Uncertainty: Hierarchical planning in real homes/kitchen scenarios (e.g., ALFRED, RC benchmarks (Song et al., 2022, Skreta et al., 8 Jan 2024, Kim et al., 23 Dec 2024)), vision-grounded plan repair and reflection, language-driven re-planning with embedded perception.
• Physical Simulation: Planning under uncertain material properties, scene geometry, with physics-informed replanning for contact tasks (Elhafsi et al., 20 May 2025).
• Web Automation and GUI Interaction: LLM-driven agents decompose and adapt navigation, extraction, and execution subtasks as new information or interface affordances are encountered (Yang et al., 8 Oct 2025, Wang et al., 24 Oct 2024).
• Real-World Deployment: Onboard robots (Toyota HSR, Jetbot Pro, Franka Panda) demonstrate online, live replanning under sensor noise, occlusion, or dynamically changing goals/obstacles (Finean et al., 2021, Alanazi et al., 13 Oct 2025).

7. Limitations and Future Directions

Common limitations across current dynamic grounded re-planning approaches include:

• Dependency on Perception Accuracy: Framework success is often tightly coupled to the fidelity of environmental perception, especially object detection and segmentation modules; failure to recognize new or unseen obstacles leads to plan failures (Finean et al., 2021, Skreta et al., 8 Jan 2024).
• Computational Overhead: Despite selective re-planning, high-dimensional planning, Monte Carlo inference, or diffusion models remain computationally expensive, requiring both algorithmic (e.g., uncertainty-driven triggers (Punyamoorty et al., 25 Sep 2024)) and architectural (e.g., reuse of memory (Kagaya et al., 29 Sep 2025)) optimizations.
• Partial Grounding and Hybridization: Many frameworks still rely on partial offline planning, with (sometimes fragile) transitions to reactive execution routines when model gaps are present (Pan et al., 5 Jun 2024).
• Incremental and Continual Learning: Autonomous, efficient integration of new skills and experience into action selection and plan repair remains under-explored, though recent memory and skill-centric approaches make progress (Yang et al., 21 Aug 2025, Kagaya et al., 29 Sep 2025).
• Scalability and Domain Generalization: Demonstrated scalability exists in PR2/HSR/Jetbot and ALFRED scenarios, but generalized deployment across complex, open-world domains still faces challenges.

Future work is converging towards: real-world multi-modal integration, more principled and scalable skill acquisition and reuse, probabilistic and robust planning under deep uncertainty (imperfect knowledge of obstacle trajectories, noisy sensors), and more interpretable/reflexive symbolic-neural hybrid architectures. A recurring trajectory in the field will be the tighter unification of perception, reasoning, and reactive/optimal control—always with live environmental grounding as central to robust, adaptive behavior.