---
title: Dynamic Grounded Re-planning
url: https://www.emergentmind.com/topics/dynamic-grounded-re-planning-16f93be4-c499-4295-940c-a80d1d6f74d1
type: topic
---

# Dynamic Grounded Re-planning

Dynamic grounded re-planning is a family of methodologies for online, feedback-driven adjustment of plans in dynamic environments, characterized by continual or event-triggered plan updates that leverage high-fidelity environment grounding—whether geometric, semantic, visual, or physical. The objective is to ensure optimal, safe, and robust task or motion execution despite unpredictably evolving obstacles, environmental changes, or incomplete prior knowledge. This paradigm integrates classical motion planning, modern deep learning, model-predictive/reactive control, and high-level symbolic (potentially language- or vision-driven) reasoning to allow agents—robots, virtual assistants, or embodied LLM-driven systems—to act reliably “in the real,” responding rapidly and adaptively as the state of the world deviates from initial assumptions.

## 1. Foundational Principles and Approaches

Dynamic grounded re-planning targets settings where static plans are insufficient due to non-stationarity, partial observability, or adversarial uncertainty. Core principles, as shown in the adaptive dimensionality work [1605.06853], are as follows:
- **Selective High-dimensional Reasoning**: Many methods avoid computation over the entire space-time or geometric-semantic manifold. Instead, expensive dimensions (e.g., time in dynamic obstacle avoidance) are only added locally where required (e.g., near predicted collisions, or in regions where observed scene deviates from the prior plan).
- **Feedback-driven Adaptation**: Agents explicitly monitor for execution failures, unexpected environmental events, or plan performance degradations, triggering local or global re-planning loops tethered to real-time perception, semantic labeling, or physical simulation state [2011.00155, 2103.03958, 2401.04157].
- **Grounding of Planning**: Dynamic re-planning is “grounded” by integrating real-world sensor input—such as RGB-D or point clouds in mapping/planning systems [2103.03958], explicit object and status detection in embodied agents [2212.04088, 2412.17288], or detailed physical parameterization inferred via vision-language models [2505.14938].
- **Hierarchical and Modular Architectures**: Many systems are architected hierarchically, with high-level symbolic or task planners generating sequences of subtasks, and low-level reactive modules (controllers, learned policies, or behaviors) responsible for physical execution and immediate reactivity. Discrepancies or failures at any layer can percolate up or down to trigger re-planning [1811.04333, 2406.03641].

## 2. Algorithmic Methodologies

Dynamic grounded re-planning subsumes a wide array of algorithmic approaches:

| Paradigm              | Representative Algorithms/Frameworks | Key Features/Mechanisms                                      |
|-----------------------|--------------------------------------|--------------------------------------------------------------|
| Adaptive Dimensionality| [1605.06853]                         | Locally increase plan fidelity (e.g., add time dim) only near dynamic events; projection/inverse-projection per spatial location; tracking tunnels; cost-bound guarantees |
| Sampling-based Tree Adaptation | Informed RRT*, branch-and-bound, prior-trees reuse [2209.05573, 1704.04585, 2306.09229] | Sample only meaningful regions, prune invalid nodes, locally rewire tree at runtime, exploit previous valid paths for rapid updates|
| Continuous Trajectory Optimization | GPMP2 with live factor graphs and receding horizon replanning [2103.03958] | Real-time EDT update, collision-checking at 10Hz, warm-start from latest plan, use of signed/unsigned distance fields|
| Deep Visuo-Motor and RL Approaches | CNN+RL fixed-horizon policies, Transformer-based motion policies [2011.00155, 2509.06953] | End-to-end reactive policies; direct sensor-to-actuation; path-conditioned reward shaping, curriculum-based adaptation|
| Task and Motion Planning with Execution Gaps | TAMP with behavior-based repair [2406.03641], RC loops with VLM diagnosis [2401.04157] | Plan up to known gaps, fill with sensor-driven behaviors; execution failures trigger symbolic re-planning with updated world state|
| Memory-Augmented LLM Planning | Coarse-to-fine memory, skill memory, visual-grounded plan correction [2508.15305, 2509.24219, 2412.17288] | Episodic trajectory reflection, context-based example retrieval, skill re-use to stabilize future execution, self-QA for anomaly handling|
| LLM/VLM-Driven Symbolic Planning | Few-shot/in-context planning with perception loop [2212.04088, 2412.17288] | Immediate incorporation of observed objects, subgoal re-generation, feedback coupling between low-level failures and task planner, plan representation as (action, object, relation) triplets|
| Grid-based/Symbolic Navigation | Dynamic scene graphs, D*-replanning, semantic anchor chaining [2510.10865, 2510.06587] | Open-vocabulary object/scene graphs, real-time occupancy grid/failure detection, LLM-based introspection, path relay with multi-hop co-occurrence chains|

## 3. Grounding Modalities and Environmental Feedback

Grounding in dynamic re-planning is achieved via:
- **Geometric Grounding**: Continuous updates to signed or unsigned distance fields from live RGB-D input enable geometric collision checking, as in GPU-Voxels + GPMP2 pipelines [2103.03958].
- **Semantic and Visual Grounding**: Foundation object detection and segmentation models (YOLOv8, OWLv2, SAM), dynamic scene graphs, or language-vision models (CLIP, GPT-4o) provide pixel-to-symbol mapping for object-aware planning or correction [2505.14938, 2510.10865].
- **Physical/Material Grounding**: Vision-language models infer material properties per object (e.g., friction, restitution), with outputs directly inserted into physics simulations for physically-informed plan rollouts [2505.14938].
- **Skill Memory and Experience Retrieval**: Successful plans, once grounded and verified, are cached for replay in future executions, minimizing re-computation and improving consistency under observation-to-action variability [2509.24219, 2508.15305].

## 4. Theoretical Guarantees and Performance Tradeoffs

Several frameworks offer formal guarantees:
- **Completeness and Bounded Suboptimality**: The adaptive dimensionality algorithm provides guaranteed solution discovery (completeness) and path cost within a configurable bound of optimal in dynamic environments [1605.06853].
- **Correctness and Robustness by Construction**: Reactive task and motion planning architectures with LTL synthesis (temporal logic games) guarantee safe execution under specified model and disturbance assumptions [1811.04333].
- **Empirical and Practical Metrics**: Most empirical work evaluates dynamic grounded re-planning using metrics such as planning time, trajectory cost, success rate, action efficiency (SPL, SAE), robustness against unmodeled disturbances, and computational cost (such as network function evaluations for diffusion-based planners [2409.16950]).

Empirical results often highlight:
- Major speedups compared to full-dimensional planning (up to an order of magnitude [1605.06853]).
- Substantial jump in task success rate in out-of-distribution or unseen environments, especially when re-planning is informed by perception [2212.04088, 2412.17288].
- Improved efficiency in step count (up to 14.7 fewer navigation steps [2510.06587]), reduced failures under dynamic adversarial scenarios, and robust sim-to-real performance on physical robot platforms [2103.03958, 2510.10865].

## 5. Comparison of Strategies and System Variants

| System/Paradigm                      | Core Re-planning Trigger   | Grounding Modalities                                  | Target Use Cases                  |
|-------------------------------------- |---------------------------|-------------------------------------------------------|-----------------------------------|
| Adaptive Dimensionality [1605.06853] | Local cost/consistency failures| Spatial-temporal reasoning, time-augmented planning | Mobile navigation in dynamic scenes|
| Sampling-based Tree Methods           | Obstacle encounters along path| Occupancy/scene geometry, sensor-detected events      | Manipulator path replanning       |
| Deep End-to-End/Visuo-Motor [2011.00155, 2509.06953] | Sensor input deviation      | Depth/point cloud, visual waypoints, RL reward shaping| Manipulation, reaching, navigation|
| Skill/Memory-Augmented LLMs [2508.15305, 2509.24219]| Action anomaly, low task confidence| Trajectory and subgoal retrieval, example-based tips| Embodied task planning, web agents|
| Physics-Driven Simulators [2505.14938] | Object state change, scene update | Physically parameterized scenes, 3D splatting, VLM-inferred materials | Robot manipulation, quadrotor landing|
| Symbolic/Hybrid Planners [2510.10865, 2406.03641] | Unreachable subgoals, sensor failure| Dynamic scene graphs, open-vocab object detection   | Semantic navigation, cluttered environments|

## 6. Applications and Benchmark Tasks

Dynamic grounded re-planning has been demonstrated and benchmarked across diverse domains:
- **Mobile Robot and Vehicle Navigation**: Path planning with static and dynamic obstacles (maze, hallway, outdoor, urban, adversarial scenarios), receding-horizon trajectory optimization with live distance field updates [1605.06853, 1704.04585, 2103.03958, 2411.12014].
- **Manipulation and Pick-and-Place**: Reactive arm motion, peg insertion, object rearrangement, obstacle avoidance, robust handling of goal or environmental changes [2011.00155, 2509.06953].
- **Closed-Loop Locomotion**: Whole-body and legged locomotion with formal safety and progress guarantees, dynamically selective contact switching [1811.04333].
- **Task Planning under Uncertainty**: Hierarchical planning in real homes/kitchen scenarios (e.g., ALFRED, RC benchmarks [2212.04088, 2401.04157, 2412.17288]), vision-grounded plan repair and reflection, language-driven re-planning with embedded perception.
- **Physical Simulation**: Planning under uncertain material properties, scene geometry, with physics-informed replanning for contact tasks [2505.14938].
- **Web Automation and GUI Interaction**: LLM-driven agents decompose and adapt navigation, extraction, and execution subtasks as new information or interface affordances are encountered [2510.06587, 2410.18963].
- **Real-World Deployment**: Onboard robots (Toyota HSR, Jetbot Pro, Franka Panda) demonstrate online, live replanning under sensor noise, occlusion, or dynamically changing goals/obstacles [2103.03958, 2510.10865].

## 7. Limitations and Future Directions

Common limitations across current dynamic grounded re-planning approaches include:
- **Dependency on Perception Accuracy**: Framework success is often tightly coupled to the fidelity of environmental perception, especially object detection and segmentation modules; failure to recognize new or unseen obstacles leads to plan failures [2103.03958, 2401.04157].
- **Computational Overhead**: Despite selective re-planning, high-dimensional planning, Monte Carlo inference, or diffusion models remain computationally expensive, requiring both algorithmic (e.g., uncertainty-driven triggers [2409.16950]) and architectural (e.g., reuse of memory [2509.24219]) optimizations.
- **Partial Grounding and Hybridization**: Many frameworks still rely on partial offline planning, with (sometimes fragile) transitions to reactive execution routines when model gaps are present [2406.03641].
- **Incremental and Continual Learning**: Autonomous, efficient integration of new skills and experience into action selection and plan repair remains under-explored, though recent memory and skill-centric approaches make progress [2508.15305, 2509.24219].
- **Scalability and Domain Generalization**: Demonstrated scalability exists in PR2/HSR/Jetbot and ALFRED scenarios, but generalized deployment across complex, open-world domains still faces challenges.

Future work is converging towards: real-world multi-modal integration, more principled and scalable skill acquisition and reuse, probabilistic and robust planning under deep uncertainty (imperfect knowledge of obstacle trajectories, noisy sensors), and more interpretable/reflexive symbolic-neural hybrid architectures. A recurring trajectory in the field will be the tighter unification of perception, reasoning, and reactive/optimal control—always with live environmental grounding as central to robust, adaptive behavior.

Source: https://www.emergentmind.com/topics/dynamic-grounded-re-planning-16f93be4-c499-4295-940c-a80d1d6f74d1