Understanding Task and Motion Planning (TAMP)
- Task and Motion Planning (TAMP) refers to a class of robotic problems combining symbolic task planning with geometric, kinematic, and dynamic motion planning to ensure feasible and executable robot tasks.
- The traditional TAMP approach integrates combinatorial search, constraint satisfaction, and trajectory optimization, accommodating robot configurations, object poses, and intermediate contact points.
- TAMP systems may use various representations, such as PDDLStream, Logic-Geometric Programming (LGP), or object-centric formulations with parallelized differentiable optimizations, to develop and solve robotic plans.
- follow_up_questions_list
Task and Motion Planning (TAMP) is a class of robotic planning problems that integrates discrete task planning with continuous geometric, kinematic, dynamic, and motion planning. A TAMP system must determine both what actions should be performed—such as picking, placing, opening, pushing, navigating, or transferring an object—and how those actions can be realized through feasible grasps, configurations, trajectories, contacts, and controls. Symbolic validity alone does not ensure executability, while geometric motion planning alone does not establish that a sequence of actions achieves a long-horizon task. TAMP consequently combines combinatorial search, hybrid constraint satisfaction, motion planning, trajectory optimization, and increasingly perception, learning, feedback, and uncertainty modeling (Garrett et al., 2020).
1. Problem definition and formal structure
A classical task-planning problem consists of a discrete state space, an action set, a transition relation, an initial state, and a goal set. In a STRIPS-style formulation, an action has parameters, preconditions, and effects. A symbolic planner searches for an action skeleton
such that the induced symbolic states satisfy
Typical symbolic predicates include inhand, on, and inworkspace. A pick action may require an empty gripper and produce the predicate inhand(a), whereas a place action removes inhand(a) and establishes on(a,b) (Migimatsu et al., 2019).
Motion planning is formulated over a continuous configuration space. For a robot configuration space , a feasible trajectory must connect an initial configuration to a goal configuration while remaining in the collision-free subset. In dynamic manipulation, however, the relevant continuous state may include robot configurations, object poses, attached bodies, and articulated mechanisms. This creates a multimodal state space whose feasible regions depend on kinematic relationships such as resting, grasping, support, or transfer modes (Garrett et al., 2020).
A TAMP domain therefore combines symbolic and continuous variables. A plan may contain discrete action parameters such as object identities and continuous parameters such as grasps, placements, configurations, and trajectories. For a fixed plan skeleton, the resulting problem is a hybrid constraint-satisfaction problem:
The continuous variables may include inverse-kinematic solutions, object poses, contact configurations, and discretized trajectory waypoints. The skeleton is executable only when both symbolic preconditions and geometric constraints are satisfied.
A useful conceptual decomposition is:
- Task planning selects a symbolic action sequence.
- Geometric reasoning assigns or tests poses, grasps, and placements.
- Motion planning or trajectory optimization produces a collision-free, dynamically or kinematically valid trajectory.
- Execution and monitoring determine whether the predicted transition occurred.
- Replanning responds to geometric failure, perception error, action failure, or environmental change.
The decomposition is not necessarily sequential. Modern TAMP methods interleave these stages, use geometric information to guide symbolic search, or replace explicit discrete decisions with continuous relaxations.
2. Representations, modes, and constraints
Symbolic modes and kinematic structure
A world configuration may be represented as a robot-object configuration . Because the robot cannot independently control every object pose, is generally under-actuated. Kinematic graphs encode attachments between bodies, with edges representing relative transformations in . Picking up an object or placing it on another object changes this graph and therefore changes the active kinematic mode (Garrett et al., 2020).
A multimodal state can be written as
where specifies the active kinematic relationships and constraints. Common modes include free transit, object-holding transfer, support, grasp, release, pushing, and contact. A mode transition is feasible only at a configuration in the intersection of the two mode spaces:
0
These intersections are often lower-dimensional manifolds. For example, a grasp transition may impose several end-effector pose constraints. Naive sampling in the full world space therefore has negligible probability of landing exactly on the relevant constraint manifold.
Logic-Geometric Programming
Logic-Geometric Programming (LGP) combines symbolic action sequencing with nonlinear optimization. A symbolic sequence selects a sequence of modes, while trajectory optimization determines continuous variables satisfying path and switch constraints. A generic LGP problem contains:
- continuous state and control variables;
- path equalities and inequalities;
- switch constraints associated with discrete actions;
- symbolic successor-state constraints;
- a terminal goal condition;
- a running or terminal cost.
LGP methods may evaluate candidate skeletons progressively, beginning with relaxed sequence-level problems and advancing to full path optimization. Multi-Bound Tree Search uses this structure to filter symbolic candidates before invoking more expensive trajectory optimization (Braun et al., 2021).
Joint-space and object-centric formulations
Conventional joint-space formulations optimize robot configurations together with free-body object variables. Later placement constraints may depend on a large portion of the preceding trajectory. A placement can depend on the object’s pose relative to the gripper at grasp time, which itself depends on earlier robot configurations.
Object-centric formulations instead optimize relative Cartesian poses. For a control frame and target frame, the geometric variable is a relative pose
1
For example:
pick(a)relates the end-effector to object 2;place(a,b)relates object 3 to support object 4;- a pushing action relates a tool to the pushed object and the pushed object to its support.
This representation makes many action constraints local in the relevant object frames. A kinematic tree composes relative transformations into absolute world poses. Consequently, an object-relative command remains meaningful when the object moves, provided its current pose is observed and used to recompute the world-frame target (Migimatsu et al., 2019).
Object-centric abstractions
U-TAMP moves selected motion constraints into symbolic search by representing functional object parts such as on, under, left, right, front, back, and in. A grasp is represented by the object surfaces contacting the palm and fingers; a placement is represented by the contacting parts of the manipulated and supporting objects. These abstract relations determine object and hand poses algebraically.
For example, placing the bottom of object 5 on the top of object 6 corresponds to
7
The relative placement pose is then computed from the bounding-box dimensions. Such abstractions can allow conventional heuristic search to produce physically meaningful manipulation plans without repeatedly invoking general-purpose geometric reasoning, although their validity depends on the adequacy of the bounding-box and functional-part model (Agostini et al., 2023).
Collision, contact, and dynamics
TAMP constraints may encode:
- inverse kinematics;
- joint, velocity, and acceleration limits;
- self-collision and environment collision;
- grasp and placement stability;
- support and contact;
- friction and force consistency;
- object attachment;
- dynamic equations;
- temporal specifications;
- robot capacity and connectivity.
Optimization-based TAMP may use rigid-body dynamics of the form
8
where 9 denotes contact forces. Contact switching, complementarity, and nonsmooth collision constraints make such problems highly nonconvex and numerically difficult (Zhao et al., 2024).
3. Algorithmic families
TAMP algorithms differ in how they solve continuous subproblems and how they integrate continuous reasoning with discrete search.
Sequencing-first and hierarchical methods
Sequencing-first methods generate symbolic plan skeletons and attempt to refine them geometrically. If refinement fails, the planner backtracks to another skeleton. This approach benefits from efficient symbolic planning but may waste substantial computation on logically valid yet geometrically impossible plans.
PDDLStream represents geometric computations as streams that generate poses, grasps, configurations, and trajectories. Its strength is a unified symbolic-geometric representation, but stream-generated objects can enlarge the symbolic planning problem substantially. COAST addresses this issue by excluding stream objects from the initial PDDL problem, first generating a symbolic plan, then constructing only the geometric stream instances required by that plan. Failed refinements are translated into PDDL constraints (Vu et al., 2024).
Interleaved search
Interleaved methods alternate between action-sequence expansion and continuous parameter generation. Geometric feasibility can prune partial plans before a complete skeleton is formed. Such methods may use conditional samplers, lazy motion planning, incremental constraint solving, optimistic objects, or geometric failure explanations.
Extended Tree Search for TAMP constructs a single MCTS structure whose root represents candidate symbolic skeletons and whose descendants bind geometric variables. Top-0 symbolic planning generates skeleton alternatives, while progressive-widening UCT balances exploration across skeletons and exploitation of promising motion-variable bindings (Ren et al., 2021).
Sketch-based TAMP provides another interleaved architecture. A sketch specifies subproblem structure rather than a complete plan. Iterated Width search solves one subproblem at a time, while motion feasibility is checked during search. If a candidate action is infeasible, search resumes within the current subproblem instead of restarting the entire plan. Under assumptions concerning feature accuracy, free space, reachability, and sampling, the proposed sketch has width 1 for the targeted pick-and-place families (Dalmau-Moreno et al., 2024).
Constraint-based feedback
Some methods convert motion-planning failures into symbolic constraints. COAST supports action constraints, sequence or timestep constraints, and collision-generalization constraints. A failed action can be prohibited globally, prohibited only after a particular prefix, or excluded under a generalized condition such as a blocked target location.
Topological-refinement methods extract broader explanations from failed motion-planning searches. If a sampling-based planner explores a region approximated by a convex hull and fails to reach a target, the framework can identify several symbolic destinations outside the explored region and the movable obstacles that caused collisions. The resulting refinement eliminates families of task plans rather than only the failed action. Tampest retains these refinements in an incremental SMT encoding (Tosello et al., 2024).
Optimization-based TAMP
For a fixed skeleton, joint optimization assigns continuous parameters and minimizes a trajectory or execution cost. It is effective when constraints are differentiable and a useful initialization is available, but it is vulnerable to local minima, disconnected feasible regions, and inaccurate collision derivatives.
Receding-horizon Logic-Geometric Programming restricts optimization to a finite action window. After executing the optimized segment, the horizon shifts and the problem is solved again. RHH-LGP combines this structure with fast geometry-based heuristics that assign finite costs to plausible actions and 1 to actions deemed unreachable. The method was applied to long-horizon manipulation, locomotion, obstacle avoidance, modular robots, and heterogeneous multi-robot structures (Braun et al., 2021).
Sampling and optimization hybrids
Sampling-based methods explore high-dimensional and nonconvex spaces without requiring differentiability. Optimization refines coupled variables efficiently when gradients are available. Hybrid methods sample grasps, placements, or initial trajectories and then optimize the coupled continuous assignment.
Differentiable GPU-parallelized TAMP maintains a batch of particles for one skeleton. Each particle contains jointly optimized grasps, placements, configurations, and trajectories. Differentiable costs encode collision avoidance, joint limits, kinematics, goal constraints, and motion quality. GPU parallelism permits thousands of local optimizations to be evaluated concurrently, increasing the likelihood that at least one particle reaches a feasible basin (Shen et al., 2024).
STAMP extends this principle by relaxing symbolic actions with softmax variables and applying Stein Variational Gradient Descent. Its particles represent both symbolic-plan logits and continuous controls. The attraction term moves particles toward low-cost plans, while the repulsive kernel term preserves diversity across task and motion modes. The final discrete plan is obtained by selecting the maximum-logit action at each stage (Lee et al., 2023).
Differentiable task assignment and motion planning uses association weights 2 between robot 3 and object 4. Pick, place, release, and handover events emerge as temporal patterns in these weights. Kinematic constraints are activated by multiplying them by the association weights, permitting a single nonlinear optimization to represent robot assignment, handovers, resting objects, and some environmental interactions (Envall et al., 2023).
4. Planning under change, uncertainty, and temporal specifications
Receding-horizon and reactive TAMP
A precomputed plan can become invalid when objects move, perception changes, an action fails, or a human intervenes. Receding Horizon TAMP repeatedly solves a finite future window, executes only the first action, observes the resulting state, and replans. The horizon trades look-ahead against responsiveness: larger horizons provide more global information but become stale more easily, whereas smaller horizons respond faster but may incur more replanning (Castaman et al., 2020).
Reactive behavior-tree architectures integrate symbolic planning with execution monitoring. A transition system and an LTL automaton are combined into a product automaton, and the resulting symbolic plan is converted into a dynamically reconfigurable behavior tree. The system can reuse valid subtrees after object relocation and invoke full replanning only when local recovery is insufficient (Li et al., 2021).
Object-centric Cartesian TAMP is reactive at the controller level but not fully closed-loop at the symbolic level. It can compensate for moving targets and moderate perception or control errors, but a failed grasp, disappearing object, or major symbolic-state change requires symbolic replanning (Migimatsu et al., 2019).
Partial observability and risk
Classical TAMP commonly assumes complete state knowledge and deterministic actions. TAMPURA instead models TAMP as planning over beliefs. Closed-loop controllers such as Pick, Place, Look, Push, and Navigate are represented through uncertain-effect operators containing preconditions, guaranteed effects, uncertain effects, and conditions influencing outcome distributions.
A concrete belief is abstracted into Boolean belief predicates such as pose certainty, object classification, visibility, hand status, localization confidence, or collision risk. Bayesian transition models are learned through simulated controller executions, and a sparse abstract belief-state MDP is solved using LAO*. Deterministic planning is used to identify relevant transitions for model learning, while final decision-making remains stochastic and can prefer information gathering or risk avoidance (Curtis et al., 2024).
Learned feasibility and visual grounding
Learning can guide TAMP without replacing symbolic planning or geometric refinement. GROP uses a fully convolutional network to transform overhead images into feasibility heatmaps. The heatmap estimates the probability that a robot pose supports navigation and subsequent manipulation. Task-level action feasibility combines these predictions with motion and execution cost:
5
This allows the planner to trade success probability against navigation cost in mobile manipulation (Zhang et al., 2022).
PIGINet predicts the feasibility of an entire task plan rather than only the next action. A Transformer receives action sequences, goals, initial-state literals, continuous values, and visual object representations. Candidate symbolic plans are ranked by predicted refinement feasibility before conventional geometric refinement. The predictor is therefore a search-ordering and pruning heuristic; the underlying task planner, samplers, collision checking, and refinement remain conventional (Yang et al., 2022).
Temporal logic
Temporal logic represents requirements involving ordering, persistence, and eventuality. An LTL specification may require that all objects eventually enter a region and remain there:
6
The task transition system is composed with a Büchi automaton to form a product automaton. Search then finds paths satisfying both the task transitions and the temporal specification. Metric and Signal Temporal Logic extend this representation to timing and continuous signals such as position, velocity, force, and contact (Li et al., 2021, Zhao et al., 2024).
5. Execution architectures and applications
TAMP has been applied to manipulation, navigation, mobile manipulation, rearrangement, locomotion, multi-robot coordination, modular robots, and human-robot interaction.
Manipulation and rearrangement
Representative manipulation primitives include picking, placing, pushing, cooking, cleaning, opening drawers, stacking, and transporting objects. Non-monotonic rearrangement requires temporary displacement and restoration of objects. The relevant difficulty is that a locally correct action may obstruct a later grasp or placement.
U-TAMP evaluated green-block arrangement and cooking tasks involving object-support and object-container relations. The reported planning times were 7 seconds for a 46-action green-block plan and 8 seconds for a 57-action cooking plan, with successful execution in both tested cases (Agostini et al., 2023).
Mobile manipulation
Mobile manipulation couples navigation with manipulation. GROP evaluated tasks involving a mobile base, a UR5e arm, obstacles, loading and unloading, and multiple transported objects. Its learned feasibility maps capture approach-side effects that are difficult to express through deterministic symbolic locations.
The method was trained using a reported dataset of 9 simulated mobile-manipulation trials or instances. In the velocity experiment, reported completion rates for GROP were 0, 1, and 2 at speeds of 3, 4, and 5 meters per second, respectively (Zhang et al., 2022).
Navigation among movable obstacles
Navigation Among Movable Obstacles requires the robot to decide whether to move through a route, open a door, relocate an object, load a parcel, or change another task-state-dependent obstacle. Meta-engine and topological-refinement methods use failed motion searches to identify blocked regions and relevant obstacles, thereby guiding symbolic decisions.
Benchmark domains include Doors, Maze, Delivery, and Rovers. The studied actions include movement, opening, loading, unloading, sampling, calibration, photographing, and communication. The experiments emphasize infeasible symbolic actions, large task spaces, non-monotonicity, and task–motion trade-offs (Tosello et al., 2024).
Locomotion and modular robots
RHH-LGP was applied to crawler robots, mobile bases, Panda arms, obstacle avoidance, climbing, and modular robot connection. Actions included step, stepTogether, connect, and disconnect. Connecting modules changes the kinematic structure and can produce heterogeneous robots such as a crawler attached to a mobile base.
The reported tasks include one- and two-crawler climbing, multi-goal climbing, multi-robot pick-and-place, modular pick-and-place, and obstacle avoidance. Receding-horizon planning reduced planning time for several long-horizon tasks and enabled instances that the non-iterative solver did not solve (Braun et al., 2021).
Human-robot interaction and safety
Reactive TAMP systems can model human interventions as changes in the observed task state. A human may relocate, add, or remove objects, provide a tray, or alter the available workspace. Behavior-tree recovery can sometimes continue execution without task-level replanning, while changes to the object or region set may require transition-system reconstruction.
TAMPURA additionally models human-safety risk through probabilistic occupancy grids whose values decay over time. Collision probabilities are incorporated into motion-level feasibility and high-level action selection, allowing the robot to select object orders or wait for a human to clear the workspace (Li et al., 2021, Curtis et al., 2024).
6. Guarantees, limitations, and research directions
Completeness and optimality
TAMP guarantees depend on the representation and solver. Symbolic planning may be complete over a finite discrete model while remaining incomplete with respect to physical feasibility. Sampling-based motion planners are often probabilistically complete under suitable assumptions, but finite failure does not prove infeasibility. Local trajectory optimization is generally not complete and may converge to local minima. Mixed-integer formulations can support stronger global-search semantics but are computationally difficult.
COAST establishes probabilistic completeness under assumptions that the task planner is complete, streams are probabilistically complete, adaptive sampling eventually finds feasible refinements, and failed planning states are revisited (Vu et al., 2024). Sketch-based methods provide conditional width-1 and probabilistic-completeness arguments under assumptions concerning features, free space, sampling, and motion planning (Dalmau-Moreno et al., 2024). Reactive LTL methods provide completeness primarily for the finite modeled symbolic transition system and available motion skills, not arbitrary continuous motion (Li et al., 2021).
Optimization-based and differentiable TAMP methods generally do not guarantee global optimality. Particle methods improve basin coverage but remain sensitive to initialization, penalty weights, differentiability, and GPU capacity. Learned feasibility predictors can introduce false positives and false negatives; strict rejection of low-scoring plans can compromise completeness (Yang et al., 2022).
Common limitations
Important limitations across TAMP systems include:
- Symbolic-state mismatch: abstract predicates may omit reachability, clearance, graspability, or stability.
- Geometric nonconvexity: narrow passages, contact modes, and collision constraints create disconnected feasible regions.
- Local minima: nonlinear optimization may fail despite the existence of a feasible solution.
- Sampling inefficiency: independently sampled grasps, placements, and trajectories may be mutually incompatible.
- Model dependence: performance depends on accurate geometry, dynamics, perception, and action models.
- Execution uncertainty: grasp failure, tracking error, object motion, and perception error invalidate open-loop plans.
- Domain-engineering burden: action schemas, streams, geometric constraints, features, sketches, predicates, and controllers must be specified.
- Scalability: object count, plan horizon, robot degrees of freedom, contact modes, and continuous parameter dimension increase computational cost.
- Incomplete robot-level feasibility: some object-centric methods do not fully model robot-link collision, joint limits, torque, or dynamics.
- Restricted uncertainty models: many systems assume deterministic transitions and full observability.
Architectural trade-offs
No single TAMP architecture dominates all problem classes.
- Hierarchical TAMP is modular and efficient when symbolic plans are geometrically informative, but requires backtracking when downward refinement fails.
- Sampling-based TAMP handles black-box geometry and high-dimensional spaces, but can be sample-inefficient and cannot generally certify infeasibility.
- Optimization-based TAMP captures strong coupling and can optimize motion quality, but is local and initialization-sensitive.
- Receding-horizon TAMP adapts to changes and limits optimization size, but may sacrifice long-horizon foresight.
- Object-centric representations make manipulation relationships robust to object motion, but may leave robot-level feasibility to lower layers.
- Learned feasibility models reduce expensive refinement calls, but can misclassify rare or out-of-distribution plans.
- Differentiable relaxations enable gradient-based joint optimization, but may lose exact discrete semantics and struggle with discontinuous contacts.
- Constraint-refinement methods exploit reusable failure explanations, but overly strong constraints can remove feasible alternatives.
- Belief-space TAMP supports information gathering and risk awareness, but requires controllers, belief predicates, and transition models.
Future directions
Research directions include:
- integrating symbolic planning, continuous optimization, and probabilistic belief-state reasoning;
- learning task-relevant predicates, sketches, streams, and action abstractions;
- combining sampling-based global exploration with differentiable local refinement;
- improving failure explanations and unsatisfiable-core extraction;
- extending TAMP to deformable objects, liquids, articulated bodies, force interaction, and non-quasistatic dynamics;
- supporting multiple simultaneously moving agents and human collaborators;
- incorporating richer temporal logic and continuous-time specifications;
- improving perception-aware, risk-sensitive, and belief-space planning;
- developing scalable GPU-parallel and distributed solvers;
- integrating LLMs and vision-LLMs with formal planning and motion verification;
- reducing sim-to-real mismatch through learned residual dynamics, execution monitoring, and online adaptation.
The central research problem remains the effective integration of symbolic structure with continuous physical feasibility. TAMP systems increasingly treat this integration not as a fixed pipeline but as a bidirectional process:
6
Its major methodological alternatives—object-centric representations, receding-horizon planning, topological refinements, sketch decompositions, learned feasibility prediction, stochastic belief abstraction, differentiable relaxations, and GPU-parallel optimization—address different parts of the same discrete-continuous search problem. The resulting field spans classical symbolic planning, motion planning, nonlinear optimization, sampling, machine learning, formal methods, and feedback control.