PlanGPT: Multi-Modal Planning Systems
- PlanGPT is a label for diverse planning-oriented large language model systems used in urban planning, classical PDDL planning, and vision-language map interpretation.
- It employs retrieval-based techniques, localized fine-tuning, and automated tool orchestration to convert complex planning inputs into structured outputs.
- Empirical evaluations reveal strengths in contextual plan generation alongside challenges with domain dependency, representational symmetry, and long-horizon reasoning.
PlanGPT is a label used in recent arXiv literature for multiple planning-oriented large-language-model systems rather than a single uniform architecture. In its most explicit form, it names a domain-specific stack for urban and spatial planning that combines retrieval, local fine-tuning, and tool orchestration (Birr et al., 2024). In a different line of work, later studies use “PlanGPT” to denote a GPT-2-style planner for classical PDDL domains that predicts action sequences from initial and goal descriptions (Abdelkader et al., 9 Jun 2026). The name has also been extended to multimodal map interpretation in urban planning through PlanGPT-VL (Zhu et al., 20 May 2025). Across these usages, the common theme is the attempt to convert planning problems, planning documents, or planning-relevant perceptual inputs into structured plans, planner-like outputs, or operational recommendations.
1. Scope, terminology, and antecedents
The research context for PlanGPT is shaped by two earlier agendas. One is formal evaluation of planning competence. PlanBench defines a planning problem as and evaluates eight tasks: plan generation, cost-optimal planning, plan verification, reasoning about plan execution, robustness to goal reformulation, ability to reuse plans, replanning, and plan generalization (Valmeekam et al., 2022). Its central finding is that LLMs can appear competent on surface-level goal reformulation while remaining weak on core reasoning about actions and change, including plan generation and execution reasoning (Valmeekam et al., 2022).
The second agenda is text-to-plan extraction. GPT3-to-plan frames plan extraction as few-shot prompting: natural-language procedure descriptions are presented as TEXT, structured outputs as ACTIONS, and GPT-3 is asked to complete the plan directly (Olmo et al., 2021). The paper reports that GPT-3 is promising for action-name extraction and can sometimes infer ordering, but trails specialized systems on full structured plan extraction, especially for arguments (Olmo et al., 2021). This established a concrete template for treating planning as a translation problem from prose to structured action sequences.
Within that landscape, the label “PlanGPT” has been attached to at least three distinct research objects: a specialized urban-planning assistant, a classical automated planner in PDDL, and a vision-language extension for planning-map interpretation. This suggests that the term functions less as a single canonical model family than as a recurrent naming pattern for planning-specialized GPT-derived systems.
2. PlanGPT in urban and spatial planning
In urban and spatial planning, PlanGPT is presented as “the first specialized LLM tailored for urban and spatial planning” (Birr et al., 2024). The motivation is operational rather than purely benchmark-driven: planners must draft official documents, retrieve relevant standards, review plans against regulatory requirements, and evaluate legality, feasibility, novelty, and utility, while general-purpose models often fail on document style, terminology, low signal-to-noise density, and privacy-sensitive deployment constraints (Birr et al., 2024).
The system is explicitly modular. Its three major components are a retrieval-oriented Vector-LLM, a domain-adapted Local-LLM, and a PlanAgent. Retrieval is formalized as span selection,
and is implemented through Plan-Emb and Plan-HS. Plan-Emb is a domain-specific embedding model trained first on general Chinese text labels and then fine-tuned on self-collected urban-planning data with a contrastive objective plus a KL regularization term to reduce forgetting (Birr et al., 2024). Plan-HS uses keyword extraction, vector search, hard keyword scoring, and cross-attention reranking to address the low signal-to-noise ratio of planning documents (Birr et al., 2024).
The Local-LLM component is built through “Urban planning Knowledge Activation” and “Specific Capability Development.” The paper reports nearly 50k high-quality instruction pairs after urban-planning annotation, and over 4,000 historical official plans collected from provinces, cities, districts, and counties for downstream task development (Birr et al., 2024). Task-specific datasets are created for text generation, style transfer, information extraction, and text evaluation. PlanAgent then adds autonomous todo-list generation, orienteering web search, professional tool invocation, and information integration across retrieved documents, local models, and external tools (Birr et al., 2024).
Empirically, the paper reports that PlanGPT attains the best overall average score of 83.34 across common urban planning tasks, outperforming general chat baselines in human and automatic evaluation (Birr et al., 2024). On urban planning knowledge assessment, the local LLM reaches 63.0 on the v1 exam-style set and 51.2 on the v2 set, with average 57.10 and (Birr et al., 2024). For retrieval, Plan-Emb reports URSTS-B 0.754 and AVG 0.589, slightly ahead of the strongest listed general semantic baselines on the in-domain test (Birr et al., 2024). The paper’s overall position is that effective planning assistance in this domain requires not only language modeling, but also localized retrieval, planner-aligned stylistic adaptation, and deployability in privacy-sensitive institutional settings (Birr et al., 2024).
3. PlanGPT as a classical automated planner
A distinct usage of PlanGPT appears in automated planning over PDDL domains. A complementary study describes PlanGPT as a from-scratch LLM inspired by GPT-2, trained to map PDDL problem descriptions to complete plans in the classical planning setting (Abdelkader et al., 9 Jun 2026). The model does not tokenize the full PDDL file; instead, it focuses on fluents from the :init and :goal sections and predicts actions and their objects one token at a time until an end token is produced (Abdelkader et al., 9 Jun 2026). It is not a single universal planner: the study emphasizes that it uses one model per domain and originally supports eight domains (Abdelkader et al., 9 Jun 2026).
The reported training setup uses 70,000 generated planning problems, with multiple valid plans per problem produced by LPG and randomized object renaming to reduce spurious correlations from identifiers (Abdelkader et al., 9 Jun 2026). The domains used in the later re-evaluation include Blocksworld, Depot, Driverlog, Logistics, Satellite, Zenotravel, and Floortile (Abdelkader et al., 9 Jun 2026).
A later symmetry-aware study further characterizes this PlanGPT as a decoder-only GPT-2-style transformer trained with standard next-token cross-entropy for plan generation (Fritzsche et al., 11 Aug 2025). Its input representation consists of tokenized predicate and object-name sequences for initial state and goal, followed by an <action> marker, after which the model autoregressively predicts the plan (Fritzsche et al., 11 Aug 2025). The same study argues that random object renaming and atom sorting do not solve the fundamental symmetry problem of planning inputs: object names are arbitrary identifiers, atom order is arbitrary, and learned positional embeddings create a length-generalization issue for harder instances (Fritzsche et al., 11 Aug 2025).
In this automated-planning sense, PlanGPT is therefore an action-sequence predictor over symbolic planning instances. Its appeal lies in treating planning as language generation over domain-specific training corpora; its core controversy lies in whether next-token action prediction is an adequate inductive bias for planning at all.
4. Empirical critiques and alternative inductive biases
Subsequent work has assessed PlanGPT critically. The 2026 complementary study finds that the original coverage claims are broadly reproducible, with coverage percentages “practically the same” and about a 10% average difference overall, but concludes that PlanGPT is not better than a Greedy search strategy once plan cost and generation time are included (Abdelkader et al., 9 Jun 2026). The reported aggregate numbers are 79% coverage for PlanGPT, 46% for A*, and 88% for Greedy; IPC cost scores of 5.02 for PlanGPT, 3.26 for A*, and 5.36 for Greedy; and IPC time scores of 4.95 for PlanGPT, 2.46 for A*, and 5.82 for Greedy (Abdelkader et al., 9 Jun 2026). The study emphasizes that these timings exclude training, which would further reduce PlanGPT’s practical attractiveness (Abdelkader et al., 9 Jun 2026).
The symmetry-aware transformer paper attributes much of PlanGPT’s extrapolation failure to representational symmetry. If a domain has , then object renaming alone yields $8!=40320$ equivalent assignments for the same underlying planning problem (Fritzsche et al., 11 Aug 2025). Because atom order is also arbitrary, the number of equivalent representations grows further by for state and goal permutations (Fritzsche et al., 11 Aug 2025). The paper reports that PlanGPT performs reasonably only on interpolation-size problems and essentially fails on extrapolation; by contrast, symmetry-aware encoder-decoder models solve many more extrapolation instances in Blocksworld, Gripper, and Visitall (Fritzsche et al., 11 Aug 2025).
A more direct conceptual critique appears in the learned-transition-models paper, which positions PlanGPT as an action-centric generalized planner that predicts plans autoregressively as action strings instead of learning the transition function (Gupta et al., 26 Feb 2026). The paper argues that this bypasses explicit world-state evolution and causes state drift in long-horizon settings. Its strongest empirical statement is that, under strict size extrapolation, all PlanGPT variants achieve 0.00 success across all four tested domains (Gupta et al., 26 Feb 2026). The same paper reports PlanGPT’s approximate size as ~125M parameters and contrasts it with much smaller learned transition models that can outperform direct action-sequence prediction on out-of-distribution satisficing-plan success in several domains (Gupta et al., 26 Feb 2026).
Taken together, these studies recast PlanGPT from “LLM planner” to a specific action-generation baseline whose main weaknesses are domain dependence, poor extrapolation, vulnerability to symmetries, and the absence of an explicit transition model.
5. Specialized descendants and multimodal expansion
The PlanGPT naming pattern extends beyond pure text planning. PlanGPT-VL is presented as “the first domain-specific Vision-LLM tailored specifically for urban planning maps” (Zhu et al., 20 May 2025). It addresses a different problem from PDDL plan generation: the interpretation of planning maps containing land-use zoning, infrastructure layouts, ecological restrictions, and policy-linked annotations (Zhu et al., 20 May 2025). The system combines PlanAnno-V for synthetic planning-map VQA data, Critical Point Thinking for Generate–Verify–Revise reasoning, and supervised fine-tuning with frozen vision encoder parameters (Zhu et al., 20 May 2025). On the 300-example PlanBench-V benchmark, PlanGPT-VL-7B achieves the best overall score of 1.566, with reported gains of 59.2% on specialized planning tasks and 72.1% over its Qwen2-7B base configuration (Zhu et al., 20 May 2025).
PlanBench-V later formalizes the evaluation gap that motivated this extension. It introduces the Spatial Planning Map Database with 223 planning maps and 1,629 question-answer pairs, organized around four capabilities: Perception, Reasoning, Association, and Implementation (Chen et al., 4 Jun 2026). The benchmark explicitly positions PlanGPT and PlanGPT-VL as prior planning-aware systems while arguing that planning-map-grounded multimodal understanding remained under-evaluated (Chen et al., 4 Jun 2026). Even after measurable progress from 2025 to 2026 models, the paper reports that all systems still struggle with implementation-oriented tasks that require evaluative judgment, policy sensitivity, and constraint-aware decision-making (Chen et al., 4 Jun 2026).
Related application-specific systems show the same design logic under different names. IncidentResponseGPT takes incident reports plus region-specific guidelines and synthesizes constrained traffic incident response plans, with guideline summarization, action formalization over lane, speed, VMS, and closure variables, and simulation-based validation (Grigorev et al., 2024). DisasterResponseGPT uses in-context learning with embedded FEMA guidance to generate three candidate plans of action that users can iteratively refine in natural language (Goecks et al., 2023). These systems are not named PlanGPT, but they instantiate the same broader pattern: domain-grounded plan generation supported by explicit operational schemas, retrieved or embedded guidance, and human-in-the-loop revision.
6. Limitations and research directions
Across its variants, PlanGPT exposes a recurrent tension between fluent language generation and dependable planning. In urban planning, the system improves document generation, retrieval, and evaluation, yet nuanced judgments remain difficult, prompt engineering alone is insufficient, and multimodal or rapidly changing regulatory content remains an open challenge (Birr et al., 2024). In classical automated planning, the PDDL-based PlanGPT remains domain-limited, uses separate models per domain, and is not reliably superior to simple symbolic baselines on cost and time (Abdelkader et al., 9 Jun 2026). The broader benchmark literature reinforces this skepticism: PlanBench concludes that even GPT-4 remains “pretty ineffective” on many critical planning capabilities, especially plan generation and reasoning about execution (Valmeekam et al., 2022).
Recent work points toward architectures that shift away from unconstrained action-token generation. GenePlan evolves domain-dependent Python planners and reports average SAT score 0.91, close to 0.93 for state-of-the-art planners, with generated planners solving new instances in average 0.49 seconds per task (Murray et al., 10 Mar 2026). A formal-verification study shows that natural-language plans can be translated into Kripke structures and LTL for model checking, with GPT-5 achieving F1 96.3% on a simplified verification task, while also showing that syntactic correctness does not guarantee semantic fidelity (Ramani et al., 3 Oct 2025). PlanCompiler separates planning from execution through typed JSON plans, static validation, and deterministic compilation, reaching 278/300 first-pass successes overall against 202/300 and 187/300 for direct free-form code-generation baselines (Harikumar, 8 Apr 2026).
This suggests a broader redefinition of what a successful PlanGPT-like system may need to be. Rather than a standalone autoregressive planner, the emerging pattern is a hybrid stack: domain-adaptive retrieval, constrained or typed intermediate representations, explicit verification, simulation or model-checking backstops, and reusable planner artifacts such as code or symbolic policies. Under that interpretation, PlanGPT is less a settled architecture than a research program centered on the specialization of LLMs for planning tasks that remain structurally resistant to unconstrained next-token prediction.
