Modular Prompting (MoT) Techniques

Updated 27 November 2025

Modular Prompting (MoT) is defined by decomposing complex tasks into specialized submodules, enabling structured reasoning and improved output composition.
Techniques include explicit reasoning graphs, learned modular routing, and differentiable modules, which together enhance performance in areas like code generation and text classification.
Empirical studies show that MoT methods yield superior performance, parameter efficiency, and adaptability across varied domains such as programming, multimodal learning, and adaptive tutoring.

Modular Prompting (MoT) encompasses a spectrum of techniques that restructure how LLMs are guided, trained, and adapted to handle complex and heterogeneous tasks. By organizing prompts, reasoning, or learned control in discrete, specialized modules, MoT methods address limitations of monolithic or linear prompt formats, yielding superior performance, interpretability, and adaptability across domains including code generation, text classification, adaptive tutoring, and multimodal learning.

1. Foundational Principles and Formalism

Modular Prompting is defined by the decomposition of complex tasks into smaller, independent units which are mapped to either explicit prompt components, structured reasoning graphs, or learned prompt modules. The general formalism involves:

Decomposition mechanism: Partitioning the original task $T$ into a set of subtasks or modules $\{T_i\}$ , either through explicit multi-level reasoning graphs (e.g., MLR Graphs for code (Pan et al., 16 Mar 2025)) or latent modular architectures (e.g., PRopS (Pilault et al., 2023)).
Module representation: Each module is annotated or implemented as a prompt fragment, continuous embedding, or function reflecting a distinct subproblem, reasoning step, or expert behavior.
Compositional execution: The final solution is constructed by combining module outputs, either hierarchically (traversal of a reasoning graph), sequentially, or via learned gating/routing.

For example, MoT in code generation defines a reasoning graph $\mathrm{MLR\_G} = (V, E)$ with nodes partitioned into high-level ( $V^{(H)}$ ), intermediate ( $V^{(I)}$ ), and detailed ( $V^{(D)}$ ) tasks, annotated with purpose, rationale, and strategy attributes, and generates code by hierarchically instantiating each module (Pan et al., 16 Mar 2025).

Modular Prompting can be instantiated as:

Rule-based decomposition frameworks (explicit program generation (Khot et al., 2022), function-split code (Li et al., 2023)),
Continuous modular prompts (differentiable rule modules with gating (Pilault et al., 2023), composable label prompts (Chen et al., 2022), mixture-of-prompts with soft selection (Dun et al., 2023)),
Mixture-of-expert prompting (automated region-wise assignment of instructions and demos (Wang et al., 28 Jun 2024)).

2. Modular Prompting Methodologies

2.1 Explicit Reasoning and Code Generation

In software-related tasks, Modularization-of-Thought (MoT) applies hierarchical decomposition, such as mapping a programming description $T$ to an MLR Graph. Each node $n \in V$ is labeled with Task Purpose $\pi(n)$ , Decision Rationale $\rho(n)$ , and Execution Strategy $\sigma(n)$ . The system then traverses from $V^{(H)}$ to $V^{(D)}$ , generating modular code components (functions, blocks), yielding improved alignment between reasoning and code structure (Pan et al., 16 Mar 2025, Li et al., 2023).

2.2 Programmable Prompt Libraries

Decomposed Prompting (“DecomP”) treats the overall solution as a program built by a decomposer LLM, whose subroutines are themselves specialist prompt modules or symbolic APIs. Subtasks can be further decomposed recursively, achieving efficient breakdown of symbolic reasoning, long-context, or multi-hop QA (Khot et al., 2022).

2.3 Differentiable Modular Prompting

Prompt Production System (PRopS) learns a set of rule modules $\{r_i\}$ , each a differentiable function, with input-dependent sparse gating $\alpha(x)$ , yielding the composed prompt $p = \sum_i \alpha_i(x) r_i(h)$ (Pilault et al., 2023). This enables conditional and compositional adaptation, with strong compositional generalization and parameter efficiency.

2.4 Mixture-of-Prompts and Modular Routing

Mixture-of-Prompts (MoP) methods maintain a bank of $K$ learned prompt modules, with a (learned) gating or routing mechanism to select or blend modules per input instance. These include:

Smart MoP: Softmax/MLP gating over a bank of trainable prompt matrices, dynamically creating a composite prompt $P^*(x) = \sum_i w_i(x)P_i$ (Dun et al., 2023).
Automated MoE prompt construction: Cluster demos based on embedding similarity, assign region-specific instructions, and route test instances to the nearest expert cluster (Wang et al., 28 Jun 2024).
Multi-task modular prompt tuning: Pretrain $K$ modular prompts across multiple tasks and learn sparse router weights for downstream adaptation (Sun et al., 2022).

2.5 Label-Modular Prompting

In non-stationary text classification, ModularPrompt assigns each class label a separate learnable soft-prompt. At inference, prompts for the current label space are composed and prepended to the input, enabling robust subset invariant classification and modular continual learning (Chen et al., 2022).

2.6 Structured Prompting in Multimodal and Conversational Systems

PromptFuse: Per-modality soft prompt blocks serve as “bridges” between frozen encoders and the core LLM, enabling modularity and parameter efficiency (Liang et al., 2022).
Modular Prompted Chatbot (MPC): Conversational systems structure modules for clarification, retrieval, memory processing, and summarization as individually prompted LLM components, improving long-turn consistency (Lee et al., 2023).

3. Empirical Performance and Comparative Results

Multiple studies demonstrate that MoT-based techniques systematically outperform baselines relying on monolithic prompt tuning or simple Chain-of-Thought (CoT):

Method	Task/Benchmark	Score/Metric	Paper
MoT (GPT-4o-mini)	HumanEval Pass@1	92.1% (+3-8 pts vs baselines)	(Pan et al., 16 Mar 2025)
MoTCoder	CodeContests pass@5	12.73% (vs 1.55–3.20%)	(Li et al., 2023)
PRopS	SCAN Seq. Accuracy	92.1% (vs prompt tuning 82.5%)	(Pilault et al., 2023)
ModularPrompt	Stage-agnostic F1	+14–18pp over previous methods	(Chen et al., 2022)
MoP	Win-rate	81% (vs prior arts, NLP tasks)	(Wang et al., 28 Jun 2024)
PromptFuse	VQAv2 (128-shot)	28.3% (vs 26.8% full finetune)	(Liang et al., 2022)
MPC	Chat SCE-p	83% (vs BB3-30B, vanilla LM)	(Lee et al., 2023)

Ablation experiments reveal that:

Removing graph-based modularization or structured prompt composition causes significant drops (e.g., –7–19% pass@1) (Pan et al., 16 Mar 2025).
Label modularity is critical: dropping the ground-truth label prompt reduces accuracy to 1–4% in ModularPrompt (Chen et al., 2022).

MoT methods yield particular gains in:

Compositional generalization (transferring to novel mixtures/task subsets) (Pilault et al., 2023, Sun et al., 2022).
Adaptability to heterogeneity (multi-task/federated settings) (Dun et al., 2023, Wang et al., 28 Jun 2024).
Parameter efficiency in low-resource settings, with prompt parameter counts orders of magnitude below full model tuning (Pilault et al., 2023, Liang et al., 2022).
Robustness under changing or dynamic label/task regimes (Chen et al., 2022).

4. Design Patterns and Implementation Strategies

4.1 Prompt Template Engineering

Explicit prompting templates define modular structure for the LLM (e.g., listing phases for module header generation, then implementation). Examples include:

Hierarchical code prompts specifying function-level decomposition, linking docstring annotations to reasoning attributes (Pan et al., 16 Mar 2025, Li et al., 2023).

Templates for modular prompting may request explicit reasoning graphs (e.g., MLR) or hierarchically structured bullet lists, then procedurally generate code or outputs for each module in order (Pan et al., 16 Mar 2025).

4.2 Learned Modular Routing

Modular prompt production systems utilize learned encoders, differentiable rule modules, and sparse input-dependent routing to select an optimal composition of prompt fragments per instance (Pilault et al., 2023, Dun et al., 2023, Sun et al., 2022). Softmax, Gumbel-softmax, or binary concrete relaxations are used for gating, with routers trained either by gradient descent or black-box optimization.

4.3 Compositional Adaptation and Transfer

MoT frameworks support rapid adaptation to new tasks via:

Sparse combination of a frozen prompt module bank, adjusting only a small number of router weights (Sun et al., 2022).
Label-wise prompt transfer via similarity initialization, supporting continual label-space extension (Chen et al., 2022).
Automated region-wise prompt assignment by semantic clustering and joint instruction search (Wang et al., 28 Jun 2024).

This compositionality allows prompt modules to be recombined for unseen task mixtures, domain shifts, or changing output spaces, with empirical evidence for smooth transfer.

4.4 Multimodal and Control Prompt Modularity

Prompt modules are assigned per modality (vision, text, audio); at inference, the modular prompt block is simply concatenated with the projected input features, providing a scalable scheme for adding new modalities or domains (Liang et al., 2022).

In adaptive/scaffolded prompting (e.g., education), boundary prompts, fuzzy-logic control schemas, and adaptation modules are composed at runtime according to user or task state, supporting interpretable and traceable control (Figueiredo, 8 Aug 2025).

5. Limitations, Challenges, and Open Directions

Although modular prompting delivers significant performance and adaptation benefits, several limitations remain:

Coverage and granularity: Fixed module or rule banks may not capture optimal granularity of task decomposition (Pilault et al., 2023, Sun et al., 2022).
Module search and assignment: Automated cluster-based module discovery may yield unbalanced or suboptimal clusters; dependency on prompt proposal algorithms (APE) affects MoP (Wang et al., 28 Jun 2024).
Order and routing sensitivity: Theoretical insight into order effects and the robustness of prompt routing (especially in MoE paradigms) remains incomplete (Wang et al., 28 Jun 2024, Chen et al., 2022).
Robustness to sampling and language variance: Stochasticity in prompt decomposition can yield divergent reasoning graphs or code, requiring consensus methods (Pan et al., 16 Mar 2025).
Language and domain specificity: Most frameworks are tuned for Python/code or English NLP; extending to structured, statically typed, or non-English domains requires prompt and annotation redesign (Pan et al., 16 Mar 2025, Sun et al., 2022).

Future research directions include:

Automated feedback-driven correction loops for reasoning graphs (Pan et al., 16 Mar 2025).
Lifelong learning schemes that dynamically expand the modular prompt bank as new tasks arise (Pilault et al., 2023).
Integrating retrieval-augmented prompting for prompt module reuse (Pan et al., 16 Mar 2025).
Joint optimization of prompt assignment, module order, and explicit inter-module interfaces (Wang et al., 28 Jun 2024).

6. Impact and Broader Connections

Modular Prompting represents a fundamental rethinking of LLM task adaptation, emphasizing compositionality, interpretability, and parameter economy. It unifies diverse approaches—hierarchical reasoning, program synthesis, mixture-of-experts, and compositional multi-tasking—under a broad principle of structured, modular decomposition. This paradigm offers a principled and empirically effective alternative to both monolithic prompt tuning and full-model finetuning in a wide range of language, code, and multimodal problems.

Key works exemplifying the modular prompting framework include "Modularization is Better: Effective Code Generation with Modular Prompting" (Pan et al., 16 Mar 2025), "MoTCoder: Elevating LLMs with Modular of Thought for Challenging Programming Tasks" (Li et al., 2023), "On Conditional and Compositional LLM Differentiable Prompting" (Pilault et al., 2023), "Learning Label Modular Prompts for Text Classification in the Wild" (Chen et al., 2022), "One Prompt is not Enough: Automated Construction of a Mixture-of-Expert Prompts" (Wang et al., 28 Jun 2024), "Decomposed Prompting: A Modular Approach for Solving Complex Tasks" (Khot et al., 2022), and "Prompted LLMs as Chatbot Modules for Long Open-domain Conversation" (Lee et al., 2023).