---
title: Prompt-Chaining Approach
url: https://www.emergentmind.com/topics/prompt-chaining-approach
type: topic
---

# Prompt-Chaining Approach

Prompt-chaining is a methodology for decomposing complex language-model tasks into structured sequences of discrete LLM invocations, where the output of each stage is fed as input to the subsequent stage. The primary rationale is to orchestrate sub-task specialization, error isolation, and explicit intermediate representation, yielding gains in controllability, transparency, and accuracy—particularly when input sequences are long or domain complexity is high. In legal document analysis, prompt-chaining pipelines can overcome fundamental bottlenecks of context window limits, information overload, and domain-specific language, making them especially well-suited for tasks such as long-text classification, contract QA, and similar high-stakes applications [2308.04138][2410.12840].

## 1. Architectural Principles and Formal Definition

A prompt-chaining pipeline is a composition of modular LLM calls, each encapsulated as a function $f_i$, often with distinct prompt templates and domain-adapted constraints. Formally, if $x$ is the original input (e.g., full document), the chain is
$$
y_{n} = f_{n}( \dots f_2( f_1(x) ) \dots )
$$
with each $f_i$ potentially depending on intermediate outputs $y_{i-1}, y_{i-2}, ...$ as required.

In long legal document classification [2308.04138], the canonical three-stage chain consists of:
1. **Summarization ($f_1$)**: Compress and distill input $x$ to a 128-token, task-relevant summary using a domain-tuned abstractive model (e.g., PRIMERA).
2. **Exemplar Retrieval ($f_2$)**: Retrieve $k$ labeled in-context exemplars by computing cosine similarity in an embedding space (e.g., LegalBERT sentence-transformer).
3. **Few-shot Generation ($f_3$)**: Construct a prompt that concatenates the $k$ nearest (summary, label) pairs and requests a label prediction for the new summary via LLM decoding.

For contract QA [2410.12840], a two-stage chain is employed:
- **Stage 1** ($f_1$): Extract only those facts in the clause relevant to the posed question (“distillation”).
- **Stage 2** ($f_2$): Map the distilled intermediate to a discrete set of options.

## 2. Prompt-Chaining Workflows in Legal Domains

The prompt-chaining approach enables efficient processing of documents far exceeding standard transformer windows (e.g., ECHR, SCOTUS opinions spanning 2048+ tokens):

**Document Summarization:**
- Input is iteratively segmented into context-window-compatible spans (1,024–2,048 tokens).
- Each span is summarized using PRIMERA (fine-tuned on legal opinion data).
- Sequence summaries are recursively condensed until a single global summary of 128 tokens remains.
- Default LLMs (e.g., GPT-NeoX, Flan-UL2) using generic summary prompts were found to dilute high-value legal signals; PRIMERA summaries retained legal issue structure and proved empirically superior [2308.04138].

**Semantic-Similarity Retrieval:**
- All documents (including test, train, dev) are pre-summarized.
- Each summary is embedded; for a given test query, $s(q, e) = \cos(\text{Emb}(q), \text{Emb}(e))$ is computed against training samples, yielding the top $k=8$ most similar exemplars.
- Exemplar (summary, label) pairs serve as explicit in-context learning signals for the subsequent prompt.

**Few-Shot Label Prompting:**
- Final prompt concatenates 8 retrieved pairs in descending similarity.
- For binary tasks (ECHR): label set is {YES, NO}; for multi-class (SCOTUS): restricted to labels present among nearest neighbors.
- Greedy decoding ($T=0$) or majority voting over $n$ samples (self-consistency) is used for the final label.

**Contract QA Chaining:**
- Stage 1 prompts force the LLM to report only explicit, question-relevant facts—no inference or assumption.
- Stage 2 receives this distilled content and applies structured option selection, outputting selected answers in JSON.

## 3. Empirical Evaluation and Performance Metrics

### Classification Task Results [2308.04138]:

**ECHR (binary):**
- Zero-shot GPT-NeoX (full doc): micro-F₁ = 0.709 (dev), 0.728 (test)
- Chained GPT-NeoX (8-shot, summaries): 0.770 (dev; +0.061), 0.756 (test; +0.028)
- NO-class F₁ improved from ≈ 0.15 (zero-shot) to 0.25 (few-shot)

**SCOTUS (13-way):**
- Zero-shot ChatGPT: micro-F₁ ≈ 0.438
- Flan-UL2 (chained, 8-shot): micro-F₁ = 0.545 (dev), 0.483 (test; +0.045 over ChatGPT)

Micro-F₁ improvements over strong zero-shot and single-prompt baselines are consistent, despite using smaller models.

### Contract QA [2410.12840]:
Summary of Prompt Chaining vs. single-stage:

| Prompt | Q1 EM | Q3 EM | Q4 EM |
|--------|-------|-------|-------|
| P1     | 0.47  | 0.51  | 0.68  |
| P3     | 0.51  | 0.51  | 0.54  |
| P4     | 0.57  | 0.58  | 0.68  |

Chaining (P4) yields +6–7 point EM gains on complex questions.

### Metrics

- Micro-F₁ and Macro-F₁ for classification
- Exact Match (EM), Precision, Recall for multi-select QA
- Self-consistency: negligible gains over greedy decoding, suggesting improvements are due to exemplar context, not majority voting.

## 4. Design Patterns, Component Modularity, and Practical Considerations

### Modularity:
- Each pipeline step is swappable. For example, summarizer, embedder, k-neighbor count, or decision rule can be replaced without affecting downstream structure.
- In contract QA, Stage 1 can be realized as summarization, slot-filling, or explicit right/obligation extraction; Stage 2 as any structured answer selection.

### Prompt Engineering:
- Task-specific summaries outperform generic LLM-generated ones; human review confirms greater retention of legal issues.
- Including answer options in Stage 1 summaries can guide focus but risks “pocket prompting” leakage—favors experiment-driven tuning.

### Limitations:
- Chaining exposes error propagation: poor Stage 1 outputs limit final answer quality.
- Linguistic variability (e.g., in force majeure scenarios) still leads to failures; neither exhaustive prompt definitions nor chaining fully solve high-variation semantics.

## 5. Theoretical and Methodological Rationale

Prompt chaining, by construction, enables fine-grained control over task decomposition and input filtering:
- **Error Isolation**: Partitioning allows upstream errors to be detected and corrected before contaminating final decisions.
- **Context Compression**: Summarization circumvents transformer context window bottlenecks.
- **Few-shot Efficiency**: Chained few-shot prompts yield performance matching or exceeding larger model zero-shot runs, reflecting improved in-context signal [2308.04138].

In contract QA, chaining aligns with the expert process: extract relevant facts, then reason/disambiguate on the distilled set [2410.12840].

## 6. Broader Implications and Future Work

The modular chaining paradigm positions each sub-task for focused error analysis and component-wise optimization. Future research avenues include:
- Task-specific intermediate extraction (beyond summarization) such as party rights/obligations, structured slot filling, or intent detection [2410.12840].
- Robustness across LLM versions (GPT-4, Llama-2, Mistral).
- Chaining more than two stages, with justification-explanation decoupling (e.g., requiring rationales in option selection).
- Model-level enhancements: pretraining on chains of contract QA pairs, data augmentation for high-variation phenomena.
- Addressing overfitting to known answer structures by prompt randomization and cross-exemplar testing.

Taken together, prompt-chaining delivers a scalable, transparent, and empirically validated approach for handling long, complex, and structured language modeling tasks in legal and other high-precision domains, systematically outperforming single-stage prompting and larger zero-shot LLMs [2308.04138][2410.12840].

Source: https://www.emergentmind.com/topics/prompt-chaining-approach