Three-Stage LLM Pipeline

• Three-stage LLM pipelines are structured workflows that decompose complex model tasks into sequential phases for improved quality, scalability, and cost efficiency.
• This architecture employs specialized stages—initial filtering, intermediate generation, and final validation—to optimize applications like data curation, QA benchmarking, and active learning.
• Empirical studies demonstrate that these pipelines reduce annotation costs while maintaining high performance through modular design and adaptive quality control.

A three-stage LLM pipeline is a modular, multi-phase workflow that decomposes complex LLM-centric tasks—such as dataset construction, inference, annotation, or downstream evaluation—into three distinct, logically sequential stages. Each stage fulfills a specialized function, interacting with LLMs (or other models) through carefully designed interfaces, algorithmic checkpoints, and quality-control mechanisms. This architecture has become central in large-scale data curation (Kim et al., 18 Nov 2024), multi-stage QA benchmarking (Lei et al., 22 Sep 2025), cost-efficient deployment (Ni et al., 18 Apr 2025), active learning workflows (Chen et al., 9 Sep 2025), human-like annotation and relevance assessment (Schnabel et al., 24 Jan 2025), and a range of applications where efficiency, transparency, and robustness are paramount.

1. Rationale and Principle of Three-Stage Decomposition

The use of three-stage LLM pipelines is motivated by the need to address the scalability, modularity, quality, and cost constraints inherent in LLM-centric workflows. Key objectives include:

• Specialization: Partitioning a complex pipeline into discrete logical phases (e.g., filtering, generation, validation) each targeted at a single operational goal. This allows for dedicated algorithmic or model choices per phase.
• Efficiency: Early pipeline stages typically prune the input space or perform lightweight filtering, preventing unnecessary computation or annotation burden downstream (Kim et al., 18 Nov 2024Schnabel et al., 24 Jan 2025).
• Quality Control and Robustness: Interleaving rule-based, programmatic, and model-based validation steps ensures that outputs are reliable and closely aligned with operational requirements (Lei et al., 22 Sep 2025Fang et al., 30 May 2025).
• Modularity and Reusability: Each stage operates as an independent module, enabling easy reconfiguration and update as model, data, or domain-specific requirements evolve (Pehlke et al., 10 Nov 2025Ni et al., 18 Apr 2025).

This staged decomposition is typically implemented via programmatic orchestration logic that governs both the flow of data and the feedback between stages, often augmented with adaptive feedback loops to optimize performance or data quality.

2. Methodological Patterns and Instantiations

There are several canonical three-stage LLM pipeline architectures, each adapted to the target workflow. Key instantiations include:

• Data Curation Pipelines (Kim et al., 18 Nov 2024):
  • Stage 1: Dataset extraction and initial deduplication (HTML cleaning, language ID, domain filtering).
  • Stage 2: Heuristic and model-based filtering (rule filters, global deduplication via MinHash LSH, probabilistic quality scoring).
  • Stage 3: Domain curation and packaging (domain classification, partitioning for downstream model training).
• Multi-Stage QA Benchmark Generation (Lei et al., 22 Sep 2025):
  • Stage 1: Dynamic sampling balancing “seed” reuse and new content generation using precisely controlled probabilities.
  • Stage 2: Iterative LLM-driven question generation and stepwise answer construction leveraging in-context learning.
  • Stage 3: Multi-level quality control—rule-based, programmatic (perplexity, similarity), and adjudicator LLM-based filtering.
• Active Learning for Demonstration Selection (Chen et al., 9 Sep 2025):
  • Stage 1: Diversity sampling for maximal input coverage.
  • Stage 2: Similarity sampling targeting high-utility demonstrations.
  • Stage 3: Uncertainty sampling focusing annotation budget on “hard” cases with low demonstration similarity.
• Cost-Efficient Model Deployment (Ni et al., 18 Apr 2025):
  • Stage 1: Prototyping with a function-call-driven LLM as a data-generating teacher.
  • Stage 2: Knowledge transfer via supervised fine-tuning, RL, and adaptive knowledge distillation.
  • Stage 3: Model compression by pruning and quantization.
• Relevance Annotation for Search (Schnabel et al., 24 Jan 2025):
  • Stage 1: Binary irrelevance filter.
  • Stage 2: Coarse, multi-grade relevance classification.
  • Stage 3: Escalation to high-fidelity judgment for ambiguous items.

These patterns generalize: initial stages aggressively reduce the input volume, intermediate stages expand or refine the semantic content, and terminal stages perform high-precision validation or escalation.

3. Algorithmic and Formulaic Underpinnings

Formally, each stage involves structured algorithms or decision rules. Typical examples include:

• Dynamic Sampling (Lei et al., 22 Sep 2025):

$$P^{(i)}_s = \frac{1}{\gamma}\bigl(1 - \frac{i}{N}\bigr);\quad n^{(i)}_s = \lfloor P^{(i)}_s \cdot T \rfloor;\quad n^{(i)}_g = T-n^{(i)}_s$$

• Entity Recognition Demonstration Corpus Construction (Chen et al., 9 Sep 2025):

$$\mathrm{dist}(x_i, \mathcal{D}_{\mathrm{selected}}) = \min_{x_j\in\mathcal{D}_{\mathrm{selected}}} (1 - U(x_i,x_j)),\quad \rho(a) = \max_{d\in\mathcal{D}_1} U(a,d)$$

• Quality Filtering Score (Kim et al., 18 Nov 2024):

$$Q(d) = \log P_{\text{class}="good"}(d) - \log P_{\text{class}="bad"}(d);\quad Q(d)\geq \kappa$$

• Pipeline Evaluation Metrics (Lei et al., 22 Sep 2025, Schnabel et al., 24 Jan 2025):
  • ROUGE-L F₁: Longest common subsequence for multi-stage answer coverage.
  • Robustness Ratio: $$\frac{\text{ROUGE}_\text{Hard}}{\text{ROUGE}_\text{Easy}}$$
  • Krippendorff's $\alpha$ (relevance): Used for assessing inter-rater reliability across pipeline outputs.
  • Cost-Efficiency Measures: Computed as USD/token processed per stage (Schnabel et al., 24 Jan 2025).

4. Quality Control, Feedback, and Escalation

A salient characteristic of three-stage pipelines is multi-level, often interleaved, quality control:

• Rule-based format checks: Language consistency, artifact stripping, JSON schema validation.
• Programmatic filters: Perplexity thresholds, near-duplicate detection (e.g., via similarity matrices).
• Model-based, professional assessment: Adjudicator LLMs or secondary models evaluate critical features (e.g., multi-stage step coverage, naturalistic answer structure, practical feasibility) and triage outputs with explicit cutoff scores ($\mathrm{Score}(d) < \tau$; cf. pipeline in (Lei et al., 22 Sep 2025)).
• Human and Artificial Evaluation: Performance is often assessed via both expert human raters (“Turing Test accuracy,” manual relevance/judgment scores) and automated metrics (ROUGE, Krippendorff's α).

Feedback from QC modules informs upstream prompt templates and data generation, creating an active loop that iteratively refines the pipeline until data quality targets are met (Lei et al., 22 Sep 2025, Ni et al., 18 Apr 2025).

5. Applications Across Domains

Three-stage LLM pipelines are widely applied:

Application	Typical Stages	Principal Metrics
Data Curation (Kim et al., 18 Nov 2024)	Extraction → Filtering → Curation	Precision, recall, cost
QA Benchmarking (Lei et al., 22 Sep 2025)	Sampling → Generation → Quality Control	ROUGE-L, robustness, Turing acc
Relevance Annotation (Schnabel et al., 24 Jan 2025)	Filter → Coarse Grading → Adjudication	Krippendorff’s α, cost
Active Learning (Chen et al., 9 Sep 2025)	Diversity → Similarity → Uncertainty	F1, annotation convergence
Model Deployment (Ni et al., 18 Apr 2025)	Prototyping → Transfer → Compression	AR, throughput, QPS, latency

These pipelines are engineered to yield scalable, high-fidelity outputs within annotation cost budgets as low as 5–10% of full dataset annotation required for maximal performance (Chen et al., 9 Sep 2025), and to provide transparent, auditable intermediates for downstream audit or improvement (Pehlke et al., 10 Nov 2025).

6. Empirical Outcomes and Comparative Analysis

Empirically, three-stage LLM pipelines deliver:

• Cost-effectiveness: In modular pipelines for relevance assessment, cost per million tokens drops from $5 (flagship, single-stage) to$0.2–$0.25 with only a minor drop—or even an increase—in accuracy ($\alpha$ up +18.4%) (Schnabel et al., 24 Jan 2025).
• Annotation efficiency: Annotating only 5–10% of a pool via three-stage active learning matches full-annotation F1 (Chen et al., 9 Sep 2025).
• Quality gains: Multi-level QA pipelines systematically yield absolute gains in both coverage and robustness, for instance: commercial LLMs outperform open-source, but all lose quality on noisy inputs; the robustness ratio quantifies the generalization gap across task complexity (Lei et al., 22 Sep 2025).
• Transparent, auditable AI: Modular three-stage reasoning (e.g., factor analysis → normal form games → sequential games) provides traceable, intermediate artifacts aligned with expert workflows (Pehlke et al., 10 Nov 2025).

7. Limitations and Extensions

Current limitations of three-stage LLM pipelines include:

• Trade-offs between granularity, speed, and resource usage, particularly when CPU-based pipelines are used in lieu of larger GPU reward models (Kim et al., 18 Nov 2024).
• Domain and language coverage restrictions: Many pipelines are currently optimized for high-resource languages and specific domains, although approaches for Basque and other low-resource languages have been described (Corral et al., 18 Dec 2024).
• Upstream dependency propagation: Errors or misclassifications in early stages can propagate, motivating the design of adaptive re-filtering or escalation mechanisms.

Extensions include retraining or swapping of individual modules to accommodate new domains, new linguistic settings, or expanded functional requirements, leveraging the intrinsic modularity of the three-stage approach (Pehlke et al., 10 Nov 2025, Ni et al., 18 Apr 2025). A plausible implication is that continued advances in orchestration logic and automated evaluation will further extend the generality and efficiency of three-stage LLM pipelines across and beyond NLP domains.