Cost-Aware Knowledge Extraction Cascade

• The paper demonstrates that cost-aware cascades leverage a low-cost extractor initially and escalate to stronger models or human experts only when necessary.
• It details various architectures, such as post-text routing and edge-cloud-human pipelines, showing how thresholding and calibration balance accuracy with cost.
• Empirical findings reveal that cascaded systems can nearly match the strongest model's performance while substantially reducing expensive model invocations.

A cost-aware knowledge-extraction cascade is a staged decision system that allocates increasingly expensive extraction mechanisms only when cheaper stages are unlikely to suffice. Across recent work, the core pattern is to begin with a low-cost extractor, estimate whether its output is reliable enough, and then either accept, escalate to a stronger model or modality, or abstain to a human expert. The extracted “knowledge” may take the form of answers, structured fields, retrieved evidence, or model outputs used as stand-ins for labels, but the recurring objective is the same: optimize quality under explicit cost, latency, or access constraints (Dwivedi et al., 17 Jun 2026, Hou et al., 23 Dec 2025, Wu et al., 1 Jun 2026).

1. Concept and scope

The recent literature uses the term in several closely related senses. In multimodal question answering, SAFE-Cascade is a “post-text, multi-stage cascade” that runs OCR, obtains a provisional text-only answer, and then escalates to a VLM only when a learned router predicts that “visual grounding is needed” (Dwivedi et al., 17 Jun 2026). In telecom knowledge systems, the cascade is an edge-cloud-expert pipeline in which “an efficient edge model handles routine queries, a more capable cloud model addresses complex cases, and human experts are involved only when necessary,” with an explicit objective of minimizing expected processing cost under a misalignment constraint (Hou et al., 23 Dec 2025). In retrieval-augmented settings, the same idea appears as budgeted evidence acquisition across cost tiers, where the system decides “when to retrieve, which tier to access, and when to stop” (Wu et al., 1 Jun 2026).

The scope is broader than model-size routing alone. Some systems route across modalities rather than only across models, as in text-only versus VLM chart answering (Dwivedi et al., 17 Jun 2026). Others insert abstention as a first-class action, so that the system may stop without querying a larger model when the expected value of escalation is low (Zellinger et al., 13 Feb 2025). Some formulations place humans at the terminal stage rather than assuming a fully automated endpoint (Hou et al., 23 Dec 2025, Fanconi et al., 13 Jun 2025). There are also broader uses of the cascade idea outside inference-time LLM routing. FedKEM organizes client-side “knowledge extraction via deep mutual learning” and server-side ensemble distillation to reduce communication cost in heterogeneous federated learning (Nguyen et al., 2022), while KCas treats a small student as a cheap stage that transfers hyperparameters to an expensive teacher through asymptotic scaling laws (Fang et al., 24 Jun 2026). This suggests that the unifying property is not any single architecture, but staged allocation of scarce computation, access, or expert effort.

2. Canonical architectural patterns

Several recurring architectures appear across the literature.

Pattern	Stages	Representative systems
Post-text modality routing	OCR → text-only LM → learned router → optional VLM	SAFE-Cascade (Dwivedi et al., 17 Jun 2026)
Edge-cloud-human triage	edge LLM → cloud LLM → human expert	Telecom cascade (Hou et al., 23 Dec 2025)
Early-abstention cascade	small model → accept / escalate / early abstain	Early abstention (Zellinger et al., 13 Feb 2025)
Clustered routing with escalation	query clustering → per-cluster model choice → QE-based escalation	Cluster, Route, Escalate (Moslem et al., 25 Jun 2026)
Budgeted evidence cascade	retrieve low-cost evidence first, then higher-cost tiers if justified	Cost-aware RAG (Wu et al., 1 Jun 2026)

SAFE-Cascade is a particularly explicit instance of staged extraction. Its four stages are OCR extraction with Azure Document Intelligence, text-only answering with gpt-5-mini, learned routing with a Random Forest, and optional VLM escalation with gemini-2.5-flash-image. The router consumes only “inference-time, cheaply available signals” such as OCR length and density, question length, numeric or comparison flags, lexical overlap, answer-length indicators, and whether the provisional answer appears in OCR (Dwivedi et al., 17 Jun 2026).

The telecom framework uses a different decomposition. At each automated tier, it computes epistemic uncertainty $U(x)$ and confidence $C(x)$, applies the tests $U(x) < \epsilon$ and $C(x) > \lambda$, and either accepts, escalates, or forwards to a human expert. The edge and cloud stages share the same threshold vector $\phi=(\epsilon,\lambda)$, while the human expert is the ultimate fallback (Hou et al., 23 Dec 2025).

Other architectures modify the interior of the cascade rather than only the endpoints. CascadeDebate inserts “multi-agent deliberation directly at each tier's escalation boundary,” so that ambiguous cases can be resolved by lightweight intra-tier debate before a larger model or human is invoked (Chang et al., 14 Apr 2026). Inter-Cascade changes the semantics of escalation by turning the strong model into both “backup helper” and “long-term teacher”: when the strong model resolves a deferred query, it also distills a reusable strategy into a retrieval repository for future weak-model use (Wu et al., 26 Sep 2025). Cluster, Route, Escalate separates offline routing from online repair: Stage 1 clusters queries and assigns each cluster to its most cost-effective model, and Stage 2 uses quality estimation to escalate only low-quality outputs (Moslem et al., 25 Jun 2026).

3. Routing signals, thresholding, and optimization

The routing problem is typically framed as thresholded selective prediction under cost. SAFE-Cascade makes this explicit. Its router predicts an escalation probability $p_{\mathrm{esc}} \in [0,1]$ from features $f(q,T,a_{\text{text}})$ and escalates iff $p_{\mathrm{esc}} \ge \theta$. The expected cost under threshold $\theta$ is

$$E[C(\theta)] = c_{OCR} + c_{text} + \Pr(\mathrm{escalate}\mid \theta)\cdot c_{VLM},$$

and the paper defines a cost-aware utility

$C(x)$0

This formulation exposes the accuracy–cost frontier directly through the threshold slider in the demo (Dwivedi et al., 17 Jun 2026).

The telecom cascade formalizes the same problem as constrained risk minimization. With misalignment loss $C(x)$1 and processing cost $C(x)$2, the objective is

$C(x)$3

Thresholds are selected by multiple hypothesis testing on a grid over $C(x)$4, exploiting monotonicity in $C(x)$5 to control the family-wise error rate (Hou et al., 23 Dec 2025).

Early-abstention cascades refine the action space further. Instead of only accept versus escalate, the small model may choose among accept $C(x)$6, escalate $C(x)$7, and early abstain $C(x)$8. The per-query expected losses are

$C(x)$9

$U(x) < \epsilon$0

$U(x) < \epsilon$1

or $U(x) < \epsilon$2 under pre-gating. The routing policy selects the minimum expected loss, which makes abstention economically meaningful rather than a mere fallback (Zellinger et al., 13 Feb 2025).

Label-free optimization variants replace task labels with a reference model. C3PO minimizes regret with respect to the most powerful model (MPM) using only unlabeled model outputs, while imposing a probabilistic budget constraint

$U(x) < \epsilon$3

Thresholds are optimized against MPM disagreement on a self-supervision pool, then filtered by conformal cost calibration on a held-out calibration set (Valkanas et al., 10 Nov 2025). BARGAIN adopts a different statistical route: it uses adaptive sampling and anytime-valid betting bounds to certify accuracy, precision, or recall targets while minimizing the use of an expensive oracle model (Zeighami et al., 2 Sep 2025).

A recurrent theme is that routing quality depends as much on calibration as on raw model accuracy. Learning to Cascade shows that “naive application of existing calibration methods to cascade inference systems sometimes performs worse,” because the relevant confidence should reflect whether routing to the expensive model improves the system-level accuracy–cost trade-off, not merely whether the cheap model is calibrated in isolation (Enomoto et al., 2021). CAT makes a related point at training time: the small model can be trained with awareness of its downstream role in a cascade, which reshapes the quality–cost frontier at inference (Wang et al., 2024).

4. Cost models and empirical behavior

The empirical literature consistently reports that cost-aware cascades can match or nearly match the strongest endpoint while reducing expensive calls, but the frontier is task- and controller-dependent.

SAFE-Cascade reports its clearest operating point at $U(x) < \epsilon$4 on a held-out ChartQA test split of 375 examples. At that threshold it achieves 69.1% unified accuracy with 73.1% VLM invocation, compared with 67.7% accuracy and 100% VLM invocation for the full-VLM baseline. Because the observed +1.4 percentage-point difference has overlapping 95% confidence intervals, the paper interprets SAFE-Cascade as matching full-VLM performance within statistical uncertainty while reducing VLM calls by 26.9% and estimated cost by 9.3% (Dwivedi et al., 17 Jun 2026).

Several later systems push the same pattern more aggressively. Inter-Cascade reports that, compared with standard LLM Cascade baselines, retrieval-based strategy teaching improves the weak model “by up to 33.06 absolute percentage points” and the overall system “by up to 5.53 absolute percentage points,” while reducing strong-model calls “by up to 48.05% relative reduction” and saving fees “by up to 49.63% relative reduction” (Wu et al., 26 Sep 2025). Cluster, Route, Escalate reports that its cascaded system retains “97-99% of the strongest model's accuracy while reducing Time Per Output Token (TPOT),” and it requires only task-correctness labels while adapting to changes in the model pool (Moslem et al., 25 Jun 2026). CascadeDebate reports that selective intra-tier deliberation yields “up to 26.75 percent” improvement over strong baselines, and that its online threshold optimizer delivers “20.98 to 52.33 percent relative improvement over fixed policies” (Chang et al., 14 Apr 2026).

The empirical picture is not uniformly monotone in budget. Cost-aware RAG provides perhaps the strongest cautionary result: “no fixed selector uniformly dominates, and larger budgets do not reliably improve answer quality, even when costly evidence is domain-matched” (Wu et al., 1 Jun 2026). The same paper reports that agentic controllers can be highly model-dependent: Qwen3-8B often stops early with low average evidence cost, whereas Llama3.1-8B-Instruct spends much more with limited gains (Wu et al., 1 Jun 2026). This directly contradicts the common assumption that more evidence or a more expensive retrieval tier automatically improves extraction quality.

5. Reliability, guarantees, and transparency

A major line of work treats the cascade as a reliability mechanism rather than only a cost-reduction heuristic. The telecom framework gives finite-sample guarantees by testing the null hypotheses $U(x) < \epsilon$5 on a threshold grid. The resulting proposition states that, with probability at least $U(x) < \epsilon$6 over the calibration sample, all thresholds in the accepted set satisfy the misalignment bound, and the selected threshold $U(x) < \epsilon$7 inherits the guarantee

$U(x) < \epsilon$8

(Hou et al., 23 Dec 2025). C3PO offers an analogous guarantee on cost rather than misalignment: under exchangeability, conformal calibration ensures that the probability of exceeding a user-specified budget remains bounded by $U(x) < \epsilon$9 (Valkanas et al., 10 Nov 2025). BARGAIN extends the guarantee family to data-processing metrics, reporting strong guarantees on accuracy, precision, or recall and showing across 8 real-world datasets that it reduces cost, on average, by up to 86% more than state-of-the-art (Zeighami et al., 2 Sep 2025).

Transparency is another recurring requirement. SAFE-Cascade explicitly surfaces “OCR evidence, text-only answer, routing probability, escalation decision, final answer, estimated cost, and estimated latency” side by side, and allows the threshold to be changed interactively so that the accuracy–cost frontier becomes observable rather than implicit (Dwivedi et al., 17 Jun 2026). This makes the routing decision auditable at the query level.

Human escalation is the reliability endpoint in several systems. In the telecom cascade, if the cloud-level knowledge or confidence test fails, the query is forwarded to a human expert (Hou et al., 23 Dec 2025). In the human-AI decision framework, abstention thresholds on calibrated predictive uncertainty determine when the cascade response is “sufficiently certain or requires human intervention,” and online learning updates the thresholds using human feedback (Fanconi et al., 13 Jun 2025). A plausible implication is that cost-aware extraction cascades are increasingly being designed not merely to avoid unnecessary API spend, but to allocate epistemic responsibility across automated and human tiers.

6. Variants beyond simple inference-time routing

The modern cascade literature includes several extensions that alter either what is routed or when the cascade learns.

One branch augments inference with memory. Inter-Cascade stores pairs $C(x) > \lambda$0 in a strategy repository indexed by all-MiniLM-L6-v2 embeddings and FAISS, then retrieves the top-$C(x) > \lambda$1 strategies for future similar queries. In this design, the strong model is not only a fallback solver but also a mechanism for online in-context knowledge transfer (Wu et al., 26 Sep 2025).

A second branch modifies test-time compute allocation inside a tier. CascadeDebate alternates single-model inference with multi-agent deliberation, using agreement-based confidence and uncertainty to determine whether a local debate should resolve an ambiguous case before escalation. This reframes escalation as a boundary between single-agent and collective reasoning, not only between model sizes (Chang et al., 14 Apr 2026).

A third branch shifts optimization to training time. CAT trains the small LLM so that it is “cascade aware,” using the downstream large model during training to improve the overall quality–cost trade-off of the cascade (Wang et al., 2024). Learning to Cascade likewise optimizes confidence so that it better serves the system-level decision rule, rather than merely matching standalone correctness frequencies (Enomoto et al., 2021). IDK Cascades presents a training-free counterpart: a general framework for composing pre-trained models using “I Don’t Know” gates so that cheap models resolve easy instances and defer only hard ones (Wang et al., 2017).

Other extensions widen the action space or the resource notion itself. Cost-aware cascading bandits study ordered lists of arms with random costs and show that, in the offline setting, the Unit Cost Ranking with Threshold 1 policy is optimal, while the online CC-UCB algorithm achieves logarithmic regret (Zhou et al., 2018). Cost-aware evidence selection in RAG turns “knowledge” into an explicitly priced resource and treats evidence access itself as the cascade (Wu et al., 1 Jun 2026). RAGFort adds a “constrained cascade generation” module with a rejection budget $C(x) > \lambda$2 to control fallback cost while suppressing sensitive intra-topic extraction in proprietary knowledge-base defense (Li et al., 13 Nov 2025). These variants suggest that the cascade abstraction now spans model invocation, evidence acquisition, verification depth, and even defensive generation control.

7. Failure modes and open problems

The dominant failure mode in many deployed cascades is missed escalation. SAFE-Cascade identifies missed escalation as the most important failure mode: the router accepts a wrong text-only answer even though the VLM would have been correct, and this failure grows as the escalation threshold increases (Dwivedi et al., 17 Jun 2026). The same paper reports a substantial “hard” bucket—28.3% of test queries—where both pathways may fail (Dwivedi et al., 17 Jun 2026). In the telecom setting, the guarantees assume an i.i.d. calibration set from the deployment distribution; the paper explicitly notes distribution shift as a limitation and recommends periodic recalibration (Hou et al., 23 Dec 2025). Cost-aware RAG identifies an orthogonal failure mode: more expensive evidence can displace cheaper but complementary evidence in a fixed context window, so larger budgets may worsen answer quality (Wu et al., 1 Jun 2026).

Calibration and controller behavior remain fragile under domain difficulty. The human-AI decision framework reports strong gains on ARC datasets but mixed gains on MedQA, attributing difficulties to verification on harder medical questions (Fanconi et al., 13 Jun 2025). CascadeDebate notes latency from sequential deliberation and the possibility of “echo chambers” when role-prompt diversity is insufficient (Chang et al., 14 Apr 2026). Cluster, Route, Escalate depends on embedding quality for clustering and on QE calibration for escalation; its offline routing tables must be recomputed when model pools or traffic distributions change (Moslem et al., 25 Jun 2026).

Several open directions recur across papers. SAFE-Cascade plans extension beyond ChartQA to DocVQA, InfographicVQA, and enterprise documents (Dwivedi et al., 17 Jun 2026). Cost-aware RAG identifies query decomposition and learned value-of-information estimators as open problems (Wu et al., 1 Jun 2026). C3PO highlights the dependence of conformal guarantees on exchangeability and suggests richer dynamic routing under cost control (Valkanas et al., 10 Nov 2025). BARGAIN’s results indicate that adaptive sampling and tight finite-sample bounds remain central when guarantees must hold at scale (Zeighami et al., 2 Sep 2025). Taken together, these strands imply that the next phase of research is likely to center on distribution shift, richer uncertainty signals, structured validators, and hybrid systems in which routing, evidence acquisition, and human escalation are optimized jointly rather than as separate modules.