LLM4Predict: Hybrid Predictive Systems

• LLM4Predict is a design pattern that integrates large language models with downstream decision pipelines to generate semantic cues, uncertainty estimates, and latent features for enhanced predictions.
• It employs architectural patterns such as meta-modeling, semantic filtering, latent feature generation, and sub-goal prediction to reduce errors and control risk.
• Empirical evaluations across moderation, clinical, financial, and planning domains show that hybrid systems using LLM4Predict improve key performance metrics and decision accuracy.

Searching arXiv for the cited LLM4Predict-related papers to ground the article. LLM4Predict denotes a family of research patterns in which LLMs are used to improve prediction systems by contributing uncertainty estimates, semantic filtering, latent-feature inference, intermediate-state prediction, or precedent- and factor-based reasoning, rather than serving only as direct end-to-end predictors. Across the literature, the term appears in distinct but technically related senses: as a mechanism for predicting when an LLM is likely to fail in human-in-the-loop moderation (Bachar et al., 11 Jan 2026), as a semantic risk-management layer over statistical discovery in prediction markets (Kim et al., 4 Feb 2026), as a feature generator for latent clinical confounders (Roy, 20 Nov 2025), and as a domain-specific intermediate-state predictor inside classical planning systems (Yu et al., 15 Aug 2025). Taken together, these works suggest a broader paradigm in which LLMs contribute predictive structure, priors, or meta-predictions that are then embedded in downstream decision pipelines rather than treated as standalone oracles.

1. Conceptual scope and definitions

In one line of work, LLM4Predict is instantiated as a system whose core function is to predict LLM performance or reliability and quantify uncertainty so that human escalation can be triggered when needed (Bachar et al., 11 Jan 2026). In another, it denotes using LLMs not as direct numerical forecasters, but as semantic risk managers that sit on top of traditional time-series methods to strengthen predictions and control risk (Kim et al., 4 Feb 2026). A further interpretation uses LLMs as auxiliary predictors that infer probabilities of confounding diseases from routine clinical data, then injects those inferred probabilities back into a structured machine-learning model (Roy, 20 Nov 2025). In classical planning, LLM4Predict is a paradigm in which a LLM is embedded as a sub-goal generator inside a classical planner and predicts intermediate states that decompose difficult planning problems (Yu et al., 15 Aug 2025).

These formulations differ in surface task, but they share a common structure. The LLM does not merely emit a final label; it contributes a predictive object that another component can use. Depending on the domain, that object may be a calibrated probability that the base LLM is correct (Bachar et al., 11 Jan 2026), a plausibility score for an economic transmission mechanism (Kim et al., 4 Feb 2026), a vector of inferred latent clinical states (Roy, 20 Nov 2025), or a domain-consistent intermediate state in a symbolic planning problem (Yu et al., 15 Aug 2025). This suggests that LLM4Predict is best understood as a design pattern for hybrid predictive systems.

A recurring theme is that off-the-shelf LLMs are often insufficient as direct predictors. In public-opinion tracking, off-the-shelf LLMs do not reliably track polls when queried in a straightforward manner, with systematic directional miscalibration for favorability estimates (Parikh et al., 6 Feb 2026). In clinical risk prediction for delirium, GPT-4 exhibited significant deficiencies in identifying positive cases and struggled to provide reliable probability estimates, while a specialist medical AI system performed substantially better (Rezk et al., 2024). These results imply that the strongest use of LLM4Predict is often indirect: the LLM is asked to generate features, priors, filters, or explanations that are coupled to more specialized predictive machinery.

2. Architectural patterns

Several architectural patterns recur across the literature. One is the “meta-model” pattern. In “LLM Performance Predictors: Learning When to Escalate in Hybrid Human-AI Moderation Systems” (Bachar et al., 11 Jan 2026), the base LLM produces structured outputs and uncertainty-related signals, and a supervised meta-model predicts whether the LLM’s moderation decision is correct. The resulting score is interpreted as an estimated probability that the LLM is correct and is used for cost-aware escalation.

A second pattern is the “semantic filter” or “risk manager” pattern. In prediction markets, the hybrid two-stage screener first applies Granger causality to identify candidate leader–follower pairs from market-implied probability time series, then uses an LLM-based semantic stage to re-rank those candidates by assessing whether the proposed direction admits a plausible economic transmission mechanism based on event descriptions (Kim et al., 4 Feb 2026). The LLM thereby filters out statistically fragile links that are prone to large losses.

A third pattern is “latent feature generation.” In breast-cancer prescreening, the original feature vector is augmented by LLM-derived probabilities for Type-2 diabetes, obesity, cardiovascular disease, and breast cancer itself, producing an augmented feature vector in $\mathbb{R}^{13}$ (Roy, 20 Nov 2025). The LLM-derived confounder features are not the final decision; they are appended to the original nine clinical variables and used by a Random Forest classifier.

A fourth pattern is “sub-goal prediction” for search-space control. In “Inspire or Predict? Exploring New Paradigms in Assisting Classical Planners with LLMs” (Yu et al., 15 Aug 2025), LLM4Predict predicts an intermediate state $\tilde{s}$ between the current state and goal state, and the predicted intermediate state is constrained by domain-specific knowledge and used to split the problem into two solvable subproblems. This uses the LLM to alter search structure rather than directly output a plan.

A fifth pattern is “causal prior construction.” In legal judgment prediction, LLMs are used to refine legal factor extraction and to disambiguate ambiguous causal directions in partially identified causal graphs, after which a causal-aware predictor constrains attention using graph-derived causal strengths (Liang et al., 12 Mar 2026). This suggests a broader LLM4Predict interpretation in which the LLM contributes structurally meaningful priors that downstream predictors can exploit.

3. Uncertainty prediction and supervised escalation

The most explicit formalization of LLM4Predict as prediction of model reliability appears in moderation. The base LLM is prompted to produce a structured JSON with a single integer label $y \in \{0,1,2,3\}$, where 0 denotes “no,” 1 denotes “yes,” 2 denotes “inconclusive_evidence,” and 3 denotes “inconclusive_definition” (Bachar et al., 11 Jan 2026). The model is run with deterministic decoding and returns token-level log-probabilities for the outcome token and, optionally, reasoning tokens.

From top-$k$ log-probabilities $\{\ell_1,\dots,\ell_k\}$, the framework computes the renormalized distribution

$$\tilde{p}_i = \frac{\exp(\ell_i)}{\sum_{j=1}^k \exp(\ell_j)}, \quad i=1,\dots,k.$$

It then derives entropy, normalized entropy, effective number of choices, confidence score, max softmax probability, top-2 probability margin, normalized margin, top-1/top-2 ratio, filtered versions over the valid schema labels $\mathcal{A}=\{0,1,2,3\}$, and log-odds margins (Bachar et al., 11 Jan 2026). The structured output also includes a scalar self-reported confidence $\hat{c} \in [0,100]$, a coarse confidence band, and two uncertainty-attribution indicators: $$u_{\text{evidence}}(x) = \mathbb{I}\{\text{outcome}(x)=2\}, \qquad u_{\text{policy}}(x) = \mathbb{I}\{\text{outcome}(x)=3\}.$$

These signals are assembled into an LLM Performance Predictor feature vector $f(x) \in \mathbb{R}^d$ (Bachar et al., 11 Jan 2026). Rather than thresholding entropy or max-probability directly, the paper trains a Ridge Regression meta-model to predict whether the base LLM is correct: $\tilde{s}$0 with objective

$\tilde{s}$1

Calibration is then applied so that the score $\tilde{s}$2 can be interpreted as an estimated probability that the LLM is correct (Bachar et al., 11 Jan 2026).

This prediction-of-correctness score drives selective classification with rejection. Given a threshold $\tilde{s}$3, the system trusts the LLM if $\tilde{s}$4 and escalates to a human otherwise. The paper defines an expected cost

$\tilde{s}$5

and an escalation ratio

$\tilde{s}$6

On the OpenAI Moderation dataset, the meta-model sharply improves cost-aware routing. For gpt-4o-mini, always-trust cost is \$\tilde{s}$$7132 with 331 escalations (37$$38 with 148 escalations (16%) (Bachar et al., 11 Jan 2026). This makes LLM4Predict here a supervised uncertainty-quantification framework for deciding when a deployed LLM should be overridden by human review.

4. Semantic filtering, mechanism reasoning, and predictive risk control

In prediction markets, LLM4Predict takes a different form. Daily YES prices $\tilde{s}$8 are transformed into log-odds

$\tilde{s}$9

and lead–lag structure is defined by whether past values of one series improve forecasts of another (Kim et al., 4 Feb 2026). Candidate directed pairs are found with Granger causality regressions of the form

$y \in \{0,1,2,3\}$0

testing $y \in \{0,1,2,3\}$1 (Kim et al., 4 Feb 2026).

The LLM is not used to estimate time-series parameters. Instead, it receives the leader event’s title and description, the follower event’s title and description, and the direction $y \in \{0,1,2,3\}$2, then judges whether there is a plausible economic transmission mechanism and what the likely sign of co-movement is (Kim et al., 4 Feb 2026). A sign variable is defined as

$y \in \{0,1,2,3\}$3

The LLM’s semantic score is used to re-rank statistically screened candidate pairs.

Trading evaluation is based on a fixed signal-triggered protocol. The one-day relative change in the leader is

$y \in \{0,1,2,3\}$4

and the follower position direction is

$y \in \{0,1,2,3\}$5

Cumulative profit and loss is

$y \in \{0,1,2,3\}$6

Across rolling evaluations on Kalshi Economics markets, the hybrid approach improves win rate from 51.4% to 54.5%, reduces average magnitude of losing trades from 649 USD to 347 USD, and increases total PnL from \$y \in \{0,1,2,3\}$712,500 (Kim et al., 4 Feb 2026).

This form of LLM4Predict is not “LLM predicts prices directly.” It is an LLM-based mechanism evaluator layered on top of statistical discovery. The main effect is risk reduction rather than rare outsized gains. This suggests that, in domains with many candidate predictive relationships and substantial multiple-testing risk, an LLM may be most effective when tasked with filtering, ranking, or rejecting candidate signals according to semantic plausibility.

5. Feature generation, causal priors, and domain-specific predictors

In clinical prediction, LLM4Predict often appears as feature augmentation. In breast-cancer prediction, each patient has an original feature vector $y \in \{0,1,2,3\}$8 and an LLM-derived vector

$y \in \{0,1,2,3\}$9

which can be concatenated to form

$k$0

A Random Forest baseline uses $k$1, whereas the augmented model uses $k$2 (Roy, 20 Nov 2025).

The study uses the Breast Cancer Coimbra dataset with 116 patients and 20 random 80/20 train-test splits (Roy, 20 Nov 2025). In the “All-confounders” configuration, Gemma-2-27B improves accuracy from 0.719 to 0.746, precision from 0.718 to 0.742, recall from 0.619 to 0.667, AUC from 0.710 to 0.740, and specificity from 0.801 to 0.813 (Roy, 20 Nov 2025). Llama-3.3-70B improves accuracy from 0.704 to 0.750, precision from 0.692 to 0.749, recall from 0.674 to 0.683, AUC from 0.711 to 0.753, and specificity from 0.749 to 0.822 (Roy, 20 Nov 2025). The paper reports average improvement over baseline across metrics of 3.9% for Gemma and 6.4% for Llama in the All-confounders configuration (Roy, 20 Nov 2025).

A related, but more structurally ambitious, use of LLMs appears in legal judgment prediction with causal modeling. The LLM-Knowledge-GCI framework first refines legal factor extraction with a coarse-to-fine hybrid mechanism combining YAKE, uniform sampling, retrieval from a legal lexicon, and LLM semantic filtering (Liang et al., 12 Mar 2026). It then uses GFCI to produce a partial ancestral graph and asks the LLM to disambiguate ambiguous edge directions under legal and temporal constraints. Candidate graphs $k$3 are weighted by

$k$4

and factor-to-label causal strengths are aggregated via

$k$5

with each $k$6 estimated through Average Treatment Effect calculations (Liang et al., 12 Mar 2026). These strengths constrain attention through

$k$7

This suggests that LLM4Predict can also denote an architecture in which LLMs help build causal abstractions or priors that shape downstream discriminative prediction.

6. Sub-goal prediction, planning, and predictive decomposition

A more literal use of the name LLM4Predict appears in classical planning. A planning problem is defined as $k$8, with actions $k$9 and transition

$\{\ell_1,\dots,\ell_k\}$0

(Yu et al., 15 Aug 2025). Large-scale planning suffers from state-space explosion of complexity $\{\ell_1,\dots,\ell_k\}$1, where $\{\ell_1,\dots,\ell_k\}$2 is branching factor and $\{\ell_1,\dots,\ell_k\}$3 plan length.

The decomposition-based planner first constructs Directed Acyclic Dependency Graphs over goal predicates and produces an ordered sub-goal sequence. If a sub-instance $\{\ell_1,\dots,\ell_k\}$4 cannot be solved directly by the classical solver, LLM4Predict is invoked to predict an intermediate state $\{\ell_1,\dots,\ell_k\}$5 between current state $\{\ell_1,\dots,\ell_k\}$6 and sub-goal $\{\ell_1,\dots,\ell_k\}$7 (Yu et al., 15 Aug 2025). The planner then constructs

$\{\ell_1,\dots,\ell_k\}$8

and, after reaching $\{\ell_1,\dots,\ell_k\}$9, resumes solving toward $$\tilde{p}_i = \frac{\exp(\ell_i)}{\sum_{j=1}^k \exp(\ell_j)}, \quad i=1,\dots,k.$$0.

The key complexity argument is that if the original problem has plan length $$\tilde{p}_i = \frac{\exp(\ell_i)}{\sum_{j=1}^k \exp(\ell_j)}, \quad i=1,\dots,k.$$1 but the predicted intermediate state yields two subproblems of lengths $$\tilde{p}_i = \frac{\exp(\ell_i)}{\sum_{j=1}^k \exp(\ell_j)}, \quad i=1,\dots,k.$$2 and $$\tilde{p}_i = \frac{\exp(\ell_i)}{\sum_{j=1}^k \exp(\ell_j)}, \quad i=1,\dots,k.$$3, then direct search costs roughly $$\tilde{p}_i = \frac{\exp(\ell_i)}{\sum_{j=1}^k \exp(\ell_j)}, \quad i=1,\dots,k.$$4 whereas decomposed search costs

$$\tilde{p}_i = \frac{\exp(\ell_i)}{\sum_{j=1}^k \exp(\ell_j)}, \quad i=1,\dots,k.$$5

which can be much smaller (Yu et al., 15 Aug 2025). The LLM prompt is domain-specific and constrained to output one or two key predicates representing a reasonable intermediate state, not the full valuation (Yu et al., 15 Aug 2025).

Empirically, LLM4Predict outperforms both classical Fast Downward and LLM4Inspire in several IPC domains. Success rates are 49/50 in Blocks, 42/42 in Logistics, 19/22 in Depot, and 15/30 in Mystery, compared with 37/50, 42/42, 17/22, and 15/30 for LLM4Inspire, and 26/50, 17/42, 5/22, and 15/30 for Fast Downward alone (Yu et al., 15 Aug 2025). This use of LLM4Predict is conceptually important because the LLM is predicting a latent waypoint that reorganizes the search problem. That differs from uncertainty prediction or feature generation, but fits the same broader theme: the LLM predicts a structured object that enhances another predictor or solver.

7. Direct prediction, limitations, and evaluation concerns

Not all domains favor direct LLM prediction. In public-opinion prediction, nine LLM configurations queried daily on exit-poll-style favorability questions systematically overpredict Kamala Harris’s favorability by 10–40% relative to polls, while biases for Donald Trump are smaller, around 5–10% and poll-dependent (Parikh et al., 6 Feb 2026). These deviations persist under temporal smoothing and are not corrected by internet-augmented retrieval (Parikh et al., 6 Feb 2026). The study concludes that off-the-shelf LLMs do not reliably track polls when queried in a straightforward manner and therefore are not suitable as direct poll replacements (Parikh et al., 6 Feb 2026).

A similar caution arises in clinical risk prediction. In delirium prediction from structured and unstructured EHR data, clinalytix Medical AI achieves precision 94.57%, recall 94.57%, F1-score 94.57%, and specificity 94.90%, whereas GPT-4 achieves precision 98.28%, recall 61.96%, F1-score 76.00%, and specificity 98.98% (Rezk et al., 2024). GPT-4 is therefore highly conservative but misses a large fraction of true positive cases, and its probability estimates are not calibrated (Rezk et al., 2024). This suggests that direct LLM outputs are often poorly suited to high-stakes predictive decision-making unless they are wrapped in specialist models or calibration layers.

The finance literature also shows that the role of the LLM matters. In intraday stock prediction using minute-level Apple price data and sentiment extracted from ten LLMs, the best overall configuration is Mamba with LLaMA 3.3–70B sentiment, achieving MSE 0.1367 on the three-day test set (Kadiyala et al., 14 Sep 2025). Reformer performs worse for every LLM, with its best configuration at MSE 2.6468 (Kadiyala et al., 14 Sep 2025). Here the LLM is neither the forecaster nor the router; it is a sentiment sensor whose outputs feed an efficient temporal model. In another financial setting, earnings-report prediction uses instruction fine-tuning with QLoRA over long textual and numeric inputs, and llama-3-8b-Instruct-4bit achieves accuracy 0.573, weighted F1 0.565, and MCC 0.154 on the full dataset, outperforming GPT-4.0 at 0.494 accuracy, 0.512 weighted F1, and 0.031 MCC (Ni et al., 2024). This suggests that direct LLM prediction may work better when the LLM is specifically adapted to the task and supplied with carefully structured financial context.

The literature also emphasizes evaluation methodology. “Statistical multi-metric evaluation and visualization of LLM system predictive performance” (Ackerman et al., 30 Jan 2025) frames predictive assessment as a multi-dimensional problem over systems, datasets, and metrics, and introduces a framework that automatically performs the correct statistical tests, properly aggregates the statistical results across metrics and datasets, and visualizes the results (Ackerman et al., 30 Jan 2025). This suggests that LLM4Predict systems should not be assessed only with point metrics on a single benchmark. When the LLM acts as a predictor, calibrator, router, or feature generator, comparisons should be made with task-appropriate statistical tests and aggregated across multiple relevant metrics.

Across these findings, a common misconception is that LLM4Predict means “use an LLM as a direct predictor of the target variable.” The evidence does not support that as a general rule. A more precise characterization is that LLM4Predict comprises a set of hybrid predictive strategies in which LLMs are used where their semantic priors, uncertainty signals, factor extraction, or structure-induction capabilities are strongest, while specialized models, retrieval systems, or symbolic solvers handle the parts of the predictive problem requiring calibration, efficient optimization, or strict validity constraints.