LLM-Filter: Frozen LLM for State Estimation

• LLM-Filter is a filtering framework that repurposes a frozen large language model to map noisy numerical observations into a latent space for state estimation.
• Its architecture uses prompt-guided, multimodal alignment to embed continuous signals into discrete tokens, enabling sequential next-token style inference.
• Empirical benchmarks show that LLM-Filter improves RMSE and runtime efficiency while System-as-Prompt enhances generalization under distribution shifts.

Searching arXiv for the primary paper and closely related uses of the term. LLM-Filter is a generalist filtering framework for optimal state estimation in dynamical systems that reuses a frozen LLM as the core inference engine. In its primary usage, the term denotes a prompt-guided, multimodal architecture that maps noisy numerical observations into the latent space of a pre-trained LLM, performs in-context inference with a fixed backbone, and projects the resulting representations back into hidden-state estimates (Liu et al., 24 Sep 2025). The framework is positioned against two limitations identified in prior filtering practice: classical Bayes filters are accurate only when the system model and noise statistics are well specified, whereas learning-based filters generalize poorly across systems. A secondary complication is terminological: the phrase “LLM-Filter” has also been used in other areas for moderation filters, semantic database filtering, recommender-system simulations of filter bubbles, and LLM-assisted discovery of filtering algorithms. In the state-estimation literature, however, LLM-Filter most specifically refers to the frozen-LLM filtering architecture introduced in “One Filters All: A Generalist Filter for State Estimation” (Liu et al., 24 Sep 2025).

1. Formal problem setting

LLM-Filter is formulated for the standard Markovian state-space model

$$\boldsymbol{x}_{t+1}=f\left(\boldsymbol{x}_{t}\right)+\boldsymbol{\xi}_{t}, \qquad \boldsymbol{y}_{t}=h\left(\boldsymbol{x}_{t}\right)+\boldsymbol{\zeta}_{t},$$

where $\boldsymbol{x}_t\in \mathbb{R}^M$ is the hidden state, $\boldsymbol{y}_t\in \mathbb{R}^N$ is the observation, $f(\cdot)$ is the transition model, $h(\cdot)$ is the observation model, and $\boldsymbol{\xi}_t,\boldsymbol{\zeta}_t$ are transition and observation noise. The equivalent probabilistic form is

$$\boldsymbol{x}_{0} \sim p(\boldsymbol{x}_0), \quad \boldsymbol{x}_{t} \sim p(\boldsymbol{x}_t|\boldsymbol{x}_{t-1}), \quad \boldsymbol{y}_{t} \sim p(\boldsymbol{y}_t|\boldsymbol{x}_{t}).$$

The filtering objective is recursive inference of $p(\boldsymbol{x}_t|\boldsymbol{y}_{1:t})$. The Bayesian recursion is written as

$$p(\boldsymbol{x}_t|\boldsymbol{y}_{1:t-1}) = \int p(\boldsymbol{x}_t|\boldsymbol{x}_{t-1}) p(\boldsymbol{x}_{t-1}|\boldsymbol{y}_{1:t-1}) \mathrm{d} \boldsymbol{x}_{t-1},$$

$$p(\boldsymbol{x}_t|\boldsymbol{y}_{1:t}) = \frac{p(\boldsymbol{y}_t|\boldsymbol{x}_t) p(\boldsymbol{x}_t|\boldsymbol{y}_{1:t-1})} {\int p(\boldsymbol{y}_t|\boldsymbol{x}_t) p(\boldsymbol{x}_t|\boldsymbol{y}_{1:t-1}) \mathrm{d} \boldsymbol{x}_t }.$$

Within this formulation, LLM-Filter treats filtering as a sequential prediction problem analogous to next-token prediction. This suggests a reinterpretation of state estimation as a modality-aligned autoregressive inference task rather than as explicit Bayes recursion alone. The framework thereby differs from classical Kalman-style or particle-based filtering, even though it is evaluated on canonical state-estimation benchmarks and compared directly with PF, EnKF, EnKFI, EnKFS, HubEnKF, MEstimator, RStateNet, ProTran, and KalmanNet (Liu et al., 24 Sep 2025).

2. Architectural design and modality alignment

The architecture uses a moving-horizon representation because the model cannot process an unbounded history. For window length $\boldsymbol{x}_t\in \mathbb{R}^M$0, the input observation window is

$\boldsymbol{x}_t\in \mathbb{R}^M$1

the corresponding state sequence is

$\boldsymbol{x}_t\in \mathbb{R}^M$2

and the prediction is

$\boldsymbol{x}_t\in \mathbb{R}^M$3

To align continuous signals with discrete LLM representations, the observation window is segmented into chunks of length $\boldsymbol{x}_t\in \mathbb{R}^M$4: $\boldsymbol{x}_t\in \mathbb{R}^M$5 with each $\boldsymbol{x}_t\in \mathbb{R}^M$6. Each segment is normalized and embedded by a trainable observation embedding module,

$\boldsymbol{x}_t\in \mathbb{R}^M$7

This embedding step is explicitly motivated by preservation of multivariate structure. The framework argues that simply flattening all variables into one series can destroy correlations such as position-velocity coupling. The embedding and projection modules are implemented as simple MLPs, while the LLM backbone remains frozen.

The LLM input is the concatenation of tokenized textual context and numeric observation embeddings: $\boldsymbol{x}_t\in \mathbb{R}^M$8 where

$\boldsymbol{x}_t\in \mathbb{R}^M$9

The frozen LLM core layers then produce

$\boldsymbol{y}_t\in \mathbb{R}^N$0

from which the observation-aligned output tokens are extracted,

$\boldsymbol{y}_t\in \mathbb{R}^N$1

and mapped back to state space by

$\boldsymbol{y}_t\in \mathbb{R}^N$2

This modality-alignment design is central to the method. The paper states that it “freeze[s] the parameters and eliminate[s] the embedding and projection layers designed for language tokens,” using only the LLM core as a general inference engine. A plausible implication is that the framework relies on the representational priors of large pretrained models while minimizing task-specific parameter updates (Liu et al., 24 Sep 2025).

3. System-as-Prompt and supervised training objective

A defining component of LLM-Filter is System-as-Prompt (SaP). SaP is the textual context $\boldsymbol{y}_t\in \mathbb{R}^N$3 prepended to the numeric observation embeddings and is designed to help the LLM understand both the system and the estimation task. The prompt contains two ingredients: “Task Instruction” and “Task Examples.” Task Instruction provides contextual knowledge about the domain and task, while Task Examples illustrate example scenarios clarifying what should be estimated. The prompt is system-specific and is used in an in-context learning style rather than through LLM finetuning (Liu et al., 24 Sep 2025).

The supervised training objective extends the usual regression loss for learning-based filters. The paper first writes the standard form as

$\boldsymbol{y}_t\in \mathbb{R}^N$4

and then introduces prompt context: $\boldsymbol{y}_t\in \mathbb{R}^N$5 The intended meaning is an MSE objective between predicted and true state windows conditioned on both observation history and prompt context.

Training is parameter-efficient. The default backbone is the core layers of LLaMA-7B. The total parameter count is 6.61B, but only 4.22M parameters are trainable. The trainable components are only $\boldsymbol{y}_t\in \mathbb{R}^N$6 and $\boldsymbol{y}_t\in \mathbb{R}^N$7, both MLPs. The reported setup uses AdamW with learning rate $\boldsymbol{y}_t\in \mathbb{R}^N$8, weight decay $\boldsymbol{y}_t\in \mathbb{R}^N$9, batch size 16, window length $f(\cdot)$0, segment length $f(\cdot)$1, hidden dimension 512, two MLP layers, and 10 epochs. Datasets contain 100 trajectories per system, each 200 time steps long, split chronologically into train, validation, and test in a 7:1:2 ratio (Liu et al., 24 Sep 2025).

An ablation variant, LLM-Filter-O, removes SaP. It remains operational but exhibits degraded accuracy and transfer, indicating that prompt structure is not an auxiliary convenience but a material component of the inference mechanism.

4. Empirical performance on canonical filtering benchmarks

The evaluation covers Tracking, Selkov, Oscillator, Hopf, Double Pendulum, Lorenz96, and VL20. These benchmarks span low-dimensional nonlinear systems and high-dimensional chaotic systems. The primary metrics are RMSE and average runtime in ms/step. On the six canonical systems reported in the main quantitative table, LLM-Filter achieves the following RMSE values (Liu et al., 24 Sep 2025):

System	RMSE
Selkov	0.4061
Oscillator	0.5247
Hopf	0.5751
Pendulum	0.8348
Lorenz96	0.9149
VL20	0.7717

Across the low-dimensional nonlinear systems, the average RMSE is 0.6712 for LLM-Filter, compared with 0.7948 for LLM-Filter-O, 0.9086 for MEstimator, 0.8558 for RStateNet, and 0.9105 for ProTran. Among online Bayes filters, EnKF averages 2.6097 and PF 3.9588. The paper states a maximum RMSE reduction of 32.00% and average improvements of 17.93% over the best learning-based baselines and 21.65% over the top online Bayes filter. For high-dimensional chaotic systems, it reports an average improvement of 11.7% over the best baselines and notes that KalmanNet diverged in those settings.

Runtime is reported as near 1 ms even as system dimension increases. The supplementary table gives 0.9577 ms/step on Lorenz96 and 0.9503 ms/step on VL20 for LLM-Filter, versus 6.7377 and 2.9397 ms/step for EnKF, respectively, and around 0.45 ms/step for RStateNet. This places the framework between lightweight recurrent estimators and more expensive online ensemble methods in the speed-accuracy trade space.

The empirical profile supports two claims made explicitly in the paper: first, that state estimation can benefit from the reasoning knowledge embedded in pre-trained LLMs once modality alignment is achieved; second, that a single frozen-backbone architecture can remain competitive across qualitatively different dynamical systems (Liu et al., 24 Sep 2025).

5. Robustness, transfer, and scaling behavior

A major focus of the paper is generalization under distribution shift. Under observation-noise mismatch, the experiments vary the true observation covariance using an observation covariance expansion ratio (OCER). Across Selkov, Oscillator, and Hopf, robust filters such as EnKFI, EnKFS, and HubEnKF degrade rapidly as OCER increases, whereas LLM-Filter remains comparatively stable.

The framework is also tested in cross-system transfer. Reported RMSE values include 1.2568 for Tracking $f(\cdot)$2 Pendulum, 1.4214 for Pendulum $f(\cdot)$3 Tracking, 0.6613 for Oscillator $f(\cdot)$4 Hopf, and 0.9069 for Hopf $f(\cdot)$5 Oscillator. LLM-Filter achieves the lowest RMSE in these transfer settings, and LLM-Filter-O performs noticeably worse. This directly supports the claim that SaP contributes significantly to out-of-distribution generalization (Liu et al., 24 Sep 2025).

Scaling behavior is reported across four backbones, with SaP omitted for fairness across tokenizers:

Backbone	Average RMSE
GPT-2 (124M)	0.9819
OPT-1.3B	0.9182
LLaMA-7B	0.7694
LLaMA-13B	0.6882

The paper interprets this as a scaling-law behavior in which larger LLMs improve state-estimation accuracy, albeit with increased training time. Additional ablations compare the frozen-LLM architecture against replacements of the backbone with MLP, RNN, or Transformer while keeping the rest of the system fixed; the reported average improvements of LLM-Filter over those alternatives are 31.66% over MLP, 32.43% over RNN, and 30.73% over Transformer.

Parameter-adaptation ablations are also informative. LoRA generally hurts performance, which the authors attribute to limited expressive power for precise state estimation. Full finetuning can improve difficult systems such as Lorenz96 and VL20, but can worsen simpler systems, suggesting a trade-off between adaptation and overfitting. This suggests that the frozen-backbone design is not merely a computational convenience but part of the framework’s generalization strategy.

6. Limitations, conceptual significance, and terminological breadth

The paper states several limitations. The current generalization claims are restricted to systems with the same dimensionality, and the evaluation does not yet cover truly variable-dimensional transfer. Performance without SaP is weaker, indicating continued dependence on prompt engineering. Full finetuning may overfit simpler systems, while LoRA may underfit. The prompt itself is system-specific, so the framework is not entirely free of task-dependent design (Liu et al., 24 Sep 2025).

Conceptually, LLM-Filter occupies an unusual position in the filtering literature. Classical adaptive and Bayesian filters remain grounded in explicit probabilistic recursions, as in probabilistic LMS (Fernandez-Bes et al., 2015), clustered-information LMMSE for Markov jump linear systems (Costa et al., 2016), or LLMMSE variants for low-dose CT sinogram denoising (Wang et al., 2019). LLM-Filter instead delegates a large share of the inference burden to a frozen LLM after embedding nonlinguistic observations into its latent space. This suggests a broader shift from model-specific estimators toward general-purpose pretrained inference engines.

At the same time, the phrase “LLM-Filter” is not unique to state estimation. Related literature uses LLMs as semantic filters for relational querying via Clustering-Sampling-Voting (Hou et al., 5 Mar 2026), as external token-level safety filters based on control barrier functions (Miyaoka et al., 2024), as bilingual moderation filters predicting harmlessness and cultural alignment in Arabic and English (Fatehkia et al., 24 Nov 2025), and as synthetic-user simulators for recommender-system filter bubbles (Sukiennik et al., 23 Mar 2025). The term has also appeared in work on LLM-assisted discovery of Kalman-like estimators through Cartesian Genetic Programming and evolutionary search (Saketos et al., 13 Aug 2025). These usages are technically distinct. In encyclopedia usage, the most precise interpretation is therefore contextual: in dynamical systems, LLM-Filter denotes the generalist frozen-LLM state-estimation framework of (Liu et al., 24 Sep 2025); in adjacent literatures, it can refer more broadly to LLM-mediated filtering mechanisms.

Taken together, these developments indicate that “filter” has become a cross-domain interface between LLMs and structured decision problems: state estimation, moderation, semantic retrieval, recommendation analysis, and algorithm synthesis. A plausible implication is that the term will remain polysemous, but the state-estimation formulation introduced in (Liu et al., 24 Sep 2025) is the clearest candidate for its canonical technical meaning.