Denoising Particle Filters: Learning State Estimation with Single-Step Objectives

Abstract: Learning-based methods commonly treat state estimation in robotics as a sequence modeling problem. While this paradigm can be effective at maximizing end-to-end performance, models are often difficult to interpret and expensive to train, since training requires unrolling sequences of predictions in time. As an alternative to end-to-end trained state estimation, we propose a novel particle filtering algorithm in which models are trained from individual state transitions, fully exploiting the Markov property in robotic systems. In this framework, measurement models are learned implicitly by minimizing a denoising score matching objective. At inference, the learned denoiser is used alongside a (learned) dynamics model to approximately solve the Bayesian filtering equation at each time step, effectively guiding predicted states toward the data manifold informed by measurements. We evaluate the proposed method on challenging robotic state estimation tasks in simulation, demonstrating competitive performance compared to tuned end-to-end trained baselines. Importantly, our method offers the desirable composability of classical filtering algorithms, allowing prior information and external sensor models to be incorporated without retraining.

• The paper introduces Denoising Particle Filters, which train dynamics and measurement models with single-step objectives and compose them through diffusion-based posterior sampling instead of backpropagation through time.
• DnPF matches or outperforms end-to-end baselines across three simulated robotic tasks, achieving a normalized NLL of 2.8 versus 4.4 under severe image-observation dropout.
• The method supports arbitrary prior initialization and zero-shot sensor fusion, while warm-started denoising and likelihood constraints reduce recursive drift and enable real-time inference.

Denoising Particle Filters (DnPF), introduced in (2602.19651), address a structural weakness in learning-based state estimation: the reliance on end-to-end training over sequences. Instead of unrolling a filter or sequence model through time and optimizing via backpropagation through time (BPTT), the authors train dynamics and measurement models exclusively with single-step objectives and compose them at inference through a diffusion-based particle filtering scheme. The result is a filter that matches tuned end-to-end baselines on simulated robotic estimation tasks while retaining the modularity and interpretability of classical Bayesian filtering.

Motivation and positioning

The paper contrasts three paradigms for state estimation. Recurrent networks and Transformers map measurement histories to state posteriors directly; they train efficiently but offer no mechanism for initializing to an arbitrary prior $p(x_0)$ and no interpretable hidden state. Differentiable Particle Filters (DPFs) restore algorithmic structure but require BPTT across every particle, making training expensive and hard to scale, and any change in the sensor suite demands retraining. A further consequence of sequence-level training is overfitting to the specific sequential structure of the training data. The authors observe that training on individual transitions would exploit the Markov property, but recursive use of single-step models typically fails because intermediate predictions drift off the training data manifold (2602.19651). DnPF is designed precisely to close this gap: single-step training, with drift handled at inference time by a denoising correction.

Method

The core idea is to sample from the posterior $p(x_t \mid y_{1:t}, u_{1:t})$ by integrating the probability-flow ODE of a diffusion model, whose score decomposes via the logarithm of the recursive Bayes filtering equation into a measurement likelihood term and a dynamics term, with the integral over ancestors approximated by the particle set. Because samples are drawn directly from the posterior, no importance sampling or resampling step is required.

Measurement model via denoising score matching. Rather than learning $p(y_t \mid x_t)$ directly — impractical when $\mathcal{Y}$ is, e.g., an image space — the authors learn an observation-conditioned denoiser $D(x_t, y_t, s)$ minimizing the standard noise-prediction objective, and recover the score $\nabla \log p_s(x_t \mid y_t)$ via the standard relation between optimal denoisers and scores. Conditioning is sporadically dropped during training so the same network also predicts the prior score $\nabla \log p_s(x_t)$, in the manner of classifier-free guidance. At inference, the likelihood score is applied with strength $\eta$, which introduces a per-timestep bias of $p(y_t \mid x_t)^{\eta} p(x_t)$. Notably, the authors do not correct this bias and argue it is beneficial: it counteracts distributional drift during recursion.

Dynamics model. The transition $p(x_t \mid x_{t-1}, u_t)$ is parametrized as Gaussian with mean and covariance heads on a feedforward network. This choice yields a closed-form score for the noise-convolved transition at every denoising step, so the dynamics network is evaluated only once per particle and timestep rather than once per denoising step — a significant efficiency gain.

Warm-started inference. Instead of denoising from pure noise, particles are initialized at noise scale $p(x_t \mid y_{1:t}, u_{1:t})$0 around the deterministic dynamics prediction $p(x_t \mid y_{1:t}, u_{1:t})$1, and only $p(x_t \mid y_{1:t}, u_{1:t})$2 denoising steps from $p(x_t \mid y_{1:t}, u_{1:t})$3 to $p(x_t \mid y_{1:t}, u_{1:t})$4 are taken. In practice, $p(x_t \mid y_{1:t}, u_{1:t})$5 with 5–25 steps suffices on the studied tasks. Implementation details support real-time use: particles are batched on GPU, observation encodings and FiLM conditioning vectors are computed once per timestep and shared across particles, and DDIM schedules with Euler integration drive the ODE.

Likelihood-constrained diffusion. When the prior and likelihood supports are nearly disjoint — common with near-deterministic dynamics — even an exact solve of the filtering equation can fail. The authors use the magnitude of the denoiser output as a proxy for distance to the likelihood manifold (justified via the ELBO interpretation of denoising score matching) and enforce a threshold constraint with an augmented Lagrangian that scales down the dynamics term per dimension when the constraint is violated. This mechanism demonstrably rescues particles after contact events, e.g., when the end-effector first touches the object in the Manipulator Spin task.

Experiments

Evaluation covers three MuJoCo tasks with partial observability, nonlinear contact dynamics, and high-dimensional states: a 7-DoF manipulator tracking an object on a spinning table through proprioception only; a 12-DoF hand estimating an object's SE(3) pose from finger joints alone, with domain randomization in friction, gains, and systematic joint biases; and a cluttered pushing task with frequently occluded and 80%-of-the-time blacked-out image observations. Performance is measured by the interquartile mean of normalized negative log-likelihood of ground truth under the particle-based Gaussian mixture, with 100 particles for all methods.

DnPF matches or exceeds the best end-to-end baselines — Transformer, GRU with particle decoder, DPF, and D2P2F — on all tasks, which is notable because the baselines are trained directly on the evaluation metric while DnPF is not. The most striking result is the out-of-distribution Cluttered Push setting, where image availability drops from 20% (matching training) to 2%: DnPF achieves a normalized NLL of 2.8 versus 4.4 for the next-best baseline, indicating substantially reduced reliance on the training sequence distribution.

The sensor-fusion experiment illustrates the compositional advantage. A simulated noisy external position sensor is added post-training by simply adding its analytically known score to the likelihood term, assuming conditional independence. Fusion improves position estimates over both proprioception-only DnPF and the sensor model alone, and — an interesting secondary finding — the additional position information also substantially improves orientation estimates for which no sensor data exists. End-to-end methods would generally require retraining to incorporate the new sensor.

Ablations on the hand task isolate the contributions of each component. Open-loop unrolling of the learned dynamics model drifts badly, confirming the manifold-exit problem; warm-started few-step denoising with the learned prior score mitigates this drift even without any measurements. Sampling from the learned likelihood alone underperforms, and the full DnPF with the likelihood constraint is best. Runtime analysis shows GPU inference is nearly independent of particle count up to parallel capacity, runtime scales linearly in the number of denoising steps, and a task-dependent sweet spot in $p(x_t \mid y_{1:t}, u_{1:t})$6 (typically 0.2–0.5 in this analysis) balances quality against cost, enabling real-time operation.

Limitations and open questions

The authors are explicit about several caveats. The classifier-free-guidance-style bias $p(x_t \mid y_{1:t}, u_{1:t})$7 is left uncorrected; while empirically beneficial, its effect is heuristic rather than principled. The likelihood constraint relies on denoiser output magnitude as a manifold-distance proxy, an approximation grounded in the ELBO rather than an exact distance. Each particle evolves from a single ancestor rather than the full mixture prior over ancestors, an approximation deferred to future work. Inference introduces four hyperparameters ($p(x_t \mid y_{1:t}, u_{1:t})$8, $p(x_t \mid y_{1:t}, u_{1:t})$9, $p(y_t \mid x_t)$0, $p(y_t \mid x_t)$1) that, although training is agnostic to them, are tuned post hoc per task via sweeps over the evaluation metric — a mild form of evaluation-set adaptation. Unknown static parameters such as friction coefficients are currently absorbed as noise in the learned models rather than explicitly estimated, which the authors identify as an open extension. Finally, all experiments are in simulation; transfer to hardware, where model error is larger and less structured, remains untested.

Conclusion

DnPF demonstrates that state estimation can be learned with purely single-step objectives and recomposed into a coherent Bayesian filter at inference, using diffusion-based posterior sampling with warm-started, likelihood-constrained denoising. It achieves competitive accuracy with end-to-end baselines — with a marked advantage under distributional shift — while supporting prior initialization and zero-shot integration of external sensor models. The main open questions concern principled treatment of the guidance bias, mixture-prior ancestry, explicit estimation of static parameters, and validation on real robotic hardware.