Input design for Bayesian identification of non-linear state-space models

Abstract: We propose an algorithm for designing optimal inputs for on-line Bayesian identification of stochastic non-linear state-space models. The proposed method relies on minimization of the posterior Cramér Rao lower bound derived for the model parameters, with respect to the input sequence. To render the optimization problem computationally tractable, the inputs are parametrized as a multi-dimensional Markov chain in the input space. The proposed approach is illustrated through a simulation example.

• The paper introduces an estimator-independent Bayesian input design framework by minimizing the Posterior Cramér-Rao Lower Bound for nonlinear state-space models.
• It leverages Monte Carlo approximations and a Markov chain parametrization to efficiently optimize long-horizon input sequences under physical constraints.
• Simulation studies demonstrate that optimal Markov chain inputs significantly reduce parameter estimation mean-square error compared to pseudo-random inputs.

Input Design for Bayesian Identification of Nonlinear State-Space Models

Introduction and Problem Setting

The paper addresses the challenge of optimal input design for Bayesian identification in stochastic nonlinear state-space models (SSMs), where the state vector is unmeasured and parameters must be estimated online using observed outputs. The identification problem is formulated in a Bayesian setting, where the goal is to sequentially infer joint posteriors over both states and parameters given observed data and user-controlled input sequences. Accurate parameter inference can be significantly improved by selecting informative input sequences—a process known as Bayesian input design.

The standard input design framework for regression and fully observed models fails to scale to nonlinear SSMs with latent states, primarily due to intractability of computing the posterior or its properties. The paper proposes minimizing the Posterior Cramér-Rao Lower Bound (PCRLB) as a surrogate objective, leveraging Monte Carlo approximations and a Markov chain-based input parameterization to obtain tractable optimization over input sequences.

Bayesian Input Design via PCRLB Minimization

Existing methods optimize criteria such as mean-square error (MSE) of parameter estimates, but for latent state-space models the required posterior quantities are analytically unavailable. The PCRLB provides an attainable lower bound for the error covariance of Bayesian estimators; its minimization with respect to input is theoretically justified, estimator-agnostic, and enables input optimization even under complex model dynamics.

Given a state-space model with process and measurement noise, parameter vector $\theta$, state $X_t$ and input $u_t$, the estimation error covariance for $\theta$ at time $t$ is bounded as

$P_{t|t}^\theta \succcurlyeq L_t^\theta$

where $L_t^\theta$ is derived as the Schur complement of the information matrix $J^z_t$ associated with the joint (parameter, state) process. For nonlinear SSMs, the recursion for $J^z_t$ (and $L_t^\theta$) involves high-dimensional expectations over both the process noise and latent state trajectory, which are not analytically tractable.

Monte Carlo Approximations for PCRLB

The recursive expectations in the information matrix update equations are approximated via Monte Carlo (MC) sampling, enabling practical computation when closed forms are unavailable. Specifically, for additive Gaussian noise SSMs, the nested integrals required for PCRLB updates are simplified using marginalization and MC estimates, thus scaling with the number of samples instead of the exponential size of the state-space. This makes the PCRLB-based input design approach computationally feasible for online and offline applications.

Markov Chain Parametrization of Inputs

Direct optimization over continuous-valued, long-horizon input sequences is computationally prohibitive. The paper discretizes the input space and models input trajectories as realizations of a finite-order Markov chain, characterized by initial and transition probability matrices. Input design is thereby recast as a lower-dimensional stochastic programming problem over these Markov parameters, rather than the entire input sequence space.

Such parameterization enables:

• Incorporation of amplitude and transition constraints.
• Analytical tractability in estimating gradients or optimizing the PCRLB objective with standard solvers.
• Efficient simulation of designed input signals in feedback control systems.

Simulation Study and Empirical Results

A simulation study on a nonlinear, univariate, parameterized SSM demonstrates the procedure. The model includes both nonlinear process dynamics and quadratic measurement relations, with noise terms and a parameter vector $\theta = [a, b, c, d]$. The identification task is to estimate $\theta$ under input constraints and fixed noise covariances.

Four cases for input Markov chain parametrization are compared: three cover optimal parametrizations found via the proposed algorithm (with increasing flexibility in the transition matrix) and one corresponds to a pseudo-random binary input.

The performance metric is the trace of the MSE for parameter estimates over 100 time steps under an SMC-based Bayesian identification filter. The PCRLB traces and actual MSE traces across cases are shown below.

Figure 1: (a) Trace of the approximate lower bound (PCRLB) across input design cases; (b) Trace of the resulting MSE for parameter estimation, showing lower error under optimally designed inputs.

The results indicate:

• The most flexible Markov chain parametrization (Case 3) yields the lowest PCRLB and actual MSE, significantly outperforming pseudo-random and less flexible chain parametrizations.
• The designed input sequence consistently produces tighter MSE, validating the PCRLB as an effective input design criterion.

Critically, the input design is shown to be independent of the Bayesian estimator, operating purely from model-adherent criteria.

Implications and Future Directions

The formalization and algorithmic procedure enable, for the first time, tractable Bayesian input design in stochastic nonlinear SSMs with unobserved state processes. Several implications emerge:

• The estimator-independent criterion broadens applicability, enabling robust, system-agnostic input strategies for parameter inference.
• Markov chain input parametrization allows the inclusion of physical constraints and facilitates ready deployment in real-world process control environments.
• The method can be extended to multivariate and higher-order SSMs, provided sufficient computational budget for MC sampling.

Future work includes addressing non-smoothness and multi-modality of the stochastic optimization landscape, potentially employing advanced stochastic gradient-based techniques, as well as exploring adaptive, online input redesign in nonstationary/uncertain environments.

Conclusion

The paper introduces a practical, estimator-agnostic framework for Bayesian input design in nonlinear SSMs by minimizing the PCRLB. Monte Carlo approximations render the approach feasible in models with intractable posteriors, and Markov input parametrization allows tractable optimization with physical constraints. Simulation results demonstrate improved parameter identifiability and reduced estimation error compared to conventional inputs, with clear implications for adaptive experimental design and real-time system identification.