Weighted Null Space Fitting (WNSF) Overview

• Weighted Null Space Fitting (WNSF) is a weighted least-squares framework that begins with a high-order non-parametric model and reduces it via null-space techniques to achieve robust system identification.
• The method bridges traditional prediction error methods and subspace identification by replacing non-convex optimization with sequential least squares and statistically informed weighting.
• WNSF is applicable in transfer-function, closed-loop, cascade-network, and state-space formulations, offering practical benefits like avoidance of SVD and improved numerical robustness.

Weighted Null Space Fitting (WNSF) is a weighted least-squares identification framework developed as a bridge between the prediction error method (PEM) and subspace identification methods (SIMs). Its defining pattern is to begin with a high-order non-parametric model estimated by least squares, reduce that model to a low-order parametric or state-space description through a null-space relation, and then apply statistically motivated weighting so that the final estimate inherits favorable asymptotic properties without requiring direct solution of a non-convex prediction-error problem (Galrinho et al., 2017, He et al., 2024). Across the literature, WNSF appears in parametric transfer-function identification, semi-parametric closed-loop estimation, cascade-network identification, and state-space realization in observer canonical form (Galrinho et al., 2017, Galrinho et al., 2018, He et al., 2 Oct 2025).

1. Historical placement and motivation

WNSF emerged from a longstanding contrast in system identification. PEM is treated as the reference method because, with Gaussian noise and a correct noise model, quadratic-cost PEM is asymptotically efficient and achieves the Cramér–Rao bound. Its drawback is that the cost function is, in general, non-convex, so practical estimation depends on iterative nonlinear optimization and good initialization (Galrinho et al., 2017).

SIMs occupy the opposite side of this trade-off. They are numerically robust, convenient for state-space models, and usually proceed through linear regression, SVD, and realization. At the same time, their asymptotic efficiency is unresolved in general, and practical use often requires choices of horizons and weighting matrices whose statistical status is unclear (He et al., 2024).

WNSF was proposed precisely to combine these two traditions. In the transfer-function setting, it was introduced as a three-step least-squares-based alternative to PEM that uses a high-order ARX model as an intermediate representation and then applies weighted least squares to recover the same asymptotic covariance as quadratic-cost PEM when the model orders are chosen according to the true system (Galrinho et al., 2017). In the closed-loop semi-parametric setting, the motivation was sharper: closed-loop data can make PEM inconsistent unless the noise model is flexible enough, but a very flexible noise model increases complexity and aggravates non-convex optimization. Semi-parametric WNSF avoids this trade-off by using a high-order non-parametric ARX model and reducing only to the dynamic model of interest, without estimating a parametric noise model in the reduction step (Galrinho et al., 2017).

The later state-space formulation reframed the same objective in a realization language. There, WNSF is presented as a method that aims to get the accuracy of PEM with the numerical robustness of SIM, while avoiding nonlinear optimization and the SVD-centered range-space extraction typical of subspace identification (He et al., 2024).

2. Canonical three-step structure

The original WNSF formulation is built around a high-order ARX approximation of a Box–Jenkins-type system. The true system is written as

$y_t = G_o(q)u_t + H_o(q)e_t,$

with corresponding infinite-order ARX representation

$A_o(q)y_t = B_o(q)u_t + e_t.$

Because an infinite model is not estimable from finite data, WNSF truncates to an ARX model of order $n=n(N)$,

$A(q,\eta^n)y_t = B(q,\eta^n)u_t + e_t,$

and estimates $\eta^n$ by ordinary least squares through the linear regression

$$y_t=(\varphi_t^n)^\prime\eta^n+e_t,\qquad \hat{\eta}^n_N=[R_N^n]^{-1}r_N^n.$$

This first step is convex and supplies the non-parametric intermediate estimate (Galrinho et al., 2017, Galrinho et al., 2017).

The second step is the null-space reduction. The low-order parametric model is linked to the high-order ARX coefficients by algebraic identities such as

$\eta_o^n - Q_n(\eta_o^n)\theta_o = 0$

for the fully parametric case, or

$b_o^n - Q_n(\eta_o^n)\theta_o = 0$

for the semi-parametric dynamic-model reduction. Replacing the unknown high-order coefficients by their least-squares estimate yields an overdetermined linear system, and the parameter vector is estimated by least squares (Galrinho et al., 2017, Galrinho et al., 2017).

The third step is weighted least squares. Its role is variance reduction and, under the stated assumptions, recovery of PEM-like asymptotic behavior. The residual relation is expressed as a linear transformation of the high-order estimation error through a Toeplitz-structured matrix:

$$\hat{b}_N^n - Q_n(\hat{\eta}_N^n)\theta_o = T_n(\theta_o)\big(\hat{\eta}_N^n-\eta_o^n\big).$$

Using the asymptotic covariance of the ARX estimate,

$$\sqrt{N}\big(\hat{\eta}_N^n-\bar{\eta}^n\big)\sim As\mathcal{N}\big(0,\sigma_o^2[\bar{R}^n]^{-1}\big),$$

the optimal weighting is built from the inverse residual covariance (Galrinho et al., 2017). In the fully parametric version, the same logic leads to a weight derived from

$A_o(q)y_t = B_o(q)u_t + e_t.$0

and the final estimate is a weighted least-squares correction of the step-2 estimate (Galrinho et al., 2017).

A central regularity condition in the foundational theory is that the high-order ARX dimension grows with sample size while remaining controlled:

$A_o(q)y_t = B_o(q)u_t + e_t.$1

for some $A_o(q)y_t = B_o(q)u_t + e_t.$2 (Galrinho et al., 2017, Galrinho et al., 2017).

3. Null-space fitting in state-space form

The state-space version of WNSF replaces transfer-function reduction by a realization problem in observer canonical form. The system is written in predictor form as

$A_o(q)y_t = B_o(q)u_t + e_t.$3

where $A_o(q)y_t = B_o(q)u_t + e_t.$4. This yields the infinite autoregressive representation

$A_o(q)y_t = B_o(q)u_t + e_t.$5

WNSF first estimates a sufficiently long truncated model

$A_o(q)y_t = B_o(q)u_t + e_t.$6

by ordinary least squares, obtaining estimated predictor Markov parameters (He et al., 2024).

The structural step is the replacement of range-space estimation by null-space estimation. For state dimension $A_o(q)y_t = B_o(q)u_t + e_t.$7, the extended observability matrix

$A_o(q)y_t = B_o(q)u_t + e_t.$8

has rank $A_o(q)y_t = B_o(q)u_t + e_t.$9, so its null space is one-dimensional. If $n=n(N)$0 are the coefficients of the characteristic polynomial of $n=n(N)$1, the Cayley–Hamilton theorem gives

$n=n(N)$2

From the first $n=n(N)$3 Markov parameters one forms a Hankel matrix $n=n(N)$4, and because

$n=n(N)$5

the same null-space relation holds for the Hankel matrix:

$n=n(N)$6

The initial estimate of the characteristic-polynomial coefficients is therefore an ordinary least-squares solution in the null-space equation (He et al., 2024).

This formulation is the explicit point of departure from SIM. SIM typically estimates the range space of the extended observability matrix through SVD. WNSF instead estimates the null space relation implied by the characteristic polynomial, thereby avoiding singular-value decomposition, singular-vector separation, and tuning of past/future horizons in the reported formulation (He et al., 2024).

The broader realization literature later made this distinction precise. In the approximate realization problem based on truncated Markov parameters, the range-space-based realization algorithm is interpreted as a total least-squares solution, whereas the null-space-based realization algorithm is interpreted as an ordinary least-squares solution. The same work argues that both are suboptimal relative to a covariance-informed weighted least-squares formulation, and establishes the asymptotic ordering

$n=n(N)$7

for the realization parameters (He et al., 26 May 2025). This suggests that WNSF is not merely a reformulation of subspace realization, but a statistically weighted second-stage estimator built around null-space structure.

After estimating the null-space coefficients, the 2024 state-space WNSF constructs the observer canonical realization

$n=n(N)$8

and then estimates the remaining free parameter vector $n=n(N)$9 by ordinary least squares because the one-step predictor is linear in $A(q,\eta^n)y_t = B(q,\eta^n)u_t + e_t,$0 once $A(q,\eta^n)y_t = B(q,\eta^n)u_t + e_t,$1 and $A(q,\eta^n)y_t = B(q,\eta^n)u_t + e_t,$2 are known (He et al., 2024).

4. Weighting, covariance structure, and asymptotic theory

The statistical core of WNSF is the derivation of the weight from the covariance of the first-stage estimate. In the state-space formulation, neglecting truncation bias, the high-order Markov-parameter estimate satisfies

$A(q,\eta^n)y_t = B(q,\eta^n)u_t + e_t,$3

Because the null-space residual can be written as a Toeplitz transformation of the Markov-parameter estimation error,

$A(q,\eta^n)y_t = B(q,\eta^n)u_t + e_t,$4

its asymptotic covariance is

$A(q,\eta^n)y_t = B(q,\eta^n)u_t + e_t,$5

and the statistically optimal weight is the inverse covariance $A(q,\eta^n)y_t = B(q,\eta^n)u_t + e_t,$6 (He et al., 2024).

The transfer-function literature uses the same template. The reduction residual is expressed as a structured linear transformation of the high-order ARX estimation error through $A(q,\eta^n)y_t = B(q,\eta^n)u_t + e_t,$7, and the weighting matrix is the inverse of the resulting covariance. In the semi-parametric closed-loop paper, this yields

$A(q,\eta^n)y_t = B(q,\eta^n)u_t + e_t,$8

whereas the implemented weight uses consistent estimates in place of the unknown population quantities (Galrinho et al., 2017).

The theoretical status of WNSF varies across formulations. For parametric transfer-function identification, WNSF is shown to provide estimates with the same asymptotic properties as PEM with a quadratic cost function when the model orders are chosen according to the true system; the stated results are consistency, asymptotic Gaussianity, and the same asymptotic covariance as PEM (Galrinho et al., 2017). For semi-parametric WNSF, the step-2 least-squares estimate is consistent, the step-3 weighted least-squares estimate is consistent, and

$A(q,\eta^n)y_t = B(q,\eta^n)u_t + e_t,$9

In open loop, this gives asymptotic efficiency; in closed loop, the same covariance is interpreted as the optimal asymptotic covariance for an infinite-order noise model (Galrinho et al., 2017).

The 2024 state-space paper is more cautious. It derives the statistically optimal weight and conjectures that WNSF is asymptotically efficient; the conjecture is supported by simulation evidence (He et al., 2024). The subsequent state-space extension strengthens this point: when a canonical parametrization is admissible, the proposed method is consistent and asymptotically efficient, including in closed loop, and the paper presents this as progress on the open question of an asymptotically efficient SIM (He et al., 2 Oct 2025).

A technically significant aspect of the semi-parametric theory is its treatment of growing-dimension variance analysis. The closed-loop covariance proof relies on a new geometric result for a matrix limit with increasing ARX order, enabling the asymptotic identification of the optimal information expression $\eta^n$0 (Galrinho et al., 2017).

5. Variants and extensions

The WNSF literature now includes several distinct formulations. The common pattern is stable: a non-parametric model is estimated first, a structured linear or null-space relation connects it to the desired low-order model, and a weighted least-squares correction uses the induced covariance.

Variant	Intermediate model	Reported property
Parametric WNSF	High-order ARX	Same asymptotic properties as PEM with a quadratic cost function (Galrinho et al., 2017)
Semi-parametric WNSF	High-order ARX	Consistent in closed loop; asymptotically efficient in open loop (Galrinho et al., 2017)
Cascade-network WNSF	Non-parametric FIR MIMO model	Estimates all modules simultaneously without non-convex optimization (Galrinho et al., 2018)
State-space WNSF	High-order AR / HOARX Markov-parameter model	Null-space realization in observer canonical form (He et al., 2024, He et al., 2 Oct 2025)

The semi-parametric version is designed for cases where the noise spectrum is too complicated to be captured well by a low-order parametric model. Its distinctive feature is that a parametric noise model does not need to be estimated in the reduction step if it is not of interest. Because the flexibility of the noise model is increased with the sample size, the method still provides consistent estimates in closed loop and asymptotically efficient estimates in open loop (Galrinho et al., 2017).

The cascade-network extension considers a serial cascade dynamic network with sensor noise. There, WNSF uses a non-parametric MIMO FIR model, rewrites the structured network equations so that they become linear in the module parameters, and then constructs a weighted least-squares estimator from the covariance of the FIR estimation error. The paper does not provide a full asymptotic proof for the network case, but argues that the extension preserves the essential WNSF structure and presents simulations suggesting asymptotic efficiency (Galrinho et al., 2018).

The state-space extension generalizes WNSF from structured transfer-function models to black-box state-space models. In the SISO case it uses observer canonical form; in the MIMO case it introduces an admissible canonical parametrization based on a Kronecker index and a corresponding echelon or overlapping state-space form. The admissibility condition is essential: if the chosen Kronecker index does not match the true structure, inconsistency can occur (He et al., 2 Oct 2025).

6. Comparative assessment, reported performance, and limitations

Relative to PEM, WNSF is intended to preserve PEM-like statistical performance while replacing nonlinear search by a sequence of least-squares or weighted least-squares problems. The literature repeatedly emphasizes the same practical consequences: no delicate initialization in the PEM sense, no local nonlinear search, and improved robustness when the PEM cost landscape is difficult (Galrinho et al., 2017, He et al., 2024).

Relative to SIM, the main distinction is the move from range-space extraction to null-space fitting. The 2024 state-space paper identifies several consequences of this choice: no SVD is required, no singular-vector separation problem arises, no ad hoc weighting around the SVD step is needed, and no past/future horizons $\eta^n$1 must be tuned in the reported formulation (He et al., 2024).

The empirical record reported in the papers is broadly consistent with those aims. In the 2024 ARMA example,

$\eta^n$2

with $\eta^n$3, $\eta^n$4, and $\eta^n$5, 1000 Monte Carlo trials were used to examine the MSE of the estimated parameter $\eta^n$6 and the fit of the impulse response. The reported findings are that the MSE of WNSF approaches the CRLB as sample size grows, WNSF performs better than SIM in the impulse-response fit, and WNSF is comparable to PEM in accuracy (He et al., 2024).

The earlier transfer-function work reports additional evidence on difficult numerical examples and random-system benchmarks. WNSF is described as competitive with state-of-the-art methods, especially when PEM suffers from bad initialization or local minima. In one reported resonant example, WNSF reaches average FIT $\eta^n$7 after two iterations, whereas PEM with Levenberg–Marquardt takes about 20 iterations to reach the same value (Galrinho et al., 2017). The cascade-network paper similarly reports that WNSF is much faster than PEM variants and avoids the same convergence failures, while PEM initialized from an over-parameterized MIMO OE model often remains far from the practically best solution (Galrinho et al., 2018). The later state-space benchmarks report favorable comparisons with N4SID, SSARX, PBSID$\eta^n$8, and default-initialized PEM, while also noting that well-initialized PEM can still win in some cases because it remains asymptotically efficient (He et al., 2 Oct 2025).

Several limitations recur in the literature. WNSF requires selection of a sufficiently large high-order ARX, AR, FIR, or HOARX model, and the asymptotic theory depends on controlled order growth (Galrinho et al., 2017, Galrinho et al., 2017). In the state-space setting, classical SIMs estimate order naturally through singular-value truncation, whereas WNSF does not; it assumes a known order and, in MIMO form, a chosen canonical parametrization (He et al., 2 Oct 2025). In the semi-parametric setting, if the noise model is not of interest, open-loop asymptotic efficiency of the dynamic model still holds, but in closed loop the asymptotic covariance is interpreted as that of PEM with an infinite-order noise model rather than full PEM efficiency for a fully specified low-order noise parametrization (Galrinho et al., 2017, Galrinho et al., 2017).

Taken together, the literature positions WNSF as a family of covariance-informed, multi-step least-squares methods whose defining innovation is the use of null-space structure in the second-stage reduction. Its significance lies in showing that statistically grounded weighting can be combined with realization-style estimation, thereby narrowing—and in some formulations closing—the gap between PEM and subspace identification (He et al., 2024, He et al., 2 Oct 2025).