Non-linearly Constrained Reconciliation (NLCR)

• NLCR is a method that reconciles forecasts by projecting predictions onto non-linear constraint manifolds, ensuring coherence in ratios and polynomial relations.
• It formulates forecast reconciliation as a constrained optimization problem solved via techniques like Sequential Quadratic Programming with covariance weighting.
• Empirical studies show that NLCR reduces forecast errors in real-world scenarios such as US mortality rates and renewable energy predictions.

Non-linearly Constrained Reconciliation (NLCR) is a class of algorithms and theoretical procedures for adjusting forecasts or solutions to ensure they satisfy non-linear constraints. In contrast to linear reconciliation—which projects unconstrained predictions onto a linear subspace defined by aggregation rules—NLCR seeks a projection onto a manifold or subset of ℝⁿ defined by nonlinear constraint functions, such as ratios or polynomial relationships. NLCR has seen recent advances in both foundational theory and practical implementation, especially in time series forecasting and structured prediction contexts.

1. Motivation and Types of Non-linear Constraints

NLCR emerges out of the practical need to reconcile forecasts in domains where important variables are linked by non-linear relationships. In demography, mortality rates are defined as ratios between deaths and population exposures. In economics, unemployment rates represent the ratio of unemployed individuals to the labor force. Wind power prediction, among numerous other applications, involves constraints that are non-linear functions of underlying variables.

Contrary to linear aggregation rules, reconciling time series forecasts under non-linear constraints such as ratio constraints ($R = D/P$, $$U = 100\, [\text{unemployed} / \text{labour force}]$$), polynomial relationships, or more complex transformations requires projection techniques that respect both the linear and nonlinear structure of the coherent manifold.

2. NLCR Algorithmic Formulation

NLCR is formalized as a constrained optimization problem. Given an incoherent base forecast vector $\hat{y} \in \mathbb{R}^n$, and a set of non-linear constraint functions $g(z) = 0$, NLCR proceeds by solving:

$$\text{find } z^* = \arg\min_{z} \; (z - \hat{y})' W^{-1} (z - \hat{y}) \;\;\text{subject to} \;\; g(z)=0$$

where $W$ is a positive-definite weighting (covariance) matrix, typically informed by forecast error covariance or set to identity for simplicity.

The Lagrangian is given by

$$\mathcal{L}(z, \lambda) = (z - \hat{y})' W^{-1} (z - \hat{y}) + \lambda' g(z)$$

and the optimization proceeds via iterative techniques, typically Sequential Quadratic Programming (SQP), due to the lack of a closed-form solution for general nonlinear $g(\cdot)$. The solution imposes the conditions

$$\frac{\partial \mathcal{L}}{\partial z} = 2W^{-1}(z - \hat{y}) - 2J' \lambda = 0$$

where $J$ is the Jacobian of $g$ at $z$.

The adjustment vector $z^* - \hat{y}$ is therefore aligned with the constraint gradients.

3. Theoretical Guarantees and Sufficient Conditions

Unlike in the linear case, NLCR does not universally guarantee improved accuracy for all base forecasts. Sufficient conditions are derived through geometric analysis of the constraint manifold. The key result asserts that if the true observation lies within a specific neighborhood ("ball") around a critical point on the manifold, reconciliation reduces mean squared error (MSE):

$d(y, \tilde{y}) < d(y, \hat{y})$

The radius $r$ of this ball depends on the Lagrange multipliers, inner products of the constraint gradients, and local curvature.

For the case of a single constraint, the expression is:

$$r = \sqrt{ \kappa^2 \langle j, j \rangle + \mu \kappa \lambda \langle j, \tilde{j} \rangle + \frac{\mu^2}{4} \lambda^2 \langle \tilde{j}, \tilde{j} \rangle }$$

where $j$ and $\tilde{j}$ represent gradients at relevant points, $\mu$ and $\lambda$ are multiplier parameters, and $\kappa$ encodes curvature.

When all constraint functions are convex and the base forecast lies within the intersection of their hypographs, reconciliation using NLCR is always guaranteed to lower forecast error. High curvature and greater distance from the manifold restrict the applicability radius.

Table: Core Theoretical Features of NLCR

Property	Linear Reconciliation	NLCR (General Case)
Universal error reduction	Yes	Only under sufficient conditions
Closed-form solution	Yes	Generally requires iteration
Covariance weighting	Possible	Recommended

4. Empirical and Simulation Evidence

Empirical studies validate NLCR across both synthetic and real-world datasets:

• Simulation 1 (Quartic Constraint): For $y_1 = y_2^4$, improved accuracy is nearly universal when the base forecast is biased toward the hypograph; sensitivity to local curvature affects improvement near transition regions.
• Simulation 2 (Ratio Constraint): For $y_1 = 100(y_2/y_3)$, reconciliation yields higher accuracy for most bias and correlation settings—even with highly non-linear constraints—with improvement probabilities commonly between 79% and 85%.
• Real Data Applications:
  • US Mortality Rates: NLCR enforces both linear aggregation and $R = D/P$ ratio constraint, outperforming both bottom-up and linearized methods in RMSE and statistical tests (Diebold–Mariano, Model Confidence Sets, Nemenyi).
  • Australian Unemployment Rates: Hierarchical time series forecasts are reconciled for the unemployment rate via NLCR, yielding statistically significant improvements.

NLCR is robust to misspecification of covariance structure provided that the weighting matrix does not diverge drastically from the true forecast error distribution.

5. Probabilistic Guarantees and Extensions

For nonlinear manifolds with varying curvature or cases where the sufficient conditions are not globally satisfied, recent theoretical advances supplement deterministic guarantees with probabilistic assurances. The estimator

$$e((\tilde{z}_i, \pi_i)) = \sum_i \pi_i \cdot \mathbf{1}\{\delta_\pi^T \tilde{\delta}_i \geq -\tfrac{1}{2} \|\delta_\pi\|^2\}$$

with sample weights $\pi_i$ evaluates, for a given realization of the true state, whether reconciliation decreases error. Clopper–Pearson intervals provide uncertainty quantification for improvement probability estimates (Nespoli et al., 30 Jul 2025).

This probabilistic assessment allows dynamic thresholding: the reconciliation is performed only when the estimated probability of improvement exceeds a practitioner-specified value.

6. Applications Across Domains

NLCR is suitable for hierarchical and grouped forecasting under non-linear structural or physical constraints (e.g., conservation laws, network constraints, aggregation rules). It extends traditional forecast reconciliation methodologies to any domain where non-linear relationships must be respected for forecasts to be coherent or usable.

Other relevant domains include:

• Power grid monitoring, renewable energy forecasting (e.g., wind power curves),
• State estimation in control and cyber–physical systems,
• Economics, finance, and demography.

Further, the same geometric and optimization principles underlie NLCR’s use in reconciling solutions in nonlinear programming and satisfaction problems across areas such as verification, scheduling, and resource allocation.

7. Limitations and Implementation Considerations

NLCR does not preserve unbiasedness in general due to the nonlinearity of the projection map. The improved accuracy guarantee is local to regions of sufficient convexity; for forecasts far from the coherent manifold, reconciliation may not improve or may diminish accuracy. Sequential Quadratic Programming (SQP) is the typical solver, and convergence can be sensitive to initialization and the local landscape. Covariance matrix specification is critical for maximum theoretical accuracy improvement.

The implementation in applied contexts is facilitated by modern open-source optimization packages, with specialized support (e.g., a JAX-based Python library (Nespoli et al., 30 Jul 2025)) offering GPU acceleration and automatic differentiation for efficient nonlinear projections.

---

NLCR systematically broadens reconciliation methods to embrace non-linear constraints, providing a rigorous optimization-based projection formalism, geometric and probabilistic improvement conditions, and robust empirical validation in demography and macroeconomics (Girolimetto et al., 24 Oct 2025, Nespoli et al., 30 Jul 2025). Its continuing development is likely to impact coherent forecasting and constrained solution tasks in scientific and engineering disciplines where nonlinear relationships are integral.