Hybrid LNN+XGBoost Model

• Hybrid LNN+XGBoost model is a fusion of continuous-time RNNs (LNN) and gradient boosting trees (XGBoost) that extracts dynamic features for improved predictions.
• It sequentially uses LNN to capture adaptive memory dynamics from time-series data and employs XGBoost to provide robust, interpretable non-linear regression.
• The modular design enables efficient, real-time inference and enhanced performance in applications like supply chain optimization and healthcare diagnostics.

A hybrid LNN+XGBoost model fuses a liquid neural network (LNN)—a class of continuous-time recurrent neural networks with biologically-inspired, adaptive memory dynamics—with an extreme gradient boosting tree ensemble (XGBoost), in a modular or sequential manner. This paradigm consistently appears in state-of-the-art sequence modeling and time-series prediction tasks in supply chain optimization and healthcare, where it achieves enhanced accuracy, noise robustness, and interpretability relative to single-model baselines (Tong, 16 Dec 2025, Tong, 28 Jul 2025, Huang et al., 20 Oct 2025). The central architectural theme is to use the LNN for dynamic feature extraction from temporal signals, and XGBoost as a powerful nonlinear regressor/classifier for downstream prediction or decision-making.

1. Architectural Principles and Data Flow

The LNN+XGBoost hybrid is a sequential pipeline with two primary components:

1. LNN dynamic feature extractor: The LNN processes time-series features (e.g., multivariate demand, inventory, sensor values) via an adaptive liquid neuron state update:

$$s_t = (1 - \alpha_t)\, s_{t-1} + \alpha_t\, a_t + \frac{dt}{\tau} (-s_{t-1} + a_t),$$

with $a_t = \phi(W_{in}\, x_t + W_{rec}\, s_{t-1} + b)$ and adaptive leak $$\alpha_t = \alpha_{0} + \beta\,\mathrm{volatility}(x_t)$$. This yields a low-dimensional dynamic state vector $s_t$ representing recent history (Tong, 16 Dec 2025, Tong, 28 Jul 2025).

2. XGBoost regressor/classifier: XGBoost receives as input $\bigl[ x_t,\, s_t \bigr]$—the engineered static features $x_t$ (e.g., lagged values, volatility, seasonality, exogenous variables) concatenated with the LNN state—and outputs the final prediction, such as demand/consumption forecast, probability estimate, or classification score.

The training procedure is typically stagewise: train the LNN on a forecasting or representation loss, freeze its parameters, extract $s_t$ on the whole dataset, and then train XGBoost on the concatenated features for the desired output. No end-to-end gradient flow traverses both models.

2. Mathematical Formulation

Let $x_t \in \mathbb{R}^m$ denote the static engineered features at time $t$, and $s_t \in \mathbb{R}^n$ the LNN dynamic state. The LNN evolution is governed by a discretized ODE:

$$s_t = (1 - \alpha_t) s_{t-1} + \alpha_t a_t + \frac{dt}{\tau}(-s_{t-1} + a_t),$$

where $\alpha_t$ is an adaptive leak parameter and $a_t$ is a nonlinear pointwise transformation. The LNN is trained to minimize a forecasting loss, typically mean squared error (MSE):

$$\mathcal{L}_{\mathrm{LNN}} = \frac{1}{N} \sum_{t=1}^N (y_t - \hat{y}_t^{(\mathrm{LNN})})^2 + \lambda \|\theta\|^2.$$

The XGBoost model is an additive tree ensemble

$$\hat{y}_t^{\mathrm{XGB}} = \sum_{k=1}^K f_k([x_t, s_t]),$$

with individual tree regularization $$\Omega(f) = \gamma T + \frac{1}{2} \lambda \sum_{j=1}^T w_j^2$$, where $T$ is the number of leaves. XGBoost optimizes a regularized objective (MSE for regression, logistic loss for classification):

$$\min_{\{f_k\}} \sum_{i=1}^N L(y_i, \hat{y}_i) + \sum_{k=1}^K \Omega(f_k).$$

3. Training Algorithms and Hyperparameter Choices

The hybrid framework is always trained in two decoupled stages:

• LNN training:
  • Loss: MSE for forecasting; one-step-ahead or multi-step as appropriate
  • Optimizer: AdamW with learning rate $[10^{-5}, 10^{-3}]$
  • Regularization: weight decay $10^{-4}$
  • Batch size: $4–8$ (sequence models)
  • Hidden units: $n \in [64,1024]$
  • Early stopping on validation loss
• XGBoost training:
  • Number of trees: $K \in [100, 300]$
  • Max depth: $[3, 7]$
  • Learning rate: $[0.01, 0.3]$
  • Regularization: $\gamma=0$, $\lambda=1$
  • Train on training set using the concatenated feature vector $[x_t, s_t]$, validate performance on out-of-sample data

Data preprocessing includes min-max normalization of features, sliding windows for the LNN input, and systematic handling of seasonality, lagged orders/sales, and volatility measures (Tong, 16 Dec 2025).

4. Empirical Performance and Benchmarking

Supply chain & time series applications:

• Bullwhip mitigation: LNN+XGBoost achieves a lower order-variance ratio (Layer 3 $\approx$ 1.15) vs. XGBoost alone ($\approx$ 1.30) or sequence models (LSTM, Transformer, DQN) (Tong, 16 Dec 2025, Tong, 28 Jul 2025).
• Forecasting error (MAE): Hybrid model MAE $\approx$ 3.5 vs. XGBoost 4.0, LSTM 5.0, Transformer 4.7.
• Profitability: Composite scores favor the hybrid model (0.6297 vs. 0.6221 for XGBoost; $p < 10^{-4}$). Under moderate demand noise, hybrid profits degrade only 9%, while standalone XGBoost suffers a 20% drop (Tong, 16 Dec 2025).
• Computational efficiency: LNN+XGBoost inference $\approx$ 5 ms/step (real-time edge deployment), with LNN training + feature extraction $<$ 2 hrs (CPU) (Tong, 16 Dec 2025).

Healthcare and classification:

The same structure generalizes to hybrid classifiers for clinical event prediction by substituting LNNs for sequence encoding of temporal biomarker/waveform data, with XGBoost operating on the learned state for robust, interpretable risk stratification (Huang et al., 20 Oct 2025).

Regression and tabular domains:

The architecture is extensible to tabular, high-dimensional, and low-data regimes, where LNNs (or other feature learners) are used to compress or denoise inputs, with XGBoost capturing residual nonlinear interactions (K, 2 Dec 2025).

5. Synergy of Local Adaptivity and Global Optimization

• Local adaptivity (LNN):

The ODE-based continuous-time nature and adaptive leak allow the LNN to track rapid local fluctuations and update internal representations in real time. This makes LNNs highly suitable for edge or low-latency environments in supply chain nodes, as well as for streaming biomedical or industrial data.

• Global optimization (XGBoost):

XGBoost leverages the LNN's state as a compact, dynamic summary while integrating global dependencies and cross-feature interactions via tree-based optimization. Regularization in XGBoost provides robustness to overfitting and noise, further stabilizing system-wide predictions (Tong, 16 Dec 2025, Tong, 28 Jul 2025).

This dual level of adaptivity and global coordination is essential for complex systems that experience nonstationary dynamics and require scalable, interpretable predictions or interventions.

6. Interpretability and Practical Implications

• Feature attribution:

XGBoost ensemble structure allows for direct computation of global and local feature importances (gain, SHAP values). This capability is preserved when the input includes the LNN-derived state, allowing practitioners to link model outputs to actionable features (e.g., which input variables or local LNN state changes drive order adjustments or clinical alerts).

• Deployability:

Due to the linear inference and low resource demands of LNNs, the hybrid model enables deployment on edge devices with constrained computation budgets, supporting real-time interventions in resource-constrained environments.

• Adaptation to regime change:

The adaptive leak mechanism of LNNs confers resilience to sudden shifts in input dynamics without requiring frequent retraining—a crucial property for supply chain resilience or online clinical monitoring (Tong, 16 Dec 2025, Tong, 28 Jul 2025).

7. Limitations and Future Directions

• Sequential training:

No end-to-end differentiability: LNN and XGBoost are trained in succession, prohibiting joint optimization. This can limit the expressiveness available via fully end-to-end architectures but improves stability and modularity.

• Hyperparameter tuning overhead:

Joint optimization of LNN (dynamic parameters, leak, time constant) and XGBoost (tree depth, shrinkage) requires extensive cross-validation and hyperparameter searches, often using Bayesian methods (Optuna TPE) for global optima (Tong, 16 Dec 2025).

• Generalization beyond time series:

While the LNN+XGBoost hybrid is highly effective for sequential and temporally structured data, analogous architectures using MLPs or CNN/Transformer feature learners have also been validated in domains ranging from EHR risk prediction to natural gas demand forecasting (Huang et al., 20 Oct 2025, Firoozeh et al., 2 Oct 2025).

---

Hybrid LNN+XGBoost frameworks establish a rigorous engineering and analytical template for combining dynamic adaptation with interpretable global optimization across a spectrum of real-world tasks, including supply chain, biomedical signal analysis, and multivariate time-series regression (Tong, 16 Dec 2025, Tong, 28 Jul 2025, Huang et al., 20 Oct 2025).