Piecewise-Linear Regeneration

• Piecewise-linear regeneration is a framework that systematically recovers continuous or discontinuous piecewise-linear functions using techniques like total variation regularization and convex optimization.
• It leverages analytic approaches, triangulation, and kernel-based methods to achieve robustness, sparsity, and stability in modeling multi-dimensional data.
• The method underpins applications in sparse regression, dynamical systems, neural network surrogates, and regression tree modeling, offering both interpretability and computational efficiency.

Piecewise-linear regeneration refers to the systematic recovery or reconstruction of continuous or discontinuous piecewise-linear (PL) or continuous piecewise-linear (CPWL) functions from data, models, or prior observations. It encompasses analytic frameworks, algorithmic approaches, and functional representations in both one and higher dimensions. This concept is foundational in sparse regression, variational learning, kernel interpolation, regression tree modeling, and dynamical systems identification, with direct connections to ReLU neural networks, adaptive spline theory, and convex optimization.

1. Fundamental Characterizations and Canonical Formulations

At its core, piecewise-linear regeneration formalizes the recovery of a function $f:\mathbb{R}^d\to\mathbb{R}$ or $f:\Omega\to\mathbb{R}$ that is affine on a partition of the domain. In the one-dimensional case, optimal recovery is achieved by total-variation (TV) regularization on the second derivative, enforcing sparsity in the set of breakpoints or “knots” of the solution: $$\min_{f\in \mathrm{BV}}\;\sum_{m=1}^{M}E(f(x_m),y_m)\;+\;\lambda\|f''\|_{\mathcal{M}}$$ where $E$ is a strictly convex data fidelity term, and $\|f''\|_{\mathcal{M}}$ denotes the total variation of the second derivative as a Radon measure. The extremal solutions are precisely PL (ReLU-spline) functions with the knot set determined by the data and regularization parameter $\lambda$ (Debarre et al., 2020).

For $d>1$, CPWL functions can be uniquely parameterized via triangulation of the domain, with the function value specified at each vertex. The function takes the form: $f(x) = \sum_{i=1}^{N_g} v_i\, \phi_i(x)$ where $\phi_i$ are barycentric “hat” functions and $v_i$ the values at grid points. The recovery problem becomes a convex generalized LASSO: $f:\Omega\to\mathbb{R}$0 where $f:\Omega\to\mathbb{R}$1 encodes jump-discontinuities of the gradient across simplex facets (Pourya et al., 2022).

2. Solution Structure, Stability, and Sparsity

Extreme-point minimizers of 1D regularized problems are of the form: $f:\Omega\to\mathbb{R}$2 For exact interpolation, uniqueness is dictated by the dual certificate $f:\Omega\to\mathbb{R}$3; if $f:\Omega\to\mathbb{R}$4 on all intervals, the solution is unique and equals the canonical PL interpolant. When saturation occurs ($f:\Omega\to\mathbb{R}$5), the solution set is a convex polytope, and all minimizers coincide with the canonical interpolant outside saturated intervals but can vary convexly or concavely within each saturated run. The union of solution graphs is the union of the canonical graph and certain convex hulls determined by neighboring data (Debarre et al., 2020).

Among non-unique minimizers, the sparsest solution is achieved by merging consecutive canonical knots. The minimal knot count is

$f:\Omega\to\mathbb{R}$6

where $f:\Omega\to\mathbb{R}$7 counts the length of each saturation run. The continuous optimality structure and minimal knot-formula are robust to penalty forms, as the penalized and constrained (“basis-pursuit”) problems are equivalent for strictly convex $f:\Omega\to\mathbb{R}$8 (Debarre et al., 2020).

For multi-dimensional CPWL, the stability of the regeneration is characterized by the Riesz-basis property of the chosen hat functions. This holds iff the triangulation is volume-regular: $f:\Omega\to\mathbb{R}$9 for all vertices $$\min_{f\in \mathrm{BV}}\;\sum_{m=1}^{M}E(f(x_m),y_m)\;+\;\lambda\|f''\|_{\mathcal{M}}$$0. The $$\min_{f\in \mathrm{BV}}\;\sum_{m=1}^{M}E(f(x_m),y_m)\;+\;\lambda\|f''\|_{\mathcal{M}}$$1 condition number is $$\min_{f\in \mathrm{BV}}\;\sum_{m=1}^{M}E(f(x_m),y_m)\;+\;\lambda\|f''\|_{\mathcal{M}}$$2 and is optimal in the case of uniform triangulation and hat functions corresponding to shifted linear box splines (Goujon et al., 2022).

3. Algorithmic and Variational Frameworks

1D TV-Regularized PL Regression

A two-step $$\min_{f\in \mathrm{BV}}\;\sum_{m=1}^{M}E(f(x_m),y_m)\;+\;\lambda\|f''\|_{\mathcal{M}}$$3 pipeline is employed. First, the discrete $$\min_{f\in \mathrm{BV}}\;\sum_{m=1}^{M}E(f(x_m),y_m)\;+\;\lambda\|f''\|_{\mathcal{M}}$$4-penalized surrogate is solved for the data vector (via ADMM or FISTA). Second, the canonical spline is constructed, its dual certificate analyzed for saturation runs, and sparsification is performed via explicit pairing-merging, producing a sparsest minimizer (Debarre et al., 2020).

Higher-dimensional CPWL via Delaunay Triangulation

Given a triangulation and scattered data, the task is to assemble the design matrix $$\min_{f\in \mathrm{BV}}\;\sum_{m=1}^{M}E(f(x_m),y_m)\;+\;\lambda\|f''\|_{\mathcal{M}}$$5 (barycentric hat evaluations at data sites) and the matrix $$\min_{f\in \mathrm{BV}}\;\sum_{m=1}^{M}E(f(x_m),y_m)\;+\;\lambda\|f''\|_{\mathcal{M}}$$6 encoding facet-wise gradient jumps, followed by generalized LASSO optimization. Fast evaluation requires explicit computation of barycentric gradients, facet normals, and simplex facet volumes, all detailed algorithmically (Pourya et al., 2022).

Kernel-based Regeneration

Piecewise-linear interpolation is recovered as kernel interpolation in a suitable RKHS (e.g., $$\min_{f\in \mathrm{BV}}\;\sum_{m=1}^{M}E(f(x_m),y_m)\;+\;\lambda\|f''\|_{\mathcal{M}}$$7 with non-standard inner product). The reproducing kernel is explicitly piecewise-linear (affine on either side of the diagonal), is the Green function of a second-order ODE with mixed boundary conditions, and the interpolation coefficients are determined by solving a $$\min_{f\in \mathrm{BV}}\;\sum_{m=1}^{M}E(f(x_m),y_m)\;+\;\lambda\|f''\|_{\mathcal{M}}$$8 system. Error rates are $$\min_{f\in \mathrm{BV}}\;\sum_{m=1}^{M}E(f(x_m),y_m)\;+\;\lambda\|f''\|_{\mathcal{M}}$$9 in $E$0 for $E$1 and $E$2 for $E$3 data (Karvonen et al., 2 Mar 2026).

Discontinuous and Tree-based Piecewise-linear Regeneration

$E$4-plane regression employs an EM-style alternating minimization: partitioning with hard or soft cluster assignments (using squared error plus cluster compactness), and regionwise least squares. The method captures both continuous and discontinuous PL structure efficiently, robust to initialization and noise, and consistently matches or outperforms hinge-based and SVR alternatives on benchmark datasets (Manwani et al., 2012).

Regression tree methods introduce PL leaf models with regularization (ridge or LASSO), and node-wise splitting criteria that directly account for predicted future splits. Generalization is quantified via Rademacher complexity bounds, with variable selection producing additional logarithmic and combinatorial overheads. GPU acceleration and batched linear algebra yield 10–100$E$5 speedups in practice (Lefakis et al., 2019).

4. Applications: Dynamical Systems and Neural Surrogates

Continuous-time PL recurrent neural networks (cPLRNNs) provide a paradigm for mechanistically tractable surrogate modeling in dynamical-system recovery and time-series analysis. The piecewise-linear structure in latent ODEs allows for analytic solutions within each activation region, event-driven region switching, and semi-analytic extraction of key topological invariants (fixed points, limit cycles, boundary surfaces). For data with hard thresholding or discontinuities (e.g., leaky integrate-and-fire neuron), piecewise-linear regeneration learns and reproduces threshold timing, limiting cycles, and long-term statistics with high fidelity, outperforming Neural ODEs in both accuracy and computational efficiency (Brändle et al., 17 Feb 2026).

5. Representational Connections: Hat Bases, Box Splines, and Max-affine Expansions

Any CPWL function over a triangulated domain can be regenerated from a local basis of hat functions at triangulation vertices. When the triangulation is regular (lattice grid), hat functions are translates of a canonical linear box spline whose Fourier properties yield sharp norm and stability estimates (Goujon et al., 2022). There is an explicit, finite linear transformation mapping these local representations to global ReLU or max-affine expansions (generalized hinging hyperplanes), providing a structural bridge between local basis and neural net representations.

6. Error Rates, Statistical Properties, and Capacity

Kernel-based superconvergence theorems guarantee that piecewise-linear regeneration via suitably constructed RKHS kernels realizes optimal $E$6 rates for $E$7 in $E$8 and $E$9 in $\|f''\|_{\mathcal{M}}$0 (uniform) norms, matching spline-theoretic predictions (Karvonen et al., 2 Mar 2026). In the context of regression trees and regularized piecewise-linear representations, generalization error is tightly controlled by uniform Rademacher complexity bounds, with explicit dependence on tree depth, leaf-type regularization, and variable selection strategies (Lefakis et al., 2019). Uniqueness/non-uniqueness and minimal-knot certification are determined algorithmically by dual certificates and are central for reproducibility and interpretability in practice (Debarre et al., 2020).

7. Practical Considerations and Hyperparameter Tuning

Piecewise-linear regeneration frameworks enable a direct tradeoff between model sparsity/complexity and data fidelity through a single hyperparameter (the regularization parameter $\|f''\|_{\mathcal{M}}$1). In practice:

• For TV-regularized problems, vary $\|f''\|_{\mathcal{M}}$2 and inspect trade-off curves to select the optimal regime; $\|f''\|_{\mathcal{M}}$3 yields a global affine fit (Debarre et al., 2020).
• In Delaunay-HTV frameworks, cross-validation is effective for tuning $\|f''\|_{\mathcal{M}}$4, controlling the number of affine regions (Pourya et al., 2022).
• Saturation analysis and post-processing algorithms permit extraction of sparsest solutions even in non-unique regimes, ensuring interpretability and parsimonious representations.
• For regression trees and $\|f''\|_{\mathcal{M}}$5-plane regression, communication between clustering and fitting via explicit regularization stabilizes solutions, reduces overfitting, and increases convergence rate (Manwani et al., 2012, Lefakis et al., 2019).

Piecewise-linear regeneration thus unifies a spectrum of recovery, learning, and interpretability techniques for linear and nonlinear models, extending from classical spline theory and interpolation to modern variational learning, neural surrogates, and scalable statistical models.