Spline Flows: Nonlinear Bijective Maps

• Spline flows are normalizing flows that use spline-based monotonic transformations to create expressive, piecewise-smooth bijections for density estimation.
• They enable efficient analytic inversion and Jacobian computation, making them ideal for coupling and autoregressive architectures in high-dimensional applications.
• Leveraging formulations like rational-quadratic, cubic, and B-spline flows, these models achieve state-of-the-art performance in generative modeling, dynamical systems, and scientific inference.

A spline flow is a normalizing flow whose invertible transformations are parameterized by spline-based monotonic maps. Spline flows generalize affine and polynomial transformations with the expressive power of nonlinear, piecewise-smooth bijections, while retaining analyticity and efficient Jacobian computation critical for density estimation, generative modeling, and scientific inference. Modern spline flows include monotonic cubic splines, rational-quadratic splines, linear rational splines, and non-uniform B-spline transformations. These models have enabled state-of-the-art flexibility in coupling and autoregressive flow architectures across density modeling, continuous-time generative models, and dynamical system learning.

1. Mathematical Formulations of Spline-based Normalizing Flows

Spline flows construct elementwise invertible maps $f: \mathbb{R}^d \to \mathbb{R}^d$ using monotonic spline basis functions. Parameterizations differ by spline order and explicit form:

• Rational-Quadratic Spline Flows: The central map within each interval $[x^{(k)}, x^{(k+1)}]$ is a monotonic rational-quadratic function specified by knot values, bin widths, and derivatives at each knot. Given $K$ bins, knot coordinates $(x^{(k)}, y^{(k)})_{k=0}^K$, and derivatives $\delta^{(k)}$, the forward transform on bin $k$ is

$$g(x) = y^{(k)} + (y^{(k+1)} - y^{(k)}) \frac{s^{(k)} \xi^2 + \delta^{(k)} \xi (1-\xi)}{s^{(k)} + (\delta^{(k+1)} + \delta^{(k)} - 2s^{(k)}) \xi (1-\xi)},$$

where $\xi = \frac{x - x^{(k)}}{w^{(k)}}$, $s^{(k)} = (y^{(k+1)}-y^{(k)})/w^{(k)}$, $w^{(k)} = x^{(k+1)}-x^{(k)}$ (Durkan et al., 2019, Shih et al., 2022, Pina-Otey et al., 2020).

• Cubic Spline Flows: Each bin utilizes a monotonic cubic polynomial, parameterized by knot values, bin derivatives, and widths. The forward map within $[x^{(k)}, x^{(k+1)}]$0 is

$[x^{(k)}, x^{(k+1)}]$1

with coefficients fixed by bin values and derivatives (Durkan et al., 2019).

• Linear Rational Splines: Each bin is split by an internal knot and interpolates via two rational functions, constructed to match endpoint values and derivatives. Forward and inverse are analytic and involve only homographic inversions, avoiding polynomial root-finding (Dolatabadi et al., 2020).
• B-spline Flows: The mapping employs a linear combination of non-uniform B-spline basis functions. For order-$[x^{(k)}, x^{(k+1)}]$2 B-splines, the map

$[x^{(k)}, x^{(k+1)}]$3

with (learned or neural-)parametrized coefficients $[x^{(k)}, x^{(k+1)}]$4 and knots $[x^{(k)}, x^{(k+1)}]$5; outside $[x^{(k)}, x^{(k+1)}]$6 the map is identity. For cubic B-splines ($[x^{(k)}, x^{(k+1)}]$7), analytic inversion is available via a root-finding step in a single bin and ensures $[x^{(k)}, x^{(k+1)}]$8 smoothness (Hong et al., 2023).

All spline flows ensure monotonicity via parameter constraints or monotonic network outputs; for example, softmax-normalized bin widths, positive softplus derivatives, and Steffen’s or other monotonicity-enforcing schemes.

2. Architectures and Neural Parameterization of Spline Flows

Spline flows are employed via coupling or autoregressive architectures, where spline parameters are predicted by neural networks conditioned on passive variables:

• Coupling Layers: The input vector is partitioned; parameters for spline maps on one subset are produced by an MLP conditioned on the remainder. Efficient per-bin parameterization and LU-decomposed global linear layers are used to maximize expressivity while preserving tractable Jacobians (Durkan et al., 2019, Durkan et al., 2019, Dolatabadi et al., 2020, Hong et al., 2023).
• Autoregressive Layers: Each variable is sequentially transformed by a spline map, with parameters dependent on previous variables via masked neural networks (e.g., MADE/ResMADE, LSTM). This allows full autoregressive density estimation with analytic invertibility (Durkan et al., 2019, Pina-Otey et al., 2020).
• Parameter Constraints: Spline width and height logits are mapped through softmax to guarantee positive widths and strictly increasing knots; derivatives at knots are enforced to be positive via softplus or minimum-slope rules, securing invertibility.
• Jacobian and Hessian Computation: The log-Jacobian determinant in all spline flow variants reduces to local computations over bins, enabling fast per-sample log-likelihood and gradient calculation. Cubic B-spline and rational-quadratic flows provide analytic first and second derivatives, essential for force-matching and physics applications (Hong et al., 2023).

3. Theoretical Properties and Regularity

Spline flows allow the construction of invertible transformations that balance expressiveness and regularity:

• Smoothness:
  • Rational-quadratic and cubic-spline flows are $[x^{(k)}, x^{(k+1)}]$9; cubic B-spline flows can enforce up to $K$0 or higher regularity, dictated by spline order (Hong et al., 2023).
  • Smooth normalizing flows can reach $K$1 regularity via bump functions but lack analytic inverse, making them computationally expensive for density calculation and sampling (Hong et al., 2023).
• Universal Approximation: Spline flows with sufficiently high bin count and flexible neural parameterizations are universal density approximators on compact domains. Autoregressive spline flows are universal for conditional densities, as shown for piecewise-linear Sig-Splines through a constructive approximation theorem (Wiese et al., 2023).
• Monotonicity & Bi-Lipschitz Bounds: Monotonicity is guaranteed by constrained parameter outputs. Bi-Lipschitz continuity is achieved by lower and upper bounding derivatives, ensuring invertibility and numerical stability even in deep compositions (Hong et al., 2023).

4. Extensions: B-Spline Flows, Flow Matching, and Geometry

• Non-uniform B-Spline Flows: By replacing uniform splines with non-uniform B-splines (variable knot placement), one can achieve higher-order smoothness and analytic inversion for $K$2. Analytic formulas for the Jacobian and Hessian support differentiable physics applications (e.g., force-matching in Boltzmann generators). Non-uniform B-spline flows are uniquely suited for tasks requiring $K$3 regularity and efficient derivatives, unattainable by rational-quadratic ($K$4) or bump-function flows (slow, non-analytic inverse) (Hong et al., 2023).
• SplineFlow for Dynamical Systems: SplineFlow integrates B-spline interpolation with the flow-matching paradigm for learning continuous normalizing flows in time. By fitting B-spline curves to time-stamped observations and defining conditional Gaussian bridges, SplineFlow constructs velocity targets with superior approximation rate and smoothness compared to linear-path baselines, providing robustness against irregular sampling and highly nonlinear dynamics (Rathod et al., 30 Jan 2026).
• Adaptive Lagrangian B-Spline Flows for Manifold Evolution: Localized, overlapping B-spline patches evolve point clouds on dynamic manifolds by updating control points under geometric flows (e.g., mean curvature flow), with analytic curvature estimation and adaptive refinement via knot insertion and point redistribution. This approach supports meshless, high-order geometric PDE evolution, robust to anisotropy and field coupling (Ammad et al., 16 Jan 2026).

5. Empirical Performance, Applications, and Benchmarks

Performance gains and application domains of spline flows are well supported by comprehensive experiments:

• Tabular and Image Density Estimation: Rational-quadratic and cubic-spline flows outperform or match prior coupling and autoregressive models on UCI, BSDS300, MNIST, CIFAR-10, and ImageNet across tabular and high-dimensional data. Spline flows achieve state-of-the-art density estimation with minimal computational overhead and significantly fewer parameters compared to prior architectures (Durkan et al., 2019, Durkan et al., 2019, Dolatabadi et al., 2020).
• Physics and Scientific Inference: Non-uniform B-spline flows achieve stable molecular dynamics force recovery, outperforming rational-quadratic spline flows (unstable due to lack of $K$5 regularity) and sampling dramatically faster than smooth bump-based alternatives. Empirical metrics show lower negative log-likelihood (NLL), more accurate force-matching error (FME), and better Kullback–Leibler divergence (KLD) for B-spline and smooth flows compared to rational-quadratic (Hong et al., 2023).
• Time Series and Dynamical Systems: SplineFlow improves mean-squared error, Wasserstein, MMD, and energy metrics over linear-path flow-matching baselines in ODE/SDE modeling and single-cell dynamics. B-spline interpolants in SplineFlow yield superior fit for nonlinear and oscillatory regimes, especially with irregular sampling (Rathod et al., 30 Jan 2026).
• Speech and Sequence Modeling: In speech attribute modeling (e.g., $K$6 trajectories), rational-quadratic spline flows enable expressive generative modeling, outperforming affine flows and matching or exceeding deterministic baselines in statistical moment matching and subjective quality in TTS (Shih et al., 2022).
• Likelihood-Free Inference: For tasks such as Bayesian neutrino oscillation parameter inference, spline flows (NSF) provide bias-reduced posterior estimation with exact, unbinned likelihoods—surpassing binned histogram methods and enabling efficient scientific computation (Pina-Otey et al., 2020).

6. Implementation: Best Practices and Limitations

Practical deployment of spline flows draws on the following guidelines:

• Parameterization: Use softmax widths/heights, softplus derivatives for monotonicity. For non-uniform B-splines, neural networks produce unconstrained increments mapped to positive slots and cumulative knots (Hong et al., 2023).
• Number of Bins: Default 8–30; higher for high-dimensional or multimodal targets. B-spline flow order $K$7 is essential for analytic inversion (Durkan et al., 2019, Hong et al., 2023).
• Domain Handling: Rational-quadratic and cubic spline flows act on bounded intervals, requiring appropriate scaling or pre-processing. Out-of-range data is mapped via linear tails or affine transform (Shih et al., 2022).
• Computational Complexity: Bin location and per-sample transformation cost is $K$8 for bin search and $K$9 for analytic transform/Jacobian evaluation per dimension.
• Limitations: Bounded support in spline flows may necessitate input scaling or channel permutations; lack of $(x^{(k)}, y^{(k)})_{k=0}^K$0 regularity in RQ, cubic, and linear rational splines preclude some physics applications.
• Reproducibility: Open-source implementations, fixed seeds, and comprehensive benchmark scripts are provided for many recent methods (Hong et al., 2023, Rathod et al., 30 Jan 2026).

7. Comparative Landscape, Advances, and Theoretical Insights

Recent research highlights the trade-offs among spline flow types:

Flow Type	Smoothness	Analytic Inverse	Key Use Cases
Rational-Quadratic Spline	$(x^{(k)}, y^{(k)})_{k=0}^K$1	Yes (quadratic)	Density modeling, image gen., general ML (Durkan et al., 2019)
Cubic-Spline	$(x^{(k)}, y^{(k)})_{k=0}^K$2	Yes (cubic)	Density modeling; closed-form inversion (Durkan et al., 2019)
Linear Rational Spline	$(x^{(k)}, y^{(k)})_{k=0}^K$3	Yes (homographic)	Mixed expressiveness/cost; analytic invert (Dolatabadi et al., 2020)
Non-uniform B-Spline	$(x^{(k)}, y^{(k)})_{k=0}^K$4	Yes ($(x^{(k)}, y^{(k)})_{k=0}^K$5)	Force-matching, physics, high-regularity flows (Hong et al., 2023)
Smooth Bump Flow	$(x^{(k)}, y^{(k)})_{k=0}^K$6	No (root-find)	Maximal smoothness, slow sampling

Spline-based architectures thus provide a spectrum from tractable, moderately flexible low-order splines (linear, quadratic) through highly expressive, efficiently invertible rational and cubic splines, up to smooth and adaptive B-spline flows supporting physics-driven computation and dynamical systems modeling.

Spline flows will likely remain a core primitive in future generative models, continuous-time flows, and scientific machine learning, as their analytic structure and differentiability ideally suit the computational requirements of modern probabilistic modeling (Hong et al., 2023, Rathod et al., 30 Jan 2026, Durkan et al., 2019).