---
title: Input-Convex KANs (ICKANs)
url: https://www.emergentmind.com/topics/input-convex-kans-ickans
type: topic
---

# Input-Convex KANs (ICKANs)

Input-Convex Kolmogorov–Arnold Networks (ICKANs) constitute a recent family of neural network architectures engineered to represent multivariate convex functions, integrating the classical Kolmogorov–Arnold representation with convexity-preserving mechanisms. These models are motivated by applications in scientific computing, mathematical finance, and engineering, where the target functions are known or required to be convex. Modern ICKANs offer both universal approximation guarantees for convex functions and practical advantages in problems such as options pricing, optimal transport, and learning polyconvex constitutive laws in mechanics [2411.12854, 2503.05617, 2505.21208].

## 1. Mathematical Foundations

ICKANs are constructed upon the interplay of convex analysis and the Kolmogorov–Arnold superposition theorem. For a convex function $f:\mathbb{R}^d\to\mathbb{R}$, the supremum-of-affine property specifies that
\[
f(x)=\sup\left\{\langle w,x\rangle + b: (w, b)\in\mathbb{R}^d\times\mathbb{R},~ \langle w, y\rangle + b \leq f(y),~ \forall y\in\mathbb{R}^d \right\}.
\]
This admits the parametric family
\[
f_{n,1}(x) = \max_{1\leq i\leq n} (\langle w_i, x\rangle + b_i),
\]
which is a neural network with a one-layer "max" output activation, inherently convex in input.

Kolmogorov–Arnold representations decompose any continuous multivariate function $f:[a,b]^n\to\mathbb{R}$ as sums of outer univariate functions applied to sums of inner univariate functions:
\[
f(x) = \sum_{i=1}^{2n+1} \psi_i\left( \sum_{j=1}^n \Phi_{i,j}(x_j) \right).
\]
In ICKANs, the $\psi$ are chosen to be convex univariate approximators; the composition and summation are arranged to guarantee that convexity is preserved through the architecture [2505.21208, 2503.05617].

## 2. Architectural Variants

ICKAN architectures fall into three principal types, each enforcing convexity by design:

- **Supremum-of-affine (“max”) networks:** These rely on stacking linear (affine) layers, followed by a convex aggregation (max or LogSumExp). The network
  \[
  f_{n,L}(x) = \phi \left( W_L W_{L-1} \cdots W_1 x + \sum_{k=1}^L W_L \cdots W_{k+1} b_k \right),
  \]
  with $\phi(z) = \max_i z_i$ or the smoothed LogSumExp output, remains convex in $x$ for any linear weights [2411.12854].

- **KAN-based, basis-expansion architectures:** A deep KAN layer takes the form
  \[
  z^{(r)}_i = \sum_{j=1}^{n_{r-1}} \phi_{r-1, i, j}\left( z^{(r-1)}_j \right),
  \]
  where each $\phi_{r-1, i, j}$ is implemented as a trainable convex univariate function via either (a) piecewise-linear (P1) expansions or (b) Hermite cubic splines [2505.21208].

- **Spline-augmented convex layers for physics:** For polyconvex constitutive laws, $W(F)$ is constructed by feeding polyconvex invariants (e.g., in hyperelasticity, $K_1, K_2, K_3$) through a KAN with convex and monotone univariate B-spline activation functions, ensuring physical admissibility [2503.05617].

The table below summarizes key characteristics of the main ICKAN instantiations:

| Variant         | Univariate Basis        | Convexity Control           |
|-----------------|------------------------|-----------------------------|
| Max/LogSumExp   | Identity (affine)      | Aggregation (max, LSE)      |
| P1-ICKAN        | Piecewise-linear (P1)  | Nondecreasing slope params  |
| Cubic-ICKAN     | Hermite cubic splines  | Convexity-band via sigmoids |
| B-spline ICKAN  | Uniform B-splines      | Linear constraints on $c_m$ |

Convexity is enforced by parameterizing the univariate basis expansions to guarantee nondecreasing (or monotone-convex) behavior, as detailed in [2505.21208, 2503.05617].

## 3. Convexity Enforcement and Theoretical Guarantees

For all ICKAN architectures, convexity with respect to input is secured by:
- Ensuring nonnegative increments for slope parameters in piecewise-linear and cubic spline basis functions;
- Linear constraints on B-spline control points, enforcing monotonicity and convexity directly;
- Composition rules for convex and nondecreasing univariate functions, ensuring global convexity under summation and layer-stacking [2505.21208].

Regarding expressivity:
- **Universal approximation:** P1-ICKANs (adaptive piecewise-linear) with sufficient mesh density and layer width are dense in the set of Lipschitz convex functions on compact sets [2505.21208, Theorem 2.1]. Supremum-of-affine ICKANs provide $O(n^{-2/d})$ uniform approximation rates for $C^1$ convex functions on $d$-dimensional cubes and similar rates in $L^r$ norms for Hölder-smooth gradients [2411.12854].
- **Higher-order splines:** Cubic and B-spline ICKANs achieve smoother gradient approximation and improved empirical convergence, although formal universal approximation results remain an open problem for the cubic variant [2505.21208].

## 4. Training Methodologies

- **Loss**: Standard supervised regression objectives, such as mean squared error,
  \[
  \mathcal{L}(\theta) = \frac{1}{N} \sum_{i=1}^N |f_\theta(x^{(i)}) - y^{(i)}|^2,
  \]
  are used for regression tasks. In applications like physics-informed learning, losses are constructed from PDE weak forms and measured reaction forces [2503.05617].

- **Optimizer**: Adam with learning rates $\sim 10^{-3}$ is used commonly. For architectures with parameter redundancy ("scrambling"), multiple (deep) affine layers are stacked to improve optimization by flattening saddle points and accelerating convergence [2411.12854].

- **Regularization**: Convexity is enforced structurally by parameter constraints, making additional explicit regularization optional. Grid adaptation is achieved via trainable mesh points, enhancing local resolution [2505.21208].

- **Symbolic Extraction**: In physical modeling (e.g., hyperelasticity), learned spline functions can be approximated post-training by convex symbolic functions from a restricted library, providing interpretability and closed-form expressions [2503.05617].

## 5. Empirical Benchmarks and Applications

ICKANs have been evaluated in diverse contexts:

- **Options Pricing**: Applied to basket, Bermudan, and swing contracts under Black–Scholes and Ornstein–Uhlenbeck dynamics, ICKANs achieve benchmark-level pricing accuracy (≤0.1% relative error for basket options; ≤1% for Bermudan; ≤0.3% for swing), matching or exceeding state-of-the-art (Deep Optimal Stopping, neural stratification) [2411.12854].

- **Multimodal Convex Regression**: Toy and synthetic benchmarks (1D, multivariate, mixed smooth/non-smooth) show that both P1- and cubic ICKANs reach mean squared errors comparable to or better than classical ICNNs, with smoother gradient approximation for cubic splines [2505.21208].

- **Optimal Transport**: In semi-dual potential parameterizations, cubic-ICKANs achieve unexplained variance percentages on distributional benchmarks (Gaussian mixtures, tensorized maps) matching or outperforming ICNNs, especially in settings with separable structure [2505.21208].

- **Physical Constitutive Modeling**: B-spline-based monotonic ICKANs model polyconvex isotropic hyperelastic energy densities, trained unsupervised from full-field (DIC-style) data and global force measurements. ICKANs achieve $R^2>0.99$ fit to ground-truth invariants and produce interpretable symbolic constitutive relationships [2503.05617].

The table below encapsulates performance domains:

| Application                     | ICKAN Variant         | Key Result                   |
|----------------------------------|-----------------------|------------------------------|
| Options Pricing                  | Max/LSE, scrambled    | ≤0.1–1% rel. error           |
| Convex Regression (1D, multivar) | P1-/Cubic-ICKAN       | MSE $\sim$ICNN, smoother $\nabla$ |
| Optimal Transport                | Cubic-ICKAN           | UVP% $\leq$ ICNN, d ≤ 32     |
| Polyconvex Constitutive Modeling | B-spline ICKAN        | $R^2>0.99$, closed-form $W(F)$ |

## 6. Interpretability, Limitations, and Open Directions

ICKANs provide increased architectural interpretability over dense ICNNs in problems where symbolic post-processing is feasible. The learned univariate convex functions are directly visualizable and can be symbolically regressed to closed-form (e.g., linear, $\exp$, or $\log(1+\exp)$ terms), yielding transparent connections to classical models [2503.05617].

Notable limitations include:
- Computational overhead due to basis expansions and grid adaptation (P1-ICKANs are ∼4.3× slower per epoch than ICNNs on similar benchmarks) [2505.21208].
- Hyperparameter tuning for spline order, mesh density, and architecture depth remains manual and nontrivial.
- Theoretical convergence guarantees are firm only for low-order (P1) variants; higher-order cubic and spline-based variants lack general proofs.
- Extrapolation outside spline-supported input domains relies on linear extension, which may not reflect complex behaviors in extreme regimes.
- Symbolic regression is constrained by the expressive power of the chosen function library.

Extending ICKANs to handle viscoelasticity, plasticity, large-deformation histories, high-dimensional optimal transport, and incorporating UQ (Bayesian ICKANs) are identified as key open research directions [2503.05617, 2505.21208].

## 7. Comparative Perspective and Outlook

ICKANs substantially broaden the toolkit for learning convex functions. They combine the dimension-wise expressivity of Kolmogorov–Arnold decompositions with robust, parameter-manifold-enforced convexity guarantees. Benchmark results indicate that ICKANs are competitive with, and sometimes outperform, traditional ICNNs, particularly in regimes favoring separability or structured input domains. Their ability to yield compact, interpretable, and physically admissible representations recommends them for applications in which convexity and physical structure are both critical [2411.12854, 2503.05617, 2505.21208].

As the field develops, advances in efficient spline evaluation, automated grid adaptation, and theoretical understanding of higher-order convex approximators will likely drive further progress in both theory and applications.

Source: https://www.emergentmind.com/topics/input-convex-kans-ickans