Structure-Preserving Reconstruction of Convex Lipschitz Functionals on Hilbert Spaces from Finite Samples

Abstract: Convex functionals are ubiquitous in applied analysis, appearing as value functions, risk measures, super-hedging prices, and loss functionals in machine learning. In many applications, however, the functional is only observed through finitely many exact pointwise evaluations. We ask whether a convex functional on a separable Hilbert space $H$ can be reconstructed, up to arbitrary uniform accuracy, by an explicit formula which preserves convexity and Lipschitz regularity and is finitely computable. We answer this affirmatively. For every compact convex $C\subseteq H$, every $L$-Lipschitz convex functional $ρ:C\to\mathbb{R}$, and every $\varepsilon>0$, we construct an explicit finite-sample reconstruction which is convex, $L$-Lipschitz, and uniformly $\varepsilon$-accurate on $C$. The construction uses only finitely many linear measurements $\langle b,\cdot\rangle_H$, with $b$ lying in a finite-dimensional subspace of $H$, and is exactly implementable by a $\operatorname{ReLU}$-MLP. Building on this, we introduce convex neural functionals (CNFs), a structured trainable architecture class containing our reconstruction, whose every admissible parameter configuration is automatically convex and Lipschitz, providing a principled foundation for learning convex functionals from finite data.

• The paper develops a finite max-min reconstruction that remains convex and exactly L-Lipschitz while achieving uniform error below ε on compact convex sets from noiseless samples.
• The method discretizes dual directions rather than primal variables, enabling exact ReLU implementation using finitely many Hilbert-space inner products with complexity governed by effective dimension d.
• The paper introduces convex neural functionals whose nonnegative-weight PReLU and max-pooling architecture certifies convexity and Lipschitzness for every admissible parameter configuration, while experiments provide illustrative numerical validation.

Problem and motivation

Convex functionals on Hilbert spaces arise as value functions, coherent risk measures, super-hedging prices, and loss functionals, yet in practice they are observed only through finitely many exact pointwise evaluations. Existing approximation guarantees fall into three unsatisfactory categories: generic universal approximation results for ReLU networks that do not preserve convexity; extension formulas (e.g., those of Azagra–Mudarra type) that involve suprema over infinitely parameterized linear functionals and are not genuinely closed-form; and finite-dimensional max-affine regression results that do not extend to infinite-dimensional domains. The paper poses the question of whether an explicit reconstruction formula exists that is simultaneously convex for every accuracy level $\varepsilon>0$, uniformly accurate over any compact convex subset $C$ of a separable Hilbert space $H$, $L$-Lipschitz whenever the target is, and exactly implementable by a ReLU-MLP using only finitely many inner products $\langle b,\cdot\rangle_H$ with $b$ in a finite-dimensional subspace. The paper answers this affirmatively with three theorems and a trainable architecture class.

The reconstruction formula

The central result constructs, from an $\varepsilon/(4L)$-net $\{\xi_n\}_{n=1}^N$ of $C$ and an $\eta$-net $C$0 of the ball $C$1 in a finite-dimensional subspace $C$2, the functional

$C$3

which is convex, exactly $C$4-Lipschitz on all of $C$5, and satisfies $C$6. The complexity parameters are explicit: $C$7, the effective linear dimension $C$8, and $C$9 where $H$0. The construction proceeds in three steps. First, a convexified Whitney–McShane extension $H$1, built from the upper McShane extension of the sample data, is shown to interpolate the data while remaining convex and $H$2-Lipschitz; it admits a semi-closed form as a minimum over the simplex $H$3. Second, Sion's minimax theorem dualizes this minimum into a supremum over the dual ball $H$4, which is then discretized: projecting onto $H$5 costs at most $H$6, and replacing the projected ball by its $H$7-net costs at most $H$8. Crucially, discretizing the dual variable preserves convexity because any pointwise maximum of affine functions is convex, whereas discretizing the primal simplex directly would destroy it — the paper explicitly identifies this as the reason a naïve primal discretization fails. Third, covering-number arguments via volume comparison yield the stated bounds on $H$9 and $L$0.

The dependence of $L$1 on dimension through $L$2 means the formula is practical only when the effective linear dimension $L$3 is small relative to the ambient space; the guarantee is uniform over compact convex sets but its cost is governed by metric entropy of $L$4 rather than $L$5.

Neural network representability

The second theorem shows the reconstruction is exactly computable by ReLU networks operating on finitely many Hilbert-space measurements. Writing $L$6, there exist directions $L$7, a convex ReLU-MLP $L$8 computing the outer maximum, and a concave ReLU-MLP $L$9 computing the inner minimum over samples, such that

$\langle b,\cdot\rangle_H$0

The complexity bounds are small and explicit: both networks have size at most $\langle b,\cdot\rangle_H$1, width at most $\langle b,\cdot\rangle_H$2, and depth at most $\langle b,\cdot\rangle_H$3 for $\langle b,\cdot\rangle_H$4 or $\langle b,\cdot\rangle_H$5, obtained from standard constructions realizing minima/maxima of $\langle b,\cdot\rangle_H$6 inputs. Notably, the formula is "in-context": the training data enter only through the concave subnetwork $\langle b,\cdot\rangle_H$7, so the same architecture accommodates arbitrary paired data. If the samples contain an $\langle b,\cdot\rangle_H$8-net of $\langle b,\cdot\rangle_H$9, the pointwise guarantee upgrades to the uniform bound $b$0.

Certifiably convex neural functionals

Because an arbitrary ReLU-MLP configuration need not be convex, the paper introduces convex neural functionals (CNFs): feedforward maps $b$1 whose input layer consists of finitely many affine coordinates $b$2, whose hidden layers apply PReLU activations with slopes in $b$3 to pre-activations computed with nonnegative weight matrices, followed by partition-based max-pooling layers, and whose output layer has nonnegative weights. The certificate theorem states that every admissible CNF parameter configuration yields a convex Lipschitz functional, with the explicit bound

$b$4

and that every reconstruction $b$5 is exactly representable by a depth-two CNF (identity PReLU, single full-partition max-pooling layer). The proof rests on the observation that nonnegative-weight PReLU-max networks are convex and coordinate-wise monotone, composed with an affine input map. The structural consequence is strong: no training algorithm operating within the CNF class can produce a non-convex or non-Lipschitz model, in contrast to unconstrained ReLU-MLPs where convexity holds only for specially constructed configurations. The class is related to input-convex neural networks and monotone networks, but operates on infinite-dimensional inputs via finite linear measurements, placing it among neural functionals.

Numerical validation

Two toy experiments support the theory. In a one-dimensional experiment, a CNF with 81,402 parameters (a ratio of 0.323 to the target's 252,001) fits a randomly initialized two-hidden-layer convex ReLU-MLP target trained on 1000 samples for 200 gradient-descent iterations. A dimensional ablation over $b$6 reports sampled Jensen gaps of order $b$7 to $b$8, with negative values indicating strict satisfaction of Jensen convexity across all sampled test pairs; the single positive value ($b$9 in dimension 1) is attributed to floating-point precision, consistent with the architectural certificate. These are illustrative rather than benchmark-scale results: no comparison against non-convex baselines, generalization guarantees, or statistical analysis is provided, and the paper states explicitly that it addresses expressivity and representation capacity rather than statistical properties.

Limitations and open questions

Several restrictions qualify the results. The uniform guarantee requires compactness of $\varepsilon/(4L)$0 and exact (noiseless) evaluations; noisy observations and non-compact domains are not treated. The complexity bound $\varepsilon/(4L)$1 inherits the curse of dimensionality through the effective linear dimension $\varepsilon/(4L)$2, so the formula's tractability depends on $\varepsilon/(4L)$3 admitting low-dimensional approximations at scale $\varepsilon/(4L)$4 — a structural assumption on the data geometry that the paper does not characterize further. The certificate covers convexity and Lipschitzness but not other regularity (e.g., smoothness), and the CNF class constrains weights to be nonnegative, which may limit optimization dynamics in ways not analyzed here. The paper leaves open the extension to separable Banach spaces (with $\varepsilon/(4L)$5, $\varepsilon/(4L)$6, as targets), applications to learning coherent risk measures and super-hedging prices from finite claim data, and any statistical learning theory for CNFs.

Conclusion

The paper establishes that convex $\varepsilon/(4L)$7-Lipschitz functionals on compact convex subsets of separable Hilbert spaces admit finite-sample reconstructions that are simultaneously convex, exactly $\varepsilon/(4L)$8-Lipschitz, uniformly accurate, and exactly computable by sparse ReLU networks of logarithmic depth, using only finitely many linear measurements. The accompanying CNF architecture provides a certifiable trainable class containing these reconstructions in which convexity and Lipschitzness hold for every admissible parameter configuration. The quantitative bounds make clear where the approach is effective — low effective linear dimension and moderate diameter-to-accuracy ratios — and the extension beyond the Hilbert-space, noiseless, compact setting remains open.