Enhanced Input Convex Neural Networks

• Enhanced ICNNs are neural network architectures that preserve convex output properties for designated inputs while integrating augmented nonconvex feature processing.
• They leverage selective convexity and upstream preprocessing to maintain global optimality in downstream tasks, as demonstrated in battery degradation and MPC applications.
• Key enhancements include solver-aware reformulations, symmetry-aware constructions, and enriched convex function classes that reduce computational overhead and improve performance.

Enhanced Input Convex Neural Network (ICNN) is best understood as an umbrella description—Editor’s term—for ICNN-based models that preserve the defining property of input convexity while augmenting the architecture, feature pipeline, or optimization embedding to better match a downstream decision problem. In the original ICNN formulation, the network is constructed so that a scalar output is convex in all or part of its inputs, allowing prediction or control to be posed as optimization over designated variables rather than as a purely feedforward map (Amos et al., 2016). Later work has extended this idea through partial input convexity, solver-aware reformulations, structured preprocessing, conic enrichments, and symmetry-aware constructions; in many of these papers, “enhanced ICNN” is a fair technical description, but not the authors’ formal name for a new universal architecture (Madahi et al., 2 Apr 2026, Mallick et al., 16 May 2025, Liu et al., 6 May 2026).

1. Conceptual scope and historical position

An ICNN is a scalar-valued neural network whose output is convex with respect to specified inputs. The foundational distinction is between the fully input-convex neural network (FICNN), in which the output is convex in the entire input, and the partially input-convex neural network (PICNN), in which convexity is enforced only in a designated subset of variables while the remaining inputs act as unconstrained conditioning variables (Amos et al., 2016). This distinction became central in later application work because many optimization problems require convexity only in the decision variables, not in exogenous context.

The term “enhanced ICNN” usually refers not to a single canonical successor of the original ICNN, but to a family of structured variants that enlarge the usable hypothesis class without sacrificing optimization compatibility where it matters. In battery degradation modeling, for example, the crucial enhancement is the use of a PICNN that is convex in charging rate while allowing nonconvex dependence on temperature and time (Mallick et al., 16 May 2025). In implicit balancing for battery-based model predictive control (MPC), the enhancement consists of a partial ICNN combined with trainable embedding and attention-based gating over exogenous market inputs, while keeping the core map convex and monotone in the battery action (Madahi et al., 2 Apr 2026).

A recurring point across the literature is that convexity is not an end in itself. It is imposed because the learned model is intended to sit inside another optimization problem—structured prediction, MPC, recourse approximation, security screening, or constitutive modeling—and a generic neural network would typically turn that downstream problem into a nonconvex MINLP or otherwise destroy global-solution guarantees. Enhanced ICNNs therefore occupy the interface between representation learning and optimization-aware modeling.

2. Canonical ICNN mechanics

The baseline FICNN uses the recurrence

$$z_{i+1}=g_i\!\left(W_i^{(z)} z_i + W_i^{(y)} y + b_i\right), \qquad f(y;\theta)=z_k,$$

with $W_0^{(z)}\equiv 0$. Convexity in $y$ is guaranteed when the hidden-to-hidden matrices $W_i^{(z)}$ are elementwise nonnegative and the activations $g_i$ are convex and nondecreasing (Amos et al., 2016). Passthrough connections from the input to every hidden layer are not incidental; they compensate for the expressive restrictions created by the nonnegativity constraint on the recurrent hidden path.

The PICNN generalizes this construction by splitting the network into a nonconvex conditioning stream and a convex decision stream. In one formulation used for battery degradation, the nonconvex variables are $x=\{T,t_{\text{temp}}\}$ and the convex input is $y=u$, the charging profile. The architecture takes the form

$u_{i+1}=\tilde{g}_i(\tilde{W}_i u_i+\tilde{b}_i),$

$$z_{i+1}=g_i\!\left(W_i^{(z)}(z_i\odot u_i^{(z)})+W_i^{(y)}(y\odot u_i^{(y)})+u_i^{(u)}\right),$$

with convexity in $y$ preserved by elementwise nonnegativity of $W_0^{(z)}\equiv 0$0 together with nondecreasing activations (Mallick et al., 16 May 2025). The mathematical role of the $W_0^{(z)}\equiv 0$1-stream is to modulate the convex $W_0^{(z)}\equiv 0$2-path without forcing the entire map to be jointly convex.

A second foundational point is that inference in an ICNN is ordinarily an optimization problem. In the original formulation, one predicts by solving

$W_0^{(z)}\equiv 0$3

or, over bounded domains, an entropy-regularized variant that is amenable to the bundle entropy method (Amos et al., 2016). That inference-by-optimization viewpoint remains the core reason ICNNs matter in later solver-integrated applications.

3. Major enhancement patterns

The first major enhancement pattern is selective convexity. Rather than enforcing convexity in all inputs, modern ICNN deployments usually isolate the true optimization variable and leave the rest unconstrained. This is explicit in PICNN-based battery degradation, where the model is convex in charging rate but nonconvex in temperature and time, and in implicit balancing, where the model is convex and monotone in battery power while conditioning on a rich exogenous market state (Mallick et al., 16 May 2025, Madahi et al., 2 Apr 2026). The practical consequence is a better expressivity–tractability trade-off than a fully convex surrogate would provide.

The second pattern is learned preprocessing outside the convex path. In implicit balancing MPC, the paper augments a partial ICNN with quarter-hour embedding and attention-based top-$W_0^{(z)}\equiv 0$4 bid selection over aFRR and mFRR merit-order data. The key design choice is that this preprocessing depends only on exogenous inputs, so it need not be converted into an MILP and does not disturb convexity in the decision variable (Madahi et al., 2 Apr 2026). This suggests a general recipe: place complex representation learning upstream of the convex core, provided it is constant with respect to the optimization variables at solve time.

The third pattern is architectural enrichment of the convex function class. In distribution-network OPF, one enhancement is signed-input augmentation: instead of feeding $W_0^{(z)}\equiv 0$5 directly, the ICNN receives $W_0^{(z)}\equiv 0$6, which the paper argues strictly improves approximation capability while preserving convexity (Cheng et al., 2024). In SOC-ICNNs, the enhancement is more structural: $W_0^{(z)}\equiv 0$7 with $W_0^{(z)}\equiv 0$8 and $W_0^{(z)}\equiv 0$9. This enriches the usual polyhedral ReLU geometry with quadratic and second-order-cone components and yields an exact SOCP value-function representation (Liu et al., 6 May 2026).

The fourth pattern is symmetry-aware and invariance-aware convex design. In nonlinear model reduction, a convex-inspired decoder is combined with an exact odd-symmetry construction,

$y$0

so that $y$1 and $y$2 by construction (Huang et al., 23 Nov 2025). In anisotropic plasticity, permutation-invariant ICNNs in principal stress space are used to encode isotropic convex structure while anisotropy is introduced through linear stress transformations; this reduces overfitting under sparse experimental supervision and improves generalization relative to pure 6D stress-space ICNNs (Jadoon et al., 21 Aug 2025).

4. Optimization embeddings and solver formulations

A central theme in enhanced ICNN research is that the architecture is chosen jointly with its optimization embedding. For ReLU-ICNNs, one important clarification is that generic Big-$y$3 graph encodings are not always necessary. For a ReLU-ICNN surrogate

$y$4

the paper on stability-constrained power-system optimization shows that the convex sublevel constraint $y$5 admits the exact LP value-function representation

$y$6

subject to

$y$7

thereby eliminating ICNN-induced binaries for the sublevel convention (Xu et al., 24 Jun 2026). The same paper also shows that reversing the inequality to a superlevel constraint generally destroys convexity and invalidates the global logic of cut-based outer approximation.

This LP-representability has immediate consequences for solver design. In ICNN-enhanced two-stage stochastic programming, the recourse surrogate is embedded as

$y$8

subject to the first-stage constraints and the linear ICNN epigraph inequalities

$y$9

Because the surrogate contributes no activation binaries, the paper reports a computationally more efficient alternative to MIP-based embeddings while preserving exactness of the ICNN inference layer (Liu et al., 8 May 2025).

In MPC, the embedding strategy depends on the activation and the objective. For explicit ML-MPC with ICNN process models, convexity of the stage cost is obtained only after a controller-oriented modification: the ICNN predicts nonnegative state magnitudes, the state-weighting matrix is diagonal with positive diagonal entries, and the last activation enforces nonnegative outputs. Under those conditions, the quadratic stage cost composed with the ICNN predictor is convex, and candidate evaluation inside the explicit MPC pipeline becomes a convex MIQP rather than a generic nonconvex problem (Wang et al., 2024). In implicit balancing MPC, a monotone partial ICNN lets the market-price surrogate be evaluated separately for charging and discharging directions and embedded as an MIQP/MILP-compatible model that remains globally solvable by branch-and-bound rather than by a generic nonconvex MINLP formulation (Madahi et al., 2 Apr 2026).

5. Application domains and empirical behavior

Enhanced ICNNs have been deployed wherever a learned surrogate must remain optimization-friendly. In Belgian implicit balancing, a partial ICNN with embedding and attention-based bid gating improved imbalance revenue by 35.4% up to 301.8% depending on battery size under noisy system-imbalance forecasts, while reducing average optimization time from 60.59 ms for the benchmark clearing approximation to 30.3 ms for the proposed model (Madahi et al., 2 Apr 2026). A noteworthy behavioral result in that study is that lower raw prediction error was not always the decisive factor; the ICNN-based controller often made better decisions because it more frequently selected idle action when gains were uncertain.

In vehicle-to-grid scheduling, a PICNN degradation surrogate trained on the NASA Ames randomized battery usage dataset achieved $W_i^{(z)}$0 on the held-out cell RW10 while preserving convexity in charging rate, which in turn made the second-stage charging problem convex and amenable to projected gradient descent with global optimality guarantees under the paper’s formulation (Mallick et al., 16 May 2025). In battery ERM optimization for PV smoothing and revenue maximization, a relaxed epigraph ICNN surrogate produced predicted and actual performance that matched on the reported test cases, in contrast to the constant-efficiency linear model whose nominal objective could look attractive but whose realized plant behavior degraded because SoC saturation was mispredicted (Omidi et al., 2024).

In power-system security applications, an ICNN classifier with reliability-enforcing scaling achieved zero false negative rate and 10–20x runtime speedup for $W_i^{(z)}$1 contingency screening on the IEEE 39-bus system, with a reported false positive rate of about 2% to 5% (Christianson et al., 2024). In two-stage stochastic programming, ICNN-enhanced 2SP maintained validation accuracy comparable to MIP-embedded ReLU surrogates while producing speedups of up to 100x on the most challenging instances (Liu et al., 8 May 2025). These studies illustrate a common pattern: enhanced ICNNs are especially attractive when exact or near-exact optimization embedding matters as much as predictive fidelity.

6. Limitations, misconceptions, and adjacent developments

A common misconception is that every “enhanced ICNN” paper introduces a fundamentally new convexity theorem or a new universal ICNN class. In several representative cases, the actual contribution is narrower and more application-structured: a PICNN with a carefully chosen input split, an upstream attention or embedding module, a symmetry construction, or a solver-level reformulation (Madahi et al., 2 Apr 2026, Mallick et al., 16 May 2025). The enhancement is often real, but it is frequently a refinement of deployment rather than a wholesale replacement of the original ICNN architecture.

A second misconception is that ICNN sign constraints characterize all convex ReLU networks. They do not. The paper “Convexity in ReLU Neural Networks: beyond ICNNs?” shows that every convex function implemented by a one-hidden-layer ReLU network can be expressed by an ICNN with the same architecture, but that this equivalence breaks once depth exceeds one hidden layer; beyond that point, ICNNs form only a strict subset of convex ReLU networks (Gagneux et al., 6 Jan 2025). This indicates that convex neural modeling “beyond ICNNs” remains an active theoretical frontier.

There are also adjacent architectures that expand the optimization-friendly design space without remaining standard ICNNs. CDiNN represents functions as differences of polyhedral convex functions and optimizes them by difference-of-convex procedures, while Input Convex Gradient Networks model convex gradients through integrated Jacobian-vector products rather than by differentiating a scalar ICNN (Sankaranarayanan et al., 2021, Richter-Powell et al., 2021). These lines suggest that enhanced ICNN research naturally shades into broader convex-structured neural modeling.

Current open directions are largely problem-driven. Implicit balancing work suggests adding longer history and more grid-related inputs, and even moving from quarter-hour to minute-based MPC if the convex surrogate remains lightweight (Madahi et al., 2 Apr 2026). SOC-ICNN theory identifies structurally degenerate inputs as the regime where clean local second-order formulas break down, leaving generalized second-order geometry as an open issue (Liu et al., 6 May 2026). In anisotropic plasticity, PI-ICNN results point toward coupled hardening and microstructure-informed constitutive design under sparse data (Jadoon et al., 21 Aug 2025). Across these developments, the central organizing principle remains unchanged: preserve convexity exactly where optimization needs it, and spend the remaining modeling freedom as deliberately as possible.