Permutation-Invariant ICNN Architectures

• Permutation-Invariant ICNNs are architectures that integrate symmetric input aggregation with ICNN convexity constraints to model physically consistent scalar functions.
• They employ mechanisms like Deep Sets–style pooling and nonnegative weight pathways to enforce both permutation invariance and convexity in principal variables.
• Applications span hyperelastic constitutive modeling and anisotropic plasticity, demonstrating improved generalization and regularization under sparse data.

Permutation-Invariant Input Convex Neural Network (PI-ICNN) denotes a family of architectures that combine two structural properties: exact invariance under permutations of designated inputs and convexity enforced by the design principles of input-convex neural networks (ICNNs). In current arXiv usage, the term is descriptive rather than fully standardized. It is used most naturally for architectures that learn convex symmetric functions of principal variables in constitutive modeling, such as principal stretches or principal stresses, and it also appears in a distinct optimization-oriented sense for ICNNs whose positivity-constrained weights are emitted by a permutation-invariant hypernetwork conditioned on a batch summary (Tepole et al., 1 Mar 2025, Jadoon et al., 21 Aug 2025, Siahkoohi et al., 22 May 2026). Across these variants, the common theme is that permutation symmetry is built into the architecture rather than learned from data, while convexity is enforced through nonnegative hidden pathways and convex, nondecreasing nonlinearities in the ICNN sense established by Amos, Xu, and Kolter (Amos et al., 2016).

1. Definitional scope and historical emergence

PI-ICNN is best understood as a structured architectural pattern rather than a single canonical model class. The defining ingredients are: a scalar-valued network or scalar potential, a permutation-invariant treatment of some input collection or spectral tuple, and ICNN-style constraints that guarantee convexity in the variables of interest. In the current literature, the most concrete realizations appear in constitutive modeling, where isotropy requires symmetry with respect to permutations of principal values and thermodynamic or variational admissibility requires convexity or polyconvexity (Tepole et al., 1 Mar 2025, Jadoon et al., 21 Aug 2025).

The term is not always the name used by the original authors. In hyperelasticity, the architecture introduced as $\lambda$-PANN is, in effect, a PI-ICNN in principal stretches: it combines permutation-invariant deep-set aggregation over spectral inputs with ICNN blocks to produce a strain-energy density that is symmetric in the stretches and polyconvex by construction (Tepole et al., 1 Mar 2025). In anisotropic plasticity, the term PI-ICNN is used explicitly for convex, permutation-invariant neural yield functions in transformed principal stress space (Jadoon et al., 21 Aug 2025). By contrast, a 2026 training paper uses permutation invariance in a different location—inside a batch-conditioned hypernetwork that emits nonnegative ICNN weights—so the “PI” refers to the weight-generation mechanism rather than to symmetry of the modeled physical variables (Siahkoohi et al., 22 May 2026).

Formulation	Permutation domain	Status relative to PI-ICNN
$\lambda$-PANN for hyperelasticity	Principal stretches and cofactor-spectrum tuples	Effective PI-ICNN (Tepole et al., 1 Mar 2025)
PI-ICNN for anisotropic plasticity	Transformed principal deviatoric stresses	Explicit PI-ICNN (Jadoon et al., 21 Aug 2025)
Lifted ICNN training	Samples in the conditioning batch	PI mechanism in training, not in input coordinates (Siahkoohi et al., 22 May 2026)

This breadth of usage matters. A sentence such as “the model is permutation-invariant” is incomplete unless it specifies what is being permuted: principal values, set elements, graph nodes, latent coordinates, or batch samples. The recent literature uses all of these symmetry notions, but only some correspond to PI-ICNN in the strict set-function or spectral-symmetry sense.

2. ICNN foundations and the convexity mechanism

The convexity side of PI-ICNNs comes directly from ICNN theory. In the fully input-convex architecture of Amos, Xu, and Kolter, a $k$-layer network is written

$$z_{i+1}=g_i\!\left(W_i^{(z)}z_i+W_i^{(y)}y+b_i\right), \qquad f(y;\theta)=z_k,$$

with $z_0,W_0^{(z)}\equiv 0$. Convexity in $y$ is guaranteed if the hidden-to-hidden matrices $W_{1:k-1}^{(z)}$ are elementwise nonnegative and each $g_i$ is convex and non-decreasing (Amos et al., 2016). The same principle appears in later ICNN formulations, including the scalar ICNN blocks used in constitutive models (Jadoon et al., 21 Aug 2025).

The logic is compositional. Affine dependence on the convex input is always convex. A nonnegative linear combination of convex hidden features remains convex. Composition with a convex, nondecreasing activation preserves convexity. This is why ICNNs restrict the recurrent hidden pathway but leave direct affine input-to-hidden shortcuts unconstrained (Amos et al., 2016). In partially input-convex architectures, only a designated subset of inputs is treated convexly; the remaining inputs can be processed through an unconstrained conditioning path while convexity in the target variables is preserved (Amos et al., 2016).

In PI-ICNN practice, this convexity machinery is often specialized to scalar-input scalar-output modules. The plasticity PI-ICNN uses $\mathcal{N}_C$, a scalar ICNN, together with $\mathcal{N}_{MC}$, a scalar ICNN additionally constrained to be monotonically non-decreasing; the latter is obtained by extending nonnegativity constraints to the first layer as well (Jadoon et al., 21 Aug 2025). In hyperelasticity, front-end scalar ICNNs are nested inside permutation-invariant aggregators, and a final ICNN combines the aggregated convex features with determinant-based inputs (Tepole et al., 1 Mar 2025).

A persistent technical consequence of ICNN convexity constraints is that optimization and initialization differ from ordinary MLPs. Nonnegative hidden weights break the centered-weight assumptions used in Xavier or He analyses. A principled initialization theory for ICNNs therefore replaces variance-only matching by joint control of mean, variance, and correlation, and the resulting initialization can make ICNNs trainable even without skip-connections (Hoedt et al., 2023). This suggests direct relevance to PI-ICNNs whenever their convex blocks rely on the same nonnegative-weight mechanism.

3. Architectural mechanisms for permutation invariance

Permutation invariance in PI-ICNNs is enforced by symmetric aggregation, not by data augmentation. The most developed mechanism in the constitutive papers is a Deep Sets–style or Hölder-power aggregation over scalar principal values. In the hyperelastic construction, for three scalar inputs $\lambda$0, the invariant module is

$\lambda$1

where $\lambda$2 is an ICNN applied elementwise and $\lambda$3 is a convex scalar function constrained to be monotonically increasing (Tepole et al., 1 Mar 2025). The sum makes the representation permutation invariant; the monotone-convex outer map preserves convexity of the composed spectral function.

The anisotropic plasticity PI-ICNN uses the same basic pattern in a more explicit notation. For one transformed principal-stress triple $\lambda$4, the network is

$\lambda$5

and the two-transform variant sums two such pooled branches with separate trainable exponents $\lambda$6 and $\lambda$7 before the monotone outer ICNN (Jadoon et al., 21 Aug 2025). The symmetry is exact with respect to permutations of each transformed principal-stress tuple.

A distinct use of permutation invariance appears in lifted ICNN training. There the constrained weight vector is generated by

$\lambda$8

so the hypernetwork conditions on a permutation-invariant batch summary (Siahkoohi et al., 22 May 2026). The ICNN itself is still an ordinary map $\lambda$9; the permutation symmetry concerns the order of samples in the conditioning batch, not the coordinates of the modeled input.

This separation between convexity and invariance is fundamental. Convexity is a property of the scalar function with respect to designated variables. Permutation invariance is a property under a group action. PI-ICNN architectures impose both simultaneously, but the two constraints operate through different mechanisms: one via nonnegative hidden pathways and monotone-convex composition, the other via symmetric pooling or spectral symmetrization.

4. Principal-stretch PI-ICNNs for hyperelastic constitutive modeling

The hyperelastic formulation in principal stretches is the clearest example of a PI-ICNN that is both physically motivated and mathematically structured (Tepole et al., 1 Mar 2025). The problem is to learn a stored-energy function $k$0 for isotropic hyperelastic soft materials such that stresses follow by differentiation, objectivity and isotropy are respected, and a generalized convexity condition ensures well-posedness. The construction works in the principal stretches $k$1, i.e. the eigenvalues of the right stretch tensor $k$2 in the polar decomposition

$k$3

The key representation-theoretic fact is that polyconvex isotropic energies can be expressed as convex functions of the principal stretches, the pairwise products $k$4, and $k$5. Since the eigenvalues of $k$6 are exactly $k$7, the network needs only to model convex symmetric functions on two unordered 3-tuples,

$k$8

plus a convex dependence on $k$9 (Tepole et al., 1 Mar 2025).

The learned energy is built from three subnetworks. Two front-end modules produce convex permutation-invariant scalars

$$z_{i+1}=g_i\!\left(W_i^{(z)}z_i+W_i^{(y)}y+b_i\right), \qquad f(y;\theta)=z_k,$$0

$$z_{i+1}=g_i\!\left(W_i^{(z)}z_i+W_i^{(y)}y+b_i\right), \qquad f(y;\theta)=z_k,$$1

followed by a final ICNN

$$z_{i+1}=g_i\!\left(W_i^{(z)}z_i+W_i^{(y)}y+b_i\right), \qquad f(y;\theta)=z_k,$$2

The inclusion of both $$z_{i+1}=g_i\!\left(W_i^{(z)}z_i+W_i^{(y)}y+b_i\right), \qquad f(y;\theta)=z_k,$$3 and $$z_{i+1}=g_i\!\left(W_i^{(z)}z_i+W_i^{(y)}y+b_i\right), \qquad f(y;\theta)=z_k,$$4 is explicit: $$z_{i+1}=g_i\!\left(W_i^{(z)}z_i+W_i^{(y)}y+b_i\right), \qquad f(y;\theta)=z_k,$$5 is needed to allow negative stress responses while preserving polyconvexity, because it is an affine scalar input to the final convex network (Tepole et al., 1 Mar 2025).

The full strain-energy density adds reference-state normalization and a growth term,

$$z_{i+1}=g_i\!\left(W_i^{(z)}z_i+W_i^{(y)}y+b_i\right), \qquad f(y;\theta)=z_k,$$6

where the constants $$z_{i+1}=g_i\!\left(W_i^{(z)}z_i+W_i^{(y)}y+b_i\right), \qquad f(y;\theta)=z_k,$$7 are chosen so that the reference state is stress free. Stresses are obtained by automatic differentiation: $$z_{i+1}=g_i\!\left(W_i^{(z)}z_i+W_i^{(y)}y+b_i\right), \qquad f(y;\theta)=z_k,$$8 This enforces polyconvexity, objectivity, isotropy, vanishing energy at the reference state, and zero stress at $$z_{i+1}=g_i\!\left(W_i^{(z)}z_i+W_i^{(y)}y+b_i\right), \qquad f(y;\theta)=z_k,$$9 by construction (Tepole et al., 1 Mar 2025).

The reported implementation uses two layers with 10 neurons and Softplus activation in each of the three subnetworks, for a total of 1285 parameters. Training uses Adam with learning rate $z_0,W_0^{(z)}\equiv 0$0, typically for $z_0,W_0^{(z)}\equiv 0$1k epochs; on experimental datasets an $z_0,W_0^{(z)}\equiv 0$2-regularization factor $z_0,W_0^{(z)}\equiv 0$3 is added to induce sparsity and avoid overfitting (Tepole et al., 1 Mar 2025). On synthetic Ogden data, the model trains on 200 points and tests on 500 extrapolation points, achieves training losses around $z_0,W_0^{(z)}\equiv 0$4, and performs best at Hölder exponent $z_0,W_0^{(z)}\equiv 0$5. On synthetic generalized-Ogden invariant data, $z_0,W_0^{(z)}\equiv 0$6-PANN with $z_0,W_0^{(z)}\equiv 0$7 shows lower extrapolation loss. On Treloar and Heuillet rubber data, trained on uniaxial and equibiaxial tests and validated on pure shear, it attains $z_0,W_0^{(z)}\equiv 0$8. In finite-element deployment for Cook’s membrane, the stress comparison with the ground-truth Ogden law yields $z_0,W_0^{(z)}\equiv 0$9 (Tepole et al., 1 Mar 2025).

The ablation results are especially informative for the PI-ICNN concept. Removing the inner elementwise ICNN $y$0 from the Hölder module slows convergence and worsens performance. Replacing the final nonlinear ICNN coupling by a purely additive decomposition $y$1 is significantly worse than the nested form $y$2. This indicates that the defining feature is not merely symmetry plus convexity in separate channels, but symmetry plus convexity plus a final nonlinear convex coupling that approximates the full polyconvex class (Tepole et al., 1 Mar 2025).

5. Principal-stress PI-ICNNs for anisotropic plasticity

In anisotropic plasticity, PI-ICNNs are used to learn thermodynamically consistent yield functions that are convex in stress and permutation invariant in principal stress space while embedding anisotropy through linear stress transformations (Jadoon et al., 21 Aug 2025). The constitutive setting is small-strain elastoplasticity with additive strain decomposition

$y$3

yield function

$y$4

and associative flow

$y$5

Convexity of $y$6 is required for thermodynamic consistency, while permutation invariance is required because an isotropic scalar function of principal stresses must be symmetric under permutations of $y$7 (Jadoon et al., 21 Aug 2025).

The model first maps deviatoric stress through one or two learned linear transformations. In 6D vectorized notation,

$y$8

where $y$9, $W_{1:k-1}^{(z)}$0, and $W_{1:k-1}^{(z)}$1 have the same Barlat-style structure derived from anisotropy coefficients and the deviatoric projector $W_{1:k-1}^{(z)}$2 (Jadoon et al., 21 Aug 2025). The transformed principal deviatoric stresses are then fed to either

$W_{1:k-1}^{(z)}$3

for $W_{1:k-1}^{(z)}$4, or

$W_{1:k-1}^{(z)}$5

for $W_{1:k-1}^{(z)}$6. Here $W_{1:k-1}^{(z)}$7 is a scalar ICNN and $W_{1:k-1}^{(z)}$8 is a monotone scalar ICNN; Softplus activations are used unless stated otherwise (Jadoon et al., 21 Aug 2025).

The central modeling choice is that anisotropy is not represented by abandoning isotropy altogether. Instead, anisotropy is encoded in the learned linear maps $W_{1:k-1}^{(z)}$9, $g_i$0, and $g_i$1, while the neural core learns an isotropic convex scalar function in transformed principal stress space. This mirrors the decomposition used by Barlat-type criteria, but replaces rigid power-law isotropic functions by a learned convex permutation-invariant map (Jadoon et al., 21 Aug 2025). The paper also incorporates a broader constrained framework including tension–compression symmetrization

$g_i$2

and a shape-consistent scaling procedure for the fact that the neural model is not positively homogeneous of degree one by construction (Jadoon et al., 21 Aug 2025).

Calibration uses the total loss

$g_i$3

with

$g_i$4

All neural models are trained with Adam at learning rate $g_i$5 for $g_i$6 epochs (Jadoon et al., 21 Aug 2025). The dataset is particularly sparse: 12 uniaxial Al-7079 specimens with measured yield stresses and Lankford ratios, split into nine train/validation/test partitions of $g_i$7 specimens (Jadoon et al., 21 Aug 2025).

The empirical results show the main claimed advantage of PI-ICNNs: better generalization under sparse data than both unconstrained 6D ICNN baselines and hybrid Hill-plus-ICNN models. The reported trainable parameter counts are 257 for ICNN, 263 for Hybrid, 234 for $g_i$8, and 272 for $g_i$9 (Jadoon et al., 21 Aug 2025).

For $\mathcal{N}_C$0 validation-based stopping, the mean absolute yield-function and $\mathcal{N}_C$1-value errors are:

Method	Validation mean $\mathcal{N}_C$2 / mean $\mathcal{N}_C$3	Test mean $\mathcal{N}_C$4 / mean $\mathcal{N}_C$5
Yld2004-18p	31.19 / 1.34	33.32 / 0.59
ICNN	92.78 / 0.32	112.10 / 0.68
Hybrid	132.11 / 0.50	142.43 / 0.64
PI-ICNN$\mathcal{N}_C$6	16.97 / 0.21	23.73 / 0.18
PI-ICNN$\mathcal{N}_C$7	23.55 / 0.21	23.97 / 0.35

These numbers are accompanied by stronger pointwise comparisons: on validation, $\mathcal{N}_C$8 achieves $\mathcal{N}_C$9 and $\mathcal{N}_{MC}$0, while Yld2004-18p gives $\mathcal{N}_{MC}$1 and $\mathcal{N}_{MC}$2; on test, $\mathcal{N}_{MC}$3 gives $\mathcal{N}_{MC}$4 and $\mathcal{N}_{MC}$5 (Jadoon et al., 21 Aug 2025). The paper interprets this as evidence that permutation invariance acts as a regularizer. That interpretation is explicitly presented as a hypothesis rather than a theorem, but it is consistent with the observed plateauing of PI-ICNN validation loss versus the stronger overfitting of the 6D ICNN and hybrid models (Jadoon et al., 21 Aug 2025).

6. Permutation-invariant lifting for ICNN training

A different but technically significant use of the PI-ICNN idea appears in the lifted training method for ICNNs (Siahkoohi et al., 22 May 2026). The problem addressed there is not symmetry of a modeled physical law, but optimization of positivity-constrained ICNN weights. Standard projected gradient descent applies a hard projection onto the nonnegative cone, while direct softplus reparameterization

$\mathcal{N}_{MC}$6

suffers from gradient attenuation on the softplus shoulder, producing dead inter-layer weights and plateaued loss (Siahkoohi et al., 22 May 2026).

The lift replaces direct optimization of the constrained coordinates by an unconstrained hypernetwork conditioned on a permutation-invariant summary of the current batch: $\mathcal{N}_{MC}$7 Because the same nonnegative readout $\mathcal{N}_{MC}$8 is still applied coordinatewise, the deployed network remains an ICNN with the same positivity constraints and therefore the same input-convexity guarantees (Siahkoohi et al., 22 May 2026). The novelty lies in the training dynamics: batch stochasticity enters the pre-readout iterate $\mathcal{N}_{MC}$9, creating an additional cross-covariance channel between iterate fluctuations and gradient fluctuations.

The theory identifies three structural ingredients as necessary for this mechanism: a learnable slack bias $\lambda$00, a batch-conditioned body $\lambda$01, and their coupling through shared batch stochasticity. The paper’s Theorem 1 states that deleting any one of these ingredients makes the slack-channel cross-covariance estimator vanish (Siahkoohi et al., 22 May 2026). Under assumptions (A1)–(A4), the expected pullback loss in lifted coordinates gains additional curvature on the slack subspace, and Corollary 1 gives an Arrhenius-style comparison of first-passage times showing that the lifted method escapes the softplus shoulder faster whenever the extra noise channel is nonzero (Siahkoohi et al., 22 May 2026).

Empirically, the lift reaches lower test loss than both PGD and direct softplus in log-concave energy-based modeling and convex-potential normalizing flows. On the 21-dimensional HEPMASS benchmark, the paper highlights a $\lambda$02-nat gap between the lift and PGD at a representative operating point (Siahkoohi et al., 22 May 2026). The reported wall-clock overhead is roughly $\lambda$03 on convex-potential-flow runs, while the cross-covariance estimator adds below $\lambda$04 overhead (Siahkoohi et al., 22 May 2026). A notable implementation detail is that the conditioning-batch size is taken as $\lambda$05 throughout experiments, so the permutation-invariant pooling is formally present but trivial in those runs (Siahkoohi et al., 22 May 2026).

This use of PI-ICNN differs sharply from the constitutive-modeling sense. Permutation invariance does not act on input coordinates, principal values, or set elements in the modeled function. It acts on the order of samples in the batch that conditions the weight-emitting hypernetwork. The result is still a legitimate PI-ICNN description, but the “PI” belongs to the training parameterization rather than to the deployed forward map.

7. Expressivity, misconceptions, and adjacent architectures

Several common confusions arise around PI-ICNNs. The first is to treat any symmetry-constrained ICNN as permutation-invariant. This is incorrect. A convex-inspired decoder with

$\lambda$06

satisfies $\lambda$07, which is odd equivariance under sign inversion, not permutation invariance over unordered inputs (Huang et al., 23 Nov 2025). It is therefore related to symmetry-constrained ICNN design, but not a PI-ICNN in the Deep Sets or spectral-symmetry sense.

The second confusion is to treat any permutation-invariant architecture as automatically compatible with ICNN convexity. A $\lambda$08-invariant network based on the exact subgroup-invariant operator

$\lambda$09

does provide exact permutation-subgroup invariance, but its elementwise products of input-dependent features generally do not preserve convexity (Kicki et al., 2020). For PI-ICNN design, this makes it a symmetry blueprint rather than a convexity-preserving construction.

A third issue concerns expressivity. For one-hidden-layer ReLU networks with weighted input skip-connections, every convex function that can be implemented can also be represented by an ICNN with the same architecture. This equivalence fails with more layers: beyond one hidden layer, convexity depends on activation-dependent sums of downstream path products, not merely on entrywise nonnegativity of hidden weights (Gagneux et al., 6 Jan 2025). This suggests that deep PI-ICNNs built by standard ICNN sign constraints may inherit the same conservatism. The current constitutive PI-ICNNs avoid this difficulty by using relatively small scalar ICNN modules and explicit symmetric pooling rather than deep generic ReLU DAGs (Tepole et al., 1 Mar 2025, Jadoon et al., 21 Aug 2025).

Optimization remains another limiting factor. ICNNs are often trained either by projection onto the nonnegative cone or by positive reparameterization, and both can be problematic. The lifted training construction addresses one part of this issue through permutation-invariant batch conditioning (Siahkoohi et al., 22 May 2026). A separate line of work on initialization shows that nonnegative weights require mean–variance–correlation control rather than classical centered-weight heuristics, and that correct initialization can remove the apparent necessity of skip-connections in plain ICNNs (Hoedt et al., 2023). A plausible implication is that PI-ICNN blocks sharing the same nonnegative-weight structure should benefit from the same initialization principles.

Finally, not every application of ICNNs that could accommodate symmetric preprocessing is already a PI-ICNN. Explicit ICNN-based model predictive control, for example, establishes convexity conditions for ICNN-based dynamics models and convex MPC objectives, but does not impose permutation invariance (Wang et al., 2024). The paper does observe that symmetric aggregates could be inserted while preserving the convexity logic if those aggregates remain convex in decision-sensitive variables; this suggests a route toward PI-ICNN control models, but it is an extrapolation rather than a reported construction (Wang et al., 2024).

Across current arXiv literature, PI-ICNNs therefore occupy a precise but nontrivial niche. They are most mature where symmetry is intrinsic to principal-variable representations and convexity is physically mandated, as in hyperelastic strain energies and yield functions (Tepole et al., 1 Mar 2025, Jadoon et al., 21 Aug 2025). They are also emerging as training-time parameterizations for difficult positivity-constrained ICNN optimization problems (Siahkoohi et al., 22 May 2026). Their future development is likely to hinge on three technical questions already visible in the literature: how to preserve convexity under richer invariant aggregations, how to reduce the expressivity gap induced by standard ICNN sign constraints in deeper models, and how to stabilize training of nonnegative-weight architectures without losing exact structural guarantees (Gagneux et al., 6 Jan 2025, Hoedt et al., 2023).