Monotone Gradient Networks (mGradNets)

• mGradNets are neural architectures that directly learn gradients of convex functions by enforcing globally positive semidefinite Jacobians.
• They offer two designs—cascaded (mGradNet-C) and modular (mGradNet-M)—balancing depth and parallelism for efficient parameterization.
• Their structure supports scalable applications in optimization, optimal transport, and control with fewer parameters than traditional ICNNs.

Searching arXiv for the primary mGradNet papers and closely related work. Monotone Gradient Networks (mGradNets) are neural architectures for vector-valued maps $f:\mathbb R^d\to\mathbb R^d$ that are constrained to satisfy $f(x)=\nabla F(x)$ for some convex, twice continuously differentiable potential $F\in C^2(\mathbb R^d)$. In the formulations introduced as C-MGN and M-MGN, and later generalized as mGradNet-C and mGradNet-M, the defining property is the global positive semidefiniteness of the Jacobian, $J_f(x)\succeq 0$ for all $x$, which makes the network a direct parameterization of monotone gradients rather than of scalar convex functions that must then be differentiated (Chaudhari et al., 2023, Chaudhari et al., 2024). This design was motivated by applications in gradient-based optimization, normalizing flows, and especially optimal transport, where Brenier’s theorem identifies the squared-Euclidean Monge map as the gradient of a convex potential. Subsequent work used mGradNets to learn such transport maps directly through Monge–Ampère-based objectives and applied them to robot swarm control (Chaudhari et al., 17 Jul 2025).

1. Mathematical characterization and motivation

A vector-valued map $g:\mathbb R^n\to\mathbb R^n$ is monotone if

$$(g(x)-g(y))^\top (x-y)\ge 0,\qquad \forall x,y\in\mathbb R^n.$$

By classical results cited in the original formulation—Rockafellar (1970) and Boyd & Vandenberghe (2004)—a continuously differentiable scalar function $f:\mathbb R^n\to\mathbb R$ is convex if and only if its gradient $\nabla f$ is monotone (Chaudhari et al., 2023). The later GradNet framework restated this in differential form: an mGradNet is precisely a gradient network whose Jacobian is symmetric positive semidefinite everywhere, $J_f(x)\succeq 0$ (Chaudhari et al., 2024).

The immediate motivation for learning monotone gradients directly is operational. Many optimization algorithms require only a gradient oracle. In the Monge problem with squared-Euclidean cost, Brenier’s theorem guarantees that the optimal transport map is the gradient of a convex function. Direct modeling of $f(x)=\nabla F(x)$0 therefore targets the object of interest itself. This differs from Input Convex Neural Networks (ICNNs), which parameterize a convex potential and require a subsequent differentiation step, and from Input Convex Gradient Networks (ICGNs), which use Hessian factorization (Chaudhari et al., 2023).

In the optimal-transport setting, the Monge formulation seeks a map $f(x)=\nabla F(x)$1 minimizing

$f(x)=\nabla F(x)$2

Under the regularity conditions stated in the GradNetOT exposition, the unique minimizer has the form

$f(x)=\nabla F(x)$3

and the pushforward condition is equivalent to the Monge–Ampère relation

$f(x)=\nabla F(x)$4

This places monotone gradients at the center of data-driven transport-map learning (Chaudhari et al., 17 Jul 2025).

2. Canonical architectures: cascaded and modular forms

The original paper introduced two architectures for directly learning gradients of convex functions: the Cascaded Monotone Gradient Network (C-MGN) and the Modular Monotone Gradient Network (M-MGN) (Chaudhari et al., 2023). Their later counterparts in the GradNet framework are denoted mGradNet-C and mGradNet-M (Chaudhari et al., 2024).

Architecture	Core structure	Monotonicity mechanism
C-MGN / mGradNet-C	Deep cascaded network with shared $f(x)=\nabla F(x)$5	Jacobian factors as $f(x)=\nabla F(x)$6
M-MGN / mGradNet-M	Sum of parallel modules	Each module contributes PSD terms

For C-MGN, the 2023 construction uses a shared linear weight $f(x)=\nabla F(x)$7, bias vectors $f(x)=\nabla F(x)$8, and layer-wise monotonic activations $f(x)=\nabla F(x)$9. With $F\in C^2(\mathbb R^d)$0 hidden layers,

$F\in C^2(\mathbb R^d)$1

$F\in C^2(\mathbb R^d)$2

and

$F\in C^2(\mathbb R^d)$3

If each $F\in C^2(\mathbb R^d)$4 is element-wise increasing, then

$F\in C^2(\mathbb R^d)$5

The 2024 mGradNet-C variant preserves the same shared-$F\in C^2(\mathbb R^d)$6 cascaded logic while introducing nonnegative layer-wise coefficients $F\in C^2(\mathbb R^d)$7, yielding the factorization $F\in C^2(\mathbb R^d)$8 with diagonal PSD $F\in C^2(\mathbb R^d)$9 (Chaudhari et al., 2024).

For M-MGN, the network is built from $J_f(x)\succeq 0$0 independent modules. Each module has $J_f(x)\succeq 0$1, a bias $J_f(x)\succeq 0$2, and a scalar convex function $J_f(x)\succeq 0$3 with $J_f(x)\succeq 0$4. The model is

$J_f(x)\succeq 0$5

$J_f(x)\succeq 0$6

Since each $J_f(x)\succeq 0$7 is convex, $J_f(x)\succeq 0$8, and the Jacobian becomes

$J_f(x)\succeq 0$9

The corresponding mGradNet-M formulation in 2024 writes

$x$0

with $x$1 convex, $x$2, and $x$3 nonnegative and nondecreasing, which again ensures that every module contributes PSD terms (Chaudhari et al., 2024).

The contrast between the two families is structural rather than semantic. C-MGN is described as a “deep” cascaded structure with one shared $x$4 and potentially more nonlinear depth; M-MGN is a “wide” structure with parallel shallow modules whose contributions are summed (Chaudhari et al., 2023).

3. Theoretical guarantees and representational scope

The foundational guarantee is monotonicity. Propositions 1 and 2 in the original paper prove $x$5 for all $x$6, hence the learned map is guaranteed to be $x$7 for some convex $x$8 (Chaudhari et al., 2023). In the later GradNet formalization, this becomes the defining criterion of an mGradNet, and several closure constructions are given: a single-layer recipe $x$9 with $g:\mathbb R^n\to\mathbb R^n$0 and $g:\mathbb R^n\to\mathbb R^n$1; a “Lipschitz flip” $g:\mathbb R^n\to\mathbb R^n$2 when a GradNet is $g:\mathbb R^n\to\mathbb R^n$3-Lipschitz; conical combinations of mGradNets; and strong-convex shifts $g:\mathbb R^n\to\mathbb R^n$4, which produce gradients of $g:\mathbb R^n\to\mathbb R^n$5-strongly-convex potentials (Chaudhari et al., 2024).

Strong convexity has immediate consequences for invertibility. The original work notes that adding a linear term $g:\mathbb R^n\to\mathbb R^n$6 can enforce $g:\mathbb R^n\to\mathbb R^n$7 when $g:\mathbb R^n\to\mathbb R^n$8. In that case the map is $g:\mathbb R^n\to\mathbb R^n$9-strongly monotone, the underlying potential is $$(g(x)-g(y))^\top (x-y)\ge 0,\qquad \forall x,y\in\mathbb R^n.$$0-strongly convex, and the resulting $$(g(x)-g(y))^\top (x-y)\ge 0,\qquad \forall x,y\in\mathbb R^n.$$1 is bijective and invertible, which is explicitly identified as useful in normalizing flows (Chaudhari et al., 2023).

A notable point in the development of the literature concerns universal approximation. The original 2023 exposition states that no formal universal-approximation bound was proved for monotone gradients, although empirical expressivity was strong (Chaudhari et al., 2023). The 2024 GradNet paper then established a universal approximation theorem for gradients of convex functions. Specifically, for the class $$(g(x)-g(y))^\top (x-y)\ge 0,\qquad \forall x,y\in\mathbb R^n.$$2, the single-layer mGradNet of Proposition 8 with scaled softmax activation,

$$(g(x)-g(y))^\top (x-y)\ge 0,\qquad \forall x,y\in\mathbb R^n.$$3

universally approximates $$(g(x)-g(y))^\top (x-y)\ge 0,\qquad \forall x,y\in\mathbb R^n.$$4 on $$(g(x)-g(y))^\top (x-y)\ge 0,\qquad \forall x,y\in\mathbb R^n.$$5. The proof sketch proceeds through uniform approximation of convex functions by scaled LogSumExp of finitely many affine hyperplanes and then differentiates that representation (Chaudhari et al., 2024).

The same paper also characterizes a larger function family: “monotone convex transformation of a sum of convex ridges,”

$$(g(x)-g(y))^\top (x-y)\ge 0,\qquad \forall x,y\in\mathbb R^n.$$6

with $$(g(x)-g(y))^\top (x-y)\ge 0,\qquad \forall x,y\in\mathbb R^n.$$7 convex $$(g(x)-g(y))^\top (x-y)\ge 0,\qquad \forall x,y\in\mathbb R^n.$$8 and $$(g(x)-g(y))^\top (x-y)\ge 0,\qquad \forall x,y\in\mathbb R^n.$$9 nondecreasing. The corresponding gradient can be implemented as an mGradNet by combining a ridge-sum gradient $f:\mathbb R^n\to\mathbb R$0 with a scalar multiplier $f:\mathbb R^n\to\mathbb R$1 (Chaudhari et al., 2024). This suggests that the architecture is not restricted to a narrow template of convex potentials.

4. Training objectives and computational profile

Three training modes recur across the mGradNet literature. The first is direct gradient regression. In the original formulation, when samples of the true gradient are available, the network is supervised with

$f:\mathbb R^n\to\mathbb R$2

or an $f:\mathbb R^n\to\mathbb R$3 variant (Chaudhari et al., 2023). The 2024 GradNet study uses the same principle in the form

$f:\mathbb R^n\to\mathbb R$4

and emphasizes that no extra Jacobian-symmetry penalty is needed because symmetry and PSD structure are built into the architecture (Chaudhari et al., 2024).

The second mode is density matching or optimal transport via a change-of-variable objective. In the 2023 paper, $f:\mathbb R^n\to\mathbb R$5 is treated as a normalizing flow and trained by minimizing a KL-based objective,

$f:\mathbb R^n\to\mathbb R$6

Optimization uses standard SGD or Adam, with the same batch sizes and learning rates for all competing models in the experiments (Chaudhari et al., 2023).

The third mode, developed in GradNetOT, imposes the pushforward condition in logarithmic residual form: $f:\mathbb R^n\to\mathbb R$7 With residual

$f:\mathbb R^n\to\mathbb R$8

the optimization problem is

$f:\mathbb R^n\to\mathbb R$9

The reported implementation uses Adam with batch size $\nabla f$0, a decaying learning rate from $\nabla f$1 to $\nabla f$2, and automatic differentiation to compute both $\nabla f$3 and $\nabla f$4 (Chaudhari et al., 17 Jul 2025).

Computationally, the architecture is intended to avoid the burdens that accompany indirect convex-potential learning. The 2024 study states that forward/backward cost is comparable to an MLP of the same size and that no costly numerical integration or PSD-projection is used. Nonnegativity constraints, such as those on $\nabla f$5 in mGradNet-C and on $\nabla f$6 in mGradNet-M, are enforced by parameterization as squares or via softplus (Chaudhari et al., 2024). This is consistent with the original motivation that direct parameterization of $\nabla f$7 avoids both the extra differentiation step of ICNNs and the heavy Hessian-factorization required by ICGNs (Chaudhari et al., 2023).

5. Empirical performance on gradient fields and transport tasks

The earliest experiments focused on a synthetic two-dimensional gradient field. In the 2023 comparison, ICGN used 15 parameters, ICNN used 78 and 163 parameters, C-MGN used 14 parameters, and M-MGN used 22 parameters. The reported MSEs in dB were: ICGN $\nabla f$8, ICNN (78p) $\nabla f$9, ICNN (163p) $J_f(x)\succeq 0$0, C-MGN (14p) $J_f(x)\succeq 0$1, and M-MGN (22p) $J_f(x)\succeq 0$2. The paper states that C-MGN beats all baselines with an order-of-magnitude fewer parameters (Chaudhari et al., 2023).

The broader 2024 GradNet evaluation used comparable parameter counts in 2D—mGradNet-C with $J_f(x)\succeq 0$3, hidden dim $J_f(x)\succeq 0$4, had $J_f(x)\succeq 0$5 parameters; mGradNet-M with $J_f(x)\succeq 0$6 modules, each dim $J_f(x)\succeq 0$7, had $J_f(x)\succeq 0$8; ICNN had $J_f(x)\succeq 0$9, ICGN $f(x)=\nabla F(x)$00, and CRR $f(x)=\nabla F(x)$01. On the convex field benchmark over $f(x)=\nabla F(x)$02, the RMSEs in dB were ICNN $f(x)=\nabla F(x)$03, ICGN $f(x)=\nabla F(x)$04, CRR $f(x)=\nabla F(x)$05, mGradNet-C $f(x)=\nabla F(x)$06, and mGradNet-M $f(x)=\nabla F(x)$07, corresponding to an improvement of approximately $f(x)=\nabla F(x)$08–$f(x)=\nabla F(x)$09 dB over the best baseline (Chaudhari et al., 2024).

Optimal-transport experiments in the original paper evaluated Gaussian optimal coupling in dimensions $f(x)=\nabla F(x)$10 and $f(x)=\nabla F(x)$11, with baselines Whitening Transform (WT), Invertible Autoregressive Flow (IAF), and Convex Potential Flow (CP Flow). For $f(x)=\nabla F(x)$12, the reported NLL and transport cost were WT $f(x)=\nabla F(x)$13; IAF $f(x)=\nabla F(x)$14; CP Flow $f(x)=\nabla F(x)$15; C-MGN $f(x)=\nabla F(x)$16; M-MGN $f(x)=\nabla F(x)$17. For $f(x)=\nabla F(x)$18, they were WT $f(x)=\nabla F(x)$19; IAF $f(x)=\nabla F(x)$20; CP Flow $f(x)=\nabla F(x)$21; C-MGN $f(x)=\nabla F(x)$22; M-MGN $f(x)=\nabla F(x)$23. The paper concludes that both C-MGN and M-MGN match or outperform CP Flow and IAF in NLL and incur lower transport cost than IAF, with simpler architectures (Chaudhari et al., 2023).

GradNetOT extended the empirical range to Gaussian-to-Gaussian transport up to $f(x)=\nabla F(x)$24. In the 2D case, a standard unconstrained feed-forward net is reported to fail because points “cross” under the mapping, whereas both mGradNet-C and mGradNet-M recover the analytic whitening transform. In higher dimensions, the mean squared error between the learned map and the true whitening transform remains near zero, with mGradNet-M slightly outperforming mGradNet-C (Chaudhari et al., 17 Jul 2025). A plausible implication is that the structural bias imposed by PSD Jacobians is especially relevant when the target map is known a priori to be a Brenier map.

6. Applications, distinctions, and open directions

A first applied use case is color domain adaptation for autonomous driving. The task is to map RGB pixel colors of daytime road scenes to sunset-style scenes by learning the Monge map between their color distributions. The reported pipeline fits a multivariate Gaussian to target pixel colors, learns a map $f(x)=\nabla F(x)$25 from day to Gaussian via M-MGN or C-MGN by minimizing Gaussian NLL, and applies the learned map to test images for visual comparison against a kernel-OT baseline (OTKer). The stated outcome is that C-MGN and M-MGN produce color transforms on par with more complex convex-relaxation OT methods while being learned end-to-end with few parameters (Chaudhari et al., 2023).

A second application is robot swarm control. GradNetOT encodes initial and desired swarm layouts as smooth densities on $f(x)=\nabla F(x)$26 via MNIST grayscale images and KDE, then learns an mGradNet-M transport map that tracks the exact Wasserstein geodesic between the distributions. In comparison to barycentric projection of the discrete Kantorovich plan, the averaged MSE over 1,000 test points is reported to be on the order of $f(x)=\nabla F(x)$27 across $f(x)=\nabla F(x)$28 source-target digit pairs (Chaudhari et al., 17 Jul 2025).

A recurrent point of clarification concerns terminology. mGradNets are not the same object as monotone variational-inequality training methods. The latter construct a monotone operator $f(x)=\nabla F(x)$29 in parameter space, solve a VI of the form “find $f(x)=\nabla F(x)$30 such that $f(x)=\nabla F(x)$31 for all $f(x)=\nabla F(x)$32,” and apply projected or extragradient schemes for training or fine-tuning FC, GNN, and CNN models (Xu et al., 2022). By contrast, mGradNets constrain the input-output Jacobian $f(x)=\nabla F(x)$33 so that the network itself represents a monotone gradient field. The shared word “monotone” therefore refers to different mathematical objects in the two lines of work.

Open directions were already identified in the original formulation: convolutional generalization by replacing fully connected $f(x)=\nabla F(x)$34 or $f(x)=\nabla F(x)$35 with convolution and its adjoint while preserving PSD Jacobians; richer module compositions; theoretical error bounds; scaling to very high dimensions; and integration into end-to-end task training (Chaudhari et al., 2023). The later GradNet framework partially addressed one earlier limitation by supplying a universal approximation theorem (Chaudhari et al., 2024). This suggests that current research is progressively shifting from existence and feasibility questions toward scalability, compositional design, and task-specific transport objectives.