Constrained Neural Parameterization for Optimization in Function Spaces

Abstract: We propose constrained neural parameterization schemes for several classes of constraints arising in optimization problems in function spaces. This is achieved by constructing smooth neural parameterizations whose image lies entirely in the admissible set while remaining asymptotically dense. In this way, the original constrained optimization problem is transformed into a smooth unconstrained problem in parameter space, enabling efficient gradient-based optimization without penalty parameters or Lagrange multipliers. We develop geometric constructions for polyhedral constraint sets in Hilbert spaces, propose smooth neural architectures for some pointwise constraints, and introduce an exact reduced neural formulation for PDE constraints that admit a separable structure. Numerical experiments demonstrate the effectiveness of the proposed methods.

• The paper introduces Constrained Neural Parameterization, a framework that embeds hard constraints directly into neural architectures for function space optimization.
• It leverages a geometric decomposition of convex polyhedra in Hilbert spaces to transform constrained problems into unconstrained ones over neural parameters.
• Numerical experiments across PDE-constrained and inverse problems demonstrate superior accuracy, stability, and faster convergence compared to traditional penalty-based methods.

Constrained Neural Parameterization for Optimization in Function Spaces

Overview and Motivation

The paper "Constrained Neural Parameterization for Optimization in Function Spaces" (2606.00855) develops a geometric and analytic framework for embedding constraints directly into neural network-based parameterizations when solving infinite-dimensional optimization problems. The approach addresses fundamental limitations of existing penalty-based and KKT-based neural solvers prevalent in PDE-constrained optimization, optimal control, and inverse problems. Instead of enforcing constraints approximately or introducing auxiliary multipliers/adjoints, the proposed Constrained Neural Parameterization (CNP) schemes ensure that the feasible set is encoded exactly and densely in the neural architecture. This advances both theoretical interpretability and computational robustness for optimization in Banach and Hilbert spaces.

Geometric Foundation and Polyhedral Constraints

The main theoretical innovation is a constructive decomposition of general convex polyhedra in infinite-dimensional Hilbert spaces, leveraging the Minkowski–Weyl theorem. The decomposition expresses any polyhedron as the sum of (i) a linear subspace orthogonal to the constraint directions, (ii) the convex hull of finitely many vertices, and (iii) the conic hull of finitely many extreme rays. For optimization problems,

$$\min_{x \in X} J(x) \quad \text{subject to} \quad x \in \mathcal{K} \subset X,$$

a smooth parametric mapping $\mathcal{T}_M: \mathbb{R}^M \to X$ is constructed so $\mathcal{T}_M(\mathbb{R}^M) \subset \mathcal{K}$ and its range is asymptotically dense as $M \to \infty$. The CNP formulation thus reduces constrained optimization to an unconstrained problem over neural parameters.

Figure 1: Loss histories demonstrate improved convergence behavior for CNP versus penalty-based neural optimization.

For polyhedral constraints specified via possibly indirect functionals, hybrid constructions involving the Riesz representation theorem and orthonormalization permit efficient parameterization regardless of whether constraint vectors are present directly in the state space or only implicitly.

Exact Enforcement of Pointwise and PDE Constraints

The paper develops CNP architectures for various classes of pointwise constraints—including unilateral and bilateral bounds, multiphase summation constraints, and boundary conditions. For PDE-constrained optimization, a central advancement is the "exact reduced neural method": if the PDE system admits a separable structure (e.g., control can be isolated as $u = \mathbb{G}(y)$), only the state is parameterized, and the control is computed analytically. The neural ansatz is designed to embed all functional, boundary, and PDE constraints exactly.

In cases with irregular domains or corner singularities where classical elliptic regularity fails, the singularity-enriched neural method incorporates analytic singular functions (localized via cutoff mechanisms at re-entrant corners) into the architecture, enabling accurate representation of low-regularity solutions.

Figure 2: Primal-dual active set methods are benchmarked against neural approaches in low-regularity state-constrained settings.

Numerical Results: Performance and Robustness

A series of numerical experiments validate the proposed schemes across 1D, high-dimensional, and 2D domains, including state-constrained optimal control, mathematical programs with complementarity conditions (without strict complementarity), Navier–Stokes control with high-frequency adjoints, and hybrid inverse problems. CNP methods consistently outperform penalty-based solvers in accuracy and converge faster, independent of penalty parameter tuning. The exact reduced neural method demonstrates superior feasibility preservation and stability, especially in scenarios where multipliers/adjoints exhibit low regularity or the KKT theory cannot be applied due to degeneracy.

Figure 3: The singularity-enriched exact reduced neural method (SEERNM) yields notably lower errors near domain corners compared to standard architectures.

The hybrid inverse problem example further demonstrates the ability of CNP to recover material property fields and source terms from noisy internal and boundary data within the PDE constraint, reflecting the generality and robustness of the framework.

Figure 4: Results for the hybrid inverse problem highlight faithful reconstruction of both absorption coefficients and source terms using CNP.

Practical and Theoretical Implications

The theoretical contribution provides a rigorous geometric foundation for constrained function space optimization, opening new avenues for neural architectures that guarantee hard feasibility and dense approximation. Practically, CNP-based approaches eliminate a major source of instability (penalty parameter tuning, projection steps, multiplier updates), facilitating gradient-based training and improved generalization. The singularity-enriched approach broadens applicability to irregular domains, while the operator-based reduction for PDEs streamlines optimization in large-scale and systems settings.

The integration of CNP with operator learning and surrogate neural models promises further advances for real-time control, high-dimensional PDE-constrained design, and hybrid data–physics inverse problems.

Speculation on Future Directions

Outstanding challenges include:

• Convergence and optimization theory for the parameterizations under non-smooth losses and adaptive architectures.
• Extension to systems and nonlinear controllers where PDE constraints cannot be analytically separated.
• Unified CNP mechanisms for mixed constraint sets (combining polyhedral, pointwise, and integral constraints) and interface problems.
• Joint incorporation with operator learning paradigms (e.g., FNOs, DeepONets) for scalable surrogate modeling in function space.

Conclusion

The paper establishes Constrained Neural Parameterization as a powerful, rigorously grounded method for embedding feasibility directly into neural optimization architectures over function spaces. The mathematical decomposition, universal approximation, and exact enforcement of constraints enable stable, accurate solutions to a wide range of challenging problems in optimal control, PDE-constrained optimization, and inverse modeling. Integration with emerging operator learning strategies and further theoretical development will likely drive substantial progress in data-driven scientific computing and AI-assisted engineering design.