Robust Deep FOSLS for Transmission Problems

Abstract: This work presents a robust, energy-based deep learning framework for solving transmission problems in heterogeneous media, including cases with discontinuous material scenarios. We introduce a weighted First-Order System Least-Squares (FOSLS) formulation involving an energy-norm Poincaré constant and prove its equivalence to a natural energy norm of the underlying equations, with constants independent of material parameters. As a result, the optimization landscape remains aligned with a meaningful error approximation even under high material contrast, where standard neural network losses often deteriorate. We further prove that the FOSLS formulation, together with its integral-loss representation, exhibits a passive variance reduction property, whereby the gradient variance progressively decreases as the loss diminishes, in contrast to methods such as VPINNs and Deep Ritz. From a numerical standpoint, we adopt a reduced-order perspective by constructing a low-dimensional space described by a neural network. The optimal coefficients are computed via a least-squares solver, and the space is subsequently improved through gradient-based updates. By selecting the activation function ReQU, the method mitigates the spurious overshoots typically observed in smooth networks when approximating discontinuities. Numerical experiments in 1D and 2D interface settings corroborate these findings.

• The paper presents a novel Deep FOSLS formulation that incorporates an energy-norm Poincaré constant to ensure robustness against high material contrasts.
• It leverages a weighted loss and ReQU activations to mitigate quasi-Gibbs oscillations, enhancing interface resolution in transmission problems.
• The approach achieves passive variance reduction, enabling efficient training with coarser quadrature rules while maintaining stable PDE solutions.

Robust Deep FOSLS for Transmission Problems: Technical Overview

Introduction and Theoretical Foundations

The paper "Robust Deep FOSLS for Transmission Problems" (2604.17549) presents a theoretically motivated framework for solving transmission problems with heterogeneous and possibly highly discontinuous material coefficients using a deep learning approach based on weighted First-Order System Least Squares (FOSLS). The authors specifically address the limitations of existing Physics-Informed Neural Networks (PINNs), Variational PINNs (VPINNs), and Deep Ritz approaches when applied to problems where coefficient discontinuities lead to reduced regularity and, consequently, inefficiency or instability in standard neural loss landscapes.

A novel aspect of this work is the introduction of a loss functional for the FOSLS formulation that incorporates an energy-norm Poincaré constant, proven to ensure equivalence with the natural energy norm of the problem. The constants in this equivalence are shown to be independent of the material parameters (cf. Theorem 1), establishing robustness with respect to high material contrast. The energy norm is designed to meaningfully reflect the physical and mathematical error structure, overcoming the scenario—common to previous FOSLS and Deep Ritz approaches—where a nearly vanishing loss does not imply physical solution accuracy for high-contrast regimes.

Weighted FOSLS Formulation and Robust Loss

Consider the standard transmission (diffusion) problem on $\Omega$: $-\operatorname{div}(\kappa \nabla u) = f,$ with homogeneous Dirichlet boundary conditions. The first-order system introduces $q = -\kappa \nabla u$, leading to the coupled system for $(u, q)$. The FOSLS approach employs a weighted quadratic loss functional: $$\mathcal{L}(u, q) = \|\kappa^{-1/2} q + \kappa^{1/2} \nabla u\|_{L^2}^2 + C_{\mathcal{L}} \| \operatorname{div} q - f \|_{L^2}^2,$$ where $C_{\mathcal{L}}$ is tied to the energy-norm Poincaré constant. By careful selection and estimation of this constant (including via a practical eigenproblem on the neural basis), the authors prove that $\mathcal{L}(u, q)$ is equivalent to the energy norm of the error with constants independent of $\kappa$.

Figure 1: The structure of the FOSLS functional enables simultaneous control over primal and flux variables, maintaining robustness across material contrasts.

This analytic result guarantees that minimization of the proposed FOSLS loss enforces control over the physically meaningful error, even in cases of extreme coefficient jumps or interface-dominated solutions.

Discretization, Neural Parametric Spaces, and Reduced-Order Perspective

The discrete solution framework is constructed via a parameterized finite-dimensional ansatz: $$u = \sum_{i=1}^{n_u} c^u_i \varphi_i, \quad q = \sum_{i=1}^{n_q} c^q_i \boldsymbol{\tau}_i,$$ where the spanning sets $\varphi_i, \boldsymbol{\tau}_i$ are generated by a feedforward neural network, including an explicit Dirichlet cut-off for $-\operatorname{div}(\kappa \nabla u) = f,$0.

The solution procedure is hybrid: for a fixed neural basis, the best coefficients are determined by solving a least-squares system (potentially with block scaling and Tikhonov stabilization), while the neural-network parameters are updated by gradient-based optimization (Adam). This realizes a solver-in-the-loop reduced-order modeling paradigm: the neural net adaptively defines a low-dimensional approximation space, improving it with each SGD update.

To ensure that the weighting in the FOSLS loss is correct at each update step, the Poincaré constant in the energy norm is recomputed for the current neural subspace via a generalized eigenvalue problem.

Passive Variance Reduction and Stochastic Quadrature

A significant disadvantage of existing variational (e.g., Deep Ritz, VPINN) approaches is the persistence of high stochastic gradient variance during training—even as the solution improves—unless costly quadrature rules are employed. In contrast, the authors demonstrate theoretically and empirically that, for the proposed robust FOSLS loss, the variance of the stochastic gradient estimator decays in proportion to the loss. This effect, termed passive variance reduction, is a direct result of aligning the integrand structure with the physically correct norm.

Figure 2: Loss-Error robustness and variance behavior demonstrate the statistical advantage of FOSLS loss over variational approaches.

Thus, their method allows for the use of significantly coarser (even MC-based) quadrature without instability or bias in optimization, which is crucial for scalability.

Neural Architecture and Mitigation of the Quasi-Gibbs Phenomenon

Standard neural networks with smooth activations (e.g., $-\operatorname{div}(\kappa \nabla u) = f,$1) exhibit quasi-Gibbs oscillations near discontinuities—manifesting as spurious overshoots or undershoots in the solution or its derivatives (especially problematic in high-contrast regimes or near interfaces). The paper utilizes the $-\operatorname{div}(\kappa \nabla u) = f,$2 activation, a $-\operatorname{div}(\kappa \nabla u) = f,$3-function, as the core element of the ansatz. This choice, motivated by constructive results in recent literature, effectively mitigates quasi-Gibbs oscillations even though the network is not granted explicit knowledge of discontinuity locations.

Figure 3: Demonstration of interface resolution and oscillation suppression using $-\operatorname{div}(\kappa \nabla u) = f,$4 versus $-\operatorname{div}(\kappa \nabla u) = f,$5 activation.

Numerical Experiments: 1D and 2D Transmission Problems

A comprehensive set of experiments validate the theoretical claims:

• Effect of Quadrature: Training with FOSLS loss remains stable with much coarser sampling per epoch compared to Deep Ritz. Overflow and loss instability are pronounced for the latter unless fine quadrature is employed.
• Robustness to Contrast: The relation between loss and physical error remains linear regardless of the contrast ratio in $-\operatorname{div}(\kappa \nabla u) = f,$6 (spanning $-\operatorname{div}(\kappa \nabla u) = f,$7 to $-\operatorname{div}(\kappa \nabla u) = f,$8), in sharp contrast to standard FOSLS.
• Interface Resolution: The combination of the robust loss and $-\operatorname{div}(\kappa \nabla u) = f,$9 network produces accurate interface representation and low excess variation even in 2D geometries with curved or planar interfaces. The breaking curves introduced by the network are shown to adaptively track major solution features.

  Figure 4: Relative errors in energy norms for $q = -\kappa \nabla u$0 and $q = -\kappa \nabla u$1 components, highlighting the increased difficulty in approximating the discontinuous component.

Implications and Future Directions

This research offers both practical and theoretical advancement in the use of neural networks for PDEs with sharp heterogeneity:

• The robust FOSLS functional ensures that the training objective is non-degenerate with respect to physical error, supporting high-contrast, interface-rich scenarios common in geophysics, materials science, and engineering.
• The established passive variance reduction property reduces the computational cost per epoch, supporting efficient training for high-dimensional or expensive-to-integrate problems.
• The $q = -\kappa \nabla u$2-based networks provide a structured, non-oscillatory approximation near interfaces without the need for prior domain decomposition or fine-tuned adaptive sampling.

Practically, these results suggest that stable, robust neural PDE solvers for real-world transmission problems are viable at moderate computational cost. Theoretically, the results close a known gap in the literature regarding the loss landscape for high-contrast problems.

Potential future avenues include:

• Extending the method to singularity-dominated regimes, potentially via hybrid bases or architectures.
• Exploration of multiple separate neural networks for representing solution and flux components, optimizing for their functional space requirements.
• Integration with adaptive or multilevel quadrature, and formulation of architectures informed by known singular structure.

Conclusion

The paper establishes a robust, energy-norm-equivalent Deep FOSLS framework, capable of stably and accurately solving transmission problems in highly heterogeneous domains. The synergy of a theoretically informed loss, practical variance control, and non-smooth neural architecture supports both high-fidelity solution recovery and computational efficiency, positioning this approach as a reference for neural PDE approaches in challenging interface regimes.