A Theory-guided Weighted $L^2$ Loss for solving the BGK model via Physics-informed neural networks

Abstract: While Physics-Informed Neural Networks offer a promising framework for solving partial differential equations, the standard $L^2$ loss formulation is fundamentally insufficient when applied to the Bhatnagar-Gross-Krook (BGK) model. Specifically, simply minimizing the standard loss does not guarantee accurate predictions of the macroscopic moments, causing the approximate solutions to fail in capturing the true physical solution. To overcome this limitation, we introduce a velocity-weighted $L^2$ loss function designed to effectively penalize errors in the high-velocity regions. By establishing a stability estimate for the proposed approach, we shows that minimizing the proposed weighted loss guarantees the convergence of the approximate solution. Also, numerical experiments demonstrate that employing this weighted PINN loss leads to superior accuracy and robustness across various benchmarks compared to the standard approach.

• The paper demonstrates that minimizing standard L2 loss in PINNs fails to control high-velocity errors impacting macroscopic moments in BGK models.
• It introduces a velocity-weighted loss function, ensuring L2 and L1 convergence with theoretical stability guarantees.
• Numerical experiments across 1D, 2D, and 3D benchmarks confirm improved shock resolution and physical fidelity.

Theory-guided Weighted $L^2$ Loss for Physics-Informed Neural Networks Solving the BGK Model

Introduction: Fundamental Limitations of Standard PINN Loss in Kinetic Modeling

The paper investigates the deployment of Physics-Informed Neural Networks (PINN) for solving the Bhatnagar–Gross–Krook (BGK) model, a widely used kinetic equation for rarefied gas dynamics. The BGK model simplifies the Boltzmann collision operator by using a relaxation term toward a local Maxwellian, with macroscopic moments computed as velocity-weighted integrals. While PINN frameworks have demonstrated promise in bypassing the curse of dimensionality in kinetic regimes, this study emphasizes that minimizing the standard squared $L^2$ norm of residuals—a conventional PINN objective—is fundamentally insufficient for the BGK model. The standard loss may not guarantee accurate macroscopic moments, especially when errors are concentrated in high-velocity tails.

Explicit Counterexamples: Failure of Standard $L^2$ PINN Loss

The paper provides explicit construction of $\varepsilon$-parameterized families of approximate solutions where the standard PINN loss $\mathcal{L}_{\mathrm{PINN}}$ converges to zero, yet the physical solution error persists.

These examples demonstrate that small high-velocity errors in the distribution function can induce significant biases in macroscopic moments (mass, momentum, energy), yielding physically incorrect solutions despite low PINN loss. The key mechanism is the insufficient penalization of errors in velocity regions where moments are sensitive.

Figure 1: Visualization of the function $K_\varepsilon(v)$ for $\varepsilon = 0.01$, concentrated in high-velocity regions with negligible $L^2$ norm but substantial energy moment.

Weighted PINN Objective: Theoretical Rationale and Mathematical Stability

To address this vulnerability, the authors introduce a velocity-weighted loss function:

$$\mathcal{L}_{w\text{-PINN}} = \mathcal{L}_{w,pde} + \lambda_{bc}\mathcal{L}_{w,bc} + \lambda_{ini}\mathcal{L}_{w,ini},$$

where each residual is weighted as

$$w(v) = 1 + \alpha |v|^\beta,\quad \alpha>0,\, \beta>7/2$$

and appropriate integrability conditions are satisfied, ensuring control over second-order velocity moments.

A rigorous stability estimate is established: minimizing $L^2$0 guarantees convergence of the neural approximation to the true solution in $L^2$1, along with $L^2$2 convergence of macroscopic moments. This is in direct contrast to the standard loss, which fails to provide such guarantees.

Numerical Experiments: Benchmarking and Hyperparameter Sensitivity

Numerical experiments validate the robust improvement of the weighted loss over standard and empirical relative loss formulations across diverse test cases:

• 1D Smooth and Riemann Problems
• 2D and 3D smooth and Riemann benchmarks
• Range of Knudsen numbers from near-continuum to rarefied ($L^2$3, $L^2$4, $L^2$5)

The ablation study reveals a U-shaped error curve as a function of the polynomial weight parameter $L^2$6, demonstrating the necessity of balancing penalization. Excessive weighting causes instability and optimization bias; insufficient weighting fails to control crucial moment errors. Empirically, $L^2$7 provides robust results across all regimes.

Figure 2: Relative error curves of $L^2$8 and macroscopic moments $L^2$9 as a function of polynomial growth rate $L^2$0 at $L^2$1, revealing optimal parameter regions.

Figure 3: Distribution function $L^2$2 and macroscopic moments for the 1D smooth problem, showcasing superiority of the weighted loss.

Figure 4: Distribution function $L^2$3 and macroscopic moments for the 1D Riemann problem at $L^2$4; weighted loss provides improved shock resolution and physical fidelity.

Comparative Analysis of Loss Weighting Strategies

The empirical relative loss ($L^2$5) penalizes low-density/high-velocity regions, similar to polynomial weighting, but suffers from instability and irregular gradient profiles, especially in discontinuous regimes. Figures compare profiles of the weights under both schemes, demonstrating the consistency and effectiveness of the polynomial-weighted PINN objective.

Figure 5: Comparison of weight shapes between the relative and proposed losses for the 1D smooth problem, illustrating their common high-velocity penalization.

Figure 6: Weight shapes for the 1D Riemann problem, indicating irregularities for empirical relative loss and stable monotonicity for polynomial weighting.

Practical and Theoretical Implications

The theory-guided weighted loss constitutes a fundamentally reliable proxy for solution accuracy in kinetic regimes governed by BGK or similar models. The mathematical analysis generalizes to other kinetic equations with nonlocal moment coupling and tail sensitivity, suggesting broad applicability for future PINN-based solvers in rarefied gas dynamics, turbulent flows, and plasma physics. The rigorous stability framework ensures that solution fidelity is dictated by physically relevant error norms rather than convenience-driven residuals, advancing analytic rigor in PINN methodology.

Conclusion

This paper rigorously demonstrates the inadequacy of the standard $L^2$6 PINN loss for the BGK model and proposes a theory-guided weighted $L^2$7 objective, providing both analytical stability guarantees and robust empirical performance across dimensional, physical, and parametric variations. The convergence results and practical efficacy distilled from extensive benchmarks are strong endorsements for the systematic inclusion of weighted residuals in PINN formulations for kinetic equations. Future research directions include extension to full Boltzmann collision models, exploration of kinetic Fokker–Planck systems, and formalization of optimization strategies for weighted objectives in high-dimensional, stiff, or turbulent flow regimes.

(2604.04971)