NNPDF Framework: Neural PDF Determination

• NNPDF Framework is a machine-learning system that determines proton, nuclear, and photon parton distribution functions using neural network ensembles and Monte Carlo replicas.
• It employs ensemble training with GPU acceleration and Bayesian hyperparameter optimization to enhance accuracy and efficiency in PDF extraction.
• Recent advancements include integrated nuclear fits, approximate N³LO evolution, and significant computational cost and energy reductions for large-scale QCD analyses.

The NNPDF (Neural Network Parton Distribution Function) framework is a comprehensive machine-learning–based system for the nonperturbative determination of proton, nuclear, and photon parton distribution functions (PDFs) from high-energy scattering data, with minimal theoretical bias and robust, data-driven uncertainty quantification. Originating in collider QCD phenomenology, NNPDF employs an ensemble-of-neural-network approach for PDF parametrization, a Monte Carlo method to propagate all sources of experimental and theoretical uncertainty, and a statistically rigorous workflow for cross-validation and hyperparameter optimization. Recent methodological advances have included GPU-accelerated ensemble training, Bayesian hyperparameter optimization, integrated treatment of proton and nuclear PDFs, and the support for approximate N³LO DGLAP evolution.

1. Theoretical Foundations

At the core of NNPDF is the view of PDF determination as an inference problem in function space, where the observable is not a finite-dimensional parameter vector but a distributional functional, $f_\ell(x,Q^2)$, for each parton flavor $\ell$, as a function of momentum fraction $x$ and energy scale $Q^2$. The extraction is accomplished via an ensemble of $N_{\text{rep}}$ Monte Carlo replicas, each representing a possible realization of the PDFs consistent with experimental data and their uncertainties:

$f_\ell^{(k)}(x,Q^2),\quad k=1\dots N_{\text{rep}}$

The ensemble mean and variance give, respectively, the PDF central value and one-sigma uncertainty:

$$\langle f_\ell(x,Q^2) \rangle = \frac{1}{N_{\text{rep}}}\sum_k f_\ell^{(k)}(x,Q^2)$$

$$\sigma_\ell(x,Q^2) = \sqrt{\frac{1}{N_{\text{rep}}-1} \sum_k \left(f_\ell^{(k)}(x,Q^2) - \langle f_\ell \rangle\right)^2 }$$

PDF evolution with $Q^2$ is performed by solving the DGLAP equations, up to NNLO or, in recent extensions, approximate N³LO (aN³LO) using constraint-based parametrizations of the $O(\alpha_s^4)$ splitting functions. The aN³LO evolution includes both missing-higher-order uncertainties (MHOU) from renormalization/factorization scale variation and incomplete-higher-order uncertainties (IHOU) from ansatz spread, combined in the theory covariance matrix for full error propagation (Hekhorn et al., 2023).

In the integrated framework for nuclear and proton PDFs, the atomic mass number $A$ is included as an explicit network input, allowing a single neural network to learn the $x$– and $A$–dependence of the distributions up to lead ($A=208$) (Rabemananjara, 2023).

2. Neural Network PDF Parametrization

Each PDF is represented at the input scale $Q_0$ as

$$f_\ell(x, Q_0) = A_\ell\, x^{\alpha_\ell} (1-x)^{\beta_\ell}\, N_\ell(x; \theta)$$

where $\{A_\ell,\,\alpha_\ell,\,\beta_\ell\}$ are normalization and preprocessing exponents (possibly $A$ dependent in the nuclear case), and $N_\ell(x; \theta)$ is a feed-forward neural network:

• Input: $x$ (and optionally $\ln x$, $A$),
• Hidden layers: typically $n_1=25$, $n_2=20$ per flavor in NNPDF4.0, with $\tanh$ activation,
• Output: linear activation,
• The total number of weights and biases per PDF is typically $\sim 200$.

Preprocessing exponents $\alpha_\ell, \beta_\ell$ are chosen randomly within broad intervals and can be further optimized as hyperparameters. Sum rules (momentum conservation, valence) and PDF positivity are enforced via Lagrange-multiplier penalty terms appended to the loss function:

$$E^{(k)}(\theta) = \frac{1}{n_{\text{dat}}}\sum_{i,j=1}^{n_{\text{dat}}}(D_i^{(k)} - T_i^{(k)}(\theta))\, [\operatorname{Cov}^{-1}]_{ij}\, (D_j^{(k)} - T_j^{(k)}(\theta)) + \sum_i \lambda_i \Phi_i(\theta)$$

The penalty strengths $\lambda_i$ themselves are subject to hyperoptimization (Cruz-Martinez et al., 21 Oct 2024).

3. Statistical and Computational Methodology

Monte Carlo Replica Procedure and Loss Function

NNPDF generates $N_{\text{rep}}$ artificial data replicas $D^{(k)}$, each constructed by fluctuating the experimental measurements according to the full covariance matrix (statistical + systematic + normalization errors). For each replica, a separate network is trained by minimizing a regularized loss,

$$\chi^2(\theta^{(k)}) = [D^{(k)} - T(\theta^{(k)})]^{\top} \operatorname{Cov}^{-1} [D^{(k)} - T(\theta^{(k)})] + \sum_\ell \lambda_\ell\,R_\ell(\theta^{(k)}),$$

where $T(\theta)$ is the theoretical prediction for the relevant observables, obtained by convoluting the neural-net parameterized PDFs with precomputed FastKernel (FK) tables for rapid evaluation of DGLAP evolution and hard cross-sections (Ball et al., 2021).

After minimization, postfit filtering discards outlier replicas (e.g., worst 10% by $\chi^2$), stabilizing ensemble statistics.

Cross-Validation, Early Stopping, and Ensemble Metrics

Crucial to avoiding over- and underfitting is cross-validation. Each replica's data are split into training and validation sets (e.g., via $k$-fold, $K\sim4$), and training is halted when the validation loss ceases to improve. Model selection employs ensemble-based statistical estimators:

• First-moment metric (generalization $\chi^2_{\text{pdf}}$): averages both fit quality and PDF spread.
• Second-moment metric ($\varphi^2$): measures PDF uncertainty in data space, particularly for extrapolation.
• $\varphi^2_{\chi^2}$ is defined as the difference between the mean replica $\chi^2$ and the $\chi^2$ of the mean prediction, and can be expressed through the prediction covariance projected onto data space.

Optimal hyperparameters minimize $L^{(\chi^2_{\text{pdf}})}$ and maximize $L^{(\varphi^2)}$, with $n_{\text{hopt}}\approx 10$ equally-good hyperparameter sets selected for replica ensemble training (Cruz-Martinez et al., 21 Oct 2024).

Hyperparameter Optimisation

The NNPDF hyperparameter set $\eta$ includes network sizes, optimizer type, clipnorm, learning rate, penalty strengths, and early stopping criteria. Bayesian optimization is performed via the Hyperopt package, guided by the aforementioned ensemble metrics, and is fully parallelized across hardware accelerators. Hyperparameter scans are distributed asynchronously over multiple GPUs (e.g., via MongoDB and Hyperopt), achieving near-linear walltime reductions with increased throughput.

4. GPU Implementation and Scaling

The major computational breakthrough is the GPU-optimized, stacked-replica workflow:

• All $N_{\text{rep}}$ neural networks are embedded in a single large model, sharing a pass through the FK-table convolution and $\alpha_s$ evolution. Distinctions between training and validation sets are enforced via per-replica masks in the computation graph.
• FastKernel tables are restructured via PineAPPL to unify $x$-grid points across all datasets, reducing memory requirements by a factor $\sim 80$ (e.g., $n_x\sim 50$ rather than $\sim 4000$ interpolation nodes).
• Tensor contractions are reordered for memory efficiency; with replica indices contiguous and contraction order optimized.
• The net result is a performance increase by $\mathcal{O}(100)$: $\sim 0.5$ replicas/hour (16-CPU) versus $\sim 60$ replicas/hour (single H100 GPU).
• GPU memory usage saturates at $\sim 9$GB for $N_{\text{rep}}=100$, a factor $\sim2$ reduction over previous implementations, with energy consumption drops of $78\%$–$91\%$ and compute-hour cost reductions of $45\%$–$55\%$ as $N_{\text{rep}}$ is increased.
• This architecture is generalizable to any ensemble-based deep learning task requiring parallelized hyperparameter and model training (Cruz-Martinez et al., 21 Oct 2024).

5. Uncertainty Quantification and Validation

Uncertainties are interpreted strictly in terms of replica ensemble variance, with no implicit linear-Gaussian assumptions:

• For any physical observable or PDF combination $\mathcal{O}$, the $N_{\text{rep}}$ predictions define the empirical mean and standard deviation.
• Correlated variations (e.g., $\alpha_s$ scans, theory systematics) are built by correlated seeding and shared data partitions.
• Filtering of non-convergent or outlier fits (e.g., discarding the highest $\chi^2$ decile) prior to ensemble averaging further stabilizes uncertainty bands without introducing bias.
• Closure tests, where pseudo-data generated from a known PDF set are re-fitted, confirm that coverage and bias are under precise control.
• For integrated proton/nuclear fits, the ensemble captures interpolation and extrapolation errors across the $A$ and $x$ domains; oscillatory features or larger errors in some regions reflect data density and rank-deficiency, not methodological artifacts.

6. Physical Results, Generalisation, and Impact

• The NNPDF4.0 hyperparameter-optimized fits (nnpdf40_newhyperopt) deliver global fit quality with $\chi^2 \approx 1.15$ (to be compared with $1.16$ for baseline NNPDF4.0) and essentially identical breakdowns for various process subsets (DIS, Drell–Yan, jets, $W/Z$, $t\bar{t}$).
• Training and validation losses are unchanged; average training length (epochs) is reduced by $\sim 15\%$.
• The ensemble $\varphi^2_{\chi^2}$ estimator increases by $\sim 10\%$, indicating a modest increase in PDF uncertainties. In the most constrained regions ($x\sim 10^{-3}$–$10^{-1}$), light quark uncertainties grow by $5$–$10\%$, while gluon and charm remain unchanged.
• LHC quark–antiquark and quark–gluon luminosity uncertainties grow by $\sim 10\%$ at low masses, but central values are statistically unchanged.
• Computational throughput gains and energy/cost savings are of order $>50\%$ for typical global-fit-scale production runs.
• The fully ensemble-based strategy and GPU implementation are transparent and reproducible, supported by open-source code, detailed documentation, and robust validation pipelines (Ball et al., 2021).

7. Extensions and Future Directions

Recent and ongoing NNPDF research programs include:

• Support for approximate N³LO evolution via constraint-based four-loop splitting kernels and uncertainty band prescriptions (MHOU/IHOU) (Hekhorn et al., 2023).
• Simultaneous, integrated proton-deuteron-nucleus PDF fits, placing all $A$-dependence on optical parity with $x$ and extending the neural network to composite inputs $(x, A)$ (Rabemananjara, 2023); global minimizations in this space are ongoing.
• General applicability of the hyperparameter-tuned, GPU-parallel framework to other deep-learning tasks involving large ensembles and bias/variance trade-offs.
• Systematic inclusion of resummation (small-$x$, threshold), mixed QCD-QED effects, theory-covariance from missing higher orders, and direct fits to lattice and other nonperturbative data.

The NNPDF framework constitutes a scalable, flexible, and transparently validated solution for large-scale, uncertainty-aware inference in QCD and particle physics, with modularity and generalisability for other ensemble-based statistical problems in computational science and deep learning.