End-to-End Optimized Summary Statistics

• The paper demonstrates that end-to-end optimized summary statistics replace separate feature extraction and inference, directly training summaries to optimize inference objectives like CLs and posterior entropy.
• This method leverages differentiable pipelines—employing techniques such as kernel density estimation and differentiable histograms—to robustly handle high-dimensional detector outputs and finite Monte Carlo statistics.
• The approach is validated in various settings, including high-energy physics and likelihood-free inference, improving sensitivity and reducing bias by aligning the summarization with the final analysis goal.

Searching arXiv for the cited papers to ground the article in current records. arxiv_search(query="(Simpson et al., 2022)", max_results=5) arxiv_search(query="(Janik et al., 9 Jul 2025)", max_results=5) arxiv_search(query="(Hoffmann et al., 2022)", max_results=5) arxiv_search(query="end-to-end optimized summary statistics inference-aware summary statistic arXiv", max_results=10) End-to-end optimized summary statistics are learned low-dimensional representations whose parameters are trained against the final inferential objective of an analysis rather than against a proxy task such as nominal signal-background classification or heuristic feature design. In high-energy physics, this formulation arises because detector-level observations are high-dimensional, exact likelihoods are often intractable, and practical analyses therefore compress events to histogrammed observables before performing likelihood-based inference (Simpson et al., 2022). In IceCube, the same issue appears in a binned forward-folding likelihood whose granularity is limited by finite Monte Carlo statistics (Janik et al., 9 Jul 2025). In likelihood-free inference and ABC, summary learning addresses the curse of dimensionality by replacing raw data with low-dimensional compressions that preserve posterior-relevant information (Hoffmann et al., 2022). Across these settings, the common principle is to learn the compression map through the downstream inferential criterion itself.

1. Conceptual basis

End-to-end optimization replaces the conventional separation between representation learning and statistical inference with a single computational graph. In the high-energy-physics formulation introduced by neos, the analysis is written as a composition

$\mathrm{CL_s} = f(\mathcal{D},\varphi) = \bigl(f_{\mathrm{sensitivity} \circ f_{\mathrm{test\,stat} \circ f_{\mathrm{likelihood} \circ f_{\mathrm{histogram} \circ f_{\mathrm{observable}\bigr)(\mathcal{D},\varphi),$

where $\mathcal{D}$ is the dataset and $\varphi$ are the parameters of the learnable summary statistic (Simpson et al., 2022). The trainable object is therefore not merely a classifier but the summary statistic itself.

The motivation is a mismatch between local training objectives and global analysis goals. In standard collider workflows, one often trains a classifier to separate signal from nominal background, then histograms the resulting score and evaluates the analysis using a profile-likelihood-based $p$-value or $\mathrm{CL}_s$. The paper argues that this mismatch can become severe in the presence of systematic uncertainties, because a score that separates nominal samples well may encode directions strongly entangled with nuisance variations and thus perform poorly after nuisance profiling (Simpson et al., 2022).

The same structural issue appears in IceCube’s diffuse Galactic-neutrino analysis. The standard approach is a binned forward-folding likelihood in reconstructed observables such as energy and direction, but increasing dimensionality or bin granularity quickly produces sparsely populated MC bins. The learned summary statistic is introduced precisely to compress more observables into a lower-dimensional representation that remains compatible with template-based likelihood analysis (Janik et al., 9 Jul 2025).

In ABC and related likelihood-free settings, the motivation is formulated in posterior terms. The 2022 unification paper distinguishes sufficient, lossless, and optimal summaries, then frames practical summary learning as the search for a low-dimensional compressor that preserves the posterior as well as possible within a restricted function class (Hoffmann et al., 2022). This suggests a unifying interpretation: end-to-end optimized summary statistics are not defined by a specific architecture or domain, but by the fact that the compression map is optimized through the inferential objective that will ultimately be used.

2. Inferential objectives

The defining feature of the literature is that the loss function is analysis-level or posterior-level rather than predictive in the ordinary supervised-learning sense. Different papers instantiate this principle with different criteria.

Setting	Learned object	Optimization target
neos (Simpson et al., 2022)	one-dimensional summary statistic $f_\varphi(d_i)$	expected $\mathrm{CL}_s$ from asymptotic profile-likelihood machinery
IceCube Galactic fit (Janik et al., 9 Jul 2025)	low-dimensional $lss$, best case $(lss_1,lss_2)$	expected variance of $\Psi_{\mathrm{CRINGE}}$ via Fisher information
ABC / likelihood-free inference (Hoffmann et al., 2022)	summary map $\mathcal{D}$0 with posterior model	expected posterior entropy

In neos, each event $\mathcal{D}$1 is mapped to

$\mathcal{D}$2

and the objective is the expected $\mathcal{D}$3, obtained through asymptotic profile-likelihood theory and the Asimov dataset (Simpson et al., 2022). The paper explicitly contrasts this with INFERNO, which optimizes a Hessian- or Fisher-derived variance estimate $\mathcal{D}$4. The key claim is that optimizing an inferential proxy such as $\mathcal{D}$5 is not identical to optimizing the final expected sensitivity, and the reported experiments show exactly that distinction (Simpson et al., 2022).

In the IceCube application, the differentiable objective is the expected variance of the Galactic-template normalization $\mathcal{D}$6. The Fisher information matrix is defined as

$\mathcal{D}$7

with the Cramér–Rao relation

$\mathcal{D}$8

and the loss is built from the expected variance of the signal parameter or a weighted sum of variances for multiple signal parameters (Janik et al., 9 Jul 2025).

In the ABC unification paper, the central criterion is expected posterior entropy (EPE) under the prior predictive distribution: $\mathcal{D}$9 The paper proves that minimizing EPE is exactly equivalent to maximizing expected surprise, maximizing mutual information $\varphi$0, and minimizing the expected KL divergence between the full posterior and the posterior conditioned on the summaries (Hoffmann et al., 2022). This is the most general theoretical statement in the cited literature: the optimal summary is the one that preserves posterior information, not merely one that improves a point estimate or a discrimination score.

A plausible implication is that end-to-end summary optimization is best understood as a family of inference-aware objectives rather than a single loss. The precise loss depends on whether the downstream problem is exclusion, parameter estimation, or likelihood-free posterior approximation.

3. Differentiable pipelines and optimization mechanics

The technical difficulty is that classical inference pipelines contain non-differentiable or nested-optimization components. The literature therefore focuses on replacing these components with differentiable surrogates or differentiable estimators.

In neos, ordinary histogramming is replaced with a differentiable surrogate based on a kernel density estimate: $\varphi$1 which is then integrated over bin intervals to produce a binned KDE with per-bin probabilities $\varphi$2 (Simpson et al., 2022). The paper emphasizes a bandwidth trade-off: in the zero-bandwidth limit, the method approaches ordinary histogramming, but gradients become unstable and high-variance; larger bandwidth smooths the objective and improves optimization but introduces bias.

The same work then builds a HistFactory likelihood through pyhf, with the profile likelihood ratio

$\varphi$3

and differentiates through the maximum-likelihood and profiled-nuisance computations by implicit differentiation of optimization routines: $\varphi$4 This avoids unrolling all iterations of the inner optimizer and is implemented with jaxopt in a JAX-style automatic-differentiation workflow (Simpson et al., 2022).

The IceCube implementation uses a different differentiable histogramming strategy. For one dimension, the hard bin indicator is replaced by

$\varphi$5

and multi-dimensional histograms are formed by outer products of these soft indicators (Janik et al., 9 Jul 2025). Because finite MC statistics are central to the problem, the method also introduces a differentiable mask based on relative MC uncertainty,

$\varphi$6

so that bins with poor MC support contribute little or nothing to the Fisher-information objective (Janik et al., 9 Jul 2025).

In the ABC framework of expected posterior entropy, differentiability is achieved by coupling a learnable compressor $\varphi$7 to a conditional density estimator $\varphi$8 and minimizing

$\varphi$9

This is a Monte Carlo estimator of EPE and is optimized jointly over compressor and posterior-model parameters (Hoffmann et al., 2022). The paper stresses that this is end-to-end with respect to raw data $p$0 summary $p$1 posterior density estimator, but not with respect to differentiating through the acceptance step of rejection ABC itself.

4. High-energy-physics and astroparticle implementations

The most explicit HEP realization is neos, a fully differentiable high-energy-physics analysis pipeline whose trainable component is a learnable one-dimensional summary statistic (Simpson et al., 2022). The demonstration benchmark is a toy 2D Gaussian-blob problem with signal, nominal background, and one nuisance parameter represented by up/down background templates. The network is a 3-layer MLP with dimensions $p$2, ReLU activations, and a sigmoid final layer; training uses Adam at learning rate $p$3, the inner maximum-likelihood fits also use Adam at learning rate $p$4, training runs for 15 epochs with batch size 2000, and test-set metrics are averaged over seven random initializations (Simpson et al., 2022). The reported empirical result is that neos achieves the lowest expected $p$5 among the compared methods, while INFERNO achieves the best $p$6 uncertainty but not the best expected sensitivity. The paper also reports that neos and BCE with systematic augmentation keep the nuisance-parameter metric comparatively low, whereas plain BCE and INFERNO show signs of over- or under-constraining the nuisance parameter (Simpson et al., 2022).

The IceCube proceedings paper applies the same broad idea to a diffuse Galactic-neutrino fit in the Northern Tracks sample of upgoing muon tracks (Janik et al., 9 Jul 2025). The baseline analysis uses a standard 3D histogram with 45 bins in reconstructed energy, 33 bins in reconstructed cosine zenith, and 180 bins in reconstructed right ascension, for a total of $p$7 bins. The optimized analysis instead learns a two-dimensional statistic $p$8, bins each learned coordinate into 40 bins, and therefore uses $p$9 bins total (Janik et al., 9 Jul 2025). The inputs include the baseline observables plus additional reconstruction-quality variables, including an estimate of the angular uncertainty and variables previously used to separate upgoing from downgoing muon tracks.

The final evaluation uses Asimov likelihood scans with MC only. In the CRINGE normalization fit, both standard and optimized analyses recover the injected value of $\mathrm{CL}_s$0, but under the Poisson likelihood the baseline excludes $\mathrm{CL}_s$1 at

$\mathrm{CL}_s$2

corresponding to about $\mathrm{CL}_s$3, whereas the optimized binning excludes $\mathrm{CL}_s$4 at

$\mathrm{CL}_s$5

corresponding to about $\mathrm{CL}_s$6 (Janik et al., 9 Jul 2025). The paper further reports that the $\mathrm{CL}_s$7 interval on the estimator improves by about $\mathrm{CL}_s$8, and that under an effective likelihood accounting for finite MC statistics the optimized method still yields a similar interval improvement together with reduced bias (Janik et al., 9 Jul 2025).

Taken together, these papers establish that end-to-end optimized summary statistics can be used either to learn nuisance-aware summary observables for profile-likelihood analyses or to compress detector observables into low-dimensional spaces that alleviate the MC-statistics bottleneck in forward-folding likelihood fits.

5. Relation to ABC and neighboring approaches

In likelihood-free inference, the 2022 EPE paper provides the most explicit general theory of learned summaries (Hoffmann et al., 2022). It defines a summary as lossless if

$\mathrm{CL}_s$9

and defines an optimal summary within a class $f_\varphi(d_i)$0 by minimizing a discrepancy between the posterior given full data and the posterior given summaries. Its main result is that minimizing expected posterior entropy under the prior predictive subsumes or unifies several earlier approaches, including mutual-information objectives, expected surprise, and minimizing expected posterior KL (Hoffmann et al., 2022).

The same paper proposes a practical method, MDN compression, in which a bottlenecked summary network is trained jointly with a conditional mixture density network. On the benchmark model with multimodal posterior and deceptive low-order moments, MDN-compressed ABC essentially matches likelihood-based inference and can outperform direct MDN posterior samples when the MDN family is too restrictive. On population-genetics and growing-tree examples, the learned summaries remain competitive and are particularly strong under negative log probability when the posterior is multimodal or the likelihood is genuinely intractable (Hoffmann et al., 2022).

A neighboring but distinct line of work is the adaptive-weighting ABC-SMC method of 2017. That paper optimizes a weight vector $f_\varphi(d_i)$1 in a weighted Euclidean distance over a fixed summary vector,

$f_\varphi(d_i)$2

and chooses $f_\varphi(d_i)$3 to maximize a Hellinger-distance-based discrepancy between the approximate posterior and the prior (Harrison et al., 2017). The paper is explicit that it does not learn new summaries from raw data; its contribution is adaptive weighting of handcrafted summaries. It is therefore best understood as adaptive metric learning in summary space rather than end-to-end summary learning in the modern sense (Harrison et al., 2017).

This distinction matters because several misconceptions recur in the literature. End-to-end optimization does not require the summary to be physically interpretable, does not imply sufficiency, and does not reduce to reweighting a fixed candidate summary vector. In the EPE formulation, maximizing mutual information or minimizing expected posterior entropy yields an optimal summary in the chosen function class, not necessarily a sufficient statistic (Hoffmann et al., 2022).

6. Misconceptions, limitations, and extensions

A first common misconception is that any inference-aware proxy is equivalent to the final analysis objective. The neos study explicitly contradicts this: INFERNO is designed around $f_\varphi(d_i)$4 and indeed obtains the best $f_\varphi(d_i)$5 uncertainty, but it does not achieve the best expected $f_\varphi(d_i)$6 (Simpson et al., 2022). The distinction between optimizing a proxy and optimizing the final inferential quantity is one of the central empirical points of the paper.

A second misconception is that end-to-end optimization simply means “train a classifier and histogram its score.” The papers show otherwise. In neos, the loss includes nuisance profiling and systematic templates, so the representation is driven away from directions where signal-background separation is strongly entangled with nuisance variation (Simpson et al., 2022). In IceCube, the learned statistic is trained for parameter precision, not AUC or classification accuracy, and the training includes a soft mechanism that suppresses poorly populated MC bins (Janik et al., 9 Jul 2025). In the EPE framework, the compressor is trained for posterior fidelity rather than posterior mean prediction or classifier performance (Hoffmann et al., 2022).

The practical limitations are substantial. In neos, training is roughly a factor of three slower than BCE on the toy problem because each batch includes all downstream analysis steps; large batches are required so that the batch adequately represents the analysis and avoids pathologies such as empty bins; and the differentiable histogram introduces a bandwidth hyperparameter that affects both stability and fidelity to standard histograms (Simpson et al., 2022). The authors state that scaling to realistic LHC problems with many nuisance parameters remains future work (Simpson et al., 2022). In IceCube, the objective is local and model-dependent because it is tied to the assumed parameterization and the Fisher-information approximation; the proceedings paper also leaves many implementation details abbreviated (Janik et al., 9 Jul 2025). In the EPE-based ABC setting, the approach still requires many simulations from the prior predictive and depends on the expressiveness and stability of the conditional density estimator (Hoffmann et al., 2022).

The cited literature also points toward broader optimization targets once the summary-statistic pipeline is differentiable. The neos paper mentions differentiably approximating cuts with sigmoids to optimize preselection, optimizing bin edges or binning schemes for maximal sensitivity, and comparing differentiable methods with black-box procedures such as Bayesian optimization (Simpson et al., 2022). The IceCube paper notes that the same method is already being applied to other analyses such as astrophysical neutrino flavor composition (Janik et al., 9 Jul 2025). A plausible implication is that end-to-end optimized summary statistics are part of a larger shift in which the full analysis workflow—compression, binning, nuisance treatment, and even selection design—is treated as an object of optimization rather than as a fixed downstream consumer of learned features.