Sample-Then-Optimize Approach

• The Sample-Then-Optimize approach decouples experimental design and sampling from subsequent optimization to guide the construction of efficient schemes.
• Rigorous analysis shows that optimal designs for measuring $n$ parameters in scan experiments require exactly $n$ points, selected by maximizing parameter-specific auxiliary functions.
• This methodology provides practical guidelines for optimally allocating experimental resources like luminosity across key scan points, leading to significant statistical efficiency gains.

The sample-then-optimize approach refers to a class of methodologies that explicitly decouple the initial sampling or design phase from the subsequent optimization or analysis, using the results or properties of the samples to guide and construct the optimal experimental or decision scheme. The referenced work provides a rigorous mathematical foundation for this approach in the context of scan experiments, particularly in high-energy physics. The main problem considered is how to design a scan—selecting energy points and allocating luminosity—so that scientific parameters of interest are estimated with maximal statistical efficiency.

1. Sampling-Based Scheme Design

The initial phase of the sample-then-optimize strategy employs a sampling-based evaluation of candidate scan schemes. Specifically, a simulated scan is constructed by discretizing the energy interval of interest into $N_{pt}$ points, assigning equal or variant luminosity, simulating observed event counts at each point (typically via Poisson processes), and then fitting the parameter(s) of interest using repeated pseudo-experiments. For each scheme, the spread (empirical standard deviation) of the fitted parameters across many simulation runs is computed to assess the efficacy of that scan scheme for parameter determination.

This method allows empirical comparison of many possible configurations, providing insight into which scan points and luminosity distributions yield the most precise parameter estimates. Notably, for a single parameter, only one well-chosen scan point (where the observable's derivative with respect to the parameter is maximized) is empirically found to be optimal. For multi-parameter problems, sampling identifies optimal energy allocations for each parameter (the "independence conjecture"), sidestepping intractable combinatorial scans.

2. Analytical Foundations and Auxiliary Functions

Building upon insights from sampling, the approach advances to rigorous analytical optimization. The scan experiment is formalized as an optimization problem where the goal is to minimize the variance of the estimated parameters, given by the inverse of the Fisher information (or Hesse matrix in nonlinear least squares). The key mathematical result is that, under general conditions, the optimal design for $n$ parameters involves exactly $n$ scan points, with each scan point positioned at the extremum of an auxiliary function:

$$g(\theta; E) = \frac{1}{\sigma(\theta; E)} \left(\frac{\partial \sigma(\theta; E)}{\partial \theta}\right)^2.$$

The auxiliary function quantifies the statistical sensitivity of each energy point for estimating a given parameter. Theorems in the paper prove the uniqueness and sufficiency of placing all luminosity for each parameter at its respective maximizer of $g$, and more generally that $n$ parameters require $n$ points for optimal joint estimation.

3. Second Optimization Theory

The "second optimization" is a theoretical construct which generalizes the principle of dual optimization: the first optimization fits parameters for a fixed scan (data), the second optimization selects scan design $(E_i, x_i)$ to minimize final statistical errors or maximize information. The analytical results show that for $n$ parameters, one should first identify the most informative scan point for each parameter (using the auxiliary function) and then allocate total experimental luminosity among these $n$ points. The independence conjecture from the sampling phase—that parameter uncertainties can be minimized independently—receives full analytical justification using affine transformations and Cholesky decomposition of the Fisher information.

4. Optimal Luminosity Allocation

Given the $n$ optimal points for $n$ parameters, the next analytical challenge is how to allocate the experimental resource (luminosity) among them to reflect the relative scientific importance of the parameters. Introducing weight factors $w_i$ for each parameter, the variance of each estimator is given by

$$v_{ii} = \frac{1}{L\epsilon} \sum_{l=1}^n \frac{\alpha_{li}^2 \sigma_l^*}{x_l},$$

where $\alpha_{li}$ encodes sensitivity, $\sigma_l^*$ is the cross-section at point $l$, and $x_l$ is the luminosity fraction. The allocation is then found by minimizing the weighted sum $\sum_{i=1}^n w_i v_{ii}$ under the constraint $\sum_l x_l = 1$, using Lagrange multipliers. The solution is

$$x_p^* \propto \sqrt{\sum_{i=1}^n w_i \alpha_{pi}^2 \sigma_p^*},$$

ensuring that more luminosity is assigned where it is most effective for reducing uncertainty in higher-priority parameters.

5. Scientific and Operational Implications

The rigorous implementation of the sample-then-optimize methodology produces practical and general guidelines for experimental design:

• For $n$ physical parameters, $n$ energy points are necessary and sufficient for optimal measurement.
• Each optimal point is chosen by maximizing the parameter-specific auxiliary function, forming a constructive, scalable design mechanism.
• The analytical solution for luminosity allocation enables tailoring for specific research goals, maximizing precision where it is most scientifically or economically valuable.
• Empirical applications (e.g., at BESIII for tau mass measurement) have demonstrated substantial improvements in statistical efficiency, operational cost, and adaptability for new experimental setups.

A comparative table in the work contrasts the empirical "sample-then-optimize" method with pure analytical optimization, highlighting aspects such as empirical confirmation of $n$-point designs, analytic parameter selection, and the robustness of the approach when models are complex or priors are uncertain.

6. Summary Table: Key Elements

Element	Sampling-Then-Optimize	Analytical Optimization
Energy point selection	Empirical (via simulation)	Auxiliary function maximization
Number of scan points	Converges empirically to $n$	Proven: exactly $n$ required
Luminosity allocation	Trend from simulated results	Lagrange multiplier analytic
Multi-parameter treatment	Empirical "independence"	Analytical independence proven
Applicability	Confirms analytic for complex models	General for physical models

The sample-then-optimize approach advanced in this work provides a universally applicable, rigorously justified method for resource-optimal, high-precision experimental design in scan experiments and may inform analogous methodologies in other parameter estimation and experimental planning domains.