Measurement-Optimization Gap

• Measurement-Versus-Optimization Gap is the misalignment between observed proxy metrics and the true deployment criteria across diverse optimization tasks.
• It manifests in hyperparameter tuning, energy-flow control, recommender systems, and deep reinforcement learning, often leading to misleading conclusions about performance.
• Understanding and quantifying this gap is crucial for designing robust experimental protocols and ensuring that optimization targets truly reflect real-world objectives.

Taken together, the cited works use the measurement-versus-optimization gap to denote a family of mismatches in which the quantity that is measured, predicted, selected, or validated is not the quantity that determines the real objective. In multi-objective hyperparameter optimization, an approximation of the Pareto front on the validation split can fail to remain Pareto-optimal on the test split (Feurer et al., 2022). In energy-flow control, node rankings induced by centrality do not necessarily identify the interventions that minimize disturbance amplification (Li et al., 23 Aug 2025). In recommender systems and computational semantics, engagement signals or predictive utility can diverge from latent Value or from scientific usability as a meaning representation (Milli et al., 2020, Plisiecki, 10 Mar 2026). Across these settings, the gap is consequential because optimization against a proxy, or evaluation with a misaligned protocol, can give systematically misleading conclusions about robustness, identifiability, or real-world effectiveness.

1. General structure of the gap

A recurring structure in the cited literature is a separation between an observable proxy and the criterion that actually matters at deployment or inference time. The observable proxy may be a validation-set Pareto approximation, a centrality score, gate depth, an engagement signal, a predictive benchmark, a measurement budget allocation, or a theorem-friendly analytical quantity. The deployment criterion may instead be test-set Pareto stability, closed-loop $L_1$-gain, logical error rate, latent value, semantic interpretability, parameter identifiability, or actual optimizer dynamics (Feurer et al., 2022, Li et al., 23 Aug 2025, Viszlai et al., 24 Jan 2026, Milli et al., 2020, Tran et al., 2024).

The gap is not always caused by simple estimation noise. In several papers, it arises because the surrogate itself has the wrong invariances or the wrong geometry. In syndrome-measurement circuits, minimizing gate count or depth does not encode how faults propagate through CNOT schedules and become ambiguous logical errors (Viszlai et al., 24 Jan 2026). In model-based design of experiments, the cheapest or easiest measurements are not necessarily those that maximize information about unknown parameters under a Fisher-information criterion (Wang et al., 2024). In networks of pairwise differences, the resource allocation that minimizes individual measurement variance is not the same as the allocation that minimizes the covariance of the final inferred vector $\vec{x}$ (Xu, 2019).

A plausible implication is that the term names not a single theorem or metric, but a diagnostic pattern: whenever optimization is conducted over a measured, estimated, or operationalized quantity, the scientific question becomes whether that quantity preserves the ranking, geometry, or inferential content of the true objective.

2. Pareto generalization in multi-objective hyperparameter optimization

In multi-objective hyperparameter optimization, the gap is formalized as a mismatch between the Pareto structure found on validation data and the Pareto structure that survives on test data. The underlying objective is

$$c : \Lambda \to \mathbb{R}^M,\qquad \min_{\lambda\in\Lambda} c(\lambda)=\min_{\lambda\in\Lambda}(c_1(\lambda),\dots,c_M(\lambda)),$$

with dominance defined by

$$c^{(1)} \prec c^{(2)} \iff \forall k\in\{1,\dots,M\}: c_k^{(1)}\le c_k^{(2)} \ \land\ \exists l\in\{1,\dots,M\}: c_l^{(1)}<c_l^{(2)}.$$

The Pareto front is

$$\mathcal{P}=\{c(\lambda)\mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\},$$

and the Pareto set is its pre-image

$$\mathcal{P}_\Lambda = c^{-1}(\mathcal{P})=\{\lambda \mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\}.$$

The central empirical phenomenon is that a configuration that is Pareto-optimal on validation may become dominated on test, while a dominated validation solution may become non-dominated on test (Feurer et al., 2022).

The paper isolates three regimes: all validation Pareto points remain non-dominated on test; the same points remain non-dominated but change order; and some validation Pareto points become dominated on test. The third regime is the key measurement-versus-optimization gap. The optimizer has improved the validation Pareto front, but the measured test Pareto front does not preserve the same set of trade-offs (Feurer et al., 2022).

A naïve response is to recompute the Pareto front on the test set and then measure hypervolume, but the paper identifies two problems. First, this overestimates generalization performance because dominated validation points are discarded even though a practitioner may still choose one of those points based on validation results. Second, it creates test-data leakage because a method can return many configurations and let the test set determine which are counted as Pareto-optimal (Feurer et al., 2022).

The proposed protocol evaluates the validation-selected Pareto set on test and constructs an optimistic Pareto front and a pessimistic Pareto front over those same returned configurations. Their hypervolumes define the approximation gap,

$\text{gap} = HV(\mathcal{P}_{\text{pessimistic})-HV(\mathcal{P}_{\text{optimistic}).$

A zero gap means the validation Pareto set transfers perfectly to test; a positive gap means some returned hyperparameter configurations are dominated on the test set. The protocol then supports three comparison criteria between optimization experiments: a hypervolume difference criterion, a Pareto dominance criterion, and an approximation gap criterion (Feurer et al., 2022).

The experiments make the abstract point concrete. On the German credit dataset, with a random forest tuned by random search for precision and recall, the validation Pareto set does not necessarily remain Pareto-optimal on test. In a second experiment, validation hypervolume can improve monotonically with more evaluations while the optimistic and pessimistic test-side hypervolumes do not improve in sync; at 50 evaluations the linear model is better by the hypervolume-difference criterion, at 100 and 500 evaluations no conclusion is justified, and at 200 evaluations the random forest is better (Feurer et al., 2022).

3. Structural proxies versus dynamical and logical objectives

In energy-flow networks, the gap appears between topological importance measurements and the optimization target of closed-loop stability. The system is modeled as a positive, directed resource-flow network adapted from the buffer-based framework of Rantzer and Valcher, with state $y_i(t)$, linear flow law

$$g_i^\mathrm{out}=\kappa_i\,y_i, \qquad v_{ij}=\alpha_{ij}\,s_{ij}\,y_i,$$

and stability optimization posed as

$$\min_{\alpha_{ij} \in \Theta} \quad \|\Sigma_\theta\|_1$$

subject to

$\vec{x}$0

The operational comparison is: compute a centrality ranking, select a fixed fraction of the highest-ranked nodes, allocate additional resources or gains to those nodes, and evaluate the resulting system by its $\vec{x}$1-norm or $\vec{x}$2-gain (Li et al., 23 Aug 2025).

Five centrality measures are compared: Katz centrality, eigenvector centrality, closeness centrality, betweenness centrality, and PageRank. The simulations use $\vec{x}$3 random graphs per topology, box plots with median, interquartile range, and outliers, 95% bootstrap confidence intervals with 10,000 resamples, and a two-sided Wilcoxon signed-rank test with $\vec{x}$4 indicating $\vec{x}$5. The main finding is that node centrality and control effectiveness are not equivalent: betweenness and eigenvector centrality often produce the best stability outcomes, closeness centrality tends to perform worse, Katz and PageRank are competitive but more topology-sensitive, and the ranking depends strongly on whether the network is scale-free, small-world, or block-diagonal (Li et al., 23 Aug 2025).

A closely related but more microscopic form of the gap appears in fault-tolerant quantum computing. In syndrome-measurement circuits, existing NISQ-era tools optimize targets such as gate depth or gate count, but these targets do not model how errors propagate through the circuit and determine the code’s logical error rate. The paper’s critique is that a minimum-depth schedule can be worse than a slightly deeper one if the CNOT ordering causes harmful hook errors, and that even circuits with the same effective distance $\vec{x}$6 can have very different logical error rates because the ambiguity structure differs (Viszlai et al., 24 Jan 2026).

PropHunt replaces depth minimization with a search for ambiguous errors, defined by

$\vec{x}$7

Combining two such patterns yields an undetected logical error,

$\vec{x}$8

Operationally, PropHunt builds the circuit-level decoding graph, finds subgraphs likely to contain ambiguous errors, solves for minimum-weight logical errors in those subgraphs using MaxSAT, and modifies CNOT ordering to remove ambiguity. The paper reports that depth-only tools can suffer as much as a 10× increase in logical error rate on surface codes, that PropHunt reduces logical error rates by 2.5× to 4× on Lifted Product and Random Quantum Tanner codes at $\vec{x}$9, and that Hook-ZNE achieves 3× to 6× lower error than Distance-Scaling ZNE (Viszlai et al., 24 Jan 2026).

4. Measurement networks, Fisher information, and optimal experiment design

In measurement-network design, the gap is between raw measurements and the best joint inference obtainable from them. For unknown quantities $$c : \Lambda \to \mathbb{R}^M,\qquad \min_{\lambda\in\Lambda} c(\lambda)=\min_{\lambda\in\Lambda}(c_1(\lambda),\dots,c_M(\lambda)),$$0, one may measure individual values $$c : \Lambda \to \mathbb{R}^M,\qquad \min_{\lambda\in\Lambda} c(\lambda)=\min_{\lambda\in\Lambda}(c_1(\lambda),\dots,c_M(\lambda)),$$1 or pairwise differences $$c : \Lambda \to \mathbb{R}^M,\qquad \min_{\lambda\in\Lambda} c(\lambda)=\min_{\lambda\in\Lambda}(c_1(\lambda),\dots,c_M(\lambda)),$$2, with measurement variances

$$c : \Lambda \to \mathbb{R}^M,\qquad \min_{\lambda\in\Lambda} c(\lambda)=\min_{\lambda\in\Lambda}(c_1(\lambda),\dots,c_M(\lambda)),$$3

under a total budget

$$c : \Lambda \to \mathbb{R}^M,\qquad \min_{\lambda\in\Lambda} c(\lambda)=\min_{\lambda\in\Lambda}(c_1(\lambda),\dots,c_M(\lambda)),$$4

Assuming independent Gaussian errors, the maximum-likelihood estimator satisfies $$c : \Lambda \to \mathbb{R}^M,\qquad \min_{\lambda\in\Lambda} c(\lambda)=\min_{\lambda\in\Lambda}(c_1(\lambda),\dots,c_M(\lambda)),$$5, with covariance

$$c : \Lambda \to \mathbb{R}^M,\qquad \min_{\lambda\in\Lambda} c(\lambda)=\min_{\lambda\in\Lambda}(c_1(\lambda),\dots,c_M(\lambda)),$$6

The design problem is then to choose the allocation $$c : \Lambda \to \mathbb{R}^M,\qquad \min_{\lambda\in\Lambda} c(\lambda)=\min_{\lambda\in\Lambda}(c_1(\lambda),\dots,c_M(\lambda)),$$7 that minimizes $$c : \Lambda \to \mathbb{R}^M,\qquad \min_{\lambda\in\Lambda} c(\lambda)=\min_{\lambda\in\Lambda}(c_1(\lambda),\dots,c_M(\lambda)),$$8, $$c : \Lambda \to \mathbb{R}^M,\qquad \min_{\lambda\in\Lambda} c(\lambda)=\min_{\lambda\in\Lambda}(c_1(\lambda),\dots,c_M(\lambda)),$$9, or $$c^{(1)} \prec c^{(2)} \iff \forall k\in\{1,\dots,M\}: c_k^{(1)}\le c_k^{(2)} \ \land\ \exists l\in\{1,\dots,M\}: c_l^{(1)}<c_l^{(2)}.$$0, corresponding to A-, D-, and E-optimality (Xu, 2019).

The core point is that one does not directly optimize measurement quality edge by edge; one optimizes the geometry of the Fisher information matrix. The paper shows that optimized designs substantially outperform naïve allocations. For random noise scales $$c^{(1)} \prec c^{(2)} \iff \forall k\in\{1,\dots,M\}: c_k^{(1)}\le c_k^{(2)} \ \land\ \exists l\in\{1,\dots,M\}: c_l^{(1)}<c_l^{(2)}.$$1 with $$c^{(1)} \prec c^{(2)} \iff \forall k\in\{1,\dots,M\}: c_k^{(1)}\le c_k^{(2)} \ \land\ \exists l\in\{1,\dots,M\}: c_l^{(1)}<c_l^{(2)}.$$2, the A-optimal network outperforms all tested naïve schemes on all criteria; compared with the best naïve allocation $$c^{(1)} \prec c^{(2)} \iff \forall k\in\{1,\dots,M\}: c_k^{(1)}\le c_k^{(2)} \ \land\ \exists l\in\{1,\dots,M\}: c_l^{(1)}<c_l^{(2)}.$$3, A-optimal reduces $$c^{(1)} \prec c^{(2)} \iff \forall k\in\{1,\dots,M\}: c_k^{(1)}\le c_k^{(2)} \ \land\ \exists l\in\{1,\dots,M\}: c_l^{(1)}<c_l^{(2)}.$$4 by about $$c^{(1)} \prec c^{(2)} \iff \forall k\in\{1,\dots,M\}: c_k^{(1)}\le c_k^{(2)} \ \land\ \exists l\in\{1,\dots,M\}: c_l^{(1)}<c_l^{(2)}.$$5, and the abstract states that optimal designs can achieve the same precision at less than half the measurement cost on average compared to naïve allocations (Xu, 2019). Structurally, E-optimal networks are trees, whereas A- and D-optimal networks are often dense and the majority of A-optimal networks are 2-connected, enabling consistency checks via multiple paths.

Model-based design of experiments extends the same logic to sensor choice, sampling times, and measurement modalities. The measurement-optimization formulation uses binary selection variables, a Fisher information matrix $$c^{(1)} \prec c^{(2)} \iff \forall k\in\{1,\dots,M\}: c_k^{(1)}\le c_k^{(2)} \ \land\ \exists l\in\{1,\dots,M\}: c_l^{(1)}<c_l^{(2)}.$$6, and either A-optimality,

$$c^{(1)} \prec c^{(2)} \iff \forall k\in\{1,\dots,M\}: c_k^{(1)}\le c_k^{(2)} \ \land\ \exists l\in\{1,\dots,M\}: c_l^{(1)}<c_l^{(2)}.$$7

or D-optimality,

$$c^{(1)} \prec c^{(2)} \iff \forall k\in\{1,\dots,M\}: c_k^{(1)}\le c_k^{(2)} \ \land\ \exists l\in\{1,\dots,M\}: c_l^{(1)}<c_l^{(2)}.$$8

The key distinction is that A-optimality can miss strong parameter correlations or near-unidentifiability, whereas D-optimality collapses when any eigenvalue is near zero (Wang et al., 2024).

Two case studies make the point operational. In a batch reactor with

$$c^{(1)} \prec c^{(2)} \iff \forall k\in\{1,\dots,M\}: c_k^{(1)}\le c_k^{(2)} \ \land\ \exists l\in\{1,\dots,M\}: c_l^{(1)}<c_l^{(2)}.$$9

A-optimality has a sharp increase around \$\mathcal{P}=\{c(\lambda)\mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\},$03k is needed for full parameter identifiability. In a rotary packed bed for CO$\mathcal{P}=\{c(\lambda)\mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\},$1 capture, there is a sharp <a href="https://www.emergentmind.com/topics/information-gain-infogain" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">information gain</a> from about <strong>\$\mathcal{P}=\{c(\lambda)\mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\},$25k, after which gains are more gradual. The paper also shows that relaxed LP or NLP formulations can select fractional time points, so integer optimization is necessary when physical implementability matters (Wang et al., 2024).

5. Latent constructs, value, and meaning representations as instruments

In recommender systems, the gap is framed explicitly as a measurement problem. Observable behaviors such as clicks, favorites, replies, or watch time are not the same as the latent construct the designer cares about, called Value. The latent-variable model represents Value as a binary latent variable $\mathcal{P}=\{c(\lambda)\mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\},$3, observed behaviors as a binary vector $\mathcal{P}=\{c(\lambda)\mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\},$4, and a special observed behavior $\mathcal{P}=\{c(\lambda)\mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\},$5 as an anchor variable. The optimization target becomes

$\mathcal{P}=\{c(\lambda)\mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\},$6

rather than a weighted sum of engagement events (Milli et al., 2020).

Identification relies on three assumptions: Value-sensitive, No children, and One-sided conditional independence. Under these conditions, observable distributions and the anchor assumption determine the latent-value posterior. In the Twitter implementation for ML-driven notifications, the anchor is See Less Often (SLO) with

$\mathcal{P}=\{c(\lambda)\mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\},$7

The paper emphasizes that validity must be evaluated through content evidence, cognitive-process evidence, internal-structure evidence, relations to other variables, and consequences of measurement. A central example is that high user active minutes is much stronger evidence of value after Open than after Click:

$\mathcal{P}=\{c(\lambda)\mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\},$8

versus

$\mathcal{P}=\{c(\lambda)\mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\},$9

The consequence is that engagement should be optimized only insofar as it provides evidence about Value (Milli et al., 2020).

A parallel argument appears in computational social science and psychology. The prediction-measurement gap holds that embeddings optimized for next-token prediction, retrieval, or benchmark accuracy may be poor scientific instruments. The paper proposes scientific usability as the relevant family of objectives: geometric legibility, interpretability and traceability to linguistic evidence, robustness to non-semantic confounds, cognitive plausibility, and compatibility with regression-style inference over semantic directions (Plisiecki, 10 Mar 2026).

The contrast between static and contextual representations is central. Static word embeddings remain attractive because centroids, projections, and linear directions are easy to compute, nearest neighbors can explain semantic directions, and they suppress many token-level confounds. Contextual transformer representations offer richer semantics and context-sensitive meaning, but they mix semantics with syntax, casing, punctuation, tokenization artifacts, and frequency effects, and they exhibit layer ambiguity, anisotropy, and weaker interpretability for measurement tasks (Plisiecki, 10 Mar 2026). The proposed agenda is therefore geometry-first design, hierarchy-aware spaces, invertible post-hoc transformations, meaning atlases, and measurement-oriented evaluation protocols.

6. Heuristics, optimization theory, and geometric barriers

For heuristic algorithms, the gap is the difference between what a fast heuristic achieves and what the optimal method could achieve on the same input. MetaOpt formalizes this as a bilevel problem,

$$\mathcal{P}_\Lambda = c^{-1}(\mathcal{P})=\{\lambda \mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\}.$$0

where the outer level chooses an adversarial input and the inner levels compute the heuristic and optimal responses. The framework automatically rewrites the bilevel formulation into a single-level form solvable by Gurobi or Z3/Zen, and it supports heuristic-vs-optimal as well as heuristic-vs-heuristic comparisons (Namyar et al., 2023).

The empirical results show that the gap is often operationally large rather than merely asymptotic. On large traffic-engineering topologies such as Cogentco, Demand Pinning (DP) can incur a 33.9% performance gap, and modifying the heuristic using patterns extracted from adversarial inputs reduces the gap by 12.5× for a representative setting. In vector bin packing, the paper proves that for every $$\mathcal{P}_\Lambda = c^{-1}(\mathcal{P})=\{\lambda \mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\}.$$1, there exists an input with $$\mathcal{P}_\Lambda = c^{-1}(\mathcal{P})=\{\lambda \mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\}.$$2 and $$\mathcal{P}_\Lambda = c^{-1}(\mathcal{P})=\{\lambda \mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\}.$$3. In packet scheduling, MetaOpt finds sequences where SP-PIFO delays the highest-priority packet by 3× relative to PIFO, and Modified-SP-PIFO reduces the gap by 2.5× (Namyar et al., 2023).

A more foundational version of the gap appears in optimization theory for deep learning. Rather than only checking whether convexity or smoothness assumptions hold globally, the empirical study measures the proof-relevant quantities that appear in convergence analyses. The most striking finding is that the classical smooth non-convex descent signal,

$$\mathcal{P}_\Lambda = c^{-1}(\mathcal{P})=\{\lambda \mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\}.$$4

estimated by an unbiased proxy, is positive on average on almost all deep learning experiments even though training succeeds. The paper also finds that empirical smoothness proxies are bounded but often lie in the range $$\mathcal{P}_\Lambda = c^{-1}(\mathcal{P})=\{\lambda \mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\}.$$5 to $$\mathcal{P}_\Lambda = c^{-1}(\mathcal{P})=\{\lambda \mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\}.$$6, and that convexity-style explanations fail badly for GPT-2 and ImageNet while surviving only partially for CIFAR-10 and BERT (Tran et al., 2024). This is a direct mismatch between the analytical measurements used in theory and the optimization dynamics observed in practice.

The Overlap Gap Property supplies a geometric explanation for why such separations can be structural rather than incidental. For some $$\mathcal{P}_\Lambda = c^{-1}(\mathcal{P})=\{\lambda \mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\}.$$7 and $$\mathcal{P}_\Lambda = c^{-1}(\mathcal{P})=\{\lambda \mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\}.$$8, if $$\mathcal{P}_\Lambda = c^{-1}(\mathcal{P})=\{\lambda \mid \nexists \lambda' \text{ s.t. } c(\lambda')\prec c(\lambda)\}.$$9 and $\text{gap} = HV(\mathcal{P}_{\text{pessimistic})-HV(\mathcal{P}_{\text{optimistic}).$0 are both $\text{gap} = HV(\mathcal{P}_{\text{pessimistic})-HV(\mathcal{P}_{\text{optimistic}).$1-optimal, then

$\text{gap} = HV(\mathcal{P}_{\text{pessimistic})-HV(\mathcal{P}_{\text{optimistic}).$2

Near-optimal solutions therefore avoid an intermediate-distance band. The paper argues that this topological disconnectivity can rigorously rule out broad classes of stable or input-insensitive algorithms, including local algorithms, sequential local algorithms, some survey-propagation or belief-propagation-guided decimation procedures, WalkSAT variants, approximate message passing in some regimes, low-degree polynomial algorithms, QAOA and related quantum local algorithms, and Langevin dynamics on certain time scales (Gamarnik, 2021). In this setting, measurement or weak recovery may remain feasible while near-optimal optimization is blocked by the geometry of the solution space.

7. Deep reinforcement learning and the distinction between finding and exploiting good experience

In deep reinforcement learning, the gap is formulated as the difference between the best experience the agent generates and the performance of the learned policy. The paper introduces a practical sub-optimality estimator

$\text{gap} = HV(\mathcal{P}_{\text{pessimistic})-HV(\mathcal{P}_{\text{optimistic}).$3

where $\text{gap} = HV(\mathcal{P}_{\text{pessimistic})-HV(\mathcal{P}_{\text{optimistic}).$4 is an experience optimal policy derived from the agent’s own collected data. In deterministic environments, $\text{gap} = HV(\mathcal{P}_{\text{pessimistic})-HV(\mathcal{P}_{\text{optimistic}).$5 is the best trajectory ever observed; in stochastic settings, the paper also uses two softer estimators based on the top 5% of all experience or of recent replay-buffer experience (Berseth, 2 Aug 2025).

The main empirical conclusion is that, across environments and algorithms, the best experience generated is 2–3× better than the policies’ learned performance, indicating that deep RL methods only exploit about half of the good experience they generate. The experiments use PPO and DQN across MuJoCo, MinAtar, Atari, and sparse-reward settings such as Montezuma’s Revenge. In difficult exploration tasks, the learned return can remain noisy and near zero while the agent still occasionally generates high-value trajectories, which the paper interprets as evidence that the dominant bottleneck is often optimization/exploitation failure rather than pure exploration failure (Berseth, 2 Aug 2025).

The same paper shows that the gap need not shrink with more training, and that common interventions can enlarge it. Random Network Distillation (RND) improves exploration and raises returns but also increases the practical sub-optimality gap, making exploration bonuses a double-edged sword. Larger models, such as ResNet-18 relative to a standard 3-layer CNN on Atari, often increase the gap as well. On AtariFive environments, DQN and PPO achieve only a little over 30% of the performance of their best experience (Berseth, 2 Aug 2025).

A common misconception, addressed across several of the cited works, is that poor final performance necessarily identifies the wrong bottleneck. In MHPO, stronger validation hypervolume does not guarantee stronger test-side Pareto performance (Feurer et al., 2022). In reinforcement learning, poor policies do not imply that good trajectories were never found (Berseth, 2 Aug 2025). In recommendation and meaning representation, high engagement or strong predictive utility does not establish that the optimized quantity is a valid measure of the intended construct (Milli et al., 2020, Plisiecki, 10 Mar 2026). The measurement-versus-optimization gap is therefore best understood as a warning that proxy success, benchmark success, and optimization success can come apart in systematic, quantifiable ways.