Across-Design Uncertainty in Short Pricing Panels: Evidence from Simulated Price Trajectories

Abstract: Short observational pricing panels can contain many observations while offering only a small number of distinct price movements. This paper studies the inferential consequences of that distinction in a synthetic data-generating process calibrated to a sparse pricing regime. We separate uncertainty conditional on a realised price trajectory from variation in estimation error across alternative trajectories generated by the same pricing process. In the baseline simulations, the latter component accounts for 97.6% of the variance of estimation error for the gradient-boosted specification. Within-panel resampling procedures use the information of one realised trajectory and do not identify this across-design component. Three results organise the analysis. First, across-design dispersion is well described by the empirical relation sigma_hat approx 0.182 V^-0.271, where V equals moves times magnitude squared. Second, adding regions sharing a common price path reduces outcome noise but does not create independent price trajectories; conversely, averaging across units with independent design-specific errors reduces dispersion at the standard square root rate. Third, a Paule-Mandel variance component estimated across independently priced units substantially increases empirical coverage in homogeneous simulations, from 0.469 to 0.931. The broader implication is a shift toward designing data-generating processes that create independent identifying variation rather than relying solely on fixed passive panels.

• The paper identifies and quantifies a between-design uncertainty component in short pricing panels, making up to 99% of the total error.
• Simulations show that conventional inferential techniques favorably widen bootstrapped intervals due to unidentified between-design uncertainty.
• Dividing small price trajectories into independently fied regions can improve error identification, whereas non-independent pricing does not.

Motivation and core distinction

Short pricing panels can contain thousands of region–week rows while offering only a handful of distinct list-price movements. This paper argues that in such settings row count is a poor proxy for identifying variation, and that conventional inferential procedures answer a different question than the one practitioners often care about. The author formalizes a decomposition of estimation error into two components: within-design uncertainty, the dispersion of $\hat\theta$ conditional on a fixed realised price trajectory $D$, and between-design uncertainty, the dispersion of the design-specific conditional mean error $b(D)=\mathbb{E}_u[\hat\theta-\theta(D)\mid D]$ across alternative trajectories generated by the same pricing process. The distinction is related to sampling-based versus design-based inference [(Kopparapu et al., 2022)-style literature; specifically Abadie et al. (2020) and Rambachan & Roth], but here it is operationalized by explicitly generating multiple price trajectories in simulation so both components are directly measurable.

The setting is a partially linear model estimated by a DML-motivated cross-fitted residual-on-residual procedure with median aggregation over five temporal folds. The estimand is deliberately defined as the convolution of the consumer elasticity with realised pass-through, $\theta=\varepsilon_c\rho^{\text{eff}}$, read from each generated panel rather than from configuration parameters—a detail that avoids conflating estimand variation with estimation error.

Quantifying between-design dominance

The baseline data-generating process is calibrated to Nakamura–Steinsson moments: 120 weeks, six regions, ten list-price changes of 3.0–7.5%, pass-through 0.85, demand shock standard deviation 0.05 in logs. With twelve designs and forty shock replications per design ($n=480$), the decomposition is stark:

Learner	Bias	$\sigma_w$	$\sigma_b$	Bootstrap SE	Between share	Coverage
Gradient boosting	0.150	0.0756	0.4815	0.1595	97.6%	0.565
Sieve (ridge basis)	0.016	0.0589	0.7187	0.1553	99.3%	0.242

Three findings deserve emphasis. First, between-design centring dispersion accounts for essentially all error variance in this DGP. The paper is careful to state this is a property of the simulated design, not a universal constant. Second, the bootstrap standard error is not simply too narrow: at 0.1595 it exceeds the measured within-design standard deviation by a factor of about 2.1 but is only one third of $\sigma_b$. The coverage failure is therefore attributable to design-dependent centring rather than underestimated conditional variance alone—a diagnosis that rules out "just widen the interval" as a fix. Third, conditional versus repeated-design coverage are distinct targets; the paper reports both and resists conflating them.

Failure of within-panel constructions

Eight familiar interval constructions—moving-block bootstraps, hierarchical block bootstrap, analytic i.i.d., week-clustered, multiway-clustered, Newey–West on weekly scores, wild cluster bootstrap-$t$—were evaluated on identical fits over 200 panels at nominal 95%. None reaches nominal coverage; the best performer, multiway clustering, achieves only 0.730 for gradient boosting. The paper is appropriately cautious: this is a finite comparison of eight procedures, not an impossibility theorem. Moreover, wider intervals do not resolve the problem under the pre-specified operational width threshold $W\le 0.6$: the higher-coverage constructions are too wide to support the intended unit-level pricing decision.

Dispersion across the simulation grid

Sweeping price-move counts (2–16), magnitudes, promotion confounding, and pass-through (288 grid points, 4,608 fits) yields the empirical relation

$D$0

with $D$1. The exponent is explicitly labelled a simulation regularity, not a scaling law, and extrapolations (e.g., halving $D$2 requires roughly a thirteenfold increase in $D$3) are offered only to convey curvature. Two sobering diagnostics emerge: no grid configuration simultaneously attains bias below 0.15 and design dispersion below 0.20, and the median ratio of repeated-design dispersion to reported standard-error scale is 2.26 for gradient boosting and 8.03 for the sieve.

Common versus independent designs: aggregation

A simple variance identity governs aggregation: averaging $D$4 units' design errors yields variance reduction at rate $D$5, i.e., $D$6 under independence and nothing under perfect correlation. This is the algebraic content behind the paper's slogan that "adding rows" under a common national list price is not equivalent to adding identification—extra regions improve nuisance estimation and reduce outcome noise but create no new independent price trajectory.

The simulations show product units with separately generated price paths come close to the independent benchmark: observed dispersion-reduction factors of 2.07–2.09 against $D$7, and 2.99 against $D$8. Notably, aggregation improves RMSE but not baseline-interval coverage, because interval width contracts alongside dispersion. Learner choice also matters: gradient boosting retains a persistent common mean bias near 0.30 across aggregation levels (so its coverage falls as intervals tighten), whereas the sieve's near-zero bias translates dispersion reduction into RMSE gains. A companion learner matrix shows no single nuisance learner dominates on bias, RMSE, and coverage simultaneously.

Variance-component intervals

When multiple independently priced units exist, excess between-unit dispersion can be estimated with a Paule–Mandel moment condition borrowed from meta-analysis. Under a random-bias working model $D$9, the resulting variance-augmented intervals raise homogeneous-scenario coverage from 0.469 (bootstrap percentile) to 0.931 (posterior-centred construction), with $b(D)=\mathbb{E}_u[\hat\theta-\theta(D)\mid D]$0 far exceeding the mean bootstrap SE of 0.144. Shrinkage alone does worse than the bootstrap (coverage 0.426), consistent with the centring mechanism: shrinkage reduces conditional variance without modelling displacement.

The gains are purchased with width: the augmented intervals are roughly 2.7 times as wide as the bootstrap and exceed the 0.6 precision threshold. Combining aggregation with variance augmentation narrows intervals substantially—at category level, widths fall to 0.83–1.25 relative to threshold—but no combination of learner, scenario, and aggregation level meets both the coverage and width criteria. An instructive back-of-envelope calculation suggests roughly ten independently priced units per published estimate would be needed, versus four available in the portfolio. In heterogeneous scenarios, $b(D)=\mathbb{E}_u[\hat\theta-\theta(D)\mid D]$1 mixes genuine effect heterogeneity with design dispersion and declines more slowly than $b(D)=\mathbb{E}_u[\hat\theta-\theta(D)\mid D]$2, warranting conservative interpretation.

Limitations

The limitations are candidly stated and substantial. Every quantitative result derives from a purpose-built generator, limiting external validity; the calibration to Nakamura–Steinsson moments establishes plausibility, not representativeness of any real category. The fitted exponent is descriptive, and $b(D)=\mathbb{E}_u[\hat\theta-\theta(D)\mid D]$3 is not derived as Fisher information. The Paule–Mandel interpretation requires independent or weakly dependent design draws, adequate within-unit variances, exchangeability, and—for clean identification—common unit truths. The implemented estimator uses median fold aggregation rather than canonical pooled DML2, so results should not be attributed to DML as a class. The 0.6 width threshold is pre-specified for the exercise, not externally validated. Reproducibility is addressed seriously via seed separation between selection and reporting runs and pre-registered verdict rules (13 of 30 pass), though the embargo-free cross-fitting is acknowledged as a known defect.

Conclusion

The paper demonstrates, in a controlled synthetic environment calibrated to empirical pricing facts, that sparse treatment variation produces a between-design component of estimation error that within-panel procedures cannot identify nonparametrically. Its practical contribution is a reframing: from refining intervals on a passive panel toward designing or exploiting data structures—multiple independently realised price paths, randomized regional assignment—that generate genuinely additional identifying variation. The central open question left by the paper is whether analogous between-design dispersion arises in real pricing panels and for other estimator classes; answering it requires computing the proposed dependence diagnostics on actual price histories, ideally where quasi-independent or randomized paths exist.