The bias of IID resampled backtests for rolling-window mean-variance portfolios

Abstract: Backtests on historical data are the basis for practical evaluations of portfolio selection rules, but their reliability is often limited by reliance on a single sample path. This can lead to high estimation variance. Resampling techniques offer a potential solution by increasing the effective sample size, but can disrupt the temporal ordering inherent in financial data and introduce significant bias. This paper investigates the critical questions: First, How large is this bias for Sharpe Ratio estimates?, and then, second: What are its primary drivers?. We focus on the canonical rolling-window mean-variance portfolio rule. Our contributions are identifying the bias mechanism, and providing a practical heuristic for gauging bias severity. We show that the bias arises from the disruption of train-test dependence linked to the return auto-covariance structure and derive bounds for the bias which show a strong dependence on the observable first-lag autocorrelation. Using simulations to confirm these findings, it is revealed that the resulting Sharpe Ratio bias is often a fraction of a typical backtest's estimation noise, benefiting from partial offsetting of component biases. Empirical analysis further illustrates that differences between IID-resampled and standard backtests align qualitatively with these drivers. Surprisingly, our results suggest that while IID resampling can disrupt temporal dependence, its resulting bias can often be tolerable. However, we highlight the need for structure-preserving resampling methods.