Sample Complexity Bounds

• Sample complexity bounds are quantitative measures defining the minimum number of data samples required to reach a target accuracy and confidence in various estimation and decision-making tasks.
• They are applied across diverse domains such as PAC learning, reinforcement learning, convex optimization, and quantum measurement to provide finite-sample guarantees and reveal performance trade-offs.
• By leveraging minimax and instance-dependent analyses along with structure-adapted techniques, these bounds guide the design of statistically efficient algorithms under differing conditions.

Sample complexity bounds quantify the minimal number of data samples required by an algorithm or estimator to achieve a specified level of accuracy and confidence in learning, system identification, estimation, or decision-making under uncertainty. These bounds are foundational in learning theory, statistics, signal processing, control, statistics, and quantum information, providing finite-sample guarantees and characterizing achievable performance. Cutting across classical PAC learning, margin-based classification, convex optimization, dynamical systems, reinforcement learning, quantum measurement learning, and modern high-dimensional models, sample complexity analysis reveals both minimax and instance-dependent rates, clarifies the trade-offs induced by structure (such as invariances, privacy, or model families), and leads to algorithmic designs optimized for statistical efficiency.

1. Foundational Definitions and Frameworks

Sample complexity bounds have diverse meanings across technical domains but are always anchored to three ingredients: a class of target objects (concepts, distributions, policies, measurements), an allowable error or loss metric (such as misclassification, total variation, prediction error, or regret), and a probability or confidence parameter.

• PAC Learning (Realizable Case): Given concept class $C$ of VC-dimension $d$, permissible error $\epsilon$, and confidence $1-\delta$, the sample complexity $m(\epsilon,\delta)$ is the smallest $m$ such that a learner $A$ produces $h$ with $\Pr[\mathrm{err}(h)\leq\epsilon] \geq 1-\delta$ for any $c \in C$. The optimal rate is $$m(\epsilon,\delta) = \Theta\left( \frac{d+\log(1/\delta)}{\epsilon} \right)$$ (Hanneke, 2015).
• Margin-based Classification: For large-margin (e.g., SVM-type) classification with margin parameter $\gamma$ and distribution $P$, the sample complexity is governed by the margin-adapted dimension $d_m(\gamma,P)$, with $$m = \widetilde{O}\left((d_m(\gamma,P) + \log(1/\delta)) / \epsilon^2\right)$$ (Sabato et al., 2012).
• Differential Privacy: In pure differentially private PAC learning, sample complexity $SCDP(C)$ for class $C$ is tightly characterized by randomized one-way public-coin communication complexity, and lower bounded by the Littlestone dimension: $SCDP(C) = \Omega(LDim(C))$, with possible gaps $SCDP(C) \gg VC(C)$ (Feldman et al., 2014).
• Convex Stochastic Programs: For SAA in convex stochastic programming, new bounds without metric entropy terms reach $N = O(\sigma^2/\epsilon^2)$ under mild regularity and bypass linear dependence on dimension $d$ (Liu et al., 1 Jan 2024).
• Quantum Learning: Quantum PAC sample complexity for measurement classes depends on the shadow-norm parameter $V_{\mathcal{C}^*}$ and the size of extreme points; $$n = O(V_{\mathcal{C}^*} \log |\mathcal{C}^*| / \epsilon^2)$$, realizing the first non-linear scaling in the quantum setting (Heidari et al., 22 Aug 2024).

2. Classical Minimax and Instance-dependent Rates

Across learning, optimization, and decision-making, both worst-case (minimax) and instance-dependent rates arise.

• Best-$k$-Arm Identification: In multi-armed bandit selection, the minimal expected samples required to identify the top-$k$ arms is governed by instance-wise gap-dependent terms ($H^+$ and $H^-$), which can be strictly smaller than the classical sum-of-inverse-gap-squared measure. Nearly instance-optimal algorithms achieve the lower bounds up to doubly-logarithmic factors (Chen et al., 2017).
• Zero-Sum Matrix Games: Approximate Nash equilibrium identification in $n\times2$ matrix games admits sample complexity lower bounds scaling as $$O(\max\{1/\Delta_{\min}^2,\,1/(\epsilon|D|)\}\log(1/\delta))$$, where $\Delta_{\min}$ and $D$ are gap parameters of the payoff matrix. For favorable instances, exponentially fewer samples are needed (Maiti et al., 2023).
• Reinforcement Learning—Actor-Critic: In infinite-horizon discounted MDPs, actor-critic methods for $\epsilon$-stationarity achieve sample complexity $O((1-\gamma)^{-2}\epsilon^{-2}\log(1/\epsilon))$ (AC) and $O((1-\gamma)^{-4}\epsilon^{-3}\log(1/\epsilon))$ (NAC), strictly better than policy gradient (PG) and natural policy gradient (NPG), where critic-based variance reduction is essential (Xu et al., 2020).
• Diffusion Model Training: For high-dimensional deep generative models (e.g., diffusion models), end-to-end sample complexity scales only polylogarithmically in the inverse desired Wasserstein error, achieving $$m = {\rm poly}(d) \cdot \operatorname{polylog}(1/\gamma)$$, closing a gap versus prior polynomial-in-$1/\gamma$ rates (Gupta et al., 2023).
• Constrained MDPs: For constrained average-reward MDPs, the minimax-optimal sample complexity is $\widetilde{O}(SA(B+H)/(\epsilon^2 \zeta^2))$ under strict feasibility; here $S,A$ are sizes, $B$ bounds transients, $H$ is the span, and $\zeta$ the Slater constant. This reveals a precise statistical penalty for feasibility constraints (Wei et al., 20 Sep 2025).

3. Structure-adapted and Distribution-dependent Bounds

Many sample complexity bounds leverage distributional or algebraic structure to obtain sharper results.

• Distributional Dependence in Margin-based Learning: The margin-adapted dimension $d_m(\gamma,P)$ exactly interpolates between worst-case (VC-dimension, $\operatorname{Tr}(\Sigma)/\gamma^2$) and favorable anisotropic regimes (effective dimension $\ll d$), yielding tight sample complexity $O(d_m(\gamma,P)/\epsilon^2)$ for large-margin classification under sub-Gaussian product laws (Sabato et al., 2012).
• Group Invariances: For distributions invariant under a group $G$ acting on a manifold $M$, sample complexity for estimating Wasserstein, Sobolev, MMD, and $L^2$/ $L^\infty$ density divergences is reduced by a factor of $|G|$ (for finite $G$) or by the normalized measure of the quotient $M/G$ (for continuous $G$), and convergence rates in $n$ improve to match the "effective dimension" $d = \dim(M/G)$ (Tahmasebi et al., 2023).
• Denoising and High-dimensional Estimation: In learning a simplex from noisy (additive Gaussian) samples in $\mathbb{R}^K$, the sample complexity is $$n = \widetilde{O}(K^2/\epsilon^2) \exp(\Omega(K/{\rm SNR}^2))$$, but with SNR $\gtrsim \sqrt{K}$, the exponential phase vanishes, matching the noiseless case (Saberi et al., 2022).

4. Algorithmic and Information-theoretic Techniques

Sample complexity bounds are derived using a range of analytical and constructive tools, often leveraging information theory, combinatorial constructions, and algorithmic design.

• Compression Schemes: For density estimation, the existence of data-dependent robust compression schemes directly yields tight upper bounds for mixtures and products (e.g., learning $k$-mixtures of $d$-dimensional Gaussians in TV requires $\widetilde{\Theta}(k d^2 / \epsilon^2)$ samples in the agnostic setting) (Ashtiani et al., 2017).
• Information-theoretic Lower Bounds: Change-of-measure (likelihood ratio) arguments, Fano-type constructions, and KL-divergence sensitivity are used to prove lower bounds that often match upper bounds up to constants or logarithms (as in system identification (Jedra et al., 2019, Chatzikiriakos et al., 17 Sep 2024), margin learning (Sabato et al., 2012), bandit feedback (Chen et al., 2017)).
• Finite-sample Monte Carlo: For Sequential Monte Carlo (SMC) schemes, the finite-sample complexity is determined by mixing properties, importance weight regularity, and the number of tempering stages, producing bounds within logarithmic factors of Markov chain Monte Carlo for log-concave and high-dimensional targets (Marion et al., 2018).
• First-order and Gradient-based Methods in Bilevel and RL: Recent advances in bilevel RL circumvent Hessian dependence using the Polyak-Łojasiewicz property and penalty surrogates, attaining $O(\epsilon^{-4})$ rates in continuous state-action domains, improving dramatically over past $O(\epsilon^{-6})$ and higher orders (Gaur et al., 22 Mar 2025).

5. Separation Results and Limits

Situations where privacy, quantum structure, or other constraints fundamentally increase the sample complexity are precisely characterized.

• Private vs. Non-private Learning: In PAC learning, pure ($\beta = 0$) differential privacy can require exponentially more samples than both non-private or approximate ($\beta>0$) DP; exact separations are constructed using representation dimension and communication complexity, with explicit classes achieving arbitrarily large SCDP/VC gaps (Feldman et al., 2014).
• Quantum Learning vs. Classical: Quantum PAC sample complexity for measurement classes can scale linearly in $|\mathcal{C}|$ in the worst case, versus log-linear in the shadow-norm-exposed extreme points: $n = O(V_{\mathcal{C}^*} \log |\mathcal{C}^*|)$, restoring a near-classical logarithmic dependence under bounded-norm circumstances (Heidari et al., 22 Aug 2024).
• Strict-constrained MDPs: Enforcing zero constraint violation in average-reward CMDPs requires $1/\zeta^2$ more samples compared to allowing $\epsilon$-approximate feasibility, reflecting a statistical hardness not present in unconstrained or discounted/finite-horizon analogues (Wei et al., 20 Sep 2025).

6. Parameter Dependence, Tightness, and Open Directions

Sample complexity bounds are sensitive to task-specific parameters, variance, model size, smoothness, problem structure, and required accuracy. In many cases, upper and lower bounds match up to logarithmic or constant factors; in others, subtle gaps remain.

Setting	Core Sample Complexity	Key Parameters	Tightness/Separations
PAC-realizable (Hanneke, 2015)	$\Theta((d+\log(1/\delta))/\epsilon)$	VC-dim $d$, error/conf	Tight, improved constant, no $\log(1/\epsilon)$
Margin-based (Sabato et al., 2012)	$\widetilde{O}(d_m(\gamma,P)/\epsilon^2)$	Margin, covariance	Tight both sides for sub-Gaussians
Private PAC (Feldman et al., 2014)	$\Omega(LDim(C))$ (pure DP)	$LDim(C)$ vs $VC(C)$	Unbounded gap possible, separation pure/approximate DP
RNN regression (Akpinar et al., 2019)	$\widetilde{O}(a^4 b/\epsilon^2)$	width, input length	Deterministic upper bound
Influence maximization (Sadeh et al., 2019)	$O(s \tau \epsilon^{-2} \log(n/\delta))$	seed size, diffusion steps	Removal of $n$-factor, near-optimal
CMDP strict feasibility (Wei et al., 20 Sep 2025)	$\widetilde{O}(SA(B+H)/(\epsilon^2 \zeta^2))$	problem size, slack	Minimax tight for CAMDPs

Many open problems persist: refining constants in optimal PAC bounds (Hanneke, 2015), closing remaining logarithmic gaps in convex stochastic programming and SAA (Liu et al., 1 Jan 2024), extending metric-entropy-free rates to nonconvex or non-Lipschitz settings, generalization to more complex reinforcement learning and stochastic control models, and fully characterizing quantum-classical separations in high-dimensional limit regimes.