Šidák Procedure: Methods and Applications

• The Šidák procedure is a statistical method with dual interpretations: as a multiplicity adjustment in independent multiple testing and as a rank-based nonparametric two-sample test.
• For multiple testing, the classical Šidák correction computes a per-test level of α' = 1-(1-α)^(1/n) to achieve exact family-wise error rate control, and its asymptotic behavior closely parallels adjusted Bonferroni methods.
• In post-selection inference, Šidák intervals benchmark simultaneous coverage, while extensions offer improved, selection-adjusted intervals and adaptations to maximal precedence–exceedance statistics.

The term Šidák procedure is used in at least two distinct ways in the cited literature. In multiple testing, it denotes the classical single-step correction that attains exact family-wise error rate control under independence by using the per-test level $\alpha' = 1-(1-\alpha)^{1/n}$. In nonparametric two-sample theory, by contrast, it denotes a rank-based test due to Šidák and Vondráček that counts precedences and exceedances relative to sample order statistics, together with later “Šidák-type” generalizations. In post-selection inference, Šidák intervals appear as a simultaneous-coverage benchmark rather than as a selection-adjusted solution. The shared name is therefore historical rather than methodological, and the surrounding hypotheses, loss criteria, and asymptotic regimes differ substantially across these uses (Datta et al., 24 Feb 2026, Stoimenova et al., 2016, Benjamini et al., 2019, Chakrabortya et al., 2022).

1. Distinct statistical meanings

The ambiguity of the term is central. The multiple-testing Šidák procedure is a multiplicity adjustment: it calibrates a common cutoff across $n$ tests so that the probability of at least one false rejection is $\alpha$ under independence. The two-sample Šidák procedure is a one-sided nonparametric test based on extreme or near-extreme order statistics from two independent samples. The post-selection literature uses “Šidák intervals” for the usual simultaneous confidence intervals under independence and then studies how much they can be shortened once a selection rule is taken into account (Datta et al., 24 Feb 2026, Stoimenova et al., 2016, Benjamini et al., 2019, Chakrabortya et al., 2022).

Usage of “Šidák procedure”	Core rule	Primary target
Multiple testing	$\alpha' = 1-(1-\alpha)^{1/n}$	FWER
Post-selection intervals	Šidák simultaneous intervals under independence	SoP baseline
Two-sample nonparametrics	precedence–exceedance statistics	ordered or two-sided alternatives

A recurrent misconception is that every reference to a “Šidák procedure” concerns the multiple-comparison correction. The two-sample papers explicitly reject that identification: their “Šidák-type” terminology refers instead to the 1957 precedence–exceedance test and its descendants, not to the multiplicity inequality used in simultaneous inference (Stoimenova et al., 2016, Chakrabortya et al., 2022).

2. Classical multiple-testing Šidák correction

In the classical setting, there are $n$ tests with independent $p$-values satisfying

$p_i \sim \mathrm{Uniform}(0,1)$

under their respective null hypotheses. The Šidák single-step procedure uses the per-test level

$\alpha_{\text{Sidak}} = 1-(1-\alpha)^{1/n}.$

For one-sided Gaussian tests with $X_i \sim N(0,1)$ under $H_{0i}$, the rejection rule is

$n$0

Under independence and the global null,

$n$1

Thus the procedure is exact, not merely conservative, in the independent case (Datta et al., 24 Feb 2026).

For two-sided Gaussian tests, the cited paper uses the symmetric rule

$n$2

The same paper also emphasizes the asymptotic equivalence between the Šidák cutoff and a suitably adjusted Bonferroni cutoff,

$n$3

through the expansion

$n$4

This yields the common asymptotic form

$n$5

which is the key device for comparing Šidák and adjusted Bonferroni thresholds in high-dimensional regimes (Datta et al., 24 Feb 2026).

3. Weak dependence and asymptotic exactness

The paper "Some Asymptotic Results on Multiple Testing under Weak Dependence" studies the same classical Šidák cutoff when the test statistics are not independent but form a weakly correlated Gaussian sequence. The model is

$n$6

with correlations $n$7 satisfying the weak dependence condition

$n$8

and a uniform bound away from $n$9 (Datta et al., 24 Feb 2026).

The main result is asymptotic exactness of the usual independence-based Šidák threshold. Under the weakly dependent standard Gaussian setting, both the adjusted Bonferroni procedure and the Šidák procedure satisfy

$\alpha$0

whenever the proportion of true nulls tends to one,

$\alpha$1

The analogous two-sided result also holds with the cutoff $\alpha$2 (Datta et al., 24 Feb 2026).

The mechanism is extreme-value asymptotics. Under the cited weak dependence condition, exceedances above high thresholds behave asymptotically as if they were independent, so the number of exceedances is approximately Poisson. The probability of no exceedance therefore converges to the same limit as in the independent case. A closely related theorem yields asymptotic formulas for the $\alpha$3-FWER through the $\alpha$4th largest order statistic of the true-null subvector. The paper also studies power through

$\alpha$5

and shows that, for both adjusted Bonferroni and Šidák, $\alpha$6 when the strongest signal dominates the $\alpha$7 threshold scale (Datta et al., 24 Feb 2026).

The simulation evidence is explicitly numerical. For $\alpha$8 and $\alpha$9, with $\alpha' = 1-(1-\alpha)^{1/n}$0 and $\alpha' = 1-(1-\alpha)^{1/n}$1 under a product correlation structure satisfying the weak dependence condition, the estimated FWER for the one-sided Šidák procedure is reported as very close to the nominal level, “e.g. around 0.099–0.100 for $\alpha' = 1-(1-\alpha)^{1/n}$2, around 0.049–0.050 for $\alpha' = 1-(1-\alpha)^{1/n}$3,” with essentially identical values for the adjusted Bonferroni benchmark (Datta et al., 24 Feb 2026).

4. Šidák intervals in post-selection inference

In the post-selection confidence-interval setting, Šidák enters as a benchmark for simultaneous coverage rather than as the final inferential target. The problem is to report intervals only for a selected subset of parameters, such as the largest $\alpha' = 1-(1-\alpha)^{1/n}$4 of $\alpha' = 1-(1-\alpha)^{1/n}$5 shift parameters, while controlling the simultaneous over selected error rate

$\alpha' = 1-(1-\alpha)^{1/n}$6

The cited paper compares newly constructed SoS-controlling intervals to standard Bonferroni and Šidák simultaneous intervals (Benjamini et al., 2019).

For independent estimators, a two-sided Šidák interval has the form

$\alpha' = 1-(1-\alpha)^{1/n}$7

with

$\alpha' = 1-(1-\alpha)^{1/n}$8

This interval controls simultaneous coverage over all $\alpha' = 1-(1-\alpha)^{1/n}$9 parameters under independence, but it does not use the knowledge that only a selected subset will actually be reported. The paper’s central comparison is that the new SoS intervals “improve substantially over Šidák intervals when $n$0 is small compared to $n$1, and approach the standard Bonferroni-corrected intervals when $n$2” (Benjamini et al., 2019).

The general SoS construction for the largest $n$3 of $n$4 independent shift estimators yields intervals

$n$5

with tuning constants chosen so that

$n$6

The multiplicity structure is asymmetric: the lower endpoints still reflect $n$7, while the upper endpoints depend only on $n$8. This is the source of the length reduction relative to Šidák intervals when $n$9 (Benjamini et al., 2019).

The paper also isolates special low-dimensional cases. For $p$0, $p$1, selection by the larger of two observations, and exchangeable symmetric errors, the unadjusted interval

$p$2

has exact SoS coverage. For the largest absolute value of two independent normal estimators, the derived SoS interval is strictly shorter than the corresponding Šidák simultaneous interval: at $p$3, its maximum width is about $p$4 of Šidák’s width, and as $p$5 its length tends to about $p$6 of Šidák’s (Benjamini et al., 2019).

5. The nonparametric two-sample Šidák test

A separate literature uses Šidák test to mean a nonparametric two-sample test based on precedence and exceedance counts. In the formulation studied in "Šidák-type tests for the two-sample problem based on precedence and exceedance statistics," two independent samples are observed,

$p$7

with $p$8 and $p$9 absolutely continuous. The null hypothesis is

$p_i \sim \mathrm{Uniform}(0,1)$0

and the alternatives are ordered alternatives expressed through stochastic ordering (Stoimenova et al., 2016).

The original statistic counts extremes. Let $p_i \sim \mathrm{Uniform}(0,1)$1 be the number of $p_i \sim \mathrm{Uniform}(0,1)$2 observations exceeding the largest $p_i \sim \mathrm{Uniform}(0,1)$3 observation, and let $p_i \sim \mathrm{Uniform}(0,1)$4 be the number of $p_i \sim \mathrm{Uniform}(0,1)$5 observations preceding the smallest $p_i \sim \mathrm{Uniform}(0,1)$6 observation. The original Šidák statistic is

$p_i \sim \mathrm{Uniform}(0,1)$7

Large values indicate a strong separation in which the $p_i \sim \mathrm{Uniform}(0,1)$8 sample accumulates at low ranks and the $p_i \sim \mathrm{Uniform}(0,1)$9 sample at high ranks. Under $\alpha_{\text{Sidak}} = 1-(1-\alpha)^{1/n}.$0, its exact distribution is distribution-free, and the cited paper records the closed form

$\alpha_{\text{Sidak}} = 1-(1-\alpha)^{1/n}.$1

This is the original two-sample Šidák statistic and distribution referenced by the later “Šidák-type” constructions (Stoimenova et al., 2016).

The generalization replaces the extreme thresholds by inner order statistics. With

$\alpha_{\text{Sidak}} = 1-(1-\alpha)^{1/n}.$2

define

$\alpha_{\text{Sidak}} = 1-(1-\alpha)^{1/n}.$3

$\alpha_{\text{Sidak}} = 1-(1-\alpha)^{1/n}.$4

and

$\alpha_{\text{Sidak}} = 1-(1-\alpha)^{1/n}.$5

When $\alpha_{\text{Sidak}} = 1-(1-\alpha)^{1/n}.$6, one recovers $\alpha_{\text{Sidak}} = 1-(1-\alpha)^{1/n}.$7. Under $\alpha_{\text{Sidak}} = 1-(1-\alpha)^{1/n}.$8, the joint pmf of $\alpha_{\text{Sidak}} = 1-(1-\alpha)^{1/n}.$9 is derived exactly, and the cdf of $X_i \sim N(0,1)$0 is obtained by summing that joint pmf over the triangular region $X_i \sim N(0,1)$1 (Stoimenova et al., 2016).

Because the distribution is discrete, exact level-$X_i \sim N(0,1)$2 comparison uses a randomized test. The rejection region is $X_i \sim N(0,1)$3, where $X_i \sim N(0,1)$4 is the smallest integer such that

$X_i \sim N(0,1)$5

If $X_i \sim N(0,1)$6 and $X_i \sim N(0,1)$7 bracket the nominal $X_i \sim N(0,1)$8, then rejection at $X_i \sim N(0,1)$9 is randomized with probability

$H_{0i}$0

The paper also gives large-sample approximations. For balanced large samples, the null law of $H_{0i}$1 is well approximated by a negative binomial distribution with parameters $H_{0i}$2 and $H_{0i}$3, and “also fairly well by a chi-square distribution,” with practical adequacy reported for sample sizes $H_{0i}$4–$H_{0i}$5 when $H_{0i}$6 and $H_{0i}$7 are similar (Stoimenova et al., 2016).

Power is studied under the Lehmann alternative

$H_{0i}$8

The exact joint pmf of $H_{0i}$9 under this alternative is derived in terms of gamma functions, and exact or Monte Carlo power calculations show that inner thresholds can improve power relative to the original extreme-threshold statistic. For example, with $n$00 and $n$01, the reported powers are $n$02 for $n$03, $n$04 for $n$05, $n$06 for $n$07, and $n$08 for $n$09 (Stoimenova et al., 2016).

6. Maximal precedence–exceedance extensions

The paper "A class of Šidák-type tests based on maximal precedence and exceedance statistic" extends the nonparametric two-sample tradition by replacing total counts with maximal local counts. Two independent samples $n$10 and $n$11 are again assumed, with $n$12 and $n$13 univariate and absolutely continuous. For integers

$n$14

the construction forms interval counts $n$15 near the lower tail of the ordered $n$16 sample and $n$17 near the upper tail, then defines

$n$18

and finally

$n$19

A common symmetric specialization sets $n$20 and writes $n$21 (Chakrabortya et al., 2022).

This statistic is presented as a generalization of the original Šidák test. The paper states that “The test defined by $n$22 includes Šidák’s test as a special case when $n$23.” The rationale for the maximal version is the “masking effect”: a strong local concentration of one sample in a narrow interval can be diluted if only total precedences or exceedances are counted. By summing maximal early and late concentrations, $n$24 is designed for a two-sided alternative, unlike earlier precedence-only or one-sided Šidák-type procedures (Chakrabortya et al., 2022).

Under $n$25, the exact distribution is obtained from the joint distribution of the full count vector

$n$26

The resulting null law depends only on $n$27 and is therefore distribution-free. The paper also derives the joint count distribution under the Lehmann alternative

$n$28

in terms of Beta functions, and thus obtains the distribution of $n$29 under the alternative as well. As with the earlier paper, the test is discrete, so exact-size comparison is implemented with a randomized rule based on the tail probabilities at the critical boundary (Chakrabortya et al., 2022).

The power comparisons are explicitly directional. For $n$30, Table 8 reports that the competing Šidák-type precedence–exceedance statistic $n$31 and the maximal precedence statistic $n$32 have power essentially $n$33 for $n$34, whereas $n$35 has substantial power in both directions. The authors therefore describe $n$36 and $n$37 as finite-sample biased for certain alternatives and present $n$38 as suitable for two-sided alternatives (Chakrabortya et al., 2022).

A real-life example concerns failure voltages for two types of cable insulation with $n$39. When one group is treated as training and the other as test, $n$40 rejects $n$41 at the $n$42 level for $n$43, whereas $n$44 rejects for all $n$45. After swapping the training and test labels, $n$46 still rejects $n$47 at the $n$48 level for all $n$49, while $n$50 fails to reject for any $n$51. This illustrates the central distinction between a two-sided maximal precedence–exceedance procedure and a one-sided Šidák-type predecessor (Chakrabortya et al., 2022).