---
title: 'Šidák Procedure: Methods and Applications'
url: https://www.emergentmind.com/topics/sidak-procedure
type: topic
---

# Šidák Procedure: Methods and Applications

The term **Šidák procedure** is used in at least two distinct ways in the cited literature. In multiple testing, it denotes the classical single-step correction that attains exact family-wise error rate control under independence by using the per-test level $\alpha' = 1-(1-\alpha)^{1/n}$. In nonparametric two-sample theory, by contrast, it denotes a rank-based test due to Šidák and Vondráček that counts precedences and exceedances relative to sample order statistics, together with later “Šidák-type” generalizations. In post-selection inference, Šidák intervals appear as a simultaneous-coverage benchmark rather than as a selection-adjusted solution. The shared name is therefore historical rather than methodological, and the surrounding hypotheses, loss criteria, and asymptotic regimes differ substantially across these uses [2602.21359] [1610.09502] [1906.00505] [2208.02521].

## 1. Distinct statistical meanings

The ambiguity of the term is central. The multiple-testing Šidák procedure is a multiplicity adjustment: it calibrates a common cutoff across $n$ tests so that the probability of at least one false rejection is $\alpha$ under independence. The two-sample Šidák procedure is a one-sided nonparametric test based on extreme or near-extreme order statistics from two independent samples. The post-selection literature uses “Šidák intervals” for the usual simultaneous confidence intervals under independence and then studies how much they can be shortened once a selection rule is taken into account [2602.21359] [1610.09502] [1906.00505] [2208.02521].

| Usage of “Šidák procedure” | Core rule | Primary target |
|---|---|---|
| Multiple testing | $\alpha' = 1-(1-\alpha)^{1/n}$ | FWER |
| Post-selection intervals | Šidák simultaneous intervals under independence | SoP baseline |
| Two-sample nonparametrics | precedence–exceedance statistics | ordered or two-sided alternatives |

A recurrent misconception is that every reference to a “Šidák procedure” concerns the multiple-comparison correction. The two-sample papers explicitly reject that identification: their “Šidák-type” terminology refers instead to the 1957 precedence–exceedance test and its descendants, not to the multiplicity inequality used in simultaneous inference [1610.09502] [2208.02521].

## 2. Classical multiple-testing Šidák correction

In the classical setting, there are $n$ tests with independent $p$-values satisfying
$$
p_i \sim \mathrm{Uniform}(0,1)
$$
under their respective null hypotheses. The Šidák single-step procedure uses the per-test level
$$
\alpha_{\text{Sidak}} = 1-(1-\alpha)^{1/n}.
$$
For one-sided Gaussian tests with $X_i \sim N(0,1)$ under $H_{0i}$, the rejection rule is
$$
X_i > c_{\text{Sid}}(n,\alpha)
\coloneqq \Phi^{-1}\bigl((1-\alpha)^{1/n}\bigr).
$$
Under independence and the global null,
$$
\mathrm{FWER}
= 1-\Pr\bigl(\forall i:\, X_i \le c_{\text{Sid}}\bigr)
= 1-\Phi(c_{\text{Sid}})^n
= \alpha.
$$
Thus the procedure is exact, not merely conservative, in the independent case [2602.21359].

For two-sided Gaussian tests, the cited paper uses the symmetric rule
$$
|X_i| > c_{\text{Sid}}(2n,\alpha)
\coloneqq \Phi^{-1}\bigl((1-\alpha)^{1/(2n)}\bigr), \qquad i=1,\dots,n.
$$
The same paper also emphasizes the asymptotic equivalence between the Šidák cutoff and a suitably adjusted Bonferroni cutoff,
$$
c_{\text{Bon}}(n,\alpha)
= \Phi^{-1}\left(1-\frac{-\log(1-\alpha)}{n}\right),
$$
through the expansion
$$
(1-\alpha)^{1/n}
= 1-\frac{-\log(1-\alpha)}{n} + O\!\left(\frac{1}{n^2}\right).
$$
This yields the common asymptotic form
$$
c_n(\alpha)
= \Phi^{-1}\left(1-\frac{-\log(1-\alpha)-o(1)}{n}\right),
$$
which is the key device for comparing Šidák and adjusted Bonferroni thresholds in high-dimensional regimes [2602.21359].

## 3. Weak dependence and asymptotic exactness

The paper "Some Asymptotic Results on Multiple Testing under Weak Dependence" studies the same classical Šidák cutoff when the test statistics are not independent but form a weakly correlated Gaussian sequence. The model is
$$
X_i \sim N(\mu_i,1), \qquad i=1,\dots,n,
$$
with correlations $\rho_{ij} = \mathrm{Corr}(X_i,X_j)$ satisfying the weak dependence condition
$$
\rho_m = o\!\left(\frac{1}{\log m}\right)
\quad \forall\, 1\le m\le n
\quad \text{as } n\to\infty,
$$
and a uniform bound away from $1$ [2602.21359].

The main result is asymptotic exactness of the usual independence-based Šidák threshold. Under the weakly dependent standard Gaussian setting, both the adjusted Bonferroni procedure and the Šidák procedure satisfy
$$
\lim_{n\to\infty}\mathrm{FWER}(n,c_{\text{Sid}}(n,\alpha),\alpha,\Sigma_n)=\alpha
$$
whenever the proportion of true nulls tends to one,
$$
\lim_{n\to\infty}\frac{n_0}{n}=1.
$$
The analogous two-sided result also holds with the cutoff $c_{\text{Sid}}(2n,\alpha)$ [2602.21359].

The mechanism is extreme-value asymptotics. Under the cited weak dependence condition, exceedances above high thresholds behave asymptotically as if they were independent, so the number of exceedances is approximately Poisson. The probability of no exceedance therefore converges to the same limit as in the independent case. A closely related theorem yields asymptotic formulas for the $k$-FWER through the $k$th largest order statistic of the true-null subvector. The paper also studies power through
$$
\mathrm{AnyPwr} = \Pr(\text{at least one false null is rejected}),
$$
and shows that, for both adjusted Bonferroni and Šidák, $\mathrm{AnyPwr}\to 1$ when the strongest signal dominates the $\sqrt{2\log n_1}$ threshold scale [2602.21359].

The simulation evidence is explicitly numerical. For $\alpha=0.10$ and $\alpha=0.05$, with $n\in\{2500,5000,7500,10000\}$ and $\delta\in\{0.1,0.25,0.5,0.75,1\}$ under a product correlation structure satisfying the weak dependence condition, the estimated FWER for the one-sided Šidák procedure is reported as very close to the nominal level, “e.g. around 0.099–0.100 for $\alpha=0.10$, around 0.049–0.050 for $\alpha=0.05$,” with essentially identical values for the adjusted Bonferroni benchmark [2602.21359].

## 4. Šidák intervals in post-selection inference

In the post-selection confidence-interval setting, Šidák enters as a benchmark for simultaneous coverage rather than as the final inferential target. The problem is to report intervals only for a selected subset of parameters, such as the largest $k$ of $m$ shift parameters, while controlling the **simultaneous over selected** error rate
$$
\mathrm{SoS}
\equiv
\Pr\bigl\{\exists i\in S(Y): \mathcal I_i(Y)\not\ni \theta_i\bigr\}.
$$
The cited paper compares newly constructed SoS-controlling intervals to standard Bonferroni and Šidák simultaneous intervals [1906.00505].

For independent estimators, a two-sided Šidák interval has the form
$$
[Y_i-c,\; Y_i+c]
$$
with
$$
c = -\sup\left\{c:\Pr(Y_i-\theta_i \le c)\le \frac{1-(1-\alpha)^{1/m}}{2}\right\}.
$$
This interval controls simultaneous coverage over all $m$ parameters under independence, but it does not use the knowledge that only a selected subset will actually be reported. The paper’s central comparison is that the new SoS intervals “improve substantially over Šidák intervals when $k$ is small compared to $m$, and approach the standard Bonferroni-corrected intervals when $k \approx m$” [1906.00505].

The general SoS construction for the largest $k$ of $m$ independent shift estimators yields intervals
$$
\mathcal I_{(i)}(y)
=
\bigl[y_{(i)}-c_i,\; y_{(i)}+\bar c_i\bigr],
\qquad i=1,\dots,k,
$$
with tuning constants chosen so that
$$
\underline{\lambda}=\delta\alpha/m,
\qquad
\bar{\lambda}=(1-\delta)\alpha/k,
\qquad \delta\in(0,1).
$$
The multiplicity structure is asymmetric: the lower endpoints still reflect $m$, while the upper endpoints depend only on $k$. This is the source of the length reduction relative to Šidák intervals when $k<m$ [1906.00505].

The paper also isolates special low-dimensional cases. For $m=2$, $k=1$, selection by the larger of two observations, and exchangeable symmetric errors, the unadjusted interval
$$
[y_{(1)}-c_{\alpha/2},\; y_{(1)}+c_{\alpha/2}]
$$
has exact SoS coverage. For the largest absolute value of two independent normal estimators, the derived SoS interval is strictly shorter than the corresponding Šidák simultaneous interval: at $\alpha=0.05$, its maximum width is about $93.6\%$ of Šidák’s width, and as $|y_1|\to\infty$ its length tends to about $88\%$ of Šidák’s [1906.00505].

## 5. The nonparametric two-sample Šidák test

A separate literature uses **Šidák test** to mean a nonparametric two-sample test based on precedence and exceedance counts. In the formulation studied in "Šidák-type tests for the two-sample problem based on precedence and exceedance statistics," two independent samples are observed,
$$
X_1,\dots,X_m \overset{i.i.d.}{\sim} F,
\qquad
Y_1,\dots,Y_n \overset{i.i.d.}{\sim} G,
$$
with $F$ and $G$ absolutely continuous. The null hypothesis is
$$
H_0: F(x)=G(x)\quad \text{for all }x,
$$
and the alternatives are ordered alternatives expressed through stochastic ordering [1610.09502].

The original statistic counts extremes. Let $A_0$ be the number of $Y$ observations exceeding the largest $X$ observation, and let $B_0$ be the number of $X$ observations preceding the smallest $Y$ observation. The original Šidák statistic is
$$
V_0 = A_0 + B_0.
$$
Large values indicate a strong separation in which the $X$ sample accumulates at low ranks and the $Y$ sample at high ranks. Under $H_0$, its exact distribution is distribution-free, and the cited paper records the closed form
$$
P(V_0 \le z)
=
\frac{
{m+n-z \choose n}
+
\sum_{j=0}^{z-1} {m+n-z-1 \choose m-j}
}{
{m+n \choose n}
}.
$$
This is the original two-sample Šidák statistic and distribution referenced by the later “Šidák-type” constructions [1610.09502].

The generalization replaces the extreme thresholds by inner order statistics. With
$$
s=\lfloor \rho m \rfloor,
\qquad
r=\lfloor \rho n \rfloor,
\qquad
0\le \rho <1,
$$
define
$$
A_s = \text{number of $Y$ observations larger than } X_{(m-s)},
$$
$$
B_r = \text{number of $X$ observations smaller than } Y_{(1+r)},
$$
and
$$
V_\rho = A_s + B_r.
$$
When $\rho=0$, one recovers $V_0$. Under $H_0$, the joint pmf of $(A_s,B_r)$ is derived exactly, and the cdf of $V_\rho$ is obtained by summing that joint pmf over the triangular region $i+k\le z$ [1610.09502].

Because the distribution is discrete, exact level-$\alpha$ comparison uses a randomized test. The rejection region is $V_\rho \ge c$, where $c$ is the smallest integer such that
$$
P(V_\rho \ge c \mid H_0)\le \alpha.
$$
If $\alpha_1=P(V_\rho\ge c)$ and $\alpha_2=P(V_\rho\ge c-1)$ bracket the nominal $\alpha$, then rejection at $V_\rho=c-1$ is randomized with probability
$$
\pi = \frac{\alpha-\alpha_1}{\alpha_2-\alpha_1}.
$$
The paper also gives large-sample approximations. For balanced large samples, the null law of $V_\rho$ is well approximated by a negative binomial distribution with parameters $2(s+1)$ and $1/2$, and “also fairly well by a chi-square distribution,” with practical adequacy reported for sample sizes $25$–$100$ when $m$ and $n$ are similar [1610.09502].

Power is studied under the Lehmann alternative
$$
G(x)=1-\bigl(1-F(x)\bigr)^{1/\eta},
\qquad \eta>1.
$$
The exact joint pmf of $(A_s,B_r)$ under this alternative is derived in terms of gamma functions, and exact or Monte Carlo power calculations show that inner thresholds can improve power relative to the original extreme-threshold statistic. For example, with $m=n=20$ and $\eta=2$, the reported powers are $0.4566$ for $V_0$, $0.5061$ for $V_1$, $0.5230$ for $V_2$, and $0.5182$ for $V_3$ [1610.09502].

## 6. Maximal precedence–exceedance extensions

The paper "A class of Šidák-type tests based on maximal precedence and exceedance statistic" extends the nonparametric two-sample tradition by replacing total counts with maximal local counts. Two independent samples $X_1,\dots,X_m\sim F$ and $Y_1,\dots,Y_n\sim G$ are again assumed, with $F$ and $G$ univariate and absolutely continuous. For integers
$$
r = \lfloor p_1 n \rfloor + 1,
\qquad
s = \lfloor p_2 n \rfloor + 1,
\qquad
0\le p_1,p_2<1,
\qquad
2\le r+s \le n,
$$
the construction forms interval counts $fp_i$ near the lower tail of the ordered $Y$ sample and $fe_i$ near the upper tail, then defines
$$
P_r = \max\{fp_i : i=1,\dots,r\},
\qquad
E_s = \max\{fe_i : i=1,\dots,s\},
$$
and finally
$$
T_{r,s} = P_r + E_s.
$$
A common symmetric specialization sets $r=s=\lfloor pn\rfloor+1$ and writes $T_r=P_r+E_r$ [2208.02521].

This statistic is presented as a generalization of the original Šidák test. The paper states that “The test defined by $T_r$ includes Šidák’s test as a special case when $r=1$.” The rationale for the maximal version is the “masking effect”: a strong local concentration of one sample in a narrow interval can be diluted if only total precedences or exceedances are counted. By summing maximal early and late concentrations, $T_{r,s}$ is designed for a two-sided alternative, unlike earlier precedence-only or one-sided Šidák-type procedures [2208.02521].

Under $H_0$, the exact distribution is obtained from the joint distribution of the full count vector
$$
\{fp_1,\dots,fp_r,fe_1,\dots,fe_s\}.
$$
The resulting null law depends only on $(m,n,r,s)$ and is therefore distribution-free. The paper also derives the joint count distribution under the Lehmann alternative
$$
H_1: G(x)=F(x)^\gamma, \qquad \gamma>0,
$$
in terms of Beta functions, and thus obtains the distribution of $T_{r,s}$ under the alternative as well. As with the earlier paper, the test is discrete, so exact-size comparison is implemented with a randomized rule based on the tail probabilities at the critical boundary [2208.02521].

The power comparisons are explicitly directional. For $m=n=25$, Table 8 reports that the competing Šidák-type precedence–exceedance statistic $V_r$ and the maximal precedence statistic $Q_r$ have power essentially $0$ for $\gamma<1$, whereas $T_r$ has substantial power in both directions. The authors therefore describe $V_r$ and $Q_r$ as finite-sample biased for certain alternatives and present $T_r$ as suitable for two-sided alternatives [2208.02521].

A real-life example concerns failure voltages for two types of cable insulation with $m=n=20$. When one group is treated as training and the other as test, $T_r$ rejects $H_0$ at the $5\%$ level for $r=3,4$, whereas $V_r$ rejects for all $r=1,2,3,4$. After swapping the training and test labels, $T_r$ still rejects $H_0$ at the $5\%$ level for all $r$, while $V_r$ fails to reject for any $r$. This illustrates the central distinction between a two-sided maximal precedence–exceedance procedure and a one-sided Šidák-type predecessor [2208.02521].

Source: https://www.emergentmind.com/topics/sidak-procedure