Conditionally IID: Theory & Applications

• CIID is a framework where random variables become both independent and identically distributed once conditioned on a latent variable or covariate.
• It underpins modern conditional independence tests by mimicking null distribution structures to assess model validity and dependency patterns.
• CIID concepts extend to statistical mechanics and representation learning, providing insights into macro-conditioning effects and invariant data transformations.

Conditionally independent and identically distributed (CIID) refers to a family or sequence of random variables that becomes both independent and identically distributed after conditioning on a specified variable, parameter, or $\sigma$-field. In standard probabilistic form, random variables $X_1,\dots,X_N$ are CIID given $Y$ if, under the conditional law $P(\cdot\mid Y=y)$, the joint factorizes as

$$P(X_1,\dots,X_N\mid Y=y)=\prod_{i=1}^N P(X_i\mid Y=y),$$

and the conditional marginal is the same for all $i$. Across recent work, this notion appears in several technically distinct roles: as the null structure in conditional independence testing, as the ensemble-level description of particle systems given macro-parameters, as the latent-measure representation behind exchangeability, and as a target of learned invariant representations rather than a blanket assumption on raw data (Sen et al., 2018).

1. Formal definition and probabilistic position

The defining feature of CIID is that the conditioning object carries the heterogeneity. Once conditioned on that object, both factorization and identical conditional marginals hold. In the conditional independence-testing literature, this is often instantiated through the conditional-independence factorization

$$p^{CI}(\mathbf{x},\mathbf{y},\mathbf{z})=p(\mathbf{z})\,p(\mathbf{y}\mid\mathbf{z})\,p(\mathbf{x}\mid\mathbf{z}),$$

so that an i.i.d. sample from $p^{CI}$ is a sequence in which, for each observation, $X_i$ and $Y_i$ are conditionally independent given $X_1,\dots,X_N$0, and the same conditional law applies across $X_1,\dots,X_N$1 (Sen et al., 2018).

CIID should be distinguished from the weaker notion of conditionally identically distributed (c.i.d.) sequences. A sequence $X_1,\dots,X_N$2 is $X_1,\dots,X_N$3-c.i.d. if

$X_1,\dots,X_N$4

for all bounded measurable $X_1,\dots,X_N$5, all $X_1,\dots,X_N$6, and all $X_1,\dots,X_N$7. This imposes equality of future conditional distributions given the past, but not conditional independence of those futures. Exchangeable sequences, by contrast, admit a de Finetti representation: there exists a random probability measure $X_1,\dots,X_N$8 such that, conditionally on $X_1,\dots,X_N$9, the sequence is i.i.d. with directing measure $Y$0. In that sense, exchangeability yields a classical CIID representation, whereas c.i.d. is strictly weaker outside the stationary setting (Fortini et al., 2016).

A recurrent misconception is that CIID is interchangeable with exchangeability. The relation is more precise than that. Exchangeability implies a conditional i.i.d. structure via a latent directing measure; c.i.d. becomes equivalent to exchangeability only for stationary sequences; and many applied constructions labeled “CIID-like” rely on conditioning on observed covariates, macro-parameters, or learned representations rather than on an exchangeable latent law (Fortini et al., 2016).

2. CIID as the null architecture in conditional independence testing

Several recent CI-testing frameworks operationalize CIID by constructing or approximating samples from a null distribution in which conditional independence is built in. In “Mimic and Classify,” the null hypothesis is

$Y$1

and the method is realized in two steps: mimic the CI distribution closely enough to recover the support, and classify to distinguish the true joint from the CI distribution. The data are assumed i.i.d. from the true joint, while under the null the sequence is CIID with respect to $Y$2: conditioned on $Y$3, the pair $Y$4 has conditional law $Y$5, identically across $Y$6. The support-overlap condition

$Y$7

is sufficient for the Bayes-optimal error-gap characterization used by the test (Sen et al., 2018).

“Model-Powered Conditional Independence Test” converts the same CI problem into binary classification by generating samples close to

$Y$8

Its nearest-neighbor bootstrap constructs synthetic samples whose single-sample marginal distribution is within $Y$9 in total variation distance of $P(\cdot\mid Y=y)$0, so the generated data are not exactly i.i.d. from the ideal CI law but are “approximately CIID in the sense of marginal distributions,” with a separate analysis for the near-independent dependence structure induced by the bootstrap (Sen et al., 2017).

“Score-based Generative Modeling for Conditional Independence Testing” replaces GAN-style mimicry with sliced conditional score matching and Langevin dynamics conditional sampling. The method learns the conditional score

$P(\cdot\mid Y=y)$1

generates null samples from an estimated $P(\cdot\mid Y=y)$2, and inserts them into a conditional randomization test. Under $P(\cdot\mid Y=y)$3, the sequence of observed and resampled triples is exchangeable, yielding valid p-values in the exact-modeling case, and the paper proves asymptotic Type I error control

$P(\cdot\mid Y=y)$4

under approximate conditional modeling (Ren et al., 29 May 2025).

Differential privacy adds another layer but retains the same i.i.d.-plus-CI structure. “Differentially Private Conditional Independence Testing” designs a private generalized covariance measure test and a private conditional randomization test, both under i.i.d. sampling of $P(\cdot\mid Y=y)$5 and the null $P(\cdot\mid Y=y)$6; the paper describes these as the first private CI tests with rigorous theoretical guarantees that work for the general case when $P(\cdot\mid Y=y)$7 is continuous (Kalemaj et al., 2023).

3. CIID, macro-conditioning, and conditional equiprobability in statistical mechanics

In “Infomechanics of Independent and Identically Distributed Particles,” the thermodynamic system is modeled by particle energy labels $P(\cdot\mid Y=y)$8 that are independent and identically distributed under the unconditional law

$P(\cdot\mid Y=y)$9

Within this framework, CIID appears when one regards temperature or related macro-parameters as conditioning variables. In the canonical setting,

$$P(X_1,\dots,X_N\mid Y=y)=\prod_{i=1}^N P(X_i\mid Y=y),$$0

and conditioning on $$P(X_1,\dots,X_N\mid Y=y)=\prod_{i=1}^N P(X_i\mid Y=y),$$1 yields a law under which the particle energies are conditionally independent and identically distributed. By contrast, conditioning on detailed macrostates such as the occupancy vector $$P(X_1,\dots,X_N\mid Y=y)=\prod_{i=1}^N P(X_i\mid Y=y),$$2 does not preserve independence of components: fixed counts induce strong correlations among the $$P(X_1,\dots,X_N\mid Y=y)=\prod_{i=1}^N P(X_i\mid Y=y),$$3 (Spalvieri, 2022).

The central conditional result of the paper is not CIID of components under fixed occupancy, but conditional equiprobability of microstate vectors. If $$P(X_1,\dots,X_N\mid Y=y)=\prod_{i=1}^N P(X_i\mid Y=y),$$4 is a fixed occupancy vector, then every compatible microstate $$P(X_1,\dots,X_N\mid Y=y)=\prod_{i=1}^N P(X_i\mid Y=y),$$5 satisfies

$$P(X_1,\dots,X_N\mid Y=y)=\prod_{i=1}^N P(X_i\mid Y=y),$$6

with $$P(X_1,\dots,X_N\mid Y=y)=\prod_{i=1}^N P(X_i\mid Y=y),$$7. This yields the interpretation of Boltzmann–Planck entropy as conditional Shannon entropy under a deterministic condition: $$P(X_1,\dots,X_N\mid Y=y)=\prod_{i=1}^N P(X_i\mid Y=y),$$8

This distinction is conceptually important. The paper explicitly separates three structures: unconditional IID particle laws in canonical or grand canonical descriptions, CIID when conditioning on macro-parameters such as temperature, and uniform conditional laws on constrained microstate sets when conditioning on detailed macrostates such as occupancy or fixed energy. A common confusion is to collapse these into a single “equally probable microstates” axiom; the infomechanical formulation instead derives approximate equiprobability inside the typical set and exact conditional equiprobability on macrostate fibers, while reserving CIID for a different conditioning layer (Spalvieri, 2022).

4. Exchangeability, c.i.d., partial c.i.d., and almost-i.i.d. generalizations

The sequence notion most directly adjacent to CIID in Bayesian probability is c.i.d. A $$P(X_1,\dots,X_N\mid Y=y)=\prod_{i=1}^N P(X_i\mid Y=y),$$9-c.i.d. sequence has predictive measures

$i$0

forming a measure-valued martingale, and every exchangeable sequence is c.i.d. with respect to its natural filtration. The converse fails in general, but for stationary sequences c.i.d. is equivalent to exchangeability. The same pattern extends to arrays: partially c.i.d. families generalize c.i.d. just as partial exchangeability generalizes exchangeability, are equivalent to partial exchangeability for stationary processes, and preserve asymptotic properties including strong laws of large numbers and two central limit theorems (Fortini et al., 2016).

This establishes a hierarchy. CIID is the strongest among these nearby notions because it requires conditional independence and conditional identicality; c.i.d. keeps only the second component; exchangeability recovers CIID through an integral mixture over a latent directing measure. The paper’s emphasis on “asymptotic agreement of predictions and empirical means” places c.i.d. and partially c.i.d. in the foundations of Bayesian statistics rather than in exact latent-product representations (Fortini et al., 2016).

Quantum information theory introduces a different generalization. “Almost-iid information theory” argues that operationally motivated symmetry assumptions, together with de Finetti theorems, imply almost-iid states rather than perfect i.i.d. states. In the paper’s terminology as elaborated in the details, a classical mixture

$i$1

is naturally read as conditionally i.i.d. with respect to the latent parameter $i$2: conditioned on $i$3, the systems are i.i.d. The almost-iid setting then permits small defect sets while preserving asymptotic entropy rates. The main theorem proves that for almost-iid states with defect size $i$4,

$i$5

and, as an application, squashed entanglement is robust for almost-iid states, asymptotically matching its value on iid states (Mazzola et al., 16 Mar 2026).

A plausible implication is that CIID is best viewed not as an isolated binary property but as one point in a spectrum of latent-variable, predictive, and symmetry-based regularities: exact conditional product structure, weaker predictive identicality, and almost-product structure with controlled defects.

5. Learned representations as induced IID or CIID structure

“Towards IID representation learning and its application on biomedical data” shifts the emphasis from assuming IID to learning it. The setup starts with independent random vectors $i$6, a representation map

$i$7

and transformed variables $i$8. For each $i$9, a task-relevant query distribution $$p^{CI}(\mathbf{x},\mathbf{y},\mathbf{z})=p(\mathbf{z})\,p(\mathbf{y}\mid\mathbf{z})\,p(\mathbf{x}\mid\mathbf{z}),$$0 is specified, and the induced distribution of the representation is the pushforward

$$p^{CI}(\mathbf{x},\mathbf{y},\mathbf{z})=p(\mathbf{z})\,p(\mathbf{y}\mid\mathbf{z})\,p(\mathbf{x}\mid\mathbf{z}),$$1

The paper defines a $$p^{CI}(\mathbf{x},\mathbf{y},\mathbf{z})=p(\mathbf{z})\,p(\mathbf{y}\mid\mathbf{z})\,p(\mathbf{x}\mid\mathbf{z}),$$2-IID symmetry by

$$p^{CI}(\mathbf{x},\mathbf{y},\mathbf{z})=p(\mathbf{z})\,p(\mathbf{y}\mid\mathbf{z})\,p(\mathbf{x}\mid\mathbf{z}),$$3

with generalization when the equality extends to $$p^{CI}(\mathbf{x},\mathbf{y},\mathbf{z})=p(\mathbf{z})\,p(\mathbf{y}\mid\mathbf{z})\,p(\mathbf{x}\mid\mathbf{z}),$$4 (Wu et al., 2022).

Because the $$p^{CI}(\mathbf{x},\mathbf{y},\mathbf{z})=p(\mathbf{z})\,p(\mathbf{y}\mid\mathbf{z})\,p(\mathbf{x}\mid\mathbf{z}),$$5 are independent, the $$p^{CI}(\mathbf{x},\mathbf{y},\mathbf{z})=p(\mathbf{z})\,p(\mathbf{y}\mid\mathbf{z})\,p(\mathbf{x}\mid\mathbf{z}),$$6 remain independent. When the query distributions are identical, the $$p^{CI}(\mathbf{x},\mathbf{y},\mathbf{z})=p(\mathbf{z})\,p(\mathbf{y}\mid\mathbf{z})\,p(\mathbf{x}\mid\mathbf{z}),$$7 are IID with respect to those query distributions. The paper is explicit that the query distribution may be marginal, conditional, or interventional. In the conditional or interventional cases, the learned symmetry is naturally interpretable as CIID-like: identicality holds at the level of the task-relevant conditional law, not necessarily in the raw observational distribution (Wu et al., 2022).

The paper uses this idea in out-of-distribution generalization on biomedical data, including molecular prediction tasks on RxRx1 and SCRC, with representation learning implemented through a Restyle encoder, a StyleGAN decoder, and instance normalization. Instance normalization enforces identical mean and variance of certain latent components across images, so the induced latent variables are described as approximately identically distributed in those coordinates. This suggests a representational view of CIID: after suitable transformation, heterogeneous environments may yield variables that are independent across samples and share the same task-relevant conditional law, even when the raw inputs do not (Wu et al., 2022).

6. Related notions, algebraic formulations, and recurring confusions

Several nearby concepts are often conflated with CIID but are technically distinct. “Conditional Inter-Causally Independent” (CICI) node distributions arise in noisy-or models for a collider structure $$p^{CI}(\mathbf{x},\mathbf{y},\mathbf{z})=p(\mathbf{z})\,p(\mathbf{y}\mid\mathbf{z})\,p(\mathbf{x}\mid\mathbf{z}),$$8. There the statement is state-specific: $$p^{CI}(\mathbf{x},\mathbf{y},\mathbf{z})=p(\mathbf{z})\,p(\mathbf{y}\mid\mathbf{z})\,p(\mathbf{x}\mid\mathbf{z}),$$9 with the defining condition that the likelihood matrix for one evidence state has rank $p^{CI}$0 or, equivalently in the binary case, determinant $p^{CI}$1. This is conditional intercausal independence at a particular evidence value, not a sequence-level CIID property (Agosta, 2013).

Algebraic statistics provides a separate language for conditional independence. For finite variables, $p^{CI}$2 corresponds to rank $p^{CI}$3 of the probability matrix $p^{CI}$4, and $p^{CI}$5 corresponds to rank $p^{CI}$6 of each slice $p^{CI}$7, equivalently vanishing of all $p^{CI}$8 minors. With hidden variables, CI statements produce higher determinantal conditions. The details explicitly note that CIID models fit this framework by combining determinantal constraints for conditional independence with symmetry or equality constraints enforcing identical conditional distributions (Clarke et al., 2019).

Three clarifications recur across the literature. First, CIID is not the same as c.i.d.; the latter omits conditional independence (Fortini et al., 2016). Second, CIID is not the same as conditional equiprobability over a constrained set; in statistical mechanics, fixing occupancy or total energy generally destroys independence among components, even though it induces a uniform law over compatible microstate vectors (Spalvieri, 2022). Third, CIID is not a universal property of raw data. In representation learning and CI testing alike, it is often a model-relative or conditioning-relative property: given $p^{CI}$9, given $X_i$0, given a latent directing measure, or given a learned representation (Wu et al., 2022).

Taken together, these uses show that CIID is best understood as a conditional product law whose meaning depends entirely on what is being conditioned on. In one setting the conditioning object is an observed covariate, in another a thermodynamic macro-parameter, in another a latent directing measure, and in another a learned invariant representation. The common mathematical core is the same, but the induced dependence structure away from that conditioning layer can differ substantially.