Probabilistic Invariance: Theory & Applications

• Probabilistic invariance is the property that probability distributions and conditional mechanisms remain unchanged under specific transformations, supporting robust causal inference and prediction.
• It underpins advanced methods in safety verification and control, enabling dynamic programming, linear programming relaxations, and invariant neural network designs.
• Applications span statistics, machine learning, and risk assessment, providing rigorous guarantees in uncertain and heterogeneous environments.

Probabilistic invariance denotes structural properties of probability distributions, conditional mechanisms, randomized rules, or probabilistic models that remain unaltered (in law or function) under specified transformations, environmental changes, feedbacks, or under certain probabilistic operations. It plays a foundational role in areas ranging from causal inference and robust prediction in statistics, to safety verification in stochastic dynamical systems, to the design of invariant neural networks and generative models, as well as in the analysis of probabilistically invariant quantum and physical systems. The notion is mathematically precise, admits deep dualities with optimization and risk theory, and enables rigorous guarantees for robustness, safety, or identifiability in environments subject to heterogeneity, adversarial or random perturbations.

1. Probabilistic Invariance in Causality and Robust Prediction

In statistical causal inference, probabilistic invariance formalizes the principle that the conditional law of the response given certain covariates remains invariant across a collection of environments, interventions, or perturbations. Given data $(Y^e, X^e)$ from environments $e \in \mathcal{E}$, probabilistic invariance asserts that there exists $S^* \subset \{1,\ldots,p\}$ such that

$$\mathcal{L}(Y^e \mid X^e_{S^*}) \equiv \mathcal{L}(Y^{e'} \mid X^{e'}_{S^*}) \quad \forall e,e' \in \mathcal{E}.$$

In linear models, this corresponds to the existence of $\beta^*$ with $\operatorname{supp}(\beta^*) = S^*$ such that $Y^e = X^e \beta^* + \epsilon^e$, $\epsilon^e \perp X^e_{S^*}$, and the noise law $\epsilon^e \sim F$ independent of $e$ (Bühlmann, 2018).

This invariance connects directly to causal identification: the set $S^*$ of stable predictors typically coincides with the set of causal parents. Learning $S^*$ via hypothesis testing across environments (as in Invariant Causal Prediction, ICP) ensures reliable causal inference with asymptotic control of type I error under standard structural equation model (SEM) assumptions.

Probabilistic invariance also provides the foundation for robust prediction formulations: minimizing worst-case risk over future or unseen environments is equivalent to minimizing over those predictors for which invariance holds. The use of anchor regression and its nonlinear extensions further generalizes invariance ideas, yielding estimators with provably minimized maximal risk over allowed perturbation classes (Bühlmann, 2018).

2. Mathematical Characterizations in Control and Verification

In stochastic control and verification, probabilistic invariance describes sets or properties that remain invariant with high probability, even under the evolution of controlled Markov or stochastic differential systems. For discrete-time stochastic systems with state $X_k$, control $U_k$, transition kernel $T(\cdot|x,u)$, and safe set $A$, the key object is the probability

$$V^{\pi}_k(x) = \Pr[X_j \in A,\,k \leq j \leq N \mid X_k = x,\,\pi]$$

under policy $\pi$. The maximal (resp. minimal) probabilistic invariant set at level $p$ consists of all $x$ such that $V_{I\uparrow,k}^*(x) \geq p$ (resp. $V_{I\downarrow,k}^*(x) \geq p$), where $V_{I\uparrow,k}^*$ and $V_{I\downarrow,k}^*$ denote extremized value functions over policies (Schmid et al., 2022).

Probabilistic invariance is dual to reachability: the optimal probability of remaining in $A$ equals $1$ minus the optimal probability of reaching $A^\complement$. These invariance properties can be characterized either by dynamic programming or as solutions to infinite-dimensional linear programs, facilitating both their theoretical paper and numerical approximation (Schmid et al., 2022, Wang et al., 23 Apr 2024, Wu et al., 13 Apr 2024).

For continuous-time SDEs, a similar concept holds, with the infinitesimal generator $A$ acting on the “probability-to-go” function $P$, such that, under well-chosen controls, the expected probability of remaining within a safe set does not decrease below a threshold over time (Wang et al., 23 Apr 2024, Wang et al., 16 Nov 2025).

3. Probabilistic Invariance in Learning and Representation

Probabilistic invariance emerges naturally in machine learning when modeling functions or distributions that are invariant under group actions or stochastic transformations. The formalism distinguishes between:

• Deterministic function-level invariance: $f(g \cdot x) = f(x)$ for all $g \in G$.
• Probabilistic (distributional) invariance: $P_X = P_{g \cdot X}$, i.e., $X \overset{d}{=} g \cdot X$.
• Probabilistic symmetry in conditional distributions or kernels: $P_{Y|X=g \cdot x}$ is $G$-invariant, or $P_{(X,Y)} = P_{(g \cdot X, Y)}$ (Bloem-Reddy et al., 2019).

General representation theorems characterize invariant or equivariant models as those constructed via suitable functions of maximal invariants and independent randomization. For stochastic neural architectures, any $G$-invariant model has the noise-outsourced form $Y = f(\zeta, M(X))$, where $M(X)$ is a maximal invariant of the group action, and $\zeta$ is independent noise. This framework unifies classical probabilistic symmetries (de Finetti, Aldous–Hoover) with modern invariant and equivariant neural network architectures (Bloem-Reddy et al., 2019).

Approximate probabilistic invariance, or high-probability invariance, appears in randomized linear classifiers and scalable generative models, where invariance holds with high probability (quantified in terms of sample and resource parameters) rather than deterministically, facilitating constant-parameter universal approximation for set, graph, and symmetric data domains (Cotta et al., 2023, Papež et al., 15 Mar 2025).

4. Invariance Principles in Probability, Physics, and Risk

Canonical probability laws arise uniquely from symmetry or invariance principles. Shift invariance implies exponential (Boltzmann) distributions; stretch (affine) invariance implies conservation of means; rotational invariance yields the normal law. In each case, invariance under a transformation family determines the form of the probability distribution by a functional equation argument, without appeal to physical mechanics or entropy maximization. Broader families (Gamma, Beta, Weibull, Pareto) result from affine-invariant measurement scales (Frank, 2016).

Probabilistic invariance also serves as a key organizing idea in statistical tests, notably via generalized invariance principles like Stolarsky’s, which undergird geometric and permutation test theory, ensuring that (for instance) permutation $p$-values can be robustly and efficiently approximated via probabilistic geometric arguments derived from invariance (He et al., 2016).

The theory extends to quantum walks, where specific time-dependent parameterizations of evolution operators can leave all observable probability distributions unchanged—reflecting an underlying probabilistic gauge invariance (Montero, 2014).

In risk and decision theory, partial and strong partial law invariance generalize the idea of risk measures (such as expected shortfall or the entropic risk) that only respond invariantly to the law of losses on “trusted” sub-$\sigma$-algebras, interpolating between full law invariance and complete sensitivity, thus enabling robustified decision schemes in the presence of model uncertainty (Shen et al., 30 Jan 2024).

5. Applications and Computational Methods

Probabilistic invariance is central in:

• Causal variable selection and robust prediction: Algorithms such as Invariant Causal Prediction (ICP) and anchor regression utilize invariance to identify causal predictors or construct minimax-optimal estimators under heterogeneous environments (Bühlmann, 2018, Henzi et al., 2023).
• Analysis and control of stochastic and uncertain dynamical systems: Both offline and online certificate or controller synthesis methods (e.g., probabilistic barrier certificates, linear program relaxations, scenario programming) leverage invariance to guarantee safety or stability against stochastic transitions, policy uncertainty, or variable system parameters (Schmid et al., 2022, Fabiani et al., 2021, Wang et al., 23 Apr 2024, Wang et al., 16 Nov 2025).
• Invariant function learning and scalable neural modeling: Design and training of group-invariant/equivariant networks, probabilistic circuits, and generative models for set, graph, or rotationally-symmetric data (Bloem-Reddy et al., 2019, Cotta et al., 2023, Papež et al., 15 Mar 2025).
• Risk assessment and finance: Construction of partially law-invariant risk measures and corresponding tractable optimization/representation formulas (Shen et al., 30 Jan 2024).
• Probabilistic program verification: Automated computation of invariant relations among distributional moments, supporting verification and analysis of complex probabilistic loops (Kofnov et al., 2022).
• Statistical testing: Highly efficient and robust permutation $p$-value estimation via invariance-based geometric or kernel methods (He et al., 2016).

6. Limitations, Extensions, and Open Problems

The detection and exploitation of probabilistic invariance require observable heterogeneity or perturbations: without sufficient environmental variation, statistical tests for invariance have diminished power, and identifiability breaks down (Bühlmann, 2018). In high dimensions, computational costs (e.g., subset intersection in ICP, basis function growth in LP relaxations) can increase rapidly, motivating approximate, scalable, or probabilistic approaches (Schmid et al., 2022, Cotta et al., 2023).

In probabilistic prediction, full (score-wise) invariance is unachievable under arbitrary shifts in heteroscedastic models: only restricted classes of environment shifts and scores render invariance feasible, and identification relies on parametric form and environment diversity (Henzi et al., 2023).

Ongoing research addresses scalable verification (e.g., via sum-of-squares programming (Wu et al., 13 Apr 2024)), efficient invariant learning under resource constraints, and the integration of probabilistic invariance with human specifications, stochastic environments, and adaptive control (Wang et al., 16 Nov 2025).

Overall, probabilistic invariance provides a unifying mathematical and algorithmic principle across contemporary research in statistics, learning, dynamical systems, verification, and decision theory. It binds invariance, causality, robustness, and tractability, and its rigorous paper yields both sharp theoretical guarantees and practical algorithmic advances.