Identification with finite environments for nonparametric representation-level invariance

Determine whether identification in representation-level invariance learning is possible with any finite number of environments when the representation map Φ: ℝ^d → ℝ^r is allowed to be nonparametric. Specifically, prove or refute the conjecture that identification is impossible for all finite |E| when Φ belongs to a nonparametric function class, in contrast to the linear case where at least r environments are necessary even under sufficient heterogeneity and known r.

Background

The paper contrasts variable-selection-level invariance with representation-level invariance (as in IRM), noting that identification requirements can be substantially different. For linear representation learning of dimension r, the authors remark that at least r distinct environments are necessary for identification, even assuming sufficient heterogeneity.

They then raise a broader question for nonparametric representation classes: whether any finite number of environments could ever suffice to identify the representation Φ and the corresponding invariant predictor. This conjecture targets the fundamental limits of environment-based identification in nonparametric settings and directly impacts the feasibility and design of representation-level invariance methods.

References

We conjecture that any finite number of environments |E|<∞ may be impossible for identification if Φ lies in some nonparametric function class.

— Causality Pursuit from Heterogeneous Environments via Neural Adversarial Invariance Learning  (2405.04715 - Gu et al., 2024) in Appendix, Discussion on the Methods (Q&A about representation-level invariance)

A central open problem in representation learning is identifiability: whether the data determine the learned representation uniquely, up to simple transformations such as permutation and element-wise reparameterization, so that it recovers the factors that generated the data rather than one of many equally predictive alternatives.

— Structure-agnostic Causal Representation Learning  (2610.00968 - Behnam et al., 1 Oct 2026) in Appendix, Section 'Applications and Broader Connections', subsection 'Identifiability beyond an assumed structure'