Unknown sparsity pattern for enforcing encoder constraints in the n=0 case

Determine the pixel-to-slot sparsity pattern in ground-truth generators f belonging to the interaction class with n=0—specifically, identify which image pixels x_i influence which latent slots z_l—so that the corresponding Jacobian sparsity constraints can be leveraged to restrict the inverse generator class G and enable compositional generalization in non-generative approaches.

Background

The authors discuss a special case (interaction degree n=0) where each pixel depends on a single slot and there are no interactions such as occlusions. In this regime, a left inverse g can be found whose Jacobian satisfies a sparsity constraint, suggesting potential for constraining encoders.

However, the practical application of this structural constraint depends on knowing the underlying pixel-to-slot mapping, which the authors explicitly state is not known a priori, making it challenging to exploit this structure for encoder design and regularization.

References

However, this remains challenging in practice because the sparsity pattern (i.e., which slots z_l depends on which pixel x_i) is not known a-priori.

— Generation is Required for Data-Efficient Perception (2512.08854 - Brady et al., 9 Dec 2025) in Special case of n=0, Theoretical Analysis (Section 3)