Learning antagonistic hidden-layer couplings

Determine whether contrastive divergence, when initialized with a random inter-module coupling matrix rather than the fixed coupling matrix \(\boldsymbol{x}=\boldsymbol{g}^{-1}\), converges autonomously to the antagonistic coupling imposed in the modular restricted Boltzmann machine, so that the hidden units learn both to classify and to compete.

Background

The modular restricted Boltzmann machine studied in the paper fixes the hidden-layer inter-module coupling matrix to x=g−1\boldsymbol{x}=\boldsymbol{g}^{-1}. This coupling encodes antagonistic interactions: modules are encouraged to activate different archetype labels, which supports pattern disentanglement. The analytical fixed-point theorem concerns learning only the intra-module weights while keeping this coupling fixed.

The paper leaves unresolved whether the antagonistic coupling itself can be learned through contrastive divergence. Resolving this question would establish whether the learning dynamics can autonomously acquire both the classification weights and the competitive interactions required for disentanglement, rather than relying on a manually imposed architectural coupling.

References

However, probably more interesting is investigating whether \bm x needs to be fixed at all. If contrastive divergence, started from a random \bm x, converges on its own to the antagonistic coupling we impose by hand, units would learn not just to classify but to compete. Hebb's postulate is that cells firing together wire together; we leave open whether cells asked to fire apart learn to stay apart.

— Classifications in modular restricted Boltzmann machines  (2610.08612 - Agliari et al., 6 Oct 2026) in Section Conclusions