Sufficient and effective representation for downstream tasks

Determine a self-supervised learning representation that is both sufficient and effective across a variety of downstream tasks, where "sufficient" means downstream tasks can be completed by composing functions only on the learned representations rather than on the original data, and "effective" means the composed functions for downstream tasks are lightweight models.

Background

The paper emphasizes that despite substantial empirical progress in self-supervised representation learning, a unified theoretical foundation is missing. The authors explicitly state that the core question of what constitutes a sufficient and effective representation for diverse downstream tasks has not been resolved. They clarify the intended meanings of sufficiency and effectiveness: sufficiency refers to enabling downstream tasks via functions on the representation alone (without needing original data), and effectiveness refers to enabling lightweight downstream models rather than large deep architectures.

This problem motivates their spectral perspective and the unified framework they develop later in the paper, but the question itself is presented as an explicit open research challenge at the outset.