On the importance of single directions for generalization (1803.06959v4)

Abstract: Despite their ability to memorize large datasets, deep neural networks often achieve good generalization performance. However, the differences between the learned solutions of networks which generalize and those which do not remain unclear. Additionally, the tuning properties of single directions (defined as the activation of a single unit or some linear combination of units in response to some input) have been highlighted, but their importance has not been evaluated. Here, we connect these lines of inquiry to demonstrate that a network's reliance on single directions is a good predictor of its generalization performance, across networks trained on datasets with different fractions of corrupted labels, across ensembles of networks trained on datasets with unmodified labels, across different hyperparameters, and over the course of training. While dropout only regularizes this quantity up to a point, batch normalization implicitly discourages single direction reliance, in part by decreasing the class selectivity of individual units. Finally, we find that class selectivity is a poor predictor of task importance, suggesting not only that networks which generalize well minimize their dependence on individual units by reducing their selectivity, but also that individually selective units may not be necessary for strong network performance.

• The paper shows that networks relying on single directions exhibit poorer generalization, with performance dropping sharply when these paths are ablated.
• The paper demonstrates through experiments that batch normalization reduces single direction reliance more effectively than dropout, enhancing network robustness.
• The paper suggests that tracking single direction reliance can serve as a practical proxy for early stopping and model selection in training deep networks.

On the Importance of Single Directions for Generalization: An Analysis

The paper "On the Importance of Single Directions for Generalization" investigates an important aspect of deep neural networks (DNNs): their reliance on single directions, which are defined as the activation of a single unit or a combination of units, and how this reliance impacts generalization performance. This analysis is conducted through extensive experiments involving different neural network architectures and datasets, providing both empirical insights and potential implications for network design and training.

Summary and Key Findings

The authors present a methodical investigation into the relationship between a neural network's propensity to rely on single unit activations and its generalization ability. A salient observation from their work is that networks which memorize their training data tend to depend significantly more on single directions than those that achieve better generalization. This relationship holds true across networks with various training labels, including corrupted labels, and those trained with unvaried datasets and hyperparameters.

• Reliance and Generalization: Networks trained with corrupted labels were observed to be more sensitive to the ablation of single directions, as removing these directions leads to rapid degradation of the network's performance. Similarly, networks with poor generalization performance, when trained on unmodified datasets, displayed increased reliance on single directions compared to networks with robust generalization capabilities.
• Use for Early Stopping and Model Selection: The paper finds that monitoring single direction reliance can potentially serve as a proxy for model selection and early stopping criteria. This insight is particularly useful for scenarios where limited data makes conventional validation less effective.
• Effect of Regularizers: The paper distinguishes between the impacts of dropout and batch normalization on single direction reliance. While dropout does not prevent over-reliance on single directions beyond the dropout fraction used in training, batch normalization effectively decreases reliance and enhances robustness by potentially reducing individual class selectivity.
• Selectivity and Importance: Interestingly, the findings indicate that individual class selectivity of units is a poor predictor of their overall importance to the network's output. Even though selective units may respond strongly to particular features, they do not necessarily contribute significantly to task performance when removed. This raises questions about commonly used interpretability methods that focus on single unit activations.
• Theoretical Implications: The results intimate that more generalized solutions in DNNs are associated with reduced reliance on low-dimensional subspaces. This might be aligned with theoretical perspectives that favor broader, flatter minima in optimization landscapes.

Implications for Future Research

The insights presented in this paper suggest multiple avenues for future work. Given that batch normalization appears to discourage reliance on single directions and improve robustness, further research could develop new forms of regularization explicitly targeting single direction reliance.

Additionally, the weak correlation between class selectivity and individual unit importance prompts a reevaluation of methodologies focused on understanding DNNs solely through the lens of single-unit activations. Exploring richer representations and distributed coding strategies might yield further dividends in model performance and interpretability.

The paper also ponders the potential for reliance metrics to estimate generalization without a held-out validation set, which could substantially optimize hyperparameter tuning and model selection processes across more complex datasets.

Conclusion

This paper's methodical approach to understanding the reliance of neural networks on single directions enriches our knowledge of how and why certain networks generalize better than others. By addressing simultaneous practical concerns—like efficient model selection—and theoretical questions—regarding network robustness and selectivity—this research offers substantial contributions to the ongoing development of DNN techniques and understanding. As neural architectures continue to evolve, these findings could greatly influence how new models are trained and interpreted.