Overfitted Brain Hypothesis (OBH)

• OBH is a theory that defines how neural systems counteract overfitting by incorporating noise, particularly through dream-like spontaneous activity.
• It draws an analogy with deep learning techniques like dropout and noise injection to regularize over-specialized representations.
• The hypothesis integrates neuroscience, machine learning, and cognitive science to explain the balance between memorization and generalization in both biological and artificial networks.

The Overfitted Brain Hypothesis (OBH) postulates that the brain’s structural and functional organization is driven not only by constraints on memory and learning but also by evolutionary pressure to maximize generalization in the face of highly repetitive, biased sensory input. OBH draws a functional analogy between biological neural circuits and artificial deep learning systems, suggesting that both are vulnerable to overfitting—the phenomenon where an adaptive system excels at recall for familiar data but fails to generalize to novel situations. Building upon contemporary machine learning theory and new neuroscientific findings, OBH contends that biological phenomena such as dreams and spontaneous activity evolved as intrinsic mechanisms to regularize the brain’s internal models, thereby enhancing generalization. The hypothesis is situated at the intersection of neuroscience, criticality theory, and statistical learning, with implications for artificial intelligence, computational neuroscience, and cognitive science.

1. Theoretical Foundations of OBH

OBH is formulated by analogy to overfitting in deep neural networks (DNNs), where a model trained on limited or non-diverse data exhibits poor performance on “test” data that depart from the training distribution (Hoel, 2020). In the biological context, the problem arises because real-world experiences—such as those encountered during daily learning—are often highly correlated, self-similar, and subject to environmental regularities. Without a mechanism for injecting variability, neural circuits may “memorize” local regularities and thus lose the capacity to generalize.

To address this, OBH asserts that the brain “injects noise” into its own learning process, most prominently in the form of dreams during sleep. Dreams, characterized by sparse, hallucinatory, and topographically abnormal sensory experiences, function analogously to stochastic regularization methods used in DNNs (dropout, noise injection, domain randomization). Formally, if the waking sensory input is $x$, then the input during sleep may be $x̃ = x + \epsilon$, with $\epsilon$ drawn from a distribution intended to corrupt or decouple the input in a way that disrupts over-specialized neural patterns.

2. Criticality, Cascades, and the Role of Spontaneous Activity

The OBH interacts fundamentally with theories of criticality in neural systems (Korchinski et al., 2019). Classical models of neural avalanches and propagation (e.g., directed percolation, branching processes) rely on clear separation between spontaneous (exogenous) activation and the propagation of activity through the network (endogenous cascades). In such models, the system’s approach to criticality is measured by quantities like the branching ratio ($\sigma$) and dynamic susceptibility ($\chi_0$), which are expected to show universal properties at the transition.

However, in real neural networks—with ongoing spontaneous activity—avalanches overlap and merge, abolishing clean timescale separation. The critical line, expressed in the mean-field regime as:

$0 = k\,(1-\sigma)^2 - (k-1)\,\sigma\,\sigma_m,$

where $\sigma_m$ encodes the merging of independent spontaneous cascades, defines a continuum between classic directed percolation (isolated avalanches, $p \to 0$) and isotropic undirected percolation (high merger rates, finite $p$). The low-$p$ scaling is:

$$\left(\frac{1}{k} - q_c\right)^3 \approx \frac{(k-1)^2 (2k-1)}{k^5}\,p_c,$$

where $q_c$ is the critical propagation probability and $p_c$ is the critical spontaneous activation probability. This duality induces a mixed scaling regime in the statistics of neural avalanche sizes, with exponents from both directed and undirected percolation. In the OBH context, continuous drive means experimental observables reflect mixed universality classes, complicating the identification of true criticality.

3. Dreaming as Biological Regularization

OBH proposes that nightly dreaming evolved specifically to counteract overfitting in neural representations (Hoel, 2020). During sleep, endogenous neural dynamics simulate “out-of-distribution” inputs, decoupled from the statistics of waking experience. These dream events, generated via top–down model-driven reactivations, result in the formation of aberrant, corrupted, and sparse sensory scenes, which serve to “anneal” high-confidence synaptic patterns and disrupt persistent, over-memorized representations.

Mechanistically, this is analogous to injecting $\epsilon$-style noise or using techniques like dropout in DNN training:

• Dropout: random “dropping” of neural units to prevent reliance on specific pathways;
• Domain randomization: exposure to data whose distribution is artificially varied to cover more of the input space;
• Simulated annealing: stochastic processes that escape local minima.

Empirical evidence supports the view that sleep, and specifically dreaming, enhances performance on generalization-heavy tasks and prevents rigidity, with sleep deprivation leading to dissociation between memorization and generalization. Tasks such as Tetris and mirror tracing, when over-trained during waking, induce dreams that incorporate “corrupted” variants, suggesting targeted regularization of overfitted circuits.

4. Generalization in Overparameterized Networks: Theoretical and Empirical Insights

Recent work in statistical learning theory provides foundational support for OBH by demonstrating that heavily overparameterized neural networks can generalize well when certain inductive biases are present. The Linear Frequency Principle (LFP) model (Zhang et al., 2021) shows that the gradient dynamics of neural network training inherently prioritize low-frequency components of the target function. Specifically, the evolution equation in the Fourier domain is:

$$\partial_{t} \hat{h}(\xi,t) = -\gamma(\xi) [\hat{h}_\rho(\xi,t) - \hat{f}^*_\rho(\xi)],$$

where $\gamma(\xi)$ decays with frequency, ensuring a preference for smooth solutions unless the data itself is high-frequency dominant. This low-frequency bias means that minimum-norm solutions selected by optimization (e.g., gradient descent) prefer smooth interpolants, thus preventing harmful overfitting even in the absence of explicit regularization.

Extending this principle, the NTK regime in shallow wide networks (Ju et al., 2021) shows that the minimum-$\ell_2$ norm interpolating solution can achieve low generalization error provided the ground-truth function resides within the network’s “learnable” function class. Contrasting the “double descent” behavior seen in simple linear models, NTK solutions do not suffer catastrophic risk escalation as the parameter count increases.

These results suggest that both biological and artificial networks may evade the worst consequences of overfitting through intrinsic inductive biases and self-organized regularization.

5. Mechanisms and Mitigation of Over-Memorization

OBH motivates investigation into the mechanisms by which both biological networks and DNNs avoid or exacerbate over-memorization. In deep networks, over-memorization manifests as sudden, persistent high-confidence predictions for specific training patterns (original and transformed), with these patterns retaining low loss even after removal from subsequent training (Lin et al., 2023). This phenomenon is detrimental, reducing generalization capacity and increasing vulnerability to adversarial inputs.

To counteract over-memorization, the Distraction Over-Memorization (DOM) framework introduces two strategies:

• DOM_RE: high-confidence examples (loss below threshold $\mathcal{T}$) are removed post-warm-up, disrupting reinforcement cycles.
• DOM_DA: high-confidence examples are iteratively augmented, their style/content perturbed until the network’s confidence drops below $\mathcal{T}$.

Pseudocode for DOM_RE:

$$\theta = \theta - \nabla_\theta (\ell_{NT}(x,y;\theta)\quad \text{if}\quad \ell_{NT} > \mathcal{T} )$$

Empirical validation across CIFAR-10, SVHN, Tiny-ImageNet, and multiple architectures demonstrates improved test accuracy and reduced generalization gap when DOM methods are used.

6. Reinforcement Learning and Synthetic Dreams

The OBH has recently been adapted to reinforcement learning agents by incorporating generative “dream-like” experiences during training (Franceschelli et al., 12 Mar 2024). Here, agents learn a world model from limited real experience ("day experience"), then perform "night training" using synthetic episodes generated via deliberate latent-state perturbations. Three main augmentations emulate dream-like stochasticity:

• Random Swing: discrete latent states are randomly re-ordered and perturbed, responding to binomial sampling and uniform shifts.
• DeepDream: latent states are adjusted via gradient ascent to amplify encoder activations, producing “hallucinatory” images.
• Value Diversification: latent states are modified to maximize prediction differences in value estimates, simulating surprise/reward hallucination.

These methods increase the diversity of experience, preventing overfitting, and yield higher generalization scores in sparse-reward scenarios compared to classic imagination-based RL.

7. Internal Bias and Overthinking in Reasoning Models

OBH extends conceptually to the domain of reasoning, where internal bias—an early, direct answer—leads to redundant reflection and “overthinking” in LLMs (Dang et al., 22 May 2025). Attention analyses reveal that excessive focus on the original input (carrying the internal bias) triggers repetitive and unnecessary verification cycles. Masking the input after the initial answer reduces reasoning length by 31–53% and increases accuracy, providing a tangible means of mitigating computational waste and improving performance. This mirrors OBH's central claim: mechanisms that disrupt rigid priors (whether sensory, internal, or representational) are critical for preserving generalization and robustness.

8. Implications, Limitations, and Perspectives

OBH synthesizes insights across neuroscience, machine learning, and cognitive theory, offering a unified explanation for biological resilience against overfitting. Key implications include:

• Redefinition of criticality in neural systems: signature measures (branching ratio, susceptibility) may be confounded by continuous spontaneous activity, complicating experimental tests of optimal computation (Korchinski et al., 2019).
• Dream-like regularization: both biological and synthetic systems benefit from stochastic, hallucinatory episodes to evade over-memorization and enhance generalization (Hoel, 2020, Franceschelli et al., 12 Mar 2024).
• Inductive bias and architecture: low-frequency bias, minimum-norm solutions, and causal structure should be considered when evaluating the generalization power of overparameterized systems (Zhang et al., 2021, Ju et al., 2021).
• Mitigation strategies: frameworks such as DOM and masking of internal biases offer practical tools for disrupting detrimental over-memorization and overthinking (Lin et al., 2023, Dang et al., 22 May 2025).

A plausible implication is that OBH's explanatory scope motivates the design of AI systems—and experimental neuroscience paradigms—that reconceptualize overfitting as not merely a bug to be suppressed but a property to be actively managed and harnessed via evolutionary and algorithmic regularization strategies. Future research may focus on quantifying the impact of dream-inspired methods, mapping the boundaries of “learnable” function classes in biological networks, and refining observables for neural criticality that account for continuous drive and cascade merging.