Log-Weighted Barron Space in Neural Approximation

• Log-weighted Barron space is a function space defined via a logarithmic Fourier weight that allows efficient approximation of functions with weak smoothness.
• It bridges classical Barron and Sobolev spaces, highlighting how deeper networks can counteract limited spectral regularity with precise embedding and capacity results.
• Deep narrow ReLU networks can approximate functions in this space with O(m⁻¹/²) error rates, emphasizing a critical depth-regularity tradeoff in high-dimensional settings.

The log-weighted Barron space, denoted $B^{\log}$, is a function space defined via a logarithmic weight in the Fourier domain. It arises as a limiting case of classical Barron spaces $B^{s}$ as $s \rightarrow 0$, and plays a central role in analyzing the depth-regularity tradeoff for approximation by deep ReLU neural networks. This space requires strictly weaker spectral regularity than any $B^s$ with $s>0$, enabling approximation of functions with less smoothness using sufficiently deep, but narrow, networks. The associated family $B^{s,\log}$ extends this concept to higher-order regularity, parameterized by $s > 0$ and with logarithmic spectral weights. The framework clarifies how depth substantially widens the class of functions that can be efficiently approximated, in contrast to classical width-oriented theories, and provides new theoretical foundations for the practical performance of deep architectures in high-dimensional settings (Song et al., 3 Jan 2026).

1. Definition and Norms of Log-Weighted Barron Spaces

Classical Barron spaces $B^s$ for $s \geq 0$ are defined by the norm

$$\|f\|_{B^s} = \int_{\mathbb{R}^d}(1 + |\xi|_1^s)\, |\widehat{f}(\xi)|\, d\xi,$$

with $$B^s = \{f \in S'(\mathbb{R}^d) : \|f\|_{B^s} < \infty\}$$.

The log-weighted Barron space $B^{\log}$ adopts a logarithmic weight:

$$\|f\|_{B^{\log}} = \int_{\mathbb{R}^d} \log_2(2 + |\xi|_1)\, |\widehat{f}(\xi)|\, d\xi,$$

$$B^{\log} = \{f \in S'(\mathbb{R}^d) : \|f\|_{B^{\log}} < \infty\}.$$

This is a Banach space due to the completeness of weighted $L^1$. For any $s>0$, $B^s \subset B^{\log}$.

The higher-order log-Barron space, $B^{s,\log}$ for $s>0$, is defined by

$$\|f\|_{B^{s,\log}} = \int_{\mathbb{R}^d}(1 + |\xi|_1^s)\log_2(2+|\xi|_1)\,|\widehat{f}(\xi)|\,d\xi,$$

$B^{s,\log} = \{ f: \|f\|_{B^{s,\log}} < \infty \}.$

2. Embedding Relations with Sobolev and Barron Spaces

A central feature is the precise position of $B^{\log}$ between Sobolev and Barron spaces. For $H^s(\mathbb{R}^d)$ the standard $L^2$-Sobolev space, the following holds [Theorem 4.1, (Song et al., 3 Jan 2026)]:

• If $s > d/2$, $$H^s(\mathbb{R}^d) \hookrightarrow B^{\log}(\mathbb{R}^d)$$.
• This embedding is sharp; for $0 \leq s \leq d/2$ there exists $f \in H^s$ with $\|f\|_{B^{\log}} = \infty$.
• Conversely, for any $s \geq 0$, there exists $f \in B^{\log}$ with $f \notin H^s$.

These results demonstrate that $B^{\log}$ is strictly larger than any classical Barron space $B^s$ with $s>0$, and neither contains nor is contained in any particular Sobolev space $H^s$ for $s \leq d/2$.

3. Rademacher Complexity and Statistical Capacity

The Rademacher complexity of the unit ball in $B^{\log}$ provides a sharp estimate of its statistical richness. For the set $F_Q = \{f \in B^{\log} : \|f\|_{B^{\log}} \leq Q\}$ and sample points $x_1,\ldots,x_n \in \Omega \subset [-1,1]^d$, the complexity satisfies [Theorem 4.5, (Song et al., 3 Jan 2026)]:

$\mathfrak{R}_n(F_Q) \leq C Q \sqrt{d/n}$

where $C$ is an absolute constant. This bound is derived using a weighted Fourier representation, dyadic frequency shells, and Dudley's entropy integral, thereby obtaining dimension-dependent, sample-size-sensitive guarantees on the richness of $B^{\log}$ for learning-theoretic analysis.

4. Deep ReLU Approximation Theorems

For $f \in B^{\log}$ and a compact $\Omega \subset \mathbb{R}^d$, the main theorem establishes that deep narrow ReLU networks approximate $f$ efficiently with explicit depth dependence [Theorem 5.1, (Song et al., 3 Jan 2026)]:

• For any $m \in \mathbb{N}$, there exist subnetworks $F_i$ of width 3 and depths $L_i$ such that

$$\left\| f - \frac{1}{m} \sum_{i=1}^m F_i \right\|_{L^2(\Omega)}^2 \leq \frac{3\pi^4}{m}|\Omega|\|f\|_{B^0}^2,$$

$$\sum_{i=1}^m L_i \leq 5m\, \frac{\|f\|_{B^{\log}}}{\|f\|_{B^0}}.$$

Merging the subnetworks yields a single network $F$ of width $d+4$ and depth $L = O\left(m\,\|f\|_{B^{\log}}/\|f\|_{B^0}\right)$ with approximation error

$$\|f - F\|_{L^2(\Omega)} \leq 2\pi^2|\Omega|^{1/2}\|f\|_{B^0}/\sqrt{m}.$$

The construction leverages an exact Fourier-based representation of $f$, decomposing it into cosine components with frequencies adapted to the logarithmic weight, and Monte Carlo sampling to construct subnetworks whose depths are governed by the log-integral of the spectral magnitude.

5. $H^1$ and Higher-Order Approximation in $B^{s,\log}$

For $f \in B^{1,\log}$, analogous results hold for $H^1$-approximation [Theorem 6.1, (Song et al., 3 Jan 2026)]:

• For any $m \in \mathbb{N}$, subnetworks $F_i$ of width 3 and appropriately bounded depths can be constructed with

$$\| f - (1/m)\sum_i F_i \|_{H^1(\Omega)}^2 \leq \frac{11\pi^4}{m}|\Omega|\|f\|_{B^1}^2,$$

$$\sum L_i \leq 5m\, \frac{\|f\|_{B^{1,\log}}}{\|f\|_{B^1}}.$$

After merging, a network of width $d+4$ and depth $O(m\,\|f\|_{B^{1,\log}}/\|f\|_{B^1})$ achieves

$$\|f - F\|_{H^1(\Omega)} \leq 4\pi^2|\Omega|^{1/2}\|f\|_{B^1}/\sqrt{m}.$$

The proofs exploit the structure of the Fourier representation to ensure both the function and its gradients are approximated in mean-square, reflecting the capacity of deep architectures to capture weakly regular, high-frequency structure.

6. Depth-Regularity Tradeoff and Theoretical Implications

The log-weighted Barron spaces precisely characterize how depth can substitute for classical smoothness in neural network approximation. Classical Barron spaces with $s>0$ require polynomial spectral decay and grant $O(n^{-1/2})$ error rates for width-$n$ two-layer networks. In contrast, $B^{\log}$ requires only log-integrability and admits $O(m^{-1/2})$ rates via depth-$m$ networks of fixed width $(d+4)$. Empirical spectral measurements (see Figure 1 in (Song et al., 3 Jan 2026)) indicate that many practical target functions exhibit slowly decaying Fourier amplitudes; deep ReLU networks equipped with sufficient depth can harness this structure for efficient approximation. This result provides a rigorous foundation for the observed superior expressivity of deep over shallow architectures in high-dimensional regimes, despite weak regularity of the target function.

7. Open Questions and Future Directions

Several open questions pertain to the boundaries and potential extensions of the log-weighted Barron framework (Song et al., 3 Jan 2026):

• Minimal width: Can the current network width of $d+4$ be further reduced, potentially via feature packing or other architectural innovations?
• Stronger norms: Do similar depth-sensitive approximation rates extend to stricter norms such as $W^{k,2}$ or $L^\infty$, possibly by suitable choices of ReLU-based representations?
• Faster rates: For smoother functions in $B^{s}$ or $B^{s,\log}$, is it possible to surpass the $m^{-1/2}$ rate by adapting depth and width, thereby aligning approximation more closely with function smoothness?
• Tightness of depth scales: For a given $\|f\|_{B^{\log}}$, what is the necessary minimal depth for achieving a prescribed approximation error? Are current depth bounds optimal up to constants?

The introduction and rigorous analysis of $B^{\log}$ and $B^{s,\log}$ offer a powerful, explicit vehicle for understanding the interplay between neural network architecture and function space regularity, particularly highlighting the unique role of depth in enabling efficient approximation of high-frequency, high-dimensional, and weakly regular targets (Song et al., 3 Jan 2026).