---
title: Deterministic Quadrature Fourier Features
url: https://www.emergentmind.com/topics/deterministic-quadrature-fourier-features-dqff
type: topic
---

# Deterministic Quadrature Fourier Features

Deterministic Quadrature Fourier Features (DQFF) are a class of explicit feature mappings for approximating shift-invariant kernels in machine learning and signal processing. DQFF replace the stochastic Monte Carlo sampling of Random Fourier Features (RFF) with deterministic, polynomially-exact quadrature rules in the frequency domain, yielding exponential improvements in approximation error, zero sampling variance, and positive-definiteness guarantees. DQFF have been developed and analyzed in the context of scalable kernel machines, adaptive filtering, Gaussian process regression, and kernel-based deep learning across a wide range of problem domains [1912.04530, 1709.02605, 2003.02658, 2408.13231, 2011.01668, 2310.14544, 2109.01545].

## 1. Theoretical Foundations: Fourier Representations and Quadrature

A shift-invariant, continuous, positive-definite kernel $k(x, z)$ on $\mathbb{R}^d$ admits a Fourier-domain representation by Bochner's theorem:
\[
k(x - z) = \int_{\mathbb{R}^d} p(\omega) e^{j\omega^\top(x-z)}\, d\omega,
\]
where $p(\omega)$ is the spectral density associated with $k$. For real-valued kernels (including the Gaussian RBF), this is equivalently
\[
k(x,z) = \int p(\omega) \cos[\omega^\top(x-z)]\, d\omega,
\]
with $p(\omega)$ often factorizing across coordinates.

RFF approximate this integral via Monte Carlo draws $\omega_i \sim p(\omega)$. DQFF, in contrast, construct a set of $D$ deterministic quadrature nodes $\{\omega_i\}$ and weights $\{w_i\}$ such that the approximation is polynomially exact up to a degree $R$, i.e.,
\[
\forall\, |r| \le R:\quad \int p(\omega) \omega^r\, d\omega = \sum_{i=1}^{D} w_i\, \omega_i^r,
\]
where $r$ is a multi-index. For the Gaussian RBF kernel, $p(\omega)$ is also Gaussian, so classical rules such as Gauss–Hermite or their sparse-grid analogues are applicable [1912.04530, 1709.02605, 2011.01668].

## 2. Deterministic Quadrature Feature Construction

Given quadrature nodes and weights, DQFF define explicit mappings:
\[
\Phi(x) = \left[
\sqrt{w_1}\cos(\omega_1^\top x),\, \sqrt{w_1}\sin(\omega_1^\top x),\, \ldots,\, \sqrt{w_D}\cos(\omega_D^\top x),\, \sqrt{w_D}\sin(\omega_D^\top x)
\right]^\top \in \mathbb{R}^{2D},
\]
such that the dot product $\Phi(x)^\top \Phi(z)$ deterministically approximates the kernel:
\[
\Phi(x)^\top \Phi(z) = \sum_{i=1}^D w_i \cos[\omega_i^\top(x-z)] \approx k(x,z).
\]
Complex-valued forms are equivalent but less standard in practical implementations [1912.04530, 1709.02605, 2003.02658].

For product kernels ($p(\omega)$ factorized), high-dimensional quadrature is constructed via tensor products of univariate rules or via sparse Smolyak grids. For isotropic kernels in high dimensions, recent approaches employ spherical–radial decompositions and quadrature rules tailored to the geometry of the measure [2408.13231].

## 3. Approximation Guarantees, Positive Definiteness, and Variance

The deterministic quadrature ensures that $\Phi(x)^\top \Phi(z)$ defines a positive-definite kernel, since each weighted trigonometric component is a positive-definite function and nonnegative weighted sums preserve this property [1912.04530]. DQFF induce a new finite-dimensional RKHS.

The approximation error is controlled by the degree of exactness $R$ and the feature count $D$. For sub-Gaussian kernels over a domain of diameter $M$, the uniform error is
\[
\sup_{\|x-z\|\le M} |k(x-z) - \Phi(x)^\top \Phi(z)| \le 3 \left(\frac{e b^2 M^2}{R}\right)^{R/2},
\]
where $b$ is the sub-Gaussian scale. To reach error $\epsilon$, DQFF require $D=O(\epsilon^{-1/\gamma})$ (for any $\gamma>0$), an exponential improvement over the $O(\epsilon^{-2})$ scaling of RFF [1912.04530, 1709.02605, 2003.02658].

DQFF incur zero sampling variance—unlike RFF, whose empirical kernel approximations fluctuate and only converge in expectation at a slow $O(1/D)$ or $O(1/\sqrt{D})$ rate [1912.04530, 1709.02605].

## 4. Computational Complexity and Practical Implementation

DQFF decouple into an offline precomputation phase and an online evaluation phase. Precomputing multidimensional quadrature nodes and weights typically costs $O(D)$ for sparse-grid or $O(D^3)$ for least-squares weight fitting; the same nodes/weights are reused for all data [1912.04530, 1709.02605, 2011.01668]. Feature evaluation for a single input vector is $O(Dd)$: each $\omega_i^\top x$ (inner product), $D$ trigonometric evaluations, and $D$ scalar multiplications.

Random Fourier features cost $O(Dd)$ per evaluation but additionally incur runtime random sampling or the need to cache randomly drawn $\omega_i$.

For tensor-product quadrature in $d$ dimensions, the feature count grows as $m^d$ (with $m$ points per dimension). Methods such as sparse grids, spherical–radial factorization, or low-rank tensor decomposition alleviate this curse in moderate to high dimensions [2408.13231, 2109.01545]. For ANOVA or locally supported kernels, DQFF exploit structured quadratures on the relevant subsets [1709.02605].

## 5. Empirical Performance across Application Domains

DQFF have been systematically evaluated in adaptive filtering, high-dimensional supervised learning, and Gaussian Process (GP) settings:

- In kernel adaptive filtering (e.g., NT-KLMS/NT-KRLS), DQFF match or outperform RFF and dictionary-based kernel methods in MSE convergence and computational efficiency, with fixed per-step costs and robust performance under noise [1912.04530].
- For GP regression and classification, DQFF achieve exponentially decaying kernel and posterior approximation error, outpace RFF in matching full-GP performance using far fewer features, and scale linearly in the number of data points [2003.02658, 2310.14544].
- In large-scale learning (e.g., MNIST, TIMIT), DQFF accelerate feature construction and kernel evaluation, attain lower RMS error, and equal or exceed the classification and regression accuracy of competing state-of-the-art kernel approximations [1709.02605, 2011.01668].
- Recent advances using spherical–radial DQFF achieve superior kernel approximation in high dimensions by carefully balancing radial and spherical quadrature components [2408.13231].

Selected comparative statistics:

| Method           | D (features) | Kernel RMS Error | Feature Gen. Time |
|------------------|-------------|------------------|-------------------|
| Random RF        | 5,000       | 0.025 ± 0.003    | 1.0×              |
| Subsampled DQFF  | 5,000       | 0.024 ± 0.002    | 0.50×             |
| Reweighted DQFF  | 5,000       | 0.008 ± 0.001    | 0.55×             |

[1709.02605]

## 6. Advances in High-Dimensional and Structured Kernels

DQFF methodology extends to scenarios where direct tensor-product quadrature would be intractable:

- Spherical–radial DQFF [2408.13231] utilize separable quadrature on the radial (Gauss–Laguerre) and spherical (uniform or optimal quadrature) components, exploiting the isotropy of the Gaussian measure. This dramatically reduces feature counts in high $d$, allowing $M=O(d)$ features for suitable precision.
- For sparse ANOVA kernels, DQFF build low-dimensional quadratures on subkernels and combine the resulting features, yielding favorable scaling in $d$ and improved sample complexity $O(\epsilon^{-1/\gamma})$ [1709.02605].
- Tensor decomposition techniques learn weight tensors as low-rank CP representations, reducing storage and computation from $O(m^d)$ to $O(dmR)$ for rank-$R$ models, and achieving competitive empirical error and generalization [2109.01545].

## 7. Hyperparameter Optimization and Practical Trade-Offs

Key hyperparameters include kernel bandwidth $\sigma$, polynomial degree $R$, and number of features $D$. Sparse-grid or random-subsampled quadrature can be used to fit budgeted $D$. Choice of quadrature level $L$ is determined by the dimensionality and precision target (e.g., $L \sim \log D / d$). Feature count should be increased until the kernel approximation error on a small validation set plateaus [1912.04530, 1709.02605].

For data with unbounded support, input normalization to $\|x\|\leq1$ is crucial to preserve sub-Gaussianity and prevent error inflation. In applications requiring variable feature dimension (e.g., stochastic online learning), hybrid rules combining deterministic quadrature and stochastic sampling have been proposed, offering unbiasedness and variance reduction over plain RFF [2011.01668].

---

**References**:  
[1912.04530], [1709.02605], [2003.02658], [2011.01668], [2408.13231], [2310.14544], [2109.01545]

Source: https://www.emergentmind.com/topics/deterministic-quadrature-fourier-features-dqff