A data-driven Fourier-mixture neural-network method for density estimation

Abstract: We propose a data-driven Fourier-trained neural-network method for estimating fixed-horizon probability densities from empirical characteristic-function (CF) information. The estimator is a positive Gaussian--Laplace mixture with closed-form CF, so training can be performed directly in Fourier space while preserving nonnegativity and unit mass. We consider two sampling settings. In the direct i.i.d. sampling setting, the method is trained against an empirical CF constructed from i.i.d. samples. In the resampling-based pseudo-sampling setting, it is trained against an empirical pseudo-CF constructed from dependent data by resampling. For the direct i.i.d. case, we derive an expected $L_2$ error bound that separates Fourier truncation, empirical training error, discretization, and CF sampling error. For the pseudo-sampling case, we obtain a conditional analogue with two additional pseudo-law discrepancy terms. We develop a multidimensional extension of the framework and analyze its computational complexity. Numerical experiments show competitive performance relative to Expectation--Maximization on Gaussian-mixture benchmarks, clear gains on heavy-tailed targets, $L_2$ error decay consistent with the theory in a well-specified setting, and effective estimation of one-year Australian equity return law from resampled dependent data.

• The paper introduces a novel Fourier-mixture neural network that estimates probability density functions from empirical characteristic functions while ensuring positivity and normalization.
• The method integrates a Gaussian-Laplace mixture model with a closed-form characteristic function, yielding theoretical L² error bounds via Plancherel’s theorem.
• The approach demonstrates competitive computational efficiency and robust performance in multivariate, heavy-tailed, and dependent data scenarios.

Data-Driven Fourier-Mixture Neural Network Method for Density Estimation

Methodology and Structural Features

The paper presents a neural-network-based probabilistic density estimator trained in the Fourier domain, targeting settings where empirical characteristic function (CF) information is available, but direct access to the underlying density is not. The estimator is constructed as a positive Gaussian-Laplace mixture parameterized via a bounded single-hidden-layer feedforward neural network (FFNN), ensuring nonnegativity and normalization throughout training, and crucially, retaining a closed-form characteristic function.

This structure facilitates direct optimization against empirical CF data—for both i.i.d. and dependent data scenarios—and allows theoretical analysis via Plancherel’s theorem, connecting Fourier-space error to physical-space density error. Closure of the error decomposition is achieved, separating statistical and algorithmic contributions: Fourier truncation, empirical CF error, discretization, and sampling error. The model’s nonnegativity and normalization make it particularly well-suited for applications necessitating proper probability density estimates.

The approach is extended to the multivariate case, with covariance matrices parameterized via Cholesky factors and the corresponding mixture structure generalized. Computational complexity is analyzed and shown to be competitive, especially as the iterative training decouples from sample size once the empirical CF is computed.

Error Analysis and Statistical Guarantees

The paper rigorously derives expected $L^2$ error bounds for density recovery. Leveraging Plancherel's theorem, the expected physical-space density error is recast as the integrated Fourier-domain error. For i.i.d. sampling, the bound comprises:

• Fourier truncation error: Controlled via interval selection.
• Empirical training error: Explicitly linked to the Fourier-domain loss function.
• Discretization error: Quantified for finite Fourier grids.
• CF sampling error: Originating from the variance of the empirical CF, scaling with sample size as expected.

For dependent observations, a resampling-based framework is used, with pseudo-samples generated from the observed history. The key innovation is that the error analysis structurally carries over, with two new pseudo-law discrepancy terms quantifying the gap between resampling and the true fixed-horizon law.

The theoretical guarantees illuminate the interplay between model capacity, sample size, and grid resolution. In well-specified regime, density $L^2$ error decays with the expected $M^{-1/2}$ rate, and discretization error saturates once the grid is sufficiently refined.

Numerical Experiments and Empirical Performance

The empirical evaluation encompasses both synthetic and real-data settings.

Gaussian-Mixture Benchmarks: The Fourier-mixture method, restricted to the Gaussian subclass, achieves accuracy comparable to EM on well-separated and overlapping mixtures, with optimal performance when the learned model matches the ground-truth component count.

Figure 1: Well-separated components; optimal density estimation with minimized $L^2$ error and per-sample negative log-likelihood occurs at matching component count.

Heavy-Tailed and Non-Gaussian Targets: Adding Laplace components substantially improves Fourier-domain fit (order-of-magnitude reduction in CF error) for heavy tails (Cauchy), while for the Kou jump-diffusion model, a sufficiently large Gaussian mixture suffices.

Multivariate Extension: In a 2D Cauchy benchmark, the Gaussian-Laplace mixture outperforms Gaussian-only models in both real and imaginary CF error metrics.

Error Decay Validation: Empirical scaling matches theoretical predictions: as sample size increases, $L^2$ error decays at rate close to $M^{-1/2}$, and grid refinement yields diminishing returns once the discretization error becomes subdominant.

Figure 2: M-dependence with $P=4000$; log-log regression demonstrates $L^2$ error decay consistent with $M^{-1/2}$ scaling for well-specified mixtures.

Pseudo-Sampling from Dependent Data: Using stationary bootstrap procedures, the framework is validated on Australian equity return data, targeting one-year horizon returns constructed from resampled monthly series. The Fourier-mixture achieves superior fit to the empirical pseudo-CF compared to EM-GMM, particularly in the Fourier space, and retains robust tail calibration.

Figure 3: Validation NLL and BIC; marginal gain diagnostic identifies the optimal EM-GMM component count in resampled Australian equity data.

Computational Complexity and Practical Implications

A salient feature is the decoupling of iterative training complexity from sample size after empirical CF preprocessing. In contrast to EM, which scales linearly with sample size at every iteration, the Fourier-mixture method’s iterative computational cost is dependent only on the Fourier grid and model capacity. This presents practical advantages in high-data or pseudo-sample scenarios.

The multivariate extension scales favorably with respect to dimension and grid resolution, and complexity remains manageable for moderate $d$ when adopting Cholesky parametrizations.

Implications and Future Directions

The methodology yields positive, normalized densities from empirical Fourier information—bridging characteristic-function-based estimation and physical-space modeling via neural networks, and providing rigorous theoretical guarantees and competitive empirical performance.

• Practical Impact: The Fourier-mixture estimator is applicable in simulation, uncertainty quantification, and fixed-horizon law estimation from dependent time-series, relevant to stochastic control and financial modeling.
• Theoretical Developments: Further analysis of pseudo-law discrepancy convergence under various resampling schemes and expansion to more general dependent-data scenarios is called for.
• Model Extensions: The Gaussian-Laplace mixture could be generalized to incorporate further flexibility for even heavier tails or non-elliptical features.

Conclusion

The paper advances a data-driven, Fourier-trained neural-network framework for density estimation, combining structural soundness, theoretical tractability, and empirical efficacy. The approach addresses both i.i.d. and dependent data settings, preserving positivity and normalization, and decouples iterative complexity from sample size. Competitive results are achieved across mixture, heavy-tailed, and resampled real-world settings. The developed methodology is poised for further extension and application in high-dimensional, stochastic, and sequential decision contexts.