2-Cats: 2D Copula Approximating Transforms (2309.16391v5)
Abstract: Copulas are powerful statistical tools for capturing dependencies across data dimensions. Applying Copulas involves estimating independent marginals, a straightforward task, followed by the much more challenging task of determining a single copulating function, $C$, that links these marginals. For bivariate data, a copula takes the form of a two-increasing function $C: (u,v)\in \mathbb{I}2 \rightarrow \mathbb{I}$, where $\mathbb{I} = [0, 1]$. This paper proposes 2-Cats, a Neural Network (NN) model that learns two-dimensional Copulas without relying on specific Copula families (e.g., Archimedean). Furthermore, via both theoretical properties of the model and a Lagrangian training approach, we show that 2-Cats meets the desiderata of Copula properties. Moreover, inspired by the literature on Physics-Informed Neural Networks and Sobolev Training, we further extend our training strategy to learn not only the output of a Copula but also its derivatives. Our proposed method exhibits superior performance compared to the state-of-the-art across various datasets while respecting (provably for most and approximately for a single other) properties of C.
- Pair-copula constructions of multiple dependence. Insurance: Mathematics and economics, 44(2), 2009.
- A new mixture copula model for spatially correlated multiple variables with an environmental application. Scientific Reports, 12(1), 2022.
- Statistics with confidence: confidence intervals and statistical guidelines. John Wiley & Sons, 2013.
- Barry C Arnold. Multivariate logistic distributions. Marcel Dekker New York, 1992.
- Yves I Ngounou Bakam and Denys Pommeret. Nonparametric estimation of copulas and copula densities by orthogonal projections. Econometrics and Statistics, 2023.
- Christopher M Bishop. Mixture density networks. Online Report., 1994.
- Yongqiang Cai. Achieve the minimum width of neural networks for universal approximation. In ICLR, 2023.
- Statistical inference. Cengage Learning, 2001.
- Copula methods in finance. John Wiley & Sons, 2004.
- Neural likelihoods via cumulative distribution functions. In UAI, 2020.
- George Cybenko. Approximation by superpositions of a sigmoidal function. Mathematics of control, signals, and systems, 2(4), 1989.
- Claudia Czado. Pair-copula constructions of multivariate copulas. In Copula Theory and Its Applications. Springer, 2010.
- Vine copula based modeling. Annual Review of Statistics and Its Application, 9, 2022.
- Sobolev training for neural networks. In NeurIPS, 2017.
- Monotone and partially monotone neural networks. IEEE Transactions on Neural Networks, 21(6), 2010.
- Pierre Dragicevic. Fair statistical communication in hci. Modern statistical methods for HCI, 2016.
- Confidence intervals rather than p values: estimation rather than hypothesis testing. Br Med J (Clin Res Ed), 292(6522), 1986.
- Probit transformation for nonparametric kernel estimation of the copula density. Bernoulli, 2017.
- Everything you always wanted to know about copula modeling but were afraid to ask. Journal of hydrologic engineering, 12(4), 2007.
- A semiparametric estimation procedure of dependence parameters in multivariate families of distributions. Biometrika, 82(3), 1995.
- Statistical tests, p values, confidence intervals, and power: a guide to misinterpretations. European journal of epidemiology, 31, 2016.
- Copulae: An overview and recent developments. Wiley Interdisciplinary Reviews: Computational Statistics, 14(3), 2022.
- Copula-like variational inference. In NeurIPS, 2019.
- Moving beyond p values: data analysis with estimation graphics. Nature methods, 16(7), 2019.
- Multilayer feedforward networks are universal approximators. Neural networks, 2(5), 1989.
- Guido W Imbens. Statistical significance, p-values, and the reporting of uncertainty. Journal of Economic Perspectives, 35(3), 2021.
- Implicit generative copulas. In NeurIPS, 2021.
- Physics-informed machine learning. Nature Reviews Physics, 3(6), 2021.
- Normalizing flows: An introduction and review of current methods. IEEE transactions on pattern analysis and machine intelligence, 43(11), 2020.
- Ryan Kortvelesy. Fixed integral neural networks. arXiv preprint arXiv:2307.14439, 2023.
- Deep archimedean copulas. In NeurIPS, 2020.
- Yucong Liu. Neural networks are integrable. arXiv preprint arXiv:2310.14394, 2023.
- Negative mixture models via squaring: Representation and learning. In The 6th Workshop on Tractable Probabilistic Modeling, 2023.
- Canonical vine copulas in the context of modern portfolio management: Are they worth it? Asymmetric Dependence in Finance: Diversification, Correlation and Portfolio Management in Market Downturns, 2018.
- Physics-informed neural networks with hard constraints for inverse design. SIAM Journal on Scientific Computing, 43(6), 2021.
- A universal approximation theorem of deep neural networks for expressing probability distributions. In NeurIPS, 2020.
- The expressive power of neural networks: A view from the width. In NeurIPS, 2017.
- Non-parametric models for non-negative functions. In NeurIPS, 2020.
- A new hybrid method to improve the ultra-short-term prediction of lod. Journal of geodesy, 94, 2020.
- Nonparametric universal copula modeling. Applied Stochastic Models in Business and Industry, 36(1), 2020.
- Michael Naaman. On the tight constant in the multivariate dvoretzky–kiefer–wolfowitz inequality. Statistics & Probability Letters, 173, 2021.
- Nonparametric estimation of simplified vine copula models: comparison of methods. Dependence Modeling, 5(1), 2017.
- Roger B Nelsen. An introduction to copulas. Springer, 2006.
- Generative archimedean copulas. In UAI, 2021.
- Approximation by finite mixtures of continuous density functions that vanish at infinity. Cogent Mathematics & Statistics, 7(1), 2020.
- Masked autoregressive flow for density estimation. In NeurIPS, 2017.
- Psd representations for effective probability models. In NeurIPS, 2021.
- G Salvadori and Carlo De Michele. Frequency analysis via copulas: Theoretical aspects and applications to hydrological events. Water resources research, 40(12), 2004.
- Derivatives and fisher information of bivariate copulas. Statistical papers, 55(2), 2014.
- Joseph Sill. Monotonic networks. In NeurIPS, 1997.
- Bernard W Silverman. Density estimation for statistics and data analysis, volume 26. CRC press, 1986.
- Abe Sklar. Random variables, distribution functions, and copulas: a personal look backward and forward. Lecture notes-monograph series, 1996.
- M Sklar. Fonctions de répartition à n dimensions et leurs marges. Annales de l’ISUP, 8(3), 1959.
- Encoding negative dependencies in probabilistic circuits. In The 6th Workshop on Tractable Probabilistic Modeling, 2023.
- Learning structured output representation using deep conditional generative models. In NeurIPS, 2015.
- B Sohrabian. Geostatistical prediction through convex combination of archimedean copulas. Spatial Statistics, 41, 2021.
- Sifting the evidence—what’s wrong with significance tests? Physical therapy, 81(8), 2001.
- Retrospective uncertainties for deep models using vine copulas. In AISTATS, 2023.
- A simple approximation for the bivariate normal integral. Communications in Statistics-Simulation and Computation, 52(4), 2023.
- Squared neural families: A new class of tractable density models. arXiv preprint arXiv:2305.13552, 2023.
- Larry Wasserman. All of statistics: a concise course in statistical inference, volume 26. Springer, 2004.
- Unconstrained monotonic neural networks. In NeurIPS, 2019.
- A coherence algorithm for 3-d seismic data analysis based on the mutual information. IEEE Geoscience and Remote Sensing Letters, 16(6), 2019.
- A mixture copula bayesian network model for multimodal genomic data. Cancer Informatics, 16, 2017.