CasTGAN: Cascaded Generative Adversarial Network for Realistic Tabular Data Synthesis (2307.00384v2)
Abstract: Generative adversarial networks (GANs) have drawn considerable attention in recent years for their proven capability in generating synthetic data which can be utilised for multiple purposes. While GANs have demonstrated tremendous successes in producing synthetic data samples that replicate the dynamics of the original datasets, the validity of the synthetic data and the underlying privacy concerns represent major challenges which are not sufficiently addressed. In this work, we design a cascaded tabular GAN framework (CasTGAN) for generating realistic tabular data with a specific focus on the validity of the output. In this context, validity refers to the the dependency between features that can be found in the real data, but is typically misrepresented by traditional generative models. Our key idea entails that employing a cascaded architecture in which a dedicated generator samples each feature, the synthetic output becomes more representative of the real data. Our experimental results demonstrate that our model is capable of generating synthetic tabular data that can be used for fitting machine learning models. In addition, our model captures well the constraints and the correlations between the features of the real data, especially the high dimensional datasets. Furthermore, we evaluate the risk of white-box privacy attacks on our model and subsequently show that applying some perturbations to the auxiliary learners in CasTGAN increases the overall robustness of our model against targeted attacks.
- J. Walonoski, M. Kramer, J. Nichols, A. Quina, C. Moesel, D. Hall, C. Duffett, K. Dube, T. Gallagher, and S. McLachlan, “Synthea: An approach, method, and software mechanism for generating synthetic patients and the synthetic electronic health care record,” Journal of the American Medical Informatics Association, vol. 25, no. 3, pp. 230–238, 2018. [Online]. Available: https://doi.org/10.1093/jamia/ocx147
- G. Pavlidis, “Financial information in the context of anti-money laundering: Broadening the access of law enforcement and facilitating information exchanges,” Journal of Money Laundering Control, vol. 23, no. 2, pp. 369–378, 2020. [Online]. Available: https://doi.org/10.1108/JMLC-10-2019-0081
- P. Regulation, “Regulation (eu) 2016/679 of the european parliament and of the council,” Regulation (eu), vol. 679, p. 2016, 2016. [Online]. Available: https://eur-lex.europa.eu/eli/reg/2016/679/2016-05-04
- I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” Advances in neural information processing systems, vol. 27, 2014.
- T. Young, D. Hazarika, S. Poria, and E. Cambria, “Recent trends in deep learning based natural language processing,” ieee Computational intelligenCe magazine, vol. 13, no. 3, pp. 55–75, 2018. [Online]. Available: https://doi.org/10.1109/MCI.2018.2840738
- D. Li, D. Chen, B. Jin, L. Shi, J. Goh, and S.-K. Ng, “Mad-gan: Multivariate anomaly detection for time series data with generative adversarial networks,” in International Conference on Artificial Neural Networks. Springer, 2019, pp. 703–716. [Online]. Available: https://doi.org/10.1007/978-3-030-30490-4_56
- M. Mirza and S. Osindero, “Conditional generative adversarial nets,” arXiv preprint, 2014. [Online]. Available: https://doi.org/10.48550/arXiv.1411.1784
- A. Makhzani, J. Shlens, N. Jaitly, I. Goodfellow, and B. Frey, “Adversarial autoencoders,” arXiv preprint, 2015. [Online]. Available: https://doi.org/10.48550/arXiv.1511.05644
- M. Esmaeilpour, N. Chaalia, A. Abusitta, F.-X. Devailly, W. Maazoun, and P. Cardinal, “Bi-discriminator gan for tabular data synthesis,” Pattern Recognition Letters, vol. 159, pp. 204–210, 2022. [Online]. Available: https://doi.org/10.1016/j.patrec.2022.05.023
- S. A. Assefa, D. Dervovic, M. Mahfouz, R. E. Tillman, P. Reddy, and M. Veloso, “Generating synthetic data in finance: opportunities, challenges and pitfalls,” in Proceedings of the First ACM International Conference on AI in Finance, 2020, pp. 1–8. [Online]. Available: https://doi.org/10.1145/3383455.3422554
- A. Odena, C. Olah, and J. Shlens, “Conditional image synthesis with auxiliary classifier gans,” in International conference on machine learning. PMLR, 2017, pp. 2642–2651.
- M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein generative adversarial networks,” in International conference on machine learning. PMLR, 2017, pp. 214–223.
- I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville, “Improved training of wasserstein gans,” Advances in neural information processing systems, vol. 30, 2017.
- R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in 2017 IEEE symposium on security and privacy (SP). IEEE, 2017, pp. 3–18. [Online]. Available: https://doi.org/10.1109/SP.2017.41
- S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha, “Privacy risk in machine learning: Analyzing the connection to overfitting,” in 2018 IEEE 31st computer security foundations symposium (CSF). IEEE, 2018, pp. 268–282. [Online]. Available: https://doi.org/10.1109/CSF.2018.00027
- C. Dwork, “Differential privacy: A survey of results,” in International conference on theory and applications of models of computation. Springer, 2008, pp. 1–19. [Online]. Available: https://doi.org/10.1007/978-3-540-79228-4_1
- J. Hayes, L. Melis, G. Danezis, and E. De Cristofaro, “Logan: Membership inference attacks against generative models,” in Proceedings on Privacy Enhancing Technologies (PoPETs), vol. 2019, no. 1. De Gruyter, 2019, pp. 133–152. [Online]. Available: https://doi.org/10.2478/popets-2019-0008
- L. Xie, K. Lin, S. Wang, F. Wang, and J. Zhou, “Differentially private generative adversarial network,” arXiv preprint, 2018. [Online]. Available: https://doi.org/10.48550/arXiv.1802.06739
- J. Jordon, J. Yoon, and M. Van Der Schaar, “Pate-gan: Generating synthetic data with differential privacy guarantees,” in International conference on learning representations, 2018.
- J. Chen, W. H. Wang, H. Gao, and X. Shi, “Par-gan: Improving the generalization of generative adversarial networks against membership inference attacks,” in Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2021, pp. 127–137. [Online]. Available: https://doi.org/10.1145/3447548.3467445
- J. Zhang, G. Cormode, C. M. Procopiuc, D. Srivastava, and X. Xiao, “Privbayes: Private data release via bayesian networks,” ACM Transactions on Database Systems (TODS), vol. 42, no. 4, pp. 1–41, 2017. [Online]. Available: https://doi.org/10.1145/3134428
- A. Goncalves, P. Ray, B. Soper, J. Stevens, L. Coyle, and A. P. Sales, “Generation and evaluation of synthetic patient data,” BMC medical research methodology, vol. 20, no. 1, pp. 1–40, 2020. [Online]. Available: https://doi.org/10.1186/s12874-020-00977-1
- J. P. Reiter, “Using cart to generate partially synthetic public use microdata,” Journal of official statistics, vol. 21, no. 3, p. 441, 2005.
- D. S. Watson, K. Blesch, J. Kapar, and M. N. Wright, “Adversarial random forests for density estimation and generative modeling,” in International Conference on Artificial Intelligence and Statistics. PMLR, 2023, pp. 5357–5375.
- D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint, 2013. [Online]. Available: https://doi.org/10.48550/arXiv.1312.6114
- Z. Wan, Y. Zhang, and H. He, “Variational autoencoder based synthetic data generation for imbalanced learning,” in 2017 IEEE symposium series on computational intelligence (SSCI). IEEE, 2017, pp. 1–7. [Online]. Available: https://doi.org/10.1109/SSCI.2017.8285168
- Z. Islam, M. Abdel-Aty, Q. Cai, and J. Yuan, “Crash data augmentation using variational autoencoder,” Accident Analysis & Prevention, vol. 151, p. 105950, 2021. [Online]. Available: https://doi.org/10.1016/j.aap.2020.105950
- L. Xu, M. Skoularidou, A. Cuesta-Infante, and K. Veeramachaneni, “Modeling tabular data using conditional gan,” Advances in neural information processing systems, vol. 32, 2019.
- I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner, “beta-vae: Learning basic visual concepts with a constrained variational framework,” in International conference on learning representations, 2017.
- J. Lee, J. Hyeong, J. Jeon, N. Park, and J. Cho, “Invertible tabular gans: Killing two birds with one stone for tabular data synthesis,” vol. 34, 2021, p. 11.
- S. Kamthe, S. Assefa, and M. Deisenroth, “Copula flows for synthetic data generation,” arXiv preprint, 2021. [Online]. Available: https://doi.org/10.48550/arXiv.2101.00598
- J. Lee, M. Kim, Y. Jeong, and Y. Ro, “Differentially private normalizing flows for synthetic tabular data generation,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 7, 2022, pp. 7345–7353. [Online]. Available: https://doi.org/10.1609/aaai.v36i7.20697
- N. Park, M. Mohammadi, K. Gorde, S. Jajodia, H. Park, and Y. Kim, “Data synthesis based on generative adversarial networks,” Proc. VLDB Endow., vol. 11, no. 10, p. 1071–1083, jun 2018. [Online]. Available: https://doi.org/10.14778/3231751.3231757
- E. Choi, S. Biswal, B. Malin, J. Duke, W. F. Stewart, and J. Sun, “Generating multi-label discrete patient records using generative adversarial networks,” in Machine learning for healthcare conference. PMLR, 2017, pp. 286–305.
- J. Engelmann and S. Lessmann, “Conditional Wasserstein GAN-based oversampling of tabular data for imbalanced learning,” Expert Systems with Applications, vol. 174, p. 114582, Jul. 2021. [Online]. Available: https://doi.org/10.1016/j.eswa.2021.114582
- Z. Zhao, A. Kunar, R. Birke, and L. Y. Chen, “Ctab-gan: Effective table data synthesizing,” in Proceedings of The 13th Asian Conference on Machine Learning, ser. Proceedings of Machine Learning Research, V. N. Balasubramanian and I. Tsang, Eds., vol. 157. PMLR, 17–19 Nov 2021, pp. 97–112.
- Z. Pan, W. Yu, X. Yi, A. Khan, F. Yuan, and Y. Zheng, “Recent progress on generative adversarial networks (gans): A survey,” IEEE access, vol. 7, pp. 36 322–36 333, 2019. [Online]. Available: https://doi.org/10.1109/ACCESS.2019.2905015
- L. Xu and K. Veeramachaneni, “Synthesizing tabular data using generative adversarial networks,” arXiv preprint, 2018. [Online]. Available: https://doi.org/10.48550/arXiv.1811.11264
- A. Srivastava, L. Valkov, C. Russell, M. U. Gutmann, and C. Sutton, “Veegan: Reducing mode collapse in gans using implicit variational learning,” Advances in neural information processing systems, vol. 30, 2017.
- E. Strelcenia and S. Prakoonwit, “A survey on gan techniques for data augmentation to address the imbalanced data issues in credit card fraud detection,” Machine Learning and Knowledge Extraction, vol. 5, no. 1, pp. 304–329, 2023. [Online]. Available: https://doi.org/10.3390/make5010019
- J. Kim, J. Jeon, J. Lee, J. Hyeong, and N. Park, “Oct-gan: Neural ode-based conditional tabular gans,” in Proceedings of the Web Conference 2021, 2021, pp. 1506–1515. [Online]. Available: https://doi.org/10.1145/3442381.3449999
- A. Waheed, M. Goyal, D. Gupta, A. Khanna, F. Al-Turjman, and P. R. Pinheiro, “Covidgan: data augmentation using auxiliary classifier gan for improved covid-19 detection,” Ieee Access, vol. 8, pp. 91 916–91 923, 2020. [Online]. Available: https://doi.org/10.1109/ACCESS.2020.2994762
- G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu, “Lightgbm: A highly efficient gradient boosting decision tree,” Advances in neural information processing systems, vol. 30, pp. 3146–3154, 2017.
- D. M. Blei and M. I. Jordan, “Variational inference for dirichlet process mixtures,” Bayesian analysis, vol. 1, no. 1, pp. 121–143, 2006. [Online]. Available: https://doi.org/10.1214/06-BA104
- T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, “Improved techniques for training gans,” Advances in neural information processing systems, vol. 29, 2016.
- J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint, 2016. [Online]. Available: https://doi.org/10.48550/arXiv.1607.06450
- R. Kohavi and B. Becker, “Adult data set,” 1996. [Online]. Available: https://archive.ics.uci.edu/ml/datasets/adult
- S. Moro, P. Cortez, and P. Rita, “Bank marketing data set,” 2012. [Online]. Available: https://archive.ics.uci.edu/ml/datasets/bank+marketing
- I.-C. Yeh and C.-H. Lien, “default of credit card clients data set,” 2016. [Online]. Available: https://archive.ics.uci.edu/ml/datasets/default+of+credit+card+clients
- A. Taboul, “Diabetes health indicators dataset,” 2021. [Online]. Available: https://www.kaggle.com/alexteboul/diabetes-health-indicators-dataset
- King County, “House sales in king county, usa,” 2021. [Online]. Available: https://www.kaggle.com/harlfoxem/housesalesprediction
- B. Tunguz, “Used cars auction prices,” 2021. [Online]. Available: https://www.kaggle.com/tunguz/used-car-auction-prices/code
- I. M. Chakravarti, R. G. Laha, and J. Roy, “Handbook of methods of applied statistics,” Wiley Series in Probability and Mathematical Statistics (USA) eng, 1967.
- M. Padala, D. Das, and S. Gujar, “Effect of input noise dimension in gans,” in International Conference on Neural Information Processing, 2021, pp. 558–569.
- I. Goodfellow, “Nips 2016 tutorial: Generative adversarial networks,” arXiv preprint, 2016. [Online]. Available: https://doi.org/10.48550/arXiv.1701.00160
- Abdallah Alshantti (2 papers)
- Damiano Varagnolo (9 papers)
- Adil Rasheed (68 papers)
- Aria Rahmati (1 paper)
- Frank Westad (2 papers)