Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

CasTGAN: Cascaded Generative Adversarial Network for Realistic Tabular Data Synthesis (2307.00384v2)

Published 1 Jul 2023 in cs.LG and cs.AI

Abstract: Generative adversarial networks (GANs) have drawn considerable attention in recent years for their proven capability in generating synthetic data which can be utilised for multiple purposes. While GANs have demonstrated tremendous successes in producing synthetic data samples that replicate the dynamics of the original datasets, the validity of the synthetic data and the underlying privacy concerns represent major challenges which are not sufficiently addressed. In this work, we design a cascaded tabular GAN framework (CasTGAN) for generating realistic tabular data with a specific focus on the validity of the output. In this context, validity refers to the the dependency between features that can be found in the real data, but is typically misrepresented by traditional generative models. Our key idea entails that employing a cascaded architecture in which a dedicated generator samples each feature, the synthetic output becomes more representative of the real data. Our experimental results demonstrate that our model is capable of generating synthetic tabular data that can be used for fitting machine learning models. In addition, our model captures well the constraints and the correlations between the features of the real data, especially the high dimensional datasets. Furthermore, we evaluate the risk of white-box privacy attacks on our model and subsequently show that applying some perturbations to the auxiliary learners in CasTGAN increases the overall robustness of our model against targeted attacks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. J. Walonoski, M. Kramer, J. Nichols, A. Quina, C. Moesel, D. Hall, C. Duffett, K. Dube, T. Gallagher, and S. McLachlan, “Synthea: An approach, method, and software mechanism for generating synthetic patients and the synthetic electronic health care record,” Journal of the American Medical Informatics Association, vol. 25, no. 3, pp. 230–238, 2018. [Online]. Available: https://doi.org/10.1093/jamia/ocx147
  2. G. Pavlidis, “Financial information in the context of anti-money laundering: Broadening the access of law enforcement and facilitating information exchanges,” Journal of Money Laundering Control, vol. 23, no. 2, pp. 369–378, 2020. [Online]. Available: https://doi.org/10.1108/JMLC-10-2019-0081
  3. P. Regulation, “Regulation (eu) 2016/679 of the european parliament and of the council,” Regulation (eu), vol. 679, p. 2016, 2016. [Online]. Available: https://eur-lex.europa.eu/eli/reg/2016/679/2016-05-04
  4. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” Advances in neural information processing systems, vol. 27, 2014.
  5. T. Young, D. Hazarika, S. Poria, and E. Cambria, “Recent trends in deep learning based natural language processing,” ieee Computational intelligenCe magazine, vol. 13, no. 3, pp. 55–75, 2018. [Online]. Available: https://doi.org/10.1109/MCI.2018.2840738
  6. D. Li, D. Chen, B. Jin, L. Shi, J. Goh, and S.-K. Ng, “Mad-gan: Multivariate anomaly detection for time series data with generative adversarial networks,” in International Conference on Artificial Neural Networks.   Springer, 2019, pp. 703–716. [Online]. Available: https://doi.org/10.1007/978-3-030-30490-4_56
  7. M. Mirza and S. Osindero, “Conditional generative adversarial nets,” arXiv preprint, 2014. [Online]. Available: https://doi.org/10.48550/arXiv.1411.1784
  8. A. Makhzani, J. Shlens, N. Jaitly, I. Goodfellow, and B. Frey, “Adversarial autoencoders,” arXiv preprint, 2015. [Online]. Available: https://doi.org/10.48550/arXiv.1511.05644
  9. M. Esmaeilpour, N. Chaalia, A. Abusitta, F.-X. Devailly, W. Maazoun, and P. Cardinal, “Bi-discriminator gan for tabular data synthesis,” Pattern Recognition Letters, vol. 159, pp. 204–210, 2022. [Online]. Available: https://doi.org/10.1016/j.patrec.2022.05.023
  10. S. A. Assefa, D. Dervovic, M. Mahfouz, R. E. Tillman, P. Reddy, and M. Veloso, “Generating synthetic data in finance: opportunities, challenges and pitfalls,” in Proceedings of the First ACM International Conference on AI in Finance, 2020, pp. 1–8. [Online]. Available: https://doi.org/10.1145/3383455.3422554
  11. A. Odena, C. Olah, and J. Shlens, “Conditional image synthesis with auxiliary classifier gans,” in International conference on machine learning.   PMLR, 2017, pp. 2642–2651.
  12. M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein generative adversarial networks,” in International conference on machine learning.   PMLR, 2017, pp. 214–223.
  13. I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville, “Improved training of wasserstein gans,” Advances in neural information processing systems, vol. 30, 2017.
  14. R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in 2017 IEEE symposium on security and privacy (SP).   IEEE, 2017, pp. 3–18. [Online]. Available: https://doi.org/10.1109/SP.2017.41
  15. S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha, “Privacy risk in machine learning: Analyzing the connection to overfitting,” in 2018 IEEE 31st computer security foundations symposium (CSF).   IEEE, 2018, pp. 268–282. [Online]. Available: https://doi.org/10.1109/CSF.2018.00027
  16. C. Dwork, “Differential privacy: A survey of results,” in International conference on theory and applications of models of computation.   Springer, 2008, pp. 1–19. [Online]. Available: https://doi.org/10.1007/978-3-540-79228-4_1
  17. J. Hayes, L. Melis, G. Danezis, and E. De Cristofaro, “Logan: Membership inference attacks against generative models,” in Proceedings on Privacy Enhancing Technologies (PoPETs), vol. 2019, no. 1.   De Gruyter, 2019, pp. 133–152. [Online]. Available: https://doi.org/10.2478/popets-2019-0008
  18. L. Xie, K. Lin, S. Wang, F. Wang, and J. Zhou, “Differentially private generative adversarial network,” arXiv preprint, 2018. [Online]. Available: https://doi.org/10.48550/arXiv.1802.06739
  19. J. Jordon, J. Yoon, and M. Van Der Schaar, “Pate-gan: Generating synthetic data with differential privacy guarantees,” in International conference on learning representations, 2018.
  20. J. Chen, W. H. Wang, H. Gao, and X. Shi, “Par-gan: Improving the generalization of generative adversarial networks against membership inference attacks,” in Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2021, pp. 127–137. [Online]. Available: https://doi.org/10.1145/3447548.3467445
  21. J. Zhang, G. Cormode, C. M. Procopiuc, D. Srivastava, and X. Xiao, “Privbayes: Private data release via bayesian networks,” ACM Transactions on Database Systems (TODS), vol. 42, no. 4, pp. 1–41, 2017. [Online]. Available: https://doi.org/10.1145/3134428
  22. A. Goncalves, P. Ray, B. Soper, J. Stevens, L. Coyle, and A. P. Sales, “Generation and evaluation of synthetic patient data,” BMC medical research methodology, vol. 20, no. 1, pp. 1–40, 2020. [Online]. Available: https://doi.org/10.1186/s12874-020-00977-1
  23. J. P. Reiter, “Using cart to generate partially synthetic public use microdata,” Journal of official statistics, vol. 21, no. 3, p. 441, 2005.
  24. D. S. Watson, K. Blesch, J. Kapar, and M. N. Wright, “Adversarial random forests for density estimation and generative modeling,” in International Conference on Artificial Intelligence and Statistics.   PMLR, 2023, pp. 5357–5375.
  25. D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint, 2013. [Online]. Available: https://doi.org/10.48550/arXiv.1312.6114
  26. Z. Wan, Y. Zhang, and H. He, “Variational autoencoder based synthetic data generation for imbalanced learning,” in 2017 IEEE symposium series on computational intelligence (SSCI).   IEEE, 2017, pp. 1–7. [Online]. Available: https://doi.org/10.1109/SSCI.2017.8285168
  27. Z. Islam, M. Abdel-Aty, Q. Cai, and J. Yuan, “Crash data augmentation using variational autoencoder,” Accident Analysis & Prevention, vol. 151, p. 105950, 2021. [Online]. Available: https://doi.org/10.1016/j.aap.2020.105950
  28. L. Xu, M. Skoularidou, A. Cuesta-Infante, and K. Veeramachaneni, “Modeling tabular data using conditional gan,” Advances in neural information processing systems, vol. 32, 2019.
  29. I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner, “beta-vae: Learning basic visual concepts with a constrained variational framework,” in International conference on learning representations, 2017.
  30. J. Lee, J. Hyeong, J. Jeon, N. Park, and J. Cho, “Invertible tabular gans: Killing two birds with one stone for tabular data synthesis,” vol. 34, 2021, p. 11.
  31. S. Kamthe, S. Assefa, and M. Deisenroth, “Copula flows for synthetic data generation,” arXiv preprint, 2021. [Online]. Available: https://doi.org/10.48550/arXiv.2101.00598
  32. J. Lee, M. Kim, Y. Jeong, and Y. Ro, “Differentially private normalizing flows for synthetic tabular data generation,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 7, 2022, pp. 7345–7353. [Online]. Available: https://doi.org/10.1609/aaai.v36i7.20697
  33. N. Park, M. Mohammadi, K. Gorde, S. Jajodia, H. Park, and Y. Kim, “Data synthesis based on generative adversarial networks,” Proc. VLDB Endow., vol. 11, no. 10, p. 1071–1083, jun 2018. [Online]. Available: https://doi.org/10.14778/3231751.3231757
  34. E. Choi, S. Biswal, B. Malin, J. Duke, W. F. Stewart, and J. Sun, “Generating multi-label discrete patient records using generative adversarial networks,” in Machine learning for healthcare conference.   PMLR, 2017, pp. 286–305.
  35. J. Engelmann and S. Lessmann, “Conditional Wasserstein GAN-based oversampling of tabular data for imbalanced learning,” Expert Systems with Applications, vol. 174, p. 114582, Jul. 2021. [Online]. Available: https://doi.org/10.1016/j.eswa.2021.114582
  36. Z. Zhao, A. Kunar, R. Birke, and L. Y. Chen, “Ctab-gan: Effective table data synthesizing,” in Proceedings of The 13th Asian Conference on Machine Learning, ser. Proceedings of Machine Learning Research, V. N. Balasubramanian and I. Tsang, Eds., vol. 157.   PMLR, 17–19 Nov 2021, pp. 97–112.
  37. Z. Pan, W. Yu, X. Yi, A. Khan, F. Yuan, and Y. Zheng, “Recent progress on generative adversarial networks (gans): A survey,” IEEE access, vol. 7, pp. 36 322–36 333, 2019. [Online]. Available: https://doi.org/10.1109/ACCESS.2019.2905015
  38. L. Xu and K. Veeramachaneni, “Synthesizing tabular data using generative adversarial networks,” arXiv preprint, 2018. [Online]. Available: https://doi.org/10.48550/arXiv.1811.11264
  39. A. Srivastava, L. Valkov, C. Russell, M. U. Gutmann, and C. Sutton, “Veegan: Reducing mode collapse in gans using implicit variational learning,” Advances in neural information processing systems, vol. 30, 2017.
  40. E. Strelcenia and S. Prakoonwit, “A survey on gan techniques for data augmentation to address the imbalanced data issues in credit card fraud detection,” Machine Learning and Knowledge Extraction, vol. 5, no. 1, pp. 304–329, 2023. [Online]. Available: https://doi.org/10.3390/make5010019
  41. J. Kim, J. Jeon, J. Lee, J. Hyeong, and N. Park, “Oct-gan: Neural ode-based conditional tabular gans,” in Proceedings of the Web Conference 2021, 2021, pp. 1506–1515. [Online]. Available: https://doi.org/10.1145/3442381.3449999
  42. A. Waheed, M. Goyal, D. Gupta, A. Khanna, F. Al-Turjman, and P. R. Pinheiro, “Covidgan: data augmentation using auxiliary classifier gan for improved covid-19 detection,” Ieee Access, vol. 8, pp. 91 916–91 923, 2020. [Online]. Available: https://doi.org/10.1109/ACCESS.2020.2994762
  43. G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu, “Lightgbm: A highly efficient gradient boosting decision tree,” Advances in neural information processing systems, vol. 30, pp. 3146–3154, 2017.
  44. D. M. Blei and M. I. Jordan, “Variational inference for dirichlet process mixtures,” Bayesian analysis, vol. 1, no. 1, pp. 121–143, 2006. [Online]. Available: https://doi.org/10.1214/06-BA104
  45. T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, “Improved techniques for training gans,” Advances in neural information processing systems, vol. 29, 2016.
  46. J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint, 2016. [Online]. Available: https://doi.org/10.48550/arXiv.1607.06450
  47. R. Kohavi and B. Becker, “Adult data set,” 1996. [Online]. Available: https://archive.ics.uci.edu/ml/datasets/adult
  48. S. Moro, P. Cortez, and P. Rita, “Bank marketing data set,” 2012. [Online]. Available: https://archive.ics.uci.edu/ml/datasets/bank+marketing
  49. I.-C. Yeh and C.-H. Lien, “default of credit card clients data set,” 2016. [Online]. Available: https://archive.ics.uci.edu/ml/datasets/default+of+credit+card+clients
  50. A. Taboul, “Diabetes health indicators dataset,” 2021. [Online]. Available: https://www.kaggle.com/alexteboul/diabetes-health-indicators-dataset
  51. King County, “House sales in king county, usa,” 2021. [Online]. Available: https://www.kaggle.com/harlfoxem/housesalesprediction
  52. B. Tunguz, “Used cars auction prices,” 2021. [Online]. Available: https://www.kaggle.com/tunguz/used-car-auction-prices/code
  53. I. M. Chakravarti, R. G. Laha, and J. Roy, “Handbook of methods of applied statistics,” Wiley Series in Probability and Mathematical Statistics (USA) eng, 1967.
  54. M. Padala, D. Das, and S. Gujar, “Effect of input noise dimension in gans,” in International Conference on Neural Information Processing, 2021, pp. 558–569.
  55. I. Goodfellow, “Nips 2016 tutorial: Generative adversarial networks,” arXiv preprint, 2016. [Online]. Available: https://doi.org/10.48550/arXiv.1701.00160
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Abdallah Alshantti (2 papers)
  2. Damiano Varagnolo (9 papers)
  3. Adil Rasheed (68 papers)
  4. Aria Rahmati (1 paper)
  5. Frank Westad (2 papers)
Citations (5)