Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Attention-based Dynamic Multilayer Graph Neural Networks for Loan Default Prediction (2402.00299v2)

Published 1 Feb 2024 in q-fin.GN and cs.LG

Abstract: Whereas traditional credit scoring tends to employ only individual borrower- or loan-level predictors, it has been acknowledged for some time that connections between borrowers may result in default risk propagating over a network. In this paper, we present a model for credit risk assessment leveraging a dynamic multilayer network built from a Graph Neural Network and a Recurrent Neural Network, each layer reflecting a different source of network connection. We test our methodology in a behavioural credit scoring context using a dataset provided by U.S. mortgage financier Freddie Mac, in which different types of connections arise from the geographical location of the borrower and their choice of mortgage provider. The proposed model considers both types of connections and the evolution of these connections over time. We enhance the model by using a custom attention mechanism that weights the different time snapshots according to their importance. After testing multiple configurations, a model with GAT, LSTM, and the attention mechanism provides the best results. Empirical results demonstrate that, when it comes to predicting probability of default for the borrowers, our proposed model brings both better results and novel insights for the analysis of the importance of connections and timestamps, compared to traditional methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (76)
  1. Attention-based recurrent neural network for multistep-ahead prediction of process performance. Computers & Chemical Engineering, 140, 106931.
  2. An LSTM network for highway trajectory prediction. In 2017 IEEE 20th international conference on intelligent transportation systems (ITSC) (pp. 353–359).
  3. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations (ICLR).
  4. Network science. Cambridge University Press.
  5. Deep neural networks and tabular data: A survey. IEEE Transactions on Neural Networks and Learning Systems, Early Access, 1–21.
  6. Evolution of credit risk using a personalized pagerank algorithm for multilayer networks. In KDD MLF 2020: KDD Workshop on Machine Learning in Finance.
  7. Long short-term memory-networks for machine reading. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (pp. 551–561).
  8. On the properties of neural machine translation: Encoder-decoder approaches. In Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation (pp. 103–111).
  9. Empirical evaluation of gated recurrent neural networks on sequence modeling. In NIPS 2014 Workshop on Deep Learning.
  10. Explainable AI for operational research: A defining framework, methods, applications, and a research agenda. European Journal of Operational Research, In press.
  11. What does your Facebook profile reveal about your creditworthiness? Using alternative data for microfinance. Journal of the Operational Research Society, 70, 353–363.
  12. Long-term recurrent convolutional networks for visual recognition and description. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39, 677–691.
  13. Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14, 179–211.
  14. Loan default correlation using an Archimedean copula approach: A case for recalibration. Economic Modelling, 47, 340–354.
  15. FreddieMac (2022). Single family loan-level dataset. URL: https://www.freddiemac.com/research/datasets/sf-loanlevel-dataset.
  16. Gordy, M. B. (2000). A comparative anatomy of credit risk models. Journal of Banking & Finance, 24, 119–149.
  17. mGNN: Generalizing the graph neural networks to the multilayer case. arXiv preprint arXiv:2109.10119, .
  18. Multi-dimensional recurrent neural networks. In International Conference on Artificial Neural Networks (ICANN 2007) (pp. 549–558).
  19. Deep learning for credit scoring: Do or don’t? European Journal of Operational Research, 295, 292–305.
  20. Haythornthwaite, C. (1996). Social network analysis: An approach and technique for the study of information exchange. Library & Information Science Research, 18, 323–342.
  21. Long short-term memory. Neural Computation, 9, 1735–1780.
  22. Structural-RNN: Deep learning on spatio-temporal graphs. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5308–5317).
  23. Representation learning for dynamic graphs: A survey. Journal of Machine Learning Research, 21, 1–73.
  24. Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations (ICLR).
  25. Multilayer networks. Journal of Complex Networks, 2, 203–271.
  26. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (pp. 1269–1278).
  27. Graph convolutional network-based credit default prediction utilizing three types of virtual distances among borrowers. Expert Systems with Applications, 168, 114411.
  28. Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research. European Journal of Operational Research, 247, 124–136.
  29. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. In 6th International Conference on Learning Representations (ICLR).
  30. Graph neural network with self-attention and multi-task learning for credit default risk prediction. In 23rd International Conference on Web Information Systems Engineering – WISE 2022 (pp. 616–629).
  31. A structured self-attentive sentence embedding. In 5th International Conference on Learning Representations (ICLR).
  32. A survey on long short-term memory networks for time series prediction. Procedia CIRP, 99, 650–655.
  33. Evaluating credit risk models. Journal of Banking & Finance, 24, 151–165.
  34. A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS’17) (pp. 4768––4777).
  35. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (pp. 1412–1421).
  36. Deep canonically correlated LSTMs. arXiv preprint arXiv:1801.05407, .
  37. Dynamic graph convolutional networks. Pattern Recognition, 97, 107000.
  38. Credit scoring methods: Latest trends and points to consider. The Journal of Finance and Data Science, 8, 180–201.
  39. MELL: Effective embedding method for multiplex networks. In Companion Proceedings of the The Web Conference 2018 (pp. 1261–1268).
  40. State initialization for recurrent neural network modeling of time-series data. In 2017 International Joint Conference on Neural Networks (IJCNN) (pp. 2330–2337).
  41. Human action recognition using attention based LSTM network with dilated CNN features. Future Generation Computer Systems, 125, 820–830.
  42. Long short-term memory recurrent neural network for automatic speech recognition. IEEE Access, 10, 30069–30079.
  43. Multilayer network analysis for improved credit risk prediction. Omega, 105, 102520.
  44. The value of big data for credit scoring: Enhancing financial inclusion using mobile phone data and social network analytics. Applied Soft Computing, 74, 26–39.
  45. Deep sentence embedding using long short-term memory networks: Analysis and application to information retrieval. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 24, 694–707.
  46. Multilingual hierarchical attention networks for document classification. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 1015–1025).
  47. EvolveGCN: Evolving graph convolutional networks for dynamic graphs. In Proceedings of the AAAI Conference on Artificial Intelligence (Volume 34) (pp. 5363–5370).
  48. Syllable-based acoustic modeling with CTC-SMBR-LSTM. In 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) (pp. 173–177).
  49. Super-app behavioral patterns in credit risk models: Financial, statistical and regulatory implications. Expert Systems with Applications, 169, 114486.
  50. Dynamic graph representation learning via self-attention networks. In Workshop on Representation Learning on Graphs and Manifolds, ICLR 2019.
  51. GrAMME: Semisupervised learning using multilayered graph attention models. IEEE Transactions on Neural Networks and Learning Systems, 31, 3977–3988.
  52. Machine learning-driven credit risk: A systemic review. Neural Computing and Applications, 34, 14327––14339.
  53. Understanding dropout for graph neural networks. In Companion Proceedings of the Web Conference 2022 (pp. 1128–1138).
  54. Foundations and modeling of dynamic networks using dynamic graph neural networks: A survey. IEEE Access, 9, 79143–79168.
  55. Šter, B. (2013). Selective recurrent neural network. Neural Processing Letters, 38, 1–15.
  56. Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems (Volume 2) (pp. 3104––3112).
  57. Graph neural network-based node classification with hard sample strategy. In 2021 International Conference on Cyber-Physical Social Intelligence (ICCSI) (pp. 1–4).
  58. Credit scoring and its applications. SIAM-Society for Industrial and Applied Mathematics.
  59. DyRep: Learning representations over dynamic graphs. In 7th International Conference on Learning Representations (ICLR).
  60. Graph clustering with graph neural networks. In Proceedings of the 16th International Workshop on Mining and Learning with Graphs (MLG).
  61. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS’17).
  62. Graph attention networks. In 6th International Conference on Learning Representations (ICLR).
  63. The effect of interfirm financial transactions on the credit risk of small and medium-sized enterprises. Journal of the Royal Statistical Society Series A: Statistics in Society, 182, 1205–1226.
  64. Financial default prediction via motif-preserving graph neural network with curriculum learning. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 2233–2242).
  65. A review on graph neural network methods in financial applications. Journal of Data Science, 20, 111–134.
  66. Graph learning based recommender systems: A review. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI-21) (pp. 4644–4652).
  67. Attentional factorization machines: Learning the weight of feature interactions via attention networks. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17) (pp. 3119–3125).
  68. Inductive representation learning on temporal graphs. In 8th International Conference on Learning Representations (ICLR).
  69. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of the 32nd International Conference on Machine Learning (Volume 37) (pp. 2048–2057).
  70. LSTM and GRU neural network performance comparison study: Taking Yelp review dataset as an example. In 2020 International Workshop on Electronic Communication and Artificial Intelligence (IWECAI) (pp. 98–101).
  71. Financial risk analysis for SMEs with graph-based supply chain mining. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence (IJCAI-20) (pp. 4661–4667).
  72. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18) (pp. 3634–3640).
  73. An improved LSTM model for correcting grammatical errors in English text. In Proceedings of the International Conference on Cloud Computing, Performance Computing, and Deep Learning (CCPCDL 2023).
  74. Link prediction based on graph neural networks. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS’18).
  75. T-GCN: A temporal graph convolutional network for traffic prediction. IEEE Transactions on Intelligent Transportation Systems, 21, 3848–3858.
  76. Graph neural networks: A review of methods and applications. AI Open, 1, 57–81.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets