Papers
Topics
Authors
Recent
Search
2000 character limit reached

Topological Feature Selection

Published 19 Feb 2023 in cs.LG and cs.AI | (2302.09543v3)

Abstract: In this paper, we introduce a novel unsupervised, graph-based filter feature selection technique which exploits the power of topologically constrained network representations. We model dependency structures among features using a family of chordal graphs (the Triangulated Maximally Filtered Graph), and we maximise the likelihood of features' relevance by studying their relative position inside the network. Such an approach presents three aspects that are particularly satisfactory compared to its alternatives: (i) it is highly tunable and easily adaptable to the nature of input data; (ii) it is fully explainable, maintaining, at the same time, a remarkable level of simplicity; (iii) it is computationally cheaper compared to its alternatives. We test our algorithm on 16 benchmark datasets from different applicative domains showing that it outperforms or matches the current state-of-the-art under heterogeneous evaluation conditions.

Authors (2)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (66)
  1. Feature selection datasets, arizona state university. https://jundongl.github.io/scikit-feature/datasets.html, 2020. Accessed: 2022-09-19.
  2. Infinite feature selection. https://github.com/fullyz/Infinite-Feature-Selection/tree/master/data, 2020. Accessed: 2022-09-19.
  3. Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proceedings of the National Academy of Sciences, 96(12):6745–6750, 1999.
  4. Aste, T. Topological regularization with information filtering networks. Information Sciences, 608:655–669, 2022.
  5. Complex networks on hyperbolic surfaces. Physica A: Statistical Mechanics and its Applications, 346(1-2):20–26, 2005.
  6. Assessing the accuracy of prediction algorithms for classification: an overview. Bioinformatics, 16(5):412–424, 2000.
  7. Parsimonious modeling with information filtering networks. Physical Review E, 94(6):062306, 2016.
  8. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Transactions on pattern analysis and machine intelligence, 19(7):711–720, 1997.
  9. Dependency structures in cryptocurrency market from high to low frequency. arXiv preprint arXiv:2206.03386, 2022.
  10. Anatomy of a stablecoin’s failure: The terra-luna case. Finance Research Letters, pp.  103358, 2022.
  11. A survey on feature selection methods. Computers & Electrical Engineering, 40(1):16–28, 2014.
  12. Christensen, A. P. Networktoolbox: Methods and measures for brain, cognitive, and psychometric network analysis in r. R J., 10(2):422, 2018.
  13. Network structure of the wisconsin schizotypy scales–short forms: Examining psychometric network filtering approaches. Behavior Research Methods, 50(6):2531–2550, 2018.
  14. Genetic, epigenetic, and environmental factors controlling oxytocin receptor gene expression. Clinical epigenetics, 13(1):1–16, 2021.
  15. Dietterich, T. G. Approximate statistical tests for comparing supervised classification learning algorithms. Neural computation, 10(7):1895–1923, 1998.
  16. Donoho, D. L. et al. High-dimensional data analysis: The curses and blessings of dimensionality. AMS math challenges lecture, 1(2000):32, 2000.
  17. UCI machine learning repository, 2017. URL http://archive.ics.uci.edu/ml.
  18. Experiences with the model-based generation of big data pipelines. Datenbanksysteme für Business, Technologie und Web (BTW 2017)-Workshopband, 2017.
  19. Mctwo: a two-step feature selection algorithm based on maximal information coefficient. BMC bioinformatics, 17(1):1–14, 2016.
  20. Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. science, 286(5439):531–537, 1999.
  21. Gorodkin, J. Comparing two k-category assignments by a k-category correlation coefficient. Computational biology and chemistry, 28(5-6):367–374, 2004.
  22. Competitive baseline methods set new standards for the nips 2003 feature selection benchmark. Pattern recognition letters, 28(12):1438–1444, 2007.
  23. Feature extraction: foundations and applications, volume 207. Springer, 2008.
  24. Data mining: concepts and techniques. Morgan kaufmann, 2022.
  25. Huang, S. H. Supervised feature selection: A tutorial. Artif. Intell. Res., 4(2):22–37, 2015.
  26. Hull, J. J. A database for handwritten text recognition research. IEEE Transactions on pattern analysis and machine intelligence, 16(5):550–554, 1994.
  27. Multi-modal functional mri to explore placental function over gestation. Magnetic resonance in medicine, 81(2):1191–1204, 2019.
  28. A comparison of mcc and cen error measures in multi-class prediction. 2012.
  29. Fundamentals of machine learning for predictive data analytics: algorithms, worked examples, and case studies. MIT press, 2020.
  30. Probabilistic graphical models: principles and techniques. MIT press, 2009.
  31. Lang, K. Newsweeder: Learning to filter netnews. In Machine Learning Proceedings 1995, pp.  331–339. Elsevier, 1995.
  32. Lee, C.-Y. Correlations among centrality measures in complex networks. arXiv preprint physics/0605220, 2006.
  33. Feature selection: A data perspective. ACM Computing Surveys (CSUR), 50(6):94, 2018.
  34. Mantegna, R. N. Hierarchical structure in financial markets. The European Physical Journal B-Condensed Matter and Complex Systems, 11(1):193–197, 1999.
  35. Network filtering for big data: Triangulated maximally filtered graph. Journal of complex Networks, 5(2):161–178, 2017.
  36. Matthews, B. W. Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure, 405(2):442–451, 1975.
  37. A survey on feature selection. Procedia Computer Science, 91:919–926, 2016.
  38. Practical text mining and statistical analysis for non-structured text data applications. Academic Press, 2012.
  39. A survey of shape feature extraction techniques. Pattern recognition, 15(7):43–90, 2008.
  40. Mosley, L. A balanced approach to the multi-class imbalance problem. Doctor of Philosophy Thesis, Iowa State University of Science and Technology, USA, 2013.
  41. Columbia object image library (coil-100). 1996.
  42. Combinatorial optimization: algorithms and complexity. Courier Corporation, 1998.
  43. Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10350, 2021.
  44. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
  45. Why and when can deep-but not shallow-networks avoid the curse of dimensionality: a review. International Journal of Automation and Computing, 14(5):503–519, 2017.
  46. Portfolio optimization with sparse multivariate modeling. Journal of Asset Management, 23(6):445–465, 2022.
  47. Infinite feature selection. In Proceedings of the IEEE International Conference on Computer Vision, pp.  4202–4210, 2015.
  48. Infinite feature selection: a graph-based feature filtering approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(12):4396–4410, 2020.
  49. Parameterisation of a stochastic model for human face identification. In Proceedings of 1994 IEEE workshop on applications of computer vision, pp.  138–142. IEEE, 1994.
  50. Quantifying impact and response in markets using information filtering networks. Journal of Physics: Complexity, 3(2):025004, 2022.
  51. A survey on semi-supervised feature selection methods. Pattern Recognition, 64:141–158, 2017.
  52. Computational study of the us stock market evolution: a rank correlation-based network model. Computational Management Science, 10(2):81–103, 2013.
  53. The cmu pose, illumination, and expression (pie) database. In Proceedings of fifth IEEE international conference on automatic face gesture recognition, pp.  53–58. IEEE, 2002.
  54. A review of unsupervised feature selection methods. Artificial Intelligence Review, 53(2):907–948, 2020.
  55. Correlation-based biological networks. In Complex Systems II, volume 6802, pp.  226–236. SPIE, 2008.
  56. Nested hierarchies in planar graphs. Discrete Applied Mathematics, 159(17):2135–2146, 2011.
  57. Hierarchical information clustering by means of topologically embedded graphs. PloS one, 7(3):e31929, 2012.
  58. Energy and policy considerations for deep learning in nlp. arXiv preprint arXiv:1906.02243, 2019.
  59. A tool for filtering information in complex systems. Proceedings of the National Academy of Sciences, 102(30):10421–10426, 2005.
  60. How correlated are network centrality measures? Connections (Toronto, Ont.), 28(1):16, 2008.
  61. Ftx’s downfall and binance’s consolidation: the fragility of centralized digital finance. arXiv preprint arXiv:2302.11371, 2023.
  62. Dynamic portfolio optimization with inverse covariance clustering. Expert Systems with Applications, pp.  118739, 2022a.
  63. Sparsification and filtering for spatial-temporal gnn in multivariate time-series. arXiv preprint arXiv:2203.03991, 2022b.
  64. Homological neural networks: A sparse architecture for multivariate complexity, 2023.
  65. Building pipelines for heterogeneous execution environments for big data processing. IEEE Software, 33(2):60–67, 2016.
  66. Distribution-balanced stratified cross-validation for accuracy estimation. Journal of Experimental & Theoretical Artificial Intelligence, 12(1):1–12, 2000.
Citations (3)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up