Inhomogeneous graph trend filtering via a l2,0 cardinality penalty (2304.05223v4)
Abstract: We study estimation of piecewise smooth signals over a graph. We propose a $\ell_{2,0}$-norm penalized Graph Trend Filtering (GTF) model to estimate piecewise smooth graph signals that exhibit inhomogeneous levels of smoothness across the nodes. We prove that the proposed GTF model is simultaneously a k-means clustering on the signal over the nodes and a minimum graph cut on the edges of the graph, where the clustering and the cut share the same assignment matrix. We propose two methods to solve the proposed GTF model: a spectral decomposition method and a method based on simulated annealing. In the experiment on synthetic and real-world datasets, we show that the proposed GTF model has a better performances compared with existing approaches on the tasks of denoising, support recovery and semi-supervised classification. We also show that the proposed GTF model can be solved more efficiently than existing models for the dataset with a large edge set.
- S. Chen, R. Varma, A. Singh, and J. Kovacevic, “Signal representations on graphs: Tools and applications,” CoRR, vol. abs/1512.05406, 2015. [Online]. Available: http://arxiv.org/abs/1512.05406
- S. Chen, R. Varma, A. Sandryhaila, and J. Kovačević, “Discrete signal processing on graphs: Sampling theory,” IEEE Transactions on Signal Processing, vol. 63, no. 24, pp. 6510–6523, 2015.
- S. Chen, A. Sandryhaila, J. M. F. Moura, and J. Kovačević, “Signal recovery on graphs: Variation minimization,” IEEE Transactions on Signal Processing, vol. 63, no. 17, pp. 4609–4624, 2015.
- D. Romero, M. Ma, and G. B. Giannakis, “Kernel-based reconstruction of graph signals,” IEEE Transactions on Signal Processing, vol. 65, no. 3, pp. 764–778, 2017.
- A. Elmoataz, O. Lezoray, and S. Bougleux, “Nonlocal discrete regularization on weighted graphs: A framework for image and manifold processing,” IEEE Transactions on Image Processing, vol. 17, no. 7, pp. 1047–1060, 2008.
- D. I. Shuman, S. K. Narang, P. Frossard, A. Ortega, and P. Vandergheynst, “The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains,” IEEE Signal Processing Magazine, vol. 30, no. 3, pp. 83–98, 2013.
- A. Ortega, P. Frossard, J. Kovačević, J. M. F. Moura, and P. Vandergheynst, “Graph signal processing: Overview, challenges, and applications,” Proceedings of the IEEE, vol. 106, no. 5, pp. 808–828, 2018.
- M. Belkin, I. Matveeva, and P. Niyogi, “Regularization and semi-supervised learning on large graphs,” in Learning Theory, J. Shawe-Taylor and Y. Singer, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2004, pp. 624–638.
- S. Bougleux, A. Elmoataz, and M. Melkemi, “Discrete regularization on weighted graphs for image and mesh filtering,” in Scale Space and Variational Methods in Computer Vision, F. Sgallari, A. Murli, and N. Paragios, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2007, pp. 128–139.
- D. A. Spielman, “Spectral graph theory and its applications,” in Proceedings of the 48th Annual IEEE Symposium on Foundations of Computer Science, ser. FOCS ’07. USA: IEEE Computer Society, 2007, p. 29–38. [Online]. Available: https://doi.org/10.1109/FOCS.2007.66
- X. Zhu, Z. Ghahramani, and J. Lafferty, “Semi-supervised learning using gaussian fields and harmonic functions,” in Proceedings of the Twentieth International Conference on International Conference on Machine Learning, ser. ICML’03. AAAI Press, 2003, p. 912–919.
- R. Varma, H. Lee, J. Kovačević, and Y. Chi, “Vector-valued graph trend filtering with non-convex penalties,” IEEE Transactions on Signal and Information Processing over Networks, vol. 6, pp. 48–62, 2020.
- Y. Wang, J. Sharpnack, A. J. Smola, and R. J. Tibshirani, “Trend filtering on graphs,” Journal of Machine Learning Research, vol. 17, no. 105, pp. 1–41, 2016. [Online]. Available: http://jmlr.org/papers/v17/15-147.html
- G. A. Pavlopoulos, M. Secrier, C. N. Moschopoulos, T. G. Soldatos, S. Kossida, J. Aerts, R. Schneider, and P. G. Bagos, “Using graph theory to analyze biological networks,” BioData Min, vol. 4, p. 10, Apr 2011.
- N. Fan and P. Pardalos, “Multi-way clustering and biclustering by the ratio cut and normalized cut in graphs,” Journal of Combinatorial Optimization, vol. 23, no. 2, pp. 224–251, Sep. 2010. [Online]. Available: https://doi.org/10.1007/s10878-010-9351-5
- M. Mahajan, P. Nimbhorkar, and K. Varadarajan, “The planar k-means problem is np-hard,” in WALCOM: Algorithms and Computation, S. Das and R. Uehara, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2009, pp. 274–285.
- M. Newman, “Finding community structure in networks using the eigenvectors of matrices,” Physical Review E, vol. 74, no. 3, Sep. 2006. [Online]. Available: https://doi.org/10.1103/physreve.74.036104
- M. Udell and A. Townsend, “Why are big data matrices approximately low rank?” SIAM Journal on Mathematics of Data Science, vol. 1, no. 1, pp. 144–160, 2019.
- C. Alpert, A. Kahng, and S.-Z. Yao, “Spectral partitioning with multiple eigenvectors,” Discrete Applied Mathematics, vol. 90, no. 1-3, pp. 3–26, Jan. 1999. [Online]. Available: https://doi.org/10.1016/s0166-218x(98)00083-3
- S. Bornholdt, “Statistical mechanics of community detection,” Physical Review E, vol. 74, no. 1, p. 016110, 2006.
- E. Carlon, “Computational Physics,” KU Leuven, Tech. Rep., 2013.
- W. K. Hastings, “Monte carlo sampling methods using markov chains and their applications,” Biometrika, vol. 57, no. 1, pp. 97–109, 1970. [Online]. Available: http://www.jstor.org/stable/2334940
- M. P. Allen and D. J. Tildesley, “Advanced Monte Carlo methods,” in Computer Simulation of Liquids. Oxford University Press, 06 2017. [Online]. Available: https://doi.org/10.1093/oso/9780198803195.003.0009
- S. Z. Selim and K. Alsultan, “A simulated annealing algorithm for the clustering problem,” Pattern Recognition, vol. 24, no. 10, pp. 1003–1008, 1991. [Online]. Available: https://www.sciencedirect.com/science/article/pii/003132039190097O
- D. S. Johnson, C. R. Aragon, L. A. McGeoch, and C. Schevon, “Optimization by simulated annealing: An experimental evaluation part i, graph partitioning,” Operations Research, vol. 37, no. 6, pp. 865–892, Dec. 1989. [Online]. Available: https://doi.org/10.1287/opre.37.6.865
- P. Talukdar and K. Crammer, “New regularized algorithms for transductive learning,” in Machine Learning and Knowledge Discovery in Databases, 2009, pp. 442–457. [Online]. Available: https://doi.org/10.1007/978-3-642-04174-7_29
- P. Talukdar and F. Pereira, “Experiments in graph-based semi-supervised learning methods for class-instance acquisition,” in Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, Uppsala, Sweden, Jul. 2010, pp. 1473–1481. [Online]. Available: https://aclanthology.org/P10-1149
- R. A. Rossi and N. K. Ahmed, “The network data repository with interactive graph analytics and visualization,” in Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, ser. AAAI’15. AAAI Press, 2015, p. 4292–4293.
- D. Dua and C. Graff, “UCI Machine Learning Repository,” 2017. [Online]. Available: http://archive.ics.uci.edu/ml