Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
11 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
3 tokens/sec
DeepSeek R1 via Azure Pro
33 tokens/sec
2000 character limit reached

Inhomogeneous graph trend filtering via a l2,0 cardinality penalty (2304.05223v4)

Published 11 Apr 2023 in cs.LG, cs.SI, and stat.ML

Abstract: We study estimation of piecewise smooth signals over a graph. We propose a $\ell_{2,0}$-norm penalized Graph Trend Filtering (GTF) model to estimate piecewise smooth graph signals that exhibit inhomogeneous levels of smoothness across the nodes. We prove that the proposed GTF model is simultaneously a k-means clustering on the signal over the nodes and a minimum graph cut on the edges of the graph, where the clustering and the cut share the same assignment matrix. We propose two methods to solve the proposed GTF model: a spectral decomposition method and a method based on simulated annealing. In the experiment on synthetic and real-world datasets, we show that the proposed GTF model has a better performances compared with existing approaches on the tasks of denoising, support recovery and semi-supervised classification. We also show that the proposed GTF model can be solved more efficiently than existing models for the dataset with a large edge set.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (29)
  1. S. Chen, R. Varma, A. Singh, and J. Kovacevic, “Signal representations on graphs: Tools and applications,” CoRR, vol. abs/1512.05406, 2015. [Online]. Available: http://arxiv.org/abs/1512.05406
  2. S. Chen, R. Varma, A. Sandryhaila, and J. Kovačević, “Discrete signal processing on graphs: Sampling theory,” IEEE Transactions on Signal Processing, vol. 63, no. 24, pp. 6510–6523, 2015.
  3. S. Chen, A. Sandryhaila, J. M. F. Moura, and J. Kovačević, “Signal recovery on graphs: Variation minimization,” IEEE Transactions on Signal Processing, vol. 63, no. 17, pp. 4609–4624, 2015.
  4. D. Romero, M. Ma, and G. B. Giannakis, “Kernel-based reconstruction of graph signals,” IEEE Transactions on Signal Processing, vol. 65, no. 3, pp. 764–778, 2017.
  5. A. Elmoataz, O. Lezoray, and S. Bougleux, “Nonlocal discrete regularization on weighted graphs: A framework for image and manifold processing,” IEEE Transactions on Image Processing, vol. 17, no. 7, pp. 1047–1060, 2008.
  6. D. I. Shuman, S. K. Narang, P. Frossard, A. Ortega, and P. Vandergheynst, “The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains,” IEEE Signal Processing Magazine, vol. 30, no. 3, pp. 83–98, 2013.
  7. A. Ortega, P. Frossard, J. Kovačević, J. M. F. Moura, and P. Vandergheynst, “Graph signal processing: Overview, challenges, and applications,” Proceedings of the IEEE, vol. 106, no. 5, pp. 808–828, 2018.
  8. M. Belkin, I. Matveeva, and P. Niyogi, “Regularization and semi-supervised learning on large graphs,” in Learning Theory, J. Shawe-Taylor and Y. Singer, Eds.   Berlin, Heidelberg: Springer Berlin Heidelberg, 2004, pp. 624–638.
  9. S. Bougleux, A. Elmoataz, and M. Melkemi, “Discrete regularization on weighted graphs for image and mesh filtering,” in Scale Space and Variational Methods in Computer Vision, F. Sgallari, A. Murli, and N. Paragios, Eds.   Berlin, Heidelberg: Springer Berlin Heidelberg, 2007, pp. 128–139.
  10. D. A. Spielman, “Spectral graph theory and its applications,” in Proceedings of the 48th Annual IEEE Symposium on Foundations of Computer Science, ser. FOCS ’07.   USA: IEEE Computer Society, 2007, p. 29–38. [Online]. Available: https://doi.org/10.1109/FOCS.2007.66
  11. X. Zhu, Z. Ghahramani, and J. Lafferty, “Semi-supervised learning using gaussian fields and harmonic functions,” in Proceedings of the Twentieth International Conference on International Conference on Machine Learning, ser. ICML’03.   AAAI Press, 2003, p. 912–919.
  12. R. Varma, H. Lee, J. Kovačević, and Y. Chi, “Vector-valued graph trend filtering with non-convex penalties,” IEEE Transactions on Signal and Information Processing over Networks, vol. 6, pp. 48–62, 2020.
  13. Y. Wang, J. Sharpnack, A. J. Smola, and R. J. Tibshirani, “Trend filtering on graphs,” Journal of Machine Learning Research, vol. 17, no. 105, pp. 1–41, 2016. [Online]. Available: http://jmlr.org/papers/v17/15-147.html
  14. G. A. Pavlopoulos, M. Secrier, C. N. Moschopoulos, T. G. Soldatos, S. Kossida, J. Aerts, R. Schneider, and P. G. Bagos, “Using graph theory to analyze biological networks,” BioData Min, vol. 4, p. 10, Apr 2011.
  15. N. Fan and P. Pardalos, “Multi-way clustering and biclustering by the ratio cut and normalized cut in graphs,” Journal of Combinatorial Optimization, vol. 23, no. 2, pp. 224–251, Sep. 2010. [Online]. Available: https://doi.org/10.1007/s10878-010-9351-5
  16. M. Mahajan, P. Nimbhorkar, and K. Varadarajan, “The planar k-means problem is np-hard,” in WALCOM: Algorithms and Computation, S. Das and R. Uehara, Eds.   Berlin, Heidelberg: Springer Berlin Heidelberg, 2009, pp. 274–285.
  17. M. Newman, “Finding community structure in networks using the eigenvectors of matrices,” Physical Review E, vol. 74, no. 3, Sep. 2006. [Online]. Available: https://doi.org/10.1103/physreve.74.036104
  18. M. Udell and A. Townsend, “Why are big data matrices approximately low rank?” SIAM Journal on Mathematics of Data Science, vol. 1, no. 1, pp. 144–160, 2019.
  19. C. Alpert, A. Kahng, and S.-Z. Yao, “Spectral partitioning with multiple eigenvectors,” Discrete Applied Mathematics, vol. 90, no. 1-3, pp. 3–26, Jan. 1999. [Online]. Available: https://doi.org/10.1016/s0166-218x(98)00083-3
  20. S. Bornholdt, “Statistical mechanics of community detection,” Physical Review E, vol. 74, no. 1, p. 016110, 2006.
  21. E. Carlon, “Computational Physics,” KU Leuven, Tech. Rep., 2013.
  22. W. K. Hastings, “Monte carlo sampling methods using markov chains and their applications,” Biometrika, vol. 57, no. 1, pp. 97–109, 1970. [Online]. Available: http://www.jstor.org/stable/2334940
  23. M. P. Allen and D. J. Tildesley, “Advanced Monte Carlo methods,” in Computer Simulation of Liquids.   Oxford University Press, 06 2017. [Online]. Available: https://doi.org/10.1093/oso/9780198803195.003.0009
  24. S. Z. Selim and K. Alsultan, “A simulated annealing algorithm for the clustering problem,” Pattern Recognition, vol. 24, no. 10, pp. 1003–1008, 1991. [Online]. Available: https://www.sciencedirect.com/science/article/pii/003132039190097O
  25. D. S. Johnson, C. R. Aragon, L. A. McGeoch, and C. Schevon, “Optimization by simulated annealing: An experimental evaluation part i, graph partitioning,” Operations Research, vol. 37, no. 6, pp. 865–892, Dec. 1989. [Online]. Available: https://doi.org/10.1287/opre.37.6.865
  26. P. Talukdar and K. Crammer, “New regularized algorithms for transductive learning,” in Machine Learning and Knowledge Discovery in Databases, 2009, pp. 442–457. [Online]. Available: https://doi.org/10.1007/978-3-642-04174-7_29
  27. P. Talukdar and F. Pereira, “Experiments in graph-based semi-supervised learning methods for class-instance acquisition,” in Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, Uppsala, Sweden, Jul. 2010, pp. 1473–1481. [Online]. Available: https://aclanthology.org/P10-1149
  28. R. A. Rossi and N. K. Ahmed, “The network data repository with interactive graph analytics and visualization,” in Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, ser. AAAI’15.   AAAI Press, 2015, p. 4292–4293.
  29. D. Dua and C. Graff, “UCI Machine Learning Repository,” 2017. [Online]. Available: http://archive.ics.uci.edu/ml
Citations (1)

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets