Hypergraph $p$-Laplacian regularization on point clouds for data interpolation (2405.01109v1)
Abstract: As a generalization of graphs, hypergraphs are widely used to model higher-order relations in data. This paper explores the benefit of the hypergraph structure for the interpolation of point cloud data that contain no explicit structural information. We define the $\varepsilon_n$-ball hypergraph and the $k_n$-nearest neighbor hypergraph on a point cloud and study the $p$-Laplacian regularization on the hypergraphs. We prove the variational consistency between the hypergraph $p$-Laplacian regularization and the continuum $p$-Laplacian regularization in a semisupervised setting when the number of points $n$ goes to infinity while the number of labeled points remains fixed. A key improvement compared to the graph case is that the results rely on weaker assumptions on the upper bound of $\varepsilon_n$ and $k_n$. To solve the convex but non-differentiable large-scale optimization problem, we utilize the stochastic primal-dual hybrid gradient algorithm. Numerical experiments on data interpolation verify that the hypergraph $p$-Laplacian regularization outperforms the graph $p$-Laplacian regularization in preventing the development of spikes at the labeled points.
- Hypergraph-based image processing. In 2020 IEEE International Conference on Image Processing (ICIP), pages 216–220. IEEE, 2020.
- Multi-modal knowledge hypergraph for diverse image retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 3376–3383, 2023.
- Hypergraphs and cellular networks. PLoS computational biology, 5(5):e1000385, 2009.
- Predicting protein interactions via parsimonious network history inference. Bioinformatics, 29(13):i237–i246, 2013.
- Social influence maximization in hypergraphs. Entropy, 23(7):796, 2021.
- Hypergraph p𝑝pitalic_p-laplacians, scale spaces, and information flow in networks. In International Conference on Scale Space and Variational Methods in Computer Vision, pages 677–690. Springer, 2023.
- The total variation on hypergraphs-learning on hypergraphs revisited. Advances in Neural Information Processing Systems, 26, 2013.
- Francis Bach et al. Learning with submodular functions: A convex optimization perspective. Foundations and Trends® in machine learning, 6(2-3):145–373, 2013.
- Re-revisiting learning on hypergraphs: confidence interval and subgradient method. In International Conference on Machine Learning, pages 4026–4034. PMLR, 2017.
- Submodular hypergraphs: p𝑝pitalic_p-laplacians, cheeger inequalities and spectral clustering. In International Conference on Machine Learning, pages 3014–3023. PMLR, 2018.
- Hypergcn: A new method for training graph convolutional networks on hypergraphs. Advances in neural information processing systems, 32, 2019.
- Hypergraph learning: Methods and practices. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(5):2548–2566, 2020.
- Learning hypergraph-regularized attribute predictors. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 409–417, 2015.
- Topic-sensitive influencer mining in interest-based social media networks via hypergraph learning. IEEE Transactions on Multimedia, 16(3):796–812, 2014.
- Continuum limit of total variation on point clouds. Archive for rational mechanics and analysis, 220:193–241, 2016.
- Analysis of p𝑝pitalic_p-laplacian regularization in semisupervised learning. SIAM Journal on Mathematical Analysis, 51(3):2085–2120, 2019.
- Sobolev spaces. Elsevier, 2003.
- A first-order primal-dual algorithm for convex problems with applications to imaging. Journal of mathematical imaging and vision, 40:120–145, 2011.
- Stochastic primal-dual hybrid gradient algorithm with arbitrary sampling and imaging applications. SIAM Journal on Optimization, 28(4):2783–2808, 2018.
- Ulrike von Luxburg and Olivier Bousquet. Distance-based classification with lipschitz functions. J. Mach. Learn. Res., 5(Jun):669–695, 2004.
- Algorithms for lipschitz learning on graphs. In Conference on Learning Theory, pages 1190–1223. PMLR, 2015.
- Jeff Calder. Consistency of lipschitz learning with infinite unlabeled data and finite labeled data. SIAM Journal on Mathematics of Data Science, 1(4):780–812, 2019.
- Continuum limit of lipschitz learning on graphs. Foundations of Computational Mathematics, 23(2):393–431, 2023.
- Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of the 20th International conference on Machine learning (ICML-03), pages 912–919, 2003.
- Learning from labeled and unlabeled data on a directed graph. In Proceedings of the 22nd international conference on Machine learning, pages 1036–1043, 2005.
- Learning on graph with Laplacian regularization. Advances in neural information processing systems, 19, 2006.
- Ulrike Von Luxburg. A tutorial on spectral clustering. Statistics and computing, 17:395–416, 2007.
- Saliency detection via graph-based manifold ranking. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3166–3173, 2013.
- Nonlocal linear image regularization and supervised segmentation. Multiscale Modeling & Simulation, 6(2):595–630, 2007.
- Statistical analysis of semi-supervised learning: The limit of infinite unlabelled data. Advances in neural information processing systems, 22, 2009.
- Weighted nonlocal Laplacian on interpolation from sparse data. Journal of Scientific Computing, 73:1164–1177, 2017.
- Properly-weighted graph Laplacian for semi-supervised learning. Applied mathematics & optimization, 82:1111–1159, 2020.
- Poisson learning: Graph based semi-supervised learning at very low label rates. In International Conference on Machine Learning, pages 1306–1316. PMLR, 2020.
- Asymptotic behavior of ℓpsubscriptℓ𝑝\ell_{p}roman_ℓ start_POSTSUBSCRIPT italic_p end_POSTSUBSCRIPT-based laplacian regularization in semi-supervised learning. In Conference on Learning Theory, pages 879–906. PMLR, 2016.
- Regularization on discrete spaces. In Joint Pattern Recognition Symposium, pages 361–368. Springer, 2005.
- On the p𝑝pitalic_p-laplacian and ∞\infty∞-laplacian on graphs with applications in image and data processing. SIAM Journal on Imaging Sciences, 8(4):2412–2451, 2015.
- Nicolas Garcia Trillos. Variational limits of k-nn graph-based functionals on data clouds. SIAM Journal on Mathematics of Data Science, 1(1):93–120, 2019.
- Mathew Penrose. Random geometric graphs, volume 5. OUP Oxford, 2003.
- Gianni Dal Maso. An introduction to ΓΓ\Gammaroman_Γ-convergence, volume 8. Springer Science & Business Media, 2012.
- Andrea Braides. Gamma-convergence for Beginners, volume 22. Clarendon Press, 2002.
- On the rate of convergence of empirical measures in ∞\infty∞-transportation distance. Canadian Journal of Mathematics, 67(6):1358–1383, 2015.
- Discrete p𝑝pitalic_p-bilaplacian operators on graphs. In Image and Signal Processing: 9th International Conference, ICISP 2020, Marrakesh, Morocco, June 4–6, 2020, Proceedings 9, pages 339–347. Springer, 2020.
- On the convergence of stochastic primal-dual hybrid gradient. SIAM Journal on Optimization, 32(2):1288–1318, 2022.
- Laurent Condat. Fast projection onto the simplex and the ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT ball. Mathematical Programming, 158(1-2):575–585, 2016.
- Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
- Vlfeat: An open and portable library of computer vision algorithms. In Proceedings of the 18th ACM international conference on Multimedia, pages 1469–1472, 2010.
- Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004.