Non Commutative Convolutional Signal Models in Neural Networks: Stability to Small Deformations (2310.03879v1)
Abstract: In this paper we discuss the results recently published in~[1] about algebraic signal models (ASMs) based on non commutative algebras and their use in convolutional neural networks. Relying on the general tools from algebraic signal processing (ASP), we study the filtering and stability properties of non commutative convolutional filters. We show how non commutative filters can be stable to small perturbations on the space of operators. We also show that although the spectral components of the Fourier representation in a non commutative signal model are associated to spaces of dimension larger than one, there is a trade-off between stability and selectivity similar to that observed for commutative models. Our results have direct implications for group neural networks, multigraph neural networks and quaternion neural networks, among other non commutative architectures. We conclude by corroborating these results through numerical experiments.
- Convolutional filters and neural networks with noncommutative algebras. IEEE Transactions on Signal Processing, 71:2683–2698, 2023.
- A. Parada-Mayorga and A. Ribeiro. Algebraic neural networks: Stability to deformations. IEEE Transactions on Signal Processing, 69:3351–3366, 2021.
- A. Parada-Mayorga and A. Ribeiro. Stability of algebraic neural networks to small perturbations. In ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5205–5209, 2021.
- S. Mallat. Group invariant scattering. Communications on Pure and Applied Mathematics, 65(10):1331–1398, 2012.
- Stability properties of graph neural networks. IEEE Transactions on Signal Processing, 68:5680–5695, 2020.
- Stability of aggregation graph neural networks. arXiv preprint arXiv:2207.03678, 2022.
- Graphon pooling in graph neural networks. In 2020 28th European Signal Processing Conference (EUSIPCO), pages 860–864, 2021.
- Graphon pooling for reducing dimensionality of signals and convolutional operators on graphs. arXiv preprint arXiv:2212.08171, 2022.
- Convolutional learning on multigraphs. IEEE Transactions on Signal Processing, 71:933–946, 2023.
- Learning with multigraph convolutional filters. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5, 2023.
- Lie group algebra convolutional filters. arXiv preprint arXiv:2305.04431, 2023.
- Algebraic convolutional filters on lie group algebras. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5, 2023.
- Quaternion convolutional neural networks for heterogeneous image processing. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8514–8518. IEEE, 2019.
- Quiver signal processing (qsp). ArXiv, abs/2010.11525, 2020.
- M. Püschel and J. M. F. Moura. Algebraic signal processing theory. ArXiv, abs/cs/0612077, 2006.
- M. Püschel and J. M. F. Moura. Algebraic signal processing theory: Foundation and 1-d time. IEEE Transactions on Signal Processing, 56(8):3572–3585, Aug 2008.
- M. Püschel and J. M. F. Moura. Algebraic signal processing theory: 1-d space. IEEE Transactions on Signal Processing, 56(8):3586–3599, Aug 2008.
- J. Kovacevic and M. Püschel. Algebraic signal processing theory: Sampling for infinite and finite 1-d space. IEEE Transactions on Signal Processing, 58(1):242–257, Jan 2010.
- A. Sandryhaila and J. M. F. Moura. Discrete signal processing on graphs. IEEE Transactions on Signal Processing, 61(7):1644–1656, April 2013.
- M. Püschel and C. Wendler. Discrete signal processing with set functions. IEEE Transactions on Signal Processing, 69:1039–1053, 2021.
- Discrete signal processing on meet/join lattices. IEEE Transactions on Signal Processing, 69:3571–3584, 2021.
- M. Lorenz. A Tour of Representation Theory. Graduate studies in mathematics. American Mathematical Society, 2018.
- Introduction to Representation Theory. Student mathematical library. American Mathematical Society, 2011.
- Eigentaste: A constant time collaborative filtering algorithm. information retrieval, 4:133–151, 2001.