2000 character limit reached
Quaternion recurrent neural network with real-time recurrent learning and maximum correntropy criterion (2402.14227v2)
Published 22 Feb 2024 in cs.LG
Abstract: We develop a robust quaternion recurrent neural network (QRNN) for real-time processing of 3D and 4D data with outliers. This is achieved by combining the real-time recurrent learning (RTRL) algorithm and the maximum correntropy criterion (MCC) as a loss function. While both the mean square error and maximum correntropy criterion are viable cost functions, it is shown that the non-quadratic maximum correntropy loss function is less sensitive to outliers, making it suitable for applications with multidimensional noisy or uncertain data. Both algorithms are derived based on the novel generalised HR (GHR) calculus, which allows for the differentiation of real functions of quaternion variables and offers the product and chain rules, thus enabling elegant and compact derivations. Simulation results in the context of motion prediction of chest internal markers for lung cancer radiotherapy, which includes regular and irregular breathing sequences, support the analysis.
- T. Parcollet, M. Morchid, and G. Linarès, “A survey of quaternion neural networks,” Artificial Intelligence Review, vol. 53, no. 4, p. 2957–2982, 2019.
- X. Zhu, Y. Xu, H. Xu, and C. Chen, “Quaternion convolutional neural networks,” in Proceedings of the European Conference on Computer Vision (ECCV), 2019.
- S. Zhang, Y. Tay, L. Yao, and Q. Liu, “Quaternion knowledge graph embeddings,” in Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), 2019.
- C. J. Gaudet and A. S. Maida, “Deep quaternion networks,” in Proceedings of the International Joint Conference on Neural Networks (IJCNN), 2018, pp. 1–8.
- T. Parcollet, M. Morchid, and G. Linares, “Quaternion convolutional neural networks for heterogeneous image processing,” in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. 8514–8518.
- K. Takahashi, E. Tano, and M. Hashimoto, “Feedforward–feedback controller based on a trained quaternion neural network using a generalised HR calculus with application to trajectory control of a three-link robot manipulator,” Machines, vol. 10, no. 5, p. 333, 2022.
- D. Comminiello, M. Lella, S. Scardapane, and A. Uncini, “Quaternion convolutional neural networks for detection and localization of 3D sound events,” in Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. 8533–8537.
- D. Xu, Y. Xia, and D. P. Mandic, “Optimization in quaternion dynamic systems: Gradient, Hessian, and learning algorithms,” IEEE Transactions on Neural Networks and Learning Systems, vol. 27, no. 2, pp. 249–261, 2016.
- D. Xu, C. Jahanchahi, C. C. Took, and D. P. Mandic, “Enabling quaternion derivatives: The generalized HR calculus,” Royal Society Open Science, vol. 2, no. 8, p. 150255, 2015.
- D. Xu, L. Zhang, and H. Zhang, “Learning algorithms in quaternion neural networks using GHR calculus,” Neural Network World, vol. 27, no. 3, pp. 271–282, 2017.
- C. Popa, “Learning algorithms for quaternion-valued neural networks,” Neural Processing Letters, vol. 47, no. 3, pp. 949–973, 2017.
- R. J. Williams and D. Zipser, “A learning algorithm for continually running fully recurrent neural networks,” Neural Computation, vol. 1, no. 2, pp. 270–280, 1989.
- T. Parcollet, M. Ravanelli, M. Morchid, G. Linarès, C. Trabelsi, R. De Mori, and Y. Bengio, “Quaternion recurrent neural networks,” in Proceedings of the International Conference on Learning Representations (ICLR), 2018.
- I. Santamaria, P. P. Pokharel, and J. C. Principe, “Generalized correlation function: Definition, properties, and application to blind equalization,” IEEE Transactions on Signal Processing, vol. 54, no. 6, pp. 2187–2197, 2006.
- A. Sudbery, “Quaternionic analysis,” Mathematical Proceedings of the Cambridge Philosophical Society, vol. 85, no. 2, pp. 199–225, 1979.
- W. Liu, P. P. Pokharel, and J. C. Principe, “Correntropy: A localized similarity measure,” in Proceedings of the International Joint Conference on Neural Networks (IJCNN), 2006, pp. 4919–4924.
- T. Ogunfunmi and T. Paul, “The quaternion maximum correntropy algorithm,” IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 62, no. 6, pp. 598–602, 2015.
- M. Pohl, M. Uesaka, K. Demachi, and R. B. Chhatkuli, “Prediction of the motion of chest internal points using a recurrent neural network trained with real-time recurrent learning for latency compensation in lung cancer radiotherapy,” Computerized Medical Imaging and Graphics, vol. 91, p. 101941, 2022.
- A. Krauss, S. Nill, and U. Oelfke, “The comparative performance of four respiratory motion predictors for real-time tumour tracking,” Physics in Medicine and Biology, vol. 56, no. 16, p. 5303–5317, 2011.
- T. P. Teo, S. B. Ahmed, P. Kawalec, N. Alayoubi, N. Bruce, E. Lyn, and S. Pistorius, “Feasibility of predicting tumor motion using online data acquired during treatment and a generalized neural network optimized with offline patient tumor trajectories,” Medical Physics, vol. 45, no. 2, p. 830–845, 2018.
- M. Mafi and S. M. Moghadam, “Real-time prediction of tumor motion using a dynamic neural network,” Medical, Biological Engineering, Computing, vol. 58, no. 3, p. 529–539, 2020.
- K. Jiang, F. Fujii, and T. Shiinoki, “Prediction of lung tumor motion using nonlinear autoregressive model with exogenous input,” Physics in Medicine & Biology, vol. 64, no. 21, p. 21NT02, 2019.
- T. Krilavicius, I. Zliobaite, H. Simonavicius, and L. Jarusevicius, “Predicting respiratory motion for real-time tumour tracking in radiotherapy,” in Proceedings of the IEEE 29th International Symposium on Computer-Based Medical Systems (CBMS), 2016.
- M. Pohl, M. Uesaka, H. Takahashi, K. Demachi, and R. Bhusal Chhatkuli, “Prediction of the position of external markers using a recurrent neural network trained with unbiased online recurrent optimization for safe lung cancer radiotherapy,” Computer Methods and Programs in Biomedicine, vol. 222, p. 106908, 2022.
- C. Took and D. Mandic, “The Quaternion LMS algorithm for adaptive filtering of hypercomplex processes,” IEEE Transactions on Signal Processing, vol. 57, no. 4, p. 1316–1327, 2009.
- N. Benvenuto and F. Piazza, “On the complex backpropagation algorithm,” IEEE Transactions on Signal Processing, vol. 40, no. 4, pp. 967–969, 1992.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
