Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

An inclusive review on deep learning techniques and their scope in handwriting recognition (2404.08011v1)

Published 10 Apr 2024 in cs.CV and cs.LG

Abstract: Deep learning expresses a category of machine learning algorithms that have the capability to combine raw inputs into intermediate features layers. These deep learning algorithms have demonstrated great results in different fields. Deep learning has particularly witnessed for a great achievement of human level performance across a number of domains in computer vision and pattern recognition. For the achievement of state-of-the-art performances in diverse domains, the deep learning used different architectures and these architectures used activation functions to perform various computations between hidden and output layers of any architecture. This paper presents a survey on the existing studies of deep learning in handwriting recognition field. Even though the recent progress indicates that the deep learning methods has provided valuable means for speeding up or proving accurate results in handwriting recognition, but following from the extensive literature survey, the present study finds that the deep learning has yet to revolutionize more and has to resolve many of the most pressing challenges in this field, but promising advances have been made on the prior state of the art. Additionally, an inadequate availability of labelled data to train presents problems in this domain. Nevertheless, the present handwriting recognition survey foresees deep learning enabling changes at both bench and bedside with the potential to transform several domains as image processing, speech recognition, computer vision, machine translation, robotics and control, medical imaging, medical information processing, bio-informatics, natural language processing, cyber security, and many others.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (121)
  1. Alan K. Mackworth David L. Poole. Artificial Intelligence: Foundations of Computational Agents. Cambridge University Press, 2010.
  2. Shivashankar B. Nair Elaine Rich, Kevin Knight. Artificial Intelligence. Tata McGraw Hill, 2010.
  3. Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education, 16(1):39, Oct 2019. ISSN 2365-9440. doi:10.1186/s41239-019-0171-0. URL https://doi.org/10.1186/s41239-019-0171-0.
  4. Artificial intelligence in education: A review. IEEE Access, 8:75264–75278, 2020. doi:10.1109/ACCESS.2020.2988510.
  5. Nils J. Nilsson. The Quest for Artificial Intelligence: A history of ideas and achievements. Cambridge University Press, 2010.
  6. Human-robot interaction: A survey. 1(3):203–275, January 2007. ISSN 1551-3955. doi:10.1561/1100000005. URL https://doi.org/10.1561/1100000005.
  7. A. L. Buczak and E. Guven. A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Communications Surveys Tutorials, 18(2):1153–1176, 2016. doi:10.1109/COMST.2015.2494502.
  8. Arash Bahrammirzaee. A comparative survey of artificial intelligence applications in finance: Artificial neural networks, expert system and hybrid intelligent systems. International Journal of Neural Computing and Application, Available online 20 June 2010, 11 2010. doi:10.1007/s00521-010-0362-z.
  9. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1798–1828, 2013. doi:10.1109/TPAMI.2013.50.
  10. Smart technology, artificial intelligence, robotics, and algorithms (stara): Employees’ perceptions of our future workplace. Journal of Management & Organization, 24(2):239–257, 2018. doi:10.1017/jmo.2016.55.
  11. Juan Gustavo Corvalan. Artificial intelligence: Challenges and opportunities-prometea: The first artificial intelligence of latin america at the service of the justice system. Revista de Investigações Constitucionais, 5:295 – 316, 04 2018. ISSN 2359-5639. URL http://www.scielo.br/scielo.php?script=sci_arttext&pid=S2359-56392018000100295&nrm=iso.
  12. Zoubin Ghahramani. Probabilistic machine learning and artificial intelligence. Nature, 521(7553):452–459, May 2015. ISSN 1476-4687. doi:10.1038/nature14541. URL https://doi.org/10.1038/nature14541.
  13. D. Castelvecchi.
  14. Carlton McDonald. Machine learning: a survey of current techniques. Artificial Intelligence Review, 3(4):243–280, Dec 1989. ISSN 1573-7462. doi:10.1007/BF00141197. URL https://doi.org/10.1007/BF00141197.
  15. An overview on application of machine learning techniques in optical networks. IEEE Communications Surveys Tutorials, 21(2):1383–1408, 2019. doi:10.1109/COMST.2018.2880039.
  16. C.M. Bishop. Pattern recognition and machine learning. Springer, 2006.
  17. Semi-supervised learning (1st ed.). The MIT Press, 2006.
  18. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12:2493–2537, 2011.
  19. When does cotraining work in real data? IEEE Transactions on Knowledge and Data Engineering.
  20. Y. Freund and R.E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119–139, 1997.
  21. Unsupervised and semisupervised clustering: A brief survey. In 7th ACM SIGMM international workshop on multimedia information retrieval, 2004.
  22. I. Guyon and A. Elisseeff. An introduction to feature extraction. Feature extraction, page 1–25, 2006.
  23. Deep learning. Nature, 521:436–444, 2015a.
  24. Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12:2825–2830, 2011.
  25. V. Vapnik. Statistical learning theory (Vol. 1). New York: Wiley, 1998.
  26. Convolutional neural networks for speech recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 22(10):1533–1545, 2014.
  27. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In The Conference on Empirical Methods in Natural Language Processing, pages 1724–1734, 2014.
  28. Recurrent convolutional neural networks for speech processing. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 5300–5304, 2017.
  29. WS. McCulloch and W. Pitts. A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys, 5, 1943.
  30. D.O. Hebb. The organization of behavior. New York: Wiley & Sons, 1949.
  31. F. Rossenblatt. The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65(6):386–408, 1958.
  32. B. Windrow and M.E. Hoff. Adaptive switching circuits. IRE WESCON Convention Record, 4(96-104), 1960.
  33. T. Kohonen. Self-organized formation of topologically correct feature maps. Biol. Cybern, 43:59–69, 1982.
  34. J.J. Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the USA, 79(8):2554–2558, 1982.
  35. Learning representations by back-propagating errors. nature, 323:533–536, 1986.
  36. The art of adaptive pattern recognition by a self-organizing neural network. Computer, 21(3):77–88, 1988.
  37. D.S. Broomhead and D. Lowe. Multivariable functional interpolation and adaptive networks. Complex Systems, 2:321–355, 1988.
  38. Kunihiko Fukushima. Neocognitron trained with winner-kill-loser rule. Neural Netw., 23(7):926–938, 2010.
  39. Simon Haykin. Neural Networks: A Comprehensive Foundation. Prentice Hall PTR, 2nd edition, 1998.
  40. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012.
  41. Going deeper with convolutions. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015a.
  42. Deepface: Closing the gap to human-level performance in face verification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1701–1708, 2014.
  43. Best practices for convolutional neural networks applied to visual document analysis. In ICDAR, volume 3, 2003.
  44. Convolutional neural network committees for handwritten character classification. In: Proceedings of the International Conference on Document Analysis and Recognition, pages 1135 – 1139, 2011.
  45. Multi-column deep neural networks for image classification. In 2012 IEEE conference on computer vision and pattern recognition, pages 3642–3649. IEEE, 2012.
  46. End-to-end text recognition with convolutional neural networks. In Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), pages 3304–3308. IEEE, 2012.
  47. Multi-digit number recognition from street view imagery using deep convolutional neural networks. arXiv preprint arXiv:1312.6082, 2013.
  48. Joint training of a convolutional network and a graphical model for human pose estimation. In Advances in neural information processing systems, pages 1799–1807, 2014.
  49. Deep learning. nature, 521(7553):436–444, 2015b.
  50. Deep Learning. MIT Press, 2016. http://www.deeplearningbook.org.
  51. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 5(2):157–166, March 1994. ISSN 1045-9227. doi:10.1109/72.279181.
  52. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, Nov 1998. ISSN 1558-2256. doi:10.1109/5.726791.
  53. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
  54. Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition. In Competition and cooperation in neural nets, pages 267–285. Springer, 1982.
  55. Artificial convolution neural network techniques and applications for lung nodule detection. IEEE transactions on medical imaging, 14(4):711–718, 1995.
  56. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):211–252, 2015.
  57. Long short-term memory recurrent neural network architectures for large scale acoustic modeling. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, pages 338–342, 01 2014.
  58. X. Li and X. Wu. Constructing long short-term memory based deep recurrent neural networks for large vocabulary speech recognition. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4520–4524, April 2015. doi:10.1109/ICASSP.2015.7178826.
  59. A novel connectionist system for unconstrained handwriting recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(5):855–868, May 2009. ISSN 0162-8828. doi:10.1109/TPAMI.2008.137.
  60. Long short-term memory. Neural Comput., 9(8):1735–1780, nov 1997. ISSN 0899-7667. doi:10.1162/neco.1997.9.8.1735. URL http://dx.doi.org/10.1162/neco.1997.9.8.1735.
  61. Recurrent nets that time and count. In Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium, volume 3, pages 189–194. IEEE, 2000.
  62. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014.
  63. M. Schuster and K. K. Paliwal. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11):2673–2681, Nov 1997. ISSN 1053-587X. doi:10.1109/78.650093.
  64. Emily Xiaoxuan Gu. Convolutional neural network based kannada-mnist classification. In 2021 IEEE International Conference on Consumer Electronics and Computer Engineering (ICCECE), pages 180–185, 2021. doi:10.1109/ICCECE51280.2021.9342474.
  65. Cnn-based multilingual handwritten numeral recognition: A fusion-free approach. Expert Systems with Applications, 165:113784, 2021.
  66. Ardis: a swedish historical handwritten digit dataset. Neural Computing and Applications, 32(21):16505–16518, 2020.
  67. Kannada-mnist classification using skip cnn. In 2019 16th International Computer Conference on Wavelet Active Media Technology and Information Processing, pages 245–248, 2019. doi:10.1109/ICCWAMTIP47768.2019.9067521.
  68. A self controlled rdp approach for feature extraction in online handwriting recognition using deep learning. Applied Intelligence, 50(7):2093–2104, 2020.
  69. Vinay Uday Prabhu. Kannada-mnist: A new handwritten digits dataset for the kannada language, 2019.
  70. On developing handwritten character image database for malayalam language script. Engineering Science and Technology, an International Journal, 22(2):637–645, 2019. ISSN 2215-0986. doi:https://doi.org/10.1016/j.jestch.2018.10.011. URL https://www.sciencedirect.com/science/article/pii/S2215098618301447.
  71. Bangla handwritten character recognition using convolutional neural network with data augmentation. In 2019 Joint 8th international conference on informatics, electronics & vision (ICIEV) and 2019 3rd international conference on imaging, vision & pattern recognition (icIVPR), pages 318–323. IEEE, 2019.
  72. Multiobjective optimization for recognition of isolated handwritten indic scripts. Pattern Recognition Letters, 128:318–325, 2019.
  73. Feature map reduction in cnn for handwritten digit recognition. In Recent Developments in Machine Learning and Data Analytics, pages 143–148. Springer, 2019.
  74. Handwriting recognition using deep learning in keras. In 2018 International Conference on Advances in Computing, Communication Control and Networking (ICACCCN), pages 142–145, 2018. doi:10.1109/ICACCCN.2018.8748540.
  75. Integrating scattering feature maps with convolutional neural networks for malayalam handwritten character recognition. International Journal on Document Analysis and Recognition (IJDAR), 21(3):187–198, 2018.
  76. Convolve, attend and spell: An attention-based sequence-to-sequence model for handwritten word recognition. In German Conference on Pattern Recognition, pages 459–472. Springer, 2018.
  77. A multi-scale deep quad tree based feature extraction method for the recognition of isolated handwritten characters of popular indic scripts. Pattern Recognition, 71:78–93, 2017. ISSN 0031-3203. doi:https://doi.org/10.1016/j.patcog.2017.05.022. URL https://www.sciencedirect.com/science/article/pii/S0031320317302200.
  78. Cnn-n-gram for handwriting word recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
  79. Handwritten hangul recognition using deep convolutional neural networks. International Journal on Document Analysis and Recognition (IJDAR), 18:1–13, 2015. doi:10.1007/s10032-014-0229-4.
  80. Regularization of neural networks using dropconnect. In International conference on machine learning, pages 1058–1066. PMLR, 2013.
  81. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012.
  82. Dropout improves recurrent neural networks for handwriting recognition. In 2014 14th international conference on frontiers in handwriting recognition, pages 285–290. IEEE, 2014.
  83. Fast and robust training of recurrent neural networks for offline handwriting recognition. In 2014 14th International Conference on Frontiers in Handwriting Recognition, pages 279–284. IEEE, 2014.
  84. Francois Chollet et al. Keras, 2015. URL https://github.com/fchollet/keras.
  85. Feature set evaluation for offline handwriting recognition systems: Application to the recurrent neural network model. IEEE Transactions on Cybernetics, 46(12):2825–2836, Dec 2016. ISSN 2168-2275. doi:10.1109/TCYB.2015.2490165.
  86. Offline handwriting recognition using lstm recurrent neural networks. In The 28th Benelux conference on artificial intelligence, 2016.
  87. Data augmentation for recognition of handwritten words and lines using a cnn-lstm network. In 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), volume 01, pages 639–645, 2017. doi:10.1109/ICDAR.2017.110.
  88. Improving cnn-rnn hybrid networks for handwriting recognition. In 2018 16th international conference on frontiers in handwriting recognition (ICFHR), pages 80–85. IEEE, 2018.
  89. Word spotting and recognition using deep embedding. In 2018 13th IAPR International Workshop on Document Analysis Systems (DAS), pages 1–6. IEEE, 2018.
  90. Offline continuous handwriting recognition using sequence to sequence neural networks. Neurocomputing, 289:119–128, 2018. ISSN 0925-2312. doi:https://doi.org/10.1016/j.neucom.2018.02.008. URL https://www.sciencedirect.com/science/article/pii/S0925231218301371.
  91. Rnn based online handwritten word recognition in devanagari and bengali scripts using horizontal zoning. Pattern Recognition, 92:203–218, 2019. ISSN 0031-3203. doi:https://doi.org/10.1016/j.patcog.2019.03.030. URL https://www.sciencedirect.com/science/article/pii/S0031320319301384.
  92. Effective offline handwritten text recognition model based on a sequence-to-sequence approach with cnn–rnn networks. Neural Computing and Applications, 33(17):10923–10934, 2021.
  93. A clockwork rnn. In International Conference on Machine Learning, pages 1863–1871. PMLR, 2014.
  94. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1–9, 2015b.
  95. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026–1034, 2015.
  96. Hyperspectral image enhancement and mixture deep-learning classification of corneal epithelium injuries. Sensors, 17(11):2644, 2017.
  97. Deep convolutional neural networks for large-scale speech tasks. Neural networks, 64:39–48, 2015.
  98. Speech recognition with deep recurrent neural networks. In 2013 IEEE international conference on acoustics, speech and signal processing, pages 6645–6649. Ieee, 2013.
  99. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE transactions on pattern analysis and machine intelligence, 39(12):2481–2495, 2017.
  100. Learning to segment every thing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4233–4241, 2018.
  101. Grammar as a foreign language. arXiv preprint arXiv:1412.7449, 2014.
  102. Deep learning in machine translation. In Deep Learning in Natural Language Processing, pages 147–183. Springer, 2018.
  103. Andrej Karpathy and Li Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3128–3137, 2015.
  104. Recurrent convolutional neural networks for scene labeling. In International conference on machine learning, pages 82–90. PMLR, 2014.
  105. A deep hybrid model for weather forecasting. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 379–386, 2015.
  106. Forecasting the weather of nevada: A deep learning approach. In 2015 international joint conference on neural networks (IJCNN), pages 1–6. IEEE, 2015.
  107. Aggnet: deep learning from crowds for mitosis detection in breast cancer histology images. IEEE transactions on medical imaging, 35(5):1313–1321, 2016.
  108. Deep learning for identifying metastatic breast cancer. arXiv preprint arXiv:1606.05718, 2016.
  109. A deep learning architecture for image representation, visual interpretability and automated basal-cell carcinoma cancer detection. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 403–410. Springer, 2013.
  110. Object recognition and detection with deep learning for autonomous driving applications. Simulation, 93(9):759–769, 2017.
  111. Deepdriving: Learning affordance for direct perception in autonomous driving. In Proceedings of the IEEE international conference on computer vision, pages 2722–2730, 2015.
  112. Achieving human parity in conversational speech recognition, 2017.
  113. English conversational telephone speech recognition by humans and machines, 2017.
  114. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Jama, 316(22):2402–2410, 2016.
  115. Dermatologist-level classification of skin cancer with deep neural networks. nature, 542(7639):115–118, 2017.
  116. Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology, 284(2):574–582, 2017.
  117. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
  118. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
  119. Big data deep learning: challenges and perspectives. IEEE access, 2:514–525, 2014.
  120. Deep learning applications and challenges in big data analytics. Journal of big data, 2(1):1–21, 2015.
  121. Theano: Deep learning on gpus with python. In NIPS 2011, BigLearning Workshop, Granada, Spain, volume 3, pages 1–48. Citeseer, 2011.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Sukhdeep Singh (52 papers)
  2. Sudhir Rohilla (1 paper)
  3. Anuj Sharma (63 papers)