Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Time-Series Forecasting and Sequence Learning Using Memristor-based Reservoir System (2405.13347v2)

Published 22 May 2024 in cs.LG, cs.AI, and cs.AR

Abstract: Pushing the frontiers of time-series information processing in the ever-growing domain of edge devices with stringent resources has been impeded by the systems' ability to process information and learn locally on the device. Local processing and learning of time-series information typically demand intensive computations and massive storage as the process involves retrieving information and tuning hundreds of parameters back in time. In this work, we developed a memristor-based echo state network accelerator that features efficient temporal data processing and in-situ online learning. The proposed design is benchmarked using various datasets involving real-world tasks, such as forecasting the load energy consumption and weather conditions. The experimental results illustrate that the hardware model experiences a marginal degradation in performance as compared to the software counterpart. This is mainly attributed to the limited precision and dynamic range of network parameters when emulated using memristor devices. The proposed system is evaluated for lifespan, robustness, and energy-delay product. It is observed that the system demonstrates reasonable robustness for device failure below 10%, which may occur due to stuck-at faults. Furthermore, 247X reduction in energy consumption is achieved when compared to a custom CMOS digital design implemented at the same technology node.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Memristive crossbar memory lifetime evaluation and reconfiguration strategies. IEEE Transactions on Emerging Topics in Computing 6, 2 (2016), 207–218.
  2. Amir F Atiya and Alexander G Parlos. 2000. New results on recurrent network training: unifying the algorithms and accelerating convergence. IEEE transactions on neural networks 11, 3 (2000), 697–709.
  3. Reservoir computing approaches for representation and classification of multivariate time series. IEEE transactions on neural networks and learning systems 32, 5 (2020), 2169–2179.
  4. EchoBay: design and optimization of echo state networks under memory and time constraints. ACM Transactions on Architecture and Code Optimization (TACO) 17, 3 (2020), 1–24.
  5. Towards oxide electronics: a roadmap. Applied surface science 482 (2019), 1–93.
  6. Forecasting chaotic weather variables with echo state networks and a novel swing training approach. Knowledge-Based Systems 269 (2023), 110506.
  7. A Cost-Efficient Digital ESN Architecture on FPGA for OFDM Symbol Detection. ACM Journal on Emerging Technologies in Computing Systems (JETC) 17, 4 (2021), 1–15.
  8. Next generation reservoir computing. Nature communications 12, 1 (2021), 5564.
  9. Reservoir computing meets smart grids: Attack detection using delayed feedback networks. IEEE Transactions on Industrial Informatics 14, 2 (2017), 734–743.
  10. Kentaro Honda and Hakaru Tamukoh. 2020. A hardware-oriented echo state network and its FPGA implementation. Journal of Robotics, Networking and Artificial Life 7, 1 (2020), 58–62.
  11. Herbert Jaeger. 2001. The “echo state” approach to analysing and training recurrent neural networks-with an erratum note. Bonn, Germany: German National Research Center for Information Technology GMD Technical Report 148, 34 (2001), 13.
  12. Integrating memristors and CMOS for better AI. Nature Electronics 2, 9 (2019), 376–377.
  13. A 2.1 TFLOPS/W mobile deep RL accelerator with transposable PE array and experience compression. In 2019 IEEE International Solid-State Circuits Conference-(ISSCC). IEEE, 136–138.
  14. Voltage divider effect for the improvement of variability and endurance of TaO x memristor. Scientific reports 6 (2016), 20085.
  15. Integer echo state networks: efficient reservoir computing for digital hardware. IEEE Transactions on Neural Networks and Learning Systems (2020).
  16. Design and analysis of a neuromemristive reservoir computing architecture for biosignal processing. Frontiers in neuroscience 9 (2016), 502.
  17. A Tuning-Free Hardware Reservoir Based on MOSFET Crossbar Array for Practical Echo State Network Implementation. In 2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC). IEEE, 458–463.
  18. VTEAM: A general model for voltage-controlled memristors. IEEE Transactions on Circuits and Systems II: Express Briefs 62, 8 (2015), 786–790.
  19. Rotating neurons for all-analog implementation of cyclic reservoir computing. Nature communications 13, 1 (2022), 1549.
  20. Mantas Lukoševičius. 2012. A practical guide to applying echo state networks. In Neural networks: Tricks of the trade. Springer, 659–686.
  21. Michael C Mackey and Leon Glass. 1977. Oscillation and chaos in physiological control systems. Science 197, 4300 (1977), 287–289.
  22. A day-ahead photovoltaic power prediction via transfer learning and deep neural networks. Forecasting 5, 1 (2023), 213–228.
  23. Temporal data classification and forecasting using a memristor-based reservoir computing system. Nature Electronics 2, 10 (2019), 480–487.
  24. ESSM: Extended Synaptic Sampling Machine with Stochastic Echo State Neuro-memristive Circuits. IEEE Journal on Emerging and Selected Topics in Circuits and Systems (2023).
  25. Micah O’Halloran and Rahul Sarpeshkar. 2004. A 10-nW 12-bit accurate analog storage cell with 10-aA leakage. IEEE journal of solid-state circuits 39, 11 (2004), 1985–1996.
  26. The era of hyper-scaling in electronics. Nature Electronics 1, 8 (2018), 442–450.
  27. Modelling and implementation of soft bio-mimetic turtle using echo state network and soft pneumatic actuators. Scientific reports 11, 1 (2021), 12076.
  28. Mariia Sorokina. 2020. Multidimensional fiber echo state network analogue. Journal of Physics: Photonics 2, 4 (2020), 044006.
  29. David Sussillo and Larry F Abbott. 2009. Generating coherent patterns of activity from chaotic neural networks. Neuron 63, 4 (2009), 544–557.
  30. Intelligent dual stream CNN and echo state network for anomaly detection. Knowledge-Based Systems 253 (2022), 109456.
  31. Ioannis Vourkas and Georgios Ch Sirakoulis. 2016. Memristor-based nanoelectronic computing circuits and architectures. Vol. 19. Springer.
  32. Impact of program/erase operation on the performances of oxide-based resistive switching memory. Nanoscale research letters 10, 1 (2015), 39.
  33. Stock market index prediction based on reservoir computing models. Expert Systems with Applications 178 (2021), 115022.
  34. Memristor-based echo state network with online least mean square. IEEE Transactions on Systems, Man, and Cybernetics: Systems 49, 9 (2018), 1787–1796.
  35. Neuromorphic applications of a multivalued [S⁢n⁢I4⁢(C6⁢H5)2⁢S⁢O2𝑆𝑛subscript𝐼4subscriptsubscript𝐶6subscript𝐻52𝑆subscript𝑂2SnI_{4}{(C_{6}H_{5})_{2}SO}_{2}italic_S italic_n italic_I start_POSTSUBSCRIPT 4 end_POSTSUBSCRIPT ( italic_C start_POSTSUBSCRIPT 6 end_POSTSUBSCRIPT italic_H start_POSTSUBSCRIPT 5 end_POSTSUBSCRIPT ) start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT italic_S italic_O start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT] memristor incorporated in the echo state machine. ACS Applied Electronic Materials 2, 2 (2020), 329–338.
  36. Optimized near-zero quantization method for flexible memristor based neural network. IEEE Access 6 (2018), 29320–29331.
  37. Chris Yakopcic. 2014. Memristor device modeling and circuit design for read out integrated circuits, memory architectures, and neuromorphic systems. University of Dayton.
  38. High switching endurance in TaO x memristive devices. Applied Physics Letters 97, 23 (2010), 232102.
  39. Heng Zhang and Danilo Vasconcellos Vargas. 2023. A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning. IEEE Access (2023).
  40. Reservoir Network With Structural Plasticity for Human Activity Recognition. IEEE Transactions on Emerging Topics in Computational Intelligence (2023), 1–11. https://doi.org/10.1109/TETCI.2023.3330422
  41. Neuromorphic system for spatial and temporal information processing. IEEE Trans. Comput. 69, 8 (2020), 1099–1112.
  42. Abdullah M Zyarah and Dhireesha Kudithipudi. 2018. Semi-trained memristive crossbar computing engine with in situ learning accelerator. ACM Journal on Emerging Technologies in Computing Systems (JETC) 14, 4 (2018), 1–16.
  43. Abdullah M Zyarah and Dhireesha Kudithipudi. 2019a. Neuromemrisitive architecture of HTM with on-device learning and neurogenesis. ACM Journal on Emerging Technologies in Computing Systems (JETC) 15, 3 (2019), 1–24.
  44. Abdullah M Zyarah and Dhireesha Kudithipudi. 2019b. Neuromemristive multi-layer random projection network with on-device learning. In 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–8.
  45. Ziksa: On-chip learning accelerator with memristor crossbars for multilevel neural networks. In 2017 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, 1–4.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets