Papers
Topics
Authors
Recent
Search
2000 character limit reached

MSTAR: Multi-Scale Backbone Architecture Search for Timeseries Classification

Published 21 Feb 2024 in cs.CV | (2402.13822v1)

Abstract: Most of the previous approaches to Time Series Classification (TSC) highlight the significance of receptive fields and frequencies while overlooking the time resolution. Hence, unavoidably suffered from scalability issues as they integrated an extensive range of receptive fields into classification models. Other methods, while having a better adaptation for large datasets, require manual design and yet not being able to reach the optimal architecture due to the uniqueness of each dataset. We overcome these challenges by proposing a novel multi-scale search space and a framework for Neural architecture search (NAS), which addresses both the problem of frequency and time resolution, discovering the suitable scale for a specific dataset. We further show that our model can serve as a backbone to employ a powerful Transformer module with both untrained and pre-trained weights. Our search space reaches the state-of-the-art performance on four datasets on four different domains while introducing more than ten highly fine-tuned models for each data.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (56)
  1. Explainable ai decision model for ecg data of cardiac disorders. Biomedical Signal Processing and Control, 75:103584, 2022.
  2. Differential evolution for neural architecture search, 2021.
  3. The UEA multivariate time series classification archive, 2018. CoRR, abs/1811.00075, 2018.
  4. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling, 2018.
  5. Smash: One-shot model architecture search through hypernetworks, 2017.
  6. Incepse: Leveraging inceptiontime’s performance with squeeze and excitaion mechanism in ecg analysis. In Proceedings of the 12th International Symposium on Information and Communication Technology, SOICT ’23, page 578–584, New York, NY, USA, 2023. Association for Computing Machinery.
  7. A novel evolutionary algorithm for hierarchical neural architecture search, 2023.
  8. The UCR time series archive. CoRR, abs/1810.07758, 2018.
  9. Rocket: exceptionally fast and accurate time series classification using random convolutional kernels. Data Mining and Knowledge Discovery, 34(5):1454–1495, July 2020.
  10. Minirocket: A very fast (almost) deterministic transform for time series classification. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, KDD ’21, page 248–257, New York, NY, USA, 2021. Association for Computing Machinery.
  11. Hydra: Competing convolutional kernels for fast and accurate time series classification. Data Min. Knowl. Discov., 37(5):1779–1805, may 2023.
  12. Xuanyi Dong and Yi Yang. Nas-bench-201: Extending the scope of reproducible neural architecture search, 2020.
  13. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
  14. Neural architecture search: a survey. J. Mach. Learn. Res., 20(1):1997–2017, jan 2019.
  15. Deep learning for time series classification and extrinsic regression: A current survey, 2023.
  16. Peter I. Frazier. A tutorial on bayesian optimization, 2018.
  17. Deep residual learning for image recognition. CoRR, abs/1512.03385, 2015.
  18. Bag of tricks for image classification with convolutional neural networks. CoRR, abs/1812.01187, 2018.
  19. Squeeze-and-excitation networks, 2019.
  20. Inceptiontime: Finding alexnet for time series classification. Data Min. Knowl. Discov., 34(6):1936–1962, nov 2020.
  21. Eegeyenet: a simultaneous electroencephalography and eye-tracking dataset and benchmark for eye movement prediction, 2021.
  22. Imagenet classification with deep convolutional neural networks. Commun. ACM, 60(6):84–90, may 2017.
  23. Application of continuous wavelet transform and convolutional neural network in decoding motor imagery brain-computer interface. Entropy, 21(12):1199, 2019.
  24. Lightning-AI. Pytorch lightning, 2024.
  25. Darts: Differentiable architecture search, 2019.
  26. Hive-cote 2.0: A new meta ensemble for time series classification. Mach. Learn., 110(11–12):3211–3243, dec 2021.
  27. Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors, 16(1), 2016.
  28. Satellite image time series analysis under time warping. IEEE Transactions on Geoscience and Remote Sensing, 50(8):3081–3095, 2012.
  29. Efficient neural architecture search via parameter sharing, 2018.
  30. Xceptiontime: Independent time-window xceptiontime architecture for hand gesture classification. In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1304–1308, 2020.
  31. Human activity recognition using smartphones. UCI Machine Learning Repository, 2012.
  32. Patrick Schäfer. The boss is concerned with time series classification in the presence of noise. Data Min. Knowl. Discov., 29(6):1505–1530, nov 2015.
  33. TS-CHIEF: A scalable and accurate forest algorithm for time series classification. CoRR, abs/1906.10329, 2019.
  34. Steven W. Smith. The scientist and engineer’s guide to digital signal processing. California Technical Publishing, USA, 1997.
  35. Deep learning for ecg analysis: Benchmarks and insights from ptb-xl. IEEE Journal of Biomedical and Health Informatics, 25(5):1519–1528, 2021.
  36. Axiomatic attribution for deep networks, 2017.
  37. Rethinking the inception architecture for computer vision. CoRR, abs/1512.00567, 2015.
  38. Multirocket: Multiple pooling operators and transformations for fast and effective time series classification. Data Min. Knowl. Discov., 36(5):1623–1646, sep 2022.
  39. Mnasnet: Platform-aware neural architecture search for mobile, 2019.
  40. Omni-scale CNNs: a simple and effective kernel size configuration for time series classification. In International Conference on Learning Representations, 2022.
  41. Frequency content and characteristics of ventricular conduction. Journal of Electrocardiology, 48(6):933–937, 2015.
  42. NAS-bench-360: Benchmarking neural architecture search on diverse tasks. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022.
  43. Attention is all you need, 2023.
  44. Ptb-xl, a large publicly available electrocardiography dataset. Scientific data, 7(1):154, 2020.
  45. Automatic ecg classification using continuous wavelet transform and convolutional neural network. Entropy, 23(1):119, 2021.
  46. Time series classification from scratch with deep neural networks: A strong baseline, 2016.
  47. BANANAS: bayesian optimization with neural architectures for neural architecture search. CoRR, abs/1910.11858, 2019.
  48. Neural architecture search: Insights from 1000 papers, 2023.
  49. Wavelet transform time-frequency image and convolutional network-based motor imagery eeg classification. Ieee Access, 7:6084–6093, 2018.
  50. Innohar: A deep neural network for complex human activity recognition. IEEE Access, 7:9893–9902, 2019.
  51. Does unsupervised architecture representation learning help neural architecture search?, 2020.
  52. Vit2eeg: Leveraging hybrid pretrained vision transformers for eeg data, 2023.
  53. Nas-bench-101: Towards reproducible neural architecture search, 2019.
  54. How transferable are features in deep neural networks? In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS’14, page 3320–3328, Cambridge, MA, USA, 2014. MIT Press.
  55. Visualizing and understanding convolutional networks. In David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors, Computer Vision – ECCV 2014, pages 818–833, Cham, 2014. Springer International Publishing.
  56. Neural architecture search with reinforcement learning, 2017.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up