Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
166 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Dataset Condensation for Time Series Classification via Dual Domain Matching (2403.07245v3)

Published 12 Mar 2024 in cs.LG

Abstract: Time series data has been demonstrated to be crucial in various research fields. The management of large quantities of time series data presents challenges in terms of deep learning tasks, particularly for training a deep neural network. Recently, a technique named \textit{Dataset Condensation} has emerged as a solution to this problem. This technique generates a smaller synthetic dataset that has comparable performance to the full real dataset in downstream tasks such as classification. However, previous methods are primarily designed for image and graph datasets, and directly adapting them to the time series dataset leads to suboptimal performance due to their inability to effectively leverage the rich information inherent in time series data, particularly in the frequency domain. In this paper, we propose a novel framework named Dataset \textit{\textbf{Cond}}ensation for \textit{\textbf{T}}ime \textit{\textbf{S}}eries \textit{\textbf{C}}lassification via Dual Domain Matching (\textbf{CondTSC}) which focuses on the time series classification dataset condensation task. Different from previous methods, our proposed framework aims to generate a condensed dataset that matches the surrogate objectives in both the time and frequency domains. Specifically, CondTSC incorporates multi-view data augmentation, dual domain training, and dual surrogate objectives to enhance the dataset condensation process in the time and frequency domains. Through extensive experiments, we demonstrate the effectiveness of our proposed framework, which outperforms other baselines and learns a condensed synthetic dataset that exhibits desirable characteristics such as conforming to the distribution of the original data.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. Automatic Early Detection of Amyotrophic Lateral Sclerosis from Intelligible Speech Using Convolutional Neural Networks.. In Interspeech. 1913–1917.
  2. A public domain dataset for human activity recognition using smartphones.. In Esann, Vol. 3. 3.
  3. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018).
  4. Dataset distillation by matching training trajectories. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4750–4759.
  5. Flying insect classification with inexpensive sensors. Journal of insect behavior 27 (2014), 657–677.
  6. Financial time series forecasting with multi-modality graph neural network. Pattern Recognition 121 (2022), 108218.
  7. Giacomo Chiarot and Claudio Silvestri. 2023. Time series compression survey. Comput. Surveys 55, 10 (2023), 1–32.
  8. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. arXiv:1412.3555 [cs.NE]
  9. Scaling up dataset distillation to imagenet-1k with constant memory. arXiv preprint arXiv:2211.10586 (2022).
  10. Multi-scale convolutional neural networks for time series classification. arXiv preprint arXiv:1603.06995 (2016).
  11. The UCR time series archive. IEEE/CAA Journal of Automatica Sinica 6, 6 (2019), 1293–1305.
  12. Zhiwei Deng and Olga Russakovsky. 2022. Remember the past: Distilling datasets into addressable memories for neural networks. Advances in Neural Information Processing Systems 35 (2022), 34391–34404.
  13. Minimizing the accumulated trajectory error to improve dataset distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3749–3758.
  14. Neural Architecture Search: A Survey. Journal of Machine Learning Research 20, 55 (2019), 1–21. http://jmlr.org/papers/v20/18-598.html
  15. Neural architecture search: A survey. The Journal of Machine Learning Research 20, 1 (2019), 1997–2017.
  16. PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. circulation 101, 23 (2000), e215–e220.
  17. A kernel two-sample test. The Journal of Machine Learning Research 13, 1 (2012), 723–773.
  18. Continual learning for multivariate time series tasks with variable input dimensions. In 2021 IEEE International Conference on Data Mining (ICDM). IEEE, 161–170.
  19. Multitask learning and benchmarking with clinical time series data. Scientific Data 6, 1 (jun 2019). https://doi.org/10.1038/s41597-019-0103-9
  20. MedFuse: Multi-modal fusion with clinical time-series data and chest X-ray images. In Machine Learning for Healthcare Conference. PMLR, 479–503.
  21. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
  22. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780.
  23. Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv:1502.03167 [cs.LG]
  24. Condensing graphs via one-step gradient matching. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 720–730.
  25. Graph condensation for graph neural networks. arXiv preprint arXiv:2110.07580 (2021).
  26. Dataset condensation via efficient synthetic-data parameterization. In International Conference on Machine Learning. PMLR, 11102–11118.
  27. Machine learning techniques and data for stock market forecasting: A literature review. Expert Systems with Applications 197 (2022), 116659.
  28. Dataset condensation with latent space knowledge factorization and sharing. arXiv preprint arXiv:2208.10494 (2022).
  29. Dataset condensation with contrastive signals. In International Conference on Machine Learning. PMLR, 12352–12364.
  30. Condition monitoring of bearing damage in electromechanical drive systems by using motor current signals of electric motors: A benchmark data set for data-driven classification. In PHM Society European Conference, Vol. 3.
  31. Dataset distillation using parameter pruning. arXiv preprint arXiv:2209.14609 (2022).
  32. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint arXiv:1707.01926 (2017).
  33. Graph condensation via receptive field distribution matching. arXiv preprint arXiv:2206.13697 (2022).
  34. Dataset distillation via factorization. Advances in Neural Information Processing Systems 35 (2022), 1100–1113.
  35. FDTI: Fine-grained Deep Traffic Inference with Roadnet-enriched Graph. arXiv preprint arXiv:2306.10945 (2023).
  36. Cross-city Few-Shot Traffic Forecasting via Traffic Pattern Bank. arXiv preprint arXiv:2308.09727 (2023).
  37. Multi-scale Traffic Pattern Bank for Cross-city Few-shot Traffic Forecasting. arXiv preprint arXiv:2402.00397 (2024).
  38. Dataset Distillation with Convexified Implicit Gradients. arXiv preprint arXiv:2302.06755 (2023).
  39. TRISTAN: Real-time analytics on massive time series using sparse dictionary compression. In 2014 IEEE International Conference on Big Data (Big Data). IEEE, 291–300.
  40. Hussein Sh Mogahed and Alexey G Yakunin. 2018. Development of a lossless data compression algorithm for multichannel environmental monitoring systems. In 2018 XIV International Scientific-Technical Conference on Actual Problems of Electronics Instrument Engineering (APEIE). IEEE, 483–486.
  41. Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML-10). 807–814.
  42. Dataset meta-learning from kernel ridge-regression. arXiv preprint arXiv:2011.00050 (2020).
  43. Dataset distillation with infinitely wide convolutional networks. Advances in Neural Information Processing Systems 34 (2021), 5186–5198.
  44. Neural architecture search for time series classification. In 2020 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–8.
  45. Infinite Recommendation Networks: A Data-Centric Approach. arXiv:2206.02626 [cs.IR]
  46. Datadam: Efficient dataset distillation with attention matching. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 17097–17107.
  47. Minimax estimation of maximum mean discrepancy with radial kernels. Advances in Neural Information Processing Systems 29 (2016).
  48. Instance Normalization: The Missing Ingredient for Fast Stylization. arXiv:1607.08022 [cs.CV]
  49. Attention is all you need. Advances in neural information processing systems 30 (2017).
  50. Cafe: Learning to condense dataset by aligning features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12196–12205.
  51. Dataset Distillation. arXiv:1811.10959 [cs.LG]
  52. Time series data augmentation for deep learning: A survey. arXiv preprint arXiv:2002.12478 (2020).
  53. CoST: Contrastive learning of disentangled seasonal-trend representations for time series forecasting. arXiv preprint arXiv:2202.01575 (2022).
  54. Ling Yang and Shenda Hong. 2022. Unsupervised time-series representation learning with iterative bilinear temporal-spectral fusion. In International Conference on Machine Learning. PMLR, 25038–25054.
  55. Accelerating dataset distillation via model augmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11950–11959.
  56. Cross reconstruction transformer for self-supervised time series representation learning. arXiv preprint arXiv:2205.09928 (2022).
  57. Self-supervised contrastive pre-training for time series via time-frequency consistency. Advances in Neural Information Processing Systems 35 (2022), 3988–4003.
  58. Bo Zhao and Hakan Bilen. 2021. Dataset condensation with differentiable siamese augmentation. In International Conference on Machine Learning. PMLR, 12674–12685.
  59. Bo Zhao and Hakan Bilen. 2023. Dataset condensation with distribution matching. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 6514–6523.
  60. Dataset condensation with gradient matching. arXiv preprint arXiv:2006.05929 (2020).
  61. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning. PMLR, 27268–27286.
  62. Dataset distillation using neural feature regression. Advances in Neural Information Processing Systems 35 (2022), 9813–9827.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets