Model Complexity of Program Phases (2310.03865v1)
Abstract: In resource limited computing systems, sequence prediction models must operate under tight constraints. Various models are available that cater to prediction under these conditions that in some way focus on reducing the cost of implementation. These resource constrained sequence prediction models, in practice, exhibit a fundamental tradeoff between the cost of implementation and the quality of its predictions. This fundamental tradeoff seems to be largely unexplored for models for different tasks. Here we formulate the necessary theory and an associated empirical procedure to explore this tradeoff space for a particular family of machine learning models such as deep neural networks. We anticipate that the knowledge of the behavior of this tradeoff may be beneficial in understanding the theoretical and practical limits of creation and deployment of models for resource constrained tasks.
- The information complexity of learning tasks, their structure and their distance. Information and Inference: A Journal of the IMA, 10(1):51–72.
- Phase-aware CPU workload forecasting.
- Learning long-term dependencies with gradient descent is difficult. Transactions in Neural Networks, 5(2):157–166.
- Perceptron-based prefetch filtering. 2019 ACM/IEEE 46th Annual International Symposium on Computer Architecture (ISCA), pages 1–13.
- Evidence-based static branch prediction using machine learning. ACM Trans. Program. Lang. Syst., 19:188–222.
- Using machine learning techniques to analyze the performance of concurrent kernel execution on GPUs. Future Gener. Comput. Syst., 113:528–540.
- Real time detection of cache-based side-channel attacks using hardware performance counters. Appl. Soft Comput., 49:1162–1174.
- Run-time program-specific phase prediction for Python programs. In ManLang ’18.
- Clustering by compression. IEEE Transactions on Information Theory, 51:1523–1545.
- Block2vec: A deep learning strategy on mining block correlations in storage systems. 2016 45th International Conference on Parallel Processing Workshops (ICPPW), pages 230–239.
- Classification of computer hardware and performance prediction using statistical learning and neural networks.
- Hochreiter, S. (1998). The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int. J. Uncertain. Fuzziness Knowl.-Based Syst., 6(2):107–116.
- Long short-term memory. Neural Computation, 9:1735–1780.
- Learning automatic schedulers with projective reparameterization.
- Miss Rate Estimation (MRE): a Novel Approach Toward L2 Cache Partitioning Algorithms for Multicore System.
- RHMD: Evasion-Resilient Hardware Malware Detectors. 2017 50th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pages 315–327.
- EnsembleHMD: Accurate Hardware Malware Detectors with Specialized Ensemble Classifiers. IEEE Transactions on Dependable and Secure Computing, 17:620–633.
- A new approach to detecting execution phases using performance monitoring counters. In Architecture of Computing Systems - ARCS 2017.
- Machine learning based online performance prediction for runtime parallelization and task scheduling. 2009 IEEE International Symposium on Performance Analysis of Systems and Software, pages 89–100.
- Assessing the ability of LSTMs to learn syntax-sensitive dependencies. Transactions of the Association for Computational Linguistics, 4:521–535.
- Approximating rate-distortion graphs of individual data: Experiments in lossy compression and denoising. IEEE Transactions on Computers, 61:395–407.
- Learning internal representations by error propagation.
- Regression as classification. 2012 Proceedings of IEEE Southeastcon, pages 1–6.
- Program phase duration prediction and its application to fine-grain power management. 2013 IEEE Computer Society Annual Symposium on VLSI (ISVLSI), pages 127–132.
- Sequence to sequence learning with neural networks. In Neural Information Processing Systems.
- Kolmogorov’s structure functions and model selection. IEEE Transactions on Information Theory, 50:3265–3290.
- GPGPU performance and power estimation using machine learning. 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA), pages 564–576.
- Google’s neural machine translation system: Bridging the gap between human and machine translation. CoRR, abs/1609.08144.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
