A Survey on Compiler Autotuning using Machine Learning (1801.04405v5)

Abstract: Since the mid-1990s, researchers have been trying to use machine-learning based approaches to solve a number of different compiler optimization problems. These techniques primarily enhance the quality of the obtained results and, more importantly, make it feasible to tackle two main compiler optimization problems: optimization selection (choosing which optimizations to apply) and phase-ordering (choosing the order of applying optimizations). The compiler optimization space continues to grow due to the advancement of applications, increasing number of compiler optimizations, and new target architectures. Generic optimization passes in compilers cannot fully leverage newly introduced optimizations and, therefore, cannot keep up with the pace of increasing options. This survey summarizes and classifies the recent advances in using machine learning for the compiler optimization field, particularly on the two major problems of (1) selecting the best optimizations and (2) the phase-ordering of optimizations. The survey highlights the approaches taken so far, the obtained results, the fine-grain classification among different approaches and finally, the influential papers of the field.

• The paper systematically classifies over 200 studies to address challenges in optimization selection and phase-ordering using machine learning.
• It reviews diverse machine learning models, including supervised, unsupervised, and reinforcement methods, to navigate the expanding optimization space.
• The findings imply significant improvements in compiler performance and adaptability for evolving hardware and complex applications.

A Survey on Compiler Autotuning using Machine Learning

The paper "A Survey on Compiler Autotuning using Machine Learning" presents an exhaustive overview of the advancements made in compiler optimization through the integration of machine learning techniques. The paper, published in ACM Computing Surveys in 2018, provides a detailed classification and categorization of research efforts aimed at tackling two critical compiler optimization problems: optimization selection and phase-ordering.

Overview

Since the mid-1990s, researchers have increasingly used machine learning to address compiler optimization challenges. Compiler optimization is crucial because the optimization space is continually expanding with new applications, compiler features, and target architectures. Traditional optimization passes cannot fully leverage these advancements, necessitating more sophisticated approaches. The survey systematically summarizes recent machine learning applications that address:

1. Optimization selection - Choosing the most suitable optimizations from a potentially vast array.
2. Phase-ordering - Determining the optimal sequence, or order, in which selected optimizations should be applied.

The paper emphasizes the exponential growth of the optimization space and the need for advanced techniques to manage this complexity efficiently. Traditional methods struggle to address phase-ordering due to its factorial nature in terms of permutations and combinations. This recognition has led to embracing machine learning models that can better navigate these challenges.

Contribution and Structure

The researchers classify over 200 papers, drawing links between different approaches taken in compiler autotuning assisted by machine learning. The paper’s organization facilitates understanding of the intricate relationships between compiler techniques and ML models, defining categorizations based on application characterization, machine learning models, prediction types, space exploration methods, target domains, and influential works.

Key Considerations

Application Characterization

Applications can be characterized through:

• Static Analysis: Source-code and IR features independent of specific architectures.
• Dynamic Analysis: Architecture-dependent counters providing runtime insights.
• Hybrid Analysis: Combining multiple features for richer profiling.

Dimension reduction techniques such as PCA and FA are employed to refine these characterizations, critical in rendering efficient machine learning models.

Machine Learning Models

The survey reveals the broad spectrum of machine learning models applied:

• Supervised Learning: Including Bayesian Networks, SVMs, and Decision Trees.
• Unsupervised Learning: Clustering techniques and various evolutionary algorithms.
• Other strategies include reinforcement learning and graph kernels.

Prediction Types

Key prediction methodologies are discussed:

• Speedup Prediction: Estimating performance gains without direct measurement.
• Compiler Sequence Prediction: Automatically generating optimal sequences.
• Intermediate Prediction: Predicting performance gain at each optimization stage.

Space Exploration Techniques

The survey categorizes:

• Adaptive Compilation: Feedback-driven dynamic modification of code.
• Iterative Compilation: Sequential exploration to refine optimization strategies.
• Non-iterative Methods: Fixed strategies for comprehensive global optimization.

Domain and Compiler Context

Various compilers (GCC, LLVM, Intel ICC) and their targets (embedded systems, desktop environments, HPC) are critically analyzed, acknowledging their unique contributions to the advancements in compiler optimization.

Implications

The implications of integrating machine learning in compiler optimization are manifold—this approach enhances optimization precision, advances automation, and provides the foundation for future compiler design that can adaptively respond to evolving hardware architectures and application demands. Furthermore, the survey suggests an increased adoption of collaborative and reproducible research methodologies to maximize these benefits.

In conclusion, the paper serves as an essential resource for researchers in the domain of compilers and machine learning, offering a comprehensive perspective on the achievements and future directions of compiler autotuning using machine learning. The myriad of models, frameworks, and categorization elaborated in the survey provide the groundwork for ongoing and future innovations in this field.