Fully-fused Multi-Layer Perceptrons on Intel Data Center GPUs (2403.17607v1)

Abstract: This paper presents a SYCL implementation of Multi-Layer Perceptrons (MLPs), which targets and is optimized for the Intel Data Center GPU Max 1550. To increase the performance, our implementation minimizes the slow global memory accesses by maximizing the data reuse within the general register file and the shared local memory by fusing the operations in each layer of the MLP. We show with a simple roofline model that this results in a significant increase in the arithmetic intensity, leading to improved performance, especially for inference. We compare our approach to a similar CUDA implementation for MLPs and show that our implementation on the Intel Data Center GPU outperforms the CUDA implementation on Nvidia's H100 GPU by a factor up to 2.84 in inference and 1.75 in training. The paper also showcases the efficiency of our SYCL implementation in three significant areas: Image Compression, Neural Radiance Fields, and Physics-Informed Machine Learning. In all cases, our implementation outperforms the off-the-shelf Intel Extension for PyTorch (IPEX) implementation on the same Intel GPU by up to a factor of 30 and the CUDA PyTorch version on Nvidia's H100 GPU by up to a factor 19. The code can be found at https://github.com/intel/tiny-dpcpp-nn.

• The paper presents a novel fully-fused MLP approach on Intel Data Center GPUs, achieving a 2.84x speedup in inference over CUDA on Nvidia H100.
• It employs SYCL with XMX matrix instructions to optimize arithmetic intensity by reducing global memory accesses, as validated by a comprehensive roofline analysis.
• The implementation significantly enhances performance in diverse applications, including image compression, NeRFs, and physics-informed machine learning.

Fully-Fused MLP Implementation on Intel Data Center GPUs Outperforms CUDA on Nvidia H100

Implementation and Optimization of Multi-Layer Perceptrons on Intel GPUs

The remarkable work executed by Yuan et al. on implementing Multi-Layer Perceptrons (MLPs) specifically tailored for Intel Data Center GPU Max 1550 via SYCL has set a new benchmark in the computational efficacy of neural networks. At the core of this achievement is the optimization strategy that skillfully reduces the dependency on slow global memory accesses by maximizing data reuse within the general register file and shared local memory. The technique of fusing operations in each layer of the MLP culminates in a notable escalation in arithmetic intensity, primarily enhancing inference performance.

This fully-fused approach significantly outstrips the conventional CUDA-based MLP implementations on Nvidia's H100 GPU across both inference and training tasks, showing an up to 2.84-fold increase in inference and a 1.75-fold increase during training scenarios. This paper details the methodological advancements, analytical comparisons using roofline models, and real-world implications of these improvements in performance.

SYCL Implementation and Performance Analysis

The paper presents a SYCL-based implementation of fully-fused MLPs optimized for the Intel GPU architecture, significantly benefitting from the XMX hardware acceleration available on the Intel Data Center GPU Max 1550. Utilizing the Intel joint_matrix SYCL extension, the implementation harnesses the XMX matrix instructions, optimizing the arithmetic intensity of MLPs and thus their performance, particularly for large batch sizes prevalent in machine learning workloads.

A comprehensive roofline analysis further elucidates the improvements brought about by this implementation, estimating significant increments in the arithmetic intensity for access to both global and shared local memory compared to existing CUDA-based solutions.

Demonstrated Efficiency Across Diverse Applications

The empirical validation of the proposed implementation across a variety of applications - from Image Compression and Neural Radiance Fields (NeRFs) to Physics-Informed Machine Learning - demonstrates performance enhancements that often reach or exceed current state-of-the-art methods:

• Image Compression: Outperforms existing solutions by up to 30 times, showcasing the ability of the fully-fused MLPs to efficiently process and learn from image data.
• Neural Radiance Fields (NeRFs): Achieves superior inference speed with an up to 19-fold improvement over CUDA PyTorch versions on Nvidia's H100 GPU, highlighting the effectiveness of the approach in 3D rendering tasks.
• Physics-Informed Machine Learning: The optimized implementation propels forward the possibilities for simulations and solving differential equations by leveraging the power of GPUs.

Future Directions and Outlook

With the open-sourcing of this implementation, the intention is to catalyze further research and developments in the optimization of neural network computations on GPU architectures. Future works might focus on extending the robustness of this method to support a broader range of layer widths and data types, thereby enhancing its applicability across more domains of machine learning and artificial intelligence.

Furthermore, exploration into Intel's ESIMD SYCL extension, aimed at finer control over register usage and cache operations, presents an inviting avenue for advancing the computational efficiency of MLPs on GPU platforms.

In conclusion, the work of Yuan et al., by devising a SYCL-based fully-fused MLP implementation optimized for Intel GPUs, marks a substantial forward leap in neural network performance optimization. The improvements in computational efficiency, particularly for inference tasks, underscore the potential of specialized hardware optimizations in future AI and machine learning endeavors.