Multi-Dimensional Vector ISA Extension for Mobile In-Cache Computing (2501.09902v1)

Abstract: In-cache computing technology transforms existing caches into long-vector compute units and offers low-cost alternatives to building expensive vector engines for mobile CPUs. Unfortunately, existing long-vector Instruction Set Architecture (ISA) extensions, such as RISC-V Vector Extension (RVV) and Arm Scalable Vector Extension (SVE), provide only one-dimensional strided and random memory accesses. While this is sufficient for typical vector engines, it fails to effectively utilize the large Single Instruction, Multiple Data (SIMD) widths of in-cache vector engines. This is because mobile data-parallel kernels expose limited parallelism across a single dimension. Based on our analysis of mobile vector kernels, we introduce a long-vector Multi-dimensional Vector ISA Extension (MVE) for mobile in-cache computing. MVE achieves high SIMD resource utilization and enables flexible programming by abstracting cache geometry and data layout. The proposed ISA features multi-dimensional strided and random memory accesses and efficient dimension-level masked execution to encode parallelism across multiple dimensions. Using a wide range of data-parallel mobile workloads, we demonstrate that MVE offers significant performance and energy reduction benefits of 2.9x and 8.8x, on average, compared to the SIMD units of a commercial mobile processor, at an area overhead of 3.6%.

• The paper presents a novel ISA extension that enables multi-dimensional vector access to simplify complex data patterns and reduce instruction counts.
• It introduces dimension-level masked execution to efficiently manage irregular data without resorting to redundant scalar operations.
• Evaluations show up to 2.9× performance and 8.8× energy efficiency improvements with minimal area overhead by repurposing mobile CPU cache.

Multi-Dimensional Vector ISA Extension for Mobile In-Cache Computing

The paper presents an innovative Instruction Set Architecture (ISA) extension named Multi-dimensional Vector Extension (MVE) designed to optimize in-cache computing for mobile systems with vector processing units. This research addresses the limitations of existing one-dimensional (1D) long-vector ISA extensions like RISC-V Vector Extension (RVV) and Arm Scalable Vector Extension (SVE) in effectively utilizing the wide Single Instruction, Multiple Data (SIMD) widths of in-cache vector engines. By supporting multi-dimensional strided and random memory accesses and efficient dimension-level masking, MVE aims to enhance data level parallelism (DLP) across multiple dimensions in mobile computing.

Key Contributions

1. Multi-Dimensional Vector Access: Conventional ISAs typically support 1D memory access patterns. The MVE introduces more versatile multi-dimensional strided vector loads and stores. This reduces instruction counts and increases computational efficiency by allowing programmers to directly encode complex data access patterns often found in mobile applications.
2. Enhanced Masked Execution: Besides traditional predicated execution, MVE introduces dimension-level masking controls, allowing certain computing dimensions to be masked more efficiently without redundant scalar operations. This capability efficiently manages irregular data-parallel patterns, which are prevalent in mobile workloads.
3. Flexible Compute-Capable Cache Design: The proposed MVE utilizes a compute-capable cache architecture that transforms cache resources into a high-throughput long-vector processing engine. With an innovative cache controller design, MVE can efficiently handle multi-dimensional data mappings and compute operations without exposing cache geometry complexities to programmers.
4. Performance and Energy Benefits: MVE achieves substantial performance and energy efficiency improvements. The paper's evaluations show an average improvement of 2.9× in performance and 8.8× in energy efficiency compared to traditional SIMD units, with only a 3.6% area overhead relative to the core. This is achieved by repurposing half of the mobile CPU’s L2 cache for in-cache computing, offering a significant boost in computational capability by utilizing untapped resources.

Implications and Speculation

1. Reduced Instruction Complexity: By compressing multi-loop computations into a single vector operation, MVE reduces the instruction fetch and issue rate bottlenecks, enhancing overall throughput. This has further implications for improving compiler optimizations in terms of instruction-level parallelism and register allocation strategies.
2. Cross-Domain Application Potential: Although aimed at mobile device optimizations, the MVE's framework can serve as a model for other application domains where memory complexity and efficiency are pressing issues. Examples include real-time communication systems, graphical rendering pipelines, and various domains utilizing multi-dimensional data structures extensively.
3. Future Development in AI and Mobile Processing: As artificial intelligence workloads continue to evolve, there is an increasing demand for efficient computation models that can handle sparse and irregular data patterns. MVE’s architectural enhancements in SIMD utilization could inform the design of future AI accelerators focused on edge computing and low-power environments.
4. Enhanced Mobile SoC Designs: The successful integration of MVE within mobile SoCs points to the potential for its adoption in commercial mobile processors, allowing for scalable, high-efficiency vector computations that could push the boundaries of what's feasible in edge computing.

Conclusion

This research extends the capabilities of standard SIMD architectures for mobile processors by enhancing data-access flexibility and compute efficiency through a novel ISA extension. The MVE's ability to reduce computational overhead and energy consumption while leveraging existing hardware resources presents a compelling approach for advancing in-cache computing. Consequently, it sets the stage for innovation in mobile processing, where balancing performance, area, and power is paramount. The adoption of multi-dimensional vector ISAs like MVE could redefine computational paradigms in various areas, with immediate applicability in domains requiring efficient processing of complex data sets.