Dark Memory and Accelerator-Rich System Optimization in the Dark Silicon Era (1602.04183v3)

Abstract: The key challenge to improving performance in the age of Dark Silicon is how to leverage transistors when they cannot all be used at the same time. In modern SOCs, these transistors are often used to create specialized accelerators which improve energy efficiency for some applications by 10-1000X. While this might seem like the magic bullet we need, for most CPU applications more energy is dissipated in the memory system than in the processor: these large gains in efficiency are only possible if the DRAM and memory hierarchy are mostly idle. We refer to this desirable state as Dark Memory, and it only occurs for applications with an extreme form of locality. To show our findings, we introduce Pareto curves in the energy/op and mm$^2$/(ops/s) metric space for compute units, accelerators, and on-chip memory/interconnect. These Pareto curves allow us to solve the power, performance, area constrained optimization problem to determine which accelerators should be used, and how to set their design parameters to optimize the system. This analysis shows that memory accesses create a floor to the achievable energy-per-op. Thus high performance requires Dark Memory, which in turn requires co-design of the algorithm for parallelism and locality, with the hardware.

Optimization in the Dark Silicon Era: Dark Memory and Accelerator-Rich Systems

The paper "Dark Memory and Accelerator-Rich System Optimization in the Dark Silicon Era" explores the complexities and solutions surrounding transistor utilization when they cannot be all active simultaneously. The paper primarily addresses the challenge of improving processor performance while balancing the constraints imposed by power consumption, a phenomenon widely associated with the term "Dark Silicon." This paper explores various methodologies for optimizing the use of available transistors, with a particular focus on specialized accelerators and the concept of "Dark Memory."

In the context of Dark Silicon, the paper discusses four main strategies: shrink, dim, specialize, and new technology. Each strategy offers a different approach to navigating the power limitations that have emerged as a result of the cessation of voltage scaling. Among these, specializing through accelerators emerges as a prominent solution, offering significant energy efficiency improvements over general-purpose computing units. However, the paper emphasizes that for the potential of accelerators to be fully harnessed, their gains in efficiency must be accompanied by dramatic reductions in memory system energy dissipation. This leads to the pivotal discussion of "Dark Memory," where the goal is to keep DRAM and other memory hierarchy levels largely idle.

The research introduces Pareto curves as a tool for evaluating trade-offs between energy-per-operation and area-related performance metrics for computing units, accelerators, and on-chip memory systems. These curves facilitate solving power, performance, and area-constrained optimization problems by illustrating which accelerators should be prioritized and how their design parameters can be fine-tuned to meet system goals. The paper reinforces the notion that memory access energy forms a baseline constraint in achieving low-energy operations, highlighting the necessity of achieving high performance by maintaining as much of the memory hierarchy in a dark state as possible.

Moreover, the paper addresses algorithmic optimization as a means to achieve high system performance. A critical insight is that optimal hardware design requires a co-design approach where algorithms are structured to maximize locality and parallelism, thus reducing memory access demands. Techniques such as loop blocking in algorithms like general matrix multiplication (GEMM) exemplify how computational processes can be reordered to enhance data locality and decrease DRAM volume.

The paper underscores the strategic importance of effectively utilizing accelerator-rich architectures for balancing energy efficiency and area costs. This involves leveraging the Pareto optimality in evaluating system designs under varying constraints—power, performance, and area—known for their repercussions on the physical and economic viability of chip production.

In conclusion, the paper presents an in-depth analysis focused on optimizing energy consumption and area efficiency in the Dark Silicon era. Through the interplay of specialized hardware, algorithmic co-design, and comprehensive use of Pareto analytics, the research provides a roadmap for addressing the inherent challenges of optimizing processor designs amidst the constraints of modern silicon technology.

Future work could extend these design principles to encompass emerging memory technologies like non-volatile memory systems, which promise to alter the landscape of memory access energy costs. Furthermore, as computational models and workloads evolve, constantly reevaluating design assumptions will be essential for maintaining optimal performance in processor architectures.