Flash-Cosmos: In-Flash Bulk Bitwise Operations Using Inherent Computation Capability of NAND Flash Memory (2209.05566v1)

Abstract: Bulk bitwise operations, i.e., bitwise operations on large bit vectors, are prevalent in a wide range of important application domains, including databases, graph processing, genome analysis, cryptography, and hyper-dimensional computing. In conventional systems, the performance and energy efficiency of bulk bitwise operations are bottlenecked by data movement between the compute units and the memory hierarchy. In-flash processing (i.e., processing data inside NAND flash chips) has a high potential to accelerate bulk bitwise operations by fundamentally reducing data movement through the entire memory hierarchy. We identify two key limitations of the state-of-the-art in-flash processing technique for bulk bitwise operations; (i) it falls short of maximally exploiting the bit-level parallelism of bulk bitwise operations; (ii) it is unreliable because it does not consider the highly error-prone nature of NAND flash memory. We propose Flash-Cosmos (Flash Computation with One-Shot Multi-Operand Sensing), a new in-flash processing technique that significantly increases the performance and energy efficiency of bulk bitwise operations while providing high reliability. Flash-Cosmos introduces two key mechanisms that can be easily supported in modern NAND flash chips: (i) Multi-Wordline Sensing (MWS), which enables bulk bitwise operations on a large number of operands with a single sensing operation, and (ii) Enhanced SLC-mode Programming (ESP), which enables reliable computation inside NAND flash memory. We demonstrate the feasibility of performing bulk bitwise operations with high reliability in Flash-Cosmos by testing 160 real 3D NAND flash chips. Our evaluation shows that Flash-Cosmos improves average performance and energy efficiency by 3.5x/32x and 3.3x/95x, respectively, over the state-of-the-art in-flash/outside-storage processing techniques across three real-world applications.

• The paper introduces a novel in-flash processing scheme using MWS and ESP techniques that enhance bulk bitwise operations.
• It details Multi-Wordline Sensing (MWS) which concurrently processes multiple operands, reducing operation counts and energy consumption.
• Enhanced SLC-mode Programming (ESP) refines voltage margins to minimize bit error rates, achieving near-zero BER for reliable computation.

Flash-Cosmos: Leveraging In-Flash Processing for Efficient Bulk Bitwise Operations

The paper, "Flash-Cosmos: In-Flash Bulk Bitwise Operations Using Inherent Computation Capability of NAND Flash Memory," offers significant advancements in enhancing the performance, energy efficiency, and reliability of in-flash processing for bulk bitwise operations. As the demand grows for efficient processing capabilities within storage systems, particularly for data-intensive applications, Flash-Cosmos introduces innovative mechanisms to optimize the use of NAND flash memory, providing critical insights into the potential of near-data processing paradigms.

Key Contributions and Mechanisms

The paper identifies two major limitations in existing in-flash processing techniques: underutilization of NAND flash's bit-level parallelism, and reliability issues due to NAND's error-prone nature. The authors propose Flash-Cosmos, which introduces two mechanisms to address these challenges:

1. Multi-Wordline Sensing (MWS): This technique leverages the structural and operational parallels between NAND flash cells and digital logic circuits to enable concurrent processing of multiple (potentially tens) operands in a single sensing operation. By using the NAND and NOR operations inherent in the flash architecture, MWS fundamentally reduces the number of operations required, thereby boosting speed and energy efficiency.
2. Enhanced SLC-mode Programming (ESP): To ensure computational reliability, ESP refines single-level cell programming by increasing voltage margins between different states, thereby minimizing bit error rates (BER). This enhancement makes Flash-Cosmos particularly suitable for a broad range of applications that demand high reliability, achieving a near-zero BER in computation results.

Performance and Evaluations

The paper compares Flash-Cosmos to outside-storage processing (OSP) and in-storage processing (ISP) baselines, as well as the state-of-the-art in-flash processing technique (ParaBit). Experimental results show Flash-Cosmos achieves a 3.5x performance improvement and a 95x increase in energy efficiency over ParaBit, particularly excelling in scenarios involving numerous operands. By executing operations within NAND flash memory, Flash-Cosmos reduces costly data transfers, which are the main bottleneck in traditional architectures.

Implications and Future Directions

Flash-Cosmos demonstrates the feasibility of extending NAND flash memory beyond storage to computational roles, configuring it as an efficient substrate for near-data processing. This aligns with the industry's shift toward minimizing data movement, a fundamental challenge in the face of rapidly growing data volumes.

Theoretically, Flash-Cosmos opens up new avenues for integrating in-storage computing with other emerging processing-in-memory and near-data processing mechanisms. Practically, it could support a crucial transition toward more diverse and efficient computing environments, particularly within large-scale data centers and AI applications requiring real-time data processing capabilities.

Conclusion

This research contributes substantially to the field by illustrating how intrinsic properties of NAND flash can be harnessed for enhanced computation, moving towards a future where storage and processing are increasingly convergent. Flash-Cosmos signifies a step forward in realizing the full potential of in-flash processing, paving the way for its broader application and integration into various domains, including databases, genomics, and secure computing. Future research could explore the integration of Flash-Cosmos with conventional computing stacks and its applicability in emerging computational paradigms like homomorphic encryption and distributed validation networks.