---
title: In-Storage Processing (ISP)
url: https://www.emergentmind.com/topics/in-storage-processing-isp
type: topic
---

# In-Storage Processing (ISP)

In-Storage Processing (ISP) refers to the deployment of general-purpose or specialized compute resources inside storage devices (notably Solid-State Drives, or SSDs) to execute user-defined or system-assistive tasks directly where the data resides. ISP architectures leverage the internal parallelism and bandwidth of modern storage subsystems, aiming to eliminate or drastically reduce host–storage data transfers, thereby improving performance, energy efficiency, and often enabling new computational paradigms for data-intensive workloads. ISP has become integral in domains ranging from graph analytics and machine learning to large-scale retrieval, genomics, and containerized analytics.

## 1. Hardware Architectures and System Abstractions

ISP systems are typically built upon computational storage devices (CSDs) that integrate one or more compute resources—embedded CPUs (e.g., ARM Cortex-A53/A9), FPGAs, or custom ASIC accelerators—alongside standard NAND flash, NVMe/PCIe interfaces, and local DRAM or SRAM pages buffers. Architectures include:

- **Firmware-based CSDs** with lightweight ARM cores executing both flash management (FTL, wear-leveling, garbage collection) and offloaded kernels (e.g., SmartSAGE on Cosmos+ OpenSSD or Newport CSDs) [2205.04711][2002.07215].
- **FPGA-augmented SSDs** for domain-specific acceleration (e.g., PreSto for RecSys data preprocessing, BlueDBM for sparse pattern processing) [2406.14571][1611.03380].
- **ASIC-integrated units** (e.g., DPZip for on-drive lossless compression) [2509.23693].
- **DRAM/flash crossbar and RCAM arrays** for associative or parallel in-data operations (e.g., PRINS) [1805.09612].
- **Virtualized/containerized ISP** enabling disaggregation via secure, managed, in-SSD microenvironments (e.g., DockerSSD) [2506.06769].

A key architectural differentiator is the location and accessibility of the compute substrate (controller CPU, DRAM, flash periphery logic), the type and size of local memory available, and the device’s support for custom firmware, networked management, and resource isolation.

## 2. Programming Models, Software Co-Design, and Security

Programming models for ISP encompass both low-level firmware extensions and higher-level container or runtime systems:

- **Firmware Co-Design:** Direct offload of phases such as sampling, filtering, or feature computation to in-firmware logic, exposed to the host via custom NVMe admin commands (e.g., SmartSAGE’s subgraph generation command, PreSto’s admin channel for per-feature task configuration) [2205.04711][2406.14571].
- **Virtualization and Containerization:** Lightweight container-like infrastructure (mini-Docker, OS-level virtualization with syscall emulation) isolates workloads and exposes standard `docker` workflows to the host, supporting distributed management and resource throttling [2506.06769].
- **Security and Trust:** Since co-tenant ISP workloads share in-SSD DRAM and may request flash access, sophisticated isolation mechanisms are essential. IceClave extends ARM TrustZone to partition controller memory and FTL tables, employing in-DRAM and flash-channel encryption, and per-entry access control (unique TEE ID check) [2205.06361]. Overhead for such TEE enforcement is low: 7.6% in throughput compared to unsecured ISP, with up to 2.31× speedup over host-based TEEs.
- **Languages and APIs:** Current practice uses low-level kernel insertion or OpenCL/bitstream deployment for FPGAs, ARM ELF binaries for embedded CPUs, and, increasingly, Docker/REST APIs for managed, distributed environments.

## 3. Task Partitioning, Offload Strategies, and Workload Mapping

Deciding what computation to offload is crucial. ISP excels when offloaded tasks are:

- **High-latency, low-compute, memory-bound:** e.g., neighbor sampling in GNN training (SmartSAGE), feature bucketing in RecSys (PreSto), or MapReduce pre-filtering [2205.04711][2406.14571][2112.12415].
- **Sparse pattern processing and filtering:** e.g., in-genomics via hash-based or chain-based filtering of reads (GenStore, MegIS) [2202.10400][2406.19113]
- **Vector and top-k/threshold pruning operations:** e.g., retrieval-augmented generation (REIS), mass-spectrometry vector similarity (FeNOMS) [2506.16444][2510.10872].
- **Compression, page hashing, content scans:** e.g., ASIC-based DPZip, BlueDBM, PRINS [2509.23693][1611.03380][1805.09612].

Task partitioning is generally achieved by profiling runtime memory, bandwidth, and computational requirements. For example, in SmartSAGE, only the “neighbor sampling” stage of GNN training is offloaded, while feature-lookup and aggregation execute on the host’s GPU [2205.04711]. PreSto offloads feature generation and normalization but leaves host–SSD decode and final assembly on host servers [2406.14571].

Offloading is also used for **privacy** (data never leaves the drive, as in STANNIS and Solana) and **energy efficiency**, with energy savings up to 69% (STANNIS) and 67% (Solana) for distributed ML and NLP workloads [2002.07215][2112.12415].

## 4. Quantitative Performance Models and Experimental Outcomes

Performance analyses in the literature employ both analytical models and extensive microbenchmarks:

- **Latency and Bandwidth:** ISP can amplify sampling throughput by 10–20× (SmartSAGE), achieving near parity with in-memory operations (within 1.1–1.3× slowdown); capacity scaling is at terabyte scale for SSDs (vs. hundreds of GB for DRAM) [2205.04711].
- **Effective Data Movement Reduction:** Only filtered, sampled, or aggregated (compact) results cross the PCIe bus, reducing DRAM↔SSD movement by up to 20× (SmartSAGE), 71× (MegIS for metagenomics), or more [2205.04711][2406.19113].
- **Throughput and Scaling:**
  - PreSto achieves 9.6× speedup in RecSys preprocessing and 11.3× better energy efficiency vs. large pools of CPU servers [2406.14571].
  - GenStore attains up to 33.6× speedup on error-prone long reads and 27× energy reduction in genomic filtering [2202.10400].
  - Containerized ISP (DockerSSD) reaches up to 2× speedup for I/O-intensive analytics and 7.9× for distributed LLM inference (native flash KV caching) [2506.06769].
  - Multi-CSD deployments exhibit near-linear scaling in throughput and energy savings (STANNIS, Solana) [2002.07215][2112.12415].
- **Comparative Placement:** In-storage CDPU (DPZip) offers 4.7 μs compression latency per 4 KB block, outperforming host/PCIe/on-chip placements (up to 2× read/write throughput and 0.5% CV in latency across VMs) [2509.23693].
- **Application-specific ISP:** Specialized architectures (e.g., FeNOMS with 3D FeNAND, PRINS with RCAM) break the memory bandwidth wall, achieving up to 224× (energy) and 10⁴× (throughput) advantage over off-storage/host approaches [2510.10872][1805.09612].

## 5. Key Design Trade-Offs, Limitations, and Security

Trade-offs in ISP design include:

- **Compute Power vs. Data Movement:** Embedded cores are generally less powerful than host CPUs/GPUs, but ISP’s data reduction and parallelization can dominate for I/O-bound or memory-bound tasks [2205.04711][2506.06769].
- **Granularity of Offload:** In-storage CDPUs operate at fixed block sizes (often 4 KB); while optimal for stream analytics, less flexible for content-aware or variable-size workloads [2509.23693].
- **Firmware and Programmability:** Deep firmware modification can impede adaptability or vendor interoperability. DockerSSD minimizes this with OS-level containers and virtual firmware abstraction [2506.06769].
- **Security and Multi-Tenancy:** Robust resource partitioning, DRAM encryption, and TEE enforcement (e.g., IceClave’s TrustZone partition, in-channel flash encryption) are necessary to prevent cross-tenant attacks or code injection [2205.06361].
- **Wear, Endurance, Resource Limits:** SLC/TLC partitions, coarse mapping, and wear-leveling are employed to balance endurance and performance. For complex or large models, limited DRAM on-device can be a constraint [2506.16444][2112.12415].
- **Workload Suitability:** ISP is least effective for compute-bound kernels with high floating-point requirements or workloads that exceed internal DRAM footprints and require significant multi-stage computation not easily mapped to local PEs [2205.04711][1805.09612].

## 6. Broader Implications and Future Research Trajectories

ISP advances not only performance and efficiency but also reconfigures the architectural boundary between storage and compute for:

- **Big Data and Scientific Analytics:** Line-rate scan/filter/aggregate on petabyte-scale data, practical on commodity form factors (E1.S/U.2) [2112.12415][2306.04323].
- **Scalable and Energy-Efficient ML Pipelines:** Enabling terabyte-scale GNNs, LLM inference pools, federated privacy-preserving ML [2205.04711][2506.06769][2002.07215].
- **Cloud and Disaggregated Storage:** Direct NVMe-over-Ethernet deployment of containerized, in-SSD code and dynamic analytics orchestration [2506.06769].
- **Capacity and TCO Planning:** Closed-form models enable break-even analysis and cost optimization for CSD/ISP deployment vs. host-centric architectures [2306.04323].

Important open areas include adaptive and ML-guided offload partitioning (e.g., Conduit), extension to heterogeneous and accelerator-filled SSDs (virtual-GPU, direct FPGA), formal verification of firmware isolation, and advances in high-level programming frameworks for dynamic and multi-tenant ISP [2601.17633][2506.06769][2205.06361].

---

**Key References**:  
- SmartSAGE: Training Large-scale Graph Neural Networks using In-Storage Processing Architectures [2205.04711]  
- Containerized In-Storage Processing and Computing-Enabled SSD Disaggregation [2506.06769]  
- In-Storage Embedded Accelerator for Sparse Pattern Processing [1611.03380]  
- STANNIS: Low-Power Acceleration of Deep Neural Network Training Using Computational Storage [2002.07215]  
- PreSto: An In-Storage Data Preprocessing System for Training Recommendation Models [2406.14571]  
- In-storage Processing of I/O Intensive Applications on Computational Storage Drives [2112.12415]  
- Conduit: Programmer-Transparent Near-Data Processing Using Multiple Compute-Capable Resources in Solid State Drives [2601.17633]  
- REIS: A High-Performance and Energy-Efficient Retrieval System with In-Storage Processing [2506.16444]  
- FeNOMS: Enhancing Open Modification Spectral Library Search with In-Storage Processing on Ferroelectric NAND (FeNAND) Flash [2510.10872]  
- ASIC-based Compression Accelerators for Storage Systems: Design, Placement, and Profiling Insights [2509.23693]  
- PRINS: Resistive CAM Processing in Storage [1805.09612]  
- MegIS: High-Performance, Energy-Efficient, and Low-Cost Metagenomic Analysis with In-Storage Processing [2406.19113]  
- Near-Data Processing for Differentiable Machine Learning Models [1610.02273]  
- Building A Trusted Execution Environment for In-Storage Computing [2205.06361]  
- GenStore: A High-Performance and Energy-Efficient In-Storage Computing System for Genome Sequence Analysis [2202.10400]  
- An Analytical Model-based Capacity Planning Approach for Building CSD-based Storage Systems [2306.04323]

Source: https://www.emergentmind.com/topics/in-storage-processing-isp