LeanStore: NVMe SSD B-tree Engine

• LeanStore is a main-memory–optimized, B-tree–based storage engine designed for NVMe SSDs that eliminates in-place writes to lower write amplification.
• It uses advanced techniques such as per-page compression, deathtime-based garbage collection, and DB–SSD alignment to boost performance and extend SSD lifespan.
• Experimental results show up to a 9.8× reduction in write amplification and more than 2× throughput improvements compared to traditional in-place designs.

LeanStore is a main-memory–optimized, B-tree–based storage engine designed for NVMe SSDs. Originally using in-place writes, its architecture was fundamentally redesigned to operate with out-of-place writes, optimizing for both performance and SSD lifespan. This transformation leverages advanced page management, compression, garbage collection, and cross-layer alignment techniques to systematically minimize write amplification (WA) across the database management system (DBMS) and SSD layers. Experimental results demonstrate dramatic improvements in transactional throughput and reductions in physical flash writes, with the modified engine seamlessly supporting recent SSD interface standards such as Zoned Namespace (ZNS) and Flexible Data Placement (FDP) (Lee et al., 10 Mar 2026).

1. Original Storage Engine Design

LeanStore is constructed around a fixed-size 4 KiB page architecture, where each page is identified by a Page Identifier (PID) and located via a direct offset calculation: $\text{offset} = \text{PID} \times 4~\mathrm{KiB}$.

Its B-tree implementation features conventional layouts for interior and leaf nodes, using keys, child PIDs, and headers containing checksums. Buffer management adopts an LRU-based cache, pinning frequently-accessed pages in memory. When a dirty page must be evicted or during checkpointing, LeanStore employs doublewrite buffering (DWB) for durability: the page is written first to a dedicated sequential DWB region, then persisted in-place at its fixed PID offset. Logging and recovery are managed via a write-ahead log (WAL), with log flushes enforced before in-place writes.

The in-place write workflow is as follows:

• Copy the dirty 4 KiB page to the DWB region
• Write the page to its fixed disk location (random I/O)
• Update in-memory metadata

This deterministic in-place overwrite pattern, however, introduces a random overwrite pattern at the SSD logical block address (LBA) level, significantly increasing SSD-internal write amplification due to frequent garbage collection and copyback operations.

2. Write Amplification and Its Implications

Write amplification in SSD-backed DBMS systems is the product of two multiplicative factors:

$$\text{Total WAF} = \text{DB WAF} \times \text{SSD WAF}$$

Where $\text{DB WAF}$ is the amplification induced by the DBMS layer, and $\text{SSD WAF}$ is the additional amplification imposed by the SSD’s internal management. Formal write amplification is defined as:

$$\text{WA} = \frac{\text{bytes written to flash}}{\text{bytes written by DBMS}}$$

Empirical results with in-place LeanStore on a 90% full Samsung PM9A3 SSD (YCSB-A, zipf θ=0.8) recorded $\text{DB WAF} \approx 2.0$ and $\text{SSD WAF} \approx 2.36$, yielding a total WA of approximately 4.7. Each 4 KiB logical page write thus incurs about 18.85 KiB of physical flash writes, substantially reducing the device's endurance and throughput. Because in-place DBMS writes are random from the SSD’s perspective, the system exacerbates SSD-level WA and blocks host-driven placement or GC optimizations.

3. Out-of-Place Optimizations

Transitioning LeanStore to out-of-place, append-only writes eliminated DWB and conferred fine-grained control over logical-to-physical placement, enabling a suite of optimizations:

3.1 Page-Wise Compression and Packing

Pages are compressed individually using standard algorithms such as LZ4 or ZSTD. Compressed pages are batch-packed into 4 KiB slots using a best-fit bin-packing algorithm, maintaining 4 KiB alignment to avoid read amplification and allowing retrieval of any page in a single 4 KiB I/O. Logical write volume is immediately reduced in proportion to the compression ratio.

3.2 Deathtime-Based Garbage Collection (GDT)

Out-of-place writes require host-level GC. Storage is divided into zones (GC units), each zone containing many pages. For each page, headers track the $n$ last write LSNs, and an Estimated Death Time (EDT) is computed:

$$\text{EDT} = \text{current\_lsn} + \frac{WH_n - WH_1}{n-1}$$

Pages with similar EDTs are grouped in the same zone ("Grouping by Death Time"), so that zones contain pages likely to become obsolete synchronously. GC reclaims the most invalidated zones, moves surviving pages in descending EDT order, and resets zones, minimizing DB-level WAF by reducing unnecessary page movement.

3.3 DB–SSD Garbage Collection Unit Alignment

To reduce SSD-level WAF, DBMS zones are aligned to SSD superblocks (groups of erase blocks). By matching these granularities, entire superblocks can be invalidated and reclaimed cleanly by the SSD with minimal overhead ($\text{SSD WAF} = 1$). Zone size discovery is empirical on standard SSDs, while FDP SSDs provide this metadata directly.

3.4 NoWA Pattern and FDP Placement Hints

On commodity SSDs lacking ZNS/FDP capabilities, the NoWA pattern is adopted. It limits the number of concurrently open zones such that only one SSD GC unit is filled at a time, and tracks write-frequency disparities to issue small compensation writes, ensuring balanced superblock filling and avoiding partial invalidation.

For FDP-enabled SSDs, NoWA is replaced by placement hints: each write carries a Placement ID (PlID) that maps the write to a distinct Reclaim Unit (RU), preventing interleaving and maintaining $$\text{Total WAF} = \text{DB WAF} \times \text{SSD WAF}$$0, provided the number of open zones does not exceed the number of RUs.

4. System Redesign and Integration

The out-of-place-optimized LeanStore integrates these techniques by extending core architectural components:

• Buffer Manager: Tracks recent write LSNs per page and computes EDT at eviction.
• I/O Interface: Implements per-page compression, packing, and selects the appropriate backend (zone-append for ZNS, io_uring for standard SSDs, or FDP hinting where supported).
• Space Manager: Replaces static PID→offset mapping with a mutable PID2OffsetTable, maintains a reverse mapping for slot-to-PID, zone statistics, and NoWA support metadata.
• Garbage Collector: Triggers when free active-zone slots are low, using GDT for victim selection, enforcing NoWA/FDP constraints, and migrating valid pages by EDT grouping.

On recovery, the system rebuilds the PID2OffsetTable and NoWA state from persisted checkpoints and WAL replay.

5. Experimental Evaluation

Extensive experiments use 64-thread YCSB-A and 32-thread TPC-C on a diverse set of SSDs, typically at 90% capacity and up to 1.6 TB data scales.

Key Results

• End-to-end WAF Reduction: In YCSB-A (800 GB), total WA drops from 4.72 (in-place) to 0.60 (all optimizations). This corresponds to a 7.8× reduction in physical flash writes per logical write.
• Throughput Improvement: YCSB-A throughput improves from 229 K OPS (in-place) to 535 K OPS (optimized), a 2.34× gain, with gains ranging 1.65×–2.24× across dataset sizes. On TPC-C (15,000 warehouses), throughput increases by 2.45×.
• Detailed Amplification Metrics:
  • DB WAF: 2.00 → 0.60 (with all techniques)
  • SSD WAF: 2.36 → 1.00 (with all techniques)
  • Logical writes/op: 1,858 B → 567 B
  • Physical writes/op: 4,378 B → 567 B
• Device Diversity: Across six SSD models, WAF reductions reach 6.2×–9.8×. Even on models with non-ideal internal WAF, optimized LeanStore achieves substantially lower amplification compared to in-place designs.

6. ZNS and FDP Interface Support

LeanStore’s redesign supports industry standards for host-managed SSDs:

• Zoned Namespace (ZNS): The engine matches its zone size to the device’s, leveraging NVMe ZNS zone-append/reset commands. GDT-based GC remains unchanged. In 800 GB YCSB-A tests, out-of-place+ZNS outperforms in-place on standard SSD by 2.1×, and yields an additional 31% benefit at 1.5 TB due to the elimination of SSD overprovisioning.
• NVMe Flexible Data Placement (FDP): The engine queries SSDs for RU size and handle count, configures matching zone parameters, and uses FDP placement hints to achieve $$\text{Total WAF} = \text{DB WAF} \times \text{SSD WAF}$$1 without NoWA compensation. On 1.6 TB YCSB-A, FDP-enabled optimized LeanStore reaches 553 K OPS with DB WAF=0.54, SSD WAF=1.00.

7. Significance and Implications

The systematic transformation of LeanStore to exploit out-of-place writes, per-page compression, GC-unit alignment, GDT-based garbage collection, and host–device coordination demonstrates a clear pathway toward SSD-lifetime-aware, high-throughput database system design. The approach requires only moderate changes to the buffer manager, I/O interface, and space manager; it enables immediate adoption of SSD advances such as ZNS and FDP without architectural overhaul. Experimental results highlight up to 9.8× lower write amplification and over 2× throughput gains in OLTP workloads, with robust improvements across heterogeneous flash hardware (Lee et al., 10 Mar 2026).