Baseline–Log Physical Separation Overview

• Baseline–log physical separation is a storage architecture that decouples the write-ahead log from data files, resulting in doubled persistence for each write.
• The dual-write model introduces increased I/O overhead and random-access penalties, impacting both throughput efficiency and recovery time.
• Unifying the persistence path in log-only designs, like LogBase, offers improved write throughput and faster crash recovery.

Baseline–log physical separation denotes the storage arrangement in which a system maintains a write-ahead log and application data as distinct persistent artifacts. In the presentation associated with LogBase, this appears as the classic WAL + Data pattern used by “many commercial and open-source systems (DB2, PostgreSQL, MySQL, HBase, BigTable, Cassandra, …),” where a client write is sent both to a Write-Ahead Log (WAL) on “stable storage” and to an in-memory buffer that is later flushed to a data file (Vo et al., 2012). Write-ahead-logging is described there as “a common approach for providing recovery capability while improving performance in most storage systems,” while the physical separation of log and data is identified as a source of write overheads in write-heavy environments.

1. Definition and architectural pattern

In the baseline arrangement, a client request is handled by a tablet server that performs two logically distinct actions: it appends the update to the WAL and also places the update in an in-memory structure such as a “memtable” or “buffer pool,” from which the data are later flushed to a persistent data file (Vo et al., 2012). The defining characteristic is therefore not merely logging, but the coexistence of a log path and a separate data path.

The pattern can be summarized as follows: the client writes to the tablet server; the server sends the update to a WAL on memory that is synchronized to a WAL file on disk; in parallel, the update enters an in-memory data buffer and is later flushed to a data file. The narrative explicitly states that “every user write thus pays two persistent writes: one to WAL and one later ‘in-place’ or appended to a data store” (Vo et al., 2012).

This separation is called “physical” because the log and the data are stored as different persistent objects. In the examples cited, the log is the recovery structure, whereas the data file is the main organized repository for keyed or paged access. A plausible implication is that the term emphasizes storage layout and update path, rather than transaction semantics alone.

2. Write-path cost model

The baseline write path is formalized by a simple I/O accounting model. If the logical write size is $W$ bytes, then

$I/O_{\text{user}} = W$

$I/O_{\text{log}} = W$

$I/O_{\text{data}} = W$

which yields

$$\alpha = \frac{I/O_{\text{log}} + I/O_{\text{data}}}{I/O_{\text{user}}} = 2.$$

This write-amplification result captures the central cost of baseline–log physical separation: the same logical update is persisted twice (Vo et al., 2012).

The same source further distinguishes the two writes by access pattern. The log write is sequential and incurs $s_{\text{log}}$, whereas the data-file update is “typically laid out by key or by page” and updated in place, so the second write is at a random position and incurs an additional seek penalty $s_{\text{data}}$. The resulting approximation is

$$T_{\text{write}} \approx s_{\text{log}} + W/B_{\text{seq}} + s_{\text{data}} + W/B_{\text{rnd}},$$

with $B_{\text{seq}} \gg B_{\text{rnd}}$ (Vo et al., 2012).

This model is significant because it makes the baseline penalty explicit: duplicated persistence is only part of the cost; the second write is also disadvantaged by random-access behavior. The source additionally notes that “splitting bandwidth between two streams also reduces peak throughput,” situating the issue as both a latency and throughput problem (Vo et al., 2012).

3. Recovery semantics under WAL + Data

Recovery in the baseline design is also affected by the separation between log and data. The recovery procedure must “(a) locate and read the data files’ metadata, (b) scan the WAL to find committed but not-yet-applied updates, and (c) replay each into the data files” (Vo et al., 2012). Thus restart is not a matter of reading a single authoritative structure; it requires reconciliation between the WAL and the persistent data files.

The recovery-time model is given as

$$T_{\text{recover,WAL+Data}} \approx L/B_{\text{seq}} + N_{\text{dirty}}\cdot(s_{\text{data}} + W/B_{\text{rnd}}),$$

where $I/O_{\text{user}} = W$0 is the log size and $I/O_{\text{user}} = W$1 is the number of updates that must be replayed (Vo et al., 2012). The first term represents sequential WAL scanning; the second term represents applying updates into data files using the slower random-write path.

The source emphasizes the operational consequence: “Large $I/O_{\text{user}} = W$2 or $I/O_{\text{user}} = W$3 ⇒ slow restart before new requests can be served” (Vo et al., 2012). This is a concise statement of why baseline–log physical separation is not only a steady-state write-path issue but also a restart-path issue.

4. Contrast with log-only storage

The LogBase design is presented as a direct rejection of baseline–log physical separation. It “replaces WAL+Data with a single append-only log that is the sole data repository,” while tablet servers are stateless except for three items: “A single log instance stored in HDFS (replicated 3× for durability),” “An in-memory multiversion index per column-group/tablet,” and “An optional small read-cache of recent records” (Vo et al., 2012).

The unified log is “chopped into 64 MB segments in HDFS,” each segment is “a strictly sequential file,” and each log entry is

$I/O_{\text{user}} = W$4

with $I/O_{\text{user}} = W$5 (Vo et al., 2012). The decisive statement is that “There is no separate data file—every write is a single append.”

The in-memory index is a B-link tree with entries

$I/O_{\text{user}} = W$6

and

$I/O_{\text{user}} = W$7

It is “Prefix-clustered by rowKey, suffix-ordered by ts,” allowing current-version lookup via the largest $I/O_{\text{user}} = W$8 and historical lookup by bounding $I/O_{\text{user}} = W$9 (Vo et al., 2012). In the write path, the log is appended first, then the index is updated with the resulting segment location. The narrative summarizes the consequence as “No second write to a data file ⇒ write amplification $I/O_{\text{log}} = W$0 and purely sequential” (Vo et al., 2012).

Recovery is correspondingly simplified. “Because the log is the data, recovery only needs to re-build the in-memory index.” With checkpointing, each index is periodically flushed to HDFS together with a checkpoint record $I/O_{\text{log}} = W$1; on restart, the system loads the index file, reads the checkpoint, and scans forward from that point in the log (Vo et al., 2012). The associated complexity is

$I/O_{\text{log}} = W$2

where $I/O_{\text{log}} = W$3 is the index-file size and $I/O_{\text{log}} = W$4 is the size of the log after checkpoint. The comparison drawn in the source is that “Because $I/O_{\text{log}} = W$5 and $I/O_{\text{log}} = W$6 (only the tail since checkpoint), LogBase recovers much faster” (Vo et al., 2012).

5. Experimental characterization

The paper reports an implementation “by forking HBase 0.90.3,” evaluated on “24× commodity nodes (quad-core, 8 GB RAM, 1 Gbps NIC)” and compared against HBase and an “LRS: a RAMCloud-like record store on disks + LevelDB LSM-tree indexes” (Vo et al., 2012). These experiments are important because they operationalize the performance consequences of retaining or eliminating baseline–log physical separation.

For mixed YCSB workloads on 3–24 nodes, the reported settings are “95% updates; 75% updates” with “1 M keys per node, 1 KB records, Zipfian=1.0.” Throughput and average latencies are said to “scale nearly linearly.” Sample numbers at 12 nodes with 95% updates are:

System	Throughput	Write Latency
HBase	40 K ops/s	0.20 ms
LRS	55 K ops/s	0.18 ms
LogBase	60 K ops/s	0.15 ms

Under the same conditions, the reported read latencies are 0.12 ms for HBase, 0.10 ms for LRS, and 0.09 ms for LogBase (Vo et al., 2012).

For transactional TPC-W on 3–12 nodes, the paper uses Browsing (5% updates), Shopping (20%), and Ordering (50%). It reports that under browsing and shopping, “LogBase scales to ∼8 K TPS at 12 nodes (flat latency 1–2 ms),” while the Ordering mix experiences more MV-OCC conflicts but “still out-performs HBase by ~30%” (Vo et al., 2012).

Crash recovery results are presented both with and without checkpointing. “Without checkpoint: scan entire log (600 MB→900 MB) ⇒ 120–160 s”; “With 500 MB checkpoint: reload index + scan only 100 MB ⇒ 25–35 s”; and the summarized “Speed-up ≈ 4×” (Vo et al., 2012). In the paper’s own summary table for the 12-node write-heavy mix, recovery times are 120 s for HBase, 80 s for LRS, and 30 s for LogBase.

6. Trade-offs, limits, and interpretation

The paper does not present the removal of baseline–log physical separation as cost-free. Several trade-offs are listed explicitly. First, although LogBase reduces write amplification from “WAL+Data $I/O_{\text{log}} = W$7” to “LogBase $I/O_{\text{log}} = W$8,” it also introduces a different read profile: “pre-compaction range scans in LogBase can be slower; compaction sorts and clusters log so later range scans ≈ 2× faster” (Vo et al., 2012).

Second, the in-memory multiversion index has a concrete memory cost. “Each entry is ≈ 24 bytes. With 40% of a 1 GB heap ⇒ ~17 million entries,” and the trade-off summary restates this as “~24 bytes per record in index ⇒ 17 M recs in 1 GB mem. Can tier to disk via LSM-tree if needed” (Vo et al., 2012). This makes clear that eliminating baseline–log physical separation shifts part of the burden from duplicated persistence to indexing memory.

Third, the recovery simplification has a specific scope. In restart processing, “All uncommitted writes are simply skipped and will be purged in the next compaction” (Vo et al., 2012). This clarifies a common misconception: removing the data-file path does not remove recovery logic; rather, it changes recovery into index reconstruction plus tail scanning.

Taken together, the paper’s conclusion is that “by eliminating the separate data store, writing each record only once (purely sequentially), and relying on an in-memory multiversion index plus lightweight MV-OCC,” LogBase demonstrates “1.5–2 × higher write throughput and 4 × faster crash recovery, at the cost of modest memory for indexing and a background log-compaction phase to restore range-scan performance” (Vo et al., 2012). This suggests that baseline–log physical separation is best understood as an architectural baseline against which log-only storage systems quantify the penalties of dual persistence and the benefits of unifying the persistence path.