Onyx-ORAM: Optimized ORAM for SSD-Backed ANN

• The paper presents Onyx-ORAM, a RingORAM-derived design that minimizes I/O request count by rebalancing costs between bandwidth and access count in SSD-based ANN systems.
• Onyx-ORAM leverages innovations like local bucket metadata, full bucket reads during eviction, and a shallow d-ary tree to optimize performance under TEE trust models.
• Experimental results demonstrate up to 5.1× higher throughput and 2.7× lower latency compared to traditional ORAMs, validating its practical efficiency in disk-resident environments.

Searching arXiv for the primary Onyx paper and closely related ORAM references. Onyx-ORAM is the ORAM component of Onyx, a TEE-based private approximate nearest neighbor (ANN) search system for disk-hosted vector indices. It is designed to hide the disk access pattern generated by ANN search when the index resides on external SSDs outside the TEE trust boundary. In the Onyx architecture, the system is divided into two co-designed layers: Onyx-ANNS, which is a bandwidth-efficient ANN search engine, and Onyx-ORAM, which is an access-count-efficient ORAM client over untrusted SSD storage. The defining design claim is an inversion of prior oblivious ANN co-design practice: rather than minimizing access count in ANN and bandwidth in ORAM, Onyx minimizes bandwidth in the ANN layer and access count in the ORAM layer, because ANN search can exploit approximation for bandwidth efficiency, whereas ORAM is argued not to face an analogous fundamental lower bound on I/O request count (Rathee et al., 22 Apr 2026).

1. Position within ORAM and private storage research

Onyx-ORAM belongs to the line of work on oblivious storage that seeks to hide access patterns to outsourced or untrusted storage, but it is specialized to a TEE setting in which the CPU package and enclave memory are trusted while the host and external SSDs are not. In this model, the host can observe the sequence of SSD I/Os issued by the enclave, and the paper treats that observation channel as first-order leakage because access-pattern attacks can recover queries or infer returned nearest neighbors (Rathee et al., 22 Apr 2026).

This specialization distinguishes Onyx-ORAM from general single-server online ORAM analyses. In the standard online ORAM model, bandwidth overhead is subject to an inherent logarithmic lower bound. “Stronger Lower Bounds for Online ORAM” proves that any statistically secure online ORAM with internal memory of size $m$ must perform $\Omega(n\log n)$ probes in expectation on input sequences of length $n$, yielding $\Omega(\log n)$ bandwidth overhead, and that this remains true even when the adversary does not know the boundaries between low-level accesses corresponding to logical requests (Hubáček et al., 2019). The same asymptotic lower bound is obtained for strong computational security, while only an $\omega(1)$ lower bound is proved for weak computational security (Hubáček et al., 2019).

That lower-bound context is directly relevant to Onyx-ORAM’s design emphasis. The Onyx paper does not claim sublogarithmic ORAM bandwidth overhead in the standard online sense. Instead, it argues that the relevant optimization axis for SSD-backed ANN inside a TEE is access count, because SSD latency and IOPS make the number of I/O operations a dominant cost, and because ORAM has no comparable fundamental lower bound on access count. This suggests that Onyx-ORAM should be read not as a refutation of classical ORAM lower bounds, but as a workload- and medium-specific rebalancing of costs under the constraints imposed by disk-hosted ANN search (Rathee et al., 22 Apr 2026).

2. Design objective and architectural role

The Onyx paper frames the central inefficiency of prior oblivious ANN systems as a cross-layer mismatch. DiskANN-style systems are effective in ordinary SSD settings because random 4 KB reads are cheap enough that access count dominates, whereas ORAM amplifies bandwidth by $O(\log N)$, making every extra byte costly. At the same time, many tree-based ORAMs incur many individual I/O requests per logical access, which is particularly harmful on SSDs because latency and throughput are limited by both raw bandwidth and IOPS (Rathee et al., 22 Apr 2026).

Onyx-ORAM addresses this by serving as an access-count-efficient ORAM beneath an ANN layer that has already been made bandwidth-efficient. The paper states that the state-of-the-art ORAM-ANN design minimizes access count at the ANN level and bandwidth at the ORAM level, with each optimization worsening the other resource, leaving the combined system overutilizing both bandwidth and access count. Onyx reverses that allocation of responsibilities: ANN search minimizes bandwidth, while ORAM minimizes access count (Rathee et al., 22 Apr 2026).

Architecturally, Onyx uses two separate ORAM instances. A traversal ORAM stores small blocks containing adjacency lists $N$ plus pruning hints $H$, while a refinement ORAM stores full-precision vectors $F$. The main text states that Setup(I) initializes both ORAMs over the ANN index and loads in-memory index state. Expensive per-block ORAM initialization can be avoided using prior bulk-loading techniques such as BULKOR and EnigMap. The paper focuses primarily on search, but it also supports Insert(p, x_p) and Delete(p), and the appendix discussion says that the decoupled Onyx-ANNS layout extends naturally to FreshDiskANN-like dynamic indices. Updated neighborhoods can be written back piggybacked on ORAM eviction, since any accessed ORAM block must be rewritten anyway (Rathee et al., 22 Apr 2026).

A plausible implication is that Onyx-ORAM is not intended as a fully general ORAM abstraction independent of workload shape. Its design is tuned to the specific access-size distribution induced by Onyx-ANNS, especially the prevalence of small traversal blocks of roughly 256–512 B.

3. Core construction: RingORAM-derived shallow-tree ORAM

Onyx-ORAM is a modified RingORAM, specifically the non-recursive variant of RingORAM with the position map stored in the TEE, and without the XOR trick because the SSD has no compute capability (Rathee et al., 22 Apr 2026). Its main innovation is a “locality-aware shallow tree design” comprising three stacked changes:

1. Store bucket metadata locally and increase bucket size.
2. Fetch entire buckets during evictions.
3. Replace the binary tree by a wide $d$-ary shallow tree (Rathee et al., 22 Apr 2026).

The baseline RingORAM organization is a tree of buckets in which each logical block is assigned a random leaf according to $\Omega(n\log n)$0 and must reside either on the root-to-leaf path of that leaf or in the client stash. Each bucket stores up to $\Omega(n\log n)$1 real blocks and is padded to $\Omega(n\log n)$2 slots with dummies. A regular RingORAM access looks up the assigned leaf, reads metadata from every bucket on that path to identify the correct slot, reads either the real block or a dummy at each level, places the block in the stash, and remaps it to a fresh random leaf. Every $\Omega(n\log n)$3 accesses, RingORAM performs deterministic eviction along a fixed path. Ignoring rare early reshuffles, the paper gives the multipliers

$\Omega(n\log n)$4

and

$\Omega(n\log n)$5

where $\Omega(n\log n)$6 is tree depth (Rathee et al., 22 Apr 2026).

Onyx-ORAM’s first optimization, Local Bucket Metadata (LBM), stores bucket metadata inside the TEE rather than on disk. The metadata stored per bucket includes a version counter $\Omega(n\log n)$7, a permutation array $\Omega(n\log n)$8, validity bits $\Omega(n\log n)$9, a dummy counter $n$0, and an access counter $n$1. This allows the client to determine exactly which slot to read without issuing a metadata read to disk, reducing regular access to one block read per tree level rather than metadata plus block. The paper says this changes the access-count expression to

$n$2

It also enables larger $n$3, since metadata cost no longer scales as disk I/O (Rathee et al., 22 Apr 2026).

The second optimization, Full Bucket Reads during Eviction (FBR), trades bandwidth for fewer I/O requests. Instead of RingORAM’s bandwidth-conscious approach of reading only $n$4 blocks per bucket during eviction, Onyx-ORAM reads the entire bucket—all $n$5 slots—in one sequential I/O. This reduces amortized per-level eviction access cost from roughly $n$6 toward $n$7, giving

$n$8

when $n$9 is sufficiently large (Rathee et al., 22 Apr 2026).

The third optimization, Shallow $\Omega(\log n)$0-ary Tree (ST), is the main structural change. Replacing the binary tree by a $\Omega(\log n)$1-ary tree reduces depth from

$\Omega(\log n)$2

to

$\Omega(\log n)$3

For $\Omega(\log n)$4, the paper states that this yields a $\Omega(\log n)$5 reduction in access count. Wider trees require more frequent eviction, and the paper proves that in $\Omega(\log n)$6-ary RingORAM the maximum eviction period satisfies

$\Omega(\log n)$7

compared with $\Omega(\log n)$8 in the binary case. However, Onyx argues that smaller $\Omega(\log n)$9 also lets $\omega(1)$0 shrink proportionally in large-bucket RingORAM, reducing dummy padding. For $\omega(1)$1, the net effect is reported as a $\omega(1)$2 reduction in access count with essentially no bandwidth penalty relative to the optimized binary version (Rathee et al., 22 Apr 2026).

The paper summarizes the resulting I/O multipliers as follows:

Design	AccessCount	Bandwidth
RingORAM	$\omega(1)$3	$\omega(1)$4
PathORAM $\omega(1)$5	$\omega(1)$6	$\omega(1)$7
PathORAM $\omega(1)$8	$\omega(1)$9	$O(\log N)$0
Onyx-ORAM	$O(\log N)$1	$O(\log N)$2

This table situates Onyx-ORAM between bandwidth-efficient RingORAM and locality-packed PathORAM, with markedly lower access count than either while remaining much closer to RingORAM on bandwidth (Rathee et al., 22 Apr 2026).

4. Protocol state and access procedure

Onyx-ORAM maintains client-side state consisting of a position map $O(\log N)$3, a stash $O(\log N)$4, per-bucket metadata $O(\log N)$5, and a global access counter $O(\log N)$6. The server stores an encrypted $O(\log N)$7-ary tree of depth $O(\log N)$8, with each bucket containing $O(\log N)$9 encrypted slots (Rathee et al., 22 Apr 2026).

For each block address $N$0,

$N$1

where $N$2 is the assigned leaf and $N$3 identifies the bucket level and logical slot currently holding the block, or $N$4 if the block is in the stash. The cached $N$5 is critical because it lets ReadPath directly fetch the correct slot at the matching level and dummies elsewhere (Rathee et al., 22 Apr 2026).

For bucket $N$6, $N$7 stores:

• $N$8: a monotonic version counter,
• $N$9: a pseudorandom permutation from logical slots to physical disk offsets,
• $H$0: validity bits,
• $H$1: which dummy slot to consume next,
• $H$2: number of accesses since the last write, used to trigger early reshuffle at $H$3 (Rathee et al., 22 Apr 2026).

Each slot is authenticated-encrypted using AES-GCM with associated data

$H$4

binding ciphertexts to bucket $H$5, physical slot offset $H$6, and version (Rathee et al., 22 Apr 2026).

The access protocol Access(a, op, data') proceeds as follows. It reads the current position $H$7 from $H$8, samples a fresh random leaf

$H$9

and immediately remaps the block by setting

$F$0

It then calls ReadPath(leaf_a, lvl_a, slot_a). If the block is not found on disk, it is removed from the stash. For writes, data is replaced by data'. The updated block is inserted into the stash with its new leaf assignment, the global counter is incremented, EvictPath() is called when $F$1, EarlyReshuffle(leaf_a) is invoked, and the block is returned for reads (Rathee et al., 22 Apr 2026).

ReadPath(leaf_a, lvl_a, slot_a) touches exactly one slot per level. At the known matching level $F$2, it reads the known logical slot $F$3; at all other levels it sets $F$4, meaning “read the next unread dummy.” The logical slot is translated to a physical offset by GetOffset, which consumes dummy slots when needed, increments the access count, maps logical slot $F$5 through $F$6, marks the slot invalid after use, and returns the physical offset (Rathee et al., 22 Apr 2026).

EvictPath() computes a deterministic eviction leaf

$F$7

reads the entire bucket at every level on path $F$8, merges readable real blocks into the stash, and then traverses levels bottom-up calling WriteBucket (Rathee et al., 22 Apr 2026). WriteBucket selects up to $F$9 stash blocks assignable to the bucket, removes them from the stash, samples a fresh random permutation

$d$0

increments the version counter, fills the first $d$1 logical slots with real blocks and the rest with dummies, encrypts every physical slot, resets metadata, updates $d$2 entries in $d$3, and writes the whole bucket back as a single I/O (Rathee et al., 22 Apr 2026).

EarlyReshuffle(leaf) checks every bucket on the accessed path and, when

$d$4

reads the whole bucket, rewrites it, and resets metadata. This ensures that no bucket is read too many times without reshuffling (Rathee et al., 22 Apr 2026).

5. Security model, formal guarantee, and analytical bounds

The Onyx paper formalizes security through a disk-access privacy game for a disk-resident ANN system running in a TEE and accessing storage via ORAM. The goal is to hide the physical disk access pattern so that an adversary observing physical disk addresses and ciphertexts cannot distinguish between two equal-size datasets and two matched-type operation sequences. The paper states:

A disk-resident ANN system provides disk-access privacy if no non-uniform PPT adversary can win the security game with probability non-negligibly higher than random guessing (Rathee et al., 22 Apr 2026).

What remains public is explicit. Public leakage includes ORAM and ANN public parameters, the operation type (Search, Insert, Delete), and the number and granularity of logical accesses per operation, because Onyx fixes these by public parameters such as $d$5, $d$6, and block sizes. Thus the system hides which blocks are accessed, but not the public fixed count or type schedule of accesses (Rathee et al., 22 Apr 2026).

The formal theorem is:

$d$7

The proof sketch combines authenticated-encryption security, the fact that Onyx-ANNS makes a fixed number of logical accesses depending only on public parameters, ORAM’s obliviousness guarantee for each logical access, and the claim that Onyx-ORAM’s modifications preserve RingORAM’s invariants and security guarantees. The hybrid proof argues that the physical disk address traces are identically distributed because ReadPath depends on random leaf assignments and hidden local metadata, EvictPath follows deterministic public reverse-lexicographic order, and EarlyReshuffle depends only on public bucket access counts (Rathee et al., 22 Apr 2026).

The main analytical theorem generalizes RingORAM stash bounds to $d$8-ary trees. For a $d$9-ary Onyx-ORAM with $\Omega(n\log n)$00 blocks, bucket capacity $\Omega(n\log n)$01, eviction period $\Omega(n\log n)$02, and depth

$\Omega(n\log n)$03

let

$\Omega(n\log n)$04

If

$\Omega(n\log n)$05

then

$\Omega(n\log n)$06

The paper notes that setting $\Omega(n\log n)$07 recovers the original RingORAM result. A key consequence is again

$\Omega(n\log n)$08

so wider trees require proportionally more frequent eviction (Rathee et al., 22 Apr 2026).

This theorem should be distinguished from the lower bounds for general online ORAM. The lower-bound work shows that statistically secure and strongly computationally secure online ORAMs incur $\Omega(n\log n)$09 bandwidth overhead and that obscuring the boundaries between logical requests does not evade that asymptotic cost (Hubáček et al., 2019). Onyx-ORAM does not dispute that claim. Instead, it modifies the tradeoff between bandwidth consumption and I/O request count within a RingORAM-derived construction under a TEE threat model and an SSD workload.

6. Performance characteristics, evaluation, and relation to prior ORAM directions

The paper’s effective overhead summary for the chosen $\Omega(n\log n)$10 regime is

$\Omega(n\log n)$11

and

$\Omega(n\log n)$12

as reported in Table 1 (Rathee et al., 22 Apr 2026). The appendix further reports measured per-block memory footprints of approximately $\Omega(n\log n)$13 bytes/block for Onyx-ORAM with $\Omega(n\log n)$14, $\Omega(n\log n)$15 bytes/block for RingORAM, $\Omega(n\log n)$16 bytes/block for PathORAM, and $\Omega(n\log n)$17 bytes/block for binary RingORAM with local bucket metadata. This indicates that the $\Omega(n\log n)$18-ary shallow tree is not only an access-count device but also a way to make local metadata economically feasible in TEE memory (Rathee et al., 22 Apr 2026).

The ORAM microbenchmarks were run on AMD EPYC 9554 with AMD SEV-SNP and Micron 7450 MAX SSDs. The primary ORAM evaluation used a 1-SSD configuration with 1 vCPU, 4 GB RAM, 1 SSD slice, 85K IOPS, 330 MB/s read bandwidth, 175 MB/s write bandwidth, and 187.5 GB storage. The evaluation used 20M blocks, matching WIKI-20M scale. Parameters included early reshuffle rate $\Omega(n\log n)$19, $\Omega(n\log n)$20 for RingORAM, $\Omega(n\log n)$21 for Onyx-ORAM, and $\Omega(n\log n)$22 for PathORAM with packed-level parameter $\Omega(n\log n)$23 tuned by block size (Rathee et al., 22 Apr 2026).

At small block sizes of 256–512 B, Onyx-ORAM sustains over 10K logical accesses/s. At 512 B on 1-SSD, it achieves up to $\Omega(n\log n)$24 higher throughput and $\Omega(n\log n)$25 lower latency than RingORAM, and up to $\Omega(n\log n)$26 higher throughput and $\Omega(n\log n)$27 lower latency than locality-optimized PathORAM. At 4-SSD, the advantage remains substantial: $\Omega(n\log n)$28 higher throughput and $\Omega(n\log n)$29 lower latency than RingORAM, and $\Omega(n\log n)$30 higher throughput and $\Omega(n\log n)$31 lower latency than PathORAM (Rathee et al., 22 Apr 2026).

The ablation study validates each component of the design: LBM removes metadata I/O, FBR collapses many eviction reads into one sequential read, and ST with varying arities further improves performance, with $\Omega(n\log n)$32 giving the best tradeoff analytically and experimentally (Rathee et al., 22 Apr 2026).

The end-to-end evaluation attributes a substantial portion of full-system gains to Onyx-ORAM. On small-vector datasets, Onyx-ORAM + DiskANN comes within roughly 10–20% of full Onyx, indicating that ORAM is the main bottleneck there. Replacing Onyx-ORAM with RingORAM or PathORAM while keeping Onyx-ANNS causes slowdowns of $\Omega(n\log n)$33–$\Omega(n\log n)$34 for Ring + Onyx-ANNS and $\Omega(n\log n)$35–$\Omega(n\log n)$36 for Path + Onyx-ANNS (Rathee et al., 22 Apr 2026).

Storage amplification for ORAM alone is reported as about $\Omega(n\log n)$37–$\Omega(n\log n)$38 plaintext, and end-to-end Onyx storage over plaintext index as $\Omega(n\log n)$39–$\Omega(n\log n)$40. The corresponding baselines are $\Omega(n\log n)$41–$\Omega(n\log n)$42 for PathORAM + DiskANN and $\Omega(n\log n)$43–$\Omega(n\log n)$44 for RingORAM (Rathee et al., 22 Apr 2026).

These results place Onyx-ORAM in contrast with other practical ORAM directions. Succinct Oblivious RAM focuses on minimizing server storage overhead, achieving server space $\Omega(n\log n)$45 while retaining worst-case $\Omega(n\log n)$46 bandwidth blowup and small client space in the single-server setting (Onodera et al., 2018). That work is a benchmark on the space-efficiency axis rather than the SSD access-count axis. By contrast, vORAM and the HIRB-based oblivious map construction target practical outsourced key/value maps, variable-size blocks, secure deletion, and bounded history leakage, with practical emphasis on map-layer efficiency and post-compromise guarantees rather than disk-resident ANN search (Roche et al., 2015). Onyx-ORAM is therefore best understood as an ORAM specialized to SSD-backed ANN inside a TEE, rather than a general replacement for succinct ORAMs or ORAM-backed oblivious maps.

7. Scope, limitations, and interpretive cautions

Several limitations are explicit in the paper. First, Onyx-ORAM increases client-side metadata by storing bucket metadata locally. The $\Omega(n\log n)$47-ary tree mitigates this, but it remains a deliberate memory tradeoff and one reason memory use can exceed that of PathORAM-based baselines (Rathee et al., 22 Apr 2026).

Second, full-bucket eviction intentionally spends extra bandwidth to reduce access count. This is favorable in the target SSD regime, especially after dummy-slot shrinkage from the shallow tree, but the tradeoff could be less favorable where bandwidth is the sole bottleneck (Rathee et al., 22 Apr 2026). Third, the design’s strongest advantage occurs at 256–512 B block sizes; at very large block sizes, Onyx-ORAM becomes bandwidth-bound and converges toward RingORAM performance (Rathee et al., 22 Apr 2026). This dependence underscores that the ORAM is co-designed with Onyx-ANNS rather than workload-agnostic.

Fourth, performance depends on practical tuning of $\Omega(n\log n)$48, $\Omega(n\log n)$49, $\Omega(n\log n)$50, $\Omega(n\log n)$51, and the early reshuffle rate. The paper reports that $\Omega(n\log n)$52 is best; $\Omega(n\log n)$53 reduces access count further but harms bandwidth (Rathee et al., 22 Apr 2026). Fifth, the security model excludes TEE side channels such as cache attacks, page-fault leakage, timing side channels, transient execution, and a compromised TEE. The paper is specifically about disk access pattern leakage (Rathee et al., 22 Apr 2026).

Sixth, the evaluation mainly focuses on search. Insertions and deletions are supported, and the dynamic-index appendix indicates likely applicability to FreshDiskANN-like structures, but update-heavy workloads are less fully characterized (Rathee et al., 22 Apr 2026). Seventh, the scheme does not hide everything: the operation type and the public fixed search budget remain visible because logical access count and granularity are fixed by public parameters (Rathee et al., 22 Apr 2026).

A common misconception would be to treat Onyx-ORAM as an argument against established ORAM lower bounds. The evidence does not support that interpretation. The lower-bound result for online ORAM states that statistically secure and strongly computationally secure online ORAMs with small client memory incur $\Omega(n\log n)$54 bandwidth overhead even when adversaries cannot identify request boundaries, so boundary obfuscation is not a route to asymptotically better online ORAM (Hubáček et al., 2019). Onyx-ORAM instead exploits a different degree of freedom: it reorganizes the internal structure of a RingORAM-derived design to reduce I/O request count while accepting a carefully managed bandwidth tradeoff, and it does so in a setting where access count is the dominant practical cost (Rathee et al., 22 Apr 2026).

This suggests a more precise characterization. Onyx-ORAM is a RingORAM-derived, shallow $\Omega(n\log n)$55-ary, access-count-optimized ORAM for hiding SSD access patterns in TEE-based disk-resident ANN search. Its main contribution is not a new asymptotic lower-bound escape hatch, nor a general-purpose ORAM definition, but a concrete protocol and evaluation showing that in SSD-backed ANN workloads, access count can be a more valuable ORAM optimization target than the bandwidth-centric metrics that dominate much of the classical ORAM literature (Rathee et al., 22 Apr 2026).