Request-Only Optimizations (ROO)

• Request-Only Optimizations (ROO) are techniques that treat the entire user request—including both user and item features—as the primary unit, reducing duplication in data processing and storage.
• ROO achieves significant efficiency gains by deduplicating computation, enabling faster training and lower resource costs, especially in recommendation systems where k-fold savings are reported.
• The approach is versatile, applied in LLM serving, Byzantine replication, transactional memory, and caching, ensuring improved throughput and quality via explicit request semantics.

Request-Only Optimizations (ROO) denotes a family of techniques that make the request itself the primary unit of optimization. In the most explicit usage, ROO in recommendation systems treats a user request, rather than an individual impression, as the unit of training data, data processing, and model computation (Guo et al., 24 Jul 2025). Closely related uses appear in batched LLM serving with request-owned mutable state, in adaptive request insertion for waterfall recommender systems, in read-only fast paths for Byzantine state machine replication, in durable read-only transactions on persistent hardware transactional memory, and in caching algorithms that operate from request-derived noisy estimates rather than exact request streams (Yang et al., 27 May 2026, Qian et al., 2022, Berger et al., 2021, Barreto et al., 2024, Mazziane et al., 2023). This suggests a broader systems pattern: computational and storage redundancy can be removed only if the semantics attached to each request—ownership, versioning, consistency, resource budget, or locality—remain explicit.

1. Defining abstractions and design space

The technical meaning of ROO is context dependent. In recommendation training, a request is written as $R=(u,\{i_1,i_2,\dots,i_k\})$, where $u$ denotes user features and each $i_j$ denotes item features and feedback. In read-write test-time training, each token decode step is exposed as an event

$$e_i=(\mathrm{owner}_i,\mathrm{version}_i,\mathrm{effect}_i),$$

with $$\mathrm{effect}_i\in\{\mathsf{READ},\mathsf{WRITE}\}$$. In PBFT read-only optimization, clients issue a [READ](https://www.emergentmind.com/topics/reconstruction-and-alignment-of-text-descriptions-read)(r,ts) directly to replicas and accept a value after collecting a quorum of matching replies. In DUMBO, read-only transactions perform no updates but must still observe a crash-consistent view. In NFPL-based caching, the online policy acts on noisy request estimates $\hat r_t$ rather than exact request vectors (Guo et al., 24 Jul 2025, Yang et al., 27 May 2026, Berger et al., 2021, Barreto et al., 2024, Mazziane et al., 2023).

Across these settings, the recurring abstractions are request identity, request-local state, request-local observations, and request-local resource accounting. The optimization target is not merely lower latency or lower storage, but lower redundancy under semantic constraints. A plausible implication is that ROO is best understood not as a single algorithmic template but as a design doctrine: represent what is private to a request only once, batch or shortcut only compatible request phases, and preserve the invariants that determine correctness.

2. Request-level data and model co-design in recommendation systems

In the recommendation setting, ROO is a training and modeling paradigm that treats a user request as the unit of data. The contrast is with impression-level training, where each item in a request becomes a separate example and duplicates the entire user feature vector. If $B_{RO}$ is the number of requests in a mini-batch and $B_{NRO}$ is the total number of impressions, typically $B_{NRO}\approx k\cdot B_{RO}$. Under impression-level storage,

$S_{\text{impr}}=B_{NRO}(S_{RO}+S_{NRO}),$

whereas request-only storage is

$u$0

The resulting storage reduction ratio is

$u$1

The same deduplication appears in preprocessing and embedding fan-out. If $u$2 is the cost of user-side processing and $u$3 is the cost of item-side processing, then impression-level cost per mini-batch is $u$4, while request-only cost is $u$5. The theoretical speedup on RO-side work is therefore $u$6 (Guo et al., 24 Jul 2025).

The data pipeline buffers events by request, accumulates all impressions for that request, and emits a single training sample with separate RO and NRO fields. Downstream preprocessing flattens these into two variable-length tensors: a $u$7 dense tensor plus a KeyedJaggedTensor for the RO side, and a $u$8 dense tensor plus a KeyedJaggedTensor for the NRO side. This data representation is paired with ROO-friendly architectures. In two-tower and early-stage ranking models, the user tower depends only on RO features and is computed once per request, then shared across the $u$9 impressions. UserArch applies a Linear Compress Embedding layer on the RO side: $i_j$0 with $i_j$1, $i_j$2, and amortized cost $i_j$3. For autoregressive or generative recommenders, impression-level training re-encodes the user sequence for each target item at cost

$i_j$4

whereas ROO encodes the concatenated sequence once at cost

$i_j$5

For $i_j$6, $i_j$7, $i_j$8, this is a $i_j$9 saving, and the production implementation is a Hierarchical Sequential Transduction Unit (HSTU).

Across three major Meta recommendation products, the reported effects span storage, throughput, and quality:

Area	Efficiency result	Quality/result highlight
Training sample volume	+43%–150%	prior down-sampling eliminated
Retrieval	+48% to +570% throughput	HSTU (ROO) at 0.99× relative FLOPs/example; Recall@100 +3.87% and +2.41%; Topline +4.77%
ESR and LSR	ESR: +125% to +266%; LSR: +32% to +100%	NE reductions and statistically significant topline lifts

The retrieval comparison on Product B is especially notable. Relative to a baseline at $$e_i=(\mathrm{owner}_i,\mathrm{version}_i,\mathrm{effect}_i),$$0 FLOPs/example, HSTU trained at impression level uses $$e_i=(\mathrm{owner}_i,\mathrm{version}_i,\mathrm{effect}_i),$$1 FLOPs/example and yields Recall@100 gains of $$e_i=(\mathrm{owner}_i,\mathrm{version}_i,\mathrm{effect}_i),$$2 and $$e_i=(\mathrm{owner}_i,\mathrm{version}_i,\mathrm{effect}_i),$$3, whereas HSTU under ROO uses $$e_i=(\mathrm{owner}_i,\mathrm{version}_i,\mathrm{effect}_i),$$4 FLOPs/example with Recall@100 gains of $$e_i=(\mathrm{owner}_i,\mathrm{version}_i,\mathrm{effect}_i),$$5 and $$e_i=(\mathrm{owner}_i,\mathrm{version}_i,\mathrm{effect}_i),$$6, plus $$e_i=(\mathrm{owner}_i,\mathrm{version}_i,\mathrm{effect}_i),$$7 topline. In ranking, reported Normalized Entropy deltas range from $$e_i=(\mathrm{owner}_i,\mathrm{version}_i,\mathrm{effect}_i),$$8 to $$e_i=(\mathrm{owner}_i,\mathrm{version}_i,\mathrm{effect}_i),$$9, with corresponding lifts in consumption and engagement. The central significance is that request-level deduplication is not confined to storage; it changes the feasible model class by making more expensive RO-side architectures trainable at production scale.

3. Adaptive request insertion in waterfall recommender systems

A different request-centric line of work concerns when to issue an additional paging request inside a waterfall recommender feed. The formal problem defines a set of decision cases $$\mathrm{effect}_i\in\{\mathsf{READ},\mathsf{WRITE}\}$$0, local context features $$\mathrm{effect}_i\in\{\mathsf{READ},\mathsf{WRITE}\}$$1, user $$\mathrm{effect}_i\in\{\mathsf{READ},\mathsf{WRITE}\}$$2, behavior history $$\mathrm{effect}_i\in\{\mathsf{READ},\mathsf{WRITE}\}$$3, and binary action $$\mathrm{effect}_i\in\{\mathsf{READ},\mathsf{WRITE}\}$$4, where $$\mathrm{effect}_i\in\{\mathsf{READ},\mathsf{WRITE}\}$$5 means inserting a request. The edge-side policy $$\mathrm{effect}_i\in\{\mathsf{READ},\mathsf{WRITE}\}$$6 maximizes cumulative feedback subject to a global budget $$\mathrm{effect}_i\in\{\mathsf{READ},\mathsf{WRITE}\}$$7: $$\mathrm{effect}_i\in\{\mathsf{READ},\mathsf{WRITE}\}$$8 subject to

$$\mathrm{effect}_i\in\{\mathsf{READ},\mathsf{WRITE}\}$$9

With approximately constant request cost $\hat r_t$0, the problem reduces to a 0–1 knapsack: $\hat r_t$1 The optimization target is therefore the conditional uplift in purchase rate generated by an inserted request (Qian et al., 2022).

AdaRequest decomposes this problem into user-intention modeling, causal uplift estimation, and budgeted decision making. The CUBE module fuses exposed items $\hat r_t$2, session clicks $\hat r_t$3, long-term clicks $\hat r_t$4, candidate items $\hat r_t$5, and coarse user/context features through GRUs, self-attention, and scaled dot-product attention. For example,

$\hat r_t$6

and the session-click matching weights are

$\hat r_t$7

The final fused representation is

$\hat r_t$8

CREST then estimates uplift under the Conditional Independence Assumption,

$\hat r_t$9

Two heads share the CUBE backbone: $B_{RO}$0 so that

$B_{RO}$1

The DRP decision layer ranks cases by $B_{RO}$2 and selects the top $B_{RO}$3 per time window, using the previous period’s threshold as a real-time proxy.

The reported experiments use $B_{RO}$4 request-invocation records, $B_{RO}$5 distinct users, and $B_{RO}$6 items from Taobao’s “Guess You Like” waterfall feed. Offline, AdaRequest achieves Qini AUUC $B_{RO}$7 versus OneModel $B_{RO}$8 and TwoModel $B_{RO}$9, Qini@50% $B_{NRO}$0 versus $B_{NRO}$1, and AUC $B_{NRO}$2 versus $B_{NRO}$3. Online, PR@10 improves by $B_{NRO}$4 versus PoolR $B_{NRO}$5, and A/B tests report a $B_{NRO}$6 lift in GMV under a $B_{NRO}$7 to $B_{NRO}$8 QPS allowance. Unlike request-level data deduplication, this variant of request optimization concerns the timing of extra requests under a resource cap; the shared idea is that requests are treated as optimizable interventions rather than passive transport events.

4. Request-owned mutable state in batched LLM serving

RW-TTT addresses a serving problem created by test-time training. In TTT, each request carries mutable model-side state—fast weights, low-rank parameter deltas, or streaming learner state. Standard batched LLM serving assumes shared static weights; serial execution is correct but slow, while naive batching can corrupt per-request state. RW-TTT formulates this as read-write TTT serving. Each decode event is tagged by owner, version, and effect, and two events are batch-compatible iff they share the same effect, run the same backend operator shape, and their versions match the runtime’s committed versions: $B_{NRO}$9 Grouping is performed by a key $B_{NRO}\approx k\cdot B_{RO}$0, so that only compatible phases are dispatched together (Yang et al., 27 May 2026).

The crucial invariant lies in the commit path. After a WRITE batch, the backend returns updated state slices $B_{NRO}\approx k\cdot B_{RO}$1 aligned with an owner map $B_{NRO}\approx k\cdot B_{RO}$2. For each batch slot $B_{NRO}\approx k\cdot B_{RO}$3,

$B_{NRO}\approx k\cdot B_{RO}$4

Each request’s new version is therefore published only to its owner. The throughput model defines

$B_{NRO}\approx k\cdot B_{RO}$5

while sequential serving is

$B_{NRO}\approx k\cdot B_{RO}$6

with speed-up

$B_{NRO}\approx k\cdot B_{RO}$7

On one A100-equivalent GPU with eight In-Place-TTT streams, the reported aggregate throughput is $B_{NRO}\approx k\cdot B_{RO}$8 tok/s, compared with $B_{NRO}\approx k\cdot B_{RO}$9 tok/s for sequential TTT and $S_{\text{impr}}=B_{NRO}(S_{RO}+S_{NRO}),$0 tok/s for $S_{\text{impr}}=B_{NRO}(S_{RO}+S_{NRO}),$1 batch-1 replicas under an equal $S_{\text{impr}}=B_{NRO}(S_{RO}+S_{NRO}),$2 GiB memory budget. The speed-ups are $S_{\text{impr}}=B_{NRO}(S_{RO}+S_{NRO}),$3 over sequential and $S_{\text{impr}}=B_{NRO}(S_{RO}+S_{NRO}),$4 over replicas, with peak memory $S_{\text{impr}}=B_{NRO}(S_{RO}+S_{NRO}),$5 GiB matching the replica baseline. The method preserves behavior on RULER and passes owner/version checks. The reported trade-offs are phase skew, state movement overhead that grows with the number of owners, and smaller effective batch sizes on WRITE-heavy workloads. The broader importance is that mutable per-request adaptation can coexist with accelerator batching only if ownership and versioning are elevated to explicit runtime contracts.

5. Read-only requests in Byzantine state machine replication

PBFT’s read-only optimization is a classical fast path for pure reads. In the normal path, an ordinary request proceeds through REQUEST, PRE-PREPARE, PREPARE, COMMIT, and then replica replies; the client accepts after collecting $S_{\text{impr}}=B_{NRO}(S_{RO}+S_{NRO}),$6 matching replies. Under the read-only optimization, the client broadcasts READ(r,ts) to all $S_{\text{impr}}=B_{NRO}(S_{RO}+S_{NRO}),$7 replicas, each replica executes the read immediately and returns a signed reply, and the client accepts after collecting a quorum $S_{\text{impr}}=B_{NRO}(S_{RO}+S_{NRO}),$8, namely $S_{\text{impr}}=B_{NRO}(S_{RO}+S_{NRO}),$9 matching replies. The optimization reduces latency from five communication steps to two, and linearizability follows from quorum intersection: any read quorum of size $u$00 intersects the update quorum so that at least one correct replica has seen the latest committed update (Berger et al., 2021).

The same work shows that this optimization can violate liveness. In the isolating-leader attack, a Byzantine primary sends PRE-PREPARE(o) only to $u$01 replicas, omitting $u$02 honest replicas. The primary and $u$03 others complete PREPARE/COMMIT, execute the update, and reply, but the client obtains at most $u$04 matching replies and never reaches the $u$05 required by the read-only fast path. Linearizability is preserved, but a correct client can remain blocked indefinitely.

Two repairs are given. Decision broadcasting causes any replica that commits $u$06 with a valid $u$07-signature COMMIT certificate to broadcast FWD-DECISION(c,v,\Gamma) to all replicas; any replica that receives a valid proof decides, executes, and re-broadcasts. Decision forwarding makes this repair on demand: an isolated replica that collects $u$08 ACCEPTs but lacks the decision sends REQ-DECISION(c) to $u$09 peers, and any decided peer returns FWD-DECISION(c,v,\Gamma). Both preserve safety by quorum-intersection arguments, and both restore liveness by ensuring that isolated replicas can eventually obtain a valid decision proof.

The BFT-SMaRt prototype quantifies the trade-off. In LAN, ROO read latency stays around $u$10 ms up to saturation, while ordered reads jump above $u$11 ms. In WAN, median ROO reads are $u$12–$u$13 ms, versus $u$14–$u$15 ms for ordered reads; on average, ROO is $u$16 faster at the median and $u$17 faster at the $u$18th percentile. Decision broadcasting adds an extra all-to-all broadcast and reduces throughput by about $u$19–$u$20, with latency up by about $u$21. Decision forwarding matches baseline performance in the fault-free case, is about $u$22 faster than broadcasting in LAN under attack, and is worse in WAN under attack because it incurs an additional REQ-DECISION round-trip. A common misconception corrected by this work is that read-only fast paths are automatically benign; the result shows that bypassing ordering can preserve linearizability yet still compromise liveness.

6. Durable read-only transactions on persistent hardware transactional memory

DUMBO studies read-only transactions in persistent hardware transactions (PHTs), where a read-only transaction performs no writes but must still observe a crash-consistent view of persistent memory. Two bottlenecks dominate prior designs such as SPHT and Pisces. First, HTM imposes read-capacity bounds: if the read set exceeds the transactional cache, the transaction aborts and may fall back to a single-global-lock path. With $u$23 denoting the probability of capacity abort on $u$24 threads,

$u$25

Second, durable opacity forces a post-commit durability wait. If $u$26 is the set of observed or concurrent updaters,

$u$27

As thread count rises, these effects can collapse throughput (Barreto et al., 2024).

DUMBO exploits IBM POWER9 support for suspending and resuming both load and store tracking in HTM. The system maintains a copy-on-write shadow of the persistent heap in DRAM, per-thread redo logs in PM, and a global circular array of durability markers. Threads export

$u$28

and a logical durability timestamp $u$29. The key operational idea is that suspend/resume allows a transaction to expand its read footprint without capacity aborts and to perform out-of-path synchronization without aborting. IsolationWait scans currently active transactions and enforces the invariant that concurrent update transactions do not read one another’s writes; DurWait prunes the wait set to earlier-committed nondurable transactions.

This changes the asymptotic read-only cost model. For SPHT,

$u$30

whereas for DUMBO,

$u$31

The experimental evaluation uses an IBM POWER9 system with 2 sockets, $u$32 cores each, SMT $u$33, for $u$34 virtual cores, with $u$35 GB DRAM and HTM flushes emulating a $u$36 ns PM latency. On TPC-C StockLevel, where read sets are large, SPHT and HTM throughput goes to zero due to capacity aborts and fallback, while DUMBO and Pisces scale to $u$37 threads, with DUMBO at $u$38 relative throughput versus Pisces $u$39. On OrderStatus, DUMBO matches HTM up to $u$40 threads and then continues scaling. On update-only Payment, DUMBO reports an $u$41–$u$42 speedup over SPHT; on Delivery, DUMBO-SI yields $u$43–$u$44 higher throughput than SPHT/HTM; and in read-dominated mixed workloads, overall throughput is $u$45–$u$46 higher. The reported headline result is up to $u$47 over the state of the art.

7. Request-only observations and insertion policies in caching

Caching work uses request-only information in two distinct ways. NFPL assumes that the cache may not know the exact request sequence and instead observes only noisy request estimates. Time is slotted, each slot collects a batch of $u$48 requests, the request vector is $u$49, and the cache decision $u$50 marks files missing from the cache. The slot cost is

$u$51

Against the best static hindsight cache $u$52, expected regret is

$u$53

NFPL chooses

$u$54

where $u$55 is an unbiased noisy estimate and $u$56. Under unbiasedness and uniform boundedness assumptions, the final guarantee is

$u$57

Special cases include NFPL-Fix and NFPL-Var, with bounds $u$58 and $u$59, respectively (Mazziane et al., 2023).

Empirically, NFPL is evaluated on Zipf-distributed requests, a real Akamai CDN trace, and an adversarial round-robin trace. On stationary Zipf and Akamai traces, LFU approaches OPT quickly and FPL/NFPL converge more slowly due to noise; on adversarial round-robin, only FPL and NFPL attain near-OPT performance, while LRU and LFU fail. A notable reported observation is that lower sampling can help on round-robin because the added randomness outperforms a static bias. Practical guidance is to set $u$60, update every $u$61 requests, and use approximate counters only when they preserve unbiasedness or controlled bias.

A second caching line studies “cache on $u$62 request” insertion under elastic storage and bandwidth costs. If storage incurs cost $u$63 per unit time and misses incur bandwidth cost $u$64, with $u$65, the offline-optimal cost for a request sequence with inter-request gaps $u$66 is

$u$67

Several online policies are analyzed: Always-on $u$68, Always-on $u$69, Single-window on $u$70, and Dual-window on $u$71. With $u$72, Always-on $u$73 has competitive ratio at most $u$74; Always-on $u$75 and Single-window on $u$76 have competitive ratio at most $u$77; and Dual-window on $u$78 with $u$79 has competitive ratio at most $u$80. Across exponential, Erlang, deterministic, and Pareto inter-request distributions, the single-parameter dual-window policy $u$81, $u$82, is reported as a robust compromise. Under exponential arrivals, the peak ratio is approximately $u$83, and on a campus-network YouTube trace the window-2nd policy yields about $u$84–$u$85 total-cost savings over “cache on 1st” (Carlsson et al., 2018).

Taken together, these caching results show two complementary meanings of request-only optimization. One treats request information as incomplete and still seeks no-regret behavior under noise; the other uses request counts themselves as admission signals and optimizes insertion thresholds against elastic costs. In both cases, direct access to full future demand is unavailable, so the request sequence becomes the operative object for online control rather than a fully known workload trace.