Cache Warming in Serverless Environments

• Cache warming is a technique that pre-populates memory with essential data or program state to mitigate cold-start latency in serverless functions.
• It employs live dependency image pools using CRIU-style checkpoints and on-demand page migration to share common libraries across functions.
• Experimental evaluations demonstrate up to 2.2× latency improvements and 90% memory savings, highlighting its impact on serverless performance.

Cache warming refers to a family of techniques designed to pre-populate memory with data or program state that is likely to be needed imminently, thereby reducing access latency by preventing on-demand (cold) loading. In the context of serverless computing, cache warming specifically targets the reduction of “cold-start” latency—a critical performance bottleneck wherein function containers require significant time to initialize their runtime environments and dependencies upon invocation after an idle period. Provider-side cache warming mechanisms, such as WarmSwap, facilitate efficient sharing of dependency images across serverless functions, thereby addressing cold-start performance for workloads with substantial dependency footprints (Li et al., 2024).

1. Architectural Foundations: Live Dependency Image Pools

WarmSwap structures cache warming at the provider level by leveraging “live dependency images” (LDIs), which are CRIU-style memory checkpoints of running processes that have already booted requisite runtimes (e.g., Python 3.9) and loaded commonly used library bundles (e.g., NumPy, Torch). Each worker node maintains a bounded pool of such LDIs in RAM. The key architectural components include:

• Dependency Pool: Hosts a limited set of LDIs per worker node.
• Dependency Manager: Tracks page locations, file descriptors, and metadata for each LDI and orchestrates on-demand migration.
• Page Server: Transfers relevant memory pages to the requesting container during function startup.
• Function Container (User Side): Employs a Migration Client that, upon cold start, requests LDI metadata, and then invokes CRIU restore in “lazy” or “bulk” (see §6) fashion, using userfaultfd to trigger page loading on demand.

This architecture supports both setup (one-time CRIU “dump” per LDI) and runtime (LDI migration/restore to user containers) phases. The approach avoids per-function prebaking, amortizes initialization costs, and constrains in-RAM usage by sharing a single LDI per software stack across multiple functions [(Li et al., 2024), Fig. 3].

2. Performance Metrics and Formulations

Cache warming efficiency is quantified through precise latency and memory utilization metrics:

• Cold-Start Probability and Frequency: With arrival rate $\lambda$ and keep-alive window $T$:

$P_{no\_inv} = e^{-\lambda T} \qquad (1)$

$$E_{cs}(\lambda) = D\,\lambda\,e^{-\lambda T} \qquad (2)$$

where $P_{no\_inv}$ is the probability of no invocation during $T$, and $E_{cs}$ is the expected number of cold starts in duration $D$.

• Latency Reduction: For legacy (baseline) and WarmSwap cold-start times $T_{c,\mathrm{old}}$ and $T_{c,\mathrm{new}}$:

$T$0

Empirical results (Section 4.2, (Li et al., 2024)) indicate $T$1–$T$2 for large dependency functions.

• Shared-Pool Cache Savings:

$T$3

For $T$4 functions sharing an LDI of size $T$5, $T$6 (90% savings). Simulations confirm 88% real-world pool space reduction [(Li et al., 2024), Fig. 8].

3. Implementation Workflow and Semantics

WarmSwap’s cache warming follows a two-phase approach:

• Setup Phase: Each dependency bundle is booted once, post-runtime and library initialization but before user logic. “criu dump” captures a minimal on-disk checkpoint for each LDI. This process generates a reusable, cold-stored LDI for subsequent RAM pool instantiation.
• Runtime Phase: At cold start, the Migration Client requests LDI metadata. CRIU then reconstitutes memory, file descriptors, and mappings, employing userfaultfd to manage on-demand bulk-or-lazy page pulling from the standing LDI Page Server.

The system ensures that only shared libraries are transferred—user state remains isolated. Pool size is strictly bounded, and capacity is managed via standard eviction policies (e.g., LRU).

4. Experimental Design and Benchmarks

Evaluation employs the FunctionBench suite, selecting Python serverless functions spanning a dependency size gradient: from minimal (helloworld, pyase) to dependency-heavy (lr_serving, cnn_serving, rnn_serving). Experiments are conducted on AWS EC2 (r5b.large, 2 vCPUs, 16GB RAM, Amazon Linux, CRIU 3.9, Python 3.9 Lambda). Metrics captured include container boot, runtime+library init, and user code intervals [(Li et al., 2024), Table 1; §4.1].

A separate simulation uses 2-week Azure trace datasets, mapping ten variants of rnn_serving (requiring Python+NumPy+Torch) to real-world cold/warm-invocation patterns under a 15-minute keepalive (Section 4.5). Performance is assessed under both WarmSwap and per-function “prebaking.”

5. Quantitative Results and Comparative Analysis

Key findings:

Function	Baseline Cold Start	WarmSwap Cold Start	Latency Speedup
lr_serving	≈ 5.3 s	≈ 4.4 s	1.2×
cnn_serving	≈ 7.2 s	≈ 4.0 s	1.8×
rnn_serving	≈ 11.6 s	≈ 5.3 s	2.2×

In dependency initialization microbenchmarks, WarmSwap achieves up to 3.2× improvement (Section 4.3, Fig. 5). Warm-start cost is unaffected, and metadata transfer overhead is negligible compared to full checkpoint size (0.9–15 MB vs 8–200 MB, Table 5).

In shared-pool simulation (Azure traces), WarmSwap reduces total memory footprint to ≈ 260 MB from ≈ 2.3 GB (prebaking), achieving 88% memory savings, and delivers a 1.4× reduction in end-to-end cold-start latency (Section 4.5, Fig. 8).

6. Design Tradeoffs and Constraints

WarmSwap’s cache warming effectiveness is highly workload dependent:

• Bulk vs. Lazy Restore: Bulk mode, the default, restores all LDI pages after the first page fault, reducing user-facing pauses and avoiding warm-start penalties. Lazy mode transfers only needed pages on demand, sparing network traffic for light workloads but incurring possible user stalls [§3.3, §4.4].
• Applicability: Overhead in metadata exchange and CRIU operations may exceed baseline for dependency-light functions, indicating that WarmSwap primarily benefits functions with large, shared dependencies.
• Operational Constraints: Requires a homogeneous runtime/library environment (consistent filesystem layout across pool node and function containers).
• Isolation: Only public (library) pages are shared; user code/state is never migrated, maintaining function-level security isolation.
• Interplay with Other Optimizations: WarmSwap complements, but does not subsume, other cold-start mitigation (e.g., Firecracker micro-VMs, container-chunk caching, Catalyzer). It does not fully eliminate cold-start overhead.

7. Broader Context and Implications

Cache warming in serverless platforms enables efficient provisioning of high-dependency workloads and addresses cloud provider memory constraints by consolidating redundant state. By shifting optimization from function-specific prebaking to bundle-level dependency management, WarmSwap enables provider-controlled tradeoffs between memory, latency, and administrative overhead (Li et al., 2024). Transparent sharing of live dependency images offers a practical middle ground between global infrastructure tuning and per-function custom checkpointing.

A plausible implication is that, with widespread adoption, provider-managed live dependency image pools could form a standard operating layer for multi-tenant serverless platforms, enabling composable and memory-efficient cache warming strategies for diverse workloads. The model presumes provider-side support and coordination, and its impact is proportional to dependency bundle commonality across deployed functions.