On-demand Container Loading in AWS Lambda

Abstract: AWS Lambda is a serverless event-driven compute service, part of a category of cloud compute offerings sometimes called Function-as-a-service (FaaS). When we first released AWS Lambda, functions were limited to 250MB of code and dependencies, packaged as a simple compressed archive. In 2020, we released support for deploying container images as large as 10GiB as Lambda functions, allowing customers to bring much larger code bases and sets of dependencies to Lambda. Supporting larger packages, while still meeting Lambda's goals of rapid scale (adding up to 15,000 new containers per second for a single customer, and much more in aggregate), high request rate (millions of requests per second), high scale (millions of unique workloads), and low start-up times (as low as 50ms) presented a significant challenge. We describe the storage and caching system we built, optimized for delivering container images on-demand, and our experiences designing, building, and operating it at scale. We focus on challenges around security, efficiency, latency, and cost, and how we addressed these challenges in a system that combines caching, deduplication, convergent encryption, erasure coding, and block-level demand loading. Since building this system, it has reliably processed hundreds of trillions of Lambda invocations for over a million AWS customers, and has shown excellent resilience to load and infrastructure failures.

• The paper introduces an on-demand container loading architecture that reduces cold-start times and scales to 15,000 containers per second.
• It combines caching, deduplication, erasure coding, and block-level demand loading to achieve up to a 23x storage reduction while ensuring data security with convergent encryption.
• The methodology leverages a multi-tiered caching hierarchy and a generational garbage collection mechanism to optimize resource utilization and enhance scalability.

On-Demand Container Loading in AWS Lambda: A Technical Overview

The paper presents a comprehensive exploration of the system architecture and mechanisms developed for on-demand container loading within AWS Lambda. AWS Lambda, a Function-as-a-Service (FaaS) platform, initially supported function deployments within 250MB limits. In response to increasing demand for extensive deployments, AWS introduced support for deploying container images up to 10GiB, posing significant challenges related to scalability, performance, and security.

Technical Challenges and Solutions

As AWS Lambda evolved to support larger container images, the primary challenge was maintaining cold-start times while scaling rapidly. The system needed to manage the deployment and execution of containers at rates that could reach 15,000 containers per second.

The AWS team identified three key properties that facilitated efficient data management: cacheability, commonality, and sparsity.

1. Cacheability: The workload's high cache potential implies repetitive access patterns.
2. Commonality: The widespread use of common base layers allows deduplication.
3. Sparsity: A significant portion of container data is often unnecessary at startup.

The novel solution integrated caching, deduplication, erasure coding, and block-level demand loading. A distinct aspect of this approach is a per-worker local agent that performs block-level data operations, which enhances the efficiency of loading only essential data at runtime.

Deduplication and Security

Deduplication leverages convergent encryption to ensure security without key-sharing among users. With 80% of new Lambda functions resulting in no unique chunks, deduplication significantly reduces data storage and movement, achieving up to a 23x reduction in storage. The convergent encryption scheme is pivotal, allowing deduplication at the block level while maintaining isolation and security, using SHA256-based key derivation for encrypting data blocks.

Tiered Caching Architecture

AWS Lambda employs a multi-tiered caching hierarchy to manage container images efficiently. The architecture consists of:

• On-Worker Cache: For immediate access to frequently used data.
• AZ-Level Distributed Cache: Reduces latency with a cache hit rate typically at 99.9%.
• Back-End Storage (S3): Provides durability for data when caches are exhausted.

The AZ-level cache uses erasure coding, following a 4 of 5 code, to balance between storage overhead and latency. This design addresses issues related to tail latency, hit rate drops, and throughput limitations.

Garbage Collection and Scaling

An innovative garbage collection mechanism is used, based on the generational approach, maintaining robustness against data loss while allowing deduplication scaling. The system avoids traditional central directory maintenance, leveraging root-based migration to manage chunk data efficiently.

Implications and Future Directions

This system has shown resilience and scalability, processing hundreds of trillions of request invocations for AWS Lambda. The practical implications include reduced latency, optimized resource utilization, and improved scalability for large-scale container deployments.

Theoretically, the paper suggests broader applications of similar techniques in other cloud environments, such as MicroVM-based operating system integration. Future work will focus on further latency optimization for AWS Lambda SnapStart and enhancing scalability measures.

Conclusion

The AWS Lambda on-demand container loading system exemplifies an impressive integration of caching, encryption, and deduplication technologies to meet modern cloud computing demands. This paper provides a blueprint for managing large-scale deployment complexity, with valuable insights for systems researchers interested in FaaS and serverless environments.