Cloud Programming Simplified: A Berkeley View on Serverless Computing (1902.03383v1)

Abstract: Serverless cloud computing handles virtually all the system administration operations needed to make it easier for programmers to use the cloud. It provides an interface that greatly simplifies cloud programming, and represents an evolution that parallels the transition from assembly language to high-level programming languages. This paper gives a quick history of cloud computing, including an accounting of the predictions of the 2009 Berkeley View of Cloud Computing paper, explains the motivation for serverless computing, describes applications that stretch the current limits of serverless, and then lists obstacles and research opportunities required for serverless computing to fulfill its full potential. Just as the 2009 paper identified challenges for the cloud and predicted they would be addressed and that cloud use would accelerate, we predict these issues are solvable and that serverless computing will grow to dominate the future of cloud computing.

• The paper introduces serverless computing by abstracting infrastructure management to boost developer productivity.
• The paper evaluates applications like ExCamera and MapReduce, exposing challenges such as high storage latency and function coordination.
• The paper outlines future research in advanced abstractions, optimized storage, improved networking, and robust security frameworks.

Serverless Computing: A Berkeley Perspective

The paper "Cloud Programming Simplified: A Berkeley View on Serverless Computing" provides a comprehensive analysis of serverless computing, tracing its evolution and positioning it as a pivotal advancement in cloud programming. The document is authored by a distinguished team from UC Berkeley and offers insights into the current state, challenges, and potential trajectory of serverless computing.

The authors begin by recounting the historical context and motivations that have shaped serverless computing. In 2009, the Berkeley team outlined six potential advantages of cloud computing, many of which have been realized over the past decade. However, serverless computing emerged in response to unresolved challenges, particularly the burdensome management of virtual machines and the inefficiencies in resource utilization.

Serverless computing, embodied in Function as a Service (FaaS) and complemented by Backend as a Service (BaaS), fundamentally shifts the paradigm. The user focuses on writing code while the infrastructure management is abstracted away, echoing the transition from assembly languages to high-level programming languages. The paper emphasizes that this abstraction allows developers to maximize productivity and resource allocation by decoupling computation from storage and imposing a usage-based billing model.

Applications and Challenges

The document presents several applications to illustrate the limits of current serverless frameworks:

1. ExCamera: A real-time video encoding service that leverages AWS Lambda for parallel processing, highlighting issues with the object store latency and function-to-function communication.
2. MapReduce: Challenges arise due to the inefficiency of shuffle operations, prompting the exploration of alternative storage solutions to handle massive data transfers more effectively.
3. Numpywren: Used for large-scale linear algebra operations, this application shows that high storage latency restricts efficient computations for smaller problem sizes.
4. Cirrus: An approach to machine learning training pipelines, Cirrus underscores the lack of support for high-speed networking to effectively manage parameter servers and data broadcasts.
5. Serverless SQLite: Demonstrates that traditional databases struggle in a serverless context due to structural assumptions about connectivity and storage requirements.

Technical Limitations

The manuscript identifies several core technical challenges in serverless computing:

• Inadequate Fine-Grained Storage: Existing storage options such as S3 exhibit high latency and costs, making them unsuitable for applications requiring frequent, small updates.
• Lack of Coordination: Applications are forced to deploy ad-hoc solutions to manage coordination and data consistency due to the absence of inherent signaling mechanisms.
• Networking Constraints: Communication patterns like broadcast and shuffle are inefficient in a serverless environment due to isolated function executions, leading to increased communication overhead.
• Startup Time: The latency involved in initial function deployment and initialization impacts performance predictability.

Future Directions and Implications

To address these limitations, the paper outlines potential research and development areas:

1. Advanced Abstractions: Proposes developing higher-level abstractions to automatically infer resource requirements and optimize deployment, drawing parallels to programming paradigms that obscure low-level detail.
2. Storage Innovations: Calls for the creation of serverless storage systems with transparent provisioning that match the performance attributes of traditional storage, segregating durable and ephemeral needs.
3. Networking Improvements: Suggests enabling efficient implementation of common communication primitives by enhancing network layer cooperation among functions.
4. Security Enhancements: Discusses the need for robust security frameworks to manage the increased complexity and ephemeral nature of serverless functions, including mechanisms for randomization and isolation.
5. Architectural Exploration: Encourages leveraging heterogeneous hardware, emphasizing the potential role of domain-specific hardware accelerators in optimizing performance for specific applications.

Conclusion

The authors predict that serverless computing will become the dominant paradigm as existing technical barriers are addressed. By streamlining cloud programming and improving efficiency through fine-grained billing and management, serverless computing promises substantial gains in productivity and cost-effectiveness. The paper positions serverless computing as a natural evolution of the cloud, poised to eclipse traditional serverful approaches, particularly as industry standards develop and new architectural and security models emerge.

In essence, this Berkeley perspective presents a measured and scholarly exploration of serverless computing, blending historical context, technical detail, and future vision to chart a course for ongoing research and development within the field.