Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Cross-Edge Orchestration of Serverless Functions with Probabilistic Caching (2310.04185v1)

Published 6 Oct 2023 in cs.NI

Abstract: Serverless edge computing adopts an event-based paradigm that provides back-end services on an as-used basis, resulting in efficient resource utilization. To improve the end-to-end latency and revenue, service providers need to optimize the number and placement of serverless containers while considering the system cost incurred by the provisioning. The particular reason for this circumstance is that frequently creating and destroying containers not only increases the system cost but also degrades the time responsiveness due to the cold-start process. Function caching is a common approach to mitigate the coldstart issue. However, function caching requires extra hardware resources and hence incurs extra system costs. Furthermore, the dynamic and bursty nature of serverless invocations remains an under-explored area. Hence, it is vitally important for service providers to conduct a context-aware request distribution and container caching policy for serverless edge computing. In this paper, we study the request distribution and container caching problem in serverless edge computing. We prove the proposed problem is NP-hard and hence difficult to find a global optimal solution. We jointly consider the distributed and resource constrained nature of edge computing and propose an optimized request distribution algorithm that adapts to the dynamics of serverless invocations with a theoretical performance guarantee. Also, we propose a context-aware probabilistic caching policy that incorporates a number of characteristics of serverless invocations. Via simulation and implementation results, we demonstrate the superiority of the proposed algorithm by outperforming existing caching policies in terms of the overall system cost and cold-start frequency by up to 62.1% and 69.1%, respectively.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Ml-faas: Towards exploiting the serverless paradigm to facilitate machine learning functions as a service. IEEE Transactions on Network and Service Management, pages 1–1, 2023.
  2. An online orchestration mechanism for general-purpose edge computing. IEEE Transactions on Services Computing, 16(2):927–940, 2023.
  3. Enabling age-aware big data analytics in serverless edge clouds. In IEEE INFOCOM 2023 - IEEE Conference on Computer Communications, pages 1–10, 2023.
  4. Disprotrack: Distributed provenance tracking over serverless applications. In IEEE INFOCOM 2023 - IEEE Conference on Computer Communications, pages 1–10, 2023.
  5. Mu: An efficient, fair and responsive serverless framework for resource-constrained edge clouds. In Proceedings of the ACM Symposium on Cloud Computing, SoCC ’21, page 168–181, New York, NY, USA, 2021. Association for Computing Machinery.
  6. When serverless computing meets edge computing: Architecture, challenges, and open issues. IEEE Wireless Communications, 28(5):126–133, 2021.
  7. Retention-aware container caching for serverless edge computing. In IEEE INFOCOM 2022 - IEEE Conference on Computer Communications, pages 1069–1078, 2022.
  8. Spright: Extracting the server from serverless computing! high-performance ebpf-based event-driven, shared-memory processing. In Proceedings of the ACM SIGCOMM 2022 Conference, SIGCOMM ’22, page 780–794, New York, NY, USA, 2022. Association for Computing Machinery.
  9. Serverledge: Decentralized function-as-a-service for the edge-cloud continuum. In 2023 IEEE International Conference on Pervasive Computing and Communications (PerCom), pages 131–140, 2023.
  10. Mxfaas: Resource sharing in serverless environments for parallelism and efficiency. In Proceedings of the 50th Annual International Symposium on Computer Architecture, ISCA ’23, New York, NY, USA, 2023. Association for Computing Machinery.
  11. Following the data, not the function: Rethinking function orchestration in serverless computing. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23), pages 1489–1504, Boston, MA, April 2023. USENIX Association.
  12. Dependent function embedding for distributed serverless edge computing. IEEE Transactions on Parallel and Distributed Systems, 33(10):2346–2357, 2022.
  13. Serverless computing: What it is, and what it is not? Commun. ACM, 66(9):80–92, aug 2023.
  14. Azure ai. https://azure.microsoft.com/en-gb/solutions/ai/. 2023.
  15. Serverless computing: State-of-the-art, challenges and opportunities. IEEE Transactions on Services Computing, 16(2):1522–1539, 2023.
  16. Specfaas: Accelerating serverless applications with speculative function execution. In 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA), pages 814–827, 2023.
  17. Lukewarm serverless functions: Characterization and optimization. In Proceedings of the 49th Annual International Symposium on Computer Architecture, ISCA ’22, page 757–770, New York, NY, USA, 2022. Association for Computing Machinery.
  18. Layer-aware collaborative microservice deployment toward maximal edge throughput. In IEEE INFOCOM 2022 - IEEE Conference on Computer Communications, pages 71–79, 2022.
  19. Time and cost-efficient cloud data transmission based on serverless computing compression. In IEEE INFOCOM 2023 - IEEE Conference on Computer Communications, pages 1–10, 2023.
  20. Online container scheduling for data-intensive applications in serverless edge computing. In IEEE INFOCOM 2023 - IEEE Conference on Computer Communications, pages 1–10, 2023.
  21. A decentralized framework for serverless edge computing in the internet of things. IEEE Transactions on Network and Service Management, 18(2):2166–2180, 2021.
  22. On efficient zygote container planning toward fast function startup in serverless edge cloud. In IEEE INFOCOM 2023 - IEEE Conference on Computer Communications, pages 1–9, 2023.
  23. Faaslight: General application-level cold-start latency optimization for function-as-a-service in serverless computing. ACM Trans. Softw. Eng. Methodol., 32(5), jul 2023.
  24. Layer aware microservice placement and request scheduling at the edge. In IEEE INFOCOM 2021 - IEEE Conference on Computer Communications, pages 1–9, 2021.
  25. Home - knative. https://knative.dev/docs/. (Accessed on 08/28/2023).
  26. An optimal auction mechanism for mobile edge caching. In 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), pages 388–399, 2018.
  27. S-cache: Function caching for serverless edge computing. In Proceedings of the 6th International Workshop on Edge Systems, Analytics and Networking, EdgeSys ’23, page 1–6, New York, NY, USA, 2023. Association for Computing Machinery.
  28. Energy-aware resource scheduling for serverless edge computing. In 2022 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid), pages 190–199, 2022.
  29. Defuse: A dependency-guided function scheduler to mitigate cold starts on faas platforms. In 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS), 2021.
  30. Faascache: Keeping serverless computing alive with greedy-dual caching. In Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, page 386–400, New York, NY, USA, 2021.
  31. Joint service placement and request routing in multi-cell mobile edge computing networks. In IEEE INFOCOM 2019, pages 10–18, 2019.
  32. Modeling and optimization of performance and cost of serverless applications. IEEE Transactions on Parallel and Distributed Systems, 32(3):615–632, 2021.
  33. Optimized container scheduling for data-intensive serverless edge computing. Future Generation Computer Systems, 114:259–271, 2021.
  34. Fine-grained performance and cost modeling and optimization for faas applications. IEEE Transactions on Parallel and Distributed Systems, 34(1):180–194, 2023.
  35. On optimal proactive caching for mobile networks with demand uncertainties. IEEE/ACM Transactions on Networking, 24(5):2715–2727, 2016.
  36. Reliability-aware dynamic service chain scheduling in 5g networks based on reinforcement learning. In IEEE INFOCOM 2021 - IEEE Conference on Computer Communications, pages 1–10, 2021.
  37. Service placement and request scheduling for data-intensive applications in edge clouds. IEEE/ACM Transactions on Networking, 29(2):779–792, 2021.
  38. Service placement with provable guarantees in heterogeneous edge computing systems. In IEEE INFOCOM 2019 - IEEE Conference on Computer Communications, pages 514–522, 2019.
  39. Two time-scale joint service caching and task offloading for uav-assisted mobile edge computing. In IEEE INFOCOM 2022 - IEEE Conference on Computer Communications, pages 1189–1198, 2022.
  40. Icebreaker: Warming serverless functions better with heterogeneity. In Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, ASPLOS ’22, page 753–767, New York, NY, USA, 2022. Association for Computing Machinery.
  41. Kubernetes. https://kubernetes.io/. 2023.
  42. Kourier. https://github.com/knative-extensions/net-kourier. 2023.
  43. A genetic algorithm for the generalised assignment problem. Computers & Operations Research, 24(1):17–23, 1997.
  44. Ludmila Cherkasova. Improving www proxies performance with greedy-dual-size-frequency caching policy. In HP Labs Technical Report 98-69, 1998.
  45. Optimal edge user allocation in edge computing with variable sized vector bin packing. In Claus Pahl, Maja Vukovic, Jianwei Yin, and Qi Yu, editors, Service-Oriented Computing, pages 230–245, Cham, 2018. Springer International Publishing.
  46. Azure. Azurepublicdataset. https://github.com/Azure/AzurePublicDataset/blob/master/AzureFunctionsDataset2019.md, 2019.
  47. My vm is lighter (and safer) than your container. In Proceedings of the 26th Symposium on Operating Systems Principles, SOSP ’17, page 218–233, New York, NY, USA, 2017. Association for Computing Machinery.
  48. Ofc: An opportunistic caching system for faas platforms. EuroSys ’21, page 228–244, New York, NY, USA, 2021. Association for Computing Machinery.
  49. Aws lambda. https://aws.amazon.com/lambda/, 2023.
  50. Proactive retention-aware caching with multi-path routing for wireless edge networks. IEEE Journal on Selected Areas in Communications, 36(6):1286–1299, 2018.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Chen Chen (753 papers)
  2. Manuel Herrera (4 papers)
  3. Ge Zheng (10 papers)
  4. Liqiao Xia (3 papers)
  5. Zhengyang Ling (2 papers)
  6. Jiangtao Wang (42 papers)
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now