Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Lessons Learned from Building Edge Software System Testbeds (2403.16869v2)

Published 25 Mar 2024 in cs.DC

Abstract: Edge computing requires the complex software interaction of geo-distributed, heterogeneous components. The growing research and industry interest in edge computing software systems has necessitated exploring ways of testing and evaluating edge software at scale without relying on physical infrastructure. Beyond simulation, virtual testbeds that emulate edge infrastructure can provide a cost-efficient yet realistic environment to evaluate edge software. In this experience paper, we share lessons learned from building a total of five edge software testbeds. We describe pitfalls in architecture and development as well as experiences from having students use our testbed tooling in distributed systems prototyping classes. While we remain confident that building custom testbed tooling is the right approach for edge computing researchers and practitioners alike, we hope this paper allows others to avoid common mistakes and benefit from our experience.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. 2020. IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems. IEEE Std. 1588-2019 (Revision of IEEE Std. 1588-2008) (June 2020), 1–499. https://doi.org/10.1109/IEEESTD.2020.9120376
  2. Firecracker: Lightweight Virtualization for Serverless Applications. In Proceedings of the 17th USENIX Symposium on Networked Systems Design and Implementation (Santa Clara, CA, USA) (NSDI ’20). USENIX Association, Berkeley, CA, USA, 419–434.
  3. A Flexible Testbed Architecture for VANET. Vehicular Communications 9 (July 2017), 115–126. https://doi.org/10.1016/j.vehcom.2017.04.004
  4. Serverless Edge Computing: Vision and Challenges. In Proceedings of the 2021 Australasian Computer Science Week Multiconference (Dunedin, New Zealand) (ACSW ’21). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3437378.3444367
  5. Oakestra: A Lightweight Hierarchical Orchestration Framework for Edge Computing. In Proceedings of the 2023 USENIX Annual Technical Conference (Boston, MA, USA) (ATC ’23). USENIX Association, Berkeley, CA, USA, 215–231.
  6. Network Emulation in Large-Scale Virtual Edge Testbeds: A Note of Caution and the Way Forward. In Proceedings of the 2nd International Workshop on Testing Distributed Internet of Things Systems (Asilomar, CA, USA) (TDIS 2022). IEEE, New York, NY, USA, 1–7. https://doi.org/10.1109/IC2E55432.2022.00007
  7. EdgePier: P2P-based Container Image Distribution in Edge Computing Environments. In Proceedings of the 2021 IEEE International Performance, Computing, and Communications Conference (Austin, TX, USA) (IPCCC ’21). IEEE, New York, NY, USA, 1–8. https://doi.org/10.1109/IPCCC51483.2021.9679447
  8. Héctor: A Framework for Testing IoT Applications Across Heterogeneous Edge and Cloud Testbeds. In Proceedings of the 12th IEEE/ACM International Conference on Utility and Cloud Computing Companion (Auckland, New Zealand) (UCC ’19 Companion). Association for Computing Machinery, New York, NY, USA, 15–20. https://doi.org/10.1145/3368235.3368832
  9. A Research Perspective on Fog Computing. In Proceedings of the 2nd Workshop on IoT Systems Provisioning & Management for Context-Aware Smart Cities (Malaga, Spain) (ISYCC 2017). Springer, Cham, Switzerland, 198–210. https://doi.org/10.1007/978-3-319-91764-1_16
  10. Cloud Service Benchmarking: Measuring Quality of Cloud Services from a Client Perspective. Springer, Cham, Switzerland.
  11. Debopam Bhattacherjee. 2021. Towards Performant Networking from Low-Earth Orbit. Ph. D. Dissertation. ETH Zurich, Zurich, Switzerland. Advisor(s) Ankit Singla.
  12. Gearing Up for the 21st Century Space Race. In Proceedings of the 17th ACM Workshop on Hot Topics in Networks (Redmond, WA, USA) (HotNets ’18). Association for Computing Machinery, New York, NY, USA, 113–119. https://doi.org/10.1145/3286062.3286079
  13. In-orbit Computing: An Outlandish thought Experiment?. In Proceedings of the 19th ACM Workshop on Hot Topics in Networks (Virtual Event, USA) (HotNets ’20). Association for Computing Machinery, New York, NY, USA, 197–204. https://doi.org/10.1145/3422604.3425937
  14. Toward Loosely Coupled Orchestration for the LEO Satellite Edge. In Proceedings of the 3rd USENIX Workshop on Hot Topics in Edge Computing (HotEdge ’20). USENIX Association, Berkeley, CA, USA.
  15. Fog Computing and its Role in the Internet of Things. In Proceedings of the First Edition of the MCC Workshop on Mobile Cloud Computing (Helsinki, Finland) (MCC ’12). Association for Computing Machinery, New York, NY, USA, 13–16. https://doi.org/10.1145/2342509.2342513
  16. Antonio Brogi and Stefano Forti. 2017. QoS-Aware Deployment of IoT Applications Through the Fog. IEEE internet of Things Journal 4, 5 (May 2017), 1185–1192. https://doi.org/10.1109/JIOT.2017.2701408
  17. Martin A. Brown. 2006. Traffic Control HOWTO. Technical Report 102. The Linux Documentation Project. https://tldp.org/HOWTO/pdf/Traffic-Control-HOWTO.pdf
  18. Benchmarking Cloud Serving Systems with YCSB. In Proceedings of the 1st ACM Symposium on Cloud Computing (Indianapolis, Indiana, USA) (SoCC ’10). Association for Computing Machinery, New York, NY, USA, 143–154. https://doi.org/10.1145/1807128.1807152
  19. Fogbed: A Rapid-Prototyping Emulation Environment for Fog Computing. In Proceedings of the 2018 IEEE International Conference on Communications (Kansas City, MO, USA) (ICC ’18). IEEE, New York, NY, USA, 1–7. https://doi.org/10.1109/ICC.2018.8423003
  20. Eric Dumazet. 2018. [PATCH net-next 0/9] tcp: switch to Early Departure Time model. Google. Retrieved March 18, 2024 from https://lwn.net/ml/netdev/[email protected]/
  21. Run User Code on Fly Machines. fly.io. Retrieved March 18, 2024 from https://fly.io/docs/machines/guides-examples/functions-with-machines/
  22. Kollaps: Decentralized and Dynamic Topology Emulation. In Proceedings of the Fifteenth European Conference on Computer Systems (Heraklion, Greece) (EuroSys ’20). Association for Computing Machinery, New York, NY, USA, 1–16. https://doi.org/10.1145/3342195.3387540
  23. Harshit Gupta and Umakishore Ramachandran. 2018. FogStore: A Geo-Distributed Key-Value Store Guaranteeing Low Latency for Strongly Consistent Access. In Proceedings of the 12th ACM International Conference on Distributed and Event-Based Systems (Hamilton, New Zealand) (DEBS ’18). Association for Computing Machinery, New York, NY, USA, 148–159. https://doi.org/10.1145/3210284.3210297
  24. FBase: A Replication Service for Data-Intensive Fog Applications. Technical Report 20191. TU Berlin & ECDF, Mobile Cloud Computing Research Group, Berlin, Germany.
  25. Towards A Replication Service for Data-Intensive Fog Applications. In Proceedings of the 35th ACM Symposium on Applied Computing, Posters Track (Brno, Czech Republic) (SAC ’20). Association for Computing Machinery, New York, NY, USA, 267–270. https://doi.org/10.1145/3341105.3374060
  26. MockFog 2.0: Automated Execution of Fog Application Experiments in the Cloud. IEEE Transactions on Cloud Computing 11, 1 (April 2021), 58–70. https://doi.org/10.1109/TCC.2021.3074988
  27. MockFog: Emulating Fog Computing Infrastructure in the Cloud. In Proceedings of the First IEEE International Conference on Fog Computing 2019 (Prague, Czech Republic) (ICFC 2019). IEEE, New York, NY, USA, 144–152. https://doi.org/10.1109/ICFC.2019.00026
  28. Managing Latency and Excess Data Dissemination in Fog-Based Publish/Subscribe Systems. In Proceedings of the Second IEEE International Conference on Fog Computing (Sydney, NSW, Australia) (ICFC 2020). IEEE, New York, NY, USA, 9–16. https://doi.org/10.1109/ICFC49376.2020.00010
  29. FogExplorer. In Proceedings of the 19th ACM/IFIP International Middleware Conference, Demos, and Posters (Rennes, France) (Middleware ’18). Association for Computing Machinery, New York, NY, USA, 1–2. https://doi.org/10.1145/3284014.3284015
  30. Supporting the Evaluation of Fog-based IoT Applications During the Design Phase. In Proceedings of the 5th Workshop on Middleware and Applications for the Internet of Things (Rennes, France) (M4IoT ’18). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3286719.3286720
  31. WoTbench: A Benchmarking Framework for the Web of Things. In Proceedings of the 9th International Conference on the Internet of Things (Bilbao, Spain) (IoT ’19). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3365871.3365897
  32. Contention Aware Web of Things Emulation Testbed. In Proceedings of the ACM/SPEC International Conference on Performance Engineering (Edmonton AB, Canada) (ICPE ’20). Association for Computing Machinery, New York, NY, USA, 246–256. https://doi.org/10.1145/3358960.3379140
  33. Stephen Hemminger. 2017. XNetEm Network Emulation using XDPp. In Proceedings of the 2017 Technical Conference on Linux Networking (Seoul, South Korea) (NetDev 2.2). The Netdev Society, Nepean, ON, Canada.
  34. Stephen Hemminger et al. 2005. Network Emulation with NetEm. Technical Report. Open Source Development Lab, San Francisco, CA, USA.
  35. An IoT Network Emulator for Analyzing the Influence of Varying Network Quality. In Proceedings of the International Conference on Simulation Tools and Techniques (Guiyang, China) (SIMUTools ’20). Springer, Cham, Switzerland, 580–599. https://doi.org/10.1007/978-3-030-72795-6_47
  36. Exploring the “Internet from space” with Hypatia. In Proceedings of the ACM Internet Measurement Conference (Virtual Event, USA) (IMC ’20). Association for Computing Machinery, New York, NY, USA, 214–229. https://doi.org/10.1145/3419394.3423635
  37. Ariane Keller. 2006. tc Packet Filtering and netem. Technical Report. ETH Zurich, Zurich, Switzerland.
  38. Benjamin Kempton and Anton Riedl. 2021. Network Simulator for Large Low Earth Orbit Satellite Networks. In Proceedings of the 2021 IEEE International Conference on Communications (Montreal, QC, Canada) (ICC ’21). IEEE, New York, NY, USA, 1–6. https://doi.org/10.1109/ICC42927.2021.9500439
  39. Linearize, Predict and Place: Minimizing the Makespan for Edge-based Stream Processing of Directed Acyclic Graphs. In Proceedings of the 4th ACM/IEEE Symposium on Edge Computing (Arlington, Virginia) (SEC ’19). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3318216.3363315
  40. A Network in a Laptop: Rapid Prototyping for Software-defined Networks. In Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks (Monterey, California) (Hotnets-IX). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/1868447.1868466
  41. EmuFog: Extensible and Scalable Emulation of Large-scale Fog Computing Infrastructures. In Proceedings of the 2017 IEEE Fog World Congress (Santa Clara, CA, USA) (FWC ’17). IEEE, New York, NY, USA, 1–6. https://doi.org/10.1109/FWC.2017.8368525
  42. Fogstore: Toward a Distributed Data Store for Fog Computing. In Proceedings of the 2017 IEEE Fog World Congress (Santa Clara, CA, USA) (FWC ’17). IEEE, New York, NY, USA, 1–6. https://doi.org/10.1109/FWC.2017.8368524
  43. Chris Munns. 2023. It’s About Time: Microsecond-Accurate Clocks on Amazon EC2 Instances. AWS Compute Blog. Retrieved March 18, 2024 from https://aws.amazon.com/blogs/compute/its-about-time-microsecond-accurate-clocks-on-amazon-ec2-instances/
  44. Tobias Pfandzelter. 2023. Serverless Abstractions for Edge Computing in Large Low-Earth Orbit Satellite Networks. In Proceedings of the 24th Middleware Demos, Posters and Doctoral Symposium (Bologna, Italy) (Middleware Demos, Posters and Doctoral Symposium ’23). Association for Computing Machinery, New York, NY, USA, 3–6. https://doi.org/10.1145/3626564.3629088
  45. Tobias Pfandzelter and David Bermbach. 2020. tinyFaaS: A Lightweight FaaS Platform for Edge Environments. In Proceedings of the Second IEEE International Conference on Fog Computing (Sydney, NSW, Australia) (ICFC 2020). IEEE, New York, NY, USA, 17–24. https://doi.org/10.1109/ICFC49376.2020.00011
  46. Tobias Pfandzelter and David Bermbach. 2022a. Celestial: Virtual Software System Testbeds for the LEO Edge. In Proceedings of the 23rd ACM/IFIP International Middleware Conference (Quebec, QC, Canada) (Middleware ’22). Association for Computing Machinery, New York, NY, USA, 69–81. https://doi.org/10.1145/3528535.3531517
  47. Tobias Pfandzelter and David Bermbach. 2022b. Testing LEO Edge Software Systems with Celestial. Technical Report 20221. TU Berlin & ECDF, Mobile Cloud Computing Research Group, Berlin, Germany.
  48. Tobias Pfandzelter and David Bermbach. 2023a. Evaluating LEO Edge Software in the Cloud with Celestial. In Proceedings of the 11th IEEE International Conference on Cloud Engineering (Boston, MA, USA) (IC2E ’23). IEEE, New York, NY, USA, 224–225. https://doi.org/10.1109/IC2E59103.2023.00034
  49. Tobias Pfandzelter and David Bermbach. 2023b. Towards a Benchmark for Fog Data Processing. In Proceedings of the 11th IEEE International Conference on Cloud Engineering (Boston, MA, USA) (IC2E ’23). IEEE, New York, NY, USA, 92–98. https://doi.org/10.1109/IC2E59103.2023.00018
  50. From Zero to Fog: Efficient Engineering of Fog-Based Internet of Things Applications. Software: Practice and Experience 51, 8 (June 2021), 1798–1821. https://doi.org/10.1002/spe.3003
  51. Towards a Computing Platform for the LEO Edge. In Proceedings of the 4th International Workshop on Edge Systems, Analytics and Networking (Online, United Kingdom) (EdgeSys ’21). Association for Computing Machinery, New York, NY, USA, 43–48. https://doi.org/10.1145/3434770.3459736
  52. Managing Data Replication and Distribution in the Fog with FReD. Software: Practice and Experience 53, 10 (Oct. 2023), 1958–1981. https://doi.org/10.1002/spe.3237
  53. Serverless Edge Computing – Where We Are and What Lies Ahead. IEEE Internet Computing 27, 3 (May 2023), 50–64. https://doi.org/10.1109/MIC.2023.3260939
  54. Constellations in the Cloud: Virtualizing Remote Sensing Systems. In Proceedings of the 2019 IEEE International Geoscience and Remote Sensing Symposium (Yokohama, Japan) (IGARSS 2019). IEEE, New York, NY, USA, 5351–5354. https://doi.org/10.1109/IGARSS.2019.8898918
  55. Weisong Shi and Schahram Dustdar. 2016. The Promise of Edge Computing. Computer 49, 5 (May 2016), 78–81. https://doi.org/10.1109/MC.2016.145
  56. Fogify: A Fog Computing Emulation Framework. In Proceedings of the 2020 IEEE/ACM Symposium on Edge Computing (San Jose, CA, USA) (SEC ’20). IEEE, New York, NY, USA, 42–54. https://doi.org/10.1109/SEC50012.2020.00011
  57. WeaveWorks. 2023. Ignite: Ignite a Firecracker microVM. Retrieved March 18, 2024 from https://github.com/weaveworks/ignite
  58. Maxinet: Distributed Emulation of Software-defined Networks. In Proceedings of the 2014 IFIP Networking Conference (Trondheim, Norway). IEEE, New York, NY, USA, 1–9. https://doi.org/10.1109/IFIPNetworking.2014.6857078
  59. Qiaozhi Xu and Junxing Zhang. 2019. piFogBed: a Fog Computing Testbed based on Raspberry Pi. In Proceedings of the 2019 IEEE 38th International Performance Computing and Communications Conference (London, UK) (IPCCC). IEEE, New York, NY, USA, 1–8. https://doi.org/10.1109/IPCCC47392.2019.8958741
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Tobias Pfandzelter (45 papers)
  2. David Bermbach (61 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets