Papers
Topics
Authors
Recent
Search
2000 character limit reached

MTS: Bringing Multi-Tenancy to Virtual Networking

Published 4 Mar 2024 in cs.CR and cs.NI | (2403.01862v1)

Abstract: Multi-tenant cloud computing provides great benefits in terms of resource sharing, elastic pricing, and scalability, however, it also changes the security landscape and introduces the need for strong isolation between the tenants, also inside the network. This paper is motivated by the observation that while multi-tenancy is widely used in cloud computing, the virtual switch designs currently used for network virtualization lack sufficient support for tenant isolation. Hence, we present, implement, and evaluate a virtual switch architecture, MTS, which brings secure design best-practice to the context of multi-tenant virtual networking: compartmentalization of virtual switches, least-privilege execution, complete mediation of all network communication, and reducing the trusted computing base shared between tenants. We build MTS from commodity components, providing an incrementally deployable and inexpensive upgrade path to cloud operators. Our extensive experiments, extending to both micro-benchmarks and cloud applications, show that, depending on the way it is deployed, MTS may produce 1.5-2x the throughput compared to state-of-the-art, with similar or better latency and modest resource overhead (1 extra CPU). MTS is available as open source software.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (75)
  1. Allclair, T. Secure Container Isolation: Problem Statement & Solution Space. https://docs.google.com/document/d/1QQ5u1RBDLXWvC8K3pscTtTRThsOeBSts_imYEoRyw8A, 2018. Accessed: 05-01-2019.
  2. Apache. ab - Apache HTTP server benchmarking tool. https://httpd.apache.org/docs/2.2/en/programs/ab.html. Accessed: 07-01-2019.
  3. SCONE: Secure linux containers with intel SGX. In Proc. Usenix Operating Systems Design Principles (OSDI) (2016).
  4. AWS. Enhanced Networking on Linux. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networking.html, 2018. Accessed: 24-01-2018.
  5. Azure, M. Create a Linux virtual machine with Accelerated Networking. https://docs.microsoft.com/en-us/azure/virtual-network/create-vm-accelerated-networking-cli, 2018. Accessed: 24-01-2018.
  6. On detecting co-resident cloud instances using network flow watermarking techniques. Springer International Journal of Information Security (2014).
  7. Beaupré, A. Updates in container isolation. https://lwn.net/Articles/754433, 2018. Accessed: 09-01-2019.
  8. BESS Comitters. BESS (Berkeley Extensible Software Switch). https://github.com/NetSys/bess, 2017. Accessed: 09-05-2017.
  9. Bishop, M. A. Introduction to computer security, vol. 50. Addison-Wesley Boston, 2005.
  10. Tolerating malicious device drivers in linux. In Proc. Usenix Annual Technical Conference (ATC) (2010).
  11. Broadcom Samples Thor, World’s First 200G Ethernet Controller with 50G PAM-4 and PCIe 4.0. https://www.broadcom.com/company/news/product-releases/2367107. Accessed: 06-05-2019.
  12. Breaking up is hard to do: Security and Functionality in a Commodity Hypervisor. In Proc. ACM Symposium on Operating System Principles (SOSP) (2011).
  13. Columbus, L. Roundup Of Cloud Computing Forecasts And Market Estimates. https://www.forbes.com/sites/louiscolumbus/2018/09/23/roundup-of-cloud-computing-forecasts-and-market-estimates-2018/, 2017. Accessed: 09-01-2019.
  14. Costin, A. All your cluster-grids are belong to us: Monitoring the (in)security of infrastructure monitoring systems. In Proc. IEEE Communications and Network Security (CNS) (Sept 2015).
  15. Policy injection: A cloud dataplane dos attack. In Proc. ACM SIGCOMM Posters and Demos (2018).
  16. Andromeda: Performance, isolation, and velocity at scale in cloud network virtualization. In Proc. Usenix Networked Systems Design and Implementation (NSDI) (2018).
  17. Dpdk. Writing Efficient Code. https://doc.dpdk.org/guides/prog_guide/writing_efficient_code.html. Accessed: 06-01-2019.
  18. [PATCH] netdev-dpdk: Add new ’dpdkvhostuserclient’ port type. https://mail.openvswitch.org/pipermail/ovs-dev/2016-August/321742.html. Accessed: 29-04-2019.
  19. Performance characteristics of virtual switching. In Proc. IEEE Conference on Cloud Networking (2014).
  20. Endace DAG 10X4-P Datasheet. https://www.endace.com/dag-10x4-p-datasheet.pdf. Accessed: 07-01-2019.
  21. Firestone, D. Vfp: A virtual switch platform for host sdn in the public cloud. In Proc. Usenix Networked Systems Design and Implementation (NSDI) (2017), pp. 315–328.
  22. Azure accelerated networking: Smartnics in the public cloud. In Proc. Usenix Networked Systems Design and Implementation (NSDI) (2018).
  23. Frazelle, J. Hard multi-tenancy in kubernetes. https://blog.jessfraz.com/post/hard-multi-tenancy-in-kubernetes, 2018. Accessed: 09-01-2019.
  24. Google Compute Engine Pricing. https://cloud.google.com/compute/pricing#network, 2018. Accessed: 03-01-2019.
  25. Geneve: Generic Network Virtualization Encapsulation. https://tools.ietf.org/html/draft-ietf-nvo3-geneve-08. Accessed: 03-01-2019.
  26. The gVisor project. https://github.com/google/gvisor, 2018. Accessed: 09-01-2019.
  27. Gospodarek, A. The Rise of SmartNICs – offloading dataplane traffic to…software. https://youtu.be/AGSy51VlKaM, 2017. Open vSwitch Conference.
  28. mswitch: a highly-scalable, modular software switch. In Proc. ACM Symposium on SDN Research (SOSR) (2015).
  29. Netvm: high performance and flexible networking using virtualization on commodity platforms. In Proc. Usenix Networked Systems Design and Implementation (NSDI) (2014).
  30. Intel. Enabling NFV to deliver on its promise. https://www-ssl.intel.com/content/www/us/en/communications/nfv-packet-processing-brief.html, 2015.
  31. iPerf - The ultimate speed test tool for TCP, UDP and SCTP. https://iperf.fr/. Accessed: 07-01-2019.
  32. Jacobson, V. Congestion avoidance and control. In ACM Computer Communication Review (CCR) (1988).
  33. Network virtualization and software defined networking for cloud computing: a survey. IEEE Communication Magazine 51, 11 (2013).
  34. Virtual switching without a hypervisor for a more secure cloud. In Proc. USENIX Workshop on Hot Topics in Management of Internet, Cloud, and Enterprise Networks and Services (HotICE) (2012).
  35. Jing, C. Zero-Copy Optimization for Alibaba Cloud Smart NIC Solution. http://www.alibabacloud.com/blog/zero-copy-optimization-for-alibaba-cloud-smart-nic-solution_593986, 2018. Accessed: 03-01-2019.
  36. The Kata Containers project. https://katacontainers.io, 2018. Accessed: 09-01-2019.
  37. Virtual ethernet bridging. http://www.ieee802.org/1/files/public/docs2009/new-hudson-vepa_seminar-20090514d.pdf. Accessed: 06-01-2019.
  38. Network virtualization in multi-tenant datacenters. In Proc. Usenix Networked Systems Design and Implementation (NSDI) (2014).
  39. Kutch, P. PCI-SIG SR-IOV primer: An introduction to SR-IOV technology. Intel application note (2011), 321211–002.
  40. The Price for Programmability in the Software Data Plane: The Vendor Perspective. IEEE J. Selected Areas in Communications (2018).
  41. HowTo Launch VM over OVS-DPDK-17.11 Using Mellanox ConnectX-4 and ConnectX-5. https://community.mellanox.com/s/article/howto-launch-vm-over-ovs-dpdk-17-11-using-mellanox-connectx-4-and-connectx-5. Accessed: 09-01-2019.
  42. Mellanox. Mellanox BlueField SmartNIC. https://bit.ly/2JaMitA, 2017. Accessed: 05-06-2018.
  43. Memcached. https://libmemcached.org/libMemcached.html. Accessed: 07-01-2019.
  44. Microsoft. Hyper-V Virtual Switch Overview. https://technet.microsoft.com/en-us/library/hh831823(v=ws.11).aspx, 2013. Accessed: 27-01-2017.
  45. Microsoft. SR-IOV VF Failover and Live Migration Support. https://docs.microsoft.com/en-us/windows-hardware/drivers/network/sr-iov-vf-failover-and-live-migration-support, 2017. Accessed: 03-01-2019.
  46. Dataplane specialization for high-performance openflow software switching. In Proc. ACM SIGCOMM (2016).
  47. Muller, A. OVS ARP Responder – Theory and Practice. https://assafmuller.com/2014/05/21/ovs-arp-responder-theory-and-practice/. Accessed: 06-01-2019.
  48. Understanding pcie performance for end host networking. In Proc. ACM SIGCOMM (2018).
  49. Sandboxing data plane programs for fun and profit. In Proc. ACM SIGCOMM Posters and Demos (2017).
  50. Sdn access control for the masses. Elsevier Computers & Security (2019).
  51. Netbricks: Taking the v out of nfv. In Proc. Usenix Operating Systems Design Principles (OSDI) (2016).
  52. Panicker, M. Enabling Hardware Offload of OVS Control & Data plane using LiquidIO. https://youtu.be/qjXBRCFhbqU, 2017. Open vSwitch Conference.
  53. Bringing platform harmony to vmware nsx. ACM SIGOPS Operating System Review (2018).
  54. OVN, Bringing Native Virtual Networking to OVS. https://networkheresy.com/2015/01/13/ovn-bringing-native-virtual-networking-to-ovs/, 2015. Accessed: 27-01-2017.
  55. Pfaff, B. Open vSwitch: Past, Present, and Future. http://openvswitch.org/slides/ppf.pdf, 2013. Accessed: 27-01-2017.
  56. The design and implementation of Open vSwitch. In Proc. Usenix Networked Systems Design and Implementation (NSDI) (2015).
  57. Hyper-switch: A scalable software virtual switching architecture. In Proc. Usenix Annual Technical Conference (ATC) (2013).
  58. Rizzo, L. Netmap: a novel framework for fast packet I/O. In Proc. Usenix Annual Technical Conference (ATC) (2012).
  59. VALE, a switched ethernet for virtual machines. In Proc. ACM CoNEXT (2012).
  60. Robin G. Open vSwitch with DPDK Overview. https://software.intel.com/en-us/articles/open-vswitch-with-dpdk-overview, 2016. Accessed: 27-01-2017.
  61. The protection of information in computer systems. Proceedings of the IEEE 63, 9 (1975), 1278–1308.
  62. SecuritytWeek. CSA’s cloud adoption, practices and priorities survey report. http://www.securityweek.com/data-security-concerns-still-challenge, 2015. Accessed: 09-01-2019.
  63. Pisces: A programmable, protocol-independent software switch. In Proc. ACM SIGCOMM (2016).
  64. Securing self-virtualizing ethernet devices. In Proc. Usenix Security Symp. (2015).
  65. Stecklina, J. Shrinking the hypervisor one subsystem at a time: A userspace packet switch for virtual machines. In Proc. ACM SIGPLAN/SIGOPS Conference on Virtual Execution Environments (VEE) (2014).
  66. Stone, R. PCI SR-IOV on FreeBSD. https://people.freebsd.org/~rstone/BSDCan_SRIOV.pdf. Accessed: 06-01-2019.
  67. The Fast Data Project. What is the Fast Data Project (FD.io)? https://fd.io/about, 2017. Accessed: 05-06-2018.
  68. Virtual network isolation: Are we there yet? In Proc. ACM Workshop on Security in Softwarized Networks: Prospects and Challenges (2018).
  69. Taking control of sdn-based cloud systems via the data plane. In Proc. ACM Symposium on SDN Research (SOSR) (2018).
  70. Accelerating open vswitch with integrated gpu. In Proc. ACM Workshop on Kernel-Bypass Networks (2017).
  71. Vanover, R. Virtual switching to become enhanced with Cisco and VMware announcement. http://www.techrepublic.com/blog/data-center/virtual-switching-to-become-enhanced-with-cisco-and-vmware-announcement, 2008. Accessed: 27-01-2017.
  72. VMware. VMware ESX 4.0 Update 1 Release Notes. https://bit.ly/2sFTuTy, 2009. Accessed: 05-06-2018.
  73. Virtual extensible local area network (VXLAN): A framework for overlaying virtualized layer 2 networks over layer 3 network. https://tools.ietf.org/html/rfc7348. Accessed: 01-06-2016.
  74. Zhao, Y. PCI: Linux kernel SR-IOV support. https://lwn.net/Articles/319651/, 2009. Accessed: 06-01-2019.
  75. All your vms are disconnected: Attacking hardware virtualized network. In Proc. ACM Conference on Data and Application Security and Privacy (CODASPY) (2017).
Citations (7)

Summary

  • The paper introduces MTS, a virtual switch architecture that compartmentalizes tenant operations into dedicated VMs to enhance security and performance.
  • The methodology leverages SR-IOV for isolating virtual functions, enabling flexible resource allocation and up to 2x throughput improvements.
  • Experimental results validate MTS's cost-effectiveness and latency enhancements, offering a scalable solution for multi-tenant cloud networks.

MTS: Bringing Multi-Tenancy to Virtual Networking

The paper "MTS: Bringing Multi-Tenancy to Virtual Networking" (2403.01862) addresses the challenges associated with multi-tenant virtual switch designs in cloud environments. It introduces a novel architecture, MTS, aimed at enhancing security while maintaining high performance and cost-effectiveness.

Introduction to Multi-Tenancy in Cloud Networking

The growing adoption of multi-tenant architectures in cloud computing necessitates robust isolation of virtual networks to ensure security and performance. The current virtual switch designs often fail to provide adequate isolation between tenants, exposing cloud infrastructure to potential security breaches. This paper identifies these shortcomings and proposes MTS, a secure multi-tenant virtual switch architecture.

MTS Architecture Overview

MTS introduces a compartmentalized virtual switch architecture to improve tenant isolation (Figure 1). The core innovation lies in moving vswitch functionalities to dedicated virtual machines (VMs), thereby compartmentalizing tenant operations. Each compartment allocates separate virtual functions (VFs) using Single Root Input/Output Virtualization (SR-IOV) to maintain high-speed intercommunication. Figure 1

Figure 1: High-level overview of MTS in security Level-2. The Red and Blue vswitch compartments (VMs) are allocated dedicated virtual functions (VFs) to communicate with external networks using the In/Out VF, their respective tenants using the Gw VF and T VF. Communication between the vswitches, tenants and the Host physical function (PF) are mediated via the SR-IOV NIC switch.

Implementation Details

Compartmentalization

MTS relies on the concept of compartmentalizing vswitch functions into dedicated VMs, allowing for isolated and secure packet processing. This separation aligns with secure system design principles like least privilege and complete mediation.

Communication and Mediation

Communication within the MTS architecture utilizes SR-IOV, a hardware technique that segregates network device resources into isolated VFs to facilitate efficient and secure tenant isolation. This ensures that tenant data traffic does not interfere with or compromise other tenant data (Figure 2). Figure 2

Figure 2: A step-by-step illustration of how packets enter and leave the Red tenant from Figure 1.

Resource Allocation

MTS supports various resource allocation strategies, ranging from shared to isolated modes to optimize security and performance trade-offs (Figure 3, Figure 4). These strategies allow flexible deployment across diverse cloud environments, ensuring scalable and cost-effective solutions. Figure 3

Figure 3

Figure 3

Figure 3: The security, throughput, latency and resource tradeoff comparison of MTS. The rows indicate the resource mode. The columns are ordered as throughput, latency and resources. The security levels used are shown in the legend.

Figure 4

Figure 4

Figure 4

Figure 4: Iperf throughput, Apache and Memcached throughput and latency comparison of MTS. The rows indicate the resource mode where the bottom row is for security Level-3 in the isolated resource mode combined with the other security levels.

Evaluation and Results

The authors conducted extensive experiments to evaluate MTS's performance compared to existing solutions. MTS demonstrated significant improvements in throughput and latency with modest resource overhead. The design achieved up to 2x throughput improvements while maintaining or enhancing latency performance.

Security Considerations

MTS addresses multi-tenancy-specific security threats by minimizing the shared components amongst tenants and implementing strict access controls. However, the reliance on SR-IOV introduces potential vulnerabilities within the VF interfaces, necessitating careful management and periodic security assessments.

Conclusions and Future Work

The paper establishes MTS as a promising multi-tenant virtual networking solution that provides enhanced isolation without compromising performance or scalability. Future research could expand on dynamically adapting resource allocations in response to real-time workload characteristics or integrating additional security layers to mitigate novel attack vectors.

In conclusion, MTS represents an incremental yet significant advancement in virtual switch design for cloud multi-tenancy, offering both practical deployment pathways and potential for further innovation.

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Collections

Sign up for free to add this paper to one or more collections.