MTS: Bringing Multi-Tenancy to Virtual Networking (2403.01862v1)
Abstract: Multi-tenant cloud computing provides great benefits in terms of resource sharing, elastic pricing, and scalability, however, it also changes the security landscape and introduces the need for strong isolation between the tenants, also inside the network. This paper is motivated by the observation that while multi-tenancy is widely used in cloud computing, the virtual switch designs currently used for network virtualization lack sufficient support for tenant isolation. Hence, we present, implement, and evaluate a virtual switch architecture, MTS, which brings secure design best-practice to the context of multi-tenant virtual networking: compartmentalization of virtual switches, least-privilege execution, complete mediation of all network communication, and reducing the trusted computing base shared between tenants. We build MTS from commodity components, providing an incrementally deployable and inexpensive upgrade path to cloud operators. Our extensive experiments, extending to both micro-benchmarks and cloud applications, show that, depending on the way it is deployed, MTS may produce 1.5-2x the throughput compared to state-of-the-art, with similar or better latency and modest resource overhead (1 extra CPU). MTS is available as open source software.
- Allclair, T. Secure Container Isolation: Problem Statement & Solution Space. https://docs.google.com/document/d/1QQ5u1RBDLXWvC8K3pscTtTRThsOeBSts_imYEoRyw8A, 2018. Accessed: 05-01-2019.
- Apache. ab - Apache HTTP server benchmarking tool. https://httpd.apache.org/docs/2.2/en/programs/ab.html. Accessed: 07-01-2019.
- SCONE: Secure linux containers with intel SGX. In Proc. Usenix Operating Systems Design Principles (OSDI) (2016).
- AWS. Enhanced Networking on Linux. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networking.html, 2018. Accessed: 24-01-2018.
- Azure, M. Create a Linux virtual machine with Accelerated Networking. https://docs.microsoft.com/en-us/azure/virtual-network/create-vm-accelerated-networking-cli, 2018. Accessed: 24-01-2018.
- On detecting co-resident cloud instances using network flow watermarking techniques. Springer International Journal of Information Security (2014).
- Beaupré, A. Updates in container isolation. https://lwn.net/Articles/754433, 2018. Accessed: 09-01-2019.
- BESS Comitters. BESS (Berkeley Extensible Software Switch). https://github.com/NetSys/bess, 2017. Accessed: 09-05-2017.
- Bishop, M. A. Introduction to computer security, vol. 50. Addison-Wesley Boston, 2005.
- Tolerating malicious device drivers in linux. In Proc. Usenix Annual Technical Conference (ATC) (2010).
- Broadcom Samples Thor, World’s First 200G Ethernet Controller with 50G PAM-4 and PCIe 4.0. https://www.broadcom.com/company/news/product-releases/2367107. Accessed: 06-05-2019.
- Breaking up is hard to do: Security and Functionality in a Commodity Hypervisor. In Proc. ACM Symposium on Operating System Principles (SOSP) (2011).
- Columbus, L. Roundup Of Cloud Computing Forecasts And Market Estimates. https://www.forbes.com/sites/louiscolumbus/2018/09/23/roundup-of-cloud-computing-forecasts-and-market-estimates-2018/, 2017. Accessed: 09-01-2019.
- Costin, A. All your cluster-grids are belong to us: Monitoring the (in)security of infrastructure monitoring systems. In Proc. IEEE Communications and Network Security (CNS) (Sept 2015).
- Policy injection: A cloud dataplane dos attack. In Proc. ACM SIGCOMM Posters and Demos (2018).
- Andromeda: Performance, isolation, and velocity at scale in cloud network virtualization. In Proc. Usenix Networked Systems Design and Implementation (NSDI) (2018).
- Dpdk. Writing Efficient Code. https://doc.dpdk.org/guides/prog_guide/writing_efficient_code.html. Accessed: 06-01-2019.
- [PATCH] netdev-dpdk: Add new ’dpdkvhostuserclient’ port type. https://mail.openvswitch.org/pipermail/ovs-dev/2016-August/321742.html. Accessed: 29-04-2019.
- Performance characteristics of virtual switching. In Proc. IEEE Conference on Cloud Networking (2014).
- Endace DAG 10X4-P Datasheet. https://www.endace.com/dag-10x4-p-datasheet.pdf. Accessed: 07-01-2019.
- Firestone, D. Vfp: A virtual switch platform for host sdn in the public cloud. In Proc. Usenix Networked Systems Design and Implementation (NSDI) (2017), pp. 315–328.
- Azure accelerated networking: Smartnics in the public cloud. In Proc. Usenix Networked Systems Design and Implementation (NSDI) (2018).
- Frazelle, J. Hard multi-tenancy in kubernetes. https://blog.jessfraz.com/post/hard-multi-tenancy-in-kubernetes, 2018. Accessed: 09-01-2019.
- Google Compute Engine Pricing. https://cloud.google.com/compute/pricing#network, 2018. Accessed: 03-01-2019.
- Geneve: Generic Network Virtualization Encapsulation. https://tools.ietf.org/html/draft-ietf-nvo3-geneve-08. Accessed: 03-01-2019.
- The gVisor project. https://github.com/google/gvisor, 2018. Accessed: 09-01-2019.
- Gospodarek, A. The Rise of SmartNICs – offloading dataplane traffic to…software. https://youtu.be/AGSy51VlKaM, 2017. Open vSwitch Conference.
- mswitch: a highly-scalable, modular software switch. In Proc. ACM Symposium on SDN Research (SOSR) (2015).
- Netvm: high performance and flexible networking using virtualization on commodity platforms. In Proc. Usenix Networked Systems Design and Implementation (NSDI) (2014).
- Intel. Enabling NFV to deliver on its promise. https://www-ssl.intel.com/content/www/us/en/communications/nfv-packet-processing-brief.html, 2015.
- iPerf - The ultimate speed test tool for TCP, UDP and SCTP. https://iperf.fr/. Accessed: 07-01-2019.
- Jacobson, V. Congestion avoidance and control. In ACM Computer Communication Review (CCR) (1988).
- Network virtualization and software defined networking for cloud computing: a survey. IEEE Communication Magazine 51, 11 (2013).
- Virtual switching without a hypervisor for a more secure cloud. In Proc. USENIX Workshop on Hot Topics in Management of Internet, Cloud, and Enterprise Networks and Services (HotICE) (2012).
- Jing, C. Zero-Copy Optimization for Alibaba Cloud Smart NIC Solution. http://www.alibabacloud.com/blog/zero-copy-optimization-for-alibaba-cloud-smart-nic-solution_593986, 2018. Accessed: 03-01-2019.
- The Kata Containers project. https://katacontainers.io, 2018. Accessed: 09-01-2019.
- Virtual ethernet bridging. http://www.ieee802.org/1/files/public/docs2009/new-hudson-vepa_seminar-20090514d.pdf. Accessed: 06-01-2019.
- Network virtualization in multi-tenant datacenters. In Proc. Usenix Networked Systems Design and Implementation (NSDI) (2014).
- Kutch, P. PCI-SIG SR-IOV primer: An introduction to SR-IOV technology. Intel application note (2011), 321211–002.
- The Price for Programmability in the Software Data Plane: The Vendor Perspective. IEEE J. Selected Areas in Communications (2018).
- HowTo Launch VM over OVS-DPDK-17.11 Using Mellanox ConnectX-4 and ConnectX-5. https://community.mellanox.com/s/article/howto-launch-vm-over-ovs-dpdk-17-11-using-mellanox-connectx-4-and-connectx-5. Accessed: 09-01-2019.
- Mellanox. Mellanox BlueField SmartNIC. https://bit.ly/2JaMitA, 2017. Accessed: 05-06-2018.
- Memcached. https://libmemcached.org/libMemcached.html. Accessed: 07-01-2019.
- Microsoft. Hyper-V Virtual Switch Overview. https://technet.microsoft.com/en-us/library/hh831823(v=ws.11).aspx, 2013. Accessed: 27-01-2017.
- Microsoft. SR-IOV VF Failover and Live Migration Support. https://docs.microsoft.com/en-us/windows-hardware/drivers/network/sr-iov-vf-failover-and-live-migration-support, 2017. Accessed: 03-01-2019.
- Dataplane specialization for high-performance openflow software switching. In Proc. ACM SIGCOMM (2016).
- Muller, A. OVS ARP Responder – Theory and Practice. https://assafmuller.com/2014/05/21/ovs-arp-responder-theory-and-practice/. Accessed: 06-01-2019.
- Understanding pcie performance for end host networking. In Proc. ACM SIGCOMM (2018).
- Sandboxing data plane programs for fun and profit. In Proc. ACM SIGCOMM Posters and Demos (2017).
- Sdn access control for the masses. Elsevier Computers & Security (2019).
- Netbricks: Taking the v out of nfv. In Proc. Usenix Operating Systems Design Principles (OSDI) (2016).
- Panicker, M. Enabling Hardware Offload of OVS Control & Data plane using LiquidIO. https://youtu.be/qjXBRCFhbqU, 2017. Open vSwitch Conference.
- Bringing platform harmony to vmware nsx. ACM SIGOPS Operating System Review (2018).
- OVN, Bringing Native Virtual Networking to OVS. https://networkheresy.com/2015/01/13/ovn-bringing-native-virtual-networking-to-ovs/, 2015. Accessed: 27-01-2017.
- Pfaff, B. Open vSwitch: Past, Present, and Future. http://openvswitch.org/slides/ppf.pdf, 2013. Accessed: 27-01-2017.
- The design and implementation of Open vSwitch. In Proc. Usenix Networked Systems Design and Implementation (NSDI) (2015).
- Hyper-switch: A scalable software virtual switching architecture. In Proc. Usenix Annual Technical Conference (ATC) (2013).
- Rizzo, L. Netmap: a novel framework for fast packet I/O. In Proc. Usenix Annual Technical Conference (ATC) (2012).
- VALE, a switched ethernet for virtual machines. In Proc. ACM CoNEXT (2012).
- Robin G. Open vSwitch with DPDK Overview. https://software.intel.com/en-us/articles/open-vswitch-with-dpdk-overview, 2016. Accessed: 27-01-2017.
- The protection of information in computer systems. Proceedings of the IEEE 63, 9 (1975), 1278–1308.
- SecuritytWeek. CSA’s cloud adoption, practices and priorities survey report. http://www.securityweek.com/data-security-concerns-still-challenge, 2015. Accessed: 09-01-2019.
- Pisces: A programmable, protocol-independent software switch. In Proc. ACM SIGCOMM (2016).
- Securing self-virtualizing ethernet devices. In Proc. Usenix Security Symp. (2015).
- Stecklina, J. Shrinking the hypervisor one subsystem at a time: A userspace packet switch for virtual machines. In Proc. ACM SIGPLAN/SIGOPS Conference on Virtual Execution Environments (VEE) (2014).
- Stone, R. PCI SR-IOV on FreeBSD. https://people.freebsd.org/~rstone/BSDCan_SRIOV.pdf. Accessed: 06-01-2019.
- The Fast Data Project. What is the Fast Data Project (FD.io)? https://fd.io/about, 2017. Accessed: 05-06-2018.
- Virtual network isolation: Are we there yet? In Proc. ACM Workshop on Security in Softwarized Networks: Prospects and Challenges (2018).
- Taking control of sdn-based cloud systems via the data plane. In Proc. ACM Symposium on SDN Research (SOSR) (2018).
- Accelerating open vswitch with integrated gpu. In Proc. ACM Workshop on Kernel-Bypass Networks (2017).
- Vanover, R. Virtual switching to become enhanced with Cisco and VMware announcement. http://www.techrepublic.com/blog/data-center/virtual-switching-to-become-enhanced-with-cisco-and-vmware-announcement, 2008. Accessed: 27-01-2017.
- VMware. VMware ESX 4.0 Update 1 Release Notes. https://bit.ly/2sFTuTy, 2009. Accessed: 05-06-2018.
- Virtual extensible local area network (VXLAN): A framework for overlaying virtualized layer 2 networks over layer 3 network. https://tools.ietf.org/html/rfc7348. Accessed: 01-06-2016.
- Zhao, Y. PCI: Linux kernel SR-IOV support. https://lwn.net/Articles/319651/, 2009. Accessed: 06-01-2019.
- All your vms are disconnected: Attacking hardware virtualized network. In Proc. ACM Conference on Data and Application Security and Privacy (CODASPY) (2017).