Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
91 tokens/sec
Gemini 2.5 Pro Premium
51 tokens/sec
GPT-5 Medium
32 tokens/sec
GPT-5 High Premium
24 tokens/sec
GPT-4o
86 tokens/sec
DeepSeek R1 via Azure Premium
95 tokens/sec
GPT OSS 120B via Groq Premium
460 tokens/sec
Kimi K2 via Groq Premium
208 tokens/sec
2000 character limit reached

Pushing the Limits of In-Network Caching for Key-Value Stores (2407.21324v3)

Published 31 Jul 2024 in cs.NI

Abstract: We present OrbitCache, a new in-network caching architecture that can cache variable-length items to balance a wide range of key-value workloads. Unlike existing works, OrbitCache does not cache hot items in the switch memory. Instead, we make hot items revisit the switch data plane continuously by exploiting packet recirculation. Our approach keeps cached key-value pairs in the switch data plane while freeing them from item size limitations caused by hardware constraints. We implement an OrbitCache prototype on an Intel Tofino switch. Our experimental results show that OrbitCache can balance highly skewed workloads and is robust to various system conditions.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. Tommyds c library. https://www.tommyds.it/, Last accessed date: August 1, 2024, 2018.
  2. Advanced congestion & flow control with programmable switches. https://opennetworking.org/wp-content/uploads/2020/04/JK-Lee-Slide-Deck.pdf, Last accessed date: August 1, 2024, April 2020.
  3. Nvidia messaging accelerator (vma). https://docs.nvidia.com/networking/display/vmav9840, Last accessed date: August 1, 2024, 2021.
  4. Redis key-value store. https://redis.io/, 2023.
  5. Rocksdb: A persistent key-value store for flash and ram storage. https://rocksdb.org/, Last accessed date: August 1, 2024, 2023.
  6. Tofino switch. https://github.com/barefootnetworks/Open-Tofino, Last accessed date: August 1, 2024, 2023.
  7. Fawn: a fast array of wimpy nodes. In Proc. of ACM SOSP, page 1–14, New York, NY, USA, 2009. Association for Computing Machinery.
  8. Workload analysis of a large-scale key-value store. In Proc. of ACM SIGMETRICS, page 53–64, New York, NY, USA, 2012.
  9. Scaling out to a single-node 80gbps memcached server with 40terabytes of memory. In Proc. of USENIX HotStorage, pages 8–12, USA, 2015.
  10. P4: Programming protocol-independent packet processors. SIGCOMM Comput. Commun. Rev., 44(3):87–95, July 2014.
  11. Forwarding metamorphosis: Fast programmable match-action processing in hardware for sdn. In Proc. of ACM SIGCOMM, pages 99–110, 2013.
  12. Millions of low-latency state insertions on asic switches. Proc. of ACM CoNEXT, 1(CoNEXT3), nov 2023.
  13. Characterizing, modeling, and benchmarking rocksdb key-value workloads at facebook. In Proc. of USENIX FAST, Santa Clara, CA, February 2020.
  14. Benchmarking cloud serving systems with ycsb. In Proc. of ACM SoCC, page 143–154, New York, NY, USA, 2010. Association for Computing Machinery.
  15. P4xos: Consensus as a network service. IEEE/ACM Trans. Netw., 28(4):1726–1738, aug 2020.
  16. Small cache, big effect: Provable load balancing for randomly partitioned cluster services. In Proc. of ACM SoCC, New York, NY, USA, 2011.
  17. Brad Fitzpatrick. Distributed caching with memcached. Linux J., 2004(124):5–, August 2004.
  18. Leed: A low-power, fast persistent key-value store on smartnic jbofs. In Proc. of ACM SIGCOMM, page 1012–1027, New York, NY, USA, 2023. Association for Computing Machinery.
  19. P4db - the case for in-network oltp. In Proc. of ACM SIGMOD, page 1375–1389, 2022.
  20. In-network support for transaction triaging. Proc. VLDB Endow., 14(9):1626–1639, may 2021.
  21. Netchain: Scale-free sub-rtt coordination. In Proc. of USENIX NSDI, pages 35–49, Renton, WA, April 2018.
  22. Netcache: Balancing key-value stores with fast in-network caching. In Proc. of ACM SOSP, pages 121–136, 2017.
  23. Tea: Enabling state-intensive network functions on programmable switches. In Proc. of ACM SIGCOMM, page 90–106, 2020.
  24. Gyuyeong Kim. Holistic in-network acceleration for heavy-tailed storage workloads. IEEE Access, 11:77416–77428, 2023.
  25. Gyuyeong Kim. Netclone: Fast, scalable, and dynamic request cloning for microsecond-scale rpcs. In Proc. of ACM SIGCOMM, page 195–207, September 2023.
  26. In-network leaderless replication for distributed data stores. Proc. VLDB Endow., 15(7):1337–1349, Mar. 2022.
  27. Kvell: the design and implementation of a fast persistent key-value store. In Proc. of ACM SOSP, page 447–461, New York, NY, USA, 2019. Association for Computing Machinery.
  28. Pegasus: Tolerating skewed workloads in distributed storage with in-network coherence directories. In Proc. of USENIX OSDI, pages 387–406, November 2020.
  29. Switchtx: Scalable in-network coordination for distributed transaction processing. Proc. VLDB Endow., 15(11):2881–2894, jul 2022.
  30. Be fast, cheap and in control with switchkv. In Proc. of USENIX NSDI, pages 31–44, USA, 2016.
  31. Mica: A holistic approach to fast in-memory key-value storage. In Proc, of USENIX NSDI, page 429–444, USA, 2014.
  32. Incbricks: Toward in-network computation with an in-network cache. In Proc. of ACM ASPLOS, page 795–809, New York, NY, USA, 2017.
  33. Distcache: Provable load balancing for large-scale storage systems with distributed caching. In Proc. of USENIX FAST, pages 143–157, Boston, MA, February 2019.
  34. Scaling memcache at facebook. In Proc. of USENIX NSDI, pages 385–398, Berkeley, CA, USA, 2013.
  35. Scaling distributed machine learning with In-Network aggregation. In Proc. of USENIX NSDI, pages 785–808, April 2021.
  36. FarReach: Write-back caching in programmable switches. In Proc. of USENIX ATC, pages 571–584, Boston, MA, July 2023. USENIX Association.
  37. Vishal Shrivastav. Stateful multi-pipelined programmable switches. In Proc. of ACM SIGCOMM, page 663–676, New York, NY, USA, 2022. Association for Computing Machinery.
  38. A large scale analysis of hundreds of in-memory cache clusters at twitter. In Proc. of USENIX OSDI, pages 191–208. USENIX Association, November 2020.
  39. Netlock: Fast, centralized lock management using programmable switches. In Proc. of ACM SIGCOMM, page 126–138, 2020.
  40. Harmonia: Near-linear scalability for replicated storage with in-network conflict detection. Proc. VLDB Endow., 13(3):376–389, November 2019.
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Authors (1)

  1. Gyuyeong Kim