Papers
Topics
Authors
Recent
Search
2000 character limit reached

Intelligent Transaction Scheduling via Conflict Prediction in OLTP DBMS

Published 3 Sep 2024 in cs.DB | (2409.01675v1)

Abstract: Current architectures for main-memory online transaction processing (OLTP) database management systems (DBMS) typically use random scheduling to assign transactions to threads. This approach achieves uniform load across threads but it ignores the likelihood of conflicts between transactions. If the DBMS could estimate the potential for transaction conflict and then intelligently schedule transactions to avoid conflicts, then the system could improve its performance. Such estimation of transaction conflict, however, is non-trivial for several reasons. First, conflicts occur under complex conditions that are far removed in time from the scheduling decision. Second, transactions must be represented in a compact and efficient manner to allow for fast conflict detection. Third, given some evidence of potential conflict, the DBMS must schedule transactions in such a way that minimizes this conflict. In this paper, we systematically explore the design decisions for solving these problems. We then empirically measure the performance impact of different representations on standard OLTP benchmarks. Our results show that intelligent scheduling using a history increases throughput by $\sim$40\% on 20-core machine.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. Coordination avoidance in database systems. Proc. VLDB Endow., 8(3):185–196, Nov. 2014.
  2. Proactive transaction scheduling for contention management. In Proceedings of the 42nd Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 42, pages 156–167, New York, NY, USA, 2009. ACM.
  3. I. T. Bowman. Scalpel: Optimizing Query Streams Using Semantic Prefetching. PhD thesis, University of Waterloo, 2005.
  4. Serializable isolation for snapshot databases. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, SIGMOD ’08, pages 729–738, New York, NY, USA, 2008. ACM.
  5. Automatic partitioning of database applications. Proc. VLDB Endow., 5(11):1471–1482, July 2012.
  6. Schism: A workload-driven approach to database replication and partitioning. Proc. VLDB Endow., 3(1-2):48–57, Sept. 2010.
  7. Size-based scheduling to improve web performance. ACM Trans. Comput. Syst., 21(2):207–233, May 2003.
  8. Improving oltp scalability using speculative lock inheritance. Proc. VLDB Endow., 2(1):479–489, Aug. 2009.
  9. A. Kemper and T. Neumann. Hyper: A hybrid oltp&olap main memory database system based on virtual memory snapshots. In Proceedings of the 2011 IEEE 27th International Conference on Data Engineering, ICDE ’11, pages 195–206, Washington, DC, USA, 2011. IEEE Computer Society.
  10. Ermia: Fast memory-optimized database system for heterogeneous workloads. Ratio, 10(3):10–2, 2016.
  11. H. Kimura. Foedus: Oltp engine for a thousand cores and nvram. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, SIGMOD ’15, pages 691–706, New York, NY, USA, 2015. ACM.
  12. High-performance concurrency control mechanisms for main-memory databases. Proc. VLDB Endow., 5(4):298–309, Dec. 2011.
  13. Morsel-driven parallelism: A numa-aware query evaluation framework for the many-core age. In Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, SIGMOD ’14, pages 743–754, New York, NY, USA, 2014. ACM.
  14. Exploiting hardware transactional memory in main-memory databases. In 2014 IEEE 30th International Conference on Data Engineering, pages 580–591, March 2014.
  15. Algorithmic improvements for fast concurrent cuckoo hashing. In Proceedings of the Ninth European Conference on Computer Systems, page 27. ACM, 2014.
  16. Holistic query transformations for dynamic web applications. In Proceedings of the IEEE 25th International Conference on Data Engineering, 2009.
  17. Fast serializable multi-version concurrency control for main-memory database systems. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, SIGMOD ’15, pages 677–689, New York, NY, USA, 2015. ACM.
  18. S. Neuvonen et al. TATP telecommunication application transaction processing (benchmark description). http://tatpbenchmark.sourceforge.net/TATP_Description.pdf.
  19. Data-oriented transaction execution. Proc. VLDB Endow., 3(1-2):928–939, Sept. 2010.
  20. Self-driving database management systems. In CIDR, 2017.
  21. Skew-aware automatic database partitioning in shared-nothing, parallel oltp systems. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, SIGMOD ’12, pages 61–72, New York, NY, USA, 2012. ACM.
  22. Oltp on hardware islands. Proc. VLDB Endow., 5(11):1447–1458, July 2012.
  23. Design principles for scaling multi-core oltp under high contention. In Proceedings of the 2016 International Conference on Management of Data, SIGMOD ’16, pages 1583–1598, New York, NY, USA, 2016. ACM.
  24. Transaction chopping: Algorithms and performance studies. ACM Trans. Database Syst., 20(3):325–363, Sept. 1995.
  25. The end of an architectural era: (it’s time for a complete rewrite). In Proceedings of the 33rd International Conference on Very Large Data Bases, VLDB ’07, pages 1150–1160. VLDB Endowment, 2007.
  26. The Transaction Processing Council. TPC-C benchmark (revision 5.9.0), 2007. http://www.tpc.org/tpcc/spec/tpcc_current.pdf.
  27. Calvin: Fast distributed transactions for partitioned database systems. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, SIGMOD ’12, pages 1–12, New York, NY, USA, 2012. ACM.
  28. Addict: Advanced instruction chasing for transactions. Proc. VLDB Endow., 7(14):1893–1904, Oct. 2014.
  29. Speedy transactions in multicore in-memory databases. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, SOSP ’13, pages 18–32, New York, NY, USA, 2013. ACM.
  30. Scaling multicore databases via constrained parallel execution. In Proceedings of the 2016 International Conference on Management of Data, SIGMOD ’16, pages 1643–1658, New York, NY, USA, 2016. ACM.
  31. Using restricted transactional memory to build a scalable in-memory database. In Proceedings of the Ninth European Conference on Computer Systems, EuroSys ’14, pages 26:1–26:15, New York, NY, USA, 2014. ACM.
  32. J.-L. Weldon. Data base administration. Springer Science & Business Media, 2013.
  33. C. Yan and A. Cheung. Leveraging lock contention to improve oltp application performance. Proceedings of the VLDB Endowment, 9(5):444–455, 2016.
  34. Staring into the abyss: An evaluation of concurrency control with one thousand cores. Proc. VLDB Endow., 8(3):209–220, Nov. 2014.
  35. Tictoc: Time traveling optimistic concurrency control. In Proceedings of the 2016 ACM SIGMOD International Conference on Management of Data, volume 8, pages 209–220. VLDB Endowment, 2016.

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up