Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash 99 tok/s
Gemini 2.5 Pro 55 tok/s Pro
GPT-5 Medium 23 tok/s
GPT-5 High 19 tok/s Pro
GPT-4o 108 tok/s
GPT OSS 120B 465 tok/s Pro
Kimi K2 179 tok/s Pro
2000 character limit reached

Effective Bug Detection in Graph Database Engines: An LLM-based Approach (2402.00292v1)

Published 1 Feb 2024 in cs.DB

Abstract: Graph database engines play a pivotal role in efficiently storing and managing graph data across various domains, including bioinformatics, knowledge graphs, and recommender systems. Ensuring data accuracy within graph database engines is paramount, as inaccuracies can yield unreliable analytical outcomes. Current bug-detection approaches are confined to specific graph query languages, limiting their applicabilities when handling graph database engines that use various graph query languages across various domains. Moreover, they require extensive prior knowledge to generate queries for detecting bugs. To address these challenges, we introduces DGDB, a novel paradigm harnessing LLMs(LLM), such as ChatGPT, for comprehensive bug detection in graph database engines. DGDB leverages ChatGPT to generate high-quality queries for different graph query languages. It subsequently employs differential testing to identify bugs in graph database engines. We applied this paradigm to graph database engines using the Gremlin query language and those using the Cypher query language, generating approximately 4,000 queries each. In the latest versions of Neo4j, Agensgraph, and JanusGraph databases, we detected 2, 5, and 3 wrong-result bugs, respectively.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Query optimizations over decentralized RDF graphs. In 2017 IEEE 33rd International Conference on Data Engineering (ICDE). IEEE, 139–142.
  2. DataStax Dylan Bethune-Waddell Expero Google Orion Health IBM Rafael Fernandes Robert Dale Seeq Amazon, Aurelius. 2023. JanusGraph. https://janusgraph.org/.
  3. Sjoerd Mullender Andreas Seltenreich, Bo Tang. 2022. SQLsmith. https://github.com/anse1/sqlsmith.
  4. Renzo Angles and Claudio Gutierrez. 2008. Survey of graph database models. ACM Computing Surveys (CSUR) 40, 1 (2008), 1–39.
  5. Dana Angluin. 1990. Negative results for equivalence queries. Machine Learning 5 (1990), 121–150.
  6. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
  7. Donald D Chamberlin and Raymond F Boyce. 1974. SEQUEL: A structured English query language. In Proceedings of the 1974 ACM SIGFIDET (now SIGMOD) workshop on Data description, access and control. 249–264.
  8. ChatGPT Informed Graph Neural Network for Stock Movement Prediction. arXiv preprint arXiv:2306.03763 (2023).
  9. Edgar Frank Codd. 1983. A relational model of data for large shared data banks. Commun. ACM 26, 1 (1983), 64–69.
  10. Chatlaw: Open-source legal large language model with integrated external knowledge bases. arXiv preprint arXiv:2306.16092 (2023).
  11. Andrzej Czerepicki. 2016. Application of graph databases for transport purposes. Bulletin of the Polish Academy of Sciences. Technical Sciences 64, 3 (2016), 457–466.
  12. SAP Enterprise. 2023. OrientDB Community Edition. http://www.orientdb.org/.
  13. Robert B Evans and Alberto Savoia. 2007. Differential testing: a new approach to change detection. In The 6th Joint Meeting on European software engineering conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering: Companion Papers. 549–552.
  14. Apache Software Foundation. 2023a. HugeGraph. https://hugegraph.apache.org/.
  15. Apache Software Foundation. 2023b. TinkerGraph. https://tinkerpop.incubator.apache.org/.
  16. Cypher: An evolving query language for property graphs. In Proceedings of the 2018 international conference on management of data. 1433–1445.
  17. Fixing recurring crash bugs via analyzing q&a sites (T). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 307–318.
  18. Graph Database-modelled Public Transportation Data for Geographic Insight Web Application. In 2022 IEEE/ACIS 23rd International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD). IEEE, 2–7.
  19. Edwin Hewitt and Leonard J Savage. 1955. Symmetric measures on Cartesian products. Trans. Amer. Math. Soc. 80, 2 (1955), 470–501.
  20. GDsmith: Detecting bugs in Cypher graph database engines. In Proceedings of ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA).
  21. Bitnine Global Inc. 2023a. Agensgraph. https://bitnine.net/agensgraph/.
  22. Neo4j Inc. 2023b. Neo4j. https://neo4j.com/.
  23. Word level predicate abstraction and refinement for verifying RTL Verilog. In Proceedings of the 42nd annual Design Automation Conference. 445–450.
  24. Matteo Kamm. 2022. Testing Graph Databases using Predicate Partitioning. Master’s thesis. ETH Zurich.
  25. Segment anything. arXiv preprint arXiv:2304.02643 (2023).
  26. Goods recommendation based on retail knowledge in a Neo4j graph database combined with an inference mechanism implemented in jess. In 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI). IEEE, 1–8.
  27. Marrying up regular expressions with neural networks: A case study for spoken language understanding. arXiv preprint arXiv:1805.05588 (2018).
  28. Llm-rec: Personalized recommendation via prompting large language models. arXiv preprint arXiv:2307.15780 (2023).
  29. Jun Ma and Bo Wang. 2023. Segment anything in medical images. arXiv preprint arXiv:2304.12306 (2023).
  30. William M McKeeman. 1998. Differential testing for software. Digital Technical Journal 10, 1 (1998), 100–107.
  31. OpenAI. 2022. ChatGPT. https://openai.com/blog/chatgpt/.
  32. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35 (2022), 27730–27744.
  33. Rob Reagan and Rob Reagan. 2018. Cosmos DB. Web Applications on Azure: Developing for Global Scale (2018), 187–255.
  34. Ensemfdet: An ensemble approach to fraud detection based on bipartite graph. In 2021 IEEE 37th International Conference on Data Engineering (ICDE). IEEE, 2039–2044.
  35. Manuel Rigger. 2022. SQLancer. https://github.com/sqlancer/sqlancer.
  36. Manuel Rigger and Zhendong Su. 2020a. Detecting optimization bugs in database engines via non-optimizing reference engine construction. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1140–1152.
  37. Manuel Rigger and Zhendong Su. 2020b. Finding bugs in database systems via query partitioning. Proceedings of the ACM on Programming Languages 4, OOPSLA (2020), 1–30.
  38. Manuel Rigger and Zhendong Su. 2020c. Testing database engines via pivoted query synthesis. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20). 667–682.
  39. Graph databases: new opportunities for connected data. ” O’Reilly Media, Inc.”.
  40. Marko A Rodriguez. 2015a. The gremlin graph traversal machine and language (invited talk). In Proceedings of the 15th Symposium on Database Programming Languages. 1–10.
  41. Marko A Rodriguez. 2015b. The gremlin graph traversal machine and language (invited talk). In Proceedings of the 15th Symposium on Database Programming Languages. 1–10.
  42. Recommendation of influenced products using association rule mining: Neo4j as a case study. SN Computer Science 2 (2021), 1–17.
  43. Donald R Slutz. 1998. Massive stochastic testing of SQL. In VLDB, Vol. 98. Citeseer, 618–622.
  44. solid IT. 2023. DB-Engines Ranking of Graph DBMS. https://db-engines.com/en/ranking/graph+dbms.
  45. Detecting Logic Bugs of Join Optimizations in DBMS. Proceedings of the ACM on Management of Data 1, 1 (2023), 1–26.
  46. An empirical study on recent graph database systems. In Knowledge Science, Engineering and Management: 13th International Conference, KSEM 2020, Hangzhou, China, August 28–30, 2020, Proceedings, Part I 13. Springer, 328–340.
  47. A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems 32, 1 (2020), 4–24.
  48. Jiawei Zhang. 2023. Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT. arXiv preprint arXiv:2304.11116 (2023).
  49. A survey of large language models. arXiv preprint arXiv:2303.18223 (2023).
  50. Finding bugs in Gremlin-based graph database systems via randomized differential testing. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis. 302–313.
Citations (1)
View on Semantic Scholar
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Ai Generate Text Spark Streamline Icon: https://streamlinehq.com

Paper Prompts

Sign up for free to create and run prompts on this paper using GPT-5.

List Streamline Icon: https://streamlinehq.com Explore 10 Community Prompts
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now