Text2GraphQuery-Bench: First Benchmark for All Mainstream Graph Query Languages
Text2GraphQuery-Bench has been launched by researchers as the inaugural benchmark that encompasses all major declarative property graph query languages, including Cypher, GQL, and SQL/PGQ. This benchmark tackles the issue of converting natural language into functional graph queries, which is essential for allowing large language models (LLMs) to act as interfaces for Graph Database Management Systems (GDBMS). Current benchmarks fall short in language diversity, depend on inflexible synthesis, and lack thorough evaluation. Comprising 267,276 pairs of (Question, Graph Query) across 34 databases and 13 domains, Text2GraphQuery-Bench is designed for adaptation from various sources. This advancement is vital for data analysis, as graph query languages are often more complex and less intuitive than SQL, posing learning challenges for users. The benchmark seeks to unify evaluation and promote advancements in text-to-graph-query systems.
Key facts
- Text2GraphQuery-Bench is the first benchmark covering Cypher, GQL, and SQL/PGQ.
- It contains 267,276 (Question, Graph Query) pairs.
- The benchmark spans 34 databases and 13 domains.
- It addresses limitations of existing benchmarks: limited language coverage, rigid synthesis, and lack of comprehensive evaluation.
- The benchmark supports adaptation from heterogeneous resources.
- Graph query languages are diverse and less familiar than SQL, making them harder to learn.
- Text2GraphQuery systems enable LLMs to serve as interfaces for GDBMS.
- The benchmark is presented in arXiv paper 2602.11745.
Entities
—