- The paper introduces SPORTSQL, a system that leverages LLMs to translate natural language queries into dynamic SQL for real-time sports analytics.
- It employs a modular pipeline with hybrid storage, prompt-guided entity recognition, and API-driven table generation to manage evolving sports data.
- Experimental results on DSQABench highlight robust performance with noted challenges in SQL predicate complexity for multi-primitive queries.
SPORTSQL: An Interactive System for Real-Time Sports Reasoning and Visualization
System Overview and Motivation
SPORTSQL is a modular pipeline designed to enable interactive, real-time querying and visualization of dynamic sports data, with a focus on the English Premier League. The system leverages LLMs for natural language-to-SQL translation, schema linking, and visualization selection while operating over a live, temporally indexed database sourced from the Fantasy Premier League API. SPORTSQL’s objective is to bridge the gap between natural language interfaces and evolving, structurally complex relational data—addressing the challenges of compositional, temporal, and relational reasoning that static datasets cannot meet.
The architecture is explicitly constructed to handle continuously updating schemas and distributed sources. Queries are translated via prompt-guided LLMs, which operate solely on schema metadata rather than data values, allowing scaling and robustness for evolving domains. The result is an interactive interface that supports both tabular and multimodal outputs, enabling both experts and non-experts to explore sports statistics with precision and visual feedback.
System Architecture
SPORTSQL comprises several critical stages:
- Database Streaming and Hybrid Storage: Structured data is sourced from the public FPL API, normalized, de-duplicated, and stored in a MariaDB backend. The two-tiered storage strategy persists core relations (players, teams, fixtures) updated nightly while query-dependent tables (e.g., "past 5 games") are materialized on demand and discarded post-query, ensuring granularity and efficiency.
- Entity Recognition: LLMs resolve nicknames, abbreviations, and informal references via prompt-guided entity grounding. Canonical player/team names are determined and a wildcard SQL query maps surface forms to unique identifiers, addressing frequent string-matching failures in dynamic domains.
- SQL Generation and Execution: Conditioned on schema and grounded entities, LLMs generate executable SQL. Custom prompt elements—table hints, synonym mappings, column cautions, derived-field formulas, and scale explanations—mitigate common errors in semantic parsing. Non-materialized tables prompt just-in-time API requests to support real-time, query-specific views.
- Visualization Generation: The pipeline detects when visual interpretation (chart, trend, ranking) is appropriate—either from explicit user requests or structural patterns in output dataframes. A code-generating LLM produces validated Python code (Matplotlib, Seaborn) for rendering, promoting multimodal analytics and interpretability.
This multi-stage process ensures efficient, accurate analytics, maintaining a persistent database under 5GB, and supports temporally indexed, player-level queries for granular analysis.
Evaluation: DSQABench Benchmark
SPORTSQL is quantitatively evaluated on DSQABench, a benchmark of 1,793 queries covering diverse reasoning skills, multiple modalities, and compositional complexity. Each query has manually authored SQL, gold answers, and database snapshots for reference.
- Dataset Composition: Queries span scalar, tabular, and visual outputs, with 398 requiring dynamic API-accessed tables.
- Annotation Process: Questions originate from template-based generation and manual crafting, focusing on linguistic diversity and compositional control.
This benchmark provides rigorous, realistic evaluation for symbolic QA systems operating in domains with evolving data.
Experimental Results
Two LLMs, GPT-4o and Gemini-2.0 Flash, are evaluated. A type-aware metric assesses string exact match and table quality via TabEval (precision, recall, macro-F1).
- Numerical Results: GPT-4o achieves 80.48% exact-match accuracy for string answers and 0.75 macro-F1 for tables. Gemini-2.0 attains 76.23% and 0.69, respectively, with GPT-4o outperforming by 4.2 points in exact match and 0.05 in macro-F1.
- Completeness vs. Correctness: Completeness scores exceed correctness for both models, indicating better column identification than row selection—highlighting SQL predicate complexity as a source of error.
Primitive-Based Analysis
Query templates are annotated by reasoning primitives (Calculate, Compare, Filter, Order, Manipulate, Retrieve):
- Single Primitive Queries: High accuracy (Retrieve: 100%, Order: 97.6%).
- Compound Queries: Accuracy declines with more primitives; dual-primitives (Filter+Calculate) drop to 22.3%, Manipulate operations score 15.4%.
- Complexity Trends: Accuracy starts at 93% for single-primitive queries, stabilizing near 50% for three or more primitives, illustrating bottlenecks in aggregation and table manipulation.
Bottleneck Clause Pairs
Pairwise primitive accuracy is highest for Retrieve+Order (≥88%) and lowest for pairs involving Manipulate or Calculate (~15%), marking these operations as critical points for system improvement in future work.
Exemplars and Qualitative Analysis
Generated queries, SQL executions, and visualizations demonstrate system capabilities in handling both scalar and tabular outputs, mapping natural queries to precise responses, and providing timely multimodal analytics.
Related Work Context
SPORTSQL uniquely addresses the limitations of static Text-to-SQL and unstructured Sports QA. Existing semantic parsing benchmarks (e.g., Spider, SyntaxSQLNet) are restricted to static schemas and lack temporal, domain-driven entity resolution. Prior sports QA systems focus on broadcast text or passage retrieval, lacking symbolic execution and aggregation. SPORTSQL's contribution is the integration of dynamic schema reasoning, temporally indexed SQL, and visualization capability in sports analytics.
Implications and Future Directions
Practical implications are significant: SPORTSQL provides domain-agnostic access to evolving structured data, enabling users—regardless of expertise—to conduct real-time, compositional reasoning and multimodal analysis. The modular architecture is generalizable to other domains (finance, healthcare, elections) where schema drift and data freshness are critical.
Theoretical implications center on prompt engineering, schema-only symbolic reasoning, and dynamic, compositional QA—advancing multimodal interpretability, entity grounding, and real-time analytics integration for LLM-driven interfaces.
Future developments will involve support for more complex, multi-turn queries, cross-season analysis, domain extension, multilingual LLMs, retrieval-augmented generation, and adaptive schema remapping to further generalize and robustify the system.
Conclusion
SPORTSQL establishes an extensible, interpretable platform for natural language-driven sports analytics, combining schema-guided prompt engineering, real-time data integration, and multimodal visualization. DSQABench provides a benchmark for dynamic QA systems, revealing performance strengths and bottlenecks in compositional reasoning. The work sets the stage for scalable domain-agnostic symbolic QA, informing future benchmarks and system development in dynamic data domains.