xBound: Join Size Lower Bounds

Abstract: Cloud database vendors invest substantial resources into their query optimizers, and for good reason. Cardinality estimation, a cornerstone of the optimizer, is critical for the selection of efficient query plans, as well as downstream tasks such as resource allocation and query scheduling. Yet, as many practitioners and researchers have noted, it is also the optimizer's Achilles heel. Prior studies on a number of industrial-strength databases show substantial cardinality estimation errors on all tested systems, with a far greater tendency to underestimate than to overestimate. Unfortunately, cardinality underestimation is more problematic than overestimation, as it misleads the optimizer to choose plans designed for small data, leading to underprovisioned CPU and memory. While previous work on pessimistic cardinality estimation has proposed provable join size upper bounds, such methods can only correct overestimation, leaving the more harmful problem of underestimation unaddressed. To fill this critical gap, we introduce xBound, the very first framework for deriving provable join size lower bounds. xBound successfully reduces underestimation in real systems: On the JOBlight benchmark, it corrects 17.5% of subexpression underestimates in DuckDB and 8.7% in PostgreSQL, while on a Microsoft enterprise workload, it fixes 36.1% of Fabric Data Warehouse's underestimates, demonstrating a significant step towards solving this long-standing problem.