Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Hydro: Adaptive Query Processing of ML Queries (2403.14902v1)

Published 22 Mar 2024 in cs.DB and cs.LG

Abstract: Query optimization in relational database management systems (DBMSs) is critical for fast query processing. The query optimizer relies on precise selectivity and cost estimates to effectively optimize queries prior to execution. While this strategy is effective for relational DBMSs, it is not sufficient for DBMSs tailored for processing ML queries. In ML-centric DBMSs, query optimization is challenging for two reasons. First, the performance bottleneck of the queries shifts to user-defined functions (UDFs) that often wrap around deep learning models, making it difficult to accurately estimate UDF statistics without profiling the query. This leads to inaccurate statistics and sub-optimal query plans. Second, the optimal query plan for ML queries is data-dependent, necessitating DBMSs to adapt the query plan on the fly during execution. So, a static query plan is not sufficient for such queries. In this paper, we present Hydro, an ML-centric DBMS that utilizes adaptive query processing (AQP) for efficiently processing ML queries. Hydro is designed to quickly evaluate UDF-based query predicates by ensuring optimal predicate evaluation order and improving the scalability of UDF execution. By integrating AQP, Hydro continuously monitors UDF statistics, routes data to predicates in an optimal order, and dynamically allocates resources for evaluating predicates. We demonstrate Hydro's efficacy through four illustrative use cases, delivering up to 11.52x speedup over a baseline system.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. GPT4All: Training an Assistant-style Chatbot with Large Scale Data Distillation from GPT-3.5-Turbo. https://github.com/nomic-ai/gpt4all.
  2. Physical Representation-Based Predicate Optimization for a Visual Analytics Database. In ICDE. 1466–1477.
  3. Ron Avnur and Joseph M Hellerstein. 2000. Eddies: Continuously Adaptive Query Processing. SIGMOD (2000), 12.
  4. Proactive re-optimization. In SIGMOD. 107–118.
  5. Seiden: Revisiting Query Processing in Video Database Systems. PVLDB (2023), 2289–2301.
  6. MIRIS: Fast Object Track Queries in Video. In SIGMOD. 1907––1921.
  7. Content-Based Routing: Different Plans for Different Data. PVLDB (2005).
  8. FiGO: Fine-Grained Query Optimization in Video Analytics. In SIGMOD. 559–572.
  9. Spatial and Temporal Constrained Ranked Retrieval over Videos. PVLDB (2022), 3226–3239.
  10. Ranked Window Query Retrieval over Video Repositories. In ICDE. 2776–2791.
  11. Evaluating Temporal Queries Over Video Feeds. In SIGMOD. 287–299.
  12. Serving Heterogeneous Machine Learning Models on Multi-GPU Servers with Spatio-Temporal Sharing. In ATC. 199–216.
  13. Zeus: Efficiently Localizing Actions in Videos using Reinforcement Learning. In SIGMOD. 545–558.
  14. InferLine: latency-aware provisioning and scaling for prediction serving pipelines. In SoCC. 477–491.
  15. Clipper: A Low-Latency Online Prediction Serving System. (2017), 613–627.
  16. Adaptive Query Processing. Foundations and Trends in Databases (2007), 1–140.
  17. GSLICE: controlled spatial sharing of GPUs for a scalable inference platform. In SoCC. 492–506.
  18. An Image Is Worth 16X16 Words: Transformers For Image Recognition At Scale. ICLR (2021).
  19. Jocher Glenn. 2020. YOLOv5 by Ultralytics. https://github.com/ultralytics/yolov5
  20. G. Graefe. 1995. The Cascades Framework for Query Optimization. IEEE Data Eng. Bull. (1995), 19–29.
  21. Serving DNNs like Clockwork: Performance Predictability from the Bottom Up. (2020), 443–462.
  22. Cocktail: A Multidimensional Optimization for Model Serving in Cloud. In NSDI. 1041–1057.
  23. Joseph M Hellerstein. 1994. Practical predicate placement. In SIGMOD. 325–335.
  24. Focus: Querying Large Video Datasets with Low Latency and Low Cost. In OSDI. 269–286.
  25. Scrooge: A Cost-Effective Deep Learning Inference System. In SoCC. 624–638.
  26. Dynamic Space-Time Scheduling for GPU Inference. http://arxiv.org/abs/1901.00041
  27. Chameleon: scalable adaptation of video analytics. In SIGCOMM. 253––266.
  28. Navin Kabra and David J Dewitt. 1998. Mid-Query Re-Optimization of Sub-Optimal Execution Plans. SIGMOD (1998).
  29. EVA: An End-to-End Exploratory Video Analytics System. In DEEM. 1–5.
  30. BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics. In VLDB. 533––546.
  31. NoScope: Optimizing Deep CNN-Based Queries over Video Streams at Scale. In VLDB. 1586––1597.
  32. Approximate Selection with Guarantees using Proxies. PVLDB (2020), 1990–2003.
  33. Accelerating Approximate Aggregation Queries with Expensive Predicates. PVLDB (2021), 2341–2354.
  34. TASTI: Semantic Indexes for Machine Learning-based Queries over Unstructured Data. In SIGMOD. 1934–1947.
  35. Extract-Transform-Load for Video Streams. PVLDB (2023), 2302–2315.
  36. Video Monitoring Queries. In ICDE. 1285–1296.
  37. Top-K Deep Video Analytics: A Probabilistic Approach. In SIGMOD. 1037–1050.
  38. Accelerating Machine Learning Inference with Probabilistic Predicates. In SIGMOD. 1493––1508.
  39. Themis: Fair and Efficient GPU Cluster Scheduling. In NSDI. 289–304.
  40. ExSample: Efficient Searches on Video Repositories through Adaptive Sampling. arXiv:2005.09141 [cs] (2020).
  41. NVIDIA. 2020. TensorRT Inference Server. [Online] Available from: https://github.com/NVIDIA/tensorrt-inference-server.
  42. NVIDIA. 2021. NVIDIA Multi-Process Service Introduction. [Online] Available from: https://docs.nvidia.com/deploy/mps/index.html.
  43. TensorFlow-Serving: Flexible, High-Performance ML Serving. http://arxiv.org/abs/1712.06139
  44. Optimizing Video Analytics with Declarative Model Relationships. PVLDB (2022), 447–460.
  45. INFaaS: Automated Model-less Inference Serving. (2021), 397–411.
  46. Nexus: a GPU cluster engine for accelerating DNN-based video analysis. In SOSP. 322–337.
  47. Ultralytics. 2023. Ultralytics YOLOv8. [Online] Available from: https://github.com/ultralytics/ultralytics.
  48. Carl A Waldspurger and William E Weihl. 1994. Lottery Scheduling: Flexible Proportional-Share Resource Management. OSDI (1994).
  49. Serving and Optimizing Machine Learning Workflows on Heterogeneous Infrastructures. PVLDB (2022), 406–419.
  50. Gandiva: Introspective Cluster Scheduling for Deep Learning. In OSDI. 595–610.
  51. EVA: A Symbolic Approach to Accelerating Exploratory Video Analytics with Materialized Views. In SIGMOD. 602–616.
  52. Optimizing Machine Learning Inference Queries with Correlative Proxy Models. PVLDB (2022), 2032–2044.
  53. Skylar Yau. 2023. Dog Breed Classifier ViT. https://huggingface.co/skyau/dog-breed-classifier-vit
  54. Peifeng Yu and Mosharaf Chowdhury. 2020. Salus: Fine-Grained GPU Sharing Primitives for Deep Learning Applications. In MLSys.
  55. EQUI-VOCAL: Synthesizing Queries for Compositional Video Events from Limited User Interactions. PVLDB (2023), 2714–2727.
  56. G-NET: Effective GPU Sharing in NFV Systems. In NSDI. 187–200.
  57. Yuhao Zhang and Arun Kumar. 2019. Panorama: A Data System for Unbounded Vocabulary Querying over Video. In VLDB. 477–491.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now