Dice Question Streamline Icon: https://streamlinehq.com

Develop versatile input/output management for indefinite-sized outputs in ExKernel

Develop a more versatile input/output management mechanism within the ExKernel programming model to safely and efficiently support operators that produce indefinite-sized outputs while operating on datasets that exceed GPU memory capacity.

Information Square Streamline Icon: https://streamlinehq.com

Background

The ExKernel interface in Vortex maps inputs and outputs into chunks that fit GPU memory and allows kernels to operate in a pipelined fashion while Exchange handles IO. For operators with indefinite output sizes, the current guidance is to conservatively size input chunks to avoid GPU memory overflow, which can be suboptimal.

A more robust input/output management mechanism is needed to handle dynamic or unknown output sizes without over-constraining chunk sizes, improving both usability and performance for a wider range of operators.

References

Developing a more versatile input/output management mechanism for indefinite-sized outputs is left as future work.

Vortex: Overcoming Memory Capacity Limitations in GPU-Accelerated Large-Scale Data Analytics (2502.09541 - Yuan et al., 13 Feb 2025) in Section 5: IO-Decoupled Programming Model, Design Details of ExKernel (Output management)