KEET: Explaining Performance of GPU Kernels Using LLM Agents

Abstract: Performance profiles of GPU kernels generated by tools such as Nsight Compute are rich in detail but are often challenging to interpret. To achieve the best performance possible on a given GPU architecture, kernel developers need to spend significant time analyzing and comparing profiles in the tool's graphical interface to identify and understand kernel performance bottlenecks. LLMs have shown promise in understanding complex data and generating natural language explanations. In this paper, we propose the Kernel Execution Explanation Toolkit (KEET), an LLM-based agentic framework for interpreting Nsight Compute profiles to generate useful and data-grounded natural language explanations of performance issues in GPU kernels, and suggestions for optimizations. We evaluate \toolname using several CUDA kernels of varying complexity on NVIDIA H100 GPUs. We find that the generated explanations, when provided as context, improve the quality of LLM code optimization and multiple-choice question answering in downstream tasks. We further demonstrate that the tool can be used to interpret performance data from large sets of profiles to improve the quality of optimization suggestions.

• The paper introduces an agentic LLM framework that decomposes GPU kernel analysis into specialized roles for source code inspection and profile interpretation.
• It demonstrates significant improvements in optimization, achieving up to 14.0x speedup and an 82% MCQ score over rule-based methods.
• Ablation studies reveal that optimal combinations of metric and profile selector agents balance information density and noise for enhanced performance.

KEET: Agentic LLM Interpretation of GPU Kernel Performance Profiles

Agentic Framework Architecture

KEET introduces an agent-based LLM framework for interpreting NVIDIA Nsight Compute (NCU) profiling data to explain GPU kernel performance, pinpoint bottlenecks, and suggest optimizations. The architecture decomposes the analysis pipeline into specialized agent roles: source code inspection, profile analysis, metric selection, and hypothesis-driven reasoning. This modular approach facilitates scalable analysis, interpretability of intermediate outputs, and enables adaptation to multi-profile and multi-source scenarios without static rule dependence.

The pipeline first employs source code inspection agents to form a semantic understanding of the kernel algorithm and generate preliminary hypotheses about performance. Subsequent agents iteratively select relevant profiles and metrics, perform in-depth empirical analysis, and finally, aggregate insights to produce data-grounded natural language reports. The design inherently allows the incorporation of external tools (e.g., DrGPU) to supplement suggestions.

Figure 1: KEET architecture showing agent roles and data flow, emphasizing decoupled Source Code Inspection and Profile Inspection stages.

Evaluation Methodology and Metrics

KEET's efficacy is rigorously assessed via two downstream tasks: multiple-choice question answering (MCQ) and code optimization (OPT). The evaluation employs a benchmark suite covering diverse algorithmic motifs and domains, including Rodinia kernels, LULESH, and XSBench, all profiled on NVIDIA H100 GPUs. The MCQ task measures context-driven interpretive accuracy (score@1), while OPT quantifies performance enhancement (speedup@1, pass@1), tracking speedup and correctness in LLM-generated kernel code.

KEET is compared against rule-based tools (DrGPU, GPA) and primitive baselines (raw code, raw metrics). Notably, KEET explanations enable downstream LLMs to implement more effective code optimizations and achieve higher MCQ scores, with statistically significant improvements in both metrics relative to baselines.

Ablation Studies: Agent Role Impact and Scalability

Ablation studies isolate the influence of key agent roles and configuration parameters:

• Metric Selector Agent: Enabling this role yields significant gains in speedup@1 for most kernels, with up to a five-point harmonic mean improvement in single-profile cases. Its impact diminishes in multi-profile contexts.
• Profile Selector Agent: Increases both mean and maximal speedup@1 in multi-profile settings, particularly for kernels with complex resource interactions (e.g., LULESH).
• Profile Count Scaling: There exists an optimal profile count (e.g., 4 for LULESH, 16 for XSBench) beyond which added profiles introduce noise and degrade performance, demonstrating the tradeoff between information density and context overload.

  Figure 2: Effect of enabling/disabling Metric Selector on speedup@1, showing increased speedup across most kernels.

  

Figure 3: Left: Improved speedup@1 with Profile Selector enabled in multi-profile settings. Right: Speedup@1 by profile count, indicating optimal scaling before noise effects.

Comparative Performance Analysis

KEET outperforms both rule-based and baseline approaches in downstream tasks, supporting robust interpretation and actionable optimization. In MCQ, KEET achieves an average score@1 of 82%, outperforming baseline and DrGPU methods for most kernels. In OPT, KEET delivers maximal speedup@1 on the majority of applications, with notable absolute speedups such as 14.0x for the gaussian kernel, surpassing baseline and DrGPU-assisted approaches.

Figure 4: Speedup@1 achieved by different LLMs when fed KEET agentic reports, highlighting sensitivity to LLM choice and KEET’s cross-model effectiveness.

Figure 5: MCQ task score@1 grouped by application and context, showing KEET’s advantage in interpretive accuracy.

Figure 6: Pass@1 for code optimization, demonstrating KEET's reliability in generating valid CUDA kernels.

Figure 7: Speedup@1 across applications and approaches, evidencing KEET’s superior performance enhancement capability.

Optimization Technique Taxonomy

Analysis of optimization attempts reveals that KEET’s agent-driven recommendations favor register-level operations and memory access restructuring, yielding higher performance than generic suggestions stereotypical of static tools. Techniques such as $\verb|__ldg|$ instructions, removal of unnecessary barriers, and tailored memory coalescing adjustments contribute markedly to speedup. KEET avoids pitfalls typical in naive block/thread tuning, thereby reducing optimization-induced slowdowns.

Figure 8: Heatmap correlating outcome categories with optimization techniques, indicating higher failure rates for loop, memory, and block configuration modifications.

Practical and Theoretical Implications

KEET demonstrates the viability of agentic LLM-driven interpretation of low-level GPU performance metrics and kernel code. The framework supports dynamic integration of new performance metrics and hardware innovations via model updates, paving the way for automated optimization pipelines in HPC. Its success underscores the utility of data-grounded natural language analysis in augmenting both LLM autonomous reasoning and human-in-the-loop workflows. KEET’s modularity further enables extension to multi-vendor profiling, broader autotuning scenarios, and integration with IDEs.

Future developments may incorporate autotuning frameworks for grid and block size selection, deeper analysis of architectural features, and expanding agent roles to contextualize cross-kernel interactions. The methodology also calls for research into context utilization efficiencies for large dataset interpretations.

Conclusion

KEET advances automated GPU kernel performance analysis by deploying specialized LLM agents for NCU profile explanation, hypothesis refinement, and adaptive optimization suggestion. Empirical evaluation substantiates KEET’s effectiveness in enabling downstream LLMs to achieve higher interpretive accuracy and superior code optimization outcomes than rule-based or bare-bones baselines. KEET’s agentic modularity, interpretability, and extensibility place it as a valuable analytical tool for both autonomous and human-guided performance engineering in modern HPC, with potential to scale to future architectures and profiling paradigms (2605.04467).