KEET Architecture Framework

• KEET Architecture is a modular LLM-agent framework for automated CUDA performance analysis and explanation.
• It orchestrates distinct agents to inspect source code, generate performance hypotheses, and analyze Nsight Compute profiles.
• The framework produces comprehensive reports with explicit metric citations, offering actionable insights for optimization.

KEET Architecture refers most directly to the Kernel Execution Explanation Toolkit, a modular LLM-agent–based framework for automatic analysis and explanation of GPU kernel performance. KEET is designed to interpret complex profiles produced by tools such as Nsight Compute, generating actionable, data-grounded analyses and optimization recommendations for CUDA kernels. Its technical foundation lies in explicit agent role decomposition and orchestration over structured inputs, producing explanations suitable both for human engineers and automated code optimization pipelines. KEET is not to be confused with unrelated similarly named TEE or IoT architectures (Davis et al., 6 May 2026).

1. System Overview and Objectives

KEET is an agentic, multi-stage toolkit targeted at the automated, interpretable diagnosis and optimization of CUDA kernel performance. Its pipeline ingests kernel source files ($.cu$, $.cuh$) and compiled Nsight Compute ($.ncu-rep$) reports, performing phased analyses that traverse code inspection, hypothesis generation, profile metric selection, bottleneck analysis, and aggregation into a comprehensive report. The primary objective is to produce, without direct human intervention:

• A summary of kernel algorithmic structure;
• Formalized, testable performance hypotheses;
• Bottleneck identification with direct metric citations;
• Concrete optimization suggestions.

Modularity, orchestrated LLM calls, and result caching are explicitly used to balance context window constraints and output reliability (Davis et al., 6 May 2026).

2. Modular Agentic Pipeline

The core design is a multi-pass sequence of discrete "agent roles," each implemented as a prompt template with orchestration logic. These include:

• Source Code Loader: Reads kernel and accompanying files to assemble the code corpus.
• Code Summarizer: Produces/upgrades algorithmic summaries for each kernel based on source contents and prior context.
• Performance Hypothesizer: Generates an explicit set of hypotheses about dominant bottlenecks or performance challenges, e.g., “because blockDim.x is small, occupancy may be low.”
• Profile Ingestion: Converts each Nsight Compute profile into structured JSON, extracting relevant per-kernel metrics.
• Profile Selector & Metric Selector: Matches untested hypotheses to available profiles and selects a minimal relevant metric subset (e.g., sm_warps_active, dram_throughput).
• Profile Analyzer: Synthesizes code context, metrics, and applied guidelines to produce natural-language explanations, each claim directly citing its metric.
• DrGPU Evaluator (Optional): Applies static DrGPU rules to metrics, incorporating them via LLM mediation.
• Analysis Aggregator: Compiles agent outputs, removes redundancy, and yields the structured analysis/report.
• Explanation Reviewer (Optional): Compares findings to the initial hypotheses, labeling each as Confirmed, Refuted, or Inconclusive.

Agents exchange data as cached JSON structures or plain-text prompt sections, with a controlling scheduler to ensure serial and, where possible, parallel operation for large input sets (Davis et al., 6 May 2026).

3. Data and Control Flow

KEET organizes its operation in three distinct programmatic phases:

1. Source Inspection Stage: Files are processed in round-robin or heuristic-determined order. The Code Summarizer and Performance Hypothesizer iterate to refine understanding and enumerate testable bottleneck conjectures.
2. Profile Inspection Stage: For each kernel profile, Metric Selector is invoked to extract metrics corresponding to existing hypotheses. The Profile Analyzer integrates these metrics, code context, and distilled performance guides to assemble an explanation, optionally calling on DrGPU suggestion filters.
3. Report Aggregation and Review: The Analysis Aggregator merges analyses, ordering bottlenecks by relevance, deduplicating claims, and organizing final report sections. Optional explanation validation compares each finding to its hypothesis.

Each LLM invocation uses explicit context caching to economize prompt token usage. If errors occur (timeouts, hallucinations, missing attributions), prompt constraints are tightened and the agent is retried once (Davis et al., 6 May 2026).

4. Algorithms, Metrics, and Heuristics

KEET implements several analytical formulas either directly (via Python drivers) or indirectly (within LLM outputs):

• Occupancy: $$\mathrm{occupancy} = \frac{\mathrm{Active\_Warps}}{\mathrm{Max\_Warps}}$$
• Memory Throughput: $$\mathrm{memory\_throughput} = \mathrm{Device\_Bandwidth} \times \mathrm{Utilization}$$
• Coalescing Efficiency: $$\mathrm{coalescing\_efficiency} = \frac{\mathrm{ideal\_bytes\_per\_sector}}{\mathrm{bytes\_per\_sector\_mem\_global\_op\_ld}}$$

Profile Analyzer outputs are required to explicitly cite underlying metric values (e.g., “L2 hit rate = 98.7% (metric dram__throughput = 5.2 GB/s)”). All derived values are computed or presented with inline attributions, ensuring data-grounding and interpretability (Davis et al., 6 May 2026).

Sample Python pseudocode (condensed from the architectural description):

def run_keet(source_files, ncu_profiles, metadata): code_summary = "" hypotheses = [] for f in source_files: file_desc = CodeSummarizer(f, code_summary) code_summary += file_desc + "\n" hypotheses = PerformanceHypothesizer(code_summary, hypotheses) analyses = [] for p in ncu_profiles: chosen_metrics = MetricSelector(p, hypotheses) analysis = ProfileAnalyzer(chosen_metrics, code_summary, hypotheses) if metadata.use_drgpu: drgpu_sugg = DrGPUEvaluator(p) analysis = merge_suggestions(analysis, drgpu_sugg) analyses.append(analysis) report = AnalysisAggregator(analyses) if metadata.review_enabled: report = ExplanationReviewer(report, hypotheses) return report

5. Agent Scheduling, Prompt Engineering, and Error Handling

Prompt engineering is central: each agent template incorporates explicit instructions to cite all evidence by raw metric name and value, strictly controlling hallucination. The orchestration logic enforces agent execution order, caches intermediate outputs by file checksum, and supports parallel agent invocations when multiple profile sets are present.

When LLM output does not meet quality requirements (missing citations, off-topic reasoning), the agent prompt is programmatically restricted and re-run. This two-tier fallback sequence improves reliability over single-pass prompting. Caching ensures previously computed code or hypothesis summaries are not recomputed when input files are unchanged (Davis et al., 6 May 2026).

6. Example Workflow and Evaluative Capabilities

A sample end-to-end execution on a “Fan2” kernel explicitly illustrates KEET’s loop:

• CodeSummarizer: “Fan2 computes a pairwise Gaussian distance matrix using two nested loops over i and j, storing results in global buffer.”
• PerformanceHypothesizer: “(H1) occupancy may be low due to blockDim.x=16; (H2) uncoalesced accesses because inner loop steps by non-unit stride.”
• MetricSelector: Extracts sm_warps_active, dramthroughput, smsp_bytes_per_sector_mem_global_op_ld.
• ProfileAnalyzer: Diagnoses memory-latency bound stall, citing “Scoreboard stalls (21.4 per warp issue), global memory coalescing poor (8.34 B/sector vs ideal 32 B/sector),” and recommending block size changes, data layout refactoring.
• DrGPU Evaluator: Suggests use of __ldg.
• AnalysisAggregator merges findings into a structured report, which is then annotated by ExplanationReviewer as having confirmed both H1 and H2.

A plausible implication is that such modular explanations substantially aid human and machine-driven code optimization tasks by providing concise, data-justified, and actionable insights (Davis et al., 6 May 2026).

7. Adaptability, Limitations, and Scope

KEET’s framework is adaptable to new GPU architectures and evolving performance metrics by updating LLMs and prompt templates, without changes to its orchestration logic. Its design requires structured input for CUDA source and profiling output to align all agent analyses. The system is geared towards interpretability and context control, rather than raw throughput. Error handling and context-window management are central design constraints. There is no evidence of integration with non-NVIDIA or non-CUDA platforms in current documentation (Davis et al., 6 May 2026).

---

KEET’s agentic and modular methodology distinguishes it among performance analysis systems. By formalizing code inspection, hypothesis-driven profiling, explicit metric selection, and orchestrated aggregation, it addresses the interpretability and usability deficits of static-rule and monolithic LLM approaches, providing a technically robust, extensible toolkit for GPU performance explanation and optimization (Davis et al., 6 May 2026).