AutoPerf: Automated Performance Engineering

Updated 4 July 2026

AutoPerf is a family of automation patterns that diagnose, experiment, and optimize performance in diverse systems including software, microprocessors, and parallel programs.
It employs machine learning techniques such as autoencoders, k-means clustering, and rule-based classifiers to detect anomalies and performance regressions.
AutoPerf also integrates with agentic recommender systems and reproducible benchmarks in C++ and autonomous driving, guiding both automated analysis and human-driven optimizations.

Searching arXiv for papers on AutoPerf and closely related usages to ground the article in current literature. arxiv_search(query="AutoPerf OR \"Automatic Microprocessor Performance Bug Detection\" OR \"A Zero-Positive Learning Approach for Diagnosing Software Performance Regressions\" OR \"AgenticRS-Architecture\" OR CppPerf OR ADPerf", max_results=10, sort_by="relevance") AutoPerf is a recurrent name in the literature on automated performance engineering, but it does not denote a single canonical system. The name appears in work on software performance regression diagnosis, microprocessor performance bug detection, task-parallel trace analysis, SPMD performance debugging, and, more recently, as a deployment-and-experimentation agent within an agentic recommender-system architecture. Related work also uses the term more broadly for automated performance-repair or performance-testing workflows in C++ repositories and autonomous driving systems (Alam et al., 2017, Barboza et al., 2020, Drebes et al., 2014, Liu et al., 2011, Zhang et al., 27 Mar 2026, Ho et al., 11 May 2026, Pham et al., 15 Oct 2025). This suggests that AutoPerf is best understood as a family of automation patterns for performance diagnosis, experimentation, and optimization rather than as a single standardized framework.

1. Terminological scope and recurring meanings

The name “AutoPerf” is attached to several distinct technical objects. In “AgenticRS-Architecture: System Design for Agentic Recommender Systems,” AutoPerf is one of three long-lived evolution agents, alongside AutoTrain and AutoFeature, responsible for performance, deployment, and online experimentation in industrial recommender systems (Zhang et al., 27 Mar 2026). In “A Zero-Positive Learning Approach for Diagnosing Software Performance Regressions,” AutoPerf is a machine-programming system for diagnosing software performance regressions using zero-positive learning, autoencoders, and hardware telemetry (Alam et al., 2017). In “Automatic Microprocessor Performance Bug Detection,” the described methodology is also presented as AutoPerf in the supplied technical explainer, where it learns the normal correlation between performance counters and IPC and detects when that relationship breaks on a new design (Barboza et al., 2020).

Other literature uses the name in a broader or analogical sense. The Aftermath extension for task-parallel programs is described as AutoPerf in the supplied explainer because it automates threshold-based and regression-based guidance over traces (Drebes et al., 2014). AutoAnalyzer for SPMD applications is likewise framed as an AutoPerf-style system because it automates bottleneck detection, localization, and root-cause analysis without apriori knowledge (Liu et al., 2011). CppPerf and ADPerf are not named AutoPerf in their titles, but the supplied descriptions explicitly position them as enabling or operationalizing AutoPerf for real-world C++ and autonomous driving, respectively (Ho et al., 11 May 2026, Pham et al., 15 Oct 2025).

Usage of AutoPerf	Domain	Core function
AutoPerf in AutoModel	Recommender systems	Performance, deployment, and online experimentation
AutoPerf with ZPL and autoencoders	Software regression diagnosis	Detect and diagnose performance regressions in changed functions
AutoPerf for microprocessor validation	Computer architecture	Detect performance bugs from counters↔IPC correlation breakage
AutoPerf in Aftermath / AutoAnalyzer narratives	Parallel-program performance debugging	Guide trace analysis or locate bottlenecks and root causes
AutoPerf in broader methodological sense	C++ repair and autonomous driving	Build reproducible performance benchmarks or generate latency-stressing tests

A common misconception is that AutoPerf always refers to automatic code repair. The surveyed literature is more heterogeneous: some systems detect the existence of bugs, some diagnose regressions in changed functions, some guide human investigation, and some manage deployment and experimentation rather than program repair (Barboza et al., 2020, Alam et al., 2017, Drebes et al., 2014, Zhang et al., 27 Mar 2026).

2. AutoPerf as an evolution agent in agentic recommender systems

Within AutoModel, AutoPerf is the evolution agent responsible for performance, deployment, and online experimentation. Its stated responsibilities are to allocate training and serving resources; select parallelism and compression strategies; manage inference optimization; manage deployment, staged rollout, and rollback; enforce risk boundaries under business requirements such as compute budget, latency, availability, and risk tolerance; configure A/B experiments including traffic split and rollback policies; monitor online metrics and risk events; and balance exploration and exploitation at the system level (Zhang et al., 27 Mar 2026).

Architecturally, AutoPerf sits at the evolution layer and operates on decision agents in the online recommendation stack, including recall, ranking, reranking, and policy components. Like the other evolution agents, it “builds its own perception–decision–execution–feedback loop behind standard interfaces.” For AutoPerf, perception ingests business requirements and internal signals such as model complexity, convergence, feature cost, and prior experiment outcomes; decision chooses resource allocations, compression and parallelism strategies, staged rollout and traffic-split policies, and risk boundaries; execution applies deployment and experimentation configurations to decision agents; and feedback evaluates decisions through online metrics and risk events, writes conclusions to the knowledge layer, and sends signals back to AutoTrain and AutoFeature (Zhang et al., 27 Mar 2026).

A central feature of the design is the coordination and knowledge layer. The coordination layer decomposes goals into task graphs and state machines governing agent invocation order, failure recovery, and termination. The knowledge layer stores problem definitions, data analyses, model and feature configurations, training and offline evaluation logs, online experiment conclusions, and reward signals and credit assignment across agents. AutoPerf both reads from and writes to this persistent memory. The paper explicitly states that “new variants and experimental outcomes are written to this layer in each iteration,” and that lifecycle state is stored as “queryable, linked metadata,” enabling reuse of prior deployments, experiment settings, and outcomes (Zhang et al., 27 Mar 2026).

The paper is specific about responsibilities and interfaces but not about many internal mechanisms. Exact schemas, indices, storage technology, detailed internal modules, concrete gating thresholds, guardrails, rollback triggers, and exact inference optimization techniques are not specified. Statistical methodology beyond A/B experiments is also not specified: multi-armed bandits, interleaving, confidence intervals, hypothesis tests, variance reduction, sequential testing, and power calculations are all absent from the paper’s description of AutoPerf (Zhang et al., 27 Mar 2026). This makes the contribution primarily architectural: deployment and experimentation become explicit optimization targets in a multi-agent recommender-system lifecycle.

3. AutoPerf for software performance regression diagnosis

In the software-regression setting, AutoPerf is a machine-programming system that automates the diagnosis of performance regressions using zero-positive learning, autoencoders, and hardware performance counters. It targets the case where regressions are rare, diverse, and difficult to label exhaustively, whereas normal performance is abundant. AutoPerf therefore models only negative data from the old version $P_i$ and detects anomalous behavior in the new version $P_{i+1}$ as deviation from the learned normal manifold (Alam et al., 2017).

Its workflow is function-level and change-focused. Given $P_i$ and $P_{i+1}$ , it automatically annotates functions that differ and assumes the regression lies among these modified functions. For each annotated function, it reads hardware performance counters at entry and exit, computes differential counts per invocation, and normalizes them with respect to instruction count and thread count. The paper reports 33 individualized counters as features and uses k-means to partition functions with similar performance signatures, training one autoencoder per cluster to improve scalability (Alam et al., 2017).

The learning objective is reconstruction of normal hardware-telemetry vectors. In the notation reported for the paper, if $x_i \in \mathbb{R}^d$ is a training sample and $A_{\text{foo}}(x)=f(g(x))$ is the autoencoder for function $\text{foo}$ , training minimizes

$\min_{\theta} \sum_{i=1}^{N_{\text{old}}} \mathcal{L}(x_i, A_{\text{foo}}(x_i)) \quad \text{with} \quad \mathcal{L}(x,\hat{x}) = \|x-\hat{x}\|_2^2.$

For a new sample $z_j$ , the anomaly score is

$E(z_j)=\|z_j-A_{\text{foo}}(z_j)\|_2^2.$

AutoPerf classifies the sample as anomalous when the reconstruction error exceeds

$P_{i+1}$ 0

where $P_{i+1}$ 1 and $P_{i+1}$ 2 are the mean and standard deviation of training reconstruction errors, and $P_{i+1}$ 3 is presented as a robust default (Alam et al., 2017).

For diagnosis, AutoPerf computes per-counter reconstruction errors and uses majority-vote ranking over anomalous samples to identify the most implicated counter. The paper ties this ranking to regression types such as true sharing, false sharing, and NUMA latency. HITM is described as indicative of true or false sharing, while OFFCORE_RESPONSE:REMOTE_DRAM is used as an indicator of NUMA latency (Alam et al., 2017).

The reported evaluation covers 10 real performance bugs across 7 benchmark and open-source programs: PARSEC applications, Phoenix applications, and open-source systems including Boost, Memcached, and MySQL. AutoPerf reports average profiling overhead of about 4%, with the highest overhead at 7% in MySQL. It achieved false-negative rate $P_{i+1}$ 4 across all anomalous programs tested and had false positives only for Boost and MySQL. The paper further reports that with $P_{i+1}$ 5, AutoPerf achieved mean TPR $P_{i+1}$ 6 and mean FPR $P_{i+1}$ 7 across 10 applications, outperforming the reported baselines based on a decision tree and a self-organizing-map approach (Alam et al., 2017).

4. Hardware and parallel-program AutoPerf variants

In microprocessor validation, AutoPerf is formulated as a two-stage machine-learning methodology for detecting performance bugs when no deterministic oracle for correct performance exists. Stage 1 trains a per-probe regression model that predicts IPC time series for performance probes from hardware performance-counter time series and optional static microarchitectural features. Stage 2 aggregates the resulting prediction errors across probes and classifies a new design as buggy or bug-free using normalized scores and a rule-based classifier (Barboza et al., 2020).

The reported stage-1 error for probe $P_{i+1}$ 8 is

$P_{i+1}$ 9

and the normalized stage-2 scores are

$P_i$ 0

The decision rule declares a design buggy if $P_i$ 1, or else if $P_i$ 2, with default thresholds $P_i$ 3 and $P_i$ 4. The best reported stage-1 model is XGBoost with 250 trees. Quantitatively, the method detects 91.5% of microprocessor core performance bugs whose average IPC impact is greater than 1% versus a bug-free design with zero false positives, and 100% of the studied memory-system bugs with zero false positives (Barboza et al., 2020).

For task-parallel programs, AutoPerf is described in the supplied explainer as an extension of the Aftermath trace-analysis tool. It adds two automated analyses over offline traces: a threshold-based high-level analysis of processor states and a linear-regression-based correlation analysis tying task durations to hardware or runtime indicators. A central threshold checks whether the machine is saturated with useful work: $P_i$ 5 The correlation analysis associates each task with counter deltas between task start and end, for example

$P_i$ 6

and then tests linear relations such as

$P_i$ 7

The paper emphasizes guidance rather than full automation of optimization; quantitative evaluation and scalability numbers are not reported (Drebes et al., 2014).

For SPMD programs, AutoAnalyzer embodies a closely related AutoPerf concept. It instruments programs into code regions, collects wall-clock time, CPU time, MPI communication time and bytes, disk I/O quantity and timing, and PAPI counters such as cycles, instructions, and cache accesses and misses. It then detects process behavior dissimilarity through clustering of per-process vectors and code-region behavior disparity through clustering of a normalized metric

$P_i$ 8

The system uses rough set theory to uncover root causes and was validated on production applications. According to the reported optimization results, performance improvements ranged from 20% to 170% after applying fixes guided by the analysis (Liu et al., 2011).

Taken together, these variants show that AutoPerf in architecture and parallel-computing settings is often oriented toward detection and localization rather than repair. The automation target is the performance-debugging workflow itself: measurement, anomaly scoring, bottleneck discovery, and root-cause narrowing.

5. AutoPerf as benchmark curation and latency-stressing test generation

The literature also uses AutoPerf in a broader methodological sense to describe automated workflows for mining performance-improving patches or generating performance-stressing tests. In C++, “CppPerf: An Automated Pipeline and Dataset for Performance-Improving C++ Commits” contributes CppPerf-Mine and CppPerf-DB as a foundation for AutoPerf in real-world repositories (Ho et al., 11 May 2026). CppPerf-Mine is a four-stage pipeline consisting of repository collection, structural commit filtering, LLM-based commit classification, and containerized build and test with fully reproducible Docker outputs. Repositories must be C++ projects with a root CMakeLists.txt and passing tests on the latest version; commits are filtered by a default 2020–2025 time window, size limit, C++-only scope, and exclusion of test modifications. The two-phase classifier uses qwen2.5:7b and qwen3:8b, and on a manually labeled sample of 405 commits reports 86.67% precision and 41.94% recall (Ho et al., 11 May 2026).

CppPerf-DB contains 347 manually verified execution-time-improving patches from 42 mature C++ repositories, with 39% multi-file patches, patch sizes from 1 to 6,069 changed lines, and a mining funnel that starts from 65,942 scanned commits and ends at 347 verified cases. For automated empirical validation where tests expose timing effects, the criterion is at least a 5% improvement and $P_i$ 9 under the Mann–Whitney test after 31 runs with the first run discarded as warm-up. Only 35 of the 347 patches have tests that directly demonstrate a statistically significant execution-time improvement under that protocol. In a preliminary evaluation, OpenHands configured with gpt-5-mini achieves 13.5% Correct Patch overall and 7.4% on multi-file cases, confirming that repository-level C++ performance repair remains an open challenge (Ho et al., 11 May 2026).

In autonomous driving, ADPerf is described as operationalizing AutoPerf for obstacle-detection latency testing. It generates realistic point-cloud modifications that increase 3D detection latency, measures latency and frame dropping, and evaluates downstream trajectory-prediction degradation. Latency is defined as the elapsed time between module input and output topics in the middleware. The per-frame delay is

$P_{i+1}$ 0

and availability is estimated as

$P_{i+1}$ 1

The paper reports, for Apollo on a nuScenes 10-scene sample, approximately 8 ms for 2D detection, approximately 116 ms for 3D detection, and approximately 4 ms for MSF fusion, yielding approximately 128 ms end-to-end obstacle detection; for Autoware in AWSIM random scenarios, it reports approximately 128.1 ms average latency for 3D detection (Pham et al., 15 Oct 2025).

ADPerf’s test-generation operators include adding noise just outside obstacle bounding boxes, adding obstacles by duplicating realistic obstacles, and moving obstacles toward the scene’s $P_{i+1}$ 2-center. The number of added noise points is defined by

$P_{i+1}$ 3

The evaluation reports that 3D obstacle detection is the primary performance bottleneck in both Apollo and Autoware, that latency grows with the number of detections, and that performance losses propagate downstream. It further reports detector-specific sensitivity patterns: PointPillars is more sensitive to added noise, while CenterPoint is more sensitive to added obstacles (Pham et al., 15 Oct 2025).

These two strands differ in domain and measurement substrate, but both make reproducibility central. CppPerf packages before-and-after versions with compilers, dependencies, and logs inside Docker images, while ADPerf uses explicit latency instrumentation, frame-drop rules, and queuing-network or Queuing Petri Net models to reason about end-to-end effects (Ho et al., 11 May 2026, Pham et al., 15 Oct 2025).

6. Common design patterns, limits, and interpretive themes

Across the surveyed literature, several technical patterns recur. First, AutoPerf systems almost always rely on telemetry that preserves operational realism: hardware performance counters for software regressions and microprocessor validation, trace events and counter samples for task-parallel debugging, code-region metrics and I/O counts for SPMD programs, online metrics and risk events for recommender-system deployment, and live middleware timestamps for autonomous driving (Alam et al., 2017, Barboza et al., 2020, Drebes et al., 2014, Liu et al., 2011, Zhang et al., 27 Mar 2026, Pham et al., 15 Oct 2025). Second, they elevate performance diagnosis from a monolithic benchmark score to structured evidence over localized units such as changed functions, probes, tasks, code regions, decision agents, or containerized patches.

Third, most systems close some form of feedback loop, but the loop differs by domain. AutoPerf in AutoModel is explicitly a perception–decision–execution–feedback agent with persistent memory in a knowledge layer (Zhang et al., 27 Mar 2026). The software-regression system trains on $P_{i+1}$ 4 and diagnoses $P_{i+1}$ 5, with retraining on newer “good” versions presented as a plausible update path (Alam et al., 2017). The microprocessor system learns from legacy designs and applies the learned counters↔IPC relation to a new design (Barboza et al., 2020). CppPerf-Mine iterates from mining to build-and-test packaging, and ADPerf iterates from generated scene perturbations to latency and availability measurements (Ho et al., 11 May 2026, Pham et al., 15 Oct 2025).

The limits are equally consistent. Several papers provide architectural blueprints or methodological pipelines but leave key internals unspecified. AutoPerf in AutoModel does not specify detailed algorithms for experiment design, statistical testing, drift detection, scaling policies, or API schemas (Zhang et al., 27 Mar 2026). The Aftermath extension does not report quantitative evaluation metrics or scalability measurements (Drebes et al., 2014). AutoAnalyzer emphasizes lightweight data collection but does not quantify overhead numerically in the supplied summary (Liu et al., 2011). CppPerf is limited to CMake repositories and automated timing evidence exists for only 35 of 347 patches (Ho et al., 11 May 2026). The microprocessor AutoPerf detects the presence of a bug but “is not a root-cause localization tool” (Barboza et al., 2020). ADPerf reports simulation and workload constraints, including limited scenario coverage and randomness in preprocessing (Pham et al., 15 Oct 2025).

A second misconception is that “automation” implies full autonomy. In the literature, some AutoPerf systems remain decisional aids rather than self-sufficient optimizers. Aftermath’s AutoPerf guides users to relevant trace slices and indicators rather than automatically transforming code (Drebes et al., 2014). AutoAnalyzer produces core attributes and CCRs that direct optimization, but the optimizations themselves remain external actions (Liu et al., 2011). Even in the recommender-system setting, the paper emphasizes interpretable, auditable orchestration and states that detailed validation and safety checks are not specified (Zhang et al., 27 Mar 2026).

In aggregate, AutoPerf names a research trajectory that treats performance engineering as an object of automation. Its concrete instantiations differ sharply—autoencoders over hardware telemetry, rule-based anomaly scores over IPC predictions, clustering and rough sets over code-region metrics, online rollout control in recommender systems, Dockerized mining pipelines for C++, and latency-stressing test generation for autonomous driving—but all share the premise that performance failures can be made legible through structured measurement, automated decision procedures, and explicit feedback loops (Alam et al., 2017, Barboza et al., 2020, Liu et al., 2011, Zhang et al., 27 Mar 2026, Ho et al., 11 May 2026, Pham et al., 15 Oct 2025).