ProfilingAgent for Hardware-Aware Model Compression

Updated 10 July 2026

ProfilingAgent is a profiling-guided multi-agent system that integrates static and dynamic metrics to generate architecture-specific compression strategies for vision models.
It uses LLM-driven agents to execute structured pruning and dynamic quantization, optimizing accuracy, parameter count, and runtime based on per-layer performance data.
Empirical evaluations on ImageNet and CIFAR datasets show that ProfilingAgent can maintain or improve accuracy while achieving significant memory savings and speedups.

ProfilingAgent is a profiling-guided, multi-agent system that uses LLMs as decision-making components to automate hardware-aware pruning and quantization of vision foundation models. It integrates static metrics such as MACs and parameter counts with dynamic signals such as per-layer latency and memory, and uses those signals to generate architecture-specific compression strategies rather than uniform, architecture-agnostic heuristics. The system was evaluated on ImageNet-1K, CIFAR-10, and CIFAR-100 with ResNet-101, ViT-B/16, Swin-B, and DeiT-B/16, where pruning maintained competitive or improved accuracy and quantization achieved up to 74% memory savings with consistent inference speedups of up to 1.74 times faster (Jafari et al., 6 Sep 2025).

1. Problem setting and conceptual basis

ProfilingAgent addresses a specific limitation of conventional model compression pipelines. Modern foundation models, including CNNs such as ResNet-101 and transformer-based vision models such as ViT-B/16, Swin-B, and DeiT-B/16, have tens to hundreds of millions of parameters and require billions of MACs per inference. Traditional compression methods such as structured pruning and post-training quantization are often applied with uniform or static heuristics, for example L1/L2-norm pruning or uniform PTQ, and therefore ignore architectural and runtime heterogeneity. Different layers contribute differently to latency, memory usage, and accuracy sensitivity, and those contributions depend on the deployment substrate rather than on parameter count alone (Jafari et al., 6 Sep 2025).

The defining claim of ProfilingAgent is that profilers already expose rich per-layer metrics, but those signals are rarely integrated into automated compression workflows. ProfilingAgent closes that loop by elevating profiling to a first-class input to agentic reasoning. The system reasons jointly over layer names and types, MACs and parameter counts, per-layer latency and memory, and overall model performance metrics, then converts those observations into machine-executable pruning and quantization plans. This places profiling not after optimization as a diagnostic artifact, but inside optimization as the primary basis for decision making.

This also shifts the optimization target from uniform sparsification to a hardware-aware trade-off. In the iterative pruning component, model comparison follows a hierarchical objective: first maximize accuracy, then minimize parameter count, and finally minimize latency. A plausible implication is that ProfilingAgent is designed less as a pure compression engine than as a profile-guided selector over a three-way Pareto surface of accuracy, size, and runtime.

2. Multi-agent architecture

ProfilingAgent is organized into three principal components: Profiling, Optimization, and Iterative Pruning. Within these components, the system includes the Acquisition Agent, Input Shape Resolver Agent, ProfilingAgent, Analysis Agent, Pruning Agent, Quantization Agent, Evaluation Agent, and Iterative Pruning Agent (Jafari et al., 6 Sep 2025).

Component	Agents	Function
Profiling	Acquisition, Input Shape Resolver, ProfilingAgent, Analysis	Retrieve model, resolve input shape, collect metrics, propose strategies
Optimization	Pruning, Quantization, Evaluation	Apply compression and benchmark original and modified models
Iterative Pruning	Iterative Pruning Agent	Refine pruning across multiple rounds using prior analysis and evaluation

The Acquisition Agent retrieves pretrained models and processors from Hugging Face and configures the AutoImageProcessor to match the model’s preprocessing requirements. The Input Shape Resolver Agent uses GPT-4o to determine the expected input tensor shape and returns a JSON object with fields channels, height, width, and sequence_length. The ProfilingAgent collects static metrics with ptflops and dynamic metrics with PyTorch Profiler plus manual per-submodule timing. The Analysis Agent then reads the profiling report and asks GPT-4o for structured pruning and quantization recommendations in JSON form.

The optimization side is deliberately split. The Pruning Agent applies regex-based recommendations using a DependencyGraph so that structural pruning remains dependency-safe. The Quantization Agent applies post-training dynamic quantization using PyTorch’s quantize_dynamic. The Evaluation Agent measures top-1 accuracy, average inference latency, parameter count, and memory usage. Finally, the Iterative Pruning Agent closes the loop by loading prior analysis and evaluation logs, querying the LLM for a new pruning configuration, applying it, and updating the best model according to the hierarchical objective.

The architecture is notable for separating profiling, reasoning, execution, and evaluation into distinct agent roles. This separation is explicit in the workflow rather than merely conceptual, and it gives ProfilingAgent the character of a modular optimization stack rather than a single monolithic compressor.

3. Profiling pipeline and LLM-guided analysis

ProfilingAgent’s profiling stage combines static and dynamic measurement. Static metrics are obtained with ptflops, which computes MACs and parameter counts. Dynamic metrics are obtained with PyTorch Profiler on CPU and GPU together with manual per-layer timing hooks, yielding per-layer latency, memory usage, tensor shapes, operator-level breakdowns, number of calls, and snapshots of overall inference latency and peak memory. The profiling experiments were run on the NCSA Delta cluster using AMD EPYC 7763 64-core CPUs, NVIDIA A100-SXM4-40GB GPUs, 251 GB RAM, CUDA 11.8, and PyTorch 2.6.0 (Jafari et al., 6 Sep 2025).

The Input Shape Resolver Agent removes a common source of friction in model profiling. For the model google/vit-base-patch16-224, the system prompt asks for channels, height, width, and sequence_length, and GPT-4o returns the required JSON specification. This automated shape discovery is important because profiling is only meaningful when the model is exercised under the correct preprocessing and input assumptions.

The Analysis Agent is the system’s main reasoning layer. It constructs a prompt that includes layer names and types, MACs and parameter counts per layer or block, latency and memory statistics, overall model performance metrics, and deployment constraints, and then requests structured recommendations. The returned JSON has separate pruning and quantization sections with fields such as layer, pruning_type, pruning_ratio, quantization_type, dtype, and justification. A representative recommendation for a ResNet model uses a regex such as encoder\.stages\d+\.layer\.\d+\.convolution, specifies "structured" pruning at ratio 0.2, and justifies it as reducing Conv2d parameters and MACs to improve latency (Jafari et al., 6 Sep 2025).

This analysis stage is not described as a free-form assistant. It is a constrained planner over structured profiler outputs. The system prompt explicitly binds the LLM to architecture-aware reasoning over measured bottlenecks, which is why ProfilingAgent differs from compression methods that merely ask an LLM to “prune the model” without a profiling substrate.

ProfilingAgent supports two optimization mechanisms. The first is structured pruning, defined as removing entire structural units so that tensor shapes remain dense and compatible with standard kernels. For CNNs such as ResNet-101, this means output-channel or filter pruning in Conv2d layers. For transformer-based models such as ViT-B/16, Swin-B, and DeiT-B/16, it includes pruning rows of Linear layers and pruning attention heads from multi-head self-attention. The Pruning Agent reads regex layer patterns, pruning types, and pruning ratios from the Analysis Agent, constructs a DependencyGraph, and applies dependency-safe structural edits to the baseline model (Jafari et al., 6 Sep 2025).

The second mechanism is post-training dynamic quantization. The Quantization Agent applies PyTorch’s torch.quantization.quantize_dynamic to nn.Linear layers, using either qint8 or float16. In the reported experiments, the LLM consistently recommends full dynamic QInt8 quantization of all linear layers, expressed as a JSON policy such as "layer": "all", "quantization_type": "dynamic", and "dtype": "qint8". Pruning and quantization are described as decoupled but composable: pruning modifies the graph structure, while quantization reduces precision on the original or pruned graph.

The most distinctive mechanism is the Iterative Pruning Agent. Its loop begins with an initial pruned model $M_0$ , evaluates its latency and accuracy, and stores it as the current best model $M_{\text{best}}$ . For each subsequent iteration $i$ , the agent loads prior profiling data $P$ , previous analysis $A_{i-1}$ , and previous evaluation $E_{i-1}$ , queries the LLM for a new pruning configuration $R_i$ , applies that configuration to produce $M_i$ , evaluates $(\mathrm{lat}_i, \mathrm{acc}_i)$ , and updates the best model according to the lexicographic criterion

$\max \mathrm{Acc}(M') \quad \text{then} \quad \min \text{Params}(M') \quad \text{then} \quad \min \mathrm{Latency}(M').$

Comparative experiments with GPT-4o and GPT-4-Turbo highlight that reasoning quality materially changes pruning behavior. With GPT-4o, ProfilingAgent reports $M_{\text{best}}$ 0Acc of $M_{\text{best}}$ 1 on DeiT-B/16, $M_{\text{best}}$ 2 on ResNet-101, $M_{\text{best}}$ 3 on ViT-B/16, and $M_{\text{best}}$ 4 on Swin-B, whereas GPT-4-Turbo produces more aggressive pruning with $M_{\text{best}}$ 5 on DeiT-B/16 and $M_{\text{best}}$ 6 on ResNet-101. The paper interprets this as evidence that iterative pruning quality depends on the LLM’s ability to reason over profiler outputs rather than merely emit pruning ratios (Jafari et al., 6 Sep 2025).

5. Empirical evaluation

ProfilingAgent was evaluated on ImageNet-1K, Imagenette, CIFAR-10, and CIFAR-100 using ResNet-101, ViT-B/16, Swin-B, and DeiT-B/16. The pruning baselines were L1-norm structured pruning, L2-norm structured pruning, and random structured pruning at fixed global ratios of 1%, 10%, and 20%. The quantization baseline was ONNX Runtime dynamic PTQ on MatMul and Gemm nodes. The reported evaluation metrics include top-1 accuracy, parameter count, memory footprint, average inference latency, and relative speedup (Jafari et al., 6 Sep 2025).

On Imagenette pruning, ProfilingAgent reports $M_{\text{best}}$ 7Acc of $M_{\text{best}}$ 8 with 86.6M → 80.9M parameters for DeiT-B/16, $M_{\text{best}}$ 9 with 44.5M → 43.0M for ResNet-101, $i$ 0 with 87.8M → 81.1M for Swin-B, and $i$ 1 with 86.6M → 78.1M for ViT-B/16. By contrast, at 20% global pruning, L1/L2 baselines cause severe drops on ResNet-101, including $i$ 2 accuracy. The paper’s interpretation is that ProfilingAgent achieves comparable or better parameter reduction with much smaller accuracy loss and, in some cases, improved generalization on smaller datasets.

On CIFAR-10 and CIFAR-100 pruning without post-pruning fine-tuning, the reported results remain small in magnitude but consistent across architectures. ResNet-101 moves from 84.10 to 84.20 on CIFAR-10 with 48.81M → 47.30M parameters and from 83.30 to 78.40 on CIFAR-100 with 42.71M → 41.20M. ViT-B/16 moves from 98.50 to 97.90 on CIFAR-10 with 87.12M → 81.45M and from 90.00 to 88.10 on CIFAR-100 with 85.88M → 83.04M. Swin-Base and DeiT-B/16 show similar behavior, typically with modest memory reduction and sub-1% degradation on several settings.

On ImageNet-1K quantization, the most prominent gains come from dynamic quantization rather than pruning. For ViT-B/16, ONNX reports 74.1% memory reduction and 0.3869 → 0.2179 s average inference, whereas ProfilingAgent reports 74.2% memory reduction, 0.2320 → 0.1316 s, a 1.66× speed gain, and $i$ 3Acc of 0%. For Swin-Base, ProfilingAgent reports 74.7% memory reduction, 0.2442 → 0.1556 s, a 1.46× speed gain, and $i$ 4Acc of 0%. For DeiT-B/16, it reports 74.2% memory reduction, 0.2312 → 0.1317 s, a 1.65× speed gain, and $i$ 5Acc of $i$ 6. ResNet-101 shows smaller memory gains, 3.4%, with 0.1211 → 0.1199 s and $i$ 7Acc of 0%.

The separate ImageNet-1K pruning results clarify the system’s trade-off profile. ViT-B/16 goes from 80.3 to 79.5 with 3.3% memory and parameter reduction and 1.02× speedup; Swin-Base goes from 85.3 to 84.7 with 1.9% reduction and 1.02× speedup; DeiT-B/16 goes from 81.0 to 80.2 with 2.0% reduction and 0.98× speedup. ResNet-101 shows 82.4 → 79.7 with 2.9% reduction and 0.97× slowdown. The paper notes that slight slowdowns on ResNet-101 can occur due to channel misalignment effects after structured pruning, which underscores that pruning does not automatically translate into faster kernels on real hardware.

6. Position in the literature and limitations

ProfilingAgent sits at the intersection of model compression, profiling-guided systems optimization, and agentic LLM systems. In adjacent work, profiling-centered agents or profiling layers serve different functions: unified provenance for agentic workflows and downstream reliability analysis in PROV-AGENT (Souza et al., 4 Aug 2025), causal graph tracing for post-hoc root-cause localization in deployed multi-agent systems in AgentTrace (Wang, 16 Mar 2026), self-evolving capability profiles for adaptive routing in FlyRoute (Li et al., 21 May 2026), behavioral trajectory monitoring for computer-use safety in ProjGuard (Contreras et al., 13 May 2026), and dynamic profiling for tabular workflow synthesis in ProfiliTable (Liu et al., 12 May 2026). This suggests that “ProfilingAgent” names not only a specific compression system but also a broader design pattern in which profiling is promoted from passive observability to active control.

Within that landscape, the distinctive contribution of ProfilingAgent is to use profiling traces as the direct substrate for architecture-specific pruning and quantization of vision models. Its contributions are a profiling-guided multi-agent architecture, LLM-driven structured pruning and dynamic quantization, an Iterative Pruning Agent with empirical feedback, and empirical validation across multiple backbones and datasets. The system therefore treats profiling as both measurement and policy input, rather than as a retrospective diagnostic.

The paper also identifies several limitations. Collecting extensive per-layer profiling adds overhead. The system relies on proprietary LLMs, especially GPT-4o, which raises reproducibility and availability concerns. Profiling results are hardware-specific, so strategies learned on A100/H200 GPUs and EPYC/Intel CPUs may not transfer directly without re-profiling. The reported methods are entirely post-training and do not integrate pruning-aware training or quantization-aware training. Pruning and quantization are applied sequentially and evaluated mostly separately, so joint optimization remains limited. A plausible implication is that ProfilingAgent is best understood as a strong proof of concept for profiling-guided agentic optimization rather than a final account of end-to-end hardware–model co-design.

The broader significance of the system lies in its inversion of the usual relationship between profiling and optimization. Rather than hand-designing compression heuristics and then using profilers to validate them, ProfilingAgent uses profilers to generate the heuristics themselves. That formulation is likely to remain relevant even if the specific agent stack, backbone models, or compression operators change.