RL-Guided Pruning for Network Compression

• The paper demonstrates that reinforcement-learning-guided pruning formulates the compression process as sequential decision problems, including finite MDPs, constrained MDPs, and contextual bandits.
• It explores diverse state, action, and reward designs that adjust pruning levels based on layer sensitivity, topology, and resource constraints across multiple architectures.
• Empirical results highlight significant sparsity gains and efficiency improvements while maintaining competitive accuracy in applications ranging from CNNs to transformers.

Reinforcement-learning-guided pruning is a family of compression methods that casts pruning as a sequential decision problem, a constrained decision problem, a Markov Game, or a bandit-style search procedure in which an agent chooses pruning ratios, masks, heads, tokens, or patterns and is evaluated by rewards tied to accuracy, sparsity, FLOPs, parameter count, perplexity, SynOps, or related objectives. In published work, the paradigm appears in structured weight pruning for CNNs, attention-head pruning for backdoor defense in LLMs, token pruning in Vision Transformers, structured pruning in spiking neural networks, graph-based auto-pruning, joint pruning and quantization, and even decision-tree pruning (Gupta et al., 2020, Chapagain et al., 27 Aug 2025, Lu et al., 30 Mar 2025, Shanto et al., 8 Aug 2025). The central premise is that pruning allocation is rarely uniform across layers or components, and that a learned decision process can exploit sensitivity differences, topology, and resource constraints more effectively than fixed heuristics (Wang et al., 2024, Baghdadi et al., 24 Jun 2026).

1. Formal problem classes

RL-guided pruning is not tied to a single mathematical formalization. PuRL models structured weight pruning as a finite Markov Decision Process $(S, A, R, T, \gamma)$, with layer-wise state, discrete threshold actions, and dense immediate rewards after each pruning step (Gupta et al., 2020). Constrained RL pruning models the layer-wise pruning decision as a finite-horizon Constrained MDP $$\mathcal{M}=(\mathcal{S},\mathcal{A},p,r,c,\gamma,T)$$ with a reward objective and an explicit cost budget on arbitrary, possibly non-differentiable, functions (Malik et al., 2021). In attention-head pruning for backdoor defense in LLMs, the state is the current pruning mask

$P_t = \{P_t^{(l)} : \text{for all layers } l\},$

the action space is the set of candidate heads not yet pruned,

$A_t = \{(l,h)\mid h\notin P_t^{(l)}\},$

and pruning terminates when validation accuracy falls below the pre-set accuracy threshold $\tau=0.85$ (Chapagain et al., 27 Aug 2025).

Other formulations depart further from the classical single-agent MDP. RL-based token pruning in Vision Transformers models pruning as a Markov Game in which each token is controlled by an agent that chooses keep or prune decisions across layers (Lu et al., 30 Mar 2025). FastForward Pruning reduces layer-wise sparsity allocation in LLMs to a single-step RL problem and explicitly states that the problem reduces to a contextual bandit (Yuan et al., 24 Nov 2025). Decision-tree pruning has also been framed as a single-state bandit problem, where each arm corresponds to a candidate internal branch-node of the trained tree (Shanto et al., 8 Aug 2025).

These formulations show that the phrase “reinforcement-learning-guided pruning” covers a broad design space rather than a single canonical algorithm (Gupta et al., 2020, Chapagain et al., 27 Aug 2025, Lu et al., 30 Mar 2025, Yuan et al., 24 Nov 2025).

Formulation	Decision unit	Representative use
Finite MDP	Layer or weight threshold	Structured network pruning
Constrained MDP	Layer-wise sparsity under budget	Budgeted CNN pruning
Markov Game	Token-wise keep/prune	ViT token pruning
Single-step contextual bandit	Global layer allocation	LLM sparsity allocation
Single-state bandit	Branch-node pruning	Decision-tree pruning

2. State, action, and reward design

State design ranges from compact hand-crafted descriptors to learned graph and environment representations. PuRL experiments with a low-dimensional 3-D state $s^t=\langle l^t,A^t,P^t\rangle$ and a high-dimensional $2L$-D state $\langle A^1,P^1,\dots,A^L,P^L\rangle$ (Gupta et al., 2020). Constrained RL pruning uses the descriptor

$$s_t=(t,\text{in\_channels},\text{out\_filters},\text{kernel\_size},\text{stride},\text{padding})$$

for each convolutional layer (Malik et al., 2021). RL-guided pruning with learnable agent guidance and alignment augments a raw layer state with a learnable epoch embedding and a GRU-based environment code $z_t$ to represent the changing reward dynamics caused by evolving CNN weights (Ganjdanesh et al., 2024). GraphPruning uses GCN-aggregated node features as RL states rather than hand-craft features (Zhang et al., 2019), while AutoSculpt rebuilds the current network graph and encodes it with a GATv2-based graph embedding before action selection (Jing et al., 2024).

Action spaces are equally heterogeneous. PuRL chooses a scalar $$\mathcal{M}=(\mathcal{S},\mathcal{A},p,r,c,\gamma,T)$$0 that determines which weights in the current layer are pruned by the threshold $$\mathcal{M}=(\mathcal{S},\mathcal{A},p,r,c,\gamma,T)$$1 (Gupta et al., 2020). Many structured pruning methods instead output a continuous pruning ratio $$\mathcal{M}=(\mathcal{S},\mathcal{A},p,r,c,\gamma,T)$$2 for each layer or block (Ganjdanesh et al., 2024, Musat et al., 2023, Xie et al., 28 Jun 2025). RL4EViT assigns a binary action $$\mathcal{M}=(\mathcal{S},\mathcal{A},p,r,c,\gamma,T)$$3 to every token agent at each pruning layer (Lu et al., 30 Mar 2025). AutoSculpt maps graph embeddings to pattern scores, then samples pruning patterns for operators from a predefined pattern library (Jing et al., 2024). In LLM backdoor defense, the action is the permanent removal of one attention head, selected by an $$\mathcal{M}=(\mathcal{S},\mathcal{A},p,r,c,\gamma,T)$$4-greedy rule over pre-computed head-importance scores:

$$\mathcal{M}=(\mathcal{S},\mathcal{A},p,r,c,\gamma,T)$$5

(Chapagain et al., 27 Aug 2025)

Reward design is the point of greatest variation. PuRL uses dense immediate rewards shaped jointly by accuracy and sparsity targets,

$$\mathcal{M}=(\mathcal{S},\mathcal{A},p,r,c,\gamma,T)$$6

with $$\mathcal{M}=(\mathcal{S},\mathcal{A},p,r,c,\gamma,T)$$7 in experiments (Gupta et al., 2020). Constrained RL pruning places the reward at the terminal step as the negative fine-tuning loss on a held-out batch and the cost as a terminal complexity measurement such as parameters, FLOPs, or wall-clock inference time (Malik et al., 2021). The entropy-minimization method replaces accuracy reward with the mean spatial entropy of convolutional activations and sets the final reward to $$\mathcal{M}=(\mathcal{S},\mathcal{A},p,r,c,\gamma,T)$$8 (Musat et al., 2023). SPEAR defines a target-aware reward that adds an accuracy term to a SynOps penalty driven by the LRE estimator (Xie et al., 28 Jun 2025). FastForward uses a scale-invariant relative perplexity gain,

$$\mathcal{M}=(\mathcal{S},\mathcal{A},p,r,c,\gamma,T)$$9

after mapping raw policy outputs to a budget-feasible retention policy (Yuan et al., 24 Nov 2025).

A common implication is that RL-guided pruning is less defined by a particular reward function than by the use of feedback-driven policy search over pruning decisions.

3. Policy classes and optimization algorithms

The algorithmic spectrum includes value-based, actor-critic, constrained-policy, multi-agent, and non-parametric controllers. PuRL uses a value-based DQN with an experience replay buffer, a separate target network, and no actor-critic baseline (Gupta et al., 2020). GraphPruning and the entropy-minimization filter-pruning method use Deep Deterministic Policy Gradient, with continuous actions and replay-based Bellman updates (Zhang et al., 2019, Musat et al., 2023). SPEAR also uses DDPG because the pruning ratio is continuous (Xie et al., 28 Jun 2025). The constrained budget paper adapts Proximal Policy Optimization with a Lagrangian term and two critics, one for reward and one for cost (Malik et al., 2021).

Some methods explicitly address richer coordination structures. RL4EViT uses Multi-Agent Proximal Policy Optimization, with a shared actor and critic across token agents, and combines an “Efficiency” reward with an “Accuracy” reward so that agents simultaneously collaborate and compete (Lu et al., 30 Mar 2025). HiReLC introduces two tiers of agents: Low-Level Agents operating per block and a High-Level Agent allocating global budgets, with ensembles of PPO and A2C policies and Fisher Information-based sensitivity estimates (Baghdadi et al., 24 Jun 2026). The jointly training-and-pruning framework uses SAC for pruning-policy updates, while a GRU-based environment model and a decoder reconstruct observed rewards to represent non-stationarity (Ganjdanesh et al., 2024).

Not all methods labeled RL-guided pruning train a neural policy at all. The LLM backdoor-defense method explicitly states that no neural policy or value network is trained; the “agent” uses a simple $P_t = \{P_t^{(l)} : \text{for all layers } l\},$0-greedy selection over a scalar importance score pre-computed for every head (Chapagain et al., 27 Aug 2025). Decision-tree pruning with multi-armed bandits similarly treats pruning as an exploration-exploitation problem over arm pulls rather than a deep actor-critic optimization problem (Shanto et al., 8 Aug 2025).

This suggests that the term “RL-guided” is often used operationally: the pruning process is organized around sequential decisions, exploration-exploitation, and reward feedback, but the controller may range from deep PPO to a score-guided $P_t = \{P_t^{(l)} : \text{for all layers } l\},$1-greedy heuristic.

4. Workflow patterns, budget handling, and evaluation acceleration

Most RL-guided pruning pipelines alternate action selection with model modification and some form of recovery or evaluation. In PuRL, each episode begins by loading the original pre-trained model, then for every layer the agent observes the state, selects $P_t = \{P_t^{(l)} : \text{for all layers } l\},$2, prunes the layer, retrains the pruned model for one epoch on a small subset of 1,000 images, computes accuracy and cumulative sparsity, stores the transition, and performs a DQN update (Gupta et al., 2020). In the LLM attention-head method, the loop is simpler: form the candidate set, choose one head by $P_t = \{P_t^{(l)} : \text{for all layers } l\},$3-greedy over importance scores, permanently prune that head, fine-tune or at least re-evaluate on the clean validation set, and terminate if accuracy drops below $P_t = \{P_t^{(l)} : \text{for all layers } l\},$4 (Chapagain et al., 27 Aug 2025).

A major practical issue is budget satisfaction. Constrained RL pruning places the budget directly into the optimization objective through the Lagrangian saddle-point formulation (Malik et al., 2021). RLAL enforces the global FLOPs budget through feasible action bounds $P_t = \{P_t^{(l)} : \text{for all layers } l\},$5 rather than through an explicit FLOPs penalty in the reward (Ganjdanesh et al., 2024). FastForward decouples policy optimization from budget satisfaction: the policy emits unconstrained raw scores, and a deterministic Budget-Constrained Policy Mapping clips, ranks, greedily fills the budget, discretizes, and corrects residual error to exactly meet the target (Yuan et al., 24 Nov 2025). SPEAR does not impose SynOps as a hard action constraint; it predicts post-fine-tuning SynOps via a linear regressor and penalizes budget violations through the Target-Aware Reward (Xie et al., 28 Jun 2025).

A second recurring issue is the cost of evaluating candidate pruned models. Several systems introduce accelerators around the RL loop rather than inside the policy itself. RLAL alternates one epoch of CNN weight training with $P_t = \{P_t^{(l)} : \text{for all layers } l\},$6 RL episodes over the frozen model and uses a decoder trained with reconstruction loss to model changing reward dynamics (Ganjdanesh et al., 2024). HiReLC interleaves surrogate-guided RL optimization with post-compression fine-tuning, using a lightweight MLP surrogate and a logit-MSE proxy during cold-start; the surrogate is used for reward shaping rather than as a replacement for final post-compression evaluation (Baghdadi et al., 24 Jun 2026). FastForward adds a curriculum that simultaneously ramps sparsity difficulty and evaluation fidelity, beginning with low-cost, simple tasks and gradually increasing in complexity (Yuan et al., 24 Nov 2025). GraphPruning goes further upstream by training a PruningNet that generates reasonable weights for any size of the sub-network, allowing search with no iterative fine-tuning during search (Zhang et al., 2019).

The practical pattern is therefore hybrid rather than purely RL-centric: pruning decisions are learned, but the overall system often depends on retraining, surrogate models, budget mappings, or prediction modules.

5. Empirical behavior across architectures and tasks

Published results show that RL-guided pruning is not confined to a single architecture family. On ImageNet, PuRL reports 80.27% sparsity on ResNet-50 with final Top-1 accuracy of 75.37% after full fine-tuning, converging in approximately 55 episodes, compared to 400 episodes used by AMC (Gupta et al., 2020). Constrained RL pruning reports that, on ResNet18 with $P_t = \{P_t^{(l)} : \text{for all layers } l\},$7, it achieves 2.91 M parameters with 92.09% accuracy, and on ResNet50 with $P_t = \{P_t^{(l)} : \text{for all layers } l\},$8 it gets 12.34 M parameters and 93.60% accuracy (Malik et al., 2021). The jointly training-and-pruning framework reports, at a 50% FLOPs budget on CIFAR-10, ResNet-56 improving from 93.41% to 93.86% and MobileNet-V2 from 94.48% to 94.85% (Ganjdanesh et al., 2024).

For backdoor defense in LLMs, the evidence is task-dependent. Against HiddenKiller, RL-based pruning achieves $P_t = \{P_t^{(l)} : \text{for all layers } l\},$9 and $A_t = \{(l,h)\mid h\notin P_t^{(l)}\},$0, and is outperformed on this attack by gradient-based pruning. Against StyleBkd, RL-based pruning obtains the best trade-off of all methods with $A_t = \{(l,h)\mid h\notin P_t^{(l)}\},$1 and $A_t = \{(l,h)\mid h\notin P_t^{(l)}\},$2, slightly surpassing Bayesian pruning and the variance-only method PURE (Chapagain et al., 27 Aug 2025). The abstract of that study summarizes the result more generally: gradient-based pruning performs best while defending syntactic triggers, whereas reinforcement learning and Bayesian pruning better withstand stylistic attacks (Chapagain et al., 27 Aug 2025).

For Vision Transformers, RL4EViT reports up to 44% inference acceleration on ImageNet-1k with under 0.4% top-1 accuracy loss, and for DeiT-B the setting with $A_t = \{(l,h)\mid h\notin P_t^{(l)}\},$3 reaches 11.6 GFLOPs, 81.4% Top-1 accuracy, and 589.8 images/s versus 17.0 GFLOPs, 81.8%, and 408.8 images/s without pruning (Lu et al., 30 Mar 2025). AutoSculpt reports up to 90% pruning rates and nearly 18% improvement in FLOPs reduction across ResNet, MobileNet, VGG, and Vision Transformer, and on ImageNet-1K reports, for example, ResNet-50 with 65% FLOPs reduction and a 74.03% pruned accuracy, and ViT-B/16 with 45% FLOPs reduction and 79.22% pruned accuracy (Jing et al., 2024).

For SNNs, SPEAR reports that under a 50% SynOps target on CIFAR-10/VGG16 it achieves 52.5% SynOps, 14.4% parameters, and 91.77% top-1 accuracy, outperforming SCA-based and NetworkSliming baselines under the reported settings (Xie et al., 28 Jun 2025). For joint pruning and quantization, HiReLC reports parameter-storage compression ratios of 5.99–6.72$A_t = \{(l,h)\mid h\notin P_t^{(l)}\},$4, with a 3.83% gain in one setting and 0.55–5.62% accuracy drops elsewhere (Baghdadi et al., 24 Jun 2026). FastForward reports that on LLaMA 7B at 20% sparsity, search takes 6.13 GPU-hr and calibration 0.97 GPU-hr, for 7.10 GPU-hr total, and that on LLaMA-V1 7B at 20% sparsity it reaches perplexity 6.64 versus FLAP’s 6.90 (Yuan et al., 24 Nov 2025).

Setting	Reported outcome	Citation
ResNet-50 on ImageNet	80.27% sparsity, final Top-1 75.37%	(Gupta et al., 2020)
ResNet18 on CIFAR-10, $A_t = \{(l,h)\mid h\notin P_t^{(l)}\},$5	2.91 M params, 92.09% acc	(Malik et al., 2021)
HiddenKiller backdoor defense	ACC $A_t = \{(l,h)\mid h\notin P_t^{(l)}\},$6, LFR $A_t = \{(l,h)\mid h\notin P_t^{(l)}\},$7	(Chapagain et al., 27 Aug 2025)
StyleBkd backdoor defense	ACC $A_t = \{(l,h)\mid h\notin P_t^{(l)}\},$8, LFR $A_t = \{(l,h)\mid h\notin P_t^{(l)}\},$9	(Chapagain et al., 27 Aug 2025)
DeiT-B on ImageNet-1k	Up to 44% speedup with 0.4% top-1 drop	(Lu et al., 30 Mar 2025)
CIFAR-10/VGG16 SNN, 50% SynOps target	52.5% SynOps, 14.4% params, 91.77% acc	(Xie et al., 28 Jun 2025)

6. Misconceptions, limitations, and current directions

A common misconception is that RL-guided pruning always means training a policy network with gradient-based RL. The LLM backdoor-defense method directly contradicts this: it formulates pruning as an MDP, but trains no neural policy or value network and reports no learning rate, discount factor, or gradient-based RL update (Chapagain et al., 27 Aug 2025). A second misconception is that the reward must be validation accuracy. The entropy-minimization method uses spatial entropy of activations as the reward proxy and reports that it matches or nearly matches the accuracy-driven agent across VGG-16, MobileNetV2, and ResNet50 on CIFAR-10 (Musat et al., 2023). Decision-tree pruning uses a $\tau=0.85$0Score built from Accuracy, Loss, and $\tau=0.85$1, while SPEAR combines accuracy with a budget-violation penalty based on predicted SynOps (Shanto et al., 8 Aug 2025, Xie et al., 28 Jun 2025).

Computational cost remains a central limitation. RL-Pruner states that building the dependency graph for very deep or multi-branch networks takes a few minutes and that the full RL-sampling and pruning loop can take several hours on a single GPU (Wang et al., 2024). SPEAR reports convergence in less than or equal to 2 hours on a single GPU, while GraphPruning reports that replacing hand-crafted layer features with GCN embeddings cuts the number of episodes to convergence by approximately 30–40% on the reported CIFAR-10 replication of AMC’s setup (Xie et al., 28 Jun 2025, Zhang et al., 2019). PuRL emphasizes that dense rewards are critical and reports 85% fewer RL episodes than AMC on ResNet-50 (Gupta et al., 2020). FastForward’s core claim is that other search-based approaches are often hindered by prohibitive computational costs on large-scale models, and its curriculum and single-step formulation are explicitly introduced to reduce computational overhead (Yuan et al., 24 Nov 2025).

Non-stationarity and proxy quality are additional limitations. The jointly training-and-pruning method states that the evolving model weights result in a dynamic reward function that prevents using prominent episodic RL methods with stationary environment assumption, which motivates its epoch embeddings and recurrent environment model (Ganjdanesh et al., 2024). SPEAR notes that linear LRE may break down if network dynamics become highly non-linear post-pruning, and that TAR hyperparameters require moderate tuning (Xie et al., 28 Jun 2025). RL4EViT notes that extension beyond image classification to detection and segmentation is left for future work, and FastForward lists richer state signals, other pruning criteria, unstructured or block sparsity, and hardware latency optimization as potential extensions (Lu et al., 30 Mar 2025, Yuan et al., 24 Nov 2025).

This suggests that reinforcement-learning-guided pruning is best understood as a methodology for search and control under compression constraints, not as a fixed algorithmic recipe. A plausible implication is that future systems will continue to combine RL with graph encoders, surrogates, constraint mappings, learned environment representations, and budget predictors rather than rely on naive repeated evaluation of fully fine-tuned subnetworks.