---
title: Progressive Channel Pruning in Deep Networks
url: https://www.emergentmind.com/topics/progressive-channel-pruning-pcp
type: topic
---

# Progressive Channel Pruning in Deep Networks

Searching arXiv for the cited PCP literature to ground the article and verify metadata.
to=arxiv_search.search  大发快三是什么query":"all:\"Progressive Channel Pruning\" OR all:\"Progressive Channel-Shrinking Network\" OR id:2304.00280 OR id:2507.04792 OR id:2402.17862 OR id:1904.03508","max_results":10}平台总代理 to=arxiv_search.search code  大发快三官网  天天中彩票怎么买result 《凤凰大参考json
[{"arxiv_id":"2507.04792","title":"Model Compression using Progressive Channel Pruning","authors":["Y. Zhang","S. Liao","W. Chen","Q. Yang"],"abstract":"In this work, we propose a simple but effective channel pruning framework called Progressive Channel Pruning (PCP) to accelerate Convolutional Neural Networks (CNNs). In contrast to the existing channel pruning methods that prune channels only once per layer in a layer-by-layer fashion, our new progressive framework iteratively prunes a small number of channels from several selected layers, which consists of a three-step attempting-selecting-pruning pipeline in each iteration. In the attempting step, we attempt to prune a pre-defined number of channels from one layer by using any existing channel pruning methods and estimate the accuracy drop for this layer based on the labelled samples in the validation set. In the selecting step, based on the estimated accuracy drops for all layers, we propose a greedy strategy to automatically select a set of layers that will lead to less overall accuracy drop after pruning these layers. In the pruning step, we prune a small number of channels from these selected layers. We further extend our PCP framework to prune channels for the deep transfer learning methods like Domain Adversarial Neural Network (DANN), in which we effectively reduce the data distribution mismatch in the channel pruning process by using both labelled samples from the source domain and pseudo-labelled samples from the target domain. Our comprehensive experiments on two benchmark datasets demonstrate that our PCP framework outperforms the existing channel pruning approaches under both supervised learning and transfer learning settings.","categories":["cs.CV","cs.LG"],"published":"2025-07-07","url":"http://arxiv.org/abs/2507.04792v1"},{"arxiv_id":"2304.00280","title":"Progressive Channel-Shrinking Network","authors":["Y. Xiao","Q. Wang","Z. Chen","M. Zhang","J. Zhang"],"abstract":"Currently, salience-based channel pruning makes continuous breakthroughs in network compression. In the realization, the salience mechanism is used as a metric of channel salience to guide pruning. Therefore, salience-based channel pruning can dynamically adjust the channel width at run-time, which provides a flexible pruning scheme. However, there are two problems emerging: a gating function is often needed to truncate the specific salience entries to zero, which destabilizes the forward propagation; dynamic architecture brings more cost for indexing in inference which bottlenecks the inference speed. In this paper, we propose a Progressive Channel-Shrinking (PCS) method to compress the selected salience entries at run-time instead of roughly approximating them to zero. We also propose a Running Shrinking Policy to provide a testing-static pruning scheme that can reduce the memory access cost for filter indexing. We evaluate our method on ImageNet and CIFAR10 datasets over two prevalent networks: ResNet and VGG, and demonstrate that our PCS outperforms all baselines and achieves state-of-the-art in terms of compression-performance tradeoff. Moreover, we observe a significant and practical acceleration of inference.","categories":["cs.CV"],"published":"2023-04-01","url":"http://arxiv.org/abs/2304.00280v1"},{"arxiv_id":"2402.17862","title":"REPrune: Channel Pruning via Kernel Representative Selection","authors":["J. Lee","S. Park","Y. Choi","D. Kim"],"abstract":"Channel pruning is widely accepted to accelerate modern convolutional neural networks (CNNs). The resulting pruned model benefits from its immediate deployment on general-purpose software and hardware resources. However, its large pruning granularity, specifically at the unit of a convolution filter, often leads to undesirable accuracy drops due to the inflexibility of deciding how and where to introduce sparsity to the CNNs. In this paper, we propose REPrune, a novel channel pruning technique that emulates kernel pruning, fully exploiting the finer but structured granularity. REPrune identifies similar kernels within each channel using agglomerative clustering. Then, it selects filters that maximize the incorporation of kernel representatives while optimizing the maximum cluster coverage problem. By integrating with a simultaneous training-pruning paradigm, REPrune promotes efficient, progressive pruning throughout training CNNs, avoiding the conventional train-prune-finetune sequence. Experimental results highlight that REPrune performs better in computer vision tasks than existing methods, effectively achieving a balance between acceleration ratio and performance retention.","categories":["cs.CV"],"published":"2024-02-27","url":"http://arxiv.org/abs/2402.17862v1"},{"arxiv_id":"1904.03508","title":"C2S2: Cost-aware Channel Sparse Selection for Progressive Network Pruning","authors":["Y.-C. Lin","Y.-Y. Hsu","T.-Y. Chiu","Y.-C. F. Wang"],"abstract":"This paper describes a channel-selection approach for simplifying deep neural networks. Specifically, we propose a new type of generic network layer, called pruning layer, to seamlessly augment a given pre-trained model for compression. Each pruning layer, comprising $1 \\times 1$ depth-wise kernels, is represented with a dual format: one is real-valued and the other is binary. The former enables a two-phase optimization process of network pruning to operate with an end-to-end differentiable network, and the latter yields the mask information for channel selection. Our method progressively performs the pruning task layer-wise, and achieves channel selection according to a sparsity criterion to favor pruning more channels. We also develop a cost-aware mechanism to prevent the compression from sacrificing the expected network performance. Our results for compressing several benchmark deep networks on image classification and semantic segmentation are comparable to those by state-of-the-art.","categories":["cs.CV"],"published":"2019-04-06","url":"http://arxiv.org/abs/1904.03508v1"}]
Progressive Channel Pruning (PCP) denotes a family of structured network-compression methods that reduce computation and memory by removing channels progressively rather than in a single post-hoc step. In the broad sense, PCP aims to reduce metrics such as FLOPs or MAdds, parameters, and latency while minimizing accuracy loss by distributing pruning decisions across training steps, epochs, or iterative pruning rounds. The term also names a specific framework, "Model Compression using Progressive Channel Pruning," in which a CNN is compressed through an iterative Attempt–Select–Prune loop guided by validation accuracy estimates [2507.04792]. Across the literature, the progressive aspect appears in several forms: layer-wise iterative pruning, differentiable salience shrinking, and training-integrated schedules that interleave pruning with representation learning [2304.00280] [2402.17862] [1904.03508].

## 1. Definition and conceptual scope

PCP is best understood as a structured pruning regime in which channel removal is staged over time. In one formulation, for a convolutional layer with output feature maps $x$, a nonnegative salience vector $s$ reweights channels as $x'_c = s_c x_c$; if $s_c = 0$, the $c$-th channel is deactivated or pruned [2304.00280]. In another formulation, each layer $l$ has a channel-selection vector $\beta^{(l)} \in \mathbb{R}^{C_l}$, with $\beta_i = 0$ indicating that the corresponding input channel is pruned, and the global objective is to minimize classification loss subject to a target compression ratio [2507.04792].

A recurrent contrast in this literature is between progressive pruning and one-shot or layer-by-layer pruning. The 2025 PCP framework explicitly positions itself against methods that prune each layer once in a fixed order, arguing instead for iteratively pruning a small number of channels from several selected layers per iteration [2507.04792]. C2S2 similarly defines “progressive” as layer-wise pruning carried out over multiple iterations after augmenting a pre-trained network with pruning layers [1904.03508]. REPrune uses the term for pruning decisions made during training at scheduled epochs, rather than in a conventional train–prune–finetune pipeline [2402.17862]. This suggests that PCP is not a single algorithmic template but a broader design principle: prune incrementally, preserve structured sparsity, and let later pruning decisions reflect earlier architectural changes.

## 2. Core optimization principles

A common PCP objective is to minimize task loss while enforcing structured sparsity on channels. The 2025 framework formalizes this as
$$
\arg\min_{\tilde{\Theta},\, \mathcal{B}} \; \mathcal{L}_c\big(\mathcal{N}(\mathcal{I}; \tilde{\Theta}, \mathcal{B}), y\big)
\quad \text{s.t.} \quad
R(\mathcal{B}) \ge R_{\text{target}},
$$
where $\mathcal{B} = \{\beta^{(1)}, \dots, \beta^{(L)}\}$ denotes per-layer channel selections and $R(\cdot)$ is a compression ratio function based on FLOPs or parameter reduction [2507.04792].

When PCP is instantiated with the CP solver, the per-layer pruning subproblem is posed as reconstruction with sparsity:
$$
\min_{W,\,\beta} \; \mathcal{L}_R(Y, X, W, \beta)
\quad \text{s.t.} \quad \|\beta\|_0 \le c',
$$
with
$$
\mathcal{L}_R(Y, X, W, \beta) =
\left\| Y - \sum_{i=1}^{c} \beta_i X_i W_i^\top \right\|_F^2.
$$
This is solved by alternating LASSO-based channel selection and least-squares weight update, and those same two subproblems are reused both in the “attempting” stage and in the actual pruning stage [2507.04792].

Other PCP variants preserve the same high-level objective but change the relaxation or control mechanism. PCS replaces hard channel truncation with a shrinking loss over the $K$ smallest salience entries,
$$
\mathcal{R} = \sum_{i \le K} s'_i,
$$
and optimizes
$$
\arg\min_{\theta,\pi} \; \mathbb{E}_{x^{(0)}}\big[\mathcal{L}(\hat{y}, y) + \lambda \mathcal{R}\big],
$$
where $s'$ is the salience vector sorted in increasing order [2304.00280]. C2S2 instead inserts a pruning layer after each convolution and learns real-valued pruning weights $P$ via
$$
L_{\text{all}} = L(Y, Y_1) + \lambda_1 \|P\|_1 + \lambda_2 \|P \odot (1-P)\|_1,
$$
then binarizes with a unified threshold $\tau = 0.5$ to obtain a mask $B$ [1904.03508]. REPrune keeps the task loss unchanged, $L_{\text{total}} = L_{\text{task}}$, and introduces pruning through periodic mask updates driven by BatchNorm scaling factors and representative-based filter selection [2402.17862].

## 3. Major algorithmic realizations

Several distinct methods now fall under the PCP umbrella.

| Method | Progressive mechanism | Distinctive feature |
|---|---|---|
| PCP framework | Iterative Attempt–Select–Prune loop | Validation-driven greedy layer selection |
| PCS | Progressive shrinking of salience entries | Testing-static pruning via running salience |
| C2S2 | Layer-wise pruning with pruning layers | Cost-aware Pruning/Restoring controller |
| REPrune | Scheduled pruning during training | Kernel representative selection via clustering |

The PCP framework of "Model Compression using Progressive Channel Pruning" iteratively prunes a small number of channels from several selected layers. Each iteration has three steps. In the attempting step, the method temporarily prunes $f_t$ channels from each layer and measures the corresponding validation accuracy drop $\Delta A_l(f_t) = A_{\text{base}, t-1} - acc_t(l)$. In the selecting step, it greedily chooses a set $S_t$ of size $\text{top\_n}$ with the smallest estimated drops. In the pruning step, it performs actual pruning for those selected layers in shallow-to-deep order, updates the model to $M_t$, and stops when $\text{FLOPs}(M_0)/\text{FLOPs}(M_t) \ge R_{\text{target}}$ [2507.04792].

PCS, introduced in "Progressive Channel-Shrinking Network," starts from salience-based channel pruning and targets two specific limitations: instability caused by abrupt truncation of continuous salience values, and indexing overhead induced by input-dependent dynamic architectures. Instead of thresholding salience entries to zero, PCS progressively compresses only the smallest $K$ entries by the shrinking loss $\mathcal{R} = \sum_{i \le K} s'_i$. Its salience generator has the architecture GAP $\rightarrow$ FC $\rightarrow$ ReLU $\rightarrow$ FC $\rightarrow$ hard sigmoid, yielding $s \in [0,1]$. A Running Shrinking Policy maintains an exponential moving average $\overline{s} = (1-\alpha)\overline{s} + \alpha s$ and uses $\overline{s}$ to stabilize which entries are shrunk. At deployment, the Boolean mask is fixed as $m = \mathbb{1}_{\mathbb{R}_{\ne 0}}(\overline{s})$, and weights and biases are permanently pruned, producing a static architecture [2304.00280].

C2S2 operationalizes progressive channel pruning by inserting a depth-wise $1 \times 1$ pruning layer after each convolution. The pruning layer has a dual representation: real-valued weights $P^\ell$ and a binary mask $B^\ell \in \{0,1\}^{n_\ell}$ obtained by thresholding at $\tau = 0.5$. Phase 1 optimizes $P^\ell$ with sparsity and bipolar regularization; Phase 2 fixes $P^\ell$, applies $B^\ell$, and updates only the original network weights. A cost-aware controller monitors the exponential moving average training error $e_{\text{ema}}$ relative to a baseline error $e_{\text{base}}$, stays in a Pruning state while $e_{\text{ema}} \le c_p e_{\text{base}}$, switches to Restoring when this bound is exceeded, and terminates a layer when $e_{\text{ema}} < c_r e_{\text{base}}$ [1904.03508].

REPrune embeds pruning into training and bases channel selection on kernel representative selection. For each input channel, it clusters kernels across output filters by agglomerative clustering with Ward’s linkage, then selects filters by a greedy maximum cluster coverage procedure. Layer-wise sparsity is derived from a global threshold on BatchNorm scales, with
$$
s^l = \frac{1}{|I|} \sum_{i \in I} \mathbb{1}[\gamma_i^l \le \gamma^*].
$$
The number of reserved filters is $B = \lceil (1-s^l)n_{l+1} \rceil$, and the greedy step picks
$$
i^* = \arg\max_{i \in I} \sum_{j \in J} S(\kappa^l_{i,j}),
$$
where $S(\kappa^l_{i,j}) \in \{0,1\}$ is a coverage score that is updated as clusters become represented [2402.17862].

## 4. Dynamic versus static pruning and systems consequences

A central systems issue in PCP is the distinction between dynamic, input-dependent channel activation and static deployment masks. PCS makes this issue explicit. It notes that prior salience-based methods often need a truncation rule such as
$$
z =
\begin{cases}
z, & z \ge \eta,\\
0, & z < \eta,
\end{cases}
$$
even though salience generators are continuous and rarely output exact zeros. The reported consequence is unstable forward and backward propagation, because channels can be abruptly deactivated at different times for different inputs [2304.00280].

PCS further argues that dynamic pruning can reduce FLOPs while failing to reduce memory access cost because full kernels must remain indexable online. In that paper, memory access cost for a convolution is defined as
$$
\text{MAC} = C_{\text{in}} C_{\text{out}} K^2 + C_{\text{out}} H_{\text{out}} W_{\text{out}}.
$$
By fixing the test-time mask after training and permanently removing zeroed channels, PCS aims to reduce both MAdds and MAC, and thereby practical latency [2304.00280]. The same general concern appears in the broader PCP literature as a distinction between nominal compression ratios and realized deployment efficiency. This suggests that, within PCP, structural density after pruning is as important as sparsity itself when the target is general-purpose hardware.

REPrune is aligned with that systems perspective. It is explicitly designed to emulate finer-grained kernel pruning while still outputting a dense channel-pruned model that remains immediately deployable on general-purpose software and hardware resources [2402.17862]. C2S2 also produces dense subnetworks by unplugging pruning layers and physically removing masked channels once a layer has been finalized [1904.03508]. The 2025 PCP framework likewise performs structured channel removal and a single final fine-tuning stage rather than retaining dynamic gating at inference [2507.04792].

## 5. Empirical behavior across benchmarks

Empirical results in the PCP literature are reported on ImageNet, CIFAR-10, Office-31, COCO, and semantic segmentation benchmarks, with architectures including VGG, AlexNet, ResNet, MobileNetV2, SSD300, Mask R-CNN, and preliminary transformer experiments [2304.00280] [2402.17862] [2507.04792] [1904.03508].

For the 2025 PCP framework on ImageNet VGG-16, the original Top-5 accuracy is 89.9%. PCP reports 90.0% at $2\times$, 89.3% at $4\times$, and 89.0% at $5\times$ compression, compared with CP at 89.9%, 88.9%, and 88.2%, and AMC at 88.5% for $5\times$. On ImageNet ResNet-50, PCP reports FLOPs 44.5%, Params 59.2%, Top-1 73.4%, and Top-5 91.5%, compared with ThiNet-50 at FLOPs 44.2%, Params 48.4%, Top-1 71.0%, and Top-5 90.0% [2507.04792]. The same paper extends PCP to DANN on Office-31 by using labeled source samples and pseudo-labeled target samples during pruning; the reported PCP w/ PL results are consistently better than CP across multiple domain-transfer tasks and compression ratios [2507.04792].

PCS reports detailed latency and MAC comparisons. On ImageNet with ResNet-18, the baseline is Top-1 29.6%, Params 12M, MAdds 1.8G, MAC 14.5M, GPU 18.9ms, and ARM 43.2ms. PCS-ResNet18-B reports Top-1 29.8%, Params 4M, MAdds 1.0G, MAC 6.9M, 1080Ti 13.8ms, and ARM 26.6ms. At roughly similar MAdds, dynamic baselines such as FBS, DGC, and ManiDP retain MAC around 15.1M and exhibit worse latency [2304.00280]. On VGG-16/ImageNet, PCS-VGG16-B reports Top-1 28.5%, Params 45M, MAdds 2.8G, MAC 51.1M, 1080Ti 30.9ms, and ARM 57.0ms, compared with the baseline Top-1 28.4%, Params 138M, MAdds 15.5G, MAC 155M, 1080Ti 83.4ms, and ARM 190.4ms [2304.00280]. That paper also reports preliminary transformer results: T2T-ViT from Top-1 err 18.5%, 13.8G, 1.62ms to 18.5%, 3.9G, 1.18ms, and Swin from 18.7%, 4.5G, 1.32ms to 18.7%, 2.5G, 0.89ms [2304.00280].

REPrune reports strong results under a training-integrated progressive regime. On ImageNet, ResNet-18 moves from a 69.4% baseline at 1.81G FLOPs to 69.2% at 1.03G. ResNet-34 reports 74.3% at 2.7G and 73.9% at 2.0G from a 73.3% baseline at 3.6G. ResNet-50 reports 77.3% at 2.3G, 77.0% at 1.8G, and 75.7% at 1.0G from a 76.2% baseline at 4.1G [2402.17862]. On COCO-2017 detection with SSD300 and a pruned ResNet-50 backbone, a 50% FLOPs reduction yields mAP 25.0, AP50 42.3, and AP75 25.9 [2402.17862].

C2S2 reports that, on CIFAR-10 VGG, it can achieve up to 92.1% parameter reduction and 61.0% FLOPs reduction with no accuracy degradation in forward order. On ResNet-20/CIFAR-10, it reports that C2S2-within and C2S2 outperform Network Slimming, with an example near 50% parameter pruning where the C2S2 accuracy drop is about 1.0% versus about 1.6% for NS. On FCN-8s for PASCAL VOC 2011 segmentation, it reports 95.6% parameter and 63.7% FLOPs reduction with mean IU moving from 57.5% to 54.8% [1904.03508].

## 6. Sensitivity, limitations, and recurring misconceptions

Several limitations recur across PCP variants. The 2025 PCP framework depends on a representative validation set because layer selection is driven by estimated accuracy drops $\Delta A_l$; small or biased validation data can misestimate those drops. Its attempting step also introduces overhead because each iteration requires temporary pruning and validation evaluation for every prunable layer, although the paper reports that the total time increase versus CP is about 7.2% on VGG-16/ImageNet and about 11.8% on ResNet-50/ImageNet [2507.04792]. In unsupervised domain adaptation, poor pseudo-labels or severe domain mismatch can misguide pruning, which the paper mitigates by DANN pre-training and variance-based spatial sampling [2507.04792].

PCS highlights a different failure mode: hard truncation of salience can destabilize training. In a matched-MAdds ablation on ResNet-18, truncation yields Top-1 err 35.9%, MAC 15.0M, 1080Ti 19.4ms, and ARM 40.1ms, whereas PCS yields Top-1 err 29.8%, MAC 6.9M, 1080Ti 13.8ms, and ARM 26.6ms [2304.00280]. The same paper reports that overly large $\lambda$ too early may over-shrink the network, and that excessively large $K$ makes pruning more aggressive and can increase error [2304.00280]. REPrune notes overhead from per-channel agglomerative clustering, sensitivity to linkage choice, early-layer sensitivity, and randomness in tie-breaking among filters with equal coverage scores [2402.17862]. C2S2 notes that setting $c_r$ too aggressively can still lead to noticeable accuracy drop and that residual or shortcut-connected architectures require mask consistency across aligned paths [1904.03508].

A common misconception is to treat PCP as a single method with a single canonical workflow. The literature instead supports a narrower and a broader meaning. Narrowly, PCP refers to the three-step framework of attempting, selecting, and pruning proposed in 2025 [2507.04792]. Broadly, it refers to gradual channel removal during training or iterative compression, encompassing layer-wise progressive pruning, salience shrinkage, and scheduled in-training pruning [2304.00280] [2402.17862] [1904.03508]. Another misconception is to equate lower FLOPs with faster inference. PCS explicitly shows that memory access cost and indexing overhead can dominate realized latency, so static deployment masks and contiguous tensors matter alongside arithmetic reduction [2304.00280]. A plausible implication is that future PCP systems will increasingly optimize both algorithmic salience criteria and deployment-aware cost models rather than FLOPs alone.

Source: https://www.emergentmind.com/topics/progressive-channel-pruning-pcp