Progressive Channel Pruning in Deep Networks

• Progressive Channel Pruning (PCP) is a framework that incrementally removes CNN channels over multiple iterations rather than in a one-shot approach.
• It employs a three-stage process—attempting, selecting, and pruning—guided by validation accuracy to minimize performance loss.
• PCP variants like PCS, C2S2, and REPrune demonstrate improved FLOPs reduction, latency, and deployment efficiency across various benchmarks.

Searching arXiv for the cited PCP literature to ground the article and verify metadata. to=arxiv_search.search 大发快三是什么query":"all:\"Progressive Channel Pruning\" OR all:\"Progressive Channel-Shrinking Network\" OR id:(Pan et al., 2023) OR id:(Guo et al., 7 Jul 2025) OR id:(Park et al., 2024) OR id:(Chiu et al., 2019)","max_results":10}平台总代理 to=arxiv_search.search code 大发快三官网 天天中彩票怎么买result 《凤凰大参考json [{"arxiv_id":"(Guo et al., 7 Jul 2025)","title":"Model Compression using Progressive Channel Pruning","authors":["Y. Zhang","S. Liao","W. Chen","Q. Yang"],"abstract":"In this work, we propose a simple but effective channel pruning framework called Progressive Channel Pruning (PCP) to accelerate Convolutional Neural Networks (CNNs). In contrast to the existing channel pruning methods that prune channels only once per layer in a layer-by-layer fashion, our new progressive framework iteratively prunes a small number of channels from several selected layers, which consists of a three-step attempting-selecting-pruning pipeline in each iteration. In the attempting step, we attempt to prune a pre-defined number of channels from one layer by using any existing channel pruning methods and estimate the accuracy drop for this layer based on the labelled samples in the validation set. In the selecting step, based on the estimated accuracy drops for all layers, we propose a greedy strategy to automatically select a set of layers that will lead to less overall accuracy drop after pruning these layers. In the pruning step, we prune a small number of channels from these selected layers. We further extend our PCP framework to prune channels for the deep transfer learning methods like Domain Adversarial Neural Network (DANN), in which we effectively reduce the data distribution mismatch in the channel pruning process by using both labelled samples from the source domain and pseudo-labelled samples from the target domain. Our comprehensive experiments on two benchmark datasets demonstrate that our PCP framework outperforms the existing channel pruning approaches under both supervised learning and transfer learning settings.","categories":["cs.CV","cs.LG"],"published":"2025-07-07","url":"http://arxiv.org/abs/([2507.04792](/papers/2507.04792))v1"},{"arxiv_id":"([2304.00280](/papers/2304.00280))","title":"Progressive Channel-Shrinking Network","authors":["Y. Xiao","Q. Wang","Z. Chen","M. Zhang","J. Zhang"],"abstract":"Currently, salience-based channel pruning makes continuous breakthroughs in network compression. In the realization, the salience mechanism is used as a metric of channel salience to guide pruning. Therefore, salience-based channel pruning can dynamically adjust the channel width at run-time, which provides a flexible pruning scheme. However, there are two problems emerging: a gating function is often needed to truncate the specific salience entries to zero, which destabilizes the forward propagation; dynamic architecture brings more cost for indexing in inference which bottlenecks the inference speed. In this paper, we propose a Progressive Channel-Shrinking (PCS) method to compress the selected salience entries at run-time instead of roughly approximating them to zero. We also propose a Running Shrinking Policy to provide a testing-static pruning scheme that can reduce the memory access cost for filter indexing. We evaluate our method on ImageNet and CIFAR10 datasets over two prevalent networks: ResNet and VGG, and demonstrate that our PCS outperforms all baselines and achieves state-of-the-art in terms of compression-performance tradeoff. Moreover, we observe a significant and practical acceleration of inference.","categories":["cs.CV"],"published":"2023-04-01","url":"http://arxiv.org/abs/([2304.00280](/papers/2304.00280))v1"},{"arxiv_id":"([2402.17862](/papers/2402.17862))","title":"REPrune: Channel Pruning via Kernel Representative Selection","authors":["J. Lee","S. Park","Y. Choi","D. Kim"],"abstract":"Channel pruning is widely accepted to accelerate modern convolutional neural networks (CNNs). The resulting pruned model benefits from its immediate deployment on general-purpose software and hardware resources. However, its large pruning granularity, specifically at the unit of a convolution filter, often leads to undesirable accuracy drops due to the inflexibility of deciding how and where to introduce sparsity to the CNNs. In this paper, we propose REPrune, a novel channel pruning technique that emulates kernel pruning, fully exploiting the finer but structured granularity. REPrune identifies similar kernels within each channel using agglomerative clustering. Then, it selects filters that maximize the incorporation of kernel representatives while optimizing the maximum cluster coverage problem. By integrating with a simultaneous training-pruning paradigm, REPrune promotes efficient, progressive pruning throughout training CNNs, avoiding the conventional train-prune-finetune sequence. Experimental results highlight that REPrune performs better in computer vision tasks than existing methods, effectively achieving a balance between acceleration ratio and performance retention.","categories":["cs.CV"],"published":"2024-02-27","url":"http://arxiv.org/abs/([2402.17862](/papers/2402.17862))v1"},{"arxiv_id":"([1904.03508](/papers/1904.03508))","title":"C2S2: Cost-aware Channel Sparse Selection for Progressive Network Pruning","authors":["Y.-C. Lin","Y.-Y. Hsu","T.-Y. Chiu","Y.-C. F. Wang"],"abstract":"This paper describes a channel-selection approach for simplifying deep neural networks. Specifically, we propose a new type of generic network layer, called pruning layer, to seamlessly augment a given pre-trained model for compression. Each pruning layer, comprising $1 \times 1$ depth-wise kernels, is represented with a dual format: one is real-valued and the other is binary. The former enables a two-phase optimization process of network pruning to operate with an end-to-end differentiable network, and the latter yields the mask information for channel selection. Our method progressively performs the pruning task layer-wise, and achieves channel selection according to a sparsity criterion to favor pruning more channels. We also develop a cost-aware mechanism to prevent the compression from sacrificing the expected network performance. Our results for compressing several benchmark deep networks on image classification and semantic segmentation are comparable to those by state-of-the-art.","categories":["cs.CV"],"published":"2019-04-06","url":"http://arxiv.org/abs/([1904.03508](/papers/1904.03508))v1"}] Progressive Channel Pruning (PCP) denotes a family of structured network-compression methods that reduce computation and memory by removing channels progressively rather than in a single post-hoc step. In the broad sense, PCP aims to reduce metrics such as FLOPs or MAdds, parameters, and latency while minimizing accuracy loss by distributing pruning decisions across training steps, epochs, or iterative pruning rounds. The term also names a specific framework, "Model Compression using Progressive Channel Pruning," in which a CNN is compressed through an iterative Attempt–Select–Prune loop guided by validation accuracy estimates (Guo et al., 7 Jul 2025). Across the literature, the progressive aspect appears in several forms: layer-wise iterative pruning, differentiable salience shrinking, and training-integrated schedules that interleave pruning with representation learning (Pan et al., 2023, Park et al., 2024, Chiu et al., 2019).

1. Definition and conceptual scope

PCP is best understood as a structured pruning regime in which channel removal is staged over time. In one formulation, for a convolutional layer with output feature maps $x$, a nonnegative salience vector $s$ reweights channels as $x'_c = s_c x_c$; if $s_c = 0$, the $c$-th channel is deactivated or pruned (Pan et al., 2023). In another formulation, each layer $l$ has a channel-selection vector $\beta^{(l)} \in \mathbb{R}^{C_l}$, with $\beta_i = 0$ indicating that the corresponding input channel is pruned, and the global objective is to minimize classification loss subject to a target compression ratio (Guo et al., 7 Jul 2025).

A recurrent contrast in this literature is between progressive pruning and one-shot or layer-by-layer pruning. The 2025 PCP framework explicitly positions itself against methods that prune each layer once in a fixed order, arguing instead for iteratively pruning a small number of channels from several selected layers per iteration (Guo et al., 7 Jul 2025). C2S2 similarly defines “progressive” as layer-wise pruning carried out over multiple iterations after augmenting a pre-trained network with pruning layers (Chiu et al., 2019). REPrune uses the term for pruning decisions made during training at scheduled epochs, rather than in a conventional train–prune–finetune pipeline (Park et al., 2024). This suggests that PCP is not a single algorithmic template but a broader design principle: prune incrementally, preserve structured sparsity, and let later pruning decisions reflect earlier architectural changes.

2. Core optimization principles

A common PCP objective is to minimize task loss while enforcing structured sparsity on channels. The 2025 framework formalizes this as

$$\arg\min_{\tilde{\Theta},\, \mathcal{B}} \; \mathcal{L}_c\big(\mathcal{N}(\mathcal{I}; \tilde{\Theta}, \mathcal{B}), y\big) \quad \text{s.t.} \quad R(\mathcal{B}) \ge R_{\text{target}},$$

where $x$0 denotes per-layer channel selections and $x$1 is a compression ratio function based on FLOPs or parameter reduction (Guo et al., 7 Jul 2025).

When PCP is instantiated with the CP solver, the per-layer pruning subproblem is posed as reconstruction with sparsity:

$x$2

with

$x$3

This is solved by alternating LASSO-based channel selection and least-squares weight update, and those same two subproblems are reused both in the “attempting” stage and in the actual pruning stage (Guo et al., 7 Jul 2025).

Other PCP variants preserve the same high-level objective but change the relaxation or control mechanism. PCS replaces hard channel truncation with a shrinking loss over the $x$4 smallest salience entries,

$x$5

and optimizes

$x$6

where $x$7 is the salience vector sorted in increasing order (Pan et al., 2023). C2S2 instead inserts a pruning layer after each convolution and learns real-valued pruning weights $x$8 via

$x$9

then binarizes with a unified threshold $s$0 to obtain a mask $s$1 (Chiu et al., 2019). REPrune keeps the task loss unchanged, $s$2, and introduces pruning through periodic mask updates driven by BatchNorm scaling factors and representative-based filter selection (Park et al., 2024).

3. Major algorithmic realizations

Several distinct methods now fall under the PCP umbrella.

Method	Progressive mechanism	Distinctive feature
PCP framework	Iterative Attempt–Select–Prune loop	Validation-driven greedy layer selection
PCS	Progressive shrinking of salience entries	Testing-static pruning via running salience
C2S2	Layer-wise pruning with pruning layers	Cost-aware Pruning/Restoring controller
REPrune	Scheduled pruning during training	Kernel representative selection via clustering

The PCP framework of "Model Compression using Progressive Channel Pruning" iteratively prunes a small number of channels from several selected layers. Each iteration has three steps. In the attempting step, the method temporarily prunes $s$3 channels from each layer and measures the corresponding validation accuracy drop $s$4. In the selecting step, it greedily chooses a set $s$5 of size $s$6 with the smallest estimated drops. In the pruning step, it performs actual pruning for those selected layers in shallow-to-deep order, updates the model to $s$7, and stops when $s$8 (Guo et al., 7 Jul 2025).

PCS, introduced in "Progressive Channel-Shrinking Network," starts from salience-based channel pruning and targets two specific limitations: instability caused by abrupt truncation of continuous salience values, and indexing overhead induced by input-dependent dynamic architectures. Instead of thresholding salience entries to zero, PCS progressively compresses only the smallest $s$9 entries by the shrinking loss $x'_c = s_c x_c$0. Its salience generator has the architecture GAP $x'_c = s_c x_c$1 FC $x'_c = s_c x_c$2 ReLU $x'_c = s_c x_c$3 FC $x'_c = s_c x_c$4 hard sigmoid, yielding $x'_c = s_c x_c$5. A Running Shrinking Policy maintains an exponential moving average $x'_c = s_c x_c$6 and uses $x'_c = s_c x_c$7 to stabilize which entries are shrunk. At deployment, the Boolean mask is fixed as $x'_c = s_c x_c$8, and weights and biases are permanently pruned, producing a static architecture (Pan et al., 2023).

C2S2 operationalizes progressive channel pruning by inserting a depth-wise $x'_c = s_c x_c$9 pruning layer after each convolution. The pruning layer has a dual representation: real-valued weights $s_c = 0$0 and a binary mask $s_c = 0$1 obtained by thresholding at $s_c = 0$2. Phase 1 optimizes $s_c = 0$3 with sparsity and bipolar regularization; Phase 2 fixes $s_c = 0$4, applies $s_c = 0$5, and updates only the original network weights. A cost-aware controller monitors the exponential moving average training error $s_c = 0$6 relative to a baseline error $s_c = 0$7, stays in a Pruning state while $s_c = 0$8, switches to Restoring when this bound is exceeded, and terminates a layer when $s_c = 0$9 (Chiu et al., 2019).

REPrune embeds pruning into training and bases channel selection on kernel representative selection. For each input channel, it clusters kernels across output filters by agglomerative clustering with Ward’s linkage, then selects filters by a greedy maximum cluster coverage procedure. Layer-wise sparsity is derived from a global threshold on BatchNorm scales, with

$c$0

The number of reserved filters is $c$1, and the greedy step picks

$c$2

where $c$3 is a coverage score that is updated as clusters become represented (Park et al., 2024).

4. Dynamic versus static pruning and systems consequences

A central systems issue in PCP is the distinction between dynamic, input-dependent channel activation and static deployment masks. PCS makes this issue explicit. It notes that prior salience-based methods often need a truncation rule such as

$c$4

even though salience generators are continuous and rarely output exact zeros. The reported consequence is unstable forward and backward propagation, because channels can be abruptly deactivated at different times for different inputs (Pan et al., 2023).

PCS further argues that dynamic pruning can reduce FLOPs while failing to reduce memory access cost because full kernels must remain indexable online. In that paper, memory access cost for a convolution is defined as

$c$5

By fixing the test-time mask after training and permanently removing zeroed channels, PCS aims to reduce both MAdds and MAC, and thereby practical latency (Pan et al., 2023). The same general concern appears in the broader PCP literature as a distinction between nominal compression ratios and realized deployment efficiency. This suggests that, within PCP, structural density after pruning is as important as sparsity itself when the target is general-purpose hardware.

REPrune is aligned with that systems perspective. It is explicitly designed to emulate finer-grained kernel pruning while still outputting a dense channel-pruned model that remains immediately deployable on general-purpose software and hardware resources (Park et al., 2024). C2S2 also produces dense subnetworks by unplugging pruning layers and physically removing masked channels once a layer has been finalized (Chiu et al., 2019). The 2025 PCP framework likewise performs structured channel removal and a single final fine-tuning stage rather than retaining dynamic gating at inference (Guo et al., 7 Jul 2025).

5. Empirical behavior across benchmarks

Empirical results in the PCP literature are reported on ImageNet, CIFAR-10, Office-31, COCO, and semantic segmentation benchmarks, with architectures including VGG, AlexNet, ResNet, MobileNetV2, SSD300, Mask R-CNN, and preliminary transformer experiments (Pan et al., 2023, Park et al., 2024, Guo et al., 7 Jul 2025, Chiu et al., 2019).

For the 2025 PCP framework on ImageNet VGG-16, the original Top-5 accuracy is 89.9%. PCP reports 90.0% at $c$6, 89.3% at $c$7, and 89.0% at $c$8 compression, compared with CP at 89.9%, 88.9%, and 88.2%, and AMC at 88.5% for $c$9. On ImageNet ResNet-50, PCP reports FLOPs 44.5%, Params 59.2%, Top-1 73.4%, and Top-5 91.5%, compared with ThiNet-50 at FLOPs 44.2%, Params 48.4%, Top-1 71.0%, and Top-5 90.0% (Guo et al., 7 Jul 2025). The same paper extends PCP to DANN on Office-31 by using labeled source samples and pseudo-labeled target samples during pruning; the reported PCP w/ PL results are consistently better than CP across multiple domain-transfer tasks and compression ratios (Guo et al., 7 Jul 2025).

PCS reports detailed latency and MAC comparisons. On ImageNet with ResNet-18, the baseline is Top-1 29.6%, Params 12M, MAdds 1.8G, MAC 14.5M, GPU 18.9ms, and ARM 43.2ms. PCS-ResNet18-B reports Top-1 29.8%, Params 4M, MAdds 1.0G, MAC 6.9M, 1080Ti 13.8ms, and ARM 26.6ms. At roughly similar MAdds, dynamic baselines such as FBS, DGC, and ManiDP retain MAC around 15.1M and exhibit worse latency (Pan et al., 2023). On VGG-16/ImageNet, PCS-VGG16-B reports Top-1 28.5%, Params 45M, MAdds 2.8G, MAC 51.1M, 1080Ti 30.9ms, and ARM 57.0ms, compared with the baseline Top-1 28.4%, Params 138M, MAdds 15.5G, MAC 155M, 1080Ti 83.4ms, and ARM 190.4ms (Pan et al., 2023). That paper also reports preliminary transformer results: T2T-ViT from Top-1 err 18.5%, 13.8G, 1.62ms to 18.5%, 3.9G, 1.18ms, and Swin from 18.7%, 4.5G, 1.32ms to 18.7%, 2.5G, 0.89ms (Pan et al., 2023).

REPrune reports strong results under a training-integrated progressive regime. On ImageNet, ResNet-18 moves from a 69.4% baseline at 1.81G FLOPs to 69.2% at 1.03G. ResNet-34 reports 74.3% at 2.7G and 73.9% at 2.0G from a 73.3% baseline at 3.6G. ResNet-50 reports 77.3% at 2.3G, 77.0% at 1.8G, and 75.7% at 1.0G from a 76.2% baseline at 4.1G (Park et al., 2024). On COCO-2017 detection with SSD300 and a pruned ResNet-50 backbone, a 50% FLOPs reduction yields mAP 25.0, AP50 42.3, and AP75 25.9 (Park et al., 2024).

C2S2 reports that, on CIFAR-10 VGG, it can achieve up to 92.1% parameter reduction and 61.0% FLOPs reduction with no accuracy degradation in forward order. On ResNet-20/CIFAR-10, it reports that C2S2-within and C2S2 outperform Network Slimming, with an example near 50% parameter pruning where the C2S2 accuracy drop is about 1.0% versus about 1.6% for NS. On FCN-8s for PASCAL VOC 2011 segmentation, it reports 95.6% parameter and 63.7% FLOPs reduction with mean IU moving from 57.5% to 54.8% (Chiu et al., 2019).

6. Sensitivity, limitations, and recurring misconceptions

Several limitations recur across PCP variants. The 2025 PCP framework depends on a representative validation set because layer selection is driven by estimated accuracy drops $l$0; small or biased validation data can misestimate those drops. Its attempting step also introduces overhead because each iteration requires temporary pruning and validation evaluation for every prunable layer, although the paper reports that the total time increase versus CP is about 7.2% on VGG-16/ImageNet and about 11.8% on ResNet-50/ImageNet (Guo et al., 7 Jul 2025). In unsupervised domain adaptation, poor pseudo-labels or severe domain mismatch can misguide pruning, which the paper mitigates by DANN pre-training and variance-based spatial sampling (Guo et al., 7 Jul 2025).

PCS highlights a different failure mode: hard truncation of salience can destabilize training. In a matched-MAdds ablation on ResNet-18, truncation yields Top-1 err 35.9%, MAC 15.0M, 1080Ti 19.4ms, and ARM 40.1ms, whereas PCS yields Top-1 err 29.8%, MAC 6.9M, 1080Ti 13.8ms, and ARM 26.6ms (Pan et al., 2023). The same paper reports that overly large $l$1 too early may over-shrink the network, and that excessively large $l$2 makes pruning more aggressive and can increase error (Pan et al., 2023). REPrune notes overhead from per-channel agglomerative clustering, sensitivity to linkage choice, early-layer sensitivity, and randomness in tie-breaking among filters with equal coverage scores (Park et al., 2024). C2S2 notes that setting $l$3 too aggressively can still lead to noticeable accuracy drop and that residual or shortcut-connected architectures require mask consistency across aligned paths (Chiu et al., 2019).

A common misconception is to treat PCP as a single method with a single canonical workflow. The literature instead supports a narrower and a broader meaning. Narrowly, PCP refers to the three-step framework of attempting, selecting, and pruning proposed in 2025 (Guo et al., 7 Jul 2025). Broadly, it refers to gradual channel removal during training or iterative compression, encompassing layer-wise progressive pruning, salience shrinkage, and scheduled in-training pruning (Pan et al., 2023, Park et al., 2024, Chiu et al., 2019). Another misconception is to equate lower FLOPs with faster inference. PCS explicitly shows that memory access cost and indexing overhead can dominate realized latency, so static deployment masks and contiguous tensors matter alongside arithmetic reduction (Pan et al., 2023). A plausible implication is that future PCP systems will increasingly optimize both algorithmic salience criteria and deployment-aware cost models rather than FLOPs alone.