HASTE: A Framework for Training-Free, Dynamic, and Steerable Compression of Pre-Trained Convolutional Neural Networks

Abstract: Deploying large convolutional neural networks (CNNs) on resource-constrained devices is challenging due to their high computational cost. While dynamic execution methods are promising, existing approaches for CNNs typically require specialized training or fine-tuning, limiting their effectiveness when applied to pre-trained models and requiring data access. To address this gap, we propose HASTE (Hashing for Tractable Efficiency), a plug-and-play convolution module that enables training-free, dynamic compression of large pre-trained CNNs. At inference time, HASTE uses locality-sensitive hashing to identify and merge redundant channels of latent feature maps on a patch-wise basis. This process simultaneously compresses the depth of both input features and their corresponding filters, resulting in computationally cheaper convolutions. We conduct extensive experiments on CIFAR-10 and ImageNet across a range of architectures, demonstrating a 46.2% FLOPs reduction in a ResNet34 on CIFAR-10 with only a 1.25% drop in accuracy, without any retraining. We support our claims by comprehensive ablation studies to validate our core design choices, an analysis of the method's properties and limitations, and a discussion that connects our channel merging scheme to the conceptually related task of token merging in Vision Transformers. Our results demonstrate that HASTE provides an effective solution for steerable compression of pre-trained CNNs at runtime, opening new possibilities for the deployment of efficient deep learning methods.

• The paper introduces HASTE, a training-free CNN compression framework that uses locality-sensitive hashing to merge similar feature channels dynamically without fine-tuning or calibration data.
• HASTE reduces ResNet34 FLOPs by 46.72% on CIFAR-10 with a 1.25-point accuracy drop, while achieving smaller savings on ImageNet and wider models benefiting most from compression.
• The method lets users steer compression at runtime through the hyperparameter L, but performs poorly on pointwise-convolution-heavy networks such as MobileNetV2 and requires hardware support for efficient execution.

Motivation and problem setting

Deploying large convolutional neural networks (CNNs) on resource-constrained hardware requires compression, but existing structured pruning and dynamic gating methods share two practical constraints: they either require training from scratch with specialized sparsity losses, or fine-tuning after pruning to restore accuracy, which in turn demands access to the original training data. HASTE (Hashing for Tractable Efficiency) is proposed as a plug-and-play convolution module that removes both requirements: it compresses pre-trained CNNs at inference time, without any training, fine-tuning, or calibration data, and its compression ratio is adjustable at runtime through a single hyperparameter. The authors state that, to their knowledge, this is the first entirely training-free CNN compression method with runtime-adjustable compression — a claim supported by a comparison table in which no prior method satisfies all of dynamic inference, no training, no fine-tuning, and runtime adaptability simultaneously.

Method

HASTE exploits the distributive property of convolutions over channels. Writing the output of filter $j$ as $Y_j = \sum_c F_{j,c} \cdot X_c$, each term factors into magnitudes and cosine similarity between the filter channel slice and input channel slice. If several input channel slices are nearly collinear, their contributions can be approximated by replacing them with their mean $\bar{X}_\mathcal{C}$ and summing the corresponding filter slices:

$$\sum_{c_i \in \mathcal{C}} F_{j,c_i} \cdot X_{c_i} \approx \Big(\sum_{c_i \in \mathcal{C}} F_{j,c_i}\Big) \cdot \bar{X}_{\mathcal{C}}$$

Redundant channels are identified with random-projection locality-sensitive hashing (LSH): each input channel slice within a $(K+2)\times(K+2)$ patch (with two pixels of overlap) is mean-centered, flattened, and hashed by $L$ hyperplanes; channels colliding in the same bucket are merged. The collision probability for two slices separated by angle $\theta$ is $p = (1 - \theta/\pi)^L$, so highly similar channels collide with high probability. Because only input depth is reduced while output dimensions are preserved, downstream layers remain compatible, and the compression ratio varies patch-wise with local feature redundancy.

Two design choices control cost and steerability. First, sparse ternary hyperplanes (entries in $\{-1, 0, 1\}$ with sparsity $s$) replace dense Gaussian projections, eliminating multiplications from hashing entirely. Second, the number of hyperplanes $Y_j = \sum_c F_{j,c} \cdot X_c$0 directly steers the accuracy–FLOPs trade-off: more hyperplanes yield finer buckets, less merging, higher accuracy, and lower savings. Parameter overhead is negligible — roughly 0.04% per module for ResNet18 with $Y_j = \sum_c F_{j,c} \cdot X_c$1.

Component analysis

An ablation isolating three design axes — scope (global vs. patch-wise), criterion ($Y_j = \sum_c F_{j,c} \cdot X_c$2 norm vs. LSH), and operation (removal vs. merging) — shows that LSH-based redundancy detection is the decisive component. At matched compression targets on CIFAR-10, all four $Y_j = \sum_c F_{j,c} \cdot X_c$3-based baselines degrade severely: global removal drops VGG19-BN by 59.06 points, and even the best baseline (patch-wise $Y_j = \sum_c F_{j,c} \cdot X_c$4 + merge) loses 5.00–28.11 points depending on architecture. Full HASTE retains accuracy within 1.89–3.03 points across ResNet18/34 and VGG11-BN/19-BN. This establishes that norm-based saliency is a poor training-free criterion, whereas cosine-similarity clustering of latent channels preserves information flow without any corrective training.

Results

On CIFAR-10, HASTE reduces FLOPs substantially with small accuracy loss. Representative numbers:

Model	Setting	Accuracy drop	FLOPs reduction
ResNet34	$Y_j = \sum_c F_{j,c} \cdot X_c$5	1.25 pp	46.72%
ResNet18	$Y_j = \sum_c F_{j,c} \cdot X_c$6	0.55 pp	35.73%
VGG19-BN	$Y_j = \sum_c F_{j,c} \cdot X_c$7	0.73 pp	34.11%

These results are competitive with methods requiring dedicated training or fine-tuning (e.g., DMCP, SSL), despite being applied instantly to frozen checkpoints. On ResNet34, the trade-off is steerable from a 2.89-point drop at 51.09% FLOPs reduction ($Y_j = \sum_c F_{j,c} \cdot X_c$8) to a 0.38-point drop at 39.07% reduction ($Y_j = \sum_c F_{j,c} \cdot X_c$9).

On ImageNet, achievable savings shrink because latent features are less redundant: ResNet34 loses 1.25 points at 18.69% FLOPs reduction, and ResNet50 loses 1.36 points at 15.68%. A notable positive finding is that effectiveness scales with model size and width — WideResNet101 reaches up to 31.54% FLOPs reduction — indicating that HASTE is best suited to overparameterized models. Per-layer analysis shows deeper layers compress more aggressively, consistent with increasing feature redundancy at higher levels of abstraction.

Latency estimates (necessarily extrapolated, since standard frameworks do not support dynamic patch-wise execution efficiently) suggest approximately 1.48× speedup for ResNet18 and 1.47× for ResNet34 on CPU under a "realistic" hardware-support assumption, versus theoretical upper bounds of 1.72× and 1.88×. Activation memory is also reduced, which matters for memory-bus-bound embedded workloads.

Ablations

Three ablations validate the defaults. Patch size: $\bar{X}_\mathcal{C}$0 patches outperform larger contexts, which lose compression efficiency; the choice is justified as minimal reuse of compressed representations across nine kernel positions rather than tuned as a free hyperparameter. Hyperplane sparsity: CIFAR-10 tolerates aggressive sparsity ($\bar{X}_\mathcal{C}$1), while ImageNet requires denser hyperplanes ($\bar{X}_\mathcal{C}$2) to avoid hashing-quality degradation — an important dataset-dependent setting. Starting layer: beginning compression later sacrifices total achievable savings but permits smaller $\bar{X}_\mathcal{C}$3 and near-lossless accuracy; starting at layer 4 of ResNet18 yields a 17.13% FLOPs reduction with essentially zero accuracy drop (0.01 points).

Limitations

The paper is candid about boundaries. The most significant is architectural sensitivity to pointwise convolutions: hashing and merging costs scale worse than convolution cost as kernels shrink (hashing is only 2.8× cheaper for 1×1 vs. 3×3 convolutions, against 9× cheaper convolution FLOPs). Consequently, gains dip from ResNet34 to bottleneck-based ResNet50, and MobileNetV2 largely defeats the method: sparse hyperplanes collapse accuracy (down to 26.68% top-1 in one configuration), and dense hyperplanes can make HASTE a net negative (-7.56% FLOPs in one setting); only a conservative configuration achieves a modest 7.84% saving at a 1.59-point accuracy cost. This restricts applicability to architectures dominated by spatial kernels. Additional limitations include stochasticity from random hyperplane initialization (mitigated cheaply since seed evaluation is training-free) and dependence on custom kernels or future framework support for conditional execution to realize theoretical speedups.

Relation to token reduction in Vision Transformers

The paper draws a careful analogy to token pruning and merging in ViTs (EViT, ToMe, ToFu, DynamicViT). Both paradigms compress redundant latent features of pre-trained models without retraining, but differ fundamentally: ViTs possess built-in similarity signals via self-attention, whereas CNN features require an explicit mechanism such as LSH; and token methods reduce the sequence dimension (permissible because transformers are agnostic to token count), whereas HASTE reduces channel depth (necessary because subsequent convolutions have fixed input dimensions). The authors argue that applying LSH to tokens would be redundant given attention, and that extending channel compression to ViT linear layers would inherit exactly the pointwise-overhead problem identified above — making efficient handling of 1×1 convolutions the key open problem that would also unlock Transformer applicability.

Conclusion

HASTE demonstrates that locality-sensitive hashing over latent feature channels provides a viable, fully training-free mechanism for dynamic CNN compression, achieving up to ~47% FLOPs reduction on CIFAR-10 ResNet34 with a 1.25-point accuracy drop and scaling favorably with model width. Its principal restrictions — degraded effectiveness on pointwise-dominated architectures and reliance on runtime support for dynamic execution — are clearly delineated, and the identification of efficient 1×1 convolution handling stands as the central open question left by this work.