CRIMP: Compact & Reliable DNN Inference on In-Memory Processing via Crossbar-Aligned Compression and Non-ideality Adaptation

Abstract: Crossbar-based In-Memory Processing (IMP) accelerators achieve high-speed, low-power computing for deep neural networks (DNNs), but face three obstacles. First, floating-point (FP) arithmetic is incompatible with crossbars, and existing quantization schemes still require FP processors for scaling factors, incurring hardware overhead. Second, redundant DNN parameters occupy too many crossbars, and current IMP-aware pruning methods require data aligning across crossbars, introducing significant memory and computing overhead. Third, non-ideal crossbar behaviors such as write variations degrade the accuracy of deployed models, and existing compensation methods add substantial overhead. In this paper, we address all three problems within a single training process. We reuse bit-shift units in crossbars to approximately multiply scaling factors, avoiding FP processors. We apply kernel-group pruning and crossbar pruning to remove the hardware units needed for data aligning. We adopt runtime-aware non-ideality adaptation to relieve the impact of device non-ideality from the training stage by exploiting crossbar features. Integrating these three optimizations into one comprehensive learning framework reduces training overhead and improves accuracy. Experiments show that our quantization incurs a negligible accuracy drop, and our pruning achieves higher sparsity and accuracy than state-of-the-art methods. Our framework produces integer-only, pruned, and reliable VGG-16 and ResNet-56 models for CIFAR-10 on IMP accelerators, with accuracy drops of only 2.19% and 1.26%, respectively, without hardware overhead.

• The paper introduces a unified framework that co-optimizes integer-only quantization, crossbar-aligned multi-grained pruning, and runtime non-ideality adaptation for ReRAM in-memory processing.
• It achieves high compression with minimal accuracy loss, delivering up to 122.38× power and 19.51× area reductions compared to uncompressed baselines.
• The approach ensures robust DNN inference under realistic device non-idealities, facilitating efficient deployment on resource-constrained analog accelerators.

CRIMP: A Comprehensive Framework for Compact and Reliable DNN Inference on ReRAM IMP

Introduction and Motivation

Crossbar-based ReRAM In-Memory Processing (IMP) accelerators provide significant improvements in energy efficiency and throughput for deep neural network (DNN) inference. However, deployment of DNNs on such hardware is hindered by three key barriers: (1) incompatibility of floating-point (FP) arithmetic with crossbar operations, (2) inefficient hardware utilization due to redundant model parameters, and (3) accuracy degradation arising from intrinsic device non-idealities such as write variations (WV) and stuck-at-faults (SAF). Existing solutions—whether through quantization, IMP-aware pruning, or non-ideality compensation—are limited by hardware overhead, integration density losses, or lack of consideration for crossbar-specific execution behavior.

The paper "CRIMP: Compact & Reliable DNN Inference on In-Memory Processing via Crossbar-Aligned Compression and Non-ideality Adaptation" (2607.08015) proposes a unified framework, CRIMP, that tightly integrates integer-only quantization, crossbar-aligned pruning, and runtime-aware non-ideality adaptation into a single co-optimization process. This approach eliminates the need for FP units, maximizes crossbar utilization without auxiliary data alignment hardware, and yields robust models under realistic ReRAM device characteristics.

Architectural and Algorithmic Design

IMP Architecture and Mapping Considerations

The CRIMP framework targets typical ReRAM-based IMP architectures, focusing on efficient mapping of both fully connected and convolutional layers. A key insight is the use of fully-folded mapping schemes for convolutional layers, maximizing crossbar occupancy and parallelism (Figure 1). Two mapping strategies—semi-folded and fully-folded—are compared, with the fully-folded approach preferred for its alignment with hardware efficiency.

Figure 1: IMP architecture from ISAAC, including key units such as Sparsity Table (ST) for data-aligned pruning and multipliers (MUL) for FP scaling factors.

Furthermore, the paper details how quantized convolution operations are represented and mapped onto crossbars, noting the trade-offs between mapping strategies in both execution cycles and crossbar consumption (Figure 2).

Figure 2: (a) Quantized convolutional layer; (b-c) mapping to crossbar and resource use trade-offs.

Crossbar-Aligned Pruning: Kernel-Group and Crossbar-Level

CRIMP introduces a hybrid pruning approach combining:

• Kernel-group pruning: Entire groups of kernels are pruned, ensuring remaining channels/kernels are integer multiples of the crossbar array width, which maintains data alignment without auxiliary hardware.
• Crossbar pruning: Entire crossbar blocks of weights are pruned at once, eliminating the need for hardware mask units such as sparsity tables.

This multi-grained strategy allows aggressive parameter reduction while avoiding the hardware and integration penalties of fine-grained, unaligned methods (Figure 3).

Figure 3: Execution of convolution layer under different sparsity schemes (crossbar-column, kernel-group, crossbar).

Integer-Only Quantization Using Bit-Shift Units

The quantization scheme in CRIMP eliminates FP scaling factors by constraining all scaling coefficients to integer powers of two. This enables replacement of multipliers with bit-shift operations already available in the crossbar (register-transfer) units, thus eliminating both area and power overheads associated with FP or even fixed-point multiplication.

This quantization is carefully integrated into the training process to maintain accuracy despite the increased approximation; the quantization parameters (e.g., scaling factors) are selected to minimize error, and the training optimization adapts accordingly.

Runtime-Aware Non-Ideality Adaptation

Unlike prior works that introduce non-ideality compensation solely during inference or via hardware, CRIMP injects realistic device non-idealities (SAF, WV) into the training loop by simulating the actual inference flow on ReRAM crossbars (Figure 4). Errors caused by non-idealities are rigorously quantified as a function of the crossbar height $H$ and WV standard deviation $\epsilon$ (Figure 5), providing analytical guidance for both algorithm and hardware design.

Figure 5: Relationship between error expectation due to WV, crossbar height $H$, and variation degree $\epsilon$.

The simulation accurately reflects how error propagates and aggregates, demonstrating that for smaller crossbar heights or lower variation, quantization error can be masked by ADC rounding (Figure 6).

Figure 6: Crossbar height determines whether ADC rounding fully suppresses WV-induced error.

Figure 4: Simulated execution flow incorporating quantization, mapping, and hardware non-idealities.

End-to-End Co-Optimization and Zero-Recovery

The key innovation is joint training: pruning masks (for both kernel and crossbar pruning), quantization parameters, and non-ideality simulation are integrated into a single differentiable training loop. An explicit zero-recovery process enables previously pruned parameters to recover if needed, facilitating exploration of better-performing subnets during training. This yields high sparsity, robustness, and zero FP overhead without iterative or sequential post-processing.

Numerical Results and Evaluations

Quantization and Pruning Performance

CRIMP's integer-only quantization yields negligible to negative accuracy drops on MNIST, CIFAR-10, and ImageNet benchmarks, rivaling or surpassing state-of-the-art (Table 1 in the paper). For instance, VGG-16 on CIFAR-10 achieves a 0.11% gain over the baseline. Pruned models demonstrate sparsity rates of 88.25% (VGG-16) and 51.36% (ResNet-56), considerably higher than reported by prior IMP-aware pruning methods, with minimal accuracy degradation (≤0.43%).

Power and Area Gains

Simulation on the ISAAC architecture (with CACTI and NeuroSim models) demonstrates an average $122.38\times$ reduction in computing power and $19.51\times$ reduction in compute area relative to uncompressed baselines. The removal of ST and MUL units is a primary contributor to these gains (Figure 7).

Figure 7: Power and area consumption across models and methods; CRIMP achieves maximum reduction.

Robustness against Device Non-Idealities

CRIMP's runtime-aware adaptation enables stable inference under aggressive SAF and WV. Notably, for VGG-16 on CIFAR-10 with 88.25% pruning and $\epsilon=0.5$, accuracy drops by only 2.97% compared to a full-precision, uncompressed baseline under ideal hardware. Without the adaptation component, accuracy drops sharply in high-variation regimes (Figure 8).

Figure 8: Accuracy under different crossbar heights, pruning levels, and with/without non-ideality adaptation (NIA).

Furthermore, the empirical curves illustrate that the architectural parameter (crossbar height $H$) can be chosen to take advantage of error-masking via ADC rounding in the presence of non-ideality, validating the analytical model.

Comprehensive Benchmarking

Compared with alternatives such as R-V-W, AIIR, CorrectNet, and CTSF, CRIMP achieves comparable or higher accuracy with zero hardware/area overhead, smaller quantization bitwidths, and substantially higher model compression.

Implications and Prospects

The CRIMP framework directly addresses critical system bottlenecks in hardware-aware DNN deployment on ReRAM IMP devices. By aligning algorithmic design and hardware constraints at the training level, it eliminates three central sources of inefficiency: the FP-integer impedance mismatch, non-structured sparsity's hardware cost, and post-hoc error compensation overhead.

From a theoretical perspective, the joint co-optimization approach may inform future research in differentiable hardware mapping and co-design, particularly for analog/mixed-signal accelerators where device-level physics interacts non-trivially with computation.

In practical terms, CRIMP facilitates the deployment of complex DNNs—including ImageNet-scale models—on resource-constrained crossbar arrays, providing a pathway to integer-only, highly compressed, robust inference without custom hardware additions or iterative workflows.

Possible directions for future work include extending CRIMP to transformer architectures, continuous relaxation of quantization/rounding for stochastic devices, and tighter integration with device-circuit co-simulation frameworks.

Conclusion

CRIMP proposes a unified, hardware-software-algorithm co-optimization framework for deploying compact and reliable DNNs on ReRAM IMP accelerators. Through crossbar-aligned multi-grained pruning, integer-only quantization, and runtime-aware non-ideality adaptation—jointly optimized during training—CRIMP achieves over two orders of magnitude reductions in compute resource requirements while maintaining high inference accuracy under realistic device conditions and zero additional hardware overhead. The approach's analytical error modeling and practical results establish it as a robust foundation for future edge AI deployments on analog in-memory computing substrates.