TetrisG-SDK: Efficient Convolutional Layer Mapping with Adaptive Windows and Grouped Convolutions for Fast In-Memory Computing

Abstract: Shifted-and-Duplicated-Kernel (SDK) mapping has emerged as an effective strategy to accelerate convolutional layers on compute-in-memory (CIM) hardware. However, existing SDK variants (e.g., VWC-SDK) merely optimize mapping for a single CIM macro, leaving inter-macro parallelism unexplored. Moreover, their mapping methodologies are still suboptimal. To address these limitations, we present TetrisG-SDK, a novel framework that employs adaptive windows to boost mapping performance. The proposed windows accommodate more input channels, increase array utilization at marginal space, and adapt to different channel depths. More importantly, TetrisG-SDK reduces compute latency by searching for optimal window configurations across multiple CIM macros with a fixed hardware budget. Besides, it incorporates grouped convolution to further decrease computing cycles while maintaining near-lossless model accuracy. In addition, TetrisG-SDK integrates a validated CIM hardware simulator to provide accurate system-/application-level estimations of latency, area and energy. Compared to the single-macro VWC-SDK, the proposed framework achieves a speed-up by 1.2x, 1.3x, and 1.3x for CNN8, GoogLeNet Inception, and DenseNet40 models, respectively. When deployed on the simulator, it reduces system-level latency and energy by 2.4x and 1.7x for CNN8, 1.3x and 1.2x for Inception, and 1.3x and 1.6x for DenseNet40, respectively. When leveraging macro-level parallelism, TetrisG-SDK reduces the Energy-Delay-Area-Product (EDAP) by 70% for CNN8, 68% for Inception, and 36% for DenseNet40 compared to its non-grouped counterpart. These results manifest that TetrisG-SDK is a promising solution to efficiently mapping convolutional layers on CIM hardware.

• The paper introduces TetrisG-SDK, a novel framework that leverages adaptive windowing and grouped convolutions to significantly reduce compute cycles and energy consumption in CIM-based CNN acceleration.
• It utilizes a macro-grid search paired with specialized window adaptations (square-inclined, marginal, depth-optimal) to optimize array utilization and enhance hardware parallelism.
• Experimental results reveal up to 2.4× latency reduction, 1.7× energy savings, and 70% EDAP reduction over previous approaches, validating the framework’s efficiency and minimal accuracy loss.

TetrisG-SDK: Adaptive Window and Grouped Convolutional Layer Mapping for Efficient In-Memory Computing

Introduction

This paper introduces TetrisG-SDK, a mapping framework targeting Convolutional Neural Network (CNN) acceleration via compute-in-memory (CIM) architectures. The framework jointly exploits adaptive windowing and grouped convolution techniques to address limitations in array utilization, computational cycles, and hardware parallelism found in prior SDK-based methods such as VW-SDK and VWC-SDK. Critically, TetrisG-SDK performs a macro-grid search and incorporates sophisticated window adaptation (square-inclined, marginal, and depth-optimal) under channel partitioning, yielding significant system-level performance gains without substantial accuracy sacrifice.

The design and implementation of TetrisG-SDK are validated on SRAM-based CIM platforms using the DNN+NeuroSim simulator, demonstrating considerable improvements—most notably, up to $2.4\times$ latency reduction and $1.7\times$ energy savings on CNN8, as well as up to $70\%$ EDAP reduction—over the strongest baselines.

Background: Mapping Challenges in CIM-based CNN Acceleration

CIM architectures collocate computation and storage at the memory macro level. For convolutions, kernel weights are mapped directly onto memory bitcell arrays while input feature maps (IFMs) are loaded onto wordlines; this structural approach minimizes data movement and unlocks substantial energy benefits compared to von Neumann systems.

Figure 1: Convolution operations in a CIM macro, with weights mapped to bitcells and IFMs presented to wordlines.

SRAM-based CIM macros are prevalent due to maturity, analog/mixed-signal support, and technology compatibility. The array architecture underpins performance constraints; the number of parallel MAC operations depends on channel/weight mapping, IFM/kernels, and marginal array rows/columns.

Despite many proposed mapping frameworks (e.g., img2col, original SDK, VW-SDK, VWC-SDK, Tetris-SDK), two critical limitations remain:

1. Single-macro mapping with rigid, fixed window shapes fails to optimize cross-macro parallelism (i.e., inter-macro workload balancing).
2. Non-adaptive handling of marginal and partitioned feature map spaces, leading to suboptimal cycles and hardware underutilization.

TetrisG-SDK was designed to address these limitations.

The TetrisG-SDK Framework

Macro-Grid Search and Window Adaptation

TetrisG-SDK introduces a macro-grid search that explores all $r\times c$ macro arrangements under a fixed hardware budget and selects a configuration minimizing total compute cycles. Instead of a layer-to-macro mapping, the framework partitions a layer across multiple CIM macros, maximizing concurrency and minimizing latency.

Figure 3: Overview flowchart of TetrisG-SDK, depicting the pipeline of enabling grouped convolutions, optimal macro grid selection, and sequentially searching for optimal windows (square-inclined, marginal, depth-optimal).

Adaptive window strategies are central:

• Square-inclined windows minimize required array rows at fixed kernel sizes.
• Marginal windows efficiently handle border cases where window size does not evenly divide IFMs, reducing null input mapping and increasing utilization (as shown in Figure 4).
• Depth-optimal windows allow partition flexibility for layers with irregular channel dimensions, optionally integrating channel pruning for further savings.

  Figure 5: Illustration of marginal space mapping, showing reduction in wasted array area for Tetris-SDK versus VW-SDK.

  

  Figure 7: TetrisG-SDK with grouped convolutions: the original window (left) supports 4 kernel computations, whereas the rearranged window (right) with grouping enables 9 concurrent computations.

Grouped Convolutions

Grouped convolutions reduce the effective input/output channel dimensionality per group, dramatically shrinking the per-macro workload and enabling larger, more parallel window shapes. The grouping factor $G$ is optimized to minimize compute cycles without incurring appreciable accuracy degradation. TetrisG-SDK natively supports grouped convolutions, integrating them into all phases of window and macro-grid search.

Figure 9: Schematic comparison of standard versus grouped convolutions, highlighting the partitioning of IFMs and filters.

Cycle and Utilization Modeling

TetrisG-SDK analytically models both compute cycles and array utilization. For multiple macros, per-tile ($N$) window counts and their distribution are recomputed under the candidate macro grid, and marginal/multiple channel depth cases are dynamically handled. The effective array utilization is tracked as the ratio of active weight-mapped array cells to total array capacity.

Experimental Results

Window-level and System-level Results

Experiments on CNN8, GoogLeNet Inception, DenseNet40, and MobileNet demonstrate the generic applicability and effectiveness of TetrisG-SDK, compared to older SDK and img2col approaches.

Figure 11: Illustration of kernel and IFM computations within parallel windows—TetrisG-SDK systematically maximizes the concurrent kernel windows per array footprint.

Notably, TetrisG-SDK:

• Outperforms VWC-SDK, achieving 1.2$\times$–1.3$\times$ speedup in compute cycles across major CNN workloads.
• Attains $2.4\times$ (CNN8), $1.3\times$ (Inception, DenseNet40) latency reduction and similar energy reduction in DNN+NeuroSim system-level simulations.
• Substantially increases marginal array utilization (by $1.7\times$0, $1.7\times$1, $1.7\times$2 for CNN8, Inception, DenseNet40).
• Cuts EDAP by $1.7\times$3 (CNN8), $1.7\times$4 (Inception), and $1.7\times$5 (DenseNet40) when leveraging macro-level parallelism with grouping.

  Figure 8: Normalized EDAP improvements across multiple macro budgets—demonstrating the advantage of macro-grid-aware parallel mapping.

Grouped convolutions incur minimal accuracy penalties (typically $1.7\times$6 classification loss), and in some workloads even improve generalization, corroborating with findings in the structured pruning/grouped convolution literature.

Ablation Analysis

Ablation studies on CNN8 show that each step—enabling square-inclined windows, marginal window adaptation, depth-optimal handling, and especially grouping—progressively reduces compute cycles, energy, and latency.

Figure 14: Stepwise ablation study: grouped convolution provides the dominant performance uplift, as measured by compute cycles, normalized energy, and latency.

Theoretical and Practical Implications

This work establishes several principles for CIM mapping:

• Macro-level parallelism is workload dependent: TetrisG-SDK’s macro-grid search reveals that the optimal macro configuration varies per network topology and should be dynamically selected to minimize EDAP.
• Window adaptation is necessary for practical efficiency: Adaptive windows outperform fixed (even partitioned) windows, with square-inclined and marginal windows yielding clear hardware resource dividends.
• Grouped convolutions form a natural synergy with CIM mapping: By reducing channel tiling bottlenecks and expanding windowing options, grouping is a robust approach for both scalability and efficiency.

TetrisG-SDK’s framework is systematically applicable to a broad set of workloads and can be incorporated into the software toolchain for mapping arbitrary CNNs to CIM backends. Its contributions in cycle modeling, adaptive window generation, and holistic macro-grid optimization address central bottlenecks in mapping schemes.

Conclusion

TetrisG-SDK provides a comprehensive solution for efficient convolutional layer mapping on CIM hardware. By integrating adaptive windows, a macro-grid search, and grouped convolutions, it achieves significant system-level performance improvements—including compute cycle, energy, and latency reduction—across diverse neural network benchmarks. Given its accuracy retention and generality, TetrisG-SDK sets a new standard in the design of mapping algorithms for high-performance, energy-efficient, and scalable CIM-based AI accelerators. Future work could further investigate automated selection of grouping factors, exploit hardware heterogeneity (e.g., mixed SRAM/RRAM), or extend the strategy to more general layer types and non-convolutional architectures.