ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks (1908.03930v3)

Abstract: As designing appropriate Convolutional Neural Network (CNN) architecture in the context of a given application usually involves heavy human works or numerous GPU hours, the research community is soliciting the architecture-neutral CNN structures, which can be easily plugged into multiple mature architectures to improve the performance on our real-world applications. We propose Asymmetric Convolution Block (ACB), an architecture-neutral structure as a CNN building block, which uses 1D asymmetric convolutions to strengthen the square convolution kernels. For an off-the-shelf architecture, we replace the standard square-kernel convolutional layers with ACBs to construct an Asymmetric Convolutional Network (ACNet), which can be trained to reach a higher level of accuracy. After training, we equivalently convert the ACNet into the same original architecture, thus requiring no extra computations anymore. We have observed that ACNet can improve the performance of various models on CIFAR and ImageNet by a clear margin. Through further experiments, we attribute the effectiveness of ACB to its capability of enhancing the model's robustness to rotational distortions and strengthening the central skeleton parts of square convolution kernels.

• The paper presents ACBs that replace standard square convolution kernels to enrich feature representation without increasing inference-time computation.
• It employs a fusion of 3x3, 1x3, and 3x1 kernels, achieving up to 1.52% Top-1 accuracy gains on benchmarks like ImageNet.
• The approach integrates easily with frameworks like PyTorch and TensorFlow, offering practical enhancements for resource-constrained deployments.

ACNet: Enhancing CNNs with Asymmetric Convolution Blocks

The paper "ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks" presents a novel approach to improving Convolutional Neural Networks (CNNs) without increasing inference-time computational complexity. The researchers introduce Asymmetric Convolution Block (ACB), a design mechanism that leverages 1D asymmetric convolutions to bolster the square convolution kernels typically used in CNNs.

The central contribution of this work is the construction of Asymmetric Convolutional Networks (ACNet) by replacing traditional convolutional layers with ACBs. This substitution is aimed at enriching the feature representation without altering the original network architecture's computational demands post-training, as ACBs can be converted back into equivalent standard convolutional layers.

Methodology

1. Asymmetric Convolution Block (ACB):
   • Each ACB comprises three convolutional layers using $3 \times 3$, $1 \times 3$, and $3 \times 1$ kernels, respectively.
   • These outputs are aggregated to create a robust feature space.
   • Post-training, ACNet can be reverted to the original architecture by fusing these asymmetric kernels into the standard ones, preserving the computational budget.
2. Theoretical Foundation:
   • The approach exploits the additive property of convolutions, allowing for the merging of variously sized and shaped kernels while maintaining computational consistency.
3. Implementation and Practicality:
   • The ACBs require no additional tuning parameters.
   • They can be integrated with widely-used frameworks such as PyTorch and TensorFlow.
   • Importantly, the transformation involves no additional computational burden at inference time.

Experimental Results

Empirical evaluations across multiple architectures, including VGG-16, ResNet-56, and DenseNet-121, on datasets like CIFAR-10 and ImageNet, demonstrate clear improvements in classification accuracy:

• CIFAR-10/100: Consistent performance gains were observed across all tested models, with increases ranging from 0.27% to 1.11% on CIFAR-10.
• ImageNet: For models such as AlexNet and ResNet-18, ACNet provided improvements of up to 1.52% in Top-1 accuracy.

These results suggest that the integration of ACBs enhances representational capacity, likely due to their ability to target and strengthen the kernel's central skeleton regions, which were identified as crucial for model performance.

Implications and Future Directions

The paper challenges existing methodologies by proposing a structural augmentation that does not require further computational resources during inference. Such modifications are highly beneficial in resource-constrained environments where efficient deployment is critical, such as mobile devices or edge computing.

Additionally, the work opens avenues for further exploration into kernel design and architecture-neutral blocks. The ability of ACBs to enhance models' robustness to rotational distortions indicates a potential for addressing transformation invariance, which could be pivotal for future advancements in neural network robustness and generalization.

In conclusion, the introduction of ACBs marks a significant step towards efficiently enhancing CNN architectures. Future research could focus on extending this approach to other neural network components, such as attention mechanisms or recurrent architectures, broadening the scope of architecture-neutral enhancements.