KAConv: Kernel attention convolutions

Most of the previous network optimization works applied attention mechanism to feature maps, but neglected to embed attention into convolution kernel of the end-to-end network that is convenient to deploy. To address this issue, we present a novel attention convolution method named Kernel Attention Convolution (KAConv) to enhance the flexibility of convolution. The proposed KAConv generates different attention weights for different spatial positions of convolution kernels based on the input features, so as to dynamically adjust the parameters of convolution kernels during the forward propagation to enhance the flexibility of convolution. We decompose the convolution kernels into subkernels spatially, and generate the corresponding feature maps through which attention weights are obtained. The final refined feature maps are aggregated by the attention weighted feature maps corresponding to each subkernel. KAConv is a computationally lightweight convolution method, which not only incorporates attention into kernels but also enhances informative representations. By replacing the standard convolution with the proposed KAConv in convolutional neural networks (CNNs), the networks yield significant performance improvement. Extensive experiments on the ImageNet-1K benchmark demonstrate that KAConv outperforms existing attention mechanism-based methods. We also carry out experiments on the MS COCO and PASCAL VOC datasets to show the generalization ability of our method.