Carbon Emissions Reduction of Neural Network by Discrete Rank Pruning

Although deep neural networks (DNNs) have achieved excellent performance in computer vision applications in recent years, it’s still challenging to deploy them on resource-limited devices due to their high computation costs and memory footprint. Meanwhile, training DNNs consumes huge energy, leading to excessive carbon emissions and accelerating global warming. To solve these problems, we first propose a novel filter pruning algorithm for neural network named Discrete Rank Pruning (DRP). It is convenient to deploy large scale models on resource-limited devices. Second, we propose a novel calculation method named Neural Network Carbon Emission Calculator (NNCEC) to quantify DNNs energy consumption and carbon emission. It makes the environmental cost of neural network become transparent. Moreover, many pruning methods apply sparse regularization on the filter weights of the convolution layers to reduce the degradation of performance after pruning. We analyze these methods and find that it is necessary to consider the influence of the bias term. Based on these, we propose a novel sparse method named Consideration Bias Sparsity (CBS). Extensive experiments on MNIST, CIFAR-10 and CIFAR-100 datasets with LeNet-5, VGGNet-16, ResNet-56, GoogLeNet and DenseNet-40 demonstrate the effectiveness of CBS and DRP. For LeNet-5, CBS achieves 1.87% increase in accuracy than sparse regularization method on MNIST. For VGGNet-16, DRP achieves 66.6% reduction in FLOPs by removing 83.3% parameters with only 0.36% decrease in accuracy on CIFAR-10. For ResNet-56, DRP achieves 47.89% reduction in FLOPs by removing 42.8% parameters with only 0.82% decrease in accuracy on CIFAR-100. For GoogLeNet, DRP achieves over 50% carbon emissions reduction on CIFAR-10 and CIFAR-100.