3D-PulCNN: Pulmonary cancer classification from hyperspectral images using convolution combination unit based CNN

Pulmonary cancer is one of the most common malignancies worldwide. Accurate classification of its subtypes is required in differential diagnosis. However, existing algorithms are mostly based on color images, and the improvement of accuracy is quite challenging. In this study, we propose a convolution combination unit (CCU)-based three-dimensional convolutional neural network (3D-PulCNN) for classifying pulmonary cancer presented in microscopic hyperspectral image with both spatial and spectral information. CCU is designed to fuse the features acquired by different convolution scales. Compared with VGGNet, only two fully connected layers are used in this model, reducing the network parameters and model complexity. Experimental results show that 3D-PulCNN achieves overall average (OA) of 0.962 and Precision, Recall, and Kappa of more than 0.920, superior to 2D-VGGNet. Then, 3D-UNet is leveraged to segment cancer cells, and their morphological characteristics are calculated to supply quantitative virtual analysis data for classification results explanation and prognosis assessment.