Diagnosis of cholangiocarcinoma from microscopic hyperspectral pathological dataset by deep convolution neural networks

This paper focuses on automatic Cholangiocarcinoma (CC) diagnosis from microscopic hyperspectral (HSI) pathological dataset with deep learning method. The first benchmark based on the microscopic hyperspectral pathological images is set up. Particularly, 880 scenes of multidimensional hyperspectral Cholangiocarcinoma images are collected and manually labeled each pixel as either tumor or non-tumor for supervised learning. Moreover, each scene from the slide is given a binary label indicating whether it is from a patient or a normal person. Different from traditional RGB images, the HSI acquires pixels in multiple spectral intervals, which is added as an extension on the channel dimension of 3-channel RGB image. This work aims at fully exploiting the spatial-spectral HSI data through a deep Convolution Neural Network (CNN). The whole scene is first divided into several patches. Then they are fed into CNN for the tumor/non-tumor binary prediction and the tumor area regression. The further diagnosis on the scene is made by random forest based on the features from patch prediction. Experiments show that HSI provides a more accurate result than RGB image. Moreover, a spectral interval convolution and normalization scheme are proposed for further mining the spectral information in HSI, which demonstrates the effectiveness of the spatial-spectral data for CC diagnosis.