Crop type recognition of VGI road-side images via hierarchy structure based on semantic segmentation model Deeplabv3+

The application of artificial intelligence in the field of agricultural crop type recognition is of great significance. There are currently two methods for collecting crop information: remote sensing and volunteered geographic information (VGI). As one pixel in a remote sensed satellite image may correspond to multiple crop types due to its low resolution, more refined VGI images can be a supplementary data source for agricultural crop monitoring. A hierarchal semantic segmentation structure based on Deeplabv3+ is proposed in this paper. Inspired by causality theories, a binary classification is adopted in the first stage of the model to segment crops from background. The second stage aims to recognize the crop types including rape, corn, fallow and bare land, wheat and rice. As wheat and rice have many similar characteristics, they are considered as one category in this stage and will be further processed in the downstream. In the third stage, a priori knowledge such as the location and time of image acquisition is utilized for fine recognition between wheat and rice. We have carried out experiments on the proposed Deeplabv3+ model and other different semantic segmentation models are validated on a VGI dataset with 20% rough labels and 80% accurate labels, including thousands of road-side images collected from different image sensors, under different weather and from different geographic locations. Experiments results show that Deeplabv3+ achieves the best performance. The recognition of five categories of crops has reached a precision of 87%, a recall of 91%, and IoU of 81%, showing good potential for use in the crop type recognition for VGI images.