ScriptNet: A Two Stream CNN for Script Identification in Camera-Based Document Images

Script identification is an essential part of a document image analysis system, since documents written in different scripts may undergo different processing methods. In this paper, we address the issue of script identification in camera-based document images, which is challenging since the camera-based document images are often subject to perspective distortions, uneven illuminations, etc. We propose a novel network called ScriptNet that is composed of two streams: spatial stream and visual stream. The spatial stream captures the spatial dependencies within the image, while the visual stream describes the appearance of the image. The two streams are then fused in the network, which can be trained in an end-to-end manner. Extensive experiments demonstrate the effectiveness of the proposed approach. The two streams have been shown to be complementary to each other. An accuracy of 99.1 % has been achieved by our proposed network, which compares favourably with state-of-the-art methods. Besides, the proposed network achieves promising results even when it is trained with non-camera-based document images and tested on camera-based document images.