Response map evaluation for RGBT tracking

In recent years, RGB and thermal sensors are widely used. There is complementary information from these two types of sensors. A fundamental task which arises in this domain is RGBT tracking. It is a challenging problem to leverage RGB and thermal data. In this paper, we propose an adaptive fusion algorithm based on response map evaluation for RGBT tracking. Specifically, a hierarchical convolutional neural network is employed to extract deep features in RGB and thermal images, respectively. The target is tracked in correlation filter framework with each layer independently in RGB and thermal images. To evaluate response map of tracking status in various conditions, the average sidelobe peak response (ASPR) is proposed. Gaussian regression process is employed to provide adaptive fusion weights based on ASPR. Experimental results on two RGBT tracking datasets demonstrate the success of our method.