Inertial constrained hierarchical belief propagation for optical flow

In computer vision, optical flow estimation has attracted the researchers’ interests for decades. Loopy belief propagation (LBP) is widely used for obtaining accurate optical flow in recent years. But its time-consumption and unfitness for large displacement scenes remains a challenging problem. In order to improve the performance of belief propagation in optical flow estimation, we propose an Inertial Constrained Hierarchical Belief Propagation (IHBPFlow) to estimate accurate optical flow. We treat input images as Markov random fields (MRF) and use possible displacements as labels and perform BP on hierarchical MRFs, i.e. superpixel MRF and pixel MRF. First we perform BP on the superpixel MRF, where the step of candidate displacements is enlarged so that the label space can be reduced. Then the basic displacements obtained from the superpixel MRF are used as initial values of the pixel MRF, which effectively compresses the space of labels, thus the process on the pixel MRF can be accelerated. Furthermore, we integrate multi-frame images and previous displacements as inertial constrained information into the proposed hierarchical BP model to enhance its ability to get reliable displacements in scenes where not enough texture information can be provided. Our method performs well on accuracy and speed and obtains competitive results on MPI Sintel dataset.