Improved CNN-based facial landmarks tracking via ridge regression at 150 Fps on mobile devices

When tracking facial landmarks in a video, existing face alignment methods seem not to be so accurate as they are employed frame by frame. This paper shows that zigzags on the trace of estimated landmarks make the estimation error perceptible. The reason why the zigzags occur is that the increment of landmark position is comparable to the estimation error and the frames are processed individually. In this paper, we train a CNN facial landmark detection model as a baseline method, and develop a post-processing algorithm to address the zigzag problem. The CNN model achieves state-of-the-art performance on the 300-W dataset. The post-processing algorithm based on ridge regression exploits correlation among adjacent frames to transform random errors into bias errors. As a result zigzags are eliminated, and the traces of landmarks look smoother while the mean error remains unchanged or even slightly decreases. Our algorithm runs on a mobile device (iPhone 5s) at 150 Fps. Extensive experiments conducted on the 300-VW dataset demonstrate the effectiveness of the proposed algorithm.