Saliency guided depth prediction from a single image

With the recent surge of deep neural networks, depth prediction from a single image has seen substantial progress. Deep regression networks are typically learned from large data without much constraints about the scene structure, thus often leading to uncertainties at discontinuous regions. In this paper, we propose a structure-aware depth prediction method based on two observations: depth is relatively smooth within the same objects, and it is usually easier to model relative depth than model the absolute depth from scratch. Our network first predicts an initial depth map and takes an object saliency map as input, which helps to teach the network to learn depth refinement. Specifically, a stable anchor depth is first estimated from the detected salient objects, and the learning objective is to penalize the difference in relative depth versus the estimated anchor. We show such saliency-guided relative depth constraint unveils helpful scene structures, leading to significant gains on the RGB-D saliency dataset NLPR and depth prediction dataset NYU V2. Furthermore, our method is appealing in that it is pluggable to any depth network and is trained end-to-end with no overhead of time during testing.