TY - GEN
T1 - Improving Optical Flow Inference for Video Colorization
AU - Huang, Rulin
AU - Li, Shaohui
AU - Dai, Wenrui
AU - Li, Chenglin
AU - Zou, Junni
AU - Xiong, Hongkai
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - Recent video colorization methods optimize correspondence estimation and information propagation in an end-to-end manner. However, they usually suffer from loss of fidelity due to the inaccurate inference of correspondence measurement. In this paper, we propose a post-training optimization (PTO) strategy to refine correspondence measurement in the end-to-end optimized framework. The proposed PTO strategy introduces a pseudo loss function to well approximate the target loss and guide the direction of updates. We further develop a video colorization method that incorporates PTO and optical flow to guarantee high-fidelity colorized frames in theory. Experimental results demonstrate that the proposed method achieves state-of-the-art PSNR performance in video colorization on the DAVIS dataset and common test sequences for video coding. Furthermore, the proposed method can be employed into video compression and achieves competitive rate-distortion performance with the recent High Efficiency Video Coding (HEVC) standard.
AB - Recent video colorization methods optimize correspondence estimation and information propagation in an end-to-end manner. However, they usually suffer from loss of fidelity due to the inaccurate inference of correspondence measurement. In this paper, we propose a post-training optimization (PTO) strategy to refine correspondence measurement in the end-to-end optimized framework. The proposed PTO strategy introduces a pseudo loss function to well approximate the target loss and guide the direction of updates. We further develop a video colorization method that incorporates PTO and optical flow to guarantee high-fidelity colorized frames in theory. Experimental results demonstrate that the proposed method achieves state-of-the-art PSNR performance in video colorization on the DAVIS dataset and common test sequences for video coding. Furthermore, the proposed method can be employed into video compression and achieves competitive rate-distortion performance with the recent High Efficiency Video Coding (HEVC) standard.
UR - https://www.scopus.com/pages/publications/85142468736
U2 - 10.1109/ISCAS48785.2022.9937932
DO - 10.1109/ISCAS48785.2022.9937932
M3 - 会议稿件
AN - SCOPUS:85142468736
T3 - Proceedings - IEEE International Symposium on Circuits and Systems
SP - 3185
EP - 3189
BT - IEEE International Symposium on Circuits and Systems, ISCAS 2022
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2022 IEEE International Symposium on Circuits and Systems, ISCAS 2022
Y2 - 27 May 2022 through 1 June 2022
ER -