Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 Challenge: Report

Andrey Ignatov; Radu Timofte; Maurizio Denna; Abdel Younes; Ganzorig Gankhuyag; Jingang Huh; Myeong Kyun Kim; Kihwan Yoon; Hyeon Cheol Moon; Seungho Lee; Yoonsik Choe; Jinwoo Jeong; Sungjei Kim; Maciej Smyl; Tomasz Latkowski; Pawel Kubik; Michal Sokolski; Yujie Ma; Jiahao Chao; Zhou Zhou; Hongfan Gao; Zhengfeng Yang; Zhenbing Zeng; Zhengyang Zhuge; Chenghua Li; Dan Zhu; Mengdi Sun; Ran Duan; Yan Gao; Lingshun Kong; Long Sun; Xiang Li; Xingdong Zhang; Jiawei Zhang; Yaqi Wu; Jinshan Pan; Gaocheng Yu; Jin Zhang; Feng Zhang; Zhe Ma; Hongbin Wang; Hojin Cho; Steve Kim; Huaen Li; Yanbo Ma; Ziwei Luo; Youwei Li; Lei Yu; Zhihong Wen; Qi Wu; Haoqiang Fan; Shuaicheng Liu; Lize Zhang; Zhikai Zong; Jeremy Kwon; Junxi Zhang; Mengyuan Li; Nianxiang Fu; Guanchen Ding; Han Zhu; Zhenzhong Chen; Gen Li; Yuanfan Zhang; Lei Sun; Dafeng Zhang; Neo Yang; Fitz Liu; Jerry Zhao; Mustafa Ayazoglu; Bahri Batuhan Bilecen; Shota Hirose; Kasidis Arunruangsirilert; Luo Ao; Ho Chun Leung; Andrew Wei; Jie Liu; Qiang Liu; Dahai Yu; Ao Li; Lei Luo; Ce Zhu; Seongmin Hong; Dongwon Park; Joonhee Lee; Byeong Hyun Lee; Seunggyu Lee; Se Young Chun; Ruiyuan He; Xuhao Jiang; Haihang Ruan; Xinjian Zhang; Jing Liu; Garas Gendy; Nabil Sabor; Jingchao Hou; Guanghui He

doi:10.1007/978-3-031-25066-8_5

Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 Challenge: Report

Andrey Ignatov, Radu Timofte, Maurizio Denna, Abdel Younes, Ganzorig Gankhuyag, Jingang Huh, Myeong Kyun Kim, Kihwan Yoon, Hyeon Cheol Moon, Seungho Lee, Yoonsik Choe, Jinwoo Jeong, Sungjei Kim, Maciej Smyl, Tomasz Latkowski, Pawel Kubik, Michal Sokolski, Yujie Ma, Jiahao Chao, Zhou ZhouHongfan Gao, Zhengfeng Yang, Zhenbing Zeng, Zhengyang Zhuge, Chenghua Li, Dan Zhu, Mengdi Sun, Ran Duan, Yan Gao, Lingshun Kong, Long Sun, Xiang Li, Xingdong Zhang, Jiawei Zhang, Yaqi Wu, Jinshan Pan, Gaocheng Yu, Jin Zhang, Feng Zhang, Zhe Ma, Hongbin Wang, Hojin Cho, Steve Kim, Huaen Li, Yanbo Ma, Ziwei Luo, Youwei Li, Lei Yu, Zhihong Wen, Qi Wu, Haoqiang Fan, Shuaicheng Liu, Lize Zhang, Zhikai Zong, Jeremy Kwon, Junxi Zhang, Mengyuan Li, Nianxiang Fu, Guanchen Ding, Han Zhu, Zhenzhong Chen, Gen Li, Yuanfan Zhang, Lei Sun, Dafeng Zhang, Neo Yang, Fitz Liu, Jerry Zhao, Mustafa Ayazoglu, Bahri Batuhan Bilecen, Shota Hirose, Kasidis Arunruangsirilert, Luo Ao, Ho Chun Leung, Andrew Wei, Jie Liu, Qiang Liu, Dahai Yu, Ao Li, Lei Luo, Ce Zhu, Seongmin Hong, Dongwon Park, Joonhee Lee, Byeong Hyun Lee, Seunggyu Lee, Se Young Chun, Ruiyuan He, Xuhao Jiang, Haihang Ruan, Xinjian Zhang, Jing Liu, Garas Gendy, Nabil Sabor, Jingchao Hou, Guanghui He

Software Engineering Institute

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

18 Scopus citations

Abstract

Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose the participants to design an efficient quantized image super-resolution solution that can demonstrate a real-time performance on mobile NPUs. The participants were provided with the DIV2K dataset and trained INT8 models to do a high-quality 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU capable of accelerating quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 60 FPS rate when reconstructing Full HD resolution images. A detailed description of all models developed in the challenge is provided in this paper.

Original language	English
Title of host publication	Computer Vision – ECCV 2022 Workshops, Proceedings
Editors	Leonid Karlinsky, Tomer Michaeli, Ko Nishino
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	92-129
Number of pages	38
ISBN (Print)	9783031250651
DOIs	https://doi.org/10.1007/978-3-031-25066-8_5
State	Published - 2023
Event	Workshops held at the 17th European Conference on Computer Vision, ECCV 2022 - Tel Aviv, Israel Duration: 23 Oct 2022 → 27 Oct 2022

Publication series

Name	Lecture Notes in Computer Science
Volume	13803 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	Workshops held at the 17th European Conference on Computer Vision, ECCV 2022
Country/Territory	Israel
City	Tel Aviv
Period	23/10/22 → 27/10/22

Keywords

AI Benchmark
Deep learning
Mobile AI
Mobile AI challenge
Mobile NPUs
Super-resolution
Synaptics

Access to Document

10.1007/978-3-031-25066-8_5

Cite this

Ignatov, A., Timofte, R., Denna, M., Younes, A., Gankhuyag, G., Huh, J., Kim, M. K., Yoon, K., Moon, H. C., Lee, S., Choe, Y., Jeong, J., Kim, S., Smyl, M., Latkowski, T., Kubik, P., Sokolski, M., Ma, Y., Chao, J., ... He, G. (2023). Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 Challenge: Report. In L. Karlinsky, T. Michaeli, & K. Nishino (Eds.), Computer Vision – ECCV 2022 Workshops, Proceedings (pp. 92-129). (Lecture Notes in Computer Science; Vol. 13803 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-25066-8_5

Ignatov, Andrey ; Timofte, Radu ; Denna, Maurizio et al. / Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 Challenge : Report. Computer Vision – ECCV 2022 Workshops, Proceedings. editor / Leonid Karlinsky ; Tomer Michaeli ; Ko Nishino. Springer Science and Business Media Deutschland GmbH, 2023. pp. 92-129 (Lecture Notes in Computer Science).

@inproceedings{6948a4598c194a72b0b06ba2f852cf29,

title = "Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI \& AIM 2022 Challenge: Report",

abstract = "Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose the participants to design an efficient quantized image super-resolution solution that can demonstrate a real-time performance on mobile NPUs. The participants were provided with the DIV2K dataset and trained INT8 models to do a high-quality 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU capable of accelerating quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 60 FPS rate when reconstructing Full HD resolution images. A detailed description of all models developed in the challenge is provided in this paper.",

keywords = "AI Benchmark, Deep learning, Mobile AI, Mobile AI challenge, Mobile NPUs, Super-resolution, Synaptics",

author = "Andrey Ignatov and Radu Timofte and Maurizio Denna and Abdel Younes and Ganzorig Gankhuyag and Jingang Huh and Kim, \{Myeong Kyun\} and Kihwan Yoon and Moon, \{Hyeon Cheol\} and Seungho Lee and Yoonsik Choe and Jinwoo Jeong and Sungjei Kim and Maciej Smyl and Tomasz Latkowski and Pawel Kubik and Michal Sokolski and Yujie Ma and Jiahao Chao and Zhou Zhou and Hongfan Gao and Zhengfeng Yang and Zhenbing Zeng and Zhengyang Zhuge and Chenghua Li and Dan Zhu and Mengdi Sun and Ran Duan and Yan Gao and Lingshun Kong and Long Sun and Xiang Li and Xingdong Zhang and Jiawei Zhang and Yaqi Wu and Jinshan Pan and Gaocheng Yu and Jin Zhang and Feng Zhang and Zhe Ma and Hongbin Wang and Hojin Cho and Steve Kim and Huaen Li and Yanbo Ma and Ziwei Luo and Youwei Li and Lei Yu and Zhihong Wen and Qi Wu and Haoqiang Fan and Shuaicheng Liu and Lize Zhang and Zhikai Zong and Jeremy Kwon and Junxi Zhang and Mengyuan Li and Nianxiang Fu and Guanchen Ding and Han Zhu and Zhenzhong Chen and Gen Li and Yuanfan Zhang and Lei Sun and Dafeng Zhang and Neo Yang and Fitz Liu and Jerry Zhao and Mustafa Ayazoglu and Bilecen, \{Bahri Batuhan\} and Shota Hirose and Kasidis Arunruangsirilert and Luo Ao and Leung, \{Ho Chun\} and Andrew Wei and Jie Liu and Qiang Liu and Dahai Yu and Ao Li and Lei Luo and Ce Zhu and Seongmin Hong and Dongwon Park and Joonhee Lee and Lee, \{Byeong Hyun\} and Seunggyu Lee and Chun, \{Se Young\} and Ruiyuan He and Xuhao Jiang and Haihang Ruan and Xinjian Zhang and Jing Liu and Garas Gendy and Nabil Sabor and Jingchao Hou and Guanghui He",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; Workshops held at the 17th European Conference on Computer Vision, ECCV 2022 ; Conference date: 23-10-2022 Through 27-10-2022",

year = "2023",

doi = "10.1007/978-3-031-25066-8\_5",

language = "英语",

isbn = "9783031250651",

series = "Lecture Notes in Computer Science",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "92--129",

editor = "Leonid Karlinsky and Tomer Michaeli and Ko Nishino",

booktitle = "Computer Vision – ECCV 2022 Workshops, Proceedings",

address = "德国",

}

Ignatov, A, Timofte, R, Denna, M, Younes, A, Gankhuyag, G, Huh, J, Kim, MK, Yoon, K, Moon, HC, Lee, S, Choe, Y, Jeong, J, Kim, S, Smyl, M, Latkowski, T, Kubik, P, Sokolski, M, Ma, Y, Chao, J, Zhou, Z, Gao, H, Yang, Z, Zeng, Z, Zhuge, Z, Li, C, Zhu, D, Sun, M, Duan, R, Gao, Y, Kong, L, Sun, L, Li, X, Zhang, X, Zhang, J, Wu, Y, Pan, J, Yu, G, Zhang, J, Zhang, F, Ma, Z, Wang, H, Cho, H, Kim, S, Li, H, Ma, Y, Luo, Z, Li, Y, Yu, L, Wen, Z, Wu, Q, Fan, H, Liu, S, Zhang, L, Zong, Z, Kwon, J, Zhang, J, Li, M, Fu, N, Ding, G, Zhu, H, Chen, Z, Li, G, Zhang, Y, Sun, L, Zhang, D, Yang, N, Liu, F, Zhao, J, Ayazoglu, M, Bilecen, BB, Hirose, S, Arunruangsirilert, K, Ao, L, Leung, HC, Wei, A, Liu, J, Liu, Q, Yu, D, Li, A, Luo, L, Zhu, C, Hong, S, Park, D, Lee, J, Lee, BH, Lee, S, Chun, SY, He, R, Jiang, X, Ruan, H, Zhang, X, Liu, J, Gendy, G, Sabor, N, Hou, J & He, G 2023, Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 Challenge: Report. in L Karlinsky, T Michaeli & K Nishino (eds), Computer Vision – ECCV 2022 Workshops, Proceedings. Lecture Notes in Computer Science, vol. 13803 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 92-129, Workshops held at the 17th European Conference on Computer Vision, ECCV 2022, Tel Aviv, Israel, 23/10/22. https://doi.org/10.1007/978-3-031-25066-8_5

Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 Challenge: Report. / Ignatov, Andrey; Timofte, Radu; Denna, Maurizio et al.
Computer Vision – ECCV 2022 Workshops, Proceedings. ed. / Leonid Karlinsky; Tomer Michaeli; Ko Nishino. Springer Science and Business Media Deutschland GmbH, 2023. p. 92-129 (Lecture Notes in Computer Science; Vol. 13803 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 Challenge

T2 - Workshops held at the 17th European Conference on Computer Vision, ECCV 2022

AU - Ignatov, Andrey

AU - Timofte, Radu

AU - Denna, Maurizio

AU - Younes, Abdel

AU - Gankhuyag, Ganzorig

AU - Huh, Jingang

AU - Kim, Myeong Kyun

AU - Yoon, Kihwan

AU - Moon, Hyeon Cheol

AU - Lee, Seungho

AU - Choe, Yoonsik

AU - Jeong, Jinwoo

AU - Kim, Sungjei

AU - Smyl, Maciej

AU - Latkowski, Tomasz

AU - Kubik, Pawel

AU - Sokolski, Michal

AU - Ma, Yujie

AU - Chao, Jiahao

AU - Zhou, Zhou

AU - Gao, Hongfan

AU - Yang, Zhengfeng

AU - Zeng, Zhenbing

AU - Zhuge, Zhengyang

AU - Li, Chenghua

AU - Zhu, Dan

AU - Sun, Mengdi

AU - Duan, Ran

AU - Gao, Yan

AU - Kong, Lingshun

AU - Sun, Long

AU - Li, Xiang

AU - Zhang, Xingdong

AU - Zhang, Jiawei

AU - Wu, Yaqi

AU - Pan, Jinshan

AU - Yu, Gaocheng

AU - Zhang, Jin

AU - Zhang, Feng

AU - Ma, Zhe

AU - Wang, Hongbin

AU - Cho, Hojin

AU - Kim, Steve

AU - Li, Huaen

AU - Ma, Yanbo

AU - Luo, Ziwei

AU - Li, Youwei

AU - Yu, Lei

AU - Wen, Zhihong

AU - Wu, Qi

AU - Fan, Haoqiang

AU - Liu, Shuaicheng

AU - Zhang, Lize

AU - Zong, Zhikai

AU - Kwon, Jeremy

AU - Zhang, Junxi

AU - Li, Mengyuan

AU - Fu, Nianxiang

AU - Ding, Guanchen

AU - Zhu, Han

AU - Chen, Zhenzhong

AU - Li, Gen

AU - Zhang, Yuanfan

AU - Sun, Lei

AU - Zhang, Dafeng

AU - Yang, Neo

AU - Liu, Fitz

AU - Zhao, Jerry

AU - Ayazoglu, Mustafa

AU - Bilecen, Bahri Batuhan

AU - Hirose, Shota

AU - Arunruangsirilert, Kasidis

AU - Ao, Luo

AU - Leung, Ho Chun

AU - Wei, Andrew

AU - Liu, Jie

AU - Liu, Qiang

AU - Yu, Dahai

AU - Li, Ao

AU - Luo, Lei

AU - Zhu, Ce

AU - Hong, Seongmin

AU - Park, Dongwon

AU - Lee, Joonhee

AU - Lee, Byeong Hyun

AU - Lee, Seunggyu

AU - Chun, Se Young

AU - He, Ruiyuan

AU - Jiang, Xuhao

AU - Ruan, Haihang

AU - Zhang, Xinjian

AU - Liu, Jing

AU - Gendy, Garas

AU - Sabor, Nabil

AU - Hou, Jingchao

AU - He, Guanghui

PY - 2023

Y1 - 2023

N2 - Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose the participants to design an efficient quantized image super-resolution solution that can demonstrate a real-time performance on mobile NPUs. The participants were provided with the DIV2K dataset and trained INT8 models to do a high-quality 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU capable of accelerating quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 60 FPS rate when reconstructing Full HD resolution images. A detailed description of all models developed in the challenge is provided in this paper.

AB - Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose the participants to design an efficient quantized image super-resolution solution that can demonstrate a real-time performance on mobile NPUs. The participants were provided with the DIV2K dataset and trained INT8 models to do a high-quality 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU capable of accelerating quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 60 FPS rate when reconstructing Full HD resolution images. A detailed description of all models developed in the challenge is provided in this paper.

KW - AI Benchmark

KW - Deep learning

KW - Mobile AI

KW - Mobile AI challenge

KW - Mobile NPUs

KW - Super-resolution

KW - Synaptics

UR - https://www.scopus.com/pages/publications/85151119704

U2 - 10.1007/978-3-031-25066-8_5

DO - 10.1007/978-3-031-25066-8_5

M3 - 会议稿件

AN - SCOPUS:85151119704

SN - 9783031250651

T3 - Lecture Notes in Computer Science

SP - 92

EP - 129

BT - Computer Vision – ECCV 2022 Workshops, Proceedings

A2 - Karlinsky, Leonid

A2 - Michaeli, Tomer

A2 - Nishino, Ko

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 23 October 2022 through 27 October 2022

ER -

Ignatov A, Timofte R, Denna M, Younes A, Gankhuyag G, Huh J et al. Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 Challenge: Report. In Karlinsky L, Michaeli T, Nishino K, editors, Computer Vision – ECCV 2022 Workshops, Proceedings. Springer Science and Business Media Deutschland GmbH. 2023. p. 92-129. (Lecture Notes in Computer Science). doi: 10.1007/978-3-031-25066-8_5