FrePrompter: Frequency self-prompt for all-in-one image restoration

Deep learning-based models have shown unprecedented success in image restoration. However, existing methods are limited to task-specific restoration, where the model performance is undesirable when the type of degradation changes. This is due to the inconsistency between the actual situation and the priori adopted for model construction. In this paper, we propose a novel prompt learning method called Frequency Self-Prompt (FSP) customized for image restoration. Motivated by the frequency properties, FSP utilizes the degradation information of the input image to generate frequency prompts that dynamically guide the restoration network in removing the corresponding corruption. On the one hand, the frequency representation can disentangle image degradation and content components, which makes learning the degradation information more effective. On the other hand, the frequency domain naturally encodes the globally distributed degradation-specific information. We exploit FSP to build a universal model for all-in-one image restoration, called FrePrompter, which can be applied to various restoration tasks without any prior knowledge of degradation. Extensive experiments show that our method establishes new state-of-the-art results for various restoration tasks.