Isolation and Integration: A Strong Pre-trained Model-Based Paradigm for Class-Incremental Learning

Continual learning aims to effectively learn from streaming data, adapting to emerging new classes without forgetting old ones. Conventional models without pre-training are constructed from the ground up, suffering from severely catastrophic forgetting. In recent times, pre-training has made significant strides, opening the door to extensive pre-trained models for continual learning. To avoid obvious stage learning bottlenecks in traditional single-backbone networks, we propose a brand-new stage-isolation based class incremental learning framework, which leverages parameter-efficient tuning technique to finetune the pre-trained model for each task, thus mitigating information interference and conflicts among tasks. Simultaneously, it enables the effective utilization of the strong generalization capabilities inherent in pre-trained networks, which can be seamlessly adapted to new tasks. Then, we fuse the features acquired from the training of all backbone networks to construct a unified feature representation. This amalgamated representation retains the distinctive features of each task while incorporating the commonalities shared across all tasks. Finally, we use the selected exemplars to compute the prototype as the classifier weights to make final prediction. We conduct extensive experiments on different class incremental learning benchmarks and settings, results indicate that our method consistently outperforms other methods with a large margin.