Eliciting CLIP's intrinsic attribute knowledge through a dual-cache guided mechanism for class-incremental learning

Class-Incremental Learning (CIL) is a learning paradigm where a model progressively learns new classes over time without forgetting previously learned ones. To handle this challenge task, pre-trained vision-language models such as CLIP are introduced for its strong generalizability. Despite their effectiveness, existing methods mainly adapt CLIP for CIL using global image-text representations, limiting their ability to discern detailed attributes for each class. Recent advances applied extra Large Language Model (LLM) to supplement description of class attributes, yet causing inference deficiency. In this work, we consider the internal knowledge inherently in CLIP, aiming to elicit its class attributes for CIL. Specifically, we propose a dual cache mechanism to capture and store informative local attributes and global semantics. During inference, the stored attributes are combined and fused with the text output to convey detailed information. To further mitigate catastrophic forgetting, we introduce an orthogonalization-based incremental update strategy for dual cache after each training stage. Leveraging the saved attributes, the model can emulate human learning process, alleviating stability and plasticity dilemma. Experiments on several benchmarks show that our method achieves state-of-the-art results.