Prototype-Aware Contrastive Knowledge Distillation for Few-Shot Anomaly Detection

Knowledge distillation (KD) is widely adopted in anomaly detection but how to extend it to the few-shot setting, where a few normal samples are provided for detecting anomalies in unseen categories, has not been explored yet. To remedy this problem, we propose a novel Prototype-Aware Contrastive Knowledge Distillation (PACKD) framework. Specifically, we first design a prototype extraction and integration module (PEIM) to improve the generalization of the KD model by integrating prior information of a given category from the teacher network into the student network. The PEIM is trained to generate prototypes from few-shot normal samples to give priors and further uses them to guide the student to restore distillation targets. Subsequently, we adopt a novel contrastive distillation strategy to robustly distill both normal sample representations and inter-sample relations in the training phase. The negative and positive pairs are obtained from the feature correlations of the teacher and student. Comprehensive studies demonstrate that the proposed method outperforms the comparable few-shot methods on three benchmarks, even in more challenging cross-dataset scenarios.