Concept Drift Adaptation for Time Series Anomaly Detection via Transformer

Time series anomaly detection (TSAD) is an essential task in practical applications, such as data monitoring and network security detection. A common approach for anomaly detection is to use sequential models. As an effective sequence model, Transformer can capture the long-term dependence of the time series and is expected to better complete anomaly detection tasks. However, there are still problems to be addressed when using Transformer for anomaly detection. (1) Failing to adapt to concept drift: The vanilla Transformer assumes that the training and test data come from the same distribution. However, practical situations may often violate this assumption due to the time-varying nature of time-series data that may lead to concept drift problems. (2) High computational complexity: The time complexity of vanilla Transformer in the inference stage increase quadratically with the sequence length L. To solve the first problem, we propose the concept drift adaptation method (CDAM), a kind of distribution adaptation method, to dynamic tuning the learning rate of Transformer. CDAM aims to fully utilize the old concept data to optimize a new model on the new concept data through an online learning strategy. To address the second problem, we propose the root square sparse self-attention, which requires only O(LL) time complexity. Experimental results on several anomaly detection benchmarks show that our model outperforms many anomaly detection methods, especially in time series with concept drift.