HATDC: A holistic approach for time series data repairing

Time series data is prevalent in real life, and time series data mining is also a hot research topic nowadays. However, there may exist lots of anomalous data caused by sensor error in the real data sets, which brings difficulties for data mining. To improve the quality of data mining, it is to repair the data before data analysis. Most of the existing repairing methods use smooth-based or constraint-based techniques, but they only consider a few adjacent points and ignore global holistic information. In this paper, we propose a novel time series data repairing algorithm, named HATDC, that can exploit the holistic information of the time series. First, we use speed constraints and the probability distribution of change rates to detect the dirty data points. After that, the dynamic time warping (DTW) is applied as the distance measure to find similar subsequences in the series, and we estimate the value of these abnormal data points according to the selected similar subsequences from the whole aspect. In addition, we propose an improved algorithm for reducing the time cost based on incremental clustering. Experiments on several real datasets demonstrate that HATDC has a significantly higher repair accuracy and a lower RMS error than other methods.