滨海湿地物联网观测数据预处理方法

Effective data preprocessing is essential to an online coastal wetland ecological internet of things (IoT) observation system. Outliers always occur due to the limitations of measuring methods and harsh environmental conditions, which challenge data applications. Based on the ecological observation data of Chongming Dongtan wetland in Shanghai, the outliers were divided into three types: abnormal values, abnormal fluctuation, and abnormal events. Integrating the interactions between indicators of coastal wetlands, we proposed a preprocessing method for the outliers of the coastal wetland ecological IoT system based on the residual probabilistic outlier detection algorithm, look-up table, and multi-indicator time series model. Compared with the traditional methods, this method can not only ensure the accuracy of outlier detection, but also better distinguish abnormal events from sensor problems to reduce false positives. Through the analysis of more than 50 000 data records of nine indicators, two abnormal events and 0.18%-8.12% abnormal values and abnormal fluctuations were detected with the threshold of 10^-8-10^-20. Through the analysis of the preprocessed data, we find that the observation principle and observation season will affect the stability of sensors, and the human activities in the observation area are the main factors causing abnormal events.