Privacy-Preserving Locally Weighted Linear Regression over Encrypted Millions of Data

The emerging development of cloud computing makes a trend that the cloud becomes a outsourced agglomeration for storing big data that generally contains numerous information. To mine the rich value involved in big data, the machine learning methodology is widespread employed due to its ability to adapt to data changes. However, the data mining process may involve the privacy issues of the users, hence they are reluctant to share their information. This is the reason why the outsourced data need to be dealt with securely, where data encryption is considered to be the most straightforward method to keep the privacy of data, but machine learning on the data in ciphertext domain is more complicated than the plaintext, since the relationship structure between data is no longer maintained, in such a way that we focus on the machine learning over encrypted big data. In this work, we study locally weighted linear regression (LWLR), a widely used classic machine learning algorithm in real-world, such as predict and find the best-fit curve through numerous data points. To tackle the privacy concerns in utilizing the LWLR algorithm, we present a system for privacy-preserving locally weighted linear regression, where the system not only protects the privacy of users but also encrypts the best-fit curve. Therefore, we use Paillier homomorphic encryption as the building modular to encrypt data and then apply the stochastic gradient descent in encrypted domain. After given a security analysis, we study how to let Paillier encryption deal with real numbers and implement the system in Python language with a couple of experiments on real-world data sets to evaluate the effectiveness, and show that it outperforms the state-of-the-art and occurs negligible errors compared with performing locally weighted linear regression in the clear.