FedCo: Self-Supervised Learning in Federated Learning with Momentum Contrast

Federated learning(FL) enables multiple partici-pants to build a common, robust machine learning model without sharing data, which is a key technique to address data privacy, security, access rights, and access to heterogeneous data. However, existing FL algorithms still fall short of expectations in unsupervised learning and Non-iid data, which are two major challenges that limit the applicability and accuracy. For example, the lack of data labels or skewed feature distributions leads to loss of representativeness of the model, at the same time, tagging client data will increase a lot of costs in real application scenarios, therefore, supervised federated learning greatly limits the applicability of FL. In this paper, we propose a new unsupervised FL algorithm FedCO. In particular, at the parameter aggregation phase, we apply the momentum update queue to improve the training performance, enhance the model accuracy, and lower the labeling costs. At the local client training phase, the common queue that could be regarded as a big dictionary solve the data heterogeneity. The experiment on the benchmark data set shows that our method can be compared to other supervised and semi-supervised federated learning models, which proves the effectiveness of FedCo.