Secure Mutual Learning with Low Interactions for Deep Model Training

The paper proposes SMuLe, a secure mutual learning protocol for two-party deep model training, to support low demand for interaction complexity and communication overhead. The strategy is that two parties exchange their blind predictions securely on each other's dataset and the underlying models can thereby benefit from not only the true label of the data but the prediction of the other party, as being different from federated learning and prior art that counts on secure multi-party computation. After this protocol, each participant occupies a well trained plaintext model privately. The contributions include the following: (i) the communication cost of SMuLe is lower than state-of-the-art two-party training protocols; (ii) our solution is flexible with different secure inference schemes; (iii) SMuLe can resist malicious attacks through poisoning samples. The experiments show that on CifarlO, SMuLe can obtain desirable accuracy even in a small convnet and the communication cost in each epoch is less than 75 MB as expected.