Privacy Leakage in Privacy-Preserving Neural Network Inference

The community has seen many attempts to secure machine learning algorithms from multi-party computation or other cryptographic primitives. An interesting 3-party framework (SCSDF hereafter) for privacy-preserving neural network inference was presented at ESORICS 2020. SCSDF defines several protocols for non-linear activation functions including ReLU, Sigmoid, etc. In particular, these protocols reckon on a protocol DReLU (derivative computation for ReLU function) they proposed as a building block. All protocols are claimed secure (against one single semi-honest corruption and against one malicious corruption). Unfortunately, the paper shows that there exists grievous privacy leakage of private inputs during SCSDF executions. This would completely destroy the framework security. We first give detailed cryptanalysis on SCSDF from the perspective of the real-ideal simulation paradigm and indicate that these claimed-secure protocols do not meet the underlying security model. We then go into particular steps in SCSDF and demonstrate that the signs of input data would be inevitably revealed to the (either semi-honest or malicious) third party responsible for assisting protocol executions. To show such leakage more explicitly, we perform plenteous experiment evaluations on the MNIST dataset, the CIFAR-10 dataset, and CFD (Chicago Face Database) for both ReLU and Sigmoid non-linear activation functions. All experiments succeed in disclosing original private data of the data owner in the inference process. Potential countermeasures are recommended and demonstrated as well.