High-Quality Triggers Based Fragile Watermarking for Optical Character Recognition Model

Deep neural networks have demonstrated exceptional performance in numerous domains, yet they also face significant security issues. To safeguard the integrity of deep neural networks and prevent unauthorized parameter manipulation, researchers have proposed fragile model watermarking techniques. However, current methods concentrate on image classification models and are not suitable for text image recognition models. Consequently, this paper proposes a remote black-box integrity authentication method for text image recognition models. Specifically, we embed a trigger set into the model to be protected as its watermark. This trigger set is carefully constructed by adding tiny noise to meticulously selected training set samples, with each trigger set sample labeled using model-specific sentence. Utilizing the principle of information entropy theory, we employ an optimal approach to fine-tuning the model, aiming to ensure that the outputs of the trigger set exhibit a pronounced sensitivity to variations within the model. Experiments indicate that with the structural similarity between the trigger set samples and the original samples exceeding 0.9, the embedded watermark has a negligible impact on model performance. The minimum decrease in accuracy on the test set is less than 0.5%. Moreover, even after undergoing modifications limited to fine-tuning only the last two layers and altering a mere one ten-thousandth of the parameters, the model watermarking is effective in identifying such changes.