Robustness Analysis on Natural Language Processing Based AI Q&A Robots

Recently, the natural language processing (NLP) based intelligent question and answering (Q&A) robots have been used in a wide range of applications, such as smart assistant, smart customer service, government business. However, the robustness and security issues of these NLP based artificial intelligence (AI) Q&A robots have not been studied yet. In this paper, we analyze the robustness problems in current Q&A robots, which include four aspects: (1) semantic slot settings are incomplete; (2) sensitive words are not filtered efficiently and completely; (3) Q&A robots return the search results directly; (4) unsatisfactory matching algorithms and inappropriate matching threshold settings. Then, we design and implement two types of evaluation tests, bad language and user’s typos, to evaluate the robustness of several state-of-the-art Q&A robots. Experiment results show that these common inputs (bad language and user’s typos) can successfully make these Q&A robots malfunction, denial of service, or speaking dirty words. Besides, we also propose possible countermeasures to enhance the robustness of these Q&A robots. To the best of the authors’ knowledge, this is the first work on analyzing the robustness and security problems of intelligent Q&A robots. This work can hopefully help provide guidelines to design robust and secure Q&A robots.