Topic Crawler for OpenStack QA Knowledge Base

With the rapid development of cloud computing, building a knowledge base on a specific cloud platform for developers and operation staff is expected to be of great value and significance. When developers and operation staff encounter problems that cannot be solved, they generally attempt to solve problems through a search engine such as Google. However, sometimes, certain domain-specific problems cannot be effectively resolved by using a generic search engine. Furthermore, the large amount of information obtained from the search engine can have a negative effect on the accuracy of the search results. To solve this problem, a topic crawler is proposed in this paper; this crawler can be used to construct a QA knowledge base for OpenStack, which is a well-known cloud platform. Firstly, a topic clustering algorithm is used to construct an OpenStack topic corpus. A vector space model is used to calculate the topic similarity for the content of the web pages, and a topic feedback model is used to update the topic corpus. Finally, we construct an OpenStack domain-specific knowledge base with the topic crawler.