面向上下位关系预测的词嵌入投影模型

A hypernymy ("is-a") relation is an important concept in the field of Natural Language Processing (NLP) and computational linguistics. This type of semantic relations is often used to describe the subordination relation between two semantic concepts, such as "(dog, animal)", "(rose, plant)" and "(sofa, furniture)". The accurate extraction and prediction of hypernymy relations from massive text corpora is extremely important for mining the inherent hierarchy among semantic concepts and entities, as well as building large-scale semantic networks, ontologies, knowledge graphs and other knowledge-intensive information systems. This task is also beneficial to a variety of downstream NLP tasks, including natural language inference, personalized recommendation, query understanding and so on. Most traditional hypernymy prediction algorithms rely on relatively fixed language patterns, such as the Hearst patterns in English. These approaches have several potential drawbacks such as the low coverage of relations in texts and the high degree of manual intervention required to train these machine learning models. In addition, the textual patterns used for hypernymy extraction are highly correlated with the characteristics of the target language itself. For languages with low regularity in text expressions such as Chinese, pattern-based methods are not sufficiently accurate. Distributional models for hypernymy prediction are more precise and can avoid the occurrence sparsity problem of concepts, but likely to suffer from the "lexical memorization" problem. With the rapid development of deep learning techniques in NLP, word embeddings which learned from neural language models are frequently employed to model the semantic relations between words, without a lot of linguistic knowledge. Especially, word embedding projection models learn how to map the embeddings of hyponyms to those of their hypernyms, modeling the representations of hypernymy relations in the embedding space explicitly. In view of existing classical and latest research, this paper introduces the development process and the latest breakthrough of word embedding projection models, in order to predict hypernymy relations accurately. We give a unified mathematical framework of these models and discuss how these models are developed, including the improvements of projection learning based on deep iterative, transductive and adversarial learning. Specifically, iterative learning methods consider the situation where hypernymy relations from different domains have diverse representations, and employ iterative, semi-supervised learning technique to learn multiple projection matrices from the embeddings of hyponyms to hypernyms. Transductive models learn the projection matrices of hypernymy and non-hypernymy relations at the same time, and consider the semantic differences between concepts in the training and testing sets. Because there are a large number of hypernymy relations in modern taxonomies, deep adversarial models learn neural network-based projection models over taxonomies and training sets, and train adversarial classifiers to make the two neural networks to learn from each other. In the experiments, we evaluate all these projection learning models under a unified framework, including multiple general-domain and domain-specific benchmark datasets in English and Chinese languages. We also compare the advantages and disadvantages of these projection learning models under different learning circumstances. Finally, the future research directions of this work are discussed, which focus on domain-specific and long-tail hypernymy prediction.