TY - GEN
T1 - Entire information attentive GRU for text representation
AU - He, Guoxiu
AU - Lu, Wei
N1 - Publisher Copyright:
© 2018 Association for Computing Machinery.
PY - 2018/9/10
Y1 - 2018/9/10
N2 - Recurrent Neural Networks (RNNs), such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), have been widely utilized in sequence representation. However, RNNs neglect variational information and long-term dependency. In this paper, we propose a new neural network structure for extracting a comprehension sequence embedding by handling the entire representation of the sequence. Unlike previous works that put attention mechanism after all steps of GRU, we add the entire representation to the input of the GRU which means the GRU model takes the entire information of the sequence into consideration in every step. We provide three various strategies to adding the entire information which are the Convolutional Neural Network (CNN) based attentive GRU (CBAG), the GRU inner attentive GRU (GIAG) and the pre-trained GRU inner attentive GRU (Pre-GIAG). To evaluate our proposed methods, we conduct extensive experiments on a benchmark sentiment classification dataset. Our experimental results show that our models outperform state-of-the-art baselines significantly.
AB - Recurrent Neural Networks (RNNs), such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), have been widely utilized in sequence representation. However, RNNs neglect variational information and long-term dependency. In this paper, we propose a new neural network structure for extracting a comprehension sequence embedding by handling the entire representation of the sequence. Unlike previous works that put attention mechanism after all steps of GRU, we add the entire representation to the input of the GRU which means the GRU model takes the entire information of the sequence into consideration in every step. We provide three various strategies to adding the entire information which are the Convolutional Neural Network (CNN) based attentive GRU (CBAG), the GRU inner attentive GRU (GIAG) and the pre-trained GRU inner attentive GRU (Pre-GIAG). To evaluate our proposed methods, we conduct extensive experiments on a benchmark sentiment classification dataset. Our experimental results show that our models outperform state-of-the-art baselines significantly.
KW - Document Representation
KW - Gated Recurrent Unit
KW - Sentiment Analysis
UR - https://www.scopus.com/pages/publications/85063423135
U2 - 10.1145/3234944.3234947
DO - 10.1145/3234944.3234947
M3 - 会议稿件
AN - SCOPUS:85063423135
T3 - ICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval
SP - 163
EP - 166
BT - ICTIR 2018 - Proceedings of the 2018 ACM SIGIR International Conference on the Theory of Information Retrieval
PB - Association for Computing Machinery, Inc
T2 - 8th ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2018
Y2 - 14 September 2018 through 17 September 2018
ER -