跳到主要导航 跳到搜索 跳到主要内容

HABERTOR: An efficient and effective deep hatespeech detector

  • Thanh Tran
  • , Yifan Hu
  • , Changwei Hu
  • , Kevin Yen
  • , Fei Tan
  • , Kyumin Lee
  • , Serim Park
  • Worcester Polytechnic Institute
  • Yahoo Research Labs
  • Twitter

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

摘要

We present our HABERTOR model for detecting hatespeech in large scale user-generated content. Inspired by the recent success of the BERT model, we propose several modifications to BERT to enhance the performance on the downstream hatespeech classification task. HABERTOR inherits BERT's architecture, but is different in four aspects: (i) it generates its own vocabularies and is pre-trained from the scratch using the largest scale hatespeech dataset; (ii) it consists of Quaternion-based factorized components, resulting in a much smaller number of parameters, faster training and inferencing, as well as less memory usage; (iii) it uses our proposed multi-source ensemble heads with a pooling layer for separate input sources, to further enhance its effectiveness; and (iv) it uses a regularized adversarial training with our proposed fine-grained and adaptive noise magnitude to enhance its robustness. Through experiments on the large-scale real-world hatespeech dataset with 1.4M annotated comments, we show that HABERTOR works better than 15 state-of-the-art hatespeech detection methods, including fine-tuning Language Models. In particular, comparing with BERT, our HABERTOR is 4∼5 times faster in the training/inferencing phase, uses less than 1/3 of the memory, and has better performance, even though we pretrain it by using less than 1% of the number of words. Our generalizability analysis shows that HABERTOR transfers well to other unseen hatespeech datasets and is a more efficient and effective alternative to BERT for the hatespeech classification.

源语言英语
主期刊名EMNLP 2020 - 2020 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference
出版商Association for Computational Linguistics (ACL)
页7486-7502
页数17
ISBN(电子版)9781952148606
出版状态已出版 - 2020
已对外发布是
活动2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020 - Virtual, Online
期限: 16 11月 2020 → 20 11月 2020

丛书

姓名EMNLP 2020 - 2020 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference

会议

会议2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020
市Virtual, Online
时期16/11/20 → 20/11/20

学术指纹

探究 'HABERTOR: An efficient and effective deep hatespeech detector' 的科研主题。它们共同构成独一无二的学术指纹。

引用此