论文标题
基于神经语言模型的词汇替代方法的比较研究
A Comparative Study of Lexical Substitution Approaches based on Neural Language Models
论文作者
论文摘要
Lexical substitution in context is an extremely powerful technology that can be used as a backbone of various NLP applications, such as word sense induction, lexical relation extraction, data augmentation, etc. In this paper, we present a large-scale comparative study of popular neural language and masked language models (LMs and MLMs), such as context2vec, ELMo, BERT, XLNet, applied to the task of lexical substitution.我们表明，如果正确注入有关目标单词的信息，可以进一步改进SOTA LMS/MLMS所取得的竞争性结果，并比较几种目标注入方法。此外，我们还提供了不同模型产生的目标与替代品之间的语义关系类型的分析，从而提供了有关替代者真正生成或给出哪种单词的见解。
Lexical substitution in context is an extremely powerful technology that can be used as a backbone of various NLP applications, such as word sense induction, lexical relation extraction, data augmentation, etc. In this paper, we present a large-scale comparative study of popular neural language and masked language models (LMs and MLMs), such as context2vec, ELMo, BERT, XLNet, applied to the task of lexical substitution. We show that already competitive results achieved by SOTA LMs/MLMs can be further improved if information about the target word is injected properly, and compare several target injection methods. In addition, we provide analysis of the types of semantic relations between the target and substitutes generated by different models providing insights into what kind of words are really generated or given by annotators as substitutes.
