论文标题
在多方对话中学习层次背景的变压器以基于跨度的问题回答
Transformers to Learn Hierarchical Contexts in Multiparty Dialogue for Span-based Question Answering
论文作者
论文摘要
我们为变压器介绍了一种新颖的方法，该方法在多方对话中学习了层次的表示。首先，使用三个语言建模任务来预先培训变压器，令牌和话语级的语言建模和话语顺序预测，这些预测同时学习令牌和话语嵌入，以在对话环境中更好地理解。然后，将话语预测和令牌跨度预测之间的多任务学习应用于基于跨度的问题回答（QA）的微调。我们的方法在FriendsQA数据集上进行了评估，并显示出比两种最先进的变压器模型分别提高了3.8％和1.4％。
We introduce a novel approach to transformers that learns hierarchical representations in multiparty dialogue. First, three language modeling tasks are used to pre-train the transformers, token- and utterance-level language modeling and utterance order prediction, that learn both token and utterance embeddings for better understanding in dialogue contexts. Then, multi-task learning between the utterance prediction and the token span prediction is applied to fine-tune for span-based question answering (QA). Our approach is evaluated on the FriendsQA dataset and shows improvements of 3.8% and 1.4% over the two state-of-the-art transformer models, BERT and RoBERTa, respectively.
