论文标题
提高质量检查模型的鲁棒性，以挑战各种问题 - 答案对生成
Improving the Robustness of QA Models to Challenge Sets with Variational Question-Answer Pair Generation
论文作者
论文摘要
用于阅读理解的问题回答（QA）模型已在分布测试集上实现了人级的准确性。但是，它们被证明缺乏挑战集的稳健性，其分布与培训集的分布不同。现有的数据增强方法通过简单地通过与挑战集相同的分布相同的合成示例来增加训练集来减轻此问题。但是，这些方法假定挑战集的分布是先验的，这使得它们不适用于看不见的挑战集。在这项研究中，我们专注于提问对生成（QAG）来减轻此问题。尽管大多数现有的QAG方法旨在提高合成示例的质量，但我们猜想促进多样性的QAG可以减轻训练集的稀疏性并带来更好的鲁棒性。我们提出了一个差异QAG模型，该模型从段落中生成了多种不同的QA对。我们的实验表明，我们的方法可以提高12个挑战集的准确性以及分布精度。我们的代码和数据可在https://github.com/kazutoshishinoda/vqag上找到。
Question answering (QA) models for reading comprehension have achieved human-level accuracy on in-distribution test sets. However, they have been demonstrated to lack robustness to challenge sets, whose distribution is different from that of training sets. Existing data augmentation methods mitigate this problem by simply augmenting training sets with synthetic examples sampled from the same distribution as the challenge sets. However, these methods assume that the distribution of a challenge set is known a priori, making them less applicable to unseen challenge sets. In this study, we focus on question-answer pair generation (QAG) to mitigate this problem. While most existing QAG methods aim to improve the quality of synthetic examples, we conjecture that diversity-promoting QAG can mitigate the sparsity of training sets and lead to better robustness. We present a variational QAG model that generates multiple diverse QA pairs from a paragraph. Our experiments show that our method can improve the accuracy of 12 challenge sets, as well as the in-distribution accuracy. Our code and data are available at https://github.com/KazutoshiShinoda/VQAG.
