论文标题
面具 - 主管：自我监督的神经词对齐
Mask-Align: Self-Supervised Neural Word Alignment
论文作者
论文摘要
旨在使源句子和目标句子之间的翻译等效单词保持一致的单词对齐单词在许多自然语言处理任务中起着重要作用。当前的无监督神经对准方法着重于诱导神经机器翻译模型的比对，该模型不会利用目标序列中的完整上下文。在本文中，我们提出了一个自我监督的单词对齐模型，它利用目标侧的完整上下文。我们的模型掩盖了每个目标令牌，并预测其在源和其余目标令牌上都可以进行。这个两步过程是基于以下假设：应该对齐恢复蒙版目标令牌的源代币。我们还引入了一种名为“泄漏注意力”的注意变体，该变体减轻了特殊令牌（例如时期）意外的高跨意外权重的问题。四个语言对的实验表明，我们的模型表现优于先前无监督的神经对准器，并获得了新的最新结果。
Word alignment, which aims to align translationally equivalent words between source and target sentences, plays an important role in many natural language processing tasks. Current unsupervised neural alignment methods focus on inducing alignments from neural machine translation models, which does not leverage the full context in the target sequence. In this paper, we propose Mask-Align, a self-supervised word alignment model that takes advantage of the full context on the target side. Our model masks out each target token and predicts it conditioned on both source and the remaining target tokens. This two-step process is based on the assumption that the source token contributing most to recovering the masked target token should be aligned. We also introduce an attention variant called leaky attention, which alleviates the problem of unexpected high cross-attention weights on special tokens such as periods. Experiments on four language pairs show that our model outperforms previous unsupervised neural aligners and obtains new state-of-the-art results.
