论文标题
马尔可夫变形金刚的级联文字生成
Cascaded Text Generation with Markov Transformers
论文作者
论文摘要
神经文本生成的两种主要方法是使用串行束搜索解码的完全自动回归模型，而非自动回形模型则使用无输出依赖性的并行解码。这项工作提出了一个具有亚线性平行时间生成的自回旋模型。指出具有有界环境的条件随机字段可以并行解码，我们提出了一种有效的级联解码方法来生成高质量输出。为了参数化此级联，我们引入了Markov Transformer，这是流行的完全自动回归模型的变体，该模型使我们可以同时用特定的自动回归上下文上下文截止。与五个机器翻译数据集中的现有方法相比，这种方法仅需要从标准自回归训练中进行的少量修改，同时表现出竞争精度/速度折衷。
The two dominant approaches to neural text generation are fully autoregressive models, using serial beam search decoding, and non-autoregressive models, using parallel decoding with no output dependencies. This work proposes an autoregressive model with sub-linear parallel time generation. Noting that conditional random fields with bounded context can be decoded in parallel, we propose an efficient cascaded decoding approach for generating high-quality output. To parameterize this cascade, we introduce a Markov transformer, a variant of the popular fully autoregressive model that allows us to simultaneously decode with specific autoregressive context cutoffs. This approach requires only a small modification from standard autoregressive training, while showing competitive accuracy/speed tradeoff compared to existing methods on five machine translation datasets.
