论文标题
通过混凝土和抽象信息从图像中产生中国诗歌
Generating Chinese Poetry from Images via Concrete and Abstract Information
论文作者
论文摘要
近年来，自动一代古典中国诗歌取得了长足的进步。除了专注于提高生成的诗歌的质量外，还有一个关于从图像中产生诗歌的新话题。但是，该主题的现有方法仍然存在主题漂移和语义不一致的问题，并且在训练这些模型时，很难构建图像诗对数据集。在本文中，我们从图像中提取和集成了具体和抽象信息以解决这些问题。我们提出了一个基于填充的中国诗歌生成模型，该模型可以以明确的方式将具体的关键字填充到每一首诗中，并嵌入一个抽象信息，以将抽象信息整合到生成的诗中。此外，我们在培训期间使用非并行数据，并构建单独的图像数据集和诗数据集来训练框架中的不同组件。自动和人类评估结果都表明，我们的方法可以产生与图像具有更好一致性的诗，而不会失去质量。
In recent years, the automatic generation of classical Chinese poetry has made great progress. Besides focusing on improving the quality of the generated poetry, there is a new topic about generating poetry from an image. However, the existing methods for this topic still have the problem of topic drift and semantic inconsistency, and the image-poem pairs dataset is hard to be built when training these models. In this paper, we extract and integrate the Concrete and Abstract information from images to address those issues. We proposed an infilling-based Chinese poetry generation model which can infill the Concrete keywords into each line of poems in an explicit way, and an abstract information embedding to integrate the Abstract information into generated poems. In addition, we use non-parallel data during training and construct separate image datasets and poem datasets to train the different components in our framework. Both automatic and human evaluation results show that our approach can generate poems which have better consistency with images without losing the quality.
