论文标题
表征和避免有问题的变异自动编码器的全球最佳
Characterizing and Avoiding Problematic Global Optima of Variational Autoencoders
论文作者
论文摘要
变量自动编码器（VAE）是由两个组件组成的深生成潜在变量模型：一种生成模型，该模型通过在潜在空间上转换一个分布p（z）来捕获数据分布p（x），以及一种推断模型，并且可以吸收每个数据点（Kingma and Welling，2013年）可能会吸收潜在的潜在代码。最近的工作表明，传统的培训方法倾向于产生违反建模的Desiderata的解决方案：（1）学习的生成模型捕获了观察到的数据分布，但在忽略潜在代码的同时这样做，从而导致代码不代表数据（例如Van Den Oord等人（2017）； Kim等人（2018）（2018））; （2）学习的潜在代码的汇总与先前的p（z）不匹配。这种不匹配意味着学习的生成模型将无法使用P（z）的样品生成逼真的数据（例如Makhzani等人（2015）； Tomczak和Welling（2017））。在本文中，我们证明了这两个问题源于以下事实：VAE培训目标的全球最佳选择通常与不良的解决方案相对应。我们的分析以两个观察结果为基础：（1）生成模型无法识别 - 存在许多生成模型，这些模型可以很好地解释数据，每个模型都具有不同的（并且可能有害的）特性，并且（2）VAE目标中的偏见 - VAE目标可能更喜欢生成模型，这些模型可能会解释数据，但可以很好地解释数据，但具有易于大致近似的后代。我们提出了一种新颖的推理方法，利比（Libi）减轻了分析中发现的问题。在合成数据集上，我们表明Libi可以学习生成模型，以捕获数据分布和推理模型，这些模型在传统方法难以这样做时可以更好地满足建模假设。
Variational Auto-encoders (VAEs) are deep generative latent variable models consisting of two components: a generative model that captures a data distribution p(x) by transforming a distribution p(z) over latent space, and an inference model that infers likely latent codes for each data point (Kingma and Welling, 2013). Recent work shows that traditional training methods tend to yield solutions that violate modeling desiderata: (1) the learned generative model captures the observed data distribution but does so while ignoring the latent codes, resulting in codes that do not represent the data (e.g. van den Oord et al. (2017); Kim et al. (2018)); (2) the aggregate of the learned latent codes does not match the prior p(z). This mismatch means that the learned generative model will be unable to generate realistic data with samples from p(z)(e.g. Makhzani et al. (2015); Tomczak and Welling (2017)). In this paper, we demonstrate that both issues stem from the fact that the global optima of the VAE training objective often correspond to undesirable solutions. Our analysis builds on two observations: (1) the generative model is unidentifiable - there exist many generative models that explain the data equally well, each with different (and potentially unwanted) properties and (2) bias in the VAE objective - the VAE objective may prefer generative models that explain the data poorly but have posteriors that are easy to approximate. We present a novel inference method, LiBI, mitigating the problems identified in our analysis. On synthetic datasets, we show that LiBI can learn generative models that capture the data distribution and inference models that better satisfy modeling assumptions when traditional methods struggle to do so.
