论文标题
Jensen-Shannon基于贝叶斯神经网络的新型损失功能
Jensen-Shannon Divergence Based Novel Loss Functions for Bayesian Neural Networks
论文作者
论文摘要
贝叶斯神经网络（BNNS）是最先进的机器学习方法，可以使用其随机参数自然地正规化并系统地量化不确定性。 BNN中使用的基于Kullback-Leibler（KL）基于差异的变异推断，由于KL差异的无限性质，在近似光尾后期的近似尾尾的挑战中受到了不稳定的优化和挑战。为了解决这些问题，我们基于对广义詹森 - 香农（JS）差异的新修改为BNN制定了新的损失函数，该差异是有限的。此外，我们提出了基于JS差异的几何损失，这在计算上是有效的，因为可以通过分析进行评估。我们发现，基于JS差异的变异推断是棘手的，因此采用约束优化框架来制定这些损失。我们对多元回归和分类数据集的理论分析和经验实验表明，所提出的损失的性能优于基于KL差异的损失，尤其是当数据集嘈杂或有偏见时。具体而言，分别为噪声添加的CIFAR-10数据集和回归数据集的精度提高了约5％和8％。偏见的组织病理学数据集的虚假阴性预测降低了约13％。此外，我们量化和比较了回归和分类任务的不确定性指标。
Bayesian neural networks (BNNs) are state-of-the-art machine learning methods that can naturally regularize and systematically quantify uncertainties using their stochastic parameters. Kullback-Leibler (KL) divergence-based variational inference used in BNNs suffers from unstable optimization and challenges in approximating light-tailed posteriors due to the unbounded nature of the KL divergence. To resolve these issues, we formulate a novel loss function for BNNs based on a new modification to the generalized Jensen-Shannon (JS) divergence, which is bounded. In addition, we propose a Geometric JS divergence-based loss, which is computationally efficient since it can be evaluated analytically. We found that the JS divergence-based variational inference is intractable, and hence employed a constrained optimization framework to formulate these losses. Our theoretical analysis and empirical experiments on multiple regression and classification data sets suggest that the proposed losses perform better than the KL divergence-based loss, especially when the data sets are noisy or biased. Specifically, there are approximately 5% and 8% improvements in accuracy for a noise-added CIFAR-10 dataset and a regression dataset, respectively. There is about a 13% reduction in false negative predictions of a biased histopathology dataset. In addition, we quantify and compare the uncertainty metrics for the regression and classification tasks.
