论文标题
适应性地解决深度神经网络的局部最低问题
Adaptively Solving the Local-Minimum Problem for Deep Neural Networks
论文作者
论文摘要
本文旨在克服深度神经网络（DNNS）理论和应用中的基本问题。我们提出了一种直接解决训练DNN中局部最小问题的方法。我们的方法是基于跨凝结损失标准通过将跨透镜损失转换为风险避免误差（RAE）标准的凸的。为了减轻数值困难，采用了标准化的RAE（NRAE）。随着其风险敏感性指数（RSI）的增加，跨透明度损失的凸区域扩大。我们的方法可以充分利用凸区域，开始使用广泛的RSI开始训练，逐渐减少它，并在RAE在数字上可行后立即切换到RAE。训练收敛后，预计最终的深度学习机器将在横向渗透损失的全球最小值的吸引力盆地内。提供数值结果以显示所提出的方法的有效性。
This paper aims to overcome a fundamental problem in the theory and application of deep neural networks (DNNs). We propose a method to solve the local minimum problem in training DNNs directly. Our method is based on the cross-entropy loss criterion's convexification by transforming the cross-entropy loss into a risk averting error (RAE) criterion. To alleviate numerical difficulties, a normalized RAE (NRAE) is employed. The convexity region of the cross-entropy loss expands as its risk sensitivity index (RSI) increases. Making the best use of the convexity region, our method starts training with an extensive RSI, gradually reduces it, and switches to the RAE as soon as the RAE is numerically feasible. After training converges, the resultant deep learning machine is expected to be inside the attraction basin of a global minimum of the cross-entropy loss. Numerical results are provided to show the effectiveness of the proposed method.
