论文标题
梯度的最近基于邻居的估计值：尖锐的非扰动边界和应用
Nearest Neighbour Based Estimates of Gradients: Sharp Nonasymptotic Bounds and Applications
论文作者
论文摘要
从各种应用中进行的激励，从随机优化到降低尺寸，通过可变选择，准确估算梯度的问题在统计和学习理论中至关重要。我们在这里考虑经典的回归设置，其中真正有价值的正方形集成R.V. $ y $将在观察一个（可能高维的）随机矢量$ x $的情况下，通过预测功能$ f（x）$在于点的意义上尽可能准确，并研究基于最接近的neighbour的最佳预测功能的梯度，回归函数$ m（x）= \ mathbb {e e}在经典的平滑度条件下，再加上$ y-m（x）$的尾巴是次高斯的假设，我们证明非扰动界限会改善替代估计方法所获得的范围。除了建立了新的理论结果外，还进行了一些说明性的数值实验。后者提供了有力的经验证据，表明估计方法提出的效果非常适合涉及梯度估计的各种统计问题，即降低维度，随机梯度下降优化和量化分解。
Motivated by a wide variety of applications, ranging from stochastic optimization to dimension reduction through variable selection, the problem of estimating gradients accurately is of crucial importance in statistics and learning theory. We consider here the classic regression setup, where a real valued square integrable r.v. $Y$ is to be predicted upon observing a (possibly high dimensional) random vector $X$ by means of a predictive function $f(X)$ as accurately as possible in the mean-squared sense and study a nearest-neighbour-based pointwise estimate of the gradient of the optimal predictive function, the regression function $m(x)=\mathbb{E}[Y\mid X=x]$. Under classic smoothness conditions combined with the assumption that the tails of $Y-m(X)$ are sub-Gaussian, we prove nonasymptotic bounds improving upon those obtained for alternative estimation methods. Beyond the novel theoretical results established, several illustrative numerical experiments have been carried out. The latter provide strong empirical evidence that the estimation method proposed works very well for various statistical problems involving gradient estimation, namely dimensionality reduction, stochastic gradient descent optimization and quantifying disentanglement.
