论文标题
用硬性负面样本进行监督对比度学习
Supervised Contrastive Learning with Hard Negative Samples
论文作者
论文摘要
通过最小化适当的损失函数（例如Infonce损失），对比度学习（CL）通过将正面样本彼此接近，同时将负样本推向嵌入空间中远距离，从而学习了有用的表示函数。正面样本通常是使用“标记性的”增强物（即给定基准或锚定的域特异性转换）创建的。在没有类信息的情况下，在无监督的CL（UCL）中，负样品通常是从整个数据集中的预设负抽样分布中随机和独立于锚定的。这导致UCL中的课堂碰撞。监督CL（SCL），通过将负抽样分布调节到具有与锚定标签不同的样品的样品来避免此类碰撞。在Hard-UCL（H-UCL）中，已被证明是进一步增强UCL的有效方法，通过硬化功能有条件地倾斜了负采样分布，朝着更接近锚的样品。由此激励，在本文中，我们提出了硬scl（h-scl）{其中}类有条件负抽样分布{倾斜}的类别负面采样分布。 Our simulation results confirm the utility of H-SCL over SCL with significant performance gains {in downstream classification tasks.} Analytically, we show that {in the} limit of infinite negative samples per anchor and a suitable assumption, the {H-SCL loss} is upper bounded by the {H-UCL loss}, thereby justifying the utility of H-UCL {for controlling} the H-SCL loss in the absence of标签信息。通过在几个数据集上的实验，我们验证了H-UCL和H-SCL损失之间的假设以及所要求的不平等。我们还提供了一个合理的方案，其中H-SCL损失受UCL损失的限制，这表明UCL在控制H-SCL损失方面的效用有限。
Through minimization of an appropriate loss function such as the InfoNCE loss, contrastive learning (CL) learns a useful representation function by pulling positive samples close to each other while pushing negative samples far apart in the embedding space. The positive samples are typically created using "label-preserving" augmentations, i.e., domain-specific transformations of a given datum or anchor. In absence of class information, in unsupervised CL (UCL), the negative samples are typically chosen randomly and independently of the anchor from a preset negative sampling distribution over the entire dataset. This leads to class-collisions in UCL. Supervised CL (SCL), avoids this class collision by conditioning the negative sampling distribution to samples having labels different from that of the anchor. In hard-UCL (H-UCL), which has been shown to be an effective method to further enhance UCL, the negative sampling distribution is conditionally tilted, by means of a hardening function, towards samples that are closer to the anchor. Motivated by this, in this paper we propose hard-SCL (H-SCL) {wherein} the class conditional negative sampling distribution {is tilted} via a hardening function. Our simulation results confirm the utility of H-SCL over SCL with significant performance gains {in downstream classification tasks.} Analytically, we show that {in the} limit of infinite negative samples per anchor and a suitable assumption, the {H-SCL loss} is upper bounded by the {H-UCL loss}, thereby justifying the utility of H-UCL {for controlling} the H-SCL loss in the absence of label information. Through experiments on several datasets, we verify the assumption as well as the claimed inequality between H-UCL and H-SCL losses. We also provide a plausible scenario where H-SCL loss is lower bounded by UCL loss, indicating the limited utility of UCL in controlling the H-SCL loss.
