论文标题
知识转移的不对称度量学习
Asymmetric metric learning for knowledge transfer
论文作者
论文摘要
最近已经研究了从大型教师模型到较小的学生模型的知识转移以进行公制学习，重点是细粒分类。在这项工作中，着眼于实例级图像检索，我们研究了一项不对称的测试任务，其中数据库由教师代表，并由学生查询。受这项任务的启发，我们引入了不对称度量学习，这是在训练中使用不对称表示的新型范式。这是知识转移与原始度量学习任务的简单组合。 我们系统地评估了新的不对称测试以及标准对称测试任务的不同教师和学生模型，指标学习和知识转移损失功能，在该任务中，数据库和查询由同一模型表示。我们发现，与更复杂的知识转移机制相比，普通回归在非对称测试方面起作用最佳。有趣的是，我们的非对称度量学习方法在对称测试中最有效，使学生甚至表现优于老师。
Knowledge transfer from large teacher models to smaller student models has recently been studied for metric learning, focusing on fine-grained classification. In this work, focusing on instance-level image retrieval, we study an asymmetric testing task, where the database is represented by the teacher and queries by the student. Inspired by this task, we introduce asymmetric metric learning, a novel paradigm of using asymmetric representations at training. This acts as a simple combination of knowledge transfer with the original metric learning task. We systematically evaluate different teacher and student models, metric learning and knowledge transfer loss functions on the new asymmetric testing as well as the standard symmetric testing task, where database and queries are represented by the same model. We find that plain regression is surprisingly effective compared to more complex knowledge transfer mechanisms, working best in asymmetric testing. Interestingly, our asymmetric metric learning approach works best in symmetric testing, allowing the student to even outperform the teacher.
