论文标题
关于不变学习与对抗性培训之间的联系
On the Connection between Invariant Learning and Adversarial Training for Out-of-Distribution Generalization
论文作者
论文摘要
尽管在许多任务中取得了令人印象深刻的成功，但深度学习模型被证明依赖于虚假特征，在概括到分发数据（OOD）数据时，灾难性失败。提出了不变风险最小化（IRM），以通过提取域名特征以进行OOD概括来减轻此问题。然而，最近的工作表明，IRM仅对某种类型的分配变化有效（例如，相关转移），而对于其他情况（例如，多样性转移）失败了。同时，方法的另一个线程，对抗训练（AT），显示出更好的域传输性能，表明它有可能成为提取域不变特征的有效候选者。本文通过探索IRM和目标之间的相似性来调查这种可能性。受此连接的启发，我们提出了域名对抗训练（DAT），这是一种通过特异性扰动来减轻分布转移的启发方法。广泛的实验表明，我们提出的DAT可以有效地消除与域变化的特征，并在相关转移和多样性转移下改善OOD的概括。
Despite impressive success in many tasks, deep learning models are shown to rely on spurious features, which will catastrophically fail when generalized to out-of-distribution (OOD) data. Invariant Risk Minimization (IRM) is proposed to alleviate this issue by extracting domain-invariant features for OOD generalization. Nevertheless, recent work shows that IRM is only effective for a certain type of distribution shift (e.g., correlation shift) while it fails for other cases (e.g., diversity shift). Meanwhile, another thread of method, Adversarial Training (AT), has shown better domain transfer performance, suggesting that it has the potential to be an effective candidate for extracting domain-invariant features. This paper investigates this possibility by exploring the similarity between the IRM and AT objectives. Inspired by this connection, we propose Domainwise Adversarial Training (DAT), an AT-inspired method for alleviating distribution shift by domain-specific perturbations. Extensive experiments show that our proposed DAT can effectively remove domain-varying features and improve OOD generalization under both correlation shift and diversity shift.
