论文标题
从失败中学习目标
Learning Goals from Failure
论文作者
论文摘要
我们介绍了一个框架，可以预测视频中可观察到的人类行动背后的目标。在发育心理学中的证据中，我们利用无意采取行动的视频来学习目标的视频表示，而无需直接监督。我们的方法模型视频是代表低级运动和高级动作功能的上下文轨迹。实验和可视化表明，我们训练有素的模型能够预测无意采取行动视频中的基本目标。我们还提出了一种方法，通过利用模型的梯度信号来调整潜在轨迹，以“自动纠正”无意的动作。尽管该模型受到最低限度的监督培训，但它具有在成功执行的大型（有监督的）数据集中训练的基线或胜过基线，这表明观察到无意识的行动对于学习视频中的目标至关重要。项目页面：https：//aha.cs.columbia.edu/
We introduce a framework that predicts the goals behind observable human action in video. Motivated by evidence in developmental psychology, we leverage video of unintentional action to learn video representations of goals without direct supervision. Our approach models videos as contextual trajectories that represent both low-level motion and high-level action features. Experiments and visualizations show our trained model is able to predict the underlying goals in video of unintentional action. We also propose a method to "automatically correct" unintentional action by leveraging gradient signals of our model to adjust latent trajectories. Although the model is trained with minimal supervision, it is competitive with or outperforms baselines trained on large (supervised) datasets of successfully executed goals, showing that observing unintentional action is crucial to learning about goals in video. Project page: https://aha.cs.columbia.edu/
