论文标题
对称网络具有基于自然语言的车辆检索的空间关系建模
Symmetric Network with Spatial Relationship Modeling for Natural Language-based Vehicle Retrieval
论文作者
论文摘要
基于自然语言（NL）的车辆检索旨在搜索给定文本描述的特定车辆。不同于基于图像的车辆检索，基于NL的车辆检索不仅需要考虑车辆外观，还需要考虑周围环境和时间关系。在本文中，我们提出了一个具有空间关系建模（SSM）方法的对称网络，用于基于NL的车辆检索。具体而言，我们设计了一个对称网络，以了解文本描述和车辆图像之间的统一跨模式表示，其中保留了车辆外观细节和车辆轨迹全球信息。此外，为了更好地利用位置信息，我们提出了一种空间关系建模方法，以考虑周围环境和相互关系的考虑。定性和定量实验验证了所提出方法的有效性。我们在基于自然语言的车辆检索轨道上的第六届AI城市挑战赛的测试集上获得了43.92％的MRR准确性，在公众排行榜上所有有效的提交中都获得了第一名。该代码可从https://github.com/hbchen121/aicity2022_track2_ssm获得。
Natural language (NL) based vehicle retrieval aims to search specific vehicle given text description. Different from the image-based vehicle retrieval, NL-based vehicle retrieval requires considering not only vehicle appearance, but also surrounding environment and temporal relations. In this paper, we propose a Symmetric Network with Spatial Relationship Modeling (SSM) method for NL-based vehicle retrieval. Specifically, we design a symmetric network to learn the unified cross-modal representations between text descriptions and vehicle images, where vehicle appearance details and vehicle trajectory global information are preserved. Besides, to make better use of location information, we propose a spatial relationship modeling methods to take surrounding environment and mutual relationship between vehicles into consideration. The qualitative and quantitative experiments verify the effectiveness of the proposed method. We achieve 43.92% MRR accuracy on the test set of the 6th AI City Challenge on natural language-based vehicle retrieval track, yielding the 1st place among all valid submissions on the public leaderboard. The code is available at https://github.com/hbchen121/AICITY2022_Track2_SSM.
