计算机科学
稳健性(进化)
人工智能
BitTorrent跟踪器
眼动
语言模型
机器学习
蒸馏
自然语言处理
感知
计算机视觉
杠杆(统计)
可视化
视觉对象识别的认知神经科学
模态(人机交互)
语言理解
目标检测
语言识别
视频跟踪
训练集
视觉感受
答疑
视觉语言
可视对象
模式识别(心理学)
数据建模
Boosting(机器学习)
对象(语法)
合成数据
作者
Guangtong Zhang,Bineng Zhong,Shirui Yang,Yang Wang,Tian Bai
标识
DOI:10.1609/aaai.v40i15.38237
摘要
Vision-language object tracking overcomes the limitations of relying solely on visual features by leveraging language descriptions of objects to provide cross-modal semantic information, thereby enhancing model robustness in complex scenarios. However, most existing high-performance vision-language trackers are trained jointly on pure visual data and vision-language multimodal data. Due to the relative sparsity of language annotations in the data, the trackers tend to prioritize the localization role of visual features, diminishing the model's attention to language information. To mitigate this issue, we propose a novel vision-language tracker: Aware Distillation for Robust Vision-Language Tracking under Linguistic Sparsity (ADTrack). We introduce a knowledge distillation framework employing a knowledge-rich teacher model and a lightweight student model to establish modality correlations between vision and language, enabling efficient modeling between visual information and language descriptions. Specifically, our lightweight student module simultaneously distills language encoding capabilities from large language models through teacher-guided learning on input language, while performing target-aware perception on template images using language descriptions to generate more effective template features for subsequent visual extraction. Furthermore, to ensure perceptual robustness in linguistically sparse scenarios, we simulate language-deficient conditions during training and employ contrastive learning to enhance model adaptability. Extensive experiments demonstrate that ADTrack reduces parameters by over 50% while achieving state-of-the-art (SOTA) performance and speed on vision-language tracking benchmarks, including LaSOT, LaSOText, TNL2K, OTB-Lang and MGIT.
科研通智能强力驱动
Strongly Powered by AbleSci AI