Visual place recognition with fusion event cameras

计算机视觉 人工智能 事件(粒子物理) 融合 计算机科学 模式识别(心理学) 物理 语言学 量子力学 哲学
作者
Yichen Liu,Lei Yu,Huai Yu,Wen Yang
出处
期刊:Journal of Image and Graphics [University of Portsmouth]
卷期号:29 (4): 1018-1029 被引量:1
标识
DOI:10.11834/jig.230003
摘要

目的 传统视觉场景识别(visual place recognition,VPR)算法的性能依赖光学图像的成像质量,因此高速和高动态范围场景导致的图像质量下降会进一步影响视觉场景识别算法的性能。针对此问题,提出一种融合事件相机的视觉场景识别算法,利用事件相机的低延时和高动态范围的特性,提升视觉场景识别算法在高速和高动态范围等极端场景下的识别性能。方法 本文提出的方法首先使用图像特征提取模块提取质量良好的参考图像的特征,然后使用多模态特征融合模块提取查询图像及其曝光区间事件信息的多模态融合特征,最后通过特征匹配查找与查询图像最相似的参考图像。结果 在MVSEC(multi-vehicle stereo event camera dataset)和RobotCar两个数据集上的实验表明,本文方法对比现有视觉场景识别算法在高速和高动态范围场景下具有明显优势。在高速高动态范围场景下,本文方法在MVSEC数据集上相较对比算法最优值在召回率与精度上分别提升5.39%和8.55%,在Robot‐Car数据集上相较对比算法最优值在召回率与精度上分别提升3.36%与4.41%。结论 本文提出了融合事件相机的视觉场景识别算法,利用了事件相机在高速和高动态范围场景的成像优势,有效提升了视觉场景识别算法在高速和高动态范围场景下的场景识别性能。;Objective The performance of traditional visual place recognition(VPR)algorithms depends on the imaging quality of optical images.However,optical cameras suffer from low temporal resolution and dynamic range.For example,in a scene with high-speed motion,continuously capturing the rapid changes in the position of the scene in the imaging plane is difficult for an optical camera,resulting in motion blur in the output image.When the scene brightness exceeds the recording range of the photosensitive chip of the camera,output image degradation of the optical camera such as underexposure and overexposure may occur.The blurring,underexposure,and overexposure of optical images will lead to the loss of image texture structure information,which will result in the performance reduction of visual scene recognition algorithms.Therefore,the recognition performance of image-based VPR algorithms is poor in high-speed and high dynamic range(HDR)scenarios.Event camera is a new type of visual sensor inspired by biological vision.This camera has the characteristics of low latency and HDR.Using event cameras can effectively improve the recognition performance of VPR algorithms in high-speed and HDR scenes.Therefore,this paper proposes a VPR algorithm fused with event cameras,which utilizes the low latency and HDR characteristics of event cameras to improve the recognition performance of VPR algorithms in extreme scenarios such as high speed and HDR.Method The proposed method first fuses the information of the query image and the events within its exposure time interval to obtain the multimodal features of the query location.The method then retrieves the reference image closest to the multimodal features of the query location in the reference image database.This method also extracts the features of the reference image with good quality using the image feature extraction module and then inputs query image and its events within the exposure time interval to the multimodal to compare the multimodal query information with the reference image.Multimodal fusion features are obtained by the multimodal feature fusion module,and the reference image most similar to the query image is finally obtained through feature matching retrieval,thereby completing visual scene recognition.The network training is supervised by a triplet loss.The triplet loss drives the network to learn in the direction where the vector distance between the query and positive features is smaller,and the vector distance between the negative feature is larger,until the difference between the negative distance and the positive distance is not less than the similarity distance constant.Therefore,distinguishing reference images with similar and different fields of view from the query image according to the similarity in the feature vector space is possible,further completing the VPR task.Result The experiments are conducted on the MVSEC and RobotCar datasets.The proposed method is compared in experiments with image-based method,event camera-based method,and methods that utilize image and event camera information.Under different exposure and high-speed scenarios,the proposed method has advantages over existing visual scene recognition algorithms.Specifically,on the MVSEC dataset,the proposed method can reach a maximum recall rate of 99.36%and a maximum recognition accuracy of 96.34%,which improves the recall rate and precision by 5.39% and 8.55%,respectively,compared with the existing VPR methods.On the RobotCar dataset,the proposed method can reach a maximum recall rate of 97.33%and a maximum recognition accuracy of 93.30%,which improves the recall rate and precision by 3.36% and 4.41%,respectively,compared with the existing VPR methods.Experimental results show that in the high-speed and HDR scene,the proposed method has advantages compared with the existing VPR algorithm and demonstrates a remarkable improvement in the recognition performance.Conclusion This paper proposes a VPR algorithm that fuses event cameras,which utilizes the characteristics of low latency and HDR of event cameras and overcomes the problem of image information loss in high-speed and HDR scenes.This method effectively fuses information from image and event modalities,thereby improving the performance of VPR in high-speed and HDR scenarios.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
东风知我欲完成签到,获得积分10
1秒前
Sodagreen2023完成签到 ,获得积分10
1秒前
人类完成签到 ,获得积分10
2秒前
汐儿完成签到 ,获得积分10
2秒前
Yiwen完成签到,获得积分10
2秒前
3秒前
Cathy完成签到,获得积分10
3秒前
ccc发布了新的文献求助10
3秒前
中科路2020发布了新的文献求助10
3秒前
amigo完成签到 ,获得积分10
4秒前
占那个完成签到 ,获得积分10
4秒前
4秒前
6秒前
金陵小帅完成签到,获得积分10
6秒前
上官若男的应助被sheep采纳,获得10
7秒前
Chase完成签到,获得积分10
7秒前
guo完成签到,获得积分10
8秒前
Lisa完成签到,获得积分10
10秒前
11秒前
liZZZZZ完成签到,获得积分10
13秒前
田志超完成签到,获得积分10
13秒前
Fairy完成签到,获得积分10
14秒前
周鑫完成签到,获得积分10
16秒前
吕程校完成签到 ,获得积分10
17秒前
WXR完成签到,获得积分10
18秒前
知识进脑子吧完成签到 ,获得积分10
18秒前
xudaniel完成签到,获得积分10
19秒前
zz完成签到,获得积分10
21秒前
中科路2020完成签到,获得积分10
22秒前
高高的电源完成签到,获得积分10
22秒前
23秒前
放青松完成签到 ,获得积分10
24秒前
橘子海完成签到 ,获得积分10
24秒前
lulala完成签到,获得积分10
25秒前
YoungLee完成签到 ,获得积分10
26秒前
wdufhgk发布了新的文献求助10
26秒前
KisaragiSabrina完成签到,获得积分10
27秒前
cdercder的应助被小小木采纳,获得10
27秒前
27秒前
manman完成签到 ,获得积分10
28秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
The World Atlas of Wine 9th edition 1000
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
八旗制度の研究 600
高分子基功能界面水蒸发材料与应用研究 500
外场增强的水汽发电研究 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8185571
求助须知:如何正确求助?哪些是违规求助? 9574290
关于积分的说明 21834928
捐赠科研通 7783274
什么是DOI,文献DOI怎么找? 3434577
关于科研通互助平台的介绍 2558973
邀请新用户注册赠送积分活动 2481370