Semi-Automatic Validation and Verification Framework for CV&AI-Enhanced Railway Signaling and Landmark Detector

The automation of railway operations is an activity in constant growth. Different railway stakeholders are already developing their research activities for the future driverless autonomous driving based on Computer Vision (CV) and Artificial Intelligence (AI) enhanced perception technologies (e.g., obstacle detection). Unfortunately, the AI models are opaque in nature and here is no certification accepted rules for CV&AI-enhanced functionality certification. Capturing and labelling camera image in real environment is expensive in terms of time and resources and it does not guarantee enough variation in edge visibility conditions, which makes resulting database less valuable for the V&V processes. To meet the increasing needs of trusted CV&AI-based solutions, numerous Validation and Verification (V&V) approaches have been proposed in other sectors like automotive, most of the based on virtual simulators. Unfortunately, there is currently no virtual perception simulator for railway scenario. This work aims to create an semi-automatic system based on virtual scenarios measuring the CV&AI-enhanced system performance facing different visibility conditions. It will be based on the global accuracy metrics and detected potential safety and operation rules violations. This work also demonstrates the quantitative and qualitative improvements while reducing current V&V cost.