Automated Hallucination Detection for Synthetic CT Images used in MR-only Radiotherapy Workflows

Abstract Objective. AI-generated synthetic CT (sCT) images have become commercially available to provide electron densities and reference anatomies in MR-only radiotherapy workflows. However, hallucinations (false regions of bone or air) introduced in AI-generated sCT images may affect the accuracy of dose calculation and patient setup verification. We developed a tool to detect bone hallucinations and/or inaccuracies in AI-generated pelvic sCT images used in MR-only workflows. Approach. A deep learning auto segmentation (DLAS) model was trained to auto-segment bone on MR images. The model was implemented with a 3D SegResNet network architecture using the MONAI framework with a training dataset of 86 Dixon MR image sets paired with their corresponding ground truth contours derived from planning CT images deformed to the MR images. The model performance was then assessed on an independent testing dataset (n = 10). Main Results. The DLAS model-based hallucination screener identified hallucinations in bone structures using daily MR images and accurately flagged these regions on sCT images. The sensitivity of the screener is adjustable based on the distance of discrepancies between bone regions derived from sCT to bone contours generated by the DLAS. The average specificity of the DLAS model was 0.78, 0.93 and 0.98 for distance parameters of 0.8, 1.0 and 1.2 cm, respectively. The screener identified false high-density hallucination regions in the abdomen of AI-generated sCT images for all testing patients, highlighting potential issues with the training data used for the AI sCT model. Significance. A hallucination screener for AI-generated pelvic sCT images was developed and implemented for routine clinical use. The screener serves as an important quality assurance tool for MR-only radiotherapy workflows. By identifying potential AI-generated errors, the hallucination screener may improve the safety and accuracy of sCT images used for dose calculation and image guidance.