Towards reliable cardiac image segmentation: Assessing image-level and pixel-level segmentation quality via self-reflective references

• An effective quality assessment framework on cardiac segmentations for reliable computer-aided cardiovascular diagnosis. • Leverages self-reflective references to expose segmentation failures. • The semantic class-aware compactness constrain is exploited for more effective segmentation quality assessment. • State-of-the-art performance on two public datasets in both pixel-level and image-level segmentation quality assessment. • Robust performance on the segmentations with diverse qualities, including low-quality, medium-quality and high-quality ones. Cardiac image segmentation is a fundamental step in cardiovascular disease diagnosis, where many deep learning models have achieved promising performance. However, when deploying these well-trained models for real clinical usage, the network will inevitably produce inferior results due to domain shifts, motion artifacts, etc. How to avoid the potential poor-quality segmentations involved in clinical decision making is crucial for reliable computer-aided cardiac disease diagnosis. To this end, we develop a quality control method to identify failure segmentations by measuring their qualities, and report them to physicians for professional opinions. In specific, we propose a reference-based framework to assess the image-level quality ( i.e. per-class Dice) for overall evaluation and pixel-level quality ( i.e. pixel-wise correct map) to locate mis-segmented regions. Following previous works, we create informative references first, and investigate their relative relationships ( e.g. differences) to the inputs to expose segmentation failures. However, we generate and leverage the references in different ways. We instantiate the references by recovering input images from segmentations by a self-reflective reference generator. If the segmentation is of good quality, the reference ( i.e. the reconstructed image) will be close to the input image, and the inconsistency between them would be a good indicator to deduce the qualities. To effectively explore these inconsistency, we employ a difference investigator equipped with semantic class-aware compactness constraint to force the correctly-segmented features more separable to the wrongly-segmented ones. The experiments on ACDC and MSCMR datasets demonstrated our method could effectively capture segmentation failures, and the results on low-quality ( D i c e ∈ [ 0 , 0.6 ) ), medium-quality ( D i c e ∈ [ 0.6 , 0.8 ) ) and high-quality ( D i c e ∈ [ 0.8 , 1.0 ) ) segmentations showed satisfying robustness of our method.