The impact of phantom design and material‐dependence on repeatability and reproducibility of CT‐based radiomics features

Abstract Purpose To understand the design of radiomics phantom and material‐dependence on repeatability and reproducibility of computed tomography (CT) radiomics features. Methods A radiomics phantom consisting of various materials with uniformity, textural, and biological components, was constructed. The phantom was scanned with different manufacturer CT scanners and the scans were repeated multiple times on the same scanner with different acquisition settings as kVp, mAs, orientation, field of view (FOV), slice thickness, pitch, reconstruction kernels, and acquisition mode. A total of 72 phantom scans were included. For each scan, 18 different regions of interest (ROI) were contoured and 708 radiomics features were extracted from each ROI via an open source radiomics tool, IBEX. To relate the phantom data to patient data, the radiomics features from different phantom materials were compared with those extracted from 50 patients’ images of five disease sites as brain, head‐and‐neck, breast, liver, and lung cases using box‐plots comparison and principal component analysis (PCA). The temporal stability of imaging features was then evaluated with respect to a controlled scenario (test–retest) via the intra‐class correlation coefficient (ICC). The reproducibility of radiomics features with respect to different scanners or acquisition settings were further evaluated with concordance correlation coefficients (CCC). Results Among all phantom materials, the biological component had feature values closest to human tissues, especially for tumors in brain and liver. The textural component showed similar ranges of variation to lung lesions, particularly for cartridges of rice, cereal, and the 3D‐printed textural phantom with fine and rough grid. It also showed that certain materials, such as polystyrene foam, plaster, and peanuts, did not have comparable values to human tissue and could be excluded for future phantom design. High repeatability was observed in the test–retest study as indicated by an ICC value of 0.998 ± 0.020. All materials were used for feature stability analysis. For the inter‐scanner study, shape‐related features were the most‐reliable category with 94% of features having CCC ≥ 0.9, while gradient orientation histogram (GOH) were the least‐reliable with only 14.6% meeting the criteria. For the intra‐scanner study, the reproducibility of CT‐based radiomics features showed material‐dependence. In general, the instability of radiomics features introduced by kVp, mAs, pitch, acquisition mode, and orientation were relatively mild. However, the homogeneous materials were more vulnerable to those changes compared to materials with textural patterns. Regardless of material compositions, resolution parameters like FOV and slice thickness, could have large impact on feature stability. Switching between standard and bone reconstruction kernels could also result significant changes to feature reproducibility. Conclusion We have built a radiomics phantom using materials that cover a wide span of tumor textures seen in oncological CT images. The designed phantom presents a preliminary opportunity for investigating reproducibility of radiomics features and the reproducibility can be material dependent. Thus, in the radiomics quality assurance design, it is important to choose appropriate materials that can provide a close range of radiomics features to patients with specific disease sites dependency taken into consideration.