Exploring the potential of large language models in identifying metabolic dysfunction‐associated steatotic liver disease: A comparative study of non‐invasive tests and artificial intelligence‐generated responses

Abstract Background and Aims This study sought to assess the capabilities of large language models (LLMs) in identifying clinically significant metabolic dysfunction‐associated steatotic liver disease (MASLD). Methods We included individuals from NHANES 2017–2018. The validity and reliability of MASLD diagnosis by GPT‐3.5 and GPT‐4 were quantitatively examined and compared with those of the Fatty Liver Index (FLI) and United States FLI (USFLI). A receiver operating characteristic curve was conducted to assess the accuracy of MASLD diagnosis via different scoring systems. Additionally, GPT‐4V's potential in clinical diagnosis using ultrasound images from MASLD patients was evaluated to provide assessments of LLM capabilities in both textual and visual data interpretation. Results GPT‐4 demonstrated comparable performance in MASLD diagnosis to FLI and USFLI with the AUROC values of .831 (95% CI .796–.867), .817 (95% CI .797–.837) and .827 (95% CI .807–.848), respectively. GPT‐4 exhibited a trend of enhanced accuracy, clinical relevance and efficiency compared to GPT‐3.5 based on clinician evaluation. Additionally, Pearson's r values between GPT‐4 and FLI, as well as USFLI, were .718 and .695, respectively, indicating robust and moderate correlations. Moreover, GPT‐4V showed potential in understanding characteristics from hepatic ultrasound imaging but exhibited limited interpretive accuracy in diagnosing MASLD compared to skilled radiologists. Conclusions GPT‐4 achieved performance comparable to traditional risk scores in diagnosing MASLD and exhibited improved convenience, versatility and the capacity to offer user‐friendly outputs. The integration of GPT‐4V highlights the capacities of LLMs in handling both textual and visual medical data, reinforcing their expansive utility in healthcare practice.