Guideline adherence in surgical decisions for T1 colorectal cancer after endoscopic resection: large language models vs clinicians

Background: Patients with T1 colorectal cancer (CRC) often show poor adherence to guideline-recommended treatment strategies after endoscopic resection. To address this challenge and improve clinical decision-making, this study aims to compare the accuracy of surgical management recommendations between large language models (LLMs) and clinicians. Methods: This retrospective study enrolled 202 patients with T1 CRC who underwent endoscopic resection at three hospitals. We compared the decision-making accuracy of two representative LLMs (ChatGPT-4o and DeepSeek) with that of 29 clinicians in determining whether additional surgery was required after endoscopic resection of T1 CRC. To optimize the inputs for the LLMs, we applied a prompt-engineering strategy that combined role prompting, in-context learning, and few-shot learning. Results: In clinical practice, we found that the guideline adherence rate after endoscopic resection for T1 CRC was below 80%. We analyzed the pathology reports of 200 patients with T1 CRC, and the results showed that both ChatGPT-4o and DeepSeek significantly outperformed clinical physicians. Subgroup analyses demonstrated that LLMs outperformed doctors regardless of years of experience or professional background. Additionally, the use of Chinese or English input had no significant impact on the performance of the LLMs. Conclusion: This study highlights the potential of LLMs to improve guideline adherence in the post-endoscopic resection management of T1 CRC.