Assessing the knowledge of ChatGPT and Google Gemini in answering peripheral artery disease-related questions

Introduction To assess and compare the knowledge of ChatGPT and Google Gemini in answering public-based and scientific questions about peripheral artery disease (PAD). Methods Frequently asked questions (FAQs) about PAD were generated by evaluating posts on social media, and the latest edition of the European Society of Cardiology (ESC) guideline was evaluated and recommendations about PAD were translated into questions. All questions were prepared in English and were asked to ChatGPT 4 and Google Gemini (formerly Google Bard) applications. The specialists assigned a Global Quality Score (GQS) for each response. Results Finally, 72 FAQs and 63 ESC guideline-based questions were identified. In total, 51 (70.8%) answers by ChatGPT for FAQs were categorized as GQS 5. Moreover, 44 (69.8%) ChatGPT answers to ESC guideline-based questions about PAD scored GQS 5. A total of 40 (55.6%) answers by Google Gemini for FAQs related with PAD obtained GQS 5. In addition, 50.8% (32 of 63) Google Gemini answers to ESC guideline-based questions were classified as GQS 5. Comparison of ChatGPT and Google Gemini with regards to GQS score revealed that both for FAQs about PAD, and ESC guideline-based scientific questions about PAD, ChatGPT gave more accurate and satisfactory answers ( p = 0.031 and p = 0.026). In contrast, response time was significantly shorter for Google Gemini for both FAQs and scientific questions about PAD ( p = 0.008 and p = 0.001). Conclusion Our findings revealed that both ChatGPT and Google Gemini had limited capacity to answer FAQs and scientific questions related with PDA, but accuracy and satisfactory rate of answers for both FAQs and scientific questions about PAD were significantly higher in favor of ChatGPT.