AI Chatbot for Cancer Patient Support: Development and Evaluation Using Llama 3.1, Mistral 7B, and PHI 3B

This study develops and evaluates a question-answering (Q&A) system for breast cancer patients using generative AI technologies. We compared the performance of three language models—Llama 3.1, Mistral 7B and Phi 3.5. The goal is to integrate this system into a patient-facing application, providing personalised, interactive support based on reliable sources such as Cancer Research UK. However, findings indicate that misinformation remains a significant concern. Medical chatbots utilising retrieval augmented generation (RAG) providing clinical information require significant refinement before being suitable for clinical use.