MedSumGraph: enhancing GraphRAG for medical QA with summarization and optimized prompts

The rapid development of large language models (LLMs) has accelerated research into applying artificial intelligence (AI) to domains such as medical question answering and clinical decision support. However, LLMs face substantial limitations in medical contexts due to challenges in understanding specialized terminology, complex contextual information, hallucination issues (i.e., generating incorrect responses), and the black-box nature of their reasoning processes. To address these issues, methods like retrieval-augmented generation (RAG) and its graph-based variant, GraphRAG, have been proposed to incorporate external knowledge into LLMs. Nonetheless, these approaches often rely heavily on external resources and increase system complexity. In this study, we introduce MedSumGraph, a medical question-answering system that enhances GraphRAG by integrating structured medical knowledge summaries and optimized prompt designs. Our method enables LLMs to better interpret domain-specific knowledge without requiring additional training, and it enhances the reliability and interpretability of responses by directly embedding factual evidence and graph-based reasoning into the generation process. MedSumGraph achieves competitive performance on two out of eight multiple-choice medical QA benchmarks, including MedQA (USMLE), outperforming closed-source LLMs and domain-specific foundation models. Moreover, it generalizes effectively to open-domain QA tasks, yielding significant gains in reasoning over common knowledge and evaluating the truthfulness of answers. These findings demonstrate the potential of structured summarization and graph-based reasoning in enhancing the trustworthiness and versatility of LLM-driven medical AI systems. • MedSumGraph is a novel hybrid approach that integrates medical question summarization and GraphRAG to enhance LLMs for medical QA without fine-tuning. • It is an efficient method for constructing a knowledge graph to summarize structured medical knowledge, along with a novel response generation approach using medical question summarization-based global search and graph-based local search. • It advances the development of trustworthy medical AI and explainable AI by providing fact-based graphical results that support the answers generated by LLM. • The proposed MedSumGraph achieves a new SOTA performance beyond that of medical-based LLMs, as well as methods for injecting external knowledge. • The proposed MedSumGraph has proven its potential to extend beyond the medical domain into open areas.