Artificial intelligence (AI) is integrated into medical diagnosis, which has changed the medical industry by making the diagnosis more accurate and efficient. The large language models (LLMs) like GPT-4, Med - PaLM, etc. Have surfaced up as efficient tools for handling and incorporating large amounts of medical information, solving the problems with different types of data and complicated reasoning about them in medicine. In this paper, we explore the application of LLMs in the field of AI assisted medical diagnosis and its potential for knowledge integration, natural language understanding of electronic health records (EHRs), and real time decisions. We analyze the existing frameworks and conduct empirical evaluations to show that LLMs excel at processing unstructured medical data over traditional ML models with 89.2% diagnostic accuracy for general disease and 81.7% for rare ones. And also, the user satisfaction surveys shows that 87% of clinicians, 91% of patient, find LLM-based system intuitive & helpful. But data bias issues, privacy, clinical validations, this kind of challenge remains important. This study shows that LLMs might change diagnostic procedures and calls for people from different fields to work together to deal with ethical and technical problems.