AI In Medicine: ChatGPT-4 Outshines All AI Platforms in Neurosurgery Oral Board Exams With A Score Of 82.9 Percent While Google Bard Fails!

AI In Medicine: The world of artificial intelligence (AI) continues to make groundbreaking strides in the medical field, with recent studies showcasing the potential of AI-based chatbots like ChatGPT-4, GPT-3.5, and Google Bard in revolutionizing the way doctors train for their oral board exams. The potential impact of AI on neurosurgery, in particular, has garnered much attention, as ChatGPT-4 outperforms its predecessors and competitors in a simulated neurosurgery oral board examination.


 
In a remarkable study, researchers from the United States evaluated the performance of three general Large Language Models (LLMs), ChatGPT (GPT-3.5), GPT-4, and Google Bard, on higher-order questions typically found in the American Board of Neurological Surgery (ABNS) oral board examination.
 
The findings of this study not only highlight the significant advancements in AI In Medicine but also indicate a potential shift in the way medical education and examinations will be conducted in the future.
 
The ChatGPT series has already demonstrated impressive capabilities in passing medical board exams with multiple-choice questions. In a prior study, ChatGPT achieved a 73.4% score on a 500-question module simulating the neurosurgery written board exams. Its upgraded version, GPT-4, became available to the public on March 14, 2023, and has also garnered passing scores in over 25 standardized exams. The AI model showed an impressive >20% performance improvement on the United States Medical Licensing Exam (USMLE).
https://www.thailandmedical.news/news/revolutionizing-healthcare-with-chatgpt-unveiling-its-potential-and-future-prospects
 
Google Bard, another AI-based chatbot, features real-time web crawling capabilities, allowing it to provide more contextually relevant information when generating responses. These capabilities make Google Bard a potential game-changer for standardized exams in various fields, including medicine, business, and law.
 
The ABNS neurosurgery oral board examination is considered more rigorous than the written counterpart, taken by doctors two to three years after completing their residency. With a pass rate that hasn't exceeded 90% since 2018, it serves as a stringent test of a doctor's knowledge and decision-making abilities.
 
In this ground-breaking study, the researchers assessed the performance of GPT-3.5, GPT-4, and Google Bard on a 149-question module imitating the neurosurgery oral board exam. The Self-Assessment Neurosurgery Exam (SANS) indications exam covered complex topics, such as neurosurgical indications and interventional decision-making. The AI models were tested on one best-answer multiple-choice question format, and their performance on various question categories was compared.
 
The results were astonishing: GPT-4 attained an impressive score of 82.6%, outperforming both ChatGPT's score of 62.4% and Google Bard's score of 44.2%.
 
In addition, GPT-4 demonstrated significantly higher performance in the Spine subspecialty (90.5% vs. 64.3%). Interestin gly, GPT-4's performance on imaging-related questions was better than ChatGPT's (68.6% vs. 47.1%) and comparable to Google Bard's (68.6% vs. 66.7%).
 
These results indicate that AI models like GPT-4 have the potential to transform medical education, particularly in the field of neurosurgery. With its ability to navigate challenging concepts such as medical futility and its reduced rates of hallucination, GPT-4 could serve as a valuable tool for neurosurgical trainees preparing for their board exams.
 
However, there are ethical and legal implications that arise from the use of AI in medical training and practice. The potential for these AI models to replace or augment human decision-making in medical situations raises questions about liability, privacy, and the doctor-patient relationship.
 
One critical concern is that reliance on AI models could lead to the erosion of clinical skills among physicians, as they become more dependent on AI for decision-making. It is essential to strike a balance between leveraging AI's capabilities and preserving the human touch in patient care. The integration of AI should be gradual, with medical professionals and AI systems working together to complement and enhance each other's strengths.
 
Another potential issue is the possibility of AI systems perpetuating existing biases in medical practice. AI models learn from the data they are trained on, and if the data contains biases, the AI system is likely to perpetuate those biases. Efforts must be made to ensure that AI systems are trained on unbiased, representative data to minimize potential harm to patients.
 
Privacy concerns also arise when it comes to AI in medicine. As AI systems become more powerful and integrated into healthcare, the risks of data breaches and misuse of patient information increase. Strict regulations and robust data security measures must be implemented to ensure that patient privacy is maintained.
 
Moreover, the legal and ethical implications of AI-assisted decision-making in medicine must be addressed. The question of liability in cases where AI systems provide incorrect or suboptimal recommendations is yet to be resolved. Clear guidelines and regulations regarding the use of AI in medicine must be established to protect both patients and healthcare providers.
 
Despite these challenges, the potential benefits of AI in neurosurgery and medical education are too significant to ignore. The use of AI chatbots like GPT-4, GPT-3.5, and Google Bard in medical training and practice could revolutionize the way doctors prepare for their board exams, leading to improved performance and, ultimately, better patient outcomes.
 
The AI revolution in neurosurgery has the potential to change the face of medical education, making it more accessible, efficient, and effective. With the right balance of human input and AI assistance, the future of neurosurgery and medicine as a whole could see tremendous advancements in patient care and outcomes.
 
The continued development and refinement of AI models like GPT-4 are essential to unlocking the full potential of AI in medicine. As these models become more advanced, their ability to handle complex questions and situations will increase, making them even more valuable tools for medical professionals.
Ultimately, the integration of AI in neurosurgery and medical education represents an exciting and promising future. However, it is crucial to address the ethical, legal, and practical implications of AI in medicine to ensure that these powerful tools are used responsibly and in the best interests of patients and healthcare providers alike.
 
As we continue to explore the capabilities of AI models like GPT-4, it is important to remember that they are not replacements for human expertise and judgment but rather powerful tools that can help medical professionals make better, more informed decisions. By striking the right balance between AI and human input, the medical community can work together to harness the incredible potential of AI to revolutionize neurosurgery, medical education, and patient care.
 
The study was published on a preprint server: medRxiv.
https://www.medrxiv.org/content/10.1101/2023.04.06.23288265v1
 
For the latest developments of AI In Medicine, keep on logging to Thailand Medical News.