ChatGPT can almost pass the US Medical Licensing Exam.

Question

ChatGPT, a breakthrough in AI, has been shown in studies to almost pass the U.S. Medical Licensing Exam. The improvement of medical education by artificial intelligence is worthy of attention.

Answer 1

This is true. This article references a study published in PLOS Digital Health, an online journal that compiles research related to healthcare. The study demonstrates how ChatGPT can achieve scores "at or near the passing threshold for all three exams" of the USMLE, an exam physicians with an MD must pass to receive their license. Here is the abstract of the article posted for this study:

"We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making."

Link to Study: https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0000198

Answer 2

I found in an article from MedPage Today that this claim is true. The chatbot was able to achieve >50% accuracy across sections with some even being able to be greater than 60%, which is around the passing threshold year over year.

https://www.medpagetoday.com/special-reports/exclusives/102705

Answer 3

Yes, according to a study conducted and published in an open-access journal, ChatGP can "can score at or around the approximately 60 percent passing threshold for the United States Medical Licensing Exam (USMLE), with responses that make coherent, internal sense and contain frequent insights". 

The USMLE passing threshold is approximately 60%, and ChatGPT managed to average between 52.4% and 75.0%, making it likely on occasion to pass. 

Claiming the software is "almost" passing would be accurate.

https://www.eurekalert.org/news-releases/978878#:~:text=ChatGPT%20can%20score%20at%20or,PLOS%20Digital%20Health%20by%20Tiffany

Answer 4

The key word in this claim is almost, and yes, this claim is true. While ChatGBT has yet to pass the U.S. Medical Licensing Exam successfully, it has been proven to achieve an MBA and it is on the path to passing other big exams such as The Bar as well. The AI continues to grow knowledge and sooner rather than later, it will be able to pass the U.S. Medical Licensing Exam.

Link to related article - https://www.msn.com/en-us/news/technology/chatgpt-can-pass-the-united-states-medical-licensing-exam-and-the-bar-exam/ar-AA16ElJ3

Answer 5

Yes, ChatGPT was recently almost able to pass a US Medical Licensing Exam. The exam was the USMILE (United States Medical Liscesning Examination) which physicians with a Doctor of Medicine (MD) degree are required to pass with a 60% or higher. Normally, most people are able to get a 90%. Despite this, the fact that ChatGPT was able to get close to this score is very impressive. However, one has to remember that this test is made hard for humans to pass, not computers. In some ways, however, this could also be the very early stages of attempting to integrate AI technology in medical practice alongside humans as a helpful resource. 

https://www.sciencemediacentre.org/expert-reaction-to-study-on-chatgpt-almost-passing-the-us-medical-licensing-exam/#:~:text=According%20to%20the%20researchers%2C%20ChatGPT,multiple%2Dchoice%20with%20justification).