+1 vote
in General Factchecking by Apprentice (1.4k points)
ChatGPT, a breakthrough in AI, has been shown in studies to almost pass the U.S. Medical Licensing Exam. The improvement of medical education by artificial intelligence is worthy of attention.

5 Answers

+2 votes
by Apprentice (1.7k points)
selected by
Best answer

This is true. This article references a study published in PLOS Digital Health, an online journal that compiles research related to healthcare. The study demonstrates how ChatGPT can achieve scores "at or near the passing threshold for all three exams" of the USMLE, an exam physicians with an MD must pass to receive their license. Here is the abstract of the article posted for this study:

"We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making."

Link to Study: https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0000198

+1 vote
by Apprentice (1.1k points)
I found in an article from MedPage Today that this claim is true. The chatbot was able to achieve >50% accuracy across sections with some even being able to be greater than 60%, which is around the passing threshold year over year.

0 votes
by (140 points)

Yes, according to a study conducted and published in an open-access journal, ChatGP can "can score at or around the approximately 60 percent passing threshold for the United States Medical Licensing Exam (USMLE), with responses that make coherent, internal sense and contain frequent insights". 

The USMLE passing threshold is approximately 60%, and ChatGPT managed to average between 52.4% and 75.0%, making it likely on occasion to pass. 

Claiming the software is "almost" passing would be accurate.


0 votes
by Novice (880 points)

The key word in this claim is almost, and yes, this claim is true. While ChatGBT has yet to pass the U.S. Medical Licensing Exam successfully, it has been proven to achieve an MBA and it is on the path to passing other big exams such as The Bar as well. The AI continues to grow knowledge and sooner rather than later, it will be able to pass the U.S. Medical Licensing Exam.

Link to related article - https://www.msn.com/en-us/news/technology/chatgpt-can-pass-the-united-states-medical-licensing-exam-and-the-bar-exam/ar-AA16ElJ3

0 votes
by Novice (920 points)

Yes, ChatGPT was recently almost able to pass a US Medical Licensing Exam. The exam was the USMILE (United States Medical Liscesning Examination) which physicians with a Doctor of Medicine (MD) degree are required to pass with a 60% or higher. Normally, most people are able to get a 90%. Despite this, the fact that ChatGPT was able to get close to this score is very impressive. However, one has to remember that this test is made hard for humans to pass, not computers. In some ways, however, this could also be the very early stages of attempting to integrate AI technology in medical practice alongside humans as a helpful resource. 



Community Rules

Be respectful.

There is bound to be disagreement on a site about misinformation. Assume best intentions on everyone's part.

If you are new to factchecking, take some time to learn about it. "How to Factcheck" has some resources for getting started. Even if you disagree with these materials, they'll help you understand the language of this community better.

News Detective is for uncovering misinformation and rumors. This is not a general interest question-answer site for things someone could Google.


The title is the "main claim" that you're trying to factcheck.

Factcheck This: Birds don't exist

If possible, LINK TO to the place you saw the claim.


LINK TO YOUR EVIDENCE or otherwise explain the source ("I called this person, I found it in this book, etc.")

But don't just drop a link. Give an explanation, copy and paste the relevant information, etc.

News Detective is not responsible for anything anyone posts on the platform.