Kush_EXE

Would your chatbot send you to the emergency room for this headache?

· Runs 2:39

Two volunteers in an Oxford study, published in Nature Medicine in February 2026, acted out the same case written by doctors: a bleed on the brain, a rare type of stroke. Both typed a terrible headache, a stiff neck and light that hurt their eyes into GPT-4o. One was told it might be a migraine and to rest in a dark, quiet room. The other also said the headache came on suddenly and was told to go to the emergency room. Across 1,298 UK participants, the chatbots reading the full case alone named the right condition about 95% of the time. People typing to the same chatbots got there in fewer than 34.5% of cases, and the group without a chatbot, mostly using web search or what they already knew, did better at naming it.

The AI mostly knew. The conversation is where it broke: in 16 of 30 chats checked, the first message left things out, and when the bot did mention the right condition, people often didn't pick it. The limits: these were volunteers playing patients, on 2024 chatbots. Newer models may score higher, but the authors say it's unclear if that helps real people. A separate study found OpenAI's ChatGPT Health sent about half of its test emergencies to a doctor's visit instead of the ER. So here are three things to do. Tell it everything, especially when it started and how fast. Ask it what else it needs to know. If it gives two answers, ask which is more dangerous and what means go now. A sudden, extremely painful headache that won't go away is an emergency: the NHS says call emergency services (999 in the UK). This video is general information, not medical advice.

Watch on YouTube Subscribe

In this video

  1. Same headache, opposite advice
  2. The detail that changed the answer
  3. The Oxford test: 95% alone vs 1 in 3 with people
  4. Where the conversation broke
  5. The limits, and the right next step
  6. ChatGPT Health under-triaged emergencies
  7. Three things to do
  8. When it's not a chatbot question

Same headache, opposite advice 0:00

You type a bad headache into a chatbot. So did two people. One was told to rest in a dark room; the other, go to the emergency room. The first, a volunteer acting out a case three doctors wrote, types: a terrible headache, a stiff neck, light that hurts their eyes. GPT-4o says it might be a migraine. Try resting in a dark, quiet room.

The detail that changed the answer 0:23

Except the case was a bleed on the brain, a rare type of stroke. The second describes the same, plus the detail the paper singles out: it came on suddenly. Same chatbot, opposite answer: go to the emergency room. The paper calls it an extreme case. If you ask a chatbot about your symptoms, this is about you. Stay for three things to do.

The Oxford test: 95% alone vs 1 in 3 with people 0:45

In this Oxford study, nearly 1,300 people each worked through two of ten cases; most got one of three chatbots, the rest their usual way. Reading the full case alone, the chatbots named the right condition about 95 percent of the time. When people typed, they got there about one time in three, at best. People without a chatbot, mostly using web search or what they knew, did better at naming it.

Where the conversation broke 1:10

So the AI mostly knew; the conversation broke. In 16 of 30 chats checked, the first message left things out. In about two out of three chats, the bot mentioned a relevant condition. But it offered two or so, and people often didn't pick it. Everyone, chatbot or not, tended to think their case was less serious than it was.

The limits, and the right next step 1:31

These were volunteers playing patients, on 2024 chatbots. Newer models may score higher, but the authors say it's unclear if that helps real people. Even alone, the bots picked the right next step only a bit over half the time.

ChatGPT Health under-triaged emergencies 1:45

Another study found OpenAI's newer ChatGPT Health sent about half of test emergencies to a doctor's visit, not the ER, though it caught obvious ones like stroke. OpenAI says it's built for back-and-forth. As Oxford co-author and GP Rebecca Payne put it: despite all the hype, AI just isn't ready to take on the role of the physician.

Three things to do 2:06

One: tell it everything, especially when it started and how fast. Two: ask it, what else do you need to know? Doctors ask because patients may not know which symptoms matter. Three: if it gives two answers, ask which is more dangerous, and what means go now.

When it's not a chatbot question 2:23

A sudden, extremely painful headache that won't go away? The NHS says call emergency services. That's not a chatbot question. The chatbot can often spot the condition, but it only knows what you tell it. So make it a second opinion, never the decision.

What the sources say

  • A randomized, preregistered study led by researchers at the University of Oxford and published in Nature Medicine on 9 February 2026 had 1,298 participants, each assigned either to a chatbot (GPT-4o, Llama 3 or Command R+) or to a source of their own choice, across ten medical scenarios. PubMed Central, Open Access Government
  • In a study published in Nature Medicine in February 2026, the chatbots GPT-4o, Llama 3 and Command R+, tested on their own on ten medical scenarios, identified the conditions correctly in 94.9% of cases and chose the right next step in 56.3% of cases on average. PubMed Central
  • In a 1,298-participant Nature Medicine study published in February 2026, people using the chatbots GPT-4o, Llama 3 or Command R+ on medical scenarios identified relevant conditions in fewer than 34.5% of cases and the right next step in fewer than 44.2%. Its abstract calls both no better than the control group. PubMed Central
  • In a February 2026 Nature Medicine study of chatbots as medical assistants, the control group used whatever they normally would, mostly internet search or their own knowledge. They had 1.76 times the odds of identifying a relevant condition compared with chatbot users, with no significant difference on the next step. PubMed Central
  • A February 2026 Nature Medicine study reports that two participants sent the chatbot GPT-4o very similar messages describing symptoms of a subarachnoid hemorrhage, a brain bleed. One was told to rest in a dark, quiet room; the other, who said the headache came on suddenly, was told to go to the emergency room. Reliability of LLMs as medical assistants for the general public: a randomized preregistered study, Extended Data Table 2: Interaction Transcript: Inconsistent Responses
  • In a February 2026 Nature Medicine study of people using chatbots on medical scenarios, the first message held only partial information in 16 of 30 sampled conversations. The chatbots mentioned a relevant condition in 65.7% to 73.2% of conversations, but users did not consistently include them in their final answers. PubMed Central
  • A Nature Medicine study of chatbots as medical assistants tested UK adults on scenarios written by doctors, not their own illnesses, with data collected from 21 August to 14 October 2024. Its authors say newer models may score higher on benchmarks, but it is unclear whether that carries over to real users. PubMed Central
  • A February 2026 Nature Medicine study of chatbots as medical assistants concludes none of the tested models were ready for direct patient care and recommends testing with real users before public deployment in healthcare. Co-author Dr Rebecca Payne said AI "just isn't ready to take on the role of the physician." PubMed Central, Open Access Government
  • A Nature Medicine study by Mount Sinai, reported by NBC News on 3 March 2026, found OpenAI's ChatGPT Health advised a doctor's visit instead of the emergency room in 51.6% of emergency cases, though it triaged stroke correctly 100% of the time. OpenAI said the study does not reflect how the product is used. NBC News
  • The NHS lists a thunderclap headache, an extremely painful headache that starts suddenly and does not go away, first among the main symptoms of a subarachnoid haemorrhage, a rare type of stroke that causes bleeding on the brain. NHS guidance is to call 999 for a sudden, extremely painful headache that does not go away. NHS

Sources

  1. Reliability of LLMs as medical assistants for the general public: a randomized preregistered study PubMed Central
  2. Reliability of LLMs as medical assistants for the general public: a randomized preregistered study Nature Medicine
  3. Extended Data Table 2: Interaction Transcript: Inconsistent Responses PubMed Central
  4. New study warns risks of AI chatbots giving medical advice to patients Open Access Government
  5. Medical chatbots: the questions or the answers StatsChat
  6. ChatGPT Health 'under-triaged' half of medical emergencies in a new study NBC News
  7. Subarachnoid haemorrhage NHS

More in AI news explained