Short answer: AI chatbots can help you understand health information, but they are not safe as a stand-in for a doctor. A careful new study in The Lancet found a closely supervised medical chatbot worked well with real patients, while an earlier audit found about half of answers from popular free chatbots were problematic. The safest way to use them is as a starting point you check with a clinician, not a final answer.
Key takeaways
- OpenAI says more than 300 million people ask ChatGPT health questions every week.
- A Lancet study of 98 patients found no safety stops when a medical chatbot interviewed them before visits, but a doctor watched every chat in real time.
- A BMJ Open audit of five free chatbots rated 49.6% of health answers problematic, and the bots refused to answer only 0.8% of the time.
- Researchers warn that harm from bad AI health advice is “structurally hidden,” because chat logs stay with the company.
- Most people still want a human in the loop: 63% in one U.S. survey chose a slower, clinician-reviewed answer over an instant AI one.
How many people ask AI for health advice?
A lot. When OpenAI expanded its Health feature to all adult U.S. users on July 23, it said, “Every week, more than 300 million people turn to ChatGPT with health-related questions.” The feature can connect to Apple Health and some medical records if you allow it. OpenAI says that connected data is not used to train its models or target ads, and it adds that ChatGPT “does not replace the care and judgment of qualified medical professionals.”
People are not always telling their doctors. A study of 400 cancer patients and caregivers in Turkey, reported this week from the journal Supportive Care in Cancer, found 24% had used AI tools for cancer information. Of those users, 69% had not told their physician. Yet 73% of all participants said they would use AI tools if their doctor checked them first.
What did the new Lancet chatbot study find?
What happened: On October 8, researchers at Beth Israel Deaconess Medical Center and Harvard Medical School, working with Google, published what they believe is the first real-world study of a patient-facing AI chatbot in primary care. Patients who booked urgent appointments chatted from home with AMIE, a research chatbot from Google. It took their history, suggested possible diagnoses for the doctor, and wrote a summary.
| Measure | Result |
|---|---|
| Patients who completed both the chat and the visit | 98 of 114 enrolled |
| Chats a supervising doctor had to stop for safety | 0 |
| Hallucinations spotted | 1 (a past surgery date given as a future date) |
| Doctors who said the summary helped them prepare | 75% of 44 visits where they read it first |
| Chatbot’s possible diagnoses matched the final diagnosis | 90%, per Google |
Why it matters: This is a rare test with real patients, not exam questions. Patients praised the chatbot’s listening and explanations, and their attitudes toward AI improved. “AI tools must first be shown to operate safely and acceptably in real care workflows,” said Dr. Adam Rodman, the corresponding author.
The other side: The study was small, at one clinic, and built to test safety and feasibility, not health outcomes. A doctor monitored every conversation and could step in, so it says little about how the tool would do alone. Alphabet, Google’s parent company, funded the work. One physician rated a chat as somewhat harmful because listing lymphoma as a possibility may have made the patient anxious. Google itself says larger trials are needed.
How accurate are popular chatbots on health questions?
Everyday chatbots are a different story from a supervised clinical tool. In an audit published in BMJ Open in April, experts sent 250 health prompts to five free chatbots: Gemini, DeepSeek, Llama, ChatGPT and Grok, in versions available in February 2025. Topics were cancer, vaccines, stem cells, nutrition and athletic performance.
| Finding | Result |
|---|---|
| Answers rated problematic | 49.6% (30% somewhat, 19.6% highly) |
| Highly problematic, open-ended questions | 32% |
| Highly problematic, closed-ended questions | 7.2% |
| Worst topics | Nutrition and athletic performance |
| Questions the bots refused to answer | 0.8% |
| Median completeness of cited references | 40% |
Bots did better on settled topics like vaccines and cancer, and worse on open-ended questions where the science is mixed. They almost never declined to answer. The authors urge users to be “extremely critical when seeking medical advice from AI chatbots.” One caveat: the models tested are older versions, and newer ones may perform differently.
Why is harm from AI health advice hard to track?
A perspective paper in Nature Health, from researchers at Binghamton, Stanford, Texas A&M and Indiana universities, argues the problem is not only wrong answers. It is that nobody can see them. Chats stay on the company’s platform, users have no standard way to report a bad answer, and outside researchers cannot measure the damage. The authors call these harms “structurally hidden from the clinicians, researchers, and regulators.”
Their example is a widely reported case of a 60-year-old man who asked ChatGPT how to cut chloride from his diet, swapped table salt for sodium bromide, and was hospitalized with bromide poisoning. He could not retrieve the chat, so his doctors could not tell exactly what he had been told. Kaicheng Yang of Binghamton University, one of the authors, says no one is watching for these harms today. “We do not think this is something we should rely on the companies to do,” he said. The authors want companies to let users export health chats for their doctors, build systems to flag and investigate bad guidance, and allow independent monitoring.
Do patients trust AI for medical answers?
Mostly when a human checks it. A survey of 1,185 U.S. adults, published October 14 in JAMA Network Open by researchers at Columbia University School of Nursing and Vanderbilt University Medical Center, asked people to choose between an instant AI reply and one a doctor or nurse reviewed within 48 hours. For medical questions, 63.1% picked the slower, reviewed answer. For scheduling and refills, 57.4% preferred the fast AI reply. The survey was run in mid-2024, so views may have shifted since.
“When people have a medical question, they want to hear from their doctor, not AI, even if AI may give an immediate answer,” said lead author Natalie Benda.
The other side
For many people, the alternative to a chatbot is not a doctor. It is a late-night search, a long wait, or nothing. Dr. Philip Bernard of Children’s Health Dallas told KERA that chatbots could widen access in rural and low-income communities, “so long as the patient using them is also consulting a trusted physician.” The Lancet study also shows that, with guardrails, patients can find these tools helpful. The fair picture is a useful tool with real blind spots, not a miracle or a menace.
The TrustNoRobot Risk Check
| Question | What we see |
|---|---|
| Exposure | High. Hundreds of millions of people ask chatbots health questions each week, and many do not tell their doctor. |
| Evidence | Mixed. Supervised clinical tools look promising in small studies. Audits of free consumer bots show frequent errors, especially on open-ended questions. |
| Speed | Fast. Health features are now built into mainstream chatbots, search results and social media. |
| Cushion | Thin. There is no standard way to report bad advice or share chat logs with clinicians, and liability rules are unsettled. |
What this means for you
Use AI to understand, not to decide. It is good at explaining a medical term, helping you write questions before an appointment, or summarizing what a doctor said. Be most careful with open-ended questions about nutrition and fitness, where the audit found the most bad answers. Never change a medication, dose or diet based only on a chatbot. Save or screenshot any health chat you act on, and show it to your clinician. If you have symptoms that feel urgent, call a doctor or emergency services instead.
Privacy matters too. Before connecting medical records, read what the app does with them. Our guide on how to stop AI from training on your chats shows where the settings are. For kids and teens, see what a new safety test found about ChatGPT for Teens. And if you work in health care and wonder how AI may change your role, try our Job Exposure Explorer and our guide on how to measure AI risk to your job.
Frequently asked questions
Is it safe to ask ChatGPT medical questions?
It can help you understand information, but answers can be wrong and sound confident. Treat it as a starting point and check anything important with a clinician.
How often do AI chatbots give bad health advice?
A 2026 BMJ Open audit of five free chatbots rated 49.6% of answers to 250 health prompts as problematic, with open-ended questions faring worst.
Did the Lancet study prove AI chatbots are safe for patients?
No. It found no safety stops among 98 patients, but a doctor watched every chat, and it did not measure health outcomes. Larger trials are needed.
Should I tell my doctor I used AI for health advice?
Yes. Sharing what the chatbot said helps your doctor catch errors. Researchers say missing chat records make it hard for doctors to trace harm.
Sources
- OpenAI, “Health in ChatGPT” (July 23, 2026)
- Harvard Medical School, “A Real-World Study of Patient-Facing AI in Primary Care” (Oct. 13, 2026)
- Google, AMIE clinical study in The Lancet (Oct. 8, 2026)
- News-Medical, “Study finds popular AI chatbots often give problematic health advice” (Apr. 16, 2026), on Tiller et al., BMJ Open
- Binghamton University, “When AI chatbots give bad health advice, no one can see the damage” (Oct. 2026), on a perspective in Nature Health
- Medical Economics, on Benda et al., “Public Perceptions of Timeliness for AI Use and Human Oversight,” JAMA Network Open (Oct. 14, 2026)
- Bioengineer.org, on Ismayilov et al., Supportive Care in Cancer (Oct. 19, 2026)
- KERA, “AI chatbots could make healthcare more accessible, but some doctors are still wary” (Oct. 16, 2026)
Leave a Reply