Some questions are too broad to elicit a helpful answer from an AI chatbot
Nearly 70% of Medicare beneficiaries did not compare Medicare coverage options during the annual open-enrollment period, healthcare policy firm KFF found.
Using artificial-intelligence tools to help navigate Medicare's open-enrollment season might seem like an easy way to simplify what can be an overwhelming decision.
But AI chatbots are more likely to suggest only certain insurers and may fail to give nuanced advice that suits an individual's healthcare needs, new research has found.
After running more than 60 consumer prompts through ChatGPT, Claude, Perplexity, Gemini and Google (GOOG) AI Overviews to see which insurers got recommended, digital-marketing firm 5W AI Communications found that the leading insurers got named the most.
UnitedHealthcare, the largest Medicare Advantage company, led with the most AI citations, at roughly 15%, followed by Humana at 12%. Kaiser Permanente, which is a fraction of the size of those two companies, ranked third at 9% because the AI tools weighted consumer-satisfaction data more heavily than enrollment size, 5W AI found.
UnitedHealthcare has about 8.4 million Medicare Advantage members, while Humana has about 7.2 million and Kaiser Permanente has more than 1.9 million.
"Chatbots don't have your doctor network information or prescription or formulary information. If it's a very targeted question, maybe they can answer it. But it's not going to give nuanced, personalized information," said Cobi Blumenfeld-Gantz, chief executive of Chapter, a company that helps subscribers choose Medicare and Medicare Advantage plans. "If they give the right answer, it will be pure happenstance."
'AI is not a good replacement for a human being who might be able to ask you what your healthcare utilization looks like, but it can help with some basic explanations.'Miranda Yaver, assistant professor in the department of health policy and management at the University of Pittsburgh
Medicare Advantage - the private-plan alternative to traditional Medicare - offers extra benefits such as vision and dental coverage and caps annual out-of-pocket costs but has a limited choice of doctors and extra requirements for specialist referrals. Medicare Advantage controls more than half of the overall Medicare market of roughly 69.6 million subscribers.
So should you use AI to find a Medicare Advantage plan?
Using AI tools to decide on a Medicare Advantage plan may be complicated at best and inaccurate at worst, analysts said. A lot will depend on how you use the tool, what questions you ask and whether you verify the sources used and double-check the findings.
"AI is good at some things and not good at other things," Miranda Yaver, an assistant professor in the department of health policy and management at the University of Pittsburgh and the author of "Coverage Denied: How Health Insurers Drive Inequality in the United States."
"AI is good at simplified versions of things. It can help translate what deductibles are, how HMOs are different from PPOs and how coinsurance works. Where AI falls short is that it may not be able to appreciate your medical history and the complex insurance rules that are changing rapidly," Yaver said.
"AI is not a good replacement for a human being who might be able to ask you what your healthcare utilization looks like, but it can help with some basic explanations," Yaver said. "Healthcare rules are really complex and opaque."
The Centers for Medicare & Medicaid Services will release plan data for 2027 coverage on or around Oct. 1. Beneficiaries should look at their annual notice-of-change letter, which will spell out what may have changed in their current plan.
The annual Medicare open-enrollment period runs from Oct. 15 to Dec. 7. During that time, people can switch from traditional Medicare to a private Medicare Advantage plan, change from one Medicare Advantage plan to another, and join or change a Medicare Part D prescription-drug plan. The changes go into effect on Jan. 1.
However, most Medicare beneficiaries don't compare their options during open enrollment. In fact, nearly 70% of Medicare beneficiaries did not compare Medicare coverage options during the 2022 open-enrollment period, healthcare policy firm KFF found.
In 2026, the average person had 32 Medicare Advantage plans with prescription-drug coverage to choose from, KFF said.
"Medicare is a pretty complex program. AI may be helpful on a big-picture basis, or offer the basics around open enrollment itself," said Meredith Freed, senior policy manager with KFF's program on Medicare policy. "AI is one tool to help in the selection process. A lot depends on the types of questions you ask it. It may not be able to pull from all available sources, and you might need to confirm the answers."
Make AI work for you
Mary Johnson, an independent Social Security and Medicare policy analyst, said asking specific questions can be more helpful than simply inputting a general query when using AI tools.
"What I learned is that rephrasing and making narrow, specific questions can be very helpful. The wording of the question is important. Don't just ask what's the best drug or MA plan. I asked, 'What is the best method to find a good Medicare Advantage plan?'" Johnson said.
"That question brought me an excellent checklist sourced from AARP. Sources need to be checked out, but what I saw were sources that I would recommend. It leads with a link to the Medicare Plan Finder Tool and then advice for how to get the most out of that tool," Johnson said.
Johnson also asked a chatbot which Medicare Advantage plan serving her county in Virginia had the most complaints. AI answered correctly, saying that complaints are not publicly published but that there are Medicare star ratings that can be helpful in making selections.
KFF's Freed said the best resources are the Medicare Plan Finder, 1-800-MEDICARE and state health insurance assistance programs, known as SHIPs, which are free, independent counseling services for Medicare beneficiaries.
-Jessica Hall
Comments