Researchers: If Medical Workers Misuse AI, It Could Be Harmful

Hospitals and health care systems are reportedly using artificial intelligence (AI), according to a Stanford School of Medicine study conducted in California. AI systems are being used by healthcare practitioners to evaluate medical records and arrange physician notes regarding patient health. But the researchers caution that widely used AI programs may contain inaccurate medical information or information that they deemed to be “racist.” Some fear that the instruments may make Black patients’ health inequities worse. This month, the work was published in Digital Medicine. Researchers found that AI models provided inaccurate responses, including made-up and racially biased responses, to questions concerning Black patients.

Artificial intelligence (AI) systems, such as chatbots like ChatGPT and Google’s Bard, “learn” using data that is gathered from the internet. Certain specialists express concern that these methods may have negative effects and perpetuate long-standing instances of medical racism. They fear that as more doctors utilize chatbots to handle routine tasks like emailing patients or collaborating with insurance providers, this will continue. Four tools were tested for the report. They were Claude from Anthropic, Bard from Google, and ChatGPT and GPT-4 from OpenAI. The researchers reported that when posed medical questions about skin thickness, lung volume, and kidney function, all four tools performed poorly.

They seemed to be repeating myths about biological distinctions between Black and White people in certain instances. Experts claim that efforts have been made to eradicate myths from medical associations. certain claim that these ideas lead certain medical professionals to misidentify health issues, fail to recognize the agony that Black patients experience, and prescribe less assistance. Dr. Roxana Daneshjou teaches biomedical data science at Stanford University. The paper was under her supervision. “Getting this wrong can have very real-world consequences that can impact health disparities,” the speaker stated. She claimed that she has been working with others to get such incorrect ideas out of medicine. She finds the emergence of those beliefs to be “deeply concerning.”