Skip to Main ContentBanner Image

Artificial Intelligence in Health Care and Medicine

AI In health care and medical scholarship

Health Care AI Concerns

Errors: AI systems may make mistakes, leading to incorrect diagnoses or treatment recommendations, which could potentially harm patients. Liability resulting from inaccuracies or misuse is an evolving concern.

 

Patient Privacy and HIPAA: The collection and analysis of large amounts of patient data raise concerns about privacy and sharing of confidential information. At present, chatbots cannot comply with the Health Insurance Portability and Accountability Act (HIPAA) in any meaningful way despite industry assurances. (See ChatGPT Confirms Data Breach, Raising Security Concerns)
 
User Privacy: Information you enter into many chatbots, including your personal information may be used unless you opt out. See these example privacy policy and terms of use from OpenAI.
 
**Security: Similar to privacy issues, a Gen AI system must continuously defend itself from unauthorized access.  More...
 
Job loss: The automation of certain tasks may lead to job displacement for healthcare professionals, particularly those performing repetitive tasks. Some specialists fear this predicts a diminution in their numbers. (See Confronting the Disruption of the Infectious Diseases Workforce by Artificial Intelligence: What This Means for Us and What We Can Do About It. (Open Forum Infect Dis. 2024 Jan 31;11(3)

 

Missing the human touch: Some are concerned that increased use of AI could reduce the number of opportunities to have caring and compassionate interactions with patients. 

 

Biased data impacting equity and fairness: AI systems can reflect and perpetuate human biases in healthcare as they are trained on human-generated data. This may lead to unequal treatment or outcomes for certain patient groups. This training method could also impact equity, as the dominance of languages like English in AI models can marginalize Indigenous languages and cultures. More...

 

Overdependence on technology: Critical thinking is a skill that must be practiced.  “Automation bias” occurs when people trust the information that comes out of a machine, bypassing critical thinking.  In health care this is an important patient quality and safety issue. Overreliance on AI systems may lead to unthinking acceptance of AI-created information. (See Artificial Intelligence Can Improve Health Care - but Not Without Human Oversight; Pew)

 

Hallucinations:  These occur when a generative AI algorithm creates text or audio that is either nonsensical or appears credible but is factually inaccurate.

 

Ethical considerations and intellectual property: The use of AI in medicine raises ethical questions about the role of technology in healthcare, informed consent, patient autonomy, and the potential of AI being used for unethical purposes. Using existing data to train others in AI raises intellectual property and copyright concerns. (See Experts Doubt Ethical AI Design Will Be Broadly Adopted as the Norm Within the Next Decade, Pew Research Center.)
 
We don't always know how it works.  In some cases, AI developers cannot explain exactly how their program arrived at its answer. This is called the "black box problem" of AI. It results in situations where errors cannot be fixed and biases are hidden.

 

Regulatory challenges: When technology leaps forward, as AI is doing now, guidelines and regulations trail far behind. There are legitimate concerns about safety, efficacy, and oversight.

 

No More Radiologists? Generative AI can locate and measure abnormalities without getting tired.  It can analyze images more quickly than a human.  Predictions are that Radiologists will have more time to devote to complex cases. Others see this as the thin edge of a wedge leading to job losses in this specialty.

Bias in AI (1:50) National Institutes of Standards and Technology

All pages of this Artificial Intelligence (AI) Guide provide an introduction to this evolving field for  HonorHealth employees, faculty, fellows, residents, and staff. Due to the rapid advancement of this emerging technology, information may become outdated at times.  All links, organizations, and products presented here are for informational purposes only.  This information supplements but does not supersede information from HonorHealth policies and guidelines.