The imperative for regulatory oversight of large language models (or generative AI) in healthcare - npj Digital Medicine
www.nature.com
The rapid advancements in artificial intelligence (AI) have led to the development of sophisticated large language models (LLMs) such as GPT-4 and Bard. The potential implementation of LLMs in healthcare settings has already garnered considerable attention because of their diverse applications that include facilitating clinical documentation, obtaining insurance pre-authorization, summarizing research papers, or working as a chatbot to answer questions for patients about their specific data and concerns. While offering transformative potential, LLMs warrant a very cautious approach since these models are trained differently from AI-based medical technologies that are regulated already, especially within the critical context of caring for patients. The newest version, GPT-4, that was released in March, 2023, brings the potentials of this technology to support multiple medical tasks; and risks from mishandling results it provides to varying reliability to a new level. Besides being an advanced LLM, it will be able to read texts on images and analyze the context of those images. The regulation of GPT-4 and generative AI in medicine and healthcare without damaging their exciting and transformative potential is a timely and critical challenge to ensure safety, maintain ethical standards, and protect patient privacy. We argue that regulatory oversight should assure medical professionals and patients can use LLMs without causing harm or compromising their data or privacy. This paper summarizes our practical recommendations for what we can expect from regulators to bring this vision to reality.

Title: The Imperative for Regulatory Oversight of Large Language Models (or Generative AI) in Healthcare

Author(s): Bertalan Meskó & Eric J. Topol Word count: 2,222

Estimated average read time: 10 minutes

Summary: This article highlights the need for regulatory oversight of large language models (LLMs), such as GPT-4 and Bard, in healthcare settings. LLMs have the potential to transform healthcare by facilitating clinical documentation, summarizing research papers, and assisting with diagnoses and treatment plans. However, these models come with significant risks, including unreliable outputs, biased information, and privacy concerns.

The authors argue that LLMs should be regulated differently from other AI-based medical technologies due to their unique characteristics, including their scale, complexity, broad applicability, real-time adaptation, and potential societal impact. They emphasize the importance of addressing issues such as transparency, accountability, fairness, and data privacy in the regulatory framework.

The article also discusses the challenges of regulating LLMs, including the need for a new regulatory category, consideration of future iterations with advanced capabilities, and the integration of LLMs into already approved medical technologies.

The authors propose practical recommendations for regulators to bring this vision to reality, including creating a new regulatory category, providing guidance for deployment of LLMs, covering different types of interactions (text, sound, video), and focusing on companies developing LLMs rather than regulating each iteration individually.

Evaluation for Applicability to Applications Development: This article provides valuable insights into the regulatory challenges and considerations related to large language models in healthcare. While it primarily focuses on the medical field, the principles and recommendations discussed can be applicable to applications development using large language models or generative AI systems in various domains.

Developers working on applications that utilize large language models should be aware of the potential risks and ethical concerns associated with these models. They should also consider the need for regulatory compliance and the importance of transparency, fairness, data privacy, and accountability in their applications.

Additionally, developers may find the proposed practical recommendations for regulators helpful in shaping their own strategies for responsible and compliant development of applications using large language models. Understanding the regulatory landscape and being proactive in addressing potential risks and challenges can lead to the successful deployment and use of these models in various applications.