As discussed previously, AI technologies have the capability to enhance healthcare and improve patient outcomes. There is a technology that stands alone in its promise to disrupt and change healthcare called ChatGPT. Although there are similar proprietary technologies, ChatGPT (Generative Pretrained Transformer) currently has received an enormous amount of press and scrutiny in the mainstream media and scientific communities.
ChatGPT is a Natural Language Processing model (NLP), also known as Large Language Model (LLM) AI software. What makes LLM AI technology unique is its ability to create novel sequences of text never previously observed. ChatGPT is trained on vast amounts of information available on the Internet with 175 billion parameters. The interface allows the user to ask a question or give a command, and ChatGPT will answer in a format that replicates human writing or conversation.
The story of ChatGPT is astounding. Launched in November of 2022, it took only two months to gain 100 million users. For comparison, it took Netflix 18 years to have 100 million subscribers. Additionally, ChatGPTās website reached a billion page visits in just 3 months. It is available in 161 countries, but has been banned in some nations, such as Afghanistan, China, Iran, and Russia, among others. There is a free version and a paid subscription option for a monthly fee of $20. The estimated revenue from ChatGPT will be $200 million by the end of 2023. The parent company, OpenAI, has a valuation of $29 billion.
ChatGPTās online interface is simple. There is a textbox where the user can enter a command or question. These are referred to as āprompts.ā ChatGPT will then generate a response. The crafting of the prompt has a huge impact on the quality of ChatGPTās output, depending on the details and parameters given. Another versatile feature is a prompt may be modified and submitted again if the output does not meet the standards of the user. Here are some example prompts:
- Write a letter of medical necessity for FGFR3 genetic testing for a 2-year-old female patient with short stature, short limbs, macrocephaly, and frontal bossing; include literature found in PubMed.
- Write a discharge summary for Jane Doe, a 62-year-old female who had a left knee replacement 4 days ago without complication and now is being discharged to go home.
- Summarize the risk for cancer for a 30-year-old male with a BRCA2 pathogenic variant.
The applications of ChatGPT are tremendous in the healthcare space. One area of interest is ChatGPTās ability to pass different professional licensing and certification exams. These professional exams take years of training and study for humans to pass. Interestingly, ChatGPT has passed sections of the United States Medical Licensing Exam (USMLE) and practice medical board exams from AMBOSS and the National Board of Medical Examiners. The LLM has also performed similar to humans on genetics board exam preparation questions (68% vs. 67% respectively). The software scored higher for memorization questions compared with critical thinking questions. Although this is impressive, there are still limitations to implement ChatGPT clinically in a real-world environment. Furthermore, ChatGPTās performance has been studied for the BAR exam and the University of Pennsylvaniaās Wharton MBA exam, among others.
The incorporation of ChatGPT into clinical care could increase efficiency for certain workflows. For example, it can write discharge notes based on the patientās clinical history and medical records at a much faster rate than a physician can dictate. ChatGPT can also write clinic notes, patient letters, and letters of medical necessity with a high degree of accuracy and are comparable to the grammar and prose of a human author (also known as āhumannessā). Some speculate that ChatGPT, or some other LLM could improve disease management, drug discovery, and translational research.
There are over 300,000 CLIA-certified clinical labs in the United States, which collectively produce massive amounts of data. Logically, ChatGPT integration into the clinical laboratory is being studied by multiple groups. The European Federation of Clinical Chemistry and Laboratory Medicine created 10 artificial common lab reports and prompted to ChatGPT for interpretation, and results were reviewed by medical professionals. ChatGPTās output had high relevance and safety. It never recommended a course of treatment and always referred the imaginary patient back to their imaginary doctor. Conversely, it could not differentiate between slightly and severely out of range lab values, could not consider preanalytical issues, and some answers were misleading.
ChatGPT is not without its shortcomings and many in the medical community are recommending caution. One laboratory group posed 65 questions to ChatGPT that included topics such as toxicology, microbiology, coagulation, and laboratory management. The LLM correctly answered only 50.7% of questions. Furthermore, it also generated incomplete (23.1%), misleading (16.9%), and irrelevant (9.3%) answers. Furthermore, ChatGPT may not be consistent in its answers to the same prompt, which is a considerable liability in medical practice. There may be legal issues if there are medical errors based on incorrect output from ChatGPT. As with other technologies, privacy, security, and regulation must be addressed.
ChatGPT represents a substantial upgrade in AI chatbot technology that is available to the general population. The LLM truly is a disruptive innovation and can be compared to the global migration of paper books and records to the Internet as the main source people use for information. Science and medicine must address not if, but how, ChatGPT (or similar LLM AI) will be integrated into medical practice, laboratory testing, and research. The benefits of integrating ChatGPT seem limitless. However, professional medical associations and regulators must work with the companies that own these proprietary technologies in order to promote proper, ethical, and reliable healthcare for all.
References
Bommarito, Michael James and Katz, Daniel Martin, GPT Takes the Bar Exam (December 29, 2022).
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4314839
Cadamuro J, Cabitza F, Debeljak Z, De Bruyne S, Frans G, Perez SM, Ozdemir H, Tolios A, Carobene A, Padoan A. Potentials and pitfalls of ChatGPT and natural-language artificial intelligence models for the understanding of laboratory medicine test results. An assessment by the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Working Group on Artificial Intelligence (WG-AI). Clin Chem Lab Med. 2023 Apr 24;61(7):1158-1166. PMID: 37083166.
https://pubmed.ncbi.nlm.nih.gov/37083166/
Irwin P, Jones D, Fealy S. What is ChatGPT and what do we do with it? Implications of the age of AI for nursing and midwifery practice and education: An editorial. Nurse Educ Today. 2023 Aug;127:105835. PMID: 37267643.
https://pubmed.ncbi.nlm.nih.gov/37267643/
Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor RA, Chartash D. How Does ChatGPT Perform on the United States Medical Licensing Examination? The Implications of Large Language Models for Medical Education and Knowledge Assessment. JMIR Med Educ. 2023 Feb 8;9:e45312. PMID: 36753318.
https://pubmed.ncbi.nlm.nih.gov/36753318/
Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, ElepaƱo C, Madriaga M, Aggabao R, Diaz-Candido G, Maningo J, Tseng V. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023 Feb 9;2(2):e0000198. PMID: 36812645.
https://pubmed.ncbi.nlm.nih.gov/36812645/
Munoz-Zuluaga C, Zhao Z, Wang F, Greenblatt MB, Yang HS. Assessing the Accuracy and Clinical Utility of ChatGPT in Laboratory Medicine. Clin Chem. 2023 May 26:hvad058. doi: 10.1093/clinchem/hvad058. Epub ahead of print. PMID: 37231970.
https://pubmed.ncbi.nlm.nih.gov/37231970/
Patel SB, Lam K. ChatGPT: the future of discharge summaries? Lancet Digit Health. 2023 Mar;5(3):e107-e108. PMID: 36754724.
https://pubmed.ncbi.nlm.nih.gov/36754724/
Sallam M. ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns. Healthcare (Basel). 2023 Mar 19;11(6):887. PMID: 36981544.
https://pubmed.ncbi.nlm.nih.gov/36981544/
Ali SR, Dobbs TD, Hutchings HA, Whitaker IS. Using ChatGPT to write patient clinic letters. Lancet Digit Health. 2023 Apr;5(4):e179-e181. PMID: 36894409.
https://pubmed.ncbi.nlm.nih.gov/36894409/
2022 U.S. Laboratory Market Report: Improved Service Capabilities and Higher Rates of Chronic Disease Drive Growth – ResearchAndMarkets.com. Accessed 06/26/2023.
59+ ChatGPT Statistics ā Users, Trends & More (June 2023). Demand Sage. Accessed 06/25/2023. https://www.demandsage.com/chatgpt-statistics/
Introducing ChatGPT. OpenAI. Accessed 06/25/2023. https://openai.com/blog/chatgpt
Would Chat GPT Get a Wharton MBA? University of Pennsylvania. Accessed 06/25/2023.