This paper is in the following e-collection/theme issue:

Generative Language Models Including ChatGPT (201) Artificial Intelligence (1186) Clinical Information and Decision Making (1314) Chatbots and Conversational Agents (458) Decision Support for Health Professionals (1127)

Published on in Vol 25 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/48659, first published .
Assessing the Utility of ChatGPT Throughout the Entire Clinical Workflow: Development and Usability Study

Assessing the Utility of ChatGPT Throughout the Entire Clinical Workflow: Development and Usability Study

Assessing the Utility of ChatGPT Throughout the Entire Clinical Workflow: Development and Usability Study

Authors of this article:

Arya Rao1, 2, 3 Author Orcid Image ;   Michael Pang1, 2, 3 Author Orcid Image ;   John Kim1, 2, 3 Author Orcid Image ;   Meghana Kamineni1, 2, 3 Author Orcid Image ;   Winston Lie1, 2, 3 Author Orcid Image ;   Anoop K Prasad1, 2, 3 Author Orcid Image ;   Adam Landman2, 4 Author Orcid Image ;   Keith Dreyer2, 5 Author Orcid Image ;   Marc D Succi1, 2, 3, 6 Author Orcid Image
Article Authors Cited by (59) Tweetations (33) Metrics

Journals

  1. El Haj M, Boutoleau-Bretonnière C, Chapelet G. ChatGPT's dance with neuropsychological data: A case study in Alzheimer’s disease. Ageing Research Reviews 2023;92:102117 View
  2. Benedicenti F, Pessarelli T, Corradi M, Michelon M, Nandi N, Lampertico P, Vecchi M, Scaramella L, Elli L. Mirror, mirror on the wall, who is the best of them all? Artificial intelligence versus gastroenterologists in solving clinical problems. Gastroenterology Report 2022;11 View
  3. Weiner S, Wakeman S. Stigma in Large Language Models: A Chatbot Responds. Journal of Addiction Medicine 2024;18(1):90 View
  4. Liu J, Liu F, Fang J, Liu S. The application of Chat Generative Pre-trained Transformer in nursing education. Nursing Outlook 2023;71(6):102064 View
  5. Griewing S, Gremke N, Wagner U, Lingenfelder M, Kuhn S, Boekhoff J. Challenging ChatGPT 3.5 in Senology—An Assessment of Concordance with Breast Cancer Tumor Board Decision Making. Journal of Personalized Medicine 2023;13(10):1502 View
  6. Bushuven S, Bentele M, Bentele S, Gerber B, Bansbach J, Ganter J, Trifunovic-Koenig M, Ranisch R. “ChatGPT, Can You Help Me Save My Child’s Life?” - Diagnostic Accuracy and Supportive Capabilities to Lay Rescuers by ChatGPT in Prehospital Basic Life Support and Paediatric Advanced Life Support Cases – An In-silico Analysis. Journal of Medical Systems 2023;47(1) View
  7. Guillen-Grima F, Guillen-Aguinaga S, Guillen-Aguinaga L, Alas-Brun R, Onambele L, Ortega W, Montejo R, Aguinaga-Ontoso E, Barach P, Aguinaga-Ontoso I. Evaluating the Efficacy of ChatGPT in Navigating the Spanish Medical Residency Entrance Examination (MIR): Promising Horizons for AI in Clinical Medicine. Clinics and Practice 2023;13(6):1460 View
  8. Li J. Security Implications of AI Chatbots in Health Care. Journal of Medical Internet Research 2023;25:e47551 View
  9. Maursetter L. Will ChatGPT Be the Next Nephrologist?. Clinical Journal of the American Society of Nephrology 2024;19(1):2 View
  10. Chen J, Liu L, Ruan S, Li M, Yin C. Are Different Versions of ChatGPT’s Ability Comparable to the Clinical Diagnosis Presented in Case Reports? A Descriptive Study. Journal of Multidisciplinary Healthcare 2023;Volume 16:3825 View
  11. Zawiah M, Al-Ashwal F, Gharaibeh L, Abu Farha R, Alzoubi K, Abu Hammour K, Qasim Q, Abrah F. ChatGPT and Clinical Training: Perception, Concerns, and Practice of Pharm-D Students. Journal of Multidisciplinary Healthcare 2023;Volume 16:4099 View
  12. Alotaibi S, Rehman A, Hasnain M. Revolutionizing ocular cancer management: a narrative review on exploring the potential role of ChatGPT. Frontiers in Public Health 2023;11 View
  13. Non L. All aboard the ChatGPT steamroller: Top 10 ways to make artificial intelligence work for healthcare professionals. Antimicrobial Stewardship & Healthcare Epidemiology 2023;3(1) View
  14. Ćirković A, Katz T. Exploring the Potential of ChatGPT-4 in Predicting Refractive Surgery Categorizations: Comparative Study. JMIR Formative Research 2023;7:e51798 View
  15. Sezgin E. Redefining Virtual Assistants in Health Care: The Future With Large Language Models. Journal of Medical Internet Research 2024;26:e53225 View
  16. Piryani R, Piryani S, Piryani R. Medical schools of South Asian countries need to incorporate artificial intelligence-powered chatbots in existing undergraduate medical curricula. Journal of Integrative Medicine and Public Health 2023;2(2):37 View
  17. Sallam M, Barakat M, Sallam M. METRICS: Establishing a Preliminary Checklist to Standardize the Design and Reporting of Generative Artificial Intelligence-Based Studies in Healthcare Education and Practice (Preprint). Interactive Journal of Medical Research 2023 View
  18. Andrew A. Potential applications and implications of large language models in primary care. Family Medicine and Community Health 2024;12(Suppl 1):e002602 View
  19. Robleto E, Habashi A, Kaplan M, Riley R, Zhang C, Bianchi L, Shehadeh L. Medical students’ perceptions of an artificial intelligence (AI) assisted diagnosing program. Medical Teacher 2024:1 View
  20. Yu J, Matava C. ChatGPT for Parents of Children Seeking Emergency Care – so much Hope, so much Caution. Journal of Medical Systems 2024;48(1) View
  21. Arslan B, Eyupoglu G, Korkut S, Turkdogan K, Altinbilek E. The accuracy of AI-assisted chatbots on the annual assessment test for emergency medicine residents. Journal of Medicine, Surgery, and Public Health 2024;3:100070 View
  22. Teixeira-Marques F, Medeiros N, Nazaré F, Alves S, Lima N, Ribeiro L, Gama R, Oliveira P. Exploring the role of ChatGPT in clinical decision-making in otorhinolaryngology: a ChatGPT designed study. European Archives of Oto-Rhino-Laryngology 2024;281(4):2023 View
  23. Milad D, Antaki F, Milad J, Farah A, Khairy T, Mikhail D, Giguère C, Touma S, Bernstein A, Szigiato A, Nayman T, Mullie G, Duval R. Assessing the medical reasoning skills of GPT-4 in complex ophthalmology cases. British Journal of Ophthalmology 2024:bjo-2023-325053 View
  24. Rudroff T. Revealing the Complexity of Fatigue: A Review of the Persistent Challenges and Promises of Artificial Intelligence. Brain Sciences 2024;14(2):186 View
  25. Ward M, Unadkat P, Toscano D, Kashanian A, Lynch D, Horn A, D'Amico R, Mittler M, Baum G. A Quantitative Assessment of ChatGPT as a Neurosurgical Triaging Tool. Neurosurgery 2024 View
  26. Liu D, Mazurek M, Whitehead D, Hood M, Choi P, Gupte A, Ottensmeyer M, Fintelmann F, Uppot R, Andriole K, Gee M, Brink J, Succi M. A Novel Design-Thinking, Hospital Innovation Core Certificate Curriculum for Radiologists and Trainees: Creation, Implementation, and Multiyear Results. Academic Radiology 2024;31(2):417 View
  27. Moreno A, Bitterman D. Toward Clinical-Grade Evaluation of Large Language Models. International Journal of Radiation Oncology*Biology*Physics 2024;118(4):916 View
  28. Lim A, Nicolaides A. Becoming-with-AI: Rethinking Professional Knowledge Through Generative Knowing. Reflective Practice 2024;25(3):391 View
  29. Burns J, Gichoya J, Kohli M, Jones J, Purkayastha S, Liang H. Theory of radiologist interaction with instant messaging decision support tools: A sequential-explanatory study. PLOS Digital Health 2024;3(2):e0000297 View
  30. Ghanem Y, Rouhi A, Al-Houssan A, Saleh Z, Moccia M, Joshi H, Dumon K, Hong Y, Spitz F, Joshi A, Kwiatt M. Dr. Google to Dr. ChatGPT: assessing the content and quality of artificial intelligence-generated medical information on appendicitis. Surgical Endoscopy 2024;38(5):2887 View
  31. Al Tibi G, Alexander M, Miller S, Chronos N. A Retrospective Comparison of Medication Recommendations Between a Cardiologist and ChatGPT-4 for Hypertension Patients in a Rural Clinic. Cureus 2024 View
  32. Beaulieu-Jones B, Berrigan M, Shah S, Marwaha J, Lai S, Brat G. Evaluating capabilities of large language models: Performance of GPT-4 on surgical knowledge assessments. Surgery 2024;175(4):936 View
  33. Sebo P. What Is the Performance of ChatGPT in Determining the Gender of Individuals Based on Their First and Last Names?. JMIR AI 2024;3:e53656 View
  34. El Haj M, Boutoleau-Bretonnière C, Gallouj K, Wagemann N, Antoine P, Kapogiannis D, Chapelet G. ChatGPT as a Diagnostic Aid in Alzheimer’s Disease: An Exploratory Study. Journal of Alzheimer's Disease Reports 2024;8(1):495 View
  35. Alexopoulos G. Artificial Intelligence and the Sense of Self of Older Adults: A Philosophy of Science Perspective. The American Journal of Geriatric Psychiatry 2024;32(6):657 View
  36. Warrier A, Singh R, Haleem A, Zaki H, Eloy J. The Comparative Diagnostic Capability of Large Language Models in Otolaryngology. The Laryngoscope 2024 View
  37. Spotnitz M, Idnay B, Gordon E, Shyu R, Zhang G, Liu C, Cimino J, Weng C. A Survey of Clinicians' Views of the Utility of Large Language Models. Applied Clinical Informatics 2024;15(02):306 View
  38. Rosselló-Jiménez D, Docampo S, Collado Y, Cuadra-Llopart L, Riba F, Llonch-Masriera M. Geriatrics and artificial intelligence in Spain (Ger-IA project): talking to ChatGPT, a nationwide survey. European Geriatric Medicine 2024 View
  39. Turan E, Baydemir A, Özcan F, Şahin A. Evaluating the accuracy of ChatGPT-4 in predicting ASA scores: A prospective multicentric study ChatGPT-4 in ASA score prediction. Journal of Clinical Anesthesia 2024;96:111475 View
  40. Quttainah M, Mishra V, Madakam S, Lurie Y, Mark S. Cost, Usability, Credibility, Fairness, Accountability, Transparency, and Explainability Framework for Safe and Effective Large Language Models in Medical Education: Narrative Review and Qualitative Study. JMIR AI 2024;3:e51834 View
  41. Rojas A. An Investigation into ChatGPT’s Application for a Scientific Writing Assignment. Journal of Chemical Education 2024;101(5):1959 View
  42. Vaira L, Lechien J, Abbate V, Allevi F, Audino G, Beltramini G, Bergonzani M, Boscolo-Rizzo P, Califano G, Cammaroto G, Chiesa-Estomba C, Committeri U, Crimi S, Curran N, di Bello F, di Stadio A, Frosolini A, Gabriele G, Gengler I, Lonardi F, Maglitto F, Mayo-Yáñez M, Petrocelli M, Pucci R, Saibene A, Saponaro G, Tel A, Trabalzini F, Trecca E, Vellone V, Salzano G, De Riu G. Validation of the Quality Analysis of Medical Artificial Intelligence (QAMAI) tool: a new tool to assess the quality of health information provided by AI platforms. European Archives of Oto-Rhino-Laryngology 2024 View
  43. Chiu W, Ko W, Cho W, Hui S, Chan W, Kuo M. Evaluating the Diagnostic Performance of Large Language Models on Complex Multimodal Medical Cases. Journal of Medical Internet Research 2024;26:e53724 View
  44. Scott I, Zuccon G. The new paradigm in machine learning – foundation models, large language models and beyond: a primer for physicians. Internal Medicine Journal 2024;54(5):705 View
  45. Shikino K, Shimizu T, Otsuka Y, Tago M, Takahashi H, Watari T, Sasaki Y, Iizuka G, Tamura H, Nakashima K, Kunitomo K, Suzuki M, Aoyama S, Kosaka S, Kawahigashi T, Matsumoto T, Orihara F, Morikawa T, Nishizawa T, Hoshina Y, Yamamoto Y, Matsuo Y, Unoki Y, Kimura H, Tokushima M, Watanuki S, Saito T, Otsuka F, Tokuda Y. Evaluation of ChatGPT-Generated Differential Diagnosis for Common Diseases With Atypical Presentation: Descriptive Research. JMIR Medical Education 2024;10:e58758 View
  46. Buldur M, Sezer B. Evaluating the accuracy of Chat Generative Pre-trained Transformer version 4 (ChatGPT-4) responses to United States Food and Drug Administration (FDA) frequently asked questions about dental amalgam. BMC Oral Health 2024;24(1) View
  47. Kaneda Y, Tayuinosho A, Tomoyose R, Takita M, Hamaki T, Tanimoto T, Ozaki A. Evaluating ChatGPT's effectiveness and tendencies in Japanese internal medicine. Journal of Evaluation in Clinical Practice 2024 View
  48. Petrella R. The AI Future of Emergency Medicine. Annals of Emergency Medicine 2024 View
  49. MEKOV Е, YAMAKOVA Y, KHAIRALLAH R, PETKOV R. ChatGPT in medical practice: what we could, what we should, and what we would. Gazzetta Medica Italiana Archivio per le Scienze Mediche 2024;183(3) View
  50. Nasef H, Patel H, Amin Q, Baum S, Ratnasekera A, Ang D, Havron W, Nakayama D, Elkbuli A. Evaluating the Accuracy, Comprehensiveness, and Validity of ChatGPT Compared to Evidence-Based Sources Regarding Common Surgical Conditions: Surgeons’ Perspectives. The American Surgeon™ 2024 View
  51. Munir F, Gehres A, Wai D, Song L. Evaluation of ChatGPT as a Tool for Answering Clinical Questions in Pharmacy Practice. Journal of Pharmacy Practice 2024 View
  52. Griewing S, Knitza J, Boekhoff J, Hillen C, Lechner F, Wagner U, Wallwiener M, Kuhn S. Evolution of publicly available large language models for complex decision-making in breast cancer care. Archives of Gynecology and Obstetrics 2024;310(1):537 View
  53. Harada Y, Suzuki T, Harada T, Sakamoto T, Ishizuka K, Miyagami T, Kawamura R, Kunitomo K, Nagano H, Shimizu T, Watari T. Performance evaluation of ChatGPT in detecting diagnostic errors and their contributing factors: an analysis of 545 case reports of diagnostic errors. BMJ Open Quality 2024;13(2):e002654 View
  54. Li J, Tang T, Wu E, Zhao J, Zong H, Wu R, Feng W, Zhang K, Wang D, Qin Y, Shen Z, Qin Y, Ren S, Zhan C, Yang L, Wei Q, Shen B. RARPKB: a knowledge-guide decision support platform for personalized robot-assisted surgery in prostate cancer. International Journal of Surgery 2024;110(6):3412 View