Published on in Vol 25 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/47479, first published .
Reliability of Medical Information Provided by ChatGPT: Assessment Against Clinical Guidelines and Patient Information Quality Instrument

Reliability of Medical Information Provided by ChatGPT: Assessment Against Clinical Guidelines and Patient Information Quality Instrument

Reliability of Medical Information Provided by ChatGPT: Assessment Against Clinical Guidelines and Patient Information Quality Instrument

Journals

  1. Levkovich I, Elyoseph Z. Suicide Risk Assessments Through the Eyes of ChatGPT-3.5 Versus ChatGPT-4: Vignette Study. JMIR Mental Health 2023;10:e51232 View
  2. ALİMEN N. Makine çevirisinden sohbet robotu çevirisine: ChatGPT ile deneysel bir çalışma. RumeliDE Dil ve Edebiyat Araştırmaları Dergisi 2023;(36):1532 View
  3. Hu J, Liu F, Chu C, Chang Y. Health Care Trainees’ and Professionals’ Perceptions of ChatGPT in Improving Medical Knowledge Training: Rapid Survey Study. Journal of Medical Internet Research 2023;25:e49385 View
  4. Abuyaman O. Strengths and Weaknesses of ChatGPT Models for Scientific Writing About Medical Vitamin B12: Mixed Methods Study. JMIR Formative Research 2023;7:e49459 View
  5. Mese I, Taslicay C, Sivrioglu A. Improving radiology workflow using ChatGPT and artificial intelligence. Clinical Imaging 2023;103:109993 View
  6. Orlando N, Qiu C, ElNemer W, Tuffaha S. Google Trends Analysis of Peripheral Nerve Disease and Surgery. World Neurosurgery 2023;180:e135 View
  7. Gao Z, Li L, Ma S, Wang Q, Hemphill L, Xu R. Examining the Potential of ChatGPT on Biomedical Information Retrieval: Fact-Checking Drug-Disease Associations. Annals of Biomedical Engineering 2024;52(8):1919 View
  8. Singh A, Das S, Mishra R, Agrawal A. Artificial intelligence and machine learning in healthcare: Scope and opportunities to use ChatGPT. Journal of Neurosciences in Rural Practice 2023;14:391 View
  9. Barrington N, Gupta N, Musmar B, Doyle D, Panico N, Godbole N, Reardon T, D’Amico R. A Bibliometric Analysis of the Rise of ChatGPT in Medical Research. Medical Sciences 2023;11(3):61 View
  10. Ayre J, Mac O, McCaffery K, McKay B, Liu M, Shi Y, Rezwan A, Dunn A. New Frontiers in Health Literacy: Using ChatGPT to Simplify Health Information for People in the Community. Journal of General Internal Medicine 2024;39(4):573 View
  11. Mondal H, Dash I, Mondal S, Behera J. ChatGPT in Answering Queries Related to Lifestyle-Related Diseases and Disorders. Cureus 2023 View
  12. Hernandez C, Vazquez Gonzalez A, Polianovskaia A, Amoro Sanchez R, Muyolema Arce V, Mustafa A, Vypritskaya E, Perez Gutierrez O, Bashir M, Eighaei Sedeh A. The Future of Patient Education: AI-Driven Guide for Type 2 Diabetes. Cureus 2023 View
  13. Scquizzato T, Semeraro F, Swindell P, Simpson R, Angelini M, Gazzato A, Sajjad U, Bignami E, Landoni G, Keeble T, Mion M. Testing ChatGPT ability to answer laypeople questions about cardiac arrest and cardiopulmonary resuscitation. Resuscitation 2024;194:110077 View
  14. Mohammad‐Rahimi H, Ourang S, Pourhoseingholi M, Dianat O, Dummer P, Nosrat A. Validity and reliability of artificial intelligence chatbots as public sources of information on endodontics. International Endodontic Journal 2024;57(3):305 View
  15. Yurdakurban E, Topsakal K, Duran G. A comparative analysis of AI-based chatbots: Assessing data quality in orthognathic surgery related patient information. Journal of Stomatology, Oral and Maxillofacial Surgery 2024;125(5):101757 View
  16. Feng R, Zhang C, Zhang Y. Large language models for biomolecular analysis: From methods to applications. TrAC Trends in Analytical Chemistry 2024;171:117540 View
  17. Blease C, Worthen A, Torous J. Psychiatrists’ experiences and opinions of generative artificial intelligence in mental healthcare: An online mixed methods survey. Psychiatry Research 2024;333:115724 View
  18. Patel H, Zanos T, Hewitt D. Deep Learning Applications in Pancreatic Cancer. Cancers 2024;16(2):436 View
  19. Yang J, Ardavanis K, Slack K, Fernando N, Della Valle C, Hernandez N. Chat Generative Pretrained Transformer (ChatGPT) and Bard: Artificial Intelligence Does not yet Provide Clinically Supported Answers for Hip and Knee Osteoarthritis. The Journal of Arthroplasty 2024;39(5):1184 View
  20. Jin X, Frock A, Nagaraja S, Wallqvist A, Reifman J. AI algorithm for personalized resource allocation and treatment of hemorrhage casualties. Frontiers in Physiology 2024;15 View
  21. Sallam M, Barakat M, Sallam M. METRICS: Establishing a Preliminary Checklist to Standardize the Design and Reporting of Generative Artificial Intelligence-Based Studies in Healthcare Education and Practice (Preprint). Interactive Journal of Medical Research 2023 View
  22. Dyckhoff-Shen S, Koedel U, Brouwer M, Bodilsen J, Klein M. ChatGPT fails challenging the recent ESCMID brain abscess guideline. Journal of Neurology 2024;271(4):2086 View
  23. Mohammad-Rahimi H, Khoury Z, Alamdari M, Rokhshad R, Motie P, Parsa A, Tavares T, Sciubba J, Price J, Sultan A. Performance of AI chatbots on controversial topics in oral medicine, pathology, and radiology. Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology 2024;137(5):508 View
  24. Kumar S, Rao P, Singhania S, Verma S, Kheterpal M. Will artificial intelligence drive the advancements in higher education? A tri-phased exploration. Technological Forecasting and Social Change 2024;201:123258 View
  25. McMahon H, McMahon B. Automating untruths: ChatGPT, self-managed medication abortion, and the threat of misinformation in a post-Roe world. Frontiers in Digital Health 2024;6 View
  26. Eleiwa T, Elhusseiny A. Re: Kianian et al.: Enhancing the assessment of large language models in medical information generation (Ophthalmol Retina. 2024;8:195-201). Ophthalmology Retina 2024;8(5):e15 View
  27. Mishra V, Jafri F, Abdul Kareem N, Aboobacker R, Noora F. Evaluation of accuracy and potential harm of ChatGPT in medical nutrition therapy - a case-based approach. F1000Research 2024;13:137 View
  28. Wang L, Chen X, Deng X, Wen H, You M, Liu W, Li Q, Li J. Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs. npj Digital Medicine 2024;7(1) View
  29. Bektaş M, Pereira J, Daams F, van der Peet D. ChatGPT in surgery: a revolutionary innovation?. Surgery Today 2024;54(8):964 View
  30. Fuchs A, Trachsel T, Weiger R, Eggmann F. ChatGPT’s performance in dentistry and allergyimmunology assessments: a comparative study. SWISS DENTAL JOURNAL SSO – Science and Clinical Topics 2023;134(2):1 View
  31. Topsakal O, Sawyer P, Akinci T, Topsakal E, Celikoyar M. Reliability and Agreement of Free Web-Based 3D Software for Computing Facial Area and Volume Measurements. BioMedInformatics 2024;4(1):690 View
  32. Mu Y, He D. The Potential Applications and Challenges of ChatGPT in the Medical Field. International Journal of General Medicine 2024;Volume 17:817 View
  33. Çoban E, Altay B. Assessing the Potential Role of Artificial Intelligence in Medication-Related Osteonecrosis of the Jaw Information Sharing. Journal of Oral and Maxillofacial Surgery 2024;82(6):699 View
  34. Chen Y, Esmaeilzadeh P. Generative AI in Medical Practice: In-Depth Exploration of Privacy and Security Challenges. Journal of Medical Internet Research 2024;26:e53008 View
  35. Chow J, Wong V, Li K. Generative Pre-Trained Transformer-Empowered Healthcare Conversations: Current Trends, Challenges, and Future Directions in Large Language Model-Enabled Medical Chatbots. BioMedInformatics 2024;4(1):837 View
  36. Kernberg A, Gold J, Mohan V. Using ChatGPT-4 to Create Structured Medical Notes From Audio Recordings of Physician-Patient Encounters: Comparative Study. Journal of Medical Internet Research 2024;26:e54419 View
  37. Staubli S, Jobeir B, Spiro M, Raptis D. Invitation to join the Healthcare AI Language Group: HeALgroup.AI Initiative. BMJ Health & Care Informatics 2024;31(1):e100884 View
  38. Mastrokostas P, Mastrokostas L, Emara A, Wellington I, Ginalis E, Houten J, Khalsa A, Saleh A, Razi A, Ng M. GPT-4 as a Source of Patient Information for Anterior Cervical Discectomy and Fusion: A Comparative Analysis Against Google Web Search. Global Spine Journal 2024;14(8):2389 View
  39. Parikh A, Oca M, Conger J, McCoy A, Chang J, Zhang-Nunes S. Accuracy and Bias in Artificial Intelligence Chatbot Recommendations for Oculoplastic Surgeons. Cureus 2024 View
  40. Hanai A, Ishikawa T, Kawauchi S, Iida Y, Kawakami E. Generative artificial intelligence and non-pharmacological bias: an experimental study on cancer patient sexual health communications. BMJ Health & Care Informatics 2024;31(1):e100924 View
  41. Shiraishi M, Tanigawa K, Tomioka Y, Miyakuni A, Moriwaki Y, Yang R, Oba J, Okazaki M. Blepharoptosis Consultation with Artificial Intelligence: Aesthetic Surgery Advice and Counseling from Chat Generative Pre-Trained Transformer (ChatGPT). Aesthetic Plastic Surgery 2024;48(11):2057 View
  42. Luo S, Canavese F, Aroojis A, Andreacchio A, Anticevic D, Bouchard M, Castaneda P, De Rosa V, Fiogbe M, Frick S, Hui J, Johari A, Loro A, Lyu X, Matsushita M, Omeroglu H, Roye D, Shah M, Yong B, Li L. Are Generative Pretrained Transformer 4 Responses to Developmental Dysplasia of the Hip Clinical Scenarios Universal? An International Review. Journal of Pediatric Orthopaedics 2024;44(6):e504 View
  43. Deng L, Wang T, Yangzhang , Zhai Z, Tao W, Li J, Zhao Y, Luo S, Xu J. Evaluation of large language models in breast cancer clinical scenarios: a comparative analysis based on ChatGPT-3.5, ChatGPT-4.0, and Claude2. International Journal of Surgery 2024;110(4):1941 View
  44. Huo B, Calabrese E, Sylla P, Kumar S, Ignacio R, Oviedo R, Hassan I, Slater B, Kaiser A, Walsh D, Vosburg W. The performance of artificial intelligence large language model-linked chatbots in surgical decision-making for gastroesophageal reflux disease. Surgical Endoscopy 2024;38(5):2320 View
  45. Guo S, Li R, Li G, Chen W, Huang J, He L, Ma Y, Wang L, Zheng H, Tian C, Zhao Y, Pan X, Wan H, Liu D, Li Z, Lei J. Comparing ChatGPT's and Surgeon's Responses to Thyroid-related Questions From Patients. The Journal of Clinical Endocrinology & Metabolism 2024 View
  46. Amaral J, Schultz R, Martin B, Taylor T, Touban B, McGraw-Heinrich J, McKay S, Rosenfeld S, Smith B. Evaluating Chat Generative Pre-trained Transformer Responses to Common Pediatric In-toeing Questions. Journal of Pediatric Orthopaedics 2024;44(7):e592 View
  47. Moulaei K, Yadegari A, Baharestani M, Farzanbakhsh S, Sabet B, Reza Afrash M. Generative artificial intelligence in healthcare: A scoping review on benefits, challenges and applications. International Journal of Medical Informatics 2024;188:105474 View
  48. Grimm D, Lee Y, Hu K, Liu L, Garcia O, Balakrishnan K, Ayoub N. The utility of ChatGPT as a generative medical translator. European Archives of Oto-Rhino-Laryngology 2024;281(11):6161 View
  49. Gibson D, Jackson S, Shanmugasundaram R, Seth I, Siu A, Ahmadi N, Kam J, Mehan N, Thanigasalam R, Jeffery N, Patel M, Leslie S. Evaluating the Efficacy of ChatGPT as a Patient Education Tool in Prostate Cancer: Multimetric Assessment. Journal of Medical Internet Research 2024;26:e55939 View
  50. Ozden I, Gokyar M, Ozden M, Sazak Ovecoglu H. Assessment of artificial intelligence applications in responding to dental trauma. Dental Traumatology 2024;40(6):722 View
  51. Buldur M, Sezer B. Evaluating the accuracy of Chat Generative Pre-trained Transformer version 4 (ChatGPT-4) responses to United States Food and Drug Administration (FDA) frequently asked questions about dental amalgam. BMC Oral Health 2024;24(1) View
  52. Bhattaru A, Yanamala N, Sengupta P. Revolutionizing Cardiology With Words: Unveiling the Impact of Large Language Models in Medical Science Writing. Canadian Journal of Cardiology 2024;40(10):1950 View
  53. Thongsri N, Tripak O, Bao Y. Do learners exhibit a willingness to use ChatGPT? An advanced two-stage SEM-neural network approach for forecasting factors influencing ChatGPT adoption. Interactive Technology and Smart Education 2024 View
  54. Min L, Fan Z, Dou F, Sun J, Luo C, Lv Q. Adaption BERT for Medical Information Processing with ChatGPT and Contrastive Learning. Electronics 2024;13(13):2431 View
  55. Akuffo-Addo E, Samman L, Munawar L, Akbik M, Kokikian N, Wescott R, Wu J. Assessing GPT-4’s diagnostic accuracy with darker skin tones: underperformance and implications. Clinical and Experimental Dermatology 2024;49(10):1244 View
  56. Huisman T, Huisman T. Artificial Intelligence in Newborn Medicine. Newborn 2024;3(2):96 View
  57. Abou Karam G. Revolutionizing Medical Education: ChatGPT3.5 Ability to Behave as a Virtual Patient. Medical Science Educator 2024 View
  58. Abou Chaar M, Grigsby-Rocca G, Huang M, Blackmon S. ChatGPT vs Expert-Guided Care Pathways for Postesophagectomy Symptom Management. Annals of Thoracic Surgery Short Reports 2024 View
  59. Han Y, Ceross A, Bourgeois F, Savaget P, Bergmann J. Evaluation of large language models for the classification of medical device software. Bio-Design and Manufacturing 2024;7(5):819 View
  60. Gencer A. Readability analysis of ChatGPT's responses on lung cancer. Scientific Reports 2024;14(1) View
  61. Schulz A, Bohnet-Joschko S. Enhancing patient informed consent in elective skin cancer surgeries: a comparative study of traditional and digital approaches in a German public hospital. BMC Health Services Research 2024;24(1) View
  62. Duran A, Cortuk O, Ok B. Future Perspective of Risk Prediction in Aesthetic Surgery: Is Artificial Intelligence Reliable?. Aesthetic Surgery Journal 2024;44(11):NP839 View
  63. Ahmed H, Thrishulamurthy C. Evaluating ChatGPT's efficacy and readability to common pediatric ophthalmology and strabismus-related questions. European Journal of Ophthalmology 2024 View
  64. Nazi Z, Peng W. Large Language Models in Healthcare and Medical Domain: A Review. Informatics 2024;11(3):57 View
  65. Tsai C, Cheng P, Deng J, Jaw F, Yii S. ChatGPT v4 outperforming v3.5 on cancer treatment recommendations in quality, clinical guideline, and expert opinion concordance. DIGITAL HEALTH 2024;10 View
  66. Venosa M, Calvisi V, Iademarco G, Romanini E, Ciminello E, Cerciello S, Logroscino G. Evaluation of the Quality of ChatGPT’s Responses to Top 20 Questions about Robotic Hip and Knee Arthroplasty: Findings, Perspectives and Critical Remarks on Healthcare Education. Prosthesis 2024;6(4):913 View
  67. Qiu J, Luo L, Zhou Y. Accuracy of ChatGPT3.5 in answering clinical questions on guidelines for severe acute pancreatitis. BMC Gastroenterology 2024;24(1) View
  68. Huo B, Marfo N, Sylla P, Calabrese E, Kumar S, Slater B, Walsh D, Vosburg W. Clinical artificial intelligence: teaching a large language model to generate recommendations that align with guidelines for the surgical management of GERD. Surgical Endoscopy 2024;38(10):5668 View
  69. Hancı V, Ergün B, Gül Ş, Uzun Ö, Erdemir İ, Hancı F. Assessment of readability, reliability, and quality of ChatGPT®, BARD®, Gemini®, Copilot®, Perplexity® responses on palliative care. Medicine 2024;103(33):e39305 View
  70. Kıyak Y. Beginner-Level Tips for Medical Educators: Guidance on Selection, Prompt Engineering, and the Use of Artificial Intelligence Chatbots. Medical Science Educator 2024 View
  71. Pirkle S, Yang J, Blumberg T. Do ChatGPT and Gemini Provide Appropriate Recommendations for Pediatric Orthopaedic Conditions?. Journal of Pediatric Orthopaedics 2024 View
  72. Dihan Q, Chauhan M, Eleiwa T, Brown A, Hassan A, Khodeiry M, Elsheikh R, Oke I, Nihalani B, VanderVeen D, Sallam A, Elhusseiny A. Large language models: a new frontier in paediatric cataract patient education. British Journal of Ophthalmology 2024;108(10):1470 View
  73. Reicher L, Lutsker G, Michaan N, Grisaru D, Laskov I. Exploring the role of artificial intelligence, large language models: Comparing patient‐focused information and clinical decision support capabilities to the gynecologic oncology guidelines. International Journal of Gynecology & Obstetrics 2024 View
  74. Abdelwahed N. Recognizing the Role of ChatGPT in Decision-Making and Recognition of Mental Health Disorders among Entrepreneurs. OBM Neurobiology 2024;08(03):1 View
  75. Rashid M, Atilgan N, Dobres J, Day S, Penkova V, Küçük M, Clapp S, Sawyer B. Humanizing AI in Education: A Readability Comparison of LLM and Human-Created Educational Content. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2024 View
  76. Yan W, Hu B, Liu Y, Li C, Song C. Does usage scenario matter? Investigating user perceptions, attitude and support for policies towards ChatGPT. Information Processing & Management 2024;61(6):103867 View
  77. Paran M, Almog A, Dreznik Y, Nesher N, Kravarusic D. A New Era in Medical Information: ChatGPT Outperforms Medical Information Provided by Online Information Sheets About Congenital Malformations. Journal of Pediatric Surgery 2024:161894 View
  78. Tong L, Zhang C, Liu R, Yang J, Sun Z. Comparative performance analysis of large language models: ChatGPT-3.5, ChatGPT-4 and Google Gemini in glucocorticoid-induced osteoporosis. Journal of Orthopaedic Surgery and Research 2024;19(1) View
  79. Bettoli V, Naldi L, Santoro E, Valetto M, Bolzon A, Cassalia F, Cazzaniga S, Cima S, Danese A, Emendi S, Ponzano M, Scarpa N, Dri P. ChatGPT and acne: Accuracy and reliability of the information provided—The AI‐check study. Journal of the European Academy of Dermatology and Venereology 2024 View
  80. Chow J, Li K. Ethical Considerations in Human-Centered AI: Advancing Oncology Chatbots Through Large Language Models. JMIR Bioinformatics and Biotechnology 2024;5:e64406 View
  81. Wang L, Wan Z, Ni C, Song Q, Li Y, Clayton E, Malin B, Yin Z. Applications and Concerns of ChatGPT and Other Conversational Large Language Models in Health Care: Systematic Review. Journal of Medical Internet Research 2024;26:e22769 View
  82. Fan K, Fan K. Dermatological Knowledge and Image Analysis Performance of Large Language Models Based on Specialty Certificate Examination in Dermatology. Dermato 2024;4(4):124 View
  83. Kim H, Yoon P, Yoon J, Kim H, Choi Y, Park S, Moon J. Discrepancies in ChatGPT’s Hip Fracture Recommendations in Older Adults for 2021 AAOS Evidence-Based Guidelines. Journal of Clinical Medicine 2024;13(19):5971 View
  84. Piao Y, Chen H, Wu S, Li X, Li Z, Yang D. Assessing the performance of large language models (LLMs) in answering medical questions regarding breast cancer in the Chinese context. DIGITAL HEALTH 2024;10 View
  85. Shamil E, Ko T, Fan K, Schuster-Bruce J, Jaafar M, Khwaja S, Eynon-Lewis N, D'Souza A, Andrews P. Assessing the Quality and Readability of Online Patient Information: ENT UK Patient Information e-Leaflets versus Responses by a Generative Artificial Intelligence. Facial Plastic Surgery 2024 View
  86. Anees M, Shaikh F, Shaikh H, Siddiqui N, Rehman Z. Assessing the quality of ChatGPT's responses to questions related to radiofrequency ablation for varicose veins. Journal of Vascular Surgery: Venous and Lymphatic Disorders 2024:101985 View
  87. Spuur K, Currie G, Al-Mousa D, Pape R. Suitability of ChatGPT as a Source of Patient Information for Screening Mammography. Health Promotion Practice 2024 View
  88. Zhou S, Luo X, Chen C, Jiang H, Yang C, Ran G, Yu J, Yin C. The performance of large language model-powered chatbots compared to oncology physicians on colorectal cancer queries. International Journal of Surgery 2024;110(10):6509 View
  89. Johnson A, Singh T, Gupta A, Sankar H, Gill I, Shalini M, Mohan N. Evaluation of validity and reliability of AI Chatbots as public sources of information on dental trauma. Dental Traumatology 2024 View
  90. Abdulnazar A, Roller R, Schulz S, Kreuzthaler M. Large Language Models for Clinical Text Cleansing Enhance Medical Concept Normalization. IEEE Access 2024;12:147981 View
  91. Lois A, Yates R, Ivy M, Inaba C, Tatum R, Cetrulo L, Parr Z, Chen J, Khandelwal S, Wright A. Accuracy of natural language processors for patients seeking inguinal hernia information. Surgical Endoscopy 2024 View
  92. García-Rudolph A, Sanchez-Pinsach D, Opisso E. ChatGPT’s performance in the Specialist Health Practitioner exam for Hospital Emergency, responses from GPT-3.5 and GPT-4.0 to 150 multiple-choice questions. European Journal of Emergency Medicine 2024;31(6):438 View
  93. Kharko A, McMillan B, Hagström J, Muli I, Davidge G, Hägglund M, Blease C. Generative artificial intelligence writing open notes: A mixed methods assessment of the functionality of GPT 3.5 and GPT 4.0. DIGITAL HEALTH 2024;10 View
  94. Asfuroğlu Z, Yağar H, Gümüşoğlu E. High accuracy but limited readability of large language model-generated responses to frequently asked questions about Kienböck’s disease. BMC Musculoskeletal Disorders 2024;25(1) View
  95. Lang S, Vitale J, Galbusera F, Fekete T, Boissiere L, Charles Y, Yucekul A, Yilgor C, Núñez-Pereira S, Haddad S, Gomez-Rice A, Mehta J, Pizones J, Pellisé F, Obeid I, Alanay A, Kleinstück F, Loibl M. Is the information provided by large language models valid in educating patients about adolescent idiopathic scoliosis? An evaluation of content, clarity, and empathy. Spine Deformity 2024 View
  96. Youssef Y, Youssef S, Melcher P, Henkelmann R, Osterhoff G, Theopold J. How accurately can ChatGPT 3.5 answer frequently asked questions by patients on glenohumeral osteoarthritis?. Obere Extremität 2024 View