%0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e65681 %T Bridging Data Silos in Oncology with Modular Software for Federated Analysis on Fast Healthcare Interoperability Resources: Multisite Implementation Study %A Ziegler,Jasmin %A Erpenbeck,Marcel Pascal %A Fuchs,Timo %A Saibold,Anna %A Volkmer,Paul-Christian %A Schmidt,Guenter %A Eicher,Johanna %A Pallaoro,Peter %A De Souza Falguera,Renata %A Aubele,Fabio %A Hagedorn,Marlien %A Vansovich,Ekaterina %A Raffler,Johannes %A Ringshandl,Stephan %A Kerscher,Alexander %A Maurer,Julia Karolin %A Kühnel,Brigitte %A Schenkirsch,Gerhard %A Kampf,Marvin %A Kapsner,Lorenz A %A Ghanbarian,Hadieh %A Spengler,Helmut %A Soto-Rey,Iñaki %A Albashiti,Fady %A Hellwig,Dirk %A Ertl,Maximilian %A Fette,Georg %A Kraska,Detlef %A Boeker,Martin %A Prokosch,Hans-Ulrich %A Gulden,Christian %+ Medical Informatics, Friedrich-Alexander-Universität Erlangen-Nürnberg, Wetterkreuz 15, Erlangen, 91058, Germany, 49 91318526720, jasmin.ziegler@uk-erlangen.de %K real-world data %K real-world evidence %K oncology %K electronic health records %K federated analysis %K HL7 FHIR %K cancer registries %K interoperability %K observational research network %D 2025 %7 15.4.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Real-world data (RWD) from sources like administrative claims, electronic health records, and cancer registries offer insights into patient populations beyond the tightly regulated environment of randomized controlled trials. To leverage this and to advance cancer research, 6 university hospitals in Bavaria have established a joint research IT infrastructure. Objective: This study aimed to outline the design, implementation, and deployment of a modular data transformation pipeline that transforms oncological RWD into a Health Level 7 (HL7) Fast Healthcare Interoperability Resources (FHIR) format and then into a tabular format in preparation for a federated analysis (FA) across the 6 Bavarian Cancer Research Center university hospitals. Methods: To harness RWD effectively, we designed a pipeline to convert the oncological basic dataset (oBDS) into HL7 FHIR format and prepare it for FA. The pipeline handles diverse IT infrastructures and systems while maintaining privacy by keeping data decentralized for analysis. To assess the functionality and validity of our implementation, we defined a cohort to address two specific medical research questions. We evaluated our findings by comparing the results of the FA with reports from the Bavarian Cancer Registry and the original data from local tumor documentation systems. Results: We conducted an FA of 17,885 cancer cases from 2021/2022. Breast cancer was the most common diagnosis at 3 sites, prostate cancer ranked in the top 2 at 4 sites, and malignant melanoma was notably prevalent. Gender-specific trends showed larynx and esophagus cancers were more common in males, while breast and thyroid cancers were more frequent in females. Discrepancies between the Bavarian Cancer Registry and our data, such as higher rates of malignant melanoma (3400/63,771, 5.3% vs 1921/17,885, 10.7%) and lower representation of colorectal cancers (8100/63,771, 12.7% vs 1187/17,885, 6.6%) likely result from differences in the time periods analyzed (2019 vs 2021/2022) and the scope of data sources used. The Bavarian Cancer Registry reports approximately 3 times more cancer cases than the 6 university hospitals alone. Conclusions: The modular pipeline successfully transformed oncological RWD across 6 hospitals, and the federated approach preserved privacy while enabling comprehensive analysis. Future work will add support for recent oBDS versions, automate data quality checks, and integrate additional clinical data. Our findings highlight the potential of federated health data networks and lay the groundwork for future research that can leverage high-quality RWD, aiming to contribute valuable knowledge to the field of cancer research. %M 40233352 %R 10.2196/65681 %U https://www.jmir.org/2025/1/e65681 %U https://doi.org/10.2196/65681 %U http://www.ncbi.nlm.nih.gov/pubmed/40233352 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 13 %N %P e63665 %T Optimizing an Electronic Health Record System Used to Help Health Care Professionals Comply With a Standardized Care Pathway for Heart Failure During the Transition From Hospital To Chronic Care: Qualitative Semistructured Interview Study %A Font,Marta %A Davoody,Nadia %+ Department of Learning, Informatics, Management and Ethics, Health Informatics Centre, Karolinska Institutet, Tomtebodavägen 18a, Stockholm, SE-171 77, Sweden, 46 852486486, nadia.davoody@ki.se %K care pathway %K heart failure %K electronic health record %K sociotechnical system %K health care professional %D 2025 %7 15.4.2025 %9 Original Paper %J JMIR Med Inform %G English %X Background: In Spain, the prevalence of heart failure is twice the European average, partly due to inadequate patient management. To address this issue, a standardized care model, the Care Model for Patients with Heart Failure (Modelos Asistenciales de Atención al Paciente con Insuficiencia Cardíaca), was developed. This model emphasizes the importance of sequential visits from hospital discharge until the patient transitions to chronic care to prevent rehospitalization. The standardized care pathway has been implemented in certain areas of the Andalusia Health Service. However, there is uncertainty about whether the region’s electronic health record system, Diraya, can effectively support this model. If not properly integrated, it could lead to data inaccuracies and noncompliance with the standardized care pathway. Objective: This study aimed to explore how to improve Diraya to better support health care professionals in adhering to the transition standardized care model for patients with heart failure as they move from hospital care to chronic care. Methods: In total, 16 semistructured interviews were conducted with nurses and physicians from both hospital and primary care settings. Thematic analysis was used to analyze the data and recommendations for improvements that were developed based on the findings. These recommendations were further supported by existing literature and validated through additional interviews. Results: In total, 65 codes, 23 subthemes, and 8 themes were identified. The main themes included optimizing medical data management for enhanced clinical workflow, agreement on standardization and enhancement of the discharge report, enhancing clinical decision support through updated guidelines and automated tools, optimizing interoperability as a solution for better management of patients with heart failure, and encouraging communication based on digital tools and personal connection. In total, 15 improvements were proposed, such as standardizing technology across Andalusia Health Service facilities and offering targeted training programs. These measures aim to enhance interoperability, streamline communication between different health care settings, and reduce the administrative burden for health care professionals. Conclusions: Diraya currently does not adequately support the transition standardized care model, placing a significant administrative burden on health care professionals, often with ethically concerning implications. To ensure effective implementation of the standardized care model, major updates are necessary for Diraya’s clinical information management, system functionality, and organizational structure within the Andalusia Health Service. %M 40233354 %R 10.2196/63665 %U https://medinform.jmir.org/2025/1/e63665 %U https://doi.org/10.2196/63665 %U http://www.ncbi.nlm.nih.gov/pubmed/40233354 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e68344 %T Developing the Digital Health Communication Maturity Model: Systematic Review %A Kim,Grace Jeonghyun %A Namkoong,Kang %+ , Department of Communication, University of Maryland, College Park, Skinner Building, 4300 Chapel Ln, College Park, MD, 20782, United States, 1 2028920233, jkim0501@umd.edu %K digital health %K maturity model %K integrated model %K digital health communication %K health communication %K systematic review %K model development %K health care innovation %K digital transformation %K organizational readiness %K evaluation metrics %K health care technology %K digital strategy %D 2025 %7 14.4.2025 %9 Review %J J Med Internet Res %G English %X Background: Digital health has become integral to public health care, advancing how services are accessed, delivered, and managed. Health organizations increasingly assess their digital health maturity to leverage these innovations fully. However, existing digital health maturity models (DHMMs) primarily focus on technology and infrastructure, often neglecting critical communication components. Objective: This systematic review addresses gaps in DHMMs by identifying deficiencies in user communication elements and proposing the digital health communication maturity model (DHCMM). The DHCMM integrates critical health communication dimensions such as satisfaction, engagement, personalization, and customization to provide a comprehensive evaluation framework. Methods: We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines to conduct a systematic review of studies selected from 3 databases: EBSCO, PubMed, and ProQuest. Studies were screened and included based on their focus on digital health maturity and communication elements, with the final selection limited to English-language research addressing DHMMs. Results: Of the 1138 initially identified studies, 31 (2.72%) met the inclusion criteria. Current DHMMs heavily emphasize infrastructure while overlooking user engagement and communication; for instance, only 35% (11/31) of the reviewed models incorporated user satisfaction, and less than one-fifth (6/31, 19%) addressed personalization or customization. The DHCMM addresses these gaps with 7 maturity levels, ranging from initial to engaged, and emphasizes user-centered metrics and governance. Quantitative analysis showed substantial variations in communication metrics, with satisfaction metrics incorporated at an average rate of 22% (7/31) across the reviewed models. Conclusions: The DHCMM shifts the focus of digital health maturity assessments by emphasizing communication and user engagement. This model provides health care organizations with a structured framework to enhance digital health initiatives, leading to better patient outcomes and system-wide efficiencies. The model delivers actionable insights for organizations aiming to achieve advanced digital maturity by addressing underrepresented dimensions. Future research should implement and refine the DHCMM across diverse health care contexts to enhance its effectiveness. The adoption of this model could result in more equitable, user-centered health care systems that integrate technological advancements with human-centered care. %M 40228239 %R 10.2196/68344 %U https://www.jmir.org/2025/1/e68344 %U https://doi.org/10.2196/68344 %U http://www.ncbi.nlm.nih.gov/pubmed/40228239 %0 Journal Article %@ 2817-1705 %I JMIR Publications %V 4 %N %P e67144 %T Comparison of Deep Learning Approaches Using Chest Radiographs for Predicting Clinical Deterioration: Retrospective Observational Study %A Rahman,Mahmudur %A Gao,Jifan %A Carey,Kyle A %A Edelson,Dana P %A Afshar,Askar %A Garrett,John W %A Chen,Guanhua %A Afshar,Majid %A Churpek,Matthew M %K chest X-ray %K critical care %K deep learning %K chest radiographs %K radiographs %K clinical deterioration %K prediction %K predictive %K deterioration %K retrospective %K data %K dataset %K artificial intelligence %K AI %K chest %K patient %K hospitalized %D 2025 %7 10.4.2025 %9 %J JMIR AI %G English %X Background: The early detection of clinical deterioration and timely intervention for hospitalized patients can improve patient outcomes. The currently existing early warning systems rely on variables from structured data, such as vital signs and laboratory values, and do not incorporate other potentially predictive data modalities. Because respiratory failure is a common cause of deterioration, chest radiographs are often acquired in patients with clinical deterioration, which may be informative for predicting their risk of intensive care unit (ICU) transfer. Objective: This study aimed to compare and validate different computer vision models and data augmentation approaches with chest radiographs for predicting clinical deterioration. Methods: This retrospective observational study included adult patients hospitalized at the University of Wisconsin Health System between 2009 and 2020 with an elevated electronic cardiac arrest risk triage (eCART) score, a validated clinical deterioration early warning score, on the medical-surgical wards. Patients with a chest radiograph obtained within 48 hours prior to the elevated score were included in this study. Five computer vision model architectures (VGG16, DenseNet121, Vision Transformer, ResNet50, and Inception V3) and four data augmentation methods (histogram normalization, random flip, random Gaussian noise, and random rotate) were compared using the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC) for predicting clinical deterioration (ie, ICU transfer or ward death in the following 24 hours). Results: The study included 21,817 patient admissions, of which 1655 (7.6%) experienced clinical deterioration. The DenseNet121 model pretrained on chest radiograph datasets with histogram normalization and random Gaussian noise augmentation had the highest discrimination (AUROC 0.734 and AUPRC 0.414), while the vision transformer having 24 transformer blocks with random rotate augmentation had the lowest discrimination (AUROC 0.598). Conclusions: The study shows the potential of chest radiographs in deep learning models for predicting clinical deterioration. The DenseNet121 architecture pretrained with chest radiographs performed better than other architectures in most experiments, and the addition of histogram normalization with random Gaussian noise data augmentation may enhance the performance of DenseNet121 and pretrained VGG16 architectures. %R 10.2196/67144 %U https://ai.jmir.org/2025/1/e67144 %U https://doi.org/10.2196/67144 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e66366 %T Developing a Machine Learning Model for Predicting 30-Day Major Adverse Cardiac and Cerebrovascular Events in Patients Undergoing Noncardiac Surgery: Retrospective Study %A Kwun,Ju-Seung %A Ahn,Houng-Beom %A Kang,Si-Hyuck %A Yoo,Sooyoung %A Kim,Seok %A Song,Wongeun %A Hyun,Junho %A Oh,Ji Seon %A Baek,Gakyoung %A Suh,Jung-Won %+ Cardiovascular Center, Department of Internal Medicine, Seoul National University Bundang Hospital, 82 Gumi-ro, 173 Beon-gil, Bundang-gu, Gyeonggi-do, Seongnam-si, 13620, Republic of Korea, 82 01076615931, suhjw1@gmail.com %K perioperative risk evaluation %K noncardiac surgery %K prediction models %K machine learning %K common data model %K ML %K predictive modeling %K cerebrovascular %K electronic health records %K EHR %K clinical practice %K risk %K noncardiac surgeries %K perioperative %D 2025 %7 9.4.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Considering that most patients with low or no significant risk factors can safely undergo noncardiac surgery without additional cardiac evaluation, and given the excessive evaluations often performed in patients undergoing intermediate or higher risk noncardiac surgeries, practical preoperative risk assessment tools are essential to reduce unnecessary delays for urgent outpatient services and manage medical costs more efficiently. Objective: This study aimed to use the Observational Medical Outcomes Partnership Common Data Model to develop a predictive model by applying machine learning algorithms that can effectively predict major adverse cardiac and cerebrovascular events (MACCE) in patients undergoing noncardiac surgery. Methods: This retrospective observational network study collected data by converting electronic health records into a standardized Observational Medical Outcomes Partnership Common Data Model format. The study was conducted in 2 tertiary hospitals. Data included demographic information, diagnoses, laboratory results, medications, surgical types, and clinical outcomes. A total of 46,225 patients were recruited from Seoul National University Bundang Hospital and 396,424 from Asan Medical Center. We selected patients aged 65 years and older undergoing noncardiac surgeries, excluding cardiac or emergency surgeries, and those with less than 30 days of observation. Using these observational health care data, we developed machine learning–based prediction models using the observational health data sciences and informatics open-source patient-level prediction package in R (version 4.1.0; R Foundation for Statistical Computing). A total of 5 machine learning algorithms, including random forest, were developed and validated internally and externally, with performance assessed through the area under the receiver operating characteristic curve (AUROC), the area under the precision-recall curve, and calibration plots. Results: All machine learning prediction models surpassed the Revised Cardiac Risk Index in MACCE prediction performance (AUROC=0.704). Random forest showed the best results, achieving AUROC values of 0.897 (95% CI 0.883-0.911) internally and 0.817 (95% CI 0.815-0.819) externally, with an area under the precision-recall curve of 0.095. Among 46,225 patients of the Seoul National University Bundang Hospital, MACCE occurred in 4.9% (2256/46,225), including myocardial infarction (907/46,225, 2%) and stroke (799/46,225, 1.7%), while in-hospital mortality was 0.9% (419/46,225). For Asan Medical Center, 6.3% (24,861/396,424) of patients experienced MACCE, with 1.5% (6017/396,424) stroke and 3% (11,875/396,424) in-hospital mortality. Furthermore, the significance of predictors linked to previous diagnoses and laboratory measurements underscored their critical role in effectively predicting perioperative risk. Conclusions: Our prediction models outperformed the widely used Revised Cardiac Risk Index in predicting MACCE within 30 days after noncardiac surgery, demonstrating superior calibration and generalizability across institutions. Its use can optimize preoperative evaluations, minimize unnecessary testing, and streamline perioperative care, significantly improving patient outcomes and resource use. We anticipate that applying this model to actual electronic health records will benefit clinical practice. %M 40203300 %R 10.2196/66366 %U https://www.jmir.org/2025/1/e66366 %U https://doi.org/10.2196/66366 %U http://www.ncbi.nlm.nih.gov/pubmed/40203300 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e66959 %T Factors Influencing Information Distortion in Electronic Nursing Records: Qualitative Study %A Wang,Jianan %A Xu,Yihong %A Yang,Zhichao %A Zhang,Jie %A Zhang,Xiaoxiao %A Li,Wen %A Sun,Yushu %A Pan,Hongying %+ Department of Nursing, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, 3 Qingchun East Road, Hangzhou, 310016, China, 86 13857188922, 3191016@zju.edu.cn %K information distortion %K electronic health record %K qualitative research %K ethics %K nursing %D 2025 %7 9.4.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Information distortion in nursing records poses significant risks to patient safety and impedes the enhancement of care quality. The introduction of information technologies, such as decision support systems and predictive models, expands the possibilities for using health data but also complicates the landscape of information distortion. Only by identifying influencing factors about information distortion can care quality and patient safety be ensured. Objective: This study aims to explore the factors influencing information distortion in electronic nursing records (ENRs) within the context of China’s health care system and provide appropriate recommendations to address these distortions. Methods: This qualitative study used semistructured interviews conducted with 14 nurses from a Class-A tertiary hospital. Participants were primarily asked about their experiences with and observations of information distortion in clinical practice, as well as potential influencing factors and corresponding countermeasures. Data were analyzed using inductive content analysis, which involved initial preparation, line-by-line coding, the creation of categories, and abstraction. Results: The analysis identified 4 categories and 10 subcategories: (1) nurse-related factors—skills, awareness, and work habits; (2) patient-related factors—willingness and ability; (3) operational factors—work characteristics and system deficiencies; and (4) organizational factors—management system, organizational climate, and team collaboration. Conclusions: Although some factors influencing information distortion in ENRs are similar to those observed in paper-based records, others are unique to the digital age. As health care continues to embrace digitalization, it is crucial to develop and implement strategies to mitigate information distortion. Regular training and education programs, robust systems and mechanisms, and optimized human resources and organizational practices are strongly recommended. %M 40202777 %R 10.2196/66959 %U https://www.jmir.org/2025/1/e66959 %U https://doi.org/10.2196/66959 %U http://www.ncbi.nlm.nih.gov/pubmed/40202777 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e57782 %T Impact of Electronic Transition and Prefilled Templates on Drug Prescription Compliance: Retrospective Study %A Lambert,Aurélien %A Hombourger,Benoit %A Salleron,Julia %A Chergui,Fadila %A Vallance,Catherine %A Nicolas,Nadège %A Moussouni,Marie %A Cherif,Lounisse %A Chenot,Emile %A Gavoille,Céline %A Massard,Vincent %+ , Institut de Cancérologie de Lorraine, 6 avenue de bourgogne, Vandoeuvre-lès-Nancy, 54500, France, 33 383598400, a.lambert@nancy.unicancer.fr %K drug prescription %K electronic prescription %K handwriting %K medical oncology %K ambulatory care %D 2025 %7 9.4.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: The transition from traditional handwritten prescriptions to electronic prescribing systems represents a significant advancement, with the potential to enhance treatment efficacy, patient safety, and professional communication. Objective: This study aimed to examine the impact of this transition within a medical oncology service, assessing the compliance of electronic prescriptions with established good practice standards and exploring the associated risks. Methods: In this retrospective analysis, we compared handwritten prescriptions from the pre-electronic era (January to May 2018) with electronic prescriptions (January to May 2021) following the implementation of the electronic prescribing system PandaLab Pro (PandaLab SAS). The inclusion criteria focused on outpatient oncology treatments, with a clear set of exclusion parameters to ensure a focused study scope. We defined good compliance as the written mention of the evaluated terms. The compliance rates were then compared using a chi-square test. Results: Our findings, based on a sample size of 260 prescriptions (randomized among 30,526 archived prescriptions), indicate a substantial improvement in electronic prescriptions’ compliance with prescribers and patient details, treatment accuracy, and overall adherence to regulatory standards. Notably, electronic formats achieved a remarkable 80.8% accuracy rate in compliance with safety criteria compared with 8.5% for handwritten prescriptions (P<.001). The use of prefilled prescriptions significantly increased compliance from a safety perspective (56% vs 96.2%; P<.001) compared with electronic prescriptions from scratch. Conclusions: The analysis further underscores the advantages of prefilled electronic prescription templates, which significantly improved compliance rates compared with manually filled electronic and handwritten prescriptions. Furthermore, the study revealed a marked shift in prescribing behaviors, with electronic prescriptions tending to be more concise yet more numerous, suggesting an impact on medication management and patient adherence, which warrants further investigation. The study supports the transition to electronic prescribing systems in oncology, highlighting enhanced traceability, compliance with health authority standards, and patient safety. The implementation of prefilled templates supported by pharmacists has emerged as a pivotal factor in this improved process. While acknowledging certain limitations, such as the nonquantitative assessment of time savings and acceptability, this research advocates for the widespread adoption of electronic prescriptions and serves as a benchmark for future e-prescription initiatives in France. %M 40202779 %R 10.2196/57782 %U https://www.jmir.org/2025/1/e57782 %U https://doi.org/10.2196/57782 %U http://www.ncbi.nlm.nih.gov/pubmed/40202779 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e59231 %T Media Framing and Portrayals of Ransomware Impacts on Informatics, Employees, and Patients: Systematic Media Literature Review %A Avery,Atiya %A Baker,Elizabeth White %A Wright,Brittany %A Avery,Ishmael %A Gomez,Dream %+ , Harbert College of Business, Auburn University, 405 W Magnolia Ave, Auburn, AL, 36849, United States, 1 334 844 2908, atiya.avery@auburn.edu %K cybersecurity %K media frames %K medical informatics %K practitioners %K health care provider %K systematic review %K employees %K patient %K mortality %K morbidity %K news media %K ransomware %K health information system %K database %K health care service %D 2025 %7 8.4.2025 %9 Review %J J Med Internet Res %G English %X Background: Ransomware attacks on health care provider information systems have the potential to impact patient mortality and morbidity, and event details are relayed publicly through news stories. Despite this, little research exists on how these events are depicted in the media and the subsequent impacts of these events. Objective: This study used collaborative qualitative analysis to understand how news media frames and portrays the impacts of ransomware attacks on health informatic systems, employees, and patients. Methods: We developed and implemented a systematic search protocol across academic news databases, which included (1) the Associated Press Newswires, (2) Newspaper Source, and (3) Access World News (Newsbank), using the search string “(hospital OR healthcare OR clinic OR medical) AND (ransomware OR denial of service OR cybersecurity).” In total, 4 inclusion and 4 exclusion criteria were applied as part of the search protocol. For articles included in the study, we performed an inductive and deductive analysis of the news articles, which included their article characteristics, impact portrayals, media framings, and discussions of the core functions outlined in the National Institute of Standards and Technologies (NIST) Cybersecurity Framework 2.0. Results: The search returned 2195 articles, among which 48 news articles published from 2009 to 2023 were included in the study. First, an analysis of the geographic prevalence showed that the United States (34/48, 71%), followed to a lesser extent by India (4/48, 8%) and Canada (3/48, 6%), featured more prominently in our sample. Second, there were no apparent year-to-year patterns in the occurrence of reported events of ransomware attacks on health care provider information systems. Third, ransomware attacks on health care provider information systems appeared to cascade from a single point of failure. Fourth, media frames regarding “human interest” and “responsibility” were equally representative in the sample. The “response” function of the NIST Cybersecurity Framework 2.0 was noted in 36 of the 48 (75%) articles. Finally, we noted that 17 (14%) of the articles assessed for eligibility were excluded from this study as they promoted a product or service or spoke hypothetically about ransomware events among health care providers. Conclusions: Organizational response represented a substantial aspect of the news articles in our corpus. To address the perception of health care providers’ management of ransomware attacks, they should take measures to influence perceptions of (1) health care service continuity, despite a lack of availability of health informatics; (2) responsibility for the patient experience; and (3) acknowledgment of the strain on health care practitioners and patients through a public declaration of support and gratitude. Furthermore, the media portrayals revealed a prevalence of single points of failure in the health informatics system, thus providing guidance for the implementation of safety protocols that could significantly reduce cascading impacts. %M 40198915 %R 10.2196/59231 %U https://www.jmir.org/2025/1/e59231 %U https://doi.org/10.2196/59231 %U http://www.ncbi.nlm.nih.gov/pubmed/40198915 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 13 %N %P e62978 %T Large-Scale Evaluation and Liver Disease Risk Prediction in Finland’s National Electronic Health Record System: Feasibility Study Using Real-World Data %A Männikkö,Viljami %A Tommola,Janne %A Tikkanen,Emmi %A Hätinen,Olli-Pekka %A Åberg,Fredrik %+ Atostek Oy, Hermiankatu 3 A, Tampere, 33720, Finland, 358 45 7834 70, viljami.mannikko@tuni.fi %K Kanta archive %K national patient data repository %K real world data %K risk prediction %K chronic liver disease %K mortality %K risk detection %K alcoholic liver %K prediction %K obesity %K overweight %K electronic health record %K wearables %K smartwatch %D 2025 %7 2.4.2025 %9 Original Paper %J JMIR Med Inform %G English %X Background: Globally, the incidence and mortality of chronic liver disease are escalating. Early detection of liver disease remains a challenge, often occurring at symptomatic stages when preventative measures are less effective. The Chronic Liver Disease score (CLivD) is a predictive risk model developed using Finnish health care data, aiming to forecast an individual’s risk of developing chronic liver disease in subsequent years. The Kanta Service is a national electronic health record system in Finland that stores comprehensive health care data including patient medical histories, prescriptions, and laboratory results, to facilitate health care delivery and research. Objective: This study aimed to evaluate the feasibility of implementing an automatic CLivD score with the current Kanta platform and identify and suggest improvements for Kanta that would enable accurate automatic risk detection. Methods: In this study, a real-world data repository (Kanta) was used as a data source for “The ClivD score” risk calculation model. Our dataset consisted of 96,200 individuals’ whole medical history from Kanta. For real-world data use, we designed processes to handle missing input in the calculation process. Results: We found that Kanta currently lacks many CLivD risk model input parameters in the structured format required to calculate precise risk scores. However, the risk scores can be improved by using the unstructured text in patient reports and by approximating variables by using other health data–like diagnosis information. Using structured data, we were able to identify only 33 out of 51,275 individuals in the “low risk” category and 308 out of 51,275 individuals (<1%) in the “moderate risk” category. By adding diagnosis information approximation and free text use, we were able to identify 18,895 out of 51,275 (37%) individuals in the “low risk” category and 2125 out of 51,275 (4%) individuals in the “moderate risk” category. In both cases, we were not able to identify any individuals in the “high-risk” category because of the missing waist-hip ratio measurement. We evaluated 3 scenarios to improve the coverage of waist-hip ratio data in Kanta and these yielded the most substantial improvement in prediction accuracy. Conclusions: We conclude that the current structured Kanta data is not enough for precise risk calculation for CLivD or other diseases where obesity, smoking, and alcohol use are important risk factors. Our simulations show up to 14% improvement in risk detection when adding support for missing input variables. Kanta shows the potential for implementing nationwide automated risk detection models that could result in improved disease prevention and public health. %M 40172947 %R 10.2196/62978 %U https://medinform.jmir.org/2025/1/e62978 %U https://doi.org/10.2196/62978 %U http://www.ncbi.nlm.nih.gov/pubmed/40172947 %0 Journal Article %@ 2369-1999 %I JMIR Publications %V 11 %N %P e69672 %T AI-Based Identification Method for Cervical Transformation Zone Within Digital Colposcopy: Development and Multicenter Validation Study %A Wu,Tong %A Wang,Yuting %A Cui,Xiaoli %A Xue,Peng %A Qiao,Youlin %+ School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, 31 Yard, Beijige Santiao, Beijing, 100730, China, 86 10 8778 8489, qiaoy@cicams.ac.cn %K artificial intelligence %K AI %K cervical cancer screening %K transformation zone %K diagnosis and early treatment %K lightweight neural network %D 2025 %7 31.3.2025 %9 Original Paper %J JMIR Cancer %G English %X Background: In low- and middle-income countries, cervical cancer remains a leading cause of death and morbidity for women. Early detection and treatment of precancerous lesions are critical in cervical cancer prevention, and colposcopy is a primary diagnostic tool for identifying cervical lesions and guiding biopsies. The transformation zone (TZ) is where a stratified squamous epithelium develops from the metaplasia of simple columnar epithelium and is the most common site of precancerous lesions. However, inexperienced colposcopists may find it challenging to accurately identify the type and location of the TZ during a colposcopy examination. Objective: This study aims to present an artificial intelligence (AI) method for identifying the TZ to enhance colposcopy examination and evaluate its potential clinical application. Methods: The study retrospectively collected data from 3616 women who underwent colposcopy at 6 tertiary hospitals in China between 2019 and 2021. A dataset from 4 hospitals was collected for model conduction. An independent dataset was collected from the other 2 geographic hospitals to validate model performance. There is no overlap between the training and validation datasets. Anonymized digital records, including each colposcopy image, baseline clinical characteristics, colposcopic findings, and pathological outcomes, were collected. The classification model was proposed as a lightweight neural network with multiscale feature enhancement capabilities and designed to classify the 3 types of TZ. The pretrained FastSAM model was first implemented to identify the location of the new squamocolumnar junction for segmenting the TZ. Overall accuracy, average precision, and recall were evaluated for the classification and segmentation models. The classification performance on the external validation was assessed by sensitivity and specificity. Results: The optimal TZ classification model performed with 83.97% classification accuracy on the test set, which achieved average precision of 91.84%, 89.06%, and 95.62% for types 1, 2, and 3, respectively. The recall and mean average precision of the TZ segmentation model were 0.78 and 0.75, respectively. The proposed model demonstrated outstanding performance in predicting 3 types of the TZ, achieving the sensitivity with 95% CIs for TZ1, TZ2, and TZ3 of 0.78 (0.74-0.81), 0.81 (0.78-0.82), and 0.8 (0.74-0.87), respectively, with specificity with 95% CIs of 0.94 (0.92-0.96), 0.83 (0.81-0.86), and 0.91 (0.89-0.92), based on a comprehensive external dataset of 1335 cases from 2 of the 6 hospitals. Conclusions: Our proposed AI-based identification system classified the type of cervical TZs and delineated their location on multicenter, colposcopic, high-resolution images. The findings of this study have shown its potential to predict TZ types and specific regions accurately. It was developed as a valuable assistant to encourage precise colposcopic examination in clinical practice. %M 40163848 %R 10.2196/69672 %U https://cancer.jmir.org/2025/1/e69672 %U https://doi.org/10.2196/69672 %U http://www.ncbi.nlm.nih.gov/pubmed/40163848 %0 Journal Article %@ 2561-7605 %I JMIR Publications %V 8 %N %P e65178 %T Unsupervised Deep Learning of Electronic Health Records to Characterize Heterogeneity Across Alzheimer Disease and Related Dementias: Cross-Sectional Study %A West,Matthew %A Cheng,You %A He,Yingnan %A Leng,Yu %A Magdamo,Colin %A Hyman,Bradley T %A Dickson,John R %A Serrano-Pozo,Alberto %A Blacker,Deborah %A Das,Sudeshna %+ Massachusetts General Hospital, 65 Landsdowne Street, Cambridge, MA, 02139, United States, 1 617 768 8254, sdas5@mgh.harvard.edu %K Alzheimer disease and related dementias %K electronic health records %K large language models %K clustering %K unsupervised learning %D 2025 %7 31.3.2025 %9 Original Paper %J JMIR Aging %G English %X Background: Alzheimer disease and related dementias (ADRD) exhibit prominent heterogeneity. Identifying clinically meaningful ADRD subtypes is essential for tailoring treatments to specific patient phenotypes. Objective: We aimed to use unsupervised learning techniques on electronic health records (EHRs) from memory clinic patients to identify ADRD subtypes. Methods: We used pretrained embeddings of non-ADRD diagnosis codes (International Classification of Diseases, Ninth Revision) and large language model (LLM)–derived embeddings of clinical notes from patient EHRs. Hierarchical clustering of these embeddings was used to identify ADRD subtypes. Clusters were characterized regarding their demographic and clinical features. Results: We analyzed a cohort of 3454 patients with ADRD from a memory clinic at Massachusetts General Hospital, each with a specialist diagnosis. Clustering pretrained embeddings of the non-ADRD diagnosis codes in patient EHRs revealed the following 3 patient subtypes: one with skin conditions, another with psychiatric disorders and an earlier age of onset, and a third with diabetes complications. Similarly, using LLM-derived embeddings of clinical notes, we identified 3 subtypes of patients as follows: one with psychiatric manifestations and higher prevalence of female participants (prevalence ratio: 1.59), another with cardiovascular and motor problems and higher prevalence of male participants (prevalence ratio: 1.75), and a third one with geriatric health disorders. Notably, we observed significant overlap between clusters from both data modalities (χ24=89.4; P<.001). Conclusions: By integrating International Classification of Diseases, Ninth Revision codes and LLM-derived embeddings, our analysis delineated 2 distinct ADRD subtypes with sex-specific comorbid and clinical presentations, offering insights for potential precision medicine approaches. %M 40163031 %R 10.2196/65178 %U https://aging.jmir.org/2025/1/e65178 %U https://doi.org/10.2196/65178 %U http://www.ncbi.nlm.nih.gov/pubmed/40163031 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e59649 %T The Role of Machine Learning in Cognitive Impairment in Parkinson Disease: Systematic Review and Meta-Analysis %A Wu,Yanyun %A Cheng,Yangfan %A Xiao,Yi %A Shang,Huifang %A Ou,Ruwei %+ Department of Neurology, West China Hospital of Sichuan University, No.37, Guoxue Lane, Chengdu, 610041, China, 86 18980607525, ouruwei@aliyun.com %K Parkinson disease %K cognitive impairment %K machine learning %K systematic review %K meta-analysis %D 2025 %7 14.3.2025 %9 Review %J J Med Internet Res %G English %X Background: Parkinson disease (PD) is a common neurodegenerative disease characterized by both motor and nonmotor symptoms. Cognitive impairment often occurs early in the disease and can persist throughout its progression, severely impacting patients’ quality of life. The utilization of machine learning (ML) has recently shown promise in identifying cognitive impairment in patients with PD. Objective: This study aims to summarize different ML models applied to cognitive impairment in patients with PD and to identify determinants for improving diagnosis and predictive power for early detection of cognitive impairment. Methods: PubMed, Cochrane, Embase, and Web of Science were searched for relevant articles on March 2, 2024. The risk of bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2). Bivariate meta-analysis was used to estimate pooled sensitivity and specificity results, presented as odds ratio (OR) and 95% CI. A summary receiver operator characteristic (SROC) curve was used. Results: A total of 38 articles met the criteria, involving 8564 patients with PD and 1134 healthy controls. Overall, 120 models reported sensitivity and specificity, with mean values of 71.07% (SD 13.72%) and 77.01% (SD 14.31%), respectively. Predictors commonly used in ML models included clinical features, neuroimaging features, and other variables. No significant heterogeneity was observed in the bivariate meta-analysis, which included 12 studies. Using sensitivity as the metric, the combined sensitivity and specificity were 0.76 (95% CI 0.67-0.83) and 0.83 (95% CI 0.76-0.88), respectively. When specificity was used, the combined values were 0.77 (95% CI 0.65-0.86) and 0.76 (95% CI 0.63-0.85), respectively. The area under the curves of the SROC were 0.87 (95% CI 0.83-0.89) and 0.83 (95% CI 0.80-0.86) respectively. Conclusions: Our findings provide a comprehensive summary of various ML models and demonstrate the effectiveness of ML as a tool for diagnosing and predicting cognitive impairment in patients with PD. Trial Registration: PROSPERO CRD42023480196; https://www.crd.york.ac.uk/PROSPERO/view/CRD42023480196 %M 40153789 %R 10.2196/59649 %U https://www.jmir.org/2025/1/e59649/ %U https://doi.org/10.2196/59649 %U http://www.ncbi.nlm.nih.gov/pubmed/40153789 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e65537 %T Large Language Model–Driven Knowledge Graph Construction in Sepsis Care Using Multicenter Clinical Databases: Development and Usability Study %A Yang,Hao %A Li,Jiaxi %A Zhang,Chi %A Sierra,Alejandro Pazos %A Shen,Bairong %+ Department of Critical Care Medicine, Joint Laboratory of Artifcial Intelligence for Critical Care Medicine, Frontiers Science Center for Disease-related Molecular Network, Institutes for Systems Genetics, Sichuan University, West China Hospital, No. 37, Guo Xue Xiang, Chengdu, 610041, China, 86 85164199, bairong.shen@scu.edu.cn %K sepsis %K knowledge graph %K large language models %K prompt engineering %K real-world %K GPT-4.0 %D 2025 %7 27.3.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Sepsis is a complex, life-threatening condition characterized by significant heterogeneity and vast amounts of unstructured data, posing substantial challenges for traditional knowledge graph construction methods. The integration of large language models (LLMs) with real-world data offers a promising avenue to address these challenges and enhance the understanding and management of sepsis. Objective: This study aims to develop a comprehensive sepsis knowledge graph by leveraging the capabilities of LLMs, specifically GPT-4.0, in conjunction with multicenter clinical databases. The goal is to improve the understanding of sepsis and provide actionable insights for clinical decision-making. We also established a multicenter sepsis database (MSD) to support this effort. Methods: We collected clinical guidelines, public databases, and real-world data from 3 major hospitals in Western China, encompassing 10,544 patients diagnosed with sepsis. Using GPT-4.0, we used advanced prompt engineering techniques for entity recognition and relationship extraction, which facilitated the construction of a nuanced sepsis knowledge graph. Results: We established a sepsis database with 10,544 patient records, including 8497 from West China Hospital, 690 from Shangjin Hospital, and 357 from Tianfu Hospital. The sepsis knowledge graph comprises of 1894 nodes and 2021 distinct relationships, encompassing nine entity concepts (diseases, symptoms, biomarkers, imaging examinations, etc) and 8 semantic relationships (complications, recommended medications, laboratory tests, etc). GPT-4.0 demonstrated superior performance in entity recognition and relationship extraction, achieving an F1-score of 76.76 on a sepsis-specific dataset, outperforming other models such as Qwen2 (43.77) and Llama3 (48.39). On the CMeEE dataset, GPT-4.0 achieved an F1-score of 65.42 using few-shot learning, surpassing traditional models such as BERT-CRF (62.11) and Med-BERT (60.66). Building upon this, we compiled a comprehensive sepsis knowledge graph, comprising of 1894 nodes and 2021 distinct relationships. Conclusions: This study represents a pioneering effort in using LLMs, particularly GPT-4.0, to construct a comprehensive sepsis knowledge graph. The innovative application of prompt engineering, combined with the integration of multicenter real-world data, has significantly enhanced the efficiency and accuracy of knowledge graph construction. The resulting knowledge graph provides a robust framework for understanding sepsis, supporting clinical decision-making, and facilitating further research. The success of this approach underscores the potential of LLMs in medical research and sets a new benchmark for future studies in sepsis and other complex medical conditions. %M 40146985 %R 10.2196/65537 %U https://www.jmir.org/2025/1/e65537 %U https://doi.org/10.2196/65537 %U http://www.ncbi.nlm.nih.gov/pubmed/40146985 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e59220 %T Implementation of Medication-Related Technology and Its Impact on Pharmacy Workflow: Real-World Evidence Usability Study %A Yu,Wei-Ning %A Cheng,Yih-Dih %A Hou,Yu-Chi %A Hsieh,Yow-Wen %+ , Department of Pharmacy, China Medical University Hospital, No 2, Yude Rd, North Dist, Taichung City, 404327, Taiwan, 886 422052121 ext 12272, tovis168@gmail.com %K medication error %K dispensing error %K medication-related technology %K pharmacy %K smart dispensing counter %D 2025 %7 27.3.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Medication errors constitute a major contributor to patient harm, driving up health care costs and representing a preventable cause of medical incidents. Over the past decade, many hospitals have integrated various medication-related technologies into their pharmacy operations. However, real-world evidence of the impact of these advanced systems on clinical prescription dispensing error rates remains limited. Objective: This study aims to prospectively detect and record the categories and rates of dispensing errors to illustrate how medication-related technologies, such as automated dispensing cabinet (ADC), barcode medication administration (BCMA), and smart dispensing counter (SDC), can be used to minimize dispensing errors. Methods: This study used a before-and-after design at a 2202-bed academic medical center in Taiwan to assess the impact of implementing medication-related technologies (ADC, BCMA, and SDC) on patient medication safety. Dispensing error rates were analyzed from January 1, 2017, to December 31, 2023, using data from the China Medical University Hospital Patient Safety Database. The study periods were defined as stage 0 (preintervention, January to November 2017), stage 1 (post-ADC intervention, December 2017 to June 2018), stage 2 (post-BCMA intervention, July 2018 to October 2020), and stage 3 (post-SDC intervention, November 2020 to December 2023). Medication errors were defined according to the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP). Chi-square or Fisher exact tests were used to analyze differences between intervention periods, with Bonferroni correction for multiple comparisons. Statistical significance was set at P<.05. Results: Following the introduction of medication-related technologies, the average dispensing error incidence rate significantly decreased by 39.68%, 44.44%, and 77.78%, from 0.0063% in stage 0 to 0.0038%, 0.0035%, and 0.0014% in stages 1, 2, and 3, respectively (P<.001). The frequency of “wrong drug” errors, the most common error type in stage 0, significantly decreased by 51.15%, 56.85%, and 81.26% in stages 1, 2, and 3, respectively. All error types, except for “wrong dosage form,” “wrong strength,” “wrong time,” and “others,” demonstrated statistically significant differences (P<.001). The majority of harm severities were categorized as “A” (no error; 97%-98.8%) and “B-D” (error, no harm; 1.2%-3%) according to the NCC MERP classification. The severity of “no error” (category A) significantly decreased at each stage (P<.001). Statistically significant differences in dispensing error rates were observed between all stages (P<.001), except between stages 2 and 1 (P>.99). Conclusions: This study provides significant evidence that the implementation of medication-related technologies, including ADC, BCMA, and SDC, effectively reduces dispensing errors in a hospital pharmacy setting. Specifically, we observed a substantial decrease in the average dispensing error rate across 3 stages of technology implementation. Importantly, this study appears to be the first to investigate the combined impact of these 3 specific technologies on dispensing error rates within a hospital pharmacy. %M 40019479 %R 10.2196/59220 %U https://www.jmir.org/2025/1/e59220 %U https://doi.org/10.2196/59220 %U http://www.ncbi.nlm.nih.gov/pubmed/40019479 %0 Journal Article %@ 2817-1705 %I JMIR Publications %V 4 %N %P e62985 %T Limitations of Binary Classification for Long-Horizon Diagnosis Prediction and Advantages of a Discrete-Time Time-to-Event Approach: Empirical Analysis %A Loh,De Rong %A Hill,Elliot D %A Liu,Nan %A Dawson,Geraldine %A Engelhard,Matthew M %K machine learning %K artificial intelligence %K deep learning %K predictive models %K practical models %K early detection %K electronic health records %K right-censoring %K survival analysis %K distributional shifts %D 2025 %7 27.3.2025 %9 %J JMIR AI %G English %X Background: A major challenge in using electronic health records (EHR) is the inconsistency of patient follow-up, resulting in right-censored outcomes. This becomes particularly problematic in long-horizon event predictions, such as autism and attention-deficit/hyperactivity disorder (ADHD) diagnoses, where a significant number of patients are lost to follow-up before the outcome can be observed. Consequently, fully supervised methods such as binary classification (BC), which are trained to predict observed diagnoses, are substantially affected by the probability of sufficient follow-up, leading to biased results. Objective: This empirical analysis aims to characterize BC’s inherent limitations for long-horizon diagnosis prediction from EHR; and quantify the benefits of a specific time-to-event (TTE) approach, the discrete-time neural network (DTNN). Methods: Records within the Duke University Health System EHR were analyzed, extracting features such as ICD-10 (International Classification of Diseases, Tenth Revision) diagnosis codes, medications, laboratories, and procedures. We compared a DTNN to 3 BC approaches and a deep Cox proportional hazards model across 4 clinical conditions to examine distributional patterns across various subgroups. Time-varying area under the receiving operating characteristic curve (AUCt) and time-varying average precision (APt) were our primary evaluation metrics. Results: TTE models consistently had comparable or higher AUCt and APt than BC for all conditions. At clinically relevant operating time points, the area under the receiving operating characteristic curve (AUC) values for DTNNYOB≤2020 (year-of-birth) and DCPHYOB≤2020 (deep Cox proportional hazard) were 0.70 (95% CI 0.66‐0.77) and 0.72 (95% CI 0.66‐0.78) at t=5 for autism, 0.72 (95% CI 0.65‐0.76) and 0.68 (95% CI 0.62‐0.74) at t=7 for ADHD, 0.72 (95% CI 0.70‐0.75) and 0.71 (95% CI 0.69‐0.74) at t=1 for recurrent otitis media, and 0.74 (95% CI 0.68‐0.82) and 0.71 (95% CI 0.63‐0.77) at t=1 for food allergy, compared to 0.6 (95% CI 0.55‐0.66), 0.47 (95% CI 0.40‐0.54), 0.73 (95% CI 0.70‐0.75), and 0.77 (95% CI 0.71‐0.82) for BCYOB≤2020, respectively. The probabilities predicted by BC models were positively correlated with censoring times, particularly for autism and ADHD prediction. Filtering strategies based on YOB or length of follow-up only partially corrected these biases. In subgroup analyses, only DTNN predicted diagnosis probabilities that accurately reflect actual clinical prevalence and temporal trends. Conclusions: BC models substantially underpredicted diagnosis likelihood and inappropriately assigned lower probability scores to individuals with earlier censoring. Common filtering strategies did not adequately address this limitation. TTE approaches, particularly DTNN, effectively mitigated bias from the censoring distribution, resulting in superior discrimination and calibration performance and more accurate prediction of clinical prevalence. Machine learning practitioners should recognize the limitations of BC for long-horizon diagnosis prediction and adopt TTE approaches. The DTNN in particular is well-suited to mitigate the effects of right-censoring and maximize prediction performance in this setting. %R 10.2196/62985 %U https://ai.jmir.org/2025/1/e62985 %U https://doi.org/10.2196/62985 %0 Journal Article %@ 2371-4379 %I JMIR Publications %V 10 %N %P e66831 %T Applications of AI in Predicting Drug Responses for Type 2 Diabetes %A Garg,Shilpa %A Kitchen,Robert %A Gupta,Ramneek %A Pearson,Ewan %K type 2 diabetes %K artificial intelligence %K machine learning %K drug response %K treatment response prediction %K ML %K AI %K deep learning %D 2025 %7 27.3.2025 %9 %J JMIR Diabetes %G English %X Type 2 diabetes mellitus has seen a continuous rise in prevalence in recent years, and a similar trend has been observed in the increased availability of glucose-lowering drugs. There is a need to understand the variation in treatment response to these drugs to be able to predict people who will respond well or poorly to a drug. Electronic health records, clinical trials, and observational studies provide a huge amount of data to explore predictors of drug response. The use of artificial intelligence (AI), which includes machine learning and deep learning techniques, has the capacity to improve the prediction of treatment response in patients. AI can assist in the analysis of vast datasets to identify patterns and may provide valuable information on selecting an effective drug. Predicting an individual’s response to a drug can aid in treatment selection, optimizing therapy, exploring new therapeutic options, and personalized medicine. This viewpoint highlights the growing evidence supporting the potential of AI-based methods to predict drug response with accuracy. Furthermore, the methods highlight a trend toward using ensemble methods as preferred models in drug response prediction studies. %R 10.2196/66831 %U https://diabetes.jmir.org/2025/1/e66831 %U https://doi.org/10.2196/66831 %0 Journal Article %@ 1929-073X %I JMIR Publications %V 14 %N %P e53849 %T Using an Electronic Immunization Registry (Aplikasi Sehat IndonesiaKu) in Indonesia: Cross-Sectional Study %A Aisyah,Dewi Nur %A Utami,Astri %A Rahman,Fauziah Mauly %A Adriani,Nathasya Humaira %A Fitransyah,Fiqi %A Endryantoro,M Thoriqul Aziz %A Hutapea,Prima Yosephine %A Tandy,Gertrudis %A Manikam,Logan %A Kozlakidis,Zisis %+ Department of Epidemiology and Public Health, Institute of Epidemiology and Health Care, University College London, 1-19 Torrington Place, London, WC1E 7HB, United Kingdom, 44 2076792000, logan.manikam.10@ucl.ac.uk %K immunization %K registry %K digital %K puskesmas %K public health center %K mobile app %D 2025 %7 27.3.2025 %9 Original Paper %J Interact J Med Res %G English %X Background: Electronic immunization registries (EIRs) are being increasingly used in low- and middle-income countries. In 2022, Indonesia’s Ministry of Health introduced its first EIR, named Aplikasi Sehat IndonesiaKu (ASIK), as part of a comprehensive nationwide immunization program. This marked a conversion from traditional paper-based immunization reports to digital routine records encompassing a network of 10,000 primary health centers (puskesmas). Objective: This paper provides an overview of the use of ASIK as the first EIR in Indonesia. It describes the coverage of the nationwide immunization program (Bulan Imunisasi Anak Nasional) using ASIK data and assesses the implementation challenges associated with the adoption of the EIR in the context of Indonesia. Methods: Data were collected from primary care health workers’ submitted reports using ASIK. The data were reported in real time, analyzed, and presented using a structured dashboard. Data on ASIK use were collected from the ASIK website. A quantitative assessment was conducted through a cross-sectional survey between September 2022 and October 2022. A set of questionnaires was used to collect feedback from ASIK users. Results: A total of 93.5% (9708/10,382) of public health centers, 93.5% (6478/6928) of subdistricts, and 97.5% (501/514) of districts and cities in 34 provinces reported immunization data using ASIK. With >21 million data points recorded, the national coverage for immunization campaigns for measles-rubella; oral polio vaccine; inactivated polio vaccine; and diphtheria, pertussis, tetanus, hepatitis B, and Haemophilus influenzae type B vaccine were 50.1% (18,301,057/36,497,694), 36.2% (938,623/2,595,240), 30.7% (1,276,668/4,158,289), and 40.2% (1,371,104/3,407,900), respectively. The quantitative survey showed that, generally, users had a good understanding of ASIK as the EIR (650/809, 80.3%), 61.7% (489/793) of the users expressed that the user interface and user experience were overall good but could still be improved, 54% (422/781) of users expressed that the ASIK variable fit their needs yet could be improved further, and 59.1% (463/784) of users observed sporadic system interference. Challenges faced during the implementation of ASIK included a heavy workload burden for health workers, inadequate access to the internet at some places, system integration and readiness, and dual reporting using the paper-based format. Conclusions: The EIR is beneficial and helpful for monitoring vaccination coverage. Implementation and adoption of ASIK as Indonesia’s first EIR still faces challenges related to human resources and digital infrastructure as the country transitions from paper-based reports to electronic or digital immunization reports. Continuous improvement, collaboration, and monitoring efforts are crucial to encourage the use of the EIR in Indonesia. %M 40146988 %R 10.2196/53849 %U https://www.i-jmr.org/2025/1/e53849 %U https://doi.org/10.2196/53849 %U http://www.ncbi.nlm.nih.gov/pubmed/40146988 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e65241 %T The Role of the Installed Base in Information Exchange Among General Practitioners in Germany: Mixed Methods Study %A Holetzek,Tim %A Häusler,Andreas %A Gödde,Kathrin %A Rapp,Michael %A Spallek,Jacob %A Holmberg,Christine %+ Institute of Social Medicine and Epidemiology, Brandenburg Medical School Theodor Fontane, Hochstrasse 15, Brandenburg/Havel, 14770, Germany, 49 03381411282, tim.holetzek@mhb-fontane.de %K digitalization %K general practitioners %K Germany %K information and communication technologies %K information exchange %K primary health care %K digital transformation %K mixed methods study %K digital health %K health application %K qualitative interview %D 2025 %7 24.3.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Digitalization is steadily advancing on a global scale, exerting a profound influence on health care systems. To facilitate acceptance of the digital transformation, guiding principles emphasize the need for digital health structures to be person-centered and promote high-quality care. This paper examines the implementation challenges within the German health care system, with a particular focus on how change initiatives engage with existing infrastructures and organizational modes of health care delivery. This approach provides a framework for analyzing how established infrastructure determines new developments while also highlighting the procedural dynamics of change and the integration of innovations within existing information infrastructures. These established infrastructures are referred to as the installed base. Objective: The aim of the study is to examine the installed base encountered by the digital transformation within the German health care system by investigating information exchange practices among general practitioners (GPs) and their communication with other health care actors. Methods: A mixed methods study including a quantitative survey and semistructured qualitative interviews was conducted. The study sample consisted of all publicly accessible GP practices (N=1348) situated in the state of Brandenburg, Germany. The survey captured demographic data, communication practices, and perceived barriers to digitalization. The interviews explored experiences with digital applications. Quantitative data were analyzed using R (R Foundation for Statistical Computing), and qualitative data were managed and analyzed in MAXQDA (VERBI Software GmbH) through content analysis. Results: A total of 250 questionnaires (response rate 18.5%) and 10 interviews with GPs were included in the analysis. GPs primarily use the telephone (n=138, 55.2%, SD 24.64), fax (n=109, 43.9%, SD 25.40), or post (n=50, 20.2%, SD 9.46) to exchange information. Newer digital communication channels such as messenger applications (n=2, 0.8%, SD 0.72) and Communication in the Medical Sector (n=1, 0.5%, SD 0.97) play a minor role. We identified three intertwined clusters displaying diverse barriers to the digitalization of GPs’ communication practices: (1) incompatibility issues and technical immaturity, (2) lack of knowledge and technical requirements, and (3) additional technical, financial, and time-related burdens. These barriers were perceived as significant deterrents to the adoption of digital tools, with older GPs more reliant on analog systems and more likely to view digitalization as a source of frustration. Conclusions: Newly established communication channels in the German health care system compete with the existing information infrastructure, which is deeply integrated into GPs’ practice routines and care processes. However, this installed base has been largely overlooked in digital transformation initiatives. While newer channels hold potential, they often malfunction and are incompatible with long-established, individualized GP workflows. Addressing these issues rather than imposing coercive measures is crucial for increasing adoption. Incorporating health care providers’ perspectives and aligning new channels with established routines can prevent frustration and facilitate a smoother digital transformation. %M 40127672 %R 10.2196/65241 %U https://www.jmir.org/2025/1/e65241 %U https://doi.org/10.2196/65241 %U http://www.ncbi.nlm.nih.gov/pubmed/40127672 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 11 %N %P e59971 %T Meeting Global Health Needs via Infectious Disease Forecasting: Development of a Reliable Data-Driven Framework %A Keshavamurthy,Ravikiran %A Pazdernik,Karl T %A Ham,Colby %A Dixon,Samuel %A Erwin,Samantha %A Charles,Lauren E %K disease forecasting %K machine learning %K deep learning %K epidemiology %K One Health %K decision-making %K data visualization %D 2025 %7 21.3.2025 %9 %J JMIR Public Health Surveill %G English %X Background: Infectious diseases (IDs) have a significant detrimental impact on global health. Timely and accurate ID forecasting can result in more informed implementation of control measures and prevention policies. Objective: To meet the operational decision-making needs of real-world circumstances, we aimed to build a standardized, reliable, and trustworthy ID forecasting pipeline and visualization dashboard that is generalizable across a wide range of modeling techniques, IDs, and global locations. Methods: We forecasted 6 diverse, zoonotic diseases (brucellosis, campylobacteriosis, Middle East respiratory syndrome, Q fever, tick-borne encephalitis, and tularemia) across 4 continents and 8 countries. We included a wide range of statistical, machine learning, and deep learning models (n=9) and trained them on a multitude of features (average n=2326) within the One Health landscape, including demography, landscape, climate, and socioeconomic factors. The pipeline and dashboard were created in consideration of crucial operational metrics—prediction accuracy, computational efficiency, spatiotemporal generalizability, uncertainty quantification, and interpretability—which are essential to strategic data-driven decisions. Results: While no single best model was suitable for all disease, region, and country combinations, our ensemble technique selects the best-performing model for each given scenario to achieve the closest prediction. For new or emerging diseases in a region, the ensemble model can predict how the disease may behave in the new region using a pretrained model from a similar region with a history of that disease. The data visualization dashboard provides a clean interface of important analytical metrics, such as ID temporal patterns, forecasts, prediction uncertainties, and model feature importance across all geographic locations and disease combinations. Conclusions: As the need for real-time, operational ID forecasting capabilities increases, this standardized and automated platform for data collection, analysis, and reporting is a major step forward in enabling evidence-based public health decisions and policies for the prevention and mitigation of future ID outbreaks. %R 10.2196/59971 %U https://publichealth.jmir.org/2025/1/e59971 %U https://doi.org/10.2196/59971 %0 Journal Article %@ 2563-6316 %I JMIR Publications %V 6 %N %P e65263 %T Large Language Models for Pediatric Differential Diagnoses in Rural Health Care: Multicenter Retrospective Cohort Study Comparing GPT-3 With Pediatrician Performance %A Mansoor,Masab %A Ibrahim,Andrew F %A Grindem,David %A Baig,Asad %K natural language processing %K NLP %K machine learning %K ML %K artificial intelligence %K language model %K large language model %K LLM %K generative pretrained transformer %K GPT %K pediatrics %D 2025 %7 19.3.2025 %9 %J JMIRx Med %G English %X Background: Rural health care providers face unique challenges such as limited specialist access and high patient volumes, making accurate diagnostic support tools essential. Large language models like GPT-3 have demonstrated potential in clinical decision support but remain understudied in pediatric differential diagnosis. Objective: This study aims to evaluate the diagnostic accuracy and reliability of a fine-tuned GPT-3 model compared to board-certified pediatricians in rural health care settings. Methods: This multicenter retrospective cohort study analyzed 500 pediatric encounters (ages 0‐18 years; n=261, 52.2% female) from rural health care organizations in Central Louisiana between January 2020 and December 2021. The GPT-3 model (DaVinci version) was fine-tuned using the OpenAI application programming interface and trained on 350 encounters, with 150 reserved for testing. Five board-certified pediatricians (mean experience: 12, SD 5.8 years) provided reference standard diagnoses. Model performance was assessed using accuracy, sensitivity, specificity, and subgroup analyses. Results: The GPT-3 model achieved an accuracy of 87.3% (131/150 cases), sensitivity of 85% (95% CI 82%‐88%), and specificity of 90% (95% CI 87%‐93%), comparable to pediatricians’ accuracy of 91.3% (137/150 cases; P=.47). Performance was consistent across age groups (0‐5 years: 54/62, 87%; 6‐12 years: 47/53, 89%; 13‐18 years: 30/35, 86%) and common complaints (fever: 36/39, 92%; abdominal pain: 20/23, 87%). For rare diagnoses (n=20), accuracy was slightly lower (16/20, 80%) but comparable to pediatricians (17/20, 85%; P=.62). Conclusions: This study demonstrates that a fine-tuned GPT-3 model can provide diagnostic support comparable to pediatricians, particularly for common presentations, in rural health care. Further validation in diverse populations is necessary before clinical implementation. %R 10.2196/65263 %U https://xmed.jmir.org/2025/1/e65263 %U https://doi.org/10.2196/65263 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e66568 %T Machine Learning to Assist in Managing Acute Kidney Injury in General Wards: Multicenter Retrospective Study %A Cho,Nam-Jun %A Jeong,Inyong %A Ahn,Se-Jin %A Gil,Hyo-Wook %A Kim,Yeongmin %A Park,Jin-Hyun %A Kang,Sanghee %A Lee,Hwamin %+ , Department of Biomedical Informatics, Korea University College of Medicine, Seongbuk-gu, 73 Goryeodae-ro, Seoul, 02841, Republic of Korea, 82 1063205109, hwamin@korea.ac.kr %K acute kidney injury %K machine learning %K recovery of function %K creatinine %K kidney %K patient rooms %D 2025 %7 18.3.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Most artificial intelligence–based research on acute kidney injury (AKI) prediction has focused on intensive care unit settings, limiting their generalizability to general wards. The lack of standardized AKI definitions and reliance on intensive care units further hinder the clinical applicability of these models. Objective: This study aims to develop and validate a machine learning–based framework to assist in managing AKI and acute kidney disease (AKD) in general ward patients, using a refined operational definition of AKI to improve predictive performance and clinical relevance. Methods: This retrospective multicenter cohort study analyzed electronic health record data from 3 hospitals in South Korea. AKI and AKD were defined using a refined version of the Kidney Disease: Improving Global Outcomes criteria, which included adjustments to baseline serum creatinine estimation and a stricter minimum increase threshold to reduce misclassification due to transient fluctuations. The primary outcome was the development of machine learning models for early prediction of AKI (within 3 days before onset) and AKD (nonrecovery within 7 days after AKI). Results: The final analysis included 135,068 patients. A total of 7658 (8%) patients in the internal cohort and 2898 (7.3%) patients in the external cohort developed AKI. Among the 5429 patients in the internal cohort and 1998 patients in the external cohort for whom AKD progression could be assessed, 896 (16.5%) patients and 287 (14.4%) patients, respectively, progressed to AKD. Using the refined criteria, 2898 cases of AKI were identified, whereas applying the standard Kidney Disease: Improving Global Outcomes criteria resulted in the identification of 5407 cases. Among the 2509 patients who were not classified as having AKI under the refined criteria, 2242 had a baseline serum creatinine level below 0.6 mg/dL, while the remaining 267 experienced a decrease in serum creatinine before the onset of AKI. The final selected early prediction model for AKI achieved an area under the receiver operating characteristic curve of 0.9053 in the internal cohort and 0.8860 in the external cohort. The early prediction model for AKD achieved an area under the receiver operating characteristic curve of 0.8202 in the internal cohort and 0.7833 in the external cohort. Conclusions: The proposed machine learning framework successfully predicted AKI and AKD in general ward patients with high accuracy. The refined AKI definition significantly reduced the classification of patients with transient serum creatinine fluctuations as AKI cases compared to the previous criteria. These findings suggest that integrating this machine learning framework into hospital workflows could enable earlier interventions, optimize resource allocation, and improve patient outcomes. %M 40101226 %R 10.2196/66568 %U https://www.jmir.org/2025/1/e66568 %U https://doi.org/10.2196/66568 %U http://www.ncbi.nlm.nih.gov/pubmed/40101226 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e57358 %T Enhancing Patient Outcome Prediction Through Deep Learning With Sequential Diagnosis Codes From Structured Electronic Health Record Data: Systematic Review %A Hama,Tuankasfee %A Alsaleh,Mohanad M %A Allery,Freya %A Choi,Jung Won %A Tomlinson,Christopher %A Wu,Honghan %A Lai,Alvina %A Pontikos,Nikolas %A Thygesen,Johan H %+ Institute of Health Informatics, University College London, 222 Euston Road, London, NW1 2DA, United Kingdom, 44 0207679200, tuankasfee.hama.21@ucl.ac.uk %K deep learning %K electronic health records %K EHR %K diagnosis codes %K prediction %K patient outcomes %K systematic review %D 2025 %7 18.3.2025 %9 Review %J J Med Internet Res %G English %X Background: The use of structured electronic health records in health care systems has grown rapidly. These systems collect huge amounts of patient information, including diagnosis codes representing temporal medical history. Sequential diagnostic information has proven valuable for predicting patient outcomes. However, the extent to which these types of data have been incorporated into deep learning (DL) models has not been examined. Objective: This systematic review aims to describe the use of sequential diagnostic data in DL models, specifically to understand how these data are integrated, whether sample size improves performance, and whether the identified models are generalizable. Methods: Relevant studies published up to May 15, 2023, were identified using 4 databases: PubMed, Embase, IEEE Xplore, and Web of Science. We included all studies using DL algorithms trained on sequential diagnosis codes to predict patient outcomes. We excluded review articles and non–peer-reviewed papers. We evaluated the following aspects in the included papers: DL techniques, characteristics of the dataset, prediction tasks, performance evaluation, generalizability, and explainability. We also assessed the risk of bias and applicability of the studies using the Prediction Model Study Risk of Bias Assessment Tool (PROBAST). We used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist to report our findings. Results: Of the 740 identified papers, 84 (11.4%) met the eligibility criteria. Publications in this area increased yearly. Recurrent neural networks (and their derivatives; 47/84, 56%) and transformers (22/84, 26%) were the most commonly used architectures in DL-based models. Most studies (45/84, 54%) presented their input features as sequences of visit embeddings. Medications (38/84, 45%) were the most common additional feature. Of the 128 predictive outcome tasks, the most frequent was next-visit diagnosis (n=30, 23%), followed by heart failure (n=18, 14%) and mortality (n=17, 13%). Only 7 (8%) of the 84 studies evaluated their models in terms of generalizability. A positive correlation was observed between training sample size and model performance (area under the receiver operating characteristic curve; P=.02). However, 59 (70%) of the 84 studies had a high risk of bias. Conclusions: The application of DL for advanced modeling of sequential medical codes has demonstrated remarkable promise in predicting patient outcomes. The main limitation of this study was the heterogeneity of methods and outcomes. However, our analysis found that using multiple types of features, integrating time intervals, and including larger sample sizes were generally related to an improved predictive performance. This review also highlights that very few studies (7/84, 8%) reported on challenges related to generalizability and less than half (38/84, 45%) of the studies reported on challenges related to explainability. Addressing these shortcomings will be instrumental in unlocking the full potential of DL for enhancing health care outcomes and patient care. Trial Registration: PROSPERO CRD42018112161; https://tinyurl.com/yc6h9rwu %M 40100249 %R 10.2196/57358 %U https://www.jmir.org/2025/1/e57358 %U https://doi.org/10.2196/57358 %U http://www.ncbi.nlm.nih.gov/pubmed/40100249 %0 Journal Article %@ 2817-1705 %I JMIR Publications %V 4 %N %P e55277 %T Creation of Scientific Response Documents for Addressing Product Medical Information Inquiries: Mixed Method Approach Using Artificial Intelligence %A Lau,Jerry %A Bisht,Shivani %A Horton,Robert %A Crisan,Annamaria %A Jones,John %A Gantotti,Sandeep %A Hermes-DeSantis,Evelyn %+ phactMI, 5931 NW 1st Place, Gainesville, FL, 32607, United States, 1 2155881585, evelyn@phactmi.org %K AI %K LLM %K GPT %K biopharmaceutical %K medical information %K content generation %K artificial intelligence %K pharmaceutical %K scientific response %K documentation %K information %K clinical data %K strategy %K reference %K feasibility %K development %K machine learning %K large language model %K accuracy %K context %K traceability %K accountability %K survey %K scientific response documentation %K SRD %K benefit %K content generator %K content analysis %K Generative Pre-trained Transformer %D 2025 %7 13.3.2025 %9 Original Paper %J JMIR AI %G English %X Background: Pharmaceutical manufacturers address health care professionals’ information needs through scientific response documents (SRDs), offering evidence-based answers to medication and disease state questions. Medical information departments, staffed by medical experts, develop SRDs that provide concise summaries consisting of relevant background information, search strategies, clinical data, and balanced references. With an escalating demand for SRDs and the increasing complexity of therapies, medical information departments are exploring advanced technologies and artificial intelligence (AI) tools like large language models (LLMs) to streamline content development. While AI and LLMs show promise in generating draft responses, a synergistic approach combining an LLM with traditional machine learning classifiers in a series of human-supervised and -curated steps could help address limitations, including hallucinations. This will ensure accuracy, context, traceability, and accountability in the development of the concise clinical data summaries of an SRD. Objective: This study aims to quantify the challenges of SRD development and develop a framework exploring the feasibility and value addition of integrating AI capabilities in the process of creating concise summaries for an SRD. Methods: To measure the challenges in SRD development, a survey was conducted by phactMI, a nonprofit consortium of medical information leaders in the pharmaceutical industry, assessing aspects of SRD creation among its member companies. The survey collected data on the time and tediousness of various activities related to SRD development. Another working group, consisting of medical information professionals and data scientists, used AI to aid SRD authoring, focusing on data extraction and abstraction. They used logistic regression on semantic embedding features to train classification models and transformer-based summarization pipelines to generate concise summaries. Results: Of the 33 companies surveyed, 64% (21/33) opened the survey, and 76% (16/21) of those responded. On average, medical information departments generate 614 new documents and update 1352 documents each year. Respondents considered paraphrasing scientific articles to be the most tedious and time-intensive task. In the project’s second phase, sentence classification models showed the ability to accurately distinguish target categories with receiver operating characteristic scores ranging from 0.67 to 0.85 (all P<.001), allowing for accurate data extraction. For data abstraction, the comparison of the bilingual evaluation understudy (BLEU) score and semantic similarity in the paraphrased texts yielded different results among reviewers, with each preferring different trade-offs between these metrics. Conclusions: This study establishes a framework for integrating LLM and machine learning into SRD development, supported by a pharmaceutical company survey emphasizing the challenges of paraphrasing content. While machine learning models show potential for section identification and content usability assessment in data extraction and abstraction, further optimization and research are essential before full-scale industry implementation. The working group’s insights guide an AI-driven content analysis; address limitations; and advance efficient, precise, and responsive frameworks to assist with pharmaceutical SRD development. %M 40080808 %R 10.2196/55277 %U https://ai.jmir.org/2025/1/e55277 %U https://doi.org/10.2196/55277 %U http://www.ncbi.nlm.nih.gov/pubmed/40080808 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 13 %N %P e63216 %T Large Language Model–Based Critical Care Big Data Deployment and Extraction: Descriptive Analysis %A Yang,Zhongbao %A Xu,Shan-Shan %A Liu,Xiaozhu %A Xu,Ningyuan %A Chen,Yuqing %A Wang,Shuya %A Miao,Ming-Yue %A Hou,Mengxue %A Liu,Shuai %A Zhou,Yi-Min %A Zhou,Jian-Xin %A Zhang,Linlin %K big data %K critical care–related databases %K database deployment %K large language model %K database extraction %K intensive care unit %K ICU %K GPT %K artificial intelligence %K AI %K LLM %D 2025 %7 12.3.2025 %9 %J JMIR Med Inform %G English %X Background: Publicly accessible critical care–related databases contain enormous clinical data, but their utilization often requires advanced programming skills. The growing complexity of large databases and unstructured data presents challenges for clinicians who need programming or data analysis expertise to utilize these systems directly. Objective: This study aims to simplify critical care–related database deployment and extraction via large language models. Methods: The development of this platform was a 2-step process. First, we enabled automated database deployment using Docker container technology, with incorporated web-based analytics interfaces Metabase and Superset. Second, we developed the intensive care unit–generative pretrained transformer (ICU-GPT), a large language model fine-tuned on intensive care unit (ICU) data that integrated LangChain and Microsoft AutoGen. Results: The automated deployment platform was designed with user-friendliness in mind, enabling clinicians to deploy 1 or multiple databases in local, cloud, or remote environments without the need for manual setup. After successfully overcoming GPT’s token limit and supporting multischema data, ICU-GPT could generate Structured Query Language (SQL) queries and extract insights from ICU datasets based on request input. A front-end user interface was developed for clinicians to achieve code-free SQL generation on the web-based client. Conclusions: By harnessing the power of our automated deployment platform and ICU-GPT model, clinicians are empowered to easily visualize, extract, and arrange critical care–related databases more efficiently and flexibly than manual methods. Our research could decrease the time and effort spent on complex bioinformatics methods and advance clinical research. %R 10.2196/63216 %U https://medinform.jmir.org/2025/1/e63216 %U https://doi.org/10.2196/63216 %0 Journal Article %@ 2563-3570 %I JMIR Publications %V 6 %N %P e65645 %T Investigating Associations Between Prognostic Factors in Gliomas: Unsupervised Multiple Correspondence Analysis %A Goes Job,Maria Eduarda %A Fukumasu,Heidge %A Malta,Tathiane Maistro %A Porfirio Xavier,Pedro Luiz %K brain tumors %K bioinformatics %K stemness %K multiple correspondence analysis %D 2025 %7 12.3.2025 %9 %J JMIR Bioinform Biotech %G English %X Background: Multiple correspondence analysis (MCA) is an unsupervised data science methodology that aims to identify and represent associations between categorical variables. Gliomas are an aggressive type of cancer characterized by diverse molecular and clinical features that serve as key prognostic factors. Thus, advanced computational approaches are essential to enhance the analysis and interpretation of the associations between clinical and molecular features in gliomas. Objective: This study aims to apply MCA to identify associations between glioma prognostic factors and also explore their associations with stemness phenotype. Methods: Clinical and molecular data from 448 patients with brain tumors were obtained from the Cancer Genome Atlas. The DNA methylation stemness index, derived from DNA methylation patterns, was built using a one-class logistic regression. Associations between variables were evaluated using the χ² test with k degrees of freedom, followed by analysis of the adjusted standardized residuals (ASRs >1.96 indicate a significant association between variables). MCA was used to uncover associations between glioma prognostic factors and stemness. Results: Our analysis revealed significant associations among molecular and clinical characteristics in gliomas. Additionally, we demonstrated the capability of MCA to identify associations between stemness and these prognostic factors. Our results exhibited a strong association between higher DNA methylation stemness index and features related to poorer prognosis such as glioblastoma cancer type (ASR: 8.507), grade 4 (ASR: 8.507), isocitrate dehydrogenase wild type (ASR:15.904), unmethylated MGMT (methylguanine methyltransferase) Promoter (ASR: 9.983), and telomerase reverse transcriptase expression (ASR: 3.351), demonstrating the utility of MCA as an analytical tool for elucidating potential prognostic factors. Conclusions: MCA is a valuable tool for understanding the complex interdependence of prognostic markers in gliomas. MCA facilitates the exploration of large-scale datasets and enhances the identification of significant associations. %R 10.2196/65645 %U https://bioinform.jmir.org/2025/1/e65645 %U https://doi.org/10.2196/65645 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e67488 %T Accuracy of Large Language Models for Literature Screening in Thoracic Surgery: Diagnostic Study %A Dai,Zhang-Yi %A Wang,Fu-Qiang %A Shen,Cheng %A Ji,Yan-Li %A Li,Zhi-Yang %A Wang,Yun %A Pu,Qiang %+ Department of Thoracic Surgery, West China Hospital of Sichuan University, No.37, Guoxue Alley, Chengdu, 610041, China, 86 18980606738, puqiang100@163.com %K accuracy %K large language models %K meta-analysis %K literature screening %K thoracic surgery %D 2025 %7 11.3.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Systematic reviews and meta-analyses rely on labor-intensive literature screening. While machine learning offers potential automation, its accuracy remains suboptimal. This raises the question of whether emerging large language models (LLMs) can provide a more accurate and efficient approach. Objective: This paper evaluates the sensitivity, specificity, and summary receiver operating characteristic (SROC) curve of LLM-assisted literature screening. Methods: We conducted a diagnostic study comparing the accuracy of LLM-assisted screening versus manual literature screening across 6 thoracic surgery meta-analyses. Manual screening by 2 investigators served as the reference standard. LLM-assisted screening was performed using ChatGPT-4o (OpenAI) and Claude-3.5 (Anthropic) sonnet, with discrepancies resolved by Gemini-1.5 pro (Google). In addition, 2 open-source, machine learning–based screening tools, ASReview (Utrecht University) and Abstrackr (Center for Evidence Synthesis in Health, Brown University School of Public Health), were also evaluated. We calculated sensitivity, specificity, and 95% CIs for the title and abstract, as well as full-text screening, generating pooled estimates and SROC curves. LLM prompts were revised based on a post hoc error analysis. Results: LLM-assisted full-text screening demonstrated high pooled sensitivity (0.87, 95% CI 0.77-0.99) and specificity (0.96, 95% CI 0.91-0.98), with the area under the curve (AUC) of 0.96 (95% CI 0.94-0.97). Title and abstract screening achieved a pooled sensitivity of 0.73 (95% CI 0.57-0.85) and specificity of 0.99 (95% CI 0.97-0.99), with an AUC of 0.97 (95% CI 0.96-0.99). Post hoc revisions improved sensitivity to 0.98 (95% CI 0.74-1.00) while maintaining high specificity (0.98, 95% CI 0.94-0.99). In comparison, the pooled sensitivity and specificity of ASReview tool-assisted screening were 0.58 (95% CI 0.53-0.64) and 0.97 (95% CI 0.91-0.99), respectively, with an AUC of 0.66 (95% CI 0.62-0.70). The pooled sensitivity and specificity of Abstrackr tool-assisted screening were 0.48 (95% CI 0.35-0.62) and 0.96 (95% CI 0.88-0.99), respectively, with an AUC of 0.78 (95% CI 0.74-0.82). A post hoc meta-analysis revealed comparable effect sizes between LLM-assisted and conventional screening. Conclusions: LLMs hold significant potential for streamlining literature screening in systematic reviews, reducing workload without sacrificing quality. Importantly, LLMs outperformed traditional machine learning-based tools (ASReview and Abstrackr) in both sensitivity and AUC values, suggesting that LLMs offer a more accurate and efficient approach to literature screening. %M 40068152 %R 10.2196/67488 %U https://www.jmir.org/2025/1/e67488 %U https://doi.org/10.2196/67488 %U http://www.ncbi.nlm.nih.gov/pubmed/40068152 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e59792 %T Generative AI Models in Time-Varying Biomedical Data: Scoping Review %A He,Rosemary %A Sarwal,Varuni %A Qiu,Xinru %A Zhuang,Yongwen %A Zhang,Le %A Liu,Yue %A Chiang,Jeffrey %+ Department of Neurosurgery, David Geffen School of Medicine, University of California, Los Angeles, 300 Stein Plaza, Suite 560, Los Angeles, CA, 90095, United States, 1 310 825 5111, njchiang@g.ucla.edu %K generative artificial intelligence %K artificial intelligence %K time series %K electronic health records %K electronic medical records %K systematic reviews %K disease trajectory %K machine learning %K algorithms %K forecasting %D 2025 %7 10.3.2025 %9 Review %J J Med Internet Res %G English %X Background: Trajectory modeling is a long-standing challenge in the application of computational methods to health care. In the age of big data, traditional statistical and machine learning methods do not achieve satisfactory results as they often fail to capture the complex underlying distributions of multimodal health data and long-term dependencies throughout medical histories. Recent advances in generative artificial intelligence (AI) have provided powerful tools to represent complex distributions and patterns with minimal underlying assumptions, with major impact in fields such as finance and environmental sciences, prompting researchers to apply these methods for disease modeling in health care. Objective: While AI methods have proven powerful, their application in clinical practice remains limited due to their highly complex nature. The proliferation of AI algorithms also poses a significant challenge for nondevelopers to track and incorporate these advances into clinical research and application. In this paper, we introduce basic concepts in generative AI and discuss current algorithms and how they can be applied to health care for practitioners with little background in computer science. Methods: We surveyed peer-reviewed papers on generative AI models with specific applications to time-series health data. Our search included single- and multimodal generative AI models that operated over structured and unstructured data, physiological waveforms, medical imaging, and multi-omics data. We introduce current generative AI methods, review their applications, and discuss their limitations and future directions in each data modality. Results: We followed the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines and reviewed 155 articles on generative AI applications to time-series health care data across modalities. Furthermore, we offer a systematic framework for clinicians to easily identify suitable AI methods for their data and task at hand. Conclusions: We reviewed and critiqued existing applications of generative AI to time-series health data with the aim of bridging the gap between computational methods and clinical application. We also identified the shortcomings of existing approaches and highlighted recent advances in generative AI that represent promising directions for health care modeling. %M 40063929 %R 10.2196/59792 %U https://www.jmir.org/2025/1/e59792 %U https://doi.org/10.2196/59792 %U http://www.ncbi.nlm.nih.gov/pubmed/40063929 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 13 %N %P e59801 %T Applying Robotic Process Automation to Monitor Business Processes in Hospital Information Systems: Mixed Method Approach %A Park,Adam %A Jung,Se Young %A Yune,Ilha %A Lee,Ho-Young %+ , Office of eHealth Research and Business, Seoul National University Bundang Hospital, Dolmar-ro 182, Seongnam-si, 13605, Republic of Korea, 82 31 787 0114, debobkr@gmail.com %K robotic process automation %K RPA %K electronic medical records %K EMR %K system monitoring %K health care information systems %K user-centric monitoring %K performance evaluation %K business process management %K BPM %K healthcare technology %K mixed methods research %K process automation in health care %D 2025 %7 7.3.2025 %9 Original Paper %J JMIR Med Inform %G English %X Background: Electronic medical records (EMRs) have undergone significant changes due to advancements in technology, including artificial intelligence, the Internet of Things, and cloud services. The increasing complexity within health care systems necessitates enhanced process reengineering and system monitoring approaches. Robotic process automation (RPA) provides a user-centric approach to monitoring system complexity by mimicking end user interactions, thus presenting potential improvements in system performance and monitoring. Objective: This study aimed to explore the application of RPA in monitoring the complexities of EMR systems within a hospital environment, focusing on RPA’s ability to perform end-to-end performance monitoring that closely reflects real-time user experiences. Methods: The research was conducted at Seoul National University Bundang Hospital using a mixed methods approach. It included the iterative development and integration of RPA bots programmed to simulate and monitor typical user interactions with the hospital’s EMR system. Quantitative data from RPA process outputs and qualitative insights from interviews with system engineers and managers were used to evaluate the effectiveness of RPA in system monitoring. Results: RPA bots effectively identified and reported system inefficiencies and failures, providing a bridge between end user experiences and engineering assessments. The bots were particularly useful in detecting delays and errors immediately following system updates or interactions with external services. Over 3 years, RPA monitoring highlighted discrepancies between user-reported experiences and traditional engineering metrics, with the bots frequently identifying critical system issues that were not evident from standard component-level monitoring. Conclusions: RPA enhances system monitoring by providing insights that reflect true end user experiences, which are often overlooked by traditional monitoring methods. The study confirms the potential of RPA to act as a comprehensive monitoring tool within complex health care systems, suggesting that RPA can significantly contribute to the maintenance and improvement of EMR systems by providing a more accurate and timely reflection of system performance and user satisfaction. %M 40053771 %R 10.2196/59801 %U https://medinform.jmir.org/2025/1/e59801 %U https://doi.org/10.2196/59801 %U http://www.ncbi.nlm.nih.gov/pubmed/40053771 %0 Journal Article %@ 2817-1705 %I JMIR Publications %V 4 %N %P e64279 %T Deep Learning Models to Predict Diagnostic and Billing Codes Following Visits to a Family Medicine Practice: Development and Validation Study %A Rajaram,Akshay %A Judd,Michael %A Barber,David %K machine learning %K ML %K artificial intelligence %K algorithm %K predictive model %K predictive analytics %K predictive system %K family medicine %K primary care %K family doctor %K family physician %K income %K billing code %K electronic notes %K electronic health record %K electronic medical record %K EMR %K patient record %K health record %K personal health record %D 2025 %7 7.3.2025 %9 %J JMIR AI %G English %X Background: Despite significant time spent on billing, family physicians routinely make errors and miss billing opportunities. In other disciplines, machine learning models have predicted Current Procedural Terminology codes with high accuracy. Objective: Our objective was to derive machine learning models capable of predicting diagnostic and billing codes from notes recorded in the electronic medical record. Methods: We conducted a retrospective algorithm development and validation study involving an academic family medicine practice. Visits between July 1, 2015, and June 30, 2020, containing a physician-authored note and an invoice in the electronic medical record were eligible for inclusion. We trained 2 deep learning models and compared their predictions to codes submitted for reimbursement. We calculated accuracy, recall, precision, F1-score, and area under the receiver operating characteristic curve. Results: Of the 245,045 visits eligible for inclusion, 198,802 (81%) were included in model development. Accuracy was 99.8% and 99.5% for the diagnostic and billing code models, respectively. Recall was 49.4% and 70.3% for the diagnostic and billing code models, respectively. Precision was 55.3% and 76.7% for the diagnostic and billing code models, respectively. The area under the receiver operating characteristic curve was 0.983 for the diagnostic code model and 0.993 for the billing code model. Conclusions: We developed models capable of predicting diagnostic and billing codes from electronic notes following visits to a family medicine practice. The billing code model outperformed the diagnostic code model in terms of recall and precision, likely due to fewer codes being predicted. Work is underway to further enhance model performance and assess the generalizability of these models to other family medicine practices. %R 10.2196/64279 %U https://ai.jmir.org/2025/1/e64279 %U https://doi.org/10.2196/64279 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e64721 %T Emerging Domains for Measuring Health Care Delivery With Electronic Health Record Metadata %A Tawfik,Daniel %A Rule,Adam %A Alexanian,Aram %A Cross,Dori %A Holmgren,A Jay %A Lou,Sunny S %A McPeek Hinz,Eugenia %A Rose,Christian %A Viswanadham,Ratnalekha V N %A Mishuris,Rebecca G %A Rodríguez-Fernández,Jorge M %A Ford,Eric W %A Florig,Sarah T %A Sinsky,Christine A %A Apathy,Nate C %+ Department of Pediatrics, Stanford University School of Medicine, 770 Welch Road, Suite 435, Palo Alto, CA, 94304, United States, 1 6507239902, dtawfik@stanford.edu %K metadata %K health services research %K audit logs %K event logs %K electronic health record data %K health care delivery %K patient care %K healthcare teams %K clinician-patient relationship %K cognitive environment %D 2025 %7 6.3.2025 %9 Viewpoint %J J Med Internet Res %G English %X This article aims to introduce emerging measurement domains made feasible through the electronic health record (EHR) use metadata, to inform the changing landscape of health care delivery. We reviewed emerging domains in which EHR metadata may be used to measure health care delivery, outlining a framework for evaluating measures based on desirability, feasibility, and viability. We argue that EHR use metadata may be leveraged to develop and operationalize novel measures in the domains of team structure and dynamics, workflows, and cognitive environment to provide a clearer understanding of modern health care delivery. Examples of measures feasible using metadata include quantification of teamwork and collaboration, patient continuity measures, workflow conformity measures, and attention switching. By enabling measures that can be used to inform the next generation of health care delivery, EHR metadata may be used to improve the quality of patient care and support clinician well-being. Careful attention is needed to ensure that these measures are desirable, feasible, and viable. %M 40053814 %R 10.2196/64721 %U https://www.jmir.org/2025/1/e64721 %U https://doi.org/10.2196/64721 %U http://www.ncbi.nlm.nih.gov/pubmed/40053814 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 11 %N %P e52119 %T Building and Developing a Tool (PANDEM-2 Dashboard) to Strengthen Pandemic Management: Participatory Design Study %A Tighe,Carlos %A Ngongalah,Lem %A Sentís,Alexis %A Orchard,Francisco %A Pacurar,Gheorghe-Aurel %A Hayes,Conor %A Hayes,Jessica S %A Toader,Adrian %A Connolly,Máire A %+ School of Health Sciences, University of Galway, University of Galway, University Road, Galway, Ireland, Galway, H91 TK33, Ireland, 353 91524411, jessica.hayes@universityofgalway.ie %K pandemic preparedness and response %K COVID-19 %K cross-border collaboration %K surveillance %K data collection %K data standardization %K data sharing %K dashboard %K IT system %K IT tools %D 2025 %7 5.3.2025 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: The COVID-19 pandemic exposed challenges in pandemic management, particularly in real-time data sharing and effective decision-making. Data protection concerns and the lack of data interoperability and standardization hindered the collection, analysis, and interpretation of critical information. Effective data visualization and customization are essential to facilitate decision-making. Objective: This study describes the development of the PANDEM-2 dashboard, a system providing a standardized and interactive platform for decision-making in pandemic management. It outlines the participatory approaches used to involve expert end users in its development and addresses key considerations of privacy, data protection, and ethical and social issues. Methods: Development was informed by a review of 25 publicly available COVID-19 dashboards, leading to the creation of a visualization catalog. User requirements were gathered through workshops and consultations with 20 experts from various health care and public health professions in 13 European Union countries. These were further refined by mapping variables and indicators required to fulfill the identified needs. Through a participatory design process, end users interacted with a preprototype platform, explored potential interface designs, and provided feedback to refine the system’s components. Potential privacy, data protection, and ethical and social risks associated with the technology, along with mitigation strategies, were identified through an iterative impact assessment. Results: Key variables incorporated into the PANDEM-2 dashboard included case rates, number of deaths, mortality rates, hospital resources, hospital admissions, testing, contact tracing, and vaccination uptake. Cases, deaths, and vaccination uptake were prioritized as the most relevant and readily available variables. However, data gaps, particularly in contact tracing and mortality rates, highlighted the need for better data collection and reporting mechanisms. User feedback emphasized the importance of diverse data visualization formats combining different data types, as well as analyzing data across various time frames. Users also expressed interest in generating custom visualizations and reports, especially on the impact of government interventions. Participants noted challenges in data reporting, such as inconsistencies in reporting levels, time intervals, the need for standardization between member states, and General Data Protection Regulation concerns for data sharing. Identified risks included ethical concerns (accessibility, user autonomy, responsible use, transparency, and accountability), privacy and data protection (security and access controls and data reidentification), and social issues (unintentional bias, data quality and accuracy, dependency on technology, and collaborative development). Mitigation measures focused on designing user-friendly interfaces, implementing robust security protocols, and promoting cross-member state collaboration. Conclusions: The PANDEM-2 dashboard provides an adaptable, user-friendly platform for pandemic preparedness and response. Our findings highlight the critical role of data interoperability, cross-border collaboration, and custom IT tools in strengthening future health crisis management. They also offer valuable insights into the challenges and opportunities in developing IT solutions to support pandemic preparedness. %M 40053759 %R 10.2196/52119 %U https://publichealth.jmir.org/2025/1/e52119 %U https://doi.org/10.2196/52119 %U http://www.ncbi.nlm.nih.gov/pubmed/40053759 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e59217 %T Opportunities and Challenges in Using Electronic Health Record Systems to Study Postacute Sequelae of SARS-CoV-2 Infection: Insights From the NIH RECOVER Initiative %A Mandel,Hannah L %A Shah,Shruti N %A Bailey,L Charles %A Carton,Thomas %A Chen,Yu %A Esquenazi-Karonika,Shari %A Haendel,Melissa %A Hornig,Mady %A Kaushal,Rainu %A Oliveira,Carlos R %A Perlowski,Alice A %A Pfaff,Emily %A Rao,Suchitra %A Razzaghi,Hanieh %A Seibert,Elle %A Thomas,Gelise L %A Weiner,Mark G %A Thorpe,Lorna E %A Divers,Jasmin %A , %+ Department of Population Health, New York University Grossman School of Medicine, 180 Madison Avenue, New York, NY, 10016, United States, 1 732 314 1595, Hannah.Mandel@nyulangone.org %K COVID-19 %K SARS-CoV-2 %K Long COVID, post-acute COVID-19 syndrome %K electronic health records %K machine learning %K public health surveillance %K post-infection syndrome %K medical informatics %K electronic medical record %K electronic health record network %K electronic health record data %K clinical research network %K clinical data research network %K common data model %K digital health %K infection %K respiratory %K infectious %K epidemiological %K pandemic %D 2025 %7 5.3.2025 %9 Viewpoint %J J Med Internet Res %G English %X The benefits and challenges of electronic health records (EHRs) as data sources for clinical and epidemiologic research have been well described. However, several factors are important to consider when using EHR data to study novel, emerging, and multifaceted conditions such as postacute sequelae of SARS-CoV-2 infection or long COVID. In this article, we present opportunities and challenges of using EHR data to improve our understanding of long COVID, based on lessons learned from the National Institutes of Health (NIH)–funded RECOVER (REsearching COVID to Enhance Recovery) Initiative, and suggest steps to maximize the usefulness of EHR data when performing long COVID research. %M 40053748 %R 10.2196/59217 %U https://www.jmir.org/2025/1/e59217 %U https://doi.org/10.2196/59217 %U http://www.ncbi.nlm.nih.gov/pubmed/40053748 %0 Journal Article %@ 1929-073X %I JMIR Publications %V 14 %N %P e63291 %T Investigating the Association Between Mean Arterial Pressure on 28-Day Mortality Risk in Patients With Sepsis: Retrospective Cohort Study Based on the MIMIC-IV Database %A Chen,Qimin %A Li,Wei %A Wang,Ying %A Chen,Xianjun %A He,Dehua %A Liu,Ming %A Yuan,Jia %A Xiao,Chuan %A Li,Qing %A Chen,Lu %A Shen,Feng %K mean arterial pressure %K 28-day mortality %K sepsis %K MIMIC-Ⅳ %K retrospective study %K Medical Information Mart for Intensive Care IV %D 2025 %7 5.3.2025 %9 %J Interact J Med Res %G English %X Background: Sepsis is a globally recognized health issue that continues to contribute significantly to mortality and morbidity in intensive care units (ICUs). The association between mean arterial pressure (MAP) and prognosis among patients with patients is yet to be demonstrated. Objective: The aim of this study was to explore the association between MAP and 28-day mortality in ICU patients with sepsis using data from a large, multicenter database. Methods: This is a retrospective cohort study. We extracted data of 35,010 patients with sepsis from the MIMIC-IV (Medical Information Mart for Intensive Care) database between 2008 and 2019, according to the Sepsis 3.0 diagnostic criteria. The MAP was calculated as the average of the highest and lowest readings within the first 24 hours of ICU admission, and patients were divided into 4 groups based on the mean MAP, using the quadruple classification approach. Other worst-case indications from the first 24 hours of ICU admission, such as vital signs, severity of illness scores, laboratory indicators, and therapies, were also gathered as baseline data. The independent effects of MAP on 28-day mortality were explored using binary logistic regression and a two-piecewise linear model, with MAP as the exposure and 28-day mortality as the outcome variables, respectively. To address the nonlinearity relationship, curve fitting and a threshold effect analysis were performed. Results: A total of 34,981 patients with sepsis were included in the final analysis, the mean age was 66.67 (SD 16.01) years, and the 28-day mortality rate was 16.27% (5691/34,981). The generalized additive model and smoothed curve fitting found a U-shaped relationship between MAP and 28-day mortality in these patients. The recursive algorithm determined the low and high inflection points as 70 mm and 82 mm Hg, respectively. Our data demonstrated that MAP was negatively associated with 28-day mortality in the range of 34.05 mm Hg-69.34 mm Hg (odds ratio [OR] 0.93, 95% CI 0.92-0.94; P<.001); however, once the MAP exceeded 82 mm Hg, a positive association existed between MAP and 28-day mortality of patients with sepsis (OR 1.01; 95% CI 1.01-1.02, P=.002). Conclusions: There is a U-shaped association between MAP and the probability of 28-day mortality in patients with sepsis. Both the lower and higher MAP were related with a higher risk of mortality in patients with sepsis. These patients have a decreased risk of mortality when their MAP remains between 70 and 82 mm Hg. %R 10.2196/63291 %U https://www.i-jmr.org/2025/1/e63291 %U https://doi.org/10.2196/63291 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 13 %N %P e63740 %T Linking Electronic Health Record Prescribing Data and Pharmacy Dispensing Records to Identify Patient-Level Factors Associated With Psychotropic Medication Receipt: Retrospective Study %A Wu,Peng %A Hurst,Jillian H %A French,Alexis %A Chrestensen,Michael %A Goldstein,Benjamin A %K electronic health records %K pharmacy dispensing %K psychotropic medications %K prescriptions %K predictive modeling %D 2025 %7 4.3.2025 %9 %J JMIR Med Inform %G English %X Background: Pharmacoepidemiology studies using electronic health record (EHR) data typically rely on medication prescriptions to determine which patients have received a medication. However, such data do not affirmatively indicate whether these prescriptions have been filled. External dispensing databases can bridge this information gap; however, few established methods exist for linking EHR data and pharmacy dispensing records. Objective: We described a process for linking EHR prescribing data with pharmacy dispensing records from Surescripts. As a use case, we considered the prescriptions and resulting fills for psychotropic medications among pediatric patients. We evaluated how dispensing information affects identifying patients receiving prescribed medications and assessing the association between filling prescriptions and subsequent health behaviors. Methods: This retrospective study identified all new psychotropic prescriptions to patients younger than 18 years of age at Duke University Health System in 2021. We linked dispensing to prescribing data using proximate dates and matching codes between RxNorm concept unique identifiers and National Drug Codes. We described demographic, clinical, and service use characteristics to assess differences between patients who did versus did not fill prescriptions. We fit a least absolute shrinkage and selection operator (LASSO) regression model to evaluate the predictability of a fill. We then fit time-to-event models to assess the association between whether a patient filled a prescription and a future provider visit. Results: We identified 1254 pediatric patients with a new psychotropic prescription. In total, 976 (77.8%) patients filled their prescriptions within 30 days of their prescribing encounters. Thus, we set 30 days as a cut point for defining a valid prescription fill. Patients who filled prescriptions differed from those who did not in several key factors. Those who did not fill had slightly higher BMIs, lived in more disadvantaged neighborhoods, were more likely to have public insurance or self-pay, and included a higher proportion of male patients. Patients with prior well-child visits or prescriptions from primary care providers were more likely to fill. Additionally, patients with anxiety diagnoses and those prescribed selective serotonin reuptake inhibitors were more likely to fill prescriptions. The LASSO model achieved an area under the receiver operator characteristic curve of 0.816. The time to the follow-up visit with the same provider was censored at 90 days after the initial encounter. Patients who filled prescriptions showed higher levels of follow-up visits. The marginal hazard ratio of a follow-up visit with the same provider was 1.673 (95% CI 1.463‐1.913) for patients who filled their prescriptions. Using the LASSO model as a propensity-based weight, we calculated the weighted hazard ratio as 1.447 (95% CI 1.257‐1.665). Conclusions: Systematic differences existed between patients who did versus did not fill prescriptions. Incorporating external dispensing databases into EHR-based studies informs medication receipt and associated health outcomes. %R 10.2196/63740 %U https://medinform.jmir.org/2025/1/e63740 %U https://doi.org/10.2196/63740 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e60077 %T 25 Years of Electronic Health Record Implementation Processes: Scoping Review %A Finnegan,Harriet %A Mountford,Nicola %+ School of Business, Maynooth University, 3rd Floor TSI Building, Maynooth, Kildare, W23 X04D, Ireland, 353 17083609, harriet.finnegan.2017@mumail.ie %K electronic health record system %K EHR %K electronic medical record %K EMR %K scoping review %K process %K implementation %D 2025 %7 3.3.2025 %9 Review %J J Med Internet Res %G English %X Background: Electronic health record (EHR) systems have undergone substantial evolution over the past 25 years, transitioning from rudimentary digital repositories to sophisticated tools that are integral to modern health care delivery. These systems have the potential to increase efficiency and improve patient care. However, for these systems to reach their potential, we need to understand how the process of EHR implementation works. Objective: This scoping review aimed to examine the implementation process of EHRs from 1999 to 2024 and to articulate process-focused recommendations for future EHR implementations that build on this history of EHR research. Methods: We conducted a scoping literature review following a systematic methodological framework. A total of 5 databases were selected from the disciplines of medicine and business: EBSCO, PubMed, Embase, IEEE Explore, and Scopus. The search included studies published from 1999 to 2024 that addressed the process of implementing an EHR. Keywords included “EHR,” “EHRS,” “Electronic Health Record*,” “EMR,” “EMRS,” “Electronic Medical Record*,” “implemen*,” and “process.” The findings were reported in accordance with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) checklist. The selected literature was thematically coded using NVivo qualitative analysis software, with the results reported qualitatively. Results: This review included 90 studies that described the process of EHR implementation in different settings. The studies identified key elements, such as the role of the government and vendors, the importance of communication and relationships, the provision of training and support, and the implementation approach and cost. Four process-related categories emerged from these results: compliance processes, collaboration processes, competence-development processes, and process costs. Conclusions: Although EHRs hold immense promise in improving patient care, enhancing research capabilities, and optimizing health care efficiency, there is a pressing need to examine the actual implementation process to understand how to approach implementation. Our findings offer 7 process-focused recommendations for EHR implementation formed from analysis of the selected literature. %M 40053758 %R 10.2196/60077 %U https://www.jmir.org/2025/1/e60077 %U https://doi.org/10.2196/60077 %U http://www.ncbi.nlm.nih.gov/pubmed/40053758 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 9 %N %P e56254 %T Daily Treatment Monitoring for Patients Receiving Home-Based Peritoneal Dialysis and Prediction of Heart Failure Risk: mHealth Tool Development and Modeling Study %A Wu,Jia %A Zeng,Youjia %A Yang,Jun %A Yao,Yutong %A Xu,Xiuling %A Song,Gaofeng %A Yi,Wuyong %A Wang,Taifen %A Zheng,Yihou %A Jia,Zhongwei %A Yan,Xiangyu %+ , School of Disaster and Emergency Medicine, Tianjin University, No 92 Weijin Road, Tianjin, 300072, China, 86 02287370177 ext 307, yanxiangyu1123@163.com %K peritoneal dialysis %K mHealth %K patient management %K heart failure %K prediction model %D 2025 %7 3.3.2025 %9 Original Paper %J JMIR Form Res %G English %X Background: Peritoneal dialysis is one of the major renal replacement modalities for patients with end-stage renal disease. Heart failure is a common adverse event among patients who undergo peritoneal dialysis treatment, especially for those who undergo continuous ambulatory peritoneal dialysis at home, because of the lack of professional input-output volume monitoring and management during treatment. Objective: This study aims to develop novel mobile health (mHealth) tools to improve the quality of home-based continuous ambulatory peritoneal dialysis treatment and to build a prediction model of heart failure based on the system’s daily treatment monitoring data. Methods: The mHealth tools with a 4-layer system were designed and developed using Spring Boot, MyBatis Plus, MySQL, and Redis as backend technology stack, and Vue, Element User Interface, and WeChat Mini Program as front-end technology stack. Patients were recruited to use the tool during daily peritoneal dialysis treatment from January 1, 2017, to April 20, 2023. Logistic regression models based on real-time treatment monitoring data were used for heart failure prediction. The sensitivity, specificity, accuracy, and Youden index were calculated to evaluate the performance of the prediction model. In the sensitivity analysis, the ratio of patients with and without heart failure was set to 1:4 and 1:10, respectively, to better evaluate the stability of the prediction model. Results: A WeChat Mini Program named Futou Bao for patients and a patient data management platform for doctors was developed. Futou Bao included an intelligent data upload function module and an auxiliary function module. The doctor’s data management platform consisted of 4 function modules, that is, patient management, data visualization and marking, data statistics, and system management. During the study period, the records of 6635 patients who received peritoneal dialysis treatment were uploaded in Futou Bao, with 0.71% (47/6635) of them experiencing heart failure. The prediction model that included sex, age, and diastolic blood pressure was considered as the optimal model, wherein the sensitivity, specificity, accuracy, and Youden index were 0.75, 0.91, 0.89, and 0.66, respectively, with an area under the curve value of 0.879 (95% CI 0.772-0.986) using the validation dataset. The sensitivity analysis showed stable results. Conclusions: This study provides a new home-based peritoneal dialysis management paradigm that enables the daily monitoring and early warning of heart failure risk. This novel paradigm is of great value for improving the efficiency, security, and personalization of peritoneal dialysis. %M 40053710 %R 10.2196/56254 %U https://formative.jmir.org/2025/1/e56254 %U https://doi.org/10.2196/56254 %U http://www.ncbi.nlm.nih.gov/pubmed/40053710 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e55492 %T Complete Blood Count and Monocyte Distribution Width–Based Machine Learning Algorithms for Sepsis Detection: Multicentric Development and External Validation Study %A Campagner,Andrea %A Agnello,Luisa %A Carobene,Anna %A Padoan,Andrea %A Del Ben,Fabio %A Locatelli,Massimo %A Plebani,Mario %A Ognibene,Agostino %A Lorubbio,Maria %A De Vecchi,Elena %A Cortegiani,Andrea %A Piva,Elisa %A Poz,Donatella %A Curcio,Francesco %A Cabitza,Federico %A Ciaccio,Marcello %+ Department of Computer Science, Systems and Communication, University of Milano-Bicocca, Piazza dell'Ateneo Nuovo, 1, Milano, 20126, Italy, 39 0264487888, federico.cabitza@unimib.it %K sepsis %K medical machine learning %K external validation %K complete blood count %K controllable AI %K machine learning %K artificial intelligence %K development study %K validation study %K organ %K organ dysfunction %K detection %K clinical signs %K clinical symptoms %K biomarker %K diagnostic %K machine learning model %K sepsis detection %K early detection %K data distribution %D 2025 %7 26.2.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Sepsis is an organ dysfunction caused by a dysregulated host response to infection. Early detection is fundamental to improving the patient outcome. Laboratory medicine can play a crucial role by providing biomarkers whose alteration can be detected before the onset of clinical signs and symptoms. In particular, the relevance of monocyte distribution width (MDW) as a sepsis biomarker has emerged in the previous decade. However, despite encouraging results, MDW has poor sensitivity and positive predictive value when compared to other biomarkers. Objective: This study aims to investigate the use of machine learning (ML) to overcome the limitations mentioned earlier by combining different parameters and therefore improving sepsis detection. However, making ML models function in clinical practice may be problematic, as their performance may suffer when deployed in contexts other than the research environment. In fact, even widely used commercially available models have been demonstrated to generalize poorly in out-of-distribution scenarios. Methods: In this multicentric study, we developed ML models whose intended use is the early detection of sepsis on the basis of MDW and complete blood count parameters. In total, data from 6 patient cohorts (encompassing 5344 patients) collected at 5 different Italian hospitals were used to train and externally validate ML models. The models were trained on a patient cohort encompassing patients enrolled at the emergency department, and it was externally validated on 5 different cohorts encompassing patients enrolled at both the emergency department and the intensive care unit. The cohorts were selected to exhibit a variety of data distribution shifts compared to the training set, including label, covariate, and missing data shifts, enabling a conservative validation of the developed models. To improve generalizability and robustness to different types of distribution shifts, the developed ML models combine traditional methodologies with advanced techniques inspired by controllable artificial intelligence (AI), namely cautious classification, which gives the ML models the ability to abstain from making predictions, and explainable AI, which provides health operators with useful information about the models’ functioning. Results: The developed models achieved good performance on the internal validation (area under the receiver operating characteristic curve between 0.91 and 0.98), as well as consistent generalization performance across the external validation datasets (area under the receiver operating characteristic curve between 0.75 and 0.95), outperforming baseline biomarkers and state-of-the-art ML models for sepsis detection. Controllable AI techniques were further able to improve performance and were used to derive an interpretable set of diagnostic rules. Conclusions: Our findings demonstrate how controllable AI approaches based on complete blood count and MDW may be used for the early detection of sepsis while also demonstrating how the proposed methodology can be used to develop ML models that are more resistant to different types of data distribution shifts. %M 40009841 %R 10.2196/55492 %U https://www.jmir.org/2025/1/e55492 %U https://doi.org/10.2196/55492 %U http://www.ncbi.nlm.nih.gov/pubmed/40009841 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 11 %N %P e66218 %T Establishing Syndromic Surveillance of Acute Coronary Syndrome, Myocardial Infarction, and Stroke: Registry Study Based on Routine Data From German Emergency Departments %A Schranz,Madlen %A Rupprecht,Mirjam %A Aigner,Annette %A Benning,Leo %A Schlump,Carmen %A Charfeddine,Nesrine %A Diercke,Michaela %A Grabenhenrich,Linus %A Ullrich,Alexander %A Neuhauser,Hannelore %A Maier,Birga %A , %A Hans,Felix Patricius %A Blaschke,Sabine %K emergency medicine %K routinely collected health data %K public health surveillance %K syndromic surveillance %K acute coronary syndrome %K myocardial infarction %K stroke %K routine data %K Germany %K emergency department %K accuracy %K syndrome %K diagnosis %K public health %K health surveillance %D 2025 %7 25.2.2025 %9 %J JMIR Public Health Surveill %G English %X Background: Emergency department (ED) routine data offer a unique opportunity for syndromic surveillance of communicable and noncommunicable diseases (NCDs). In 2020, the Robert Koch Institute established a syndromic surveillance system using ED data from the AKTIN registry. The system provides daily insights into ED utilization for infectious diseases. Adding NCD indicators to the surveillance is of great public health importance, especially during acute events, where timely monitoring enables targeted public health responses and communication. Objective: This study aimed to develop and validate syndrome definitions for the NCD indicators of acute coronary syndrome (ACS), myocardial infarction (MI), and stroke (STR). Methods: First, syndrome definitions were developed with clinical experts combining ED diagnosis, chief complaints, diagnostic certainty, and discharge information. Then, using the multicenter retrospective routine ED data provided by the AKTIN registry, we conducted internal validation by linking ED cases fulfilling the syndrome definition with the hospital discharge diagnoses and calculating sensitivity, specificity, and accuracy. Lastly, external validation comprised the comparison of the ED cases fulfilling the syndrome definition with the federal German hospital diagnosis statistic. Ratios comparing the relative number of cases for all syndrome definitions were calculated and stratified by age and sex. Results: We analyzed data from 9 EDs, totaling 704,797 attendances from January 1, 2019, to March 5, 2021. Syndrome definitions were based on ICD-10 (International Statistical Classification of Diseases and Related Health Problems 10th Revision-German Modification) diagnoses, chief complaints, and discharge information. We identified 4.3% of all cases as ACS, 0.6% as MI, and 3.2% as STR. Patients with ACS and MI were more likely to be male (58.3% and 64.7%), compared to the overall attendances (52.7%). For all syndrome definitions, the prevalence was higher in the older age groups (60‐79 years and >80 years), and the highest proportions of cases were assigned an urgency level (3=urgent or 2=very urgent). The internal validation showed accuracy and specificity levels above 96% for all syndrome definitions. The sensitivity was 85.3% for ACS, 56.6% for MI, and 80.5% for STR. The external validation showed high levels of correspondence between the ED data and the German hospital statistics, with most ratios ranging around 1, indicating congruence, particularly in older age groups. The highest differences were noted in younger age groups, with the highest ratios in women aged between 20 and 39 years (4.57 for MI and 4.17 for ACS). Conclusions: We developed NCD indicators for ACS, MI, and STR that showed high levels of internal and external validity. The integration of these indicators into the syndromic surveillance system for EDs could enable daily monitoring of NCD patterns and trends to enhance timely public health surveillance in Germany. %R 10.2196/66218 %U https://publichealth.jmir.org/2025/1/e66218 %U https://doi.org/10.2196/66218 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 12 %N %P e63622 %T Multimodal Digital Phenotyping Study in Patients With Major Depressive Episodes and Healthy Controls (Mobile Monitoring of Mood): Observational Longitudinal Study %A Aledavood,Talayeh %A Luong,Nguyen %A Baryshnikov,Ilya %A Darst,Richard %A Heikkilä,Roope %A Holmén,Joel %A Ikäheimonen,Arsi %A Martikkala,Annasofia %A Riihimäki,Kirsi %A Saleva,Outi %A Triana,Ana Maria %A Isometsä,Erkki %+ , Department of Computer Science, Aalto University, Konemiehentie 2, Espoo, 02150, Finland, 358 505632634, talayeh.aledavood@aalto.fi %K digital health %K mental disorders %K depression %K digital phenotyping %K smartphones %K mobile devices %K multisensor %K mobile phone %D 2025 %7 21.2.2025 %9 Original Paper %J JMIR Ment Health %G English %X Background: Mood disorders are among the most common mental health conditions worldwide. Wearables and consumer-grade personal digital devices create digital traces that can be collected, processed, and analyzed, offering a unique opportunity to quantify and monitor individuals with mental disorders in their natural living environments. Objective: This study comprised (1) 3 subcohorts of patients with a major depressive episode, either with major depressive disorder, bipolar disorder, or concurrent borderline personality disorder, and (2) a healthy control group. We investigated whether differences in behavioral patterns could be observed at the group level, that is, patients versus healthy controls. We studied the volume and temporal patterns of smartphone screen and app use, communication, sleep, mobility, and physical activity. We investigated whether patients or controls exhibited more homogenous temporal patterns of activity when compared with other individuals in the same group. We examined which variables were associated with the severity of depression. Methods: In total, 188 participants were recruited to complete a 2-phase study. In the first 2 weeks, data from bed sensors, actigraphy, smartphones, and 5 sets of daily questions were collected. In the second phase, which lasted up to 1 year, only passive smartphone data and biweekly 9-item Patient Health Questionnaire data were collected. Survival analysis, statistical tests, and linear mixed models were performed. Results: Survival analysis showed no statistically significant difference in adherence. Most participants did not stay in the study for 1 year. Weekday location variance showed lower values for patients (control: mean –10.04, SD 2.73; patient: mean –11.91, SD 2.50; Mann-Whitney U [MWU] test P=.004). Normalized entropy of location was lower among patients (control: mean 2.10, SD 1.38; patient: mean 1.57, SD 1.10; MWU test P=.05). The temporal communication patterns of controls were more diverse compared to those of patients (MWU test P<.001). In contrast, patients exhibited more varied temporal patterns of smartphone use compared to the controls. We found that the duration of incoming calls (β=–0.08, 95% CI –0.12 to –0.04; P<.001) and the SD of activity magnitude (β=–2.05, 95% CI –4.18 to –0.20; P=.02) over the 14 days before the 9-item Patient Health Questionnaire records were negatively associated with depression severity. Conversely, the duration of outgoing calls showed a positive association with depression severity (β=0.05, 95% CI 0.00-0.09; P=.02). Conclusions: Our work shows the important features for future analyses of behavioral markers of mood disorders. However, among outpatients with mild to moderate depressive disorders, the group-level differences from healthy controls in any single modality remain relatively modest. Therefore, future studies need to combine data from multiple modalities to detect more subtle differences and identify individualized signatures. The high dropout rates for longer study periods remain a challenge and limit the generalizability. %M 39984168 %R 10.2196/63622 %U https://mental.jmir.org/2025/1/e63622 %U https://doi.org/10.2196/63622 %U http://www.ncbi.nlm.nih.gov/pubmed/39984168 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 9 %N %P e51939 %T Assessing the Feasibility and Utility of Patient-Specific 3D Advanced Visualization Modeling in Cerebrovascular Disease: Retrospective Analysis and Prospective Survey Pilot Study %A Sarkar,Korak %A Bhimarasetty,Vishal %A Rahim,Abdul %A Curtis,Colin %A Hughes,Kimberly %+ Department of Academics, Ochsner BioDesign Lab, Ochsner Health, 1514 Jefferson Highway, New Orleans, LA, 70119, United States, 1 504 894 2799, korak.sarkar.md@gmail.com %K cerebrovascular disease %K advanced visualization %K 3D modeling %K cerebrovascular %K intracerebral arteriovenous malformations %K artery %K vein %K vessel %K medical extended reality %K 3D printing %K medical simulation %K virtual reality %K augmented reality %K usability %K survey %K stroke %K brain %K cerebral %D 2025 %7 21.2.2025 %9 Original Paper %J JMIR Form Res %G English %X Background: The prevalence, clinical burden, and health care costs (>US $100 billion) associated with cerebrovascular disease (CVD) will increase significantly as the US population grows and ages over the next 25 years. Existing 2D imaging modalities have inherent limitations in visualizing complex CVD, which may be mitigated with the use of patient-specific 3D advanced visualization (AV) technologies. There remain gaps in knowledge, however, regarding how and with what impact these technologies are being used in CVD. Objective: The aim of this study was to characterize the clinical attributes and reported utility associated with the use of 3D AV modeling in CVDs, specifically intracerebral arteriovenous malformations. Methods: This pilot study employs a combination of retrospective analysis and prospective surveys to describe the utilization and utility of patient-specific AV models at a single high-volume certified comprehensive stroke center. Results: From July 2017 to February 2023, 25 AV models were created for 4 different clinicians. The average patient age was 37.4 years; 44% (11/25) of the patients were African Americans, 52% (13/25) were on public insurance, and 56% (14/25) were associated with a neurovascular procedure. In this study, 18 clinicians with diverse experience responded to AV model surveys, with a 92.2% (166/180) completion rate. There was an average reported utility of 8.0 on a 0-10 scale, with higher scores reflecting increased utility. Compared to 2D viewing, AV models allowed staff to appreciate novel abnormal anatomy, and therefore, they would have changed their therapeutic approach in 45% (23/51) of the cases. Conclusions: AV models were used in complex CVDs associated with young, publicly insured individuals requiring resource-intensive interventions. There was strong and diverse clinician engagement with overall report of substantial utility of AV models. Staff clinicians frequently reported novel anatomical and therapeutic insights based on AV models compared to traditional 2D viewing. This study establishes the infrastructure for future larger randomized studies that can be repeated for CVDs or other disease states and incorporate assessments of other AV modalities such as 3D printing and medical extended reality. %M 39983107 %R 10.2196/51939 %U https://formative.jmir.org/2025/1/e51939 %U https://doi.org/10.2196/51939 %U http://www.ncbi.nlm.nih.gov/pubmed/39983107 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e56306 %T Finding Consensus on Trust in AI in Health Care: Recommendations From a Panel of International Experts %A Starke,Georg %A Gille,Felix %A Termine,Alberto %A Aquino,Yves Saint James %A Chavarriaga,Ricardo %A Ferrario,Andrea %A Hastings,Janna %A Jongsma,Karin %A Kellmeyer,Philipp %A Kulynych,Bogdan %A Postan,Emily %A Racine,Elise %A Sahin,Derya %A Tomaszewska,Paulina %A Vold,Karina %A Webb,Jamie %A Facchini,Alessandro %A Ienca,Marcello %+ Institute for History and Ethics of Medicine, Technical University of Munich, Ismaninger Str. 22, Munich, 81675, Germany, 49 8941404041, georg.starke@tum.de %K expert consensus %K trust %K artificial intelligence %K clinical decision support %K assistive technologies %K public health surveillance %K framework analysis %D 2025 %7 19.2.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: The integration of artificial intelligence (AI) into health care has become a crucial element in the digital transformation of health systems worldwide. Despite the potential benefits across diverse medical domains, a significant barrier to the successful adoption of AI systems in health care applications remains the prevailing low user trust in these technologies. Crucially, this challenge is exacerbated by the lack of consensus among experts from different disciplines on the definition of trust in AI within the health care sector. Objective: We aimed to provide the first consensus-based analysis of trust in AI in health care based on an interdisciplinary panel of experts from different domains. Our findings can be used to address the problem of defining trust in AI in health care applications, fostering the discussion of concrete real-world health care scenarios in which humans interact with AI systems explicitly. Methods: We used a combination of framework analysis and a 3-step consensus process involving 18 international experts from the fields of computer science, medicine, philosophy of technology, ethics, and social sciences. Our process consisted of a synchronous phase during an expert workshop where we discussed the notion of trust in AI in health care applications, defined an initial framework of important elements of trust to guide our analysis, and agreed on 5 case studies. This was followed by a 2-step iterative, asynchronous process in which the authors further developed, discussed, and refined notions of trust with respect to these specific cases. Results: Our consensus process identified key contextual factors of trust, namely, an AI system’s environment, the actors involved, and framing factors, and analyzed causes and effects of trust in AI in health care. Our findings revealed that certain factors were applicable across all discussed cases yet also pointed to the need for a fine-grained, multidisciplinary analysis bridging human-centered and technology-centered approaches. While regulatory boundaries and technological design features are critical to successful AI implementation in health care, ultimately, communication and positive lived experiences with AI systems will be at the forefront of user trust. Our expert consensus allowed us to formulate concrete recommendations for future research on trust in AI in health care applications. Conclusions: This paper advocates for a more refined and nuanced conceptual understanding of trust in the context of AI in health care. By synthesizing insights into commonalities and differences among specific case studies, this paper establishes a foundational basis for future debates and discussions on trusting AI in health care. %M 39969962 %R 10.2196/56306 %U https://www.jmir.org/2025/1/e56306 %U https://doi.org/10.2196/56306 %U http://www.ncbi.nlm.nih.gov/pubmed/39969962 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 13 %N %P e64204 %T A Risk Warning Model for Anemia Based on Facial Visible Light Reflectance Spectroscopy: Cross-Sectional Study %A Zhang,Yahan %A Chun,Yi %A Fu,Hongyuan %A Jiao,Wen %A Bao,Jizhang %A Jiang,Tao %A Cui,Longtao %A Hu,Xiaojuan %A Cui,Ji %A Qiu,Xipeng %A Tu,Liping %A Xu,Jiatuo %K anemia %K hemoglobin %K spectroscopy %K machine learning %K risk warning model %K Shapley additive explanation %D 2025 %7 14.2.2025 %9 %J JMIR Med Inform %G English %X Background: Anemia is a global public health issue causing symptoms such as fatigue, weakness, and cognitive decline. Furthermore, anemia is associated with various diseases and increases the risk of postoperative complications and mortality. Frequent invasive blood tests for diagnosis also pose additional discomfort and risks to patients. Objective: This study aims to assess the facial spectral characteristics of patients with anemia and to develop a predictive model for anemia risk using machine learning approaches. Methods: Between August 2022 and September 2023, we collected facial image data from 78 anemic patients who met the inclusion criteria from the Hematology Department of Shanghai Hospital of Traditional Chinese Medicine. Between March 2023 and September 2023, we collected data from 78 healthy adult participants from Shanghai Jiading Community Health Center and Shanghai Gaohang Community Health Center. A comprehensive statistical analysis was performed to evaluate differences in spectral characteristics between the anemic patients and healthy controls. Then, we used 10 different machine learning algorithms to create a predictive model for anemia. The least absolute shrinkage and selection operator was used to analyze the predictors. We integrated multiple machine learning classification models to identify the optimal model and developed Shapley additive explanations (SHAP) for personalized risk assessment. Results: The study identified significant differences in facial spectral features between anemic patients and healthy controls. The support vector machine classifier outperformed other classification models, achieving an accuracy of 0.875 (95% CI 0.825-0.925) for distinguishing between the anemia and healthy control groups. In the SHAP interpretation of the model, forehead-570 nm, right cheek-520 nm, right zygomatic-570 nm, jaw-570 nm, and left cheek-610 nm were the features with the highest contributions. Conclusions: Facial spectral data demonstrated clinical significance in anemia diagnosis, and the early warning model for anemia risk constructed based on spectral information demonstrated a high accuracy rate. %R 10.2196/64204 %U https://medinform.jmir.org/2025/1/e64204 %U https://doi.org/10.2196/64204 %0 Journal Article %@ 2561-7605 %I JMIR Publications %V 8 %N %P e60107 %T Best Practices for Implementing Electronic Care Records in Adult Social Care: Rapid Scoping Review %A Snow,Martha %A Silva-Ribeiro,Wagner %A Baginsky,Mary %A Di Giorgio,Sonya %A Farrelly,Nicola %A Larkins,Cath %A Poole,Karen %A Steils,Nicole %A Westwood,Joanne %A Malley,Juliette %+ Care Policy and Evaluation Centre, London School of Economics and Political Science (LSE), PAN 8.01J, Houghton Street, London, WC2A 2AE, United Kingdom, 44 (0)20 7405 7686, m.snow@lse.ac.uk %K digital care records %K adult social care %K digitization %K domiciliary care %K care homes %K electronic care records %K PRISMA %D 2025 %7 14.2.2025 %9 Review %J JMIR Aging %G English %X Background: In the past decade, the use of digital or electronic records in social care has risen worldwide, capturing key information for service delivery. The COVID-19 pandemic accelerated digitization in health and social care. For example, the UK government created a fund specifically for adult social care provider organizations to adopt digital social care records. These developments offer valuable learning opportunities for implementing digital care records in adult social care settings. Objective: This rapid scoping review aimed to understand what is known about the implementation of digital care records in adult social care and how implementation varies across use cases, settings, and broader contexts. Methods: A scoping review methodology was used, with amendments made to enable a rapid review. Comprehensive searches based on the concepts of digital care records, social care, and interoperability were conducted across the MEDLINE, EmCare, Web of Science Core Collection, HMIC Health Management Information Consortium, Social Policy and Practice, and Social Services Abstracts databases. Studies published between 2018 and 2023 in English were included. One reviewer screened titles and abstracts, while 2 reviewers extracted data. Thematic analysis mapped findings against the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework. Results: Our search identified 2499 references. After screening titles and abstracts, 71 records were selected for full-text review, resulting in 31 references from 29 studies. Studies originated from 11 countries, including 1 multicountry study, with the United Kingdom being the most represented (10/29, 34%). Studies were most often conducted in nursing homes or facilities (7/29, 24%) with older people as the target population (6/29, 21%). Health records were the most investigated record type (12/29, 41%). We identified 45 facilitators and 102 barriers to digital care record implementation across 28 studies, spanning 6 of the 7 NASSS framework domains and aligning with 5 overarching themes that require greater active management regarding implementation. Intended or actual implementation outcomes were reported in 17 (59%) of the 29 studies. Conclusions: The findings suggest that implementation is complex due to a lack of consensus on what digital care records and expected outcomes and impacts should look like. The literature often lacks clear definitions and robust study designs. To be successful, implementation should consider complexity, while studies should use robust frameworks and mixed methods or quantitative designs where appropriate. Future research should define the target population, gather data on carer or service user experiences, and focus on digital care records specifically used in social care. %M 39951702 %R 10.2196/60107 %U https://aging.jmir.org/2025/1/e60107 %U https://doi.org/10.2196/60107 %U http://www.ncbi.nlm.nih.gov/pubmed/39951702 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 9 %N %P e68436 %T Machine Learning Approach to Identifying Wrong-Site Surgeries Using Centers for Medicare and Medicaid Services Dataset: Development and Validation Study %A Chen,Yuan-Hsin %A Lin,Ching-Hsuan %A Fan,Chiao-Hsin %A Long,An Jim %A Scholl,Jeremiah %A Kao,Yen-Pin %A Iqbal,Usman %A Li,Yu-Chuan Jack %+ Graduate Institute of Biomedical Informatics, College of Medical Science and Technology, Taipei Medical University, No. 301, Yuantong Rd. Zhonghe Dist, New Taipei City, 235, Taiwan, 886 266382736, jack@tmu.edu.tw %K patient safety %K wrong site surgery %K medical errors %K machine learning %K claim data %D 2025 %7 13.2.2025 %9 Original Paper %J JMIR Form Res %G English %X Background: Wrong-site surgery (WSS) is a critical but preventable medical error, often resulting in severe patient harm and substantial financial costs. While protocols exist to reduce wrong-site surgery, underreporting and inconsistent documentation continue to contribute to its persistence. Machine learning (ML) models, which have shown success in detecting medication errors, may offer a solution by identifying unusual procedure-diagnosis combinations. This study investigated whether an ML approach can effectively adapt to detect surgical errors. Objective: This study aimed to evaluate the transferability and effectiveness of an ML-based model for detecting inconsistencies within surgical documentation, particularly focusing on laterality discrepancies. Methods: We used claims data from the Centers for Medicare and Medicaid Services Limited Data Set (CMS-LDS) from 2017 to 2020, focusing on surgical procedures with documented laterality. We developed an adapted Association Outlier Pattern (AOP) ML model to identify uncommon procedure-diagnosis combinations, specifically targeting discrepancies in laterality. The model was trained on data from 2017 to 2019 and tested on 2020 orthopedic procedures, using ICD-10-PCS (International Classification of Diseases, Tenth Revision, Procedure Coding System) codes to distinguish body part and laterality. Test cases were classified based on alignment between procedural and diagnostic laterality, with 2 key subgroups (right-left and left-right mismatches) identified for evaluation. Model performance was assessed by comparing precision-recall curves and accuracy against rule-based methods. Results: The findings here included 346,382 claims, of which 2170 claims demonstrated with significant laterality discrepancies between procedures and diagnoses. Among patients with left-side procedures and right-side diagnoses (603/1106), 54.5% were confirmed as errors after clinical review. For right-side procedures with left-side diagnoses (541/1064), 50.8% were classified as errors. The AOP model identified 697 and 655 potentially unusual combinations in the left-right and right-left subgroups, respectively, with over 80% of these cases confirmed as errors following clinical review. Most confirmed errors involved discrepancies in laterality for the same body part, while nonerror cases typically involved general diagnoses without specified laterality. Conclusions: This investigation showed that the AOP model effectively detects inconsistencies between surgical procedures and diagnoses using CMS-LDS data. The AOP model outperformed traditional rule-based methods, offering higher accuracy in identifying errors. Moreover, the model’s transferability from medication-disease associations to procedure-diagnosis verification highlights its broad applicability. By improving the precision of identifying laterality discrepancies, the AOP model can reduce surgical errors, particularly in orthopedic care. These findings suggest that the model enhances patient safety and has the potential to improve clinical decision-making and outcomes. %M 39946709 %R 10.2196/68436 %U https://formative.jmir.org/2025/1/e68436 %U https://doi.org/10.2196/68436 %U http://www.ncbi.nlm.nih.gov/pubmed/39946709 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e66910 %T Using Structured Codes and Free-Text Notes to Measure Information Complementarity in Electronic Health Records: Feasibility and Validation Study %A Seinen,Tom M %A Kors,Jan A %A van Mulligen,Erik M %A Rijnbeek,Peter R %+ Department of Medical Informatics, Erasmus University Medical Center, Dr. Molewaterplein 40, Rotterdam, 3015 GD, Netherlands, 31 010 7044122, t.seinen@erasmusmc.nl %K natural language processing %K named entity recognition %K clinical concept extraction %K machine learning %K electronic health records %K EHR %K word embeddings %K clinical concept similarity %K text mining %K code %K free-text %K information %K electronic record %K data %K patient records %K framework %K structured data %K unstructured data %D 2025 %7 13.2.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Electronic health records (EHRs) consist of both structured data (eg, diagnostic codes) and unstructured data (eg, clinical notes). It is commonly believed that unstructured clinical narratives provide more comprehensive information. However, this assumption lacks large-scale validation and direct validation methods. Objective: This study aims to quantitatively compare the information in structured and unstructured EHR data and directly validate whether unstructured data offers more extensive information across a patient population. Methods: We analyzed both structured and unstructured data from patient records and visits in a large Dutch primary care EHR database between January 2021 and January 2024. Clinical concepts were identified from free-text notes using an extraction framework tailored for Dutch and compared with concepts from structured data. Concept embeddings were generated to measure semantic similarity between structured and extracted concepts through cosine similarity. A similarity threshold was systematically determined via annotated matches and minimized weighted Gini impurity. We then quantified the concept overlap between structured and unstructured data across various concept domains and patient populations. Results: In a population of 1.8 million patients, only 13% of extracted concepts from patient records and 7% from individual visits had similar structured counterparts. Conversely, 42% of structured concepts in records and 25% in visits had similar matches in unstructured data. Condition concepts had the highest overlap, followed by measurements and drug concepts. Subpopulation visits, such as those with chronic conditions or psychological disorders, showed different proportions of data overlap, indicating varied reliance on structured versus unstructured data across clinical contexts. Conclusions: Our study demonstrates the feasibility of quantifying the information difference between structured and unstructured data, showing that the unstructured data provides important additional information in the studied database and populations. The annotated concept matches are made publicly available for the clinical natural language processing community. Despite some limitations, our proposed methodology proves versatile, and its application can lead to more robust and insightful observational clinical research. %M 39946687 %R 10.2196/66910 %U https://www.jmir.org/2025/1/e66910 %U https://doi.org/10.2196/66910 %U http://www.ncbi.nlm.nih.gov/pubmed/39946687 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e59841 %T Trends and Gaps in Digital Precision Hypertension Management: Scoping Review %A Clifford,Namuun %A Tunis,Rachel %A Ariyo,Adetimilehin %A Yu,Haoxiang %A Rhee,Hyekyun %A Radhakrishnan,Kavita %+ , School of Nursing, The University of Texas at Austin, 1710 Red River St, Austin, TX, 78701, United States, 1 (512) 471 7913, namuun.clifford@utexas.edu %K precision health %K hypertension %K digital health %K prediction models %K personalization %K phenotyping %K machine learning %K algorithms %K mobile apps %K mobile health %D 2025 %7 10.2.2025 %9 Review %J J Med Internet Res %G English %X Background: Hypertension (HTN) is the leading cause of cardiovascular disease morbidity and mortality worldwide. Despite effective treatments, most people with HTN do not have their blood pressure under control. Precision health strategies emphasizing predictive, preventive, and personalized care through digital tools offer notable opportunities to optimize the management of HTN. Objective: This scoping review aimed to fill a research gap in understanding the current state of precision health research using digital tools for the management of HTN in adults. Methods: This study used a scoping review framework to systematically search for articles in 5 databases published between 2013 and 2023. The included articles were thematically analyzed based on their precision health focus: personalized interventions, prediction models, and phenotyping. Data were extracted and summarized for study and sample characteristics, precision health focus, digital health technology, disciplines involved, and characteristics of personalized interventions. Results: After screening 883 articles, 46 were included; most studies had a precision health focus on personalized digital interventions (34/46, 74%), followed by prediction models (8/46, 17%) and phenotyping (4/46, 9%). Most studies (38/46, 82%) were conducted in or used data from North America or Europe, and 63% (29/46) of the studies came exclusively from the medical and health sciences, with 33% (15/46) of studies involving 2 or more disciplines. The most commonly used digital technologies were mobile phones (33/46, 72%), blood pressure monitors (18/46, 39%), and machine learning algorithms (11/46, 24%). In total, 45% (21/46) of the studies either did not report race or ethnicity data (14/46, 30%) or partially reported this information (7/46, 15%). For personalized intervention studies, nearly half (14/30, 47%) used 2 or less types of data for personalization, with only 7% (2/30) of the studies using social determinants of health data and no studies using physical environment or digital literacy data. Personalization characteristics of studies varied, with 43% (13/30) of studies using fully automated personalization approaches, 33% (10/30) using human-driven personalization, and 23% (7/30) using a hybrid approach. Conclusions: This scoping review provides a comprehensive mapping of the literature on the current trends and gaps in digital precision health research for the management of HTN in adults. Personalized digital interventions were the primary focus of most studies; however, the review highlighted the need for more precise definitions of personalization and the integration of more diverse data sources to improve the tailoring of interventions and promotion of health equity. In addition, there were significant gaps in the reporting of race and ethnicity data of participants, underuse of wearable devices for passive data collection, and the need for greater interdisciplinary collaboration to advance precision health research in digital HTN management. Trial Registration: OSF Registries osf.io/yuzf8; https://osf.io/yuzf8 %M 39928934 %R 10.2196/59841 %U https://www.jmir.org/2025/1/e59841 %U https://doi.org/10.2196/59841 %U http://www.ncbi.nlm.nih.gov/pubmed/39928934 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e60888 %T Machine Learning in the Management of Patients Undergoing Catheter Ablation for Atrial Fibrillation: Scoping Review %A Luo,Aijing %A Chen,Wei %A Zhu,Hongtao %A Xie,Wenzhao %A Chen,Xi %A Liu,Zhenjiang %A Xin,Zirui %+ The Second Xiangya Hospital, Central South University, No. 139 Renmin Middle Road, Changsha, 410011, China, 86 15211017166, xinzirui@csu.edu.cn %K atrial fibrillation %K catheter ablation %K deep learning %K patient management %K prognosis %K quality assessment tools %K cardiac arrhythmia %K public health %K quality of life %K severe medical condition %K electrocardiogram %K electronic health record %K morbidity %K mortality %K thromboembolism %K clinical intervention %D 2025 %7 10.2.2025 %9 Review %J J Med Internet Res %G English %X Background: Although catheter ablation (CA) is currently the most effective clinical treatment for atrial fibrillation, its variable therapeutic effects among different patients present numerous problems. Machine learning (ML) shows promising potential in optimizing the management and clinical outcomes of patients undergoing atrial fibrillation CA (AFCA). Objective: This scoping review aimed to evaluate the current scientific evidence on the application of ML for managing patients undergoing AFCA, compare the performance of various models across specific clinical tasks within AFCA, and summarize the strengths and limitations of ML in this field. Methods: Adhering to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines, relevant studies published up to October 7, 2023, were searched from PubMed, Web of Science, Embase, the Cochrane Library, and ScienceDirect. The final included studies were confirmed based on inclusion and exclusion criteria and manual review. The PROBAST (Prediction model Risk Of Bias Assessment Tool) and QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies-2) methodological quality assessment tools were used to review the included studies, and narrative data synthesis was performed on the modeled results provided by these studies. Results: The analysis of 23 included studies showcased the contributions of ML in identifying potential ablation targets, improving ablation strategies, and predicting patient prognosis. The patient data used in these studies comprised demographics, clinical characteristics, various types of imaging (9/23, 39%), and electrophysiological signals (7/23, 30%). In terms of model type, deep learning, represented by convolutional neural networks, was most frequently applied (14/23, 61%). Compared with traditional clinical scoring models or human clinicians, the model performance reported in the included studies was generally satisfactory, but most models (14/23, 61%) showed a high risk of bias due to lack of external validation. Conclusions: Our evidence-based findings suggest that ML is a promising tool for improving the effectiveness and efficiency of managing patients undergoing AFCA. While guiding data preparation and model selection for future studies, this review highlights the need to address prevalent limitations, including lack of external validation, and to further explore model generalization and interpretability. %M 39928932 %R 10.2196/60888 %U https://www.jmir.org/2025/1/e60888 %U https://doi.org/10.2196/60888 %U http://www.ncbi.nlm.nih.gov/pubmed/39928932 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 14 %N %P e60559 %T A Digital Behavior Change Intervention for Health Promotion for Adults in Midlife: Protocol for a Multidimensional Assessment Study %A Soleymani,Dagmar %A Pougheon-Bertrand,Dominique %A Gagnayre,Rémi %+ , Health Promotion and Prevention Division, Santé publique France, 12 rue due Val d'Osne, Saint-Maurice, 94415, France, 33 0171482134, dagmar.soleymani@santepubliquefrance.fr %K digital behavior change intervention %K assessment protocol %K middle-aged adults %K health promotion %K user account %K mixed assessments %K health information technologies %D 2025 %7 7.2.2025 %9 Protocol %J JMIR Res Protoc %G English %X Background: To support lifelong health promotion and disease prevention, Santé publique France studied the methodology for building a social marketing scheme with a digital intervention targeting middle-aged adults, specifically socioeconomically disadvantaged groups. The digital intervention aims to encourage people aged 40-55 years to look after their health in the short and medium terms by adopting small actions relating to 8 health determinants: nutrition, physical activity, smoking, alcohol, stress, cognitive health, sleep, and environmental health. In the long term, the intervention intends to prevent frailty and reduce the burden of multimorbidities in older age, particularly for lower socioeconomic groups. Objective: This study aims to measure behavior changes among registered users of the future website. The protocol assesses the impact of the website based on users’ implementation of small actions relating to the 8 health determinants. Specifically, it intends to evaluate the website’s performance in terms of engaging a specific population, triggering behavior change, raising awareness about a multifactorial approach to health, and encouraging user interaction with the website’s resources. Methods: The methodology is based on clinical assessments developed alongside the website according to the functionalities offered to registered users in their personalized space. The assessment tool design draws on logic models for digital interventions, and their consistency for digital applications is verified. The target audience is clearly defined from the outset. The protocol sets out a 3-step assessment: upon registration, after 3 weeks of use, and after 10 weeks of use (end of assessment). Users are divided into 2 groups (socioeconomically disadvantaged users and others) to characterize differences and make corrections. The protocol uses a mixed assessment approach based on website traffic and user login data. Specific and identifiable behavior changes are documented by monitoring the same individuals from T0 to T2, using verbatim comments to classify them into profiles and conducting semistructured individual interviews with a sample of users. Results: The protocol creates a multidimensional assessment of digital intervention, showing that during a given timeline, interactions with users can reveal their capabilities, opportunities, and motivations to adopt healthy lifestyles. The protocol’s principles were integrated into the development of a personal account to assess users’ behavior changes. Given the delayed launch of the website, no recruitment or effects analysis of the protocol took place. Conclusions: As no multidimensional assessment protocol is currently available for digital behavior change interventions, our methods reveal that the different framework stages can strengthen the effect measurement, consolidate the choice of assumptions used within the logic model and steer the digital intervention toward action while reducing the burden of information. The suitability of the assessment protocol remains to be evaluated given the delayed launch of the website. International Registered Report Identifier (IRRID): PRR1-10.2196/60559 %M 39919300 %R 10.2196/60559 %U https://www.researchprotocols.org/2025/1/e60559 %U https://doi.org/10.2196/60559 %U http://www.ncbi.nlm.nih.gov/pubmed/39919300 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 13 %N %P e55825 %T Diagnosis of Chronic Kidney Disease Using Retinal Imaging and Urine Dipstick Data: Multimodal Deep Learning Approach %A Bhak,Youngmin %A Lee,Yu Ho %A Kim,Joonhyung %A Lee,Kiwon %A Lee,Daehwan %A Jang,Eun Chan %A Jang,Eunjeong %A Lee,Christopher Seungkyu %A Kang,Eun Seok %A Park,Sehee %A Han,Hyun Wook %A Nam,Sang Min %K multimodal deep learning %K chronic kidney disease %K fundus image %K saliency map %K urine dipstick %D 2025 %7 7.2.2025 %9 %J JMIR Med Inform %G English %X Background: Chronic kidney disease (CKD) is a prevalent condition with significant global health implications. Early detection and management are critical to prevent disease progression and complications. Deep learning (DL) models using retinal images have emerged as potential noninvasive screening tools for CKD, though their performance may be limited, especially in identifying individuals with proteinuria and in specific subgroups. Objective: We aim to evaluate the efficacy of integrating retinal images and urine dipstick data into DL models for enhanced CKD diagnosis. Methods: The 3 models were developed and validated: eGFR-RIDL (estimated glomerular filtration rate–retinal image deep learning), eGFR-UDLR (logistic regression using urine dipstick data), and eGFR-MMDL (multimodal deep learning combining retinal images and urine dipstick data). All models were trained to predict an eGFR<60 mL/min/1.73 m², a key indicator of CKD, calculated using the 2009 CKD-EPI (Chronic Kidney Disease Epidemiology Collaboration) equation. This study used a multicenter dataset of participants aged 20‐79 years, including a development set (65,082 people) and an external validation set (58,284 people). Wide Residual Networks were used for DL, and saliency maps were used to visualize model attention. Sensitivity analyses assessed the impact of numerical variables. Results: eGFR-MMDL outperformed eGFR-RIDL in both the test and external validation sets, with area under the curves of 0.94 versus 0.90 and 0.88 versus 0.77 (P<.001 for both, DeLong test). eGFR-UDLR outperformed eGFR-RIDL and was comparable to eGFR-MMDL, particularly in the external validation. However, in the subgroup analysis, eGFR-MMDL showed improvement across all subgroups, while eGFR-UDLR demonstrated no such gains. This suggested that the enhanced performance of eGFR-MMDL was not due to urine data alone, but rather from the synergistic integration of both retinal images and urine data. The eGFR-MMDL model demonstrated the best performance in individuals younger than 65 years or those with proteinuria. Age and proteinuria were identified as critical factors influencing model performance. Saliency maps indicated that urine data and retinal images provide complementary information, with urine offering insights into retinal abnormalities and retinal images, particularly the arcade vessels, being key for predicting kidney function. Conclusions: The MMDL model integrating retinal images and urine dipstick data show significant promise for noninvasive CKD screening, outperforming the retinal image–only model. However, routine blood tests are still recommended for individuals aged 65 years and older due to the model’s limited performance in this age group. %R 10.2196/55825 %U https://medinform.jmir.org/2025/1/e55825 %U https://doi.org/10.2196/55825 %0 Journal Article %@ 2561-7605 %I JMIR Publications %V 8 %N %P e55929 %T Revisits, Readmission, and Mortality From Emergency Department Admissions for Older Adults With Vague Presentations: Longitudinal Observational Study %A Alvarez Avendano,Sebastian Alejandro %A Cochran,Amy %A Odeh Couvertier,Valerie %A Patterson,Brian %A Shah,Manish %A Zayas-Caban,Gabriel %K gerontology %K geriatric %K older adults %K elderly %K older people %K aging %K emergency department %K emergency room %K ED %K disposition decision %K disposition %K discharge %K admission %K revisit %K readmission %K observational study %K health %K hospital %D 2025 %7 6.2.2025 %9 %J JMIR Aging %G English %X Background: Older adults (65 years and older) often present to the emergency department (ED) with an unclear need for hospitalization, leading to potentially harmful and costly care. This underscores the importance of measuring the trade-off between admission and discharge for these patients in terms of patient outcomes. Objective: This study aimed to measure the relationship between disposition decisions and 3-day, 9-day, and 30-day revisits, readmission, and mortality, using causal inference methods that adjust for potential measured and unmeasured confounding. Methods: A longitudinal observational study (n=3591) was conducted using electronic health records from a large tertiary teaching hospital with an ED between January 1, 2014 and September 27, 2018. The sample consisted of older adult patients with 1 of 6 presentations with significant variability in admission: falls, weakness, syncope, urinary tract infection, pneumonia, and cellulitis. The exposure under consideration was the ED disposition decision (admission to the hospital or discharge). Nine outcome variables were considered: ED revisits, hospital readmission, and mortality within 3, 9, and 30 days of being discharged from either the hospital for admitted patients or the ED for discharged patients. Results: Admission was estimated to significantly decrease the risk of an ED revisit after discharge (30-day window: −6.4%, 95% CI −7.8 to −5.0), while significantly increasing the risk of hospital readmission (30-day window: 5.8%, 95% CI 5.0 to 6.5) and mortality (30-day window: 1.0%, 95% CI 0.4 to 1.6). Admission was found to be especially adverse for patients with weakness and pneumonia, and relatively less adverse for older adult patients with falls and syncope. Conclusions: Admission may not be the safe option for older adults with gray area presentations, and while revisits and readmissions are commonly used to evaluate the quality of care in the ED, their divergence suggests that caution should be used when interpreting either in isolation. %R 10.2196/55929 %U https://aging.jmir.org/2025/1/e55929 %U https://doi.org/10.2196/55929 %0 Journal Article %@ 2561-7605 %I JMIR Publications %V 8 %N %P e68267 %T Predictive Validity of Hospital-Associated Complications of Older People Identified Using Diagnosis Procedure Combination Data From an Acute Care Hospital in Japan: Observational Study %A Mitsutake,Seigo %A Ishizaki,Tatsuro %A Yano,Shohei %A Hirata,Takumi %A Ito,Kae %A Furuta,Ko %A Shimazaki,Yoshitomo %A Ito,Hideki %A Mudge,Alison %A Toba,Kenji %+ Human Care Research Team, Tokyo Metropolitan Institute for Geriatrics and Gerontology, 35-2 Sakae-cho, Itabashi-ku, Tokyo, 173 0015, Japan, 81 3 3964 3241 ext 4229, mitsu@tmig.or.jp %K delirium %K functional decline %K Japan %K older adult %K routinely collected health data %K elder %K hospital complication %K HAC-OP %K incontinence %K pressure injury %K inpatient care %K diagnosis procedure combination %K predictive validity %K hospital length of stay %K administrative data %K acute care %K index hospitalization %K diagnostic code %K linear regression %K logistic regression %K long-term care %K retrospective cohort %K observational study %K patient care %K gerontology %K hospital care %K patient complication %D 2025 %7 6.2.2025 %9 Original Paper %J JMIR Aging %G English %X Background: A composite outcome of hospital-associated complications of older people (HAC-OP; comprising functional decline, delirium, incontinence, falls, and pressure injuries) has been proposed as an outcome measure reflecting quality of acute hospital care. Estimating HAC-OP from routinely collected administrative data could facilitate the rapid and standardized evaluation of interventions in the clinical setting, thereby supporting the development, improvement, and wider implementation of effective interventions. Objective: This study aimed to create a Diagnosis Procedure Combination (DPC) data version of the HAC-OP measure (HAC-OP-DPC) and demonstrate its predictive validity by assessing its associations with hospital length of stay (LOS) and discharge destination. Methods: This retrospective cohort study acquired DPC data (routinely collected administrative data) from a general acute care hospital in Tokyo, Japan. We included data from index hospitalizations for patients aged ≥65 years hospitalized for ≥3 days and discharged between July 2016 and March 2021. HAC-OP-DPC were identified using diagnostic codes for functional decline, incontinence, delirium, pressure injury, and falls occurring during the index hospitalization. Generalized linear regression models were used to examine the associations between HAC-OP-DPC and LOS, and logistic regression models were used to examine the associations between HAC-OP-DPC and discharge to other hospitals and long-term care facilities (LTCFs). Results: Among 15,278 patients, 3610 (23.6%) patients had coding evidence of one or more HAC-OP-DPC (1: 18.8% and ≥2: 4.8%). Using “no HAC-OP-DPC” as the reference category, the analysis showed a significant and graded association with longer LOS (adjusted risk ratio for patients with one HAC-OP-DPC 1.29, 95% CI 1.25-1.33; adjusted risk ratio for ≥2 HAC-OP-DPC 1.97, 95% CI 1.87-2.08), discharge to another hospital (adjusted odds ratio [AOR] for one HAC-OP-DPC 2.36, 95% CI 2.10-2.65; AOR for ≥2 HAC-OP-DPC 6.96, 95% CI 5.81-8.35), and discharge to LTCFs (AOR for one HAC-OP-DPC 1.35, 95% CI 1.09-1.67; AOR for ≥2 HAC-OP-DPC 1.68, 95% CI 1.18-2.39). Each individual HAC-OP was also significantly associated with longer LOS and discharge to another hospital, but only delirium was associated with discharge to LTCF. Conclusions: This study demonstrated the predictive validity of the HAC-OP-DPC measure for longer LOS and discharge to other hospitals and LTCFs. To attain a more robust understanding of these relationships, additional studies are needed to verify our findings in other hospitals and regions. The clinical implementation of HAC-OP-DPC, which is identified using routinely collected administrative data, could support the evaluation of integrated interventions aimed at optimizing inpatient care for older adults. %M 39913911 %R 10.2196/68267 %U https://aging.jmir.org/2025/1/e68267 %U https://doi.org/10.2196/68267 %U http://www.ncbi.nlm.nih.gov/pubmed/39913911 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e58779 %T An Easy and Quick Risk-Stratified Early Forewarning Model for Septic Shock in the Intensive Care Unit: Development, Validation, and Interpretation Study %A Liu,Guanghao %A Zheng,Shixiang %A He,Jun %A Zhang,Zi-Mei %A Wu,Ruoqiong %A Yu,Yingying %A Fu,Hao %A Han,Li %A Zhu,Haibo %A Xu,Yichang %A Shao,Huaguo %A Yan,Haidan %A Chen,Ting %A Shen,Xiaopei %+ Department of Bioinformatics, School of Medical Technology and Engineering, Fujian Medical University, 1 Xue Yuan Road, University Town, Fuzhou, 350122, China, 86 17366882017, xshen@fjmu.edu.cn %K septic shock %K early forewarning %K risk stratification %K machine learning %D 2025 %7 6.2.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Septic shock (SS) is a syndrome with high mortality. Early forewarning and diagnosis of SS, which are critical in reducing mortality, are still challenging in clinical management. Objective: We propose a simple and fast risk-stratified forewarning model for SS to help physicians recognize patients in time. Moreover, further insights can be gained from the application of the model to improve our understanding of SS. Methods: A total of 5125 patients with sepsis from the Medical Information Mart for Intensive Care-IV (MIMIC-IV) database were divided into training, validation, and test sets. In addition, 2180 patients with sepsis from the eICU Collaborative Research Database (eICU) were used as an external validation set. We developed a simplified risk-stratified early forewarning model for SS based on the weight of evidence and logistic regression, which was compared with multi-feature complex models, and clinical characteristics among risk groups were evaluated. Results: Using only vital signs and rapid arterial blood gas test features according to feature importance, we constructed the Septic Shock Risk Predictor (SORP), with an area under the curve (AUC) of 0.9458 in the test set, which is only slightly lower than that of the optimal multi-feature complex model (0.9651). A median forewarning time of 13 hours was calculated for SS patients. 4 distinct risk groups (high, medium, low, and ultralow) were identified by the SORP 6 hours before onset, and the incidence rates of SS in the 4 risk groups in the postonset interval were 88.6% (433/489), 34.5% (262/760), 2.5% (67/2707), and 0.3% (4/1301), respectively. The severity increased significantly with increasing risk in both clinical features and survival. Clustering analysis demonstrated a high similarity of pathophysiological characteristics between the high-risk patients without SS diagnosis (NS_HR) and the SS patients, while a significantly worse overall survival was shown in NS_HR patients. On further exploring the characteristics of the treatment and comorbidities of the NS_HR group, these patients demonstrated a significantly higher incidence of mean blood pressure <65 mmHg, significantly lower vasopressor use and infused volume, and more severe renal dysfunction. The above findings were further validated by multicenter eICU data. Conclusions: The SORP demonstrated accurate forewarning and a reliable risk stratification capability. Among patients forewarned as high risk, similar pathophysiological phenotypes and high mortality were observed in both those subsequently diagnosed as having SS and those without such a diagnosis. NS_HR patients, overlooked by the Sepsis-3 definition, may provide further insights into the pathophysiological processes of SS onset and help to complement its diagnosis and precise management. The importance of precise fluid resuscitation management in SS patients with renal dysfunction is further highlighted. For convenience, an online service for the SORP has been provided. %M 39913913 %R 10.2196/58779 %U https://www.jmir.org/2025/1/e58779 %U https://doi.org/10.2196/58779 %U http://www.ncbi.nlm.nih.gov/pubmed/39913913 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e62836 %T Mortality Risk Prediction in Patients With Antimelanoma Differentiation–Associated, Gene 5 Antibody–Positive, Dermatomyositis–Associated Interstitial Lung Disease: Algorithm Development and Validation %A Li,Hui %A Zou,Ruyi %A Xin,Hongxia %A He,Ping %A Xi,Bin %A Tian,Yaqiong %A Zhao,Qi %A Yan,Xin %A Qiu,Xiaohua %A Gao,Yujuan %A Liu,Yin %A Cao,Min %A Chen,Bi %A Han,Qian %A Chen,Juan %A Wang,Guochun %A Cai,Hourong %+ Department of Respiratory and Critical Care Medicine, Nanjing Drum Tower Hospital, Affiliated Hospital of Medical School, Nanjing University, 321 Zhongshan Rd., Gulou District, Nanjing, 210008, China, 86 025 83106666, caihourong2013@163.com %K antimelanoma differentiation–associated gene 5 antibody %K dermatomyositis %K interstitial lung disease %K 3-month mortality %K machine learning %K ML %K tool %K web based %K mortality %K idiopathic inflammatory myopathy %K myopathy %K lung disease %K melanoma %K imaging %K clinical outcome %D 2025 %7 5.2.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Patients with antimelanoma differentiation–associated gene 5 antibody–positive dermatomyositis–associated interstitial lung disease (anti-MDA5+DM-ILD) are susceptible to rapidly progressive interstitial lung disease (RP-ILD) and have a high risk of mortality. There is an urgent need for a reliable prediction model, accessible via an easy-to-use web-based tool, to evaluate the risk of death. Objective: This study aimed to develop and validate a risk prediction model of 3-month mortality using machine learning (ML) in a large multicenter cohort of patients with anti-MDA5+DM-ILD in China. Methods: In total, 609 consecutive patients with anti-MDA5+DM-ILD were retrospectively enrolled from 6 hospitals across China. Patient demographics and laboratory and clinical parameters were collected on admission. The primary endpoint was 3-month mortality due to all causes. Six ML algorithms (Extreme Gradient Boosting [XGBoost], logistic regression (LR), Light Gradient Boosting Machine [LightGBM], random forest [RF], support vector machine [SVM], and k-nearest neighbor [KNN]) were applied to construct and evaluate the model. Results: After applying inclusion and exclusion criteria, 509 (83.6%) of the 609 patients were included in our study, divided into a training cohort (n=203, 39.9%), an internal validation cohort (n=51, 10%), and 2 external validation cohorts (n=92, 18.1%, and n=163, 32%). ML identified 8 important variables as critical for model construction: RP-ILD, erythrocyte sedimentation rate (ESR), serum albumin (ALB) level, age, C-reactive protein (CRP) level, aspartate aminotransferase (AST) level, lactate dehydrogenase (LDH) level, and the neutrophil-to-lymphocyte ratio (NLR). LR was chosen as the best algorithm for model construction, and the model demonstrated excellent performance, with an area under the receiver operating characteristic (ROC) curve (AUC) of 0.866, a sensitivity of 84.8%, and a specificity of 84.4% on the validation data set and an AUC of 0.90, a sensitivity of 85.0%, and a specificity of 83.9% on the training data set. Calibration curves and decision curve analysis (DCA) confirmed the model’s accuracy and clinical applicability. Moreover, the model showed strong predictive performance in the external validation cohorts (cohort 1: AUC=0.836, 95% CI 0.754-0.916; cohort 2: AUC=0.915, 95% CI 0.871-0.959), indicating good generalizability. This model was integrated into a web-based tool to predict the 3-month mortality for patients with anti-MDA5+DM-ILD. Conclusions: We successfully developed a robust clinical prediction model and an accompanying web tool to estimate the 3-month mortality risk for patients with anti-MDA5+DM-ILD. %R 10.2196/62836 %U https://www.jmir.org/2025/1/e62836 %U https://doi.org/10.2196/62836 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e63897 %T The Prognostic Significance of Sleep and Circadian Rhythm for Myocardial Infarction Outcomes: Case-Control Study %A Chin,Wei-Chih %A Chu,Pao-Hsien %A Wu,Lung-Sheng %A Lee,Kuang-Tso %A Lin,Chen %A Ho,Chien-Te %A Yang,Wei-Sheng %A Chung,I-Hang %A Huang,Yu-Shu %+ Division of Psychiatry and Sleep Center, Chang Gung Memorial Hospital, No. 5, Fuxing St., Guishan, Taoyuan, 333423, Taiwan, 886 3 3281200 ext 2479, yushuhuang1212@gmail.com %K myocardial infarction %K circadian rhythm %K actigraphy %K nonparametric analysis %K prognosis %K sleep %K heart rate variability %K activity %D 2025 %7 4.2.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Myocardial infarction (MI) is a medical emergency resulting from coronary artery occlusion. Patients with acute MI often experience disturbed sleep and circadian rhythm. Most previous studies assessed the premorbid sleep and circadian rhythm of patients with MI and their correlations with cardiovascular disease. However, little is known about post-MI sleep and circadian rhythm and their impacts on prognosis. The use of actigraphy with different algorithms to evaluate sleep and circadian rhythm after acute MI has the potential for predicting outcomes and preventing future disease progression. Objective: We aimed to evaluate how sleep patterns and disrupted circadian rhythm affect the prognosis of MI, using actigraphy and heart rate variability (HRV). Nonparametric analysis of actigraphy data was performed to examine the circadian rhythm of patients. Methods: Patients with MI in the intensive care unit (ICU) were enrolled alongside age- and gender-matched healthy controls. Actigraphy was used to evaluate sleep and circadian rhythm, while HRV was monitored for 24 hours to assess autonomic nerve function. Nonparametric indicators were calculated to quantify the active-rest patterns, including interdaily stability, intradaily variability, the most active 10 consecutive hours (M10), the least active 5 consecutive hours (L5), the relative amplitude, and the actigraphic dichotomy index. Follow-ups were conducted at 3 and 6 months after discharge to evaluate prognosis, including the duration of current admission, the number and duration of readmission and ICU admission, and catheterization. Independent sample t tests and analysis of covariance were used to compare group differences. Pearson correlation tests were used to explore the correlations of the parameters of actigraphy and HRV with prognosis. Results: The study included 34 patients with MI (mean age 57.65, SD 9.03 years) and 17 age- and gender-matched controls. MI patients had significantly more wake after sleep onset, an increased number of awakenings, and a lower sleep efficiency than controls. Circadian rhythm analysis revealed significantly lower daytime activity in MI patients. Moreover, these patients had a lower relative amplitude and dichotomy index and a higher intradaily variability and midpoint of M10, suggesting less sleep and wake activity changes, more fragmentation of the rest-activity patterns, and a more delayed circadian rhythm. Furthermore, significant correlations were found between the parameters of circadian rhythm analysis, including nighttime activity, time of M10 and L5, and daytime and nighttime activitySD, and patient prognosis. Conclusions: Patients with acute MI experienced significantly worse sleep and disturbed circadian rhythm compared with healthy controls. Our actigraphy-based analysis revealed a disturbed circadian rhythm, including reduced daytime activities, greater fluctuation in hourly activities, and a weak rest-activity rhythm, which were correlated with prognosis. The evaluation of sleep and circadian rhythm in patients with acute MI can serve as a valuable indicator for prognosis and should be further studied. %M 39903495 %R 10.2196/63897 %U https://www.jmir.org/2025/1/e63897 %U https://doi.org/10.2196/63897 %U http://www.ncbi.nlm.nih.gov/pubmed/39903495 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e58338 %T Challenges and Opportunities for Data Sharing Related to Artificial Intelligence Tools in Health Care in Low- and Middle-Income Countries: Systematic Review and Case Study From Thailand %A Kaushik,Aprajita %A Barcellona,Capucine %A Mandyam,Nikita Kanumoory %A Tan,Si Ying %A Tromp,Jasper %+ Saw Swee Hock School of Public Health, National University of Singapore, Tahir Foundation Building, 12 Science Drive 2, #10-01, Singapore, 117549, Singapore, 65 6516 4988, jasper_tromp@nus.edu.sg %K artificial intelligence %K data sharing %K health care %K low- and middle-income countries %K AI tools %K systematic review %K case study %K Thailand %K computing machinery %K academic experts %K technology developers %K health care providers %K internet connectivity %K data systems %K low health data literacy %K cybersecurity %K standardized data formats %K AI development %K PRISMA %D 2025 %7 4.2.2025 %9 Review %J J Med Internet Res %G English %X Background: Health care systems in low- and middle-income countries (LMICs) can greatly benefit from artificial intelligence (AI) interventions in various use cases such as diagnostics, treatment, and public health monitoring but face significant challenges in sharing data for developing and deploying AI in health care. Objective: This study aimed to identify barriers and enablers to data sharing for AI in health care in LMICs and to test the relevance of these in a local context. Methods: First, we conducted a systematic literature search using PubMed, SCOPUS, Embase, Web of Science, and ACM using controlled vocabulary. Primary research studies, perspectives, policy landscape analyses, and commentaries performed in or involving an LMIC context were included. Studies that lacked a clear connection to health information exchange systems or were not reported in English were excluded from the review. Two reviewers independently screened titles and abstracts of the included articles and critically appraised each study. All identified barriers and enablers were classified according to 7 categories as per the predefined framework—technical, motivational, economic, political, legal and policy, ethical, social, organisational, and managerial. Second, we tested the local relevance of barriers and enablers in Thailand through stakeholder interviews with 15 academic experts, technology developers, regulators, policy makers, and health care providers. The interviewers took notes and analyzed data using framework analysis. Coding procedures were standardized to enhance the reliability of our approach. Coded data were reverified and themes were readjusted where necessary to avoid researcher bias. Results: We identified 22 studies, the majority of which were conducted across Africa (n=12, 55%) and Asia (n=6, 27%). The most important data-sharing challenges were unreliable internet connectivity, lack of equipment, poor staff and management motivation, uneven resource distribution, and ethical concerns. Possible solutions included improving IT infrastructure, enhancing funding, introducing user-friendly software, and incentivizing health care organizations and personnel to share data for AI-related tools. In Thailand, inconsistent data systems, limited staff time, low health data literacy, complex and unclear policies, and cybersecurity issues were important data-sharing challenges. Key solutions included building a conducive digital ecosystem—having shared data input platforms for health facilities to ensure data uniformity and to develop easy-to-understand consent forms, having standardized guidelines for data sharing, and having compensation policies for data breach victims. Conclusions: Although AI in LMICs has the potential to overcome health inequalities, these countries face technical, political, legal, policy, and organizational barriers to sharing data, which impede effective AI development and deployment. When tested in a local context, most of these barriers were relevant. Although our findings might not be generalizable to other contexts, this study can be used by LMICs as a framework to identify barriers and strengths within their health care systems and devise localized solutions for enhanced data sharing. Trial Registration: PROSPERO CRD42022360644; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=360644 %M 39903508 %R 10.2196/58338 %U https://www.jmir.org/2025/1/e58338 %U https://doi.org/10.2196/58338 %U http://www.ncbi.nlm.nih.gov/pubmed/39903508 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e62670 %T Electronic Health Record Data Collection Practices to Advance Standardization and Interoperability of Patient Preferences for Interpretation Services: Qualitative Study %A Heaney-Huls,Krysta %A Shams,Rida %A Nwefo,Ruth %A Kane,Rachel %A Gordon,Janna %A Laffan,Alison M %A Stare,Scott %A Dullabh,Prashila %+ , NORC at the University of Chicago, 55 E Monroe St 30th Floor, Chicago, IL, 60603, United States, 1 7734017110, heaney-huls-krysta@norc.org %K health information exchange %K interoperability %K electronic health records %K interpreter %K limited English proficiency %K communication barriers %D 2025 %7 31.1.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Poor health outcomes are well documented among patients with a non-English language preference (NELP). The use of interpreters can improve the quality of care for patients with NELP. Despite a growing and unmet need for interpretation services in the US health care system, rates of interpreter use in the care setting are consistently low. Standardized collection and exchange of patient interpretation needs can improve access to appropriate language assistance services. Objective: This study aims to examine current practices for collecting, documenting, and exchanging information on a patient’s self-reported preference for an interpreter in the electronic health record (EHR) and the implementation maturity and adoption level of available data standards. The paper identifies standards implementation; data collection workflows; use cases for collecting, documenting, and exchanging information on a patient’s self-reported preference for an interpreter; challenges to data collection and use; and opportunities to advance standardization of the interpreter needed data element to facilitate patient-centered care. Methods: We conducted a narrative review to describe the availability of terminology standards to facilitate health care organization documentation of a patient’s self-reported preference for an interpreter in the EHR. Key informant discussions with EHR developers, health systems, clinicians, a practice-based research organization, a national standards collaborative, a professional health care association, and Federal agency representatives filled in gaps from the narrative review. Results: The findings indicate that health care organizations value standardized collection and exchange of patient language assistance service needs and preferences. Informants identified three use cases for collecting, documenting, and exchanging information on a patient’s self-reported preference for an interpreter, which are (1) person-centered care, (2) transitions of care, and (3) health care administration. The discussions revealed that EHR developers provide a data field for documenting interpreter needed data, which are routinely collected across health care organizations through commonly used data collection workflows. However, this data element is not mapped to standard terminologies, such as Logical Observation Identifiers Names and Codes (LOINC) or Systematized Medical Nomenclature for Medicine–Clinical Terminology (SNOMED-CT), consequently limiting the opportunities to electronically share these data between health systems and community-based organizations. The narrative review and key informant discussions identified three potential challenges to using information on a patient’s self-reported preference for an interpreter for person-centered care and quality improvement, which are (1) lack of adoption of available data standards, (2) limited electronic exchange, and (3) patient mistrust. Conclusions: Collecting and documenting patient’s self-reported interpreter preferences can improve the quality of services provided, patient care experiences, and equitable health care delivery without invoking a significant burden on the health care system. Although there is routine collection and documentation of patient interpretation needs, the lack of standardization limits the exchange of this information among health care and community-based organizations. %M 39888652 %R 10.2196/62670 %U https://www.jmir.org/2025/1/e62670 %U https://doi.org/10.2196/62670 %U http://www.ncbi.nlm.nih.gov/pubmed/39888652 %0 Journal Article %@ 2561-7605 %I JMIR Publications %V 8 %N %P e66104 %T Comparison of 3 Aging Metrics in Dual Declines to Capture All-Cause Dementia and Mortality Risk: Cohort Study %A Bai,Anying %A He,Shan %A Jiang,Yu %A Xu,Weihao %A Lin,Zhanyi %K gerontology %K geriatrics %K older adults %K older people %K aging %K motoric cognitive risk syndrome %K MCR %K physio-cognitive decline syndrome %K PCDS %K cognitive frailty %K CF %K frailty %K discrimination %K risk factors %K prediction %K dementia risk %K mortality risk %D 2025 %7 30.1.2025 %9 %J JMIR Aging %G English %X Background: The utility of aging metrics that incorporate cognitive and physical function is not fully understood. Objective: We aim to compare the predictive capacities of 3 distinct aging metrics—motoric cognitive risk syndrome (MCR), physio-cognitive decline syndrome (PCDS), and cognitive frailty (CF)—for incident dementia and all-cause mortality among community-dwelling older adults. Methods: We used longitudinal data from waves 10-15 of the Health and Retirement Study. Cox proportional hazards regression analysis was employed to evaluate the effects of MCR, PCDS, and CF on incident all-cause dementia and mortality, controlling for socioeconomic and lifestyle factors, as well as medical comorbidities. Discrimination analysis was conducted to assess and compare the predictive accuracy of the 3 aging metrics. Results: A total of 2367 older individuals aged 65 years and older, with no baseline prevalence of dementia or disability, were ultimately included. The prevalence rates of MCR, PCDS, and CF were 5.4%, 6.3%, and 1.3%, respectively. Over a decade-long follow-up period, 341 cases of dementia and 573 deaths were recorded. All 3 metrics were predictive of incident all-cause dementia and mortality when adjusting for multiple confounders, with variations in the strength of their associations (incident dementia: MCR odds ratio [OR] 1.90, 95% CI 1.30‐2.78; CF 5.06, 95% CI 2.87‐8.92; PCDS 3.35, 95% CI 2.44‐4.58; mortality: MCR 1.60, 95% CI 1.17‐2.19; CF 3.26, 95% CI 1.99‐5.33; and PCDS 1.58, 95% CI 1.17‐2.13). The C-index indicated that PCDS and MCR had the highest discriminatory accuracy for all-cause dementia and mortality, respectively. Conclusions: Despite the inherent differences among the aging metrics that integrate cognitive and physical functions, they consistently identified risks of dementia and mortality. This underscores the importance of implementing targeted preventive strategies and intervention programs based on these metrics to enhance the overall quality of life and reduce premature deaths in aging populations. %R 10.2196/66104 %U https://aging.jmir.org/2025/1/e66104 %U https://doi.org/10.2196/66104 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e58760 %T Identification of Intracranial Germ Cell Tumors Based on Facial Photos: Exploratory Study on the Use of Deep Learning for Software Development %A Li,Yanong %A He,Yixuan %A Liu,Yawei %A Wang,Bingchen %A Li,Bo %A Qiu,Xiaoguang %+ Department of Radiation Oncology, Beijing Tiantan Hospital, Capital Medical University, 119 West Southern 4th Ring Road, Fengtai District, Beijing, 100070, China, 86 10 59975581, qiuxiaoguang@bjtth.org %K deep learning %K facial recognition %K intracranial germ cell tumors %K endocrine indicators %K software development %K artificial intelligence %K machine learning models %K software engineering %K neural networks %K algorithms %K cohort studies %D 2025 %7 30.1.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Primary intracranial germ cell tumors (iGCTs) are highly malignant brain tumors that predominantly occur in children and adolescents, with an incidence rate ranking third among primary brain tumors in East Asia (8%-15%). Due to their insidious onset and impact on critical functional areas of the brain, these tumors often result in irreversible abnormalities in growth and development, as well as cognitive and motor impairments in affected children. Therefore, early diagnosis through advanced screening techniques is vital for improving patient outcomes and quality of life. Objective: This study aimed to investigate the application of facial recognition technology in the early detection of iGCTs in children and adolescents. Early diagnosis through advanced screening techniques is vital for improving patient outcomes and quality of life. Methods: A multicenter, phased approach was adopted for the development and validation of a deep learning model, GVisageNet, dedicated to the screening of midline brain tumors from normal controls (NCs) and iGCTs from other midline brain tumors. The study comprised the collection and division of datasets into training (n=847, iGCTs=358, NCs=300, other midline brain tumors=189) and testing (n=212, iGCTs=79, NCs=70, other midline brain tumors=63), with an additional independent validation dataset (n=336, iGCTs=130, NCs=100, other midline brain tumors=106) sourced from 4 medical institutions. A regression model using clinically relevant, statistically significant data was developed and combined with GVisageNet outputs to create a hybrid model. This integration sought to assess the incremental value of clinical data. The model’s predictive mechanisms were explored through correlation analyses with endocrine indicators and stratified evaluations based on the degree of hypothalamic-pituitary-target axis damage. Performance metrics included area under the curve (AUC), accuracy, sensitivity, and specificity. Results: On the independent validation dataset, GVisageNet achieved an AUC of 0.938 (P<.01) in distinguishing midline brain tumors from NCs. Further, GVisageNet demonstrated significant diagnostic capability in distinguishing iGCTs from the other midline brain tumors, achieving an AUC of 0.739, which is superior to the regression model alone (AUC=0.632, P<.001) but less than the hybrid model (AUC=0.789, P=.04). Significant correlations were found between the GVisageNet’s outputs and 7 endocrine indicators. Performance varied with hypothalamic-pituitary-target axis damage, indicating a further understanding of the working mechanism of GVisageNet. Conclusions: GVisageNet, capable of high accuracy both independently and with clinical data, shows substantial potential for early iGCTs detection, highlighting the importance of combining deep learning with clinical insights for personalized health care. %M 39883924 %R 10.2196/58760 %U https://www.jmir.org/2025/1/e58760 %U https://doi.org/10.2196/58760 %U http://www.ncbi.nlm.nih.gov/pubmed/39883924 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 12 %N %P e53630 %T Evaluating the Development, Reliability, and Validation of the Tele-Primary Care Oral Health Clinical Information System Questionnaire: Cross-Sectional Questionnaire Study %A Sutan,Rosnah %A Ismail,Shahida %A Ibrahim,Roszita %+ Department of Public Health Medicine, Faculty of Medicine, Universiti Kebangsaan Malaysia, Jalan Yaakob Latiff, Bandar Tun Razak Cheras, Cheras Kuala Lumpur, 56000, Malaysia, 60 391459587, rosnah.sutan@hctm.ukm.edu.my %K telehealth %K electronic health %K eHealth %K public health information system %K psychometric analysis %D 2025 %7 29.1.2025 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Evaluating digital health service delivery in primary health care requires a validated questionnaire to comprehensively assess users’ ability to implement tasks customized to the program’s needs. Objective: This study aimed to develop, test the reliability of, and validate the Tele-Primary Care Oral Health Clinical Information System (TPC-OHCIS) questionnaire for evaluating the implementation of maternal and child digital health information systems. Methods: A cross-sectional study was conducted in 2 phases. The first phase focused on content item development and was validated by a group of 10 experts using the content validity index. The second phase was to assess its psychometric testing for reliability and validity. Results: A structured questionnaire of 65 items was constructed to assess the TPC-OHCIS delivery for primary health care use based on literature and has been validated by 10 experts, and 319 respondents answered the 65-item TPC-OHCIS questionnaire, with mean item scores ranging from 1.99 (SD 0.67) to 2.85 (SD 1.019). The content validity, reliability, and face validity showed a scale-level content validity index of 0.90, scale-level content validation ratio of 0.90, and item-level face validity index of 0.76, respectively. The internal reliability was calculated as a Cronbach α value of 0.90, with an intraclass correlation coefficient of 0.91. Scales were determined by the scree plot with eigenvalues >1, and 13 subscales were identified based on principal component analysis. The Kaiser-Meyer-Olkin value was 0.90 (P<.049). The total variance explained was 76.07%, and factor loading scores for all variables were >0.7. The Bartlett test of sphericity, determining construct validity, was found to be significant (P<.049). Conclusions: The TPC-OHCIS questionnaire is valid to be used at the primary health care level to evaluate the TPC-OHCIS implementation. It can assess health care workers’ work performance and job acceptance and improve the quality of care. %M 39879614 %R 10.2196/53630 %U https://humanfactors.jmir.org/2025/1/e53630 %U https://doi.org/10.2196/53630 %U http://www.ncbi.nlm.nih.gov/pubmed/39879614 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 11 %N %P e63809 %T An Explainable Artificial Intelligence Text Classifier for Suicidality Prediction in Youth Crisis Text Line Users: Development and Validation Study %A Thomas,Julia %A Lucht,Antonia %A Segler,Jacob %A Wundrack,Richard %A Miché,Marcel %A Lieb,Roselind %A Kuchinke,Lars %A Meinlschmidt,Gunther %+ Division of Clinical Psychology and Epidemiology, Faculty of Psychology, University of Basel, Missionsstrasse 60/62, Basel, 4055, Switzerland, 49 30 57714627, julia.thomas@krisenchat.de %K deep learning %K explainable artificial intelligence (XAI) %K large language model (LLM) %K machine learning %K neural network %K prevention %K risk monitoring %K suicide %K transformer model %K suicidality %K suicidal ideation %K self-murder %K self-harm %K youth %K adolescent %K adolescents %K public health %K language model %K language models %K chat protocols %K crisis helpline %K help-seeking behaviors %K German %K Shapley %K decision-making %K mental health %K health informatics %K mobile phone %D 2025 %7 29.1.2025 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: Suicide represents a critical public health concern, and machine learning (ML) models offer the potential for identifying at-risk individuals. Recent studies using benchmark datasets and real-world social media data have demonstrated the capability of pretrained large language models in predicting suicidal ideation and behaviors (SIB) in speech and text. Objective: This study aimed to (1) develop and implement ML methods for predicting SIBs in a real-world crisis helpline dataset, using transformer-based pretrained models as a foundation; (2) evaluate, cross-validate, and benchmark the model against traditional text classification approaches; and (3) train an explainable model to highlight relevant risk-associated features. Methods: We analyzed chat protocols from adolescents and young adults (aged 14-25 years) seeking assistance from a German crisis helpline. An ML model was developed using a transformer-based language model architecture with pretrained weights and long short-term memory layers. The model predicted suicidal ideation (SI) and advanced suicidal engagement (ASE), as indicated by composite Columbia-Suicide Severity Rating Scale scores. We compared model performance against a classical word-vector-based ML model. We subsequently computed discrimination, calibration, clinical utility, and explainability information using a Shapley Additive Explanations value-based post hoc estimation model. Results: The dataset comprised 1348 help-seeking encounters (1011 for training and 337 for testing). The transformer-based classifier achieved a macroaveraged area under the curve (AUC) receiver operating characteristic (ROC) of 0.89 (95% CI 0.81-0.91) and an overall accuracy of 0.79 (95% CI 0.73-0.99). This performance surpassed the word-vector-based baseline model (AUC-ROC=0.77, 95% CI 0.64-0.90; accuracy=0.61, 95% CI 0.61-0.80). The transformer model demonstrated excellent prediction for nonsuicidal sessions (AUC-ROC=0.96, 95% CI 0.96-0.99) and good prediction for SI and ASE, with AUC-ROCs of 0.85 (95% CI 0.97-0.86) and 0.87 (95% CI 0.81-0.88), respectively. The Brier Skill Score indicated a 44% improvement in classification performance over the baseline model. The Shapley Additive Explanations model identified language features predictive of SIBs, including self-reference, negation, expressions of low self-esteem, and absolutist language. Conclusions: Neural networks using large language model–based transfer learning can accurately identify SI and ASE. The post hoc explainer model revealed language features associated with SI and ASE. Such models may potentially support clinical decision-making in suicide prevention services. Future research should explore multimodal input features and temporal aspects of suicide risk. %M 39879608 %R 10.2196/63809 %U https://publichealth.jmir.org/2025/1/e63809 %U https://doi.org/10.2196/63809 %U http://www.ncbi.nlm.nih.gov/pubmed/39879608 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e51689 %T Cross-Platform Ecological Momentary Assessment App (JTrack-EMA+): Development and Usability Study %A Sahandi Far,Mehran %A Fischer,Jona M %A Senge,Svea %A Rathmakers,Robin %A Meissner,Thomas %A Schneble,Dominik %A Narava,Mamaka %A Eickhoff,Simon B %A Dukart,Juergen %+ Research Centre Jülich, Institute of Neuroscience and Medicine, Brain and Behaviour (INM-7), Wilhelm-Johnen-Straße, Jülich, 52428, Germany, 49 17636977109, m.sahandi.far@fz-juelich.de %K digital biomarkers %K mobile health %K remote monitoring %K smartphone %K mobile phone %K monitoring %K biomarker %K ecological momentary assessment %K application %K costly %K user experience %K data management %K mobility %D 2025 %7 28.1.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Traditional in-clinic methods of collecting self-reported information are costly, time-consuming, subjective, and often limited in the quality and quantity of observation. However, smartphone-based ecological momentary assessments (EMAs) provide complementary information to in-clinic visits by collecting real-time, frequent, and longitudinal data that are ecologically valid. While these methods are promising, they are often prone to various technical obstacles. However, despite the potential of smartphone-based EMAs, they face technical obstacles that impact adaptability, availability, and interoperability across devices and operating systems. Deficiencies in these areas can contribute to selection bias by excluding participants with unsupported devices or limited digital literacy, increase development and maintenance costs, and extend deployment timelines. Moreover, these limitations not only impede the configurability of existing solutions but also hinder their adoption for addressing diverse clinical challenges. Objective: The primary aim of this research was to develop a cross-platform EMA app that ensures a uniform user experience and core features across various operating systems. Emphasis was placed on maximizing the integration and adaptability to various study designs, all while maintaining strict adherence to security and privacy protocols. JTrack-EMA+ was designed and implemented per the FAIR (findable, accessible, interpretable, and reusable) principles in both its architecture and data management layers, thereby reducing the burden of integration for clinicians and researchers. Methods: JTrack-EMA+ was built using the Flutter framework, enabling it to run seamlessly across different platforms. This platform comprises two main components. JDash (Research Centre Jülich, Institute of Neuroscience and Medicine, Brain and Behaviour [INM-7]) is an online management tool created using Python (Python Software Foundation) with the Django (Django Software Foundation) framework. This online dashboard offers comprehensive study management tools, including assessment design, user administration, data quality control, and a reminder casting center. The JTrack-EMA+ app supports a wide range of question types, allowing flexibility in assessment design. It also has configurable assessment logic and the ability to include supplementary materials for a richer user experience. It strongly commits to security and privacy and complies with the General Data Protection Regulations to safeguard user data and ensure confidentiality. Results: We investigated our platform in a pilot study with 480 days of follow-up to assess participants’ compliance. The 6-month average compliance was 49.3%, significantly declining (P=.004) from 66.7% in the first month to 42% in the sixth month. Conclusions: The JTrack-EMA+ platform prioritizes platform-independent architecture, providing an easy entry point for clinical researchers to deploy EMA in their respective clinical studies. Remote and home-based assessments of EMA using this platform can provide valuable insights into patients’ daily lives, particularly in a population with limited mobility or inconsistent access to health care services. %M 39874571 %R 10.2196/51689 %U https://www.jmir.org/2025/1/e51689 %U https://doi.org/10.2196/51689 %U http://www.ncbi.nlm.nih.gov/pubmed/39874571 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 13 %N %P e59452 %T The Social Construction of Categorical Data: Mixed Methods Approach to Assessing Data Features in Publicly Available Datasets %A Willem,Theresa %A Wollek,Alessandro %A Cheslerean-Boghiu,Theodor %A Kenney,Martha %A Buyx,Alena %+ Institute of History and Ethics in Medicine, School of Medicine and Health, Technical University of Munich, Ismaningerstraße 22, Munich, 81675, Germany, 49 89 4140 4041, theresa.willem@tum.de %K machine learning %K categorical data %K social context dependency %K mixed methods %K dermatology %K dataset analysis %D 2025 %7 28.1.2025 %9 Original Paper %J JMIR Med Inform %G English %X Background: In data-sparse areas such as health care, computer scientists aim to leverage as much available information as possible to increase the accuracy of their machine learning models’ outputs. As a standard, categorical data, such as patients’ gender, socioeconomic status, or skin color, are used to train models in fusion with other data types, such as medical images and text-based medical information. However, the effects of including categorical data features for model training in such data-scarce areas are underexamined, particularly regarding models intended to serve individuals equitably in a diverse population. Objective: This study aimed to explore categorical data’s effects on machine learning model outputs, rooted the effects in the data collection and dataset publication processes, and proposed a mixed methods approach to examining datasets’ data categories before using them for machine learning training. Methods: Against the theoretical background of the social construction of categories, we suggest a mixed methods approach to assess categorical data’s utility for machine learning model training. As an example, we applied our approach to a Brazilian dermatological dataset (Dermatological and Surgical Assistance Program at the Federal University of Espírito Santo [PAD-UFES] 20). We first present an exploratory, quantitative study that assesses the effects when including or excluding each of the unique categorical data features of the PAD-UFES 20 dataset for training a transformer-based model using a data fusion algorithm. We then pair our quantitative analysis with a qualitative examination of the data categories based on interviews with the dataset authors. Results: Our quantitative study suggests scattered effects of including categorical data for machine learning model training across predictive classes. Our qualitative analysis gives insights into how the categorical data were collected and why they were published, explaining some of the quantitative effects that we observed. Our findings highlight the social constructedness of categorical data in publicly available datasets, meaning that the data in a category heavily depend on both how these categories are defined by the dataset creators and the sociomedico context in which the data are collected. This reveals relevant limitations of using publicly available datasets in contexts different from those of the collection of their data. Conclusions: We caution against using data features of publicly available datasets without reflection on the social construction and context dependency of their categorical data features, particularly in data-sparse areas. We conclude that social scientific, context-dependent analysis of available data features using both quantitative and qualitative methods is helpful in judging the utility of categorical data for the population for which a model is intended. %M 39874567 %R 10.2196/59452 %U https://medinform.jmir.org/2025/1/e59452 %U https://doi.org/10.2196/59452 %U http://www.ncbi.nlm.nih.gov/pubmed/39874567 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 13 %N %P e66444 %T The Impact of Data Control and Delayed Discounting on the Public’s Willingness to Share Different Types of Health Care Data: Empirical Study %A Wei,Dongle %A Gao,Pan %A Zhai,Yunkai %K health data control %K delay discounting rate %K mental accounting %K health data %K data sharing %K willingness %K patient-generated data %K clinical medical data %K disease prevention %K precision medicine %K health care %K portability %K accountability %K app %K web-based survey %K data security %K data privacy %K mobile phone %D 2025 %7 22.1.2025 %9 %J JMIR Med Inform %G English %X Background: Health data typically include patient-generated data and clinical medical data. Different types of data contribute to disease prevention, precision medicine, and the overall improvement of health care. With the introduction of regulations such as the Health Insurance Portability and Accountability Act (HIPAA), individuals play a key role in the sharing and application of personal health data. Objective: This study aims to explore the impact of different types of health data on users’ willingness to share. Additionally, it analyzes the effect of data control and delay discounting rate on this process. Methods: The results of a web-based survey were analyzed to examine individuals’ perceptions of sharing different types of health data and how data control and delay discounting rates influenced their decisions. We recruited participants for our study through the web-based platform “Wenjuanxing.” After screening, we obtained 257 valid responses. Regression analysis was used to investigate the impact of data control, delayed discounting, and mental accounting on the public’s willingness to share different types of health care data. Results: Our findings indicate that the type of health data does not significantly affect the perceived benefits of data sharing. Instead, it negatively influences willingness to share by indirectly affecting data acquisition costs and perceived risks. Our results also show that data control reduces the perceived risks associated with sharing, while higher delay discounting rates lead to an overestimation of data acquisition costs and perceived risks. Conclusions: Individuals’ willingness to share data is primarily influenced by costs. To promote the acquisition and development of personal health data, stakeholders should strengthen individuals’ control over their data or provide direct short-term incentives. %R 10.2196/66444 %U https://medinform.jmir.org/2025/1/e66444 %U https://doi.org/10.2196/66444 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 14 %N %P e66094 %T Applications of Natural Language Processing and Large Language Models for Social Determinants of Health: Protocol for a Systematic Review %A Rajwal,Swati %A Zhang,Ziyuan %A Chen,Yankai %A Rogers,Hannah %A Sarker,Abeed %A Xiao,Yunyu %+ Department of Computer Science, Emory University, Mathematics & Science Center, Suite W401, 400 Dowman Drive, Atlanta, GA, 30322, United States, 1 4704478469, swati.rajwal@emory.edu %K social determinants of health %K SDOH %K natural language processing %K NLP %K systematic review protocol %K large language models %K LLM %D 2025 %7 21.1.2025 %9 Protocol %J JMIR Res Protoc %G English %X Background: In recent years, the intersection of natural language processing (NLP) and public health has opened innovative pathways for investigating social determinants of health (SDOH) in textual datasets. Despite the promise of NLP in the SDOH domain, the literature is dispersed across various disciplines, and there is a need to consolidate existing knowledge, identify knowledge gaps in the literature, and inform future research directions in this emerging field. Objective: This research protocol describes a systematic review to identify and highlight NLP techniques, including large language models, used for SDOH-related studies. Methods: A search strategy will be executed across PubMed, Web of Science, IEEE Xplore, Scopus, PsycINFO, HealthSource: Academic Nursing, and ACL Anthology to find studies published in English between 2014 and 2024. Three reviewers (SR, ZZ, and YC) will independently screen the studies to avoid voting bias, and two (AS and YX) additional reviewers will resolve any conflicts during the screening process. We will further screen studies that cited the included studies (forward search). Following the title abstract and full-text screening, the characteristics and main findings of the included studies and resources will be tabulated, visualized, and summarized. Results: The search strategy was formulated and run across the 7 databases in August 2024. We expect the results to be submitted for peer review publication in early 2025. As of December 2024, the title and abstract screening was underway. Conclusions: This systematic review aims to provide a comprehensive study of existing research on the application of NLP for various SDOH tasks across multiple textual datasets. By rigorously evaluating the methodologies, tools, and outcomes of eligible studies, the review will identify gaps in current knowledge and suggest directions for future research in the form of specific research questions. The findings will be instrumental in developing more effective NLP models for SDOH, ultimately contributing to improved health outcomes and a better understanding of social determinants in diverse populations. International Registered Report Identifier (IRRID): DERR1-10.2196/66094 %M 39836952 %R 10.2196/66094 %U https://www.researchprotocols.org/2025/1/e66094 %U https://doi.org/10.2196/66094 %U http://www.ncbi.nlm.nih.gov/pubmed/39836952 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 13 %N %P e58649 %T Interpretable Machine Learning Model for Predicting Postpartum Depression: Retrospective Study %A Zhang,Ren %A Liu,Yi %A Zhang,Zhiwei %A Luo,Rui %A Lv,Bin %K postpartum depression %K machine learning %K predictive model %K risk factors %K XGBoost %K extreme gradient boosting %K PPD %D 2025 %7 20.1.2025 %9 %J JMIR Med Inform %G English %X Background: Postpartum depression (PPD) is a prevalent mental health issue with significant impacts on mothers and families. Exploring reliable predictors is crucial for the early and accurate prediction of PPD, which remains challenging. Objective: This study aimed to comprehensively collect variables from multiple aspects, develop and validate machine learning models to achieve precise prediction of PPD, and interpret the model to reveal clinical implications. Methods: This study recruited pregnant women who delivered at the West China Second University Hospital, Sichuan University. Various variables were collected from electronic medical record data and screened using least absolute shrinkage and selection operator penalty regression. Participants were divided into training (1358/2055, 66.1%) and validation (697/2055, 33.9%) sets by random sampling. Machine learning–based predictive models were developed in the training cohort. Models were validated in the validation cohort with receiver operating curve and decision curve analysis. Multiple model interpretation methods were implemented to explain the optimal model. Results: We recruited 2055 participants in this study. The extreme gradient boosting model was the optimal predictive model with the area under the receiver operating curve of 0.849. Shapley Additive Explanation indicated that the most influential predictors of PPD were antepartum depression, lower fetal weight, elevated thyroid-stimulating hormone, declined thyroid peroxidase antibodies, elevated serum ferritin, and older age. Conclusions: This study developed and validated a machine learning–based predictive model for PPD. Several significant risk factors and how they impact the prediction of PPD were revealed. These findings provide new insights into the early screening of individuals with high risk for PPD, emphasizing the need for comprehensive screening approaches that include both physiological and psychological factors. %R 10.2196/58649 %U https://medinform.jmir.org/2025/1/e58649 %U https://doi.org/10.2196/58649 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 11 %N %P e66179 %T Prevalence and Economic Impact of Acute Respiratory Failure in the Prehospital Emergency Medical Service of the Madrid Community: Retrospective Cohort Study %A Cintora-Sanz,Ana María %A Horrillo-García,Cristina %A Quesada-Cubo,Víctor %A Pérez-Alonso,Ana María %A Gutiérrez-Misis,Alicia %K acute respiratory failure %K COVID-19 %K chronic obstructive respiratory insufficiency %K congestive heart failure %K bronchospasm %K emergency medical services costs %K ambulances %K SARS-CoV-2 %K coronavirus %K respiratory %K pulmonary %K pandemic %K economic impact %K observational %K Madrid %K community %K medical records %K health records %K medical advanced life support %K ALS %K acute pulmonary edema %K chronic obstructive pulmonary disease %K COPD %K prevalence %D 2025 %7 16.1.2025 %9 %J JMIR Public Health Surveill %G English %X Background: Chronic obstructive pulmonary disease (COPD), congestive heart failure (CHF), and acute pulmonary edema (APE) are serious illnesses that often require acute care from prehospital emergency medical services (EMSs). These respiratory diseases that cause acute respiratory failure (ARF) are one of the main reasons for hospitalization and death, generating high health care costs. The prevalence of the main respiratory diseases treated in a prehospital environment in the prepandemic period and during the COVID-19 pandemic in Spain is unknown. The Madrid Community EMS is a public service that serves all types of populations and represents an epidemiological reference for supporting a population of 6.4 million inhabitants. The high volume of patients treated by Madrid’s medical advanced life supports (ALSs) allows us to analyze this little-studied problem. Objectives: Our goal was to lay the groundwork for comprehensive data collection and surveillance of respiratory failure, with an emphasis on the most prevalent diseases that cause it, an aspect that has been largely overlooked in previous initiatives. By achieving these objectives, we hope to inform efforts to address respiratory failure and establish a standardized methodology and framework that can facilitate expansion to a continuous community-wide registry in Madrid, driving advances in emergency care and care practices in these pathologies. The aim of this retrospective observational study was to determine the pathologies that have mainly caused respiratory failure in patients and required medicalized ALS and to evaluate the cost of care for these pathologies collected through this pilot registry. Methods: A multicenter descriptive study was carried out in the Madrid Community EMS. The anonymized medical records of patients treated with medical ALS, who received any of the following medical diagnoses, were extracted: ARF not related to chronic respiratory disease, ARF in chronic respiratory failure, exacerbations of COPD, APE, CHF, and bronchospasm (not from asthma or COPD). The prevalence of each pathology, its evolution from 2014 to 2020, and the economic impact of the Medical ALSs were calculated. Results: The study included 96,221 patients. The most common pathology was exacerbation of COPD, with a prevalence of 0.07% in 2014; it decreased to 0.03% in 2020. CHF followed at 0.06% in 2014 and 0.03% in 2020. APE had a prevalence of 0.01% in 2014, decreasing to 0.005% in 2020 with the pandemic. The greatest economic impact was on exacerbation of COPD in 2015, with an annual cost of €2,726,893 (which equals to US $2,864,628). Conclusions: COPD exacerbations had the higher prevalence in the Madrid region among the respiratory diseases studied. With the COVID-19 pandemic, the prevalence and costs of almost all these diseases decreased, except for ARF not related to chronic disease. The cost of these pathologies over 5 years was €58,791,031 (US $61,832,879). %R 10.2196/66179 %U https://publichealth.jmir.org/2025/1/e66179 %U https://doi.org/10.2196/66179 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 14 %N %P e63875 %T In Silico Evaluation of Algorithm-Based Clinical Decision Support Systems: Protocol for a Scoping Review %A Dorosan,Michael %A Chen,Ya-Lin %A Zhuang,Qingyuan %A Lam,Shao Wei Sean %+ Health Services Research Centre, Singapore Health Services Pte Ltd, Health Services Research Institute (HSRI) Academia, Ngee Ann Kongsi Discovery Tower Level 6, 20 College Road, Singapore, 169856, Singapore, 65 65767140, gmslasws@nus.edu.sg %K clinical decision support algorithms %K in silico evaluation %K clinical workflow simulation %K health care modeling %K digital twin %K quadruple aims %K clinical decision %K decision-making %K decision support %K workflow %K support system %K protocol %K scoping review %K algorithm-based %K screening %K thematic analysis %K descriptive analysis %K clinical decision-making %D 2025 %7 16.1.2025 %9 Protocol %J JMIR Res Protoc %G English %X Background: Integrating algorithm-based clinical decision support (CDS) systems poses significant challenges in evaluating their actual clinical value. Such CDS systems are traditionally assessed via controlled but resource-intensive clinical trials. Objective: This paper presents a review protocol for preimplementation in silico evaluation methods to enable broadened impact analysis under simulated environments before clinical trials. Methods: We propose a scoping review protocol that follows an enhanced Arksey and O’Malley framework and PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines to investigate the scope and research gaps in the in silico evaluation of algorithm-based CDS models—specifically CDS decision-making end points and objectives, evaluation metrics used, and simulation paradigms used to assess potential impacts. The databases searched are PubMed, Embase, CINAHL, PsycINFO, Cochrane, IEEEXplore, Web of Science, and arXiv. A 2-stage screening process identified pertinent articles. The information extracted from articles was iteratively refined. The review will use thematic, trend, and descriptive analyses to meet scoping aims. Results: We conducted an automated search of the databases above in May 2023, with most title and abstract screenings completed by November 2023 and full-text screening extended from December 2023 to May 2024. Concurrent charting and full-text analysis were carried out, with the final analysis and manuscript preparation set for completion in July 2024. Publication of the review results is targeted from July 2024 to February 2025. As of April 2024, a total of 21 articles have been selected following a 2-stage screening process; these will proceed to data extraction and analysis. Conclusions: We refined our data extraction strategy through a collaborative, multidisciplinary approach, planning to analyze results using thematic analyses to identify approaches to in silico evaluation. Anticipated findings aim to contribute to developing a unified in silico evaluation framework adaptable to various clinical workflows, detailing clinical decision-making characteristics, impact measures, and reusability of methods. The study’s findings will be published and presented in forums combining artificial intelligence and machine learning, clinical decision-making, and health technology impact analysis. Ultimately, we aim to bridge the development-deployment gap through in silico evaluation-based potential impact assessments. International Registered Report Identifier (IRRID): DERR1-10.2196/63875 %M 39819973 %R 10.2196/63875 %U https://www.researchprotocols.org/2025/1/e63875 %U https://doi.org/10.2196/63875 %U http://www.ncbi.nlm.nih.gov/pubmed/39819973 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e52385 %T A Digital Tool for Clinical Evidence–Driven Guideline Development by Studying Properties of Trial Eligible and Ineligible Populations: Development and Usability Study %A Mumtaz,Shahzad %A McMinn,Megan %A Cole,Christian %A Gao,Chuang %A Hall,Christopher %A Guignard-Duff,Magalie %A Huang,Huayi %A McAllister,David A %A Morales,Daniel R %A Jefferson,Emily %A Guthrie,Bruce %+ Division of Population Health and Genomics, School of Medicine, University of Dundee, The Health Informatics Centre, Ninewells Hospital and Medical School, Dundee, DD2 1FD, United Kingdom, 44 01382383943, e.r.jefferson@dundee.ac.uk %K multimorbidity %K clinical practice guideline %K gout %K Trusted Research Environment %K National Institute for Health and Care Excellence %K Scottish Intercollegiate Guidelines Network %K clinical practice %K development %K efficacy %K validity %K epidemiological data %K epidemiology %K epidemiological %K digital tool %K tool %K age %K gender %K ethnicity %K mortality %K feedback %K availability %D 2025 %7 16.1.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Clinical guideline development preferentially relies on evidence from randomized controlled trials (RCTs). RCTs are gold-standard methods to evaluate the efficacy of treatments with the highest internal validity but limited external validity, in the sense that their findings may not always be applicable to or generalizable to clinical populations or population characteristics. The external validity of RCTs for the clinical population is constrained by the lack of tailored epidemiological data analysis designed for this purpose due to data governance, consistency of disease or condition definitions, and reduplicated effort in analysis code. Objective: This study aims to develop a digital tool that characterizes the overall population and differences between clinical trial eligible and ineligible populations from the clinical populations of a disease or condition regarding demography (eg, age, gender, ethnicity), comorbidity, coprescription, hospitalization, and mortality. Currently, the process is complex, onerous, and time-consuming, whereas a real-time tool may be used to rapidly inform a guideline developer’s judgment about the applicability of evidence. Methods: The National Institute for Health and Care Excellence—particularly the gout guideline development group—and the Scottish Intercollegiate Guidelines Network guideline developers were consulted to gather their requirements and evidential data needs when developing guidelines. An R Shiny (R Foundation for Statistical Computing) tool was designed and developed using electronic primary health care data linked with hospitalization and mortality data built upon an optimized data architecture. Disclosure control mechanisms were built into the tool to ensure data confidentiality. The tool was deployed within a Trusted Research Environment, allowing only trusted preapproved researchers to conduct analysis. Results: The tool supports 128 chronic health conditions as index conditions and 161 conditions as comorbidities (33 in addition to the 128 index conditions). It enables 2 types of analyses via the graphic interface: overall population and stratified by user-defined eligibility criteria. The analyses produce an overview of statistical tables (eg, age, gender) of the index condition population and, within the overview groupings, produce details on, for example, electronic frailty index, comorbidities, and coprescriptions. The disclosure control mechanism is integral to the tool, limiting tabular counts to meet local governance needs. An exemplary result for gout as an index condition is presented to demonstrate the tool’s functionality. Guideline developers from the National Institute for Health and Care Excellence and the Scottish Intercollegiate Guidelines Network provided positive feedback on the tool. Conclusions: The tool is a proof-of-concept, and the user feedback has demonstrated that this is a step toward computer-interpretable guideline development. Using the digital tool can potentially improve evidence-driven guideline development through the availability of real-world data in real time. %M 39819848 %R 10.2196/52385 %U https://www.jmir.org/2025/1/e52385 %U https://doi.org/10.2196/52385 %U http://www.ncbi.nlm.nih.gov/pubmed/39819848 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e56463 %T Effectiveness of Remote Patient Monitoring Equipped With an Early Warning System in Tertiary Care Hospital Wards: Retrospective Cohort Study %A Lakshman,Pavithra %A Gopal,Priyanka T %A Khurdi,Sheen %+ Hospital Administration, Ramaiah Memorial Hospital, New BEL Rd, M S Ramaiah Nagar, MSRIT Post, Bengaluru, Karnataka, 560054, India, 91 9741592241, dr.pavithra.lakshman@gmail.com %K continuous vitals monitoring %K remote patient monitoring %K early warning system %K hospital wards %K retrospective %K cohort study %K early deterioration monitoring %K patient care %K decision making %K clinical information %D 2025 %7 15.1.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Monitoring vital signs in hospitalized patients is crucial for evaluating their clinical condition. While early warning scores like the modified early warning score (MEWS) are typically calculated 3 to 4 times daily through spot checks, they might not promptly identify early deterioration. Leveraging technologies that provide continuous monitoring of vital signs, combined with an early warning system, has the potential to identify clinical deterioration sooner. This approach empowers health care providers to intervene promptly and effectively. Objective: This study aimed to assess the impact of a Remote Patient Monitoring System (RPMS) with an automated early warning system (R-EWS) on patient safety in noncritical care at a tertiary hospital. R-EWS performance was compared with a simulated Modified Early Warning System (S-MEWS) and a simulated threshold-based alert system (S-Threshold). Methods: Patient outcomes, including intensive care unit (ICU) transfers due to deterioration and discharges for nondeteriorating cases, were analyzed in Ramaiah Memorial Hospital’s general wards with RPMS. Sensitivity, specificity, chi-square test for alert frequency distribution equality, and the average time from the first alert to ICU transfer in the last 24 hours was determined. Alert and patient distribution by tiers and vitals in R-EWS groups were examined. Results: Analyzing 905 patients, including 38 with deteriorations, R-EWS, S-Threshold, and S-MEWS generated more alerts for deteriorating cases. R-EWS showed high sensitivity (97.37%) and low specificity (23.41%), S-Threshold had perfect sensitivity (100%) but low specificity (0.46%), and S-MEWS demonstrated moderate sensitivity (47.37%) and high specificity (81.31%). The average time from initial alert to clinical deterioration was at least 18 hours for RPMS and S-Threshold in deteriorating participants. R-EWS had increased alert frequency and a higher proportion of critical alerts for deteriorating cases. Conclusions: This study underscores R-EWS role in early deterioration detection, emphasizing timely interventions for improved patient outcomes. Continuous monitoring enhances patient safety and optimizes care quality. %M 39813676 %R 10.2196/56463 %U https://www.jmir.org/2025/1/e56463 %U https://doi.org/10.2196/56463 %U http://www.ncbi.nlm.nih.gov/pubmed/39813676 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 12 %N %P e58073 %T Predictive Factors and the Predictive Scoring System for Falls in Acute Care Inpatients: Retrospective Cohort Study %A Saito,Chihiro %A Nakatani,Eiji %A Sasaki,Hatoko %A E Katsuki,Naoko %A Tago,Masaki %A Harada,Kiyoshi %K falls %K inpatient falls %K acute care hospital %K predictive factor %K risk factors %D 2025 %7 13.1.2025 %9 %J JMIR Hum Factors %G English %X Background: Falls in hospitalized patients are a serious problem, resulting in physical injury, secondary complications, impaired activities of daily living, prolonged hospital stays, and increased medical costs. Establishing a fall prediction scoring system to identify patients most likely to fall can help prevent falls among hospitalized patients. Objectives: This study aimed to identify predictive factors of falls in acute care hospital patients, develop a scoring system, and evaluate its validity. Methods: This single-center, retrospective cohort study involved patients aged 20 years or older admitted to Shizuoka General Hospital between April 2019 and September 2020. Demographic data, candidate predictors at admission, and fall occurrence reports were collected from medical records. The outcome was the time from admission to a fall requiring medical resources. Two-thirds of cases were randomly selected as the training set for analysis, and univariable and multivariable Cox regression analyses were used to identify factors affecting fall risk. We scored the fall risk based on the estimated hazard ratios (HRs) and constructed a fall prediction scoring system. The remaining one-third of cases was used as the test set to evaluate the predictive performance of the new scoring system. Results: A total of 13,725 individuals were included. During the study period, 2.4% (326/13,725) of patients experienced a fall. In the training dataset (n=9150), Cox regression analysis identified sex (male: HR 1.60, 95% CI 1.21‐2.13), age (65 to <80 years: HR 2.26, 95% CI 1.48‐3.44; ≥80 years: HR 2.50, 95% CI 1.60‐3.92 vs 20-<65 years), BMI (18.5 to <25 kg/m²: HR 1.36, 95% CI 0.94‐1.97; <18.5 kg/m²: HR 1.57, 95% CI 1.01‐2.44 vs ≥25 kg/m²), independence degree of daily living for older adults with disabilities (bedriddenness rank A: HR 1.81, 95% CI 1.26‐2.60; rank B: HR 2.03, 95% CI 1.31‐3.14; rank C: HR 1.23, 95% CI 0.83‐1.83 vs rank J), department (internal medicine: HR 1.23, 95% CI 0.92‐1.64; emergency department: HR 1.81, 95% CI 1.26‐2.60 vs department of surgery), and history of falls within 1 year (yes: HR 1.66, 95% CI 1.21‐2.27) as predictors of falls. Using these factors, we developed a fall prediction scoring system categorizing patients into 3 risk groups: low risk (0-4 points), intermediate risk (5-9 points), and high risk (10-15 points). The c-index indicating predictive performance in the test set (n=4575) was 0.733 (95% CI 0.684‐0.782). Conclusions: We developed a new fall prediction scoring system for patients admitted to acute care hospitals by identifying predictors of falls in Japan. This system may be useful for preventive interventions in patient populations with a high likelihood of falling in acute care settings. %R 10.2196/58073 %U https://humanfactors.jmir.org/2025/1/e58073 %U https://doi.org/10.2196/58073 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e55721 %T An Intelligent System for Classifying Patient Complaints Using Machine Learning and Natural Language Processing: Development and Validation Study %A Li,Xiadong %A Shu,Qiang %A Kong,Canhong %A Wang,Jinhu %A Li,Gang %A Fang,Xin %A Lou,Xiaomin %A Yu,Gang %+ Children's Hospital, Zhejiang University School of Medicine, National Clinical Research Center For Child Health, No. 3333, Binsheng Road, Binjiang District, Hangzhou, Hang Zhou, 310020, China, 86 13588773370, yugbme@zju.edu.cn %K complaint analysis %K text classification %K natural language processing %K NLP %K machine learning %K ML %K patient complaints %D 2025 %7 8.1.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Accurate classification of patient complaints is crucial for enhancing patient satisfaction management in health care settings. Traditional manual methods for categorizing complaints often lack efficiency and precision. Thus, there is a growing demand for advanced and automated approaches to streamline the classification process. Objective: This study aimed to develop and validate an intelligent system for automatically classifying patient complaints using machine learning (ML) and natural language processing (NLP) techniques. Methods: An ML-based NLP technology was proposed to extract frequently occurring dissatisfactory words related to departments, staff, and key treatment procedures. A dataset containing 1465 complaint records from 2019 to 2023 was used for training and validation, with an additional 376 complaints from Hangzhou Cancer Hospital serving as an external test set. Complaints were categorized into 4 types—communication problems, diagnosis and treatment issues, management problems, and sense of responsibility concerns. The imbalanced data were balanced using the Synthetic Minority Oversampling Technique (SMOTE) algorithm to ensure equal representation across all categories. A total of 3 ML algorithms (Multifactor Logistic Regression, Multinomial Naive Bayes, and Support Vector Machines [SVM]) were used for model training and validation. The best-performing model was tested using a 5-fold cross-validation on external data. Results: The original dataset consisted of 719, 376, 260, and 86 records for communication problems, diagnosis and treatment issues, management problems, and sense of responsibility concerns, respectively. The Multifactor Logistic Regression and SVM models achieved weighted average accuracies of 0.89 and 0.93 in the training set, and 0.83 and 0.87 in the internal test set, respectively. Ngram-level term frequency–inverse document frequency did not significantly improve classification performance, with only a marginal 1% increase in precision, recall, and F1-score when implementing Ngram-level term frequency–inverse document frequency (n=2) from 0.91 to 0.92. The SVM algorithm performed best in prediction, achieving an average accuracy of 0.91 on the external test set with a 95% CI of 0.87-0.97. Conclusions: The NLP-driven SVM algorithm demonstrates effective classification performance in automatically categorizing patient complaint texts. It showed superior performance in both internal and external test sets for communication and management problems. However, caution is advised when using it for classifying sense of responsibility complaints. This approach holds promises for implementation in medical institutions with high complaint volumes and limited resources for addressing patient feedback. %M 39778195 %R 10.2196/55721 %U https://www.jmir.org/2025/1/e55721 %U https://doi.org/10.2196/55721 %U http://www.ncbi.nlm.nih.gov/pubmed/39778195 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e55015 %T Digital Twins for Clinical and Operational Decision-Making: Scoping Review %A Riahi,Vahid %A Diouf,Ibrahima %A Khanna,Sankalp %A Boyle,Justin %A Hassanzadeh,Hamed %+ Australian e-Health Research Centre, Commonwealth Scientific and Industrial Research Organisation, 343 Royal Parade Parkville Vic 3052, Melbourne, Australia, 61 469842309, vahid.riahi@csiro.au %K digital twin %K health care %K clinical decision-making %K CDM %K operational decision-making %K ODM %K scoping review %D 2025 %7 8.1.2025 %9 Review %J J Med Internet Res %G English %X Background: The health care industry must align with new digital technologies to respond to existing and new challenges. Digital twins (DTs) are an emerging technology for digital transformation and applied intelligence that is rapidly attracting attention. DTs are virtual representations of products, systems, or processes that interact bidirectionally in real time with their actual counterparts. Although DTs have diverse applications from personalized care to treatment optimization, misconceptions persist regarding their definition and the extent of their implementation within health systems. Objective: This study aimed to review DT applications in health care, particularly for clinical decision-making (CDM) and operational decision-making (ODM). It provides a definition and framework for DTs by exploring their unique elements and characteristics. Then, it assesses the current advances and extent of DT applications to support CDM and ODM using the defined DT characteristics. Methods: We conducted a scoping review following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) protocol. We searched multiple databases, including PubMed, MEDLINE, and Scopus, for original research articles describing DT technologies applied to CDM and ODM in health systems. Papers proposing only ideas or frameworks or describing DT capabilities without experimental data were excluded. We collated several available types of information, for example, DT characteristics, the environment that DTs were tested within, and the main underlying method, and used descriptive statistics to analyze the synthesized data. Results: Out of 5537 relevant papers, 1.55% (86/5537) met the predefined inclusion criteria, all published after 2017. The majority focused on CDM (75/86, 87%). Mathematical modeling (24/86, 28%) and simulation techniques (17/86, 20%) were the most frequently used methods. Using International Classification of Diseases, 10th Revision coding, we identified 3 key areas of DT applications as follows: factors influencing diseases of the circulatory system (14/86, 16%); health status and contact with health services (12/86, 14%); and endocrine, nutritional, and metabolic diseases (10/86, 12%). Only 16 (19%) of 86 studies tested the developed system in a real environment, while the remainder were evaluated in simulated settings. Assessing the studies against defined DT characteristics reveals that the developed systems have yet to materialize the full capabilities of DTs. Conclusions: This study provides a comprehensive review of DT applications in health care, focusing on CDM and ODM. A key contribution is the development of a framework that defines important elements and characteristics of DTs in the context of related literature. The DT applications studied in this paper reveal encouraging results that allow us to envision that, in the near future, they will play an important role not only in the diagnosis and prevention of diseases but also in other areas, such as efficient clinical trial design, as well as personalized and optimized treatments. %M 39778199 %R 10.2196/55015 %U https://www.jmir.org/2025/1/e55015 %U https://doi.org/10.2196/55015 %U http://www.ncbi.nlm.nih.gov/pubmed/39778199 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 11 %N %P e53957 %T Trends in Prescription of Stimulants and Narcoleptic Drugs in Switzerland: Longitudinal Health Insurance Claims Analysis for the Years 2014-2021 %A Scharf,Tamara %A Huber,Carola A %A Näpflin,Markus %A Zhang,Zhongxing %A Khatami,Ramin %K prescription trends %K claims data %K cross-sectional data %K narcolepsy %K prescribers %K prescribing practices %K medical care %K stimulants %K stimulant medication %D 2025 %7 7.1.2025 %9 %J JMIR Public Health Surveill %G English %X Background: Stimulants are potent treatments for central hypersomnolence disorders or attention-deficit/hyperactivity disorders/attention deficit disorders but concerns have been raised about their potential negative consequences and their increasing prescription rates. Objective: We aimed to describe stimulant prescription trends in Switzerland from 2014 to 2021. Second, we aimed to analyze the characteristics of individuals who received stimulant prescriptions in 2021 and investigate the link between stimulant prescriptions and hospitalization rates in 2021, using hospitalization as a potential indicator of adverse health outcomes. Methods: Longitudinal and cross-sectional data from a large Swiss health care insurance were analyzed from all insureds older than 6 years. The results were extrapolated to the Swiss general population. We identified prescriptions for methylphenidate, lisdexamfetamine, modafinil, and sodium oxybate and calculated prevalences of each drug prescription over the period from 2014 to 2021. For 2021 we provide detailed information on the prescribers and evaluate the association of stimulant prescription and the number and duration of hospitalization using logistic regression models. Results: We observed increasing prescription rates of all stimulants in all age groups from 2014 to 2021 (0.55% to 0.81%, 43,848 to 66,113 insureds with a prescription). In 2021, 37.1% (28,057 prescriptions) of the medications were prescribed by psychiatrists, followed by 36.1% (n=27,323) prescribed by general practitioners and 1% (n=748) by neurologists. Only sodium oxybate, which is highly specific for narcolepsy treatment, was most frequently prescribed by neurologists (27.8%, 37 prescriptions). Comorbid psychiatric disorders were common in patients receiving stimulants. Patients hospitalized in a psychiatric institution were 5.3 times (odds ratio 5.3, 95% CI 4.63‐6.08, P<.001) more likely to have a stimulant prescription than those without hospitalization. There were no significant associations between stimulant prescription and the total length of inpatient stay (odds ratio 1, 95% CI 1‐1, P=.13). Conclusions: The prescription of stimulant medication in Switzerland increased slightly but continuously over years, but at lower rates compared to the estimated prevalence of central hypersomnolence disorders and attention-deficit/hyperactivity disorders/attention deficit disorders. Most stimulants are prescribed by psychiatrists, closely followed by general practitioners. The increased odds for hospitalization to psychiatric institutions for stimulant receivers reflects the severity of disease and the higher psychiatric comorbidities in these patients. %R 10.2196/53957 %U https://publichealth.jmir.org/2025/1/e53957 %U https://doi.org/10.2196/53957 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 9 %N %P e64936 %T Real-Time Analytics and AI for Managing No-Show Appointments in Primary Health Care in the United Arab Emirates: Before-and-After Study %A AlSerkal,Yousif Mohamed %A Ibrahim,Naseem Mohamed %A Alsereidi,Aisha Suhail %A Ibrahim,Mubaraka %A Kurakula,Sudheer %A Naqvi,Sadaf Ahsan %A Khan,Yasir %A Oottumadathil,Neema Preman %K electronic health record %K EHR %K artificial intelligence %K AI %K no-show appointments %K real-time data %K primary health care %K risk prediction %K clinic waiting time %K operational efficiency %D 2025 %7 6.1.2025 %9 %J JMIR Form Res %G English %X Background: Primary health care (PHC) services face operational challenges due to high patient volumes, leading to complex management needs. Patients access services through booked appointments and walk-in visits, with walk-in visits often facing longer waiting times. No-show appointments are significant contributors to inefficiency in PHC operations, which can lead to an estimated 3%-14% revenue loss, disrupt resource allocation, and negatively impact health care quality. Emirates Health Services (EHS) PHC centers handle over 140,000 visits monthly. Baseline data indicate a 21% no-show rate and an average patient wait time exceeding 16 minutes, necessitating an advanced scheduling and resource management system to enhance patient experiences and operational efficiency. Objective: The objective of this study was to evaluate the impact of an artificial intelligence (AI)-driven solution that was integrated with an interactive real-time data dashboard on reducing no-show appointments and improving patient waiting times at the EHS PHCs. Methods: This study introduced an innovative AI-based data application to enhance PHC efficiency. Leveraging our electronic health record system, we deployed an AI model with an 86% accuracy rate to predict no-shows by analyzing historical data and categorizing appointments based on no-show risk. The model was integrated with a real-time dashboard to monitor patient journeys and wait times. Clinic coordinators used the dashboard to proactively manage high-risk appointments and optimize resource allocation. The intervention was assessed through a before-and-after comparison of PHC appointment dynamics and wait times, analyzing data from 135,393 appointments (67,429 before implementation and 67,964 after implementation). Results: Implementation of the AI-powered no-show prediction model resulted in a significant 50.7% reduction in no-show rates (P<.001). The odds ratio for no-shows after implementation was 0.43 (95% CI 0.42-0.45; P<.001), indicating a 57% reduction in the likelihood of no-shows. Additionally, patient wait times decreased by an average of 5.7 minutes overall (P<.001), with some PHCs achieving up to a 50% reduction in wait times. Conclusions: This project demonstrates that integrating AI with a data analytics platform and an electronic health record systems can significantly improve operational efficiency and patient satisfaction in PHC settings. The AI model enabled daily assessments of wait times and allowed for real-time adjustments, such as reallocating patients to different clinicians, thus reducing wait times and optimizing resource use. These findings illustrate the transformative potential of AI and real-time data analytics in health care delivery. %R 10.2196/64936 %U https://formative.jmir.org/2025/1/e64936 %U https://doi.org/10.2196/64936 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e66356 %T Unpacking Performance Factors of Innovation Systems and Studying Germany’s Attempt to Foster the Role of the Patient Through a Market Access Pathway for Digital Health Applications (DiGAs): Exploratory Mixed Methods Study %A Gehder,Sara %A Goeldner,Moritz %+ Working Group for Data-Driven Innovation, Hamburg University of Technology, Am Schwarzenberg-Campus 4, Hamburg, 21073, Germany, 49 40 428784777, moritz.goeldner@tuhh.de %K regulatory market access pathways %K digital health application %K DiGA %K patient-relevant structural and procedural improvement %K pSVV pathway analysis %K qualitative and systemic analysis %K policy and stakeholder insights %K innovation system analysis %D 2025 %7 6.1.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Health care innovation faces significant challenges, including system inertia and diverse stakeholders, making regulated market access pathways essential for facilitating the adoption of new technologies. The German Digital Healthcare Act, introduced in 2019, offers a model by enabling digital health applications (DiGAs) to be reimbursed by statutory health insurance, improving market access and patient empowerment. However, the factors influencing the success of these pathways in driving innovation remain unclear. Objective: This study aims to identify the key performance factors of the innovation system shaped by the patient-relevant structural and procedural improvement (pSVV) pathway within the DiGA model. By examining how this pathway supports the entry of innovative digital health technologies, we seek to uncover the systemic dynamics that influence its effectiveness in fostering patient-centered digital health solutions. Methods: This study, conducted from May 2023 to November 2024, used a mixed methods approach. A descriptive analysis assessed how DiGA manufacturers use positive health care effects, giving a market overview of the pSVV technology. A qualitative analysis using grounded theory and Gioia methodology provided insights into stakeholder perspectives, focusing on manufacturers and regulatory bodies. A functional-structural analysis examined how components of the innovation system, such as actors, institutions, interactions, and infrastructure, interact and impact the effectiveness of the pathway. Results: The descriptive analysis showed that only 11 (20%) of the 56 DiGAs available in Germany used the pSVV pathway, with only 1 (2%) provisionally listed DiGA using pSVV as a primary end point; 6 of 9 (67%) pSVV key areas were used. The qualitative analysis revealed that manufacturers prioritize demonstrating medical benefits over pSVV due to evidence requirements and uncertainties around pSVV acceptance. Operational barriers hindered the adoption of pSVV, despite a positive reception among stakeholders. The systemic analysis identified key issues, including a lack of entrepreneurial focus on pSVV, limited regulatory experience, inadequate measurement methods, and entrenched practices prioritizing medical benefits, that hinder market formation and legitimacy. Conclusions: This study identifies key factors for effectively implementing innovation systems through regulated market access pathways, including content and format security, clearer framework specification, active innovation process management, and market formation stimulation. Addressing these factors can reduce uncertainties and promote wider adoption of digital health technologies. The findings highlight the need for future research on patient empowerment and the development of methodologies beyond traditional therapeutic outcomes. %M 39761562 %R 10.2196/66356 %U https://www.jmir.org/2025/1/e66356 %U https://doi.org/10.2196/66356 %U http://www.ncbi.nlm.nih.gov/pubmed/39761562 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e52786 %T Major Adverse Kidney Events in Hospitalized Older Patients With Acute Kidney Injury: Machine Learning–Based Model Development and Validation Study %A Luo,Xiao-Qin %A Zhang,Ning-Ya %A Deng,Ying-Hao %A Wang,Hong-Shen %A Kang,Yi-Xin %A Duan,Shao-Bin %+ Department of Nephrology, Hunan Key Laboratory of Kidney Disease and Blood Purification, The Second Xiangya Hospital of Central South University, 139 Renmin Road, Changsha, 410011, China, 86 73185295100, duansb528@csu.edu.cn %K major adverse kidney events within 30 days %K older %K acute kidney injury %K machine learning %K prediction model %D 2025 %7 3.1.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Acute kidney injury (AKI) is a common complication in hospitalized older patients, associated with increased morbidity, mortality, and health care costs. Major adverse kidney events within 30 days (MAKE30), a composite of death, new renal replacement therapy, or persistent renal dysfunction, has been recommended as a patient-centered endpoint for clinical trials involving AKI. Objective: This study aimed to develop and validate a machine learning–based model to predict MAKE30 in hospitalized older patients with AKI. Methods: A total of 4266 older patients (aged ≥ 65 years) with AKI admitted to the Second Xiangya Hospital of Central South University from January 1, 2015, to December 31, 2020, were included and randomly divided into a training set and an internal test set in a ratio of 7:3. An additional cohort of 11,864 eligible patients from the Medical Information Mart for Intensive Care Ⅳ database served as an external test set. The Boruta algorithm was used to select the most important predictor variables from 53 candidate variables. The eXtreme Gradient Boosting algorithm was applied to establish a prediction model for MAKE30. Model discrimination was evaluated by the area under the receiver operating characteristic curve (AUROC). The SHapley Additive exPlanations method was used to interpret model predictions. Results: The overall incidence of MAKE30 in the 2 study cohorts was 28.3% (95% CI 26.9%-29.7%) and 26.7% (95% CI 25.9%-27.5%), respectively. The prediction model for MAKE30 exhibited adequate predictive performance, with an AUROC of 0.868 (95% CI 0.852-0.881) in the training set and 0.823 (95% CI 0.798-0.846) in the internal test set. Its simplified version achieved an AUROC of 0.744 (95% CI 0.735-0.754) in the external test set. The SHapley Additive exPlanations method showed that the use of vasopressors, mechanical ventilation, blood urea nitrogen level, red blood cell distribution width-coefficient of variation, and serum albumin level were closely associated with MAKE30. Conclusions: An interpretable eXtreme Gradient Boosting model was developed and validated to predict MAKE30, which provides opportunities for risk stratification, clinical decision-making, and the conduct of clinical trials involving AKI. Trial Registration: Chinese Clinical Trial Registry ChiCTR2200061610; https://tinyurl.com/3smf9nuw %M 39752664 %R 10.2196/52786 %U https://www.jmir.org/2025/1/e52786 %U https://doi.org/10.2196/52786 %U http://www.ncbi.nlm.nih.gov/pubmed/39752664 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e62764 %T Spread and Scale of the Integrated Nutrition Pathway for Acute Care Across Canada: Protocol for the Advancing Malnutrition Care Program %A Ford,Katherine L %A Laur,Celia %A Dhaliwal,Rupinder %A Nasser,Roseann %A Gramlich,Leah %A Allard,Johane P %A Keller,Heather %A , %+ Department of Kinesiology and Health Sciences, University of Waterloo, 200 University Ave W, Waterloo, ON, N2L 3G1, Canada, 1 519 904 0660 ext 4205, heather.keller@uwaterloo.ca %K malnutrition %K nutrition screening, nutrition assessment %K hospital %K malnutrition care %K nutrition %K acute care %K clinicians %K mixed-methods design %K decision making %K mentor-champion model %K virtual training %K peer support %K virtual community of practice %D 2024 %7 31.12.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: A high proportion of patients admitted to hospital are at nutritional risk or have malnutrition. However, this risk is often not identified at admission, which may result in longer hospital stays and increased likelihood of death. The Integrated Nutrition Pathway for Acute Care (INPAC) was developed to provide clinicians with a standardized approach to prevent, detect, and treat malnutrition in hospital. Objective: The purpose of this study was to determine if the Advancing Malnutrition Care (AMC) program can be used to spread and scale-up improvements to nutrition care in Canadian hospitals. Methods: A prospective, longitudinal, mixed methods design is proposed to evaluate the spread and scale of INPAC best practices across Canadian hospitals using a mentor-champion model. Purposive and snowball sampling are used to recruit mentors and hospital champions to participate in the AMC program. Mentors are persons with experience improving nutrition care in a clinical setting and champions are health care providers with a commitment to implementing best care practices. Mentors and champions are trained digitally on their roles and activities. Mentors meet with champions in their area monthly to support them with making practice change. Champions created a site implementation team to target practice change in a specific area related to malnutrition care and use AMC program-specific tools and resources to implement improvements and collect site information through quarterly audits of patient charts to track implementation of nutrition care best practices. An online community of practice is held every 3-4 months to provide further implementation resources and foster connection between mentors and champions at a national level. A prospective evaluation will be conducted to assess the impact of the program and explore how it can be sustainably spread and scaled across Canada. Semistructured interviews will be used to gain a deeper understanding of mentor and champion experiences in the program. The capabilities, opportunities, and motivations of behavior model will be used to evaluate behavior change and the Kirkpatrick 4-level framework will facilitate assessment of barriers to change. Aggregated chart audits will assess the impact of implemented care practices. Descriptive analyses will be used to describe baseline mentor and champion and hospital characteristics and mentor and champion experiences; Friedman test will describe these changes over time. Directed content analysis will guide interpretation of interview data. Results: Data collection began in September 2022 and is anticipated to end in June 2025, at which time data analysis will begin. Conclusions: Evaluation of the AMC program will strengthen decision-making, future programming, and will inform program changes that reflect implementation of best practices in nutrition care while supporting regional mentors and hospital champions. This work will address the sustainability of AMC and the critical challenges related to hospital-based malnutrition, ultimately improving nutrition care for patients across Canada. International Registered Report Identifier (IRRID): DERR1-10.2196/62764 %M 39740211 %R 10.2196/62764 %U https://www.researchprotocols.org/2024/1/e62764 %U https://doi.org/10.2196/62764 %U http://www.ncbi.nlm.nih.gov/pubmed/39740211 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e56382 %T Leveraging Machine Learning to Identify Subgroups of Misclassified Patients in the Emergency Department: Multicenter Proof-of-Concept Study %A Wyatt,Sage %A Lunde Markussen,Dagfinn %A Haizoune,Mounir %A Vestbø,Anders Strand %A Sima,Yeneabeba Tilahun %A Sandboe,Maria Ilene %A Landschulze,Marcus %A Bartsch,Hauke %A Sauer,Christopher Martin %+ Institute for Artificial Intelligence in Medicine, University Hospital Essen, Girardetstraße 2, Essen, 45131, Germany, 49 201 723 0, sauerc@mit.edu %K emergency department %K triage %K machine learning %K real world evidence %K random forest %K classification %K subgroup %K misclassification %K patient %K multi-center %K proof-of-concept %K hospital %K clinical feature %K Norway %K retrospective %K cohort study %K electronic health system %K electronic health record %D 2024 %7 31.12.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Hospitals use triage systems to prioritize the needs of patients within available resources. Misclassification of a patient can lead to either adverse outcomes in a patient who did not receive appropriate care in the case of undertriage or a waste of hospital resources in the case of overtriage. Recent advances in machine learning algorithms allow for the quantification of variables important to under- and overtriage. Objective: This study aimed to identify clinical features most strongly associated with triage misclassification using a machine learning classification model to capture nonlinear relationships. Methods: Multicenter retrospective cohort data from 2 big regional hospitals in Norway were extracted. The South African Triage System is used at Bergen University Hospital, and the Rapid Emergency Triage and Treatment System is used at Trondheim University Hospital. Variables included triage score, age, sex, arrival time, subject area affiliation, reason for emergency department contact, discharge location, level of care, and time of death were retrieved. Random forest classification models were used to identify features with the strongest association with overtriage and undertriage in clinical practice in Bergen and Trondheim. We reported variable importance as SHAP (SHapley Additive exPlanations)-values. Results: We collected data on 205,488 patient records from Bergen University Hospital and 304,997 patient records from Trondheim University Hospital. Overall, overtriage was very uncommon at both hospitals (all <0.1%), with undertriage differing between both locations, with 0.8% at Bergen and 0.2% at Trondheim University Hospital. Demographics were similar for both hospitals. However, the percentage given a high-priority triage score (red or orange) was higher in Bergen (24%) compared with 9% in Trondheim. The clinical referral department was found to be the variable with the strongest association with undertriage (mean SHAP +0.62 and +0.37 for Bergen and Trondheim, respectively). Conclusions: We identified subgroups of patients consistently undertriaged using 2 common triage systems. While the importance of clinical patient characteristics to triage misclassification varies by triage system and location, we found consistent evidence between the two locations that the clinical referral department is the most important variable associated with triage misclassification. Replication of this approach at other centers could help to further improve triage scoring systems and improve patient care worldwide. %M 39451101 %R 10.2196/56382 %U https://www.jmir.org/2024/1/e56382 %U https://doi.org/10.2196/56382 %U http://www.ncbi.nlm.nih.gov/pubmed/39451101 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e57824 %T Applying AI to Structured Real-World Data for Pharmacovigilance Purposes: Scoping Review %A Dimitsaki,Stella %A Natsiavas,Pantelis %A Jaulent,Marie-Christine %+ Laboratoire d'Informatique Médicale et d'Ingénierie des Connaissances en e-Santé - LIMICS, Inserm, Université Sorbonne Paris-Nord, Sorbonne Université, 15 Rue de l'École de Médecine, Paris, 75006, France, 33 767968072, Stella.Dimitsaki@etu.sorbonne-universite.fr %K pharmacovigilance %K drug safety %K artificial intelligence %K machine learning %K real-world data %K scoping review %D 2024 %7 30.12.2024 %9 Review %J J Med Internet Res %G English %X Background: Artificial intelligence (AI) applied to real-world data (RWD; eg, electronic health care records) has been identified as a potentially promising technical paradigm for the pharmacovigilance field. There are several instances of AI approaches applied to RWD; however, most studies focus on unstructured RWD (conducting natural language processing on various data sources, eg, clinical notes, social media, and blogs). Hence, it is essential to investigate how AI is currently applied to structured RWD in pharmacovigilance and how new approaches could enrich the existing methodology. Objective: This scoping review depicts the emerging use of AI on structured RWD for pharmacovigilance purposes to identify relevant trends and potential research gaps. Methods: The scoping review methodology is based on the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology. We queried the MEDLINE database through the PubMed search engine. Relevant scientific manuscripts published from January 2010 to January 2024 were retrieved. The included studies were “mapped” against a set of evaluation criteria, including applied AI approaches, code availability, description of the data preprocessing pipeline, clinical validation of AI models, and implementation of trustworthy AI criteria following the guidelines of the FUTURE (Fairness, Universality, Traceability, Usability, Robustness, and Explainability)-AI initiative. Results: The scoping review ultimately yielded 36 studies. There has been a significant increase in relevant studies after 2019. Most of the articles focused on adverse drug reaction detection procedures (23/36, 64%) for specific adverse effects. Furthermore, a substantial number of studies (34/36, 94%) used nonsymbolic AI approaches, emphasizing classification tasks. Random forest was the most popular machine learning approach identified in this review (17/36, 47%). The most common RWD sources used were electronic health care records (28/36, 78%). Typically, these data were not available in a widely acknowledged data model to facilitate interoperability, and they came from proprietary databases, limiting their availability for reproducing results. On the basis of the evaluation criteria classification, 10% (4/36) of the studies published their code in public registries, 16% (6/36) tested their AI models in clinical environments, and 36% (13/36) provided information about the data preprocessing pipeline. In addition, in terms of trustworthy AI, 89% (32/36) of the studies followed at least half of the trustworthy AI initiative guidelines. Finally, selection and confounding biases were the most common biases in the included studies. Conclusions: AI, along with structured RWD, constitutes a promising line of work for drug safety and pharmacovigilance. However, in terms of AI, some approaches have not been examined extensively in this field (such as explainable AI and causal AI). Moreover, it would be helpful to have a data preprocessing protocol for RWD to support pharmacovigilance processes. Finally, because of personal data sensitivity, evaluation procedures have to be investigated further. %M 39753222 %R 10.2196/57824 %U https://www.jmir.org/2024/1/e57824 %U https://doi.org/10.2196/57824 %U http://www.ncbi.nlm.nih.gov/pubmed/39753222 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e67056 %T Automated Pathologic TN Classification Prediction and Rationale Generation From Lung Cancer Surgical Pathology Reports Using a Large Language Model Fine-Tuned With Chain-of-Thought: Algorithm Development and Validation Study %A Kim,Sanghwan %A Jang,Sowon %A Kim,Borham %A Sunwoo,Leonard %A Kim,Seok %A Chung,Jin-Haeng %A Nam,Sejin %A Cho,Hyeongmin %A Lee,Donghyoung %A Lee,Keehyuck %A Yoo,Sooyoung %+ Office of eHealth Research and Business, Seoul National University Bundang Hospital, Healthcare Innovation Park, Seongnam, 13605, Republic of Korea, 82 317878980, yoosoo0@snubh.org %K AJCC Cancer Staging Manual 8th edition %K American Joint Committee on Cancer %K large language model %K chain-of-thought %K rationale %K lung cancer %K report analysis %K AI %K surgery %K pathology reports %K tertiary hospital %K generative language models %K efficiency %K accuracy %K automated %D 2024 %7 20.12.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Traditional rule-based natural language processing approaches in electronic health record systems are effective but are often time-consuming and prone to errors when handling unstructured data. This is primarily due to the substantial manual effort required to parse and extract information from diverse types of documentation. Recent advancements in large language model (LLM) technology have made it possible to automatically interpret medical context and support pathologic staging. However, existing LLMs encounter challenges in rapidly adapting to specialized guideline updates. In this study, we fine-tuned an LLM specifically for lung cancer pathologic staging, enabling it to incorporate the latest guidelines for pathologic TN classification. Objective: This study aims to evaluate the performance of fine-tuned generative language models in automatically inferring pathologic TN classifications and extracting their rationale from lung cancer surgical pathology reports. By addressing the inefficiencies and extensive parsing efforts associated with rule-based methods, this approach seeks to enable rapid and accurate reclassification aligned with the latest cancer staging guidelines. Methods: We conducted a comparative performance evaluation of 6 open-source LLMs for automated TN classification and rationale generation, using 3216 deidentified lung cancer surgical pathology reports based on the American Joint Committee on Cancer (AJCC) Cancer Staging Manual8th edition, collected from a tertiary hospital. The dataset was preprocessed by segmenting each report according to lesion location and morphological diagnosis. Performance was assessed using exact match ratio (EMR) and semantic match ratio (SMR) as evaluation metrics, which measure classification accuracy and the contextual alignment of the generated rationales, respectively. Results: Among the 6 models, the Orca2_13b model achieved the highest performance with an EMR of 0.934 and an SMR of 0.864. The Orca2_7b model also demonstrated strong performance, recording an EMR of 0.914 and an SMR of 0.854. In contrast, the Llama2_7b model achieved an EMR of 0.864 and an SMR of 0.771, while the Llama2_13b model showed an EMR of 0.762 and an SMR of 0.690. The Mistral_7b and Llama3_8b models, on the other hand, showed lower performance, with EMRs of 0.572 and 0.489, and SMRs of 0.377 and 0.456, respectively. Overall, the Orca2 models consistently outperformed the others in both TN stage classification and rationale generation. Conclusions: The generative language model approach presented in this study has the potential to enhance and automate TN classification in complex cancer staging, supporting both clinical practice and oncology data curation. With additional fine-tuning based on cancer-specific guidelines, this approach can be effectively adapted to other cancer types. %M 39705675 %R 10.2196/67056 %U https://medinform.jmir.org/2024/1/e67056 %U https://doi.org/10.2196/67056 %U http://www.ncbi.nlm.nih.gov/pubmed/39705675 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e51493 %T Strengthening Cause of Death Statistics in Selected Districts of 3 States in India: Protocol for an Uncontrolled, Before-After, Mixed Method Study %A Grover,Ashoo %A Nair,Saritha %A Sharma,Saurabh %A Gupta,Shefali %A Shrivastava,Suyesh %A Singh,Pushpendra %A Kanungo,Srikanta %A Ovung,Senthanro %A Singh,Charan %A Khan,Abdul Mabood %A Sharma,Sandeep %A Palo,Subrata Kumar %A Chakma,Tapas %A Bajaj,Anjali %+ Indian Council of Medical Research, PO Box 491, Ansari Nagar, New Delhi, 110029, India, 91 9871087189, nairs@icmr.gov.in %K cause of death %K Medical Certification of Cause of Death %K capacity building %K Civil Registration and Vital Statistics %K training %D 2024 %7 20.12.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Mortality statistics are vital for health policy development, epidemiological research, and health care service planning. A robust surveillance system is essential for obtaining vital information such as cause of death (CoD) information. Objective: This study aims to develop a comprehensive model to strengthen the CoD information in the selected study sites. The specific objectives are (1) to identify the best practices and challenges in the functioning of the Civil Registration and Vital Statistics (CRVS) system with respect to mortality statistics and CoD information; (2) to develop and implement interventions to strengthen the CoD information; (3) to evaluate the quality improvement of the Medical Certification of Cause of Death (MCCD); and (4) to improve the CoD information at the population level through verbal autopsy for noninstitutional deaths in the selected study sites. Methods: An uncontrolled, before-after, mixed method study will be conducted in 3 blocks located in the districts of 3 states (Madhya Pradesh, Uttar Pradesh, and Odisha) in India. A baseline assessment to identify the best practices and challenges in the functioning of the CRVS system, along with a quality assessment of the MCCD, will be conducted. An intervention informed by existing literature and the baseline assessment will be developed and implemented in the study sites. The major components of intervention will include a Training of Trainers workshop, orientation of stakeholders in the functioning of the CRVS system, training of physicians and medical officers in the MCCD, and training of community health workers in World Health Organization Verbal Autopsy 2022 instrument. Postintervention evaluation will be carried out to assess the impact made by the intervention on the availability and quality improvement of CoD information in the selected study sites. The outcome will be measured in terms of the quality improvement of the MCCD and the availability of CoD information at population level through verbal autopsy in the selected study sites. Results: The project has been funded, and regulatory approval has been obtained from the Institutional Ethics Committee. The data collection process began in May 2023. The duration of the study will be for 24 months. Conclusions: Our study is expected to provide a valuable contribution toward strengthening CoD information, which could be helpful for policy making and further research. The intervention model will be developed in collaboration with the existing functionaries of the health and CRVS systems in the selected study sites that are engaged in reporting and recording CoD information; this will ensure sustainability and provide lessons for upscaling, with the aim to improve the reporting of CoD information in the country. International Registered Report Identifier (IRRID): DERR1-10.2196/51493 %M 39705697 %R 10.2196/51493 %U https://www.researchprotocols.org/2024/1/e51493 %U https://doi.org/10.2196/51493 %U http://www.ncbi.nlm.nih.gov/pubmed/39705697 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e60665 %T An Automatic and End-to-End System for Rare Disease Knowledge Graph Construction Based on Ontology-Enhanced Large Language Models: Development Study %A Cao,Lang %A Sun,Jimeng %A Cross,Adam %K rare disease %K clinical informatics %K LLM %K natural language processing %K machine learning %K artificial intelligence %K large language models %K data extraction %K ontologies %K knowledge graphs %K text mining %D 2024 %7 18.12.2024 %9 %J JMIR Med Inform %G English %X Background: Rare diseases affect millions worldwide but sometimes face limited research focus individually due to low prevalence. Many rare diseases do not have specific International Classification of Diseases, Ninth Edition (ICD-9) and Tenth Edition (ICD-10), codes and therefore cannot be reliably extracted from granular fields like “Diagnosis” and “Problem List” entries, which complicates tasks that require identification of patients with these conditions, including clinical trial recruitment and research efforts. Recent advancements in large language models (LLMs) have shown promise in automating the extraction of medical information, offering the potential to improve medical research, diagnosis, and management. However, most LLMs lack professional medical knowledge, especially concerning specific rare diseases, and cannot effectively manage rare disease data in its various ontological forms, making it unsuitable for these tasks. Objective: Our aim is to create an end-to-end system called automated rare disease mining (AutoRD), which automates the extraction of rare disease–related information from medical text, focusing on entities and their relations to other medical concepts, such as signs and symptoms. AutoRD integrates up-to-date ontologies with other structured knowledge and demonstrates superior performance in rare disease extraction tasks. We conducted various experiments to evaluate AutoRD’s performance, aiming to surpass common LLMs and traditional methods. Methods: AutoRD is a pipeline system that involves data preprocessing, entity extraction, relation extraction, entity calibration, and knowledge graph construction. We implemented this system using GPT-4 and medical knowledge graphs developed from the open-source Human Phenotype and Orphanet ontologies, using techniques such as chain-of-thought reasoning and prompt engineering. We quantitatively evaluated our system’s performance in entity extraction, relation extraction, and knowledge graph construction. The experiment used the well-curated dataset RareDis2023, which contains medical literature focused on rare disease entities and their relations, making it an ideal dataset for training and testing our methodology. Results: On the RareDis2023 dataset, AutoRD achieved an overall entity extraction F1-score of 56.1% and a relation extraction F1-score of 38.6%, marking a 14.4% improvement over the baseline LLM. Notably, the F1-score for rare disease entity extraction reached 83.5%, indicating high precision and recall in identifying rare disease mentions. These results demonstrate the effectiveness of integrating LLMs with medical ontologies in extracting complex rare disease information. Conclusions: AutoRD is an automated end-to-end system for extracting rare disease information from text to build knowledge graphs, addressing critical limitations of existing LLMs by improving identification of these diseases and connecting them to related clinical features. This work underscores the significant potential of LLMs in transforming health care, particularly in the rare disease domain. By leveraging ontology-enhanced LLMs, AutoRD constructs a robust medical knowledge base that incorporates up-to-date rare disease information, facilitating improved identification of patients and resulting in more inclusive research and trial candidacy efforts. %R 10.2196/60665 %U https://medinform.jmir.org/2024/1/e60665 %U https://doi.org/10.2196/60665 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e60879 %T A Digital Approach for Addressing Suicidal Ideation and Behaviors in Youth Mental Health Services: Observational Study %A Chong,Min K %A Hickie,Ian B %A Ottavio,Antonia %A Rogers,David %A Dimitropoulos,Gina %A LaMonica,Haley M %A Borgnolo,Luke J %A McKenna,Sarah %A Scott,Elizabeth M %A Iorfino,Frank %+ Brain and Mind Centre, The University of Sydney, 94 Mallett Street, Camperdown, 2050, Australia, 61 (02) 9114 2199, min.chong@sydney.edu.au %K mental health service %K youth mental health %K suicide management %K clinical decision support %K primary care %K personalization %K suicide %K suicidal %K youth %K mental health %K mental health care %K suicide prevention %K digital technology %K online assessment %K clinician %K digital health %K health informatics %K clinical information %D 2024 %7 18.12.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Long wait times for mental health treatments may cause delays in early detection and management of suicidal ideation and behaviors, which are crucial for effective mental health care and suicide prevention. The use of digital technology is a potential solution for prompt identification of youth with high suicidality. Objective: The primary aim of this study was to evaluate the use of a digital suicidality notification system designed to detect and respond to suicidal needs in youth mental health services. Second, the study aimed to characterize young people at different levels of suicidal ideation and behaviors. Methods: Young people aged between 16 and 25 years completed multidimensional assessments using a digital platform, collecting demographic, clinical, social, functional, and suicidality information. When the suicidality score exceeded a predetermined threshold, established based on clinical expertise and service policies, a rule-based algorithm configured within the platform immediately generated an alert for treating clinicians. Subsequent clinical actions and response times were analyzed. Results: A total of 2021 individuals participated, of whom 266 (11%) triggered one or more high suicidal ideation and behaviors notification. Of the 292 notifications generated, 76% (222/292) were resolved, with a median response time of 1.9 (range 0-50.8) days. Clinical actions initiated to address suicidality included creating safety plans (60%, 134/222), conducting safety checks (18%, 39/222), psychological therapy (8%, 17/222), transfer to another service (3%, 8/222), and scheduling of new appointments (2%, 4/222). Young people with high levels of suicidality were more likely to present with more severe and comorbid symptoms, including low engagement in work or education, heterogenous psychopathology, substance misuse, and recurrent illness. Conclusions: The digital suicidality notification system facilitated prompt clinical actions by alerting clinicians to high levels of suicidal ideation and behaviors detected among youth. Further, the multidimensional assessment revealed complex and comorbid symptoms exhibited in youth with high suicidality. By expediting and personalizing care for those displaying elevated suicidality, the digital notification system can play a pivotal role in preventing rapid symptom progression and its detrimental impacts on young people’s mental health. %M 39693140 %R 10.2196/60879 %U https://www.jmir.org/2024/1/e60879 %U https://doi.org/10.2196/60879 %U http://www.ncbi.nlm.nih.gov/pubmed/39693140 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e55982 %T Insights and Trends in Open Note Access: Retrospective Observational Study %A Badwal,Randeep Singh %A Cavo,Paul %A Panesar,Mandip %+ Erie County Medical Center, 462 Grider Street, Buffalo, NY, 14215, United States, 1 716 898 1417, MPanesar@ecmc.edu %K open note trends %K open notes %K open note access %K open note use, open note sex %K open note specialty %K clinical note views %K patient portal %K patients %K trends %K hospitals %K engagement %K retrospective observational study %K outpatient %K assessment %K older patients %K adults %K pandemic %K COVID-19 %D 2024 %7 17.12.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: As of 2021, at least 4 out of every 5 hospitals offered patients access to clinical notes via a web-based patient portal, a number that is expected to grow because of the 21st Century Cures Act. There is limited data on how open note use may have evolved over time or which types of clinical interactions were viewed most in the outpatient setting. Objective: This study aims to analyze trends in outpatient open note access over time; characterize usage in terms of age, sex, and clinical interaction type; and assess the method of access to help uncover areas of improvement in patient engagement and identify further areas of research. Methods: A retrospective observational study was conducted at Erie County Medical Center from November 1, 2021, to December 31, 2022, to coincide with the time that open notes went live. Outpatient note access and account logs were downloaded from the portal and combined into a single dataset consisting of 18,384 note accesses by 4615 users, with column headings of the patient index, sex, age, note title that was accessed, clinical interaction type, time stamp of note creation, time stamp of access, and method of access (web vs mobile). A separate table was created with sex data for all 35,273 portal accounts. Microsoft Excel and Microsoft Power Query were used to combine and analyze the data. Results: During the study period, 4615 portal users viewed 12,150 documents for a total of 18,384 times, averaging 2.6 notes per patient viewed 4 times. Only 13.1% (4615/35,273) of all portal inpatient and outpatient registrants viewed their outpatient notes. There was a female predominance in those who viewed notes (2926/4615, 63.4%; P<.001), while 56.8% (20,047/35,273) of all portal registrants were female. Users in their 30s and 50s accessed more notes than other age groups. The ratio of mobile-to-web access of notes tended to decrease as a function of increasing age, which was not observed in those aged ≥90 years. Notes regarding COVID-19 assessments were the most accessed among all clinical interactions (4725/12,150, 38.9%). Overall, the number of users accessing notes reached a maximum of 1968 before declining to 1027 by the end of the study period. Conclusions: Open note access was largely dominated by COVID-19 assessments, and the number of users viewing their notes has declined over time as the pandemic subsided. Furthermore, female patients and those aged in their 30s as well as 50s viewed more notes than other groups. Finally, the percentage of notes viewed via a mobile device tended to decrease as a function of increasing age, showing that web-based access of open notes is an important modality for older patients. %M 39689311 %R 10.2196/55982 %U https://www.jmir.org/2024/1/e55982 %U https://doi.org/10.2196/55982 %U http://www.ncbi.nlm.nih.gov/pubmed/39689311 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e65626 %T Appropriately Matching Transport Care Units to Patients in Interhospital Transport Care: Implementation Study %A Hasavari,Shirin %A Esmaeilzadeh,Pouyan %+ Department of Information Science & Systems, Graves School of Business & Management, Morgan State University, 21251, 4100 Hillen Rd, Baltimore, MD, 21218, United States, 1 3015090562, shirin.hasavari@morgan.edu %K interfacility transport care %K electronic health records %K data sharing %K blockchain %K hyperledger fabric %K privacy %K implementation %K EMS %K emergency medical services %D 2024 %7 13.12.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: In interfacility transport care, a critical challenge exists in accurately matching ambulance response levels to patients’ needs, often hindered by limited access to essential patient data at the time of transport requests. Existing systems cannot integrate patient data from sending hospitals’ electronic health records (EHRs) into the transfer request process, primarily due to privacy concerns, interoperability challenges, and the sensitive nature of EHR data. We introduce a distributed digital health platform, Interfacility Transport Care (ITC)–InfoChain, designed to solve this problem without compromising EHR security or data privacy. Objective: This study aimed to detail the implementation of ITC-InfoChain, a secure, blockchain-based platform designed to enhance real-time data sharing without compromising data privacy or EHR security. Methods: The ITC-InfoChain platform prototype was implemented on Amazon Web Services cloud infrastructure, using Hyperledger Fabric as a permissioned blockchain. Key elements included participant registration, identity management, and patient data collection isolated from the sending hospital’s EHR system. The client program submits encrypted patient data to a distributed ledger, accessible to the receiving facility’s critical care unit at the time of transport request and emergency medical services (EMS) teams during transport through the PatienTrack web app. Performance was evaluated through key performance indicators such as data transaction times and scalability across transaction loads. Results: The ITC-InfoChain demonstrated strong performance and scalability. Data transaction times averaged 3.1 seconds for smaller volumes (1-20 transactions) and 6.4 seconds for 100 transactions. Optimized configurations improved processing times to 1.8-1.9 seconds for 400 transactions. These results confirm the platform’s capacity to handle high transaction volumes, supporting timely, real-time data access for decision-making during transport requests and patient transfers. Conclusions: The ITC-InfoChain platform addresses the challenge of matching appropriate transport units to patient needs by ensuring data privacy, integrity, and real-time data sharing, enhancing the coordination of patient care. The platform’s success suggests potential for regional pilots and broader adoption in secure health care systems. Stakeholder resistance due to blockchain unfamiliarity and data privacy concerns remains. Funding has been sought to support a pilot program to address these challenges through targeted education and engagement. %M 39540868 %R 10.2196/65626 %U https://formative.jmir.org/2024/1/e65626 %U https://doi.org/10.2196/65626 %U http://www.ncbi.nlm.nih.gov/pubmed/39540868 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e51409 %T Longitudinal Model Shifts of Machine Learning–Based Clinical Risk Prediction Models: Evaluation Study of Multiple Use Cases Across Different Hospitals %A Cabanillas Silva,Patricia %A Sun,Hong %A Rezk,Mohamed %A Roccaro-Waldmeyer,Diana M %A Fliegenschmidt,Janis %A Hulde,Nikolai %A von Dossow,Vera %A Meesseman,Laurent %A Depraetere,Kristof %A Stieg,Joerg %A Szymanowsky,Ralph %A Dahlweid,Fried-Michael %+ Dedalus HealthCare, Roderveldlaan 2, Antwerp, 2600, Belgium, 32 0784244010, mohamed.rezk@dedalus.com %K model shift %K model monitoring %K prediction models %K acute kidney injury %K AKI %K sepsis %K delirium %K decision curve analysis %K DCA %D 2024 %7 13.12.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: In recent years, machine learning (ML)–based models have been widely used in clinical domains to predict clinical risk events. However, in production, the performances of such models heavily rely on changes in the system and data. The dynamic nature of the system environment, characterized by continuous changes, has significant implications for prediction models, leading to performance degradation and reduced clinical efficacy. Thus, monitoring model shifts and evaluating their impact on prediction models are of utmost importance. Objective: This study aimed to assess the impact of a model shift on ML-based prediction models by evaluating 3 different use cases—delirium, sepsis, and acute kidney injury (AKI)—from 2 hospitals (M and H) with different patient populations and investigate potential model deterioration during the COVID-19 pandemic period. Methods: We trained prediction models using retrospective data from earlier years and examined the presence of a model shift using data from more recent years. We used the area under the receiver operating characteristic curve (AUROC) to evaluate model performance and analyzed the calibration curves over time. We also assessed the influence on clinical decisions by evaluating the alert rate, the rates of over- and underdiagnosis, and the decision curve. Results: The 2 data sets used in this study contained 189,775 and 180,976 medical cases for hospitals M and H, respectively. Statistical analyses (Z test) revealed no significant difference (P>.05) between the AUROCs from the different years for all use cases and hospitals. For example, in hospital M, AKI did not show a significant difference between 2020 (AUROC=0.898) and 2021 (AUROC=0.907, Z=–1.171, P=.242). Similar results were observed in both hospitals and for all use cases (sepsis and delirium) when comparing all the different years. However, when evaluating the calibration curves at the 2 hospitals, model shifts were observed for the delirium and sepsis use cases but not for AKI. Additionally, to investigate the clinical utility of our models, we performed decision curve analysis (DCA) and compared the results across the different years. A pairwise nonparametric statistical comparison showed no differences in the net benefit at the probability thresholds of interest (P>.05). The comprehensive evaluations performed in this study ensured robust model performance of all the investigated models across the years. Moreover, neither performance deteriorations nor alert surges were observed during the COVID-19 pandemic period. Conclusions: Clinical risk prediction models were affected by the dynamic and continuous evolution of clinical practices and workflows. The performance of the models evaluated in this study appeared stable when assessed using AUROCs, showing no significant variations over the years. Additional model shift investigations suggested that a calibration shift was present for certain use cases (delirium and sepsis). However, these changes did not have any impact on the clinical utility of the models based on DCA. Consequently, it is crucial to closely monitor data changes and detect possible model shifts, along with their potential influence on clinical decision-making. %M 39671571 %R 10.2196/51409 %U https://www.jmir.org/2024/1/e51409 %U https://doi.org/10.2196/51409 %U http://www.ncbi.nlm.nih.gov/pubmed/39671571 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e63289 %T Survival After Radical Cystectomy for Bladder Cancer: Development of a Fair Machine Learning Model %A Carbunaru,Samuel %A Neshatvar,Yassamin %A Do,Hyungrok %A Murray,Katie %A Ranganath,Rajesh %A Nayan,Madhur %+ Department of Urology, New York University School of Medicine, 550 1st Avenue, New York, NY, 10016, United States, 1 (646) 825 6300, samuel.carbunaru@nyulangone.org %K machine learning %K bladder cancer %K survival %K prediction %K model %K bias %K fairness %K radical cystectomy %K mortality rate %K algorithmic fairness %K health equity %K healthcare disparities %D 2024 %7 13.12.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Prediction models based on machine learning (ML) methods are being increasingly developed and adopted in health care. However, these models may be prone to bias and considered unfair if they demonstrate variable performance in population subgroups. An unfair model is of particular concern in bladder cancer, where disparities have been identified in sex and racial subgroups. Objective: This study aims (1) to develop a ML model to predict survival after radical cystectomy for bladder cancer and evaluate for potential model bias in sex and racial subgroups; and (2) to compare algorithm unfairness mitigation techniques to improve model fairness. Methods: We trained and compared various ML classification algorithms to predict 5-year survival after radical cystectomy using the National Cancer Database. The primary model performance metric was the F1-score. The primary metric for model fairness was the equalized odds ratio (eOR). We compared 3 algorithm unfairness mitigation techniques to improve eOR. Results: We identified 16,481 patients; 23.1% (n=3800) were female, and 91.5% (n=15,080) were “White,” 5% (n=832) were “Black,” 2.3% (n=373) were “Hispanic,” and 1.2% (n=196) were “Asian.” The 5-year mortality rate was 75% (n=12,290). The best naive model was extreme gradient boosting (XGBoost), which had an F1-score of 0.860 and eOR of 0.619. All unfairness mitigation techniques increased the eOR, with correlation remover showing the highest increase and resulting in a final eOR of 0.750. This mitigated model had F1-scores of 0.86, 0.904, and 0.824 in the full, Black male, and Asian female test sets, respectively. Conclusions: The ML model predicting survival after radical cystectomy exhibited bias across sex and racial subgroups. By using algorithm unfairness mitigation techniques, we improved algorithmic fairness as measured by the eOR. Our study highlights the role of not only evaluating for model bias but also actively mitigating such disparities to ensure equitable health care delivery. We also deployed the first web-based fair ML model for predicting survival after radical cystectomy. %R 10.2196/63289 %U https://medinform.jmir.org/2024/1/e63289 %U https://doi.org/10.2196/63289 %0 Journal Article %@ 2561-7605 %I JMIR Publications %V 7 %N %P e59234 %T Factors Influencing Drug Prescribing for Patients With Hospitalization History in Circulatory Disease–Patient Severity, Composite Adherence, and Physician-Patient Relationship: Retrospective Cohort Study %A Takura,Tomoyuki %A Yokoi,Hiroyoshi %A Honda,Asao %+ Department of Health Care Services Management, Nihon University School of Medicine, 30-1 Oyaguchi Kamicho, Itabashi-ku, Tokyo, 113-8655, Japan, 81 03 3972 8111 ext 2282, takura.tomoyuki@nihon-u.ac.jp %K medication adherence %K drug prescription switch %K generic drug %K logistic model %K long-term longitudinal study %K patient severity %K systolic blood pressure %K serum creatinine %K aging %K big data %D 2024 %7 6.12.2024 %9 Original Paper %J JMIR Aging %G English %X Background: With countries promoting generic drug prescribing, their growth may plateau, warranting further investigation into the factors influencing this trend, including physician and patient perspectives. Additional strategies may be needed to maximize the switch to generic drugs while ensuring health care system sustainability, focusing on factors beyond mere low cost. Emphasizing affordability and clarifying other prescription considerations are essential. Objective: This study aimed to provide initial insights into how patient severity, composite adherence, and physician-patient relationships impact generic switching. Methods: This study used a long-term retrospective cohort design by analyzing data from a national health care database. The population included patients of all ages, primarily older adults, who required primary-to-tertiary preventive actions with a history of hospitalization for cardiovascular diseases (ICD-10 [International Statistical Classification of Diseases, Tenth Revision]) from April 2014 to March 2018 (4 years). We focused on switching to generic drugs, with temporal variations in clinical parameters as independent variables. Lifestyle factors (smoking and drinking) were also considered. Adherence was measured as a composite score comprising 11 elements. The physician-patient relationship was established based on the interval between physician change and prescription. Logistic regression analysis and propensity score matching were used, along with complementary analysis of physician-patient relationships, proportion of days covered, and adherence for a subset of the population. Results: The study included 48,456 patients with an average follow-up of 36.1 (SD 8.8) months. The mean age was 68.3 (SD 9.9) years; BMI, 23.4 (SD 3.4) kg/m2; systolic blood pressure, 131.2 (SD 15) mm Hg; low-density lipoprotein cholesterol level, 116.6 (SD 29.3) mg/dL; hemoglobin A1c (HbA1c), 5.9% (SD 0.8%); and serum creatinine level, 0.9 (SD 0.8) mg/dL. Logistic regression analysis revealed significant associations between generic switching and systolic blood pressure (odds ratio [OR] 0.996, 95% CI 0.993-0.999), serum creatinine levels (OR 0.837, 95% CI 0.729-0.962), glutamic oxaloacetic transaminase levels (OR 0.994, 95% CI 0.990-0.997), proportion of days covered score (OR 0.959, 95% CI 0.948-0.97), and adherence score (OR 0.910, 95% CI 0.875-0.947). In addition, generic drug rates increased with improvements in the HbA1c level band and smoking level (P<.01 and P<.001). The group with a superior physician-patient relationship after propensity score matching had a significantly higher rate of generic drug prescribing (51.6%, SD 15.2%) than the inferior relationship group (47.7%, SD17.7%; P<.001). Conclusions: Although physicians’ understanding influences the choice of generic drugs, patient condition (severity) and adherence also impact this decision. For example, improved creatinine levels are associated with generic drug choice, while stronger physician-patient relationships correlate with higher rates of generic drug use. These findings may contribute to the appropriate prescription of pharmaceuticals if the policy diffusion of generic drugs begins to slow down. Thus, preventing serious illness while building trust may result in clinical benefits and positive socioeconomic outcomes. %M 39421979 %R 10.2196/59234 %U https://aging.jmir.org/2024/1/e59234 %U https://doi.org/10.2196/59234 %U http://www.ncbi.nlm.nih.gov/pubmed/39421979 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e57929 %T Evaluation Methods, Indicators, and Outcomes in Learning Health Systems: Protocol for a Jurisdictional Scan %A Vanderhout,Shelley %A Bird,Marissa %A Giannarakos,Antonia %A Panesar,Balpreet %A Whitmore,Carly %+ Institute of Health Policy, Management, and Evaluation, University of Toronto, Toronto, ON, Canada, shelley.vanderhout@thp.ca %K learning health systems %K evaluation %K jurisdictional scan %K counterfactuals %K LHS %K health system %K real-time evidence %K informatics %K organizational culture %K learning cycles %K benchmark %K patient care %K gaps %K health care %K inequities %K development %K implementation %K intervention %K new approach %D 2024 %7 6.12.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: In learning health systems (LHSs), real-time evidence, informatics, patient-provider partnerships and experiences, and organizational culture are combined to conduct “learning cycles” that support improvements in care. Although the concept of LHSs is fairly well established in the literature, evaluation methods, mechanisms, and indicators are less consistently described. Furthermore, LHSs often use “usual care” or “status quo” as a benchmark for comparing new approaches to care, but disentangling usual care from multifarious care modalities found across settings is challenging. There is a need to identify which evaluation methods are used within LHSs, describe how LHS growth and maturity are conceptualized, and determine what tools and measures are being used to evaluate LHSs at the system level. Objective: This study aimed to (1) identify international examples of LHSs and describe their evaluation approaches, frameworks, indicators, and outcomes; and (2) describe common characteristics, emphases, assumptions, or challenges in establishing counterfactuals in LHSs. Methods: A jurisdictional scan, which is a method used to explore, understand, and assess how problems have been framed by others in a given field, will be conducted according to modified PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. LHSs will be identified through a search of peer-reviewed and gray literature using Ovid MEDLINE, EBSCO CINAHL, Ovid Embase, Clarivate Web of Science, PubMed non-MEDLINE databases, and the web. We will describe evaluation approaches used both at the LHS learning cycle and system levels. To gain a comprehensive understanding of each LHS, including details specific to evaluation, self-identified LHSs will be included if they are described according to at least 4 of 11 prespecified criteria (core functionalities, analytics, use of evidence, co-design or implementation, evaluation, change management or governance structures, data sharing, knowledge sharing, training or capacity building, equity, and sustainability). Search results will be screened, extracted, and analyzed to inform a descriptive review pertaining to our main objectives. Evaluation methods and approaches, both within learning cycles and at the system level, as well as frameworks, indicators, and target outcomes, will be identified and summarized descriptively. Across evaluations, common challenges, assumptions, contextual factors, and mechanisms will be described. Results: As of October 2024, the database searches described above yielded 3503 citations after duplicate removal. Full-text screening of 117 articles is complete, and 49 articles are under analysis. Results are expected in early 2025. Conclusions: This research will characterize the current landscape of LHS evaluation approaches and provide a foundation for developing consistent and scalable metrics of LHS growth, maturity, and success. This work will also serve to identify opportunities for improving the alignment of current evaluation approaches and metrics with population health needs, community priorities, equity, and health system strategic aims. Trial Registration: Open Science Framework b5u7e; https://osf.io/b5u7e International Registered Report Identifier (IRRID): DERR1-10.2196/57929 %R 10.2196/57929 %U https://www.researchprotocols.org/2024/1/e57929 %U https://doi.org/10.2196/57929 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e45763 %T Patient-Centric Mobile Medical Services Accessed Through Smartphones in the Top 100 Chinese Public Hospitals: Cross-Sectional Survey Study %A Huang,Xuan %A Wang,Ying %A Yang,Xixian %A Jiang,Ruo %A Liu,Yicheng %A Wang,Hui %+ Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, 600 Yishan Road, Shanghai, 200233, China, 86 21 24058249, wangh2005@alumni.sjtu.edu.cn %K mobile health technology %K smartphones %K mobile phone %K internet hospital %K China %D 2024 %7 4.12.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Smartphone-based technology has been used to enhance the delivery of health care services to the public in numerous countries. Objective: This study aims to investigate the application of patient-centric mobile medical services accessed through smartphones in the top 100 Chinese public hospitals. Methods: Data on 124 tertiary public hospitals, ranked among the top 100 by the China Hospital Science and Technology Evaluation Metrics of the Chinese Academy of Medical Sciences (2019) and China’s Hospital Rankings of the Hospital Management Institute of Fudan University (2019), were collected from the WeChat platform (Tencent Inc), mobile phone apps, and official websites until February 10, 2021. Results: A total of 124 tertiary public hospitals, all of which were among the top 100 hospitals according to the 2 ranking lists, were selected for this study. Almost all (122/124, 98.39%) of the hospitals offered basic services such as appointment scheduling, registration, and health education. The majority also provided online access to test reports (95/124, 76.61%), consultations (72/124, 58.06%), and prescriptions (61/124, 49.19%). Among the hospitals offering online prescriptions, the majority (54/61, 88.52%) supported home delivery through third-party carriers. Slightly less than half (57/124, 45.97%) used artificial intelligence for medical guidance. Only a small fraction (8/124, 6.45%) managed chronic diseases through online monitoring and supervision by experienced doctors. Approximately half (60/124, 48.39%) of the included hospitals were officially licensed as internet hospitals approved to provide full online services. Hospitals with official internet hospital licenses provided more extensive digital health offerings. A significantly higher proportion of approved hospitals offered online consultations (29.69% vs 88.33%, r=43.741; P<.001), test reports (62.5% vs 91.67%, r=14.703; P<.001), and chronic disease management (1.56% vs 11.67%, r=5.238; P<.05). These officially approved hospitals tended to provide over 6 mobile medical services, mainly in the regions of Shanghai and Guangdong. This geographic distribution aligned with the overall layout of hospitals included in the study. Conclusions: Patient-centric mobile medical services offered by the top 100 Chinese public hospitals accessed through smartphones primarily focus on online appointment scheduling, registration, health education, and accessing test reports. The most popular features include online consultations, prescriptions, medication delivery, medical guidance, and early-stage chronic disease management. Approved internet hospitals offer a significantly greater variety of patient-centric mobile medical services compared with unapproved ones. %M 39631758 %R 10.2196/45763 %U https://formative.jmir.org/2024/1/e45763 %U https://doi.org/10.2196/45763 %U http://www.ncbi.nlm.nih.gov/pubmed/39631758 %0 Journal Article %@ 2369-1999 %I JMIR Publications %V 10 %N %P e53229 %T Examining Racial Disparities in Colorectal Cancer Screening and the Role of Online Medical Record Use: Findings From a Cross-Sectional Study of a National Survey %A Ewing,Aldenise P %A Tounkara,Fode %A Marshall,Daniel %A Henry,Abhishek V %A Abdel-Rasoul,Mahmoud %A McElwain,Skylar %A Clark,Justice %A Hefner,Jennifer L %A Zaire,Portia J %A Nolan,Timiya S %A Tarver,Willi L %A Doubeni,Chyke A %K colorectal cancer %K cancer screening %K early detection %K Health Information National Trends Survey %K cancer disparities %K online medical records %K secondary data analysis %D 2024 %7 4.12.2024 %9 %J JMIR Cancer %G English %X Background: Colorectal cancer (CRC) is the second leading cause of cancer-related deaths in the United States. Early detection via routine CRC screening can significantly lower risks for CRC-specific morbidity and mortality. Public health initiatives between 2000 and 2015 nearly doubled CRC screening rates for some US adults. However, screening rates remain lowest for adults aged 45‐49 years (20%), patients of safety net health care facilities (42%), adults without insurance (44%), and other subgroups compared with national averages (72%). Given the evolving landscape of digital health care and trends in web-based health information–seeking behaviors, leveraging online medical record (OMR) systems may be an underutilized resource to promote CRC screening utilization. Recognizing trends in OMR usage and patient demographics may enhance digital inclusion—a key social determinant of health—and support equitable web-based interventions aimed at boosting CRC screening across diverse populations. Objective: This study examined the association of accessing an OMR with CRC screening utilization and corresponding sociodemographic characteristics of US adults. Methods: In 2023, we conducted a secondary data analysis using a pooled, weighted sample from Health Information National Trends Survey (HINTS) 5 cycles, 2, 3, and 4 (2018‐2020), a nationally representative survey assessing how US adults access and use health-related information. We analyzed the association between sociodemographic characteristics, medical conditions, OMR access, and CRC screening behaviors via logistic regression. Results: The sample included adults aged 45‐75 years (N=5143). The mean age was 59 (SD 8) years for those who reported CRC screening and 52 (SD 6) years for those never screened. Nearly 70% (4029/5143) of participants reported CRC screening and 52% (2707/5143) reported OMR access in the past year. Adjusted odds of CRC screening were higher among non-Hispanic African American or Black adults than among non-Hispanic White adults (odds ratio [OR] 1.76, 95% CI 1.22‐2.53), adults who accessed an OMR (OR 1.89, 95% CI 1.45‐2.46), older individuals (OR 1.18, 95% CI 1.16‐1.21), the insured (OR 3.69, 95% CI 2.34‐5.82), and those with a professional or graduate degree versus those with a high school diploma or less (OR 2.65, 95% CI 1.28‐5.47). Individuals aged 65‐75 years were significantly more likely (P<.001) to be screened (1687/1831, 91%) than those aged 45‐49 years (190/610, 29%). Conclusions: Promoting OMR access, especially among the most disadvantaged Americans, may assist in reaching national screening goals. Emphasis should be placed on the mutability of OMR use compared with most other statistically significant associations with CRC screening behaviors. OMR access provides an intervenable means of promoting CRC education and screening, especially among those facing structural barriers to cancer diagnoses and care. Future research should focus on tailored and accessible interventions that expand OMR access, particularly for younger populations. %R 10.2196/53229 %U https://cancer.jmir.org/2024/1/e53229 %U https://doi.org/10.2196/53229 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e52516 %T Supporting Patients’ Use of Digital Services in Primary Health Care in England: Synthesis of Evidence From a Mixed Methods Study of “Digital Facilitation” %A Sussex,Jon %A Atherton,Helen %A Abel,Gary %A Clark,Christopher %A Cockcroft,Emma %A Leach,Brandi %A Marriott,Christine %A Newbould,Jennifer %A Pitchforth,Emma %A Winder,Rachel %A Campbell,John %K web-based health services %K primary care %K digital facilitation %K evidence synthesis %K medical practitioners %K digital services %K digital intervention %K mixed methods study %K scoping review %K ethnography %D 2024 %7 4.12.2024 %9 %J JMIR Hum Factors %G English %X Background: General medical practitioners and other staff at primary care medical practices have an important role in facilitating patient access to online services in the National Health Service in England. These services range from online ordering of repeat prescriptions to conducting online consultations with health care professionals. We have defined “digital facilitation” as that range of processes, procedures, and personnel that seeks to support patients in their uptake and use of online services. Objective: We report how we have synthesized the evidence from a mixed methods study of digital facilitation in primary care in England. The study’s objectives were to identify, characterize, and explore the benefits and challenges of different models of digital facilitation in general medical practices in England and to design a framework for evaluation of the effectiveness and costs of digital facilitation interventions. Methods: Our study comprised scoping review of literature, survey of staff in general practices, survey of patients, and ethnography at case study practices plus stakeholder interviews. We compiled a triangulation matrix of the findings from individual work packages through an iterative process whereby each work package’s results were first analyzed separately and were then cumulatively combined across work packages in 3 successive workshops. From the resulting matrix, we developed a program theory and an implementation theory and constructed a framework for evaluations of digital facilitation in primary care. The final step of the synthesis process was to discuss the results with national and regional National Health Service stakeholders. Results: Triangulation yielded a combined set of findings summarized within 11 thematic groupings: 3 setting the scene within which digital facilitation takes place, and 8 related to different types of digital facilitation, their implementation, and effectiveness. Some thematic groupings were evident in the findings of all 4 of the research work packages; others were not addressed in all the work packages but were evident from those where they were addressed. Throughout the synthesis, there were no instances where findings from one work package contradicted the findings of another. Findings either reinforced each other or offered complementary or additional insights. The discussion at the stakeholder meeting held at the end of the study resulted in the research team clarifying some findings but not changing any of them. Conclusions: Digital facilitation can take many forms, though much of what is currently done in primary care practices in England is reactive and passive. Clear lines of responsibility, digital tools and platforms that work well for patients and practice staff, and investment in staff time and training are all needed if digital facilitation is to deliver on its promise. We propose a framework for future evaluations of the effectiveness and costs of digital facilitation interventions. %R 10.2196/52516 %U https://humanfactors.jmir.org/2024/1/e52516 %U https://doi.org/10.2196/52516 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 10 %N %P e59844 %T The University of California Study of Outcomes in Mothers and Infants (a Population-Based Research Resource): Retrospective Cohort Study %A Baer,Rebecca J %A Bandoli,Gretchen %A Jelliffe-Pawlowski,Laura %A Chambers,Christina D %+ Department of Pediatrics, University of California San Diego, 9500 Gilman Drive, La Jolla, CA, 92093, United States, 1 2063510850, rjbaer@ucsd.edu %K birth certificate %K vital statistics %K hospital discharge %K administrative data %K linkage %K pregnancy outcome %K birth outcome %K infant outcome %K adverse outcome %K preterm birth %K birth defects %K pregnancy %K prenatal %K California %K policy %K disparities %K children %K data collection %D 2024 %7 3.12.2024 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: Population-based databases are valuable for perinatal research. The California Department of Health Care Access and Information (HCAI) created a linked birth file covering the years 1991 through 2012. This file includes birth and fetal death certificate records linked to the hospital discharge records of the birthing person and infant. In 2019, the University of California Study of Outcomes in Mothers and Infants received approval to create similar linked birth files for births from 2011 onward, with 2 years of overlapping birth files to allow for linkage comparison. Objective: This paper aims to describe the University of California Study of Outcomes in Mothers and Infants linkage methodology, examine the linkage quality, and discuss the benefits and limitations of the approach. Methods: Live birth and fetal death certificates were linked to hospital discharge records for California infants between 2005 and 2020. The linkage algorithm includes variables such as birth hospital and date of birth, and linked record selection is made based on a “link score.” The complete file includes California Vital Statistics and HCAI hospital discharge records for the birthing person (1 y before delivery and 1 y after delivery) and infant (1 y after delivery). Linkage quality was assessed through a comparison of linked files and California Vital Statistics only. Comparisons were made to previous linked birth files created by the HCAI for 2011 and 2012. Results: Of the 8,040,000 live births, 7,427,738 (92.38%) California Vital Statistics live birth records were linked to HCAI records for birthing people, 7,680,597 (95.53%) birth records were linked to HCAI records for the infant, and 7,285,346 (90.61%) California Vital Statistics birth records were linked to HCAI records for both the birthing person and the infant. The linkage rates were 92.44% (976,526/1,056,358) for Asian and 86.27% (28,601/33,151) for Hawaiian or Pacific Islander birthing people. Of the 44,212 fetal deaths, 33,355 (75.44%) had HCAI records linked to the birthing person. When assessing variables in both California Vital Statistics and hospital records, the percentage was greatest when using both sources: the rates of gestational diabetes were 4.52% (329,128/7,285,345) in the California Vital Statistics records, 8.2% (597,534/7,285,345) in the HCAI records, and 9.34% (680,757/7,285,345) when using both data sources. Conclusions: We demonstrate that the linkage strategy used for this data platform is similar in linkage rate and linkage quality to the previous linked birth files created by the HCAI. The linkage provides higher rates of crucial variables, such as diabetes, compared to birth certificate records alone, although selection bias from the linkage must be considered. This platform has been used independently to examine health outcomes, has been linked to environmental datasets and residential data, and has been used to obtain and examine maternal serum and newborn blood spots. %M 39625748 %R 10.2196/59844 %U https://publichealth.jmir.org/2024/1/e59844 %U https://doi.org/10.2196/59844 %U http://www.ncbi.nlm.nih.gov/pubmed/39625748 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e60258 %T Digital Information Ecosystems in Modern Care Coordination and Patient Care Pathways and the Challenges and Opportunities for AI Solutions %A Chen,You %A Lehmann,Christoph U %A Malin,Bradley %+ Department of Biomedical Informatics, Vanderbilt University Medical Center, 2525 West End Ave, Nashville, TN, 37203, United States, 1 6153431939, you.chen@vanderbilt.edu %K patient care pathway %K care journey %K care coordination %K digital information ecosystem %K digital technologies %K artificial intelligence %K information interoperability %K information silos %K workload %K information retrieval %K care transitions %K patient-reported outcome measures %K clinical workflow %K usability %K user experience workflow %K health care information systems %K networks of health care professionals %K patient information flow %D 2024 %7 2.12.2024 %9 Viewpoint %J J Med Internet Res %G English %X The integration of digital technologies into health care has significantly enhanced the efficiency and effectiveness of care coordination. Our perspective paper explores the digital information ecosystems in modern care coordination, focusing on the processes of information generation, updating, transmission, and exchange along a patient’s care pathway. We identify several challenges within this ecosystem, including interoperability issues, information silos, hard-to-map patient care journeys, increased workload on health care professionals, coordination and communication gaps, and compliance with privacy regulations. These challenges are often associated with inefficiencies and diminished care quality. We also examine how emerging artificial intelligence (AI) tools have the potential to enhance the management of patient information flow. Specifically, AI can boost interoperability across diverse health systems; optimize and monitor patient care pathways; improve information retrieval and care transitions; humanize health care by integrating patients’ desired outcomes and patient-reported outcome measures; and optimize clinical workflows, resource allocation, and digital tool usability and user experiences. By strategically leveraging AI, health care systems can establish a more robust and responsive digital information ecosystem, improving care coordination and patient outcomes. This perspective underscores the importance of continued research and investment in AI technologies in patient care pathways. We advocate for a thoughtful integration of AI into health care practices to fully realize its potential in revolutionizing care coordination. %M 39622048 %R 10.2196/60258 %U https://www.jmir.org/2024/1/e60258 %U https://doi.org/10.2196/60258 %U http://www.ncbi.nlm.nih.gov/pubmed/39622048 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e58007 %T A Cross-Disciplinary Analysis of the Complexities of Scaling Up eHealth Innovation %A Allers,Sanne %A Carboni,Chiara %A Eijkenaar,Frank %A Wehrens,Rik %+ Erasmus School of Health Policy & Management, Erasmus University Rotterdam, Burgemeester van Oudlaan 50, Rotterdam, 3062PA, Netherlands, 31 104081213, allers@eshpm.eur.nl %K innovation %K eHealth %K remote patient monitoring %K scale-up %K cross-disciplinary %K qualitative case study %K health care systems %K adaptation %K complexity %K health care %K framework %K ecological perspective %K barriers and facilitators %D 2024 %7 2.12.2024 %9 Viewpoint %J J Med Internet Res %G English %X Innovative eHealth technologies are becoming increasingly common worldwide, with researchers and policy makers advocating their scale-up within and across health care systems. However, examples of successful scale-up remain extremely rare. Although this issue is widely acknowledged, there is still only a limited understanding of why scaling up eHealth technologies is so challenging. This article aims to contribute to a better understanding of the complexities innovators encounter when attempting to scale up eHealth technologies and their strategies for addressing these complexities. We draw on different theoretical perspectives as well as the findings of an interview-based case study of a prominent remote patient monitoring (RPM) innovation in the Netherlands. Specifically, we create a cross-disciplinary theoretical framework bringing together 3 perspectives on scale-up: a structural perspective (focusing on structural barriers and facilitators), an ecological perspective (focusing on local complexities), and a critical perspective (focusing on mutual adaptation between innovation and setting). We then mobilize these perspectives to analyze how various stakeholders (n=14) experienced efforts to scale up RPM technology. We provide 2 key insights: (1) the complexities and strategies associated with local eHealth scale-up are disconnected from those that actors encounter at a broader level scale-up, and this translates into a simultaneous need for stability and malleability, which catches stakeholders in an impasse, and (2) pre-existing circumstances and associated path dependencies shape the complexities of the local context and facilitate or constrain opportunities for the scale-up of eHealth innovation. The 3 theoretical perspectives used in this article, with their diverging assumptions about innovation scale-up, should be viewed as complementary and highlight different aspects of the complexities perceived as playing an important role. Using these perspectives, we conclude that the level at which scale-up is envisaged and the pre-existing local circumstances (2 factors whose importance is often neglected) contribute to an impasse in the scale-up of eHealth innovation at the broader level of scale. %M 39622044 %R 10.2196/58007 %U https://www.jmir.org/2024/1/e58007 %U https://doi.org/10.2196/58007 %U http://www.ncbi.nlm.nih.gov/pubmed/39622044 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e52715 %T Perspectives of Clients and Health Care Professionals on the Opportunities for Digital Health Interventions in Cerebrovascular Disease Care: Qualitative Descriptive Study %A Härkönen,Henna %A Myllykangas,Kirsi %A Kärppä,Mikko %A Rasmus,Kirsi Maaria %A Gomes,Julius Francis %A Immonen,Milla %A Hyvämäki,Piia %A Jansson,Miia %+ Research Unit of Health Sciences and Technology (HST), University of Oulu, PO BOX 8000, Oulu, FI-90014, Finland, 358 504689544, henna.harkonen@oulu.fi %K cerebrovascular disease %K stroke %K digitalization %K interventions %K health care professional %K client %K patient %K mHealth %K mobile health %K application %K digital health %K smartphones %K health system %K qualitative %K descriptive study %K brain %K blood vessel disease %K cerebrovascular disorder %K Finland %K interviews %K efficiency %K information %K quality %K accountability %K neurology %K neuroscience %K brain injury %K mobile phone %D 2024 %7 2.12.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Cerebrovascular diseases (CVDs) are a major and potentially increasing burden to public health. Digital health interventions (DHIs) could support access to and provision of high-quality health care (eg, outcomes, safety, and satisfaction), but the design and development of digital solutions and technologies lack the assessment of user needs. Research is needed to identify opportunities to address health system challenges and improve CVD care with primary users of services as the key informants of everyday requirements. Objective: This study aims to identify opportunities for DHIs from clients’ and health care professionals’ perspectives to address health system challenges and improve CVD care. Methods: This study used a qualitative, descriptive approach. Semistructured, in-person interviews were conducted with 22 clients and 26 health care professionals in a single tertiary-level hospital in Finland between August 2021 and March 2022. The data were analyzed using a deductive and inductive content analysis. Results: Identified opportunities for DHIs in CVD care were organized according to clients, health care professionals, and data services and classified into 14 main categories and 27 generic categories, with 126 subcategories of requirements. DHIs for clients could support the long-term management of health and life changes brought on by CVD. They could provide access to personal health data and offer health information, support, and communication possibilities for clients and their caregivers. Health care professionals would benefit from access to relevant patient data, along with systems and tools that support competence and decision-making. Intersectoral and professional collaboration could be promoted with digital platforms and care pathways. DHIs for data services could enhance care planning and coordination with novel predictive data and interoperable systems for data exchange. Conclusions: The combined study of client and health care professional perspectives identified several opportunities and requirements for DHIs that related to the information, availability, quality, acceptability, utilization, efficiency, and accountability challenges of health systems. These findings provide valuable social insights into digital transformation and the emerging design, development, and use of user-centered technologies and applications to address challenges and improve CVD care and health care. %M 39622027 %R 10.2196/52715 %U https://www.jmir.org/2024/1/e52715 %U https://doi.org/10.2196/52715 %U http://www.ncbi.nlm.nih.gov/pubmed/39622027 %0 Journal Article %@ 2561-7605 %I JMIR Publications %V 7 %N %P e58980 %T Enhancing Frailty Assessments for Transcatheter Aortic Valve Replacement Patients Using Structured and Unstructured Data: Real-World Evidence Study %A Mardini,Mamoun T %A Bai,Chen %A Bavry,Anthony A %A Zaghloul,Ahmed %A Anderson,R David %A Price,Catherine E Crenshaw %A Al-Ani,Mohammad A Z %K transcatheter aortic valve replacement %K frailty %K cardiology %K machine learning %K TAVR %K minimally invasive surgery %K cardiac surgery %K real-world data %K topic modeling %K clinical notes %K electronic health record %K EHR %D 2024 %7 27.11.2024 %9 %J JMIR Aging %G English %X Background: Transcatheter aortic valve replacement (TAVR) is a commonly used treatment for severe aortic stenosis. As degenerative aortic stenosis is primarily a disease afflicting older adults, a frailty assessment is essential to patient selection and optimal periprocedural outcomes. Objective: This study aimed to enhance frailty assessments of TAVR candidates by integrating real-world structured and unstructured data. Methods: This study analyzed data from 14,000 patients between January 2018 and December 2019 to assess frailty in TAVR patients at the University of Florida. Frailty was identified using the Fried criteria, which includes weight loss, exhaustion, walking speed, grip strength, and physical activity. Latent Dirichlet allocation for topic modeling and Extreme Gradient Boosting for frailty prediction were applied to unstructured clinical notes and structured electronic health record (EHR) data. We also used least absolute shrinkage and selection operator regression for feature selection. Model performance was rigorously evaluated using nested cross-validation, ensuring the generalizability of the findings. Results: Model performance was significantly improved by combining unstructured clinical notes with structured EHR data, achieving an area under the receiver operating characteristic curve of 0.82 (SD 0.07), which surpassed the EHR-only model’s area under the receiver operating characteristic curve of 0.64 (SD 0.08). The Shapley Additive Explanations analysis found that congestive heart failure management, back problems, and atrial fibrillation were the top frailty predictors. Additionally, the latent Dirichlet allocation topic modeling identified 7 key topics, highlighting the role of specific medical treatments in predicting frailty. Conclusions: Integrating unstructured clinical notes and structured EHR data led to a notable enhancement in predicting frailty. This method shows great potential for standardizing frailty assessments using real-world data and improving patient selection for TAVR. %R 10.2196/58980 %U https://aging.jmir.org/2024/1/e58980 %U https://doi.org/10.2196/58980 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e51056 %T Patient Preferences for Direct-to-Consumer Telemedicine Services: Replication and Extension of a Nationwide Survey %A Ivanova,Julia %A Wilczewski,Hattie %A Klocksieben,Farina %A Cummins,Mollie %A Soni,Hiral %A Ong,Triton %A Barrera,Janelle %A Harvey,Jillian %A O'Connell,Nathaniel %A McElligott,James %A Welch,Brandon %A Bunnell,Brian %K telemedicine %K survey %K patient preferences %K direct-to-consumer telemedicine %K patient-provider relationship %K inequity %K consumer %K patient experience %K willingness %K income %K association %K satisfaction %K mobile phone %D 2024 %7 27.11.2024 %9 %J JMIR Hum Factors %G English %X Background: A 2017 survey of patient perspectives showed overall willingness and comfort to use telemedicine, but low actual use. Given recent growth and widespread exposure of patients to telemedicine, patient preferences are likely to have changed. Objective: This study aimed to (1) identify demographic trends in patient preferences and experiences; (2) measure ease of use and satisfaction of telemedicine; and (3) measure changes in telemedicine use, willingness, and comfort since 2017. Methods: We replicated a 2017 study with a nationwide survey of US adults. The survey, an extended version of the previous study, measured patient health care access as well as knowledge, experiences, and preferences regarding telemedicine encounters. We recruited participants using SurveyMonkey Audience in July 2022. We used descriptive statistics and generalized estimating equations to measure change and identify trends. Results: We accrued 4577 complete responses. Patient experience with telemedicine was substantially higher in 2022 than in 2017, with 61.1% (vs 5.3%) of participants aware that their primary care provider offered telemedicine and 34.5% (vs 3.5%) reporting use of telemedicine with their primary care provider. This study also reported ease of use and satisfaction rates to be similar to in-person visits, while overall willingness and comfort in using telemedicine increased from 2017. Individuals at the poverty line were significantly less likely to report satisfaction with telemedicine visits. We found increased interpersonal distance in a patient and health care professional relationship significantly reduced patient ease of use, willingness, and comfort in using telemedicine. Conclusions: This study identified an association between income and patient satisfaction, conveying the importance of understanding telemedicine in relation to health care access and equity. Telemedicine ease of use and satisfaction were comparable to in-person visits. Individuals reported greater use and higher positive perceptions of telemedicine willingness and comfort since 2017. %R 10.2196/51056 %U https://humanfactors.jmir.org/2024/1/e51056 %U https://doi.org/10.2196/51056 %0 Journal Article %@ 1929-073X %I JMIR Publications %V 13 %N %P e58036 %T Benefits of Clinical Decision Support Systems for the Management of Noncommunicable Chronic Diseases: Targeted Literature Review %A Grechuta,Klaudia %A Shokouh,Pedram %A Alhussein,Ahmad %A Müller-Wieland,Dirk %A Meyerhoff,Juliane %A Gilbert,Jeremy %A Purushotham,Sneha %A Rolland,Catherine %+ Boehringer Ingelheim International GmbH, Binger Strasse 173, Ingelheim am Rhein, 55218, Germany, 49 15143151983, klaudia.grechuta@boehringer-ingelheim.com %K clinical decision support system %K digital health %K chronic disease management %K electronic health records %K noncommunicable diseases %K targeted literature review %K mobile phone %D 2024 %7 27.11.2024 %9 Review %J Interact J Med Res %G English %X Background: Clinical decision support systems (CDSSs) are designed to assist in health care delivery by supporting medical practice with clinical knowledge, patient information, and other relevant types of health information. CDSSs are integral parts of health care technologies assisting in disease management, including diagnosis, treatment, and monitoring. While electronic medical records (EMRs) serve as data repositories, CDSSs are used to assist clinicians in providing personalized, context-specific recommendations derived by comparing individual patient data to evidence-based guidelines. Objective: This targeted literature review (TLR) aimed to identify characteristics and features of both stand-alone and EMR-integrated CDSSs that influence their outcomes and benefits based on published scientific literature. Methods: A TLR was conducted using the Embase, MEDLINE, and Cochrane databases to identify data on CDSSs published in a 10-year frame (2012-2022). Studies on computerized, guideline-based CDSSs used by health care practitioners with a focus on chronic disease areas and reporting outcomes for CDSS utilization were eligible for inclusion. Results: A total of 49 publications were included in the TLR. Studies predominantly reported on EMR-integrated CDSSs (ie, connected to an EMR database; n=32, 65%). The implementation of CDSSs varied globally, with substantial utilization in the United States and within the domain of cardio-renal-metabolic diseases. CDSSs were found to positively impact “quality assurance” (n=35, 69%) and provide “clinical benefits” (n=20, 41%), compared to usual care. Among CDSS features, treatment guidance and flagging were consistently reported as the most frequent elements for enhancing health care, followed by risk level estimation, diagnosis, education, and data export. The effectiveness of a CDSS was evaluated most frequently in primary care settings (n=34, 69%) across cardio-renal-metabolic disease areas (n=32, 65%), especially in diabetes (n=13, 26%). Studies reported CDSSs to be commonly used by a mixed group (n=27, 55%) of users including physicians, specialists, nurses or nurse practitioners, and allied health care professionals. Conclusions: Overall, both EMR-integrated and stand-alone CDSSs showed positive results, suggesting their benefits to health care providers and potential for successful adoption. Flagging and treatment recommendation features were commonly used in CDSSs to improve patient care; other features such as risk level estimation, diagnosis, education, and data export were tailored to specific requirements and collectively contributed to the effectiveness of health care delivery. While this TLR demonstrated that both stand-alone and EMR-integrated CDSSs were successful in achieving clinical outcomes, the heterogeneity of included studies reflects the evolving nature of this research area, underscoring the need for further longitudinal studies to elucidate aspects that may impact their adoption in real-world scenarios. %M 39602213 %R 10.2196/58036 %U https://www.i-jmr.org/2024/1/e58036 %U https://doi.org/10.2196/58036 %U http://www.ncbi.nlm.nih.gov/pubmed/39602213 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e54597 %T A Computable Phenotype Algorithm for Postvaccination Myocarditis/Pericarditis Detection Using Real-World Data: Validation Study %A Deady,Matthew %A Duncan,Raymond %A Sonesen,Matthew %A Estiandan,Renier %A Stimpert,Kelly %A Cho,Sylvia %A Beers,Jeffrey %A Goodness,Brian %A Jones,Lance Daniel %A Forshee,Richard %A Anderson,Steven A %A Ezzeldin,Hussein %+ Center for Biologics Evaluation and Research, Food and Drug Administration, 10903 New Hampshire Ave, Silver Spring, MD, 20993, United States, 1 240 205 2215, hussein.ezzeldin@fda.hhs.gov %K adverse event %K vaccine safety %K interoperability %K computable phenotype %K postmarket surveillance system %K fast healthcare interoperability resources %K FHIR %K real-world data %K validation study %K Food and Drug Administration %K electronic health records %K COVID-19 vaccine %D 2024 %7 25.11.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Adverse events (AEs) associated with vaccination have traditionally been evaluated by epidemiological studies. More recently, they have gained attention due to the emergency use authorization of several COVID-19 vaccines. As part of its responsibility to conduct postmarket surveillance, the US Food and Drug Administration continues to monitor several AEs of interest to ensure the safety of vaccines, including those for COVID-19. Objective: This study is part of the Biologics Effectiveness and Safety Initiative, which aims to improve the US Food and Drug Administration’s postmarket surveillance capabilities while minimizing the burden of collecting clinical data on suspected postvaccination AEs. The objective of this study was to enhance active surveillance efforts through a pilot platform that can receive automatically reported AE cases through a health care data exchange. Methods: We detected cases by sharing and applying computable phenotype algorithms to real-world data in health care providers’ electronic health records databases. Using the fast healthcare interoperability resources standard for secure data transmission, we implemented a computable phenotype algorithm on a new health care system. The study focused on the algorithm's positive predictive value, validated through clinical records, assessing both the time required for implementation and the accuracy of AE detection. Results: The algorithm required 200-250 hours to implement and optimize. Of the 6,574,420 clinical encounters across 694,151 patients, 30 cases were identified as potential myocarditis/pericarditis. Of these, 26 cases were retrievable, and 24 underwent clinical validation. In total, 14 cases were confirmed as definite or probable myocarditis/pericarditis, yielding a positive predictive value of 58.3% (95% CI 37.3%-76.9%). These findings underscore the algorithm's capability for real-time detection of AEs, though they also highlight variability in performance across different health care systems. Conclusions: The study advocates for the ongoing refinement and application of distributed computable phenotype algorithms to enhance AE detection capabilities. These tools are crucial for comprehensive postmarket surveillance and improved vaccine safety monitoring. The outcomes suggest the need for further optimization to achieve more consistent results across diverse health care settings. %M 39586081 %R 10.2196/54597 %U https://www.jmir.org/2024/1/e54597 %U https://doi.org/10.2196/54597 %U http://www.ncbi.nlm.nih.gov/pubmed/39586081 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e54357 %T Antihypertensive Drug Recommendations for Reducing Arterial Stiffness in Patients With Hypertension: Machine Learning–Based Multicohort (RIGIPREV) Study %A Cavero-Redondo,Iván %A Martinez-Rodrigo,Arturo %A Saz-Lara,Alicia %A Moreno-Herraiz,Nerea %A Casado-Vicente,Veronica %A Gomez-Sanchez,Leticia %A Garcia-Ortiz,Luis %A Gomez-Marcos,Manuel A %A , %+ CarVasCare Research Group, Facultad de Enfermería de Cuenca, Universidad de Castilla-La Mancha, C. Santa Teresa Jornet s/n, Cuenca, 16001, Spain, 34 969179100, alicia.delsaz@uclm.es %K antihypertensive %K drugs %K models %K patients %K pulse wave velocity %K recommendations %K hypertension %K machine learning %K drug recommendations %K arterial stiffness %K RIGIPREV %D 2024 %7 25.11.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: High systolic blood pressure is one of the leading global risk factors for mortality, contributing significantly to cardiovascular diseases. Despite advances in treatment, a large proportion of patients with hypertension do not achieve optimal blood pressure control. Arterial stiffness (AS), measured by pulse wave velocity (PWV), is an independent predictor of cardiovascular events and overall mortality. Various antihypertensive drugs exhibit differential effects on PWV, but the extent to which these effects vary depending on individual patient characteristics is not well understood. Given the complexity of selecting the most appropriate antihypertensive medication for reducing PWV, machine learning (ML) techniques offer an opportunity to improve personalized treatment recommendations. Objective: This study aims to develop an ML model that provides personalized recommendations for antihypertensive medications aimed at reducing PWV. The model considers individual patient characteristics, such as demographic factors, clinical data, and cardiovascular measurements, to identify the most suitable antihypertensive agent for improving AS. Methods: This study, known as the RIGIPREV study, used data from the EVA, LOD-DIABETES, and EVIDENT studies involving individuals with hypertension with baseline and follow-up measurements. Antihypertensive drugs were grouped into classes such as angiotensin-converting enzyme inhibitors (ACEIs), angiotensin receptor blockers (ARBs), β-blockers, diuretics, and combinations of diuretics with ACEIs or ARBs. The primary outcomes were carotid-femoral and brachial-ankle PWV, while the secondary outcomes included various cardiovascular, anthropometric, and biochemical parameters. A multioutput regressor using 6 random forest models was used to predict the impact of each antihypertensive class on PWV reduction. Model performance was evaluated using the coefficient of determination (R2) and mean squared error. Results: The random forest models exhibited strong predictive capabilities, with internal validation yielding R2 values between 0.61 and 0.74, while external validation showed a range of 0.26 to 0.46. The mean squared values ranged from 0.08 to 0.22 for internal validation and from 0.29 to 0.45 for external validation. Variable importance analysis revealed that glycated hemoglobin and weight were the most critical predictors for ACEIs, while carotid-femoral PWV and total cholesterol were key variables for ARBs. The decision tree model achieved an accuracy of 84.02% in identifying the most suitable antihypertensive drug based on individual patient characteristics. Furthermore, the system’s recommendations for ARBs matched 55.3% of patients’ original prescriptions. Conclusions: This study demonstrates the utility of ML techniques in providing personalized treatment recommendations for antihypertensive therapy. By accounting for individual patient characteristics, the model improves the selection of drugs that control blood pressure and reduce AS. These findings could significantly aid clinicians in optimizing hypertension management and reducing cardiovascular risk. However, further studies with larger and more diverse populations are necessary to validate these results and extend the model’s applicability. %M 39585738 %R 10.2196/54357 %U https://www.jmir.org/2024/1/e54357 %U https://doi.org/10.2196/54357 %U http://www.ncbi.nlm.nih.gov/pubmed/39585738 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e57486 %T Identification of a Susceptible and High-Risk Population for Postoperative Systemic Inflammatory Response Syndrome in Older Adults: Machine Learning–Based Predictive Model %A Mai,Haiyan %A Lu,Yaxin %A Fu,Yu %A Luo,Tongsen %A Li,Xiaoyue %A Zhang,Yihan %A Liu,Zifeng %A Zhang,Yuenong %A Zhou,Shaoli %A Chen,Chaojin %+ Department of Anesthesiology, The Third Affiliated Hospital of Sun Yat-sen University, 600 Tianhe Road, Tianhe District, Guangzhou, 510631, China, 86 020 85253333, chenchj28@mail.sysu.edu.cn %K older adult patients %K postoperative SIRS %K sepsis %K machine learning %K prediction model %D 2024 %7 22.11.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Systemic inflammatory response syndrome (SIRS) is a serious postoperative complication among older adult surgical patients that frequently develops into sepsis or even death. Notably, the incidences of SIRS and sepsis steadily increase with age. It is important to identify the risk of postoperative SIRS for older adult patients at a sufficiently early stage, which would allow preemptive individualized enhanced therapy to be conducted to improve the prognosis of older adult patients. In recent years, machine learning (ML) models have been deployed by researchers for many tasks, including disease prediction and risk stratification, exhibiting good application potential. Objective: We aimed to develop and validate an individualized predictive model to identify susceptible and high-risk populations for SIRS in older adult patients to instruct appropriate early interventions. Methods: Data for surgical patients aged ≥65 years from September 2015 to September 2020 in 3 independent medical centers were retrieved and analyzed. The eligible patient cohort in the Third Affiliated Hospital of Sun Yat-sen University was randomly separated into an 80% training set (2882 patients) and a 20% internal validation set (720 patients). We developed 4 ML models to predict postoperative SIRS. The area under the receiver operating curve (AUC), F1 score, Brier score, and calibration curve were used to evaluate the model performance. The model with the best performance was further validated in the other 2 independent data sets involving 844 and 307 cases, respectively. Results: The incidences of SIRS in the 3 medical centers were 24.3% (876/3602), 29.6% (250/844), and 6.5% (20/307), respectively. We identified 15 variables that were significantly associated with postoperative SIRS and used in 4 ML models to predict postoperative SIRS. A balanced cutoff between sensitivity and specificity was chosen to ensure as high a true positive as possible. The random forest classifier (RF) model showed the best overall performance to predict postoperative SIRS, with an AUC of 0.751 (95% CI 0.709-0.793), sensitivity of 0.682, specificity of 0.681, and F1 score of 0.508 in the internal validation set and higher AUCs in the external validation-1 set (0.759, 95% CI 0.723-0.795) and external validation-2 set (0.804, 95% CI 0.746-0.863). Conclusions: We developed and validated a generalizable RF model to predict postoperative SIRS in older adult patients, enabling clinicians to screen susceptible and high-risk patients and implement early individualized interventions. An online risk calculator to make the RF model accessible to anesthesiologists and peers around the world was developed. %M 39501984 %R 10.2196/57486 %U https://www.jmir.org/2024/1/e57486 %U https://doi.org/10.2196/57486 %U http://www.ncbi.nlm.nih.gov/pubmed/39501984 %0 Journal Article %@ 2562-7600 %I JMIR Publications %V 7 %N %P e59619 %T Calculating Optimal Patient to Nursing Capacity: Comparative Analysis of Traditional and New Methods %A Ware,Anna %A Blumke,Terri %A Hoover,Peter %A Arreola,David %K nurse scheduling %K nurse %K patient ratio %K nursing hours per patient day %K NHPPD %K nursing administration %K workload %K comparative analysis %K nursing %K staffing %K nurse staffing %K registered nurses %K nurse assistants %K staff allocation %K patient capacity %D 2024 %7 22.11.2024 %9 %J JMIR Nursing %G English %X Background: Optimal nurse staffing levels have been shown to impact patients’ prognoses and safety, as well as staff burnout. The predominant method for calculating staffing levels has been patient-to-nurse (P/N) ratios and nursing hours per patient day. However, both methods fall short of addressing the dynamic nature of staffing needs that often fluctuate throughout the day as patients’ clinical status changes and new patients are admitted or discharged from the unit. Objective: In this evaluation, the Veterans Affairs Palo Alto Health Care System (VAPAHCS) piloted a new dynamic bed count calculation in an effort to target optimal staffing levels every hour to provide greater temporal resolution on nurse staffing levels within the Veterans Health Administration. Methods: The dynamic bed count uses elements from both the nursing hours per patient day and P/N ratio to calculate current and target staffing levels, every hour, while balancing across nurse types (registered nurses to nurse assistants) to provide improved temporal insight into staff allocation. The dynamic bed count was compared with traditional P/N ratio methods of calculating patient capacity at the VAPAHCS, to assess optimal patient capacity within their acute care ward from January 1, 2023, through May 25, 2023. Descriptive statistics summarized patient capacity variables across the intensive care unit (ICU), medical-surgical ICU, and 3 acute care units. Student t tests (2-tailed) were used to analyze differences between patient capacity measures. Results: Hourly analysis of patient capacity information displayed how the dynamic bed count provided improved temporal resolution on patient capacity. Comparing the dynamic bed count to the P/N ratio, we found the patient capacity, as determined by the P/N ratio, was, on average, higher than that of the dynamic bed count across VAPAHCS acute care units and the medical-surgical ICU (P<.001). For example, in acute care unit 3C, the average dynamic bed count was 21.6 (SD 4.2) compared with a P/N ratio of 28.6 (SD 3.2). This suggests that calculating patient capacity using P/N ratios alone could lead to units taking on more patients than what the dynamic bed count suggests the unit can optimally handle. Conclusions: As a new patient capacity calculation, the dynamic bed count provided additional details and timely information about clinical staffing levels, patient acuity, and patient turnover. Implementing this calculation into the management process has the potential to empower departments to further optimize staffing and patient care. %R 10.2196/59619 %U https://nursing.jmir.org/2024/1/e59619 %U https://doi.org/10.2196/59619 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e59260 %T Hospital Length of Stay Prediction for Planned Admissions Using Observational Medical Outcomes Partnership Common Data Model: Retrospective Study %A Lee,Haeun %A Kim,Seok %A Moon,Hui-Woun %A Lee,Ho-Young %A Kim,Kwangsoo %A Jung,Se Young %A Yoo,Sooyoung %+ Department of Family Medicine, Seoul National University Bundang Hospital, 172, Dolma-ro bundang-gu, Seongnam-si, 13605, Republic of Korea, 82 0317878845, syjung@snubh.org %K length of stay %K machine learning %K Observational Medical Outcomes Partnership Common Data Model %K allocation of resources %K reproducibility of results %K hospital %K admission %K retrospective study %K prediction model %K electronic health record %K EHR %K South Korea %K logistic regression %K algorithm %K Shapley Additive Explanation %K health care %K clinical informatics %D 2024 %7 22.11.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Accurate hospital length of stay (LoS) prediction enables efficient resource management. Conventional LoS prediction models with limited covariates and nonstandardized data have limited reproducibility when applied to the general population. Objective: In this study, we developed and validated a machine learning (ML)–based LoS prediction model for planned admissions using the Observational Medical Outcomes Partnership Common Data Model (OMOP CDM). Methods: Retrospective patient-level prediction models used electronic health record (EHR) data converted to the OMOP CDM (version 5.3) from Seoul National University Bundang Hospital (SNUBH) in South Korea. The study included 137,437 hospital admission episodes between January 2016 and December 2020. Covariates from the patient, condition occurrence, medication, observation, measurement, procedure, and visit occurrence tables were included in the analysis. To perform feature selection, we applied Lasso regularization in the logistic regression. The primary outcome was an LoS of 7 days or longer, while the secondary outcome was an LoS of 3 days or longer. The prediction models were developed using 6 ML algorithms, with the training and test set split in a 7:3 ratio. The performance of each model was evaluated based on the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). Shapley Additive Explanations (SHAP) analysis measured feature importance, while calibration plots assessed the reliability of the prediction models. External validation of the developed models occurred at an independent institution, the Seoul National University Hospital. Results: The final sample included 129,938 patient entry events in the planned admissions. The Extreme Gradient Boosting (XGB) model achieved the best performance in binary classification for predicting an LoS of 7 days or longer, with an AUROC of 0.891 (95% CI 0.887-0.894) and an AUPRC of 0.819 (95% CI 0.813-0.826) on the internal test set. The Light Gradient Boosting (LGB) model performed the best in the multiclassification for predicting an LoS of 3 days or more, with an AUROC of 0.901 (95% CI 0.898-0.904) and an AUPRC of 0.770 (95% CI 0.762-0.779). The most important features contributing to the models were the operation performed, frequency of previous outpatient visits, patient admission department, age, and day of admission. The RF model showed robust performance in the external validation set, achieving an AUROC of 0.804 (95% CI 0.802-0.807). Conclusions: The use of the OMOP CDM in predicting hospital LoS for planned admissions demonstrates promising predictive capabilities for stays of varying durations. It underscores the advantage of standardized data in achieving reproducible results. This approach should serve as a model for enhancing operational efficiency and patient care coordination across health care settings. %M 39576284 %R 10.2196/59260 %U https://www.jmir.org/2024/1/e59260 %U https://doi.org/10.2196/59260 %U http://www.ncbi.nlm.nih.gov/pubmed/39576284 %0 Journal Article %@ 2369-3762 %I JMIR Publications %V 10 %N %P e59902 %T Performance Comparison of Junior Residents and ChatGPT in the Objective Structured Clinical Examination (OSCE) for Medical History Taking and Documentation of Medical Records: Development and Usability Study %A Huang,Ting-Yun %A Hsieh,Pei Hsing %A Chang,Yung-Chun %K large language model %K medical history taking %K clinical documentation %K simulation-based evaluation %K OSCE standards %K LLM %D 2024 %7 21.11.2024 %9 %J JMIR Med Educ %G English %X Background: This study explores the cutting-edge abilities of large language models (LLMs) such as ChatGPT in medical history taking and medical record documentation, with a focus on their practical effectiveness in clinical settings—an area vital for the progress of medical artificial intelligence. Objective: Our aim was to assess the capability of ChatGPT versions 3.5 and 4.0 in performing medical history taking and medical record documentation in simulated clinical environments. The study compared the performance of nonmedical individuals using ChatGPT with that of junior medical residents. Methods: A simulation involving standardized patients was designed to mimic authentic medical history–taking interactions. Five nonmedical participants used ChatGPT versions 3.5 and 4.0 to conduct medical histories and document medical records, mirroring the tasks performed by 5 junior residents in identical scenarios. A total of 10 diverse scenarios were examined. Results: Evaluation of the medical documentation created by laypersons with ChatGPT assistance and those created by junior residents was conducted by 2 senior emergency physicians using audio recordings and the final medical records. The assessment used the Objective Structured Clinical Examination benchmarks in Taiwan as a reference. ChatGPT-4.0 exhibited substantial enhancements over its predecessor and met or exceeded the performance of human counterparts in terms of both checklist and global assessment scores. Although the overall quality of human consultations remained higher, ChatGPT-4.0’s proficiency in medical documentation was notably promising. Conclusions: The performance of ChatGPT 4.0 was on par with that of human participants in Objective Structured Clinical Examination evaluations, signifying its potential in medical history and medical record documentation. Despite this, the superiority of human consultations in terms of quality was evident. The study underscores both the promise and the current limitations of LLMs in the realm of clinical practice. %R 10.2196/59902 %U https://mededu.jmir.org/2024/1/e59902 %U https://doi.org/10.2196/59902 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e60334 %T Chinese Clinical Named Entity Recognition With Segmentation Synonym Sentence Synthesis Mechanism: Algorithm Development and Validation %A Tang,Jian %A Huang,Zikun %A Xu,Hongzhen %A Zhang,Hao %A Huang,Hailing %A Tang,Minqiong %A Luo,Pengsheng %A Qin,Dong %K clinical named entity recognition %K word embedding %K Chinese electronic medical records %K RoBERTa %K entity recognition %K segmentation %K natural language processing %K AI %K artificial intelligence %K dataset %K dataset augmentation %K algorithm %K entity %K EMR %D 2024 %7 21.11.2024 %9 %J JMIR Med Inform %G English %X Background: Clinical named entity recognition (CNER) is a fundamental task in natural language processing used to extract named entities from electronic medical record texts. In recent years, with the continuous development of machine learning, deep learning models have replaced traditional machine learning and template-based methods, becoming widely applied in the CNER field. However, due to the complexity of clinical texts, the diversity and large quantity of named entity types, and the unclear boundaries between different entities, existing advanced methods rely to some extent on annotated databases and the scale of embedded dictionaries. Objective: This study aims to address the issues of data scarcity and labeling difficulties in CNER tasks by proposing a dataset augmentation algorithm based on proximity word calculation. Methods: We propose a Segmentation Synonym Sentence Synthesis (SSSS) algorithm based on neighboring vocabulary, which leverages existing public knowledge without the need for manual expansion of specialized domain dictionaries. Through lexical segmentation, the algorithm replaces new synonymous vocabulary by recombining from vast natural language data, achieving nearby expansion expressions of the dataset. We applied the SSSS algorithm to the Robustly Optimized Bidirectional Encoder Representations from Transformers Pretraining Approach (RoBERTa) + conditional random field (CRF) and RoBERTa + Bidirectional Long Short-Term Memory (BiLSTM) + CRF models and evaluated our models (SSSS + RoBERTa + CRF; SSSS + RoBERTa + BiLSTM + CRF) on the China Conference on Knowledge Graph and Semantic Computing (CCKS) 2017 and 2019 datasets. Results: Our experiments demonstrated that the models SSSS + RoBERTa + CRF and SSSS + RoBERTa + BiLSTM + CRF achieved F1-scores of 91.30% and 91.35% on the CCKS-2017 dataset, respectively. They also achieved F1-scores of 83.21% and 83.01% on the CCKS-2019 dataset, respectively. Conclusions: The experimental results indicated that our proposed method successfully expanded the dataset and remarkably improved the performance of the model, effectively addressing the challenges of data acquisition, annotation difficulties, and insufficient model generalization performance. %R 10.2196/60334 %U https://medinform.jmir.org/2024/1/e60334 %U https://doi.org/10.2196/60334 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e52817 %T Factors Contributing to Successful Information System Implementation and Employee Well-Being in Health Care and Social Welfare Professionals: Comparative Cross-Sectional Study %A Nadav,Janna %A Kaihlanen,Anu-Marja %A Kujala,Sari %A Keskimäki,Ilmo %A Viitanen,Johanna %A Salovaara,Samuel %A Saukkonen,Petra %A Vänskä,Jukka %A Vehko,Tuulikki %A Heponiemi,Tarja %K implementation %K health information systems %K client information systems %K stress %K healthcare professionals %K social welfare professionals %K clinician well-being %K workplace stress %D 2024 %7 21.11.2024 %9 %J JMIR Med Inform %G English %X Background: The integration of information systems in health care and social welfare organizations has brought significant changes in patient and client care. This integration is expected to offer numerous benefits, but simultaneously the implementation of health information systems and client information systems can also introduce added stress due to the increased time and effort required by professionals. Objective: This study aimed to examine whether professional groups and the factors that contribute to successful implementation (participation in information systems development and satisfaction with software providers’ development work) are associated with the well-being of health care and social welfare professionals. Methods: Data were obtained from 3 national cross-sectional surveys (n=9240), which were carried out among Finnish health care and social welfare professionals (registered nurses, physicians, and social welfare professionals) in 2020‐2021. Self-rated stress and stress related to information systems were used as indicators of well-being. Analyses were conducted using linear and logistic regression analysis. Results: Registered nurses were more likely to experience self-rated stress than physicians (odds ratio [OR] –0.47; P>.001) and social welfare professionals (OR –0.68; P<.001). They also had a higher likelihood of stress related to information systems than physicians (b=–.11; P<.001). Stress related to information systems was less prevalent among professionals who did not participate in information systems development work (b=–.14; P<.001). Higher satisfaction with software providers’ development work was associated with a lower likelihood of self-rated stress (OR –0.23; P<.001) and stress related to information systems (b=–.36 P<.001). When comparing the professional groups, we found that physicians who were satisfied with software providers’ development work had a significantly lower likelihood of stress related to information systems (b=–.12; P<.001) compared with registered nurses and social welfare professionals. Conclusions: Organizations can enhance the well-being of professionals and improve the successful implementation of information systems by actively soliciting and incorporating professional feedback, dedicating time for information systems development, fostering collaboration with software providers, and addressing the unique needs of different professional groups. %R 10.2196/52817 %U https://medinform.jmir.org/2024/1/e52817 %U https://doi.org/10.2196/52817 %0 Journal Article %@ 1947-2579 %I JMIR Publications %V 16 %N %P e60261 %T Population Digital Health: Continuous Health Monitoring and Profiling at Scale %A Hossein Motlagh,Naser %A Zuniga,Agustin %A Thi Nguyen,Ngoc %A Flores,Huber %A Wang,Jiangtao %A Tarkoma,Sasu %A Prosperi,Mattia %A Helal,Sumi %A Nurmi,Petteri %K digital health %K population health %K modeling, health data %K health monitoring %K monitoring %K wearable devices %K wearables %K machine learning %K networking infrastructure %K cost-effectiveness %K device %K sensor %K PDH %K equity %D 2024 %7 20.11.2024 %9 %J Online J Public Health Inform %G English %X This paper introduces population digital health (PDH)—the use of digital health information sourced from health internet of things (IoT) and wearable devices for population health modeling—as an emerging research domain that offers an integrated approach for continuous monitoring and profiling of diseases and health conditions at multiple spatial resolutions. PDH combines health data sourced from health IoT devices, machine learning, and ubiquitous computing or networking infrastructure to increase the scale, coverage, equity, and cost-effectiveness of population health. This contrasts with the traditional population health approach, which relies on data from structured clinical records (eg, electronic health records) or health surveys. We present the overall PDH approach and highlight its key research challenges, provide solutions to key research challenges, and demonstrate the potential of PDH through three case studies that address (1) data inadequacy, (2) inaccuracy of the health IoT devices’ sensor measurements, and (3) the spatiotemporal sparsity in the available digital health information. Finally, we discuss the conditions, prerequisites, and barriers for adopting PDH drawing on from real-world examples from different geographic regions. %R 10.2196/60261 %U https://ojphi.jmir.org/2024/1/e60261 %U https://doi.org/10.2196/60261 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e58088 %T Toward Guidelines for Designing Holistic Integrated Information Visualizations for Time-Critical Contexts: Systematic Review %A Patel,Ahmed Mohammed %A Baxter,Weston %A Porat,Talya %+ Dyson School of Design Engineering, Imperial College London, Imperial College Rd, South Kensington, London, SW7 2DB, United Kingdom, 44 07990035581, ap19@ic.ac.uk %K visualization %K design %K holistic %K integrated %K time-critical %K guidelines %K pre-attentive processing %K gestalt theory %K situation awareness %K decision-making %K mobile phone %D 2024 %7 20.11.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: With the extensive volume of information from various and diverse data sources, it is essential to present information in a way that allows for quick understanding and interpretation. This is particularly crucial in health care, where timely insights into a patient’s condition can be lifesaving. Holistic visualizations that integrate multiple data variables into a single visual representation can enhance rapid situational awareness and support informed decision-making. However, despite the existence of numerous guidelines for different types of visualizations, this study reveals that there are currently no specific guidelines or principles for designing holistic integrated information visualizations that enable quick processing and comprehensive understanding of multidimensional data in time-critical contexts. Addressing this gap is essential for enhancing decision-making in time-critical scenarios across various domains, particularly in health care. Objective: This study aims to establish a theoretical foundation supporting the argument that holistic integrated visualizations are a distinct type of visualization for time-critical contexts and identify applicable design principles and guidelines that can be used to design for such cases. Methods: We systematically searched the literature for peer-reviewed research on visualization strategies, guidelines, and taxonomies. The literature selection followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. The search was conducted across 6 databases: ACM Digital Library, Google Scholar, IEEE Xplore, PubMed, Scopus, and Web of Science. The search was conducted up to August 2024 using the terms (“visualisations” OR “visualizations”) AND (“guidelines” OR “taxonomy” OR “taxonomies”), with studies restricted to the English language. Results: Of 936 papers, 46 (4.9%) were included in the final review. In total, 48% (22/46) related to providing a holistic understanding and overview of multidimensional data; 28% (13/46) focused on integrated presentation, that is, integrating or combining multidimensional data into a single visual representation; and 35% (16/46) pertained to time and designing for rapid information processing. In total, 65% (30/46) of the papers presented general information visualization or visual communication guidelines and principles. No specific guidelines or principles were found that addressed all the characteristics of holistic, integrated visualizations in time-critical contexts. A summary of the key guidelines and principles from the 46 papers was extracted, collated, and categorized into 60 guidelines that could aid in designing holistic integrated visualizations. These were grouped according to different characteristics identified in the systematic review (eg, gestalt principles, reduction, organization, abstraction, and task complexity) and further condensed into 5 main proposed guidelines. Conclusions: Holistic integrated information visualizations in time-critical domains are a unique use case requiring a unique set of design guidelines. Our proposed 5 main guidelines, derived from existing design theories and guidelines, can serve as a starting point to enable both holistic and rapid processing of information, facilitating better-informed decisions in time-critical contexts. %M 39566050 %R 10.2196/58088 %U https://www.jmir.org/2024/1/e58088 %U https://doi.org/10.2196/58088 %U http://www.ncbi.nlm.nih.gov/pubmed/39566050 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e56473 %T The Impact of Patient Access to Electronic Health Records on Health Care Engagement: Systematic Review %A Alomar,Dalia %A Almashmoum,Maryam %A Eleftheriou,Iliada %A Whelan,Pauline %A Ainsworth,John %+ Division of Informatics, Imaging and Data Sciences, School of Health Sciences, Faculty of Biology, Medicine and Health, Manchester Academic Health Science System, The University of Manchester, Vaughan House, Portsmouth St, Manchester, M13 9GB, United Kingdom, 44 7917941877, dalia.alomar@postgrad.manchester.ac.uk %K electronic health records %K personal health record %K health care engagement %K empowerment %K patient experience %K patient satisfaction %K health care services %K systematic review %D 2024 %7 20.11.2024 %9 Review %J J Med Internet Res %G English %X Background: Health information technologies, including electronic health records (EHRs), have revolutionized health care delivery. These technologies promise to enhance the efficiency and quality of care through improved patient health information management. Despite the transformative potential of EHRs, the extent to which patient access contributes to increased engagement with health care services within different clinical setting remains a distinct and underexplored facet. Objective: This systematic review aims to investigate the impact of patient access to EHRs on health care engagement. Specifically, we seek to determine whether providing patients with access to their EHRs contributes to improved engagement with health care services. Methods: A comprehensive systematic review search was conducted across various international databases, including Ovid MEDLINE, Embase, PsycINFO, and CINAHL, to identify relevant studies published from January 1, 2010, to November 15, 2023. The search on these databases was conducted using a combination of keywords and Medical Subject Heading terms related to patient access to electronic health records, patient engagement, and health care services. Studies were included if they assessed the impact of patient access to EHRs on health care engagement and provided evidence (quantitative or qualitative) for that. The guidelines of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 statement were followed for study selection, data extraction, and quality assessment. The included studies were assessed for quality using the Mixed Methods Appraisal Tool, and the results were reported using a narrative synthesis. Results: The initial search from the databases yielded 1737 studies, to which, after scanning their reference lists, we added 10 studies. Of these 1747 studies, 18 (1.03%) met the inclusion criteria for the final review. The synthesized evidence from these studies revealed a positive relationship between patient access to EHRs and health care engagement, addressing 6 categories of health care engagement dimensions and outcomes, including treatment adherence and self-management, patient involvement and empowerment, health care communication and relationship, patient satisfaction and health outcomes, use of health care resources, and usability concerns and barriers. Conclusions: The findings suggested a positive association between patient access to EHRs and health care engagement. The implications of these findings for health care providers, policy makers, and patients should be considered, highlighting the potential benefits and challenges associated with implementing and promoting patient access to EHRs. Further research directions have been proposed to deepen our understanding of this dynamic relationship. %M 39566058 %R 10.2196/56473 %U https://www.jmir.org/2024/1/e56473 %U https://doi.org/10.2196/56473 %U http://www.ncbi.nlm.nih.gov/pubmed/39566058 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e58329 %T Evaluation Framework of Large Language Models in Medical Documentation: Development and Usability Study %A Seo,Junhyuk %A Choi,Dasol %A Kim,Taerim %A Cha,Won Chul %A Kim,Minha %A Yoo,Haanju %A Oh,Namkee %A Yi,YongJin %A Lee,Kye Hwa %A Choi,Edward %+ Department of Digital Health, Samsung Advanced Institute of Health Sciences and Technology (SAIHST), Sungkyunkwan University, 115, Irwon-ro, Gangnam-gu, Seoul, 06355, Republic of Korea, 82 010 7114 2342, taerim.j.kim@gmail.com %K large language models %K health care documentation %K clinical evaluation %K emergency department %K artificial intelligence %K medical record accuracy %D 2024 %7 20.11.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: The advancement of large language models (LLMs) offers significant opportunities for health care, particularly in the generation of medical documentation. However, challenges related to ensuring the accuracy and reliability of LLM outputs, coupled with the absence of established quality standards, have raised concerns about their clinical application. Objective: This study aimed to develop and validate an evaluation framework for assessing the accuracy and clinical applicability of LLM-generated emergency department (ED) records, aiming to enhance artificial intelligence integration in health care documentation. Methods: We organized the Healthcare Prompt-a-thon, a competitive event designed to explore the capabilities of LLMs in generating accurate medical records. The event involved 52 participants who generated 33 initial ED records using HyperCLOVA X, a Korean-specialized LLM. We applied a dual evaluation approach. First, clinical evaluation: 4 medical professionals evaluated the records using a 5-point Likert scale across 5 criteria—appropriateness, accuracy, structure/format, conciseness, and clinical validity. Second, quantitative evaluation: We developed a framework to categorize and count errors in the LLM outputs, identifying 7 key error types. Statistical methods, including Pearson correlation and intraclass correlation coefficients (ICC), were used to assess consistency and agreement among evaluators. Results: The clinical evaluation demonstrated strong interrater reliability, with ICC values ranging from 0.653 to 0.887 (P<.001), and a test-retest reliability Pearson correlation coefficient of 0.776 (P<.001). Quantitative analysis revealed that invalid generation errors were the most common, constituting 35.38% of total errors, while structural malformation errors had the most significant negative impact on the clinical evaluation score (Pearson r=–0.654; P<.001). A strong negative correlation was found between the number of quantitative errors and clinical evaluation scores (Pearson r=–0.633; P<.001), indicating that higher error rates corresponded to lower clinical acceptability. Conclusions: Our research provides robust support for the reliability and clinical acceptability of the proposed evaluation framework. It underscores the framework’s potential to mitigate clinical burdens and foster the responsible integration of artificial intelligence technologies in health care, suggesting a promising direction for future research and practical applications in the field. %M 39566044 %R 10.2196/58329 %U https://www.jmir.org/2024/1/e58329 %U https://doi.org/10.2196/58329 %U http://www.ncbi.nlm.nih.gov/pubmed/39566044 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e64844 %T Comparative Analysis of Diagnostic Performance: Differential Diagnosis Lists by LLaMA3 Versus LLaMA2 for Case Reports %A Hirosawa,Takanobu %A Harada,Yukinori %A Tokumasu,Kazuki %A Shiraishi,Tatsuya %A Suzuki,Tomoharu %A Shimizu,Taro %+ Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, 880 Kitakobayashi, Mibu-cho, Shimotsuga, 321-0293, Japan, 81 0282861111, hirosawa@dokkyomed.ac.jp %K artificial intelligence %K clinical decision support system %K generative artificial intelligence %K large language models %K natural language processing %K NLP %K AI %K clinical decision making %K decision support %K decision making %K LLM: diagnostic %K case report %K diagnosis %K generative AI %K LLaMA %D 2024 %7 19.11.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Generative artificial intelligence (AI), particularly in the form of large language models, has rapidly developed. The LLaMA series are popular and recently updated from LLaMA2 to LLaMA3. However, the impacts of the update on diagnostic performance have not been well documented. Objective: We conducted a comparative evaluation of the diagnostic performance in differential diagnosis lists generated by LLaMA3 and LLaMA2 for case reports. Methods: We analyzed case reports published in the American Journal of Case Reports from 2022 to 2023. After excluding nondiagnostic and pediatric cases, we input the remaining cases into LLaMA3 and LLaMA2 using the same prompt and the same adjustable parameters. Diagnostic performance was defined by whether the differential diagnosis lists included the final diagnosis. Multiple physicians independently evaluated whether the final diagnosis was included in the top 10 differentials generated by LLaMA3 and LLaMA2. Results: In our comparative evaluation of the diagnostic performance between LLaMA3 and LLaMA2, we analyzed differential diagnosis lists for 392 case reports. The final diagnosis was included in the top 10 differentials generated by LLaMA3 in 79.6% (312/392) of the cases, compared to 49.7% (195/392) for LLaMA2, indicating a statistically significant improvement (P<.001). Additionally, LLaMA3 showed higher performance in including the final diagnosis in the top 5 differentials, observed in 63% (247/392) of cases, compared to LLaMA2’s 38% (149/392, P<.001). Furthermore, the top diagnosis was accurately identified by LLaMA3 in 33.9% (133/392) of cases, significantly higher than the 22.7% (89/392) achieved by LLaMA2 (P<.001). The analysis across various medical specialties revealed variations in diagnostic performance with LLaMA3 consistently outperforming LLaMA2. Conclusions: The results reveal that the LLaMA3 model significantly outperforms LLaMA2 per diagnostic performance, with a higher percentage of case reports having the final diagnosis listed within the top 10, top 5, and as the top diagnosis. Overall diagnostic performance improved almost 1.5 times from LLaMA2 to LLaMA3. These findings support the rapid development and continuous refinement of generative AI systems to enhance diagnostic processes in medicine. However, these findings should be carefully interpreted for clinical application, as generative AI, including the LLaMA series, has not been approved for medical applications such as AI-enhanced diagnostics. %M 39561356 %R 10.2196/64844 %U https://formative.jmir.org/2024/1/e64844 %U https://doi.org/10.2196/64844 %U http://www.ncbi.nlm.nih.gov/pubmed/39561356 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e63445 %T Using Large Language Models to Abstract Complex Social Determinants of Health From Original and Deidentified Medical Notes: Development and Validation Study %A Ralevski,Alexandra %A Taiyab,Nadaa %A Nossal,Michael %A Mico,Lindsay %A Piekos,Samantha %A Hadlock,Jennifer %+ Institute for Systems Biology, 401 Terry Ave N, Seattle, WA, 98121, United States, 1 732 1359, jhadlock@isbscience.org %K housing instability %K housing insecurity %K housing %K machine learning %K artificial intelligence %K AI %K large language model %K LLM %K natural language processing %K NLP %K electronic health record %K EHR %K electronic medical record %K EMR %K social determinants of health %K exposome %K pregnancy %K obstetric %K deidentification %D 2024 %7 19.11.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Social determinants of health (SDoH) such as housing insecurity are known to be intricately linked to patients’ health status. More efficient methods for abstracting structured data on SDoH can help accelerate the inclusion of exposome variables in biomedical research and support health care systems in identifying patients who could benefit from proactive outreach. Large language models (LLMs) developed from Generative Pre-trained Transformers (GPTs) have shown potential for performing complex abstraction tasks on unstructured clinical notes. Objective: Here, we assess the performance of GPTs on identifying temporal aspects of housing insecurity and compare results between both original and deidentified notes. Methods: We compared the ability of GPT-3.5 and GPT-4 to identify instances of both current and past housing instability, as well as general housing status, from 25,217 notes from 795 pregnant women. Results were compared with manual abstraction, a named entity recognition model, and regular expressions. Results: Compared with GPT-3.5 and the named entity recognition model, GPT-4 had the highest performance and had a much higher recall (0.924) than human abstractors (0.702) in identifying patients experiencing current or past housing instability, although precision was lower (0.850) compared with human abstractors (0.971). GPT-4’s precision improved slightly (0.936 original, 0.939 deidentified) on deidentified versions of the same notes, while recall dropped (0.781 original, 0.704 deidentified). Conclusions: This work demonstrates that while manual abstraction is likely to yield slightly more accurate results overall, LLMs can provide a scalable, cost-effective solution with the advantage of greater recall. This could support semiautomated abstraction, but given the potential risk for harm, human review would be essential before using results for any patient engagement or care decisions. Furthermore, recall was lower when notes were deidentified prior to LLM abstraction. %M 39561354 %R 10.2196/63445 %U https://www.jmir.org/2024/1/e63445 %U https://doi.org/10.2196/63445 %U http://www.ncbi.nlm.nih.gov/pubmed/39561354 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e57754 %T Data Ownership in the AI-Powered Integrative Health Care Landscape %A Liu,Shuimei %A Guo,L Raymond %+ School of Juris Master, China University of Political Science and Law, 25 Xitucheng Rd, Hai Dian Qu, Beijing, 100088, China, 1 (734) 358 3970, shuiliu0802@alumni.iu.edu %K data ownership %K integrative healthcare %K artificial intelligence %K AI %K ownership %K data science %K governance %K consent %K privacy %K security %K access %K model %K framework %K transparency %D 2024 %7 19.11.2024 %9 Viewpoint %J JMIR Med Inform %G English %X In the rapidly advancing landscape of artificial intelligence (AI) within integrative health care (IHC), the issue of data ownership has become pivotal. This study explores the intricate dynamics of data ownership in the context of IHC and the AI era, presenting the novel Collaborative Healthcare Data Ownership (CHDO) framework. The analysis delves into the multifaceted nature of data ownership, involving patients, providers, researchers, and AI developers, and addresses challenges such as ambiguous consent, attribution of insights, and international inconsistencies. Examining various ownership models, including privatization and communization postulates, as well as distributed access control, data trusts, and blockchain technology, the study assesses their potential and limitations. The proposed CHDO framework emphasizes shared ownership, defined access and control, and transparent governance, providing a promising avenue for responsible and collaborative AI integration in IHC. This comprehensive analysis offers valuable insights into the complex landscape of data ownership in IHC and the AI era, potentially paving the way for ethical and sustainable advancements in data-driven health care. %M 39560980 %R 10.2196/57754 %U https://medinform.jmir.org/2024/1/e57754 %U https://doi.org/10.2196/57754 %U http://www.ncbi.nlm.nih.gov/pubmed/39560980 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e58933 %T A 25-Year Retrospective of Health IT Infrastructure Building: The Example of the Catalonia Region %A Piera-Jiménez,Jordi %A Carot-Sans,Gerard %A Ramiro-Pareta,Marina %A Nogueras,Maria Mercedes %A Folguera-Profitós,Júlia %A Ródenas,Pepi %A Jiménez-Rueda,Alba %A de Pando Navarro,Thais %A Mira Palacios,Josep Antoni %A Fajardo,Joan Carles %A Ustrell Campillo,Joan %A Vela,Emili %A Monterde,David %A Valero-Bover,Damià %A Bonet,Tara %A Tarrasó-Urios,Guillermo %A Cantenys-Sabà,Roser %A Fabregat-Fabregat,Pau %A Gómez Oliveros,Beatriz %A Berdún,Jesús %A Michelena,Xabier %A Cano,Isaac %A González-Colom,Rubèn %A Roca,Josep %A Solans,Oscar %A Pontes,Caridad %A Pérez-Sust,Pol %+ Catalan Health Service, Gran Via de les Corts Catalanes 587, Barcelona, 08007, Spain, 34 934643013, jpiera@catsalut.cat %K health ITs %K eHealth %K integrated care %K open platforms %K interoperability %K Catalonia %K digitalization %K health care structure %K health care delivery %K integrated pathway %K integrated treatment plan %K process management %D 2024 %7 18.11.2024 %9 Viewpoint %J J Med Internet Res %G English %X Over the past decades, health care systems have significantly evolved due to aging populations, chronic diseases, and higher-quality care expectations. Concurrently with the added health care needs, information and communications technology advancements have transformed health care delivery. Technologies such as telemedicine, electronic health records, and mobile health apps promise enhanced accessibility, efficiency, and patient outcomes, leading to more personalized, data-driven care. However, organizational, political, and cultural barriers and the fragmented approach to health information management are challenging the integration of these technologies to effectively support health care delivery. This fragmentation collides with the need for integrated care pathways that focus on holistic health and wellness. Catalonia (northeast Spain), a region of 8 million people with universal health care coverage and a single public health insurer but highly heterogeneous health care service providers, has experienced outstanding digitalization and integration of health information over the past 25 years, when the first transition from paper to digital support occurred. This Viewpoint describes the implementation of health ITs at a system level, discusses the hits and misses encountered in this journey, and frames this regional implementation within the global context. We present the architectures and use trends of the health information platforms over time. This provides insightful information that can be used by other systems worldwide in the never-ending transformation of health care structure and services. %M 39556831 %R 10.2196/58933 %U https://www.jmir.org/2024/1/e58933 %U https://doi.org/10.2196/58933 %U http://www.ncbi.nlm.nih.gov/pubmed/39556831 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e53622 %T Distributed Statistical Analyses: A Scoping Review and Examples of Operational Frameworks Adapted to Health Analytics %A Camirand Lemyre,Félix %A Lévesque,Simon %A Domingue,Marie-Pier %A Herrmann,Klaus %A Ethier,Jean-François %K distributed algorithms %K generalized linear models %K horizontally partitioned data %K GLMs %K learning health systems %K distributed analysis %K federated analysis %K data science %K data custodians %K algorithms %K statistics %K synthesis %K review methods %K searches %K scoping %D 2024 %7 14.11.2024 %9 %J JMIR Med Inform %G English %X Background: Data from multiple organizations are crucial for advancing learning health systems. However, ethical, legal, and social concerns may restrict the use of standard statistical methods that rely on pooling data. Although distributed algorithms offer alternatives, they may not always be suitable for health frameworks. Objective: This study aims to support researchers and data custodians in three ways: (1) providing a concise overview of the literature on statistical inference methods for horizontally partitioned data, (2) describing the methods applicable to generalized linear models (GLMs) and assessing their underlying distributional assumptions, and (3) adapting existing methods to make them fully usable in health settings. Methods: A scoping review methodology was used for the literature mapping, from which methods presenting a methodological framework for GLM analyses with horizontally partitioned data were identified and assessed from the perspective of applicability in health settings. Statistical theory was used to adapt methods and derive the properties of the resulting estimators. Results: From the review, 41 articles were selected and 6 approaches were extracted to conduct standard GLM-based statistical analysis. However, these approaches assumed evenly and identically distributed data across nodes. Consequently, statistical procedures were derived to accommodate uneven node sample sizes and heterogeneous data distributions across nodes. Workflows and detailed algorithms were developed to highlight information sharing requirements and operational complexity. Conclusions: This study contributes to the field of health analytics by providing an overview of the methods that can be used with horizontally partitioned data by adapting these methods to the context of heterogeneous health data and clarifying the workflows and quantities exchanged by the methods discussed. Further analysis of the confidentiality preserved by these methods is needed to fully understand the risk associated with the sharing of summary statistics. %R 10.2196/53622 %U https://medinform.jmir.org/2024/1/e53622 %U https://doi.org/10.2196/53622 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e59634 %T An Electronic Medical Record–Based Prognostic Model for Inpatient Falls: Development and Internal-External Cross-Validation %A Parsons,Rex %A Blythe,Robin %A Cramb,Susanna %A Abdel-Hafez,Ahmad %A McPhail,Steven %+ Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Queensland University of Technology, 60 Musk Ave, Kelvin Grove, 4059, Australia, 61 31380905, rex.parsons@hdr.qut.edu.au %K clinical prediction model %K falls %K patient safety %K prognostic %K electronic medical record %K EMR %K intervention %K hospital %K risk assessment %K clinical decision %K support system %K in-hospital fall %K survival model %K inpatient falls %D 2024 %7 13.11.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Effective fall prevention interventions in hospitals require appropriate allocation of resources early in admission. To address this, fall risk prediction tools and models have been developed with the aim to provide fall prevention strategies to patients at high risk. However, fall risk assessment tools have typically been inaccurate for prediction, ineffective in prevention, and time-consuming to complete. Accurate, dynamic, individualized estimates of fall risk for admitted patients using routinely recorded data may assist in prioritizing fall prevention efforts. Objective: The objective of this study was to develop and validate an accurate and dynamic prognostic model for inpatient falls among a cohort of patients using routinely recorded electronic medical record data. Methods: We used routinely recorded data from 5 Australian hospitals to develop and internally-externally validate a prediction model for inpatient falls using a Cox proportional hazards model with time-varying covariates. The study cohort included patients admitted during 2018-2021 to any ward, with no age restriction. Predictors used in the model included admission-related administrative data, length of stay, and number of previous falls during the admission (updated every 12 hours up to 14 days after admission). Model calibration was assessed using Poisson regression and discrimination using the area under the time-dependent receiver operating characteristic curve. Results: There were 1,107,556 inpatient admissions, 6004 falls, and 5341 unique fallers. The area under the time-dependent receiver operating characteristic curve was 0.899 (95% CI 0.88-0.91) at 24 hours after admission and declined throughout admission (eg, 0.765, 95% CI 0.75-0.78 on the seventh day after admission). Site-dependent overestimation and underestimation of risk was observed on the calibration plots. Conclusions: Using a large dataset from multiple hospitals and robust methods to model development and validation, we developed a prognostic model for inpatient falls. It had high discrimination, suggesting the model has the potential for operationalization in clinical decision support for prioritizing inpatients for fall prevention. Performance was site dependent, and model recalibration may lead to improved performance. %M 39536309 %R 10.2196/59634 %U https://www.jmir.org/2024/1/e59634 %U https://doi.org/10.2196/59634 %U http://www.ncbi.nlm.nih.gov/pubmed/39536309 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e51412 %T Influence of Blood Sampling Service Process Reengineering on Medical Services Supply: Quasi-Experimental Study %A Liao,Wenmin %A He,Rong %A He,Zhonglian %A Shi,Nan %A Li,Dan %A Zhuang,Aihua %A Gan,Feng %A Sun,Ying %A Li,Chaofeng %+ State Key Laboratory of Oncology, Guangdong Provincial Clinical Research Center for Cancer, Sun Yat-sen University Cancer Center, Dongfengdong road 651#, Guangzhou, 510060, China, 86 87343292, lichaofeng@sysucc.org.cn %K process reengineering %K blood sampling %K hospital administration %K medical informatics %K digital health %K patient experience %D 2024 %7 12.11.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Tertiary hospitals in China are confronted with significant challenges due to limited spatial capacity and workforce constraints, leading to saturated allocation of medical resources and restricted growth in medical service provision. The incorporation of digital health into medical service process reengineering (MSPR) marks a pivotal transformation and restructuring of conventional health service delivery models. Specifically, the application of MSPR to blood sampling services processes reengineering (BSSPR) holds promise for substantially enhancing the efficiency and quality of medical services through streamlining and optimizing these procedures. However, the comprehensive impact of BSSPR has been infrequently quantified in existing research. Objective: This study aims to investigate the influence of BSSPR on the efficiency and quality of medical services and to elucidate the key informative technological support points underpinning BSSPR. Methods: Data were collected from both the new and old laboratory information systems from August 1, 2019, to December 31, 2021. A combination of statistical description, chi-square test, and t test was used to compare check-in time and waiting time of outpatients before and after the implementation of BSSPR. An interrupted time-series design was used to analyze the impact of BSSPR on medical service efficiency and quality, enabling the control of confounding variables, including changes in medical human resources and both long- and short-term temporal trends. Results: BSSPR had an impact on the efficiency and quality of medical services. Notably, there was a significant increase in the number of patients receiving blood sampling services, with a daily service volume increase of ~150 individuals (P=.04). The average waiting time for patients decreased substantially from 29 (SD 36) to 11 (SD 11) minutes, indicating a marked improvement in patient experience. During the peak period, the number of patients receiving blood sampling services per working hour statistically increased from 9.56 to 16.77 (P<.001). The interrupted time-series model results demonstrated a reduction in patients’ waiting time by an average of 26.1 (SD 3.8; 95% CI –33.64 to –18.57) minutes. Although there was an initial decline in the number of outpatients admitted following BSSPR implementation, an upward trend was observed over time (β=1.13, 95% CI 0.91-1.36). Conclusions: BSSPR implementation for outpatients not only reduced waiting time and improved patients’ experience but also augmented the hospital’s capacity to provide medical services. This study’s findings offer valuable insights into the potential advantages of BSSPR and underscore the significance of harnessing digital technologies to optimize medical service processes. This research serves as a foundational basis and provides scientific support for the promotion and application of BSSPR in other health care contexts. By continuing to explore and refine the integration of digital technologies in health care, we can further enhance patient outcomes and elevate the overall quality of medical services. %M 39531265 %R 10.2196/51412 %U https://www.jmir.org/2024/1/e51412 %U https://doi.org/10.2196/51412 %U http://www.ncbi.nlm.nih.gov/pubmed/39531265 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e54018 %T In-Depth Examination of the Functionality and Performance of the Internet Hospital Information Platform: Development and Usability Study %A Zhang,Guang-Wei %A Li,Bin %A Gu,Zheng-Min %A Yang,Wei-Feng %A Wang,Yi-Ran %A Li,Hui-Jun %A Zheng,Han-Bing %A Yue,Ying-Xu %A Wang,Kui-Zhong %A Gong,Mengchun %A Gong,Da-Xin %+ Department of Smart Hospital, The First Hospital of China Medical University, 155 Nanjingbei Street, Shenyang, 110001, China, 86 02483283350, ydyyzhyy@163.com %K internet hospital %K smart hospital %K mobile applications %K operational data %K information system %K online service %K patient service %K management tool %K electronic prescriptions %K medical education %K integration %D 2024 %7 8.11.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Internet hospitals (IHs) have rapidly developed as a promising strategy to address supply-demand imbalances in China’s medical industry, with their capabilities directly dependent on information platform functionality. Furthermore, a novel theory of “Trinity” smart hospital has provided advanced guidelines on IH constructions. Objective: This study aimed to explore the construction experience, construction models, and development prospects based on operational data from IHs. Methods: Based on existing information systems and internet service functionalities, our hospital has built a “Smart Hospital Internet Information Platform (SHIIP)” for IH operations, actively to expand online services, digitalize traditional health care, and explore health care services modes throughout the entire process and lifecycle. This article encompasses the platform architecture design, technological applications, patient service content and processes, health care professional support features, administrative management tools, and associated operational data. Results: Our platform has presented a set of data, including 82,279,669 visits, 420,120 online medical consultations, 124,422 electronic prescriptions, 92,285 medication deliveries, 6,965,566 prediagnosis triages, 4,995,824 offline outpatient appointments, 2025 medical education articles with a total of 15,148,310 views, and so on. These data demonstrate the significant role of IH as an indispensable component of our physical hospital services, with deep integration between online and offline health care systems. Conclusions: The upward trends in various data metrics indicate that our IH has gained significant recognition and usage among both the public and healthcare workers, and may have promising development prospects. Additionally, the platform construction approach, which prioritizes comprehensive service digitization and the 'Trinity' of the public, healthcare workers, and managers, serves as an effective means of promoting the development of Internet Hospitals. Such insights may prove invaluable in guiding the development of IH and facilitating the continued evolution of the Internet healthcare sector. %M 39168813 %R 10.2196/54018 %U https://www.jmir.org/2024/1/e54018 %U https://doi.org/10.2196/54018 %U http://www.ncbi.nlm.nih.gov/pubmed/39168813 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e54022 %T Patterns and Perceptions of Standard Order Set Use Among Physicians Working Within a Multihospital System: Mixed Methods Study %A Naicker,Sundresan %A Tariq,Amina %A Donovan,Raelene %A Magon,Honor %A White,Nicole %A Simmons,Joshua %A McPhail,Steven M %+ Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology, 60 Musk Avenue, Kelvin Grove, 4059, Australia, 61 449876034, sundresan.naicker@qut.edu.au %K medical informatics %K adoption and implementation %K behavior %K health systems %K testing %K electronic medical records %K behavioral model %K quantitative data %K semistructured interview %K clinical practice %K user preference %K user %K user experience %D 2024 %7 8.11.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Electronic standard order sets automate the ordering of specific treatment, testing, and investigative protocols by physicians. These tools may help reduce unwarranted clinical variation and improve health care efficiency. Despite their routine implementation within electronic medical records (EMRs), little is understood about how they are used and what factors influence their adoption in practice. Objective: This study aims to (1) describe the patterns of use of standard order sets implemented in a widely used EMR (PowerPlans and Cerner Millennium) within a multihospital digital health care system; (2) explore the experiences and perceptions of implementers and users regarding the factors contributing to the use of these standard order sets; and (3) map these findings to the Capability, Opportunity, and Motivation Behavior (COM-B) model of behavior change to assist those planning to develop, improve, implement, and iterate the use of standard order sets in hospital settings. Methods: Quantitative data on standard order set usage were captured from 5 hospitals over 5-month intervals for 3 years (2019, 2020, and 2021). Qualitative data, comprising unstructured and semistructured interviews (n=15), were collected and analyzed using a reflexive thematic approach. Interview themes were then mapped to a theory-informed model of behavior change (COM-B) to identify determinants of standard order set usage in routine clinical practice. The COM-B model is an evidence-based, multicomponent framework that posits that human actions result from multiple contextual influences, which can be categorized across 3 dimensions: capability, opportunity, and motivation, all of which intersect. Results: The total count of standard order set usage across the health system during the 2019 observation period was 267,253, increasing to 293,950 in 2020 and 335,066 in 2021. There was a notable shift toward using specialty order sets that received upgrades during the study period. Four emergent themes related to order set use were derived from clinician interviews: (1) Knowledge and Skills; (2) Perceptions; (3) Technical Dependencies; and (4) Unintended Consequences, all of which were mapped to the COM-B model. Findings indicate a user preference for customized order sets that respond to local context and user experience. Conclusions: The study findings suggest that ongoing investment in the development and functionality of specialty order sets has the potential to enhance usage as these sets continue to be customized in response to local context and user experience. Sustained and continuous uptake of appropriate Computerized Provider Order Entry use may require implementation strategies that address the capability, opportunity, and motivational influencers of behavior. %M 39514274 %R 10.2196/54022 %U https://formative.jmir.org/2024/1/e54022 %U https://doi.org/10.2196/54022 %U http://www.ncbi.nlm.nih.gov/pubmed/39514274 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e58413 %T Development and Validation of Deep Learning–Based Infectivity Prediction in Pulmonary Tuberculosis Through Chest Radiography: Retrospective Study %A Chung,Wou young %A Yoon,Jinsik %A Yoon,Dukyong %A Kim,Songsoo %A Kim,Yujeong %A Park,Ji Eun %A Kang,Young Ae %+ Department of Internal Medicine, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea, 82 2 2228 1954, mdkang@yuhs.ac %K pulmonary tuberculosis %K chest radiography %K artificial intelligence %K tuberculosis %K TB %K smear %K smear test %K culture test %K diagnosis %K treatment %K deep learning %K CXR %K PTB %K management %K cost effective %K asymptomatic infection %K diagnostic tools %K infectivity %K AI tool %K cohort %D 2024 %7 7.11.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Pulmonary tuberculosis (PTB) poses a global health challenge owing to the time-intensive nature of traditional diagnostic tests such as smear and culture tests, which can require hours to weeks to yield results. Objective: This study aimed to use artificial intelligence (AI)–based chest radiography (CXR) to evaluate the infectivity of patients with PTB more quickly and accurately compared with traditional methods such as smear and culture tests. Methods: We used DenseNet121 and visualization techniques such as gradient-weighted class activation mapping and local interpretable model-agnostic explanations to demonstrate the decision-making process of the model. We analyzed 36,142 CXR images of 4492 patients with PTB obtained from Severance Hospital, focusing specifically on the lung region through segmentation and cropping with TransUNet. We used data from 2004 to 2020 to train the model, data from 2021 for testing, and data from 2022 to 2023 for internal validation. In addition, we used 1978 CXR images of 299 patients with PTB obtained from Yongin Severance Hospital for external validation. Results: In the internal validation, the model achieved an accuracy of 73.27%, an area under the receiver operating characteristic curve of 0.79, and an area under the precision-recall curve of 0.77. In the external validation, it exhibited an accuracy of 70.29%, an area under the receiver operating characteristic curve of 0.77, and an area under the precision-recall curve of 0.8. In addition, gradient-weighted class activation mapping and local interpretable model-agnostic explanations provided insights into the decision-making process of the AI model. Conclusions: This proposed AI tool offers a rapid and accurate alternative for evaluating PTB infectivity through CXR, with significant implications for enhancing screening efficiency by evaluating infectivity before sputum test results in clinical settings, compared with traditional smear and culture tests. %M 39509691 %R 10.2196/58413 %U https://www.jmir.org/2024/1/e58413 %U https://doi.org/10.2196/58413 %U http://www.ncbi.nlm.nih.gov/pubmed/39509691 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e58276 %T Clinical Decision Support to Increase Emergency Department Naloxone Coprescribing: Implementation Report %A Sommers,Stuart W %A Tolle,Heather J %A Trinkley,Katy E %A Johnston,Christine G %A Dietsche,Caitlin L %A Eldred,Stephanie V %A Wick,Abraham T %A Hoppe,Jason A %K clinical decision support systems %K order sets %K drug monitoring %K opioid analgesic %K opioid use %K opioid prescribing %K drug overdose %K opioid overdose %K naloxone %K naloxone coprescribing %K harm reduction %K harm minimization %D 2024 %7 6.11.2024 %9 %J JMIR Med Inform %G English %X Background: Coprescribing naloxone with opioid analgesics is a Centers for Disease Control and Prevention (CDC) best practice to mitigate the risk of fatal opioid overdose, yet coprescription by emergency medicine clinicians is rare, occurring less than 5% of the time it is indicated. Clinical decision support (CDS) has been associated with increased naloxone prescribing; however, key CDS design characteristics and pragmatic outcome measures necessary to understand replicability and effectiveness have not been reported. Objective: This study aimed to rigorously evaluate and quantify the impact of CDS designed to improve emergency department (ED) naloxone coprescribing. We hypothesized CDS would increase naloxone coprescribing and the number of naloxone prescriptions filled by patients discharged from EDs in a large health care system. Methods: Following user-centered design principles, we designed and implemented a fully automated, interruptive, electronic health record–based CDS to nudge clinicians to coprescribe naloxone with high-risk opioid prescriptions. “High-risk” opioid prescriptions were defined as any opioid analgesic prescription ≥90 total morphine milligram equivalents per day or for patients with a prior diagnosis of opioid use disorder or opioid overdose. The Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework was used to evaluate pragmatic CDS outcomes of reach, effectiveness, adoption, implementation, and maintenance. Effectiveness was the primary outcome of interest and was assessed by (1) constructing a Bayesian structural time-series model of the number of ED visits with naloxone coprescriptions before and after CDS implementation and (2) calculating the percentage of naloxone prescriptions associated with CDS that were filled at an outpatient pharmacy. Mann-Kendall tests were used to evaluate longitudinal trends in CDS adoption. All outcomes were analyzed in R (version 4.2.2; R Core Team). Implementation (Results): Between November 2019 and July 2023, there were 1,994,994 ED visits. CDS reached clinicians in 0.83% (16,566/1,994,994) of all visits and 15.99% (16,566/103,606) of ED visits where an opioid was prescribed at discharge. Clinicians adopted CDS, coprescribing naloxone in 34.36% (6613/19,246) of alerts. CDS was effective, increasing naloxone coprescribing from baseline by 18.1 (95% CI 17.9‐18.3) coprescriptions per week or 2,327% (95% CI 3390‐3490). Patients filled 43.80% (1989/4541) of naloxone coprescriptions. The CDS was implemented simultaneously at every ED and no adaptations were made to CDS postimplementation. CDS was maintained beyond the study period and maintained its effect, with adoption increasing over time (τ=0.454; P<.001). Conclusions: Our findings advance the evidence that electronic health record–based CDS increases the number of naloxone coprescriptions and improves the distribution of naloxone. Our time series analysis controls for secular trends and strongly suggests that minimally interruptive CDS significantly improves process outcomes. %R 10.2196/58276 %U https://medinform.jmir.org/2024/1/e58276 %U https://doi.org/10.2196/58276 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 11 %N %P e58068 %T Building Mutually Beneficial Collaborations Between Digital Navigators, Mental Health Professionals, and Clients: Naturalistic Observational Case Study %A Gorban,Carla %A McKenna,Sarah %A Chong,Min K %A Capon,William %A Battisti,Robert %A Crowley,Alison %A Whitwell,Bradley %A Ottavio,Antonia %A Scott,Elizabeth M %A Hickie,Ian B %A Iorfino,Frank %K digital navigator %K digital coach %K clinical technology specialist %K mental health services %K shared decision-making %K lived experience %K implementation %K poor engagement %K decision-making %K mental health %K digital mental health %K digital mental health technology %D 2024 %7 6.11.2024 %9 %J JMIR Ment Health %G English %X Despite the efficacy of digital mental health technologies (DMHTs) in clinical trials, low uptake and poor engagement are common in real-world settings. Accordingly, digital technology experts or “digital navigators” are increasingly being used to enhance engagement and shared decision-making between health professionals and clients. However, this area is relatively underexplored and there is a lack of data from naturalistic settings. In this paper, we report observational findings from the implementation of a digital navigator in a multidisciplinary mental health clinic in Sydney, Australia. The digital navigator supported clients and health professionals to use a measurement-based DMHT (the Innowell platform) for improved multidimensional outcome assessment and to guide personalized decision-making. Observational data are reported from implementation logs, platform usage statistics, and response rates to digital navigator emails and phone calls. Ultimately, support from the digital navigator led to improved data collection and clearer communications about goals for using the DMHT to track client outcomes; however, this required strong partnerships between health professionals, the digital navigator, and clients. The digital navigator helped to facilitate the integration of DMHT into care, rather than providing a stand-alone service. Thus, collaborations between health professionals and digital navigators are mutually beneficial and empower clients to be more engaged in their own care. %R 10.2196/58068 %U https://mental.jmir.org/2024/1/e58068 %U https://doi.org/10.2196/58068 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e58130 %T Electronic Health Record Data Quality and Performance Assessments: Scoping Review %A Penev,Yordan P %A Buchanan,Timothy R %A Ruppert,Matthew M %A Liu,Michelle %A Shekouhi,Ramin %A Guan,Ziyuan %A Balch,Jeremy %A Ozrazgat-Baslanti,Tezcan %A Shickel,Benjamin %A Loftus,Tyler J %A Bihorac,Azra %K electronic health record %K EHR %K record %K data quality %K data performance %K clinical informatics %K performance %K data science %K synthesis %K review methods %K review methodology %K search %K scoping %D 2024 %7 6.11.2024 %9 %J JMIR Med Inform %G English %X Background: Electronic health records (EHRs) have an enormous potential to advance medical research and practice through easily accessible and interpretable EHR-derived databases. Attainability of this potential is limited by issues with data quality (DQ) and performance assessment. Objective: This review aims to streamline the current best practices on EHR DQ and performance assessments as a replicable standard for researchers in the field. Methods: PubMed was systematically searched for original research articles assessing EHR DQ and performance from inception until May 7, 2023. Results: Our search yielded 26 original research articles. Most articles had 1 or more significant limitations, including incomplete or inconsistent reporting (n=6, 30%), poor replicability (n=5, 25%), and limited generalizability of results (n=5, 25%). Completeness (n=21, 81%), conformance (n=18, 69%), and plausibility (n=16, 62%) were the most cited indicators of DQ, while correctness or accuracy (n=14, 54%) was most cited for data performance, with context-specific supplementation by recency (n=7, 27%), fairness (n=6, 23%), stability (n=4, 15%), and shareability (n=2, 8%) assessments. Artificial intelligence–based techniques, including natural language data extraction, data imputation, and fairness algorithms, were demonstrated to play a rising role in improving both dataset quality and performance. Conclusions: This review highlights the need for incentivizing DQ and performance assessments and their standardization. The results suggest the usefulness of artificial intelligence–based techniques for enhancing DQ and performance to unlock the full potential of EHRs to improve medical research and practice. %R 10.2196/58130 %U https://medinform.jmir.org/2024/1/e58130 %U https://doi.org/10.2196/58130 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 10 %N %P e60001 %T Spatial Analyses of Crisis Pregnancy Centers and Abortion Facilities in the United States, 2021 (Pre-Dobbs): Cross-Sectional Study %A Swartzendruber,Andrea %A Luisi,Nicole %A Johnson,Erin R %A Lambert,Danielle N %+ Department of Epidemiology and Biostatistics, College of Public Health, University of Georgia, 101 Buck Road, Athens, GA, 30602, United States, 1 706 583 8149, aswartz@uga.edu %K crisis pregnancy center %K abortion, induced %K reproductive health %K policy %K access to information %K internet %K directory %K geographic information system %K spatial analyses %D 2024 %7 6.11.2024 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: Crisis pregnancy centers (CPCs) are religious nonprofit organizations with a primary mission of diverting people from having abortions. One CPC tactic has been to locate near abortion facilities. Despite medical groups’ warnings that CPCs do not adhere to medical and ethical standards and pose risks, government support for CPCs has significantly increased. Objective: This study aims to map CPCs, abortion facilities, and geographical areas in the United States into 4 zones based on their proximity to CPCs and abortion facilities. We sought to describe the number and percentage of reproductive-aged women living in each zone and the proximity of CPCs to abortion facilities. Methods: Using 2021 data from CPC Map and the Advancing New Standards in Reproductive Health Abortion Facility Database, we determined the ratio of CPCs to abortion facilities. Along with census data, we categorized and mapped US block groups into 4 distinct zones based on locations of block group centroids within 15-mile (1 mile is approximately 1.609 km) radii of CPCs and abortion facilities, namely “no presence,” “CPC only,” “abortion facility only,” and “dual presence.” We calculated the number and percentage of block groups and reproductive-aged (15-49 years) women living in each zone. We calculated driving distances and drive times from abortion facilities to the nearest CPC and mapped abortion facilities with CPCs in close proximity. All analyses were conducted nationally and by region, division, and state. Results: Nationally, the ratio of CPCs to abortion facilities was 3.4, and 54.9% (131,410/239,462) of block groups were categorized in the “dual presence” zone, 26.6% (63,679/239,462) as “CPC only,” and 0.8% (63,679/239,462) as “abortion facility only.” Most reproductive-aged women (45,150,110/75,582,028, 59.7%) lived in a “dual presence” zone, 26.1% (19,696,572/75,582,028) in a “CPC only” zone, and 0.8% (625,403/75,582,028) in an “abortion facility only” zone. The number of block groups and women classified as living in each zone varied by region, division, and state. Nationally, the median distance from abortion facilities to the nearest CPC was 2 miles, and the median drive time was 5.5 minutes. Minimum drive times were <1 minute in all but 11 states. The percentages of abortion facilities with a CPC within 0.25, 0.5, 1, and 3 miles were 14.1% (107/757), 22.6% (171/757), 36.1% (273/757), and 66.3% (502/757), respectively. Conclusions: The findings suggest that CPCs’ tactic of locating near abortion facilities was largely realized before the 2022 US Supreme Court decision that overturned the federal right to abortion. Research on CPCs’ locations and tactics should continue given the dynamic abortion policy landscape and risks posed by CPCs. Tailored programming to raise awareness about CPCs and help people identify and access safe sources of health care may mitigate harm. Increased regulation of CPCs and government divestment may also mitigate CPC harms. %M 39504544 %R 10.2196/60001 %U https://publichealth.jmir.org/2024/1/e60001 %U https://doi.org/10.2196/60001 %U http://www.ncbi.nlm.nih.gov/pubmed/39504544 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e55218 %T A Food Intake Estimation System Using an Artificial Intelligence–Based Model for Estimating Leftover Hospital Liquid Food in Clinical Environments: Development and Validation Study %A Tagi,Masato %A Hamada,Yasuhiro %A Shan,Xiao %A Ozaki,Kazumi %A Kubota,Masanori %A Amano,Sosuke %A Sakaue,Hiroshi %A Suzuki,Yoshiko %A Konishi,Takeshi %A Hirose,Jun %+ Medical Informatics, Institute of Biomedical Sciences, Tokushima University Graduate School, 3-18-15, Kuramoto-cho, Tokushima, 7708503, Japan, 81 88 633 9178, tagi@tokushima-u.ac.jp %K artificial intelligence %K machine learning %K system development %K food intake %K dietary intake %K dietary assessment %K food consumption %K image visual estimation %K AI estimation %K direct visual estimation %D 2024 %7 5.11.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Medical staff often conduct assessments, such as food intake and nutrient sufficiency ratios, to accurately evaluate patients’ food consumption. However, visual estimations to measure food intake are difficult to perform with numerous patients. Hence, the clinical environment requires a simple and accurate method to measure dietary intake. Objective: This study aims to develop a food intake estimation system through an artificial intelligence (AI) model to estimate leftover food. The accuracy of the AI’s estimation was compared with that of visual estimation for liquid foods served to hospitalized patients. Methods: The estimations were evaluated by a dietitian who looked at the food photo (image visual estimation) and visual measurement evaluation was carried out by a nurse who looked directly at the food (direct visual estimation) based on actual measurements. In total, 300 dishes of liquid food (100 dishes of thin rice gruel, 100 of vegetable soup, 31 of fermented milk, and 18, 12, 13, and 26 of peach, grape, orange, and mixed juices, respectively) were used. The root-mean-square error (RMSE) and coefficient of determination (R2) were used as metrics to determine the accuracy of the evaluation process. Corresponding t tests and Spearman rank correlation coefficients were used to verify the accuracy of the measurements by each estimation method with the weighing method. Results: The RMSE obtained by the AI estimation approach was 8.12 for energy. This tended to be smaller and larger than that obtained by the image visual estimation approach (8.49) and direct visual estimation approach (4.34), respectively. In addition, the R2 value for the AI estimation tended to be larger and smaller than the image and direct visual estimations, respectively. There was no difference between the AI estimation (mean 71.7, SD 23.9 kcal, P=.82) and actual values with the weighing method. However, the mean nutrient intake from the image visual estimation (mean 75.5, SD 23.2 kcal, P<.001) and direct visual estimation (mean 73.1, SD 26.4 kcal, P=.007) were significantly different from the actual values. Spearman rank correlation coefficients were high for energy (ρ=0.89-0.97), protein (ρ=0.94-0.97), fat (ρ=0.91-0.94), and carbohydrate (ρ=0.89-0.97). Conclusions: The measurement from the food intake estimation system by an AI-based model to estimate leftover liquid food intake in patients showed a high correlation with the actual values with the weighing method. Furthermore, it also showed a higher accuracy than the image visual estimation. The errors of the AI estimation method were within the acceptable range of the weighing method, which indicated that the AI-based food intake estimation system could be applied in clinical environments. However, its lower accuracy than that of direct visual estimation was still an issue. %M 39500491 %R 10.2196/55218 %U https://formative.jmir.org/2024/1/e55218 %U https://doi.org/10.2196/55218 %U http://www.ncbi.nlm.nih.gov/pubmed/39500491 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 10 %N %P e64270 %T Digital Contact Tracing Implementation Among Leaders and Health Care Workers in a Pediatric Hospital During the COVID-19 Pandemic: Qualitative Interview Study %A O'Dwyer,Brynn %A Jaana,Mirou %A Hui,Charles %A Chreim,Samia %A Ellis,Jennifer %+ Telfer School of Management, University of Ottawa, 55 Laurier Ave E, Ottawa, ON, K1N 6N5, Canada, 1 6135625731 ext 3400, jaana@telfer.uottawa.ca %K COVID-19 %K surveillance %K technology %K digital contact tracing %K qualitative %K hospitals %K Reach, Effectiveness, Adoption, Implementation, and Maintenance framework %K RE-AIM %D 2024 %7 5.11.2024 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: Health systems had to rapidly implement infection control strategies to sustain their workforces during the COVID-19 pandemic. Various outbreak response tools, such as digital contact tracing (DCT), have been developed to monitor exposures and symptoms of health care workers (HCWs). Limited research evidence exists on the experiences with these technologies and the impacts of DCT innovations from the perspective of stakeholders in health care environments. Objective: This study aims to identify the factors influencing the adoption of DCT, highlight variations in perspectives across 3 key stakeholder groups concerning the impact of DCT, and provide benchmarking evidence for future pandemic preparedness. Methods: Guided by the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework, we conducted an exploratory qualitative study to investigate the implementation and impact of DCT at the Children’s Hospital of Eastern Ontario between December 2022 and April 2023. We conducted 21 semistructured interviews with key stakeholders, including health care administrators (6/21, 29%), occupational health and safety specialists (8/21, 38%), and HCWs (7/21, 33%). Stakeholders were asked about the factors influencing engagement with the DCT tool, organizational-level uptake, the implementation process, long-term use and sustainability of DCT, and unintended consequences. Verbatim transcripts were subject to thematic analysis using NVivo (QSR International). Results: The implementation of DCT was viable and well received. End users indicated that their engagement with the DCT tool was facilitated by its perceived ease of use and the ability to gain awareness of probable COVID-19 exposures; however, risk assessment consequences and access concerns were reported as barriers (reach). Participants commonly agreed that the DCT technology had a positive influence on the hospital’s capacity to meet the demands of COVID-19 (effectiveness). Implementors and occupational specialists referred to negative staffing impacts and the loss of nuanced information as unintended consequences (effectiveness). Safety-focused communication strategies and having a DCT tool that was human-centered were crucial factors driving staff adoption of the technology. Conversely, adoption was challenged by the misaligned delivery of the DCT tool with HCWs’ standard practices, alongside the evolving perceived threat of COVID-19. Stakeholders collectively agreed on the viability of DCT and its applicability to infectious disease practices (maintenance). Conclusions: Hospital stakeholders were highly satisfied with DCT technology and it was perceived as feasible, efficient, and having a positive impact on organizational safety. Challenges related to the alignment and delivery of DCT, alongside the evolving perspectives on COVID-19, posed obstacles to continued adoption by HCWs. Our findings contribute to evidence-based practices and present benchmarks that can inform preparedness for future pandemics and infectious disease outbreaks and help other organizations implement similar technologies. %M 39499919 %R 10.2196/64270 %U https://publichealth.jmir.org/2024/1/e64270 %U https://doi.org/10.2196/64270 %U http://www.ncbi.nlm.nih.gov/pubmed/39499919 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e50631 %T Use of Artificial Intelligence in Cobb Angle Measurement for Scoliosis: Retrospective Reliability and Accuracy Study of a Mobile App %A Li,Haodong %A Qian,Chuang %A Yan,Weili %A Fu,Dong %A Zheng,Yiming %A Zhang,Zhiqiang %A Meng,Junrong %A Wang,Dahui %+ Department of Orthopedics, Children’s Hospital of Fudan University, National Children’s Medical Center, Wanyuan Rd, Minhang District, Shanghai, 201102, China, 86 02164931101, wangdahui@fudan.edu.cn %K scoliosis %K photogrammetry %K artificial intelligence %K deep learning %D 2024 %7 1.11.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Scoliosis is a spinal deformity in which one or more spinal segments bend to the side or show vertebral rotation. Some artificial intelligence (AI) apps have already been developed for measuring the Cobb angle in patients with scoliosis. These apps still require doctors to perform certain measurements, which can lead to interobserver variability. The AI app (cobbAngle pro) in this study will eliminate the need for doctor measurements, achieving complete automation. Objective: We aimed to evaluate the reliability and accuracy of our new AI app that is based on deep learning to automatically measure the Cobb angle in patients with scoliosis. Methods: A retrospective analysis was conducted on the clinical data of children with scoliosis who were treated at the Pediatric Orthopedics Department of the Children’s Hospital affiliated with Fudan University from July 2019 to July 2022. Three measurers used the Picture Archiving and Communication System (PACS) to measure the coronal main curve Cobb angle in 802 full-length anteroposterior and lateral spine X-rays of 601 children with scoliosis, and recorded the results of each measurement. After an interval of 2 weeks, the mobile AI app was used to remeasure the Cobb angle once. The Cobb angle measurements from the PACS were used as the reference standard, and the accuracy of the Cobb angle measurements by the app was analyzed through the Bland-Altman test. The intraclass correlation coefficient (ICC) was used to compare the repeatability within measurers and the consistency between measurers. Results: Among 601 children with scoliosis, 89 were male and 512 were female (age range: 10-17 years), and 802 full-length spinal X-rays were analyzed. Two functionalities of the app (photography and photo upload) were compared with the PACS for measuring the Cobb angle. The consistency was found to be excellent. The average absolute errors of the Cobb angle measured by the photography and upload methods were 2.00 and 2.08, respectively. Using a clinical allowance maximum error of 5°, the 95% limits of agreement (LoAs) for Cobb angle measurements by the photography and upload methods were –4.7° to 4.9° and –4.9° to 4.9°, respectively. For the photography and upload methods, the 95% LoAs for measuring Cobb angles were –4.3° to 4.6° and –4.4° to 4.7°, respectively, in mild scoliosis patients; –4.9° to 5.2° and –5.1° to 5.1°, respectively, in moderate scoliosis patients; and –5.2° to 5.0° and –6.0° to 4.8°, respectively, in severe scoliosis patients. The Cobb angle measured by the 3 observers twice before and after using the photography method had good repeatability (P<.001). The consistency between the observers was excellent (P<.001). Conclusions: The new AI platform is accurate and repeatable in the automatic measurement of the Cobb angle of the main curvature in patients with scoliosis. %M 39486021 %R 10.2196/50631 %U https://www.jmir.org/2024/1/e50631 %U https://doi.org/10.2196/50631 %U http://www.ncbi.nlm.nih.gov/pubmed/39486021 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e55766 %T Health Care Professionals’ Experience of Using AI: Systematic Review With Narrative Synthesis %A Ayorinde,Abimbola %A Mensah,Daniel Opoku %A Walsh,Julia %A Ghosh,Iman %A Ibrahim,Siti Aishah %A Hogg,Jeffry %A Peek,Niels %A Griffiths,Frances %+ Division of Health Sciences, Warwick Medical School, University of Warwick, Medical School Building, Gibbet Hill Road, Coventry, CV4 7AL, United Kingdom, 44 2476151098, a.ayorinde.1@warwick.ac.uk %K artificial intelligence %K clinical decision support systems %K CDSS %K decision-making %K quality assessment %K clinician experience %K health care professionals %K health care delivery %D 2024 %7 30.10.2024 %9 Review %J J Med Internet Res %G English %X Background: There has been a substantial increase in the development of artificial intelligence (AI) tools for clinical decision support. Historically, these were mostly knowledge-based systems, but recent advances include non–knowledge-based systems using some form of machine learning. The ability of health care professionals to trust technology and understand how it benefits patients or improves care delivery is known to be important for their adoption of that technology. For non–knowledge-based AI tools for clinical decision support, these issues are poorly understood. Objective: The aim of this study is to qualitatively synthesize evidence on the experiences of health care professionals in routinely using non–knowledge-based AI tools to support their clinical decision-making. Methods: In June 2023, we searched 4 electronic databases, MEDLINE, Embase, CINAHL, and Web of Science, with no language or date limit. We also contacted relevant experts and searched reference lists of the included studies. We included studies of any design that reported the experiences of health care professionals using non–knowledge-based systems for clinical decision support in their work settings. We completed double independent quality assessment for all included studies using the Mixed Methods Appraisal Tool. We used a theoretically informed thematic approach to synthesize the findings. Results: After screening 7552 titles and 182 full-text articles, we included 25 studies conducted in 9 different countries. Most of the included studies were qualitative (n=13), and the remaining were quantitative (n=9) and mixed methods (n=3). Overall, we identified 7 themes: health care professionals’ understanding of AI applications, level of trust and confidence in AI tools, judging the value added by AI, data availability and limitations of AI, time and competing priorities, concern about governance, and collaboration to facilitate the implementation and use of AI. The most frequently occurring are the first 3 themes. For example, many studies reported that health care professionals were concerned about not understanding the AI outputs or the rationale behind them. There were issues with confidence in the accuracy of the AI applications and their recommendations. Some health care professionals believed that AI provided added value and improved decision-making, and some reported that it only served as a confirmation of their clinical judgment, while others did not find it useful at all. Conclusions: Our review identified several important issues documented in various studies on health care professionals’ use of AI tools in real-world health care settings. Opinions of health care professionals regarding the added value of AI tools for supporting clinical decision-making varied widely, and many professionals had concerns about their understanding of and trust in this technology. The findings of this review emphasize the need for concerted efforts to optimize the integration of AI tools in real-world health care settings. Trial Registration: PROSPERO CRD42022336359; https://tinyurl.com/2yunvkmb %M 39476382 %R 10.2196/55766 %U https://www.jmir.org/2024/1/e55766 %U https://doi.org/10.2196/55766 %U http://www.ncbi.nlm.nih.gov/pubmed/39476382 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e46983 %T Characterization of Telecare Conversations on Lifestyle Management and Their Relation to Health Care Utilization for Patients with Heart Failure: Mixed Methods Study %A Erdt,Mojisola %A Yusof,Sakinah Binte %A Chai,Liquan %A Md Salleh,Siti Umairah %A Liu,Zhengyuan %A Sarim,Halimah Binte %A Lim,Geok Choo %A Lim,Hazel %A Suhaimi,Nur Farah Ain %A Yulong,Lin %A Guo,Yang %A Ng,Angela %A Ong,Sharon %A Choo,Bryan Peide %A Lee,Sheldon %A Weiliang,Huang %A Oh,Hong Choon %A Wolters,Maria Klara %A Chen,Nancy F %A Krishnaswamy,Pavitra %+ Institute for Infocomm Research (I²R), Agency for Science, Technology and Research (A*STAR), 1 Fusionopolis Way, #21-01, Connexis South Tower, Singapore, 138632, Singapore, 65 6408 2450, pavitrak@i2r.a-star.edu.sg %K telehealth %K telecare %K heart failure %K chronic disease %K self-management %K lifestyle management %K behavior %K health care utilization %K conversation %K dialogue %K medical informatics %D 2024 %7 30.10.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Telehealth interventions where providers offer support and coaching to patients with chronic conditions such as heart failure (HF) and type 2 diabetes mellitus (T2DM) are effective in improving health outcomes. However, the understanding of the content and structure of these interactions and how they relate to health care utilization remains incomplete. Objective: This study aimed to characterize the content and structure of telecare conversations on lifestyle management for patients with HF and investigate how these conversations relate to health care utilization. Methods: We leveraged real-world data from 50 patients with HF enrolled in a postdischarge telehealth program, with the primary intervention comprising a series of telephone calls from nurse telecarers over a 12-month period. For the full cohort, we transcribed 729 English-language calls and annotated conversation topics. For a subcohort (25 patients with both HF and T2DM), we annotated lifestyle management content with fine-grained dialogue acts describing typical conversational structures. For each patient, we identified calls with unusually high ratios of utterances on lifestyle management as lifestyle-focused calls. We further extracted structured data for inpatient admissions from 6 months before to 6 months after the intervention period. First, to understand conversational structures and content of lifestyle-focused calls, we compared the number of utterances, dialogue acts, and symptom attributes in lifestyle-focused calls to those in calls containing but not focused on lifestyle management. Second, to understand the perspectives of nurse telecarers on these calls, we conducted an expert evaluation where 2 nurse telecarers judged levels of concern and follow-up actions for lifestyle-focused and other calls (not focused on lifestyle management content). Finally, we assessed how the number of lifestyle-focused calls relates to the number of admissions, and to the average length of stay per admission. Results: In comparative analyses, lifestyle-focused calls had significantly fewer utterances (P=.01) and more dialogue acts (Padj=.005) than calls containing but not focused on lifestyle management. Lifestyle-focused calls did not contain deeper discussions on clinical symptoms. These findings indicate that lifestyle-focused calls entail short, intense discussions with greater emphasis on understanding patient experience and coaching than on clinical content. In the expert evaluation, nurse telecarers identified 24.2% (29/120) of calls assessed as concerning enough for follow-up. For these 29 calls, nurse telecarers were more attuned to concerns about symptoms and vitals (19/29, 65.5%) than lifestyle management concerns (4/29, 13.8%). The number of lifestyle-focused calls a patient had was modestly (but not significantly) associated with a lower average length of stay for inpatient admissions (Spearman ρ=-0.30; Padj=.06), but not with the number of admissions (Spearman ρ=-0.03; Padj=.84). Conclusions: Our approach and findings offer novel perspectives on the content, structure, and clinical associations of telehealth conversations on lifestyle management for patients with HF. Hence, our study could inform ways to enhance telehealth programs for self-care management in chronic conditions. %M 39476370 %R 10.2196/46983 %U https://www.jmir.org/2024/1/e46983 %U https://doi.org/10.2196/46983 %U http://www.ncbi.nlm.nih.gov/pubmed/39476370 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e51711 %T Evolution of the “Internet Plus Health Care” Mode Enabled by Artificial Intelligence: Development and Application of an Outpatient Triage System %A Yang,Lingrui %A Pang,Jiali %A Zuo,Song %A Xu,Jian %A Jin,Wei %A Zuo,Feng %A Xue,Kui %A Xiao,Zhongzhou %A Peng,Xinwei %A Xu,Jie %A Zhang,Xiaofan %A Chen,Ruiyao %A Luo,Shuqing %A Zhang,Shaoting %A Sun,Xin %+ Clinical Research and Innovation Unit, Xinhua Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, 1665 Kongjiang Road, Shanghai, China, 86 02125077480, sunxin@xinhuamed.com.cn %K artificial intelligence %K triage system %K all department recommendation %K subspecialty department recommendation %K “internet plus healthcare” %K “internet plus health care” %D 2024 %7 30.10.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Although new technologies have increased the efficiency and convenience of medical care, patients still struggle to identify specialized outpatient departments in Chinese tertiary hospitals due to a lack of medical knowledge. Objective: The objective of our study was to develop a precise and subdividable outpatient triage system to improve the experiences and convenience of patient care. Methods: We collected 395,790 electronic medical records (EMRs) and 500 medical dialogue groups. The EMRs were divided into 3 data sets to design and train the triage model (n=387,876, 98%) and test (n=3957, 1%) and validate (n=3957, 1%) it. The triage system was altered based on the current BERT (Bidirectional Encoder Representations from Transformers) framework and evaluated by recommendation accuracies in Xinhua Hospital using the cancellation rates in 2021 and 2022, from October 29 to December 5. Finally, a prospective observational study containing 306 samples was conducted to compare the system’s performance with that of triage nurses, which was evaluated by calculating precision, accuracy, recall of the top 3 recommended departments (recall@3), and time consumption. Results: With 3957 (1%) records each, the testing and validation data sets achieved an accuracy of 0.8945 and 0.8941, respectively. Implemented in Xinhua Hospital, our triage system could accurately recommend 79 subspecialty departments and reduce the number of registration cancellations from 16,037 (3.83%) of the total 418,714 to 15,338 (3.53%) of the total 434200 (P<.05). In comparison to the triage system, the performance of the triage nurses was more accurate (0.9803 vs 0.9153) and precise (0.9213 vs 0.9049) since the system could identify subspecialty departments, whereas triage nurses or even general physicians can only recommend main departments. In addition, our triage system significantly outperformed triage nurses in recall@3 (0.6230 vs 0.5266; P<.001) and time consumption (10.11 vs 14.33 seconds; P<.001). Conclusions: The triage system demonstrates high accuracy in outpatient triage of all departments and excels in subspecialty department recommendations, which could decrease the cancellation rate and time consumption. It also improves the efficiency and convenience of clinical care to fulfill better the usage of medical resources, expand hospital effectiveness, and improve patient satisfaction in Chinese tertiary hospitals. %M 39476375 %R 10.2196/51711 %U https://www.jmir.org/2024/1/e51711 %U https://doi.org/10.2196/51711 %U http://www.ncbi.nlm.nih.gov/pubmed/39476375 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e53636 %T Question Answering for Electronic Health Records: Scoping Review of Datasets and Models %A Bardhan,Jayetri %A Roberts,Kirk %A Wang,Daisy Zhe %+ Department of Computer and Information Science and Engineering, University of Florida, 1889 Museum Rd, Gainesville, FL, 32606, United States, 1 3528716584, jayetri.bardhan@ufl.edu %K medical question answering %K electronic health record %K EHR %K electronic medical records %K EMR %K relational database %K knowledge graph %D 2024 %7 30.10.2024 %9 Review %J J Med Internet Res %G English %X Background: Question answering (QA) systems for patient-related data can assist both clinicians and patients. They can, for example, assist clinicians in decision-making and enable patients to have a better understanding of their medical history. Substantial amounts of patient data are stored in electronic health records (EHRs), making EHR QA an important research area. Because of the differences in data format and modality, this differs greatly from other medical QA tasks that use medical websites or scientific papers to retrieve answers, making it critical to research EHR QA. Objective: This study aims to provide a methodological review of existing works on QA for EHRs. The objectives of this study were to identify the existing EHR QA datasets and analyze them, study the state-of-the-art methodologies used in this task, compare the different evaluation metrics used by these state-of-the-art models, and finally elicit the various challenges and the ongoing issues in EHR QA. Methods: We searched for articles from January 1, 2005, to September 30, 2023, in 4 digital sources, including Google Scholar, ACL Anthology, ACM Digital Library, and PubMed, to collect relevant publications on EHR QA. Our systematic screening process followed PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. A total of 4111 papers were identified for our study, and after screening based on our inclusion criteria, we obtained 47 papers for further study. The selected studies were then classified into 2 non–mutually exclusive categories depending on their scope: “EHR QA datasets” and “EHR QA models.” Results: A systematic screening process obtained 47 papers on EHR QA for final review. Out of the 47 papers, 53% (n=25) were about EHR QA datasets, and 79% (n=37) papers were about EHR QA models. It was observed that QA on EHRs is relatively new and unexplored. Most of the works are fairly recent. In addition, it was observed that emrQA is by far the most popular EHR QA dataset, both in terms of citations and usage in other papers. We have classified the EHR QA datasets based on their modality, and we have inferred that Medical Information Mart for Intensive Care (MIMIC-III) and the National Natural Language Processing Clinical Challenges datasets (ie, n2c2 datasets) are the most popular EHR databases and corpuses used in EHR QA. Furthermore, we identified the different models used in EHR QA along with the evaluation metrics used for these models. Conclusions: EHR QA research faces multiple challenges, such as the limited availability of clinical annotations, concept normalization in EHR QA, and challenges faced in generating realistic EHR QA datasets. There are still many gaps in research that motivate further work. This study will assist future researchers in focusing on areas of EHR QA that have possible future research directions. %M 39475821 %R 10.2196/53636 %U https://www.jmir.org/2024/1/e53636 %U https://doi.org/10.2196/53636 %U http://www.ncbi.nlm.nih.gov/pubmed/39475821 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e59811 %T Perceptions Toward Using Artificial Intelligence and Technology for Asthma Attack Risk Prediction: Qualitative Exploration of Māori Views %A Jayamini,Widana Kankanamge Darsha %A Mirza,Farhaan %A Bidois-Putt,Marie-Claire %A Naeem,M Asif %A Chan,Amy Hai Yan %+ Department of Computer Science, School of Engineering, Computer and Mathematical Sciences, Auckland University of Technology, Building WZ, Level 8, 6th St Paul Street, Auckland, 1010, New Zealand, 64 210504680, darsha.jayamini@autuni.ac.nz %K asthma risk prediction %K artificial intelligence %K machine learning %K māori perceptions %K health system development %K mobile phone %D 2024 %7 30.10.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Asthma is a significant global health issue, impacting over 500,000 individuals in New Zealand and disproportionately affecting Māori communities in New Zealand, who experience worse asthma symptoms and attacks. Digital technologies, including artificial intelligence (AI) and machine learning (ML) models, are increasingly popular for asthma risk prediction. However, these AI models may underrepresent minority ethnic groups and introduce bias, potentially exacerbating disparities. Objective: This study aimed to explore the views and perceptions that Māori have toward using AI and ML technologies for asthma self-management, identify key considerations for developing asthma attack risk prediction models, and ensure Māori are represented in ML models without worsening existing health inequities. Methods: Semistructured interviews were conducted with 20 Māori participants with asthma, 3 male and 17 female, aged 18-76 years. All the interviews were conducted one-on-one, except for 1 interview, which was conducted with 2 participants. Altogether, 10 web-based interviews were conducted, while the rest were kanohi ki te kanohi (face-to-face). A thematic analysis was conducted to identify the themes. Further, sentiment analysis was carried out to identify the sentiments using a pretrained Bidirectional Encoder Representations from Transformers model. Results: We identified four key themes: (1) concerns about AI use, (2) interest in using technology to support asthma, (3) desired characteristics of AI-based systems, and (4) experience with asthma management and opportunities for technology to improve care. AI was relatively unfamiliar to many participants, and some of them expressed concerns about whether AI technology could be trusted, kanohi ki te kanohi interaction, and inadequate knowledge of AI and technology. These concerns are exacerbated by the Māori experience of colonization. Most of the participants were interested in using technology to support their asthma management, and we gained insights into user preferences regarding computer-based health care applications. Participants discussed their experiences, highlighting problems with health care quality and limited access to resources. They also mentioned the factors that trigger their asthma control level. Conclusions: The exploration revealed that there is a need for greater information about AI and technology for Māori communities and a need to address trust issues relating to the use of technology. Expectations in relation to computer-based applications for health purposes were expressed. The research outcomes will inform future investigations on AI and technology to enhance the health of people with asthma, in particular those designed for Indigenous populations in New Zealand. %M 39475765 %R 10.2196/59811 %U https://formative.jmir.org/2024/1/e59811 %U https://doi.org/10.2196/59811 %U http://www.ncbi.nlm.nih.gov/pubmed/39475765 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e58683 %T Virtual Hospitals and Patient Experience: Protocol for a Mixed Methods Observational Study %A Jackson,Tim Michael %A Ward,Kanesha %A Saad,Shannon %A White,Sarah J %A Poudel,Shila %A Raffan,Freya %A Amanatidis,Sue %A Bartyn,Jenna %A Hutchings,Owen %A Coiera,Enrico %A Chan,Kevin %A Lau,Annie Y S %+ Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, 75 Talavera Road, Sydney, 2113, Australia, 61 98502400, tim.jackson@mq.edu.au %K virtual care %K patient experience %K virtual hospital %K mixed method %K co-design %K barriers %K facilitators %K virtual services %D 2024 %7 29.10.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Virtual care is increasingly incorporated within routine health care settings to improve patient experience and access to care. A patient’s experience encompasses all the interactions an individual has with the health care system. This includes a greater emphasis on actively involving carers in the decisions and activities surrounding a patient’s health care. Objective: This study aimed to investigate the variety of health care delivery challenges encountered in a virtual hospital and explore potential ways to improve the patient experience. Methods: Focusing on acute respiratory, this protocol outlines a mixed methods study exploring the patient experience of a virtual hospital in Australia, Royal Prince Alfred Virtual Hospital (rpavirtual). We will use an exploratory mixed methods approach comprising of secondary data analysis, observations, interviews, and co-design focus groups. Participants will include patients, their carers, and health care workers who are involved in the acute respiratory virtual hospital model of care. Together, the data will be triangulated to explore views and experiences of using this model of care, as well as co-designing recommendations for further improvement. Results: Findings from this study will identify current barriers and facilitators to implementing virtual care, such as work-as-done versus work-as-imagined, equity of care, the role of carers, and patient safety during virtual care. As of August 2024, a total of 25 participants have been interviewed. Conclusions: This protocol outlines a mixed methods case study on the acute respiratory model of care from Australia’s first virtual hospital, rpavirtual. This study will collect the experiences of patients, carers, and health care workers to co-design a series of recommendations to improve the patient experience. International Registered Report Identifier (IRRID): DERR1-10.2196/58683 %M 39471375 %R 10.2196/58683 %U https://www.researchprotocols.org/2024/1/e58683 %U https://doi.org/10.2196/58683 %U http://www.ncbi.nlm.nih.gov/pubmed/39471375 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e53594 %T Provision of Digital Primary Health Care Services: Overview of Reviews %A Fava,Virgínia Maria Dalfior %A Lapão,Luís Velez %+ Centro de Estudos Estratégicos Antonio Ivo de Carvalho, Fundação Oswaldo Cruz (Fiocruz), Ministério da Saúde, Av. Brasil, 4036, 10º andar, sala 1001, Manguinhos, Rio de Janeiro, 21040-361, Brazil, 55 21 3882 9133, virginiafava@gmail.com %K primary health care %K digital health %K implementation %K health service quality %K patients’ clinical conditions %K digital skills %K mobile phone %D 2024 %7 29.10.2024 %9 Review %J J Med Internet Res %G English %X Background: Digital health is a growing field, and many digital interventions have been implemented on a large scale since the COVID-19 pandemic, mainly in primary health care (PHC). The development of digital health interventions and their application in PHC are encouraged by the World Health Organization. The increased number of published scientific papers on this topic has resulted in an overwhelming amount of information, but there is no overview of reviews to summarize this evidence. Objective: This study aims to provide policy makers, health managers, and researchers with a summary of evidence on digital interventions used in PHC. Methods: This overview of reviews searched the Web of Science and MEDLINE databases for systematic and scoping reviews on assessments of digital technologies implemented in PHC published from January 2007 to March 2023. Only reviews that addressed digital interventions whose targets were real patients or health care providers (HCPs) were included. Results: A total of 236 records were identified from the search strategy, of which 42 (17.8%) full-text papers were selected for analysis, and 18 (7.6%) reviews met the eligibility criteria. In total, 61% (11/18) of the reviews focused their analysis on specific digital health interventions (client-to-provider telemedicine, provider-to-provider telemedicine, health worker decision support systems, systems for tracking patients’ health status, client participation and self-care platforms, and provision of education and training to health workers), and 39% (7/18) of the reviews focused on specific topics related to PHC (preventive care, chronic disease management, behavioral health disorders, the COVID-19 pandemic, multicomponent PHC interventions, and care coordination). Most studies in the included reviews agreed on barriers to implementation, such as software and apps developed without involving end users, the lack of training of HCPs and patients in digital technology use, and the lack of reimbursement and billing strategies for remote consultations. However, they showed several mixed results related to health service quality and patients’ clinical conditions and behavior changes. Conclusions: Research in digital health applied to PHC is still concentrated in high-income countries, mainly in North America and Europe. The mixed results related to health service quality and patients’ clinical conditions or behavior changes may have been caused by deficiencies in the process of implementing digital interventions. It is necessary to examine the entire impact pathway and the causal relationship among implementation, health service quality, and clinical condition outcomes to support the spread of digital health in PHC settings. %M 39471374 %R 10.2196/53594 %U https://www.jmir.org/2024/1/e53594 %U https://doi.org/10.2196/53594 %U http://www.ncbi.nlm.nih.gov/pubmed/39471374 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e56898 %T Critical Success Factors and Acceptance of the Casemix System Implementation Within the Total Hospital Information System: Exploratory Factor Analysis of a Pilot Study %A Mustafa,Noor Khairiyah %A Ibrahim,Roszita %A Aizuddin,Azimatun Noor %A Aljunid,Syed Mohamed %A Awang,Zainudin %+ Department of Public Health Medicine, Faculty of Medicine, Universiti Kebangsaan Malaysia, 6th Floor, Pre-Clinical Block, Cheras, 56000, Malaysia, 60 391455887 ext 5888, roszita@ppukm.ukm.edu.my %K critical success factors %K exploratory factor analysis %K Casemix system %K acceptance %K Total Hospital Information System %D 2024 %7 29.10.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: The health care landscape is evolving rapidly due to rising costs, an aging population, and the increasing prevalence of diseases. To address these challenges, the Ministry of Health of Malaysia implemented transformation strategies such as the Casemix system and hospital information system to enhance health care quality, resource allocation, and cost-effectiveness. However, successful implementation relies not just on the technology itself but on the acceptance and engagement of the users involved. Objective: This study aims to develop and refine items of a quantitative instrument measuring the critical success factors influencing acceptance of Casemix system implementation within the Ministry of Health’s Total Hospital Information System (THIS). Methods: A cross-sectional pilot study collected data from medical doctors at a hospital equipped with the THIS in the federal territory of Putrajaya, Malaysia. This pilot study’s minimum sample size was 125, achieved through proportionate stratified random sampling. Data were collected using a web-based questionnaire adapted from the human, organization, and technology-fit evaluation framework and the technology acceptance model. The pilot data were analyzed using exploratory factor analysis (EFA), and the Cronbach α assessed internal reliability. Both analyses were conducted in SPSS (version 25.0; IBM Corp). Results: This study obtained 106 valid responses, equivalent to an 84.8% (106/125) response rate. The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.859, and the Bartlett test of sphericity yielded statistically significant results (P<.001). Principal component analysis identified 9 components explaining 84.07% of the total variance, surpassing the minimum requirement of 60%. In total, 9 unique slopes indicated the identification of 9 components through EFA. While no new components emerged from the other 7 constructs, only the organizational factors construct was divided into 2 components, later named organizational structure and organizational environment. In total, 98% (41/42) of the items had factor loadings of >0.6, leading to the removal of 1 item for the final instrument for the field study. EFA ultimately identified 8 main constructs influencing Casemix implementation within the THIS: system quality, information quality, service quality, organizational characteristics, perceived ease of use, perceived usefulness, intention to use, and acceptance. Internal reliability measured using the Cronbach α ranged from 0.914 to 0.969, demonstrating high reliability. Conclusions: This study provides insights into the complexities of EFA and the distinct dimensions underlying the constructs that influence Casemix system acceptance in the THIS. While the findings align with extensive technology acceptance literature, the results accentuate the necessity for further research to develop a consensus regarding the most critical factors for successful Casemix adoption. The developed instrument is a substantial step toward better understanding the multidimensional challenges of health care system transformations in Malaysia, postulating an underpinning for future fieldwork and broader application across other hospitals. %M 39470697 %R 10.2196/56898 %U https://formative.jmir.org/2024/1/e56898 %U https://doi.org/10.2196/56898 %U http://www.ncbi.nlm.nih.gov/pubmed/39470697 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e54246 %T A New Natural Language Processing–Inspired Methodology (Detection, Initial Characterization, and Semantic Characterization) to Investigate Temporal Shifts (Drifts) in Health Care Data: Quantitative Study %A Paiva,Bruno %A Gonçalves,Marcos André %A da Rocha,Leonardo Chaves Dutra %A Marcolino,Milena Soriano %A Lana,Fernanda Cristina Barbosa %A Souza-Silva,Maira Viana Rego %A Almeida,Jussara M %A Pereira,Polianna Delfino %A de Andrade,Claudio Moisés Valiense %A Gomes,Angélica Gomides dos Reis %A Ferreira,Maria Angélica Pires %A Bartolazzi,Frederico %A Sacioto,Manuela Furtado %A Boscato,Ana Paula %A Guimarães-Júnior,Milton Henriques %A dos Reis,Priscilla Pereira %A Costa,Felício Roberto %A Jorge,Alzira de Oliveira %A Coelho,Laryssa Reis %A Carneiro,Marcelo %A Sales,Thaís Lorenna Souza %A Araújo,Silvia Ferreira %A Silveira,Daniel Vitório %A Ruschel,Karen Brasil %A Santos,Fernanda Caldeira Veloso %A Cenci,Evelin Paola de Almeida %A Menezes,Luanna Silva Monteiro %A Anschau,Fernando %A Bicalho,Maria Aparecida Camargos %A Manenti,Euler Roberto Fernandes %A Finger,Renan Goulart %A Ponce,Daniela %A de Aguiar,Filipe Carrilho %A Marques,Luiza Margoto %A de Castro,Luís César %A Vietta,Giovanna Grünewald %A Godoy,Mariana Frizzo de %A Vilaça,Mariana do Nascimento %A Morais,Vivian Costa %+ Computer Science Department, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil, Street Daniel de Carvalho, 1846, apto 201, Belo Horizonte, 30431310, Brazil, 55 31999710134, angelfire7@gmail.com %K health care %K machine learning %K data drifts %K temporal drifts %D 2024 %7 28.10.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Proper analysis and interpretation of health care data can significantly improve patient outcomes by enhancing services and revealing the impacts of new technologies and treatments. Understanding the substantial impact of temporal shifts in these data is crucial. For example, COVID-19 vaccination initially lowered the mean age of at-risk patients and later changed the characteristics of those who died. This highlights the importance of understanding these shifts for assessing factors that affect patient outcomes. Objective: This study aims to propose detection, initial characterization, and semantic characterization (DIS), a new methodology for analyzing changes in health outcomes and variables over time while discovering contextual changes for outcomes in large volumes of data. Methods: The DIS methodology involves 3 steps: detection, initial characterization, and semantic characterization. Detection uses metrics such as Jensen-Shannon divergence to identify significant data drifts. Initial characterization offers a global analysis of changes in data distribution and predictive feature significance over time. Semantic characterization uses natural language processing–inspired techniques to understand the local context of these changes, helping identify factors driving changes in patient outcomes. By integrating the outcomes from these 3 steps, our results can identify specific factors (eg, interventions and modifications in health care practices) that drive changes in patient outcomes. DIS was applied to the Brazilian COVID-19 Registry and the Medical Information Mart for Intensive Care, version IV (MIMIC-IV) data sets. Results: Our approach allowed us to (1) identify drifts effectively, especially using metrics such as the Jensen-Shannon divergence, and (2) uncover reasons for the decline in overall mortality in both the COVID-19 and MIMIC-IV data sets, as well as changes in the cooccurrence between different diseases and this particular outcome. Factors such as vaccination during the COVID-19 pandemic and reduced iatrogenic events and cancer-related deaths in MIMIC-IV were highlighted. The methodology also pinpointed shifts in patient demographics and disease patterns, providing insights into the evolving health care landscape during the study period. Conclusions: We developed a novel methodology combining machine learning and natural language processing techniques to detect, characterize, and understand temporal shifts in health care data. This understanding can enhance predictive algorithms, improve patient outcomes, and optimize health care resource allocation, ultimately improving the effectiveness of machine learning predictive algorithms applied to health care data. Our methodology can be applied to a variety of scenarios beyond those discussed in this paper. %M 39467275 %R 10.2196/54246 %U https://medinform.jmir.org/2024/1/e54246 %U https://doi.org/10.2196/54246 %U http://www.ncbi.nlm.nih.gov/pubmed/39467275 %0 Journal Article %@ 2561-7605 %I JMIR Publications %V 7 %N %P e54839 %T Using Existing Clinical Data to Measure Older Adult Inpatients’ Frailty at Admission and Discharge: Hospital Patient Register Study %A Wernli,Boris %A Verloo,Henk %A von Gunten,Armin %A Pereira,Filipa %+ University of Applied Sciences and Arts Western Switzerland (HES-SO), 5 Chemin de l'Agasse, Sion, 1950, Switzerland, 41 0787698990, henk.verloo@hevs.ch %K frailty %K frailty assessment %K electronic patient records %K functional independence measure %K routinely collected data %K hospital register %K patient records %K medical records %K clinical data %K older adults %K cluster analysis %K hierarchical clustering %D 2024 %7 28.10.2024 %9 Original Paper %J JMIR Aging %G English %X Background: Frailty is a widespread geriatric syndrome among older adults, including hospitalized older inpatients. Some countries use electronic frailty measurement tools to identify frailty at the primary care level, but this method has rarely been investigated during hospitalization in acute care hospitals. An electronic frailty measurement instrument based on population-based hospital electronic health records could effectively detect frailty, frailty-related problems, and complications as well be a clinical alert. Identifying frailty among older adults using existing patient health data would greatly aid the management and support of frailty identification and could provide a valuable public health instrument without additional costs. Objective: We aim to explore a data-driven frailty measurement instrument for older adult inpatients using data routinely collected at hospital admission and discharge. Methods: A retrospective electronic patient register study included inpatients aged ≥65 years admitted to and discharged from a public hospital between 2015 and 2017. A dataset of 53,690 hospitalizations was used to customize this data-driven frailty measurement instrument inspired by the Edmonton Frailty Scale developed by Rolfson et al. A 2-step hierarchical cluster procedure was applied to compute e-Frail-CH (Switzerland) scores at hospital admission and discharge. Prevalence, central tendency, comparative, and validation statistics were computed. Results: Mean patient age at admission was 78.4 (SD 7.9) years, with more women admitted (28,018/53,690, 52.18%) than men (25,672/53,690, 47.81%). Our 2-step hierarchical clustering approach computed 46,743 inputs of hospital admissions and 47,361 for discharges. Clustering solutions scored from 0.5 to 0.8 on a scale from 0 to 1. Patients considered frail comprised 42.02% (n=19,643) of admissions and 48.23% (n=22,845) of discharges. Within e-Frail-CH’s 0-12 range, a score ≥6 indicated frailty. We found a statistically significant mean e-Frail-CH score change between hospital admission (5.3, SD 2.6) and discharge (5.75, SD 2.7; P<.001). Sensitivity and specificity cut point values were 0.82 and 0.88, respectively. The area under the receiver operating characteristic curve was 0.85. Comparing the e-Frail-CH instrument to the existing Functional Independence Measure (FIM) instrument, FIM scores indicating severe dependence equated to e-Frail-CH scores of ≥9, with a sensitivity and specificity of 0.97 and 0.88, respectively. The area under the receiver operating characteristic curve was 0.92. There was a strong negative association between e-Frail-CH scores at hospital discharge and FIM scores (rs=–0.844; P<.001). Conclusions: An electronic frailty measurement instrument was constructed and validated using patient data routinely collected during hospitalization, especially at admission and discharge. The mean e-Frail-CH score was higher at discharge than at admission. The routine calculation of e-Frail-CH scores during hospitalization could provide very useful clinical alerts on the health trajectories of older adults and help select interventions for preventing or mitigating frailty. %M 39467281 %R 10.2196/54839 %U https://aging.jmir.org/2024/1/e54839 %U https://doi.org/10.2196/54839 %U http://www.ncbi.nlm.nih.gov/pubmed/39467281 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e53657 %T Caregiver and Youth Characteristics That Influence Trust in Digital Health Platforms in Pediatric Care: Mixed Methods Study %A Chow,Eric %A Virani,Alice %A Pinkney,Susan %A Abdulhussein,Fatema S %A van Rooij,Tibor %A Görges,Matthias %A Wasserman,Wyeth %A Bone,Jeffrey %A Longstaff,Holly %A Amed,Shazhan %+ Department of Pediatrics, BC Children’s Hospital, 4480 Oak Street, Room K4-206, Vancouver, BC, V6H 3V4, Canada, 1 604 875 2117, SAmed@cw.bc.ca %K pediatrics %K patient trust %K security %K data privacy %K data sharing %K caregivers %K patient engagement %K co-design %K personal health information %K secondary use of data %D 2024 %7 28.10.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Combining patient-generated health data and digital health platforms may improve patient experience and population health, mitigate rising health care costs, reduce clinician burnout, and enable health equity. However, lack of trust may be a notable barrier to the data-sharing required by such platforms. Understanding sociodemographic, health, and personal characteristics will enable developers and implementers of such technologies to consider these in their technical design requirements. Objective: This study aims to understand relationships between sociodemographic characteristics of caregivers of children or adolescents and trust in and willingness to use digital platforms to store and share personal health information for clinical care and research. Methods: This study used a mixed methods approach, including surveys of caregivers of youth aged <18 years living in Canada or the United States and youth aged 16 to 17 years living in Canada, as well as web-based bulletin board discussions to further explore topics of trust in data sharing. Sociodemographic and survey data were tabulated and explored using proportional odds ordinal regression models. Comments from web-based group discussions were analyzed thematically using a coding approach to identify issues important to the participants. Results: Survey data from 1128 caregivers (female participants: n=549, 48.7%; 36-50 years old: n=660, 58.5%; Canadian: n=603, 53.5%; urban population: n=494, 43.8%) were collected, of which 685 (60.7%) completed all questions. Data from 173 youth (female participants: n=73, 42.2%; urban population: n=94, 54.3%) were collected, of which 129 (74.6%) completed all questions, and data were available for analysis. Furthermore, among 40 participants, 23 (58%) caregivers contributed to the web-based discussion boards. Related to trust, living in a rural area (vs urban; odds ratio [OR] 0.66, 95% CI 0.46-0.95) resulted in lower concern for data privacy and security, while having an undergraduate (OR 1.82, 95% CI 1.30-2.55) or graduate degree (vs secondary or trade school; OR 2.50, 95% CI 1.68-3.73) resulted in higher levels of concern. Living with a chronic disease (OR 1.81, 95% CI 1.35-2.44) increased levels of concern regarding data privacy and security. Interestingly, those with chronic disease were more willing to use digital platforms for clinical care and share personal health information for not-for-profit research. Caregivers were most concerned about data breaches involving data from their children but also highlighted that digital platforms would allow for better coordination of care for their children. Conclusions: Our research confirms the willingness of caregivers and youth to use digital platforms for both clinical care delivery and research and suggests that the value of a digital platform may outweigh the risks of its use. Engagement of end users in co-designing such platforms has the potential to enhance digital trust. However, digital trust varies across sociodemographic groups; therefore, diverse end user engagement is necessary when designing digital applications. %M 39467279 %R 10.2196/53657 %U https://www.jmir.org/2024/1/e53657 %U https://doi.org/10.2196/53657 %U http://www.ncbi.nlm.nih.gov/pubmed/39467279 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e59791 %T The Evolution of Health Information Technology for Enhanced Patient-Centric Care in the United States: Data-Driven Descriptive Study %A Barker,Wesley %A Chang,Wei %A Everson,Jordan %A Gabriel,Meghan %A Patel,Vaishali %A Richwine,Chelsea %A Strawley,Catherine %+ Office of the Assistant Secretary for Technology Policy, US Department of Health and Human Services, 330 C St SW, Floor 7, Washington, DC, 20201, United States, 1 202 465 0597, meghan.gabriel@hhs.gov %K interoperability %K e-prescribing %K electronic public health reporting %K patient access to health information %K electronic health records %K health IT %D 2024 %7 28.10.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Health information technology (health IT) has revolutionized health care in the United States through interoperable clinical care data exchange, e-prescribing, electronic public health reporting, and electronic patient access to health information. Objective: This study aims to examine progress in health IT adoption and its alignment with the Office of the Assistant Secretary for Technology Policy/Office of the National Coordinator for Health IT (ASTP's) mission to enhance health care through data access and exchange. Methods: This study leverages data on end users of health IT to capture trends in engagement in interoperable clinical care data exchange (ability to find, send, receive, and integrate information from outside organizations), e-prescribing, electronic public health reporting, and capabilities to enable patient access to electronic health information. Data were primarily sourced from the American Hospital Association Annual Survey IT Supplement (2008 to 2023), Surescripts e-prescribing use data (2008 to 2023), the National Cancer Institute’s Health Information National Trends Survey (2014 to 2022), and the National Center for Health Statistics’ National Electronic Health Records Survey (2009 to 2023). Results: Since 2009, there has been a 10-fold increase in electronic health record (EHR) use among hospitals and a 5-fold increase among physicians. This enabled the interoperable exchange of electronic health information, e- prescribing, electronic public health data exchange, and the means for patients and their caregivers to access crucial personal health information digitally. As of 2023, 70% of hospitals are interoperable, with many providers integrated within EHR systems. Nearly all pharmacies and 92% of prescribers possess e-prescribing capabilities, an 85%-point increase since 2008. In 2013, 40% of hospitals and one-third of physicians allowed patients to view their online medical records. Patient access has improved, with 97% of hospitals and 65% of physicians possessing EHRs that enable patients to access their online medical records. As of 2022, three-fourths of individuals report being offered access to patient portals, and over half (57%) report engaging with their health information through their patient portal. Electronic public health reporting has also seen an increase, with most hospitals and physicians actively engaged in key reporting types. Conclusions: Federal incentives have contributed to the widespread adoption of EHRs and broad digitization in health care, while efforts to promote interoperability have encouraged collaboration across health care entities. As a result, interoperable clinical care data exchange, e-prescribing, electronic public health reporting, and patient access to health information have grown substantially over the past quarter century and have been shown to improve health care outcomes. However, interoperability hurdles, usability issues, data security concerns, and inequitable patient access persist. Addressing these issues will require collaborative efforts among stakeholders to promote data standardization, implement governance structures, and establish robust health information exchange networks. %M 39466303 %R 10.2196/59791 %U https://www.jmir.org/2024/1/e59791 %U https://doi.org/10.2196/59791 %U http://www.ncbi.nlm.nih.gov/pubmed/39466303 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e57035 %T Targeted Development and Validation of Clinical Prediction Models in Secondary Care Settings: Opportunities and Challenges for Electronic Health Record Data %A van Maurik,I S %A Doodeman,H J %A Veeger-Nuijens,B W %A Möhringer,R P M %A Sudiono,D R %A Jongbloed,W %A van Soelen,E %K clinical prediction model %K electronic health record %K targeted validation %K EHR %K EMR %K prediction models %K validation %K CPM %K secondary care %K machine learning %K artificial intelligence %K AI %D 2024 %7 24.10.2024 %9 %J JMIR Med Inform %G English %X Before deploying a clinical prediction model (CPM) in clinical practice, its performance needs to be demonstrated in the population of intended use. This is also called “targeted validation.” Many CPMs developed in tertiary settings may be most useful in secondary care, where the patient case mix is broad and practitioners need to triage patients efficiently. However, since structured or rich datasets of sufficient quality from secondary to assess the performance of a CPM are scarce, a validation gap exists that hampers the implementation of CPMs in secondary care settings. In this viewpoint, we highlight the importance of targeted validation and the use of CPMs in secondary care settings and discuss the potential and challenges of using electronic health record (EHR) data to overcome the existing validation gap. The introduction of software applications for text mining of EHRs allows the generation of structured “big” datasets, but the imperfection of EHRs as a research database requires careful validation of data quality. When using EHR data for the development and validation of CPMs, in addition to widely accepted checklists, we propose considering three additional practical steps: (1) involve a local EHR expert (clinician or nurse) in the data extraction process, (2) perform validity checks on the generated datasets, and (3) provide metadata on how variables were constructed from EHRs. These steps help to generate EHR datasets that are statistically powerful, of sufficient quality and replicable, and enable targeted development and validation of CPMs in secondary care settings. This approach can fill a major gap in prediction modeling research and appropriately advance CPMs into clinical practice. %R 10.2196/57035 %U https://medinform.jmir.org/2024/1/e57035 %U https://doi.org/10.2196/57035 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e53510 %T Occurrence of Stigmatizing Documentation Among Hospital Medicine Encounters With Opioid-Related Diagnosis Codes: Cohort Study %A Bradford,William S %A Bratches,Reed W R %A Porras,Hollie %A Chen,David R %A Gagnon,Kelly W %A Ascher,Simon B %+ Division of Infectious Diseases, University of Alabama Birmingham, Boshell Building 8th floor, 1808 7th Ave S, Birmingham, AL, 35233, United States, 1 205 934 8610, wsbradford@uabmc.edu %K stigmatizing language %K OUD %K opioid use disorder %K people with opioid use disorder (POUD) %K people who inject drugs (PWID) %K people who use drugs (PWUD) %K drug use %D 2024 %7 24.10.2024 %9 Short Paper %J JMIR Form Res %G English %X Background: Physician use of stigmatizing language in the clinical documentation of hospitalized adults with opioid use is common. However, patient factors associated with stigmatizing language in this setting remain poorly characterized. Objective: This study aimed to determine whether specific demographic factors and clinical outcomes are associated with the presence of stigmatizing language by physicians in the clinical documentation of encounters with opioid-related ICD-10 (International Statistical Classification of Diseases, Tenth Revision) codes. Methods: Hospital encounters with one or more associated opioid-related ICD-10 admission diagnoses on the hospital medicine service during the 2020 calendar year were analyzed for the presence of stigmatizing language in history and physical and discharge summaries. Multivariable adjusted logistic regression models were used to determine associations of age, race, gender, medication for addiction treatment use, against medical advice discharge, homelessness, comorbid polysubstance use, comorbid psychiatric disorder, comorbid chronic pain, cost, and 30-day readmission with the presence of stigmatizing language. Results: A total of 221 encounters were identified, of which 64 (29%) encounters had stigmatizing language present in physician documentation. Most stigmatizing language was due to use of “substance abuse” rather than the preferred term “substance use” (63/66 instances). Polysubstance use and homelessness were independently associated with the presence of stigmatizing language (adjusted odds ratio [aOR] 7.83; 95% CI 3.42-19.24 and aOR 2.44; 95% CI 1.03-5.90) when controlling for chronic pain and other covariates. Conclusions: Among hospital medicine encounters with an opioid-related diagnosis, stigmatizing language by physicians in clinical documentation was common and independently associated with comorbid polysubstance use and homelessness. %M 39447164 %R 10.2196/53510 %U https://formative.jmir.org/2024/1/e53510 %U https://doi.org/10.2196/53510 %U http://www.ncbi.nlm.nih.gov/pubmed/39447164 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e58418 %T Aligning Large Language Models for Enhancing Psychiatric Interviews Through Symptom Delineation and Summarization: Pilot Study %A So,Jae-hee %A Chang,Joonhwan %A Kim,Eunji %A Na,Junho %A Choi,JiYeon %A Sohn,Jy-yong %A Kim,Byung-Hoon %A Chu,Sang Hui %+ Department of Applied Statistics, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea, 82 2 2123 2472, jysohn1108@gmail.com %K large language model %K psychiatric interview %K interview summarization %K symptom delineation %D 2024 %7 24.10.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Recent advancements in large language models (LLMs) have accelerated their use across various domains. Psychiatric interviews, which are goal-oriented and structured, represent a significantly underexplored area where LLMs can provide substantial value. In this study, we explore the application of LLMs to enhance psychiatric interviews by analyzing counseling data from North Korean defectors who have experienced traumatic events and mental health issues. Objective: This study aims to investigate whether LLMs can (1) delineate parts of the conversation that suggest psychiatric symptoms and identify those symptoms, and (2) summarize stressors and symptoms based on the interview dialogue transcript. Methods: Given the interview transcripts, we align the LLMs to perform 3 tasks: (1) extracting stressors from the transcripts, (2) delineating symptoms and their indicative sections, and (3) summarizing the patients based on the extracted stressors and symptoms. These 3 tasks address the 2 objectives, where delineating symptoms is based on the output from the second task, and generating the summary of the interview incorporates the outputs from all 3 tasks. In this context, the transcript data were labeled by mental health experts for the training and evaluation of the LLMs. Results: First, we present the performance of LLMs in estimating (1) the transcript sections related to psychiatric symptoms and (2) the names of the corresponding symptoms. In the zero-shot inference setting using the GPT-4 Turbo model, 73 out of 102 transcript segments demonstrated a recall mid-token distance d<20 for estimating the sections associated with the symptoms. For evaluating the names of the corresponding symptoms, the fine-tuning method demonstrates a performance advantage over the zero-shot inference setting of the GPT-4 Turbo model. On average, the fine-tuning method achieves an accuracy of 0.82, a precision of 0.83, a recall of 0.82, and an F1-score of 0.82. Second, the transcripts are used to generate summaries for each interviewee using LLMs. This generative task was evaluated using metrics such as Generative Evaluation (G-Eval) and Bidirectional Encoder Representations from Transformers Score (BERTScore). The summaries generated by the GPT-4 Turbo model, utilizing both symptom and stressor information, achieve high average G-Eval scores: coherence of 4.66, consistency of 4.73, fluency of 2.16, and relevance of 4.67. Furthermore, it is noted that the use of retrieval-augmented generation did not lead to a significant improvement in performance. Conclusions: LLMs, using either (1) appropriate prompting techniques or (2) fine-tuning methods with data labeled by mental health experts, achieved an accuracy of over 0.8 for the symptom delineation task when measured across all segments in the transcript. Additionally, they attained a G-Eval score of over 4.6 for coherence in the summarization task. This research contributes to the emerging field of applying LLMs in psychiatric interviews and demonstrates their potential effectiveness in assisting mental health practitioners. %M 39447159 %R 10.2196/58418 %U https://formative.jmir.org/2024/1/e58418 %U https://doi.org/10.2196/58418 %U http://www.ncbi.nlm.nih.gov/pubmed/39447159 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e63456 %T Exploring Health Care Professionals’ Perspectives on the Use of a Medication and Care Support System and Recommendations for Designing a Similar Tool for Family Caregivers: Interview Study Among Health Care Professionals %A Ashimwe,Aimerence %A Davoody,Nadia %+ Karolinska Institutet, Tomtebodavägen 18 A, Stockholm, S-17177, Sweden, 46 (0)8 524 864 86, nadia.davoody@ki.se %K eHealth %K telemedicine %K mobile health %K mHealth %K medication management %K home care %K family caregivers %K mobile phone %D 2024 %7 23.10.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: With the aging population on the rise, the demand for effective health care solutions to address adverse drug events is becoming increasingly urgent. Telemedicine has emerged as a promising solution for strengthening health care delivery in home care settings and mitigating drug errors. Due to the indispensable role of family caregivers in daily patient care, integrating digital health tools has the potential to streamline medication management processes and enhance the overall quality of patient care. Objective: This study aims to explore health care professionals’ perspectives on the use of a medication and care support system (MCSS) and collect recommendations for designing a similar tool for family caregivers. Methods: Fifteen interviews with health care professionals in a home care center were conducted. Thematic analysis was used, and 5 key themes highlighting the importance of using the MCSS tool to improve medication management in home care were identified. Results: All participants emphasized the necessity of direct communication between health care professionals and family caregivers and stated that family caregivers need comprehensive information about medication administration, patient conditions, and symptoms. Furthermore, the health care professionals recommended features and functions customized for family caregivers. Conclusions: This study underscored the importance of clear communication between health care professionals and family caregivers and the provision of comprehensive instructions to promote safe medication practices. By equipping family caregivers with essential information via a tool similar to the MCSS, a proactive approach to preventing errors and improving outcomes is advocated. %M 39442168 %R 10.2196/63456 %U https://medinform.jmir.org/2024/1/e63456 %U https://doi.org/10.2196/63456 %U http://www.ncbi.nlm.nih.gov/pubmed/39442168 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e60491 %T Exploring Contactless Vital Signs Collection in Video Telehealth Visits Among Veterans Affairs Providers and Patients: Pilot Usability Study %A Garvin,Lynn %A Richardson,Eric %A Heyworth,Leonie %A McInnes,D Keith %+ Center for Healthcare Optimization and Implementation Research, Veterans Affairs Boston Healthcare System, 150 South Huntington Avenue, Boston, MA, 02130, United States, 1 617 390 4315, Lynn.Garvin@va.gov %K veteran %K provider %K video-based care %K vital statistics %K telemonitoring %K usability %K mobile health app %K telemedicine %K health care access %K vital sign %K video %K telehealth %K patient %K Veterans Affairs %K telehealth platform %K photoplethysmography %K camera %K web-based survey %K electronic medical record %K home-based biometric devices %K mHealth %D 2024 %7 23.10.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: To expand veterans’ access to health care, the Veterans Affairs (VA) Office of Connected Care explored a novel software feature called “Vitals” on its VA Video Connect telehealth platform. Vitals uses contactless, video-based, remote photoplethysmography (rPPG) through the infrared camera on veterans’ smartphones (and other devices) to automatically scan their faces to provide real-time vital statistics on screen to both the provider and patient. Objective: This study aimed to assess VA clinical provider and veteran patient attitudes regarding the usability of Vitals. Methods: We conducted a mixed methods evaluation of Vitals among VA providers and patients, collecting data in July and August 2023 at the VA Boston Healthcare System and VA San Diego Healthcare System. We conducted analyses in October 2023. In-person usability testing sessions consisted of a think-aloud procedure while using the software, a semistructured interview, and a 26-item web-based survey. Results: Usability test sessions with 20 VA providers and 13 patients demonstrated that both groups found Vitals “useful” and “easy to use,” and they rated its usability highly (86 and 82 points, respectively, on a 100-point scale). Regarding acceptability or willingness/intent to use, providers and patients generally expressed confidence and trust in Vitals readings, with high ratings of 90 and 85 points, respectively. Providers and patients rated Vitals highly for its feasibility and appropriateness for context (90 and 90 points, respectively). Finally, providers noted that Vitals’ flexibility makes it appropriate and advantageous for implementation in a wide range of clinical contexts, particularly in specialty care. Providers believed that most clinical teams would readily integrate Vitals into their routine workflow because it saves time; delivers accurate, consistently collected vitals; and may reduce reporting errors. Providers and veterans suggested training and support materials that could improve Vitals adoption and implementation. Conclusions: While remote collection of vital readings has been described in the literature, this is one of the first accounts of testing a contactless vital signs measurement tool among providers and patients. If ongoing initiatives demonstrate accuracy in its readings, Vitals could enhance telemedicine by providing accurate and automatic reporting and recording of vitals; sending patients’ vital readings (pending provider approval) directly to their electronic medical record; saving provider and patient time; and potentially reducing necessity of some home-based biometric devices. Understanding usability issues before US Food and Drug Administration approval of Vitals and its implementation could contribute to a seamless introduction of Vitals to VA providers and patients. %M 39441645 %R 10.2196/60491 %U https://formative.jmir.org/2024/1/e60491 %U https://doi.org/10.2196/60491 %U http://www.ncbi.nlm.nih.gov/pubmed/39441645 %0 Journal Article %@ 2561-6722 %I JMIR Publications %V 7 %N %P e54598 %T Touching Technology—Parents’ Experiences of Remote Consultations for Children With Severe Congenital Cardiac Conditions: Quasi-Experimental Cohort Study %A McCullough,Julie Elizabeth May %A Sinclair,Marlene %A Gillender,Jonathan %A McCrossan,Brian %A Slater,Paul F %A Browne,Rosie %A Casey,Frank %K congenital heart disease %K pediatric cardiology %K pediatric cardiologist %K pediatric %K parent %K digital health %K digital technology %K digital intervention %K telemedicine %K telehealth %K virtual care %K virtual health %K virtual medicine %K remote consultation %K telephone consultation %K video consultation %K remote patient monitoring %K technology acceptance %K videoconferencing consultations %D 2024 %7 22.10.2024 %9 %J JMIR Pediatr Parent %G English %X Background: Remote consultations (RCs) using videoconferencing was recommended by the General Medical Council as the method for clinicians to provide patient consultations during the COVID-19 pandemic. Facilitating this while providing high-quality care depends on the usability and acceptability of the technology. Objective: This project aimed to investigate parents’ experiences of using videoconferencing technology for real-time RCs with children who had congenital heart defects during the COVID-19 pandemic lockdown. Methods: This study’s design was quasi-experimental and was underpinned by the Unified Theory of Acceptance and Use of Technology model that seeks to explain and predict an individual’s intention to use a technology. Parents were informed of this study by the medical team, posters were made available in the wards and clinics, and leaflets were left for browsing. Clinician screening of potential participants led to the identification of 33 children and parents who were enrolled on this study. The intervention was a web-based RC by medical staff using a secure, interactive videoconferencing platform (Pexip). Each child and their mother or father received 8 RCs with the same specialist doctor or nurse. Measurements were taken using web-based questionnaires pre and post consultation at the first, middle, and last events; questions were focused on the acceptability, usability, and clinical applicability of RCs. Parents’ experiences were explored using recorded interviews and analyzed thematically. Results: In total, 29 children aged 4‐1052 (mean 95, SD 191.14) days completed the project, receiving a total of 189 RCs as part of their routine care. Parents’ prior experience of consultation via videoconference was low; however, as time progressed, their use and acceptance of the technology increased. The intervention was warmly received by all parents who found the face-to-face component particularly useful for discussion with their child’s medical team. Furthermore, parents noted the savings on time, money, and childcare. Conclusions: While in-person consultations are considered the gold standard of patient care, increasing pressures on health services and staff reduce availability. Given the ease of access and additional benefits experienced by parents and their children, it is proposed that hybrid models of consultation and care provision are equal, if not superior, to in-person consultations in the management of children with severe congenital heart defects while reducing costs and pressure on the health service and parents. %R 10.2196/54598 %U https://pediatrics.jmir.org/2024/1/e54598 %U https://doi.org/10.2196/54598 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e50023 %T True Mitotic Count Prediction in Gastrointestinal Stromal Tumors: Bayesian Network Model and PROMETheus (Preoperative Mitosis Estimator Tool) Application Development %A Renne,Salvatore Lorenzo %A Cammelli,Manuela %A Santori,Ilaria %A Tassan-Mangina,Marta %A Samà,Laura %A Ruspi,Laura %A Sicoli,Federico %A Colombo,Piergiuseppe %A Terracciano,Luigi Maria %A Quagliuolo,Vittorio %A Cananzi,Ferdinando Carlo Maria %+ Department of Biomedical Sciences, Humanitas University, Via Rita Levi Montalcini 4, Pieve Emanuele, Milan, 20072, Italy, 39 0282247743, salvatore.renne@hunimed.eu %K GIST mitosis %K risk classification %K mHealth %K mobile health %K neoadjuvant therapy %K patient stratification %K Gastrointestinal Stroma %K preoperative risk %D 2024 %7 22.10.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Gastrointestinal stromal tumors (GISTs) present a complex clinical landscape, where precise preoperative risk assessment plays a pivotal role in guiding therapeutic decisions. Conventional methods for evaluating mitotic count, such as biopsy-based assessments, encounter challenges stemming from tumor heterogeneity and sampling biases, thereby underscoring the urgent need for innovative approaches to enhance prognostic accuracy. Objective: The primary objective of this study was to develop a robust and reliable computational tool, PROMETheus (Preoperative Mitosis Estimator Tool), aimed at refining patient stratification through the precise estimation of mitotic count in GISTs. Methods: Using advanced Bayesian network methodologies, we constructed a directed acyclic graph (DAG) integrating pertinent clinicopathological variables essential for accurate mitotic count prediction on the surgical specimen. Key parameters identified and incorporated into the model encompassed tumor size, location, mitotic count from biopsy specimens, surface area evaluated during biopsy, and tumor response to therapy, when applicable. Rigorous testing procedures, including prior predictive simulations, validation utilizing synthetic data sets were employed. Finally, the model was trained on a comprehensive cohort of real-world GIST cases (n=80), drawn from the repository of the Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS) Humanitas Research Hospital, with a total of 160 cases analyzed. Results: Our computational model exhibited excellent diagnostic performance on synthetic data. Different model architecture were selected based on lower deviance and robust out-of-sample predictive capabilities. Posterior predictive checks (retrodiction) further corroborated the model’s accuracy. Subsequently, PROMETheus was developed. This is an intuitive tool that dynamically computes predicted mitotic count and risk assessment on surgical specimens based on tumor-specific attributes, including size, location, surface area, and biopsy-derived mitotic count, using posterior probabilities derived from the model. Conclusions: The deployment of PROMETheus represents a potential advancement in preoperative risk stratification for GISTs, offering clinicians a precise and reliable means to anticipate mitotic counts on surgical specimens and a solid base to stratify patients for clinical studies. By facilitating tailored therapeutic strategies, this innovative tool is poised to revolutionize clinical decision-making paradigms, ultimately translating into improved patient outcomes and enhanced prognostic precision in the management of GISTs. %M 39437385 %R 10.2196/50023 %U https://www.jmir.org/2024/1/e50023 %U https://doi.org/10.2196/50023 %U http://www.ncbi.nlm.nih.gov/pubmed/39437385 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e54902 %T Accessibility, Cost, and Quality of an Online Regular Follow-Up Visit Service at an Internet Hospital in China: Mixed Methods Study %A Wang,Kun %A Zou,Wenxin %A Lai,Yingsi %A Hao,Chun %A Liu,Ning %A Ling,Xiang %A Liu,Xiaohan %A Liu,Ting %A Yang,Xin %A Zu,Chenxi %A Wu,Shaolong %+ School of Government, Sun Yat-sen University, No. 135 Xingang Xi Road, Guangzhou, 510000, China, 86 02087335706, wushlong@mail.sysu.edu.cn %K internet hospital %K medical service %K accessibility %K cost %K quality %K regular follow-up %D 2024 %7 21.10.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Telemedicine provides remote health care services to overcome constraints of time and space in accessing medical care. Similarly, internet hospitals in China support and provide remote health care services. Due to the COVID-19 pandemic, there has been a proliferation of internet hospitals. Many new services, including online consultations and regular online follow-up visit services, can now be accessed via internet hospitals in China. However, the accessibility, cost, and quality advantages of regular online follow-up visit services remain unclear. Objective: This study aimed to evaluate the accessibility, costs, and quality of an online regular follow-up visit service provided by an internet hospital in China. By analyzing the accessibility, costs, and quality of this service from the supply and demand sides, we can summarize the practical and theoretical experiences. Methods: A mixed methods study was conducted using clinical records from 18,473 patients receiving 39,239 online regular follow-up visit services at an internet hospital in 2021, as well as interviews with 7 physicians, 2 head nurses, and 3 administrative staff members. The quantitative analysis examined patient demographics, diagnoses, prescriptions, geographic distribution, physician characteristics, accessibility (travel time and costs), and service hours. The qualitative analysis elucidated physician perspectives on ensuring the quality of online health care. Results: Patients were predominantly middle-aged men with chronic diseases like viral hepatitis who were located near the hospital. The vast majority were from Guangdong province where the hospital is based, especially concentrated in Guangzhou city. The online regular follow-up visit service reduced travel time by 1 hour to 9 hours and costs by ¥6 to ¥991 (US $0.86-$141.32) depending on proximity, with greater savings for patients farther from the hospital. Consultation times were roughly equivalent between online and in-person visits. Physicians provided most online services during lunch breaks (12 PM to 2 PM) or after work hours (7 PM to 11 PM), indicating increased workload. The top departments providing online regular follow-up visit services were Infectious Diseases, Rheumatology, and Dermatology. The most commonly prescribed medications aligned with the prevalent chronic diagnoses. To ensure quality, physicians conducted initial in-person consultations to fully evaluate patients before allowing online regular follow-up visits, during which they communicated with patients to assess conditions and determine if in-person care was warranted. They also periodically reminded patients to come in person for more comprehensive evaluations. However, they acknowledged online visits cannot fully replace face-to-face care. Conclusions: Telemedicine services such as online regular follow-up visit services provided by internet hospitals must strictly adhere to fundamental medical principles of diagnosis, prescription, and treatment. For patients with chronic diseases, online regular follow-up visit services improve accessibility and reduce cost but cannot fully replace in-person evaluations. Physicians leverage various strategies to ensure the quality of online care. %M 39432365 %R 10.2196/54902 %U https://www.jmir.org/2024/1/e54902 %U https://doi.org/10.2196/54902 %U http://www.ncbi.nlm.nih.gov/pubmed/39432365 %0 Journal Article %@ 2562-0959 %I JMIR Publications %V 7 %N %P e60346 %T Challenges in Teledermoscopy Diagnostic Outcome Studies: Scoping Review of Heterogeneous Study Characteristics %A van Sinderen,Femke %A Langermans,Anne P %A Kushniruk,Andre W %A Borycki,Elizabeth M %A Jaspers,Monique M %A Peute,Linda W %+ Department of Medical Informatics, Amsterdam UMC location University of Amsterdam, Meibergdreef 9, Amsterdam, 1105 AZ, Netherlands, 31 205666204, f.vansinderen@amsterdamumc.nl %K teledermatology %K teledermoscopy %K dermatologist %K dermatology %K telemedicine %K e-health %K telehealth %K scoping review %K heterogeneity %K variability %K diagnostic %K content analysis %K mobile phone %D 2024 %7 18.10.2024 %9 Review %J JMIR Dermatol %G English %X Background: Teledermoscopy has demonstrated benefits such as decreased costs and enhanced access to dermatology care for skin cancer detection. However, the heterogeneity among teledermoscopy studies hinders the systematic reviews’ synopsis of diagnostic outcomes, impeding trust and adoption in general practice and limiting overall health care benefits. Objective: This study aims to improve understanding and standardization of teledermoscopy diagnostic studies, by identifying and categorizing study characteristics contributing to heterogeneity. Subsequently, the variability and consistency of these characteristics were assessed. Methods: A review of systematic reviews regarding the diagnostic outcomes of teledermoscopy was performed to discern reported study characteristics contributing to heterogeneity. These characteristics were thematically grouped into 3 domains (population, index test, and reference standard), forming a data extraction framework. A scoping review on teledermoscopy diagnostic outcomes studies was performed, guided by the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist. Data pertaining to study characteristics from included studies were extracted and analyzed through descriptive content analysis. Systematic reviews’ reference lists validated the scoping review query. Results: The literature search yielded 4 systematic reviews, revealing 15 heterogeneous studies across the population, index test, and reference standard domains. The scoping review identified 49 studies, with 27 overlapping with the systematic reviews. Population characteristics varied, with one-third (16/49, 33%) of studies reporting fewer than 100 samples; most studies (41/49, 84%) reported on the type of lesion, and most (20/49, 41%) teledermoscopy consultations took place in secondary care. One-fifth (11/49, 22%) did not describe inclusion or exclusion criteria, or the criteria varied highly. Index test characteristics showed differences in clinical expertise, profession, and training in dermatoscopic photography, and 59% (29/49) did not report on 1 or more index test characteristics. Image quality and clinical information reporting likewise varied. Reference standard characteristics involved teledermatologists’ assessment, but 16 studies did not report teledermatologists’ experience levels. Most studies (26/49, 53%) used histopathology as a gold standard. Conclusions: The heterogeneity in the population, index tests, and reference standard domains across teledermoscopy diagnostic outcome studies underscores the need for standardized reporting. This hinders the synopsis of teledermoscopy diagnostic outcomes in systematic reviews and limits the integration of research results into practice. Adopting a (tailored) STARD (Standards for Reporting Diagnostic Accuracy Studies) checklist for teledermoscopy diagnostic outcome studies is recommended to enhance the consistency and comparability of outcomes. We suggest performing a Delphi study to gather consensus on the tailored STARD guideline. %M 39423370 %R 10.2196/60346 %U https://derma.jmir.org/2024/1/e60346 %U https://doi.org/10.2196/60346 %U http://www.ncbi.nlm.nih.gov/pubmed/39423370 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e60306 %T Enhancing Clinicians’ Use of Electronic Patient-Reported Outcome Measures in Outpatient Care: Mixed Methods Study %A van Engen,Veerle %A Bonfrer,Igna %A Ahaus,Kees %A Den Hollander-Ardon,Monique %A Peters,Ingrid %A Buljac-Samardzic,Martina %+ Erasmus School of Health Policy & Management, Erasmus University Rotterdam, Burgemeester Oudlaan 50, Rotterdam, 3062 PA, Netherlands, 31 010 408 8555, vanengen@eshpm.eur.nl %K patient-reported outcome measure %K value-based health care %K implementation %K clinician %K behavior %K barrier %K facilitator %K strategies %K professionalization %K mixed methods %D 2024 %7 18.10.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Despite the increasing use of patient-reported outcome measures (PROMs) for collecting self-reported data among hospital outpatients, clinicians’ use of these data remains suboptimal. Insight into this issue and strategies to enhance the use of PROMs are critical but limited. Objective: This study aimed to examine clinicians’ use of PROM data for value-based outpatient consultations and identify efforts to enhance their use of PROMs in a Dutch university hospital. First, we aimed to investigate clinicians’ use of outpatients’ PROM data in 2023, focusing on adoption, implementation, and maintenance. Second, we aimed to develop insights into the organizational-level strategies implemented to enhance clinicians’ use of PROM data from 2020 to 2023. This included understanding the underlying rationales for these strategies and identifying strategies that appeared to be missing to address barriers or leverage facilitators. Third, we aimed to explore the key factors driving and constraining clinicians’ use of PROMs in 2023. Methods: We integrated data from 4 sources: 1-year performance data on clinicians’ use of PROMs (n=70 subdepartments), internal hospital documents from a central support team (n=56), a survey among clinicians (n=47), and interviews with individuals contributing to the organizational-level implementation of PROMs (n=20). The Reach, Effectiveness, Adoption, Implementation, and Maintenance framework was used to analyze clinicians’ adoption, implementation, and maintenance of PROMs. Strategies were analyzed using the Expert Recommendations for Implementing Change taxonomy, and results were structured around the constructs of capability, opportunity, and motivation. Results: On average, around 2023, clinicians accessed PROM data for approximately 3 of 20 (14%) patients during their outpatient consultation, despite numerous strategies to improve this practice. We identified issues in adoption, implementation, and maintenance. The hospital’s strategies, shaped organically and pragmatically, were related to 27 (37%) out of 73 Expert Recommendations for Implementing Change strategies. These strategies focused on enhancing clinicians’ capability, opportunity, and motivation. We found shortcomings in the quality of execution and completeness of strategies in relation to addressing all barriers and leveraging facilitators. We identified variations in the factors influencing the use of PROMs among frequent PROM users, occasional users, and nonusers. Challenges to effective facilitation were apparent, with certain desired strategies being unfeasible or impeded. Conclusions: Enhancing clinicians’ use of PROMs has remained challenging despite various strategies aimed at improving their capability, opportunity, and motivation. The use of PROMs may require more substantial changes than initially expected, necessitating a shift in clinicians’ professional attitudes and practices. Hospitals can facilitate rather than manage clinicians’ genuine use of PROMs. They must prioritize efforts to engage clinicians with PROMs for value-based outpatient care. Specific attention to their professionalization may be warranted. Tailored strategies, designed to address within-group differences in clinicians’ needs and motivation, hold promise for future efforts. %M 39422999 %R 10.2196/60306 %U https://www.jmir.org/2024/1/e60306 %U https://doi.org/10.2196/60306 %U http://www.ncbi.nlm.nih.gov/pubmed/39422999 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e50730 %T Automatic Recommender System of Development Platforms for Smart Contract–Based Health Care Insurance Fraud Detection Solutions: Taxonomy and Performance Evaluation %A Kaafarani,Rima %A Ismail,Leila %A Zahwe,Oussama %+ Intelligent Distributed Computing and Systems Laboratory, Department of Computer Science and Software Engineering, College of Information Technology, United Arab Emirates University, 15551 Al Maqam Campus, Al Ain, Abu Dhabi, 15551, United Arab Emirates, 971 37673333 ext 5530, leila@uaeu.ac.ae %K blockchain %K blockchain development platform %K eHealth %K fraud detection %K fraud scenarios %K health care %K health care insurance %K health insurance %K machine learning %K medical informatics %K recommender system %K smart contract %K taxonomy %D 2024 %7 18.10.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Health care insurance fraud is on the rise in many ways, such as falsifying information and hiding third-party liability. This can result in significant losses for the medical health insurance industry. Consequently, fraud detection is crucial. Currently, companies employ auditors who manually evaluate records and pinpoint fraud. However, an automated and effective method is needed to detect fraud with the continually increasing number of patients seeking health insurance. Blockchain is an emerging technology and is constantly evolving to meet business needs. With its characteristics of immutability, transparency, traceability, and smart contracts, it demonstrates its potential in the health care domain. In particular, self-executable smart contracts are essential to reduce the costs associated with traditional paradigms, which are mostly manual, while preserving privacy and building trust among health care stakeholders, including the patient and the health insurance networks. However, with the proliferation of blockchain development platform options, selecting the right one for health care insurance can be difficult. This study addressed this void and developed an automated decision map recommender system to select the most effective blockchain platform for insurance fraud detection. Objective: This study aims to develop smart contracts for detecting health care insurance fraud efficiently. Therefore, we provided a taxonomy of fraud scenarios and implemented their detection using a blockchain platform that was suitable for health care insurance fraud detection. To automatically and efficiently select the best platform, we proposed and implemented a decision map–based recommender system. For developing the decision-map, we proposed a taxonomy of 102 blockchain platforms. Methods: We developed smart contracts for 12 fraud scenarios that we identified in the literature. We used the top 2 blockchain platforms selected by our proposed decision-making map–based recommender system, which is tailored for health care insurance fraud. The map used our taxonomy of 102 blockchain platforms classified according to their application domains. Results: The recommender system demonstrated that Hyperledger Fabric was the best blockchain platform for identifying health care insurance fraud. We validated our recommender system by comparing the performance of the top 2 platforms selected by our system. The blockchain platform taxonomy that we created revealed that 59 blockchain platforms are suitable for all application domains, 25 are suitable for financial services, and 18 are suitable for various application domains. We implemented fraud detection based on smart contracts. Conclusions: Our decision map recommender system, which was based on our proposed taxonomy of 102 platforms, automatically selected the top 2 platforms, which were Hyperledger Fabric and Neo, for the implementation of health care insurance fraud detection. Our performance evaluation of the 2 platforms indicated that Fabric surpassed Neo in all performance metrics, as depicted by our recommender system. We provided an implementation of fraud detection based on smart contracts. %M 39423005 %R 10.2196/50730 %U https://www.jmir.org/2024/1/e50730 %U https://doi.org/10.2196/50730 %U http://www.ncbi.nlm.nih.gov/pubmed/39423005 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e47814 %T Fine-Tuned Bidirectional Encoder Representations From Transformers Versus ChatGPT for Text-Based Outpatient Department Recommendation: Comparative Study %A Jo,Eunbeen %A Yoo,Hakje %A Kim,Jong-Ho %A Kim,Young-Min %A Song,Sanghoun %A Joo,Hyung Joon %+ Department of Medical Informatics, Korea University College of Medicine, 73, Inchon-ro, Seoul, 02841, Republic of Korea, 82 2 920 5445, drjoohj@gmail.com %K natural language processing %K bidirectional encoder representations from transformers %K large language model %K generative pretrained transformer %K medical specialty prediction %K quality of care %K health care application %K ChatGPT %K BERT %K AI technology %K conversational agent %K AI %K artificial intelligence %K chatbot %K application %K health care %D 2024 %7 18.10.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Patients often struggle with determining which outpatient specialist to consult based on their symptoms. Natural language processing models in health care offer the potential to assist patients in making these decisions before visiting a hospital. Objective: This study aimed to evaluate the performance of ChatGPT in recommending medical specialties for medical questions. Methods: We used a dataset of 31,482 medical questions, each answered by doctors and labeled with the appropriate medical specialty from the health consultation board of NAVER (NAVER Corp), a major Korean portal. This dataset includes 27 distinct medical specialty labels. We compared the performance of the fine-tuned Korean Medical bidirectional encoder representations from transformers (KM-BERT) and ChatGPT models by analyzing their ability to accurately recommend medical specialties. We categorized responses from ChatGPT into those matching the 27 predefined specialties and those that did not. Both models were evaluated using performance metrics of accuracy, precision, recall, and F1-score. Results: ChatGPT demonstrated an answer avoidance rate of 6.2% but provided accurate medical specialty recommendations with explanations that elucidated the underlying pathophysiology of the patient’s symptoms. It achieved an accuracy of 0.939, precision of 0.219, recall of 0.168, and an F1-score of 0.134. In contrast, the KM-BERT model, fine-tuned for the same task, outperformed ChatGPT with an accuracy of 0.977, precision of 0.570, recall of 0.652, and an F1-score of 0.587. Conclusions: Although ChatGPT did not surpass the fine-tuned KM-BERT model in recommending the correct medical specialties, it showcased notable advantages as a conversational artificial intelligence model. By providing detailed, contextually appropriate explanations, ChatGPT has the potential to significantly enhance patient comprehension of medical information, thereby improving the medical referral process. %M 39423004 %R 10.2196/47814 %U https://formative.jmir.org/2024/1/e47814 %U https://doi.org/10.2196/47814 %U http://www.ncbi.nlm.nih.gov/pubmed/39423004 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e57727 %T Semiology Extraction and Machine Learning–Based Classification of Electronic Health Records for Patients With Epilepsy: Retrospective Analysis %A Xia,Yilin %A He,Mengqiao %A Basang,Sijia %A Sha,Leihao %A Huang,Zijie %A Jin,Ling %A Duan,Yifei %A Tang,Yusha %A Li,Hua %A Lai,Wanlin %A Chen,Lei %K epilepsy %K natural language processing %K machine learning %K electronic health record %K unstructured text %K semiology %K health records %K retrospective analysis %K diagnosis %K treatment %K decision support tools %K symptom %K ontology %K China %K Chinese %K seizure %D 2024 %7 17.10.2024 %9 %J JMIR Med Inform %G English %X Background: Obtaining and describing semiology efficiently and classifying seizure types correctly are crucial for the diagnosis and treatment of epilepsy. Nevertheless, there exists an inadequacy in related informatics resources and decision support tools. Objective: We developed a symptom entity extraction tool and an epilepsy semiology ontology (ESO) and used machine learning to achieve an automated binary classification of epilepsy in this study. Methods: Using present history data of electronic health records from the Southwest Epilepsy Center in China, we constructed an ESO and a symptom-entity extraction tool to extract seizure duration, seizure symptoms, and seizure frequency from the unstructured text by combining manual annotation with natural language processing techniques. In addition, we achieved automatic classification of patients in the study cohort with high accuracy based on the extracted seizure feature data using multiple machine learning methods. Results: Data included present history from 10,925 cases between 2010 and 2020. Six annotators labeled a total of 2500 texts to obtain 5844 words of semiology and construct an ESO with 702 terms. Based on the ontology, the extraction tool achieved an accuracy rate of 85% in symptom extraction. Furthermore, we trained a stacking ensemble learning model combining XGBoost and random forest with an F1-score of 75.03%. The random forest model had the highest area under the curve (0.985). Conclusions: This work demonstrated the feasibility of natural language processing–assisted structural extraction of epilepsy medical record texts and downstream tasks, providing open ontology resources for subsequent related work. %R 10.2196/57727 %U https://medinform.jmir.org/2024/1/e57727 %U https://doi.org/10.2196/57727 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e60402 %T Applying the Non-Adoption, Abandonment, Scale-up, Spread, and Sustainability Framework Across Implementation Stages to Identify Key Strategies to Facilitate Clinical Decision Support System Integration Within a Large Metropolitan Health Service: Interview and Focus Group Study %A Fernando,Manasha %A Abell,Bridget %A McPhail,Steven M %A Tyack,Zephanie %A Tariq,Amina %A Naicker,Sundresan %+ Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology, Q Block, 60 Musk Avenue, Brisbane, 4059, Australia, 61 3138 6454, sundresan.naicker@qut.edu.au %K medical informatics %K adoption and implementation %K behavior %K health systems %D 2024 %7 17.10.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Computerized clinical decision support systems (CDSSs) enhance patient care through real-time, evidence-based guidance for health care professionals. Despite this, the effective implementation of these systems for health services presents multifaceted challenges, leading to inappropriate use and abandonment over the course of time. Using the Non-Adoption, Abandonment, Scale-Up, Spread, and Sustainability (NASSS) framework, this qualitative study examined CDSS adoption in a metropolitan health service, identifying determinants across implementation stages to optimize CDSS integration into health care practice. Objective: This study aims to identify the theory-informed (NASSS) determinants, which included multiple CDSS interventions across a 2-year period, both at the health-service level and at the individual hospital setting, that either facilitate or hinder the application of CDSSs within a metropolitan health service. In addition, this study aimed to map these determinants onto specific stages of the implementation process, thereby developing a system-level understanding of CDSS application across implementation stages. Methods: Participants involved in various stages of the implementation process were recruited (N=30). Participants took part in interviews and focus groups. We used a hybrid inductive-deductive qualitative content analysis and a framework mapping approach to categorize findings into barriers, enablers, or neutral determinants aligned to NASSS framework domains. These determinants were also mapped to implementation stages using the Active Implementation Framework stages approach. Results: Participants comprised clinical adopters (14/30, 47%), organizational champions (5/30, 16%), and those with roles in organizational clinical informatics (5/30, 16%). Most determinants were mapped to the organization level, technology, and adopter subdomains. However, the study findings also demonstrated a relative lack of long-term implementation planning. Consequently, determinants were not uniformly distributed across the stages of implementation, with 61.1% (77/126) identified in the exploration stage, 30.9% (39/126) in the full implementation stage, and 4.7% (6/126) in the installation stages. Stakeholders engaged in more preimplementation and full-scale implementation activities, with fewer cycles of monitoring and iteration activities identified. Conclusions: These findings addressed a substantial knowledge gap in the literature using systems thinking principles to identify the interdependent dynamics of CDSS implementation. A lack of sustained implementation strategies (ie, training and longer-term, adopter-level championing) weakened the sociotechnical network between developers and adopters, leading to communication barriers. More rigorous implementation planning, encompassing all 4 implementation stages, may, in a way, help in addressing the barriers identified and enhancing enablers. %M 39419497 %R 10.2196/60402 %U https://medinform.jmir.org/2024/1/e60402 %U https://doi.org/10.2196/60402 %U http://www.ncbi.nlm.nih.gov/pubmed/39419497 %0 Journal Article %@ 2817-1705 %I JMIR Publications %V 3 %N %P e58463 %T Machine Learning–Based Prediction for High Health Care Utilizers by Using a Multi-Institutional Diabetes Registry: Model Training and Evaluation %A Tan,Joshua Kuan %A Quan,Le %A Salim,Nur Nasyitah Mohamed %A Tan,Jen Hong %A Goh,Su-Yen %A Thumboo,Julian %A Bee,Yong Mong %+ Health Services Research Unit, Singapore General Hospital, 10 Hospital Blvd, Singapore, 168582, Singapore, 65 6222 3322, joshua.tank@mohh.com.sg %K diabetes mellitus %K type 2 diabetes %K health care utilization %K population health management %K population health %K machine learning %K artificial intelligence %K predictive model %K predictive system %K practical model %D 2024 %7 17.10.2024 %9 Original Paper %J JMIR AI %G English %X Background: The cost of health care in many countries is increasing rapidly. There is a growing interest in using machine learning for predicting high health care utilizers for population health initiatives. Previous studies have focused on individuals who contribute to the highest financial burden. However, this group is small and represents a limited opportunity for long-term cost reduction. Objective: We developed a collection of models that predict future health care utilization at various thresholds. Methods: We utilized data from a multi-institutional diabetes database from the year 2019 to develop binary classification models. These models predict health care utilization in the subsequent year across 6 different outcomes: patients having a length of stay of ≥7, ≥14, and ≥30 days and emergency department attendance of ≥3, ≥5, and ≥10 visits. To address class imbalance, random and synthetic minority oversampling techniques were employed. The models were then applied to unseen data from 2020 and 2021 to predict health care utilization in the following year. A portfolio of performance metrics, with priority on area under the receiver operating characteristic curve, sensitivity, and positive predictive value, was used for comparison. Explainability analyses were conducted on the best performing models. Results: When trained with random oversampling, 4 models, that is, logistic regression, multivariate adaptive regression splines, boosted trees, and multilayer perceptron consistently achieved high area under the receiver operating characteristic curve (>0.80) and sensitivity (>0.60) across training-validation and test data sets. Correcting for class imbalance proved critical for model performance. Important predictors for all outcomes included age, number of emergency department visits in the present year, chronic kidney disease stage, inpatient bed days in the present year, and mean hemoglobin A1c levels. Explainability analyses using partial dependence plots demonstrated that for the best performing models, the learned patterns were consistent with real-world knowledge, thereby supporting the validity of the models. Conclusions: We successfully developed machine learning models capable of predicting high service level utilization with strong performance and valid explainability. These models can be integrated into wider diabetes-related population health initiatives. %M 39418089 %R 10.2196/58463 %U https://ai.jmir.org/2024/1/e58463 %U https://doi.org/10.2196/58463 %U http://www.ncbi.nlm.nih.gov/pubmed/39418089 %0 Journal Article %@ 1947-2579 %I JMIR Publications %V 16 %N %P e53226 %T Improving Vaccine Clinic Efficiency Through the CANImmunize Platform %A Kuatsidzo,Ananda %A Wilson,Kumanan %A Ruller,Sydney %A Daly,Blake %A Halil,Roland %A Kobewka,Daniel %K digital solutions %K vaccine %K CANImmunize platform %K CANImmunize %K platform %K Canada %K Canadian %K workflow %K booking %K health care %K digital health %K hospital %K patient %K personnel %D 2024 %7 16.10.2024 %9 %J Online J Public Health Inform %G English %X Our objective was to evaluate the CANImmunize digital solution and measure the impact on workflow and appointment booking at Bruyère Hospital. %R 10.2196/53226 %U https://ojphi.jmir.org/2024/1/e53226 %U https://doi.org/10.2196/53226 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e44494 %T Reinforcement Learning to Optimize Ventilator Settings for Patients on Invasive Mechanical Ventilation: Retrospective Study %A Liu,Siqi %A Xu,Qianyi %A Xu,Zhuoyang %A Liu,Zhuo %A Sun,Xingzhi %A Xie,Guotong %A Feng,Mengling %A See,Kay Choong %+ Saw Swee Hock School of Public Health, National University of Singapore, 12 Science Drive 2, Singapore, 117549, Singapore, 65 65164984, ephfm@nus.edu.sg %K mechanical ventilation %K reinforcement learning %K artificial intelligence %K validation study %K critical care %K treatment %K intensive care unit %K critically ill %K patient %K monitoring %K database %K mortality rate %K decision support %K support tool %K survival %K prognosis %K respiratory support %D 2024 %7 16.10.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: One of the significant changes in intensive care medicine over the past 2 decades is the acknowledgment that improper mechanical ventilation settings substantially contribute to pulmonary injury in critically ill patients. Artificial intelligence (AI) solutions can optimize mechanical ventilation settings in intensive care units (ICUs) and improve patient outcomes. Specifically, machine learning algorithms can be trained on large datasets of patient information and mechanical ventilation settings. These algorithms can then predict patient responses to different ventilation strategies and suggest personalized ventilation settings for individual patients. Objective: In this study, we aimed to design and evaluate an AI solution that could tailor an optimal ventilator strategy for each critically ill patient who requires mechanical ventilation. Methods: We proposed a reinforcement learning–based AI solution using observational data from multiple ICUs in the United States. The primary outcome was hospital mortality. Secondary outcomes were the proportion of optimal oxygen saturation and the proportion of optimal mean arterial blood pressure. We trained our AI agent to recommend low, medium, and high levels of 3 ventilator settings—positive end-expiratory pressure, fraction of inspired oxygen, and ideal body weight–adjusted tidal volume—according to patients’ health conditions. We defined a policy as rules guiding ventilator setting changes given specific clinical scenarios. Off-policy evaluation metrics were applied to evaluate the AI policy. Results: We studied 21,595 and 5105 patients’ ICU stays from the e-Intensive Care Unit Collaborative Research (eICU) and Medical Information Mart for Intensive Care IV (MIMIC-IV) databases, respectively. Using the learned AI policy, we estimated the hospital mortality rate (eICU 12.1%, SD 3.1%; MIMIC-IV 29.1%, SD 0.9%), the proportion of optimal oxygen saturation (eICU 58.7%, SD 4.7%; MIMIC-IV 49%, SD 1%), and the proportion of optimal mean arterial blood pressure (eICU 31.1%, SD 4.5%; MIMIC-IV 41.2%, SD 1%). Based on multiple quantitative and qualitative evaluation metrics, our proposed AI solution outperformed observed clinical practice. Conclusions: Our study found that customizing ventilation settings for individual patients led to lower estimated hospital mortality rates compared to actual rates. This highlights the potential effectiveness of using reinforcement learning methodology to develop AI models that analyze complex clinical data for optimizing treatment parameters. Additionally, our findings suggest the integration of this model into a clinical decision support system for refining ventilation settings, supporting the need for prospective validation trials. %M 39219230 %R 10.2196/44494 %U https://www.jmir.org/2024/1/e44494 %U https://doi.org/10.2196/44494 %U http://www.ncbi.nlm.nih.gov/pubmed/39219230 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e54572 %T The Effects of Electronic Health Records on Medical Error Reduction: Extension of the DeLone and McLean Information System Success Model %A Chimbo,Bester %A Motsi,Lovemore %+ Department of Information Systems, University of South Africa, Cnr of Christiaan de Wet Road & Pioneer Avenue Florida, Johannesburg, 1709, South Africa, 27 82 333 8815, chimbb@unisa.ac.za %K medication error %K patient safety %K information system %K information systems %K electronic health record %K service quality %D 2024 %7 16.10.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Medical errors are becoming a major problem for health care providers and those who design health policies. These errors cause patients’ illnesses to worsen over time and can make recovery impossible. For the benefit of patients and the welfare of health care providers, a decrease in these errors is required to maintain safe, high-quality patient care. Objective: This study aimed to improve the ability of health care professionals to diagnose diseases and reduce medical errors. Methods: Data collection was performed at Dr George Mukhari Academic Hospital using convenience sampling. In total, 300 health care professionals were given a self-administered questionnaire, including doctors, dentists, pharmacists, physiologists, and nurses. To test the study hypotheses, multiple linear regression was used to evaluate empirical data. Results: In the sample of 300 health care professionals, no significant correlation was found between medical error reduction (MER) and knowledge quality (KQ) (β=.043, P=.48). A nonsignificant negative relationship existed between MER and information quality (IQ) (β=–.080, P=.19). However, a significant positive relationship was observed between MER and electronic health records (EHR; β=.125, 95% CI 0.005-0.245, P=.042). Conclusions: Increasing patient access to medical records for health care professionals may significantly improve patient health and well-being. The effectiveness of health care organizations’ operations can also be increased through better health information systems. To lower medical errors and enhance patient outcomes, policy makers should provide financing and support for EHR adoption as a top priority. Health care administrators should also concentrate on providing staff with the training they need to operate these systems efficiently. Empirical surveys in other public and private hospitals can be used to further test the validated survey instrument. %M 39412857 %R 10.2196/54572 %U https://medinform.jmir.org/2024/1/e54572 %U https://doi.org/10.2196/54572 %U http://www.ncbi.nlm.nih.gov/pubmed/39412857 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e60081 %T Primary Care Informatics: Vitalizing the Bedrock of Health Care %A You,Jacqueline Guan-Ting %A Leung,Tiffany I %A Pandita,Deepti %A Sakumoto,Matthew %+ Department of Medicine, University of California San Francisco, 533 Parnassus Avenue, U127, San Francisco, CA, 94143, United States, 1 4154761000, matthew.sakumoto@ucsf.edu %K health care delivery %K primary care %K primary health care %K primary prevention %K quality of health care %K holistic care %K holistic medicine %K people-centric care %K person-centric care %K medical informatics applications %K primary care informatics %K medical informatics %K health informatics %K information science %K data science %D 2024 %7 15.10.2024 %9 Viewpoint %J J Med Internet Res %G English %X Primary care informatics (PCI) professionals address workflow and technology solutions in a wide spectrum of health, ranging from optimizing the experience of the individual patient in the clinic room to supporting the health of populations and augmenting the work of frontline primary care clinical teams. PCI overlaps uniquely with 2 disciplines with an impact on societal health—primary care and health informatics. Primary care is a gateway to health care access and aims to synthesize and coordinate numerous, complex elements of patients’ health and medical care in a holistic manner. However, over the past 25 years, primary care has become a specialty in crisis: in a post–COVID-19 world, workforce shortages, clinician burnout, and continuing challenges in health care access all contribute to difficulties in sustaining primary care. Informatics professionals are poised to change this trajectory. In this viewpoint, we aim to inform readers of the discipline of PCI and its importance in the design, support, and maintenance of essential primary care services. Although this work focuses on primary care in the United States, which includes general internal medicine, family medicine, and pediatrics (and depending on definition, includes specialties such as obstetrics and gynecology), many of the principles outlined can also be applied to comparable health care services and settings in other countries. We highlight (1) common global challenges in primary care, (2) recent trends in the evolution of PCI (personalized medicine, population health, social drivers of health, and team-based care), and (3) opportunities to move forward PCI with current and emerging technologies using the 4Cs of primary care framework. In summary, PCI offers important contributions to health care and the informatics field, and there are many opportunities for informatics professionals to enhance the primary care experience for patients, families, and their care teams. %M 39405512 %R 10.2196/60081 %U https://www.jmir.org/2024/1/e60081 %U https://doi.org/10.2196/60081 %U http://www.ncbi.nlm.nih.gov/pubmed/39405512 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e56949 %T Views and Uses of Sepsis Digital Alerts in National Health Service Trusts in England: Qualitative Study With Health Care Professionals %A Lazzarino,Runa %A Borek,Aleksandra J %A Honeyford,Kate %A Welch,John %A Brent,Andrew J %A Kinderlerer,Anne %A Cooke,Graham %A Patil,Shashank %A Gordon,Anthony %A Glampson,Ben %A Goodman,Philippa %A Ghazal,Peter %A Daniels,Ron %A Costelloe,Céire E %A Tonkin-Crine,Sarah %+ Nuffield Department of Primary Care Health Sciences, Medical Division, University of Oxford, Radcliffe Observatory Quarter, Woodstock Road, Oxford, OX2 6GG, United Kingdom, 44 01865 289300, r.lazzarino@ymail.com %K digital alerts %K electronic health records %K computerized clinical decision support systems %K sepsis %K patient deterioration %K decision-making %K secondary care %K emergency care %K intensive care %K England %K qualitative study %D 2024 %7 15.10.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Sepsis is a common cause of serious illness and death. Sepsis management remains challenging and suboptimal. To support rapid sepsis diagnosis and treatment, screening tools have been embedded into hospital digital systems to appear as digital alerts. The implementation of digital alerts to improve the management of sepsis and deterioration is a complex intervention that has to fit with team workflow and the views and practices of hospital staff. Despite the importance of human decision-making and behavior in optimal implementation, there are limited qualitative studies that explore the views and experiences of health care professionals regarding digital alerts as sepsis or deterioration computerized clinician decision support systems (CCDSSs). Objective: This study aims to explore the views and experiences of health care professionals on the use of sepsis or deterioration CCDSSs and to identify barriers and facilitators to their implementation and use in National Health Service (NHS) hospitals. Methods: We conducted a qualitative, multisite study with unstructured observations and semistructured interviews with health care professionals from emergency departments, outreach teams, and intensive or acute units in 3 NHS hospital trusts in England. Data from both interviews and observations were analyzed together inductively using thematic analysis. Results: A total of 22 health care professionals were interviewed, and 12 observation sessions were undertaken. A total of four themes regarding digital alerts were identified: (1) support decision-making as nested in electronic health records, but never substitute professionals’ knowledge and experience; (2) remind to take action according to the context, such as the hospital unit and the job role; (3) improve the alerts and their introduction, by making them more accessible, easy to use, not intrusive, more accurate, as well as integrated across the whole health care system; and (4) contextual factors affecting views and use of alerts in the NHS trusts. Digital alerts are more optimally used in general hospital units with a lower senior decision maker:patient ratio and by health care professionals with experience of a similar technology. Better use of the alerts was associated with quality improvement initiatives and continuous sepsis training. The trusts’ features, such as the presence of a 24/7 emergency outreach team, good technological resources, and staffing and teamwork, favored a more optimal use. Conclusions: Trust implementation of sepsis or deterioration CCDSSs requires support on multiple levels and at all phases of the intervention, starting from a prego-live analysis addressing organizational needs and readiness. Advancements toward minimally disruptive and smart digital alerts as sepsis or deterioration CCDSSs, which are more accurate and specific but at the same time scalable and accessible, require policy changes and investments in multidisciplinary research. %M 39405513 %R 10.2196/56949 %U https://humanfactors.jmir.org/2024/1/e56949 %U https://doi.org/10.2196/56949 %U http://www.ncbi.nlm.nih.gov/pubmed/39405513 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e64085 %T Understanding Patient Portal Uses and Needs: Cross-Sectional Study in a State Fair Setting %A Rajamani,Sripriya %A Austin,Robin %A Richwine,Chelsea %A Britt-Lalich,Malin %A Thakur,Madhur %A Odowa,Yasmin %A Jantraporn,Ratchada %A Marquard,Jenna %+ University of Minnesota, 6-174 Weaver Densford 308 Harvard St SE, Minneapolis, MN, 55455, United States, 1 651 278 7426, sripriya@umn.edu %K patient portals %K patient engagement %K health information technology %K consumer health informatics %K health informatics %K use %K online access %K medical records %K data access %K functionality %D 2024 %7 11.10.2024 %9 Research Letter %J JMIR Form Res %G English %X This study identified 22 features that are used and the needs for desired features/data in patient portals that enable online access to medical records. Data collected at a Midwestern state fair indicates that while most participants used patient portals, use and desirability of specific features varied widely. Identified needs for enhanced data access, portal functionality, and usability can be used to inform effective patient portal design. %M 39393063 %R 10.2196/64085 %U https://formative.jmir.org/2024/1/e64085 %U https://doi.org/10.2196/64085 %U http://www.ncbi.nlm.nih.gov/pubmed/39393063 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 12 %N %P e58035 %T Digital Health Readiness: Making Digital Health Care More Inclusive %A Bober,Timothy %A Rollman,Bruce L %A Handler,Steven %A Watson,Andrew %A Nelson,Lyndsay A %A Faieta,Julie %A Rosland,Ann-Marie %+ Division of General Internal Medicine, University of Pittsburgh School of Medicine, UPMC Montefiore Hospital, Suite W933, Pittsburgh, PA, 15213, United States, 1 412 692 4821, bobertm@upmc.edu %K digital health %K digital health literacy %K informatics %K digital disparities %K digital health readiness %K inclusivity %K digital health tool %K literacy %K patient support %K health system %D 2024 %7 9.10.2024 %9 Viewpoint %J JMIR Mhealth Uhealth %G English %X This paper proposes an approach to assess digital health readiness in clinical settings to understand how prepared, experienced, and equipped individual people are to participate in digital health activities. Existing digital health literacy and telehealth prediction tools exist but do not assess technological aptitude for particular tasks or incorporate available electronic health record data to improve efficiency and efficacy. As such, we propose a multidomain digital health readiness assessment that incorporates a person’s stated goals and motivations for use of digital health, a focused digital health literacy assessment, passively collected data from the electronic health record, and a focused aptitude assessment for critical skills needed to achieve a person’s goals. This combination of elements should allow for easy integration into clinical workflows and make the assessment as actionable as possible for health care providers and in-clinic digital health navigators. Digital health readiness profiles could be used to match individuals with support interventions to promote the use of digital tools like telehealth, mobile apps, and remote monitoring, especially for those who are motivated but do not have adequate experience. Moreover, while effective and holistic digital health readiness assessments could contribute to increased use and greater equity in digital health engagement, they must also be designed with inclusivity in mind to avoid worsening known disparities in digital health care. %M 39383524 %R 10.2196/58035 %U https://mhealth.jmir.org/2024/1/e58035 %U https://doi.org/10.2196/58035 %U http://www.ncbi.nlm.nih.gov/pubmed/39383524 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e52411 %T Clinical Data Flow in Botswana Clinics: Protocol for a Mixed-Methods Assessment %A Faulkenberry,Grey %A Masizana,Audrey %A Mosesane,Badisa %A Ndlovu,Kagiso %+ Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Roberts Center for Pediatric Research, 734 Schuylkill Ave, 2716 South Street, Philadelphia, PA, 19146, United States, 1 2154393265, faulkenbej@chop.edu %K global health %K health information systems %K electronic health care records %K EHRs %K interoperability %K data flow %K access to information %K health information interoperability %K pediatric %K pediatrics %K infant %K infants %K clinical data %K mixed method %K Botswana %K health care information %K child health %K tuberculosis %K HIV %K eLearning %K survey %K health informatics %K communication %D 2024 %7 9.10.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Botswana has made significant investments in its health care information infrastructure, including vertical programs for child health and nutrition, HIV care, and tuberculosis. However, effectively integrating the more than 18 systems in place for data collection and reporting has proved to be challenging. The Botswana Health Data Collaborative Roadmap Strategy (2020-24) states that “there exists parallel reporting systems and data is not integrated into the mainstream reports at the national level,” seconded by the Botswana National eLearning strategy (2020), which states that “there is inadequate information flow at all levels, proliferation of systems, reporting tools are not synthesized; hence too many systems are not communicating.” Objective: The objectives of this study are to (1) create a visual representation of how data are processed and the inputs and outputs through each health care system level; (2) understand how frontline workers perceive health care data sharing across existing platforms and the impact of data on health care service delivery. Methods: The setting included a varied range of 30 health care facilities across Botswana, aiming to capture insights from multiple perspectives into data flow and system integration challenges. The study design combined qualitative and quantitative methodologies, informed by the rapid assessment process and the technology assessment model for resource limited settings. The study used a participatory research approach to ensure comprehensive stakeholder engagement from its inception. Survey instruments were designed to capture the intricacies of data processing, sharing, and integration among health care workers. A purposive sampling strategy was used to ensure a wide representation of participants across different health care roles and settings. Data collection used both digital surveys and in-depth interviews. Preliminary themes for analysis include perceptions of the value of health care data and experiences in data collection and sharing. Ethical approvals were comprehensively obtained, reflecting the commitment to uphold research integrity and participant welfare throughout the study. Results: The study recruited almost 44 health care facilities, spanning a variety of health care facilities. Of the 44 recruited facilities, 27 responded to the surveys and participated in the interviews. A total of 75% (112/150) of health care professionals participating came from clinics, 20% (30/150) from hospitals, and 5% (8/150) from health posts and mobile clinics. As of October 10, 2023, the study had collected over 200 quantitative surveys and conducted 90 semistructured interviews. Conclusions: This study has so far shown enthusiastic engagement from the health care community, underscoring the relevance and necessity of this study’s objectives. We believe the methodology, centered around extensive community engagement, is pivotal in capturing a nuanced understanding of the health care data ecosystem. The focus will now shift to the analysis phase of the study, with the aim of developing comprehensive recommendations for improving data flow within Botswana's health care system. International Registered Report Identifier (IRRID): DERR1-10.2196/52411 %R 10.2196/52411 %U https://www.researchprotocols.org/2024/1/e52411 %U https://doi.org/10.2196/52411 %0 Journal Article %@ 2369-3762 %I JMIR Publications %V 10 %N %P e54083 %T Integrating Digital Assistive Technologies Into Care Processes: Mixed Methods Study %A Hofstetter,Sebastian %A Zilezinski,Max %A Behr,Dominik %A Kraft,Bernhard %A Buhtz,Christian %A Paulicke,Denny %A Wolf,Anja %A Klus,Christina %A Stoevesandt,Dietrich %A Schwarz,Karsten %A Jahn,Patrick %+ AG Versorgungsforschung Pflege im Krankenhaus, Departement of Internal Medicine, University Medicine Halle (Saale), Ernst-Grube-Str. 40, 06120 Halle (Saale), Halle (Saale), Germany, 49 345 557 4064, sebastian.hofstetter@medizin.uni-halle.de %K digital assistive technologies %K education concept %K intention to use %K learning effects %K digital transformation %D 2024 %7 9.10.2024 %9 Original Paper %J JMIR Med Educ %G English %X Background: Current challenges in patient care have increased research on technology use in nursing and health care. Digital assistive technologies (DATs) are one option that can be incorporated into care processes. However, how the application of DATs should be introduced to nurses and care professionals must be clarified. No structured and effective education concepts for the patient-oriented integration of DATs in the nursing sector are currently available. Objective: This study aims to examine how a structured and guided integration and education concept, herein termed the sensitization, evaluative introduction, qualification, and implementation (SEQI) education concept, can support the integration of DATs into nursing practices. Methods: This study used an explanatory, sequential study design with a mixed methods approach. The SEQI intervention was run in 26 long-term care facilities oriented toward older adults in Germany after a 5-day training course in each. The participating care professionals were asked to test 1 of 6 DATs in real-world practice over 3 days. Surveys (n=112) were then administered that recorded the intention to use DATs at 3 measurement points, and guided qualitative interviews with care professionals (n=12) were conducted to evaluate the learning concepts and effects of the intervention. Results: As this was a pilot study, no sample size calculation was carried out, and P values were not reported. The participating care professionals were generally willing to integrate DATs—as an additional resource—into nursing processes even before the 4-stage SEQI intervention was presented. However, the intervention provided additional background knowledge and sensitized care professionals to the digital transformation, enabling them to evaluate how DATs fit in the health care sector, what qualifies these technologies for correct application, and what promotes their use. The care professionals expressed specific ideas and requirements for both technology-related education concepts and nursing DATs. Conclusions: Actively matching technical support, physical limitations, and patients’ needs is crucial when selecting DATs and integrating them into nursing processes. To this end, using a structured process such as SEQI that strengthens care professionals’ ability to integrate DATs can help improve the benefits of such technology in the health care setting. Practical, application-oriented learning can promote the long-term implementation of DATs. %M 39383526 %R 10.2196/54083 %U https://mededu.jmir.org/2024/1/e54083 %U https://doi.org/10.2196/54083 %U http://www.ncbi.nlm.nih.gov/pubmed/39383526 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e44294 %T Digital Interventions for Managing Medication and Health Care Service Delivery in West Africa: Systematic Review %A Oluokun,Emmanuel Oluwatosin %A Adedoyin,Festus Fatai %A Dogan,Huseyin %A Jiang,Nan %+ Department of Computing and Informatics, Faculty of Science and Technology, Bournemouth University, Fern Barrow, Talbot, Poole, BH12 5BB, United Kingdom, 44 01202 524111, eoluokun@bournemouth.ac.uk %K digital interventions %K medications delivery %K phone-based intervention %K tele and e-based intervention %K West Africa %K management, technology %K intervention %K medication %K tool %K smartphone %D 2024 %7 9.10.2024 %9 Review %J J Med Internet Res %G English %X Background: As a result of the recent advancements in technology, the incorporation of digital interventions into the health care system has gained a lot of attention and adoption globally. However, these interventions have not been fully adopted, thereby limiting their impact on health care delivery in West Africa. Objective: This review primarily aims at evaluating the current digital interventions for medication and health care delivery in West Africa. Its secondary aim is to assess the impacts of digital interventions in managing medication and health care service delivery with the intent of providing vital recommendations that would contribute to an excellent adoption of digital intervention tools in the health care space in West Africa. Methods: In line with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), a comprehensive search through various databases yielded 529 results. After a rigorous screening, 29 articles that provided information on 3 broad digital health intervention tools were found eligible for this review. Results: Out of 29 studies, 16 (55%) studies examined phone-based interventions, 9 (31%) studies focused on tele- and e-based interventions, and 4 (14%) studies evaluated digital interventions. These interventions were used for diverse purposes, some of which are monitoring adverse drug reactions, general health, sexual and reproductive health, and training of health care practitioners. The phone-based intervention appears to be the most known and impactful of all the interventions, followed by tele- and e-based, while digital interventions were scarcely used. Conclusions: Digital interventions have had a considerable level of impact on medication and health care delivery across West Africa. However, the overall impact is limited. Therefore, strategies must be developed to address the challenges limiting the use of digital intervention tools so that these tools can be fully incorporated into the health care space in West Africa. %M 39383531 %R 10.2196/44294 %U https://www.jmir.org/2024/1/e44294 %U https://doi.org/10.2196/44294 %U http://www.ncbi.nlm.nih.gov/pubmed/39383531 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e56263 %T Health Care Worker Usage of Large-Scale Health Information Exchanges in Japan: User-Level Audit Log Analysis Study %A Suzumoto,Jun %A Mori,Yukiko %A Kuroda,Tomohiro %K health information exchange %K audit log %K Japan %K HIE %K audit %K logs %K usage %K medical informatics %K rate %K hospitals %K electronic health record %D 2024 %7 9.10.2024 %9 %J JMIR Med Inform %G English %X Background: Over 200 health information exchanges (HIEs) are currently operational in Japan. The most common feature of HIEs is remote on-demand viewing or searching of aggregated patient health data from multiple institutions. However, the usage of this feature by individual users and institutions remains unknown. Objective: This study aims to understand usage of the on-demand patient data viewing feature of large-scale HIEs by individual health care workers and institutions in Japan. Methods: We conducted audit log analyses of large-scale HIEs. The research subjects were HIEs connected to over 100 institutions and with over 10,000 patients. Each health care worker’s profile and audit log data for HIEs were collected. We conducted four types of analyses on the extracted audit log. First, we calculated the ratio of the number of days of active HIE use for each hospital-affiliated doctor account. Second, we calculated cumulative monthly usage days of HIEs by each institution in financial year (FY) 2021/22. Third, we calculated each facility type’s monthly active institution ratio in FY2021/22. Fourth, we compared the monthly active institution ratio by medical institution for each HIE and the proportion of cumulative usage days by user type for each HIE. Results: We identified 24 HIEs as candidates for data collection and we analyzed data from 7 HIEs. Among hospital doctors, 93.5% (7326/7833) had never used HIEs during the available period in FY2021/22, while 19 doctors used them at least 30% of days. The median (IQR) monthly active institution ratios were 0.482 (0.470‐0.487) for hospitals, 0.243 (0.230‐0.247) for medical clinics, and 0.030 (0.024‐0.048) for dental clinics. In 51.9% (1781/3434) of hospitals, the cumulative monthly usage days of HIEs was 0, while in 26.8% (921/3434) of hospitals, it was between 1 and 10, and in 3% (103/3434) of hospitals, it was 100 or more. The median (IQR) monthly active institution ratio in medical institutions was 0.511 (0.487‐0.529) for the most used HIE and 0.109 (0.0927‐0.117) for the least used. The proportion of cumulative usage days of HIE by user type was complex for each HIE, and no consistent trends could be discerned. Conclusions: In the large-scale HIEs surveyed in this study, the overall usage of the on-demand patient data viewing feature was low, consistent with past official reports. User-level analyses of audit logs revealed large disparities in the number of days of HIE use among health care workers and institutions. There were also large disparities in HIE use by facility type or HIE; the percentage of cumulative HIE usage days by user type also differed by HIE. This study indicates the need for further research into why there are large disparities in demand for HIEs in Japan as well as the need to design comprehensive audit logs that can be matched with other official datasets. %R 10.2196/56263 %U https://medinform.jmir.org/2024/1/e56263 %U https://doi.org/10.2196/56263 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e55472 %T Applying Implementation Science to Advance Electronic Health Record–Driven Learning Health Systems: Case Studies, Challenges, and Recommendations %A Trinkley,Katy E %A Maw,Anna M %A Torres,Cristina Huebner %A Huebschmann,Amy G %A Glasgow,Russell E %+ Department of Family Medicine, School of Medicine, University of Colorado Anschutz Medical Campus, 12631 E 17th Ave, Mail Stop F496, Aurora, CO, 80045, United States, 1 303 724 3103, katy.trinkley@cuanschutz.edu %K learning health systems %K implementation science %K chronic care %K electronic health record %K evidence-based medicine %K information technology %K research and technology %D 2024 %7 7.10.2024 %9 Viewpoint %J J Med Internet Res %G English %X With the widespread implementation of electronic health records (EHRs), there has been significant progress in developing learning health systems (LHSs) aimed at improving health and health care delivery through rapid and continuous knowledge generation and translation. To support LHSs in achieving these goals, implementation science (IS) and its frameworks are increasingly being leveraged to ensure that LHSs are feasible, rapid, iterative, reliable, reproducible, equitable, and sustainable. However, 6 key challenges limit the application of IS to EHR-driven LHSs: barriers to team science, limited IS experience, data and technology limitations, time and resource constraints, the appropriateness of certain IS approaches, and equity considerations. Using 3 case studies from diverse health settings and 1 IS framework, we illustrate these challenges faced by LHSs and offer solutions to overcome the bottlenecks in applying IS and utilizing EHRs, which often stymie LHS progress. We discuss the lessons learned and provide recommendations for future research and practice, including the need for more guidance on the practical application of IS methods and a renewed emphasis on generating and accessing inclusive data. %M 39374069 %R 10.2196/55472 %U https://www.jmir.org/2024/1/e55472 %U https://doi.org/10.2196/55472 %U http://www.ncbi.nlm.nih.gov/pubmed/39374069 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e48580 %T Patient Safety Incident Reporting and Learning Guidelines Implemented by Health Care Professionals in Specialized Care Units: Scoping Review %A Gqaleni,Tusiwe Mabel %A Mkhize,Sipho Wellington %A Chironda,Geldine %+ School of Nursing and Public Health, University of KwaZulu-Natal, 238 Mazisi Kunene Road, College of Health Sciences, Durban, 4041, South Africa, 27 031260400 ext 1559, gqalenit@ukzn.ac.za %K patient safety incidents %K adverse events %K harm %K near misses %K reporting guidelines %K implementation guidelines %K implementation practices %K intervention strategies %K critical care units %K intensive care units %K patient safety %K specialized care unit %K guidelines %K clinical practice %K healthcare professional %K ICU %D 2024 %7 4.10.2024 %9 Review %J J Med Internet Res %G English %X Background: Implementing Patient Safety Incident Reporting and Learning (PSIRL) guidelines is critical in guiding clinical practice and improving clinical outcomes in specialized care units (SCUs). There is limited research on the evidence of the implemented PSIRL guidelines in SCUs at the global level. Objective: This review aims to map the evidence of PSIRL guidelines implemented by health care professionals in specialized care units globally. Methods: A scoping review methodology, according to Joanna Briggs Institute, was adopted. The eligibility criteria were guided by the Population, Concept, and Context (PCC) framework, with the Population including health care professionals, the Concept including PSIRL guidelines, and the Context including specialized units globally. Papers written in English were searched from relevant databases and search engines. The PRISMA-ScR (Preferred Reporting Items for Scoping Reviews and Meta-Analyses extension for Scoping Reviews) checklist for used. Results: The 13 selected studies were published from 2003 to 2023. Most articles are from the Netherlands and Switzerland (n=3), followed by South Africa (n=2). The nature of implemented PSIRL guidelines was computer-based (n=11) and paper-based incident reporting (n=2). The reporting system was intended for all the health care professionals within the specialized units, focusing on patients, staff members, and families. The outcomes of implemented incident reporting guidelines were positive, as evidenced by improved reporting of incidents, including medication errors (n=8) and decreased rate of incidents and errors (n=4). Furthermore, 1 study showed no change (n=1) in implementing the incident reporting guidelines. Conclusions: The implementation of reporting of patient safety incidents (PSIs) in specialized units started to be reported around 2002; however, the frequency of yearly publications remains very low. Although some specialized units are still using multifaceted interventions and paper reporting systems in reporting PSIs, the implementation of electronic and computer-based reporting systems is gaining momentum. The effective implementation of an electronic-based reporting system should extend into other units beyond critical care units, as it increases the reporting of PSIs, reducing time to make an informed reporting of PSIs and immediate accessibility to information when needed for analysis. The evidence on the implementation of PSI reporting guidelines in SCUs comes from 5 different continents (Asia, Africa, Australia, Europe, and North America). However, the number identified for certain countries within each continent is very minimal. %M 39365987 %R 10.2196/48580 %U https://www.jmir.org/2024/1/e48580 %U https://doi.org/10.2196/48580 %U http://www.ncbi.nlm.nih.gov/pubmed/39365987 %0 Journal Article %@ 2817-1705 %I JMIR Publications %V 3 %N %P e57673 %T The Utility and Implications of Ambient Scribes in Primary Care %A Seth,Puneet %A Carretas,Romina %A Rudzicz,Frank %+ Department of Family Medicine, McMaster University, 100 Main Street West, Hamilton, ON, L8P 1H6, Canada, 1 416 671 5114, sethp1@mcmaster.ca %K artificial intelligence %K AI %K large language model %K LLM %K digital scribe %K ambient scribe %K organizational efficiency %K electronic health record %K documentation burden %K administrative burden %D 2024 %7 4.10.2024 %9 Viewpoint %J JMIR AI %G English %X Ambient scribe technology, utilizing large language models, represents an opportunity for addressing several current pain points in the delivery of primary care. We explore the evolution of ambient scribes and their current use in primary care. We discuss the suitability of primary care for ambient scribe integration, considering the varied nature of patient presentations and the emphasis on comprehensive care. We also propose the stages of maturation in the use of ambient scribes in primary care and their impact on care delivery. Finally, we call for focused research on safety, bias, patient impact, and privacy in ambient scribe technology, emphasizing the need for early training and education of health care providers in artificial intelligence and digital health tools. %M 39365655 %R 10.2196/57673 %U https://ai.jmir.org/2024/1/e57673 %U https://doi.org/10.2196/57673 %U http://www.ncbi.nlm.nih.gov/pubmed/39365655 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e60601 %T Ascle—A Python Natural Language Processing Toolkit for Medical Text Generation: Development and Evaluation Study %A Yang,Rui %A Zeng,Qingcheng %A You,Keen %A Qiao,Yujie %A Huang,Lucas %A Hsieh,Chia-Chun %A Rosand,Benjamin %A Goldwasser,Jeremy %A Dave,Amisha %A Keenan,Tiarnan %A Ke,Yuhe %A Hong,Chuan %A Liu,Nan %A Chew,Emily %A Radev,Dragomir %A Lu,Zhiyong %A Xu,Hua %A Chen,Qingyu %A Li,Irene %+ Information Technology Center, University of Tokyo, 6-2-3 Kashiwanoha, Kashiwa, 277-8582, Japan, 81 09014707813, ireneli@ds.itc.u-tokyo.ac.jp %K natural language processing %K machine learning %K deep learning %K generative artificial intelligence %K large language models %K retrieval-augmented generation %K healthcare %D 2024 %7 3.10.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Medical texts present significant domain-specific challenges, and manually curating these texts is a time-consuming and labor-intensive process. To address this, natural language processing (NLP) algorithms have been developed to automate text processing. In the biomedical field, various toolkits for text processing exist, which have greatly improved the efficiency of handling unstructured text. However, these existing toolkits tend to emphasize different perspectives, and none of them offer generation capabilities, leaving a significant gap in the current offerings. Objective: This study aims to describe the development and preliminary evaluation of Ascle. Ascle is tailored for biomedical researchers and clinical staff with an easy-to-use, all-in-one solution that requires minimal programming expertise. For the first time, Ascle provides 4 advanced and challenging generative functions: question-answering, text summarization, text simplification, and machine translation. In addition, Ascle integrates 12 essential NLP functions, along with query and search capabilities for clinical databases. Methods: We fine-tuned 32 domain-specific language models and evaluated them thoroughly on 27 established benchmarks. In addition, for the question-answering task, we developed a retrieval-augmented generation (RAG) framework for large language models that incorporated a medical knowledge graph with ranking techniques to enhance the reliability of generated answers. Additionally, we conducted a physician validation to assess the quality of generated content beyond automated metrics. Results: The fine-tuned models and RAG framework consistently enhanced text generation tasks. For example, the fine-tuned models improved the machine translation task by 20.27 in terms of BLEU score. In the question-answering task, the RAG framework raised the ROUGE-L score by 18% over the vanilla models. Physician validation of generated answers showed high scores for readability (4.95/5) and relevancy (4.43/5), with a lower score for accuracy (3.90/5) and completeness (3.31/5). Conclusions: This study introduces the development and evaluation of Ascle, a user-friendly NLP toolkit designed for medical text generation. All code is publicly available through the Ascle GitHub repository. All fine-tuned language models can be accessed through Hugging Face. %M 39361955 %R 10.2196/60601 %U https://www.jmir.org/2024/1/e60601 %U https://doi.org/10.2196/60601 %U http://www.ncbi.nlm.nih.gov/pubmed/39361955 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e54991 %T Health Care Usage During the COVID-19 Pandemic and the Adoption of Telemedicine: Retrospective Study of Chronic Disease Cohorts %A Bjarnadóttir,Margrét Vilborg %A Anderson,David %A Anderson,Kelley M %A Aljwfi,Omar %A Peluso,Alina %A Ghannoum,Adam %A Balba,Gayle %A Shara,Nawar %+ Decisions, Operations and Information Technology, University of Maryland, College Park, 4353 Van Munching Hall, College Park, MD, 20742, United States, 1 301 405 2061, mbjarnad@umd.edu %K telehealth utilization %K health care utilization %K demographic differences %K cohort study %K telehealth %K COVID-19 %K telehealth adaption %D 2024 %7 3.10.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: The COVID-19 pandemic accelerated telehealth adoption across disease cohorts of patients. For many patients, routine medical care was no longer an option, and others chose not to visit medical offices in order to minimize COVID-19 exposure. In this study, we take a comprehensive multidisease approach in studying the impact of the COVID-19 pandemic on health care usage and the adoption of telemedicine through the first 12 months of the COVID-19 pandemic. Objective: We studied the impact of the COVID-19 pandemic on in-person health care usage and telehealth adoption across chronic diseases to understand differences in telehealth adoption across disease cohorts and patient demographics (such as the Social Vulnerability Index [SVI]). Methods: We conducted a retrospective cohort study of 6 different disease cohorts (anxiety: n=67,578; depression: n=45,570; diabetes: n=81,885; kidney failure: n=29,284; heart failure: n=21,152; and cancer: n=35,460). We used summary statistics to characterize changes in usage and regression analysis to study how patient characteristics relate to in-person health care and telehealth adoption and usage during the first 12 months of the pandemic. Results: We observed a reduction in in-person health care usage across disease cohorts (ranging from 10% to 24%). For most diseases we study, telehealth appointments offset the reduction in in-person visits. Furthermore, for anxiety and depression, the increase in telehealth usage exceeds the reduction in in-person visits (by up to 5%). We observed that younger patients and men have higher telehealth usage after accounting for other covariates. Patients from higher SVI areas are less likely to use telehealth; however, if they do, they have a higher number of telehealth visits, after accounting for other covariates. Conclusions: The COVID-19 pandemic affected health care usage across diseases, and the role of telehealth in replacing in-person visits varies by disease cohort. Understanding these differences can inform current practices and provides opportunities to further guide modalities of in-person and telehealth visits. Critically, further study is needed to understand barriers to telehealth service usage for patients in higher SVI areas. A better understanding of the role of social determinants of health may lead to more support for patients and help individual health care providers improve access to care for patients with chronic conditions. %M 39361360 %R 10.2196/54991 %U https://www.jmir.org/2024/1/e54991 %U https://doi.org/10.2196/54991 %U http://www.ncbi.nlm.nih.gov/pubmed/39361360 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e55267 %T A Collection of Components to Design Clinical Dashboards Incorporating Patient-Reported Outcome Measures: Qualitative Study %A Bischof,Anja Yvonne %A Kuklinski,David %A Salvi,Irene %A Walker,Carla %A Vogel,Justus %A Geissler,Alexander %+ Chair of Health Economics, Policy and Management, School of Medicine, University of St. Gallen, St. Jakob-Strasse 21, St. Gallen, 9000, Switzerland, 41 712243220, anja.bischof@unisg.ch %K clinical dashboards %K design components %K patient-reported outcome measures (PROMs) %K patient-reported outcomes (PROs) %D 2024 %7 2.10.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: A clinical dashboard is a data-driven clinical decision support tool visualizing multiple key performance indicators in a single report while minimizing time and effort for data gathering. Studies have shown that including patient-reported outcome measures (PROMs) in clinical dashboards supports the clinician’s understanding of how treatments impact patients’ health status, helps identify changes in health-related quality of life at an early stage, and strengthens patient-physician communication. Objective: This study aims to determine design components for clinical dashboards incorporating PROMs to inform software producers and users (ie, physicians). Methods: We conducted interviews with software producers and users to test preselected design components. Furthermore, the interviews allowed us to derive additional components that are not outlined in existing literature. Finally, we used inductive and deductive coding to derive a guide on which design components need to be considered when building a clinical dashboard incorporating PROMs. Results: A total of 25 design components were identified, of which 16 were already surfaced during the literature search. Furthermore, 9 additional components were derived inductively during our interviews. The design components are clustered in a generic dashboard, PROM-related, adjacent information, and requirements for adoption components. Both software producers and users agreed on the primary purpose of a clinical dashboard incorporating PROMs to enhance patient communication in outpatient settings. Dashboard benefits include enhanced data visualization and improved workflow efficiency, while interoperability and data collection were named as adoption challenges. Consistency in dashboard design components is preferred across different episodes of care, with adaptations only for disease-specific PROMs. Conclusions: Clinical dashboards have the potential to facilitate informed treatment decisions if certain design components are followed. This study establishes a comprehensive framework of design components to guide the development of effective clinical dashboards incorporating PROMs in health care practice. %M 39357042 %R 10.2196/55267 %U https://www.jmir.org/2024/1/e55267 %U https://doi.org/10.2196/55267 %U http://www.ncbi.nlm.nih.gov/pubmed/39357042 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e64092 %T Retention in HIV Primary Care Using a Web-Based Patient Engagement Platform: Multistate Case-Control Study %A Sukhija-Cohen,Adam Carl %A Patani,Henna %A Blasingame,Michael Foxworth %A Vu,Kathy Linh %A Bastani,Ramin %+ Sutter Health, Palo Alto Medical Foundation Research Institute, 795 El Camino Real, Ames Building, Palo Alto, CA, 94301, United States, 1 (650) 330 5963, adam.sukhija-cohen@sutterhealth.org %K HIV %K primary health care %K retention in care %K digital technology %K appointments and schedules %D 2024 %7 2.10.2024 %9 Short Paper %J J Med Internet Res %G English %X Background: Digital interventions to improve retention in HIV care are critical to ensure viral suppression and prevent further transmission. AIDS Healthcare Foundation Healthcare Centers are centers across the United States that provide primary HIV care. Traditionally, the Healthcare Centers conduct phone calls with patients to schedule and confirm appointments, as well as share laboratory results. In 2017, Healthvana piloted a digital platform at AIDS Healthcare Foundation Healthcare Centers to send patients SMS text message appointment reminders and allow patients to review their upcoming appointment and view their laboratory results in the web-based patient portal. Objective: A national implementation in 15 US states and Washington, DC, of this digital intervention pilot by Healthvana aims to determine whether SMS appointment reminders and web-based patient portal logins improved retention in care compared to traditional methods. Methods: A retrospective analysis of 40,028 patients living with HIV was conducted at the 61 AIDS Healthcare Foundation Healthcare Centers between January 2, 2017, and May 22, 2018. Patients were invited to enroll in Healthvana’s digital intervention pilot, allowing for a natural, organization-wide case-control study. Separate binary logistic regression models evaluated the relationship between receiving SMS appointment reminders and completing scheduled appointments, as well as the relationship between logging into the web-based patient portal and completing scheduled appointments. Four scheduled consecutive appointments for each patient were included in the analysis to account for 1 full year of data per patient. Results: Patients who received the SMS appointment reminder were 1.7 times more likely to complete appointment 1 compared to patients who did not receive the SMS appointment reminder (P<.001). In addition, patients who received the SMS appointment reminder were 1.6 times more likely to complete appointment 2 (P<.001), 1.7 times more likely to complete appointment 3 (P<.001), and 1.8 times more likely to complete appointment 4 (P<.001) compared to patients who did not receive the SMS appointment reminder. Patients who logged in to the web-based patient portal prior to their scheduled appointment were 7.4 times more likely to complete appointment 1 compared to patients who did not log in (P<.001). In addition, patients who logged in to the web-based patient portal prior to their scheduled appointment were 3.6 times more likely to complete appointment 2 (P<.001), 3.2 times more likely to complete appointment 3 (P<.001), and 2.8 times more likely to complete appointment 4 (P<.001) compared to patients who did not log in. Conclusions: HIV primary care appointment completion was higher when patients engaged with Healthvana’s digital platform. Digital technology interventions to ensure patients complete their scheduled HIV care appointments are imperative to curb the HIV epidemic. %M 39357049 %R 10.2196/64092 %U https://www.jmir.org/2024/1/e64092 %U https://doi.org/10.2196/64092 %U http://www.ncbi.nlm.nih.gov/pubmed/39357049 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e51198 %T Harnessing the Power of Complementarity Between Smart Tracking Technology and Associated Health Information Technologies: Longitudinal Study %A Tao,Youyou %A Zhu,Ruilin %A Wu,Dezhi %+ Department of Management Science, Lancaster University, Bailrigg, Lancaster, LA1 4YX, United Kingdom, 44 1524592938, ruilin.zhu@lancaster.ac.uk %K health IT %K smart tracking technology %K mobile IT %K health information exchange %K electronic health record %K readmission risk %K complementarity effects %K mobile phone %D 2024 %7 1.10.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Smart tracking technology (STT) that was applied for clinical use has the potential to reduce 30-day all-cause readmission risk through streamlining clinical workflows with improved accuracy, mobility, and efficiency. However, previously published literature has inadequately addressed the joint effects of STT for clinical use and its complementary health ITs (HITs) in this context. Furthermore, while previous studies have discussed the symbiotic and pooled complementarity effects among different HITs, there is a lack of evidence-based research specifically examining the complementarity effects between STT for clinical use and other relevant HITs. Objective: Through a complementarity theory lens, this study aims to examine the joint effects of STT for clinical use and 3 relevant HITs on 30-day all-cause readmission risk. These HITs are STT for supply chain management, mobile IT, and health information exchange (HIE). Specifically, this study examines whether the pooled complementarity effect exists between STT for clinical use and STT for supply chain management, and whether symbiotic complementarity effects exist between STT for clinical use and mobile IT and between STT for clinical use and HIE. Methods: This study uses a longitudinal in-patient dataset, including 879,122 in-patient hospital admissions for 347,949 patients in 61 hospitals located in Florida and New York in the United States, from 2014 to 2015. Logistic regression was applied to assess the effect of HITs on readmission risks. Time and hospital fixed effects were controlled in the regression model. Robust standard errors (SEs) were used to account for potential heteroskedasticity. These errors were further clustered at the patient level to consider possible correlations within the patient groups. Results: The interaction between STT for clinical use and STT for supply chain management, mobile IT, and HIE was negatively associated with 30-day readmission risk, with coefficients of –0.0352 (P=.003), –0.0520 (P<.001), and –0.0216 (P=.04), respectively. These results indicate that the pooled complementarity effect exists between STT for clinical use and STT for supply chain management, and symbiotic complementarity effects exist between STT for clinical use and mobile IT and between STT for clinical use and HIE. Furthermore, the joint effects of these HITs varied depending on the hospital affiliation and patients’ disease types. Conclusions: Our results reveal that while individual HIT implementations have varying impacts on 30-day readmission risk, their joint effects are often associated with a reduction in 30-day readmission risk. This study substantially contributes to HIT value literature by quantifying the complementarity effects among 4 different types of HITs: STT for clinical use, STT for supply chain management, mobile IT, and HIE. It further offers practical implications for hospitals to maximize the benefits of their complementary HITs in reducing the 30-day readmission risk in their respective care scenarios. %M 39353192 %R 10.2196/51198 %U https://formative.jmir.org/2024/1/e51198 %U https://doi.org/10.2196/51198 %U http://www.ncbi.nlm.nih.gov/pubmed/39353192 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e58740 %T Enhancing Performance of the National Field Triage Guidelines Using Machine Learning: Development of a Prehospital Triage Model to Predict Severe Trauma %A Chen,Qi %A Qin,Yuchen %A Jin,Zhichao %A Zhao,Xinxin %A He,Jia %A Wu,Cheng %A Tang,Bihan %+ Department of Health Management, Naval Medical University, No 800 Xiangyin Road, Shanghai, 200433, China, 86 02181871425, mangotangbihan@126.com %K severe trauma %K field triage %K machine learning %K prediction model %D 2024 %7 30.9.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Prehospital trauma triage is essential to get the right patient to the right hospital. However, the national field triage guidelines proposed by the American College of Surgeons have proven to be relatively insensitive when identifying severe traumas. Objective: This study aimed to build a prehospital triage model to predict severe trauma and enhance the performance of the national field triage guidelines. Methods: This was a multisite prediction study, and the data were extracted from the National Trauma Data Bank between 2017 and 2019. All patients with injury, aged 16 years of age or older, and transported by ambulance from the injury scene to any trauma center were potentially eligible. The data were divided into training, internal, and external validation sets of 672,309; 288,134; and 508,703 patients, respectively. As the national field triage guidelines recommended, age, 7 vital signs, and 8 injury patterns at the prehospital stage were included as candidate variables for model development. Outcomes were severe trauma with an Injured Severity Score ≥16 (primary) and critical resource use within 24 hours of emergency department arrival (secondary). The triage model was developed using an extreme gradient boosting model and Shapley additive explanation analysis. The model’s accuracy regarding discrimination, calibration, and clinical benefit was assessed. Results: At a fixed specificity of 0.5, the model showed a sensitivity of 0.799 (95% CI 0.797-0.801), an undertriage rate of 0.080 (95% CI 0.079-0.081), and an overtriage rate of 0.743 (95% CI 0.742-0.743) for predicting severe trauma. The model showed a sensitivity of 0.774 (95% CI 0.772-0.776), an undertriage rate of 0.158 (95% CI 0.157-0.159), and an overtriage rate of 0.609 (95% CI 0.608-0.609) when predicting critical resource use, fixed at 0.5 specificity. The triage model’s areas under the curve were 0.755 (95% CI 0.753-0.757) for severe trauma prediction and 0.736 (95% CI 0.734-0.737) for critical resource use prediction. The triage model’s performance was better than those of the Glasgow Coma Score, Prehospital Index, revised trauma score, and the 2011 national field triage guidelines RED criteria. The model’s performance was consistent in the 2 validation sets. Conclusions: The prehospital triage model is promising for predicting severe trauma and achieving an undertriage rate of <10%. Moreover, machine learning enhances the performance of field triage guidelines. %M 39348683 %R 10.2196/58740 %U https://www.jmir.org/2024/1/e58740 %U https://doi.org/10.2196/58740 %U http://www.ncbi.nlm.nih.gov/pubmed/39348683 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e49691 %T Inefficient Processes and Associated Factors in Primary Care Nursing: System Configuration Analysis %A Tarver,Willi L %A Savoy,April %A Patel,Himalaya %A Weiner,Michael %A Holden,Richard J %+ School of Industrial Engineering, Purdue University, 799 W. Michigan St. ET 201, Indianapolis, IN, 46202, United States, 1 3172782194, asavoy@purdue.edu %K health information technology %K mobile devices %K nursing and nursing systems %K outpatient care %K SEIPS 2.0 %K work-system analysis %D 2024 %7 30.9.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Industrywide, primary care nurses’ work is increasing in complexity and team orientation. Mobile health information technologies (HITs) designed to aid nurses with indirect care tasks, including charting, have had mixed success. Failed introductions of HIT may be explained by insufficient integration into nurses’ work processes, owing to an incomplete or incorrect understanding of the underlying work systems. Despite this need for context, published evidence has focused more on inpatient settings than on primary care. Objective: This study aims to characterize nurses’ and health technicians’ perceptions of process inefficiencies in the primary care setting and identify related work system factors. Methods: Guided by the Systems Engineering Initiative for Patient Safety (SEIPS) 2.0 model, we conducted an exploratory work system analysis with a convenience sample of primary care nurses and health technicians. Semistructured contextual interviews were conducted in 2 sets of primary care clinics in the Midwestern United States, one in an urban tertiary care center and the other in a rural community-based outpatient facility. Using directed qualitative content analysis of transcripts, we identified tasks participants perceived as frequent, redundant, or difficult, related processes, and recommendations for improvement. In addition, we conducted configuration analyses to identify associations between process inefficiencies and work system factors. Results: We interviewed a convenience sample of 20 primary care nurses and 2 health technicians, averaging approximately 12 years of experience in their current role. Across sites, participants perceived 2 processes, managing patient calls and clinic walk-in visits, as inefficient. Among work system factors, participants described organizational and technological factors associated with inefficiencies. For example, new organization policies to decrease patient waiting invoked frequent, repetitive, and difficult tasks, including chart review and check-in using tablet computers. Participants reported that issues with policy implementation and technology usability contributed to process inefficiencies. Organizational and technological factors were also perceived among participants as the most adaptable. Suggested technology changes included new tools for walk-in triage and patient self-reporting of symptoms. Conclusions: In response to changes to organizational policy and technology, without compensative changes elsewhere in their primary care work system, participants reported process adaptations. These adaptations indicate inefficient work processes. Understanding how the implementation of organizational policies affects other factors in the primary care work system may improve the quality of such implementations and, in turn, increase the effectiveness and efficiency of primary care nurse processes. Furthermore, the design and implementation of HIT interventions should consider influential work system factors and their effects on work processes. %M 39348682 %R 10.2196/49691 %U https://humanfactors.jmir.org/2024/1/e49691 %U https://doi.org/10.2196/49691 %U http://www.ncbi.nlm.nih.gov/pubmed/39348682 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e48294 %T Quality, Usability, and Trust Challenges to Effective Data Use in the Deployment and Use of the Bangladesh Nutrition Information System Dashboard: Qualitative Study %A Fesshaye,Berhaun %A Pandya,Shivani %A Kan,Lena %A Kalbarczyk,Anna %A Alland,Kelsey %A Rahman,SM Mustafizur %A Bulbul,Md. Mofijul Islam %A Mustaphi,Piyali %A Siddique,Muhammad Abu Bakr %A Tanim,Md. Imtiaz Alam %A Chowdhury,Mridul %A Rumman,Tajkia %A Labrique,Alain B %+ Department of International Health, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, Baltimore, MD, 21205, United States, 1 (410) 955 3934, bfessha1@jh.edu %K digital health %K nutrition %K data for decision-making %K health information systems %K information system %K information systems %K LMIC %K low- and middle-income countries %K nutritional %K dashboard %K experience %K experiences %K interview %K interviews %K service %K services %K delivery %K health care management %D 2024 %7 30.9.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Evidence-based decision-making is essential to improve public health benefits and resources, especially in low- and middle-income countries (LMICs), but the mechanisms of its implementation remain less straightforward. The availability of high-quality, reliable, and sufficient data in LMICs can be challenging due to issues such as a lack of human resource capacity and weak digital infrastructure, among others. Health information systems (HISs) have been critical for aggregating and integrating health-related data from different sources to support evidence-based decision-making. Nutrition information systems (NISs), which are nutrition-focused HISs, collect and report on nutrition-related indicators to improve issues related to malnutrition and food security—and can assist in improving populations’ nutritional statuses and the integration of nutrition programming into routine health services. Data visualization tools (DVTs) such as dashboards have been recommended to support evidence-based decision-making, leveraging data from HISs or NISs. The use of such DVTs to support decision-making has largely been unexplored within LMIC contexts. In Bangladesh, the Mukto dashboard was developed to display and visualize nutrition-related performance indicators at the national and subnational levels. However, despite this effort, the current use of nutrition data to guide priorities and decisions remains relatively nascent and underused. Objective: The goal of this study is to better understand how Bangladesh’s NIS, including the Mukto dashboard, has been used and areas for improvement to facilitate its use for evidence-based decision-making toward ameliorating nutrition-related service delivery and the health status of communities in Bangladesh. Methods: Primary data collection was conducted through qualitative semistructured interviews with key policy-level stakeholders (n=24). Key informants were identified through purposive sampling and were asked questions about the experiences and challenges with the NIS and related nutrition dashboards. Results: Main themes such as trust, data usability, personal power, and data use for decision-making emerged from the data. Trust in both data collection and quality was lacking among many stakeholders. Poor data usability stemmed from unstandardized indicators, irregular data collection, and differences between rural and urban data. Insufficient personal power and staff training coupled with infrastructural challenges can negatively affect data at the input stage. While stakeholders understood and expressed the importance of evidence-based decision-making, ultimately, they noted that the data were not being used to their maximum potential. Conclusions: Leveraging DVTs can improve the use of data for evidence-based decision-making, but decision makers must trust that the data are believable, credible, timely, and responsive. The results support the significance of a tailored data ecosystem, which has not reached its full potential in Bangladesh. Recommendations to reach this potential include ensuring a clear intended user base and accountable stakeholders are present. Systems should also have the capacity to ensure data credibility and support ongoing personal power requirements. %M 39348172 %R 10.2196/48294 %U https://www.jmir.org/2024/1/e48294 %U https://doi.org/10.2196/48294 %U http://www.ncbi.nlm.nih.gov/pubmed/39348172 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e55648 %T Accuracy of a Commercial Large Language Model (ChatGPT) to Perform Disaster Triage of Simulated Patients Using the Simple Triage and Rapid Treatment (START) Protocol: Gage Repeatability and Reproducibility Study %A Franc,Jeffrey Micheal %A Hertelendy,Attila Julius %A Cheng,Lenard %A Hata,Ryan %A Verde,Manuela %+ Department of Emergency Medicine, University of Alberta, 736c University Terrace, 8203-112 Street NW, Edmonton, AB, T6R2Z6, Canada, 1 7807006730, jeffrey.franc@ualberta.ca %K disaster medicine %K large language models %K triage %K disaster %K emergency %K disasters %K emergencies %K LLM %K LLMs %K GPT %K ChatGPT %K language model %K language models %K NLP %K natural language processing %K artificial intelligence %K repeatability %K reproducibility %K accuracy %K accurate %K reproducible %K repeatable %D 2024 %7 30.9.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: The release of ChatGPT (OpenAI) in November 2022 drastically reduced the barrier to using artificial intelligence by allowing a simple web-based text interface to a large language model (LLM). One use case where ChatGPT could be useful is in triaging patients at the site of a disaster using the Simple Triage and Rapid Treatment (START) protocol. However, LLMs experience several common errors including hallucinations (also called confabulations) and prompt dependency. Objective: This study addresses the research problem: “Can ChatGPT adequately triage simulated disaster patients using the START protocol?” by measuring three outcomes: repeatability, reproducibility, and accuracy. Methods: Nine prompts were developed by 5 disaster medicine physicians. A Python script queried ChatGPT Version 4 for each prompt combined with 391 validated simulated patient vignettes. Ten repetitions of each combination were performed for a total of 35,190 simulated triages. A reference standard START triage code for each simulated case was assigned by 2 disaster medicine specialists (JMF and MV), with a third specialist (LC) added if the first two did not agree. Results were evaluated using a gage repeatability and reproducibility study (gage R and R). Repeatability was defined as variation due to repeated use of the same prompt. Reproducibility was defined as variation due to the use of different prompts on the same patient vignette. Accuracy was defined as agreement with the reference standard. Results: Although 35,102 (99.7%) queries returned a valid START score, there was considerable variability. Repeatability (use of the same prompt repeatedly) was 14% of the overall variation. Reproducibility (use of different prompts) was 4.1% of the overall variation. The accuracy of ChatGPT for START was 63.9% with a 32.9% overtriage rate and a 3.1% undertriage rate. Accuracy varied by prompt with a maximum of 71.8% and a minimum of 46.7%. Conclusions: This study indicates that ChatGPT version 4 is insufficient to triage simulated disaster patients via the START protocol. It demonstrated suboptimal repeatability and reproducibility. The overall accuracy of triage was only 63.9%. Health care professionals are advised to exercise caution while using commercial LLMs for vital medical determinations, given that these tools may commonly produce inaccurate data, colloquially referred to as hallucinations or confabulations. Artificial intelligence–guided tools should undergo rigorous statistical evaluation—using methods such as gage R and R—before implementation into clinical settings. %M 39348189 %R 10.2196/55648 %U https://www.jmir.org/2024/1/e55648 %U https://doi.org/10.2196/55648 %U http://www.ncbi.nlm.nih.gov/pubmed/39348189 %0 Journal Article %@ 2561-1011 %I JMIR Publications %V 8 %N %P e60503 %T Identifying the Severity of Heart Valve Stenosis and Regurgitation Among a Diverse Population Within an Integrated Health Care System: Natural Language Processing Approach %A Xie,Fagen %A Lee,Ming-sum %A Allahwerdy,Salam %A Getahun,Darios %A Wessler,Benjamin %A Chen,Wansu %+ Department of Research and Evaluation, Kaiser Permanente Southern California, 100 S Los Robles Ave, 2nd Floor, Pasadena, CA, 91101, United States, 1 6265643294, fagen.xie@kp.org %K echocardiography report %K heart valve %K stenosis %K regurgitation %K natural language processing %K algorithm %D 2024 %7 30.9.2024 %9 Original Paper %J JMIR Cardio %G English %X Background: Valvular heart disease (VHD) is a leading cause of cardiovascular morbidity and mortality that poses a substantial health care and economic burden on health care systems. Administrative diagnostic codes for ascertaining VHD diagnosis are incomplete. Objective: This study aimed to develop a natural language processing (NLP) algorithm to identify patients with aortic, mitral, tricuspid, and pulmonic valve stenosis and regurgitation from transthoracic echocardiography (TTE) reports within a large integrated health care system. Methods: We used reports from echocardiograms performed in the Kaiser Permanente Southern California (KPSC) health care system between January 1, 2011, and December 31, 2022. Related terms/phrases of aortic, mitral, tricuspid, and pulmonic stenosis and regurgitation and their severities were compiled from the literature and enriched with input from clinicians. An NLP algorithm was iteratively developed and fine-trained via multiple rounds of chart review, followed by adjudication. The developed algorithm was applied to 200 annotated echocardiography reports to assess its performance and then the study echocardiography reports. Results: A total of 1,225,270 TTE reports were extracted from KPSC electronic health records during the study period. In these reports, valve lesions identified included 111,300 (9.08%) aortic stenosis, 20,246 (1.65%) mitral stenosis, 397 (0.03%) tricuspid stenosis, 2585 (0.21%) pulmonic stenosis, 345,115 (28.17%) aortic regurgitation, 802,103 (65.46%) mitral regurgitation, 903,965 (73.78%) tricuspid regurgitation, and 286,903 (23.42%) pulmonic regurgitation. Among the valves, 50,507 (4.12%), 22,656 (1.85%), 1685 (0.14%), and 1767 (0.14%) were identified as prosthetic aortic valves, mitral valves, tricuspid valves, and pulmonic valves, respectively. Mild and moderate were the most common severity levels of heart valve stenosis, while trace and mild were the most common severity levels of regurgitation. Males had a higher frequency of aortic stenosis and all 4 valvular regurgitations, while females had more mitral, tricuspid, and pulmonic stenosis. Non-Hispanic Whites had the highest frequency of all 4 valvular stenosis and regurgitations. The distribution of valvular stenosis and regurgitation severity was similar across race/ethnicity groups. Frequencies of aortic stenosis, mitral stenosis, and regurgitation of all 4 heart valves increased with age. In TTE reports with stenosis detected, younger patients were more likely to have mild aortic stenosis, while older patients were more likely to have severe aortic stenosis. However, mitral stenosis was opposite (milder in older patients and more severe in younger patients). In TTE reports with regurgitation detected, younger patients had a higher frequency of severe/very severe aortic regurgitation. In comparison, older patients had higher frequencies of mild aortic regurgitation and severe mitral/tricuspid regurgitation. Validation of the NLP algorithm against the 200 annotated TTE reports showed excellent precision, recall, and F1-scores. Conclusions: The proposed computerized algorithm could effectively identify heart valve stenosis and regurgitation, as well as the severity of valvular involvement, with significant implications for pharmacoepidemiological studies and outcomes research. %M 39348175 %R 10.2196/60503 %U https://cardio.jmir.org/2024/1/e60503 %U https://doi.org/10.2196/60503 %U http://www.ncbi.nlm.nih.gov/pubmed/39348175 %0 Journal Article %@ 2561-9128 %I JMIR Publications %V 7 %N %P e63817 %T Blood Bonds: Transforming Blood Donation Through Innovation, Inclusion, and Engagement %A Sagar,Ankita %+ Creighton University School of Medicine, CommonSpirit Health, 4 Interlaken Road, Monmouth Junction, NJ, 08852, United States, 1 9179136278, Ankita.Sagar@commonspirit.org %K blood donor %K engagement %K digital health %K health technology %K EBM %K inclusive %K inclusivity %K shared decision-making %K evidence-based medicine %K blood shortage %K blood product %K transfusion %K transfusions %K donor %K donation %K hematology %K perioperative medicine %K surgery %D 2024 %7 27.9.2024 %9 Viewpoint %J JMIR Perioper Med %G English %X The journey of receiving blood as a patient with transfusion-dependent beta thalassemia has profoundly shaped my understanding of the life-saving power of blood donation. This personal experience underscores the critical importance of blood donors, not just for individual recipients but for the broader community, enhancing public health, productivity, and well-being. There are several challenges to securing a blood donor pool in current health care climate. Solutions that focus on the engagement of donors, clinicians, and patients are key to improving the donor pool and utilizing the blood supply in a judicious manner. %M 39331421 %R 10.2196/63817 %U https://periop.jmir.org/2024/1/e63817 %U https://doi.org/10.2196/63817 %U http://www.ncbi.nlm.nih.gov/pubmed/39331421 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e55099 %T Triage Accuracy and the Safety of User-Initiated Symptom Assessment With an Electronic Symptom Checker in a Real-Life Setting: Instrument Validation Study %A Liu,Ville %A Kaila,Minna %A Koskela,Tuomas %+ Faculty of Medicine, University of Helsinki, Ruusulankatu 21 B 32, Helsinki, 00250, Finland, 358 400642104, villeliu@hotmail.com %K nurse triage %K emergency department triage %K triage %K symptom assessment %K health services accessibility %K telemedicine %K eHealth %K remote consultation %K eHealth %K primary health care %K primary care %K urgent care %K health services research %K health services %D 2024 %7 26.9.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Previous studies have evaluated the accuracy of the diagnostics of electronic symptom checkers (ESCs) and triage using clinical case vignettes. National Omaolo digital services (Omaolo) in Finland consist of an ESC for various symptoms. Omaolo is a medical device with a Conformité Européenne marking (risk class: IIa), based on Duodecim Clinical Decision Support, EBMEDS. Objective: This study investigates how well triage performed by the ESC nurse triage within the chief symptom list available in Omaolo (anal region symptoms, cough, diarrhea, discharge from the eye or watery or reddish eye, headache, heartburn, knee symptom or injury, lower back pain or injury, oral health, painful or blocked ear, respiratory tract infection, sexually transmitted disease, shoulder pain or stiffness or injury, sore throat or throat symptom, and urinary tract infection). In addition, the accuracy, specificity, sensitivity, and safety of the Omaolo ESC were assessed. Methods: This is a clinical validation study in a real-life setting performed at multiple primary health care (PHC) centers across Finland. The included units were of the walk-in model of primary care, where no previous phone call or contact was required. Upon arriving at the PHC center, users (patients) answered the ESC questions and received a triage recommendation; a nurse then assessed their triage. Findings on 877 patients were analyzed by matching the ESC recommendations with triage by the triage nurse. Results: Safe assessments by the ESC accounted for 97.6% (856/877; 95% CI 95.6%-98.0%) of all assessments made. The mean of the exact match for all symptom assessments was 53.7% (471/877; 95% CI 49.2%-55.9%). The mean value of the exact match or overly conservative but suitable for all (ESC’s assessment was 1 triage level higher than the nurse’s triage) symptom assessments was 66.6% (584/877; 95% CI 63.4%-69.7%). When the nurse concluded that urgent treatment was needed, the ESC’s exactly matched accuracy was 70.9% (244/344; 95% CI 65.8%-75.7%). Sensitivity for the Omaolo ESC was 62.6% and specificity of 69.2%. A total of 21 critical assessments were identified for further analysis: there was no indication of compromised patient safety. Conclusions: The primary objectives of this study were to evaluate the safety and to explore the accuracy, specificity, and sensitivity of the Omaolo ESC. The results indicate that the ESC is safe in a real-life setting when appraised with assessments conducted by triage nurses. Furthermore, the Omaolo ESC exhibits the potential to guide patients to appropriate triage destinations effectively, helping them to receive timely and suitable care. International Registered Report Identifier (IRRID): RR2-10.2196/41423 %M 39326038 %R 10.2196/55099 %U https://humanfactors.jmir.org/2024/1/e55099 %U https://doi.org/10.2196/55099 %U http://www.ncbi.nlm.nih.gov/pubmed/39326038 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e57852 %T Processing of Short-Form Content in Clinical Narratives: Systematic Scoping Review %A Kugic,Amila %A Martin,Ingrid %A Modersohn,Luise %A Pallaoro,Peter %A Kreuzthaler,Markus %A Schulz,Stefan %A Boeker,Martin %+ Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz, Auenbruggerplatz 2/5, Graz, 8036, Austria, 43 316 385 13591, markus.kreuzthaler@medunigraz.at %K electronic health records %K EHR %K clinical narratives %K natural language processing %K machine learning %K deep learning %K rule-based approach %K short-form expression %K disambiguation %K word embedding %K vector representations %K language modeling %K human-in-the-loop, feature extraction %D 2024 %7 26.9.2024 %9 Review %J J Med Internet Res %G English %X Background: Clinical narratives are essential components of electronic health records. The adoption of electronic health records has increased documentation time for hospital staff, leading to the use of abbreviations and acronyms more frequently. This brevity can potentially hinder comprehension for both professionals and patients. Objective: This review aims to provide an overview of the types of short forms found in clinical narratives, as well as the natural language processing (NLP) techniques used for their identification, expansion, and disambiguation. Methods: In the databases Web of Science, Embase, MEDLINE, EBMR (Evidence-Based Medicine Reviews), and ACL Anthology, publications that met the inclusion criteria were searched according to PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines for a systematic scoping review. Original, peer-reviewed publications focusing on short-form processing in human clinical narratives were included, covering the period from January 2018 to February 2023. Short-form types were extracted, and multidimensional research methodologies were assigned to each target objective (identification, expansion, and disambiguation). NLP study recommendations and study characteristics were systematically assigned occurrence rates for evaluation. Results: Out of a total of 6639 records, only 19 articles were included in the final analysis. Rule-based approaches were predominantly used for identifying short forms, while string similarity and vector representations were applied for expansion. Embeddings and deep learning approaches were used for disambiguation. Conclusions: The scope and types of what constitutes a clinical short form were often not explicitly defined by the authors. This lack of definition poses challenges for reproducibility and for determining whether specific methodologies are suitable for different types of short forms. Analysis of a subset of NLP recommendations for assessing quality and reproducibility revealed only partial adherence to these recommendations. Single-character abbreviations were underrepresented in studies on clinical narrative processing, as were investigations in languages other than English. Future research should focus on these 2 areas, and each paper should include descriptions of the types of content analyzed. %M 39325515 %R 10.2196/57852 %U https://www.jmir.org/2024/1/e57852 %U https://doi.org/10.2196/57852 %U http://www.ncbi.nlm.nih.gov/pubmed/39325515 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e49720 %T Assessing the Utility of a Patient-Facing Diagnostic Tool Among Individuals With Hypermobile Ehlers-Danlos Syndrome: Focus Group Study %A Goehringer,Jessica %A Kosmin,Abigail %A Laible,Natalie %A Romagnoli,Katrina %+ Department of Genomic Health, Geisinger, 100 North Academy Avenue, Dept of Genomic Health, Danville, PA, 17822, United States, 1 5702141005, jgoehringer@geisinger.edu %K diagnostic tool %K hypermobile Ehlers-Danlos syndrome %K patient experiences %K diagnostic odyssey %K affinity mapping %K mobile health app %K mobile phone %D 2024 %7 26.9.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Hypermobile Ehlers-Danlos syndrome (hEDS), characterized by joint hypermobility, skin laxity, and tissue fragility, is thought to be the most common inherited connective tissue disorder, with millions affected worldwide. Diagnosing this condition remains a challenge that can impact quality of life for individuals with hEDS. Many with hEDS describe extended diagnostic odysseys involving exorbitant time and monetary investment. This delay is due to the complexity of diagnosis, symptom overlap with other conditions, and limited access to providers. Many primary care providers are unfamiliar with hEDS, compounded by genetics clinics that do not accept referrals for hEDS evaluation and long waits for genetics clinics that do evaluate for hEDS, leaving patients without sufficient options. Objective: This study explored the user experience, quality, and utility of a prototype of a patient-facing diagnostic tool intended to support clinician diagnosis for individuals with symptoms of hEDS. The questions included within the prototype are aligned with the 2017 international classification of Ehlers-Danlos syndromes. This study explored how this tool may help patients communicate information about hEDS to their physicians, influencing the diagnosis of hEDS and affecting patient experience. Methods: Participants clinically diagnosed with hEDS were recruited from either a medical center or private groups on a social media platform. Interested participants provided verbal consent, completed questionnaires about their diagnosis, and were invited to join an internet-based focus group to share their thoughts and opinions on a diagnostic tool prototype. Participants were invited to complete the Mobile App Rating Scale (MARS) to evaluate their experience viewing the diagnostic tool. The MARS is a framework for evaluating mobile health apps across 4 dimensions: engagement, functionality, esthetics, and information quality. Qualitative data were analyzed using affinity mapping to organize information and inductively create themes that were categorized within the MARS framework dimensions to help identify strengths and weaknesses of the diagnostic tool prototype. Results: In total, 15 individuals participated in the internet-based focus groups; 3 (20%) completed the MARS. Through affinity diagramming, 2 main categories of responses were identified, including responses related to the user interface and responses related to the application of the tool. Each category included several themes and subthemes that mapped well to the 4 MARS dimensions. The analysis showed that the tool held value and utility among the participants diagnosed with hEDS. The shareable ending summary sheet provided by the tool stood out as a strength for facilitating communication between patient and provider during the diagnostic evaluation. Conclusions: The results provide insights on the perceived utility and value of the tool, including preferred phrasing, layout and design preferences, and tool accessibility. The participants expressed that the tool may improve the hEDS diagnostic odyssey and help educate providers about the diagnostic process. %M 39325533 %R 10.2196/49720 %U https://formative.jmir.org/2024/1/e49720 %U https://doi.org/10.2196/49720 %U http://www.ncbi.nlm.nih.gov/pubmed/39325533 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e59505 %T Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook %A AlSaad,Rawan %A Abd-alrazaq,Alaa %A Boughorbel,Sabri %A Ahmed,Arfan %A Renault,Max-Antoine %A Damseh,Rafat %A Sheikh,Javaid %+ Weill Cornell Medicine-Qatar, Education City, Street 2700, Doha, Qatar, 974 44928830, rta4003@qatar-med.cornell.edu %K artificial intelligence %K large language models %K multimodal large language models %K multimodality %K multimodal generative artificial intelligence %K multimodal generative AI %K generative artificial intelligence %K generative AI %K health care %D 2024 %7 25.9.2024 %9 Viewpoint %J J Med Internet Res %G English %X In the complex and multidimensional field of medicine, multimodal data are prevalent and crucial for informed clinical decisions. Multimodal data span a broad spectrum of data types, including medical images (eg, MRI and CT scans), time-series data (eg, sensor data from wearable devices and electronic health records), audio recordings (eg, heart and respiratory sounds and patient interviews), text (eg, clinical notes and research articles), videos (eg, surgical procedures), and omics data (eg, genomics and proteomics). While advancements in large language models (LLMs) have enabled new applications for knowledge retrieval and processing in the medical field, most LLMs remain limited to processing unimodal data, typically text-based content, and often overlook the importance of integrating the diverse data modalities encountered in clinical practice. This paper aims to present a detailed, practical, and solution-oriented perspective on the use of multimodal LLMs (M-LLMs) in the medical field. Our investigation spanned M-LLM foundational principles, current and potential applications, technical and ethical challenges, and future research directions. By connecting these elements, we aimed to provide a comprehensive framework that links diverse aspects of M-LLMs, offering a unified vision for their future in health care. This approach aims to guide both future research and practical implementations of M-LLMs in health care, positioning them as a paradigm shift toward integrated, multimodal data–driven medical practice. We anticipate that this work will spark further discussion and inspire the development of innovative approaches in the next generation of medical M-LLM systems. %M 39321458 %R 10.2196/59505 %U https://www.jmir.org/2024/1/e59505 %U https://doi.org/10.2196/59505 %U http://www.ncbi.nlm.nih.gov/pubmed/39321458 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e57633 %T A Clinical Decision Support Tool for Intimate Partner Violence Screening Among Women Veterans: Development and Qualitative Evaluation of Provider Perspectives %A Rossi,Fernanda S %A Wu,Justina %A Timko,Christine %A Nevedal,Andrea L %A Wiltsey Stirman,Shannon %+ Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, 1070 Arastradero Rd, Stanford, CA, 94304, United States, 1 650 721 3990, fsrossi@stanford.edu %K intimate partner violence %K clinical decision support %K intimate partner violence screening %K women veterans %D 2024 %7 25.9.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Women veterans, compared to civilian women, are especially at risk of experiencing intimate partner violence (IPV), pointing to the critical need for IPV screening and intervention in the Veterans Health Administration (VHA). However, implementing paper-based IPV screening and intervention in the VHA has revealed substantial barriers, including health care providers’ inadequate IPV training, competing demands, time constraints, and discomfort addressing IPV and making decisions about the appropriate type or level of intervention. Objective: This study aimed to address IPV screening implementation barriers and hence developed and tested a novel IPV clinical decision support (CDS) tool for physicians in the Women’s Health Clinic (WHC), a primary care clinic within the Veterans Affairs Palo Alto Health Care System. This tool provides intelligent, evidence-based, step-by-step guidance on how to conduct IPV screening and intervention. Methods: Informed by existing CDS development frameworks, developing the IPV CDS tool prototype involved six steps: (1) identifying the scope of the tool, (2) identifying IPV screening and intervention content, (3) incorporating IPV-related VHA and clinic resources, (4) identifying the tool’s components, (5) designing the tool, and (6) conducting initial tool revisions. We obtained preliminary physician feedback on user experience and clinical utility of the CDS tool via the System Usability Scale (SUS) and semistructured interviews with 6 WHC physicians. SUS scores were examined using descriptive statistics. Interviews were analyzed using rapid qualitative analysis to extract actionable feedback to inform design updates and improvements. Results: This study includes a detailed description of the IPV CDS tool. Findings indicated that the tool was generally well received by physicians, who indicated good tool usability (SUS score: mean 77.5, SD 12.75). They found the tool clinically useful, needed in their practice, and feasible to implement in primary care. They emphasized that it increased their confidence in managing patients reporting IPV but expressed concerns regarding its length, workflow integration, flexibility, and specificity of information. Several physicians, for example, found the tool too time consuming when encountering patients at high risk; they suggested multiple uses of the tool (eg, an educational tool for less-experienced health care providers and a checklist for more-experienced health care providers) and including more detailed information (eg, a list of local shelters). Conclusions: Physician feedback on the IPV CDS tool is encouraging and will be used to improve the tool. This study offers an example of an IPV CDS tool that clinics can adapt to potentially enhance the quality and efficiency of their IPV screening and intervention process. Additional research is needed to determine the tool’s clinical utility in improving IPV screening and intervention rates and patient outcomes (eg, increased patient safety, reduced IPV risk, and increased referrals to mental health treatment). %M 39321455 %R 10.2196/57633 %U https://formative.jmir.org/2024/1/e57633 %U https://doi.org/10.2196/57633 %U http://www.ncbi.nlm.nih.gov/pubmed/39321455 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e56049 %T Integrating Social Determinants of Health in Machine Learning–Driven Decision Support for Diabetes Case Management: Protocol for a Sequential Mixed Methods Study %A Lee,Seung-Yup %A Hayes,Leslie W %A Ozaydin,Bunyamin %A Howard,Steven %A Garretson,Alison M %A Bradley,Heather M %A Land,Andrew M %A DeLaney,Erin W %A Pritchett,Amy O %A Furr,Amanda L %A Allgood,Ashleigh %A Wyatt,Matthew C %A Hall,Allyson G %A Banaszak-Holl,Jane C %+ School of Health Professions, University of Alabama at Birmingham, 1716 9th Ave S, Birmingham, AL, 35233, United States, 1 205 934 4315, slee9@uab.edu %K diabetes %K case management %K case manager %K social work %K case mix %K social determinants of health %K clinical decision support %K decision support %K predictive analytics %K disparities %K health disparities %K data warehouse %K tertiary care %K health care system %K chronic disease management %D 2024 %7 25.9.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: The use of both clinical factors and social determinants of health (SDoH) in referral decision-making for case management may improve optimal use of resources and reduce outcome disparities among patients with diabetes. Objective: This study proposes the development of a data-driven decision-support system incorporating interactions between clinical factors and SDoH into an algorithm for prioritizing who receives case management services. The paper presents a design for prediction validation and preimplementation assessment that uses a mixed methods approach to guide the implementation of the system. Methods: Our study setting is a large, tertiary care academic medical center in the Deep South of the United States, where SDoH contribute to disparities in diabetes-specific hospitalizations and emergency department (ED) visits. This project will develop an interpretable artificial intelligence model for a population with diabetes using SDoH and clinical data to identify which posthospitalization cases have a higher likelihood of subsequent ED use. The electronic health record data collected for the study include demographics, SDoH, comorbidities, hospitalization-related factors, laboratory test results, and medication use to predict posthospitalization ED visits. Subsequently, a mixed methods approach will be used to validate prediction outcomes and develop an implementation strategy from insights into patient outcomes from case managers, clinicians, and quality and patient safety experts. Results: As of December 2023, we had abstracted data on 174,871 inpatient encounters between January 2018 and September 2023, involving 89,355 unique inpatients meeting inclusion criteria. Both clinical and SDoH data items were included for these patient encounters. In total, 85% of the inpatient visits (N=148,640) will be used for training (learning from the data) and the remaining 26,231 inpatient visits will be used for mixed-methods validation (testing). Conclusions: By integrating a critical suite of SDoH with clinical data related to diabetes, the proposed data-driven risk stratification model can enable individualized risk estimation and inform health professionals (eg, case managers) about the risk of patients’ upcoming ED use. The prediction outcome could potentially automate case management referrals, helping to better prioritize services. By taking a mixed methods approach, we aim to align the model with the hospital’s specific quality and patient safety considerations for the quality of patient care and the optimization of case management resource allocation. International Registered Report Identifier (IRRID): DERR1-10.2196/56049 %M 39321449 %R 10.2196/56049 %U https://www.researchprotocols.org/2024/1/e56049 %U https://doi.org/10.2196/56049 %U http://www.ncbi.nlm.nih.gov/pubmed/39321449 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e49387 %T Health Professionals’ Views on the Use of Conversational Agents for Health Care: Qualitative Descriptive Study %A MacNeill,A Luke %A MacNeill,Lillian %A Luke,Alison %A Doucet,Shelley %+ Centre for Research in Integrated Care, University of New Brunswick, 355 Campus Ring Road, Saint John, NB, E2L 4L5, Canada, 1 506 648 5777, luke.macneill@unb.ca %K conversational agents %K chatbots %K health care %K health professionals %K health personnel %K qualitative %K interview %D 2024 %7 25.9.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: In recent years, there has been an increase in the use of conversational agents for health promotion and service delivery. To date, health professionals’ views on the use of this technology have received limited attention in the literature. Objective: The purpose of this study was to gain a better understanding of how health professionals view the use of conversational agents for health care. Methods: Physicians, nurses, and regulated mental health professionals were recruited using various web-based methods. Participants were interviewed individually using the Zoom (Zoom Video Communications, Inc) videoconferencing platform. Interview questions focused on the potential benefits and risks of using conversational agents for health care, as well as the best way to integrate conversational agents into the health care system. Interviews were transcribed verbatim and uploaded to NVivo (version 12; QSR International, Inc) for thematic analysis. Results: A total of 24 health professionals participated in the study (19 women, 5 men; mean age 42.75, SD 10.71 years). Participants said that the use of conversational agents for health care could have certain benefits, such as greater access to care for patients or clients and workload support for health professionals. They also discussed potential drawbacks, such as an added burden on health professionals (eg, program familiarization) and the limited capabilities of these programs. Participants said that conversational agents could be used for routine or basic tasks, such as screening and assessment, providing information and education, and supporting individuals between appointments. They also said that health professionals should have some oversight in terms of the development and implementation of these programs. Conclusions: The results of this study provide insight into health professionals’ views on the use of conversational agents for health care, particularly in terms of the benefits and drawbacks of these programs and how they should be integrated into the health care system. These collective findings offer useful information and guidance to stakeholders who have an interest in the development and implementation of this technology. %M 39320936 %R 10.2196/49387 %U https://www.jmir.org/2024/1/e49387 %U https://doi.org/10.2196/49387 %U http://www.ncbi.nlm.nih.gov/pubmed/39320936 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e58978 %T Evaluating the Bias in Hospital Data: Automatic Preprocessing of Patient Pathways Algorithm Development and Validation Study %A Uhl,Laura %A Augusto,Vincent %A Dalmas,Benjamin %A Alexandre,Youenn %A Bercelli,Paolo %A Jardinaud,Fanny %A Aloui,Saber %+ Mines Saint-Etienne Centre Ingénierie Santé, Unité Mixte de Recherche (UMR) 6158 Laboratoire d'Informatique, de Modélisation et d'Optimisation des Systèmes (LIMOS), Centre national de la recherche scientifique (CNRS), 158 cours Fauriel, Saint-Etienne, 42000, France, 33 477420123, l.uhl@emse.fr %K preprocessing %K framework %K health care data %K patient pathway %K bed management %D 2024 %7 23.9.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: The optimization of patient care pathways is crucial for hospital managers in the context of a scarcity of medical resources. Assuming unlimited capacities, the pathway of a patient would only be governed by pure medical logic to meet at best the patient’s needs. However, logistical limitations (eg, resources such as inpatient beds) are often associated with delayed treatments and may ultimately affect patient pathways. This is especially true for unscheduled patients—when a patient in the emergency department needs to be admitted to another medical unit without disturbing the flow of planned hospitalizations. Objective: In this study, we proposed a new framework to automatically detect activities in patient pathways that may be unrelated to patients’ needs but rather induced by logistical limitations. Methods: The scientific contribution lies in a method that transforms a database of historical pathways with bias into 2 databases: a labeled pathway database where each activity is labeled as relevant (related to a patient’s needs) or irrelevant (induced by logistical limitations) and a corrected pathway database where each activity corresponds to the activity that would occur assuming unlimited resources. The labeling algorithm was assessed through medical expertise. In total, 2 case studies quantified the impact of our method of preprocessing health care data using process mining and discrete event simulation. Results: Focusing on unscheduled patient pathways, we collected data covering 12 months of activity at the Groupe Hospitalier Bretagne Sud in France. Our algorithm had 87% accuracy and demonstrated its usefulness for preprocessing traces and obtaining a clean database. The 2 case studies showed the importance of our preprocessing step before any analysis. The process graphs of the processed data had, on average, 40% (SD 10%) fewer variants than the raw data. The simulation revealed that 30% of the medical units had >1 bed difference in capacity between the processed and raw data. Conclusions: Patient pathway data reflect the actual activity of hospitals that is governed by medical requirements and logistical limitations. Before using these data, these limitations should be identified and corrected. We anticipate that our approach can be generalized to obtain unbiased analyses of patient pathways for other hospitals. %M 39312289 %R 10.2196/58978 %U https://medinform.jmir.org/2024/1/e58978 %U https://doi.org/10.2196/58978 %U http://www.ncbi.nlm.nih.gov/pubmed/39312289 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e52678 %T The Impact of Collaborative Documentation on Person-Centered Care: Textual Analysis of Clinical Notes %A Stanhope,Victoria %A Yoo,Nari %A Matthews,Elizabeth %A Baslock,Daniel %A Hu,Yuanyuan %K person-centered care %K collaborative documentation %K natural language processing %K concurrent documentation %K clinical documentations %K visit notes %K community %K health center %K mental health center %K textual analysis %K clinical informatics %K behavioral health %K mental health %K linguistic %K linguistic inquiry %K dictionary-based %K sentence fragment %K psychology %K psychological %K clinical information %K decision-making %K mental health services %K clinical notes %K NLP %D 2024 %7 20.9.2024 %9 %J JMIR Med Inform %G English %X Background: Collaborative documentation (CD) is a behavioral health practice involving shared writing of clinic visit notes by providers and consumers. Despite widespread dissemination of CD, research on its effectiveness or impact on person-centered care (PCC) has been limited. Principles of PCC planning, a recovery-based approach to service planning that operationalizes PCC, can inform the measurement of person-centeredness within clinical documentation. Objective: This study aims to use the clinical informatics approach of natural language processing (NLP) to examine the impact of CD on person-centeredness in clinic visit notes. Using a dictionary-based approach, this study conducts a textual analysis of clinic notes from a community mental health center before and after staff were trained in CD. Methods: This study used visit notes (n=1981) from 10 providers in a community mental health center 6 months before and after training in CD. LIWC-22 was used to assess all notes using the Linguistic Inquiry and Word Count (LIWC) dictionary, which categorizes over 5000 linguistic and psychological words. Twelve LIWC categories were selected and mapped onto PCC planning principles through the consensus of 3 domain experts. The LIWC-22 contextualizer was used to extract sentence fragments from notes corresponding to LIWC categories. Then, fixed-effects modeling was used to identify differences in notes before and after CD training while accounting for nesting within the provider. Results: Sentence fragments identified by the contextualizing process illustrated how visit notes demonstrated PCC. The fixed effects analysis found a significant positive shift toward person-centeredness; this was observed in 6 of the selected LIWC categories post CD. Specifically, there was a notable increase in words associated with achievement (β=.774, P<.001), power (β=.831, P<.001), money (β=.204, P<.001), physical health (β=.427, P=.03), while leisure words decreased (β=−.166, P=.002). Conclusions: By using a dictionary-based approach, the study identified how CD might influence the integration of PCC principles within clinical notes. Although the results were mixed, the findings highlight the potential effectiveness of CD in enhancing person-centeredness in clinic notes. By leveraging NLP techniques, this research illuminated the value of narrative clinical notes in assessing the quality of care in behavioral health contexts. These findings underscore the promise of NLP for quality assurance in health care settings and emphasize the need for refining algorithms to more accurately measure PCC. %R 10.2196/52678 %U https://medinform.jmir.org/2024/1/e52678 %U https://doi.org/10.2196/52678 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e58278 %T Evaluating a Natural Language Processing–Driven, AI-Assisted International Classification of Diseases, 10th Revision, Clinical Modification, Coding System for Diagnosis Related Groups in a Real Hospital Environment: Algorithm Development and Validation Study %A Dai,Hong-Jie %A Wang,Chen-Kai %A Chen,Chien-Chang %A Liou,Chong-Sin %A Lu,An-Tai %A Lai,Chia-Hsin %A Shain,Bo-Tsz %A Ke,Cheng-Rong %A Wang,William Yu Chung %A Mir,Tatheer Hussain %A Simanjuntak,Mutiara %A Kao,Hao-Yun %A Tsai,Ming-Ju %A Tseng,Vincent S %+ Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Kaohsiung Medical University Hospital, Kaohsiung Medical University, No 100, Tzyou 1st Road, Sanmin District, Kaohsiung, 80756, Taiwan, 886 73121101 ext 4660035, mjt@kmu.edu.tw %K natural language processing %K International Classification of Diseases %K deep learning %K electronic medical record %K Taiwan diagnosis related groups %D 2024 %7 20.9.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: International Classification of Diseases codes are widely used to describe diagnosis information, but manual coding relies heavily on human interpretation, which can be expensive, time consuming, and prone to errors. With the transition from the International Classification of Diseases, Ninth Revision, to the International Classification of Diseases, Tenth Revision (ICD-10), the coding process has become more complex, highlighting the need for automated approaches to enhance coding efficiency and accuracy. Inaccurate coding can result in substantial financial losses for hospitals, and a precise assessment of outcomes generated by a natural language processing (NLP)–driven autocoding system thus assumes a critical role in safeguarding the accuracy of the Taiwan diagnosis related groups (Tw-DRGs). Objective: This study aims to evaluate the feasibility of applying an International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM), autocoding system that can automatically determine diagnoses and codes based on free-text discharge summaries to facilitate the assessment of Tw-DRGs, specifically principal diagnosis and major diagnostic categories (MDCs). Methods: By using the patient discharge summaries from Kaohsiung Medical University Chung-Ho Memorial Hospital (KMUCHH) from April 2019 to December 2020 as a reference data set we developed artificial intelligence (AI)–assisted ICD-10-CM coding systems based on deep learning models. We constructed a web-based user interface for the AI-assisted coding system and deployed the system to the workflow of the certified coding specialists (CCSs) of KMUCHH. The data used for the assessment of Tw-DRGs were manually curated by a CCS with the principal diagnosis and MDC was determined from discharge summaries collected at KMUCHH from February 2023 to April 2023. Results: Both the reference data set and real hospital data were used to assess performance in determining ICD-10-CM coding, principal diagnosis, and MDC for Tw-DRGs. Among all methods, the GPT-2 (OpenAI)-based model achieved the highest F1-score, 0.667 (F1-score 0.851 for the top 50 codes), on the KMUCHH test set and a slightly lower F1-score, 0.621, in real hospital data. Cohen κ evaluation for the agreement of MDC between the models and the CCS revealed that the overall average κ value for GPT-2 (κ=0.714) was approximately 12.2 percentage points higher than that of the hierarchy attention network (κ=0.592). GPT-2 demonstrated superior agreement with the CCS across 6 categories of MDC, with an average κ value of approximately 0.869 (SD 0.033), underscoring the effectiveness of the developed AI-assisted coding system in supporting the work of CCSs. Conclusions: An NLP-driven AI-assisted coding system can assist CCSs in ICD-10-CM coding by offering coding references via a user interface, demonstrating the potential to reduce the manual workload and expedite Tw-DRG assessment. Consistency in performance affirmed the effectiveness of the system in supporting CCSs in ICD-10-CM coding and the judgment of Tw-DRGs. %M 39302714 %R 10.2196/58278 %U https://www.jmir.org/2024/1/e58278 %U https://doi.org/10.2196/58278 %U http://www.ncbi.nlm.nih.gov/pubmed/39302714 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 10 %N %P e46485 %T The Use of Online Consultation Systems or Remote Consulting in England Characterized Through the Primary Care Health Records of 53 Million People in the OpenSAFELY Platform: Retrospective Cohort Study %A Fonseca,Martina %A MacKenna,Brian %A Mehrkar,Amir %A , %A Walters,Caroline E %A Hickman,George %A Pearson,Jonathan %A Fisher,Louis %A Inglesby,Peter %A Bacon,Seb %A Davy,Simon %A Hulme,William %A Goldacre,Ben %A Koffman,Ofra %A Bakhai,Minal %+ NHS England, Wellington House, 133-155 Waterloo Road, London, SE1 6LH, United Kingdom, 44 6526746523, martinabfonseca@gmail.com %K online consultation system %K remote monitoring %K triage %K primary care research %K health informatics %K general practice %K digital primary care %K electronic health record coding %K OpenSAFELY %K trusted research environment %D 2024 %7 18.9.2024 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: The National Health Service (NHS) Long Term Plan, published in 2019, committed to ensuring that every patient in England has the right to digital-first primary care by 2023-2024. The COVID-19 pandemic and infection prevention and control measures accelerated work by the NHS to enable and stimulate the use of online consultation (OC) systems across all practices for improved access to primary care. Objective: We aimed to explore general practice coding activity associated with the use of OC systems in terms of trends, COVID-19 effect, variation, and quality. Methods: With the approval of NHS England, the OpenSAFELY platform was used to query and analyze the in situ electronic health records of suppliers The Phoenix Partnership (TPP) and Egton Medical Information Systems, covering >53 million patients in >6400 practices, mainly in 2019-2020. Systematized Medical Nomenclature for Medicine–Clinical Terminology (SNOMED-CT) codes relevant to OC systems and written OCs were identified including eConsultation. Events were described by volumes and population rates, practice coverage, and trends before and after the COVID-19 pandemic. Variation was characterized among practices, by sociodemographics, and by clinical history of long-term conditions. Results: Overall, 3,550,762 relevant coding events were found in practices using TPP, with the code eConsultation detected in 84.56% (2157/2551) of practices. Activity related to digital forms of interaction increased rapidly from March 2020, the onset of the pandemic; namely, in the second half of 2020, >9 monthly eConsultation coding events per 1000 registered population were registered compared to <1 a year prior. However, we found large variations among regions and practices: December 2020 saw the median practice have 0.9 coded instances per 1000 population compared to at least 36 for the highest decile of practices. On sociodemographics, the TPP cohort with OC instances, when compared (univariate analysis) to the cohort with general practitioner consultations, was more predominantly female (661,235/1,087,919, 60.78% vs 9,172,833/17,166,765, 53.43%), aged 18 to 40 years (349,162/1,080,589, 32.31% vs 4,295,711/17,000,942, 25.27%), White (730,389/1,087,919, 67.14% vs 10,887,858/17,166,765, 63.42%), and less deprived (167,889/1,068,887, 15.71% vs 3,376,403/16,867,074, 20.02%). Looking at the eConsultation code through multivariate analysis, it was more commonly recorded among patients with a history of asthma (adjusted odds ratio [aOR] 1.131, 95% CI 1.124-1.137), depression (aOR 1.144, 95% CI 1.138-1.151), or atrial fibrillation (aOR 1.119, 95% CI 1.099-1.139) when compared to other patients with general practitioner consultations, adjusted for long-term conditions, age, and gender. Conclusions: We successfully queried general practice coding activity relevant to the use of OC systems, showing increased adoption and key areas of variation during the pandemic at both sociodemographic and clinical levels. The work can be expanded to support monitoring of coding quality and underlying activity. This study suggests that large-scale impact evaluation studies can be implemented within the OpenSAFELY platform, namely looking at patient outcomes. %M 39292500 %R 10.2196/46485 %U https://publichealth.jmir.org/2024/1/e46485 %U https://doi.org/10.2196/46485 %U http://www.ncbi.nlm.nih.gov/pubmed/39292500 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e62890 %T Early Prediction of Cardiac Arrest in the Intensive Care Unit Using Explainable Machine Learning: Retrospective Study %A Kim,Yun Kwan %A Seo,Won-Doo %A Lee,Sun Jung %A Koo,Ja Hyung %A Kim,Gyung Chul %A Song,Hee Seok %A Lee,Minji %+ Department of Biomedical Software Engineering, The Catholic University of Korea, 43, Jibong-ro, Bucheon-si, Gyeonggi-do, 14662, Republic of Korea, 82 2 2164 4364, minjilee@catholic.ac.kr %K early cardiac arrest warning system %K electric medical record %K explainable clinical decision support system %K pseudo-real-time evaluation %K ensemble learning %K cost-sensitive learning %D 2024 %7 17.9.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Cardiac arrest (CA) is one of the leading causes of death among patients in the intensive care unit (ICU). Although many CA prediction models with high sensitivity have been developed to anticipate CA, their practical application has been challenging due to a lack of generalization and validation. Additionally, the heterogeneity among patients in different ICU subtypes has not been adequately addressed. Objective: This study aims to propose a clinically interpretable ensemble approach for the timely and accurate prediction of CA within 24 hours, regardless of patient heterogeneity, including variations across different populations and ICU subtypes. Additionally, we conducted patient-independent evaluations to emphasize the model’s generalization performance and analyzed interpretable results that can be readily adopted by clinicians in real-time. Methods: Patients were retrospectively analyzed using data from the Medical Information Mart for Intensive Care-IV (MIMIC-IV) and the eICU-Collaborative Research Database (eICU-CRD). To address the problem of underperformance, we constructed our framework using feature sets based on vital signs, multiresolution statistical analysis, and the Gini index, with a 12-hour window to capture the unique characteristics of CA. We extracted 3 types of features from each database to compare the performance of CA prediction between high-risk patient groups from MIMIC-IV and patients without CA from eICU-CRD. After feature extraction, we developed a tabular network (TabNet) model using feature screening with cost-sensitive learning. To assess real-time CA prediction performance, we used 10-fold leave-one-patient-out cross-validation and a cross–data set method. We evaluated MIMIC-IV and eICU-CRD across different cohort populations and subtypes of ICU within each database. Finally, external validation using the eICU-CRD and MIMIC-IV databases was conducted to assess the model’s generalization ability. The decision mask of the proposed method was used to capture the interpretability of the model. Results: The proposed method outperformed conventional approaches across different cohort populations in both MIMIC-IV and eICU-CRD. Additionally, it achieved higher accuracy than baseline models for various ICU subtypes within both databases. The interpretable prediction results can enhance clinicians’ understanding of CA prediction by serving as a statistical comparison between non-CA and CA groups. Next, we tested the eICU-CRD and MIMIC-IV data sets using models trained on MIMIC-IV and eICU-CRD, respectively, to evaluate generalization ability. The results demonstrated superior performance compared with baseline models. Conclusions: Our novel framework for learning unique features provides stable predictive power across different ICU environments. Most of the interpretable global information reveals statistical differences between CA and non-CA groups, demonstrating its utility as an indicator for clinical decisions. Consequently, the proposed CA prediction system is a clinically validated algorithm that enables clinicians to intervene early based on CA prediction information and can be applied to clinical trials in digital health. %R 10.2196/62890 %U https://www.jmir.org/2024/1/e62890 %U https://doi.org/10.2196/62890 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e56370 %T A Behavior-Based Model to Validate Electronic Systems Designed to Collect Patient-Reported Outcomes: Model Development and Application %A Attamimi,Sultan %A Marshman,Zoe %A Deery,Christopher %A Radley,Stephen %A Gilchrist,Fiona %+ Academic Unit of Oral Health Dentistry and Society, University of Sheffield, 19 Claremont Cres, Broomhall, Sheffield, S10 2TA, United Kingdom, 44 0114 2717990, su.altamimi@uoh.edu.sa %K patient-reported outcome %K PRO %K electronic PRO %K user acceptance testing %K system validation %K patient-reported outcomes %K electronic PROs %K user acceptance %K validation model %K paediatric dentistry %D 2024 %7 17.9.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: The merits of technology have been adopted in capturing patient-reported outcomes (PROs) by incorporating PROs into electronic systems. Following the development of an electronic system, evaluation of system performance is crucial to ensuring the collection of meaningful data. In contemporary PRO literature, electronic system validation is overlooked, and evidence on validation methods is lacking. Objective: This study aims to introduce a generalized concept to guide electronic patient-reported outcome (ePRO) providers in planning for system-specific validation methods. Methods: Since electronic systems are essentially products of software engineering endeavors, electronic systems used to collect PRO should be viewed from a computer science perspective with consideration to the health care environment. On this basis, a testing model was blueprinted and applied to a newly developed ePRO system designed for clinical use in pediatric dentistry (electronic Personal Assessment Questionnaire-Paediatric Dentistry) to investigate its thoroughness. Results: A behavior-based model of ePRO system validation was developed based on the principles of user acceptance testing and patient-centered care. The model allows systematic inspection of system specifications and identification of technical errors through simulated positive and negative usage pathways in open and closed environments. The model was able to detect 15 positive errors with 1 unfavorable response when applied to electronic Personal Assessment Questionnaire-Paediatric Dentistry system testing. Conclusions: The application of the behavior-based model to a newly developed ePRO system showed a high ability for technical error detection in a systematic fashion. The proposed model will increase confidence in the validity of ePRO systems as data collection tools in future research and clinical practice. %M 39288407 %R 10.2196/56370 %U https://formative.jmir.org/2024/1/e56370 %U https://doi.org/10.2196/56370 %U http://www.ncbi.nlm.nih.gov/pubmed/39288407 %0 Journal Article %@ 2561-9128 %I JMIR Publications %V 7 %N %P e63076 %T Association of a Novel Electronic Form for Preoperative Cardiac Risk Assessment With Reduction in Cardiac Consultations and Testing: Retrospective Cohort Study %A Kumar,Mandeep %A Wilkinson,Kathryn %A Li,Ya-Huei %A Masih,Rohit %A Gandhi,Mehak %A Saadat,Haleh %A Culmone,Julie %+ Pre-Admission Testing Center, Perioperative Medicine, Hartford HealthCare, 85 Seymour St, Suite 601, Hartford, CT, 06106, United States, 1 860 972 2334, mandeep.kumar@hhchealth.org %K preoperative %K cardiology consultations %K decrease low value care %K cardiology %K cardiac %K cohort %K surgery %K surgical %K EMR %K EMRs %K EHR %K EHRs %K electronic medical record %K electronic medical records %K electronic health record %K electronic health records %K form %K forms %K assessment %K assessments %K risk %K risks %K referral %K consultation %K consultations %K testing %K diagnosis %K diagnoses %K diagnostic %K diagnostics %D 2024 %7 13.9.2024 %9 Original Paper %J JMIR Perioper Med %G English %X Background: Preoperative cardiac risk assessment is an integral part of preoperative evaluation; however, there is significant variation among providers, leading to inappropriate referrals for cardiology consultation or excessive low-value cardiac testing. We implemented a novel electronic medical record (EMR) form in our preoperative clinics to decrease variation. Objective: This study aimed to investigate the impact of the EMR form on the preoperative utilization of cardiology consultation and cardiac diagnostic testing (echocardiograms, stress tests, and cardiac catheterization) and evaluate postoperative outcomes. Methods: A retrospective cohort study was conducted. Patients who underwent outpatient preoperative evaluation prior to an elective surgery over 2 years were divided into 2 cohorts: from July 1, 2021, to June 30, 2022 (pre–EMR form implementation), and from July 1, 2022, to June 30, 2023 (post–EMR form implementation). Demographics, comorbidities, resource utilization, and surgical characteristics were analyzed. Propensity score matching was used to adjust for differences between the 2 cohorts. The primary outcomes were the utilization of preoperative cardiology consultation, cardiac testing, and 30-day postoperative major adverse cardiac events (MACE). Results: A total of 25,484 patients met the inclusion criteria. Propensity score matching yielded 11,645 well-matched pairs. The post–EMR form, matched cohort had lower cardiology consultation (pre–EMR form: n=2698, 23.2% vs post–EMR form: n=2088, 17.9%; P<.001) and echocardiogram (pre–EMR form: n=808, 6.9% vs post–EMR form: n=591, 5.1%; P<.001) utilization. There were no significant differences in the 30-day postoperative outcomes, including MACE (all P>.05). While patients with “possible indications” for cardiology consultation had higher MACE rates, the consultations did not reduce MACE risk. Most algorithm end points, except for active cardiac conditions, had MACE rates <1%. Conclusions: In this cohort study, preoperative cardiac risk assessment using a novel EMR form was associated with a significant decrease in cardiology consultation and testing utilization, with no adverse impact on postoperative outcomes. Adopting this approach may assist perioperative medicine clinicians and anesthesiologists in efficiently decreasing unnecessary preoperative resource utilization without compromising patient safety or quality of care. %M 39269754 %R 10.2196/63076 %U https://periop.jmir.org/2024/1/e63076 %U https://doi.org/10.2196/63076 %U http://www.ncbi.nlm.nih.gov/pubmed/39269754 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e57949 %T Natural Language Processing Versus Diagnosis Code–Based Methods for Postherpetic Neuralgia Identification: Algorithm Development and Validation %A Zheng,Chengyi %A Ackerson,Bradley %A Qiu,Sijia %A Sy,Lina S %A Daily,Leticia I Vega %A Song,Jeannie %A Qian,Lei %A Luo,Yi %A Ku,Jennifer H %A Cheng,Yanjun %A Wu,Jun %A Tseng,Hung Fu %K postherpetic neuralgia %K herpes zoster %K natural language processing %K electronic health record %K real-world data %K artificial intelligence %K development %K validation %K diagnosis %K EHR %K algorithm %K EHR data %K sensitivity %K specificity %K validation data %K neuralgia %K recombinant zoster vaccine %D 2024 %7 10.9.2024 %9 %J JMIR Med Inform %G English %X Background: Diagnosis codes and prescription data are used in algorithms to identify postherpetic neuralgia (PHN), a debilitating complication of herpes zoster (HZ). Because of the questionable accuracy of codes and prescription data, manual chart review is sometimes used to identify PHN in electronic health records (EHRs), which can be costly and time-consuming. Objective: This study aims to develop and validate a natural language processing (NLP) algorithm for automatically identifying PHN from unstructured EHR data and to compare its performance with that of code-based methods. Methods: This retrospective study used EHR data from Kaiser Permanente Southern California, a large integrated health care system that serves over 4.8 million members. The source population included members aged ≥50 years who received an incident HZ diagnosis and accompanying antiviral prescription between 2018 and 2020 and had ≥1 encounter within 90‐180 days of the incident HZ diagnosis. The study team manually reviewed the EHR and identified PHN cases. For NLP development and validation, 500 and 800 random samples from the source population were selected, respectively. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F-score, and Matthews correlation coefficient (MCC) of NLP and the code-based methods were evaluated using chart-reviewed results as the reference standard. Results: The NLP algorithm identified PHN cases with a 90.9% sensitivity, 98.5% specificity, 82% PPV, and 99.3% NPV. The composite scores of the NLP algorithm were 0.89 (F-score) and 0.85 (MCC). The prevalences of PHN in the validation data were 6.9% (reference standard), 7.6% (NLP), and 5.4%‐13.1% (code-based). The code-based methods achieved a 52.7%‐61.8% sensitivity, 89.8%‐98.4% specificity, 27.6%‐72.1% PPV, and 96.3%‐97.1% NPV. The F-scores and MCCs ranged between 0.45 and 0.59 and between 0.32 and 0.61, respectively. Conclusions: The automated NLP-based approach identified PHN cases from the EHR with good accuracy. This method could be useful in population-based PHN research. %R 10.2196/57949 %U https://medinform.jmir.org/2024/1/e57949 %U https://doi.org/10.2196/57949 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e57195 %T Automated Identification of Postoperative Infections to Allow Prediction and Surveillance Based on Electronic Health Record Data: Scoping Review %A van der Meijden,Siri Lise %A van Boekel,Anna M %A van Goor,Harry %A Nelissen,Rob GHH %A Schoones,Jan W %A Steyerberg,Ewout W %A Geerts,Bart F %A de Boer,Mark GJ %A Arbous,M Sesmu %+ Intensive Care Unit, Leiden University Medical Center, Albinusdreef 2, Leiden, 2333 ZA, Netherlands, 31 526 9111, S.L.van_der_meijden@lumc.nl %K postoperative infections %K surveillance %K prediction %K surgery %K artificial intelligence %K chart review %K electronic health record %K scoping review %K postoperative %K surgical %K infection %K infections %K predictions %K predict %K predictive %K bacterial %K machine learning %K record %K records %K EHR %K EHRs %K synthesis %K review methods %K review methodology %K search %K searches %K searching %K scoping %D 2024 %7 10.9.2024 %9 Review %J JMIR Med Inform %G English %X Background: Postoperative infections remain a crucial challenge in health care, resulting in high morbidity, mortality, and costs. Accurate identification and labeling of patients with postoperative bacterial infections is crucial for developing prediction models, validating biomarkers, and implementing surveillance systems in clinical practice. Objective: This scoping review aimed to explore methods for identifying patients with postoperative infections using electronic health record (EHR) data to go beyond the reference standard of manual chart review. Methods: We performed a systematic search strategy across PubMed, Embase, Web of Science (Core Collection), the Cochrane Library, and Emcare (Ovid), targeting studies addressing the prediction and fully automated surveillance (ie, without manual check) of diverse bacterial infections in the postoperative setting. For prediction modeling studies, we assessed the labeling methods used, categorizing them as either manual or automated. We evaluated the different types of EHR data needed for the surveillance and labeling of postoperative infections, as well as the performance of fully automated surveillance systems compared with manual chart review. Results: We identified 75 different methods and definitions used to identify patients with postoperative infections in studies published between 2003 and 2023. Manual labeling was the predominant method in prediction modeling research, 65% (49/75) of the identified methods use structured data, and 45% (34/75) use free text and clinical notes as one of their data sources. Fully automated surveillance systems should be used with caution because the reported positive predictive values are between 0.31 and 0.76. Conclusions: There is currently no evidence to support fully automated labeling and identification of patients with infections based solely on structured EHR data. Future research should focus on defining uniform definitions, as well as prioritizing the development of more scalable, automated methods for infection detection using structured EHR data. %M 39255011 %R 10.2196/57195 %U https://medinform.jmir.org/2024/1/e57195 %U https://doi.org/10.2196/57195 %U http://www.ncbi.nlm.nih.gov/pubmed/39255011 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e58705 %T Identifications of Similarity Metrics for Patients With Cancer: Protocol for a Scoping Review %A Manuilova,Iryna %A Bossenz,Jan %A Weise,Annemarie Bianka %A Boehm,Dominik %A Strantz,Cosima %A Unberath,Philipp %A Reimer,Niklas %A Metzger,Patrick %A Pauli,Thomas %A Werle,Silke D %A Schulze,Susann %A Hiemer,Sonja %A Ustjanzew,Arsenij %A Kestler,Hans A %A Busch,Hauke %A Brors,Benedikt %A Christoph,Jan %+ Junior Research Group (Bio-) Medical Data Science, Faculty of Medicine, Martin Luther University Halle-Wittenberg, Magdeburger Str 8, Halle (Saale), 06112, Germany, 49 345 557 2651, Iryna.Manuilova@uk-halle.de %K patient similarity %K cancer research %K patient similarity applications %K precision medicine %K cancer similarity metrics %K scoping review protocol %D 2024 %7 4.9.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Understanding the similarities of patients with cancer is essential to advancing personalized medicine, improving patient outcomes, and developing more effective and individualized treatments. It enables researchers to discover important patterns, biomarkers, and treatment strategies that can have a significant impact on cancer research and oncology. In addition, the identification of previously successfully treated patients supports oncologists in making treatment decisions for a new patient who is clinically or molecularly similar to the previous patient. Objective: The planned review aims to systematically summarize, map, and describe existing evidence to understand how patient similarity is defined and used in cancer research and clinical care. Methods: To systematically identify relevant studies and to ensure reproducibility and transparency of the review process, a comprehensive literature search will be conducted in several bibliographic databases, including Web of Science, PubMed, LIVIVIVO, and MEDLINE, covering the period from 1998 to February 2024. After the initial duplicate deletion phase, a study selection phase will be applied using Rayyan, which consists of 3 distinct steps: title and abstract screening, disagreement resolution, and full-text screening. To ensure the integrity and quality of the selection process, each of these steps is preceded by a pilot testing phase. This methodological process will culminate in the presentation of the final research results in a structured form according to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) flowchart. The protocol has been registered in the Journal of Medical Internet Research. Results: This protocol outlines the methodologies used in conducting the scoping review. A search of the specified electronic databases and after removing duplicates resulted in 1183 unique records. As of March 2024, the review process has moved to the full-text evaluation phase. At this stage, data extraction will be conducted using a pretested chart template. Conclusions: The scoping review protocol, centered on these main concepts, aims to systematically map the available evidence on patient similarity among patients with cancer. By defining the types of data sources, approaches, and methods used in the field, and aligning these with the research questions, the review will provide a foundation for future research and clinical application in personalized cancer care. This protocol will guide the literature search, data extraction, and synthesis of findings to achieve the review’s objectives. International Registered Report Identifier (IRRID): DERR1-10.2196/58705 %M 39230952 %R 10.2196/58705 %U https://www.researchprotocols.org/2024/1/e58705 %U https://doi.org/10.2196/58705 %U http://www.ncbi.nlm.nih.gov/pubmed/39230952 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e46608 %T Predictors of Medical and Dental Clinic Closure by Machine Learning Methods: Cross-Sectional Study Using Empirical Data %A Park,Young-Taek %A Kim,Donghan %A Jeon,Ji Soo %A Kim,Kwang Gi %+ Department of Biomedical Engineering, College of Medicine, Gil Medical Center, Gachon University, 58-13 Docjemro, NamdongGum, Inchon, 21565, Republic of Korea, 82 324582770, kimkg@gachon.ac.kr %K machine learning %K health facility closure %K hospital closure %K clinic closure %K clinic bankruptcy %K hospital bankruptcy %K health clinic %K prediction %K healthcare resources %K artificial intelligence %K medical clinic %K health insurance %K health facility closure %D 2024 %7 30.8.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Small clinics are important in providing health care in local communities. Accurately predicting their closure would help manage health care resource allocation. There have been few studies on the prediction of clinic closure using machine learning techniques. Objective: This study aims to test the feasibility of predicting the closure of medical and dental clinics (MCs and DCs, respectively) and investigate important factors associated with their closure using machine running techniques. Methods: The units of analysis were MCs and DCs. This study used health insurance administrative data. The participants of this study ran and closed clinics between January 1, 2020, and December 31, 2021. Using all closed clinics, closed and run clinics were selected at a ratio of 1:2 based on the locality of study participants using the propensity matching score of logistic regression. This study used 23 and 19 variables to predict the closure of MCs and DCs, respectively. Key variables were extracted using permutation importance and the sequential feature selection technique. Finally, this study used 5 and 6 variables of MCs and DCs, respectively, for model learning. Furthermore, four machine learning techniques were used: (1) logistic regression, (2) support vector machine, (3) random forest (RF), and (4) Extreme Gradient Boost. This study evaluated the modeling accuracy using the area under curve (AUC) method and presented important factors critically affecting closures. This study used SAS (version 9.4; SAS Institute Inc) and Python (version 3.7.9; Python Software Foundation). Results: The best-fit model for the closure of MCs with cross-validation was the support vector machine (AUC 0.762, 95% CI 0.746-0.777; P<.001) followed by RF (AUC 0.736, 95% CI 0.720-0.752; P<.001). The best-fit model for DCs was Extreme Gradient Boost (AUC 0.700, 95% CI 0.675-0.725; P<.001) followed by RF (AUC 0.687, 95% CI 0.661-0.712; P<.001). The most significant factor associated with the closure of MCs was years of operation, followed by population growth, population, and percentage of medical specialties. In contrast, the main factor affecting the closure of DCs was the number of patients, followed by annual variation in the number of patients, year of operation, and percentage of dental specialists. Conclusions: This study showed that machine running methods are useful tools for predicting the closure of small medical facilities with a moderate level of accuracy. Essential factors affecting medical facility closure also differed between MCs and DCs. Developing good models would prevent unnecessary medical facility closures at the national level. %M 39213534 %R 10.2196/46608 %U https://www.jmir.org/2024/1/e46608 %U https://doi.org/10.2196/46608 %U http://www.ncbi.nlm.nih.gov/pubmed/39213534 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e56170 %T Optimizing Response Rates to Examine Health IT Maturity and Nurse Practitioner Care Environments in US Nursing Homes: Mixed Mode Survey Recruitment Protocol %A Alexander,Gregory L %A Poghosyan,Lusine %A Zhao,Yihong %A Hobensack,Mollie %A Kisselev,Sergey %A Norful,Allison A %A McHugh,John %A Wise,Keely %A Schrimpf,M Brooke %A Kolanowski,Ann %A Bhatia,Tamanna %A Tasnova,Sabrina %+ School of Nursing, Columbia University, 560 W. 168 Room 628, New York, NY, 10032, United States, 1 5733013131, ga2545@cumc.columbia.edu %K surveys and questionnaires %K survey methods %K health care surveys %K survey %K survey design %K mixed-mMode survey %K nursing homes %K nursing home %K clinical informatics research %K electronic health records %K electronic health record %K clinicians %K HIT Maturity %K Care Environments %K United States %D 2024 %7 29.8.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Survey-driven research is a reliable method for large-scale data collection. Investigators incorporating mixed-mode survey designs report benefits for survey research including greater engagement, improved survey access, and higher response rate. Mix-mode survey designs combine 2 or more modes for data collection including web, phone, face-to-face, and mail. Types of mixed-mode survey designs include simultaneous (ie, concurrent), sequential, delayed concurrent, and adaptive. This paper describes a research protocol using mixed-mode survey designs to explore health IT (HIT) maturity and care environments reported by administrators and nurse practitioners (NPs), respectively, in US nursing homes (NHs). Objective: The aim of this study is to describe a research protocol using mixed-mode survey designs in research using 2 survey tools to explore HIT maturity and NP care environments in US NHs. Methods: We are conducting a national survey of 1400 NH administrators and NPs. Two data sets (ie, Care Compare and IQVIA) were used to identify eligible facilities at random. The protocol incorporates 2 surveys to explore how HIT maturity (survey 1 collected by administrators) impacts care environments where NPs work (survey 2 collected by NPs). Higher HIT maturity collected by administrators indicates greater IT capabilities, use, and integration in resident care, clinical support, and administrative activities. The NP care environment survey measures relationships, independent practice, resource availability, and visibility. The research team conducted 3 iterative focus groups, including 14 clinicians (NP and NH experts) and recruiters from 2 national survey teams experienced with these populations to achieve consensus on which mixed-mode designs to use. During focus groups we identified the pros and cons of using mixed-mode designs in these settings. We determined that 2 mixed-mode designs with regular follow-up calls (Delayed Concurrent Mode and Sequential Mode) is effective for recruiting NH administrators while a concurrent mixed-mode design is best to recruit NPs. Results: Participant recruitment for the project began in June 2023. As of April 22, 2024, a total of 98 HIT maturity surveys and 81 NP surveys have been returned. Recruitment of NH administrators and NPs is anticipated through July 2025. About 71% of the HIT maturity surveys have been submitted using the electronic link and 23% were submitted after a QR code was sent to the administrator. Approximately 95% of the NP surveys were returned with electronic survey links. Conclusions: Pros of mixed-mode designs for NH research identified by the team were that delayed concurrent, concurrent, and sequential mixed-mode methods of delivering surveys to potential participants save on recruitment time compared to single mode delivery methods. One disadvantage of single-mode strategies is decreased versatility and adaptability to different organizational capabilities (eg, access to email and firewalls), which could reduce response rates. International Registered Report Identifier (IRRID): DERR1-10.2196/56170 %M 39207828 %R 10.2196/56170 %U https://www.researchprotocols.org/2024/1/e56170 %U https://doi.org/10.2196/56170 %U http://www.ncbi.nlm.nih.gov/pubmed/39207828 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e59617 %T Viability of Open Large Language Models for Clinical Documentation in German Health Care: Real-World Model Evaluation Study %A Heilmeyer,Felix %A Böhringer,Daniel %A Reinhard,Thomas %A Arens,Sebastian %A Lyssenko,Lisa %A Haverkamp,Christian %K machine learning %K ML %K artificial intelligence %K AI %K large language model %K large language models %K LLM %K LLMs %K natural language processing %K NLP %K deep learning %K algorithm %K algorithms %K model %K models %K analytics %K practical model %K practical models %K medical documentation %K writing assistance %K medical administration %K writing assistance for physicians %D 2024 %7 28.8.2024 %9 %J JMIR Med Inform %G English %X Background: The use of large language models (LLMs) as writing assistance for medical professionals is a promising approach to reduce the time required for documentation, but there may be practical, ethical, and legal challenges in many jurisdictions complicating the use of the most powerful commercial LLM solutions. Objective: In this study, we assessed the feasibility of using nonproprietary LLMs of the GPT variety as writing assistance for medical professionals in an on-premise setting with restricted compute resources, generating German medical text. Methods: We trained four 7-billion–parameter models with 3 different architectures for our task and evaluated their performance using a powerful commercial LLM, namely Anthropic’s Claude-v2, as a rater. Based on this, we selected the best-performing model and evaluated its practical usability with 2 independent human raters on real-world data. Results: In the automated evaluation with Claude-v2, BLOOM-CLP-German, a model trained from scratch on the German text, achieved the best results. In the manual evaluation by human experts, 95 (93.1%) of the 102 reports generated by that model were evaluated as usable as is or with only minor changes by both human raters. Conclusions: The results show that even with restricted compute resources, it is possible to generate medical texts that are suitable for documentation in routine clinical practice. However, the target language should be considered in the model selection when processing non-English text. %R 10.2196/59617 %U https://medinform.jmir.org/2024/1/e59617 %U https://doi.org/10.2196/59617 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e50935 %T Evaluation of a Natural Language Processing Approach to Identify Diagnostic Errors and Analysis of Safety Learning System Case Review Data: Retrospective Cohort Study %A Tabaie,Azade %A Tran,Alberta %A Calabria,Tony %A Bennett,Sonita S %A Milicia,Arianna %A Weintraub,William %A Gallagher,William James %A Yosaitis,John %A Schubel,Laura C %A Hill,Mary A %A Smith,Kelly Michelle %A Miller,Kristen %+ Center for Biostatistics, Informatics, and Data Science, MedStar Health Research Institute, 3007 Tilden Street NW, Washington, DC, 20008, United States, 1 202 244 9810, azade.tabaie@medstar.net %K diagnostic error %K electronic health records %K machine learning %K natural language processing %K NLP %K mortality %K hospital %K risk %K length of stay %K patient harm %K diagnostic %K EHR %D 2024 %7 26.8.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Diagnostic errors are an underappreciated cause of preventable mortality in hospitals and pose a risk for severe patient harm and increase hospital length of stay. Objective: This study aims to explore the potential of machine learning and natural language processing techniques in improving diagnostic safety surveillance. We conducted a rigorous evaluation of the feasibility and potential to use electronic health records clinical notes and existing case review data. Methods: Safety Learning System case review data from 1 large health system composed of 10 hospitals in the mid-Atlantic region of the United States from February 2016 to September 2021 were analyzed. The case review outcome included opportunities for improvement including diagnostic opportunities for improvement. To supplement case review data, electronic health record clinical notes were extracted and analyzed. A simple logistic regression model along with 3 forms of logistic regression models (ie, Least Absolute Shrinkage and Selection Operator, Ridge, and Elastic Net) with regularization functions was trained on this data to compare classification performances in classifying patients who experienced diagnostic errors during hospitalization. Further, statistical tests were conducted to find significant differences between female and male patients who experienced diagnostic errors. Results: In total, 126 (7.4%) patients (of 1704) had been identified by case reviewers as having experienced at least 1 diagnostic error. Patients who had experienced diagnostic error were grouped by sex: 59 (7.1%) of the 830 women and 67 (7.7%) of the 874 men. Among the patients who experienced a diagnostic error, female patients were older (median 72, IQR 66-80 vs median 67, IQR 57-76; P=.02), had higher rates of being admitted through general or internal medicine (69.5% vs 47.8%; P=.01), lower rates of cardiovascular-related admitted diagnosis (11.9% vs 28.4%; P=.02), and lower rates of being admitted through neurology department (2.3% vs 13.4%; P=.04). The Ridge model achieved the highest area under the receiver operating characteristic curve (0.885), specificity (0.797), positive predictive value (PPV; 0.24), and F1-score (0.369) in classifying patients who were at higher risk of diagnostic errors among hospitalized patients. Conclusions: Our findings demonstrate that natural language processing can be a potential solution to more effectively identifying and selecting potential diagnostic error cases for review and therefore reducing the case review burden. %M 39186764 %R 10.2196/50935 %U https://www.jmir.org/2024/1/e50935 %U https://doi.org/10.2196/50935 %U http://www.ncbi.nlm.nih.gov/pubmed/39186764 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e46936 %T Sex-Based Performance Disparities in Machine Learning Algorithms for Cardiac Disease Prediction: Exploratory Study %A Straw,Isabel %A Rees,Geraint %A Nachev,Parashkev %+ University College London, 222 Euston Road, London, NW1 2DA, United Kingdom, 44 020 3549 5969, isabelstraw@doctors.org.uk %K artificial intelligence %K machine learning %K cardiology %K health care %K health equity %K medicine %K cardiac %K quantitative evaluation %K inequality %K cardiac disease %K performance %K sex %K management %K heart failure %D 2024 %7 26.8.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: The presence of bias in artificial intelligence has garnered increased attention, with inequities in algorithmic performance being exposed across the fields of criminal justice, education, and welfare services. In health care, the inequitable performance of algorithms across demographic groups may widen health inequalities. Objective: Here, we identify and characterize bias in cardiology algorithms, looking specifically at algorithms used in the management of heart failure. Methods: Stage 1 involved a literature search of PubMed and Web of Science for key terms relating to cardiac machine learning (ML) algorithms. Papers that built ML models to predict cardiac disease were evaluated for their focus on demographic bias in model performance, and open-source data sets were retained for our investigation. Two open-source data sets were identified: (1) the University of California Irvine Heart Failure data set and (2) the University of California Irvine Coronary Artery Disease data set. We reproduced existing algorithms that have been reported for these data sets, tested them for sex biases in algorithm performance, and assessed a range of remediation techniques for their efficacy in reducing inequities. Particular attention was paid to the false negative rate (FNR), due to the clinical significance of underdiagnosis and missed opportunities for treatment. Results: In stage 1, our literature search returned 127 papers, with 60 meeting the criteria for a full review and only 3 papers highlighting sex differences in algorithm performance. In the papers that reported sex, there was a consistent underrepresentation of female patients in the data sets. No papers investigated racial or ethnic differences. In stage 2, we reproduced algorithms reported in the literature, achieving mean accuracies of 84.24% (SD 3.51%) for data set 1 and 85.72% (SD 1.75%) for data set 2 (random forest models). For data set 1, the FNR was significantly higher for female patients in 13 out of 16 experiments, meeting the threshold of statistical significance (–17.81% to –3.37%; P<.05). A smaller disparity in the false positive rate was significant for male patients in 13 out of 16 experiments (–0.48% to +9.77%; P<.05). We observed an overprediction of disease for male patients (higher false positive rate) and an underprediction of disease for female patients (higher FNR). Sex differences in feature importance suggest that feature selection needs to be demographically tailored. Conclusions: Our research exposes a significant gap in cardiac ML research, highlighting that the underperformance of algorithms for female patients has been overlooked in the published literature. Our study quantifies sex disparities in algorithmic performance and explores several sources of bias. We found an underrepresentation of female patients in the data sets used to train algorithms, identified sex biases in model error rates, and demonstrated that a series of remediation techniques were unable to address the inequities present. %M 39186324 %R 10.2196/46936 %U https://www.jmir.org/2024/1/e46936 %U https://doi.org/10.2196/46936 %U http://www.ncbi.nlm.nih.gov/pubmed/39186324 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e54616 %T AI-Driven Diagnostic Assistance in Medical Inquiry: Reinforcement Learning Algorithm Development and Validation %A Zou,Xuan %A He,Weijie %A Huang,Yu %A Ouyang,Yi %A Zhang,Zhen %A Wu,Yu %A Wu,Yongsheng %A Feng,Lili %A Wu,Sheng %A Yang,Mengqi %A Chen,Xuyan %A Zheng,Yefeng %A Jiang,Rui %A Chen,Ting %+ Department of Computer Science and Technology, Tsinghua University, Room 3-609, Future Internet Technology Research Center, Tsinghua University, Beijing, 100084, China, 86 010 62797101, tingchen@tsinghua.edu.cn %K inquiry and diagnosis %K electronic health record %K reinforcement learning %K natural language processing %K artificial intelligence %D 2024 %7 23.8.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: For medical diagnosis, clinicians typically begin with a patient’s chief concerns, followed by questions about symptoms and medical history, physical examinations, and requests for necessary auxiliary examinations to gather comprehensive medical information. This complex medical investigation process has yet to be modeled by existing artificial intelligence (AI) methodologies. Objective: The aim of this study was to develop an AI-driven medical inquiry assistant for clinical diagnosis that provides inquiry recommendations by simulating clinicians’ medical investigating logic via reinforcement learning. Methods: We compiled multicenter, deidentified outpatient electronic health records from 76 hospitals in Shenzhen, China, spanning the period from July to November 2021. These records consisted of both unstructured textual information and structured laboratory test results. We first performed feature extraction and standardization using natural language processing techniques and then used a reinforcement learning actor-critic framework to explore the rational and effective inquiry logic. To align the inquiry process with actual clinical practice, we segmented the inquiry into 4 stages: inquiring about symptoms and medical history, conducting physical examinations, requesting auxiliary examinations, and terminating the inquiry with a diagnosis. External validation was conducted to validate the inquiry logic of the AI model. Results: This study focused on 2 retrospective inquiry-and-diagnosis tasks in the emergency and pediatrics departments. The emergency departments provided records of 339,020 consultations including mainly children (median age 5.2, IQR 2.6-26.1 years) with various types of upper respiratory tract infections (250,638/339,020, 73.93%). The pediatrics department provided records of 561,659 consultations, mainly of children (median age 3.8, IQR 2.0-5.7 years) with various types of upper respiratory tract infections (498,408/561,659, 88.73%). When conducting its own inquiries in both scenarios, the AI model demonstrated high diagnostic performance, with areas under the receiver operating characteristic curve of 0.955 (95% CI 0.953-0.956) and 0.943 (95% CI 0.941-0.944), respectively. When the AI model was used in a simulated collaboration with physicians, it notably reduced the average number of physicians’ inquiries to 46% (6.037/13.26; 95% CI 6.009-6.064) and 43% (6.245/14.364; 95% CI 6.225-6.269) while achieving areas under the receiver operating characteristic curve of 0.972 (95% CI 0.970-0.973) and 0.968 (95% CI 0.967-0.969) in the scenarios. External validation revealed a normalized Kendall τ distance of 0.323 (95% CI 0.301-0.346), indicating the inquiry consistency of the AI model with physicians. Conclusions: This retrospective analysis of predominantly respiratory pediatric presentations in emergency and pediatrics departments demonstrated that an AI-driven diagnostic assistant had high diagnostic performance both in stand-alone use and in simulated collaboration with clinicians. Its investigation process was found to be consistent with the clinicians’ medical investigation logic. These findings highlight the diagnostic assistant’s promise in assisting the decision-making processes of health care professionals. %M 39178403 %R 10.2196/54616 %U https://www.jmir.org/2024/1/e54616 %U https://doi.org/10.2196/54616 %U http://www.ncbi.nlm.nih.gov/pubmed/39178403 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e57615 %T Data Quality–Driven Improvement in Health Care: Systematic Literature Review %A Lighterness,Anthony %A Adcock,Michael %A Scanlon,Lauren Abigail %A Price,Gareth %+ Clinical Outcomes and Data Unit, The Christie NHS Foundation Trust, 550 Wilmslow Road, Manchester, M20 4BX, United Kingdom, 44 7305054646, anthony.lighterness@nhs.net %K real-world data %K data quality %K quality improvement %K systematic literature review %K PRISMA %D 2024 %7 22.8.2024 %9 Review %J J Med Internet Res %G English %X Background: The promise of real-world evidence and the learning health care system primarily depends on access to high-quality data. Despite widespread awareness of the prevalence and potential impacts of poor data quality (DQ), best practices for its assessment and improvement are unknown. Objective: This review aims to investigate how existing research studies define, assess, and improve the quality of structured real-world health care data. Methods: A systematic literature search of studies in the English language was implemented in the Embase and PubMed databases to select studies that specifically aimed to measure and improve the quality of structured real-world data within any clinical setting. The time frame for the analysis was from January 1945 to June 2023. We standardized DQ concepts according to the Data Management Association (DAMA) DQ framework to enable comparison between studies. After screening and filtering by 2 independent authors, we identified 39 relevant articles reporting DQ improvement initiatives. Results: The studies were characterized by considerable heterogeneity in settings and approaches to DQ assessment and improvement. Affiliated institutions were from 18 different countries and 18 different health domains. DQ assessment methods were largely manual and targeted completeness and 1 other DQ dimension. Use of DQ frameworks was limited to the Weiskopf and Weng (3/6, 50%) or Kahn harmonized model (3/6, 50%). Use of standardized methodologies to design and implement quality improvement was lacking, but mainly included plan-do-study-act (PDSA) or define-measure-analyze-improve-control (DMAIC) cycles. Most studies reported DQ improvements using multiple interventions, which included either DQ reporting and personalized feedback (24/39, 61%), IT-related solutions (21/39, 54%), training (17/39, 44%), improvements in workflows (5/39, 13%), or data cleaning (3/39, 8%). Most studies reported improvements in DQ through a combination of these interventions. Statistical methods were used to determine significance of treatment effect (22/39, 56% times), but only 1 study implemented a randomized controlled study design. Variability in study designs, approaches to delivering interventions, and reporting DQ changes hindered a robust meta-analysis of treatment effects. Conclusions: There is an urgent need for standardized guidelines in DQ improvement research to enable comparison and effective synthesis of lessons learned. Frameworks such as PDSA learning cycles and the DAMA DQ framework can facilitate this unmet need. In addition, DQ improvement studies can also benefit from prioritizing root cause analysis of DQ issues to ensure the most appropriate intervention is implemented, thereby ensuring long-term, sustainable improvement. Despite the rise in DQ improvement studies in the last decade, significant heterogeneity in methodologies and reporting remains a challenge. Adopting standardized frameworks for DQ assessment, analysis, and improvement can enhance the effectiveness, comparability, and generalizability of DQ improvement initiatives. %M 39173155 %R 10.2196/57615 %U https://www.jmir.org/2024/1/e57615 %U https://doi.org/10.2196/57615 %U http://www.ncbi.nlm.nih.gov/pubmed/39173155 %0 Journal Article %@ 1929-073X %I JMIR Publications %V 13 %N %P e46946 %T Exploring Computational Techniques in Preprocessing Neonatal Physiological Signals for Detecting Adverse Outcomes: Scoping Review %A Rahman,Jessica %A Brankovic,Aida %A Tracy,Mark %A Khanna,Sankalp %+ Commonwealth Scientific and Industrial Research Organisation (CSIRO) Australian e-Health Research Centre, Australia, 160 Hawkesbury Road, Sydney, 2145, Australia, 61 02 9325 3016, jessica.rahman@csiro.au %K physiological signals %K preterm %K neonatal intensive care unit %K morbidity %K signal processing %K signal analysis %K adverse outcomes %K predictive and diagnostic models %D 2024 %7 20.8.2024 %9 Review %J Interact J Med Res %G English %X Background: Computational signal preprocessing is a prerequisite for developing data-driven predictive models for clinical decision support. Thus, identifying the best practices that adhere to clinical principles is critical to ensure transparency and reproducibility to drive clinical adoption. It further fosters reproducible, ethical, and reliable conduct of studies. This procedure is also crucial for setting up a software quality management system to ensure regulatory compliance in developing software as a medical device aimed at early preclinical detection of clinical deterioration. Objective: This scoping review focuses on the neonatal intensive care unit setting and summarizes the state-of-the-art computational methods used for preprocessing neonatal clinical physiological signals; these signals are used for the development of machine learning models to predict the risk of adverse outcomes. Methods: Five databases (PubMed, Web of Science, Scopus, IEEE, and ACM Digital Library) were searched using a combination of keywords and MeSH (Medical Subject Headings) terms. A total of 3585 papers from 2013 to January 2023 were identified based on the defined search terms and inclusion criteria. After removing duplicates, 2994 (83.51%) papers were screened by title and abstract, and 81 (0.03%) were selected for full-text review. Of these, 52 (64%) were eligible for inclusion in the detailed analysis. Results: Of the 52 articles reviewed, 24 (46%) studies focused on diagnostic models, while the remainder (n=28, 54%) focused on prognostic models. The analysis conducted in these studies involved various physiological signals, with electrocardiograms being the most prevalent. Different programming languages were used, with MATLAB and Python being notable. The monitoring and capturing of physiological data used diverse systems, impacting data quality and introducing study heterogeneity. Outcomes of interest included sepsis, apnea, bradycardia, mortality, necrotizing enterocolitis, and hypoxic-ischemic encephalopathy, with some studies analyzing combinations of adverse outcomes. We found a partial or complete lack of transparency in reporting the setting and the methods used for signal preprocessing. This includes reporting methods to handle missing data, segment size for considered analysis, and details regarding the modification of the state-of-the-art methods for physiological signal processing to align with the clinical principles for neonates. Only 7 (13%) of the 52 reviewed studies reported all the recommended preprocessing steps, which could have impacts on the downstream analysis. Conclusions: The review found heterogeneity in the techniques used and inconsistent reporting of parameters and procedures used for preprocessing neonatal physiological signals, which is necessary to confirm adherence to clinical and software quality management system practices, usefulness, and choice of best practices. Enhancing transparency in reporting and standardizing procedures will boost study interpretation and reproducibility and expedite clinical adoption, instilling confidence in the research findings and streamlining the translation of research outcomes into clinical practice, ultimately contributing to the advancement of neonatal care and patient outcomes. %M 39163610 %R 10.2196/46946 %U https://www.i-jmr.org/2024/1/e46946 %U https://doi.org/10.2196/46946 %U http://www.ncbi.nlm.nih.gov/pubmed/39163610 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e46455 %T Effective Privacy Protection Strategies for Pregnancy and Gestation Information From Electronic Medical Records: Retrospective Study in a National Health Care Data Network in China %A Liu,Chao %A Jiao,Yuanshi %A Su,Licong %A Liu,Wenna %A Zhang,Haiping %A Nie,Sheng %A Gong,Mengchun %+ School of Biomedical Engineering, Guangdong Medical University, No 2, Wenming East Road, Xiashan District, Zhanjiang, 524000, China, 86 18611768672, gmc@nrdrs.org %K pregnancy %K electronic medical record %K privacy protection %K risk stratification %K rule-based %D 2024 %7 20.8.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Pregnancy and gestation information is routinely recorded in electronic medical record (EMR) systems across China in various data sets. The combination of data on the number of pregnancies and gestations can imply occurrences of abortions and other pregnancy-related issues, which is important for clinical decision-making and personal privacy protection. However, the distribution of this information inside EMR is variable due to inconsistent IT structures across different EMR systems. A large-scale quantitative evaluation of the potential exposure of this sensitive information has not been previously performed, ensuring the protection of personal information is a priority, as emphasized in Chinese laws and regulations. Objective: This study aims to perform the first nationwide quantitative analysis of the identification sites and exposure frequency of sensitive pregnancy and gestation information. The goal is to propose strategies for effective information extraction and privacy protection related to women’s health. Methods: This study was conducted in a national health care data network. Rule-based protocols for extracting pregnancy and gestation information were developed by a committee of experts. A total of 6 different sub–data sets of EMRs were used as schemas for data analysis and strategy proposal. The identification sites and frequencies of identification in different sub–data sets were calculated. Manual quality inspections of the extraction process were performed by 2 independent groups of reviewers on 1000 randomly selected records. Based on these statistics, strategies for effective information extraction and privacy protection were proposed. Results: The data network covered hospitalized patients from 19 hospitals in 10 provinces of China, encompassing 15,245,055 patients over an 11-year period (January 1, 2010-December 12, 2020). Among women aged 14-50 years, 70% were randomly selected from each hospital, resulting in a total of 1,110,053 patients. Of these, 688,268 female patients with sensitive reproductive information were identified. The frequencies of identification were variable, with the marriage history in admission medical records being the most frequent at 63.24%. Notably, more than 50% of female patients were identified with pregnancy and gestation history in nursing records, which is not generally considered a sub–data set rich in reproductive information. During the manual curation and review process, 1000 cases were randomly selected, and the precision and recall rates of the information extraction method both exceeded 99.5%. The privacy-protection strategies were designed with clear technical directions. Conclusions: Significant amounts of critical information related to women’s health are recorded in Chinese routine EMR systems and are distributed in various parts of the records with different frequencies. This requires a comprehensive protocol for extracting and protecting the information, which has been demonstrated to be technically feasible. Implementing a data-based strategy will enhance the protection of women’s privacy and improve the accessibility of health care services. %M 39163593 %R 10.2196/46455 %U https://www.jmir.org/2024/1/e46455 %U https://doi.org/10.2196/46455 %U http://www.ncbi.nlm.nih.gov/pubmed/39163593 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e48320 %T The Use of Deep Learning and Machine Learning on Longitudinal Electronic Health Records for the Early Detection and Prevention of Diseases: Scoping Review %A Swinckels,Laura %A Bennis,Frank C %A Ziesemer,Kirsten A %A Scheerman,Janneke F M %A Bijwaard,Harmen %A de Keijzer,Ander %A Bruers,Josef Jan %+ Department of Oral Public Health, Academic Centre for Dentistry Amsterdam (ACTA), University of Amsterdam and Vrije Universiteit, Gustav Mahlerlaan 3004, Amsterdam, 1081 LA, Netherlands, 31 205980308, L.Swinckels@acta.nl %K artificial intelligence %K big data %K detection %K electronic health records %K machine learning %K personalized health care %K prediction %K prevention %D 2024 %7 20.8.2024 %9 Review %J J Med Internet Res %G English %X Background: Electronic health records (EHRs) contain patients’ health information over time, including possible early indicators of disease. However, the increasing amount of data hinders clinicians from using them. There is accumulating evidence suggesting that machine learning (ML) and deep learning (DL) can assist clinicians in analyzing these large-scale EHRs, as algorithms thrive on high volumes of data. Although ML has become well developed, studies mainly focus on engineering but lack medical outcomes. Objective: This study aims for a scoping review of the evidence on how the use of ML on longitudinal EHRs can support the early detection and prevention of disease. The medical insights and clinical benefits that have been generated were investigated by reviewing applications in a variety of diseases. Methods: This study was conducted according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. A literature search was performed in 2022 in collaboration with a medical information specialist in the following databases: PubMed, Embase, Web of Science Core Collection (Clarivate Analytics), and IEEE Xplore Digital Library and computer science bibliography. Studies were eligible when longitudinal EHRs were used that aimed for the early detection of disease via ML in a prevention context. Studies with a technical focus or using imaging or hospital admission data were beyond the scope of this review. Study screening and selection and data extraction were performed independently by 2 researchers. Results: In total, 20 studies were included, mainly published between 2018 and 2022. They showed that a variety of diseases could be detected or predicted, particularly diabetes; kidney diseases; diseases of the circulatory system; and mental, behavioral, and neurodevelopmental disorders. Demographics, symptoms, procedures, laboratory test results, diagnoses, medications, and BMI were frequently used EHR data in basic recurrent neural network or long short-term memory techniques. By developing and comparing ML and DL models, medical insights such as a high diagnostic performance, an earlier detection, the most important predictors, and additional health indicators were obtained. A clinical benefit that has been evaluated positively was preliminary screening. If these models are applied in practice, patients might also benefit from personalized health care and prevention, with practical benefits such as workload reduction and policy insights. Conclusions: Longitudinal EHRs proved to be helpful for support in health care. Current ML models on EHRs can support the detection of diseases in terms of accuracy and offer preliminary screening benefits. Regarding the prevention of diseases, ML and specifically DL models can accurately predict or detect diseases earlier than current clinical diagnoses. Adding personally responsible factors allows targeted prevention interventions. While ML models based on textual EHRs are still in the developmental stage, they have high potential to support clinicians and the health care system and improve patient outcomes. %M 39163096 %R 10.2196/48320 %U https://www.jmir.org/2024/1/e48320 %U https://doi.org/10.2196/48320 %U http://www.ncbi.nlm.nih.gov/pubmed/39163096 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e57153 %T Evaluating and Enhancing the Fitness-for-Purpose of Electronic Health Record Data: Qualitative Study on Current Practices and Pathway to an Automated Approach Within the Medical Informatics for Research and Care in University Medicine Consortium %A Kamdje Wabo,Gaetan %A Moorthy,Preetha %A Siegel,Fabian %A Seuchter,Susanne A %A Ganslandt,Thomas %+ Center for Preventive Medicine and Digital Health Baden-Wuerttemberg, Department of Biomedical Informatics, Medical Faculty of Mannheim, University of Heidelberg, Building 3, Level 4, Theodor-Kutzer-Ufer 1-3, Mannheim, 68167, Germany, 49 621 383 8088, gaetankamdje.wabo@medma.uni-heidelberg.de %K data quality %K fitness-for-purpose %K secondary use %K thematic analysis %K EHR data %K electronic health record %K data integration center %K Medical Informatics Initiative %K MIRACUM consortium %K Medical Informatics for Research and Care in University Medicine %K data science %K integration %K data use %K visualization %K visualizations %K record %K records %K EHR %K EHRs %K survey %K surveys %K medical informatics %D 2024 %7 19.8.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Leveraging electronic health record (EHR) data for clinical or research purposes heavily depends on data fitness. However, there is a lack of standardized frameworks to evaluate EHR data suitability, leading to inconsistent quality in data use projects (DUPs). This research focuses on the Medical Informatics for Research and Care in University Medicine (MIRACUM) Data Integration Centers (DICs) and examines empirical practices on assessing and automating the fitness-for-purpose of clinical data in German DIC settings. Objective: The study aims (1) to capture and discuss how MIRACUM DICs evaluate and enhance the fitness-for-purpose of observational health care data and examine the alignment with existing recommendations and (2) to identify the requirements for designing and implementing a computer-assisted solution to evaluate EHR data fitness within MIRACUM DICs. Methods: A qualitative approach was followed using an open-ended survey across DICs of 10 German university hospitals affiliated with MIRACUM. Data were analyzed using thematic analysis following an inductive qualitative method. Results: All 10 MIRACUM DICs participated, with 17 participants revealing various approaches to assessing data fitness, including the 4-eyes principle and data consistency checks such as cross-system data value comparison. Common practices included a DUP-related feedback loop on data fitness and using self-designed dashboards for monitoring. Most experts had a computer science background and a master’s degree, suggesting strong technological proficiency but potentially lacking clinical or statistical expertise. Nine key requirements for a computer-assisted solution were identified, including flexibility, understandability, extendibility, and practicability. Participants used heterogeneous data repositories for evaluating data quality criteria and practical strategies to communicate with research and clinical teams. Conclusions: The study identifies gaps between current practices in MIRACUM DICs and existing recommendations, offering insights into the complexities of assessing and reporting clinical data fitness. Additionally, a tripartite modular framework for fitness-for-purpose assessment was introduced to streamline the forthcoming implementation. It provides valuable input for developing and integrating an automated solution across multiple locations. This may include statistical comparisons to advanced machine learning algorithms for operationalizing frameworks such as the 3×3 data quality assessment framework. These findings provide foundational evidence for future design and implementation studies to enhance data quality assessments for specific DUPs in observational health care settings. %M 39158950 %R 10.2196/57153 %U https://medinform.jmir.org/2024/1/e57153 %U https://doi.org/10.2196/57153 %U http://www.ncbi.nlm.nih.gov/pubmed/39158950 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e48997 %T Five-Feature Models to Predict Preeclampsia Onset Time From Electronic Health Record Data: Development and Validation Study %A Ballard,Hailey K %A Yang,Xiaotong %A Mahadevan,Aditya D %A Lemas,Dominick J %A Garmire,Lana X %+ Department of Computational Medicine and Bioinformatics, University of Michigan Medical School, Room 3366, Building 520, NCRC, 1600 Huron Parkway, Ann Arbor, MI, 48105, United States, 1 734 615 0514, lgarmire@gmail.com %K preeclampsia %K survival analysis %K risk prediction %K pregnancy %K prognosis %K survival %K risk %K mortality %K EHR %K health records %K maternal %K machine learning %K electronic health records %D 2024 %7 14.8.2024 %9 Original Paper %J J Med Internet Res %G English %X Background:  Preeclampsia is a potentially fatal complication during pregnancy, characterized by high blood pressure and the presence of excessive proteins in the urine. Due to its complexity, the prediction of preeclampsia onset is often difficult and inaccurate. Objective:  This study aimed to create quantitative models to predict the onset gestational age of preeclampsia using electronic health records. Methods:  We retrospectively collected 1178 preeclamptic pregnancy records from the University of Michigan Health System as the discovery cohort, and 881 records from the University of Florida Health System as the validation cohort. We constructed 2 Cox-proportional hazards models: 1 baseline model using maternal and pregnancy characteristics, and the other full model with additional laboratory findings, vitals, and medications. We built the models using 80% of the discovery data, tested the remaining 20% of the discovery data, and validated with the University of Florida data. We further stratified the patients into high- and low-risk groups for preeclampsia onset risk assessment. Results:  The baseline model reached Concordance indices of 0.64 and 0.61 in the 20% testing data and the validation data, respectively, while the full model increased these Concordance indices to 0.69 and 0.61, respectively. For preeclampsia diagnosed at 34 weeks, the baseline and full models had area under the curve (AUC) values of 0.65 and 0.70, and AUC values of 0.69 and 0.70 for preeclampsia diagnosed at 37 weeks, respectively. Both models contain 5 selective features, among which the number of fetuses in the pregnancy, hypertension, and parity are shared between the 2 models with similar hazard ratios and significant P values. In the full model, maximum diastolic blood pressure in early pregnancy was the predominant feature. Conclusions:  Electronic health records data provide useful information to predict the gestational age of preeclampsia onset. Stratification of the cohorts using 5-predictor Cox-proportional hazards models provides clinicians with convenient tools to assess the onset time of preeclampsia in patients. %M 39141914 %R 10.2196/48997 %U https://www.jmir.org/2024/1/e48997 %U https://doi.org/10.2196/48997 %U http://www.ncbi.nlm.nih.gov/pubmed/39141914 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e49542 %T Transforming Primary Care Data Into the Observational Medical Outcomes Partnership Common Data Model: Development and Usability Study %A Fruchart,Mathilde %A Quindroit,Paul %A Jacquemont,Chloé %A Beuscart,Jean-Baptiste %A Calafiore,Matthieu %A Lamer,Antoine %K data reuse %K Observational Medical Outcomes Partnership %K common data model %K data warehouse %K reproducible research %K primary care %K dashboard %K electronic health record %K patient tracking system %K patient monitoring %K EHR %K primary care data %D 2024 %7 13.8.2024 %9 %J JMIR Med Inform %G English %X Background: Patient-monitoring software generates a large amount of data that can be reused for clinical audits and scientific research. The Observational Health Data Sciences and Informatics (OHDSI) consortium developed the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) to standardize electronic health record data and promote large-scale observational and longitudinal research. Objective: This study aimed to transform primary care data into the OMOP CDM format. Methods: We extracted primary care data from electronic health records at a multidisciplinary health center in Wattrelos, France. We performed structural mapping between the design of our local primary care database and the OMOP CDM tables and fields. Local French vocabularies concepts were mapped to OHDSI standard vocabularies. To validate the implementation of primary care data into the OMOP CDM format, we applied a set of queries. A practical application was achieved through the development of a dashboard. Results: Data from 18,395 patients were implemented into the OMOP CDM, corresponding to 592,226 consultations over a period of 20 years. A total of 18 OMOP CDM tables were implemented. A total of 17 local vocabularies were identified as being related to primary care and corresponded to patient characteristics (sex, location, year of birth, and race), units of measurement, biometric measures, laboratory test results, medical histories, and drug prescriptions. During semantic mapping, 10,221 primary care concepts were mapped to standard OHDSI concepts. Five queries were used to validate the OMOP CDM by comparing the results obtained after the completion of the transformations with the results obtained in the source software. Lastly, a prototype dashboard was developed to visualize the activity of the health center, the laboratory test results, and the drug prescription data. Conclusions: Primary care data from a French health care facility have been implemented into the OMOP CDM format. Data concerning demographics, units, measurements, and primary care consultation steps were already available in OHDSI vocabularies. Laboratory test results and drug prescription data were mapped to available vocabularies and structured in the final model. A dashboard application provided health care professionals with feedback on their practice. %R 10.2196/49542 %U https://medinform.jmir.org/2024/1/e49542 %U https://doi.org/10.2196/49542 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 10 %N %P e54967 %T Liver Cancer Mortality Disparities at a Fine Scale Among Subpopulations in China: Nationwide Analysis of Spatial and Temporal Trends %A Gan,Ting %A Liu,Yunning %A Bambrick,Hilary %A Zhou,Maigeng %A Hu,Wenbiao %K liver cancer %K mortality %K year of life lost %K spatial distribution %K temporal trend %D 2024 %7 8.8.2024 %9 %J JMIR Public Health Surveill %G English %X Background: China has the highest number of liver cancers worldwide, and liver cancer is at the forefront of all cancers in China. However, current research on liver cancer in China primarily relies on extrapolated data or relatively lagging data, with limited focus on subregions and specific population groups. Objective: The purpose of this study is to identify geographic disparities in liver cancer by exploring the spatial and temporal trends of liver cancer mortality and the years of life lost (YLL) caused by it within distinct geographical regions, climate zones, and population groups in China. Methods: Data from the National Death Surveillance System between 2013 and 2020 were used to calculate the age-standardized mortality rate of liver cancer (LASMR) and YLL from liver cancer in China. The spatial distribution and temporal trends of liver cancer were analyzed in subgroups by sex, age, region, and climate classification. Estimated annual percentage change was used to describe liver cancer trends in various regions, and partial correlation was applied to explore associations between LASMR and latitude. Results: In China, the average LASMR decreased from 28.79 in 2013 to 26.38 per 100,000 in 2020 among men and 11.09 to 9.83 per 100,000 among women. This decline in mortality was consistent across all age groups. Geographically, Guangxi had the highest LASMR for men in China, with a rate of 50.15 per 100,000, while for women, it was Heilongjiang, with a rate of 16.64 per 100,000. Within these regions, the LASMR among men in most parts of Guangxi ranged from 32.32 to 74.98 per 100,000, whereas the LASMR among women in the majority of Heilongjiang ranged from 13.72 to 21.86 per 100,000. The trend of LASMR varied among regions. For both men and women, Guizhou showed an increasing trend in LASMR from 2013 to 2020, with estimated annual percentage changes ranging from 10.05% to 29.07% and from 10.09% to 21.71%, respectively. Both men and women observed an increase in LASMR with increasing latitude below the 40th parallel. However, overall, LASMR in men was positively correlated with latitude (R=0.225; P<.001), while in women, it showed a negative correlation (R=0.083; P=.04). High LASMR areas among men aligned with subtropical zones, like Cwa and Cfa. The age group 65 years and older, the southern region, and the Cwa climate zone had the highest YLL rates at 4850.50, 495.50, and 440.17 per 100,000, respectively. However, the overall trends in these groups showed a decline over the period. Conclusions: Despite the declining overall trend of liver cancer in China, there are still marked disparities between regions and populations. Future prevention and control should focus on high-risk regions and populations to further reduce the burden of liver cancer in China. %R 10.2196/54967 %U https://publichealth.jmir.org/2024/1/e54967 %U https://doi.org/10.2196/54967 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e53369 %T Assessing Opportunities and Barriers to Improving the Secondary Use of Health Care Data at the National Level: Multicase Study in the Kingdom of Saudi Arabia and Estonia %A Metsallik,Janek %A Draheim,Dirk %A Sabic,Zlatan %A Novak,Thomas %A Ross,Peeter %+ E-Medicine Centre, Department of Health Technologies, School of Information Technologies, Tallinn University of Technology, Akadeemia 15a, Tallinn, 12616, Estonia, 372 56485978, janek.metsallik@taltech.ee %K health data governance %K secondary use %K health information sharing maturity %K large-scale interoperability %K health data stewardship %K health data custodianship %K health information purpose %K health data policy %D 2024 %7 8.8.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Digitization shall improve the secondary use of health care data. The Government of the Kingdom of Saudi Arabia ordered a project to compile the National Master Plan for Health Data Analytics, while the Government of Estonia ordered a project to compile the Person-Centered Integrated Hospital Master Plan. Objective: This study aims to map these 2 distinct projects’ problems, approaches, and outcomes to find the matching elements for reuse in similar cases. Methods: We assessed both health care systems’ abilities for secondary use of health data by exploratory case studies with purposive sampling and data collection via semistructured interviews and documentation review. The collected content was analyzed qualitatively and coded according to a predefined framework. The analytical framework consisted of data purpose, flow, and sharing. The Estonian project used the Health Information Sharing Maturity Model from the Mitre Corporation as an additional analytical framework. The data collection and analysis in the Kingdom of Saudi Arabia took place in 2019 and covered health care facilities, public health institutions, and health care policy. The project in Estonia collected its inputs in 2020 and covered health care facilities, patient engagement, public health institutions, health care financing, health care policy, and health technology innovations. Results: In both cases, the assessments resulted in a set of recommendations focusing on the governance of health care data. In the Kingdom of Saudi Arabia, the health care system consists of multiple isolated sectors, and there is a need for an overarching body coordinating data sets, indicators, and reports at the national level. The National Master Plan of Health Data Analytics proposed a set of organizational agreements for proper stewardship. Despite Estonia’s national Digital Health Platform, the requirements remain uncoordinated between various data consumers. We recommended reconfiguring the stewardship of the national health data to include multipurpose data use into the scope of interoperability standardization. Conclusions: Proper data governance is the key to improving the secondary use of health data at the national level. The data flows from data providers to data consumers shall be coordinated by overarching stewardship structures and supported by interoperable data custodians. %M 39116424 %R 10.2196/53369 %U https://www.jmir.org/2024/1/e53369 %U https://doi.org/10.2196/53369 %U http://www.ncbi.nlm.nih.gov/pubmed/39116424 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e53427 %T Pediatric Sedation Assessment and Management System (PSAMS) for Pediatric Sedation in China: Development and Implementation Report %A Zhu,Ziyu %A Liu,Lan %A Du,Min %A Ye,Mao %A Xu,Ximing %A Xu,Ying %K electronic data capture %K information systems %K pediatric sedation %K sedation management %K workflow optimization %D 2024 %7 7.8.2024 %9 %J JMIR Med Inform %G English %X Background: Recently, the growing demand for pediatric sedation services outside the operating room has imposed a heavy burden on pediatric centers in China. There is an urgent need to develop a novel system for improved sedation services. Objective: This study aimed to develop and implement a computerized system, the Pediatric Sedation Assessment and Management System (PSAMS), to streamline pediatric sedation services at a major children’s hospital in Southwest China. Methods: PSAMS was designed to reflect the actual workflow of pediatric sedation. It consists of 3 main components: server-hosted software; client applications on tablets and computers; and specialized devices like gun-type scanners, desktop label printers, and pulse oximeters. With the participation of a multidisciplinary team, PSAMS was developed and refined during its application in the sedation process. This study analyzed data from the first 2 years after the system’s deployment. Implementation (Results): From January 2020 to December 2021, a total of 127,325 sedations were performed on 85,281 patients using the PSAMS database. Besides basic variables imported from Hospital Information Systems (HIS), the PSAMS database currently contains 33 additional variables that capture comprehensive information from presedation assessment to postprocedural recovery. The recorded data from PSAMS indicates a one-time sedation success rate of 97.1% (50,752/52,282) in 2020 and 97.5% (73,184/75,043) in 2021. The observed adverse events rate was 3.5% (95% CI 3.4%‐3.7%) in 2020 and 2.8% (95% CI 2.7%-2.9%) in 2021. Conclusions: PSAMS streamlined the entire sedation workflow, reduced the burden of data collection, and laid a foundation for future cooperation of multiple pediatric health care centers. %R 10.2196/53427 %U https://medinform.jmir.org/2024/1/e53427 %U https://doi.org/10.2196/53427 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 10 %N %P e53371 %T Social Determinants of Health Phenotypes and Cardiometabolic Condition Prevalence Among Patients in a Large Academic Health System: Latent Class Analysis %A Howell,Carrie R %A Zhang,Li %A Clay,Olivio J %A Dutton,Gareth %A Horton,Trudi %A Mugavero,Michael J %A Cherrington,Andrea L %K social determinants of health %K electronic medical record %K phenotypes %K diabetes %K obesity %K cardiovascular disease %K obese %K social determinants %K social determinant %K cardiometabolic %K risk factors %K risk factor %K latent class analysis %K cardiometabolic disease %K EMR %K EHR %K electronic medical record %K electronic health record %D 2024 %7 7.8.2024 %9 %J JMIR Public Health Surveill %G English %X Background: Adverse social determinants of health (SDoH) have been associated with cardiometabolic disease; however, disparities in cardiometabolic outcomes are rarely the result of a single risk factor. Objective: This study aimed to identify and characterize SDoH phenotypes based on patient-reported and neighborhood-level data from the institutional electronic medical record and evaluate the prevalence of diabetes, obesity, and other cardiometabolic diseases by phenotype status. Methods: Patient-reported SDoH were collected (January to December 2020) and neighborhood-level social vulnerability, neighborhood socioeconomic status, and rurality were linked via census tract to geocoded patient addresses. Diabetes status was coded in the electronic medical record using International Classification of Diseases codes; obesity was defined using measured BMI ≥30 kg/m2. Latent class analysis was used to identify clusters of SDoH (eg, phenotypes); we then examined differences in the prevalence of cardiometabolic conditions based on phenotype status using prevalence ratios (PRs). Results: Complete data were available for analysis for 2380 patients (mean age 53, SD 16 years; n=1405, 59% female; n=1198, 50% non-White). Roughly 8% (n=179) reported housing insecurity, 30% (n=710) reported resource needs (food, health care, or utilities), and 49% (n=1158) lived in a high-vulnerability census tract. We identified 3 patient SDoH phenotypes: (1) high social risk, defined largely by self-reported SDoH (n=217, 9%); (2) adverse neighborhood SDoH (n=1353, 56%), defined largely by adverse neighborhood-level measures; and (3) low social risk (n=810, 34%), defined as low individual- and neighborhood-level risks. Patients with an adverse neighborhood SDoH phenotype had higher prevalence of diagnosed type 2 diabetes (PR 1.19, 95% CI 1.06‐1.33), hypertension (PR 1.14, 95% CI 1.02‐1.27), peripheral vascular disease (PR 1.46, 95% CI 1.09‐1.97), and heart failure (PR 1.46, 95% CI 1.20‐1.79). Conclusions: Patients with the adverse neighborhood SDoH phenotype had higher prevalence of poor cardiometabolic conditions compared to phenotypes determined by individual-level characteristics, suggesting that neighborhood environment plays a role, even if individual measures of socioeconomic status are not suboptimal. %R 10.2196/53371 %U https://publichealth.jmir.org/2024/1/e53371 %U https://doi.org/10.2196/53371 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e46823 %T Using Discrete-Event Simulation to Model Web-Based Crisis Counseling Service Operation: Evaluation Study %A Chiang,Byron %A Law,Yik Wa %A Yip,Paul Siu Fai %+ Centre of Suicide Research and Prevention, University of Hong Kong, 2/F, The Hong Kong Jockey Club Building for Interdisciplinary Research, 5 Sassoon Rd, Pokfulam, Hong Kong, China (Hong Kong), 852 2831 5232, sfpyip@hku.hk %K discrete-event simulation %K community operational research %K queuing %K web-based counseling %K service management %K repeat users %D 2024 %7 7.8.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: According to the Organisation for Economic Co-operation and Development, its member states experienced worsening mental health during the COVID-19 pandemic, leading to an increase of 60% to 1000% in digital counseling access. Hong Kong, too, witnessed a surge in demand for crisis intervention services during the pandemic, attracting both nonrepeat and repeat service users during the process. As a result of the continuing demand, platforms offering short-term emotional support are facing an efficiency challenge in managing caller responses. Objective: This aim of this paper was to assess the queuing performance of a 24-hour text-based web-based crisis counseling platform using a Python-based discrete-event simulation (DES) model. The model evaluates the staff combinations needed to meet demand and informs service priority decisions. It is able to account for unbalanced and overlapping shifts, unequal simultaneous serving capacities among custom worker types, time-dependent user arrivals, and the influence of user type (nonrepeat users vs repeat users) and suicide risk on service durations. Methods: Use and queue statistics by user type and staffing conditions were tabulated from past counseling platform database records. After calculating the data distributions, key parameters were incorporated into the DES model to determine the supply-demand equilibrium and identify potential service bottlenecks. An unobserved-components time-series model was fitted to make 30-day forecasts of the arrival rate, with the results piped back to the DES model to estimate the number of workers needed to staff each work shift, as well as the number of repeat service users encountered during a service operation. Results: The results showed a marked increase (from 3401/9202, 36.96% to 5042/9199, 54.81%) in the overall conversion rate after the strategic deployment of human resources according to the values set in the simulations, with an 85% chance of queuing users receiving counseling service within 10 minutes and releasing an extra 39.57% (3631/9175) capacity to serve nonrepeat users at potential risk. Conclusions: By exploiting scientifically informed data models with DES, nonprofit web-based counseling platforms, even those with limited resources, can optimize service capacity strategically to manage service bottlenecks and increase service uptake. %M 39110974 %R 10.2196/46823 %U https://formative.jmir.org/2024/1/e46823 %U https://doi.org/10.2196/46823 %U http://www.ncbi.nlm.nih.gov/pubmed/39110974 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e46407 %T Evaluating Artificial Intelligence in Clinical Settings—Let Us Not Reinvent the Wheel %A Cresswell,Kathrin %A de Keizer,Nicolette %A Magrabi,Farah %A Williams,Robin %A Rigby,Michael %A Prgomet,Mirela %A Kukhareva,Polina %A Wong,Zoie Shui-Yee %A Scott,Philip %A Craven,Catherine K %A Georgiou,Andrew %A Medlock,Stephanie %A Brender McNair,Jytte %A Ammenwerth,Elske %+ Usher Institute, The University of Edinburgh, Usher Building, 5-7 Little France Road, Edinburgh, EH16 4UX, United Kingdom, 44 131 650 6984, kathrin.cresswell@ed.ac.uk %K artificial intelligence %K evaluation %K theory %K patient safety %K optimisation %K health care %K optimization %D 2024 %7 7.8.2024 %9 Viewpoint %J J Med Internet Res %G English %X Given the requirement to minimize the risks and maximize the benefits of technology applications in health care provision, there is an urgent need to incorporate theory-informed health IT (HIT) evaluation frameworks into existing and emerging guidelines for the evaluation of artificial intelligence (AI). Such frameworks can help developers, implementers, and strategic decision makers to build on experience and the existing empirical evidence base. We provide a pragmatic conceptual overview of selected concrete examples of how existing theory-informed HIT evaluation frameworks may be used to inform the safe development and implementation of AI in health care settings. The list is not exhaustive and is intended to illustrate applications in line with various stakeholder requirements. Existing HIT evaluation frameworks can help to inform AI-based development and implementation by supporting developers and strategic decision makers in considering relevant technology, user, and organizational dimensions. This can facilitate the design of technologies, their implementation in user and organizational settings, and the sustainability and scalability of technologies. %M 39110494 %R 10.2196/46407 %U https://www.jmir.org/2024/1/e46407 %U https://doi.org/10.2196/46407 %U http://www.ncbi.nlm.nih.gov/pubmed/39110494 %0 Journal Article %@ 2371-4379 %I JMIR Publications %V 9 %N %P e53338 %T A Machine Learning Model for Risk Stratification of Postdiagnosis Diabetic Ketoacidosis Hospitalization in Pediatric Type 1 Diabetes: Retrospective Study %A Subramanian,Devika %A Sonabend,Rona %A Singh,Ila %+ Department of Pathology and Immunology, Baylor College of Medicine, One Baylor Plaza, Houston, TX, 77030, United States, 1 832 824 7449, irsingh@texaschildrens.org %K pediatric type 1 diabetes %K postdiagnosis diabetic ketoacidosis %K risk prediction and stratification %K XGBoost %K Shapley values %K ketoacidosis %K risks %K predict %K prediction %K predictive %K gradient-boosted ensemble model %K diabetes %K pediatrics %K children %K machine learning %D 2024 %7 7.8.2024 %9 Original Paper %J JMIR Diabetes %G English %X Background: Diabetic ketoacidosis (DKA) is the leading cause of morbidity and mortality in pediatric type 1 diabetes (T1D), occurring in approximately 20% of patients, with an economic cost of $5.1 billion/year in the United States. Despite multiple risk factors for postdiagnosis DKA, there is still a need for explainable, clinic-ready models that accurately predict DKA hospitalization in established patients with pediatric T1D. Objective: We aimed to develop an interpretable machine learning model to predict the risk of postdiagnosis DKA hospitalization in children with T1D using routinely collected time-series of electronic health record (EHR) data. Methods: We conducted a retrospective case-control study using EHR data from 1787 patients from among 3794 patients with T1D treated at a large tertiary care US pediatric health system from January 2010 to June 2018. We trained a state-of-the-art; explainable, gradient-boosted ensemble (XGBoost) of decision trees with 44 regularly collected EHR features to predict postdiagnosis DKA. We measured the model’s predictive performance using the area under the receiver operating characteristic curve–weighted F1-score, weighted precision, and recall, in a 5-fold cross-validation setting. We analyzed Shapley values to interpret the learned model and gain insight into its predictions. Results: Our model distinguished the cohort that develops DKA postdiagnosis from the one that does not (P<.001). It predicted postdiagnosis DKA risk with an area under the receiver operating characteristic curve of 0.80 (SD 0.04), a weighted F1-score of 0.78 (SD 0.04), and a weighted precision and recall of 0.83 (SD 0.03) and 0.76 (SD 0.05) respectively, using a relatively short history of data from routine clinic follow-ups post diagnosis. On analyzing Shapley values of the model output, we identified key risk factors predicting postdiagnosis DKA both at the cohort and individual levels. We observed sharp changes in postdiagnosis DKA risk with respect to 2 key features (diabetes age and glycated hemoglobin at 12 months), yielding time intervals and glycated hemoglobin cutoffs for potential intervention. By clustering model-generated Shapley values, we automatically stratified the cohort into 3 groups with 5%, 20%, and 48% risk of postdiagnosis DKA. Conclusions: We have built an explainable, predictive, machine learning model with potential for integration into clinical workflow. The model risk-stratifies patients with pediatric T1D and identifies patients with the highest postdiagnosis DKA risk using limited follow-up data starting from the time of diagnosis. The model identifies key time points and risk factors to direct clinical interventions at both the individual and cohort levels. Further research with data from multiple hospital systems can help us assess how well our model generalizes to other populations. The clinical importance of our work is that the model can predict patients most at risk for postdiagnosis DKA and identify preventive interventions based on mitigation of individualized risk factors. %M 39110490 %R 10.2196/53338 %U https://diabetes.jmir.org/2024/1/e53338 %U https://doi.org/10.2196/53338 %U http://www.ncbi.nlm.nih.gov/pubmed/39110490 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e56316 %T Digital Maturity as a Predictor of Quality and Safety Outcomes in US Hospitals: Cross-Sectional Observational Study %A Snowdon,Anne %A Hussein,Abdulkadir %A Danforth,Melissa %A Wright,Alexandra %A Oakes,Reid %+ Department of Mathematics & Statistics, University of Windsor, Windsor, ON, Canada, 1 4164001956, alexandra.wright@uwindsor.ca %K digital health %K readiness %K cross sectional %K observational %K regression %K digital maturity %K association %K associations %K correlation %K correlations %K quality and safety %K hospital performance %K workforce %K health outcomes %K safety %K service %K services %K healthcare system %K healthcare systems %K hospital %K hospitals %D 2024 %7 6.8.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: This study demonstrates that digital maturity contributes to strengthened quality and safety performance outcomes in US hospitals. Advanced digital maturity is associated with more digitally enabled work environments with automated flow of data across information systems to enable clinicians and leaders to track quality and safety outcomes. This research illustrates that an advanced digitally enabled workforce is associated with strong safety leadership and culture and better patient health and safety outcomes. Objective: This study aimed to examine the relationship between digital maturity and quality and safety outcomes in US hospitals. Methods: The data sources were hospital safety letter grades as well as quality and safety scores on a continuous scale published by The Leapfrog Group. We used the digital maturity level (measured using the Electronic Medical Record Assessment Model [EMRAM]) of 1026 US hospitals. This was a cross-sectional, observational study. Logistic, linear, and Tweedie regression analyses were used to explore the relationships among The Leapfrog Group's Hospital Safety Grades, individual Leapfrog safety scores, and digital maturity levels classified as advanced or fully developed digital maturity (EMRAM levels 6 and 7) or underdeveloped maturity (EMRAM level 0). Digital maturity was a predictor while controlling for hospital characteristics including teaching status, urban or rural location, hospital size measured by number of beds, whether the hospital was a referral center, and type of hospital ownership as confounding variables. Hospitals were divided into the following 2 groups to compare safety and quality outcomes: hospitals that were digitally advanced and hospitals with underdeveloped digital maturity. Data from The Leapfrog Group's Hospital Safety Grades report published in spring 2019 were matched to the hospitals with completed EMRAM assessments in 2019. Hospital characteristics such as number of hospital beds were obtained from the CMS database. Results: The results revealed that the odds of achieving a higher Leapfrog Group Hospital Safety Grade was statistically significantly higher, by 3.25 times, for hospitals with advanced digital maturity (EMRAM maturity of 6 or 7; odds ratio 3.25, 95% CI 2.33-4.55). Conclusions: Hospitals with advanced digital maturity had statistically significantly reduced infection rates, reduced adverse events, and improved surgical safety outcomes. The study findings suggest a significant difference in quality and safety outcomes among hospitals with advanced digital maturity compared with hospitals with underdeveloped digital maturity. %M 39106100 %R 10.2196/56316 %U https://www.jmir.org/2024/1/e56316 %U https://doi.org/10.2196/56316 %U http://www.ncbi.nlm.nih.gov/pubmed/39106100 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e57224 %T Predictors of Health Care Practitioners’ Intention to Use AI-Enabled Clinical Decision Support Systems: Meta-Analysis Based on the Unified Theory of Acceptance and Use of Technology %A Dingel,Julius %A Kleine,Anne-Kathrin %A Cecil,Julia %A Sigl,Anna Leonie %A Lermer,Eva %A Gaube,Susanne %+ Human-AI-Interaction Group, Center for Leadership and People Management, Ludwig Maximilian University of Munich, Geschwister-Scholl-Platz 1, Munich, 80539, Germany, 49 8921809775, anne-kathrin.kleine@psy.lmu.de %K Unified Theory of Acceptance and Use of Technology %K UTAUT %K artificial intelligence–enabled clinical decision support systems %K AI-CDSSs %K meta-analysis %K health care practitioners %D 2024 %7 5.8.2024 %9 Review %J J Med Internet Res %G English %X Background: Artificial intelligence–enabled clinical decision support systems (AI-CDSSs) offer potential for improving health care outcomes, but their adoption among health care practitioners remains limited. Objective: This meta-analysis identified predictors influencing health care practitioners’ intention to use AI-CDSSs based on the Unified Theory of Acceptance and Use of Technology (UTAUT). Additional predictors were examined based on existing empirical evidence. Methods: The literature search using electronic databases, forward searches, conference programs, and personal correspondence yielded 7731 results, of which 17 (0.22%) studies met the inclusion criteria. Random-effects meta-analysis, relative weight analyses, and meta-analytic moderation and mediation analyses were used to examine the relationships between relevant predictor variables and the intention to use AI-CDSSs. Results: The meta-analysis results supported the application of the UTAUT to the context of the intention to use AI-CDSSs. The results showed that performance expectancy (r=0.66), effort expectancy (r=0.55), social influence (r=0.66), and facilitating conditions (r=0.66) were positively associated with the intention to use AI-CDSSs, in line with the predictions of the UTAUT. The meta-analysis further identified positive attitude (r=0.63), trust (r=0.73), anxiety (r=–0.41), perceived risk (r=–0.21), and innovativeness (r=0.54) as additional relevant predictors. Trust emerged as the most influential predictor overall. The results of the moderation analyses show that the relationship between social influence and use intention becomes weaker with increasing age. In addition, the relationship between effort expectancy and use intention was stronger for diagnostic AI-CDSSs than for devices that combined diagnostic and treatment recommendations. Finally, the relationship between facilitating conditions and use intention was mediated through performance and effort expectancy. Conclusions: This meta-analysis contributes to the understanding of the predictors of intention to use AI-CDSSs based on an extended UTAUT model. More research is needed to substantiate the identified relationships and explain the observed variations in effect sizes by identifying relevant moderating factors. The research findings bear important implications for the design and implementation of training programs for health care practitioners to ease the adoption of AI-CDSSs into their practice. %M 39102675 %R 10.2196/57224 %U https://www.jmir.org/2024/1/e57224 %U https://doi.org/10.2196/57224 %U http://www.ncbi.nlm.nih.gov/pubmed/39102675 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e56627 %T Advancing Accuracy in Multimodal Medical Tasks Through Bootstrapped Language-Image Pretraining (BioMedBLIP): Performance Evaluation Study %A Naseem,Usman %A Thapa,Surendrabikram %A Masood,Anum %+ Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, B4-135, Realfagbygget Building., Gloshaugen Campus, Trondheim, 7491, Norway, 47 92093743, anum.msd@gmail.com %K biomedical text mining %K BioNLP %K vision-language pretraining %K multimodal models %K medical image analysis %D 2024 %7 5.8.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Medical image analysis, particularly in the context of visual question answering (VQA) and image captioning, is crucial for accurate diagnosis and educational purposes. Objective: Our study aims to introduce BioMedBLIP models, fine-tuned for VQA tasks using specialized medical data sets such as Radiology Objects in Context and Medical Information Mart for Intensive Care-Chest X-ray, and evaluate their performance in comparison to the state of the art (SOTA) original Bootstrapping Language-Image Pretraining (BLIP) model. Methods: We present 9 versions of BioMedBLIP across 3 downstream tasks in various data sets. The models are trained on a varying number of epochs. The findings indicate the strong overall performance of our models. We proposed BioMedBLIP for the VQA generation model, VQA classification model, and BioMedBLIP image caption model. We conducted pretraining in BLIP using medical data sets, producing an adapted BLIP model tailored for medical applications. Results: In VQA generation tasks, BioMedBLIP models outperformed the SOTA on the Semantically-Labeled Knowledge-Enhanced (SLAKE) data set, VQA in Radiology (VQA-RAD), and Image Cross-Language Evaluation Forum data sets. In VQA classification, our models consistently surpassed the SOTA on the SLAKE data set. Our models also showed competitive performance on the VQA-RAD and PathVQA data sets. Similarly, in image captioning tasks, our model beat the SOTA, suggesting the importance of pretraining with medical data sets. Overall, in 20 different data sets and task combinations, our BioMedBLIP excelled in 15 (75%) out of 20 tasks. BioMedBLIP represents a new SOTA in 15 (75%) out of 20 tasks, and our responses were rated higher in all 20 tasks (P<.005) in comparison to SOTA models. Conclusions: Our BioMedBLIP models show promising performance and suggest that incorporating medical knowledge through pretraining with domain-specific medical data sets helps models achieve higher performance. Our models thus demonstrate their potential to advance medical image analysis, impacting diagnosis, medical education, and research. However, data quality, task-specific variability, computational resources, and ethical considerations should be carefully addressed. In conclusion, our models represent a contribution toward the synergy of artificial intelligence and medicine. We have made BioMedBLIP freely available, which will help in further advancing research in multimodal medical tasks. %M 39102281 %R 10.2196/56627 %U https://medinform.jmir.org/2024/1/e56627 %U https://doi.org/10.2196/56627 %U http://www.ncbi.nlm.nih.gov/pubmed/39102281 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e60336 %T Patient-Representing Population's Perceptions of GPT-Generated Versus Standard Emergency Department Discharge Instructions: Randomized Blind Survey Assessment %A Huang,Thomas %A Safranek,Conrad %A Socrates,Vimig %A Chartash,David %A Wright,Donald %A Dilip,Monisha %A Sangal,Rohit B %A Taylor,Richard Andrew %+ Department of Emergency Medicine, Yale School of Medicine, 333 Cedar Street, New Haven, CT, 06510, United States, 1 2034324771, richard.taylor@yale.edu %K machine learning %K artificial intelligence %K large language models %K natural language processing %K ChatGPT %K discharge instructions %K emergency medicine %K emergency department %K discharge instructions %K surveys and questionaries %D 2024 %7 2.8.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Discharge instructions are a key form of documentation and patient communication in the time of transition from the emergency department (ED) to home. Discharge instructions are time-consuming and often underprioritized, especially in the ED, leading to discharge delays and possibly impersonal patient instructions. Generative artificial intelligence and large language models (LLMs) offer promising methods of creating high-quality and personalized discharge instructions; however, there exists a gap in understanding patient perspectives of LLM-generated discharge instructions. Objective: We aimed to assess the use of LLMs such as ChatGPT in synthesizing accurate and patient-accessible discharge instructions in the ED. Methods: We synthesized 5 unique, fictional ED encounters to emulate real ED encounters that included a diverse set of clinician history, physical notes, and nursing notes. These were passed to GPT-4 in Azure OpenAI Service (Microsoft) to generate LLM-generated discharge instructions. Standard discharge instructions were also generated for each of the 5 unique ED encounters. All GPT-generated and standard discharge instructions were then formatted into standardized after-visit summary documents. These after-visit summaries containing either GPT-generated or standard discharge instructions were randomly and blindly administered to Amazon MTurk respondents representing patient populations through Amazon MTurk Survey Distribution. Discharge instructions were assessed based on metrics of interpretability of significance, understandability, and satisfaction. Results: Our findings revealed that survey respondents’ perspectives regarding GPT-generated and standard discharge instructions were significantly (P=.01) more favorable toward GPT-generated return precautions, and all other sections were considered noninferior to standard discharge instructions. Of the 156 survey respondents, GPT-generated discharge instructions were assigned favorable ratings, “agree” and “strongly agree,” more frequently along the metric of interpretability of significance in discharge instruction subsections regarding diagnosis, procedures, treatment, post-ED medications or any changes to medications, and return precautions. Survey respondents found GPT-generated instructions to be more understandable when rating procedures, treatment, post-ED medications or medication changes, post-ED follow-up, and return precautions. Satisfaction with GPT-generated discharge instruction subsections was the most favorable in procedures, treatment, post-ED medications or medication changes, and return precautions. Wilcoxon rank-sum test of Likert responses revealed significant differences (P=.01) in the interpretability of significant return precautions in GPT-generated discharge instructions compared to standard discharge instructions but not for other evaluation metrics and discharge instruction subsections. Conclusions: This study demonstrates the potential for LLMs such as ChatGPT to act as a method of augmenting current documentation workflows in the ED to reduce the documentation burden of physicians. The ability of LLMs to provide tailored instructions for patients by improving readability and making instructions more applicable to patients could improve upon the methods of communication that currently exist. %M 39094112 %R 10.2196/60336 %U https://www.jmir.org/2024/1/e60336 %U https://doi.org/10.2196/60336 %U http://www.ncbi.nlm.nih.gov/pubmed/39094112 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e49655 %T Implementing AI in Hospitals to Achieve a Learning Health System: Systematic Review of Current Enablers and Barriers %A Kamel Rahimi,Amir %A Pienaar,Oliver %A Ghadimi,Moji %A Canfell,Oliver J %A Pole,Jason D %A Shrapnel,Sally %A van der Vegt,Anton H %A Sullivan,Clair %+ Queensland Digital Health Centre, Faculty of Medicine, The University of Queensland, Health Sciences Building, Herston Campus, Brisbane, QLD4006, Australia, 61 0733465350, amir.kamel@uq.edu.au %K life cycle %K medical informatics %K decision support system %K clinical %K electronic health records %K artificial intelligence %K machine learning %K routinely collected health data %D 2024 %7 2.8.2024 %9 Review %J J Med Internet Res %G English %X Background: Efforts are underway to capitalize on the computational power of the data collected in electronic medical records (EMRs) to achieve a learning health system (LHS). Artificial intelligence (AI) in health care has promised to improve clinical outcomes, and many researchers are developing AI algorithms on retrospective data sets. Integrating these algorithms with real-time EMR data is rare. There is a poor understanding of the current enablers and barriers to empower this shift from data set–based use to real-time implementation of AI in health systems. Exploring these factors holds promise for uncovering actionable insights toward the successful integration of AI into clinical workflows. Objective: The first objective was to conduct a systematic literature review to identify the evidence of enablers and barriers regarding the real-world implementation of AI in hospital settings. The second objective was to map the identified enablers and barriers to a 3-horizon framework to enable the successful digital health transformation of hospitals to achieve an LHS. Methods: The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were adhered to. PubMed, Scopus, Web of Science, and IEEE Xplore were searched for studies published between January 2010 and January 2022. Articles with case studies and guidelines on the implementation of AI analytics in hospital settings using EMR data were included. We excluded studies conducted in primary and community care settings. Quality assessment of the identified papers was conducted using the Mixed Methods Appraisal Tool and ADAPTE frameworks. We coded evidence from the included studies that related to enablers of and barriers to AI implementation. The findings were mapped to the 3-horizon framework to provide a road map for hospitals to integrate AI analytics. Results: Of the 1247 studies screened, 26 (2.09%) met the inclusion criteria. In total, 65% (17/26) of the studies implemented AI analytics for enhancing the care of hospitalized patients, whereas the remaining 35% (9/26) provided implementation guidelines. Of the final 26 papers, the quality of 21 (81%) was assessed as poor. A total of 28 enablers was identified; 8 (29%) were new in this study. A total of 18 barriers was identified; 5 (28%) were newly found. Most of these newly identified factors were related to information and technology. Actionable recommendations for the implementation of AI toward achieving an LHS were provided by mapping the findings to a 3-horizon framework. Conclusions: Significant issues exist in implementing AI in health care. Shifting from validating data sets to working with live data is challenging. This review incorporated the identified enablers and barriers into a 3-horizon framework, offering actionable recommendations for implementing AI analytics to achieve an LHS. The findings of this study can assist hospitals in steering their strategic planning toward successful adoption of AI. %M 39094106 %R 10.2196/49655 %U https://www.jmir.org/2024/1/e49655 %U https://doi.org/10.2196/49655 %U http://www.ncbi.nlm.nih.gov/pubmed/39094106 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e55090 %T Research on Traditional Chinese Medicine: Domain Knowledge Graph Completion and Quality Evaluation %A Liu,Chang %A Li,Zhan %A Li,Jianmin %A Qu,Yiqian %A Chang,Ying %A Han,Qing %A Cao,Lingyong %A Lin,Shuyuan %+ School of Basic Medical Sciences, Zhejiang Chinese Medical University, 548 Binwen Road, Binjiang District, Hangzhou, 310053, China, 86 057186633015, lin_shuyuan@foxmail.com %K graph completion %K traditional Chinese medicine %K graph quality evaluation %K graph representation %K knowledge graph %D 2024 %7 2.8.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Knowledge graphs (KGs) can integrate domain knowledge into a traditional Chinese medicine (TCM) intelligent syndrome differentiation model. However, the quality of current KGs in the TCM domain varies greatly, related to the lack of knowledge graph completion (KGC) and evaluation methods. Objective: This study aims to investigate KGC and evaluation methods tailored for TCM domain knowledge. Methods: In the KGC phase, according to the characteristics of TCM domain knowledge, we proposed a 3-step “entity-ontology-path” completion approach. This approach uses path reasoning, ontology rule reasoning, and association rules. In the KGC quality evaluation phase, we proposed a 3-dimensional evaluation framework that encompasses completeness, accuracy, and usability, using quantitative metrics such as complex network analysis, ontology reasoning, and graph representation. Furthermore, we compared the impact of different graph representation models on KG usability. Results: In the KGC phase, 52, 107, 27, and 479 triples were added by outlier analysis, rule-based reasoning, association rules, and path-based reasoning, respectively. In addition, rule-based reasoning identified 14 contradictory triples. In the KGC quality evaluation phase, in terms of completeness, KG had higher density and lower sparsity after completion, and there were no contradictory rules within the KG. In terms of accuracy, KG after completion was more consistent with prior knowledge. In terms of usability, the mean reciprocal ranking, mean rank, and hit rate of the first N tail entities predicted by the model (Hits@N) of the TransE, RotatE, DistMult, and ComplEx graph representation models all showed improvement after KGC. Among them, the RotatE model achieved the best representation. Conclusions: The 3-step completion approach can effectively improve the completeness, accuracy, and availability of KGs, and the 3-dimensional evaluation framework can be used for comprehensive KGC evaluation. In the TCM field, the RotatE model performed better at KG representation. %M 39094109 %R 10.2196/55090 %U https://medinform.jmir.org/2024/1/e55090 %U https://doi.org/10.2196/55090 %U http://www.ncbi.nlm.nih.gov/pubmed/39094109 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e54009 %T An App for Navigating Patient Transportation and Acute Stroke Care in Northwestern Ontario Using Machine Learning: Retrospective Study %A Hassan,Ayman %A Benlamri,Rachid %A Diner,Trina %A Cristofaro,Keli %A Dillistone,Lucas %A Khallouki,Hajar %A Ahghari,Mahvareh %A Littlefield,Shalyn %A Siddiqui,Rabail %A MacDonald,Russell %A Savage,David W %+ Thunder Bay Regional Health Research Institute, 980 Oliver Road, Thunder Bay, ON, P7B 6Z6, Canada, 1 8076847580, rabail.siddiqui@tbh.net %K stroke care %K acute stroke %K northwestern %K Ontario %K prediction %K models %K machine learning %K stroke %K cardiovascular %K brain %K neuroscience %K TIA %K transient ischemic attack %K coordinated care %K navigation %K navigating %K mHealth %K mobile health %K app %K apps %K applications %K geomapping %K geography %K geographical %K location %K spatial %K predict %K predictions %K predictive %D 2024 %7 1.8.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: A coordinated care system helps provide timely access to treatment for suspected acute stroke. In Northwestern Ontario (NWO), Canada, communities are widespread with several hospitals offering various diagnostic equipment and services. Thus, resources are limited, and health care providers must often transfer patients with stroke to different hospital locations to ensure the most appropriate care access within recommended time frames. However, health care providers frequently situated temporarily (locum) in NWO or providing care remotely from other areas of Ontario may lack sufficient information and experience in the region to access care for a patient with a time-sensitive condition. Suboptimal decision-making may lead to multiple transfers before definitive stroke care is obtained, resulting in poor outcomes and additional health care system costs. Objective: We aimed to develop a tool to inform and assist NWO health care providers in determining the best transfer options for patients with stroke to provide the most efficient care access. We aimed to develop an app using a comprehensive geomapping navigation and estimation system based on machine learning algorithms. This app uses key stroke-related timelines including the last time the patient was known to be well, patient location, treatment options, and imaging availability at different health care facilities. Methods: Using historical data (2008-2020), an accurate prediction model using machine learning methods was developed and incorporated into a mobile app. These data contained parameters regarding air (Ornge) and land medical transport (3 services), which were preprocessed and cleaned. For cases in which Ornge air services and land ambulance medical transport were both involved in a patient transport process, data were merged and time intervals of the transport journey were determined. The data were distributed for training (35%), testing (35%), and validation (30%) of the prediction model. Results: In total, 70,623 records were collected in the data set from Ornge and land medical transport services to develop a prediction model. Various learning models were analyzed; all learning models perform better than the simple average of all points in predicting output variables. The decision tree model provided more accurate results than the other models. The decision tree model performed remarkably well, with the values from testing, validation, and the model within a close range. This model was used to develop the “NWO Navigate Stroke” system. The system provides accurate results and demonstrates that a mobile app can be a significant tool for health care providers navigating stroke care in NWO, potentially impacting patient care and outcomes. Conclusions: The NWO Navigate Stroke system uses a data-driven, reliable, accurate prediction model while considering all variations and is simultaneously linked to all required acute stroke management pathways and tools. It was tested using historical data, and the next step will to involve usability testing with end users. %R 10.2196/54009 %U https://formative.jmir.org/2024/1/e54009 %U https://doi.org/10.2196/54009 %0 Journal Article %@ 1947-2579 %I JMIR Publications %V 16 %N %P e56237 %T Making Metadata Machine-Readable as the First Step to Providing Findable, Accessible, Interoperable, and Reusable Population Health Data: Framework Development and Implementation Study %A Amadi,David %A Kiwuwa-Muyingo,Sylvia %A Bhattacharjee,Tathagata %A Taylor,Amelia %A Kiragga,Agnes %A Ochola,Michael %A Kanjala,Chifundo %A Gregory,Arofan %A Tomlin,Keith %A Todd,Jim %A Greenfield,Jay %+ Department of Population Health, Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, Keppel St, London, WC1E 7HT, United Kingdom, 44 20 7636 8636, david.amadi@lshtm.ac.uk %K FAIR data principles %K metadata %K machine-readable metadata %K DDI %K Data Documentation Initiative %K standardization %K JSON-LD %K JavaScript Object Notation for Linked Data %K OMOP CDM %K Observational Medical Outcomes Partnership Common Data Model %K data science %K data models %D 2024 %7 1.8.2024 %9 Original Paper %J Online J Public Health Inform %G English %X Background: Metadata describe and provide context for other data, playing a pivotal role in enabling findability, accessibility, interoperability, and reusability (FAIR) data principles. By providing comprehensive and machine-readable descriptions of digital resources, metadata empower both machines and human users to seamlessly discover, access, integrate, and reuse data or content across diverse platforms and applications. However, the limited accessibility and machine-interpretability of existing metadata for population health data hinder effective data discovery and reuse. Objective: To address these challenges, we propose a comprehensive framework using standardized formats, vocabularies, and protocols to render population health data machine-readable, significantly enhancing their FAIRness and enabling seamless discovery, access, and integration across diverse platforms and research applications. Methods: The framework implements a 3-stage approach. The first stage is Data Documentation Initiative (DDI) integration, which involves leveraging the DDI Codebook metadata and documentation of detailed information for data and associated assets, while ensuring transparency and comprehensiveness. The second stage is Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) standardization. In this stage, the data are harmonized and standardized into the OMOP CDM, facilitating unified analysis across heterogeneous data sets. The third stage involves the integration of Schema.org and JavaScript Object Notation for Linked Data (JSON-LD), in which machine-readable metadata are generated using Schema.org entities and embedded within the data using JSON-LD, boosting discoverability and comprehension for both machines and human users. We demonstrated the implementation of these 3 stages using the Integrated Disease Surveillance and Response (IDSR) data from Malawi and Kenya. Results: The implementation of our framework significantly enhanced the FAIRness of population health data, resulting in improved discoverability through seamless integration with platforms such as Google Dataset Search. The adoption of standardized formats and protocols streamlined data accessibility and integration across various research environments, fostering collaboration and knowledge sharing. Additionally, the use of machine-interpretable metadata empowered researchers to efficiently reuse data for targeted analyses and insights, thereby maximizing the overall value of population health resources. The JSON-LD codes are accessible via a GitHub repository and the HTML code integrated with JSON-LD is available on the Implementation Network for Sharing Population Information from Research Entities website. Conclusions: The adoption of machine-readable metadata standards is essential for ensuring the FAIRness of population health data. By embracing these standards, organizations can enhance diverse resource visibility, accessibility, and utility, leading to a broader impact, particularly in low- and middle-income countries. Machine-readable metadata can accelerate research, improve health care decision-making, and ultimately promote better health outcomes for populations worldwide. %M 39088253 %R 10.2196/56237 %U https://ojphi.jmir.org/2024/1/e56237 %U https://doi.org/10.2196/56237 %U http://www.ncbi.nlm.nih.gov/pubmed/39088253 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e58764 %T The McMaster Health Information Research Unit: Over a Quarter-Century of Health Informatics Supporting Evidence-Based Medicine %A Lokker,Cynthia %A McKibbon,K Ann %A Afzal,Muhammad %A Navarro,Tamara %A Linkins,Lori-Ann %A Haynes,R Brian %A Iorio,Alfonso %+ Health Information Research Unit, Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main St W, CRL 137, Hamilton, ON, L8S 4K1, Canada, 1 2897883272, lokkerc@mcmaster.ca %K health informatics %K evidence-based medicine %K information retrieval %K evidence-based %K health information %K Boolean %K natural language processing %K NLP %K journal %K article %K Health Information Research Unit %K HiRU %D 2024 %7 31.7.2024 %9 Viewpoint %J J Med Internet Res %G English %X Evidence-based medicine (EBM) emerged from McMaster University in the 1980-1990s, which emphasizes the integration of the best research evidence with clinical expertise and patient values. The Health Information Research Unit (HiRU) was created at McMaster University in 1985 to support EBM. Early on, digital health informatics took the form of teaching clinicians how to search MEDLINE with modems and phone lines. Searching and retrieval of published articles were transformed as electronic platforms provided greater access to clinically relevant studies, systematic reviews, and clinical practice guidelines, with PubMed playing a pivotal role. In the early 2000s, the HiRU introduced Clinical Queries—validated search filters derived from the curated, gold-standard, human-appraised Hedges dataset—to enhance the precision of searches, allowing clinicians to hone their queries based on study design, population, and outcomes. Currently, almost 1 million articles are added to PubMed annually. To filter through this volume of heterogenous publications for clinically important articles, the HiRU team and other researchers have been applying classical machine learning, deep learning, and, increasingly, large language models (LLMs). These approaches are built upon the foundation of gold-standard annotated datasets and humans in the loop for active machine learning. In this viewpoint, we explore the evolution of health informatics in supporting evidence search and retrieval processes over the past 25+ years within the HiRU, including the evolving roles of LLMs and responsible artificial intelligence, as we continue to facilitate the dissemination of knowledge, enabling clinicians to integrate the best available evidence into their clinical practice. %M 39083765 %R 10.2196/58764 %U https://www.jmir.org/2024/1/e58764 %U https://doi.org/10.2196/58764 %U http://www.ncbi.nlm.nih.gov/pubmed/39083765 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e52896 %T Unsupervised Feature Selection to Identify Important ICD-10 and ATC Codes for Machine Learning on a Cohort of Patients With Coronary Heart Disease: Retrospective Study %A Ghasemi,Peyman %A Lee,Joon %K unsupervised feature selection %K ICD-10 %K International Classification of Diseases %K ATC %K Anatomical Therapeutic Chemical %K concrete autoencoder %K Laplacian score %K unsupervised feature selection for multicluster data %K autoencoder-inspired unsupervised feature selection %K principal feature analysis %K machine learning %K artificial intelligence %K case study %K coronary artery disease %K artery disease %K patient cohort %K artery %K mortality prediction %K mortality %K data set %K interpretability %K International Classification of Diseases, Tenth Revision %D 2024 %7 26.7.2024 %9 %J JMIR Med Inform %G English %X Background: The application of machine learning in health care often necessitates the use of hierarchical codes such as the International Classification of Diseases (ICD) and Anatomical Therapeutic Chemical (ATC) systems. These codes classify diseases and medications, respectively, thereby forming extensive data dimensions. Unsupervised feature selection tackles the “curse of dimensionality” and helps to improve the accuracy and performance of supervised learning models by reducing the number of irrelevant or redundant features and avoiding overfitting. Techniques for unsupervised feature selection, such as filter, wrapper, and embedded methods, are implemented to select the most important features with the most intrinsic information. However, they face challenges due to the sheer volume of ICD and ATC codes and the hierarchical structures of these systems. Objective: The objective of this study was to compare several unsupervised feature selection methods for ICD and ATC code databases of patients with coronary artery disease in different aspects of performance and complexity and select the best set of features representing these patients. Methods: We compared several unsupervised feature selection methods for 2 ICD and 1 ATC code databases of 51,506 patients with coronary artery disease in Alberta, Canada. Specifically, we used the Laplacian score, unsupervised feature selection for multicluster data, autoencoder-inspired unsupervised feature selection, principal feature analysis, and concrete autoencoders with and without ICD or ATC tree weight adjustment to select the 100 best features from over 9000 ICD and 2000 ATC codes. We assessed the selected features based on their ability to reconstruct the initial feature space and predict 90-day mortality following discharge. We also compared the complexity of the selected features by mean code level in the ICD or ATC tree and the interpretability of the features in the mortality prediction task using Shapley analysis. Results: In feature space reconstruction and mortality prediction, the concrete autoencoder–based methods outperformed other techniques. Particularly, a weight-adjusted concrete autoencoder variant demonstrated improved reconstruction accuracy and significant predictive performance enhancement, confirmed by DeLong and McNemar tests (P<.05). Concrete autoencoders preferred more general codes, and they consistently reconstructed all features accurately. Additionally, features selected by weight-adjusted concrete autoencoders yielded higher Shapley values in mortality prediction than most alternatives. Conclusions: This study scrutinized 5 feature selection methods in ICD and ATC code data sets in an unsupervised context. Our findings underscore the superiority of the concrete autoencoder method in selecting salient features that represent the entire data set, offering a potential asset for subsequent machine learning research. We also present a novel weight adjustment approach for the concrete autoencoders specifically tailored for ICD and ATC code data sets to enhance the generalizability and interpretability of the selected features. %R 10.2196/52896 %U https://medinform.jmir.org/2024/1/e52896 %U https://doi.org/10.2196/52896 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e54458 %T Complex Hospital-Based Electronic Prescribing–Based Intervention to Support Antimicrobial Stewardship: Qualitative Study %A Cresswell,Kathrin %A Hinder,Susan %A Sheikh,Aziz %A Watson,Neil %A Price,David %A Heed,Andrew %A Pontefract,Sarah Katie %A Coleman,Jamie %A Beggs,Jillian %A Chuter,Antony %A Slee,Ann %A Williams,Robin %+ Usher Institute, University of Edinburgh, Usher Building, 5‒7 Little France Road, Edinburgh, EH16 4UX, United Kingdom, 44 (0)131 651 4151, Kathrin.Cresswell@ed.ac.uk %K antimicrobial stewardship %K electronic prescribing %K evaluation %K healthcare %K qualitative study %K hospital-based %K electronic prescribing %K e-prescribing %K prescribing %K prescription %K ePAMS+ %K antimicrobial resistance %K AMR %K complex intervention %K complex interventions %K educational %K behavioral %K technological %K public health %K implementation %K AMS %K hospital %K hospitals %K development %K in-depth %K interview %K interviews %K observation %K observations %K prescriber %K prescribers %K nurse %K nurses %K pharmacist %K pharmacists %K microbiologist %K microbiologists %K thematic analysis %K antimicrobial %K antimicrobials %D 2024 %7 26.7.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Antimicrobial resistance (AMR) represents a growing concern for public health. Objective: We sought to explore the challenges associated with development and implementation of a complex intervention designed to improve AMS in hospitals. Methods: We conducted a qualitative evaluation of a complex AMS intervention with educational, behavioral, and technological components in 5 wards of an English hospital. At 2 weeks and 7 weeks after initiating the intervention, we interviewed 25 users of the intervention, including senior and junior prescribers, a senior nurse, a pharmacist, and a microbiologist. Topics discussed included perceived impacts of different elements of the intervention and facilitators and barriers to effective use. Interviews were supplemented by 2 observations of ward rounds to gain insights into AMS practices. Data were audio-recorded, transcribed, and inductively and deductively analyzed thematically using NVivo12. Results: Tracing the adoption and impact of the various components of the intervention was difficult, as it had been introduced into a setting with competing pressures. These particularly affected behavioral and educational components (eg, training, awareness-building activities), which were often delivered ad hoc. We found that the participatory intervention design had addressed typical use cases but had not catered for edge cases that only became visible when the intervention was delivered in real-world settings (eg, variations in prescribing workflows across different specialties and conditions). Conclusions: Effective user-focused design of complex interventions to promote AMS can support acceptance and use. However, not all requirements and potential barriers to use can be fully anticipated or tested in advance of full implementation in real-world settings. %M 39059001 %R 10.2196/54458 %U https://formative.jmir.org/2024/1/e54458 %U https://doi.org/10.2196/54458 %U http://www.ncbi.nlm.nih.gov/pubmed/39059001 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e49142 %T Prediction of In-Hospital Cardiac Arrest in the Intensive Care Unit: Machine Learning–Based Multimodal Approach %A Lee,Hsin-Ying %A Kuo,Po-Chih %A Qian,Frank %A Li,Chien-Hung %A Hu,Jiun-Ruey %A Hsu,Wan-Ting %A Jhou,Hong-Jie %A Chen,Po-Huang %A Lee,Cho-Hao %A Su,Chin-Hua %A Liao,Po-Chun %A Wu,I-Ju %A Lee,Chien-Chang %K cardiac arrest %K machine learning %K intensive care %K mortality %K medical emergency team %K early warning scores %D 2024 %7 23.7.2024 %9 %J JMIR Med Inform %G English %X Background: Early identification of impending in-hospital cardiac arrest (IHCA) improves clinical outcomes but remains elusive for practicing clinicians. Objective: We aimed to develop a multimodal machine learning algorithm based on ensemble techniques to predict the occurrence of IHCA. Methods: Our model was developed by the Multiparameter Intelligent Monitoring of Intensive Care (MIMIC)–IV database and validated in the Electronic Intensive Care Unit Collaborative Research Database (eICU-CRD). Baseline features consisting of patient demographics, presenting illness, and comorbidities were collected to train a random forest model. Next, vital signs were extracted to train a long short-term memory model. A support vector machine algorithm then stacked the results to form the final prediction model. Results: Of 23,909 patients in the MIMIC-IV database and 10,049 patients in the eICU-CRD database, 452 and 85 patients, respectively, had IHCA. At 13 hours in advance of an IHCA event, our algorithm had already demonstrated an area under the receiver operating characteristic curve of 0.85 (95% CI 0.815‐0.885) in the MIMIC-IV database. External validation with the eICU-CRD and National Taiwan University Hospital databases also presented satisfactory results, showing area under the receiver operating characteristic curve values of 0.81 (95% CI 0.763-0.851) and 0.945 (95% CI 0.934-0.956), respectively. Conclusions: Using only vital signs and information available in the electronic medical record, our model demonstrates it is possible to detect a trajectory of clinical deterioration up to 13 hours in advance. This predictive tool, which has undergone external validation, could forewarn and help clinicians identify patients in need of assessment to improve their overall prognosis. %R 10.2196/49142 %U https://medinform.jmir.org/2024/1/e49142 %U https://doi.org/10.2196/49142 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e58141 %T Construction of a Multi-Label Classifier for Extracting Multiple Incident Factors From Medication Incident Reports in Residential Care Facilities: Natural Language Processing Approach %A Kizaki,Hayato %A Satoh,Hiroki %A Ebara,Sayaka %A Watabe,Satoshi %A Sawada,Yasufumi %A Imai,Shungo %A Hori,Satoko %+ Division of Drug Informatics, Keio University Faculty of Pharmacy, 1-5-30 Shibakoen, Minato-ku, Tokyo, Japan, 81 354002799, hayatokizaki625@keio.jp %K residential facilities %K incidents %K non-medical staff %K natural language processing %K risk management %D 2024 %7 23.7.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Medication safety in residential care facilities is a critical concern, particularly when nonmedical staff provide medication assistance. The complex nature of medication-related incidents in these settings, coupled with the psychological impact on health care providers, underscores the need for effective incident analysis and preventive strategies. A thorough understanding of the root causes, typically through incident-report analysis, is essential for mitigating medication-related incidents. Objective: We aimed to develop and evaluate a multilabel classifier using natural language processing to identify factors contributing to medication-related incidents using incident report descriptions from residential care facilities, with a focus on incidents involving nonmedical staff. Methods: We analyzed 2143 incident reports, comprising 7121 sentences, from residential care facilities in Japan between April 1, 2015, and March 31, 2016. The incident factors were annotated using sentences based on an established organizational factor model and previous research findings. The following 9 factors were defined: procedure adherence, medicine, resident, resident family, nonmedical staff, medical staff, team, environment, and organizational management. To assess the label criteria, 2 researchers with relevant medical knowledge annotated a subset of 50 reports; the interannotator agreement was measured using Cohen κ. The entire data set was subsequently annotated by 1 researcher. Multiple labels were assigned to each sentence. A multilabel classifier was developed using deep learning models, including 2 Bidirectional Encoder Representations From Transformers (BERT)–type models (Tohoku-BERT and a University of Tokyo Hospital BERT pretrained with Japanese clinical text: UTH-BERT) and an Efficiently Learning Encoder That Classifies Token Replacements Accurately (ELECTRA), pretrained on Japanese text. Both sentence- and report-level training were performed; the performance was evaluated by the F1-score and exact match accuracy through 5-fold cross-validation. Results: Among all 7121 sentences, 1167, 694, 2455, 23, 1905, 46, 195, 1104, and 195 included “procedure adherence,” “medicine,” “resident,” “resident family,” “nonmedical staff,” “medical staff,” “team,” “environment,” and “organizational management,” respectively. Owing to limited labels, “resident family” and “medical staff” were omitted from the model development process. The interannotator agreement values were higher than 0.6 for each label. A total of 10, 278, and 1855 reports contained no, 1, and multiple labels, respectively. The models trained using the report data outperformed those trained using sentences, with macro F1-scores of 0.744, 0.675, and 0.735 for Tohoku-BERT, UTH-BERT, and ELECTRA, respectively. The report-trained models also demonstrated better exact match accuracy, with 0.411, 0.389, and 0.399 for Tohoku-BERT, UTH-BERT, and ELECTRA, respectively. Notably, the accuracy was consistent even when the analysis was confined to reports containing multiple labels. Conclusions: The multilabel classifier developed in our study demonstrated potential for identifying various factors associated with medication-related incidents using incident reports from residential care facilities. Thus, this classifier can facilitate prompt analysis of incident factors, thereby contributing to risk management and the development of preventive strategies. %M 39042454 %R 10.2196/58141 %U https://medinform.jmir.org/2024/1/e58141 %U https://doi.org/10.2196/58141 %U http://www.ncbi.nlm.nih.gov/pubmed/39042454 %0 Journal Article %@ 2291-9694 %I %V 12 %N %P e55959 %T The Information and Communication Technology Maturity Assessment at Primary Health Care Services Across 9 Provinces in Indonesia: Evaluation Study %A Aisyah,Dewi Nur %A Setiawan,Agus Heri %A Lokopessy,Alfiano Fawwaz %A Faradiba,Nadia %A Setiaji,Setiaji %A Manikam,Logan %A Kozlakidis,Zisis %K public health centers %K Puskesmas %K digital maturity %K infrastructure %K primary health care %K district health office %K primary care clinics %K Asia %K Asian %K Indonesia %K ICT %K information and communication technologies %K information and communication technology %K maturity %K adoption %K readiness %K implementation %K eHealth %K telehealth %K telemedicine %K cross sectional %K survey %K surveys %K questionnaire %K questionnaires %K primary care %D 2024 %7 18.7.2024 %9 %J JMIR Med Inform %G English %X Background: Indonesia has rapidly embraced digital health, particularly during the COVID-19 pandemic, with over 15 million daily health application users. To advance its digital health vision, the government is prioritizing the development of health data and application systems into an integrated health care technology ecosystem. This initiative involves all levels of health care, from primary to tertiary, across all provinces. In particular, it aims to enhance primary health care services (as the main interface with the general population) and contribute to Indonesia’s digital health transformation. Objective: This study assesses the information and communication technology (ICT) maturity in Indonesian health care services to advance digital health initiatives. ICT maturity assessment tools, specifically designed for middle-income countries, were used to evaluate digital health capabilities in 9 provinces across 5 Indonesian islands. Methods: A cross-sectional survey was conducted from February to March 2022, in 9 provinces across Indonesia, representing the country’s diverse conditions on its major islands. Respondents included staff from public health centers (Puskesmas), primary care clinics (Klinik Pratama), and district health offices (Dinas Kesehatan Kabupaten/Kota). The survey used adapted ICT maturity assessment questionnaires, covering human resources, software and system, hardware, and infrastructure. It was administered electronically and involved 121 public health centers, 49 primary care clinics, and 67 IT staff from district health offices. Focus group discussions were held to delve deeper into the assessment results and gain more descriptive insights. Results: In this study, 237 participants represented 3 distinct categories: 121 public health centers, 67 district health offices, and 49 primary clinics. These instances were selected from a sample of 9 of the 34 provinces in Indonesia. Collected data from interviews and focus group discussions were transformed into scores on a scale of 1 to 5, with 1 indicating low ICT readiness and 5 indicating high ICT readiness. On average, the breakdown of ICT maturity scores was as follows: 2.71 for human resources’ capability in ICT use and system management, 2.83 for software and information systems, 2.59 for hardware, and 2.84 for infrastructure, resulting in an overall average score of 2.74. According to the ICT maturity level pyramid, the ICT maturity of health care providers in Indonesia fell between the basic and good levels. The need to pursue best practices also emerged strongly. Further analysis of the ICT maturity scores, when examined by province, revealed regional variations. Conclusions: The maturity of ICT use is influenced by several critical components. Enhancing human resources, ensuring infrastructure, the availability of supportive hardware, and optimizing information systems are imperative to attain ICT maturity in health care services. In the context of ICT maturity assessment, significant score variations were observed across health care levels in the 9 provinces, underscoring the diversity in ICT readiness and the need for regionally customized follow-up actions. %R 10.2196/55959 %U https://medinform.jmir.org/2024/1/e55959 %U https://doi.org/10.2196/55959 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e48600 %T Acceptance of AI in Health Care for Short- and Long-Term Treatments: Pilot Development Study of an Integrated Theoretical Model %A Wichmann,Johannes %A Gesk,Tanja Sophie %A Leyer,Michael %+ Working group Digitalization and Process Management, Department of Business, Philipps-University Marburg, Barfuessertor 2, Marburg, 35037, Germany, 49 64212823712, johannes.wichmann@wiwi.uni-marburg.de %K health information systems %K integrated theoretical model %K artificial intelligence %K health care %K technology acceptance %K long-term treatments %K short-term treatments %K mobile phone %D 2024 %7 18.7.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: As digital technologies and especially artificial intelligence (AI) become increasingly important in health care, it is essential to determine whether and why potential users intend to use related health information systems (HIS). Several theories exist, but they focus mainly on aspects of health care or information systems, in addition to general psychological theories, and hence provide a small number of variables to explain future behavior. Thus, research that provides a larger number of variables by combining several theories from health care, information systems, and psychology is necessary. Objective: This study aims to investigate the intention to use new HIS for decisions concerning short- and long-term medical treatments using an integrated approach with several variables to explain future behavior. Methods: We developed an integrated theoretical model based on theories from health care, information systems, and psychology that allowed us to analyze the duality approach of adaptive and nonadaptive appraisals and their influence on the intention to use HIS. We applied the integrated theoretical model to the short-term treatment using AI-based HIS for surgery and the long-term treatment of diabetes tracking using survey data with structured equation modeling. To differentiate between certain levels of AI involvement, we used several scenarios that include treatments by physicians only, physicians with AI support, and AI only to understand how individuals perceive the influence of AI. Results: Our results showed that for short- and long-term treatments, the variables perceived threats, fear (disease), perceived efficacy, attitude (HIS), and perceived norms are important to consider when determining the intention to use AI-based HIS. Furthermore, the results revealed that perceived efficacy and attitude (HIS) are the most important variables to determine intention to use for all treatments and scenarios. In contrast, abilities (HIS) were important for short-term treatments only. For our 9 scenarios, adaptive and nonadaptive appraisals were both important to determine intention to use, depending on whether the treatment is known. Furthermore, we determined R² values that varied between 57.9% and 81.7% for our scenarios, which showed that the explanation power of our model is medium to good. Conclusions: We contribute to HIS literature by highlighting the importance of integrating disease- and technology-related factors and by providing an integrated theoretical model. As such, we show how adaptive and nonadaptive appraisals should be arranged to report on medical decisions in the future, especially in the short and long terms. Physicians and HIS developers can use our insights to identify promising rationale for HIS adoption concerning short- and long-term treatments and adapt and develop HIS accordingly. Specifically, HIS developers should ensure that future HIS act in terms of HIS functions, as our study shows that efficient HIS lead to a positive attitude toward the HIS and ultimately to a higher intention to use. %M 39024565 %R 10.2196/48600 %U https://formative.jmir.org/2024/1/e48600 %U https://doi.org/10.2196/48600 %U http://www.ncbi.nlm.nih.gov/pubmed/39024565 %0 Journal Article %@ 2291-9694 %I %V 12 %N %P e54590 %T Data Lake, Data Warehouse, Datamart, and Feature Store: Their Contributions to the Complete Data Reuse Pipeline %A Lamer,Antoine %A Saint-Dizier,Chloé %A Paris,Nicolas %A Chazard,Emmanuel %K data reuse %K data lake %K data warehouse %K feature extraction %K datamart %K feature store %D 2024 %7 17.7.2024 %9 %J JMIR Med Inform %G English %X The growing adoption and use of health information technology has generated a wealth of clinical data in electronic format, offering opportunities for data reuse beyond direct patient care. However, as data are distributed across multiple software, it becomes challenging to cross-reference information between sources due to differences in formats, vocabularies, and technologies and the absence of common identifiers among software. To address these challenges, hospitals have adopted data warehouses to consolidate and standardize these data for research. Additionally, as a complement or alternative, data lakes store both source data and metadata in a detailed and unprocessed format, empowering exploration, manipulation, and adaptation of the data to meet specific analytical needs. Subsequently, datamarts are used to further refine data into usable information tailored to specific research questions. However, for efficient analysis, a feature store is essential to pivot and denormalize the data, simplifying queries. In conclusion, while data warehouses are crucial, data lakes, datamarts, and feature stores play essential and complementary roles in facilitating data reuse for research and analysis in health care. %R 10.2196/54590 %U https://medinform.jmir.org/2024/1/e54590 %U https://doi.org/10.2196/54590 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e56095 %T The Paradoxes of Digital Tools in Hospitals: Qualitative Interview Study %A Wosny,Marie %A Strasser,Livia Maria %A Hastings,Janna %+ School of Medicine, University of St Gallen, St.Jakob-Strasse 21, St.Gallen, 9000, Switzerland, 41 712243249, mariejohanna.wosny@unisg.ch %K health care %K health care technology %K health care information technology %K hospital information technology %K clinical information systems %K health care professionals %K experience %K frustration %K clinician burnout %K technology implementation %K paradoxes %K digital tool %K digital tools %K hospital %K hospitals %K qualitative interview study %K interview %K interviews %K Switzerland %K thematic analysis %D 2024 %7 15.7.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Digital tools are progressively reshaping the daily work of health care professionals (HCPs) in hospitals. While this transformation holds substantial promise, it leads to frustrating experiences, raising concerns about negative impacts on clinicians’ well-being. Objective: The goal of this study was to comprehensively explore the lived experiences of HCPs navigating digital tools throughout their daily routines. Methods: Qualitative in-depth interviews with 52 HCPs representing 24 medical specialties across 14 hospitals in Switzerland were performed. Results: Inductive thematic analysis revealed 4 main themes: digital tool use, workflow and processes, HCPs’ experience of care delivery, and digital transformation and management of change. Within these themes, 6 intriguing paradoxes emerged, and we hypothesized that these paradoxes might partly explain the persistence of the challenges facing hospital digitalization: the promise of efficiency and the reality of inefficiency, the shift from face to face to interface, juggling frustration and dedication, the illusion of information access and trust, the complexity and intersection of workflows and care paths, and the opportunities and challenges of shadow IT. Conclusions: Our study highlights the central importance of acknowledging and considering the experiences of HCPs to support the transformation of health care technology and to avoid or mitigate any potential negative experiences that might arise from digitalization. The viewpoints of HCPs add relevant insights into long-standing informatics problems in health care and may suggest new strategies to follow when tackling future challenges. %M 39008341 %R 10.2196/56095 %U https://www.jmir.org/2024/1/e56095 %U https://doi.org/10.2196/56095 %U http://www.ncbi.nlm.nih.gov/pubmed/39008341 %0 Journal Article %@ 2561-7605 %I JMIR Publications %V 7 %N %P e54748 %T Self-Explainable Graph Neural Network for Alzheimer Disease and Related Dementias Risk Prediction: Algorithm Development and Validation Study %A Hu,Xinyue %A Sun,Zenan %A Nian,Yi %A Wang,Yichen %A Dang,Yifang %A Li,Fang %A Feng,Jingna %A Yu,Evan %A Tao,Cui %+ Department of Artificial Intelligence and Informatics, Mayo Clinic, 4500 San Pablo Rd S, Jacksonville, FL, 32224, United States, 1 904 956 3256, tao.cui@mayo.edu %K Alzheimer disease and related dementias %K risk prediction %K graph neural network %K relation importance %K machine learning %D 2024 %7 8.7.2024 %9 Original Paper %J JMIR Aging %G English %X Background: Alzheimer disease and related dementias (ADRD) rank as the sixth leading cause of death in the United States, underlining the importance of accurate ADRD risk prediction. While recent advancements in ADRD risk prediction have primarily relied on imaging analysis, not all patients undergo medical imaging before an ADRD diagnosis. Merging machine learning with claims data can reveal additional risk factors and uncover interconnections among diverse medical codes. Objective: The study aims to use graph neural networks (GNNs) with claim data for ADRD risk prediction. Addressing the lack of human-interpretable reasons behind these predictions, we introduce an innovative, self-explainable method to evaluate relationship importance and its influence on ADRD risk prediction. Methods: We used a variationally regularized encoder-decoder GNN (variational GNN [VGNN]) integrated with our proposed relation importance method for estimating ADRD likelihood. This self-explainable method can provide a feature-important explanation in the context of ADRD risk prediction, leveraging relational information within a graph. Three scenarios with 1-year, 2-year, and 3-year prediction windows were created to assess the model’s efficiency, respectively. Random forest (RF) and light gradient boost machine (LGBM) were used as baselines. By using this method, we further clarify the key relationships for ADRD risk prediction. Results: In scenario 1, the VGNN model showed area under the receiver operating characteristic (AUROC) scores of 0.7272 and 0.7480 for the small subset and the matched cohort data set. It outperforms RF and LGBM by 10.6% and 9.1%, respectively, on average. In scenario 2, it achieved AUROC scores of 0.7125 and 0.7281, surpassing the other models by 10.5% and 8.9%, respectively. Similarly, in scenario 3, AUROC scores of 0.7001 and 0.7187 were obtained, exceeding 10.1% and 8.5% than the baseline models, respectively. These results clearly demonstrate the significant superiority of the graph-based approach over the tree-based models (RF and LGBM) in predicting ADRD. Furthermore, the integration of the VGNN model and our relation importance interpretation could provide valuable insight into paired factors that may contribute to or delay ADRD progression. Conclusions: Using our innovative self-explainable method with claims data enhances ADRD risk prediction and provides insights into the impact of interconnected medical code relationships. This methodology not only enables ADRD risk modeling but also shows potential for other image analysis predictions using claims data. %M 38976869 %R 10.2196/54748 %U https://aging.jmir.org/2024/1/e54748 %U https://doi.org/10.2196/54748 %U http://www.ncbi.nlm.nih.gov/pubmed/38976869 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e51931 %T Investigating Patient Use and Experience of Online Appointment Booking in Primary Care: Mixed Methods Study %A Atherton,Helen %A Eccles,Abi %A Poltawski,Leon %A Dale,Jeremy %A Campbell,John %A Abel,Gary %+ Primary Care Research Centre, School of Primary Care, Population Science, and Medical Education, University of Southampton, Aldermoor Health Centre, Southampton, SO16 5ST, United Kingdom, 44 023 8059 5000, h.atherton@soton.ac.uk %K appointment %K patient appointments %K online systems %K primary health care %K general practice %K qualitative research %K secondary data analysis %K mobile phone %D 2024 %7 8.7.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Online appointment booking is a commonly used tool in several industries. There is limited evidence about the benefits and challenges of using online appointment booking in health care settings. Potential benefits include convenience and the ability to track appointments, although some groups of patients may find it harder to engage with online appointment booking. We sought to understand how patients in England used and experienced online appointment booking. Objective: This study aims to describe and compare the characteristics of patients in relation to their use of online appointment booking in general practice and investigate patients’ views regarding online appointment booking arrangements. Methods: This was a mixed methods study set in English general practice comprising a retrospective analysis of the General Practice Patient Survey (GPPS) and semistructured interviews with patients. Data used in the retrospective analysis comprised responses to the 2018 and 2019 GPPS analyzed using mixed-effects logistic regression. Semistructured interviews with purposively sampled patients from 11 general practices in England explored experiences of and views on online appointment booking. Framework analysis was used to allow for comparison with the findings of the retrospective analysis. Results: The retrospective analysis included 1,327,693 GPPS responders (2018-2019 combined). We conducted 43 interviews with patients with a variety of experiences and awareness of online appointment booking; of these 43 patients, 6 (14%) were from ethnic minority groups. In the retrospective analysis, more patients were aware that online appointment booking was available (581,224/1,288,341, 45.11%) than had experience using it (203,184/1,301,694, 15.61%). There were deprivation gradients for awareness and use and a substantial decline in both awareness and use in patients aged >75 years. For interview participants, age and life stage were factors influencing experiences and perceptions, working patients valued convenience, and older patients preferred to use the telephone. Patients with long-term conditions were more aware of (odds ratio [OR] 1.43, 95% CI 1.41-1.44) and more likely to use (OR 1.65, 95% CI 1.63-1.67) online appointment booking. Interview participants with long-term conditions described online appointment booking as useful for routine nonurgent appointments. Patients in deprived areas were clustered in practices with low awareness and use of online appointment booking among GPPS respondents (OR for use 0.65, 95% CI 0.64-0.67). Other key findings included the influence of the availability of appointments online and differences in the registration process for accessing online booking. Conclusions: Whether and how patients engage with online appointment booking is influenced by the practice with which they are registered, whether they live with long-term conditions, and their deprivation status. These factors should be considered in designing and implementing online appointment booking and have implications for patient engagement with the wider range of online services offered in general practice. %M 38976870 %R 10.2196/51931 %U https://www.jmir.org/2024/1/e51931 %U https://doi.org/10.2196/51931 %U http://www.ncbi.nlm.nih.gov/pubmed/38976870 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e57981 %T Combining Federated Machine Learning and Qualitative Methods to Investigate Novel Pediatric Asthma Subtypes: Protocol for a Mixed Methods Study %A Xu,Jie %A Talankar,Sankalp %A Pan,Jinqian %A Harmon,Ira %A Wu,Yonghui %A Fedele,David A %A Brailsford,Jennifer %A Fishe,Jennifer Noel %+ Department of Emergency Medicine, Center for Data Solutions, University of Florida College of Medicine - Jacksonville, 655 W 8th St., Jacksonville, FL, 32209, United States, 1 904 244 4046, Jennifer.Fishe@jax.ufl.edu %K pediatric asthma %K machine learning %K federated learning %K qualitative research %D 2024 %7 8.7.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Pediatric asthma is a heterogeneous disease; however, current characterizations of its subtypes are limited. Machine learning (ML) methods are well-suited for identifying subtypes. In particular, deep neural networks can learn patient representations by leveraging longitudinal information captured in electronic health records (EHRs) while considering future outcomes. However, the traditional approach for subtype analysis requires large amounts of EHR data, which may contain protected health information causing potential concerns regarding patient privacy. Federated learning is the key technology to address privacy concerns while preserving the accuracy and performance of ML algorithms. Federated learning could enable multisite development and implementation of ML algorithms to facilitate the translation of artificial intelligence into clinical practice. Objective: The aim of this study is to develop a research protocol for implementation of federated ML across a large clinical research network to identify and discover pediatric asthma subtypes and their progression over time. Methods: This mixed methods study uses data and clinicians from the OneFlorida+ clinical research network, which is a large regional network covering linked and longitudinal patient-level real-world data (RWD) of over 20 million patients from Florida, Georgia, and Alabama in the United States. To characterize the subtypes, we will use OneFlorida+ data from 2011 to 2023 and develop a research-grade pediatric asthma computable phenotype and clinical natural language processing pipeline to identify pediatric patients with asthma aged 2-18 years. We will then apply federated learning to characterize pediatric asthma subtypes and their temporal progression. Using the Promoting Action on Research Implementation in Health Services framework, we will conduct focus groups with practicing pediatric asthma clinicians within the OneFlorida+ network to investigate the clinical utility of the subtypes. With a user-centered design, we will create prototypes to visualize the subtypes in the EHR to best assist with the clinical management of children with asthma. Results: OneFlorida+ data from 2011 to 2023 have been collected for 411,628 patients aged 2-18 years along with 11,156,148 clinical notes. We expect to complete the computable phenotyping within the first year of the project, followed by subtyping during the second and third years, and then will perform the focus groups and establish the user-centered design in the fourth and fifth years of the project. Conclusions: Pediatric asthma subtypes incorporating RWD from diverse populations could improve patient outcomes by moving the field closer to precision pediatric asthma care. Our privacy-preserving federated learning methodology and qualitative implementation work will address several challenges of applying ML to large, multicenter RWD data. International Registered Report Identifier (IRRID): DERR1-10.2196/57981 %M 38976313 %R 10.2196/57981 %U https://www.researchprotocols.org/2024/1/e57981 %U https://doi.org/10.2196/57981 %U http://www.ncbi.nlm.nih.gov/pubmed/38976313 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e54263 %T Electronic Health Record–Oriented Knowledge Graph System for Collaborative Clinical Decision Support Using Multicenter Fragmented Medical Data: Design and Application Study %A Shang,Yong %A Tian,Yu %A Lyu,Kewei %A Zhou,Tianshu %A Zhang,Ping %A Chen,Jianghua %A Li,Jingsong %+ Research Center for Data Hub and Security, Zhejiang Laboratory, No.1 Kechuang Avenue, Zhongtai Sub-District, Yuhang District, Hangzhou, 310000, China, 86 0571 58005162, ljs@zju.edu.cn %K knowledge graph %K electronic health record %K ontology %K data fragmentation %K data privacy %K knowledge graphs %K visualization %K ontologies %K data science %K privacy %K security %K collaborative %K collaboration %K kidney %K CKD %K nephrology %K EHR %K health record %K hypernym %K encryption %K encrypt %K encrypted %K decision support %K semantic %K vocabulary %K blockchain %D 2024 %7 5.7.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: The medical knowledge graph provides explainable decision support, helping clinicians with prompt diagnosis and treatment suggestions. However, in real-world clinical practice, patients visit different hospitals seeking various medical services, resulting in fragmented patient data across hospitals. With data security issues, data fragmentation limits the application of knowledge graphs because single-hospital data cannot provide complete evidence for generating precise decision support and comprehensive explanations. It is important to study new methods for knowledge graph systems to integrate into multicenter, information-sensitive medical environments, using fragmented patient records for decision support while maintaining data privacy and security. Objective: This study aims to propose an electronic health record (EHR)–oriented knowledge graph system for collaborative reasoning with multicenter fragmented patient medical data, all the while preserving data privacy. Methods: The study introduced an EHR knowledge graph framework and a novel collaborative reasoning process for utilizing multicenter fragmented information. The system was deployed in each hospital and used a unified semantic structure and Observational Medical Outcomes Partnership (OMOP) vocabulary to standardize the local EHR data set. The system transforms local EHR data into semantic formats and performs semantic reasoning to generate intermediate reasoning findings. The generated intermediate findings used hypernym concepts to isolate original medical data. The intermediate findings and hash-encrypted patient identities were synchronized through a blockchain network. The multicenter intermediate findings were collaborated for final reasoning and clinical decision support without gathering original EHR data. Results: The system underwent evaluation through an application study involving the utilization of multicenter fragmented EHR data to alert non-nephrology clinicians about overlooked patients with chronic kidney disease (CKD). The study covered 1185 patients in nonnephrology departments from 3 hospitals. The patients visited at least two of the hospitals. Of these, 124 patients were identified as meeting CKD diagnosis criteria through collaborative reasoning using multicenter EHR data, whereas the data from individual hospitals alone could not facilitate the identification of CKD in these patients. The assessment by clinicians indicated that 78/91 (86%) patients were CKD positive. Conclusions: The proposed system was able to effectively utilize multicenter fragmented EHR data for clinical application. The application study showed the clinical benefits of the system with prompt and comprehensive decision support. %M 38968598 %R 10.2196/54263 %U https://www.jmir.org/2024/1/e54263 %U https://doi.org/10.2196/54263 %U http://www.ncbi.nlm.nih.gov/pubmed/38968598 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e56127 %T Collaborative Human–Computer Vision Operative Video Analysis Algorithm for Analyzing Surgical Fluency and Surgical Interruptions in Endonasal Endoscopic Pituitary Surgery: Cohort Study %A Wong,Chia-En %A Chen,Pei-Wen %A Hsu,Heng-Jui %A Cheng,Shao-Yang %A Fan,Chen-Che %A Chen,Yen-Chang %A Chiu,Yi-Pei %A Lee,Jung-Shun %A Liang,Sheng-Fu %+ Department of Computer Science and Information Engineering, National Cheng Kung University, No 1, University Road, Tainan, 701, Taiwan, 886 62757575, sfliang@ncku.edu.tw %K algorithm %K computer vision %K endonasal endoscopic approach %K pituitary %K transsphenoidal surgery %D 2024 %7 4.7.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: The endonasal endoscopic approach (EEA) is effective for pituitary adenoma resection. However, manual review of operative videos is time-consuming. The application of a computer vision (CV) algorithm could potentially reduce the time required for operative video review and facilitate the training of surgeons to overcome the learning curve of EEA. Objective: This study aimed to evaluate the performance of a CV-based video analysis system, based on OpenCV algorithm, to detect surgical interruptions and analyze surgical fluency in EEA. The accuracy of the CV-based video analysis was investigated, and the time required for operative video review using CV-based analysis was compared to that of manual review. Methods: The dominant color of each frame in the EEA video was determined using OpenCV. We developed an algorithm to identify events of surgical interruption if the alterations in the dominant color pixels reached certain thresholds. The thresholds were determined by training the current algorithm using EEA videos. The accuracy of the CV analysis was determined by manual review, and the time spent was reported. Results: A total of 46 EEA operative videos were analyzed, with 93.6%, 95.1%, and 93.3% accuracies in the training, test 1, and test 2 data sets, respectively. Compared with manual review, CV-based analysis reduced the time required for operative video review by 86% (manual review: 166.8 and CV analysis: 22.6 minutes; P<.001). The application of a human-computer collaborative strategy increased the overall accuracy to 98.5%, with a 74% reduction in the review time (manual review: 166.8 and human-CV collaboration: 43.4 minutes; P<.001). Analysis of the different surgical phases showed that the sellar phase had the lowest frequency (nasal phase: 14.9, sphenoidal phase: 15.9, and sellar phase: 4.9 interruptions/10 minutes; P<.001) and duration (nasal phase: 67.4, sphenoidal phase: 77.9, and sellar phase: 31.1 seconds/10 minutes; P<.001) of surgical interruptions. A comparison of the early and late EEA videos showed that increased surgical experience was associated with a decreased number (early: 4.9 and late: 2.9 interruptions/10 minutes; P=.03) and duration (early: 41.1 and late: 19.8 seconds/10 minutes; P=.02) of surgical interruptions during the sellar phase. Conclusions: CV-based analysis had a 93% to 98% accuracy in detecting the number, frequency, and duration of surgical interruptions occurring during EEA. Moreover, CV-based analysis reduced the time required to analyze the surgical fluency in EEA videos compared to manual review. The application of CV can facilitate the training of surgeons to overcome the learning curve of endoscopic skull base surgery. Trial Registration: ClinicalTrials.gov NCT06156020; https://clinicaltrials.gov/study/NCT06156020 %M 38963694 %R 10.2196/56127 %U https://www.jmir.org/2024/1/e56127 %U https://doi.org/10.2196/56127 %U http://www.ncbi.nlm.nih.gov/pubmed/38963694 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e52139 %T Artificial Intelligence–Based Electrocardiographic Biomarker for Outcome Prediction in Patients With Acute Heart Failure: Prospective Cohort Study %A Cho,Youngjin %A Yoon,Minjae %A Kim,Joonghee %A Lee,Ji Hyun %A Oh,Il-Young %A Lee,Chan Joo %A Kang,Seok-Min %A Choi,Dong-Ju %+ Division of Cardiology, Department of Internal Medicine, Seoul National University Bundang Hospital, Seoul National University College of Medicine, 82 Gumi-ro 173 Beon-gil, Bundang-gu, Seongnam, Gyeonggi-do, 13620, Republic of Korea, 82 317877007, djchoi@snubh.org %K acute heart failure %K electrocardiography %K artificial intelligence %K deep learning %D 2024 %7 3.7.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Although several biomarkers exist for patients with heart failure (HF), their use in routine clinical practice is often constrained by high costs and limited availability. Objective: We examined the utility of an artificial intelligence (AI) algorithm that analyzes printed electrocardiograms (ECGs) for outcome prediction in patients with acute HF. Methods: We retrospectively analyzed prospectively collected data of patients with acute HF at two tertiary centers in Korea. Baseline ECGs were analyzed using a deep-learning system called Quantitative ECG (QCG), which was trained to detect several urgent clinical conditions, including shock, cardiac arrest, and reduced left ventricular ejection fraction (LVEF). Results: Among the 1254 patients enrolled, in-hospital cardiac death occurred in 53 (4.2%) patients, and the QCG score for critical events (QCG-Critical) was significantly higher in these patients than in survivors (mean 0.57, SD 0.23 vs mean 0.29, SD 0.20; P<.001). The QCG-Critical score was an independent predictor of in-hospital cardiac death after adjustment for age, sex, comorbidities, HF etiology/type, atrial fibrillation, and QRS widening (adjusted odds ratio [OR] 1.68, 95% CI 1.47-1.92 per 0.1 increase; P<.001), and remained a significant predictor after additional adjustments for echocardiographic LVEF and N-terminal prohormone of brain natriuretic peptide level (adjusted OR 1.59, 95% CI 1.36-1.87 per 0.1 increase; P<.001). During long-term follow-up, patients with higher QCG-Critical scores (>0.5) had higher mortality rates than those with low QCG-Critical scores (<0.25) (adjusted hazard ratio 2.69, 95% CI 2.14-3.38; P<.001). Conclusions: Predicting outcomes in patients with acute HF using the QCG-Critical score is feasible, indicating that this AI-based ECG score may be a novel biomarker for these patients. Trial Registration: ClinicalTrials.gov NCT01389843; https://clinicaltrials.gov/study/NCT01389843 %M 38959500 %R 10.2196/52139 %U https://www.jmir.org/2024/1/e52139 %U https://doi.org/10.2196/52139 %U http://www.ncbi.nlm.nih.gov/pubmed/38959500 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 10 %N %P e49127 %T Factors Influencing Data Quality in Electronic Health Record Systems in 50 Health Facilities in Rwanda and the Role of Clinical Alerts: Cross-Sectional Observational Study %A Fraser,Hamish S F %A Mugisha,Michael %A Bacher,Ian %A Ngenzi,Joseph Lune %A Seebregts,Christopher %A Umubyeyi,Aline %A Condo,Jeanine %+ Brown Center for Biomedical Informatics, Brown University, 233 Richmond Street, Providence, RI, 02903, United States, 1 4018631815, hamish_fraser@brown.edu %K data quality %K electronic health record %K EHR %K electronic medical record %K EMR %K HIV %K Rwanda %D 2024 %7 3.7.2024 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: Electronic health records (EHRs) play an increasingly important role in delivering HIV care in low- and middle-income countries. The data collected are used for direct clinical care, quality improvement, program monitoring, public health interventions, and research. Despite widespread EHR use for HIV care in African countries, challenges remain, especially in collecting high-quality data. Objective: We aimed to assess data completeness, accuracy, and timeliness compared to paper-based records, and factors influencing data quality in a large-scale EHR deployment in Rwanda. Methods: We randomly selected 50 health facilities (HFs) using OpenMRS, an EHR system that supports HIV care in Rwanda, and performed a data quality evaluation. All HFs were part of a larger randomized controlled trial, with 25 HFs receiving an enhanced EHR with clinical decision support systems. Trained data collectors visited the 50 HFs to collect 28 variables from the paper charts and the EHR system using the Open Data Kit app. We measured data completeness, timeliness, and the degree of matching of the data in paper and EHR records, and calculated concordance scores. Factors potentially affecting data quality were drawn from a previous survey of users in the 50 HFs. Results: We randomly selected 3467 patient records, reviewing both paper and EHR copies (194,152 total data items). Data completeness was >85% threshold for all data elements except viral load (VL) results, second-line, and third-line drug regimens. Matching scores for data values were close to or >85% threshold, except for dates, particularly for drug pickups and VL. The mean data concordance was 10.2 (SD 1.28) for 15 (68%) variables. HF and user factors (eg, years of EHR use, technology experience, EHR availability and uptime, and intervention status) were tested for correlation with data quality measures. EHR system availability and uptime was positively correlated with concordance, whereas users’ experience with technology was negatively correlated with concordance. The alerts for missing VL results implemented at 11 intervention HFs showed clear evidence of improving timeliness and completeness of initially low matching of VL results in the EHRs and paper records (11.9%-26.7%; P<.001). Similar effects were seen on the completeness of the recording of medication pickups (18.7%-32.6%; P<.001). Conclusions: The EHR records in the 50 HFs generally had high levels of completeness except for VL results. Matching results were close to or >85% threshold for nondate variables. Higher EHR stability and uptime, and alerts for entering VL both strongly improved data quality. Most data were considered fit for purpose, but more regular data quality assessments, training, and technical improvements in EHR forms, data reports, and alerts are recommended. The application of quality improvement techniques described in this study should benefit a wide range of HFs and data uses for clinical care, public health, and disease surveillance. %M 38959048 %R 10.2196/49127 %U https://publichealth.jmir.org/2024/1/e49127 %U https://doi.org/10.2196/49127 %U http://www.ncbi.nlm.nih.gov/pubmed/38959048 %0 Journal Article %@ 1929-073X %I JMIR Publications %V 13 %N %P e44906 %T Intramural Health Care Through Video Consultations and the Need for Referrals and Hospital Admissions: Retrospective Quantitative Subanalysis of an Evaluation Study %A Schmalstieg-Bahr,Katharina %A Colombo,Miriam Giovanna %A Koch,Roland %A Szecsenyi,Joachim %A Völker,Friedrich %A Blozik,Eva Elisabeth %A Scherer,Martin %+ Department of General Practice and Primary Care, University Medical Center Eppendorf, Martinistrasse 52, Bldg. W37, 5th Fl, Hamburg, 20246, Germany, 49 40 7410 52400, k.schmalstieg-bahr@uke.de %K intramural health care %K prison %K telemedicine %K primary care %K family medicine %K referral %K hospital admission %K admission rate %K intramural %K penal %K video consult %K e-consult %K remote care %K virtual care %K health care delivery %K service delivery %K health care system %D 2024 %7 28.6.2024 %9 Original Paper %J Interact J Med Res %G English %X Background: In comparison to the general population, prison inmates are at a higher risk for drug abuse and psychiatric, as well as infectious, diseases. Although intramural health care has to be equivalent to extramural services, prison inmates have less access to primary and secondary care. Furthermore, not every prison is constantly staffed with a physician. Since transportation to the nearest extramural medical facility is often resource-intensive, video consultations may offer cost-effective health care for prison inmates. Objective: This study aims to quantify the need for referrals to secondary care services and hospital admissions when video consultations with family physicians and psychiatrists are offered in prison. Methods: In 5 German prisons, a mixed methods evaluation study was conducted to assess feasibility, acceptance, and reasons for conducting video consultations with family physicians and psychiatrists. This analysis uses quantitative data from these consultations (June 2018 to February 2019) in addition to data from a sixth prison added in January 2019 focusing on referral and admission rates, as well as reasons for encounters. Results: At the initiation of the project, 2499 prisoners were detained in the 6 prisons. A total of 435 video consultations were conducted by 12 physicians (3 female and 7 male family physicians, and 2 male psychiatrists during the study period). The majority were scheduled consultations (341/435, 78%). In 68% (n=294) of all encounters, the patient was asked to consult a physician again if symptoms persisted or got worse. In 26% (n=115), a follow-up appointment with either the video consultant or prison physician was scheduled. A referral to other specialties, most often psychiatry, was necessary in 4% (n=17) of the cases. Only in 2% (n=8) of the consultations, a hospital admission was needed. Usually, hospital admissions were the result of unscheduled consultations, and the videoconferencing system was the method of communication in 88% (n=7) of these cases, while 12% (n=1) were carried out over the phone. Reasons for admissions were severe abdominal pain, hypotension, unstable angina or suspected myocardial infarction, or a suspected schizophrenic episode. Conclusions: Most scheduled and unscheduled consultations did not require subsequent patient transport to external health care providers. Using telemedicine services allowed a prompt patient-physician encounter with the possibility to refer patients to other specialties or to admit them to a hospital if necessary. %M 38941595 %R 10.2196/44906 %U https://www.i-jmr.org/2024/1/e44906 %U https://doi.org/10.2196/44906 %U http://www.ncbi.nlm.nih.gov/pubmed/38941595 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e58491 %T AI: Bridging Ancient Wisdom and Modern Innovation in Traditional Chinese Medicine %A Lu,Linken %A Lu,Tangsheng %A Tian,Chunyu %A Zhang,Xiujun %+ School of Psychology and Mental Health, North China University of Science and Technology, 21 Bohai Avenue, Caofeidian New Town, Tangshan, Hebei Province, 063210, China, 86 0315 8805970, zhxj@ncst.edu.cn %K traditional Chinese medicine %K TCM %K artificial intelligence %K AI %K diagnosis %D 2024 %7 28.6.2024 %9 Viewpoint %J JMIR Med Inform %G English %X The pursuit of groundbreaking health care innovations has led to the convergence of artificial intelligence (AI) and traditional Chinese medicine (TCM), thus marking a new frontier that demonstrates the promise of combining the advantages of ancient healing practices with cutting-edge advancements in modern technology. TCM, which is a holistic medical system with >2000 years of empirical support, uses unique diagnostic methods such as inspection, auscultation and olfaction, inquiry, and palpation. AI is the simulation of human intelligence processes by machines, especially via computer systems. TCM is experience oriented, holistic, and subjective, and its combination with AI has beneficial effects, which presumably arises from the perspectives of diagnostic accuracy, treatment efficacy, and prognostic veracity. The role of AI in TCM is highlighted by its use in diagnostics, with machine learning enhancing the precision of treatment through complex pattern recognition. This is exemplified by the greater accuracy of TCM syndrome differentiation via tongue images that are analyzed by AI. However, integrating AI into TCM also presents multifaceted challenges, such as data quality and ethical issues; thus, a unified strategy, such as the use of standardized data sets, is required to improve AI understanding and application of TCM principles. The evolution of TCM through the integration of AI is a key factor for elucidating new horizons in health care. As research continues to evolve, it is imperative that technologists and TCM practitioners collaborate to drive innovative solutions that push the boundaries of medical science and honor the profound legacy of TCM. We can chart a future course wherein AI-augmented TCM practices contribute to more systematic, effective, and accessible health care systems for all individuals. %M 38941141 %R 10.2196/58491 %U https://medinform.jmir.org/2024/1/e58491 %U https://doi.org/10.2196/58491 %U http://www.ncbi.nlm.nih.gov/pubmed/38941141 %0 Journal Article %@ 2561-7605 %I %V 7 %N %P e56345 %T The Frailty Trajectory’s Additional Edge Over the Frailty Index: Retrospective Cohort Study of Veterans With Heart Failure %A Razjouyan,Javad %A Orkaby,Ariela R %A Horstman,Molly J %A Goyal,Parag %A Intrator,Orna %A Naik,Aanand D %K gerontology %K geriatric %K geriatrics %K older adult %K older adults %K elder %K elderly %K older person %K older people %K ageing %K aging %K frailty %K frailty index %K frailty trajectory %K frail %K weak %K weakness %K heart failure %K HF %K cardiovascular disease %K CVD %K congestive heart failure %K CHF %K myocardial infarction %K MI %K unstable angina %K angina %K cardiac arrest %K atherosclerosis %K cardiology %K cardiac %K cardiologist %K cardiologists %D 2024 %7 27.6.2024 %9 %J JMIR Aging %G English %X %R 10.2196/56345 %U https://aging.jmir.org/2024/1/e56345 %U https://doi.org/10.2196/56345 %0 Journal Article %@ 2291-9694 %I %V 12 %N %P e52934 %T Data Flow Construction and Quality Evaluation of Electronic Source Data in Clinical Trials: Pilot Study Based on Hospital Electronic Medical Records in China %A Yuan,Yannan %A Mei,Yun %A Zhao,Shuhua %A Dai,Shenglong %A Liu,Xiaohong %A Sun,Xiaojing %A Fu,Zhiying %A Zhou,Liheng %A Ai,Jie %A Ma,Liheng %A Jiang,Min %K clinical trials %K electronic source data %K EHRs %K electronic data capture systems %K data quality %K electronic health records %D 2024 %7 27.6.2024 %9 %J JMIR Med Inform %G English %X Background: The traditional clinical trial data collection process requires a clinical research coordinator who is authorized by the investigators to read from the hospital’s electronic medical record. Using electronic source data opens a new path to extract patients’ data from electronic health records (EHRs) and transfer them directly to an electronic data capture (EDC) system; this method is often referred to as eSource. eSource technology in a clinical trial data flow can improve data quality without compromising timeliness. At the same time, improved data collection efficiency reduces clinical trial costs. Objective: This study aims to explore how to extract clinical trial–related data from hospital EHR systems, transform the data into a format required by the EDC system, and transfer it into sponsors’ environments, and to evaluate the transferred data sets to validate the availability, completeness, and accuracy of building an eSource dataflow. Methods: A prospective clinical trial study registered on the Drug Clinical Trial Registration and Information Disclosure Platform was selected, and the following data modules were extracted from the structured data of 4 case report forms: demographics, vital signs, local laboratory data, and concomitant medications. The extracted data was mapped and transformed, deidentified, and transferred to the sponsor’s environment. Data validation was performed based on availability, completeness, and accuracy. Results: In a secure and controlled data environment, clinical trial data was successfully transferred from a hospital EHR to the sponsor’s environment with 100% transcriptional accuracy, but the availability and completeness of the data could be improved. Conclusions: Data availability was low due to some required fields in the EDC system not being available directly in the EHR. Some data is also still in an unstructured or paper-based format. The top-level design of the eSource technology and the construction of hospital electronic data standards should help lay a foundation for a full electronic data flow from EHRs to EDC systems in the future. %R 10.2196/52934 %U https://medinform.jmir.org/2024/1/e52934 %U https://doi.org/10.2196/52934 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e59267 %T Evaluating ChatGPT-4’s Accuracy in Identifying Final Diagnoses Within Differential Diagnoses Compared With Those of Physicians: Experimental Study for Diagnostic Cases %A Hirosawa,Takanobu %A Harada,Yukinori %A Mizuta,Kazuya %A Sakamoto,Tetsu %A Tokumasu,Kazuki %A Shimizu,Taro %+ Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, 880 Kitakobayashi, Mibu-cho, Shimotsuga, Tochigi, 321-0293, Japan, 81 282861111, hirosawa@dokkyomed.ac.jp %K decision support system %K diagnostic errors %K diagnostic excellence %K diagnosis %K large language model %K LLM %K natural language processing %K GPT-4 %K ChatGPT %K diagnoses %K physicians %K artificial intelligence %K AI %K chatbots %K medical diagnosis %K assessment %K decision-making support %K application %K applications %K app %K apps %D 2024 %7 26.6.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: The potential of artificial intelligence (AI) chatbots, particularly ChatGPT with GPT-4 (OpenAI), in assisting with medical diagnosis is an emerging research area. However, it is not yet clear how well AI chatbots can evaluate whether the final diagnosis is included in differential diagnosis lists. Objective: This study aims to assess the capability of GPT-4 in identifying the final diagnosis from differential-diagnosis lists and to compare its performance with that of physicians for case report series. Methods: We used a database of differential-diagnosis lists from case reports in the American Journal of Case Reports, corresponding to final diagnoses. These lists were generated by 3 AI systems: GPT-4, Google Bard (currently Google Gemini), and Large Language Models by Meta AI 2 (LLaMA2). The primary outcome was focused on whether GPT-4’s evaluations identified the final diagnosis within these lists. None of these AIs received additional medical training or reinforcement. For comparison, 2 independent physicians also evaluated the lists, with any inconsistencies resolved by another physician. Results: The 3 AIs generated a total of 1176 differential diagnosis lists from 392 case descriptions. GPT-4’s evaluations concurred with those of the physicians in 966 out of 1176 lists (82.1%). The Cohen κ coefficient was 0.63 (95% CI 0.56-0.69), indicating a fair to good agreement between GPT-4 and the physicians’ evaluations. Conclusions: GPT-4 demonstrated a fair to good agreement in identifying the final diagnosis from differential-diagnosis lists, comparable to physicians for case report series. Its ability to compare differential diagnosis lists with final diagnoses suggests its potential to aid clinical decision-making support through diagnostic feedback. While GPT-4 showed a fair to good agreement for evaluation, its application in real-world scenarios and further validation in diverse clinical environments are essential to fully understand its utility in the diagnostic process. %M 38924784 %R 10.2196/59267 %U https://formative.jmir.org/2024/1/e59267 %U https://doi.org/10.2196/59267 %U http://www.ncbi.nlm.nih.gov/pubmed/38924784 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e55228 %T Antibiotic Prescribing by Digital Health Care Providers as Compared to Traditional Primary Health Care Providers: Cohort Study Using Register Data %A Wallman,Andy %A Svärdsudd,Kurt %A Bobits,Kent %A Wallman,Thorne %+ Department of Medical and Translational Biology, Umeå University, Biology Building (House H), Linnaeus Väg 9, Umeå, 901 87, Sweden, 46 0705500971, andy.wallman@umu.se %K telehealth prescribing %K physical-primary health care %K internet-primary health care %K antibiotics %K prescription %K infectious disease %K antibiotic %K prescriptions %K prescribing %K telehealth %K health care %K traditional %K digital %K telemedicine %K virtual care %K Swedish %K Sweden %K primary care %K quality of care %K online setting %K ePrescription %K ePrescriptions %K ePrescribing %K eHealth %K compare %K comparison %K online consultation %K digital care %K patient record %K patient records %K mobile phone %D 2024 %7 26.6.2024 %9 Original Paper %J J Med Internet Res %G English %X Background:  “Direct-to-consumer (DTC) telemedicine” is increasing worldwide and changing the map of primary health care (PHC). Virtual care has increased in the last decade and with the ongoing COVID-19 pandemic, patients’ use of online care has increased even further. In Sweden, online consultations are a part of government-supported health care today, and there are several digital care providers on the Swedish market, which makes it possible to get in touch with a doctor within a few minutes. The fast expansion of this market has raised questions about the quality of primary care provided only in an online setting without any physical appointments. Antibiotic prescribing is a common treatment in PHC. Objective:  This study aimed to compare antibiotic prescribing between digital PHC providers (internet-PHC) and traditional physical PHC providers (physical-PHC) and to determine whether prescriptions for specific diagnoses differed between internet-PHC and physical-PHC appointments, adjusted for the effects of attained age at the time of appointment, gender, and time relative to the COVID-19 pandemic. Methods:  Antibiotic prescribing data based on Anatomical Therapeutic Chemical (ATC) codes were obtained for Region Sörmland residents from January 2020 until March 2021 from the Regional Administrative Office. In total, 160,238 appointments for 68,332 Sörmland residents were included (124,398 physical-PHC and 35,840 internet-PHC appointments). Prescriptions issued by internet-PHC or physical-PHC physicians were considered. Information on the appointment date, staff category serving the patient, ICD-10 (International Statistical Classification of Diseases, Tenth Revision) diagnosis codes, ATC codes of prescribed medicines, and patient-attained age and gender were used. Results:  A total of 160,238 health care appointments were registered, of which 18,433 led to an infection diagnosis. There were large differences in gender and attained age distributions among physical-PHC and internet-PHC appointments. Physical-PHC appointments peaked among patients aged 60-80 years while internet-PHC appointments peaked at 20-30 years of age for both genders. Antibiotics with the ATC codes J01A-J01X were prescribed in 9.3% (11,609/124,398) of physical-PHC appointments as compared with 6.1% (2201/35,840) of internet-PHC appointments. In addition, 61.3% (6412/10,454) of physical-PHC infection appointments resulted in antibiotic prescriptions, as compared with only 25.8% (2057/7979) of internet-PHC appointments. Analyses of the prescribed antibiotics showed that internet-PHC followed regional recommendations for all diagnoses. Physical-PHC also followed the recommendations but used a wider spectrum of antibiotics. The odds ratio of receiving an antibiotic prescription (after adjustments for attained age at the time of appointment, patient gender, and whether the prescription was issued before or during the COVID-19 pandemic) during an internet-PHC appointment was 0.23-0.39 as compared with a physical-PHC appointment. Conclusions:  Internet-PHC appointments resulted in a significantly lower number of antibiotics prescriptions than physical-PHC appointments, adjusted for the large differences in the characteristics of patients who consult internet-PHC and physical-PHC. Internet-PHC prescribers showed appropriate prescribing according to guidelines. %M 38924783 %R 10.2196/55228 %U https://www.jmir.org/2024/1/e55228 %U https://doi.org/10.2196/55228 %U http://www.ncbi.nlm.nih.gov/pubmed/38924783 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e56241 %T Clinical Simulation in the Regulation of Software as a Medical Device: An eDelphi Study %A O'Driscoll,Fiona %A O'Brien,Niki %A Guo,Chaohui %A Prime,Matthew %A Darzi,Ara %A Ghafur,Saira %+ Institute of Global Health Innovation, Imperial College London, Room 1035, Queen Elizabeth Queen Mother Wing, St Mary's Campus, South Wharf Road, London, W2 1NY, United Kingdom, 44 020 7594 1419, saira.ghafur13@imperial.ac.uk %K digital health technology %K software as a medical device %K clinical simulation %K Delphi study %K eDelphi study %K artificial intelligence %K digital health %D 2024 %7 25.6.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Accelerated digitalization in the health sector requires the development of appropriate evaluation methods to ensure that digital health technologies (DHTs) are safe and effective. Software as a medical device (SaMD) is a commonly used DHT by clinicians to provide care to patients. Traditional research methods for evaluating health care products, such as randomized clinical trials, may not be suitable for DHTs, such as SaMD. However, evidence to show their safety and efficacy is needed by regulators before they can be used in practice. Clinical simulation can be used by researchers to test SaMD in an agile and low-cost way; yet, there is limited research on criteria to assess the robustness of simulations and, subsequently, their relevance for a regulatory decision. Objective: The objective of this study was to gain consensus on the criteria that should be used to assess clinical simulation from a regulatory perspective when it is used to generate evidence for SaMD. Methods: An eDelphi study approach was chosen to develop a set of criteria to assess clinical simulation when used to evaluate SaMD. Participants were recruited through purposive and snowball sampling based on their experience and knowledge in relevant sectors. They were guided through an initial scoping questionnaire with key themes identified from the literature to obtain a comprehensive list of criteria. Participants voted upon these criteria in 2 Delphi rounds, with criteria being excluded if consensus was not met. Participants were invited to add qualitative comments during rounds and qualitative analysis was performed on the comments gathered during the first round. Consensus was predefined by 2 criteria: if <10% of the panelists deemed the criteria as “not important” or “not important at all” and >60% “important” or “very important.” Results: In total, 33 international experts in the digital health field, including academics, regulators, policy makers, and industry representatives, completed both Delphi rounds, and 43 criteria gained consensus from the participants. The research team grouped these criteria into 7 domains—background and context, overall study design, study population, delivery of the simulation, fidelity, software and artificial intelligence, and study analysis. These 7 domains were formulated into the simulation for regulation of SaMD framework. There were key areas of concern identified by participants regarding the framework criteria, such as the importance of how simulation fidelity is achieved and reported and the avoidance of bias throughout all stages. Conclusions: This study proposes the simulation for regulation of SaMD framework, developed through an eDelphi consensus process, to evaluate clinical simulation when used to assess SaMD. Future research should prioritize the development of safe and effective SaMD, while implementing and refining the framework criteria to adapt to new challenges. %M 38917454 %R 10.2196/56241 %U https://formative.jmir.org/2024/1/e56241 %U https://doi.org/10.2196/56241 %U http://www.ncbi.nlm.nih.gov/pubmed/38917454 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e54265 %T Making Science Computable Using Evidence-Based Medicine on Fast Healthcare Interoperability Resources: Standards Development Project %A Soares,Andrey %A Schilling,Lisa M %A Richardson,Joshua %A Kommadi,Bhagvan %A Subbian,Vignesh %A Dehnbostel,Joanne %A Shahin,Khalid %A Robinson,Karen A %A Afzal,Muhammad %A Lehmann,Harold P %A Kunnamo,Ilkka %A Alper,Brian S %+ Department of Medicine, University of Colorado Anschutz Medical Campus, 1890 North Revere Court, Mailstop F443, Aurora, CO, 80045, United States, 1 3037242825, andrey.soares@cuanschutz.edu %K evidence-based medicine %K FHIR %K Fast Healthcare Interoperability Resources %K computable evidence %K EBMonFHIR %K evidence-based medicine on Fast Healthcare Interoperability Resources %D 2024 %7 25.6.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Evidence-based medicine (EBM) has the potential to improve health outcomes, but EBM has not been widely integrated into the systems used for research or clinical decision-making. There has not been a scalable and reusable computer-readable standard for distributing research results and synthesized evidence among creators, implementers, and the ultimate users of that evidence. Evidence that is more rapidly updated, synthesized, disseminated, and implemented would improve both the delivery of EBM and evidence-based health care policy. Objective: This study aimed to introduce the EBM on Fast Healthcare Interoperability Resources (FHIR) project (EBMonFHIR), which is extending the methods and infrastructure of Health Level Seven (HL7) FHIR to provide an interoperability standard for the electronic exchange of health-related scientific knowledge. Methods: As an ongoing process, the project creates and refines FHIR resources to represent evidence from clinical studies and syntheses of those studies and develops tools to assist with the creation and visualization of FHIR resources. Results: The EBMonFHIR project created FHIR resources (ie, ArtifactAssessment, Citation, Evidence, EvidenceReport, and EvidenceVariable) for representing evidence. The COVID-19 Knowledge Accelerator (COKA) project, now Health Evidence Knowledge Accelerator (HEvKA), took this work further and created FHIR resources that express EvidenceReport, Citation, and ArtifactAssessment concepts. The group is (1) continually refining FHIR resources to support the representation of EBM; (2) developing controlled terminology related to EBM (ie, study design, statistic type, statistical model, and risk of bias); and (3) developing tools to facilitate the visualization and data entry of EBM information into FHIR resources, including human-readable interfaces and JSON viewers. Conclusions: EBMonFHIR resources in conjunction with other FHIR resources can support relaying EBM components in a manner that is interoperable and consumable by downstream tools and health information technology systems to support the users of evidence. %M 38916936 %R 10.2196/54265 %U https://www.jmir.org/2024/1/e54265 %U https://doi.org/10.2196/54265 %U http://www.ncbi.nlm.nih.gov/pubmed/38916936 %0 Journal Article %@ 2291-9694 %I %V 12 %N %P e50194 %T Implementing a Biomedical Data Warehouse From Blueprint to Bedside in a Regional French University Hospital Setting: Unveiling Processes, Overcoming Challenges, and Extracting Clinical Insight %A Karakachoff,Matilde %A Goronflot,Thomas %A Coudol,Sandrine %A Toublant,Delphine %A Bazoge,Adrien %A Constant Dit Beaufils,Pacôme %A Varey,Emilie %A Leux,Christophe %A Mauduit,Nicolas %A Wargny,Matthieu %A Gourraud,Pierre-Antoine %K data warehouse %K biomedical data warehouse %K clinical data repository %K electronic health records %K data reuse %K secondary use %K clinical routine data %K real-world data %K implementation report %D 2024 %7 24.6.2024 %9 %J JMIR Med Inform %G English %X Background: Biomedical data warehouses (BDWs) have become an essential tool to facilitate the reuse of health data for both research and decisional applications. Beyond technical issues, the implementation of BDWs requires strong institutional data governance and operational knowledge of the European and national legal framework for the management of research data access and use. Objective: In this paper, we describe the compound process of implementation and the contents of a regional university hospital BDW. Methods: We present the actions and challenges regarding organizational changes, technical architecture, and shared governance that took place to develop the Nantes BDW. We describe the process to access clinical contents, give details about patient data protection, and use examples to illustrate merging clinical insights. Implementation (Results): More than 68 million textual documents and 543 million pieces of coded information concerning approximately 1.5 million patients admitted to CHUN between 2002 and 2022 can be queried and transformed to be made available to investigators. Since its creation in 2018, 269 projects have benefited from the Nantes BDW. Access to data is organized according to data use and regulatory requirements. Conclusions: Data use is entirely determined by the scientific question posed. It is the vector of legitimacy of data access for secondary use. Enabling access to a BDW is a game changer for research and all operational situations in need of data. Finally, data governance must prevail over technical issues in institution data strategy vis-à-vis care professionals and patients alike. %R 10.2196/50194 %U https://medinform.jmir.org/2024/1/e50194 %U https://doi.org/10.2196/50194 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e53162 %T Multicentric Assessment of a Multimorbidity-Adjusted Disability Score to Stratify Depression-Related Risks Using Temporal Disease Maps: Instrument Validation Study %A González-Colom,Rubèn %A Mitra,Kangkana %A Vela,Emili %A Gezsi,Andras %A Paajanen,Teemu %A Gál,Zsófia %A Hullam,Gabor %A Mäkinen,Hannu %A Nagy,Tamas %A Kuokkanen,Mikko %A Piera-Jiménez,Jordi %A Roca,Josep %A Antal,Peter %A Juhasz,Gabriella %A Cano,Isaac %+ Fundació de Recerca Clínic Barcelona - Institut d’Investigacions Biomèdiques August Pi i Sunyer, C/Rosselló 149-153, Barcelona, 08036, Spain, 34 932275707, rgonzalezc@recerca.clinic.cat %K health risk assessment %K multimorbidity %K disease trajectories %K major depressive disorder %D 2024 %7 24.6.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Comprehensive management of multimorbidity can significantly benefit from advanced health risk assessment tools that facilitate value-based interventions, allowing for the assessment and prediction of disease progression. Our study proposes a novel methodology, the Multimorbidity-Adjusted Disability Score (MADS), which integrates disease trajectory methodologies with advanced techniques for assessing interdependencies among concurrent diseases. This approach is designed to better assess the clinical burden of clusters of interrelated diseases and enhance our ability to anticipate disease progression, thereby potentially informing targeted preventive care interventions. Objective: This study aims to evaluate the effectiveness of the MADS in stratifying patients into clinically relevant risk groups based on their multimorbidity profiles, which accurately reflect their clinical complexity and the probabilities of developing new associated disease conditions. Methods: In a retrospective multicentric cohort study, we developed the MADS by analyzing disease trajectories and applying Bayesian statistics to determine disease-disease probabilities combined with well-established disability weights. We used major depressive disorder (MDD) as a primary case study for this evaluation. We stratified patients into different risk levels corresponding to different percentiles of MADS distribution. We statistically assessed the association of MADS risk strata with mortality, health care resource use, and disease progression across 1 million individuals from Spain, the United Kingdom, and Finland. Results: The results revealed significantly different distributions of the assessed outcomes across the MADS risk tiers, including mortality rates; primary care visits; specialized care outpatient consultations; visits in mental health specialized centers; emergency room visits; hospitalizations; pharmacological and nonpharmacological expenditures; and dispensation of antipsychotics, anxiolytics, sedatives, and antidepressants (P<.001 in all cases). Moreover, the results of the pairwise comparisons between adjacent risk tiers illustrate a substantial and gradual pattern of increased mortality rate, heightened health care use, increased health care expenditures, and a raised pharmacological burden as individuals progress from lower MADS risk tiers to higher-risk tiers. The analysis also revealed an augmented risk of multimorbidity progression within the high-risk groups, aligned with a higher incidence of new onsets of MDD-related diseases. Conclusions: The MADS seems to be a promising approach for predicting health risks associated with multimorbidity. It might complement current risk assessment state-of-the-art tools by providing valuable insights for tailored epidemiological impact analyses of clusters of interrelated diseases and by accurately assessing multimorbidity progression risks. This study paves the way for innovative digital developments to support advanced health risk assessment strategies. Further validation is required to generalize its use beyond the initial case study of MDD. %M 38913991 %R 10.2196/53162 %U https://www.jmir.org/2024/1/e53162 %U https://doi.org/10.2196/53162 %U http://www.ncbi.nlm.nih.gov/pubmed/38913991 %0 Journal Article %@ 2562-7600 %I JMIR Publications %V 7 %N %P e55793 %T A Scalable and Extensible Logical Data Model of Electronic Health Record Audit Logs for Temporal Data Mining (RNteract): Model Conceptualization and Formulation %A Tiase,Victoria L %A Sward,Katherine A %A Facelli,Julio C %+ Department of Biomedical Informatics, University of Utah, 421 Wakara Way, Salt Lake City, UT, 84108, United States, 1 801 585 3945, victoria.tiase@utah.edu %K burnout %K professional %K nursing %K nurse %K electronic health record %K EHR %K data modeling %K data set %K temporal machine learning %K machine learning %K ML %K artificial intelligence %K AI %K algorithm %K predictive model %K predictive analytics %K practical model %D 2024 %7 24.6.2024 %9 Original Paper %J JMIR Nursing %G English %X Background: Increased workload, including workload related to electronic health record (EHR) documentation, is reported as a main contributor to nurse burnout and adversely affects patient safety and nurse satisfaction. Traditional methods for workload analysis are either administrative measures (such as the nurse-patient ratio) that do not represent actual nursing care or are subjective and limited to snapshots of care (eg, time-motion studies). Observing care and testing workflow changes in real time can be obstructive to clinical care. An examination of EHR interactions using EHR audit logs could provide a scalable, unobtrusive way to quantify the nursing workload, at least to the extent that nursing work is represented in EHR documentation. EHR audit logs are extremely complex; however, simple analytical methods cannot discover complex temporal patterns, requiring use of state-of-the-art temporal data-mining approaches. To effectively use these approaches, it is necessary to structure the raw audit logs into a consistent and scalable logical data model that can be consumed by machine learning (ML) algorithms. Objective: We aimed to conceptualize a logical data model for nurse-EHR interactions that would support the future development of temporal ML models based on EHR audit log data. Methods: We conducted a preliminary review of EHR audit logs to understand the types of nursing-specific data captured. Using concepts derived from the literature and our previous experience studying temporal patterns in biomedical data, we formulated a logical data model that can describe nurse-EHR interactions, the nurse-intrinsic and situational characteristics that may influence those interactions, and outcomes of relevance to the nursing workload in a scalable and extensible manner. Results: We describe the data structure and concepts from EHR audit log data associated with nursing workload as a logical data model named RNteract. We conceptually demonstrate how using this logical data model could support temporal unsupervised ML and state-of-the-art artificial intelligence (AI) methods for predictive modeling. Conclusions: The RNteract logical data model appears capable of supporting a variety of AI-based systems and should be generalizable to any type of EHR system or health care setting. Quantitatively identifying and analyzing temporal patterns of nurse-EHR interactions is foundational for developing interventions that support the nursing documentation workload and address nurse burnout. %M 38913994 %R 10.2196/55793 %U https://nursing.jmir.org/2024/1/e55793 %U https://doi.org/10.2196/55793 %U http://www.ncbi.nlm.nih.gov/pubmed/38913994 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e49613 %T Dermoscopy Differential Diagnosis Explorer (D3X) Ontology to Aggregate and Link Dermoscopic Patterns to Differential Diagnoses: Development and Usability Study %A Lin,Rebecca Z %A Amith,Muhammad Tuan %A Wang,Cynthia X %A Strickley,John %A Tao,Cui %+ Department of Artificial Intelligence and Informatics, Mayo Clinic, 4500 San Pablo Road, Jacksonville, FL, 32224, United States, 1 9049530255, Tao.Cui@mayo.edu %K medical informatics %K biomedical ontology %K ontology %K ontologies %K vocabulary %K OWL %K web ontology language %K skin %K semiotic %K web app %K web application %K visual %K visualization %K dermoscopic %K diagnosis %K diagnoses %K diagnostic %K information storage %K information retrieval %K skin lesion %K skin diseases %K dermoscopy differential diagnosis explorer %K dermatology %K dermoscopy %K differential diagnosis %K information storage and retrieval %D 2024 %7 21.6.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Dermoscopy is a growing field that uses microscopy to allow dermatologists and primary care physicians to identify skin lesions. For a given skin lesion, a wide variety of differential diagnoses exist, which may be challenging for inexperienced users to name and understand. Objective: In this study, we describe the creation of the dermoscopy differential diagnosis explorer (D3X), an ontology linking dermoscopic patterns to differential diagnoses. Methods: Existing ontologies that were incorporated into D3X include the elements of visuals ontology and dermoscopy elements of visuals ontology, which connect visual features to dermoscopic patterns. A list of differential diagnoses for each pattern was generated from the literature and in consultation with domain experts. Open-source images were incorporated from DermNet, Dermoscopedia, and open-access research papers. Results: D3X was encoded in the OWL 2 web ontology language and includes 3041 logical axioms, 1519 classes, 103 object properties, and 20 data properties. We compared D3X with publicly available ontologies in the dermatology domain using a semiotic theory–driven metric to measure the innate qualities of D3X with others. The results indicate that D3X is adequately comparable with other ontologies of the dermatology domain. Conclusions: The D3X ontology is a resource that can link and integrate dermoscopic differential diagnoses and supplementary information with existing ontology-based resources. Future directions include developing a web application based on D3X for dermoscopy education and clinical practice. %M 38904996 %R 10.2196/49613 %U https://medinform.jmir.org/2024/1/e49613 %U https://doi.org/10.2196/49613 %U http://www.ncbi.nlm.nih.gov/pubmed/38904996 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 10 %N %P e54250 %T Development of a Consolidated Health Facility Masterlist Using Data From Polio Electronic Surveillance in the World Health Organization African Region %A Babona Nshuti,Marie Aimee %A Touray,Kebba %A Muluh,Ticha Johnson %A Ubong,Godwin Akpan %A Ngofa,Reuben Opara %A Mohammed,Bello Isa %A Roselyne,Ishimwe %A Oviaesu,David %A Bakata,Evans Mawa Oliver %A Lau,Fiona %A Kipterer,John %A Green,Hugh Henry W %A Seaman,Vincent %A Ahmed,Jamal A %A Ndoutabe,Modjirom %+ World Health Organization Regional Office for Africa, Cite du Djoué, Bacongo, PO Box 06, Brazzaville, Congo, 242 069248040, shutaime03@gmail.com %K African region %K electronic surveillance %K geographic information systems %K Global Polio Eradication Initiative %K integrated supportive supervision %K polio %D 2024 %7 21.6.2024 %9 Viewpoint %J JMIR Public Health Surveill %G English %X Geospatial data reporting from surveillance and immunization efforts is a key aspect of the World Health Organization (WHO) Global Polio Eradication Initiative in Africa. These activities are coordinated through the WHO Regional Office for Africa Geographic Information Systems Centre. To ensure the accuracy of field-collected data, the WHO Regional Office for Africa Geographic Information Systems Centre has developed mobile phone apps such as electronic surveillance (eSURV) and integrated supportive supervision (ISS) geospatial data collection programs. While eSURV and ISS have played a vital role in efforts to eradicate polio and control other communicable diseases in Africa, disease surveillance efforts have been hampered by incomplete and inaccurate listings of health care sites throughout the continent. To address this shortcoming, data compiled from eSURV and ISS are being used to develop, update, and validate a Health Facility master list for the WHO African region that contains comprehensive listings of the names, locations, and types of health facilities in each member state. The WHO and Ministry of Health field officers are responsible for documenting and transmitting the relevant geospatial location information regarding health facilities and traditional medicine sites using the eSURV and ISS form; this information is then used to update the Health Facility master list and is also made available to national ministries of health to update their respective health facility lists. This consolidation of health facility information into a single registry is expected to improve disease surveillance and facilitate epidemiologic research for the Global Polio Eradication Initiative, as well as aid public health efforts directed at other diseases across the African continent. This review examines active surveillance using eSURV at the district, country, and regional levels, highlighting its role in supporting polio surveillance and immunization efforts, as well as its potential to serve as a fundamental basis for broader public health initiatives and research throughout Africa. %M 38904997 %R 10.2196/54250 %U https://publichealth.jmir.org/2024/1/e54250 %U https://doi.org/10.2196/54250 %U http://www.ncbi.nlm.nih.gov/pubmed/38904997 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e46691 %T Effect of Digital Early Warning Scores on Hospital Vital Sign Observation Protocol Adherence: Stepped-Wedge Evaluation %A Wong,David Chi-Wai %A Bonnici,Timothy %A Gerry,Stephen %A Birks,Jacqueline %A Watkinson,Peter J %+ Leeds Institute of Health Sciences, School of Medicine, University of Leeds, Level 10, Worsley Building, Leeds, LS2 9JT, United Kingdom, 44 113 343 0806, d.c.wong@leeds.ac.uk %K vital signs %K early warning score %K track and trigger %K electronic charting %K stepped-wedge %K vital %K charting %K documentation %K deterioration %K hospital management %K clinical intervention %K decision-making %K patient risk %K hospital %K ICU %K intensive care unit %K UK %K United Kingdom %K intervention %D 2024 %7 20.6.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Early warning scores (EWS) are routinely used in hospitals to assess a patient’s risk of deterioration. EWS are traditionally recorded on paper observation charts but are increasingly recorded digitally. In either case, evidence for the clinical effectiveness of such scores is mixed, and previous studies have not considered whether EWS leads to changes in how deteriorating patients are managed. Objective: This study aims to examine whether the introduction of a digital EWS system was associated with more frequent observation of patients with abnormal vital signs, a precursor to earlier clinical intervention. Methods: We conducted a 2-armed stepped-wedge study from February 2015 to December 2016, over 4 hospitals in 1 UK hospital trust. In the control arm, vital signs were recorded using paper observation charts. In the intervention arm, a digital EWS system was used. The primary outcome measure was time to next observation (TTNO), defined as the time between a patient’s first elevated EWS (EWS ≥3) and subsequent observations set. Secondary outcomes were time to death in the hospital, length of stay, and time to unplanned intensive care unit admission. Differences between the 2 arms were analyzed using a mixed-effects Cox model. The usability of the system was assessed using the system usability score survey. Results: We included 12,802 admissions, 1084 in the paper (control) arm and 11,718 in the digital EWS (intervention) arm. The system usability score was 77.6, indicating good usability. The median TTNO in the control and intervention arms were 128 (IQR 73-218) minutes and 131 (IQR 73-223) minutes, respectively. The corresponding hazard ratio for TTNO was 0.99 (95% CI 0.91-1.07; P=.73). Conclusions: We demonstrated strong clinical engagement with the system. We found no difference in any of the predefined patient outcomes, suggesting that the introduction of a highly usable electronic system can be achieved without impacting clinical care. Our findings contrast with previous claims that digital EWS systems are associated with improvement in clinical outcomes. Future research should investigate how digital EWS systems can be integrated with new clinical pathways adjusting staff behaviors to improve patient outcomes. %M 38900529 %R 10.2196/46691 %U https://www.jmir.org/2024/1/e46691 %U https://doi.org/10.2196/46691 %U http://www.ncbi.nlm.nih.gov/pubmed/38900529 %0 Journal Article %@ 2291-9694 %I %V 12 %N %P e49978 %T Effect of Implementing an Informatization Case Management Model on the Management of Chronic Respiratory Diseases in a General Hospital: Retrospective Controlled Study %A Xiao,Yi-Zhen %A Chen,Xiao-Jia %A Sun,Xiao-Ling %A Chen,Huan %A Luo,Yu-Xia %A Chen,Yuan %A Liang,Ye-Mei %K chronic disease management %K chronic respiratory disease %K hospital information system %K informatization %K information system %K respiratory %K pulmonary %K breathing %K implementation %K care management %K disease management %K chronic obstructive pulmonary disease %K case management %D 2024 %7 19.6.2024 %9 %J JMIR Med Inform %G English %X Background: The use of chronic disease information systems in hospitals and communities plays a significant role in disease prevention, control, and monitoring. However, there are several limitations to these systems, including that the platforms are generally isolated, the patient health information and medical resources are not effectively integrated, and the “Internet Plus Healthcare” technology model is not implemented throughout the patient consultation process. Objective: The aim of this study was to evaluate the efficiency of the application of a hospital case management information system in a general hospital in the context of chronic respiratory diseases as a model case. Methods: A chronic disease management information system was developed for use in general hospitals based on internet technology, a chronic disease case management model, and an overall quality management model. Using this system, the case managers provided sophisticated inpatient, outpatient, and home medical services for patients with chronic respiratory diseases. Chronic respiratory disease case management quality indicators (number of managed cases, number of patients accepting routine follow-up services, follow-up visit rate, pulmonary function test rate, admission rate for acute exacerbations, chronic respiratory diseases knowledge awareness rate, and patient satisfaction) were evaluated before (2019‐2020) and after (2021‐2022) implementation of the chronic disease management information system. Results: Before implementation of the chronic disease management information system, 1808 cases were managed in the general hospital, and an average of 603 (SD 137) people were provided with routine follow-up services. After use of the information system, 5868 cases were managed and 2056 (SD 211) patients were routinely followed-up, representing a significant increase of 3.2 and 3.4 times the respective values before use (U=342.779; P<.001). With respect to the quality of case management, compared to the indicators measured before use, the achievement rate of follow-up examination increased by 50.2%, the achievement rate of the pulmonary function test increased by 26.2%, the awareness rate of chronic respiratory disease knowledge increased by 20.1%, the retention rate increased by 16.3%, and the patient satisfaction rate increased by 9.6% (all P<.001), while the admission rate of acute exacerbation decreased by 42.4% (P<.001) after use of the chronic disease management information system. Conclusions: Use of a chronic disease management information system improves the quality of chronic respiratory disease case management and reduces the admission rate of patients owing to acute exacerbations of their diseases. %R 10.2196/49978 %U https://medinform.jmir.org/2024/1/e49978 %U https://doi.org/10.2196/49978 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e50209 %T Retrieval-Based Diagnostic Decision Support: Mixed Methods Study %A Abdullahi,Tassallah %A Mercurio,Laura %A Singh,Ritambhara %A Eickhoff,Carsten %+ School of Medicine, University of Tübingen, Schaffhausenstr, 77, Tübingen, 72072, Germany, 49 7071 29 843, carsten.eickhoff@uni-tuebingen.de %K clinical decision support %K rare diseases %K ensemble learning %K retrieval-augmented learning %K machine learning %K electronic health records %K natural language processing %K retrieval augmented generation %K RAG %K electronic health record %K EHR %K data sparsity %K information retrieval %D 2024 %7 19.6.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Diagnostic errors pose significant health risks and contribute to patient mortality. With the growing accessibility of electronic health records, machine learning models offer a promising avenue for enhancing diagnosis quality. Current research has primarily focused on a limited set of diseases with ample training data, neglecting diagnostic scenarios with limited data availability. Objective: This study aims to develop an information retrieval (IR)–based framework that accommodates data sparsity to facilitate broader diagnostic decision support. Methods: We introduced an IR-based diagnostic decision support framework called CliniqIR. It uses clinical text records, the Unified Medical Language System Metathesaurus, and 33 million PubMed abstracts to classify a broad spectrum of diagnoses independent of training data availability. CliniqIR is designed to be compatible with any IR framework. Therefore, we implemented it using both dense and sparse retrieval approaches. We compared CliniqIR’s performance to that of pretrained clinical transformer models such as Clinical Bidirectional Encoder Representations from Transformers (ClinicalBERT) in supervised and zero-shot settings. Subsequently, we combined the strength of supervised fine-tuned ClinicalBERT and CliniqIR to build an ensemble framework that delivers state-of-the-art diagnostic predictions. Results: On a complex diagnosis data set (DC3) without any training data, CliniqIR models returned the correct diagnosis within their top 3 predictions. On the Medical Information Mart for Intensive Care III data set, CliniqIR models surpassed ClinicalBERT in predicting diagnoses with <5 training samples by an average difference in mean reciprocal rank of 0.10. In a zero-shot setting where models received no disease-specific training, CliniqIR still outperformed the pretrained transformer models with a greater mean reciprocal rank of at least 0.10. Furthermore, in most conditions, our ensemble framework surpassed the performance of its individual components, demonstrating its enhanced ability to make precise diagnostic predictions. Conclusions: Our experiments highlight the importance of IR in leveraging unstructured knowledge resources to identify infrequently encountered diagnoses. In addition, our ensemble framework benefits from combining the complementary strengths of the supervised and retrieval-based models to diagnose a broad spectrum of diseases. %M 38896468 %R 10.2196/50209 %U https://medinform.jmir.org/2024/1/e50209 %U https://doi.org/10.2196/50209 %U http://www.ncbi.nlm.nih.gov/pubmed/38896468 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e54518 %T Key Considerations for Designing Clinical Studies to Evaluate Digital Health Solutions %A Bolinger,Elaina %A Tyl,Benoit %+ Integrated Evidence Generation & Business Innovation, Bayer HealthCare SAS, 10 Place de Belgique, La Garenne Colombes, F-92254, France, 33 6 80 29 07 79, benoit.tyl@bayer.com %K evidence generation %K clinical robustness %K clinical trials %K digital health %K solutions %K digital health solutions %K DHS %K health care system %K patients %K patient %K individuals %K individual %K healthcare system %K control arm adaptations %K randomization methods %K real-world data %K platform research %D 2024 %7 17.6.2024 %9 Viewpoint %J J Med Internet Res %G English %X Evidence of clinical impact is critical to unlock the potential of digital health solutions (DHSs), yet many solutions are failing to deliver positive clinical results. We argue in this viewpoint that this failure is linked to current approaches to DHS evaluation design, which neglect numerous key characteristics (KCs) requiring specific scientific and design considerations. We first delineate the KCs of DHSs: (1) they are implemented at health care system and patient levels; (2) they are “complex” interventions; (3) they can drive multiple clinical outcomes indirectly through a multitude of smaller clinical benefits; (4) their mechanism of action can vary between individuals and change over time based on patient needs; and (5) they develop through short, iterative cycles—optimally within a real-world use context. Following our objective to drive better alignment between clinical evaluation design and the unique traits of DHSs, we then provide methodological suggestions that better address these KCs, including tips on mechanism-of-action mapping, alternative randomization methods, control-arm adaptations, and novel end-point selection, as well as innovative methods utilizing real-world data and platform research. %M 38885020 %R 10.2196/54518 %U https://www.jmir.org/2024/1/e54518 %U https://doi.org/10.2196/54518 %U http://www.ncbi.nlm.nih.gov/pubmed/38885020 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e54910 %T Communication and Contextual Factors in Robotic-Assisted Surgical Teams: Protocol for Developing a Taxonomy %A Nyein,Kyi Phyu %A Condron,Claire %+ SIM Centre for Simulation Education and Research, Royal College of Surgeons in Ireland, University of Medicine and Health Sciences, 123 St Stephen’s Green, Dublin 2, Dublin, D02 YN77, Ireland, 353 (01) 402 2100, kyiphyunyein@rcsi.ie %K communication %K teams %K robotic surgery %K robotic-assisted %K simulation %D 2024 %7 17.6.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Robotic-assisted surgery (RAS) has been rapidly integrated into surgical practice in the past few decades. The setup in the operating theater for RAS differs from that for open or laparoscopic surgery such that the operating surgeon sits at a console separate from the rest of the surgical team and the patient. Communication and team dynamics are altered due to this physical separation and visual barriers imposed by the robotic equipment. However, the factors that might comprise effective communication among members of RAS teams and the contextual factors that facilitate or inhibit effective communication in RAS remain unknown. Objective: We aim to develop a taxonomy of communication behaviors and contextual factors that influence communication in RAS teams. We also aim to examine the patterns of communication behaviors based on gender. Methods: We will first perform a scoping review on communication in RAS to develop a preliminary taxonomy of communication based on the existing literature. We will then conduct semistructured interviews with RAS team members, including the surgeon, assisting surgeon or trainee, bedside or first assistant, nurses, and anesthetists. Participants will represent different disciplines, including urology, general surgery, and gynecology, and have a range of experiences in RAS. We will use a reflexive thematic analysis to analyze the data and further refine the taxonomy. We will also observe live robotic surgeries at Royal College of Surgeons in Ireland (RCSI)–affiliated hospitals. We will observe varying lengths and conditions of RAS procedures to a capture a wide range of communication behaviors and contextual factors to help finalize the taxonomy. Although we anticipate conducting 30 interviews and 30 observations, we will collect data until we achieve data sufficiency. We will conduct data collection in parallel with data analysis such that if we identify a new behavior in an interview, we will follow up with questions related to that behavior in additional interviews and/or observations. Results: The taxonomy from this project will include a list of actionable communication behaviors, contextual factors, their descriptions, and examples. As of May 2024, this project has been approved by the RCSI Research and Ethics Committee. Data collection started in June 2024 and will continue throughout the year. We plan to publish the findings as meaningful results emerge in our data analysis in 2024 and 2025. Conclusions: The results from this project will be used to observe and train surgical teams in a simulated environment to effectively communicate with each other and prevent communication breakdowns. The developed taxonomy will also add to the knowledge base on the role of gender in communication in RAS and produce recommendations that can be incorporated into training. Overall, this project will contribute to the improvement of communication skills of surgical teams and the quality and safety of patient care. International Registered Report Identifier (IRRID): PRR1-10.2196/54910 %M 38885018 %R 10.2196/54910 %U https://www.researchprotocols.org/2024/1/e54910 %U https://doi.org/10.2196/54910 %U http://www.ncbi.nlm.nih.gov/pubmed/38885018 %0 Journal Article %@ 2563-3570 %I JMIR Publications %V 5 %N %P e55632 %T It Is in Our DNA: Bringing Electronic Health Records and Genomic Data Together for Precision Medicine %A Robertson,Alan J %A Mallett,Andrew J %A Stark,Zornitza %A Sullivan,Clair %+ Queensland Digital Health Centre, University of Queensland, Health Sciences Building, Herston Campus, Royal Brisbane and Women's Hospital, Brisbane, 4029, Australia, 61 733465343, c.sullivan1@uq.edu.au %K genomics %K digital health %K genetics %K precision medicine %K genomic %K genomic data %K electronic health records %K DNA %K supports %K decision-making %K timeliness %K diagnosis %K risk reduction %K electronic medical records %D 2024 %7 13.6.2024 %9 Viewpoint %J JMIR Bioinform Biotech %G English %X Health care is at a turning point. We are shifting from protocolized medicine to precision medicine, and digital health systems are facilitating this shift. By providing clinicians with detailed information for each patient and analytic support for decision-making at the point of care, digital health technologies are enabling a new era of precision medicine. Genomic data also provide clinicians with information that can improve the accuracy and timeliness of diagnosis, optimize prescribing, and target risk reduction strategies, all of which are key elements for precision medicine. However, genomic data are predominantly seen as diagnostic information and are not routinely integrated into the clinical workflows of electronic medical records. The use of genomic data holds significant potential for precision medicine; however, as genomic data are fundamentally different from the information collected during routine practice, special considerations are needed to use this information in a digital health setting. This paper outlines the potential of genomic data integration with electronic records, and how these data can enable precision medicine. %M 38935958 %R 10.2196/55632 %U https://bioinform.jmir.org/2024/1/e55632 %U https://doi.org/10.2196/55632 %U http://www.ncbi.nlm.nih.gov/pubmed/38935958 %0 Journal Article %@ 2563-6316 %I %V 5 %N %P e45973 %T Performance Drift in Machine Learning Models for Cardiac Surgery Risk Prediction: Retrospective Analysis %A Dong,Tim %A Sinha,Shubhra %A Zhai,Ben %A Fudulu,Daniel %A Chan,Jeremy %A Narayan,Pradeep %A Judge,Andy %A Caputo,Massimo %A Dimagli,Arnaldo %A Benedetto,Umberto %A Angelini,Gianni D %K cardiac surgery %K artificial intelligence %K risk prediction %K machine learning %K operative mortality %K data set drift %K performance drift %K national data set %K adult %K data %K cardiac %K surgery %K cardiology %K heart %K risk %K prediction %K United Kingdom %K mortality %K performance %K model %D 2024 %7 12.6.2024 %9 %J JMIRx Med %G English %X Background: The Society of Thoracic Surgeons and European System for Cardiac Operative Risk Evaluation (EuroSCORE) II risk scores are the most commonly used risk prediction models for in-hospital mortality after adult cardiac surgery. However, they are prone to miscalibration over time and poor generalization across data sets; thus, their use remains controversial. Despite increased interest, a gap in understanding the effect of data set drift on the performance of machine learning (ML) over time remains a barrier to its wider use in clinical practice. Data set drift occurs when an ML system underperforms because of a mismatch between the data it was developed from and the data on which it is deployed. Objective: In this study, we analyzed the extent of performance drift using models built on a large UK cardiac surgery database. The objectives were to (1) rank and assess the extent of performance drift in cardiac surgery risk ML models over time and (2) investigate any potential influence of data set drift and variable importance drift on performance drift. Methods: We conducted a retrospective analysis of prospectively, routinely gathered data on adult patients undergoing cardiac surgery in the United Kingdom between 2012 and 2019. We temporally split the data 70:30 into a training and validation set and a holdout set. Five novel ML mortality prediction models were developed and assessed, along with EuroSCORE II, for relationships between and within variable importance drift, performance drift, and actual data set drift. Performance was assessed using a consensus metric. Results: A total of 227,087 adults underwent cardiac surgery during the study period, with a mortality rate of 2.76% (n=6258). There was strong evidence of a decrease in overall performance across all models (P<.0001). Extreme gradient boosting (clinical effectiveness metric [CEM] 0.728, 95% CI 0.728-0.729) and random forest (CEM 0.727, 95% CI 0.727-0.728) were the overall best-performing models, both temporally and nontemporally. EuroSCORE II performed the worst across all comparisons. Sharp changes in variable importance and data set drift from October to December 2017, from June to July 2018, and from December 2018 to February 2019 mirrored the effects of performance decrease across models. Conclusions: All models show a decrease in at least 3 of the 5 individual metrics. CEM and variable importance drift detection demonstrate the limitation of logistic regression methods used for cardiac surgery risk prediction and the effects of data set drift. Future work will be required to determine the interplay between ML models and whether ensemble models could improve on their respective performance advantages. %R 10.2196/45973 %U https://xmed.jmir.org/2024/1/e45973 %U https://doi.org/10.2196/45973 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e56686 %T Integrated Real-World Data Warehouses Across 7 Evolving Asian Health Care Systems: Scoping Review %A Shau,Wen-Yi %A Santoso,Handoko %A Jip,Vincent %A Setia,Sajita %+ Transform Medical Communications Limited, 184 Glasgow Street, Wanganui, 4500, New Zealand, 64 276175433, sajita.setia@transform-medcomms.com %K Asia %K health care databases %K cross-country comparison %K electronic health records %K electronic medical records %K data warehousing %K information storage and retrieval %K real-world data %K real-world evidence %K registries %K scoping review %D 2024 %7 11.6.2024 %9 Review %J J Med Internet Res %G English %X Background: Asia consists of diverse nations with extremely variable health care systems. Integrated real-world data (RWD) research warehouses provide vast interconnected data sets that uphold statistical rigor. Yet, their intricate details remain underexplored, restricting their broader applications. Objective: Building on our previous research that analyzed integrated RWD warehouses in India, Thailand, and Taiwan, this study extends the research to 7 distinct health care systems: Hong Kong, Indonesia, Malaysia, Pakistan, the Philippines, Singapore, and Vietnam. We aimed to map the evolving landscape of RWD, preferences for methodologies, and database use and archetype the health systems based on existing intrinsic capability for RWD generation. Methods: A systematic scoping review methodology was used, centering on contemporary English literature on PubMed (search date: May 9, 2023). Rigorous screening as defined by eligibility criteria identified RWD studies from multiple health care facilities in at least 1 of the 7 target Asian nations. Point estimates and their associated errors were determined for the data collected from eligible studies. Results: Of the 1483 real-world evidence citations identified on May 9, 2023, a total of 369 (24.9%) fulfilled the requirements for data extraction and subsequent analysis. Singapore, Hong Kong, and Malaysia contributed to ≥100 publications, with each country marked by a higher proportion of single-country studies at 51% (80/157), 66.2% (86/130), and 50% (50/100), respectively, and were classified as solo scholars. Indonesia, Pakistan, Vietnam, and the Philippines had fewer publications and a higher proportion of cross-country collaboration studies (CCCSs) at 79% (26/33), 58% (18/31), 74% (20/27), and 86% (19/22), respectively, and were classified as global collaborators. Collaboration with countries outside the 7 target nations appeared in 84.2% to 97.7% of the CCCSs of each nation. Among target nations, Singapore and Malaysia emerged as preferred research partners for other nations. From 2018 to 2023, most nations showed an increasing trend in study numbers, with Vietnam (24.5%) and Pakistan (21.2%) leading the growth; the only exception was the Philippines, which declined by –14.5%. Clinical registry databases were predominant across all CCCSs from every target nation. For single-country studies, Indonesia, Malaysia, and the Philippines favored clinical registries; Singapore had a balanced use of clinical registries and electronic medical or health records, whereas Hong Kong, Pakistan, and Vietnam leaned toward electronic medical or health records. Overall, 89.9% (310/345) of the studies took >2 years from completion to publication. Conclusions: The observed variations in contemporary RWD publications across the 7 nations in Asia exemplify distinct research landscapes across nations that are partially explained by their diverse economic, clinical, and research settings. Nevertheless, recognizing these variations is pivotal for fostering tailored, synergistic strategies that amplify RWD’s potential in guiding future health care research and policy decisions. International Registered Report Identifier (IRRID): RR2-10.2196/43741 %M 38749399 %R 10.2196/56686 %U https://www.jmir.org/2024/1/e56686 %U https://doi.org/10.2196/56686 %U http://www.ncbi.nlm.nih.gov/pubmed/38749399 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e57678 %T Explainable AI Method for Tinnitus Diagnosis via Neighbor-Augmented Knowledge Graph and Traditional Chinese Medicine: Development and Validation Study %A Yin,Ziming %A Kuang,Zhongling %A Zhang,Haopeng %A Guo,Yu %A Li,Ting %A Wu,Zhengkun %A Wang,Lihua %+ Department of Otolaryngology, Shanghai Municipal Hospital of Traditional Chinese Medicine, Shanghai University of Traditional Chinese Medicine, 274 Zhijiang Middle Road, Jing'an District, Shanghai, 200071, China, 86 18116013561, lihuahanhan@126.com %K knowledge graph %K syndrome differentiation %K tinnitus %K traditional Chinese medicine %K explainable %K ear %K audiology %K TCM %K algorithm %K diagnosis %K AI %K artificial intelligence %D 2024 %7 10.6.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Tinnitus diagnosis poses a challenge in otolaryngology owing to an extremely complex pathogenesis, lack of effective objectification methods, and factor-affected diagnosis. There is currently a lack of explainable auxiliary diagnostic tools for tinnitus in clinical practice. Objective: This study aims to develop a diagnostic model using an explainable artificial intelligence (AI) method to address the issue of low accuracy in tinnitus diagnosis. Methods: In this study, a knowledge graph–based tinnitus diagnostic method was developed by combining clinical medical knowledge with electronic medical records. Electronic medical record data from 1267 patients were integrated with traditional Chinese clinical medical knowledge to construct a tinnitus knowledge graph. Subsequently, weights were introduced, which measured patient similarity in the knowledge graph based on mutual information values. Finally, a collaborative neighbor algorithm was proposed, which scored patient similarity to obtain the recommended diagnosis. We conducted 2 group experiments and 1 case derivation to explore the effectiveness of our models and compared the models with state-of-the-art graph algorithms and other explainable machine learning models. Results: The experimental results indicate that the method achieved 99.4% accuracy, 98.5% sensitivity, 99.6% specificity, 98.7% precision, 98.6% F1-score, and 99% area under the receiver operating characteristic curve for the inference of 5 tinnitus subtypes among 253 test patients. Additionally, it demonstrated good interpretability. The topological structure of knowledge graphs provides transparency that can explain the reasons for the similarity between patients. Conclusions: This method provides doctors with a reliable and explainable diagnostic tool that is expected to improve tinnitus diagnosis accuracy. %M 38857077 %R 10.2196/57678 %U https://medinform.jmir.org/2024/1/e57678 %U https://doi.org/10.2196/57678 %U http://www.ncbi.nlm.nih.gov/pubmed/38857077 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e50049 %T Creation of Standardized Common Data Elements for Diagnostic Tests in Infectious Disease Studies: Semantic and Syntactic Mapping %A Stellmach,Caroline %A Hopff,Sina Marie %A Jaenisch,Thomas %A Nunes de Miranda,Susana Marina %A Rinaldi,Eugenia %A , %+ Berlin Institute of Health, Charité - Universitätsmedizin Berlin, Anna-Louisa-Karsch-Str 2, Berlin, 10178, Germany, 49 15752614677, caroline.stellmach@charite.de %K core data element %K CDE %K case report form %K CRF %K interoperability %K semantic standards %K infectious disease %K diagnostic test %K covid19 %K COVID-19 %K mpox %K ZIKV %K patient data %K data model %K syntactic interoperability %K clinical data %K FHIR %K SNOMED CT %K LOINC %K virus infection %K common element %D 2024 %7 10.6.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: It is necessary to harmonize and standardize data variables used in case report forms (CRFs) of clinical studies to facilitate the merging and sharing of the collected patient data across several clinical studies. This is particularly true for clinical studies that focus on infectious diseases. Public health may be highly dependent on the findings of such studies. Hence, there is an elevated urgency to generate meaningful, reliable insights, ideally based on a high sample number and quality data. The implementation of core data elements and the incorporation of interoperability standards can facilitate the creation of harmonized clinical data sets. Objective: This study’s objective was to compare, harmonize, and standardize variables focused on diagnostic tests used as part of CRFs in 6 international clinical studies of infectious diseases in order to, ultimately, then make available the panstudy common data elements (CDEs) for ongoing and future studies to foster interoperability and comparability of collected data across trials. Methods: We reviewed and compared the metadata that comprised the CRFs used for data collection in and across all 6 infectious disease studies under consideration in order to identify CDEs. We examined the availability of international semantic standard codes within the Systemized Nomenclature of Medicine - Clinical Terms, the National Cancer Institute Thesaurus, and the Logical Observation Identifiers Names and Codes system for the unambiguous representation of diagnostic testing information that makes up the CDEs. We then proposed 2 data models that incorporate semantic and syntactic standards for the identified CDEs. Results: Of 216 variables that were considered in the scope of the analysis, we identified 11 CDEs to describe diagnostic tests (in particular, serology and sequencing) for infectious diseases: viral lineage/clade; test date, type, performer, and manufacturer; target gene; quantitative and qualitative results; and specimen identifier, type, and collection date. Conclusions: The identification of CDEs for infectious diseases is the first step in facilitating the exchange and possible merging of a subset of data across clinical studies (and with that, large research projects) for possible shared analysis to increase the power of findings. The path to harmonization and standardization of clinical study data in the interest of interoperability can be paved in 2 ways. First, a map to standard terminologies ensures that each data element’s (variable’s) definition is unambiguous and that it has a single, unique interpretation across studies. Second, the exchange of these data is assisted by “wrapping” them in a standard exchange format, such as Fast Health care Interoperability Resources or the Clinical Data Interchange Standards Consortium’s Clinical Data Acquisition Standards Harmonization Model. %M 38857066 %R 10.2196/50049 %U https://www.jmir.org/2024/1/e50049 %U https://doi.org/10.2196/50049 %U http://www.ncbi.nlm.nih.gov/pubmed/38857066 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e56493 %T Individual-Level Digital Determinants of Health and Technology Acceptance of Patient Portals: Cross-Sectional Assessment %A Philpot,Lindsey M %A Ramar,Priya %A Roellinger,Daniel L %A Njeru,Jane W %A Ebbert,Jon O %+ Department of Medicine, Mayo Clinic, 200 First Street SW, Rochester, MN, 55902, United States, 1 507 538 1882, Philpot.Lindsey@mayo.edu %K electronic health records %K digital determinants of health %K patient portals %K eHealth %K digital health %K technology acceptance model %K digital health literacy %K digital inclusion %K mobile phone %D 2024 %7 10.6.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Digital determinants of health (DDoH), including access to technological tools and digital health literacy, function independently as barriers to health. Assessment for DDoH is not routine within most health care systems, although addressing DDoH could help mitigate differential health outcomes and the digital divide. Objective: This study aims to assess the role of individual-level DDoH factors on patient enrollment in and use of the patient portal. Methods: We developed a multimodal, cross-sectional survey and deployed it to 11,424 individuals based on their preferred mode and language documented within the electronic medical record. Based on the Technology Acceptance Model, enrollment in and intent to use the patient portal were the outcomes of interest. Perceived usefulness and ease of use were assessed to determine construct validity, and exploratory investigations included individual-level DDoH, including internet and device access, availability of technological support, medical complexity, individual relationship with the health care system, and digital health literacy. Counts (n) and proportions (%) were used to describe response categories, and adjusted and unadjusted odds ratios are reported. Results: This study included 1850 respondents (11,424 invited, 16.2% response rate), who were mostly female (1048/1850, 56.6%) and White (1240/1850, 67%), with an average age of 63 years. In the validation of the Technology Acceptance Model, measures of perceived ease of use (ie, using the patient portal will require a lot of mental effort; the patient portal will be very easy to use) and perceived usefulness (ie, the usefulness of the patient portal to send and receive messages with providers, schedule appointments, and refill medications) were positively associated with both enrollment in and intent to use the patient portal. Within adjusted models, perceived ease of use and perceived usefulness constructs, in addition to constructs of digital health literacy, knowing what health resources are available on the internet (adjusted odds ratio [aOR] 3.5, 95% CI 1.8-6.6), portal ease of use (aOR 2.8, 95% CI 1.6-5), and portal usefulness (aOR 2.4, 95% CI 1.4-4.2) were significantly associated with patient portal enrollment. Other factors associated with patient portal enrollment and intent to use included being comfortable reading and speaking English, reported use of the internet to surf the web or to send or receive emails, home internet access, and access to technology devices (computer, tablet, smartphone, etc). Conclusions: Assessing for and addressing individual-level DDoH, including digital health literacy, access to digital tools and technologies, and support of the relational aspects between patients, social support systems, and health care providers, could help mitigate disparities in health. By focusing efforts to assess for and address individual-level DDoH, an opportunity exists to improve digitally driven health care delivery outcomes like access and structural outcomes like bias built within algorithms created with incomplete representation across communities. %M 38695754 %R 10.2196/56493 %U https://formative.jmir.org/2024/1/e56493 %U https://doi.org/10.2196/56493 %U http://www.ncbi.nlm.nih.gov/pubmed/38695754 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 12 %N %P e54642 %T Use of Mobile Technologies to Streamline Pretriage Patient Flow in the Emergency Department: Observational Usability Study %A Wang,Panzhang %A Yu,Lei %A Li,Tao %A Zhou,Liang %A Ma,Xin %+ Department of Orthopedics, Shanghai Sixth People's Hospital, 600 Yishan Rd, Shanghai, 200233, China, 86 38297230, maxinsix@sina.com %K overcrowding %K overcrowded %K crowding %K smartphone %K queueing %K pretriage %K self-service %K triage %K emergency %K urgent %K ambulatory %K mHealth %K mobile health %K workflow %K health care management %K hospital %D 2024 %7 7.6.2024 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: In emergency departments (EDs), triage nurses are under tremendous daily pressure to rapidly assess the acuity level of patients and log the collected information into computers. With self-service technologies, patients could complete data entry on their own, allowing nurses to focus on higher-order tasks. Kiosks are a popular working example of such self-service technologies; however, placing a sufficient number of unwieldy and fixed machines demands a spatial change in the greeting area and affects pretriage flow. Mobile technologies could offer a solution to these issues. Objective: The aim of this study was to investigate the use of mobile technologies to improve pretriage flow in EDs. Methods: The proposed stack of mobile technologies includes patient-carried smartphones and QR technology. The web address of the self-registration app is encoded into a QR code, which was posted directly outside the walk-in entrance to be seen by every ambulatory arrival. Registration is initiated immediately after patients or their proxies scan the code using their smartphones. Patients could complete data entry at any site on the way to the triage area. Upon completion, the result is saved locally on smartphones. At the triage area, the result is automatically decoded by a portable code reader and then loaded into the triage computer. This system was implemented in three busy metropolitan EDs in Shanghai, China. Both kiosks and smartphones were evaluated randomly while being used to direct pretriage patient flow. Data were collected during a 20-day period in each center. Timeliness and usability of medical students simulating ED arrivals were assessed with the After-Scenario Questionnaire. Usability was assessed by triage nurses with the Net Promoter Score (NPS). Observations made during system implementation were subject to qualitative thematic analysis. Results: Overall, 5928 of 8575 patients performed self-registration on kiosks, and 7330 of 8532 patients checked in on their smartphones. Referring effort was significantly reduced (43.7% vs 8.8%; P<.001) and mean pretriage waiting times were significantly reduced (4.4, SD 1.7 vs 2.9, SD 1.0 minutes; P<.001) with the use of smartphones compared to kiosks. There was a significant difference in mean usability scores for “ease of task completion” (4.4, SD 1.5 vs 6.7, SD 0.7; P<.001), “satisfaction with completion time” (4.5, SD 1.4 vs 6.8, SD 0.6; P<.001), and “satisfaction with support” (4.9, SD 1.9 vs 6.6, SD 1.2; P<.001). Triage nurses provided a higher NPS after implementation of mobile self-registration compared to the use of kiosks (13.3% vs 93.3%; P<.001). A modified queueing model was identified and qualitative findings were grouped by sequential steps. Conclusions: This study suggests patient-carried smartphones as a useful tool for ED self-registration. With increased usability and a tailored queueing model, the proposed system is expected to minimize pretriage waiting for patients in the ED. %M 38848554 %R 10.2196/54642 %U https://mhealth.jmir.org/2024/1/e54642 %U https://doi.org/10.2196/54642 %U http://www.ncbi.nlm.nih.gov/pubmed/38848554 %0 Journal Article %@ 2368-7959 %I %V 11 %N %P e57965 %T Development of Recommendations for the Digital Sharing of Notes With Adolescents in Mental Health Care: Delphi Study %A Nielsen,Martine Stecher %A Steinsbekk,Aslak %A Nøst,Torunn Hatlen %K electronic health record %K EHR %K electronic health records %K EHRs %K electronic medical record %K EMR %K electronic medical records %K EMRs %K patient record %K health record %K health records %K personal health record %K PHR %K online access to electronic health records %K open notes %K clinical notes %K adolescent mental health care %K adolescent mental health %K child mental health %K mental health %K mental illness %K mental illnesses %K mental disorder %K mental disorders %K recommendations %K Delphi study %K digital mental health %K e-health %K eHealth %K e–mental health %K health care professionals %K digital health care %D 2024 %7 6.6.2024 %9 %J JMIR Ment Health %G English %X Background: In many countries, health care professionals are legally obliged to share information from electronic health records with patients. However, concerns have been raised regarding the sharing of notes with adolescents in mental health care, and health care professionals have called for recommendations to guide this practice. Objective: The aim was to reach a consensus among authors of scientific papers on recommendations for health care professionals’ digital sharing of notes with adolescents in mental health care and to investigate whether staff at child and adolescent specialist mental health care clinics agreed with the recommendations. Methods: A Delphi study was conducted with authors of scientific papers to reach a consensus on recommendations. The process of making the recommendations involved three steps. First, scientific papers meeting the eligibility criteria were identified through a PubMed search where the references were screened. Second, the results from the included papers were coded and transformed into recommendations in an iterative process. Third, the authors of the included papers were asked to provide feedback and consider their agreement with each of the suggested recommendations in two rounds. After the Delphi process, a cross-sectional study was conducted among staff at specialist child and adolescent mental health care clinics to assess whether they agreed with the recommendations that reached a consensus. Results: Of the 84 invited authors, 27 responded. A consensus was reached on 17 recommendations on areas related to digital sharing of notes with adolescents in mental health care. The recommendations considered how to introduce digital access to notes, write notes, and support health care professionals, and when to withhold notes. Of the 41 staff members at child and adolescent specialist mental health care clinics, 60% or more agreed with the 17 recommendations. No consensus was reached regarding the age at which adolescents should receive digital access to their notes and the timing of digitally sharing notes with parents. Conclusions: A total of 17 recommendations related to key aspects of health care professionals’ digital sharing of notes with adolescents in mental health care achieved consensus. Health care professionals can use these recommendations to guide their practice of sharing notes with adolescents in mental health care. However, the effects and experiences of following these recommendations should be tested in clinical practice. %R 10.2196/57965 %U https://mental.jmir.org/2024/1/e57965 %U https://doi.org/10.2196/57965 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e51666 %T The Solutions in Health Analytics for Rural Equity Across the Northwest (SHARE-NW) Dashboard for Health Equity in Rural Public Health: Usability Evaluation %A Heitkemper,Elizabeth %A Hulse,Scott %A Bekemeier,Betty %A Schultz,Melinda %A Whitman,Greg %A Turner,Anne M %+ School of Nursing, The University of Texas at Austin, 1710 Red River Street, Austin, TX, 78712, United States, 1 512 232 4228, e.heit@utexas.edu %K data dashboard %K rural health %K health equity %K usability %K nursing informatics %K dashboard %K rural %K informatics %K satisfaction %K think aloud %K content analysis %K user experience %K public health %K visualization %K information systems %D 2024 %7 5.6.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Given the dearth of resources to support rural public health practice, the solutions in health analytics for rural equity across the northwest dashboard (SHAREdash) was created to support rural county public health departments in northwestern United States with accessible and relevant data to identify and address health disparities in their jurisdictions. To ensure the development of useful dashboards, assessment of usability should occur at multiple stages throughout the system development life cycle. SHAREdash was refined via user-centered design methods, and upon completion, it is critical to evaluate the usability of SHAREdash. Objective: This study aims to evaluate the usability of SHAREdash based on the system development lifecycle stage 3 evaluation goals of efficiency, satisfaction, and validity. Methods: Public health professionals from rural health departments from Washington, Idaho, Oregon, and Alaska were enrolled in the usability study from January to April 2022. The web-based evaluation consisted of 2 think-aloud tasks and a semistructured qualitative interview. Think-aloud tasks assessed efficiency and effectiveness, and the interview investigated satisfaction and overall usability. Verbatim transcripts from the tasks and interviews were analyzed using directed content analysis. Results: Of the 9 participants, all were female and most worked at a local health department (7/9, 78%). A mean of 10.1 (SD 1.4) clicks for task 1 (could be completed in 7 clicks) and 11.4 (SD 2.0) clicks for task 2 (could be completed in 9 clicks) were recorded. For both tasks, most participants required no prompting—89% (n=8) participants for task 1 and 67% (n=6) participants for task 2, respectively. For effectiveness, all participants were able to complete each task accurately and comprehensively. Overall, the participants were highly satisfied with the dashboard with everyone remarking on the utility of using it to support their work, particularly to compare their jurisdiction to others. Finally, half of the participants stated that the ability to share the graphs from the dashboard would be “extremely useful” for their work. The only aspect of the dashboard cited as problematic is the amount of missing data that was present, which was a constraint of the data available about rural jurisdictions. Conclusions: Think-aloud tasks showed that the SHAREdash allows users to complete tasks efficiently. Overall, participants reported being very satisfied with the dashboard and provided multiple ways they planned to use it to support their work. The main usability issue identified was the lack of available data indicating the importance of addressing the ongoing issues of missing and fragmented public health data, particularly for rural communities. %M 38837192 %R 10.2196/51666 %U https://humanfactors.jmir.org/2024/1/e51666 %U https://doi.org/10.2196/51666 %U http://www.ncbi.nlm.nih.gov/pubmed/38837192 %0 Journal Article %@ 2291-9694 %I %V 12 %N %P e54428 %T Event Analysis for Automated Estimation of Absent and Persistent Medication Alerts: Novel Methodology %A Bittmann,Janina A %A Scherkl,Camilo %A Meid,Andreas D %A Haefeli,Walter E %A Seidling,Hanna M %K clinical decision support system %K CDSS %K medication alert system %K alerting %K alert acceptance %K event analysis %D 2024 %7 4.6.2024 %9 %J JMIR Med Inform %G English %X Background: Event analysis is a promising approach to estimate the acceptance of medication alerts issued by computerized physician order entry (CPOE) systems with an integrated clinical decision support system (CDSS), particularly when alerts cannot be interactively confirmed in the CPOE-CDSS due to its system architecture. Medication documentation is then reviewed for documented evidence of alert acceptance, which can be a time-consuming process, especially when performed manually. Objective: We present a new automated event analysis approach, which was applied to a large data set generated in a CPOE-CDSS with passive, noninterruptive alerts. Methods: Medication and alert data generated over 3.5 months within the CPOE-CDSS at Heidelberg University Hospital were divided into 24-hour time intervals in which the alert display was correlated with associated prescription changes. Alerts were considered “persistent” if they were displayed in every consecutive 24-hour time interval due to a respective active prescription until patient discharge and were considered “absent” if they were no longer displayed during continuous prescriptions in the subsequent interval. Results: Overall, 1670 patient cases with 11,428 alerts were analyzed. Alerts were displayed for a median of 3 (IQR 1-7) consecutive 24-hour time intervals, with the shortest alerts displayed for drug-allergy interactions and the longest alerts displayed for potentially inappropriate medication for the elderly (PIM). Among the total 11,428 alerts, 56.1% (n=6413) became absent, most commonly among alerts for drug-drug interactions (1915/2366, 80.9%) and least commonly among PIM alerts (199/499, 39.9%). Conclusions: This new approach to estimate alert acceptance based on event analysis can be flexibly adapted to the automated evaluation of passive, noninterruptive alerts. This enables large data sets of longitudinal patient cases to be processed, allows for the derivation of the ratios of persistent and absent alerts, and facilitates the comparison and prospective monitoring of these alerts. %R 10.2196/54428 %U https://medinform.jmir.org/2024/1/e54428 %U https://doi.org/10.2196/54428 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e55798 %T Using the Natural Language Processing System Medical Named Entity Recognition-Japanese to Analyze Pharmaceutical Care Records: Natural Language Processing Analysis %A Ohno,Yukiko %A Kato,Riri %A Ishikawa,Haruki %A Nishiyama,Tomohiro %A Isawa,Minae %A Mochizuki,Mayumi %A Aramaki,Eiji %A Aomori,Tohru %+ Faculty of Pharmacy, Takasaki University of Health and Welfare, 37-1 Nakaorui-machi, Takasaki-shi, Gunma, 370-0033, Japan, 81 273521290, aomori-t@takasaki-u.ac.jp %K natural language processing %K NLP %K named entity recognition %K pharmaceutical care records %K machine learning %K cefazolin sodium %K electronic medical record %K EMR %K extraction %K Japanese %D 2024 %7 4.6.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Large language models have propelled recent advances in artificial intelligence technology, facilitating the extraction of medical information from unstructured data such as medical records. Although named entity recognition (NER) is used to extract data from physicians’ records, it has yet to be widely applied to pharmaceutical care records. Objective: In this study, we aimed to investigate the feasibility of automatic extraction of the information regarding patients’ diseases and symptoms from pharmaceutical care records. The verification was performed using Medical Named Entity Recognition-Japanese (MedNER-J), a Japanese disease-extraction system designed for physicians’ records. Methods: MedNER-J was applied to subjective, objective, assessment, and plan data from the care records of 49 patients who received cefazolin sodium injection at Keio University Hospital between April 2018 and March 2019. The performance of MedNER-J was evaluated in terms of precision, recall, and F1-score. Results: The F1-scores of NER for subjective, objective, assessment, and plan data were 0.46, 0.70, 0.76, and 0.35, respectively. In NER and positive-negative classification, the F1-scores were 0.28, 0.39, 0.64, and 0.077, respectively. The F1-scores of NER for objective (0.70) and assessment data (0.76) were higher than those for subjective and plan data, which supported the superiority of NER performance for objective and assessment data. This might be because objective and assessment data contained many technical terms, similar to the training data for MedNER-J. Meanwhile, the F1-score of NER and positive-negative classification was high for assessment data alone (F1-score=0.64), which was attributed to the similarity of its description format and contents to those of the training data. Conclusions: MedNER-J successfully read pharmaceutical care records and showed the best performance for assessment data. However, challenges remain in analyzing records other than assessment data. Therefore, it will be necessary to reinforce the training data for subjective data in order to apply the system to pharmaceutical care records. %M 38833694 %R 10.2196/55798 %U https://formative.jmir.org/2024/1/e55798 %U https://doi.org/10.2196/55798 %U http://www.ncbi.nlm.nih.gov/pubmed/38833694 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e47682 %T A Taxonomy for Health Information Systems %A Janssen,Anna %A Donnelly,Candice %A Shaw,Tim %+ Faculty of Medicine and Health, The University of Sydney, Level 2, Charles Perkins Centre D17, Sydney, 2066, Australia, 61 02 9036 9406, anna.janssen@sydney.edu.au %K eHealth %K digital health %K electronic health data %K data revolution %K actionable data %K mobile phone %D 2024 %7 31.5.2024 %9 Viewpoint %J J Med Internet Res %G English %X The health sector is highly digitized, which is enabling the collection of vast quantities of electronic data about health and well-being. These data are collected by a diverse array of information and communication technologies, including systems used by health care organizations, consumer and community sources such as information collected on the web, and passively collected data from technologies such as wearables and devices. Understanding the breadth of IT that collect these data and how it can be actioned is a challenge for the significant portion of the digital health workforce that interact with health data as part of their duties but are not for informatics experts. This viewpoint aims to present a taxonomy categorizing common information and communication technologies that collect electronic data. An initial classification of key information systems collecting electronic health data was undertaken via a rapid review of the literature. Subsequently, a purposeful search of the scholarly and gray literature was undertaken to extract key information about the systems within each category to generate definitions of the systems and describe the strengths and limitations of these systems. %M 38820575 %R 10.2196/47682 %U https://www.jmir.org/2024/1/e47682 %U https://doi.org/10.2196/47682 %U http://www.ncbi.nlm.nih.gov/pubmed/38820575 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e46954 %T Experiences of Electronic Health Records’ and Client Information Systems’ Use on a Mobile Device and Factors Associated With Work Time Savings Among Practical Nurses: Cross-Sectional Study %A Paatela,Satu %A Kyytsönen,Maiju %A Saranto,Kaija %A Kinnunen,Ulla-Mari %A Vehko,Tuulikki %+ Health and Social Service System Research, Finnish Institute for Health and Welfare, Mannerheimintie 166, Helsinki, 00271, Finland, 358 29 524 77 22, satu.paatela@thl.fi %K practical nurse %K information and communication technology %K electronic health record %K client information system %K documentation %K mobile technology %D 2024 %7 29.5.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: The transmission of clinical information in nursing predominantly occurs through digital solutions, such as computers and mobile devices, in today’s era. Various technological systems, including electronic health records (EHRs) and client information systems (CISs), can be seamlessly integrated with mobile devices. The use of mobile devices is anticipated to rise, particularly as long-term care is increasingly delivered in environments such as clients’ homes, where computers are not readily accessible. However, there is a growing need for more user-centered data to ensure that mobile devices effectively support practical nurses in their daily activities. Objective: This study aims to analyze practical nurses’ experiences of using EHRs or CISs on a mobile device in their daily practice. In addition, it aims to examine the factors associated with work time savings when using EHRs/CISs on a mobile device. Methods: A cross-sectional study using an electronic survey was conducted in spring 2022. A total of 3866 practical nurses participated in the survey based on self-assessment. The sample was limited to practical nurses who used EHRs or CISs on a mobile device and worked in home care or service housing within the social welfare or health care sector (n=1014). Logistic regression analysis was used to explore the factors associated with work time savings. Results: The likelihood of perceiving work time savings was higher among more experienced EHR/CIS users compared with those with less experience (odds ratio [OR] 1.59, 95% CI 1.30-1.94). Participants with 0-5 years of work experience were more likely to experience work time savings compared with those who had worked 21 years or more (OR 2.41, 95% CI 1.43-4.07). Practical nurses in home care were also more likely to experience work time savings compared with those working in service housing (OR 1.95, 95% CI 1.23-3.07). A lower grade given for EHRs/CISs was associated with a reduced likelihood of experiencing work time savings (OR 0.76, 95% CI 0.66-0.89). Participants who documented client data in a public area were more likely to experience work time savings compared with those who did so in the nurses’ office (OR 2.33, 95% CI 1.27-4.25). Practical nurses who found documentation of client data on a mobile device easy (OR 3.05, 95% CI 2.14-4.34) were more likely to experience work time savings compared with those who did not. Similarly, participants who believed that documentation of client data on a mobile device reduced the need to memorize things (OR 4.10, 95% CI 2.80-6.00) were more likely to experience work time savings compared with those who did not. Conclusions: To enhance the proportion of practical nurses experiencing work time savings, we recommend that organizations offer comprehensive orientation and regular education sessions tailored for mobile device users who have less experience using EHRs or CISs and find mobile devices less intuitive to use. %M 38809583 %R 10.2196/46954 %U https://www.jmir.org/2024/1/e46954 %U https://doi.org/10.2196/46954 %U http://www.ncbi.nlm.nih.gov/pubmed/38809583 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e52027 %T Enabling Health Information Recommendation Using Crowdsourced Refinement in Web-Based Health Information Applications: User-Centered Design Approach and EndoZone Informatics Case Study %A Li,Wenhao %A O'Hara,Rebecca %A Hull,M Louise %A Slater,Helen %A Sirohi,Diksha %A Parker,Melissa A %A Bidargaddi,Niranjan %+ College of Medicine and Public Health, Flinders University, 1284 South Road, Clovelly Park, 5042, Australia, 61 423416543, niranjan.bidargaddi@flinders.edu.au %K information recommendation %K crowdsourcing %K health informatics %K digital health %K endometriosis %D 2024 %7 29.5.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: In the digital age, search engines and social media platforms are primary sources for health information, yet their commercial interests–focused algorithms often prioritize irrelevant content. Web-based health applications by reputable sources offer a solution to circumvent these biased algorithms. Despite this advantage, there remains a significant gap in research on the effective integration of content-ranking algorithms within these specialized health applications to ensure the delivery of personalized and relevant health information. Objective: This study introduces a generic methodology designed to facilitate the development and implementation of health information recommendation features within web-based health applications. Methods: We detail our proposed methodology, covering conceptual foundation and practical considerations through the stages of design, development, operation, review, and optimization in the software development life cycle. Using a case study, we demonstrate the practical application of the proposed methodology through the implementation of recommendation functionalities in the EndoZone platform, a platform dedicated to providing targeted health information on endometriosis. Results: Application of the proposed methodology in the EndoZone platform led to the creation of a tailored health information recommendation system known as EndoZone Informatics. Feedback from EndoZone stakeholders as well as insights from the implementation process validate the methodology’s utility in enabling advanced recommendation features in health information applications. Preliminary assessments indicate that the system successfully delivers personalized content, adeptly incorporates user feedback, and exhibits considerable flexibility in adjusting its recommendation logic. While certain project-specific design flaws were not caught in the initial stages, these issues were subsequently identified and rectified in the review and optimization stages. Conclusions: We propose a generic methodology to guide the design and implementation of health information recommendation functionality within web-based health information applications. By harnessing user characteristics and feedback for content ranking, this methodology enables the creation of personalized recommendations that align with individual user needs within trusted health applications. The successful application of our methodology in the development of EndoZone Informatics marks a significant progress toward personalized health information delivery at scale, tailored to the specific needs of users. %M 38809588 %R 10.2196/52027 %U https://humanfactors.jmir.org/2024/1/e52027 %U https://doi.org/10.2196/52027 %U http://www.ncbi.nlm.nih.gov/pubmed/38809588 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e54334 %T Exploring Consumers’ Negative Electronic Word-of-Mouth of 5 Military Hospitals in Taiwan Through SERVQUAL and Flower of Services: Web Scraping Analysis %A Huang,Ching-Yuan %A Lee,Po-Chun %A Chen,Long-Hui %+ Department of Internal Medicine, Kaohsiung Armed Forces General Hospital, No.2, Zhongzheng 1st Rd., Lingya Dist., Kaohsiung, 80284, Taiwan, 886 953332550, chyun0124@gmail.com %K electronic word-of-mouth %K eWOM %K service quality %K SERVQUAL scale %K Flower of Services %K health care service quality %K military hospitals %D 2024 %7 29.5.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: In recent years, with the widespread use of the internet, the influence of electronic word-of-mouth (eWOM) has been increasingly recognized, particularly the significance of negative eWOM, which has surpassed positive eWOM in importance. Such reviews play a pivotal role in research related to service industry management, particularly in intangible service sectors such as hospitals, where they have become a reference point for improving service quality. Objective: This study comprehensively collected negative eWOM from 5 military hospitals in Taiwan that were at or above the level of regional teaching hospitals. It aimed to investigate service quality issues before and after the pandemic. The findings provide important references for formulating strategies to improve service quality. Methods: In this study, we used web scraping techniques to gather 1259 valid negative eWOM, covering the period from the inception of the first review to December 31, 2022. These reviews were categorized using content analysis based on the modified Parasuraman, Zeithaml, and Berry service quality (PZB SERVQUAL) scale and Flower of Services. Statistical data analysis was conducted to investigate the performance of service quality. Results: The annual count of negative reviews for each hospital has exhibited a consistent upward trajectory over the years, with a more pronounced increase following the onset of the pandemic. In the analysis, among the 5 dimensions of PZB SERVQUAL framework, the “Assurance” dimension yielded the least favorable results, registering a negative review rate as high as 58.3%. Closely trailing, the “Responsiveness” dimension recorded a negative review rate of 34.2%. When evaluating the service process, the subitem “In Service: Diagnosis/Examination/Medical/Hospitalization” exhibited the least satisfactory performance, with a negative review rate of 46.2%. This was followed by the subitem “In Service: Pre-diagnosis Waiting,” which had a negative review rate of 20.2%. To evaluate the average scores of negative reviews before and during the onset of the COVID-19 pandemic, independent sample t tests (2-tailed) were used. The analysis revealed statistically significant differences (P<.001). Furthermore, an ANOVA was conducted to investigate whether the length of the negative reviews impacted their ratings, which also showed significant differences (P=.01). Conclusions: Before and during the pandemic, there were significant differences in evaluating hospital services, and a higher word count in negative reviews indicated greater dissatisfaction with the service. Therefore, it is recommended that hospitals establish more comprehensive service quality management mechanisms, carefully respond to negative reviews, and categorize significant service deficiencies as critical events to prevent a decrease in overall service quality. Furthermore, during the service process, customers are particularly concerned about the attitude and responsiveness of health care personnel in the treatment process. Therefore, hospitals should enhance training and management in this area. %M 38809602 %R 10.2196/54334 %U https://formative.jmir.org/2024/1/e54334 %U https://doi.org/10.2196/54334 %U http://www.ncbi.nlm.nih.gov/pubmed/38809602 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e55676 %T An Extensible Evaluation Framework Applied to Clinical Text Deidentification Natural Language Processing Tools: Multisystem and Multicorpus Study %A Heider,Paul M %A Meystre,Stéphane M %+ Biomedical Informatics Center, Medical University of South Carolina, 22 WestEdge Street, Suite 200, Charleston, SC, 29403, United States, 1 843 792 3385, heiderp@musc.edu %K natural language processing %K evaluation methodology %K deidentification %K privacy protection %K de-identification %K secondary use %K patient privacy %D 2024 %7 28.5.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Clinical natural language processing (NLP) researchers need access to directly comparable evaluation results for applications such as text deidentification across a range of corpus types and the means to easily test new systems or corpora within the same framework. Current systems, reported metrics, and the personally identifiable information (PII) categories evaluated are not easily comparable. Objective: This study presents an open-source and extensible end-to-end framework for comparing clinical NLP system performance across corpora even when the annotation categories do not align. Methods: As a use case for this framework, we use 6 off-the-shelf text deidentification systems (ie, CliniDeID, deid from PhysioNet, MITRE Identity Scrubber Toolkit [MIST], NeuroNER, National Library of Medicine [NLM] Scrubber, and Philter) across 3 standard clinical text corpora for the task (2 of which are publicly available) and 1 private corpus (all in English), with annotation categories that are not directly analogous. The framework is built on shell scripts that can be extended to include new systems, corpora, and performance metrics. We present this open tool, multiple means for aligning PII categories during evaluation, and our initial timing and performance metric findings. Code for running this framework with all settings needed to run all pairs are available via Codeberg and GitHub. Results: From this case study, we found large differences in processing speed between systems. The fastest system (ie, MIST) processed an average of 24.57 (SD 26.23) notes per second, while the slowest (ie, CliniDeID) processed an average of 1.00 notes per second. No system uniformly outperformed the others at identifying PII across corpora and categories. Instead, a rich tapestry of performance trade-offs emerged for PII categories. CliniDeID and Philter prioritize recall over precision (with an average recall 6.9 and 11.2 points higher, respectively, for partially matching spans of text matching any PII category), while the other 4 systems consistently have higher precision (with MIST’s precision scoring 20.2 points higher, NLM Scrubber scoring 4.4 points higher, NeuroNER scoring 7.2 points higher, and deid scoring 17.1 points higher). The macroaverage recall across corpora for identifying names, one of the more sensitive PII categories, included deid (48.8%) and MIST (66.9%) at the low end and NeuroNER (84.1%), NLM Scrubber (88.1%), and CliniDeID (95.9%) at the high end. A variety of metrics across categories and corpora are reported with a wider variety (eg, F2-score) available via the tool. Conclusions: NLP systems in general and deidentification systems and corpora in our use case tend to be evaluated in stand-alone research articles that only include a limited set of comparators. We hold that a single evaluation pipeline across multiple systems and corpora allows for more nuanced comparisons. Our open pipeline should reduce barriers to evaluation and system advancement. %M 38805692 %R 10.2196/55676 %U https://www.jmir.org/2024/1/e55676 %U https://doi.org/10.2196/55676 %U http://www.ncbi.nlm.nih.gov/pubmed/38805692 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e54095 %T Advances in the Application of AI Robots in Critical Care: Scoping Review %A Li,Yun %A Wang,Min %A Wang,Lu %A Cao,Yuan %A Liu,Yuyan %A Zhao,Yan %A Yuan,Rui %A Yang,Mengmeng %A Lu,Siqian %A Sun,Zhichao %A Zhou,Feihu %A Qian,Zhirong %A Kang,Hongjun %+ The First Medical Centre, Chinese PLA General Hospital, 28 Fuxing Road, Haidian District,, Beijing, 100853, China, 86 13811989878, doctorkang301@163.com %K critical care medicine %K artificial intelligence %K AI %K robotics %K intensive care unit %K ICU %D 2024 %7 27.5.2024 %9 Review %J J Med Internet Res %G English %X Background: In recent epochs, the field of critical medicine has experienced significant advancements due to the integration of artificial intelligence (AI). Specifically, AI robots have evolved from theoretical concepts to being actively implemented in clinical trials and applications. The intensive care unit (ICU), known for its reliance on a vast amount of medical information, presents a promising avenue for the deployment of robotic AI, anticipated to bring substantial improvements to patient care. Objective: This review aims to comprehensively summarize the current state of AI robots in the field of critical care by searching for previous studies, developments, and applications of AI robots related to ICU wards. In addition, it seeks to address the ethical challenges arising from their use, including concerns related to safety, patient privacy, responsibility delineation, and cost-benefit analysis. Methods: Following the scoping review framework proposed by Arksey and O’Malley and the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, we conducted a scoping review to delineate the breadth of research in this field of AI robots in ICU and reported the findings. The literature search was carried out on May 1, 2023, across 3 databases: PubMed, Embase, and the IEEE Xplore Digital Library. Eligible publications were initially screened based on their titles and abstracts. Publications that passed the preliminary screening underwent a comprehensive review. Various research characteristics were extracted, summarized, and analyzed from the final publications. Results: Of the 5908 publications screened, 77 (1.3%) underwent a full review. These studies collectively spanned 21 ICU robotics projects, encompassing their system development and testing, clinical trials, and approval processes. Upon an expert-reviewed classification framework, these were categorized into 5 main types: therapeutic assistance robots, nursing assistance robots, rehabilitation assistance robots, telepresence robots, and logistics and disinfection robots. Most of these are already widely deployed and commercialized in ICUs, although a select few remain under testing. All robotic systems and tools are engineered to deliver more personalized, convenient, and intelligent medical services to patients in the ICU, concurrently aiming to reduce the substantial workload on ICU medical staff and promote therapeutic and care procedures. This review further explored the prevailing challenges, particularly focusing on ethical and safety concerns, proposing viable solutions or methodologies, and illustrating the prospective capabilities and potential of AI-driven robotic technologies in the ICU environment. Ultimately, we foresee a pivotal role for robots in a future scenario of a fully automated continuum from admission to discharge within the ICU. Conclusions: This review highlights the potential of AI robots to transform ICU care by improving patient treatment, support, and rehabilitation processes. However, it also recognizes the ethical complexities and operational challenges that come with their implementation, offering possible solutions for future development and optimization. %M 38801765 %R 10.2196/54095 %U https://www.jmir.org/2024/1/e54095 %U https://doi.org/10.2196/54095 %U http://www.ncbi.nlm.nih.gov/pubmed/38801765 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e52185 %T Developing a Cost-Effective Surgical Scheduling System Applying Lean Thinking and Toyota’s Methods for Surgery-Related Big Data for Improved Data Use in Hospitals: User-Centered Design Approach %A Lin,Chien-Chung %A Shen,Jian-Hong %A Chen,Shu-Fang %A Chen,Hung-Ming %A Huang,Hung-Meng %+ Department of Orthopedic Surgery, Taipei City Hospital, Number 33, Section 2, Chung-Hwa Road, Taipei, 100, Taiwan, 886 223889595 ext 2102, ericdoctor@gmail.com %K algorithm %K process %K computational thinking %K continuous improvement %K customer needs %K lean principles %K problem solving %K Toyota Production System %K value stream map %K need %K needs %K operating room %D 2024 %7 24.5.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Surgical scheduling is pivotal in managing daily surgical sequences, impacting patient experience and hospital resources significantly. With operating rooms costing approximately US $36 per minute, efficient scheduling is vital. However, global practices in surgical scheduling vary, largely due to challenges in predicting individual surgeon times for diverse patient conditions. Inspired by the Toyota Production System’s efficiency in addressing similar logistical challenges, we applied its principles as detailed in the book “Lean Thinking” by Womack and Jones, which identifies processes that do not meet customer needs as wasteful. This insight is critical in health care, where waste can compromise patient safety and medical quality. Objective: This study aims to use lean thinking and Toyota methods to develop a more efficient surgical scheduling system that better aligns with user needs without additional financial burdens. Methods: We implemented the 5 principles of the Toyota system: specifying value, identifying the value stream, enabling flow, establishing pull, and pursuing perfection. Value was defined in terms of meeting the customer’s needs, which in this context involved developing a responsive and efficient scheduling system. Our approach included 2 subsystems: one handling presurgery patient data and another for intraoperative and postoperative data. We identified inefficiencies in the presurgery data subsystem and responded by creating a comprehensive value stream map of the surgical process. We developed 2 Excel (Microsoft Corporation) macros using Visual Basic for Applications. The first calculated average surgery times from intra- or postoperative historic data, while the second estimated surgery durations and generated concise, visually engaging scheduling reports from presurgery data. We assessed the effectiveness of the new system by comparing task completion times and user satisfaction between the old and new systems. Results: The implementation of the revised scheduling system significantly reduced the overall scheduling time from 301 seconds to 261 seconds (P=.02), with significant time reductions in the revised process from 99 seconds to 62 seconds (P<.001). Despite these improvements, approximately 21% of nurses preferred the older system for its familiarity. The new system protects patient data privacy and streamlines schedule dissemination through a secure LINE group (LY Corp), ensuring seamless flow. The design of the system allows for real-time updates and has been effectively monitoring surgical durations daily for over 3 years. The “pull” principle was demonstrated when an unplanned software issue prompted immediate, user-led troubleshooting, enhancing system reliability. Continuous improvement efforts are ongoing, except for the preoperative patient confirmation step, which requires further enhancement to ensure optimal patient safety. Conclusions: Lean principles and Toyota’s methods, combined with computer programming, can revitalize surgical scheduling processes. They offer effective solutions for surgical scheduling challenges and enable the creation of a novel surgical scheduling system without incurring additional costs. %M 38787610 %R 10.2196/52185 %U https://formative.jmir.org/2024/1/e52185 %U https://doi.org/10.2196/52185 %U http://www.ncbi.nlm.nih.gov/pubmed/38787610 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e54996 %T Barriers to Implementing Registered Nurse–Driven Clinical Decision Support for Antibiotic Stewardship: Retrospective Case Study %A Stevens,Elizabeth R %A Xu,Lynn %A Kwon,JaeEun %A Tasneem,Sumaiya %A Henning,Natalie %A Feldthouse,Dawn %A Kim,Eun Ji %A Hess,Rachel %A Dauber-Decker,Katherine L %A Smith,Paul D %A Halm,Wendy %A Gautam-Goyal,Pranisha %A Feldstein,David A %A Mann,Devin M %+ Department of Population Health, New York University Grossman School of Medicine, 180 Madison Ave, Room 17-13, New York, NY, 10016, United States, 1 6465012558, elizabeth.stevens@nyulangone.org %K integrated clinical prediction rules %K EHR %K electronic health record %K implementation %K barriers %K acute respiratory infections %K antibiotics %K CDS %K clinical decision support %K decision support %K antibiotic %K prescribe %K prescription %K acute respiratory infection %K barrier %K effectiveness %K registered nurse %K RN %K RN-driven intervention %K personnel availability %K workflow variability %K infrastructure %K infrastructures %K law %K laws %K policy %K policies %K clinical-care setting %K clinical setting %K electronic health records %K RN-driven %K antibiotic stewardship %K retrospective analysis %K Consolidated Framework for Implementation Research %K CFIR %K CDS-based intervention %K urgent care %K New York %K chart review %K interview %K interviews %K staff change %K staff changes %K RN shortage %K RN shortages %K turnover %K health system %K nurse %K nurses %K researcher %K researchers %D 2024 %7 23.5.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Up to 50% of antibiotic prescriptions for upper respiratory infections (URIs) are inappropriate. Clinical decision support (CDS) systems to mitigate unnecessary antibiotic prescriptions have been implemented into electronic health records, but their use by providers has been limited. Objective: As a delegation protocol, we adapted a validated electronic health record–integrated clinical prediction rule (iCPR) CDS-based intervention for registered nurses (RNs), consisting of triage to identify patients with low-acuity URI followed by CDS-guided RN visits. It was implemented in February 2022 as a randomized controlled stepped-wedge trial in 43 primary and urgent care practices within 4 academic health systems in New York, Wisconsin, and Utah. While issues were pragmatically addressed as they arose, a systematic assessment of the barriers to implementation is needed to better understand and address these barriers. Methods: We performed a retrospective case study, collecting quantitative and qualitative data regarding clinical workflows and triage-template use from expert interviews, study surveys, routine check-ins with practice personnel, and chart reviews over the first year of implementation of the iCPR intervention. Guided by the updated CFIR (Consolidated Framework for Implementation Research), we characterized the initial barriers to implementing a URI iCPR intervention for RNs in ambulatory care. CFIR constructs were coded as missing, neutral, weak, or strong implementation factors. Results: Barriers were identified within all implementation domains. The strongest barriers were found in the outer setting, with those factors trickling down to impact the inner setting. Local conditions driven by COVID-19 served as one of the strongest barriers, impacting attitudes among practice staff and ultimately contributing to a work infrastructure characterized by staff changes, RN shortages and turnover, and competing responsibilities. Policies and laws regarding scope of practice of RNs varied by state and institutional application of those laws, with some allowing more clinical autonomy for RNs. This necessitated different study procedures at each study site to meet practice requirements, increasing innovation complexity. Similarly, institutional policies led to varying levels of compatibility with existing triage, rooming, and documentation workflows. These workflow conflicts were compounded by limited available resources, as well as an implementation climate of optional participation, few participation incentives, and thus low relative priority compared to other clinical duties. Conclusions: Both between and within health care systems, significant variability existed in workflows for patient intake and triage. Even in a relatively straightforward clinical workflow, workflow and cultural differences appreciably impacted intervention adoption. Takeaways from this study can be applied to other RN delegation protocol implementations of new and innovative CDS tools within existing workflows to support integration and improve uptake. When implementing a system-wide clinical care intervention, considerations must be made for variability in culture and workflows at the state, health system, practice, and individual levels. Trial Registration: ClinicalTrials.gov NCT04255303; https://clinicaltrials.gov/ct2/show/NCT04255303 %M 38781006 %R 10.2196/54996 %U https://formative.jmir.org/2024/1/e54996 %U https://doi.org/10.2196/54996 %U http://www.ncbi.nlm.nih.gov/pubmed/38781006 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e54705 %T AI Quality Standards in Health Care: Rapid Umbrella Review %A Kuziemsky,Craig E %A Chrimes,Dillon %A Minshall,Simon %A Mannerow,Michael %A Lau,Francis %+ MacEwan University, 10700 104 Avenue, 7-257, Edmonton, AB, T5J4S2, Canada, 1 7806333290, kuziemskyc@macewan.ca %K artificial intelligence %K health care artificial intelligence %K health care AI %K rapid review %K umbrella review %K quality standard %D 2024 %7 22.5.2024 %9 Review %J J Med Internet Res %G English %X Background: In recent years, there has been an upwelling of artificial intelligence (AI) studies in the health care literature. During this period, there has been an increasing number of proposed standards to evaluate the quality of health care AI studies. Objective: This rapid umbrella review examines the use of AI quality standards in a sample of health care AI systematic review articles published over a 36-month period. Methods: We used a modified version of the Joanna Briggs Institute umbrella review method. Our rapid approach was informed by the practical guide by Tricco and colleagues for conducting rapid reviews. Our search was focused on the MEDLINE database supplemented with Google Scholar. The inclusion criteria were English-language systematic reviews regardless of review type, with mention of AI and health in the abstract, published during a 36-month period. For the synthesis, we summarized the AI quality standards used and issues noted in these reviews drawing on a set of published health care AI standards, harmonized the terms used, and offered guidance to improve the quality of future health care AI studies. Results: We selected 33 review articles published between 2020 and 2022 in our synthesis. The reviews covered a wide range of objectives, topics, settings, designs, and results. Over 60 AI approaches across different domains were identified with varying levels of detail spanning different AI life cycle stages, making comparisons difficult. Health care AI quality standards were applied in only 39% (13/33) of the reviews and in 14% (25/178) of the original studies from the reviews examined, mostly to appraise their methodological or reporting quality. Only a handful mentioned the transparency, explainability, trustworthiness, ethics, and privacy aspects. A total of 23 AI quality standard–related issues were identified in the reviews. There was a recognized need to standardize the planning, conduct, and reporting of health care AI studies and address their broader societal, ethical, and regulatory implications. Conclusions: Despite the growing number of AI standards to assess the quality of health care AI studies, they are seldom applied in practice. With increasing desire to adopt AI in different health topics, domains, and settings, practitioners and researchers must stay abreast of and adapt to the evolving landscape of health care AI quality standards and apply these standards to improve the quality of their AI studies. %M 38776538 %R 10.2196/54705 %U https://www.jmir.org/2024/1/e54705 %U https://doi.org/10.2196/54705 %U http://www.ncbi.nlm.nih.gov/pubmed/38776538 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e51013 %T Using a Semiautomated Procedure (CleanADHdata.R Script) to Clean Electronic Adherence Monitoring Data: Tutorial %A Bandiera,Carole %A Pasquier,Jérôme %A Locatelli,Isabella %A Schneider,Marie P %+ School of Pharmaceutical Sciences, University of Geneva, Rue Michel-Servet 1, Geneva, 1205, Switzerland, 41 223795316, marie.schneider@unige.ch %K medication adherence %K digital technology %K digital pharmacy %K electronic adherence monitoring %K data management %K data cleaning %K research methodology %K algorithms %K R %K semiautomated %K code %K coding %K computer science %K computer programming %K medications %K computer script %D 2024 %7 22.5.2024 %9 Tutorial %J JMIR Form Res %G English %X Background: Patient adherence to medications can be assessed using interactive digital health technologies such as electronic monitors (EMs). Changes in treatment regimens and deviations from EM use over time must be characterized to establish the actual level of medication adherence. Objective: We developed the computer script CleanADHdata.R to clean raw EM adherence data, and this tutorial is a guide for users. Methods: In addition to raw EM data, we collected adherence start and stop monitoring dates and identified the prescribed regimens, the expected number of EM openings per day based on the prescribed regimen, EM use deviations, and patients’ demographic data. The script formats the data longitudinally and calculates each day’s medication implementation. Results: We provided a simulated data set for 10 patients, for which 15 EMs were used over a median period of 187 (IQR 135-342) days. The median patient implementation before and after EM raw data cleaning was 83.3% (IQR 71.5%-93.9%) and 97.3% (IQR 95.8%-97.6%), respectively (Δ+14%). This difference is substantial enough to consider EM data cleaning to be capable of avoiding data misinterpretation and providing a cleaned data set for the adherence analysis in terms of implementation and persistence. Conclusions: The CleanADHdata.R script is a semiautomated procedure that increases standardization and reproducibility. This script has broader applicability within the realm of digital health, as it can be used to clean adherence data collected with diverse digital technologies. %M 38776539 %R 10.2196/51013 %U https://formative.jmir.org/2024/1/e51013 %U https://doi.org/10.2196/51013 %U http://www.ncbi.nlm.nih.gov/pubmed/38776539 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 11 %N %P e53894 %T Data-Driven Exploration of National Health Service Talking Therapies Care Pathways Using Process Mining: Retrospective Cohort Study %A Yardley,Elizabeth %A Davis,Alice %A Eldridge,Chris %A Vasilakis,Christos %+ School of Management, University of Bath, Claverton Down, Bath, BA2 7AY, United Kingdom, 44 01249 701100, egmy20@bath.ac.uk %K electronic health record %K EHR %K electronic health records %K EHRs %K health record %K data science %K secondary data analysis %K mental health services %K mental health %K health information system %K HIS %K information system %K information systems %K process mining %K flow %K flows %K path %K pathway %K pathways %K delivery %K visualization %D 2024 %7 21.5.2024 %9 Original Paper %J JMIR Ment Health %G English %X Background: The National Health Service (NHS) Talking Therapies program treats people with common mental health problems in England according to “stepped care,” in which lower-intensity interventions are offered in the first instance, where clinically appropriate. Limited resources and pressure to achieve service standards mean that program providers are exploring all opportunities to evaluate and improve the flow of patients through their service. Existing research has found variation in clinical performance and stepped care implementation across sites and has identified associations between service delivery and patient outcomes. Process mining offers a data-driven approach to analyzing and evaluating health care processes and systems, enabling comparison of presumed models of service delivery and their actual implementation in practice. The value and utility of applying process mining to NHS Talking Therapies data for the analysis of care pathways have not been studied. Objective: A better understanding of systems of service delivery will support improvements and planned program expansion. Therefore, this study aims to demonstrate the value and utility of applying process mining to NHS Talking Therapies care pathways using electronic health records. Methods: Routine collection of a wide variety of data regarding activity and patient outcomes underpins the Talking Therapies program. In our study, anonymized individual patient referral records from two sites over a 2-year period were analyzed using process mining to visualize the care pathway process by mapping the care pathway and identifying common pathway routes. Results: Process mining enabled the identification and visualization of patient flows directly from routinely collected data. These visualizations illustrated waiting periods and identified potential bottlenecks, such as the wait for higher-intensity cognitive behavioral therapy (CBT) at site 1. Furthermore, we observed that patients discharged from treatment waiting lists appeared to experience longer wait durations than those who started treatment. Process mining allowed analysis of treatment pathways, showing that patients commonly experienced treatment routes that involved either low- or high-intensity interventions alone. Of the most common routes, >5 times as many patients experienced direct access to high-intensity treatment rather than stepped care. Overall, 3.32% (site 1: 1507/45,401) and 4.19% (site 2: 527/12,590) of all patients experienced stepped care. Conclusions: Our findings demonstrate how process mining can be applied to Talking Therapies care pathways to evaluate pathway performance, explore relationships among performance issues, and highlight systemic issues, such as stepped care being relatively uncommon within a stepped care system. Integration of process mining capability into routine monitoring will enable NHS Talking Therapies service stakeholders to explore such issues from a process perspective. These insights will provide value to services by identifying areas for service improvement, providing evidence for capacity planning decisions, and facilitating better quality analysis into how health systems can affect patient outcomes. %M 38771630 %R 10.2196/53894 %U https://mental.jmir.org/2024/1/e53894 %U https://doi.org/10.2196/53894 %U http://www.ncbi.nlm.nih.gov/pubmed/38771630 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e53985 %T Longitudinal Changes in Diagnostic Accuracy of a Differential Diagnosis List Developed by an AI-Based Symptom Checker: Retrospective Observational Study %A Harada,Yukinori %A Sakamoto,Tetsu %A Sugimoto,Shu %A Shimizu,Taro %+ Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, 880 Kitakobayashi, Shimotsuga, 321-0293, Japan, 81 282 86 1111, yharada@dokkyomed.ac.jp %K atypical presentations %K diagnostic accuracy %K diagnosis %K diagnostics %K symptom checker %K uncommon diseases %K symptom checkers %K uncommon %K rare %K artificial intelligence %D 2024 %7 17.5.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Artificial intelligence (AI) symptom checker models should be trained using real-world patient data to improve their diagnostic accuracy. Given that AI-based symptom checkers are currently used in clinical practice, their performance should improve over time. However, longitudinal evaluations of the diagnostic accuracy of these symptom checkers are limited. Objective: This study aimed to assess the longitudinal changes in the accuracy of differential diagnosis lists created by an AI-based symptom checker used in the real world. Methods: This was a single-center, retrospective, observational study. Patients who visited an outpatient clinic without an appointment between May 1, 2019, and April 30, 2022, and who were admitted to a community hospital in Japan within 30 days of their index visit were considered eligible. We only included patients who underwent an AI-based symptom checkup at the index visit, and the diagnosis was finally confirmed during follow-up. Final diagnoses were categorized as common or uncommon, and all cases were categorized as typical or atypical. The primary outcome measure was the accuracy of the differential diagnosis list created by the AI-based symptom checker, defined as the final diagnosis in a list of 10 differential diagnoses created by the symptom checker. To assess the change in the symptom checker’s diagnostic accuracy over 3 years, we used a chi-square test to compare the primary outcome over 3 periods: from May 1, 2019, to April 30, 2020 (first year); from May 1, 2020, to April 30, 2021 (second year); and from May 1, 2021, to April 30, 2022 (third year). Results: A total of 381 patients were included. Common diseases comprised 257 (67.5%) cases, and typical presentations were observed in 298 (78.2%) cases. Overall, the accuracy of the differential diagnosis list created by the AI-based symptom checker was 172 (45.1%), which did not differ across the 3 years (first year: 97/219, 44.3%; second year: 32/72, 44.4%; and third year: 43/90, 47.7%; P=.85). The accuracy of the differential diagnosis list created by the symptom checker was low in those with uncommon diseases (30/124, 24.2%) and atypical presentations (12/83, 14.5%). In the multivariate logistic regression model, common disease (P<.001; odds ratio 4.13, 95% CI 2.50-6.98) and typical presentation (P<.001; odds ratio 6.92, 95% CI 3.62-14.2) were significantly associated with the accuracy of the differential diagnosis list created by the symptom checker. Conclusions: A 3-year longitudinal survey of the diagnostic accuracy of differential diagnosis lists developed by an AI-based symptom checker, which has been implemented in real-world clinical practice settings, showed no improvement over time. Uncommon diseases and atypical presentations were independently associated with a lower diagnostic accuracy. In the future, symptom checkers should be trained to recognize uncommon conditions. %M 38758588 %R 10.2196/53985 %U https://formative.jmir.org/2024/1/e53985 %U https://doi.org/10.2196/53985 %U http://www.ncbi.nlm.nih.gov/pubmed/38758588 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e52843 %T National Public Health Dashboards: Protocol for a Scoping Review %A Yanovitzky,Itzhak %A Stahlman,Gretchen %A Quow,Justine %A Ackerman,Matthew %A Perry,Yehuda %A Kim,Miriam %+ School of Communication & Information, Rutgers University, 4 Huntington St, New Brunswick, NJ, 08901, United States, 1 848 932 8852, itzhak@rutgers.edu %K dashboard %K scoping review %K public health %K design %K development %K implementation %K evaluation %K user need %K protocol %K data dashboards %K audiences %K audience %K systematic treatment %K public health data dashboards %K PRISMA-ScR %K snowballing techniques %K gray literature sources %K evidence-informed framework %K framework %K COVID-19 %K pandemic %D 2024 %7 16.5.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: The COVID-19 pandemic highlighted the importance of robust public health data systems and the potential utility of data dashboards for ensuring access to critical public health data for diverse groups of stakeholders and decision makers. As dashboards are becoming ubiquitous, it is imperative to consider how they may be best integrated with public health data systems and the decision-making routines of diverse audiences. However, additional progress on the continued development, improvement, and sustainability of these tools requires the integration and synthesis of a largely fragmented scholarship regarding the purpose, design principles and features, successful implementation, and decision-making supports provided by effective public health data dashboards across diverse users and applications. Objective: This scoping review aims to provide a descriptive and thematic overview of national public health data dashboards including their purpose, intended audiences, health topics, design elements, impact, and underlying mechanisms of use and usefulness of these tools in decision-making processes. It seeks to identify gaps in the current literature on the topic and provide the first-of-its-kind systematic treatment of actionability as a critical design element of public health data dashboards. Methods: The scoping review follows the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines. The review considers English-language, peer-reviewed journal papers, conference proceedings, book chapters, and reports that describe the design, implementation, and evaluation of a public health dashboard published between 2000 and 2023. The search strategy covers scholarly databases (CINAHL, PubMed, Medline, and Web of Science) and gray literature sources and uses snowballing techniques. An iterative process of testing for and improving intercoder reliability was implemented to ensure that coders are properly trained to screen documents according to the inclusion criteria prior to beginning the full review of relevant papers. Results: The search process initially identified 2544 documents, including papers located via databases, gray literature searching, and snowballing. Following the removal of duplicate documents (n=1416), nonrelevant items (n=839), and items classified as literature reviews and background information (n=73), 216 documents met the inclusion criteria: US case studies (n=90) and non-US case studies (n=126). Data extraction will focus on key variables, including public health data characteristics; dashboard design elements and functionalities; intended users, usability, logistics, and operation; and indicators of usefulness and impact reported. Conclusions: The scoping review will analyze the goals, design, use, usefulness, and impact of public health data dashboards. The review will also inform the continued development and improvement of these tools by analyzing and synthesizing current practices and lessons emerging from the literature on the topic and proposing a theory-grounded and evidence-informed framework for designing, implementing, and evaluating public health data dashboards. International Registered Report Identifier (IRRID): DERR1-10.2196/52843 %M 38753428 %R 10.2196/52843 %U https://www.researchprotocols.org/2024/1/e52843 %U https://doi.org/10.2196/52843 %U http://www.ncbi.nlm.nih.gov/pubmed/38753428 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e56267 %T Natural Language Processing for Work-Related Stress Detection Among Health Professionals: Protocol for a Scoping Review %A Bieri,Jannic Stefan %A Ikae,Catherine %A Souissi,Souhir Ben %A Müller,Thomas Jörg %A Schlunegger,Margarithe Charlotte %A Golz,Christoph %+ Department of Health Professions, Bern University of Applied Sciences, Murtenstrasse 10, Bern, 3008, Switzerland, 41 31 848 52 73, jannic.bieri@bfh.ch %K health professionals %K natural language processing %K text-mining %K work-related stress %K healthcare %K occupational well-being %K automatic detection %K scoping review protocol %K methodology %K synthesis %D 2024 %7 15.5.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: There is an urgent need worldwide for qualified health professionals. High attrition rates among health professionals, combined with a predicted rise in life expectancy, further emphasize the need for additional health professionals. Work-related stress is a major concern among health professionals, affecting both the well-being of health professionals and the quality of patient care. Objective: This scoping review aims to identify processes and methods for the automatic detection of work-related stress among health professionals using natural language processing (NLP) and text mining techniques. Methods: This review follows Joanna Briggs Institute Methodology and PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines. The inclusion criteria for this scoping review encompass studies involving health professionals using NLP for work-related stress detection while excluding studies involving other professions or children. The review focuses on various aspects, including NLP applications for stress detection, criteria for stress identification, technical aspects of NLP, and implications of stress detection through NLP. Studies within health care settings using diverse NLP techniques are considered, including experimental and observational designs, aiming to provide a comprehensive understanding of NLP’s role in detecting stress among health professionals. Studies published in English, German, or French from 2013 to present will be considered. The databases to be searched include MEDLINE (via PubMed), CINAHL, PubMed, Cochrane, ACM Digital Library, and IEEE Xplore. Sources of unpublished studies and gray literature to be searched will include ProQuest Dissertations & Theses and OpenGrey. Two reviewers will independently retrieve full-text studies and extract data. The collected data will be organized in tables, graphs, and a qualitative narrative summary. This review will use tables and graphs to present data on studies’ distribution by year, country, activity field, and research methods. Results synthesis involves identifying, grouping, and categorizing. The final scoping review will include a narrative written report detailing the search and study selection process, a visual representation using a PRISMA-ScR flow diagram, and a discussion of implications for practice and research. Results: We anticipate the outcomes will be presented in a systematic scoping review by June 2024. Conclusions: This review fills a literature gap by identifying automated work-related stress detection among health professionals using NLP and text mining, providing insights on an innovative approach, and identifying research needs for further systematic reviews. Despite promising outcomes, acknowledging limitations in the reviewed studies, including methodological constraints, sample biases, and potential oversight, is crucial to refining methodologies and advancing automatic stress detection among health professionals. International Registered Report Identifier (IRRID): PRR1-10.2196/56267 %M 38749026 %R 10.2196/56267 %U https://www.researchprotocols.org/2024/1/e56267 %U https://doi.org/10.2196/56267 %U http://www.ncbi.nlm.nih.gov/pubmed/38749026 %0 Journal Article %@ 2368-7959 %I %V 11 %N %P e56812 %T Coding of Childhood Psychiatric and Neurodevelopmental Disorders in Electronic Health Records of a Large Integrated Health Care System: Validation Study %A Shi,Jiaxiao M %A Chiu,Vicki Y %A Avila,Chantal C %A Lewis,Sierra %A Park,Daniella %A Peltier,Morgan R %A Getahun,Darios %K autism %K autism spectrum disorder %K ASD %K attention deficit hyperactivity disorder %K ADHD %K disruptive behavioral disorders %K DBD %K anxiety disorders %K AD %K major depression disorder %K MDD %K autistic %K coding %K neurodevelopmental %K psychiatric %K electronic health record %K electronic health records %K validation %K accuracy %K mental health %K emotional %K behavior %K behaviors %K behavioral %K disorder %K disorders %K pediatric %K pediatrics %K paediatric %K infant %K paediatrics %K infants %K infancy %K baby %K babies %K neonate %K neotnates %K neonatal %K toddler %K toddlers %K child %K children %K hospital %K hospitals %D 2024 %7 14.5.2024 %9 %J JMIR Ment Health %G English %X Background: Mental, emotional, and behavioral disorders are chronic pediatric conditions, and their prevalence has been on the rise over recent decades. Affected children have long-term health sequelae and a decline in health-related quality of life. Due to the lack of a validated database for pharmacoepidemiological research on selected mental, emotional, and behavioral disorders, there is uncertainty in their reported prevalence in the literature. Objectives: We aimed to evaluate the accuracy of coding related to pediatric mental, emotional, and behavioral disorders in a large integrated health care system’s electronic health records (EHRs) and compare the coding quality before and after the implementation of the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) coding as well as before and after the COVID-19 pandemic. Methods: Medical records of 1200 member children aged 2-17 years with at least 1 clinical visit before the COVID-19 pandemic (January 1, 2012, to December 31, 2014, the ICD-9-CM coding period; and January 1, 2017, to December 31, 2019, the ICD-10-CM coding period) and after the COVID-19 pandemic (January 1, 2021, to December 31, 2022) were selected with stratified random sampling from EHRs for chart review. Two trained research associates reviewed the EHRs for all potential cases of autism spectrum disorder (ASD), attention-deficit hyperactivity disorder (ADHD), major depression disorder (MDD), anxiety disorder (AD), and disruptive behavior disorders (DBD) in children during the study period. Children were considered cases only if there was a mention of any one of the conditions (yes for diagnosis) in the electronic chart during the corresponding time period. The validity of diagnosis codes was evaluated by directly comparing them with the gold standard of chart abstraction using sensitivity, specificity, positive predictive value, negative predictive value, the summary statistics of the F-score, and Youden J statistic. κ statistic for interrater reliability among the 2 abstractors was calculated. Results: The overall agreement between the identification of mental, behavioral, and emotional conditions using diagnosis codes compared to medical record abstraction was strong and similar across the ICD-9-CM and ICD-10-CM coding periods as well as during the prepandemic and pandemic time periods. The performance of AD coding, while strong, was relatively lower compared to the other conditions. The weighted sensitivity, specificity, positive predictive value, and negative predictive value for each of the 5 conditions were as follows: 100%, 100%, 99.2%, and 100%, respectively, for ASD; 100%, 99.9%, 99.2%, and 100%, respectively, for ADHD; 100%, 100%, 100%, and 100%, respectively for DBD; 87.7%, 100%, 100%, and 99.2%, respectively, for AD; and 100%, 100%, 99.2%, and 100%, respectively, for MDD. The F-score and Youden J statistic ranged between 87.7% and 100%. The overall agreement between abstractors was almost perfect (κ=95%). Conclusions: Diagnostic codes are quite reliable for identifying selected childhood mental, behavioral, and emotional conditions. The findings remained similar during the pandemic and after the implementation of the ICD-10-CM coding in the EHR system. %R 10.2196/56812 %U https://mental.jmir.org/2024/1/e56812 %U https://doi.org/10.2196/56812 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e45593 %T The Effects of Displaying the Time Targets of the Manchester Triage System to Emergency Department Personnel: Prospective Crossover Study %A Bienzeisler,Jonas %A Becker,Guido %A Erdmann,Bernadett %A Kombeiz,Alexander %A Majeed,Raphael W %A Röhrig,Rainer %A Greiner,Felix %A Otto,Ronny %A Otto-Sobotka,Fabian %A , %+ Institute of Medical Informatics, Medical Faculty, RWTH Aachen University, Pauwelsstraße 30, Aachen, 52074, Germany, 49 24180 ext 88870, jbienzeisler@ukaachen.de %K EHR %K emergency medicine %K AKTIN, process management %K crowding %K triage system %K electronic health record %K health care %K treatment %K emergency department %D 2024 %7 14.5.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: The use of triage systems such as the Manchester Triage System (MTS) is a standard procedure to determine the sequence of treatment in emergency departments (EDs). When using the MTS, time targets for treatment are determined. These are commonly displayed in the ED information system (EDIS) to ED staff. Using measurements as targets has been associated with a decline in meeting those targets. Objective: This study investigated the impact of displaying time targets for treatment to physicians on processing times in the ED. Methods: We analyzed the effects of displaying time targets to ED staff on waiting times in a prospective crossover study, during the introduction of a new EDIS in a large regional hospital in Germany. The old information system version used a module that showed the time target determined by the MTS, while the new system version used a priority list instead. Evaluation was based on 35,167 routinely collected electronic health records from the preintervention period and 10,655 records from the postintervention period. Electronic health records were extracted from the EDIS, and data were analyzed using descriptive statistics and generalized additive models. We evaluated the effects of the intervention on waiting times and the odds of achieving timely treatment according to the time targets set by the MTS. Results: The average ED length of stay and waiting times increased when the EDIS that did not display time targets was used (average time from admission to treatment: preintervention phase=median 15, IQR 6-39 min; postintervention phase=median 11, IQR 5-23 min). However, severe cases with high acuity (as indicated by the triage score) benefited from lower waiting times (0.15 times as high as in the preintervention period for MTS1, only 0.49 as high for MTS2). Furthermore, these patients were less likely to receive delayed treatment, and we observed reduced odds of late treatment when crowding occurred. Conclusions: Our results suggest that it is beneficial to use a priority list instead of displaying time targets to ED personnel. These time targets may lead to false incentives. Our work highlights that working better is not the same as working faster. %M 38743464 %R 10.2196/45593 %U https://www.jmir.org/2024/1/e45593 %U https://doi.org/10.2196/45593 %U http://www.ncbi.nlm.nih.gov/pubmed/38743464 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e53724 %T Evaluating the Diagnostic Performance of Large Language Models on Complex Multimodal Medical Cases %A Chiu,Wan Hang Keith %A Ko,Wei Sum Koel %A Cho,William Chi Shing %A Hui,Sin Yu Joanne %A Chan,Wing Chi Lawrence %A Kuo,Michael D %+ Ensemble Group, 10541 E Firewheel Drive, Scottsdale, AZ, 85259, United States, 1 4084512341, mikedkuo@gmail.com %K large language model %K hospital %K health center %K Massachusetts %K statistical analysis %K chi-square %K ANOVA %K clinician %K physician %K performance %K proficiency %K disease etiology %D 2024 %7 13.5.2024 %9 Research Letter %J J Med Internet Res %G English %X Large language models showed interpretative reasoning in solving diagnostically challenging medical cases. %M 38739441 %R 10.2196/53724 %U https://www.jmir.org/2024/1/e53724 %U https://doi.org/10.2196/53724 %U http://www.ncbi.nlm.nih.gov/pubmed/38739441 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e53787 %T The Role of Large Language Models in Transforming Emergency Medicine: Scoping Review %A Preiksaitis,Carl %A Ashenburg,Nicholas %A Bunney,Gabrielle %A Chu,Andrew %A Kabeer,Rana %A Riley,Fran %A Ribeira,Ryan %A Rose,Christian %+ Department of Emergency Medicine, Stanford University School of Medicine, 900 Welch Road, Suite 350, Palo Alto, CA, 94304, United States, 1 650 723 6576, cpreiksaitis@stanford.edu %K large language model %K LLM %K emergency medicine %K clinical decision support %K workflow efficiency %K medical education %K artificial intelligence %K AI %K natural language processing %K NLP %K AI literacy %K ChatGPT %K Bard %K Pathways Language Model %K Med-PaLM %K Bidirectional Encoder Representations from Transformers %K BERT %K generative pretrained transformer %K GPT %K United States %K US %K China %K scoping review %K Preferred Reporting Items for Systematic Reviews and Meta-Analyses %K PRISMA %K decision support %K workflow efficiency %K risk %K ethics %K education %K communication %K medical training %K physician %K health literacy %K emergency care %D 2024 %7 10.5.2024 %9 Review %J JMIR Med Inform %G English %X Background: Artificial intelligence (AI), more specifically large language models (LLMs), holds significant potential in revolutionizing emergency care delivery by optimizing clinical workflows and enhancing the quality of decision-making. Although enthusiasm for integrating LLMs into emergency medicine (EM) is growing, the existing literature is characterized by a disparate collection of individual studies, conceptual analyses, and preliminary implementations. Given these complexities and gaps in understanding, a cohesive framework is needed to comprehend the existing body of knowledge on the application of LLMs in EM. Objective: Given the absence of a comprehensive framework for exploring the roles of LLMs in EM, this scoping review aims to systematically map the existing literature on LLMs’ potential applications within EM and identify directions for future research. Addressing this gap will allow for informed advancements in the field. Methods: Using PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) criteria, we searched Ovid MEDLINE, Embase, Web of Science, and Google Scholar for papers published between January 2018 and August 2023 that discussed LLMs’ use in EM. We excluded other forms of AI. A total of 1994 unique titles and abstracts were screened, and each full-text paper was independently reviewed by 2 authors. Data were abstracted independently, and 5 authors performed a collaborative quantitative and qualitative synthesis of the data. Results: A total of 43 papers were included. Studies were predominantly from 2022 to 2023 and conducted in the United States and China. We uncovered four major themes: (1) clinical decision-making and support was highlighted as a pivotal area, with LLMs playing a substantial role in enhancing patient care, notably through their application in real-time triage, allowing early recognition of patient urgency; (2) efficiency, workflow, and information management demonstrated the capacity of LLMs to significantly boost operational efficiency, particularly through the automation of patient record synthesis, which could reduce administrative burden and enhance patient-centric care; (3) risks, ethics, and transparency were identified as areas of concern, especially regarding the reliability of LLMs’ outputs, and specific studies highlighted the challenges of ensuring unbiased decision-making amidst potentially flawed training data sets, stressing the importance of thorough validation and ethical oversight; and (4) education and communication possibilities included LLMs’ capacity to enrich medical training, such as through using simulated patient interactions that enhance communication skills. Conclusions: LLMs have the potential to fundamentally transform EM, enhancing clinical decision-making, optimizing workflows, and improving patient outcomes. This review sets the stage for future advancements by identifying key research areas: prospective validation of LLM applications, establishing standards for responsible use, understanding provider and patient perceptions, and improving physicians’ AI literacy. Effective integration of LLMs into EM will require collaborative efforts and thorough evaluation to ensure these technologies can be safely and effectively applied. %M 38728687 %R 10.2196/53787 %U https://medinform.jmir.org/2024/1/e53787 %U https://doi.org/10.2196/53787 %U http://www.ncbi.nlm.nih.gov/pubmed/38728687 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e49848 %T Development and Validation of an Explainable Deep Learning Model to Predict In-Hospital Mortality for Patients With Acute Myocardial Infarction: Algorithm Development and Validation Study %A Xie,Puguang %A Wang,Hao %A Xiao,Jun %A Xu,Fan %A Liu,Jingyang %A Chen,Zihang %A Zhao,Weijie %A Hou,Siyu %A Wu,Dongdong %A Ma,Yu %A Xiao,Jingjing %+ Bio-Med Informatics Research Centre & Clinical Research Centre, Xinqiao Hospital, Army Medical University, No. 183 Xinqiao Street, Shapingba District, Chongqing, 400037, China, 86 18502299862, shine636363@sina.com %K acute myocardial infarction %K mortality %K deep learning %K explainable model %K prediction %D 2024 %7 10.5.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Acute myocardial infarction (AMI) is one of the most severe cardiovascular diseases and is associated with a high risk of in-hospital mortality. However, the current deep learning models for in-hospital mortality prediction lack interpretability. Objective: This study aims to establish an explainable deep learning model to provide individualized in-hospital mortality prediction and risk factor assessment for patients with AMI. Methods: In this retrospective multicenter study, we used data for consecutive patients hospitalized with AMI from the Chongqing University Central Hospital between July 2016 and December 2022 and the Electronic Intensive Care Unit Collaborative Research Database. These patients were randomly divided into training (7668/10,955, 70%) and internal test (3287/10,955, 30%) data sets. In addition, data of patients with AMI from the Medical Information Mart for Intensive Care database were used for external validation. Deep learning models were used to predict in-hospital mortality in patients with AMI, and they were compared with linear and tree-based models. The Shapley Additive Explanations method was used to explain the model with the highest area under the receiver operating characteristic curve in both the internal test and external validation data sets to quantify and visualize the features that drive predictions. Results: A total of 10,955 patients with AMI who were admitted to Chongqing University Central Hospital or included in the Electronic Intensive Care Unit Collaborative Research Database were randomly divided into a training data set of 7668 (70%) patients and an internal test data set of 3287 (30%) patients. A total of 9355 patients from the Medical Information Mart for Intensive Care database were included for independent external validation. In-hospital mortality occurred in 8.74% (670/7668), 8.73% (287/3287), and 9.12% (853/9355) of the patients in the training, internal test, and external validation cohorts, respectively. The Self-Attention and Intersample Attention Transformer model performed best in both the internal test data set and the external validation data set among the 9 prediction models, with the highest area under the receiver operating characteristic curve of 0.86 (95% CI 0.84-0.88) and 0.85 (95% CI 0.84-0.87), respectively. Older age, high heart rate, and low body temperature were the 3 most important predictors of increased mortality, according to the explanations of the Self-Attention and Intersample Attention Transformer model. Conclusions: The explainable deep learning model that we developed could provide estimates of mortality and visual contribution of the features to the prediction for a patient with AMI. The explanations suggested that older age, unstable vital signs, and metabolic disorders may increase the risk of mortality in patients with AMI. %M 38728685 %R 10.2196/49848 %U https://www.jmir.org/2024/1/e49848 %U https://doi.org/10.2196/49848 %U http://www.ncbi.nlm.nih.gov/pubmed/38728685 %0 Journal Article %@ 2291-9694 %I %V 12 %N %P e51274 %T Evaluation of SNOMED CT Grouper Accuracy and Coverage in Organizing the Electronic Health Record Problem List by Clinical System: Observational Study %A Senior,Rashaud %A Tsai,Timothy %A Ratliff,William %A Nadler,Lisa %A Balu,Suresh %A Malcolm,Elizabeth %A McPeek Hinz,Eugenia %K electronic health record %K problem List %K problem list organization %K problem list management %K SNOMED CT %K SNOMED CT Groupers %K Systematized Nomenclature of Medicine %K clinical term %K ICD-10 %K International Classification of Diseases %D 2024 %7 9.5.2024 %9 %J JMIR Med Inform %G English %X Background: The problem list (PL) is a repository of diagnoses for patients’ medical conditions and health-related issues. Unfortunately, over time, our PLs have become overloaded with duplications, conflicting entries, and no-longer-valid diagnoses. The lack of a standardized structure for review adds to the challenges of clinical use. Previously, our default electronic health record (EHR) organized the PL primarily via alphabetization, with other options available, for example, organization by clinical systems or priority settings. The system’s PL was built with limited groupers, resulting in many diagnoses that were inconsistent with the expected clinical systems or not associated with any clinical systems at all. As a consequence of these limited EHR configuration options, our PL organization has poorly supported clinical use over time, particularly as the number of diagnoses on the PL has increased. Objective: We aimed to measure the accuracy of sorting PL diagnoses into PL system groupers based on Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT) concept groupers implemented in our EHR. Methods: We transformed and developed 21 system- or condition-based groupers, using 1211 SNOMED CT hierarchal concepts refined with Boolean logic, to reorganize the PL in our EHR. To evaluate the clinical utility of our new groupers, we extracted all diagnoses on the PLs from a convenience sample of 50 patients with 3 or more encounters in the previous year. To provide a spectrum of clinical diagnoses, we included patients from all ages and divided them by sex in a deidentified format. Two physicians independently determined whether each diagnosis was correctly attributed to the expected clinical system grouper. Discrepancies were discussed, and if no consensus was reached, they were adjudicated by a third physician. Descriptive statistics and Cohen κ statistics for interrater reliability were calculated. Results: Our 50-patient sample had a total of 869 diagnoses (range 4-59; median 12, IQR 9-24). The reviewers initially agreed on 821 system attributions. Of the remaining 48 items, 16 required adjudication with the tie-breaking third physician. The calculated κ statistic was 0.7. The PL groupers appropriately associated diagnoses to the expected clinical system with a sensitivity of 97.6%, a specificity of 58.7%, a positive predictive value of 96.8%, and an F1-score of 0.972. Conclusions: We found that PL organization by clinical specialty or condition using SNOMED CT concept groupers accurately reflects clinical systems. Our system groupers were subsequently adopted by our vendor EHR in their foundation system for PL organization. %R 10.2196/51274 %U https://medinform.jmir.org/2024/1/e51274 %U https://doi.org/10.2196/51274 %0 Journal Article %@ 2563-3570 %I JMIR Publications %V 5 %N %P e52700 %T ChatGPT and Medicine: Together We Embrace the AI Renaissance %A Hacking,Sean %+ NYU Langone, Tisch Hospital, 560 First Avenue, Suite TH 461, New York, NY, 10016, United States, 1 6466836133, hackingsean1@gmail.com %K ChatGPT %K generative AI %K NLP %K medicine %K bioinformatics %K AI democratization %K AI renaissance %K artificial intelligence %K natural language processing %D 2024 %7 7.5.2024 %9 Editorial %J JMIR Bioinform Biotech %G English %X The generative artificial intelligence (AI) model ChatGPT holds transformative prospects in medicine. The development of such models has signaled the beginning of a new era where complex biological data can be made more accessible and interpretable. ChatGPT is a natural language processing tool that can process, interpret, and summarize vast data sets. It can serve as a digital assistant for physicians and researchers, aiding in integrating medical imaging data with other multiomics data and facilitating the understanding of complex biological systems. The physician’s and AI’s viewpoints emphasize the value of such AI models in medicine, providing tangible examples of how this could enhance patient care. The editorial also discusses the rise of generative AI, highlighting its substantial impact in democratizing AI applications for modern medicine. While AI may not supersede health care professionals, practitioners incorporating AI into their practices could potentially have a competitive edge. %M 38935938 %R 10.2196/52700 %U https://bioinform.jmir.org/2024/1/e52700 %U https://doi.org/10.2196/52700 %U http://www.ncbi.nlm.nih.gov/pubmed/38935938 %0 Journal Article %@ 2563-3570 %I JMIR Publications %V 5 %N %P e56884 %T The Roles of NOTCH3 p.R544C and Thrombophilia Genes in Vietnamese Patients With Ischemic Stroke: Study Involving a Hierarchical Cluster Analysis %A Bui,Huong Thi Thu %A Nguyễn Thị Phương,Quỳnh %A Cam Tu,Ho %A Nguyen Phuong,Sinh %A Pham,Thuy Thi %A Vu,Thu %A Nguyen Thi Thu,Huyen %A Khanh Ho,Lam %A Nguyen Tien,Dung %+ Department of Internal Medicine, Thai Nguyen University of Medicine and Pharmacy, 284 Luong Ngoc Quyen, Quang Trung, Thai Nguyen, 250000, Vietnam, 84 913516863, dung.nt@tnmc.edu.vn %K Glasgow Coma Scale %K ischemic stroke %K hierarchical cluster analysis %K clustering %K machine learning %K MTHFR %K NOTCH3 %K modified Rankin scale %K National Institutes of Health Stroke Scale %K prothrombin %K thrombophilia %K mutations %K genetics %K genomics %K ischemia %K risk %K risk analysis %D 2024 %7 7.5.2024 %9 Original Paper %J JMIR Bioinform Biotech %G English %X Background: The etiology of ischemic stroke is multifactorial. Several gene mutations have been identified as leading causes of cerebral autosomal dominant arteriopathy with subcortical infarcts and leukoencephalopathy (CADASIL), a hereditary disease that causes stroke and other neurological symptoms. Objective: We aimed to identify the variants of NOTCH3 and thrombophilia genes, and their complex interactions with other factors. Methods: We conducted a hierarchical cluster analysis (HCA) on the data of 100 patients diagnosed with ischemic stroke. The variants of NOTCH3 and thrombophilia genes were identified by polymerase chain reaction with confronting 2-pair primers and real-time polymerase chain reaction. The overall preclinical characteristics, cumulative cutpoint values, and factors associated with these somatic mutations were analyzed in unidimensional and multidimensional scaling models. Results: We identified the following optimal cutpoints: creatinine, 83.67 (SD 9.19) µmol/L; age, 54 (SD 5) years; prothrombin (PT) time, 13.25 (SD 0.17) seconds; and international normalized ratio (INR), 1.02 (SD 0.03). Using the Nagelkerke method, cutpoint 50% values of the Glasgow Coma Scale score; modified Rankin scale score; and National Institutes of Health Stroke Scale scores at admission, after 24 hours, and at discharge were 12.77, 2.86 (SD 1.21), 9.83 (SD 2.85), 7.29 (SD 2.04), and 6.85 (SD 2.90), respectively. Conclusions: The variants of MTHFR (C677T and A1298C) and NOTCH3 p.R544C may influence the stroke severity under specific conditions of PT, creatinine, INR, and BMI, with risk ratios of 4.8 (95% CI 1.53-15.04) and 3.13 (95% CI 1.60-6.11), respectively (Pfisher<.05). It is interesting that although there are many genes linked to increased atrial fibrillation risk, not all of them are associated with ischemic stroke risk. With the detection of stroke risk loci, more information can be gained on their impacts and interconnections, especially in young patients. %M 38935968 %R 10.2196/56884 %U https://bioinform.jmir.org/2024/1/e56884 %U https://doi.org/10.2196/56884 %U http://www.ncbi.nlm.nih.gov/pubmed/38935968 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e54363 %T Improving the Prognostic Evaluation Precision of Hospital Outcomes for Heart Failure Using Admission Notes and Clinical Tabular Data: Multimodal Deep Learning Model %A Gao,Zhenyue %A Liu,Xiaoli %A Kang,Yu %A Hu,Pan %A Zhang,Xiu %A Yan,Wei %A Yan,Muyang %A Yu,Pengming %A Zhang,Qing %A Xiao,Wendong %A Zhang,Zhengbo %+ Center for Artificial Intelligence in Medicine, The General Hospital of People's Liberation Army, 28 Fuxing Road, Beijing, 100853, China, 86 010 68295454, zhangzhengbo@301hospital.com.cn %K heart failure %K multimodal deep learning %K mortality prediction %K admission notes %K clinical tabular data %K tabular %K notes %K deep learning %K machine learning %K cardiology %K heart %K cardiac %K documentation %K prognostic %K prognosis %K prognoses %K predict %K prediction %K predictions %K predictive %D 2024 %7 2.5.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Clinical notes contain contextualized information beyond structured data related to patients’ past and current health status. Objective: This study aimed to design a multimodal deep learning approach to improve the evaluation precision of hospital outcomes for heart failure (HF) using admission clinical notes and easily collected tabular data. Methods: Data for the development and validation of the multimodal model were retrospectively derived from 3 open-access US databases, including the Medical Information Mart for Intensive Care III v1.4 (MIMIC-III) and MIMIC-IV v1.0, collected from a teaching hospital from 2001 to 2019, and the eICU Collaborative Research Database v1.2, collected from 208 hospitals from 2014 to 2015. The study cohorts consisted of all patients with critical HF. The clinical notes, including chief complaint, history of present illness, physical examination, medical history, and admission medication, as well as clinical variables recorded in electronic health records, were analyzed. We developed a deep learning mortality prediction model for in-hospital patients, which underwent complete internal, prospective, and external evaluation. The Integrated Gradients and SHapley Additive exPlanations (SHAP) methods were used to analyze the importance of risk factors. Results: The study included 9989 (16.4%) patients in the development set, 2497 (14.1%) patients in the internal validation set, 1896 (18.3%) in the prospective validation set, and 7432 (15%) patients in the external validation set. The area under the receiver operating characteristic curve of the models was 0.838 (95% CI 0.827-0.851), 0.849 (95% CI 0.841-0.856), and 0.767 (95% CI 0.762-0.772), for the internal, prospective, and external validation sets, respectively. The area under the receiver operating characteristic curve of the multimodal model outperformed that of the unimodal models in all test sets, and tabular data contributed to higher discrimination. The medical history and physical examination were more useful than other factors in early assessments. Conclusions: The multimodal deep learning model for combining admission notes and clinical tabular data showed promising efficacy as a potentially novel method in evaluating the risk of mortality in patients with HF, providing more accurate and timely decision support. %M 38696251 %R 10.2196/54363 %U https://www.jmir.org/2024/1/e54363 %U https://doi.org/10.2196/54363 %U http://www.ncbi.nlm.nih.gov/pubmed/38696251 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e49910 %T IT-Related Barriers and Facilitators to the Implementation of a New European eHealth Solution, the Digital Survivorship Passport (SurPass Version 2.0): Semistructured Digital Survey %A de Beijer,Ismay A E %A van den Oever,Selina R %A Charalambous,Eliana %A Cangioli,Giorgio %A Balaguer,Julia %A Bardi,Edit %A Alfes,Marie %A Cañete Nieto,Adela %A Correcher,Marisa %A Pinto da Costa,Tiago %A Degelsegger-Márquez,Alexander %A Düster,Vanessa %A Filbert,Anna-Liesa %A Grabow,Desiree %A Gredinger,Gerald %A Gsell,Hannah %A Haupt,Riccardo %A van Helvoirt,Maria %A Ladenstein,Ruth %A Langer,Thorsten %A Laschkolnig,Anja %A Muraca,Monica %A Pluijm,Saskia M F %A Rascon,Jelena %A Schreier,Günter %A Tomášikova,Zuzana %A Trauner,Florian %A Trinkūnas,Justas %A Trunner,Kathrin %A Uyttebroeck,Anne %A Kremer,Leontien C M %A van der Pal,Helena J H %A Chronaki,Catherine %A , %+ Princess Máxima Center for Pediatric Oncology, Heidelberglaan 25, Utrecht, 3484 CS, Netherlands, 31 638960162, i.a.e.debeijer-3@prinsesmaximacentrum.nl %K pediatric oncology %K long-term follow up care %K survivorship %K cancer survivors %K Survivorship Passport %K SurPass, eHealth %K information and technology %D 2024 %7 2.5.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: To overcome knowledge gaps and optimize long-term follow-up (LTFU) care for childhood cancer survivors, the concept of the Survivorship Passport (SurPass) has been invented. Within the European PanCareSurPass project, the semiautomated and interoperable SurPass (version 2.0) will be optimized, implemented, and evaluated at 6 LTFU care centers representing 6 European countries and 3 distinct health system scenarios: (1) national electronic health information systems (EHISs) in Austria and Lithuania, (2) regional or local EHISs in Italy and Spain, and (3) cancer registries or hospital-based EHISs in Belgium and Germany. Objective: We aimed to identify and describe barriers and facilitators for SurPass (version 2.0) implementation concerning semiautomation of data input, interoperability, data protection, privacy, and cybersecurity. Methods: IT specialists from the 6 LTFU care centers participated in a semistructured digital survey focusing on IT-related barriers and facilitators to SurPass (version 2.0) implementation. We used the fit-viability model to assess the compatibility and feasibility of integrating SurPass into existing EHISs. Results: In total, 13/20 (65%) invited IT specialists participated. The main barriers and facilitators in all 3 health system scenarios related to semiautomated data input and interoperability included unaligned EHIS infrastructure and the use of interoperability frameworks and international coding systems. The main barriers and facilitators related to data protection or privacy and cybersecurity included pseudonymization of personal health data and data retention. According to the fit-viability model, the first health system scenario provides the best fit for SurPass implementation, followed by the second and third scenarios. Conclusions: This study provides essential insights into the information and IT-related influencing factors that need to be considered when implementing the SurPass (version 2.0) in clinical practice. We recommend the adoption of Health Level Seven Fast Healthcare Interoperability Resources and data security measures such as encryption, pseudonymization, and multifactor authentication to protect personal health data where applicable. In sum, this study offers practical insights into integrating digital health solutions into existing EHISs. %M 38696248 %R 10.2196/49910 %U https://www.jmir.org/2024/1/e49910 %U https://doi.org/10.2196/49910 %U http://www.ncbi.nlm.nih.gov/pubmed/38696248 %0 Journal Article %@ 2291-9694 %I %V 12 %N %P e50164 %T An Electronic Health Record–Integrated Application for Standardizing Care and Monitoring Patients With Autosomal Dominant Polycystic Kidney Disease Enrolled in a Tolvaptan Clinic: Design and Implementation Study %A Chedid,Maroun %A Chebib,Fouad T %A Dahlen,Erin %A Mueller,Theodore %A Schnell,Theresa %A Gay,Melissa %A Hommos,Musab %A Swaminathan,Sundararaman %A Garg,Arvind %A Mao,Michael %A Amberg,Brigid %A Balderes,Kirk %A Johnson,Karen F %A Bishop,Alyssa %A Vaughn,Jackqueline Kay %A Hogan,Marie %A Torres,Vicente %A Chaudhry,Rajeev %A Zoghby,Ziad %K ADPKD %K autosomal dominant polycystic kidney disease %K polycystic kidney disease %K tolvaptan %K EHR %K electronic health record %K digital health solutions %K monitoring %K kidney disease %K drug-related toxicity %K digital application %K management %K chronic disease %D 2024 %7 1.5.2024 %9 %J JMIR Med Inform %G English %X Background: Tolvaptan is the only US Food and Drug Administration–approved drug to slow the progression of autosomal dominant polycystic kidney disease (ADPKD), but it requires strict clinical monitoring due to potential serious adverse events. Objective: We aimed to share our experience in developing and implementing an electronic health record (EHR)–based application to monitor patients with ADPKD who were initiated on tolvaptan. Methods: The application was developed in collaboration with clinical informatics professionals based on our clinical protocol with frequent laboratory test monitoring to detect early drug-related toxicity. The application streamlined the clinical workflow and enabled our nursing team to take appropriate actions in real time to prevent drug-related serious adverse events. We retrospectively analyzed the characteristics of the enrolled patients. Results: As of September 2022, a total of 214 patients were enrolled in the tolvaptan program across all Mayo Clinic sites. Of these, 126 were enrolled in the Tolvaptan Monitoring Registry application and 88 in the Past Tolvaptan Patients application. The mean age at enrollment was 43.1 (SD 9.9) years. A total of 20 (9.3%) patients developed liver toxicity, but only 5 (2.3%) had to discontinue the drug. The 2 EHR-based applications allowed consolidation of all necessary patient information and real-time data management at the individual or population level. This approach facilitated efficient staff workflow, monitoring of drug-related adverse events, and timely prescription renewal. Conclusions: Our study highlights the feasibility of integrating digital applications into the EHR workflow to facilitate efficient and safe care delivery for patients enrolled in a tolvaptan program. This workflow needs further validation but could be extended to other health care systems managing chronic diseases requiring drug monitoring. %R 10.2196/50164 %U https://medinform.jmir.org/2024/1/e50164 %U https://doi.org/10.2196/50164 %0 Journal Article %@ 2291-9694 %I %V 12 %N %P e53535 %T Semantic Interoperability of Electronic Health Records: Systematic Review of Alternative Approaches for Enhancing Patient Information Availability %A Palojoki,Sari %A Lehtonen,Lasse %A Vuokko,Riikka %K electronic health record %K health records %K EHR %K EHRs %K semantic %K health care data %K semantic interoperability %K interoperability %K standardize %K standardized %K standardization %K cross-border data exchange %K systematic review %K synthesis %K syntheses %K review methods %K review methodology %K search %K searches %K searching %K systematic %K data exchange %K information sharing %K ontology %K ontologies %K terminology %K terminologies %K standard %K standards %K classification %K PRISMA %K data sharing %K Preferred Reporting Items for Systematic Reviews and Meta-Analyses %D 2024 %7 25.4.2024 %9 %J JMIR Med Inform %G English %X Background: Semantic interoperability facilitates the exchange of and access to health data that are being documented in electronic health records (EHRs) with various semantic features. The main goals of semantic interoperability development entail patient data availability and use in diverse EHRs without a loss of meaning. Internationally, current initiatives aim to enhance semantic development of EHR data and, consequently, the availability of patient data. Interoperability between health information systems is among the core goals of the European Health Data Space regulation proposal and the World Health Organization’s Global Strategy on Digital Health 2020-2025. Objective: To achieve integrated health data ecosystems, stakeholders need to overcome challenges of implementing semantic interoperability elements. To research the available scientific evidence on semantic interoperability development, we defined the following research questions: What are the key elements of and approaches for building semantic interoperability integrated in EHRs? What kinds of goals are driving the development? and What kinds of clinical benefits are perceived following this development? Methods: Our research questions focused on key aspects and approaches for semantic interoperability and on possible clinical and semantic benefits of these choices in the context of EHRs. Therefore, we performed a systematic literature review in PubMed by defining our study framework based on previous research. Results: Our analysis consisted of 14 studies where data models, ontologies, terminologies, classifications, and standards were applied for building interoperability. All articles reported clinical benefits of the selected approach to enhancing semantic interoperability. We identified 3 main categories: increasing the availability of data for clinicians (n=6, 43%), increasing the quality of care (n=4, 29%), and enhancing clinical data use and reuse for varied purposes (n=4, 29%). Regarding semantic development goals, data harmonization and developing semantic interoperability between different EHRs was the largest category (n=8, 57%). Enhancing health data quality through standardization (n=5, 36%) and developing EHR-integrated tools based on interoperable data (n=1, 7%) were the other identified categories. The results were closely coupled with the need to build usable and computable data out of heterogeneous medical information that is accessible through various EHRs and databases (eg, registers). Conclusions: When heading toward semantic harmonization of clinical data, more experiences and analyses are needed to assess how applicable the chosen solutions are for semantic interoperability of health care data. Instead of promoting a single approach, semantic interoperability should be assessed through several levels of semantic requirements A dual model or multimodel approach is possibly usable to address different semantic interoperability issues during development. The objectives of semantic interoperability are to be achieved in diffuse and disconnected clinical care environments. Therefore, approaches for enhancing clinical data availability should be well prepared, thought out, and justified to meet economically sustainable and long-term outcomes. %R 10.2196/53535 %U https://medinform.jmir.org/2024/1/e53535 %U https://doi.org/10.2196/53535 %0 Journal Article %@ 2561-1011 %I JMIR Publications %V 8 %N %P e53091 %T Use of Machine Learning for Early Detection of Maternal Cardiovascular Conditions: Retrospective Study Using Electronic Health Record Data %A Shara,Nawar %A Mirabal-Beltran,Roxanne %A Talmadge,Bethany %A Falah,Noor %A Ahmad,Maryam %A Dempers,Ramon %A Crovatt,Samantha %A Eisenberg,Steven %A Anderson,Kelley %+ School of Nursing, Georgetown University, 3700 Reservoir Road, NW, Washington, DC, 20057, United States, 1 2026873496, rm1910@georgetown.edu %K machine learning %K preeclampsia %K cardiovascular %K maternal %K obstetrics %K health disparities %K woman %K women %K pregnancy %K pregnant %K cardiovascular %K cardiovascular condition %K retrospective study %K electronic health record %K EHR %K technology %K decision-making %K health disparity %K virtual server %K thromboembolism %K kidney failure %K HOPE-CAT %D 2024 %7 22.4.2024 %9 Original Paper %J JMIR Cardio %G English %X Background: Cardiovascular conditions (eg, cardiac and coronary conditions, hypertensive disorders of pregnancy, and cardiomyopathies) were the leading cause of maternal mortality between 2017 and 2019. The United States has the highest maternal mortality rate of any high-income nation, disproportionately impacting those who identify as non-Hispanic Black or Hispanic. Novel clinical approaches to the detection and diagnosis of cardiovascular conditions are therefore imperative. Emerging research is demonstrating that machine learning (ML) is a promising tool for detecting patients at increased risk for hypertensive disorders during pregnancy. However, additional studies are required to determine how integrating ML and big data, such as electronic health records (EHRs), can improve the identification of obstetric patients at higher risk of cardiovascular conditions. Objective: This study aimed to evaluate the capability and timing of a proprietary ML algorithm, Healthy Outcomes for all Pregnancy Experiences-Cardiovascular-Risk Assessment Technology (HOPE-CAT), to detect maternal-related cardiovascular conditions and outcomes. Methods: Retrospective data from the EHRs of a large health care system were investigated by HOPE-CAT in a virtual server environment. Deidentification of EHR data and standardization enabled HOPE-CAT to analyze data without pre-existing biases. The ML algorithm assessed risk factors selected by clinical experts in cardio-obstetrics, and the algorithm was iteratively trained using relevant literature and current standards of risk identification. After refinement of the algorithm’s learned risk factors, risk profiles were generated for every patient including a designation of standard versus high risk. The profiles were individually paired with clinical outcomes pertaining to cardiovascular pregnancy conditions and complications, wherein a delta was calculated between the date of the risk profile and the actual diagnosis or intervention in the EHR. Results: In total, 604 pregnancies resulting in birth had records or diagnoses that could be compared against the risk profile; the majority of patients identified as Black (n=482, 79.8%) and aged between 21 and 34 years (n=509, 84.4%). Preeclampsia (n=547, 90.6%) was the most common condition, followed by thromboembolism (n=16, 2.7%) and acute kidney disease or failure (n=13, 2.2%). The average delta was 56.8 (SD 69.7) days between the identification of risk factors by HOPE-CAT and the first date of diagnosis or intervention of a related condition reported in the EHR. HOPE-CAT showed the strongest performance in early risk detection of myocardial infarction at a delta of 65.7 (SD 81.4) days. Conclusions: This study provides additional evidence to support ML in obstetrical patients to enhance the early detection of cardiovascular conditions during pregnancy. ML can synthesize multiday patient presentations to enhance provider decision-making and potentially reduce maternal health disparities. %M 38648629 %R 10.2196/53091 %U https://cardio.jmir.org/2024/1/e53091 %U https://doi.org/10.2196/53091 %U http://www.ncbi.nlm.nih.gov/pubmed/38648629 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e54419 %T Using ChatGPT-4 to Create Structured Medical Notes From Audio Recordings of Physician-Patient Encounters: Comparative Study %A Kernberg,Annessa %A Gold,Jeffrey A %A Mohan,Vishnu %+ Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Sciences University, 3181 SW Sam Jackson Park Road, Portland, OR, 97239, United States, 1 5034944469, mohanV@ohsu.edu %K generative AI %K generative artificial intelligence %K ChatGPT %K simulation %K large language model %K clinical documentation %K quality %K accuracy %K reproducibility %K publicly available %K medical note %K medical notes %K generation %K medical documentation %K documentation %K documentations %K AI %K artificial intelligence %K transcript %K transcripts %K ChatGPT-4 %D 2024 %7 22.4.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Medical documentation plays a crucial role in clinical practice, facilitating accurate patient management and communication among health care professionals. However, inaccuracies in medical notes can lead to miscommunication and diagnostic errors. Additionally, the demands of documentation contribute to physician burnout. Although intermediaries like medical scribes and speech recognition software have been used to ease this burden, they have limitations in terms of accuracy and addressing provider-specific metrics. The integration of ambient artificial intelligence (AI)–powered solutions offers a promising way to improve documentation while fitting seamlessly into existing workflows. Objective: This study aims to assess the accuracy and quality of Subjective, Objective, Assessment, and Plan (SOAP) notes generated by ChatGPT-4, an AI model, using established transcripts of History and Physical Examination as the gold standard. We seek to identify potential errors and evaluate the model’s performance across different categories. Methods: We conducted simulated patient-provider encounters representing various ambulatory specialties and transcribed the audio files. Key reportable elements were identified, and ChatGPT-4 was used to generate SOAP notes based on these transcripts. Three versions of each note were created and compared to the gold standard via chart review; errors generated from the comparison were categorized as omissions, incorrect information, or additions. We compared the accuracy of data elements across versions, transcript length, and data categories. Additionally, we assessed note quality using the Physician Documentation Quality Instrument (PDQI) scoring system. Results: Although ChatGPT-4 consistently generated SOAP-style notes, there were, on average, 23.6 errors per clinical case, with errors of omission (86%) being the most common, followed by addition errors (10.5%) and inclusion of incorrect facts (3.2%). There was significant variance between replicates of the same case, with only 52.9% of data elements reported correctly across all 3 replicates. The accuracy of data elements varied across cases, with the highest accuracy observed in the “Objective” section. Consequently, the measure of note quality, assessed by PDQI, demonstrated intra- and intercase variance. Finally, the accuracy of ChatGPT-4 was inversely correlated to both the transcript length (P=.05) and the number of scorable data elements (P=.05). Conclusions: Our study reveals substantial variability in errors, accuracy, and note quality generated by ChatGPT-4. Errors were not limited to specific sections, and the inconsistency in error types across replicates complicated predictability. Transcript length and data complexity were inversely correlated with note accuracy, raising concerns about the model’s effectiveness in handling complex medical cases. The quality and reliability of clinical notes produced by ChatGPT-4 do not meet the standards required for clinical use. Although AI holds promise in health care, caution should be exercised before widespread adoption. Further research is needed to address accuracy, variability, and potential errors. ChatGPT-4, while valuable in various applications, should not be considered a safe alternative to human-generated clinical documentation at this time. %M 38648636 %R 10.2196/54419 %U https://www.jmir.org/2024/1/e54419 %U https://doi.org/10.2196/54419 %U http://www.ncbi.nlm.nih.gov/pubmed/38648636 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e56262 %T TAILR (Nursing-Sensitive Events and Their Association With Individual Nurse Staffing Levels) Project: Protocol for an International Longitudinal Multicenter Study %A Bachnick,Stefanie %A Unbeck,Maria %A Ahmadi Shad,Maryam %A Falta,Katja %A Grossmann,Nicole %A Holle,Daniela %A Bartakova,Jana %A Musy,Sarah N %A Hellberg,Sarah %A Dillner,Pernilla %A Atoof,Fatemeh %A Khorasanizadeh,Mohammadhossein %A Kelly-Pettersson,Paula %A Simon,Michael %+ Department of Nursing Science, University of Applied Sciences, Gesundheitscampus 6 – 8, Bochum, 44801, Germany, 49 234 77727 748, stefanie.bachnick@hs-gesundheit.de %K adverse events %K electronic health record %K hospital care %K no-harm incidents %K nursing care %K nursing-sensitive events %K nurse staffing %K patient safety %K systematic record review %D 2024 %7 22.4.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Nursing-sensitive events (NSEs) are common, accounting for up to 77% of adverse events in hospitalized patients (eg, fall-related harm, pressure ulcers, and health care–associated infections). NSEs lead to adverse patient outcomes and impose an economic burden on hospitals due to increased medical costs through a prolonged hospital stay and additional medical procedures. To reduce NSEs and ensure high-quality nursing care, appropriate nurse staffing levels are needed. Although the link between nurse staffing and NSEs has been described in many studies, appropriate nurse staffing levels are lacking. Existing studies describe constant staffing exposure at the unit or hospital level without assessing patient-level exposure to nurse staffing during the hospital stay. Few studies have assessed nurse staffing and patient outcomes using a single-center longitudinal design, with limited generalizability. There is a need for multicenter longitudinal studies with improved potential for generalizing the association between individual nurse staffing levels and NSEs. Objective: This study aimed (1) to determine the prevalence, preventability, type, and severity of NSEs; (2) to describe individual patient-level nurse staffing exposure across hospitals; (3) to assess the effect of nurse staffing on NSEs in patients; and (4) to identify thresholds of safe nurse staffing levels and test them against NSEs in hospitalized patients. Methods: This international multicenter study uses a longitudinal and observational research design; it involves 4 countries (Switzerland, Sweden, Germany, and Iran), with participation from 14 hospitals and 61 medical, surgery, and mixed units. The 16-week observation period will collect NSEs using systematic retrospective record reviews. A total of 3680 patient admissions will be reviewed, with 60 randomly selected admissions per unit. To be included, patients must have been hospitalized for at least 48 hours. Nurse staffing data (ie, the number of nurses and their education level) will be collected daily for each shift to assess the association between NSEs and individual nurse staffing levels. Additionally, hospital data (ie, type, teaching status, and ownership) and unit data (ie, service line and number of beds) will be collected. Results: As of January 2024, the verification process for the plausibility and comprehensibility of patients’ and nurse staffing data is underway across all 4 countries. Data analyses are planned to be completed by spring 2024, with the first results expected to be published in late 2024. Conclusions: This study will provide comprehensive information on NSEs, including their prevalence, preventability, type, and severity, across countries. Moreover, it seeks to enhance understanding of NSE mechanisms and the potential impact of nurse staffing on these events. We will evaluate within- and between-hospital variability to identify productive strategies to ensure safe nurse staffing levels, thereby reducing NSEs in hospitalized patients. The TAILR (Nursing-Sensitive Events and Their Association With Individual Nurse Staffing Levels) study will focus on the optimization of scarce staffing resources. International Registered Report Identifier (IRRID): DERR1-10.2196/56262 %M 38648083 %R 10.2196/56262 %U https://www.researchprotocols.org/2024/1/e56262 %U https://doi.org/10.2196/56262 %U http://www.ncbi.nlm.nih.gov/pubmed/38648083 %0 Journal Article %@ 2291-9694 %I %V 12 %N %P e52343 %T CHDmap: One Step Further Toward Integrating Medicine-Based Evidence Into Practice %A Van den Eynde,Jef %K artificial intelligence %K clinical practice %K congenital heart disease %K decision-making %K evidence-based medicine %K machine learning %K medicine-based evidence %K patient similarity networks %K precision medicine %K randomized controlled trials %D 2024 %7 19.4.2024 %9 %J JMIR Med Inform %G English %X Evidence-based medicine, rooted in randomized controlled trials, offers treatment estimates for the average patient but struggles to guide individualized care. This challenge is amplified in complex conditions like congenital heart disease due to disease variability and limited trial applicability. To address this, medicine-based evidence was proposed to synthesize information for personalized care. A recent article introduced a patient similarity network, CHDmap, which represents a promising technical rendition of the medicine-based evidence concept. Leveraging comprehensive clinical and echocardiographic data, CHDmap creates an interactive patient map representing individuals with similar attributes. Using a k-nearest neighbor algorithm, CHDmap interactively identifies closely resembling patient groups based on specific characteristics. These approximate matches form the foundation for predictive analyses, including outcomes like hospital length of stay and complications. A key finding is the tool’s dual capacity: not only did it corroborate clinical intuition in many scenarios, but in specific instances, it prompted a reevaluation of cases, culminating in an enhancement of overall performance across various classification tasks. While an important first step, future versions of CHDmap may aim to expand mapping complexity, increase data granularity, consider long-term outcomes, allow for treatment comparisons, and implement artificial intelligence–driven weighting of various input variables. Successful implementation of CHDmap and similar tools will require training for practitioners, robust data infrastructure, and interdisciplinary collaboration. Patient similarity networks may become valuable in multidisciplinary discussions, complementing clinicians’ expertise. The symbiotic approach bridges evidence, experience, and real-life care, enabling iterative learning for future physicians. %R 10.2196/52343 %U https://medinform.jmir.org/2024/1/e52343 %U https://doi.org/10.2196/52343 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e55202 %T A Health Information Technology Protocol to Enhance Colorectal Cancer Screening %A Baus,Adam %A Boatman,Dannell D %A Calkins,Andrea %A Pollard,Cecil %A Conn,Mary Ellen %A Subramanian,Sujha %A Kennedy-Rea,Stephenie %+ Department of Social and Behavioral Sciences, School of Public Health, West Virginia University, 64 Medical Center Drive, PO Box 9190, Morgantown, WV, 26506, United States, 1 304 293 1083, abaus@hsc.wvu.edu %K electronic health record %K EHR %K colorectal cancer screening %K health information technology %K cancer %K colorectal cancer %D 2024 %7 19.4.2024 %9 Research Letter %J JMIR Form Res %G English %X This study addresses barriers to electronic health records–based colorectal cancer screening and follow-up in primary care through the development and implementation of a health information technology protocol. %M 38640474 %R 10.2196/55202 %U https://formative.jmir.org/2024/1/e55202 %U https://doi.org/10.2196/55202 %U http://www.ncbi.nlm.nih.gov/pubmed/38640474 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e52592 %T Academic Detailing as a Health Information Technology Implementation Method: Supporting the Design and Implementation of an Emergency Department–Based Clinical Decision Support Tool to Prevent Future Falls %A Barton,Hanna J %A Maru,Apoorva %A Leaf,Margaret A %A Hekman,Daniel J %A Wiegmann,Douglas A %A Shah,Manish N %A Patterson,Brian W %+ BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, 800 University Bay Dr., Madison, WI, 53705, United States, 1 (608) 890 8682, hbarton@wisc.edu %K emergency medicine %K clinical decision support %K health IT %K human factors %K work systems %K SEIPS %K Systems Engineering Initiative for Patient Safety %K educational outreach %K academic detailing %K implementation method %K department-based %K CDS %K clinical care %K evidence-based %K CDS tool %K gerontology %K geriatric %K geriatrics %K older adult %K older adults %K elder %K elderly %K older person %K older people %K preventative intervention %K team-based analysis %K machine learning %K high-risk patient %K high-risk patients %K pharmaceutical %K pharmaceutical sales %K United States %K fall-risk prediction %K EHR %K electronic health record %K interview %K ED environment %K emergency department %D 2024 %7 18.4.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Clinical decision support (CDS) tools that incorporate machine learning–derived content have the potential to transform clinical care by augmenting clinicians’ expertise. To realize this potential, such tools must be designed to fit the dynamic work systems of the clinicians who use them. We propose the use of academic detailing—personal visits to clinicians by an expert in a specific health IT tool—as a method for both ensuring the correct understanding of that tool and its evidence base and identifying factors influencing the tool’s implementation. Objective: This study aimed to assess academic detailing as a method for simultaneously ensuring the correct understanding of an emergency department–based CDS tool to prevent future falls and identifying factors impacting clinicians’ use of the tool through an analysis of the resultant qualitative data. Methods: Previously, our team designed a CDS tool to identify patients aged 65 years and older who are at the highest risk of future falls and prompt an interruptive alert to clinicians, suggesting the patient be referred to a mobility and falls clinic for an evidence-based preventative intervention. We conducted 10-minute academic detailing interviews (n=16) with resident emergency medicine physicians and advanced practice providers who had encountered our CDS tool in practice. We conducted an inductive, team-based content analysis to identify factors that influenced clinicians’ use of the CDS tool. Results: The following categories of factors that impacted clinicians’ use of the CDS were identified: (1) aspects of the CDS tool’s design (2) clinicians’ understanding (or misunderstanding) of the CDS or referral process, (3) the busy nature of the emergency department environment, (4) clinicians’ perceptions of the patient and their associated fall risk, and (5) the opacity of the referral process. Additionally, clinician education was done to address any misconceptions about the CDS tool or referral process, for example, demonstrating how simple it is to place a referral via the CDS and clarifying which clinic the referral goes to. Conclusions: Our study demonstrates the use of academic detailing for supporting the implementation of health information technologies, allowing us to identify factors that impacted clinicians’ use of the CDS while concurrently educating clinicians to ensure the correct understanding of the CDS tool and intervention. Thus, academic detailing can inform both real-time adjustments of a tool’s implementation, for example, refinement of the language used to introduce the tool, and larger scale redesign of the CDS tool to better fit the dynamic work environment of clinicians. %M 38635318 %R 10.2196/52592 %U https://humanfactors.jmir.org/2024/1/e52592 %U https://doi.org/10.2196/52592 %U http://www.ncbi.nlm.nih.gov/pubmed/38635318 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e54838 %T Global Trends of Medical Misadventures Using International Classification of Diseases, Tenth Revision Cluster Y62-Y69 Comparing Pre–, Intra–, and Post–COVID-19 Pandemic Phases: Protocol for a Retrospective Analysis Using the TriNetX Platform %A Caruso,Rosario %A Di Muzio,Marco %A Di Simone,Emanuele %A Dionisi,Sara %A Magon,Arianna %A Conte,Gianluca %A Stievano,Alessandro %A Girani,Emanuele %A Boveri,Sara %A Menicanti,Lorenzo %A Dolansky,Mary A %+ Health Professions Research and Development Unit, IRCCS Policlinico San Donato, via morandi 30, San Donato Milanese, 20097, Italy, 39 025277 ext 4940, rosario.caruso@grupposandonato.it %K COVID-19 %K curve-fitting analyses %K health care quality %K health care safety %K International Classification of Diseases, Tenth Revision %K ICD-10 %K incidence rates %K safety %K TriNetX %D 2024 %7 17.4.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: The COVID-19 pandemic has sharpened the focus on health care safety and quality, underscoring the importance of using standardized metrics such as the International Classification of Diseases, Tenth Revision (ICD-10). In this regard, the ICD-10 cluster Y62-Y69 serves as a proxy assessment of safety and quality in health care systems, allowing researchers to evaluate medical misadventures. Thus far, extensive research and reports support the need for more attention to safety and quality in health care. The study aims to leverage the pandemic’s unique challenges to explore health care safety and quality trends during prepandemic, intrapandemic, and postpandemic phases, using the ICD-10 cluster Y62-Y69 as a key tool for their evaluation. Objective: This research aims to perform a comprehensive retrospective analysis of incidence rates associated with ICD-10 cluster Y62-Y69, capturing both linear and nonlinear trends across prepandemic, intrapandemic, and postpandemic phases over an 8-year span. Therefore, it seeks to understand how these trends inform health care safety and quality improvements, policy, and future research. Methods: This study uses the extensive data available through the TriNetX platform, using an observational, retrospective design and applying curve-fitting analyses and quadratic models to comprehend the relationships between incidence rates over an 8-year span (from 2015 to 2023). These techniques will enable the identification of nuanced trends in the data, facilitating a deeper understanding of the impacts of the COVID-19 pandemic on medical misadventures. The anticipated results aim to outline complex patterns in health care safety and quality during the COVID-19 pandemic, using global real-world data for robust and generalizable conclusions. This study will explore significant shifts in health care practices and outcomes, with a special focus on geographical variations and key clinical conditions in cardiovascular and oncological care, ensuring a comprehensive analysis of the pandemic’s impact across different regions and medical fields. Results: This study is currently in the data collection phase, with funding secured in November 2023 through the Ricerca Corrente scheme of the Italian Ministry of Health. Data collection via the TriNetX platform is anticipated to be completed in May 2024, covering an 8-year period from January 2015 to December 2023. This dataset spans pre-pandemic, intra-pandemic, and early post-pandemic phases, enabling a comprehensive analysis of trends in medical misadventures using the ICD-10 cluster Y62-Y69. The final analytics are anticipated to be completed by June 2024. The study's findings aim to provide actionable insights for enhancing healthcare safety and quality, reflecting on the pandemic's transformative impact on global healthcare systems. Conclusions: This study is anticipated to contribute significantly to health care safety and quality literature. It will provide actionable insights for health care professionals, policy makers, and researchers. It will highlight critical areas for intervention and funding to enhance health care safety and quality globally by examining the incidence rates of medical misadventures before, during, and after the pandemic. In addition, the use of global real-world data enhances the study’s strength by providing a practical view of health care safety and quality, paving the way for initiatives that are informed by data and tailored to specific contexts worldwide. This approach ensures the findings are applicable and actionable across different health care settings, contributing significantly to the global understanding and improvement of health care safety and quality. International Registered Report Identifier (IRRID): PRR1-10.2196/54838 %M 38630516 %R 10.2196/54838 %U https://www.researchprotocols.org/2024/1/e54838 %U https://doi.org/10.2196/54838 %U http://www.ncbi.nlm.nih.gov/pubmed/38630516 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e50475 %T Integrating Explainable Machine Learning in Clinical Decision Support Systems: Study Involving a Modified Design Thinking Approach %A Shulha,Michael %A Hovdebo,Jordan %A D’Souza,Vinita %A Thibault,Francis %A Harmouche,Rola %+ Lady Davis Institute for Medical Research, Jewish General Hospital, Centre intégré universitaire de santé et de services sociaux (CIUSSS) du Centre-Ouest-de-l'Île-de-Montréal, Pavilion B-274, 3755 Chem. de la Côte-Sainte-Catherine, Montreal, QC, H3T 1E2, Canada, 1 514 340 8222, michael.shulha.ccomtl@ssss.gouv.qc.ca %K explainable machine learning %K XML %K design thinking approach %K NASSS framework %K clinical decision support %K clinician engagement %K clinician-facing interface %K clinician trust in machine learning %K COVID-19 %K chest x-ray %K severity prediction %D 2024 %7 16.4.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Though there has been considerable effort to implement machine learning (ML) methods for health care, clinical implementation has lagged. Incorporating explainable machine learning (XML) methods through the development of a decision support tool using a design thinking approach is expected to lead to greater uptake of such tools. Objective: This work aimed to explore how constant engagement of clinician end users can address the lack of adoption of ML tools in clinical contexts due to their lack of transparency and address challenges related to presenting explainability in a decision support interface. Methods: We used a design thinking approach augmented with additional theoretical frameworks to provide more robust approaches to different phases of design. In particular, in the problem definition phase, we incorporated the nonadoption, abandonment, scale-up, spread, and sustainability of technology in health care (NASSS) framework to assess these aspects in a health care network. This process helped focus on the development of a prognostic tool that predicted the likelihood of admission to an intensive care ward based on disease severity in chest x-ray images. In the ideate, prototype, and test phases, we incorporated a metric framework to assess physician trust in artificial intelligence (AI) tools. This allowed us to compare physicians’ assessments of the domain representation, action ability, and consistency of the tool. Results: Physicians found the design of the prototype elegant, and domain appropriate representation of data was displayed in the tool. They appreciated the simplified explainability overlay, which only displayed the most predictive patches that cumulatively explained 90% of the final admission risk score. Finally, in terms of consistency, physicians unanimously appreciated the capacity to compare multiple x-ray images in the same view. They also appreciated the ability to toggle the explainability overlay so that both options made it easier for them to assess how consistently the tool was identifying elements of the x-ray image they felt would contribute to overall disease severity. Conclusions: The adopted approach is situated in an evolving space concerned with incorporating XML or AI technologies into health care software. We addressed the alignment of AI as it relates to clinician trust, describing an approach to wire framing and prototyping, which incorporates the use of a theoretical framework for trust in the design process itself. Moreover, we proposed that alignment of AI is dependent upon integration of end users throughout the larger design process. Our work shows the importance and value of engaging end users prior to tool development. We believe that the described approach is a unique and valuable contribution that outlines a direction for ML experts, user experience designers, and clinician end users on how to collaborate in the creation of trustworthy and usable XML-based clinical decision support tools. %M 38625728 %R 10.2196/50475 %U https://formative.jmir.org/2024/1/e50475 %U https://doi.org/10.2196/50475 %U http://www.ncbi.nlm.nih.gov/pubmed/38625728 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e52412 %T Novel Approach for Detecting Respiratory Syncytial Virus in Pediatric Patients Using Machine Learning Models Based on Patient-Reported Symptoms: Model Development and Validation Study %A Kawamoto,Shota %A Morikawa,Yoshihiko %A Yahagi,Naohisa %+ Graduate School of Media and Governance, Keio University, 5322 Endo, Fujisawa, 252-0882, Japan, 81 466 49 3404, yahagin@sfc.keio.ac.jp %K respiratory syncytial virus %K machine learning %K self-reported information %K clinical decision support system %K decision support %K decision-making %K artificial intelligence %K model development %K evaluation study %K detection %K respiratory %K respiratory virus %K virus %K machine learning model %K pediatric %K Japan %K detection model %D 2024 %7 12.4.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Respiratory syncytial virus (RSV) affects children, causing serious infections, particularly in high-risk groups. Given the seasonality of RSV and the importance of rapid isolation of infected individuals, there is an urgent need for more efficient diagnostic methods to expedite this process. Objective: This study aimed to investigate the performance of a machine learning model that leverages the temporal diversity of symptom onset for detecting RSV infections and elucidate its discriminatory ability. Methods: The study was conducted in pediatric and emergency outpatient settings in Japan. We developed a detection model that remotely confirms RSV infection based on patient-reported symptom information obtained using a structured electronic template incorporating the differential points of skilled pediatricians. An extreme gradient boosting–based machine learning model was developed using the data of 4174 patients aged ≤24 months who underwent RSV rapid antigen testing. These patients visited either the pediatric or emergency department of Yokohama City Municipal Hospital between January 1, 2009, and December 31, 2015. The primary outcome was the diagnostic accuracy of the machine learning model for RSV infection, as determined by rapid antigen testing, measured using the area under the receiver operating characteristic curve. The clinical efficacy was evaluated by calculating the discriminative performance based on the number of days elapsed since the onset of the first symptom and exclusion rates based on thresholds of reasonable sensitivity and specificity. Results: Our model demonstrated an area under the receiver operating characteristic curve of 0.811 (95% CI 0.784-0.833) with good calibration and 0.746 (95% CI 0.694-0.794) for patients within 3 days of onset. It accurately captured the temporal evolution of symptoms; based on adjusted thresholds equivalent to those of a rapid antigen test, our model predicted that 6.9% (95% CI 5.4%-8.5%) of patients in the entire cohort would be positive and 68.7% (95% CI 65.4%-71.9%) would be negative. Our model could eliminate the need for additional testing in approximately three-quarters of all patients. Conclusions: Our model may facilitate the immediate detection of RSV infection in outpatient settings and, potentially, in home environments. This approach could streamline the diagnostic process, reduce discomfort caused by invasive tests in children, and allow rapid implementation of appropriate treatments and isolation at home. The findings underscore the potential of machine learning in augmenting clinical decision-making in the early detection of RSV infection. %M 38608268 %R 10.2196/52412 %U https://formative.jmir.org/2024/1/e52412 %U https://doi.org/10.2196/52412 %U http://www.ncbi.nlm.nih.gov/pubmed/38608268 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e55499 %T Impact of Electronic Health Record Use on Cognitive Load and Burnout Among Clinicians: Narrative Review %A Asgari,Elham %A Kaur,Japsimar %A Nuredini,Gani %A Balloch,Jasmine %A Taylor,Andrew M %A Sebire,Neil %A Robinson,Robert %A Peters,Catherine %A Sridharan,Shankar %A Pimenta,Dominic %+ Tortus AI, 193-197 High Holborn, London, WC1V 7BD, United Kingdom, 44 7763891802, asgelham@gmail.com %K electronic health record %K cognitive load %K burnout %K technology %K clinician %D 2024 %7 12.4.2024 %9 Viewpoint %J JMIR Med Inform %G English %X The cognitive load theory suggests that completing a task relies on the interplay between sensory input, working memory, and long-term memory. Cognitive overload occurs when the working memory’s limited capacity is exceeded due to excessive information processing. In health care, clinicians face increasing cognitive load as the complexity of patient care has risen, leading to potential burnout. Electronic health records (EHRs) have become a common feature in modern health care, offering improved access to data and the ability to provide better patient care. They have been added to the electronic ecosystem alongside emails and other resources, such as guidelines and literature searches. Concerns have arisen in recent years that despite many benefits, the use of EHRs may lead to cognitive overload, which can impact the performance and well-being of clinicians. We aimed to review the impact of EHR use on cognitive load and how it correlates with physician burnout. Additionally, we wanted to identify potential strategies recommended in the literature that could be implemented to decrease the cognitive burden associated with the use of EHRs, with the goal of reducing clinician burnout. Using a comprehensive literature review on the topic, we have explored the link between EHR use, cognitive load, and burnout among health care professionals. We have also noted key factors that can help reduce EHR-related cognitive load, which may help reduce clinician burnout. The research findings suggest that inadequate efforts to present large amounts of clinical data to users in a manner that allows the user to control the cognitive burden in the EHR and the complexity of the user interfaces, thus adding more “work” to tasks, can lead to cognitive overload and burnout; this calls for strategies to mitigate these effects. Several factors, such as the presentation of information in the EHR, the specialty, the health care setting, and the time spent completing documentation and navigating systems, can contribute to this excess cognitive load and result in burnout. Potential strategies to mitigate this might include improving user interfaces, streamlining information, and reducing documentation burden requirements for clinicians. New technologies may facilitate these strategies. The review highlights the importance of addressing cognitive overload as one of the unintended consequences of EHR adoption and potential strategies for mitigation, identifying gaps in the current literature that require further exploration. %M 38607672 %R 10.2196/55499 %U https://medinform.jmir.org/2024/1/e55499 %U https://doi.org/10.2196/55499 %U http://www.ncbi.nlm.nih.gov/pubmed/38607672 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e52612 %T Applications of Artificial Intelligence in Emergency Departments to Improve Wait Times: Protocol for an Integrative Living Review %A Ahmadzadeh,Bahareh %A Patey,Christopher %A Hurley,Oliver %A Knight,John %A Norman,Paul %A Farrell,Alison %A Czarnuch,Stephen %A Asghari,Shabnam %+ Centre for Rural Health Studies, Faculty of Medicine, Memorial University of Newfoundland, 300 Prince Philip Drive, St. John's, NL, A1B3V6, Canada, 1 709 777 2142, sasghari@mun.ca %K emergency department %K ED %K wait time %K artificial intelligence %K AI %K living systematic review %K LSR %D 2024 %7 12.4.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Long wait times in the emergency department (ED) are a major issue for health care systems all over the world. The application of artificial intelligence (AI) is a novel strategy to reduce ED wait times when compared to the interventions included in previous research endeavors. To date, comprehensive systematic reviews that include studies involving AI applications in the context of EDs have covered a wide range of AI implementation issues. However, the lack of an iterative update strategy limits the use of these reviews. Since the subject of AI development is cutting edge and is continuously changing, reviews in this area must be frequently updated to remain relevant. Objective: This study aims to provide a summary of the evidence that is currently available regarding how AI can affect ED wait times; discuss the applications of AI in improving wait times; and periodically assess the depth, breadth, and quality of the evidence supporting the application of AI in reducing ED wait times. Methods: We plan to conduct a living systematic review (LSR). Our strategy involves conducting continuous monitoring of evidence, with biannual search updates and annual review updates. Upon completing the initial round of the review, we will refine the search strategy and establish clear schedules for updating the LSR. An interpretive synthesis using Whittemore and Knafl’s framework will be performed to compile and summarize the findings. The review will be carried out using an integrated knowledge translation strategy, and knowledge users will be involved at all stages of the review to guarantee applicability, usability, and clarity of purpose. Results: The literature search was completed by September 22, 2023, and identified 17,569 articles. The title and abstract screening were completed by December 9, 2023. In total, 70 papers were eligible. The full-text screening is in progress. Conclusions: The review will summarize AI applications that improve ED wait time. The LSR enables researchers to maintain high methodological rigor while enhancing the timeliness, applicability, and value of the review. International Registered Report Identifier (IRRID): DERR1-10.2196/52612 %M 38607662 %R 10.2196/52612 %U https://www.researchprotocols.org/2024/1/e52612 %U https://doi.org/10.2196/52612 %U http://www.ncbi.nlm.nih.gov/pubmed/38607662 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e46698 %T Usability of an Automated System for Real-Time Monitoring of Shared Decision-Making for Surgery: Mixed Methods Evaluation %A Hoffmann,Christin %A Avery,Kerry %A Macefield,Rhiannon %A Dvořák,Tadeáš %A Snelgrove,Val %A Blazeby,Jane %A Hopkins,Della %A Hickey,Shireen %A Gibbison,Ben %A Rooshenas,Leila %A Williams,Adam %A Aning,Jonathan %A Bekker,Hilary L %A McNair,Angus GK %A , %+ National Institute for Health and Care Research Bristol Biomedical Research Centre, Bristol Centre for Surgical Research, Bristol Medical School: Population Health Sciences, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS, United Kingdom, 44 117 455 5993, c.hoffmann@bristol.ac.uk %K surgery %K shared decision-making %K patient participation %K mixed methods %K surgery %K real-time measurement %K patient-reported measure %K electronic data collection %K usability %K data collection %K patient reported %K satisfaction %K mobile phone %D 2024 %7 10.4.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Improving shared decision-making (SDM) for patients has become a health policy priority in many countries. Achieving high-quality SDM is particularly important for approximately 313 million surgical treatment decisions patients make globally every year. Large-scale monitoring of surgical patients’ experience of SDM in real time is needed to identify the failings of SDM before surgery is performed. We developed a novel approach to automating real-time data collection using an electronic measurement system to address this. Examining usability will facilitate its optimization and wider implementation to inform interventions aimed at improving SDM. Objective: This study examined the usability of an electronic real-time measurement system to monitor surgical patients’ experience of SDM. We aimed to evaluate the metrics and indicators relevant to system effectiveness, system efficiency, and user satisfaction. Methods: We performed a mixed methods usability evaluation using multiple participant cohorts. The measurement system was implemented in a large UK hospital to measure patients’ experience of SDM electronically before surgery using 2 validated measures (CollaboRATE and SDM-Q-9). Quantitative data (collected between April 1 and December 31, 2021) provided measurement system metrics to assess system effectiveness and efficiency. We included adult patients booked for urgent and elective surgery across 7 specialties and excluded patients without the capacity to consent for medical procedures, those without access to an internet-enabled device, and those undergoing emergency or endoscopic procedures. Additional groups of service users (group 1: public members who had not engaged with the system; group 2: a subset of patients who completed the measurement system) completed user-testing sessions and semistructured interviews to assess system effectiveness and user satisfaction. We conducted quantitative data analysis using descriptive statistics and calculated the task completion rate and survey response rate (system effectiveness) as well as the task completion time, task efficiency, and relative efficiency (system efficiency). Qualitative thematic analysis identified indicators of and barriers to good usability (user satisfaction). Results: A total of 2254 completed surveys were returned to the measurement system. A total of 25 service users (group 1: n=9; group 2: n=16) participated in user-testing sessions and interviews. The task completion rate was high (169/171, 98.8%) and the survey response rate was good (2254/5794, 38.9%). The median task completion time was 3 (IQR 2-13) minutes, suggesting good system efficiency and effectiveness. The qualitative findings emphasized good user satisfaction. The identified themes suggested that the measurement system is acceptable, easy to use, and easy to access. Service users identified potential barriers and solutions to acceptability and ease of access. Conclusions: A mixed methods evaluation of an electronic measurement system for automated, real-time monitoring of patients’ experience of SDM showed that usability among patients was high. Future pilot work will optimize the system for wider implementation to ultimately inform intervention development to improve SDM. International Registered Report Identifier (IRRID): RR2-10.1136/bmjopen-2023-079155 %M 38598276 %R 10.2196/46698 %U https://humanfactors.jmir.org/2024/1/e46698 %U https://doi.org/10.2196/46698 %U http://www.ncbi.nlm.nih.gov/pubmed/38598276 %0 Journal Article %@ 2563-6316 %I %V 5 %N %P e52198 %T Insider Threats to the Military Health System: A Systematic Background Check of TRICARE West Providers %A Bychkov,David %K TRICARE %K health care fraud %K Defense Health Agency %K fraud %K fraudulent %K insurance %K coverage %K beneficiary %K beneficiaries %K background check %K background checks %K demographic %K security clearance %K FDA %K Medicaid %K Medicare %K provider %K provider referral %K military %K false claims act %K HIPAA breach %K OIG-LEIE %K inspector general %K misconduct %K insider threat %K information system %K zero trust %K data management %K Food and Drug Administration %K Health Insurance Portability and Accountability Act breach %K Office of the Inspector General's List of Excluded Individuals and Entities %D 2024 %7 9.4.2024 %9 %J JMIRx Med %G English %X Background: To address the pandemic, the Defense Health Agency (DHA) expanded its TRICARE civilian provider network by 30.1%. In 2022, the DHA Annual Report stated that TRICARE’s provider directories were only 80% accurate. Unlike Medicare, the DHA does not publicly reveal National Provider Identification (NPI) numbers. As a result, TRICARE’s 9.6 million beneficiaries lack the means to verify their doctor’s credentials. Since 2013, the Department of Health and Human Services’ (HHS) Office of Inspector General (OIG) has excluded 17,706 physicians and other providers from federal health programs due to billing fraud, neglect, drug-related convictions, and other offenses. These providers and their NPIs are included on the OIG’s List of Excluded Individuals and Entities (LEIE). Patients who receive care from excluded providers face higher risks of hospitalization and mortality. Objective: We sought to assess the extent to which TRICARE screens health care provider names on their referral website against criminal databases. Methods: Between January 1-31, 2023, we used TRICARE West’s provider directory to search for all providers within a 5-mile radius of 798 zip codes (38 per state, ≥10,000 residents each, randomly entered). We then copied and pasted all directory results’ first and last names, business names, addresses, phone numbers, fax numbers, degree types, practice specialties, and active or closed statuses into a CSV file. We cross-referenced the search results against US and state databases for medical and criminal misconduct, including the OIG-LEIE and General Services Administration’s (GSA) SAM.gov exclusion lists, the HHS Office of Civil Rights Health Insurance Portability and Accountability Act (HIPAA) breach reports, 15 available state Medicaid exclusion lists (state), the International Trade Administration’s Consolidated Screening List (CSL), 3 Food and Drug Administration (FDA) debarment lists, the Federal Bureau of Investigation’s (FBI) list of January 6 federal defendants, and the OIG-HHS list of fugitives (FUG). Results: Our provider search yielded 111,619 raw results; 54 zip codes contained no data. After removing 72,156 (64.65%) duplicate entries, closed offices, and non-TRICARE West locations, we identified 39,463 active provider names. Within this baseline sample group, there were 2398 (6.08%) total matches against all exclusion and sanction databases, including 2197 on the OIG-LEIE, 2311 on the GSA-SAM.gov list, 2 on the HIPAA list, 54 on the state Medicaid exclusion lists, 69 on the CSL, 3 on the FDA lists, 53 on the FBI list, and 10 on the FUG. Conclusions: TRICARE’s civilian provider roster merits further scrutiny by law enforcement. Following the National Institute of Standards and Technology 800, the DHA can mitigate privacy, safety, and security clearance threats by implementing an insider threat management model, robust enforcement of the False Claims Act, and mandatory security risk assessments. These are the views of the author, not the Department of Defense or the US government. %R 10.2196/52198 %U https://xmed.jmir.org/2024/1/e52198 %U https://doi.org/10.2196/52198 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e55779 %T Converge or Collide? Making Sense of a Plethora of Open Data Standards in Health Care %A Tsafnat,Guy %A Dunscombe,Rachel %A Gabriel,Davera %A Grieve,Grahame %A Reich,Christian %+ Evidentli Pty Ltd, 50 Holt St, Suite 516, Surry Hills, 2010, Australia, 61 415481043, guyt@evidentli.com %K interoperability %K clinical data %K open data standards %K health care %K digital health %K health care data %D 2024 %7 9.4.2024 %9 Editorial %J J Med Internet Res %G English %X Practitioners of digital health are familiar with disjointed data environments that often inhibit effective communication among different elements of the ecosystem. This fragmentation leads in turn to issues such as inconsistencies in services versus payments, wastage, and notably, care delivered being less than best-practice. Despite the long-standing recognition of interoperable data as a potential solution, efforts in achieving interoperability have been disjointed and inconsistent, resulting in numerous incompatible standards, despite the widespread agreement that fewer standards would enhance interoperability. This paper introduces a framework for understanding health care data needs, discussing the challenges and opportunities of open data standards in the field. It emphasizes the necessity of acknowledging diverse data standards, each catering to specific viewpoints and needs, while proposing a categorization of health care data into three domains, each with its distinct characteristics and challenges, along with outlining overarching design requirements applicable to all domains and specific requirements unique to each domain. %M 38593431 %R 10.2196/55779 %U https://www.jmir.org/2024/1/e55779 %U https://doi.org/10.2196/55779 %U http://www.ncbi.nlm.nih.gov/pubmed/38593431 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e55627 %T Evaluating ChatGPT-4’s Diagnostic Accuracy: Impact of Visual Data Integration %A Hirosawa,Takanobu %A Harada,Yukinori %A Tokumasu,Kazuki %A Ito,Takahiro %A Suzuki,Tomoharu %A Shimizu,Taro %+ Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, 880 Kitakobayashi, Mibu-cho, Shimotsuga, 321-0293, Japan, 81 282 87 2498, hirosawa@dokkyomed.ac.jp %K artificial intelligence %K large language model %K LLM %K LLMs %K language model %K language models %K ChatGPT %K GPT %K ChatGPT-4V %K ChatGPT-4 Vision %K clinical decision support %K natural language processing %K decision support %K NLP %K diagnostic excellence %K diagnosis %K diagnoses %K diagnose %K diagnostic %K diagnostics %K image %K images %K imaging %D 2024 %7 9.4.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: In the evolving field of health care, multimodal generative artificial intelligence (AI) systems, such as ChatGPT-4 with vision (ChatGPT-4V), represent a significant advancement, as they integrate visual data with text data. This integration has the potential to revolutionize clinical diagnostics by offering more comprehensive analysis capabilities. However, the impact on diagnostic accuracy of using image data to augment ChatGPT-4 remains unclear. Objective: This study aims to assess the impact of adding image data on ChatGPT-4’s diagnostic accuracy and provide insights into how image data integration can enhance the accuracy of multimodal AI in medical diagnostics. Specifically, this study endeavored to compare the diagnostic accuracy between ChatGPT-4V, which processed both text and image data, and its counterpart, ChatGPT-4, which only uses text data. Methods: We identified a total of 557 case reports published in the American Journal of Case Reports from January 2022 to March 2023. After excluding cases that were nondiagnostic, pediatric, and lacking image data, we included 363 case descriptions with their final diagnoses and associated images. We compared the diagnostic accuracy of ChatGPT-4V and ChatGPT-4 without vision based on their ability to include the final diagnoses within differential diagnosis lists. Two independent physicians evaluated their accuracy, with a third resolving any discrepancies, ensuring a rigorous and objective analysis. Results: The integration of image data into ChatGPT-4V did not significantly enhance diagnostic accuracy, showing that final diagnoses were included in the top 10 differential diagnosis lists at a rate of 85.1% (n=309), comparable to the rate of 87.9% (n=319) for the text-only version (P=.33). Notably, ChatGPT-4V’s performance in correctly identifying the top diagnosis was inferior, at 44.4% (n=161), compared with 55.9% (n=203) for the text-only version (P=.002, χ2 test). Additionally, ChatGPT-4’s self-reports showed that image data accounted for 30% of the weight in developing the differential diagnosis lists in more than half of cases. Conclusions: Our findings reveal that currently, ChatGPT-4V predominantly relies on textual data, limiting its ability to fully use the diagnostic potential of visual information. This study underscores the need for further development of multimodal generative AI systems to effectively integrate and use clinical image data. Enhancing the diagnostic performance of such AI systems through improved multimodal data integration could significantly benefit patient care by providing more accurate and comprehensive diagnostic insights. Future research should focus on overcoming these limitations, paving the way for the practical application of advanced AI in medicine. %M 38592758 %R 10.2196/55627 %U https://medinform.jmir.org/2024/1/e55627 %U https://doi.org/10.2196/55627 %U http://www.ncbi.nlm.nih.gov/pubmed/38592758 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e55318 %T An Empirical Evaluation of Prompting Strategies for Large Language Models in Zero-Shot Clinical Natural Language Processing: Algorithm Development and Validation Study %A Sivarajkumar,Sonish %A Kelley,Mark %A Samolyk-Mazzanti,Alyssa %A Visweswaran,Shyam %A Wang,Yanshan %+ Department of Health Information Management, University of Pittsburgh, 6026 Forbes Tower, Pittsburgh, PA, 15260, United States, 1 4123832712, yanshan.wang@pitt.edu %K large language model %K LLM %K LLMs %K natural language processing %K NLP %K in-context learning %K prompt engineering %K evaluation %K zero-shot %K few shot %K prompting %K GPT %K language model %K language %K models %K machine learning %K clinical data %K clinical information %K extraction %K BARD %K Gemini %K LLaMA-2 %K heuristic %K prompt %K prompts %K ensemble %D 2024 %7 8.4.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Large language models (LLMs) have shown remarkable capabilities in natural language processing (NLP), especially in domains where labeled data are scarce or expensive, such as the clinical domain. However, to unlock the clinical knowledge hidden in these LLMs, we need to design effective prompts that can guide them to perform specific clinical NLP tasks without any task-specific training data. This is known as in-context learning, which is an art and science that requires understanding the strengths and weaknesses of different LLMs and prompt engineering approaches. Objective: The objective of this study is to assess the effectiveness of various prompt engineering techniques, including 2 newly introduced types—heuristic and ensemble prompts, for zero-shot and few-shot clinical information extraction using pretrained language models. Methods: This comprehensive experimental study evaluated different prompt types (simple prefix, simple cloze, chain of thought, anticipatory, heuristic, and ensemble) across 5 clinical NLP tasks: clinical sense disambiguation, biomedical evidence extraction, coreference resolution, medication status extraction, and medication attribute extraction. The performance of these prompts was assessed using 3 state-of-the-art language models: GPT-3.5 (OpenAI), Gemini (Google), and LLaMA-2 (Meta). The study contrasted zero-shot with few-shot prompting and explored the effectiveness of ensemble approaches. Results: The study revealed that task-specific prompt tailoring is vital for the high performance of LLMs for zero-shot clinical NLP. In clinical sense disambiguation, GPT-3.5 achieved an accuracy of 0.96 with heuristic prompts and 0.94 in biomedical evidence extraction. Heuristic prompts, alongside chain of thought prompts, were highly effective across tasks. Few-shot prompting improved performance in complex scenarios, and ensemble approaches capitalized on multiple prompt strengths. GPT-3.5 consistently outperformed Gemini and LLaMA-2 across tasks and prompt types. Conclusions: This study provides a rigorous evaluation of prompt engineering methodologies and introduces innovative techniques for clinical information extraction, demonstrating the potential of in-context learning in the clinical domain. These findings offer clear guidelines for future prompt-based clinical NLP research, facilitating engagement by non-NLP experts in clinical NLP advancements. To the best of our knowledge, this is one of the first works on the empirical evaluation of different prompt engineering approaches for clinical NLP in this era of generative artificial intelligence, and we hope that it will inspire and inform future research in this area. %M 38587879 %R 10.2196/55318 %U https://medinform.jmir.org/2024/1/e55318 %U https://doi.org/10.2196/55318 %U http://www.ncbi.nlm.nih.gov/pubmed/38587879 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e54109 %T Fast Healthcare Interoperability Resources–Based Support System for Predicting Delivery Type: Model Development and Evaluation Study %A Coutinho-Almeida,João %A Cardoso,Alexandrina %A Cruz-Correia,Ricardo %A Pereira-Rodrigues,Pedro %+ Faculty of Medicine, University of Porto, Alameda Professor Hernâni Monteiro, Porto, 4200-319, Portugal, 351 225513600, joaofilipe90@gmail.com %K obstetrics %K machine-learning %K clinical decision support %K interoperability %K interoperable %K obstetric %K cesarean delivery %K cesarean %K cesarean deliveries %K decision support %K pregnant %K pregnancy %K maternal %K algorithm %K algorithms %K simulation %K simulations %D 2024 %7 8.4.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: The escalating prevalence of cesarean delivery globally poses significant health impacts on mothers and newborns. Despite this trend, the underlying reasons for increased cesarean delivery rates, which have risen to 36.3% in Portugal as of 2020, remain unclear. This study delves into these issues within the Portuguese health care context, where national efforts are underway to reduce cesarean delivery occurrences. Objective: This paper aims to introduce a machine learning, algorithm-based support system designed to assist clinical teams in identifying potentially unnecessary cesarean deliveries. Key objectives include developing clinical decision support systems for cesarean deliveries using interoperability standards, identifying predictive factors influencing delivery type, assessing the economic impact of implementing this tool, and comparing system outputs with clinicians’ decisions. Methods: This study used retrospective data collected from 9 public Portuguese hospitals, encompassing maternal and fetal data and delivery methods from 2019 to 2020. We used various machine learning algorithms for model development, with light gradient-boosting machine (LightGBM) selected for deployment due to its efficiency. The model’s performance was compared with clinician assessments through questionnaires. Additionally, an economic simulation was conducted to evaluate the financial impact on Portuguese public hospitals. Results: The deployed model, based on LightGBM, achieved an area under the receiver operating characteristic curve of 88%. In the trial deployment phase at a single hospital, 3.8% (123/3231) of cases triggered alarms for potentially unnecessary cesarean deliveries. Financial simulation results indicated potential benefits for 30% (15/48) of Portuguese public hospitals with the implementation of our tool. However, this study acknowledges biases in the model, such as combining different vaginal delivery types and focusing on potentially unwarranted cesarean deliveries. Conclusions: This study presents a promising system capable of identifying potentially incorrect cesarean delivery decisions, with potentially positive implications for medical practice and health care economics. However, it also highlights the challenges and considerations necessary for real-world application, including further evaluation of clinical decision-making impacts and understanding the diverse reasons behind delivery type choices. This study underscores the need for careful implementation and further robust analysis to realize the full potential and real-world applicability of such clinical support systems. %M 38587885 %R 10.2196/54109 %U https://formative.jmir.org/2024/1/e54109 %U https://doi.org/10.2196/54109 %U http://www.ncbi.nlm.nih.gov/pubmed/38587885 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e54278 %T Effect of Performance-Based Nonfinancial Incentives on Data Quality in Individual Medical Records of Institutional Births: Quasi-Experimental Study %A Taye,Biniam Kefiyalew %A Gezie,Lemma Derseh %A Atnafu,Asmamaw %A Mengiste,Shegaw Anagaw %A Kaasbøll,Jens %A Gullslett,Monika Knudsen %A Tilahun,Binyam %+ Ministry of Health, The Federal Democratic Republic of Ethiopia, Zambia street, Addis Ababa, Ethiopia, 251 910055867, bini.bhi2013@gmail.com %K individual medical records %K data quality %K completeness %K consistency %K nonfinancial incentives %K institutional birth %K health care quality %K quasi-experimental design %K Ethiopia %D 2024 %7 5.4.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Despite the potential of routine health information systems in tackling persistent maternal deaths stemming from poor service quality at health facilities during and around childbirth, research has demonstrated their suboptimal performance, evident from the incomplete and inaccurate data unfit for practical use. There is a consensus that nonfinancial incentives can enhance health care providers’ commitment toward achieving the desired health care quality. However, there is limited evidence regarding the effectiveness of nonfinancial incentives in improving the data quality of institutional birth services in Ethiopia. Objective: This study aimed to evaluate the effect of performance-based nonfinancial incentives on the completeness and consistency of data in the individual medical records of women who availed institutional birth services in northwest Ethiopia. Methods: We used a quasi-experimental design with a comparator group in the pre-post period, using a sample of 1969 women’s medical records. The study was conducted in the “Wegera” and “Tach-armacheho” districts, which served as the intervention and comparator districts, respectively. The intervention comprised a multicomponent nonfinancial incentive, including smartphones, flash disks, power banks, certificates, and scholarships. Personal records of women who gave birth within 6 months before (April to September 2020) and after (February to July 2021) the intervention were included. Three distinct women’s birth records were examined: the integrated card, integrated individual folder, and delivery register. The completeness of the data was determined by examining the presence of data elements, whereas the consistency check involved evaluating the agreement of data elements among women’s birth records. The average treatment effect on the treated (ATET), with 95% CIs, was computed using a difference-in-differences model. Results: In the intervention district, data completeness in women’s personal records was nearly 4 times higher (ATET 3.8, 95% CI 2.2-5.5; P=.02), and consistency was approximately 12 times more likely (ATET 11.6, 95% CI 4.18-19; P=.03) than in the comparator district. Conclusions: This study indicates that performance-based nonfinancial incentives enhance data quality in the personal records of institutional births. Health care planners can adapt these incentives to improve the data quality of comparable medical records, particularly pregnancy-related data within health care facilities. Future research is needed to assess the effectiveness of nonfinancial incentives across diverse contexts to support successful scale-up. %M 38578684 %R 10.2196/54278 %U https://medinform.jmir.org/2024/1/e54278 %U https://doi.org/10.2196/54278 %U http://www.ncbi.nlm.nih.gov/pubmed/38578684 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e48862 %T Interpretable Deep Learning System for Identifying Critical Patients Through the Prediction of Triage Level, Hospitalization, and Length of Stay: Prospective Study %A Lin,Yu-Ting %A Deng,Yuan-Xiang %A Tsai,Chu-Lin %A Huang,Chien-Hua %A Fu,Li-Chen %+ Department of Computer Science and Information Engineering, National Taiwan University, CSIE Der Tian Hall No. 1, Sec. 4, Roosevelt Road, Taipei, 106319, Taiwan, 886 935545846, lichen@ntu.edu.tw %K emergency department %K triage system %K hospital admission %K length of stay %K multimodal integration %D 2024 %7 1.4.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Triage is the process of accurately assessing patients’ symptoms and providing them with proper clinical treatment in the emergency department (ED). While many countries have developed their triage process to stratify patients’ clinical severity and thus distribute medical resources, there are still some limitations of the current triage process. Since the triage level is mainly identified by experienced nurses based on a mix of subjective and objective criteria, mis-triage often occurs in the ED. It can not only cause adverse effects on patients, but also impose an undue burden on the health care delivery system. Objective: Our study aimed to design a prediction system based on triage information, including demographics, vital signs, and chief complaints. The proposed system can not only handle heterogeneous data, including tabular data and free-text data, but also provide interpretability for better acceptance by the ED staff in the hospital. Methods: In this study, we proposed a system comprising 3 subsystems, with each of them handling a single task, including triage level prediction, hospitalization prediction, and length of stay prediction. We used a large amount of retrospective data to pretrain the model, and then, we fine-tuned the model on a prospective data set with a golden label. The proposed deep learning framework was built with TabNet and MacBERT (Chinese version of bidirectional encoder representations from transformers [BERT]). Results: The performance of our proposed model was evaluated on data collected from the National Taiwan University Hospital (901 patients were included). The model achieved promising results on the collected data set, with accuracy values of 63%, 82%, and 71% for triage level prediction, hospitalization prediction, and length of stay prediction, respectively. Conclusions: Our system improved the prediction of 3 different medical outcomes when compared with other machine learning methods. With the pretrained vital sign encoder and repretrained mask language modeling MacBERT encoder, our multimodality model can provide a deeper insight into the characteristics of electronic health records. Additionally, by providing interpretability, we believe that the proposed system can assist nursing staff and physicians in taking appropriate medical decisions. %M 38557661 %R 10.2196/48862 %U https://medinform.jmir.org/2024/1/e48862 %U https://doi.org/10.2196/48862 %U http://www.ncbi.nlm.nih.gov/pubmed/38557661 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e52920 %T A Novel Electronic Record System for Documentation and Efficient Workflow for Community Health Workers: Development and Usability Study %A Acharya,Harshdeep %A Sykes,Kevin J %A Neira,Ton Mirás %A Scott,Angela %A Pacheco,Christina M %A Sanner,Matthew %A Ablah,Elizabeth %A Oyowe,Kevin %A Ellerbeck,Edward F %A Greiner,K Allen %A Corriveau,Erin A %A Finocchario-Kessler,Sarah %+ Health and Wellness Center, Baylor Scott and White Health, 4500 Spring Avenue, Dallas, TX, 75210, United States, 1 820 0111 ext 214, Kevin.Sykes@bswhealth.org %K public health %K database %K community health worker %K social determinants of health %K health worker %K health workers %K CHW %K CHWs %K community-based %K data collection %K functionality %K develop %K development %K EHR %K EHRs %K EMR %K EMRs %K dashboard %K dashboards %K health record %K health records %K documentation %K medical record %K medical records %K equity %K inequity %K inequities %D 2024 %7 1.4.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: The COVID-19 pandemic added to the decades of evidence that public health institutions are routinely stretched beyond their capacity. Community health workers (CHWs) can be a crucial extension of public health resources to address health inequities, but systems to document CHW efforts are often fragmented and prone to unneeded redundancy, errors, and inefficiency. Objective: We sought to develop a more efficient data collection system for recording the wide range of community-based efforts performed by CHWs. Methods: The Communities Organizing to Promote Equity (COPE) project is an initiative to address health disparities across Kansas, in part, through the deployment of CHWs. Our team iteratively designed and refined the features of a novel data collection system for CHWs. Pilot tests with CHWs occurred over several months to ensure that the functionality supported their daily use. Following implementation of the database, procedures were set to sustain the collection of feedback from CHWs, community partners, and organizations with similar systems to continually modify the database to meet the needs of users. A continuous quality improvement process was conducted monthly to evaluate CHW performance; feedback was exchanged at team and individual levels regarding the continuous quality improvement results and opportunities for improvement. Further, a 15-item feedback survey was distributed to all 33 COPE CHWs and supervisors for assessing the feasibility of database features, accessibility, and overall satisfaction. Results: At launch, the database had 60 active users in 20 counties. Documented client interactions begin with needs assessments (modified versions of the Arizona Self-sufficiency Matrix and PRAPARE [Protocol for Responding to and Assessing Patient Assets, Risks, and Experiences]) and continue with the longitudinal tracking of progress toward goals. A user-specific automated alerts-based dashboard displays clients needing follow-up and upcoming events. The database contains over 55,000 documented encounters across more than 5079 clients. Available resources from over 2500 community organizations have been documented. Survey data indicated that 84% (27/32) of the respondents considered the overall navigation of the database as very easy. The majority of the respondents indicated they were overall very satisfied (14/32, 44%) or satisfied (15/32, 48%) with the database. Open-ended responses indicated the database features, documentation of community organizations and visual confirmation of consent form and data storage on a Health Insurance Portability and Accountability Act–compliant record system, improved client engagement, enrollment processes, and identification of resources. Conclusions: Our database extends beyond conventional electronic medical records and provides flexibility for ever-changing needs. The COPE database provides real-world data on CHW accomplishments, thereby improving the uniformity of data collection to enhance monitoring and evaluation. This database can serve as a model for community-based documentation systems and be adapted for use in other community settings. %M 38557671 %R 10.2196/52920 %U https://formative.jmir.org/2024/1/e52920 %U https://doi.org/10.2196/52920 %U http://www.ncbi.nlm.nih.gov/pubmed/38557671 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e54857 %T Investigating the Impact of AI on Shared Decision-Making in Post-Kidney Transplant Care (PRIMA-AI): Protocol for a Randomized Controlled Trial %A Osmanodja,Bilgin %A Sassi,Zeineb %A Eickmann,Sascha %A Hansen,Carla Maria %A Roller,Roland %A Burchardt,Aljoscha %A Samhammer,David %A Dabrock,Peter %A Möller,Sebastian %A Budde,Klemens %A Herrmann,Anne %+ Department of Nephrology and Medical Intensive Care, Charité – Universitätsmedizin Berlin, Charitéplatz 1, Berlin, 10117, Germany, 49 30450614368, bilgin.osmanodja@charite.de %K shared decision-making %K SDM %K kidney transplantation %K artificial intelligence %K AI %K decision-support system %K DSS %K qualitative research %D 2024 %7 1.4.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Patients after kidney transplantation eventually face the risk of graft loss with the concomitant need for dialysis or retransplantation. Choosing the right kidney replacement therapy after graft loss is an important preference-sensitive decision for kidney transplant recipients. However, the rate of conversations about treatment options after kidney graft loss has been shown to be as low as 13% in previous studies. It is unknown whether the implementation of artificial intelligence (AI)–based risk prediction models can increase the number of conversations about treatment options after graft loss and how this might influence the associated shared decision-making (SDM). Objective: This study aims to explore the impact of AI-based risk prediction for the risk of graft loss on the frequency of conversations about the treatment options after graft loss, as well as the associated SDM process. Methods: This is a 2-year, prospective, randomized, 2-armed, parallel-group, single-center trial in a German kidney transplant center. All patients will receive the same routine post–kidney transplant care that usually includes follow-up visits every 3 months at the kidney transplant center. For patients in the intervention arm, physicians will be assisted by a validated and previously published AI-based risk prediction system that estimates the risk for graft loss in the next year, starting from 3 months after randomization until 24 months after randomization. The study population will consist of 122 kidney transplant recipients >12 months after transplantation, who are at least 18 years of age, are able to communicate in German, and have an estimated glomerular filtration rate <30 mL/min/1.73 m2. Patients with multi-organ transplantation, or who are not able to communicate in German, as well as underage patients, cannot participate. For the primary end point, the proportion of patients who have had a conversation about their treatment options after graft loss is compared at 12 months after randomization. Additionally, 2 different assessment tools for SDM, the CollaboRATE mean score and the Control Preference Scale, are compared between the 2 groups at 12 months and 24 months after randomization. Furthermore, recordings of patient-physician conversations, as well as semistructured interviews with patients, support persons, and physicians, are performed to support the quantitative results. Results: The enrollment for the study is ongoing. The first results are expected to be submitted for publication in 2025. Conclusions: This is the first study to examine the influence of AI-based risk prediction on physician-patient interaction in the context of kidney transplantation. We use a mixed methods approach by combining a randomized design with a simple quantitative end point (frequency of conversations), different quantitative measurements for SDM, and several qualitative research methods (eg, records of physician-patient conversations and semistructured interviews) to examine the implementation of AI-based risk prediction in the clinic. Trial Registration: ClinicalTrials.gov NCT06056518; https://clinicaltrials.gov/study/NCT06056518 International Registered Report Identifier (IRRID): PRR1-10.2196/54857 %M 38557315 %R 10.2196/54857 %U https://www.researchprotocols.org/2024/1/e54857 %U https://doi.org/10.2196/54857 %U http://www.ncbi.nlm.nih.gov/pubmed/38557315 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e53343 %T Identification of Predictors for Clinical Deterioration in Patients With COVID-19 via Electronic Nursing Records: Retrospective Observational Study %A Sung,Sumi %A Kim,Youlim %A Kim,Su Hwan %A Jung,Hyesil %+ Department of Nursing, College of Medicine, Inha University, 100 Inha-ro, Michuhol-gu, Incheon, 22212, Republic of Korea, 82 32 860 8206, hsjung@inha.ac.kr %K COVID-19 %K infectious %K respiratory %K SARS-CoV-2 %K nursing records %K SNOMED CT %K random forest %K logistic regression %K EHR %K EHRs %K machine learning %K documentation %K deterioration %K health records %K health record %K patient record %K patient records %K nursing %K standardization %K standard %K standards %K standardized %K standardize %K nomenclature %K term %K terms %K terminologies %K terminology %D 2024 %7 29.3.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Few studies have used standardized nursing records with Systematized Nomenclature of Medicine–Clinical Terms (SNOMED CT) to identify predictors of clinical deterioration. Objective: This study aims to standardize the nursing documentation records of patients with COVID-19 using SNOMED CT and identify predictive factors of clinical deterioration in patients with COVID-19 via standardized nursing records. Methods: In this study, 57,558 nursing statements from 226 patients with COVID-19 were analyzed. Among these, 45,852 statements were from 207 patients in the stable (control) group and 11,706 from 19 patients in the exacerbated (case) group who were transferred to the intensive care unit within 7 days. The data were collected between December 2019 and June 2022. These nursing statements were standardized using the SNOMED CT International Edition released on November 30, 2022. The 260 unique nursing statements that accounted for the top 90% of 57,558 statements were selected as the mapping source and mapped into SNOMED CT concepts based on their meaning by 2 experts with more than 5 years of SNOMED CT mapping experience. To identify the main features of nursing statements associated with the exacerbation of patient condition, random forest algorithms were used, and optimal hyperparameters were selected for nursing problems or outcomes and nursing procedure–related statements. Additionally, logistic regression analysis was conducted to identify features that determine clinical deterioration in patients with COVID-19. Results: All nursing statements were semantically mapped to SNOMED CT concepts for “clinical finding,” “situation with explicit context,” and “procedure” hierarchies. The interrater reliability of the mapping results was 87.7%. The most important features calculated by random forest were “oxygen saturation below reference range,” “dyspnea,” “tachypnea,” and “cough” in “clinical finding,” and “oxygen therapy,” “pulse oximetry monitoring,” “temperature taking,” “notification of physician,” and “education about isolation for infection control” in “procedure.” Among these, “dyspnea” and “inadequate food diet” in “clinical finding” increased clinical deterioration risk (dyspnea: odds ratio [OR] 5.99, 95% CI 2.25-20.29; inadequate food diet: OR 10.0, 95% CI 2.71-40.84), and “oxygen therapy” and “notification of physician” in “procedure” also increased the risk of clinical deterioration in patients with COVID-19 (oxygen therapy: OR 1.89, 95% CI 1.25-3.05; notification of physician: OR 1.72, 95% CI 1.02-2.97). Conclusions: The study used SNOMED CT to express and standardize nursing statements. Further, it revealed the importance of standardized nursing records as predictive variables for clinical deterioration in patients. %M 38414056 %R 10.2196/53343 %U https://www.jmir.org/2024/1/e53343 %U https://doi.org/10.2196/53343 %U http://www.ncbi.nlm.nih.gov/pubmed/38414056 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e49696 %T Patient Perspectives on Communication Pathways After Orthopedic Surgery and Discharge and Evaluation of Team-Based Digital Communication: Qualitative Exploratory Study %A Jensen,Lili Worre Høpfner %A Rahbek,Ole %A Lauritsen,Rikke Emilie Kildahl %A Kold,Søren %A Dinesen,Birthe %+ Interdisciplinary Orthopaedics, Orthopaedic Surgery Department, Aalborg University Hospital, Hobrovej 18-22, Aalborg, 9000, Denmark, 45 60229406, lili.jensen@rn.dk %K digital communication %K patient-provider communication %K continuity of care %K interdisciplinary communication %K hospital discharge %K orthopedic surgery %K postoperative care %K text messaging %K mobile phone %D 2024 %7 29.3.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The transition from hospital to home after orthopedic surgery requires smooth communication and coordination between patients and their team of care to avoid fragmented care pathways. Digital communication is increasingly being used to facilitate easy and accessible asynchronous communication between patients and health care professionals across settings. A team-based approach to digital communication may provide optimized quality of care in the postoperative period following orthopedic surgery and hospital discharge. Objective: This study was divided into two phases that aimed to (1) explore the perspectives of patients undergoing orthopedic surgery on current communication pathways at a tertiary hospital in Denmark and (2) test and explore patients’ experiences and use of team-based digital communication following hospital discharge (eDialogue). Methods: A triangulation of qualitative data collection techniques was applied: document analysis, participant observations (n=16 hours), semistructured interviews with patients before (n=31) and after (n=24) their access to eDialogue, and exploration of use data. Results: Findings show that patients experience difficult communication pathways after hospital discharge and a lack of information due to inadequate coordination of care. eDialogue was used by 84% (26/31) of the patients, and they suggested that it provided a sense of security, coherence, and proximity in the aftercare rearranging communication pathways for the better. Specific drivers and barriers to use were identified, and these call for further exploration of eDialogue. Conclusions: In conclusion, patients evaluated eDialogue positively and suggested that it could support them after returning home following orthopedic surgery. %M 38551641 %R 10.2196/49696 %U https://humanfactors.jmir.org/2024/1/e49696 %U https://doi.org/10.2196/49696 %U http://www.ncbi.nlm.nih.gov/pubmed/38551641 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e47914 %T A Mobile App (Concerto) to Empower Hospitalized Patients in a Swiss University Hospital: Development, Design, and Implementation Report %A Dietrich,Damien %A Bornet dit Vorgeat,Helena %A Perrin Franck,Caroline %A Ligier,Quentin %+ Geneva Hub for Global Digital Health, Faculty of Medicine, University of Geneva, Campus Biotech, 9 Chemin des Mines, Geneva, 1202, Switzerland, 41 227714730, damien.dietrich@gmail.com %K patient empowerment %K mobile apps %K digital health %K mobile health %K implementation science %K health care system %K hospital information system %K health promotion %D 2024 %7 28.3.2024 %9 Implementation Report %J JMIR Med Inform %G English %X Background: Patient empowerment can be associated with better health outcomes, especially in the management of chronic diseases. Digital health has the potential to promote patient empowerment. Objective: Concerto is a mobile app designed to promote patient empowerment in an in-patient setting. This implementation report focuses on the lessons learned during its implementation. Methods: The app was conceptualized and prototyped during a hackathon. Concerto uses hospital information system (HIS) data to offer the following key functionalities: a care schedule, targeted medical information, practical information, information about the on-duty care team, and a medical round preparation module. Funding was obtained following a feasibility study, and the app was developed and implemented in four pilot divisions of a Swiss University Hospital using institution-owned tablets. Implementation (Results): The project lasted for 2 years with effective implementation in the four pilot divisions and was maintained within budget. The induced workload on caregivers impaired project sustainability and warranted a change in our implementation strategy. The presence of a killer function would have facilitated the deployment. Furthermore, our experience is in line with the well-accepted need for both high-quality user training and a suitable selection of superusers. Finally, by presenting HIS data directly to the patient, Concerto highlighted the data that are not fit for purpose and triggered data curation and standardization initiatives. Conclusions: This implementation report presents a real-world example of designing, developing, and implementing a patient-empowering mobile app in a university hospital in-patient setting with a particular focus on the lessons learned. One limitation of the study is the lack of definition of a “key success” indicator. %M 38546728 %R 10.2196/47914 %U https://medinform.jmir.org/2024/1/e47914 %U https://doi.org/10.2196/47914 %U http://www.ncbi.nlm.nih.gov/pubmed/38546728 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e41065 %T Development and External Validation of Machine Learning Models for Diabetic Microvascular Complications: Cross-Sectional Study With Metabolites %A He,Feng %A Ng Yin Ling,Clarissa %A Nusinovici,Simon %A Cheng,Ching-Yu %A Wong,Tien Yin %A Li,Jialiang %A Sabanayagam,Charumathi %+ Singapore Eye Research Institute, Singapore National Eye Centre, The Academia, 20 College Road, Discovery Tower Level 6, Singapore, 169856, Singapore, 65 6576 7286, charumathi.sabanayagam@seri.com.sg %K machine learning %K diabetic microvascular complication %K diabetic kidney disease %K diabetic retinopathy %K biomarkers %K metabolomics %K complication %K adult %K cardiovascular disease %K metabolites %K biomedical big data %K kidney disease %D 2024 %7 28.3.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Diabetic kidney disease (DKD) and diabetic retinopathy (DR) are major diabetic microvascular complications, contributing significantly to morbidity, disability, and mortality worldwide. The kidney and the eye, having similar microvascular structures and physiological and pathogenic features, may experience similar metabolic changes in diabetes. Objective: This study aimed to use machine learning (ML) methods integrated with metabolic data to identify biomarkers associated with DKD and DR in a multiethnic Asian population with diabetes, as well as to improve the performance of DKD and DR detection models beyond traditional risk factors. Methods: We used ML algorithms (logistic regression [LR] with Least Absolute Shrinkage and Selection Operator and gradient-boosting decision tree) to analyze 2772 adults with diabetes from the Singapore Epidemiology of Eye Diseases study, a population-based cross-sectional study conducted in Singapore (2004-2011). From 220 circulating metabolites and 19 risk factors, we selected the most important variables associated with DKD (defined as an estimated glomerular filtration rate <60 mL/min/1.73 m2) and DR (defined as an Early Treatment Diabetic Retinopathy Study severity level ≥20). DKD and DR detection models were developed based on the variable selection results and externally validated on a sample of 5843 participants with diabetes from the UK biobank (2007-2010). Machine-learned model performance (area under the receiver operating characteristic curve [AUC] with 95% CI, sensitivity, and specificity) was compared to that of traditional LR adjusted for age, sex, diabetes duration, hemoglobin A1c, systolic blood pressure, and BMI. Results: Singapore Epidemiology of Eye Diseases participants had a median age of 61.7 (IQR 53.5-69.4) years, with 49.1% (1361/2772) being women, 20.2% (555/2753) having DKD, and 25.4% (685/2693) having DR. UK biobank participants had a median age of 61.0 (IQR 55.0-65.0) years, with 35.8% (2090/5843) being women, 6.7% (374/5570) having DKD, and 6.1% (355/5843) having DR. The ML algorithms identified diabetes duration, insulin usage, age, and tyrosine as the most important factors of both DKD and DR. DKD was additionally associated with cardiovascular disease history, antihypertensive medication use, and 3 metabolites (lactate, citrate, and cholesterol esters to total lipids ratio in intermediate-density lipoprotein), while DR was additionally associated with hemoglobin A1c, blood glucose, pulse pressure, and alanine. Machine-learned models for DKD and DR detection outperformed traditional LR models in both internal (AUC 0.838 vs 0.743 for DKD and 0.790 vs 0.764 for DR) and external validation (AUC 0.791 vs 0.691 for DKD and 0.778 vs 0.760 for DR). Conclusions: This study highlighted diabetes duration, insulin usage, age, and circulating tyrosine as important factors in detecting DKD and DR. The integration of ML with biomedical big data enables biomarker discovery and improves disease detection beyond traditional risk factors. %M 38546730 %R 10.2196/41065 %U https://www.jmir.org/2024/1/e41065 %U https://doi.org/10.2196/41065 %U http://www.ncbi.nlm.nih.gov/pubmed/38546730 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e55209 %T Telehealth Evaluation in the United States: Protocol for a Scoping Review %A Zhang,Yunxi %A Lin,Yueh-Yun %A Lal,Lincy S %A Reneker,Jennifer C %A Hinton,Elizabeth G %A Chandra,Saurabh %A Swint,J Michael %+ Department of Data Science, John D Bower School of Population Health, University of Mississippi Medical Center, 2500 North State Street, Jackson, MS, 39216-4505, United States, 1 6018153477, yzhang4@umc.edu %K cost %K effectiveness %K evaluation %K framework %K healthcare delivery %K measurement %K quality %K scoping review %K telehealth %K United States %D 2024 %7 28.3.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: The rapid expansion of telehealth services, driven by the COVID-19 pandemic, necessitates systematic evaluation to guarantee the quality, effectiveness, and cost-effectiveness of telehealth services and programs in the United States. While numerous evaluation frameworks have emerged, crafted by various stakeholders, their comprehensiveness is limited, and the overall state of telehealth evaluation remains unclear. Objective: The overarching goal of this scoping review is to create a comprehensive overview of telehealth evaluation, incorporating perspectives from multiple stakeholder categories. Specifically, we aim to (1) map the existing landscape of telehealth evaluation, (2) identify key concepts for evaluation, (3) synthesize existing evaluation frameworks, and (4) identify measurements and assessments considered in the United States. Methods: We will conduct this scoping review in accordance with the Joanna Briggs Institute (JBI) methodology for scoping reviews and in line with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews). This scoping review will consider documents, including reviews, reports, and white papers, published since January 1, 2019. It will focus on evaluation frameworks and associated measurements of telehealth services and programs in the US health care system, developed by telehealth stakeholders, professional organizations, and authoritative sources, excluding those developed by individual researchers, to collect data that reflect the collective expertise and consensus of experts within the respective professional group. Results: The data extracted from selected documents will be synthesized using tools such as tables and figures. Visual aids like Venn diagrams will be used to illustrate the relationships between the evaluation frameworks from various sources. A narrative summary will be crafted to further describe how the results align with the review objectives, facilitating a comprehensive overview of the findings. This scoping review is expected to conclude by August 2024. Conclusions: By addressing critical gaps in telehealth evaluation, this scoping review protocol lays the foundation for a comprehensive and multistakeholder assessment of telehealth services and programs. Its findings will inform policy makers, health care providers, researchers, and other stakeholders in advancing the quality, effectiveness, and cost-effectiveness of telehealth in the US health care system. Trial Registration: OSF Registries osf.io/aytus; https://osf.io/aytus International Registered Report Identifier (IRRID): DERR1-10.2196/55209 %M 38546709 %R 10.2196/55209 %U https://www.researchprotocols.org/2024/1/e55209 %U https://doi.org/10.2196/55209 %U http://www.ncbi.nlm.nih.gov/pubmed/38546709 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e46971 %T The Political Economy of Digital Health Equity: Structural Analysis %A Shaw,James %A Glover,Wiljeana %+ Department of Physical Therapy, Temerty Faculty of Medicine, University of Toronto, 500 University Avenue, Toronto, ON, M5G1V7, Canada, 1 4169780315, jay.shaw@wchospital.ca %K digital health equity %K health equity %K health policy %K telemedicine %K digital care %K political economy %K race %K ethnicity %K socioeconomic %K policy %D 2024 %7 26.3.2024 %9 Viewpoint %J J Med Internet Res %G English %X Digital technologies have produced many innovations in care delivery and enabled continuity of care for many people when in-person care was impossible. However, a growing body of research suggests that digital health can also exacerbate health inequities for those excluded from its benefits for reasons of cost, digital literacy, and structural discrimination related to characteristics such as age, race, ethnicity, and socioeconomic status. In this paper, we draw on a political economy perspective to examine structural barriers to progress in advancing digital health equity at the policy level. Considering the incentive structures and investments of powerful actors in the field, we outline how characteristics of neoliberal capitalism in Western contexts produce and sustain digital health inequities by describing 6 structural challenges to the effort to promote health equity through digital health, as follows: (1) the revenue-first incentives of technology corporations, (2) the influence of venture capital, (3) inequitable access to the internet and digital devices, (4) underinvestment in digital health literacy, (5) uncertainty about future reimbursement of digital health, and (6) justified mistrust of digital health. Building on these important challenges, we propose future immediate and long-term directions for work to support meaningful change for digital health equity. %M 38530341 %R 10.2196/46971 %U https://www.jmir.org/2024/1/e46971 %U https://doi.org/10.2196/46971 %U http://www.ncbi.nlm.nih.gov/pubmed/38530341 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e47248 %T Best Practices in Evolving Privacy Frameworks for Patient Age Data: Census Data Study %A Moffatt,Colin %A Leshin,Jonah %+ Datavant, 44 Montgomery St 3rd floor, San Francisco, CA, 94104, United States, 1 415 520 1171, jonah@datavant.com %K census %K date of birth %K deidentification %K HIPAA %K Health Insurance Portability and Accountability Act %K k-anonymity %K patient privacy %K policy %K reidentification risk %D 2024 %7 25.3.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Over the previous 4 decennial censuses, the population of the United States has grown older, with the proportion of individuals aged at least 90 years old in the 2010 census being more than 2 and a half times what it was in the 1980 census. This suggests that the threshold for constraining age introduced in the Safe Harbor method of the HIPAA (Health Insurance Portability and Accountability Act) in 1996 may be increased without exceeding the original levels of risk. This is desirable to maintain or even increase the utility of affected data sets without compromising privacy. Objective: In light of the upcoming release of 2020 census data, this study presents a straightforward recipe for updating age-constrained thresholds in the context of new census data and derives recommendations for new thresholds from the 2010 census. Methods: Using census data dating back to 1980, we used group size considerations to analyze the risk associated with various maximum age thresholds over time. We inferred the level of risk of the age cutoff of 90 years at the time of HIPAA’s inception in 1996 and used this as a baseline from which to recommend updated cutoffs. Results: The maximum age threshold may be increased by at least 2 years without exceeding the levels of risk conferred in HIPAA’s original recommendations. Moreover, in the presence of additional information that restricts the population in question to a known subgroup with increased longevity (for example, restricting to female patients), the threshold may be increased further. Conclusions: Increasing the maximum age threshold would enable the data user to gain more utility from the data without introducing risk beyond what was originally envisioned with the enactment of HIPAA. Going forward, a recurring update of such thresholds is advised, in line with the considerations detailed in the paper. %M 38526530 %R 10.2196/47248 %U https://formative.jmir.org/2024/1/e47248 %U https://doi.org/10.2196/47248 %U http://www.ncbi.nlm.nih.gov/pubmed/38526530 %0 Journal Article %@ 2561-6722 %I JMIR Publications %V 7 %N %P e46813 %T Provincial Maternal and Child Information System in Inner Mongolia, China: Descriptive Implementation Study %A Yan,Yiwei %A Xing,Congyan %A Chen,Jian %A Zheng,Yingbin %A Li,Xiaobin %A Liu,Yirong %A Wang,Zhanxiang %A Gong,Kai %+ Biomedical Big Data Center, The First Affiliated Hospital of Xiamen University, 10 Shanggu Road, Siming District, Xiamen, 361003, China, 86 15160003918, freatink@xmu.edu.cn %K information system %K maternal and child health care %K system construction %K system implementation %K regional health %K Inner Mongolia Autonomous Region %D 2024 %7 25.3.2024 %9 Original Paper %J JMIR Pediatr Parent %G English %X Background: After the implementation of 2- and 3-child policies, the rising proportion of high-age and high-risk pregnancies put enormous pressure on maternal and child health (MCH) services for China. This populous nation with an increasing population flow imperatively required the support of large-scale information systems for management. Municipal MCH information systems were commonly applied in developed cities of eastern provinces in China. However, implementation of provincial MCH information systems in relatively low-income areas is lacking. In 2020, the implementation of a regional maternal and child information system (RMCIS) in Inner Mongolia filled this gap. Objective: This paper aimed to demonstrate the construction process and evaluate the implementation effect of an RMCIS in improving the regional MCH in Inner Mongolia. Methods: We conducted a descriptive study for the implementation of an RMCIS in Inner Mongolia. Based on the role analysis and information reporting process, the system architecture design had 10 modules, supporting basic health care services, special case management, health support, and administration and supervision. Five-color management was applied for pregnancy risk stratification. We collected data on the construction cost, key characteristics of patients, and use count of the main services from January 1, 2020, to October 31, 2022, in Inner Mongolia. Descriptive analysis was used to demonstrate the implementation effects of the RMCIS. Results: The construction and implementation of the RMCIS cost CNY 8 million (US $1.1 million), with a duration of 13 months. Between 2020 and 2022, the system recorded 221,772 registered pregnant women, with a 44.75% early pregnancy registry rate and 147,264 newborns, covering 278 hospitals and 225 community health care centers in 12 cities. Five-color management of high-risk pregnancies resulted in 76,975 (45.45%) pregnancies stratified as yellow (general risk), 36,627 (21.63%) as orange (relatively high risk), 156 (0.09%) as red (high risk), and 3888 (2.30%) as purple (infectious disease). A scarred uterus (n=28,159, 36.58%), BMI≥28 (n=14,164, 38.67%), aggressive placenta praevia (n=32, 20.51%), and viral hepatitis (n=1787, 45.96%) were the top factors of high-risk pregnancies (yellow, orange, red, and purple). In addition, 132,079 pregnancies, including 65,018 (49.23%) high-risk pregnancies, were registered in 2022 compared to 32,466 pregnancies, including 21,849 (67.30%) high-risk pregnancies, registered in 2020. Conclusions: The implementation of an RMCIS in Inner Mongolia achieved the provincial MCH data interconnection for basic services and obtained both social and economic benefits, which could provide valuable experience to medical administration departments, practitioners, and medical informatics constructors worldwide. %M 38526553 %R 10.2196/46813 %U https://pediatrics.jmir.org/2024/1/e46813 %U https://doi.org/10.2196/46813 %U http://www.ncbi.nlm.nih.gov/pubmed/38526553 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e56933 %T Definitions and Measurements for Atypical Presentations at Risk for Diagnostic Errors in Internal Medicine: Protocol for a Scoping Review %A Harada,Yukinori %A Kawamura,Ren %A Yokose,Masashi %A Shimizu,Taro %A Singh,Hardeep %+ Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, 880 Kitakobayashi, Mibu, 321-0293, Japan, 81 282 86 1111, yharada@dokkyomed.ac.jp %K atypical presentations %K diagnostic errors %K internal medicine %K scoping review protocol %K atypical presentation %K high risk %K data extraction %K descriptive statistics %K criteria %K qualitative %K content analysis %K inductive approach %K clinical informatics %K clinical decision support %D 2024 %7 25.3.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Atypical presentations have been increasingly recognized as a significant contributing factor to diagnostic errors in internal medicine. However, research to address associations between atypical presentations and diagnostic errors has not been evaluated due to the lack of widely applicable definitions and criteria for what is considered an atypical presentation. Objective: The aim of the study is to describe how atypical presentations are defined and measured in studies of diagnostic errors in internal medicine and use this new information to develop new criteria to identify atypical presentations at high risk for diagnostic errors. Methods: This study will follow an established framework for conducting scoping reviews. Inclusion criteria are developed according to the participants, concept, and context framework. This review will consider studies that fulfill all of the following criteria: include adult patients (participants); explore the association between atypical presentations and diagnostic errors using any definition, criteria, or measurement to identify atypical presentations and diagnostic errors (concept); and focus on internal medicine (context). Regarding the type of sources, this scoping review will consider quantitative, qualitative, and mixed methods study designs; systematic reviews; and opinion papers for inclusion. Case reports, case series, and conference abstracts will be excluded. The data will be extracted through MEDLINE, Web of Science, CINAHL, Embase, Cochrane Library, and Google Scholar searches. No limits will be applied to language, and papers indexed from database inception to December 31, 2023, will be included. Two independent reviewers (YH and RK) will conduct study selection and data extraction. The data extracted will include specific details about the patient characteristics (eg, age, sex, and disease), the definitions and measuring methods for atypical presentations and diagnostic errors, clinical settings (eg, department and outpatient or inpatient), type of evidence source, and the association between atypical presentations and diagnostic errors relevant to the review question. The extracted data will be presented in tabular format with descriptive statistics, allowing us to identify the key components or types of atypical presentations and develop new criteria to identify atypical presentations for future studies of diagnostic errors. Developing the new criteria will follow guidance for a basic qualitative content analysis with an inductive approach. Results: As of January 2024, a literature search through multiple databases is ongoing. We will complete this study by December 2024. Conclusions: This scoping review aims to provide rigorous evidence to develop new criteria to identify atypical presentations at high risk for diagnostic errors in internal medicine. Such criteria could facilitate the development of a comprehensive conceptual model to understand the associations between atypical presentations and diagnostic errors in internal medicine. Trial Registration: Open Science Framework; www.osf.io/27d5m International Registered Report Identifier (IRRID): DERR1-10.2196/56933 %M 38526541 %R 10.2196/56933 %U https://www.researchprotocols.org/2024/1/e56933 %U https://doi.org/10.2196/56933 %U http://www.ncbi.nlm.nih.gov/pubmed/38526541 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e53400 %T Forecasting Hospital Room and Ward Occupancy Using Static and Dynamic Information Concurrently: Retrospective Single-Center Cohort Study %A Seo,Hyeram %A Ahn,Imjin %A Gwon,Hansle %A Kang,Heejun %A Kim,Yunha %A Choi,Heejung %A Kim,Minkyoung %A Han,Jiye %A Kee,Gaeun %A Park,Seohyun %A Ko,Soyoung %A Jung,HyoJe %A Kim,Byeolhee %A Oh,Jungsik %A Jun,Tae Joon %A Kim,Young-Hak %+ Division of Cardiology, Department of Information Medicine, Asan Medical Center & University of Ulsan College of Medicine, 88, Olympic-ro 43-gil, Songpa-gu, Seoul, 05505, Republic of Korea, 82 2 3010 0955, mdyhkim@amc.seoul.kr %K hospital bed occupancy %K electronic medical records %K time series forecasting %K short-term memory %K combining static and dynamic variables %D 2024 %7 21.3.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Predicting the bed occupancy rate (BOR) is essential for efficient hospital resource management, long-term budget planning, and patient care planning. Although macro-level BOR prediction for the entire hospital is crucial, predicting occupancy at a detailed level, such as specific wards and rooms, is more practical and useful for hospital scheduling. Objective: The aim of this study was to develop a web-based support tool that allows hospital administrators to grasp the BOR for each ward and room according to different time periods. Methods: We trained time-series models based on long short-term memory (LSTM) using individual bed data aggregated hourly each day to predict the BOR for each ward and room in the hospital. Ward training involved 2 models with 7- and 30-day time windows, and room training involved models with 3- and 7-day time windows for shorter-term planning. To further improve prediction performance, we added 2 models trained by concatenating dynamic data with static data representing room-specific details. Results: We confirmed the results of a total of 12 models using bidirectional long short-term memory (Bi-LSTM) and LSTM, and the model based on Bi-LSTM showed better performance. The ward-level prediction model had a mean absolute error (MAE) of 0.067, mean square error (MSE) of 0.009, root mean square error (RMSE) of 0.094, and R2 score of 0.544. Among the room-level prediction models, the model that combined static data exhibited superior performance, with a MAE of 0.129, MSE of 0.050, RMSE of 0.227, and R2 score of 0.600. Model results can be displayed on an electronic dashboard for easy access via the web. Conclusions: We have proposed predictive BOR models for individual wards and rooms that demonstrate high performance. The results can be visualized through a web-based dashboard, aiding hospital administrators in bed operation planning. This contributes to resource optimization and the reduction of hospital resource use. %M 38513229 %R 10.2196/53400 %U https://medinform.jmir.org/2024/1/e53400 %U https://doi.org/10.2196/53400 %U http://www.ncbi.nlm.nih.gov/pubmed/38513229 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e52073 %T Preliminary Evidence of the Use of Generative AI in Health Care Clinical Services: Systematic Narrative Review %A Yim,Dobin %A Khuntia,Jiban %A Parameswaran,Vijaya %A Meyers,Arlen %+ University of Colorado Denver, 1475 Lawrence St., Denver, CO, United States, 1 3038548024, jiban.khuntia@ucdenver.edu %K generative artificial intelligence tools and applications %K GenAI %K service %K clinical %K health care %K transformation %K digital %D 2024 %7 20.3.2024 %9 Review %J JMIR Med Inform %G English %X Background: Generative artificial intelligence tools and applications (GenAI) are being increasingly used in health care. Physicians, specialists, and other providers have started primarily using GenAI as an aid or tool to gather knowledge, provide information, train, or generate suggestive dialogue between physicians and patients or between physicians and patients’ families or friends. However, unless the use of GenAI is oriented to be helpful in clinical service encounters that can improve the accuracy of diagnosis, treatment, and patient outcomes, the expected potential will not be achieved. As adoption continues, it is essential to validate the effectiveness of the infusion of GenAI as an intelligent technology in service encounters to understand the gap in actual clinical service use of GenAI. Objective: This study synthesizes preliminary evidence on how GenAI assists, guides, and automates clinical service rendering and encounters in health care The review scope was limited to articles published in peer-reviewed medical journals. Methods: We screened and selected 0.38% (161/42,459) of articles published between January 1, 2020, and May 31, 2023, identified from PubMed. We followed the protocols outlined in the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines to select highly relevant studies with at least 1 element on clinical use, evaluation, and validation to provide evidence of GenAI use in clinical services. The articles were classified based on their relevance to clinical service functions or activities using the descriptive and analytical information presented in the articles. Results: Of 161 articles, 141 (87.6%) reported using GenAI to assist services through knowledge access, collation, and filtering. GenAI was used for disease detection (19/161, 11.8%), diagnosis (14/161, 8.7%), and screening processes (12/161, 7.5%) in the areas of radiology (17/161, 10.6%), cardiology (12/161, 7.5%), gastrointestinal medicine (4/161, 2.5%), and diabetes (6/161, 3.7%). The literature synthesis in this study suggests that GenAI is mainly used for diagnostic processes, improvement of diagnosis accuracy, and screening and diagnostic purposes using knowledge access. Although this solves the problem of knowledge access and may improve diagnostic accuracy, it is oriented toward higher value creation in health care. Conclusions: GenAI informs rather than assisting or automating clinical service functions in health care. There is potential in clinical service, but it has yet to be actualized for GenAI. More clinical service–level evidence that GenAI is used to streamline some functions or provides more automated help than only information retrieval is needed. To transform health care as purported, more studies related to GenAI applications must automate and guide human-performed services and keep up with the optimism that forward-thinking health care organizations will take advantage of GenAI. %M 38506918 %R 10.2196/52073 %U https://medinform.jmir.org/2024/1/e52073 %U https://doi.org/10.2196/52073 %U http://www.ncbi.nlm.nih.gov/pubmed/38506918 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e49822 %T Investigating the Roles and Responsibilities of Institutional Signing Officials After Data Sharing Policy Reform for Federally Funded Research in the United States: National Survey %A Baek,Jinyoung %A Lawson,Jonathan %A Rahimzadeh,Vasiliki %+ Center for Medical Ethics and Health Policy, Baylor College of Medicine, 1 Baylor Plaza, Suite 310DF, Houston, TX, 77030, United States, 1 (713) 798 3500, vasiliki.rahimzadeh@bcm.edu %K biomedical research %K survey %K surveys %K data sharing %K data management %K secondary use %K National Institutes of Health %K signing official %K information sharing %K exchange %K access %K data science %K accessibility %K policy %K policies %D 2024 %7 20.3.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: New federal policies along with rapid growth in data generation, storage, and analysis tools are together driving scientific data sharing in the United States. At the same, triangulating human research data from diverse sources can also create situations where data are used for future research in ways that individuals and communities may consider objectionable. Institutional gatekeepers, namely, signing officials (SOs), are therefore at the helm of compliant management and sharing of human data for research. Of those with data governance responsibilities, SOs most often serve as signatories for investigators who deposit, access, and share research data between institutions. Although SOs play important leadership roles in compliant data sharing, we know surprisingly little about their scope of work, roles, and oversight responsibilities. Objective: The purpose of this study was to describe existing institutional policies and practices of US SOs who manage human genomic data access, as well as how these may change in the wake of new Data Management and Sharing requirements for National Institutes of Health–funded research in the United States. Methods: We administered an anonymous survey to institutional SOs recruited from biomedical research institutions across the United States. Survey items probed where data generated from extramurally funded research are deposited, how researchers outside the institution access these data, and what happens to these data after extramural funding ends. Results: In total, 56 institutional SOs participated in the survey. We found that SOs frequently approve duplicate data deposits and impose stricter access controls when data use limitations are unclear or unspecified. In addition, 21% (n=12) of SOs knew where data from federally funded projects are deposited after project funding sunsets. As a consequence, most investigators deposit their scientific data into “a National Institutes of Health–funded repository” to meet the Data Management and Sharing requirements but also within the “institution’s own repository” or a third-party repository. Conclusions: Our findings inform 5 policy recommendations and best practices for US SOs to improve coordination and develop comprehensive and consistent data governance policies that balance the need for scientific progress with effective human data protections. %M 38506894 %R 10.2196/49822 %U https://formative.jmir.org/2024/1/e49822 %U https://doi.org/10.2196/49822 %U http://www.ncbi.nlm.nih.gov/pubmed/38506894 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e53951 %T Clinical Decision Support System Used in Spinal Disorders: Scoping Review %A Toh,Zheng An %A Berg,Bjørnar %A Han,Qin Yun Claudia %A Hey,Hwee Weng Dennis %A Pikkarainen,Minna %A Grotle,Margreth %A He,Hong-Gu %+ National University Hospital, National University Health System, 5 Lower Kent Ridge Road, Singapore, 119074, Singapore, 65 92289289, E0191325@u.nus.edu %K back pain %K clinical decision support systems %K CDSS %K diagnosis %K imaging %K predictive %K prognosis %K spine %D 2024 %7 19.3.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Spinal disorders are highly prevalent worldwide with high socioeconomic costs. This cost is associated with the demand for treatment and productivity loss, prompting the exploration of technologies to improve patient outcomes. Clinical decision support systems (CDSSs) are computerized systems that are increasingly used to facilitate safe and efficient health care. Their applications range in depth and can be found across health care specialties. Objective: This scoping review aims to explore the use of CDSSs in patients with spinal disorders. Methods: We used the Joanna Briggs Institute methodological guidance for this scoping review and reported according to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) statement. Databases, including PubMed, Embase, Cochrane, CINAHL, Web of Science, Scopus, ProQuest, and PsycINFO, were searched from inception until October 11, 2022. The included studies examined the use of digitalized CDSSs in patients with spinal disorders. Results: A total of 4 major CDSS functions were identified from 31 studies: preventing unnecessary imaging (n=8, 26%), aiding diagnosis (n=6, 19%), aiding prognosis (n=11, 35%), and recommending treatment options (n=6, 20%). Most studies used the knowledge-based system. Logistic regression was the most commonly used method, followed by decision tree algorithms. The use of CDSSs to aid in the management of spinal disorders was generally accepted over the threat to physicians’ clinical decision-making autonomy. Conclusions: Although the effectiveness was frequently evaluated by examining the agreement between the decisions made by the CDSSs and the health care providers, comparing the CDSS recommendations with actual clinical outcomes would be preferable. In addition, future studies on CDSS development should focus on system integration, considering end user’s needs and preferences, and external validation and impact studies to assess effectiveness and generalizability. Trial Registration: OSF Registries osf.io/dyz3f; https://osf.io/dyz3f %M 38502157 %R 10.2196/53951 %U https://www.jmir.org/2024/1/e53951 %U https://doi.org/10.2196/53951 %U http://www.ncbi.nlm.nih.gov/pubmed/38502157 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e52071 %T An Electronic Health Record–Based Automated Self-Rescheduling Tool to Improve Patient Access: Retrospective Cohort Study %A Ganeshan,Smitha %A Liu,Andrew W %A Kroeger,Anne %A Anand,Prerna %A Seefeldt,Richard %A Regner,Alexis %A Vaughn,Diana %A Odisho,Anobel Y %A Mourad,Michelle %+ Department of Medicine, University of California San Francisco, 505 Parnassus Avenue, #M1493, San Francisco, CA, 94117‭, United States, 1 415 514 1000, smitha.ganeshan@ucsf.edu %K appointment %K consultation %K cost %K digital health %K digital tools %K electronic health record %K EHR %K informatics %K patient access %K retrospective review %K revenue %K self-rescheduling tool %K self-scheduling %K waiting time %D 2024 %7 19.3.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: In many large health centers, patients face long appointment wait times and difficulties accessing care. Last-minute cancellations and patient no-shows leave unfilled slots in a clinician’s schedule, exacerbating delays in care from poor access. The mismatch between the supply of outpatient appointments and patient demand has led health systems to adopt many tools and strategies to minimize appointment no-show rates and fill open slots left by patient cancellations. Objective: We evaluated an electronic health record (EHR)–based self-scheduling tool, Fast Pass, at a large academic medical center to understand the impacts of the tool on the ability to fill cancelled appointment slots, patient access to earlier appointments, and clinical revenue from visits that may otherwise have gone unscheduled. Methods: In this retrospective cohort study, we extracted Fast Pass appointment offers and scheduling data, including patient demographics, from the EHR between June 18, 2022, and March 9, 2023. We analyzed the outcomes of Fast Pass offers (accepted, declined, expired, and unavailable) and the outcomes of scheduled appointments resulting from accepted Fast Pass offers (completed, canceled, and no-show). We stratified outcomes based on appointment specialty. For each specialty, the patient service revenue from appointments filled by Fast Pass was calculated using the visit slots filled, the payer mix of the appointments, and the contribution margin by payer. Results: From June 18 to March 9, 2023, there were a total of 60,660 Fast Pass offers sent to patients for 21,978 available appointments. Of these offers, 6603 (11%) were accepted across all departments, and 5399 (8.9%) visits were completed. Patients were seen a median (IQR) of 14 (4-33) days sooner for their appointments. In a multivariate logistic regression model with primary outcome Fast Pass offer acceptance, patients who were aged 65 years or older (vs 20-40 years; P=.005 odds ratio [OR] 0.86, 95% CI 0.78-0.96), other ethnicity (vs White; P<.001, OR 0.84, 95% CI 0.77-0.91), primarily Chinese speakers (P<.001; OR 0.62, 95% CI 0.49-0.79), and other language speakers (vs English speakers; P=.001; OR 0.71, 95% CI 0.57-0.87) were less likely to accept an offer. Fast Pass added 2576 patient service hours to the clinical schedule, with a median (IQR) of 251 (216-322) hours per month. The estimated value of physician fees from these visits scheduled through 9 months of Fast Pass scheduling in professional fees at our institution was US $3 million. Conclusions: Self-scheduling tools that provide patients with an opportunity to schedule into cancelled or unfilled appointment slots have the potential to improve patient access and efficiently capture additional revenue from filling unfilled slots. The demographics of the patients accepting these offers suggest that such digital tools may exacerbate inequities in access. %M 38502159 %R 10.2196/52071 %U https://www.jmir.org/2024/1/e52071 %U https://doi.org/10.2196/52071 %U http://www.ncbi.nlm.nih.gov/pubmed/38502159 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e50177 %T Implementing and Evaluating a National Integrated Digital Registry and Clinical Decision Support System in Early Intervention in Psychosis Services (Early Psychosis Informatics Into Care): Co-Designed Protocol %A Griffiths,Siân Lowri %A Murray,Graham K %A Logeswaran,Yanakan %A Ainsworth,John %A Allan,Sophie M %A Campbell,Niyah %A Drake,Richard J %A Katshu,Mohammad Zia Ul Haq %A Machin,Matthew %A Pope,Megan A %A Sullivan,Sarah A %A Waring,Justin %A Bogatsu,Tumelo %A Kane,Julie %A Weetman,Tyler %A Johnson,Sonia %A Kirkbride,James B %A Upthegrove,Rachel %+ Institute for Mental Health, University of Birmingham, Wolfson Centre, 52 Pritchatts Road, Birmingham, B15 2TT, United Kingdom, 44 01214146241, r.upthegrove@bham.ac.uk %K Early Intervention in Psychosis %K digital registry %K clinical decision support system %K participatory co-design %K participatory %K co-design %K registry %K psychosis %K mental health %K psychiatry %K decision support %K study protocol %D 2024 %7 19.3.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Early intervention in psychosis (EIP) services are nationally mandated in England to provide multidisciplinary care to people experiencing first-episode psychosis, which disproportionately affects deprived and ethnic minority youth. Quality of service provision varies by region, and people from historically underserved populations have unequal access. In other disease areas, including stroke and dementia, national digital registries coupled with clinical decision support systems (CDSSs) have revolutionized the delivery of equitable, evidence-based interventions to transform patient outcomes and reduce population-level disparities in care. Given psychosis is ranked the third most burdensome mental health condition by the World Health Organization, it is essential that we achieve the same parity of health improvements. Objective: This paper reports the protocol for the program development phase of this study, in which we aimed to co-design and produce an evidence-based, stakeholder-informed framework for the building, implementation, piloting, and evaluation of a national integrated digital registry and CDSS for psychosis, known as EPICare (Early Psychosis Informatics into Care). Methods: We conducted 3 concurrent work packages, with reciprocal knowledge exchange between each. In work package 1, using a participatory co-design framework, key stakeholders (clinicians, academics, policy makers, and patient and public contributors) engaged in 4 workshops to review, refine, and identify a core set of essential and desirable measures and features of the EPICare registry and CDSS. Using a modified Delphi approach, we then developed a consensus of data priorities. In work package 2, we collaborated with National Health Service (NHS) informatics teams to identify relevant data currently captured in electronic health records, understand data retrieval methods, and design the software architecture and data model to inform future implementation. In work package 3, observations of stakeholder workshops and individual interviews with representative stakeholders (n=10) were subject to interpretative qualitative analysis, guided by normalization process theory, to identify factors likely to influence the adoption and implementation of EPICare into routine practice. Results: Stage 1 of the EPICare study took place between December 2021 and September 2022. The next steps include stage 2 building, piloting, implementation, and evaluation of EPICare in 5 demonstrator NHS Trusts serving underserved and diverse populations with substantial need for EIP care in England. If successful, this will be followed by stage 3, in which we will seek NHS adoption of EPICare for rollout to all EIP services in England. Conclusions: By establishing a multistakeholder network and engaging them in an iterative co-design process, we have identified essential and desirable elements of the EPICare registry and CDSS; proactively identified and minimized potential challenges and barriers to uptake and implementation; and addressed key questions related to informatics architecture, infrastructure, governance, and integration in diverse NHS Trusts, enabling us to proceed with the building, piloting, implementation, and evaluation of EPICare. International Registered Report Identifier (IRRID): DERR1-10.2196/50177 %M 38502175 %R 10.2196/50177 %U https://www.researchprotocols.org/2024/1/e50177 %U https://doi.org/10.2196/50177 %U http://www.ncbi.nlm.nih.gov/pubmed/38502175 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e45070 %T Outcomes and Costs of the Transition From a Paper-Based Immunization System to a Digital Immunization System in Vietnam: Mixed Methods Study %A Dang,Thi Thanh Huyen %A Carnahan,Emily %A Nguyen,Linh %A Mvundura,Mercy %A Dao,Sang %A Duong,Thi Hong %A Nguyen,Trung %A Nguyen,Doan %A Nguyen,Tu %A Werner,Laurie %A Ryman,Tove K %A Nguyen,Nga %+ PATH, 1101, 11th floor, Hanoi Towers, 49 Hai Ba Trung Street, Hanoi, 100000, Vietnam, 84 243936221 ext 130, ntnguyen@path.org %K eHealth %K digital health %K immunization information system %K electronic immunization registry %K immunization %K data quality %K data use %K costing %D 2024 %7 18.3.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: The electronic National Immunization Information System (NIIS) was introduced nationwide in Vietnam in 2017. Health workers were expected to use the NIIS alongside the legacy paper-based system. Starting in 2018, Hanoi and Son La provinces transitioned to paperless reporting. Interventions to support this transition included data guidelines and training, internet-based data review meetings, and additional supportive supervision visits. Objective: This study aims to assess (1) changes in NIIS data quality and use, (2) changes in immunization program outcomes, and (3) the economic costs of using the NIIS versus the traditional paper system. Methods: This mixed methods study took place in Hanoi and Son La provinces. It aimed to analyses pre- and postintervention data from various sources including the NIIS; household and health facility surveys; and interviews to measure NIIS data quality, data use, and immunization program outcomes. Financial data were collected at the national, provincial, district, and health facility levels through record review and interviews. An activity-based costing approach was conducted from a health system perspective. Results: NIIS data timeliness significantly improved from pre- to postintervention in both provinces. For example, the mean number of days from birth date to NIIS registration before and after intervention dropped from 18.6 (SD 65.5) to 5.7 (SD 31.4) days in Hanoi (P<.001) and from 36.1 (SD 94.2) to 11.7 (40.1) days in Son La (P<.001). Data from Son La showed that the completeness and accuracy improved, while Hanoi exhibited mixed results, possibly influenced by the COVID-19 pandemic. Data use improved; at postintervention, 100% (667/667) of facilities in both provinces used NIIS data for activities beyond monthly reporting compared with 34.8% (202/580) in Hanoi and 29.4% (55/187) in Son La at preintervention. Across nearly all antigens, the percentage of children who received the vaccine on time was higher in the postintervention cohort compared with the preintervention cohort. Up-front costs associated with developing and deploying the NIIS were estimated at US $0.48 per child in the study provinces. The commune health center level showed cost savings from changing from the paper system to the NIIS, mainly driven by human resource time savings. At the administrative level, incremental costs resulted from changing from the paper system to the NIIS, as some costs increased, such as labor costs for supportive supervision and additional capital costs for equipment associated with the NIIS. Conclusions: The Hanoi and Son La provinces successfully transitioned to paperless reporting while maintaining or improving NIIS data quality and data use. However, improvements in data quality were not associated with improvements in the immunization program outcomes in both provinces. The COVID-19 pandemic likely had a negative influence on immunization program outcomes, particularly in Hanoi. These improvements entail up-front financial costs. %M 38498020 %R 10.2196/45070 %U https://www.jmir.org/2024/1/e45070 %U https://doi.org/10.2196/45070 %U http://www.ncbi.nlm.nih.gov/pubmed/38498020 %0 Journal Article %@ 2371-4379 %I JMIR Publications %V 9 %N %P e52688 %T New Approach to Equitable Intervention Planning to Improve Engagement and Outcomes in a Digital Health Program: Simulation Study %A Killian,Jackson A %A Jain,Manish %A Jia,Yugang %A Amar,Jonathan %A Huang,Erich %A Tambe,Milind %+ Verily Life Sciences, 269 East Grand Avenue, South San Francisco, CA, 94080, United States, 1 9143744981, yugang@verily.com %K chronic disease %K type-2 diabetes %K T2D %K restless multiarmed bandits %K multi-armed bandit %K multi-armed bandits %K machine learning %K resource allocation %K digital health %K equity %D 2024 %7 15.3.2024 %9 Original Paper %J JMIR Diabetes %G English %X Background: Digital health programs provide individualized support to patients with chronic diseases and their effectiveness is measured by the extent to which patients achieve target individual clinical outcomes and the program’s ability to sustain patient engagement. However, patient dropout and inequitable intervention delivery strategies, which may unintentionally penalize certain patient subgroups, represent challenges to maximizing effectiveness. Therefore, methodologies that optimize the balance between success factors (achievement of target clinical outcomes and sustained engagement) equitably would be desirable, particularly when there are resource constraints. Objective: Our objectives were to propose a model for digital health program resource management that accounts jointly for the interaction between individual clinical outcomes and patient engagement, ensures equitable allocation as well as allows for capacity planning, and conducts extensive simulations using publicly available data on type 2 diabetes, a chronic disease. Methods: We propose a restless multiarmed bandit (RMAB) model to plan interventions that jointly optimize long-term engagement and individual clinical outcomes (in this case measured as the achievement of target healthy glucose levels). To mitigate the tendency of RMAB to achieve good aggregate performance by exacerbating disparities between groups, we propose new equitable objectives for RMAB and apply bilevel optimization algorithms to solve them. We formulated a model for the joint evolution of patient engagement and individual clinical outcome trajectory to capture the key dynamics of interest in digital chronic disease management programs. Results: In simulation exercises, our optimized intervention policies lead to up to 10% more patients reaching healthy glucose levels after 12 months, with a 10% reduction in dropout compared to standard-of-care baselines. Further, our new equitable policies reduce the mean absolute difference of engagement and health outcomes across 6 demographic groups by up to 85% compared to the state-of-the-art. Conclusions: Planning digital health interventions with individual clinical outcome objectives and long-term engagement dynamics as considerations can be both feasible and effective. We propose using an RMAB sequential decision-making framework, which may offer additional capabilities in capacity planning as well. The integration of an equitable RMAB algorithm further enhances the potential for reaching equitable solutions. This approach provides program designers with the flexibility to switch between different priorities and balance trade-offs across various objectives according to their preferences. %M 38488828 %R 10.2196/52688 %U https://diabetes.jmir.org/2024/1/e52688 %U https://doi.org/10.2196/52688 %U http://www.ncbi.nlm.nih.gov/pubmed/38488828 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e50737 %T Implementing a Sodium-Glucose Cotransporter 2 Inhibitor Module With a Software Tool (Future Health Today): Qualitative Study %A Suen,Matthew %A Manski-Nankervis,Jo-Anne %A McBride,Caroline %A Lumsden,Natalie %A Hunter,Barbara %+ Department of General Practice and Primary Care, University of Melbourne, Level 3, North Wing, Building, 181 Grattan Street, Medical Building, Parkville, 3010, Australia, 61 383443369, matthew.suen@unimelb.edu.au %K type 2 diabetes %K CP-FIT %K electronic health %K clinical decision support tool %K primary care %K SGLT2 inhibitor %K complication %K tool %K digital health intervention %K thematic analysis %K decision support %K diabetes management %D 2024 %7 13.3.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Primary care plays a key role in the management of type 2 diabetes. Sodium-glucose cotransporter 2 (SGLT2) inhibitors have been demonstrated to reduce hospitalization and cardiac and renal complications. Tools that optimize management, including appropriate prescribing, are a priority for treating chronic diseases. Future Health Today (FHT) is software that facilitates clinical decision support and quality improvement. FHT applies algorithms to data stored in electronic medical records in general practice to identify patients who are at risk of a chronic disease or who have a chronic disease that may benefit from intensification of management. The platform continues to evolve because of rigorous evaluation, continuous improvement, and expansion of the conditions hosted on the platform. FHT currently displays recommendations for the identification and management of chronic kidney disease, cardiovascular disease, type 2 diabetes, and cancer risk. A new module will be introduced to FHT focusing on SGLT2 inhibitors in patients with type 2 diabetes who have chronic kidney diseases, cardiovascular diseases, or risk factors for cardiovascular disease. Objective: The study aims to explore the barriers and enablers to the implementation of an SGLT2 inhibitor module within the Future Health Today software. Methods: Clinic staff were recruited to participate in interviews on their experience in their use of a tool to improve prescribing behavior for SGLT2 inhibitors. Thematic analysis was guided by Clinical Performance Feedback Intervention Theory. Results: In total, 16 interviews were completed. Identified enablers of use included workflow alignment, clinical appropriateness, and active delivery of the module. Key barriers to use were competing priorities, staff engagement, and knowledge of the clinical topic. Conclusions: There is a recognized benefit to the use of a clinical decision support tool to support type 2 diabetes management, but barriers were identified that impeded the usability and actionability of the module. Successful and effective implementation of this tool could support the optimization of patient management of type 2 diabetes in primary care. %M 38477973 %R 10.2196/50737 %U https://formative.jmir.org/2024/1/e50737 %U https://doi.org/10.2196/50737 %U http://www.ncbi.nlm.nih.gov/pubmed/38477973 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e54732 %T Development of a Social Risk Score in the Electronic Health Record to Identify Social Needs Among Underserved Populations: Retrospective Study %A Hatef,Elham %A Chang,Hsien-Yen %A Richards,Thomas M %A Kitchen,Christopher %A Budaraju,Janya %A Foroughmand,Iman %A Lasser,Elyse C %A Weiner,Jonathan P %+ Division of General Internal Medicine, Department of Medicine, Johns Hopkins School of Medicine, 624 N Broadway, Room 502, Baltimore, MD, 21205, United States, 1 4109788006, ehatef1@jhu.edu %K AI %K algorithms %K artificial intelligence %K community health %K deep learning %K EHR %K electronic health record %K machine learning %K ML %K population demographics %K population health %K practical models %K predictive analytics %K predictive modeling %K predictive modelling %K predictive models %K predictive system %K public health %K public surveillance %K SDOH %K social determinants of health %K social needs %K social risks %D 2024 %7 12.3.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Patients with unmet social needs and social determinants of health (SDOH) challenges continue to face a disproportionate risk of increased prevalence of disease, health care use, higher health care costs, and worse outcomes. Some existing predictive models have used the available data on social needs and SDOH challenges to predict health-related social needs or the need for various social service referrals. Despite these one-off efforts, the work to date suggests that many technical and organizational challenges must be surmounted before SDOH-integrated solutions can be implemented on an ongoing, wide-scale basis within most US-based health care organizations. Objective: We aimed to retrieve available information in the electronic health record (EHR) relevant to the identification of persons with social needs and to develop a social risk score for use within clinical practice to better identify patients at risk of having future social needs. Methods: We conducted a retrospective study using EHR data (2016-2021) and data from the US Census American Community Survey. We developed a prospective model using current year-1 risk factors to predict future year-2 outcomes within four 2-year cohorts. Predictors of interest included demographics, previous health care use, comorbidity, previously identified social needs, and neighborhood characteristics as reflected by the area deprivation index. The outcome variable was a binary indicator reflecting the likelihood of the presence of a patient with social needs. We applied a generalized estimating equation approach, adjusting for patient-level risk factors, the possible effect of geographically clustered data, and the effect of multiple visits for each patient. Results: The study population of 1,852,228 patients included middle-aged (mean age range 53.76-55.95 years), White (range 324,279/510,770, 63.49% to 290,688/488,666, 64.79%), and female (range 314,741/510,770, 61.62% to 278,488/448,666, 62.07%) patients from neighborhoods with high socioeconomic status (mean area deprivation index percentile range 28.76-30.31). Between 8.28% (37,137/448,666) and 11.55% (52,037/450,426) of patients across the study cohorts had at least 1 social need documented in their EHR, with safety issues and economic challenges (ie, financial resource strain, employment, and food insecurity) being the most common documented social needs (87,152/1,852,228, 4.71% and 58,242/1,852,228, 3.14% of overall patients, respectively). The model had an area under the curve of 0.702 (95% CI 0.699-0.705) in predicting prospective social needs in the overall study population. Previous social needs (odds ratio 3.285, 95% CI 3.237-3.335) and emergency department visits (odds ratio 1.659, 95% CI 1.634-1.684) were the strongest predictors of future social needs. Conclusions: Our model provides an opportunity to make use of available EHR data to help identify patients with high social needs. Our proposed social risk score could help identify the subset of patients who would most benefit from further social needs screening and data collection to avoid potentially more burdensome primary data collection on all patients in a target population of interest. %M 38470477 %R 10.2196/54732 %U https://formative.jmir.org/2024/1/e54732 %U https://doi.org/10.2196/54732 %U http://www.ncbi.nlm.nih.gov/pubmed/38470477 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e47715 %T The Impact of Digital Hospitals on Patient and Clinician Experience: Systematic Review and Qualitative Evidence Synthesis %A Canfell,Oliver J %A Woods,Leanna %A Meshkat,Yasaman %A Krivit,Jenna %A Gunashanhar,Brinda %A Slade,Christine %A Burton-Jones,Andrew %A Sullivan,Clair %+ Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Level 5 Health Sciences Building, Central, Fig Tree Cres, Brisbane, 4006, Australia, 61 731765530, o.canfell@uq.edu.au %K electronic medical record %K electronic health record %K health care professionals %K patients %K patient satisfaction %K hospitals %K eHealth %K attitude %K perception %K systematic %K digital hospital %K digital hospitals %K experience %K satisfaction %D 2024 %7 11.3.2024 %9 Review %J J Med Internet Res %G English %X Background: The digital transformation of health care is advancing rapidly. A well-accepted framework for health care improvement is the Quadruple Aim: improved clinician experience, improved patient experience, improved population health, and reduced health care costs. Hospitals are attempting to improve care by using digital technologies, but the effectiveness of these technologies is often only measured against cost and quality indicators, and less is known about the clinician and patient experience. Objective: This study aims to conduct a systematic review and qualitative evidence synthesis to assess the clinician and patient experience of digital hospitals. Methods: The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and ENTREQ (Enhancing the Transparency in Reporting the Synthesis of Qualitative Research) guidelines were followed. The PubMed, Embase, Scopus, CINAHL, and PsycINFO databases were searched from January 2010 to June 2022. Studies that explored multidisciplinary clinician or adult inpatient experiences of digital hospitals (with a full electronic medical record) were included. Study quality was assessed using the Mixed Methods Appraisal Tool. Data synthesis was performed narratively for quantitative studies. Qualitative evidence synthesis was performed via (1) automated machine learning text analytics using Leximancer (Leximancer Pty Ltd) and (2) researcher-led inductive synthesis to generate themes. Results: A total of 61 studies (n=39, 64% quantitative; n=15, 25% qualitative; and n=7, 11% mixed methods) were included. Most studies (55/61, 90%) investigated clinician experiences, whereas few (10/61, 16%) investigated patient experiences. The study populations ranged from 8 to 3610 clinicians, 11 to 34,425 patients, and 5 to 2836 hospitals. Quantitative outcomes indicated that clinicians had a positive overall satisfaction (17/24, 71% of the studies) with digital hospitals, and most studies (11/19, 58%) reported a positive sentiment toward usability. Data accessibility was reported positively, whereas adaptation, clinician-patient interaction, and workload burnout were reported negatively. The effects of digital hospitals on patient safety and clinicians’ ability to deliver patient care were mixed. The qualitative evidence synthesis of clinician experience studies (18/61, 30%) generated 7 themes: inefficient digital documentation, inconsistent data quality, disruptions to conventional health care relationships, acceptance, safety versus risk, reliance on hybrid (digital and paper) workflows, and patient data privacy. There was weak evidence of a positive association between digital hospitals and patient satisfaction scores. Conclusions: Clinicians’ experience of digital hospitals appears positive according to high-level indicators (eg, overall satisfaction and data accessibility), but the qualitative evidence synthesis revealed substantive tensions. There is insufficient evidence to draw a definitive conclusion on the patient experience within digital hospitals, but indications appear positive or agnostic. Future research must prioritize equitable investigation and definition of the digital clinician and patient experience to achieve the Quadruple Aim of health care. %M 38466978 %R 10.2196/47715 %U https://www.jmir.org/2024/1/e47715 %U https://doi.org/10.2196/47715 %U http://www.ncbi.nlm.nih.gov/pubmed/38466978 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e52744 %T Machine Learning Model for Readmission Prediction of Patients With Heart Failure Based on Electronic Health Records: Protocol for a Quasi-Experimental Study for Impact Assessment %A Nair,Monika %A Lundgren,Lina E %A Soliman,Amira %A Dryselius,Petra %A Fogelberg,Ebba %A Petersson,Marcus %A Hamed,Omar %A Triantafyllou,Miltiadis %A Nygren,Jens %+ School of Health and Welfare, Halmstad University, Kristian IV:s väg 3, Halmstad, 30118, Sweden, 46 707993854, monika.nair@hh.se %K artificial intelligence %K machine learning %K readmission prediction %K heart failure %K clinical decision support %K machine learning model %K CHF %K congestive heart failure %K readmission %K prediction %K electronic health records %K electronic health record %K EHR %K quasi-experimental study %K decision-making process %K risk assessment %K risk assessment tool %K predictive models %K predictive model %K Sweden %K physician %K nurse %K nurses %K clinician %K clinicians %D 2024 %7 11.3.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Care for patients with heart failure (HF) causes a substantial load on health care systems where a prominent challenge is the elevated rate of readmissions within 30 days following initial discharge. Clinical professionals face high levels of uncertainty and subjectivity in the decision-making process on the optimal timing of discharge. Unwanted hospital stays generate costs and cause stress to patients and potentially have an impact on care outcomes. Recent studies have aimed to mitigate the uncertainty by developing and testing risk assessment tools and predictive models to identify patients at risk of readmission, often using novel methods such as machine learning (ML). Objective: This study aims to investigate how a developed clinical decision support (CDS) tool alters the decision-making processes of health care professionals in the specific context of discharging patients with HF, and if so, in which ways. Additionally, the aim is to capture the experiences of health care practitioners as they engage with the system’s outputs to analyze usability aspects and obtain insights related to future implementation. Methods: A quasi-experimental design with randomized crossover assessment will be conducted with health care professionals on HF patients’ scenarios in a region located in the South of Sweden. In total, 12 physicians and nurses will be randomized into control and test groups. The groups shall be provided with 20 scenarios of purposefully sampled patients. The clinicians will be asked to take decisions on the next action regarding a patient. The test group will be provided with the 10 scenarios containing patient data from electronic health records and an outcome from an ML-based CDS model on the risk level for readmission of the same patients. The control group will have 10 other scenarios without the CDS model output and containing only the patients’ data from electronic medical records. The groups will switch roles for the next 10 scenarios. This study will collect data through interviews and observations. The key outcome measures are decision consistency, decision quality, work efficiency, perceived benefits of using the CDS model, reliability, validity, and confidence in the CDS model outcome, integrability in the routine workflow, ease of use, and intention to use. This study will be carried out in collaboration with Cambio Healthcare Systems. Results: The project is part of the Center for Applied Intelligent Systems Research Health research profile, funded by the Knowledge Foundation (2021-2028). Ethical approval for this study was granted by the Swedish ethical review authority (2022-07287-02). The recruitment process of the clinicians and the patient scenario selection will start in September 2023 and last till March 2024. Conclusions: This study protocol will contribute to the development of future formative evaluation studies to test ML models with clinical professionals. International Registered Report Identifier (IRRID): PRR1-10.2196/52744 %M 38466983 %R 10.2196/52744 %U https://www.researchprotocols.org/2024/1/e52744 %U https://doi.org/10.2196/52744 %U http://www.ncbi.nlm.nih.gov/pubmed/38466983 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e53008 %T Generative AI in Medical Practice: In-Depth Exploration of Privacy and Security Challenges %A Chen,Yan %A Esmaeilzadeh,Pouyan %+ Department of Information Systems and Business Analytics, College of Business, Florida International University, Modesto A Maidique Campus, 11200 SW 8th St, RB 261 B, Miami, FL, 33199, United States, 1 3053483302, pesmaeil@fiu.edu %K artificial intelligence %K AI %K generative artificial intelligence %K generative AI %K medical practices %K potential benefits %K security and privacy threats %D 2024 %7 8.3.2024 %9 Viewpoint %J J Med Internet Res %G English %X As advances in artificial intelligence (AI) continue to transform and revolutionize the field of medicine, understanding the potential uses of generative AI in health care becomes increasingly important. Generative AI, including models such as generative adversarial networks and large language models, shows promise in transforming medical diagnostics, research, treatment planning, and patient care. However, these data-intensive systems pose new threats to protected health information. This Viewpoint paper aims to explore various categories of generative AI in health care, including medical diagnostics, drug discovery, virtual health assistants, medical research, and clinical decision support, while identifying security and privacy threats within each phase of the life cycle of such systems (ie, data collection, model development, and implementation phases). The objectives of this study were to analyze the current state of generative AI in health care, identify opportunities and privacy and security challenges posed by integrating these technologies into existing health care infrastructure, and propose strategies for mitigating security and privacy risks. This study highlights the importance of addressing the security and privacy threats associated with generative AI in health care to ensure the safe and effective use of these systems. The findings of this study can inform the development of future generative AI systems in health care and help health care organizations better understand the potential benefits and risks associated with these systems. By examining the use cases and benefits of generative AI across diverse domains within health care, this paper contributes to theoretical discussions surrounding AI ethics, security vulnerabilities, and data privacy regulations. In addition, this study provides practical insights for stakeholders looking to adopt generative AI solutions within their organizations. %M 38457208 %R 10.2196/53008 %U https://www.jmir.org/2024/1/e53008 %U https://doi.org/10.2196/53008 %U http://www.ncbi.nlm.nih.gov/pubmed/38457208 %0 Journal Article %@ 2561-9128 %I JMIR Publications %V 7 %N %P e57012 %T Blood Management: A Current Opportunity in Perioperative Medicine %A Auron,Moises %+ Department of Hospital Medicine, Cleveland Clinic, 9500 Euclid Ave, Cleveland, OH, 44195, United States, 1 2164458383, auronm@ccf.org %K blood management %K perioperative %K anemia %K plasma %K transfusion %D 2024 %7 8.3.2024 %9 Viewpoint %J JMIR Perioper Med %G English %X The purpose of this viewpoint is to provide awareness of the current opportunities to enhance a high-value care approach to blood product transfusion. It provides a historical context to the evolution of blood management, as well as of the patient safety and high-value care movement. Leveraging current technology for enhanced education, as well as clinical decision support, is also discussed. %M 38457232 %R 10.2196/57012 %U https://periop.jmir.org/2024/1/e57012 %U https://doi.org/10.2196/57012 %U http://www.ncbi.nlm.nih.gov/pubmed/38457232 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e46817 %T Comparison of the Discrimination Performance of AI Scoring and the Brixia Score in Predicting COVID-19 Severity on Chest X-Ray Imaging: Diagnostic Accuracy Study %A Tenda,Eric Daniel %A Yunus,Reyhan Eddy %A Zulkarnaen,Benny %A Yugo,Muhammad Reynalzi %A Pitoyo,Ceva Wicaksono %A Asaf,Moses Mazmur %A Islamiyati,Tiara Nur %A Pujitresnani,Arierta %A Setiadharma,Andry %A Henrina,Joshua %A Rumende,Cleopas Martin %A Wulani,Vally %A Harimurti,Kuntjoro %A Lydia,Aida %A Shatri,Hamzah %A Soewondo,Pradana %A Yusuf,Prasandhya Astagiri %+ Department of Medical Physiology and Biophysics/ Medical Technology Cluster IMERI, Faculty of Medicine, Universitas Indonesia, Jalan Salemba Raya No.6, Jakarta, 10430, Indonesia, 62 812 8459 4272, prasandhya.a.yusuf@ui.ac.id %K artificial intelligence %K Brixia %K chest x-ray %K COVID-19 %K CAD4COVID %K pneumonia %K radiograph %K artificial intelligence scoring system %K AI scoring system %K prediction %K disease severity %D 2024 %7 7.3.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: The artificial intelligence (AI) analysis of chest x-rays can increase the precision of binary COVID-19 diagnosis. However, it is unknown if AI-based chest x-rays can predict who will develop severe COVID-19, especially in low- and middle-income countries. Objective: The study aims to compare the performance of human radiologist Brixia scores versus 2 AI scoring systems in predicting the severity of COVID-19 pneumonia. Methods: We performed a cross-sectional study of 300 patients suspected with and with confirmed COVID-19 infection in Jakarta, Indonesia. A total of 2 AI scores were generated using CAD4COVID x-ray software. Results: The AI probability score had slightly lower discrimination (area under the curve [AUC] 0.787, 95% CI 0.722-0.852). The AI score for the affected lung area (AUC 0.857, 95% CI 0.809-0.905) was almost as good as the human Brixia score (AUC 0.863, 95% CI 0.818-0.908). Conclusions: The AI score for the affected lung area and the human radiologist Brixia score had similar and good discrimination performance in predicting COVID-19 severity. Our study demonstrated that using AI-based diagnostic tools is possible, even in low-resource settings. However, before it is widely adopted in daily practice, more studies with a larger scale and that are prospective in nature are needed to confirm our findings. %M 38451633 %R 10.2196/46817 %U https://formative.jmir.org/2024/1/e46817 %U https://doi.org/10.2196/46817 %U http://www.ncbi.nlm.nih.gov/pubmed/38451633 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e52469 %T Patients’ Experiences of Digital Health Interventions for the Self-Management of Chronic Pain: Protocol for a Systematic Review and Thematic Synthesis %A Main,Ashleigh %A McCartney,Haruno %A Ibrar,Maryam %A Rai,Harleen Kaur %A Muirhead,Fiona %A Mavroeidi,Alexandra %A Maguire,Roma %+ Department of Computer and Information Sciences, University of Strathclyde, 26 Richmond Street, Glasgow, G1 1XH, United Kingdom, 44 (0)141 552 4400, ashleigh.main@strath.ac.uk %K chronic pain %K digital health %K digital tool %K digital health intervention %K mHealth %K eHealth %K pain-management %K person-centered %K experience %K protocol %K patients' experiences %K patient experiences %K self-management %K systematic review %K thematic synthesis %K protocol. %D 2024 %7 7.3.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Chronic pain is a highly prevalent condition that requires multidisciplinary treatment. However, in the United Kingdom, access to specialist pain clinics where patients can receive medical multidisciplinary treatment is limited, and provision varies between health boards. As such, self-management of chronic pain using digital tools has been gaining traction recently, but evidence of its effectiveness from clinical-based trials focuses mainly on quantitative outcomes. Objective: This systematic review aims to identify, appraise, and synthesize qualitative evidence on patients’ experiences with digital health interventions (DHIs) for the management of chronic pain. Methods: This systematic review will consider qualitative and mixed methods studies that explore the experience of patients (aged 18 years and older) with chronic pain engaging in DHIs to manage their pain. MEDLINE Ovid, PubMed, Embase, CINAHL, PsycINFO, and Scopus databases will be searched for published studies. The systematic review will be conducted in accordance with the ENTREQ (Enhancing Transparency in Reporting the Synthesis of Qualitative Research) guidelines. Following the 3-step thematic synthesis methodology of Thomas and Harden, titles and abstracts will be screened by 2 independent reviewers (AM and HM), and a third reviewer (MI or FM) will resolve any conflict that arises before the full-text screening. The Critical Appraisal Skills Programme checklist tool will be used to critically appraise the included studies. The extracted data will be imported to NVivo (QSR International), where thematic synthesis will be used to derive analytical themes from the included studies. Results: Themes that encapsulate the patient experience will be identified from qualitative evidence, and these themes will shed light on the perceived benefits and disadvantages, usability, acceptability, and the overall impact digital tools can have on the lives of those with chronic pain. Conclusions: This systematic review will identify, appraise, and synthesize the overall experience of patients engaging in DHI to manage a diverse range of chronic pain conditions. By elaborating the patient experience through qualitative analysis, the findings from this review will enhance our current understanding of the experiences of patients with chronic pain using digital tools for the self-management of their pain and highlight what person-centered elements are essential for future DHI development. Trial Registration: PROSPERO CRD42023445100; http://tinyurl.com/4z77khfs International Registered Report Identifier (IRRID): DERR1-10.2196/52469 %M 38451694 %R 10.2196/52469 %U https://www.researchprotocols.org/2024/1/e52469 %U https://doi.org/10.2196/52469 %U http://www.ncbi.nlm.nih.gov/pubmed/38451694 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e47744 %T Real-World Data Quality Framework for Oncology Time to Treatment Discontinuation Use Case: Implementation and Evaluation Study %A Ru,Boshu %A Sillah,Arthur %A Desai,Kaushal %A Chandwani,Sheenu %A Yao,Lixia %A Kothari,Smita %+ Center for Observational and Real-world Evidence (CORE), Merck & Co, Inc, 770 Sumneytown Pike, WP37A, West Point, PA, 19486, United States, 1 215 652 4301, boshu.ru@merck.com %K data quality assessment %K real-world data %K real-world time to treatment discontinuation %K systemic anticancer therapy %K Use Case Specific Relevance and Quality Assessment %K UReQA framework %D 2024 %7 6.3.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: The importance of real-world evidence is widely recognized in observational oncology studies. However, the lack of interoperable data quality standards in the fragmented health information technology landscape represents an important challenge. Therefore, adopting validated systematic methods for evaluating data quality is important for oncology outcomes research leveraging real-world data (RWD). Objective: This study aims to implement real-world time to treatment discontinuation (rwTTD) for a systemic anticancer therapy (SACT) as a new use case for the Use Case Specific Relevance and Quality Assessment, a framework linking data quality and relevance in fit-for-purpose RWD assessment. Methods: To define the rwTTD use case, we mapped the operational definition of rwTTD to RWD elements commonly available from oncology electronic health record–derived data sets. We identified 20 tasks to check the completeness and plausibility of data elements concerning SACT use, line of therapy (LOT), death date, and length of follow-up. Using descriptive statistics, we illustrated how to implement the Use Case Specific Relevance and Quality Assessment on 2 oncology databases (Data sets A and B) to estimate the rwTTD of an SACT drug (target SACT) for patients with advanced head and neck cancer diagnosed on or after January 1, 2015. Results: A total of 1200 (24.96%) of 4808 patients in Data set A and 237 (5.92%) of 4003 patients in Data set B received the target SACT, suggesting better relevance of the former in estimating the rwTTD of the target SACT. The 2 data sets differed with regard to the terminology used for SACT drugs, LOT format, and target SACT LOT distribution over time. Data set B appeared to have less complete SACT records, longer lags in incorporating the latest data, and incomplete mortality data, suggesting a lack of fitness for estimating rwTTD. Conclusions: The fit-for-purpose data quality assessment demonstrated substantial variability in the quality of the 2 real-world data sets. The data quality specifications applied for rwTTD estimation can be expanded to support a broad spectrum of oncology use cases. %M 38446504 %R 10.2196/47744 %U https://medinform.jmir.org/2024/1/e47744 %U https://doi.org/10.2196/47744 %U http://www.ncbi.nlm.nih.gov/pubmed/38446504 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e47081 %T Intention to Use an Electronic Community Health Information System Among Health Extension Workers in Rural Northwest Ethiopia: Cross-Sectional Study Using the Unified Theory of Acceptance and Use of Technology 2 Model %A Hailemariam,Tesfahun %A Atnafu,Asmamaw %A Gezie,Lemma %A Kaasbøll,Jens %A Klein,Jorn %A Tilahun,Binyam %+ Department of Health Informatics, Institute of Public Health, College of Medicine and Health Sciences, University of Gondar, Chechela street, Gondar, 196, Ethiopia, 251 913305250, tesfahunhailemariam@gmail.com %K data capturing %K data use %K eCHIS %K electronic community health information system %K health extension worker %K HEW %K intention to use %K service provision %K Unified Theory of Acceptance and Use of Technology 2 %K UTAUT2 model %D 2024 %7 4.3.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: IT has brought remarkable change in bridging the digital gap in resource-constrained regions and advancing the health care system worldwide. Community-based information systems and mobile apps have been extensively developed and deployed to quantify and support health services delivered by community health workers. The success and failure of a digital health information system depends on whether and how it is used. Ethiopia is scaling up its electronic community health information system (eCHIS) to support the work of health extension workers (HEWs). For successful implementation, more evidence was required about the factors that may affect the willingness of HEWs to use the eCHIS. Objective: This study aimed to assess HEWs’ intentions to use the eCHIS for health data management and service provision. Methods: A cross-sectional study design was conducted among 456 HEWs in 6 pilot districts of the Central Gondar zone, Northwest Ethiopia. A Unified Theory of Acceptance and Use of Technology model was used to investigate HEWs’ intention to use the eCHIS. Data were cleaned, entered into Epi-data (version 4.02; EpiData Association), and exported to SPSS (version 26; IBM Corp) for analysis using the AMOS 23 Structural Equation Model. The statistical significance of dependent and independent variables in the model was reported using a 95% CI with a corresponding P value of <.05. Results: A total of 456 HEWs participated in the study, with a response rate of 99%. The mean age of the study participants was 28 (SD 4.8) years. Our study revealed that about 179 (39.3%; 95% CI 34.7%-43.9%) participants intended to use the eCHIS for community health data generation, use, and service provision. Effort expectancy (β=0.256; P=.007), self-expectancy (β=0.096; P=.04), social influence (β=0.203; P=.02), and hedonic motivation (β=0.217; P=.03) were significantly associated with HEWs’ intention to use the eCHIS. Conclusions: HEWs need to be computer literate and understand their role with the eCHIS. Ensuring that the system is easy and enjoyable for them to use is important for implementation and effective health data management. %M 38437008 %R 10.2196/47081 %U https://humanfactors.jmir.org/2024/1/e47081 %U https://doi.org/10.2196/47081 %U http://www.ncbi.nlm.nih.gov/pubmed/38437008 %0 Journal Article %@ 2561-1011 %I JMIR Publications %V 8 %N %P e45130 %T Cloud-Based Machine Learning Platform to Predict Clinical Outcomes at Home for Patients With Cardiovascular Conditions Discharged From Hospital: Clinical Trial %A Yang,Phillip C %A Jha,Alokkumar %A Xu,William %A Song,Zitao %A Jamp,Patrick %A Teuteberg,Jeffrey J %+ Stanford University School of Medicine, 300 Pasteur Dr # H2157 Stanford, Palo Alto, CA, 94305-2200, United States, 1 6508048828, phillip@stanford.edu %K smart sensor %K wearable technology %K moving average %K physical activity %K artificial intelligence %K AI %D 2024 %7 1.3.2024 %9 Original Paper %J JMIR Cardio %G English %X Background: Hospitalizations account for almost one-third of the US $4.1 trillion health care cost in the United States. A substantial portion of these hospitalizations are attributed to readmissions, which led to the establishment of the Hospital Readmissions Reduction Program (HRRP) in 2012. The HRRP reduces payments to hospitals with excess readmissions. In 2018, >US $700 million was withheld; this is expected to exceed US $1 billion by 2022. More importantly, there is nothing more physically and emotionally taxing for readmitted patients and demoralizing for hospital physicians, nurses, and administrators. Given this high uncertainty of proper home recovery, intelligent monitoring is needed to predict the outcome of discharged patients to reduce readmissions. Physical activity (PA) is one of the major determinants for overall clinical outcomes in diabetes, hypertension, hyperlipidemia, heart failure, cancer, and mental health issues. These are the exact comorbidities that increase readmission rates, underlining the importance of PA in assessing the recovery of patients by quantitative measurement beyond the questionnaire and survey methods. Objective: This study aims to develop a remote, low-cost, and cloud-based machine learning (ML) platform to enable the precision health monitoring of PA, which may fundamentally alter the delivery of home health care. To validate this technology, we conducted a clinical trial to test the ability of our platform to predict clinical outcomes in discharged patients. Methods: Our platform consists of a wearable device, which includes an accelerometer and a Bluetooth sensor, and an iPhone connected to our cloud-based ML interface to analyze PA remotely and predict clinical outcomes. This system was deployed at a skilled nursing facility where we collected >17,000 person-day data points over 2 years, generating a solid training database. We used these data to train our extreme gradient boosting (XGBoost)–based ML environment to conduct a clinical trial, Activity Assessment of Patients Discharged from Hospital-I, to test the hypothesis that a comprehensive profile of PA would predict clinical outcome. We developed an advanced data-driven analytic platform that predicts the clinical outcome based on accurate measurements of PA. Artificial intelligence or an ML algorithm was used to analyze the data to predict short-term health outcome. Results: We enrolled 52 patients discharged from Stanford Hospital. Our data demonstrated a robust predictive system to forecast health outcome in the enrolled patients based on their PA data. We achieved precise prediction of the patients’ clinical outcomes with a sensitivity of 87%, a specificity of 79%, and an accuracy of 85%. Conclusions: To date, there are no reliable clinical data, using a wearable device, regarding monitoring discharged patients to predict their recovery. We conducted a clinical trial to assess outcome data rigorously to be used reliably for remote home care by patients, health care professionals, and caretakers. %M 38427393 %R 10.2196/45130 %U https://cardio.jmir.org/2024/1/e45130 %U https://doi.org/10.2196/45130 %U http://www.ncbi.nlm.nih.gov/pubmed/38427393 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e47846 %T Integration of Patient-Reported Outcome Data Collected Via Web Applications and Mobile Apps Into a Nation-Wide COVID-19 Research Platform Using Fast Healthcare Interoperability Resources: Development Study %A Oehm,Johannes Benedict %A Riepenhausen,Sarah Luise %A Storck,Michael %A Dugas,Martin %A Pryss,Rüdiger %A Varghese,Julian %+ Institute of Medical Informatics, University of Münster, Albert-Schweizer-Campus 1, Gebäude 11, Münster, 48149, Germany, 49 251 83 58247, johannes.oehm@uni-muenster.de %K Fast Healthcare Interoperability Resources %K FHIR %K FHIR Questionnaire %K patient-reported outcome %K mobile health %K mHealth %K research compatibility %K interoperability %K Germany %K harmonized data collection %K findable, accessible, interoperable, and reusable %K FAIR data %K mobile phone %D 2024 %7 27.2.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: The Network University Medicine projects are an important part of the German COVID-19 research infrastructure. They comprise 2 subprojects: COVID-19 Data Exchange (CODEX) and Coordination on Mobile Pandemic Apps Best Practice and Solution Sharing (COMPASS). CODEX provides a centralized and secure data storage platform for research data, whereas in COMPASS, expert panels were gathered to develop a reference app framework for capturing patient-reported outcomes (PROs) that can be used by any researcher. Objective: Our study aims to integrate the data collected with the COMPASS reference app framework into the central CODEX platform, so that they can be used by secondary researchers. Although both projects used the Fast Healthcare Interoperability Resources (FHIR) standard, it was not used in a way that data could be shared directly. Given the short time frame and the parallel developments within the CODEX platform, a pragmatic and robust solution for an interface component was required. Methods: We have developed a means to facilitate and promote the use of the German Corona Consensus (GECCO) data set, a core data set for COVID-19 research in Germany. In this way, we ensured semantic interoperability for the app-collected PRO data with the COMPASS app. We also developed an interface component to sustain syntactic interoperability. Results: The use of different FHIR types by the COMPASS reference app framework (the general-purpose FHIR Questionnaire) and the CODEX platform (eg, Patient, Condition, and Observation) was found to be the most significant obstacle. Therefore, we developed an interface component that realigns the Questionnaire items with the corresponding items in the GECCO data set and provides the correct resources for the CODEX platform. We extended the existing COMPASS questionnaire editor with an import function for GECCO items, which also tags them for the interface component. This ensures syntactic interoperability and eases the reuse of the GECCO data set for researchers. Conclusions: This paper shows how PRO data, which are collected across various studies conducted by different researchers, can be captured in a research-compatible way. This means that the data can be shared with a central research infrastructure and be reused by other researchers to gain more insights about COVID-19 and its sequelae. %M 38411999 %R 10.2196/47846 %U https://www.jmir.org/2024/1/e47846 %U https://doi.org/10.2196/47846 %U http://www.ncbi.nlm.nih.gov/pubmed/38411999 %0 Journal Article %@ 2561-9128 %I JMIR Publications %V 7 %N %P e45126 %T Comparing Anesthesia and Surgery Controlled Time for Primary Total Knee and Hip Arthroplasty Between an Academic Medical Center and a Community Hospital: Retrospective Cohort Study %A Nguyen,Thy B %A Weitzel,Nathaen %A Hogan,Craig %A Kacmar,Rachel M %A Williamson,Kayla M %A Pattee,Jack %A Jevtovic-Todorovic,Vesna %A Simmons,Colby G %A Faruki,Adeel Ahmad %+ Department of Anesthesiology and Perioperative Medicine, MD Anderson Cancer Center, Faculty Center, Floor 13, FC13.2000, 1400 Holcombe Blvd, Unit 409, Houston, TX, United States, 1 713 792 6911, aafaruki@mdanderson.org %K anesthesia controlled time %K surgery-controlled time %K total joint arthroplasty %K healthcare operations %K efficiency %K total joint replacement %K knee %K hip %K arthroplasty %K anesthesia %K surgery %K surgical duration %K community hospital %K surgeon %K reliability %K operating room %K anesthesiology %K orthopedics %K perioperative %K medicine %D 2024 %7 26.2.2024 %9 Original Paper %J JMIR Perioper Med %G English %X Background: Osteoarthritis is a significant cause of disability, resulting in increased joint replacement surgeries and health care costs. Establishing benchmarks that more accurately predict surgical duration could help to decrease costs, maximize efficiency, and improve patient experience. We compared the anesthesia-controlled time (ACT) and surgery-controlled time (SCT) of primary total knee (TKA) and total hip arthroplasties (THA) between an academic medical center (AMC) and a community hospital (CH) for 2 orthopedic surgeons. Objective: This study aims to validate and compare benchmarking times for ACT and SCT in a single patient population at both an AMC and a CH. Methods: This retrospective 2-center observational cohort study was conducted at the University of Colorado Hospital (AMC) and UCHealth Broomfield Hospital (CH). Cases with current procedural terminology codes for THA and TKA between January 1, 2019, and December 31, 2020, were assessed. Cases with missing data were excluded. The primary outcomes were ACT and SCT. Primary outcomes were tested for association with covariates of interest. The primary covariate of interest was the location of the procedure (CH vs AMC); secondary covariates of interest included the American Society of Anesthesiologists (ASA) classification and anesthetic type. Linear regression models were used to assess the relationships. Results: Two surgeons performed 1256 cases at the AMC and CH. A total of 10 THA cases and 12 TKA cases were excluded due to missing data. After controlling for surgeon, the ACT was greater at the AMC for THA by 3.77 minutes and for TKA by 3.58 minutes (P<.001). SCT was greater at the AMC for THA by 11.14 minutes and for TKA by 14.04 minutes (P<.001). ASA III/IV classification increased ACT for THA by 3.76 minutes (P<.001) and increased SCT for THA by 6.33 minutes after controlling for surgeon and location (P=.008). General anesthesia use was higher at the AMC for both THA (29.2% vs 7.3%) and TKA (23.8% vs 4.2%). No statistically significant association was observed between either ACT or SCT and anesthetic type (neuraxial or general) after adjusting for surgeon and location (all P>.05). Conclusions: We observed lower ACT and SCT at the CH for both TKA and THA after controlling for the surgeon of record and ASA classification. These findings underscore the efficiency advantages of performing primary joint replacements at the CH, showcasing an average reduction of 16 minutes in SCT and 4 minutes in ACT per case. Overall, establishing more accurate benchmarks to improve the prediction of surgical duration for THA and TKA in different perioperative environments can increase the reliability of surgical duration predictions and optimize scheduling. Future studies with study populations at multiple community hospitals and academic medical centers are needed before extrapolating these findings. %M 38407957 %R 10.2196/45126 %U https://periop.jmir.org/2024/1/e45126 %U https://doi.org/10.2196/45126 %U http://www.ncbi.nlm.nih.gov/pubmed/38407957 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 10 %N %P e44062 %T The Use of ICD-9-CM Coding to Identify COVID-19 Diagnoses and Determine Risk Factors for 30-Day Death Rate in Hospitalized Patients in Italy: Retrospective Study %A Giordani,Barbara %A Burgio,Alessandra %A Grippo,Francesco %A Barone,Alessandra %A Eugeni,Erica %A Baglio,Giovanni %+ Research, National Outcomes Evaluation Programme (PNE) and International Relations Unit, Italian National Agency for Regional Healthcare Services, Via Piemonte 60, Rome, 00187, Italy, 39 06 42749713, giordani@agenas.it %K COVID-19 %K ICD-9-CM coding %K hospitalizations %K SARS-CoV-2 %K coronavirus %K risk factor %K Italy %K death rate %K monitoring %K hospital records %K coding %K algorithm %D 2024 %7 23.2.2024 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: In Italy, it has been difficult to accurately quantify hospital admissions of patients with a COVID-19 diagnosis using the Hospital Information System (HIS), mainly due to the heterogeneity of codes used in the hospital discharge records during different waves of the COVID-19 pandemic. Objective: The objective of this study was to define a specific combination of codes to identify the COVID-19 hospitalizations within the HIS and to investigate the risk factors associated with mortality due to COVID-19 among patients admitted to Italian hospitals in 2020. Methods: A retrospective study was conducted using the hospital discharge records, provided by more than 1300 public and private Italian hospitals. Inpatient hospitalizations were detected by implementing an algorithm based on specific International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) code combinations. Hospitalizations were analyzed by different clinical presentations associated with COVID-19 diagnoses. In addition, 2 multivariable Cox regression models were performed among patients hospitalized “due to COVID-19” from January 1 to December 31, 2020, to investigate potential risk factors associated with 30-day death and the temporal changes over the course of the pandemic; in particular, the 30-day death rates during the first and the second waves were analyzed across 3 main geographical areas (North, Center, and South and Islands) and by discharge wards (ordinary and intensive care). Results: We identified a total of 325,810 hospitalizations with COVID-19–related diagnosis codes. Among these, 73.4% (n=239,114) were classified as “due to COVID-19,” 14.5% (n=47,416) as “SARS-CoV-2 positive, but not due to COVID-19,” and 12.1% (n=39,280) as “suspected COVID-19” hospitalizations. The cohort of patients hospitalized “due to COVID-19” included 205,048 patients, with a median age of 72 years and a higher prevalence of male patients (n=124,181, 60.6%). The overall 30-day death rate among hospitalized patients due to COVID-19 was 9.9 per 1000 person-days. Mortality was lower for women (hazard ratio [HR]=0.83; P<.001) and for patients coming from high migration pressure countries, especially Northern Africans (HR=0.65; P<.001) and Central and Eastern Europeans (HR=0.66; P<.001), compared to patients coming from Italy and high-income countries. In the southern regions and the Islands, mortality was higher compared to the northern regions (HR=1.17; P<.001), especially during the second wave of COVID-19 among patients with a transfer to intensive care units (HR=2.52; P<.001). Conclusions: To our knowledge, the algorithm is the first attempt to define, at a national level, selection criteria for identifying COVID-19 hospitalizations within the HIS. The implemented algorithm will be used to monitor the pandemic over time, and the patients selected in 2020 will be followed up in the next years to assess the long-term effects of COVID-19. %M 38393763 %R 10.2196/44062 %U https://publichealth.jmir.org/2024/1/e44062 %U https://doi.org/10.2196/44062 %U http://www.ncbi.nlm.nih.gov/pubmed/38393763 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e51153 %T Ethical and Quality of Care-Related Challenges of Digital Health Twins in Older Care Settings: Protocol for a Scoping Review %A Jabin,Md Shafiqur Rahman %A Yaroson,Emillia Vann %A Ilodibe,Adaobi %A Eldabi,Tillal %+ Faculty of Health Studies, University of Bradford, Horton A, Room: 2.10, Richmond Road, Bradford, BD7 1DP, United Kingdom, 44 7915673612, mjabin@bradford.ac.uk %K accessibility %K data security %K effectiveness %K equality %K health equity %K patient safety %K right to privacy %K social care %D 2024 %7 23.2.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Digital health twins (DHTs) have been evolving with their diverse applications in medicine, specifically in older care settings, with the increasing demands of older adults. DHTs have already contributed to improving the quality of dementia and trauma care, cardiac treatment, and health care services for older individuals. Despite its many benefits, the optimum implementation of DHTs has faced several challenges associated with ethical issues, quality of care, management and leadership, and design considerations in older care settings. Since the need for such care is continuously rising and there is evident potential for DHTs to meet those needs, this review aims to map key concepts to address the gaps in the research knowledge to improve DHT implementation. Objective: The review aims to compile and synthesize the best available evidence regarding the problems encountered by older adults and care providers associated with the application of DHTs. The synthesis will collate the evidence of the issues associated with quality of care, the ethical implications of DHTs, and the strategies undertaken to overcome those challenges in older care settings. Methods: The review will follow the Joanna Briggs Institute (JBI) methodology. The published studies will be searched through CINAHL, MEDLINE, JBI, and Web of Science, and the unpublished studies through Mednar, Trove, OCLC WorldCat, and Dissertations and Theses. Studies published in English from 2002 will be considered. This review will include studies of older individuals (aged 65 years or older) undergoing care delivery associated with DHTs and their respective care providers. The concept will include the application of the technology, and the context will involve studies based on the older care setting. A broad scope of evidence, including quantitative, qualitative, text and opinion studies, will be considered. A total of 2 independent reviewers will screen the titles and abstracts and then review the full text. Data will be extracted from the included studies using a data extraction tool developed for this study. Results: The results will be presented in a PRISMA-ScR (Preferred Reporting Items for Systematic Review and Meta-Analysis extension for Scoping Reviews) flow diagram. A draft charting table will be developed as a data extraction tool. The results will be presented as a “map” of the data in a logical, diagrammatic, or tabular form in a descriptive format. Conclusions: The evidence synthesis is expected to uncover the shreds of evidence required to address the ethical and care quality-related challenges associated with applying DHTs. A synthesis of various strategies used to overcome identified challenges will provide more prospects for adopting them elsewhere and create a resource allocation model for older individuals. International Registered Report Identifier (IRRID): DERR1-10.2196/51153 %M 38393771 %R 10.2196/51153 %U https://www.researchprotocols.org/2024/1/e51153 %U https://doi.org/10.2196/51153 %U http://www.ncbi.nlm.nih.gov/pubmed/38393771 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e51002 %T Enhancing the Efficiency of a Radiation Oncology Department Using Electronic Medical Records: Protocol for Preparing Radiotherapy %A Cheng,Hao-Shen %A You,Weir-Chiang %A Chen,Ni-Wei %A Hsieh,Mu-Chih %A Tsai,Che-Fu %A Ho,Chia-Jing %A Chen,Chien-Chih %+ Taichung Veterans General Hospital, 1650 Taiwan Boulevard Sect. 4, Taichung, 40705, Taiwan, 886 4 2359 2525 ext 5601, chiencheh@gmail.com %K efficiency %K electronic medical records %K Hospital Information System %K protocol %K radiation oncology %D 2024 %7 23.2.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Electronic medical records (EMRs) streamline medical processes, improve quality control, and facilitate data sharing among hospital departments. They also reduce maintenance costs and storage space needed for paper records, while saving time and providing structured data for future research. Objective: This study aimed to investigate whether the integration of the radiation oncology information system and the hospital information system enhances the efficiency of the department of radiation oncology. Methods: We held multidisciplinary discussions among physicians, physicists, medical radiation technologists, nurses, and engineers. We integrated paper records from the radiation oncology department into the existing hospital information system within the hospital. A new electronic interface was designed. A comparison was made between the time taken to retrieve information from either the paper records or the EMRs for radiation preparation. A total of 30 cases were randomly allocated in both the old paper-based system and the new EMR system. The time spent was calculated manually at every step during the process, and we performed an independent 1-tailed t test to evaluate the difference between the 2 systems. Results: Since the system was launched in August 2020, more than 1000 medical records have been entered into the system, and this figure continues to increase. The total time needed for the radiation preparation process was reduced from 286.8 minutes to 154.3 minutes (P<.001)—a reduction of 46.2%. There was no longer any need to arrange for a nurse to organize the radiotherapy paper records, saving a workload of 16 hours per month. Conclusions: The implementation of the integrated EMR system has resulted in a significant reduction in the number of steps involved in radiotherapy preparation, as well as a decrease in the amount of time required for the process. The new EMR system has provided numerous benefits for the department, including a decrease in workload, a simplified workflow, and conserving more patient data within a confined space. %M 38393753 %R 10.2196/51002 %U https://www.researchprotocols.org/2024/1/e51002 %U https://doi.org/10.2196/51002 %U http://www.ncbi.nlm.nih.gov/pubmed/38393753 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e50146 %T A Virtual Hospital Model of Care for Low Back Pain, Back@Home: Protocol for a Hybrid Effectiveness-Implementation Type-I Study %A Melman,Alla %A Teng,Min Jiat %A Coombs,Danielle M %A Li,Qiang %A Billot,Laurent %A Lung,Thomas %A Rogan,Eileen %A Marabani,Mona %A Hutchings,Owen %A Maher,Chris G %A Machado,Gustavo C %A , %+ Sydney Musculoskeletal Health, The University of Sydney and Sydney Local Health District, Level 10N, King George V Building, Royal Prince Alfred Hospital, Missenden Road, Camperdown, 2050, Australia, 61 286276263, alla.melman@sydney.edu.au %K length of stay %K back pain %K musculoskeletal pain %K telemedicine %K hospital-based home care %K mobile phone %K home care %K virtual care %K remote care %K virtual hospital %K pain %K telehealth %K eHealth %K musculoskeletal %K implementation %K model of care %K back %K cost %K economic %K readmission %K hospital stay %D 2024 %7 22.2.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Low back pain (LBP) was the fifth most common reason for an emergency department (ED) visit in 2020-2021 in Australia, with >145,000 presentations. A total of one-third of these patients were subsequently admitted to the hospital. The admitted patient care accounts for half of the total health care expenditure on LBP in Australia. Objective: The primary aim of the Back@Home study is to assess the effectiveness of a virtual hospital model of care to reduce the length of admission in people presenting to ED with musculoskeletal LBP. A secondary aim is to evaluate the acceptability and feasibility of the virtual hospital and our implementation strategy. We will also investigate rates of traditional hospital admission from the ED, representations and readmissions to the traditional hospital, demonstrate noninferiority of patient-reported outcomes, and assess cost-effectiveness of the new model. Methods: This is a hybrid effectiveness-implementation type-I study. To evaluate effectiveness, we plan to conduct an interrupted time-series study at 3 metropolitan hospitals in Sydney, New South Wales, Australia. Eligible patients will include those aged 16 years or older with a primary diagnosis of musculoskeletal LBP presenting to the ED. The implementation strategy includes clinician education using multimedia resources, staff champions, and an “audit and feedback” process. The implementation of “Back@Home” will be evaluated over 12 months and compared to a 48-month preimplementation period using monthly time-series trends in the average length of hospital stay as the primary outcome. We will construct a plot of the observed and expected lines of trend based on the preimplementation period. Linear segmented regression will identify changes in the level and slope of fitted lines, indicating immediate effects of the intervention, as well as effects over time. The data will be fully anonymized, with informed consent collected for patient-reported outcomes. Results: As of December 6, 2023, a total of 108 patients have been cared for through Back@Home. A total of 6 patients have completed semistructured interviews regarding their experience of virtual hospital care for nonserious back pain. All outcomes will be evaluated at 6 months (August 2023) and 12 months post implementation (February 2024). Conclusions: This study will serve to inform ongoing care delivery and implementation strategies of a novel model of care. If found to be effective, it may be adopted by other health districts, adapting the model to their unique local contexts. International Registered Report Identifier (IRRID): PRR1-10.2196/50146 %M 38386370 %R 10.2196/50146 %U https://www.researchprotocols.org/2024/1/e50146 %U https://doi.org/10.2196/50146 %U http://www.ncbi.nlm.nih.gov/pubmed/38386370 %0 Journal Article %@ 2369-3762 %I JMIR Publications %V 10 %N %P e48507 %T Occupational Therapy Students’ Evidence-Based Practice Skills as Reported in a Mobile App: Cross-Sectional Study %A Johnson,Susanne G %A Espehaug,Birgitte %A Larun,Lillebeth %A Ciliska,Donna %A Olsen,Nina Rydland %+ Department of Health and Functioning, Western Norway University of Applied Sciences, Inndalseveien 28, Bergen, 5063, Norway, 47 92213202, susanne.grodem.johnson@hvl.no %K active learning strategies %K application %K cross-sectional study %K development %K education %K higher education %K interactive %K mobile application %K mobile app %K occupational therapy students %K occupational therapy %K students %K usability %K use %D 2024 %7 21.2.2024 %9 Original Paper %J JMIR Med Educ %G English %X Background: Evidence-based practice (EBP) is an important aspect of the health care education curriculum. EBP involves following the 5 EBP steps: ask, assess, appraise, apply, and audit. These 5 steps reflect the suggested core competencies covered in teaching and learning programs to support future health care professionals applying EBP. When implementing EBP teaching, assessing outcomes by documenting the student’s performance and skills is relevant. This can be done using mobile devices. Objective: The aim of this study was to assess occupational therapy students’ EBP skills as reported in a mobile app. Methods: We applied a cross-sectional design. Descriptive statistics were used to present frequencies, percentages, means, and ranges of data regarding EBP skills found in the EBPsteps app. Associations between students’ ability to formulate the Population, Intervention, Comparison, and Outcome/Population, Interest, and Context (PICO/PICo) elements and identifying relevant research evidence were analyzed with the chi-square test. Results: Of 4 cohorts with 150 students, 119 (79.3%) students used the app and produced 240 critically appraised topics (CATs) in the app. The EBP steps “ask,” “assess,” and “appraise” were often correctly performed. The clinical question was formulated correctly in 53.3% (128/240) of the CATs, and students identified research evidence in 81.2% (195/240) of the CATs. Critical appraisal checklists were used in 81.2% (195/240) of the CATs, and most of these checklists were assessed as relevant for the type of research evidence identified (165/195, 84.6%). The least frequently correctly reported steps were “apply” and “audit.” In 39.6% (95/240) of the CATs, it was reported that research evidence was applied. Only 61% (58/95) of these CATs described how the research was applied to clinical practice. Evaluation of practice changes was reported in 38.8% (93/240) of the CATs. However, details about practice changes were lacking in all these CATs. A positive association was found between correctly reporting the "population" and "interventions/interest" elements of the PICO/PICo and identifying research evidence (P<.001). Conclusions: We assessed the students’ EBP skills based on how they documented following the EBP steps in the EBPsteps app, and our results showed variations in how well the students mastered the steps. “Apply” and “audit” were the most difficult EBP steps for the students to perform, and this finding has implications and gives directions for further development of the app and educational instruction in EBP. The EBPsteps app is a new and relevant app for students to learn and practice EBP, and it can be used to assess students’ EBP skills objectively. %M 38381475 %R 10.2196/48507 %U https://mededu.jmir.org/2024/1/e48507 %U https://doi.org/10.2196/48507 %U http://www.ncbi.nlm.nih.gov/pubmed/38381475 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e51727 %T Enhancing Health Care Accessibility and Equity Through a Geoprocessing Toolbox for Spatial Accessibility Analysis: Development and Case Study %A Hashtarkhani,Soheil %A Schwartz,David L %A Shaban-Nejad,Arash %+ Center for Biomedical Informatics, Department of Pediatrics, College of Medicine, The University of Tennessee Health Science Center, 50 N Dunlap Street, R492, Memphis, TN, 38103, United States, 1 9012875863, ashabann@uthsc.edu %K geographical information system %K geoprocessing tool %K health disparities %K health equity %K health services management %K hemodialysis services %K spatial accessibility %D 2024 %7 21.2.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Access to health care services is a critical determinant of population health and well-being. Measuring spatial accessibility to health services is essential for understanding health care distribution and addressing potential inequities. Objective: In this study, we developed a geoprocessing toolbox including Python script tools for the ArcGIS Pro environment to measure the spatial accessibility of health services using both classic and enhanced versions of the 2-step floating catchment area method. Methods: Each of our tools incorporated both distance buffers and travel time catchments to calculate accessibility scores based on users’ choices. Additionally, we developed a separate tool to create travel time catchments that is compatible with both locally available network data sets and ArcGIS Online data sources. We conducted a case study focusing on the accessibility of hemodialysis services in the state of Tennessee using the 4 versions of the accessibility tools. Notably, the calculation of the target population considered age as a significant nonspatial factor influencing hemodialysis service accessibility. Weighted populations were calculated using end-stage renal disease incidence rates in different age groups. Results: The implemented tools are made accessible through ArcGIS Online for free use by the research community. The case study revealed disparities in the accessibility of hemodialysis services, with urban areas demonstrating higher scores compared to rural and suburban regions. Conclusions: These geoprocessing tools can serve as valuable decision-support resources for health care providers, organizations, and policy makers to improve equitable access to health care services. This comprehensive approach to measuring spatial accessibility can empower health care stakeholders to address health care distribution challenges effectively. %M 38381503 %R 10.2196/51727 %U https://formative.jmir.org/2024/1/e51727 %U https://doi.org/10.2196/51727 %U http://www.ncbi.nlm.nih.gov/pubmed/38381503 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e54681 %T Exploring Shared Implementation Leadership of Point of Care Nursing Leadership Teams on Inpatient Hospital Units: Protocol for a Collective Case Study %A Castiglione,Sonia Angela %A Lavoie-Tremblay,Mélanie %A Kilpatrick,Kelley %A Gifford,Wendy %A Semenic,Sonia Elizabeth %+ Ingram School of Nursing, McGill University, #1800, 680 Rue Sherbrooke O, Montreal, QC, H3A 2M7, Canada, 1 514 398 4144, sonia.castiglione@mcgill.ca %K case study %K evidence-based practices %K implementation leadership %K inpatient hospital units %K nursing leadership %K point of care %D 2024 %7 19.2.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Nursing leadership teams at the point of care (POC), consisting of both formal and informal leaders, are regularly called upon to support the implementation of evidence-based practices (EBPs) in hospital units. However, current conceptualizations of effective leadership for successful implementation typically focus on the behaviors of individual leaders in managerial roles. Little is known about how multiple nursing leaders in formal and informal roles share implementation leadership (IL), representing an important knowledge gap. Objective: This study aims to explore shared IL among formal and informal nursing leaders in inpatient hospital units. The central research question is as follows: How is IL shared among members of POC nursing leadership teams on inpatient hospital units? The subquestions are as follows: (1) What IL behaviors are enacted and shared by formal and informal leaders? (2) What social processes enable shared IL by formal and informal leaders? and (3) What factors influence shared IL in nursing leadership teams? Methods: We will use a collective case study approach to describe and generate an in-depth understanding of shared IL in nursing. We will select nursing leadership teams on 2 inpatient hospital units that have successfully implemented an EBP as instrumental cases. We will construct data through focus groups and individual interviews with key informants (leaders, unit staff, and senior nurse leaders), review of organizational documents, and researcher-generated field notes. We have developed a conceptual framework of shared IL to guide data analysis, which describes effective IL behaviors, formal and informal nursing leaders’ roles at the POC, and social processes generating shared leadership and influencing contextual factors. We will use the Framework Method to systematically generate data matrices from deductive and inductive thematic analysis of each case. We will then generate assertions about shared IL following a cross-case analysis. Results: The study protocol received research ethics approval (2022-8408) on February 24, 2022. Data collection began in June 2022, and we have recruited 2 inpatient hospital units and 25 participants. Data collection was completed in December 2023, and data analysis is ongoing. We anticipate findings to be published in a peer-reviewed journal by late 2024. Conclusions: The anticipated results will shed light on how multiple and diverse members of the POC nursing leadership team enact and share IL. This study addresses calls to advance knowledge in promoting effective implementation of EBPs to ensure high-quality health care delivery by further developing the concept of shared IL in a nursing context. We will identify strategies to strengthen shared IL in nursing leadership teams at the POC, informing future intervention studies. International Registered Report Identifier (IRRID): DERR1-10.2196/54681 %M 38373024 %R 10.2196/54681 %U https://www.researchprotocols.org/2024/1/e54681 %U https://doi.org/10.2196/54681 %U http://www.ncbi.nlm.nih.gov/pubmed/38373024 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e53654 %T Development of Cost-Effective Fatty Liver Disease Prediction Models in a Chinese Population: Statistical and Machine Learning Approaches %A Zhang,Liang %A Huang,Yueqing %A Huang,Min %A Zhao,Chun-Hua %A Zhang,Yan-Jun %A Wang,Yi %+ Department of General Practice, The Affiliated Suzhou Hospital of Nanjing Medical University, 16 Baitaxi Road, Gusu District, Suzhou, 215000, China, 86 13812757566, huangyq_sz@163.com %K NAFLD %K artificial intelligence %K public health %K transient elastography %K diagnosis %D 2024 %7 16.2.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: The increasing prevalence of nonalcoholic fatty liver disease (NAFLD) in China presents a significant public health concern. Traditional ultrasound, commonly used for fatty liver screening, often lacks the ability to accurately quantify steatosis, leading to insufficient follow-up for patients with moderate-to-severe steatosis. Transient elastography (TE) provides a more quantitative diagnosis of steatosis and fibrosis, closely aligning with biopsy results. Moreover, machine learning (ML) technology holds promise for developing more precise diagnostic models for NAFLD using a variety of laboratory indicators. Objective: This study aims to develop a novel ML-based diagnostic model leveraging TE results for staging hepatic steatosis. The objective was to streamline the model’s input features, creating a cost-effective and user-friendly tool to distinguish patients with NAFLD requiring follow-up. This innovative approach merges TE and ML to enhance diagnostic accuracy and efficiency in NAFLD assessment. Methods: The study involved a comprehensive analysis of health examination records from Suzhou Municipal Hospital, spanning from March to May 2023. Patient data and questionnaire responses were meticulously inputted into Microsoft Excel 2019, followed by thorough data cleaning and model development using Python 3.7, with libraries scikit-learn and numpy to ensure data accuracy. A cohort comprising 978 residents with complete medical records and TE results was included for analysis. Various classification models, including logistic regression (LR), k-nearest neighbor (KNN), support vector machine (SVM), random forest (RF), light gradient boosting machine (LightGBM), and extreme gradient boosting (XGBoost), were constructed and evaluated based on the area under the receiver operating characteristic curve (AUROC). Results: Among the 916 patients included in the study, 273 were diagnosed with moderate-to-severe NAFLD. The concordance rate between traditional ultrasound and TE for detecting moderate-to-severe NAFLD was 84.6% (231/273). The AUROC values for the RF, LightGBM, XGBoost, SVM, KNN, and LR models were 0.91, 0.86, 0.83, 0.88, 0.77, and 0.81, respectively. These models achieved accuracy rates of 84%, 81%, 78%, 81%, 76%, and 77%, respectively. Notably, the RF model exhibited the best performance. A simplified RF model was developed with an AUROC of 0.88, featuring 62% sensitivity and 90% specificity. This simplified model used 6 key features: waist circumference, BMI, fasting plasma glucose, uric acid, total bilirubin, and high-sensitivity C-reactive protein. This approach offers a cost-effective and user-friendly tool while streamlining feature acquisition for training purposes. Conclusions: The study introduces a groundbreaking, cost-effective ML algorithm that leverages health examination data for identifying moderate-to-severe NAFLD. This model has the potential to significantly impact public health by enabling targeted investigations and interventions for NAFLD. By integrating TE and ML technologies, the study showcases innovative approaches to advancing NAFLD diagnostics. %M 38363597 %R 10.2196/53654 %U https://formative.jmir.org/2024/1/e53654 %U https://doi.org/10.2196/53654 %U http://www.ncbi.nlm.nih.gov/pubmed/38363597 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e52967 %T Use of Metadata-Driven Approaches for Data Harmonization in the Medical Domain: Scoping Review %A Peng,Yuan %A Bathelt,Franziska %A Gebler,Richard %A Gött,Robert %A Heidenreich,Andreas %A Henke,Elisa %A Kadioglu,Dennis %A Lorenz,Stephan %A Vengadeswaran,Abishaa %A Sedlmayr,Martin %+ Institute for Medical Informatics and Biometry, Carl Gustav Carus Faculty of Medicine, Technische Universität Dresden, Fetscherstraße 74, Dresden, 01307, Germany, 49 3514583648, yuan.peng@tu-dresden.de %K ETL %K ELT %K Extract-Load-Transform %K Extract-Transform-Load %K interoperability %K metadata-driven %K medical domain %K data harmonization %D 2024 %7 14.2.2024 %9 Review %J JMIR Med Inform %G English %X Background: Multisite clinical studies are increasingly using real-world data to gain real-world evidence. However, due to the heterogeneity of source data, it is difficult to analyze such data in a unified way across clinics. Therefore, the implementation of Extract-Transform-Load (ETL) or Extract-Load-Transform (ELT) processes for harmonizing local health data is necessary, in order to guarantee the data quality for research. However, the development of such processes is time-consuming and unsustainable. A promising way to ease this is the generalization of ETL/ELT processes. Objective: In this work, we investigate existing possibilities for the development of generic ETL/ELT processes. Particularly, we focus on approaches with low development complexity by using descriptive metadata and structural metadata. Methods: We conducted a literature review following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. We used 4 publication databases (ie, PubMed, IEEE Explore, Web of Science, and Biomed Center) to search for relevant publications from 2012 to 2022. The PRISMA flow was then visualized using an R-based tool (Evidence Synthesis Hackathon). All relevant contents of the publications were extracted into a spreadsheet for further analysis and visualization. Results: Regarding the PRISMA guidelines, we included 33 publications in this literature review. All included publications were categorized into 7 different focus groups (ie, medicine, data warehouse, big data, industry, geoinformatics, archaeology, and military). Based on the extracted data, ontology-based and rule-based approaches were the 2 most used approaches in different thematic categories. Different approaches and tools were chosen to achieve different purposes within the use cases. Conclusions: Our literature review shows that using metadata-driven (MDD) approaches to develop an ETL/ELT process can serve different purposes in different thematic categories. The results show that it is promising to implement an ETL/ELT process by applying MDD approach to automate the data transformation from Fast Healthcare Interoperability Resources to Observational Medical Outcomes Partnership Common Data Model. However, the determining of an appropriate MDD approach and tool to implement such an ETL/ELT process remains a challenge. This is due to the lack of comprehensive insight into the characterizations of the MDD approaches presented in this study. Therefore, our next step is to evaluate the MDD approaches presented in this study and to determine the most appropriate MDD approaches and the way to integrate them into the ETL/ELT process. This could verify the ability of using MDD approaches to generalize the ETL process for harmonizing medical data. %M 38354027 %R 10.2196/52967 %U https://medinform.jmir.org/2024/1/e52967 %U https://doi.org/10.2196/52967 %U http://www.ncbi.nlm.nih.gov/pubmed/38354027 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e42271 %T Improving Prediction of Survival for Extremely Premature Infants Born at 23 to 29 Weeks Gestational Age in the Neonatal Intensive Care Unit: Development and Evaluation of Machine Learning Models %A Li,Angie %A Mullin,Sarah %A Elkin,Peter L %+ Department of Biomedical Informatics, Jacobs School of Medicine and Biomedical Sciences, University at Buffalo, 77 Goodell Street, Suite 540, Buffalo, NY, 14203, United States, 1 716 888 4858, ali83@buffalo.edu %K reproductive informatics %K pregnancy complications %K premature birth %K neonatal mortality %K machine learning %K clinical decision support %K preterm %K pediatrics %K intensive care unit outcome %K health care outcome %K survival prediction %K maternal health %K decision tree model %K socioeconomic %D 2024 %7 14.2.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Infants born at extremely preterm gestational ages are typically admitted to the neonatal intensive care unit (NICU) after initial resuscitation. The subsequent hospital course can be highly variable, and despite counseling aided by available risk calculators, there are significant challenges with shared decision-making regarding life support and transition to end-of-life care. Improving predictive models can help providers and families navigate these unique challenges. Objective: Machine learning methods have previously demonstrated added predictive value for determining intensive care unit outcomes, and their use allows consideration of a greater number of factors that potentially influence newborn outcomes, such as maternal characteristics. Machine learning–based models were analyzed for their ability to predict the survival of extremely preterm neonates at initial admission. Methods: Maternal and newborn information was extracted from the health records of infants born between 23 and 29 weeks of gestation in the Medical Information Mart for Intensive Care III (MIMIC-III) critical care database. Applicable machine learning models predicting survival during the initial NICU admission were developed and compared. The same type of model was also examined using only features that would be available prepartum for the purpose of survival prediction prior to an anticipated preterm birth. Features most correlated with the predicted outcome were determined when possible for each model. Results: Of included patients, 37 of 459 (8.1%) expired. The resulting random forest model showed higher predictive performance than the frequently used Score for Neonatal Acute Physiology With Perinatal Extension II (SNAPPE-II) NICU model when considering extremely preterm infants of very low birth weight. Several other machine learning models were found to have good performance but did not show a statistically significant difference from previously available models in this study. Feature importance varied by model, and those of greater importance included gestational age; birth weight; initial oxygenation level; elements of the APGAR (appearance, pulse, grimace, activity, and respiration) score; and amount of blood pressure support. Important prepartum features also included maternal age, steroid administration, and the presence of pregnancy complications. Conclusions: Machine learning methods have the potential to provide robust prediction of survival in the context of extremely preterm births and allow for consideration of additional factors such as maternal clinical and socioeconomic information. Evaluation of larger, more diverse data sets may provide additional clarity on comparative performance. %M 38354033 %R 10.2196/42271 %U https://medinform.jmir.org/2024/1/e42271 %U https://doi.org/10.2196/42271 %U http://www.ncbi.nlm.nih.gov/pubmed/38354033 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e47441 %T A Digital Respiratory Ward in Leicester, Leicestershire, and Rutland, England, for Patients With COVID-19: Economic Evaluation of the Impact on Acute Capacity and Wider National Health Service Resource Use %A Swift,Jim %A O'Kelly,Noel %A Barker,Chris %A Woodward,Alex %A Ghosh,Sudip %+ Spirit Health, Spirit House, Saffron Way, Leicester, LE2 6UP, United Kingdom, 44 1162865000, jim.swift@spirit-health.com %K Covid-19 %K telemedicine %K digital technology %K home transition %K length of stay %K cost-effectiveness analysis %K cost %K costs %K economic %K economics %K telehealth %K hospitalization %K hospital %K hospitals %K hospitalizations %K resource %K resources %K hospital stay %K ward %K wards %K virtual care %K remote care %K financial %K finance %K finances %K remote %K respiratory %K SARS-CoV-2 %K pulmonary %K lung %K lungs %K service %K services %K delivery %D 2024 %7 13.2.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: The COVID-19 pandemic stressed global health care systems’ acute capacity and caused a diversion of resources from elective care to the treatment of acute respiratory disease. In preparing for a second wave of COVID-19 infections, England’s National Health Service (NHS) in Leicester, Leicestershire, and Rutland sought to protect acute capacity in the winter of 2020-2021. Their plans included the introduction of a digital ward where patients were discharged home early and supported remotely by community-based respiratory specialists, who were informed about patient health status by a digital patient monitoring system. Objective: The objective of the digital ward was to maintain acute capacity through safe, early discharge of patients with COVID-19 respiratory disease. The study objective was to establish what impact this digital ward had on overall NHS resource use. Methods: There were no expected differences in patient outcomes. A cost minimization was performed to demonstrate the impact on the NHS resource use from discharging patients into a digital COVID-19 respiratory ward, compared to acute care length of stay (LOS). This evaluation included all 310 patients enrolled in the service from November 2020 (service commencement) to November 2021. Two primary methods, along with sensitivity analyses, were used to help overcome the uncertainty associated with the estimated comparators for the observational data on COVID-19 respiratory acute LOS, compared with the actual LOS of the 279 (90%) patients who were not discharged on oxygen nor were in critical care. Historic comparative LOS and an ordinary least squares model based on local monthly COVID-19 respiratory median LOS were used as comparators. Actual comparator data were sourced for the 31 (10%) patients who were discharged home and into the digital ward for oxygen weaning. Resource use associated with delivering care in the digital ward was sourced from the digital system and respiratory specialists. Results: In the base case, the digital ward delivered estimated health care system savings of 846.5 bed-days and US $504,197 in net financial savings across the 2 key groups of patients—those on oxygen and those not on oxygen at acute discharge (both P<.001). The mean gross and net savings per patient were US $1850 and US $1626 in the base case, respectively, without including any savings associated with a potential reduction in readmissions. The 30-day readmission rate was 2.9%, which was below comparative data. The mean cost of the intervention was US $223.53 per patient, 12.1% of the estimated gross savings. It was not until the costs were increased and the effect reduced simultaneously by 78.4% in the sensitivity analysis that the intervention was no longer cost saving. Conclusions: The digital ward delivered increased capacity and substantial financial savings and did so with a high degree of confidence, at a very low absolute and relative cost. %M 38349716 %R 10.2196/47441 %U https://formative.jmir.org/2024/1/e47441 %U https://doi.org/10.2196/47441 %U http://www.ncbi.nlm.nih.gov/pubmed/38349716 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e53453 %T Changepoint Detection in Heart Rate Variability Indices in Older Patients Without Cancer at End of Life Using Ballistocardiography Signals: Preliminary Retrospective Study %A Yanagisawa,Naotake %A Nishizaki,Yuji %A Yao,Bingwei %A Zhang,Jianting %A Kasai,Takatoshi %+ Department of Cardiovascular Biology and Medicine, Juntendo University Graduate School of Medicine, 3-1-3 Hongo, Bunkyo-ku, Tokyo, 113-8421, Japan, 81 338133111, kasai-t@mx6.nisiq.net %K ballistocardiography %K BCG %K nonnvasive monitoring %K heart rate variability %K end-of-life care %K prognosis prediction %D 2024 %7 12.2.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: In an aging society such as Japan, where the number of older people continues to increase, providing in-hospital end-of-life care for all deaths, and end-of-life care outside of hospitals, such as at home or in nursing homes, will be difficult. In end-of-life care, monitoring patients is important to understand their condition and predict survival time; this information gives family members and caregivers time to prepare for the end of life. However, with no clear indicators, health care providers must subjectively decide if an older patient is in the end-of-life stage, considering factors such as condition changes and decreased food intake. This complicates decisions for family members, especially during home-based care. Objective: The purpose of this preliminary retrospective study was to determine whether and how changes in heart rate variability (HRV) indices estimated from ballistocardiography (BCG) occur before the date of death in terminally ill older patients, and ultimately to predict the date of death from the changepoint. Methods: This retrospective pilot study assessed the medical records of 15 older patients admitted to a special nursing home between August 2019 and December 2021. Patient characteristics and time-domain HRV indices such as the average normal-to-normal (ANN) interval, SD of the normal-to-normal (SDNN) interval, and root mean square of successive differences (RMSSD) from at least 2 months before the date of death were collected. Overall trends of indices were examined by drawing a restricted cubic spline curve. A repeated measures ANOVA was performed to evaluate changes in the indices over the observation period. To explore more detailed changes in HRV, a piecewise regression analysis was conducted to estimate the changepoint of HRV indices. Results: The 15 patients included 8 men and 7 women with a median age of 93 (IQR 91-96) years. The cubic spline curve showed a gradual decline of indices from approximately 30 days before the patients’ deaths. The repeated measures ANOVA showed that when compared with 8 weeks before death, the ratio of the geometric mean of ANN (0.90, 95% CI 0.84-0.98; P=.005) and RMSSD (0.83, 95% CI 0.70-0.99; P=.03) began to decrease 3 weeks before death. The piecewise regression analysis estimated the changepoints for ANN, SDNN, and RMSSD at –34.5 (95% CI –42.5 to –26.5; P<.001), –33.0 (95% CI –40.9 to –25.1; P<.001), and –35.0 (95% CI –42.3 to –27.7; P<.001) days, respectively, before death. Conclusions: This preliminary study identified the changepoint of HRV indices before death in older patients at end of life. Although few data were examined, our findings indicated that HRV indices from BCG can be useful for monitoring and predicting survival time in older patients at end of life. The study and results suggest the potential for more objective and accurate prognostic tools in predicting end-of-life outcomes. %M 38345857 %R 10.2196/53453 %U https://formative.jmir.org/2024/1/e53453 %U https://doi.org/10.2196/53453 %U http://www.ncbi.nlm.nih.gov/pubmed/38345857 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e45751 %T The Implementation of Federated Digital Identifiers in Health Care: Rapid Review %A Ramamoorthi,Karishini %A Stamenova,Vess %A Liu,Rebecca H %A Bhattacharyya,Onil %+ Institute for Health System Solutions and Virtual Care, Women's College Hospital, 76 Grenville St, Toronto, ON, M5S 1B2, Canada, 1 4163236400, karishini.ramamoorthi@wchospital.ca %K digital identity %K electronic health record %K environmental scan %K identity management %K identity verification %K national electronic health record %K online access %K PAEHR %K patient records %K patient-accessible electronic health records %D 2024 %7 8.2.2024 %9 Review %J J Med Internet Res %G English %X Background: Federated digital identifiers (FDIs) have been cited to improve the interoperability of data and information management while enhancing the privacy of individuals verifying their identity on the web. Many countries around the world have implemented FDIs in various sectors, such as banking and government. Similarly, FDIs could improve the experience for those wanting to access their health care information; however, they have only been introduced in a few jurisdictions around the world, and their impact remains unclear. Objective: The main objective of this environmental scan was to describe how FDIs have been established and implemented to enable patients’ access to health care. Methods: We conducted this study in 2 stages, with the primary stage being a rapid review, which was supplemented by a targeted gray literature search. Specifically, the rapid review was conducted through a database search of MEDLINE and Embase, which generated a list of countries and their services that use FDIs in health care. This list was then used to conduct a targeted gray literature search using the Google search engine. Results: A total of 93 references from the database and targeted Google searches were included in this rapid review. FDIs were implemented in health care in 11 countries (Australia, Belgium, Canada, Denmark, Estonia, Finland, Iceland, Norway, Singapore, Sweden, and Taiwan) and exclusively used with a patient-accessible electronic health record system through a single sign-on interface. The most common FDIs were implemented nationally or provincially, and establishing them usually required individuals to visit a bank or government office in person. In contrast, some countries, such as Australia, allow individuals to verify their identities entirely on the web. We found that despite the potential of FDIs for use in health care to facilitate the amalgamation of health information from different data sources into one platform, the adoption of most health care services that use FDIs remained below 30%. The exception to this was Australia, which had an adoption rate of 90%, which could be correlated with the fact that it leveraged an opt-out consent model. Conclusions: This rapid review highlights key features of FDIs across regions and elements associated with higher adoption of the patient-accessible electronic health record systems that use them, like opt-out registration. Although FDIs have been reported to facilitate the collation of data from multiple sources through a single sign-on interface, there is little information on their impact on care or patient experience. If FDIs are used to their fullest potential and implemented across sectors, adoption rates within health care may also improve. %M 38329799 %R 10.2196/45751 %U https://www.jmir.org/2024/1/e45751 %U https://doi.org/10.2196/45751 %U http://www.ncbi.nlm.nih.gov/pubmed/38329799 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e53216 %T Investigating the Impact of Prompt Engineering on the Performance of Large Language Models for Standardizing Obstetric Diagnosis Text: Comparative Study %A Wang,Lei %A Bi,Wenshuai %A Zhao,Suling %A Ma,Yinyao %A Lv,Longting %A Meng,Chenwei %A Fu,Jingru %A Lv,Hanlin %+ BGI Research, Building 11, Beishan Industrial Zone, Yantian District, Shenzhen, 518083, China, 86 18707190886, lvhanlin@genomics.cn %K obstetric data %K similarity embedding %K term standardization %K large language models %K LLMs %D 2024 %7 8.2.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: The accumulation of vast electronic medical records (EMRs) through medical informatization creates significant research value, particularly in obstetrics. Diagnostic standardization across different health care institutions and regions is vital for medical data analysis. Large language models (LLMs) have been extensively used for various medical tasks. Prompt engineering is key to use LLMs effectively. Objective: This study aims to evaluate and compare the performance of LLMs with various prompt engineering techniques on the task of standardizing obstetric diagnostic terminology using real-world obstetric data. Methods: The paper describes a 4-step approach used for mapping diagnoses in electronic medical records to the International Classification of Diseases, 10th revision, observation domain. First, similarity measures were used for mapping the diagnoses. Second, candidate mapping terms were collected based on similarity scores above a threshold, to be used as the training data set. For generating optimal mapping terms, we used two LLMs (ChatGLM2 and Qwen-14B-Chat [QWEN]) for zero-shot learning in step 3. Finally, a performance comparison was conducted by using 3 pretrained bidirectional encoder representations from transformers (BERTs), including BERT, whole word masking BERT, and momentum contrastive learning with BERT (MC-BERT), for unsupervised optimal mapping term generation in the fourth step. Results: LLMs and BERT demonstrated comparable performance at their respective optimal levels. LLMs showed clear advantages in terms of performance and efficiency in unsupervised settings. Interestingly, the performance of the LLMs varied significantly across different prompt engineering setups. For instance, when applying the self-consistency approach in QWEN, the F1-score improved by 5%, with precision increasing by 7.9%, outperforming the zero-shot method. Likewise, ChatGLM2 delivered similar rates of accurately generated responses. During the analysis, the BERT series served as a comparative model with comparable results. Among the 3 models, MC-BERT demonstrated the highest level of performance. However, the differences among the versions of BERT in this study were relatively insignificant. Conclusions: After applying LLMs to standardize diagnoses and designing 4 different prompts, we compared the results to those generated by the BERT model. Our findings indicate that QWEN prompts largely outperformed the other prompts, with precision comparable to that of the BERT model. These results demonstrate the potential of unsupervised approaches in improving the efficiency of aligning diagnostic terms in daily research and uncovering hidden information values in patient data. %M 38329787 %R 10.2196/53216 %U https://formative.jmir.org/2024/1/e53216 %U https://doi.org/10.2196/53216 %U http://www.ncbi.nlm.nih.gov/pubmed/38329787 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e32690 %T Vision-Language Model for Generating Textual Descriptions From Clinical Images: Model Development and Validation Study %A Ji,Jia %A Hou,Yongshuai %A Chen,Xinyu %A Pan,Youcheng %A Xiang,Yang %+ Peng Cheng Laboratory, No. 2 Xingke 1st Street, Shenzhen, 518000, China, 86 18566668732, panyoucheng4@gmail.com %K clinical image %K radiology report generation %K vision-language model %K multistage fine-tuning %K prior knowledge %D 2024 %7 8.2.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: The automatic generation of radiology reports, which seeks to create a free-text description from a clinical radiograph, is emerging as a pivotal intersection between clinical medicine and artificial intelligence. Leveraging natural language processing technologies can accelerate report creation, enhancing health care quality and standardization. However, most existing studies have not yet fully tapped into the combined potential of advanced language and vision models. Objective: The purpose of this study was to explore the integration of pretrained vision-language models into radiology report generation. This would enable the vision-language model to automatically convert clinical images into high-quality textual reports. Methods: In our research, we introduced a radiology report generation model named ClinicalBLIP, building upon the foundational InstructBLIP model and refining it using clinical image-to-text data sets. A multistage fine-tuning approach via low-rank adaptation was proposed to deepen the semantic comprehension of the visual encoder and the large language model for clinical imagery. Furthermore, prior knowledge was integrated through prompt learning to enhance the precision of the reports generated. Experiments were conducted on both the IU X-RAY and MIMIC-CXR data sets, with ClinicalBLIP compared to several leading methods. Results: Experimental results revealed that ClinicalBLIP obtained superior scores of 0.570/0.365 and 0.534/0.313 on the IU X-RAY/MIMIC-CXR test sets for the Metric for Evaluation of Translation with Explicit Ordering (METEOR) and the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) evaluations, respectively. This performance notably surpasses that of existing state-of-the-art methods. Further evaluations confirmed the effectiveness of the multistage fine-tuning and the integration of prior information, leading to substantial improvements. Conclusions: The proposed ClinicalBLIP model demonstrated robustness and effectiveness in enhancing clinical radiology report generation, suggesting significant promise for real-world clinical applications. %M 38329788 %R 10.2196/32690 %U https://formative.jmir.org/2024/1/e32690 %U https://doi.org/10.2196/32690 %U http://www.ncbi.nlm.nih.gov/pubmed/38329788 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e42140 %T The Environmental Impacts of Electronic Medical Records Versus Paper Records at a Large Eye Hospital in India: Life Cycle Assessment Study %A Kwon,Cordelia %A Essayei,Lernik %A Spencer,Michael %A Etheridge,Tom %A Venkatesh,Rengaraj %A Vengadesan,Natrajan %A Thiel,Cassandra L %+ Center for Healthcare Innovation and Delivery Science, Department of Population Health, NYU Langone Health, 227 E. 30th St., Room #636, New York, NY, 10016, United States, 1 6083871985, cassandra.thiel@nyulangone.org %K carbon emissions %K electronic health records %K electronic medical records %K environmental impact %K greenhouse gases %K life cycle assessment %K low middle income country %K medical records %K paper medical records %K sustainability %D 2024 %7 6.2.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Health care providers worldwide are rapidly adopting electronic medical record (EMR) systems, replacing paper record-keeping systems. Despite numerous benefits to EMRs, the environmental emissions associated with medical record-keeping are unknown. Given the need for urgent climate action, understanding the carbon footprint of EMRs will assist in decarbonizing their adoption and use. Objective: We aimed to estimate and compare the environmental emissions associated with paper medical record-keeping and its replacement EMR system at a high-volume eye care facility in southern India. Methods: We conducted the life cycle assessment methodology per the ISO (International Organization for Standardization) 14040 standard, with primary data supplied by the eye care facility. Data on the paper record-keeping system include the production, use, and disposal of paper and writing utensils in 2016. The EMR system was adopted at this location in 2018. Data on the EMR system include the allocated production and disposal of capital equipment (such as computers and routers); the production, use, and disposal of consumable goods like paper and writing utensils; and the electricity required to run the EMR system. We excluded built infrastructure and cooling loads (eg. buildings and ventilation) from both systems. We used sensitivity analyses to model the effects of practice variation and data uncertainty and Monte Carlo assessments to statistically compare the 2 systems, with and without renewable electricity sources. Results: This location’s EMR system was found to emit substantially more greenhouse gases (GHGs) than their paper medical record system (195,000 kg carbon dioxide equivalents [CO2e] per year or 0.361 kg CO2e per patient visit compared with 20,800 kg CO2e per year or 0.037 kg CO2e per patient). However, sensitivity analyses show that the effect of electricity sources is a major factor in determining which record-keeping system emits fewer GHGs. If the study hospital sourced all electricity from renewable sources such as solar or wind power rather than the Indian electric grid, their EMR emissions would drop to 24,900 kg CO2e (0.046 kg CO2e per patient), a level comparable to the paper record-keeping system. Energy-efficient EMR equipment (such as computers and monitors) is the next largest factor impacting emissions, followed by equipment life spans. Multimedia Appendix 1 includes other emissions impact categories. Conclusions: The climate-changing emissions associated with an EMR system are heavily dependent on the sources of electricity. With a decarbonized electricity source, the EMR system’s GHG emissions are on par with paper medical record-keeping, and decarbonized grids would likely have a much broader benefit to society. Though we found that the EMR system produced more emissions than a paper record-keeping system, this study does not account for potential expanded environmental gains from EMRs, including expanding access to care while reducing patient travel and operational efficiencies that can reduce unnecessary or redundant care. %M 38319701 %R 10.2196/42140 %U https://www.jmir.org/2024/1/e42140 %U https://doi.org/10.2196/42140 %U http://www.ncbi.nlm.nih.gov/pubmed/38319701 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e52080 %T The Current Status and Promotional Strategies for Cloud Migration of Hospital Information Systems in China: Strengths, Weaknesses, Opportunities, and Threats Analysis %A Xu,Jian %+ Department of Health Policy, Beijing Municipal Health Big Data and Policy Research Center, Building 1, Number 6 Daji Street, Tongzhou District, Beijing, 101160, China, 86 01055532146, _xujian@163.com %K hospital information system %K HIS %K cloud computing %K cloud migration %K Strengths, Weaknesses, Opportunities, and Threats analysis %D 2024 %7 5.2.2024 %9 Viewpoint %J JMIR Med Inform %G English %X Background: In the 21st century, Chinese hospitals have witnessed innovative medical business models, such as online diagnosis and treatment, cross-regional multidepartment consultation, and real-time sharing of medical test results, that surpass traditional hospital information systems (HISs). The introduction of cloud computing provides an excellent opportunity for hospitals to address these challenges. However, there is currently no comprehensive research assessing the cloud migration of HISs in China. This lack may hinder the widespread adoption and secure implementation of cloud computing in hospitals. Objective: The objective of this study is to comprehensively assess external and internal factors influencing the cloud migration of HISs in China and propose promotional strategies. Methods: Academic articles from January 1, 2007, to February 21, 2023, on the topic were searched in PubMed and HuiyiMd databases, and relevant documents such as national policy documents, white papers, and survey reports were collected from authoritative sources for analysis. A systematic assessment of factors influencing cloud migration of HISs in China was conducted by combining a Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis and literature review methods. Then, various promotional strategies based on different combinations of external and internal factors were proposed. Results: After conducting a thorough search and review, this study included 94 academic articles and 37 relevant documents. The analysis of these documents reveals the increasing application of and research on cloud computing in Chinese hospitals, and that it has expanded to 22 disciplinary domains. However, more than half (n=49, 52%) of the documents primarily focused on task-specific cloud-based systems in hospitals, while only 22% (n=21 articles) discussed integrated cloud platforms shared across the entire hospital, medical alliance, or region. The SWOT analysis showed that cloud computing adoption in Chinese hospitals benefits from policy support, capital investment, and social demand for new technology. However, it also faces threats like loss of digital sovereignty, supplier competition, cyber risks, and insufficient supervision. Factors driving cloud migration for HISs include medical big data analytics and use, interdisciplinary collaboration, health-centered medical service provision, and successful cases. Barriers include system complexity, security threats, lack of strategic planning and resource allocation, relevant personnel shortages, and inadequate investment. This study proposes 4 promotional strategies: encouraging more hospitals to migrate, enhancing hospitals’ capabilities for migration, establishing a provincial-level unified medical hybrid multi-cloud platform, strengthening legal frameworks, and providing robust technical support. Conclusions: Cloud computing is an innovative technology that has gained significant attention from both the Chinese government and the global community. In order to effectively support the rapid growth of a novel, health-centered medical industry, it is imperative for Chinese health authorities and hospitals to seize this opportunity by implementing comprehensive strategies aimed at encouraging hospitals to migrate their HISs to the cloud. %M 38315519 %R 10.2196/52080 %U https://medinform.jmir.org/2024/1/e52080 %U https://doi.org/10.2196/52080 %U http://www.ncbi.nlm.nih.gov/pubmed/38315519 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e53302 %T Clinical Informatics Team Members’ Perspectives on Health Information Technology Safety After Experiential Learning and Safety Process Development: Qualitative Descriptive Study %A Recsky,Chantelle %A Rush,Kathy L %A MacPhee,Maura %A Stowe,Megan %A Blackburn,Lorraine %A Muniak,Allison %A Currie,Leanne M %+ School of Nursing, University of British Columbia, T201-2211 Wesbrook Mall, Vancouver, BC, V6T 2B5, Canada, 1 604 822 7417, chantelle.recsky@ubc.ca %K informatics %K community health services %K knowledge translation %K qualitative research %K patient safety %D 2024 %7 5.2.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Although intended to support improvement, the rapid adoption and evolution of technologies in health care can also bring about unintended consequences related to safety. In this project, an embedded researcher with expertise in patient safety and clinical education worked with a clinical informatics team to examine safety and harm related to health information technologies (HITs) in primary and community care settings. The clinical informatics team participated in learning activities around relevant topics (eg, human factors, high reliability organizations, and sociotechnical systems) and cocreated a process to address safety events related to technology (ie, safety huddles and sociotechnical analysis of safety events). Objective: This study aimed to explore clinical informaticians’ experiences of incorporating safety practices into their work. Methods: We used a qualitative descriptive design and conducted web-based focus groups with clinical informaticians. Thematic analysis was used to analyze the data. Results: A total of 10 informants participated. Barriers to addressing safety and harm in their context included limited prior knowledge of HIT safety, previous assumptions and perspectives, competing priorities and organizational barriers, difficulty with the reporting system and processes, and a limited number of reports for learning. Enablers to promoting safety and mitigating harm included participating in learning sessions, gaining experience analyzing reported events, participating in safety huddles, and role modeling and leadership from the embedded researcher. Individual outcomes included increased ownership and interest in HIT safety, the development of a sociotechnical systems perspective, thinking differently about safety, and increased consideration for user perspectives. Team outcomes included enhanced communication within the team, using safety events to inform future work and strategic planning, and an overall promotion of a culture of safety. Conclusions: As HITs are integrated into care delivery, it is important for clinical informaticians to recognize the risks related to safety. Experiential learning activities, including reviewing safety event reports and participating in safety huddles, were identified as particularly impactful. An HIT safety learning initiative is a feasible approach for clinical informaticians to become more knowledgeable and engaged in HIT safety issues in their work. %M 38315544 %R 10.2196/53302 %U https://formative.jmir.org/2024/1/e53302 %U https://doi.org/10.2196/53302 %U http://www.ncbi.nlm.nih.gov/pubmed/38315544 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 12 %N %P e51510 %T Health Care Professionals’ Experiences With a Mobile Self-Care Solution for Low Complex Orthopedic Injuries: Mixed Methods Study %A Spierings,Jelle %A Willinge,Gijs %A Kokke,Marike %A Twigt,Bas %A de Lange,Wendela %A Geerdink,Thijs %A van der Velde,Detlef %A Repping,Sjoerd %A Goslings,Carel %+ Department of Traumasurgery, St Antonius Hospital, Soestwetering 1, Utrecht, 3543AZ, Netherlands, 31 883203000, j.spierings@antoniusziekenhuis.nl %K application %K direct discharge %K eHealth %K experience %K healthcare professional %K mixed method study %K orthopaedic surgery %K orthopaedic %K policy %K policymaker %K self-care application %K self-care %K trauma surgery %K utilization %K virtual fracture clinic %D 2024 %7 2.2.2024 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: To cope with the rising number of patients with trauma in an already constrained Dutch health care system, Direct Discharge (DD) has been introduced in over 25 hospitals in the Netherlands since 2019. With DD, no routine follow-up appointments are scheduled after the emergency department (ED) visit, and patients are supported through information leaflets, a smartphone app, and a telephone helpline. DD reduces secondary health care use, with comparable patient satisfaction and primary health care use. Currently, little is known about the experiences of in-hospital health care professionals with DD. Objective: The aim of this study was to explore the experiences of health care professionals with the DD protocol to enhance durable adoption and improve the protocol. Methods: We conducted a mixed methods study parallel to the implementation of DD in 3 hospitals. Data were collected through a preimplementation survey, a postimplementation survey, and semistructured interviews. Quantitative data were reported descriptively, and qualitative data were reported using thematic analysis. Outcomes included the Bowen feasibility parameters: implementation, acceptability, preliminary efficacy, demand, and applicability. Preimplementation expectations were compared with postimplementation experiences. Health care professionals involved in the daily clinical care of patients with low-complex, stable injuries were eligible for this study. Results: Of the 217 eligible health care professionals, 128 started the primary survey, 37 completed both surveys (response rate of 17%), and 15 participated in semistructured interviews. Health care professionals expressed satisfaction with the DD protocol (median 7.8, IQR 6.8-8.9) on a 10-point scale, with 82% (30/37) of participants noting improved information quality and uniformity and 73% (27/37) of patients perceiving reduced outpatient follow-up and imaging. DD was perceived as safe by 79% (28/37) of participants in its current form, but a feedback system to reassure health care professionals that patients had recovered adequately was suggested to improve DD. The introduction of DD had varying effects on workload and job satisfaction among different occupations. Health care professionals expressed intentions to continue using DD due to increased efficiency, patient empowerment, and self-management. Conclusions: Health care professionals perceive DD as an acceptable, applicable, safe, and efficacious alternative to traditional treatment. A numerical in-app feedback system (eg, in-app communication tools or recovery scores) could alleviate health care professionals’ concerns about adequate recovery and further improve DD protocols. DD can reduce health care use, which is important in times of constrained resources. Nonetheless, both advantages and disadvantages should be considered while evaluating this type of treatment. In the future, clinicians and policy makers can use these insights to further optimize and implement DD in clinical practice and guidelines. %M 38306162 %R 10.2196/51510 %U https://mhealth.jmir.org/2024/1/e51510 %U https://doi.org/10.2196/51510 %U http://www.ncbi.nlm.nih.gov/pubmed/38306162 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e49497 %T Data Representation Structure to Support Clinical Decision-Making in the Pediatric Intensive Care Unit: Interview Study and Preliminary Decision Support Interface Design %A Yakob,Najia %A Laliberté,Sandrine %A Doyon-Poulin,Philippe %A Jouvet,Philippe %A Noumeir,Rita %+ Pediatric Intensive Care Unit, CHU Sainte-Justine, 3175 Côte-Sainte-Catherine, Montreal, QC, H3T 1C5, Canada, 1 514 345 4927, philippe.jouvet@umontreal.ca %K data representation %K decision support %K critical care %K clinical workflow %K clinical decision-making %K prototype %K design %K intensive care unit %D 2024 %7 1.2.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Clinical decision-making is a complex cognitive process that relies on the interpretation of a large variety of data from different sources and involves the use of knowledge bases and scientific recommendations. The representation of clinical data plays a key role in the speed and efficiency of its interpretation. In addition, the increasing use of clinical decision support systems (CDSSs) provides assistance to clinicians in their practice, allowing them to improve patient outcomes. In the pediatric intensive care unit (PICU), clinicians must process high volumes of data and deal with ever-growing workloads. As they use multiple systems daily to assess patients’ status and to adjust the health care plan, including electronic health records (EHR), clinical systems (eg, laboratory, imaging and pharmacy), and connected devices (eg, bedside monitors, mechanical ventilators, intravenous pumps, and syringes), clinicians rely mostly on their judgment and ability to trace relevant data for decision-making. In these circumstances, the lack of optimal data structure and adapted visual representation hinder clinician’s cognitive processes and clinical decision-making skills. Objective: In this study, we designed a prototype to optimize the representation of clinical data collected from existing sources (eg, EHR, clinical systems, and devices) via a structure that supports the integration of a home-developed CDSS in the PICU. This study was based on analyzing end user needs and their clinical workflow. Methods: First, we observed clinical activities in a PICU to secure a better understanding of the workflow in terms of staff tasks and their use of EHR on a typical work shift. Second, we conducted interviews with 11 clinicians from different staff categories (eg, intensivists, fellows, nurses, and nurse practitioners) to compile their needs for decision support. Third, we structured the data to design a prototype that illustrates the proposed representation. We used a brain injury care scenario to validate the relevance of integrated data and the utility of main functionalities in a clinical context. Fourth, we held design meetings with 5 clinicians to present, revise, and adapt the prototype to meet their needs. Results: We created a structure with 3 levels of abstraction—unit level, patient level, and system level—to optimize clinical data representation and display for efficient patient assessment and to provide a flexible platform to host the internally developed CDSS. Subsequently, we designed a preliminary prototype based on this structure. Conclusions: The data representation structure allows prioritizing patients via criticality indicators, assessing their conditions using a personalized dashboard, and monitoring their courses based on the evolution of clinical values. Further research is required to define and model the concepts of criticality, problem recognition, and evolution. Furthermore, feasibility tests will be conducted to ensure user satisfaction. %M 38300695 %R 10.2196/49497 %U https://formative.jmir.org/2024/1/e49497 %U https://doi.org/10.2196/49497 %U http://www.ncbi.nlm.nih.gov/pubmed/38300695 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e49347 %T Evaluation of Eligibility Criteria Relevance for the Purpose of IT-Supported Trial Recruitment: Descriptive Quantitative Analysis %A Blasini,Romina %A Strantz,Cosima %A Gulden,Christian %A Helfer,Sven %A Lidke,Jakub %A Prokosch,Hans-Ulrich %A Sohrabi,Keywan %A Schneider,Henning %+ Institute of Medical Informatics, Justus Liebig University, Rudolf-Buchheim-Strasse 6, Giessen, 35392, Germany, 49 06419941386, romina.blasini@informatik.med.uni-giessen.de %K CTRSS %K clinical trial recruitment support system %K PRS %K patient recruitment system %K clinical trials %K classifications %K data groups %K data elements %K data classification %K criteria %K relevance %K automated clinical trials %K participants %K clinical trial %D 2024 %7 31.1.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Clinical trials (CTs) are crucial for medical research; however, they frequently fall short of the requisite number of participants who meet all eligibility criteria (EC). A clinical trial recruitment support system (CTRSS) is developed to help identify potential participants by performing a search on a specific data pool. The accuracy of the search results is directly related to the quality of the data used for comparison. Data accessibility can present challenges, making it crucial to identify the necessary data for a CTRSS to query. Prior research has examined the data elements frequently used in CT EC but has not evaluated which criteria are actually used to search for participants. Although all EC must be met to enroll a person in a CT, not all criteria have the same importance when searching for potential participants in an existing data pool, such as an electronic health record, because some of the criteria are only relevant at the time of enrollment. Objective: In this study, we investigated which groups of data elements are relevant in practice for finding suitable participants and whether there are typical elements that are not relevant and can therefore be omitted. Methods: We asked trial experts and CTRSS developers to first categorize the EC of their CTs according to data element groups and then to classify them into 1 of 3 categories: necessary, complementary, and irrelevant. In addition, the experts assessed whether a criterion was documented (on paper or digitally) or whether it was information known only to the treating physicians or patients. Results: We reviewed 82 CTs with 1132 unique EC. Of these 1132 EC, 350 (30.9%) were considered necessary, 224 (19.8%) complementary, and 341 (30.1%) total irrelevant. To identify the most relevant data elements, we introduced the data element relevance index (DERI). This describes the percentage of studies in which the corresponding data element occurs and is also classified as necessary or supplementary. We found that the query of “diagnosis” was relevant for finding participants in 79 (96.3%) of the CTs. This group was followed by “date of birth/age” with a DERI of 85.4% (n=70) and “procedure” with a DERI of 35.4% (n=29). Conclusions: The distribution of data element groups in CTs has been heterogeneously described in previous works. Therefore, we recommend identifying the percentage of CTs in which data element groups can be found as a more reliable way to determine the relevance of EC. Only necessary and complementary criteria should be included in this DERI. %M 38294862 %R 10.2196/49347 %U https://formative.jmir.org/2024/1/e49347 %U https://doi.org/10.2196/49347 %U http://www.ncbi.nlm.nih.gov/pubmed/38294862 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 13 %N %P e54213 %T Model for Doctor of Nursing Practice Projects Based on Cross-Fertilization Between Improvement and Implementation Sciences: Protocol for Quality Improvement and Program Evaluation Studies %A Sowan,Azizeh %A Chinman,Matthew %+ School of Nursing, The University of Texas Health Science Center at San Antonio, 7703 Floyd Curl Dr, San Antonio, TX, 7975, United States, 1 210 567 5799, sowan@uthscsa.edu %K quality improvement %K implementation %K Doctor of Nursing Practice %K model %K methodology %K Nursing %K Doctor of Nursing %K hybrid approach %K implementation sciences %K scholarship %K scholars %K Nursing Practice Program %K nursing program %D 2024 %7 31.1.2024 %9 Protocol %J JMIR Res Protoc %G English %X Background: Hundreds of nursing professionals graduate each year from Doctor of Nursing Practice (DNP) programs, entrusted with roles as practice scholars and leaders. Graduates are tasked to lead multidisciplinary knowledge implementation projects to improve safety, quality, and key performance metrics. Nevertheless, there is a continued lack of agreement and faculty dissatisfaction with the format, focus, and results of the DNP graduation projects. The use of a wide range of models and methodologies from different sciences for knowledge implementation introduces challenges to DNP students; affects the scientific rigor of the projects; and results in the overuse, superficial use, or misuse of the models. Quality improvement (QI) and program evaluation studies are substantial investments that may lead to waste and even harm if not well conducted. Traditional QI methodologies, commonly used in DNP projects, were found to be uncertain in improving health care outcomes. The complexity of health care systems calls for cross-fertilization between improvement and implementation sciences to improve health care outcomes. Objective: This study describes the development, implementation, and evaluation of a hybrid model for QI and program evaluation studies to guide scholarship in the DNP program. Methods: The hybrid model was based on cross-fertilization between improvement and implementation sciences. The model adapted the Getting to Outcome (GTO) and Knowledge to Action (KTA) models as the overarching process models for knowledge implementation. Within each phase of the GTO and KTA models, expected barriers and facilitators for the implementation and adoption of innovation were identified based on the CFIR (Consolidated Framework for Implementation Research). Accordingly, strategies to facilitate the implementation and adoption of innovations were identified based on a refined list of implementation strategies and QI tools. The choice of these models was based on the top 5 criteria for selecting implementation science theories and frameworks. Seven DNP students used the hybrid model to conduct QI projects. Students evaluated their experiences by responding to a Qualtrics survey. Results: The hybrid model encouraged a comprehensive systematic way of thinking, provided tools essential to implementation success, emphasized the need for adaptability in implementation, maintained rigor in QI, and guided the sustainability of change initiatives. Some of the challenges faced by students included finding reliable and valid measures, attaining and maintaining staff buy-in, and competing organizational priorities. Conclusions: Cross-fertilization between improvement and implementation sciences provided a roadmap and systematic thinking for successful QI projects in the DNP program. The integration of the CFIR with the GTO or KTA process models, enforced by the use of evidence-based implementation strategies and QI tools, reflected the complexity of health care systems and emphasized the need for adaptability in implementation. International Registered Report Identifier (IRRID): RR1-10.2196/54213 %M 38294860 %R 10.2196/54213 %U https://www.researchprotocols.org/2024/1/e54213 %U https://doi.org/10.2196/54213 %U http://www.ncbi.nlm.nih.gov/pubmed/38294860 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e46857 %T Natural Language Processing of Referral Letters for Machine Learning–Based Triaging of Patients With Low Back Pain to the Most Appropriate Intervention: Retrospective Study %A Fudickar,Sebastian %A Bantel,Carsten %A Spieker,Jannik %A Töpfer,Heinrich %A Stegeman,Patrick %A Schiphorst Preuper,Henrica R %A Reneman,Michiel F %A Wolff,André P %A Soer,Remko %+ Institute of Medical Informatics, University of Lübeck, Ratzeburger Allee 160, Lübeck, D-23538, Germany, 49 160 7979077, sebastian.fudickar@uni-luebeck.de %K decision support %K triaging %K NLP %K natural language processing %K neural network %K LBP %K low back pain %K back %K pain %K decision-making %K machine learning %K artificial intelligence %K clinical application %K patient records %K qualitative data %K support system %K questionnaire %K quality of life %K psychosocial %D 2024 %7 30.1.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Decision support systems (DSSs) for suggesting optimal treatments for individual patients with low back pain (LBP) are currently insufficiently accurate for clinical application. Most of the input provided to train these systems is based on patient-reported outcome measures. However, with the appearance of electronic health records (EHRs), additional qualitative data on reasons for referrals and patients’ goals become available for DSSs. Currently, no decision support tools cover a wide range of biopsychosocial factors, including referral letter information to help clinicians triage patients to the optimal LBP treatment. Objective: The objective of this study was to investigate the added value of including qualitative data from EHRs and referral letters to the accuracy of a quantitative DSS for patients with LBP. Methods: A retrospective study was conducted in a clinical cohort of Dutch patients with LBP. Patients filled out a baseline questionnaire about demographics, pain, disability, work status, quality of life, medication, psychosocial functioning, comorbidity, history, and duration of pain. Referral reasons and patient requests for help (patient goals) were extracted via natural language processing (NLP) and enriched in the data set. For decision support, these data were considered independent factors for triage to neurosurgery, anesthesiology, rehabilitation, or minimal intervention. Support vector machine, k-nearest neighbor, and multilayer perceptron models were trained for 2 conditions: with and without consideration of the referral letter content. The models’ accuracies were evaluated via F1-scores, and confusion matrices were used to predict the treatment path (out of 4 paths) with and without additional referral parameters. Results: Data from 1608 patients were evaluated. The evaluation indicated that 2 referral reasons from the referral letters (for anesthesiology and rehabilitation intervention) increased the F1-score accuracy by up to 19.5% for triaging. The confusion matrices confirmed the results. Conclusions: This study indicates that data enriching by adding NLP-based extraction of the content of referral letters increases the model accuracy of DSSs in suggesting optimal treatments for individual patients with LBP. Overall model accuracies were considered low and insufficient for clinical application. %M 38289669 %R 10.2196/46857 %U https://www.jmir.org/2024/1/e46857 %U https://doi.org/10.2196/46857 %U http://www.ncbi.nlm.nih.gov/pubmed/38289669 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e45209 %T Electronic Health Record and Semantic Issues Using Fast Healthcare Interoperability Resources: Systematic Mapping Review %A Amar,Fouzia %A April,Alain %A Abran,Alain %+ École de technologie supérieure - ETS, 1100 Notre Dame Ouest, Montreal, QC, H3C 1K3, Canada, 1 514 396 8800, famar2022@gmail.com %K electronic health record %K EHR %K Health Level Seven International Fast Healthcare Interoperability Resources %K HL7 FHIR %K interoperability, web ontology language %K OWL %K ontology %K semantic %K terminology %K resource description framework %K RDF %K machine learning %K ML %K natural language processing %K NLP %D 2024 %7 30.1.2024 %9 Review %J J Med Internet Res %G English %X Background: The increasing use of electronic health records and the Internet of Things has led to interoperability issues at different levels (structural and semantic). Standards are important not only for successfully exchanging data but also for appropriately interpreting them (semantic interoperability). Thus, to facilitate the semantic interoperability of data exchanged in health care, considerable resources have been deployed to improve the quality of shared clinical data by structuring and mapping them to the Fast Healthcare Interoperability Resources (FHIR) standard. Objective: The aims of this study are 2-fold: to inventory the studies on FHIR semantic interoperability resources and terminologies and to identify and classify the approaches and contributions proposed in these studies. Methods: A systematic mapping review (SMR) was conducted using 10 electronic databases as sources of information for inventory and review studies published during 2012 to 2022 on the development and improvement of semantic interoperability using the FHIR standard. Results: A total of 70 FHIR studies were selected and analyzed to identify FHIR resource types and terminologies from a semantic perspective. The proposed semantic approaches were classified into 6 categories, namely mapping (31/126, 24.6%), terminology services (18/126, 14.3%), resource description framework or web ontology language–based proposals (24/126, 19%), annotation proposals (18/126, 14.3%), machine learning (ML) and natural language processing (NLP) proposals (20/126, 15.9%), and ontology-based proposals (15/126, 11.9%). From 2012 to 2022, there has been continued research in 6 categories of approaches as well as in new and emerging annotations and ML and NLP proposals. This SMR also classifies the contributions of the selected studies into 5 categories: framework or architecture proposals, model proposals, technique proposals, comparison services, and tool proposals. The most frequent type of contribution is the proposal of a framework or architecture to enable semantic interoperability. Conclusions: This SMR provides a classification of the different solutions proposed to address semantic interoperability using FHIR at different levels: collecting, extracting and annotating data, modeling electronic health record data from legacy systems, and applying transformation and mapping to FHIR models and terminologies. The use of ML and NLP for unstructured data is promising and has been applied to specific use case scenarios. In addition, terminology services are needed to accelerate their use and adoption; furthermore, techniques and tools to automate annotation and ontology comparison should help reduce human interaction. %M 38289660 %R 10.2196/45209 %U https://www.jmir.org/2024/1/e45209 %U https://doi.org/10.2196/45209 %U http://www.ncbi.nlm.nih.gov/pubmed/38289660 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e53516 %T Using a Natural Language Processing Approach to Support Rapid Knowledge Acquisition %A Koonce,Taneya Y %A Giuse,Dario A %A Williams,Annette M %A Blasingame,Mallory N %A Krump,Poppy A %A Su,Jing %A Giuse,Nunzia B %+ Center for Knowledge Management, Vanderbilt University Medical Center, 3401 West End, Suite 304, Nashville, TN, 37203, United States, 1 6159365790, taneya.koonce@vumc.org %K natural language processing %K electronic health records %K machine learning %K data mining %K knowledge management %K NLP %D 2024 %7 30.1.2024 %9 Viewpoint %J JMIR Med Inform %G English %X Implementing artificial intelligence to extract insights from large, real-world clinical data sets can supplement and enhance knowledge management efforts for health sciences research and clinical care. At Vanderbilt University Medical Center (VUMC), the in-house developed Word Cloud natural language processing system extracts coded concepts from patient records in VUMC’s electronic health record repository using the Unified Medical Language System terminology. Through this process, the Word Cloud extracts the most prominent concepts found in the clinical documentation of a specific patient or population. The Word Cloud provides added value for clinical care decision-making and research. This viewpoint paper describes a use case for how the VUMC Center for Knowledge Management leverages the condition-disease associations represented by the Word Cloud to aid in the knowledge generation needed to inform the interpretation of phenome-wide association studies. %M 38289670 %R 10.2196/53516 %U https://medinform.jmir.org/2024/1/e53516 %U https://doi.org/10.2196/53516 %U http://www.ncbi.nlm.nih.gov/pubmed/38289670 %0 Journal Article %@ 2371-4379 %I JMIR Publications %V 9 %N %P e46096 %T Effectiveness of a Continuous Remote Temperature Monitoring Program to Reduce Foot Ulcers and Amputations: Multicenter Postmarket Registry Study %A Shih,Chia-Ding %A Scholten,Henk Jan %A Ripp,Gavin %A Srikanth,Kirthana %A Smith,Caileigh %A Ma,Ran %A Fu,Jie %A Reyzelman,Alexander M %+ Siren Care Inc, 1256 Folsom St, San Francisco, CA, 94103, United States, 1 6284449603, henkjan.scholten@siren.care %K neuropathy %K neuropathic foot ulcer %K diabetes %K diabetic foot ulcer %K amputation %K remote patient monitoring %K temperature monitoring %K prevention %K socks %D 2024 %7 29.1.2024 %9 Original Paper %J JMIR Diabetes %G English %X Background: Neuropathic foot ulcers are the leading cause of nontraumatic foot amputations, particularly among patients with diabetes. Traditional methods of monitoring and managing these patients are periodic in-person clinic visits, which are passive and may be insufficient for preventing neuropathic foot ulcers and amputations. Continuous remote temperature monitoring has the potential to capture the critical period before the foot ulcers develop and to improve outcomes by providing real-time data and early interventions. For the first time, the effectiveness of such a strategy to prevent neuropathic foot ulcers and related complications among high-risk patients in a real-world commercial setting is reported. Objective: This study aims to evaluate the effectiveness of a real-world continuous remote temperature monitoring program in preventing neuropathic foot ulcers and amputations in patients with diabetes. Methods: In this retrospective analysis of a real-world continuous remote temperature monitoring program, 115 high-risk patients identified by clinical providers from 15 geographically diverse private podiatry offices were analyzed. Patients received continuous remote monitoring socks as part of the program. The enrollment was based on medical necessity as decided by their managing physician. We evaluated data from up to 2 years before enrollment and up to 3 years during the program. The primary outcome was the rate of wound development. Secondary outcomes included amputation rate, the severity of the foot ulcers, and the number of visits to an outpatient podiatry clinic after enrolling in the program. Results: We observed significantly lower rates of foot ulceration (relative risk reduction [RRR] 0.68; 95% CI 0.52-0.79; number needed to treat [NNT] 5.0; P<.001), less moderate to severe ulcers (RRR 0.86; 95% CI 0.70-0.93; NNT 16.2; P<.001), less amputations (RRR 0.83; 95% CI 0.39-0.95; NNT 41.7; P=.006), and less hospitalizations (RRR 0.63; 95% CI 0.33-0.80; NNT 5.7; P<.002). We found a decrease in outpatient podiatry office visits during the program (RRR 0.31; 95% CI 0.24-0.37; NNT 0.46; P<.001). Conclusions: Our findings suggested that a real-world continuous remote temperature monitoring program was an effective strategy to prevent foot ulcer development and nontraumatic foot amputation among high-risk patients. %M 38285493 %R 10.2196/46096 %U https://diabetes.jmir.org/2024/1/e46096 %U https://doi.org/10.2196/46096 %U http://www.ncbi.nlm.nih.gov/pubmed/38285493 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e54274 %T Development and Implementation of Digital Diagnostic Algorithms for Neonatal Units in Zimbabwe and Malawi: Development and Usability Study %A Gannon,Hannah %A Larsson,Leyla %A Chimhuya,Simbarashe %A Mangiza,Marcia %A Wilson,Emma %A Kesler,Erin %A Chimhini,Gwendoline %A Fitzgerald,Felicity %A Zailani,Gloria %A Crehan,Caroline %A Khan,Nushrat %A Hull-Bailey,Tim %A Sassoon,Yali %A Baradza,Morris %A Heys,Michelle %A Chiume,Msandeni %+ Population, Policy and Practice, Institute of Child Health, University College London, 30 Guildford Street, London, WC1N 1EH, United Kingdom, 44 (0) 20 7905 ext 2600, h.gannon@ucl.ac.uk %K mobile health %K mHealth %K neonatology %K digital health %K mobile apps %K newborn %K Malawi, Zimbabwe %K usability %K clinical decision support %D 2024 %7 26.1.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Despite an increase in hospital-based deliveries, neonatal mortality remains high in low-resource settings. Due to limited laboratory diagnostics, there is significant reliance on clinical findings to inform diagnoses. Accurate, evidence-based identification and management of neonatal conditions could improve outcomes by standardizing care. This could be achieved through digital clinical decision support (CDS) tools. Neotree is a digital, quality improvement platform that incorporates CDS, aiming to improve neonatal care in low-resource health care facilities. Before this study, first-phase CDS development included developing and implementing neonatal resuscitation algorithms, creating initial versions of CDS to address a range of neonatal conditions, and a Delphi study to review key algorithms. Objective: This second-phase study aims to codevelop and implement neonatal digital CDS algorithms in Malawi and Zimbabwe. Methods: Overall, 11 diagnosis-specific web-based workshops with Zimbabwean, Malawian, and UK neonatal experts were conducted (August 2021 to April 2022) encompassing the following: (1) review of available evidence, (2) review of country-specific guidelines (Essential Medicines List and Standard Treatment Guidelinesfor Zimbabwe and Care of the Infant and Newborn, Malawi), and (3) identification of uncertainties within the literature for future studies. After agreement of clinical content, the algorithms were programmed into a test script, tested with the respective hospital’s health care professionals (HCPs), and refined according to their feedback. Once finalized, the algorithms were programmed into the Neotree software and implemented at the tertiary-level implementation sites: Sally Mugabe Central Hospital in Zimbabwe and Kamuzu Central Hospital in Malawi, in December 2021 and May 2022, respectively. In Zimbabwe, usability was evaluated through 2 usability workshops and usability questionnaires: Post-Study System Usability Questionnaire (PSSUQ) and System Usability Scale (SUS). Results: Overall, 11 evidence-based diagnostic and management algorithms were tailored to local resource availability. These refined algorithms were then integrated into Neotree. Where national management guidelines differed, country-specific guidelines were created. In total, 9 HCPs attended the usability workshops and completed the SUS, among whom 8 (89%) completed the PSSUQ. Both usability scores (SUS mean score 75.8 out of 100 [higher score is better]; PSSUQ overall score 2.28 out of 7 [lower score is better]) demonstrated high usability of the CDS function but highlighted issues around technical complexity, which continue to be addressed iteratively. Conclusions: This study describes the successful development and implementation of the only known neonatal CDS system, incorporated within a bedside data capture system with the ability to deliver up-to-date management guidelines, tailored to local resource availability. This study highlighted the importance of collaborative participatory design. Further implementation evaluation is planned to guide and inform the development of health system and program strategies to support newborn HCPs, with the ultimate goal of reducing preventable neonatal morbidity and mortality in low-resource settings. %M 38277198 %R 10.2196/54274 %U https://formative.jmir.org/2024/1/e54274 %U https://doi.org/10.2196/54274 %U http://www.ncbi.nlm.nih.gov/pubmed/38277198 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e47572 %T Improving Medical Photography in a Level 1 Trauma Center by Implementing a Specialized Smartphone-Based App in Comparison to the Usage of Digital Cameras: Prospective Panel Study %A El Barbari,Jan Siad %A Fikuart,Maxim %A Beisemann,Nils %A Müller,Michael %A Syrek,Hannah %A Grützner,Paul Alfred %A Franke,Jochen %A Vetter,Sven Yves %+ Department of Orthopaedics and Traumatology, BG Klinik Ludwigshafen, Ludwig-Guttmann-Str 13, Ludwigshafen am Rhein, 67071, Germany, 49 621 6810 2480, sven.vetter@bgu-ludwigshafen.de %K app %K device usability %K digital camera %K medical photo %K medical photography %K mRay app %K PACS %K patient care %K patient education %K picture archiving and communication system %K questionnaire %K smartphone %D 2024 %7 25.1.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Medical photography plays a pivotal role in modern health care, serving multiple purposes ranging from patient care to medical documentation and education. Specifically, it aids in wound management, surgical planning, and medical training. While digital cameras have traditionally been used, smartphones equipped with specialized apps present an intriguing alternative. Smartphones offer several advantages, including increased usability and efficiency and the capability to uphold medicolegal standards more effectively and consistently. Objective: This study aims to assess whether implementing a specialized smartphone app could lead to more frequent and efficient use of medical photography. Methods: We carried out this study as a comprehensive single-center panel investigation at a level 1 trauma center, encompassing various settings including the emergency department, operating theaters, and surgical wards, over a 6-month period from June to November 2020. Using weekly questionnaires, health care providers were asked about their experiences and preferences with using both digital cameras and smartphones equipped with a specialized medical photography app. Parameters such as the frequency of use, time taken for image upload, and general usability were assessed. Results: A total of 65 questionnaires were assessed for digital camera use and 68 for smartphone use. Usage increased significantly by 5.4 (SD 1.9) times per week (95% CI 1.7-9.2; P=.005) when the smartphone was used. The time it took to upload pictures to the clinical picture and archiving system was significantly shorter for the app (mean 1.8, SD 1.2 min) than for the camera (mean 14.9, SD 24.0 h; P<.001). Smartphone usage also outperformed the digital camera in terms of technical failure (4.4% vs 9.7%; P=.04) and for the technical process of archiving (P<.001) pictures to the picture archiving and communication system (PACS) and display images (P<.001) from it. No difference was found in regard to the photographer’s intent (P=.31) or reasoning (P=.94) behind the pictures. Additionally, the study highlighted that potential concerns regarding data security and patient confidentiality were also better addressed through the smartphone app, given its encryption capabilities and password protection. Conclusions: Specialized smartphone apps provide a secure, rapid, and user-friendly platform for medical photography, showing significant advantages over traditional digital cameras. This study supports the notion that these apps not only have the potential to improve patient care, particularly in the realm of wound management, but also offer substantial medicolegal and economic benefits. Future research should focus on additional aspects such as patient comfort and preference, image resolution, and the quality of photographs, as well as seek to corroborate these findings through a larger sample size. %M 38271087 %R 10.2196/47572 %U https://formative.jmir.org/2024/1/e47572 %U https://doi.org/10.2196/47572 %U http://www.ncbi.nlm.nih.gov/pubmed/38271087 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e47197 %T Understanding the Integrated Health Management System Policy in China From Multiple Perspectives: Systematic Review and Content Analysis %A Yu,Yang %A Wang,Sufen %A You,Lijue %+ Glorious Sun School of Business and Management, Donghua University, 1882 West Yan-an Road, Shanghai, 200051, China, 86 18116350361, sf_wang@dhu.edu.cn %K integrated health management system %K medical association %K medical consortium %K policy tools %K content analysis %K PRISMA %K Preferred Reporting Items for Systematic Reviews and Meta-Analyses %D 2024 %7 24.1.2024 %9 Review %J J Med Internet Res %G English %X Background: The integrated health management system (IHMS), which unites all health care–related institutions under a health-centered organizational framework, is of great significance to China in promoting the hierarchical treatment system and improving the new health care reform. China’s IHMS policy consists of multiple policies at different levels and at different times; however, there is a lack of comprehensive interpretation and analysis of these policies, which is not conducive to the further development of the IHMS in China. Objective: This study aims to comprehensively analyze and understand the characteristics, development, and evolution of China’s IHMS policy to inform the design and improvement of the system. Methods: We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines to collect 152 policy documents. With the perspective of policy tools and policy orientation as the core, a comprehensive 6D framework including policy level, policy nature, release time, policy tools, stakeholders, and policy orientation was established by combining the content of policy texts. These dimensions were then analyzed using content analysis. Results: First, we found that, regarding the coordination of policy tools and stakeholders, China’s IHMS policy was more inclined to use environment-based policy tools (1089/1929, 56.45%), which suggests a need for further balance in the internal structure of policy tools. Attention to different actors varied, and the participation of physicians and residents needs further improvement (65/2019, 3.22% and 11/2019, 0.54%, respectively). Second, in terms of level differences, Shanghai’s IHMS policy used fewer demand-based policy tools (43/483, 8.9%), whereas the national IHMS policy and those of other provinces and cities used fewer supply-based tools (61/357, 17.1% and 248/357, 69.5%, respectively). The national IHMS strategy placed more emphasis on the construction of smart health care (including digital health; 10/275, 3.6%), whereas Shanghai was a leader in the development of healthy community and healthy China (9/158, 5.7% and 4/158, 2.5%, respectively). Third, in terms of time evolution, the various policy tools showed an increasing and then decreasing trend from 2014 to 2021, with relatively more use of environment-based policy tools and less use of demand-based policy tools in the last 3 years. The growth of China’s IHMS policy can be divided into 3 stages: the disease-centered period (2014-2017), the e-health technology development period (2017-2019), and the health-centered period (2018-2021). Conclusions: Policy makers should make several adjustments, such as coordinating policy tools and the uneven relationships among stakeholders; grasping key policy priorities in the context of local characteristics; and focusing on horizontal, multidimensional integration of health resources starting from the community. This study expands the objects of policy research and improves the framework for policy analysis. The findings provide some possible lessons for future policy formulation and optimization. %M 38265862 %R 10.2196/47197 %U https://www.jmir.org/2024/1/e47197 %U https://doi.org/10.2196/47197 %U http://www.ncbi.nlm.nih.gov/pubmed/38265862 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e52524 %T Value of Electronic Health Records Measured Using Financial and Clinical Outcomes: Quantitative Study %A Modi,Shikha %A Feldman,Sue S %A Berner,Eta S %A Schooley,Benjamin %A Johnston,Allen %+ The University of Alabama in Huntsville, 1610 Ben Graves Dr NW, Huntsville, AL, 35816, United States, 1 2568242437, ssm0031@uah.edu %K acceptance %K admission %K adoption %K clinical outcome %K cost %K economic %K EHR adoption %K EHR %K electronic health record %K finance %K financial outcome %K financial %K health outcome %K health record %K hospital %K hospitalization %K length of stay %K margin %K moderation analysis %K multivariate %K operating margin %K operating %K operation %K operational %K profit %K project management %K readmission rate %K readmission %K total margin %K value analysis %K value engineering %K value management %D 2024 %7 24.1.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: The Health Information Technology for Economic and Clinical Health Act of 2009 was legislated to reduce health care costs, improve quality, and increase patient safety. Providers and organizations were incentivized to exhibit meaningful use of certified electronic health record (EHR) systems in order to achieve this objective. EHR adoption is an expensive investment, given the resources and capital that are invested. Due to the cost of the investment, a return on the EHR adoption investment is expected. Objective: This study performed a value analysis of EHRs. The objective of this study was to investigate the relationship between EHR adoption levels and financial and clinical outcomes by combining both financial and clinical outcomes into one conceptual model. Methods: We examined the multivariate relationships between different levels of EHR adoption and financial and clinical outcomes, along with the time variant control variables, using moderation analysis with a longitudinal fixed effects model. Since it is unknown as to when hospitals begin experiencing improvements in financial outcomes, additional analysis was conducted using a 1- or 2-year lag for profit margin ratios. Results: A total of 5768 hospital-year observations were analyzed over the course of 4 years. According to the results of the moderation analysis, as the readmission rate increases by 1 unit, the effect of a 1-unit increase in EHR adoption level on the operating margin decreases by 5.38%. Hospitals with higher readmission payment adjustment factors have lower penalties. Conclusions: This study fills the gap in the literature by evaluating individual relationships between EHR adoption levels and financial and clinical outcomes, in addition to evaluating the relationship between EHR adoption level and financial outcomes, with clinical outcomes as moderators. This study provided statistically significant evidence (P<.05), indicating that there is a relationship between EHR adoption level and operating margins when this relationship is moderated by readmission rates, meaning hospitals that have adopted EHRs could see a reduction in their readmission rates and an increase in operating margins. This finding could further be supported by evaluating more recent data to analyze whether hospitals increasing their level of EHR adoption would decrease readmission rates, resulting in an increase in operating margins. Hospitals would incur lower penalties as a result of improved readmission rates, which would contribute toward improved operating margins. %M 38265848 %R 10.2196/52524 %U https://medinform.jmir.org/2024/1/e52524 %U https://doi.org/10.2196/52524 %U http://www.ncbi.nlm.nih.gov/pubmed/38265848 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 12 %N %P e48842 %T Barriers and Implications of 5G Technology Adoption for Hospitals in Western China: Integrated Interpretive Structural Modeling and Decision-Making Trial and Evaluation Laboratory Analysis %A Zhou,Linyun %A Jiang,Minghuan %A Duan,Ran %A Zuo,Feng %A Li,Zongfang %A Xu,Songhua %+ Institute of Medical Artificial Intelligence, The Second Affiliated Hospital of Xi’an Jiaotong University, No 5 Jianqiang Road, Xi’an, 710016, China, 86 029 86320798, songhuaxu@126.com %K 5G health care %K 5G adoption barriers %K 5G adoption strategy %K smart health care %K Western China hospitals %D 2024 %7 23.1.2024 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: 5G technology is gaining traction in Chinese hospitals for its potential to enhance patient care and internal management. However, various barriers hinder its implementation in clinical settings, and studies on their relevance and importance are scarce. Objective: This study aimed to identify critical barriers hampering the effective implementation of 5G in hospitals in Western China, to identify interaction relationships and priorities of the above-identified barriers, and to assess the intensity of the relationships and cause-and-effect relations between the adoption barriers. Methods: This paper uses the Delphi expert consultation method to determine key barriers to 5G adoption in Western China hospitals, the interpretive structural modeling to uncover interaction relationships and priorities, and the decision-making trial and evaluation laboratory method to reveal cause-and-effect relationships and their intensity levels. Results: In total, 14 barriers were determined by literature review and the Delphi method. Among these, “lack of policies on ethics, rights, and responsibilities in core health care scenarios” emerged as the fundamental influencing factor in the entire system, as it was the only factor at the bottom level of the interpretive structural model. Overall, 8 barriers were classified as the “cause group,” and 6 as the “effect group” by the decision-making trial and evaluation laboratory method. “High expense” and “organizational barriers within hospitals” were determined as the most significant driving barrier (the highest R–C value of 1.361) and the most critical barrier (the highest R+C value of 4.317), respectively. Conclusions: Promoting the integration of 5G in hospitals in Western China faces multiple complex and interrelated barriers. The study provides valuable quantitative evidence and a comprehensive approach for regulatory authorities, hospitals, and telecom operators, helping them develop strategic pathways for promoting widespread 5G adoption in health care. It is suggested that the stakeholders cooperate to explore and solve the problems in the 5G medical care era, aiming to achieve the coverage of 5G medical care across the country. To our best knowledge, this study is the first academic exploration systematically analyzing factors resisting 5G integration in Chinese hospitals, and it may give subsequent researchers a solid foundation for further studying the application and development of 5G in health care. %M 38261368 %R 10.2196/48842 %U https://mhealth.jmir.org/2024/1/e48842 %U https://doi.org/10.2196/48842 %U http://www.ncbi.nlm.nih.gov/pubmed/38261368 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e52085 %T Big 5 Personality Traits and Individual- and Practice-Related Characteristics as Influencing Factors of Digital Maturity in General Practices: Quantitative Web-Based Survey Study %A Weik,Lisa %A Fehring,Leonard %A Mortsiefer,Achim %A Meister,Sven %+ Health Care Informatics, Faculty of Health, School of Medicine, Witten/Herdecke University, Pferdebachstr. 11, Witten, 58448, Germany, 49 230292678629, sven.meister@uni-wh.de %K digital health %K eHealth %K digital maturity %K maturity assessment %K general practitioners %K primary care physicians %K primary care %K family medicine %K personality %K digital affinity %K digital health adoption %D 2024 %7 22.1.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Various studies propose the significance of digital maturity in ensuring effective patient care and enabling improved health outcomes, a successful digital transformation, and optimized service delivery. Although previous research has centered around inpatient health care settings, research on digital maturity in general practices is still in its infancy. Objective: As general practitioners (GPs) are the first point of contact for most patients, we aimed to shed light on the pivotal role of GPs’ inherent characteristics, especially their personality, in the digital maturity of general practices. Methods: In the first step, we applied a sequential mixed methods approach involving a literature review and expert interviews with GPs to construct the digital maturity scale used in this study. Next, we designed a web-based survey to assess digital maturity on a 5-point Likert-type scale and analyze the relationship with relevant inherent characteristics using ANOVAs and regression analysis. Results: Our web-based survey with 219 GPs revealed that digital maturity was overall moderate (mean 3.31, SD 0.64) and substantially associated with several characteristics inherent to the GP. We found differences in overall digital maturity based on GPs’ gender, the expected future use of digital health solutions, the perceived digital affinity of medical assistants, GPs’ level of digital affinity, and GPs’ level of extraversion and neuroticism. In a regression model, a higher expected future use, a higher perceived digital affinity of medical assistants, a higher digital affinity of GPs, and lower neuroticism were substantial predictors of overall digital maturity. Conclusions: Our study highlights the impact of GPs’ inherent characteristics, especially their personality, on the digital maturity of general practices. By identifying these inherent influencing factors, our findings support targeted approaches to drive digital maturity in general practice settings. %M 38252468 %R 10.2196/52085 %U https://www.jmir.org/2024/1/e52085 %U https://doi.org/10.2196/52085 %U http://www.ncbi.nlm.nih.gov/pubmed/38252468 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e49986 %T A Nationwide Chronic Disease Management Solution via Clinical Decision Support Services: Software Development and Real-Life Implementation Report %A Ulgu,Mustafa Mahir %A Laleci Erturkmen,Gokce Banu %A Yuksel,Mustafa %A Namli,Tuncay %A Postacı,Şenan %A Gencturk,Mert %A Kabak,Yildiray %A Sinaci,A Anil %A Gonul,Suat %A Dogac,Asuman %A Özkan Altunay,Zübeyde %A Ekinci,Banu %A Aydin,Sahin %A Birinci,Suayip %+ Software Research Development and Consultancy Corporation, Orta Dogu Teknik Universitesi Teknokent Silikon Blok Kat 1 No 16, Ankara, 06800, Turkey, 90 3122101763, gokce@srdc.com.tr %K chronic disease management %K clinical decision support services %K integrated care %K interoperability %K evidence-based medicine %K medicine %K disease management %K management %K implementation %K decision support %K clinical decision %K support %K chronic disease %K physician-centered %K risk assessment %K tracking %K diagnosis %D 2024 %7 19.1.2024 %9 Implementation Report %J JMIR Med Inform %G English %X Background: The increasing population of older adults has led to a rise in the demand for health care services, with chronic diseases being a major burden. Person-centered integrated care is required to address these challenges; hence, the Turkish Ministry of Health has initiated strategies to implement an integrated health care model for chronic disease management. We aim to present the design, development, nationwide implementation, and initial performance results of the national Disease Management Platform (DMP). Objective: This paper’s objective is to present the design decisions taken and technical solutions provided to ensure successful nationwide implementation by addressing several challenges, including interoperability with existing IT systems, integration with clinical workflow, enabling transition of care, ease of use by health care professionals, scalability, high performance, and adaptability. Methods: The DMP is implemented as an integrated care solution that heavily uses clinical decision support services to coordinate effective screening and management of chronic diseases in adherence to evidence-based clinical guidelines and, hence, to increase the quality of health care delivery. The DMP is designed and implemented to be easily integrated with the existing regional and national health IT systems via conformance to international health IT standards, such as Health Level Seven Fast Healthcare Interoperability Resources. A repeatable cocreation strategy has been used to design and develop new disease modules to ensure extensibility while ensuring ease of use and seamless integration into the regular clinical workflow during patient encounters. The DMP is horizontally scalable in case of high load to ensure high performance. Results: As of September 2023, the DMP has been used by 25,568 health professionals to perform 73,715,269 encounters for 16,058,904 unique citizens. It has been used to screen and monitor chronic diseases such as obesity, cardiovascular risk, diabetes, and hypertension, resulting in the diagnosis of 3,545,573 patients with obesity, 534,423 patients with high cardiovascular risk, 490,346 patients with diabetes, and 144,768 patients with hypertension. Conclusions: It has been demonstrated that the platform can scale horizontally and efficiently provides services to thousands of family medicine practitioners without performance problems. The system seamlessly interoperates with existing health IT solutions and runs as a part of the clinical workflow of physicians at the point of care. By automatically accessing and processing patient data from various sources to provide personalized care plan guidance, it maximizes the effect of evidence-based decision support services by seamless integration with point-of-care electronic health record systems. As the system is built on international code systems and standards, adaptation and deployment to additional regional and national settings become easily possible. The nationwide DMP as an integrated care solution has been operational since January 2020, coordinating effective screening and management of chronic diseases in adherence to evidence-based clinical guidelines. %M 38241077 %R 10.2196/49986 %U https://medinform.jmir.org/2024/1/e49986 %U https://doi.org/10.2196/49986 %U http://www.ncbi.nlm.nih.gov/pubmed/38241077 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e53002 %T Application of Failure Mode and Effects Analysis to Improve the Quality of the Front Page of Electronic Medical Records in China: Cross-Sectional Data Mapping Analysis %A Zhan,Siyi %A Ding,Liping %A Li,Hui %A Su,Aonan %+ Zhejiang Provincial People's Hospital, No. 158, Shangtang Rd, Hangzhou, 310000, China, 86 18814885258, suaonan_512917@126.com %K front page %K EMR system %K electronic medical record %K failure mode and effects analysis %K FMEA %K measures %D 2024 %7 19.1.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: The completeness and accuracy of the front pages of electronic medical records (EMRs) are crucial for evaluating hospital performance and for health insurance payments to inpatients. However, the quality of the first page of EMRs in China's medical system is not satisfactory, which can be partly attributed to deficiencies in the EMR system. Failure mode and effects analysis (FMEA) is a proactive risk management tool that can be used to investigate the potential failure modes in an EMR system and analyze the possible consequences. Objective: The purpose of this study was to preemptively identify the potential failures of the EMR system in China and their causes and effects in order to prevent such failures from recurring. Further, we aimed to implement corresponding improvements to minimize system failure modes. Methods: From January 1, 2020, to May 31, 2022, 10 experts, including clinicians, engineers, administrators, and medical record coders, in Zhejiang People’s Hospital conducted FMEA to improve the quality of the front page of the EMR. The completeness and accuracy of the front page and the risk priority numbers were compared before and after the implementation of specific improvement measures. Results: We identified 2 main processes and 6 subprocesses for improving the EMR system. We found that there were 13 potential failure modes, including data messaging errors, data completion errors, incomplete quality control, and coding errors. A questionnaire survey administered to random physicians and coders showed 7 major causes for these failure modes. Therefore, we established quality control rules for medical records and embedded them in the system. We also integrated the medical insurance system and the front page of the EMR on the same interface and established a set of intelligent front pages in the EMR management system. Further, we revamped the quality management systems such as communicating with physicians regularly and conducting special training seminars. The overall accuracy and integrity rate of the front page (P<.001) of the EMR increased significantly after implementation of the improvement measures, while the risk priority number decreased. Conclusions: In this study, we were able to identify the potential failure modes in the front page of the EMR system by using the FMEA method and implement corresponding improvement measures in order to minimize recurring errors in the health care services in China. %M 38241064 %R 10.2196/53002 %U https://medinform.jmir.org/2024/1/e53002 %U https://doi.org/10.2196/53002 %U http://www.ncbi.nlm.nih.gov/pubmed/38241064 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e47761 %T The Implementation of an Electronic Medical Record in a German Hospital and the Change in Completeness of Documentation: Longitudinal Document Analysis %A Wurster,Florian %A Beckmann,Marina %A Cecon-Stabel,Natalia %A Dittmer,Kerstin %A Hansen,Till Jes %A Jaschke,Julia %A Köberlein-Neu,Juliane %A Okumu,Mi-Ran %A Rusniok,Carsten %A Pfaff,Holger %A Karbach,Ute %+ Chair of Quality Development and Evaluation in Rehabilitation, Institute of Medical Sociology, Health Services Research, and Rehabilitation Science, Faculty of Human Sciences & Faculty of Medicine and University Hospital Cologne, University of Cologne, Eupener Str. 129, Cologne, 50933, Germany, 49 22147897116, florian.wurster@uni-koeln.de %K clinical documentation %K digital transformation %K document analysis %K electronic medical record %K EMR %K Germany %K health services research %K hospital %K implementation %D 2024 %7 19.1.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Electronic medical records (EMR) are considered a key component of the health care system’s digital transformation. The implementation of an EMR promises various improvements, for example, in the availability of information, coordination of care, or patient safety, and is required for big data analytics. To ensure those possibilities, the included documentation must be of high quality. In this matter, the most frequently described dimension of data quality is the completeness of documentation. In this regard, little is known about how and why the completeness of documentation might change after the implementation of an EMR. Objective: This study aims to compare the completeness of documentation in paper-based medical records and EMRs and to discuss the possible impact of an EMR on the completeness of documentation. Methods: A retrospective document analysis was conducted, comparing the completeness of paper-based medical records and EMRs. Data were collected before and after the implementation of an EMR on an orthopaedical ward in a German academic teaching hospital. The anonymized records represent all treated patients for a 3-week period each. Unpaired, 2-tailed t tests, chi-square tests, and relative risks were calculated to analyze and compare the mean completeness of the 2 record types in general and of 10 specific items in detail (blood pressure, body temperature, diagnosis, diet, excretions, height, pain, pulse, reanimation status, and weight). For this purpose, each of the 10 items received a dichotomous score of 1 if it was documented on the first day of patient care on the ward; otherwise, it was scored as 0. Results: The analysis consisted of 180 medical records. The average completeness was 6.25 (SD 2.15) out of 10 in the paper-based medical record, significantly rising to an average of 7.13 (SD 2.01) in the EMR (t178=–2.469; P=.01; d=–0.428). When looking at the significant changes of the 10 items in detail, the documentation of diet (P<.001), height (P<.001), and weight (P<.001) was more complete in the EMR, while the documentation of diagnosis (P<.001), excretions (P=.02), and pain (P=.008) was less complete in the EMR. The completeness remained unchanged for the documentation of pulse (P=.28), blood pressure (P=.47), body temperature (P=.497), and reanimation status (P=.73). Conclusions: Implementing EMRs can influence the completeness of documentation, with a possible change in both increased and decreased completeness. However, the mechanisms that determine those changes are often neglected. There are mechanisms that might facilitate an improved completeness of documentation and could decrease or increase the staff’s burden caused by documentation tasks. Research is needed to take advantage of these mechanisms and use them for mutual profit in the interests of all stakeholders. Trial Registration: German Clinical Trials Register DRKS00023343; https://drks.de/search/de/trial/DRKS00023343 %M 38241076 %R 10.2196/47761 %U https://medinform.jmir.org/2024/1/e47761 %U https://doi.org/10.2196/47761 %U http://www.ncbi.nlm.nih.gov/pubmed/38241076 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e52880 %T Machine Learning Approaches for the Image-Based Identification of Surgical Wound Infections: Scoping Review %A Tabja Bortesi,Juan Pablo %A Ranisau,Jonathan %A Di,Shuang %A McGillion,Michael %A Rosella,Laura %A Johnson,Alistair %A Devereaux,PJ %A Petch,Jeremy %+ Centre for Data Science and Digital Health, Hamilton Health Sciences, 175 Longwood Road South, Suite 207, Hamilton, ON, L8P 0A1, Canada, 1 9055212100, petchj@hhsc.ca %K surgical site infection %K machine learning %K postoperative surveillance %K wound imaging %K mobile phone %D 2024 %7 18.1.2024 %9 Review %J J Med Internet Res %G English %X Background: Surgical site infections (SSIs) occur frequently and impact patients and health care systems. Remote surveillance of surgical wounds is currently limited by the need for manual assessment by clinicians. Machine learning (ML)–based methods have recently been used to address various aspects of the postoperative wound healing process and may be used to improve the scalability and cost-effectiveness of remote surgical wound assessment. Objective: The objective of this review was to provide an overview of the ML methods that have been used to identify surgical wound infections from images. Methods: We conducted a scoping review of ML approaches for visual detection of SSIs following the JBI (Joanna Briggs Institute) methodology. Reports of participants in any postoperative context focusing on identification of surgical wound infections were included. Studies that did not address SSI identification, surgical wounds, or did not use image or video data were excluded. We searched MEDLINE, Embase, CINAHL, CENTRAL, Web of Science Core Collection, IEEE Xplore, Compendex, and arXiv for relevant studies in November 2022. The records retrieved were double screened for eligibility. A data extraction tool was used to chart the relevant data, which was described narratively and presented using tables. Employment of TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) guidelines was evaluated and PROBAST (Prediction Model Risk of Bias Assessment Tool) was used to assess risk of bias (RoB). Results: In total, 10 of the 715 unique records screened met the eligibility criteria. In these studies, the clinical contexts and surgical procedures were diverse. All papers developed diagnostic models, though none performed external validation. Both traditional ML and deep learning methods were used to identify SSIs from mostly color images, and the volume of images used ranged from under 50 to thousands. Further, 10 TRIPOD items were reported in at least 4 studies, though 15 items were reported in fewer than 4 studies. PROBAST assessment led to 9 studies being identified as having an overall high RoB, with 1 study having overall unclear RoB. Conclusions: Research on the image-based identification of surgical wound infections using ML remains novel, and there is a need for standardized reporting. Limitations related to variability in image capture, model building, and data sources should be addressed in the future. %M 38236623 %R 10.2196/52880 %U https://www.jmir.org/2024/1/e52880 %U https://doi.org/10.2196/52880 %U http://www.ncbi.nlm.nih.gov/pubmed/38236623 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 12 %N %P e48345 %T Evidence of How Physicians and Their Patients Adopt mHealth Apps in Germany: Exploratory Qualitative Study %A Schroeder,Tanja %A Haug,Maximilian %A Georgiou,Andrew %A Seaman,Karla %A Gewald,Heiko %+ Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Macquarie University, Level 6, 75 Talavera Road, Sydney, 2109, Australia, 61 2 9850 ext 6281, tanja.schroeder@mq.edu.au %K mobile health apps %K DiGA %K adoption %K prescription %K mHealth %K aging and individual differences %D 2024 %7 17.1.2024 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: The enactment of the “Act to Improve Healthcare Provision through Digitalisation and Innovation ” (Digital Healthcare Act; Digitale-Versorgung-Gesetz [DVG]) in Germany has introduced a paradigm shift in medical practice, allowing physicians to prescribe mobile health (mHealth) apps alongside traditional medications. This transformation imposes a dual responsibility on physicians to acquaint themselves with qualifying apps and align them with patient diagnoses, while requiring patients to adhere to the prescribed app use, similar to pharmaceutical adherence. This transition, particularly challenging for older generations who are less skilled with technology, underscores a significant evolution in Germany’s medical landscape. Objective: This study aims to investigate physicians’ responses to this novel treatment option, their strategies for adapting to this form of prescription, and the willingness of patients to adhere to prescribed mHealth apps. Methods: Using an exploratory qualitative study design, we conducted semistructured interviews with 28 physicians and 30 potential patients aged 50 years and older from August 2020 to June 2021. Results: The findings reveal several factors influencing the adoption of mHealth apps, prompting a nuanced understanding of adoption research. Notably, both physicians and patients demonstrated a lack of information regarding mHealth apps and their positive health impacts, contributing to a deficiency in trust. Physicians’ self-perceived digital competence and their evaluation of patients’ digital proficiency emerge as pivotal factors influencing the prescription of mHealth apps. Conclusions: Our study provides comprehensive insights into the prescription process and the fundamental factors shaping the adoption of mHealth apps in Germany. The identified information gaps on both the physicians’ and patients’ sides contribute to a trust deficit and hindered digital competence. This research advances the understanding of adoption dynamics regarding digital health technologies and highlights crucial considerations for the successful integration of digital health apps into medical practice. %M 38231550 %R 10.2196/48345 %U https://mhealth.jmir.org/2024/1/e48345 %U https://doi.org/10.2196/48345 %U http://www.ncbi.nlm.nih.gov/pubmed/38231550 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e49007 %T Additional Value From Free-Text Diagnoses in Electronic Health Records: Hybrid Dictionary and Machine Learning Classification Study %A Mehra,Tarun %A Wekhof,Tobias %A Keller,Dagmar Iris %+ Department for Medical Oncology and Hematology, University Hospital of Zurich, Rämistrasse 100, Zurich, 8091, Switzerland, 41 44255 ext 1111, tarun.mehra@usz.ch %K electronic health records %K free text %K natural language processing %K NLP %K artificial intelligence %K AI %D 2024 %7 17.1.2024 %9 Original Paper %J JMIR Med Inform %G English %X Background: Physicians are hesitant to forgo the opportunity of entering unstructured clinical notes for structured data entry in electronic health records. Does free text increase informational value in comparison with structured data? Objective: This study aims to compare information from unstructured text-based chief complaints harvested and processed by a natural language processing (NLP) algorithm with clinician-entered structured diagnoses in terms of their potential utility for automated improvement of patient workflows. Methods: Electronic health records of 293,298 patient visits at the emergency department of a Swiss university hospital from January 2014 to October 2021 were analyzed. Using emergency department overcrowding as a case in point, we compared supervised NLP-based keyword dictionaries of symptom clusters from unstructured clinical notes and clinician-entered chief complaints from a structured drop-down menu with the following 2 outcomes: hospitalization and high Emergency Severity Index (ESI) score. Results: Of 12 symptom clusters, the NLP cluster was substantial in predicting hospitalization in 11 (92%) clusters; 8 (67%) clusters remained significant even after controlling for the cluster of clinician-determined chief complaints in the model. All 12 NLP symptom clusters were significant in predicting a low ESI score, of which 9 (75%) remained significant when controlling for clinician-determined chief complaints. The correlation between NLP clusters and chief complaints was low (r=−0.04 to 0.6), indicating complementarity of information. Conclusions: The NLP-derived features and clinicians’ knowledge were complementary in explaining patient outcome heterogeneity. They can provide an efficient approach to patient flow management, for example, in an emergency medicine setting. We further demonstrated the feasibility of creating extensive and precise keyword dictionaries with NLP by medical experts without requiring programming knowledge. Using the dictionary, we could classify short and unstructured clinical texts into diagnostic categories defined by the clinician. %M 38231569 %R 10.2196/49007 %U https://medinform.jmir.org/2024/1/e49007 %U https://doi.org/10.2196/49007 %U http://www.ncbi.nlm.nih.gov/pubmed/38231569 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e44653 %T Virtual and Interprofessional Objective Structured Clinical Examination in Dentistry and Dental Technology: Development and User Evaluations %A Pang,MengWei %A Dong,YanLing %A Zhao,XiaoHan %A Wan,JiaWu %A Jiang,Li %A Song,JinLin %A Ji,Ping %A Jiang,Lin %+ Stomatological Hospital of Chongqing Medical University, 426# Songshibei Road, Yubei District, Chongqing, 401147, China, 86 15922650133, jianglin@hospital.cqmu.edu.cn %K dentist %K dental technician %K objective structured clinical examination %K OSCE %K interprofessional education %K interprofessional collaborative practice %D 2024 %7 17.1.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Interprofessional education (IPE) facilitates interprofessional collaborative practice (IPCP) to encourage teamwork among dental care professionals and is increasingly becoming a part of training programs for dental and dental technology students. However, the focus of previous IPE and IPCP studies has largely been on subjective student and instructor perceptions without including objective assessments of collaborative practice as an outcome measure. Objective: The purposes of this study were to develop the framework for a novel virtual and interprofessional objective structured clinical examination (viOSCE) applicable to dental and dental technology students, to assess the effectiveness of the framework as a tool for measuring the outcomes of IPE, and to promote IPCP among dental and dental technology students. Methods: The framework of the proposed novel viOSCE was developed using the modified Delphi method and then piloted. The lead researcher and a group of experts determined the content and scoring system. Subjective data were collected using the Readiness for Interprofessional Learning Scale and a self-made scale, and objective data were collected using examiner ratings. Data were analyzed using nonparametric tests. Results: We successfully developed a viOSCE framework applicable to dental and dental technology students. Of 50 students, 32 (64%) participated in the pilot study and completed the questionnaires. On the basis of the Readiness for Interprofessional Learning Scale, the subjective evaluation indicated that teamwork skills were improved, and the only statistically significant difference in participant motivation between the 2 professional groups was in the mutual evaluation scale (P=.004). For the viOSCE evaluation scale, the difference between the professional groups in removable prosthodontics was statistically significant, and a trend for negative correlation between subjective and objective scores was noted, but it was not statistically significant. Conclusions: The results confirm that viOSCE can be used as an objective evaluation tool to assess the outcomes of IPE and IPCP. This study also revealed an interesting relationship between mutual evaluation and IPCP results, further demonstrating that the IPE and IPCP results urgently need to be supplemented with objective evaluation tools. Therefore, the implementation of viOSCE as part of a large and more complete objective structured clinical examination to test the ability of students to meet undergraduate graduation requirements will be the focus of our future studies. %M 38231556 %R 10.2196/44653 %U https://formative.jmir.org/2024/1/e44653 %U https://doi.org/10.2196/44653 %U http://www.ncbi.nlm.nih.gov/pubmed/38231556 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e45391 %T Clinical Needs Assessment of a Machine Learning–Based Asthma Management Tool: User-Centered Design Approach %A Zheng,Lu %A Ohde,Joshua W %A Overgaard,Shauna M %A Brereton,Tracey A %A Jose,Kristelle %A Wi,Chung-Il %A Peterson,Kevin J %A Juhn,Young J %+ Center for Digital Health, Mayo Clinic, 200 1st Street South West, Rochester, MN, United States, 1 480 758 0664, zheng.lu@mayo.edu %K asthma %K formative research %K user-centered design %K machine learning (ML) %K artificial intelligence (AI) %K qualitative %K user needs. %D 2024 %7 15.1.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Personalized asthma management depends on a clinician’s ability to efficiently review patient’s data and make timely clinical decisions. Unfortunately, efficient and effective review of these data is impeded by the varied format, location, and workflow of data acquisition, storage, and processing in the electronic health record. While machine learning (ML) and clinical decision support tools are well-positioned as potential solutions, the translation of such frameworks requires that barriers to implementation be addressed in the formative research stages. Objective: We aimed to use a structured user-centered design approach (double-diamond design framework) to (1) qualitatively explore clinicians’ experience with the current asthma management system, (2) identify user requirements to improve algorithm explainability and Asthma Guidance and Prediction System prototype, and (3) identify potential barriers to ML-based clinical decision support system use. Methods: At the “discovery” phase, we first shadowed to understand the practice context. Then, semistructured interviews were conducted digitally with 14 clinicians who encountered pediatric asthma patients at 2 outpatient facilities. Participants were asked about their current difficulties in gathering information for patients with pediatric asthma, their expectations of ideal workflows and tools, and suggestions on user-centered interfaces and features. At the “define” phase, a synthesis analysis was conducted to converge key results from interviewees’ insights into themes, eventually forming critical “how might we” research questions to guide model development and implementation. Results: We identified user requirements and potential barriers associated with three overarching themes: (1) usability and workflow aspects of the ML system, (2) user expectations and algorithm explainability, and (3) barriers to implementation in context. Even though the responsibilities and workflows vary among different roles, the core asthma-related information and functions they requested were highly cohesive, which allows for a shared information view of the tool. Clinicians hope to perceive the usability of the model with the ability to note patients’ high risks and take proactive actions to manage asthma efficiently and effectively. For optimal ML algorithm explainability, requirements included documentation to support the validity of algorithm development and output logic, and a request for increased transparency to build trust and validate how the algorithm arrived at the decision. Acceptability, adoption, and sustainability of the asthma management tool are implementation outcomes that are reliant on the proper design and training as suggested by participants. Conclusions: As part of our comprehensive informatics-based process centered on clinical usability, we approach the problem using a theoretical framework grounded in user experience research leveraging semistructured interviews. Our focus on meeting the needs of the practice with ML technology is emphasized by a user-centered approach to clinician engagement through upstream technology design. %M 38224482 %R 10.2196/45391 %U https://formative.jmir.org/2024/1/e45391 %U https://doi.org/10.2196/45391 %U http://www.ncbi.nlm.nih.gov/pubmed/38224482 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e51200 %T Dentists’ Information Needs and Opinions on Accessing Patient Information via Health Information Exchange: Survey Study %A Li,Shuning %A Felix Gomez,Grace Gomez %A Xu,Huiping %A Rajapuri,Anushri Singh %A Dixon,Brian E %A Thyvalikakath,Thankam %+ Department of Dental Public Health and Dental Informatics, Indiana University School of Dentistry, 415 Lansing St, Indianapolis, IN, 46201, United States, 1 3172745460, tpt@iu.edu %K dentistry %K medical history %K integrated medical and dental records %K health information exchange %K medical record %K dental record %K dental %K medical information %K dental care %K adverse drug effect %K medication %K allergies %K cost %K data safety %K data accuracy %D 2024 %7 11.1.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: The integration of medical and dental records is gaining significance over the past 2 decades. However, few studies have evaluated the opinions of practicing dentists on patient medical histories. Questions remain on dentists’ information needs; their perception of the reliability of patient-reported medical history; satisfaction with the available information and the methods to gather this information; and their attitudes to other options, such as a health information exchange (HIE) network, to collect patient medical history. Objective: This study aims to determine Indiana dentists’ information needs regarding patients’ medical information and their opinions about accessing it via an HIE. Methods: We administered a web-based survey to Indiana Dental Association members to assess their current medical information-retrieval approaches, the information critical for dental care, and their willingness to access or share information via an HIE. We used descriptive statistics to summarize survey results and multivariable regression to examine the associations between survey respondents’ characteristics and responses. Results: Of the 161 respondents (161/2148, 7.5% response rate), 99.5% (n=160) respondents considered patients’ medical histories essential to confirm no contraindications, including allergies or the need for antibiotic prophylaxis during dental care and other adverse drug events. The critical information required were medical conditions or diagnosis, current medications, and allergies, which were gathered from patient reports. Furthermore, 88.2% (n=142) of respondents considered patient-reported histories reliable; however, they experienced challenges obtaining information from patients and physicians. Additionally, 70.2% (n=113) of respondents, especially those who currently access an HIE or electronic health record, were willing to use an HIE to access or share their patient’s information, and 91.3% (n=147) shared varying interests in such a service. However, usability, data accuracy, data safety, and cost are the driving factors in adopting an HIE. Conclusions: Patients’ medical histories are essential for dentists to optimize dental care, especially for those with chronic conditions. In addition, most dentists are interested in using an HIE to access patient medical histories. The findings from this study can provide an alternative option for improving communications between dental and medical professionals and help the health information technology system or tool developers identify critical requirements for more user-friendly designs. %M 38206667 %R 10.2196/51200 %U https://formative.jmir.org/2024/1/e51200 %U https://doi.org/10.2196/51200 %U http://www.ncbi.nlm.nih.gov/pubmed/38206667 %0 Journal Article %@ 2562-0959 %I JMIR Publications %V 7 %N %P e48940 %T REDCap as a Platform for Cutaneous Disease Management in Street Medicine: Descriptive Study %A Eachus,Emily %A Schwartz,Kayla %A Rasul,Taha %A Bergholz,Daniel %A Keri,Jonette %A Henderson,Armen %+ Miami Street Medicine, University of Miami Miller School of Medicine, Suite 1149, 1600 NW 10th Avenue, Miami, FL, 33136, United States, 1 954 610 8779, eeachus@med.miami.edu %K REDCap %K unsheltered homelessness %K street medicine %K informatics %K cutaneous %K homeless %K homelessness %K data capture %K data collection %K skin %K dermatology %K vulnerable %K low income %K low resource %K database %K chart %K health record %K health records %K EHR %K electronic health record %D 2024 %7 9.1.2024 %9 Research Letter %J JMIR Dermatol %G English %X %M 38194246 %R 10.2196/48940 %U https://derma.jmir.org/2024/1/e48940 %U https://doi.org/10.2196/48940 %U http://www.ncbi.nlm.nih.gov/pubmed/38194246 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e45020 %T Key Factors Influencing the Operationalization and Effectiveness of Telemedicine Services in Henan Province, China: Cross-Sectional Analysis %A He,Xianying %A Cui,Fangfang %A Lyu,Minzhao %A Sun,Dongxu %A Zhang,Xu %A Shi,Jinming %A Zhang,Yinglan %A Jiang,Shuai %A Zhao,Jie %+ National Engineering Laboratory for Internet Medical Systems and Applications, The First Affiliated Hospital of Zhengzhou University, 1 Jianshe Road, Zhengzhou, 450052, China, 86 037167966286, zhaojie@zzu.edu.cn %K telemedicine %K service statistics %K efficiency %K quality management %D 2024 %7 5.1.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Telemedicine has demonstrated its potential in alleviating the unbalanced distribution of medical resources across different regions. Henan, a province in China with a population of approximately 100 million, is especially affected by a health care divide. The province has taken a proactive step by establishing a regional collaborative platform for telemedicine services provided by top-tier provincial hospitals. Objective: We aim to identify the key factors that influence the current operationalization and effectiveness of telemedicine services in Henan province. The insights gained from this study will serve as valuable references for enhancing the efficient operation of telemedicine platforms in low- and middle-income regions. Methods: We analyzed service reports from the performance management system of telemedicine services in Henan province throughout 2020. Using descriptive statistics and graphical methods, we examined key influencing factors, such as management competency; device configuration; and hospital capability, capacity, and service efficacy, across hospitals at 2 different tiers. In addition, we used generalized linear models and multiple linear regression models to identify key operational factors that significantly affect the service volume and efficacy of 2 major telemedicine services, namely teleconsultation and tele-education. Results: Among the 89 tier 3 hospitals and 97 tier 2 hospitals connected to the collaborative telemedicine platform, 65 (73%) and 55 (57%), respectively, have established standardized management procedures for telemedicine services. As the primary delivery method for telemedicine services, 90% (80/89) of the tier 3 hospitals and 94% (91/97) of the tier 2 hospitals host videoconferencing consultations through professional hardware terminals rather than generic computers. Teleconsultation is the dominant service type, with an average annual service volume per institution of 173 (IQR 37-372) and 60 (IQR 14-271) teleconsultations for tier 3 and tier 2 hospitals, respectively. Key factors influencing the service volume at each hospital include available funding, management competency, the number of connected upper tiers, and the number of professional staff. After receiving teleconsultations from tier 3 (65/89, 73%) and tier 2 (61/97, 63%) hospitals, patients reported significant improvements in their medical conditions. In addition, we observed that service efficacy is positively influenced by management competency, financial incentives, the number of connected upper or lower tiers, and the involvement of participating medical professionals. Conclusions: Telemedicine has become increasingly popular in Henan province, with a notable focus on teleconsultation and tele-education services. Despite its popularity, many medical institutions, especially tier 2 hospitals, face challenges related to management competency. In addition to enhancing the effectiveness of existing telemedicine services, health care decision-makers in Henan province and other low- and middle-income regions should consider expanding the service categories, such as including remote emergency care and telesurgery, which have promise in addressing crucial health care needs in these regions. %M 38180795 %R 10.2196/45020 %U https://www.jmir.org/2024/1/e45020 %U https://doi.org/10.2196/45020 %U http://www.ncbi.nlm.nih.gov/pubmed/38180795 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 12 %N %P e48716 %T Application of eHealth Tools in Anticoagulation Management After Cardiac Valve Replacement: Scoping Review Coupled With Bibliometric Analysis %A Wu,Ying %A Wang,Xiaohui %A Zhou,Mengyao %A Huang,Zhuoer %A Liu,Lijuan %A Cong,Li %+ School of Medicine, Hunan Normal University, 371 Tongzipo Road, Changsha, 410013, China, 86 0731 889124, congli@hunnu.edu.cn %K eHealth tool %K cardiac valve replacement %K anticoagulation management %K scoping review %K bibliometrics analysis %K rehabilitation %D 2024 %7 5.1.2024 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: Anticoagulation management can effectively prevent complications in patients undergoing cardiac valve replacement (CVR). The emergence of eHealth tools provides new prospects for the management of long-term anticoagulants. However, there is no comprehensive summary of the application of eHealth tools in anticoagulation management after CVR. Objective: Our objective is to clarify the current state, trends, benefits, and challenges of using eHealth tools in the anticoagulation management of patients after CVR and provide future directions and recommendations for development in this field. Methods: This scoping review follows the 5-step framework developed by Arksey and O’Malley. We searched 5 databases such as PubMed, MEDLINE, Web of Science, CINAHL, and Embase using keywords such as “eHealth,” “anticoagulation,” and “valve replacement.” We included papers on the practical application of eHealth tools and excluded papers describing the underlying mechanisms for developing eHealth tools. The search time ranged from the database inception to March 1, 2023. The study findings were reported according to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews). Additionally, VOSviewer (version 1.6.18) was used to construct visualization maps of countries, institutions, authors, and keywords to investigate the internal relations of included literature and to explore research hotspots and frontiers. Results: This study included 25 studies that fulfilled the criteria. There were 27,050 participants in total, with the sample size of the included studies ranging from 49 to 13,219. The eHealth tools mainly include computer-based support systems, electronic health records, telemedicine platforms, and mobile apps. Compared to traditional anticoagulation management, eHealth tools can improve time in therapeutic range and life satisfaction. However, there is no significant impact observed in terms of economic benefits and anticoagulation-related complications. Bibliometric analysis suggests the potential for increased collaboration and opportunities among countries and academic institutions. Italy had the widest cooperative relationships. Machine learning and artificial intelligence are the popular research directions in anticoagulation management. Conclusions: eHealth tools exhibit promise for clinical applications in anticoagulation management after CVR, with the potential to enhance postoperative rehabilitation. Further high-quality research is needed to explore the economic benefits of eHealth tools in long-term anticoagulant therapy and the potential to reduce the occurrence of adverse events. %M 38180783 %R 10.2196/48716 %U https://mhealth.jmir.org/2024/1/e48716 %U https://doi.org/10.2196/48716 %U http://www.ncbi.nlm.nih.gov/pubmed/38180783 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 12 %N %P e46744 %T Documentation Completeness and Nurses’ Perceptions of a Novel Electronic App for Medical Resuscitation in the Emergency Room: Mixed Methods Approach %A Cheung,Kin %A Yip,Chak Sum %+ School of Nursing, The Hong Kong Polytechnic University, 11 Yuk Choi Road, Hung Hom, Hong Kong, China (Hong Kong), 852 2766 6773, kin.cheung@polyu.edu.hk %K tablet computer %K nursing documentation %K paper resuscitation record %K electronic resuscitation record %K medical resuscitation %K electronic medical record %K documentation %K resuscitation %K electronic health record %K nurses’ perception %K traditional paper record %K nurse %D 2024 %7 5.1.2024 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: Complete documentation of critical care events in the accident and emergency department (AED) is essential. Due to the fast-paced and complex nature of resuscitation cases, missing data is a common issue during emergency situations. Objective: This study aimed to evaluate the impact of a tablet-based resuscitation record on documentation completeness during medical resuscitations and nurses’ perceptions of the use of the tablet app. Methods: A mixed methods approach was adopted. To collect quantitative data, randomized retrospective reviews of paper-based resuscitation records before implementation of the tablet (Pre-App Paper; n=176), paper-based resuscitation records after implementation of the tablet (Post-App Paper; n=176), and electronic tablet-based resuscitation records (Post-App Electronic; n=176) using a documentation completeness checklist were conducted. The checklist was validated by 4 experts in the emergency medicine field. The content validity index (CVI) was calculated using the scale CVI (S-CVI). The universal agreement S-CVI was 0.822, and the average S-CVI was 0.939. The checklist consisted of the following 5 domains: basic information, vital signs, procedures, investigations, and medications. To collect qualitative data, nurses’ perceptions of the app for electronic resuscitation documentation were obtained using individual interviews. Reporting of the qualitative data was guided by Consolidated Criteria for Reporting Qualitative Studies (COREQ) to enhance rigor. Results: A significantly higher documentation rate in all 5 domains (ie, basic information, vital signs, procedures, investigations, and medications) was present with Post-App Electronic than with Post-App Paper, but there were no significant differences in the 5 domains between Pre-App Paper and Post-App Paper. The qualitative analysis resulted in main categories of “advantages of tablet-based documentation of resuscitation records,” “challenges with tablet-based documentation of resuscitation records,” and “areas for improvement of tablet-based resuscitation records.” Conclusions: This study demonstrated that higher documentation completion rates are achieved with electronic tablet-based resuscitation records than with traditional paper records. During the transition period, the nurse documenters faced general problems with resuscitation documentation such as multitasking and unique challenges such as software updates and a need to familiarize themselves with the app’s layout. Automation should be considered during future app development to improve documentation and redistribute more time for patient care. Nurses should continue to provide feedback on the app’s usability and functionality during app refinement to ensure a successful transition and future development of electronic documentation records. %M 38180801 %R 10.2196/46744 %U https://mhealth.jmir.org/2024/1/e46744 %U https://doi.org/10.2196/46744 %U http://www.ncbi.nlm.nih.gov/pubmed/38180801 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 11 %N %P e46030 %T A Novel Continuous Real-Time Vital Signs Viewer for Intensive Care Units: Design and Evaluation Study %A Yang,Shiming %A Galvagno,Samuel %A Badjatia,Neeraj %A Stein,Deborah %A Teeter,William %A Scalea,Thomas %A Shackelford,Stacy %A Fang,Raymond %A Miller,Catriona %A Hu,Peter %A , %+ Department of Anesthesiology, University of Maryland School of Medicine, 11 S Paca St. LL01, Baltimore, MD, 21201, United States, 1 4103284179, syang@som.umaryland.edu %K clinical decision-making %K health information technology %K intensive care units %K patient care prioritization %K physiological monitoring %K visualization %K vital signs %D 2024 %7 5.1.2024 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Clinicians working in intensive care units (ICUs) are immersed in a cacophony of alarms and a relentless onslaught of data. Within this frenetic environment, clinicians make high-stakes decisions using many data sources and are often oversaturated with information of varying quality. Traditional bedside monitors only depict static vital signs data, and these data are not easily viewable remotely. Clinicians must rely on separate nursing charts—handwritten or electric—to review physiological patterns, including signs of potential clinical deterioration. An automated physiological data viewer has been developed to provide at-a-glance summaries and to assist with prioritizing care for multiple patients who are critically ill. Objective: This study aims to evaluate a novel vital signs viewer system in a level 1 trauma center by subjectively assessing the viewer’s utility in a high-volume ICU setting. Methods: ICU attendings were surveyed during morning rounds. Physicians were asked to conduct rounds normally, using data reported from nurse charts and briefs from fellows to inform their clinical decisions. After the physician finished their assessment and plan for the patient, they were asked to complete a questionnaire. Following completion of the questionnaire, the viewer was presented to ICU physicians on a tablet personal computer that displayed the patient’s physiologic data (ie, shock index, blood pressure, heart rate, temperature, respiratory rate, and pulse oximetry), summarized for up to 72 hours. After examining the viewer, ICU physicians completed a postview questionnaire. In both questionnaires, the physicians were asked questions regarding the patient’s stability, status, and need for a higher or lower level of care. A hierarchical clustering analysis was used to group participating ICU physicians and assess their general reception of the viewer. Results: A total of 908 anonymous surveys were collected from 28 ICU physicians from February 2015 to June 2017. Regarding physicians’ perception of whether the viewer enhanced the ability to assess multiple patients in the ICU, 5% (45/908) strongly agreed, 56.6% (514/908) agreed, 35.3% (321/908) were neutral, 2.9% (26/908) disagreed, and 0.2% (2/908) strongly disagreed. Conclusions: Morning rounds in a trauma center ICU are conducted in a busy environment with many data sources. This study demonstrates that organized physiologic data and visual assessment can improve situation awareness, assist clinicians with recognizing changes in patient status, and prioritize care. %M 38180791 %R 10.2196/46030 %U https://humanfactors.jmir.org/2024/1/e46030 %U https://doi.org/10.2196/46030 %U http://www.ncbi.nlm.nih.gov/pubmed/38180791 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e46501 %T Design and Implementation of an Inpatient Fall Risk Management Information System %A Wang,Ying %A Jiang,Mengyao %A He,Mei %A Du,Meijie %+ School of Management, Wuhan University of Technology, 122 Luoshi Road, Hongshan District, Wuhan, 430070, China, 86 027 83662317, wangying_tjh@hotmail.com %K fall %K hospital information system %K patient safety %K quality improvement %K management %K implementation %D 2024 %7 2.1.2024 %9 Implementation Report %J JMIR Med Inform %G English %X Background: Falls had been identified as one of the nursing-sensitive indicators for nursing care in hospitals. With technological progress, health information systems make it possible for health care professionals to manage patient care better. However, there is a dearth of research on health information systems used to manage inpatient falls. Objective: This study aimed to design and implement a novel hospital-based fall risk management information system (FRMIS) to prevent inpatient falls and improve nursing quality. Methods: This implementation was conducted at a large academic medical center in central China. We established a nurse-led multidisciplinary fall prevention team in January 2016. The hospital’s fall risk management problems were summarized by interviewing fall-related stakeholders, observing fall prevention workflow and post–fall care process, and investigating patients' satisfaction. The FRMIS was developed using an iterative design process, involving collaboration among health care professionals, software developers, and system architects. We used process indicators and outcome indicators to evaluate the implementation effect. Results: The FRMIS includes a fall risk assessment platform, a fall risk warning platform, a fall preventive strategies platform, fall incident reporting, and a tracking improvement platform. Since the implementation of the FRMIS, the inpatient fall rate was significantly lower than that before implementation (P<.05). In addition, the percentage of major fall-related injuries was significantly lower than that before implementation. The implementation rate of fall-related process indicators and the reporting rate of high risk of falls were significantly different before and after system implementation (P<.05). Conclusions: The FRMIS provides support to nursing staff in preventing falls among hospitalized patients while facilitating process control for nursing managers. %M 38165733 %R 10.2196/46501 %U https://medinform.jmir.org/2024/1/e46501 %U https://doi.org/10.2196/46501 %U http://www.ncbi.nlm.nih.gov/pubmed/38165733 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e48834 %T Developing a Machine Learning Algorithm to Predict the Probability of Medical Staff Work Mode Using Human-Smartphone Interaction Patterns: Algorithm Development and Validation Study %A Chen,Hung-Hsun %A Lu,Henry Horng-Shing %A Weng,Wei-Hung %A Lin,Yu-Hsuan %+ Institute of Population Health Sciences, National Health Research Institutes, 35 Keyan Road Zhunan, Miaoli County, 35053, Taiwan, 886 37 206 166 ext 36383, yuhsuanlin@nhri.edu.tw %K human-smartphone interaction %K digital phenotyping %K work hours %K machine learning %K deep learning %K probability in work mode %K one-dimensional convolutional neural network %K extreme gradient-boosted trees %D 2023 %7 29.12.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Traditional methods for investigating work hours rely on an employee’s physical presence at the worksite. However, accurately identifying break times at the worksite and distinguishing remote work outside the worksite poses challenges in work hour estimations. Machine learning has the potential to differentiate between human-smartphone interactions at work and off work. Objective: In this study, we aimed to develop a novel approach called “probability in work mode,” which leverages human-smartphone interaction patterns and corresponding GPS location data to estimate work hours. Methods: To capture human-smartphone interactions and GPS locations, we used the “Staff Hours” app, developed by our team, to passively and continuously record participants’ screen events, including timestamps of notifications, screen on or off occurrences, and app usage patterns. Extreme gradient boosted trees were used to transform these interaction patterns into a probability, while 1-dimensional convolutional neural networks generated successive probabilities based on previous sequence probabilities. The resulting probability in work mode allowed us to discern periods of office work, off-work, breaks at the worksite, and remote work. Results: Our study included 121 participants, contributing to a total of 5503 person-days (person-days represent the cumulative number of days across all participants on which data were collected and analyzed). The developed machine learning model exhibited an average prediction performance, measured by the area under the receiver operating characteristic curve, of 0.915 (SD 0.064). Work hours estimated using the probability in work mode (higher than 0.5) were significantly longer (mean 11.2, SD 2.8 hours per day) than the GPS-defined counterparts (mean 10.2, SD 2.3 hours per day; P<.001). This discrepancy was attributed to the higher remote work time of 111.6 (SD 106.4) minutes compared to the break time of 54.7 (SD 74.5) minutes. Conclusions: Our novel approach, the probability in work mode, harnessed human-smartphone interaction patterns and machine learning models to enhance the precision and accuracy of work hour investigation. By integrating human-smartphone interactions and GPS data, our method provides valuable insights into work patterns, including remote work and breaks, offering potential applications in optimizing work productivity and well-being. %M 38157232 %R 10.2196/48834 %U https://www.jmir.org/2023/1/e48834 %U https://doi.org/10.2196/48834 %U http://www.ncbi.nlm.nih.gov/pubmed/38157232 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e48702 %T Can OpenEHR, ISO 13606, and HL7 FHIR Work Together? An Agnostic Approach for the Selection and Application of Electronic Health Record Standards to the Next-Generation Health Data Spaces %A Pedrera-Jiménez,Miguel %A García-Barrio,Noelia %A Frid,Santiago %A Moner,David %A Boscá-Tomás,Diego %A Lozano-Rubí,Raimundo %A Kalra,Dipak %A Beale,Thomas %A Muñoz-Carrero,Adolfo %A Serrano-Balazote,Pablo %+ Data Science Unit, Hospital Universitario 12 de Octubre, Av. de Córdoba, s/n, Madrid, 28041, Spain, 34 634209791, mpedrerajimenez@gmail.com %K electronic health records %K FAIR principles %K health information standards %K HL7 FHIR %K ISO 13606 %K OpenEHR %K semantics %D 2023 %7 28.12.2023 %9 Viewpoint %J J Med Internet Res %G English %X In order to maximize the value of electronic health records (EHRs) for both health care and secondary use, it is necessary for the data to be interoperable and reusable without loss of the original meaning and context, in accordance with the findable, accessible, interoperable, and reusable (FAIR) principles. To achieve this, it is essential for health data platforms to incorporate standards that facilitate addressing needs such as formal modeling of clinical knowledge (health domain concepts) as well as the harmonized persistence, query, and exchange of data across different information systems and organizations. However, the selection of these specifications has not been consistent across the different health data initiatives, often applying standards to address needs for which they were not originally designed. This issue is essential in the current scenario of implementing the European Health Data Space, which advocates harmonization, interoperability, and reuse of data without regulating the specific standards to be applied for this purpose. Therefore, this viewpoint aims to establish a coherent, agnostic, and homogeneous framework for the use of the most impactful EHR standards in the new-generation health data spaces: OpenEHR, International Organization for Standardization (ISO) 13606, and Health Level 7 (HL7) Fast Healthcare Interoperability Resources (FHIR). Thus, a panel of EHR standards experts has discussed several critical points to reach a consensus that will serve decision-making teams in health data platform projects who may not be experts in these EHR standards. It was concluded that these specifications possess different capabilities related to modeling, flexibility, and implementation resources. Because of this, in the design of future data platforms, these standards must be applied based on the specific needs they were designed for, being likewise fully compatible with their combined functional and technical implementation. %M 38153779 %R 10.2196/48702 %U https://www.jmir.org/2023/1/e48702 %U https://doi.org/10.2196/48702 %U http://www.ncbi.nlm.nih.gov/pubmed/38153779 %0 Journal Article %@ 2369-3762 %I JMIR Publications %V 9 %N %P e51199 %T Empathy and Equity: Key Considerations for Large Language Model Adoption in Health Care %A Koranteng,Erica %A Rao,Arya %A Flores,Efren %A Lev,Michael %A Landman,Adam %A Dreyer,Keith %A Succi,Marc %+ Massachusetts General Hospital, 55 Fruit St, Boston, 02114, United States, 1 617 935 9144, msucci@mgh.harvard.edu %K ChatGPT %K AI %K artificial intelligence %K large language models %K LLMs %K ethics %K empathy %K equity %K bias %K language model %K health care application %K patient care %K care %K development %K framework %K model %K ethical implication %D 2023 %7 28.12.2023 %9 Viewpoint %J JMIR Med Educ %G English %X The growing presence of large language models (LLMs) in health care applications holds significant promise for innovative advancements in patient care. However, concerns about ethical implications and potential biases have been raised by various stakeholders. Here, we evaluate the ethics of LLMs in medicine along 2 key axes: empathy and equity. We outline the importance of these factors in novel models of care and develop frameworks for addressing these alongside LLM deployment. %M 38153778 %R 10.2196/51199 %U https://mededu.jmir.org/2023/1/e51199 %U https://doi.org/10.2196/51199 %U http://www.ncbi.nlm.nih.gov/pubmed/38153778 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e48544 %T Economic Evaluations and Equity in the Use of Artificial Intelligence in Imaging Exams for Medical Diagnosis in People With Skin, Neurological, and Pulmonary Diseases: Protocol for a Systematic Review %A Santana,Giulia Osório %A Couto,Rodrigo de Macedo %A Loureiro,Rafael Maffei %A Furriel,Brunna Carolinne Rocha Silva %A Rother,Edna Terezinha %A de Paiva,Joselisa Péres Queiroz %A Correia,Lucas Reis %+ PROADI-SUS, Hospital Israelita Albert Einstein, Madre Cabrini Street, 462, Tower A, 5th Floor, São Paulo, Brazil, 55 11 97444 8995, giulia.santana@einstein.br %K artificial intelligence %K economic evaluation %K equity %K medical diagnosis %K health care system %K technology %K systematic review %K cost-effectiveness %K imaging exam %K intervention %D 2023 %7 28.12.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Traditional health care systems face long-standing challenges, including patient diversity, geographical disparities, and financial constraints. The emergence of artificial intelligence (AI) in health care offers solutions to these challenges. AI, a multidisciplinary field, enhances clinical decision-making. However, imbalanced AI models may enhance health disparities. Objective: This systematic review aims to investigate the economic performance and equity impact of AI in diagnostic imaging for skin, neurological, and pulmonary diseases. The research question is “To what extent does the use of AI in imaging exams for diagnosing skin, neurological, and pulmonary diseases result in improved economic outcomes, and does it promote equity in health care systems?” Methods: The study is a systematic review of economic and equity evaluations following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and CHEERS (Consolidated Health Economic Evaluation Reporting Standards) guidelines. Eligibility criteria include articles reporting on economic evaluations or equity considerations related to AI-based diagnostic imaging for specified diseases. Data will be collected from PubMed, Embase, Scopus, Web of Science, and reference lists. Data quality and transferability will be assessed according to CHEC (Consensus on Health Economic Criteria), EPHPP (Effective Public Health Practice Project), and Welte checklists. Results: This systematic review began in March 2023. The literature search identified 9,526 publications and, after full-text screening, 9 publications were included in the study. We plan to submit a manuscript to a peer-reviewed journal once it is finalized, with an expected completion date in January 2024. Conclusions: AI in diagnostic imaging offers potential benefits but also raises concerns about equity and economic impact. Bias in algorithms and disparities in access may hinder equitable outcomes. Evaluating the economic viability of AI applications is essential for resource allocation and affordability. Policy makers and health care stakeholders can benefit from this review’s insights to make informed decisions. Limitations, including study variability and publication bias, will be considered in the analysis. This systematic review will provide valuable insights into the economic and equity implications of AI in diagnostic imaging. It aims to inform evidence-based decision-making and contribute to more efficient and equitable health care systems. International Registered Report Identifier (IRRID): DERR1-10.2196/48544 %M 38153775 %R 10.2196/48544 %U https://www.researchprotocols.org/2023/1/e48544 %U https://doi.org/10.2196/48544 %U http://www.ncbi.nlm.nih.gov/pubmed/38153775 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e47840 %T Users’ Experiences With Online Access to Electronic Health Records in Mental and Somatic Health Care: Cross-Sectional Study %A Wang,Bo %A Kristiansen,Eli %A Fagerlund,Asbjørn Johansen %A Zanaboni,Paolo %A Hägglund,Maria %A Bärkås,Annika %A Kujala,Sari %A Cajander,Åsa %A Blease,Charlotte %A Kharko,Anna %A Huvila,Isto %A Kane,Bridget %A Johansen,Monika Alise %+ Norwegian Centre for E-health Research, University Hospital of North Norway, Sykehusvegen 23, Forskningsparken, Tromsø, 9019, Norway, 47 46366629, bo.wang@ehealthresearch.no %K patient empowerment %K online access to electronic health records %K patient-accessible electronic health record %K patient access %K user perspective %K psychiatry %K electronic health record %K health data %K patient portal %K online records access %D 2023 %7 25.12.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Patient-accessible electronic health records (PAEHRs) hold promise for empowering patients, but their impact may vary between mental and somatic health care. Medical professionals and ethicists have expressed concerns about the potential challenges of PAEHRs for patients, especially those receiving mental health care. Objective: This study aims to investigate variations in the experiences of online access to electronic health records (EHRs) among persons receiving mental and somatic health care, as well as to understand how these experiences and perceptions vary among those receiving mental health care at different levels of point of care. Methods: Using Norwegian data from the NORDeHEALTH 2022 Patient Survey, we conducted a cross-sectional descriptive analysis of service use and perceptions of perceived mistakes, omissions, and offensive comments by mental and somatic health care respondents. Content analysis was used to analyze free-text responses to understand how respondents experienced the most serious errors in their EHR. Results: Among 9505 survey participants, we identified 2008 mental health care respondents and 7086 somatic health care respondents. A higher percentage of mental health care respondents (1385/2008, 68.97%) reported that using PAEHR increased their trust in health care professionals compared with somatic health care respondents (4251/7086, 59.99%). However, a significantly larger proportion (P<.001) of mental health care respondents (976/2008, 48.61%) reported perceiving errors in their EHR compared with somatic health care respondents (1893/7086, 26.71%). Mental health care respondents also reported significantly higher odds (P<.001) of identifying omissions (758/2008, 37.75%) and offensive comments (729/2008, 36.3%) in their EHR compared with the somatic health care group (1867/7086, 26.35% and 826/7086, 11.66%, respectively). Mental health care respondents in hospital inpatient settings were more likely to identify errors (398/588, 67.7%; P<.001) and omissions (251/588, 42.7%; P<.001) than those in outpatient care (errors: 422/837, 50.4% and omissions: 336/837, 40.1%; P<.001) and primary care (errors: 32/100, 32% and omissions: 29/100, 29%; P<.001). Hospital inpatients also reported feeling more offended (344/588, 58.5%; P<.001) by certain content in their EHR compared with respondents in primary (21/100, 21%) and outpatient care (287/837, 34.3%) settings. Our qualitative findings showed that both mental and somatic health care respondents identified the most serious errors in their EHR in terms of medical history, communication, diagnosis, and medication. Conclusions: Most mental and somatic health care respondents showed a positive attitude toward PAEHRs. However, mental health care respondents, especially those with severe and chronic concerns, expressed a more critical attitude toward certain content in their EHR compared with somatic health care respondents. A PAEHR can provide valuable information and foster trust, but it requires careful attention to the use of clinical terminology to ensure accurate, nonjudgmental documentation, especially for persons belonging to health care groups with unique sensitivities. %M 38145466 %R 10.2196/47840 %U https://www.jmir.org/2023/1/e47840 %U https://doi.org/10.2196/47840 %U http://www.ncbi.nlm.nih.gov/pubmed/38145466 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e48244 %T Explainable Artificial Intelligence Warning Model Using an Ensemble Approach for In-Hospital Cardiac Arrest Prediction: Retrospective Cohort Study %A Kim,Yun Kwan %A Koo,Ja Hyung %A Lee,Sun Jung %A Song,Hee Seok %A Lee,Minji %+ Department of Biomedical Software Engineering, The Catholic University of Korea, 43, Jibong-ro, Bucheon, Gyeonggi, 14662, Republic of Korea, 82 2 2164 4364, minjilee@catholic.ac.kr %K cardiac arrest prediction %K ensemble learning %K temporal pattern changes %K cost-sensitive learning %K electronic medical records %D 2023 %7 22.12.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Cardiac arrest (CA) is the leading cause of death in critically ill patients. Clinical research has shown that early identification of CA reduces mortality. Algorithms capable of predicting CA with high sensitivity have been developed using multivariate time series data. However, these algorithms suffer from a high rate of false alarms, and their results are not clinically interpretable. Objective: We propose an ensemble approach using multiresolution statistical features and cosine similarity–based features for the timely prediction of CA. Furthermore, this approach provides clinically interpretable results that can be adopted by clinicians. Methods: Patients were retrospectively analyzed using data from the Medical Information Mart for Intensive Care-IV database and the eICU Collaborative Research Database. Based on the multivariate vital signs of a 24-hour time window for adults diagnosed with heart failure, we extracted multiresolution statistical and cosine similarity–based features. These features were used to construct and develop gradient boosting decision trees. Therefore, we adopted cost-sensitive learning as a solution. Then, 10-fold cross-validation was performed to check the consistency of the model performance, and the Shapley additive explanation algorithm was used to capture the overall interpretability of the proposed model. Next, external validation using the eICU Collaborative Research Database was performed to check the generalization ability. Results: The proposed method yielded an overall area under the receiver operating characteristic curve (AUROC) of 0.86 and area under the precision-recall curve (AUPRC) of 0.58. In terms of the timely prediction of CA, the proposed model achieved an AUROC above 0.80 for predicting CA events up to 6 hours in advance. The proposed method simultaneously improved precision and sensitivity to increase the AUPRC, which reduced the number of false alarms while maintaining high sensitivity. This result indicates that the predictive performance of the proposed model is superior to the performances of the models reported in previous studies. Next, we demonstrated the effect of feature importance on the clinical interpretability of the proposed method and inferred the effect between the non-CA and CA groups. Finally, external validation was performed using the eICU Collaborative Research Database, and an AUROC of 0.74 and AUPRC of 0.44 were obtained in a general intensive care unit population. Conclusions: The proposed framework can provide clinicians with more accurate CA prediction results and reduce false alarm rates through internal and external validation. In addition, clinically interpretable prediction results can facilitate clinician understanding. Furthermore, the similarity of vital sign changes can provide insights into temporal pattern changes in CA prediction in patients with heart failure–related diagnoses. Therefore, our system is sufficiently feasible for routine clinical use. In addition, regarding the proposed CA prediction system, a clinically mature application has been developed and verified in the future digital health field. %M 38133922 %R 10.2196/48244 %U https://www.jmir.org/2023/1/e48244 %U https://doi.org/10.2196/48244 %U http://www.ncbi.nlm.nih.gov/pubmed/38133922 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e49301 %T The Necessity of Interoperability to Uncover the Full Potential of Digital Health Devices %A Schwab,Julian D %A Werle,Silke D %A Hühne,Rolf %A Spohn,Hannah %A Kaisers,Udo X %A Kestler,Hans A %+ Ulm University, Albert-Einstein-Allee 11, Ulm, 89081, Germany, 49 731 500 24500, hans.kestler@uni-ulm.de %K semantic terminology %K semantic %K terminology %K terminologies %K data linkage %K interoperability %K data exchange %K SNOMED CT %K LOINC %K eHealth %K patient-reported outcome questionnaires %K requirement for standards %K standard %K standards %K PRO %K PROM %K patient reported %D 2023 %7 22.12.2023 %9 Viewpoint %J JMIR Med Inform %G English %X Personalized health care can be optimized by including patient-reported outcomes. Standardized and disease-specific questionnaires have been developed and are routinely used. These patient-reported outcome questionnaires can be simple paper forms given to the patient to fill out with a pen or embedded in digital devices. Regardless of the format used, they provide a snapshot of the patient’s feelings and indicate when therapies need to be adjusted. The advantage of digitizing these questionnaires is that they can be automatically analyzed, and patients can be monitored independently of doctor visits. Although the questions of most clinical patient-reported outcome questionnaires follow defined standards and are evaluated by clinical trials, these standards do not exist for data processing. Interoperable data formats and structures would benefit multilingual and cross-study data exchange. Linking questionnaires to standardized terminologies such as the Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT) and Logical Observation Identifiers, Names, and Codes (LOINC) would improve this interoperability. However, linking clinically validated patient-reported outcome questionnaires to clinical terms available in SNOMED CT or LOINC is not as straightforward as it sounds. Here, we report our approach to link patient-reported outcomes from health applications to SNOMED CT or LOINC codes. We highlight current difficulties in this process and outline ways to minimize them. %M 38133917 %R 10.2196/49301 %U https://medinform.jmir.org/2023/1/e49301 %U https://doi.org/10.2196/49301 %U http://www.ncbi.nlm.nih.gov/pubmed/38133917 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e51471 %T Effects of Internal and External Factors on Hospital Data Breaches: Quantitative Study %A Dolezel,Diane %A Beauvais,Brad %A Stigler Granados,Paula %A Fulton,Lawrence %A Kruse,Clemens Scott %+ Health Informatics & Information Management Department, Texas State University, 100 Bobcat Way, Round Rock, TX, 78665, United States, 1 512 716 2840, dd30@txstate.edu %K data breach %K security %K geospatial %K predictive %K mobile phone %D 2023 %7 21.12.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Health care data breaches are the most rapidly increasing type of cybercrime; however, the predictors of health care data breaches are uncertain. Objective: This quantitative study aims to develop a predictive model to explain the number of hospital data breaches at the county level. Methods: This study evaluated data consolidated at the county level from 1032 short-term acute care hospitals. We considered the association between data breach occurrence (a dichotomous variable), predictors based on county demographics, and socioeconomics, average hospital workload, facility type, and average performance on several hospital financial metrics using 3 model types: logistic regression, perceptron, and support vector machine. Results: The model coefficient performance metrics indicated convergent validity across the 3 model types for all variables except bad debt and the factor level accounting for counties with >20% and up to 40% Hispanic populations, both of which had mixed coefficient directionality. The support vector machine model performed the classification task best based on all metrics (accuracy, precision, recall, F1-score). All the 3 models performed the classification task well with directional congruence of weights. From the logistic regression model, the top 5 odds ratios (indicating a higher risk of breach) included inpatient workload, medical center status, pediatric trauma center status, accounts receivable, and the number of outpatient visits, in high to low order. The bottom 5 odds ratios (indicating the lowest odds of experiencing a data breach) occurred for counties with Black populations of >20% and <40%, >80% and <100%, and >40% but <60%, as well as counties with ≤20% Asian or between 80% and 100% Hispanic individuals. Our results are in line with those of other studies that determined that patient workload, facility type, and financial outcomes were associated with the likelihood of health care data breach occurrence. Conclusions: The results of this study provide a predictive model for health care data breaches that may guide health care managers to reduce the risk of data breaches by raising awareness of the risk factors. %M 38127426 %R 10.2196/51471 %U https://www.jmir.org/2023/1/e51471 %U https://doi.org/10.2196/51471 %U http://www.ncbi.nlm.nih.gov/pubmed/38127426 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e42505 %T A Biobanking System for Diagnostic Images: Architecture Development, COVID-19–Related Use Cases, and Performance Evaluation %A Esposito,Giuseppina %A Allarà,Ciro %A Randon,Marco %A Aiello,Marco %A Salvatore,Marco %A Aceto,Giuseppe %A Pescapè,Antonio %+ Bio Check Up Srl, Via Riviera di Chiaia, 9a, Naples, 80122, Italy, 39 08119322515, gesposito@biocheckup.net %K biobank %K diagnostics %K COVID-19 %K network performance %K eHealth %D 2023 %7 21.12.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Systems capable of automating and enhancing the management of research and clinical data represent a significant contribution of information and communication technologies to health care. A recent advancement is the development of imaging biobanks, which are now enabling the collection and storage of diagnostic images, clinical reports, and demographic data to allow researchers identify associations between lifestyle and genetic factors and imaging-derived phenotypes. Objective: The aim of this study was to design and evaluate the system performance of a network for an operating biobank of diagnostic images, the Bio Check Up Srl (BCU) Imaging Biobank, based on the Extensible Neuroimaging Archive Toolkit open-source platform. Methods: Three usage cases were designed focusing on evaluation of the memory and computing consumption during imaging collections upload and during interactions between two kinds of users (researchers and radiologists) who inspect chest computed tomography scans of a COVID-19 cohort. The experiments considered three network setups: (1) a local area network, (2) virtual private network, and (3) wide area network. The experimental setup recorded the activity of a human user interacting with the biobank system, which was continuously replayed multiple times. Several metrics were extracted from network traffic traces and server logs captured during the activity replay. Results: Regarding the diagnostic data transfer, two types of containers were considered: the Web and the Database containers. The Web appeared to be the more memory-hungry container with a higher computational load (average 2.7 GB of RAM) compared to that of the database. With respect to user access, both users demonstrated the same network performance level, although higher resource consumption was registered for two different actions: DOWNLOAD & LOGOUT (100%) for the researcher and OPEN VIEWER (20%-50%) for the radiologist. Conclusions: This analysis shows that the current setup of BCU Imaging Biobank is well provisioned for satisfying the planned number of concurrent users. More importantly, this study further highlights and quantifies the resource demands of specific user actions, providing a guideline for planning, setting up, and using an image biobanking system. %M 38064636 %R 10.2196/42505 %U https://formative.jmir.org/2023/1/e42505 %U https://doi.org/10.2196/42505 %U http://www.ncbi.nlm.nih.gov/pubmed/38064636 %0 Journal Article %@ 2369-3762 %I JMIR Publications %V 9 %N %P e50903 %T Reimagining Core Entrustable Professional Activities for Undergraduate Medical Education in the Era of Artificial Intelligence %A Jacobs,Sarah Marie %A Lundy,Neva Nicole %A Issenberg,Saul Barry %A Chandran,Latha %+ Department of Medical Education, University of Miami Miller School of Medicine, 1120 NW 14th Street, Miami, FL, 33136, United States, 1 3052436491, bissenbe@miami.edu %K artificial intelligence %K entrustable professional activities %K medical education %K competency-based education %K educational technology %K machine learning %D 2023 %7 19.12.2023 %9 Viewpoint %J JMIR Med Educ %G English %X The proliferation of generative artificial intelligence (AI) and its extensive potential for integration into many aspects of health care signal a transformational shift within the health care environment. In this context, medical education must evolve to ensure that medical trainees are adequately prepared to navigate the rapidly changing health care landscape. Medical education has moved toward a competency-based education paradigm, leading the Association of American Medical Colleges (AAMC) to define a set of Entrustable Professional Activities (EPAs) as its practical operational framework in undergraduate medical education. The AAMC’s 13 core EPAs for entering residencies have been implemented with varying levels of success across medical schools. In this paper, we critically assess the existing core EPAs in the context of rapid AI integration in medicine. We identify EPAs that require refinement, redefinition, or comprehensive change to align with the emerging trends in health care. Moreover, this perspective proposes a set of “emerging” EPAs, informed by the changing landscape and capabilities presented by generative AI technologies. We provide a practical evaluation of the EPAs, alongside actionable recommendations on how medical education, viewed through the lens of the AAMC EPAs, can adapt and remain relevant amid rapid technological advancements. By leveraging the transformative potential of AI, we can reshape medical education to align with an AI-integrated future of medicine. This approach will help equip future health care professionals with technological competence and adaptive skills to meet the dynamic and evolving demands in health care. %M 38052721 %R 10.2196/50903 %U https://mededu.jmir.org/2023/1/e50903 %U https://doi.org/10.2196/50903 %U http://www.ncbi.nlm.nih.gov/pubmed/38052721 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e50342 %T Existing Barriers Faced by and Future Design Recommendations for Direct-to-Consumer Health Care Artificial Intelligence Apps: Scoping Review %A He,Xin %A Zheng,Xi %A Ding,Huiyuan %+ School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Luoyu Road 1037, Hongshan District, Wuhan, 430074, China, 86 18707149470, xinh@hust.edu.cn %K artificial intelligence %K medical %K health care %K consumer %K consumers %K app %K apps %K application %K applications %K DTC %K direct to consumer %K barrier %K barriers %K implementation %K design %K scoping %K review methods %K review methodology %D 2023 %7 18.12.2023 %9 Review %J J Med Internet Res %G English %X Background: Direct-to-consumer (DTC) health care artificial intelligence (AI) apps hold the potential to bridge the spatial and temporal disparities in health care resources, but they also come with individual and societal risks due to AI errors. Furthermore, the manner in which consumers interact directly with health care AI is reshaping traditional physician-patient relationships. However, the academic community lacks a systematic comprehension of the research overview for such apps. Objective: This paper systematically delineated and analyzed the characteristics of included studies, identified existing barriers and design recommendations for DTC health care AI apps mentioned in the literature and also provided a reference for future design and development. Methods: This scoping review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews guidelines and was conducted according to Arksey and O’Malley’s 5-stage framework. Peer-reviewed papers on DTC health care AI apps published until March 27, 2023, in Web of Science, Scopus, the ACM Digital Library, IEEE Xplore, PubMed, and Google Scholar were included. The papers were analyzed using Braun and Clarke’s reflective thematic analysis approach. Results: Of the 2898 papers retrieved, 32 (1.1%) covering this emerging field were included. The included papers were recently published (2018-2023), and most (23/32, 72%) were from developed countries. The medical field was mostly general practice (8/32, 25%). In terms of users and functionalities, some apps were designed solely for single-consumer groups (24/32, 75%), offering disease diagnosis (14/32, 44%), health self-management (8/32, 25%), and health care information inquiry (4/32, 13%). Other apps connected to physicians (5/32, 16%), family members (1/32, 3%), nursing staff (1/32, 3%), and health care departments (2/32, 6%), generally to alert these groups to abnormal conditions of consumer users. In addition, 8 barriers and 6 design recommendations related to DTC health care AI apps were identified. Some more subtle obstacles that are particularly worth noting and corresponding design recommendations in consumer-facing health care AI systems, including enhancing human-centered explainability, establishing calibrated trust and addressing overtrust, demonstrating empathy in AI, improving the specialization of consumer-grade products, and expanding the diversity of the test population, were further discussed. Conclusions: The booming DTC health care AI apps present both risks and opportunities, which highlights the need to explore their current status. This paper systematically summarized and sorted the characteristics of the included studies, identified existing barriers faced by, and made future design recommendations for such apps. To the best of our knowledge, this is the first study to systematically summarize and categorize academic research on these apps. Future studies conducting the design and development of such systems could refer to the results of this study, which is crucial to improve the health care services provided by DTC health care AI apps. %M 38109173 %R 10.2196/50342 %U https://www.jmir.org/2023/1/e50342 %U https://doi.org/10.2196/50342 %U http://www.ncbi.nlm.nih.gov/pubmed/38109173 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e51003 %T Use of Epic Electronic Health Record System for Health Care Research: Scoping Review %A Chishtie,Jawad %A Sapiro,Natalie %A Wiebe,Natalie %A Rabatach,Leora %A Lorenzetti,Diane %A Leung,Alexander A %A Rabi,Doreen %A Quan,Hude %A Eastwood,Cathy A %+ Center for Health Informatics, University of Calgary, 3280 Hospital Drive Northwest - 5E04, Calgary, AB, Canada, 1 403 210 9426, jac161@gmail.com %K electronic health record %K EHR %K Epic %K research %K health care %K electronic medical record %K EMR %K health system %D 2023 %7 15.12.2023 %9 Review %J J Med Internet Res %G English %X Background: Electronic health records (EHRs) enable health data exchange across interconnected systems from varied settings. Epic is among the 5 leading EHR providers and is the most adopted EHR system across the globe. Despite its global reach, there is a gap in the literature detailing how EHR systems such as Epic have been used for health care research. Objective: The objective of this scoping review is to synthesize the available literature on use cases of the Epic EHR for research in various areas of clinical and health sciences. Methods: We used established scoping review methods and searched 9 major information repositories, including databases and gray literature sources. To categorize the research data, we developed detailed criteria for 5 major research domains to present the results. Results: We present a comprehensive picture of the method types in 5 research domains. A total of 4669 articles were screened by 2 independent reviewers at each stage, while 206 articles were abstracted. Most studies were from the United States, with a sharp increase in volume from the year 2015 onwards. Most articles focused on clinical care, health services research and clinical decision support. Among research designs, most studies used longitudinal designs, followed by interventional studies implemented at single sites in adult populations. Important facilitators and barriers to the use of Epic and EHRs in general were identified. Important lessons to the use of Epic and other EHRs for research purposes were also synthesized. Conclusions: The Epic EHR provides a wide variety of functions that are helpful toward research in several domains, including clinical and population health, quality improvement, and the development of clinical decision support tools. As Epic is reported to be the most globally adopted EHR, researchers can take advantage of its various system features, including pooled data, integration of modules and developing decision support tools. Such research opportunities afforded by the system can contribute to improving quality of care, building health system efficiencies, and conducting population-level studies. Although this review is limited to the Epic EHR system, the larger lessons are generalizable to other EHRs. %M 38100185 %R 10.2196/51003 %U https://www.jmir.org/2023/1/e51003 %U https://doi.org/10.2196/51003 %U http://www.ncbi.nlm.nih.gov/pubmed/38100185 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e44599 %T Tensorial Principal Component Analysis in Detecting Temporal Trajectories of Purchase Patterns in Loyalty Card Data: Retrospective Cohort Study %A Autio,Reija %A Virta,Joni %A Nordhausen,Klaus %A Fogelholm,Mikael %A Erkkola,Maijaliisa %A Nevalainen,Jaakko %+ Faculty of Social Sciences (Health Sciences), Tampere University, P.O. Box 100, Tampere, FI-33014, Finland, 358 50 318 7364, reija.autio@tuni.fi %K tensorial data %K principal components %K loyalty card data %K purchase pattern %K food expenditure %K seasonality %K food %K diet %D 2023 %7 15.12.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Loyalty card data automatically collected by retailers provide an excellent source for evaluating health-related purchase behavior of customers. The data comprise information on every grocery purchase, including expenditures on product groups and the time of purchase for each customer. Such data where customers have an expenditure value for every product group for each time can be formulated as 3D tensorial data. Objective: This study aimed to use the modern tensorial principal component analysis (PCA) method to uncover the characteristics of health-related purchase patterns from loyalty card data. Another aim was to identify card holders with distinct purchase patterns. We also considered the interpretation, advantages, and challenges of tensorial PCA compared with standard PCA. Methods: Loyalty card program members from the largest retailer in Finland were invited to participate in this study. Our LoCard data consist of the purchases of 7251 card holders who consented to the use of their data from the year 2016. The purchases were reclassified into 55 product groups and aggregated across 52 weeks. The data were then analyzed using tensorial PCA, allowing us to effectively reduce the time and product group-wise dimensions simultaneously. The augmentation method was used for selecting the suitable number of principal components for the analysis. Results: Using tensorial PCA, we were able to systematically search for typical food purchasing patterns across time and product groups as well as detect different purchasing behaviors across groups of card holders. For example, we identified customers who purchased large amounts of meat products and separated them further into groups based on time profiles, that is, customers whose purchases of meat remained stable, increased, or decreased throughout the year or varied between seasons of the year. Conclusions: Using tensorial PCA, we can effectively examine customers’ purchasing behavior in more detail than with traditional methods because it can handle time and product group dimensions simultaneously. When interpreting the results, both time and product dimensions must be considered. In further analyses, these time and product groups can be directly associated with additional consumer characteristics such as socioeconomic and demographic predictors of dietary patterns. In addition, they can be linked to external factors that impact grocery purchases such as inflation and unexpected pandemics. This enables us to identify what types of people have specific purchasing patterns, which can help in the development of ways in which consumers can be steered toward making healthier food choices. %M 38100168 %R 10.2196/44599 %U https://www.jmir.org/2023/1/e44599 %U https://doi.org/10.2196/44599 %U http://www.ncbi.nlm.nih.gov/pubmed/38100168 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e44119 %T Performance Test of a Well-Trained Model for Meningioma Segmentation in Health Care Centers: Secondary Analysis Based on Four Retrospective Multicenter Data Sets %A Chen,Chaoyue %A Teng,Yuen %A Tan,Shuo %A Wang,Zizhou %A Zhang,Lei %A Xu,Jianguo %+ Neurosurgery Department, West China Hospital, Sichuan University, West China Hosptial, No 37, GuoXue Alley, Chengdu, 610041, China, 86 18980602049, drjianguoxu@gmail.com %K meningioma segmentation %K magnetic resonance imaging %K MRI %K convolutional neural network %K model test and verification %K CNN %K radiographic image interpretation %D 2023 %7 15.12.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Convolutional neural networks (CNNs) have produced state-of-the-art results in meningioma segmentation on magnetic resonance imaging (MRI). However, images obtained from different institutions, protocols, or scanners may show significant domain shift, leading to performance degradation and challenging model deployment in real clinical scenarios. Objective: This research aims to investigate the realistic performance of a well-trained meningioma segmentation model when deployed across different health care centers and verify the methods to enhance its generalization. Methods: This study was performed in four centers. A total of 606 patients with 606 MRIs were enrolled between January 2015 and December 2021. Manual segmentations, determined through consensus readings by neuroradiologists, were used as the ground truth mask. The model was previously trained using a standard supervised CNN called Deeplab V3+ and was deployed and tested separately in four health care centers. To determine the appropriate approach to mitigating the observed performance degradation, two methods were used: unsupervised domain adaptation and supervised retraining. Results: The trained model showed a state-of-the-art performance in tumor segmentation in two health care institutions, with a Dice ratio of 0.887 (SD 0.108, 95% CI 0.903-0.925) in center A and a Dice ratio of 0.874 (SD 0.800, 95% CI 0.854-0.894) in center B. Whereas in the other health care institutions, the performance declined, with Dice ratios of 0.631 (SD 0.157, 95% CI 0.556-0.707) in center C and 0.649 (SD 0.187, 95% CI 0.566-0.732) in center D, as they obtained the MRI using different scanning protocols. The unsupervised domain adaptation showed a significant improvement in performance scores, with Dice ratios of 0.842 (SD 0.073, 95% CI 0.820-0.864) in center C and 0.855 (SD 0.097, 95% CI 0.826-0.886) in center D. Nonetheless, it did not overperform the supervised retraining, which achieved Dice ratios of 0.899 (SD 0.026, 95% CI 0.889-0.906) in center C and 0.886 (SD 0.046, 95% CI 0.870-0.903) in center D. Conclusions: Deploying the trained CNN model in different health care institutions may show significant performance degradation due to the domain shift of MRIs. Under this circumstance, the use of unsupervised domain adaptation or supervised retraining should be considered, taking into account the balance between clinical requirements, model performance, and the size of the available data. %M 38100181 %R 10.2196/44119 %U https://www.jmir.org/2023/1/e44119 %U https://doi.org/10.2196/44119 %U http://www.ncbi.nlm.nih.gov/pubmed/38100181 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e49996 %T Toward an Interdisciplinary Approach to Constructing Care Delivery Pathways From Electronic Health Care Databases to Support Integrated Care in Chronic Conditions: Systematic Review of Quantification and Visualization Methods %A Siqueira do Prado,Luiza %A Allemann,Samuel %A Viprey,Marie %A Schott,Anne-Marie %A Dediu,Dan %A Dima,Alexandra Lelia %+ INSERM Unit U1290—Research on Healthcare Performance, University Claude Bernard Lyon 1, 8 Avenue Rockefeller, Lyon, 69373, France, 33 426688223, prado.luiza@gmail.com %K long-term care %K electronic health care databases %K patient pathway %K data visualization %K systematic review %D 2023 %7 14.12.2023 %9 Review %J J Med Internet Res %G English %X Background: Electronic health care databases are increasingly used for informing clinical decision-making. In long-term care, linking and accessing information on health care delivered by different providers could improve coordination and health outcomes. Several methods for quantifying and visualizing this information into data-driven care delivery pathways (CDPs) have been proposed. To be integrated effectively and sustainably into routine care, these methods need to meet a range of prerequisites covering 3 broad domains: clinical, technological, and behavioral. Although advances have been made, development to date lacks a comprehensive interdisciplinary approach. As the field expands, it would benefit from developing common standards of development and reporting that integrate clinical, technological, and behavioral aspects. Objective: We aimed to describe the content and development of long-term CDP quantification and visualization methods and to propose recommendations for future work. Methods: We conducted a systematic review following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) recommendations. We searched peer-reviewed publications in English and reported the CDP methods by using the following data in the included studies: long-term care data and extracted data on clinical information and aims, technological development and characteristics, and user behaviors. The data are summarized in tables and presented narratively. Results: Of the 2921 records identified, 14 studies were included, of which 13 (93%) were descriptive reports and 1 (7%) was a validation study. Clinical aims focused primarily on treatment decision-making (n=6, 43%) and care coordination (n=7, 50%). Technological development followed a similar process from scope definition to tool validation, with various levels of detail in reporting. User behaviors (n=3, 21%) referred to accessing CDPs, planning care, adjusting treatment, or supporting adherence. Conclusions: The use of electronic health care databases for quantifying and visualizing CDPs in long-term care is an emerging field. Detailed and standardized reporting of clinical and technological aspects is needed. Early consideration of how CDPs would be used, validated, and implemented in clinical practice would likely facilitate further development and adoption. Trial Registration: PROSPERO CRD42019140494; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=140494 International Registered Report Identifier (IRRID): RR2-10.1136/bmjopen-2019-033573 %M 38096009 %R 10.2196/49996 %U https://www.jmir.org/2023/1/e49996 %U https://doi.org/10.2196/49996 %U http://www.ncbi.nlm.nih.gov/pubmed/38096009 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e45238 %T Adoption of Electronic Health Record Among Substance Use Disorder Treatment Programs: Nationwide Cross-Sectional Survey Study %A Frimpong,Jemima A %A Liu,Xun %A Liu,Lingrui %A Zhang,Ruoqiuyan %+ New York University Abu Dhabi, Social Science Division, Abu Dhabi, 00000, United Arab Emirates, 971 2 628 8732, jafrimpong@nyu.edu %K adoption of technology %K barriers to adoption %K electronic health record %K health information technology %K opioid treatment programs %K substance use disorder %D 2023 %7 14.12.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Electronic health record (EHR) systems have been shown to be associated with improvements in care processes, quality of care, and patient outcomes. EHR also has a crucial role in the delivery of substance use disorder (SUD) treatment and is considered important for addressing SUD crises, including the opioid epidemic. However, little is known about the adoption of EHR in SUD treatment programs or the organizational-level factors associated with the adoption of EHR in SUD treatment. Objective: We examined the adoption of EHR in SUD programs, with a focus on changes in adoption from 2014 to 2017, and identified organizational-level factors associated with EHR adoption. Methods: We used data from the 2014 and 2017 National Drug Abuse Treatment System Surveys. Our analysis included 1027 SUD programs (531 in 2014 and 496 in 2017). We used chi-square and Mann-Whitney U tests for categorical and continuous variables, respectively, to assess changes in EHR adoption, technology use, program, and client characteristics. We also investigated differences in characteristics and barriers to adoption by EHR adoption status (adopted EHR vs had not adopted or were planning to adopt EHR). We then conducted multivariate logistic regressions to examine internal and external factors associated with EHR adoption. Results: The adoption of EHR increased significantly from 57.6% (306/531) in 2014 to 69.2% (343/496) in 2017 (P<.001), showing that nearly one-third (153/496, 30.8%) of SUD programs had not yet adopted an EHR system by 2017. We identified a significant increase in technology use and ownership by a parent company (P=.01 and P<.001) and a decrease in the percentage of uninsured patients in 2017 (P<.001), compared to 2014. Our analysis further showed significant differences by adoption status for three major barriers to adoption: (1) start-up costs, (2) ongoing financial costs, and (3) privacy or security concerns (P<.001). Programs that used computerized scheduling (adjusted odds ratio [AOR] 3.02, 95% CI 2.23-4.09) and billing systems (AOR 2.29, 95% CI 1.62-3.25) were more likely to adopt EHR. Similarly, ownership type, such as private nonprofit (AOR 1.86, 95% CI 1.31-2.65) and public (AOR 2.14, 95% CI 1.27-3.67), or interest in participating in a patient-centered medical home (AOR 1.93, 95% CI 1.29-2.92), were associated with an increased likelihood to adopt EHR. Overall, SUD programs were more likely to adopt an EHR system in 2017 compared to 2014 (AOR 1.44, 95% CI 1.07-1.94). Conclusions: Our findings highlighted that SUD programs may be on track to achieve widespread EHR adoption. However, there is a need for focused strategies, resources, and policies explicitly designed to systematically address barriers and tackle obstacles to expanding the adoption of EHR systems. These efforts must be holistic and address factors at multiple organizational levels. %M 38096006 %R 10.2196/45238 %U https://www.jmir.org/2023/1/e45238 %U https://doi.org/10.2196/45238 %U http://www.ncbi.nlm.nih.gov/pubmed/38096006 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e45979 %T A Machine Learning Algorithm Predicting Acute Kidney Injury in Intensive Care Unit Patients (NAVOY Acute Kidney Injury): Proof-of-Concept Study %A Persson,Inger %A Grünwald,Adam %A Morvan,Ludivine %A Becedas,David %A Arlbrandt,Martin %+ Department of Statistics, Uppsala University, Box 513, Uppsala, SE 751 20, Sweden, 46 738275861, inger.persson@statistik.uu.se %K acute kidney injury %K AKI %K algorithm %K early detection %K electronic health records %K ICU %K intensive care unit %K machine learning %K nephrology %K prediction %K software as a medical device %D 2023 %7 14.12.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Acute kidney injury (AKI) represents a significant global health challenge, leading to increased patient distress and financial health care burdens. The development of AKI in intensive care unit (ICU) settings is linked to prolonged ICU stays, a heightened risk of long-term renal dysfunction, and elevated short- and long-term mortality rates. The current diagnostic approach for AKI is based on late indicators, such as elevated serum creatinine and decreased urine output, which can only detect AKI after renal injury has transpired. There are no treatments to reverse or restore renal function once AKI has developed, other than supportive care. Early prediction of AKI enables proactive management and may improve patient outcomes. Objective: The primary aim was to develop a machine learning algorithm, NAVOY Acute Kidney Injury, capable of predicting the onset of AKI in ICU patients using data routinely collected in ICU electronic health records. The ultimate goal was to create a clinical decision support tool that empowers ICU clinicians to proactively manage AKI and, consequently, enhance patient outcomes. Methods: We developed the NAVOY Acute Kidney Injury algorithm using a hybrid ensemble model, which combines the strengths of both a Random Forest (Leo Breiman and Adele Cutler) and an XGBoost model (Tianqi Chen). To ensure the accuracy of predictions, the algorithm used 22 clinical variables for hourly predictions of AKI as defined by the Kidney Disease: Improving Global Outcomes guidelines. Data for algorithm development were sourced from the Massachusetts Institute of Technology Lab for Computational Physiology Medical Information Mart for Intensive Care IV clinical database, focusing on ICU patients aged 18 years or older. Results: The developed algorithm, NAVOY Acute Kidney Injury, uses 4 hours of input and can, with high accuracy, predict patients with a high risk of developing AKI 12 hours before onset. The prediction performance compares well with previously published prediction algorithms designed to predict AKI onset in accordance with Kidney Disease: Improving Global Outcomes diagnosis criteria, with an impressive area under the receiver operating characteristics curve (AUROC) of 0.91 and an area under the precision-recall curve (AUPRC) of 0.75. The algorithm’s predictive performance was externally validated on an independent hold-out test data set, confirming its ability to predict AKI with exceptional accuracy. Conclusions: NAVOY Acute Kidney Injury is an important development in the field of critical care medicine. It offers the ability to predict the onset of AKI with high accuracy using only 4 hours of data routinely collected in ICU electronic health records. This early detection capability has the potential to strengthen patient monitoring and management, ultimately leading to improved patient outcomes. Furthermore, NAVOY Acute Kidney Injury has been granted Conformite Europeenne (CE)–marking, marking a significant milestone as the first CE-marked AKI prediction algorithm for commercial use in European ICUs. %M 38096015 %R 10.2196/45979 %U https://formative.jmir.org/2023/1/e45979 %U https://doi.org/10.2196/45979 %U http://www.ncbi.nlm.nih.gov/pubmed/38096015 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e51578 %T Caries Detection in Primary Teeth Using Intraoral Scanners Featuring Fluorescence: Protocol for a Diagnostic Agreement Study %A Jones,Bree %A Michou,Stavroula %A Chen,Tong %A Moreno-Betancur,Margarita %A Kilpatrick,Nicky %A Burgner,David %A Vannahme,Christoph %A Silva,Mihiri %+ Murdoch Children's Research Institute, Royal Children's Hospital, 5 Densley Court, Darley, 3340, Australia, bree.jones@unimelb.edu.au %K dental caries %K diagnosis %K oral %K technology %K dental %K image interpretation %K computer-assisted %K imaging %K 3D %K quantitative light-induced fluorescence %K diagnostic agreement %K intra oral scanners %K oral health %K teeth %K 3D model %K color %K fluorescence %K intraoral scanner %K device %K dentistry %D 2023 %7 14.12.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Digital methods that enable early caries identification can streamline data collection in research and optimize dental examinations for young children. Intraoral scanners are devices used for creating 3D models of teeth in dentistry and are being rapidly adopted into clinical workflows. Integrating fluorescence technology into scanner hardware can support early caries detection. However, the performance of caries detection methods using 3D models featuring color and fluorescence in primary teeth is unknown. Objective: This study aims to assess the diagnostic agreement between visual examination (VE), on-screen assessment of 3D models in approximate natural colors with and without fluorescence, and application of an automated caries scoring system to the 3D models with fluorescence for caries detection in primary teeth. Methods: The study sample will be drawn from eligible participants in a randomized controlled trial at the Royal Children’s Hospital, Melbourne, Australia, where a dental assessment was conducted, including VE using the International Caries Detection and Assessment System (ICDAS) and intraoral scan using the TRIOS 4 (3Shape TRIOS A/S). Participant clinical records will be collected, and all records meeting eligibility criteria will be subject to an on-screen assessment of 3D models by 4 dental practitioners. First, all primary tooth surfaces will be examined for caries based on 3D geometry and color, using a merged ICDAS index. Second, the on-screen assessment of 3D models will include fluorescence, where caries will be classified using a merged ICDAS index that has been modified to incorporate fluorescence criteria. After 4 weeks, all examiners will repeat the on-screen assessment for all 3D models. Finally, an automated caries scoring system will be used to classify caries on primary occlusal surfaces. The agreement in the total number of caries detected per person between methods will be assessed using a Bland-Altman analysis and intraclass correlation coefficients. At a tooth surface level, agreement between methods will be estimated using multilevel models to account for the clustering of dental data. Results: Automated caries scoring of 3D models was completed as of October 2023, with the publication of results expected by July 2024. On-screen assessment has commenced, with the expected completion of scoring and data analysis by March 2024. Results will be disseminated by the end of 2024. Conclusions: The study outcomes may inform new practices that use digital models to facilitate dental assessments. Novel approaches that enable remote dental examination without compromising the accuracy of VE have wide applications in the research environment, clinical practice, and the provision of teledentistry. Trial Registration: Australian New Zealand Clinical Trials Registry ACTRN12622001237774; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=384632 International Registered Report Identifier (IRRID): DERR1-10.2196/51578 %M 38096003 %R 10.2196/51578 %U https://www.researchprotocols.org/2023/1/e51578 %U https://doi.org/10.2196/51578 %U http://www.ncbi.nlm.nih.gov/pubmed/38096003 %0 Journal Article %@ 2562-0959 %I JMIR Publications %V 6 %N %P e43821 %T The Reporting and Methodological Quality of Systematic Reviews Underpinning Clinical Practice Guidelines Focused on the Management of Cutaneous Melanoma: Cross-Sectional Analysis %A Khalid,Mahnoor %A Sutterfield,Bethany %A Minley,Kirstien %A Ottwell,Ryan %A Abercrombie,McKenna %A Heath,Christopher %A Torgerson,Trevor %A Hartwell,Micah %A Vassar,Matt %+ Office of Medical Student Research, Oklahoma State University Center for Health Sciences, 1111 W 17th St, Tulsa, OK, 74107, United States, 1 (918) 853 9938, mahnoor.khalid@okstate.edu %K clinical practice guidelines %K clinical %K cutaneous melanoma %K decision making %K evidence %K management %K melanoma %K practice guideline %K review %K systematic review %D 2023 %7 7.12.2023 %9 Original Paper %J JMIR Dermatol %G English %X Background: Clinical practice guidelines (CPGs) inform evidence-based decision-making in the clinical setting; however, systematic reviews (SRs) that inform these CPGs may vary in terms of reporting and methodological quality, which affects confidence in summary effect estimates. Objective: Our objective was to appraise the methodological and reporting quality of the SRs used in CPGs for cutaneous melanoma and evaluate differences in these outcomes between Cochrane and non-Cochrane reviews. Methods: We conducted a cross-sectional analysis by searching PubMed for cutaneous melanoma guidelines published between January 1, 2015, and May 21, 2021. Next, we extracted SRs composing these guidelines and appraised their reporting and methodological rigor using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and AMSTAR (A Measurement Tool to Assess Systematic Reviews) checklists. Lastly, we compared these outcomes between Cochrane and non-Cochrane SRs. All screening and data extraction occurred in a masked, duplicate fashion. Results: Of the SRs appraised, the mean completion rate was 66.5% (SD 12.29%) for the PRISMA checklist and 44.5% (SD 21.05%) for AMSTAR. The majority of SRs (19/50, 53%) were of critically low methodological quality, with no SRs being appraised as high quality. There was a statistically significant association (P<.001) between AMSTAR and PRISMA checklists. Cochrane SRs had higher PRISMA mean completion rates and higher methodological quality than non-Cochrane SRs. Conclusions: SRs supporting CPGs focused on the management of cutaneous melanoma vary in reporting and methodological quality, with the majority of SRs being of low quality. Increasing adherence to PRISMA and AMSTAR checklists will likely increase the quality of SRs, thereby increasing the level of evidence supporting cutaneous melanoma CPGs. %M 38060306 %R 10.2196/43821 %U https://derma.jmir.org/2023/1/e43821 %U https://doi.org/10.2196/43821 %U http://www.ncbi.nlm.nih.gov/pubmed/38060306 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e50027 %T Traceable Research Data Sharing in a German Medical Data Integration Center With FAIR (Findability, Accessibility, Interoperability, and Reusability)-Geared Provenance Implementation: Proof-of-Concept Study %A Gierend,Kerstin %A Waltemath,Dagmar %A Ganslandt,Thomas %A Siegel,Fabian %+ Department of Biomedical Informatics at the Center for Preventive Medicine and Digital Health, Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, Mannheim, 68167, Germany, 49 621383 ext 8087, kerstin.gierend@medma.uni-heidelberg.de %K provenance %K traceability %K data management %K metadata %K data integrity %K data integration center %K medical informatics %D 2023 %7 7.12.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Secondary investigations into digital health records, including electronic patient data from German medical data integration centers (DICs), pave the way for enhanced future patient care. However, only limited information is captured regarding the integrity, traceability, and quality of the (sensitive) data elements. This lack of detail diminishes trust in the validity of the collected data. From a technical standpoint, adhering to the widely accepted FAIR (Findability, Accessibility, Interoperability, and Reusability) principles for data stewardship necessitates enriching data with provenance-related metadata. Provenance offers insights into the readiness for the reuse of a data element and serves as a supplier of data governance. Objective: The primary goal of this study is to augment the reusability of clinical routine data within a medical DIC for secondary utilization in clinical research. Our aim is to establish provenance traces that underpin the status of data integrity, reliability, and consequently, trust in electronic health records, thereby enhancing the accountability of the medical DIC. We present the implementation of a proof-of-concept provenance library integrating international standards as an initial step. Methods: We adhered to a customized road map for a provenance framework, and examined the data integration steps across the ETL (extract, transform, and load) phases. Following a maturity model, we derived requirements for a provenance library. Using this research approach, we formulated a provenance model with associated metadata and implemented a proof-of-concept provenance class. Furthermore, we seamlessly incorporated the internationally recognized Word Wide Web Consortium (W3C) provenance standard, aligned the resultant provenance records with the interoperable health care standard Fast Healthcare Interoperability Resources, and presented them in various representation formats. Ultimately, we conducted a thorough assessment of provenance trace measurements. Results: This study marks the inaugural implementation of integrated provenance traces at the data element level within a German medical DIC. We devised and executed a practical method that synergizes the robustness of quality- and health standard–guided (meta)data management practices. Our measurements indicate commendable pipeline execution times, attaining notable levels of accuracy and reliability in processing clinical routine data, thereby ensuring accountability in the medical DIC. These findings should inspire the development of additional tools aimed at providing evidence-based and reliable electronic health record services for secondary use. Conclusions: The research method outlined for the proof-of-concept provenance class has been crafted to promote effective and reliable core data management practices. It aims to enhance biomedical data by imbuing it with meaningful provenance, thereby bolstering the benefits for both research and society. Additionally, it facilitates the streamlined reuse of biomedical data. As a result, the system mitigates risks, as data analysis without knowledge of the origin and quality of all data elements is rendered futile. While the approach was initially developed for the medical DIC use case, these principles can be universally applied throughout the scientific domain. %M 38060305 %R 10.2196/50027 %U https://formative.jmir.org/2023/1/e50027 %U https://doi.org/10.2196/50027 %U http://www.ncbi.nlm.nih.gov/pubmed/38060305 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e48145 %T OpenDeID Pipeline for Unstructured Electronic Health Record Text Notes Based on Rules and Transformers: Deidentification Algorithm Development and Validation Study %A Liu,Jiaxing %A Gupta,Shalini %A Chen,Aipeng %A Wang,Chen-Kai %A Mishra,Pratik %A Dai,Hong-Jie %A Wong,Zoie Shui-Yee %A Jonnagaddala,Jitendra %+ School of Population Health, UNSW Sydney, F25 Samuels Building, Samuel Terry Ave, Kensington, NSW, 2033, Australia, 61 (02) 9385 2517, z3339253@unsw.edu.au %K deidentification %K scrubbing %K anonymization %K surrogate generation %K unstructured EHRs %K electronic health records %K BERT %K Bidirectional Encoder Representations from Transformers %D 2023 %7 6.12.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Electronic health records (EHRs) in unstructured formats are valuable sources of information for research in both the clinical and biomedical domains. However, before such records can be used for research purposes, sensitive health information (SHI) must be removed in several cases to protect patient privacy. Rule-based and machine learning–based methods have been shown to be effective in deidentification. However, very few studies investigated the combination of transformer-based language models and rules. Objective: The objective of this study is to develop a hybrid deidentification pipeline for Australian EHR text notes using rules and transformers. The study also aims to investigate the impact of pretrained word embedding and transformer-based language models. Methods: In this study, we present a hybrid deidentification pipeline called OpenDeID, which is developed using an Australian multicenter EHR-based corpus called OpenDeID Corpus. The OpenDeID corpus consists of 2100 pathology reports with 38,414 SHI entities from 1833 patients. The OpenDeID pipeline incorporates a hybrid approach of associative rules, supervised deep learning, and pretrained language models. Results: The OpenDeID achieved a best F1-score of 0.9659 by fine-tuning the Discharge Summary BioBERT model and incorporating various preprocessing and postprocessing rules. The OpenDeID pipeline has been deployed at a large tertiary teaching hospital and has processed over 8000 unstructured EHR text notes in real time. Conclusions: The OpenDeID pipeline is a hybrid deidentification pipeline to deidentify SHI entities in unstructured EHR text notes. The pipeline has been evaluated on a large multicenter corpus. External validation will be undertaken as part of our future work to evaluate the effectiveness of the OpenDeID pipeline. %M 38055317 %R 10.2196/48145 %U https://www.jmir.org/2023/1/e48145 %U https://doi.org/10.2196/48145 %U http://www.ncbi.nlm.nih.gov/pubmed/38055317 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e49374 %T Investigating the Impact of Automation on the Health Care Workforce Through Autonomous Telemedicine in the Cataract Pathway: Protocol for a Multicenter Study %A Khavandi,Sarah %A Zaghloul,Fatema %A Higham,Aisling %A Lim,Ernest %A de Pennington,Nick %A Celi,Leo Anthony %+ Ufonia, 104 Gloucester Green, Oxford, OX1 2BU, United Kingdom, 44 07931531022, sk@ufonia.co %K artificial intelligence %K autonomous telemedicine %K clinician burnout %K clinician wellbeing %K conversational agent %K digital health %K health communication %K health information technology %K health services %K healthcare %K medical informatics %K socio-technical system approach %K systems approach %K technology acceptability %D 2023 %7 5.12.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: While digital health innovations are increasingly being adopted by health care organizations, implementation is often carried out without considering the impacts on frontline staff who will be using the technology and who will be affected by its introduction. The enthusiasm surrounding the use of artificial intelligence (AI)–enabled digital solutions in health care is tempered by uncertainty around how it will change the working lives and practices of health care professionals. Digital enablement can be viewed as facilitating enhanced effectiveness and efficiency by improving services and automating cognitive labor, yet the implementation of such AI technology comes with challenges related to changes in work practices brought by automation. This research explores staff experiences before and after care pathway automation with an autonomous clinical conversational assistant, Dora (Ufonia Ltd), that is able to automate routine clinical conversations. Objective: The primary objective is to examine the impact of AI-enabled automation on clinicians, allied health professionals, and administrators who provide or facilitate health care to patients in high-volume, low-complexity care pathways. In the process of transforming care pathways through automation of routine tasks, staff will increasingly “work at the top of their license.” The impact of this fundamental change on the professional identity, well-being, and work practices of the individual is poorly understood at present. Methods: We will adopt a multiple case study approach, combining qualitative and quantitative data collection methods, over 2 distinct phases, namely phase A (preimplementation) and phase B (postimplementation). Results: The analysis is expected to reveal the interrelationship between Dora and those affected by its introduction. This will reveal how tasks and responsibilities have changed or shifted, current tensions and contradictions, ways of working, and challenges, benefits, and opportunities as perceived by those on the frontlines of the health care system. The findings will enable a better understanding of the resistance or susceptibility of different stakeholders within the health care workforce and encourage managerial awareness of differing needs, demands, and uncertainties. Conclusions: The implementation of AI in the health care sector, as well as the body of research on this topic, remain in their infancy. The project’s key contribution will be to understand the impact of AI-enabled automation on the health care workforce and their work practices. International Registered Report Identifier (IRRID): PRR1-10.2196/49374 %M 38051569 %R 10.2196/49374 %U https://www.researchprotocols.org/2023/1/e49374 %U https://doi.org/10.2196/49374 %U http://www.ncbi.nlm.nih.gov/pubmed/38051569 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e51387 %T Exploring Whether the Electronic Optimization of Routine Health Assessments Can Increase Testing for Sexually Transmitted Infections and Provider Acceptability at an Aboriginal Community Controlled Health Service: Mixed Methods Evaluation %A McCormack,Heather %A Wand,Handan %A Newman,Christy E %A Bourne,Christopher %A Kennedy,Catherine %A Guy,Rebecca %+ Kirby Institute, University of New South Wales, Wallace Wurth Building (C27), High St, Kensington, 2052, Australia, 61 93481086, hmccormack@kirby.unsw.edu.au %K sexual health %K sexually transmitted infection %K STI %K primary care %K Indigenous health %K electronic medical record %K EMR %K medical records %K electronic health record %K EHR %K health record %K health records %K Indigenous %K Native %K Aboriginal %K sexual transmission %K sexually transmitted %K time series %K testing %K uptake %K acceptance %K acceptability %K adoption %K syphilis %K sexually transmitted disease %K STD %K systems change %K health assessment %K health assessments %K prompt %K prompts %K implementation %K youth %K young people %K adolescent %K adolescents %D 2023 %7 30.11.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: In the context of a syphilis outbreak in neighboring states, a multifaceted systems change to increase testing for sexually transmitted infections (STIs) among young Aboriginal people aged 15 to 29 years was implemented at an Aboriginal Community Controlled Health Service (ACCHS) in New South Wales, Australia. The components included electronic medical record prompts and automated pathology test sets to increase STI testing in annual routine health assessments, the credentialing of nurses and Aboriginal health practitioners to conduct STI tests independently, pathology request forms presigned by a physician, and improved data reporting. Objective: We aimed to determine whether the systems change increased the integration of STI testing into routine health assessments by clinicians between April 2019 and March 2020, the inclusion of syphilis tests in STI testing, and STI testing uptake overall. We also explored the understandings of factors contributing to the acceptability and normalization of the systems change among staff. Methods: We used a mixed methods design to evaluate the effectiveness and acceptability of the systems change implemented in 2019. We calculated the annual proportion of health assessments that included tests for chlamydia, gonorrhea, and syphilis, as well as an internal control (blood glucose level). We conducted an interrupted time series analysis of quarterly proportions 24 months before and 12 months after the systems change and in-depth semistructured interviews with ACCHS staff using normalization process theory. Results: Among 2461 patients, the annual proportion of health assessments that included any STI test increased from 16% (38/237) in the first year of the study period to 42.9% (94/219) after the implementation of the systems change. There was an immediate and large increase when the systems change occurred (coefficient=0.22; P=.003) with no decline for 12 months thereafter. The increase was greater for male individuals, with no change for the internal control. Qualitative data indicated that nurse- and Aboriginal health practitioner–led testing and presigned pathology forms proved more difficult to normalize than electronic prompts and shortcuts. The interviews identified that staff understood the modifications to have encouraged cultural change around the role of sexual health care in routine practice. Conclusions: This study provides evidence for the first time that optimizing health assessments electronically is an effective and acceptable strategy to increase and sustain clinician integration and the completeness of STI testing among young Aboriginal people attending an ACCHS. Future strategies should focus on increasing the uptake of health assessments and promote whole-of-service engagement and accountability. %M 38032729 %R 10.2196/51387 %U https://medinform.jmir.org/2023/1/e51387 %U https://doi.org/10.2196/51387 %U http://www.ncbi.nlm.nih.gov/pubmed/38032729 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e47145 %T Patient and Physician Perspectives on the Use of a Connected Ecosystem for Diabetes Management: International Cross-Sectional Observational Study %A Benito-Garcia,Elizabeth %A Vega,Julio %A Daza,Eric J %A Lee,Wei-Nchih %A Kennedy,Adee %A Chantelot,Jean-Marc %+ Sanofi, 54 Rue La Boétie, Paris, 75008, France, 33 0153774000, Jean-Marc.Chantelot@sanofi.com %K type 2 diabetes mellitus %K insulin treatment %K connected ecosystems %K surveys %K diabetes %K diabetic %K ecosystem %K ecosystems %K telehealth %K telemedicine %K eHealth %K digital health %K health technology %K adoption %K perception %K attitude %K intention %K acceptance %D 2023 %7 30.11.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Collaboration between people with type 2 diabetes (T2DM) and their health care teams is important for optimal control of the disease and outcomes. Digital technologies could potentially tie together several health care-related devices and platforms into connected ecosystems (CES), but attitudes about CES are unknown. Objective: We surveyed convenience samples of patients and physicians to better understand which patient characteristics are associated with higher likelihoods of (1) participating in a potential CES program, as self-reported by patients with T2DM and (2) clinical benefit from participation in a potential CES program, as reported by physicians. Methods: Adults self-reporting a diagnosis of T2DM and current insulin use (n=197), and 33 physicians whose practices included ≥20% of such patients, were enrolled in the United States, France, and Germany. We surveyed both groups about the likelihood of patient participation in a CES. We then examined the associations between patients’ clinical and sociodemographic characteristics and this likelihood. We also described characteristics of patients likely to clinically benefit from CES use, according to physicians. Results: Compared with patients in Germany and France, US patients were younger (mean age 45.3 [SD 11.9] years vs 61.9 [SD 9.2] and 65.8 [SD 9.4] years, respectively), more often female, more highly educated, and more often working full-time. In all, 51 (44.7%) US patients, 16 (36.4%) German patients, and 18 (46.3%) French patients indicated strong interest in a CES program, and 115 (78.7%) reported currently using ≥1 connected device or app. However, physicians believed that only 11.3%-19.2% of their patients were using connected devices or apps to manage their disease. Physicians also reported infrequently recommending or prescribing connected devices to their patients, although ≥80% (n=28) of them thought that a CES could help support their patients in managing their disease. The factors most predictive of patient likelihood of participating in a CES program were cost, inclusion of medication reminders, and linking blood glucose levels to behaviors such as eating and exercise. In all countries, the most common patient expectations for a CES program were that it could help them eat more healthfully, increase their physical activity, increase their understanding of how blood glucose relates to behavior such as exercise and eating, and reduce stress. Physicians thought that newly diagnosed patients, sicker patients—those who had been hospitalized for diabetes, were currently using insulin, or who had any comorbid condition—and patients who were nonadherent to treatment were most likely to benefit from CES use. Conclusions: In this study, there was a high degree of interest in the future use of CES, although additional education is needed among both patients with T2DM and their physicians to achieve the full potential of such systems to improve self-management and clinical care for the disease. %M 38032701 %R 10.2196/47145 %U https://formative.jmir.org/2023/1/e47145 %U https://doi.org/10.2196/47145 %U http://www.ncbi.nlm.nih.gov/pubmed/38032701 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e44773 %T Standardized Comparison of Voice-Based Information and Documentation Systems to Established Systems in Intensive Care: Crossover Study %A Peine,Arne %A Gronholz,Maike %A Seidl-Rathkopf,Katharina %A Wolfram,Thomas %A Hallawa,Ahmed %A Reitz,Annika %A Celi,Leo Anthony %A Marx,Gernot %A Martin,Lukas %+ Department of Intensive Care Medicine and Intermediate Care, University Hospital RWTH Aachen, Pauwelsstr. 30, Aachen, 52070, Germany, 49 241 800, apeine@ukaachen.de %K artificial intelligence %K documentation %K ICU %K intensive care medicine %K speech-recognition %K user perception %K workload %D 2023 %7 28.11.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: The medical teams in intensive care units (ICUs) spend increasing amounts of time at computer systems for data processing, input, and interpretation purposes. As each patient creates about 1000 data points per hour, the available information is abundant, making the interpretation difficult and time-consuming. This data flood leads to a decrease in time for evidence-based, patient-centered care. Information systems, such as patient data management systems (PDMSs), are increasingly used at ICUs. However, they often create new challenges arising from the increasing documentation burden. Objective: New concepts, such as artificial intelligence (AI)–based assistant systems, are hence introduced to the workflow to cope with these challenges. However, there is a lack of standardized, published metrics in order to compare the various data input and management systems in the ICU setting. The objective of this study is to compare established documentation and retrieval processes with newer methods, such as PDMSs and voice information and documentation systems (VIDSs). Methods: In this crossover study, we compare traditional, paper-based documentation systems with PDMSs and newer AI-based VIDSs in terms of performance (required time), accuracy, mental workload, and user experience in an intensive care setting. Performance is assessed on a set of 6 standardized, typical ICU tasks, ranging from documentation to medical interpretation. Results: A total of 60 ICU-experienced medical professionals participated in the study. The VIDS showed a statistically significant advantage compared to the other 2 systems. The tasks were completed significantly faster with the VIDS than with the PDMS (1-tailed t59=12.48; Cohen d=1.61; P<.001) or paper documentation (t59=20.41; Cohen d=2.63; P<.001). Significantly fewer errors were made with VIDS than with the PDMS (t59=3.45; Cohen d=0.45; P=.03) and paper-based documentation (t59=11.2; Cohen d=1.45; P<.001). The analysis of the mental workload of VIDS and PDMS showed no statistically significant difference (P=.06). However, the analysis of subjective user perception showed a statistically significant perceived benefit of the VIDS compared to the PDMS (P<.001) and paper documentation (P<.001). Conclusions: The results of this study show that the VIDS reduced error rate, documentation time, and mental workload regarding the set of 6 standardized typical ICU tasks. In conclusion, this indicates that AI-based systems such as the VIDS tested in this study have the potential to reduce this workload and improve evidence-based and safe patient care. %M 38015593 %R 10.2196/44773 %U https://medinform.jmir.org/2023/1/e44773 %U https://doi.org/10.2196/44773 %U http://www.ncbi.nlm.nih.gov/pubmed/38015593 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e44639 %T Patient Information Summarization in Clinical Settings: Scoping Review %A Keszthelyi,Daniel %A Gaudet-Blavignac,Christophe %A Bjelogrlic,Mina %A Lovis,Christian %+ Division of Medical Information Sciences, University Hospitals of Geneva, Rue Gabrielle-Perret-Gentil 4, Geneva, 1205, Switzerland, 41 223726201, Daniel.Keszthelyi@unige.ch %K summarization %K electronic health records %K EHR %K medical record %K visualization %K dashboard %K natural language processing %D 2023 %7 28.11.2023 %9 Review %J JMIR Med Inform %G English %X Background: Information overflow, a common problem in the present clinical environment, can be mitigated by summarizing clinical data. Although there are several solutions for clinical summarization, there is a lack of a complete overview of the research relevant to this field. Objective: This study aims to identify state-of-the-art solutions for clinical summarization, to analyze their capabilities, and to identify their properties. Methods: A scoping review of articles published between 2005 and 2022 was conducted. With a clinical focus, PubMed and Web of Science were queried to find an initial set of reports, later extended by articles found through a chain of citations. The included reports were analyzed to answer the questions of where, what, and how medical information is summarized; whether summarization conserves temporality, uncertainty, and medical pertinence; and how the propositions are evaluated and deployed. To answer how information is summarized, methods were compared through a new framework “collect—synthesize—communicate” referring to information gathering from data, its synthesis, and communication to the end user. Results: Overall, 128 articles were included, representing various medical fields. Exclusively structured data were used as input in 46.1% (59/128) of papers, text in 41.4% (53/128) of articles, and both in 10.2% (13/128) of papers. Using the proposed framework, 42.2% (54/128) of the records contributed to information collection, 27.3% (35/128) contributed to information synthesis, and 46.1% (59/128) presented solutions for summary communication. Numerous summarization approaches have been presented, including extractive (n=13) and abstractive summarization (n=19); topic modeling (n=5); summary specification (n=11); concept and relation extraction (n=30); visual design considerations (n=59); and complete pipelines (n=7) using information extraction, synthesis, and communication. Graphical displays (n=53), short texts (n=41), static reports (n=7), and problem-oriented views (n=7) were the most common types in terms of summary communication. Although temporality and uncertainty information were usually not conserved in most studies (74/128, 57.8% and 113/128, 88.3%, respectively), some studies presented solutions to treat this information. Overall, 115 (89.8%) articles showed results of an evaluation, and methods included evaluations with human participants (median 15, IQR 24 participants): measurements in experiments with human participants (n=31), real situations (n=8), and usability studies (n=28). Methods without human involvement included intrinsic evaluation (n=24), performance on a proxy (n=10), or domain-specific tasks (n=11). Overall, 11 (8.6%) reports described a system deployed in clinical settings. Conclusions: The scientific literature contains many propositions for summarizing patient information but reports very few comparisons of these proposals. This work proposes to compare these algorithms through how they conserve essential aspects of clinical information and through the “collect—synthesize—communicate” framework. We found that current propositions usually address these 3 steps only partially. Moreover, they conserve and use temporality, uncertainty, and pertinent medical aspects to varying extents, and solutions are often preliminary. %M 38015588 %R 10.2196/44639 %U https://medinform.jmir.org/2023/1/e44639 %U https://doi.org/10.2196/44639 %U http://www.ncbi.nlm.nih.gov/pubmed/38015588 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e49358 %T Learnings in Digital Health Design: Insights From a Pilot Web App for Structured Note-Taking for Patients With Rheumatoid Arthritis %A Srivastava,Ujwal %A Dasari,Shobha %A Shah,Neha %+ Department of Computer Science, Stanford University, 900 Blake Wilbur Dr Rm W2081 2nd Fl, Stanford, CA, 94305, United States, 1 (650) 723 6961, ujwal@stanford.edu %K digital health %K biodesign %K technology %K software %K web app %K codesign %K patient empowerment %K note-taking %K medical information %K web application %K web-based %K technology engagement %D 2023 %7 28.11.2023 %9 Viewpoint %J JMIR Form Res %G English %X Background: Patients fail to accurately remember 40% to 80% of medical information relayed during doctor appointments, and most standard after-visit summaries fail to effectively help patients comply with behaviors to manage their health conditions. The value of technology to empower and engage patients in their health management has been shown, and here we apply technology to help patients remember and act upon information communicated during their medical appointments. Objective: We describe the development of WellNote, a digital notebook designed for patients to create a customized plan to manage their condition, plan for their appointments, track important actions (eg, medications and labs), and receive reminders for appointments and labs. Methods: For this pilot, we chose to focus on rheumatoid arthritis, a chronic condition that relies on many of these features. The development of WellNote followed a structured method based on design thinking and co-design principles, with the app built in close collaboration with patients and a physician partner to ensure clinical relevance. Our design process consisted of 3 rounds: patient and physician interviews, visual prototypes, and a functional pilot app. Results: Over the course of the design process, WellNote’s features were refined, with the final version being a digital notebook designed for patients with rheumatoid arthritis to manage their health by helping them track medications and labs and plan for appointments. It features several pages, like a dashboard, patient profile, appointment notes, preplanning, medication management, lab tracking, appointment archives, reminders, and a pillbox for medication visualization. Conclusions: WellNote’s active and structured note-taking features allow patients to clearly document the information from their physician without detracting from the conversation, helping the patient to become more empowered and engaged in their health management. The co-design process empowered these stakeholders to share their needs and participate in the development of a solution that truly solves pain points for these groups. This viewpoint highlights the role of digital health tools and the co-design of new health care innovations to empower patients and support clinicians. %M 38015609 %R 10.2196/49358 %U https://formative.jmir.org/2023/1/e49358 %U https://doi.org/10.2196/49358 %U http://www.ncbi.nlm.nih.gov/pubmed/38015609 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e49886 %T A Large Language Model Screening Tool to Target Patients for Best Practice Alerts: Development and Validation %A Savage,Thomas %A Wang,John %A Shieh,Lisa %+ Division of Hospital Medicine, Department of Medicine, Stanford University, 300 Pasteur Drive, Palo Alto, CA, 94304, United States, 1 6507234000, tsavage@stanford.edu %K large language models %K language models %K language model %K EHR %K health record %K health records %K quality improvement %K Artificial Intelligence %K Natural Language Processing %D 2023 %7 27.11.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: Best Practice Alerts (BPAs) are alert messages to physicians in the electronic health record that are used to encourage appropriate use of health care resources. While these alerts are helpful in both improving care and reducing costs, BPAs are often broadly applied nonselectively across entire patient populations. The development of large language models (LLMs) provides an opportunity to selectively identify patients for BPAs. Objective: In this paper, we present an example case where an LLM screening tool is used to select patients appropriate for a BPA encouraging the prescription of deep vein thrombosis (DVT) anticoagulation prophylaxis. The artificial intelligence (AI) screening tool was developed to identify patients experiencing acute bleeding and exclude them from receiving a DVT prophylaxis BPA. Methods: Our AI screening tool used a BioMed-RoBERTa (Robustly Optimized Bidirectional Encoder Representations from Transformers Pretraining Approach; AllenAI) model to perform classification of physician notes, identifying patients without active bleeding and thus appropriate for a thromboembolism prophylaxis BPA. The BioMed-RoBERTa model was fine-tuned using 500 history and physical notes of patients from the MIMIC-III (Medical Information Mart for Intensive Care) database who were not prescribed anticoagulation. A development set of 300 MIMIC patient notes was used to determine the model’s hyperparameters, and a separate test set of 300 patient notes was used to evaluate the screening tool. Results: Our MIMIC-III test set population of 300 patients included 72 patients with bleeding (ie, were not appropriate for a DVT prophylaxis BPA) and 228 without bleeding who were appropriate for a DVT prophylaxis BPA. The AI screening tool achieved impressive accuracy with a precision-recall area under the curve of 0.82 (95% CI 0.75-0.89) and a receiver operator curve area under the curve of 0.89 (95% CI 0.84-0.94). The screening tool reduced the number of patients who would trigger an alert by 20% (240 instead of 300 alerts) and increased alert applicability by 14.8% (218 [90.8%] positive alerts from 240 total alerts instead of 228 [76%] positive alerts from 300 total alerts), compared to nonselectively sending alerts for all patients. Conclusions: These results show a proof of concept on how language models can be used as a screening tool for BPAs. We provide an example AI screening tool that uses a HIPAA (Health Insurance Portability and Accountability Act)–compliant BioMed-RoBERTa model deployed with minimal computing power. Larger models (eg, Generative Pre-trained Transformers–3, Generative Pre-trained Transformers–4, and Pathways Language Model) will exhibit superior performance but require data use agreements to be HIPAA compliant. We anticipate LLMs to revolutionize quality improvement in hospital medicine. %M 38010803 %R 10.2196/49886 %U https://medinform.jmir.org/2023/1/e49886 %U https://doi.org/10.2196/49886 %U http://www.ncbi.nlm.nih.gov/pubmed/38010803 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e46835 %T Exploring Current Practices, Needs, and Barriers for Expanding Distributed Medical Education and Scholarship in Psychiatry: Protocol for an Environmental Scan Using a Formal Information Search Approach and Explanatory Design %A Hazelton,Lara %A da Luz Dias,Raquel %A Esliger,Mandy %A Tibbo,Philip %A Sinha,Nachiketa %A Njoku,Anthony %A Satyanarayana,Satyendra %A Siddhartha,Sanjay %A Alexiadis-Brown,Peggy %A Rahman,Faisal %A Maguire,Hugh %A Gray,Gerald %A Bosma,Mark %A Parker,Deborah %A Connolly,Owen %A Raji,Adewale %A Manning,Alexandra %A Bagnell,Alexa %A Israel Opoku Agyapong,Vincent %+ Department of Psychiatry, Faculty of Medicine, Dalhousie University, 5909 Veterans' Memorial Lane, 8th Floor Abbie J. Lane Memorial Building QEII Health Sciences Centre, Halifax, NS, B3H 2E2, Canada, 1 902 473 6214, vn602367@dal.ca %K distributed learning sites %K medical education %K psychiatry %K environmental scan %K needs assessment %K strategic plan %K distributed medical education %K rural area %K physician %K mixed methods approach %K education program %D 2023 %7 27.11.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Distributed medical education (DME) offers manifold benefits, such as increased training capacity, enhanced clinical learning, and enhanced rural physician recruitment. Engaged faculty are pivotal to DME's success, necessitating efforts from the academic department to promote integration into scholarly and research activities. Environmental scanning has been used to gather, analyze, and apply information for strategic planning purposes. It helps organizations identify current practices, assess needs and barriers, and respond to emerging risks and opportunities. There are process models and conceptual frameworks developed for environmental scanning in the business and educational sectors. However, the literature lacks methodological direction on how to go about designing and implementing this strategy to guide research and practice in DME, especially in the psychiatry field. Objective: This paper presents a protocol for an environmental scanning that aims to understand current practices and identify needs and barriers that must be addressed to facilitate the integration of psychiatrists from the Dalhousie University Faculty of Medicine’s distributed education sites in Nova Scotia and New Brunswick into the Department of Psychiatry, contributing for the expansion of DME in both provinces and informing strategic planning and decision-making within the organization. Methods: This protocol adopts an innovative approach combining a formal information search and an explanatory design that includes quantitative and qualitative data. About 120 psychiatrists from 8 administrative health zones of both provinces will be invited to complete an anonymous web-based survey with questions about demographics, participants' experience and interest in undergraduate, postgraduate, and continuing medical education, research and scholarly activities, quality improvement, and knowledge translation. Focus group sessions will be conducted with a purposive sample of psychiatrists to collect qualitative data on their perspectives on the expansion of DME. Results: Results are expected within 6 months of data collection and will inform policy options for expanding Dalhousie University’s psychiatry residency and fellowship programs using the infrastructure and human resources at distributed learning sites, leveraging opportunities regionally, especially in rural areas. Conclusions: This paper proposes a comprehensive environmental scan procedure adapted from existing approaches. It does this by collecting important characteristics that affect psychiatrists' desire to be involved with research and scholarly activities, which is crucial for the DME expansion. Furthermore, its concordance with the literature facilitates interpretation and comparison. The protocol's new method also fills DME information gaps, allowing one to identify insights and patterns that may shape psychiatric education. This environmental scan's results will answer essential questions about how training programs could involve therapists outside the academic core and make the most of training experiences in semiurban and rural areas. This could help other psychiatry and medical units outside tertiary care establish residency and fellowship programs. International Registered Report Identifier (IRRID): DERR1-10.2196/46835 %M 38010790 %R 10.2196/46835 %U https://www.researchprotocols.org/2023/1/e46835 %U https://doi.org/10.2196/46835 %U http://www.ncbi.nlm.nih.gov/pubmed/38010790 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e47505 %T Innovation Process and Industrial System of US Food and Drug Administration–Approved Software as a Medical Device: Review and Content Analysis %A Yu,Jiakan %A Zhang,Jiajie %A Sengoku,Shintaro %+ Department of Innovation Science, School of Environment and Society, Tokyo Institute of Technology, Tokyo Campus Innovation Center 9th Floor Room 908N, 3-3-6 Shibaura, Minato-ku, Tokyo, 108-0023, Japan, 81 03 3454 8907, sengoku.s.aa@m.titech.ac.jp %K digital health %K digital therapeutics %K software as a medical device %K innovation process %K artificial intelligence %D 2023 %7 24.11.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: There has been a surge in academic and business interest in software as a medical device (SaMD). SaMD enables medical professionals to streamline existing medical practices and make innovative medical processes such as digital therapeutics a reality. Furthermore, SaMD is a billion-dollar market. However, SaMD is not clearly understood as a technological change and emerging industry. Objective: This study aims to review the landscape of SaMD in response to increasing interest in SaMD within health systems and regulation. The objectives of the study are to (1) clarify the innovation process of SaMD, (2) identify the prevailing typology of such innovation, and (3) elucidate the underlying mechanisms driving the SaMD innovation process. Methods: We collected product information on 581 US Food and Drug Administration–approved SaMDs from the OpenFDA website and 268 company profiles of the corresponding manufacturers from Crunchbase, Bloomberg, PichBook.com, and other company websites. In addition to assessing the metadata of SaMD, we used correspondence and business process analysis to assess the distribution of intended use and how SaMDs interact with other devices in the medical process. Results: The current SaMD industry is highly concentrated in medical image processing and radiological analysis. Incumbents in the medical device industry currently lead the market and focus on incremental innovation, whereas new entrants, particularly startups, produce more disruptive innovation. We found that hardware medical device functions as a complementary asset for SaMD, whereas how SaMD interacts with the complementary asset differs according to its intended use. Based on these findings, we propose a regime map that illustrates the SaMD innovation process. Conclusions: SaMD, as an industry, is nascent and dominated by incremental innovation. The innovation process of the present SaMD industry is shaped by data accessibility, which is key to building disruptive innovation. %M 37999948 %R 10.2196/47505 %U https://www.jmir.org/2023/1/e47505 %U https://doi.org/10.2196/47505 %U http://www.ncbi.nlm.nih.gov/pubmed/37999948 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e47874 %T Circular Business Model for Digital Health Solutions: Protocol for a Scoping Review %A Rønn,Camille %A Wieland,Andreas %A Lehrer,Christiane %A Márton,Attila %A LaRoche,Jason %A Specker,Adrien %A Leroy,Pascal %A Fürstenau,Daniel %+ Department of Business IT, IT University of Copenhagen, Rued Langgaards Vej 7, Copenhagen, 2300, Denmark, 45 72185198, daniel.fuerstenau@itu.dk %K business model %K circular economy %K digital health solution %K digital health %K digital tool %K digital %K healthcare %K life cycle %K MedTech device %K monitoring device %K technology %D 2023 %7 24.11.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: The circular economy reshapes the linear “take, make, and dispose” approach and evolves around minimizing waste and recapturing resources in a closed-loop system. The health sector accounts for 4.6% of global greenhouse gas emissions and has, over the decades, been built to rely on single-use devices and deal with high volumes of medical waste. With the increase in the adoption of digital health solutions in the health care industry, leading the industry into a new paradigm of how we provide health care, a focus must be put on the amount of waste that will follow. Digital health solutions will shape health care through the use of technology and lead to improved patient care, but they will also make medical waste more complex to deal with due to the e-waste component. Therefore, a transformation of the health care industry to a circular economy is a crucial cornerstone in decreasing the impact on the environment. Objective: This study aims to address the lack of direction in the current literature on circular business models. It will consider micro, meso, and macro factors that would impact the operational validity of circular models using the digital health solutions ePaper label (medical packaging), smart wearable sensor (health monitoring devices), smart pill box (medication management), and endo-cutter (surgical equipment) as examples. Methods: The study will systematically perform a scoping review through a database and snowball search. We will analyze and classify the studies from a predetermined set of categories and then summarize them into an evidence map. Based on the review, the study will develop a 2D framework for businesses to follow or for future research to take a standpoint from. Results: Preliminarily, the review has analyzed 26 studies in total. The results are close to equally distributed among the micro (8/26, 31%), meso (10/26, 38%), and macro (8/26, 31%) levels. Circular economy studies emphasize several circular practices such as recycling (17/26, 65%), reusing (18/26, 69%), reducing (15/26, 58%), and remanufacturing (8/26, 31%). The value proposition in the examined business model is mostly dominated by stand-alone products (18/26, 69%) compared to product as a service (7/26, 27%), involving stakeholders such as health care professionals or hospitals (20/26, 77%), manufacturers (11/26, 42%), and consumers (9/26, 35%). All studies encompass societal (12/26, 46%), economic (23/26, 88%), and environmental (24/26, 92%) viewpoints. Conclusions: The study argues that each digital health solution would have to be accessed individually to find the optimal business model to follow. This is due to their differing life cycles and complexity. The manufacturer will need a layered value proposition, implementing several business models dependent on their respective product portfolios. The need to incorporate several business models implies an ecosystem perspective that is relevant to consider. International Registered Report Identifier (IRRID): DERR1-10.2196/47874 %M 37999949 %R 10.2196/47874 %U https://www.researchprotocols.org/2023/1/e47874 %U https://doi.org/10.2196/47874 %U http://www.ncbi.nlm.nih.gov/pubmed/37999949 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e52286 %T Authors’ Reply: Modernizing Gender, Sex, and Sexual Orientation Data Through Engagement and Education %A Queen,Roz %A Courtney,Karen L %A Lau,Francis %A Davison,Kelly %A Devor,Aaron %A Antonio,Marcy G %+ School of Health Information Science, University of Victoria, PO Box 1700 STN CSC, Victoria, BC, V8W 2Y2, Canada, 1 2507218575, rozomqueen@gmail.com %K data sharing %K digital health systems %K digital health %K gender, sex, and sexual orientation %K electronic health records %K GSSO %K health information standards %K LGBT health %K LGBT %K policy %D 2023 %7 15.11.2023 %9 Letter to the Editor %J J Med Internet Res %G English %X %M 37966876 %R 10.2196/52286 %U https://www.jmir.org/2023/1/e52286 %U https://doi.org/10.2196/52286 %U http://www.ncbi.nlm.nih.gov/pubmed/37966876 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e51632 %T Modernizing Gender, Sex, and Sexual Orientation Data Through Engagement and Education %A Ginaldi,Lia %A De Martinis,Massimo %+ Department of Life, Health and Environmental Sciences, University of L'Aquila, Piazzale Salvatore Tommasi n.1, L'Aquila, 67100, Italy, 39 0861 429548, demartinis@cc.univaq.it %K data sharing %K digital health systems %K digital health %K gender, sex, and sexual orientation %K electronic health records %K GSSO %K health information standards %K LGBT health %K LGBT %K policy %K LGBTQIA+ %K gender medicine %D 2023 %7 15.11.2023 %9 Letter to the Editor %J J Med Internet Res %G English %X %M 37966895 %R 10.2196/51632 %U https://www.jmir.org/2023/1/e51632 %U https://doi.org/10.2196/51632 %U http://www.ncbi.nlm.nih.gov/pubmed/37966895 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e44763 %T Machine Learning Algorithms Predict Successful Weaning From Mechanical Ventilation Before Intubation: Retrospective Analysis From the Medical Information Mart for Intensive Care IV Database %A Kim,Jinchul %A Kim,Yun Kwan %A Kim,Hyeyeon %A Jung,Hyojung %A Koh,Soonjeong %A Kim,Yujeong %A Yoon,Dukyong %A Yi,Hahn %A Kim,Hyung-Jun %+ Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Seoul National University Bundang Hospital, 82 Gumi-ro 173beon-gil, Bundang-gu, Seongnam, 13620, Republic of Korea, 82 31 787 7844, dr.hjkim@snubh.org %K algorithms %K clinical decision-making %K intensive care units %K noninvasive ventilation %K organ dysfunction scores %D 2023 %7 14.11.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: The prediction of successful weaning from mechanical ventilation (MV) in advance of intubation can facilitate discussions regarding end-of-life care before unnecessary intubation. Objective: We aimed to develop a machine learning–based model that predicts successful weaning from ventilator support based on routine clinical and laboratory data taken before or immediately after intubation. Methods: We used the Medical Information Mart for Intensive Care IV database, which is an open-access database covering 524,740 admissions of 382,278 patients in Beth Israel Deaconess Medical Center, United States, from 2008 to 2019. We selected adult patients who underwent MV in the intensive care unit (ICU). Clinical and laboratory variables that are considered relevant to the prognosis of the patient in the ICU were selected. Data collected before or within 24 hours of intubation were used to develop machine learning models that predict the probability of successful weaning within 14 days of ventilator support. Developed models were integrated into an ensemble model. Performance metrics were calculated by 5-fold cross-validation for each model, and a permutation feature importance and Shapley additive explanations analysis was conducted to better understand the impacts of individual variables on outcome prediction. Results: Of the 23,242 patients, 19,025 (81.9%) patients were successfully weaned from MV within 14 days. Using the preselected 46 clinical and laboratory variables, the area under the receiver operating characteristic curve of CatBoost classifier, random forest classifier, and regularized logistic regression classifier models were 0.860 (95% CI 0.852-0.868), 0.855 (95% CI 0.848-0.863), and 0.823 (95% CI 0.813-0.832), respectively. Using the ensemble voting classifier using the 3 models above, the final model revealed the area under the receiver operating characteristic curve of 0.861 (95% CI 0.853-0.869), which was significantly better than that of Simplified Acute Physiology Score II (0.749, 95% CI 0.742-0.756) and Sequential Organ Failure Assessment (0.588, 95% CI 0.566-0.609). The top features included lactate and anion gap. The model’s performance achieved a plateau with approximately the top 21 variables. Conclusions: We developed machine learning algorithms that can predict successful weaning from MV in advance to intubation in the ICU. Our models can aid the appropriate management for patients who hesitate to decide on ventilator support or meaningless end-of-life care. %M 37962939 %R 10.2196/44763 %U https://formative.jmir.org/2023/1/e44763 %U https://doi.org/10.2196/44763 %U http://www.ncbi.nlm.nih.gov/pubmed/37962939 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e48809 %T The Status of Data Management Practices Across German Medical Data Integration Centers: Mixed Methods Study %A Gierend,Kerstin %A Freiesleben,Sherry %A Kadioglu,Dennis %A Siegel,Fabian %A Ganslandt,Thomas %A Waltemath,Dagmar %+ Department of Biomedical Informatics at the Center for Preventive Medicine and Digital Health, Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, Mannheim, 68167, Germany, 49 621383 ext 8087, kerstin.gierend@medma.uni-heidelberg.de %K data management %K provenance %K traceability %K metadata %K data integration center %K maturity model %D 2023 %7 8.11.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: In the context of the Medical Informatics Initiative, medical data integration centers (DICs) have implemented complex data flows to transfer routine health care data into research data repositories for secondary use. Data management practices are of importance throughout these processes, and special attention should be given to provenance aspects. Insufficient knowledge can lead to validity risks and reduce the confidence and quality of the processed data. The need to implement maintainable data management practices is undisputed, but there is a great lack of clarity on the status. Objective: Our study examines the current data management practices throughout the data life cycle within the Medical Informatics in Research and Care in University Medicine (MIRACUM) consortium. We present a framework for the maturity status of data management practices and present recommendations to enable a trustful dissemination and reuse of routine health care data. Methods: In this mixed methods study, we conducted semistructured interviews with stakeholders from 10 DICs between July and September 2021. We used a self-designed questionnaire that we tailored to the MIRACUM DICs, to collect qualitative and quantitative data. Our study method is compliant with the Good Reporting of a Mixed Methods Study (GRAMMS) checklist. Results: Our study provides insights into the data management practices at the MIRACUM DICs. We identify several traceability issues that can be partially explained with a lack of contextual information within nonharmonized workflow steps, unclear responsibilities, missing or incomplete data elements, and incomplete information about the computational environment information. Based on the identified shortcomings, we suggest a data management maturity framework to reach more clarity and to help define enhanced data management strategies. Conclusions: The data management maturity framework supports the production and dissemination of accurate and provenance-enriched data for secondary use. Our work serves as a catalyst for the derivation of an overarching data management strategy, abiding data integrity and provenance characteristics as key factors. We envision that this work will lead to the generation of fairer and maintained health research data of high quality. %M 37938878 %R 10.2196/48809 %U https://www.jmir.org/2023/1/e48809 %U https://doi.org/10.2196/48809 %U http://www.ncbi.nlm.nih.gov/pubmed/37938878 %0 Journal Article %@ 2561-6722 %I JMIR Publications %V 6 %N %P e41393 %T An After-Hours Virtual Care Service for Children With Medical Complexity and New Medical Technology: Mixed Methods Feasibility Study %A Babayan,Katherine %A Keilty,Krista %A Esufali,Jessica %A Grajales III,Francisco J %A , %+ Institute of Health Policy, Management and Evaluation, University of Toronto, 155 College Street, Toronto, ON, M5T 3M6, Canada, 1 (416) 978 4326, katherine.babayan@gmail.com %K children with medical complexity %K technology dependence %K medical devices %K family caregivers %K virtual care %K home and community care %K emergency department visits %K enteral feeding tubes %K hospital-to-home transition %K feasibility %K mixed methods %D 2023 %7 8.11.2023 %9 Original Paper %J JMIR Pediatr Parent %G English %X Background: Family caregivers (FCs) of children with medical complexity require specialized support to promote the safe management of new medical technologies (eg, gastrostomy tubes) during hospital-to-home transitions. With limited after-hours services available to families in home and community care, medical device complications that arise often lead to increased FC stress and unplanned emergency department (ED) visits. To improve FC experiences, enable safer patient discharge, and reduce after-hours ED visits, this study explores the feasibility of piloting a 24/7 virtual care service (Connected Care Live) with families to provide real-time support by clinicians expert in the use of pediatric home care technologies. Objective: This study aims to establish the economic, operational, and technical feasibility of piloting the expansion of an existing nurse-led after-hours virtual care service offered to home and community care providers to FCs of children with newly inserted medical devices after hospital discharge at Toronto’s Hospital for Sick Children (SickKids). Methods: This exploratory study, conducted from October 2020 to August 2021, used mixed data sources to inform service expansion feasibility. Semistructured interviews were conducted with FCs, nurses, and hospital leadership to assess the risks, benefits, and technical and operational requirements for sustainable and cost-effective future service operations. Time and travel savings were estimated using ED visit data in SickKids’ electronic medical records (Epic) with a chief complaint of “medical device problems,” after-hours medical device inquiries from clinician emails and voicemails, and existing service operational data. Results: A total of 30 stakeholders were interviewed and voiced the need for the proposed service. Safer and more timely management of medical device complications, improved caregiver and provider experiences, and strengthened partnerships were identified as expected benefits, while service demand, nursing practice, and privacy and security were identified as potential risks. A total of 47 inquiries were recorded over 2 weeks from March 26, 2021, to April 8, 2021, with 51% (24/47) assessed as manageable via service expansion. This study forecasted annual time and travel savings of 558 hours for SickKids and 904 hours and 22,740 km for families. Minimal technical and operational requirements were needed to support service expansion by leveraging an existing platform and clinical staff. Of the 212 ED visits related to “medical device problems” over 6 months from September 1, 2020, to February 28, 2021, enteral feeding tubes accounted for nearly two-thirds (n=137, 64.6%), with 41.6% (57/137) assessed as virtually manageable. Conclusions: Our findings indicate that it is feasible to pilot the expansion of Connected Care Live to FCs of children with newly inserted enteral feeding tubes. This nurse-led virtual caregiver service is a promising tool to promote safe hospital-to-home transitions, improve FC experiences, and reduce after-hours ED visits. %M 37938869 %R 10.2196/41393 %U https://pediatrics.jmir.org/2023/1/e41393 %U https://doi.org/10.2196/41393 %U http://www.ncbi.nlm.nih.gov/pubmed/37938869 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e45636 %T Clinical Decision Support to Reduce Opioid Prescriptions for Dental Extractions using SMART on FHIR: Implementation Report %A Rindal,D Brad %A Pasumarthi,Dhavan Prasad %A Thirumalai,Vijayakumar %A Truitt,Anjali R %A Asche,Stephen E %A Worley,Donald C %A Kane,Sheryl M %A Gryczynski,Jan %A Mitchell,Shannon G %+ HealthPartners Institute, 8170 33rd Ave S, Minneapolis, MN, 55425, United States, 1 952 967 5026, donald.b.rindal@healthpartners.com %K clinical decision support systems %K dentistry %K analgesics %K electronic health records %K EHR %K algorithm %K design %K implementation %K decision support %K development %K dentists %K pain management %K patient care %K application %K tool %K Fast Healthcare Interoperability Resources %K FHIR %K Substitutable Medical Applications and Reusable Technologies %K SMART %D 2023 %7 7.11.2023 %9 Implementation Report %J JMIR Med Inform %G English %X Background: Clinical decision support (CDS) has the potential to improve clinical decision-making consistent with evidence-based care. CDS can be designed to save health care providers time and help them provide safe and personalized analgesic prescribing. Objective: The aim of this report is to describe the development of a CDS system designed to provide dentists with personalized pain management recommendations to reduce opioid prescribing following extractions. The use of CDS is also examined. Methods: This study was conducted in HealthPartners, which uses an electronic health record (EHR) system that integrates both medical and dental information upon which the CDS application was developed based on SMART (Substitutable Medical Applications and Reusable Technologies) on FHIR (Fast Healthcare Interoperability Resources). The various tools used to bring relevant medical conditions, medications, patient history, and other relevant data into the CDS interface are described. The CDS application runs a drug interaction algorithm developed by our organization and provides patient-specific recommendations. The CDS included access to the state Prescription Monitoring Program database. Implementation (Results): The pain management CDS was implemented as part of a study examining opioid prescribing among patients undergoing dental extraction procedures from February 17, 2020, to May 14, 2021. Provider-level use of CDS at extraction encounters ranged from 0% to 87.4% with 12.1% of providers opening the CDS for no encounters, 39.4% opening the CDS for 1%-20% of encounters, 36.4% opening it for 21%-50% of encounters, and 12.1% opening it for 51%-87% of encounters. Conclusions: The pain management CDS is an EHR-embedded, provider-facing tool to help dentists make personalized pain management recommendations following dental extractions. The SMART on FHIR–based pain management CDS adapted well to the point-of-care dental setting and led to the design of a scalable CDS tool that is EHR vendor agnostic. Trial Registration: ClinicalTrials.gov NCT03584789; https://clinicaltrials.gov/study/NCT03584789 %M 37934572 %R 10.2196/45636 %U https://medinform.jmir.org/2023/1/e45636 %U https://doi.org/10.2196/45636 %U http://www.ncbi.nlm.nih.gov/pubmed/37934572 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 9 %N %P e46708 %T Optimal Look-Back Period to Identify True Incident Cases of Diabetes in Medical Insurance Data in the Chinese Population: Retrospective Analysis Study %A Yang,Wenyi %A Wang,Baohua %A Ma,Shaobo %A Wang,Jingxin %A Ai,Limei %A Li,Zhengyu %A Wan,Xia %+ Institute of Basic Medical Sciences, Chinese Academy of Medical Science, Dongdan Street, 5th, Beijing, 100052, China, 86 01065233870, xiawan@ibms.pumc.edu.cn %K diabetes %K incident cases %K administrative data %K look-back period %K retrograde survival function %D 2023 %7 6.11.2023 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: Accurate estimation of incidence and prevalence is vital for preventing and controlling diabetes. Administrative data (including insurance data) could be a good source to estimate the incidence of diabetes. However, how to determine the look-back period (LP) to remove cases with preceding records remains a problem for administrative data. A short LP will cause overestimation of incidence, whereas a long LP will limit the usefulness of a database. Therefore, it is necessary to determine the optimal LP length for identifying incident cases in administrative data. Objective: This study aims to offer different methods to identify the optimal LP for diabetes by using medical insurance data from the Chinese population with reference to other diseases in the administrative data. Methods: Data from the insurance database of the city of Weifang, China from between January 2016 and December 2020 were used. To identify the incident cases in 2020, we removed prevalent patients with preceding records of diabetes between 2016 and 2019 (ie, a 4-year LP). Using this 4-year LP as a reference, consistency examination indexes (CEIs), including positive predictive values, the κ coefficient, and overestimation rate, were calculated to determine the level of agreement between different LPs and an LP of 4 years (the longest LP). Moreover, we constructed a retrograde survival function, in which survival (ie, incident cases) means not having a preceding record at the given time and the survival time is the difference between the date of the last record in 2020 and the most recent previous record in the LP. Based on the survival outcome and survival time, we established the survival function and survival hazard function. When the survival probability, S(t), remains stable, and survival hazard converges to zero, we obtain the optimal LP. Combined with the results of these two methods, we determined the optimal LP for Chinese diabetes patients. Results: The κ agreement was excellent (0.950), with a high positive predictive value (92.2%) and a low overestimation rate (8.4%) after a 2-year LP. As for the retrograde survival function, S(t) dropped rapidly during the first 1-year LP (from 1.00 to 0.11). At a 417-day LP, the hazard function reached approximately zero (ht=0.000459), S(t) remained at 0.10, and at 480 days, the frequency of S(t) did not increase. Combining the two methods, we found that the optimal LP is 2 years for Chinese diabetes patients. Conclusions: The retrograde survival method and CEIs both showed effectiveness. A 2-year LP should be considered when identifying incident cases of diabetes using insurance data in the Chinese population. %M 37930785 %R 10.2196/46708 %U https://publichealth.jmir.org/2023/1/e46708 %U https://doi.org/10.2196/46708 %U http://www.ncbi.nlm.nih.gov/pubmed/37930785 %0 Journal Article %@ 2561-1011 %I JMIR Publications %V 7 %N %P e44732 %T Physician- and Patient-Elicited Barriers and Facilitators to Implementation of a Machine Learning–Based Screening Tool for Peripheral Arterial Disease: Preimplementation Study With Physician and Patient Stakeholders %A Ho,Vy %A Brown Johnson,Cati %A Ghanzouri,Ilies %A Amal,Saeed %A Asch,Steven %A Ross,Elsie %+ Division of Vascular Surgery, Department of Surgery, Stanford University School of Medicine, 500 Pasteur Drive, Stanford, CA, 94043, United States, 1 6507232185, vivianho@stanford.edu %K artificial intelligence %K cardiovascular disease %K machine learning %K peripheral arterial disease %K preimplementation study %D 2023 %7 6.11.2023 %9 Original Paper %J JMIR Cardio %G English %X Background: Peripheral arterial disease (PAD) is underdiagnosed, partially due to a high prevalence of atypical symptoms and a lack of physician and patient awareness. Implementing clinical decision support tools powered by machine learning algorithms may help physicians identify high-risk patients for diagnostic workup. Objective: This study aims to evaluate barriers and facilitators to the implementation of a novel machine learning–based screening tool for PAD among physician and patient stakeholders using the Consolidated Framework for Implementation Research (CFIR). Methods: We performed semistructured interviews with physicians and patients from the Stanford University Department of Primary Care and Population Health, Division of Cardiology, and Division of Vascular Medicine. Participants answered questions regarding their perceptions toward machine learning and clinical decision support for PAD detection. Rapid thematic analysis was performed using templates incorporating codes from CFIR constructs. Results: A total of 12 physicians (6 primary care physicians and 6 cardiovascular specialists) and 14 patients were interviewed. Barriers to implementation arose from 6 CFIR constructs: complexity, evidence strength and quality, relative priority, external policies and incentives, knowledge and beliefs about intervention, and individual identification with the organization. Facilitators arose from 5 CFIR constructs: intervention source, relative advantage, learning climate, patient needs and resources, and knowledge and beliefs about intervention. Physicians felt that a machine learning–powered diagnostic tool for PAD would improve patient care but cited limited time and authority in asking patients to undergo additional screening procedures. Patients were interested in having their physicians use this tool but raised concerns about such technologies replacing human decision-making. Conclusions: Patient- and physician-reported barriers toward the implementation of a machine learning–powered PAD diagnostic tool followed four interdependent themes: (1) low familiarity or urgency in detecting PAD; (2) concerns regarding the reliability of machine learning; (3) differential perceptions of responsibility for PAD care among primary care versus specialty physicians; and (4) patient preference for physicians to remain primary interpreters of health care data. Facilitators followed two interdependent themes: (1) enthusiasm for clinical use of the predictive model and (2) willingness to incorporate machine learning into clinical care. Implementation of machine learning–powered diagnostic tools for PAD should leverage provider support while simultaneously educating stakeholders on the importance of early PAD diagnosis. High predictive validity is necessary for machine learning models but not sufficient for implementation. %M 37930755 %R 10.2196/44732 %U https://cardio.jmir.org/2023/1/e44732 %U https://doi.org/10.2196/44732 %U http://www.ncbi.nlm.nih.gov/pubmed/37930755 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e51884 %T Developing Health Management Competency for Digital Health Transformation: Protocol for a Qualitative Study %A Brommeyer,Mark %A Liang,Zhanming %A Whittaker,Maxine %A Mackay,Mark %+ College of Public Health, Medical and Veterinary Science, James Cook University, James Cook Drive, Townsville, 4810, Australia, 61 400825555, mark.brommeyer@flinders.edu.au %K health care management %K health service manager %K digital health %K health informatics %K competency %K workforce development %K innovation %K research protocol %K informatics %K manager %K managers %K service %K services %K delivery %K organization %K organizational %K workforce %K management %K managerial %K qualification %K qualifications %K focus group %K focus groups %K interview %K interviews %K scoping %K review methods %K review methodology %D 2023 %7 3.11.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Globally, the health care system is experiencing a period of rapid and radical change. In response, innovative service models have been adopted for the delivery of high-quality care that require a health workforce with skills to support transformation and new ways of working. Objective: The aim of this research protocol is to describe research that will contribute to developing the capability of health service managers in the digital health era and enabling digital transformation within the Australian health care environment. It also explains the process of preparing and finalizing the research design and methodologies by seeking answers to the following three research questions: (1) To what extent can the existing health service management and digital health competency frameworks guide the development of competence for health service managers in understanding and managing in the digital health space? (2) What are the competencies that are necessary for health service managers to acquire in order to effectively work with and manage in the digital health context? (3) What are the key factors that enable and inhibit health service managers to develop and demonstrate digital health competence in the workplace? Methods: The study has adopted a qualitative approach, guided by the empirically validated management competency identification process, using four steps: (1) health management and digital health competency mapping, (2) scoping review of literature and policy analysis, (3) focus group discussions with health service managers, and (4) semistructured interviews with digital health leaders. The first 2 steps were to confirm the need for updating the current health service management curriculum to address changing competency requirements of health service managers in the digital health context. Results: Two initial steps have been completed confirming the significance of the study and study design. Step 1, competency mapping, found that nearly half of the digital competencies were only partially or not addressed at all by the health management competency framework. The scoping review articulated the competencies health service managers need to effectively demonstrate digital health competence in the workplace. The findings effectively support the importance of the current research and also the appropriateness of the proposed steps 3 and 4 in answering the research questions and achieving the research aim. Conclusions: This study will provide insights into the health service management workforce performance and development needs for digital health and inform credentialing and professional development requirements. This will guide health service managers in leading and managing the adoption and implementation of digital health as a contemporary tool for health care delivery. The study will develop an in-depth understanding of Australian health service managers’ experiences and views. This research process could be applied in other contexts, noting that the results need contextualization to individual country jurisdictions and environments. International Registered Report Identifier (IRRID): DERR1-10.2196/51884 %M 37921855 %R 10.2196/51884 %U https://www.researchprotocols.org/2023/1/e51884 %U https://doi.org/10.2196/51884 %U http://www.ncbi.nlm.nih.gov/pubmed/37921855 %0 Journal Article %@ 2369-3762 %I JMIR Publications %V 9 %N %P e47532 %T The Accuracy and Potential Racial and Ethnic Biases of GPT-4 in the Diagnosis and Triage of Health Conditions: Evaluation Study %A Ito,Naoki %A Kadomatsu,Sakina %A Fujisawa,Mineto %A Fukaguchi,Kiyomitsu %A Ishizawa,Ryo %A Kanda,Naoki %A Kasugai,Daisuke %A Nakajima,Mikio %A Goto,Tadahiro %A Tsugawa,Yusuke %+ TXP Medical Co Ltd, 41-1 H¹O Kanda 706, Tokyo, 101-0042, Japan, 81 03 5615 8433, tag695@mail.harvard.edu %K GPT-4 %K racial and ethnic bias %K typical clinical vignettes %K diagnosis %K triage %K artificial intelligence %K AI %K race %K clinical vignettes %K physician %K efficiency %K decision-making %K bias %K GPT %D 2023 %7 2.11.2023 %9 Original Paper %J JMIR Med Educ %G English %X Background: Whether GPT-4, the conversational artificial intelligence, can accurately diagnose and triage health conditions and whether it presents racial and ethnic biases in its decisions remain unclear. Objective: We aim to assess the accuracy of GPT-4 in the diagnosis and triage of health conditions and whether its performance varies by patient race and ethnicity. Methods: We compared the performance of GPT-4 and physicians, using 45 typical clinical vignettes, each with a correct diagnosis and triage level, in February and March 2023. For each of the 45 clinical vignettes, GPT-4 and 3 board-certified physicians provided the most likely primary diagnosis and triage level (emergency, nonemergency, or self-care). Independent reviewers evaluated the diagnoses as “correct” or “incorrect.” Physician diagnosis was defined as the consensus of the 3 physicians. We evaluated whether the performance of GPT-4 varies by patient race and ethnicity, by adding the information on patient race and ethnicity to the clinical vignettes. Results: The accuracy of diagnosis was comparable between GPT-4 and physicians (the percentage of correct diagnosis was 97.8% (44/45; 95% CI 88.2%-99.9%) for GPT-4 and 91.1% (41/45; 95% CI 78.8%-97.5%) for physicians; P=.38). GPT-4 provided appropriate reasoning for 97.8% (44/45) of the vignettes. The appropriateness of triage was comparable between GPT-4 and physicians (GPT-4: 30/45, 66.7%; 95% CI 51.0%-80.0%; physicians: 30/45, 66.7%; 95% CI 51.0%-80.0%; P=.99). The performance of GPT-4 in diagnosing health conditions did not vary among different races and ethnicities (Black, White, Asian, and Hispanic), with an accuracy of 100% (95% CI 78.2%-100%). P values, compared to the GPT-4 output without incorporating race and ethnicity information, were all .99. The accuracy of triage was not significantly different even if patients’ race and ethnicity information was added. The accuracy of triage was 62.2% (95% CI 46.5%-76.2%; P=.50) for Black patients; 66.7% (95% CI 51.0%-80.0%; P=.99) for White patients; 66.7% (95% CI 51.0%-80.0%; P=.99) for Asian patients, and 62.2% (95% CI 46.5%-76.2%; P=.69) for Hispanic patients. P values were calculated by comparing the outputs with and without conditioning on race and ethnicity. Conclusions: GPT-4’s ability to diagnose and triage typical clinical vignettes was comparable to that of board-certified physicians. The performance of GPT-4 did not vary by patient race and ethnicity. These findings should be informative for health systems looking to introduce conversational artificial intelligence to improve the efficiency of patient diagnosis and triage. %M 37917120 %R 10.2196/47532 %U https://mededu.jmir.org/2023/1/e47532 %U https://doi.org/10.2196/47532 %U http://www.ncbi.nlm.nih.gov/pubmed/37917120 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e49605 %T Characteristics and Admission Preferences of Pediatric Emergency Patients and Their Waiting Time Prediction Using Electronic Medical Record Data: Retrospective Comparative Analysis %A Guo,Lin Lin %A Guo,Lin Ying %A Li,Jiao %A Gu,Yao Wen %A Wang,Jia Yang %A Cui,Ying %A Qian,Qing %A Chen,Ting %A Jiang,Rui %A Zheng,Si %+ Institute of Medical Information, Chinese Academy of Medical Sciences and Peking Union Medical College, 3 Yabao Rd, Chaoyang District, Beijing, 100020, China, 86 010 52328745, zheng.si@imicams.ac.cn %K pediatric emergency department %K characteristics %K admission preferences %K waiting time %K machine learning %K electronic medical record %D 2023 %7 1.11.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: The growing number of patients visiting pediatric emergency departments could have a detrimental impact on the care provided to children who are triaged as needing urgent attention. Therefore, it has become essential to continuously monitor and analyze the admissions and waiting times of pediatric emergency patients. Despite the significant challenge posed by the shortage of pediatric medical resources in China’s health care system, there have been few large-scale studies conducted to analyze visits to the pediatric emergency room. Objective: This study seeks to examine the characteristics and admission patterns of patients in the pediatric emergency department using electronic medical record (EMR) data. Additionally, it aims to develop and assess machine learning models for predicting waiting times for pediatric emergency department visits. Methods: This retrospective analysis involved patients who were admitted to the emergency department of Children’s Hospital Capital Institute of Pediatrics from January 1, 2021, to December 31, 2021. Clinical data from these admissions were extracted from the electronic medical records, encompassing various variables of interest such as patient demographics, clinical diagnoses, and time stamps of clinical visits. These indicators were collected and compared. Furthermore, we developed and evaluated several computational models for predicting waiting times. Results: In total, 183,024 eligible admissions from 127,368 pediatric patients were included. During the 12-month study period, pediatric emergency department visits were most frequent among children aged less than 5 years, accounting for 71.26% (130,423/183,024) of the total visits. Additionally, there was a higher proportion of male patients (104,147/183,024, 56.90%) compared with female patients (78,877/183,024, 43.10%). Fever (50,715/183,024, 27.71%), respiratory infection (43,269/183,024, 23.64%), celialgia (9560/183,024, 5.22%), and emesis (6898/183,024, 3.77%) were the leading causes of pediatric emergency room visits. The average daily number of admissions was 501.44, and 18.76% (34,339/183,204) of pediatric emergency department visits resulted in discharge without a prescription or further tests. The median waiting time from registration to seeing a doctor was 27.53 minutes. Prolonged waiting times were observed from April to July, coinciding with an increased number of arrivals, primarily for respiratory diseases. In terms of waiting time prediction, machine learning models, specifically random forest, LightGBM, and XGBoost, outperformed regression methods. On average, these models reduced the root-mean-square error by approximately 17.73% (8.951/50.481) and increased the R2 by approximately 29.33% (0.154/0.525). The SHAP method analysis highlighted that the features “wait.green” and “department” had the most significant influence on waiting times. Conclusions: This study offers a contemporary exploration of pediatric emergency room visits, revealing significant variations in admission rates across different periods and uncovering certain admission patterns. The machine learning models, particularly ensemble methods, delivered more dependable waiting time predictions. Patient volume awaiting consultation or treatment and the triage status emerged as crucial factors contributing to prolonged waiting times. Therefore, strategies such as patient diversion to alleviate congestion in emergency departments and optimizing triage systems to reduce average waiting times remain effective approaches to enhance the quality of pediatric health care services in China. %M 37910168 %R 10.2196/49605 %U https://www.jmir.org/2023/1/e49605 %U https://doi.org/10.2196/49605 %U http://www.ncbi.nlm.nih.gov/pubmed/37910168 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e48236 %T Implications for Electronic Surveys in Inpatient Settings Based on Patient Survey Response Patterns: Cross-Sectional Study %A Gregory,Megan E %A Sova,Lindsey N %A Huerta,Timothy R %A McAlearney,Ann Scheck %+ The Center for the Advancement of Team Science, Analytics, and Systems Thinking in Health Services and Implementation Science Research (CATALYST), College of Medicine, The Ohio State University, 700 Ackerman Rd, Suite 4000, Columbus, OH, 43202, United States, 1 614 293 8973, Ann.McAlearney@osumc.edu %K surveys %K patient satisfaction %K patient experience %K patient surveys %K electronic survey %K cross-sectional study %K quality of life %K mental health %K symptoms %K data quality %K hospitalization %D 2023 %7 1.11.2023 %9 Original Paper %J J Med Internet Res %G English %X Background:  Surveys of hospitalized patients are important for research and learning about unobservable medical issues (eg, mental health, quality of life, and symptoms), but there has been little work examining survey data quality in this population whose capacity to respond to survey items may differ from the general population. Objective:  The aim of this study is to determine what factors drive response rates, survey drop-offs, and missing data in surveys of hospitalized patients. Methods:  Cross-sectional surveys were distributed on an inpatient tablet to patients in a large, midwestern US hospital. Three versions were tested: 1 with 174 items and 2 with 111 items; one 111-item version had missing item reminders that prompted participants when they did not answer items. Response rate, drop-off rate (abandoning survey before completion), and item missingness (skipping items) were examined to investigate data quality. Chi-square tests, Kaplan-Meyer survival curves, and distribution charts were used to compare data quality among survey versions. Response duration was computed for each version. Results: Overall, 2981 patients responded. Response rate did not differ between the 174- and 111-item versions (81.7% vs 83%, P=.53). Drop-off was significantly reduced when the survey was shortened (65.7% vs 20.2% of participants dropped off, P<.001). Approximately one-quarter of participants dropped off by item 120, with over half dropping off by item 158. The percentage of participants with missing data decreased substantially when missing item reminders were added (77.2% vs 31.7% of participants, P<.001). The mean percentage of items with missing data was reduced in the shorter survey (40.7% vs 20.3% of items missing); with missing item reminders, the percentage of items with missing data was further reduced (20.3% vs 11.7% of items missing). Across versions, for the median participant, each item added 24.6 seconds to a survey’s duration. Conclusions:  Hospitalized patients may have a higher tolerance for longer surveys than the general population, but surveys given to hospitalized patients should have a maximum of 120 items to ensure high rates of completion. Missing item prompts should be used to reduce missing data. Future research should examine generalizability to nonhospitalized individuals. %M 37910163 %R 10.2196/48236 %U https://www.jmir.org/2023/1/e48236 %U https://doi.org/10.2196/48236 %U http://www.ncbi.nlm.nih.gov/pubmed/37910163 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e49324 %T Large Language Models for Therapy Recommendations Across 3 Clinical Specialties: Comparative Study %A Wilhelm,Theresa Isabelle %A Roos,Jonas %A Kaczmarczyk,Robert %+ Department of Dermatology and Allergy, School of Medicine, Technical University of Munich, Biedersteiner Str 29, Munich, 80802, Germany, 49 08941403033, Robert.Kaczmarczyk@tum.de %K dermatology %K ophthalmology %K orthopedics %K therapy %K large language models %K artificial intelligence %K LLM %K ChatGPT %K chatbot %K chatbots %K orthopedic %K recommendation %K recommendations %K medical information %K health information %K quality %K reliability %K accuracy %K safety %K reliable %K medical advice %D 2023 %7 30.10.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: As advancements in artificial intelligence (AI) continue, large language models (LLMs) have emerged as promising tools for generating medical information. Their rapid adaptation and potential benefits in health care require rigorous assessment in terms of the quality, accuracy, and safety of the generated information across diverse medical specialties. Objective: This study aimed to evaluate the performance of 4 prominent LLMs, namely, Claude-instant-v1.0, GPT-3.5-Turbo, Command-xlarge-nightly, and Bloomz, in generating medical content spanning the clinical specialties of ophthalmology, orthopedics, and dermatology. Methods: Three domain-specific physicians evaluated the AI-generated therapeutic recommendations for a diverse set of 60 diseases. The evaluation criteria involved the mDISCERN score, correctness, and potential harmfulness of the recommendations. ANOVA and pairwise t tests were used to explore discrepancies in content quality and safety across models and specialties. Additionally, using the capabilities of OpenAI’s most advanced model, GPT-4, an automated evaluation of each model’s responses to the diseases was performed using the same criteria and compared to the physicians’ assessments through Pearson correlation analysis. Results: Claude-instant-v1.0 emerged with the highest mean mDISCERN score (3.35, 95% CI 3.23-3.46). In contrast, Bloomz lagged with the lowest score (1.07, 95% CI 1.03-1.10). Our analysis revealed significant differences among the models in terms of quality (P<.001). Evaluating their reliability, the models displayed strong contrasts in their falseness ratings, with variations both across models (P<.001) and specialties (P<.001). Distinct error patterns emerged, such as confusing diagnoses; providing vague, ambiguous advice; or omitting critical treatments, such as antibiotics for infectious diseases. Regarding potential harm, GPT-3.5-Turbo was found to be the safest, with the lowest harmfulness rating. All models lagged in detailing the risks associated with treatment procedures, explaining the effects of therapies on quality of life, and offering additional sources of information. Pearson correlation analysis underscored a substantial alignment between physician assessments and GPT-4’s evaluations across all established criteria (P<.01). Conclusions: This study, while comprehensive, was limited by the involvement of a select number of specialties and physician evaluators. The straightforward prompting strategy (“How to treat…”) and the assessment benchmarks, initially conceptualized for human-authored content, might have potential gaps in capturing the nuances of AI-driven information. The LLMs evaluated showed a notable capability in generating valuable medical content; however, evident lapses in content quality and potential harm signal the need for further refinements. Given the dynamic landscape of LLMs, this study’s findings emphasize the need for regular and methodical assessments, oversight, and fine-tuning of these AI tools to ensure they produce consistently trustworthy and clinically safe medical advice. Notably, the introduction of an auto-evaluation mechanism using GPT-4, as detailed in this study, provides a scalable, transferable method for domain-agnostic evaluations, extending beyond therapy recommendation assessments. %M 37902826 %R 10.2196/49324 %U https://www.jmir.org/2023/1/e49324 %U https://doi.org/10.2196/49324 %U http://www.ncbi.nlm.nih.gov/pubmed/37902826 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e46547 %T Architectural Design of a Blockchain-Enabled, Federated Learning Platform for Algorithmic Fairness in Predictive Health Care: Design Science Study %A Liang,Xueping %A Zhao,Juan %A Chen,Yan %A Bandara,Eranga %A Shetty,Sachin %+ Department of Information Systems and Business Analytics, Florida International University, 11200 SW 8th St, Miami, FL, 33199, United States, 1 305 348 2830, xuliang@fiu.edu %K fairness %K federated learning %K bias %K health care %K blockchain %K software %K proof of concept %K implementation %K privacy %D 2023 %7 30.10.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Developing effective and generalizable predictive models is critical for disease prediction and clinical decision-making, often requiring diverse samples to mitigate population bias and address algorithmic fairness. However, a major challenge is to retrieve learning models across multiple institutions without bringing in local biases and inequity, while preserving individual patients’ privacy at each site. Objective: This study aims to understand the issues of bias and fairness in the machine learning process used in the predictive health care domain. We proposed a software architecture that integrates federated learning and blockchain to improve fairness, while maintaining acceptable prediction accuracy and minimizing overhead costs. Methods: We improved existing federated learning platforms by integrating blockchain through an iterative design approach. We used the design science research method, which involves 2 design cycles (federated learning for bias mitigation and decentralized architecture). The design involves a bias-mitigation process within the blockchain-empowered federated learning framework based on a novel architecture. Under this architecture, multiple medical institutions can jointly train predictive models using their privacy-protected data effectively and efficiently and ultimately achieve fairness in decision-making in the health care domain. Results: We designed and implemented our solution using the Aplos smart contract, microservices, Rahasak blockchain, and Apache Cassandra–based distributed storage. By conducting 20,000 local model training iterations and 1000 federated model training iterations across 5 simulated medical centers as peers in the Rahasak blockchain network, we demonstrated how our solution with an improved fairness mechanism can enhance the accuracy of predictive diagnosis. Conclusions: Our study identified the technical challenges of prediction biases faced by existing predictive models in the health care domain. To overcome these challenges, we presented an innovative design solution using federated learning and blockchain, along with the adoption of a unique distributed architecture for a fairness-aware system. We have illustrated how this design can address privacy, security, prediction accuracy, and scalability challenges, ultimately improving fairness and equity in the predictive health care domain. %M 37902833 %R 10.2196/46547 %U https://www.jmir.org/2023/1/e46547 %U https://doi.org/10.2196/46547 %U http://www.ncbi.nlm.nih.gov/pubmed/37902833 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e46905 %T A Hybrid Decision Tree and Deep Learning Approach Combining Medical Imaging and Electronic Medical Records to Predict Intubation Among Hospitalized Patients With COVID-19: Algorithm Development and Validation %A Nguyen,Kim-Anh-Nhi %A Tandon,Pranai %A Ghanavati,Sahar %A Cheetirala,Satya Narayana %A Timsina,Prem %A Freeman,Robert %A Reich,David %A Levin,Matthew A %A Mazumdar,Madhu %A Fayad,Zahi A %A Kia,Arash %+ Institute for Healthcare Delivery Science, Icahn School of Medicine at Mount Sinai, One Gustave L Levy Place, 1255 5th Ave, Suite C-2, New York, NY, 10029, United States, 1 8572851577, kim-anh-nhi.nguyen@mountsinai.org %K COVID-19 %K medical imaging %K machine learning %K chest radiograph %K mechanical ventilation %K electronic health records %K intubation %K decision trees %K hybrid model %K clinical informatics %D 2023 %7 26.10.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Early prediction of the need for invasive mechanical ventilation (IMV) in patients hospitalized with COVID-19 symptoms can help in the allocation of resources appropriately and improve patient outcomes by appropriately monitoring and treating patients at the greatest risk of respiratory failure. To help with the complexity of deciding whether a patient needs IMV, machine learning algorithms may help bring more prognostic value in a timely and systematic manner. Chest radiographs (CXRs) and electronic medical records (EMRs), typically obtained early in patients admitted with COVID-19, are the keys to deciding whether they need IMV. Objective: We aimed to evaluate the use of a machine learning model to predict the need for intubation within 24 hours by using a combination of CXR and EMR data in an end-to-end automated pipeline. We included historical data from 2481 hospitalizations at The Mount Sinai Hospital in New York City. Methods: CXRs were first resized, rescaled, and normalized. Then lungs were segmented from the CXRs by using a U-Net algorithm. After splitting them into a training and a test set, the training set images were augmented. The augmented images were used to train an image classifier to predict the probability of intubation with a prediction window of 24 hours by retraining a pretrained DenseNet model by using transfer learning, 10-fold cross-validation, and grid search. Then, in the final fusion model, we trained a random forest algorithm via 10-fold cross-validation by combining the probability score from the image classifier with 41 longitudinal variables in the EMR. Variables in the EMR included clinical and laboratory data routinely collected in the inpatient setting. The final fusion model gave a prediction likelihood for the need of intubation within 24 hours as well. Results: At a prediction probability threshold of 0.5, the fusion model provided 78.9% (95% CI 59%-96%) sensitivity, 83% (95% CI 76%-89%) specificity, 0.509 (95% CI 0.34-0.67) F1-score, 0.874 (95% CI 0.80-0.94) area under the receiver operating characteristic curve (AUROC), and 0.497 (95% CI 0.32-0.65) area under the precision recall curve (AUPRC) on the holdout set. Compared to the image classifier alone, which had an AUROC of 0.577 (95% CI 0.44-0.73) and an AUPRC of 0.206 (95% CI 0.08-0.38), the fusion model showed significant improvement (P<.001). The most important predictor variables were respiratory rate, C-reactive protein, oxygen saturation, and lactate dehydrogenase. The imaging probability score ranked 15th in overall feature importance. Conclusions: We show that, when linked with EMR data, an automated deep learning image classifier improved performance in identifying hospitalized patients with severe COVID-19 at risk for intubation. With additional prospective and external validation, such a model may assist risk assessment and optimize clinical decision-making in choosing the best care plan during the critical stages of COVID-19. %M 37883177 %R 10.2196/46905 %U https://formative.jmir.org/2023/1/e46905 %U https://doi.org/10.2196/46905 %U http://www.ncbi.nlm.nih.gov/pubmed/37883177 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e47119 %T Methods for the Clinical Validation of Digital Endpoints: Protocol for a Scoping Review Abstract %A Rego,Sílvia %A Henriques,Ana Rita %A Serra,Sofia Silvério %A Costa,Teresa %A Rodrigues,Ana Maria %A Nunes,Francisco %+ Fraunhofer Portugal Research Center for Assistive Information and Communication Solutions, Rua Alfredo Allen, 455-461, Porto, 4200-135 Porto, Portugal, 351 220430300, silvia.s.rego@gmail.com %K digital endpoint %K digital biomarker %K mobile health technologies %K mobile health %K mHealth %K remote monitoring %K wearable technology %K scoping review %K review method %K validate %K validation %K outcome measure %K sensor %K wearable %D 2023 %7 26.10.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Clinical trials often use digital technologies to collect data continuously outside the clinic and use the derived digital endpoints as trial endpoints. Digital endpoints are also being developed to support diagnosis, monitoring, or therapeutic interventions in clinical care. However, clinical validation stands as a significant challenge, as there are no specific guidelines orienting the validation of digital endpoints. Objective: This paper presents the protocol for a scoping review that aims to map the existing methods for the clinical validation of digital endpoints. Methods: The scoping review will comprise searches from the electronic literature databases MEDLINE (PubMed), Scopus (including conference proceedings), Embase, IEEE (Institute of Electrical and Electronics Engineers) Xplore, ACM (Association for Computing Machinery) Digital Library, CENTRAL (Cochrane Central Register of Controlled Trials), Web of Science Core Collection (including conference proceedings), and Joanna Briggs Institute Database of Systematic Reviews and Implementation Reports. We will also include various sources of gray literature with search terms related to digital endpoints. The methodology will adhere to the Joanna Briggs Institute Scoping Review and the Guidance for Conducting Systematic Scoping Reviews. Results: A search for reviews on the existing evidence related to this topic was conducted and has shown that no such review was previously undertaken. This review will provide a systematic assessment of the literature on methods for the clinical validation of digital endpoints and highlight any potential need for harmonization or reporting of methods. The results will include the methods for the clinical validation of digital endpoints according to device, digital endpoint, and clinical application goal of digital endpoints. The study started in January 2023 and is expected to end by December 2023, with results to be published in a peer-reviewed journal. Conclusions: A scoping review of methodologies that validate digital endpoints is necessary. This review will be unique in its breadth since it will comprise digital endpoints collected from several devices and not focus on a specific disease area. The results of our work should help guide researchers in choosing validation methods, identify potential gaps in the literature, or inform the development of novel methods to optimize the clinical validation of digital endpoints. Resolving these gaps is the key to presenting evidence in a consistent way to regulators and other parties and obtaining regulatory acceptance of digital endpoints for patient benefit. International Registered Report Identifier (IRRID): PRR1-10.2196/47119 %M 37883152 %R 10.2196/47119 %U https://www.researchprotocols.org/2023/1/e47119 %U https://doi.org/10.2196/47119 %U http://www.ncbi.nlm.nih.gov/pubmed/37883152 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e44065 %T Integrating Clinical Decision Support Into Electronic Health Record Systems Using a Novel Platform (EvidencePoint): Developmental Study %A Solomon,Jeffrey %A Dauber-Decker,Katherine %A Richardson,Safiya %A Levy,Sera %A Khan,Sundas %A Coleman,Benjamin %A Persaud,Rupert %A Chelico,John %A King,D'Arcy %A Spyropoulos,Alex %A McGinn,Thomas %+ Institute of Health System Science, Feinstein Institutes for Medical Research, 350 Community Drive, Manhasset, NY, 11030, United States, 1 516 600 1422, jsolomon6@northwell.edu %K clinical decision support system %K cloud based %K decision support %K development %K EHR %K electronic health record %K evidence-based medicine %K health information technology %K platform %K user-centered design %D 2023 %7 19.10.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Through our work, we have demonstrated how clinical decision support (CDS) tools integrated into the electronic health record (EHR) assist providers in adopting evidence-based practices. This requires confronting technical challenges that result from relying on the EHR as the foundation for tool development; for example, the individual CDS tools need to be built independently for each different EHR. Objective: The objective of our research was to build and implement an EHR-agnostic platform for integrating CDS tools, which would remove the technical constraints inherent in relying on the EHR as the foundation and enable a single set of CDS tools that can work with any EHR. Methods: We developed EvidencePoint, a novel, cloud-based, EHR-agnostic CDS platform, and we will describe the development of EvidencePoint and the deployment of its initial CDS tools, which include EHR-integrated applications for clinical use cases such as prediction of hospitalization survival for patients with COVID-19, venous thromboembolism prophylaxis, and pulmonary embolism diagnosis. Results: The results below highlight the adoption of the CDS tools, the International Medical Prevention Registry on Venous Thromboembolism-D-Dimer, the Wells’ criteria, and the Northwell COVID-19 Survival (NOCOS), following development, usability testing, and implementation. The International Medical Prevention Registry on Venous Thromboembolism-D-Dimer CDS was used in 5249 patients at the 2 clinical intervention sites. The intervention group tool adoption was 77.8% (4083/5249 possible uses). For the NOCOS tool, which was designed to assist with triaging patients with COVID-19 for hospital admission in the event of constrained hospital resources, the worst-case resourcing scenario never materialized and triaging was never required. As a result, the NOCOS tool was not frequently used, though the EvidencePoint platform’s flexibility and customizability enabled the tool to be developed and deployed rapidly under the emergency conditions of the pandemic. Adoption rates for the Wells’ criteria tool will be reported in a future publication. Conclusions: The EvidencePoint system successfully demonstrated that a flexible, user-friendly platform for hosting CDS tools outside of a specific EHR is feasible. The forthcoming results of our outcomes analyses will demonstrate the adoption rate of EvidencePoint tools as well as the impact of behavioral economics “nudges” on the adoption rate. Due to the EHR-agnostic nature of EvidencePoint, the development process for additional forms of CDS will be simpler than traditional and cumbersome IT integration approaches and will benefit from the capabilities provided by the core system of EvidencePoint. %M 37856193 %R 10.2196/44065 %U https://formative.jmir.org/2023/1/e44065 %U https://doi.org/10.2196/44065 %U http://www.ncbi.nlm.nih.gov/pubmed/37856193 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e47254 %T The BioRef Infrastructure, a Framework for Real-Time, Federated, Privacy-Preserving, and Personalized Reference Intervals: Design, Development, and Application %A Blatter,Tobias Ueli %A Witte,Harald %A Fasquelle-Lopez,Jules %A Nakas,Christos Theodoros %A Raisaro,Jean Louis %A Leichtle,Alexander Benedikt %+ University Institute of Clinical Chemistry, University Hospital Bern, Freiburgstrasse 10, Bern, 3010, Switzerland, 41 31 632 83 30, harald.witte@extern.insel.ch %K personalized health %K laboratory medicine %K reference interval %K research infrastructure %K sensitive data %K confidential data %K data security %K differential privacy %K precision medicine %D 2023 %7 18.10.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Reference intervals (RIs) for patient test results are in standard use across many medical disciplines, allowing physicians to identify measurements indicating potentially pathological states with relative ease. The process of inferring cohort-specific RIs is, however, often ignored because of the high costs and cumbersome efforts associated with it. Sophisticated analysis tools are required to automatically infer relevant and locally specific RIs directly from routine laboratory data. These tools would effectively connect clinical laboratory databases to physicians and provide personalized target ranges for the respective cohort population. Objective: This study aims to describe the BioRef infrastructure, a multicentric governance and IT framework for the estimation and assessment of patient group–specific RIs from routine clinical laboratory data using an innovative decentralized data-sharing approach and a sophisticated, clinically oriented graphical user interface for data analysis. Methods: A common governance agreement and interoperability standards have been established, allowing the harmonization of multidimensional laboratory measurements from multiple clinical databases into a unified “big data” resource. International coding systems, such as the International Classification of Diseases, Tenth Revision (ICD-10); unique identifiers for medical devices from the Global Unique Device Identification Database; type identifiers from the Global Medical Device Nomenclature; and a universal transfer logic, such as the Resource Description Framework (RDF), are used to align the routine laboratory data of each data provider for use within the BioRef framework. With a decentralized data-sharing approach, the BioRef data can be evaluated by end users from each cohort site following a strict “no copy, no move” principle, that is, only data aggregates for the intercohort analysis of target ranges are exchanged. Results: The TI4Health distributed and secure analytics system was used to implement the proposed federated and privacy-preserving approach and comply with the limitations applied to sensitive patient data. Under the BioRef interoperability consensus, clinical partners enable the computation of RIs via the TI4Health graphical user interface for query without exposing the underlying raw data. The interface was developed for use by physicians and clinical laboratory specialists and allows intuitive and interactive data stratification by patient factors (age, sex, and personal medical history) as well as laboratory analysis determinants (device, analyzer, and test kit identifier). This consolidated effort enables the creation of extremely detailed and patient group–specific queries, allowing the generation of individualized, covariate-adjusted RIs on the fly. Conclusions: With the BioRef-TI4Health infrastructure, a framework for clinical physicians and researchers to define precise RIs immediately in a convenient, privacy-preserving, and reproducible manner has been implemented, promoting a vital part of practicing precision medicine while streamlining compliance and avoiding transfers of raw patient data. This new approach can provide a crucial update on RIs and improve patient care for personalized medicine. %M 37851984 %R 10.2196/47254 %U https://www.jmir.org/2023/1/e47254 %U https://doi.org/10.2196/47254 %U http://www.ncbi.nlm.nih.gov/pubmed/37851984 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e45163 %T Using Theories, Models, and Frameworks to Inform Implementation Cycles of Computerized Clinical Decision Support Systems in Tertiary Health Care Settings: Scoping Review %A Fernando,Manasha %A Abell,Bridget %A Tyack,Zephanie %A Donovan,Thomasina %A McPhail,Steven M %A Naicker,Sundresan %+ Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology, Q Block, 60 Musk Avenue, Kelvin Grove QLD, Brisbane, 4059, Australia, 61 3138 6454, sundresan.naicker@qut.edu.au %K computerized clinical decision support systems %K CDSS %K implementation science %K hospital %K theories %K models %K frameworks %K mobile phone %D 2023 %7 18.10.2023 %9 Review %J J Med Internet Res %G English %X Background: Computerized clinical decision support systems (CDSSs) are essential components of modern health system service delivery, particularly within acute care settings such as hospitals. Theories, models, and frameworks may assist in facilitating the implementation processes associated with CDSS innovation and its use within these care settings. These processes include context assessments to identify key determinants, implementation plans for adoption, promoting ongoing uptake, adherence, and long-term evaluation. However, there has been no prior review synthesizing the literature regarding the theories, models, and frameworks that have informed the implementation and adoption of CDSSs within hospitals. Objective: This scoping review aims to identify the theory, model, and framework approaches that have been used to facilitate the implementation and adoption of CDSSs in tertiary health care settings, including hospitals. The rationales reported for selecting these approaches, including the limitations and strengths, are described. Methods: A total of 5 electronic databases were searched (CINAHL via EBSCOhost, PubMed, Scopus, PsycINFO, and Embase) to identify studies that implemented or adopted a CDSS in a tertiary health care setting using an implementation theory, model, or framework. No date or language limits were applied. A narrative synthesis was conducted using full-text publications and abstracts. Implementation phases were classified according to the “Active Implementation Framework stages”: exploration (feasibility and organizational readiness), installation (organizational preparation), initial implementation (initiating implementation, ie, training), full implementation (sustainment), and nontranslational effectiveness studies. Results: A total of 81 records (42 full text and 39 abstracts) were included. Full-text studies and abstracts are reported separately. For full-text studies, models (18/42, 43%), followed by determinants frameworks (14/42,33%), were most frequently used to guide adoption and evaluation strategies. Most studies (36/42, 86%) did not list the limitations associated with applying a specific theory, model, or framework. Conclusions: Models and related quality improvement methods were most frequently used to inform CDSS adoption. Models were not typically combined with each other or with theory to inform full-cycle implementation strategies. The findings highlight a gap in the application of implementation methods including theories, models, and frameworks to facilitate full-cycle implementation strategies for hospital CDSSs. %M 37851492 %R 10.2196/45163 %U https://www.jmir.org/2023/1/e45163 %U https://doi.org/10.2196/45163 %U http://www.ncbi.nlm.nih.gov/pubmed/37851492 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e49731 %T Enhancing Transsectoral Interdisciplinary Patient-Centered Care for Patients With Rare Cancers: Protocol for a Mixed Methods Process Evaluation %A Hinneburg,Jana %A Zacher,Sandro %A Berger-Höger,Birte %A Berger-Thürmel,Karin %A Kratzer,Vanessa %A Steckelberg,Anke %A Lühnen,Julia %A , %+ Institute for Health and Nursing Science, Medical Faculty, Martin Luther University Halle-Wittenberg, Magdeburger Str 8, Halle (Saale), 06112, Germany, 49 345 557 1220, sandro.zacher@medizin.uni-halle.de %K process evaluation %K study protocol %K logic model %K complex intervention %K coordination of care %K rare cancer %K mobile phone %D 2023 %7 12.10.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Rare cancers account for approximately 24% of all new cancers. The category of rare tumor diseases includes almost 200 different entities. In particular, the treatment of patients with extensive care needs requires cooperation between service providers, both between sectors (outpatient and inpatient) and within sectors (eg, between different medical disciplines). The treatment pathway is associated with a high need for coordination and information sharing between providers. When crossing sectoral boundaries in the German health care system, interface problems between the outpatient and inpatient sectors can lead to gaps in care delivery. The multicomponent program Trans-sectoral Personalised Care Concept for Patients with Rare Cancers aims to optimize transsectoral cooperation and coordination of care to enhance patient involvement and the medical care coordination of patients with rare cancers. Objective: This process evaluation will contribute to answering questions about intervention fidelity and the implementation of transsectoral communication, identifying and describing the intended and nonintended effects of the intervention, and exploring the barriers to and facilitators of the implementation. Methods: We will include patients who participate in the intervention phase; all persons and staff involved in the development and implementation of the intervention (Onco Coach, psychologists, physicians on the contact platform, IT staff, and staff of the Bavarian Association of Statutory Health Insurance Physicians); physicians from the Ludwig-Maximilians-University Hospital Munich and the hospital of the Technical University Munich who are involved in the treatment of patients during the course of the project; and participating office–based hematologists and oncologists. Data collection will be conducted at the beginning, during, and at the end of the intervention using mixed methods. Data will be collected from questionnaires, document analyses, semistructured interviews, and structured observations and will cover different aspects of process evaluation. These include examining the context to explore existing patterns, changes in patterns, attitudes, and interactions; analyzing the implementation of intervention elements; and exploring the complex causal pathways and mediators of the intervention. Qualitative data will be analyzed using thematic analysis. The data will then be combined using between-methods triangulation. Results: This project received funding on March 1, 2022. The intervention phase and recruitment for the process evaluation began on March 1, 2023, and the recruitment is expected to end on September 30, 2025. At the time of protocol submission in June 2023, a total of 8 doctors from hematology and oncology practices were enrolled. Data collection began on March 14, 2023. Conclusions: The Trans-sectoral Personalised Care Concept for Patients with Rare Cancers project is a complex intervention that is to be implemented in an equally complex health care context. The process evaluation will help understand the influence of contextual factors and assess the mechanisms of change. Trial Registration: ISRCTN registry ISRCTN16441179; https://doi.org/10.1186/ISRCTN16441179 International Registered Report Identifier (IRRID): DERR1-10.2196/49731 %M 37824180 %R 10.2196/49731 %U https://www.researchprotocols.org/2023/1/e49731 %U https://doi.org/10.2196/49731 %U http://www.ncbi.nlm.nih.gov/pubmed/37824180 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e46992 %T Digital Health Technology for Real-World Clinical Outcome Measurement Using Patient-Generated Data: Systematic Scoping Review %A Pyper,Evelyn %A McKeown,Sarah %A Hartmann-Boyce,Jamie %A Powell,John %+ Department for Continuing Education, University of Oxford, Rewley House, 1 Wellington Square, Oxford, United Kingdom, 44 (0)1865 270360, evelyn.pyper@stcatz.ox.ac.uk %K real-world evidence %K real-world data %K digital tools %K digital health %K digital biomarkers %K patient-generated health data %K mobile health %K mHealth %K wearables %K digital health management %K clinical intervention %K electronic health record %K health outcomes %K mobile phone %D 2023 %7 11.10.2023 %9 Review %J J Med Internet Res %G English %X Background: Digital health technologies (DHTs) play an ever-expanding role in health care management and delivery. Beyond their use as interventions, DHTs also serve as a vehicle for real-world data collection to characterize patients, their care journeys, and their responses to other clinical interventions. There is a need to comprehensively map the evidence—across all conditions and technology types—on DHT measurement of patient outcomes in the real world. Objective: We aimed to investigate the use of DHTs to measure real-world clinical outcomes using patient-generated data. Methods: We conducted this systematic scoping review in accordance with the Joanna Briggs Institute methodology. Detailed eligibility criteria documented in a preregistered protocol informed a search strategy for the following databases: MEDLINE (Ovid), CINAHL, Cochrane (CENTRAL), Embase, PsycINFO, ClinicalTrials.gov, and the EU Clinical Trials Register. We considered studies published between 2000 and 2022 wherein digital health data were collected, passively or actively, from patients with any specified health condition outside of clinical visits. Categories for key concepts, such as DHT type and analytical applications, were established where needed. Following screening and full-text review, data were extracted and analyzed using predefined fields, and findings were reported in accordance with established guidelines. Results: The search strategy identified 11,015 publications, with 7308 records after duplicates and reviews were removed. After screening and full-text review, 510 studies were included for extraction. These studies encompassed 169 different conditions in over 20 therapeutic areas and 44 countries. The DHTs used for mental health and addictions research (111/510, 21.8%) were the most prevalent. The most common type of DHT, mobile apps, was observed in approximately half of the studies (250/510, 49%). Most studies used only 1 DHT (346/510, 67.8%); however, the majority of technologies used were able to collect more than 1 type of data, with the most common being physiological data (189/510, 37.1%), clinical symptoms data (188/510, 36.9%), and behavioral data (171/510, 33.5%). Overall, there has been real growth in the depth and breadth of evidence, number of DHT types, and use of artificial intelligence and advanced analytics over time. Conclusions: This scoping review offers a comprehensive view of the variety of types of technology, data, collection methods, analytical approaches, and therapeutic applications within this growing body of evidence. To unlock the full potential of DHT for measuring health outcomes and capturing digital biomarkers, there is a need for more rigorous research that goes beyond technology validation to demonstrate whether robust real-world data can be reliably captured from patients in their daily life and whether its capture improves patient outcomes. This study provides a valuable repository of DHT studies to inform subsequent research by health care providers, policy makers, and the life sciences industry. Trial Registration: Open Science Framework 5TMKY; https://osf.io/5tmky/ %M 37819698 %R 10.2196/46992 %U https://www.jmir.org/2023/1/e46992 %U https://doi.org/10.2196/46992 %U http://www.ncbi.nlm.nih.gov/pubmed/37819698 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e46379 %T Implementing Electronic Discharge Communication Tools in Pediatric Emergency Departments: Multicountry, Cross-Sectional Readiness Survey of Nurses and Physicians %A Curran,Janet %A Wozney,Lori %A Tavender,Emma %A Wilson,Catherine %A Ritchie,Krista C %A Wong,Helen %A Gallant,Allyson %A Somerville,Mari %A Archambault,Patrick M %A Cassidy,Christine %A Jabbour,Mona %A Mackay,Rebecca %A Plint,Amy C %+ IWK Health Centre, 5850/5980 University Ave, Halifax, NS, B3K 6R8, Canada, 1 9027199285, lori.wozney@iwk.nshealth.ca %K discharge communication %K pediatric %K emergency department %K medical informatics %K implementation science %K electronic medical record %K mobile phone %D 2023 %7 11.10.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Pediatric emergency departments (ED) in many countries are implementing electronic tools such as kiosks, mobile apps, and electronic patient portals, to improve the effectiveness of discharge communication. Objective: This study aimed to survey nurse and physician readiness to adopt these tools. Methods: An electronic, cross-sectional survey was distributed to a convenience sample of currently practicing ED nurses and physicians affiliated with national pediatric research organizations in Canada, Australia, and New Zealand. Survey development was informed by the nonadoption, abandonment, scale-up, spread, sustainability framework. Measures of central tendency, and parametric and nonparametric tests were used to describe and compare nurse and physician responses. Results: Out of the 270 participants, the majority were physicians (61%, 164/270), female (65%, 176/270), and had 5 or more years of ED experience (76%, 205/270). There were high levels of consensus related to the value proposition of electronic discharge communication tools (EDCTs) with 82% (221/270) of them agreeing that they help parents and patients with comprehension and recall. Lower levels of consensus were observed for organizational factors with only 37% (100/270) agreeing that their staff is equipped to handle challenges with communication technologies. Nurses and physicians showed significant differences on 3 out of 21 readiness factors. Compared to physicians, nurses were significantly more likely to report that EDs have a responsibility to integrate EDCTs as part of a modern system (P<.001) and that policies are in place to guide safe and secure electronic communication (P=.02). Physicians were more likely to agree that using an EDCT would change their routine tasks (P=.04). One third (33%, 89/270) of participants indicated that they use or have used EDCT. Conclusions: Despite low levels of uptake, both nurses and physicians in multiple countries view EDCTs as a valuable support to families visiting pediatric ED. Leadership for technology change, unclear impact on workflow, and disparities in digital literacy skills require focused research effort. %M 37819696 %R 10.2196/46379 %U https://humanfactors.jmir.org/2023/1/e46379 %U https://doi.org/10.2196/46379 %U http://www.ncbi.nlm.nih.gov/pubmed/37819696 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e48808 %T ChatGPT-Generated Differential Diagnosis Lists for Complex Case–Derived Clinical Vignettes: Diagnostic Accuracy Evaluation %A Hirosawa,Takanobu %A Kawamura,Ren %A Harada,Yukinori %A Mizuta,Kazuya %A Tokumasu,Kazuki %A Kaji,Yuki %A Suzuki,Tomoharu %A Shimizu,Taro %+ Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, 880 Kitakobayashi, Mibu-cho, Shimotsuga, Tochigi, 321-0293, Japan, 81 282861111, hirosawa@dokkyomed.ac.jp %K artificial intelligence %K AI chatbot %K ChatGPT %K large language models %K clinical decision support %K natural language processing %K diagnostic excellence %K language model %K vignette %K case study %K diagnostic %K accuracy %K decision support %K diagnosis %D 2023 %7 9.10.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: The diagnostic accuracy of differential diagnoses generated by artificial intelligence chatbots, including ChatGPT models, for complex clinical vignettes derived from general internal medicine (GIM) department case reports is unknown. Objective: This study aims to evaluate the accuracy of the differential diagnosis lists generated by both third-generation ChatGPT (ChatGPT-3.5) and fourth-generation ChatGPT (ChatGPT-4) by using case vignettes from case reports published by the Department of GIM of Dokkyo Medical University Hospital, Japan. Methods: We searched PubMed for case reports. Upon identification, physicians selected diagnostic cases, determined the final diagnosis, and displayed them into clinical vignettes. Physicians typed the determined text with the clinical vignettes in the ChatGPT-3.5 and ChatGPT-4 prompts to generate the top 10 differential diagnoses. The ChatGPT models were not specially trained or further reinforced for this task. Three GIM physicians from other medical institutions created differential diagnosis lists by reading the same clinical vignettes. We measured the rate of correct diagnosis within the top 10 differential diagnosis lists, top 5 differential diagnosis lists, and the top diagnosis. Results: In total, 52 case reports were analyzed. The rates of correct diagnosis by ChatGPT-4 within the top 10 differential diagnosis lists, top 5 differential diagnosis lists, and top diagnosis were 83% (43/52), 81% (42/52), and 60% (31/52), respectively. The rates of correct diagnosis by ChatGPT-3.5 within the top 10 differential diagnosis lists, top 5 differential diagnosis lists, and top diagnosis were 73% (38/52), 65% (34/52), and 42% (22/52), respectively. The rates of correct diagnosis by ChatGPT-4 were comparable to those by physicians within the top 10 (43/52, 83% vs 39/52, 75%, respectively; P=.47) and within the top 5 (42/52, 81% vs 35/52, 67%, respectively; P=.18) differential diagnosis lists and top diagnosis (31/52, 60% vs 26/52, 50%, respectively; P=.43) although the difference was not significant. The ChatGPT models’ diagnostic accuracy did not significantly vary based on open access status or the publication date (before 2011 vs 2022). Conclusions: This study demonstrates the potential diagnostic accuracy of differential diagnosis lists generated using ChatGPT-3.5 and ChatGPT-4 for complex clinical vignettes from case reports published by the GIM department. The rate of correct diagnoses within the top 10 and top 5 differential diagnosis lists generated by ChatGPT-4 exceeds 80%. Although derived from a limited data set of case reports from a single department, our findings highlight the potential utility of ChatGPT-4 as a supplementary tool for physicians, particularly for those affiliated with the GIM department. Further investigations should explore the diagnostic accuracy of ChatGPT by using distinct case materials beyond its training data. Such efforts will provide a comprehensive insight into the role of artificial intelligence in enhancing clinical decision-making. %M 37812468 %R 10.2196/48808 %U https://medinform.jmir.org/2023/1/e48808 %U https://doi.org/10.2196/48808 %U http://www.ncbi.nlm.nih.gov/pubmed/37812468 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e46809 %T Toward Shared Decision-Making in Degenerative Cervical Myelopathy: Protocol for a Mixed Methods Study %A Sangeorzan,Irina %A Antonacci,Grazia %A Martin,Anne %A Grodzinski,Ben %A Zipser,Carl M %A Murphy,Rory K J %A Andriopoulou,Panoraia %A Cook,Chad E %A Anderson,David B %A Guest,James %A Furlan,Julio C %A Kotter,Mark R N %A Boerger,Timothy F %A Sadler,Iwan %A Roberts,Elizabeth A %A Wood,Helen %A Fraser,Christine %A Fehlings,Michael G %A Kumar,Vishal %A Jung,Josephine %A Milligan,James %A Nouri,Aria %A Martin,Allan R %A Blizzard,Tammy %A Vialle,Luiz Roberto %A Tetreault,Lindsay %A Kalsi-Ryan,Sukhvinder %A MacDowall,Anna %A Martin-Moore,Esther %A Burwood,Martin %A Wood,Lianne %A Lalkhen,Abdul %A Ito,Manabu %A Wilson,Nicky %A Treanor,Caroline %A Dugan,Sheila %A Davies,Benjamin M %+ Department of Clinical Neurosurgery, University of Cambridge, Box 167, Cambridge Biomedical Campus, Addenbrooke’s Hospital, Cambridge, CB2 0QQ, United Kingdom, 44 7766 692608, bd375@cam.ac.uk %K degenerative cervical myelopathy %K spine %K spinal cord %K chronic %K aging %K geriatric %K patient engagement %K shared decision-making %K process mapping %K core information set %K decision-making %K patient education %K common data element %K Research Objectives and Common Data Elements for Degenerative Cervical Myelopathy %K RECODE-DCM %D 2023 %7 9.10.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Health care decisions are a critical determinant in the evolution of chronic illness. In shared decision-making (SDM), patients and clinicians work collaboratively to reach evidence-based health decisions that align with individual circumstances, values, and preferences. This personalized approach to clinical care likely has substantial benefits in the oversight of degenerative cervical myelopathy (DCM), a type of nontraumatic spinal cord injury. Its chronicity, heterogeneous clinical presentation, complex management, and variable disease course engenders an imperative for a patient-centric approach that accounts for each patient’s unique needs and priorities. Inadequate patient knowledge about the condition and an incomplete understanding of the critical decision points that arise during the course of care currently hinder the fruitful participation of health care providers and patients in SDM. This study protocol presents the rationale for deploying SDM for DCM and delineates the groundwork required to achieve this. Objective: The study’s primary outcome is the development of a comprehensive checklist to be implemented upon diagnosis that provides patients with essential information necessary to support their informed decision-making. This is known as a core information set (CIS). The secondary outcome is the creation of a detailed process map that provides a diagrammatic representation of the global care workflows and cognitive processes involved in DCM care. Characterizing the critical decision points along a patient’s journey will allow for an effective exploration of SDM tools for routine clinical practice to enhance patient-centered care and improve clinical outcomes. Methods: Both CISs and process maps are coproduced iteratively through a collaborative process involving the input and consensus of key stakeholders. This will be facilitated by Myelopathy.org, a global DCM charity, through its Research Objectives and Common Data Elements for Degenerative Cervical Myelopathy community. To develop the CIS, a 3-round, web-based Delphi process will be used, starting with a baseline list of information items derived from a recent scoping review of educational materials in DCM, patient interviews, and a qualitative survey of professionals. A priori criteria for achieving consensus are specified. The process map will be developed iteratively using semistructured interviews with patients and professionals and validated by key stakeholders. Results: Recruitment for the Delphi consensus study began in April 2023. The pilot-testing of process map interview participants started simultaneously, with the formulation of an initial baseline map underway. Conclusions: This protocol marks the first attempt to provide a starting point for investigating SDM in DCM. The primary work centers on developing an educational tool for use in diagnosis to enable enhanced onward decision-making. The wider objective is to aid stakeholders in developing SDM tools by identifying critical decision junctures in DCM care. Through these approaches, we aim to provide an exhaustive launchpad for formulating SDM tools in the wider DCM community. International Registered Report Identifier (IRRID): DERR1-10.2196/46809 %M 37812472 %R 10.2196/46809 %U https://www.researchprotocols.org/2023/1/e46809 %U https://doi.org/10.2196/46809 %U http://www.ncbi.nlm.nih.gov/pubmed/37812472 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e48583 %T Characterizing the Patterns of Electronic Health Record–Integrated Secure Messaging Use: Cross-Sectional Study %A Baratta,Laura R %A Harford,Derek %A Sinsky,Christine A %A Kannampallil,Thomas %A Lou,Sunny S %+ Department of Anesthesiology, Washington University School of Medicine, 660 South Euclid, Campus Box 8054, Saint Louis, MO, 63110, United States, 1 314 362 1196, slou@wustl.edu %K clinical care %K clinician burden %K communication %K electronic health record %K EHR %K interprofessional communication %K medical assistant %K messaging %K nurses %K observational study %K physicians %K secure messaging %K users %D 2023 %7 6.10.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Communication among health care professionals is essential for the delivery of safe clinical care. Secure messaging has rapidly emerged as a new mode of asynchronous communication. Despite its popularity, relatively little is known about how secure messaging is used and how such use contributes to communication burden. Objective: This study aims to characterize the use of an electronic health record–integrated secure messaging platform across 14 hospitals and 263 outpatient clinics within a large health care system. Methods: We collected metadata on the use of the Epic Systems Secure Chat platform for 6 months (July 2022 to January 2023). Information was retrieved on message volume, response times, message characteristics, messages sent and received by users, user roles, and work settings (inpatient vs outpatient). Results: A total of 32,881 users sent 9,639,149 messages during the study. Median daily message volume was 53,951 during the first 2 weeks of the study and 69,526 during the last 2 weeks, resulting in an overall increase of 29% (P=.03). Nurses were the most frequent users of secure messaging (3,884,270/9,639,149, 40% messages), followed by physicians (2,387,634/9,639,149, 25% messages), and medical assistants (1,135,577/9,639,149, 12% messages). Daily message frequency varied across users; inpatient advanced practice providers and social workers interacted with the highest number of messages per day (median 19). Conversations were predominantly between 2 users (1,258,036/1,547,879, 81% conversations), with a median of 2 conversational turns and a median response time of 2.4 minutes. The largest proportion of inpatient messages was from nurses to physicians (972,243/4,749,186, 20% messages) and physicians to nurses (606,576/4,749,186, 13% messages), while the largest proportion of outpatient messages was from physicians to nurses (344,048/2,192,488, 16% messages) and medical assistants to other medical assistants (236,694/2,192,488, 11% messages). Conclusions: Secure messaging was widely used by a diverse range of health care professionals, with ongoing growth throughout the study and many users interacting with more than 20 messages per day. The short message response times and high messaging volume observed highlight the interruptive nature of secure messaging, raising questions about its potentially harmful effects on clinician workflow, cognition, and errors. %M 37801359 %R 10.2196/48583 %U https://www.jmir.org/2023/1/e48583 %U https://doi.org/10.2196/48583 %U http://www.ncbi.nlm.nih.gov/pubmed/37801359 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e49944 %T A Natural Language Processing Model for COVID-19 Detection Based on Dutch General Practice Electronic Health Records by Using Bidirectional Encoder Representations From Transformers: Development and Validation Study %A Homburg,Maarten %A Meijer,Eline %A Berends,Matthijs %A Kupers,Thijmen %A Olde Hartman,Tim %A Muris,Jean %A de Schepper,Evelien %A Velek,Premysl %A Kuiper,Jeroen %A Berger,Marjolein %A Peters,Lilian %+ Department of Primary- and Long-Term Care, University Medical Center Groningen, Home Post Code FA21, PO Box 196, Groningen, 9700 RB, Netherlands, 31 050 3616161, t.m.homburg@umcg.nl %K natural language processing %K primary care %K COVID-19 %K EHR %K electronic health records %K public health %K multidisciplinary %K NLP %K disease identification %K BERT model %K model development %K prediction %D 2023 %7 4.10.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Natural language processing (NLP) models such as bidirectional encoder representations from transformers (BERT) hold promise in revolutionizing disease identification from electronic health records (EHRs) by potentially enhancing efficiency and accuracy. However, their practical application in practice settings demands a comprehensive and multidisciplinary approach to development and validation. The COVID-19 pandemic highlighted challenges in disease identification due to limited testing availability and challenges in handling unstructured data. In the Netherlands, where general practitioners (GPs) serve as the first point of contact for health care, EHRs generated by these primary care providers contain a wealth of potentially valuable information. Nonetheless, the unstructured nature of free-text entries in EHRs poses challenges in identifying trends, detecting disease outbreaks, or accurately pinpointing COVID-19 cases. Objective: This study aims to develop and validate a BERT model for detecting COVID-19 consultations in general practice EHRs in the Netherlands. Methods: The BERT model was initially pretrained on Dutch language data and fine-tuned using a comprehensive EHR data set comprising confirmed COVID-19 GP consultations and non–COVID-19–related consultations. The data set was partitioned into a training and development set, and the model’s performance was evaluated on an independent test set that served as the primary measure of its effectiveness in COVID-19 detection. To validate the final model, its performance was assessed through 3 approaches. First, external validation was applied on an EHR data set from a different geographic region in the Netherlands. Second, validation was conducted using results of polymerase chain reaction (PCR) test data obtained from municipal health services. Lastly, correlation between predicted outcomes and COVID-19–related hospitalizations in the Netherlands was assessed, encompassing the period around the outbreak of the pandemic in the Netherlands, that is, the period before widespread testing. Results: The model development used 300,359 GP consultations. We developed a highly accurate model for COVID-19 consultations (accuracy 0.97, F1-score 0.90, precision 0.85, recall 0.85, specificity 0.99). External validations showed comparable high performance. Validation on PCR test data showed high recall but low precision and specificity. Validation using hospital data showed significant correlation between COVID-19 predictions of the model and COVID-19–related hospitalizations (F1-score 96.8; P<.001; R2=0.69). Most importantly, the model was able to predict COVID-19 cases weeks before the first confirmed case in the Netherlands. Conclusions: The developed BERT model was able to accurately identify COVID-19 cases among GP consultations even preceding confirmed cases. The validated efficacy of our BERT model highlights the potential of NLP models to identify disease outbreaks early, exemplifying the power of multidisciplinary efforts in harnessing technology for disease identification. Moreover, the implications of this study extend beyond COVID-19 and offer a blueprint for the early recognition of various illnesses, revealing that such models could revolutionize disease surveillance. %M 37792444 %R 10.2196/49944 %U https://www.jmir.org/2023/1/e49944 %U https://doi.org/10.2196/49944 %U http://www.ncbi.nlm.nih.gov/pubmed/37792444 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 11 %N %P e49995 %T Comparison of Diagnostic and Triage Accuracy of Ada Health and WebMD Symptom Checkers, ChatGPT, and Physicians for Patients in an Emergency Department: Clinical Data Analysis Study %A Fraser,Hamish %A Crossland,Daven %A Bacher,Ian %A Ranney,Megan %A Madsen,Tracy %A Hilliard,Ross %+ Brown Center for Biomedical Informatics, The Warren Alpert Medical School of Brown University, 233 Richmond Street, Providence, RI, 02912, United States, 1 401863 1815, hamish_fraser@brown.edu %K diagnosis %K triage %K symptom checker %K emergency patient %K ChatGPT %K LLM %K diagnose %K self-diagnose %K self-diagnosis %K app %K application %K language model %K accuracy %K ChatGPT-3.5 %K ChatGPT-4.0 %K emergency %K machine learning %D 2023 %7 3.10.2023 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: Diagnosis is a core component of effective health care, but misdiagnosis is common and can put patients at risk. Diagnostic decision support systems can play a role in improving diagnosis by physicians and other health care workers. Symptom checkers (SCs) have been designed to improve diagnosis and triage (ie, which level of care to seek) by patients. Objective: The aim of this study was to evaluate the performance of the new large language model ChatGPT (versions 3.5 and 4.0), the widely used WebMD SC, and an SC developed by Ada Health in the diagnosis and triage of patients with urgent or emergent clinical problems compared with the final emergency department (ED) diagnoses and physician reviews. Methods: We used previously collected, deidentified, self-report data from 40 patients presenting to an ED for care who used the Ada SC to record their symptoms prior to seeing the ED physician. Deidentified data were entered into ChatGPT versions 3.5 and 4.0 and WebMD by a research assistant blinded to diagnoses and triage. Diagnoses from all 4 systems were compared with the previously abstracted final diagnoses in the ED as well as with diagnoses and triage recommendations from three independent board-certified ED physicians who had blindly reviewed the self-report clinical data from Ada. Diagnostic accuracy was calculated as the proportion of the diagnoses from ChatGPT, Ada SC, WebMD SC, and the independent physicians that matched at least one ED diagnosis (stratified as top 1 or top 3). Triage accuracy was calculated as the number of recommendations from ChatGPT, WebMD, or Ada that agreed with at least 2 of the independent physicians or were rated “unsafe” or “too cautious.” Results: Overall, 30 and 37 cases had sufficient data for diagnostic and triage analysis, respectively. The rate of top-1 diagnosis matches for Ada, ChatGPT 3.5, ChatGPT 4.0, and WebMD was 9 (30%), 12 (40%), 10 (33%), and 12 (40%), respectively, with a mean rate of 47% for the physicians. The rate of top-3 diagnostic matches for Ada, ChatGPT 3.5, ChatGPT 4.0, and WebMD was 19 (63%), 19 (63%), 15 (50%), and 17 (57%), respectively, with a mean rate of 69% for physicians. The distribution of triage results for Ada was 62% (n=23) agree, 14% unsafe (n=5), and 24% (n=9) too cautious; that for ChatGPT 3.5 was 59% (n=22) agree, 41% (n=15) unsafe, and 0% (n=0) too cautious; that for ChatGPT 4.0 was 76% (n=28) agree, 22% (n=8) unsafe, and 3% (n=1) too cautious; and that for WebMD was 70% (n=26) agree, 19% (n=7) unsafe, and 11% (n=4) too cautious. The unsafe triage rate for ChatGPT 3.5 (41%) was significantly higher (P=.009) than that of Ada (14%). Conclusions: ChatGPT 3.5 had high diagnostic accuracy but a high unsafe triage rate. ChatGPT 4.0 had the poorest diagnostic accuracy, but a lower unsafe triage rate and the highest triage agreement with the physicians. The Ada and WebMD SCs performed better overall than ChatGPT. Unsupervised patient use of ChatGPT for diagnosis and triage is not recommended without improvements to triage accuracy and extensive clinical evaluation. %M 37788063 %R 10.2196/49995 %U https://mhealth.jmir.org/2023/1/e49995 %U https://doi.org/10.2196/49995 %U http://www.ncbi.nlm.nih.gov/pubmed/37788063 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e46491 %T Implementing Technologies to Enhance Coordinated Specialty Care Framework: Implementation Outcomes From a Development and Usability Study %A Green,James B %A Rodriguez,Joey %A Keshavan,Matcheri %A Lizano,Paulo %A Torous,John %+ Beth Israel Deaconess Medical Center, Harvard Medical School, 330 Brookline Avenue, Boston, MA, 02215, United States, 1 508 397 0853, jgreen8@bidmc.harvard.edu %K psychosis %K digital health %K digital mental health %K coordinated specialty care %K digital navigator %K clinical high risk %K schizophrenia %K implementation science %K technology %K mobile phone %D 2023 %7 3.10.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Coordinated specialty care (CSC) has demonstrated efficacy in improving outcomes in individuals at clinical high risk for psychosis and individuals with first-episode psychosis. Given the limitations of scalability and staffing needs, the augmentation of services using digital mental health interventions (DMHIs) may be explored to help support CSC service delivery. Objective: In this study, we aimed to understand the methods to implement and support technology in routine CSC and offered insights from a quality improvement study assessing the implementation outcomes of DMHIs in CSC. Methods: Patients and clinicians including psychiatrists, therapists, and supported education and employment specialists from a clinical-high-risk-for-psychosis clinic (Center for Early Detection Assessment and Response to Risk [CEDAR]) and a first-episode–psychosis clinic (Advancing Services for Psychosis Integration and Recovery [ASPIRE]) participated in a quality improvement project exploring the feasibility of DMHIs following the Access, Alignment, Connection, Care, and Scalability framework to implement mindLAMP, a flexible and evidenced-based DMHI. Digital navigators were used at each site to assist clinicians and patients in implementing mindLAMP. To explore the differences in implementation outcomes associated with the app format, a menu-style format was delivered at CEDAR, and a modular approach was used at ASPIRE. Qualitative baseline and follow-up data were collected to assess the specific implementation outcomes. Results: In total, 5 patients (ASPIRE: n=3, 60%; CEDAR: n=2, 40%) were included: 3 (60%) White individuals, 2 (40%) male and 2 (40%) female patients, and 1 (20%) transgender man, with a mean age of 19.6 (SD 2.05) years. Implementation outcome data revealed that patients and clinicians demonstrated high accessibility, acceptability, interest, and belief in the sustainability of DMHIs. Clinicians and patients presented a wide range of interest in unique use cases of DMHI in CSC and expressed variable feasibility and appropriateness associated with nuanced barriers and needs. In addition, the results suggest that adoption, penetration, feasibility, and appropriateness outcomes were moderate and might continue to be explored and targeted. Conclusions: Implementation outcomes from this project suggest the need for a patient- and clinician-centered approach that is guided by digital navigators and provides versatility, autonomy, and structure. Leveraging these insights has the potential to build on growing research regarding the need for versatility, autonomy, digital navigator support, and structured applications. We anticipate that by continuing to research and improve implementation barriers impeding the adoption and penetration of DMHIs in CSC, accessibility and uptake of DMHIs will improve, therefore connecting patients to the demonstrated benefits of technology-augmented care. %M 37788066 %R 10.2196/46491 %U https://formative.jmir.org/2023/1/e46491 %U https://doi.org/10.2196/46491 %U http://www.ncbi.nlm.nih.gov/pubmed/37788066 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e46381 %T Evaluation of a Digital Decision Support System to Integrate Type 2 Diabetes Mellitus and Periodontitis Care: Case-Vignette Study in Simulated Environments %A Kalmus,Olivier %A Smits,Kirsten %A Seitz,Max %A Haux,Christian %A Robra,Bernt-Peter %A Listl,Stefan %+ Department of Dentistry, Quality and Safety of Oral Healthcare, Radboud Institute for Health Sciences, Radboud University Medical Center, Philips van Leydenlaan 25, Nijmegen, 6533XL, Netherlands, 31 652735619, stefan.listl@radboudumc.nl %K digital health %K integrated care %K decision support %K oral health %K diabetes %K periodontitis %K decision support %K oral care %K type 2 diabetes %K evaluation %K survey %K hemoglobin %K diagnostic device %K telemedicine %D 2023 %7 2.10.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: As highlighted by the recent World Health Organization Oral Health Resolution, there is an urgent need to better integrate primary and oral health care. Despite evidence and guidelines substantiating the relevance of integrating type 2 diabetes mellitus (T2DM) and periodontitis care, the fragmentation of primary and oral health care persists. Objective: This paper reports on the evaluation of a prototype digital decision support system (DSS) that was developed to enhance the integration of T2DM and periodontitis care. Methods: The effects of the prototype DSS were assessed in web-based simulated environments, using 2 different sets of case vignettes in combination with evaluation surveys among 202 general dental practitioners (GDPs) and 206 general practitioners (GPs). Each participant evaluated 3 vignettes, one of which, chosen at random, was assisted by the DSS. Logistic regression analyses were conducted at the participant and case levels. Results: Under DSS assistance, GPs had 8.3 (95% CI 4.32-16.03) times higher odds of recommending a GDP visit. There was no significant impact of DSS assistance on GP advice about common risk factors for T2DM and periodontal disease. GDPs had 4.3 (95% CI 2.08-9.04) times higher odds of recommending a GP visit, 1.6 (95% CI 1.03-2.33) times higher odds of giving advice on disease correlations, and 3.2 (95% CI 1.63-6.35) times higher odds of asking patients about their glycated hemoglobin value. Conclusions: The findings of this study provide a proof of concept for a digital DSS to integrate T2DM and periodontal care. Future updating and testing is warranted to continuously enhance the functionalities of the DSS in terms of interoperability with various types of data sources and diagnostic devices; incorporation of other (oral) health dimensions; application in various settings, including via telemedicine; and further customization of end-user interfaces. %M 37782539 %R 10.2196/46381 %U https://www.jmir.org/2023/1/e46381 %U https://doi.org/10.2196/46381 %U http://www.ncbi.nlm.nih.gov/pubmed/37782539 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e46521 %T Assessment of Upper Extremity Function in Multiple Sclerosis: Feasibility of a Digital Pinching Test %A Graves,Jennifer S %A Elantkowski,Marcin %A Zhang,Yan-Ping %A Dondelinger,Frank %A Lipsmeier,Florian %A Bernasconi,Corrado %A Montalban,Xavier %A Midaglia,Luciana %A Lindemann,Michael %+ F. Hoffmann-La Roche Ltd, Grenzacherstrasse 124, Basel, 4070, Switzerland, 41 61 687 79 09, florian.lipsmeier@roche.com %K multiple sclerosis %K smartphone sensor %K digital health technology tools %K upper extremity function %K hand-motor dexterity %D 2023 %7 2.10.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: The development of touchscreen-based assessments of upper extremity function could benefit people with multiple sclerosis (MS) by allowing convenient, quantitative assessment of their condition. The Pinching Test forms a part of the Floodlight smartphone app (F. Hoffmann-La Roche Ltd, Basel, Switzerland) for people with MS and was designed to capture upper extremity function. Objective: This study aimed to evaluate the Pinching Test as a tool for remotely assessing upper extremity function in people with MS. Methods: Using data from the 24-week, prospective feasibility study investigating the Floodlight Proof-of-Concept app for remotely assessing MS, we examined 13 pinching, 11 inertial measurement unit (IMU)–based, and 13 fatigability features of the Pinching Test. We assessed the test-retest reliability using intraclass correlation coefficients [second model, first type; ICC(2,1)], age- and sex-adjusted cross-sectional Spearman rank correlation, and known-groups validity (data aggregation: median [all features], SD [fatigability features]). Results: We evaluated data from 67 people with MS (mean Expanded Disability Status Scale [EDSS]: 2.4 [SD 1.4]) and 18 healthy controls. In this cohort of early MS, pinching features were reliable [ICC(2,1)=0.54-0.81]; correlated with standard clinical assessments, including the Nine-Hole Peg Test (9HPT) (|r|=0.26-0.54; 10/13 features), EDSS (|r|=0.25-0.36; 7/13 features), and the arm items of the 29-item Multiple Sclerosis Impact Scale (MSIS-29) (|r|=0.31-0.52; 7/13 features); and differentiated people with MS-Normal from people with MS-Abnormal (area under the curve: 0.68-0.78; 8/13 features). IMU-based features showed similar test-retest reliability [ICC(2,1)=0.47-0.84] but showed little correlations with standard clinical assessments. In contrast, fatigability features (SD aggregation) correlated with 9HPT time (|r|=0.26-0.61; 10/13 features), EDSS (|r|=0.26-0.41; 8/13 features), and MSIS-29 arm items (|r|=0.32-0.46; 7/13 features). Conclusions: The Pinching Test provides a remote, objective, and granular assessment of upper extremity function in people with MS that can potentially complement standard clinical evaluation. Future studies will validate it in more advanced MS. Trial Registration: ClinicalTrials.gov NCT02952911; https://clinicaltrials.gov/study/NCT02952911 %M 37782540 %R 10.2196/46521 %U https://formative.jmir.org/2023/1/e46521 %U https://doi.org/10.2196/46521 %U http://www.ncbi.nlm.nih.gov/pubmed/37782540 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e49804 %T Identifying Inequities in Video and Audio Telehealth Services for Primary Care Encounters During COVID-19: Repeated Cross-Sectional, Observational Study %A Buis,Lorraine R %A Brown,Lindsay K %A Plegue,Melissa A %A Kadri,Reema %A Laurie,Anna R %A Guetterman,Timothy C %A Vydiswaran,V G Vinod %A Li,Jiazhao %A Veinot,Tiffany C %+ Department of Family Medicine, University of Michigan, 1018 Fuller Street, Ann Arbor, MI, 48104, United States, 1 734 998 7120, buisl@umich.edu %K COVID-19 %K telemedicine %K health equity %K clinical encounters %K electronic health records %D 2023 %7 29.9.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: The COVID-19 pandemic resulted in rapid changes in how patient care was provided, particularly through the expansion of telehealth and audio-only phone-based care. Objective: The goal of this study was to evaluate inequities in video and audio-only care during various time points including the initial wave of the COVID-19 pandemic, later stages of the pandemic, and a historical control. We sought to understand the characteristics of care during this time for a variety of different groups of patients that may experience health care inequities. Methods: We conducted a retrospective analysis of electronic health record (EHR) data from encounters from 34 family medicine and internal medicine primary care clinics in a large, Midwestern health system, using a repeated cross-sectional, observational study design. These data included patient demographic data, as well as encounter, diagnosis, and procedure records. Data were obtained for all in-person and telehealth encounters (including audio-only phone-based care) that occurred during 3 separate time periods: an initial COVID-19 period (T2: March 16, 2020, to May 3, 2020), a later COVID-19 period (T3: May 4, 2020, to September 30, 2020), and a historical control period from the previous year (T1: March 16, 2019, to September 30, 2019). Primary analysis focused on the status of each encounter in terms of whether it was completed as scheduled, it was canceled, or the patient missed the appointment. A secondary analysis was performed to evaluate the likelihood of an encounter being completed based on visit modality (phone, video, in-person). Results: In total, there were 938,040 scheduled encounters during the 3 time periods, with 178,747 unique patients, that were included for analysis. Patients with completed encounters were more likely to be younger than 65 years old (71.8%-74.1%), be female (58.8%-61.8%), be White (75.6%-76.7%), and have no significant comorbidities (63.2%-66.8%) or disabilities (53.2%-61.1%) in all time periods than those who had only canceled or missed encounters. Effects on different subpopulations are discussed herein. Conclusions: Findings from this study demonstrate that primary care utilization across delivery modalities (in person, video, and phone) was not equivalent across all groups before and during the COVID-19 pandemic and different groups were differentially impacted at different points. Understanding how different groups of patients responded to these rapid changes and how health care inequities may have been affected is an important step in better understanding implementation strategies for digital solutions in the future. %M 37773609 %R 10.2196/49804 %U https://www.jmir.org/2023/1/e49804 %U https://doi.org/10.2196/49804 %U http://www.ncbi.nlm.nih.gov/pubmed/37773609 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e48976 %T A Web-Based Tool to Report Adverse Drug Reactions by Community Pharmacists in Australia: Usability Testing Study %A Fossouo Tagne,Joel %A Yakob,Reginald Amin %A Mcdonald,Rachael %A Wickramasinghe,Nilmini %+ School of Health Sciences and Biostatistics, Swinburne University of Technology, John Street, Hawthorn, Melbourne, 3122, Australia, 61 0412478610, jfossouo@gmail.com %K ADR %K adverse drug reaction %K pharmacovigilance %K community pharmacy %K digital health evaluation %K usability testing %D 2023 %7 29.9.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Adverse drug reactions (ADRs) are unintended and harmful events associated with medication use. Despite their significance in postmarketing surveillance, quality improvement, and drug safety research, ADRs are vastly underreported. Enhanced digital-based communication of ADR information to regulators and among care providers could significantly improve patient safety. Objective: This paper presents a usability evaluation of the commercially available GuildCare Adverse Event Recording system, a web-based ADR reporting system widely used by community pharmacists (CPs) in Australia. Methods: We developed a structured interview protocol encompassing remote observation, think-aloud moderating techniques, and retrospective questioning to gauge the overall user experience, complemented by the System Usability Scale (SUS) assessment. Thematic analysis was used to analyze field notes from the interviews. Results: A total of 7 CPs participated in the study, who perceived the system to have above-average usability (SUS score of 68.57). Nonetheless, the structured approach to usability testing unveiled specific functional and user interpretation issues, such as unnecessary information, lack of system clarity, and redundant data fields—critical insights not captured by the SUS results. Design elements like drop-down menus, free-text entry, checkboxes, and prefilled or auto-populated data fields were perceived as useful for enhancing system navigation and facilitating ADR reporting. Conclusions: The user-centric design of technology solutions, like the one discussed herein, is crucial to meeting CPs’ information needs and ensuring effective ADR reporting. Developers should adopt a structured approach to usability testing during the developmental phase to address identified issues comprehensively. Such a methodological approach may promote the adoption of ADR reporting systems by CPs and ultimately enhance patient safety. %M 37773620 %R 10.2196/48976 %U https://formative.jmir.org/2023/1/e48976 %U https://doi.org/10.2196/48976 %U http://www.ncbi.nlm.nih.gov/pubmed/37773620 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e40959 %T Successes and Barriers of Health Information Exchange Participation Across Hospitals in South Carolina From 2014 to 2020: Longitudinal Observational Study %A Li,Zhong %A Merrell,Melinda A %A Eberth,Jan M %A Wu,Dezhi %A Hung,Peiyin %+ Rural and Minority Health Research Center, Arnold School of Public Health, University of South Carolina, 220 Stoneridge Dr, Suite 204, Columbia, SC, 29210, United States, 1 8037779867, hungp@mailbox.sc.edu %K health information exchange %K electronic health records %K interoperability %K meaningful use %K hospital %D 2023 %7 28.9.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: The 2009 Health Information Technology for Economic and Clinical Health Act sets three stages of Meaningful Use requirements for the electronic health records incentive program. Health information exchange (HIE) technologies are critical in the meaningful use of electronic health records to support patient care coordination. However, HIE use trends and barriers remain unclear across hospitals in South Carolina (SC), a state with the earliest HIE implementation. Objective: This study aims to explore changes in the proportion of HIE participation and factors associated with HIE participation, and barriers to exchange and interoperability across SC hospitals. Methods: This study derived data from a longitudinal data set of the 2014-2020 American Hospital Association Information Technology Supplement for 69 SC hospitals. The primary outcome was whether a hospital participated in HIE in a year. A cross-sectional multivariable logistic regression model, clustered at the hospital level and weighted by bed size, was used to identify factors associated with HIE participation. The second outcome was barriers to sending, receiving, or finding patient health information to or from other organizations or hospital systems. The frequency of hospitals reporting each barrier related to exchange and interoperability were then calculated. Results: Hospitals in SC have been increasingly participating in HIE, improving from 43% (24/56) in 2014 to 82% (54/66) in 2020. After controlling for other hospital factors, teaching hospitals (adjusted odds ratio [AOR] 3.7, 95% CI 1.0-13.3), system-affiliated hospitals (AOR 6.6, 95% CI 3.2-13.7), and rural referral hospitals (AOR 8.0, 95% CI 1.2-53.4) had higher odds to participate in HIE than their counterparts, whereas critical access hospitals (AOR 0.1, 95% CI 0.02-0.6) were less likely to participate in HIE than their counterparts reimbursed by the prospective payment system. Hospitals with greater ratios of Medicare or Medicaid inpatient days to total inpatient days also reported higher odds of HIE participation. Despite the majority of hospitals reporting HIE participation in 2020, barriers to exchange and interoperability remained, including lack of provider contacts (27/40, 68%), difficulty in finding patient health information (27/40, 68%), adapting different vendor platforms (26/40, 65%), difficulty matching or identifying same patients between systems (23/40, 58%), and providers that do not typically exchange patient data (23/40, 58%). Conclusions: HIE participation has been widely adopted in SC hospitals. Our findings highlight the need to incentivize optimization of HIE and seamless information exchange by facilitating and implementing standardization of health information across various HIE systems and by addressing other technical issues, including providing providers’ addresses and training HIE stakeholders to find relevant information. Policies and efforts should include more collaboration with vendors to reduce platform compatibility issues and more user engagement and technical training and support to facilitate effective, accurate, and efficient exchange of provider contacts and patient health information. %M 37768730 %R 10.2196/40959 %U https://medinform.jmir.org/2023/1/e40959 %U https://doi.org/10.2196/40959 %U http://www.ncbi.nlm.nih.gov/pubmed/37768730 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e47486 %T Toward Personalized Medicine Approaches for Parkinson Disease Using Digital Technologies %A Khanna,Amit %A Jones,Graham %+ GDD Connected Health and Innovation Group, Novartis Pharmaceuticals, 1 Health Plaza, East Hanover, NJ, 07936, United States, 1 8572757045, graham.jones@novartis.com %K digital health %K monitoring %K personalized medicine %K Parkinson disease %K wearables %K neurodegenerative disorder %K cognitive impairment %K economic burden %K digital technology %K symptom management %K disease control %K debilitating disease %K intervention %D 2023 %7 27.9.2023 %9 Viewpoint %J JMIR Form Res %G English %X Parkinson disease (PD) is a complex neurodegenerative disorder that afflicts over 10 million people worldwide, resulting in debilitating motor and cognitive impairment. In the United States alone (with approximately 1 million cases), the economic burden for treating and caring for persons with PD exceeds US $50 billion and myriad therapeutic approaches are under development, including both symptomatic- and disease-modifying agents. The challenges presented in addressing PD are compounded by observations that numerous, statistically distinct patient phenotypes present with a wide variety of motor and nonmotor symptomatic profiles, varying responses to current standard-of-care symptom-alleviating medications (L-DOPA and dopaminergic agonists), and different disease trajectories. The existence of these differing phenotypes highlights the opportunities in personalized approaches to symptom management and disease control. The prodromal period of PD can span across several decades, allowing the potential to leverage the unique array of composite symptoms presented to trigger early interventions. This may be especially beneficial as disease progression in PD (alongside Alzheimer disease and Huntington disease) may be influenced by biological processes such as oxidative stress, offering the potential for individual lifestyle factors to be tailored to delay disease onset. In this viewpoint, we offer potential scenarios where emerging diagnostic and monitoring strategies might be tailored to the individual patient under the tenets of P4 medicine (predict, prevent, personalize, and participate). These approaches may be especially relevant as the causative factors and biochemical pathways responsible for the observed neurodegeneration in patients with PD remain areas of fluid debate. The numerous observational patient cohorts established globally offer an excellent opportunity to test and refine approaches to detect, characterize, control, modify the course, and ultimately stop progression of this debilitating disease. Such approaches may also help development of parallel interventive strategies in other diseases such as Alzheimer disease and Huntington disease, which share common traits and etiologies with PD. In this overview, we highlight near-term opportunities to apply P4 medicine principles for patients with PD and introduce the concept of composite orthogonal patient monitoring. %M 37756050 %R 10.2196/47486 %U https://formative.jmir.org/2023/1/e47486 %U https://doi.org/10.2196/47486 %U http://www.ncbi.nlm.nih.gov/pubmed/37756050 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e49303 %T Centering Public Perceptions on Translating AI Into Clinical Practice: Patient and Public Involvement and Engagement Consultation Focus Group Study %A Lammons,William %A Silkens,Milou %A Hunter,Jamie %A Shah,Sudhir %A Stavropoulou,Charitini %+ National Institute of Health and Care Research, Applied Research Collaboration North Thames, Department of Applied Health Research, University College London, 1-19 Torrington Place, London, WC1E 7HB, United Kingdom, 44 (0)20 8059 0939, william.lammons@ucl.ac.uk %K acceptance %K AI in health care %K AI %K artificial intelligence %K health care research %K health care %K patient and public engagement and involvement %K patient engagement %K public engagement %K transition %D 2023 %7 26.9.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Artificial intelligence (AI) is widely considered to be the new technical advancement capable of a large-scale modernization of health care. Considering AI’s potential impact on the clinician-patient relationship, health care provision, and health care systems more widely, patients and the wider public should be a part of the development, implementation, and embedding of AI applications in health care. Failing to establish patient and public engagement and involvement (PPIE) can limit AI’s impact. Objective: This study aims to (1) understand patients’ and the public’s perceived benefits and challenges for AI and (2) clarify how to best conduct PPIE in projects on translating AI into clinical practice, given public perceptions of AI. Methods: We conducted this qualitative PPIE focus-group consultation in the United Kingdom. A total of 17 public collaborators representing 7 National Institute of Health and Care Research Applied Research Collaborations across England participated in 1 of 3 web-based semistructured focus group discussions. We explored public collaborators’ understandings, experiences, and perceptions of AI applications in health care. Transcripts were coanalyzed iteratively with 2 public coauthors using thematic analysis. Results: We identified 3 primary deductive themes with 7 corresponding inductive subthemes. Primary theme 1, advantages of implementing AI in health care, had 2 subthemes: system improvements and improve quality of patient care and shared decision-making. Primary theme 2, challenges of implementing AI in health care, had 3 subthemes: challenges with security, bias, and access; public misunderstanding of AI; and lack of human touch in care and decision-making. Primary theme 3, recommendations on PPIE for AI in health care, had 2 subthemes: experience, empowerment, and raising awareness; and acknowledging and supporting diversity in PPIE. Conclusions: Patients and the public can bring unique perspectives on the development, implementation, and embedding of AI in health care. Early PPIE is therefore crucial not only to safeguard patients but also to increase the chances of acceptance of AI by the public and the impact AI can make in terms of outcomes. %M 37751234 %R 10.2196/49303 %U https://www.jmir.org/2023/1/e49303 %U https://doi.org/10.2196/49303 %U http://www.ncbi.nlm.nih.gov/pubmed/37751234 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e49963 %T A Future of Smarter Digital Health Empowered by Generative Pretrained Transformer %A Miao,Hongyu %A Li,Chengdong %A Wang,Jing %+ College of Nursing, Florida State University, 98 Varsity Way, Tallahassee, FL, 32306, United States, 1 8506443299, JingWang@nursing.fsu.edu %K generative pretrained model %K artificial intelligence %K digital health %K generative pretrained transformer %K ChatGPT %K precision medicine %K AI %K privacy %K ethics %D 2023 %7 26.9.2023 %9 Viewpoint %J J Med Internet Res %G English %X Generative pretrained transformer (GPT) tools have been thriving, as ignited by the remarkable success of OpenAI’s recent chatbot product. GPT technology offers countless opportunities to significantly improve or renovate current health care research and practice paradigms, especially digital health interventions and digital health–enabled clinical care, and a future of smarter digital health can thus be expected. In particular, GPT technology can be incorporated through various digital health platforms in homes and hospitals embedded with numerous sensors, wearables, and remote monitoring devices. In this viewpoint paper, we highlight recent research progress that depicts the future picture of a smarter digital health ecosystem through GPT-facilitated centralized communications, automated analytics, personalized health care, and instant decision-making. %M 37751243 %R 10.2196/49963 %U https://www.jmir.org/2023/1/e49963 %U https://doi.org/10.2196/49963 %U http://www.ncbi.nlm.nih.gov/pubmed/37751243 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e43963 %T Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study %A Wang,Sabrina M %A Hogg,H D Jeffry %A Sangvai,Devdutta %A Patel,Manesh R %A Weissler,E Hope %A Kellogg,Katherine C %A Ratliff,William %A Balu,Suresh %A Sendak,Mark %+ Duke Institute for Health Innovation, 200 Morris St, Durham, NC, 27701, United States, 1 (919) 684 4389, mark.sendak@duke.edu %K machine learning %K implementation %K integration %K support %K quality %K peripheral arterial disease %K algorithm %K efficacy %K structure %K barrier %K clinical %K engagement %K development %K translation %K detection %D 2023 %7 21.9.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Machine learning (ML)–driven clinical decision support (CDS) continues to draw wide interest and investment as a means of improving care quality and value, despite mixed real-world implementation outcomes. Objective: This study aimed to explore the factors that influence the integration of a peripheral arterial disease (PAD) identification algorithm to implement timely guideline-based care. Methods: A total of 12 semistructured interviews were conducted with individuals from 3 stakeholder groups during the first 4 weeks of integration of an ML-driven CDS. The stakeholder groups included technical, administrative, and clinical members of the team interacting with the ML-driven CDS. The ML-driven CDS identified patients with a high probability of having PAD, and these patients were then reviewed by an interdisciplinary team that developed a recommended action plan and sent recommendations to the patient’s primary care provider. Pseudonymized transcripts were coded, and thematic analysis was conducted by a multidisciplinary research team. Results: Three themes were identified: positive factors translating in silico performance to real-world efficacy, organizational factors and data structure factors affecting clinical impact, and potential challenges to advancing equity. Our study found that the factors that led to successful translation of in silico algorithm performance to real-world impact were largely nontechnical, given adequate efficacy in retrospective validation, including strong clinical leadership, trustworthy workflows, early consideration of end-user needs, and ensuring that the CDS addresses an actionable problem. Negative factors of integration included failure to incorporate the on-the-ground context, the lack of feedback loops, and data silos limiting the ML-driven CDS. The success criteria for each stakeholder group were also characterized to better understand how teams work together to integrate ML-driven CDS and to understand the varying needs across stakeholder groups. Conclusions: Longitudinal and multidisciplinary stakeholder engagement in the development and integration of ML-driven CDS underpins its effective translation into real-world care. Although previous studies have focused on the technical elements of ML-driven CDS, our study demonstrates the importance of including administrative and operational leaders as well as an early consideration of clinicians’ needs. Seeing how different stakeholder groups have this more holistic perspective also permits more effective detection of context-driven health care inequities, which are uncovered or exacerbated via ML-driven CDS integration through structural and organizational challenges. Many of the solutions to these inequities lie outside the scope of ML and require coordinated systematic solutions for mitigation to help reduce disparities in the care of patients with PAD. %M 37733427 %R 10.2196/43963 %U https://formative.jmir.org/2023/1/e43963 %U https://doi.org/10.2196/43963 %U http://www.ncbi.nlm.nih.gov/pubmed/37733427 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 9 %N %P e49968 %T Nasopharyngeal Cancer Incidence and Mortality in 185 Countries in 2020 and the Projected Burden in 2040: Population-Based Global Epidemiological Profiling %A Zhang,Yanting %A Rumgay,Harriet %A Li,Mengmeng %A Cao,Sumei %A Chen,Wanqing %+ Department of Epidemiology and Health Statistics, School of Public Health, Guangdong Medical University, No.1 Xincheng Road, Dongguan, 523808, China, 86 076922896050, zhangyt@gdmu.edu.cn %K nasopharyngeal cancer %K incidence %K mortality %K epidemiology %K worldwide %D 2023 %7 20.9.2023 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: Nasopharyngeal cancer (NPC) is one of the most common head and neck cancers. Objective: This study describes the global epidemiological profiles of NPC incidence and mortality in 185 countries in 2020 and the projected burden in 2040. Methods: The estimated numbers of NPC cases and deaths were retrieved from the GLOBOCAN 2020 data set. Age-standardized incidence rates (ASIRs) and age-standardized mortality rates (ASMRs) were calculated using the world standard. The future number of NPC cases and deaths by 2040 were estimated based on global demographic projections. Results: Globally, approximately 133,354 cases and 80,008 deaths from NPC were estimated in 2020 corresponding to ASIRs and ASMRs of 1.5 and 0.9 per 100,000 person-years, respectively. The largest numbers of both global cases and deaths from NPC occurred in Eastern Asia (65,866/133,354, 49.39% and 36,453/80,008, 45.56%, respectively), in which China contributed most to this burden (62,444/133,354, 46.82% and 34,810/80,008, 43.50%, respectively). The ASIRs and ASMRs in men were approximately 3-fold higher than those in women. Incidence rates varied across world regions, with the highest ASIRs for both men and women detected in South-Eastern Asia (7.7 and 2.5 per 100,000 person-years, respectively) and Eastern Asia (3.9 and 1.5 per 100,000 person-years, respectively). The highest ASMRs for both men and women were found in South-Eastern Asia (5.4 and 1.5 per 100,000 person-years, respectively). By 2040, the annual number of cases and deaths will increase to 179,476 (46,122/133,354, a 34.58% increase from the year 2020) and 113,851 (33,843/80,008, a 42.29% increase), respectively. Conclusions: Disparities in NPC incidence and mortality persist worldwide. Our study highlights the urgent need to develop and accelerate NPC control initiatives to tackle the NPC burden in certain regions and countries (eg, South-Eastern Asia, China). %M 37728964 %R 10.2196/49968 %U https://publichealth.jmir.org/2023/1/e49968 %U https://doi.org/10.2196/49968 %U http://www.ncbi.nlm.nih.gov/pubmed/37728964 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e48636 %T The Australian Genetic Heart Disease Registry: Protocol for a Data Linkage Study %A Butters,Alexandra %A Blanch,Bianca %A Kemp-Casey,Anna %A Do,Judy %A Yeates,Laura %A Leslie,Felicity %A Semsarian,Christopher %A Nedkoff,Lee %A Briffa,Tom %A Ingles,Jodie %A Sweeting,Joanna %+ Clinical Genomics Laboratory, Centre for Population Genomics, Garvan Institute of Medical Research, 384 Victoria Street, Darlinghurst, 2010, Australia, 61 2 9359 8049, joanna.sweeting@populationgenomics.org.au %K data linkage %K genetic heart diseases %K health care use %K cardiomyopathies %K arrhythmia %K cardiology %K heart %K genetics %K registry %K registries %K risk %K mortality %K national %K big data %K harmonization %K probabilistic matching %D 2023 %7 20.9.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Genetic heart diseases such as hypertrophic cardiomyopathy can cause significant morbidity and mortality, ranging from syncope, chest pain, and palpitations to heart failure and sudden cardiac death. These diseases are inherited in an autosomal dominant fashion, meaning family members of affected individuals have a 1 in 2 chance of also inheriting the disease (“at-risk relatives”). The health care use patterns of individuals with a genetic heart disease, including emergency department presentations and hospital admissions, are poorly understood. By linking genetic heart disease registry data to routinely collected health data, we aim to provide a more comprehensive clinical data set to examine the burden of disease on individuals, families, and health care systems. Objective: The objective of this study is to link the Australian Genetic Heart Disease (AGHD) Registry with routinely collected whole-population health data sets to investigate the health care use of individuals with a genetic heart disease and their at-risk relatives. This linked data set will allow for the investigation of differences in outcomes and health care use due to disease, sex, socioeconomic status, and other factors. Methods: The AGHD Registry is a nationwide data set that began in 2007 and aims to recruit individuals with a genetic heart disease and their family members. In this study, demographic, clinical, and genetic data (available from 2007 to 2019) for AGHD Registry participants and at-risk relatives residing in New South Wales (NSW), Australia, were linked to routinely collected health data. These data included NSW-based data sets covering hospitalizations (2001-2019), emergency department presentations (2005-2019), and both state-wide and national mortality registries (2007-2019). The linkage was performed by the Centre for Health Record Linkage. Investigations stratifying by diagnosis, age, sex, socioeconomic status, and gene status will be undertaken and reported using descriptive statistics. Results: NSW AGHD Registry participants were linked to routinely collected health data sets using probabilistic matching (November 2019). Of 1720 AGHD Registry participants, 1384 had linkages with 11,610 hospital records, 7032 emergency department records, and 60 death records. Data assessment and harmonization were performed, and descriptive data analysis is underway. Conclusions: We intend to provide insights into the health care use patterns of individuals with a genetic heart disease and their at-risk relatives, including frequency of hospital admissions and differences due to factors such as disease, sex, and socioeconomic status. Identifying disparities and potential barriers to care may highlight specific health care needs (eg, between sexes) and factors impacting health care access and use. International Registered Report Identifier (IRRID): DERR1-10.2196/48636 %M 37728963 %R 10.2196/48636 %U https://www.researchprotocols.org/2023/1/e48636 %U https://doi.org/10.2196/48636 %U http://www.ncbi.nlm.nih.gov/pubmed/37728963 %0 Journal Article %@ 2561-9128 %I JMIR Publications %V 6 %N %P e47398 %T Dashboard of Short-Term Postoperative Patient Outcomes for Anesthesiologists: Development and Preliminary Evaluation %A Sreepada,Rama Syamala %A Chang,Ai Ching %A West,Nicholas C %A Sujan,Jonath %A Lai,Brendan %A Poznikoff,Andrew K %A Munk,Rebecca %A Froese,Norbert R %A Chen,James C %A Görges,Matthias %+ Research Institute, BC Children's Hospital, Rm V3-317 - 950 West 28th Avenue, Vancouver, BC, V5Z 4H4, Canada, 1 6048752000 ext 5616, mgoerges@bcchr.ca %K quality improvement %K feedback %K anesthesiologists %K patient reported outcome measures %K data display %K user-centered design %K surgical outcome %K discharge %K anesthesiology %K postoperative care %K registry %K dashboard %K interactive %K practice %K performance %K patient outcome %K mobile phone %D 2023 %7 19.9.2023 %9 Original Paper %J JMIR Perioper Med %G English %X Background: Anesthesiologists require an understanding of their patients’ outcomes to evaluate their performance and improve their practice. Traditionally, anesthesiologists had limited information about their surgical outpatients’ outcomes due to minimal contact post discharge. Leveraging digital health innovations for analyzing personal and population outcomes may improve perioperative care. BC Children’s Hospital’s postoperative follow-up registry for outpatient surgeries collects short-term outcomes such as pain, nausea, and vomiting. Yet, these data were previously not available to anesthesiologists. Objective: This quality improvement study aimed to visualize postoperative outcome data to allow anesthesiologists to reflect on their care and compare their performance with their peers. Methods: The postoperative follow-up registry contains nurse-reported postoperative outcomes, including opioid and antiemetic administration in the postanesthetic care unit (PACU), and family-reported outcomes, including pain, nausea, and vomiting, within 24 hours post discharge. Dashboards were iteratively co-designed with 5 anesthesiologists, and a department-wide usability survey gathered anesthesiologists’ feedback on the dashboards, allowing further design improvements. A final dashboard version has been deployed, with data updated weekly. Results: The dashboard contains three sections: (1) 24-hour outcomes, (2) PACU outcomes, and (3) a practice profile containing individual anesthesiologist’s case mix, grouped by age groups, sex, and surgical service. At the time of evaluation, the dashboard included 24-hour data from 7877 cases collected from September 2020 to February 2023 and PACU data from 8716 cases collected from April 2021 to February 2023. The co-design process and usability evaluation indicated that anesthesiologists preferred simpler designs for data summaries but also required the ability to explore details of specific outcomes and cases if needed. Anesthesiologists considered security and confidentiality to be key features of the design and most deemed the dashboard information useful and potentially beneficial for their practice. Conclusions: We designed and deployed a dynamic, personalized dashboard for anesthesiologists to review their outpatients’ short-term postoperative outcomes. This dashboard facilitates personal reflection on individual practice in the context of peer and departmental performance and, hence, the opportunity to evaluate iterative practice changes. Further work is required to establish their effect on improving individual and department performance and patient outcomes. %M 37725426 %R 10.2196/47398 %U https://periop.jmir.org/2023/1/e47398 %U https://doi.org/10.2196/47398 %U http://www.ncbi.nlm.nih.gov/pubmed/37725426 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e42409 %T Electronic Health Record–Driven Approaches in Primary Care to Strengthen Hypertension Management Among Racial and Ethnic Minoritized Groups in the United States: Systematic Review %A Ose,Dominik %A Adediran,Emmanuel %A Owens,Robert %A Gardner,Elena %A Mervis,Matthew %A Turner,Cindy %A Carlson,Emily %A Forbes,Danielle %A Jasumback,Caitlyn Lydia %A Stuligross,John %A Pohl,Susan %A Kiraly,Bernadette %+ Department of Family and Preventive Medicine, University of Utah, 375 Chipeta Way A, Salt Lake City, UT, 84108, United States, 1 801 581 7234, dominik.ose@hsc.utah.edu %K hypertension %K electronic health record (EHR) %K health data %K EHR-driven %K primary care %K racial and ethnic minority groups %D 2023 %7 15.9.2023 %9 Review %J J Med Internet Res %G English %X Background: Managing hypertension in racial and ethnic minoritized groups (eg, African American/Black patients) in primary care is highly relevant. However, evidence on whether or how electronic health record (EHR)–driven approaches in primary care can help improve hypertension management for patients of racial and ethnic minoritized groups in the United States remains scarce. Objective: This review aims to examine the role of the EHR in supporting interventions in primary care to strengthen the hypertension management of racial and ethnic minoritized groups in the United States. Methods: A search strategy based on the PICO (Population, Intervention, Comparison, and Outcome) guidelines was utilized to query and identify peer-reviewed articles on the Web of Science and PubMed databases. The search strategy was based on terms related to racial and ethnic minoritized groups, hypertension, primary care, and EHR-driven interventions. Articles were excluded if the focus was not hypertension management in racial and ethnic minoritized groups or if there was no mention of health record data utilization. Results: A total of 29 articles were included in this review. Regarding populations, Black/African American patients represented the largest population (26/29, 90%) followed by Hispanic/Latino (18/29, 62%), Asian American (7/29, 24%), and American Indian/Alaskan Native (2/29, 7%) patients. No study included patients who identified as Native Hawaiian/Pacific Islander. The EHR was used to identify patients (25/29, 86%), drive the intervention (21/29, 72%), and monitor results and outcomes (7/29, 59%). Most often, EHR-driven approaches were used for health coaching interventions, disease management programs, clinical decision support (CDS) systems, and best practice alerts (BPAs). Regarding outcomes, out of 8 EHR-driven health coaching interventions, only 3 (38%) reported significant results. In contrast, all the included studies related to CDS and BPA applications reported some significant results with respect to improving hypertension management. Conclusions: This review identified several use cases for the integration of the EHR in supporting primary care interventions to strengthen hypertension management in racial and ethnic minoritized patients in the United States. Some clinical-based interventions implementing CDS and BPA applications showed promising results. However, more research is needed on community-based interventions, particularly those focusing on patients who are Asian American, American Indian/Alaskan Native, and Native Hawaiian/Pacific Islander. The developed taxonomy comprising “identifying patients,” “driving intervention,” and “monitoring results” to classify EHR-driven approaches can be a helpful tool to facilitate this. %M 37713256 %R 10.2196/42409 %U https://www.jmir.org/2023/1/e42409 %U https://doi.org/10.2196/42409 %U http://www.ncbi.nlm.nih.gov/pubmed/37713256 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e45376 %T Developing a Semantically Based Query Recommendation for an Electronic Medical Record Search Engine: Query Log Analysis and Design Implications %A Wu,Danny T Y %A Hanauer,David %A Murdock,Paul %A Vydiswaran,V G Vinod %A Mei,Qiaozhu %A Zheng,Kai %+ Department of Biomedical Informatics, University of Cincinnati College of Medicine, 231 Albert Sabin Way, ML0840, Cincinnati, OH, 45267, United States, 1 5135586464, wutz@ucmail.uc.edu %K electronic health records %K information retrieval %K user-centered evaluation %K query recommendation %K query log analysis %K clinical research informatics %D 2023 %7 15.9.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: An effective and scalable information retrieval (IR) system plays a crucial role in enabling clinicians and researchers to harness the valuable information present in electronic health records. In a previous study, we developed a prototype medical IR system, which incorporated a semantically based query recommendation (SBQR) feature. The system was evaluated empirically and demonstrated high perceived performance by end users. To delve deeper into the factors contributing to this perceived performance, we conducted a follow-up study using query log analysis. Objective: One of the primary challenges faced in IR is that users often have limited knowledge regarding their specific information needs. Consequently, an IR system, particularly its user interface, needs to be thoughtfully designed to assist users through the iterative process of refining their queries as they encounter relevant documents during their search. To address these challenges, we incorporated “query recommendation” into our Electronic Medical Record Search Engine (EMERSE), drawing inspiration from the success of similar features in modern IR systems for general purposes. Methods: The query log data analyzed in this study were collected during our previous experimental study, where we developed EMERSE with the SBQR feature. We implemented a logging mechanism to capture user query behaviors and the output of the IR system (retrieved documents). In this analysis, we compared the initial query entered by users with the query formulated with the assistance of the SBQR. By examining the results of this comparison, we could examine whether the use of SBQR helped in constructing improved queries that differed from the original ones. Results: Our findings revealed that the first query entered without SBQR and the final query with SBQR assistance were highly similar (Jaccard similarity coefficient=0.77). This suggests that the perceived positive performance of the system was primarily attributed to the automatic query expansion facilitated by the SBQR rather than users manually manipulating their queries. In addition, through entropy analysis, we observed that search results converged in scenarios of moderate difficulty, and the degree of convergence correlated strongly with the perceived system performance. Conclusions: The study demonstrated the potential contribution of the SBQR in shaping participants' positive perceptions of system performance, contingent upon the difficulty of the search scenario. Medical IR systems should therefore consider incorporating an SBQR as a user-controlled option or a semiautomated feature. Future work entails redesigning the experiment in a more controlled manner and conducting multisite studies to demonstrate the effectiveness of EMERSE with SBQR for patient cohort identification. By further exploring and validating these findings, we can enhance the usability and functionality of medical IR systems in real-world settings. %M 37713239 %R 10.2196/45376 %U https://formative.jmir.org/2023/1/e45376 %U https://doi.org/10.2196/45376 %U http://www.ncbi.nlm.nih.gov/pubmed/37713239 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e49593 %T Integrated Real-World Study Databases in 3 Diverse Asian Health Care Systems in Taiwan, India, and Thailand: Scoping Review %A Shau,Wen-Yi %A Setia,Sajita %A Chen,Ying-Jan %A Ho,Tsu-yun %A Prakash Shinde,Salil %A Santoso,Handoko %A Furtner,Daniel %+ Executive Office, Transform Medical Communications Limited, 184 Glasgow Street, Wanganui, 4500, New Zealand, 64 0276175433, sajita.setia@transform-medcomms.com %K Asia %K health care databases %K real-world data %K real-world evidence %K scoping review %D 2023 %7 11.9.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: The use of real-world data (RWD) warehouses for research in Asia is on the rise, but current trends remain largely unexplored. Given the varied economic and health care landscapes in different Asian countries, understanding these trends can offer valuable insights. Objective: We sought to discern the contemporary landscape of linked RWD warehouses and explore their trends and patterns in 3 Asian countries with contrasting economies and health care systems: Taiwan, India, and Thailand. Methods: Using a systematic scoping review methodology, we conducted an exhaustive literature search on PubMed with filters for the English language and the past 5 years. The search combined Medical Subject Heading terms and specific keywords. Studies were screened against strict eligibility criteria to identify eligible studies using RWD databases from more than one health care facility in at least 1 of the 3 target countries. Results: Our search yielded 2277 studies, of which 833 (36.6%) met our criteria. Overall, single-country studies (SCS) dominated at 89.4% (n=745), with cross-country collaboration studies (CCCS) being at 10.6% (n=88). However, the country-wise breakdown showed that of all the SCS, 623 (83.6%) were from Taiwan, 81 (10.9%) from India, and 41 (5.5%) from Thailand. Among the total studies conducted in each country, India at 39.1% (n=133) and Thailand at 43.1% (n=72) had a significantly higher percentage of CCCS compared to Taiwan at 7.6% (n=51). Over a 5-year span from 2017 to 2022, India and Thailand experienced an annual increase in RWD studies by approximately 18.2% and 13.8%, respectively, while Taiwan’s contributions remained consistent. Comparative effectiveness research (CER) was predominant in Taiwan (n=410, or 65.8% of SCS) but less common in India (n=12, or 14.8% of SCS) and Thailand (n=11, or 26.8% of SCS). CER percentages in CCCS were similar across the 3 countries, ranging from 19.2% (n=10) to 29% (n=9). The type of RWD source also varied significantly across countries, with India demonstrating a high reliance on electronic medical records or electronic health records at 55.6% (n=45) of SCS and Taiwan showing an increasing trend in their use over the period. Registries were used in 26 (83.9%) CCCS and 31 (75.6%) SCS from Thailand but in <50% of SCS from Taiwan and India. Health insurance/administrative claims data were used in most of the SCS from Taiwan (n=458, 73.5%). There was a consistent predominant focus on cardiology/metabolic disorders in all studies, with a noticeable increase in oncology and infectious disease research from 2017 to 2022. Conclusions: This review provides a comprehensive understanding of the evolving landscape of RWD research in Taiwan, India, and Thailand. The observed differences and trends emphasize the unique economic, clinical, and research settings in each country, advocating for tailored strategies for leveraging RWD for future health care research and decision-making. International Registered Report Identifier (IRRID): RR2-10.2196/43741 %M 37615085 %R 10.2196/49593 %U https://www.jmir.org/2023/1/e49593 %U https://doi.org/10.2196/49593 %U http://www.ncbi.nlm.nih.gov/pubmed/37615085 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e45224 %T Strengths, Weaknesses, Opportunities, and Threats Analysis of the Use of Digital Health Technologies in Primary Health Care in the Sub-Saharan African Region: Qualitative Study %A O'Brien,Niki %A Li,Edmond %A Chaibva,Cynthia N %A Gomez Bravo,Raquel %A Kovacevic,Lana %A Kwame Ayisi-Boateng,Nana %A Lounsbury,Olivia %A Nwabufo,Ngnedjou Francoise F %A Senkyire,Ephraim Kumi %A Serafini,Alice %A Surafel Abay,Eleleta %A van de Vijver,Steven %A Wanjala,Mercy %A Wangari,Marie-Claire %A Moosa,Shabir %A Neves,Ana Luisa %+ Institute of Global Health Innovation, Imperial College London, Room 1035/7, QEQM Wing, St Mary's Hospital, London, W2 1NY, United Kingdom, 44 020 7594 1419, n.obrien@imperial.ac.uk %K digital health %K digital health technology %K telemedicine %K remote care %K primary care %K primary health carel PHC %K COVID-19 %K global health %K sub-Saharan Africa %K eHealth %D 2023 %7 7.9.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Digital health technologies (DHTs) have become increasingly commonplace as a means of delivering primary care. While DHTs have been postulated to reduce inequalities, increase access, and strengthen health systems, how the implementation of DHTs has been realized in the sub-Saharan Africa (SSA) health care environment remains inadequately explored. Objective: This study aims to capture the multidisciplinary experiences of primary care professionals using DHTs to explore the strengths and weaknesses, as well as opportunities and threats, regarding the implementation and use of DHTs in SSA primary care settings. Methods: A combination of qualitative approaches was adopted (ie, focus groups and semistructured interviews). Participants were recruited through the African Forum for Primary Care and researchers’ contact networks using convenience sampling and included if having experience with digital technologies in primary health care in SSA. Focus and interviews were conducted, respectively, in November 2021 and January-March 2022. Topic guides were used to cover relevant topics in the interviews, using the strengths, weaknesses, opportunities, and threats framework. Transcripts were compiled verbatim and systematically reviewed by 2 independent reviewers using framework analysis to identify emerging themes. The COREQ (Consolidated Criteria for Reporting Qualitative Research) checklist was used to ensure the study met the recommended standards of qualitative data reporting. Results: A total of 33 participants participated in the study (n=13 and n=23 in the interviews and in focus groups, respectively; n=3 participants participated in both). The strengths of using DHTs ranged from improving access to care, supporting the continuity of care, and increasing care satisfaction and trust to greater collaboration, enabling safer decision-making, and hastening progress toward universal health coverage. Weaknesses included poor digital literacy, health inequalities, lack of human resources, inadequate training, lack of basic infrastructure and equipment, and poor coordination when implementing DHTs. DHTs were perceived as an opportunity to improve patient digital literacy, increase equity, promote more patient-centric design in upcoming DHTs, streamline expenditure, and provide a means to learn international best practices. Threats identified include the lack of buy-in from both patients and providers, insufficient human resources and local capacity, inadequate governmental support, overly restrictive regulations, and a lack of focus on cybersecurity and data protection. Conclusions: The research highlights the complex challenges of implementing DHTs in the SSA context as a fast-moving health delivery modality, as well as the need for multistakeholder involvement. Future research should explore the nuances of these findings across different technologies and settings in the SSA region and implications on health and health care equity, capitalizing on mixed-methods research, including the use of real-world quantitative data to understand patient health needs. The promise of digital health will only be realized when informed by studies that incorporate patient perspective at every stage of the research cycle. %M 37676721 %R 10.2196/45224 %U https://www.jmir.org/2023/1/e45224 %U https://doi.org/10.2196/45224 %U http://www.ncbi.nlm.nih.gov/pubmed/37676721 %0 Journal Article %@ 2369-1999 %I JMIR Publications %V 9 %N %P e45547 %T Data-Efficient Computational Pathology Platform for Faster and Cheaper Breast Cancer Subtype Identifications: Development of a Deep Learning Model %A Bae,Kideog %A Jeon,Young Seok %A Hwangbo,Yul %A Yoo,Chong Woo %A Han,Nayoung %A Feng,Mengling %+ Saw Swee Hock School of Public Health, National University of Singapore, 12 Science Drive 2, Tahir foundation MD1 #09-01, Singapore, 117549, Singapore, 65 65164984, ephfm@nus.edu.sg %K deep learning %K self-supervised learning %K immunohistochemical staining %K machine learning %K histology %K pathology %K computation %K predict %K diagnosis %K diagnose %K carcinoma %K cancer %K oncology %K breast cancer %D 2023 %7 5.9.2023 %9 Original Paper %J JMIR Cancer %G English %X Background: Breast cancer subtyping is a crucial step in determining therapeutic options, but the molecular examination based on immunohistochemical staining is expensive and time-consuming. Deep learning opens up the possibility to predict the subtypes based on the morphological information from hematoxylin and eosin staining, a much cheaper and faster alternative. However, training the predictive model conventionally requires a large number of histology images, which is challenging to collect by a single institute. Objective: We aimed to develop a data-efficient computational pathology platform, 3DHistoNet, which is capable of learning from z-stacked histology images to accurately predict breast cancer subtypes with a small sample size. Methods: We retrospectively examined 401 cases of patients with primary breast carcinoma diagnosed between 2018 and 2020 at the Department of Pathology, National Cancer Center, South Korea. Pathology slides of the patients with breast carcinoma were prepared according to the standard protocols. Age, gender, histologic grade, hormone receptor (estrogen receptor [ER], progesterone receptor [PR], and androgen receptor [AR]) status, erb-B2 receptor tyrosine kinase 2 (HER2) status, and Ki-67 index were evaluated by reviewing medical charts and pathological records. Results: The area under the receiver operating characteristic curve and decision curve were analyzed to evaluate the performance of our 3DHistoNet platform for predicting the ER, PR, AR, HER2, and Ki67 subtype biomarkers with 5-fold cross-validation. We demonstrated that 3DHistoNet can predict all clinically important biomarkers (ER, PR, AR, HER2, and Ki67) with performance exceeding the conventional multiple instance learning models by a considerable margin (area under the receiver operating characteristic curve: 0.75-0.91 vs 0.67-0.8). We further showed that our z-stack histology scanning method can make up for insufficient training data sets without any additional cost incurred. Finally, 3DHistoNet offered an additional capability to generate attention maps that reveal correlations between Ki67 and histomorphological features, which renders the hematoxylin and eosin image in higher fidelity to the pathologist. Conclusions: Our stand-alone, data-efficient pathology platform that can both generate z-stacked images and predict key biomarkers is an appealing tool for breast cancer diagnosis. Its development would encourage morphology-based diagnosis, which is faster, cheaper, and less error-prone compared to the protein quantification method based on immunohistochemical staining. %M 37669090 %R 10.2196/45547 %U https://cancer.jmir.org/2023/1/e45547 %U https://doi.org/10.2196/45547 %U http://www.ncbi.nlm.nih.gov/pubmed/37669090 %0 Journal Article %@ 2561-1011 %I JMIR Publications %V 7 %N %P e44983 %T Digital Transformation in the Diagnostics and Therapy of Cardiovascular Diseases: Comprehensive Literature Review %A Stremmel,Christopher %A Breitschwerdt,Rüdiger %+ Department of Cardiology, LMU University Hospital, Marchioninistr. 15, Munich, 81377, Germany, 49 894400712622, christopher.stremmel@med.uni-muenchen.de %K cardiovascular %K digital medicine %K telehealth %K artificial intelligence %K telemedicine %K mobile phone %K review %D 2023 %7 30.8.2023 %9 Review %J JMIR Cardio %G English %X Background: The digital transformation of our health care system has experienced a clear shift in the last few years due to political, medical, and technical innovations and reorganization. In particular, the cardiovascular field has undergone a significant change, with new broad perspectives in terms of optimized treatment strategies for patients nowadays. Objective: After a short historical introduction, this comprehensive literature review aimed to provide a detailed overview of the scientific evidence regarding digitalization in the diagnostics and therapy of cardiovascular diseases (CVDs). Methods: We performed an extensive literature search of the PubMed database and included all related articles that were published as of March 2022. Of the 3021 studies identified, 1639 (54.25%) studies were selected for a structured analysis and presentation (original articles: n=1273, 77.67%; reviews or comments: n=366, 22.33%). In addition to studies on CVDs in general, 829 studies could be assigned to a specific CVD with a diagnostic and therapeutic approach. For data presentation, all 829 publications were grouped into 6 categories of CVDs. Results: Evidence-based innovations in the cardiovascular field cover a wide medical spectrum, starting from the diagnosis of congenital heart diseases or arrhythmias and overoptimized workflows in the emergency care setting of acute myocardial infarction to telemedical care for patients having chronic diseases such as heart failure, coronary artery disease, or hypertension. The use of smartphones and wearables as well as the integration of artificial intelligence provides important tools for location-independent medical care and the prevention of adverse events. Conclusions: Digital transformation has opened up multiple new perspectives in the cardiovascular field, with rapidly expanding scientific evidence. Beyond important improvements in terms of patient care, these innovations are also capable of reducing costs for our health care system. In the next few years, digital transformation will continue to revolutionize the field of cardiovascular medicine and broaden our medical and scientific horizons. %M 37647103 %R 10.2196/44983 %U https://cardio.jmir.org/2023/1/e44983 %U https://doi.org/10.2196/44983 %U http://www.ncbi.nlm.nih.gov/pubmed/37647103 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e44114 %T A Data Taxonomy for Adaptive Multifactor Authentication in the Internet of Health Care Things %A Suleski,Tance %A Ahmed,Mohiuddin %+ School of Science, Edith Cowan University, 270 Joondalup Dr, Joondalup WA, Perth, 6027, Australia, 61 13 43 28, tsuleski@our.ecu.edu.au %K health care %K authentication %K contextual data model %K Internet of Health Care Things %K multifactor %K mobile phone %D 2023 %7 29.8.2023 %9 Viewpoint %J J Med Internet Res %G English %X The health care industry has faced various challenges over the past decade as we move toward a digital future where services and data are available on demand. The systems of interconnected devices, users, data, and working environments are referred to as the Internet of Health Care Things (IoHT). IoHT devices have emerged in the past decade as cost-effective solutions with large scalability capabilities to address the constraints on limited resources. These devices cater to the need for remote health care services outside of physical interactions. However, IoHT security is often overlooked because the devices are quickly deployed and configured as solutions to meet the demands of a heavily saturated industry. During the COVID-19 pandemic, studies have shown that cybercriminals are exploiting the health care industry, and data breaches are targeting user credentials through authentication vulnerabilities. Poor password use and management and the lack of multifactor authentication security posture within IoHT cause a loss of millions according to the IBM reports. Therefore, it is important that health care authentication security moves toward adaptive multifactor authentication (AMFA) to replace the traditional approaches to authentication. We identified a lack of taxonomy for data models that particularly focus on IoHT data architecture to improve the feasibility of AMFA. This viewpoint focuses on identifying key cybersecurity challenges in a theoretical framework for a data model that summarizes the main components of IoHT data. The data are to be used in modalities that are suited for health care users in modern IoHT environments and in response to the COVID-19 pandemic. To establish the data taxonomy, a review of recent IoHT papers was conducted to discuss the related work in IoHT data management and use in next-generation authentication systems. Reports, journal articles, conferences, and white papers were reviewed for IoHT authentication data technologies in relation to the problem statement of remote authentication and user management systems. Only publications written in English from the last decade were included (2012-2022) to identify key issues within the current health care practices and their management of IoHT devices. We discuss the components of the IoHT architecture from the perspective of data management and sensitivity to ensure privacy for all users. The data model addresses the security requirements of IoHT users, environments, and devices toward the automation of AMFA in health care. We found that in health care authentication, the significant threats occurring were related to data breaches owing to weak security options and poor user configuration of IoHT devices. The security requirements of IoHT data architecture and identified impactful methods of cybersecurity for health care devices, data, and their respective attacks are discussed. Data taxonomy provides better understanding, solutions, and improvements of user authentication in remote working environments for security features. %M 37490633 %R 10.2196/44114 %U https://www.jmir.org/2023/1/e44114 %U https://doi.org/10.2196/44114 %U http://www.ncbi.nlm.nih.gov/pubmed/37490633 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e49283 %T An Artificial Intelligence Model for Predicting Trauma Mortality Among Emergency Department Patients in South Korea: Retrospective Cohort Study %A Lee,Seungseok %A Kang,Wu Seong %A Kim,Do Wan %A Seo,Sang Hyun %A Kim,Joongsuck %A Jeong,Soon Tak %A Yon,Dong Keon %A Lee,Jinseok %+ Department of Biomedical Engineering, Kyung Hee University, 1732 Deogyeong-daero, Giheung-gu, Yongin, 17104, Republic of Korea, 82 312012570, gonasago@khu.ac.kr %K artificial intelligence %K trauma %K mortality prediction %K international classification of disease %K emergency department %K ICD %K model %K models %K mortality %K predict %K prediction %K predictive %K emergency %K death %K traumatic %K nationwide %K national %K cohort %K retrospective %D 2023 %7 29.8.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Within the trauma system, the emergency department (ED) is the hospital’s first contact and is vital for allocating medical resources. However, there is generally limited information about patients that die in the ED. Objective: The aim of this study was to develop an artificial intelligence (AI) model to predict trauma mortality and analyze pertinent mortality factors for all patients visiting the ED. Methods: We used the Korean National Emergency Department Information System (NEDIS) data set (N=6,536,306), incorporating over 400 hospitals between 2016 and 2019. We included the International Classification of Disease 10th Revision (ICD-10) codes and chose the following input features to predict ED patient mortality: age, sex, intentionality, injury, emergent symptom, Alert/Verbal/Painful/Unresponsive (AVPU) scale, Korean Triage and Acuity Scale (KTAS), and vital signs. We compared three different feature set performances for AI input: all features (n=921), ICD-10 features (n=878), and features excluding ICD-10 codes (n=43). We devised various machine learning models with an ensemble approach via 5-fold cross-validation and compared the performance of each model with that of traditional prediction models. Lastly, we investigated explainable AI feature effects and deployed our final AI model on a public website, providing access to our mortality prediction results among patients visiting the ED. Results: Our proposed AI model with the all-feature set achieved the highest area under the receiver operating characteristic curve (AUROC) of 0.9974 (adaptive boosting [AdaBoost], AdaBoost + light gradient boosting machine [LightGBM]: Ensemble), outperforming other state-of-the-art machine learning and traditional prediction models, including extreme gradient boosting (AUROC=0.9972), LightGBM (AUROC=0.9973), ICD-based injury severity scores (AUC=0.9328 for the inclusive model and AUROC=0.9567 for the exclusive model), and KTAS (AUROC=0.9405). In addition, our proposed AI model outperformed a cutting-edge AI model designed for in-hospital mortality prediction (AUROC=0.7675) for all ED visitors. From the AI model, we also discovered that age and unresponsiveness (coma) were the top two mortality predictors among patients visiting the ED, followed by oxygen saturation, multiple rib fractures (ICD-10 code S224), painful response (stupor, semicoma), and lumbar vertebra fracture (ICD-10 code S320). Conclusions: Our proposed AI model exhibits remarkable accuracy in predicting ED mortality. Including the necessity for external validation, a large nationwide data set would provide a more accurate model and minimize overfitting. We anticipate that our AI-based risk calculator tool will substantially aid health care providers, particularly regarding triage and early diagnosis for trauma patients. %M 37642984 %R 10.2196/49283 %U https://www.jmir.org/2023/1/e49283 %U https://doi.org/10.2196/49283 %U http://www.ncbi.nlm.nih.gov/pubmed/37642984 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e45013 %T Initiatives, Concepts, and Implementation Practices of the Findable, Accessible, Interoperable, and Reusable Data Principles in Health Data Stewardship: Scoping Review %A Inau,Esther Thea %A Sack,Jean %A Waltemath,Dagmar %A Zeleke,Atinkut Alamirrew %+ Department of Medical Informatics, Institute for Community Medicine, University Medicine Greifswald, Walther-Rathenau-Str 48, Greifswald, D-17475, Germany, 49 3834867548, inaue@uni-greifswald.de %K data stewardship %K findable, accessible, interoperable, and reusable data principles %K FAIR data principles %K health research %K Preferred Reporting Items for Systematic Reviews and Meta-Analyses %K PRISMA %K qualitative analysis %K scoping review %K information retrieval %K health information exchange %D 2023 %7 28.8.2023 %9 Review %J J Med Internet Res %G English %X Background: Thorough data stewardship is a key enabler of comprehensive health research. Processes such as data collection, storage, access, sharing, and analytics require researchers to follow elaborate data management strategies properly and consistently. Studies have shown that findable, accessible, interoperable, and reusable (FAIR) data leads to improved data sharing in different scientific domains. Objective: This scoping review identifies and discusses concepts, approaches, implementation experiences, and lessons learned in FAIR initiatives in health research data. Methods: The Arksey and O’Malley stage-based methodological framework for scoping reviews was applied. PubMed, Web of Science, and Google Scholar were searched to access relevant publications. Articles written in English, published between 2014 and 2020, and addressing FAIR concepts or practices in the health domain were included. The 3 data sources were deduplicated using a reference management software. In total, 2 independent authors reviewed the eligibility of each article based on defined inclusion and exclusion criteria. A charting tool was used to extract information from the full-text papers. The results were reported using the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. Results: A total of 2.18% (34/1561) of the screened articles were included in the final review. The authors reported FAIRification approaches, which include interpolation, inclusion of comprehensive data dictionaries, repository design, semantic interoperability, ontologies, data quality, linked data, and requirement gathering for FAIRification tools. Challenges and mitigation strategies associated with FAIRification, such as high setup costs, data politics, technical and administrative issues, privacy concerns, and difficulties encountered in sharing health data despite its sensitive nature were also reported. We found various workflows, tools, and infrastructures designed by different groups worldwide to facilitate the FAIRification of health research data. We also uncovered a wide range of problems and questions that researchers are trying to address by using the different workflows, tools, and infrastructures. Although the concept of FAIR data stewardship in the health research domain is relatively new, almost all continents have been reached by at least one network trying to achieve health data FAIRness. Documented outcomes of FAIRification efforts include peer-reviewed publications, improved data sharing, facilitated data reuse, return on investment, and new treatments. Successful FAIRification of data has informed the management and prognosis of various diseases such as cancer, cardiovascular diseases, and neurological diseases. Efforts to FAIRify data on a wider variety of diseases have been ongoing since the COVID-19 pandemic. Conclusions: This work summarises projects, tools, and workflows for the FAIRification of health research data. The comprehensive review shows that implementing the FAIR concept in health data stewardship carries the promise of improved research data management and transparency in the era of big data and open research publishing. International Registered Report Identifier (IRRID): RR2-10.2196/22505 %M 37639292 %R 10.2196/45013 %U https://www.jmir.org/2023/1/e45013 %U https://doi.org/10.2196/45013 %U http://www.ncbi.nlm.nih.gov/pubmed/37639292 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e46722 %T Changes in Documentation Due to Patient Access to Electronic Health Records: Protocol for a Scoping Review %A Meier-Diedrich,Eva %A Davidge,Gail %A Hägglund,Maria %A Kharko,Anna %A Lyckblad,Camilla %A McMillan,Brian %A Blease,Charlotte %A Schwarz,Julian %+ Brandenburg Medical School, Immanuel Hospital Rüdersdorf, University Clinic for Psychiatry and Psychotherapy, Seebad 82/83, Rüdersdorf, 15562, Germany, 49 176 22 65 26 2, julian.schwarz@mhb-fontane.de %K electronic health record %K patient-accessible electronic health record %K patient portal %K electronic portal %K scoping review %K patient access %K documentation change %K natural language processing %K patient-clinician relationship %K online record access %K review method %K library science %K librarian %D 2023 %7 28.8.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Internationally, patient-accessible electronic health records (PAEHRs) are increasingly being implemented. Despite reported benefits to patients, the innovation has prompted concerns among health care professionals (HCPs), including the possibility that access incurs a “dumbing down” of clinical records. Currently, no review has investigated empirical evidence of whether and how documentation changes after introducing PAEHRs. Objective: This paper presents the protocol for a scoping review examining potential subjective and objective changes in HCPs documentation after using PAEHRs. Methods: This scoping review will be carried out based on the framework of Arksey and O’Malley. Several databases will be used to conduct a literature search (APA PsycInfo, CINAHL, PubMed, and Web of Science Core Collection). Authors will participate in screening identified papers to explore the research questions: How do PAEHRs affect HCPs’ documentation practices? and What subjective and objective changes to the clinical notes arise after patient access? Only studies that relate to actual use experiences, and not merely prior expectations about PAEHRs, will be selected in the review. Data abstraction will include but will not be limited to publication type, publication year, country, sample characteristics, setting, study aim, research question, and conclusions. The Mixed Methods Appraisal Tool will be used to assess the quality of the studies included. Results: The results from this scoping review will be presented as a narrative synthesis structured along the key themes of the corpus of evidence. Additional data will be prepared in charts or tabular format. We anticipate the results to be presented in a scoping review at a later date. They will be disseminated at scientific conferences and through publication in a peer-reviewed journal. Conclusions: This is the first scoping review that considers potential change in documentation after implementation of PAEHRs. The results can potentially help affirm or refute prior opinions and expectations among various stakeholders about the use of PAEHRs and thereby help to address uncertainties. Results may help to provide guidance to clinicians in writing notes and thus have immediate practical relevance to care. In addition, the review will help to identify any substantive research gaps in this field of research. In the longer term, our findings may contribute to the development of shared documentation guidelines, which in turn are central to improving patient communication and safety. International Registered Report Identifier (IRRID): PRR1-10.2196/46722 %M 37639298 %R 10.2196/46722 %U https://www.researchprotocols.org/2023/1/e46722 %U https://doi.org/10.2196/46722 %U http://www.ncbi.nlm.nih.gov/pubmed/37639298 %0 Journal Article %@ 1929-073X %I JMIR Publications %V 12 %N %P e46322 %T Web-Based Application Based on Human-in-the-Loop Deep Learning for Deidentifying Free-Text Data in Electronic Medical Records: Development and Usability Study %A Liu,Leibo %A Perez-Concha,Oscar %A Nguyen,Anthony %A Bennett,Vicki %A Blake,Victoria %A Gallego,Blanca %A Jorm,Louisa %+ Centre for Big Data Research in Health, University of New South Wales, Level 2, AGSM Building, G27, Botany St, Sydney, 2052, Australia, 61 290657847, z5250377@ad.unsw.edu.au %K web-based system %K deidentification %K electronic medical records %K deep learning %K narrative free text %K human in the loop %K free text %K unstructured data %K electronic health records %K machine learning %D 2023 %7 25.8.2023 %9 Original Paper %J Interact J Med Res %G English %X Background: The narrative free-text data in electronic medical records (EMRs) contain valuable clinical information for analysis and research to inform better patient care. However, the release of free text for secondary use is hindered by concerns surrounding personally identifiable information (PII), as protecting individuals' privacy is paramount. Therefore, it is necessary to deidentify free text to remove PII. Manual deidentification is a time-consuming and labor-intensive process. Numerous automated deidentification approaches and systems have been attempted to overcome this challenge over the past decade. Objective: We sought to develop an accurate, web-based system deidentifying free text (DEFT), which can be readily and easily adopted in real-world settings for deidentification of free text in EMRs. The system has several key features including a simple and task-focused web user interface, customized PII types, use of a state-of-the-art deep learning model for tagging PII from free text, preannotation by an interactive learning loop, rapid manual annotation with autosave, support for project management and team collaboration, user access control, and central data storage. Methods: DEFT comprises frontend and backend modules and communicates with central data storage through a filesystem path access. The frontend web user interface provides end users with a user-friendly workspace for managing and annotating free text. The backend module processes the requests from the frontend and performs relevant persistence operations. DEFT manages the deidentification workflow as a project, which can contain one or more data sets. Customized PII types and user access control can also be configured. The deep learning model is based on a Bidirectional Long Short-Term Memory-Conditional Random Field (BiLSTM-CRF) with RoBERTa as the word embedding layer. The interactive learning loop is further integrated into DEFT to speed up the deidentification process and increase its performance over time. Results: DEFT has many advantages over existing deidentification systems in terms of its support for project management, user access control, data management, and an interactive learning process. Experimental results from DEFT on the 2014 i2b2 data set obtained the highest performance compared to 5 benchmark models in terms of microaverage strict entity–level recall and F1-scores of 0.9563 and 0.9627, respectively. In a real-world use case of deidentifying clinical notes, extracted from 1 referral hospital in Sydney, New South Wales, Australia, DEFT achieved a high microaverage strict entity–level F1-score of 0.9507 on a corpus of 600 annotated clinical notes. Moreover, the manual annotation process with preannotation demonstrated a 43% increase in work efficiency compared to the process without preannotation. Conclusions: DEFT is designed for health domain researchers and data custodians to easily deidentify free text in EMRs. DEFT supports an interactive learning loop and end users with minimal technical knowledge can perform the deidentification work with only a shallow learning curve. %M 37624624 %R 10.2196/46322 %U https://www.i-jmr.org/2023/1/e46322 %U https://doi.org/10.2196/46322 %U http://www.ncbi.nlm.nih.gov/pubmed/37624624 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e47335 %T Barriers and Enablers for Implementation of an Artificial Intelligence–Based Decision Support Tool to Reduce the Risk of Readmission of Patients With Heart Failure: Stakeholder Interviews %A Nair,Monika %A Andersson,Jonas %A Nygren,Jens M %A Lundgren,Lina E %+ School of Business, Innovation and Sustainability, Halmstad University, Kristian IV:s väg 3, Halmstad, 30118, Sweden, 46 707227544, lina.lundgren@hh.se %K implementation %K AI systems %K health care %K interviews %K artificial Intelligence %K AI %K decision support tool %K readmission %K prediction %K heart failure %K digital tool %D 2023 %7 23.8.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Artificial intelligence (AI) applications in health care are expected to provide value for health care organizations, professionals, and patients. However, the implementation of such systems should be carefully planned and organized in order to ensure quality, safety, and acceptance. The gathered view of different stakeholders is a great source of information to understand the barriers and enablers for implementation in a specific context. Objective: This study aimed to understand the context and stakeholder perspectives related to the future implementation of a clinical decision support system for predicting readmissions of patients with heart failure. The study was part of a larger project involving model development, interface design, and implementation planning of the system. Methods: Interviews were held with 12 stakeholders from the regional and municipal health care organizations to gather their views on the potential effects implementation of such a decision support system could have as well as barriers and enablers for implementation. Data were analyzed based on the categories defined in the nonadoption, abandonment, scale-up, spread, sustainability (NASSS) framework. Results: Stakeholders had in general a positive attitude and curiosity toward AI-based decision support systems, and mentioned several barriers and enablers based on the experiences of previous implementations of information technology systems. Central aspects to consider for the proposed clinical decision support system were design aspects, access to information throughout the care process, and integration into the clinical workflow. The implementation of such a system could lead to a number of effects related to both clinical outcomes as well as resource allocation, which are all important to address in the planning of implementation. Stakeholders saw, however, value in several aspects of implementing such system, emphasizing the increased quality of life for those patients who can avoid being hospitalized. Conclusions: Several ideas were put forward on how the proposed AI system would potentially affect and provide value for patients, professionals, and the organization, and implementation aspects were important parts of that. A successful system can help clinicians to prioritize the need for different types of treatments but also be used for planning purposes within the hospital. However, the system needs not only technological and clinical precision but also a carefully planned implementation process. Such a process should take into consideration the aspects related to all the categories in the NASSS framework. This study further highlighted the importance to study stakeholder needs early in the process of development, design, and implementation of decision support systems, as the data revealed new information on the potential use of the system and the placement of the application in the care process. %M 37610799 %R 10.2196/47335 %U https://formative.jmir.org/2023/1/e47335 %U https://doi.org/10.2196/47335 %U http://www.ncbi.nlm.nih.gov/pubmed/37610799 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e44842 %T Designing Interoperable Health Care Services Based on Fast Healthcare Interoperability Resources: Literature Review %A Nan,Jingwen %A Xu,Li-Qun %+ Health IT Research, China Mobile (Chengdu) Industrial Research Institute, Unit 2, Block C1, AI Innovation Center, Hele Second Street, Gaoxin District, Chengdu, 610213, China, 86 (28) 60103585, xuliqun@chinamobile.com %K Health level 7 Fast Healthcare Interoperability Resources %K HL7 FHIR %K interoperability %K literature review %K practice guideline %K mobile phone %D 2023 %7 21.8.2023 %9 Review %J JMIR Med Inform %G English %X Background: With the advent of the digital economy and the aging population, the demand for diversified health care services and innovative care delivery models has been overwhelming. This trend has accelerated the urgency to implement effective and efficient data exchange and service interoperability, which underpins coordinated care services among tiered health care institutions, improves the quality of oversight of regulators, and provides vast and comprehensive data collection to support clinical medicine and health economics research, thus improving the overall service quality and patient satisfaction. To meet this demand and facilitate the interoperability of IT systems of stakeholders, after years of preparation, Health Level 7 formally introduced, in 2014, the Fast Healthcare Interoperability Resources (FHIR) standard. It has since continued to evolve. FHIR depends on the Implementation Guide (IG) to ensure feasibility and consistency while developing an interoperable health care service. The IG defines rules with associated documentation on how FHIR resources are used to tackle a particular problem. However, a gap remains between IGs and the process of building actual services because IGs are rules without specifying concrete methods, procedures, or tools. Thus, stakeholders may feel it nontrivial to participate in the ecosystem, giving rise to the need for a more actionable practice guideline (PG) for promoting FHIR’s fast adoption. Objective: This study aimed to propose a general FHIR PG to facilitate stakeholders in the health care ecosystem to understand FHIR and quickly develop interoperable health care services. Methods: We selected a collection of FHIR-related papers about the latest studies or use cases on designing and building FHIR-based interoperable health care services and tagged each use case as belonging to 1 of the 3 dominant innovation feature groups that are also associated with practice stages, that is, data standardization, data management, and data integration. Next, we reviewed each group’s detailed process and key techniques to build respective care services and collate a complete FHIR PG. Finally, as an example, we arbitrarily selected a use case outside the scope of the reviewed papers and mapped it back to the FHIR PG to demonstrate the effectiveness and generalizability of the PG. Results: The FHIR PG includes 2 core elements: one is a practice design that defines the responsibilities of stakeholders and outlines the complete procedure from data to services, and the other is a development architecture for practice design, which lists the available tools for each practice step and provides direct and actionable recommendations. Conclusions: The FHIR PG can bridge the gap between IGs and the process of building actual services by proposing actionable methods, procedures, and tools. It assists stakeholders in identifying participants’ roles, managing the scope of responsibilities, and developing relevant modules, thus helping promote FHIR-based interoperable health care services. %M 37603388 %R 10.2196/44842 %U https://medinform.jmir.org/2023/1/e44842 %U https://doi.org/10.2196/44842 %U http://www.ncbi.nlm.nih.gov/pubmed/37603388 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e41552 %T Alerts and Collections for Automating Patients’ Sensemaking and Organizing of Their Electronic Health Record Data for Reflection, Planning, and Clinical Visits: Qualitative Research-Through-Design Study %A Nakikj,Drashko %A Kreda,David %A Gehlenborg,Nils %+ Department of Biomedical Informatics, Harvard Medical School, Harvard University, 10 Shattuck Street, Suite 514, Boston, MA, 02115, United States, 1 857 272 5075, nils@hms.harvard.edu %K patients %K electronic health records %K sensemaking %K pattern detection %K data organization %K alerts %K reports %K collections %D 2023 %7 21.8.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Electronic health record (EHR) data from multiple providers often exhibit important but convoluted and complex patterns that patients find hard and time-consuming to identify and interpret. However, existing patient-facing applications lack the capability to incorporate automatic pattern detection robustly and toward supporting making sense of the patient’s EHR data. In addition, there is no means to organize EHR data in an efficient way that suits the patient’s needs and makes them more actionable in real-life settings. These shortcomings often result in a skewed and incomplete picture of the patient’s health status, which may lead to suboptimal decision-making and actions that put the patient at risk. Objective: Our main goal was to investigate patients’ attitudes, needs, and use scenarios with respect to automatic support for surfacing important patterns in their EHR data and providing means for organizing them that best suit patients’ needs. Methods: We conducted an inquisitive research-through-design study with 14 participants. Presented in the context of a cutting-edge application with strong emphasis on independent EHR data sensemaking, called Discovery, we used high-level mock-ups for the new features that were supposed to support automatic identification of important data patterns and offer recommendations—Alerts—and means for organizing the medical records based on patients’ needs, much like photos in albums—Collections. The combined audio recording transcripts and in-study notes were analyzed using the reflexive thematic analysis approach. Results: The Alerts and Collections can be used for raising awareness, reflection, planning, and especially evidence-based patient-provider communication. Moreover, patients desired carefully designed automatic pattern detection with safe and actionable recommendations, which produced a well-tailored and scoped landscape of alerts for both potential threats and positive progress. Furthermore, patients wanted to contribute their own data (eg, progress notes) and log feelings, daily observations, and measurements to enrich the meaning and enable easier sensemaking of the alerts and collections. On the basis of the findings, we renamed Alerts to Reports for a more neutral tone and offered design implications for contextualizing the reports more deeply for increased actionability; automatically generating the collections for more expedited and exhaustive organization of the EHR data; enabling patient-generated data input in various formats to support coarser organization, richer pattern detection, and learning from experience; and using the reports and collections for efficient, reliable, and common-ground patient-provider communication. Conclusions: Patients need to have a flexible and rich way to organize and annotate their EHR data; be introduced to insights from these data—both positive and negative; and share these artifacts with their physicians in clinical visits or via messaging for establishing shared mental models for clear goals, agreed-upon priorities, and feasible actions. %M 37603400 %R 10.2196/41552 %U https://humanfactors.jmir.org/2023/1/e41552 %U https://doi.org/10.2196/41552 %U http://www.ncbi.nlm.nih.gov/pubmed/37603400 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e47434 %T A Deep Learning Model for the Normalization of Institution Names by Multisource Literature Feature Fusion: Algorithm Development Study %A Chen,Yifei %A Li,Xiaoying %A Li,Aihua %A Li,Yongjie %A Yang,Xuemei %A Lin,Ziluo %A Yu,Shirui %A Tang,Xiaoli %+ Institute of Medical Information, Chinese Academy of Medical Sciences, 69, Dongdan North Street, Beijing, 100005, China, 86 10 52328902, tang.xiaoli@imicams.ac.cn %K multisource literature %K institution name normalization %K deep learning %K bidirectional encoder representations from transformers %K BERT %D 2023 %7 18.8.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: The normalization of institution names is of great importance for literature retrieval, statistics of academic achievements, and evaluation of the competitiveness of research institutions. Differences in authors’ writing habits and spelling mistakes lead to various names of institutions, which affects the analysis of publication data. With the development of deep learning models and the increasing maturity of natural language processing methods, training a deep learning–based institution name normalization model can increase the accuracy of institution name normalization at the semantic level. Objective: This study aimed to train a deep learning–based model for institution name normalization based on the feature fusion of affiliation data from multisource literature, which would realize the normalization of institution name variants with the help of authority files and achieve a high specification accuracy after several rounds of training and optimization. Methods: In this study, an institution name normalization–oriented model was trained based on bidirectional encoder representations from transformers (BERT) and other deep learning models, including the institution classification model, institutional hierarchical relation extraction model, and institution matching and merging model. The model was then trained to automatically learn institutional features by pretraining and fine-tuning, and institution names were extracted from the affiliation data of 3 databases to complete the normalization process: Dimensions, Web of Science, and Scopus. Results: It was found that the trained model could achieve at least 3 functions. First, the model could identify the institution name that is consistent with the authority files and associate the name with the files through the unique institution ID. Second, it could identify the nonstandard institution name variants, such as singular forms, plural changes, and abbreviations, and update the authority files. Third, it could identify the unregistered institutions and add them to the authority files, so that when the institution appeared again, the model could identify and regard it as a registered institution. Moreover, the test results showed that the accuracy of the normalization model reached 93.79%, indicating the promising performance of the model for the normalization of institution names. Conclusions: The deep learning–based institution name normalization model trained in this study exhibited high accuracy. Therefore, it could be widely applied in the evaluation of the competitiveness of research institutions, analysis of research fields of institutions, and construction of interinstitutional cooperation networks, among others, showing high application value. %M 37594844 %R 10.2196/47434 %U https://formative.jmir.org/2023/1/e47434 %U https://doi.org/10.2196/47434 %U http://www.ncbi.nlm.nih.gov/pubmed/37594844 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e48155 %T Clinical Integration of Digital Patient-Reported Outcome Measures in Primary Health Care for Chronic Disease Management: Protocol for a Systematic Review %A Sasseville,Maxime %A Supper,Wilfried %A Gartner,Jean-Baptiste %A Layani,Géraldine %A Amil,Samira %A Sheffield,Peter %A Gagnon,Marie-Pierre %A Hudon,Catherine %A Lambert,Sylvie %A Attisso,Eugène %A Bureau Lagarde,Victoria %A Breton,Mylaine %A Poitras,Marie-Eve %A Pluye,Pierre %A Roux-Levy,Pierre-Henri %A Plaisimond,James %A Bergeron,Frédéric %A Ashcroft,Rachelle %A Wong,Sabrina %A Groulx,Antoine %A Beaudet,Nicolas %A Paquette,Jean-Sébastien %A D'Anjou,Natasha %A Langlois,Sylviane %A LeBlanc,Annie %+ Université Laval, 1050, rue de la Médecine, Québec, QC, G1V 0A6, Canada, 1 418 656 2131, maxime.sasseville@fsi.ulaval.ca %K systematic review %K patient-reported outcome measure %K primary healthcare %K health care %K implementation science %D 2023 %7 18.8.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Health measurement guides policies and health care decisions are necessary to describe and attain the quintuple aim of improving patient experience, population health, care team well-being, health care costs, and equity. In the primary care setting, patient-reported outcome measurement allows outcome comparisons within and across settings and helps improve the clinical management of patients. However, these digital patient-reported outcome measures (PROMs) are still not adapted to the clinical context of primary health care, which is an indication of the complexity of integrating these tools in this context. We must then gather evidence of their impact on chronic disease management in primary health care and understand the characteristics of effective implementation. Objective: We will conduct a systematic review to identify and assess the impact of electronic PROMs (ePROMs) implementation in primary health care for chronic disease management. Our specific objectives are to (1) determine the impact of ePROMs in primary health care for chronic disease management and (2) compare and contrast characteristics of effective ePROMs’ implementation strategies. Methods: We will conduct a systematic review of the literature in accordance with the guidelines of the Cochrane Methods Group and in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines for its reporting. A specific search strategy was developed for relevant databases to identify studies. Two reviewers will independently apply the inclusion criteria using full texts and will extract the data. We will use a 2-phase sequential mixed methods synthesis design by conducting a qualitative synthesis first, and use its results to perform a quantitative synthesis. Results: This study was initiated in June 2022 by assembling the research team and the knowledge transfer committee. The preliminary search strategy will be developed and completed in September 2022. The main search strategy, data collection, study selection, and application of inclusion criteria were completed between October and December 2022. Conclusions: Results from this review will help support implementation efforts to accelerate innovations and digital adoption for primary health care and will be relevant for improving clinical management of chronic diseases and health care services and policies. Trial Registration: PROSPERO International Prospective Register of Systematic Reviews CRD42022333513; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=333513 International Registered Report Identifier (IRRID): DERR1-10.2196/48155 %M 37594780 %R 10.2196/48155 %U https://www.researchprotocols.org/2023/1/e48155 %U https://doi.org/10.2196/48155 %U http://www.ncbi.nlm.nih.gov/pubmed/37594780 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e49933 %T Epidemiology of Motoric Cognitive Risk Syndrome in the Kerala Einstein Study: Protocol for a Prospective Cohort Study %A Sathyan,Sanish %A Ayers,Emmeline %A Blumen,Helena %A Weiss,Erica F %A Adhikari,Dristi %A Stimmel,Marnina %A Abdulsalam,Kizhakkaniyakath %A Noone,Mohan %A George,Roy K %A Ceide,Mirnova %A Ambrose,Anne Felicia %A Wang,Cuiling %A Narayanan,Poornima %A Sureshbabu,Sachin %A Shaji,Kunnukatil S %A Sigamani,Alben %A Mathuranath,Pavagada S %A Pradeep,Vayyattu G %A Verghese,Joe %+ Department of Neurology, Albert Einstein College of Medicine, 1225 Morris Park Avenue, Bronx, NY, 10461, United States, 1 7184303877, joe.verghese@einsteinmed.edu %K motoric cognitive risk %K Kerala %K India %K dementia %K cognitive decline %K neuroimaging %D 2023 %7 17.8.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: The southern India state of Kerala has among the highest proportion of older adults in its population in the country. An increase in chronic age-related diseases such as dementia is expected in the older Kerala population. Identifying older individuals early in the course of cognitive decline offers the best hope of introducing preventive measures early and planning management. However, the epidemiology and pathogenesis of predementia syndromes at the early stages of cognitive decline in older adults are not well established in India. Objective: The Kerala Einstein Study (KES) is a community-based cohort study that was established in 2008 and is based in the Kozhikode district in Kerala state. KES aims to establish risk factors and brain substrates of motoric cognitive risk syndrome (MCR), a predementia syndrome characterized by the presence of slow gait and subjective cognitive concerns in individuals without dementia or disability. This protocol describes the study design and procedures for this KES project. Methods: KES is proposing to enroll a sample of 1000 adults ≥60 years old from urban and rural areas in the Kozhikode district of Kerala state: 200 recruited in the previous phase of KES and 800 new participants to be recruited in this project. MCR is the cognitive phenotype of primary interest. The associations between previously established risk factors for dementia as well as novel risk factors (apathy and traumatic brain injury) and MCR will be examined in KES. Risk factor profiles for MCR will be compared between urban and rural residents as well as with individuals who meet the criteria for mild cognitive impairment (MCI). Cognitive and physical function, medical history and medications, sociodemographic characteristics, lifestyle patterns, and activities of daily living will be evaluated. Participants will also undergo magnetic resonance imaging and electrocardiogram investigations. Longitudinal follow-up is planned in a subset of participants as a prelude to future longitudinal studies. Results: KES (2R01AG039330-07) was funded by the US National Institutes of Health in September 2019 and received approval from the Indian Medical Council of Research to start the study in June 2021. We had recruited 433 new participants from urban and rural sites in Kozhikode as of May 2023: 41.1% (178/433) women, 67.7% (293/433) rural residents, and 13.4% (58/433) MCR cases. Enrollment is actively ongoing at all the KES recruitment sites. Conclusions: KES will provide new insights into risk factors and brain substrates associated with MCR in India and will help guide future development of regionally specific preventive interventions for dementia. International Registered Report Identifier (IRRID): DERR1-10.2196/49933 %M 37590054 %R 10.2196/49933 %U https://www.researchprotocols.org/2023/1/e49933 %U https://doi.org/10.2196/49933 %U http://www.ncbi.nlm.nih.gov/pubmed/37590054 %0 Journal Article %@ 2562-7600 %I JMIR Publications %V 6 %N %P e46819 %T Sociotechnical Challenges of Digital Health in Nursing Practice During the COVID-19 Pandemic: National Study %A Livesay,Karen %A Petersen,Sacha %A Walter,Ruby %A Zhao,Lin %A Butler-Henderson,Kerryn %A Abdolkhani,Robab %+ School of Health and Biomedical Sciences, Science, Technology, Engineering, and Mathematics College, Royal Melbourne Institute of Technology University, 289 McKimmies Rd, Bundoora, Melbourne, 3083, Australia, 61 98098654, robab.abdolkhani@rmit.edu.au %K nursing informatics %K digital health %K COVID-19 pandemic %K workforce %K sociotechnical approach %D 2023 %7 16.8.2023 %9 Original Paper %J JMIR Nursing %G English %X Background: The COVID-19 pandemic has accelerated the use of digital health innovations, which has greatly impacted nursing practice. However, little is known about the use of digital health services by nurses and how this has changed during the pandemic. Objective: This study explored the sociotechnical challenges that nurses encountered in using digital health services implemented during the pandemic and, accordingly, what digital health capabilities they expect from the emerging workforce. Methods: Five groups of nurses, including chief nursing information officers, nurses, clinical educators, nurse representatives at digital health vendor companies, and nurse representatives in government bodies across Australia were interviewed. They were asked about their experience of digital health during the pandemic, their sociotechnical challenges, and their expectations of the digital health capabilities of emerging nurses to overcome these challenges. Interviews were deductively analyzed based on 8 sociotechnical themes, including technical challenges, nurse-technology interaction, clinical content management, training and human resources, communication and workflow, internal policies and guidelines, external factors, and effectiveness assessment of digital health for postpandemic use. Results: Sixteen participants were interviewed. Human factors and clinical workflow challenges were highly mentioned. Nurses’ lack of knowledge and involvement in digital health implementation and evaluation led to inefficient use of these technologies during the pandemic. They expected the emerging workforce to be digitally literate and actively engaged in digital health interventions beyond documentation, such as data analytics and decision-making. Conclusions: Nurses should be involved in digital health interventions to efficiently use these technologies and provide safe and quality care. Collaborative efforts among policy makers, vendors, and clinical and academic industries can leverage digital health capabilities in the nursing workforce. %M 37585256 %R 10.2196/46819 %U https://nursing.jmir.org/2023/1/e46819 %U https://doi.org/10.2196/46819 %U http://www.ncbi.nlm.nih.gov/pubmed/37585256 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e48009 %T Ethical Considerations of Using ChatGPT in Health Care %A Wang,Changyu %A Liu,Siru %A Yang,Hao %A Guo,Jiulin %A Wu,Yuxuan %A Liu,Jialin %+ Information Center, West China Hospital, Sichuan University, No 37 Guo Xue Xiang, Chengdu, 610041, China, 86 28 85422306, DLJL8@163.com %K ethics %K ChatGPT %K artificial intelligence %K AI %K large language models %K health care %K artificial intelligence development %K development %K algorithm %K patient safety %K patient privacy %K safety %K privacy %D 2023 %7 11.8.2023 %9 Viewpoint %J J Med Internet Res %G English %X ChatGPT has promising applications in health care, but potential ethical issues need to be addressed proactively to prevent harm. ChatGPT presents potential ethical challenges from legal, humanistic, algorithmic, and informational perspectives. Legal ethics concerns arise from the unclear allocation of responsibility when patient harm occurs and from potential breaches of patient privacy due to data collection. Clear rules and legal boundaries are needed to properly allocate liability and protect users. Humanistic ethics concerns arise from the potential disruption of the physician-patient relationship, humanistic care, and issues of integrity. Overreliance on artificial intelligence (AI) can undermine compassion and erode trust. Transparency and disclosure of AI-generated content are critical to maintaining integrity. Algorithmic ethics raise concerns about algorithmic bias, responsibility, transparency and explainability, as well as validation and evaluation. Information ethics include data bias, validity, and effectiveness. Biased training data can lead to biased output, and overreliance on ChatGPT can reduce patient adherence and encourage self-diagnosis. Ensuring the accuracy, reliability, and validity of ChatGPT-generated content requires rigorous validation and ongoing updates based on clinical practice. To navigate the evolving ethical landscape of AI, AI in health care must adhere to the strictest ethical standards. Through comprehensive ethical guidelines, health care professionals can ensure the responsible use of ChatGPT, promote accurate and reliable information exchange, protect patient privacy, and empower patients to make informed decisions about their health care. %M 37566454 %R 10.2196/48009 %U https://www.jmir.org/2023/1/e48009 %U https://doi.org/10.2196/48009 %U http://www.ncbi.nlm.nih.gov/pubmed/37566454 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e45043 %T Examining Care Planning Efficiency and Clinical Decision Support Adoption in a System Tailoring to Nurses’ Graph Literacy: National, Web-Based Randomized Controlled Trial %A Yao,Yingwei %A Dunn Lopez,Karen %A Bjarnadottir,Ragnhildur I %A Macieira,Tamara G R %A Dos Santos,Fabiana Cristina %A Madandola,Olatunde O %A Cho,Hwayoung %A Priola,Karen J B %A Wolf,Jessica %A Wilkie,Diana J %A Keenan,Gail %+ University of Florida College of Nursing, 1225 Center Dr, Gainesville, FL, 32610, United States, 1 352 273 6352, gkeenan@ufl.edu %K clinical decision support %K nurse decision-making %K nurse care planning %K simulation %K remote testing %K tailored interfaces %K graph literacy %K cognitive workload %D 2023 %7 11.8.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: The proliferation of health care data in electronic health records (EHRs) is fueling the need for clinical decision support (CDS) that ensures accuracy and reduces cognitive processing and documentation burden. The CDS format can play a key role in achieving the desired outcomes. Building on our laboratory-based pilot study with 60 registered nurses (RNs) from 1 Midwest US metropolitan area indicating the importance of graph literacy (GL), we conducted a fully powered, innovative, national, and web-based randomized controlled trial with 203 RNs. Objective: This study aimed to compare care planning time (CPT) and the adoption of evidence-based CDS recommendations by RNs randomly assigned to 1 of 4 CDS format groups: text only (TO), text+table (TT), text+graph (TG), and tailored (based on the RN’s GL score). We hypothesized that the tailored CDS group will have faster CPT (primary) and higher adoption rates (secondary) than the 3 nontailored CDS groups. Methods: Eligible RNs employed in an adult hospital unit within the past 2 years were recruited randomly from 10 State Board of Nursing lists representing the 5 regions of the United States (Northeast, Southeast, Midwest, Southwest, and West) to participate in a randomized controlled trial. RNs were randomly assigned to 1 of 4 CDS format groups—TO, TT, TG, and tailored (based on the RN’s GL score)—and interacted with the intervention on their PCs. Regression analysis was performed to estimate the effect of tailoring and the association between CPT and RN characteristics. Results: The differences between the tailored (n=46) and nontailored (TO, n=55; TT, n=54; and TG, n=48) CDS groups were not significant for either the CPT or the CDS adoption rate. RNs with low GL had longer CPT interacting with the TG CDS format than the TO CDS format (P=.01). The CPT in the TG CDS format was associated with age (P=.02), GL (P=.02), and comfort with EHRs (P=.047). Comfort with EHRs was also associated with CPT in the TT CDS format (P<.001). Conclusions: Although tailoring based on GL did not improve CPT or adoption, the study reinforced previous pilot findings that low GL is associated with longer CPT when graphs were included in care planning CDS. Higher GL, younger age, and comfort with EHRs were associated with shorter CPT. These findings are robust based on our new innovative testing strategy in which a diverse national sample of RN participants (randomly derived from 10 State Board of Nursing lists) interacted on the web with the intervention on their PCs. Future studies applying our innovative methodology are recommended to cost-effectively enhance the understanding of how the RN’s GL, combined with additional factors, can inform the development of efficient CDS for care planning and other EHR components before use in practice. %M 37566456 %R 10.2196/45043 %U https://www.jmir.org/2023/1/e45043 %U https://doi.org/10.2196/45043 %U http://www.ncbi.nlm.nih.gov/pubmed/37566456 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e46471 %T Data Quality– and Utility-Compliant Anonymization of Common Data Model–Harmonized Electronic Health Record Data: Protocol for a Scoping Review %A Kamdje Wabo,Gaetan %A Prasser,Fabian %A Gierend,Kerstin %A Siegel,Fabian %A Ganslandt,Thomas %+ Department of Biomedical Informatics, Center for Preventive Medicine and Digital Health Baden-Württemberg, Mannheim Medical Faculty of the University of Heidelberg, Theodor-Kutzer-Ufer 1-3, House 3, Floor 4, Mannheim, 68167, Germany, 49 621 383 8088, gaetankamdje.wabo@medma.uni-heidelberg.de %K EHR %K electronic health record %K data quality %K common data model %K data standard %K data privacy models %K data anonymization %D 2023 %7 11.8.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: The anonymization of Common Data Model (CDM)–converted EHR data is essential to ensure the data privacy in the use of harmonized health care data. However, applying data anonymization techniques can significantly affect many properties of the resulting data sets and thus biases research results. Few studies have reviewed these applications with a reflection of approaches to manage data utility and quality concerns in the context of CDM-formatted health care data. Objective: Our intended scoping review aims to identify and describe (1) how formal anonymization methods are carried out with CDM-converted health care data, (2) how data quality and utility concerns are considered, and (3) how the various CDMs differ in terms of their suitability for recording anonymized data. Methods: The planned scoping review is based on the framework of Arksey and O'Malley. By using this, only articles published in English will be included. The retrieval of literature items should be based on a literature search string combining keywords related to data anonymization, CDM standards, and data quality assessment. The proposed literature search query should be validated by a librarian, accompanied by manual searches to include further informal sources. Eligible articles will first undergo a deduplication step, followed by the screening of titles. Second, a full-text reading will allow the 2 reviewers involved to reach the final decision about article selection, while a domain expert will support the resolution of citation selection conflicts. Additionally, key information will be extracted, categorized, summarized, and analyzed by using a proposed template into an iterative process. Tabular and graphical analyses should be addressed in alignment with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) checklist. We also performed some tentative searches on Web of Science for estimating the feasibility of reaching eligible articles. Results: Tentative searches on Web of Science resulted in 507 nonduplicated matches, suggesting the availability of (potential) relevant articles. Further analysis and selection steps will allow us to derive a final literature set. Furthermore, the completion of this scoping review study is expected by the end of the fourth quarter of 2023. Conclusions: Outlining the approaches of applying formal anonymization methods on CDM-formatted health care data while taking into account data quality and utility concerns should provide useful insights to understand the existing approaches and future research direction based on identified gaps. This protocol describes a schedule to perform a scoping review, which should support the conduction of follow-up investigations. International Registered Report Identifier (IRRID): PRR1-10.2196/46471 %M 37566443 %R 10.2196/46471 %U https://www.researchprotocols.org/2023/1/e46471 %U https://doi.org/10.2196/46471 %U http://www.ncbi.nlm.nih.gov/pubmed/37566443 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e48363 %T Challenges and Solutions in Implementing eSource Technology for Real-World Studies in China: Qualitative Study Among Different Stakeholders %A Wang,Bin %A Lai,Junkai %A Liao,Xiwen %A Jin,Feifei %A Yao,Chen %+ Peking University Clinical Research Institute, Peking University First Hospital, No. 8 Xishiku Street, Xicheng District, Beijing, 100191, China, 86 01066551053, yaochen@hsc.pku.edu.cn %K electronic medical record %K electronic source %K eSource %K challenge %K real-world data %K interoperability %D 2023 %7 10.8.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: eSources consist of data that were initially documented in an electronic structure. Typically, an eSource encompasses the direct acquisition, compilation, and retention of electronic information (such as electronic health records [EHRs] or wearable devices), which serves to streamline clinical research. eSources have the potential to enhance the accuracy of data, promote patient safety, and minimize expenses associated with clinical trials. An opinion study published in September 2020 by TransCelerate outlined a road map for the future application of eSource technology and identified 5 key areas of challenges. The background of this study concerns the use of eSource technology in clinical research. Objective: The aim of this study was to present challenges and possible solutions for the implementation of eSource technology in real-world studies by summarizing team experiences and lessons learned from an eSource record (ESR) project. Methods: After initially developing a simple prototype of the ESR software that can be demonstrated systematically, the researchers conducted in-depth interviews and interacted with different stakeholders to obtain guidance and suggestions. The researchers selected 5 different roles for interviewees: regulatory authorities, pharmaceutical company representatives, hospital information department employees, medical system providers, and clinicians. Results: After screening all consultants, the researchers concluded that there were 25 representative consultants. The hospital information department needs to implement many demands from various stakeholders, which makes the existing EHR system unable to meet all the demands of eSources. The emergence of an ESR is intended to divert the burden of the hospital information department from the enormous functional requirements of the outdated EHR system. The entire research process emphasizes multidisciplinary and multibackground expert opinions and considers the complexity of the knowledge backgrounds of personnel involved in the chain of clinical source data collection, processing, quality control, and application in real-world scenarios. To increase the readability of the results, the researchers classified the main results in accordance with the paragraph titles in “Use of Electronic Health Record Data in Clinical Investigations,” a guide released by the US Food and Drug Administration. Conclusions: This study introduces the requirement dependencies of different stakeholders and the challenges and recommendations for designing ESR software when implementing eSource technology in China. Experiences based on ESR projects will provide new insights into the disciplines that advance the eSource research field. Future studies should engage patients directly to understand their experiences, concerns, and preferences regarding the implementation of eSource technology. Moreover, involving additional stakeholders, including community health care providers and social workers, will provide valuable insights into the challenges and potential solutions across various health care settings. %M 37561551 %R 10.2196/48363 %U https://formative.jmir.org/2023/1/e48363 %U https://doi.org/10.2196/48363 %U http://www.ncbi.nlm.nih.gov/pubmed/37561551 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e46761 %T Chatbots to Improve Sexual and Reproductive Health: Realist Synthesis %A Mills,Rhiana %A Mangone,Emily Rose %A Lesh,Neal %A Mohan,Diwakar %A Baraitser,Paula %+ SH24, 35A Westminster Bridge Road, London, SE1 7JB, United Kingdom, 44 7742932445, rhiana@sh24.org.uk %K chatbot %K sexual and reproductive health %K realist synthesis %K social networks %K service networks %K disclosure %K artificial intelligence %K sexual %K reproductive %K social media %K counseling %K treatment %K development %K theory %K digital device %K device %D 2023 %7 9.8.2023 %9 Review %J J Med Internet Res %G English %X Background: Digital technologies may improve sexual and reproductive health (SRH) across diverse settings. Chatbots are computer programs designed to simulate human conversation, and there is a growing interest in the potential for chatbots to provide responsive and accurate information, counseling, linkages to products and services, or a companion on an SRH journey. Objective: This review aimed to identify assumptions about the value of chatbots for SRH and collate the evidence to support them. Methods: We used a realist approach that starts with an initial program theory and generates causal explanations in the form of context, mechanism, and outcome configurations to test and develop that theory. We generated our program theory, drawing on the expertise of the research team, and then searched the literature to add depth and develop this theory with evidence. Results: The evidence supports our program theory, which suggests that chatbots are a promising intervention for SRH information and service delivery. This is because chatbots offer anonymous and nonjudgmental interactions that encourage disclosure of personal information, provide complex information in a responsive and conversational tone that increases understanding, link to SRH conversations within web-based and offline social networks, provide immediate support or service provision 24/7 by automating some tasks, and provide the potential to develop long-term relationships with users who return over time. However, chatbots may be less valuable where people find any conversation about SRH (even with a chatbot) stigmatizing, for those who lack confidential access to digital devices, where conversations do not feel natural, and where chatbots are developed as stand-alone interventions without reference to service contexts. Conclusions: Chatbots in SRH could be developed further to automate simple tasks and support service delivery. They should prioritize achieving an authentic conversational tone, which could be developed to facilitate content sharing in social networks, should support long-term relationship building with their users, and should be integrated into wider service networks. %M 37556194 %R 10.2196/46761 %U https://www.jmir.org/2023/1/e46761 %U https://doi.org/10.2196/46761 %U http://www.ncbi.nlm.nih.gov/pubmed/37556194 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e50231 %T Examining the Use of Text Messages Among Multidisciplinary Care Teams to Reduce Avoidable Hospitalization of Nursing Home Residents with Dementia: Protocol for a Secondary Analysis %A Powell,Kimberly R %A Popescu,Mihail %A Lee,Suhwon %A Mehr,David R %A Alexander,Gregory L %+ Sinclair School of Nursing, University of Missouri, 915 Hitt Street, Columbia, MO, 65211, United States, 1 5026407556, powellk@missouri.edu %K age-friendly health systems %K Alzheimer disease %K communication %K dementia %K nursing homes %K older adults %D 2023 %7 9.8.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Reducing avoidable nursing home (NH)–to-hospital transfers of residents with Alzheimer disease or a related dementia (ADRD) has become a national priority due to the physical and emotional toll it places on residents and the high costs to Medicare and Medicaid. Technologies supporting the use of clinical text messages (TMs) could improve communication among health care team members and have considerable impact on reducing avoidable NH-to-hospital transfers. Although text messaging is a widely accepted mechanism of communication, clinical models of care using TMs are sparsely reported in the literature, especially in NHs. Protocols for assessing technologies that integrate TMs into care delivery models would be beneficial for end users of these systems. Without evidence to support clinical models of care using TMs, users are left to design their own methods and protocols for their use, which can create wide variability and potentially increase disparities in resident outcomes. Objective: Our aim is to describe the protocol of a study designed to understand how members of the multidisciplinary team communicate using TMs and how salient and timely communication can be used to avert poor outcomes for NH residents with ADRD, including hospitalization. Methods: This project is a secondary analysis of data collected from a Centers for Medicare & Medicaid Services (CMS)–funded demonstration project designed to reduce avoidable hospitalizations for long-stay NH residents. We will use two data sources: (1) TMs exchanged among the multidisciplinary team across the 7-year CMS study period (August 2013-September 2020) and (2) an adapted acute care transfer tool completed by advanced practice registered nurses to document retrospective details about NH-to-hospital transfers. The study is guided by an age-friendly model of care called the 4Ms (What Matters, Medications, Mentation, and Mobility) framework. We will use natural language processing, statistical methods, and social network analysis to generate a new ontology and to compare communication patterns found in TMs occurring around the time NH-to-hospital transfer decisions were made about residents with and without ADRD. Results: After accounting for inclusion and exclusion criteria, we will analyze over 30,000 TMs pertaining to over 3600 NH-to-hospital transfers. Development of the 4M ontology is in progress, and the 3-year project is expected to run until mid-2025. Conclusions: To our knowledge, this project will be the first to explore the content of TMs exchanged among a multidisciplinary team of care providers as they make decisions about NH-to-hospital resident transfers. Understanding how the presence of evidence-based elements of high-quality care relate to avoidable hospitalizations among NH residents with ADRD will generate knowledge regarding the future scalability of behavioral interventions. Without this knowledge, NHs will continue to rely on ineffective and outdated communication methods that fail to account for evidence-based elements of age-friendly care. International Registered Report Identifier (IRRID): DERR1-10.2196/50231 %M 37556199 %R 10.2196/50231 %U https://www.researchprotocols.org/2023/1/e50231 %U https://doi.org/10.2196/50231 %U http://www.ncbi.nlm.nih.gov/pubmed/37556199 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e47958 %T Automated Electronic Health Record to Electronic Data Capture Transfer in Clinical Studies in the German Health Care System: Feasibility Study and Gap Analysis %A Mueller,Christian %A Herrmann,Patrick %A Cichos,Stephan %A Remes,Bernhard %A Junker,Erwin %A Hastenteufel,Tobias %A Mundhenke,Markus %+ Bayer Vital GmbH, Kaiser-Wilhelm-Allee 70, Leverkusen, 51373, Germany, 49 175 3005134, christian.mueller4@bayer.com %K digital transformation %K automated data transfer %K electronic medical record %K electronic data capture %K EDC %K data transfer %K electronic health record %K EHR %K digital transfer %K barrier %K clinical practice %K EHR2EDC %K health care system %D 2023 %7 4.8.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Data transfer between electronic health records (EHRs) at the point of care and electronic data capture (EDC) systems for clinical research is still mainly carried out manually, which is error-prone as well as cost- and time-intensive. Automated digital transfer from EHRs to EDC systems (EHR2EDC) would enable more accurate and efficient data capture but has so far encountered technological barriers primarily related to data format and the technological environment: in Germany, health care data are collected at the point of care in a variety of often individualized practice management systems (PMSs), most of them not interoperable. Data quality for research purposes within EDC systems must meet the requirements of regulatory authorities for standardized submission of clinical trial data and safety reports. Objective: We aimed to develop a model for automated data transfer as part of an observational study that allows data of sufficient quality to be captured at the point of care, extracted from various PMSs, and automatically transferred to electronic case report forms in EDC systems. This required addressing aspects of data security, as well as the lack of compatibility between EHR health care data and the data quality required in EDC systems for clinical research. Methods: The SaniQ software platform (Qurasoft GmbH) is already used to extract and harmonize predefined variables from electronic medical records of different Compu Group Medical–hosted PMSs. From there, data are automatically transferred to the validated AlcedisTRIAL EDC system (Alcedis GmbH) for data collection and management. EHR2EDC synchronization occurs automatically overnight, and real-time updates can be initiated manually following each data entry in the EHR. The electronic case report form (eCRF) contains 13 forms with 274 variables. Of these, 5 forms with 185 variables contain 67 automatically transferable variables (67/274, 24% of all variables and 67/185, 36% of eligible variables). Results: This model for automated data transfer bridges the current gap between clinical practice data capture at the point of care and the data sets required by regulatory agencies; it also enables automated EHR2EDC data transfer in compliance with the General Data Protection Regulation (GDPR). It addresses feasibility, connectivity, and system compatibility of currently used PMSs in health care and clinical research and is therefore directly applicable. Conclusions: This use case demonstrates that secure, consistent, and automated end-to-end data transmission from the treating physician to the regulatory authority is feasible. Automated data transmission can be expected to reduce effort and save resources and costs while ensuring high data quality. This may facilitate the conduct of studies for both study sites and sponsors, thereby accelerating the development of new drugs. Nevertheless, the industry-wide implementation of EHR2EDC requires policy decisions that set the framework for the use of research data based on routine PMS data. %M 37540555 %R 10.2196/47958 %U https://www.jmir.org/2023/1/e47958 %U https://doi.org/10.2196/47958 %U http://www.ncbi.nlm.nih.gov/pubmed/37540555 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e45116 %T Methods Used in the Development of Common Data Models for Health Data: Scoping Review %A Ahmadi,Najia %A Zoch,Michele %A Kelbert,Patricia %A Noll,Richard %A Schaaf,Jannik %A Wolfien,Markus %A Sedlmayr,Martin %+ Institute for Medical Informatics and Biometry, Carl Gustav Carus Faculty of Medicine, Technische Universität Dresden, Fetscherstr 74, Dresden, 01307, Germany, 49 351458 87 7704, najia.ahmadi@tu-dresden.de %K common data model %K common data elements %K health data %K electronic health record %K Observational Medical Outcomes Partnership %K stakeholder involvement %K Data harmonisation %K Interoperability %K Standardized Data Repositories %K Suggestive Development Process %K Healthcare %K Medical Informatics %K %D 2023 %7 3.8.2023 %9 Review %J JMIR Med Inform %G English %X Background: Common data models (CDMs) are essential tools for data harmonization, which can lead to significant improvements in the health domain. CDMs unite data from disparate sources and ease collaborations across institutions, resulting in the generation of large standardized data repositories across different entities. An overview of existing CDMs and methods used to develop these data sets may assist in the development process of future models for the health domain, such as for decision support systems. Objective: This scoping review investigates methods used in the development of CDMs for health data. We aim to provide a broad overview of approaches and guidelines that are used in the development of CDMs (ie, common data elements or common data sets) for different health domains on an international level. Methods: This scoping review followed the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist. We conducted the literature search in prominent databases, namely, PubMed, Web of Science, Science Direct, and Scopus, starting from January 2000 until March 2022. We identified and screened 1309 articles. The included articles were evaluated based on the type of adopted method, which was used in the conception, users’ needs collection, implementation, and evaluation phases of CDMs, and whether stakeholders (such as medical experts, patients’ representatives, and IT staff) were involved during the process. Moreover, the models were grouped into iterative or linear types based on the imperativeness of the stages during development. Results: We finally identified 59 articles that fit our eligibility criteria. Of these articles, 45 specifically focused on common medical conditions, 10 focused on rare medical conditions, and the remaining 4 focused on both conditions. The development process usually involved stakeholders but in different ways (eg, working group meetings, Delphi approaches, interviews, and questionnaires). Twenty-two models followed an iterative process. Conclusions: The included articles showed the diversity of methods used to develop a CDM in different domains of health. We highlight the need for more specialized CDM development methods in the health domain and propose a suggestive development process that might ease the development of CDMs in the health domain in the future. %M 37535410 %R 10.2196/45116 %U https://medinform.jmir.org/2023/1/e45116 %U https://doi.org/10.2196/45116 %U http://www.ncbi.nlm.nih.gov/pubmed/37535410 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e37447 %T Examining Diversity in Digital Therapeutics Clinical Trials: Descriptive Analysis %A Adu-Brimpong,Joel %A Pugh,Jennifer %A Darko,David Agyen %A Shieh,Lisa %+ School of Medicine, Stanford University, 291 Campus Drive, Stanford, CA, 94305, United States, 1 269 487 7766, jadu@stanford.edu %K digital therapeutics, DTx %K clinical trials, health informatics %K digital health %K digital medicine %K DTx clinical trials %K health equity %K digital therapy %K demographic %K sociodemographic %K representation %K software %K telehealth %K telemedicine %D 2023 %7 2.8.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Digital therapeutics (DTx) are an emerging class of software-based medical therapies helping to improve care access and delivery. As we leverage these digital health therapies broadly in clinical care, it is important to consider sociodemographic representation underlying clinical trials data to ensure broad application to all groups. Objective: We review current sociodemographic representation in DTx clinical trials using data from the Digital Therapeutics Alliance Product Library database. Methods: We conducted a descriptive analysis of DTx products. We analyzed 15 manuscripts associated with 13 DTx products. Sociodemographic information was retrieved and compared with the US population’s demographic distribution. Results: The median study size and age of participants were 252 and 43.3 years, respectively. Of the 15 studies applicable to this study, 10 (67%) reported that females made up 65% or greater of the study cohort. A total of 14 studies reported race data with Black or African American and Asian American individuals underrepresented in 9 and 11 studies, respectively. In 7 studies that reported ethnicity, Hispanics were underrepresented in all 7 studies. Furthermore, 8 studies reported education levels, with 5 studies reporting populations in which 70% or greater had at least some college education. Only 3 studies reported health insurance information, each reporting a study cohort in which 100% of members were privately insured. Conclusions: Our findings indicate opportunities for improved sociodemographic representation in DTx clinical trials, especially for underserved populations typically underrepresented in clinical trials. This review is a step in examining sociodemographic representation in DTx clinical trials to help inform the path forward for DTx development and testing. %M 37531157 %R 10.2196/37447 %U https://www.jmir.org/2023/1/e37447 %U https://doi.org/10.2196/37447 %U http://www.ncbi.nlm.nih.gov/pubmed/37531157 %0 Journal Article %@ 2817-092X %I JMIR Publications %V 2 %N %P e46021 %T A Semantic Relatedness Model for the Automatic Cluster Analysis of Phonematic and Semantic Verbal Fluency Tasks Performed by People With Parkinson Disease: Prospective Multicenter Study %A Hähnel,Tom %A Feige,Tim %A Kunze,Julia %A Epler,Andrea %A Frank,Anika %A Bendig,Jonas %A Schnalke,Nils %A Wolz,Martin %A Themann,Peter %A Falkenburger,Björn %+ Department of Neurology, University Hospital and Faculty of Medicine Carl Gustav Carus, Technische Universität Dresden, Fetscherstraße 74, Dresden, 01307, Germany, 49 351 458 ext 11880, tom.haehnel@uniklinikum-dresden.de %K cognition %K executive function %K language function %K mild cognitive impairment %K Parkinson disease %K Parkinson disease dementia %K semantic clusters %K semantic relatedness %K verbal fluency tasks %D 2023 %7 2.8.2023 %9 Original Paper %J JMIR Neurotech %G English %X Background: Phonematic and semantic verbal fluency tasks (VFTs) are widely used to capture cognitive deficits in people with neurodegenerative diseases. Counting the total number of words produced within a given time frame constitutes the most commonly used analysis for VFTs. The analysis of semantic and phonematic word clusters can provide additional information about frontal and temporal cognitive functions. Traditionally, clusters in the semantic VFT are identified using fixed word lists, which need to be created manually, lack standardization, and are language specific. Furthermore, it is not possible to identify semantic clusters in the phonematic VFT using this technique. Objective: The objective of this study was to develop a method for the automated analysis of semantically related word clusters for semantic and phonematic VFTs. Furthermore, we aimed to explore the cognitive domains captured by this analysis for people with Parkinson disease (PD). Methods: People with PD performed tablet-based semantic (51/85, 60%) and phonematic (69/85, 81%) VFTs. For both tasks, semantic word clusters were determined using a semantic relatedness model based on a neural network trained on the Wikipedia (Wikimedia Foundation) text corpus. The cluster characteristics derived from this model were compared with those derived from traditional evaluation methods of VFTs and a set of neuropsychological parameters. Results: For the semantic VFT, the cluster characteristics obtained through automated analyses showed good correlations with the cluster characteristics obtained through the traditional method. Cluster characteristics from automated analyses of phonematic and semantic VFTs correlated with the overall cognitive function reported by the Montreal Cognitive Assessment, executive function reported by the Frontal Assessment Battery and the Trail Making Test, and language function reported by the Boston Naming Test. Conclusions: Our study demonstrated the feasibility of standardized automated cluster analyses of VFTs using semantic relatedness models. These models do not require manually creating and updating categorized word lists and, therefore, can be easily and objectively implemented in different languages, potentially allowing comparison of results across different languages. Furthermore, this method provides information about semantic clusters in phonematic VFTs, which cannot be obtained from traditional methods. Hence, this method could provide easily accessible digital biomarkers for executive and language functions in people with PD. %R 10.2196/46021 %U https://neuro.jmir.org/2023/1/e46021 %U https://doi.org/10.2196/46021 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e49034 %T Effects of Combinational Use of Additional Differential Diagnostic Generators on the Diagnostic Accuracy of the Differential Diagnosis List Developed by an Artificial Intelligence–Driven Automated History–Taking System: Pilot Cross-Sectional Study %A Harada,Yukinori %A Tomiyama,Shusaku %A Sakamoto,Tetsu %A Sugimoto,Shu %A Kawamura,Ren %A Yokose,Masashi %A Hayashi,Arisa %A Shimizu,Taro %+ Department of Diagnostic and Generalist Medicine, Dokkyo Medical University, 880 Kitakobayashi, Mibu, Shimotsugagun, 321-0293, Japan, 81 282 86 1111, yharada@dokkyomed.ac.jp %K collective intelligence %K differential diagnosis generator %K diagnostic accuracy %K automated medical history taking system %K artificial intelligence %K AI %D 2023 %7 2.8.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Low diagnostic accuracy is a major concern in automated medical history–taking systems with differential diagnosis (DDx) generators. Extending the concept of collective intelligence to the field of DDx generators such that the accuracy of judgment becomes higher when accepting an integrated diagnosis list from multiple people than when accepting a diagnosis list from a single person may be a possible solution. Objective: The purpose of this study is to assess whether the combined use of several DDx generators improves the diagnostic accuracy of DDx lists. Methods: We used medical history data and the top 10 DDx lists (index DDx lists) generated by an artificial intelligence (AI)–driven automated medical history–taking system from 103 patients with confirmed diagnoses. Two research physicians independently created the other top 10 DDx lists (second and third DDx lists) per case by imputing key information into the other 2 DDx generators based on the medical history generated by the automated medical history–taking system without reading the index lists generated by the automated medical history–taking system. We used the McNemar test to assess the improvement in diagnostic accuracy from the index DDx lists to the three types of combined DDx lists: (1) simply combining DDx lists from the index, second, and third lists; (2) creating a new top 10 DDx list using a 1/n weighting rule; and (3) creating new lists with only shared diagnoses among DDx lists from the index, second, and third lists. We treated the data generated by 2 research physicians from the same patient as independent cases. Therefore, the number of cases included in analyses in the case using 2 additional lists was 206 (103 cases × 2 physicians’ input). Results: The diagnostic accuracy of the index lists was 46% (47/103). Diagnostic accuracy was improved by simply combining the other 2 DDx lists (133/206, 65%, P<.001), whereas the other 2 combined DDx lists did not improve the diagnostic accuracy of the DDx lists (106/206, 52%, P=.05 in the collective list with the 1/n weighting rule and 29/206, 14%, P<.001 in the only shared diagnoses among the 3 DDx lists). Conclusions: Simply adding each of the top 10 DDx lists from additional DDx generators increased the diagnostic accuracy of the DDx list by approximately 20%, suggesting that the combinational use of DDx generators early in the diagnostic process is beneficial. %M 37531164 %R 10.2196/49034 %U https://formative.jmir.org/2023/1/e49034 %U https://doi.org/10.2196/49034 %U http://www.ncbi.nlm.nih.gov/pubmed/37531164 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e44399 %T An Assessment of Patient Portal Messaging Use by Patients With Multiple Chronic Conditions Living in Rural Communities: Retrospective Analysis %A Chivela,Fernando L %A Burch,Ashley E %A Asagbra,Oghale %+ Department of Health Services and Information Management, East Carolina University, 4340N Health Sciences Building, 2150 West 5th Street, Greenville, NC, 27834, United States, 1 2527441205, burchas15@ecu.edu %K patient portal %K multimorbidity %K chronic disease %K patient messaging %K rural %K mobile phone %D 2023 %7 1.8.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Patient portals can facilitate the delivery of health care services and support self-management for patients with multiple chronic conditions. Despite their benefits, the evidence of patient portal use among patients with multimorbidity in rural communities is limited. Objective: This study aimed to explore the factors associated with portal messaging use by rural patients. Methods: We assessed patient portal use among patients with ≥1 chronic diagnoses who sent or received messages via the Epic MyChart (Epic Systems Corporation) portal between January 1, 2015, and November 9, 2021. Patient portal use was defined as sending or receiving a message through the portal during the study period. We fit a zero-inflated negative binomial model to predict portal use based on the patient’s number of chronic conditions, sex, race, age, marital status, and insurance type. County-level characteristics, based on the patient’s home address, were also included in the model to assess the influence of community factors on portal use. County-level factors included educational attainment, smartphone ownership, median income, and primary care provider density. Results: A total of 65,178 patients (n=38,587, 59.2% female and n=21,454, 32.92% Black) were included in the final data set, of which 38,380 (58.88%) sent at least 1 message via the portal during the 7-year study period. As the number of chronic diagnoses increased, so did portal messaging use; however, this relationship was driven primarily by younger patients. Patients with 2 chronic conditions were 1.57 times more likely to send messages via the portal than those with 1 chronic condition (P<.001). In comparison, patients with ≥7 chronic conditions were approximately 11 times more likely to send messages than patients with 1 chronic condition (P<.001). A robustness check confirmed the interaction effect of age and the number of diagnoses on portal messaging. In the model including only patients aged <65 years, there was a significant effect of increased portal messaging corresponding to the number of chronic conditions (P<.001). Conversely, this relationship was not significant for the model consisting of older patients. Other significant factors associated with increased portal use include being female; White; married; having private insurance; and living in an area with a higher average level of educational attainment, greater medical provider density, and a lower median income. Conclusions: Patients’ use of the portal to send messages to providers was incrementally related to their number of diagnoses. As the number of chronic diagnoses increased, so did portal messaging use. Patients of all ages, particularly those living in rural areas, could benefit from the convenience and cost-effectiveness of portal communication. Health care systems and providers are encouraged to increase the use of patient portals by implementing educational interventions to promote the advantages of portal communication, particularly among patients with multimorbidity. %M 37526967 %R 10.2196/44399 %U https://www.jmir.org/2023/1/e44399 %U https://doi.org/10.2196/44399 %U http://www.ncbi.nlm.nih.gov/pubmed/37526967 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e46477 %T Structure of Health Information With Different Information Models: Evaluation Study With Competency Questions %A Rossander,Anna %A Karlsson,Daniel %+ Department of Applied IT, University of Gothenburg, Department of Applied Information Technology, Division of Informatics, Box 100, Gothenburg, 405 30, Sweden, 46 735989141, anna.rossander@gu.se %K informatics %K health care %K information model %K terminology %K terminologies %K interoperability %K competency question %K interoperable %K competency %K EHR %K electronic health record %K guideline %K standard %K recommendation %K information system %D 2023 %7 31.7.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: There is a flora of health care information models but no consensus on which to use. This leads to poor information sharing and duplicate modelling work. The amount and type of differences between models has, to our knowledge, not been evaluated. Objective: This work aims to explore how information structured with various information models differ in practice. Our hypothesis is that differences between information models are overestimated. This work will also assess the usability of competency questions as a method for evaluation of information models within health care. Methods: In this study, 4 information standards, 2 standards for secondary use, and 2 electronic health record systems were included as material. Competency questions were developed for a random selection of recommendations from a clinical guideline. The information needed to answer the competency questions was modelled according to each included information model, and the results were analyzed. Differences in structure and terminology were quantified for each combination of standards. Results: In this study, 36 competency questions were developed and answered. In general, similarities between the included information models were larger than the differences. The demarcation between information model and terminology was overall similar; on average, 45% of the included structures were identical between models. Choices of terminology differed within and between models; on average, 11% was usable in interaction with each other. The information models included in this study were able to represent most information required for answering the competency questions. Conclusions: Different but same same; in practice, different information models structure much information in a similar fashion. To increase interoperability within and between systems, it is more important to move toward structuring information with any information model rather than finding or developing a perfect information model. Competency questions are a feasible way of evaluating how information models perform in practice. %M 37523221 %R 10.2196/46477 %U https://medinform.jmir.org/2023/1/e46477 %U https://doi.org/10.2196/46477 %U http://www.ncbi.nlm.nih.gov/pubmed/37523221 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e44641 %T The Minimum Data Set for Rare Diseases: Systematic Review %A Bernardi,Filipe Andrade %A Mello de Oliveira,Bibiana %A Bettiol Yamada,Diego %A Artifon,Milena %A Schmidt,Amanda Maria %A Machado Scheibe,Victória %A Alves,Domingos %A Félix,Têmis Maria %+ Medical Genetics Service, Hospital de Clinicas de Porto Alegre, Rua Ramiro Barcelos, 2350 - Santa Cecilia, Porto Alegre, 90035-903, Brazil, 55 5133598011, tfelix@hcpa.edu.br %K health network %K health care system %K minimum data set %K public health %K rare disease %D 2023 %7 27.7.2023 %9 Review %J J Med Internet Res %G English %X Background: The minimum data set (MDS) is a collection of data elements to be grouped using a standard approach to allow the use of data for clinical and research purposes. Health data are typically voluminous, complex, and sometimes too ambiguous to generate indicators that can provide knowledge and information on health. This complexity extends further to the rare disease (RD) domain. MDSs are essential for health surveillance as they help provide services and generate recommended population indicators. There is a bottleneck in international literature that reveals a global problem with data collection, recording, and structuring in RD. Objective: This study aimed to identify and analyze the MDSs used for RD in health care networks worldwide and compare them with World Health Organization (WHO) guidelines. Methods: The population, concept, and context methodology proposed by the Joanna Briggs Institute was used to define the research question of this systematic review. A total of 4 databases were reviewed, and all the processes were reported using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology. The data elements were analyzed, extracted, and organized into 10 categories according to WHO digital health guidelines. The quality assessment used the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) checklist. Results: We included 20 studies in our review, 70% (n=14) of which focused on a specific health domain and 30% (n=6) of which referred to RD in general. WHO recommends that health systems and networks use standard terminology to exchange data, information, knowledge, and intelligence in health. However, there was a lack of terminological standardization of the concepts in MDSs. Moreover, the selected studies did not follow the same standard structure for classifying the data from their MDSs. All studies presented MDSs with limitations or restrictions because they covered only a specific RD, or their scope of application was restricted to a specific context or geographic region. Data science methods and clinical experience were used to design, structure, and recommend a fundamental global MDS for RD patient records in health care networks. Conclusions: Our study highlights the difficulties in standardizing and categorizing findings from MDSs for RD because of the varying structures used in different studies. The fundamental RD MDS designed in this study comprehensively covers the data needs in the clinical and management sectors. These results can help public policy makers support other aspects of their policies. We highlight the potential of our results to help strategic decisions related to RD. Trial Registration: PROSPERO CRD42021221593; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=221593 International Registered Report Identifier (IRRID): RR2-10.1016/j.procs.2021.12.034 %M 37498666 %R 10.2196/44641 %U https://www.jmir.org/2023/1/e44641 %U https://doi.org/10.2196/44641 %U http://www.ncbi.nlm.nih.gov/pubmed/37498666 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e45166 %T Operational Implementation of Remote Patient Monitoring Within a Large Ambulatory Health System: Multimethod Qualitative Case Study %A Lawrence,Katharine %A Singh,Nina %A Jonassen,Zoe %A Groom,Lisa L %A Alfaro Arias,Veronica %A Mandal,Soumik %A Schoenthaler,Antoinette %A Mann,Devin %A Nov,Oded %A Dove,Graham %+ Department of Population Health, New York University Grossman School of Medicine, 227 E 30th St, 6th Fl, New York, NY, 10016, United States, 1 646 301 3400, katharine.lawrence@nyulangone.org %K digital health %K remote patient monitoring %K RPM %K human-centered design %K human-computer interaction %K implementation science %D 2023 %7 27.7.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Remote patient monitoring (RPM) technologies can support patients living with chronic conditions through self-monitoring of physiological measures and enhance clinicians’ diagnostic and treatment decisions. However, to date, large-scale pragmatic RPM implementation within health systems has been limited, and understanding of the impacts of RPM technologies on clinical workflows and care experience is lacking. Objective: In this study, we evaluate the early implementation of operational RPM initiatives for chronic disease management within the ambulatory network of an academic medical center in New York City, focusing on the experiences of “early adopter” clinicians and patients. Methods: Using a multimethod qualitative approach, we conducted (1) interviews with 13 clinicians across 9 specialties considered as early adopters and supporters of RPM and (2) speculative design sessions exploring the future of RPM in clinical care with 21 patients and patient representatives, to better understand experiences, preferences, and expectations of pragmatic RPM use for health care delivery. Results: We identified themes relevant to RPM implementation within the following areas: (1) data collection and practices, including impacts of taking real-world measures and issues of data sharing, security, and privacy; (2) proactive and preventive care, including proactive and preventive monitoring, and proactive interventions and support; and (3) health disparities and equity, including tailored and flexible care and implicit bias. We also identified evidence for mitigation and support to address challenges in each of these areas. Conclusions: This study highlights the unique contexts, perceptions, and challenges regarding the deployment of RPM in clinical practice, including its potential implications for clinical workflows and work experiences. Based on these findings, we offer implementation and design recommendations for health systems interested in deploying RPM-enabled health care. %M 37498668 %R 10.2196/45166 %U https://humanfactors.jmir.org/2023/1/e45166 %U https://doi.org/10.2196/45166 %U http://www.ncbi.nlm.nih.gov/pubmed/37498668 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e46773 %T What’s Next for Modernizing Gender, Sex, and Sexual Orientation Terminology in Digital Health Systems? Viewpoint on Research and Implementation Priorities %A Queen,Roz %A Courtney,Karen L %A Lau,Francis %A Davison,Kelly %A Devor,Aaron %A Antonio,Marcy G %+ School of Health Information Science, University of Victoria, PO Box 1700 STN CSC, Victoria, BC, V8W 2Y2, Canada, 1 2507218575, rozomqueen@gmail.com %K data sharing %K digital health systems %K digital health %K gender, sex, and sexual orientation %K electronic health records %K GSSO %K Health Information Standards %K LGBT health %K LGBT %K policy %D 2023 %7 25.7.2023 %9 Viewpoint %J J Med Internet Res %G English %X In 2021, Canada Health Infoway and the University of Victoria's Gender, Sex, and Sexual Orientation Research Team hosted a series of discussions to successfully and safely modernize gender, sex, and sexual orientation information practices within digital health systems. Five main topic areas were covered: (1) terminology standards; (2) digital health and electronic health record functions; (3) policy and practice implications; (4) primary care settings; and (5) acute and tertiary care settings. In this viewpoint paper, we provide priorities for future research and implementation projects and recommendations that emerged from these discussions. %M 37490327 %R 10.2196/46773 %U https://www.jmir.org/2023/1/e46773 %U https://doi.org/10.2196/46773 %U http://www.ncbi.nlm.nih.gov/pubmed/37490327 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e48000 %T Mapping Factors That Affect the Uptake of Digital Therapeutics Within Health Systems: Scoping Review %A van Kessel,Robin %A Roman-Urrestarazu,Andres %A Anderson,Michael %A Kyriopoulos,Ilias %A Field,Samantha %A Monti,Giovanni %A Reed,Shelby D %A Pavlova,Milena %A Wharton,George %A Mossialos,Elias %+ LSE Health, Department of Health Policy, London School of Economics and Political Science, London, WC2A 2AE, United Kingdom, 44 7772 707841, e.a.mossialos@lse.ac.uk %K digital health %K uptake %K implementation %K adoption %K framework %K digital therapeutics %K scoping review %K thematic analysis %K digital medicine %K policy %D 2023 %7 25.7.2023 %9 Review %J J Med Internet Res %G English %X Background: Digital therapeutics are patient-facing digital health interventions that can significantly alter the health care landscape. Despite digital therapeutics being used to successfully treat a range of conditions, their uptake in health systems remains limited. Understanding the full spectrum of uptake factors is essential to identify ways in which policy makers and providers can facilitate the adoption of effective digital therapeutics within a health system, as well as the steps developers can take to assist in the deployment of products. Objective: In this review, we aimed to map the most frequently discussed factors that determine the integration of digital therapeutics into health systems and practical use of digital therapeutics by patients and professionals. Methods: A scoping review was conducted in MEDLINE, Web of Science, Cochrane Database of Systematic Reviews, and Google Scholar. Relevant data were extracted and synthesized using a thematic analysis. Results: We identified 35,541 academic and 221 gray literature reports, with 244 (0.69%) included in the review, covering 35 countries. Overall, 85 factors that can impact the uptake of digital therapeutics were extracted and pooled into 5 categories: policy and system, patient characteristics, properties of digital therapeutics, characteristics of health professionals, and outcomes. The need for a regulatory framework for digital therapeutics was the most stated factor at the policy level. Demographic characteristics formed the most iterated patient-related factor, whereas digital literacy was considered the most important factor for health professionals. Among the properties of digital therapeutics, their interoperability across the broader health system was most emphasized. Finally, the ability to expand access to health care was the most frequently stated outcome measure. Conclusions: The map of factors developed in this review offers a multistakeholder approach to recognizing the uptake factors of digital therapeutics in the health care pathway and provides an analytical tool for policy makers to assess their health system’s readiness for digital therapeutics. %M 37490322 %R 10.2196/48000 %U https://www.jmir.org/2023/1/e48000 %U https://doi.org/10.2196/48000 %U http://www.ncbi.nlm.nih.gov/pubmed/37490322 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e36121 %T Evaluation of 2 Artificial Intelligence Software for Chest X-Ray Screening and Pulmonary Tuberculosis Diagnosis: Protocol for a Retrospective Case-Control Study %A Mohd Hisham,Muhammad Faiz %A Lodz,Noor Aliza %A Muhammad,Eida Nurhadzira %A Asari,Filza Noor %A Mahmood,Mohd Ihsani %A Abu Bakar,Zamzurina %+ Institute for Public Health, National Institute of Health, Ministry of Health Malaysia, No 1, Jalan Setia Murni U13/52, Setia Alam, Shah Alam, 40170, Malaysia, 60 333628888 ext 8722, faizhisham86@gmail.com %K artificial intelligence %K AI %K evaluation %K pulmonary tuberculosis %K PTB %K chest x-ray %K CXR %K screening %D 2023 %7 25.7.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: According to the World Bank, Malaysia reported an estimated 97 tuberculosis cases per 100,000 people in 2021. Chest x-ray (CXR) remains the best conventional method for the early detection of pulmonary tuberculosis (PTB) infection. The intervention of artificial intelligence (AI) in PTB diagnosis could efficiently aid human interpreters and reduce health professionals’ work burden. To date, no AI studies have been evaluated in Malaysia. Objective: This study aims to evaluate the performance of Putralytica and Qure.ai software for CXR screening and PTB diagnosis among the Malaysian population. Methods: We will conduct a retrospective case-control study at the Respiratory Medicine Institute, National Cancer Institute, and Sungai Buloh Health Clinic. A total of 1500 CXR images of patients who completed treatments or check-ups will be selected and categorized into three groups: (1) abnormal PTB cases, (2) abnormal non-PTB cases, and (3) normal cases. These CXR images, along with their clinical findings, will be the reference standard in this study. All patient data, including sociodemographic characteristics and clinical history, will be collected prior to screening via Putralytica and Qure.ai software and readers’ interpretation, which are the index tests for this study. Interpretation from all 3 index tests will be compared with the reference standard, and significant statistical analysis will be computed. Results: Data collection is expected to commence in August 2023. It is anticipated that 1 year will be needed to conduct the study. Conclusions: This study will measure the accuracy of Putralytica and Qure.ai software and whether their findings will concur with readers’ interpretation and the reference standard, thus providing evidence toward the effectiveness of implementing AI in the medical setting. International Registered Report Identifier (IRRID): PRR1-10.2196/36121 %M 37490330 %R 10.2196/36121 %U https://www.researchprotocols.org/2023/1/e36121 %U https://doi.org/10.2196/36121 %U http://www.ncbi.nlm.nih.gov/pubmed/37490330 %0 Journal Article %@ 1929-073X %I JMIR Publications %V 12 %N %P e43384 %T Electronic Phenotype for Advanced Chronic Kidney Disease in a Veteran Health Care System Clinical Database: Systems-Based Strategy for Model Development and Evaluation %A Chamarthi,Gajapathiraju %A Orozco,Tatiana %A Shell,Popy %A Fu,Devin %A Hale-Gallardo,Jennifer %A Jia,Huanguang %A Shukla,Ashutosh M %+ Advanced Chronic Kidney Disease and Home Dialysis Program, North Florida/South Georgia Veteran Healthcare System, 1600 Archer Road, Gainesville, FL, 32610, United States, 1 352 273 8821, Ashutosh.Shukla@medicine.ufl.edu %K advanced chronic kidney disease %K EHR phenotype %K Veteran Health System %K CKD cohort %K kidney disease %K chronic %K clinical %K database %K data %K diagnosis %K risk %K disease %D 2023 %7 24.7.2023 %9 Original Paper %J Interact J Med Res %G English %X Background: Identifying advanced (stages 4 and 5) chronic kidney disease (CKD) cohorts in clinical databases is complicated and often unreliable. Accurately identifying these patients can allow targeting this population for their specialized clinical and research needs. Objective: This study was conducted as a system-based strategy to identify all prevalent Veterans with advanced CKD for subsequent enrollment in a clinical trial. We aimed to examine the prevalence and accuracy of conventionally used diagnosis codes and estimated glomerular filtration rate (eGFR)-based phenotypes for advanced CKD in an electronic health record (EHR) database. We sought to develop a pragmatic EHR phenotype capable of improving the real-time identification of advanced CKD cohorts in a regional Veterans health care system. Methods: Using the Veterans Affairs Informatics and Computing Infrastructure services, we extracted the source cohort of Veterans with advanced CKD based on a combination of the latest eGFR value ≤30 ml·min–1·1.73 m–2 or existing International Classification of Diseases (ICD)-10 diagnosis codes for advanced CKD (N18.4 and N18.5) in the last 12 months. We estimated the prevalence of advanced CKD using various prior published EHR phenotypes (ie, advanced CKD diagnosis codes, using the latest single eGFR <30 ml·min–1·1.73 m–2, utilizing two eGFR values) and our operational EHR phenotypes of a high-, intermediate-, and low-risk advanced CKD cohort. We evaluated the accuracy of these phenotypes by examining the likelihood of a sustained reduction of eGFR <30 ml·min–1·1.73 m–2 over a 6-month follow-up period. Results: Of the 133,756 active Veteran enrollees at North Florida/South Georgia Veterans Health System (NF/SG VHS), we identified a source cohort of 1759 Veterans with advanced nondialysis CKD. Among these, 1102 (62.9%) Veterans had diagnosis codes for advanced CKD; 1391(79.1%) had the index eGFR <30 ml·min–1·1.73 m–2; and 928 (52.7%), 480 (27.2%), and 315 (17.9%) Veterans had high-, intermediate-, and low-risk advanced CKD, respectively. The prevalence of advanced CKD among Veterans at NF/SG VHS varied between 1% and 1.5% depending on the EHR phenotype. At the 6-month follow-up, the probability of Veterans remaining in the advanced CKD stage was 65.3% in the group defined by the ICD-10 codes and 90% in the groups defined by eGFR values. Based on our phenotype, 94.2% of high-risk, 71% of intermediate-risk, and 16.1% of low-risk groups remained in the advanced CKD category. Conclusions: While the prevalence of advanced CKD has limited variation between different EHR phenotypes, the accuracy can be improved by utilizing two eGFR values in a stratified manner. We report the development of a pragmatic EHR-based model to identify advanced CKD within a regional Veterans health care system in real time with a tiered approach that allows targeting the needs of the groups at risk of progression to end-stage kidney disease. %M 37486757 %R 10.2196/43384 %U https://www.i-jmr.org/2023/1/e43384 %U https://doi.org/10.2196/43384 %U http://www.ncbi.nlm.nih.gov/pubmed/37486757 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e40639 %T A Neuro-Informatics Pipeline for Cerebrovascular Disease: Research Registry Development %A Potter,Thomas B H %A Pratap,Sharmila %A Nicolas,Juan Carlos %A Khan,Osman S %A Pan,Alan P %A Bako,Abdulaziz T %A Hsu,Enshuo %A Johnson,Carnayla %A Jefferson,Imory N %A Adegbindin,Sofiat K %A Baig,Eman %A Kelly,Hannah R %A Jones,Stephen L %A Britz,Gavin W %A Tannous,Jonika %A Vahidy,Farhaan S %+ Department of Neurosurgery, Houston Methodist, JRB Building, 4th Fl., 7550 Greenbriar Dr, Houston, TX, 77030, United States, 1 346 356 1479, fvahidy@houstonmethodist.org %K clinical outcome %K intracerebral hemorrhage %K acute ischemic stroke %K transient ischemic attack %K subarachnoid hemorrhage %K cerebral amyloid angiopathy %K learning health system %K electronic health record %K data curation %K database %D 2023 %7 21.7.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Although stroke is well recognized as a critical disease, treatment options are often limited. Inpatient stroke encounters carry critical information regarding the mechanisms of stroke and patient outcomes; however, these data are typically formatted to support administrative functions instead of research. To support improvements in the care of patients with stroke, a substantive research data platform is needed. Objective: To advance a stroke-oriented learning health care system, we sought to establish a comprehensive research repository of stroke data using the Houston Methodist electronic health record (EHR) system. Methods: Dedicated processes were developed to import EHR data of patients with primary acute ischemic stroke, intracerebral hemorrhage (ICH), transient ischemic attack, and subarachnoid hemorrhage under a review board–approved protocol. Relevant patients were identified from discharge diagnosis codes and assigned registry patient identification numbers. For identified patients, extract, transform, and load processes imported EHR data of primary cerebrovascular disease admissions and available data from any previous or subsequent admissions. Data were loaded into patient-focused SQL objects to enable cross-sectional and longitudinal analyses. Primary data domains (admission details, comorbidities, laboratory data, medications, imaging data, and discharge characteristics) were loaded into separate relational tables unified by patient and encounter identification numbers. Computed tomography, magnetic resonance, and angiography images were retrieved. Imaging data from patients with ICH were assessed for hemorrhage characteristics and cerebral small vessel disease markers. Patient information needed to interface with other local and national databases was retained. Prospective patient outreach was established, with patients contacted via telephone to assess functional outcomes 30, 90, 180, and 365 days after discharge. Dashboards were constructed to provide investigators with data summaries to support access. Results: The Registry of Neurological Endpoint Assessments among Patients with Ischemic and Hemorrhagic Stroke (REINAH) database was constructed as a series of relational category-specific SQL objects. Encounter summaries and dashboards were constructed to draw from these objects, providing visual data summaries for investigators seeking to build studies based on REINAH data. As of June 2022, the database contains 18,061 total patients, including 1809 (10.02%) with ICH, 13,444 (74.43%) with acute ischemic stroke, 1221 (6.76%) with subarachnoid hemorrhage, and 3165 (17.52%) with transient ischemic attack. Depending on the cohort, imaging data from computed tomography are available for 85.83% (1048/1221) to 98.4% (1780/1809) of patients, with magnetic resonance imaging available for 27.85% (340/1221) to 85.54% (11,500/13,444) of patients. Outcome assessment has successfully contacted 56.1% (240/428) of patients after ICH, with 71.3% (171/240) of responders providing consent for assessment. Responders reported a median modified Rankin Scale score of 3 at 90 days after discharge. Conclusions: A highly curated and clinically focused research platform for stroke data will establish a foundation for future research that may fundamentally improve poststroke patient care and outcomes. %M 37477961 %R 10.2196/40639 %U https://formative.jmir.org/2023/1/e40639 %U https://doi.org/10.2196/40639 %U http://www.ncbi.nlm.nih.gov/pubmed/37477961 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e45868 %T Perceived Impact of Digital Health Maturity on Patient Experience, Population Health, Health Care Costs, and Provider Experience: Mixed Methods Case Study %A Woods,Leanna %A Dendere,Ronald %A Eden,Rebekah %A Grantham,Brittany %A Krivit,Jenna %A Pearce,Andrew %A McNeil,Keith %A Green,Damian %A Sullivan,Clair %+ Centre for Health Services Research, Faculty of Medicine, The University of Queensland, L5 Health Sciences Building, Royal Brisbane and Women's Hospital, Herston, 4006, Australia, 61 731765530, lee.woods@uq.edu.au %K digital health %K health information systems %K digital maturity %K digital hospital %K evaluation study %K impact %K outcome assessment %K qualitative research %K health services research %D 2023 %7 18.7.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Health care organizations understand the importance of new technology implementations; however, the best strategy for implementing successful digital transformations is often unclear. Digital health maturity assessments allow providers to understand the progress made toward technology-enhanced health service delivery. Existing models have been criticized for their lack of depth and breadth because of their technology focus and neglect of meaningful outcomes. Objective: We aimed to examine the perceived impacts of digital health reported by health care staff employed in health care organizations across a spectrum of digital health maturity. Methods: A mixed methods case study was conducted. The digital health maturity of public health care systems (n=16) in Queensland, Australia, was examined using the quantitative Digital Health Indicator (DHI) self-assessment survey. The lower and upper quartiles of DHI scores were calculated and used to stratify sites into 3 groups. Using qualitative methods, health care staff (n=154) participated in interviews and focus groups. Transcripts were analyzed assisted by automated text-mining software. Impacts were grouped according to the digital maturity of the health care worker’s facility and mapped to the quadruple aims of health care: improved patient experience, improved population health, reduced health care cost, and enhanced provider experience. Results: DHI scores ranged between 78 and 193 for the 16 health care systems. Health care systems in the high-maturity category (n=4, 25%) had a DHI score of ≥166.75 (the upper quartile); low-maturity sites (n=4, 25%) had a DHI score of ≤116.75 (the lower quartile); and intermediate-maturity sites (n=8, 50%) had a DHI score ranging from 116.75 to 166.75 (IQR). Overall, 18 perceived impacts were identified. Generally, a greater number of positive impacts were reported in health care systems of higher digital health maturity. For patient experiences, higher maturity was associated with maintaining a patient health record and tracking patient experience data, while telehealth enabled access and flexibility across all digital health maturity categories. For population health, patient journey tracking and clinical risk mitigation were reported as positive impacts at higher-maturity sites, and telehealth enabled health care access and efficiencies across all maturity categories. Limited interoperability and organizational factors (eg, strategy, policy, and vision) were universally negative impacts affecting health service delivery. For health care costs, the resource burden of ongoing investments in digital health and a sustainable skilled workforce was reported. For provider experiences, the negative impacts of poor usability and change fatigue were universal, while network and infrastructure issues were negative impacts at low-maturity sites. Conclusions: This is one of the first studies to show differences in the perceived impacts of digital maturity of health care systems at scale. Higher digital health maturity was associated with more positive reported impacts, most notably in achieving outcomes for the population health aim. %M 37463008 %R 10.2196/45868 %U https://www.jmir.org/2023/1/e45868 %U https://doi.org/10.2196/45868 %U http://www.ncbi.nlm.nih.gov/pubmed/37463008 %0 Journal Article %@ 2563-3570 %I JMIR Publications %V 4 %N %P e44700 %T Secure Comparisons of Single Nucleotide Polymorphisms Using Secure Multiparty Computation: Method Development %A Woods,Andrew %A Kramer,Skyler T %A Xu,Dong %A Jiang,Wei %+ Department of Electrical Engineering and Computer Science, University of Missouri, 227 Naka Hall, Columbia, MO, 65211-0001, United States, 1 5738822299, xudong@missouri.edu %K secure multiparty computation %K single nucleotide polymorphism %K Variant Call Format %K Jaccard similarity %D 2023 %7 18.7.2023 %9 Original Paper %J JMIR Bioinform Biotech %G English %X Background: While genomic variations can provide valuable information for health care and ancestry, the privacy of individual genomic data must be protected. Thus, a secure environment is desirable for a human DNA database such that the total data are queryable but not directly accessible to involved parties (eg, data hosts and hospitals) and that the query results are learned only by the user or authorized party. Objective: In this study, we provide efficient and secure computations on panels of single nucleotide polymorphisms (SNPs) from genomic sequences as computed under the following set operations: union, intersection, set difference, and symmetric difference. Methods: Using these operations, we can compute similarity metrics, such as the Jaccard similarity, which could allow querying a DNA database to find the same person and genetic relatives securely. We analyzed various security paradigms and show metrics for the protocols under several security assumptions, such as semihonest, malicious with honest majority, and malicious with a malicious majority. Results: We show that our methods can be used practically on realistically sized data. Specifically, we can compute the Jaccard similarity of two genomes when considering sets of SNPs, each with 400,000 SNPs, in 2.16 seconds with the assumption of a malicious adversary in an honest majority and 0.36 seconds under a semihonest model. Conclusions: Our methods may help adopt trusted environments for hosting individual genomic data with end-to-end data security. %R 10.2196/44700 %U https://bioinform.jmir.org/2023/1/e44700 %U https://doi.org/10.2196/44700 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e45651 %T Effects of Using Different Indirect Techniques on the Calculation of Reference Intervals: Observational Study %A Yang,Dan %A Su,Zihan %A Mu,Runqing %A Diao,Yingying %A Zhang,Xin %A Liu,Yusi %A Wang,Shuo %A Wang,Xu %A Zhao,Lei %A Wang,Hongyi %A Zhao,Min %+ National Clinical Research Center for Laboratory Medicine, Department of Laboratory Medicine, The First Hospital of China Medical University, Nanjin North Street, No 155, Shenyang, 110001, China, 86 13898169877, minzhao@cmu.edu.cn %K comparative study %K data transformation %K indirect method %K outliers %K reference interval %K clinical decision-making %K complete blood count %K red blood cells %K white blood cells %K platelets %K laboratory %K clinical %D 2023 %7 17.7.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Reference intervals (RIs) play an important role in clinical decision-making. However, due to the time, labor, and financial costs involved in establishing RIs using direct means, the use of indirect methods, based on big data previously obtained from clinical laboratories, is getting increasing attention. Different indirect techniques combined with different data transformation methods and outlier removal might cause differences in the calculation of RIs. However, there are few systematic evaluations of this. Objective: This study used data derived from direct methods as reference standards and evaluated the accuracy of combinations of different data transformation, outlier removal, and indirect techniques in establishing complete blood count (CBC) RIs for large-scale data. Methods: The CBC data of populations aged ≥18 years undergoing physical examination from January 2010 to December 2011 were retrieved from the First Affiliated Hospital of China Medical University in northern China. After exclusion of repeated individuals, we performed parametric, nonparametric, Hoffmann, Bhattacharya, and truncation points and Kolmogorov–Smirnov distance (kosmic) indirect methods, combined with log or BoxCox transformation, and Reed–Dixon, Tukey, and iterative mean (3SD) outlier removal methods in order to derive the RIs of 8 CBC parameters and compared the results with those directly and previously established. Furthermore, bias ratios (BRs) were calculated to assess which combination of indirect technique, data transformation pattern, and outlier removal method is preferrable. Results: Raw data showed that the degrees of skewness of the white blood cell (WBC) count, platelet (PLT) count, mean corpuscular hemoglobin (MCH), mean corpuscular hemoglobin concentration (MCHC), and mean corpuscular volume (MCV) were much more obvious than those of other CBC parameters. After log or BoxCox transformation combined with Tukey or iterative mean (3SD) processing, the distribution types of these data were close to Gaussian distribution. Tukey-based outlier removal yielded the maximum number of outliers. The lower-limit bias of WBC (male), PLT (male), hemoglobin (HGB; male), MCH (male/female), and MCV (female) was greater than that of the corresponding upper limit for more than half of 30 indirect methods. Computational indirect choices of CBC parameters for males and females were inconsistent. The RIs of MCHC established by the direct method for females were narrow. For this, the kosmic method was markedly superior, which contrasted with the RI calculation of CBC parameters with high |BR| qualification rates for males. Among the top 10 methodologies for the WBC count, PLT count, HGB, MCV, and MCHC with a high-BR qualification rate among males, the Bhattacharya, Hoffmann, and parametric methods were superior to the other 2 indirect methods. Conclusions: Compared to results derived by the direct method, outlier removal methods and indirect techniques markedly influence the final RIs, whereas data transformation has negligible effects, except for obviously skewed data. Specifically, the outlier removal efficiency of Tukey and iterative mean (3SD) methods is almost equivalent. Furthermore, the choice of indirect techniques depends more on the characteristics of the studied analyte itself. This study provides scientific evidence for clinical laboratories to use their previous data sets to establish RIs. %M 37459170 %R 10.2196/45651 %U https://www.jmir.org/2023/1/e45651 %U https://doi.org/10.2196/45651 %U http://www.ncbi.nlm.nih.gov/pubmed/37459170 %0 Journal Article %@ 1929-073X %I JMIR Publications %V 12 %N %P e42016 %T The Utility of Predictive Modeling and a Systems Process Approach to Reduce Emergency Department Crowding: A Position Paper %A Monahan,Ann Corneille %A Feldman,Sue S %+ University of Alabama at Birmingham, 1720 University Blvd, Birmingham, AL, 35294, United States, 1 205 934 4011, monahanannc@gmail.com %K emergency care, prehospital %K information systems %K crowding %K healthcare service %K healthcare system %K emergency department %K boarding %K exit block %K medical informatics, application %K health services research %K personalized medicine %K predictive medicine %K model, probabilistic %K polynomial model %K decision support technique %K systems approach %K predict %K evidence based health care %K hospital bed management %K management information systems %K position paper %D 2023 %7 10.7.2023 %9 Viewpoint %J Interact J Med Res %G English %X Emergency department (ED) crowding and its main causes, exit block and boarding, continue to threaten the quality and safety of ED care. Most interventions to reduce crowding have not been comprehensive or system solutions, only focusing on part of the care procession and not directly affecting boarding reduction. This position paper proposes that the ED crowding problem can be optimally addressed by applying a systems approach using predictive modeling to identify patients at risk of being admitted to the hospital and uses that information to initiate the time-consuming bed management process earlier in the care continuum, shortening the time during which patients wait in the ED for an inpatient bed assignment, thus removing the exit block that causes boarding and subsequently reducing crowding. %M 37428536 %R 10.2196/42016 %U https://www.i-jmr.org/2023/1/e42016 %U https://doi.org/10.2196/42016 %U http://www.ncbi.nlm.nih.gov/pubmed/37428536 %0 Journal Article %@ 2561-1011 %I JMIR Publications %V 7 %N %P e44003 %T Electrocardiogram Devices for Home Use: Technological and Clinical Scoping Review %A Zepeda-Echavarria,Alejandra %A van de Leur,Rutger R %A van Sleuwen,Meike %A Hassink,Rutger J %A Wildbergh,Thierry X %A Doevendans,Pieter A %A Jaspers,Joris %A van Es,René %+ Department of Cardiology, Division of Heart and Lungs, University Medical Center Utrecht, Heidelberglaan 100, Utrecht, 3584 CX, Netherlands, 31 88 757 3453, R.vanEs@umcutrecht.nl %K electrocardiogram %K mobile ECG %K home use ECG %K wearables %K medical devices %K ECG clinical validation, ECG technical characteristics %D 2023 %7 7.7.2023 %9 Review %J JMIR Cardio %G English %X Background: Electrocardiograms (ECGs) are used by physicians to record, monitor, and diagnose the electrical activity of the heart. Recent technological advances have allowed ECG devices to move out of the clinic and into the home environment. There is a great variety of mobile ECG devices with the capabilities to be used in home environments. Objective: This scoping review aimed to provide a comprehensive overview of the current landscape of mobile ECG devices, including the technology used, intended clinical use, and available clinical evidence. Methods: We conducted a scoping review to identify studies concerning mobile ECG devices in the electronic database PubMed. Secondarily, an internet search was performed to identify other ECG devices available in the market. We summarized the devices’ technical information and usability characteristics based on manufacturer data such as datasheets and user manuals. For each device, we searched for clinical evidence on the capabilities to record heart disorders by performing individual searches in PubMed and ClinicalTrials.gov, as well as the Food and Drug Administration (FDA) 510(k) Premarket Notification and De Novo databases. Results: From the PubMed database and internet search, we identified 58 ECG devices with available manufacturer information. Technical characteristics such as shape, number of electrodes, and signal processing influence the capabilities of the devices to record cardiac disorders. Of the 58 devices, only 26 (45%) had clinical evidence available regarding their ability to detect heart disorders such as rhythm disorders, more specifically atrial fibrillation. Conclusions: ECG devices available in the market are mainly intended to be used for the detection of arrhythmias. No devices are intended to be used for the detection of other cardiac disorders. Technical and design characteristics influence the intended use of the devices and use environments. For mobile ECG devices to be intended to detect other cardiac disorders, challenges regarding signal processing and sensor characteristics should be solved to increase their detection capabilities. Devices recently released include the use of other sensors on ECG devices to increase their detection capabilities. %M 37418308 %R 10.2196/44003 %U https://cardio.jmir.org/2023/1/e44003 %U https://doi.org/10.2196/44003 %U http://www.ncbi.nlm.nih.gov/pubmed/37418308 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e46427 %T Deep Learning–Assisted Gait Parameter Assessment for Neurodegenerative Diseases: Model Development and Validation %A Jing,Yu %A Qin,Peinuan %A Fan,Xiangmin %A Qiang,Wei %A Wencheng,Zhu %A Sun,Wei %A Tian,Feng %A Wang,Dakuo %+ Institute of Software, Chinese Academy of Sciences, No. 4, South 4th Street, Haidian District, Beijing, 100190, China, 86 18810117223, sanqsunwei@gmail.com %K deep learning %K neurodegenerative disease %K auxiliary medical care %K gait parameter assessment %D 2023 %7 5.7.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Neurodegenerative diseases (NDDs) are prevalent among older adults worldwide. Early diagnosis of NDD is challenging yet crucial. Gait status has been identified as an indicator of early-stage NDD changes and can play a significant role in diagnosis, treatment, and rehabilitation. Historically, gait assessment has relied on intricate but imprecise scales by trained professionals or required patients to wear additional equipment, causing discomfort. Advancements in artificial intelligence may completely transform this and offer a novel approach to gait evaluation. Objective: This study aimed to use cutting-edge machine learning techniques to offer patients a noninvasive, entirely contactless gait assessment and provide health care professionals with precise gait assessment results covering all common gait-related parameters to assist in diagnosis and rehabilitation planning. Methods: Data collection involved motion data from 41 different participants aged 25 to 85 (mean 57.51, SD 12.93) years captured in motion sequences using the Azure Kinect (Microsoft Corp; a 3D camera with a 30-Hz sampling frequency). Support vector machine (SVM) and bidirectional long short-term memory (Bi-LSTM) classifiers trained using spatiotemporal features extracted from raw data were used to identify gait types in each walking frame. Gait semantics could then be obtained from the frame labels, and all the gait parameters could be calculated accordingly. For optimal generalization performance of the model, the classifiers were trained using a 10-fold cross-validation strategy. The proposed algorithm was also compared with the previous best heuristic method. Qualitative and quantitative feedback from medical staff and patients in actual medical scenarios was extensively collected for usability analysis. Results: The evaluations comprised 3 aspects. Regarding the classification results from the 2 classifiers, Bi-LSTM achieved an average precision, recall, and F1-score of 90.54%, 90.41%, and 90.38%, respectively, whereas these metrics were 86.99%, 86.62%, and 86.67%, respectively, for SVM. Moreover, the Bi-LSTM–based method attained 93.2% accuracy in gait segmentation evaluation (tolerance set to 2), whereas that of the SVM-based method achieved only 77.5% accuracy. For the final gait parameter calculation result, the average error rate of the heuristic method, SVM, and Bi-LSTM was 20.91% (SD 24.69%), 5.85% (SD 5.45%), and 3.17% (SD 2.75%), respectively. Conclusions: This study demonstrated that the Bi-LSTM–based approach can effectively support accurate gait parameter assessment, assisting medical professionals in making early diagnoses and reasonable rehabilitation plans for patients with NDD. %M 37405831 %R 10.2196/46427 %U https://www.jmir.org/2023/1/e46427 %U https://doi.org/10.2196/46427 %U http://www.ncbi.nlm.nih.gov/pubmed/37405831 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e47479 %T Reliability of Medical Information Provided by ChatGPT: Assessment Against Clinical Guidelines and Patient Information Quality Instrument %A Walker,Harriet Louise %A Ghani,Shahi %A Kuemmerli,Christoph %A Nebiker,Christian Andreas %A Müller,Beat Peter %A Raptis,Dimitri Aristotle %A Staubli,Sebastian Manuel %+ Royal Free London NHS Foundation Trust, Pond Street, London, NW3 2QG, United Kingdom, 44 20 7794 0500, s.staubli@nhs.net %K artificial intelligence %K internet information %K patient information %K ChatGPT %K EQIP tool %K chatbot %K chatbots %K conversational agent %K conversational agents %K internal medicine %K pancreas %K liver %K hepatic %K biliary %K gall %K bile %K gallstone %K pancreatitis %K pancreatic %K medical information %D 2023 %7 30.6.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: ChatGPT-4 is the latest release of a novel artificial intelligence (AI) chatbot able to answer freely formulated and complex questions. In the near future, ChatGPT could become the new standard for health care professionals and patients to access medical information. However, little is known about the quality of medical information provided by the AI. Objective: We aimed to assess the reliability of medical information provided by ChatGPT. Methods: Medical information provided by ChatGPT-4 on the 5 hepato-pancreatico-biliary (HPB) conditions with the highest global disease burden was measured with the Ensuring Quality Information for Patients (EQIP) tool. The EQIP tool is used to measure the quality of internet-available information and consists of 36 items that are divided into 3 subsections. In addition, 5 guideline recommendations per analyzed condition were rephrased as questions and input to ChatGPT, and agreement between the guidelines and the AI answer was measured by 2 authors independently. All queries were repeated 3 times to measure the internal consistency of ChatGPT. Results: Five conditions were identified (gallstone disease, pancreatitis, liver cirrhosis, pancreatic cancer, and hepatocellular carcinoma). The median EQIP score across all conditions was 16 (IQR 14.5-18) for the total of 36 items. Divided by subsection, median scores for content, identification, and structure data were 10 (IQR 9.5-12.5), 1 (IQR 1-1), and 4 (IQR 4-5), respectively. Agreement between guideline recommendations and answers provided by ChatGPT was 60% (15/25). Interrater agreement as measured by the Fleiss κ was 0.78 (P<.001), indicating substantial agreement. Internal consistency of the answers provided by ChatGPT was 100%. Conclusions: ChatGPT provides medical information of comparable quality to available static internet information. Although currently of limited quality, large language models could become the future standard for patients and health care professionals to gather medical information. %M 37389908 %R 10.2196/47479 %U https://www.jmir.org/2023/1/e47479 %U https://doi.org/10.2196/47479 %U http://www.ncbi.nlm.nih.gov/pubmed/37389908 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e44331 %T Optimizing Patient Record Linkage in a Master Patient Index Using Machine Learning: Algorithm Development and Validation %A Nelson,Walter %A Khanna,Nityan %A Ibrahim,Mohamed %A Fyfe,Justin %A Geiger,Maxwell %A Edwards,Keith %A Petch,Jeremy %+ Centre for Data Science and Digital Health, Hamilton Health Sciences, 175 Longwood Road South, Hamilton, ON, L8P 0A1, Canada, 1 9055212100, walterj.nelson@mail.utoronto.ca %K medical record linkage %K electronic health records %K medical record systems %K computerized %K machine learning %K quality of care %K health care system %K open-source software %K Bayesian optimization %K pilot %K data linkage %K master patient index %K master index %K record link %K matching algorithm %K FEBRL %D 2023 %7 29.6.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: To provide quality care, modern health care systems must match and link data about the same patient from multiple sources, a function often served by master patient index (MPI) software. Record linkage in the MPI is typically performed manually by health care providers, guided by automated matching algorithms. These matching algorithms must be configured in advance, such as by setting the weights of patient attributes, usually by someone with knowledge of both the matching algorithm and the patient population being served. Objective: We aimed to develop and evaluate a machine learning–based software tool, which automatically configures a patient matching algorithm by learning from pairs of patient records previously linked by humans already present in the database. Methods: We built a free and open-source software tool to optimize record linkage algorithm parameters based on historical record linkages. The tool uses Bayesian optimization to identify the set of configuration parameters that lead to optimal matching performance in a given patient population, by learning from prior record linkages by humans. The tool is written assuming only the existence of a minimal HTTP application programming interface (API), and so is agnostic to the choice of MPI software, record linkage algorithm, and patient population. As a proof of concept, we integrated our tool with SantéMPI, an open-source MPI. We validated the tool using several synthetic patient populations in SantéMPI by comparing the performance of the optimized configuration in held-out data to SantéMPI’s default matching configuration using sensitivity and specificity. Results: The machine learning–optimized configurations correctly detect over 90% of true record linkages as definite matches in all data sets, with 100% specificity and positive predictive value in all data sets, whereas the baseline detects none. In the largest data set examined, the baseline matching configuration detects possible record linkages with a sensitivity of 90.2% (95% CI 88.4%-92.0%) and specificity of 100%. By comparison, the machine learning–optimized matching configuration attains a sensitivity of 100%, with a decreased specificity of 95.9% (95% CI 95.9%-96.0%). We report significant gains in sensitivity in all data sets examined, at the cost of only marginally decreased specificity. The configuration optimization tool, data, and data set generator have been made freely available. Conclusions: Our machine learning software tool can be used to significantly improve the performance of existing record linkage algorithms, without knowledge of the algorithm being used or specific details of the patient population being served. %M 37384382 %R 10.2196/44331 %U https://formative.jmir.org/2023/1/e44331 %U https://doi.org/10.2196/44331 %U http://www.ncbi.nlm.nih.gov/pubmed/37384382 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 9 %N %P e42816 %T Evaluating the Discriminatory Ability of the Sickle Cell Data Collection Program’s Administrative Claims Case Definition in Identifying Adults With Sickle Cell Disease: Validation Study %A Singh,Ashima %A Sontag,Marci K %A Zhou,Mei %A Dasgupta,Mahua %A Crume,Tessa %A McLemore,Morgan %A Galadanci,Najibah %A Randall,Eldrida %A Steiner,Nicole %A Brandow,Amanda M %A Koch,Kathryn %A Field,Joshua J %A Hassell,Kathryn %A Snyder,Angela B %A Kanter,Julie %+ Department of Medicine, University of Alabama Birmingham, 1720 2nd street South, NP2510, Birmingham, AL, 35209, United States, 1 205 801 9034, jkanter@uabmc.edu %K surveillance using administrative data %K rare conditions %K sickle cell disease %K disease %K surveillance %K genetic %K prevention %K data %K adults %K epidemiology %K utilization %D 2023 %7 28.6.2023 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: Sickle cell disease (SCD) was first recognized in 1910 and identified as a genetic condition in 1949. However, there is not a universal clinical registry that can be used currently to estimate its prevalence. The Sickle Cell Data Collection (SCDC) program, funded by the Centers for Disease Control and Prevention, funds state-level grantees to compile data within their states from various sources including administrative claims to identify individuals with SCD. The performance of the SCDC administrative claims case definition has been validated in a pediatric population with SCD, but it has not been tested in adults. Objective: The objective of our study is to evaluate the discriminatory ability of the SCDC administrative claims case definition to accurately identify adults with SCD using Medicaid insurance claims data. Methods: Our study used Medicaid claims data in combination with hospital-based medical record data from the Alabama, Georgia, and Wisconsin SCDC programs to identify individuals aged 18 years or older meeting the SCDC administrative claims case definition. In order to validate this definition, our study included only those individuals who were identified in both Medicaid’s and the partnering clinical institution’s records. We used clinical laboratory tests and diagnostic algorithms to determine the true SCD status of this subset of patients. Positive predictive values (PPV) are reported overall and by state under several scenarios. Results: There were 1219 individuals (354 from Alabama and 865 from Georgia) who were identified through a 5-year time period. The 5-year time period yielded a PPV of 88.4% (91% for data from Alabama and 87% for data from Georgia), when only using data with laboratory-confirmed (gold standard) cases as true positives. With a narrower time period (3-year period) and data from 3 states (Alabama, Georgia, and Wisconsin), a total of 1432 individuals from these states were included in our study. The overall 3-year PPV was 89.4% (92%, 93%, and 81% for data from Alabama, Georgia, and Wisconsin, respectively) when only considering laboratory-confirmed cases as true cases. Conclusions: Adults identified as having SCD from administrative claims data based on the SCDC case definition have a high probability of truly having the disease, especially if those hospitals have active SCD programs. Administrative claims are thus a valuable data source to identify adults with SCD in a state and understand their epidemiology and health care service usage. %M 37379070 %R 10.2196/42816 %U https://publichealth.jmir.org/2023/1/e42816 %U https://doi.org/10.2196/42816 %U http://www.ncbi.nlm.nih.gov/pubmed/37379070 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e48072 %T Acquisition of a Lexicon for Family History Information: Bidirectional Encoder Representations From Transformers–Assisted Sublanguage Analysis %A Wang,Liwei %A He,Huan %A Wen,Andrew %A Moon,Sungrim %A Fu,Sunyang %A Peterson,Kevin J %A Ai,Xuguang %A Liu,Sijia %A Kavuluru,Ramakanth %A Liu,Hongfang %+ Department of Artificial Intelligence and Informatics, Mayo Clinic, 200 First Street SW, Rochester, MN, 55905, United States, 1 507 293 0057, liu.hongfang@mayo.edu %K electronic health record %K natural language processing %K family history %K sublanguage analysis %K rule-based system %K deep learning %D 2023 %7 27.6.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: A patient’s family history (FH) information significantly influences downstream clinical care. Despite this importance, there is no standardized method to capture FH information in electronic health records and a substantial portion of FH information is frequently embedded in clinical notes. This renders FH information difficult to use in downstream data analytics or clinical decision support applications. To address this issue, a natural language processing system capable of extracting and normalizing FH information can be used. Objective: In this study, we aimed to construct an FH lexical resource for information extraction and normalization. Methods: We exploited a transformer-based method to construct an FH lexical resource leveraging a corpus consisting of clinical notes generated as part of primary care. The usability of the lexicon was demonstrated through the development of a rule-based FH system that extracts FH entities and relations as specified in previous FH challenges. We also experimented with a deep learning–based FH system for FH information extraction. Previous FH challenge data sets were used for evaluation. Results: The resulting lexicon contains 33,603 lexicon entries normalized to 6408 concept unique identifiers of the Unified Medical Language System and 15,126 codes of the Systematized Nomenclature of Medicine Clinical Terms, with an average number of 5.4 variants per concept. The performance evaluation demonstrated that the rule-based FH system achieved reasonable performance. The combination of the rule-based FH system with a state-of-the-art deep learning–based FH system can improve the recall of FH information evaluated using the BioCreative/N2C2 FH challenge data set, with the F1 score varied but comparable. Conclusions: The resulting lexicon and rule-based FH system are freely available through the Open Health Natural Language Processing GitHub. %M 37368483 %R 10.2196/48072 %U https://medinform.jmir.org/2023/1/e48072 %U https://doi.org/10.2196/48072 %U http://www.ncbi.nlm.nih.gov/pubmed/37368483 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e46771 %T The Effects of Internet-Based Cognitive Behavioral Therapy for Suicidal Ideation or Behaviors on Depression, Anxiety, and Hopelessness in Individuals With Suicidal Ideation: Systematic Review and Meta-Analysis of Individual Participant Data %A Sander,Lasse B %A Beisemann,Marie %A Doebler,Philipp %A Micklitz,Hannah Moon %A Kerkhof,Ad %A Cuijpers,Pim %A Batterham,Philip %A Calear,Alison %A Christensen,Helen %A De Jaegere,Eva %A Domhardt,Matthias %A Erlangsen,Annette %A Eylem-van Bergeijk,Ozlem %A Hill,Ryan %A Mühlmann,Charlotte %A Österle,Marie %A Pettit,Jeremy %A Portzky,Gwendolyn %A Steubl,Lena %A van Spijker,Bregje %A Tighe,Joseph %A Werner-Seidler,Aliza %A Büscher,Rebekka %+ Medical Psychology and Medical Sociology, Faculty of Medicine, University of Freiburg, Hebelstraße 29, Freiburg, 79104, Germany, 49 761 203 5519, Lasse.Sander@mps.uni-freiburg.de %K meta-analysis %K internet-based cognitive behavioral therapy %K suicidal ideation %K anxiety %K depression %K hopelessness %K depressive %K mental health %K systematic review %K review method %K suicide %K suicidal %K psychotherapy %K CBT %K cognitive behavioral therapy %D 2023 %7 26.6.2023 %9 Review %J J Med Internet Res %G English %X Background: Suicide is a global public health problem. Digital interventions are considered a low-threshold treatment option for people with suicidal ideation or behaviors. Internet-based cognitive behavioral therapy (iCBT) targeting suicidal ideation has demonstrated effectiveness in reducing suicidal ideation. However, suicidal ideation often is related to additional mental health problems, which should be addressed for optimal care. Yet, the effects of iCBT on related symptoms, such as depression, anxiety, and hopelessness, remain unclear. Objective: We aimed to analyze whether digital interventions targeting suicidal ideation had an effect on related mental health symptoms (depression, anxiety, and hopelessness). Methods: We systematically searched CENTRAL, PsycInfo, Embase, and PubMed for randomized controlled trials that investigated guided or unguided iCBT for suicidal ideation or behaviors. Participants reporting baseline suicidal ideation were eligible. Individual participant data (IPD) were collected from eligible trials. We conducted a 1-stage IPD meta-analysis on the effects on depression, anxiety, and hopelessness—analyzed as 2 indices: symptom severity and treatment response. Results: We included IPD from 8 out of 9 eligible trials comprising 1980 participants with suicidal ideation. iCBT was associated with significant reductions in depression severity (b=−0.17; 95% CI −0.25 to −0.09; P<.001) and higher treatment response (ie, 50% reduction of depressive symptoms; b=0.36; 95% CI 0.12-0.60; P=.008) after treatment. We did not find significant effects on anxiety and hopelessness. Conclusions: iCBT for people with suicidal ideation revealed significant effects on depression outcomes but only minor or no effects on anxiety and hopelessness. Therefore, individuals with comorbid symptoms of anxiety or hopelessness may require additional treatment components to optimize care. Studies that monitor symptoms with higher temporal resolution and consider a broader spectrum of factors influencing suicidal ideation are needed to understand the complex interaction of suicidality and related mental health symptoms. %M 37358893 %R 10.2196/46771 %U https://www.jmir.org/2023/1/e46771 %U https://doi.org/10.2196/46771 %U http://www.ncbi.nlm.nih.gov/pubmed/37358893 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e45849 %T Development of a Corpus Annotated With Mentions of Pain in Mental Health Records: Natural Language Processing Approach %A Chaturvedi,Jaya %A Chance,Natalia %A Mirza,Luwaiza %A Vernugopan,Veshalee %A Velupillai,Sumithra %A Stewart,Robert %A Roberts,Angus %+ Department of Biostatistics and Health Informatics, King's College London, 16 De Crespigny Park, London, SE5 8AF, United Kingdom, 44 7380695133, jaya.1.chaturvedi@kcl.ac.uk %K pain %K mental health %K natural language processing %K annotation %K information extraction %D 2023 %7 26.6.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Pain is a widespread issue, with 20% of adults (1 in 5) experiencing it globally. A strong association has been demonstrated between pain and mental health conditions, and this association is known to exacerbate disability and impairment. Pain is also known to be strongly related to emotions, which can lead to damaging consequences. As pain is a common reason for people to access health care facilities, electronic health records (EHRs) are a potential source of information on this pain. Mental health EHRs could be particularly beneficial since they can show the overlap of pain with mental health. Most mental health EHRs contain the majority of their information within the free-text sections of the records. However, it is challenging to extract information from free text. Natural language processing (NLP) methods are therefore required to extract this information from the text. Objective: This research describes the development of a corpus of manually labeled mentions of pain and pain-related entities from the documents of a mental health EHR database, for use in the development and evaluation of future NLP methods. Methods: The EHR database used, Clinical Record Interactive Search, consists of anonymized patient records from The South London and Maudsley National Health Service Foundation Trust in the United Kingdom. The corpus was developed through a process of manual annotation where pain mentions were marked as relevant (ie, referring to physical pain afflicting the patient), negated (ie, indicating absence of pain), or not relevant (ie, referring to pain affecting someone other than the patient, or metaphorical and hypothetical mentions). Relevant mentions were also annotated with additional attributes such as anatomical location affected by pain, pain character, and pain management measures, if mentioned. Results: A total of 5644 annotations were collected from 1985 documents (723 patients). Over 70% (n=4028) of the mentions found within the documents were annotated as relevant, and about half of these mentions also included the anatomical location affected by the pain. The most common pain character was chronic pain, and the most commonly mentioned anatomical location was the chest. Most annotations (n=1857, 33%) were from patients who had a primary diagnosis of mood disorders (International Classification of Diseases—10th edition, chapter F30-39). Conclusions: This research has helped better understand how pain is mentioned within the context of mental health EHRs and provided insight into the kind of information that is typically mentioned around pain in such a data source. In future work, the extracted information will be used to develop and evaluate a machine learning–based NLP application to automatically extract relevant pain information from EHR databases. %M 37358897 %R 10.2196/45849 %U https://formative.jmir.org/2023/1/e45849 %U https://doi.org/10.2196/45849 %U http://www.ncbi.nlm.nih.gov/pubmed/37358897 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e45179 %T Conceptualizing Interprofessional Digital Communication and Collaboration in Health Care: Protocol for a Scoping Review %A Nordmann,Kim %A Sauter,Stefanie %A Möbius-Lerch,Patricia %A Redlich,Marie-Christin %A Schaller,Michael %A Fischer,Florian %+ Bavarian Research Center for Digital Health and Social Care, Albert-Einstein-Straße 6, Kempten, 87437, Germany, 49 831 870 235 0, kim.nordmann@hs-kempten.de %K digital technologies %K interdisciplinary communication %K intersectoral collaboration %K nurse %K physician %K collaboration %K interdisciplinary %K scoping review %K communication %K interprofessional %D 2023 %7 26.6.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Effective communication and collaboration among health professionals are essential prerequisites for patient-centered care. However, interprofessional teams require suitable structures and tools to efficiently use their professional competencies in the service of high-quality care appropriate to the patient’s life situation. In this context, digital tools potentially enhance interprofessional communication and collaboration and lead to an organizationally, socially, and ecologically sustainable health care system. However, there is a lack of studies systematically assessing the critical factors for successfully implementing tools for digitally supported interprofessional communication and collaboration in the health care setting. Furthermore, an operationalization of this concept is missing. Objective: The aim of the proposed scoping review is to (1) identify factors influencing the development, implementation, and adoption processes of digital tools for interprofessional communication in the health care sector and (2) analyze and synthesize the (implicit) definition, dimensions, and concepts of digitally supported communication and collaboration among health care professionals in the health care setting. Studies focusing on digital communication and collaboration practices among health care professionals, including medical doctors and qualified medical assistants, in any health care setting will be included in this review. Methods: To address these objectives, an in-depth analysis of heterogeneous studies is needed, which is best achieved through a scoping review. Within this proposed scoping review, which adheres to the Joanna Briggs Institute methodology, 5 databases (SCOPUS, CINAHL, PubMed, Embase, and PsycInfo) will be searched for studies assessing digital communication and collaboration among various health care professionals in different health care settings. Studies focusing on health care providers or patient interaction through digital tools and non–peer-reviewed studies will be excluded. Results: Key characteristics of the studies included will be summarized through descriptive analysis, using diagrams and tables. We will synthesize and map the data and conduct a qualitative in-depth thematic analysis of definitions and dimensions of interprofessional digital communication and collaboration among health care and nursing professionals. Conclusions: Results from this scoping review may help in establishing digitally supported collaborations between various stakeholders in the health care setting and successfully implementing new forms of interprofessional communication and collaboration. This could facilitate the transition to better coordinated care and encourage the development of digital frameworks. International Registered Report Identifier (IRRID): PRR1-10.2196/45179 %M 37358886 %R 10.2196/45179 %U https://www.researchprotocols.org/2023/1/e45179 %U https://doi.org/10.2196/45179 %U http://www.ncbi.nlm.nih.gov/pubmed/37358886 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e45614 %T Representation Learning and Spectral Clustering for the Development and External Validation of Dynamic Sepsis Phenotypes: Observational Cohort Study %A Boussina,Aaron %A Wardi,Gabriel %A Shashikumar,Supreeth Prajwal %A Malhotra,Atul %A Zheng,Kai %A Nemati,Shamim %+ Division of Biomedical Informatics, University of California, San Diego, 9500 Gilman Dr. MC 0990, La Jolla, CA, 92093, United States, 1 858 534 2230, aboussina@health.ucsd.edu %K sepsis %K phenotype %K emergency service, hospital %K disease progression %K artificial intelligence %K machine learning %K emergency %K infection %K clinical phenotype %K clinical phenotyping %K transition model %K transition modeling %D 2023 %7 23.6.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Recent attempts at clinical phenotyping for sepsis have shown promise in identifying groups of patients with distinct treatment responses. Nonetheless, the replicability and actionability of these phenotypes remain an issue because the patient trajectory is a function of both the patient’s physiological state and the interventions they receive. Objective: We aimed to develop a novel approach for deriving clinical phenotypes using unsupervised learning and transition modeling. Methods: Forty commonly used clinical variables from the electronic health record were used as inputs to a feed-forward neural network trained to predict the onset of sepsis. Using spectral clustering on the representations from this network, we derived and validated consistent phenotypes across a diverse cohort of patients with sepsis. We modeled phenotype dynamics as a Markov decision process with transitions as a function of the patient’s current state and the interventions they received. Results: Four consistent and distinct phenotypes were derived from over 11,500 adult patients who were admitted from the University of California, San Diego emergency department (ED) with sepsis between January 1, 2016, and January 31, 2020. Over 2000 adult patients admitted from the University of California, Irvine ED with sepsis between November 4, 2017, and August 4, 2022, were involved in the external validation. We demonstrate that sepsis phenotypes are not static and evolve in response to physiological factors and based on interventions. We show that roughly 45% of patients change phenotype membership within the first 6 hours of ED arrival. We observed consistent trends in patient dynamics as a function of interventions including early administration of antibiotics. Conclusions: We derived and describe 4 sepsis phenotypes present within 6 hours of triage in the ED. We observe that the administration of a 30 mL/kg fluid bolus may be associated with worse outcomes in certain phenotypes, whereas prompt antimicrobial therapy is associated with improved outcomes. %M 37351927 %R 10.2196/45614 %U https://www.jmir.org/2023/1/e45614 %U https://doi.org/10.2196/45614 %U http://www.ncbi.nlm.nih.gov/pubmed/37351927 %0 Journal Article %@ 2561-1011 %I JMIR Publications %V 7 %N %P e45611 %T Smartphone-Based Remote Monitoring in Heart Failure With Reduced Ejection Fraction: Retrospective Cohort Study of Secondary Care Use and Costs %A Zaman,Sameer %A Padayachee,Yorissa %A Shah,Moulesh %A Samways,Jack %A Auton,Alice %A Quaife,Nicholas M %A Sweeney,Mark %A Howard,James P %A Tenorio,Indira %A Bachtiger,Patrik %A Kamalati,Tahereh %A Pabari,Punam A %A Linton,Nick W F %A Mayet,Jamil %A Peters,Nicholas S %A Barton,Carys %A Cole,Graham D %A Plymen,Carla M %+ Imperial College London, Du Cane Road, London, W12 0HS, United Kingdom, 44 2033131000, graham.cole3@nhs.net %K heart failure %K remote monitoring %K smartphone care %K telemonitoring %K self-management %K admission prevention %K cohort study %K hospitalization %K noninvasive %K smartphone %K vital signs %K diagnosis %D 2023 %7 23.6.2023 %9 Original Paper %J JMIR Cardio %G English %X Background: Despite effective therapies, the economic burden of heart failure with reduced ejection fraction (HFrEF) is driven by frequent hospitalizations. Treatment optimization and admission avoidance rely on frequent symptom reviews and monitoring of vital signs. Remote monitoring (RM) aims to prevent admissions by facilitating early intervention, but the impact of noninvasive, smartphone-based RM of vital signs on secondary health care use and costs in the months after a new diagnosis of HFrEF is unknown. Objective: The purpose of this study is to conduct a secondary care health use and health-economic evaluation for patients with HFrEF using smartphone-based noninvasive RM and compare it with matched controls receiving usual care without RM. Methods: We conducted a retrospective study of 2 cohorts of newly diagnosed HFrEF patients, matched 1:1 for demographics, socioeconomic status, comorbidities, and HFrEF severity. They are (1) the RM group, with patients using the RM platform for >3 months and (2) the control group, with patients referred before RM was available who received usual heart failure care without RM. Emergency department (ED) attendance, hospital admissions, outpatient use, and the associated costs of this secondary care activity were extracted from the Discover data set for a 3-month period after diagnosis. Platform costs were added for the RM group. Secondary health care use and costs were analyzed using Kaplan-Meier event analysis and Cox proportional hazards modeling. Results: A total of 146 patients (mean age 63 years; 42/146, 29% female) were included (73 in each group). The groups were well-matched for all baseline characteristics except hypertension (P=.03). RM was associated with a lower hazard of ED attendance (hazard ratio [HR] 0.43; P=.02) and unplanned admissions (HR 0.26; P=.02). There were no differences in elective admissions (HR 1.03, P=.96) or outpatient use (HR 1.40; P=.18) between the 2 groups. These differences were sustained by a univariate model controlling for hypertension. Over a 3-month period, secondary health care costs were approximately 4-fold lower in the RM group than the control group, despite the additional cost of RM itself (mean cost per patient GBP £465, US $581 vs GBP £1850, US $2313, respectively; P=.04). Conclusions: This retrospective cohort study shows that smartphone-based RM of vital signs is feasible for HFrEF. This type of RM was associated with an approximately 2-fold reduction in ED attendance and a 4-fold reduction in emergency admissions over just 3 months after a new diagnosis with HFrEF. Costs were significantly lower in the RM group without increasing outpatient demand. This type of RM could be adjunctive to standard care to reduce admissions, enabling other resources to help patients unable to use RM. %M 37351921 %R 10.2196/45611 %U https://cardio.jmir.org/2023/1/e45611 %U https://doi.org/10.2196/45611 %U http://www.ncbi.nlm.nih.gov/pubmed/37351921 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e46842 %T Trauma-Informed Care in Digital Health Technologies: Protocol for a Scoping Review %A Abdulai,Abdul-Fatawu %A Naghdali,Hasti %A Tekie Ghirmay,Eden %A Adam,Fuseini %A Bawafaa,Eunice %+ School of Nursing, Faculty of Applied Sciences, University of British Columbia, T201-2211 Wesbrook Mall, Vancouver, BC, V6T 2B5, Canada, 1 6048227214, fatawu.abdulai@ubc.ca %K clinical intervention %K digital health technologies %K digital health %K psychological trauma %K stress %K trauma %K trauma-informed care %D 2023 %7 23.6.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: The use of digital health technologies is becoming increasingly common across the globe as they offer immense potential to enhance health care delivery by promoting accessibility, flexibility, and personalized care, connecting patients to health care professionals, and offering more efficient services and treatments to remote residents. At the same time, there is an increasing recognition of how digital health can inadvertently foment psychological trauma. This phenomenon has led to the adoption of trauma-informed care in designing and deploying digital health technologies. However, how trauma-informed care is defined and characterized, and the various trauma-informed care strategies used in designing and deploying digital health technologies remain unexplored. Objective: This scoping review aims to explore and synthesize the literature on how trauma-informed care is defined and characterized in digital health and the various trauma-informed care principles, strategies, or recommendations used in designing and deploying digital health. Methods: This review will draw on the Joanna Briggs Institute’s updated methodological guidance for scoping reviews. A search will be conducted on CINAHL, PubMed, Embase, Compendex Engineering Village, Web of Science, Scopus, and PsycINFO. This review will consider published research studies and unpublished work (gray literature). Studies will be included if they applied trauma-informed care in designing or deploying digital health for patients across all geographical locations or provide trauma-informed recommendations on how web developers should develop digital health. Studies will be limited to publications within the past 10 years and studies in all languages will be considered. Two independent reviewers will screen the titles and abstracts, and then perform a full-text review. Data will be extracted into a data extraction tool developed for this study. Results: The scoping review was undergoing a full search as of April 2023. The main results will synthesize the peer-reviewed and gray literature on adopting trauma-informed care practices in digital health research and development. The study is expected to be completed by December 2023 and the results are expected to be published in a peer-reviewed journal. Conclusions: This review is expected to provide the knowledge base on the adoption of trauma-informed care in designing and deploying digital health. This knowledge can lead to more engaging, and likely, more effective digital health interventions that have less potential for harm. A synthesis of the various trauma-informed care strategies in digital health will also provide a trauma-informed language by enabling researchers and digital health developers to consider trauma as a critical factor in each stage of the design process. International Registered Report Identifier (IRRID): DERR1-10.2196/46842 %M 37351935 %R 10.2196/46842 %U https://www.researchprotocols.org/2023/1/e46842 %U https://doi.org/10.2196/46842 %U http://www.ncbi.nlm.nih.gov/pubmed/37351935 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e43333 %T Digital Education for the Deployment of Artificial Intelligence in Health Care %A Malerbi,Fernando Korn %A Nakayama,Luis Filipe %A Gayle Dychiao,Robyn %A Zago Ribeiro,Lucas %A Villanueva,Cleva %A Celi,Leo Anthony %A Regatieri,Caio Vinicius %+ Laboratory for Computational Physiology, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA, 02139, United States, 1 617 253 7818, luisnaka@mit.edu %K artificial intelligence %K digital health %K health education %K machine learning %K digital education %K digital %K education %K transformation %K neural %K network %K evaluation %K dataset %K data %K set %K clinical %D 2023 %7 22.6.2023 %9 Viewpoint %J J Med Internet Res %G English %X Artificial Intelligence (AI) represents a significant milestone in health care's digital transformation. However, traditional health care education and training often lack digital competencies. To promote safe and effective AI implementation, health care professionals must acquire basic knowledge of machine learning and neural networks, critical evaluation of data sets, integration within clinical workflows, bias control, and human-machine interaction in clinical settings. Additionally, they should understand the legal and ethical aspects of digital health care and the impact of AI adoption. Misconceptions and fears about AI systems could jeopardize its real-life implementation. However, there are multiple barriers to promoting electronic health literacy, including time constraints, overburdened curricula, and the shortage of capacitated professionals. To overcome these challenges, partnerships among developers, professional societies, and academia are essential. Integrating specialists from different backgrounds, including data specialists, lawyers, and social scientists, can significantly contribute to combating digital illiteracy and promoting safe AI implementation in health care. %M 37347537 %R 10.2196/43333 %U https://www.jmir.org/2023/1/e43333 %U https://doi.org/10.2196/43333 %U http://www.ncbi.nlm.nih.gov/pubmed/37347537 %0 Journal Article %@ 2561-7605 %I JMIR Publications %V 6 %N %P e42274 %T Usability and Acceptability of Clinical Dashboards in Aged Care: Systematic Review %A Siette,Joyce %A Dodds,Laura %A Sharifi,Fariba %A Nguyen,Amy %A Baysari,Melissa %A Seaman,Karla %A Raban,Magdalena %A Wabe,Nasir %A Westbrook,Johanna %+ The MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Level 6, Westmead Innovation Quarter, 160 Hawkesbury Rd, Westmead, 2145, Australia, 61 2 9772 6648, joyce.siette@westernsydney.edu.au %K dashboard %K visualization %K usability %K acceptability %K user interface design %K health information technology %K aged care %K clinical %K database %K development %K aged care %D 2023 %7 19.6.2023 %9 Review %J JMIR Aging %G English %X Background: The use of clinical dashboards in aged care systems to support performance review and improve outcomes for older adults receiving care is increasing. Objective: Our aim was to explore evidence from studies of the acceptability and usability of clinical dashboards including their visual features and functionalities in aged care settings. Methods: A systematic review was conducted using 5 databases (MEDLINE, Embase, PsycINFO, Cochrane Library, and CINAHL) from inception to April 2022. Studies were included in the review if they were conducted in aged care environments (home-based community care, retirement villages, and long-term care) and reported a usability or acceptability evaluation of a clinical dashboard for use in aged care environments, including specific dashboard visual features (eg, a qualitative summary of individual user experience or metrics from a usability scale). Two researchers independently reviewed the articles and extracted the data. Data synthesis was performed via narrative review, and the risk of bias was measured using the Mixed Methods Appraisal Tool. Results: In total, 14 articles reporting on 12 dashboards were included. The quality of the articles varied. There was considerable heterogeneity in implementation setting (home care 8/14, 57%), dashboard user groups (health professionals 9/14, 64%), and sample size (range 3-292). Dashboard features included a visual representation of information (eg, medical condition prevalence), analytic capability (eg, predictive), and others (eg, stakeholder communication). Dashboard usability was mixed (4 dashboards rated as high), and dashboard acceptability was high for 9 dashboards. Most users considered dashboards to be informative, relevant, and functional, highlighting the use and intention of using this resource in the future. Dashboards that had the presence of one or more of these features (bar charts, radio buttons, checkboxes or other symbols, interactive displays, and reporting capabilities) were found to be highly acceptable. Conclusions: A comprehensive summary of clinical dashboards used in aged care is provided to inform future dashboard development, testing, and implementation. Further research is required to optimize visualization features, usability, and acceptability of dashboards in aged care. %M 37335599 %R 10.2196/42274 %U https://aging.jmir.org/2023/1/e42274 %U https://doi.org/10.2196/42274 %U http://www.ncbi.nlm.nih.gov/pubmed/37335599 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e45823 %T Comparing Decentralized Learning Methods for Health Data Models to Nondecentralized Alternatives: Protocol for a Systematic Review %A Diniz,José Miguel %A Vasconcelos,Henrique %A Souza,Júlio %A Rb-Silva,Rita %A Ameijeiras-Rodriguez,Carolina %A Freitas,Alberto %+ PhD Program in Health Data Science, Faculty of Medicine, University of Porto, Rua Dr Plácido da Costa, 4200-450, Porto, Portugal, 351 225 513 622, jmdiniz.med@gmail.com %K decentralized learning %K distributed learning %K federated learning %K centralized learning %K privacy %K health %K health data %K secondary data use %K health data model %K blockchain %K health care %K data science %D 2023 %7 19.6.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Considering the soaring health-related costs directed toward a growing, aging, and comorbid population, the health sector needs effective data-driven interventions while managing rising care costs. While health interventions using data mining have become more robust and adopted, they often demand high-quality big data. However, growing privacy concerns have hindered large-scale data sharing. In parallel, recently introduced legal instruments require complex implementations, especially when it comes to biomedical data. New privacy-preserving technologies, such as decentralized learning, make it possible to create health models without mobilizing data sets by using distributed computation principles. Several multinational partnerships, including a recent agreement between the United States and the European Union, are adopting these techniques for next-generation data science. While these approaches are promising, there is no clear and robust evidence synthesis of health care applications. Objective: The main aim is to compare the performance among health data models (eg, automated diagnosis and mortality prediction) developed using decentralized learning approaches (eg, federated and blockchain) to those using centralized or local methods. Secondary aims are comparing the privacy compromise and resource use among model architectures. Methods: We will conduct a systematic review using the first-ever registered research protocol for this topic following a robust search methodology, including several biomedical and computational databases. This work will compare health data models differing in development architecture, grouping them according to their clinical applications. For reporting purposes, a PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 flow diagram will be presented. CHARMS (Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies)–based forms will be used for data extraction and to assess the risk of bias, alongside PROBAST (Prediction Model Risk of Bias Assessment Tool). All effect measures in the original studies will be reported. Results: The queries and data extractions are expected to start on February 28, 2023, and end by July 31, 2023. The research protocol was registered with PROSPERO, under the number 393126, on February 3, 2023. With this protocol, we detail how we will conduct the systematic review. With that study, we aim to summarize the progress and findings from state-of-the-art decentralized learning models in health care in comparison to their local and centralized counterparts. Results are expected to clarify the consensuses and heterogeneities reported and help guide the research and development of new robust and sustainable applications to address the health data privacy problem, with applicability in real-world settings. Conclusions: We expect to clearly present the status quo of these privacy-preserving technologies in health care. With this robust synthesis of the currently available scientific evidence, the review will inform health technology assessment and evidence-based decisions, from health professionals, data scientists, and policy makers alike. Importantly, it should also guide the development and application of new tools in service of patients’ privacy and future research. Trial Registration: PROSPERO 393126; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=393126 International Registered Report Identifier (IRRID): PRR1-10.2196/45823 %M 37335606 %R 10.2196/45823 %U https://www.researchprotocols.org/2023/1/e45823 %U https://doi.org/10.2196/45823 %U http://www.ncbi.nlm.nih.gov/pubmed/37335606 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e46938 %T The Impact of Point-of-Care Testing for Influenza on Antimicrobial Stewardship (PIAMS) in UK Primary Care: Protocol for a Mixed Methods Study %A Hoang,Uy %A Williams,Alice %A Smylie,Jessica %A Aspden,Carole %A Button,Elizabeth %A Macartney,Jack %A Okusi,Cecilia %A Byford,Rachel %A Ferreira,Filipa %A Leston,Meredith %A Xie,Charis Xuan %A Joy,Mark %A Marsden,Gemma %A Clark,Tristan %A de Lusignan,Simon %+ Nuffield Department of Primary Care Health Sciences, University of Oxford, Eagle House, Walton Well Road, Oxford, OX2 6ED, United Kingdom, 44 01865289341, uy.hoang@phc.ox.ac.uk %K medical records systems, computerized %K influenza point-of-care systems %K general practice %K RSV %K implementation %K outcome assessment %K health care %K antimicrobial stewardship %K acute respiratory infection %K antimicrobial %K influenza %K primary care %K respiratory symptom %D 2023 %7 16.6.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Molecular point-of-care testing (POCT) used in primary care can inform whether a patient presenting with an acute respiratory infection has influenza. A confirmed clinical diagnosis, particularly early in the disease, could inform better antimicrobial stewardship. Social distancing and lockdowns during the COVID-19 pandemic have disturbed previous patterns of influenza infections in 2021. However, data from samples taken in the last quarter of 2022 suggest that influenza represents 36% of sentinel network positive virology, compared with 24% for respiratory syncytial virus. Problems with integration into the clinical workflow is a known barrier to incorporating technology into routine care. Objective: This study aims to report the impact of POCT for influenza on antimicrobial prescribing in primary care. We will additionally describe severe outcomes of infection (hospitalization and mortality) and how POCT is integrated into primary care workflows. Methods: The impact of POCT for influenza on antimicrobial stewardship (PIAMS) in UK primary care is an observational study being conducted between December 2022 and May 2023 and involving 10 practices that contribute data to the English sentinel network. Up to 1000 people who present to participating practices with respiratory symptoms will be swabbed and tested with a rapid molecular POCT analyzer in the practice. Antimicrobial prescribing and other study outcomes will be collected by linking information from the POCT analyzer with data from the patient’s computerized medical record. We will collect data on how POCT is incorporated into practice using data flow diagrams, unified modeling language use case diagrams, and Business Process Modeling Notation. Results: We will present the crude and adjusted odds of antimicrobial prescribing (all antibiotics and antivirals) given a POCT diagnosis of influenza, stratifying by whether individuals have a respiratory or other relevant diagnosis (eg, bronchiectasis). We will also present the rates of hospital referrals and deaths related to influenza infection in PIAMS study practices compared with a set of matched practices in the sentinel network and the rest of the network. We will describe any difference in implementation models in terms of staff involved and workflow. Conclusions: This study will generate data on the impact of POCT testing for influenza in primary care as well as help to inform about the feasibility of incorporating POCT into primary care workflows. It will inform the design of future larger studies about the effectiveness and cost-effectiveness of POCT to improve antimicrobial stewardship and any impact on severe outcomes. International Registered Report Identifier (IRRID): DERR1-10.2196/46938 %M 37327029 %R 10.2196/46938 %U https://www.researchprotocols.org/2023/1/e46938 %U https://doi.org/10.2196/46938 %U http://www.ncbi.nlm.nih.gov/pubmed/37327029 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e47672 %T A Practice-Proven Adaptive Case Management Approach for Innovative Health Care Services (Health Circuit): Cluster Randomized Clinical Pilot and Descriptive Observational Study %A Herranz,Carmen %A Martín-Moreno Banegas,Laura %A Dana Muzzio,Fernando %A Siso-Almirall,Antoni %A Roca,Josep %A Cano,Isaac %+ Physiopathological Mechanisms of Respiratory Illnesses Group, Institut d'Investigacions Biomèdiques August Pi i Sunyer, C/ del Rosselló, 149, Barcelona, 08036, Spain, 34 932275400, iscano@recerca.clinic.cat %K continuum of care management %K innovative healthcare services %K collaborative tools %K digital health transformation %K usability %K acceptability %K health care service %K Health Circuit %K health management %K management %K support %K digital aid %K aid %K care %K prototype %K surgery %K testing %D 2023 %7 14.6.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Digital health tools may facilitate the continuity of care. Enhancement of digital aid is imperative to prevent information gaps or redundancies, as well as to facilitate support of flexible care plans. Objective: The study presents Health Circuit, an adaptive case management approach that empowers health care professionals and patients to implement personalized evidence-based interventions, thanks to dynamic communication channels and patient-centered service workflows; analyze the health care impact; and determine its usability and acceptability among health care professionals and patients. Methods: From September 2019 to March 2020, the health impact, usability (measured with the system usability scale; SUS), and acceptability (measured with the net promoter score; NPS) of an initial prototype of Health Circuit were tested in a cluster randomized clinical pilot (n=100) in patients with high risk for hospitalization (study 1). From July 2020 to July 2021, a premarket pilot study of usability (with the SUS) and acceptability (with the NPS) was conducted among 104 high-risk patients undergoing prehabilitation before major surgery (study 2). Results: In study 1, Health Circuit resulted in a reduction of emergency room visits (4/7, 13% vs 7/16, 44%), enhanced patients’ empowerment (P<.001) and showed good acceptability and usability scores (NPS: 31; SUS: 54/100). In study 2, the NPS was 40 and the SUS was 85/100. The acceptance rate was also high (mean score of 8.4/10). Conclusions: Health Circuit showed potential for health care value generation and good acceptability and usability despite being a prototype system, prompting the need for testing a completed system in real-world scenarios. Trial Registration: ClinicalTrials.gov NCT04056663; https://clinicaltrials.gov/ct2/show/NCT04056663 %M 37314850 %R 10.2196/47672 %U https://www.jmir.org/2023/1/e47672 %U https://doi.org/10.2196/47672 %U http://www.ncbi.nlm.nih.gov/pubmed/37314850 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e43896 %T Expectations of Anesthesiology and Intensive Care Professionals Toward Artificial Intelligence: Observational Study %A Kloka,Jan Andreas %A Holtmann,Sophie C %A Nürenberg-Goloub,Elina %A Piekarski,Florian %A Zacharowski,Kai %A Friedrichson,Benjamin %+ Department of Anaesthesiology, Intensive Care Medicine and Pain Therapy, University Hospital Frankfurt, Goethe University, Theodor-Stern Kai 7, Frankfurt, 60590, Germany, 49 630183876, JanAndreas.Kloka@kgu.de %K anesthesiology %K artificial intelligence %K health care %K intensive care %K medical informatics %K technology acceptance %K Europe-wide survey %D 2023 %7 12.6.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Artificial intelligence (AI) applications offer numerous opportunities to improve health care. To be used in the intensive care unit, AI must meet the needs of staff, and potential barriers must be addressed through joint action by all stakeholders. It is thus critical to assess the needs and concerns of anesthesiologists and intensive care physicians related to AI in health care throughout Europe. Objective: This Europe-wide, cross-sectional observational study investigates how potential users of AI systems in anesthesiology and intensive care assess the opportunities and risks of the new technology. The web-based questionnaire was based on the established analytic model of acceptance of innovations by Rogers to record 5 stages of innovation acceptance. Methods: The questionnaire was sent twice in 2 months (March 11, 2021, and November 5, 2021) through the European Society of Anaesthesiology and Intensive Care (ESAIC) member email distribution list. A total of 9294 ESAIC members were reached, of whom 728 filled out the questionnaire (response rate 728/9294, 8%). Due to missing data, 27 questionnaires were excluded. The analyses were conducted with 701 participants. Results: A total of 701 questionnaires (female: n=299, 42%) were analyzed. Overall, 265 (37.8%) of the participants have been in contact with AI and evaluated the benefits of this technology higher (mean 3.22, SD 0.39) than participants who stated no previous contact (mean 3.01, SD 0.48). Physicians see the most benefits of AI application in early warning systems (335/701, 48% strongly agreed, and 358/701, 51% agreed). Major potential disadvantages were technical problems (236/701, 34% strongly agreed, and 410/701, 58% agreed) and handling difficulties (126/701, 18% strongly agreed, and 462/701, 66% agreed), both of which could be addressed by Europe-wide digitalization and education. In addition, the lack of a secure legal basis for the research and use of medical AI in the European Union leads doctors to expect problems with legal liability (186/701, 27% strongly agreed, and 374/701, 53% agreed) and data protection (148/701, 21% strongly agreed, and 343/701, 49% agreed). Conclusions: Anesthesiologists and intensive care personnel are open to AI applications in their professional field and expect numerous benefits for staff and patients. Regional differences in the digitalization of the private sector are not reflected in the acceptance of AI among health care professionals. Physicians anticipate technical difficulties and lack a stable legal basis for the use of AI. Training for medical staff could increase the benefits of AI in professional medicine. Therefore, we suggest that the development and implementation of AI in health care require a solid technical, legal, and ethical basis, as well as adequate education and training of users. %M 37307038 %R 10.2196/43896 %U https://formative.jmir.org/2023/1/e43896 %U https://doi.org/10.2196/43896 %U http://www.ncbi.nlm.nih.gov/pubmed/37307038 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e45342 %T New Web-Based System for Recording Public Health Nursing Practices and Determining Best Practices: Protocol of an Exploratory Sequential Design %A Yoshioka-Maeda,Kyoko %A Matsumoto,Hiroshige %A Honda,Chikako %A Shiomi,Misa %A Taira,Kazuya %A Hosoya,Noriko %A Sato,Miki %A Sumikawa,Yuka %A Fujii,Hitoshi %A Miura,Takahiro %+ Department of Community Health Nursing, Division of Health Sciences & Nursing, Graduate School of Medicine, The University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, 113-0033, Japan, 81 358413597, kyokoy-tky@g.ecc.u-tokyo.ac.jp %K community-based activity %K evidence-based practice %K individual care %K information and communication technology %K program development %K public health nursing %K quality assurance %K digitalization %K eHealth %K electronic record %D 2023 %7 12.6.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Digitalization and information and communication technology (ICT) promote effective, efficient individual and community care. Clinical terminology or taxonomy and its framework visualize individual patients’ and nursing interventions’ classifications to improve their outcomes and care quality. Public health nurses (PHNs) provide lifelong individual care and community-based activities while developing projects to promote community health. The linkage between these practices and clinical assessment remains tacit. Owing to Japan’s lagging digitalization, supervisory PHNs face difficulties in monitoring each department’s activities and staff members’ performances and competencies. Randomly selected prefectural or municipal PHNs collect data on daily activities and required hours every 3 years. No study has adopted these data for public health nursing care management. PHNs need ICTs to manage their work and improve care quality; it may help identify health needs and suggest best public health nursing practices. Objective: We aim to develop and validate an electronic recording and management system for evaluating different public health nursing practice needs, including individual care, community-based activities, and project development, and for determining their best practices. Methods: We used a 2-phase exploratory sequential design (in Japan) comprising 2 phases. In phase 1, we developed the system’s architectural framework and a hypothetical algorithm to determine the need for practice review through a literature review and a panel discussion. We designed a cloud-based practice recording system, including a daily record system and a termly review system. The panels included 3 supervisors who were prior PHNs at the prefectural or municipal government, and 1 was the executive director of the Japanese Nursing Association. The panels agreed that the draft architectural framework and hypothetical algorithm were reasonable. The system was not linked to electronic nursing records to protect patient privacy. Phase 2 validated each item through interviews with supervisory PHNs using a web-based meeting system. A nationwide survey was distributed to supervisory and midcareer PHNs across local governments. Results: This study was funded in March 2022 and approved by all ethics review boards from July to September and November 2022. Data collection was completed in January 2023. Five PHNs participated in the interviews. In the nationwide survey, responses were obtained from 177 local governments of supervisory PHNs and 196 midcareer ones. Conclusions: This study will reveal PHNs’ tacit knowledge about their practices, assess needs for different approaches, and determine best practices. Additionally, this study will promote ICT-based practices in public health nursing. The system will enable PHNs to record their daily activities and share them with their supervisors to reflect on and improve their performance, and the quality of care to promote health equity in community settings. The system will support supervisory PHNs in creating performance benchmarks for their staff and departments to promote evidence-based human resource development and management. Trial Registration: UMIN-ICDR UMIN000049411; https://tinyurl.com/yfvxscfm International Registered Report Identifier (IRRID): DERR1-10.2196/45342 %M 37307040 %R 10.2196/45342 %U https://www.researchprotocols.org/2023/1/e45342 %U https://doi.org/10.2196/45342 %U http://www.ncbi.nlm.nih.gov/pubmed/37307040 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e44081 %T Issue of Data Imbalance on Low Birthweight Baby Outcomes Prediction and Associated Risk Factors Identification: Establishment of Benchmarking Key Machine Learning Models With Data Rebalancing Strategies %A Ren,Yang %A Wu,Dezhi %A Tong,Yan %A López-DeFede,Ana %A Gareau,Sarah %+ Department of Integrated Information Technology, University of South Carolina, 550 Assembly Street, Columbia, SC, 29298, United States, 1 8033774691, dezhiwu@cec.sc.edu %K low birthweight %K machine learning %K risk factor %K benchmark %K data rebalance %D 2023 %7 31.5.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Low birthweight (LBW) is a leading cause of neonatal mortality in the United States and a major causative factor of adverse health effects in newborns. Identifying high-risk patients early in prenatal care is crucial to preventing adverse outcomes. Previous studies have proposed various machine learning (ML) models for LBW prediction task, but they were limited by small and imbalanced data sets. Some authors attempted to address this through different data rebalancing methods. However, most of their reported performances did not reflect the models’ actual performance in real-life scenarios. To date, few studies have successfully benchmarked the performance of ML models in maternal health; thus, it is critical to establish benchmarks to advance ML use to subsequently improve birth outcomes. Objective: This study aimed to establish several key benchmarking ML models to predict LBW and systematically apply different rebalancing optimization methods to a large-scale and extremely imbalanced all-payer hospital record data set that connects mother and baby data at a state level in the United States. We also performed feature importance analysis to identify the most contributing features in the LBW classification task, which can aid in targeted intervention. Methods: Our large data set consisted of 266,687 birth records across 6 years, and 8.63% (n=23,019) of records were labeled as LBW. To set up benchmarking ML models to predict LBW, we applied 7 classic ML models (ie, logistic regression, naive Bayes, random forest, extreme gradient boosting, adaptive boosting, multilayer perceptron, and sequential artificial neural network) while using 4 different data rebalancing methods: random undersampling, random oversampling, synthetic minority oversampling technique, and weight rebalancing. Owing to ethical considerations, in addition to ML evaluation metrics, we primarily used recall to evaluate model performance, indicating the number of correctly predicted LBW cases out of all actual LBW cases, as false negative health care outcomes could be fatal. We further analyzed feature importance to explore the degree to which each feature contributed to ML model prediction among our best-performing models. Results: We found that extreme gradient boosting achieved the highest recall score—0.70—using the weight rebalancing method. Our results showed that various data rebalancing methods improved the prediction performance of the LBW group substantially. From the feature importance analysis, maternal race, age, payment source, sum of predelivery emergency department and inpatient hospitalizations, predelivery disease profile, and different social vulnerability index components were important risk factors associated with LBW. Conclusions: Our findings establish useful ML benchmarks to improve birth outcomes in the maternal health domain. They are informative to identify the minority class (ie, LBW) based on an extremely imbalanced data set, which may guide the development of personalized LBW early prevention, clinical interventions, and statewide maternal and infant health policy changes. %M 37256674 %R 10.2196/44081 %U https://www.jmir.org/2023/1/e44081 %U https://doi.org/10.2196/44081 %U http://www.ncbi.nlm.nih.gov/pubmed/37256674 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e44567 %T Migrating a Well-Established Longitudinal Cohort Database From Oracle SQL to Research Electronic Data Entry (REDCap): Data Management Research and Design Study %A Kusejko,Katharina %A Smith,Daniel %A Scherrer,Alexandra %A Paioni,Paolo %A Kohns Vasconcelos,Malte %A Aebi-Popp,Karoline %A Kouyos,Roger D %A Günthard,Huldrych F %A Kahlert,Christian R %A , %+ Institute of Medical Virology, University of Zurich, Universitaetsstrasse 84, Zurich, 8006, Switzerland, 41 44 634 1913, katharina.kusejko@usz.ch %K REDCap %K cohort study %K data collection %K electronic case report forms %K eCRF %K software %K digital solution %K electronic data entry %K HIV %D 2023 %7 31.5.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Providing user-friendly electronic data collection tools for large multicenter studies is key for obtaining high-quality research data. Research Electronic Data Capture (REDCap) is a software solution developed for setting up research databases with integrated graphical user interfaces for electronic data entry. The Swiss Mother and Child HIV Cohort Study (MoCHiV) is a longitudinal cohort study with around 2 million data entries dating back to the early 1980s. Until 2022, data collection in MoCHiV was paper-based. Objective: The objective of this study was to provide a user-friendly graphical interface for electronic data entry for physicians and study nurses reporting MoCHiV data. Methods: MoCHiV collects information on obstetric events among women living with HIV and children born to mothers living with HIV. Until 2022, MoCHiV data were stored in an Oracle SQL relational database. In this project, R and REDCap were used to develop an electronic data entry platform for MoCHiV with migration of already collected data. Results: The key steps for providing an electronic data entry option for MoCHiV were (1) design, (2) data cleaning and formatting, (3) migration and compliance, and (4) add-on features. In the first step, the database structure was defined in REDCap, including the specification of primary and foreign keys, definition of study variables, and the hierarchy of questions (termed “branching logic”). In the second step, data stored in Oracle were cleaned and formatted to adhere to the defined database structure. Systematic data checks ensured compliance to all branching logic and levels of categorical variables. REDCap-specific variables and numbering of repeated events for enabling a relational data structure in REDCap were generated using R. In the third step, data were imported to REDCap and then systematically compared to the original data. In the last step, add-on features, such as data access groups, redirections, and summary reports, were integrated to facilitate data entry in the multicenter MoCHiV study. Conclusions: By combining different software tools—Oracle SQL, R, and REDCap—and building a systematic pipeline for data cleaning, formatting, and comparing, we were able to migrate a multicenter longitudinal cohort study from Oracle SQL to REDCap. REDCap offers a flexible way for developing customized study designs, even in the case of longitudinal studies with different study arms (ie, obstetric events, women, and mother-child pairs). However, REDCap does not offer built-in tools for preprocessing large data sets before data import. Additional software is needed (eg, R) for data formatting and cleaning to achieve the predefined REDCap data structure. %M 37256686 %R 10.2196/44567 %U https://formative.jmir.org/2023/1/e44567 %U https://doi.org/10.2196/44567 %U http://www.ncbi.nlm.nih.gov/pubmed/37256686 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e41725 %T Machine Learning and Causal Approaches to Predict Readmissions and Its Economic Consequences Among Canadian Patients With Heart Disease: Retrospective Study %A Rajkumar,Ethan %A Nguyen,Kevin %A Radic,Sandra %A Paa,Jubelle %A Geng,Qiyang %+ Department of Chemistry, Faculty of Science, The University of British Columbia, 2036 Main Mall, Vancouver, BC, Canada, 1 (604) 822 3266, er12da@student.ubc.ca %K patient readmission %K health care economics %K ensemble %K prediction model %K classification %K linear regression resource intensity value %K hospital %K health care %K principal component analysis %K PCA %D 2023 %7 26.5.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Unplanned patient readmissions within 30 days of discharge pose a substantial challenge in Canadian health care economics. To address this issue, risk stratification, machine learning, and linear regression paradigms have been proposed as potential predictive solutions. Ensemble machine learning methods, such as stacked ensemble models with boosted tree algorithms, have shown promise for early risk identification in specific patient groups. Objective: This study aims to implement an ensemble model with submodels for structured data, compare metrics, evaluate the impact of optimized data manipulation with principal component analysis on shorter readmissions, and quantitatively verify the causal relationship between expected length of stay (ELOS) and resource intensity weight (RIW) value for a comprehensive economic perspective. Methods: This retrospective study used Python 3.9 and streamlined libraries to analyze data obtained from the Discharge Abstract Database covering 2016 to 2021. The study used 2 sub–data sets, clinical and geographical data sets, to predict patient readmission and analyze its economic implications, respectively. A stacking classifier ensemble model was used after principal component analysis to predict patient readmission. Linear regression was performed to determine the relationship between RIW and ELOS. Results: The ensemble model achieved precision and slightly higher recall (0.49 and 0.68), indicating a higher instance of false positives. The model was able to predict cases better than other models in the literature. Per the ensemble model, readmitted women and men aged 40 to 44 and 35 to 39 years, respectively, were more likely to use resources. The regression tables verified the causality of the model and confirmed the trend that patient readmission is much more costly than continued hospital stay without discharge for both the patient and health care system. Conclusions: This study validates the use of hybrid ensemble models for predicting economic cost models in health care with the goal of reducing the bureaucratic and utility costs associated with hospital readmissions. The availability of robust and efficient predictive models, as demonstrated in this study, can help hospitals focus more on patient care while maintaining low economic costs. This study predicts the relationship between ELOS and RIW, which can indirectly impact patient outcomes by reducing administrative tasks and physicians’ burden, thereby reducing the cost burdens placed on patients. It is recommended that changes to the general ensemble model and linear regressions be made to analyze new numerical data for predicting hospital costs. Ultimately, the proposed work hopes to emphasize the advantages of implementing hybrid ensemble models in forecasting health care economic cost models, empowering hospitals to prioritize patient care while simultaneously decreasing administrative and bureaucratic expenses. %M 37234042 %R 10.2196/41725 %U https://formative.jmir.org/2023/1/e41725 %U https://doi.org/10.2196/41725 %U http://www.ncbi.nlm.nih.gov/pubmed/37234042 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e45662 %T Generate Analysis-Ready Data for Real-world Evidence: Tutorial for Harnessing Electronic Health Records With Advanced Informatic Technologies %A Hou,Jue %A Zhao,Rachel %A Gronsbell,Jessica %A Lin,Yucong %A Bonzel,Clara-Lea %A Zeng,Qingyi %A Zhang,Sinian %A Beaulieu-Jones,Brett K %A Weber,Griffin M %A Jemielita,Thomas %A Wan,Shuyan Sabrina %A Hong,Chuan %A Cai,Tianrun %A Wen,Jun %A Ayakulangara Panickan,Vidul %A Liaw,Kai-Li %A Liao,Katherine %A Cai,Tianxi %+ Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Room 434, Boston, MA, 02115, United States, 1 617 432 4923, tcai@hsph.harvard.edu %K electronic health records %K real-world evidence %K data curation %K medical informatics %K randomized controlled trials %K reproducibility %D 2023 %7 25.5.2023 %9 Tutorial %J J Med Internet Res %G English %X Although randomized controlled trials (RCTs) are the gold standard for establishing the efficacy and safety of a medical treatment, real-world evidence (RWE) generated from real-world data has been vital in postapproval monitoring and is being promoted for the regulatory process of experimental therapies. An emerging source of real-world data is electronic health records (EHRs), which contain detailed information on patient care in both structured (eg, diagnosis codes) and unstructured (eg, clinical notes and images) forms. Despite the granularity of the data available in EHRs, the critical variables required to reliably assess the relationship between a treatment and clinical outcome are challenging to extract. To address this fundamental challenge and accelerate the reliable use of EHRs for RWE, we introduce an integrated data curation and modeling pipeline consisting of 4 modules that leverage recent advances in natural language processing, computational phenotyping, and causal modeling techniques with noisy data. Module 1 consists of techniques for data harmonization. We use natural language processing to recognize clinical variables from RCT design documents and map the extracted variables to EHR features with description matching and knowledge networks. Module 2 then develops techniques for cohort construction using advanced phenotyping algorithms to both identify patients with diseases of interest and define the treatment arms. Module 3 introduces methods for variable curation, including a list of existing tools to extract baseline variables from different sources (eg, codified, free text, and medical imaging) and end points of various types (eg, death, binary, temporal, and numerical). Finally, module 4 presents validation and robust modeling methods, and we propose a strategy to create gold-standard labels for EHR variables of interest to validate data curation quality and perform subsequent causal modeling for RWE. In addition to the workflow proposed in our pipeline, we also develop a reporting guideline for RWE that covers the necessary information to facilitate transparent reporting and reproducibility of results. Moreover, our pipeline is highly data driven, enhancing study data with a rich variety of publicly available information and knowledge sources. We also showcase our pipeline and provide guidance on the deployment of relevant tools by revisiting the emulation of the Clinical Outcomes of Surgical Therapy Study Group Trial on laparoscopy-assisted colectomy versus open colectomy in patients with early-stage colon cancer. We also draw on existing literature on EHR emulation of RCTs together with our own studies with the Mass General Brigham EHR. %M 37227772 %R 10.2196/45662 %U https://www.jmir.org/2023/1/e45662 %U https://doi.org/10.2196/45662 %U http://www.ncbi.nlm.nih.gov/pubmed/37227772 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e47173 %T Low Adoption of Video Consultations in Post–COVID-19 General Practice in Northern Europe: Barriers to Use and Potential Action Points %A Assing Hvidt,Elisabeth %A Atherton,Helen %A Keuper,Jelle %A Kristiansen,Eli %A Lüchau,Elle Christine %A Lønnebakke Norberg,Børge %A Steinhäuser,Jost %A van den Heuvel,Johannes %A van Tuyl,Lilian %+ Research Unit of General Practice, Department of Public Health, University of Southern Denmark, JB Winsløwsvej 9 A, Odense, 5000, Denmark, 45 65504247, ehvidt@health.sdu.dk %K video consultation %K adoption %K general practice %K Northern Europe %K barriers %K action potential %K Europe %K viewpoint %K consultation %K barrier %K clinician %K digital care %K care %K implementation %K practitioner %K COVID-19 %K research %D 2023 %7 22.5.2023 %9 Viewpoint %J J Med Internet Res %G English %X In the wake of the COVID-19 pandemic, video consultation was introduced in general practice in many countries around the world as a solution to provide remote health care to patients. It was assumed that video consultation would find widespread adoption in post–COVID-19 general practice. However, adoption rates remain low across countries in Northern Europe, suggesting that barriers to its use exist among general practitioners and other practice staff. In this viewpoint, we take a comparative approach, reflecting on similarities and differences in implementation conditions of video consultations in 5 Northern European countries’ general practice settings that might have created barriers to its use within general practice. We convened at a cross-disciplinary seminar in May 2022 with researchers and clinicians from 5 Northern European countries with expertise in digital care in general practice, and this viewpoint emerged out of dialogues from that seminar. We have reflected on barriers across general practice settings in our countries, such as lacking technological and financial support for general practitioners, that we feel are critical for adoption of video consultation in the coming years. Furthermore, there is a need to further investigate the contribution of cultural elements, such as professional norms and values, to adoption. This viewpoint may inform policy work to ensure that a sustainable level of video consultation use can be reached in the future, one that reflects the reality of general practice settings rather than policy optimism. %M 37213196 %R 10.2196/47173 %U https://www.jmir.org/2023/1/e47173 %U https://doi.org/10.2196/47173 %U http://www.ncbi.nlm.nih.gov/pubmed/37213196 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e41808 %T Using a Clinical Data Warehouse to Calculate and Present Key Metrics for the Radiology Department: Implementation and Performance Evaluation %A Liman,Leon %A May,Bernd %A Fette,Georg %A Krebs,Jonathan %A Puppe,Frank %+ Chair of Computer Science VI, Würzburg University, Am Hubland, Würzburg, 97074, Germany, 49 9313189250, leon.liman@uni-wuerzburg.de %K data warehouse %K electronic health records %K radiology %K statistics and numerical data %K hospital data %K eHealth %K medical records %D 2023 %7 22.5.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: Due to the importance of radiologic examinations, such as X-rays or computed tomography scans, for many clinical diagnoses, the optimal use of the radiology department is 1 of the primary goals of many hospitals. Objective: This study aims to calculate the key metrics of this use by creating a radiology data warehouse solution, where data from radiology information systems (RISs) can be imported and then queried using a query language as well as a graphical user interface (GUI). Methods: Using a simple configuration file, the developed system allowed for the processing of radiology data exported from any kind of RIS into a Microsoft Excel, comma-separated value (CSV), or JavaScript Object Notation (JSON) file. These data were then imported into a clinical data warehouse. Additional values based on the radiology data were calculated during this import process by implementing 1 of several provided interfaces. Afterward, the query language and GUI of the data warehouse were used to configure and calculate reports on these data. For the most common types of requested reports, a web interface was created to view their numbers as graphics. Results: The tool was successfully tested with the data of 4 different German hospitals from 2018 to 2021, with a total of 1,436,111 examinations. The user feedback was good, since all their queries could be answered if the available data were sufficient. The initial processing of the radiology data for using them with the clinical data warehouse took (depending on the amount of data provided by each hospital) between 7 minutes and 1 hour 11 minutes. Calculating 3 reports of different complexities on the data of each hospital was possible in 1-3 seconds for reports with up to 200 individual calculations and in up to 1.5 minutes for reports with up to 8200 individual calculations. Conclusions: A system was developed with the main advantage of being generic concerning the export of different RISs as well as concerning the configuration of queries for various reports. The queries could be configured easily using the GUI of the data warehouse, and their results could be exported into the standard formats Excel and CSV for further processing. %M 37213191 %R 10.2196/41808 %U https://medinform.jmir.org/2023/1/e41808 %U https://doi.org/10.2196/41808 %U http://www.ncbi.nlm.nih.gov/pubmed/37213191 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e48109 %T Comprehensive Acute Kidney Injury Survivor Care: Protocol for the Randomized Acute Kidney Injury in Care Transitions Pilot Trial %A May,Heather P %A Griffin,Joan M %A Herges,Joseph R %A Kashani,Kianoush B %A Kattah,Andrea G %A Mara,Kristin C %A McCoy,Rozalina G %A Rule,Andrew D %A Tinaglia,Angeliki G %A Barreto,Erin F %+ Mayo Clinic, 200 1st Ave SW, Rochester, MN, 55905, United States, 1 5072555866, barreto.erin@mayo.edu %K acute kidney injury %K acute renal failure %K care transitions %K chronic kidney disease %K nephrologists %K randomized controlled trials %D 2023 %7 22.5.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Innovative care models are needed to address gaps in kidney care follow-up among acute kidney injury (AKI) survivors. We developed the multidisciplinary AKI in Care Transitions (ACT) program, which embeds post-AKI care in patients’ primary care clinic. Objective: The objective of this randomized pilot trial is to test the feasibility and acceptability of the ACT program and study protocol, including recruitment and retention, procedures, and outcome measures. Methods: The study will be conducted at Mayo Clinic in Rochester, Minnesota, a tertiary care center with a local primary care practice. Individuals who are included have stage 3 AKI during their hospitalization, do not require dialysis at discharge, have a local primary care provider, and are discharged to their home. Patients unable or unwilling to provide informed consent and recipients of any transplant within 100 days of enrollment are excluded. Consented patients are randomized to receive the intervention (ie, ACT program) or usual care. The ACT program intervention includes predischarge kidney health education from nurses and coordinated postdischarge laboratory monitoring (serum creatinine and urine protein assessment) and follow-up with a primary care provider and pharmacist within 14 days. The usual care group receives no specific study-related intervention, and any aspects of AKI care are at the direction of the treating team. This study will examine the feasibility of the ACT program, including recruitment, randomization and retention in a trial setting, and intervention fidelity. The feasibility and acceptability of participating in the ACT program will also be examined in qualitative interviews with patients and staff and through surveys. Qualitative interviews will be deductively and inductively coded and themes compared across data types. Observations of clinical encounters will be examined for discussion and care plans related to kidney health. Descriptive analyses will summarize quantitative measures of the feasibility and acceptability of ACT. Participants’ knowledge about kidney health, quality of life, and process outcomes (eg, type and timing of laboratory assessments) will be described for both groups. Clinical outcomes (eg, unplanned rehospitalization) up to 12 months will be compared with Cox proportional hazards models. Results: This study received funding from the Agency for Health Care Research and Quality on April 21, 2021, and was approved by the Institutional Review Board on December 14, 2021. As of March 14, 2023, seventeen participants each have been enrolled in the intervention and usual care groups. Conclusions: Feasible and generalizable AKI survivor care delivery models are needed to improve care processes and health outcomes. This pilot trial will test the ACT program, which uses a multidisciplinary model focused on primary care to address this gap. Trial Registration: ClinicalTrials.gov NCT05184894; https://www.clinicaltrials.gov/ct2/show/NCT05184894 International Registered Report Identifier (IRRID): DERR1-10.2196/48109 %M 37213187 %R 10.2196/48109 %U https://www.researchprotocols.org/2023/1/e48109 %U https://doi.org/10.2196/48109 %U http://www.ncbi.nlm.nih.gov/pubmed/37213187 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e45645 %T Eliciting Insights From Chat Logs of the 25X5 Symposium to Reduce Documentation Burden: Novel Application of Topic Modeling %A Moy,Amanda J %A Withall,Jennifer %A Hobensack,Mollie %A Yeji Lee,Rachel %A Levy,Deborah R %A Rossetti,Sarah C %A Rosenbloom,S Trent %A Johnson,Kevin %A Cato,Kenrick %+ Department of Biomedical Informatics, Columbia University, 622 West 168th Street, PH20, New York, NY, 10032, United States, 1 6504270678, am3458@cumc.columbia.edu %K topic modeling %K content analysis %K online chat %K virtual conference %K documentation burden %K burnout %K physicians %K nurses %K policy %K symposium %K chat bot %D 2023 %7 17.5.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Addressing clinician documentation burden through “targeted solutions” is a growing priority for many organizations ranging from government and academia to industry. Between January and February 2021, the 25 by 5: Symposium to Reduce Documentation Burden on US Clinicians by 75% (25X5 Symposium) convened across 2 weekly 2-hour sessions among experts and stakeholders to generate actionable goals for reducing clinician documentation over the next 5 years. Throughout this web-based symposium, we passively collected attendees’ contributions to a chat functionality—with their knowledge that the content would be deidentified and made publicly available. This presented a novel opportunity to synthesize and understand participants’ perceptions and interests from chat messages. We performed a content analysis of 25X5 Symposium chat logs to identify themes about reducing clinician documentation burden. Objective: The objective of this study was to explore unstructured chat log content from the web-based 25X5 Symposium to elicit latent insights on clinician documentation burden among clinicians, health care leaders, and other stakeholders using topic modeling. Methods: Across the 6 sessions, we captured 1787 messages among 167 unique chat participants cumulatively; 14 were private messages not included in the analysis. We implemented a latent Dirichlet allocation (LDA) topic model on the aggregated dataset to identify clinician documentation burden topics mentioned in the chat logs. Coherence scores and manual examination informed optimal model selection. Next, 5 domain experts independently and qualitatively assigned descriptive labels to model-identified topics and classified them into higher-level categories, which were finalized through a panel consensus. Results: We uncovered ten topics using the LDA model: (1) determining data and documentation needs (422/1773, 23.8%); (2) collectively reassessing documentation requirements in electronic health records (EHRs) (252/1773, 14.2%); (3) focusing documentation on patient narrative (162/1773, 9.1%); (4) documentation that adds value (147/1773, 8.3%); (5) regulatory impact on clinician burden (142/1773, 8%); (6) improved EHR user interface and design (128/1773, 7.2%); (7) addressing poor usability (122/1773, 6.9%); (8) sharing 25X5 Symposium resources (122/1773, 6.9%); (9) capturing data related to clinician practice (113/1773, 6.4%); and (10) the role of quality measures and technology in burnout (110/1773, 6.2%). Among these 10 topics, 5 high-level categories emerged: consensus building (821/1773, 46.3%), burden sources (365/1773, 20.6%), EHR design (250/1773, 14.1%), patient-centered care (162/1773, 9.1%), and symposium comments (122/1773, 6.9%). Conclusions: We conducted a topic modeling analysis on 25X5 Symposium multiparticipant chat logs to explore the feasibility of this novel application and elicit additional insights on clinician documentation burden among attendees. Based on the results of our LDA analysis, consensus building, burden sources, EHR design, and patient-centered care may be important themes to consider when addressing clinician documentation burden. Our findings demonstrate the value of topic modeling in discovering topics associated with clinician documentation burden using unstructured textual content. Topic modeling may be a suitable approach to examine latent themes presented in web-based symposium chat logs. %M 37195741 %R 10.2196/45645 %U https://www.jmir.org/2023/1/e45645 %U https://doi.org/10.2196/45645 %U http://www.ncbi.nlm.nih.gov/pubmed/37195741 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e43518 %T Factors Influencing Admission Decisions in Skilled Nursing Facilities: Retrospective Quantitative Study %A Strickland,Caroline %A Chi,Nancy %A Ditz,Laura %A Gomez,Luisa %A Wagner,Brittin %A Wang,Stanley %A Lizotte,Daniel J %+ Department of Computer Science, University of Western Ontario, 1151 Richmond St, London, ON, N6A 3K7, Canada, 1 519 661 2111, cstrick4@uwo.ca %K decision-making %K skilled nursing facility %K patient admission %K decision %K nursing %K clinical %K database %K health informatics %K diagnosis %K modeling %K connection %K patient %D 2023 %7 17.5.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Occupancy rates within skilled nursing facilities (SNFs) in the United States have reached a record low. Understanding drivers of occupancy, including admission decisions, is critical for assessing the recovery of the long-term care sector as a whole. We provide the first comprehensive analysis of financial, clinical, and operational factors that impact whether a patient referral to an SNF is accepted or denied, using a large health informatics database. Objective: Our key objectives were to describe the distribution of referrals sent to SNFs in terms of key referral- and facility-level features; analyze key financial, clinical, and operational variables and their relationship to admission decisions; and identify the key potential reasons behind referral decisions in the context of learning health systems. Methods: We extracted and cleaned referral data from 627 SNFs from January 2020 to March 2022, including information on SNF daily operations (occupancy and nursing hours), referral-level factors (insurance type and primary diagnosis), and facility-level factors (overall 5-star rating and urban versus rural status). We computed descriptive statistics and applied regression modeling to identify and describe the relationships between these factors and referral decisions, considering them individually and controlling for other factors to understand their impact on the decision-making process. Results: When analyzing daily operation values, no significant relationship between SNF occupancy or nursing hours and referral acceptance was observed (P>.05). By analyzing referral-level factors, we found that the primary diagnosis category and insurance type of the patient were significantly related to referral acceptance (P<.05). Referrals with primary diagnoses within the category “Diseases of the Musculoskeletal System” are least often denied whereas those with diagnoses within the “Mental Illness” category are most often denied (compared with other diagnosis categories). Furthermore, private insurance holders are least often denied whereas “medicaid” holders are most often denied (compared with other insurance types). When analyzing facility-level factors, we found that the overall 5-star rating and urban versus rural status of an SNF are significantly related to referral acceptance (P<.05). We found a positive but nonmonotonic relationship between the 5-star rating and referral acceptance rates, with the highest acceptance rates found among 5-star facilities. In addition, we found that SNFs in urban areas have lower acceptance rates than their rural counterparts. Conclusions: While many factors may influence a referral acceptance, care challenges associated with individual diagnoses and financial challenges associated with different remuneration types were found to be the strongest drivers. Understanding these drivers is essential in being more intentional in the process of accepting or denying referrals. We have interpreted our results using an adaptive leadership framework and suggested how SNFs can be more purposeful with their decisions while striving to achieve appropriate occupancy levels in ways that meet their goals and patients’ needs. %M 37195755 %R 10.2196/43518 %U https://www.jmir.org/2023/1/e43518 %U https://doi.org/10.2196/43518 %U http://www.ncbi.nlm.nih.gov/pubmed/37195755 %0 Journal Article %@ 2561-1011 %I JMIR Publications %V 7 %N %P e45190 %T Continuous Data-Driven Monitoring in Critical Congenital Heart Disease: Clinical Deterioration Model Development %A Zoodsma,Ruben S %A Bosch,Rian %A Alderliesten,Thomas %A Bollen,Casper W %A Kappen,Teus H %A Koomen,Erik %A Siebes,Arno %A Nijman,Joppe %+ Department of Paediatric Intensive Care, University Medical Center Utrecht, Office Number KG0.2.306.1, Lundlaan 6, Utrecht, 3584 EA, Netherlands, 31 88 7575092, j.nijman@umcutrecht.nl %K artificial intelligence %K aberration detection %K clinical deterioration %K classification model %K paediatric intensive care %K pediatric intensive care %K congenital heart disease %K cardiac monitoring %K machine learning %K peri-operative %K perioperative %K surgery %D 2023 %7 16.5.2023 %9 Original Paper %J JMIR Cardio %G English %X Background: Critical congenital heart disease (cCHD)—requiring cardiac intervention in the first year of life for survival—occurs globally in 2-3 of every 1000 live births. In the critical perioperative period, intensive multimodal monitoring at a pediatric intensive care unit (PICU) is warranted, as their organs—especially the brain—may be severely injured due to hemodynamic and respiratory events. These 24/7 clinical data streams yield large quantities of high-frequency data, which are challenging in terms of interpretation due to the varying and dynamic physiology innate to cCHD. Through advanced data science algorithms, these dynamic data can be condensed into comprehensible information, reducing the cognitive load on the medical team and providing data-driven monitoring support through automated detection of clinical deterioration, which may facilitate timely intervention. Objective: This study aimed to develop a clinical deterioration detection algorithm for PICU patients with cCHD. Methods: Retrospectively, synchronous per-second data of cerebral regional oxygen saturation (rSO2) and 4 vital parameters (respiratory rate, heart rate, oxygen saturation, and invasive mean blood pressure) in neonates with cCHD admitted to the University Medical Center Utrecht, the Netherlands, between 2002 and 2018 were extracted. Patients were stratified based on mean oxygen saturation during admission to account for physiological differences between acyanotic and cyanotic cCHD. Each subset was used to train our algorithm in classifying data as either stable, unstable, or sensor dysfunction. The algorithm was designed to detect combinations of parameters abnormal to the stratified subpopulation and significant deviations from the patient’s unique baseline, which were further analyzed to distinguish clinical improvement from deterioration. Novel data were used for testing, visualized in detail, and internally validated by pediatric intensivists. Results: A retrospective query yielded 4600 hours and 209 hours of per-second data in 78 and 10 neonates for, respectively, training and testing purposes. During testing, stable episodes occurred 153 times, of which 134 (88%) were correctly detected. Unstable episodes were correctly noted in 46 of 57 (81%) observed episodes. Twelve expert-confirmed unstable episodes were missed in testing. Time-percentual accuracy was 93% and 77% for, respectively, stable and unstable episodes. A total of 138 sensorial dysfunctions were detected, of which 130 (94%) were correct. Conclusions: In this proof-of-concept study, a clinical deterioration detection algorithm was developed and retrospectively evaluated to classify clinical stability and instability, achieving reasonable performance considering the heterogeneous population of neonates with cCHD. Combined analysis of baseline (ie, patient-specific) deviations and simultaneous parameter-shifting (ie, population-specific) proofs would be promising with respect to enhancing applicability to heterogeneous critically ill pediatric populations. After prospective validation, the current—and comparable—models may, in the future, be used in the automated detection of clinical deterioration and eventually provide data-driven monitoring support to the medical team, allowing for timely intervention. %M 37191988 %R 10.2196/45190 %U https://cardio.jmir.org/2023/1/e45190 %U https://doi.org/10.2196/45190 %U http://www.ncbi.nlm.nih.gov/pubmed/37191988 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e44455 %T An Electronic Dashboard to Improve Dosing of Hydroxychloroquine Within the Veterans Health Care System: Time Series Analysis %A Montgomery,Anna %A Tarasovsky,Gary %A Izadi,Zara %A Shiboski,Stephen %A Whooley,Mary A %A Dana,Jo %A Ehiorobo,Iziegbe %A Barton,Jennifer %A Bennett,Lori %A Chung,Lorinda %A Reiter,Kimberly %A Wahl,Elizabeth %A Subash,Meera %A Schmajuk,Gabriela %+ University of California San Francisco, 4150 Clement St, San Francisco, CA, 94121, United States, 1 415 221 4810, Gabriela.schmajuk@ucsf.edu %K medical informatics %K patient safety %K health IT %K hydroxychloroquine %K dashboard %K Veterans Health Administration %K audit and feedback %K electronic health record %D 2023 %7 12.5.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: Hydroxychloroquine (HCQ) is commonly used for patients with autoimmune conditions. Long-term use of HCQ can cause retinal toxicity, but this risk can be reduced if high doses are avoided. Objective: We developed and piloted an electronic health record–based dashboard to improve the safe prescribing of HCQ within the Veterans Health Administration (VHA). We observed pilot facilities over a 1-year period to determine whether they were able to improve the proportion of patients receiving inappropriate doses of HCQ. Methods: Patients receiving HCQ were identified from the VHA corporate data warehouse. Using PowerBI (Microsoft Corp), we constructed a dashboard to display patient identifiers and the most recent HCQ dose and weight (flagged if ≥5.2 mg/kg/day). Six VHA pilot facilities were enlisted to test the dashboard and invited to participate in monthly webinars. We performed an interrupted time series analysis using synthetic controls to assess changes in the proportion of patients receiving HCQ ≥5.2 mg/kg/day between October 2020 and November 2021. Results: At the start of the study period, we identified 18,525 total users of HCQ nationwide at 128 facilities in the VHA, including 1365 patients at the 6 pilot facilities. Nationwide, at baseline, 19.8% (3671/18,525) of patients were receiving high doses of HCQ. We observed significant improvements in the proportion of HCQ prescribed at doses ≥5.2 mg/kg/day among pilot facilities after the dashboard was deployed (–0.06; 95% CI –0.08 to –0.04). The difference in the postintervention linear trend for pilot versus synthetic controls was also significant (–0.06; 95% CI –0.08 to –0.05). Conclusions: The use of an electronic health record–based dashboard reduced the proportion of patients receiving higher than recommended doses of HCQ and significantly improved performance at 6 VHA facilities. National roll-out of the dashboard will enable further improvements in the safe prescribing of HCQ. %M 37171858 %R 10.2196/44455 %U https://medinform.jmir.org/2023/1/e44455 %U https://doi.org/10.2196/44455 %U http://www.ncbi.nlm.nih.gov/pubmed/37171858 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e46694 %T iCHECK-DH: Guidelines and Checklist for the Reporting on Digital Health Implementations %A Perrin Franck,Caroline %A Babington-Ashaye,Awa %A Dietrich,Damien %A Bediang,Georges %A Veltsos,Philippe %A Gupta,Pramendra Prasad %A Juech,Claudia %A Kadam,Rigveda %A Collin,Maxime %A Setian,Lucy %A Serrano Pons,Jordi %A Kwankam,S Yunkap %A Garrette,Béatrice %A Barbe,Solenne %A Bagayoko,Cheick Oumar %A Mehl,Garrett %A Lovis,Christian %A Geissbuhler,Antoine %+ Department of Radiology and Medical Informatics, Faculty of Medicine, University of Geneva, Chemin des Mines 9, Geneva, CH-1202, Switzerland, 41 0787997725, caroline.perrin@gmx.de %K implementation science %K knowledge management %K reporting standards %K publishing standards %K guideline %K Digital Health Hub %K reporting guideline %K digital health implementation %K health outcome %D 2023 %7 10.5.2023 %9 Implementation Report %J J Med Internet Res %G English %X Background: Implementation of digital health technologies has grown rapidly, but many remain limited to pilot studies due to challenges, such as a lack of evidence or barriers to implementation. Overcoming these challenges requires learning from previous implementations and systematically documenting implementation processes to better understand the real-world impact of a technology and identify effective strategies for future implementation. Objective: A group of global experts, facilitated by the Geneva Digital Health Hub, developed the Guidelines and Checklist for the Reporting on Digital Health Implementations (iCHECK-DH, pronounced “I checked”) to improve the completeness of reporting on digital health implementations. Methods: A guideline development group was convened to define key considerations and criteria for reporting on digital health implementations. To ensure the practicality and effectiveness of the checklist, it was pilot-tested by applying it to several real-world digital health implementations, and adjustments were made based on the feedback received. The guiding principle for the development of iCHECK-DH was to identify the minimum set of information needed to comprehensively define a digital health implementation, to support the identification of key factors for success and failure, and to enable others to replicate it in different settings. Results: The result was a 20-item checklist with detailed explanations and examples in this paper. The authors anticipate that widespread adoption will standardize the quality of reporting and, indirectly, improve implementation standards and best practices. Conclusions: Guidelines for reporting on digital health implementations are important to ensure the accuracy, completeness, and consistency of reported information. This allows for meaningful comparison and evaluation of results, transparency, and accountability and informs stakeholder decision-making. i-CHECK-DH facilitates standardization of the way information is collected and reported, improving systematic documentation and knowledge transfer that can lead to the development of more effective digital health interventions and better health outcomes. %M 37163336 %R 10.2196/46694 %U https://www.jmir.org/2023/1/e46694 %U https://doi.org/10.2196/46694 %U http://www.ncbi.nlm.nih.gov/pubmed/37163336 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e44597 %T Chinese Clinical Named Entity Recognition From Electronic Medical Records Based on Multisemantic Features by Using Robustly Optimized Bidirectional Encoder Representation From Transformers Pretraining Approach Whole Word Masking and Convolutional Neural Networks: Model Development and Validation %A Wang,Weijie %A Li,Xiaoying %A Ren,Huiling %A Gao,Dongping %A Fang,An %+ Institute of Medical Information and Library, Chinese Academy of Medical Sciences & Peking Union Medical College, 69 Dongdan N St, Beijing, 100005, China, 86 010 52328911, ren.huiling@imicams.ac.cn %K Chinese clinical named entity recognition %K multisemantic features %K image feature %K Robustly Optimized Bidirectional Encoder Representation from Transformers Pretraining Approach Whole Word Masking %K RoBERTa-wwm %K convolutional neural network %K CNN %D 2023 %7 10.5.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: Clinical electronic medical records (EMRs) contain important information on patients’ anatomy, symptoms, examinations, diagnoses, and medications. Large-scale mining of rich medical information from EMRs will provide notable reference value for medical research. With the complexity of Chinese grammar and blurred boundaries of Chinese words, Chinese clinical named entity recognition (CNER) remains a notable challenge. Follow-up tasks such as medical entity structuring, medical entity standardization, medical entity relationship extraction, and medical knowledge graph construction largely depend on medical named entity recognition effects. A promising CNER result would provide reliable support for building domain knowledge graphs, knowledge bases, and knowledge retrieval systems. Furthermore, it would provide research ideas for scientists and medical decision-making references for doctors and even guide patients on disease and health management. Therefore, obtaining excellent CNER results is essential. Objective: We aimed to propose a Chinese CNER method to learn semantics-enriched representations for comprehensively enhancing machines to understand deep semantic information of EMRs by using multisemantic features, which makes medical information more readable and understandable. Methods: First, we used Robustly Optimized Bidirectional Encoder Representation from Transformers Pretraining Approach Whole Word Masking (RoBERTa-wwm) with dynamic fusion and Chinese character features, including 5-stroke code, Zheng code, phonological code, and stroke code, extracted by 1-dimensional convolutional neural networks (CNNs) to obtain fine-grained semantic features of Chinese characters. Subsequently, we converted Chinese characters into square images to obtain Chinese character image features from another modality by using a 2-dimensional CNN. Finally, we input multisemantic features into Bidirectional Long Short-Term Memory with Conditional Random Fields to achieve Chinese CNER. The effectiveness of our model was compared with that of the baseline and existing research models, and the features involved in the model were ablated and analyzed to verify the model’s effectiveness. Results: We collected 1379 Yidu-S4K EMRs containing 23,655 entities in 6 categories and 2007 self-annotated EMRs containing 118,643 entities in 7 categories. The experiments showed that our model outperformed the comparison experiments, with F1-scores of 89.28% and 84.61% on the Yidu-S4K and self-annotated data sets, respectively. The results of the ablation analysis demonstrated that each feature and method we used could improve the entity recognition ability. Conclusions: Our proposed CNER method would mine the richer deep semantic information in EMRs by multisemantic embedding using RoBERTa-wwm and CNNs, enhancing the semantic recognition of characters at different granularity levels and improving the generalization capability of the method by achieving information complementarity among different semantic features, thus making the machine semantically understand EMRs and improving the CNER task accuracy. %M 37163343 %R 10.2196/44597 %U https://medinform.jmir.org/2023/1/e44597 %U https://doi.org/10.2196/44597 %U http://www.ncbi.nlm.nih.gov/pubmed/37163343 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e44776 %T Traumatic Brain Injury Intensive Evaluation and Treatment Program: Protocol for a Partnered Evaluation Initiative Mixed Methods Study %A Haun,Jolie N %A Nakase-Richardson,Risa %A Melillo,Christine %A Kean,Jacob %A Benzinger,Rachel C %A Schneider,Tali %A Pugh,Mary Jo V %+ James A. Haley Veterans' Hospital, Research Service, 8200 Grand Oak Circle, Tampa, FL, 33637, United States, 1 813 558 7622, rachel.benzinger@va.gov %K service member %K rehabilitation %K traumatic brain injury %K TBI %K posttraumatic stress disorder %K PTSD %K pain %K military %K brain injury %K trauma %K traumatic %K participatory %K recovery %K veteran %K implementation %K service delivery %K protocol %K treatment program %K implementation %K health care implementation %K Consolidated Framework for Implementation Research %K CFIR %K cognitive %K cognition %K brain %K script %K Bayesian %K network analysis %K directed acyclic graph %K effect size %K missing data %K inpatient %K modality %D 2023 %7 9.5.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: The traumatic brain injury (TBI) Intensive Evaluation and Treatment Program (IETP) is an innovative modality for delivering evidence-based treatments in a residential, inpatient format to special operational forces service members and veterans with mild TBI. IETPs provide bundled evidence-based assessment, treatment, referral, and case management in concordance with the existing guidelines for mild TBI and commonly co-occurring comorbidities. To date, there has been no formal characterization or evaluation of the IETP to understand the determinants of implementation across the system of care. The goal of our partnered evaluation initiative (PEI) with an operational partner, the Physical Medicine and Rehabilitation National Program Office, is to facilitate the full implementation of the IETP across all 5 Veterans Health Administration TBI–Centers of Excellence (TBI-COE) and to inform minimum standards while supporting the unique characteristics of each site. Objective: This IETP partnered evaluation will describe each of the 5 TBI-COE IETP services and state of implementation to identify opportunities for adaptation and scale, characterize the relationship between patient characteristics and clinical services received, evaluate the outcomes for participants in the IETP, and inform ongoing implementation and knowledge translation efforts to support IETP expansion. In alignment with the goals of the protocol, ineffective treatment components will be targeted for deimplementation. Methods: A 3-year concurrent mixed methods evaluation using a participatory approach in collaboration with the operational partner and TBI-COE site leadership will be conducted. Qualitative observations, semistructured focus groups, and interviewing methods will be used to describe IETP, stakeholder experiences and needs, and suggestions for IETP implementation. Quantitative methods will include primary data collection from patients in the IETP at each site to characterize long-term outcomes and patient satisfaction with treatment and secondary data collection to quantitatively characterize patient-level and care system–level data. Finally, data sets will be triangulated to share data findings with partners to inform ongoing implementation efforts. Results: Data collection began in December 2021 and is currently ongoing. The results and deliverables will inform IETP characterization, evaluation, implementation, and knowledge translation. Conclusions: The results of this evaluation seek to provide an understanding of the determinants affecting the implementation of IETPs. Service member, staff, and stakeholder insights will inform the state of implementation at each site, and quantitative measures will provide options for standardized outcome measures. This evaluation is expected to inform national Physical Medicine and Rehabilitation Office policies and processes and knowledge translation efforts to improve and expand the IETP. Future work may include cost evaluations and rigorous research, such as randomized controlled trials. International Registered Report Identifier (IRRID): DERR1-10.2196/44776 %M 37159250 %R 10.2196/44776 %U https://www.researchprotocols.org/2023/1/e44776 %U https://doi.org/10.2196/44776 %U http://www.ncbi.nlm.nih.gov/pubmed/37159250 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e43695 %T Effectiveness of a Mobile App (PIMPmyHospital) in Reducing Therapeutic Turnaround Times in an Emergency Department: Protocol for a Pre- and Posttest Study %A Ehrler,Frederic %A Tuor,Carlotta %A Rey,Robin %A Trompier,Rémy %A Berger,Antoine %A Ramusi,Michael %A Courvoisier,Delphine S %A Siebert,Johan N %+ Geneva Children’s Hospital, Department of Pediatric Emergency Medicine, Geneva University Hospitals, Avenue de la Roseraie, 47, Geneva, 1205, Switzerland, 41 795534072, Johan.Siebert@hcuge.ch %K clinical laboratory information systems %K laboratory results %K digital technology %K emergency department %K emergency service %K hospital %K length of stay %K mobile app %K mobile health %K mHealth %K pediatrics %K therapeutic turnaround time %D 2023 %7 3.5.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Delays in reviewing issued laboratory results in emergency departments (EDs) can adversely affect efficiency and quality of care. One opportunity to improve therapeutic turnaround time could be to provide real-time access to laboratory results on mobile devices available to every caregiver. We developed a mobile app named “Patients In My Pocket in my Hospital” (PIMPmyHospital) to help ED caregivers automatically obtain and share relevant information about the patients they care for including laboratory results. Objective: This pre- and posttest study aims to explore whether the implementation of the PIMPmyHospital app impacts the timeliness with which ED physicians and nurses remotely access laboratory results while actively working in their real-world environment, including ED length of stay, technology acceptance and usability among users, and how specific in-app alerts impact on its effectiveness. Methods: This single-center study of nonequivalent pre- and posttest comparison group design will be conducted before and after the implementation of the app in a tertiary pediatric ED in Switzerland. The retrospective period will cover the previous 12 months, and the prospective period will cover the following 6 months. Participants will be postgraduate residents pursuing a ≤6-year residency in pediatrics, pediatric emergency medicine fellows, and registered nurses from the pediatric ED. The primary outcome will be the mean elapsed time in minutes from delivery of laboratory results to caregivers’ consideration by accessing them either through the hospital’s electronic medical records or through the app before and after the implementation of the app, respectively. As secondary outcomes, participants will be queried about the acceptance and usability of the app using the Unified Theory of Acceptance and Use of Technology model and the System Usability Scale. ED length of stay will be compared before and after the implementation of the app for patients with laboratory results. The impact of specific alerts on the app, such as a flashing icon or sound for reported pathological values, will be reported. Results: Retrospective data collection gathered from the institutional data set will span a 12-month period from October 2021 to October 2022, while the 6-month prospective collection will begin with the implementation of the app in November 2022 and is expected to cease at the end of April 2023. We expect the results of the study to be published in a peer-reviewed journal in late 2023. Conclusions: This study will show the potential reach, effectiveness, acceptance, and use of the PIMPmyHospital app among ED caregivers. The findings of this study will serve as the basis for future research on the app and any further development to improve its effectiveness. Trial Registration: ClinicalTrials.gov NCT05557331; https://clinicaltrials.gov/ct2/show/NCT05557331 Trial Registration: ClinicalTrials.gov NCT05557331; https://clinicaltrials.gov/ct2/show/NCT05557331 International Registered Report Identifier (IRRID): PRR1-10.2196/43695 %M 37133909 %R 10.2196/43695 %U https://www.researchprotocols.org/2023/1/e43695 %U https://doi.org/10.2196/43695 %U http://www.ncbi.nlm.nih.gov/pubmed/37133909 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e32962 %T The Gap Between AI and Bedside: Participatory Workshop on the Barriers to the Integration, Translation, and Adoption of Digital Health Care and AI Startup Technology Into Clinical Practice %A Olaye,Iredia M %A Seixas,Azizi A %+ Department of Medicine, Weill Cornell Medicine, Cornell University, 1300 York Avenue, Box #46, New York, NY, 10065, United States, 1 646 962 5050, imo4001@med.cornell.edu %K digital health %K startups %K venture capital %K artificial intelligence %K AI translation %K clinical practice %K early-stage %K funding %K bedside %K machine learning %K technology %K tech %K qualitative %K workshop %K entrepreneurs %D 2023 %7 2.5.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Artificial intelligence (AI) and digital health technological innovations from startup companies used in clinical practice can yield better health outcomes, reduce health care costs, and improve patients' experience. However, the integration, translation, and adoption of these technologies into clinical practice are plagued with many challenges and are lagging. Furthermore, explanations of the impediments to clinical translation are largely unknown and have not been systematically studied from the perspective of AI and digital health care startup founders and executives. Objective: The aim of this paper is to describe the barriers to integrating early-stage technologies in clinical practice and health care systems from the perspectives of digital health and health care AI founders and executives. Methods: A stakeholder focus group workshop was conducted with a sample of 10 early-stage digital health and health care AI founders and executives. Digital health, health care AI, digital health–focused venture capitalists, and physician executives were represented. Using an inductive thematic analysis approach, transcripts were organized, queried, and analyzed for thematic convergence. Results: We identified the following four categories of barriers in the integration of early-stage digital health innovations into clinical practice and health care systems: (1) lack of knowledge of health system technology procurement protocols and best practices, (2) demanding regulatory and validation requirements, (3) challenges within the health system technology procurement process, and (4) disadvantages of early-stage digital health companies compared to large technology conglomerates. Recommendations from the study participants were also synthesized to create a road map to mitigate the barriers to integrating early-stage or novel digital health technologies in clinical practice. Conclusions: Early-stage digital health and health care AI entrepreneurs identified numerous barriers to integrating digital health solutions into clinical practice. Mitigation initiatives should create opportunities for early-stage digital health technology companies and health care providers to interact, develop relationships, and use evidence-based research and best practices during health care technology procurement and evaluation processes. %M 37129947 %R 10.2196/32962 %U https://www.jmir.org/2023/1/e32962 %U https://doi.org/10.2196/32962 %U http://www.ncbi.nlm.nih.gov/pubmed/37129947 %0 Journal Article %@ 2561-9128 %I JMIR Publications %V 6 %N %P e36172 %T Toward Enhanced Clinical Decision Support for Patients Undergoing a Hip or Knee Replacement: Focus Group and Interview Study With Surgeons %A Grant,Sabrina %A Tonkin,Emma %A Craddock,Ian %A Blom,Ashley %A Holmes,Michael %A Judge,Andrew %A Masullo,Alessandro %A Perello Nieto,Miquel %A Song,Hao %A Whitehouse,Michael %A Flach,Peter %A Gooberman-Hill,Rachael %+ Digital Health, Faculty of Engineering, University of Bristol, Bristol, BS8 1TH, United Kingdom, 44 117 42 82343 ext 82343, cselt@bristol.ac.uk %K arthroplasty %K knee replacement %K hip replacement %K orthopedic surgery %K clinical decision-making %K postoperative follow-up %K home monitoring %K wearables %K video %D 2023 %7 24.4.2023 %9 Original Paper %J JMIR Perioper Med %G English %X Background: The current assessment of recovery after total hip or knee replacement is largely based on the measurement of health outcomes through self-report and clinical observations at follow-up appointments in clinical settings. Home activity-based monitoring may improve assessment of recovery by enabling the collection of more holistic information on a continuous basis. Objective: This study aimed to introduce orthopedic surgeons to time-series analyses of patient activity data generated from a platform of sensors deployed in the homes of patients who have undergone primary total hip or knee replacement and understand the potential role of these data in postoperative clinical decision-making. Methods: Orthopedic surgeons and registrars were recruited through a combination of convenience and snowball sampling. Inclusion criteria were a minimum required experience in total joint replacement surgery specific to the hip or knee or familiarity with postoperative recovery assessment. Exclusion criteria included a lack of specific experience in the field. Of the 9 approached participants, 6 (67%) orthopedic surgeons and 3 (33%) registrars took part in either 1 of 3 focus groups or 1 of 2 interviews. Data were collected using an action-based approach in which stimulus materials (mock data visualizations) provided imaginative and creative interactions with the data. The data were analyzed using a thematic analysis approach. Results: Each data visualization was presented sequentially followed by a discussion of key illustrative commentary from participants, ending with a summary of key themes emerging across the focus group and interview data set. Conclusions: The limitations of the evidence are as follows. The data presented are from 1 English hospital. However, all data reflect the views of surgeons following standard national approaches and training. Although convenience sampling was used, participants’ background, skills, and experience were considered heterogeneous. Passively collected home monitoring data offered a real opportunity to more objectively characterize patients’ recovery from surgery. However, orthopedic surgeons highlighted the considerable difficulty in navigating large amounts of complex data within short medical consultations with patients. Orthopedic surgeons thought that a proposed dashboard presenting information and decision support alerts would fit best with existing clinical workflows. From this, the following guidelines for system design were developed: minimize the risk of misinterpreting data, express a level of confidence in the data, support clinicians in developing relevant skills as time-series data are often unfamiliar, and consider the impact of patient engagement with data in the future. International Registered Report Identifier (IRRID): RR2-10.1136/bmjopen-2018-021862 %M 37093626 %R 10.2196/36172 %U https://periop.jmir.org/2023/1/e36172 %U https://doi.org/10.2196/36172 %U http://www.ncbi.nlm.nih.gov/pubmed/37093626 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 9 %N %P e40622 %T The Impact of Digital Transformation on Inpatient Care: Mixed Methods Study %A Koebe,Philipp %A Bohnet-Joschko,Sabine %+ Faculty of Management, Economics and Society, Witten/Herdecke University, Alfred-Herrhausen-Str. 50, Witten, 58455, Germany, 49 17621951063, philipp.koebe@uni-wh.de %K digital transformation %K digitization %K health care provision %K hospital %K trends %D 2023 %7 21.4.2023 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: In the context of the digital transformation of all areas of society, health care providers are also under pressure to change. New technologies and a change in patients’ self-perception and health awareness require rethinking the provision of health care services. New technologies and the extensive use of data can change provision processes, optimize them, or replace them with new services. The inpatient sector, which accounts for a particularly large share of health care spending, plays a major role in this regard. Objective: This study examined the influences of current trends in digitization on inpatient service delivery. Methods: We conducted a scoping review. This was applied to identify the international trends in digital transformation as they relate to hospitals. Future trends were considered from different perspectives. Using the defined inclusion criteria, international peer-reviewed articles published between 2016 and 2021 were selected. The extracted core trends were then contextualized for the German hospital sector with 12 experts. Results: We included 44 articles in the literature analysis. From these, 8 core trends could be deduced. A heuristic impact model of the trends was derived from the data obtained and the experts’ assessments. This model provides a development corridor for the interaction of the trends with regard to technological intensity and supply quality. Trend accelerators and barriers were identified. Conclusions: The impact analysis showed the dependencies of a successful digital transformation in the hospital sector. Although data interoperability is of particular importance for technological intensity, the changed self-image of patients was shown to be decisive with regard to the quality of care. We show that hospitals must find their role in new digitally driven ecosystems, adapt their business models to customer expectations, and use up-to-date information and communications technologies. %M 37083473 %R 10.2196/40622 %U https://publichealth.jmir.org/2023/1/e40622 %U https://doi.org/10.2196/40622 %U http://www.ncbi.nlm.nih.gov/pubmed/37083473 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e44977 %T Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults %A Afshar,Majid %A Adelaine,Sabrina %A Resnik,Felice %A Mundt,Marlon P %A Long,John %A Leaf,Margaret %A Ampian,Theodore %A Wills,Graham J %A Schnapp,Benjamin %A Chao,Michael %A Brown,Randy %A Joyce,Cara %A Sharma,Brihat %A Dligach,Dmitriy %A Burnside,Elizabeth S %A Mahoney,Jane %A Churpek,Matthew M %A Patterson,Brian W %A Liao,Frank %+ University of Wisconsin - Madison, 1685 Highland Avenue, 5158 Medical Foundation Centennial Building, Madison, WI, 53705, United States, 1 3125459462, majid.afshar@wisc.edu %K clinical decision support %K natural language processing %K medical informatics %K opioid related disorder %K opioid use %K electronic health record %K clinical note %K cloud service %K artificial intelligence %K AI %D 2023 %7 20.4.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: The clinical narrative in electronic health records (EHRs) carries valuable information for predictive analytics; however, its free-text form is difficult to mine and analyze for clinical decision support (CDS). Large-scale clinical natural language processing (NLP) pipelines have focused on data warehouse applications for retrospective research efforts. There remains a paucity of evidence for implementing NLP pipelines at the bedside for health care delivery. Objective: We aimed to detail a hospital-wide, operational pipeline to implement a real-time NLP-driven CDS tool and describe a protocol for an implementation framework with a user-centered design of the CDS tool. Methods: The pipeline integrated a previously trained open-source convolutional neural network model for screening opioid misuse that leveraged EHR notes mapped to standardized medical vocabularies in the Unified Medical Language System. A sample of 100 adult encounters were reviewed by a physician informaticist for silent testing of the deep learning algorithm before deployment. An end user interview survey was developed to examine the user acceptability of a best practice alert (BPA) to provide the screening results with recommendations. The planned implementation also included a human-centered design with user feedback on the BPA, an implementation framework with cost-effectiveness, and a noninferiority patient outcome analysis plan. Results: The pipeline was a reproducible workflow with a shared pseudocode for a cloud service to ingest, process, and store clinical notes as Health Level 7 messages from a major EHR vendor in an elastic cloud computing environment. Feature engineering of the notes used an open-source NLP engine, and the features were fed into the deep learning algorithm, with the results returned as a BPA in the EHR. On-site silent testing of the deep learning algorithm demonstrated a sensitivity of 93% (95% CI 66%-99%) and specificity of 92% (95% CI 84%-96%), similar to published validation studies. Before deployment, approvals were received across hospital committees for inpatient operations. Five interviews were conducted; they informed the development of an educational flyer and further modified the BPA to exclude certain patients and allow the refusal of recommendations. The longest delay in pipeline development was because of cybersecurity approvals, especially because of the exchange of protected health information between the Microsoft (Microsoft Corp) and Epic (Epic Systems Corp) cloud vendors. In silent testing, the resultant pipeline provided a BPA to the bedside within minutes of a provider entering a note in the EHR. Conclusions: The components of the real-time NLP pipeline were detailed with open-source tools and pseudocode for other health systems to benchmark. The deployment of medical artificial intelligence systems in routine clinical care presents an important yet unfulfilled opportunity, and our protocol aimed to close the gap in the implementation of artificial intelligence–driven CDS. Trial Registration: ClinicalTrials.gov NCT05745480; https://www.clinicaltrials.gov/ct2/show/NCT05745480 %M 37079367 %R 10.2196/44977 %U https://medinform.jmir.org/2023/1/e44977 %U https://doi.org/10.2196/44977 %U http://www.ncbi.nlm.nih.gov/pubmed/37079367 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e44035 %T Information and Communications Technologies Enabling Integrated Primary Care for Patients With Complex Care Needs: Scoping Review %A Tahsin,Farah %A Armas,Alana %A Kirakalaprathapan,Apery %A Kadu,Mudathira %A Sritharan,Jasvinei %A Steele Gray,Carolyn %+ Institute of Health Policy, Management and Evaluation, University of Toronto, 155 College St, Toronto, ON, M5T 1P8, Canada, 1 647 825 4684, farah.tahsin@mail.utoronto.ca %K information and communications technology %K multimorbidity %K integrated care %K primary care %K review method %K scoping %K complex care %K mobile phone %D 2023 %7 19.4.2023 %9 Review %J J Med Internet Res %G English %X Background: Information and communications technologies (ICTs) are recognized as critical enablers of integrated primary care to support patients with multiple chronic conditions. Although ICT-enabled integrated primary care holds promise in supporting patients with complex care needs through team-based and continued care, critical implementation factors regarding what ICTs are available and how they enable this model are yet to be mapped in the literature. Objective: This scoping review addressed the current knowledge gap by answering the following research question: What ICTs are used in delivering integrated primary care to patients with complex care needs? Methods: The Arksey and O’Malley method enhanced by the work by Levac et al was used to guide this scoping review. In total, 4 electronic medical databases were accessed—MEDLINE, Embase, CINAHL, and PsycINFO—collecting studies published between January 2000 and December 2021. Identified peer-reviewed articles were screened. Relevant studies were charted, collated, and analyzed using the Rainbow Model of Integrated Care and the eHealth Enhanced Chronic Care Model. Results: A total of 52,216 articles were identified, of which 31 (0.06%) met the review’s eligibility criteria. In the current literature, ICTs are used to serve the following functions in the integrated primary care setting: information sharing, self-management support, clinical decision-making, and remote service delivery. Integration efforts are supported by ICTs by promoting teamwork and coordinating clinical services across teams and organizations. Patient, provider, organizational, and technological implementation factors are considered important for ICT-based interventions in the integrated primary care setting. Conclusions: ICTs play a critical role in enabling clinical and professional integration in the primary care setting to meet the health system–related needs of patients with complex care needs. Future research is needed to explore how to integrate technologies at an organizational and system level to create a health system that is well prepared to optimize technologies to support patients with complex care needs. %M 37074779 %R 10.2196/44035 %U https://www.jmir.org/2023/1/e44035 %U https://doi.org/10.2196/44035 %U http://www.ncbi.nlm.nih.gov/pubmed/37074779 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e44237 %T Data-Driven Identification of Unusual Prescribing Behavior: Analysis and Use of an Interactive Data Tool Using 6 Months of Primary Care Data From 6500 Practices in England %A Hopcroft,Lisa EM %A Massey,Jon %A Curtis,Helen J %A Mackenna,Brian %A Croker,Richard %A Brown,Andrew D %A O'Dwyer,Thomas %A Macdonald,Orla %A Evans,David %A Inglesby,Peter %A Bacon,Sebastian CJ %A Goldacre,Ben %A Walker,Alex J %+ Bennett Institute for Applied Data Science, Nuffield Department of Primary Care Health Sciences, University of Oxford, Radcliffe Primary Care Building, 32 Woodstock Road, Oxford, OX2 6GG, United Kingdom, 44 01865617855, alex.walker@phc.ox.ac.uk %K dashboard %K data science %K EHR %K electronic health records %K general practice %K outliers %K prescribing %K primary care %D 2023 %7 19.4.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: Approaches to addressing unwarranted variation in health care service delivery have traditionally relied on the prospective identification of activities and outcomes, based on a hypothesis, with subsequent reporting against defined measures. Practice-level prescribing data in England are made publicly available by the National Health Service (NHS) Business Services Authority for all general practices. There is an opportunity to adopt a more data-driven approach to capture variability and identify outliers by applying hypothesis-free, data-driven algorithms to national data sets. Objective: This study aimed to develop and apply a hypothesis-free algorithm to identify unusual prescribing behavior in primary care data at multiple administrative levels in the NHS in England and to visualize these results using organization-specific interactive dashboards, thereby demonstrating proof of concept for prioritization approaches. Methods: Here we report a new data-driven approach to quantify how “unusual” the prescribing rates of a particular chemical within an organization are as compared to peer organizations, over a period of 6 months (June-December 2021). This is followed by a ranking to identify which chemicals are the most notable outliers in each organization. These outlying chemicals are calculated for all practices, primary care networks, clinical commissioning groups, and sustainability and transformation partnerships in England. Our results are presented via organization-specific interactive dashboards, the iterative development of which has been informed by user feedback. Results: We developed interactive dashboards for every practice (n=6476) in England, highlighting the unusual prescribing of 2369 chemicals (dashboards are also provided for 42 sustainability and transformation partnerships, 106 clinical commissioning groups, and 1257 primary care networks). User feedback and internal review of case studies demonstrate that our methodology identifies prescribing behavior that sometimes warrants further investigation or is a known issue. Conclusions: Data-driven approaches have the potential to overcome existing biases with regard to the planning and execution of audits, interventions, and policy making within NHS organizations, potentially revealing new targets for improved health care service delivery. We present our dashboards as a proof of concept for generating candidate lists to aid expert users in their interpretation of prescribing data and prioritize further investigations and qualitative research in terms of potential targets for improved performance. %M 37074763 %R 10.2196/44237 %U https://medinform.jmir.org/2023/1/e44237 %U https://doi.org/10.2196/44237 %U http://www.ncbi.nlm.nih.gov/pubmed/37074763 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e46127 %T A SNOMED CT Mapping Guideline for the Local Terms Used to Document Clinical Findings and Procedures in Electronic Medical Records in South Korea: Methodological Study %A Sung,Sumi %A Park,Hyeoun-Ae %A Jung,Hyesil %A Kang,Hannah %+ College of Nursing, Seoul National University, 103 Daehak-ro, Jongno-gu, Seoul, 03080, Republic of Korea, 82 2 740 8827, hapark@snu.ac.kr %K semantic interoperability %K Systematized Nomenclature of Medicine–Clinical Terms %K mapping guideline %K local terms %K mapping %K guideline %K SNOMED %K nomenclature %K interoperable %K interoperability %K terminology %K medical term %K health term %K terminologies %K ontologies %D 2023 %7 18.4.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: South Korea joined SNOMED International as the 39th member country. To ensure semantic interoperability, South Korea introduced SNOMED CT (Systemized Nomenclature of Medicine–Clinical Terms) in 2020. However, there is no methodology to map local Korean terms to SNOMED CT. Instead, this is performed sporadically and independently at each local medical institution. The quality of the mapping, therefore, cannot be guaranteed. Objective: This study aimed to develop and introduce a guideline to map local Korean terms to the SNOMED CT used to document clinical findings and procedures in electronic health records at health care institutions in South Korea. Methods: The guidelines were developed from December 2020 to December 2022. An extensive literature review was conducted. The overall structures and contents of the guidelines with diverse use cases were developed by referencing the existing SNOMED CT mapping guidelines, previous studies related to SNOMED CT mapping, and the experiences of the committee members. The developed guidelines were validated by a guideline review panel. Results: The SNOMED CT mapping guidelines developed in this study recommended the following 9 steps: define the purpose and scope of the map, extract terms, preprocess source terms, preprocess source terms using clinical context, select a search term, use search strategies to find SNOMED CT concepts using a browser, classify mapping correlations, validate the map, and build the final map format. Conclusions: The guidelines developed in this study can support the standardized mapping of local Korean terms into SNOMED CT. Mapping specialists can use this guideline to improve the mapping quality performed at individual local medical institutions. %M 37071456 %R 10.2196/46127 %U https://medinform.jmir.org/2023/1/e46127 %U https://doi.org/10.2196/46127 %U http://www.ncbi.nlm.nih.gov/pubmed/37071456 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e41775 %T Comparison of Machine Learning Algorithms for Predicting Hospital Readmissions and Worsening Heart Failure Events in Patients With Heart Failure With Reduced Ejection Fraction: Modeling Study %A Ru,Boshu %A Tan,Xi %A Liu,Yu %A Kannapur,Kartik %A Ramanan,Dheepan %A Kessler,Garin %A Lautsch,Dominik %A Fonarow,Gregg %+ Ahmanson-UCLA Cardiomyopathy Center, University of California, Los Angeles, 10833 LeConte Avenue, Los Angeles, CA, 90095, United States, 1 310 206 9112, GFonarow@mednet.ucla.edu %K deep learning %K machine learning %K hospital readmission %K heart failure %K heart failure with reduced ejection fraction %K worsening heart failure event %K Bidirectional Encoder Representations From Transformers %K BERT %K clinical registry %K medical claims %K real-world data %D 2023 %7 17.4.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Heart failure (HF) is highly prevalent in the United States. Approximately one-third to one-half of HF cases are categorized as HF with reduced ejection fraction (HFrEF). Patients with HFrEF are at risk of worsening HF, have a high risk of adverse outcomes, and experience higher health care use and costs. Therefore, it is crucial to identify patients with HFrEF who are at high risk of subsequent events after HF hospitalization. Objective: Machine learning (ML) has been used to predict HF-related outcomes. The objective of this study was to compare different ML prediction models and feature construction methods to predict 30-, 90-, and 365-day hospital readmissions and worsening HF events (WHFEs). Methods: We used the Veradigm PINNACLE outpatient registry linked to Symphony Health’s Integrated Dataverse data from July 1, 2013, to September 30, 2017. Adults with a confirmed diagnosis of HFrEF and HF-related hospitalization were included. WHFEs were defined as HF-related hospitalizations or outpatient intravenous diuretic use within 1 year of the first HF hospitalization. We used different approaches to construct ML features from clinical codes, including frequencies of clinical classification software (CCS) categories, Bidirectional Encoder Representations From Transformers (BERT) trained with CCS sequences (BERT + CCS), BERT trained on raw clinical codes (BERT + raw), and prespecified features based on clinical knowledge. A multilayer perceptron neural network, extreme gradient boosting (XGBoost), random forest, and logistic regression prediction models were applied and compared. Results: A total of 30,687 adult patients with HFrEF were included in the analysis; 11.41% (3184/27,917) of adults experienced a hospital readmission within 30 days of their first HF hospitalization, and nearly half (9231/21,562, 42.81%) of the patients experienced at least 1 WHFE within 1 year after HF hospitalization. The prediction models and feature combinations with the best area under the receiver operating characteristic curve (AUC) for each outcome were XGBoost with CCS frequency (AUC=0.595) for 30-day readmission, random forest with CCS frequency (AUC=0.630) for 90-day readmission, XGBoost with CCS frequency (AUC=0.649) for 365-day readmission, and XGBoost with CCS frequency (AUC=0.640) for WHFEs. Our ML models could discriminate between readmission and WHFE among patients with HFrEF. Our model performance was mediocre, especially for the 30-day readmission events, most likely owing to limitations of the data, including an imbalance between positive and negative cases and high missing rates of many clinical variables and outcome definitions. Conclusions: We predicted readmissions and WHFEs after HF hospitalizations in patients with HFrEF. Features identified by data-driven approaches may be comparable with those identified by clinical domain knowledge. Future work may be warranted to validate and improve the models using more longitudinal electronic health records that are complete, are comprehensive, and have a longer follow-up time. %M 37067873 %R 10.2196/41775 %U https://formative.jmir.org/2023/1/e41775 %U https://doi.org/10.2196/41775 %U http://www.ncbi.nlm.nih.gov/pubmed/37067873 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e38159 %T Establishing Requirements for Technology to Support Clinical Trial Retention: Systematic Scoping Review and Analysis Using Self-determination Theory %A Gamble,Eoin %A Linehan,Conor %A Heavin,Ciara %+ School of Applied Psychology, University College Cork, N Mall, Kilbarry Enterprise Centre, Cork, , Ireland, 353 (0)21 490 4551, 118225493@umail.ucc.ie %K clinical trial %K clinical research %K retention strategies %K participant retention %K technology strategy %K decentralized clinical trial %K participant motivation %K patient centric %K engagement strategies %K self-determination theory %D 2023 %7 13.4.2023 %9 Review %J J Med Internet Res %G English %X Background: Retaining participants in clinical trials is an established challenge. Currently, the industry is moving to a technology-mediated, decentralized model for running trials. The shift presents an opportunity for technology design to aid the participant experience and promote retention; however, there are many open questions regarding how this can be best supported. We advocate the adoption of a stronger theoretical position to improve the quality of design decisions for clinical trial technology to promote participant engagement. Objective: This study aimed to identify and analyze the types of retention strategies used in published clinical trials that successfully retain participants. Methods: A systematic scoping review was carried out on 6 electronic databases for articles published from 1990 to September 2020, namely CINAHL, The Cochrane Library, EBSCO, Embase, PsycINFO, and PubMed, using the concepts “retention,” “strategy,” “clinal trial,” and “clinical research.” This was followed by an analysis of the included articles through the lens of self-determination theory, an evidence-based theory of human motivation. Results: A total of 26 articles were included in this review. The motivational strategies identified in the clinical trials in our sample were categorized into 8 themes: autonomy; competence; relatedness; controlled motivation; branding, communication material, and marketing literature; contact, tracking, and scheduling methods and data collection; convenience to contribute to data collection; and organizational competence. The trials used a wide range of motivational strategies. Notably, the trials often relied on controlled motivation interventions and underused strategies to support intrinsic motivation. Moreover, traditional clinical trials relied heavily on human interaction and “relatedness” to support motivation and retention, which may cause problems in the move to technology-led decentralized trials. We found inconsistency in the data-reporting methods and that motivational theory–based approaches were not evident in strategy design. Conclusions: This study offers direction and a framework to guide digital technology design decisions for future decentralized clinical trials to enhance participant retention during clinical trials. This research defines previous clinical trial retention strategies in terms of participant motivation, identifies motivational strategies, and offers a rationale for selecting strategies that will improve retention. It emphasizes the benefits of using theoretical frameworks to analyze strategic approaches and aid decision-making to improve the quality of technology design decisions. %M 37052985 %R 10.2196/38159 %U https://www.jmir.org/2023/1/e38159 %U https://doi.org/10.2196/38159 %U http://www.ncbi.nlm.nih.gov/pubmed/37052985 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e45002 %T Patients’ and Members of the Public’s Wishes Regarding Transparency in the Context of Secondary Use of Health Data: Scoping Review %A Cumyn,Annabelle %A Ménard,Jean-Frédéric %A Barton,Adrien %A Dault,Roxanne %A Lévesque,Frédérique %A Ethier,Jean-François %+ Groupe de recherche interdisciplinaire en informatique de la santé, Faculté des sciences/Faculté de médecine et des sciences de la santé, Université de Sherbrooke, 2500 boul. Université, Sherbrooke, QC, J1K 2R1, Canada, 1 819 346 1110 ext 74977, jean-francois.ethier@usherbrooke.ca %K transparency %K information %K means of communication %K public %K patients %K secondary use %K health data %K learning health systems %D 2023 %7 13.4.2023 %9 Review %J J Med Internet Res %G English %X Background: Secondary use of health data has reached unequaled potential to improve health systems governance, knowledge, and clinical care. Transparency regarding this secondary use is frequently cited as necessary to address deficits in trust and conditional support and to increase patient awareness. Objective: We aimed to review the current published literature to identify different stakeholders’ perspectives and recommendations on what information patients and members of the public want to learn about the secondary use of health data for research purposes and how and in which situations. Methods: Using PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines, we conducted a scoping review using Medline, CINAHL, PsycINFO, Scopus, Cochrane Library, and PubMed databases to locate a broad range of studies published in English or French until November 2022. We included articles reporting a stakeholder’s perspective or recommendations of what information patients and members of the public want to learn about the secondary use of health data for research purposes and how or in which situations. Data were collected and analyzed with an iterative thematic approach using NVivo. Results: Overall, 178 articles were included in this scoping review. The type of information can be divided into generic and specific content. Generic content includes information on governance and regulatory frameworks, technical aspects, and scientific aims. Specific content includes updates on the use of one’s data, return of results from individual tests, information on global results, information on data sharing, and how to access one’s data. Recommendations on how to communicate the information focused on frequency, use of various supports, formats, and wording. Methods for communication generally favored broad approaches such as nationwide publicity campaigns, mainstream and social media for generic content, and mixed approaches for specific content including websites, patient portals, and face-to-face encounters. Content should be tailored to the individual as much as possible with regard to length, avoidance of technical terms, cultural competence, and level of detail. Finally, the review outlined 4 major situations where communication was deemed necessary: before a new use of data, when new test results became available, when global research results were released, and in the advent of a breach in confidentiality. Conclusions: This review highlights how different types of information and approaches to communication efforts may serve as the basis for achieving greater transparency. Governing bodies could use the results: to elaborate or evaluate strategies to educate on the potential benefits; to provide some knowledge and control over data use as a form of reciprocity; and as a condition to engage citizens and build and maintain trust. Future work is needed to assess which strategies achieve the greatest outreach while striking a balance between meeting information needs and use of resources. %M 37052967 %R 10.2196/45002 %U https://www.jmir.org/2023/1/e45002 %U https://doi.org/10.2196/45002 %U http://www.ncbi.nlm.nih.gov/pubmed/37052967 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e38941 %T Using Clinical Data Visualizations in Electronic Health Record User Interfaces to Enhance Medical Student Diagnostic Reasoning: Randomized Experiment %A Cheng,Lucille %A Senathirajah,Yalini %+ School of Medicine, University of Pittsburgh, 3550 Terrace St, Pittsburgh, PA, 15213, United States, 1 (412) 624 5100, luc56@pitt.edu %K electronic health record %K EHR %K System-1–type diagnostic reasoning %K type-1 reasoning %K diagnostic %K diagnosis %K user interface %K user design %K heuristics %K medical education %K clinical reasoning %K reasoning process %K data visualization %K hGraph %K cognitive burden %K cognitive load %K medical student %K medical school %D 2023 %7 13.4.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: In medicine, the clinical decision-making process can be described using the dual-process theory consisting of the fast, intuitive “System 1,” commonly seen in seasoned physicians, and the slow, deliberative “System 2,” associated with medical students. System-1—type diagnostic reasoning is thought to be less cognitively burdensome, thereby reducing physician error. To date, limited literature exists on inducing System-1–type diagnosis in medical students through cognitive heuristics, particularly while using modern electronic health record (EHR) interfaces. Objective: In this experimental pilot study, we aimed to (1) attempt to induce System-1—type diagnostic reasoning in inexperienced medical students through the acquisition of cognitive user interface heuristics and (2) understand the impact of clinical patient data visualizations on students' cognitive load and medical education. Methods: The participants were third- and fourth-year medical students recruited from the University of Pittsburgh School of Medicine who had completed 1+ clinical rotations. The students were presented 8 patient cases on a novel EHR, featuring a prominent data visualization designed to foster at-a-glance rapid case assessment, and asked to diagnose the patient. Half of the participants were shown 4 of the 8 cases repeatedly, up to 4 times with 30 seconds per case (Group A), and the other half of the participants were shown cases twice with 2 minutes per case (Group B). All participants were then asked to provide full diagnoses of all 8 cases. Finally, the participants were asked to evaluate and elaborate on their experience with the system; content analysis was subsequently performed on these user experience interviews. Results: A total of 15 students participated. The participants in Group A scored slightly higher on average than those in Group B, with a mean percentage correct of 76% (95% CI 0.68-0.84) versus 69% (95% CI 0.58-0.80), and spent on average 50% less time per question than Group B diagnosing patients (13.98 seconds vs 19.13 seconds, P=.03, respectively). When comparing the novel EHR design to previously used EHRs, 73% (n=11) of participants rated the new version on par or higher (3+/5). Ease of use and intuitiveness of this new system rated similarly high (mean score 3.73/5 and 4.2/5, respectively). In qualitative thematic analysis of poststudy interviews, most participants (n=11, 73%) spoke to “pattern-recognition” cognitive heuristic strategies consistent with System 1 decision-making. Conclusions: These results support the possibility of inducing type-1 diagnostics in learners and the potential for data visualization and user design heuristics to reduce cognitive burden in clinical settings. Clinical data presentation in the diagnostic reasoning process is ripe for innovation, and further research is needed to explore the benefit of using such visualizations in medical education. %M 37053000 %R 10.2196/38941 %U https://humanfactors.jmir.org/2023/1/e38941 %U https://doi.org/10.2196/38941 %U http://www.ncbi.nlm.nih.gov/pubmed/37053000 %0 Journal Article %@ 1929-073X %I JMIR Publications %V 12 %N %P e42685 %T Evolution of Health Information Sharing Between Health Care Organizations: Potential of Nonfungible Tokens %A Esmaeilzadeh,Pouyan %+ Department of Information Systems and Business Analytics, College of Business, Florida International University, Modesto A Maidique Campus, 11200 SW 8th St, RB 261B, Miami, FL, 33199, United States, 1 305 348 3302, pesmaeil@fiu.edu %K health information exchange %K HIE %K personal health information %K PHI %K blockchain %K nonfungible token %K NFT %K evolution of technology %D 2023 %7 12.4.2023 %9 Viewpoint %J Interact J Med Res %G English %X This study attempts to explain the development and progress of the technology used for sharing health information across health care organizations (such as hospitals and physicians’ offices). First, we describe the strengths and weaknesses of traditional sharing models, health information exchange (HIE), and blockchain-based HIE. Second, the potential use of nonfungible token (NFT) protocols in HIE models is proposed as the next possible move for information-sharing initiatives in health care. In addition to some potential opportunities and distinguishing features (eg, ownability, verifiability, and incentivization), we identify the uncertainty and risks associated with the application of NFTs, such as the lack of a dedicated regulatory framework for legal ownership of digital patient data. This paper is among the first to discuss the potential of NFTs in health care. The use of NFTs in HIE networks could generate a new stream of research for future studies. This study provides practical insights into how the technological foundations of information-sharing efforts in health care have developed and diversified from earlier forms. %M 37043269 %R 10.2196/42685 %U https://www.i-jmr.org/2023/1/e42685 %U https://doi.org/10.2196/42685 %U http://www.ncbi.nlm.nih.gov/pubmed/37043269 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e41223 %T Novel Note Templates to Enhance Signal and Reduce Noise in Medical Documentation: Prospective Improvement Study %A Feldman,Jonah %A Goodman,Adam %A Hochman,Katherine %A Chakravartty,Eesha %A Austrian,Jonathan %A Iturrate,Eduardo %A Bosworth,Brian %A Saxena,Archana %A Moussa,Marwa %A Chenouda,Dina %A Volpicelli,Frank %A Adler,Nicole %A Weisstuch,Joseph %A Testa,Paul %+ Department of Medicine, NYU Langone Health, 550 1st avenue, New York, NY, 10016, United States, 1 (212) 263 5800, eesha.chakravartty@nyulangone.org %K medical informatics %K decision support %K hospital data %K clinical documentation %K clinical informatics %D 2023 %7 12.4.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: The introduction of electronic workflows has allowed for the flow of raw uncontextualized clinical data into medical documentation. As a result, many electronic notes have become replete of “noise” and deplete clinically significant “signals.” There is an urgent need to develop and implement innovative approaches in electronic clinical documentation that improve note quality and reduce unnecessary bloating. Objective: This study aims to describe the development and impact of a novel set of templates designed to change the flow of information in medical documentation. Methods: This is a multihospital nonrandomized prospective improvement study conducted on the inpatient general internal medicine service across 3 hospital campuses at the New York University Langone Health System. A group of physician leaders representing each campus met biweekly for 6 months. The output of these meetings included (1) a conceptualization of the note bloat problem as a dysfunction in information flow, (2) a set of guiding principles for organizational documentation improvement, (3) the design and build of novel electronic templates that reduced the flow of extraneous information into provider notes by providing link outs to best practice data visualizations, and (4) a documentation improvement curriculum for inpatient medicine providers. Prior to go-live, pragmatic usability testing was performed with the new progress note template, and the overall user experience was measured using the System Usability Scale (SUS). Primary outcome measures after go-live include template utilization rate and note length in characters. Results: In usability testing among 22 medicine providers, the new progress note template averaged a usability score of 90.6 out of 100 on the SUS. A total of 77% (17/22) of providers strongly agreed that the new template was easy to use, and 64% (14/22) strongly agreed that they would like to use the template frequently. In the 3 months after template implementation, general internal medicine providers wrote 67% (51,431/76,647) of all inpatient notes with the new templates. During this period, the organization saw a 46% (2768/6191), 47% (3505/7819), and 32% (3427/11,226) reduction in note length for general medicine progress notes, consults, and history and physical notes, respectively, when compared to a baseline measurement period prior to interventions. Conclusions: A bundled intervention that included the deployment of novel templates for inpatient general medicine providers significantly reduced average note length on the clinical service. Templates designed to reduce the flow of extraneous information into provider notes performed well during usability testing, and these templates were rapidly adopted across all hospital campuses. Further research is needed to assess the impact of novel templates on note quality, provider efficiency, and patient outcomes. %M 36821760 %R 10.2196/41223 %U https://formative.jmir.org/2023/1/e41223 %U https://doi.org/10.2196/41223 %U http://www.ncbi.nlm.nih.gov/pubmed/36821760 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e43127 %T Asymmetric Interoperability as a Strategy Among Provider Group Health Information Exchange: Directional Analysis %A Shah,Rohin Rathin %A Bailey,Joseph Peter %+ The Robert H. Smith School of Business, University of Maryland, 7699 Mowatt Ln, College Park, MD, 20742-1815, United States, 1 301 405 2174, jpbailey@umd.edu %K health information exchange %K quality payment program %K electronic health records %K electronic referrals %K medical informatics %K technology adoption %K health information interoperability %D 2023 %7 6.4.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: High levels of seamless, bidirectional health information exchange continue to be broadly limited among provider groups despite the vast array of benefits that interoperability entails for patient care and the many persistent efforts across the health care ecosystem directed at advancing interoperability. As provider groups seek to act in their strategic best interests, they are often interoperable and exchange information in certain directions but not others, leading to the formation of asymmetries. Objective: We aimed to examine the correlation at the provider group level between the distinct directions of interoperability with regard to sending health information and receiving health information, to describe how this correlation varies across provider group types and provider group sizes, and to analyze the symmetries and asymmetries that arise in the exchange of patient health information across the health care ecosystem as a result. Methods: We used data from the Centers for Medicare & Medicaid Services (CMS), which included interoperability performance information for 2033 provider groups within the Quality Payment Program Merit-based Incentive Payment System and maintained distinct performance measures for sending health information and receiving health information. In addition to compiling descriptive statistics, we also conducted a cluster analysis to identify differences among provider groups—particularly with respect to symmetric versus asymmetric interoperability. Results: We found that the examined directions of interoperability—sending health information and receiving health information—have relatively low bivariate correlation (0.4147) with a significant number of observations exhibiting asymmetric interoperability (42.5%). Primary care providers are generally more likely to exchange information asymmetrically than specialty providers, being more inclined to receive health information than to send health information. Finally, we found that larger provider groups are significantly less likely to be bidirectionally interoperable than smaller groups, although both are asymmetrically interoperable at similar rates. Conclusions: The adoption of interoperability by provider groups is more nuanced than traditionally considered and should not be seen as a binary determination (ie, to be interoperable or not). Asymmetric interoperability—and its pervasive presence among provider groups—reiterates how the manner in which provider groups exchange patient health information is a strategic choice and may pose similar implications and potential harms as the practice of information blocking has in the past. Differences in the operational paradigms among provider groups of varying types and sizes may explain their varying extents of health information exchange for sending and receiving health information. There continues to remain substantial room for improvement on the path to achieving a fully interoperable health care ecosystem, and future policy efforts directed at advancing interoperability should consider the practice of being asymmetrically interoperable among provider groups. %M 37023418 %R 10.2196/43127 %U https://www.jmir.org/2023/1/e43127 %U https://doi.org/10.2196/43127 %U http://www.ncbi.nlm.nih.gov/pubmed/37023418 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e47695 %T “To Err Is Evolution”: We Need the Implementation Report to Learn %A Perrin Franck,Caroline %A Geissbuhler,Antoine %A Lovis,Christian %+ Campus Biotech, University of Geneva, Chemin des Mines 9, Geneva, 1202, Switzerland, 41 787997725, caroline.perrin@unige.ch %K implementation science %K knowledge management %K knowledge sharing %K digital health %K implementation report %D 2023 %7 4.4.2023 %9 Editorial %J JMIR Med Inform %G English %X JMIR Medical Informatics is pleased to offer implementation reports as a new article type. Implementation reports present real-world accounts of the implementation of health technologies and clinical interventions. This new article type is intended to promote the rapid documentation and dissemination of the perspectives and experiences of those involved in implementing digital health interventions and assessing the effectiveness of digital health projects. %M 37014675 %R 10.2196/47695 %U https://medinform.jmir.org/2023/1/e47695 %U https://doi.org/10.2196/47695 %U http://www.ncbi.nlm.nih.gov/pubmed/37014675 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e42615 %T Digital Health Data Quality Issues: Systematic Review %A Syed,Rehan %A Eden,Rebekah %A Makasi,Tendai %A Chukwudi,Ignatius %A Mamudu,Azumah %A Kamalpour,Mostafa %A Kapugama Geeganage,Dakshi %A Sadeghianasl,Sareh %A Leemans,Sander J J %A Goel,Kanika %A Andrews,Robert %A Wynn,Moe Thandar %A ter Hofstede,Arthur %A Myers,Trina %+ School of Information Systems, Faculty of Science, Queensland University of Technology, 2 George Street, Brisbane, 4000, Australia, 61 7 3138 9360, r.syed@qut.edu.au %K data quality %K digital health %K electronic health record %K eHealth %K systematic reviews %D 2023 %7 31.3.2023 %9 Review %J J Med Internet Res %G English %X Background: The promise of digital health is principally dependent on the ability to electronically capture data that can be analyzed to improve decision-making. However, the ability to effectively harness data has proven elusive, largely because of the quality of the data captured. Despite the importance of data quality (DQ), an agreed-upon DQ taxonomy evades literature. When consolidated frameworks are developed, the dimensions are often fragmented, without consideration of the interrelationships among the dimensions or their resultant impact. Objective: The aim of this study was to develop a consolidated digital health DQ dimension and outcome (DQ-DO) framework to provide insights into 3 research questions: What are the dimensions of digital health DQ? How are the dimensions of digital health DQ related? and What are the impacts of digital health DQ? Methods: Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, a developmental systematic literature review was conducted of peer-reviewed literature focusing on digital health DQ in predominately hospital settings. A total of 227 relevant articles were retrieved and inductively analyzed to identify digital health DQ dimensions and outcomes. The inductive analysis was performed through open coding, constant comparison, and card sorting with subject matter experts to identify digital health DQ dimensions and digital health DQ outcomes. Subsequently, a computer-assisted analysis was performed and verified by DQ experts to identify the interrelationships among the DQ dimensions and relationships between DQ dimensions and outcomes. The analysis resulted in the development of the DQ-DO framework. Results: The digital health DQ-DO framework consists of 6 dimensions of DQ, namely accessibility, accuracy, completeness, consistency, contextual validity, and currency; interrelationships among the dimensions of digital health DQ, with consistency being the most influential dimension impacting all other digital health DQ dimensions; 5 digital health DQ outcomes, namely clinical, clinician, research-related, business process, and organizational outcomes; and relationships between the digital health DQ dimensions and DQ outcomes, with the consistency and accessibility dimensions impacting all DQ outcomes. Conclusions: The DQ-DO framework developed in this study demonstrates the complexity of digital health DQ and the necessity for reducing digital health DQ issues. The framework further provides health care executives with holistic insights into DQ issues and resultant outcomes, which can help them prioritize which DQ-related problems to tackle first. %M 37000497 %R 10.2196/42615 %U https://www.jmir.org/2023/1/e42615 %U https://doi.org/10.2196/42615 %U http://www.ncbi.nlm.nih.gov/pubmed/37000497 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e42330 %T A Description of Personal Health Information Management Work With a Spotlight on the Practices of Older Adults: Qualitative e-Delphi Study With Professional Organizers %A Seale,Deborah E %A LeRouge,Cynthia M %A Kolotylo-Kulkarni,Malgorzata %+ Department of Information Management & Business Analytics, Zimpleman College of Business, Drake University, 356 Aliber Hall, 2507 University Ave, Des Moines, IA, 50311, United States, 1 5152712007, malgorzata.kolotylo-kulkarni@drake.edu %K patient work system %K consumer health informatics %K personal health information management %K PHIM %K patient participation %K medical informatics %K information management %D 2023 %7 31.3.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Personal health information (PHI) is created on behalf of and by health care consumers to support their care and wellness. Available tools designed to support PHI management (PHIM) remain insufficient. A comprehensive understanding of PHIM work is required, particularly for older adults, to offer more effective PHIM tools and support. Objective: The primary objective of this study was to use the Patient Work System model to provide a holistic description of PHIM work from the perspective of professional organizers with experience assisting health care consumers, including older adults, in managing their PHI. A secondary objective was to examine how factors associated with 4 Patient Work System components (person, tasks, tools and technologies, and context) interact to support or compromise PHIM work performance. Methods: A modified e‐Delphi methodology was used to complete 3 web-based rounds of open-ended questions and obtain consensus among a panel of 16 experts in professional organizing. Data were collected between April and December 2017. The Patient Work System model was used as a coding schema and guided the interpretation of findings during the analysis. Results: The PHIM work of adults who sought assistance focused on the tasks of acquiring, organizing, and storing 3 classifications of PHI (medical, financial, and reference) and then processing, reconciling, and storing the medical and financial classifications to tend to their health, health care, and health finances. We also found that the complexities of PHI and PHIM-related work often exceeded the abilities and willingness of those who sought assistance. A total of 6 factors contributed to the complexity of PHIM work. The misalignment of these factors was found to increase the PHIM workload, particularly for older adults. The life changes that often accompanied aging, coupled with obscure and fragmented health care provider- and insurer-generated PHI, created the need for much PHIM work. Acquiring and integrating obscure and fragmented PHI, detecting and reconciling PHI discrepancies, and protecting PHI held by health care consumers were among the most burdensome tasks, especially for older adults. Consequently, personal stakeholders (paid and unpaid) were called upon or voluntarily stepped in to assist with PHIM work. Conclusions: Streamlining and automating 2 of the most common and burdensome PHIM undertakings could drastically reduce health care consumers’ PHIM workload: developing and maintaining accurate current and past health summaries and tracking medical bills and insurance claims to reconcile discrepancies. Other improvements that hold promise are the simplification and standardization of commonly used financial and medical PHI; standardization and automation of commonly used PHI acquisition interfaces; and provision of secure, Health Insurance Portability and Accountability Act (HIPAA)–certified PHI tools and technologies that control multiperson access for PHI stored by health care consumers in electronic and paper formats. %M 37000478 %R 10.2196/42330 %U https://www.jmir.org/2023/1/e42330 %U https://doi.org/10.2196/42330 %U http://www.ncbi.nlm.nih.gov/pubmed/37000478 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e42452 %T Real-Time Prediction of Sepsis in Critical Trauma Patients: Machine Learning–Based Modeling Study %A Li,Jiang %A Xi,Fengchan %A Yu,Wenkui %A Sun,Chuanrui %A Wang,Xiling %+ School of Public Health and Key Laboratory of Public Health Safety, Fudan University, No 130 Dongan Road, Xuhui District, Shanghai, 200032, China, 86 021 54237051, erinwang@fudan.edu.cn %K sepsis %K trauma %K intensive care unit %K machine learning %K real-time prediction %D 2023 %7 31.3.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Sepsis is a leading cause of death in patients with trauma, and the risk of mortality increases significantly for each hour of delay in treatment. A hypermetabolic baseline and explosive inflammatory immune response mask clinical signs and symptoms of sepsis in trauma patients, making early diagnosis of sepsis more challenging. Machine learning–based predictive modeling has shown great promise in evaluating and predicting sepsis risk in the general intensive care unit (ICU) setting, but there has been no sepsis prediction model specifically developed for trauma patients so far. Objective: To develop a machine learning model to predict the risk of sepsis at an hourly scale among ICU-admitted trauma patients. Methods: We extracted data from adult trauma patients admitted to the ICU at Beth Israel Deaconess Medical Center between 2008 and 2019. A total of 42 raw variables were collected, including demographics, vital signs, arterial blood gas, and laboratory tests. We further derived a total of 485 features, including measurement pattern features, scoring features, and time-series variables, from the raw variables by feature engineering. The data set was randomly split into 70% for model development with stratified 5-fold cross-validation, 15% for calibration, and 15% for testing. An Extreme Gradient Boosting (XGBoost) model was developed to predict the hourly risk of sepsis at prediction windows of 4, 6, 8, 12, and 24 hours. We evaluated model performance for discrimination and calibration both at time-step and outcome levels. Clinical applicability of the model was evaluated with varying levels of precision, and the potential clinical net benefit was assessed with decision curve analysis (DCA). A Shapley additive explanation algorithm was applied to show the effect of features on the prediction model. In addition, we trained an L2-regularized logistic regression model to compare its performance with XGBoost. Results: We included 4603 trauma patients in the study, 1196 (26%) of whom developed sepsis. The XGBoost model achieved an area under the receiver operating characteristics curve (AUROC) ranging from 0.83 to 0.88 at the 4-to-24-hour prediction window in the test set. With a ratio of 9 false alerts for every true alert, it predicted 73% (386/529) of sepsis-positive timesteps and 91% (163/179) of sepsis events in the subsequent 6 hours. The DCA showed our model had a positive net benefit in the threshold probability range of 0 to 0.6. In comparison, the logistic regression model achieved lower performance, with AUROC ranging from 0.76 to 0.84 at the 4-to-24-hour prediction window. Conclusions: The machine learning–based model had good discrimination and calibration performance for sepsis prediction in critical trauma patients. Using the model in clinical practice might help to identify patients at risk of sepsis in a time window that enables personalized intervention and early treatment. %M 37000488 %R 10.2196/42452 %U https://formative.jmir.org/2023/1/e42452 %U https://doi.org/10.2196/42452 %U http://www.ncbi.nlm.nih.gov/pubmed/37000488 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e41588 %T Federated Machine Learning, Privacy-Enhancing Technologies, and Data Protection Laws in Medical Research: Scoping Review %A Brauneck,Alissa %A Schmalhorst,Louisa %A Kazemi Majdabadi,Mohammad Mahdi %A Bakhtiari,Mohammad %A Völker,Uwe %A Baumbach,Jan %A Baumbach,Linda %A Buchholtz,Gabriele %+ Hamburg University Faculty of Law, University of Hamburg, Rothenbaumchaussee 33, Hamburg, 20148, Germany, 49 40 42838 2328, alissa.brauneck@uni-hamburg.de %K federated learning %K data protection regulation %K data protection by design %K privacy protection %K General Data Protection Regulation compliance %K GDPR compliance %K privacy-preserving technologies %K differential privacy %K secure multiparty computation %D 2023 %7 30.3.2023 %9 Review %J J Med Internet Res %G English %X Background: The collection, storage, and analysis of large data sets are relevant in many sectors. Especially in the medical field, the processing of patient data promises great progress in personalized health care. However, it is strictly regulated, such as by the General Data Protection Regulation (GDPR). These regulations mandate strict data security and data protection and, thus, create major challenges for collecting and using large data sets. Technologies such as federated learning (FL), especially paired with differential privacy (DP) and secure multiparty computation (SMPC), aim to solve these challenges. Objective: This scoping review aimed to summarize the current discussion on the legal questions and concerns related to FL systems in medical research. We were particularly interested in whether and to what extent FL applications and training processes are compliant with the GDPR data protection law and whether the use of the aforementioned privacy-enhancing technologies (DP and SMPC) affects this legal compliance. We placed special emphasis on the consequences for medical research and development. Methods: We performed a scoping review according to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews). We reviewed articles on Beck-Online, SSRN, ScienceDirect, arXiv, and Google Scholar published in German or English between 2016 and 2022. We examined 4 questions: whether local and global models are “personal data” as per the GDPR; what the “roles” as defined by the GDPR of various parties in FL are; who controls the data at various stages of the training process; and how, if at all, the use of privacy-enhancing technologies affects these findings. Results: We identified and summarized the findings of 56 relevant publications on FL. Local and likely also global models constitute personal data according to the GDPR. FL strengthens data protection but is still vulnerable to a number of attacks and the possibility of data leakage. These concerns can be successfully addressed through the privacy-enhancing technologies SMPC and DP. Conclusions: Combining FL with SMPC and DP is necessary to fulfill the legal data protection requirements (GDPR) in medical research dealing with personal data. Even though some technical and legal challenges remain, for example, the possibility of successful attacks on the system, combining FL with SMPC and DP creates enough security to satisfy the legal requirements of the GDPR. This combination thereby provides an attractive technical solution for health institutions willing to collaborate without exposing their data to risk. From a legal perspective, the combination provides enough built-in security measures to satisfy data protection requirements, and from a technical perspective, the combination provides secure systems with comparable performance with centralized machine learning applications. %M 36995759 %R 10.2196/41588 %U https://www.jmir.org/2023/1/e41588 %U https://doi.org/10.2196/41588 %U http://www.ncbi.nlm.nih.gov/pubmed/36995759 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e41082 %T Turning When Using Smartphone in Persons With and Those Without Neurologic Conditions: Observational Study %A Bianchini,Edoardo %A Warmerdam,Elke %A Romijnders,Robbin %A Stürner,Klarissa Hanja %A Baron,Ralf %A Heinzel,Sebastian %A Pontieri,Francesco Ernesto %A Hansen,Clint %A Maetzler,Walter %+ Department of Neurology, Kiel University, Klinik für Neurologie UKSH, Campus Kiel, Arnold-Heller-Strasse 3, Haus D, Kiel, 24105, Germany, 49 431500 23981, w.maetzler@neurologie.uni-kiel.de %K turning %K turning coordination %K smartphone %K dual task %K dual task cost %K Parkinson disease %K Parkinson %K stroke %K multiple sclerosis %K low back pain %K neurology %K neurological %K movement %K biomechanics %K gait %K balance %K walk %K kinesiology %K fall %D 2023 %7 30.3.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Turning during walking is a relevant and common everyday movement and it depends on a correct top-down intersegmental coordination. This could be reduced in several conditions (en bloc turning), and an altered turning kinematics has been linked to increased risk of falls. Smartphone use has been associated with poorer balance and gait; however, its effect on turning-while-walking has not been investigated yet. This study explores turning intersegmental coordination during smartphone use in different age groups and neurologic conditions. Objective: This study aims to evaluate the effect of smartphone use on turning behavior in healthy individuals of different ages and those with various neurological diseases. Methods: Younger (aged 18-60 years) and older (aged >60 years) healthy individuals and those with Parkinson disease, multiple sclerosis, subacute stroke (<4 weeks), or lower-back pain performed turning-while-walking alone (single task [ST]) and while performing 2 different cognitive tasks of increasing complexity (dual task [DT]). The mobility task consisted of walking up and down a 5-m walkway at self-selected speed, thus including 180° turns. Cognitive tasks consisted of a simple reaction time test (simple DT [SDT]) and a numerical Stroop test (complex DT [CDT]). General (turn duration and the number of steps while turning), segmental (peak angular velocity), and intersegmental turning parameters (intersegmental turning onset latency and maximum intersegmental angle) were extracted for head, sternum, and pelvis using a motion capture system and a turning detection algorithm. Results: In total, 121 participants were enrolled. All participants, irrespective of age and neurologic disease, showed a reduced intersegmental turning onset latency and a reduced maximum intersegmental angle of both pelvis and sternum relative to head, thus indicating an en bloc turning behavior when using a smartphone. With regard to change from the ST to turning when using a smartphone, participants with Parkinson disease reduced their peak angular velocity the most, which was significantly different from lower-back pain relative to the head (P<.01). Participants with stroke showed en bloc turning already without smartphone use. Conclusions: Smartphone use during turning-while-walking may lead to en bloc turning and thus increase fall risk across age and neurologic disease groups. This behavior is probably particularly dangerous for those groups with the most pronounced changes in turning parameters during smartphone use and the highest fall risk, such as individuals with Parkinson disease. Moreover, the experimental paradigm presented here might be useful in differentiating individuals with lower-back pain without and those with early or prodromal Parkinson disease. In individuals with subacute stroke, en bloc turning could represent a compensative strategy to overcome the newly occurring mobility deficit. Considering the ubiquitous smartphone use in daily life, this study should stimulate future studies in the area of fall risk and neurological and orthopedic diseases. Trial Registration: German Clinical Trials Register DRKS00022998; https://drks.de/search/en/trial/DRKS00022998 %M 36995756 %R 10.2196/41082 %U https://www.jmir.org/2023/1/e41082 %U https://doi.org/10.2196/41082 %U http://www.ncbi.nlm.nih.gov/pubmed/36995756 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e46700 %T mHealth Systems Need a Privacy-by-Design Approach: Commentary on “Federated Machine Learning, Privacy-Enhancing Technologies, and Data Protection Laws in Medical Research: Scoping Review” %A Tewari,Ambuj %+ Department of Statistics, University of Michigan, 1085 S University Ave, Ann Arbor, MI, 48109-1107, United States, 1 734 615 0928, tewaria@umich.edu %K mHealth %K differential privacy %K private synthetic data %K federated learning %K data protection regulation %K data protection by design %K privacy protection %K General Data Protection Regulation %K GDPR compliance %K privacy-preserving technologies %K secure multiparty computation %K multiparty computation %K machine learning %K privacy %D 2023 %7 30.3.2023 %9 Commentary %J J Med Internet Res %G English %X Brauneck and colleagues have combined technical and legal perspectives in their timely and valuable paper “Federated Machine Learning, Privacy-Enhancing Technologies, and Data Protection Laws in Medical Research: Scoping Review.” Researchers who design mobile health (mHealth) systems must adopt the same privacy-by-design approach that privacy regulations (eg, General Data Protection Regulation) do. In order to do this successfully, we will have to overcome implementation challenges in privacy-enhancing technologies such as differential privacy. We will also have to pay close attention to emerging technologies such as private synthetic data generation. %M 36995757 %R 10.2196/46700 %U https://www.jmir.org/2023/1/e46700 %U https://doi.org/10.2196/46700 %U http://www.ncbi.nlm.nih.gov/pubmed/36995757 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e43277 %T No-show Prediction Model Performance Among People With HIV: External Validation Study %A Mason,Joseph A %A Friedman,Eleanor E %A Rojas,Juan C %A Ridgway,Jessica P %+ The Chicago Center for HIV Elimination, Department of Medicine, University of Chicago, 5841 S Maryland Ave, Chicago, IL, 60637, United States, 1 773 702 9586, joemason@bsd.uchicago.edu %K no-show %K prediction model %K Epic systems %K people with HIV %K human immunodeficiency virus %K electronic medical record %K external validation %K technology %K model %K care %K patient %K HIV %D 2023 %7 29.3.2023 %9 Short Paper %J J Med Internet Res %G English %X Background: Regular medical care is important for people living with HIV. A no-show predictive model among people with HIV could improve clinical care by allowing providers to proactively engage patients at high risk of missing appointments. Epic, a major provider of electronic medical record systems, created a model that predicts a patient’s probability of being a no-show for an outpatient health care appointment; however, this model has not been externally validated in people with HIV. Objective: We examined the performance of Epic’s no-show model among people with HIV at an academic medical center and assessed whether the performance was impacted by the addition of demographic and HIV clinical information. Methods: We obtained encounter data from all in-person appointments among people with HIV from January 21 to March 30, 2022, at the University of Chicago Medicine. We compared the predicted no-show probability at the time of the encounter to the actual outcome of these appointments. We also examined the performance of the Epic model among people with HIV for only HIV care appointments in the infectious diseases department. We further compared the no-show model among people with HIV for HIV care appointments to an alternate random forest model we created using a subset of seven readily accessible features used in the Epic model and four additional features related to HIV clinical care or demographics. Results: We identified 674 people with HIV who contributed 1406 total scheduled in-person appointments during the study period. Of those, we identified 331 people with HIV who contributed 440 HIV care appointments. The performance of the Epic model among people with HIV for all appointments in any outpatient clinic had an area under the receiver operating characteristic curve (AUC) of 0.65 (95% CI 0.63-0.66) and for only HIV care appointments had an AUC of 0.63 (95% CI 0.59-0.67). The alternate model we created for people with HIV attending HIV care appointments had an AUC of 0.78 (95% CI 0.75-0.82), a significant improvement over the Epic model restricted to HIV care appointments (P<.001). Features identified as important in the alternate model included lead time, appointment length, HIV viral load >200 copies per mL, lower CD4 T cell counts (both 50 to <200 cells/mm3 and 200 to <350 cells/mm3), and female sex. Conclusions: For both models among people with HIV, performance was significantly lower than reported by Epic. The improvement in the performance of the alternate model over the proprietary Epic model demonstrates that, among people with HIV, the inclusion of demographic information may enhance the prediction of appointment attendance. The alternate model further reveals that the prediction of appointment attendance in people with HIV can be improved by using HIV clinical information such as CD4 count and HIV viral load test results as features in the model. %M 36989038 %R 10.2196/43277 %U https://www.jmir.org/2023/1/e43277 %U https://doi.org/10.2196/43277 %U http://www.ncbi.nlm.nih.gov/pubmed/36989038 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e39972 %T Machine Learning Approaches for Predicting Psoriatic Arthritis Risk Using Electronic Medical Records: Population-Based Study %A Lee,Leon Tsung-Ju %A Yang,Hsuan-Chia %A Nguyen,Phung Anh %A Muhtar,Muhammad Solihuddin %A Li,Yu-Chuan Jack %+ Department of Dermatology, Taipei Municipal Wanfang Hospital, Taipei Medical University, No 111, Section 3, Hsing-Long Rd, Taipei, 116, Taiwan, 886 02 2930 7930, jack@tmu.edu.tw %K convolutional neural network %K deep learning, machine learning %K prediction model %K psoriasis %K psoriatic arthritis %K temporal phenomic map %K electronic medical records %D 2023 %7 28.3.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Psoriasis (PsO) is a chronic, systemic, immune-mediated disease with multiorgan involvement. Psoriatic arthritis (PsA) is an inflammatory arthritis that is present in 6%-42% of patients with PsO. Approximately 15% of patients with PsO have undiagnosed PsA. Predicting patients with a risk of PsA is crucial for providing them with early examination and treatment that can prevent irreversible disease progression and function loss. Objective: The aim of this study was to develop and validate a prediction model for PsA based on chronological large-scale and multidimensional electronic medical records using a machine learning algorithm. Methods: This case-control study used Taiwan’s National Health Insurance Research Database from January 1, 1999, to December 31, 2013. The original data set was split into training and holdout data sets in an 80:20 ratio. A convolutional neural network was used to develop a prediction model. This model used 2.5-year diagnostic and medical records (inpatient and outpatient) with temporal-sequential information to predict the risk of PsA for a given patient within the next 6 months. The model was developed and cross-validated using the training data and was tested using the holdout data. An occlusion sensitivity analysis was performed to identify the important features of the model. Results: The prediction model included a total of 443 patients with PsA with earlier diagnosis of PsO and 1772 patients with PsO without PsA for the control group. The 6-month PsA risk prediction model that uses sequential diagnostic and drug prescription information as a temporal phenomic map yielded an area under the receiver operating characteristic curve of 0.70 (95% CI 0.559-0.833), a mean sensitivity of 0.80 (SD 0.11), a mean specificity of 0.60 (SD 0.04), and a mean negative predictive value of 0.93 (SD 0.04). Conclusions: The findings of this study suggest that the risk prediction model can identify patients with PsO at a high risk of PsA. This model may help health care professionals to prioritize treatment for target high-risk populations and prevent irreversible disease progression and functional loss. %M 36976633 %R 10.2196/39972 %U https://www.jmir.org/2023/1/e39972 %U https://doi.org/10.2196/39972 %U http://www.ncbi.nlm.nih.gov/pubmed/36976633 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e39767 %T Factors Influencing the Clinical Adoption of Quantitative Gait Analysis Technologies for Adult Patient Populations With a Focus on Clinical Efficacy and Clinician Perspectives: Protocol for a Scoping Review %A Sharma,Yashoda %A Cheung,Lovisa %A Patterson,Kara K %A Iaboni,Andrea %+ KITE - Toronto Rehabilitation Institute, University Health Network, 550 University Avenue, Toronto, ON, M5G2A2, Canada, 1 4165973422 ext 3027, andrea.iaboni@uhn.ca %K quantitative gait analysis %K clinical adoption %K clinical efficacy %K clinician perspectives %K barriers %K facilitators %K adults %D 2023 %7 22.3.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: Quantitative gait analysis can support clinical decision-making. These analyses can be performed using wearable sensors, nonwearable sensors, or a combination of both. However, to date, they have not been widely adopted in clinical practice. Technology adoption literature has highlighted the clinical efficacy of technology and the users’ perspective on the technology (eg, ease of use and usefulness) as some factors that influence their widespread adoption. Objective: To assist with the clinical adoption of quantitative gait technologies, this scoping review will synthesize the literature on their clinical efficacy and clinician perspectives on their use in the clinical care of adult patient populations. Methods: This scoping review protocol follows the Joanna Briggs Institute methodology for scoping reviews. The review will include both peer-reviewed and gray literature (ie, conference abstracts) regarding the clinical efficacy of quantitative gait technologies and clinician perspectives on their use in the clinical care of adult patient populations. A comprehensive search strategy was created in MEDLINE (Ovid), which was then translated to 4 other databases: CENTRAL (Ovid), Embase (Ovid), CINAHL (EBSCO), and SPORTDiscus (EBSCO). The title and abstract screening, full-text review, and data extraction of relevant articles will be performed independently by 2 reviewers, with a third reviewer involved to support the resolution of conflicts. Data will be analyzed using content analysis and summarized in tabular and diagram formats. Results: A search of relevant articles will be conducted in all 5 databases, and through hand-searching in Google Scholar and PEDro, including articles published up until December 2022. The research team plans to submit the final scoping review for publication in a peer-reviewed journal in 2023. Conclusions: The findings of this review will be presented at clinical science conferences and published in a peer-reviewed journal. This review will inform future studies designed to develop, evaluate, or implement quantitative gait analysis technologies in clinical practice. International Registered Report Identifier (IRRID): DERR1-10.2196/39767 %M 36947120 %R 10.2196/39767 %U https://www.researchprotocols.org/2023/1/e39767 %U https://doi.org/10.2196/39767 %U http://www.ncbi.nlm.nih.gov/pubmed/36947120 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e42335 %T The Next Frontier of Remote Patient Monitoring: Hospital at Home %A Whitehead,David %A Conley,Jared %+ Department of Emergency Medicine, Massachusetts General Hospital, 55 Fruit Street, Boston, MA, 02114, United States, 1 617 726 2000, dwhitehead.med@gmail.com %K hospital at home %K remote patient monitoring %K RPM %K digital health %K remote monitoring %K remote care %K vital sign %K telemetry %K fall %K cost %K care delivery %K service delivery %D 2023 %7 16.3.2023 %9 Viewpoint %J J Med Internet Res %G English %X Remote patient monitoring (RPM) has shown promise in aiding safe and efficient remote care for chronic conditions; however, its use remains more limited within the hospital at home (HaH) model of care despite a significant opportunity to increase patient eligibility, improve safety, and decrease costs. HaH could achieve these goals by further adopting the 3 primary modalities of RPM (ie, vital sign, continuous single-lead electrocardiogram, and fall monitoring). With only 2 in-person vital sign checks required per day, HaH patient eligibility is currently often limited to lower-acuity cases. The use of vital sign RPM within HaH could better match the standard clinical practice of vital sign checks every 4-8 hours and enable safe care for appropriate moderate-acuity medical and surgical floor-level patients not traditionally enrolled in HaH. Robust, efficient collection of more frequent vital signs via RPM could expand patient eligibility for HaH and create a digital health safety net that enables high quality care. Similarly, our experience at Massachusetts General Hospital has demonstrated that appropriate use of continuous single-lead electrocardiogram RPM can also expand HaH enrollment, particularly for patients with acute decompensated heart failure. Through increasing enrollment of patients in HaH, RPM stands to enable more patients to reap the potential safety benefits of home hospitalization, including decreased rates of delirium and hospital-acquired infections, and better avoid aspects of posthospital syndrome. Furthermore, instituting fall detection RPM allows care teams to further HaH patient safety during their episode of acute care and develop enhanced mitigation strategies to avoid falls post home hospitalization. RPM also has the potential to assist HaH in achieving greater economies of scale and decreasing direct variable costs. By expanding HaH eligibility, RPM could enable HaH programs, which have traditionally operated under capacity, to care for a larger census and decrease allocated fixed costs per hospitalization. Additionally, RPM for HaH could further optimize hybrid in-home and remote nurse or physician evaluations, decreasing costs on a per-episode basis by up to an estimated 3.5%. Overall, RPM holds great promise to increase patient eligibility and patient safety while decreasing costs. However, it is in its infancy in achieving its potential to advance the HaH model of care; further research and experience that inform operational and technical as well as policy considerations are needed. %M 36928088 %R 10.2196/42335 %U https://www.jmir.org/2023/1/e42335 %U https://doi.org/10.2196/42335 %U http://www.ncbi.nlm.nih.gov/pubmed/36928088 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e45095 %T Making Sense of Theories, Models, and Frameworks in Digital Health Behavior Change Design: Qualitative Descriptive Study %A Voorheis,Paula %A Bhuiya,Aunima R %A Kuluski,Kerry %A Pham,Quynh %A Petch,Jeremy %+ Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, 155 College St 4th Floor, Toronto, ON, M5T 3M6, Canada, 1 7058163180, paula.voorheis@mail.utoronto.ca %K behavioral science %K behavior change %K health behavior %K digital health %K mobile health %K theories %K models %K frameworks %D 2023 %7 15.3.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Digital health interventions are increasingly being designed to support health behaviors. Although digital health interventions informed by behavioral science theories, models, and frameworks (TMFs) are more likely to be effective than those designed without them, design teams often struggle to use these evidence-informed tools. Until now, little work has been done to clarify the ways in which behavioral science TMFs can add value to digital health design. Objective: The aim of this study was to better understand how digital health design leaders select and use TMFs in design practice. The questions that were addressed included how do design leaders perceive the value of TMFs in digital health design, what considerations do design leaders make when selecting and applying TMFs, and what do design leaders think is needed in the future to advance the utility of TMFs in digital health design? Methods: This study used a qualitative description design to understand the experiences and perspectives of digital health design leaders. The participants were identified through purposive and snowball sampling. Semistructured interviews were conducted via Zoom software. Interviews were audio-recorded and transcribed using Otter.ai software. Furthermore, 3 researchers coded a sample of interview transcripts and confirmed the coding strategy. One researcher completed the qualitative analysis using a codebook thematic analysis approach. Results: Design leaders had mixed opinions on the value of behavioral science TMFs in digital health design. Leaders suggested that TMFs added the most value when viewed as a starting point rather than the final destination for evidence-informed design. Specifically, these tools added value when they acted as a gateway drug to behavioral science, supported health behavior conceptualization, were balanced with expert knowledge and user-centered design principles, were complementary to existing design methods, and supported both individual- and systems-level thinking. Design leaders also felt that there was a considerable nuance in selecting the most value-adding TMFs. Considerations should be made regarding their source, appropriateness, complexity, accessibility, adaptability, evidence base, purpose, influence, audience, fit with team expertise, fit with team culture, and fit with external pressures. Design leaders suggested multiple opportunities to advance the use of TMFs. These included improving TMF reporting, design, and accessibility, as well as improving design teams' capacity to use TMFs appropriately in practice. Conclusions: When designing a digital health behavior change intervention, using TMFs can help design teams to systematically integrate behavioral insights. The future of digital health behavior change design demands an easier way for designers to integrate evidence-based TMFs into practice. %M 36920442 %R 10.2196/45095 %U https://www.jmir.org/2023/1/e45095 %U https://doi.org/10.2196/45095 %U http://www.ncbi.nlm.nih.gov/pubmed/36920442 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 9 %N %P e43725 %T Electronic Health Record–Based Absolute Risk Prediction Model for Esophageal Cancer in the Chinese Population: Model Development and External Validation %A Han,Yuting %A Zhu,Xia %A Hu,Yizhen %A Yu,Canqing %A Guo,Yu %A Hang,Dong %A Pang,Yuanjie %A Pei,Pei %A Ma,Hongxia %A Sun,Dianjianyi %A Yang,Ling %A Chen,Yiping %A Du,Huaidong %A Yu,Min %A Chen,Junshi %A Chen,Zhengming %A Huo,Dezheng %A Jin,Guangfu %A Lv,Jun %A Hu,Zhibin %A Shen,Hongbing %A Li,Liming %+ Department of Epidemiology and Biostatistics, School of Public Health, Peking University, No 38 Xueyuan Rd, Haidian District, Beijing, 100191, China, 86 10 82801528, lvjun@bjmu.edu.cn %K esophageal cancer %K prediction model %K absolute risk %K China %K prospective cohort %K screening %K primary prevention %K development %K external validation %K electronic health record %D 2023 %7 15.3.2023 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: China has the largest burden of esophageal cancer (EC). Prediction models can be used to identify high-risk individuals for intensive lifestyle interventions and endoscopy screening. However, the current prediction models are limited by small sample size and a lack of external validation, and none of them can be embedded into the booming electronic health records (EHRs) in China. Objective: This study aims to develop and validate absolute risk prediction models for EC in the Chinese population. In particular, we assessed whether models that contain only EHR-available predictors performed well. Methods: A prospective cohort recruiting 510,145 participants free of cancer from both high EC-risk and low EC-risk areas in China was used to develop EC models. Another prospective cohort of 18,441 participants was used for validation. A flexible parametric model was used to develop a 10-year absolute risk model by considering the competing risks (full model). The full model was then abbreviated by keeping only EHR-available predictors. We internally and externally validated the models by using the area under the receiver operating characteristic curve (AUC) and calibration plots and compared them based on classification measures. Results: During a median of 11.1 years of follow-up, we observed 2550 EC incident cases. The models consisted of age, sex, regional EC-risk level (high-risk areas: 2 study regions; low-risk areas: 8 regions), education, family history of cancer (simple model), smoking, alcohol use, BMI (intermediate model), physical activity, hot tea consumption, and fresh fruit consumption (full model). The performance was only slightly compromised after the abbreviation. The simple and intermediate models showed good calibration and excellent discriminating ability with AUCs (95% CIs) of 0.822 (0.783-0.861) and 0.830 (0.792-0.867) in the external validation and 0.871 (0.858-0.884) and 0.879 (0.867-0.892) in the internal validation, respectively. Conclusions: Three nested 10-year EC absolute risk prediction models for Chinese adults aged 30-79 years were developed and validated, which may be particularly useful for populations in low EC-risk areas. Even the simple model with only 5 predictors available from EHRs had excellent discrimination and good calibration, indicating its potential for broader use in tailored EC prevention. The simple and intermediate models have the potential to be widely used for both primary and secondary prevention of EC. %M 36781293 %R 10.2196/43725 %U https://publichealth.jmir.org/2023/1/e43725 %U https://doi.org/10.2196/43725 %U http://www.ncbi.nlm.nih.gov/pubmed/36781293 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 9 %N %P e38868 %T Automating Case Reporting of Chlamydia and Gonorrhea to Public Health Authorities in Illinois Clinics: Implementation and Evaluation of Findings %A Mishra,Ninad %A Grant,Reynaldo %A Patel,Megan Toth %A Guntupalli,Siva %A Hamilton,Andrew %A Carr,Jeremy %A McKnight,Elizabeth %A Wise,Wendy %A deRoode,David %A Jellison,Jim %A Collins,Natalie Viator %A Pérez,Alejandro %A Karki,Saugat %+ Division of STD Prevention, Centers for Disease Control and Prevention, 1600 Clifton Road, Atlanta, GA, 30329, United States, 1 4047187483, oql7@cdc.gov %K public health surveillance %K sexually transmitted diseases %K gonorrhoea %K chlamydia %K electronic case reporting %K eCR %K health information interoperability %K electronic health records %K EHR %K case reporting %K automated %K reporting %K recording %K patient records %K cases %K health care system %K semantic %K interoperability %K implementation %D 2023 %7 14.3.2023 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: Chlamydia and gonorrhea cases continue to rise in Illinois, increasing by 16.4% and 70.9% in 2019, respectively, compared with 2015. Providers are required to report both chlamydia and gonorrhea, as mandated by public health laws. Manual reporting remains a huge burden; 90%-93% of cases were reported to Illinois Department of Public Health (IDPH) via electronic laboratory reporting (ELR), and the remaining were reported through web-based data entry platforms, faxes, and phone calls. However, cases reported via ELRs only contain information available to a laboratory facility and do not contain additional data needed for public health. Such data are typically found in an electronic health record (EHR). Electronic case reports (eCRs) were developed and automated the generation of case reports from EHRs to be reported to public health agencies. Objective: Prior studies consolidated trigger criteria for eCRs, and compared with manual reporting, found it to be more complete. The goal of this project is to pilot standards-based eCR for chlamydia and gonorrhea. We evaluated the throughput, completeness, and timeliness of eCR compared to ELR, as well as the implementation experience at a large health center–controlled network in Illinois. Methods: For this study, we selected 8 clinics located on the north, west, and south sides of Chicago to implement the eCRs; these cases were reported to IDPH. The study period was 52 days. The centralized EHR used by these clinics leveraged 2 of the 3 case detection scenarios, which were previously defined as the trigger, to generate an eCR. These messages were successfully transmitted via Health Level 7 electronic initial case report standard. Upon receipt by IDPH, these eCRs were parsed and housed in a staging database. Results: During the study period, 183 eCRs representing 135 unique patients were received by IDPH. eCR reported 95% (n=113 cases) of all the chlamydia cases and 97% (n=70 cases) of all the gonorrhea cases reported from the participating clinical sites. eCR found an additional 14 (19%) cases of gonorrhea that were not reported via ELR. However, ELR reported an additional 6 cases of chlamydia and 2 cases of gonorrhea, which were not reported via eCR. ELR reported 100% of chlamydia cases but only 81% of gonorrhea cases. While key elements such as patient and provider names were complete in both eCR and ELR, eCR was found to report additional clinical data, including history of present illness, reason for visit, symptoms, diagnosis, and medications. Conclusions: eCR successfully identified and created automated reports for chlamydia and gonorrhea cases in the implementing clinics in Illinois. eCR demonstrated a more complete case report and represents a promising future of reducing provider burden for reporting cases while achieving greater semantic interoperability between health care systems and public health. %M 36917153 %R 10.2196/38868 %U https://publichealth.jmir.org/2023/1/e38868 %U https://doi.org/10.2196/38868 %U http://www.ncbi.nlm.nih.gov/pubmed/36917153 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e43103 %T The Priorities of End Users of Emergency Department Electronic Health Records: Modified Delphi Study %A Yip,Matthew %A Ackery,Alun %A Jamieson,Trevor %A Mehta,Shaun %+ Temerty Faculty of Medicine, University of Toronto, 1 King's College Circle, Toronto, ON, M5S 1A8, Canada, 1 6479092917, matt.yip@mail.utoronto.ca %K Delphi %K EHR %K electronic health record %K emergency medicine %K emergency %K functionality %K health information exchange %K health system %K medical informatics %K patient-physician relationship %K usability %D 2023 %7 10.3.2023 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The needs of the emergency department (ED) pose unique challenges to modern electronic health record (EHR) systems. A diverse case load of high-acuity, high-complexity presentations, and ambulatory patients, all requiring multiple transitions of care, creates a rich environment through which to critically examine EHRs. Objective: This investigation aims to capture and analyze the perspective of end users of EHR about the strengths, limitations, and future priorities for EHR in the setting of the ED. Methods: In the first phase of this investigation, a literature search was conducted to identify 5 key usage categories of ED EHRs. Using key usage categories in the first phase, a modified Delphi study was conducted with a group of 12 panelists with expertise in both emergency medicine and health informatics. Across 3 rounds of surveys, panelists generated and refined a list of strengths, limitations, and key priorities. Results: The findings from this investigation highlighted the preference of panelists for features maximizing functionality of basic clinical features relative to features of disruptive innovation. Conclusions: By capturing the perspectives of end users in the ED, this investigation highlights areas for the improvement or development of future EHRs in acute care settings. %M 36897633 %R 10.2196/43103 %U https://humanfactors.jmir.org/2023/1/e43103 %U https://doi.org/10.2196/43103 %U http://www.ncbi.nlm.nih.gov/pubmed/36897633 %0 Journal Article %@ 2561-7605 %I JMIR Publications %V 6 %N %P e41936 %T The Impact of Health Information Exchange on In-Hospital and Postdischarge Mortality in Older Adults with Alzheimer Disease Readmitted to a Different Hospital Within 30 Days of Discharge: Cohort Study of Medicare Beneficiaries %A Turbow,Sara %A Vaughan,Camille P %A Culler,Steven D %A Hepburn,Kenneth W %A Rask,Kimberly J %A Perkins,Molly M %A Clevenger,Carolyn K %A Ali,Mohammed K %+ Division of General Internal Medicine, Department of Medicine, Emory University School of Medicine, 49 Jesse Hill Jr Dr SE, Atlanta, GA, 30303, United States, 1 404 251 8897, sara.turbow@emory.edu %K readmissions %K care fragmentation %K health information exchange %K mortality %K Alzheimer disease %K electronic health information %K information sharing %K older adults %K information exchange %K hospital system %K health informatics %D 2023 %7 10.3.2023 %9 Original Paper %J JMIR Aging %G English %X Background: Although electronic health information sharing is expanding nationally, it is unclear whether electronic health information sharing improves patient outcomes, particularly for patients who are at the highest risk of communication challenges, such as older adults with Alzheimer disease. Objective: To determine the association between hospital-level health information exchange (HIE) participation and in-hospital or postdischarge mortality among Medicare beneficiaries with Alzheimer disease or 30-day readmissions to a different hospital following an admission for one of several common conditions. Methods: This was a cohort study of Medicare beneficiaries with Alzheimer disease who had one or more 30-day readmissions in 2018 following an initial admission for select Hospital Readmission Reduction Program conditions (acute myocardial infarction, congestive heart failure, chronic obstructive pulmonary disease, and pneumonia) or common reasons for hospitalization among older adults with Alzheimer disease (dehydration, syncope, urinary tract infection, or behavioral issues). Using unadjusted and adjusted logistic regression, we examined the association between electronic information sharing and in-hospital mortality during the readmission or mortality in the 30 days following the readmission. Results: A total of 28,946 admission-readmission pairs were included. Beneficiaries with same-hospital readmissions were older (aged 81.1, SD 8.6 years) than beneficiaries with readmissions to different hospitals (age range 79.8-80.3 years, P<.001). Compared to admissions and readmissions to the same hospital, beneficiaries who had a readmission to a different hospital that shared an HIE with the admission hospital had 39% lower odds of dying during the readmission (adjusted odds ratio [AOR] 0.61, 95% CI 0.39-0.95). There were no differences in in-hospital mortality observed for admission-readmission pairs to different hospitals that participated in different HIEs (AOR 1.02, 95% CI 0.82-1.28) or to different hospitals where one or both hospitals did not participate in HIE (AOR 1.25, 95% CI 0.93-1.68), and there was no association between information sharing and postdischarge mortality. Conclusions: These results indicate that information sharing between unrelated hospitals via a shared HIE may be associated with lower in-hospital, but not postdischarge, mortality for older adults with Alzheimer disease. In-hospital mortality during a readmission to a different hospital was higher if the admission and readmission hospitals participated in different HIEs or if one or both hospitals did not participate in an HIE. Limitations of this analysis include that HIE participation was measured at the hospital level, rather than at the provider level. This study provides some evidence that HIEs can improve care for vulnerable populations receiving acute care from different hospitals. %M 36897638 %R 10.2196/41936 %U https://aging.jmir.org/2023/1/e41936 %U https://doi.org/10.2196/41936 %U http://www.ncbi.nlm.nih.gov/pubmed/36897638 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e42822 %T A Data Transformation Methodology to Create Findable, Accessible, Interoperable, and Reusable Health Data: Software Design, Development, and Evaluation Study %A Sinaci,A Anil %A Gencturk,Mert %A Teoman,Huseyin Alper %A Laleci Erturkmen,Gokce Banu %A Alvarez-Romero,Celia %A Martinez-Garcia,Alicia %A Poblador-Plou,Beatriz %A Carmona-Pírez,Jonás %A Löbe,Matthias %A Parra-Calderon,Carlos Luis %+ Software Research & Development and Consultancy Corporation (SRDC), Orta Dogu Teknik Universitesi Teknokent K1-16, Cankaya, 06800, Turkey, 90 3122101763, anil@srdc.com.tr %K Health Level 7 Fast Healthcare Interoperability Resources %K HL7 FHIR %K Findable, Accessible, Interoperable, and Reusable principles %K FAIR principles %K health data sharing %K health data transformation %K secondary use %D 2023 %7 8.3.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Sharing health data is challenging because of several technical, ethical, and regulatory issues. The Findable, Accessible, Interoperable, and Reusable (FAIR) guiding principles have been conceptualized to enable data interoperability. Many studies provide implementation guidelines, assessment metrics, and software to achieve FAIR-compliant data, especially for health data sets. Health Level 7 (HL7) Fast Healthcare Interoperability Resources (FHIR) is a health data content modeling and exchange standard. Objective: Our goal was to devise a new methodology to extract, transform, and load existing health data sets into HL7 FHIR repositories in line with FAIR principles, develop a Data Curation Tool to implement the methodology, and evaluate it on health data sets from 2 different but complementary institutions. We aimed to increase the level of compliance with FAIR principles of existing health data sets through standardization and facilitate health data sharing by eliminating the associated technical barriers. Methods: Our approach automatically processes the capabilities of a given FHIR end point and directs the user while configuring mappings according to the rules enforced by FHIR profile definitions. Code system mappings can be configured for terminology translations through automatic use of FHIR resources. The validity of the created FHIR resources can be automatically checked, and the software does not allow invalid resources to be persisted. At each stage of our data transformation methodology, we used particular FHIR-based techniques so that the resulting data set could be evaluated as FAIR. We performed a data-centric evaluation of our methodology on health data sets from 2 different institutions. Results: Through an intuitive graphical user interface, users are prompted to configure the mappings into FHIR resource types with respect to the restrictions of selected profiles. Once the mappings are developed, our approach can syntactically and semantically transform existing health data sets into HL7 FHIR without loss of data utility according to our privacy-concerned criteria. In addition to the mapped resource types, behind the scenes, we create additional FHIR resources to satisfy several FAIR criteria. According to the data maturity indicators and evaluation methods of the FAIR Data Maturity Model, we achieved the maximum level (level 5) for being Findable, Accessible, and Interoperable and level 3 for being Reusable. Conclusions: We developed and extensively evaluated our data transformation approach to unlock the value of existing health data residing in disparate data silos to make them available for sharing according to the FAIR principles. We showed that our method can successfully transform existing health data sets into HL7 FHIR without loss of data utility, and the result is FAIR in terms of the FAIR Data Maturity Model. We support institutional migration to HL7 FHIR, which not only leads to FAIR data sharing but also eases the integration with different research networks. %M 36884270 %R 10.2196/42822 %U https://www.jmir.org/2023/1/e42822 %U https://doi.org/10.2196/42822 %U http://www.ncbi.nlm.nih.gov/pubmed/36884270 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e44547 %T An Ontology-Based Approach for Consolidating Patient Data Standardized With European Norm/International Organization for Standardization 13606 (EN/ISO 13606) Into Joint Observational Medical Outcomes Partnership (OMOP) Repositories: Description of a Methodology %A Frid,Santiago %A Pastor Duran,Xavier %A Bracons Cucó,Guillem %A Pedrera-Jiménez,Miguel %A Serrano-Balazote,Pablo %A Muñoz Carrero,Adolfo %A Lozano-Rubí,Raimundo %+ Clinical Foundations Department, Universitat de Barcelona, Casanova 143, Barcelona, 08036, Spain, 34 934035258, santifrid@gmail.com %K health information interoperability %K health research %K health information standards %K dual model %K secondary use of health data %K Observational Medical Outcomes Partnership Common Data Model %K European Norm/International Organization for Standardization 13606 %K health records %K ontologies %K clinical data %D 2023 %7 8.3.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: To discover new knowledge from data, they must be correct and in a consistent format. OntoCR, a clinical repository developed at Hospital Clínic de Barcelona, uses ontologies to represent clinical knowledge and map locally defined variables to health information standards and common data models. Objective: The aim of the study is to design and implement a scalable methodology based on the dual-model paradigm and the use of ontologies to consolidate clinical data from different organizations in a standardized repository for research purposes without loss of meaning. Methods: First, the relevant clinical variables are defined, and the corresponding European Norm/International Organization for Standardization (EN/ISO) 13606 archetypes are created. Data sources are then identified, and an extract, transform, and load process is carried out. Once the final data set is obtained, the data are transformed to create EN/ISO 13606–normalized electronic health record (EHR) extracts. Afterward, ontologies that represent archetyped concepts and map them to EN/ISO 13606 and Observational Medical Outcomes Partnership Common Data Model (OMOP CDM) standards are created and uploaded to OntoCR. Data stored in the extracts are inserted into its corresponding place in the ontology, thus obtaining instantiated patient data in the ontology-based repository. Finally, data can be extracted via SPARQL queries as OMOP CDM–compliant tables. Results: Using this methodology, EN/ISO 13606–standardized archetypes that allow for the reuse of clinical information were created, and the knowledge representation of our clinical repository by modeling and mapping ontologies was extended. Furthermore, EN/ISO 13606–compliant EHR extracts of patients (6803), episodes (13,938), diagnosis (190,878), administered medication (222,225), cumulative drug dose (222,225), prescribed medication (351,247), movements between units (47,817), clinical observations (6,736,745), laboratory observations (3,392,873), limitation of life-sustaining treatment (1,298), and procedures (19,861) were created. Since the creation of the application that inserts data from extracts into the ontologies is not yet finished, the queries were tested and the methodology was validated by importing data from a random subset of patients into the ontologies using a locally developed Protégé plugin (“OntoLoad”). In total, 10 OMOP CDM–compliant tables (“Condition_occurrence,” 864 records; “Death,” 110; “Device_exposure,” 56; “Drug_exposure,” 5609; “Measurement,” 2091; “Observation,” 195; “Observation_period,” 897; “Person,” 922; “Visit_detail,” 772; and “Visit_occurrence,” 971) were successfully created and populated. Conclusions: This study proposes a methodology for standardizing clinical data, thus allowing its reuse without any changes in the meaning of the modeled concepts. Although this paper focuses on health research, our methodology suggests that the data be initially standardized per EN/ISO 13606 to obtain EHR extracts with a high level of granularity that can be used for any purpose. Ontologies constitute a valuable approach for knowledge representation and standardization of health information in a standard-agnostic manner. With the proposed methodology, institutions can go from local raw data to standardized, semantically interoperable EN/ISO 13606 and OMOP repositories. %M 36884279 %R 10.2196/44547 %U https://medinform.jmir.org/2023/1/e44547 %U https://doi.org/10.2196/44547 %U http://www.ncbi.nlm.nih.gov/pubmed/36884279 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e43005 %T Identification of Postpartum Depression in Electronic Health Records: Validation in a Large Integrated Health Care System %A Slezak,Jeff %A Sacks,David %A Chiu,Vicki %A Avila,Chantal %A Khadka,Nehaa %A Chen,Jiu-Chiuan %A Wu,Jun %A Getahun,Darios %+ Kaiser Permanente Southern California, 100 S. Los Robles Ave, Pasadena, CA, 91101, United States, 1 626 564 3477, Jeff.M.Slezak@kp.org %K validation %K postpartum depression %K electronic health records %K pregnancy %K health care system %K diagnosis codes %K pharmacy records %K health data %K data collection %K implementation %K eHealth record %K depression %K mental well-being %K women's health %D 2023 %7 1.3.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: The accuracy of electronic health records (EHRs) for identifying postpartum depression (PPD) is not well studied. Objective: This study aims to evaluate the accuracy of PPD reporting in EHRs and compare the quality of PPD data collected before and after the implementation of the International Classification of Diseases, Tenth Revision (ICD-10) coding in the health care system. Methods: Information on PPD was extracted from a random sample of 400 eligible Kaiser Permanente Southern California patients’ EHRs. Clinical diagnosis codes and pharmacy records were abstracted for two time periods: January 1, 2012, through December 31, 2014 (International Classification of Diseases, Ninth Revision [ICD-9] period), and January 1, 2017, through December 31, 2019 (ICD-10 period). Manual chart reviews of clinical records for PPD were considered the gold standard and were compared with corresponding electronically coded diagnosis and pharmacy records using sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Kappa statistic was calculated to measure agreement. Results: Overall agreement between the identification of depression using combined diagnosis codes and pharmacy records with that of medical record review was strong (κ=0.85, sensitivity 98.3%, specificity 83.3%, PPV 93.7%, NPV 95.0%). Using only diagnosis codes resulted in much lower sensitivity (65.4%) and NPV (50.5%) but good specificity (88.6%) and PPV (93.5%). Separately, examining agreement between chart review and electronic coding among diagnosis codes and pharmacy records showed sensitivity, specificity, and NPV higher with prescription use records than with clinical diagnosis coding for PPD, 96.5% versus 72.0%, 96.5% versus 65.0%, and 96.5% versus 65.0%, respectively. There was no notable difference in agreement between ICD-9 (overall κ=0.86) and ICD-10 (overall κ=0.83) coding periods. Conclusions: PPD is not reliably captured in the clinical diagnosis coding of EHRs. The accuracy of PPD identification can be improved by supplementing clinical diagnosis with pharmacy use records. The completeness of PPD data remained unchanged after the implementation of the ICD-10 diagnosis coding. %M 36857123 %R 10.2196/43005 %U https://medinform.jmir.org/2023/1/e43005 %U https://doi.org/10.2196/43005 %U http://www.ncbi.nlm.nih.gov/pubmed/36857123 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 10 %N %P e43966 %T Exploring Patient Journey Mapping and the Learning Health System: Scoping Review %A Joseph,Amanda L %A Monkman,Helen %A Kushniruk,Andre %A Quintana,Yuri %+ School of Health Information Science, University of Victoria, PO Box 1700 STN CSC, Victoria, BC, V8W 2Y2, Canada, 1 250 721 8575, amandalynnjoseph@uvic.ca %K patient journey map %K journey map %K patient health information %K learning health system %K learning health care system %K delivery of health care %K service delivery %K scoping review %K health informatics %K user experience %K data integration %D 2023 %7 27.2.2023 %9 Review %J JMIR Hum Factors %G English %X Background: Journey maps are visualization tools that can facilitate the diagrammatical representation of stakeholder groups by interest or function for comparative visual analysis. Therefore, journey maps can illustrate intersections and relationships between organizations and consumers using products or services. We propose that some synergies may exist between journey maps and the concept of a learning health system (LHS). The overarching goal of an LHS is to use health care data to inform clinical practice and improve service delivery processes and patient outcomes. Objective: The purpose of this review was to assess the literature and establish a relationship between journey mapping techniques and LHSs. Specifically, in this study, we explored the current state of the literature to answer the following research questions: (1) Is there a relationship between journey mapping techniques and an LHS in the literature? (2) Is there a way to integrate the data from journey mapping activities into an LHS? (3) How can the data gleaned from journey map activities be used to inform an LHS? Methods: A scoping review was conducted by querying the following electronic databases: Cochrane Database of Systematic Reviews (Ovid), IEEE Xplore, PubMed, Web of Science, Academic Search Complete (EBSCOhost), APA PsycInfo (EBSCOhost), CINAHL (EBSCOhost), and MEDLINE (EBSCOhost). Two researchers applied the inclusion criteria and assessed all articles by title and abstract in the first screen, using Covidence. Following this, a full-text review of included articles was done, with relevant data extracted, tabulated, and assessed thematically. Results: The initial search yielded 694 studies. Of those, 179 duplicates were removed. Following this, 515 articles were assessed during the first screening phase, and 412 were excluded, as they did not meet the inclusion criteria. Next, 103 articles were read in full, and 95 were excluded, resulting in a final sample of 8 articles that satisfied the inclusion criteria. The article sample can be subsumed into 2 overarching themes: (1) the need to evolve service delivery models in health care, and (2) the potential value of using patient journey data in an LHS. Conclusions: This scoping review demonstrated the gap in knowledge regarding integrating the data from journey mapping activities into an LHS. Our findings highlighted the importance of using the data from patient experiences to enrich an LHS and provide holistic care. To satisfy this gap, the authors intend to continue this investigation to establish the relationship between journey mapping and the concept of LHSs. This scoping review will serve as phase 1 of an investigative series. Phase 2 will entail the creation of a holistic framework to guide and streamline data integration from journey mapping activities into an LHS. Lastly, phase 3 will provide a proof of concept to demonstrate how patient journey mapping activities could be integrated into an LHS. %M 36848189 %R 10.2196/43966 %U https://humanfactors.jmir.org/2023/1/e43966 %U https://doi.org/10.2196/43966 %U http://www.ncbi.nlm.nih.gov/pubmed/36848189 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e43848 %T Perspectives on Challenges and Opportunities for Interoperability: Findings From Key Informant Interviews With Stakeholders in Ohio %A Walker,Daniel M %A Tarver,Willi L %A Jonnalagadda,Pallavi %A Ranbom,Lorin %A Ford,Eric W %A Rahurkar,Saurabh %+ Department of Family and Community Medicine, College of Medicine, The Ohio State University, Suite 5000, 700 Ackerman Rd, Columbus, OH, 43202, United States, 1 203 988 1800, daniel.walker@osumc.edu %K interoperability %K health information exchange %K health information technology %K electronic health record %K usability %D 2023 %7 24.2.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: Interoperability—the exchange and integration of data across the health care system—remains a challenge despite ongoing policy efforts aimed at promoting interoperability. Objective: This study aimed to identify current challenges and opportunities to advancing interoperability across stakeholders. Methods: Primary data were collected through qualitative, semistructured interviews with stakeholders (n=24) in Ohio from July to October 2021. Interviewees were sampled using a stratified purposive sample of key informants from 4 representative groups as follows: acute care and children’s hospital leaders, primary care providers, behavioral health providers, and regional health information exchange networks. Interviews focused on key informant perspectives on electronic health record implementation, the alignment of public policy with organizational strategy, interoperability implementation challenges, and opportunities for health information technology. The interviews were transcribed verbatim followed by rigorous qualitative analysis using directed content analysis. Results: The findings illuminate themes related to challenges and opportunities for interoperability that align with technological (ie, implementation challenges, mismatches in interoperability capabilities across stakeholders, and opportunities to leverage new technology and integrate social determinants of health data), organizational (ie, facilitators of interoperability and strategic alignment of participation in value-based payment programs with interoperability), and environmental (ie, policy) domains. Conclusions: Interoperability, although technically feasible for most providers, remains challenging for technological, organizational, and environmental reasons. Our findings suggest that the incorporation of end user considerations into health information technology development, implementation, policy, and standard deployment may support interoperability advancement. %M 36826979 %R 10.2196/43848 %U https://medinform.jmir.org/2023/1/e43848 %U https://doi.org/10.2196/43848 %U http://www.ncbi.nlm.nih.gov/pubmed/36826979 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e40634 %T User Testing of the Veteran Delegation Tool: Qualitative Inquiry %A Haun,Jolie N %A Melillo,Christine %A Schneider,Tali %A Merzier,Marie M %A Klanchar,S Angelina %A Fowler,Christopher A %A Benzinger,Rachel C %+ James A Haley Veterans Hospital, 8900 Grand Oak Circle, Tampa, FL, 33637, United States, 1 813 558 3938, christine.melillo@va.gov %K electronic health portal %K human-centered design %K delegate %K electronic resources %K delegation %K care partner %K veteran %K Veteran Delegation Tool %K Veterans Health Administration %D 2023 %7 23.2.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Informal caregivers, or care partners, provide critical support to care recipients when managing health care. Veterans Health Administration (VHA) priorities identify care partners as vital in supporting veterans’ care management. The Veteran Delegation Tool (VDT) is VHA’s Health Insurance Portability and Accountability Act–compliant solution for care partners to comanage veterans’ care through VHA’s electronic health portal. Human-centered design approaches in VDT development are needed to inform enhancements aimed at promoting uptake and sustained use. Objective: The objective of this prospective descriptive quality improvement project was to use a human-centered design approach to examine VDT use perceptions and practical experiences. Methods: This project was conducted using a 4-phase approach: frame, discover, design, and deliver. The frame phase designed the protocol and prepared the VDT system for testing. This paper reports on the discover phase, which used semistructured and follow-up interviews and user testing to examine VDT’s benefits, facilitators, and barriers. The discover phase data informed the design and deliver phases, which are underway. Results: Veterans (24/54, 44%), care partners (21/54, 39%), and individuals who represented dual roles (9/54, 17%)—namely veteran care partner (4/54, 7%), veteran clinical provider (2/54, 4%), and care partner provider (3/54, 6%)—participated in semistructured interviews in the discover phase. A subsample of these participants (3/54, 6%) participated in the follow-up interviews and user testing. Analysis of the semistructured interviews indicated convergence on the respondents’ perceptions of VDT’s benefits, facilitators, and barriers and recommendations for improving VDT. The perceived benefits were authorized access, comanagement of care needs on the web, communication with the clinical team, access to resources, and ease of burden. Perceived barriers were nonrecognition of the benefits of VDT, technical literacy access issues, increased stress in or burden on care partners, and personal health information security. Participant experiences across 4 VDT activity domains were upgrade to My HealtheVet Premium account, registration, sign-in, and use. User testing demonstrated users’ challenges to register, navigate, and use VDT. Findings informed VDT development enhancements and recommendations. Conclusions: Care partners need Health Insurance Portability and Accountability Act–compliant access to electronic health portals to assist with care management. VDT is VHA’s solution, allowing communication among delegates, veterans, and clinical care teams. Users value VDT’s potential use and benefits, while access and navigation improvements to ensure uptake and sustained use are needed. Future efforts need to iteratively evaluate the human-centered phases, design and deliver, of VDT to target audiences. Continued efforts to understand and respond to care partners’ needs are warranted. %M 36821364 %R 10.2196/40634 %U https://www.jmir.org/2023/1/e40634 %U https://doi.org/10.2196/40634 %U http://www.ncbi.nlm.nih.gov/pubmed/36821364 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e43496 %T Patient Online Record Access in English Primary Care: Qualitative Survey Study of General Practitioners’ Views %A Blease,Charlotte %A Torous,John %A Dong,Zhiyong %A Davidge,Gail %A DesRoches,Catherine %A Kharko,Anna %A Turner,Andrew %A Jones,Ray %A Hägglund,Maria %A McMillan,Brian %+ Division of General Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, General Medicine and Primary Care, Beth Israel Deaconess Medical Center Beth Israel Deaconess Medical Center, 330 Brookline Ave, Boston, MA, 02215, United States, 1 7921587211, charlotteblease@gmail.com %K electronic health records %K attitudes %K general practice %K patients %K online record access %K open notes %K opinions %K primary care %K qualitative research %D 2023 %7 22.2.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: In 2022, NHS England announced plans to ensure that all adult primary care patients in England would have full online access to new data added to their general practitioner (GP) record. However, this plan has not yet been fully implemented. Since April 2020, the GP contract in England has already committed to offering patients full online record access on a prospective basis and on request. However, there has been limited research into UK GPs’ experiences and opinions about this practice innovation. Objective: This study aimed to explore the experiences and opinions of GPs in England about patients’ access to their full web-based health record, including clinicians’ free-text summaries of the consultation (so-called “open notes”). Methods: In March 2022, using a convenience sample, we administered a web-based mixed methods survey of 400 GPs in the United Kingdom to explore their experiences and opinions about the impact on patients and GPs’ practices to offer patients full online access to their health records. Participants were recruited using the clinician marketing service Doctors.net.uk from registered GPs currently working in England. We conducted a qualitative descriptive analysis of written responses (“comments”) to 4 open-ended questions embedded in a web-based questionnaire. Results: Of 400 GPs, 224 (56%) left comments that were classified into 4 major themes: increased strain on GP practices, the potential to harm patients, changes to documentation, and legal concerns. GPs believed that patient access would lead to extra work for them, reduced efficiency, and increased burnout. The participants also believed that access would increase patient anxiety and incur risks to patient safety. Experienced and perceived documentation changes included reduced candor and changes to record functionality. Anticipated legal concerns encompassed fears about increased litigation risks and lack of legal guidance to GPs about how to manage documentation that would be read by patients and potential third parties. Conclusions: This study provides timely information on the views of GPs in England regarding patient access to their web-based health records. Overwhelmingly, GPs were skeptical about the benefits of access both for patients and to their practices. These views are similar to those expressed by clinicians in other countries, including Nordic countries and the United States before patient access. The survey was limited by the convenience sample, and it is not possible to infer that our sample was representative of the opinions of GPs in England. More extensive, qualitative research is required to understand the perspectives of patients in England after experiencing access to their web-based records. Finally, further research is needed to explore objective measures of the impact of patient access to their records on health outcomes, clinician workload, and changes to documentation. %M 36811939 %R 10.2196/43496 %U https://www.jmir.org/2023/1/e43496 %U https://doi.org/10.2196/43496 %U http://www.ncbi.nlm.nih.gov/pubmed/36811939 %0 Journal Article %@ 2369-3762 %I JMIR Publications %V 9 %N %P e42364 %T Implementation of a Biopsychosocial History and Physical Exam Template in the Electronic Health Record: Mixed Methods Study %A Rieger,Erin Y %A Anderson,Irsk J %A Press,Valerie G %A Cui,Michael X %A Arora,Vineet M %A Williams,Brent C %A Tang,Joyce W %+ Department of Medicine, University of Chicago, 5841 S Maryland Avenue, Chicago, IL, 60637, United States, 1 773 702 1111, jtang@bsd.uchicago.edu %K medical education %K electronic health record %K hospital medicine %K psychosocial factors %K chronic condition %K chronic %K disease %K management %K prevention %K clinical %K engagement %D 2023 %7 21.2.2023 %9 Original Paper %J JMIR Med Educ %G English %X Background: Patients’ perspectives and social contexts are critical for prevention of hospital readmissions; however, neither is routinely assessed using the traditional history and physical (H&P) examination nor commonly documented in the electronic health record (EHR). The H&P 360 is a revised H&P template that integrates routine assessment of patient perspectives and goals, mental health, and an expanded social history (behavioral health, social support, living environment and resources, function). Although the H&P 360 has shown promise in increasing psychosocial documentation in focused teaching contexts, its uptake and impact in routine clinical settings are unknown. Objective: The aim of this study was to assess the feasibility, acceptability, and impact on care planning of implementing an inpatient H&P 360 template in the EHR for use by fourth-year medical students. Methods: A mixed methods study design was used. Fourth-year medical students on internal medicine subinternship (subI) services were given a brief training on the H&P 360 and access to EHR-based H&P 360 templates. Students not working in the intensive care unit (ICU) were asked to use the templates at least once per call cycle, whereas use by ICU students was elective. An EHR query was used to identify all H&P 360 and traditional H&P admission notes authored by non-ICU students at University of Chicago (UC) Medicine. Of these notes, all H&P 360 notes and a sample of traditional H&P notes were reviewed by two researchers for the presence of H&P 360 domains and impact on patient care. A postcourse survey was administered to query all students for their perspectives on the H&P 360. Results: Of the 13 non-ICU subIs at UC Medicine, 6 (46%) used the H&P 360 templates at least once, which accounted for 14%-92% of their authored admission notes (median 56%). Content analysis was performed with 45 H&P 360 notes and 54 traditional H&P notes. Psychosocial documentation across all H&P 360 domains (patient perspectives and goals, mental health, expanded social history elements) was more common in H&P 360 compared with traditional notes. Related to impact on patient care, H&P 360 notes more commonly identified needs (20% H&P 360; 9% H&P) and described interdisciplinary coordination (78% H&P 360; 41% H&P). Of the 11 subIs completing surveys, the vast majority (n=10, 91%) felt the H&P 360 helped them understand patient goals and improved the patient-provider relationship. Most students (n=8, 73%) felt the H&P 360 took an appropriate amount of time. Conclusions: Students who applied the H&P 360 using templated notes in the EHR found it feasible and helpful. These students wrote notes reflecting enhanced assessment of goals and perspectives for patient-engaged care and contextual factors important to preventing rehospitalization. Reasons some students did not use the templated H&P 360 should be examined in future studies. Uptake may be enhanced through earlier and repeated exposure and greater engagement by residents and attendings. Larger-scale implementation studies can help further elucidate the complexities of implementing nonbiomedical information within EHRs. %M 36802337 %R 10.2196/42364 %U https://mededu.jmir.org/2023/1/e42364 %U https://doi.org/10.2196/42364 %U http://www.ncbi.nlm.nih.gov/pubmed/36802337 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e43758 %T Design, Development, and Evaluation of an Automated Solution for Electronic Information Exchange Between Acute and Long-term Postacute Care Facilities: Design Science Research %A Gottumukkala,Madhu %+ College of Business & Information Systems, Dakota State University, 820 Washington Ave N, Madison, SD, 57042, United States, 1 6052565111, madhu.gottumukkala@trojans.dsu.edu %K information exchange %K interoperability %K care transition %K health information technology %K health information exchange %K open standards %K long-term and postacute care %K LTPAC %K design science research %D 2023 %7 17.2.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Information exchange is essential for transitioning high-quality care between care settings. Inadequate or delayed information exchange can result in medication errors, missed test results, considerable delays in care, and even readmissions. Unfortunately, long-term and postacute care facilities often lag behind other health care facilities in adopting health information technologies, increasing difficulty in facilitating care transitions through electronic information exchange. The research gap is most evident when considering the implications of the inability to electronically transfer patients’ health records between these facilities. Objective: This study aimed to design and evaluate an open standards–based interoperability solution that facilitates seamless bidirectional information exchange between acute care and long-term and postacute care facilities using 2 vendor electronic health record (EHR) systems. Methods: Using the design science research methodology, we designed an interoperability solution that improves the bidirectional information exchange between acute care and long-term care (LTC) facilities using different EHR systems. Different approaches were applied in the study with a focus on the relevance cycle, including eliciting detailed requirements from stakeholders in the health system who understand the complex data formats, constraints, and workflows associated with transferring patient records between 2 different EHR systems. We performed literature reviews and sought experts in the health care industry from different organizations with a focus on the rigor cycle to identify the components relevant to the interoperability solution. The design cycle focused on iterating between the core activities of implementing and evaluating the proposed artifact. The artifact was evaluated at a health care organization with a combined footprint of acute and postacute care operations using 2 different EHR systems. Results: The resulting interoperability solution offered integrations with source systems and was proven to facilitate bidirectional information exchange for patients transferring between an acute care facility using an Epic EHR system and an LTC facility using a PointClickCare EHR system. This solution serves as a proof of concept for bidirectional data exchange between Epic and PointClickCare for medications, yet the solution is designed to expand to additional data elements such as allergies, problem lists, and diagnoses. Conclusions: Historically, the interoperability topic has centered on hospital-to-hospital data exchange, making it more challenging to evaluate the efficacy of data exchange between other care settings. In acute and LTC settings, there are differences in patients’ needs and delivery of care workflows that are distinctly unique. In addition, the health care system’s components that offer long-term and acute care in the United States have evolved independently and separately. This study demonstrates that the interoperability solution improves the information exchange between acute and LTC facilities by simplifying data transfer, eliminating manual processes, and reducing data discrepancies using a design science research methodology. %M 36800213 %R 10.2196/43758 %U https://formative.jmir.org/2023/1/e43758 %U https://doi.org/10.2196/43758 %U http://www.ncbi.nlm.nih.gov/pubmed/36800213 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e40685 %T Thought Leader Perspectives on the Benefits, Barriers, and Enablers for Routinely Collected Electronic Health Data to Support Professional Development: Qualitative Study %A Bucalon,Bernard %A Whitelock-Wainwright,Emma %A Williams,Chris %A Conley,Jeanette %A Veysey,Martin %A Kay,Judy %A Shaw,Tim %+ Human Centred Technology Research Cluster, School of Computer Science, The University of Sydney, Level 3, Computer Science Building J12, Sydney, 2006, Australia, 61 2 8627 0010, bernard.bucalon@sydney.edu.au %K practice analytics %K data visualization %K continuing professional development %K professional practice %K reflective practice %K lifelong learning %K electronic health records %K EHR %D 2023 %7 16.2.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Hospitals routinely collect large amounts of administrative data such as length of stay, 28-day readmissions, and hospital-acquired complications; yet, these data are underused for continuing professional development (CPD). First, these clinical indicators are rarely reviewed outside of existing quality and safety reporting. Second, many medical specialists view their CPD requirements as time-consuming, having minimal impact on practice change and improving patient outcomes. There is an opportunity to build new user interfaces based on these data, designed to support individual and group reflection. Data-informed reflective practice has the potential to generate new insights about performance, bridging the gap between CPD and clinical practice. Objective: This study aims to understand why routinely collected administrative data have not yet become widely used to support reflective practice and lifelong learning. Methods: We conducted semistructured interviews (N=19) with thought leaders from a range of backgrounds, including clinicians, surgeons, chief medical officers, information and communications technology professionals, informaticians, researchers, and leaders from related industries. Interviews were thematically analyzed by 2 independent coders. Results: Respondents identified visibility of outcomes, peer comparison, group reflective discussions, and practice change as potential benefits. The key barriers included legacy technology, distrust with data quality, privacy, data misinterpretation, and team culture. Respondents suggested recruiting local champions for co-design, presenting data for understanding rather than information, coaching by specialty group leaders, and timely reflection linked to CPD as enablers to successful implementation. Conclusions: Overall, there was consensus among thought leaders, bringing together insights from diverse backgrounds and medical jurisdictions. We found that clinicians are interested in repurposing administrative data for professional development despite concerns with underlying data quality, privacy, legacy technology, and visual presentation. They prefer group reflection led by supportive specialty group leaders, rather than individual reflection. Our findings provide novel insights into the specific benefits, barriers, and benefits of potential reflective practice interfaces based on these data sets. They can inform the design of new models of in-hospital reflection linked to the annual CPD planning-recording-reflection cycle. %M 36795463 %R 10.2196/40685 %U https://www.jmir.org/2023/1/e40685 %U https://doi.org/10.2196/40685 %U http://www.ncbi.nlm.nih.gov/pubmed/36795463 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e39876 %T The Current Status of Secondary Use of Claims, Electronic Medical Records, and Electronic Health Records in Epidemiology in Japan: Narrative Literature Review %A Zhao,Yang %A Tsubota,Tadashi %+ Audit & Assurance Deloitte Analytics R&D, Deloitte Touche Tohmatsu LLC, Marunouchi Nijubashi Building, 3-2-3 Marunouchi, Chiyoda-ku, Tokyo, 1008360, Japan, 81 80 9350 0848, yang1.zhao@tohmatsu.co.jp %K claims %K electronic medical records %K EMRs %K electronic health records %K EHRs %K epidemiology %K narrative literature review %D 2023 %7 14.2.2023 %9 Review %J JMIR Med Inform %G English %X Background: Real-world data, such as claims, electronic medical records (EMRs), and electronic health records (EHRs), are increasingly being used in clinical epidemiology. Understanding the current status of existing approaches can help in designing high-quality epidemiological studies. Objective: We conducted a comprehensive narrative literature review to clarify the secondary use of claims, EMRs, and EHRs in clinical epidemiology in Japan. Methods: We searched peer-reviewed publications in PubMed from January 1, 2006, to June 30, 2021 (the date of search), which met the following 3 inclusion criteria: involvement of claims, EMRs, EHRs, or medical receipt data; mention of Japan; and published from January 1, 2006, to June 30, 2021. Eligible articles that met any of the following 6 exclusion criteria were filtered: review articles; non–disease-related articles; articles in which the Japanese population is not the sample; articles without claims, EMRs, or EHRs; full text not available; and articles without statistical analysis. Investigations of the titles, abstracts, and full texts of eligible articles were conducted automatically or manually, from which 7 categories of key information were collected. The information included organization, study design, real-world data type, database, disease, outcome, and statistical method. Results: A total of 620 eligible articles were identified for this narrative literature review. The results of the 7 categories suggested that most of the studies were conducted by academic institutes (n=429); the cohort study was the primary design that longitudinally measured outcomes of proper patients (n=533); 594 studies used claims data; the use of databases was concentrated in well-known commercial and public databases; infections (n=105), cardiovascular diseases (n=100), neoplasms (n=78), and nutritional and metabolic diseases (n=75) were the most studied diseases; most studies have focused on measuring treatment patterns (n=218), physiological or clinical characteristics (n=184), and mortality (n=137); and multivariate models were commonly used (n=414). Most (375/414, 90.6%) of these multivariate modeling studies were performed for confounder adjustment. Logistic regression was the first choice for assessing many of the outcomes, with the exception of hospitalization or hospital stay and resource use or costs, for both of which linear regression was commonly used. Conclusions: This literature review provides a good understanding of the current status and trends in the use of claims, EMRs, and EHRs data in clinical epidemiology in Japan. The results demonstrated appropriate statistical methods regarding different outcomes, Japan-specific trends of disease areas, and the lack of use of artificial intelligence techniques in existing studies. In the future, a more precise comparison of relevant domestic research with worldwide research will be conducted to clarify the Japan-specific status and challenges. %M 36787161 %R 10.2196/39876 %U https://medinform.jmir.org/2023/1/e39876 %U https://doi.org/10.2196/39876 %U http://www.ncbi.nlm.nih.gov/pubmed/36787161 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e43486 %T Optimizing the Implementation of Clinical Predictive Models to Minimize National Costs: Sepsis Case Study %A Rogers,Parker %A Boussina,Aaron E %A Shashikumar,Supreeth P %A Wardi,Gabriel %A Longhurst,Christopher A %A Nemati,Shamim %+ Department of Economics, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA, 92093, United States, 1 405 850 4751, parogers@ucsd.edu %K sepsis %K machine learning %K evaluation %K utility assessment %K workflow simulation %K simulation %K model %K implementation %K data %K acute kidney injury %K injury %K technology %K care %K diagnosis %K clinical %K cost %D 2023 %7 13.2.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Sepsis costs and incidence vary dramatically across diagnostic categories, warranting a customized approach for implementing predictive models. Objective: The aim of this study was to optimize the parameters of a sepsis prediction model within distinct patient groups to minimize the excess cost of sepsis care and analyze the potential effect of factors contributing to end-user response to sepsis alerts on overall model utility. Methods: We calculated the excess costs of sepsis to the Centers for Medicare and Medicaid Services (CMS) by comparing patients with and without a secondary sepsis diagnosis but with the same primary diagnosis and baseline comorbidities. We optimized the parameters of a sepsis prediction algorithm across different diagnostic categories to minimize these excess costs. At the optima, we evaluated diagnostic odds ratios and analyzed the impact of compliance factors such as noncompliance, treatment efficacy, and tolerance for false alarms on the net benefit of triggering sepsis alerts. Results: Compliance factors significantly contributed to the net benefit of triggering a sepsis alert. However, a customized deployment policy can achieve a significantly higher diagnostic odds ratio and reduced costs of sepsis care. Implementing our optimization routine with powerful predictive models could result in US $4.6 billion in excess cost savings for CMS. Conclusions: We designed a framework for customizing sepsis alert protocols within different diagnostic categories to minimize excess costs and analyzed model performance as a function of false alarm tolerance and compliance with model recommendations. We provide a framework that CMS policymakers could use to recommend minimum adherence rates to the early recognition and appropriate care of sepsis that is sensitive to hospital department-level incidence rates and national excess costs. Customizing the implementation of clinical predictive models by accounting for various behavioral and economic factors may improve the practical benefit of predictive models. %M 36780203 %R 10.2196/43486 %U https://www.jmir.org/2023/1/e43486 %U https://doi.org/10.2196/43486 %U http://www.ncbi.nlm.nih.gov/pubmed/36780203 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 9 %N %P e41450 %T Small Area Forecasting of Opioid-Related Mortality: Bayesian Spatiotemporal Dynamic Modeling Approach %A Bauer,Cici %A Zhang,Kehe %A Li,Wenjun %A Bernson,Dana %A Dammann,Olaf %A LaRochelle,Marc R %A Stopka,Thomas J %+ Department of Biostatistics and Data Science, School of Public Health, The University of Texas Health Science Center at Houston, 1200 Pressler Street, Room E819, Houston, TX, 77030, United States, 1 713 500 9581, cici.x.bauer@uth.tmc.edu %K opioid-related mortality %K small area estimation %K spatiotemporal models %K Bayesian %K forecasting %D 2023 %7 10.2.2023 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: Opioid-related overdose mortality has remained at crisis levels across the United States, increasing 5-fold and worsened during the COVID-19 pandemic. The ability to provide forecasts of opioid-related mortality at granular geographical and temporal scales may help guide preemptive public health responses. Current forecasting models focus on prediction on a large geographical scale, such as states or counties, lacking the spatial granularity that local public health officials desire to guide policy decisions and resource allocation. Objective: The overarching objective of our study was to develop Bayesian spatiotemporal dynamic models to predict opioid-related mortality counts and rates at temporally and geographically granular scales (ie, ZIP Code Tabulation Areas [ZCTAs]) for Massachusetts. Methods: We obtained decedent data from the Massachusetts Registry of Vital Records and Statistics for 2005 through 2019. We developed Bayesian spatiotemporal dynamic models to predict opioid-related mortality across Massachusetts’ 537 ZCTAs. We evaluated the prediction performance of our models using the one-year ahead approach. We investigated the potential improvement of prediction accuracy by incorporating ZCTA-level demographic and socioeconomic determinants. We identified ZCTAs with the highest predicted opioid-related mortality in terms of rates and counts and stratified them by rural and urban areas. Results: Bayesian dynamic models with the full spatial and temporal dependency performed best. Inclusion of the ZCTA-level demographic and socioeconomic variables as predictors improved the prediction accuracy, but only in the model that did not account for the neighborhood-level spatial dependency of the ZCTAs. Predictions were better for urban areas than for rural areas, which were more sparsely populated. Using the best performing model and the Massachusetts opioid-related mortality data from 2005 through 2019, our models suggested a stabilizing pattern in opioid-related overdose mortality in 2020 and 2021 if there were no disruptive changes to the trends observed for 2005-2019. Conclusions: Our Bayesian spatiotemporal models focused on opioid-related overdose mortality data facilitated prediction approaches that can inform preemptive public health decision-making and resource allocation. While sparse data from rural and less populated locales typically pose special challenges in small area predictions, our dynamic Bayesian models, which maximized information borrowing across geographic areas and time points, were used to provide more accurate predictions for small areas. Such approaches can be replicated in other jurisdictions and at varying temporal and geographical levels. We encourage the formation of a modeling consortium for fatal opioid-related overdose predictions, where different modeling techniques could be ensembled to inform public health policy. %M 36763450 %R 10.2196/41450 %U https://publichealth.jmir.org/2023/1/e41450 %U https://doi.org/10.2196/41450 %U http://www.ncbi.nlm.nih.gov/pubmed/36763450 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e41344 %T Functional Requirements for Medical Data Integration into Knowledge Management Environments: Requirements Elicitation Approach Based on Systematic Literature Analysis %A Kinast,Benjamin %A Ulrich,Hannes %A Bergh,Björn %A Schreiweis,Björn %+ Institute for Medical Informatics and Statistics, Kiel University and University Hospital Schleswig-Holstein, Arnold-Heller-Straße 3, Kiel, 24105, Germany, 49 431500 ext 31601, benjamin.kinast@uksh.de %K data integration %K requirements engineering %K requirements %K knowledge management %K software engineering %D 2023 %7 9.2.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: In patient care, data are historically generated and stored in heterogeneous databases that are domain specific and often noninteroperable or isolated. As the amount of health data increases, the number of isolated data silos is also expected to grow, limiting the accessibility of the collected data. Medical informatics is developing ways to move from siloed data to a more harmonized arrangement in information architectures. This paradigm shift will allow future research to integrate medical data at various levels and from various sources. Currently, comprehensive requirements engineering is working on data integration projects in both patient care– and research-oriented contexts, and it is significantly contributing to the success of such projects. In addition to various stakeholder-based methods, document-based requirement elicitation is a valid method for improving the scope and quality of requirements. Objective: Our main objective was to provide a general catalog of functional requirements for integrating medical data into knowledge management environments. We aimed to identify where integration projects intersect to derive consistent and representative functional requirements from the literature. On the basis of these findings, we identified which functional requirements for data integration exist in the literature and thus provide a general catalog of requirements. Methods: This work began by conducting a literature-based requirement elicitation based on a broad requirement engineering approach. Thus, in the first step, we performed a web-based systematic literature review to identify published articles that dealt with the requirements for medical data integration. We identified and analyzed the available literature by applying the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. In the second step, we screened the results for functional requirements using the requirements engineering method of document analysis and derived the requirements into a uniform requirement syntax. Finally, we classified the elicited requirements into a category scheme that represents the data life cycle. Results: Our 2-step requirements elicitation approach yielded 821 articles, of which 61 (7.4%) were included in the requirement elicitation process. There, we identified 220 requirements, which were covered by 314 references. We assigned the requirements to different data life cycle categories as follows: 25% (55/220) to data acquisition, 35.9% (79/220) to data processing, 12.7% (28/220) to data storage, 9.1% (20/220) to data analysis, 6.4% (14/220) to metadata management, 2.3% (5/220) to data lineage, 3.2% (7/220) to data traceability, and 5.5% (12/220) to data security. Conclusions: The aim of this study was to present a cross-section of functional data integration–related requirements defined in the literature by other researchers. The aim was achieved with 220 distinct requirements from 61 publications. We concluded that scientific publications are, in principle, a reliable source of information for functional requirements with respect to medical data integration. Finally, we provide a broad catalog to support other scientists in the requirement elicitation phase. %M 36757764 %R 10.2196/41344 %U https://www.jmir.org/2023/1/e41344 %U https://doi.org/10.2196/41344 %U http://www.ncbi.nlm.nih.gov/pubmed/36757764 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e42796 %T Experiences and Expectations of Information and Communication Technologies in Flexible Assertive Community Treatment Teams: Qualitative Study %A Bønes,Erlend %A Granja,Conceição %A Solvoll,Terje %+ Norwegian Centre for e-Health Research, University Hospital of North Norway, PO Box 35, Tromsø, 9019, Norway, 47 97655680, erlend.bones@ehealthresearch.no %K mental health %K FACT %K electronic health records %K eHealth %K EHR %K electronic whiteboards %K community %K treatment %K qualitative %K COVID-19 %K patient care %K mental illness %K information technology %K thematic analysis %K data access %K information and communication solutions %K ICT %K Norway %K semistructured interviews %D 2023 %7 9.2.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Flexible Assertive Community Treatment (FACT) is a model of integrated care for patients with long-term serious mental illness. FACT teams deliver services using assertive outreach to treat patients who can be hard to reach by the health care service, and focus on both the patient’s health and their social situation. However, in Norway, FACT team members have challenges with their information and communication (ICT) solutions. Objective: The aim of this study was to explore Norwegian FACT teams’ experiences and expectations of their ICT solutions, including electronic health records, electronic whiteboards, and calendars. Methods: We gathered data in two phases. In the first phase, we conducted semistructured interviews with team leaders and team coordinators, and made observations in FACT teams targeting adults. In the second phase, we conducted semistructured group interviews in FACT teams targeting youth. We performed a thematic analysis of the data in a theoretical manner to address the specific objectives of the study. Results: A total of 8 teams were included, with 5 targeting adults and 3 targeting youth. Due to the COVID-19 pandemic, we were not able to perform observations in 2 of the teams targeting adults. Team leaders and coordinators in all 5 teams targeting adults were interviewed, with a total of 7 team members participating in the teams targeting youth. We found various challenges with communication, documentation, and organization for FACT teams. The COVID-19 pandemic was challenging for the teams and changed the way they used ICT solutions. There were issues with some technical solutions used in the teams, including electronic health records, electronic whiteboards, and calendars. Lack of integration and access to data were some of the main issues identified. Conclusions: Despite the FACT model being successfully implemented in Norway, there are several issues regarding the ICT solutions they use, mainly related to access to data and integration. Further research is required to detail how improved ICT solutions should be designed. While FACT teams targeting adults and youth differ in some ways, their needs for ICT solutions are largely similar. %M 36730062 %R 10.2196/42796 %U https://formative.jmir.org/2023/1/e42796 %U https://doi.org/10.2196/42796 %U http://www.ncbi.nlm.nih.gov/pubmed/36730062 %0 Journal Article %@ 2561-9128 %I JMIR Publications %V 6 %N %P e40455 %T A Neural Network Model Using Pain Score Patterns to Predict the Need for Outpatient Opioid Refills Following Ambulatory Surgery: Algorithm Development and Validation %A Gabriel,Rodney Allanigue %A Simpson,Sierra %A Zhong,William %A Burton,Brittany Nicole %A Mehdipour,Soraya %A Said,Engy Tadros %+ Department of Anesthesiology, University of California San Diego, 9400 Campus Pt Dr, La Jolla, CA, 92037, United States, 1 8586637747, ragabriel@health.ucsd.edu %K opioids %K ambulatory surgery %K machine learning %K surgery %K outpatient %K pain medication %K pain %K pain management %K patient needs %K predict %K algorithms %K clinical decision support %K pain care %D 2023 %7 8.2.2023 %9 Original Paper %J JMIR Perioper Med %G English %X Background: Expansion of clinical guidance tools is crucial to identify patients at risk of requiring an opioid refill after outpatient surgery. Objective: The objective of this study was to develop machine learning algorithms incorporating pain and opioid features to predict the need for outpatient opioid refills following ambulatory surgery. Methods: Neural networks, regression, random forest, and a support vector machine were used to evaluate the data set. For each model, oversampling and undersampling techniques were implemented to balance the data set. Hyperparameter tuning based on k-fold cross-validation was performed, and feature importance was ranked based on a Shapley Additive Explanations (SHAP) explainer model. To assess performance, we calculated the average area under the receiver operating characteristics curve (AUC), F1-score, sensitivity, and specificity for each model. Results: There were 1333 patients, of whom 144 (10.8%) refilled their opioid prescription within 2 weeks after outpatient surgery. The average AUC calculated from k-fold cross-validation was 0.71 for the neural network model. When the model was validated on the test set, the AUC was 0.75. The features with the highest impact on model output were performance of a regional nerve block, postanesthesia care unit maximum pain score, postanesthesia care unit median pain score, active smoking history, and total perioperative opioid consumption. Conclusions: Applying machine learning algorithms allows providers to better predict outcomes that require specialized health care resources such as transitional pain clinics. This model can aid as a clinical decision support for early identification of at-risk patients who may benefit from transitional pain clinic care perioperatively in ambulatory surgery. %M 36753316 %R 10.2196/40455 %U https://periop.jmir.org/2023/1/e40455 %U https://doi.org/10.2196/40455 %U http://www.ncbi.nlm.nih.gov/pubmed/36753316 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e43734 %T Explainable Machine Learning Techniques To Predict Amiodarone-Induced Thyroid Dysfunction Risk: Multicenter, Retrospective Study With External Validation %A Lu,Ya-Ting %A Chao,Horng-Jiun %A Chiang,Yi-Chun %A Chen,Hsiang-Yin %+ Department of Clinical Pharmacy, School of Pharmacy, Taipei Medical University, R714, 7th Floor, Health and Science Building No.250, Wuxing Street, Xinyi Distict, Taipei, 110, Taiwan, 886 2 2736 1661 ext 6175, shawn@tmu.edu.tw %K amiodarone %K thyroid dysfunction %K machine learning %K oversampling %K extreme gradient boosting %K adverse effect %K resampling %K thyroid %K predict %K risk %D 2023 %7 7.2.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Machine learning offers new solutions for predicting life-threatening, unpredictable amiodarone-induced thyroid dysfunction. Traditional regression approaches for adverse-effect prediction without time-series consideration of features have yielded suboptimal predictions. Machine learning algorithms with multiple data sets at different time points may generate better performance in predicting adverse effects. Objective: We aimed to develop and validate machine learning models for forecasting individualized amiodarone-induced thyroid dysfunction risk and to optimize a machine learning–based risk stratification scheme with a resampling method and readjustment of the clinically derived decision thresholds. Methods: This study developed machine learning models using multicenter, delinked electronic health records. It included patients receiving amiodarone from January 2013 to December 2017. The training set was composed of data from Taipei Medical University Hospital and Wan Fang Hospital, while data from Taipei Medical University Shuang Ho Hospital were used as the external test set. The study collected stationary features at baseline and dynamic features at the first, second, third, sixth, ninth, 12th, 15th, 18th, and 21st months after amiodarone initiation. We used 16 machine learning models, including extreme gradient boosting, adaptive boosting, k-nearest neighbor, and logistic regression models, along with an original resampling method and 3 other resampling methods, including oversampling with the borderline-synthesized minority oversampling technique, undersampling–edited nearest neighbor, and over- and undersampling hybrid methods. The model performance was compared based on accuracy; Precision, recall, F1-score, geometric mean, area under the curve of the receiver operating characteristic curve (AUROC), and the area under the precision-recall curve (AUPRC). Feature importance was determined by the best model. The decision threshold was readjusted to identify the best cutoff value and a Kaplan-Meier survival analysis was performed. Results: The training set contained 4075 patients from Taipei Medical University Hospital and Wan Fang Hospital, of whom 583 (14.3%) developed amiodarone-induced thyroid dysfunction, while the external test set included 2422 patients from Taipei Medical University Shuang Ho Hospital, of whom 275 (11.4%) developed amiodarone-induced thyroid dysfunction. The extreme gradient boosting oversampling machine learning model demonstrated the best predictive outcomes among all 16 models. The accuracy; Precision, recall, F1-score, G-mean, AUPRC, and AUROC were 0.923, 0.632, 0.756, 0.688, 0.845, 0.751, and 0.934, respectively. After readjusting the cutoff, the best value was 0.627, and the F1-score reached 0.699. The best threshold was able to classify 286 of 2422 patients (11.8%) as high-risk subjects, among which 275 were true-positive patients in the testing set. A shorter treatment duration; higher levels of thyroid-stimulating hormone and high-density lipoprotein cholesterol; and lower levels of free thyroxin, alkaline phosphatase, and low-density lipoprotein were the most important features. Conclusions: Machine learning models combined with resampling methods can predict amiodarone-induced thyroid dysfunction and serve as a support tool for individualized risk prediction and clinical decision support. %M 36749620 %R 10.2196/43734 %U https://www.jmir.org/2023/1/e43734 %U https://doi.org/10.2196/43734 %U http://www.ncbi.nlm.nih.gov/pubmed/36749620 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e41614 %T Unique Device Identification–Based Linkage of Hierarchically Accessible Data Domains in Prospective Surgical Hospital Data Ecosystems: User-Centered Design Approach %A Kozak,Karol %A Seidel,André %A Matvieieva,Nataliia %A Neupetsch,Constanze %A Teicher,Uwe %A Lemme,Gordon %A Ben Achour,Anas %A Barth,Martin %A Ihlenfeldt,Steffen %A Drossel,Welf-Guntram %+ Fraunhofer Institute for Machine Tools and Forming Technology IWU, Pforzheimer Straße 7a, Dresden, 01187, Germany, 49 351 4772 ext 2620, uwe.teicher@iwu.fraunhofer.de %K electronic health record %K unique device identification %K cyber-physical production systems %K mHealth %K data integration ecosystem %K hierarchical data access %K shell embedded role model %D 2023 %7 27.1.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: The electronic health record (EHR) targets systematized collection of patient-specific, electronically stored health data. The EHR is an evolving concept driven by ongoing developments and open or unclear legal issues concerning medical technologies, cross-domain data integration, and unclear access roles. Consequently, an interdisciplinary discourse based on representative pilot scenarios is required to connect previously unconnected domains. Objective: We address cross-domain data integration including access control using the specific example of a unique device identification (UDI)–expanded hip implant. In fact, the integration of technical focus data into the hospital information system (HIS) is considered based on surgically relevant information. Moreover, the acquisition of social focus data based on mobile health (mHealth) is addressed, covering data integration and networking with therapeutic intervention and acute diagnostics data. Methods: In addition to the additive manufacturing of a hip implant with the integration of a UDI, we built a database that combines database technology and a wrapper layer known from extract, transform, load systems and brings it into a SQL database, WEB application programming interface (API) layer (back end), interface layer (rest API), and front end. It also provides semantic integration through connection mechanisms between data elements. Results: A hip implant is approached by design, production, and verification while linking operation-relevant specifics like implant-bone fit by merging patient-specific image material (computed tomography, magnetic resonance imaging, or a biomodel) and the digital implant twin for well-founded selection pairing. This decision-facilitating linkage, which improves surgical planning, relates to patient-specific postoperative influencing factors during the healing phase. A unique product identification approach is presented, allowing a postoperative read-out with state-of-the-art hospital technology while enabling future access scenarios for patient and implant data. The latter was considered from the manufacturing perspective using the process manufacturing chain for a (patient-specific) implant to identify quality-relevant data for later access. In addition, sensor concepts were identified to use to monitor the patient-implant interaction during the healing phase using wearables, for example. A data aggregation and integration concept for heterogeneous data sources from the considered focus domains is also presented. Finally, a hierarchical data access concept is shown, protecting sensitive patient data from misuse using existing scenarios. Conclusions: Personalized medicine requires cross-domain linkage of data, which, in turn, require an appropriate data infrastructure and adequate hierarchical data access solutions in a shared and federated data space. The hip implant is used as an example for the usefulness of cross-domain data linkage since it bundles social, medical, and technical aspects of the implantation. It is necessary to open existing databases using interfaces for secure integration of data from end devices and to assure availability through suitable access models while guaranteeing long-term, independent data persistence. A suitable strategy requires the combination of technical solutions from the areas of identity and trust, federated data storage, cryptographic procedures, and software engineering as well as organizational changes. %M 36705946 %R 10.2196/41614 %U https://medinform.jmir.org/2023/1/e41614 %U https://doi.org/10.2196/41614 %U http://www.ncbi.nlm.nih.gov/pubmed/36705946 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e42151 %T The Usability of Homelab, a Digital Self-service at a Dutch General Practice, for Diagnostic Tests: Pilot Study With a Questionnaire %A Schnoor,Kyma %A Versluis,Anke %A Chavannes,Niels H %A Talboom-Kamp,Esther P W A %+ Public Health and Primary Care, Leiden University Medical Center, Hippocratespad 21, Leiden, 2333 ZA, Netherlands, 31 71526 8433, k.schnoor@lumc.nl %K eHealth %K diagnostic testing %K general practitioner %K general practice %K GP %K referral %K online testing %K diagnostic %K laboratory test %K usability %K digital health %K health care service %K service delivery %D 2023 %7 26.1.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: eHealth potentially can make health care more accessible and efficient and help reduce the workload in primary health care. Homelab is an eHealth tool implemented in a general practice environment, and it offers relatively simple laboratory diagnostic tests without the referral of the general practitioner. After logging in this eHealth tool, patients select and order a diagnostic test based on their symptoms. The test results are presented online to the general practitioner and the patient. Objective: This study aims to evaluate the use, usability, and user characteristics of Homelab. Further, it aims to evaluate whether Homelab replaces an appointment with the general practitioner. Methods: Homelab has been implemented since May 2021 as a pilot in a Dutch general practice. The number of requests and the ordered diagnostic packages are monitored. After using Homelab, patients are invited to complete a short questionnaire. The questionnaire contains demographic questions and assesses usability using the System Usability Scale (10 items). In addition, questions about requesting an appointment with the general practitioner without Homelab are included. All data were anonymous. Results: The questionnaire was filled by 74 individual patients. The mean age of the patients was 40.33 (SD 12.11) years, and half of them were females (39/74, 53%). The majority of the patients were highly educated (56/74, 76%) and employed (53/74, 72%). Approximately 81% (60/74) of the patients reported that they would use Homelab again in the future and 66% (49/74) reported that they would have gone to the general practitioner if they had not used Homelab. The usability of Homelab was perceived higher by the younger age group (mean 73.96, SD 14.74) than by the older age group (mean 61.59, SD 14.37). In total, 106 test packages were ordered over 1 year, and the most requested diagnostic package was “Am I still healthy? I want to do my annual health checkup.” Homelab was used the most during the months of the COVID-19 lockdown. Conclusions: The use of Homelab, a digital self-service for ordering diagnostic tests, was monitored in this study, and its usability was perceived as above average. Our findings showed that patients are willing to use Homelab in the future and they would use it most of the time as a replacement for regular consultations. Homelab offers opportunities for more accessible and efficient health care for both the patient and the general practitioner. %M 36701183 %R 10.2196/42151 %U https://formative.jmir.org/2023/1/e42151 %U https://doi.org/10.2196/42151 %U http://www.ncbi.nlm.nih.gov/pubmed/36701183 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e41706 %T Recording of Social Determinants in Computerized Medical Records in Primary Care Consultations: Quasi-Experimental Study %A Rodoreda-Pallàs,Berta %A Lumillo-Gutiérrez,Iris %A Miró Catalina,Queralt %A Torra Escarrer,Eva %A Sanahuja Juncadella,Jaume %A Morin Fraile,Victoria %+ Health Promotion in Rural Areas Research Group, Institut Català de la Salut, Carrer Pica d'Estats, 36,, Sant Fruitós de Bages, 08272, Spain, 34 936930040, brodoreda.cc.ics@gencat.cat %K recording of social determinants of health %K computerised medical records %K electronic health record coding %K non-clinical diagnoses %K Z-coding %K primary care %K medical records %K intervention %K medical %K treatment %D 2023 %7 25.1.2023 %9 Original Paper %J JMIR Form Res %G English %X Background:  Social determinants of health may be more important than medical or lifestyle choices in influencing people's health. Even so, there is a deficit in recording these in patients' computerized medical histories. The Spanish administration and the World Health Organization are promoting the recording of diagnoses in computerized clinical histories with the aim of benefiting the individual, the professional, and the community. In most cases, professionals tend to record only clinical diagnoses despite evidence in the literature documenting that addressing the social determinants of health can lead to improvements in health and reductions in social disparities in disease. Objective:  This study aims to develop and evaluate the effectiveness of a mixed intervention (face-to-face-digital) aimed at improving the quantity and quality of the records of the social determinants of health in computerized medical records at primary care clinics. Methods:  A quasi-experimental, nonrandomized, controlled, multicenter study with 2 parallel study arms was conducted in the area of Central Catalonia (Spain) with primary care professionals of the Institut Català de la Salut (ICS), working from September 23, 2019, to March 31, 2020. All interested professionals were accepted. In total, 22 basic health areas were involved in the study. In Spain and Catalonia, the International Classification of Diseases is used, in which there is a coding of the social determinants of health. Five social determinants were selected by a physician, a nurse, and a social worker; these professionals had experience in primary care and were experts in community health. The choice was made taking into account the ease of use, benefit, and existing terminology. The intervention, based on the integration of a checklist, was integrated as part of the usual multidisciplinary clinical workflow in primary care consultations to influence the recording of these determinants in the patient's computerized medical record. Results:  After 6 months of implementing the intervention, the volume and quantity of records of 5 social determinants of health were compared, and a significant increase in the median number of pre- and postintervention diagnoses was observed (P≤.001). There was also an increase in the diversity of selected social determinants. Using the linear regression model, the significant mean increase of the experimental group with respect to the control group was estimated with a coefficient of 8.18 (95% CI 5.11-11.26). Conclusions:  The intervention described in this study is an effective tool for coding the social determinants of health designed by a multidisciplinary team to be incorporated into the workflow of primary care practices. The effectiveness of its usability and the description of the intervention described here should be generalizable to any environment. Trial Registration: ClinicalTrials.gov NCT04151056; https://clinicaltrials.gov/ct2/show/NCT04151056 %M 36696168 %R 10.2196/41706 %U https://formative.jmir.org/2023/1/e41706 %U https://doi.org/10.2196/41706 %U http://www.ncbi.nlm.nih.gov/pubmed/36696168 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e43028 %T The Ability of Austrian Qualified Physiotherapists to Make Accurate Keep-Refer Decisions and to Detect Serious Pathologies Based on Clinical Vignettes: Protocol for a Cross-sectional Web-Based Survey %A Lackenbauer,Wolfgang %A Gasselich,Simon %A Lickel,Martina Edda %A Beikircher,Reinhard %A Keip,Christian %A Rausch,Florian %A Wieser,Manfred %A Selfe,James %A Janssen,Jessie %+ Institute of Therapeutic and Midwifery Sciences, Department of Health Sciences, University of Applied Sciences Krems, Piaristengasse 1, Krems, 3500, Austria, 43 2732 802 0, wolfgang.lackenbauer@fh-krems.ac.at %K red flags %K clinical reasoning %K physiotherapy %K screening %K referral %K musculoskeletal %D 2023 %7 24.1.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: The recognition of serious pathologies affecting the musculoskeletal (MSK) system, especially in the early stage of a disease, is an important but challenging task. The prevalence of such serious pathologies is currently low. However, in our progressing aging population, it is anticipated that serious pathologies affecting the MSK system will be on the rise. Physiotherapists, as part of a wider health care team, can play a valuable role in the recognition of serious pathologies. It is at present unknown how accurately Austrian qualified physiotherapists can detect the presence of serious pathologies affecting the MSK system and therefore determine whether physiotherapy management is indicated (keep patients) or not (refer patients to a medical doctor). Objective: We will explore the current ability of Austrian qualified physiotherapists to recognize serious pathologies by using validated clinical vignettes. Methods: As part of an electronic web-based survey, these vignettes will be distributed among a convenience sample of qualified Austrian physiotherapists working in a hospital or private setting. The survey will consist of four sections: (1) demographics and general information, (2) the clinical vignettes, (3) questions concerning the clinical vignettes, and (4) self-perceived knowledge gaps and learning preferences from the perspective of study participants. Results will further be used for (1) international comparison with similar studies from the existing literature and (2) gaining insight into the participants’ self-perceived knowledge gaps and learning preferences for increasing their knowledge level about keep-refer decision-making and detecting serious pathologies. Results: Data collection took place between May 2022 and June 2022. As of June 2022, a total of 479 Austrian physiotherapists completed the survey. Data analysis has started, and we aim to publish the results in 2023. Conclusions: The results of this survey will provide insights into the ability of Austrian physiotherapists to make accurate keep-refer decisions and to recognize the presence of serious pathologies using clinical vignettes. The results of this survey are expected to serve as a basis for future training in this area. International Registered Report Identifier (IRRID): DERR1-10.2196/43028 %M 36692940 %R 10.2196/43028 %U https://www.researchprotocols.org/2023/1/e43028 %U https://doi.org/10.2196/43028 %U http://www.ncbi.nlm.nih.gov/pubmed/36692940 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e38861 %T A Linked Open Data–Based Terminology to Describe Libre/Free and Open-source Software: Incremental Development Study %A Jahn,Franziska %A Ammenwerth,Elske %A Dornauer,Verena %A Höffner,Konrad %A Bindel,Michelle %A Karopka,Thomas %A Winter,Alfred %+ Institute of Medical Informatics, Statistics and Epidemiology, Faculty of Medicine, Leipzig University, Härtelstr. 16-18, Leipzig, 04107, Germany, 49 341 97 16194, franziska.jahn@imise.uni-leipzig.de %K health informatics %K ontology %K free/libre open-source software %K software applications %K health IT %K terminology %D 2023 %7 20.1.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: There is a variety of libre/free and open-source software (LIFOSS) products for medicine and health care. To support health care and IT professionals select an appropriate software product for given tasks, several comparison studies and web platforms, such as Medfloss.org, are available. However, due to the lack of a uniform terminology for health informatics, ambiguous or imprecise terms are used to describe the functionalities of LIFOSS. This makes comparisons of LIFOSS difficult and may lead to inappropriate software selection decisions. Using Linked Open Data (LOD) promises to address these challenges. Objective: We describe LIFOSS systematically with the help of the underlying Health Information Technology Ontology (HITO). We publish HITO and HITO-based software product descriptions using LOD to obtain the following benefits: (1) linking and reusing existing terminologies and (2) using Semantic Web tools for viewing and querying the LIFOSS data on the World Wide Web. Methods: HITO was incrementally developed and implemented. First, classes for the description of software products in health IT evaluation studies were identified. Second, requirements for describing LIFOSS were elicited by interviewing domain experts. Third, to describe domain-specific functionalities of software products, existing catalogues of features and enterprise functions were analyzed and integrated into the HITO knowledge base. As a proof of concept, HITO was used to describe 25 LIFOSS products. Results: HITO provides a defined set of classes and their relationships to describe LIFOSS in medicine and health care. With the help of linked or integrated catalogues for languages, programming languages, licenses, features, and enterprise functions, the functionalities of LIFOSS can be precisely described and compared. We publish HITO and the LIFOSS descriptions as LOD; they can be queried and viewed using different Semantic Web tools, such as Resource Description Framework (RDF) browsers, SPARQL Protocol and RDF Query Language (SPARQL) queries, and faceted searches. The advantages of providing HITO as LOD are demonstrated by practical examples. Conclusions: HITO is a building block to achieving unambiguous communication among health IT professionals and researchers. Providing LIFOSS product information as LOD enables barrier-free and easy access to data that are often hidden in user manuals of software products or are not available at all. Efforts to establish a unique terminology of medical and health informatics should be further supported and continued. %M 36662569 %R 10.2196/38861 %U https://medinform.jmir.org/2023/1/e38861 %U https://doi.org/10.2196/38861 %U http://www.ncbi.nlm.nih.gov/pubmed/36662569 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e41820 %T Investigation and Countermeasures Research of Hospital Information Construction of Tertiary Class-A Public Hospitals in China: Questionnaire Study %A Shu,Chang %A Chen,Yueyue %A Yang,Huiyuan %A Tao,Ran %A Chen,Xiaoping %A Yu,Jingjing %+ Hepatic Surgery Center, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, 1095 Jiefang Avenue, Wuhan, 430030, China, 86 83662495, yujingjingtjh@163.com %K public hospital %K hospital information construction %K current situation %K development %K countermeasures %D 2023 %7 20.1.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Medical informatization has initially demonstrated its advantages in improving the medical service industry. Over the past decade, the Chinese government have made a lot of effort to complete infrastructural information construction in the medical and health domain, and smart hospitals will be the next priority according to policies released by Chinese government in recent years. Objective: To provide strategic support for further development of medical information construction in China, this study aimed to investigate the current situation of medical information construction in tertiary class-A public hospitals and analyze the existing problems and countermeasures. Methods: This study surveyed 23 tertiary class-A public hospitals in China who voluntarily responded to a self-designed questionnaire distributed in April 2020 to investigate the current medical information construction status. Descriptive statistics were used to summarize the current configurations of hospital information department, hospital information systems, hospital internet service and its application, and the satisfaction of hospital information construction. Interviews were also conducted with the respondents in this study for requirement analysis. Results: The results show that hospital information construction has become one of the priorities of the hospitals’ daily work, and the medical information infrastructural construction and internet service application of the hospitals are good; however, a remarkable gap among the different level of hospitals can be observed. Although most hospitals had built their own IT team to undertake information construction work, the actual utilization rate of big data collected and stored in the hospital information system was not satisfactory. Conclusions: Support for the construction of information technology in primary care institutions should be increased to balance the level of development of medical informatization in medical institutions at all levels. The training of complex talents with both IT and medical backgrounds should be emphasized, and specialized disease information standards should be developed to lay a solid data foundation for data utilization and improve the utilization of medical big data. %M 36662565 %R 10.2196/41820 %U https://formative.jmir.org/2023/1/e41820 %U https://doi.org/10.2196/41820 %U http://www.ncbi.nlm.nih.gov/pubmed/36662565 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e38184 %T Development and Evaluation of Health Recommender Systems: Systematic Scoping Review and Evidence Mapping %A Sun,Yue %A Zhou,Jia %A Ji,Mengmeng %A Pei,Lusi %A Wang,Zhiwen %+ School of Nursing, Peking University, No. 38 Xueyuan Road, Haidian District, Beijng, 100191, China, 86 15901566817, wzwjing@sina.com %K health recommender systems %K systematic review %K evidence map %K scoping review %K recommender system %D 2023 %7 19.1.2023 %9 Review %J J Med Internet Res %G English %X Background: Health recommender systems (HRSs) are information retrieval systems that provide users with relevant items according to the users’ needs, which can motivate and engage users to change their behavior. Objective: This study aimed to identify the development and evaluation of HRSs and create an evidence map. Methods: A total of 6 databases were searched to identify HRSs reported in studies from inception up to June 30, 2022, followed by forward citation and grey literature searches. Titles, abstracts, and full texts were screened independently by 2 reviewers, with discrepancies resolved by a third reviewer, when necessary. Data extraction was performed by one reviewer and checked by a second reviewer. This review was conducted in accordance with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) statement. Results: A total of 51 studies were included for data extraction. Recommender systems were used across different health domains, such as general health promotion, lifestyle, and generic health service. A total of 23 studies had reported the use of a combination of recommender techniques, classified as hybrid recommender systems, which are the most commonly used recommender techniques in HRSs. In the HRS design stage, only 10 of 51 (19.6%) recommender systems considered personal preferences of end users in the design or development of the system; a total of 29 studies reported the user interface of HRSs, and most HRSs worked on users’ mobile interfaces, usually a mobile app. Two categories of HRS evaluations were used, and evaluations of HRSs varied greatly; 62.7% (32/51) of the studies used the offline evaluations using computational methods (no user), and 33.3% (17/51) of the studies included end users in their HRS evaluation. Conclusions: Through this scoping review, nonmedical professionals and policy makers can visualize and better understand HRSs for future studies. The health care professionals and the end users should be encouraged to participate in the future design and development of HRSs to optimize their utility and successful implementation. Detailed evaluations of HRSs in a user-centered approach are needed in future studies. %M 36656630 %R 10.2196/38184 %U https://www.jmir.org/2023/1/e38184 %U https://doi.org/10.2196/38184 %U http://www.ncbi.nlm.nih.gov/pubmed/36656630 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e43053 %T Ontologies Applied in Clinical Decision Support System Rules: Systematic Review %A Jing,Xia %A Min,Hua %A Gong,Yang %A Biondich,Paul %A Robinson,David %A Law,Timothy %A Nohr,Christian %A Faxvaag,Arild %A Rennert,Lior %A Hubig,Nina %A Gimbel,Ronald %+ Department of Public Health Sciences, Clemson University, 519 Edwards Hall, Clemson, SC, 29634, United States, 1 8646563347, xjing@clemson.edu %K clinical decision support system rules %K clinical decision support systems %K interoperability %K ontology %K Semantic Web technology %D 2023 %7 19.1.2023 %9 Review %J JMIR Med Inform %G English %X Background: Clinical decision support systems (CDSSs) are important for the quality and safety of health care delivery. Although CDSS rules guide CDSS behavior, they are not routinely shared and reused. Objective: Ontologies have the potential to promote the reuse of CDSS rules. Therefore, we systematically screened the literature to elaborate on the current status of ontologies applied in CDSS rules, such as rule management, which uses captured CDSS rule usage data and user feedback data to tailor CDSS services to be more accurate, and maintenance, which updates CDSS rules. Through this systematic literature review, we aim to identify the frontiers of ontologies used in CDSS rules. Methods: The literature search was focused on the intersection of ontologies; clinical decision support; and rules in PubMed, the Association for Computing Machinery (ACM) Digital Library, and the Nursing & Allied Health Database. Grounded theory and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 guidelines were followed. One author initiated the screening and literature review, while 2 authors validated the processes and results independently. The inclusion and exclusion criteria were developed and refined iteratively. Results: CDSSs were primarily used to manage chronic conditions, alerts for medication prescriptions, reminders for immunizations and preventive services, diagnoses, and treatment recommendations among 81 included publications. The CDSS rules were presented in Semantic Web Rule Language, Jess, or Jena formats. Despite the fact that ontologies have been used to provide medical knowledge, CDSS rules, and terminologies, they have not been used in CDSS rule management or to facilitate the reuse of CDSS rules. Conclusions: Ontologies have been used to organize and represent medical knowledge, controlled vocabularies, and the content of CDSS rules. So far, there has been little reuse of CDSS rules. More work is needed to improve the reusability and interoperability of CDSS rules. This review identified and described the ontologies that, despite their limitations, enable Semantic Web technologies and their applications in CDSS rules. %M 36534739 %R 10.2196/43053 %U https://medinform.jmir.org/2023/1/e43053 %U https://doi.org/10.2196/43053 %U http://www.ncbi.nlm.nih.gov/pubmed/36534739 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e38150 %T An End-to-End Natural Language Processing Application for Prediction of Medical Case Coding Complexity: Algorithm Development and Validation %A Xu,He Ayu %A Maccari,Bernard %A Guillain,Hervé %A Herzen,Julien %A Agri,Fabio %A Raisaro,Jean Louis %+ Biomedical Data Science Center, Lausanne University Hospital, CHUV, Centre hospitalier universitaire vaudois Rue du Bugnon 21, Lausanne, 1011, Switzerland, 41 0795566886, he.xu@chuv.ch %K medical coding %K natural language processing %K NLP %K complexity prediction %K prediction %K decision support %K machine learning %K model %K clinical decision support application %K multimodal modeling %K coding %K algorithm %K documentation %K health record %K electronic health record %K EHR %K development %D 2023 %7 19.1.2023 %9 Original Paper %J JMIR Med Inform %G English %X Background: Medical coding is the process that converts clinical documentation into standard medical codes. Codes are used for several key purposes in a hospital (eg, insurance reimbursement and performance analysis); therefore, their optimization is crucial. With the rapid growth of natural language processing technologies, several solutions based on artificial intelligence have been proposed to aid in medical coding by automatically suggesting relevant codes for clinical documents. However, their effectiveness is still limited to simple cases, and it is not yet clear how much value they can bring in improving coding efficiency and accuracy. Objective: This study aimed to bring more efficiency to the coding process to improve the selection of codes by medical coders. To achieve this, we developed an innovative multimodal machine learning–based solution that, instead of predicting codes, detects the degree of coding complexity before coding is performed. The notion of coding complexity was used to better dispatch work among medical coders to eventually minimize errors and improve throughput. Methods: To train and evaluate our approach, we collected 2060 cases rated by coders in terms of coding complexity from 1 (simplest) to 4 (most complex). We asked 2 expert coders to rate 3.01% (62/2060) of the cases as the gold standard. The agreements between experts were used as benchmarks for model evaluation. A case contains both clinical text and patient metadata from the hospital electronic health record. We extracted both text features and metadata features, then concatenated and fed them into several machine learning models. Finally, we selected 2 models. The first used cross-validated training on 1751 cases and testing on 309 cases aiming to assess the predictive power of the proposed approach and its generalizability. The second model was trained on 1998 cases and tested on the gold standard to validate the best model performance against human benchmarks. Results: Our first model achieved a macro–F1-score of 0.51 and an accuracy of 0.59 on classifying the 4-scale complexity. The model distinguished well between the simple (combined complexity 1-2) and complex (combined complexity 3-4) cases with a macro–F1-score of 0.65 and an accuracy of 0.71. Our second model achieved 61% agreement with experts’ ratings and a macro–F1-score of 0.62 on the gold standard, whereas the 2 experts had a 66% (41/62) agreement ratio with a macro–F1-score of 0.67. Conclusions: We propose a multimodal machine learning approach that leverages information from both clinical text and patient metadata to predict the complexity of coding a case in the precoding phase. By integrating this model into the hospital coding system, distribution of cases among coders can be done automatically with performance comparable with that of human expert coders, thus improving coding efficiency and accuracy at scale. %M 36656627 %R 10.2196/38150 %U https://medinform.jmir.org/2023/1/e38150 %U https://doi.org/10.2196/38150 %U http://www.ncbi.nlm.nih.gov/pubmed/36656627 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 12 %N %P e42653 %T Nudging Health Care Providers’ Adoption of Clinical Decision Support: Protocol for the User-Centered Development of a Behavioral Economics–Inspired Electronic Health Record Tool %A Richardson,Safiya %A Dauber-Decker,Katherine %A Solomon,Jeffrey %A Khan,Sundas %A Barnaby,Douglas %A Chelico,John %A Qiu,Michael %A Liu,Yan %A Mann,Devin %A Pekmezaris,Renee %A McGinn,Thomas %A Diefenbach,Michael %+ New York University Grossman School of Medicine, 550 1st Ave, New York, NY, 10016, United States, 1 2122638313, srichard12@northwell.edu %K health informatics %K clinical decision support %K electronic health record %K implementation science %K behavioral economics %K user-centered design %K pulmonary embolism %D 2023 %7 18.1.2023 %9 Protocol %J JMIR Res Protoc %G English %X Background: The improvements in care resulting from clinical decision support (CDS) have been significantly limited by consistently low health care provider adoption. Health care provider attitudes toward CDS, specifically psychological and behavioral barriers, are not typically addressed during any stage of CDS development, although they represent an important barrier to adoption. Emerging evidence has shown the surprising power of using insights from the field of behavioral economics to address psychological and behavioral barriers. Nudges are formal applications of behavioral economics, defined as positive reinforcement and indirect suggestions that have a nonforced effect on decision-making. Objective: Our goal is to employ a user-centered design process to develop a CDS tool—the pulmonary embolism (PE) risk calculator—for PE risk stratification in the emergency department that incorporates a behavior theory–informed nudge to address identified behavioral barriers to use. Methods: All study activities took place at a large academic health system in the New York City metropolitan area. Our study used a user-centered and behavior theory–based approach to achieve the following two aims: (1) use mixed methods to identify health care provider barriers to the use of an active CDS tool for PE risk stratification and (2) develop a new CDS tool—the PE risk calculator—that addresses behavioral barriers to health care providers’ adoption of CDS by incorporating nudges into the user interface. These aims were guided by the revised Observational Research Behavioral Information Technology model. A total of 50 clinicians who used the original version of the tool were surveyed with a quantitative instrument that we developed based on a behavior theory framework—the Capability-Opportunity-Motivation-Behavior framework. A semistructured interview guide was developed based on the survey responses. Inductive methods were used to analyze interview session notes and audio recordings from 12 interviews. Revised versions of the tool were developed that incorporated nudges. Results: Functional prototypes were developed by using Axure PRO (Axure Software Solutions) software and usability tested with end users in an iterative agile process (n=10). The tool was redesigned to address 4 identified major barriers to tool use; we included 2 nudges and a default. The 6-month pilot trial for the tool was launched on October 1, 2021. Conclusions: Clinicians highlighted several important psychological and behavioral barriers to CDS use. Addressing these barriers, along with conducting traditional usability testing, facilitated the development of a tool with greater potential to transform clinical care. The tool will be tested in a prospective pilot trial. International Registered Report Identifier (IRRID): DERR1-10.2196/42653 %M 36652293 %R 10.2196/42653 %U https://www.researchprotocols.org/2023/1/e42653 %U https://doi.org/10.2196/42653 %U http://www.ncbi.nlm.nih.gov/pubmed/36652293 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e37545 %T Digital Maturity Consulting and Strategizing to Optimize Services: Overview %A Phiri,Peter %A Cavalini,Heitor %A Shetty,Suchith %A Delanerolle,Gayathri %+ Research & Innovation Department, Southern Health NHS Foundation Trust, Clinical Trials Facility, Tom Rudd Unit, Moorgreen Hospital, Botley Road, West End, Southampton, SO30 3JB, United Kingdom, 44 2380475112, P.Phiri@soton.ac.uk %K digital maturity model %K health care system %K electronic medical records %K health record %K information %K UK %K medical service %K care provider %K integration %K interoperability %K digital health %K digital record %K workflow %D 2023 %7 17.1.2023 %9 Viewpoint %J J Med Internet Res %G English %X The National Health Service (NHS), the health care system of the United Kingdom, is one of the largest health care entities in the world and has been successfully serving the UK population for decades. The NHS is also the fourth-largest employer globally. True to its reputation, some of the most modern and technically advanced medical services are available in the United Kingdom. However, between the acute, primary, secondary, and tertiary care providers of the NHS, there needs to be seamless integration and interoperability to provide timely holistic care to patients at a national level. Various efforts have been taken and programs launched since 2002 to achieve digital transformation in the NHS but with partial success rates. As it is important to understand a problem before trying to solve it, in this paper, we focus on tools used to assess the digital maturity of NHS trusts and organizations. Additionally, we aim to present the impact of ongoing transformation attempts on secondary services, particularly mental health. This paper considered the literature on digital maturity and performed a rapid review of currently available tools to measure digital maturity. We have performed a multivocal literature review that included white papers and web-based documents in addition to peer-reviewed literature. Further, the paper also provides a perspective of the ground reality from a mental health service provider’s point of view. Assessment tools adopted from the global market, later modified and tailor-made to suit local preferences, are currently being used. However, there is a need for a robust framework that assesses status, allows target setting, and tracks progress across diverse providers. %M 36649060 %R 10.2196/37545 %U https://www.jmir.org/2023/1/e37545 %U https://doi.org/10.2196/37545 %U http://www.ncbi.nlm.nih.gov/pubmed/36649060 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e39231 %T Accuracy of COVID-19–Like Illness Diagnoses in Electronic Health Record Data: Retrospective Cohort Study %A Rao,Suchitra %A Bozio,Catherine %A Butterfield,Kristen %A Reynolds,Sue %A Reese,Sarah E %A Ball,Sarah %A Steffens,Andrea %A Demarco,Maria %A McEvoy,Charlene %A Thompson,Mark %A Rowley,Elizabeth %A Porter,Rachael M %A Fink,Rebecca V %A Irving,Stephanie A %A Naleway,Allison %+ Department of Pediatrics, Hospital Medicine and Infectious Diseases, University of Colorado School of Medicine, 13123 E 16th Ave, Aurora, CO, 80045, United States, 1 7207772823, suchitra.rao@childrenscolorado.org %K COVID-19 %K COVID-like illness %K COVID-19 case definition %K sensitivity %K specificity %K positive predictive value %K negative predictive value %D 2023 %7 17.1.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Electronic health record (EHR) data provide a unique opportunity to study the epidemiology of COVID-19, clinical outcomes of the infection, comparative effectiveness of therapies, and vaccine effectiveness but require a well-defined computable phenotype of COVID-19–like illness (CLI). Objective: The objective of this study was to evaluate the performance of pathogen-specific and other acute respiratory illness (ARI) International Statistical Classification of Diseases-9 and -10 codes in identifying COVID-19 cases in emergency department (ED) or urgent care (UC) and inpatient settings. Methods: We conducted a retrospective observational cohort study using EHR, claims, and laboratory information system data of ED or UC and inpatient encounters from 4 health systems in the United States. Patients who were aged ≥18 years, had an ED or UC or inpatient encounter for an ARI, and underwent a SARS-CoV-2 polymerase chain reaction test between March 1, 2020, and March 31, 2021, were included. We evaluated various CLI definitions using combinations of International Statistical Classification of Diseases-10 codes as follows: COVID-19–specific codes; CLI definition used in VISION network studies; ARI signs, symptoms, and diagnosis codes only; signs and symptoms of ARI only; and random forest model definitions. We evaluated the sensitivity, specificity, positive predictive value, and negative predictive value of each CLI definition using a positive SARS-CoV-2 polymerase chain reaction test as the reference standard. We evaluated the performance of each CLI definition for distinct hospitalization and ED or UC cohorts. Results: Among 90,952 hospitalizations and 137,067 ED or UC visits, 5627 (6.19%) and 9866 (7.20%) were positive for SARS-CoV-2, respectively. COVID-19–specific codes had high sensitivity (91.6%) and specificity (99.6%) in identifying patients with SARS-CoV-2 positivity among hospitalized patients. The VISION CLI definition maintained high sensitivity (95.8%) but lowered specificity (45.5%). By contrast, signs and symptoms of ARI had low sensitivity and positive predictive value (28.9% and 11.8%, respectively) but higher specificity and negative predictive value (85.3% and 94.7%, respectively). ARI diagnoses, signs, and symptoms alone had low predictive performance. All CLI definitions had lower sensitivity for ED or UC encounters. Random forest approaches identified distinct CLI definitions with high performance for hospital encounters and moderate performance for ED or UC encounters. Conclusions: COVID-19–specific codes have high sensitivity and specificity in identifying adults with positive SARS-CoV-2 test results. Separate combinations of COVID-19-specific codes and ARI codes enhance the utility of CLI definitions in studies using EHR data in hospital and ED or UC settings. %M 36383633 %R 10.2196/39231 %U https://formative.jmir.org/2023/1/e39231 %U https://doi.org/10.2196/39231 %U http://www.ncbi.nlm.nih.gov/pubmed/36383633 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e41043 %T An Accurate Deep Learning–Based System for Automatic Pill Identification: Model Development and Validation %A Heo,Junyeong %A Kang,Youjin %A Lee,SangKeun %A Jeong,Dong-Hwa %A Kim,Kang-Min %+ Department of Artificial Intelligence, The Catholic University of Korea, T908 Michael Building, The Catholic University of Korea, 43 Jibong-ro, Bucheon, 14662, Republic of Korea, 82 10 6707 6977, donghwa@catholic.ac.kr %K pill identification %K pill retrieval %K pill recognition %K automatic pill search %K deep learning %K machine learning %K character-level language model %D 2023 %7 13.1.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Medication errors account for a large proportion of all medical errors. In most homes, patients take a variety of medications for a long period. However, medication errors frequently occur because patients often throw away the containers of their medications. Objective: We proposed a deep learning–based system for reducing medication errors by accurately identifying prescription pills. Given the pill images, our system located the pills in the respective pill databases in South Korea and the United States. Methods: We organized the system into a pill recognition step and pill retrieval step, and we applied deep learning models to train not only images of the pill but also imprinted characters. In the pill recognition step, there are 3 modules that recognize the 3 features of pills and their imprints separately and correct the recognized imprint to fit the actual data. We adopted image classification and text detection models for the feature and imprint recognition modules, respectively. In the imprint correction module, we introduced a language model for the first time in the pill identification system and proposed a novel coordinate encoding technique for effective correction in the language model. We identified pills using similarity scores of pill characteristics with those in the database. Results: We collected the open pill database from South Korea and the United States in May 2022. We used a total of 24,404 pill images in our experiments. The experimental results show that the predicted top-1 candidates achieve accuracy levels of 85.6% (South Korea) and 74.5% (United States) for the types of pills not trained on 2 different databases (South Korea and the United States). Furthermore, the predicted top-1 candidate accuracy of our system was 78% with consumer-granted images, which was achieved by training only 1 image per pill. The results demonstrate that our system could identify and retrieve new pills without additional model updates. Finally, we confirmed through an ablation study that the language model that we emphasized significantly improves the pill identification ability of the system. Conclusions: Our study proposes the possibility of reducing medical errors by showing that the introduction of artificial intelligence can identify numerous pills with high precision in real time. Our study suggests that the proposed system can reduce patients’ misuse of medications and help medical staff focus on higher-level tasks by simplifying time-consuming lower-level tasks such as pill identification. %M 36637893 %R 10.2196/41043 %U https://www.jmir.org/2023/1/e41043 %U https://doi.org/10.2196/41043 %U http://www.ncbi.nlm.nih.gov/pubmed/36637893 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e40179 %T Interpretable Deep-Learning Approaches for Osteoporosis Risk Screening and Individualized Feature Analysis Using Large Population-Based Data: Model Development and Performance Evaluation %A Suh,Bogyeong %A Yu,Heejin %A Kim,Hyeyeon %A Lee,Sanghwa %A Kong,Sunghye %A Kim,Jin-Woo %A Choi,Jongeun %+ School of Mechanical Engineering, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea, 82 2 2123 2813, jongeunchoi@yonsei.ac.kr %K osteoporosis %K artificial intelligence %K deep learning %K machine learning %K risk factors %K screening %D 2023 %7 13.1.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Osteoporosis is one of the diseases that requires early screening and detection for its management. Common clinical tools and machine-learning (ML) models for screening osteoporosis have been developed, but they show limitations such as low accuracy. Moreover, these methods are confined to limited risk factors and lack individualized explanation. Objective: The aim of this study was to develop an interpretable deep-learning (DL) model for osteoporosis risk screening with clinical features. Clinical interpretation with individual explanations of feature contributions is provided using an explainable artificial intelligence (XAI) technique. Methods: We used two separate data sets: the National Health and Nutrition Examination Survey data sets from the United States (NHANES) and South Korea (KNHANES) with 8274 and 8680 respondents, respectively. The study population was classified according to the T-score of bone mineral density at the femoral neck or total femur. A DL model for osteoporosis diagnosis was trained on the data sets and significant risk factors were investigated with local interpretable model-agnostic explanations (LIME). The performance of the DL model was compared with that of ML models and conventional clinical tools. Additionally, contribution ranking of risk factors and individualized explanation of feature contribution were examined. Results: Our DL model showed area under the curve (AUC) values of 0.851 (95% CI 0.844-0.858) and 0.922 (95% CI 0.916-0.928) for the femoral neck and total femur bone mineral density, respectively, using the NHANES data set. The corresponding AUC values for the KNHANES data set were 0.827 (95% CI 0.821-0.833) and 0.912 (95% CI 0.898-0.927), respectively. Through the LIME method, significant features were induced, and each feature’s integrated contribution and interpretation for individual risk were determined. Conclusions: The developed DL model significantly outperforms conventional ML models and clinical tools. Our XAI model produces high-ranked features along with the integrated contributions of each feature, which facilitates the interpretation of individual risk. In summary, our interpretable model for osteoporosis risk screening outperformed state-of-the-art methods. %M 36482780 %R 10.2196/40179 %U https://www.jmir.org/2023/1/e40179 %U https://doi.org/10.2196/40179 %U http://www.ncbi.nlm.nih.gov/pubmed/36482780 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 9 %N %P e34123 %T Trusted Data Spaces as a Viable and Sustainable Solution for Networks of Population-Based Patient Registries %A Nicholson,Nicholas %A Caldeira,Sandra %A Furtado,Artur %A Nicholl,Ciaran %+ European Commission, Joint Research Centre, via E. Fermi 2749, Ispra, 21027, Italy, 39 0332 78 ext 9365, Nicholas.Nicholson@ec.europa.eu %K population-based patient registries %K trusted research environments %K registry network model %K data federation %K cancer registries %K noncommunicable diseases %D 2023 %7 13.1.2023 %9 Viewpoint %J JMIR Public Health Surveill %G English %X Harmonization and integration of health data remain as the focus of many ongoing efforts toward the goal of optimizing health and health care policies. Population-based patient registries constitute a critical element of these endeavors. Although their main function is monitoring and surveillance of a particular disease within a given population, they are also an important data source for epidemiology. Comparing indicators across national boundaries brings an extra dimension to the use of registry data, especially in regions where supranational initiatives are or could be coordinated to leverage good practices; this is particularly relevant for the European Union. However, strict data protection laws can unintentionally hamper the efforts of data harmonization to ensure the removal of statistical bias in the individual data sets, thereby compromising the integrated value of registries’ data. Consequently, there is the motivation for creating a new paradigm to ensure that registries can operate in an environment that is not unnecessarily restrictive and to allow accurate comparison of data to better ascertain the measures and practices that are most conducive to the public health of societies. The pan-European organizational model of cancer registries, owing to its long and successful establishment, was considered as a sound basis from which to proceed toward such a paradigm. However, it has certain drawbacks, particularly regarding governance, scalability, and resourcing, which are essential elements to consider for a generic patient registry model. These issues are addressed in a proposal of an adapted model that promises a valuable pan-European data resource for epidemiological research, while providing a closely regulated environment for the processing of pseudonymized patient summary data on a broader scale than has hitherto been possible. %M 36637894 %R 10.2196/34123 %U https://publichealth.jmir.org/2023/1/e34123 %U https://doi.org/10.2196/34123 %U http://www.ncbi.nlm.nih.gov/pubmed/36637894 %0 Journal Article %@ 2562-7600 %I JMIR Publications %V 6 %N %P e41331 %T The Use and Structure of Emergency Nurses’ Triage Narrative Data: Scoping Review %A Picard,Christopher %A Kleib,Manal %A Norris,Colleen %A O'Rourke,Hannah M %A Montgomery,Carmel %A Douma,Matthew %+ Faculty of Nursing, University of Alberta, Graduate Office, 4-171 Edmonton Clinic Health Academy, Edmonton, AB, T6G 1C9, Canada, 1 (780) 492 4567, picard.ct@gmail.com %K nursing %K artificial intelligence %K machine learning %K triage %K review %K narrative %D 2023 %7 13.1.2023 %9 Review %J JMIR Nursing %G English %X Background: Emergency departments use triage to ensure that patients with the highest level of acuity receive care quickly and safely. Triage is typically a nursing process that is documented as structured and unstructured (free text) data. Free-text triage narratives have been studied for specific conditions but never reviewed in a comprehensive manner. Objective: The objective of this paper was to identify and map the academic literature that examines triage narratives. The paper described the types of research conducted, identified gaps in the research, and determined where additional review may be warranted. Methods: We conducted a scoping review of unstructured triage narratives. We mapped the literature, described the use of triage narrative data, examined the information available on the form and structure of narratives, highlighted similarities among publications, and identified opportunities for future research. Results: We screened 18,074 studies published between 1990 and 2022 in CINAHL, MEDLINE, Embase, Cochrane, and ProQuest Central. We identified 0.53% (96/18,074) of studies that directly examined the use of triage nurses’ narratives. More than 12 million visits were made to 2438 emergency departments included in the review. In total, 82% (79/96) of these studies were conducted in the United States (43/96, 45%), Australia (31/96, 32%), or Canada (5/96, 5%). Triage narratives were used for research and case identification, as input variables for predictive modeling, and for quality improvement. Overall, 31% (30/96) of the studies offered a description of the triage narrative, including a list of the keywords used (27/96, 28%) or more fulsome descriptions (such as word counts, character counts, abbreviation, etc; 7/96, 7%). We found limited use of reporting guidelines (8/96, 8%). Conclusions: The breadth of the identified studies suggests that there is widespread routine collection and research use of triage narrative data. Despite the use of triage narratives as a source of data in studies, the narratives and nurses who generate them are poorly described in the literature, and data reporting is inconsistent. Additional research is needed to describe the structure of triage narratives, determine the best use of triage narratives, and improve the consistent use of triage-specific data reporting guidelines. International Registered Report Identifier (IRRID): RR2-10.1136/bmjopen-2021-055132 %M 36637881 %R 10.2196/41331 %U https://nursing.jmir.org/2023/1/e41331 %U https://doi.org/10.2196/41331 %U http://www.ncbi.nlm.nih.gov/pubmed/36637881 %0 Journal Article %@ 1929-073X %I JMIR Publications %V 12 %N %P e40721 %T Electronic Medical Record System Use and Determinants in Ethiopia: Systematic Review and Meta-Analysis %A Tegegne,Masresha Derese %A Wubante,Sisay Maru %A Kalayou,Mulugeta Hayelom %A Melaku,Mequannent Sharew %A Tilahun,Binyam %A Yilma,Tesfahun Melese %A Dessie,Hiwote Simane %+ Department of Health Informatics, Institute of Public Health, College of Medicine and Health Sciences, University of Gondar, PO Box 196, Gondar, Ethiopia, 251 935474755, masresha1derese@gmail.com %K electronic medical record system %K health professional %K utilization %K determinants %K Ethiopia %K medical record %K EMR %K EHR %K electronic health record %K health information technology %K systematic review %D 2023 %7 11.1.2023 %9 Review %J Interact J Med Res %G English %X Background: The strategic plan of the Ethiopian Ministry of Health recommends an electronic medical record (EMR) system to enhance health care delivery and streamline data systems. However, only a few exhaustive systematic reviews and meta-analyses have been conducted on the degree of EMR use in Ethiopia and the factors influencing success. This will emphasize the factors that make EMR effective and increase awareness of its widespread use among future implementers in Ethiopia. Objective: This study aims to determine the pooled estimate of EMR use and success determinants among health professionals in Ethiopia. Methods: We developed a protocol and searched PubMed, Web of Sciences, African Journals OnLine, Embase, MEDLINE, and Scopus to identify relevant studies. To assess the quality of each included study, we used the Joanna Briggs Institute quality assessment tool using 9 criteria. The applicable data were extracted using Microsoft Excel 2019, and the data were then analyzed using Stata software (version 11; StataCorp). The presence of total heterogeneity across included studies was calculated using the index of heterogeneity I2 statistics. The pooled size of EMR use was estimated using a random effect model with a 95% CI. Results: After reviewing 11,026 research papers, 5 papers with a combined total of 2439 health workers were included in the evaluation and meta-analysis. The pooled estimate of EMR usage in Ethiopia was 51.85% (95% CI 37.14%-66.55%). The subgroup study found that the northern Ethiopian region had the greatest EMR utilization rate (58.75%) and that higher (54.99%) utilization was also seen in publications published after 2016. Age groups <30 years, access to an EMR manual, EMR-related training, and managerial support were identified factors associated with EMR use among health workers. Conclusions: The use of EMR systems in Ethiopia is relatively low. Belonging to a young age group, accessing an EMR manual, receiving EMR-related training, and managerial support were identified as factors associated with EMR use among health workers. As a result, to increase the use of EMRs by health care providers, it is essential to provide management support and an EMR training program and make the EMR manual accessible to health professionals. %M 36630161 %R 10.2196/40721 %U https://www.i-jmr.org/2023/1/e40721 %U https://doi.org/10.2196/40721 %U http://www.ncbi.nlm.nih.gov/pubmed/36630161 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e41142 %T Machine Learning–Based Prediction of Acute Kidney Injury Following Pediatric Cardiac Surgery: Model Development and Validation Study %A Luo,Xiao-Qin %A Kang,Yi-Xin %A Duan,Shao-Bin %A Yan,Ping %A Song,Guo-Bao %A Zhang,Ning-Ya %A Yang,Shi-Kun %A Li,Jing-Xin %A Zhang,Hui %+ Department of Nephrology, The Second Xiangya Hospital of Central South University, 139 Renmin Road, Changsha, 410011, China, 86 731 85295100, duansb528@csu.edu.cn %K cardiac surgery %K acute kidney injury %K pediatric %K machine learning %D 2023 %7 5.1.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Cardiac surgery–associated acute kidney injury (CSA-AKI) is a major complication following pediatric cardiac surgery, which is associated with increased morbidity and mortality. The early prediction of CSA-AKI before and immediately after surgery could significantly improve the implementation of preventive and therapeutic strategies during the perioperative periods. However, there is limited clinical information on how to identify pediatric patients at high risk of CSA-AKI. Objective: The study aims to develop and validate machine learning models to predict the development of CSA-AKI in the pediatric population. Methods: This retrospective cohort study enrolled patients aged 1 month to 18 years who underwent cardiac surgery with cardiopulmonary bypass at 3 medical centers of Central South University in China. CSA-AKI was defined according to the 2012 Kidney Disease: Improving Global Outcomes criteria. Feature selection was applied separately to 2 data sets: the preoperative data set and the combined preoperative and intraoperative data set. Multiple machine learning algorithms were tested, including K-nearest neighbor, naive Bayes, support vector machines, random forest, extreme gradient boosting (XGBoost), and neural networks. The best performing model was identified in cross-validation by using the area under the receiver operating characteristic curve (AUROC). Model interpretations were generated using the Shapley additive explanations (SHAP) method. Results: A total of 3278 patients from one of the centers were used for model derivation, while 585 patients from another 2 centers served as the external validation cohort. CSA-AKI occurred in 564 (17.2%) patients in the derivation cohort and 51 (8.7%) patients in the external validation cohort. Among the considered machine learning models, the XGBoost models achieved the best predictive performance in cross-validation. The AUROC of the XGBoost model using only the preoperative variables was 0.890 (95% CI 0.876-0.906) in the derivation cohort and 0.857 (95% CI 0.800-0.903) in the external validation cohort. When the intraoperative variables were included, the AUROC increased to 0.912 (95% CI 0.899-0.924) and 0.889 (95% CI 0.844-0.920) in the 2 cohorts, respectively. The SHAP method revealed that baseline serum creatinine level, perfusion time, body length, operation time, and intraoperative blood loss were the top 5 predictors of CSA-AKI. Conclusions: The interpretable XGBoost models provide practical tools for the early prediction of CSA-AKI, which are valuable for risk stratification and perioperative management of pediatric patients undergoing cardiac surgery. %M 36603200 %R 10.2196/41142 %U https://www.jmir.org/2023/1/e41142 %U https://doi.org/10.2196/41142 %U http://www.ncbi.nlm.nih.gov/pubmed/36603200 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e40976 %T The Assessment of Medical Device Software Supporting Health Care Services for Chronic Patients in a Tertiary Hospital: Overarching Study %A Baltaxe,Erik %A Hsieh,Hsin Wen %A Roca,Josep %A Cano,Isaac %+ Hospital Clinic de Barcelona, Institut d’Investigacions Biomèdiques August Pi i Sunyer, Universitat de Barcelona, Villarroel 170, Barcelona, 08036, Spain, 34 932275540, iscano@recerca.clinic.cat %K chronic patients %K digital health %K health technology assessment %K implementation research %K integrated care %D 2023 %7 4.1.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Innovative digital health tools are increasingly being evaluated and, in some instances, integrated at scale into health systems. However, the applicability of assessment methodologies in real-life scenarios to demonstrate value generation and consequently foster sustainable adoption of digitally enabled health interventions has some bottlenecks. Objective: We aimed to build on the process of premarket assessment of 4 digital health interventions piloted at the Hospital Clinic de Barcelona (HCB), as well as on the analysis of current medical device software regulations and postmarket surveillance in the European Union and United States in order to generate recommendations and lessons learnt for the sustainable adoption of digitally enabled health interventions. Methods: Four digital health interventions involving prototypes were piloted at the HCB (studies 1-4). Cocreation and quality improvement methodologies were used to consolidate a pragmatic evaluation method to assess the perceived usability and satisfaction of end users (both patients and health care professionals) by means of the System Usability Scale and the Net Promoter Score, including general questions about satisfaction. Analyses of both medical software device regulations and postmarket surveillance in the European Union and United States (2017-2021) were performed. Finally, an overarching analysis on lessons learnt was conducted considering 4 domains (technical, clinical, usability, and cost), as well as differentiating among 3 different eHealth strategies (telehealth, integrated care, and digital therapeutics). Results: Among the participant stakeholders, the System Usability Scale score was consistently higher in patients (studies 1, 2, 3, and 4: 78, 67, 56, and 76, respectively) than in health professionals (studies 2, 3, and 4: 52, 43, and 54, respectively). In general, use of the supporting digital health tools was recommended more by patients (studies 1, 2, 3, and 4: Net Promoter Scores of −3%, 31%, −21%, and 31%, respectively) than by professionals (studies 2, 3, and 4: Net Promoter Scores of −67%, 1%, and −80%, respectively). The overarching analysis resulted in pragmatic recommendations for the digital health evaluation domains and the eHealth strategies considered. Conclusions: Lessons learnt on the digitalization of health resulted in practical recommendations that could contribute to future deployment experiences. %M 36598817 %R 10.2196/40976 %U https://www.jmir.org/2023/1/e40976 %U https://doi.org/10.2196/40976 %U http://www.ncbi.nlm.nih.gov/pubmed/36598817 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 11 %N %P e37805 %T Practical Considerations for Developing Clinical Natural Language Processing Systems for Population Health Management and Measurement %A Tamang,Suzanne %A Humbert-Droz,Marie %A Gianfrancesco,Milena %A Izadi,Zara %A Schmajuk,Gabriela %A Yazdany,Jinoos %+ Division of Rheumatology, University of California, San Francisco, 10 Koret Way, Room K-219, San Francisco, CA, 94143, United States, 1 415 576 1000, jinoos.yazdany@ucsf.edu %K clinical natural language processing %K electronic health records %K population health science %K clinical decision support %K information extraction %D 2023 %7 3.1.2023 %9 Viewpoint %J JMIR Med Inform %G English %X Experts have noted a concerning gap between clinical natural language processing (NLP) research and real-world applications, such as clinical decision support. To help address this gap, in this viewpoint, we enumerate a set of practical considerations for developing an NLP system to support real-world clinical needs and improve health outcomes. They include determining (1) the readiness of the data and compute resources for NLP, (2) the organizational incentives to use and maintain the NLP systems, and (3) the feasibility of implementation and continued monitoring. These considerations are intended to benefit the design of future clinical NLP projects and can be applied across a variety of settings, including large health systems or smaller clinical practices that have adopted electronic medical records in the United States and globally. %M 36595345 %R 10.2196/37805 %U https://medinform.jmir.org/2023/1/e37805 %U https://doi.org/10.2196/37805 %U http://www.ncbi.nlm.nih.gov/pubmed/36595345 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 9 %N 12 %P e39747 %T Cross-Platform Detection of Psychiatric Hospitalization via Social Media Data: Comparison Study %A Nguyen,Viet Cuong %A Lu,Nathaniel %A Kane,John M %A Birnbaum,Michael L %A De Choudhury,Munmun %+ School of Interactive Computing, Georgia Institute of Technology, 756 W Peachtree St NW, Atlanta, GA, 30318, United States, 1 404 279 2941, johnny.nguyen@gatech.edu %K schizophrenia %K mental health %K machine learning %K clinical informatics %K social media %K mobile phone %D 2022 %7 30.12.2022 %9 Original Paper %J JMIR Ment Health %G English %X Background: Previous research has shown the feasibility of using machine learning models trained on social media data from a single platform (eg, Facebook or Twitter) to distinguish individuals either with a diagnosis of mental illness or experiencing an adverse outcome from healthy controls. However, the performance of such models on data from novel social media platforms unseen in the training data (eg, Instagram and TikTok) has not been investigated in previous literature. Objective: Our study examined the feasibility of building machine learning classifiers that can effectively predict an upcoming psychiatric hospitalization given social media data from platforms unseen in the classifiers’ training data despite the preliminary evidence on identity fragmentation on the investigated social media platforms. Methods: Windowed timeline data of patients with a diagnosis of schizophrenia spectrum disorder before a known hospitalization event and healthy controls were gathered from 3 platforms: Facebook (254/268, 94.8% of participants), Twitter (51/268, 19% of participants), and Instagram (134/268, 50% of participants). We then used a 3 × 3 combinatorial binary classification design to train machine learning classifiers and evaluate their performance on testing data from all available platforms. We further compared results from models in intraplatform experiments (ie, training and testing data belonging to the same platform) to those from models in interplatform experiments (ie, training and testing data belonging to different platforms). Finally, we used Shapley Additive Explanation values to extract the top predictive features to explain and compare the underlying constructs that predict hospitalization on each platform. Results: We found that models in intraplatform experiments on average achieved an F1-score of 0.72 (SD 0.07) in predicting a psychiatric hospitalization because of schizophrenia spectrum disorder, which is 68% higher than the average of models in interplatform experiments at an F1-score of 0.428 (SD 0.11). When investigating the key drivers for divergence in construct validities between models, an analysis of top features for the intraplatform models showed both low predictive feature overlap between the platforms and low pairwise rank correlation (<0.1) between the platforms’ top feature rankings. Furthermore, low average cosine similarity of data between platforms within participants in comparison with the same measurement on data within platforms between participants points to evidence of identity fragmentation of participants between platforms. Conclusions: We demonstrated that models built on one platform’s data to predict critical mental health treatment outcomes such as hospitalization do not generalize to another platform. In our case, this is because different social media platforms consistently reflect different segments of participants’ identities. With the changing ecosystem of social media use among different demographic groups and as web-based identities continue to become fragmented across platforms, further research on holistic approaches to harnessing these diverse data sources is required. %M 36583932 %R 10.2196/39747 %U https://mental.jmir.org/2022/12/e39747 %U https://doi.org/10.2196/39747 %U http://www.ncbi.nlm.nih.gov/pubmed/36583932 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 12 %P e37783 %T Patients Managing Their Medical Data in Personal Electronic Health Records: Scoping Review %A Damen,Debby J %A Schoonman,Guus G %A Maat,Barbara %A Habibović,Mirela %A Krahmer,Emiel %A Pauws,Steffen %+ Department of Communication and Cognition, Tilburg School of Humanities and Digital Sciences, Tilburg University, Warandelaan 2, Tilburg, 5037 AB, Netherlands, 31 0630215926, d.j.damen@tilburguniversity.edu %K patient-generated data %K patient portal %K personal electronic health record %K patient activation %K patient engagement %D 2022 %7 27.12.2022 %9 Review %J J Med Internet Res %G English %X Background: Personal electronic health records (PEHRs) allow patients to view, generate, and manage their personal and medical data that are relevant across illness episodes, such as their medications, allergies, immunizations, and their medical, social, and family health history. Thus, patients can actively participate in the management of their health care by ensuring that their health care providers have an updated and accurate overview of the patients’ medical records. However, the uptake of PEHRs remains low, especially in terms of patients entering and managing their personal and medical data in their PEHR. Objective: This scoping review aimed to explore the barriers and facilitators that patients face when deciding to review, enter, update, or modify their personal and medical data in their PEHR. This review also explores the extent to which patient-generated and -managed data affect the quality and safety of care, patient engagement, patient satisfaction, and patients’ health and health care services. Methods: We searched the MEDLINE, Embase, CINAHL, PsycINFO, Cochrane Library, Web of Science, and Google Scholar web-based databases, as well as reference lists of all primary and review articles using a predefined search query. Results: Of the 182 eligible papers, 37 (20%) provided sufficient information about patients’ data management activities. The results showed that patients tend to use their PEHRs passively rather than actively. Patients refrain from generating and managing their medical data in a PEHR, especially when these data are complex and sensitive. The reasons for patients’ passive data management behavior were related to their concerns about the validity, applicability, and confidentiality of patient-generated data. Our synthesis also showed that patient-generated and -managed health data ensures that the medical record is complete and up to date and is positively associated with patient engagement and patient satisfaction. Conclusions: The findings of this study suggest recommendations for implementing design features within the PEHR and the construal of a dedicated policy to inform both clinical staff and patients about the added value of patient-generated data. Moreover, clinicians should be involved as important ambassadors in informing, reminding, and encouraging patients to manage the data in their PEHR. %M 36574275 %R 10.2196/37783 %U https://www.jmir.org/2022/12/e37783 %U https://doi.org/10.2196/37783 %U http://www.ncbi.nlm.nih.gov/pubmed/36574275 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 12 %P e31433 %T Understanding Emergency Room Visits for Nontraumatic Oral Health Conditions in a Hospital Serving Rural Appalachia: Dental Informatics Study %A Khanna,Raj K %A Cecchetti,Alfred A %A Bhardwaj,Niharika %A Muto,Bobbi Steele %A Murughiyan,Usha %+ Department of Clinical and Translational Science, Joan C Edwards School of Medicine, Marshall University, 1600 Medical Center Drive, Huntington, WV, 25701, United States, 1 3046911585, cecchetti@marshall.edu %K dental informatics %K visualization %K nontraumatic dental care %K emergency room %K cost %K utilization %K economic impact %D 2022 %7 23.12.2022 %9 Original Paper %J JMIR Form Res %G English %X Background: In the Appalachian region, a variety of factors will impact the ability of patients to maintain good oral health, which is essential for overall health and well-being. Oral health issues have led to high costs within the Appalachian hospital system. Dental informatics examines preventable dental conditions to understand the problem and suggest cost containment. Objective: We aimed to demonstrate the value of dental informatics in dental health care in rural Appalachia by presenting a research study that measured emergency room (ER) use for nontraumatic dental conditions (NTDCs) and the associated economic impact in a hospital system that primarily serves rural Appalachia. Methods: The Appalachian Clinical and Translational Science Institute’s oral health data mart with relevant data on patients (n=8372) with ER encounters for NTDC between 2010 and 2018 was created using Appalachian Clinical and Translational Science Institute’s research data warehouse. Exploratory analysis was then performed by developing an interactive Tableau dashboard. Dental Informatics provided the platform whereby the overall burden of these encounters, along with disparities in burden by age groups, gender, and primary payer, was assessed. Results: Dental informatics was essential in understanding the overall problem and provided an interactive and easily comprehensible visualization of the situation. We found that ER visits for NTDCs declined by 40% from 2010 to 2018, but a higher percentage of visits required inpatient care and surgical intervention. Conclusions: Dental informatics can provide the necessary tools and support to health care systems and state health departments across Appalachia to address serious dental problems. In this case, informatics helped identify that although inappropriate ER use for NTDCs diminished due to ER diversion efforts, they remain a significant burden. Through its visualization and data extraction techniques, dental informatics can help produce policy changes by promoting models that improve access to preventive care. %M 36563041 %R 10.2196/31433 %U https://formative.jmir.org/2022/12/e31433 %U https://doi.org/10.2196/31433 %U http://www.ncbi.nlm.nih.gov/pubmed/36563041 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 11 %N 12 %P e42754 %T Clinical Source Data Production and Quality Control in Real-world Studies: Proposal for Development of the eSource Record System %A Wang,Bin %A Lai,Junkai %A Jin,Feifei %A Liao,Xiwen %A Zhu,Huan %A Yao,Chen %+ Peking University Clinical Research Institute, Peking University First Hospital, No. 8, Xishiku Street, Xicheng District, Beijing, 100034, China, 86 01083325822, yaochen@hsc.pku.edu.cn %K electronic medical record %K electronic health record %K eSource %K real-world data %K eSource record %K clinical research %K data collection %K data transcription %K data quality %K interoperability %D 2022 %7 23.12.2022 %9 Proposal %J JMIR Res Protoc %G English %X Background: An eSource generally includes the direct capture, collection, and storage of electronic data to simplify clinical research. It can improve data quality and patient safety and reduce clinical trial costs. There has been some eSource-related research progress in relatively large projects. However, most of these studies focused on technical explorations to improve interoperability among systems to reuse retrospective data for research. Few studies have explored source data collection and quality control during prospective data collection from a methodological perspective. Objective: This study aimed to design a clinical source data collection method that is suitable for real-world studies and meets the data quality standards for clinical research and to improve efficiency when writing electronic medical records (EMRs). Methods: On the basis of our group’s previous research experience, TransCelerate BioPharm Inc eSource logical architecture, and relevant regulations and guidelines, we designed a source data collection method and invited relevant stakeholders to optimize it. On the basis of this method, we proposed the eSource record (ESR) system as a solution and invited experts with different roles in the contract research organization company to discuss and design a flowchart for data connection between the ESR and electronic data capture (EDC). Results: The ESR method included 5 steps: research project preparation, initial survey collection, in-hospital medical record writing, out-of-hospital follow-up, and electronic case report form (eCRF) traceability. The data connection between the ESR and EDC covered the clinical research process from creating the eCRF to collecting data for the analysis. The intelligent data acquisition function of the ESR will automatically complete the empty eCRF to create an eCRF with values. When the clinical research associate and data manager conduct data verification, they can query the certified copy database through interface traceability and send data queries. The data queries are transmitted to the ESR through the EDC interface. The EDC and EMR systems interoperate through the ESR. The EMR and EDC systems transmit data to the ESR system through the data standards of the Health Level Seven Clinical Document Architecture and the Clinical Data Interchange Standards Consortium operational data model, respectively. When the implemented data standards for a given system are not consistent, the ESR will approach the problem by first automating mappings between standards and then handling extensions or corrections to a given data format through human evaluation. Conclusions: The source data collection method proposed in this study will help to realize eSource’s new strategy. The ESR solution is standardized and sustainable. It aims to ensure that research data meet the attributable, legible, contemporaneous, original, accurate, complete, consistent, enduring, and available standards for clinical research data quality and to provide a new model for prospective data collection in real-world studies. %M 36563036 %R 10.2196/42754 %U https://www.researchprotocols.org/2022/12/e42754 %U https://doi.org/10.2196/42754 %U http://www.ncbi.nlm.nih.gov/pubmed/36563036 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 12 %P e43086 %T Patient-Centered Digital Health Records and Their Effects on Health Outcomes: Systematic Review %A Brands,Martijn R %A Gouw,Samantha C %A Beestrum,Molly %A Cronin,Robert M %A Fijnvandraat,Karin %A Badawy,Sherif M %+ Department of Pediatrics, Feinberg School of Medicine, Northwestern University, 420 E Superior St, Chicago, IL, 60611, United States, 1 312 227 4789, sherif.badawy@northwestern.edu %K telemedicine %K health records %K personal %K electronic health records %K outcome assessment %K health care %D 2022 %7 22.12.2022 %9 Review %J J Med Internet Res %G English %X Background: eHealth tools such as patient portals and personal health records, also known as patient-centered digital health records, can engage and empower individuals with chronic health conditions. Patients who are highly engaged in their care have improved disease knowledge, self-management skills, and clinical outcomes. Objective: We aimed to systematically review the effects of patient-centered digital health records on clinical and patient-reported outcomes, health care utilization, and satisfaction among patients with chronic conditions and to assess the feasibility and acceptability of their use. Methods: We searched MEDLINE, Cochrane, CINAHL, Embase, and PsycINFO databases between January 2000 and December 2021. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were followed. Eligible studies were those evaluating digital health records intended for nonhospitalized adult or pediatric patients with a chronic condition. Patients with a high disease burden were a subgroup of interest. Primary outcomes included clinical and patient-reported health outcomes and health care utilization. Secondary outcomes included satisfaction, feasibility, and acceptability. Joanna Briggs Institute critical appraisal tools were used for quality assessment. Two reviewers screened titles, abstracts, and full texts. Associations between health record use and outcomes were categorized as beneficial, neutral or clinically nonrelevant, or undesired. Results: Of the 7716 unique publications examined, 81 (1%) met the eligibility criteria, with a total of 1,639,556 participants across all studies. The most commonly studied diseases included diabetes mellitus (37/81, 46%), cardiopulmonary conditions (21/81, 26%), and hematology-oncology conditions (14/81, 17%). One-third (24/81, 30%) of the studies were randomized controlled trials. Of the 81 studies that met the eligibility criteria, 16 (20%) were of high methodological quality. Reported outcomes varied across studies. The benefits of patient-centered digital health records were most frequently reported in the category health care utilization on the “use of recommended care services” (10/13, 77%), on the patient-reported outcomes “disease knowledge” (7/10, 70%), “patient engagement” (13/28, 56%), “treatment adherence” (10/18, 56%), and “self-management and self-efficacy” (10/19, 53%), and on the clinical outcome “laboratory parameters,” including HbA1c and low-density lipoprotein (LDL; 16/33, 48%). Beneficial effects on “health-related quality of life” were seen in only 27% (4/15) of studies. Patient satisfaction (28/30, 93%), feasibility (15/19, 97%), and acceptability (23/26, 88%) were positively evaluated. More beneficial effects were reported for digital health records that predominantly focus on active features. Beneficial effects were less frequently observed among patients with a high disease burden and among high-quality studies. No unfavorable effects were observed. Conclusions: The use of patient-centered digital health records in nonhospitalized individuals with chronic health conditions is potentially associated with considerable beneficial effects on health care utilization, treatment adherence, and self-management or self-efficacy. However, for firm conclusions, more studies of high methodological quality are required. Trial Registration: PROSPERO (International Prospective Register of Systematic Reviews) CRD42020213285; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=213285 %M 36548034 %R 10.2196/43086 %U https://www.jmir.org/2022/12/e43086 %U https://doi.org/10.2196/43086 %U http://www.ncbi.nlm.nih.gov/pubmed/36548034 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 12 %P e40534 %T Natural Language Processing and Graph Theory: Making Sense of Imaging Records in a Novel Representation Frame %A Binsfeld Gonçalves,Laurent %A Nesic,Ivan %A Obradovic,Marko %A Stieltjes,Bram %A Weikert,Thomas %A Bremerich,Jens %+ Clinic of Radiology & Nuclear Medicine, University Hospital Basel, University of Basel, Petersgraben, 4, Basel, 4031, Switzerland, 352 621517916, laurent.binsfeld@gmail.com %K radiology %K deep learning %K NLP %K radiology reports %K imaging record %K temporal referrals %K date extraction %K graph theory %K health care information system %K resource planning. %D 2022 %7 21.12.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: A concise visualization framework of related reports would increase readability and improve patient management. To this end, temporal referrals to prior comparative exams are an essential connection to previous exams in written reports. Due to unstructured narrative texts' variable structure and content, their extraction is hampered by poor computer readability. Natural language processing (NLP) permits the extraction of structured information from unstructured texts automatically and can serve as an essential input for such a novel visualization framework. Objective: This study proposes and evaluates an NLP-based algorithm capable of extracting the temporal referrals in written radiology reports, applies it to all the radiology reports generated for 10 years, introduces a graphical representation of imaging reports, and investigates its benefits for clinical and research purposes. Methods: In this single-center, university hospital, retrospective study, we developed a convolutional neural network capable of extracting the date of referrals from imaging reports. The model's performance was assessed by calculating precision, recall, and F1-score using an independent test set of 149 reports. Next, the algorithm was applied to our department's radiology reports generated from 2011 to 2021. Finally, the reports and their metadata were represented in a modulable graph. Results: For extracting the date of referrals, the named-entity recognition (NER) model had a high precision of 0.93, a recall of 0.95, and an F1-score of 0.94. A total of 1,684,635 reports were included in the analysis. Temporal reference was mentioned in 53.3% (656,852/1,684,635), explicitly stated as not available in 21.0% (258,386/1,684,635), and omitted in 25.7% (317,059/1,684,635) of the reports. Imaging records can be visualized in a directed and modulable graph, in which the referring links represent the connecting arrows. Conclusions: Automatically extracting the date of referrals from unstructured radiology reports using deep learning NLP algorithms is feasible. Graphs refined the selection of distinct pathology pathways, facilitated the revelation of missing comparisons, and enabled the query of specific referring exam sequences. Further work is needed to evaluate its benefits in clinics, research, and resource planning. %M 36542426 %R 10.2196/40534 %U https://medinform.jmir.org/2022/12/e40534 %U https://doi.org/10.2196/40534 %U http://www.ncbi.nlm.nih.gov/pubmed/36542426 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 12 %P e42664 %T Telehealth System Based on the Ontology Design of a Diabetes Management Pathway Model in China: Development and Usability Study %A Fan,ZhiYuan %A Cui,LiYuan %A Ye,Ying %A Li,ShouCheng %A Deng,Ning %+ College of Biomedical Engineering and Instrument Science, Ministry of Education Key Laboratory of Biomedical Engineering, Zhejiang University, Zhouyiqing Bldg 512, 38 Zheda Rd, Hangzhou, 310000, China, 86 571 2295 2693, zju.dengning@gmail.com %K diabetes %K chronic disease management %K Chronic Disease Management Pathway %K ontology %K Semantic Web Rule Language rules %K SWRL rules %D 2022 %7 19.12.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Diabetes needs to be under control through management and intervention. Management of diabetes through mobile health is a practical approach; however, most diabetes mobile health management systems do not meet expectations, which may be because of the lack of standardized management processes in the systems and the lack of intervention implementation recommendations in the management knowledge base. Objective: In this study, we aimed to construct a diabetes management care pathway suitable for the actual situation in China to express the diabetes management care pathway using ontology and develop a diabetes closed-loop system based on the construction results of the diabetes management pathway and apply it practically. Methods: This study proposes a diabetes management care pathway model in which the management process of diabetes is divided into 9 management tasks, and the Diabetes Care Pathway Ontology (DCPO) is constructed to represent the knowledge contained in this pathway model. A telehealth system, which can support the comprehensive management of patients with diabetes while providing active intervention by physicians, was designed and developed based on the DCPO. A retrospective study was performed based on the data records extracted from the system to analyze the usability and treatment effects of the DCPO. Results: The diabetes management pathway ontology constructed in this study contains 119 newly added classes, 28 object properties, 58 data properties, 81 individuals, 426 axioms, and 192 Semantic Web Rule Language rules. The developed mobile medical system was applied to 272 patients with diabetes. Within 3 months, the average fasting blood glucose of the patients decreased by 1.34 mmol/L (P=.003), and the average 2-hour postprandial blood glucose decreased by 2.63 mmol/L (P=.003); the average systolic and diastolic blood pressures decreased by 11.84 mmHg (P=.02) and 8.8 mmHg (P=.02), respectively. In patients who received physician interventions owing to abnormal attention or low-compliance warnings, the average fasting blood glucose decreased by 2.45 mmol/L (P=.003), and the average 2-hour postprandial blood glucose decreased by 2.89 mmol/L (P=.003) in all patients with diabetes; the average systolic and diastolic blood pressure decreased by 20.06 mmHg (P=.02) and 17.37 mmHg (P=.02), respectively, in patients with both hypertension and diabetes during the 3-month management period. Conclusions: This study helps guide the timing and content of interactive interventions between physicians and patients and regulates physicians’ medical service behavior. Different management plans are formulated for physicians and patients according to different characteristics to comprehensively manage various cardiovascular risk factors. The application of the DCPO in the diabetes management system can provide effective and adequate management support for patients with diabetes and those with both diabetes and hypertension. %M 36534448 %R 10.2196/42664 %U https://medinform.jmir.org/2022/12/e42664 %U https://doi.org/10.2196/42664 %U http://www.ncbi.nlm.nih.gov/pubmed/36534448 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 12 %P e42379 %T Construction of Cohorts of Similar Patients From Automatic Extraction of Medical Concepts: Phenotype Extraction Study %A Gérardin,Christel %A Mageau,Arthur %A Mékinian,Arsène %A Tannier,Xavier %A Carrat,Fabrice %+ Institute Pierre Louis Epidemiology and Public Health, Institut National de la Santé et de la Recherche Médicale, Sorbonne Université, 27 rue de Chaligny, Paris, 75012, France, 33 678148466, christel.ducroz-gerardin@iplesp.upmc.fr %K natural language processing %K similar patient cohort %K phenotype %K systemic disease %K NLP %K algorithm %K automatic extraction %K automated extraction %K named entity %K MeSH %K medical subject heading %K data extraction %K text extraction %D 2022 %7 19.12.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Reliable and interpretable automatic extraction of clinical phenotypes from large electronic medical record databases remains a challenge, especially in a language other than English. Objective: We aimed to provide an automated end-to-end extraction of cohorts of similar patients from electronic health records for systemic diseases. Methods: Our multistep algorithm includes a named-entity recognition step, a multilabel classification using medical subject headings ontology, and the computation of patient similarity. A selection of cohorts of similar patients on a priori annotated phenotypes was performed. Six phenotypes were selected for their clinical significance: P1, osteoporosis; P2, nephritis in systemic erythematosus lupus; P3, interstitial lung disease in systemic sclerosis; P4, lung infection; P5, obstetric antiphospholipid syndrome; and P6, Takayasu arteritis. We used a training set of 151 clinical notes and an independent validation set of 256 clinical notes, with annotated phenotypes, both extracted from the Assistance Publique-Hôpitaux de Paris data warehouse. We evaluated the precision of the 3 patients closest to the index patient for each phenotype with precision-at-3 and recall and average precision. Results: For P1-P4, the precision-at-3 ranged from 0.85 (95% CI 0.75-0.95) to 0.99 (95% CI 0.98-1), the recall ranged from 0.53 (95% CI 0.50-0.55) to 0.83 (95% CI 0.81-0.84), and the average precision ranged from 0.58 (95% CI 0.54-0.62) to 0.88 (95% CI 0.85-0.90). P5-P6 phenotypes could not be analyzed due to the limited number of phenotypes. Conclusions: Using a method close to clinical reasoning, we built a scalable and interpretable end-to-end algorithm for extracting cohorts of similar patients. %M 36534446 %R 10.2196/42379 %U https://medinform.jmir.org/2022/12/e42379 %U https://doi.org/10.2196/42379 %U http://www.ncbi.nlm.nih.gov/pubmed/36534446 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 12 %P e37833 %T Implementation of Machine Learning Pipelines for Clinical Practice: Development and Validation Study %A Kanbar,Lara J %A Wissel,Benjamin %A Ni,Yizhao %A Pajor,Nathan %A Glauser,Tracy %A Pestian,John %A Dexheimer,Judith W %+ Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, 3333 Burnet Avenue, Cincinnati, OH, 45229, United States, 1 5138032962, judith.dexheimer@cchmc.org %K electronic health record %K natural language processing %K epilepsy %K clinical decision support %K machine learning %K emergency medicine %K artificial intelligence %D 2022 %7 16.12.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Artificial intelligence (AI) technologies, such as machine learning and natural language processing, have the potential to provide new insights into complex health data. Although powerful, these algorithms rarely move from experimental studies to direct clinical care implementation. Objective: We aimed to describe the key components for successful development and integration of two AI technology–based research pipelines for clinical practice. Methods: We summarized the approach, results, and key learnings from the implementation of the following two systems implemented at a large, tertiary care children’s hospital: (1) epilepsy surgical candidate identification (or epilepsy ID) in an ambulatory neurology clinic; and (2) an automated clinical trial eligibility screener (ACTES) for the real-time identification of patients for research studies in a pediatric emergency department. Results: The epilepsy ID system performed as well as board-certified neurologists in identifying surgical candidates (with a sensitivity of 71% and positive predictive value of 77%). The ACTES system decreased coordinator screening time by 12.9%. The success of each project was largely dependent upon the collaboration between machine learning experts, research and operational information technology professionals, longitudinal support from clinical providers, and institutional leadership. Conclusions: These projects showcase novel interactions between machine learning recommendations and providers during clinical care. Our deployment provides seamless, real-time integration of AI technology to provide decision support and improve patient care. %M 36525289 %R 10.2196/37833 %U https://medinform.jmir.org/2022/12/e37833 %U https://doi.org/10.2196/37833 %U http://www.ncbi.nlm.nih.gov/pubmed/36525289 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 12 %P e43229 %T Electronic Source Data Transcription for Electronic Case Report Forms in China: Validation of the Electronic Source Record Tool in a Real-world Ophthalmology Study %A Wang,Bin %A Lai,Junkai %A Liu,Mimi %A Jin,Feifei %A Peng,Yifei %A Yao,Chen %+ Peking University Clinical Research Institute, Peking University First Hospital, No 8, Xishiku Street, Xicheng District, Beijing, 100034, China, 86 01083325822, yaochen@hsc.pku.edu.cn %K electronic medical record %K electronic health record %K electronic source %K eSource %K eSource record tool %K real-world data %K data transcription %K data quality %K System Usability Scale %K ophthalmology %D 2022 %7 16.12.2022 %9 Original Paper %J JMIR Form Res %G English %X Background: As researchers are increasingly interested in real-world studies (RWSs), improving data collection efficiency and data quality has become an important challenge. An electronic source (eSource) generally includes direct capture, collection, and storage of electronic data to simplify clinical research. It can improve data quality and patient safety and reduce clinical trial costs. Although there are already large projects on eSource technology, there is a lack of experience in using eSource technology to implement RWSs. Our team designed and developed an eSource record (ESR) system in China. In a preliminary prospective study, we selected a cosmetic medical device project to evaluate ESR software’s effect on data collection and transcription. As the previous case verification was simple, we plan to choose more complicated ophthalmology projects to further evaluate the ESR. Objective: We aimed to evaluate the data transcription efficiency and quality of ESR software in retrospective studies to verify the feasibility of using eSource as an alternative to traditional manual transcription of data in RWS projects. Methods: The approved ophthalmic femtosecond laser project was used for ESR case validation. This study compared the efficiency and quality of data transcription between the eSource method using ESR software and the traditional clinical research model of manually transcribing the data. Usability refers to the quality of a user’s experience when interacting with products or systems including websites, software, devices, or applications. To evaluate the system availability of ESR, we used the System Usability Scale (SUS). The questionnaire consisted of the following 2 parts: participant information and SUS evaluation of the electronic medical record (EMR), electronic data capture (EDC), and ESR systems. By accessing log data from the EDC system previously used by the research project, all the time spent from the beginning to the end of the study could be counted. Results: In terms of transcription time cost per field, the eSource method can reduce the time cost by 81.8% (11.2/13.7). Compared with traditional manual data transcription, the eSource method has higher data transcription quality (correct entry rate of 2356/2400, 98.17% vs 47,991/51,424, 93.32%). A total of 15 questionnaires were received with a response rate of 100%. In terms of usability, the average overall SUS scores of the EMR, EDC, and ESR systems were 50.3 (SD 21.9), 51.5 (SD 14.2), and 63.0 (SD 11.3; contract research organization experts: 69.5, SD 11.5; clinicians: 59.8, SD 10.2), respectively. The Cronbach α for the SUS items of the EMR, EDC, and ESR systems were 0.591 (95% CI −0.012 to 0.903), 0.588 (95% CI −0.288 to 0.951), and 0.785 (95% CI 0.576-0.916), respectively. Conclusions: In real-world ophthalmology studies, the eSource approach based on the ESR system can replace the traditional clinical research model that relies on the manual transcription of data. %M 36525285 %R 10.2196/43229 %U https://formative.jmir.org/2022/12/e43229 %U https://doi.org/10.2196/43229 %U http://www.ncbi.nlm.nih.gov/pubmed/36525285 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 12 %P e40743 %T State-of-the-Art Evidence Retriever for Precision Medicine: Algorithm Development and Validation %A Jin,Qiao %A Tan,Chuanqi %A Chen,Mosha %A Yan,Ming %A Zhang,Ningyu %A Huang,Songfang %A Liu,Xiaozhong %+ Alibaba Group, No. 969 West Wen Yi Road, Yuhang District, Hangzhou, 311121, China, 86 15201162567, chuanqi.tcq@alibaba-inc.com %K precision medicine %K evidence-based medicine %K information retrieval %K active learning %K pretrained language models %K digital health intervention %K data retrieval %K big data %K algorithm development %D 2022 %7 15.12.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Under the paradigm of precision medicine (PM), patients with the same disease can receive different personalized therapies according to their clinical and genetic features. These therapies are determined by the totality of all available clinical evidence, including results from case reports, clinical trials, and systematic reviews. However, it is increasingly difficult for physicians to find such evidence from scientific publications, whose size is growing at an unprecedented pace. Objective: In this work, we propose the PM-Search system to facilitate the retrieval of clinical literature that contains critical evidence for or against giving specific therapies to certain cancer patients. Methods: The PM-Search system combines a baseline retriever that selects document candidates at a large scale and an evidence reranker that finely reorders the candidates based on their evidence quality. The baseline retriever uses query expansion and keyword matching with the ElasticSearch retrieval engine, and the evidence reranker fits pretrained language models to expert annotations that are derived from an active learning strategy. Results: The PM-Search system achieved the best performance in the retrieval of high-quality clinical evidence at the Text Retrieval Conference PM Track 2020, outperforming the second-ranking systems by large margins (0.4780 vs 0.4238 for standard normalized discounted cumulative gain at rank 30 and 0.4519 vs 0.4193 for exponential normalized discounted cumulative gain at rank 30). Conclusions: We present PM-Search, a state-of-the-art search engine to assist the practicing of evidence-based PM. PM-Search uses a novel Bidirectional Encoder Representations from Transformers for Biomedical Text Mining–based active learning strategy that models evidence quality and improves the model performance. Our analyses show that evidence quality is a distinct aspect from general relevance, and specific modeling of evidence quality beyond general relevance is required for a PM search engine. %M 36409468 %R 10.2196/40743 %U https://medinform.jmir.org/2022/12/e40743 %U https://doi.org/10.2196/40743 %U http://www.ncbi.nlm.nih.gov/pubmed/36409468 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 12 %P e43757 %T Model for Predicting In-Hospital Mortality of Physical Trauma Patients Using Artificial Intelligence Techniques: Nationwide Population-Based Study in Korea %A Lee,Seungseok %A Kang,Wu Seong %A Seo,Sanghyun %A Kim,Do Wan %A Ko,Hoon %A Kim,Joongsuck %A Lee,Seonghwa %A Lee,Jinseok %+ Department of Biomedical Engineering, Kyung Hee University, 1732, Deogyeong-daero, Giheung-gu, Yong-in, 17104, Republic of Korea, 82 312012570, gonasago@khu.ac.kr %K artificial intelligence %K trauma %K mortality prediction %K international classification of disease %K injury %K prediction model %K severity score %K emergency department %K Information system %K deep neural network %D 2022 %7 13.12.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Physical trauma–related mortality places a heavy burden on society. Estimating the mortality risk in physical trauma patients is crucial to enhance treatment efficiency and reduce this burden. The most popular and accurate model is the Injury Severity Score (ISS), which is based on the Abbreviated Injury Scale (AIS), an anatomical injury severity scoring system. However, the AIS requires specialists to code the injury scale by reviewing a patient's medical record; therefore, applying the model to every hospital is impossible. Objective: We aimed to develop an artificial intelligence (AI) model to predict in-hospital mortality in physical trauma patients using the International Classification of Disease 10th Revision (ICD-10), triage scale, procedure codes, and other clinical features. Methods: We used the Korean National Emergency Department Information System (NEDIS) data set (N=778,111) compiled from over 400 hospitals between 2016 and 2019. To predict in-hospital mortality, we used the following as input features: ICD-10, patient age, gender, intentionality, injury mechanism, and emergent symptom, Alert/Verbal/Painful/Unresponsive (AVPU) scale, Korean Triage and Acuity Scale (KTAS), and procedure codes. We proposed the ensemble of deep neural networks (EDNN) via 5-fold cross-validation and compared them with other state-of-the-art machine learning models, including traditional prediction models. We further investigated the effect of the features. Results: Our proposed EDNN with all features provided the highest area under the receiver operating characteristic (AUROC) curve of 0.9507, outperforming other state-of-the-art models, including the following traditional prediction models: Adaptive Boosting (AdaBoost; AUROC of 0.9433), Extreme Gradient Boosting (XGBoost; AUROC of 0.9331), ICD-based ISS (AUROC of 0.8699 for an inclusive model and AUROC of 0.8224 for an exclusive model), and KTAS (AUROC of 0.1841). In addition, using all features yielded a higher AUROC than any other partial features, namely, EDNN with the features of ICD-10 only (AUROC of 0.8964) and EDNN with the features excluding ICD-10 (AUROC of 0.9383). Conclusions: Our proposed EDNN with all features outperforms other state-of-the-art models, including the traditional diagnostic code-based prediction model and triage scale. %M 36512392 %R 10.2196/43757 %U https://www.jmir.org/2022/12/e43757 %U https://doi.org/10.2196/43757 %U http://www.ncbi.nlm.nih.gov/pubmed/36512392 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 12 %P e41312 %T Pan-Canadian Electronic Medical Record Diagnostic and Unstructured Text Data for Capturing PTSD: Retrospective Observational Study %A Kosowan,Leanne %A Singer,Alexander %A Zulkernine,Farhana %A Zafari,Hasan %A Nesca,Marcello %A Muthumuni,Dhasni %+ Department of Family Medicine, Rady Faculty of Health Sciences, University of Manitoba, D009-780 Bannatyne Ave., Winnipeg, MB, R3E0W2, Canada, 1 204 789 3314, alexander.singer@umanitoba.ca %K electronic health records %K EHR %K natural language processing %K NLP %K medical informatics %K primary health care %K stress disorders, posttraumatic %K posttraumatic stress disorder %K PTSD %D 2022 %7 13.12.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: The availability of electronic medical record (EMR) free-text data for research varies. However, access to short diagnostic text fields is more widely available. Objective: This study assesses agreement between free-text and short diagnostic text data from primary care EMR for identification of posttraumatic stress disorder (PTSD). Methods: This retrospective cross-sectional study used EMR data from a pan-Canadian repository representing 1574 primary care providers at 265 clinics using 11 EMR vendors. Medical record review using free text and short diagnostic text fields of the EMR produced reference standards for PTSD. Agreement was assessed with sensitivity, specificity, positive predictive value, negative predictive value, and accuracy. Results: Our reference set contained 327 patients with free text and short diagnostic text. Among these patients, agreement between free text and short diagnostic text had an accuracy of 93.6% (CI 90.4%-96.0%). In a single Canadian province, case definitions 1 and 4 had a sensitivity of 82.6% (CI 74.4%-89.0%) and specificity of 99.5% (CI 97.4%-100%). However, when the reference set was expanded to a pan-Canada reference (n=12,104 patients), case definition 4 had the strongest agreement (sensitivity: 91.1%, CI 90.1%-91.9%; specificity: 99.1%, CI 98.9%-99.3%). Conclusions: Inclusion of free-text encounter notes during medical record review did not lead to improved capture of PTSD cases, nor did it lead to significant changes in case definition agreement. Within this pan-Canadian database, jurisdictional differences in diagnostic codes and EMR structure suggested the need to supplement diagnostic codes with natural language processing to capture PTSD. When unavailable, short diagnostic text can supplement free-text data for reference set creation and case validation. Application of the PTSD case definition can inform PTSD prevalence and characteristics. %M 36512389 %R 10.2196/41312 %U https://medinform.jmir.org/2022/12/e41312 %U https://doi.org/10.2196/41312 %U http://www.ncbi.nlm.nih.gov/pubmed/36512389 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 11 %N 12 %P e43741 %T Contemporary Databases in Real-world Studies Regarding the Diverse Health Care Systems of India, Thailand, and Taiwan: Protocol for a Scoping Review %A Shau,Wen-Yi %A Setia,Sajita %A Shinde,Salil Prakash %A Santoso,Handoko %A Furtner,Daniel %+ Executive Office, Transform Medical Communications Limited, 184 Glasgow Street, Wanganui, 4500, New Zealand, 64 276175433, sajita.setia@transform-medcomms.com %K Asia %K health care databases %K real-world data %K real-world evidence %K scoping review %D 2022 %7 13.12.2022 %9 Protocol %J JMIR Res Protoc %G English %X Background: Real-world data (RWD) related to patient health status or health care delivery can be broadly defined as data collected outside of conventional clinical trials, including those from databases, treatment and disease registries, electronic medical records, insurance claims, and information directly contributed by health care professionals or patients. RWD are used to generate real-world evidence (RWE), which is increasingly relevant to policy makers in Asia, who use RWE to support decision-making in several areas, including public health policy, regulatory health technology assessment, and reimbursement; set priorities; or inform clinical practice. Objective: To support the achievement of the benefits of RWE in Asian health care strategies and policies, we sought to identify the linked contemporary databases used in real-world studies from three representative countries—India, Thailand, and Taiwan—and explore variations in results based on these countries’ economies and health care reimbursement systems by performing a systematic scoping review. Herein, we describe the protocol and preliminary findings of our scoping review. Methods: The PubMed search strategy covered 3 concepts. Concept 1 was designed to identify potential RWE and RWD studies by applying various Medical Subject Headings (MeSH) terms (“Treatment Outcome,” “Evidence-Based Medicine,” “Retrospective Studies,” and “Time Factors”) and related keywords (eg, “real-world,” “actual life,” and “actual practice”). Concept 2 introduced the three countries—India, Taiwan, and Thailand. Concept 3 focused on data types, using a combination of MeSH terms (“Electronic Health Records,” “Insurance, Health,” “Registries,” “Databases, Pharmaceutical,” and “Pharmaceutical Services”) and related keywords (eg, “electronic medical record,” “electronic healthcare record,” “EMR,” “EHR,” “administrative database,” and “registry”). These searches were conducted with filters for language (English) and publication date (publications in the last 5 years before the search). The retrieved articles will undergo 2 screening phases (phase 1: review of titles and abstracts; phase 2: review of full texts) to identify relevant and eligible articles for data extraction. The data to be extracted from eligible studies will include the characteristics of databases, the regions covered, and the patient populations. Results: The literature search was conducted on September 27, 2022. We retrieved 3,172,434, 1,094,125, and 672,794 articles for concepts 1, 2, and 3, respectively. After applying all 3 concepts and the language and publication date filters, 2277 articles were identified. These will be further screened to identify eligible studies. Based on phase 1 screening and our progress to date, approximately 44% (1003/2277) of articles have undergone phase 2 screening to judge their eligibility. Around 800 studies will be used for data extraction. Conclusions: Our research will be crucial for nurturing advancement in RWD generation within Asia by identifying linked clinical RWD databases and new avenues for public-private partnerships and multiple collaborations for expanding the scope and spectrum of high-quality, robust RWE generation in Asia. International Registered Report Identifier (IRRID): DERR1-10.2196/43741 %M 36512386 %R 10.2196/43741 %U https://www.researchprotocols.org/2022/12/e43741 %U https://doi.org/10.2196/43741 %U http://www.ncbi.nlm.nih.gov/pubmed/36512386 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 12 %P e41819 %T A Machine Learning-Based Approach to Predict Prognosis and Length of Hospital Stay in Adults and Children With Traumatic Brain Injury: Retrospective Cohort Study %A Fang,Cheng %A Pan,Yifeng %A Zhao,Luotong %A Niu,Zhaoyi %A Guo,Qingguo %A Zhao,Bing %+ Department of Neurosurgery, The Second Affiliated Hospital of Anhui Medical University, Anhui Medical University, No. 678, Furong Raod, Hefei, 230601, China, 86 138 6611 2073, aydzhb@126.com %K convolutional neural network %K machine learning %K neurosurgery %K support vector machine %K support vector regression %K traumatic brain injury %D 2022 %7 9.12.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: The treatment and care of adults and children with traumatic brain injury (TBI) constitute an intractable global health problem. Predicting the prognosis and length of hospital stay of patients with TBI may improve therapeutic effects and significantly reduce societal health care burden. Applying novel machine learning methods to the field of TBI may be valuable for determining the prognosis and cost-effectiveness of clinical treatment. Objective: We aimed to combine multiple machine learning approaches to build hybrid models for predicting the prognosis and length of hospital stay for adults and children with TBI. Methods: We collected relevant clinical information from patients treated at the Neurosurgery Center of the Second Affiliated Hospital of Anhui Medical University between May 2017 and May 2022, of which 80% was used for training the model and 20% for testing via screening and data splitting. We trained and tested the machine learning models using 5 cross-validations to avoid overfitting. In the machine learning models, 11 types of independent variables were used as input variables and Glasgow Outcome Scale score, used to evaluate patients’ prognosis, and patient length of stay were used as output variables. Once the models were trained, we obtained and compared the errors of each machine learning model from 5 rounds of cross-validation to select the best predictive model. The model was then externally tested using clinical data of patients treated at the First Affiliated Hospital of Anhui Medical University from June 2021 to February 2022. Results: The final convolutional neural network–support vector machine (CNN-SVM) model predicted Glasgow Outcome Scale score with an accuracy of 93% and 93.69% in the test and external validation sets, respectively, and an area under the curve of 94.68% and 94.32% in the test and external validation sets, respectively. The mean absolute percentage error of the final built convolutional neural network–support vector regression (CNN-SVR) model predicting inpatient time in the test set and external validation set was 10.72% and 10.44%, respectively. The coefficient of determination (R2) was 0.93 and 0.92 in the test set and external validation set, respectively. Compared with back-propagation neural network, CNN, and SVM models built separately, our hybrid model was identified to be optimal and had high confidence. Conclusions: This study demonstrates the clinical utility of 2 hybrid models built by combining multiple machine learning approaches to accurately predict the prognosis and length of stay in hospital for adults and children with TBI. Application of these models may reduce the burden on physicians when assessing TBI and assist clinicians in the medical decision-making process. %M 36485032 %R 10.2196/41819 %U https://www.jmir.org/2022/12/e41819 %U https://doi.org/10.2196/41819 %U http://www.ncbi.nlm.nih.gov/pubmed/36485032 %0 Journal Article %@ 2561-9128 %I JMIR Publications %V 5 %N 1 %P e40831 %T An Accessible Clinical Decision Support System to Curtail Anesthetic Greenhouse Gases in a Large Health Network: Implementation Study %A Ramaswamy,Priya %A Shah,Aalap %A Kothari,Rishi %A Schloemerkemper,Nina %A Methangkool,Emily %A Aleck,Amalia %A Shapiro,Anne %A Dayal,Rakhi %A Young,Charlotte %A Spinner,Jon %A Deibler,Carly %A Wang,Kaiyi %A Robinowitz,David %A Gandhi,Seema %+ Department of Anesthesia and Perioperative Care, University of California, San Francisco, 521 Parnassus Avenue, Box 0648, San Francisco, CA, 94143-0131, United States, 1 415 476 9043, priya.ramaswamy@ucsf.edu %K clinical decision support %K sustainability %K intraoperative %K perioperative %K anesthetic gas %K waste reduction %K fresh gas flow %D 2022 %7 8.12.2022 %9 Original Paper %J JMIR Perioper Med %G English %X Background: Inhaled anesthetics in the operating room are potent greenhouse gases and are a key contributor to carbon emissions from health care facilities. Real-time clinical decision support (CDS) systems lower anesthetic gas waste by prompting anesthesia professionals to reduce fresh gas flow (FGF) when a set threshold is exceeded. However, previous CDS systems have relied on proprietary or highly customized anesthesia information management systems, significantly reducing other institutions’ accessibility to the technology and thus limiting overall environmental benefit. Objective: In 2018, a CDS system that lowers anesthetic gas waste using methods that can be easily adopted by other institutions was developed at the University of California San Francisco (UCSF). This study aims to facilitate wider uptake of our CDS system and further reduce gas waste by describing the implementation of the FGF CDS toolkit at UCSF and the subsequent implementation at other medical campuses within the University of California Health network. Methods: We developed a noninterruptive active CDS system to alert anesthesia professionals when FGF rates exceeded 0.7 L per minute for common volatile anesthetics. The implementation process at UCSF was documented and assembled into an informational toolkit to aid in the integration of the CDS system at other health care institutions. Before implementation, presentation-based education initiatives were used to disseminate information regarding the safety of low FGF use and its relationship to environmental sustainability. Our FGF CDS toolkit consisted of 4 main components for implementation: sustainability-focused education of anesthesia professionals, hardware integration of the CDS technology, software build of the CDS system, and data reporting of measured outcomes. Results: The FGF CDS system was successfully deployed at 5 University of California Health network campuses. Four of the institutions are independent from the institution that created the CDS system. The CDS system was deployed at each facility using the FGF CDS toolkit, which describes the main components of the technology and implementation. Each campus made modifications to the CDS tool to best suit their institution, emphasizing the versatility and adoptability of the technology and implementation framework. Conclusions: It has previously been shown that the FGF CDS system reduces anesthetic gas waste, leading to environmental and fiscal benefits. Here, we demonstrate that the CDS system can be transferred to other medical facilities using our toolkit for implementation, making the technology and associated benefits globally accessible to advance mitigation of health care–related emissions. %M 36480254 %R 10.2196/40831 %U https://periop.jmir.org/2022/1/e40831 %U https://doi.org/10.2196/40831 %U http://www.ncbi.nlm.nih.gov/pubmed/36480254 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 12 %P e37239 %T A Framework for Modeling and Interpreting Patient Subgroups Applied to Hospital Readmission: Visual Analytical Approach %A Bhavnani,Suresh K %A Zhang,Weibin %A Visweswaran,Shyam %A Raji,Mukaila %A Kuo,Yong-Fang %+ School of Public and Population Health, University of Texas Medical Branch, Institute for Translational Sciences, 301 University Blvd., Galveston, TX, 77555-0129, United States, 1 (409) 772 1928, subhavna@utmb.edu %K visual analytics %K Bipartite Network analysis %K hospital readmission %K precision medicine %K modeling %K Medicare %D 2022 %7 7.12.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: A primary goal of precision medicine is to identify patient subgroups and infer their underlying disease processes with the aim of designing targeted interventions. Although several studies have identified patient subgroups, there is a considerable gap between the identification of patient subgroups and their modeling and interpretation for clinical applications. Objective: This study aimed to develop and evaluate a novel analytical framework for modeling and interpreting patient subgroups (MIPS) using a 3-step modeling approach: visual analytical modeling to automatically identify patient subgroups and their co-occurring comorbidities and determine their statistical significance and clinical interpretability; classification modeling to classify patients into subgroups and measure its accuracy; and prediction modeling to predict a patient’s risk of an adverse outcome and compare its accuracy with and without patient subgroup information. Methods: The MIPS framework was developed using bipartite networks to identify patient subgroups based on frequently co-occurring high-risk comorbidities, multinomial logistic regression to classify patients into subgroups, and hierarchical logistic regression to predict the risk of an adverse outcome using subgroup membership compared with standard logistic regression without subgroup membership. The MIPS framework was evaluated for 3 hospital readmission conditions: chronic obstructive pulmonary disease (COPD), congestive heart failure (CHF), and total hip arthroplasty/total knee arthroplasty (THA/TKA) (COPD: n=29,016; CHF: n=51,550; THA/TKA: n=16,498). For each condition, we extracted cases defined as patients readmitted within 30 days of hospital discharge. Controls were defined as patients not readmitted within 90 days of discharge, matched by age, sex, race, and Medicaid eligibility. Results: In each condition, the visual analytical model identified patient subgroups that were statistically significant (Q=0.17, 0.17, 0.31; P<.001, <.001, <.05), significantly replicated (Rand Index=0.92, 0.94, 0.89; P<.001, <.001, <.01), and clinically meaningful to clinicians. In each condition, the classification model had high accuracy in classifying patients into subgroups (mean accuracy=99.6%, 99.34%, 99.86%). In 2 conditions (COPD and THA/TKA), the hierarchical prediction model had a small but statistically significant improvement in discriminating between readmitted and not readmitted patients as measured by net reclassification improvement (0.059, 0.11) but not as measured by the C-statistic or integrated discrimination improvement. Conclusions: Although the visual analytical models identified statistically and clinically significant patient subgroups, the results pinpoint the need to analyze subgroups at different levels of granularity for improving the interpretability of intra- and intercluster associations. The high accuracy of the classification models reflects the strong separation of patient subgroups, despite the size and density of the data sets. Finally, the small improvement in predictive accuracy suggests that comorbidities alone were not strong predictors of hospital readmission, and the need for more sophisticated subgroup modeling methods. Such advances could improve the interpretability and predictive accuracy of patient subgroup models for reducing the risk of hospital readmission, and beyond. %M 35537203 %R 10.2196/37239 %U https://medinform.jmir.org/2022/12/e37239 %U https://doi.org/10.2196/37239 %U http://www.ncbi.nlm.nih.gov/pubmed/35537203 %0 Journal Article %@ 1929-073X %I JMIR Publications %V 11 %N 2 %P e38655 %T Levels of Autonomous Radiology %A Ghuwalewala,Suraj %A Kulkarni,Viraj %A Pant,Richa %A Kharat,Amit %+ DeepTek Medical Imaging Pvt Ltd, 3rd Floor, Ideas to Impact, 3, Baner Rd, Pallod Farms, Baner, Pune, 411045, India, 91 72760 60080, richa.pant@deeptek.ai %K artificial intelligence %K automation %K machine learning %K radiology %K explainability %K model decay %K generalizability %K fairness and bias %K distributed learning %K autonomous radiology %K AI assistance %D 2022 %7 7.12.2022 %9 Viewpoint %J Interact J Med Res %G English %X Radiology, being one of the younger disciplines of medicine with a history of just over a century, has witnessed tremendous technological advancements and has revolutionized the way we practice medicine today. In the last few decades, medical imaging modalities have generated seismic amounts of medical data. The development and adoption of artificial intelligence applications using this data will lead to the next phase of evolution in radiology. It will include automating laborious manual tasks such as annotations, report generation, etc, along with the initial radiological assessment of patients and imaging features to aid radiologists in their diagnostic and treatment planning workflow. We propose a level-wise classification for the progression of automation in radiology, explaining artificial intelligence assistance at each level with the corresponding challenges and solutions. We hope that such discussions can help us address challenges in a structured way and take the necessary steps to ensure the smooth adoption of new technologies in radiology. %M 36476422 %R 10.2196/38655 %U https://www.i-jmr.org/2022/2/e38655 %U https://doi.org/10.2196/38655 %U http://www.ncbi.nlm.nih.gov/pubmed/36476422 %0 Journal Article %@ 2562-7600 %I JMIR Publications %V 5 %N 1 %P e37562 %T Code-Switching Automatic Speech Recognition for Nursing Record Documentation: System Development and Evaluation %A Hou,Shih-Yen %A Wu,Ya-Lun %A Chen,Kai-Ching %A Chang,Ting-An %A Hsu,Yi-Min %A Chuang,Su-Jung %A Chang,Ying %A Hsu,Kai-Cheng %+ Artificial Intelligence Center for Medical Diagnosis, China Medical University Hospital, No 2, Yude Rd, North Dist, Taichung City, 40459, Taiwan, 886 +886911284382 ext 168, D35842@mail.cmuh.org.tw %K nursing records %K automatic speech recognition %K code-switching %K transfer learning %K meta–transfer learning %D 2022 %7 7.12.2022 %9 Original Paper %J JMIR Nursing %G English %X Background: Taiwan has insufficient nursing resources due to the high turnover rate of health care providers. Therefore, reducing the heavy workload of these employees is essential. Herein, speech transcription, which has various potential clinical applications, was employed for the documentation of nursing records. The requirement of including only one speaker per transcription facilitated data collection and system development. Moreover, authorization from patients was unnecessary. Objective: The aim of this study was to construct a speech recognition system for nursing records such that health care providers can complete nursing records without typing or with only a few edits. Methods: Nursing records in Taiwan are mainly written in Mandarin, with technical terms and abbreviations presented in both Mandarin and English. Therefore, the training set consisted of English code-switching information. Next, transfer learning (TL) and meta-TL (MTL) methods, which perform favorably in code-switching scenarios, were applied. Results: As of September 2021, the China Medical University Hospital Artificial Intelligence Speech (CMaiSpeech) data set was established by manually annotating approximately 100 hours of recordings from 525 speakers. The word error rate (WER) of the benchmark model of syllable-based TL was 29.54% in code-switching. The WER of the proposed model of syllable-based MTL was 22.20% in code-switching. The test set comprised 17,247 words. Moreover, in a clinical case, the proposed model of syllable-based MTL yielded a WER of 31.06% in code-switching. The clinical test set contained 1159 words. Conclusions: This paper has two main contributions. First, the CMaiSpeech data set—a Mandarin-English corpus—has been established. Health care providers in Taiwan are often compelled to use a mixture of Mandarin and English in nursing records. Second, an automatic speech recognition system for nursing record document conversion was proposed. The proposed system can shorten the work handover time and further reduce the workload of health care providers. %M 36476781 %R 10.2196/37562 %U https://nursing.jmir.org/2022/1/e37562 %U https://doi.org/10.2196/37562 %U http://www.ncbi.nlm.nih.gov/pubmed/36476781 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 12 %P e41889 %T Assessment of Clinical Information Quality in Digital Health Technologies: International eDelphi Study %A Fadahunsi,Kayode Philip %A Wark,Petra A %A Mastellos,Nikolaos %A Neves,Ana Luisa %A Gallagher,Joseph %A Majeed,Azeem %A Webster,Andrew %A Smith,Anthony %A Choo-Kang,Brian %A Leon,Catherine %A Edwards,Christopher %A O'Shea,Conor %A Heitz,Elizabeth %A Kayode,Olamide Valentine %A Nash,Makeba %A Kowalski,Martin %A Jiwani,Mateen %A O'Callaghan,Michael Edmund %A Zary,Nabil %A Henderson,Nicola %A Chavannes,Niels H %A Čivljak,Rok %A Olubiyi,Olubunmi Abiola %A Mahapatra,Piyush %A Panday,Rishi Nannan %A Oriji,Sunday O %A Fox,Tatiana Erlikh %A Faint,Victoria %A Car,Josip %+ Department of Primary Care and Public Health, Imperial College London, The Reynolds Building, St Dunstan’s Road, London, W6 8RP, United Kingdom, 44 020 7594 0799, josip.car@imperial.ac.uk %K information quality %K digital health technology %K patient safety %K perspective %K digital health technologies %K DHT %K thematic analysis %K clarity %K understandable %K understandability %K readability %K searchability %K security %K decision support system %K framework development %K framework %D 2022 %7 6.12.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Digital health technologies (DHTs), such as electronic health records and prescribing systems, are transforming health care delivery around the world. The quality of information in DHTs is key to the quality and safety of care. We developed a novel clinical information quality (CLIQ) framework to assess the quality of clinical information in DHTs. Objective: This study explored clinicians’ perspectives on the relevance, definition, and assessment of information quality dimensions in the CLIQ framework. Methods: We used a systematic and iterative eDelphi approach to engage clinicians who had information governance roles or personal interest in information governance; the clinicians were recruited through purposive and snowball sampling techniques. Data were collected using semistructured online questionnaires until consensus was reached on the information quality dimensions in the CLIQ framework. Responses on the relevance of the dimensions were summarized to inform decisions on retention of the dimensions according to prespecified rules. Thematic analysis of the free-text responses was used to revise definitions and the assessment of dimensions. Results: Thirty-five clinicians from 10 countries participated in the study, which was concluded after the second round. Consensus was reached on all dimensions and categories in the CLIQ framework: informativeness (accuracy, completeness, interpretability, plausibility, provenance, and relevance), availability (accessibility, portability, security, and timeliness), and usability (conformance, consistency, and maintainability). A new dimension, searchability, was introduced in the availability category to account for the ease of finding needed information in the DHTs. Certain dimensions were renamed, and some definitions were rephrased to improve clarity. Conclusions: The CLIQ framework reached a high expert consensus and clarity of language relating to the information quality dimensions. The framework can be used by health care managers and institutions as a pragmatic tool for identifying and forestalling information quality problems that could compromise patient safety and quality of care. International Registered Report Identifier (IRRID): RR2-10.1136/bmjopen-2021-057430 %M 36472901 %R 10.2196/41889 %U https://www.jmir.org/2022/12/e41889 %U https://doi.org/10.2196/41889 %U http://www.ncbi.nlm.nih.gov/pubmed/36472901 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 11 %P e42185 %T Artificial Intelligence in Intensive Care Medicine: Bibliometric Analysis %A Tang,Ri %A Zhang,Shuyi %A Ding,Chenling %A Zhu,Mingli %A Gao,Yuan %+ Department of Intensive Care Medicine, Renji Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, No 1630, Dongfang Road, Pudong New District, Shanghai, 200127, China, 86 13917816250, shuishui286@qq.com %K intensive care medicine %K artificial intelligence %K bibliometric analysis %K machine learning %K sepsis %D 2022 %7 30.11.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Interest in critical care–related artificial intelligence (AI) research is growing rapidly. However, the literature is still lacking in comprehensive bibliometric studies that measure and analyze scientific publications globally. Objective: The objective of this study was to assess the global research trends in AI in intensive care medicine based on publication outputs, citations, coauthorships between nations, and co-occurrences of author keywords. Methods: A total of 3619 documents published until March 2022 were retrieved from the Scopus database. After selecting the document type as articles, the titles and abstracts were checked for eligibility. In the final bibliometric study using VOSviewer, 1198 papers were included. The growth rate of publications, preferred journals, leading research countries, international collaborations, and top institutions were computed. Results: The number of publications increased steeply between 2018 and 2022, accounting for 72.53% (869/1198) of all the included papers. The United States and China contributed to approximately 55.17% (661/1198) of the total publications. Of the 15 most productive institutions, 9 were among the top 100 universities worldwide. Detecting clinical deterioration, monitoring, predicting disease progression, mortality, prognosis, and classifying disease phenotypes or subtypes were some of the research hot spots for AI in patients who are critically ill. Neural networks, decision support systems, machine learning, and deep learning were all commonly used AI technologies. Conclusions: This study highlights popular areas in AI research aimed at improving health care in intensive care units, offers a comprehensive look at the research trend in AI application in the intensive care unit, and provides an insight into potential collaboration and prospects for future research. The 30 articles that received the most citations were listed in detail. For AI-based clinical research to be sufficiently convincing for routine critical care practice, collaborative research efforts are needed to increase the maturity and robustness of AI-driven models. %M 36449345 %R 10.2196/42185 %U https://www.jmir.org/2022/11/e42185 %U https://doi.org/10.2196/42185 %U http://www.ncbi.nlm.nih.gov/pubmed/36449345 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 11 %P e37455 %T The Socioeconomic Indicators Linked to Parent Health-Related Technology Use: Cross-sectional Survey %A McCall,Madison P %A Hineline,Megan T %A Anton,Margaret T %A Highlander,April %A Jones,Deborah J %+ Department of Psychology and Neuroscience, University of North Carolina at Chapel Hill, 235 E Cameron Avenue, Chapel Hill, NC, 27599, United States, 1 919 843 2351, mmccall@unc.edu %K parenting %K child %K health behavior %K information seeking %K health-related technology use %K health information %K digital health %K mobile health %K socioeconomic status %K accessibility %D 2022 %7 30.11.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Despite the prevalence of parent health information seeking on the internet and its impact on parenting behavior, there is a paucity of research on parents of young children (ages 3 to 8 years). Given the importance of this developmental period, exploring how family socioeconomic indicators linked to the digital divide and health inequities affect parent proxy- and self-seeking is critical to further understanding variability in health information seeking and associated outcomes. Objective: This study aimed to explore parental health-related technology use (HTU), the process by which parents engage in support, advice, and information-seeking behavior related to their (self-seeking) and their children’s (proxy seeking) health across a range of hardware devices (eg, tablet, wearable, smartphone, laptop, and desktop computer) and sources (eg, search engines, mobile applications, social media, and other digital media). Methods: A cross-sectional study including 313 parents and guardians of children ages 3 to 8 years recruited through Amazon Mechanical Turk (MTurk) was conducted. Parents were asked to complete a self-administered questionnaire on a broad range of parenting and parent-related constructs, including sociodemographic information, technology device ownership, and engagement in and use, features, and perceptions of HTU. Descriptive and bivariate analyses (chi-square tests) were performed to identify patterns and investigate associations between family socioeconomic indicators and parent HTU. Results: The overwhelming majority (301/313, 96%) of parents of young children reported engaging in HTU, of which 99% (300/301) reported using search engines (eg, Google), followed by social media (62%, 188/301), other forms of digital media (eg, podcasts; 145/301, 48%), and mobile applications (114/301, 38%). Parents who engaged in HTU reported seeking information about their child’s behavior and discipline practices (260/313, 83%), mental or physical health (181/313, 58%), and academic performance (142/313, 45%). Additionally, nearly half (134/313, 43%) of parents reported searching for advice on managing their stress. Among parents who reported using each source, an overwhelming majority (280/300, 93%) indicated that search engines were a helpful online source for proxy- and self-seeking, followed by social media (89%, 167/188), other digital media (120/145, 83%), and mobile apps (87/114, 76%). Among parents who reported using any technology source, approximately one-fifth reported that technology sources were most comfortable (61/311, 20%), most understanding (69/311, 22%), and most influential toward behavior change (73/312, 23%) compared to traditional sources of health information–seeking, including mental health professionals, other health care professionals, school professionals, community leaders, friends, and family members. Indicators of family socioeconomic status were differentially associated with frequency and perceptions of and search content associated with parent HTU across technology sources. Conclusions: The findings of this study underscore critical considerations in the design and dissemination of digital resources, programs, and interventions targeting parent and child health, especially for families in traditionally underserved communities. %M 36449346 %R 10.2196/37455 %U https://www.jmir.org/2022/11/e37455 %U https://doi.org/10.2196/37455 %U http://www.ncbi.nlm.nih.gov/pubmed/36449346 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 11 %P e40361 %T The Generation of a Lung Cancer Health Factor Distribution Using Patient Graphs Constructed From Electronic Medical Records: Retrospective Study %A Chen,Anjun %A Huang,Ran %A Wu,Erman %A Han,Ruobing %A Wen,Jian %A Li,Qinghua %A Zhang,Zhiyong %A Shen,Bairong %+ Institutes for System Genetics, West China Hospital, 2222 Xingchuan Road, Chengdu, 610212, China, 86 15995854635, bairong.shen@scu.edu.cn %K lung cancer %K risk factor %K patient graph %K UMLS knowledge graph %K Unified Medical Language System %K connection delta ratio %K EMR %K electronic health record %K EHR %K electronic health record %K cancer %D 2022 %7 25.11.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Electronic medical records (EMRs) of patients with lung cancer (LC) capture a variety of health factors. Understanding the distribution of these factors will help identify key factors for risk prediction in preventive screening for LC. Objective: We aimed to generate an integrated biomedical graph from EMR data and Unified Medical Language System (UMLS) ontology for LC, and to generate an LC health factor distribution from a hospital EMR of approximately 1 million patients. Methods: The data were collected from 2 sets of 1397 patients with and those without LC. A patient-centered health factor graph was plotted with 108,000 standardized data, and a graph database was generated to integrate the graphs of patient health factors and the UMLS ontology. With the patient graph, we calculated the connection delta ratio (CDR) for each of the health factors to measure the relative strength of the factor’s relationship to LC. Results: The patient graph had 93,000 relations between the 2794 patient nodes and 650 factor nodes. An LC graph with 187 related biomedical concepts and 188 horizontal biomedical relations was plotted and linked to the patient graph. Searching the integrated biomedical graph with any number or category of health factors resulted in graphical representations of relationships between patients and factors, while searches using any patient presented the patient’s health factors from the EMR and the LC knowledge graph (KG) from the UMLS in the same graph. Sorting the health factors by CDR in descending order generated a distribution of health factors for LC. The top 70 CDR-ranked factors of disease, symptom, medical history, observation, and laboratory test categories were verified to be concordant with those found in the literature. Conclusions: By collecting standardized data of thousands of patients with and those without LC from the EMR, it was possible to generate a hospital-wide patient-centered health factor graph for graph search and presentation. The patient graph could be integrated with the UMLS KG for LC and thus enable hospitals to bring continuously updated international standard biomedical KGs from the UMLS for clinical use in hospitals. CDR analysis of the graph of patients with LC generated a CDR-sorted distribution of health factors, in which the top CDR-ranked health factors were concordant with the literature. The resulting distribution of LC health factors can be used to help personalize risk evaluation and preventive screening recommendations. %M 36427233 %R 10.2196/40361 %U https://www.jmir.org/2022/11/e40361 %U https://doi.org/10.2196/40361 %U http://www.ncbi.nlm.nih.gov/pubmed/36427233 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 11 %P e38095 %T Medical Text Simplification Using Reinforcement Learning (TESLEA): Deep Learning–Based Text Simplification Approach %A Phatak,Atharva %A Savage,David W %A Ohle,Robert %A Smith,Jonathan %A Mago,Vijay %+ Department of Computer Science, Lakehead University, 955 Oliver Road, Thunder Bay, ON, P7B 5E1, Canada, 1 8073558351, phataka@lakeheadu.ca %K medical text simplification %K reinforcement learning %K natural language processing %K manual evaluation %D 2022 %7 18.11.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: In most cases, the abstracts of articles in the medical domain are publicly available. Although these are accessible by everyone, they are hard to comprehend for a wider audience due to the complex medical vocabulary. Thus, simplifying these complex abstracts is essential to make medical research accessible to the general public. Objective: This study aims to develop a deep learning–based text simplification (TS) approach that converts complex medical text into a simpler version while maintaining the quality of the generated text. Methods: A TS approach using reinforcement learning and transformer–based language models was developed. Relevance reward, Flesch-Kincaid reward, and lexical simplicity reward were optimized to help simplify jargon-dense complex medical paragraphs to their simpler versions while retaining the quality of the text. The model was trained using 3568 complex-simple medical paragraphs and evaluated on 480 paragraphs via the help of automated metrics and human annotation. Results: The proposed method outperformed previous baselines on Flesch-Kincaid scores (11.84) and achieved comparable performance with other baselines when measured using ROUGE-1 (0.39), ROUGE-2 (0.11), and SARI scores (0.40). Manual evaluation showed that percentage agreement between human annotators was more than 70% when factors such as fluency, coherence, and adequacy were considered. Conclusions: A unique medical TS approach is successfully developed that leverages reinforcement learning and accurately simplifies complex medical paragraphs, thereby increasing their readability. The proposed TS approach can be applied to automatically generate simplified text for complex medical text data, which would enhance the accessibility of biomedical research to a wider audience. %M 36399375 %R 10.2196/38095 %U https://medinform.jmir.org/2022/11/e38095 %U https://doi.org/10.2196/38095 %U http://www.ncbi.nlm.nih.gov/pubmed/36399375 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 11 %P e38677 %T Extraction and Quantification of Words Representing Degrees of Diseases: Combining the Fuzzy C-Means Method and Gaussian Membership %A Han,Feng %A Zhang,ZiHeng %A Zhang,Hongjian %A Nakaya,Jun %A Kudo,Kohsuke %A Ogasawara,Katsuhiko %+ Graduate School of Health Sciences, Medical Management and Informatics, Hokkaido University, N12 W5, Sapporo, 060-0812, Japan, 81 011 706 3409, oga@hs.hokudai.ac.jp %K medical text %K fuzzy c-means %K cluster %K algorithm %K machine learning %K word quantification %K fuzzification %K Gauss %K radiology %K medical report %K documentation %K text mining %K data mining %K extraction %K unstructured %K free text %K quantification %K fuzzy %K diagnosis %K diagnostic %K EHR %K support system %D 2022 %7 18.11.2022 %9 Original Paper %J JMIR Form Res %G English %X Background: Due to the development of medical data, a large amount of clinical data has been generated. These unstructured data contain substantial information. Extracting useful knowledge from this data and making scientific decisions for diagnosing and treating diseases have become increasingly necessary. Unstructured data, such as in the Marketplace for Medical Information in Intensive Care III (MIMIC-III) data set, contain several ambiguous words that demonstrate the subjectivity of doctors, such as descriptions of patient symptoms. These data could be used to further improve the accuracy of medical diagnostic system assessments. To the best of our knowledge, there is currently no method for extracting subjective words that express the extent of these symptoms (hereinafter, “degree words”). Objective: Therefore, we propose using the fuzzy c-means (FCM) method and Gaussian membership to quantify the degree words in the clinical medical data set MIMIC-III. Methods: First, we preprocessed the 381,091 radiology reports collected in MIMIC-III, and then we used the FCM method to extract degree words from unstructured text. Thereafter, we used the Gaussian membership method to quantify the extracted degree words, which transform the fuzzy words extracted from the medical text into computer-recognizable numbers. Results: The results showed that the digitization of ambiguous words in medical texts is feasible. The words representing each degree of each disease had a range of corresponding values. Examples of membership medians were 2.971 (atelectasis), 3.121 (pneumonia), 2.899 (pneumothorax), 3.051 (pulmonary edema), and 2.435 (pulmonary embolus). Additionally, all extracted words contained the same subjective words (low, high, etc), which allows for an objective evaluation method. Furthermore, we will verify the specific impact of the quantification results of ambiguous words such as symptom words and degree words on the use of medical texts in subsequent studies. These same ambiguous words may be used as a new set of feature values to represent the disorders. Conclusions: This study proposes an innovative method for handling subjective words. We used the FCM method to extract the subjective degree words in the English-interpreted report of the MIMIC-III and then used the Gaussian functions to quantify the subjective degree words. In this method, words containing subjectivity in unstructured texts can be automatically processed and transformed into numerical ranges by digital processing. It was concluded that the digitization of ambiguous words in medical texts is feasible. %M 36399376 %R 10.2196/38677 %U https://formative.jmir.org/2022/11/e38677 %U https://doi.org/10.2196/38677 %U http://www.ncbi.nlm.nih.gov/pubmed/36399376 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 11 %P e42955 %T Postpartum Migraine Headache Coding in Electronic Health Records of a Large Integrated Health Care System: Validation Study %A Shi,Jiaxiao %A Fassett,Michael J %A Chiu,Vicki Y %A Avila,Chantal C %A Khadka,Nehaa %A Brown,Brittany %A Patel,Pooja %A Mensah,Nana %A Xie,Fagen %A Peltier,Morgan R %A Getahun,Darios %+ Department of Research and Evaluation, Kaiser Permanente Southern California, 100 S Los Robles Avenue, Pasadena, CA, 91101, United States, 1 626 564 5658, Darios.T.Getahun@kp.org %K migraine headache %K validation %K diagnosis %K pharmacy %K postpartum %K medical record %K health plan %K electronic health record %K coding %K pharmacy record %K diagnostic code %K EHR system %D 2022 %7 17.11.2022 %9 Original Paper %J JMIR Form Res %G English %X Background: Migraine is a common neurological disorder characterized by repeated headaches of varying intensity. The prevalence and severity of migraine headaches disproportionally affects women, particularly during the postpartum period. Moreover, migraines during pregnancy have been associated with adverse maternal outcomes, including preeclampsia and postpartum stroke. However, due to the lack of a validated instrument for uniform case ascertainment on postpartum migraine headache, there is uncertainty in the reported prevalence in the literature. Objective: The aim of this study was to evaluate the completeness and accuracy of reporting postpartum migraine headache coding in a large integrated health care system’s electronic health records (EHRs) and to compare the coding quality before and after the implementation of the International Classification of Diseases, 10th revision, Clinical Modification (ICD-10-CM) codes and pharmacy records in EHRs. Methods: Medical records of 200 deliveries in all 15 Kaiser Permanente Southern California hospitals during 2 time periods, that is, January 1, 2012 through December 31, 2014 (International Classification of Diseases, 9th revision, Clinical Modification [ICD-9-CM] coding period) and January 1, 2017 through December 31, 2019 (ICD-10-CM coding period), were randomly selected from EHRs for chart review. Two trained research associates reviewed the EHRs for all 200 women for postpartum migraine headache cases documented within 1 year after delivery. Women were considered to have postpartum migraine headache if either a mention of migraine headache (yes for diagnosis) or a prescription for treatment of migraine headache (yes for pharmacy records) was noted in the electronic chart. Results from the chart abstraction served as the gold standard and were compared with corresponding diagnosis and pharmacy prescription utilization records for both ICD-9-CM and ICD-10-CM coding periods through comparisons of sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), as well as the summary statistics of F-score and Youden J statistic (J). The kappa statistic (κ) for interrater reliability was calculated. Results: The overall agreement between the identification of migraine headache using diagnosis codes and pharmacy records compared to the medical record review was strong. Diagnosis coding (F-score=87.8%; J=82.5%) did better than pharmacy records (F-score=72.7%; J=57.5%) when identifying cases, but combining both of these sources of data produced much greater accuracy in the identification of postpartum migraine cases (F-score=96.9%; J=99.7%) with sensitivity, specificity, PPV, and NPV of 100%, 99.7%, 93.9%, and 100%, respectively. Results were similar across the ICD-9-CM (F-score=98.7%, J=99.9%) and ICD-10-CM coding periods (F-score=94.9%; J=99.6%). The interrater reliability between the 2 research associates for postpartum migraine headache was 100%. Conclusions: Neither diagnostic codes nor pharmacy records alone are sufficient for identifying postpartum migraine cases reliably, but when used together, they are quite reliable. The completeness of the data remained similar after the implementation of the ICD-10-CM coding in the EHR system. %M 36394937 %R 10.2196/42955 %U https://formative.jmir.org/2022/11/e42955 %U https://doi.org/10.2196/42955 %U http://www.ncbi.nlm.nih.gov/pubmed/36394937 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 11 %N 11 %P e39536 %T Developing an Artificial Intelligence Model for Reading Chest X-rays: Protocol for a Prospective Validation Study %A Miró Catalina,Queralt %A Fuster-Casanovas,Aïna %A Solé-Casals,Jordi %A Vidal-Alaball,Josep %+ Unitat de Suport a la Recerca de la Catalunya Central, Fundació Institut Universitari per a la Recerca a l'Atenció Primària de Salut Jordi Gol i Gurina, C/ Pica d'Estats 13-15, Sant Fruitós de Bages, 08272, Spain, 34 634810263, qmiro.cc.ics@gencat.cat %K artificial intelligence %K machine learning %K chest x-ray %K radiology %K validation %D 2022 %7 16.11.2022 %9 Protocol %J JMIR Res Protoc %G English %X Background: Chest x-rays are the most commonly used type of x-rays today, accounting for up to 26% of all radiographic tests performed. However, chest radiography is a complex imaging modality to interpret. Several studies have reported discrepancies in chest x-ray interpretations among emergency physicians and radiologists. It is of vital importance to be able to offer a fast and reliable diagnosis for this kind of x-ray, using artificial intelligence (AI) to support the clinician. Oxipit has developed an AI algorithm for reading chest x-rays, available through a web platform called ChestEye. This platform is an automatic computer-aided diagnosis system where a reading of the inserted chest x-ray is performed, and an automatic report is returned with a capacity to detect 75 pathologies, covering 90% of diagnoses. Objective: The overall objective of the study is to perform validation with prospective data of the ChestEye algorithm as a diagnostic aid. We wish to validate the algorithm for a single pathology and multiple pathologies by evaluating the accuracy, sensitivity, and specificity of the algorithm. Methods: A prospective validation study will be carried out to compare the diagnosis of the reference radiologists for the users attending the primary care center in the Osona region (Spain), with the diagnosis of the ChestEye AI algorithm. Anonymized chest x-ray images will be acquired and fed into the AI algorithm interface, which will return an automatic report. A radiologist will evaluate the same chest x-ray, and both assessments will be compared to calculate the precision, sensitivity, specificity, and accuracy of the AI algorithm. Results will be represented globally and individually for each pathology using a confusion matrix and the One-vs-All methodology. Results: Patient recruitment was conducted from February 7, 2022, and it is expected that data can be obtained in 5 to 6 months. In June 2022, more than 450 x-rays have been collected, so it is expected that 600 samples will be gathered in July 2022. We hope to obtain sufficient evidence to demonstrate that the use of AI in the reading of chest x-rays can be a good tool for diagnostic support. However, there is a decreasing number of radiology professionals and, therefore, it is necessary to develop and validate tools to support professionals who have to interpret these tests. Conclusions: If the results of the validation of the model are satisfactory, it could be implemented as a support tool and allow an increase in the accuracy and speed of diagnosis, patient safety, and agility in the primary care system, while reducing the cost of unnecessary tests. International Registered Report Identifier (IRRID): PRR1-10.2196/39536 %M 36383419 %R 10.2196/39536 %U https://www.researchprotocols.org/2022/11/e39536 %U https://doi.org/10.2196/39536 %U http://www.ncbi.nlm.nih.gov/pubmed/36383419 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 11 %P e37976 %T Automatic Estimation of the Most Likely Drug Combination in Electronic Health Records Using the Smooth Algorithm: Development and Validation Study %A Ouchi,Dan %A Giner-Soriano,Maria %A Gómez-Lumbreras,Ainhoa %A Vedia Urgell,Cristina %A Torres,Ferran %A Morros,Rosa %+ Fundació Institut Universitari per a la recerca a l'Atenció Primària de Salut Jordi Gol i Gurina, Gran Via de les Corts Catalanes 587, àtic, Barcelona, 08007, Spain, 34 934824110, douchi@idiapjgol.info %K electronic health records %K data mining %K complex drug patterns %K algorithms %K drug utilization %K polypharmacy %K EHR %K medication %K drug combination %K therapy %K automation %K drug exposition %K treatment %K adherence %D 2022 %7 15.11.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Since the use of electronic health records (EHRs) in an automated way, pharmacovigilance or pharmacoepidemiology studies have been used to characterize the therapy using different algorithms. Although progress has been made in this area for monotherapy, with combinations of 2 or more drugs the challenge to characterize the treatment increases significantly, and more research is needed. Objective: The goal of the research was to develop and describe a novel algorithm that automatically returns the most likely therapy of one drug or combinations of 2 or more drugs over time. Methods: We used the Information System for Research in Primary Care as our reference EHR platform for the smooth algorithm development. The algorithm was inspired by statistical methods based on moving averages and depends on a parameter Wt, a flexible window that determines the level of smoothing. The effect of Wt was evaluated in a simulation study on the same data set with different window lengths. To understand the algorithm performance in a clinical or pharmacological perspective, we conducted a validation study. We designed 4 pharmacological scenarios and asked 4 independent professionals to compare a traditional method against the smooth algorithm. Data from the simulation and validation studies were then analyzed. Results: The Wt parameter had an impact over the raw data. As we increased the window length, more patient were modified and the number of smoothed patients augmented, although we rarely observed changes of more than 5% of the total data. In the validation study, significant differences were obtained in the performance of the smooth algorithm over the traditional method. These differences were consistent across pharmacological scenarios. Conclusions: The smooth algorithm is an automated approach that standardizes, simplifies, and improves data processing in drug exposition studies using EHRs. This algorithm can be generalized to almost any pharmacological medication and model the drug exposure to facilitate the detection of treatment switches, discontinuations, and terminations throughout the study period. %M 36378514 %R 10.2196/37976 %U https://medinform.jmir.org/2022/11/e37976 %U https://doi.org/10.2196/37976 %U http://www.ncbi.nlm.nih.gov/pubmed/36378514 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 11 %P e40124 %T Effects of Hospital Digitization on Clinical Outcomes and Patient Satisfaction: Nationwide Multiple Regression Analysis Across German Hospitals %A von Wedel,Philip %A Hagist,Christian %A Liebe,Jan-David %A Esdar,Moritz %A Hübner,Ursula %A Pross,Christoph %+ Chair of Economic and Social Policy, WHU - Otto Beisheim School of Management, Burgplatz 2, Vallendar, 56179, Germany, 49 02616509 ext 255, philip.wedel@whu.edu %K health care information technology %K electronic health records %K hospital digitization %K quality of care %K clinical outcomes %K patient satisfaction %K user-perceived value %D 2022 %7 10.11.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: The adoption of health information technology (HIT) by health care providers is commonly believed to improve the quality of care. Policy makers in the United States and Germany follow this logic and deploy nationwide HIT adoption programs to fund hospital investments in digital technologies. However, scientific evidence for the beneficial effects of HIT on care quality at a national level remains mostly US based, is focused on electronic health records (EHRs), and rarely accounts for the quality of digitization from a hospital user perspective. Objective: This study aimed to examine the effects of digitization on clinical outcomes and patient experience in German hospitals. Hence, this study adds to the small stream of literature in this field outside the United States. It goes beyond assessing the effects of mere HIT adoption and also considers user-perceived HIT value. In addition, the impact of a variety of technologies beyond EHRs was examined. Methods: Multiple linear regression models were estimated using emergency care outcomes, elective care outcomes, and patient satisfaction as dependent variables. The adoption and user-perceived value of HIT represented key independent variables, and case volume, hospital size, ownership status, and teaching status were included as controls. Care outcomes were captured via risk-adjusted, observed-to-expected outcome ratios for patients who had stroke, myocardial infarction, or hip replacement. The German Patient Experience Questionnaire of Weisse Liste provided information on patient satisfaction. Information on the adoption and user-perceived value of 10 subdomains of HIT and EHRs was derived from the German 2020 Healthcare IT Report. Results: Statistical analysis was based on an overall sample of 383 German hospitals. The analyzed data set suggested no significant effect of HIT or EHR adoption on clinical outcomes or patient satisfaction. However, a higher user-perceived value or quality of the installed tools did improve outcomes. Emergency care outcomes benefited from user-friendly overall digitization (β=−.032; P=.04), which was especially driven by the user-friendliness of admission HIT (β=−.023; P=.07). Elective care outcomes were positively impacted by user-friendly EHR installations (β=−.138; P=.008). Similarly, the results suggested user-friendly, overall digitization to have a moderate positive effect on patient satisfaction (β=−.009; P=.01). Conclusions: The results of this study suggest that hospital digitization is not an end in itself. Policy makers and hospitals are well advised to not only focus on the mere adoption of digital technologies but also continuously work toward digitization that is perceived as valuable by physicians and nurses who rely on it every day. Furthermore, hospital digitization strategies should consider that the assumed benefits of single technologies are not realized across all care domains. %M 36355423 %R 10.2196/40124 %U https://www.jmir.org/2022/11/e40124 %U https://doi.org/10.2196/40124 %U http://www.ncbi.nlm.nih.gov/pubmed/36355423 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 11 %P e37945 %T The Real-World Experiences of Persons With Multiple Sclerosis During the First COVID-19 Lockdown: Application of Natural Language Processing %A Chiavi,Deborah %A Haag,Christina %A Chan,Andrew %A Kamm,Christian Philipp %A Sieber,Chloé %A Stanikić,Mina %A Rodgers,Stephanie %A Pot,Caroline %A Kesselring,Jürg %A Salmen,Anke %A Rapold,Irene %A Calabrese,Pasquale %A Manjaly,Zina-Mary %A Gobbi,Claudio %A Zecca,Chiara %A Walther,Sebastian %A Stegmayer,Katharina %A Hoepner,Robert %A Puhan,Milo %A von Wyl,Viktor %+ Epidemiology, Biostatistics and Prevention Institute, University of Zurich, Hirschengraben 84, Zurich, 8001, Switzerland, 41 44 63 46380, viktor.vonwyl@uzh.ch %K natural language processing %K multiple sclerosis %K COVID-19 %K nervous system disease %K nervous system disorder %K textual data %K health data %K patient data %K topic modeling %K sentiment analysis %K linguistic inquiry %K medical informatics %K clinical informatics %D 2022 %7 10.11.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: The increasing availability of “real-world” data in the form of written text holds promise for deepening our understanding of societal and health-related challenges. Textual data constitute a rich source of information, allowing the capture of lived experiences through a broad range of different sources of information (eg, content and emotional tone). Interviews are the “gold standard” for gaining qualitative insights into individual experiences and perspectives. However, conducting interviews on a large scale is not always feasible, and standardized quantitative assessment suitable for large-scale application may miss important information. Surveys that include open-text assessments can combine the advantages of both methods and are well suited for the application of natural language processing (NLP) methods. While innovations in NLP have made large-scale text analysis more accessible, the analysis of real-world textual data is still complex and requires several consecutive steps. Objective: We developed and subsequently examined the utility and scientific value of an NLP pipeline for extracting real-world experiences from textual data to provide guidance for applied researchers. Methods: We applied the NLP pipeline to large-scale textual data collected by the Swiss Multiple Sclerosis (MS) registry. Such textual data constitute an ideal use case for the study of real-world text data. Specifically, we examined 639 text reports on the experienced impact of the first COVID-19 lockdown from the perspectives of persons with MS. The pipeline has been implemented in Python and complemented by analyses of the “Linguistic Inquiry and Word Count” software. It consists of the following 5 interconnected analysis steps: (1) text preprocessing; (2) sentiment analysis; (3) descriptive text analysis; (4) unsupervised learning–topic modeling; and (5) results interpretation and validation. Results: A topic modeling analysis identified the following 4 distinct groups based on the topics participants were mainly concerned with: “contacts/communication;” “social environment;” “work;” and “errands/daily routines.” Notably, the sentiment analysis revealed that the “contacts/communication” group was characterized by a pronounced negative emotional tone underlying the text reports. This observed heterogeneity in emotional tonality underlying the reported experiences of the first COVID-19–related lockdown is likely to reflect differences in emotional burden, individual circumstances, and ways of coping with the pandemic, which is in line with previous research on this matter. Conclusions: This study illustrates the timely and efficient applicability of an NLP pipeline and thereby serves as a precedent for applied researchers. Our study thereby contributes to both the dissemination of NLP techniques in applied health sciences and the identification of previously unknown experiences and burdens of persons with MS during the pandemic, which may be relevant for future treatment. %M 36252126 %R 10.2196/37945 %U https://medinform.jmir.org/2022/11/e37945 %U https://doi.org/10.2196/37945 %U http://www.ncbi.nlm.nih.gov/pubmed/36252126 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 11 %P e35709 %T The Application of Graph Theoretical Analysis to Complex Networks in Medical Malpractice in China: Qualitative Study %A Dong,Shengjie %A Shi,Chenshu %A Zeng,Wu %A Jia,Zhiying %A Dong,Minye %A Xiao,Yuyin %A Li,Guohong %+ School of Public Health, Shanghai Jiao Tong University, No.227 South Chongqing Road, Huangpu District, Shanghai, 200025, China, 86 21 63846590, guohongli@sjtu.edu.cn %K medical malpractice %K complex network %K scale-free network %K hub nodes %K patient safety management %K health systems %D 2022 %7 3.11.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Studies have shown that hospitals or physicians with multiple malpractice claims are more likely to be involved in new claims. This finding indicates that medical malpractice may be clustered by institutions. Objective: We aimed to identify the underlying mechanisms of medical malpractice that, in the long term, may contribute to developing interventions to reduce future claims and patient harm. Methods: This study extracted the semantic network in 6610 medical litigation records (unstructured data) obtained from a public judicial database in China. They represented the most serious cases of malpractice in the country. The medical malpractice network of China was presented as a knowledge graph based on the complex network theory; it uses the International Classification of Patient Safety from the World Health Organization as a reference. Results: We found that the medical malpractice network of China was a scale-free network—the occurrence of medical malpractice in litigation cases was not random, but traceable. The results of the hub nodes revealed that orthopedics, obstetrics and gynecology, and the emergency department were the 3 most frequent specialties that incurred malpractice; inadequate informed consent work constituted the most errors. Nontechnical errors (eg, inadequate informed consent) showed a higher centrality than technical errors. Conclusions: Hospitals and medical boards could apply our approach to detect hub nodes that are likely to benefit from interventions; doing so could effectively control medical risks. %M 36326815 %R 10.2196/35709 %U https://medinform.jmir.org/2022/11/e35709 %U https://doi.org/10.2196/35709 %U http://www.ncbi.nlm.nih.gov/pubmed/36326815 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 10 %P e38640 %T Tooth-Related Disease Detection System Based on Panoramic Images and Optimization Through Automation: Development Study %A Kim,Changgyun %A Jeong,Hogul %A Park,Wonse %A Kim,Donghyun %+ AI Cloud R&D Center, InVisionLab Inc, G114, 128, Beobwon-ro, Songpa-gu, Seoul, 05854, Republic of Korea, 82 70 4415 2229, rari98@naver.com %K object detection %K tooth %K diagnosis %K panorama %K dentistry %K dental health %K oral health %K dental caries %K image analysis %K artificial intelligence %K detection model %K machine learning %K automation %K diagnosis system %D 2022 %7 31.10.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Early detection of tooth-related diseases in patients plays a key role in maintaining their dental health and preventing future complications. Since dentists are not overly attentive to tooth-related diseases that may be difficult to judge visually, many patients miss timely treatment. The 5 representative tooth-related diseases, that is, coronal caries or defect, proximal caries, cervical caries or abrasion, periapical radiolucency, and residual root can be detected on panoramic images. In this study, a web service was constructed for the detection of these diseases on panoramic images in real time, which helped shorten the treatment planning time and reduce the probability of misdiagnosis. Objective: This study designed a model to assess tooth-related diseases in panoramic images by using artificial intelligence in real time. This model can perform an auxiliary role in the diagnosis of tooth-related diseases by dentists and reduce the treatment planning time spent through telemedicine. Methods: For learning the 5 tooth-related diseases, 10,000 panoramic images were modeled: 4206 coronal caries or defects, 4478 proximal caries, 6920 cervical caries or abrasion, 8290 periapical radiolucencies, and 1446 residual roots. To learn the model, the fast region-based convolutional network (Fast R-CNN), residual neural network (ResNet), and inception models were used. Learning about the 5 tooth-related diseases completely did not provide accurate information on the diseases because of indistinct features present in the panoramic pictures. Therefore, 1 detection model was applied to each tooth-related disease, and the models for each of the diseases were integrated to increase accuracy. Results: The Fast R-CNN model showed the highest accuracy, with an accuracy of over 90%, in diagnosing the 5 tooth-related diseases. Thus, Fast R-CNN was selected as the final judgment model as it facilitated the real-time diagnosis of dental diseases that are difficult to judge visually from radiographs and images, thereby assisting the dentists in their treatment plans. Conclusions: The Fast R-CNN model showed the highest accuracy in the real-time diagnosis of dental diseases and can therefore play an auxiliary role in shortening the treatment planning time after the dentists diagnose the tooth-related disease. In addition, by updating the captured panoramic images of patients on the web service developed in this study, we are looking forward to increasing the accuracy of diagnosing these 5 tooth-related diseases. The dental diagnosis system in this study takes 2 minutes for diagnosing 5 diseases in 1 panoramic image. Therefore, this system plays an effective role in setting a dental treatment schedule. %M 36315222 %R 10.2196/38640 %U https://medinform.jmir.org/2022/10/e38640 %U https://doi.org/10.2196/38640 %U http://www.ncbi.nlm.nih.gov/pubmed/36315222 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 10 %P e39616 %T Evaluating the Impact on Clinical Task Efficiency of a Natural Language Processing Algorithm for Searching Medical Documents: Prospective Crossover Study %A Park,Eunsoo H %A Watson,Hannah I %A Mehendale,Felicity V %A O'Neil,Alison Q %A , %+ Edinburgh Medical School, College of Medicine and Veterinary Medicine, University of Edinburgh, The Chancellor's Building, 49 Little France Crescent, Edinburgh, EH16 4SB, United Kingdom, 44 1312426792, e.park-7@sms.ed.ac.uk %K clinical decision support %K electronic health records %K natural language processing %K semantic search %K clinical informatics %D 2022 %7 26.10.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Information retrieval (IR) from the free text within electronic health records (EHRs) is time consuming and complex. We hypothesize that natural language processing (NLP)–enhanced search functionality for EHRs can make clinical workflows more efficient and reduce cognitive load for clinicians. Objective: This study aimed to evaluate the efficacy of 3 levels of search functionality (no search, string search, and NLP-enhanced search) in supporting IR for clinical users from the free text of EHR documents in a simulated clinical environment. Methods: A clinical environment was simulated by uploading 3 sets of patient notes into an EHR research software application and presenting these alongside 3 corresponding IR tasks. Tasks contained a mixture of multiple-choice and free-text questions. A prospective crossover study design was used, for which 3 groups of evaluators were recruited, which comprised doctors (n=19) and medical students (n=16). Evaluators performed the 3 tasks using each of the search functionalities in an order in accordance with their randomly assigned group. The speed and accuracy of task completion were measured and analyzed, and user perceptions of NLP-enhanced search were reviewed in a feedback survey. Results: NLP-enhanced search facilitated more accurate task completion than both string search (5.14%; P=.02) and no search (5.13%; P=.08). NLP-enhanced search and string search facilitated similar task speeds, both showing an increase in speed compared to the no search function, by 11.5% (P=.008) and 16.0% (P=.007) respectively. Overall, 93% of evaluators agreed that NLP-enhanced search would make clinical workflows more efficient than string search, with qualitative feedback reporting that NLP-enhanced search reduced cognitive load. Conclusions: To the best of our knowledge, this study is the largest evaluation to date of different search functionalities for supporting target clinical users in realistic clinical workflows, with a 3-way prospective crossover study design. NLP-enhanced search improved both accuracy and speed of clinical EHR IR tasks compared to browsing clinical notes without search. NLP-enhanced search improved accuracy and reduced the number of searches required for clinical EHR IR tasks compared to direct search term matching. %M 36287591 %R 10.2196/39616 %U https://medinform.jmir.org/2022/10/e39616 %U https://doi.org/10.2196/39616 %U http://www.ncbi.nlm.nih.gov/pubmed/36287591 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 10 %P e38041 %T Visualization Techniques of Time-Oriented Data for the Comparison of Single Patients With Multiple Patients or Cohorts: Scoping Review %A Scheer,Jan %A Volkert,Alisa %A Brich,Nicolas %A Weinert,Lina %A Santhanam,Nandhini %A Krone,Michael %A Ganslandt,Thomas %A Boeker,Martin %A Nagel,Till %+ Human Data Interaction Lab, Mannheim University of Applied Sciences, Paul-Wittsack-Straße 10, Mannheim, 68163, Germany, 49 621 292 6886, t.nagel@hs-mannheim.de %K patient data %K comparison %K visualization systems %K visual analytics %K information visualization %K cohorts %K multiple patients %K single patients %K time-oriented data %D 2022 %7 24.10.2022 %9 Review %J J Med Internet Res %G English %X Background: Visual analysis and data delivery in the form of visualizations are of great importance in health care, as such forms of presentation can reduce errors and improve care and can also help provide new insights into long-term disease progression. Information visualization and visual analytics also address the complexity of long-term, time-oriented patient data by reducing inherent complexity and facilitating a focus on underlying and hidden patterns. Objective: This review aims to provide an overview of visualization techniques for time-oriented data in health care, supporting the comparison of patients. We systematically collected literature and report on the visualization techniques supporting the comparison of time-based data sets of single patients with those of multiple patients or their cohorts and summarized the use of these techniques. Methods: This scoping review used the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist. After all collected articles were screened by 16 reviewers according to the criteria, 6 reviewers extracted the set of variables under investigation. The characteristics of these variables were based on existing taxonomies or identified through open coding. Results: Of the 249 screened articles, we identified 22 (8.8%) that fit all criteria and reviewed them in depth. We collected and synthesized findings from these articles for medical aspects such as medical context, medical objective, and medical data type, as well as for the core investigated aspects of visualization techniques, interaction techniques, and supported tasks. The extracted articles were published between 2003 and 2019 and were mostly situated in clinical research. These systems used a wide range of visualization techniques, most frequently showing changes over time. Timelines and temporal line charts occurred 8 times each, followed by histograms with 7 occurrences and scatterplots with 5 occurrences. We report on the findings quantitatively through visual summarization, as well as qualitatively. Conclusions: The articles under review in general mitigated complexity through visualization and supported diverse medical objectives. We identified 3 distinct patient entities: single patients, multiple patients, and cohorts. Cohorts were typically visualized in condensed form, either through prior data aggregation or through visual summarization, whereas visualization of individual patients often contained finer details. All the systems provided mechanisms for viewing and comparing patient data. However, explicitly comparing a single patient with multiple patients or a cohort was supported only by a few systems. These systems mainly use basic visualization techniques, with some using novel visualizations tailored to a specific task. Overall, we found the visual comparison of measurements between single and multiple patients or cohorts to be underdeveloped, and we argue for further research in a systematic review, as well as the usefulness of a design space. %M 36279164 %R 10.2196/38041 %U https://www.jmir.org/2022/10/e38041 %U https://doi.org/10.2196/38041 %U http://www.ncbi.nlm.nih.gov/pubmed/36279164 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 11 %N 10 %P e39193 %T Medication Adherence and Cardiometabolic Control Indicators Among American Indian Adults Receiving Tribal Health Services: Protocol for a Longitudinal Electronic Health Records Study %A Scarton,Lisa %A Nelson,Tarah %A Yao,Yingwei %A Segal,Richard %A Donahoo,William T %A Goins,R Turner %A DeVaughan-Circles,Ashley %A Manson,Spero M %A Wilkie,Diana J %+ College of Nursing, University of Florida, 1225 Center Dr, Gainesville, FL, 32607, United States, 1 352 274 6417, lscarton@ufl.edu %K medication adherence %K American Indian %K type 2 diabetes %D 2022 %7 24.10.2022 %9 Protocol %J JMIR Res Protoc %G English %X Background: American Indian adults have the highest prevalence of type 2 diabetes (T2D) in any racial or ethnic group and experience high rates of comorbidities. Uncontrolled cardiometabolic risk factors—insulin resistance, resulting in impaired glucose tolerance, dyslipidemia, and hypertension—increase the risk of mortality. Mortality is significantly reduced by glucose- and lipid-lowering and antihypertensive medication adherence. Medication adherence is low among American Indian adults living in non–Indian Health Service health care settings. Virtually nothing is known about the nature and extent of medication adherence among reservation-dwelling American Indian adults who primarily receive their medications without cost from Indian Health Service or tribal facilities. Electronic health records (EHRs) offer a rich but underused data source regarding medication adherence and its potential to predict cardiometabolic control indicators (C-MCIs). With the support of the Choctaw Nation of Oklahoma (CNO), we address this oversight by using EHR data generated by this large, state-of-the-art tribal health care system to investigate C-MCIs. Objective: Our specific aims are to determine, using 2018 EHR data, the bivariate relationships between medication adherence and C-MCIs, demographics, and comorbidities and each C-MCI and demographics and comorbidities; develop machine learning models for predicting future C-MCIs from the previous year’s medication adherence, demographics, comorbidities, and common laboratory tests; and identify facilitators of and barriers to medication adherence within the context of social determinants of health (SDOH), EHR-derived medication adherence, and C-MCIs. Methods: Drawing on the tribe’s EHR (2018-2021) data for CNO patients with T2D, we will characterize the relationships among medication adherence (to glucose- and lipid-lowering and antihypertensive drugs) and C-MCIs (hemoglobin A1c ≤7%, low-density lipoprotein cholesterol <100 mg/dL, and systolic blood pressure <130 mm Hg); patient demographics (eg, age, sex, SDOH, and residence location); and comorbidities (eg, BMI ≥30, cardiovascular disease, and chronic kidney disease). We will also characterize the association of each C-MCI with demographics and comorbidities. Prescription and pharmacy refill data will be used to calculate the proportion of days covered with medications, a typical measure of medication adherence. Using machine learning techniques, we will develop prediction models for future (2019-2021) C-MCIs based on medication adherence, patient demographics, comorbidities, and common laboratory tests (eg, lipid panel) from the previous year. Finally, key informant interviews (N=90) will explore facilitators of and barriers to medication adherence within the context of local SDOH. Results: Funding was obtained in early 2022. The University of Florida and CNO approved the institutional review board protocols and executed the data use agreements. Data extraction is in process. We expect to obtain results from aims 1 and 2 in 2024. Conclusions: Our findings will yield insights into improving medication adherence and C-MCIs among American Indian adults, consistent with CNO’s State of the Nation’s Health Report 2017 goal of reducing T2D and its complications. International Registered Report Identifier (IRRID): PRR1-10.2196/39193 %M 36279173 %R 10.2196/39193 %U https://www.researchprotocols.org/2022/10/e39193 %U https://doi.org/10.2196/39193 %U http://www.ncbi.nlm.nih.gov/pubmed/36279173 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 10 %P e35860 %T Characterizing Thrombotic Complication Risk Factors Associated With COVID-19 via Heterogeneous Patient Data: Retrospective Observational Study %A Rosario,Bedda %A Zhang,Andrew %A Patel,Mehool %A Rajmane,Amol %A Xie,Ning %A Weeraratne,Dilhan %A Alterovitz,Gil %+ Biomedical Cybernetics Laboratory, Brigham and Women's Hospital, 75 Francis Street, Boston, MA, 02115, United States, 1 617 329 1445, ga@alum.mit.edu %K COVID-19 %K thrombotic complications %K logistic regression %K EHR %K electronic health record %K insurance claims data %D 2022 %7 21.10.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: COVID-19 has been observed to be associated with venous and arterial thrombosis. The inflammatory disease prolongs hospitalization, and preexisting comorbidities can intensity the thrombotic burden in patients with COVID-19. However, venous thromboembolism, arterial thrombosis, and other vascular complications may go unnoticed in critical care settings. Early risk stratification is paramount in the COVID-19 patient population for proactive monitoring of thrombotic complications. Objective: The aim of this exploratory research was to characterize thrombotic complication risk factors associated with COVID-19 using information from electronic health record (EHR) and insurance claims databases. The goal is to develop an approach for analysis using real-world data evidence that can be generalized to characterize thrombotic complications and additional conditions in other clinical settings as well, such as pneumonia or acute respiratory distress syndrome in COVID-19 patients or in the intensive care unit. Methods: We extracted deidentified patient data from the insurance claims database IBM MarketScan, and formulated hypotheses on thrombotic complications in patients with COVID-19 with respect to patient demographic and clinical factors using logistic regression. The hypotheses were then verified with analysis of deidentified patient data from the Research Patient Data Registry (RPDR) Mass General Brigham (MGB) patient EHR database. Data were analyzed according to odds ratios, 95% CIs, and P values. Results: The analysis identified significant predictors (P<.001) for thrombotic complications in 184,831 COVID-19 patients out of the millions of records from IBM MarketScan and the MGB RPDR. With respect to age groups, patients 60 years and older had higher odds (4.866 in MarketScan and 6.357 in RPDR) to have thrombotic complications than those under 60 years old. In terms of gender, men were more likely (odds ratio of 1.245 in MarketScan and 1.693 in RPDR) to have thrombotic complications than women. Among the preexisting comorbidities, patients with heart disease, cerebrovascular diseases, hypertension, and personal history of thrombosis all had significantly higher odds of developing a thrombotic complication. Cancer and obesity were also associated with odds>1. The results from RPDR validated the IBM MarketScan findings, as they were largely consistent and afford mutual enrichment. Conclusions: The analysis approach adopted in this study can work across heterogeneous databases from diverse organizations and thus facilitates collaboration. Searching through millions of patient records, the analysis helped to identify factors influencing a phenotype. Use of thrombotic complications in COVID-19 patients represents only a case study; however, the same design can be used across other disease areas by extracting corresponding disease-specific patient data from available databases. %M 36044652 %R 10.2196/35860 %U https://www.jmir.org/2022/10/e35860 %U https://doi.org/10.2196/35860 %U http://www.ncbi.nlm.nih.gov/pubmed/36044652 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 10 %P e38557 %T Lifting Hospital Electronic Health Record Data Treasures: Challenges and Opportunities %A Maletzky,Alexander %A Böck,Carl %A Tschoellitsch,Thomas %A Roland,Theresa %A Ludwig,Helga %A Thumfart,Stefan %A Giretzlehner,Michael %A Hochreiter,Sepp %A Meier,Jens %+ Research Department Medical Informatics, RISC Software GmbH, Softwarepark 32a, Hagenberg, 4232, Austria, 43 7236 93028406, alexander.maletzky@risc-software.at %K electronic health record %K medical data preparation %K machine learning %K retrospective data analysis %D 2022 %7 21.10.2022 %9 Viewpoint %J JMIR Med Inform %G English %X Electronic health records (EHRs) have been successfully used in data science and machine learning projects. However, most of these data are collected for clinical use rather than for retrospective analysis. This means that researchers typically face many different issues when attempting to access and prepare the data for secondary use. We aimed to investigate how raw EHRs can be accessed and prepared in retrospective data science projects in a disciplined, effective, and efficient way. We report our experience and findings from a large-scale data science project analyzing routinely acquired retrospective data from the Kepler University Hospital in Linz, Austria. The project involved data collection from more than 150,000 patients over a period of 10 years. It included diverse data modalities, such as static demographic data, irregularly acquired laboratory test results, regularly sampled vital signs, and high-frequency physiological waveform signals. Raw medical data can be corrupted in many unexpected ways that demand thorough manual inspection and highly individualized data cleaning solutions. We present a general data preparation workflow, which was shaped in the course of our project and consists of the following 7 steps: obtain a rough overview of the available EHR data, define clinically meaningful labels for supervised learning, extract relevant data from the hospital’s data warehouses, match data extracted from different sources, deidentify them, detect errors and inconsistencies therein through a careful exploratory analysis, and implement a suitable data processing pipeline in actual code. Only few of the data preparation issues encountered in our project were addressed by generic medical data preprocessing tools that have been proposed recently. Instead, highly individualized solutions for the specific data used in one’s own research seem inevitable. We believe that the proposed workflow can serve as a guidance for practitioners, helping them to identify and address potential problems early and avoid some common pitfalls. %M 36269654 %R 10.2196/38557 %U https://medinform.jmir.org/2022/10/e38557 %U https://doi.org/10.2196/38557 %U http://www.ncbi.nlm.nih.gov/pubmed/36269654 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 4 %P e36976 %T Desired Characteristics of a Clinical Decision Support System for Early Sepsis Recognition: Interview Study Among Hospital-Based Clinicians %A Silvestri,Jasmine A %A Kmiec,Tyler E %A Bishop,Nicholas S %A Regli,Susan H %A Weissman,Gary E %+ Palliative and Advanced Illness Research Center, University of Pennsylvania Perelman School of Medicine, 300 Blockley Hall, 423 Guardian Drive, Philadelphia, PA, 19104, United States, 1 215 746 2887, gary.weissman@pennmedicine.upenn.edu %K sepsis %K predictive information %K clinical decision support %K human factors %K sepsis onset %D 2022 %7 21.10.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Sepsis is a major burden for health care systems in the United States, with over 750,000 cases annually and a total cost of approximately US $20 billion. The hallmark of sepsis treatment is early and appropriate initiation of antibiotic therapy. Although sepsis clinical decision support (CDS) systems can provide clinicians with early predictions of suspected sepsis or imminent clinical decline, such systems have not reliably demonstrated improvements in clinical outcomes or care processes. Growing evidence suggests that the challenges of integrating sepsis CDS systems into clinical workflows, gaining the trust of clinicians, and making sepsis CDS systems clinically relevant at the bedside are all obstacles to successful deployment. However, there are significant knowledge gaps regarding the achievement of these implementation and deployment goals. Objective: We aimed to identify perceptions of predictive information in sepsis CDS systems based on clinicians’ past experiences, explore clinicians’ perceptions of a hypothetical sepsis CDS system, and identify the characteristics of a CDS system that would be helpful in promoting timely recognition and management of suspected sepsis in a multidisciplinary, team-based clinical setting. Methods: We conducted semistructured interviews with practicing bedside nurses, advanced practice providers, and physicians at a large academic medical center between September 2020 and March 2021. We used modified human factor methods (contextual interview and cognitive walkthrough performed over video calls because of the COVID-19 pandemic) and conducted a thematic analysis using an abductive approach for coding to identify important patterns and concepts in the interview transcripts. Results: We interviewed 6 bedside nurses and 9 clinicians responsible for ordering antibiotics (advanced practice providers or physicians) who had a median of 4 (IQR 4-6.5) years of experience working in an inpatient setting. We then synthesized critical content from the thematic analysis of the data into four domains: clinician perceptions of prediction models and alerts; previous experiences of clinician encounters with predictive information and risk scores; desired characteristics of a CDS system build, including predictions, supporting information, and delivery methods for a potential alert; and the clinical relevance and potential utility of a CDS system. These 4 domains were strongly linked to clinicians’ perceptions of the likelihood of adoption and the impact on clinical workflows when diagnosing and managing patients with suspected sepsis. Ultimately, clinicians desired a trusted and actionable CDS system to improve sepsis care. Conclusions: Building a trusted and actionable sepsis CDS alert is paramount to achieving acceptability and use among clinicians. These findings can inform the development, implementation, and deployment strategies for CDS systems that support the early detection and treatment of sepsis. This study also highlights several key opportunities when eliciting clinician input before the development and deployment of prediction models. %M 36269653 %R 10.2196/36976 %U https://humanfactors.jmir.org/2022/4/e36976 %U https://doi.org/10.2196/36976 %U http://www.ncbi.nlm.nih.gov/pubmed/36269653 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 10 %P e41136 %T Relation Extraction in Biomedical Texts Based on Multi-Head Attention Model With Syntactic Dependency Feature: Modeling Study %A Li,Yongbin %A Hui,Linhu %A Zou,Liping %A Li,Huyang %A Xu,Luo %A Wang,Xiaohua %A Chua,Stephanie %+ School of Medical Information Engineering, Zunyi Medical University, 6 Xuefu Road West, Xinpu New District, Zunyi, 563000, China, 86 18311545098, bynn456@126.com %K biomedical relation extraction %K deep learning %K feature combination %K multi-head attention %K additive attention %K syntactic dependency feature %K syntactic dependency graph %K shortest dependency path %D 2022 %7 20.10.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: With the rapid expansion of biomedical literature, biomedical information extraction has attracted increasing attention from researchers. In particular, relation extraction between 2 entities is a long-term research topic. Objective: This study aimed to perform 2 multiclass relation extraction tasks of Biomedical Natural Language Processing Workshop 2019 Open Shared Tasks: relation extraction of Bacteria-Biotope (BB-rel) task and binary relation extraction of plant seed development (SeeDev-binary) task. In essence, these 2 tasks are aimed at extracting the relation between annotated entity pairs from biomedical texts, which is a challenging problem. Methods: Traditional research methods adopted feature- or kernel-based methods and achieved good performance. For these tasks, we propose a deep learning model based on a combination of several distributed features, such as domain-specific word embedding, part-of-speech embedding, entity-type embedding, distance embedding, and position embedding. The multi-head attention mechanism is used to extract the global semantic features of an entire sentence. Meanwhile, we introduced a dependency-type feature and the shortest dependency path connecting 2 candidate entities in the syntactic dependency graph to enrich the feature representation. Results: Experiments show that our proposed model has excellent performance in biomedical relation extraction, achieving F1 scores of 65.56% and 38.04% on the test sets of the BB-rel and SeeDev-binary tasks. Especially in the SeeDev-binary task, the F1 score of our model is superior to that of other existing models and achieves state-of-the-art performance. Conclusions: We demonstrated that the multi-head attention mechanism can learn relevant syntactic and semantic features in different representation subspaces and different positions to extract comprehensive feature representation. Moreover, syntactic dependency features can improve the performance of the model by learning dependency relation between the entities in biomedical texts. %M 36264604 %R 10.2196/41136 %U https://medinform.jmir.org/2022/10/e41136 %U https://doi.org/10.2196/41136 %U http://www.ncbi.nlm.nih.gov/pubmed/36264604 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 10 %P e40876 %T An Increase in Antibiotic Prescribing for Respiratory Tract Infections Through Telehealth Consultations: Retrospective Study in Australian General Practice %A Imai,Chisato %A Amin,Janaki %A Prgomet,Mirela %A Pearce,Christopher %A Georgiou,Andrew %+ Australian Institute of Health Innovation, Macquarie University, ​75 ​Talavera Road, Level 6, Sydney, 2109, Australia, 61 02 9850 2415, chrissy.imai@mq.edu.au %K general practice %K anti-infective agents %K antibiotics %K medication %K prescriptions %K respiratory tract infections %K infection %K telehealth %K telemedicine %K Australia %D 2022 %7 18.10.2022 %9 Research Letter %J J Med Internet Res %G English %X %M 36256826 %R 10.2196/40876 %U https://www.jmir.org/2022/10/e40876 %U https://doi.org/10.2196/40876 %U http://www.ncbi.nlm.nih.gov/pubmed/36256826 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 10 %P e38936 %T Standardized Description of the Feature Extraction Process to Transform Raw Data Into Meaningful Information for Enhancing Data Reuse: Consensus Study %A Lamer,Antoine %A Fruchart,Mathilde %A Paris,Nicolas %A Popoff,Benjamin %A Payen,Anaïs %A Balcaen,Thibaut %A Gacquer,William %A Bouzillé,Guillaume %A Cuggia,Marc %A Doutreligne,Matthieu %A Chazard,Emmanuel %+ Univ. Lille, CHU Lille, ULR 2694 - METRICS: Évaluation des Technologies de santé et des Pratiques médicales, 1 place de Verdun, Lille, 59000, France, 33 320626969, antoine.lamer@univ-lille.fr %K feature extraction %K data reuse %K data warehouse %K database %K algorithm %K Observation Medical Outcomes Partnership %D 2022 %7 17.10.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Despite the many opportunities data reuse offers, its implementation presents many difficulties, and raw data cannot be reused directly. Information is not always directly available in the source database and needs to be computed afterwards with raw data for defining an algorithm. Objective: The main purpose of this article is to present a standardized description of the steps and transformations required during the feature extraction process when conducting retrospective observational studies. A secondary objective is to identify how the features could be stored in the schema of a data warehouse. Methods: This study involved the following 3 main steps: (1) the collection of relevant study cases related to feature extraction and based on the automatic and secondary use of data; (2) the standardized description of raw data, steps, and transformations, which were common to the study cases; and (3) the identification of an appropriate table to store the features in the Observation Medical Outcomes Partnership (OMOP) common data model (CDM). Results: We interviewed 10 researchers from 3 French university hospitals and a national institution, who were involved in 8 retrospective and observational studies. Based on these studies, 2 states (track and feature) and 2 transformations (track definition and track aggregation) emerged. “Track” is a time-dependent signal or period of interest, defined by a statistical unit, a value, and 2 milestones (a start event and an end event). “Feature” is time-independent high-level information with dimensionality identical to the statistical unit of the study, defined by a label and a value. The time dimension has become implicit in the value or name of the variable. We propose the 2 tables “TRACK” and “FEATURE” to store variables obtained in feature extraction and extend the OMOP CDM. Conclusions: We propose a standardized description of the feature extraction process. The process combined the 2 steps of track definition and track aggregation. By dividing the feature extraction into these 2 steps, difficulty was managed during track definition. The standardization of tracks requires great expertise with regard to the data, but allows the application of an infinite number of complex transformations. On the contrary, track aggregation is a very simple operation with a finite number of possibilities. A complete description of these steps could enhance the reproducibility of retrospective studies. %M 36251369 %R 10.2196/38936 %U https://medinform.jmir.org/2022/10/e38936 %U https://doi.org/10.2196/38936 %U http://www.ncbi.nlm.nih.gov/pubmed/36251369 %0 Journal Article %@ 2561-6722 %I JMIR Publications %V 5 %N 4 %P e38879 %T Assessing the Racial and Socioeconomic Disparities in Postpartum Depression Using Population-Level Hospital Discharge Data: Longitudinal Retrospective Study %A Liu,Star %A Ding,Xiyu %A Belouali,Anas %A Bai,Haibin %A Raja,Kanimozhi %A Kharrazi,Hadi %+ Johns Hopkins University School of Medicine, 2024 E Monument St. S 1-200, Baltimore, MD, 21205, United States, 1 470 538 5974, sliu197@jhmi.edu %K health disparity %K hospital discharge summary %K phenotyping %K data quality %K vulnerable population %K postpartum depression %K maternal health %D 2022 %7 17.10.2022 %9 Original Paper %J JMIR Pediatr Parent %G English %X Background: In the United States, >3.6 million deliveries occur annually. Among them, up to 20% (approximately 700,000) of women experience postpartum depression (PPD) according to the Centers for Disease Control and Prevention. Absence of accurate reporting and diagnosis has made phenotyping of patients with PPD difficult. Existing literature has shown that factors such as race, socioeconomic status, and history of substance abuse are associated with the differential risks of PPD. However, limited research has considered differential temporal associations with the outcome. Objective: This study aimed to estimate the disparities in the risk of PPD and time to diagnosis for patients of different racial and socioeconomic backgrounds. Methods: This is a longitudinal retrospective study using the statewide hospital discharge data from Maryland. We identified 160,066 individuals who had a hospital delivery from 2017 to 2019. We applied logistic regression and Cox regression to study the risk of PPD across racial and socioeconomic strata. Multinomial regression was used to estimate the risk of PPD at different postpartum stages. Results: The cumulative incidence of PPD diagnosis was highest for White patients (8779/65,028, 13.5%) and lowest for Asian and Pacific Islander patients (248/10,760, 2.3%). Compared with White patients, PPD diagnosis was less likely to occur for Black patients (odds ratio [OR] 0.31, 95% CI 0.30-0.33), Asian or Pacific Islander patients (OR 0.17, 95% CI 0.15-0.19), and Hispanic patients (OR 0.21, 95% CI 0.19-0.22). Similar findings were observed from the Cox regression analysis. Multinomial regression showed that compared with White patients, Black patients (relative risk 2.12, 95% CI 1.73-2.60) and Asian and Pacific Islander patients (relative risk 2.48, 95% CI 1.46-4.21) were more likely to be diagnosed with PPD after 8 weeks of delivery. Conclusions: Compared with White patients, PPD diagnosis is less likely to occur in individuals of other races. We found disparate timing in PPD diagnosis across different racial groups and socioeconomic backgrounds. Our findings serve to enhance intervention strategies and policies for phenotyping patients at the highest risk of PPD and to highlight needs in data quality to support future work on racial disparities in PPD. %M 36103575 %R 10.2196/38879 %U https://pediatrics.jmir.org/2022/4/e38879 %U https://doi.org/10.2196/38879 %U http://www.ncbi.nlm.nih.gov/pubmed/36103575 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 10 %P e42429 %T Fast Healthcare Interoperability Resources for Inpatient Deterioration Detection With Time-Series Vital Signs: Design and Implementation Study %A Tseng,Tzu-Wei %A Su,Chang-Fu %A Lai,Feipei %+ Department of Computer Science and Information Engineering, National Taiwan University, No. 1, Sec. 4, Roosevelt Rd., Taipei City, 106319, Taiwan, 886 963079621, chaaa463@gmail.com %K Fast Healthcare Interoperability Resources %K FHIR %K Health Level 7 %K HL7 %K health research %K data sharing %K health information technology %K clinical research %D 2022 %7 13.10.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Vital signs have been widely adopted in in-hospital cardiac arrest (IHCA) assessment, which plays an important role in inpatient deterioration detection. As the number of early warning systems and artificial intelligence applications increases, health care information exchange and interoperability are becoming more complex and difficult. Although Health Level 7 Fast Healthcare Interoperability Resources (FHIR) have already developed a vital signs profile, it is not sufficient to support IHCA applications or machine learning–based models. Objective: In this paper, for IHCA instances with vital signs, we define a new implementation guide that includes data mapping, a system architecture, a workflow, and FHIR applications. Methods: We interviewed 10 experts regarding health care system integration and defined an implementation guide. We then developed the FHIR Extract Transform Load to map data to FHIR resources. We also integrated an early warning system and machine learning pipeline. Results: The study data set includes electronic health records of adult inpatients who visited the En-Chu-Kong hospital. Medical staff regularly measured these vital signs at least 2 to 3 times per day during the day, night, and early morning. We used pseudonymization to protect patient privacy. Then, we converted the vital signs to FHIR observations in the JSON format using the FHIR Extract Transform Load application. The measured vital signs include systolic blood pressure, diastolic blood pressure, heart rate, respiratory rate, and body temperature. According to clinical requirements, we also extracted the electronic health record information to the FHIR server. Finally, we integrated an early warning system and machine learning pipeline using the FHIR RESTful application programming interface. Conclusions: We successfully demonstrated a process that standardizes health care information for inpatient deterioration detection using vital signs. Based on the FHIR definition, we also provided an implementation guide that includes data mapping, an integration process, and IHCA assessment using vital signs. We also proposed a clarifying system architecture and possible workflows. Based on FHIR, we integrated the 3 different systems in 1 dashboard system, which can effectively solve the complexity of the system in the medical staff workflow. %M 36227636 %R 10.2196/42429 %U https://medinform.jmir.org/2022/10/e42429 %U https://doi.org/10.2196/42429 %U http://www.ncbi.nlm.nih.gov/pubmed/36227636 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 10 %P e41503 %T Coronary Artery Computed Tomography Angiography for Preventing Cardio-Cerebrovascular Disease: Observational Cohort Study Using the Observational Health Data Sciences and Informatics’ Common Data Model %A Bae,Woo Kyung %A Cho,Jihoon %A Kim,Seok %A Kim,Borham %A Baek,Hyunyoung %A Song,Wongeun %A Yoo,Sooyoung %+ Healthcare Information and Communication Technology Research Center, Office of eHealth Research and Business, Seoul National University Bundang Hospital, Republic of Korea, 172, Dolma-ro, Bundang-gu, Seongnam-si, 13605, Republic of Korea, 82 010 9053 7094, yoosoo0@snubh.org %K cardiovascular diseases %K coronary artery computed tomography angiography %K observational study %K common data model %K population level estimation %K cardiology %K vascular disease %K medical informatics %K computed tomography %K angiography %K electronic health record %K risk score %K health data science %K data modeling %D 2022 %7 13.10.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Cardio-cerebrovascular diseases (CVDs) result in 17.5 million deaths annually worldwide, accounting for 46.2% of noncommunicable causes of death, and are the leading cause of death, followed by cancer, respiratory disease, and diabetes mellitus. Coronary artery computed tomography angiography (CCTA), which detects calcification in the coronary arteries, can be used to detect asymptomatic but serious vascular disease. It allows for noninvasive and quick testing despite involving radiation exposure. Objective: The objective of our study was to investigate the effectiveness of CCTA screening on CVD outcomes by using the Observational Health Data Sciences and Informatics’ Observational Medical Outcomes Partnership Common Data Model (OMOP-CDM) data and the population-level estimation method. Methods: Using electronic health record–based OMOP-CDM data, including health questionnaire responses, adults (aged 30-74 years) without a history of CVD were selected, and 5-year CVD outcomes were compared between patients undergoing CCTA (target group) and a comparison group via 1:1 propensity score matching. Participants were stratified into low-risk and high-risk groups based on the American College of Cardiology/American Heart Association atherosclerotic cardiovascular disease (ASCVD) risk score and Framingham risk score (FRS) for subgroup analyses. Results: The 2-year and 5-year risk scores were compared as secondary outcomes between the two groups. In total, 8787 participants were included in both the target group and comparison group. No significant differences (calibration P=.37) were found between the hazard ratios of the groups at 5 years. The subgroup analysis also revealed no significant differences between the ASCVD risk scores and FRSs of the groups at 5 years (ASCVD risk score: P=.97; FRS: P=.85). However, the CCTA group showed a significantly lower increase in risk scores at 2 years (ASCVD risk score: P=.03; FRS: P=.02). Conclusions: Although we could not confirm a significant difference in the preventive effects of CCTA screening for CVDs over a long period of 5 years, it may have a beneficial effect on risk score management over 2 years. %M 36227638 %R 10.2196/41503 %U https://medinform.jmir.org/2022/10/e41503 %U https://doi.org/10.2196/41503 %U http://www.ncbi.nlm.nih.gov/pubmed/36227638 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 10 %P e39187 %T A Recurrent Neural Network Model for Predicting Activated Partial Thromboplastin Time After Treatment With Heparin: Retrospective Study %A Boie,Sebastian Daniel %A Engelhardt,Lilian Jo %A Coenen,Nicolas %A Giesa,Niklas %A Rubarth,Kerstin %A Menk,Mario %A Balzer,Felix %+ Charité – Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Institute of Medical Informatics, Charitéplatz 1, Berlin, 10117, Germany, 49 30 450580877, Sebastian-Daniel.Boie@charite.de %K machine learning %K health care %K recurrent neural network %K heparin %K activated partial thromboplastin time (aPTT) %K deep learning %K ICU %K critical care %D 2022 %7 13.10.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Anticoagulation therapy with heparin is a frequent treatment in intensive care units and is monitored by activated partial thromboplastin clotting time (aPTT). It has been demonstrated that reaching an established anticoagulation target within 24 hours is associated with favorable outcomes. However, patients respond to heparin differently and reaching the anticoagulation target can be challenging. Machine learning algorithms may potentially support clinicians with improved dosing recommendations. Objective: This study evaluates a range of machine learning algorithms on their capability of predicting the patients’ response to heparin treatment. In this analysis, we apply, for the first time, a model that considers time series. Methods: We extracted patient demographics, laboratory values, dialysis and extracorporeal membrane oxygenation treatments, and scores from the hospital information system. We predicted the numerical values of aPTT laboratory values 24 hours after continuous heparin infusion and evaluated 7 different machine learning models. The best-performing model was compared to recently published models on a classification task. We considered all data before and within the first 12 hours of continuous heparin infusion as features and predicted the aPTT value after 24 hours. Results: The distribution of aPTT in our cohort of 5926 hospital admissions was highly skewed. Most patients showed aPTT values below 75 s, while some outliers showed much higher aPTT values. A recurrent neural network that consumes a time series of features showed the highest performance on the test set. Conclusions: A recurrent neural network that uses time series of features instead of only static and aggregated features showed the highest performance in predicting aPTT after heparin treatment. %M 36227653 %R 10.2196/39187 %U https://medinform.jmir.org/2022/10/e39187 %U https://doi.org/10.2196/39187 %U http://www.ncbi.nlm.nih.gov/pubmed/36227653 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 10 %P e40344 %T Successful Integration of EN/ISO 13606–Standardized Extracts From a Patient Mobile App Into an Electronic Health Record: Description of a Methodology %A Frid,Santiago %A Fuentes Expósito,Maria Angeles %A Grau-Corral,Inmaculada %A Amat-Fernandez,Clara %A Muñoz Mateu,Montserrat %A Pastor Duran,Xavier %A Lozano-Rubí,Raimundo %+ Medical Informatics Unit, Hospital Clínic de Barcelona, Villarroel 170, Barcelona, 08036, Spain, 34 932 27 54 00 ext 3344, santifrid@gmail.com %K health information interoperability %K mobile app %K health information standards %K artificial intelligence %K electronic health records %K machine learning %D 2022 %7 12.10.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: There is an increasing need to integrate patient-generated health data (PGHD) into health information systems (HISs). The use of health information standards based on the dual model allows the achievement of semantic interoperability among systems. Although there is evidence in the use of the Substitutable Medical Applications and Reusable Technologies on Fast Healthcare Interoperability Resources (SMART on FHIR) framework for standardized communication between mobile apps and electronic health records (EHRs), the use of European Norm/International Organization for Standardization (EN/ISO) 13606 has not been explored yet, despite some advantages over FHIR in terms of modeling and formalization of clinical knowledge, as well as flexibility in the creation of new concepts. Objective: This study aims to design and implement a methodology based on the dual-model paradigm to communicate clinical information between a patient mobile app (Xemio Research) and an institutional ontology-based clinical repository (OntoCR) without loss of meaning. Methods: This paper is framed within Artificial intelligence Supporting CAncer Patients across Europe (ASCAPE), a project that aims to use artificial intelligence (AI)/machine learning (ML) mechanisms to support cancer patients’ health status and quality of life (QoL). First, the variables “side effect” and “daily steps” were defined and represented with EN/ISO 13606 archetypes. Next, ontologies that model archetyped concepts and map them to the standard were created and uploaded to OntoCR, where they were ready to receive instantiated patient data. Xemio Research used a conversion module in the ASCAPE Local Edge to transform data entered into the app to create EN/ISO 13606 extracts, which were sent to an Application Programming Interface (API) in OntoCR that maps each element in the normalized XML files to its corresponding location in the ontology. This way, instantiated data of patients are stored in the clinical repository. Results: Between December 22, 2020, and April 4, 2022, 1100 extracts of 47 patients were successfully communicated (234/1100, 21.3%, extracts of side effects and 866/1100, 78.7%, extracts of daily activity). Furthermore, the creation of EN/ISO 13606–standardized archetypes allows the reuse of clinical information regarding daily activity and side effects, while with the creation of ontologies, we extended the knowledge representation of our clinical repository. Conclusions: Health information interoperability is one of the requirements for continuity of health care. The dual model allows the separation of knowledge and information in HISs. EN/ISO 13606 was chosen for this project because of the operational mechanisms it offers for data exchange, as well as its flexibility for modeling knowledge and creating new concepts. To the best of our knowledge, this is the first experience reported in the literature of effective communication of EN/ISO 13606 EHR extracts between a patient mobile app and an institutional clinical repository using a scalable standard-agnostic methodology that can be applied to other projects, data sources, and institutions. %M 36222792 %R 10.2196/40344 %U https://medinform.jmir.org/2022/10/e40344 %U https://doi.org/10.2196/40344 %U http://www.ncbi.nlm.nih.gov/pubmed/36222792 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 4 %P e39102 %T Answering Hospital Caregivers’ Questions at Any Time: Proof-of-Concept Study of an Artificial Intelligence–Based Chatbot in a French Hospital %A Daniel,Thomas %A de Chevigny,Alix %A Champrigaud,Adeline %A Valette,Julie %A Sitbon,Marine %A Jardin,Meryam %A Chevalier,Delphine %A Renet,Sophie %+ Department of Pharmacy, Paris Saint-Joseph Hospital Group, 185 Raymond Losserand Street, Paris, 75014, France, 33 144127191, srenet@ghpsj.fr %K chatbot %K artificial intelligence %K pharmacy %K hospital %K health care %K drugs %K medication %K information quality %K health information %K caregiver %K healthcare staff %K digital health tool %K COVID-19 %K information technology %D 2022 %7 11.10.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Access to accurate information in health care is a key point for caregivers to avoid medication errors, especially with the reorganization of staff and drug circuits during health crises such as the COVID‑19 pandemic. It is, therefore, the role of the hospital pharmacy to answer caregivers’ questions. Some may require the expertise of a pharmacist, some should be answered by pharmacy technicians, but others are simple and redundant, and automated responses may be provided. Objective: We aimed at developing and implementing a chatbot to answer questions from hospital caregivers about drugs and pharmacy organization 24 hours a day and to evaluate this tool. Methods: The ADDIE (Analysis, Design, Development, Implementation, and Evaluation) model was used by a multiprofessional team composed of 3 hospital pharmacists, 2 members of the Innovation and Transformation Department, and the IT service provider. Based on an analysis of the caregivers’ needs about drugs and pharmacy organization, we designed and developed a chatbot. The tool was then evaluated before its implementation into the hospital intranet. Its relevance and conversations with testers were monitored via the IT provider’s back office. Results: Needs analysis with 5 hospital pharmacists and 33 caregivers from 5 health services allowed us to identify 7 themes about drugs and pharmacy organization (such as opening hours and specific prescriptions). After a year of chatbot design and development, the test version obtained good evaluation scores: its speed was rated 8.2 out of 10, usability 8.1 out of 10, and appearance 7.5 out of 10. Testers were generally satisfied (70%) and were hoping for the content to be enhanced. Conclusions: The chatbot seems to be a relevant tool for hospital caregivers, helping them obtain reliable and verified information they need on drugs and pharmacy organization. In the context of significant mobility of nursing staff during the health crisis due to the COVID-19 pandemic, the chatbot could be a suitable tool for transmitting relevant information related to drug circuits or specific procedures. To our knowledge, this is the first time that such a tool has been designed for caregivers. Its development further continued by means of tests conducted with other users such as pharmacy technicians and via the integration of additional data before the implementation on the 2 hospital sites. %M 35930555 %R 10.2196/39102 %U https://humanfactors.jmir.org/2022/4/e39102 %U https://doi.org/10.2196/39102 %U http://www.ncbi.nlm.nih.gov/pubmed/35930555 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 4 %P e37905 %T The Introduction of Robotics to an Outpatient Dispensing and Medication Management Process in Saudi Arabia: Retrospective Review of a Pharmacy-led Multidisciplinary Six Sigma Performance Improvement Project %A Al Nemari,Manal %A Waterson,James %+ Medication Management Solutions, Medical Affairs, Becton Dickinson, 11F Blue Bay Tower, Business Bay, Dubai, 52279, United Arab Emirates, 971 0566035154, james.waterson@bd.com %K inventory waste %K mislabeling events %K no-show returns %K inventory stock levels %K staff education %K task realignment %K outpatient %K Six Sigma %K medication management %K medication adherence %K risk %K pharmacy %K health care professional %K dispensing %K robotics %K automation %K pharmaceuticals %K inventory %D 2022 %7 11.10.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Outpatient pharmacy management aims for improved patient safety, improved quality of service, and cost reduction. The Six Sigma method improves quality by eliminating variability, with the goal of a nearly error-free process. Automation of pharmacy tasks potentially offers greater efficiency and safety. Objective: The goal was to measure the impact that integration of automation made to service, safety and efficiency, staff reallocation and reorientation, and workflow in the outpatient pharmacy department. The Six Sigma problem definition to be resolved was as follows: The current system of outpatient dispensing denies quality to patients in terms of waiting time and contact time with pharmacy professionals, incorporates risks to the patient in terms of mislabeling of medications and the incomplete dispensing of prescriptions, and is potentially wasteful in terms of time and resources. Methods: We described the process of introducing automation to a large outpatient pharmacy department in a university hospital. The Six Sigma approach was used as it focuses on continuous improvement and also produces a road map that integrates tracking and monitoring into its process. A review of activity in the outpatient department focused on non-value-added (NVA) pharmacist tasks, improving the patient experience and patient safety. Metrics to measure the impact of change were established, and a process map analysis with turnaround times (TATs) for each stage of service was created. Discrete events were selected for correction, improvement, or mitigation. From the review, the team selected key outcome metrics, including storage, picking and delivery dispensing rates, patient and prescription load per day, average packs and lines per prescription, and lines held. Our goal was total automation of stock management. We deployed 2 robotic dispensing units to feed 9 dispensing desks. The automated units were integrated with hospital information technology (HIT) that supports appointments, medication records, and prescriptions. Results: Postautomation, the total patient time in the department, including the time interacting with the pharmacist for medication education and counseling, dropped from 17.093 to 11.812 digital minutes, with an appreciable increase in patient-pharmacist time. The percentage of incomplete prescriptions dispensed versus orders decreased from 3.0% to 1.83%. The dispensing error rate dropped from 1.00% to 0.24%. Assessed via a “basket” of medications, wastage cost was reduced by 83.9%. During implementation, it was found that NVA tasks that were replaced by automated processes were responsible for an extensive loss of pharmacist time. The productivity ratio postautomation was 1.26. Conclusions: The Six Sigma methodology allowed for rapid transformation of the medication management process. The risk priority numbers (RPNs) for the “wrong patient-wrong medication error” reduced by a ratio of 5.25:1 and for “patient leaves unit with inadequate counseling” postautomation by 2.5:1. Automation allowed for ring-fencing of patient-pharmacist time. This time needs to be structured for optimal effectiveness. %M 36222805 %R 10.2196/37905 %U https://humanfactors.jmir.org/2022/4/e37905 %U https://doi.org/10.2196/37905 %U http://www.ncbi.nlm.nih.gov/pubmed/36222805 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 10 %P e36313 %T The Factors Contributing to Physicians’ Current Use of and Satisfaction With Electronic Health Records in Kuwait’s Public Health Care: Cross-sectional Questionnaire Study %A Al-Otaibi,Jawaher %A Tolma,Eleni %A Alali,Walid %A Alhuwail,Dari %A Aljunid,Syed Mohamed %+ Department of Health Policy and Management, College of Public Health, Kuwait University, Sabah Al-Salem University City, Kuwait Ministry of Health Bldg, Medical Record Jamal Abdel Nasser Street, Sulaibikat Club Circle, Al-ASima area, Kuwait City, 12009, Kuwait, 965 55615073, jawaher309m@gmail.com %K health informatics %K information systems adoption %K electronic health record %K EHR %K public health informatics %D 2022 %7 7.10.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Electronic health record (EHR) has emerged as a backbone health care organization that aims to integrate health care records and automate clinical workflow. With the adoption of the eHealth care system, health information communication technologies and EHRs are offering significant health care advantages in the form of error reduction, improved communication, and patient satisfaction. Objective: This study aimed to (1) investigate factors associated with physicians’ EHR adoption status and prevalence of EHRs in Kuwait and (2) identify factors predicting physician satisfaction with EHRs in public hospitals in Kuwait. Methods: This study was conducted at Kuwait’s public Al-Jahra hospital from May to September 2019, using quantitative research methods. Primary data were gathered via questionnaires distributed among 295 physicians recruited using convenience sampling. Data were analyzed in SPSS using descriptive, bivariate, and multivariate linear regression, adjusted for demographics. Results: Results of the study revealed that the controlled variable of gender (β=–.197; P=.02) along with explanatory variables, such as training quality (β=.068; P=.005), perception of barriers (β=–.107; P=.04), and effect on physician (β=.521; P<.001) have a significant statistical relationship with physicians’ EHR adoption status. Furthermore, findings also suggested that controlled variables of gender (β=–.193; P=.02), education (β=–.164; P=.03), effect on physician (β=.417; P<.001), and level of ease of use (β=.254; P<.001) are significant predictors of the degree of physician satisfaction with the EHR system. Conclusions: The findings of this study had significant managerial and practical implications for creating an inductive environment for the acceptance of EHR systems across a broad spectrum of health care system in Kuwait. %M 36206039 %R 10.2196/36313 %U https://medinform.jmir.org/2022/10/e36313 %U https://doi.org/10.2196/36313 %U http://www.ncbi.nlm.nih.gov/pubmed/36206039 %0 Journal Article %@ 2563-3570 %I JMIR Publications %V 3 %N 1 %P e36660 %T Multiple-Inputs Convolutional Neural Network for COVID-19 Classification and Critical Region Screening From Chest X-ray Radiographs: Model Development and Performance Evaluation %A Li,Zhongqiang %A Li,Zheng %A Yao,Luke %A Chen,Qing %A Zhang,Jian %A Li,Xin %A Feng,Ji-Ming %A Li,Yanping %A Xu,Jian %+ Division of Electrical and Computer Engineering, College of Engineering, Louisiana State University, Patrick F Taylor Hall, 3304 S Quad Dr, Baton Rouge, LA, 70803, United States, 1 (225) 578 4483, jianxu1@lsu.edu %K COVID-19 %K chest X-ray radiography %K multiple-inputs convolutional neural network %K screening critical COVID regions %D 2022 %7 4.10.2022 %9 Original Paper %J JMIR Bioinform Biotech %G English %X Background: The COVID-19 pandemic is becoming one of the largest, unprecedented health crises, and chest X-ray radiography (CXR) plays a vital role in diagnosing COVID-19. However, extracting and finding useful image features from CXRs demand a heavy workload for radiologists. Objective: The aim of this study was to design a novel multiple-inputs (MI) convolutional neural network (CNN) for the classification of COVID-19 and extraction of critical regions from CXRs. We also investigated the effect of the number of inputs on the performance of our new MI-CNN model. Methods: A total of 6205 CXR images (including 3021 COVID-19 CXRs and 3184 normal CXRs) were used to test our MI-CNN models. CXRs could be evenly segmented into different numbers (2, 4, and 16) of individual regions. Each region could individually serve as one of the MI-CNN inputs. The CNN features of these MI-CNN inputs would then be fused for COVID-19 classification. More importantly, the contributions of each CXR region could be evaluated through assessing the number of images that were accurately classified by their corresponding regions in the testing data sets. Results: In both the whole-image and left- and right-lung region of interest (LR-ROI) data sets, MI-CNNs demonstrated good efficiency for COVID-19 classification. In particular, MI-CNNs with more inputs (2-, 4-, and 16-input MI-CNNs) had better efficiency in recognizing COVID-19 CXRs than the 1-input CNN. Compared to the whole-image data sets, the efficiency of LR-ROI data sets showed approximately 4% lower accuracy, sensitivity, specificity, and precision (over 91%). In considering the contributions of each region, one of the possible reasons for this reduced performance was that nonlung regions (eg, region 16) provided false-positive contributions to COVID-19 classification. The MI-CNN with the LR-ROI data set could provide a more accurate evaluation of the contribution of each region and COVID-19 classification. Additionally, the right-lung regions had higher contributions to the classification of COVID-19 CXRs, whereas the left-lung regions had higher contributions to identifying normal CXRs. Conclusions: Overall, MI-CNNs could achieve higher accuracy with an increasing number of inputs (eg, 16-input MI-CNN). This approach could assist radiologists in identifying COVID-19 CXRs and in screening the critical regions related to COVID-19 classifications. %M 36277075 %R 10.2196/36660 %U https://bioinform.jmir.org/2022/1/e36660 %U https://doi.org/10.2196/36660 %U http://www.ncbi.nlm.nih.gov/pubmed/36277075 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 9 %P e33775 %T The Data-Adaptive Fellegi-Sunter Model for Probabilistic Record Linkage: Algorithm Development and Validation for Incorporating Missing Data and Field Selection %A Li,Xiaochun %A Xu,Huiping %A Grannis,Shaun %+ Department of Biostatistics and Health Data Science, Indiana University School of Medicine, The Richard M. Fairbanks School of Public Health, HITS, Suite 3000, 410 W 10th St., Indianapolis, IN, 46202, United States, 1 317 274 2696, xiaochun@iu.edu %K record linkage %K Fellegi-Sunter model %K latent class model %K missing at random %K matching field selection %D 2022 %7 29.9.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Quality patient care requires comprehensive health care data from a broad set of sources. However, missing data in medical records and matching field selection are 2 real-world challenges in patient-record linkage. Objective: In this study, we aimed to evaluate the extent to which incorporating the missing at random (MAR)–assumption in the Fellegi-Sunter model and using data-driven selected fields improve patient-matching accuracy using real-world use cases. Methods: We adapted the Fellegi-Sunter model to accommodate missing data using the MAR assumption and compared the adaptation to the common strategy of treating missing values as disagreement with matching fields specified by experts or selected by data-driven methods. We used 4 use cases, each containing a random sample of record pairs with match statuses ascertained by manual reviews. Use cases included health information exchange (HIE) record deduplication, linkage of public health registry records to HIE, linkage of Social Security Death Master File records to HIE, and deduplication of newborn screening records, which represent real-world clinical and public health scenarios. Matching performance was evaluated using the sensitivity, specificity, positive predictive value, negative predictive value, and F1-score. Results: Incorporating the MAR assumption in the Fellegi-Sunter model maintained or improved F1-scores, regardless of whether matching fields were expert-specified or selected by data-driven methods. Combining the MAR assumption and data-driven fields optimized the F1-scores in the 4 use cases. Conclusions: MAR is a reasonable assumption in real-world record linkage applications: it maintains or improves F1-scores regardless of whether matching fields are expert-specified or data-driven. Data-driven selection of fields coupled with MAR achieves the best overall performance, which can be especially useful in privacy-preserving record linkage. %M 36173664 %R 10.2196/33775 %U https://www.jmir.org/2022/9/e33775 %U https://doi.org/10.2196/33775 %U http://www.ncbi.nlm.nih.gov/pubmed/36173664 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 9 %P e40547 %T On the Current Connection and Relation Between Health Informatics and Social Informatics %A Smutny,Zdenek %A Vehovar,Vasja %+ Faculty of Informatics and Statistics, Prague University of Economics and Business, W. Churchill Sq. 1938/4, Prague, 13067, Czech Republic, 420 224095473, zdenek.smutny@vse.cz %K biomedical informatics %K conceptual view %K clinical informatics %K international perspective %K medical informatics %D 2022 %7 28.9.2022 %9 Viewpoint %J J Med Internet Res %G English %X Scholars from the health and medical sciences have recently proposed the term social informatics (SI) as a new scientific subfield of health informatics (HI). However, SI is not a new academic concept; in fact, it has been continuously used in the social sciences and informatics since the 1970s. Although the dominant understanding of SI was established in the 1990s in the United States, a rich international perspective on SI has existed since the 1970s in other regions of the world. When that perspective is considered, the fields of understanding can be structured into 7 SI schools of thought. Against that conceptual background, this paper contributes to the discussion on the relationship between SI and HI, outlining possible perspectives of SI that are associated with health, medical, and clinical aspects. This paper argues against the multiplication and inconsistent appearance of the term SI when newly used in health and medical sciences. A more explicit name for the area that uses health and social data to advance individual and population health might be helpful to overcome this issue; giving an identity to this new field would help it to be understood more precisely and bring greater separation. This labeling could be fruitful for further segmentation of HI, which is rapidly expanding. %M 36169995 %R 10.2196/40547 %U https://www.jmir.org/2022/9/e40547 %U https://doi.org/10.2196/40547 %U http://www.ncbi.nlm.nih.gov/pubmed/36169995 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 3 %P e38926 %T COVID-19’s Impact on Digital Health Adoption: The Growing Gap Between a Technological and a Cultural Transformation %A Meskó,Bertalan %+ The Medical Futurist Institute, Povl Bang-Jensen u 2/B1 4/1, Budapest, 1118, Hungary, 36 36703807260, berci@medicalfuturist.com %K COVID-19 %K digital health %K future %K cultural transformation %K medical information %K technology adoption %K health care %K physician burnout %K burnout %D 2022 %7 19.9.2022 %9 Viewpoint %J JMIR Hum Factors %G English %X Health care in the 21st century has started undergoing major changes due to the rising number of patients with chronic conditions; increased access to new technologies, medical information, and peer support via the internet; and the ivory tower of medicine breaking down. This marks the beginning of a cultural transformation called digital health. This has also led to a shift in the roles of patients and medical professionals, resulting in a new, equal partnership. When COVID-19 hit, the adoption of digital health technologies skyrocketed. The technological revolution we had been aiming for in health care took place in just months due to the pandemic, but the cultural transition is lagging. This creates a dangerous gap between what is possible technologically through remote care, at-home lab tests, or health sensors and what patients and physicians are actually longing for. If we do it well enough now, we can spare a decade of technological transformations and bring that long-term vision of patients becoming the point of care to the practical reality of today. This is a historic opportunity we might not want to waste. %M 36121692 %R 10.2196/38926 %U https://humanfactors.jmir.org/2022/3/e38926 %U https://doi.org/10.2196/38926 %U http://www.ncbi.nlm.nih.gov/pubmed/36121692 %0 Journal Article %@ 1929-073X %I JMIR Publications %V 11 %N 2 %P e34533 %T Predicting Therapeutic Response to Unfractionated Heparin Therapy: Machine Learning Approach %A Abdel-Hafez,Ahmad %A Scott,Ian A %A Falconer,Nazanin %A Canaris,Stephen %A Bonilla,Oscar %A Marxen,Sven %A Van Garderen,Aaron %A Barras,Michael %+ Clinical Informatics, Metro South Health, Queensland Health, 199 Ipswich rd, Brisbane, QLD 4102, Australia, 61 431059039, ahmad.abdel-hafez@health.qld.gov.au %K heparin %K activated partial thromboplastin time %K aPTT %K predictive modeling %K machine learning %K personalized medicine %D 2022 %7 19.9.2022 %9 Original Paper %J Interact J Med Res %G English %X Background: Unfractionated heparin (UFH) is an anticoagulant drug that is considered a high-risk medication because an excessive dose can cause bleeding, whereas an insufficient dose can lead to a recurrent embolic event. Therapeutic response to the initiation of intravenous UFH is monitored using activated partial thromboplastin time (aPTT) as a measure of blood clotting time. Clinicians iteratively adjust the dose of UFH toward a target, indication-defined therapeutic aPTT range using nomograms, but this process can be imprecise and can take ≥36 hours to achieve the target range. Thus, a more efficient approach is required. Objective: In this study, we aimed to develop and validate a machine learning (ML) algorithm to predict aPTT within 12 hours after a specified bolus and maintenance dose of UFH. Methods: This was a retrospective cohort study of 3019 patient episodes of care from January 2017 to August 2020 using data collected from electronic health records of 5 hospitals in Queensland, Australia. Data from 4 hospitals were used to build and test ensemble models using cross-validation, whereas data from the fifth hospital were used for external validation. We built 2 ML models: a regression model to predict the aPTT value after a UFH bolus dose and a multiclass model to predict the aPTT, classified as subtherapeutic (aPTT <70 seconds), therapeutic (aPTT 70-100 seconds), or supratherapeutic (aPTT >100 seconds). Modeling was performed using Driverless AI (H2O), an automated ML tool, and 17 different experiments were iteratively conducted to optimize model accuracy. Results: In predicting aPTT, the best performing model was an ensemble with 4x LightGBM models with a root mean square error of 31.35 (SD 1.37). In predicting the aPTT class using a repurposed data set, the best performing ensemble model achieved an accuracy of 0.599 (SD 0.0289) and an area under the receiver operating characteristic curve of 0.735. External validation yielded similar results: root mean square error of 30.52 (SD 1.29) for the aPTT prediction model, and accuracy of 0.568 (SD 0.0315) and area under the receiver operating characteristic curve of 0.724 for the aPTT multiclassification model. Conclusions: To the best of our knowledge, this is the first ML model applied to intravenous UFH dosing that has been developed and externally validated in a multisite adult general medical and surgical inpatient setting. We present the processes of data collection, preparation, and feature engineering for replication. %M 35993617 %R 10.2196/34533 %U https://www.i-jmr.org/2022/2/e34533 %U https://doi.org/10.2196/34533 %U http://www.ncbi.nlm.nih.gov/pubmed/35993617 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 9 %P e38144 %T The Impact of Electronic Health Record Interoperability on Safety and Quality of Care in High-Income Countries: Systematic Review %A Li,Edmond %A Clarke,Jonathan %A Ashrafian,Hutan %A Darzi,Ara %A Neves,Ana Luisa %+ National Institute for Health and Care Research (NIHR) Imperial Patient Safety Translational Research Centre, Institute of Global Health Innovation, Imperial College London, Room 1035/7, QEQM Wing, St Mary’s Hospital Campus, London, W2 1NY, United Kingdom, 44 02033127259, edmond.li19@imperial.ac.uk %K electronic health records %K interoperability %K patient safety %K systematic literature review %K health information exchange %K digital health %D 2022 %7 15.9.2022 %9 Review %J J Med Internet Res %G English %X Background: Electronic health records (EHRs) and poor system interoperability are well-known issues in the use of health information technologies in most high-income countries worldwide. Despite the abundance of literature exploring their relationship, their practical implications on patient safety and quality of care remain unclear. Objective: This study aimed to examine how EHR interoperability affects patient safety, or other dimensions of care quality, in high-income health care settings. Methods: A systematic search was conducted using 4 web-based medical journal repositories and grey literature sources. The publications included were published in English between 2010 and 2022, pertaining to EHR use, interoperability, and patient safety or care quality in high-income settings. Screening was completed by 3 researchers in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Risk of bias assessments were performed using the Risk of Bias in Non-randomized Studies of Interventions and the Cochrane Risk of Bias 2 tools. The findings were presented as a narrative synthesis and mapped based on the Institute of Medicine’s framework for health care quality. Results: A total of 12 studies met the inclusion criteria to be included in our review. The findings were categorized into 6 common outcome measure categories: patient safety events, medication safety, data accuracy and errors, care effectiveness, productivity, and cost savings. EHR interoperability positively influenced medication safety, reduced patient safety events, and reduced costs. Improvements in time saving and clinical workflow are mixed. However, true measures of effect are difficult to determine with certainty because of the heterogeneity in the outcome measures used and notable variation in study quality. Conclusions: The benefits of EHR interoperability on the quality and safety of care remain unclear and reflect extensive heterogeneity in the interventions, designs, and outcome measures used. The establishment of common health information technology research outcome measures would support higher-quality research on the topic. Future research efforts should focus on both the positive and negative impacts of interoperable EHR interventions and explore patient perspectives, given the growing trend for patient involvement and stewardship over their own electronic clinical data. Trial Registration: PROSPERO CRD42020209285; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=209285 International Registered Report Identifier (IRRID): RR2-10.1136/bmjopen-2020-044941 %M 36107486 %R 10.2196/38144 %U https://www.jmir.org/2022/9/e38144 %U https://doi.org/10.2196/38144 %U http://www.ncbi.nlm.nih.gov/pubmed/36107486 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 9 %P e29927 %T Business Process Model and Notation and openEHR Task Planning for Clinical Pathway Standards in Infections: Critical Analysis %A Iglesias,Natalia %A Juarez,Jose M %A Campos,Manuel %+ Instituto de Investigación de Tecnologías de la Información y las Comunicaciones Orientadas, University of Murcia, Faculty of Computer Science, Campus Espinardo, University of Murcia, Murcia, 30100, Spain, 34 868887864, natalia.iglesias@um.es %K openEHR task planning %K business process model and notation %K BPMN %K clinical pathways %K catheter-related bloodstream infection %K CR-BSI %K clinical guidelines %D 2022 %7 15.9.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Clinical pathways (CPs) are usually expressed by means of workflow formalisms, providing health care personnel with an easy-to-understand, high-level conceptual model of medical steps in specific patient conditions, thereby improving overall health care process quality in clinical practice. From a standardized perspective, the business process model and notation (BPMN), a widely spread general-purpose process formalism, has been used for conceptual modeling in clinical domains, mainly because of its easy-to-use graphical notation, facilitating the common understanding and communication of the parties involved in health care. However, BPMN is not particularly oriented toward the peculiarities of complex clinical processes such as infection diagnosis and treatment, in which time plays a critical role, which is why much of the BPMN clinical-oriented research has revolved around how to extend the standard to address these special needs. The shift from an agnostic, general-purpose BPMN notation to a natively clinical-oriented notation such as openEHR Task Planning (TP) could constitute a major step toward clinical process improvement, enhancing the representation of CPs for infection treatment and other complex scenarios. Objective: Our work aimed to analyze the suitability of a clinical-oriented formalism (TP) to successfully represent typical process patterns in infection treatment, identifying domain-specific improvements to the standard that could help enhance its modeling capabilities, thereby promoting the widespread adoption of CPs to improve medical practice and overall health care quality. Methods: Our methodology consisted of 4 major steps: identification of key features of infection CPs through literature review, clinical guideline analysis, and BPMN extensions; analysis of the presence of key features in TP; modeling of relevant process patterns of catheter-related bloodstream infection as a case study; and analysis and proposal of extensions in view of the results. Results: We were able to easily represent the same logic applied in the extended BPMN-based process models in our case study using out-of-the-box standard TP primitives. However, we identified possible improvements to the current version of TP to allow for simpler conceptual models of infection CPs and possibly of other complex clinical scenarios. Conclusions: Our study showed that the clinical-oriented TP specification is able to successfully represent the most complex catheter-related bloodstream infection process patterns depicted in our case study and identified possible extensions that can help increase its adequacy for modeling infection CPs and possibly other complex clinical conditions. %M 36107480 %R 10.2196/29927 %U https://www.jmir.org/2022/9/e29927 %U https://doi.org/10.2196/29927 %U http://www.ncbi.nlm.nih.gov/pubmed/36107480 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 3 %P e34568 %T Evaluation of a Health Information Exchange System for Geriatric Health Care in Rural Areas: Development and Technical Acceptance Study %A Pfeuffer,Nils %A Beyer,Angelika %A Penndorf,Peter %A Leiz,Maren %A Radicke,Franziska %A Hoffmann,Wolfgang %A van den Berg,Neeltje %+ Section Epidemiology of Health Care and Community Health, Institute for Community Medicine, University Medicine Greifswald, Ellernholzstr. 1-2, Greifswald, 17489, Germany, 49 3834867618, nils.pfeuffer@med.uni-greifswald.de %K electronic health records %K health information exchange %K geriatrics %K community-based participatory research %K technical acceptance %K usability %K health information network %K postacute care %K patient-centered care %D 2022 %7 15.9.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Patients of geriatrics are often treated by several health care providers at the same time. The spatial, informational, and organizational separation of these health care providers can hinder the effective treatment of these patients. Objective: This study aimed to develop a regional health information exchange (HIE) system to improve HIE in geriatric treatment. This study also evaluated the usability of the regional HIE system and sought to identify barriers to and facilitators of its implementation. Methods: The development of the regional HIE system followed the community-based participatory research approach. The primary outcomes were the usability of the regional HIE system, expected implementation barriers and facilitators, and the quality of the developmental process. Data were collected and analyzed using a mixed methods approach. Results: A total of 3 focus regions were identified, 22 geriatric health care providers participated in the development of the regional HIE system, and 11 workshops were conducted between October 2019 and September 2020. In total, 12 participants responded to a questionnaire. The main results were that the regional HIE system should support the exchange of assessments, diagnoses, medication, assistive device supply, and social information. The regional HIE system was expected to be able to improve the quality and continuity of care. In total, 5 adoption facilitators were identified. The main points were adaptability of the regional HIE system to local needs, availability to different patient groups and treatment documents, web-based design, trust among the users, and computer literacy. A total of 13 barriers to adoption were identified. The main expected barriers to implementation were lack of resources, interoperability issues, computer illiteracy, lack of trust, privacy concerns, and ease-of-use issues. Conclusions: Participating health care professionals shared similar motivations for developing the regional HIE system, including improved quality of care, reduction of unnecessary examinations, and more effective health care provision. An overly complicated registration process for health care professionals and the patients’ free choice of their health care providers hinder the effectiveness of the regional HIE system, resulting in incomplete patient health information. However, the web-based design of the system bridges interoperability problems that exist owing to the different technical and organizational structures of the health care facilities involved. The regional HIE system is better accepted by health care professionals who are already engaged in an interdisciplinary, geriatric-focused network. This might indicate that pre-existing cross-organizational structures and processes are prerequisites for using HIE systems. The participatory design supports the development of technologies that are adaptable to regional needs. Health care providers are interested in participating in the development of an HIE system, but they often lack the required time, knowledge, and resources. %M 36107474 %R 10.2196/34568 %U https://humanfactors.jmir.org/2022/3/e34568 %U https://doi.org/10.2196/34568 %U http://www.ncbi.nlm.nih.gov/pubmed/36107474 %0 Journal Article %@ 2563-3570 %I JMIR Publications %V 3 %N 1 %P e37701 %T Diagnosis of a Single-Nucleotide Variant in Whole-Exome Sequencing Data for Patients With Inherited Diseases: Machine Learning Study Using Artificial Intelligence Variant Prioritization %A Huang,Yu-Shan %A Hsu,Ching %A Chune,Yu-Chang %A Liao,I-Cheng %A Wang,Hsin %A Lin,Yi-Lin %A Hwu,Wuh-Liang %A Lee,Ni-Chung %A Lai,Feipei %+ Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, Number 1, Roosevelt Road, Section 4, Taipei City, 106319, Taiwan, 886 2 33664924, flai@ntu.edu.tw %K next-generation sequencing %K genetic variation analysis %K machine learning %K artificial intelligence %K whole-exome sequencing %D 2022 %7 15.9.2022 %9 Original Paper %J JMIR Bioinform Biotech %G English %X Background: In recent years, thanks to the rapid development of next-generation sequencing (NGS) technology, an entire human genome can be sequenced in a short period. As a result, NGS technology is now being widely introduced into clinical diagnosis practice, especially for diagnosis of hereditary disorders. Although the exome data of single-nucleotide variant (SNV) can be generated using these approaches, processing the DNA sequence data of a patient requires multiple tools and complex bioinformatics pipelines. Objective: This study aims to assist physicians to automatically interpret the genetic variation information generated by NGS in a short period. To determine the true causal variants of a patient with genetic disease, currently, physicians often need to view numerous features on every variant manually and search for literature in different databases to understand the effect of genetic variation. Methods: We constructed a machine learning model for predicting disease-causing variants in exome data. We collected sequencing data from whole-exome sequencing (WES) and gene panel as training set, and then integrated variant annotations from multiple genetic databases for model training. The model built ranked SNVs and output the most possible disease-causing candidates. For model testing, we collected WES data from 108 patients with rare genetic disorders in National Taiwan University Hospital. We applied sequencing data and phenotypic information automatically extracted by a keyword extraction tool from patient’s electronic medical records into our machine learning model. Results: We succeeded in locating 92.5% (124/134) of the causative variant in the top 10 ranking list among an average of 741 candidate variants per person after filtering. AI Variant Prioritizer was able to assign the target gene to the top rank for around 61.1% (66/108) of the patients, followed by Variant Prioritizer, which assigned it for 44.4% (48/108) of the patients. The cumulative rank result revealed that our AI Variant Prioritizer has the highest accuracy at ranks 1, 5, 10, and 20. It also shows that AI Variant Prioritizer presents better performance than other tools. After adopting the Human Phenotype Ontology (HPO) terms by looking up the databases, the top 10 ranking list can be increased to 93.5% (101/108). Conclusions: We successfully applied sequencing data from WES and free-text phenotypic information of patient’s disease automatically extracted by the keyword extraction tool for model training and testing. By interpreting our model, we identified which features of variants are important. Besides, we achieved a satisfactory result on finding the target variant in our testing data set. After adopting the HPO terms by looking up the databases, the top 10 ranking list can be increased to 93.5% (101/108). The performance of the model is similar to that of manual analysis, and it has been used to help National Taiwan University Hospital with a genetic diagnosis. %R 10.2196/37701 %U https://bioinform.jmir.org/2022/1/e37701 %U https://doi.org/10.2196/37701 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 9 %P e35675 %T Dynamic Digital Twin: Diagnosis, Treatment, Prediction, and Prevention of Disease During the Life Course %A Mulder,Skander Tahar %A Omidvari,Amir-Houshang %A Rueten-Budde,Anja J %A Huang,Pei-Hua %A Kim,Ki-Hun %A Bais,Babette %A Rousian,Melek %A Hai,Rihan %A Akgun,Can %A van Lennep,Jeanine Roeters %A Willemsen,Sten %A Rijnbeek,Peter R %A Tax,David MJ %A Reinders,Marcel %A Boersma,Eric %A Rizopoulos,Dimitris %A Visch,Valentijn %A Steegers-Theunissen,Régine %+ Obstetrics and Gynaecology, Erasmus Medical Center, Dr. Molewaterplein 40, Rotterdam, 3015GD, Netherlands, 31 10 7038256, r.steegers@erasmusmc.nl %K digital health %K digital twin %K machine learning %K artifical intelligence %K obstetrics %K cardiovascular %K disease %K health %D 2022 %7 14.9.2022 %9 Viewpoint %J J Med Internet Res %G English %X A digital twin (DT), originally defined as a virtual representation of a physical asset, system, or process, is a new concept in health care. A DT in health care is not a single technology but a domain-adapted multimodal modeling approach incorporating the acquisition, management, analysis, prediction, and interpretation of data, aiming to improve medical decision-making. However, there are many challenges and barriers that must be overcome before a DT can be used in health care. In this viewpoint paper, we build on the current literature, address these challenges, and describe a dynamic DT in health care for optimizing individual patient health care journeys, specifically for women at risk for cardiovascular complications in the preconception and pregnancy periods and across the life course. We describe how we can commit multiple domains to developing this DT. With our cross-domain definition of the DT, we aim to define future goals, trade-offs, and methods that will guide the development of the dynamic DT and implementation strategies in health care. %M 36103220 %R 10.2196/35675 %U https://www.jmir.org/2022/9/e35675 %U https://doi.org/10.2196/35675 %U http://www.ncbi.nlm.nih.gov/pubmed/36103220 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 3 %P e40064 %T Patient Experience and Feedback After Using an Electronic Health Record–Integrated COVID-19 Symptom Checker: Survey Study %A Liu,Andrew W %A Odisho,Anobel Y %A Brown III,William %A Gonzales,Ralph %A Neinstein,Aaron B %A Judson,Timothy J %+ Center for Digital Health Innovation, University of California, San Francisco, 1700 Owens St, Suite 541, San Francisco, CA, 94158, United States, 1 415 514 8755, timothy.judson@ucsf.edu %K COVID-19 %K patient portals %K digital health %K diagnostic self evaluation %K medical informatics %K internet %K telemedicine %K triage %K feedback %K medical records systems %K San Francisco %K user experience %K user satisfaction %K self-triage %K symptom checker %K health system %K workflow %K feedback %K integration %K electronic health record %D 2022 %7 13.9.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Symptom checkers have been widely used during the COVID-19 pandemic to alleviate strain on health systems and offer patients a 24-7 self-service triage option. Although studies suggest that users may positively perceive web-based symptom checkers, no studies have quantified user feedback after use of an electronic health record–integrated COVID-19 symptom checker with self-scheduling functionality. Objective: In this paper, we aimed to understand user experience, user satisfaction, and user-reported alternatives to the use of a COVID-19 symptom checker with self-triage and self-scheduling functionality. Methods: We launched a patient-portal–based self-triage and self-scheduling tool in March 2020 for patients with COVID-19 symptoms, exposures, or questions. We made an optional, anonymous Qualtrics survey available to patients immediately after they completed the symptom checker. Results: Between December 16, 2021, and March 28, 2022, there were 395 unique responses to the survey. Overall, the respondents reported high satisfaction across all demographics, with a median rating of 8 out of 10 and 288/395 (47.6%) of the respondents giving a rating of 9 or 10 out of 10. User satisfaction scores were not associated with any demographic factors. The most common user-reported alternatives had the web-based tool not been available were calling the COVID-19 telephone hotline and sending a patient-portal message to their physician for advice. The ability to schedule a test online was the most important symptom checker feature for the respondents. The most common categories of user feedback were regarding other COVID-19 services (eg, telephone hotline), policies, or procedures, and requesting additional features or functionality. Conclusions: This analysis suggests that COVID-19 symptom checkers with self-triage and self-scheduling functionality may have high overall user satisfaction, regardless of user demographics. By allowing users to self-triage and self-schedule tests and visits, tools such as this may prevent unnecessary calls and messages to clinicians. Individual feedback suggested that the user experience for this type of tool is highly dependent on the organization's operational workflows for COVID-19 testing and care. This study provides insight for the implementation and improvement of COVID-19 symptom checkers to ensure high user satisfaction. %M 35960593 %R 10.2196/40064 %U https://humanfactors.jmir.org/2022/3/e40064 %U https://doi.org/10.2196/40064 %U http://www.ncbi.nlm.nih.gov/pubmed/35960593 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 9 %P e39235 %T Issues With Variability in Electronic Health Record Data About Race and Ethnicity: Descriptive Analysis of the National COVID Cohort Collaborative Data Enclave %A Cook,Lily %A Espinoza,Juan %A Weiskopf,Nicole G %A Mathews,Nisha %A Dorr,David A %A Gonzales,Kelly L %A Wilcox,Adam %A Madlock-Brown,Charisse %A , %+ Department of Medical Informatics and Clinical Epidemiology, School of Medicine, Oregon Health & Science University, Biomedical Information Communication Center, 3280 S.W. Sam Jackson Park Rd., Portland, OR, 97239, United States, 1 503 494 4502, lilyjune25@gmail.com %K social determinants of health %K health equity %K bias %K data quality %K data harmonization %K data standards %K terminology %K data aggregation %D 2022 %7 6.9.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: The adverse impact of COVID-19 on marginalized and under-resourced communities of color has highlighted the need for accurate, comprehensive race and ethnicity data. However, a significant technical challenge related to integrating race and ethnicity data in large, consolidated databases is the lack of consistency in how data about race and ethnicity are collected and structured by health care organizations. Objective: This study aims to evaluate and describe variations in how health care systems collect and report information about the race and ethnicity of their patients and to assess how well these data are integrated when aggregated into a large clinical database. Methods: At the time of our analysis, the National COVID Cohort Collaborative (N3C) Data Enclave contained records from 6.5 million patients contributed by 56 health care institutions. We quantified the variability in the harmonized race and ethnicity data in the N3C Data Enclave by analyzing the conformance to health care standards for such data. We conducted a descriptive analysis by comparing the harmonized data available for research purposes in the database to the original source data contributed by health care institutions. To make the comparison, we tabulated the original source codes, enumerating how many patients had been reported with each encoded value and how many distinct ways each category was reported. The nonconforming data were also cross tabulated by 3 factors: patient ethnicity, the number of data partners using each code, and which data models utilized those particular encodings. For the nonconforming data, we used an inductive approach to sort the source encodings into categories. For example, values such as “Declined” were grouped with “Refused,” and “Multiple Race” was grouped with “Two or more races” and “Multiracial.” Results: “No matching concept” was the second largest harmonized concept used by the N3C to describe the race of patients in their database. In addition, 20.7% of the race data did not conform to the standard; the largest category was data that were missing. Hispanic or Latino patients were overrepresented in the nonconforming racial data, and data from American Indian or Alaska Native patients were obscured. Although only a small proportion of the source data had not been mapped to the correct concepts (0.6%), Black or African American and Hispanic/Latino patients were overrepresented in this category. Conclusions: Differences in how race and ethnicity data are conceptualized and encoded by health care institutions can affect the quality of the data in aggregated clinical databases. The impact of data quality issues in the N3C Data Enclave was not equal across all races and ethnicities, which has the potential to introduce bias in analyses and conclusions drawn from these data. Transparency about how data have been transformed can help users make accurate analyses and inferences and eventually better guide clinical care and public policy. %M 35917481 %R 10.2196/39235 %U https://medinform.jmir.org/2022/9/e39235 %U https://doi.org/10.2196/39235 %U http://www.ncbi.nlm.nih.gov/pubmed/35917481 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 9 %P e38385 %T Evaluating the Impact of a Point-of-Care Cardiometabolic Clinical Decision Support Tool on Clinical Efficiency Using Electronic Health Record Audit Log Data: Algorithm Development and Validation %A Yan,Xiaowei %A Husby,Hannah %A Mudiganti,Satish %A Gbotoe,Madina %A Delatorre-Reimer,Jake %A Knobel,Kevin %A Hudnut,Andrew %A Jones,J B %+ Center for Health Systems Research, Sutter Health, 2121 N California Blvd, Suite 310, Walnut Creek, CA, 94596, United States, 1 925 287 4025, YanSX@sutterhealth.org %K digital health %K electronic health record %K EHR audit logs %K workflow efficiency %K cardiometabolic conditions %D 2022 %7 6.9.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Electronic health record (EHR) systems are becoming increasingly complicated, leading to concerns about rising physician burnout, particularly for primary care physicians (PCPs). Managing the most common cardiometabolic chronic conditions by PCPs during a limited clinical time with a patient is challenging. Objective: This study aimed to evaluate a Cardiometabolic Sutter Health Advanced Reengineered Encounter (CM-SHARE), a web-based application to visualize key EHR data, on the EHR use efficiency. Methods: We developed algorithms to identify key clinic workflow measures (eg, total encounter time, total physician time in the examination room, and physician EHR time in the examination room) using audit data, and we validated and calibrated the measures with time-motion data. We used a pre-post parallel design to identify propensity score–matched CM-SHARE users (cases), nonusers (controls), and nested-matched patients. Cardiometabolic encounters from matched case and control patients were used for the workflow evaluation. Outcome measures were compared between the cases and controls. We applied this approach separately to both the CM-SHARE pilot and spread phases. Results: Time-motion observation was conducted on 101 primary care encounters for 9 PCPs in 3 clinics. There was little difference (<0.8 minutes) between the audit data–derived workflow measures and the time-motion observation. Two key unobservable times from audit data, physician entry into and exiting the examination room, were imputed based on time-motion studies. CM-SHARE was launched with 6 pilot PCPs in April 2016. During the prestudy period (April 1, 2015, to April 1, 2016), 870 control patients with 2845 encounters were matched with 870 case patients and encounters, and 727 case patients with 852 encounters were matched with 727 control patients and 3754 encounters in the poststudy period (June 1, 2016, to June 30, 2017). Total encounter time was slightly shorter (mean −2.7, SD 1.4 minutes, 95% CI −4.7 to −0.9; mean –1.6, SD 1.1 minutes, 95% CI −3.2 to −0.1) for cases than controls for both periods. CM-SHARE saves physicians approximately 2 minutes EHR time in the examination room (mean −2.0, SD 1.3, 95% CI −3.4 to −0.9) compared with prestudy period and poststudy period controls (mean −1.9, SD 0.9, 95% CI −3.8 to −0.5). In the spread phase, 48 CM-SHARE spread PCPs were matched with 84 control PCPs and 1272 cases with 3412 control patients, having 1119 and 4240 encounters, respectively. A significant reduction in total encounter time for the CM-SHARE group was observed for short appointments (≤20 minutes; 5.3-minute reduction on average) only. Total physician EHR time was significantly reduced for both longer and shorter appointments (17%-33% reductions). Conclusions: Combining EHR audit log files and clinical information, our approach offers an innovative and scalable method and new measures that can be used to evaluate clinical EHR efficiency of digital tools used in clinical settings. %M 36066940 %R 10.2196/38385 %U https://medinform.jmir.org/2022/9/e38385 %U https://doi.org/10.2196/38385 %U http://www.ncbi.nlm.nih.gov/pubmed/36066940 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 9 %P e37896 %T Identification of Preterm Labor Evaluation Visits and Extraction of Cervical Length Measures from Electronic Health Records Within a Large Integrated Health Care System: Algorithm Development and Validation %A Xie,Fagen %A Khadka,Nehaa %A Fassett,Michael J %A Chiu,Vicki Y %A Avila,Chantal C %A Shi,Jiaxiao %A Yeh,Meiyu %A Kawatkar,Aniket %A Mensah,Nana A %A Sacks,David A %A Getahun,Darios %+ Department of Research and Evaluation, Kaiser Permanente Southern California, 100 S. Los Robles Avenue, 2nd Floor, Pasadena, CA, 91101, United States, 1 626 564 3294, fagen.xie@kp.org %K preterm labor %K preterm birth %K fetal fibronectin %K transvaginal ultrasound %K cervical length %K natural language processing %K computerized algorithm %K data extraction %K patient records %K clinical notes %K evaluation notes %K patient care %K patient notes %K electronic health records %D 2022 %7 6.9.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Preterm birth (PTB) represents a significant public health problem in the United States and throughout the world. Accurate identification of preterm labor (PTL) evaluation visits is the first step in conducting PTB-related research. Objective: We aimed to develop a validated computerized algorithm to identify PTL evaluation visits and extract cervical length (CL) measures from electronic health records (EHRs) within a large integrated health care system. Methods: We used data extracted from the EHRs at Kaiser Permanente Southern California between 2009 and 2020. First, we identified triage and hospital encounters with fetal fibronectin (fFN) tests, transvaginal ultrasound (TVUS) procedures, PTL medications, or PTL diagnosis codes within 240/7-346/7 gestational weeks. Second, clinical notes associated with triage and hospital encounters within 240/7-346/7 gestational weeks were extracted from EHRs. A computerized algorithm and an automated process were developed and refined by multiple iterations of chart review and adjudication to search the following PTL indicators: fFN tests, TVUS procedures, abdominal pain, uterine contractions, PTL medications, and descriptions of PTL evaluations. An additional process was constructed to extract the CLs from the corresponding clinical notes of these identified PTL evaluation visits. Results: A total of 441,673 live birth pregnancies were identified between 2009 and 2020. Of these, 103,139 pregnancies (23.35%) had documented PTL evaluation visits identified by the computerized algorithm. The trend of pregnancies with PTL evaluation visits slightly decreased from 24.41% (2009) to 17.42% (2020). Of the first 103,139 PTL visits, 19,439 (18.85%) and 44,423 (43.97%) had an fFN test and a TVUS, respectively. The percentage of first PTL visits with an fFN test decreased from 18.06% at 240/7 gestational weeks to 2.32% at 346/7 gestational weeks, and TVUS from 54.67% at 240/7 gestational weeks to 12.05% in 346/7 gestational weeks. The mean (SD) of the CL was 3.66 (0.99) cm with a mean range of 3.61-3.69 cm that remained stable across the study period. Of the pregnancies with PTL evaluation visits, the rate of PTB remained stable over time (20,399, 19.78%). Validation of the computerized algorithms against 100 randomly selected records from these potential PTL visits showed positive predictive values of 97%, 94.44%, 100%, and 96.43% for the PTL evaluation visits, fFN tests, TVUS, and CL, respectively, along with sensitivity values of 100%, 90%, and 90%, and specificity values of 98.8%, 100%, and 98.6% for the fFN test, TVUS, and CL, respectively. Conclusions: The developed computerized algorithm effectively identified PTL evaluation visits and extracted the corresponding CL measures from the EHRs. Validation against this algorithm achieved a high level of accuracy. This computerized algorithm can be used for conducting PTL- or PTB-related pharmacoepidemiologic studies and patient care reviews. %M 36066930 %R 10.2196/37896 %U https://medinform.jmir.org/2022/9/e37896 %U https://doi.org/10.2196/37896 %U http://www.ncbi.nlm.nih.gov/pubmed/36066930 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 8 %P e29431 %T An mHealth-Based Health Management Information System Among Health Workers in Volta and Eastern Regions of Ghana: Pre-Post Comparison Analysis %A Lee,Young-ji %A Lee,Seohyun %A Kim,SeYeon %A Choi,Wonil %A Jeong,Yoojin %A Rhim,Nina Jin Joo %A Seo,Ilwon %A Kim,Sun-Young %+ Department of Public Health Sciences, Graduate School of Public Health, Seoul National University, Gwanak Campus, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, Republic of Korea, 82 28802768, sykim22@snu.ac.kr %K mobile health %K mHealth %K e-Tracker %K health information system %K HIS %K health information management system %K HIMS %K District Health Information Management System %K DHIMS %K maternal and child health %K MCH %K electronic health record %K EHR %K health workers %D 2022 %7 31.8.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Despite the increasing attention to electronic health management information systems (HMISs) in global health, most African countries still depend on inefficient paper-based systems. Good Neighbors International and Evaluate 4 Health have recently supported the Ghana Health Service on the rollout of a mobile health–based HMIS called the e-Tracker system in 2 regions in Ghana. The e-Tracker is an Android-based tracker capture app that electronically manages maternal and child health (MCH) data. The Ghana Health Service has implemented this new system in Community Health Planning and Services in the 2 regions (Volta and Eastern). Objective: This study aims to evaluate changes in health workers’ capacity and behavior after using the e-Tracker to deliver MCH services. Specifically, the study assesses the changes in knowledge, attitude, and practice (KAP) of the health workers toward the e-Tracker system by comparing the pre- and postsurvey results. Methods: The KAP of frontline health workers was measured through self-administered surveys before and after using the e-Tracker system to assess their capacity and behavioral change toward the system. A total of 1124 health workers from the Volta and Eastern regions responded to the pre-post surveys. This study conducted the McNemar chi-square test and Wilcoxon signed-rank test for a pre-post comparison analysis. In addition, random-effects ordered logistic regression analysis and random-effects panel analysis were conducted to identify factors associated with KAP level. Results: The pre-post comparison analysis showed significant improvement in health workers’ capacity, with higher knowledge and practice levels after using the e-Tracker system. As for knowledge, there was a 9.9%-point increase (from 559/1109, 50.41% to 669/1109, 60.32%) in the proportion of the respondents who were able to generate basic statistics on the number of children born in a random month within 30 minutes. In the practice section, the percentage of respondents who had scheduled clientencounters increased from 91.41% (968/1059) to 97.83% (1036/1059). By contrast, responses to the attitude (acceptability) became less favorable after experiencing the actual system. For instance, 48.53% (544/1121) initially expressed their preferences for an electronic system; however, the proportion decreased to 33.45% (375/1121) after the intervention. Random-effects ordered logistic regression showed that days of overwork were significantly associated with health workers’ attitudes toward the e-Tracker system. Conclusions: This study provides empirical evidence that the e-Tracker system is conducive to enhancing capacity in MCH data management for providing necessary MCH services. However, the change in attitude implies that the users appear to feel less comfortable using the new system. As Ghana plans to scale up the electronic HMIS system using the e-Tracker to the national level, strategies to enhance health workers’ attitudes are necessary to sustain this new system. %M 36044256 %R 10.2196/29431 %U https://medinform.jmir.org/2022/8/e29431 %U https://doi.org/10.2196/29431 %U http://www.ncbi.nlm.nih.gov/pubmed/36044256 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 8 %P e40384 %T Development and Evaluation of a Natural Language Processing Annotation Tool to Facilitate Phenotyping of Cognitive Status in Electronic Health Records: Diagnostic Study %A Noori,Ayush %A Magdamo,Colin %A Liu,Xiao %A Tyagi,Tanish %A Li,Zhaozhi %A Kondepudi,Akhil %A Alabsi,Haitham %A Rudmann,Emily %A Wilcox,Douglas %A Brenner,Laura %A Robbins,Gregory K %A Moura,Lidia %A Zafar,Sahar %A Benson,Nicole M %A Hsu,John %A R Dickson,John %A Serrano-Pozo,Alberto %A Hyman,Bradley T %A Blacker,Deborah %A Westover,M Brandon %A Mukerji,Shibani S %A Das,Sudeshna %+ Department of Neurology, Massachusetts General Hospital, 55 Fruit Street, Boston, MA, 02114, United States, 1 617 726 2000, SDAS5@mgh.harvard.edu %K chart review %K cognition %K cognitive status %K dementia %K diagnostic %K electronic health record %K health care %K natural language processing %K research cohort %D 2022 %7 30.8.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Electronic health records (EHRs) with large sample sizes and rich information offer great potential for dementia research, but current methods of phenotyping cognitive status are not scalable. Objective: The aim of this study was to evaluate whether natural language processing (NLP)–powered semiautomated annotation can improve the speed and interrater reliability of chart reviews for phenotyping cognitive status. Methods: In this diagnostic study, we developed and evaluated a semiautomated NLP-powered annotation tool (NAT) to facilitate phenotyping of cognitive status. Clinical experts adjudicated the cognitive status of 627 patients at Mass General Brigham (MGB) health care, using NAT or traditional chart reviews. Patient charts contained EHR data from two data sets: (1) records from January 1, 2017, to December 31, 2018, for 100 Medicare beneficiaries from the MGB Accountable Care Organization and (2) records from 2 years prior to COVID-19 diagnosis to the date of COVID-19 diagnosis for 527 MGB patients. All EHR data from the relevant period were extracted; diagnosis codes, medications, and laboratory test values were processed and summarized; clinical notes were processed through an NLP pipeline; and a web tool was developed to present an integrated view of all data. Cognitive status was rated as cognitively normal, cognitively impaired, or undetermined. Assessment time and interrater agreement of NAT compared to manual chart reviews for cognitive status phenotyping was evaluated. Results: NAT adjudication provided higher interrater agreement (Cohen κ=0.89 vs κ=0.80) and significant speed up (time difference mean 1.4, SD 1.3 minutes; P<.001; ratio median 2.2, min-max 0.4-20) over manual chart reviews. There was moderate agreement with manual chart reviews (Cohen κ=0.67). In the cases that exhibited disagreement with manual chart reviews, NAT adjudication was able to produce assessments that had broader clinical consensus due to its integrated view of highlighted relevant information and semiautomated NLP features. Conclusions: NAT adjudication improves the speed and interrater reliability for phenotyping cognitive status compared to manual chart reviews. This study underscores the potential of an NLP-based clinically adjudicated method to build large-scale dementia research cohorts from EHRs. %M 36040790 %R 10.2196/40384 %U https://www.jmir.org/2022/8/e40384 %U https://doi.org/10.2196/40384 %U http://www.ncbi.nlm.nih.gov/pubmed/36040790 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 8 %P e37472 %T Investigating a Work System Approach to Implement an Emergency Department Surge Management System: Case Study %A Jewer,Jennifer %+ Faculty of Business Administration, Memorial University of Newfoundland, 300 Prince Philip Drive, St. John's, NL, A1B 3X5, Canada, 1 7098643094, jenniferj@mun.ca %K emergency department %K surge management %K work system %K system implementation %K emergency department information system %K mobile phone %D 2022 %7 25.8.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Emergency department (ED) crowding is a global health care issue. eHealth systems have the potential to reduce crowding; however, the true benefits are seldom realized because the systems are not integrated into clinicians’ work. We sought a deep understanding of how an eHealth system implementation can be structured to truly integrate the system into the workflow. Objective: The specific objectives of this study were to examine whether work system theory (WST) is a good approach to structure the implementation of an eHealth system by incorporating the entire work system, and not just the eHealth system, in the implementation framework; identify the role that specific elements of WST’s static framework and dynamic work system life cycle model play in the implementation; and demonstrate how WST can be applied in the health care setting to guide the implementation of an eHealth system. Methods: Through a case study of an ED in a rural hospital, we used a mixed methods approach to examine the implementation of a surge management system through the lens of WST. We conducted 14 hours of observation in the ED; 20 interviews with clinicians, management, and members of the implementation team; and a survey of 23 clinicians; reviewed related documentation; and analyzed ED data to measure wait times. We used template analysis based on WST to structure our analysis of qualitative data and descriptive statistics for quantitative data. Results: The surge management system helped to reduce crowding in the ED, staff was satisfied with the implementation, and wait time improvements have been maintained for several years. Although study participants indicated changes to their workflow, 72% (13/18) of survey participants were satisfied with their use of the system, and 82% (14/17) indicated that it was integrated with their workflow. Examining the implementation through the lens of WST enabled us to identify the aspects of the implementation that made it so successful. By applying the WST static framework, we saw how the implementation team incorporated the elements of the ED work system, assessed their alignment, and designed interventions to address areas of misalignment. The dynamic work system life cycle model captured how planned and unplanned changes were managed throughout the iterative implementation cycle—83% (15/18) of participants indicated that there was sufficient management support for the changes and 80% (16/20) indicated the change served an important purpose. Conclusions: The broad scope and holistic approach of WST is well suited to guide eHealth system implementations as it focuses efforts on the entire work system and not just the IT artifact. We broaden the focus of WST by applying it to the implementation of an ED surge management system. These findings will guide further studies and implementations of eHealth systems using WST. %M 36006684 %R 10.2196/37472 %U https://www.jmir.org/2022/8/e37472 %U https://doi.org/10.2196/37472 %U http://www.ncbi.nlm.nih.gov/pubmed/36006684 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 11 %N 8 %P e38026 %T Assessing the Clinical and Socioeconomic Burden of Respiratory Syncytial Virus in Children Aged Under 5 Years in Primary Care: Protocol for a Prospective Cohort Study in England and Report on the Adaptations of the Study to the COVID-19 Pandemic %A Hoang,Uy %A Button,Elizabeth %A Armstrong,Miguel %A Okusi,Cecilia %A Ellis,Joanna %A Zambon,Maria %A Anand,Sneha %A Delanerolle,Gayathri %A Hobbs,F D Richard %A van Summeren,Jojanneke %A Paget,John %A de Lusignan,Simon %+ Nuffield Department of Primary Care Health Sciences, University of Oxford, Eagle House, Walton Well Road, Oxford, OX2 6ED, United Kingdom, 44 01865617283, simon.delusignan@phc.ox.ac.uk %K medical records systems, computerized %K respiratory syncytial virus %K general practitioners %K pandemics %K COVID-19 %K general practice %K primary health care %K outcome assessment, health care %K respiratory %K children %K pediatric %D 2022 %7 25.8.2022 %9 Protocol %J JMIR Res Protoc %G English %X Background: Respiratory syncytial virus (RSV) commonly causes lower respiratory tract infections and hospitalization in children. In 2019-2020, the Europe-wide RSV ComNet standardized study protocol was developed to measure the clinical and socioeconomic disease burden of RSV infections among children aged <5 years in primary care. RSV has a recognized seasonality in England. Objective: We aimed to describe (1) the adaptations of the RSV ComNet standardized study protocol for England and (2) the challenges of conducting the study during the COVID-19 pandemic. Methods: This study was conducted by the Oxford-Royal College of General Practitioners Research and Surveillance Centre—the English national primary care sentinel network. We invited all (N=248) general practices within the network that undertook virology sampling to participate in the study by recruiting eligible patients (registered population: n=3,056,583). Children aged <5 years with the following case definition of RSV infection were included in the study: those consulting a health care practitioner in primary care with symptoms meeting the World Health Organization’s definition of acute respiratory illness or influenza-like illness who have laboratory-confirmed RSV infection. The parents/guardians of these cases were asked to complete 2 previously validated questionnaires (14 and 30 days postsampling). A sample size of at least 100 RSV-positive cases is required to estimate the percentage of children that consult in primary care who need hospitalization. Assuming a swab positivity rate of 20% in children aged <5 years, we estimated that 500 swabs are required. We adapted our method for the pandemic by extending sampling planned for winter 2020-2021 to a rolling data collection, allowing verbal consent and introducing home swabbing because of increased web-based consultations during the COVID-19 pandemic. Results: The preliminary results of the data collection between International Organization for Standardization (ISO) weeks 1-41 in 2021 are described. There was no RSV detected in the winter of 2020-2021 through the study. The first positive RSV swab collected through the sentinel network in England was collected in ISO week 17 and then every week since ISO week 25. In total, 16 (N=248, 6.5%) of the virology-sampling practices volunteered to participate; these were high-sampling practices collecting the majority of eligible swabs across the sentinel network—200 (43.8%) out of 457 swabs, of which 54 (N=200, 27%) were positive for RSV. Conclusions: Measures to control the COVID-19 pandemic meant there was no circulating RSV last winter; however, RSV has circulated out of season, as detected by the sentinel network. The sentinel network practices have collected 40% (200/500) of the required samples, and 27% (54/200) were RSV positive. We have demonstrated the feasibility of implementing a European-standardized RSV disease burden study protocol in England during a pandemic, and we now need to recruit to this adapted protocol. International Registered Report Identifier (IRRID): DERR1-10.2196/38026 %M 35960819 %R 10.2196/38026 %U https://www.researchprotocols.org/2022/8/e38026 %U https://doi.org/10.2196/38026 %U http://www.ncbi.nlm.nih.gov/pubmed/35960819 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 8 %P e37284 %T Interactive Medical Image Labeling Tool to Construct a Robust Convolutional Neural Network Training Data Set: Development and Validation Study %A Reifs,David %A Reig-Bolaño,Ramon %A Casals,Marta %A Grau-Carrion,Sergi %+ Digital Care Research Group, Centre for Health and Social Care, Universitat of Vic-Central University of Catalonia, Carrer de la Sagrada Família, 7, Vic, 08500, Spain, 34 938861222, david.reifs@uvic.cat %K wound assessment %K pressure ulcers %K wound tissue classification %K labeling %K machine learning %D 2022 %7 22.8.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Skin ulcers are an important cause of morbidity and mortality everywhere in the world and occur due to several causes, including diabetes mellitus, peripheral neuropathy, immobility, pressure, arteriosclerosis, infections, and venous insufficiency. Ulcers are lesions that fail to undergo an orderly healing process and produce functional and anatomical integrity in the expected time. In most cases, the methods of analysis used nowadays are rudimentary, which leads to errors and the use of invasive and uncomfortable techniques on patients. There are many studies that use a convolutional neural network to classify the different tissues in a wound. To obtain good results, the network must be trained with a correctly labeled data set by an expert in wound assessment. Typically, it is difficult to label pixel by pixel using a professional photo editor software, as this requires extensive time and effort from a health professional. Objective: The aim of this paper is to implement a new, fast, and accurate method of labeling wound samples for training a neural network to classify different tissues. Methods: We developed a support tool and evaluated its accuracy and reliability. We also compared the support tool classification with a digital gold standard (labeling the data with an image editing software). Results: The obtained comparison between the gold standard and the proposed method was 0.9789 for background, 0.9842 for intact skin, 0.8426 for granulation tissue, 0.9309 for slough, and 0.9871 for necrotic. The obtained speed on average was 2.6, compared to that of an advanced image editing user. Conclusions: This method increases tagging speed on average compared to an advanced image editing user. This increase is greater with untrained users. The samples obtained with the new system are indistinguishable from the samples made with the gold standard. %M 35994311 %R 10.2196/37284 %U https://medinform.jmir.org/2022/8/e37284 %U https://doi.org/10.2196/37284 %U http://www.ncbi.nlm.nih.gov/pubmed/35994311 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 8 %P e38714 %T The Interplay of Work, Digital Health Usage, and the Perceived Effects of Digitalization on Physicians’ Work: Network Analysis Approach %A Saukkonen,Petra %A Elovainio,Marko %A Virtanen,Lotta %A Kaihlanen,Anu-Marja %A Nadav,Janna %A Lääveri,Tinja %A Vänskä,Jukka %A Viitanen,Johanna %A Reponen,Jarmo %A Heponiemi,Tarja %+ Finnish Institute for Health and Welfare, Mannerheimintie 166, Helsinki, PO Box 30, Finland, 358 29 524 8289, petra.saukkonen@thl.fi %K network analysis %K mixed graphical model %K physicians %K health care digitalization %K digitalization of work %K work in transformation %K digital health %D 2022 %7 17.8.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: In health care, the benefits of digitalization need to outweigh the risks, but there is limited knowledge about the factors affecting this balance in the work environment of physicians. To achieve the benefits of digitalization, a more comprehensive understanding of this complex phenomenon related to the digitalization of physicians’ work is needed. Objective: The aim of this study was to examine physicians’ perceptions of the effects of health care digitalization on their work and to analyze how these perceptions are associated with multiple factors related to work and digital health usage. Methods: A representative sample of 4630 (response rate 24.46%) Finnish physicians (2960/4617, 64.11% women) was used. Statements measuring the perceived effects of digitalization on work included the patients’ active role, preventive work, interprofessional cooperation, decision support, access to patient information, and faster consultations. Network analysis of the perceived effects of digitalization and factors related to work and digital health usage was conducted using mixed graphical modeling. Adjusted and standardized regression coefficients are denoted by b. Centrality statistics were examined to evaluate the relative influence of each variable in terms of node strength. Results: Nearly half of physicians considered that digitalization has promoted an active role for patients in their own care (2104/4537, 46.37%) and easier access to patient information (1986/4551, 43.64%), but only 1 in 10 (445/4529, 9.82%) felt that the impact has been positive on consultation times with patients. Almost half of the respondents estimated that digitalization has neither increased nor decreased the possibilities for preventive work (2036/4506, 45.18%) and supportiveness of clinical decision support systems (1941/4458, 43.54%). When all variables were integrated into the network, the most influential variables were purpose of using health information systems, employment sector, and specialization status. However, the grade given to the electronic health record (EHR) system that was primarily used had the strongest direct links to faster consultations (b=0.32) and facilitated access to patient information (b=0.28). At least 6 months of use of the main EHR was associated with facilitated access to patient information (b=0.18). Conclusions: The results highlight the complex interdependence of multiple factors associated with the perceived effects of digitalization on physicians’ work. It seems that a high-quality EHR system is critical for promoting smooth clinical practice. In addition, work-related factors may influence other factors that affect digital health success. These factors should be considered when developing and implementing new digital health technologies or services for physicians’ work. The adoption of digital health is not just a technological project but a project that changes existing work practices. %M 35976692 %R 10.2196/38714 %U https://www.jmir.org/2022/8/e38714 %U https://doi.org/10.2196/38714 %U http://www.ncbi.nlm.nih.gov/pubmed/35976692 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 8 %P e34858 %T Physicians’ Perceptions as Predictors of the Future Use of the National Death Information System in Peru: Cross-sectional Study %A Vargas-Herrera,Javier %A Meneses,Giovanni %A Cortez-Escalante,Juan %+ Department of Preventive Medicine and Public Health, National University of San Marcos, Av German Amezaga 375, Lima, 15081, Peru, 51 945029342, jvargash@unmsm.edu.pe %K death certificates %K health information system %K mortality %K vital statistics %K Technology Acceptance Model %K model %K acceptance model %K certificates %K information system %K physicians %K predictors %K cross-sectional study %K analysis %K death %D 2022 %7 15.8.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: A computer application called the National Death Information System (SINADEF) was implemented in Peru so that physicians can prepare death certificates in electronic format and the information is available online. In 2018, only half of the estimated deaths in Peru were certified using SINADEF. When a death is certified in paper format, the probability being entered in the mortality database decreases. It is important to know, from the user’s perspective, the factors that can influence the successful implementation of SINADEF. SINADEF can only be successfully implemented if it is known whether physicians believe that it is useful and easy to operate. Objective: The aim of this study was to identify the perceptions of physicians and other factors as predictors of their behavioral intention to use SINADEF to certify a death. Methods: This study had an observational, cross-sectional design. A survey was provided to physicians working in Peru, who used SINADEF to certify a death for a period of 12 months, starting in November 2019. A questionnaire was adapted based on the Technology Acceptance Model. The questions measured the dimensions of subjective norm, image, job relevance, output quality, demonstrability of results, perceived usefulness, perceived ease of use, and behavioral intention to use. Chi-square and logistic regression tests were used in the analysis, and a confidence level of 95% was chosen to support a significant association. Results: In this study, 272 physicians responded to the survey; 184 (67.6%) were men and the average age was 45.3 (SD 10.1) years. The age range was 24 to 73 years. In the bivariate analysis, the intention to use SINADEF was found to be associated with (1) perceived usefulness, expressed as “using SINADEF avoids falsifying a death certificate” (P<.001), “using SINADEF reduces the risk of errors” (P<.001), and “using SINADEF allows for filling out a certificate in less time” (P<.001); and (2) perceived ease of use, expressed as “I think SINADEF is easy to use” (P<.001). In the logistic regression, perceived usefulness (odds ratio [OR] 8.5, 95% CI 2.2-32.3; P=.002), perceived ease of use (OR 10.1, 95% CI 2.4-41.8; P=.001), and training in filling out death certificates (OR 8.3, 95% CI 1.6-42.8; P=.01) were found to be predictors of the behavioral intention to use SINADEF. Conclusions: The behavioral intention to use SINADEF was related to the perception that it is an easy-to-use system, the belief that it improves the performance of physicians in carrying out the task at hand, and with training in filling out death certificates. %M 35969435 %R 10.2196/34858 %U https://www.jmir.org/2022/8/e34858 %U https://doi.org/10.2196/34858 %U http://www.ncbi.nlm.nih.gov/pubmed/35969435 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 8 %P e38052 %T Exploiting Intersentence Information for Better Question-Driven Abstractive Summarization: Algorithm Development and Validation %A Wang,Xin %A Wang,Jian %A Xu,Bo %A Lin,Hongfei %A Zhang,Bo %A Yang,Zhihao %+ School of Computer Science and Technology, Dalian University of Technology, No 2 Linggong Road, Dalian, 116023, China, 86 13604119266, wangjian@dlut.edu.cn %K question-driven abstractive summarization %K transformer %K multi-head attention %K pointer network %K question answering %K factual consistency %K algorithm %K validation %K natural language processing %D 2022 %7 15.8.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Question-driven summarization has become a practical and accurate approach to summarizing the source document. The generated summary should be concise and consistent with the concerned question, and thus, it could be regarded as the answer to the nonfactoid question. Existing methods do not fully exploit question information over documents and dependencies across sentences. Besides, most existing summarization evaluation tools like recall-oriented understudy for gisting evaluation (ROUGE) calculate N-gram overlaps between the generated summary and the reference summary while neglecting the factual consistency problem. Objective: This paper proposes a novel question-driven abstractive summarization model based on transformer, including a two-step attention mechanism and an overall integration mechanism, which can generate concise and consistent summaries for nonfactoid question answering. Methods: Specifically, the two-step attention mechanism is proposed to exploit the mutual information both of question to context and sentence over other sentences. We further introduced an overall integration mechanism and a novel pointer network for information integration. We conducted a question-answering task to evaluate the factual consistency between the generated summary and the reference summary. Results: The experimental results of question-driven summarization on the PubMedQA data set showed that our model achieved ROUGE-1, ROUGE-2, and ROUGE-L measures of 36.01, 15.59, and 30.22, respectively, which is superior to the state-of-the-art methods with a gain of 0.79 (absolute) in the ROUGE-2 score. The question-answering task demonstrates that the generated summaries of our model have better factual constancy. Our method achieved 94.2% accuracy and a 77.57% F1 score. Conclusions: Our proposed question-driven summarization model effectively exploits the mutual information among the question, document, and summary to generate concise and consistent summaries. %M 35969463 %R 10.2196/38052 %U https://medinform.jmir.org/2022/8/e38052 %U https://doi.org/10.2196/38052 %U http://www.ncbi.nlm.nih.gov/pubmed/35969463 %0 Journal Article %@ 2817-092X %I JMIR Publications %V 1 %N 1 %P e41122 %T JMIR Neurotechnology: Connecting Clinical Neuroscience and (Information) Technology %A Kubben,Pieter %+ Faculty of Health, Medicine and Life Sciences, School for Mental Health and Neuroscience, Maastricht University Medical Center, PO Box 616, Maastricht, 6200 MD, Netherlands, 31 43 388 2222, pieter.kubben@maastrichtuniversity.nl %K neurotechnology %K neurological disorders %K treatment tools %K chronic neurological disease %K information technology %D 2022 %7 11.8.2022 %9 Editorial %J JMIR Neurotech %G English %X %R 10.2196/41122 %U https://neuro.jmir.org/2022/1/e41122 %U https://doi.org/10.2196/41122 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 8 %P e32319 %T Using the Diagnostic Odds Ratio to Select Patterns to Build an Interpretable Pattern-Based Classifier in a Clinical Domain: Multivariate Sequential Pattern Mining Study %A Casanova,Isidoro J %A Campos,Manuel %A Juarez,Jose M %A Gomariz,Antonio %A Lorente-Ros,Marta %A Lorente,Jose A %+ AIKE Research Team (INTICO), Computer Science Faculty, University of Murcia, Edificio 32, Campus de Espinardo, Murcia, 30100, Spain, 34 868887150, isidoroj@um.es %K sequential patterns %K survival classification %K diagnostic odds ratio %K burn units %D 2022 %7 10.8.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: It is important to exploit all available data on patients in settings such as intensive care burn units (ICBUs), where several variables are recorded over time. It is possible to take advantage of the multivariate patterns that model the evolution of patients to predict their survival. However, pattern discovery algorithms generate a large number of patterns, of which only some are relevant for classification. Objective: We propose to use the diagnostic odds ratio (DOR) to select multivariate sequential patterns used in the classification in a clinical domain, rather than employing frequency properties. Methods: We used data obtained from the ICBU at the University Hospital of Getafe, where 6 temporal variables for 465 patients were registered every day during 5 days, and to model the evolution of these clinical variables, we used multivariate sequential patterns by applying 2 different discretization methods for the continuous attributes. We compared 4 ways in which to employ the DOR for pattern selection: (1) we used it as a threshold to select patterns with a minimum DOR; (2) we selected patterns whose differential DORs are higher than a threshold with regard to their extensions; (3) we selected patterns whose DOR CIs do not overlap; and (4) we proposed the combination of threshold and nonoverlapping CIs to select the most discriminative patterns. As a baseline, we compared our proposals with Jumping Emerging Patterns, one of the most frequently used techniques for pattern selection that utilizes frequency properties. Results: We have compared the number and length of the patterns eventually selected, classification performance, and pattern and model interpretability. We show that discretization has a great impact on the accuracy of the classification model, but that a trade-off must be found between classification accuracy and the physicians’ capacity to interpret the patterns obtained. We have also identified that the experiments combining threshold and nonoverlapping CIs (Option 4) obtained the fewest number of patterns but also with the smallest size, thus implying the loss of an acceptable accuracy with regard to clinician interpretation. The best classification model according to the trade-off is a JRIP classifier with only 5 patterns (20 items) that was built using unsupervised correlation preserving discretization and differential DOR in a beam search for the best pattern. It achieves a specificity of 56.32% and an area under the receiver operating characteristic curve of 0.767. Conclusions: A method for the classification of patients’ survival can benefit from the use of sequential patterns, as these patterns consider knowledge about the temporal evolution of the variables in the case of ICBU. We have proved that the DOR can be used in several ways, and that it is a suitable measure to select discriminative and interpretable quality patterns. %M 35947437 %R 10.2196/32319 %U https://medinform.jmir.org/2022/8/e32319 %U https://doi.org/10.2196/32319 %U http://www.ncbi.nlm.nih.gov/pubmed/35947437 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 9 %N 8 %P e38428 %T Predicting Patient Wait Times by Using Highly Deidentified Data in Mental Health Care: Enhanced Machine Learning Approach %A Rastpour,Amir %A McGregor,Carolyn %+ Faculty of Business and Information Technology, Ontario Tech University, 2000 Simcoe St N, Oshawa, ON, L1G 0C5, Canada, 1 905 721 8668 ext 2830, amir.rastpour@ontariotechu.ca %K mental health care %K outpatient clinics %K wait time prediction %K machine learning %K random forest %K system’s knowledge %D 2022 %7 9.8.2022 %9 Original Paper %J JMIR Ment Health %G English %X Background: Wait times impact patient satisfaction, treatment effectiveness, and the efficiency of care that the patients receive. Wait time prediction in mental health is a complex task and is affected by the difficulty in predicting the required number of treatment sessions for outpatients, high no-show rates, and the possibility of using group treatment sessions. The task of wait time analysis becomes even more challenging if the input data has low utility, which happens when the data is highly deidentified by removing both direct and quasi identifiers. Objective: The first aim of this study was to develop machine learning models to predict the wait time from referral to the first appointment for psychiatric outpatients by using real-time data. The second aim was to enhance the performance of these predictive models by utilizing the system’s knowledge while the input data were highly deidentified. The third aim was to identify the factors that drove long wait times, and the fourth aim was to build these models such that they were practical and easy-to-implement (and therefore, attractive to care providers). Methods: We analyzed retrospective highly deidentified administrative data from 8 outpatient clinics at Ontario Shores Centre for Mental Health Sciences in Canada by using 6 machine learning methods to predict the first appointment wait time for new outpatients. We used the system’s knowledge to mitigate the low utility of our data. The data included 4187 patients who received care through 30,342 appointments. Results: The average wait time varied widely between different types of mental health clinics. For more than half of the clinics, the average wait time was longer than 3 months. The number of scheduled appointments and the rate of no-shows varied widely among clinics. Despite these variations, the random forest method provided the minimum root mean square error values for 4 of the 8 clinics, and the second minimum root mean square error for the other 4 clinics. Utilizing the system’s knowledge increased the utility of our highly deidentified data and improved the predictive power of the models. Conclusions: The random forest method, enhanced with the system’s knowledge, provided reliable wait time predictions for new outpatients, regardless of low utility of the highly deidentified input data and the high variation in wait times across different clinics and patient types. The priority system was identified as a factor that contributed to long wait times, and a fast-track system was suggested as a potential solution. %M 35943774 %R 10.2196/38428 %U https://mental.jmir.org/2022/8/e38428 %U https://doi.org/10.2196/38428 %U http://www.ncbi.nlm.nih.gov/pubmed/35943774 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 8 %P e38082 %T Predicting Mortality in Intensive Care Unit Patients With Heart Failure Using an Interpretable Machine Learning Model: Retrospective Cohort Study %A Li,Jili %A Liu,Siru %A Hu,Yundi %A Zhu,Lingfeng %A Mao,Yujia %A Liu,Jialin %+ Department of Medical Informatics, West China Hospital, Sichuan University, No 37 Guoxue Road, Chengdu, 610041, China, 86 28 85422306, dljl8@163.com %K heart failure %K mortality %K intensive care unit %K prediction %K XGBoost %K SHAP %K SHapley Additive exPlanation %D 2022 %7 9.8.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Heart failure (HF) is a common disease and a major public health problem. HF mortality prediction is critical for developing individualized prevention and treatment plans. However, due to their lack of interpretability, most HF mortality prediction models have not yet reached clinical practice. Objective: We aimed to develop an interpretable model to predict the mortality risk for patients with HF in intensive care units (ICUs) and used the SHapley Additive exPlanation (SHAP) method to explain the extreme gradient boosting (XGBoost) model and explore prognostic factors for HF. Methods: In this retrospective cohort study, we achieved model development and performance comparison on the eICU Collaborative Research Database (eICU-CRD). We extracted data during the first 24 hours of each ICU admission, and the data set was randomly divided, with 70% used for model training and 30% used for model validation. The prediction performance of the XGBoost model was compared with three other machine learning models by the area under the curve. We used the SHAP method to explain the XGBoost model. Results: A total of 2798 eligible patients with HF were included in the final cohort for this study. The observed in-hospital mortality of patients with HF was 9.97%. Comparatively, the XGBoost model had the highest predictive performance among four models with an area under the curve (AUC) of 0.824 (95% CI 0.7766-0.8708), whereas support vector machine had the poorest generalization ability (AUC=0.701, 95% CI 0.6433-0.7582). The decision curve showed that the net benefit of the XGBoost model surpassed those of other machine learning models at 10%~28% threshold probabilities. The SHAP method reveals the top 20 predictors of HF according to the importance ranking, and the average of the blood urea nitrogen was recognized as the most important predictor variable. Conclusions: The interpretable predictive model helps physicians more accurately predict the mortality risk in ICU patients with HF, and therefore, provides better treatment plans and optimal resource allocation for their patients. In addition, the interpretable framework can increase the transparency of the model and facilitate understanding the reliability of the predictive model for the physicians. %M 35943767 %R 10.2196/38082 %U https://www.jmir.org/2022/8/e38082 %U https://doi.org/10.2196/38082 %U http://www.ncbi.nlm.nih.gov/pubmed/35943767 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 8 %P e34126 %T A Questionnaire-Based Ensemble Learning Model to Predict the Diagnosis of Vertigo: Model Development and Validation Study %A Yu,Fangzhou %A Wu,Peixia %A Deng,Haowen %A Wu,Jingfang %A Sun,Shan %A Yu,Huiqian %A Yang,Jianming %A Luo,Xianyang %A He,Jing %A Ma,Xiulan %A Wen,Junxiong %A Qiu,Danhong %A Nie,Guohui %A Liu,Rizhao %A Hu,Guohua %A Chen,Tao %A Zhang,Cheng %A Li,Huawei %+ Department of Otorhinolaryngology, Eye & ENT Hospital, Fudan University, Room 611, Building 9, No. 83, Fenyang Road, Xuhui District, Shanghai, 200031, China, 86 021 64377134 ext 2669, hwli@shmu.edu.cn %K vestibular disorders %K machine learning %K diagnostic model %K vertigo %K ENT %K questionnaire %D 2022 %7 3.8.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Questionnaires have been used in the past 2 decades to predict the diagnosis of vertigo and assist clinical decision-making. A questionnaire-based machine learning model is expected to improve the efficiency of diagnosis of vestibular disorders. Objective: This study aims to develop and validate a questionnaire-based machine learning model that predicts the diagnosis of vertigo. Methods: In this multicenter prospective study, patients presenting with vertigo entered a consecutive cohort at their first visit to the ENT and vertigo clinics of 7 tertiary referral centers from August 2019 to March 2021, with a follow-up period of 2 months. All participants completed a diagnostic questionnaire after eligibility screening. Patients who received only 1 final diagnosis by their treating specialists for their primary complaint were included in model development and validation. The data of patients enrolled before February 1, 2021 were used for modeling and cross-validation, while patients enrolled afterward entered external validation. Results: A total of 1693 patients were enrolled, with a response rate of 96.2% (1693/1760). The median age was 51 (IQR 38-61) years, with 991 (58.5%) females; 1041 (61.5%) patients received the final diagnosis during the study period. Among them, 928 (54.8%) patients were included in model development and validation, and 113 (6.7%) patients who enrolled later were used as a test set for external validation. They were classified into 5 diagnostic categories. We compared 9 candidate machine learning methods, and the recalibrated model of light gradient boosting machine achieved the best performance, with an area under the curve of 0.937 (95% CI 0.917-0.962) in cross-validation and 0.954 (95% CI 0.944-0.967) in external validation. Conclusions: The questionnaire-based light gradient boosting machine was able to predict common vestibular disorders and assist decision-making in ENT and vertigo clinics. Further studies with a larger sample size and the participation of neurologists will help assess the generalization and robustness of this machine learning method. %M 35921135 %R 10.2196/34126 %U https://www.jmir.org/2022/8/e34126 %U https://doi.org/10.2196/34126 %U http://www.ncbi.nlm.nih.gov/pubmed/35921135 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 7 %P e32925 %T Hemodialysis Record Sharing: Solution for Work Burden Reduction and Disaster Preparedness %A Ido,Keisuke %A Miyazaki,Mariko %A Nakayama,Masaharu %+ Department of Medical Informatics, Tohoku University Graduate School of Medicine, 2-1 Seiryo-machi, Aoba-ku, Sendai, 9808575, Japan, 81 227177572, nakayama@cardio.med.tohoku.ac.jp %K hemodialysis %K electronic health record %K EHR %K information sharing %K information exchange %K data sharing %K document sharing %K health information exchange %K disaster %K work burden %K clinical information %K clinical data %K clinical report %K medical report %K information network %K medical informatics %K renal failure %K kidney %K renal %K clinical record %K medical record %K backup %K data security %K data protection %K data recovery %D 2022 %7 22.7.2022 %9 Original Paper %J JMIR Form Res %G English %X Background: After the Great East Japan Earthquake in 2011, backup systems for clinical information were launched in Japan. The system in Miyagi Prefecture called the Miyagi Medical and Welfare Information Network (MMWIN) is used as a health information exchange network to share clinical information among various medical facilities for patients who have opted in. Hospitals and clinics specializing in chronic renal failure require patients’ data and records during hemodialysis to facilitate communication in daily clinical activity and preparedness for disasters. Objective: This study aimed to facilitate the sharing of clinical data of patients undergoing hemodialysis among different hemodialysis facilities. Methods: We introduced a document-sharing system to make hemodialysis reports available on the MMWIN. We also recruited hospitals and clinics to share the hemodialysis reports of their patients and promoted the development of a network between emergency and dialysis clinics. Results: In addition to basic patient information as well as information on diagnosis, prescription, laboratory data, hospitalization, allergy, and image data from different facilities, specific information about hemodialysis is available, as well as a backup of indispensable information in preparation for disasters. As of June 1, 2021, 12 clinics and 10 hospitals of 68 dialysis facilities in Miyagi participated in the MMWIN. The number of patients who underwent hemodialysis in Miyagi increased by more than 40%. Conclusions: Our backup system successfully developed a network of hemodialysis facilities. We have accumulated data that are beneficial to prevent the fragmentation of patient information and would be helpful in transferring patients efficiently during unpredictable disasters. %M 35867394 %R 10.2196/32925 %U https://formative.jmir.org/2022/7/e32925 %U https://doi.org/10.2196/32925 %U http://www.ncbi.nlm.nih.gov/pubmed/35867394 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 7 %P e37301 %T Experimental Implementation of NSER Mobile App for Efficient Real-Time Sharing of Prehospital Patient Information With Emergency Departments: Interrupted Time-Series Analysis %A Fukaguchi,Kiyomitsu %A Goto,Tadahiro %A Yamamoto,Tadatsugu %A Yamagami,Hiroshi %+ Department of Emergency Medicine, Shonan Kamakura General Hospital, 1370-1, Okamoto, Kamakura-shi, Kanagawa, 247-0072, Japan, 81 467 46 1717, fukaskgh@gmail.com %K emergency department %K emergency medical services %K mobile apps %K interrupted time series analysis %K emergency %K patient record %K implementation %K patient care %K app %K implement %K medical informatics %K clinical informatics %K decision support %K electronic health record %K eHealth %K digital health %D 2022 %7 6.7.2022 %9 Original Paper %J JMIR Form Res %G English %X Background: With the aging society, the number of emergency transportations has been growing. Although it is important that a patient be immediately transported to an appropriate hospital for proper management, accurate diagnosis in the prehospital setting is challenging. However, at present, patient information is mainly communicated by telephone, which has a potential risk of communication errors such as mishearing. Sharing correct and detailed prehospital information with emergency departments (EDs) should facilitate optimal patient care and resource use. Therefore, the implementation of an app that provides on-site, real-time information to emergency physicians could be useful for early preparation, intervention, and effective use of medical and human resources. Objective: In this paper, we aimed to examine whether the implementation of a mobile app for emergency medical service (EMS) would improve patient outcomes and reduce transportation time as well as communication time by phone (ie, phone-communication time). Methods: We performed an interrupted time-series analysis (ITSA) on the data from a tertiary care hospital in Japan from July 2021 to October 2021 (8 weeks before and 8 weeks after the implementation period). We included all patients transported by EMS. Using the mobile app, EMS can send information on patient demographics, vital signs, medications, and photos of the scene to the ED. The outcome measure was inpatient mortality and transportation time, as well as phone-communication time, which was the time for EMS to negotiate with ED staffs for transport requests. Results: During the study period, 1966 emergency transportations were made (n=1033, 53% patients during the preimplementation period and n=933, 47% patients after the implementation period). The ITSA did not reveal a significant decrease in patient mortality and transportation time before and after the implementation. However, the ITSA revealed a significant decrease in mean phone-communication time between pre- and postimplementation periods (from 216 to 171 seconds; −45 seconds; 95% CI −71 to −18 seconds). From the pre- to postimplementation period, the mean transportation time from EMS request to ED arrival decreased by 0.29 minutes (from 36.1 minutes to 35.9 minutes; 95% CI −2.20 to 1.60 minutes), without change in time trends. We also introduced cases where the app allowed EMS to share accurate and detailed prehospital information with the emergency department, resulting in timely intervention and reducing the burden on the ED. Conclusions: The implementation of a mobile app for EMS was associated with reduced phone-communication time by 45 seconds (22%) without increasing mortality or overall transportation time despite the implementation of new methods in the real clinical setting. In addition, real-time patient information sharing, such as the transfer of monitor images and photos of the accident site, could facilitate optimal patient care and resource use. %M 35793142 %R 10.2196/37301 %U https://formative.jmir.org/2022/7/e37301 %U https://doi.org/10.2196/37301 %U http://www.ncbi.nlm.nih.gov/pubmed/35793142 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 6 %P e37557 %T Automatic International Classification of Diseases Coding System: Deep Contextualized Language Model With Rule-Based Approaches %A Chen,Pei-Fu %A Chen,Kuan-Chih %A Liao,Wei-Chih %A Lai,Feipei %A He,Tai-Liang %A Lin,Sheng-Che %A Chen,Wei-Jen %A Yang,Chi-Yu %A Lin,Yu-Cheng %A Tsai,I-Chang %A Chiu,Chi-Hao %A Chang,Shu-Chih %A Hung,Fang-Ming %+ Department of Medical Affairs, Far Eastern Memorial Hospital, No. 21, Sec. 2, Nanya S. Rd., Banciao Dist., New Taipei City, 220216, Taiwan, 886 2 8966 7000, drphilip101@gmail.com %K deep learning %K International Classification of Diseases %K medical records %K multilabel text classification %K natural language processing %K coding system %K algorithm %K electronic health record %K data mining %D 2022 %7 29.6.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: The tenth revision of the International Classification of Diseases (ICD-10) is widely used for epidemiological research and health management. The clinical modification (CM) and procedure coding system (PCS) of ICD-10 were developed to describe more clinical details with increasing diagnosis and procedure codes and applied in disease-related groups for reimbursement. The expansion of codes made the coding time-consuming and less accurate. The state-of-the-art model using deep contextual word embeddings was used for automatic multilabel text classification of ICD-10. In addition to input discharge diagnoses (DD), the performance can be improved by appropriate preprocessing methods for the text from other document types, such as medical history, comorbidity and complication, surgical method, and special examination. Objective: This study aims to establish a contextual language model with rule-based preprocessing methods to develop the model for ICD-10 multilabel classification. Methods: We retrieved electronic health records from a medical center. We first compared different word embedding methods. Second, we compared the preprocessing methods using the best-performing embeddings. We compared biomedical bidirectional encoder representations from transformers (BioBERT), clinical generalized autoregressive pretraining for language understanding (Clinical XLNet), label tree-based attention-aware deep model for high-performance extreme multilabel text classification (AttentionXLM), and word-to-vector (Word2Vec) to predict ICD-10-CM. To compare different preprocessing methods for ICD-10-CM, we included DD, medical history, and comorbidity and complication as inputs. We compared the performance of ICD-10-CM prediction using different preprocesses, including definition training, external cause code removal, number conversion, and combination code filtering. For the ICD-10 PCS, the model was trained using different combinations of DD, surgical method, and key words of special examination. The micro F1 score and the micro area under the receiver operating characteristic curve were used to compare the model’s performance with that of different preprocessing methods. Results: BioBERT had an F1 score of 0.701 and outperformed other models such as Clinical XLNet, AttentionXLM, and Word2Vec. For the ICD-10-CM, the model had an F1 score that significantly increased from 0.749 (95% CI 0.744-0.753) to 0.769 (95% CI 0.764-0.773) with the ICD-10 definition training, external cause code removal, number conversion, and combination code filter. For the ICD-10-PCS, the model had an F1 score that significantly increased from 0.670 (95% CI 0.663-0.678) to 0.726 (95% CI 0.719-0.732) with a combination of discharge diagnoses, surgical methods, and key words of special examination. With our preprocessing methods, the model had the highest area under the receiver operating characteristic curve of 0.853 (95% CI 0.849-0.855) and 0.831 (95% CI 0.827-0.834) for ICD-10-CM and ICD-10-PCS, respectively. Conclusions: The performance of our model with the pretrained contextualized language model and rule-based preprocessing method is better than that of the state-of-the-art model for ICD-10-CM or ICD-10-PCS. This study highlights the importance of rule-based preprocessing methods based on coder coding rules. %M 35767353 %R 10.2196/37557 %U https://medinform.jmir.org/2022/6/e37557 %U https://doi.org/10.2196/37557 %U http://www.ncbi.nlm.nih.gov/pubmed/35767353 %0 Journal Article %@ 2561-6722 %I JMIR Publications %V 5 %N 2 %P e36878 %T Electronic Discharge Communication Tools Used in Pediatric Emergency Departments: Systematic Review %A Wozney,Lori %A Curran,Janet %A Archambault,Patrick %A Cassidy,Christine %A Jabbour,Mona %A Mackay,Rebecca %A Newton,Amanda %A Plint,Amy C %A Somerville,Mari %+ Mental Health and Addictions, Nova Scotia Health, 300 Pleasant St., Dartmouth, NS, B2Y 3Z9, Canada, 1 902 449 0603, loriwozney@gmail.com %K emergency department %K medical informatics %K pediatric %K systematic review %K patient discharge summaries %K patient-centered care %K technology %K hospital %D 2022 %7 24.6.2022 %9 Original Paper %J JMIR Pediatr Parent %G English %X Background: Electronic discharge communication tools (EDCTs) are increasingly common in pediatric emergency departments (EDs). These tools have been shown to improve patient-centered communication, support postdischarge care at home, and reduce unnecessary return visits to the ED. Objective: This study aimed to map and assess the evidence base for EDCTs used in pediatric EDs according to their functionalities, intended purpose, implementation context features, and outcomes. Methods: A systematic review was conducted following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) procedures for identification, screening, and eligibility. A total of 7 databases (EBSCO, MEDLINE, CINAHL, PsycINFO, EMBASE Scopus, and Web of Science) were searched for studies published between 1989 and 2021. Studies evaluating discharge communication–related outcomes using electronic tools (eg, text messages, videos, and kiosks) in pediatric EDs were included. In all, 2 researchers independently assessed the eligibility. Extracted data related to study identification, methodology, settings and demographics, intervention features, outcome implementation features, and practice, policy, and research implications. The Mixed Method Appraisal Tool was used to assess methodological quality. The synthesis of results involved structured tabulation, vote counting, recoding into common metrics, inductive thematic analysis, descriptive statistics, and heat mapping. Results: In total, 231 full-text articles and abstracts were screened for review inclusion with 49 reports (representing 55 unique tools) included. In all, 70% (26/37) of the studies met at least three of five Mixed Method Appraisal Tool criteria. The most common EDCTs were videos, text messages, kiosks, and phone calls. The time required to use the tools ranged from 120 seconds to 80 minutes. The EDCTs were evaluated for numerous presenting conditions (eg, asthma, fracture, head injury, fever, and otitis media) that required a range of at-home care needs after the ED visit. The most frequently measured outcomes were knowledge acquisition, caregiver and patient beliefs and attitudes, and health service use. Unvalidated self-report measures were typically used for measurement. Health care provider satisfaction or system-level impacts were infrequently measured in studies. The directionality of primary outcomes pointed to positive effects for the primary measure (44/55, 80%) or no significant difference (10/55, 18%). Only one study reported negative findings, with an increase in return visits to the ED after receiving the intervention compared with the control group. Conclusions: This review is the first to map the broad literature of EDCTs used in pediatric EDs. The findings suggest a promising evidence base, demonstrating that EDCTs have been successfully integrated across clinical contexts and deployed via diverse technological modalities. Although caregiver and patient satisfaction with EDCTs is high, future research should use robust trials using consistent measures of communication quality, clinician experience, cost-effectiveness, and health service use to accumulate evidence regarding these outcomes. Trial Registration: PROSPERO CRD42020157500; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=157500 %M 35608929 %R 10.2196/36878 %U https://pediatrics.jmir.org/2022/2/e36878 %U https://doi.org/10.2196/36878 %U http://www.ncbi.nlm.nih.gov/pubmed/35608929 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 6 %P e33446 %T Medication Use and Clinical Outcomes by the Dutch Institute for Clinical Auditing Medicines Program: Quantitative Analysis %A Ismail,Rawa Kamaran %A van Breeschoten,Jesper %A van der Flier,Silvia %A van Loosen,Caspar %A Pasmooij,Anna Maria Gerdina %A van Dartel,Maaike %A van den Eertwegh,Alfons %A de Boer,Anthonius %A Wouters,Michel %A Hilarius,Doranne %+ Dutch Institute for Clinical Auditing, Rijnsburgerweg 10, Leiden, 2333AA, Netherlands, 31 613382358, j.vanbreeschoten@dica.nl %K real-world data %K quality of care %K medicines %K cancer %D 2022 %7 23.6.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: The Dutch Institute for Clinical Auditing (DICA) Medicines Program was set up in September 2018 to evaluate expensive medicine use in daily practice in terms of real-world effectiveness using only existing data sources. Objective: The aim of this study is to describe the potential of the addition of declaration data to quality registries to provide participating centers with benchmark information about the use of medicines and outcomes among patients. Methods: A total of 3 national population-based registries were linked to clinical and financial data from the hospital pharmacy, the Dutch diagnosis treatment combinations information system including in-hospital activities, and survival data from health care insurers. The first results of the real-world data (RWD) linkage are presented using descriptive statistics to assess patient, tumor, and treatment characteristics. Time-to-next-treatment (TTNT) and overall survival (OS) were estimated using the Kaplan-Meier method. Results: A total of 21 Dutch hospitals participated in the DICA Medicines Program, which included 7412 patients with colorectal cancer, 1981 patients with metastasized colon cancer, 3860 patients with lung cancer, 1253 patients with metastasized breast cancer, and 7564 patients with rheumatic disease. The data were used for hospital benchmarking to gain insights into medication use in specific patient populations, treatment information, clinical outcomes, and costs. Detailed treatment information (duration and treatment steps) led to insights into differences between hospitals in daily clinical practices. Furthermore, exploratory analyses on clinical outcomes (TTNT and OS) were possible. Conclusions: The DICA Medicines Program shows that it is possible to gather and link RWD about medicines to 4 disease-specific population-based registries. Since these RWD became available with minimal registration burden and effort for hospitals, this method can be explored in other population-based registries to evaluate real-world efficacy. %M 35737449 %R 10.2196/33446 %U https://www.jmir.org/2022/6/e33446 %U https://doi.org/10.2196/33446 %U http://www.ncbi.nlm.nih.gov/pubmed/35737449 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 6 %P e34753 %T A Clinical Decision Support System for Assessing the Risk of Cervical Cancer: Development and Evaluation Study %A Chekin,Nasrin %A Ayatollahi,Haleh %A Karimi Zarchi,Mojgan %+ Health Management and Economics Research Center, Health Management Research Institute, Iran University of Medical Sciences, No 4, Yasemi St, Vali-e-Asr St, Tehran, 1996713883, Iran, 98 2188794301, ayatollahi.h@iums.ac.ir %K cervical cancer %K clinical decision support system %K risk assessment %K medical informatics %K cancer %K oncology %K decision support %K risk %K CDSS %K cervical %K prototype %K evaluation %K testing %D 2022 %7 22.6.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Cervical cancer has been recognized as a preventable type of cancer. As the assessment of all the risk factors of a disease is challenging for physicians, information technology and risk assessment models have been used to estimate the degree of risk. Objective: The aim of this study was to develop a clinical decision support system to assess the risk of cervical cancer. Methods: This study was conducted in 2 phases in 2021. In the first phase of the study, 20 gynecologists completed a questionnaire to determine the essential parameters for assessing the risk of cervical cancer, and the data were analyzed using descriptive statistics. In the second phase of the study, the prototype of the clinical decision support system was developed and evaluated. Results: The findings revealed that the most important parameters for assessing the risk of cervical cancer consisted of general and specific parameters. In total, the 8 parameters that had the greatest impact on the risk of cervical cancer were selected. After developing the clinical decision support system, it was evaluated and the mean values of sensitivity, specificity, and accuracy were 85.81%, 93.82%, and 91.39%, respectively. Conclusions: The clinical decision support system developed in this study can facilitate the process of identifying people who are at risk of developing cervical cancer. In addition, it can help to increase the quality of health care and reduce the costs associated with the treatment of cervical cancer. %M 35731549 %R 10.2196/34753 %U https://medinform.jmir.org/2022/6/e34753 %U https://doi.org/10.2196/34753 %U http://www.ncbi.nlm.nih.gov/pubmed/35731549 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 6 %P e34141 %T Finding Primary Care—Repurposing Physician Registration Data to Generate a Regionally Accurate List of Primary Care Clinics: Development and Validation of an Open-Source Algorithm %A Cooper,Ian R %A Lindsay,Cameron %A Fraser,Keaton %A Hill,Tiffany T %A Siu,Andrew %A Fletcher,Sarah %A Klimas,Jan %A Hamilton,Michee-Ana %A Frazer,Amanda D %A Humphrys,Elka %A Koepke,Kira %A Hedden,Lindsay %A Price,Morgan %A McCracken,Rita K %+ Innovation Support Unit, Department of Family Practice, University of British Columbia, 3rd Floor David Strangway Building, Vancouver, BC, V6T 1Z3, Canada, 1 604 827 4168, rita.mccracken@ubc.ca %K physicians, primary care %K primary health care %K health services accessibility %K practice patterns, physicians %K physicians’ offices %K computing methodologies %K algorithms %D 2022 %7 22.6.2022 %9 Original Paper %J JMIR Form Res %G English %X Background: Some Canadians have limited access to longitudinal primary care, despite its known advantages for population health. Current initiatives to transform primary care aim to increase access to team-based primary care clinics. However, many regions lack a reliable method to enumerate clinics, limiting estimates of clinical capacity and ongoing access gaps. A region-based complete clinic list is needed to effectively describe clinic characteristics and to compare primary care outcomes at the clinic level. Objective: The objective of this study is to show how publicly available data sources, including the provincial physician license registry, can be used to generate a verifiable, region-wide list of primary care clinics in British Columbia, Canada, using a process named the Clinic List Algorithm (CLA). Methods: The CLA has 10 steps: (1) collect data sets, (2) develop clinic inclusion and exclusion criteria, (3) process data sets, (4) consolidate data sets, (5) transform from list of physicians to initial list of clinics, (6) add additional metadata, (7) create working lists, (8) verify working lists, (9) consolidate working lists, and (10) adjust processing steps based on learnings. Results: The College of Physicians and Surgeons of British Columbia Registry contained 13,726 physicians, at 2915 unique addresses, 6942 (50.58%) of whom were family physicians (FPs) licensed to practice in British Columbia. The CLA identified 1239 addresses where primary care was delivered by 4262 (61.39%) FPs. Of the included addresses, 84.50% (n=1047) were in urban locations, and there was a median of 2 (IQR 2-4, range 1-23) FPs at each unique address. Conclusions: The CLA provides a region-wide description of primary care clinics that improves on simple counts of primary care providers or self-report lists. It identifies the number and location of primary care clinics and excludes primary care providers who are likely not providing community-based primary care. Such information may be useful for estimates of capacity of primary care, as well as for policy planning and research in regions engaged in primary care evaluation or transformation. %M 35731556 %R 10.2196/34141 %U https://formative.jmir.org/2022/6/e34141 %U https://doi.org/10.2196/34141 %U http://www.ncbi.nlm.nih.gov/pubmed/35731556 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 6 %P e35717 %T A Scalable Risk-Scoring System Based on Consumer-Grade Wearables for Inpatients With COVID-19: Statistical Analysis and Model Development %A Föll,Simon %A Lison,Adrian %A Maritsch,Martin %A Klingberg,Karsten %A Lehmann,Vera %A Züger,Thomas %A Srivastava,David %A Jegerlehner,Sabrina %A Feuerriegel,Stefan %A Fleisch,Elgar %A Exadaktylos,Aristomenis %A Wortmann,Felix %+ Department of Emergency Medicine, Inselspital, Bern, University Hospital, University of Bern, Freiburgstrasse 16C, Bern, 3010, Switzerland, 41 31632244, Aristomenis.Exadaktylos@insel.ch %K COVID-19 %K risk scoring %K wearable devices %K wearable %K smartwatches %K smartwatch %K Bayesian survival analysis %K remote monitoring %K patient monitoring %K remote patient monitoring %K smart device %K digital health %K risk score %K scalable %K general ward %K hospital %K measurement tool %K measurement instrument %D 2022 %7 21.6.2022 %9 Original Paper %J JMIR Form Res %G English %X Background: To provide effective care for inpatients with COVID-19, clinical practitioners need systems that monitor patient health and subsequently allow for risk scoring. Existing approaches for risk scoring in patients with COVID-19 focus primarily on intensive care units (ICUs) with specialized medical measurement devices but not on hospital general wards. Objective: In this paper, we aim to develop a risk score for inpatients with COVID-19 in general wards based on consumer-grade wearables (smartwatches). Methods: Patients wore consumer-grade wearables to record physiological measurements, such as the heart rate (HR), heart rate variability (HRV), and respiration frequency (RF). Based on Bayesian survival analysis, we validated the association between these measurements and patient outcomes (ie, discharge or ICU admission). To build our risk score, we generated a low-dimensional representation of the physiological features. Subsequently, a pooled ordinal regression with time-dependent covariates inferred the probability of either hospital discharge or ICU admission. We evaluated the predictive performance of our developed system for risk scoring in a single-center, prospective study based on 40 inpatients with COVID-19 in a general ward of a tertiary referral center in Switzerland. Results: First, Bayesian survival analysis showed that physiological measurements from consumer-grade wearables are significantly associated with patient outcomes (ie, discharge or ICU admission). Second, our risk score achieved a time-dependent area under the receiver operating characteristic curve (AUROC) of 0.73-0.90 based on leave-one-subject-out cross-validation. Conclusions: Our results demonstrate the effectiveness of consumer-grade wearables for risk scoring in inpatients with COVID-19. Due to their low cost and ease of use, consumer-grade wearables could enable a scalable monitoring system. Trial Registration: Clinicaltrials.gov NCT04357834; https://www.clinicaltrials.gov/ct2/show/NCT04357834 %M 35613417 %R 10.2196/35717 %U https://formative.jmir.org/2022/6/e35717 %U https://doi.org/10.2196/35717 %U http://www.ncbi.nlm.nih.gov/pubmed/35613417 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 6 %P e35317 %T Integration vs Collaborative Redesign Strategies of Health Systems’ Supply Chains in the Post-COVID-19 New Normal: Cross-sectional Survey Across the United States %A Khuntia,Jiban %A Mejia,Frances J %A Ning,Xue %A Helton,Jeff %A Stacey,Rulon %+ Health Administration Research Consortium, University of Colorado Denver, 1475 Lawrence St., Denver, CO, 80202, United States, 1 303 315 8424, jiban.khuntia@ucdenver.edu %K COVID-19 %K post-COVID-19 %K health systems %K supply chain integration %K supply chain collaboration %K supply chain resilience %D 2022 %7 15.6.2022 %9 Original Paper %J JMIR Form Res %G English %X Background: Given the widespread disruptions to supply chains in 2020 because of the COVID-19 pandemic, questions such as how health systems are shaping strategies to restore the supply chain disruptions are essential to have confidence in health systems’ supply chain model strategies. Plausibly, health systems have an opportunity for redesign, growth, and innovation by utilizing collaborative strategies now, compared to the usual strategies of integrating their existing supply chains to reduce inefficiencies. Objective: This study focuses on teasing out the nuance of supply chain integration versus collaborative redesign strategies for health systems in the post-COVID-19 new normal. We focus on 2 research questions. First, we explore the impact of perceived supply chain challenges and disruptions on health systems’ supply chain integration (SC-INTEGRATION) and collaborative redesign (SC-REDESIGN) strategies. Second, we examine the outcomes of integration and collaborative redesign strategic choices on growth and service outcomes. Methods: We used data for this study collected through a consultant from a robust group of health system chief executive officers (CEOs) across the United States from February to March 2021. Among the 625 health system CEOs contacted, 135 (21.6%) responded to our survey. We considered supply chain–relevant strategy and outcome variables from the literature and ratified them via expert consensus. We collected secondary data from the Agency for Healthcare Research and Quality (AHRQ) Compendium of the US Health Systems, leading to a matched data set from the 124 health systems. Next, we used ordered logit model estimation to examine CEO preferences for partnership strategies to address current supply disruptions and the outcomes of strategy choices. Results: Health systems with higher disruptions would choose integration (positive, P<.001) over redesign, indicating that they still trust the existing partners. Integration strategy is perceived to result in better service outcomes (P<.01), while collaborations are perceived to lead to greater growth opportunities (P<.05); however, the role of integration in growth is not entirely ruled out (combined model, P<.001). Plausibly, some health systems would choose integration and collaborative redesign models, which have a significant relationship with both services (combined model, P<.01) and growth, establishing the importance of mixed strategies for health systems. Conclusions: The cost of health care continues to rise, and supply-related costs constitute a large portion of a hospital’s expenditure. Understanding supply chain strategic choices are essential for a health system’s success. Although collaboration is an option, focusing on and improving existing integration dynamics is helpful to foster both growth and services for health systems. %M 35452405 %R 10.2196/35317 %U https://formative.jmir.org/2022/6/e35317 %U https://doi.org/10.2196/35317 %U http://www.ncbi.nlm.nih.gov/pubmed/35452405 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e35032 %T An Electronic Data Capture Tool for Data Collection During Public Health Emergencies: Development and Usability Study %A Brown,Joan %A Bhatnagar,Manas %A Gordon,Hugh %A Goodner,Jared %A Cobb,J Perren %A Lutrick,Karen %+ Clinical Operations Business Intelligence, The Keck School of Medicine of the University of Southern California, 1520 San Pablo St, Los Angeles, CA, 90033, United States, 1 310 245 8079, joancbrown@gmail.com %K clinical research design %K disaster management %K informatics %K public health emergencies %K electronic data capture %K design tenet %K public health emergency %K electronic data %K EDCT %K real time data %D 2022 %7 9.6.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The Discovery Critical Care Research Network Program for Resilience and Emergency Preparedness (Discovery PREP) partnered with a third-party technology vendor to design and implement an electronic data capture tool that addressed multisite data collection challenges during public health emergencies (PHE) in the United States. The basis of the work was to design an electronic data capture tool and to prospectively gather data on usability from bedside clinicians during national health system stress queries and influenza observational studies. Objective: The aim of this paper is to describe the lessons learned in the design and implementation of a novel electronic data capture tool with the goal of significantly increasing the nation’s capability to manage real-time data collection and analysis during PHE. Methods: A multiyear and multiphase design approach was taken to create an electronic data capture tool, which was used to pilot rapid data capture during a simulated PHE. Following the pilot, the study team retrospectively assessed the feasibility of automating the data captured by the electronic data capture tool directly from the electronic health record. In addition to user feedback during semistructured interviews, the System Usability Scale (SUS) questionnaire was used as a basis to evaluate the usability and performance of the electronic data capture tool. Results: Participants included Discovery PREP physicians, their local administrators, and data collectors from tertiary-level academic medical centers at 5 different institutions. User feedback indicated that the designed system had an intuitive user interface and could be used to automate study communication tasks making for more efficient management of multisite studies. SUS questionnaire results classified the system as highly usable (SUS score 82.5/100). Automation of 17 (61%) of the 28 variables in the influenza observational study was deemed feasible during the exploration of automated versus manual data abstraction. The creation and use of the Project Meridian electronic data capture tool identified 6 key design requirements for multisite data collection, including the need for the following: (1) scalability irrespective of the type of participant; (2) a common data set across sites; (3) automated back end administrative capability (eg, reminders and a self-service status board); (4) multimedia communication pathways (eg, email and SMS text messaging); (5) interoperability and integration with local site information technology infrastructure; and (6) natural language processing to extract nondiscrete data elements. Conclusions: The use of the electronic data capture tool in multiple multisite Discovery PREP clinical studies proved the feasibility of using the novel, cloud-based platform in practice. The lessons learned from this effort can be used to inform the improvement of ongoing global multisite data collection efforts during the COVID-19 pandemic and transform current manual data abstraction approaches into reliable, real time, and automated information exchange. Future research is needed to expand the ability to perform automated multisite data extraction during a PHE and beyond. %M 35679114 %R 10.2196/35032 %U https://humanfactors.jmir.org/2022/2/e35032 %U https://doi.org/10.2196/35032 %U http://www.ncbi.nlm.nih.gov/pubmed/35679114 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 6 %P e34295 %T Machine Learning–Based Prediction Models for Different Clinical Risks in Different Hospitals: Evaluation of Live Performance %A Sun,Hong %A Depraetere,Kristof %A Meesseman,Laurent %A Cabanillas Silva,Patricia %A Szymanowsky,Ralph %A Fliegenschmidt,Janis %A Hulde,Nikolai %A von Dossow,Vera %A Vanbiervliet,Martijn %A De Baerdemaeker,Jos %A Roccaro-Waldmeyer,Diana M %A Stieg,Jörg %A Domínguez Hidalgo,Manuel %A Dahlweid,Fried-Michael %+ Dedalus Healthcare, Roderveldlaan 2, Antwerp, 2600, Belgium, 32 3444 8108, hong.sun@dedalus.com %K machine learning %K clinical risk prediction %K prediction %K model %K model evaluation %K scalability %K risk %K live clinical workflow %K delirium %K sepsis %K acute kidney injury %K kidney %K EHR %K electronic health record %K workflow %K algorithm %D 2022 %7 7.6.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Machine learning algorithms are currently used in a wide array of clinical domains to produce models that can predict clinical risk events. Most models are developed and evaluated with retrospective data, very few are evaluated in a clinical workflow, and even fewer report performances in different hospitals. In this study, we provide detailed evaluations of clinical risk prediction models in live clinical workflows for three different use cases in three different hospitals. Objective: The main objective of this study was to evaluate clinical risk prediction models in live clinical workflows and compare their performance in these setting with their performance when using retrospective data. We also aimed at generalizing the results by applying our investigation to three different use cases in three different hospitals. Methods: We trained clinical risk prediction models for three use cases (ie, delirium, sepsis, and acute kidney injury) in three different hospitals with retrospective data. We used machine learning and, specifically, deep learning to train models that were based on the Transformer model. The models were trained using a calibration tool that is common for all hospitals and use cases. The models had a common design but were calibrated using each hospital’s specific data. The models were deployed in these three hospitals and used in daily clinical practice. The predictions made by these models were logged and correlated with the diagnosis at discharge. We compared their performance with evaluations on retrospective data and conducted cross-hospital evaluations. Results: The performance of the prediction models with data from live clinical workflows was similar to the performance with retrospective data. The average value of the area under the receiver operating characteristic curve (AUROC) decreased slightly by 0.6 percentage points (from 94.8% to 94.2% at discharge). The cross-hospital evaluations exhibited severely reduced performance: the average AUROC decreased by 8 percentage points (from 94.2% to 86.3% at discharge), which indicates the importance of model calibration with data from the deployment hospital. Conclusions: Calibrating the prediction model with data from different deployment hospitals led to good performance in live settings. The performance degradation in the cross-hospital evaluation identified limitations in developing a generic model for different hospitals. Designing a generic process for model development to generate specialized prediction models for each hospital guarantees model performance in different hospitals. %M 35502887 %R 10.2196/34295 %U https://www.jmir.org/2022/6/e34295 %U https://doi.org/10.2196/34295 %U http://www.ncbi.nlm.nih.gov/pubmed/35502887 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 6 %P e37213 %T Deep Phenotyping of Chinese Electronic Health Records by Recognizing Linguistic Patterns of Phenotypic Narratives With a Sequence Motif Discovery Tool: Algorithm Development and Validation %A Li,Shicheng %A Deng,Lizong %A Zhang,Xu %A Chen,Luming %A Yang,Tao %A Qi,Yifan %A Jiang,Taijiao %+ Institute of Systems Medicine, Chinese Academy of Medical Sciences & Peking Union Medical College, No.100 Chongwen Road, Suzhou, Jiangsu Province, China, Beijing, 215000, China, 86 13366191184, taijiaobioinfor@ism.cams.cn %K deep phenotyping %K Chinese EHRs %K linguistic pattern %K motif discovery %K pattern recognition %D 2022 %7 3.6.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Phenotype information in electronic health records (EHRs) is mainly recorded in unstructured free text, which cannot be directly used for clinical research. EHR-based deep-phenotyping methods can structure phenotype information in EHRs with high fidelity, making it the focus of medical informatics. However, developing a deep-phenotyping method for non-English EHRs (ie, Chinese EHRs) is challenging. Although numerous EHR resources exist in China, fine-grained annotation data that are suitable for developing deep-phenotyping methods are limited. It is challenging to develop a deep-phenotyping method for Chinese EHRs in such a low-resource scenario. Objective: In this study, we aimed to develop a deep-phenotyping method with good generalization ability for Chinese EHRs based on limited fine-grained annotation data. Methods: The core of the methodology was to identify linguistic patterns of phenotype descriptions in Chinese EHRs with a sequence motif discovery tool and perform deep phenotyping of Chinese EHRs by recognizing linguistic patterns in free text. Specifically, 1000 Chinese EHRs were manually annotated based on a fine-grained information model, PhenoSSU (Semantic Structured Unit of Phenotypes). The annotation data set was randomly divided into a training set (n=700, 70%) and a testing set (n=300, 30%). The process for mining linguistic patterns was divided into three steps. First, free text in the training set was encoded as single-letter sequences (P: phenotype, A: attribute). Second, a biological sequence analysis tool—MEME (Multiple Expectation Maximums for Motif Elicitation)—was used to identify motifs in the single-letter sequences. Finally, the identified motifs were reduced to a series of regular expressions representing linguistic patterns of PhenoSSU instances in Chinese EHRs. Based on the discovered linguistic patterns, we developed a deep-phenotyping method for Chinese EHRs, including a deep learning–based method for named entity recognition and a pattern recognition–based method for attribute prediction. Results: In total, 51 sequence motifs with statistical significance were mined from 700 Chinese EHRs in the training set and were combined into six regular expressions. It was found that these six regular expressions could be learned from a mean of 134 (SD 9.7) annotated EHRs in the training set. The deep-phenotyping algorithm for Chinese EHRs could recognize PhenoSSU instances with an overall accuracy of 0.844 on the test set. For the subtask of entity recognition, the algorithm achieved an F1 score of 0.898 with the Bidirectional Encoder Representations from Transformers–bidirectional long short-term memory and conditional random field model; for the subtask of attribute prediction, the algorithm achieved a weighted accuracy of 0.940 with the linguistic pattern–based method. Conclusions: We developed a simple but effective strategy to perform deep phenotyping of Chinese EHRs with limited fine-grained annotation data. Our work will promote the second use of Chinese EHRs and give inspiration to other non–English-speaking countries. %M 35657661 %R 10.2196/37213 %U https://www.jmir.org/2022/6/e37213 %U https://doi.org/10.2196/37213 %U http://www.ncbi.nlm.nih.gov/pubmed/35657661 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e35380 %T Digital Health Opportunities to Improve Primary Health Care in the Context of COVID-19: Scoping Review %A Silva,Cícera Renata Diniz Vieira %A Lopes,Rayssa Horácio %A de Goes Bay Jr,Osvaldo %A Martiniano,Claudia Santos %A Fuentealba-Torres,Miguel %A Arcêncio,Ricardo Alexandre %A Lapão,Luís Velez %A Dias,Sonia %A Uchoa,Severina Alice da Costa %+ Faculty of Health Sciences, Federal University of Rio Grande do Norte, 601 General Gustavo Cordeiro de Faria Street, Natal, 59012-570, Brazil, 55 84 3221 0862, renatadiniz_enf@yahoo.com.br %K digital health %K telehealth %K telemedicine %K primary health care %K quality of care %K COVID-19 %K pandemic %K science database %K gray literature %D 2022 %7 31.5.2022 %9 Review %J JMIR Hum Factors %G English %X Background: The COVID-19 pandemic brought social, economic, and health impacts, requiring fast adaptation of health systems. Although information and communication technologies were essential for achieving this objective, the extent to which health systems incorporated this technology is unknown. Objective: The aim of this study was to map the use of digital health strategies in primary health care worldwide and their impact on quality of care during the COVID-19 pandemic. Methods: We performed a scoping review based on the Joanna Briggs Institute manual and guided by the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) Extension for Scoping Reviews. A systematic and comprehensive three-step search was performed in June and July 2021 in multidisciplinary health science databases and the gray literature. Data extraction and eligibility were performed by two authors independently and interpreted using thematic analysis. Results: A total of 44 studies were included and six thematic groups were identified: characterization and geographic distribution of studies; nomenclatures of digital strategies adopted; types of information and communication technologies; characteristics of digital strategies in primary health care; impacts on quality of care; and benefits, limitations, and challenges of digital strategies in primary health care. The impacts on organization of quality of care were investigated by the majority of studies, demonstrating the strengthening of (1) continuity of care; (2) economic, social, geographical, time, and cultural accessibility; (3) coordination of care; (4) access; (5) integrality of care; (6) optimization of appointment time; (7) and efficiency. Negative impacts were also observed in the same dimensions, such as reduced access to services and increased inequity and unequal use of services offered, digital exclusion of part of the population, lack of planning for defining the role of professionals, disarticulation of actions with real needs of the population, fragile articulation between remote and face-to-face modalities, and unpreparedness of professionals to meet demands using digital technologies. Conclusions: The results showed the positive and negative impacts of remote strategies on quality of care in primary care and the inability to take advantage of the potential of technologies. This may demonstrate differences in the organization of fast and urgent implementation of digital strategies in primary health care worldwide. Primary health care must strengthen its response capacity, expand the use of information and communication technologies, and manage challenges using scientific evidence since digital health is important and must be integrated into public service. %M 35319466 %R 10.2196/35380 %U https://humanfactors.jmir.org/2022/2/e35380 %U https://doi.org/10.2196/35380 %U http://www.ncbi.nlm.nih.gov/pubmed/35319466 %0 Journal Article %@ 2152-7202 %I JMIR Publications %V 14 %N 1 %P e37759 %T In Anticipation of Sharing Pediatric Inpatient Notes: Focus Group Study With Stakeholders %A Smith,Catherine Arnott %A Kelly,Michelle M %+ The Information School, University of Wisconsin-Madison, 600 North Park Street, Madison, WI, 53706, United States, 1 6082632900, casmith24@wisc.edu %K medical informatics %K information sharing %K electronic health records %K patient portals %K hospitals %K pediatrics %K focus group %K inpatient care %K caregivers %D 2022 %7 30.5.2022 %9 Original Paper %J J Particip Med %G English %X Background: Patient portals are a health information technology that allows patients and their proxies, such as caregivers and family members, to access designated portions of their electronic health record using mobile devices and web browsers. The Open Notes initiative in the United States, which became federal law in April 2021, has redrawn and expanded the boundaries of medical records. Only a few studies have focused on sharing notes with parents or caregivers of pediatric patients. Objective: This study aimed to investigate the anticipated impact of increasing the flow of electronic health record information, specifically physicians’ daily inpatient progress notes, via a patient portal to parents during their child’s acute hospital stay—an understudied population and an understudied setting. Methods: A total of 5 in-person focus groups were conducted with 34 stakeholders most likely impacted by sharing of physicians’ inpatient notes with parents of hospitalized children: hospital administrators, hospitalist physicians, interns and resident physicians, nurses, and the parents themselves. Results: Distinct themes identified as benefits of pediatric inpatient Open Notes for parents emerged from all the 5 focus groups. These themes were communication, recapitulation and reinforcement, education, stress reduction, quality control, and improving family-provider relationships. Challenges identified included burden on provider, medical jargon, communication, sensitive content, and decreasing trust. Conclusions: Providing patients and, in the case of pediatrics, caregivers with access to medical records via patient portals increases the flow of information and, in turn, their ability to participate in the discourse of their care. Parents in this study demonstrated not only that they act as monitors and guardians of their children’s health but also that they are observers of the clinical processes taking place in the hospital and at their child’s bedside. This includes the clinical documentation process, from the creation of notes to the reading and sharing of the notes. Parents acknowledge not only the importance of notes in the clinicians’ workflow but also their collaboration with providers as part of the health care team. %M 35635743 %R 10.2196/37759 %U https://jopm.jmir.org/2022/1/e37759 %U https://doi.org/10.2196/37759 %U http://www.ncbi.nlm.nih.gov/pubmed/35635743 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 5 %P e33742 %T Impact of Hospital Characteristics and Governance Structure on the Adoption of Tracking Technologies for Clinical and Supply Chain Use: Longitudinal Study of US Hospitals %A Zhu,Xiao %A Tao,Youyou %A Zhu,Ruilin %A Wu,Dezhi %A Ming,Wai-kit %+ Management Science Department, Lancaster University Management School, Lancaster University, Bailrigg, Lancaster, LA1 4YX, United Kingdom, 44 1524592938, ruilin.zhu@lancaster.ac.uk %K radio frequency identification %K bar coding %K tracking technology adoption %K smart hospital %K hospital affiliation %K governance structure %K location %K clinical use %K supply chain use %D 2022 %7 26.5.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Despite the increasing adoption rate of tracking technologies in hospitals in the United States, few empirical studies have examined the factors involved in such adoption within different use contexts (eg, clinical and supply chain use contexts). To date, no study has systematically examined how governance structures impact technology adoption in different use contexts in hospitals. Given that the hospital governance structure fundamentally governs health care workflows and operations, understanding its critical role provides a solid foundation from which to explore factors involved in the adoption of tracking technologies in hospitals. Objective: This study aims to compare critical factors associated with the adoption of tracking technologies for clinical and supply chain uses and examine how governance structure types affect the adoption of tracking technologies in hospitals. Methods: This study was conducted based on a comprehensive and longitudinal national census data set comprising 3623 unique hospitals across 50 states in the United States from 2012 to 2015. Using mixed effects population logistic regression models to account for the effects within and between hospitals, we captured and examined the effects of hospital characteristics, locations, and governance structure on adjustments to the innate development of tracking technology over time. Results: From 2012 to 2015, we discovered that the proportion of hospitals in which tracking technologies were fully implemented for clinical use increased from 36.34% (782/2152) to 54.63% (1316/2409), and that for supply chain use increased from 28.58% (615/2152) to 41.3% (995/2409). We also discovered that adoption factors impact the clinical and supply chain use contexts differently. In the clinical use context, compared with hospitals located in urban areas, hospitals in rural areas (odds ratio [OR] 0.68, 95% CI 0.56-0.80) are less likely to fully adopt tracking technologies. In the context of supply chain use, the type of governance structure influences tracking technology adoption. Compared with hospitals not affiliated with a health system, implementation rates increased as hospitals affiliated with a more centralized health system—1.9-fold increase (OR 1.87, 95% CI 1.60-2.13) for decentralized or independent hospitals, 2.4-fold increase (OR 2.40, 95% CI 2.07-2.80) for moderately centralized health systems, and 3.1-fold increase for centralized health systems (OR 3.07, 95% CI 2.67-3.53). Conclusions: As the first of such type of studies, we provided a longitudinal overview of how hospital characteristics and governance structure jointly affect adoption rates of tracking technology in both clinical and supply chain use contexts, which is essential for developing intelligent infrastructure for smart hospital systems. This study informs researchers, health care providers, and policy makers that hospital characteristics, locations, and governance structures have different impacts on the adoption of tracking technologies for clinical and supply chain use and on health resource disparities among hospitals of different sizes, locations, and governance structures. %M 35617002 %R 10.2196/33742 %U https://www.jmir.org/2022/5/e33742 %U https://doi.org/10.2196/33742 %U http://www.ncbi.nlm.nih.gov/pubmed/35617002 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 5 %P e27694 %T The Accuracy of Artificial Intelligence in the Endoscopic Diagnosis of Early Gastric Cancer: Pooled Analysis Study %A Chen,Pei-Chin %A Lu,Yun-Ru %A Kang,Yi-No %A Chang,Chun-Chao %+ Division of Gastroenterology and Hepatology, Department of Internal Medicine, Taipei Medical University Hospital, No 252, Wuxing St, Taipei, 110, Taiwan, 886 227372181, chunchao@tmu.edu.tw %K artificial intelligence %K early gastric cancer %K endoscopy %D 2022 %7 16.5.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Artificial intelligence (AI) for gastric cancer diagnosis has been discussed in recent years. The role of AI in early gastric cancer is more important than in advanced gastric cancer since early gastric cancer is not easily identified in clinical practice. However, to our knowledge, past syntheses appear to have limited focus on the populations with early gastric cancer. Objective: The purpose of this study is to evaluate the diagnostic accuracy of AI in the diagnosis of early gastric cancer from endoscopic images. Methods: We conducted a systematic review from database inception to June 2020 of all studies assessing the performance of AI in the endoscopic diagnosis of early gastric cancer. Studies not concerning early gastric cancer were excluded. The outcome of interest was the diagnostic accuracy (comprising sensitivity, specificity, and accuracy) of AI systems. Study quality was assessed on the basis of the revised Quality Assessment of Diagnostic Accuracy Studies. Meta-analysis was primarily based on a bivariate mixed-effects model. A summary receiver operating curve and a hierarchical summary receiver operating curve were constructed, and the area under the curve was computed. Results: We analyzed 12 retrospective case control studies (n=11,685) in which AI identified early gastric cancer from endoscopic images. The pooled sensitivity and specificity of AI for early gastric cancer diagnosis were 0.86 (95% CI 0.75-0.92) and 0.90 (95% CI 0.84-0.93), respectively. The area under the curve was 0.94. Sensitivity analysis of studies using support vector machines and narrow-band imaging demonstrated more consistent results. Conclusions: For early gastric cancer, to our knowledge, this was the first synthesis study on the use of endoscopic images in AI in diagnosis. AI may support the diagnosis of early gastric cancer. However, the collocation of imaging techniques and optimal algorithms remain unclear. Competing models of AI for the diagnosis of early gastric cancer are worthy of future investigation. Trial Registration: PROSPERO CRD42020193223; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=193223 %M 35576561 %R 10.2196/27694 %U https://www.jmir.org/2022/5/e27694 %U https://doi.org/10.2196/27694 %U http://www.ncbi.nlm.nih.gov/pubmed/35576561 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 5 %P e35981 %T Integration of an Intensive Care Unit Visualization Dashboard (i-Dashboard) as a Platform to Facilitate Multidisciplinary Rounds: Cluster-Randomized Controlled Trial %A Lai,Chao-Han %A Li,Kai-Wen %A Hu,Fang-Wen %A Su,Pei-Fang %A Hsu,I-Lin %A Huang,Min‑Hsin %A Huang,Yen‑Ta %A Liu,Ping-Yen %A Shen,Meng-Ru %+ Department of Obstetrics and Gynecology, National Cheng Kung University Hospital, College of Medicine, National Cheng Kung University, 138, Sheng-Li Road, Tainan, 704, Taiwan, 886 062353535 ext 5505, mrshen@mail.ncku.edu.tw %K Intensive care unit %K multidisciplinary round %K visualization dashboard %K large screen %K information management strategy %K electronic health record %K medical record %K digital health %K dashboard %K i-Dashboard %K electronic medical record %K information exchange %D 2022 %7 13.5.2022 %9 Original Paper %J J Med Internet Res %G English %X Background:  Multidisciplinary rounds (MDRs) are scheduled, patient-focused communication mechanisms among multidisciplinary providers in the intensive care unit (ICU). Objective: i-Dashboard is a custom-developed visualization dashboard that supports (1) key information retrieval and reorganization, (2) time-series data, and (3) display on large touch screens during MDRs. This study aimed to evaluate the performance, including the efficiency of prerounding data gathering, communication accuracy, and information exchange, and clinical satisfaction of integrating i-Dashboard as a platform to facilitate MDRs. Methods: A cluster-randomized controlled trial was performed in 2 surgical ICUs at a university hospital. Study participants included all multidisciplinary care team members. The performance and clinical satisfaction of i-Dashboard during MDRs were compared with those of the established electronic medical record (EMR) through direct observation and questionnaire surveys. Results: Between April 26 and July 18, 2021, a total of 78 and 91 MDRs were performed with the established EMR and i-Dashboard, respectively. For prerounding data gathering, the median time was 10.4 (IQR 9.1-11.8) and 4.6 (IQR 3.5-5.8) minutes using the established EMR and i-Dashboard (P<.001), respectively. During MDRs, data misrepresentations were significantly less frequent with i-Dashboard (median 0, IQR 0-0) than with the established EMR (4, IQR 3-5; P<.001). Further, effective recommendations were significantly more frequent with i-Dashboard than with the established EMR (P<.001). The questionnaire results revealed that participants favored using i-Dashboard in association with the enhancement of care plan development and team participation during MDRs. Conclusions:  i-Dashboard increases efficiency in data gathering. Displaying i-Dashboard on large touch screens in MDRs may enhance communication accuracy, information exchange, and clinical satisfaction. The design concepts of i-Dashboard may help develop visualization dashboards that are more applicable for ICU MDRs. Trial Registration: ClinicalTrials.gov NCT04845698; https://clinicaltrials.gov/ct2/show/NCT04845698 %M 35560107 %R 10.2196/35981 %U https://www.jmir.org/2022/5/e35981 %U https://doi.org/10.2196/35981 %U http://www.ncbi.nlm.nih.gov/pubmed/35560107 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 5 %P e32006 %T Factors Predicting Engagement of Older Adults With a Coach-Supported eHealth Intervention Promoting Lifestyle Change and Associations Between Engagement and Changes in Cardiovascular and Dementia Risk: Secondary Analysis of an 18-Month Multinational Randomized Controlled Trial %A Coley,Nicola %A Andre,Laurine %A Hoevenaar-Blom,Marieke P %A Ngandu,Tiia %A Beishuizen,Cathrien %A Barbera,Mariagnese %A van Wanrooij,Lennard %A Kivipelto,Miia %A Soininen,Hilkka %A van Gool,Willem %A Brayne,Carol %A Moll van Charante,Eric %A Richard,Edo %A Andrieu,Sandrine %A , %A , %+ Center for Epidemiology and Research in Population health (CERPOP), University of Toulouse III Paul Sabatier (UPS), National Institute of Health and Medical Research (INSERM) mixed research unit (UMR) 1295, 37 allées Jules Guesde, Toulouse, 31000, France, 33 561145680, nicola.coley@inserm.fr %K aging %K eHealth %K disparities %K engagement %K prevention %K cardiovascular %K lifestyle %K risk factors %D 2022 %7 9.5.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Digital health interventions could help to prevent age-related diseases, but little is known about how older adults engage with such interventions, especially in the long term, or whether engagement is associated with changes in clinical, behavioral, or biological outcomes in this population. Disparities in engagement levels with digital health interventions may exist among older people and be associated with health inequalities. Objective: This study aimed to describe older adults’ engagement with an eHealth intervention, identify factors associated with engagement, and examine associations between engagement and changes in cardiovascular and dementia risk factors (blood pressure, cholesterol, BMI, physical activity, diet, and cardiovascular and dementia risk scores). Methods: This was a secondary analysis of the 18-month randomized controlled Healthy Ageing Through Internet Counselling in the Elderly trial of a tailored internet-based intervention encouraging behavior changes, with remote support from a lifestyle coach, to reduce cardiovascular and cognitive decline risk in 2724 individuals aged ≥65 years, recruited offline in the Netherlands, Finland, and France. Engagement was assessed via log-in frequency, number of lifestyle goals set, measurements entered and messages sent to coaches, and percentage of education materials read. Clinical and biological data were collected during in-person visits at baseline and 18 months. Lifestyle data were self-reported on a web-based platform. Results: Of the 1389 intervention group participants, 1194 (85.96%) sent at least one message. They logged in a median of 29 times, and set a median of 1 goal. Higher engagement was associated with significantly greater improvement in biological and behavioral risk factors, with evidence of a dose-response effect. Compared with the control group, the adjusted mean difference (95% CI) in 18-month change in the primary outcome, a composite z-score comprising blood pressure, BMI, and cholesterol, was −0.08 (−0.12 to −0.03), −0.04 (−0.08 to 0.00), and 0.00 (−0.08 to 0.08) in the high, moderate, and low engagement groups, respectively. Low engagers showed no improvement in any outcome measures compared with the control group. Participants not using a computer regularly before the study engaged much less with the intervention than those using a computer up to 7 (adjusted odds ratio 5.39, 95% CI 2.66-10.95) or ≥7 hours per week (adjusted odds ratio 6.58, 95% CI 3.21-13.49). Those already working on or with short-term plans for lifestyle improvement at baseline, and with better cognition, engaged more. Conclusions: Greater engagement with an eHealth lifestyle intervention was associated with greater improvement in risk factors in older adults. However, those with limited computer experience, who tended to have a lower level of education, or who had poorer cognition engaged less. Additional support or forms of intervention delivery for such individuals could help minimize potential health inequalities associated with the use of digital health interventions in older people. %M 35385395 %R 10.2196/32006 %U https://www.jmir.org/2022/5/e32006 %U https://doi.org/10.2196/32006 %U http://www.ncbi.nlm.nih.gov/pubmed/35385395 %0 Journal Article %@ 2371-4379 %I JMIR Publications %V 7 %N 2 %P e35163 %T Acceptance and Effect of Continuous Glucose Monitoring on Discharge From Hospital in Patients With Type 2 Diabetes: Open-label, Prospective, Controlled Study %A Depczynski,Barbara %A Poynten,Ann %+ Prince of Wales Hospital, Barker St, Randwick, NSW 2031, Australia, 61 293824600, barbaradepczynski@gmail.com %K CGM %K continuous glucose monitor %K hospital %K discharge %K T2DM %K type 2 diabetes %K diabetes %K glucose monitoring %D 2022 %7 9.5.2022 %9 Original Paper %J JMIR Diabetes %G English %X Background: Continuous glucose monitors (CGM) can provide detailed information on glucose excursions. There is little information on safe transitioning from hospital back to the community for patients who have had diabetes therapies adjusted in hospital and it is unclear whether newer technologies may facilitate this process. Objective: Our aim was to determine whether offering CGM on discharge would be acceptable and if CGM initiated on hospital discharge in people with type 2 diabetes (T2DM) would reduce hospital re-presentations at 1 month. Methods: This was an open-label study. Adult inpatients with T2DM, who were to be discharged home and required postdischarge glycemic stabilization, were offered usual care consisting of clinic review at 2 weeks and at 3 months. In addition to usual care, participants in the intervention arm were provided with a Libre flash glucose monitoring system (Abbott Australia). An initial run-in phase for the first 20 participants was planned, where all consenting participants were enrolled in an active arm. Subsequently, all participants were to be randomized to the active arm or usual care control group. Results: Of 237 patients screened during their hospital admission, 34 had comorbidities affecting cognition that prevented informed consent and affected their ability to learn to use the CGM device. In addition, 21 were not able to be approached as the material was only in English. Of 101 potential participants who fulfilled eligibility criteria, 19 provided consent and were enrolled. Of the 82 patients who declined to participate, 31 advised that the learning of a new task toward discharge was overwhelming or too stressful and 26 were not interested, with no other details. Due to poor recruitment, the study was terminated without entering the randomization phase to determine whether CGM could reduce readmission rate. Conclusions: These results suggest successful and equitable implementation of telemedicine programs requires that any human factors such as language, cognition, and possible disengagement be addressed. Recovery from acute illness may not be the ideal time for introduction of newer technologies or may require more novel implementation frameworks. %M 35532995 %R 10.2196/35163 %U https://diabetes.jmir.org/2022/2/e35163 %U https://doi.org/10.2196/35163 %U http://www.ncbi.nlm.nih.gov/pubmed/35532995 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 5 %P e38513 %T Through the Narrative Looking Glass: Commentary on “Impact of Electronic Health Records on Information Practices in Mental Health Contexts: Scoping Review” %A Weir,Charlene %+ Department of Biomedical Informatics, University of Utah, 421 Wakara Way, Ste 140, Salt Lake City, UT, 84108-3514, United States, 1 801 541 9462, charlene.weir@utah.edu %K electronic health records %K psychiatry %K mental health %K electronic medical records %K health informatics %K mental illness %K scoping review %K clinical decision support %D 2022 %7 4.5.2022 %9 Commentary %J J Med Internet Res %G English %X The authors of “Impact of Electronic Health Records on Information Practices in Mental Health Contexts: Scoping Review” have effectively brought to our attention the failure of the electronic health record (EHR) to represent the human context. Because mental health or behavioral disorders (and functional status in general) emerge from an interaction between the individual’s characteristics and the social context, it is essentially a failure to represent the human context. The assessment and treatment of these disorders must reflect how the person lives, their degree of social connectedness, their personal motivation, and their cultural background. This type of information is best communicated both through narrative and in collaboration with other providers and the patient—largely because human social memory is organized around situation models and natural episodes. Neither functionality is currently available in most EHRs. Narrative communication is effective for several reasons: (1) it supports the communication of goals between providers; (2) it allows the author to express their belief in others’ perspectives (theory of mind), for example, those who will be reading these notes; and (3) it supports the incorporation of the patient’s personal perspective. The failure of the EHR to support mental health information data and information practices is, therefore, essentially a failure to support the basic communication functions necessary for the narrative. The authors have rightly noted the problems of the EHR in this domain, but perhaps they did not completely link the problems to the lack of functionality to support narrative communication. Suggestions for adding design elements are discussed. %M 35507399 %R 10.2196/38513 %U https://www.jmir.org/2022/5/e38513 %U https://doi.org/10.2196/38513 %U http://www.ncbi.nlm.nih.gov/pubmed/35507399 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 5 %P e30405 %T Impact of Electronic Health Records on Information Practices in Mental Health Contexts: Scoping Review %A Kariotis,Timothy Charles %A Prictor,Megan %A Chang,Shanton %A Gray,Kathleen %+ School of Computing and Information Systems, University of Melbourne, The University of Melbourne, Parkville, 3052, Australia, 61 0488300223, Timothy.kariotis@unimelb.edu.au %K electronic health records %K psychiatry %K mental health %K electronic medical records %K health informatics %K mental illness %K scoping review %K clinical decision support %D 2022 %7 4.5.2022 %9 Review %J J Med Internet Res %G English %X Background: The adoption of electronic health records (EHRs) and electronic medical records (EMRs) has been slow in the mental health context, partly because of concerns regarding the collection of sensitive information, the standardization of mental health data, and the risk of negatively affecting therapeutic relationships. However, EHRs and EMRs are increasingly viewed as critical to improving information practices such as the documentation, use, and sharing of information and, more broadly, the quality of care provided. Objective: This paper aims to undertake a scoping review to explore the impact of EHRs on information practices in mental health contexts and also explore how sensitive information, data standardization, and therapeutic relationships are managed when using EHRs in mental health contexts. Methods: We considered a scoping review to be the most appropriate method for this review because of the relatively recent uptake of EHRs in mental health contexts. A comprehensive search of electronic databases was conducted with no date restrictions for articles that described the use of EHRs, EMRs, or associated systems in the mental health context. One of the authors reviewed all full texts, with 2 other authors each screening half of the full-text articles. The fourth author mediated the disagreements. Data regarding study characteristics were charted. A narrative and thematic synthesis approach was taken to analyze the included studies’ results and address the research questions. Results: The final review included 40 articles. The included studies were highly heterogeneous with a variety of study designs, objectives, and settings. Several themes and subthemes were identified that explored the impact of EHRs on information practices in the mental health context. EHRs improved the amount of information documented compared with paper. However, mental health–related information was regularly missing from EHRs, especially sensitive information. EHRs introduced more standardized and formalized documentation practices that raised issues because of the focus on narrative information in the mental health context. EHRs were found to disrupt information workflows in the mental health context, especially when they did not include appropriate templates or care plans. Usability issues also contributed to workflow concerns. Managing the documentation of sensitive information in EHRs was problematic; clinicians sometimes watered down sensitive information or chose to keep it in separate records. Concerningly, the included studies rarely involved service user perspectives. Furthermore, many studies provided limited information on the functionality or technical specifications of the EHR being used. Conclusions: We identified several areas in which work is needed to ensure that EHRs benefit clinicians and service users in the mental health context. As EHRs are increasingly considered critical for modern health systems, health care decision-makers should consider how EHRs can better reflect the complexity and sensitivity of information practices and workflows in the mental health context. %M 35507393 %R 10.2196/30405 %U https://www.jmir.org/2022/5/e30405 %U https://doi.org/10.2196/30405 %U http://www.ncbi.nlm.nih.gov/pubmed/35507393 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 4 %P e30898 %T Social Networking Service, Patient-Generated Health Data, and Population Health Informatics: National Cross-sectional Study of Patterns and Implications of Leveraging Digital Technologies to Support Mental Health and Well-being %A Ye,Jiancheng %A Wang,Zidan %A Hai,Jiarui %+ Feinberg School of Medicine, Northwestern University, 633 N. Saint Clair St, Chicago, IL, 60611, United States, 1 312 503 3690, jiancheng.ye@u.northwestern.edu %K patient-generated health data %K social network %K population health informatics %K mental health %K social determinants of health %K health data sharing %K technology acceptability %K mobile phone %K mobile health %D 2022 %7 29.4.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: The emerging health technologies and digital services provide effective ways of collecting health information and gathering patient-generated health data (PGHD), which provide a more holistic view of a patient’s health and quality of life over time, increase visibility into a patient’s adherence to a treatment plan or study protocol, and enable timely intervention before a costly care episode. Objective: Through a national cross-sectional survey in the United States, we aimed to describe and compare the characteristics of populations with and without mental health issues (depression or anxiety disorders), including physical health, sleep, and alcohol use. We also examined the patterns of social networking service use, PGHD, and attitudes toward health information sharing and activities among the participants, which provided nationally representative estimates. Methods: We drew data from the 2019 Health Information National Trends Survey of the National Cancer Institute. The participants were divided into 2 groups according to mental health status. Then, we described and compared the characteristics of the social determinants of health, health status, sleeping and drinking behaviors, and patterns of social networking service use and health information data sharing between the 2 groups. Multivariable logistic regression models were applied to assess the predictors of mental health. All the analyses were weighted to provide nationally representative estimates. Results: Participants with mental health issues were significantly more likely to be younger, White, female, and lower-income; have a history of chronic diseases; and be less capable of taking care of their own health. Regarding behavioral health, they slept <6 hours on average, had worse sleep quality, and consumed more alcohol. In addition, they were more likely to visit and share health information on social networking sites, write online diary blogs, participate in online forums or support groups, and watch health-related videos. Conclusions: This study illustrates that individuals with mental health issues have inequitable social determinants of health, poor physical health, and poor behavioral health. However, they are more likely to use social networking platforms and services, share their health information, and actively engage with PGHD. Leveraging these digital technologies and services could be beneficial for developing tailored and effective strategies for self-monitoring and self-management. %M 35486428 %R 10.2196/30898 %U https://www.jmir.org/2022/4/e30898 %U https://doi.org/10.2196/30898 %U http://www.ncbi.nlm.nih.gov/pubmed/35486428 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e29780 %T Health Professionals’ eHealth Literacy and System Experience Before and 3 Months After the Implementation of an Electronic Health Record System: Longitudinal Study %A Kayser,Lars %A Karnoe,Astrid %A Duminski,Emily %A Jakobsen,Svend %A Terp,Rikke %A Dansholm,Susanne %A Roeder,Michael %A From,Gustav %+ Section of Health Service Research, Department of Public Health, University of Copenhagen, Øster Farimagsgade 5, Copenhagen, 1353, Denmark, 45 28757291, lk@sund.ku.dk %K health care professionals %K eHealth literacy %K electronic health record %K implementation %K digital health %K eHealth %K health literacy %K health records %K eHealth records %K patient care %D 2022 %7 29.4.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The implementation of an integrated electronic health record (EHR) system can potentially provide health care providers with support standardization of patient care, pathways, and workflows, as well as provide medical staff with decision support, easier access, and the same interface across features and subsystems. These potentials require an implementation process in which the expectations of the medical staff and the provider of the new system are aligned with respect to the medical staff’s knowledge and skills, as well as the interface and performance of the system. Awareness of the medical staff’s level of eHealth literacy may be a way of understanding and aligning these expectations and following the progression of the implementation process. Objective: The objective of this study was to investigate how a newly developed and modified instrument measuring the medical staff’s eHealth literacy (staff eHealth Literacy Questionnaire [eHLQ]) can be used to inform the system provider and the health care organization in the implementation process and evaluate whether the medical staff’s perceptions of the ease of use change and how this may be related to their level of eHealth literacy. Methods: A modified version of the eHLQ was distributed to the staff of a medical department in Denmark before and 3 months after the implementation of a new EHR system. The survey also included questions related to users’ perceived ease of use and their self-reported information technology skills. Results: The mean age of the 194 participants before implementation was 43.1 (SD 12.4) years, and for the 198 participants after implementation, it was 42.3 (SD 12.5) years. After the implementation, the only difference compared with the preimplementation data was a small decrease in staff eHLQ5 (motivated to engage with digital services; unpaired 2-tailed t test; P=.009; effect size 0.267), and the values of the scales relating to the medical staff’s knowledge and skills (eHLQ1-3) were approximately ≥3 both before and after implementation. The range of scores was narrower after implementation, indicating that some of those with the lowest ability benefited from the training and new experiences with the EHR. There was an association between perceived ease of use and the 3 tested staff eHLQ scales, both before and after implementation. Conclusions: The staff eHLQ may be a good candidate for monitoring the medical staff’s digital competence in and response to the implementation of new digital solutions. This may enable those responsible for the implementation to tailor efforts to the specific needs of segments of users and inform them if the process is not going according to plan with respect to the staff’s information technology–related knowledge and skills, trust in data security, motivation, and experience of a coherent system that suits their needs and supports the workflows and data availability. %M 35486414 %R 10.2196/29780 %U https://humanfactors.jmir.org/2022/2/e29780 %U https://doi.org/10.2196/29780 %U http://www.ncbi.nlm.nih.gov/pubmed/35486414 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e29118 %T Effective Communication of Personalized Risks and Patient Preferences During Surgical Informed Consent Using Data Visualization: Qualitative Semistructured Interview Study With Patients After Surgery %A Gisladottir,Undina %A Nakikj,Drashko %A Jhunjhunwala,Rashi %A Panton,Jasmine %A Brat,Gabriel %A Gehlenborg,Nils %+ Department of Biomedical Informatics, Harvard Medical School, Harvard University, 10 Shattuck Street, Suite 514, Boston, MA, 02115, United States, 1 6174321535, nils@hms.harvard.edu %K data visualization %K surgical informed consent %K shared decision-making %K biomedical informatics %D 2022 %7 29.4.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: There is no consensus on which risks to communicate to a prospective surgical patient during informed consent or how. Complicating the process, patient preferences may diverge from clinical assumptions and are often not considered for discussion. Such discrepancies can lead to confusion and resentment, raising the potential for legal action. To overcome these issues, we propose a visual consent tool that incorporates patient preferences and communicates personalized risks to patients using data visualization. We used this platform to identify key effective visual elements to communicate personalized surgical risks. Objective: Our main focus is to understand how to best communicate personalized risks using data visualization. To contextualize patient responses to the main question, we examine how patients perceive risks before surgery (research question 1), how suitably the visual consent tool is able to present personalized surgical risks (research question 2), how well our visualizations convey those personalized surgical risks (research question 3), and how the visual consent tool could improve the informed consent process and how it can be used (research question 4). Methods: We designed a visual consent tool to meet the objectives of our study. To calculate and list personalized surgical risks, we used the American College of Surgeons risk calculator. We created multiple visualization mock-ups using visual elements previously determined to be well-received for risk communication. Semistructured interviews were conducted with patients after surgery, and each of the mock-ups was presented and evaluated independently and in the context of our visual consent tool design. The interviews were transcribed, and thematic analysis was performed to identify major themes. We also applied a quantitative approach to the analysis to assess the prevalence of different perceptions of the visualizations presented in our tool. Results: In total, 20 patients were interviewed, with a median age of 59 (range 29-87) years. Thematic analysis revealed factors that influenced the perception of risk (the surgical procedure, the cognitive capacity of the patient, and the timing of consent; research question 1); factors that influenced the perceived value of risk visualizations (preference for rare event communication, preference for risk visualization, and usefulness of comparison with the average; research question 3); and perceived usefulness and use cases of the visual consent tool (research questions 2 and 4). Most importantly, we found that patients preferred the visual consent tool to current text-based documents and had no unified preferences for risk visualization. Furthermore, our findings suggest that patient concerns were not often represented in existing risk calculators. Conclusions: We identified key elements that influence effective visual risk communication in the perioperative setting and pointed out the limitations of the existing calculators in addressing patient concerns. Patient preference is highly variable and should influence choices regarding risk presentation and visualization. %M 35486432 %R 10.2196/29118 %U https://humanfactors.jmir.org/2022/2/e29118 %U https://doi.org/10.2196/29118 %U http://www.ncbi.nlm.nih.gov/pubmed/35486432 %0 Journal Article %@ 2563-6316 %I JMIR Publications %V 3 %N 2 %P e22912 %T Using Structural Equation Modelling in Routine Clinical Data on Diabetes and Depression: Observational Cohort Study %A Ronaldson,Amy %A Freestone,Mark %A Zhang,Haoyuan %A Marsh,William %A Bhui,Kamaldeep %+ Wolfson Institute of Population Health, Queen Mary University of London, Centre for Psychiatry and Mental Health, Yvonne Carter Building, London, E1 2AB, United Kingdom, 44 02078822033 ext 2033, m.c.freestone@qmul.ac.uk %K depression %K diabetes %K electronic health records %K acute care %K PLS-SEM %K path analysis %K equation modelling %K accident %K emergency care %K emergency %K structural equation modelling %K clinical data %D 2022 %7 27.4.2022 %9 Original Paper %J JMIRx Med %G English %X Background: Large data sets comprising routine clinical data are becoming increasingly available for use in health research. These data sets contain many clinical variables that might not lend themselves to use in research. Structural equation modelling (SEM) is a statistical technique that might allow for the creation of “research-friendly” clinical constructs from these routine clinical variables and therefore could be an appropriate analytic method to apply more widely to routine clinical data. Objective: SEM was applied to a large data set of routine clinical data developed in East London to model well-established clinical associations. Depression is common among patients with type 2 diabetes, and is associated with poor diabetic control, increased diabetic complications, increased health service utilization, and increased health care costs. Evidence from trial data suggests that integrating psychological treatment into diabetes care can improve health status and reduce costs. Attempting to model these known associations using SEM will test the utility of this technique in routine clinical data sets. Methods: Data were cleaned extensively prior to analysis. SEM was used to investigate associations between depression, diabetic control, diabetic care, mental health treatment, and Accident & Emergency (A&E) use in patients with type 2 diabetes. The creation of the latent variables and the direction of association between latent variables in the model was based upon established clinical knowledge. Results: The results provided partial support for the application of SEM to routine clinical data. Overall, 19% (3106/16,353) of patients with type 2 diabetes had received a diagnosis of depression. In line with known clinical associations, depression was associated with worse diabetic control (β=.034, P<.001) and increased A&E use (β=.071, P<.001). However, contrary to expectation, worse diabetic control was associated with lower A&E use (β=–.055, P<.001) and receipt of mental health treatment did not impact upon diabetic control (P=.39). Receipt of diabetes care was associated with better diabetic control (β=–.072, P<.001), having depression (β=.018, P=.007), and receiving mental health treatment (β=.046, P<.001), which might suggest that comprehensive integrated care packages are being delivered in East London. Conclusions: Some established clinical associations were successfully modelled in a sample of patients with type 2 diabetes in a way that made clinical sense, providing partial evidence for the utility of SEM in routine clinical data. Several issues relating to data quality emerged. Data improvement would have likely enhanced the utility of SEM in this data set. %M 37725546 %R 10.2196/22912 %U https://med.jmirx.org/2022/2/e22912 %U https://doi.org/10.2196/22912 %U http://www.ncbi.nlm.nih.gov/pubmed/37725546 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 4 %P e28114 %T Understanding the Research Landscape of Deep Learning in Biomedical Science: Scientometric Analysis %A Nam,Seojin %A Kim,Donghun %A Jung,Woojin %A Zhu,Yongjun %+ Department of Library and Information Science, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea, 82 2 2123 2409, zhu@yonsei.ac.kr %K deep learning %K scientometric analysis %K research publications %K research landscape %K research collaboration %K knowledge diffusion %D 2022 %7 22.4.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Advances in biomedical research using deep learning techniques have generated a large volume of related literature. However, there is a lack of scientometric studies that provide a bird’s-eye view of them. This absence has led to a partial and fragmented understanding of the field and its progress. Objective: This study aimed to gain a quantitative and qualitative understanding of the scientific domain by analyzing diverse bibliographic entities that represent the research landscape from multiple perspectives and levels of granularity. Methods: We searched and retrieved 978 deep learning studies in biomedicine from the PubMed database. A scientometric analysis was performed by analyzing the metadata, content of influential works, and cited references. Results: In the process, we identified the current leading fields, major research topics and techniques, knowledge diffusion, and research collaboration. There was a predominant focus on applying deep learning, especially convolutional neural networks, to radiology and medical imaging, whereas a few studies focused on protein or genome analysis. Radiology and medical imaging also appeared to be the most significant knowledge sources and an important field in knowledge diffusion, followed by computer science and electrical engineering. A coauthorship analysis revealed various collaborations among engineering-oriented and biomedicine-oriented clusters of disciplines. Conclusions: This study investigated the landscape of deep learning research in biomedicine and confirmed its interdisciplinary nature. Although it has been successful, we believe that there is a need for diverse applications in certain areas to further boost the contributions of deep learning in addressing biomedical research problems. We expect the results of this study to help researchers and communities better align their present and future work. %M 35451980 %R 10.2196/28114 %U https://www.jmir.org/2022/4/e28114 %U https://doi.org/10.2196/28114 %U http://www.ncbi.nlm.nih.gov/pubmed/35451980 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 4 %P e31825 %T Modeling Data Journeys to Inform the Digital Transformation of Kidney Transplant Services: Observational Study %A Sharma,Videha %A Eleftheriou,Iliada %A van der Veer,Sabine N %A Brass,Andrew %A Augustine,Titus %A Ainsworth,John %+ Centre for Health Informatics, Division of Informatics, Imaging and Data Science, The University of Manchester, Vaughan House, Portsmouth Street, Manchester, M13 9GV, United Kingdom, 44 7735360958, videha.sharma@postgrad.manchester.ac.uk %K digital transformation %K health information exchange %K interoperability %K medical informatics %K data journey modelling %K kidney transplantation %D 2022 %7 21.4.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Data journey modeling is a methodology used to establish a high-level overview of information technology (IT) infrastructure in health care systems. It allows a better understanding of sociotechnical barriers and thus informs meaningful digital transformation. Kidney transplantation is a complex clinical service involving multiple specialists and providers. The referral pathway for a transplant requires the centralization of patient data across multiple IT solutions and health care organizations. At present, there is a poor understanding of the role of IT in this process, specifically regarding the management of patient data, clinical communication, and workflow support. Objective: To apply data journey modeling to better understand interoperability, data access, and workflow requirements of a regional multicenter kidney transplant service. Methods: An incremental methodology was used to develop the data journey model. This included review of service documents, domain expert interviews, and iterative modeling sessions. Results were analyzed based on the LOAD (landscape, organizations, actors, and data) framework to provide a meaningful assessment of current data management challenges and inform ways for IT to overcome these challenges. Results: Results were presented as a diagram of the organizations (n=4), IT systems (n>9), actors (n>4), and data journeys (n=0) involved in the transplant referral pathway. The diagram revealed that all movement of data was dependent on actor interaction with IT systems and manual transcription of data into Microsoft Word (Microsoft, Inc) documents. Each actor had between 2 and 5 interactions with IT systems to capture all relevant data, a process that was reported to be time consuming and error prone. There was no interoperability within or across organizations, which led to delays as clinical teams manually transferred data, such as medical history and test results, via post or email. Conclusions: Overall, data journey modeling demonstrated that human actors, rather than IT systems, formed the central focus of data movement. The IT landscape did not complement this workflow and exerted a significant administrative burden on clinical teams. Based on this study, future solutions must consider regional interoperability and specialty-specific views of data to support multi-organizational clinical services such as transplantation. %M 35451983 %R 10.2196/31825 %U https://www.jmir.org/2022/4/e31825 %U https://doi.org/10.2196/31825 %U http://www.ncbi.nlm.nih.gov/pubmed/35451983 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 4 %P e32776 %T Mechanism of Impact of Big Data Resources on Medical Collaborative Networks From the Perspective of Transaction Efficiency of Medical Services: Survey Study %A Yuan,Junyi %A Wang,Sufen %A Pan,Changqing %+ Hospital’s Office, Shanghai Chest Hospital, Shanghai Jiao Tong University, No 241 West Huaihai Road, Xuhui District, Shanghai, 200030, China, 86 21 62805080, panchangqing@shchest.org %K medical collaborative networks %K big data resources %K transaction efficiency %D 2022 %7 21.4.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: The application of big data resources and the development of medical collaborative networks (MCNs) boost each other. However, MCNs are often assumed to be exogenous. How big data resources affect the emergence, development, and evolution of endogenous MCNs has not been well explained. Objective: This study aimed to explore and understand the influence of the mechanism of a wide range of shared and private big data resources on the transaction efficiency of medical services to reveal the impact of big data resources on the emergence and development of endogenous MCNs. Methods: This study was conducted by administering a survey questionnaire to information technology staff and medical staff from 132 medical institutions in China. Data from information technology staff and medical staff were integrated. Structural equation modeling was used to test the direct impact of big data resources on transaction efficiency of medical services. For those big data resources that had no direct impact, we analyzed their indirect impact. Results: Sharing of diagnosis and treatment data (β=.222; P=.03) and sharing of medical research data (β=.289; P=.04) at the network level (as big data itself) positively directly affected the transaction efficiency of medical services. Network protection of the external link systems (β=.271; P=.008) at the level of medical institutions (as big data technology) positively directly affected the transaction efficiency of medical services. Encryption security of web-based data (as big data technology) at the level of medical institutions, medical service capacity available for external use, real-time data of diagnosis and treatment services (as big data itself) at the level of medical institutions, and policies and regulations at the network level indirectly affected the transaction efficiency through network protection of the external link systems at the level of medical institutions. Conclusions: This study found that big data technology, big data itself, and policy at the network and organizational levels interact with, and influence, each other to form the transaction efficiency of medical services. On the basis of the theory of neoclassical economics, the study highlighted the implications of big data resources for the emergence and development of endogenous MCNs. %M 35318187 %R 10.2196/32776 %U https://www.jmir.org/2022/4/e32776 %U https://doi.org/10.2196/32776 %U http://www.ncbi.nlm.nih.gov/pubmed/35318187 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e34626 %T A Mobile App for Advance Care Planning and Advance Directives (Accordons-nous): Development and Usability Study %A Schöpfer,Céline %A Ehrler,Frederic %A Berger,Antoine %A Bollondi Pauly,Catherine %A Buytaert,Laurence %A De La Serna,Camille %A Hartheiser,Florence %A Fassier,Thomas %A Clavien,Christine %+ Institute for Ethics, History, and the Humanities, University Medical Center, University of Geneva, CMU/1 rue Michel Servet, Geneva, 1211, Switzerland, 41 22 379 46 00, celine.schopfer@etu.unige.ch %K usability %K mobile apps %K advance directives %K advance care planning %K mHealth %K mobile health %K palliative care %K mobile phone %D 2022 %7 20.4.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Advance care planning, including advance directives, is an important tool that allows patients to express their preferences for care if they are no longer able to express themselves. We developed Accordons-nous, a smartphone app that informs patients about advance care planning and advance directives, facilitates communication on these sensitive topics, and helps patients express their values and preferences for care. Objective: The first objective of this study is to conduct a usability test of this app. The second objective is to collect users’ critical opinions on the usability and relevance of the tool. Methods: We conducted a usability test by means of a think-aloud method, asking 10 representative patients to complete 7 browsing tasks. We double coded the filmed sessions to obtain descriptive data on task completion (with or without help), time spent, number of clicks, and the types of problems encountered. We assessed the severity of the problems encountered and identified the modifications needed to address these problems. We evaluated the readability of the app using Scolarius, a French equivalent of the Flesch Reading Ease test. By means of a posttest questionnaire, we asked participants to assess the app’s usability (System Usability Scale), relevance (Mobile App Rating Scale, section F), and whether they would recommend the app to the target groups: patients, health professionals, and patients’ caring relatives. Results: Participants completed the 7 think-aloud tasks in 80% (56/70) of the cases without any help from the experimenter, in 16% (11/70) of the cases with some help, and failed in 4% (3/70) of the cases. The analysis of failures and difficulties encountered revealed a series of major usability problems that could be addressed with minor modifications to the app. Accordons-nous obtained high scores on readability (overall score of 87.4 on Scolarius test, corresponding to elementary school level), usability (85.3/100 on System Usability Scale test), relevance (4.3/5 on the Mobile App Rating Scale, section F), and overall subjective endorsement on 3 I would recommend questions (4.7/5). Conclusions: This usability test helped us make the final changes to our app before its official launch. %M 35442206 %R 10.2196/34626 %U https://humanfactors.jmir.org/2022/2/e34626 %U https://doi.org/10.2196/34626 %U http://www.ncbi.nlm.nih.gov/pubmed/35442206 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 4 %P e28696 %T Patient Recruitment System for Clinical Trials: Mixed Methods Study About Requirements at Ten University Hospitals %A Fitzer,Kai %A Haeuslschmid,Renate %A Blasini,Romina %A Altun,Fatma Betül %A Hampf,Christopher %A Freiesleben,Sherry %A Macho,Philipp %A Prokosch,Hans-Ulrich %A Gulden,Christian %+ Core Unit Data Integration Center, University Medicine Greifswald, Walter-Rathenau-Str 48, Greifswald, 17487, Germany, 49 383486 ext 7555, kai.fitzer@uni-greifswald.de %K patient recruitment system %K clinical trial recruitment support system %K recruitment %K patient screening %K requirements %K user needs %K clinical trial %K interview %K survey %K electronic support %K clinical information systems %K eHealth %D 2022 %7 20.4.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Clinical trials are the gold standard for advancing medical knowledge and improving patient outcomes. For their success, an appropriately sized cohort is required. However, patient recruitment remains one of the most challenging aspects of clinical trials. Information technology (IT) support systems—for instance, patient recruitment systems—may help overcome existing challenges and improve recruitment rates, when customized to the user needs and environment. Objective: The goal of our study is to describe the status quo of patient recruitment processes and to identify user requirements for the development of a patient recruitment system. Methods: We conducted a web-based survey with 56 participants as well as semistructured interviews with 33 participants from 10 German university hospitals. Results: We here report the recruitment procedures and challenges of 10 university hospitals. The recruitment process was influenced by diverse factors such as the ward, use of software, and the study inclusion criteria. Overall, clinical staff seemed more involved in patient identification, while the research staff focused on screening tasks. Ad hoc and planned screenings were common. Identifying eligible patients was still associated with significant manual efforts. The recruitment staff used Microsoft Office suite because tailored software were not available. To implement such software, data from disparate sources will need to be made available. We discussed concrete technical challenges concerning patient recruitment systems, including requirements for features, data, infrastructure, and workflow integration, and we contributed to the support of developing a successful system. Conclusions: Identifying eligible patients is still associated with significant manual efforts. To fully make use of the high potential of IT in patient recruitment, many technical and process challenges have to be solved first. We contribute and discuss concrete technical challenges for patient recruitment systems, including requirements for features, data, infrastructure, and workflow integration. %M 35442203 %R 10.2196/28696 %U https://medinform.jmir.org/2022/4/e28696 %U https://doi.org/10.2196/28696 %U http://www.ncbi.nlm.nih.gov/pubmed/35442203 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 9 %N 4 %P e21208 %T Behavioral Health Professionals’ Perceptions on Patient-Controlled Granular Information Sharing (Part 1): Focus Group Study %A Ivanova,Julia %A Tang,Tianyu %A Idouraine,Nassim %A Murcko,Anita %A Whitfield,Mary Jo %A Dye,Christy %A Chern,Darwyn %A Grando,Adela %+ School of Human Evolution and Social Change, Arizona State University, 900 Cady Mall, Tempe, AZ, 85281, United States, 1 480 965 6213, jivanova@asu.edu %K behavioral health professional %K granular information %K granular information sharing %K electronic health record %K integrated health care %K electronic consent tool %D 2022 %7 20.4.2022 %9 Original Paper %J JMIR Ment Health %G English %X Background: Patient-controlled granular information sharing (PC-GIS) allows a patient to select specific health information “granules,” such as diagnoses and medications; choose with whom the information is shared; and decide how the information can be used. Previous studies suggest that health professionals have mixed or concerned opinions about the process and impact of PC-GIS for care and research. Further understanding of behavioral health professionals’ views on PC-GIS are needed for successful implementation and use of this technology. Objective: The aim of this study was to evaluate changes in health professionals’ opinions on PC-GIS before and after a demonstrative case study. Methods: Four focus groups were conducted at two integrated health care facilities: one serious mental illness facility and one general behavioral health facility. A total of 28 participants were given access to outcomes of a previous study where patients had control over medical record sharing. Participants were surveyed before and after focus groups on their views about PC-GIS. Thematic analysis of focus group output was paired with descriptive statistics and exploratory factor analysis of surveys. Results: Behavioral health professionals showed a significant opinion shift toward concern after the focus group intervention, specifically on the topics of patient understanding (P=.001), authorized electronic health record access (P=.03), patient-professional relationship (P=.006), patient control acceptance (P<.001), and patient rights (P=.02). Qualitative methodology supported these results. The themes of professional considerations (2234/4025, 55.5% of codes) and necessity of health information (260/766, 33.9%) identified key aspects of PC-GIS concerns. Conclusions: Behavioral health professionals agreed that a trusting patient-professional relationship is integral to the optimal implementation of PC-GIS, but were concerned about the potential negative impacts of PC-GIS on patient safety and quality of care. %M 35442199 %R 10.2196/21208 %U https://mental.jmir.org/2022/4/e21208 %U https://doi.org/10.2196/21208 %U http://www.ncbi.nlm.nih.gov/pubmed/35442199 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 9 %N 4 %P e18792 %T Behavioral Health Professionals’ Perceptions on Patient-Controlled Granular Information Sharing (Part 2): Focus Group Study %A Ivanova,Julia %A Tang,Tianyu %A Idouraine,Nassim %A Murcko,Anita %A Whitfield,Mary Jo %A Dye,Christy %A Chern,Darwyn %A Grando,Adela %+ School of Human Evolution and Social Change, Arizona State University, 900 Cady Mall, Tempe, AZ, 85281, United States, 1 480 965 6213, jivanova@asu.edu %K behavioral health %K patient information %K granular information %K electronic health record %K integrated health care %K electronic consent tool %D 2022 %7 20.4.2022 %9 Original Paper %J JMIR Ment Health %G English %X Background: Patient-directed selection and sharing of health information “granules” is known as granular information sharing. In a previous study, patients with behavioral health conditions categorized their own health information into sensitive categories (eg, mental health) and chose the health professionals (eg, pharmacists) who should have access to those records. Little is known about behavioral health professionals’ perspectives of patient-controlled granular information sharing (PC-GIS). Objective: This study aimed to assess behavioral health professionals’ (1) understanding of and opinions about PC-GIS; (2) accuracy in assessing redacted medical information; (3) reactions to patient rationale for health data categorization, assignment of sensitivity, and sharing choices; and (4) recommendations to improve PC-GIS. Methods: Four 2-hour focus groups and pre- and postsurveys were conducted at 2 facilities. During the focus groups, outcomes from a previous study on patients’ choices for medical record sharing were discussed. Thematic analysis was applied to focus group transcripts to address study objectives. Results: A total of 28 health professionals were recruited. Over half (14/25, 56%) were unaware or provided incorrect definitions of granular information sharing. After PC-GIS was explained, all professionals demonstrated understanding of the terminology and process. Most (26/32 codes, 81%) recognized that key medical data had been redacted from the study case. A majority (41/62 codes, 66%) found the patient rationale for categorization and data sharing choices to be unclear. Finally, education and other approaches to inform and engage patients in granular information sharing were recommended. Conclusions: This study provides detailed insights from behavioral health professionals on granular information sharing. Outcomes will inform the development, deployment, and evaluation of an electronic consent tool for granular health data sharing. %M 35442213 %R 10.2196/18792 %U https://mental.jmir.org/2022/4/e18792 %U https://doi.org/10.2196/18792 %U http://www.ncbi.nlm.nih.gov/pubmed/35442213 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 10 %N 4 %P e34483 %T Reimagining Connected Care in the Era of Digital Medicine %A Mann,Devin M %A Lawrence,Katharine %+ Department of Population Health, NYU Grossman School of Medicine, 227 E 30th Street, New York, NY, 10016, United States, 1 2122639026, devin.mann@nyulangone.org %K health information technology %K telehealth %K remote patient monitoring %K mobile health %K mHealth %K eHealth %K digital health %K innovation %K process model %K information technology %K digital medicine %K automation %D 2022 %7 15.4.2022 %9 Viewpoint %J JMIR Mhealth Uhealth %G English %X The COVID-19 pandemic accelerated the adoption of remote patient monitoring technology, which offers exciting opportunities for expanded connected care at a distance. However, while the mode of clinicians’ interactions with patients and their health data has transformed, the larger framework of how we deliver care is still driven by a model of episodic care that does not facilitate this new frontier. Fully realizing a transformation to a system of continuous connected care augmented by remote monitoring technology will require a shift in clinicians’ and health systems’ approach to care delivery technology and its associated data volume and complexity. In this article, we present a solution that organizes and optimizes the interaction of automated technologies with human oversight, allowing for the maximal use of data-rich tools while preserving the pieces of medical care considered uniquely human. We review implications of this “augmented continuous connected care” model of remote patient monitoring for clinical practice and offer human-centered design-informed next steps to encourage innovation around these important issues. %M 35436238 %R 10.2196/34483 %U https://mhealth.jmir.org/2022/4/e34483 %U https://doi.org/10.2196/34483 %U http://www.ncbi.nlm.nih.gov/pubmed/35436238 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 4 %P e34954 %T Cluster Analysis of Primary Care Physician Phenotypes for Electronic Health Record Use: Retrospective Cohort Study %A Fong,Allan %A Iscoe,Mark %A Sinsky,Christine A %A Haimovich,Adrian D %A Williams,Brian %A O'Connell,Ryan T %A Goldstein,Richard %A Melnick,Edward %+ National Center for Human Factors in Healthcare, MedStar Health, 3007 Tilden St NW, Washington, DC, 20008, United States, 1 2022449807, allan.fong@medstar.net %K electronic health record %K phenotypes %K cluster analysis %K unsupervised machine learning %K machine learning %K EHR %K primary care %D 2022 %7 15.4.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Electronic health records (EHRs) have become ubiquitous in US office-based physician practices. However, the different ways in which users engage with EHRs remain poorly characterized. Objective: The aim of this study is to explore EHR use phenotypes among ambulatory care physicians. Methods: In this retrospective cohort analysis, we applied affinity propagation, an unsupervised clustering machine learning technique, to identify EHR user types among primary care physicians. Results: We identified 4 distinct phenotype clusters generalized across internal medicine, family medicine, and pediatrics specialties. Total EHR use varied for physicians in 2 clusters with above-average ratios of work outside of scheduled hours. This finding suggested that one cluster of physicians may have worked outside of scheduled hours out of necessity, whereas the other preferred ad hoc work hours. The two remaining clusters represented physicians with below-average EHR time and physicians who spend the largest proportion of their EHR time on documentation. Conclusions: These findings demonstrate the utility of cluster analysis for exploring EHR use phenotypes and may offer opportunities for interventions to improve interface design to better support users’ needs. %M 35275070 %R 10.2196/34954 %U https://medinform.jmir.org/2022/4/e34954 %U https://doi.org/10.2196/34954 %U http://www.ncbi.nlm.nih.gov/pubmed/35275070 %0 Journal Article %@ 2561-6722 %I JMIR Publications %V 5 %N 2 %P e35406 %T Classifying Autism From Crowdsourced Semistructured Speech Recordings: Machine Learning Model Comparison Study %A Chi,Nathan A %A Washington,Peter %A Kline,Aaron %A Husic,Arman %A Hou,Cathy %A He,Chloe %A Dunlap,Kaitlyn %A Wall,Dennis P %+ Division of Systems Medicine, Department of Pediatrics, Stanford University, 3145 Porter Drive, Palo Alto, CA, 94304, United States, 1 650 666 7676, dpwall@stanford.edu %K autism %K mHealth %K machine learning %K artificial intelligence %K speech %K audio %K child %K digital data %K mobile app %K diagnosis %D 2022 %7 14.4.2022 %9 Original Paper %J JMIR Pediatr Parent %G English %X Background: Autism spectrum disorder (ASD) is a neurodevelopmental disorder that results in altered behavior, social development, and communication patterns. In recent years, autism prevalence has tripled, with 1 in 44 children now affected. Given that traditional diagnosis is a lengthy, labor-intensive process that requires the work of trained physicians, significant attention has been given to developing systems that automatically detect autism. We work toward this goal by analyzing audio data, as prosody abnormalities are a signal of autism, with affected children displaying speech idiosyncrasies such as echolalia, monotonous intonation, atypical pitch, and irregular linguistic stress patterns. Objective: We aimed to test the ability for machine learning approaches to aid in detection of autism in self-recorded speech audio captured from children with ASD and neurotypical (NT) children in their home environments. Methods: We considered three methods to detect autism in child speech: (1) random forests trained on extracted audio features (including Mel-frequency cepstral coefficients); (2) convolutional neural networks trained on spectrograms; and (3) fine-tuned wav2vec 2.0—a state-of-the-art transformer-based speech recognition model. We trained our classifiers on our novel data set of cellphone-recorded child speech audio curated from the Guess What? mobile game, an app designed to crowdsource videos of children with ASD and NT children in a natural home environment. Results: The random forest classifier achieved 70% accuracy, the fine-tuned wav2vec 2.0 model achieved 77% accuracy, and the convolutional neural network achieved 79% accuracy when classifying children’s audio as either ASD or NT. We used 5-fold cross-validation to evaluate model performance. Conclusions: Our models were able to predict autism status when trained on a varied selection of home audio clips with inconsistent recording qualities, which may be more representative of real-world conditions. The results demonstrate that machine learning methods offer promise in detecting autism automatically from speech without specialized equipment. %M 35436234 %R 10.2196/35406 %U https://pediatrics.jmir.org/2022/2/e35406 %U https://doi.org/10.2196/35406 %U http://www.ncbi.nlm.nih.gov/pubmed/35436234 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 2 %P e30523 %T Requirements for a Bespoke Intensive Care Unit Dashboard in Response to the COVID-19 Pandemic: Semistructured Interview Study %A Davidson,Brittany %A Ferrer Portillo,Katiuska Mara %A Wac,Marceli %A McWilliams,Chris %A Bourdeaux,Christopher %A Craddock,Ian %+ Department of Electrical & Electronic Engineering, University of Bristol, 1 Cathedral Square, Bristol, BS1 5DD, United Kingdom, 44 07774286342, m.wac@bristol.ac.uk %K intensive care %K critical care %K COVID-19 %K human-centered design %K dashboard %K eHealth %K disease monitoring %K monitoring %K ICU %K design %K development %K interview %D 2022 %7 13.4.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Intensive care units (ICUs) around the world are in high demand due to patients with COVID-19 requiring hospitalization. As researchers at the University of Bristol, we were approached to develop a bespoke data visualization dashboard to assist two local ICUs during the pandemic that will centralize disparate data sources in the ICU to help reduce the cognitive load on busy ICU staff in the ever-evolving pandemic. Objective: The aim of this study was to conduct interviews with ICU staff in University Hospitals Bristol and Weston National Health Service Foundation Trust to elicit requirements for a bespoke dashboard to monitor the high volume of patients, particularly during the COVID-19 pandemic. Methods: We conducted six semistructured interviews with clinical staff to obtain an overview of their requirements for the dashboard and to ensure its ultimate suitability for end users. Interview questions aimed to understand the job roles undertaken in the ICU, potential uses of the dashboard, specific issues associated with managing COVID-19 patients, key data of interest, and any concerns about the introduction of a dashboard into the ICU. Results: From our interviews, we found the following design requirements: (1) a flexible dashboard, where the functionality can be updated quickly and effectively to respond to emerging information about the management of this new disease; (2) a mobile dashboard, which allows staff to move around on wards with a dashboard, thus potentially replacing paper forms to enable detailed and consistent data entry; (3) a customizable and intuitive dashboard, where individual users would be able to customize the appearance of the dashboard to suit their role; (4) real-time data and trend analysis via informative data visualizations that help busy ICU staff to understand a patient’s clinical trajectory; and (5) the ability to manage tasks and staff, tracking both staff and patient movements, handovers, and task monitoring to ensure the highest quality of care. Conclusions: The findings of this study confirm that digital solutions for ICU use would potentially reduce the cognitive load of ICU staff and reduce clinical errors at a time of notably high demand of intensive health care. %M 35038301 %R 10.2196/30523 %U https://humanfactors.jmir.org/2022/2/e30523 %U https://doi.org/10.2196/30523 %U http://www.ncbi.nlm.nih.gov/pubmed/35038301 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 4 %P e29982 %T Predicting Sepsis Mortality in a Population-Based National Database: Machine Learning Approach %A Park,James Yeongjun %A Hsu,Tzu-Chun %A Hu,Jiun-Ruey %A Chen,Chun-Yuan %A Hsu,Wan-Ting %A Lee,Matthew %A Ho,Joshua %A Lee,Chien-Chang %+ Department of Emergency Medicine, National Taiwan University Hospital, Number 7, Chung-Shan South Road, Taipei, 100, Taiwan, 886 223123456, hit3transparency@gmail.com %K sepsis %K mortality %K machine learning %K SuperLearner %D 2022 %7 13.4.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Although machine learning (ML) algorithms have been applied to point-of-care sepsis prognostication, ML has not been used to predict sepsis mortality in an administrative database. Therefore, we examined the performance of common ML algorithms in predicting sepsis mortality in adult patients with sepsis and compared it with that of the conventional context knowledge–based logistic regression approach. Objective: The aim of this study is to examine the performance of common ML algorithms in predicting sepsis mortality in adult patients with sepsis and compare it with that of the conventional context knowledge–based logistic regression approach. Methods: We examined inpatient admissions for sepsis in the US National Inpatient Sample using hospitalizations in 2010-2013 as the training data set. We developed four ML models to predict in-hospital mortality: logistic regression with least absolute shrinkage and selection operator regularization, random forest, gradient-boosted decision tree, and deep neural network. To estimate their performance, we compared our models with the Super Learner model. Using hospitalizations in 2014 as the testing data set, we examined the models’ area under the receiver operating characteristic curve (AUC), confusion matrix results, and net reclassification improvement. Results: Hospitalizations of 923,759 adults were included in the analysis. Compared with the reference logistic regression (AUC: 0.786, 95% CI 0.783-0.788), all ML models showed superior discriminative ability (P<.001), including logistic regression with least absolute shrinkage and selection operator regularization (AUC: 0.878, 95% CI 0.876-0.879), random forest (AUC: 0.878, 95% CI 0.877-0.880), xgboost (AUC: 0.888, 95% CI 0.886-0.889), and neural network (AUC: 0.893, 95% CI 0.891-0.895). All 4 ML models showed higher sensitivity, specificity, positive predictive value, and negative predictive value compared with the reference logistic regression model (P<.001). We obtained similar results from the Super Learner model (AUC: 0.883, 95% CI 0.881-0.885). Conclusions: ML approaches can improve sensitivity, specificity, positive predictive value, negative predictive value, discrimination, and calibration in predicting in-hospital mortality in patients hospitalized with sepsis in the United States. These models need further validation and could be applied to develop more accurate models to compare risk-standardized mortality rates across hospitals and geographic regions, paving the way for research and policy initiatives studying disparities in sepsis care. %M 35416785 %R 10.2196/29982 %U https://www.jmir.org/2022/4/e29982 %U https://doi.org/10.2196/29982 %U http://www.ncbi.nlm.nih.gov/pubmed/35416785 %0 Journal Article %@ 2369-3762 %I JMIR Publications %V 8 %N 2 %P e28965 %T Current and Future Needs for Human Resources for Ethiopia’s National Health Information System: Survey and Forecasting Study %A Tilahun,Binyam %A Endehabtu,Berhanu F %A Gashu,Kassahun D %A Mekonnen,Zeleke A %A Animut,Netsanet %A Belay,Hiwot %A Denboba,Wubshet %A Alemu,Hibret %A Mohammed,Mesoud %A Abate,Biruk %+ Department of Health Informatics, Institute of Public Health, College of Medicine and Health Sciences, University of Gondar, PO Box 196, Gondar, Ethiopia, 251 921013129, berhanufikadie@gmail.com %K forecasting %K human resources %K health information system %K workforce %K Ethiopia %K health informatics %K healthcare professionals %D 2022 %7 12.4.2022 %9 Original Paper %J JMIR Med Educ %G English %X Background: Strengthening the national health information system is one of Ethiopia’s priority transformation agendas. A well-trained and competent workforce is the essential ingredient to a strong health information system. However, this workforce has neither been quantified nor characterized well, and there is no roadmap of required human resources to enhance the national health information system. Objective: We aimed to determine the current state of the health information system workforce and to forecast the human resources needed for the health information system by 2030. Methods: We conducted a survey to estimate the current number of individuals employed in the health information system unit and the turnover rate. Document review and key-informant interviews were used to collect current human resources and available health information system position data from 110 institutions, including the Ministry of Health, federal agencies, regional health bureaus, zonal health departments, district health offices, and health facilities. The Delphi technique was used to forecast human resources required for the health information system in the next ten years: 3 rounds of workshops with experts from the Ministry of Health, universities, agencies, and regional health bureaus were held. In the first expert meeting, we set criteria, which was followed by expert suggestions and feedback. Results: As of April 2020, there were 10,344 health information system professionals working in the governmental health system. Nearly 95% (20/21) of district health offices and 86.7% (26/30) of health centers reported that the current number of health information system positions was inadequate. In the period from June 2015 to June 2019, health information technicians had high turnover (48/244, 19.7%) at all levels of the health system. In the next ten years, we estimate that 50,656 health information system professionals will be needed to effectively implement the Ethiopia's national health information system. Conclusions: Current health information system–related staffing levels were found to be inadequate. To meet the estimated need of 50,656 multidisciplinary health information system professionals by 2030, the Ministry of Health and regional health bureaus, in collaboration with partners and academic institutions, need to work on retaining existing and training additional health information system professionals. %M 35412469 %R 10.2196/28965 %U https://mededu.jmir.org/2022/2/e28965 %U https://doi.org/10.2196/28965 %U http://www.ncbi.nlm.nih.gov/pubmed/35412469 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 4 %P e35418 %T A Sense of Coherence Approach to Improving Patient Experience Using Information Infrastructure Modeling: Design Science Research %A Williams,Patricia A H %A Lovelock,Brendan %A Cabarrus,Javier Antonio %+ Flinders-Cisco Digital Health Design Lab, Flinders Digital Health Research Centre, College of Science and Engineering, Flinders University, Tonsley Campus, GPO Box 2100, Adelaide, 5001, Australia, 61 882012023, trish.williams@flinders.edu.au %K medical informatics %K information infrastructure %K digital hospitals %K patient experience %K implementation %K eHealth %D 2022 %7 12.4.2022 %9 Original Paper %J JMIR Form Res %G English %X Background: Health care provider organizations are complex and dynamic environments. Consequently, how the physical and social environment of such organizations interact with an individual is a primary driver of an individual’s experience. Increasingly, the capabilities required for them to successfully interact with those within their care are critically dependent on the information infrastructure they have in place, which enables people, both patients and staff, to work optimally together to deliver their clinical and operational objectives. Objective: This study aims to design a framework to address the challenge of how to assemble information systems in health care to support an improved sense of coherence for patients, as well as potentially innovate patients’ experiences, by connecting and orchestrating the synergy among people, processes, and systems. Methods: It is necessary to understand the needs of health care providers and patients to address this challenge at a level relevant to information process design and technology development. This paper describes the design science research method used to combine the sense of coherence, which is a core concept within the Antonosky salutogenic approach to health and well-being, with an established information infrastructure maturity framework, demonstrating the coalescence of 2 distinct conceptual perspectives on care delivery. This paper provides an approach to defining a positive and supportive health care experience and linking this to the capabilities of an information- and technology-enabled environment. Results: This research delivers a methodology for describing the patient experience in a form relevant to information infrastructure design, articulating a pathway from information infrastructure to patient experience. It proposes that patient experience can be viewed pragmatically in terms of the established sense of coherence concept, with its ability to identify and guide resources to modulate a patient’s environmental stressors. This research establishes a framework for determining and optimizing the capability of a facility’s information infrastructure to support the sense of coherence defined by the experiences of its patients. Conclusions: This groundbreaking research provides a framework for health care provider organizations to understand and assess the ability of their information infrastructure to support and improve the patient experience. The tool assists providers in defining their technology-dependent operational goals around patient experience and, consequently, in identifying the information capabilities needed to support these goals. The results demonstrate how a fundamental shift in thinking about the use of information infrastructure can transform the patient experience. This study details an approach to describing information infrastructure within an experience-oriented framework that enables the impact of technology on experience to be designed explicitly. The contribution to knowledge is a new perspective on modeling how information infrastructure can contribute to supportive health-promoting environments. Furthermore, it may significantly affect the design and deployment of future digital infrastructures in health care. %M 35307641 %R 10.2196/35418 %U https://formative.jmir.org/2022/4/e35418 %U https://doi.org/10.2196/35418 %U http://www.ncbi.nlm.nih.gov/pubmed/35307641 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 4 %P e22124 %T The Efficacy of Health Information Technology in Supporting Health Equity for Black and Hispanic Patients With Chronic Diseases: Systematic Review %A Senteio,Charles %A Murdock,Paul Joseph %+ Department of Library and Information Science, School of Communication and Information, Rutgers University, 4 Huntington St, New Brunswick, NJ, 08901-1071, United States, 1 8489327586, charles.senteio@rutgers.edu %K chronic disease %K minority health %K technology assessment %K biomedical %K self-management %K systematic review %K mobile phone %D 2022 %7 4.4.2022 %9 Review %J J Med Internet Res %G English %X Background: Racial inequity persists for chronic disease outcomes amid the proliferation of health information technology (HIT) designed to support patients in following recommended chronic disease self-management behaviors (ie, medication behavior, physical activity, and dietary behavior and attending follow-up appointments). Numerous interventions that use consumer-oriented HIT to support self-management have been evaluated, and some of the related literature has focused on racial minorities who experience disparate chronic disease outcomes. However, little is known about the efficacy of these interventions. Objective: This study aims to conduct a systematic review of the literature that describes the efficacy of consumer-oriented HIT interventions designed to support self-management involving African American and Hispanic patients with chronic diseases. Methods: We followed an a priori protocol using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses)-Equity 2012 Extension guidelines for systematic reviews that focus on health equity. Themes of interest included the inclusion and exclusion criteria. We identified 7 electronic databases, created search strings, and conducted the searches. We initially screened results based on titles and abstracts and then performed full-text screening. We then resolved conflicts and extracted relevant data from the included articles. Results: In total, there were 27 included articles. The mean sample size was 640 (SD 209.5), and 52% (14/27) of the articles focused on African American participants, 15% (4/27) of the articles focused on Hispanic participants, and 33% (9/27) included both. Most articles addressed 3 of the 4 self-management behaviors: medication (17/27, 63%), physical activity (17/27, 63%), and diet (16/27, 59%). Only 15% (4/27) of the studies focused on follow-up appointment attendance. All the articles investigated HIT for use at home, whereas 7% (2/27) included use in the hospital. Conclusions: This study addresses a key gap in research that has not sufficiently examined what technology designs and capabilities may be effective for underserved populations in promoting health behavior in concordance with recommendations. %M 35377331 %R 10.2196/22124 %U https://www.jmir.org/2022/4/e22124 %U https://doi.org/10.2196/22124 %U http://www.ncbi.nlm.nih.gov/pubmed/35377331 %0 Journal Article %@ 2561-7605 %I JMIR Publications %V 5 %N 2 %P e35373 %T Predicting Falls in Long-term Care Facilities: Machine Learning Study %A Thapa,Rahul %A Garikipati,Anurag %A Shokouhi,Sepideh %A Hurtado,Myrna %A Barnes,Gina %A Hoffman,Jana %A Calvert,Jacob %A Katzmann,Lynne %A Mao,Qingqing %A Das,Ritankar %+ Dascena Inc., 12333 Sowden Rd Ste B PMB 65148, Houston, TX, 77080-2059, United States, 1 (510) 826 950, sshokouhi@dascena.com %K vital signs %K machine learning %K blood pressure %K skilled nursing facilities %K independent living facilities %K assisted living facilities %K fall prediction %K elderly care %K elderly population %K older adult %K aging %D 2022 %7 1.4.2022 %9 Original Paper %J JMIR Aging %G English %X Background: Short-term fall prediction models that use electronic health records (EHRs) may enable the implementation of dynamic care practices that specifically address changes in individualized fall risk within senior care facilities. Objective: The aim of this study is to implement machine learning (ML) algorithms that use EHR data to predict a 3-month fall risk in residents from a variety of senior care facilities providing different levels of care. Methods: This retrospective study obtained EHR data (2007-2021) from Juniper Communities’ proprietary database of 2785 individuals primarily residing in skilled nursing facilities, independent living facilities, and assisted living facilities across the United States. We assessed the performance of 3 ML-based fall prediction models and the Juniper Communities’ fall risk assessment. Additional analyses were conducted to examine how changes in the input features, training data sets, and prediction windows affected the performance of these models. Results: The Extreme Gradient Boosting model exhibited the highest performance, with an area under the receiver operating characteristic curve of 0.846 (95% CI 0.794-0.894), specificity of 0.848, diagnostic odds ratio of 13.40, and sensitivity of 0.706, while achieving the best trade-off in balancing true positive and negative rates. The number of active medications was the most significant feature associated with fall risk, followed by a resident’s number of active diseases and several variables associated with vital signs, including diastolic blood pressure and changes in weight and respiratory rates. The combination of vital signs with traditional risk factors as input features achieved higher prediction accuracy than using either group of features alone. Conclusions: This study shows that the Extreme Gradient Boosting technique can use a large number of features from EHR data to make short-term fall predictions with a better performance than that of conventional fall risk assessments and other ML models. The integration of routinely collected EHR data, particularly vital signs, into fall prediction models may generate more accurate fall risk surveillance than models without vital signs. Our data support the use of ML models for dynamic, cost-effective, and automated fall predictions in different types of senior care facilities. %M 35363146 %R 10.2196/35373 %U https://aging.jmir.org/2022/2/e35373 %U https://doi.org/10.2196/35373 %U http://www.ncbi.nlm.nih.gov/pubmed/35363146 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 3 %P e36200 %T Physician Burnout and the Electronic Health Record Leading Up to and During the First Year of COVID-19: Systematic Review %A Kruse,Clemens Scott %A Mileski,Michael %A Dray,Gevin %A Johnson,Zakia %A Shaw,Cameron %A Shirodkar,Harsha %+ School of Health Administration, College of Health Professions, Texas State University, 601 University Dr, San Marcos, TX, 78666, United States, 1 5122454462, scottkruse@txstate.edu %K electronic health record %K physician burnout %K quality improvement %K psychiatry %K medical informatics %K COVID-19 %K pandemic %K health informatic %K health care %K health care professional %K health care infrastructure %K health care system %K mental health %K cognitive fatigue %D 2022 %7 31.3.2022 %9 Review %J J Med Internet Res %G English %X Background: Physician burnout was first documented in 1974, and the electronic health record (EHR) has been known to contribute to the symptoms of physician burnout. Authors pondered the extent of this effect, recognizing the increased use of telemedicine during the first year of COVID-19. Objective: The aim of this review was to objectively analyze the literature over the last 5 years for empirical evidence of burnout incident to the EHR and to identify barriers to, facilitators to, and associated patient satisfaction with using the EHR to improve symptoms of burnout. Methods: No human participants were involved in this review; however, 100% of participants in studies analyzed were adult physicians. We queried 4 research databases and 1 targeted journal for studies commensurate with the objective statement from January 1, 2016 through January 31, 2021 (n=25). Results: The hours spent in documentation and workflow are responsible for the sense of loss of autonomy, lack of work-life balance, lack of control of one’s schedule, cognitive fatigue, a general loss of autonomy, and poor relationships with colleagues. Researchers have identified training, local customization of templates and workflow, and the use of scribes as strategies to alleviate the administrative burden of the EHR and decrease symptoms of burnout. Conclusions: The solutions provided in the literature only addressed 2 of the 3 factors (workflow and documentation time) but not the third factor (usability). Practitioners and administrators should focus on the former 2 factors because they are within their sphere of control. EHR vendors should focus on empirical evidence to identify and improve the usability features with the greatest impact. Researchers should design experiments to explore solutions that address all 3 factors of the EHR that contribute to burnout. Trial Registration: PROSPERO International Prospective Register of Systematic Reviews CRD42020201820; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=201820 International Registered Report Identifier (IRRID): RR2-10.2196/15490 %M 35120019 %R 10.2196/36200 %U https://www.jmir.org/2022/3/e36200 %U https://doi.org/10.2196/36200 %U http://www.ncbi.nlm.nih.gov/pubmed/35120019 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 3 %P e29988 %T Cost and Effort Considerations for the Development of Intervention Studies Using Mobile Health Platforms: Pragmatic Case Study %A Thorpe,Dan %A Fouyaxis,John %A Lipschitz,Jessica M %A Nielson,Amy %A Li,Wenhao %A Murphy,Susan A %A Bidargaddi,Niranjan %+ Flinders Digital Health Research Lab, College of Medicine and Public Health, Flinders University, Tonsley Campus, 1284 South Road, Clovelly Park, 5042, Australia, 61 0872215238, dthorpe@flinders.edu.au %K health informatics %K human computer interaction %K digital health %K mobile health %K ecological momentary assessment %K ecological momentary intervention %K behavioral activation %K interventional research %K mobile health costs %D 2022 %7 31.3.2022 %9 Viewpoint %J JMIR Form Res %G English %X Background: The research marketplace has seen a flood of open-source or commercial mobile health (mHealth) platforms that can collect and use user data in real time. However, there is a lack of practical literature on how these platforms are developed, integrated into study designs, and adopted, including important information around cost and effort considerations. Objective: We intend to build critical literacy in the clinician-researcher readership into the cost, effort, and processes involved in developing and operationalizing an mHealth platform, focusing on Intui, an mHealth platform that we developed. Methods: We describe the development of the Intui mHealth platform and general principles of its operationalization across sites. Results: We provide a worked example in the form of a case study. Intui was operationalized in the design of a behavioral activation intervention in collaboration with a mental health service provider. We describe the design specifications of the study site, the developed software, and the cost and effort required to build the final product. Conclusions: Study designs, researcher needs, and technical considerations can impact effort and costs associated with the use of mHealth platforms. Greater transparency from platform developers about the impact of these factors on practical considerations relevant to end users such as clinician-researchers is crucial to increasing critical literacy around mHealth, thereby aiding in the widespread use of these potentially beneficial technologies and building clinician confidence in these tools. %M 35357313 %R 10.2196/29988 %U https://formative.jmir.org/2022/3/e29988 %U https://doi.org/10.2196/29988 %U http://www.ncbi.nlm.nih.gov/pubmed/35357313 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 11 %N 3 %P e34201 %T Leveraging Large-Scale Electronic Health Records and Interpretable Machine Learning for Clinical Decision Making at the Emergency Department: Protocol for System Development and Validation %A Liu,Nan %A Xie,Feng %A Siddiqui,Fahad Javaid %A Ho,Andrew Fu Wah %A Chakraborty,Bibhas %A Nadarajan,Gayathri Devi %A Tan,Kenneth Boon Kiat %A Ong,Marcus Eng Hock %+ Programme in Health Services and Systems Research, Duke-NUS Medical School, 8 College Road, Singapore, 169857, Singapore, 65 66016503, liu.nan@duke-nus.edu.sg %K electronic health records %K machine learning %K clinical decision making %K emergency department %D 2022 %7 25.3.2022 %9 Protocol %J JMIR Res Protoc %G English %X Background: There is a growing demand globally for emergency department (ED) services. An increase in ED visits has resulted in overcrowding and longer waiting times. The triage process plays a crucial role in assessing and stratifying patients’ risks and ensuring that the critically ill promptly receive appropriate priority and emergency treatment. A substantial amount of research has been conducted on the use of machine learning tools to construct triage and risk prediction models; however, the black box nature of these models has limited their clinical application and interpretation. Objective: In this study, we plan to develop an innovative, dynamic, and interpretable System for Emergency Risk Triage (SERT) for risk stratification in the ED by leveraging large-scale electronic health records (EHRs) and machine learning. Methods: To achieve this objective, we will conduct a retrospective, single-center study based on a large, longitudinal data set obtained from the EHRs of the largest tertiary hospital in Singapore. Study outcomes include adverse events experienced by patients, such as the need for an intensive care unit and inpatient death. With preidentified candidate variables drawn from expert opinions and relevant literature, we will apply an interpretable machine learning–based AutoScore to develop 3 SERT scores. These 3 scores can be used at different times in the ED, that is, on arrival, during ED stay, and at admission. Furthermore, we will compare our novel SERT scores with established clinical scores and previously described black box machine learning models as baselines. Receiver operating characteristic analysis will be conducted on the testing cohorts for performance evaluation. Results: The study is currently being conducted. The extracted data indicate approximately 1.8 million ED visits by over 810,000 unique patients. Modelling results are expected to be published in 2022. Conclusions: The SERT scoring system proposed in this study will be unique and innovative because of its dynamic nature and modelling transparency. If successfully validated, our proposed solution will establish a standard for data processing and modelling by taking advantage of large-scale EHRs and interpretable machine learning tools. International Registered Report Identifier (IRRID): DERR1-10.2196/34201 %M 35333179 %R 10.2196/34201 %U https://www.researchprotocols.org/2022/3/e34201 %U https://doi.org/10.2196/34201 %U http://www.ncbi.nlm.nih.gov/pubmed/35333179 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e30130 %T A Patient Outcomes–Driven Feedback Platform for Emergency Medicine Clinicians: Human-Centered Design and Usability Evaluation of Linking Outcomes Of Patients (LOOP) %A Strauss,Alexandra T %A Morgan,Cameron %A El Khuri,Christopher %A Slogeris,Becky %A Smith,Aria G %A Klein,Eili %A Toerper,Matt %A DeAngelo,Anthony %A Debraine,Arnaud %A Peterson,Susan %A Gurses,Ayse P %A Levin,Scott %A Hinson,Jeremiah %+ Department of Medicine, Johns Hopkins University School of Medicine, 600 N Wolfe St, Block 465, Baltimore, MD, 21287, United States, 1 6098280943, atstrauss13@gmail.com %K emergency medicine %K usability %K human-centered design %K health informatics %K feedback %K practice-based learning and improvement %K emergency room %K ER %K platform %K outcomes %K closed-loop learning %D 2022 %7 23.3.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The availability of patient outcomes–based feedback is limited in episodic care environments such as the emergency department. Emergency medicine (EM) clinicians set care trajectories for a majority of hospitalized patients and provide definitive care to an even larger number of those discharged into the community. EM clinicians are often unaware of the short- and long-term health outcomes of patients and how their actions may have contributed. Despite large volumes of patients and data, outcomes-driven learning that targets individual clinician experiences is meager. Integrated electronic health record (EHR) systems provide opportunity, but they do not have readily available functionality intended for outcomes-based learning. Objective: This study sought to unlock insights from routinely collected EHR data through the development of an individualizable patient outcomes feedback platform for EM clinicians. Here, we describe the iterative development of this platform, Linking Outcomes Of Patients (LOOP), under a human-centered design framework, including structured feedback obtained from its use. Methods: This multimodal study consisting of human-centered design studios, surveys (24 physicians), interviews (11 physicians), and a LOOP application usability evaluation (12 EM physicians for ≥30 minutes each) was performed between August 2019 and February 2021. The study spanned 3 phases: (1) conceptual development under a human-centered design framework, (2) LOOP technical platform development, and (3) usability evaluation comparing pre- and post-LOOP feedback gathering practices in the EHR. Results: An initial human-centered design studio and EM clinician surveys revealed common themes of disconnect between EM clinicians and their patients after the encounter. Fundamental postencounter outcomes of death (15/24, 63% respondents identified as useful), escalation of care (20/24, 83%), and return to ED (16/24, 67%) were determined high yield for demonstrating proof-of-concept in our LOOP application. The studio aided the design and development of LOOP, which integrated physicians throughout the design and content iteration. A final LOOP prototype enabled usability evaluation and iterative refinement prior to launch. Usability evaluation compared to status quo (ie, pre-LOOP) feedback gathering practices demonstrated a shift across all outcomes from “not easy” to “very easy” to obtain and from “not confident” to “very confident” in estimating outcomes after using LOOP. On a scale from 0 (unlikely) to 10 (most likely), the users were very likely (9.5) to recommend LOOP to a colleague. Conclusions: This study demonstrates the potential for human-centered design of a patient outcomes–driven feedback platform for individual EM providers. We have outlined a framework for working alongside clinicians with a multidisciplined team to develop and test a tool that augments their clinical experience and enables closed-loop learning. %M 35319469 %R 10.2196/30130 %U https://humanfactors.jmir.org/2022/1/e30130 %U https://doi.org/10.2196/30130 %U http://www.ncbi.nlm.nih.gov/pubmed/35319469 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e29019 %T Barriers to and Facilitators for Acceptance of Comprehensive Clinical Decision Support System–Driven Care Maps for Patients With Thoracic Trauma: Interview Study Among Health Care Providers and Nurses %A Jones,Emma K %A Banks,Alyssa %A Melton,Genevieve B %A Porta,Carolyn M %A Tignanelli,Christopher J %+ Department of Surgery, University of Minnesota, 420 Delaware St SE, Mayo Mail Code 195, Minneapolis, MN, 55455, United States, 1 6126261968, ctignane@umn.edu %K clinical decision support systems %K rib fractures %K trauma %K Unified Theory of Acceptance and Use of Technology %K human computer interaction %D 2022 %7 16.3.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Comprehensive clinical decision support (CDS) care maps can improve the delivery of care and clinical outcomes. However, they are frequently plagued by usability problems and poor user acceptance. Objective: This study aims to characterize factors influencing successful design and use of comprehensive CDS care maps and identify themes associated with end-user acceptance of a thoracic trauma CDS care map earlier in the process than has traditionally been done. This was a planned adaptive redesign stage of a User Acceptance and System Adaptation Design development and implementation strategy for a CDS care map. This stage was based on a previously developed prototype CDS care map guided by the Unified Theory of Acceptance and Use of Technology. Methods: A total of 22 multidisciplinary end users (physicians, advanced practice providers, and nurses) were identified and recruited using snowball sampling. Qualitative interviews were conducted, audio-recorded, and transcribed verbatim. Generation of prespecified codes and the interview guide was informed by the Unified Theory of Acceptance and Use of Technology constructs and investigative team experience. Interviews were blinded and double-coded. Thematic analysis of interview scripts was conducted and yielded descriptive themes about factors influencing the construction and potential use of an acceptable CDS care map. Results: A total of eight dominant themes were identified: alert fatigue (theme 1), automation (theme 2), redundancy (theme 3), minimalistic design (theme 4), evidence based (theme 5), prevent errors (theme 6), comprehensive across the spectrum of disease (theme 7), and malleability (theme 8). Themes 1 to 4 addressed factors directly affecting end users, and themes 5 to 8 addressed factors affecting patient outcomes. More experienced providers prioritized a system that is easy to use. Nurses prioritized a system that incorporated evidence into decision support. Clinicians across specialties, roles, and ages agreed that the amount of extra work generated should be minimal and that the system should help them administer optimal care efficiently. Conclusions: End user feedback reinforces attention toward factors that improve the acceptance and use of a CDS care map for patients with thoracic trauma. Common themes focused on system complexity, the ability of the system to fit different populations and settings, and optimal care provision. Identifying these factors early in the development and implementation process may facilitate user-centered design and improve adoption. %M 35293873 %R 10.2196/29019 %U https://humanfactors.jmir.org/2022/1/e29019 %U https://doi.org/10.2196/29019 %U http://www.ncbi.nlm.nih.gov/pubmed/35293873 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e24680 %T Acceptance of the Use of Artificial Intelligence in Medicine Among Japan’s Doctors and the Public: A Questionnaire Survey %A Tamori,Honoka %A Yamashina,Hiroko %A Mukai,Masami %A Morii,Yasuhiro %A Suzuki,Teppei %A Ogasawara,Katsuhiko %+ Faculty of Health Sciences, Hokkaido University, N12-W5, Kita-ku, Sapporo, 0600812, Japan, 81 11 706 3409, oga@hs.hokudai.ac.jp %K artificial intelligence %K technology acceptance %K surveys and questionnaires %K doctors vs public %D 2022 %7 16.3.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The use of artificial intelligence (AI) in the medical industry promises many benefits, so AI has been introduced to medical practice primarily in developed countries. In Japan, the government is preparing for the rollout of AI in the medical industry. This rollout depends on doctors and the public accepting the technology. Therefore it is necessary to consider acceptance among doctors and among the public. However, little is known about the acceptance of AI in medicine in Japan. Objective: This study aimed to obtain detailed data on the acceptance of AI in medicine by comparing the acceptance among Japanese doctors with that among the Japanese public. Methods: We conducted an online survey, and the responses of doctors and members of the public were compared. AI in medicine was defined as the use of AI to determine diagnosis and treatment without requiring a doctor. A questionnaire was prepared referred to as the unified theory of acceptance and use of technology, a model of behavior toward new technologies. It comprises 20 items, and each item was rated on a five-point scale. Using this questionnaire, we conducted an online survey in 2018 among 399 doctors and 600 members of the public. The sample-wide responses were analyzed, and then the responses of the doctors were compared with those of the public using t tests. Results: Regarding the sample-wide responses (N=999), 653 (65.4%) of the respondents believed, in the future, AI in medicine would be necessary, whereas only 447 (44.7%) expressed an intention to use AI-driven medicine. Additionally, 730 (73.1%) believed that regulatory legislation was necessary, and 734 (73.5%) were concerned about where accountability lies. Regarding the comparison between doctors and the public, doctors (mean 3.43, SD 1.00) were more likely than members of the public (mean 3.23, SD 0.92) to express intention to use AI-driven medicine (P<.001), suggesting that optimism about AI in medicine is greater among doctors compared to the public. Conclusions: Many of the respondents were optimistic about the role of AI in medicine. However, when asked whether they would like to use AI-driven medicine, they tended to give a negative response. This trend suggests that concerns about the lack of regulation and about accountability hindered acceptance. Additionally, the results revealed that doctors were more enthusiastic than members of the public regarding AI-driven medicine. For the successful implementation of AI in medicine, it would be necessary to inform the public and doctors about the relevant laws and to take measures to remove their concerns about them. %M 35293878 %R 10.2196/24680 %U https://humanfactors.jmir.org/2022/1/e24680 %U https://doi.org/10.2196/24680 %U http://www.ncbi.nlm.nih.gov/pubmed/35293878 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 3 %P e28880 %T Using a Convolutional Neural Network and Convolutional Long Short-term Memory to Automatically Detect Aneurysms on 2D Digital Subtraction Angiography Images: Framework Development and Validation %A Liao,JunHua %A Liu,LunXin %A Duan,HaiHan %A Huang,YunZhi %A Zhou,LiangXue %A Chen,LiangYin %A Wang,ChaoHua %+ Department of Neurosurgery, West China Hospital, Sichuan University, No. 37 Guoxue Lane, Wuhou District, Chengdu, 610041, China, 86 18628169123, wangchaohuaHX@163.com %K convolutional neural network %K convolutional long short-term memory %K cerebral aneurysm %K deep learning %D 2022 %7 16.3.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: It is hard to distinguish cerebral aneurysms from overlapping vessels in 2D digital subtraction angiography (DSA) images due to these images’ lack of spatial information. Objective: The aims of this study were to (1) construct a deep learning diagnostic system to improve the ability to detect posterior communicating artery aneurysms on 2D DSA images and (2) validate the efficiency of the deep learning diagnostic system in 2D DSA aneurysm detection. Methods: We proposed a 2-stage detection system. First, we established the region localization stage to automatically locate specific detection regions of raw 2D DSA sequences. Second, in the intracranial aneurysm detection stage, we constructed a bi-input+RetinaNet+convolutional long short-term memory (C-LSTM) framework to compare its performance for aneurysm detection with that of 3 existing frameworks. Each of the frameworks had a 5-fold cross-validation scheme. The receiver operating characteristic curve, the area under the curve (AUC) value, mean average precision, sensitivity, specificity, and accuracy were used to assess the abilities of different frameworks. Results: A total of 255 patients with posterior communicating artery aneurysms and 20 patients without aneurysms were included in this study. The best AUC values of the RetinaNet, RetinaNet+C-LSTM, bi-input+RetinaNet, and bi-input+RetinaNet+C-LSTM frameworks were 0.95, 0.96, 0.92, and 0.97, respectively. The mean sensitivities of the RetinaNet, RetinaNet+C-LSTM, bi-input+RetinaNet, and bi-input+RetinaNet+C-LSTM frameworks and human experts were 89% (range 67.02%-98.43%), 88% (range 65.76%-98.06%), 87% (range 64.53%-97.66%), 89% (range 67.02%-98.43%), and 90% (range 68.30%-98.77%), respectively. The mean specificities of the RetinaNet, RetinaNet+C-LSTM, bi-input+RetinaNet, and bi-input+RetinaNet+C-LSTM frameworks and human experts were 80% (range 56.34%-94.27%), 89% (range 67.02%-98.43%), 86% (range 63.31%-97.24%), 93% (range 72.30%-99.56%), and 90% (range 68.30%-98.77%), respectively. The mean accuracies of the RetinaNet, RetinaNet+C-LSTM, bi-input+RetinaNet, and bi-input+RetinaNet+C-LSTM frameworks and human experts were 84.50% (range 69.57%-93.97%), 88.50% (range 74.44%-96.39%), 86.50% (range 71.97%-95.22%), 91% (range 77.63%-97.72%), and 90% (range 76.34%-97.21%), respectively. Conclusions: According to our results, more spatial and temporal information can help improve the performance of the frameworks. Therefore, the bi-input+RetinaNet+C-LSTM framework had the best performance when compared to that of the other frameworks. Our study demonstrates that our system can assist physicians in detecting intracranial aneurysms on 2D DSA images. %M 35294371 %R 10.2196/28880 %U https://medinform.jmir.org/2022/3/e28880 %U https://doi.org/10.2196/28880 %U http://www.ncbi.nlm.nih.gov/pubmed/35294371 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 3 %P e33250 %T Electronic Health Record–Triggered Research Infrastructure Combining Real-world Electronic Health Record Data and Patient-Reported Outcomes to Detect Benefits, Risks, and Impact of Medication: Development Study %A Hek,Karin %A Rolfes,Leàn %A van Puijenbroek,Eugène P %A Flinterman,Linda E %A Vorstenbosch,Saskia %A van Dijk,Liset %A Verheij,Robert A %+ Nivel, Netherlands Institute for Health Services Research, PO Box 1568, Utrecht, 3500 BN, Netherlands, 31 302729700, k.hek@nivel.nl %K adverse drug reaction %K general practice %K patient-reported outcome %K electronic health record %K overactive bladder %K research infrastructure %K learning health systems %D 2022 %7 16.3.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Real-world data from electronic health records (EHRs) represent a wealth of information for studying the benefits and risks of medical treatment. However, they are limited in scope and should be complemented by information from the patient perspective. Objective: The aim of this study is to develop an innovative research infrastructure that combines information from EHRs with patient experiences reported in questionnaires to monitor the risks and benefits of medical treatment. Methods: We focused on the treatment of overactive bladder (OAB) in general practice as a use case. To develop the Benefit, Risk, and Impact of Medication Monitor (BRIMM) infrastructure, we first performed a requirement analysis. BRIMM’s starting point is routinely recorded general practice EHR data that are sent to the Dutch Nivel Primary Care Database weekly. Patients with OAB were flagged weekly on the basis of diagnoses and prescriptions. They were invited subsequently for participation by their general practitioner (GP), via a trusted third party. Patients received a series of questionnaires on disease status, pharmacological and nonpharmacological treatments, adverse drug reactions, drug adherence, and quality of life. The questionnaires and a dedicated feedback portal were developed in collaboration with a patient association for pelvic-related diseases, Bekkenbodem4All. Participating patients and GPs received feedback. An expert meeting was organized to assess the strengths, weaknesses, opportunities, and threats of the new research infrastructure. Results: The BRIMM infrastructure was developed and implemented. In the Nivel Primary Care Database, 2933 patients with OAB from 27 general practices were flagged. GPs selected 1636 (55.78%) patients who were eligible for the study, of whom 295 (18.0% of eligible patients) completed the first questionnaire. A total of 288 (97.6%) patients consented to the linkage of their questionnaire data with their EHR data. According to experts, the strengths of the infrastructure were the linkage of patient-reported outcomes with EHR data, comparison of pharmacological and nonpharmacological treatments, flexibility of the infrastructure, and low registration burden for GPs. Methodological weaknesses, such as susceptibility to bias, patient selection, and low participation rates among GPs and patients, were seen as weaknesses and threats. Opportunities represent usefulness for policy makers and health professionals, conditional approval of medication, data linkage to other data sources, and feedback to patients. Conclusions: The BRIMM research infrastructure has the potential to assess the benefits and safety of (medical) treatment in real-life situations using a unique combination of EHRs and patient-reported outcomes. As patient involvement is an important aspect of the treatment process, generating knowledge from clinical and patient perspectives is valuable for health care providers, patients, and policy makers. The developed methodology can easily be applied to other treatments and health problems. %M 35293877 %R 10.2196/33250 %U https://medinform.jmir.org/2022/3/e33250 %U https://doi.org/10.2196/33250 %U http://www.ncbi.nlm.nih.gov/pubmed/35293877 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 3 %P e32678 %T Modeling Access Across the Digital Divide for Intersectional Groups Seeking Web-Based Health Information: National Survey %A Medero,Kristina %A Merrill Jr,Kelly %A Ross,Morgan Quinn %+ School of Communication, Ohio State University, 154 N Oval Mall, Columbus, OH, 43210-1132, United States, 1 5313017556, medero.2@osu.edu %K Black %K African American %K first-level digital divide %K health disparities %K home computer %K internet access %K intersectionality %K Latino %K Latine %K Hispanic %K mobile %K online health information seeking %K public computer %K structural equation modeling %K work computer %K mobile phone %D 2022 %7 15.3.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: The digital divide refers to technological disparities based on demographic characteristics (eg, race and ethnicity). Lack of physical access to the internet inhibits online health information seeking (OHIS) and exacerbates health disparities. Research on the digital divide examines where and how people access the internet, whereas research on OHIS investigates how intersectional identities influence OHIS. We combine these perspectives to explicate how unique context–device access pairings operate differently across intersectional identities—particularly racial and ethnic groups—in the domain of OHIS. Objective: This study aims to examine how different types of internet access relate to OHIS for different racial and ethnic groups. We investigate relationships among predisposing characteristics (ie, age, sex, education, and income), internet access (home computer, public computer, work computer, and mobile), health needs, and OHIS. Methods: Analysis was conducted using data from the 2019 Health Information National Trends Survey. Our theoretical model of OHIS explicates the roles of internet access and health needs for racial and ethnic minority groups’ OHIS. Participant responses were analyzed using structural equation modeling. Three separate group structural equation modeling models were specified based on Black, Latine, and White self-categorizations. Results: Overall, predisposing characteristics (ie, age, sex, education, and income) were associated with internet access, health needs, and OHIS; internet access was associated with OHIS; and health needs were associated with OHIS. Home computer and mobile access were most consistently associated with OHIS. Several notable linkages between predisposing characteristics and internet access differed for Black and Latine individuals. Older racial and ethnic minorities tended to access the internet on home and public computers less frequently; home computer access was a stronger predictor of OHIS for White individuals, and mobile access was a stronger predictor of OHIS for non-White individuals. Conclusions: Our findings necessitate a deeper unpacking of how physical internet access, the foundational and multifaceted level of the digital divide, affects specific racial and ethnic groups and their OHIS. We not only find support for prior work on the digital divide but also surface new insights, including distinct impacts of context–device access pairings for OHIS and several relationships that differ between racial and ethnic groups. As such, we propose interventions with an intersectional approach to access to ameliorate the impact of the digital divide. %M 35289761 %R 10.2196/32678 %U https://www.jmir.org/2022/3/e32678 %U https://doi.org/10.2196/32678 %U http://www.ncbi.nlm.nih.gov/pubmed/35289761 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 3 %P e33046 %T Workarounds in Electronic Health Record Systems and the Revised Sociotechnical Electronic Health Record Workaround Analysis Framework: Scoping Review %A Blijleven,Vincent %A Hoxha,Florian %A Jaspers,Monique %+ Center for Marketing & Supply Chain Management, Nyenrode Business Universiteit, Straatweg 25, Breukelen, 3621 BG, Netherlands, 31 630023248, vincentblijleven@gmail.com %K electronic health records %K electronic medical records %K framework %K patient safety %K unintended consequences %K usability %K workarounds %K workflow %D 2022 %7 15.3.2022 %9 Review %J J Med Internet Res %G English %X Background: Electronic health record (EHR) system users devise workarounds to cope with mismatches between workflows designed in the EHR and preferred workflows in practice. Although workarounds appear beneficial at first sight, they frequently jeopardize patient safety, the quality of care, and the efficiency of care. Objective: This review aims to aid in identifying, analyzing, and resolving EHR workarounds; the Sociotechnical EHR Workaround Analysis (SEWA) framework was published in 2019. Although the framework was based on a large case study, the framework still required theoretical validation, refinement, and enrichment. Methods: A scoping literature review was performed on studies related to EHR workarounds published between 2010 and 2021 in the MEDLINE, Embase, CINAHL, Cochrane, or IEEE databases. A total of 737 studies were retrieved, of which 62 (8.4%) were included in the final analysis. Using an analytic framework, the included studies were investigated to uncover the rationales that EHR users have for workarounds, attributes characterizing workarounds, possible scopes, and types of perceived impacts of workarounds. Results: The SEWA framework was theoretically validated and extended based on the scoping review. Extensive support for the pre-existing rationales, attributes, possible scopes, and types of impact was found in the included studies. Moreover, 7 new rationales, 4 new attributes, and 3 new types of impact were incorporated. Similarly, the descriptions of multiple pre-existing rationales for workarounds were refined to describe each rationale more accurately. Conclusions: SEWA is now grounded in the existing body of peer-reviewed empirical evidence on EHR workarounds and, as such, provides a theoretically validated and more complete synthesis of EHR workaround rationales, attributes, possible scopes, and types of impact. The revised SEWA framework can aid researchers and practitioners in a wider range of health care settings to identify, analyze, and resolve workarounds. This will improve user-centered EHR design and redesign, ultimately leading to improved patient safety, quality of care, and efficiency of care. %M 35289752 %R 10.2196/33046 %U https://www.jmir.org/2022/3/e33046 %U https://doi.org/10.2196/33046 %U http://www.ncbi.nlm.nih.gov/pubmed/35289752 %0 Journal Article %@ 2561-9128 %I JMIR Publications %V 5 %N 1 %P e35584 %T A Canadian Weekend Elective Pediatric Surgery Program to Reduce the COVID-19–Related Backlog: Operating Room Ramp-Up After COVID-19 Lockdown Ends—Extra Lists (ORRACLE-Xtra) Implementation Study %A Matava,Clyde %A So,Jeannette %A Williams,RJ %A Kelley,Simon %A , %+ Department of Anesthesia and Pain Medicine, Hospital for Sick Children, 555 University Ave, Toronto, ON, M5G 1X8, Canada, 1 4168137445, clyde.matava@sickkids.ca %K waiting lists %K quality improvement %K patient satisfaction %K COVID-19 %K ambulatory surgery %K pandemics %K Canada %D 2022 %7 15.3.2022 %9 Original Paper %J JMIR Perioper Med %G English %X Background: The COVID-19 pandemic caused by the SARS-COV-2 virus has resulted in unprecedented challenges for the health care system. A decrease of surgical services led to substantial backlogs for time-sensitive scheduled pediatric patients. We designed and implemented a novel pilot weekend surgical quality improvement project called Operating Room Ramp-Up After COVID Lockdown Ends—Extra Lists (ORRACLE-Xtra). Objective: Our overall goals are to increase patient access to surgery (and reduce the wait list), improve operating room efficiencies, and optimize parent and staff experience. Methods: Using the DMAIC (define, measure, analyze, improve, control) framework, we implemented ORRACLE-Xtra in a tertiary care academic pediatric hospital during a quiescent period of the COVID-19 pandemic. We defined process and outcome measures based on provincial targets of out-of-window cases. Parental and staff satisfaction was tracked by surveys. Results: ORRACLE-Xtra led to 247 patients receiving surgery during the pilot period, resulting in a 5% decrease in the total number of patients on our wait list with Paediatric Canadian Access Targets for Surgery IV (147/247, 59.5%), with 38.1% (94/247) out-of-window of provincial targets. Most of the process and outcome measures were met or exceeded. Overall parental satisfaction was at 95.8% (110/121), with 79% (64/81) of staff reporting satisfaction with working weekends. Conclusions: Through the ORRACLE-Xtra pilot program, we have shown that hospitals impacted by COVID-19 can reduce the surgical backlog using innovative models of service delivery in a Canadian context. Sustained funding is critical to achieving more meaningful reductions in wait times for scheduled surgeries over the longer term and needs to be balanced with staff well-being. %M 34887242 %R 10.2196/35584 %U https://periop.jmir.org/2022/1/e35584 %U https://doi.org/10.2196/35584 %U http://www.ncbi.nlm.nih.gov/pubmed/34887242 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 3 %P e29967 %T Machine-Aided Self-diagnostic Prediction Models for Polycystic Ovary Syndrome: Observational Study %A Zigarelli,Angela %A Jia,Ziyang %A Lee,Hyunsun %+ Department of Mathematics and Statistics, University of Massachusetts Amherst, 100 Carlson Ave, Newton, MA, 02459, United States, 1 6172431886, hyunsunlee@umass.edu %K Polycystic Ovary Syndrome (PCOS) %K prediction %K machine learning %K self-diagnosis %K principal component analysis %K clustering %K CatBoost %K SHAP values %K subgroup study %D 2022 %7 15.3.2022 %9 Early Reports %J JMIR Form Res %G English %X Background: Artificial intelligence and digital health care have substantially advanced to improve and enhance medical diagnosis and treatment during the prolonged period of the COVID-19 global pandemic. In this study, we discuss the development of prediction models for the self-diagnosis of polycystic ovary syndrome (PCOS) using machine learning techniques. Objective: We aim to develop self-diagnostic prediction models for PCOS in potential patients and clinical providers. For potential patients, the prediction is based only on noninvasive measures such as anthropomorphic measures, symptoms, age, and other lifestyle factors so that the proposed prediction tool can be conveniently used without any laboratory or ultrasound test results. For clinical providers who can access patients’ medical test results, prediction models using all predictor variables can be adopted to help health providers diagnose patients with PCOS. We compare both prediction models using various error metrics. We call the former model the patient model and the latter, the provider model throughout this paper. Methods: In this retrospective study, a publicly available data set of 541 women’s health information collected from 10 different hospitals in Kerala, India, including PCOS status, was acquired and used for analysis. We adopted the CatBoost method for classification, K-fold cross-validation for estimating the performance of models, and SHAP (Shapley Additive Explanations) values to explain the importance of each variable. In our subgroup study, we used k-means clustering and Principal Component Analysis to split the data set into 2 distinct BMI subgroups and compared the prediction results as well as the feature importance between the 2 subgroups. Results: We achieved 81% to 82.5% prediction accuracy of PCOS status without any invasive measures in the patient models and achieved 87.5% to 90.1% prediction accuracy using both noninvasive and invasive predictor variables in the provider models. Among noninvasive measures, variables including acanthosis nigricans, acne, hirsutism, irregular menstrual cycle, length of menstrual cycle, weight gain, fast food consumption, and age were more important in the models. In medical test results, the numbers of follicles in the right and left ovaries and anti-Müllerian hormone were ranked highly in feature importance. We also reported more detailed results in a subgroup study. Conclusions: The proposed prediction models are ultimately expected to serve as a convenient digital platform with which users can acquire pre- or self-diagnosis and counsel for the risk of PCOS, with or without obtaining medical test results. It will enable women to conveniently access the platform at home without delay before they seek further medical care. Clinical providers can also use the proposed prediction tool to help diagnose PCOS in women. %M 35289757 %R 10.2196/29967 %U https://formative.jmir.org/2022/3/e29967 %U https://doi.org/10.2196/29967 %U http://www.ncbi.nlm.nih.gov/pubmed/35289757 %0 Journal Article %@ 2563-3570 %I JMIR Publications %V 3 %N 1 %P e31536 %T The Easy-to-Use SARS-CoV-2 Assembler for Genome Sequencing: Development Study %A Rueca,Martina %A Giombini,Emanuela %A Messina,Francesco %A Bartolini,Barbara %A Di Caro,Antonino %A Capobianchi,Maria Rosaria %A Gruber,Cesare EM %+ Laboratory of Microbiology and Biological Bank, National Institute for Infectious Diseases “Lazzaro Spallanzani”, Istituto di Ricovero e Cura a Carattere Scientifico, Via Portuense 292, Rome, 00149, Italy, 39 0655170668, francesco.messina@inmi.it %K SARS-CoV-2 genome %K bioinformatics tool %K NGS data analysis %K COVID-19 %K genome %K health informatics %K bioinformatic %K digital tools %K algorithms %D 2022 %7 14.3.2022 %9 Original Paper %J JMIR Bioinform Biotech %G English %X Background: Early sequencing and quick analysis of the SARS-CoV-2 genome have contributed to the understanding of the dynamics of COVID-19 epidemics and in designing countermeasures at a global level. Objective: Amplicon-based next-generation sequencing (NGS) methods are widely used to sequence the SARS-CoV-2 genome and to identify novel variants that are emerging in rapid succession as well as harboring multiple deletions and amino acid–changing mutations. Methods: To facilitate the analysis of NGS sequencing data obtained from amplicon-based sequencing methods, here, we propose an easy-to-use SARS-CoV-2 genome assembler: the Easy-to-use SARS-CoV-2 Assembler (ESCA) pipeline. Results: Our results have shown that ESCA could perform high-quality genome assembly from Ion Torrent and Illumina raw data and help the user in easily correct low-coverage regions. Moreover, ESCA includes the possibility of comparing assembled genomes of multisample runs through an easy table format. Conclusions: In conclusion, ESCA automatically furnished a variant table output file, fundamental to rapidly recognizing variants of interest. Our pipeline could be a useful method for obtaining a complete, rapid, and accurate analysis even with minimal knowledge in bioinformatics. %M 35309411 %R 10.2196/31536 %U https://bioinform.jmir.org/2022/1/e31536 %U https://doi.org/10.2196/31536 %U http://www.ncbi.nlm.nih.gov/pubmed/35309411 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 11 %N 3 %P e36201 %T Linking Electronic Health Records and In-Depth Interviews to Inform Efforts to Integrate Social Determinants of Health Into Health Care Delivery: Protocol for a Qualitative Research Study %A Hirsch,Annemarie %A Durden,T Elizabeth %A Silva,Jennifer %+ Department of Population Health Sciences, Geisinger, 100 N. Academy Avenue, Danville, PA, 17822, United States, 1 267 626 8110, aghirsch@geisinger.edu %K electronic health records %K social determinants of health %K poverty %K rural %K qualitative %K health system %D 2022 %7 11.3.2022 %9 Protocol %J JMIR Res Protoc %G English %X Background: Health systems are attempting to capture social determinants of health (SDoH) in electronic health records (EHR) and use these data to adjust care plans. To date, however, methods for identifying social needs, which are the SDoH prioritized by patients, have been underexplored, and there is little guidance as to how clinicians should act on SDoH data when caring for patients. Moreover, the unintended consequences of collecting and responding to SDoH are poorly understood. Objective: The objective of this study is to use two data sources, EHR data and patient interviews, to describe divergences between the EHR and patient experiences that could help identify gaps in the documentation of SDoH in the EHR; highlight potential missed opportunities for addressing social needs, and identify unintended consequences of efforts to integrate SDoH into clinical care. Methods: We are conducting a qualitative study that merges discrete and free-text data from EHRs with in-depth interviews with women residing in rural, socioeconomically deprived communities in the Mid-Atlantic region of the United States. Participants had to confirm that they had at least one visit with the large health system that serves the region. Interviews with the women included questions regarding health, interaction with the health system, and social needs. Next, with consent, we extracted discrete data (eg, diagnoses and medication orders) for each participant and free-text clinician notes from this health system’s EHRs between 1996 and the year of the interview. We used a standardized protocol to create an EHR narrative, a free-text summary of the EHR data. We used NVivo to identify themes in the interviews and the EHR narratives. Results: To date, we have interviewed 88 women, including 51 White women, 19 Black women, 14 Latina women, 2 mixed Black and Latina women, and 2 Asian Pacific women. We have completed the EHR narratives on 66 women. The women range in age from 18 to 90 years. We found corresponding EHR data on all but 4 of the interview participants. Participants had contact with a wide range of clinical departments (eg, psychiatry, neurology, and infectious disease) and received care in various clinical settings (eg, primary care clinics, emergency departments, and inpatient hospitalizations). A preliminary review of the EHR narratives revealed that the clinician notes were a source of data on a range of SDoH but did not always reflect the social needs that participants described in the interviews. Conclusions: This study will provide unique insight into the demands and consequences of integrating SDoH into clinical care. This work comes at a pivotal point in time, as health systems, payors, and policymakers accelerate attempts to deliver care within the context of social needs. International Registered Report Identifier (IRRID): DERR1-10.2196/36201 %M 35275090 %R 10.2196/36201 %U https://www.researchprotocols.org/2022/3/e36201 %U https://doi.org/10.2196/36201 %U http://www.ncbi.nlm.nih.gov/pubmed/35275090 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 3 %P e31684 %T A National Network of Safe Havens: Scottish Perspective %A Gao,Chuang %A McGilchrist,Mark %A Mumtaz,Shahzad %A Hall,Christopher %A Anderson,Lesley Ann %A Zurowski,John %A Gordon,Sharon %A Lumsden,Joanne %A Munro,Vicky %A Wozniak,Artur %A Sibley,Michael %A Banks,Christopher %A Duncan,Chris %A Linksted,Pamela %A Hume,Alastair %A Stables,Catherine L %A Mayor,Charlie %A Caldwell,Jacqueline %A Wilde,Katie %A Cole,Christian %A Jefferson,Emily %+ Health Informatics Centre, Ninewells Hospital & Medical School, University of Dundee, Mail Box 15, , Dundee, DD1 9SY, United Kingdom, 44 (0)1382 383943, e.r.jefferson@dundee.ac.uk %K electronic health records %K Safe Haven %K data governance %D 2022 %7 9.3.2022 %9 Viewpoint %J J Med Internet Res %G English %X For over a decade, Scotland has implemented and operationalized a system of Safe Havens, which provides secure analytics platforms for researchers to access linked, deidentified electronic health records (EHRs) while managing the risk of unauthorized reidentification. In this paper, a perspective is provided on the state-of-the-art Scottish Safe Haven network, including its evolution, to define the key activities required to scale the Scottish Safe Haven network’s capability to facilitate research and health care improvement initiatives. A set of processes related to EHR data and their delivery in Scotland have been discussed. An interview with each Safe Haven was conducted to understand their services in detail, as well as their commonalities. The results show how Safe Havens in Scotland have protected privacy while facilitating the reuse of the EHR data. This study provides a common definition of a Safe Haven and promotes a consistent understanding among the Scottish Safe Haven network and the clinical and academic research community. We conclude by identifying areas where efficiencies across the network can be made to meet the needs of population-level studies at scale. %M 35262495 %R 10.2196/31684 %U https://www.jmir.org/2022/3/e31684 %U https://doi.org/10.2196/31684 %U http://www.ncbi.nlm.nih.gov/pubmed/35262495 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 3 %P e30328 %T Deriving Weight From Big Data: Comparison of Body Weight Measurement–Cleaning Algorithms %A Evans,Richard %A Burns,Jennifer %A Damschroder,Laura %A Annis,Ann %A Freitag,Michelle B %A Raffa,Susan %A Wiitala,Wyndy %+ Center for Clinical Management Research, Veterans Health Administration, 2215 Fuller Road, Mail Stop 152, Ann Arbor, MI, 48105, United States, 1 248 910 3441, Richard.Evans8@va.gov %K veterans %K weight %K algorithms %K obesity %K measurement %K electronic health record %D 2022 %7 9.3.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Patient body weight is a frequently used measure in biomedical studies, yet there are no standard methods for processing and cleaning weight data. Conflicting documentation on constructing body weight measurements presents challenges for research and program evaluation. Objective: In this study, we aim to describe and compare methods for extracting and cleaning weight data from electronic health record databases to develop guidelines for standardized approaches that promote reproducibility. Methods: We conducted a systematic review of studies published from 2008 to 2018 that used Veterans Health Administration electronic health record weight data and documented the algorithms for constructing patient weight. We applied these algorithms to a cohort of veterans with at least one primary care visit in 2016. The resulting weight measures were compared at the patient and site levels. Results: We identified 496 studies and included 62 (12.5%) that used weight as an outcome. Approximately 48% (27/62) included a replicable algorithm. Algorithms varied from cutoffs of implausible weights to complex models using measures within patients over time. We found differences in the number of weight values after applying the algorithms (71,961/1,175,995, 6.12% to 1,175,177/1,175,995, 99.93% of raw data) but little difference in average weights across methods (93.3, SD 21.0 kg to 94.8, SD 21.8 kg). The percentage of patients with at least 5% weight loss over 1 year ranged from 9.37% (4933/52,642) to 13.99% (3355/23,987). Conclusions: Contrasting algorithms provide similar results and, in some cases, the results are not different from using raw, unprocessed data despite algorithm complexity. Studies using point estimates of weight may benefit from a simple cleaning rule based on cutoffs of implausible values; however, research questions involving weight trajectories and other, more complex scenarios may benefit from a more nuanced algorithm that considers all available weight data. %M 35262492 %R 10.2196/30328 %U https://medinform.jmir.org/2022/3/e30328 %U https://doi.org/10.2196/30328 %U http://www.ncbi.nlm.nih.gov/pubmed/35262492 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 3 %P e32800 %T Tackling the Burden of Electronic Health Record Use Among Physicians in a Mental Health Setting: Physician Engagement Strategy %A Tajirian,Tania %A Jankowicz,Damian %A Lo,Brian %A Sequeira,Lydia %A Strudwick,Gillian %A Almilaji,Khaled %A Stergiopoulos,Vicky %+ Information Management Group, Centre for Addiction and Mental Health, 100 Stokes Street, Toronto, ON, M6J 1H4, Canada, 1 416 535 8501 ext 30515, Tania.Tajirian@camh.ca %K burnout %K organizational strategy %K electronic health record use %K clinical informatics %K medical informatics %D 2022 %7 8.3.2022 %9 Viewpoint %J J Med Internet Res %G English %X The burden associated with using the electronic health record system continues to be a critical issue for physicians and is potentially contributing to physician burnout. At a large academic mental health hospital in Canada, we recently implemented a Physician Engagement Strategy focused on reducing the burden of electronic health record use through close collaboration with clinical leadership, information technology leadership, and physicians. Built on extensive stakeholder consultation, this strategy highlights initiatives that we have implemented (or will be implementing in the near future) under four components: engage, inspire, change, and measure. In this viewpoint paper, we share our process of developing and implementing the Physician Engagement Strategy and discuss the lessons learned and implications of this work. %M 35258473 %R 10.2196/32800 %U https://www.jmir.org/2022/3/e32800 %U https://doi.org/10.2196/32800 %U http://www.ncbi.nlm.nih.gov/pubmed/35258473 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 3 %P e37419 %T Authors’ Reply to: Interpretation Bias Toward the Positive Impacts of Digital Interventions in Health Care. Comment on “Value of the Electronic Medical Record for Hospital Care: Update From the Literature” %A Stausberg,Jürgen %A Uslu,Aykut %+ Institute for Medical Informatics, Biometry and Epidemiology, University Hospital Essen, University Duisburg-Essen, Hufelandstrasse 55, Essen, 45122, Germany, 49 201 723 77201, stausberg@ekmed.de %K cost analysis %K costs and cost analyses %K economic advantage %K electronic medical records %K electronic records %K health care %K hospitals %K computerized medical records systems %K quality of health care %K secondary data %D 2022 %7 4.3.2022 %9 Letter to the Editor %J J Med Internet Res %G English %X   %M 35254272 %R 10.2196/37419 %U https://www.jmir.org/2022/3/e37419 %U https://doi.org/10.2196/37419 %U http://www.ncbi.nlm.nih.gov/pubmed/35254272 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 3 %P e37208 %T Interpretation Bias Toward the Positive Impacts of Digital Interventions in Health Care. Comment on “Value of the Electronic Medical Record for Hospital Care: Update From the Literature” %A Shakibaei Bonakdeh,Erfan %+ Department of Management, Monash Business School, Monash University, 900 Dandenong Rd, Caulfield East, Melbourne, VIC, 3145, Australia, 61 1399032000, erfan.shakibaeibonakdeh@monash.edu %K cost analysis %K costs and cost analyses %K economic advantage %K electronic medical records %K electronic records %K health care %K hospitals %K computerized medical records system %K quality of health care %K secondary data %D 2022 %7 4.3.2022 %9 Letter to the Editor %J J Med Internet Res %G English %X   %M 35254276 %R 10.2196/37208 %U https://www.jmir.org/2022/3/e37208 %U https://doi.org/10.2196/37208 %U http://www.ncbi.nlm.nih.gov/pubmed/35254276 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 3 %P e25477 %T A Comparison of Census and Cohort Sampling Models for the Longitudinal Collection of User-Reported Data in the Maternity Care Pathway: Mixed Methods Study %A Jamieson Gilmore,Kendall %A Bonciani,Manila %A Vainieri,Milena %+ Management and Healthcare Laboratory, Department of Economics and Management in the era of Data Science, Institute of Management, Sant'Anna Scuola Superiore, 33 Piazza Martiri della Libertà, Pisa, 56127, Italy, 39 050 883111, k.jamiesongilmore@santannapisa.it %K longitudinal studies %K mothers %K pregnancy %K survival analysis %K patient-reported outcome measures %K patient-reported experience measures %K surveys %K maternity %K postpartum %K online %K digital health %K digital collection %D 2022 %7 4.3.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Typical measures of maternity performance remain focused on the technical elements of birth, especially pathological elements, with insufficient measurement of nontechnical measures and those collected pre- and postpartum. New technologies allow for patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) to be collected from large samples at multiple time points, which can be considered alongside existing administrative sources; however, such models are not widely implemented or evaluated. Since 2018, a longitudinal, personalized, and integrated user-reported data collection process for the maternal care pathway has been used in Tuscany, Italy. This model has been through two methodological iterations. Objective: The aim of this study was to compare and contrast two sampling models of longitudinal user-reported data for the maternity care pathway, exploring factors influencing participation, cost, and suitability of the models for different stakeholders. Methods: Data were collected by two modes: (1) “cohort” recruitment at the birth hospital of a predetermined sample size and (2) continuous, ongoing “census” recruitment of women at the first midwife appointment. Surveys were used to collect experiential and outcome data related to existing services. Women were included who passed 12 months after initial enrollment, meaning that they either received the surveys issued after that interval or dropped out in the intervening period. Data were collected from women in Tuscany, Italy, between September 2018 and July 2020. The total sample included 7784 individuals with 38,656 observations. The two models of longitudinal collection of user-reported data were analyzed using descriptive statistics, survival analysis, cost comparison, and a qualitative review. Results: Cohort sampling provided lower initial participation than census sampling, although very high subsequent response rates (87%) were obtained 1 year after enrollment. Census sampling had higher initial participation, but greater dropout (up to 45% at 1 year). Both models showed high response rates for online surveys. There were nonproportional dropout hazards over time. There were higher rates of dropout for women with foreign nationality (hazard ratio [HR] 1.88, P<.001), and lower rates of dropout for those who had a higher level of education (HR 0.77 and 0.61 for women completing high school and college, respectively; P<.001), were employed (HR 0.87, P=.01), in a relationship (HR 0.84, P=.04), and with previous pregnancies (HR 0.86, P=.002). The census model was initially more expensive, albeit with lower repeat costs and could become cheaper if repeated more than six times. Conclusions: The digital collection of user-reported data enables high response rates to targeted surveys in the maternity care pathway. The point at which pregnant women or mothers are recruited is relevant for response rates and sample bias. The census model of continuous enrollment and real-time data availability offers a wider set of potential benefits, but at an initially higher cost and with the requirement for more substantial data translation and managerial capacity to make use of such data. %M 35254268 %R 10.2196/25477 %U https://medinform.jmir.org/2022/3/e25477 %U https://doi.org/10.2196/25477 %U http://www.ncbi.nlm.nih.gov/pubmed/35254268 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 3 %P e31760 %T Predicting High Flow Nasal Cannula Failure in an Intensive Care Unit Using a Recurrent Neural Network With Transfer Learning and Input Data Perseveration: Retrospective Analysis %A Pappy,George %A Aczon,Melissa %A Wetzel,Randall %A Ledbetter,David %+ The Laura P. and Leland K. Whittier Virtual PICU, Children's Hospital Los Angeles, 4650 Sunset Blvd, Los Angeles, CA, 90027, United States, 1 323 717 4515, dledbetter@chla.usc.edu %K high flow nasal cannula %K HFNC failure %K predictive model %K deep learning %K transfer learning %K LSTM %K RNN %K input data perseveration %D 2022 %7 3.3.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: High flow nasal cannula (HFNC) provides noninvasive respiratory support for children who are critically ill who may tolerate it more readily than other noninvasive ventilation (NIV) techniques such as bilevel positive airway pressure and continuous positive airway pressure. Moreover, HFNC may preclude the need for mechanical ventilation (intubation). Nevertheless, NIV or intubation may ultimately be necessary for certain patients. Timely prediction of HFNC failure can provide an indication for increasing respiratory support. Objective: The aim of this study is to develop and compare machine learning (ML) models to predict HFNC failure. Methods: A retrospective study was conducted using the Virtual Pediatric Intensive Care Unit database of electronic medical records of patients admitted to a tertiary pediatric intensive care unit between January 2010 and February 2020. Patients aged <19 years, without apnea, and receiving HFNC treatment were included. A long short-term memory (LSTM) model using 517 variables (vital signs, laboratory data, and other clinical parameters) was trained to generate a continuous prediction of HFNC failure, defined as escalation to NIV or intubation within 24 hours of HFNC initiation. For comparison, 7 other models were trained: a logistic regression (LR) using the same 517 variables, another LR using only 14 variables, and 5 additional LSTM-based models using the same 517 variables as the first LSTM model and incorporating additional ML techniques (transfer learning, input perseveration, and ensembling). Performance was assessed using the area under the receiver operating characteristic (AUROC) curve at various times following HFNC initiation. The sensitivity, specificity, and positive and negative predictive values of predictions at 2 hours after HFNC initiation were also evaluated. These metrics were also computed for a cohort with primarily respiratory diagnoses. Results: A total of 834 HFNC trials (455 [54.6%] training, 173 [20.7%] validation, and 206 [24.7%] test) met the inclusion criteria, of which 175 (21%; training: 103/455, 22.6%; validation: 30/173, 17.3%; test: 42/206, 20.4%) escalated to NIV or intubation. The LSTM models trained with transfer learning generally performed better than the LR models, with the best LSTM model achieving an AUROC of 0.78 versus 0.66 for the 14-variable LR and 0.71 for the 517-variable LR 2 hours after initiation. All models except for the 14-variable LR achieved higher AUROCs in the respiratory cohort than in the general intensive care unit population. Conclusions: ML models trained using electronic medical record data were able to identify children at risk of HFNC failure within 24 hours of initiation. LSTM models that incorporated transfer learning, input data perseveration, and ensembling showed improved performance compared with the LR and standard LSTM models. %M 35238792 %R 10.2196/31760 %U https://medinform.jmir.org/2022/3/e31760 %U https://doi.org/10.2196/31760 %U http://www.ncbi.nlm.nih.gov/pubmed/35238792 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 11 %N 3 %P e30956 %T Reporting of Model Performance and Statistical Methods in Studies That Use Machine Learning to Develop Clinical Prediction Models: Protocol for a Systematic Review %A Weaver,Colin George Wyllie %A Basmadjian,Robert B %A Williamson,Tyler %A McBrien,Kerry %A Sajobi,Tolu %A Boyne,Devon %A Yusuf,Mohamed %A Ronksley,Paul Everett %+ Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, Teaching, Research, and Wellness Building 3E18B, 3280 Hospital Drive NW, Calgary, AB, T2N 4Z6, Canada, 1 403 220 8820, peronksl@ucalgary.ca %K machine learning %K clinical prediction %K research reporting %K statistics %K research methods %K clinical prediction models %K artificial intelligence %K modeling %K eHealth %K digital medicine %K prediction %D 2022 %7 3.3.2022 %9 Protocol %J JMIR Res Protoc %G English %X Background: With the growing excitement of the potential benefits of using machine learning and artificial intelligence in medicine, the number of published clinical prediction models that use these approaches has increased. However, there is evidence (albeit limited) that suggests that the reporting of machine learning–specific aspects in these studies is poor. Further, there are no reviews assessing the reporting quality or broadly accepted reporting guidelines for these aspects. Objective: This paper presents the protocol for a systematic review that will assess the reporting quality of machine learning–specific aspects in studies that use machine learning to develop clinical prediction models. Methods: We will include studies that use a supervised machine learning algorithm to develop a prediction model for use in clinical practice (ie, for diagnosis or prognosis of a condition or identification of candidates for health care interventions). We will search MEDLINE for studies published in 2019, pseudorandomly sort the records, and screen until we obtain 100 studies that meet our inclusion criteria. We will assess reporting quality with a novel checklist developed in parallel with this review, which includes content derived from existing reporting guidelines, textbooks, and consultations with experts. The checklist will cover 4 key areas where the reporting of machine learning studies is unique: modelling steps (order and data used for each step), model performance (eg, reporting the performance of each model compared), statistical methods (eg, describing the tuning approach), and presentation of models (eg, specifying the predictors that contributed to the final model). Results: We completed data analysis in August 2021 and are writing the manuscript. We expect to submit the results to a peer-reviewed journal in early 2022. Conclusions: This review will contribute to more standardized and complete reporting in the field by identifying areas where reporting is poor and can be improved. Trial Registration: PROSPERO International Prospective Register of Systematic Reviews CRD42020206167; https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=206167 International Registered Report Identifier (IRRID): RR1-10.2196/30956 %M 35238322 %R 10.2196/30956 %U https://www.researchprotocols.org/2022/3/e30956 %U https://doi.org/10.2196/30956 %U http://www.ncbi.nlm.nih.gov/pubmed/35238322 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 3 %P e34098 %T An Innovative Telemedical Network to Improve Infectious Disease Management in Critically Ill Patients and Outpatients (TELnet@NRW): Stepped-Wedge Cluster Randomized Controlled Trial %A Marx,Gernot %A Greiner,Wolfgang %A Juhra,Christian %A Elkenkamp,Svenja %A Gensorowsky,Daniel %A Lemmen,Sebastian %A Englbrecht,Jan %A Dohmen,Sandra %A Gottschalk,Antje %A Haverkamp,Miriam %A Hempen,Annette %A Flügel-Bleienheuft,Christian %A Bause,Daniela %A Schulze-Steinen,Henna %A Rademacher,Susanne %A Kistermann,Jennifer %A Hoch,Stefan %A Beckmann,Hans-Juergen %A Lanckohr,Christian %A Lowitsch,Volker %A Peine,Arne %A Juzek-Kuepper,Fabian %A Benstoem,Carina %A Sperling,Kathrin %A Deisz,Robert %+ Department of Intensive Care Medicine and Intermediate Care, Medical Faculty RWTH Aachen, Pauwelsstr. 30, Aachen, 52074, Germany, 49 2418080444, gmarx@ukaachen.de %K telemedicine %K infectious disease medicine %K sepsis %K evidence-based medicine %K eHealth %D 2022 %7 2.3.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Evidence-based infectious disease and intensive care management is more relevant than ever. Medical expertise in the two disciplines is often geographically limited to university institutions. In addition, the interconnection between inpatient and outpatient care is often insufficient (eg, no shared electronic health record and no digital transfer of patient findings). Objective: This study aims to establish and evaluate a telemedical inpatient-outpatient network based on expert teleconsultations to increase treatment quality in intensive care medicine and infectious diseases. Methods: We performed a multicenter, stepped-wedge cluster randomized trial (February 2017 to January 2020) to establish a telemedicine inpatient-outpatient network among university hospitals, hospitals, and outpatient physicians in North Rhine-Westphalia, Germany. Patients aged ≥18 years in the intensive care unit or consulting with a physician in the outpatient setting were eligible. We provided expert knowledge from intensivists and infectious disease specialists through advanced training courses and expert teleconsultations with 24/7/365 availability on demand respectively once per week to enhance treatment quality. The primary outcome was adherence to the 10 Choosing Wisely recommendations for infectious disease management. Guideline adherence was analyzed using binary logistic regression models. Results: Overall, 159,424 patients (10,585 inpatients and 148,839 outpatients) from 17 hospitals and 103 outpatient physicians were included. There was a significant increase in guideline adherence in the management of Staphylococcus aureus infections (odds ratio [OR] 4.00, 95% CI 1.83-9.20; P<.001) and in sepsis management in critically ill patients (OR 6.82, 95% CI 1.27-56.61; P=.04). There was a statistically nonsignificant decrease in sepsis-related mortality from 29% (19/66) in the control group to 23.8% (50/210) in the intervention group. Furthermore, the extension of treatment with prophylactic antibiotics after surgery was significantly less likely (OR 9.37, 95% CI 1.52-111.47; P=.04). Patients treated by outpatient physicians, who were regularly participating in expert teleconsultations, were also more likely to be treated according to guideline recommendations regarding antibiotic therapy for uncomplicated upper respiratory tract infections (OR 1.34, 95% CI 1.16-1.56; P<.001) and asymptomatic bacteriuria (OR 9.31, 95% CI 3.79-25.94; P<.001). For the other recommendations, we found no significant effects, or we had too few observations to generate models. The key limitations of our study include selection effects due to the applied on-site triage of patients as well as the limited possibilities to control for secular effects. Conclusions: Telemedicine facilitates a direct round-the-clock interaction over broad distances between intensivists or infectious disease experts and physicians who care for patients in hospitals without ready access to these experts. Expert teleconsultations increase guideline adherence and treatment quality in infectious disease and intensive care management, creating added value for critically ill patients. Trial Registration: ClinicalTrials.gov NCT03137589; https://clinicaltrials.gov/ct2/show/NCT03137589 %M 35103604 %R 10.2196/34098 %U https://www.jmir.org/2022/3/e34098 %U https://doi.org/10.2196/34098 %U http://www.ncbi.nlm.nih.gov/pubmed/35103604 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 11 %N 3 %P e34894 %T Dashboards in Health Care Settings: Protocol for a Scoping Review %A Helminski,Danielle %A Kurlander,Jacob E %A Renji,Anjana Deep %A Sussman,Jeremy B %A Pfeiffer,Paul N %A Conte,Marisa L %A Gadabu,Oliver J %A Kokaly,Alex N %A Goldberg,Rebecca %A Ranusch,Allison %A Damschroder,Laura J %A Landis-Lewis,Zach %+ Department of Internal Medicine, University of Michigan, NCRC Building 14, 2800 Plymouth Road, Ann Arbor, MI, 48109, United States, 1 7346153952, dhelmins@umich.edu %K dashboard %K mHealth %K medical informatics %K quality improvement %K scoping review %K health care %K Cochrane library %K Cochrane %K stakeholder %K health care sector %K digital health %K design %K end user %K development %K implementation %K evaluation %K user need %D 2022 %7 2.3.2022 %9 Protocol %J JMIR Res Protoc %G English %X Background: Health care organizations increasingly depend on business intelligence tools, including “dashboards,” to capture, analyze, and present data on performance metrics. Ideally, dashboards allow users to quickly visualize actionable data to inform and optimize clinical and organizational performance. In reality, dashboards are typically embedded in complex health care organizations with massive data streams and end users with distinct needs. Thus, designing effective dashboards is a challenging task and theoretical underpinnings of health care dashboards are poorly characterized; even the concept of the dashboard remains ill-defined. Researchers, informaticists, clinical managers, and health care administrators will benefit from a clearer understanding of how dashboards have been developed, implemented, and evaluated, and how the design, end user, and context influence their uptake and effectiveness. Objective: This scoping review first aims to survey the vast published literature of “dashboards” to describe where, why, and for whom they are used in health care settings, as well as how they are developed, implemented, and evaluated. Further, we will examine how dashboard design and content is informed by intended purpose and end users. Methods: In July 2020, we searched MEDLINE, Embase, Web of Science, and the Cochrane Library for peer-reviewed literature using a targeted strategy developed with a research librarian and retrieved 5188 results. Following deduplication, 3306 studies were screened in duplicate for title and abstract. Any abstracts mentioning a health care dashboard were retrieved in full text and are undergoing duplicate review for eligibility. Articles will be included for data extraction and analysis if they describe the development, implementation, or evaluation of a dashboard that was successfully used in routine workflow. Articles will be excluded if they were published before 2015, the full text is unavailable, they are in a non-English language, or they describe dashboards used for public health tracking, in settings where direct patient care is not provided, or in undergraduate medical education. Any discrepancies in eligibility determination will be adjudicated by a third reviewer. We chose to focus on articles published after 2015 and those that describe dashboards that were successfully used in routine practice to identify the most recent and relevant literature to support future dashboard development in the rapidly evolving field of health care informatics. Results: All articles have undergone dual review for title and abstract, with a total of 2019 articles mentioning use of a health care dashboard retrieved in full text for further review. We are currently reviewing all full-text articles in duplicate. We aim to publish findings by mid-2022. Findings will be reported following guidance from the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist. Conclusions: This scoping review will provide stakeholders with an overview of existing dashboard tools, highlighting the ways in which dashboards have been developed, implemented, and evaluated in different settings and for different end user groups, and identify potential research gaps. Findings will guide efforts to design and use dashboards in the health care sector more effectively. International Registered Report Identifier (IRRID): DERR1-10.2196/34894 %M 35234650 %R 10.2196/34894 %U https://www.researchprotocols.org/2022/3/e34894 %U https://doi.org/10.2196/34894 %U http://www.ncbi.nlm.nih.gov/pubmed/35234650 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 3 %P e31615 %T Performance of a Computational Phenotyping Algorithm for Sarcoidosis Using Diagnostic Codes in Electronic Medical Records: Case Validation Study From 2 Veterans Affairs Medical Centers %A Seedahmed,Mohamed I %A Mogilnicka,Izabella %A Zeng,Siyang %A Luo,Gang %A Whooley,Mary A %A McCulloch,Charles E %A Koth,Laura %A Arjomandi,Mehrdad %+ Division of Pulmonary, Critical Care, Allergy and Immunology, and Sleep, Department of Medicine, University of California San Francisco, 513 Parnassus Ave, HSE 1314, Box 0111, San Francisco, CA, 94143, United States, 1 (415) 476 0735, mohamed.seedahmed@ucsf.edu %K sarcoidosis %K electronic medical records %K EMRs %K computational phenotype %K diagnostic codes %K Veterans Affairs %K VA %K practice guidelines %D 2022 %7 2.3.2022 %9 Original Paper %J JMIR Form Res %G English %X Background: Electronic medical records (EMRs) offer the promise of computationally identifying sarcoidosis cases. However, the accuracy of identifying these cases in the EMR is unknown. Objective: The aim of this study is to determine the statistical performance of using the International Classification of Diseases (ICD) diagnostic codes to identify patients with sarcoidosis in the EMR. Methods: We used the ICD diagnostic codes to identify sarcoidosis cases by searching the EMRs of the San Francisco and Palo Alto Veterans Affairs medical centers and randomly selecting 200 patients. To improve the diagnostic accuracy of the computational algorithm in cases where histopathological data are unavailable, we developed an index of suspicion to identify cases with a high index of suspicion for sarcoidosis (confirmed and probable) based on clinical and radiographic features alone using the American Thoracic Society practice guideline. Through medical record review, we determined the positive predictive value (PPV) of diagnosing sarcoidosis by two computational methods: using ICD codes alone and using ICD codes plus the high index of suspicion. Results: Among the 200 patients, 158 (79%) had a high index of suspicion for sarcoidosis. Of these 158 patients, 142 (89.9%) had documentation of nonnecrotizing granuloma, confirming biopsy-proven sarcoidosis. The PPV of using ICD codes alone was 79% (95% CI 78.6%-80.5%) for identifying sarcoidosis cases and 71% (95% CI 64.7%-77.3%) for identifying histopathologically confirmed sarcoidosis in the EMRs. The inclusion of the generated high index of suspicion to identify confirmed sarcoidosis cases increased the PPV significantly to 100% (95% CI 96.5%-100%). Histopathology documentation alone was 90% sensitive compared with high index of suspicion. Conclusions: ICD codes are reasonable classifiers for identifying sarcoidosis cases within EMRs with a PPV of 79%. Using a computational algorithm to capture index of suspicion data elements could significantly improve the case-identification accuracy. %M 35081036 %R 10.2196/31615 %U https://formative.jmir.org/2022/3/e31615 %U https://doi.org/10.2196/31615 %U http://www.ncbi.nlm.nih.gov/pubmed/35081036 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e30883 %T Applications and User Perceptions of Smart Glasses in Emergency Medical Services: Semistructured Interview Study %A Zhang,Zhan %A Joy,Karen %A Harris,Richard %A Ozkaynak,Mustafa %A Adelgais,Kathleen %A Munjal,Kevin %+ School of Computer Science and Information Systems, Pace University, 163 William Street, New York, NY, 10078, United States, 1 2123461897, zzhang@pace.edu %K smart glasses %K hands-free technologies %K emergency medical services %K user studies %K mobile phone %D 2022 %7 28.2.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Smart glasses have been gaining momentum as a novel technology because of their advantages in enabling hands-free operation and see-what-I-see remote consultation. Researchers have primarily evaluated this technology in hospital settings; however, limited research has investigated its application in prehospital operations. Objective: The aim of this study is to understand the potential of smart glasses to support the work practices of prehospital providers, such as emergency medical services (EMS) personnel. Methods: We conducted semistructured interviews with 13 EMS providers recruited from 4 hospital-based EMS agencies in an urban area in the east coast region of the United States. The interview questions covered EMS workflow, challenges encountered, technology needs, and users’ perceptions of smart glasses in supporting daily EMS work. During the interviews, we demonstrated a system prototype to elicit more accurate and comprehensive insights regarding smart glasses. Interviews were transcribed verbatim and analyzed using the open coding technique. Results: We identified four potential application areas for smart glasses in EMS: enhancing teleconsultation between distributed prehospital and hospital providers, semiautomating patient data collection and documentation in real time, supporting decision-making and situation awareness, and augmenting quality assurance and training. Compared with the built-in touch pad, voice commands and hand gestures were indicated as the most preferred and suitable interaction mechanisms. EMS providers expressed positive attitudes toward using smart glasses during prehospital encounters. However, several potential barriers and user concerns need to be considered and addressed before implementing and deploying smart glasses in EMS practice. They are related to hardware limitations, human factors, reliability, workflow, interoperability, and privacy. Conclusions: Smart glasses can be a suitable technological means for supporting EMS work. We conclude this paper by discussing several design considerations for realizing the full potential of this hands-free technology. %M 35225816 %R 10.2196/30883 %U https://humanfactors.jmir.org/2022/1/e30883 %U https://doi.org/10.2196/30883 %U http://www.ncbi.nlm.nih.gov/pubmed/35225816 %0 Journal Article %@ 2561-9128 %I JMIR Publications %V 5 %N 1 %P e32738 %T The Case for the Anesthesiologist-Informaticist %A Lee,Robert %A Hitt,James %A Hobika,Geoffrey G %A Nader,Nader D %+ Department of Anesthesiology, VA Western New York Healthcare System, 3495 Bailey Ave, Buffalo, NY, 14215, United States, 1 716 834 9200, rlee32@buffalo.edu %K anesthesia %K anesthesiology %K AIMS %K anesthesia information management systems %K clinical informatics %K anesthesia informatics %K perioperative informatics %K health information %K perioperative medicine %K health technology %D 2022 %7 28.2.2022 %9 Viewpoint %J JMIR Perioper Med %G English %X Health care has been transformed by computerization, and the use of electronic health record systems has become widespread. Anesthesia information management systems are commonly used in the operating room to maintain records of anesthetic care delivery. The perioperative environment and the practice of anesthesia generate a large volume of data that may be reused to support clinical decision-making, research, and process improvement. Anesthesiologists trained in clinical informatics, referred to as informaticists or informaticians, may help implement and optimize anesthesia information management systems. They may also participate in clinical research, management of information systems, and quality improvement in the operating room or throughout a health care system. Here, we describe the specialty of clinical informatics, how anesthesiologists may obtain training in clinical informatics, and the considerations particular to the subspecialty of anesthesia informatics. Management of perioperative information systems, implementation of computerized clinical decision support systems in the perioperative environment, the role of virtual visits and remote monitoring, perioperative informatics research, perioperative process improvement, leadership, and change management are described from the perspective of the anesthesiologist-informaticist. %M 35225822 %R 10.2196/32738 %U https://periop.jmir.org/2022/1/e32738 %U https://doi.org/10.2196/32738 %U http://www.ncbi.nlm.nih.gov/pubmed/35225822 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 2 %P e32230 %T Automated Pulmonary Embolism Risk Assessment Using the Wells Criteria: Validation Study %A Zhang,Nasen Jonathan %A Rameau,Philippe %A Julemis,Marsophia %A Liu,Yan %A Solomon,Jeffrey %A Khan,Sundas %A McGinn,Thomas %A Richardson,Safiya %+ Northwell Health, 600 Community Dr, Manhasset, NY, 11020, United States, 1 (516) 470 3377, nasenz@gmail.com %K health informatics %K pulmonary embolism %K electronic health record %K quality improvement %K clinical decision support systems %D 2022 %7 28.2.2022 %9 Original Paper %J JMIR Form Res %G English %X Background: Computed tomography pulmonary angiography (CTPA) is frequently used in the emergency department (ED) for the diagnosis of pulmonary embolism (PE), while posing risk for contrast-induced nephropathy and radiation-induced malignancy. Objective: We aimed to create an automated process to calculate the Wells score for pulmonary embolism for patients in the ED, which could potentially reduce unnecessary CTPA testing. Methods: We designed an automated process using electronic health records data elements, including using a combinatorial keyword search method to query free-text fields, and calculated automated Wells scores for a sample of all adult ED encounters that resulted in a CTPA study for PE at 2 tertiary care hospitals in New York, over a 2-month period. To validate the automated process, the scores were compared to those derived from a 2-clinician chart review. Results: A total of 202 ED encounters resulted in a completed CTPA to form the retrospective study cohort. Patients classified as “PE likely” by the automated process (126/202, 62%) had a PE prevalence of 15.9%, whereas those classified as “PE unlikely” (76/202, 38%; Wells score >4) had a PE prevalence of 7.9%. With respect to classification of the patient as “PE likely,” the automated process achieved an accuracy of 92.1% when compared with the chart review, with sensitivity, specificity, positive predictive value, and negative predictive value of 93%, 90.5%, 94.4%, and 88.2%, respectively. Conclusions: This was a successful development and validation of an automated process using electronic health records data elements, including free-text fields, to classify risk for PE in ED visits. %M 35225812 %R 10.2196/32230 %U https://formative.jmir.org/2022/2/e32230 %U https://doi.org/10.2196/32230 %U http://www.ncbi.nlm.nih.gov/pubmed/35225812 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 2 %P e27146 %T Age- and Sex-Specific Differences in Multimorbidity Patterns and Temporal Trends on Assessing Hospital Discharge Records in Southwest China: Network-Based Study %A Wang,Liya %A Qiu,Hang %A Luo,Li %A Zhou,Li %+ School of Computer Science and Engineering, University of Electronic Science and Technology of China, No.2006, Xiyuan Ave, West Hi-Tech Zone, Chengdu, 611731, China, 86 28 61830278, qiuhang@uestc.edu.cn %K multimorbidity pattern %K temporal trend %K network analysis %K multimorbidity prevalence %K administrative data %K longitudinal study %K regional research %D 2022 %7 25.2.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Multimorbidity represents a global health challenge, which requires a more global understanding of multimorbidity patterns and trends. However, the majority of studies completed to date have often relied on self-reported conditions, and a simultaneous assessment of the entire spectrum of chronic disease co-occurrence, especially in developing regions, has not yet been performed. Objective: We attempted to provide a multidimensional approach to understand the full spectrum of chronic disease co-occurrence among general inpatients in southwest China, in order to investigate multimorbidity patterns and temporal trends, and assess their age and sex differences. Methods: We conducted a retrospective cohort analysis based on 8.8 million hospital discharge records of about 5.0 million individuals of all ages from 2015 to 2019 in a megacity in southwest China. We examined all chronic diagnoses using the ICD-10 (International Classification of Diseases, 10th revision) codes at 3 digits and focused on chronic diseases with ≥1% prevalence for each of the age and sex strata, which resulted in a total of 149 and 145 chronic diseases in males and females, respectively. We constructed multimorbidity networks in the general population based on sex and age, and used the cosine index to measure the co-occurrence of chronic diseases. Then, we divided the networks into communities and assessed their temporal trends. Results: The results showed complex interactions among chronic diseases, with more intensive connections among males and inpatients ≥40 years old. A total of 9 chronic diseases were simultaneously classified as central diseases, hubs, and bursts in the multimorbidity networks. Among them, 5 diseases were common to both males and females, including hypertension, chronic ischemic heart disease, cerebral infarction, other cerebrovascular diseases, and atherosclerosis. The earliest leaps (degree leaps ≥6) appeared at a disorder of glycoprotein metabolism that happened at 25-29 years in males, about 15 years earlier than in females. The number of chronic diseases in the community increased over time, but the new entrants did not replace the root of the community. Conclusions: Our multimorbidity network analysis identified specific differences in the co-occurrence of chronic diagnoses by sex and age, which could help in the design of clinical interventions for inpatient multimorbidity. %M 35212632 %R 10.2196/27146 %U https://www.jmir.org/2022/2/e27146 %U https://doi.org/10.2196/27146 %U http://www.ncbi.nlm.nih.gov/pubmed/35212632 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 2 %P e34492 %T Traditional Machine Learning Models and Bidirectional Encoder Representations From Transformer (BERT)–Based Automatic Classification of Tweets About Eating Disorders: Algorithm Development and Validation Study %A Benítez-Andrades,José Alberto %A Alija-Pérez,José-Manuel %A Vidal,Maria-Esther %A Pastor-Vargas,Rafael %A García-Ordás,María Teresa %+ SALBIS Research Group, Department of Electric, Systems and Automatics Engineering, University of León, Campus of Vegazana s/n, León, 24071, Spain, 34 987293628, jbena@unileon.es %K natural language processing %K NLP %K social media %K data %K bidirectional encoder representations from transformer %K BERT %K deep learning %K machine learning %K eating disorder %K mental health %K model %K classification %K Twitter %K nutrition %K diet %K weight %K disorder %K performance %D 2022 %7 24.2.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Eating disorders affect an increasing number of people. Social networks provide information that can help. Objective: We aimed to find machine learning models capable of efficiently categorizing tweets about eating disorders domain. Methods: We collected tweets related to eating disorders, for 3 consecutive months. After preprocessing, a subset of 2000 tweets was labeled: (1) messages written by people suffering from eating disorders or not, (2) messages promoting suffering from eating disorders or not, (3) informative messages or not, and (4) scientific or nonscientific messages. Traditional machine learning and deep learning models were used to classify tweets. We evaluated accuracy, F1 score, and computational time for each model. Results: A total of 1,058,957 tweets related to eating disorders were collected. were obtained in the 4 categorizations, with The bidirectional encoder representations from transformer–based models had the best score among the machine learning and deep learning techniques applied to the 4 categorization tasks (F1 scores 71.1%-86.4%). Conclusions: Bidirectional encoder representations from transformer–based models have better performance, although their computational cost is significantly higher than those of traditional techniques, in classifying eating disorder–related tweets. %M 35200156 %R 10.2196/34492 %U https://medinform.jmir.org/2022/2/e34492 %U https://doi.org/10.2196/34492 %U http://www.ncbi.nlm.nih.gov/pubmed/35200156 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 2 %P e31083 %T Computerized Clinical Decision Support Systems for the Early Detection of Sepsis Among Adult Inpatients: Scoping Review %A Ackermann,Khalia %A Baker,Jannah %A Green,Malcolm %A Fullick,Mary %A Varinli,Hilal %A Westbrook,Johanna %A Li,Ling %+ Centre for Health Systems and Safety Research, Australian Institute of Health Innovation, Level 6, Talavera Road, Macquarie University, 2109, Australia, 61 2 9850 2432, khalia.ackermann@mq.edu.au %K sepsis %K early detection of disease %K clinical decision support systems %K patient safety %K electronic health records %K sepsis care pathway %D 2022 %7 23.2.2022 %9 Review %J J Med Internet Res %G English %X Background: Sepsis is a significant cause of morbidity and mortality worldwide. Early detection of sepsis followed promptly by treatment initiation improves patient outcomes and saves lives. Hospitals are increasingly using computerized clinical decision support (CCDS) systems for the rapid identification of adult patients with sepsis. Objective: This scoping review aims to systematically describe studies reporting on the use and evaluation of CCDS systems for the early detection of adult inpatients with sepsis. Methods: The protocol for this scoping review was previously published. A total of 10 electronic databases (MEDLINE, Embase, CINAHL, the Cochrane database, LILACS [Latin American and Caribbean Health Sciences Literature], Scopus, Web of Science, OpenGrey, ClinicalTrials.gov, and PQDT [ProQuest Dissertations and Theses]) were comprehensively searched using terms for sepsis, CCDS, and detection to identify relevant studies. Title, abstract, and full-text screening were performed by 2 independent reviewers using predefined eligibility criteria. Data charting was performed by 1 reviewer with a second reviewer checking a random sample of studies. Any disagreements were discussed with input from a third reviewer. In this review, we present the results for adult inpatients, including studies that do not specify patient age. Results: A search of the electronic databases retrieved 12,139 studies following duplicate removal. We identified 124 studies for inclusion after title, abstract, full-text screening, and hand searching were complete. Nearly all studies (121/124, 97.6%) were published after 2009. Half of the studies were journal articles (65/124, 52.4%), and the remainder were conference abstracts (54/124, 43.5%) and theses (5/124, 4%). Most studies used a single cohort (54/124, 43.5%) or before-after (42/124, 33.9%) approach. Across all 124 included studies, patient outcomes were the most frequently reported outcomes (107/124, 86.3%), followed by sepsis treatment and management (75/124, 60.5%), CCDS usability (14/124, 11.3%), and cost outcomes (9/124, 7.3%). For sepsis identification, the systemic inflammatory response syndrome criteria were the most commonly used, alone (50/124, 40.3%), combined with organ dysfunction (28/124, 22.6%), or combined with other criteria (23/124, 18.5%). Over half of the CCDS systems (68/124, 54.8%) were implemented alongside other sepsis-related interventions. Conclusions: The current body of literature investigating the implementation of CCDS systems for the early detection of adult inpatients with sepsis is extremely diverse. There is substantial variability in study design, CCDS criteria and characteristics, and outcomes measured across the identified literature. Future research on CCDS system usability, cost, and impact on sepsis morbidity is needed. International Registered Report Identifier (IRRID): RR2-10.2196/24899 %M 35195528 %R 10.2196/31083 %U https://www.jmir.org/2022/2/e31083 %U https://doi.org/10.2196/31083 %U http://www.ncbi.nlm.nih.gov/pubmed/35195528 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 2 %P e34907 %T The Science of Learning Health Systems: Scoping Review of Empirical Research %A Ellis,Louise A %A Sarkies,Mitchell %A Churruca,Kate %A Dammery,Genevieve %A Meulenbroeks,Isabelle %A Smith,Carolynn L %A Pomare,Chiara %A Mahmoud,Zeyad %A Zurynski,Yvonne %A Braithwaite,Jeffrey %+ Australian Institute of Health Innovation, Macquarie University, 75 Talavera Rd, Sydney, 2113, Australia, 61 298502484, louise.ellis@mq.edu.au %K learning health systems %K learning health care systems %K implementation science %K evaluation %K health system %K health care system %K empirical research %K medical informatics %K review %D 2022 %7 23.2.2022 %9 Review %J JMIR Med Inform %G English %X Background: The development and adoption of a learning health system (LHS) has been proposed as a means to address key challenges facing current and future health care systems. The first review of the LHS literature was conducted 5 years ago, identifying only a small number of published papers that had empirically examined the implementation or testing of an LHS. It is timely to look more closely at the published empirical research and to ask the question, Where are we now? 5 years on from that early LHS review. Objective: This study performed a scoping review of empirical research within the LHS domain. Taking an “implementation science” lens, the review aims to map out the empirical research that has been conducted to date, identify limitations, and identify future directions for the field. Methods: Two academic databases (PubMed and Scopus) were searched using the terms “learning health* system*” for papers published between January 1, 2016, to January 31, 2021, that had an explicit empirical focus on LHSs. Study information was extracted relevant to the review objective, including each study’s publication details; primary concern or focus; context; design; data type; implementation framework, model, or theory used; and implementation determinants or outcomes examined. Results: A total of 76 studies were included in this review. Over two-thirds of the studies were concerned with implementing a particular program, system, or platform (53/76, 69.7%) designed to contribute to achieving an LHS. Most of these studies focused on a particular clinical context or patient population (37/53, 69.8%), with far fewer studies focusing on whole hospital systems (4/53, 7.5%) or on other broad health care systems encompassing multiple facilities (12/53, 22.6%). Over two-thirds of the program-specific studies utilized quantitative methods (37/53, 69.8%), with a smaller number utilizing qualitative methods (10/53, 18.9%) or mixed-methods designs (6/53, 11.3%). The remaining 23 studies were classified into 1 of 3 key areas: ethics, policies, and governance (10/76, 13.2%); stakeholder perspectives of LHSs (5/76, 6.6%); or LHS-specific research strategies and tools (8/76, 10.5%). Overall, relatively few studies were identified that incorporated an implementation science framework. Conclusions: Although there has been considerable growth in empirical applications of LHSs within the past 5 years, paralleling the recent emergence of LHS-specific research strategies and tools, there are few high-quality studies. Comprehensive reporting of implementation and evaluation efforts is an important step to moving the LHS field forward. In particular, the routine use of implementation determinant and outcome frameworks will improve the assessment and reporting of barriers, enablers, and implementation outcomes in this field and will enable comparison and identification of trends across studies. %M 35195529 %R 10.2196/34907 %U https://medinform.jmir.org/2022/2/e34907 %U https://doi.org/10.2196/34907 %U http://www.ncbi.nlm.nih.gov/pubmed/35195529 %0 Journal Article %@ 2562-0959 %I JMIR Publications %V 5 %N 1 %P e24956 %T Patients’ Experiences of Telemedicine for Their Skin Problems: Qualitative Study %A Chow,Aloysius %A Teo,Sok Huang %A Kong,Jing Wen %A Lee,Simon %A Heng,Yee Kiat %A van Steensel,Maurice %A Smith,Helen %+ Family Medicine and Primary Care, Lee Kong Chian School of Medicine, Nanyang Technological University, 11 Mandalay Road, Clinical Sciences Building Level 18, Singapore, 308232, Singapore, 65 6592 3926, h.e.smith@ntu.edu.sg %K teledermatology %K qualitative %K patients experience %K telemedicine %K dermatology %K Singapore %D 2022 %7 22.2.2022 %9 Original Paper %J JMIR Dermatol %G English %X Background: Teledermatology is a cost-effective treatment modality for the management of skin disorders. Most evaluations use quantitative data, and far less is understood about the patients’ experience. Objective: This qualitative study aimed to explore patients’ perceptions of a teledermatology service linking public primary care clinics to the national specialist dermatology clinic in Singapore. A better understanding of patients’ experiences can help refine and develop the care provided. Methods: Semistructured in-depth interviews were conducted with patients who had been referred to the teledermatology service. The interviews were digitally recorded and transcribed before undergoing thematic content analysis. Results: A total of 21 patients aged between 22 and 72 years were recruited. The following 3 themes were identified from the data of patients’ experiences: positive perceptions of teledermatology, concerns about teledermatology, and ideas for improving the teledermatology service. The patients found the teledermatology service convenient, saving them time and expense and liberating them from the stresses incurred when making an in-person visit to a specialist facility. They valued the confidence and reassurance they gained from having a dermatologist involved in deciding their management. The patients’ concern included data security and the quality of the images shared. Nonetheless, they were keen to see the service expanded beyond the polyclinics. Their experiences and perceptions will inform future service refinement and development. Conclusions: This narrative exploration of users’ experiences of teledermatology produced rich data enabling a better understanding of the patients’ journey, the way they understand and interpret their experiences, and ideas for service refinement. Telemedicine reduces traveling and enables safe distancing, factors that are much needed during pandemics. %M 37632855 %R 10.2196/24956 %U https://derma.jmir.org/2022/1/e24956 %U https://doi.org/10.2196/24956 %U http://www.ncbi.nlm.nih.gov/pubmed/37632855 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 2 %P e27534 %T Interactive Visualization Applications in Population Health and Health Services Research: Systematic Scoping Review %A Chishtie,Jawad %A Bielska,Iwona Anna %A Barrera,Aldo %A Marchand,Jean-Sebastien %A Imran,Muhammad %A Tirmizi,Syed Farhan Ali %A Turcotte,Luke A %A Munce,Sarah %A Shepherd,John %A Senthinathan,Arrani %A Cepoiu-Martin,Monica %A Irvine,Michael %A Babineau,Jessica %A Abudiab,Sally %A Bjelica,Marko %A Collins,Christopher %A Craven,B Catharine %A Guilcher,Sara %A Jeji,Tara %A Naraei,Parisa %A Jaglal,Susan %+ Rehabilitation Sciences Institute, Faculty of Medicine, University of Toronto, 500 University Ave, Toronto, ON, M5G 1V7, Canada, 1 6479756965, jac161@gmail.com %K interactive visualization %K data visualization %K secondary health care data %K public health informatics %K population health %K health services research %D 2022 %7 18.2.2022 %9 Review %J J Med Internet Res %G English %X Background: Simple visualizations in health research data, such as scatter plots, heat maps, and bar charts, typically present relationships between 2 variables. Interactive visualization methods allow for multiple related facets such as numerous risk factors to be studied simultaneously, leading to data insights through exploring trends and patterns from complex big health care data. The technique presents a powerful tool that can be used in combination with statistical analysis for knowledge discovery, hypothesis generation and testing, and decision support. Objective: The primary objective of this scoping review is to describe and summarize the evidence of interactive visualization applications, methods, and tools being used in population health and health services research (HSR) and their subdomains in the last 15 years, from January 1, 2005, to March 30, 2019. Our secondary objective is to describe the use cases, metrics, frameworks used, settings, target audience, goals, and co-design of applications. Methods: We adapted standard scoping review guidelines with a peer-reviewed search strategy: 2 independent researchers at each stage of screening and abstraction, with a third independent researcher to arbitrate conflicts and validate findings. A comprehensive abstraction platform was built to capture the data from diverse bodies of literature, primarily from the computer science and health care sectors. After screening 11,310 articles, we present findings from 56 applications from interrelated areas of population health and HSR, as well as their subdomains such as epidemiologic surveillance, health resource planning, access, and use and costs among diverse clinical and demographic populations. Results: In this companion review to our earlier systematic synthesis of the literature on visual analytics applications, we present findings in 6 major themes of interactive visualization applications developed for 8 major problem categories. We found a wide application of interactive visualization methods, the major ones being epidemiologic surveillance for infectious disease, resource planning, health service monitoring and quality, and studying medication use patterns. The data sources included mostly secondary administrative and electronic medical record data. In addition, at least two-thirds of the applications involved participatory co-design approaches while introducing a distinct category, embedded research, within co-design initiatives. These applications were in response to an identified need for data-driven insights into knowledge generation and decision support. We further discuss the opportunities stemming from the use of interactive visualization methods in studying global health; inequities, including social determinants of health; and other related areas. We also allude to the challenges in the uptake of these methods. Conclusions: Visualization in health has strong historical roots, with an upward trend in the use of these methods in population health and HSR. Such applications are being fast used by academic and health care agencies for knowledge discovery, hypotheses generation, and decision support. International Registered Report Identifier (IRRID): RR2-10.2196/14019 %M 35179499 %R 10.2196/27534 %U https://www.jmir.org/2022/2/e27534 %U https://doi.org/10.2196/27534 %U http://www.ncbi.nlm.nih.gov/pubmed/35179499 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 2 %P e33440 %T The Application and Comparison of Machine Learning Models for the Prediction of Breast Cancer Prognosis: Retrospective Cohort Study %A Xiao,Jialong %A Mo,Miao %A Wang,Zezhou %A Zhou,Changming %A Shen,Jie %A Yuan,Jing %A He,Yulian %A Zheng,Ying %+ Department of Cancer Prevention, Fudan University Shanghai Cancer Center, 270 Dong 'an Road, Xuhui District, Shanghai, 200000, China, 86 21 64175590, zhengying@fudan.edu.cn %K breast cancer %K machine learning %K survival analysis %K random survival forest %K support vector machine %K medical informatics %K prediction models %D 2022 %7 18.2.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Over the recent years, machine learning methods have been increasingly explored in cancer prognosis because of the appearance of improved machine learning algorithms. These algorithms can use censored data for modeling, such as support vector machines for survival analysis and random survival forest (RSF). However, it is still debated whether traditional (Cox proportional hazard regression) or machine learning-based prognostic models have better predictive performance. Objective: This study aimed to compare the performance of breast cancer prognostic prediction models based on machine learning and Cox regression. Methods: This retrospective cohort study included all patients diagnosed with breast cancer and subsequently hospitalized in Fudan University Shanghai Cancer Center between January 1, 2008, and December 31, 2016. After all exclusions, a total of 22,176 cases with 21 features were eligible for model development. The data set was randomly split into a training set (15,523 cases, 70%) and a test set (6653 cases, 30%) for developing 4 models and predicting the overall survival of patients diagnosed with breast cancer. The discriminative ability of models was evaluated by the concordance index (C-index), the time-dependent area under the curve, and D-index; the calibration ability of models was evaluated by the Brier score. Results: The RSF model revealed the best discriminative performance among the 4 models with 3-year, 5-year, and 10-year time-dependent area under the curve of 0.857, 0.838, and 0.781, a D-index of 7.643 (95% CI 6.542, 8.930) and a C-index of 0.827 (95% CI 0.809, 0.845). The statistical difference of the C-index was tested, and the RSF model significantly outperformed the Cox-EN (elastic net) model (C-index 0.816, 95% CI 0.796, 0.836; P=.01), the Cox model (C-index 0.814, 95% CI 0.794, 0.835; P=.003), and the support vector machine model (C-index 0.812, 95% CI 0.793, 0.832; P<.001). The 4 models’ 3-year, 5-year, and 10-year Brier scores were very close, ranging from 0.027 to 0.094 and less than 0.1, which meant all models had good calibration. In the context of feature importance, elastic net and RSF both indicated that TNM staging, neoadjuvant therapy, number of lymph node metastases, age, and tumor diameter were the top 5 important features for predicting the prognosis of breast cancer. A final online tool was developed to predict the overall survival of patients with breast cancer. Conclusions: The RSF model slightly outperformed the other models on discriminative ability, revealing the potential of the RSF method as an effective approach to building prognostic prediction models in the context of survival analysis. %M 35179504 %R 10.2196/33440 %U https://medinform.jmir.org/2022/2/e33440 %U https://doi.org/10.2196/33440 %U http://www.ncbi.nlm.nih.gov/pubmed/35179504 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 2 %P e30345 %T Evaluation of Natural Language Processing for the Identification of Crohn Disease–Related Variables in Spanish Electronic Health Records: A Validation Study for the PREMONITION-CD Project %A Montoto,Carmen %A Gisbert,Javier P %A Guerra,Iván %A Plaza,Rocío %A Pajares Villarroya,Ramón %A Moreno Almazán,Luis %A López Martín,María Del Carmen %A Domínguez Antonaya,Mercedes %A Vera Mendoza,Isabel %A Aparicio,Jesús %A Martínez,Vicente %A Tagarro,Ignacio %A Fernandez-Nistal,Alonso %A Canales,Lea %A Menke,Sebastian %A Gomollón,Fernando %A , %+ Takeda Farmacéutica España S.A., Edificio Torre Europa, Paseo de la Castellana, 95, Madrid, 28046, Spain, 34 917904222, Carmen.montoto@takeda.com %K natural language processing %K linguistic validation %K artificial intelligence %K electronic health records %K Crohn disease %K inflammatory bowel disease %D 2022 %7 18.2.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: The exploration of clinically relevant information in the free text of electronic health records (EHRs) holds the potential to positively impact clinical practice as well as knowledge regarding Crohn disease (CD), an inflammatory bowel disease that may affect any segment of the gastrointestinal tract. The EHRead technology, a clinical natural language processing (cNLP) system, was designed to detect and extract clinical information from narratives in the clinical notes contained in EHRs. Objective: The aim of this study is to validate the performance of the EHRead technology in identifying information of patients with CD. Methods: We used the EHRead technology to explore and extract CD-related clinical information from EHRs. To validate this tool, we compared the output of the EHRead technology with a manually curated gold standard to assess the quality of our cNLP system in detecting records containing any reference to CD and its related variables. Results: The validation metrics for the main variable (CD) were a precision of 0.88, a recall of 0.98, and an F1 score of 0.93. Regarding the secondary variables, we obtained a precision of 0.91, a recall of 0.71, and an F1 score of 0.80 for CD flare, while for the variable vedolizumab (treatment), a precision, recall, and F1 score of 0.86, 0.94, and 0.90 were obtained, respectively. Conclusions: This evaluation demonstrates the ability of the EHRead technology to identify patients with CD and their related variables from the free text of EHRs. To the best of our knowledge, this study is the first to use a cNLP system for the identification of CD in EHRs written in Spanish. %M 35179507 %R 10.2196/30345 %U https://medinform.jmir.org/2022/2/e30345 %U https://doi.org/10.2196/30345 %U http://www.ncbi.nlm.nih.gov/pubmed/35179507 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 2 %P e23355 %T Habit and Automaticity in Medical Alert Override: Cohort Study %A Wang,Le %A Goh,Kim Huat %A Yeow,Adrian %A Poh,Hermione %A Li,Ke %A Yeow,Joannas Jie Lin %A Tan,Gamaliel %A Soh,Christina %+ Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore, 65 67904808, akhgoh@ntu.edu.sg %K alert systems %K habits %K electronic medical record %K health personnel alert fatigue %D 2022 %7 16.2.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Prior literature suggests that alert dismissal could be linked to physicians’ habits and automaticity. The evidence for this perspective has been mainly observational data. This study uses log data from an electronic medical records system to empirically validate this perspective. Objective: We seek to quantify the association between habit and alert dismissal in physicians. Methods: We conducted a retrospective analysis using the log data comprising 66,049 alerts generated from hospitalized patients in a hospital from March 2017 to December 2018. We analyzed 1152 physicians exposed to a specific clinical support alert triggered in a hospital’s electronic medical record system to estimate the extent to which the physicians’ habit strength, which had been developed from habitual learning, impacted their propensity toward alert dismissal. We further examined the association between a physician’s habit strength and their subsequent incidences of alert dismissal. Additionally, we recorded the time taken by the physician to respond to the alert and collected data on other clinical and environmental factors related to the alerts as covariates for the analysis. Results: We found that a physician’s prior dismissal of alerts leads to their increased habit strength to dismiss alerts. Furthermore, a physician’s habit strength to dismiss alerts was found to be positively associated with incidences of subsequent alert dismissals after their initial alert dismissal. Alert dismissal due to habitual learning was also found to be pervasive across all physician ranks, from junior interns to senior attending specialists. Further, the dismissal of alerts had been observed to typically occur after a very short processing time. Our study found that 72.5% of alerts were dismissed in under 3 seconds after the alert appeared, and 13.2% of all alerts were dismissed in under 1 second after the alert appeared. We found empirical support that habitual dismissal is one of the key factors associated with alert dismissal. We also found that habitual dismissal of alerts is self-reinforcing, which suggests significant challenges in disrupting or changing alert dismissal habits once they are formed. Conclusions: Habitual tendencies are associated with the dismissal of alerts. This relationship is pervasive across all levels of physician rank and experience, and the effect is self-reinforcing. %M 35171102 %R 10.2196/23355 %U https://www.jmir.org/2022/2/e23355 %U https://doi.org/10.2196/23355 %U http://www.ncbi.nlm.nih.gov/pubmed/35171102 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 9 %N 2 %P e30810 %T Digital and Mobile Health Technology in Collaborative Behavioral Health Care: Scoping Review %A Moon,Khatiya %A Sobolev,Michael %A Kane,John M %+ Zucker Hillside Hospital, Northwell Health, 75-59 263rd Street, Glen Oaks, NY, 11004, United States, 1 718 470 4597, KMoon2@northwell.edu %K collaborative care %K integrated care %K augmented care %K digital health %K mobile health %K behavioral health %K review %D 2022 %7 16.2.2022 %9 Review %J JMIR Ment Health %G English %X Background: The collaborative care model (CoCM) is a well-established system of behavioral health care in primary care settings. There is potential for digital and mobile technology to augment the CoCM to improve access, scalability, efficiency, and clinical outcomes. Objective: This study aims to conduct a scoping review to synthesize the evidence available on digital and mobile health technology in collaborative care settings. Methods: This review included cohort and experimental studies of digital and mobile technologies used to augment the CoCM. Studies examining primary care without collaborative care were excluded. A literature search was conducted using 4 electronic databases (MEDLINE, Embase, Web of Science, and Google Scholar). The search results were screened in 2 stages (title and abstract screening, followed by full-text review) by 2 reviewers. Results: A total of 3982 nonduplicate reports were identified, of which 20 (0.5%) were included in the analysis. Most studies used a combination of novel technologies. The range of digital and mobile health technologies used included mobile apps, websites, web-based platforms, telephone-based interactive voice recordings, and mobile sensor data. None of the identified studies used social media or wearable devices. Studies that measured patient and provider satisfaction reported positive results, although some types of interventions increased provider workload, and engagement was variable. In studies where clinical outcomes were measured (7/20, 35%), there were no differences between groups, or the differences were modest. Conclusions: The use of digital and mobile health technologies in CoCM is still limited. This study found that technology was most successful when it was integrated into the existing workflow without relying on patient or provider initiative. However, the effect of digital and mobile health on clinical outcomes in CoCM remains unclear and requires additional clinical trials. %M 35171105 %R 10.2196/30810 %U https://mental.jmir.org/2022/2/e30810 %U https://doi.org/10.2196/30810 %U http://www.ncbi.nlm.nih.gov/pubmed/35171105 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 2 %P e34560 %T Building a Precision Medicine Delivery Platform for Clinics: The University of California, San Francisco, BRIDGE Experience %A Bove,Riley %A Schleimer,Erica %A Sukhanov,Paul %A Gilson,Michael %A Law,Sindy M %A Barnecut,Andrew %A Miller,Bruce L %A Hauser,Stephen L %A Sanders,Stephan J %A Rankin,Katherine P %+ UCSF Weill Institute for Neurosciences, University of California, San Francisco, 1651 4th Street, San Francisco, CA, 94158, United States, 1 415 353 2069, Riley.bove@ucsf.edu %K precision medicine %K clinical implementation %K in silico trials %K clinical dashboard %K precision %K implementation %K dashboard %K design %K experience %K analytic %K tool %K analysis %K decision-making %K real time %K platform %K human-centered design %D 2022 %7 15.2.2022 %9 Viewpoint %J J Med Internet Res %G English %X Despite an ever-expanding number of analytics with the potential to impact clinical care, the field currently lacks point-of-care technological tools that allow clinicians to efficiently select disease-relevant data about their patients, algorithmically derive clinical indices (eg, risk scores), and view these data in straightforward graphical formats to inform real-time clinical decisions. Thus far, solutions to this problem have relied on either bottom-up approaches that are limited to a single clinic or generic top-down approaches that do not address clinical users’ specific setting-relevant or disease-relevant needs. As a road map for developing similar platforms, we describe our experience with building a custom but institution-wide platform that enables economies of time, cost, and expertise. The BRIDGE platform was designed to be modular and scalable and was customized to data types relevant to given clinical contexts within a major university medical center. The development process occurred by using a series of human-centered design phases with extensive, consistent stakeholder input. This institution-wide approach yielded a unified, carefully regulated, cross-specialty clinical research platform that can be launched during a patient’s electronic health record encounter. The platform pulls clinical data from the electronic health record (Epic; Epic Systems) as well as other clinical and research sources in real time; analyzes the combined data to derive clinical indices; and displays them in simple, clinician-designed visual formats specific to each disorder and clinic. By integrating an application into the clinical workflow and allowing clinicians to access data sources that would otherwise be cumbersome to assemble, view, and manipulate, institution-wide platforms represent an alternative approach to achieving the vision of true personalized medicine. %M 35166689 %R 10.2196/34560 %U https://www.jmir.org/2022/2/e34560 %U https://doi.org/10.2196/34560 %U http://www.ncbi.nlm.nih.gov/pubmed/35166689 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 2 %P e31830 %T Identification of Social Engagement Indicators Associated With Autism Spectrum Disorder Using a Game-Based Mobile App: Comparative Study of Gaze Fixation and Visual Scanning Methods %A Varma,Maya %A Washington,Peter %A Chrisman,Brianna %A Kline,Aaron %A Leblanc,Emilie %A Paskov,Kelley %A Stockham,Nate %A Jung,Jae-Yoon %A Sun,Min Woo %A Wall,Dennis P %+ Department of Pediatrics and Biomedical Data Science, Stanford University, 1265 Welch Road, Stanford, CA, 94304, United States, 1 650 497 9214, dpwall@stanford.edu %K mobile health %K autism spectrum disorder %K social phenotyping %K computer vision %K gaze %K mobile diagnostics %K pattern recognition %K autism %K diagnostic %K pattern %K engagement %K gaming %K app %K insight %K vision %K video %D 2022 %7 15.2.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Autism spectrum disorder (ASD) is a widespread neurodevelopmental condition with a range of potential causes and symptoms. Standard diagnostic mechanisms for ASD, which involve lengthy parent questionnaires and clinical observation, often result in long waiting times for results. Recent advances in computer vision and mobile technology hold potential for speeding up the diagnostic process by enabling computational analysis of behavioral and social impairments from home videos. Such techniques can improve objectivity and contribute quantitatively to the diagnostic process. Objective: In this work, we evaluate whether home videos collected from a game-based mobile app can be used to provide diagnostic insights into ASD. To the best of our knowledge, this is the first study attempting to identify potential social indicators of ASD from mobile phone videos without the use of eye-tracking hardware, manual annotations, and structured scenarios or clinical environments. Methods: Here, we used a mobile health app to collect over 11 hours of video footage depicting 95 children engaged in gameplay in a natural home environment. We used automated data set annotations to analyze two social indicators that have previously been shown to differ between children with ASD and their neurotypical (NT) peers: (1) gaze fixation patterns, which represent regions of an individual’s visual focus and (2) visual scanning methods, which refer to the ways in which individuals scan their surrounding environment. We compared the gaze fixation and visual scanning methods used by children during a 90-second gameplay video to identify statistically significant differences between the 2 cohorts; we then trained a long short-term memory (LSTM) neural network to determine if gaze indicators could be predictive of ASD. Results: Our results show that gaze fixation patterns differ between the 2 cohorts; specifically, we could identify 1 statistically significant region of fixation (P<.001). In addition, we also demonstrate that there are unique visual scanning patterns that exist for individuals with ASD when compared to NT children (P<.001). A deep learning model trained on coarse gaze fixation annotations demonstrates mild predictive power in identifying ASD. Conclusions: Ultimately, our study demonstrates that heterogeneous video data sets collected from mobile devices hold potential for quantifying visual patterns and providing insights into ASD. We show the importance of automated labeling techniques in generating large-scale data sets while simultaneously preserving the privacy of participants, and we demonstrate that specific social engagement indicators associated with ASD can be identified and characterized using such data. %M 35166683 %R 10.2196/31830 %U https://www.jmir.org/2022/2/e31830 %U https://doi.org/10.2196/31830 %U http://www.ncbi.nlm.nih.gov/pubmed/35166683 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 2 %P e29541 %T Health Information Systems for Older Persons in Select Government Tertiary Hospitals and Health Centers in the Philippines: Cross-sectional Study %A Garcia,Angely P %A De La Vega,Shelley F %A Mercado,Susan P %+ Institute on Aging, National Institutes of Health, University of the Philippines Manila, Rm 211 NIH Bldg, 623 Pedro Gil St. Ermita, Manila, 1000, Philippines, 63 9064029690, apgarcia@up.edu.ph %K health information systems %K the Philippines %K aged %K hospitals %K community health centers %K database %K geriatric assessment %K elderly %K digital health %K medical records %K health policy %D 2022 %7 14.2.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: The rapid aging of the world’s population requires systems that support health facilities’ provision of integrated care at multiple levels of the health care system. The use of health information systems (HISs) at the point of care has shown positive effects on clinical processes and patient health in several settings of care. Objective: We sought to describe HISs for older persons (OPs) in select government tertiary hospitals and health centers in the Philippines. Specifically, we aimed to review the existing policies and guidelines related to HISs for OPs in the country, determine the proportion of select government hospitals and health centers with existing health information specific for OPs, and describe the challenges related to HISs in select health facilities. Methods: We utilized the data derived from the findings of the Focused Interventions for Frail Older Adults Research and Development Project (FITforFrail), a cross-sectional and ethics committee–approved study. A facility-based listing of services and human resources specific to geriatric patients was conducted in purposively sampled 27 tertiary government hospitals identified as geriatric centers and 16 health centers across all regions in the Philippines. We also reviewed the existing policies and guidelines related to HISs for OPs in the country. Results: Based on the existing guidelines, multiple agencies were involved in the provision of services for OPs, with several records containing health information of OPs. However, there is no existing HIS specific for OPs in the country. Only 14 (52%) of the 27 hospitals and 4 (25%) of the 16 health centers conduct comprehensive geriatric assessment (CGA). All tertiary hospitals and health centers are able to maintain medical records of their patients, and almost all (26/27, 96%) hospitals and all (16/16, 100%) health centers have data on top causes of morbidity and mortality. Meanwhile, the presence of specific disease registries varied per hospitals and health centers. Challenges to HISs include the inability to update databases due to inadequately trained personnel, use of an offline facility–based HIS, an unstable internet connection, and technical issues and nonuniform reporting of categories for age group classification. Conclusions: Current HISs for OPs are characterized by fragmentation, multiple sources, and inaccessibility. Barriers to achieving appropriate HISs for OPs include the inability to update HISs in hospitals and health centers and a lack of standardization by age group and disease classification. Thus, we recommend a 1-person, 1-record electronic medical record system for OPs and the disaggregation and analysis across demographic and socioeconomic parameters to inform policies and programs that address the complex needs of OPs. CGA as a required routine procedure for all OPs and its integration with the existing HISs in the country are also recommended. %M 35156927 %R 10.2196/29541 %U https://www.jmir.org/2022/2/e29541 %U https://doi.org/10.2196/29541 %U http://www.ncbi.nlm.nih.gov/pubmed/35156927 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 2 %P e34932 %T Early Identification of Maternal Cardiovascular Risk Through Sourcing and Preparing Electronic Health Record Data: Machine Learning Study %A Shara,Nawar %A Anderson,Kelley M %A Falah,Noor %A Ahmad,Maryam F %A Tavazoei,Darya %A Hughes,Justin M %A Talmadge,Bethany %A Crovatt,Samantha %A Dempers,Ramon %+ MedStar Health Research Institute, Georgetown-Howard Universities Center for Clinical and Translational Science, 6525 Belcrest Road, Hyattsville, MD, 20782, United States, 1 240 618 7859, Nawar.Shara@Medstar.net %K electronic health record %K maternal health %K machine learning %K maternal morbidity and mortality %K cardiovascular risk %K data transformation %K extract %K transform %K load %K artificial intelligence %K electronic medical record %D 2022 %7 10.2.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Health care data are fragmenting as patients seek care from diverse sources. Consequently, patient care is negatively impacted by disparate health records. Machine learning (ML) offers a disruptive force in its ability to inform and improve patient care and outcomes. However, the differences that exist in each individual’s health records, combined with the lack of health data standards, in addition to systemic issues that render the data unreliable and that fail to create a single view of each patient, create challenges for ML. Although these problems exist throughout health care, they are especially prevalent within maternal health and exacerbate the maternal morbidity and mortality crisis in the United States. Objective: This study aims to demonstrate that patient records extracted from the electronic health records (EHRs) of a large tertiary health care system can be made actionable for the goal of effectively using ML to identify maternal cardiovascular risk before evidence of diagnosis or intervention within the patient’s record. Maternal patient records were extracted from the EHRs of a large tertiary health care system and made into patient-specific, complete data sets through a systematic method. Methods: We outline the effort that was required to define the specifications of the computational systems, the data set, and access to relevant systems, while ensuring that data security, privacy laws, and policies were met. Data acquisition included the concatenation, anonymization, and normalization of health data across multiple EHRs in preparation for their use by a proprietary risk stratification algorithm designed to establish patient-specific baselines to identify and establish cardiovascular risk based on deviations from the patient’s baselines to inform early interventions. Results: Patient records can be made actionable for the goal of effectively using ML, specifically to identify cardiovascular risk in pregnant patients. Conclusions: Upon acquiring data, including their concatenation, anonymization, and normalization across multiple EHRs, the use of an ML-based tool can provide early identification of cardiovascular risk in pregnant patients. %M 35142637 %R 10.2196/34932 %U https://medinform.jmir.org/2022/2/e34932 %U https://doi.org/10.2196/34932 %U http://www.ncbi.nlm.nih.gov/pubmed/35142637 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e30512 %T Electronic Records With Tablets at the Point of Care in an Internal Medicine Unit: Before-After Time Motion Study %A Pérez-Martí,Montserrat %A Casadó-Marín,Lina %A Guillén-Villar,Abraham %+ Department of Information Technology, Consorci Sanitari Alt Penedès-Garraf, c/Espirall 61, Vilafranca del Penedès, 08720, Spain, 34 938194000 ext 7445, mperezm@csapg.cat %K electronic health records %K nursing %K computer handheld %K equipment and supplies (devices tablets mobile phones, devices and technologies) %K workflow %D 2022 %7 10.2.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: There are many benefits of nursing professionals being able to consult electronic health records (EHRs) at the point of care. It promotes quality and patient security, communication, continuity of care, and time dedicated to records. Objective: The aim of this study was to evaluate whether making EHRs available at the point of care with tablets reduces nurses’ time spent on records compared with the current system. The analysis included sociodemographic and qualitative variables, time spent per patient, and work shift. This time difference can be used for direct patient care. Methods: A before-after time motion study was carried out in the internal medicine unit. There was a total of 130 observations of 2 hours to 3 hours in duration of complete patient records that were carried out at the beginning of the nurses' work shifts. We calculated the time dedicated to measuring vital signs, patient evaluation, and EHR recording. The main variable was time spent per patient. Results: The average time spent per patient (total time/patients admitted) was lower with the tablet group (mean 4.22, SD 0.14 minutes) than with the control group (mean 4.66, SD 0.12 minutes); there were statistically significant differences (W=3.20, P=.001) and a low effect (d=.44) between groups. The tablet group saved an average of 0.44 (SD 0.13) minutes per patient. Similar results were obtained for the afternoon shift, which saved an average of 0.60 (SD 0.15) minutes per patient (t34=3.82, P=.01) and high effect (d=.77). However, although there was a mean difference of 0.26 (SD 0.22) minutes per patient for the night shift, this was not statistically significant (t29=1.16, P=.25). The “nonparticipating” average age was higher (49.57, SD 2.92 years) compared with the “afternoon shift participants” and “night shift participants” (P=.007). “Nonparticipants” of the night shift had a worse perception of the project. Conclusions: This investigation determined that, with EHRs at the point of care, the time spent for registration by the nursing staff decreases, because of reduced movements and avoiding data transcription. It eliminates unnecessary work that does not add value, and therefore, care is improved. So, we think EHRs at the point of care should be the future or natural method for nursing to undertake. However, variables that could have a negative effect include age, night shift, and nurses’ perceptions. Therefore, it is proposed that training in the different work platforms and the participation of nurses are fundamental axes that any institution should consider before their implementation. %M 35142624 %R 10.2196/30512 %U https://humanfactors.jmir.org/2022/1/e30512 %U https://doi.org/10.2196/30512 %U http://www.ncbi.nlm.nih.gov/pubmed/35142624 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e20702 %T Factors Associated With Willingness to Share Health Information: Rapid Review %A Naeem,Iffat %A Quan,Hude %A Singh,Shaminder %A Chowdhury,Nashit %A Chowdhury,Mohammad %A Saini,Vineet %A TC,Turin %+ O'Brien Institute of Public Health, University of Calgary, GD01, TRW Building, 3280 Hospital Drive NW, Calgary, AB, T2N4N1, Canada, 1 5877787866, Iffat.naeem1@ucalgary.ca %K health information %K information sharing %K health data %K EMR %K PHR %K mobile phone %D 2022 %7 9.2.2022 %9 Review %J JMIR Hum Factors %G English %X Background: To expand research and strategies to prevent disease, comprehensive and real-time data are essential. Health data are increasingly available from platforms such as pharmaceuticals, genomics, health care imaging, medical procedures, wearable devices, and internet activity. Further, health data are integrated with an individual’s sociodemographic information, medical conditions, genetics, treatments, and health care. Ultimately, health information generation and flow are controlled by the patient or participant; however, there is a lack of understanding about the factors that influence willingness to share health information. A synthesis of the current literature on the multifactorial nature of health information sharing preferences is required to understand health information exchange. Objective: The objectives of this review are to identify peer-reviewed literature that reported factors associated with health information sharing and to organize factors into cohesive themes and present a narrative synthesis of factors related to willingness to share health information. Methods: This review uses a rapid review methodology to gather literature regarding willingness to share health information within the context of eHealth, which includes electronic health records, personal health records, mobile health information, general health information, or information on social determinants of health. MEDLINE and Google Scholar were searched using keywords such as electronic health records AND data sharing OR sharing preference OR willingness to share. The search was limited to any population that excluded health care workers or practitioners, and the participants aged ≥18 years within the US or Canadian context. The data abstraction process using thematic analysis where any factors associated with sharing health information were highlighted and coded inductively within each article. On the basis of shared meaning, the coded factors were collated into major themes. Results: A total of 26 research articles met our inclusion criteria and were included in the qualitative analysis. The inductive thematic coding process revealed multiple major themes related to sharing health information. Conclusions: This review emphasized the importance of data generators’ viewpoints and the complex systems of factors that shape their decision to share health information. The themes explored in this study emphasize the importance of trust at multiple levels to develop effective information exchange partnerships. In the case of improving precision health care, addressing the factors presented here that influence willingness to share information can improve sharing capacity for individuals and allow researchers to reorient their methods to address hesitation in sharing health information. %M 35138263 %R 10.2196/20702 %U https://humanfactors.jmir.org/2022/1/e20702 %U https://doi.org/10.2196/20702 %U http://www.ncbi.nlm.nih.gov/pubmed/35138263 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e34058 %T Understanding Cardiology Practitioners’ Interpretations of Electrocardiograms: An Eye-Tracking Study %A Tahri Sqalli,Mohammed %A Al-Thani,Dena %A Elshazly,Mohamed B %A Al-Hijji,Mohammed %A Alahmadi,Alaa %A Sqalli Houssaini,Yahya %+ Information and Computing Technology Division, College of Science and Engineering, Hamad Bin Khalifa University, Qatar Foundation, Doha, 34110, Qatar, 974 50588170, mtahrisqalli@hbku.edu.qa %K eye tracking %K electrocardiogram %K ECG interpretation %K cardiology practitioners %K human-computer interaction %K cardiology %K ECG %D 2022 %7 9.2.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Visual expertise refers to advanced visual skills demonstrated when performing domain-specific visual tasks. Prior research has emphasized the fact that medical experts rely on such perceptual pattern-recognition skills when interpreting medical images, particularly in the field of electrocardiogram (ECG) interpretation. Analyzing and modeling cardiology practitioners’ visual behavior across different levels of expertise in the health care sector is crucial. Namely, understanding such acquirable visual skills may help train less experienced clinicians to interpret ECGs accurately. Objective: This study aims to quantify and analyze through the use of eye-tracking technology differences in the visual behavior and methodological practices for different expertise levels of cardiology practitioners such as medical students, cardiology nurses, technicians, fellows, and consultants when interpreting several types of ECGs. Methods: A total of 63 participants with different levels of clinical expertise took part in an eye-tracking study that consisted of interpreting 10 ECGs with different cardiac abnormalities. A counterbalanced within-subjects design was used with one independent variable consisting of the expertise level of the cardiology practitioners and two dependent variables of eye-tracking metrics (fixations count and fixation revisitations). The eye movements data revealed by specific visual behaviors were analyzed according to the accuracy of interpretation and the frequency with which interpreters visited different parts/leads on a standard 12-lead ECG. In addition, the median and SD in the IQR for the fixations count and the mean and SD for the ECG lead revisitations were calculated. Results: Accuracy of interpretation ranged between 98% among consultants, 87% among fellows, 70% among technicians, 63% among nurses, and finally 52% among medical students. The results of the eye fixations count, and eye fixation revisitations indicate that the less experienced cardiology practitioners need to interpret several ECG leads more carefully before making any decision. However, more experienced cardiology practitioners rely on their skills to recognize the visual signal patterns of different cardiac abnormalities, providing an accurate ECG interpretation. Conclusions: The results show that visual expertise for ECG interpretation is linked to the practitioner’s role within the health care system and the number of years of practical experience interpreting ECGs. Cardiology practitioners focus on different ECG leads and different waveform abnormalities according to their role in the health care sector and their expertise levels. %M 35138258 %R 10.2196/34058 %U https://humanfactors.jmir.org/2022/1/e34058 %U https://doi.org/10.2196/34058 %U http://www.ncbi.nlm.nih.gov/pubmed/35138258 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 2 %P e30351 %T Examining Diurnal Differences in Multidisciplinary Care Teams at a Pediatric Trauma Center Using Electronic Health Record Data: Social Network Analysis %A Durojaiye,Ashimiyu %A Fackler,James %A McGeorge,Nicolette %A Webster,Kristen %A Kharrazi,Hadi %A Gurses,Ayse %+ Armstrong Institute Center for Health Care Human Factors, Johns Hopkins University, 750 E. Pratt St. 15th Floor, Baltimore, MD, 21202, United States, 1 410 637 4387, agurses1@jhmi.edu %K pediatric trauma %K multidisciplinary health team %K multi-team systems %K social network analysis %K electronic health record %K process mining %K fluid teams %D 2022 %7 4.2.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: The care of pediatric trauma patients is delivered by multidisciplinary care teams with high fluidity that may vary in composition and organization depending on the time of day. Objective: This study aims to identify and describe diurnal variations in multidisciplinary care teams taking care of pediatric trauma patients using social network analysis on electronic health record (EHR) data. Methods: Metadata of clinical activities were extracted from the EHR and processed into an event log, which was divided into 6 different event logs based on shift (day or night) and location (emergency department, pediatric intensive care unit, and floor). Social networks were constructed from each event log by creating an edge among the functional roles captured within a similar time interval during a shift. Overlapping communities were identified from the social networks. Day and night network structures for each care location were compared and validated via comparison with secondary analysis of qualitatively derived care team data, obtained through semistructured interviews; and member-checking interviews with clinicians. Results: There were 413 encounters in the 1-year study period, with 65.9% (272/413) and 34.1% (141/413) beginning during day and night shifts, respectively. A single community was identified at all locations during the day and in the pediatric intensive care unit at night, whereas multiple communities corresponding to individual specialty services were identified in the emergency department and on the floor at night. Members of the trauma service belonged to all communities, suggesting that they were responsible for care coordination. Health care professionals found the networks to be largely accurate representations of the composition of the care teams and the interactions among them. Conclusions: Social network analysis was successfully used on EHR data to identify and describe diurnal differences in the composition and organization of multidisciplinary care teams at a pediatric trauma center. %M 35119372 %R 10.2196/30351 %U https://www.jmir.org/2022/2/e30351 %U https://doi.org/10.2196/30351 %U http://www.ncbi.nlm.nih.gov/pubmed/35119372 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 2 %P e27167 %T Prevalence, Factors, and Association of Electronic Communication Use With Patient-Perceived Quality of Care From the 2019 Health Information National Trends Survey 5-Cycle 3: Exploratory Study %A Yang,Rumei %A Zeng,Kai %A Jiang,Yun %+ School of Nursing, Nanjing Medical University, 818 Tianyuan E Rd, Nanjing, 211166, China, 86 02586869558, rumeiyang@njmu.edu.cn %K electronic communication %K quality of care %K person-related characteristics %K patient preference %K HINTS %D 2022 %7 4.2.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Electronic communication (e-communication), referring to communication through electronic platforms such as the web, patient portal, or mobile phone, has become increasingly important, as it extends traditional in-person communication with fewer limitations of timing and locations. However, little is known about the current status of patients’ use of e-communication with clinicians and whether the use is related to the better patient-perceived quality of care at the population level. Objective: The aim of this study was to explore the prevalence of and the factors associated with e-communication use and the association of e-communication use with patient-perceived quality of care by using the nationally representative sample of the 2019 Health Information National Trends Survey 5 (HINTS 5)-Cycle 3. Methods: Data from 5438 adult responders (mean age 49.04 years, range 18-98 years) were included in this analysis. Multiple logistic and linear regressions were conducted to explore responders’ personal characteristics related to their use of e-communication with clinicians in the past 12 months and how their use was related to perceived quality of care. Descriptive analyses for e-communication use according to age groups were also performed. All analyses considered the complex survey design using the jackknife replication method. Results: The overall prevalence of e-communication use was 60.3%, with a significantly lower prevalence in older adults (16.6%) than that in <45-year-old adults (41%) and 45-65-year-old adults (42.4%). All percentages are weighted; therefore, absolute values are not shown. American adults who used e-communication were more likely to be high school graduates (odds ratio [OR] 1.95, 95% CI 1.14-3.34; P=.02), some college degree holders (OR 3.34, 95% CI 1.84-6.05; P<.001), and college graduates or more (OR 4.89, 95% CI 2.67-8.95; P<.001). Further, people who were females (OR 1.47, 95% CI 1.18-1.82; P=.001), with a household income ≥US $50,000 (OR 1.63, 95% CI 1.23-2.16; P=.001), with more comorbidities (OR 1.22, 95% CI 1.07-1.40; P=.004), or having a regular health care provider (OR 2.62, 95% CI 1.98-3.47; P<.001), were more likely to use e-communication. In contrast, those who resided in rural areas (OR 0.61, 95% CI 0.43-0.88; P=.009) were less likely to use e-communication. After controlling for the sociodemographics, the number of comorbidities, and relationship factors (ie, having a regular provider and trusting a doctor), e-communication use was found to be significantly associated with better perceived quality of care (β=.12, 95% CI 0.02-0.22; P=.02). Conclusions: This study confirmed the positive association between e-communication use and patient-perceived quality of care and suggested that policy-level attention should be raised to engage the socially disadvantaged (ie, those with lower levels of education and income, without a regular health care provider, and living in rural areas) to maximize e-communication use and to support better patient-perceived quality of care among American adults. %M 35119369 %R 10.2196/27167 %U https://www.jmir.org/2022/2/e27167 %U https://doi.org/10.2196/27167 %U http://www.ncbi.nlm.nih.gov/pubmed/35119369 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 2 %P e30483 %T Disease Progression of Hypertrophic Cardiomyopathy: Modeling Using Machine Learning %A Pičulin,Matej %A Smole,Tim %A Žunkovič,Bojan %A Kokalj,Enja %A Robnik-Šikonja,Marko %A Kukar,Matjaž %A Fotiadis,Dimitrios I %A Pezoulas,Vasileios C %A Tachos,Nikolaos S %A Barlocco,Fausto %A Mazzarotto,Francesco %A Popović,Dejana %A Maier,Lars S %A Velicki,Lazar %A Olivotto,Iacopo %A MacGowan,Guy A %A Jakovljević,Djordje G %A Filipović,Nenad %A Bosnić,Zoran %+ Faculty of Computer and Information Science, University of Ljubljana, Večna pot 113, Ljubljana, 1000, Slovenia, 386 1 479 8226, matej.piculin@fri.uni-lj.si %K hypertrophic cardiomyopathy %K disease progression %K machine learning %K artificial intelligence %K AI %K ML %K cardiomyopathy %K cardiovascular disease %K sudden cardiac death %K SCD %K prediction %K prediction model %K validation %D 2022 %7 2.2.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Cardiovascular disorders in general are responsible for 30% of deaths worldwide. Among them, hypertrophic cardiomyopathy (HCM) is a genetic cardiac disease that is present in about 1 of 500 young adults and can cause sudden cardiac death (SCD). Objective: Although the current state-of-the-art methods model the risk of SCD for patients, to the best of our knowledge, no methods are available for modeling the patient's clinical status up to 10 years ahead. In this paper, we propose a novel machine learning (ML)-based tool for predicting disease progression for patients diagnosed with HCM in terms of adverse remodeling of the heart during a 10-year period. Methods: The method consisted of 6 predictive regression models that independently predict future values of 6 clinical characteristics: left atrial size, left atrial volume, left ventricular ejection fraction, New York Heart Association functional classification, left ventricular internal diastolic diameter, and left ventricular internal systolic diameter. We supplemented each prediction with the explanation that is generated using the Shapley additive explanation method. Results: The final experiments showed that predictive error is lower on 5 of the 6 constructed models in comparison to experts (on average, by 0.34) or a consortium of experts (on average, by 0.22). The experiments revealed that semisupervised learning and the artificial data from virtual patients help improve predictive accuracies. The best-performing random forest model improved R2 from 0.3 to 0.6. Conclusions: By engaging medical experts to provide interpretation and validation of the results, we determined the models' favorable performance compared to the performance of experts for 5 of 6 targets. %M 35107432 %R 10.2196/30483 %U https://medinform.jmir.org/2022/2/e30483 %U https://doi.org/10.2196/30483 %U http://www.ncbi.nlm.nih.gov/pubmed/35107432 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 2 %P e29978 %T Reduction of Platelet Outdating and Shortage by Forecasting Demand With Statistical Learning and Deep Neural Networks: Modeling Study %A Schilling,Maximilian %A Rickmann,Lennart %A Hutschenreuter,Gabriele %A Spreckelsen,Cord %+ Institute for Medical Informatics, University Hospital Aachen, RWTH Aachen University, Pauwelsstraße 30, Aachen, 52074, Germany, 49 1784599836, maximilian.schilling@rwth-aachen.de %K platelets %K demand forecasting %K time series forecasting %K blood inventory management %K statistical learning %K deep learning %K LASSO %K LSTM %D 2022 %7 1.2.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Platelets are a valuable and perishable blood product. Managing platelet inventory is a demanding task because of short shelf lives and high variation in daily platelet use patterns. Predicting platelet demand is a promising step toward avoiding obsolescence and shortages and ensuring optimal care. Objective: The aim of this study is to forecast platelet demand for a given hospital using both a statistical model and a deep neural network. In addition, we aim to calculate the possible reduction in waste and shortage of platelets using said predictions in a retrospective simulation of the platelet inventory. Methods: Predictions of daily platelet demand were made by a least absolute shrinkage and selection operator (LASSO) model and a recurrent neural network (RNN) with long short-term memory (LSTM). Both models used the same set of 81 clinical features. Predictions were passed to a simulation of the blood inventory to calculate the possible reduction in waste and shortage as compared with historical data. Results: From January 1, 2008, to December 31, 2018, the waste and shortage rates for platelets were 10.1% and 6.5%, respectively. In simulations of platelet inventory, waste could be lowered to 4.9% with the LASSO and 5% with the RNN, whereas shortages were 2.1% and 1.7% with the LASSO and RNN, respectively. Daily predictions of platelet demand for the next 2 days had mean absolute percent errors of 25.5% (95% CI 24.6%-26.6%) with the LASSO and 26.3% (95% CI 25.3%-27.4%) with the LSTM (P=.01). Predictions for the next 4 days had mean absolute percent errors of 18.1% (95% CI 17.6%-18.6%) with the LASSO and 19.2% (95% CI 18.6%-19.8%) with the LSTM (P<.001). Conclusions: Both models allow for predictions of platelet demand with similar and sufficient accuracy to significantly reduce waste and shortage in a retrospective simulation study. The possible improvements in platelet inventory management are roughly equivalent to US $250,000 per year. %M 35103612 %R 10.2196/29978 %U https://medinform.jmir.org/2022/2/e29978 %U https://doi.org/10.2196/29978 %U http://www.ncbi.nlm.nih.gov/pubmed/35103612 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 1 %P e29458 %T Using Electronic Health Records for Personalized Dosing of Intravenous Vancomycin in Critically Ill Neonates: Model and Web-Based Interface Development Study %A Hui,Ka Ho Matthew %A Lam,Hugh Simon %A Chow,Cheuk Hin Twinny %A Li,Yuen Shun Janice %A Leung,Pok Him Tom %A Chan,Long Yin Brian %A Lee,Chui Ping %A Ewig,Celeste Lom Ying %A Cheung,Yin Ting %A Lam,Tai Ning Teddy %+ School of Pharmacy, Faculty of Medicine, The Chinese University of Hong Kong, 8/F, Lo Kwee-Seong Integrated Biomedical Sciences Building, Area 39, Shatin, New Territories, Hong Kong, Hong Kong, 852 39436827, teddylam@cuhk.edu.hk %K digital health %K web-based user interface %K personalized medicine %K dose individualization %K therapeutic drug monitoring %K Bayesian estimation %K antibiotics %K vancomycin %K infectious disease %K neonate %D 2022 %7 31.1.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Intravenous (IV) vancomycin is used in the treatment of severe infection in neonates. However, its efficacy is compromised by elevated risks of acute kidney injury. The risk is even higher among neonates admitted to the neonatal intensive care unit (NICU), in whom the pharmacokinetics of vancomycin vary widely. Therapeutic drug monitoring is an integral part of vancomycin treatment to balance efficacy against toxicity. It involves individual dose adjustments based on the observed serum vancomycin concentration (VCs). However, the existing trough-based approach shows poor evidence for clinical benefits. The updated clinical practice guideline recommends population pharmacokinetic (popPK) model–based approaches, targeting area under curve, preferably through the Bayesian approach. Since Bayesian methods cannot be performed manually and require specialized computer programs, there is a need to provide clinicians with a user-friendly interface to facilitate accurate personalized dosing recommendations for vancomycin in critically ill neonates. Objective: We used medical data from electronic health records (EHRs) to develop a popPK model and subsequently build a web-based interface to perform model-based individual dose optimization of IV vancomycin for NICU patients in local medical institutions. Methods: Medical data of subjects prescribed IV vancomycin in the NICUs of Prince of Wales Hospital and Queen Elizabeth Hospital in Hong Kong were extracted from EHRs, namely the Clinical Information System, In-Patient Medication Order Entry, and electronic Patient Record. Patient demographics, such as body weight and postmenstrual age (PMA), serum creatinine (SCr), vancomycin administration records, and VCs were collected. The popPK model employed a 2-compartment infusion model. Various covariate models were tested against body weight, PMA, and SCr, and were evaluated for the best goodness of fit. A previously published web-based dosing interface was adapted to develop the interface in this study. Results: The final data set included EHR data extracted from 207 subjects, with a total of 689 VCs measurements. The final model chosen explained 82% of the variability in vancomycin clearance. All parameter estimates were within the bootstrapping CIs. Predictive plots, residual plots, and visual predictive checks demonstrated good model predictability. Model approximations showed that the model-based Bayesian approach consistently promoted a probability of target attainment (PTA) above 75% for all subjects, while only half of the subjects could achieve a PTA over 50% with the trough-based approach. The dosing interface was developed with the capability to optimize individual doses with the model-based empirical or Bayesian approach. Conclusions: Using EHRs, a satisfactory popPK model was verified and adopted to develop a web-based individual dose optimization interface. The interface is expected to improve treatment outcomes of IV vancomycin for severe infections among critically ill neonates. This study provides the foundation for a cohort study to demonstrate the utility of the new approach compared with previous dosing methods. %M 35099393 %R 10.2196/29458 %U https://medinform.jmir.org/2022/1/e29458 %U https://doi.org/10.2196/29458 %U http://www.ncbi.nlm.nih.gov/pubmed/35099393 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 1 %P e29015 %T Identifying Information Gaps in Electronic Health Records by Using Natural Language Processing: Gynecologic Surgery History Identification %A Moon,Sungrim %A Carlson,Luke A %A Moser,Ethan D %A Agnikula Kshatriya,Bhavani Singh %A Smith,Carin Y %A Rocca,Walter A %A Gazzuola Rocca,Liliana %A Bielinski,Suzette J %A Liu,Hongfang %A Larson,Nicholas B %+ Division of Clinical Trials and Biostatistics, Department of Quantitative Health Sciences, Mayo Clinic, 200 First Street SW, Rochester, MN, 55905, United States, 1 507 293 1700, Larson.Nicholas@mayo.edu %K information gap %K health information interoperability %K natural language processing %K electronic health records %K gynecologic surgery %K surgery %K medical informatics %K digital health %K eHealth %K gynecology %D 2022 %7 28.1.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Electronic health records (EHRs) are a rich source of longitudinal patient data. However, missing information due to clinical care that predated the implementation of EHR system(s) or care that occurred at different medical institutions impedes complete ascertainment of a patient’s medical history. Objective: This study aimed to investigate information discrepancies and to quantify information gaps by comparing the gynecological surgical history extracted from an EHR of a single institution by using natural language processing (NLP) techniques with the manually curated surgical history information through chart review of records from multiple independent regional health care institutions. Methods: To facilitate high-throughput evaluation, we developed a rule-based NLP algorithm to detect gynecological surgery history from the unstructured narrative of the Mayo Clinic EHR. These results were compared to a gold standard cohort of 3870 women with gynecological surgery status adjudicated using the Rochester Epidemiology Project medical records–linkage system. We quantified and characterized the information gaps observed that led to misclassification of the surgical status. Results: The NLP algorithm achieved precision of 0.85, recall of 0.82, and F1-score of 0.83 in the test set (n=265) relative to outcomes abstracted from the Mayo EHR. This performance attenuated when directly compared to the gold standard (precision 0.79, recall 0.76, and F1-score 0.76), with the majority of misclassifications being false negatives in nature. We then applied the algorithm to the remaining patients (n=3340) and identified 2 types of information gaps through error analysis. First, 6% (199/3340) of women in this study had no recorded surgery information or partial information in the EHR. Second, 4.3% (144/3340) of women had inconsistent or inaccurate information within the clinical narrative owing to misinterpreted information, erroneous “copy and paste,” or incorrect information provided by patients. Additionally, the NLP algorithm misclassified the surgery status of 3.6% (121/3340) of women. Conclusions: Although NLP techniques were able to adequately recreate the gynecologic surgical status from the clinical narrative, missing or inaccurately reported and recorded information resulted in much of the misclassification observed. Therefore, alternative approaches to collect or curate surgical history are needed. %M 35089141 %R 10.2196/29015 %U https://www.jmir.org/2022/1/e29015 %U https://doi.org/10.2196/29015 %U http://www.ncbi.nlm.nih.gov/pubmed/35089141 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 1 %P e29333 %T The Applied Data Analytics in Medicine Program: Lessons Learned From Four Years’ Experience With Personalizing Health Care in an Academic Teaching Hospital %A Haitjema,Saskia %A Prescott,Timothy R %A van Solinge,Wouter W %+ Central Diagnostic Laboratory, University Medical Center Utrecht, Utrecht University, Heidelberglaan 100, Room #G.03.551, Utrecht, 3584 CX, Netherlands, 31 088 7550759, s.haitjema@umcutrecht.nl %K digital health %K data-driven care %K multidisciplinarity %K lessons learned %K eHealth %K personalized medicine %K data analytics %K implementation %K collaboration %K hospital %D 2022 %7 28.1.2022 %9 Viewpoint %J JMIR Form Res %G English %X The University Medical Center (UMC) Utrecht piloted a hospital-wide innovation data analytics program over the past 4 years. The goal was, based on available data and innovative data analytics methodologies, to answer clinical questions to improve patient care. In this viewpoint, we aimed to support and inspire others pursuing similar efforts by sharing the three principles of the program: the data analytics value chain (data, insight, action, value), the innovation funnel (structured innovation approach with phases and gates), and the multidisciplinary team (patients, clinicians, and data scientists). We also discussed our most important lessons learned: the importance of a clinical question, collaboration challenges between health care professionals and different types of data scientists, the win-win result of our collaboration with external partners, the prerequisite of available meaningful data, the (legal) complexity of implementation, organizational power, and the embedding of collaborative efforts in the health care system as a whole. %M 35089145 %R 10.2196/29333 %U https://formative.jmir.org/2022/1/e29333 %U https://doi.org/10.2196/29333 %U http://www.ncbi.nlm.nih.gov/pubmed/35089145 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 1 %P e31918 %T The Development History and Research Tendency of Medical Informatics: Topic Evolution Analysis %A Han,Wenting %A Han,Xi %A Zhou,Sijia %A Zhu,Qinghua %+ School of Business Administration, Guangdong University of Finance & Economics, 21 Luntou Road, Guangzhou, 510320, China, 86 13512791305, hanx015@163.com %K medical informatics %K research hotspot %K LDA model %K topic evolution analysis %K mobile phone %D 2022 %7 27.1.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Medical informatics has attracted the attention of researchers worldwide. It is necessary to understand the development of its research hot spots as well as directions for future research. Objective: The aim of this study is to explore the evolution of medical informatics research topics by analyzing research articles published between 1964 and 2020. Methods: A total of 56,466 publications were collected from 27 representative medical informatics journals indexed by the Web of Science Core Collection. We identified the research stages based on the literature growth curve, extracted research topics using the latent Dirichlet allocation model, and analyzed topic evolution patterns by calculating the cosine similarity between topics from the adjacent stages. Results: The following three research stages were identified: early birth, early development, and rapid development. Medical informatics has entered the fast development stage, with literature growing exponentially. Research topics in medical informatics can be classified into the following two categories: data-centered studies and people-centered studies. Medical data analysis has been a research hot spot across all 3 stages, and the integration of emerging technologies into data analysis might be a future hot spot. Researchers have focused more on user needs in the last 2 stages. Another potential hot spot might be how to meet user needs and improve the usability of health tools. Conclusions: Our study provides a comprehensive understanding of research hot spots in medical informatics, as well as evolution patterns among them, which was helpful for researchers to grasp research trends and design their studies. %M 35084351 %R 10.2196/31918 %U https://medinform.jmir.org/2022/1/e31918 %U https://doi.org/10.2196/31918 %U http://www.ncbi.nlm.nih.gov/pubmed/35084351 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e23833 %T An Information Directory App InHouse Call for Streamlining Communication to Optimize Efficiency and Patient Care in a Hospital: Pilot Mixed Methods Design and Utility Study %A Schilling,George %A Villarosa,Leonardo %+ Vidant Medical Center, 2100 Statonsburg Rd, Greenville, NC, 27834, United States, 1 252 847 4441, georgeschilling@gmail.com %K InHouse Call %K communication %K hospital directory %K healthcare %K health care %K health informatics %K mHealth %K mobile app %K digital health %K patient records %K electronic health %D 2022 %7 27.1.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Communication failures disrupt physician workflow, lead to poor patient outcomes, and are associated with significant economic burden. To increase efficiency when contacting a team member in a hospital, we have designed an information directory app, InHouse Call. Objective: This study aimed to describe the design of InHouse Call, objectively compare the usefulness of the app versus that of traditional methods (operator or pocket cards, etc), and determine its subjective usefulness through user surveys and a net promoter score (NPS). Methods: This pilot study utilizing before-after trials was carried out at a tertiary academic hospital and involved 20 clinicians, including physiatrists, hospitalists, internal medicine and family medicine residents, and advanced practice providers/nurse practitioners/physician assistants. InHouse Call was designed to efficiently supply contact information to providers through a simple, user-friendly interface. The participants used InHouse Call in timed trials to contact a health care team member in the hospital via a telephone call. The effectiveness of InHouse Call in connecting the user with a contact in the hospital was measured through timed trials comparing the amount of time spent in attempting to make the connection using traditional methods versus the app. Usability was measured through exit surveys and NPS. Results: The average time spent connecting to the correct contact using traditional methods was 59.5 seconds, compared to 13.8 seconds when using InHouse Call. The degree of variance when using traditional methods was 1544.2, compared to 19.7 with InHouse Call. A call made using the traditional methods deviated from the mean by 39.3 seconds, compared to 4.4 seconds when using InHouse Call. InHouse Call achieved an NPS of 95. Conclusions: InHouse Call significantly reduced the average amount of time spent connecting with the correct contact as well as the variability to complete the task, thus proving to be the superior method of communication for health care providers. The app garnered a high NPS and positive subjective feedback. %M 35084350 %R 10.2196/23833 %U https://humanfactors.jmir.org/2022/1/e23833 %U https://doi.org/10.2196/23833 %U http://www.ncbi.nlm.nih.gov/pubmed/35084350 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 1 %P e27349 %T A Remote Monitoring System to Optimize the Home Management of Oral Anticancer Therapies (ONCO-TreC): Prospective Training–Validation Trial %A Passardi,Alessandro %A Foca,Flavia %A Caffo,Orazio %A Tondini,Carlo Alberto %A Zambelli,Alberto %A Vespignani,Roberto %A Bartolini,Giulia %A Sullo,Francesco Giulio %A Andreis,Daniele %A Dianti,Marco %A Eccher,Claudio %A Piras,Enrico Maria %A Forti,Stefano %+ IT Service, IRCCS Istituto Romagnolo per lo Studio dei Tumori “Dino Amadori”, Via P. Maroncelli 40, Meldola, 47014, Italy, 39 0543 739992, roberto.vespignani@irst.emr.it %K adherence %K oral anticancer drug %K mHealth %K ONCO-TreC %K electronic diary %D 2022 %7 26.1.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: A platform designed to support the home management of oral anticancer treatments and provide a secure web-based patient–health care professional communication modality, ONCO-TreC, was tested in 3 cancer centers in Italy. Objective: The overall aims of the trial are to customize the platform; assess the system’s ability to facilitate the shared management of oral anticancer therapies by patients and health professionals; and evaluate system usability and acceptability by patients, caregivers, and health care professionals. Methods: Patients aged ≥18 years who were candidates for oral anticancer treatment as monotherapy with an Eastern Cooperative Oncology Group performance status score of 0 to 1 and a sufficient level of familiarity with mobile devices were eligible. ONCO-TreC consisted of a mobile app for patients and a web-based dashboard for health care professionals. Adherence to treatment (pill count) and toxicities reported by patients through the app were compared with those reported by physicians in medical records. Usability and acceptability were evaluated using questionnaires. Results: A total of 40 patients were enrolled, 38 (95%) of whom were evaluable for adherence to treatment. The ability of the system to measure adherence to treatment was high, with a concordance of 97.3% (95% CI 86.1%-99.9%) between the investigator and system pill count. Only 60% (3/5) of grade 3, 54% (13/24) of grade 2, and 19% (7/36) of grade 1 adverse events reported by physicians in the case report forms were also reported in the app directly by patients. In total, 94% (33/35) of patients had ≥1 app launch each week, and the median number of daily accesses per patient was 2. Approximately 71% (27/38) and 68% (26/38) of patients used the app for messages and vital sign entering, respectively, at least once during the study period. Conclusions: ONCO-TreC is an important tool for measuring and monitoring adherence to oral anticancer drugs. System usability and acceptability were very high, whereas its reliability in registering toxicity could be improved. Trial Registration: ClinicalTrials.gov NCT02921724; https://www.clinicaltrials.gov/ct2/show/NCT02921724 %M 35080505 %R 10.2196/27349 %U https://www.jmir.org/2022/1/e27349 %U https://doi.org/10.2196/27349 %U http://www.ncbi.nlm.nih.gov/pubmed/35080505 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 1 %P e31356 %T Real-world Health Data and Precision for the Diagnosis of Acute Kidney Injury, Acute-on-Chronic Kidney Disease, and Chronic Kidney Disease: Observational Study %A Triep,Karen %A Leichtle,Alexander Benedikt %A Meister,Martin %A Fiedler,Georg Martin %A Endrich,Olga %+ Medical Directorate, Medizincontrolling, Inselspital, University Hospital Bern, Insel Gruppe, S96 C223, Schwarztorstrasse 96, Bern, 3010, Switzerland, 41 31 632 51 96, karen.triep@insel.ch %K acute kidney injury %K chronic kidney disease %K acute-on-chronic %K real-world health data %K clinical decision support %K KDIGO %K ICD coding %D 2022 %7 25.1.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: The criteria for the diagnosis of kidney disease outlined in the Kidney Disease: Improving Global Outcomes guidelines are based on a patient’s current, historical, and baseline data. The diagnosis of acute kidney injury, chronic kidney disease, and acute-on-chronic kidney disease requires previous measurements of creatinine, back-calculation, and the interpretation of several laboratory values over a certain period. Diagnoses may be hindered by unclear definitions of the individual creatinine baseline and rough ranges of normal values that are set without adjusting for age, ethnicity, comorbidities, and treatment. The classification of correct diagnoses and sufficient staging improves coding, data quality, reimbursement, the choice of therapeutic approach, and a patient’s outcome. Objective: In this study, we aim to apply a data-driven approach to assign diagnoses of acute, chronic, and acute-on-chronic kidney diseases with the help of a complex rule engine. Methods: Real-time and retrospective data from the hospital’s clinical data warehouse of inpatient and outpatient cases treated between 2014 and 2019 were used. Delta serum creatinine, baseline values, and admission and discharge data were analyzed. A Kidney Disease: Improving Global Outcomes–based SQL algorithm applied specific diagnosis-based International Classification of Diseases (ICD) codes to inpatient stays. Text mining on discharge documentation was also conducted to measure the effects on diagnosis. Results: We show that this approach yielded an increased number of diagnoses (4491 cases in 2014 vs 11,124 cases of ICD-coded kidney disease and injury in 2019) and higher precision in documentation and coding. The percentage of unspecific ICD N19-coded diagnoses of N19 codes generated dropped from 19.71% (1544/7833) in 2016 to 4.38% (416/9501) in 2019. The percentage of specific ICD N18-coded diagnoses of N19 codes generated increased from 50.1% (3924/7833) in 2016 to 62.04% (5894/9501) in 2019. Conclusions: Our data-driven method supports the process and reliability of diagnosis and staging and improves the quality of documentation and data. Measuring patient outcomes will be the next step in this project. %M 35076410 %R 10.2196/31356 %U https://medinform.jmir.org/2022/1/e31356 %U https://doi.org/10.2196/31356 %U http://www.ncbi.nlm.nih.gov/pubmed/35076410 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 1 %P e28036 %T Energy Efficiency of Inference Algorithms for Clinical Laboratory Data Sets: Green Artificial Intelligence Study %A Yu,Jia-Ruei %A Chen,Chun-Hsien %A Huang,Tsung-Wei %A Lu,Jang-Jih %A Chung,Chia-Ru %A Lin,Ting-Wei %A Wu,Min-Hsien %A Tseng,Yi-Ju %A Wang,Hsin-Yao %+ Department of Laboratory Medicine, Chang Gung Memorial Hospital at Linkou, No 5 Fuxing Street, Guishan District, Taoyuan City, 333, Taiwan, 886 978112962, mdhsinyaowang@gmail.com %K medical informatics %K machine learning %K algorithms %K energy consumption %K artificial intelligence %K energy efficient %K medical domain %K medical data sets %K informatics %D 2022 %7 25.1.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: The use of artificial intelligence (AI) in the medical domain has attracted considerable research interest. Inference applications in the medical domain require energy-efficient AI models. In contrast to other types of data in visual AI, data from medical laboratories usually comprise features with strong signals. Numerous energy optimization techniques have been developed to relieve the burden on the hardware required to deploy a complex learning model. However, the energy efficiency levels of different AI models used for medical applications have not been studied. Objective: The aim of this study was to explore and compare the energy efficiency levels of commonly used machine learning algorithms—logistic regression (LR), k-nearest neighbor, support vector machine, random forest (RF), and extreme gradient boosting (XGB) algorithms, as well as four different variants of neural network (NN) algorithms—when applied to clinical laboratory datasets. Methods: We applied the aforementioned algorithms to two distinct clinical laboratory data sets: a mass spectrometry data set regarding Staphylococcus aureus for predicting methicillin resistance (3338 cases; 268 features) and a urinalysis data set for predicting Trichomonas vaginalis infection (839,164 cases; 9 features). We compared the performance of the nine inference algorithms in terms of accuracy, area under the receiver operating characteristic curve (AUROC), time consumption, and power consumption. The time and power consumption levels were determined using performance counter data from Intel Power Gadget 3.5. Results: The experimental results indicated that the RF and XGB algorithms achieved the two highest AUROC values for both data sets (84.7% and 83.9%, respectively, for the mass spectrometry data set; 91.1% and 91.4%, respectively, for the urinalysis data set). The XGB and LR algorithms exhibited the shortest inference time for both data sets (0.47 milliseconds for both in the mass spectrometry data set; 0.39 and 0.47 milliseconds, respectively, for the urinalysis data set). Compared with the RF algorithm, the XGB and LR algorithms exhibited a 45% and 53%-60% reduction in inference time for the mass spectrometry and urinalysis data sets, respectively. In terms of energy efficiency, the XGB algorithm exhibited the lowest power consumption for the mass spectrometry data set (9.42 Watts) and the LR algorithm exhibited the lowest power consumption for the urinalysis data set (9.98 Watts). Compared with a five-hidden-layer NN, the XGB and LR algorithms achieved 16%-24% and 9%-13% lower power consumption levels for the mass spectrometry and urinalysis data sets, respectively. In all experiments, the XGB algorithm exhibited the best performance in terms of accuracy, run time, and energy efficiency. Conclusions: The XGB algorithm achieved balanced performance levels in terms of AUROC, run time, and energy efficiency for the two clinical laboratory data sets. Considering the energy constraints in real-world scenarios, the XGB algorithm is ideal for medical AI applications. %M 35076405 %R 10.2196/28036 %U https://www.jmir.org/2022/1/e28036 %U https://doi.org/10.2196/28036 %U http://www.ncbi.nlm.nih.gov/pubmed/35076405 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 1 %P e27743 %T A Platform and Multisided Market for Translational, Software-Defined Medical Procedures in the Operating Room (OP 4.1): Proof-of-Concept Study %A Görtz,Magdalena %A Byczkowski,Michael %A Rath,Mathias %A Schütz,Viktoria %A Reimold,Philipp %A Gasch,Claudia %A Simpfendörfer,Tobias %A März,Keno %A Seitel,Alexander %A Nolden,Marco %A Ross,Tobias %A Mindroc-Filimon,Diana %A Michael,Dominik %A Metzger,Jasmin %A Onogur,Sinan %A Speidel,Stefanie %A Mündermann,Lars %A Fallert,Johannes %A Müller,Michael %A von Knebel Doeberitz,Magnus %A Teber,Dogu %A Seitz,Peter %A Maier-Hein,Lena %A Duensing,Stefan %A Hohenfellner,Markus %+ Department of Urology, Heidelberg University Hospital, Im Neuenheimer Feld 420, Heidelberg, 69120, Germany, 49 6221568820, magdalena.goertz@med.uni-heidelberg.de %K cloud-based platform %K data %K eHealth %K Internet of Medical Things %K IoT %K medical apps %K multisided market %K perioperative medicine %K software-defined healthcare %K translational research %D 2022 %7 20.1.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Although digital and data-based technologies are widespread in various industries in the context of Industry 4.0, the use of smart connected devices in health care is still in its infancy. Innovative solutions for the medical environment are affected by difficult access to medical device data and high barriers to market entry because of proprietary systems. Objective: In the proof-of-concept project OP 4.1, we show the business viability of connecting and augmenting medical devices and data through software add-ons by giving companies a technical and commercial platform for the development, implementation, distribution, and billing of innovative software solutions. Methods: The creation of a central platform prototype requires the collaboration of several independent market contenders, including medical users, software developers, medical device manufacturers, and platform providers. A dedicated consortium of clinical and scientific partners as well as industry partners was set up. Results: We demonstrate the successful development of the prototype of a user-centric, open, and extensible platform for the intelligent support of processes starting with the operating room. By connecting heterogeneous data sources and medical devices from different manufacturers and making them accessible for software developers and medical users, the cloud-based platform OP 4.1 enables the augmentation of medical devices and procedures through software-based solutions. The platform also allows for the demand-oriented billing of apps and medical devices, thus permitting software-based solutions to fast-track their economic development and become commercially successful. Conclusions: The technology and business platform OP 4.1 creates a multisided market for the successful development, implementation, distribution, and billing of new software solutions in the operating room and in the health care sector in general. Consequently, software-based medical innovation can be translated into clinical routine quickly, efficiently, and cost-effectively, optimizing the treatment of patients through smartly assisted procedures. %M 35049510 %R 10.2196/27743 %U https://medinform.jmir.org/2022/1/e27743 %U https://doi.org/10.2196/27743 %U http://www.ncbi.nlm.nih.gov/pubmed/35049510 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 1 %P e17278 %T The Use of Blockchain Technology in the Health Care Sector: Systematic Review %A Elangovan,Deepa %A Long,Chiau Soon %A Bakrin,Faizah Safina %A Tan,Ching Siang %A Goh,Khang Wen %A Yeoh,Siang Fei %A Loy,Mei Jun %A Hussain,Zahid %A Lee,Kah Seng %A Idris,Azam Che %A Ming,Long Chiau %+ Faculty of Information Technology, INTI International University, Persiaran Perdana BBN Putra Nilai, Nilai, 71800, Malaysia, 60 6 798 2000, khangwen.goh@newinti.edu.my %K blockchain %K health care %K hospital information system %K data integrity %K access control %K data logging %K health informatics %D 2022 %7 20.1.2022 %9 Review %J JMIR Med Inform %G English %X Background: Blockchain technology is a part of Industry 4.0’s new Internet of Things applications: decentralized systems, distributed ledgers, and immutable and cryptographically secure technology. This technology entails a series of transaction lists with identical copies shared and retained by different groups or parties. One field where blockchain technology has tremendous potential is health care, due to the more patient-centric approach to the health care system as well as blockchain’s ability to connect disparate systems and increase the accuracy of electronic health records. Objective: The aim of this study was to systematically review studies on the use of blockchain technology in health care and to analyze the characteristics of the studies that have implemented blockchain technology. Methods: This study used a systematic review methodology to find literature related to the implementation aspect of blockchain technology in health care. Relevant papers were searched for using PubMed, SpringerLink, IEEE Xplore, Embase, Scopus, and EBSCOhost. A quality assessment of literature was performed on the 22 selected papers by assessing their trustworthiness and relevance. Results: After full screening, 22 papers were included. A table of evidence was constructed, and the results of the selected papers were interpreted. The results of scoring for measuring the quality of the publications were obtained and interpreted. Out of 22 papers, a total of 3 (14%) high-quality papers, 9 (41%) moderate-quality papers, and 10 (45%) low-quality papers were identified. Conclusions: Blockchain technology was found to be useful in real health care environments, including for the management of electronic medical records, biomedical research and education, remote patient monitoring, pharmaceutical supply chains, health insurance claims, health data analytics, and other potential areas. The main reasons for the implementation of blockchain technology in the health care sector were identified as data integrity, access control, data logging, data versioning, and nonrepudiation. The findings could help the scientific community to understand the implementation aspect of blockchain technology. The results from this study help in recognizing the accessibility and use of blockchain technology in the health care sector. %M 35049516 %R 10.2196/17278 %U https://medinform.jmir.org/2022/1/e17278 %U https://doi.org/10.2196/17278 %U http://www.ncbi.nlm.nih.gov/pubmed/35049516 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 6 %N 1 %P e27550 %T Toward Data-Driven Radiation Oncology Using Standardized Terminology as a Starting Point: Cross-sectional Study %A Cihoric,Nikola %A Badra,Eugenia Vlaskou %A Stenger-Weisser,Anna %A Aebersold,Daniel M %A Pavic,Matea %+ Inselspital, Bern University Hospital, University of Bern, Freiburgstrasse 18, Bern, 3010, Switzerland, 41 31 632 26 32, nikola.cihoric@gmail.com %K terminology %K semantic interoperability %K radiation oncology %K informatics %K medical informatics %K oncology %K lexical analysis %K eHealth %D 2022 %7 19.1.2022 %9 Original Paper %J JMIR Form Res %G English %X Background: The inability to seamlessly exchange information across radiation therapy ecosystems is a limiting factor in the pursuit of data-driven clinical practice. The implementation of semantic interoperability is a prerequisite for achieving the full capacity of the latest developments in personalized and precision medicine, such as mathematical modeling, advanced algorithmic information processing, and artificial intelligence approaches. Objective: This study aims to evaluate the state of terminology resources (TRs) dedicated to radiation oncology as a prerequisite for an oncology semantic ecosystem. The goal of this cross-sectional analysis is to quantify the state of the art in radiation therapy specific terminology. Methods: The Unified Medical Language System (UMLS) was searched for the following terms: radio oncology, radiation oncology, radiation therapy, and radiotherapy. We extracted 6509 unique concepts for further analysis. We conducted a quantitative analysis of available source vocabularies (SVs) and analyzed all UMLS SVs according to the route source, number, author, location of authors, license type, the lexical density of TR, and semantic types. Descriptive data are presented as numbers and percentages. Results: The concepts were distributed across 35 SVs. The median number of unique concepts per SV was 5 (range 1-5479), with 14% (5/35) of SVs containing 94.59% (6157/6509) of the concepts. The SVs were created by 29 authors, predominantly legal entities registered in the United States (25/35, 71%), followed by international organizations (6/35, 17%), legal entities registered in Australia (2/35, 6%), and the Netherlands and the United Kingdom with 3% (1/35) of authors each. Of the total 35 SVs, 16 (46%) did not have any restrictions on use, whereas for 19 (54%) of SVs, some level of restriction was required. Overall, 57% (20/35) of SVs were updated within the last 5 years. All concepts found within radiation therapy SVs were labeled with one of the 29 semantic types represented within UMLS. After removing the stop words, the total number of words for all SVs together was 56,219, with a median of 25 unique words per SV (range 3-50,682). The total number of unique words in all SVs was 1048, with a median of 19 unique words per vocabulary (range 3-406). The lexical density for all concepts within all SVs was 0 (0.02 rounded to 2 decimals). Median lexical density per unique SV was 0.7 (range 0.0-1.0). There were no dedicated radiation therapy SVs. Conclusions: We did not identify any dedicated TRs for radiation oncology. Current terminologies are not sufficient to cover the need of modern radiation oncology practice and research. To achieve a sufficient level of interoperability, of the creation of a new, standardized, universally accepted TR dedicated to modern radiation therapy is required. %M 35044315 %R 10.2196/27550 %U https://formative.jmir.org/2022/1/e27550 %U https://doi.org/10.2196/27550 %U http://www.ncbi.nlm.nih.gov/pubmed/35044315 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 11 %N 1 %P e34573 %T Digital Biomarkers for Supporting Transitional Care Decisions: Protocol for a Transnational Feasibility Study %A Petsani,Despoina %A Ahmed,Sara %A Petronikolou,Vasileia %A Kehayia,Eva %A Alastalo,Mika %A Santonen,Teemu %A Merino-Barbancho,Beatriz %A Cea,Gloria %A Segkouli,Sofia %A Stavropoulos,Thanos G %A Billis,Antonis %A Doumas,Michael %A Almeida,Rosa %A Nagy,Enikő %A Broeckx,Leen %A Bamidis,Panagiotis %A Konstantinidis,Evdokimos %+ Medical Physics and Digital Innovation Laboratory, School of Medicine, Aristotle University of Thessaloniki, University Campus, Thessaloniki, 54124, Greece, 30 6986177524, despoinapets@gmail.com %K Living Lab %K cocreation %K transitional care %K technology %K feasibility study %D 2022 %7 19.1.2022 %9 Protocol %J JMIR Res Protoc %G English %X Background: Virtual Health and Wellbeing Living Lab Infrastructure is a Horizon 2020 project that aims to harmonize Living Lab procedures and facilitate access to European health and well-being research infrastructures. In this context, this study presents a joint research activity that will be conducted within Virtual Health and Wellbeing Living Lab Infrastructure in the transitional care domain to test and validate the harmonized Living Lab procedures and infrastructures. The collection of data from various sources (information and communications technology and clinical and patient-reported outcome measures) demonstrated the capacity to assess risk and support decisions during care transitions, but there is no harmonized way of combining this information. Objective: This study primarily aims to evaluate the feasibility and benefit of collecting multichannel data across Living Labs on the topic of transitional care and to harmonize data processes and collection. In addition, the authors aim to investigate the collection and use of digital biomarkers and explore initial patterns in the data that demonstrate the potential to predict transition outcomes, such as readmissions and adverse events. Methods: The current research protocol presents a multicenter, prospective, observational cohort study that will consist of three phases, running consecutively in multiple sites: a cocreation phase, a testing and simulation phase, and a transnational pilot phase. The cocreation phase aims to build a common understanding among different sites, investigate the differences in hospitalization discharge management among countries, and the willingness of different stakeholders to use technological solutions in the transitional care process. The testing and simulation phase aims to explore ways of integrating observation of a patient’s clinical condition, patient involvement, and discharge education in transitional care. The objective of the simulation phase is to evaluate the feasibility and the barriers faced by health care professionals in assessing transition readiness. Results: The cocreation phase will be completed by April 2022. The testing and simulation phase will begin in September 2022 and will partially overlap with the deployment of the transnational pilot phase that will start in the same month. The data collection of the transnational pilots will be finalized by the end of June 2023. Data processing is expected to be completed by March 2024. The results will consist of guidelines and implementation pathways for large-scale studies and an analysis for identifying initial patterns in the acquired data. Conclusions: The knowledge acquired through this research will lead to harmonized procedures and data collection for Living Labs that support transitions in care. International Registered Report Identifier (IRRID): PRR1-10.2196/34573 %M 35044303 %R 10.2196/34573 %U https://www.researchprotocols.org/2022/1/e34573 %U https://doi.org/10.2196/34573 %U http://www.ncbi.nlm.nih.gov/pubmed/35044303 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e27431 %T Implementation of E-prescription for Multidose Dispensed Drugs: Qualitative Study of General Practitioners’ Experiences %A Gullslett,Monika Knudsen %A Strand Bergmo,Trine %+ Norwegian Centre for E-Health Research, University Hospital of North Norway, Postboks 35, Tromsø, 9038, Norway, 47 90784208, monika.knudsen.gullslett@ehealthresearch.no %K e-prescribing of multidose drug dispensing (eMDD) %K pharmacy %K start-up %K general practitioner (GP) %K Norway %K digital health %K digital tools %K e-prescriptions %K physicians %K qualitative study %D 2022 %7 17.1.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Increased use of pharmaceuticals challenges both capacity and safety related to medication management for patients and changes in how general practitioners (GPs) and other health personnel interact with and follow up with patients. E-prescribing of multidose drug dispensing (eMDD) is 1 of the national measures being tested in Norway. Objective: The objective of this study is to explore GPs’ experiences with the challenges and benefits of implementing eMDD in Norway. Methods: Qualitative in-depth and group interviews were conducted with a total of 25 GPs between 2018 and 2020. Transcribed files were saved in NVivo to conduct a step-by-step content analysis. NVivo is a software tool for organizing, managing, and analyzing qualitative data. Results: The study revealed that eMDD offers many benefits. At the same time, there are several challenges related to information, training, and initiation, as well as to the responsibility for the medication, interactions, and the risk of incorrect medication. An important activity in the start-up phase was an information meeting with pharmacies and technology suppliers, as well as exchanging information and instructions with pharmacies on how to get started. Four analytic themes emerged through the extraction of data: (1) start-up with eMDD (“Be patient”); (2) the need for training; (3) interaction, safety, and efficiency; and (4) the working day with eMDD. Conclusions: There is a variation in different GPs’ needs regarding training and information, and considerable variation in competence and motivation related to the use of digital tools. There are also different degrees of understanding the everyday work of the other actors in the medication chain. In particular, the harmonization of medication lists related to the use of time, expenditures, and challenges with technological solutions in the introduction phase was emphasized as a challenge. Overall, GPs who have started using the system report great benefits; these are largely related to an increased overview of patients’ total medication lists, less time spent on prescribing prescriptions, and increased collaboration with pharmacies and nurses, both in service from providers in homes and in nursing homes. %M 35037881 %R 10.2196/27431 %U https://humanfactors.jmir.org/2022/1/e27431 %U https://doi.org/10.2196/27431 %U http://www.ncbi.nlm.nih.gov/pubmed/35037881 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 1 %P e28323 %T Barriers to and Facilitators of Automated Patient Self-scheduling for Health Care Organizations: Scoping Review %A Woodcock,Elizabeth W %+ Department of Health Policy & Management, Rollins School of Public Health, Emory University, 1518 Clifton Road, Atlanta, GA, 30307, United States, 1 404 272 2274, elizabeth@elizabethwoodcock.com %K appointment %K scheduling %K outpatient %K ambulatory %K online %K self-serve %K e-book %K web-based %K automation %K patient satisfaction %K self-scheduling %K eHealth %K digital health %K mobile phone %D 2022 %7 11.1.2022 %9 Review %J J Med Internet Res %G English %X Background: Appointment management in the outpatient setting is important for health care organizations, as waits and delays lead to poor outcomes. Automated patient self-scheduling of outpatient appointments has demonstrable advantages in the form of patients’ arrival rates, labor savings, patient satisfaction, and more. Despite evidence of the potential benefits of self-scheduling, the organizational uptake of self-scheduling in health care has been limited. Objective: The objective of this scoping review is to identify and to catalog existing evidence of the barriers to and facilitators of self-scheduling for health care organizations. Methods: A scoping review was conducted by searching 4 databases (PubMed, CINAHL, Business Source Ultimate, and Scopus) and systematically reviewing peer-reviewed studies. The Consolidated Framework for Implementation Research was used to catalog the studies. Results: In total, 30 full-text articles were included in this review. The results demonstrated that self-scheduling initiatives have increased over time, indicating the broadening appeal of self-scheduling. The body of literature regarding intervention characteristics is appreciable. Outer setting factors, including national policy, competition, and the response to patients’ needs and technology access, have played an increasing role in influencing implementation over time. Self-scheduling, compared with using the telephone to schedule an appointment, was most often cited as a relative advantage. Scholarly pursuit lacked recommendations related to the framework’s inner setting, characteristics of individuals, and processes as determinants of implementation. Future discoveries regarding these Consolidated Framework for Implementation Research domains may help detect, categorize, and appreciate organizational-level barriers to and facilitators of self-scheduling to advance knowledge regarding this solution. Conclusions: This scoping review cataloged evidence of the existence, advantages, and intervention characteristics of patient self-scheduling. Automated self-scheduling may offer a solution to health care organizations striving to positively affect access. Gaps in knowledge regarding the uptake of self-scheduling by health care organizations were identified to inform future research. %M 35014968 %R 10.2196/28323 %U https://www.jmir.org/2022/1/e28323 %U https://doi.org/10.2196/28323 %U http://www.ncbi.nlm.nih.gov/pubmed/35014968 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 1 %P e25440 %T Understanding the Nature of Metadata: Systematic Review %A Ulrich,Hannes %A Kock-Schoppenhauer,Ann-Kristin %A Deppenwiese,Noemi %A Gött,Robert %A Kern,Jori %A Lablans,Martin %A Majeed,Raphael W %A Stöhr,Mark R %A Stausberg,Jürgen %A Varghese,Julian %A Dugas,Martin %A Ingenerf,Josef %+ IT Center for Clinical Research, University of Lübeck, Ratzeburger Allee 160, Lübeck, 23564, Germany, 49 45131015607, h.ulrich@uni-luebeck.de %K metadata %K metadata definition %K systematic review %K data integration %K data identification %K data classification %D 2022 %7 11.1.2022 %9 Review %J J Med Internet Res %G English %X Background: Metadata are created to describe the corresponding data in a detailed and unambiguous way and is used for various applications in different research areas, for example, data identification and classification. However, a clear definition of metadata is crucial for further use. Unfortunately, extensive experience with the processing and management of metadata has shown that the term “metadata” and its use is not always unambiguous. Objective: This study aimed to understand the definition of metadata and the challenges resulting from metadata reuse. Methods: A systematic literature search was performed in this study following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines for reporting on systematic reviews. Five research questions were identified to streamline the review process, addressing metadata characteristics, metadata standards, use cases, and problems encountered. This review was preceded by a harmonization process to achieve a general understanding of the terms used. Results: The harmonization process resulted in a clear set of definitions for metadata processing focusing on data integration. The following literature review was conducted by 10 reviewers with different backgrounds and using the harmonized definitions. This study included 81 peer-reviewed papers from the last decade after applying various filtering steps to identify the most relevant papers. The 5 research questions could be answered, resulting in a broad overview of the standards, use cases, problems, and corresponding solutions for the application of metadata in different research areas. Conclusions: Metadata can be a powerful tool for identifying, describing, and processing information, but its meaningful creation is costly and challenging. This review process uncovered many standards, use cases, problems, and solutions for dealing with metadata. The presented harmonized definitions and the new schema have the potential to improve the classification and generation of metadata by creating a shared understanding of metadata and its context. %M 35014967 %R 10.2196/25440 %U https://www.jmir.org/2022/1/e25440 %U https://doi.org/10.2196/25440 %U http://www.ncbi.nlm.nih.gov/pubmed/35014967 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 1 %P e17273 %T Characterizing Patient-Clinician Communication in Secure Medical Messages: Retrospective Study %A Huang,Ming %A Fan,Jungwei %A Prigge,Julie %A Shah,Nilay D %A Costello,Brian A %A Yao,Lixia %+ Department of Artificial Intelligence and Informatics, Mayo Clinic, 200 First Street SW, Rochester, MN, 55905, United States, 1 507 293 7953, lixia.cn.yao@gmail.com %K patient portal %K secure message %K patient-clinician communication %K workload %K response time %K message round %D 2022 %7 11.1.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Patient-clinician secure messaging is an important function in patient portals and enables patients and clinicians to communicate on a wide spectrum of issues in a timely manner. With its growing adoption and patient engagement, it is time to comprehensively study the secure messages and user behaviors in order to improve patient-centered care. Objective: The aim of this paper was to analyze the secure messages sent by patients and clinicians in a large multispecialty health system at Mayo Clinic, Rochester. Methods: We performed message-based, sender-based, and thread-based analyses of more than 5 million secure messages between 2010 and 2017. We summarized the message volumes, patient and clinician population sizes, message counts per patient or clinician, as well as the trends of message volumes and user counts over the years. In addition, we calculated the time distribution of clinician-sent messages to understand their workloads at different times of a day. We also analyzed the time delay in clinician responses to patient messages to assess their communication efficiency and the back-and-forth rounds to estimate the communication complexity. Results: During 2010-2017, the patient portal at Mayo Clinic, Rochester experienced a significant growth in terms of the count of patient users and the total number of secure messages sent by patients and clinicians. Three clinician categories, namely “physician—primary care,” “registered nurse—specialty,” and “physician—specialty,” bore the majority of message volume increase. The patient portal also demonstrated growing trends in message counts per patient and clinician. The “nurse practitioner or physician assistant—primary care” and “physician—primary care” categories had the heaviest per-clinician workload each year. Most messages by the clinicians were sent from 7 AM to 5 PM during a day. Yet, between 5 PM and 7 PM, the physicians sent 7.0% (95,785/1,377,006) of their daily messages, and the nurse practitioner or physician assistant sent 5.4% (22,121/408,526) of their daily messages. The clinicians replied to 72.2% (1,272,069/1,761,739) patient messages within 1 day and 90.6% (1,595,702/1,761,739) within 3 days. In 95.1% (1,499,316/1,576,205) of the message threads, the patients communicated with their clinicians back and forth for no more than 4 rounds. Conclusions: Our study found steady increases in patient adoption of the secure messaging system and the average workload per clinician over 8 years. However, most clinicians responded timely to meet the patients’ needs. Our study also revealed differential patient-clinician communication patterns across different practice roles and care settings. These findings suggest opportunities for care teams to optimize messaging tasks and to balance the workload for optimal efficiency. %M 35014964 %R 10.2196/17273 %U https://www.jmir.org/2022/1/e17273 %U https://doi.org/10.2196/17273 %U http://www.ncbi.nlm.nih.gov/pubmed/35014964 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 1 %P e33600 %T Current Status of the Health Information Technology Industry in China from the China Hospital Information Network Conference: Cross-sectional Study of Participating Companies %A Zhang,Zhongan %A Zheng,Xu %A An,Kai %A He,Yunfan %A Wang,Tong %A Zhou,Ruizhu %A Zheng,Qilin %A Nuo,Mingfu %A Liang,Jun %A Lei,Jianbo %+ Center for Medical Informatics, Health Science Center, Peking University, 38 Xueyuan Rd, Haidian District, Beijing, 100191, China, 86 15201230935, jblei@hsc.pku.edu.cn %K medical informatics %K China Hospital Information Network Conference %K industry analysis %K county medical community %K smart hospital %K cross-sectional study %K digital therapeutic %K information network %K health care %K hospital information %K medical information %K tertiary hospital %D 2022 %7 11.1.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: The China Hospital Information Network Conference (CHINC) is one of the most influential academic and technical exchange activities in medical informatics and medical informatization in China. It collects frontier ideas in medical information and has an important reference value for the analysis of China's medical information industry development. Objective: This study summarizes the current situation and future development of China's medical information industry and provides a future reference for China and abroad in the future by analyzing the characteristics of CHINC exhibitors in 2021. Methods: The list of enterprises and participating keywords were obtained from the official website of CHINC. Basic characteristics of the enterprises, industrial fields, applied technologies, company concepts, and other information were collected from the TianYanCha website and the VBDATA company library. Descriptive analysis was used to analyze the collected data, and we summarized the future development directions. Results: A total of 205 enterprises officially participated in the exhibition. Most of the enterprises were newly founded, of which 61.9% (127/205) were founded in the past 10 years. The majority of these enterprises were from first-tier cities, and 79.02% (162/205) were from Beijing, Zhejiang, Guangdong, Shanghai, and Jiangsu Provinces. The median registered capital is 16.67 million RMB (about US $2.61 million), and there are 35 (72.2%) enterprises with a registered capital of more than 100 million RMB (about US $15.68 million), 17 (8.3%) of which are already listed. A total of 126 enterprises were found in the VBDATA company library, of which 39 (30.9%) are information technology vendors and 57 (45.2%) are application technology vendors. In addition, 16 of the 57 (28%) use artificial intelligence technology. Smart medicine and internet hospitals were the focus of the enterprises participating in this conference. Conclusions: China's tertiary hospital informatization has basically completed the construction of the primary stage. The average grade of hospital electronic medical records exceeds grade 3, and 78.13% of the provinces have reached grade 3 or above. The characteristics are as follows: On the one hand, China's medical information industry is focusing on the construction of smart hospitals, including intelligent systems supporting doctors' scientific research, diagnosis-related group intelligent operation systems, and office automation systems supporting hospital management, single-disease clinical decision support systems assisting doctors' clinical care, and intelligent internet of things for logistics. On the other hand, the construction of a compact county medical community is becoming a new focus of enterprises under the guidance of practical needs and national policies to improve the quality of grassroots health services. In addition, whole-course management and digital therapy will also become a new hotspot in the future. %M 35014959 %R 10.2196/33600 %U https://medinform.jmir.org/2022/1/e33600 %U https://doi.org/10.2196/33600 %U http://www.ncbi.nlm.nih.gov/pubmed/35014959 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e28762 %T Comparing International Experiences With Electronic Health Records Among Emergency Medicine Physicians in the United States and Norway: Semistructured Interview Study %A Garcia,Gracie %A Crenner,Christopher %+ Department of History and Philosophy of Medicine, University of Kansas School of Medicine, 3901 Rainbow Blvd, Kansas City, KS, 66160, United States, 1 6209514171, grace.c.garcia1@gmail.com %K electronic health records %K electronic medical records %K health information technology %K health information exchange %K health policy %K international %K emergency medicine %K medical informatics %K meaningful use %K burnout %D 2022 %7 7.1.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The variability in physicians’ attitudes regarding electronic health records (EHRs) is widely recognized. Both human and technological factors contribute to user satisfaction. This exploratory study considers these variables by comparing emergency medicine physician experiences with EHRs in the United States and Norway. Objective: This study is unique as it aims to compare individual experiences with EHRs. It creates an opportunity to expand perspective, challenge the unknown, and explore how this technology affects clinicians globally. Research often highlights the challenge that health information technology has created for users: Are the negative consequences of this technology shared among countries? Does it affect medical practice? What determines user satisfaction? Can this be measured internationally? Do specific factors account for similarities or differences? This study begins by investigating these questions by comparing cohort experiences. Fundamental differences between nations will also be addressed. Methods: We used semistructured, participant-driven, in-depth interviews (N=12) for data collection in conjunction with ethnographic observations. The conversations were recorded and transcribed. Texts were then analyzed using NVivo software (QSR International) to develop codes for direct comparison among countries. Comprehensive understanding of the data required triangulation, specifically using thematic and interpretive phenomenological analysis. Narrative analysis ensured appropriate context of the NVivo (QSR International) query results. Results: Each interview resulted in mixed discussions regarding the benefits and disadvantages of EHRs. All the physicians recognized health care’s dependence on this technology. In Norway, physicians perceived more benefits compared with those based in the United States. Americans reported fewer benefits and disproportionally high disadvantages. Both cohorts believed that EHRs have increased user workload. However, this was mentioned 2.6 times more frequently by Americans (United States [n=40] vs Norway [n=15]). Financial influences regarding health information technology use were of great concern for American physicians but rarely mentioned among Norwegian physicians (United States [n=37] vs Norway [n=6]). Technology dysfunctions were the most common complaint from Norwegian physicians. Participants from each country noted increased frustration among older colleagues. Conclusions: Despite differences spanning geographical, organizational, and cultural boundaries, much is to be learned by comparing individual experiences. Both cohorts experienced EHR-related frustrations, although etiology differed. The overall number of complaints was significantly higher among American physicians. This study augments the idea that policy, regulation, and administration have compelling influence on user experience. Global EHR optimization requires additional investigation, and these results help to establish a foundation for future research. %M 34994702 %R 10.2196/28762 %U https://humanfactors.jmir.org/2022/1/e28762 %U https://doi.org/10.2196/28762 %U http://www.ncbi.nlm.nih.gov/pubmed/34994702 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 1 %P e30720 %T Sequential Data–Based Patient Similarity Framework for Patient Outcome Prediction: Algorithm Development %A Wang,Ni %A Wang,Muyu %A Zhou,Yang %A Liu,Honglei %A Wei,Lan %A Fei,Xiaolu %A Chen,Hui %+ School of Biomedical Engineering, Capital Medical University, No.10, Xitoutiao, You An Men, Fengtai District, Beijing, 100069, China, 86 010 8391 1545, chenhui@ccmu.edu.cn %K patient similarity %K electronic medical records %K time series %K acute myocardial infarction %K natural language processing %K machine learning %K deep learning %K outcome prediction %K informatics %K health data %D 2022 %7 6.1.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Sequential information in electronic medical records is valuable and helpful for patient outcome prediction but is rarely used for patient similarity measurement because of its unevenness, irregularity, and heterogeneity. Objective: We aimed to develop a patient similarity framework for patient outcome prediction that makes use of sequential and cross-sectional information in electronic medical record systems. Methods: Sequence similarity was calculated from timestamped event sequences using edit distance, and trend similarity was calculated from time series using dynamic time warping and Haar decomposition. We also extracted cross-sectional information, namely, demographic, laboratory test, and radiological report data, for additional similarity calculations. We validated the effectiveness of the framework by constructing k–nearest neighbors classifiers to predict mortality and readmission for acute myocardial infarction patients, using data from (1) a public data set and (2) a private data set, at 3 time points—at admission, on Day 7, and at discharge—to provide early warning patient outcomes. We also constructed state-of-the-art Euclidean-distance k–nearest neighbor, logistic regression, random forest, long short-term memory network, and recurrent neural network models, which were used for comparison. Results: With all available information during a hospitalization episode, predictive models using the similarity model outperformed baseline models based on both public and private data sets. For mortality predictions, all models except for the logistic regression model showed improved performances over time. There were no such increasing trends in predictive performances for readmission predictions. The random forest and logistic regression models performed best for mortality and readmission predictions, respectively, when using information from the first week after admission. Conclusions: For patient outcome predictions, the patient similarity framework facilitated sequential similarity calculations for uneven electronic medical record data and helped improve predictive performance. %M 34989682 %R 10.2196/30720 %U https://www.jmir.org/2022/1/e30720 %U https://doi.org/10.2196/30720 %U http://www.ncbi.nlm.nih.gov/pubmed/34989682 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 1 %P e33399 %T Exploring the COVID-19 Pandemic as a Catalyst for Behavior Change Among Patient Health Record App Users in Taiwan: Development and Usability Study %A Tseng,Chinyang Henry %A Chen,Ray-Jade %A Tsai,Shang-Yu %A Wu,Tsung-Ren %A Tsaur,Woei-Jiunn %A Chiu,Hung-Wen %A Yang,Cheng-Yi %A Lo,Yu-Sheng %+ Graduate Institute of Biomedical Informatics, College of Medical Science and Technology, Taipei Medical University, 250 Wuxing Street, Taipei, 110, Taiwan, 886 +886227361661, Loyusen@tmu.edu.tw %K personal health records %K COVID-19 %K My Health Bank %K blockchain %K public health %D 2022 %7 6.1.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: During the COVID-19 pandemic, personal health records (PHRs) have enabled patients to monitor and manage their medical data without visiting hospitals and, consequently, minimize their infection risk. Taiwan’s National Health Insurance Administration (NHIA) launched the My Health Bank (MHB) service, a national PHR system through which insured individuals to access their cross-hospital medical data. Furthermore, in 2019, the NHIA released the MHB software development kit (SDK), which enables development of mobile apps with which insured individuals can retrieve their MHB data. However, the NHIA MHB service has its limitations, and the participation rate among insured individuals is low. Objective: We aimed to integrate the MHB SDK with our developed blockchain-enabled PHR mobile app, which enables patients to access, store, and manage their cross-hospital PHR data. We also collected and analyzed the app’s log data to examine patients’ MHB use during the COVID-19 pandemic. Methods: We integrated our existing blockchain-enabled mobile app with the MHB SDK to enable NHIA MHB data retrieval. The app utilizes blockchain technology to encrypt the downloaded NHIA MHB data. Existing and new indexes can be synchronized between the app and blockchain nodes, and high security can be achieved for PHR management. Finally, we analyzed the app’s access logs to compare patients’ activities during high and low COVID-19 infection periods. Results: We successfully integrated the MHB SDK into our mobile app, thereby enabling patients to retrieve their cross-hospital medical data, particularly those related to COVID-19 rapid and polymerase chain reaction testing and vaccination information and progress. We retrospectively collected the app’s log data for the period of July 2019 to June 2021. From January 2020, the preliminary results revealed a steady increase in the number of people who applied to create a blockchain account for access to their medical data and the number of app subscribers among patients who visited the outpatient department (OPD) and emergency department (ED). Notably, for patients who visited the OPD and ED, the peak proportions with respect to the use of the app for OPD and ED notes and laboratory test results also increased year by year. The highest proportions were 52.40% for ED notes in June 2021, 88.10% for ED laboratory test reports in May 2021, 34.61% for OPD notes in June 2021, and 41.87% for OPD laboratory test reports in June 2021. These peaks coincided with Taiwan’s local COVID-19 outbreak lasting from May to June 2021. Conclusions: This study developed a blockchain-enabled mobile app, which can periodically retrieve and integrate PHRs from the NHIA MHB's cross-hospital data and the investigated hospital's self-pay medical data. Analysis of users’ access logs revealed that the COVID-19 pandemic substantially increased individuals’ use of PHRs and their health awareness with respect to COVID-19 prevention. %M 34951863 %R 10.2196/33399 %U https://www.jmir.org/2022/1/e33399 %U https://doi.org/10.2196/33399 %U http://www.ncbi.nlm.nih.gov/pubmed/34951863 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 1 %P e28953 %T Developing a Machine Learning Model to Predict Severe Chronic Obstructive Pulmonary Disease Exacerbations: Retrospective Cohort Study %A Zeng,Siyang %A Arjomandi,Mehrdad %A Tong,Yao %A Liao,Zachary C %A Luo,Gang %+ Department of Biomedical Informatics and Medical Education, University of Washington, UW Medicine South Lake Union, 850 Republican Street, Building C, Box 358047, Seattle, WA, 98195, United States, 1 206 221 4596, gangluo@cs.wisc.edu %K chronic obstructive pulmonary disease %K machine learning %K forecasting %K symptom exacerbation %K patient care management %D 2022 %7 6.1.2022 %9 Original Paper %J J Med Internet Res %G English %X Background: Chronic obstructive pulmonary disease (COPD) poses a large burden on health care. Severe COPD exacerbations require emergency department visits or inpatient stays, often cause an irreversible decline in lung function and health status, and account for 90.3% of the total medical cost related to COPD. Many severe COPD exacerbations are deemed preventable with appropriate outpatient care. Current models for predicting severe COPD exacerbations lack accuracy, making it difficult to effectively target patients at high risk for preventive care management to reduce severe COPD exacerbations and improve outcomes. Objective: The aim of this study is to develop a more accurate model to predict severe COPD exacerbations. Methods: We examined all patients with COPD who visited the University of Washington Medicine facilities between 2011 and 2019 and identified 278 candidate features. By performing secondary analysis on 43,576 University of Washington Medicine data instances from 2011 to 2019, we created a machine learning model to predict severe COPD exacerbations in the next year for patients with COPD. Results: The final model had an area under the receiver operating characteristic curve of 0.866. When using the top 9.99% (752/7529) of the patients with the largest predicted risk to set the cutoff threshold for binary classification, the model gained an accuracy of 90.33% (6801/7529), a sensitivity of 56.6% (103/182), and a specificity of 91.17% (6698/7347). Conclusions: Our model provided a more accurate prediction of severe COPD exacerbations in the next year compared with prior published models. After further improvement of its performance measures (eg, by adding features extracted from clinical notes), our model could be used in a decision support tool to guide the identification of patients with COPD and at high risk for care management to improve outcomes. International Registered Report Identifier (IRRID): RR2-10.2196/13783 %M 34989686 %R 10.2196/28953 %U https://www.jmir.org/2022/1/e28953 %U https://doi.org/10.2196/28953 %U http://www.ncbi.nlm.nih.gov/pubmed/34989686 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 10 %N 1 %P e28981 %T Patient Perspectives on the Digitization of Personal Health Information in the Emergency Department: Mixed Methods Study During the COVID-19 Pandemic %A Ly,Sophia %A Tsang,Ricky %A Ho,Kendall %+ Faculty of Medicine, University of British Columbia, 3312-818 West 10th Avenue, Vancouver, BC, V5Z 1M9, Canada, 1 6048220327, sophia.ly@alumni.ubc.ca %K emergency medicine %K digital health %K health informatics %K electronic health record %K patient portal %K patient-physician relationship %K COVID-19 %D 2022 %7 6.1.2022 %9 Original Paper %J JMIR Med Inform %G English %X Background: Although the digitization of personal health information (PHI) has been shown to improve patient engagement in the primary care setting, patient perspectives on its impact in the emergency department (ED) are unknown. Objective: The primary objective was to characterize the views of ED users in British Columbia, Canada, on the impacts of PHI digitization on ED care. Methods: This was a mixed methods study consisting of an online survey followed by key informant interviews with a subset of survey respondents. ED users in British Columbia were asked about their ED experiences and attitudes toward PHI digitization in the ED. Results: A total of 108 participants submitted survey responses between January and April 2020. Most survey respondents were interested in the use of electronic health records (79/105, 75%) and patient portals (91/107, 85%) in the ED and were amenable to sharing their ED PHI with ED staff (up to 90% in emergencies), family physicians (up to 91%), and family caregivers (up to 75%). In addition, 16 survey respondents provided key informant interviews in August 2020. Interviewees expected PHI digitization in the ED to enhance PHI access by health providers, patient-provider relationships, patient self-advocacy, and postdischarge care management, although some voiced concerns about patient privacy risk and limited access to digital technologies (eg, smart devices, internet connection). Many participants thought the COVID-19 pandemic could provide momentum for the digitization of health care. Conclusions: Patients overwhelmingly support PHI digitization in the form of electronic health records and patient portals in the ED. The COVID-19 pandemic may represent a critical moment for the development and implementation of these tools. %M 34818211 %R 10.2196/28981 %U https://medinform.jmir.org/2022/1/e28981 %U https://doi.org/10.2196/28981 %U http://www.ncbi.nlm.nih.gov/pubmed/34818211 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e31246 %T Key Challenges and Opportunities for Cloud Technology in Health Care: Semistructured Interview Study %A Cresswell,Kathrin %A Domínguez Hernández,Andrés %A Williams,Robin %A Sheikh,Aziz %+ Usher Institute, The University of Edinburgh, Teviot Place, Edinburgh, EH8 9AG, United Kingdom, 44 131 651 4151, Kathrin.Cresswell@ed.ac.uk %K cloud technology %K qualitative %K adoption %K implementation %K digital health %K data processing %K health care %K risk assessment %K user engagement %D 2022 %7 6.1.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The use of cloud computing (involving storage and processing of data on the internet) in health care has increasingly been highlighted as having great potential in facilitating data-driven innovations. Although some provider organizations are reaping the benefits of using cloud providers to store and process their data, others are lagging behind. Objective: We aim to explore the existing challenges and barriers to the use of cloud computing in health care settings and investigate how perceived risks can be addressed. Methods: We conducted a qualitative case study of cloud computing in health care settings, interviewing a range of individuals with perspectives on supply, implementation, adoption, and integration of cloud technology. Data were collected through a series of in-depth semistructured interviews exploring current applications, implementation approaches, challenges encountered, and visions for the future. The interviews were transcribed and thematically analyzed using NVivo 12 (QSR International). We coded the data based on a sociotechnical coding framework developed in related work. Results: We interviewed 23 individuals between September 2020 and November 2020, including professionals working across major cloud providers, health care provider organizations, innovators, small and medium-sized software vendors, and academic institutions. The participants were united by a common vision of a cloud-enabled ecosystem of applications and by drivers surrounding data-driven innovation. The identified barriers to progress included the cost of data migration and skill gaps to implement cloud technologies within provider organizations, the cultural shift required to move to externally hosted services, a lack of user pull as many benefits were not visible to those providing frontline care, and a lack of interoperability standards and central regulations. Conclusions: Implementations need to be viewed as a digitally enabled transformation of services, driven by skill development, organizational change management, and user engagement, to facilitate the implementation and exploitation of cloud-based infrastructures and to maximize returns on investment. %M 34989688 %R 10.2196/31246 %U https://humanfactors.jmir.org/2022/1/e31246 %U https://doi.org/10.2196/31246 %U http://www.ncbi.nlm.nih.gov/pubmed/34989688 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 9 %N 1 %P e28783 %T Clinician Perspectives on Unmet Needs for Mobile Technology Among Hospitalists: Workflow Analysis Based on Semistructured Interviews %A Savoy,April %A Saleem,Jason J %A Barker,Barry C %A Patel,Himalaya %A Kara,Areeba %+ Center for Health Information and Communication, Health Services Research and Development Service, Richard L Roudebush Veterans Affairs Medical Center, 1481 West 10th Street, Stop 11H, Indianapolis, IN, 46202, United States, 1 3172782194, asavoy@iu.edu %K electronic health records %K hospital medicine %K user-computer interface %K human-computer interaction %K usability %K mental workload %K workflow analysis %D 2022 %7 4.1.2022 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The hospitalist workday is cognitively demanding and dominated by activities away from patients’ bedsides. Although mobile technologies are offered as solutions, clinicians report lower expectations of mobile technology after actual use. Objective: The purpose of this study is to better understand opportunities for integrating mobile technology and apps into hospitalists’ workflows. We aim to identify difficult tasks and contextual factors that introduce inefficiencies and characterize hospitalists’ perspectives on mobile technology and apps. Methods: We conducted a workflow analysis based on semistructured interviews. At a Midwestern US medical center, we recruited physicians and nurse practitioners from hospitalist and inpatient teaching teams and internal medicine residents. Interviews focused on tasks perceived as frequent, redundant, and difficult. Additionally, participants were asked to describe opportunities for mobile technology interventions. We analyzed contributing factors, impacted workflows, and mobile app ideas. Results: Over 3 months, we interviewed 12 hospitalists. Participants collectively identified chart reviews, orders, and documentation as the most frequent, redundant, and difficult tasks. Based on those tasks, the intake, discharge, and rounding workflows were characterized as difficult and inefficient. The difficulty was associated with a lack of access to electronic health records at the bedside. Contributing factors for inefficiencies were poor usability and inconsistent availability of health information technology combined with organizational policies. Participants thought mobile apps designed to improve team communications would be most beneficial. Based on our analysis, mobile apps focused on data entry and presentation supporting specific tasks should also be prioritized. Conclusions: Based on our results, there are prioritized opportunities for mobile technology to decrease difficulty and increase the efficiency of hospitalists’ workflows. Mobile technology and task-specific mobile apps with enhanced usability could decrease overreliance on hospitalists’ memory and fragmentation of clinical tasks across locations. This study informs the design and implementation processes of future health information technologies to improve continuity in hospital-based medicine. %M 34643530 %R 10.2196/28783 %U https://humanfactors.jmir.org/2022/1/e28783 %U https://doi.org/10.2196/28783 %U http://www.ncbi.nlm.nih.gov/pubmed/34643530 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 12 %P e27386 %T Benchmarking Effectiveness and Efficiency of Deep Learning Models for Semantic Textual Similarity in the Clinical Domain: Validation Study %A Chen,Qingyu %A Rankine,Alex %A Peng,Yifan %A Aghaarabi,Elaheh %A Lu,Zhiyong %+ National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, 8600 Rockville Pike, Bethesda, MD, 20894, United States, 1 301 594 7089, luzh@ncbi.nlm.nih.gov %K semantic textual similarity %K deep learning %K biomedical and clinical text mining %K word embeddings %K sentence embeddings %K transformers %D 2021 %7 30.12.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Semantic textual similarity (STS) measures the degree of relatedness between sentence pairs. The Open Health Natural Language Processing (OHNLP) Consortium released an expertly annotated STS data set and called for the National Natural Language Processing Clinical Challenges. This work describes our entry, an ensemble model that leverages a range of deep learning (DL) models. Our team from the National Library of Medicine obtained a Pearson correlation of 0.8967 in an official test set during 2019 National Natural Language Processing Clinical Challenges/Open Health Natural Language Processing shared task and achieved a second rank. Objective: Although our models strongly correlate with manual annotations, annotator-level correlation was only moderate (weighted Cohen κ=0.60). We are cautious of the potential use of DL models in production systems and argue that it is more critical to evaluate the models in-depth, especially those with extremely high correlations. In this study, we benchmark the effectiveness and efficiency of top-ranked DL models. We quantify their robustness and inference times to validate their usefulness in real-time applications. Methods: We benchmarked five DL models, which are the top-ranked systems for STS tasks: Convolutional Neural Network, BioSentVec, BioBERT, BlueBERT, and ClinicalBERT. We evaluated a random forest model as an additional baseline. For each model, we repeated the experiment 10 times, using the official training and testing sets. We reported 95% CI of the Wilcoxon rank-sum test on the average Pearson correlation (official evaluation metric) and running time. We further evaluated Spearman correlation, R², and mean squared error as additional measures. Results: Using only the official training set, all models obtained highly effective results. BioSentVec and BioBERT achieved the highest average Pearson correlations (0.8497 and 0.8481, respectively). BioSentVec also had the highest results in 3 of 4 effectiveness measures, followed by BioBERT. However, their robustness to sentence pairs of different similarity levels varies significantly. A particular observation is that BERT models made the most errors (a mean squared error of over 2.5) on highly similar sentence pairs. They cannot capture highly similar sentence pairs effectively when they have different negation terms or word orders. In addition, time efficiency is dramatically different from the effectiveness results. On average, the BERT models were approximately 20 times and 50 times slower than the Convolutional Neural Network and BioSentVec models, respectively. This results in challenges for real-time applications. Conclusions: Despite the excitement of further improving Pearson correlations in this data set, our results highlight that evaluations of the effectiveness and efficiency of STS models are critical. In future, we suggest more evaluations on the generalization capability and user-level testing of the models. We call for community efforts to create more biomedical and clinical STS data sets from different perspectives to reflect the multifaceted notion of sentence-relatedness. %M 34967748 %R 10.2196/27386 %U https://medinform.jmir.org/2021/12/e27386 %U https://doi.org/10.2196/27386 %U http://www.ncbi.nlm.nih.gov/pubmed/34967748 %0 Journal Article %@ 2562-7600 %I JMIR Publications %V 4 %N 4 %P e31038 %T Exploring Reasons for Delayed Start-of-Care Nursing Visits in Home Health Care: Algorithm Development and Data Science Study %A Zolnoori,Maryam %A Song,Jiyoun %A McDonald,Margaret V %A Barrón,Yolanda %A Cato,Kenrick %A Sockolow,Paulina %A Sridharan,Sridevi %A Onorato,Nicole %A Bowles,Kathryn H %A Topaz,Maxim %+ School of Nursing, Columbia University, 560 W 168th St, New York, NY, 10032, United States, 1 (212) 609 1774, mt3315@cumc.columbia.edu %K delayed start-of-care nursing visit %K home healthcare services %K natural language processing %K nursing note %K NLP %K nursing %K eHealth %K home care %K clinical notes %K classification %K clinical informatics %D 2021 %7 30.12.2021 %9 Original Paper %J JMIR Nursing %G English %X Background: Delayed start-of-care nursing visits in home health care (HHC) can result in negative outcomes, such as hospitalization. No previous studies have investigated why start-of-care HHC nursing visits are delayed, in part because most reasons for delayed visits are documented in free-text HHC nursing notes. Objective: The aims of this study were to (1) develop and test a natural language processing (NLP) algorithm that automatically identifies reasons for delayed visits in HHC free-text clinical notes and (2) describe reasons for delayed visits in a large patient sample. Methods: This study was conducted at the Visiting Nurse Service of New York (VNSNY). We examined data available at the VNSNY on all new episodes of care started in 2019 (N=48,497). An NLP algorithm was developed and tested to automatically identify and classify reasons for delayed visits. Results: The performance of the NLP algorithm was 0.8, 0.75, and 0.77 for precision, recall, and F-score, respectively. A total of one-third of HHC episodes (n=16,244) had delayed start-of-care HHC nursing visits. The most prevalent identified category of reasons for delayed start-of-care nursing visits was no answer at the door or phone (3728/8051, 46.3%), followed by patient/family request to postpone or refuse some HHC services (n=2858, 35.5%), and administrative or scheduling issues (n=1465, 18.2%). In 40% (n=16,244) of HHC episodes, 2 or more reasons were documented. Conclusions: To avoid critical delays in start-of-care nursing visits, HHC organizations might examine and improve ways to effectively address the reasons for delayed visits, using effective interventions, such as educating patients or caregivers on the importance of a timely nursing visit and improving patients’ intake procedures. %M 34967749 %R 10.2196/31038 %U https://nursing.jmir.org/2021/4/e31038 %U https://doi.org/10.2196/31038 %U http://www.ncbi.nlm.nih.gov/pubmed/34967749 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 12 %P e27008 %T A Novel Deep Learning–Based System for Triage in the Emergency Department Using Electronic Medical Records: Retrospective Cohort Study %A Yao,Li-Hung %A Leung,Ka-Chun %A Tsai,Chu-Lin %A Huang,Chien-Hua %A Fu,Li-Chen %+ Department of Computer Science and Information Engineering, National Taiwan University, CSIE Der Tian Hall, No. 1, Sec. 4, Roosevelt Road, Taipei, 10617, Taiwan, 886 0935545846, lichen@ntu.edu.tw %K emergency department %K triage system %K deep learning %K hospital admission %K data to text %K electronic health record %D 2021 %7 27.12.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Emergency department (ED) crowding has resulted in delayed patient treatment and has become a universal health care problem. Although a triage system, such as the 5-level emergency severity index, somewhat improves the process of ED treatment, it still heavily relies on the nurse’s subjective judgment and triages too many patients to emergency severity index level 3 in current practice. Hence, a system that can help clinicians accurately triage a patient’s condition is imperative. Objective: This study aims to develop a deep learning–based triage system using patients’ ED electronic medical records to predict clinical outcomes after ED treatments. Methods: We conducted a retrospective study using data from an open data set from the National Hospital Ambulatory Medical Care Survey from 2012 to 2016 and data from a local data set from the National Taiwan University Hospital from 2009 to 2015. In this study, we transformed structured data into text form and used convolutional neural networks combined with recurrent neural networks and attention mechanisms to accomplish the classification task. We evaluated our performance using area under the receiver operating characteristic curve (AUROC). Results: A total of 118,602 patients from the National Hospital Ambulatory Medical Care Survey were included in this study for predicting hospitalization, and the accuracy and AUROC were 0.83 and 0.87, respectively. On the other hand, an external experiment was to use our own data set from the National Taiwan University Hospital that included 745,441 patients, where the accuracy and AUROC were similar, that is, 0.83 and 0.88, respectively. Moreover, to effectively evaluate the prediction quality of our proposed system, we also applied the model to other clinical outcomes, including mortality and admission to the intensive care unit, and the results showed that our proposed method was approximately 3% to 5% higher in accuracy than other conventional methods. Conclusions: Our proposed method achieved better performance than the traditional method, and its implementation is relatively easy, it includes commonly used variables, and it is better suited for real-world clinical settings. It is our future work to validate our novel deep learning–based triage algorithm with prospective clinical trials, and we hope to use it to guide resource allocation in a busy ED once the validation succeeds. %M 34958305 %R 10.2196/27008 %U https://www.jmir.org/2021/12/e27008 %U https://doi.org/10.2196/27008 %U http://www.ncbi.nlm.nih.gov/pubmed/34958305 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 12 %P e30805 %T Health Care Analytics With Time-Invariant and Time-Variant Feature Importance to Predict Hospital-Acquired Acute Kidney Injury: Observational Longitudinal Study %A Chua,Horng-Ruey %A Zheng,Kaiping %A Vathsala,Anantharaman %A Ngiam,Kee-Yuan %A Yap,Hui-Kim %A Lu,Liangjian %A Tiong,Ho-Yee %A Mukhopadhyay,Amartya %A MacLaren,Graeme %A Lim,Shir-Lynn %A Akalya,K %A Ooi,Beng-Chin %+ Division of Nephrology, Department of Medicine, National University Hospital, Level 10 Medicine, NUHS Tower Block, 1E Kent Ridge Road, Singapore, 119228, Singapore, 65 67726178, horng_ruey_chua@nuhs.edu.sg %K acute kidney injury %K artificial intelligence %K biomarkers %K clinical deterioration %K electronic health records %K hospital medicine %K machine learning %D 2021 %7 24.12.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Acute kidney injury (AKI) develops in 4% of hospitalized patients and is a marker of clinical deterioration and nephrotoxicity. AKI onset is highly variable in hospitals, which makes it difficult to time biomarker assessment in all patients for preemptive care. Objective: The study sought to apply machine learning techniques to electronic health records and predict hospital-acquired AKI by a 48-hour lead time, with the aim to create an AKI surveillance algorithm that is deployable in real time. Methods: The data were sourced from 20,732 case admissions in 16,288 patients over 1 year in our institution. We enhanced the bidirectional recurrent neural network model with a novel time-invariant and time-variant aggregated module to capture important clinical features temporal to AKI in every patient. Time-series features included laboratory parameters that preceded a 48-hour prediction window before AKI onset; the latter’s corresponding reference was the final in-hospital serum creatinine performed in case admissions without AKI episodes. Results: The cohort was of mean age 53 (SD 25) years, of whom 29%, 12%, 12%, and 53% had diabetes, ischemic heart disease, cancers, and baseline eGFR <90 mL/min/1.73 m2, respectively. There were 911 AKI episodes in 869 patients. We derived and validated an algorithm in the testing dataset with an AUROC of 0.81 (0.78-0.85) for predicting AKI. At a 15% prediction threshold, our model generated 699 AKI alerts with 2 false positives for every true AKI and predicted 26% of AKIs. A lowered 5% prediction threshold improved the recall to 60% but generated 3746 AKI alerts with 6 false positives for every true AKI. Representative interpretation results produced by our model alluded to the top-ranked features that predicted AKI that could be categorized in association with sepsis, acute coronary syndrome, nephrotoxicity, or multiorgan injury, specific to every case at risk. Conclusions: We generated an accurate algorithm from electronic health records through machine learning that predicted AKI by a lead time of at least 48 hours. The prediction threshold could be adjusted during deployment to optimize recall and minimize alert fatigue, while its precision could potentially be augmented by targeted AKI biomarker assessment in the high-risk cohort identified. %M 34951595 %R 10.2196/30805 %U https://www.jmir.org/2021/12/e30805 %U https://doi.org/10.2196/30805 %U http://www.ncbi.nlm.nih.gov/pubmed/34951595 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 12 %P e28632 %T Text Mining of Adverse Events in Clinical Trials: Deep Learning Approach %A Chopard,Daphne %A Treder,Matthias S %A Corcoran,Padraig %A Ahmed,Nagheen %A Johnson,Claire %A Busse,Monica %A Spasic,Irena %+ School of Computer Science & Informatics, Cardiff University, Abacws, Senghennydd Road, Cardiff, CF24 4AG, United Kingdom, 44 2920870032, spasici@cardiff.ac.uk %K natural language processing %K deep learning %K machine learning %K classification %D 2021 %7 24.12.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Pharmacovigilance and safety reporting, which involve processes for monitoring the use of medicines in clinical trials, play a critical role in the identification of previously unrecognized adverse events or changes in the patterns of adverse events. Objective: This study aims to demonstrate the feasibility of automating the coding of adverse events described in the narrative section of the serious adverse event report forms to enable statistical analysis of the aforementioned patterns. Methods: We used the Unified Medical Language System (UMLS) as the coding scheme, which integrates 217 source vocabularies, thus enabling coding against other relevant terminologies such as the International Classification of Diseases–10th Revision, Medical Dictionary for Regulatory Activities, and Systematized Nomenclature of Medicine). We used MetaMap, a highly configurable dictionary lookup software, to identify the mentions of the UMLS concepts. We trained a binary classifier using Bidirectional Encoder Representations from Transformers (BERT), a transformer-based language model that captures contextual relationships, to differentiate between mentions of the UMLS concepts that represented adverse events and those that did not. Results: The model achieved a high F1 score of 0.8080, despite the class imbalance. This is 10.15 percent points lower than human-like performance but also 17.45 percent points higher than that of the baseline approach. Conclusions: These results confirmed that automated coding of adverse events described in the narrative section of serious adverse event reports is feasible. Once coded, adverse events can be statistically analyzed so that any correlations with the trialed medicines can be estimated in a timely fashion. %M 34951601 %R 10.2196/28632 %U https://medinform.jmir.org/2021/12/e28632 %U https://doi.org/10.2196/28632 %U http://www.ncbi.nlm.nih.gov/pubmed/34951601 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 12 %P e25328 %T Can Real-time Computer-Aided Detection Systems Diminish the Risk of Postcolonoscopy Colorectal Cancer? %A Madalinski,Mariusz %A Prudham,Roger %+ Northern Care Alliance, Royal Oldham Hospital, Rochdale Rd, Oldham, OL1 2JH, United Kingdom, 44 01616240420, mariusz.madalinski@googlemail.com %K artificial intelligence %K colonoscopy %K adenoma %K real-time computer-aided detection %K colonic polyp %D 2021 %7 24.12.2021 %9 Viewpoint %J JMIR Med Inform %G English %X The adenoma detection rate is the constant subject of research and the main marker of quality in bowel cancer screening. However, by improving the quality of endoscopy via artificial intelligence methods, all polyps, including those with the potential for malignancy, can be removed, thereby reducing interval colorectal cancer rates. As such, the removal of all polyps may become the best marker of endoscopy quality. Thus, we present a viewpoint on integrating the computer-aided detection (CADe) of polyps with high-accuracy, real-time colonoscopy to challenge quality improvements in the performance of colonoscopy. Colonoscopy for bowel cancer screening involving the integration of a deep learning methodology (ie, integrating artificial intelligence with CADe systems) has been assessed in an effort to increase the adenoma detection rate. In this viewpoint, a few studies are described, and their results show that CADe systems are able to increase screening sensitivity. The detection of adenomatous polyps, which are associated with a potential risk of progression to colorectal cancer, and their removal are expected to reduce cancer incidence and mortality rates. However, so far, artificial intelligence methods do not increase the detection of cancer or large adenomatous polyps but contribute to the detection of small precancerous polyps. %M 34571490 %R 10.2196/25328 %U https://medinform.jmir.org/2021/12/e25328 %U https://doi.org/10.2196/25328 %U http://www.ncbi.nlm.nih.gov/pubmed/34571490 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 12 %P e27096 %T Electronic Health Record Implementations and Insufficient Training Endanger Nurses’ Well-being: Cross-sectional Survey Study %A Heponiemi,Tarja %A Gluschkoff,Kia %A Vehko,Tuulikki %A Kaihlanen,Anu-Marja %A Saranto,Kaija %A Nissinen,Sari %A Nadav,Janna %A Kujala,Sari %+ Finnish Institute for Health and Welfare, Mannerheimintie 166, Helsinki, 00271, Finland, 358 295247434, tarja.heponiemi@thl.fi %K electronic health records %K implementation %K information systems %K training %K stress %K cognitive failures %K time pressure %K registered nurses %D 2021 %7 23.12.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: High expectations have been set for the implementations of health information systems (HIS) in health care. However, nurses have been dissatisfied after implementations of HIS. In particular, poorly functioning electronic health records (EHRs) have been found to induce stress and cognitive workload. Moreover, the need to learn new systems may require considerable effort from nurses. Thus, EHR implementations may have an effect on the well-being of nurses. Objective: This study aimed to examine the associations of EHR-to-EHR implementations and the sufficiency of related training with perceived stress related to information systems (SRIS), time pressure, and cognitive failures among registered nurses. Moreover, we examined the moderating effect of the employment sector (hospital, primary care, social services, and others) on these associations. Methods: This study was a cross-sectional survey study of 3610 registered Finnish nurses in 2020. EHR implementation was measured by assessing whether the work unit of each respondent had implemented or will implement a new EHR (1) within the last 6 months, (2) within the last 12 months, (3) in the next 12 months, and (4) at no point within the last 12 months or in the forthcoming 12 months. The associations were examined using analyses of covariance adjusted for age, gender, and employment sector. Results: The highest levels of SRIS (adjusted mean 4.07, SE 0.05) and time pressure (adjusted mean 4.55, SE 0.06) were observed among those who had experienced an EHR implementation within the last 6 months. The lowest levels of SRIS (adjusted mean 3.26, SE 0.04), time pressure (adjusted mean 4.41, SE 0.05), and cognitive failures (adjusted mean 1.84, SE 0.02) were observed among those who did not experience any completed or forthcoming implementations within 12 months. Nurses who perceived that they had received sufficient implementation-related training experienced less SRIS (F1=153.40, P<.001), time pressure (F1=80.95, P<.001), and cognitive failures (F1=34.96, P<.001) than those who had received insufficient training. Recent implementations and insufficient training were especially strongly associated with high levels of SRIS in hospitals. Conclusions: EHR implementations and insufficient training related to these implementations may endanger the well-being of nurses and even lead to errors. Thus, it is extremely important for organizations to offer comprehensive training before, during, and after implementations. Moreover, easy-to-use systems that allow transition periods, a re-engineering approach, and user involvement may be beneficial to nurses in the implementation process. Training and other improvements would be especially important in hospitals. %M 34941546 %R 10.2196/27096 %U https://www.jmir.org/2021/12/e27096 %U https://doi.org/10.2196/27096 %U http://www.ncbi.nlm.nih.gov/pubmed/34941546 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 12 %P e26323 %T Value of the Electronic Medical Record for Hospital Care: Update From the Literature %A Uslu,Aykut %A Stausberg,Jürgen %+ Institute for Medical Informatics, Biometry and Epidemiology, University Hospital Essen, University Duisburg-Essen, Hufelandstrasse 55, Essen, 45122, Germany, 49 201 72377201, stausberg@ekmed.de %K cost analysis %K costs and cost analyses %K economic advantage %K electronic medical records %K electronic records %K health care %K hospitals %K medical records systems computerized %K quality of health care %K secondary data %D 2021 %7 23.12.2021 %9 Review %J J Med Internet Res %G English %X Background: Electronic records could improve quality and efficiency of health care. National and international bodies propagate this belief worldwide. However, the evidence base concerning the effects and advantages of electronic records is questionable. The outcome of health care systems is influenced by many components, making assertions about specific types of interventions difficult. Moreover, electronic records itself constitute a complex intervention offering several functions with possibly positive as well as negative effects on the outcome of health care systems. Objective: The aim of this review is to summarize empirical studies about the value of electronic medical records (EMRs) for hospital care published between 2010 and spring 2019. Methods: The authors adopted their method from a series of literature reviews. The literature search was performed on MEDLINE with “Medical Record System, Computerized” as the essential keyword. The selection process comprised 2 phases looking for a consent of both authors. Starting with 1345 references, 23 were finally included in the review. The evaluation combined a scoring of the studies’ quality, a description of data sources in case of secondary data analyses, and a qualitative assessment of the publications’ conclusions concerning the medical record’s impact on quality and efficiency of health care. Results: The majority of the studies stemmed from the United States (19/23, 83%). Mostly, the studies used publicly available data (“secondary data studies”; 17/23, 74%). A total of 18 studies analyzed the effect of an EMR on the quality of health care (78%), 16 the effect on the efficiency of health care (70%). The primary data studies achieved a mean score of 4.3 (SD 1.37; theoretical maximum 10); the secondary data studies a mean score of 7.1 (SD 1.26; theoretical maximum 9). From the primary data studies, 2 demonstrated a reduction of costs. There was not one study that failed to demonstrate a positive effect on the quality of health care. Overall, 9/16 respective studies showed a reduction of costs (56%); 14/18 studies showed an increase of health care quality (78%); the remaining 4 studies missed explicit information about the proposed positive effect. Conclusions: This review revealed a clear evidence about the value of EMRs. In addition to an awesome majority of economic advantages, the review also showed improvements in quality of care by all respective studies. The use of secondary data studies has prevailed over primary data studies in the meantime. Future work could focus on specific aspects of electronic records to guide their implementation and operation. %M 34941544 %R 10.2196/26323 %U https://www.jmir.org/2021/12/e26323 %U https://doi.org/10.2196/26323 %U http://www.ncbi.nlm.nih.gov/pubmed/34941544 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 9 %N 12 %P e31618 %T Identifying Data Quality Dimensions for Person-Generated Wearable Device Data: Multi-Method Study %A Cho,Sylvia %A Weng,Chunhua %A Kahn,Michael G %A Natarajan,Karthik %+ Department of Biomedical Informatics, Columbia University, 622 West 168th Street PH20, New York, NY, 10032, United States, 1 212 305 5334, sc3901@cumc.columbia.edu %K patient-generated health data %K data accuracy %K data quality %K wearable device %K fitness trackers %K qualitative research %D 2021 %7 23.12.2021 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: There is a growing interest in using person-generated wearable device data for biomedical research, but there are also concerns regarding the quality of data such as missing or incorrect data. This emphasizes the importance of assessing data quality before conducting research. In order to perform data quality assessments, it is essential to define what data quality means for person-generated wearable device data by identifying the data quality dimensions. Objective: This study aims to identify data quality dimensions for person-generated wearable device data for research purposes. Methods: This study was conducted in 3 phases: literature review, survey, and focus group discussion. The literature review was conducted following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guideline to identify factors affecting data quality and its associated data quality challenges. In addition, we conducted a survey to confirm and complement results from the literature review and to understand researchers’ perceptions on data quality dimensions that were previously identified as dimensions for the secondary use of electronic health record (EHR) data. We sent the survey to researchers with experience in analyzing wearable device data. Focus group discussion sessions were conducted with domain experts to derive data quality dimensions for person-generated wearable device data. On the basis of the results from the literature review and survey, a facilitator proposed potential data quality dimensions relevant to person-generated wearable device data, and the domain experts accepted or rejected the suggested dimensions. Results: In total, 19 studies were included in the literature review, and 3 major themes emerged: device- and technical-related, user-related, and data governance–related factors. The associated data quality problems were incomplete data, incorrect data, and heterogeneous data. A total of 20 respondents answered the survey. The major data quality challenges faced by researchers were completeness, accuracy, and plausibility. The importance ratings on data quality dimensions in an existing framework showed that the dimensions for secondary use of EHR data are applicable to person-generated wearable device data. There were 3 focus group sessions with domain experts in data quality and wearable device research. The experts concluded that intrinsic data quality features, such as conformance, completeness, and plausibility, and contextual and fitness-for-use data quality features, such as completeness (breadth and density) and temporal data granularity, are important data quality dimensions for assessing person-generated wearable device data for research purposes. Conclusions: In this study, intrinsic and contextual and fitness-for-use data quality dimensions for person-generated wearable device data were identified. The dimensions were adapted from data quality terminologies and frameworks for the secondary use of EHR data with a few modifications. Further research on how data quality can be assessed with respect to each dimension is needed. %M 34941540 %R 10.2196/31618 %U https://mhealth.jmir.org/2021/12/e31618 %U https://doi.org/10.2196/31618 %U http://www.ncbi.nlm.nih.gov/pubmed/34941540 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 12 %P e28958 %T Implementation of Fingerprint Technology for Unique Patient Matching and Identification at an HIV Care and Treatment Facility in Western Kenya: Cross-sectional Study %A Jaafa,Noah Kasiiti %A Mokaya,Benard %A Savai,Simon Muhindi %A Yeung,Ada %A Siika,Abraham Mosigisi %A Were,Martin %+ Institute of Biomedical Informatics, Moi University, Nandi Road, Eldoret, PO Box 3900-30100, Kenya, 256 753764102, kasiitinoah@gmail.com %K biometrics %K patient matching %K fingerprints %K unique patient identification %K electronic medical record systems %K low- and middle-income countries (LMICs) %D 2021 %7 22.12.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Unique patient identification remains a challenge in many health care settings in low- and middle-income countries (LMICs). Without national-level unique identifiers for whole populations, countries rely on demographic-based approaches that have proven suboptimal. Affordable biometrics-based approaches, implemented with consideration of contextual ethical, legal, and social implications, have the potential to address this challenge and improve patient safety and reporting accuracy. However, limited studies exist to evaluate the actual performance of biometric approaches and perceptions of these systems in LMICs. Objective: The aim of this study is to evaluate the performance and acceptability of fingerprint technology for unique patient matching and identification in the LMIC setting of Kenya. Methods: In this cross-sectional study conducted at an HIV care and treatment facility in Western Kenya, an open source fingerprint application was integrated within an implementation of the Open Medical Record System, an open source electronic medical record system (EMRS) that is nationally endorsed and deployed for HIV care in Kenya and in more than 40 other countries; hence, it has potential to translate the findings across multiple countries. Participants aged >18 years were conveniently sampled and enrolled into the study. Participants’ left thumbprints were captured and later used to retrieve and match records. The technology’s performance was evaluated using standard measures: sensitivity, false acceptance rate, false rejection rate, and failure to enroll rate. The Wald test was used to compare the accuracy of the technology with the probabilistic patient-matching technique of the EMRS. Time to retrieval and matching of records were compared using the independent samples 2-tailed t test. A survey was administered to evaluate patient acceptance and satisfaction with use of the technology. Results: In all, 300 participants were enrolled; their mean age was 36.3 (SD 12.2) years, and 58% (174/300) were women. The relevant values for the technology’s performance were sensitivity 89.3%, false acceptance rate 0%, false rejection rate 11%, and failure to enroll rate 2.3%. The technology’s mean record retrieval speed was 3.2 (SD 1.1) seconds versus 9.5 (SD 1.9) seconds with demographic-based record retrieval in the EMRS (P<.001). The survey results revealed that 96.3% (289/300) of the participants were comfortable with the technology and 90.3% (271/300) were willing to use it. Participants who had previously used fingerprint biometric systems for identification were estimated to have more than thrice increased odds of accepting the technology (odds ratio 3.57, 95% CI 1.0-11.92). Conclusions: Fingerprint technology performed very well in identifying adult patients in an LMIC setting. Patients reported a high level of satisfaction and acceptance. Serious considerations need to be given to the use of fingerprint technology for patient identification in LMICs, but this has to be done with strong consideration of ethical, legal, and social implications as well as security issues. %M 34941557 %R 10.2196/28958 %U https://www.jmir.org/2021/12/e28958 %U https://doi.org/10.2196/28958 %U http://www.ncbi.nlm.nih.gov/pubmed/34941557 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 12 %P e28610 %T Using the Theoretical Domains Framework to Identify Barriers and Enablers to Implementing a Virtual Tertiary–Regional Telemedicine Rounding and Consultation for Kids (TRaC-K) Model: Qualitative Study %A Bele,Sumedh %A Cassidy,Christine %A Curran,Janet %A Johnson,David W %A Bailey,J A Michelle %+ Department of Pediatrics, Cumming School of Medicine, University of Calgary, 28 Oki Dr NW, Calgary, AB, T3B 6A8, Canada, 1 403 955 3015, jamichelle.bailey@albertahealthservices.ca %K telemedicine %K eHealth %K pediatric care %K inpatient %K regional %K rural %K Canada %K Theoretical Domains Framework %K qualitative %D 2021 %7 22.12.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Inequities in access to health services are a global concern and a concern for Canadian populations living in rural areas. Rural children hospitalized at tertiary children’s hospitals have higher rates of medical complexity and experience more expensive hospitalizations and more frequent readmissions. The 2 tertiary pediatric hospitals in Alberta, Canada, have already been operating above capacity, but the pediatric beds at regional hospitals are underused. Such imbalance could lead to poor patient safety and increased readmission risk at tertiary pediatric hospitals and diminish the clinical exposure of regional pediatric health care providers, erode their confidence, and compel health systems to further reduce the capacity at regional sites. A Telemedicine Rounding and Consultation for Kids (TRaC-K) model was proposed to enable health care providers at Alberta Children’s Hospital to partner with their counterparts at Medicine Hat Regional Hospital to provide inpatient clinical care for pediatric patients who would otherwise have to travel or be transferred to the tertiary site. Objective: The aim of this study is to identify perceived barriers and enablers to implementing the TRaC-K model. Methods: This study was guided by the Theoretical Domains Framework (TDF) and used qualitative methods. We collected qualitative data from 42 participants from tertiary and regional hospitals through 31 semistructured interviews and 2 focus groups. These data were thematically analyzed to identify major subthemes within each TDF domain. These subthemes were further aggregated and categorized into barriers or enablers to implementing the TRaC-K model and were tabulated separately. Results: Our study identified 31 subthemes in 14 TDF domains, ranging from administrative issues to specific clinical conditions. We were able to merge these subthemes into larger themes and categorize them into 4 barriers and 4 enablers. Our findings showed that the barriers were lack of awareness of telemedicine, skills to provide virtual clinical care, unclear processes and resources to support TRaC-K, and concerns about clear roles and responsibilities. The enablers were health care providers’ motivation to provide care closer to home, supporting system resource stewardship, site and practice compatibility, and motivation to strengthen tertiary–regional relationships. Conclusions: This systematic inquiry into the perceived barriers and enablers to the implementation of TRaC-K helped us to gain insights from various health care providers’ and family members’ perspectives. We will use these findings to design interventions to overcome the identified barriers and harness the enablers to encourage successful implementation of TRaC-K. These findings will inform the implementation of telemedicine-based interventions in pediatric settings in other parts of Canada and beyond. International Registered Report Identifier (IRRID): RR2-10.1186/s12913-018-3859-2 %M 34941561 %R 10.2196/28610 %U https://www.jmir.org/2021/12/e28610 %U https://doi.org/10.2196/28610 %U http://www.ncbi.nlm.nih.gov/pubmed/34941561 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 12 %P e31321 %T An Integrated Model to Improve Medication Reconciliation in Oncology: Prospective Interventional Study %A Passardi,Alessandro %A Serra,Patrizia %A Donati,Caterina %A Fiori,Federica %A Prati,Sabrina %A Vespignani,Roberto %A Taglioni,Gabriele %A Farfaneti Ghetti,Patrizia %A Martinelli,Giovanni %A Nanni,Oriana %A Altini,Mattia %A Frassineti,Giovanni Luca %A Minguzzi,Martina Vittoria %+ IT Service, IRCCS Istituto Romagnolo per lo Studio dei Tumori (IRST) “Dino Amadori”, Via P. Maroncelli 40, Meldola, 47014, Italy, 39 0543 739992, roberto.vespignani@irst.emr.it %K medication recognition %K medication reconciliation %K IT platform %K community pharmacies %K healthcare transitions %K pharmacy %K oncology %K drug incompatibility %K information technology %K drug interactions %D 2021 %7 20.12.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Accurate medication reconciliation reduces the risk of drug incompatibilities and adverse events that can occur during transitions in care. Community pharmacies (CPs) are a crucial part of the health care system and could be involved in collecting essential information on conventional and supplementary drugs used at home. Objective: The aim of this paper was to establish an alliance between our cancer institute, Istituto Romagnolo per lo Studio dei Tumori (IRST), and CPs, the latter entrusted with the completion of a pharmacological recognition survey. We also aimed to integrate the national information technology (IT) platform of CPs with the electronic medical records of IRST. Methods: Cancer patients undergoing antiblastic treatments were invited to select a CP taking part in the study and to complete the pharmacological recognition step. The information collected by the pharmacist was sent to the electronic medical records of IRST through the new IT platform, after which the oncologist performed the reconciliation process. Results: A total of 66 CPs completed surveys for 134 patients. An average of 5.9 drugs per patient was used at home, with 12 or more used in the most advanced age groups. Moreover, 60% (80/134) of the patients used nonconventional products or critical foods. Some potential interactions between nonconventional medications and cancer treatments were reported. Conclusions: In the PROF-1 (Progetto di Rete in Oncologia con le Farmacie di comunità della Romagna) study, an alliance was created between our cancer center and CPs to improve medication reconciliation, and a new integrated IT platform was validated. Trial Registration: ClinicalTrials.gov NCT04796142; https://clinicaltrials.gov/ct2/show/NCT04796142 %M 34932001 %R 10.2196/31321 %U https://www.jmir.org/2021/12/e31321 %U https://doi.org/10.2196/31321 %U http://www.ncbi.nlm.nih.gov/pubmed/34932001 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 12 %P e24109 %T The Health Care Sector’s Experience of Blockchain: A Cross-disciplinary Investigation of Its Real Transformative Potential %A Yeung,Karen %+ Birmingham Law School and School of Computer Science, University of Birmingham, Edgbaston, Birmingham, B15 2TT, United Kingdom, 44 0121 414 3344, k.yeung@bham.ac.uk %K blockchain %K health information management %K health information systems %K electronic health record %K data sharing %K health services administration %K privacy of patient data %K computer security %K mobile phone %D 2021 %7 20.12.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Academic literature highlights blockchain’s potential to transform health care, particularly by seamlessly and securely integrating existing data silos while enabling patients to exercise automated, fine-grained control over access to their electronic health records. However, no serious scholarly attempt has been made to assess how these technologies have in fact been applied to real-world health care contexts. Objective: The primary aim of this paper is to assess whether blockchain’s theoretical potential to deliver transformative benefits to health care is likely to become a reality by undertaking a critical investigation of the health care sector’s actual experience of blockchain technologies to date. Methods: This mixed methods study entailed a series of iterative, in-depth, theoretically oriented, desk-based investigations and 2 focus group investigations. It builds on the findings of a companion research study documenting real-world engagement with blockchain technologies in health care. Data were sourced from academic and gray literature from multiple disciplinary perspectives concerned with the configuration, design, and functionality of blockchain technologies. The analysis proceeded in 3 stages. First, it undertook a qualitative investigation of observed patterns of blockchain for health care engagement to identify the application domains, data-sharing problems, and the challenges encountered to date. Second, it critically compared these experiences with claims about blockchain’s potential benefits in health care. Third, it developed a theoretical account of challenges that arise in implementing blockchain in health care contexts, thus providing a firmer foundation for appraising its future prospects in health care. Results: Health care organizations have actively experimented with blockchain technologies since 2016 and have demonstrated proof of concept for several applications (use cases) primarily concerned with administrative data and to facilitate medical research by enabling algorithmic models to be trained on multiple disparately located sets of patient data in a secure, privacy-preserving manner. However, blockchain technology is yet to be implemented at scale in health care, remaining largely in its infancy. These early experiences have demonstrated blockchain’s potential to generate meaningful value to health care by facilitating data sharing between organizations in circumstances where computational trust can overcome a lack of social trust that might otherwise prevent valuable cooperation. Although there are genuine prospects of using blockchain to bring about positive transformations in health care, the successful development of blockchain for health care applications faces a number of very significant, multidimensional, and highly complex challenges. Early experience suggests that blockchain is unlikely to rapidly and radically revolutionize health care. Conclusions: The successful development of blockchain for health care applications faces numerous significant, multidimensional, and complex challenges that will not be easily overcome, suggesting that blockchain technologies are unlikely to revolutionize health care in the near future. %M 34932009 %R 10.2196/24109 %U https://www.jmir.org/2021/12/e24109 %U https://doi.org/10.2196/24109 %U http://www.ncbi.nlm.nih.gov/pubmed/34932009 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 12 %P e33540 %T How Clinicians Perceive Artificial Intelligence–Assisted Technologies in Diagnostic Decision Making: Mixed Methods Approach %A Hah,Hyeyoung %A Goldin,Deana Shevit %+ Information Systems and Business Analytics, College of Business, Florida International University, 11200 SW 8th Street, Miami, FL, 33199, United States, 1 3053484342, hhah@fiu.edu %K artificial intelligence algorithms %K AI %K diagnostic capability %K virtual care %K multilevel modeling %K human-AI teaming %K natural language understanding %D 2021 %7 16.12.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: With the rapid development of artificial intelligence (AI) and related technologies, AI algorithms are being embedded into various health information technologies that assist clinicians in clinical decision making. Objective: This study aimed to explore how clinicians perceive AI assistance in diagnostic decision making and suggest the paths forward for AI-human teaming for clinical decision making in health care. Methods: This study used a mixed methods approach, utilizing hierarchical linear modeling and sentiment analysis through natural language understanding techniques. Results: A total of 114 clinicians participated in online simulation surveys in 2020 and 2021. These clinicians studied family medicine and used AI algorithms to aid in patient diagnosis. Their overall sentiment toward AI-assisted diagnosis was positive and comparable with diagnoses made without the assistance of AI. However, AI-guided decision making was not congruent with the way clinicians typically made decisions in diagnosing illnesses. In a quantitative survey, clinicians reported perceiving current AI assistance as not likely to enhance diagnostic capability and negatively influenced their overall performance (β=–0.421, P=.02). Instead, clinicians’ diagnostic capabilities tended to be associated with well-known parameters, such as education, age, and daily habit of technology use on social media platforms. Conclusions: This study elucidated clinicians’ current perceptions and sentiments toward AI-enabled diagnosis. Although the sentiment was positive, the current form of AI assistance may not be linked with efficient decision making, as AI algorithms are not well aligned with subjective human reasoning in clinical diagnosis. Developers and policy makers in health could gather behavioral data from clinicians in various disciplines to help align AI algorithms with the unique subjective patterns of reasoning that humans employ in clinical diagnosis. %M 34924356 %R 10.2196/33540 %U https://www.jmir.org/2021/12/e33540 %U https://doi.org/10.2196/33540 %U http://www.ncbi.nlm.nih.gov/pubmed/34924356 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 12 %P e26381 %T mUzima Mobile Electronic Health Record (EHR) System: Development and Implementation at Scale %A Were,Martin Chieng %A Savai,Simon %A Mokaya,Benard %A Mbugua,Samuel %A Ribeka,Nyoman %A Cholli,Preetam %A Yeung,Ada %+ Department of Biomedical Informatics, Vanderbilt University Medical Center, 2525 West End Avenue, Suite 750, Nashville, TN, 37203, United States, 1 615 322 9374, martin.c.were@vumc.org %K mobile health %K electronic medical records %K developing countries %K digital divide %K digital health %K global health %D 2021 %7 14.12.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: The predominant implementation paradigm of electronic health record (EHR) systems in low- and middle-income countries (LMICs) relies on standalone system installations at facilities. This implementation approach exacerbates the digital divide, with facilities in areas with inadequate electrical and network infrastructure often left behind. Mobile health (mHealth) technologies have been implemented to extend the reach of digital health, but these systems largely add to the problem of siloed patient data, with few seamlessly interoperating with the EHR systems that are now scaled nationally in many LMICs. Robust mHealth applications that effectively extend EHR systems are needed to improve access, improve quality of care, and ameliorate the digital divide. Objective: We report on the development and scaled implementation of mUzima, an mHealth extension of the most broadly deployed EHR system in LMICs (OpenMRS). Methods: The “Guidelines for reporting of health interventions using mobile phones: mobile (mHealth) evidence reporting assessment (mERA)” checklist was employed to report on the mUzima application. The World Health Organization (WHO) Principles for Digital Development framework was used as a secondary reference framework. Details of mUzima’s architecture, core features, functionalities, and its implementation status are provided to highlight elements that can be adapted in other systems. Results: mUzima is an open-source, highly configurable Android application with robust features including offline management, deduplication, relationship management, security, cohort management, and error resolution, among many others. mUzima allows providers with lower-end Android smartphones (version 4.4 and above) who work remotely to access historical patient data, collect new data, view media, leverage decision support, conduct store-and-forward teleconsultation, and geolocate clients. The application is supported by an active community of developers and users, with feature priorities vetted by the community. mUzima has been implemented nationally in Kenya, is widely used in Rwanda, and is gaining scale in Uganda and Mozambique. It is disease-agnostic, with current use cases in HIV, cancer, chronic disease, and COVID-19 management, among other conditions. mUzima meets all WHO’s Principles of Digital Development, and its scaled implementation success has led to its recognition as a digital global public good and its listing in the WHO Digital Health Atlas. Conclusions: Greater emphasis should be placed on mHealth applications that robustly extend reach of EHR systems within resource-limited settings, as opposed to siloed mHealth applications. This is particularly important given that health information exchange infrastructure is yet to mature in many LMICs. The mUzima application demonstrates how this can be done at scale, as evidenced by its adoption across multiple countries and for numerous care domains. %M 34904952 %R 10.2196/26381 %U https://www.jmir.org/2021/12/e26381 %U https://doi.org/10.2196/26381 %U http://www.ncbi.nlm.nih.gov/pubmed/34904952 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 8 %N 12 %P e34170 %T Sharing Clinical Notes and Electronic Health Records With People Affected by Mental Health Conditions: Scoping Review %A Schwarz,Julian %A Bärkås,Annika %A Blease,Charlotte %A Collins,Lorna %A Hägglund,Maria %A Markham,Sarah %A Hochwarter,Stefan %+ Department of Psychiatry and Psychotherapy, Immanuel Klinik Rüdersdorf, Brandenburg Medical School Theodor Fontane, Seebad 82/83, Rüdersdorf, 15562, Germany, 49 17622652628, julian.schwarz@mhb-fontane.de %K electronic health record %K open notes %K user involvement %K patient advocacy %K patient portal %K patient rights %K collaborative health care %K participation %K coproduction %K system transformation %K health care reform %D 2021 %7 14.12.2021 %9 Review %J JMIR Ment Health %G English %X Background: Electronic health records (EHRs) are increasingly implemented internationally, whereas digital sharing of EHRs with service users (SUs) is a relatively new practice. Studies of patient-accessible EHRs (PAEHRs)—often referred to as open notes—have revealed promising results within general medicine settings. However, studies carried out in mental health care (MHC) settings highlight several ethical and practical challenges that require further exploration. Objective: This scoping review aims to map available evidence on PAEHRs in MHC. We seek to relate findings with research from other health contexts, to compare different stakeholders’ perspectives, expectations, actual experiences with PAEHRs, and identify potential research gaps. Methods: A systematic scoping review was performed using 6 electronic databases. Studies that focused on the digital sharing of clinical notes or EHRs with people affected by mental health conditions up to September 2021 were included. The Mixed Methods Appraisal Tool was used to assess the quality of the studies. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) Extension for Scoping Reviews guided narrative synthesis and reporting of findings. Results: Of the 1034 papers screened, 31 were included in this review. The studies used mostly qualitative methods or surveys and were predominantly published after 2018 in the United States. PAEHRs were examined in outpatient (n=29) and inpatient settings (n=11), and a third of all research was conducted in Veterans Affairs Mental Health. Narrative synthesis allowed the integration of findings according to the different stakeholders. First, SUs reported mainly positive experiences with PAEHRs, such as increased trust in their clinician, health literacy, and empowerment. Negative experiences were related to inaccurate notes, disrespectful language use, or uncovering of undiscussed diagnoses. Second, for health care professionals, concerns outweigh the benefits of sharing EHRs, including an increased clinical burden owing to more documentation efforts and possible harm triggered by reading the notes. Third, care partners gained a better understanding of their family members’ mental problems and were able to better support them when they had access to their EHR. Finally, policy stakeholders and experts addressed ethical challenges and recommended the development of guidelines and trainings to better prepare both clinicians and SUs on how to write and read notes. Conclusions: PAEHRs in MHC may strengthen user involvement, patients’ autonomy, and shift medical treatment to a coproduced process. Acceptance issues among health care professionals align with the findings from general health settings. However, the corpus of evidence on digital sharing of EHRs with people affected by mental health conditions is limited. Above all, further research is needed to examine the clinical effectiveness, efficiency, and implementation of this sociotechnical intervention. %M 34904956 %R 10.2196/34170 %U https://mental.jmir.org/2021/12/e34170 %U https://doi.org/10.2196/34170 %U http://www.ncbi.nlm.nih.gov/pubmed/34904956 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 12 %P e30970 %T Transformation and Evaluation of the MIMIC Database in the OMOP Common Data Model: Development and Usability Study %A Paris,Nicolas %A Lamer,Antoine %A Parrot,Adrien %+ InterHop, 30 avenue du Maine, Paris, 75015, France, 33 3 20 62 69 69, nicolas.paris@riseup.net %K data reuse %K open data %K OMOP %K common data model %K critical care %K machine learning %K big data %K health informatics %K health data %K health database %K electronic health records %K open access database %K digital health %K intensive care %K health care %D 2021 %7 14.12.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: In the era of big data, the intensive care unit (ICU) is likely to benefit from real-time computer analysis and modeling based on close patient monitoring and electronic health record data. The Medical Information Mart for Intensive Care (MIMIC) is the first open access database in the ICU domain. Many studies have shown that common data models (CDMs) improve database searching by allowing code, tools, and experience to be shared. The Observational Medical Outcomes Partnership (OMOP) CDM is spreading all over the world. Objective: The objective was to transform MIMIC into an OMOP database and to evaluate the benefits of this transformation for analysts. Methods: We transformed MIMIC (version 1.4.21) into OMOP format (version 5.3.3.1) through semantic and structural mapping. The structural mapping aimed at moving the MIMIC data into the right place in OMOP, with some data transformations. The mapping was divided into 3 phases: conception, implementation, and evaluation. The conceptual mapping aimed at aligning the MIMIC local terminologies to OMOP's standard ones. It consisted of 3 phases: integration, alignment, and evaluation. A documented, tested, versioned, exemplified, and open repository was set up to support the transformation and improvement of the MIMIC community's source code. The resulting data set was evaluated over a 48-hour datathon. Results: With an investment of 2 people for 500 hours, 64% of the data items of the 26 MIMIC tables were standardized into the OMOP CDM and 78% of the source concepts mapped to reference terminologies. The model proved its ability to support community contributions and was well received during the datathon, with 160 participants and 15,000 requests executed with a maximum duration of 1 minute. Conclusions: The resulting MIMIC-OMOP data set is the first MIMIC-OMOP data set available free of charge with real disidentified data ready for replicable intensive care research. This approach can be generalized to any medical field. %M 34904958 %R 10.2196/30970 %U https://medinform.jmir.org/2021/12/e30970 %U https://doi.org/10.2196/30970 %U http://www.ncbi.nlm.nih.gov/pubmed/34904958 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 12 %P e31333 %T CE Accreditation and Barriers to CE Marking of Pediatric Drug Calculators for Mobile Devices: Scoping Review and Qualitative Analysis %A Koldeweij,Charlotte %A Clarke,Jonathan %A Nijman,Joppe %A Feather,Calandra %A de Wildt,Saskia N %A Appelbaum,Nicholas %+ Department of Pharmacology and Toxicology, Radboud Institute for Health Sciences, Radboud University Medical Center, Geert Grooteplein Zuid 21, Nijmegen, 6525 EZ, Netherlands, 31 243616192, charlotte.koldeweij@radboudumc.nl %K pediatric %K drug dosage calculator %K European regulations %K safety %K medical devices %K medical errors %K app %K application %K mobile health %K pharmacy %D 2021 %7 13.12.2021 %9 Review %J J Med Internet Res %G English %X Background: Pediatric drug calculators (PDCs) intended for clinical use qualify as medical devices under the Medical Device Directive and the Medical Device Regulation. The extent to which they comply with European standards on quality and safety is unknown. Objective: This study determines the number of PDCs available as mobile apps for use in the Netherlands that bear a CE mark, and explore the factors influencing the CE marking of such devices among app developers. Methods: A scoping review of Google Play Store and Apple App Store was conducted to identify PDCs available for download in the Netherlands. CE accreditation of the sampled apps was determined by consulting the app landing pages on app stores, by screening the United Kingdom Medicines and Healthcare products Regulatory Agency’s online registry of medical devices, and by surveying app developers. The barriers to CE accreditation were also explored through a survey of app developers. Results: Of 632 screened apps, 74 were eligible, including 60 pediatric drug dosage calculators and 14 infusion rate calculators. One app was CE marked. Of the 20 (34%) respondents to the survey, 8 considered their apps not to be medical devices based on their intent of use or functionality. Three developers had not aimed to make their app available for use in Europe. Other barriers that may explain the limited CE accreditation of sampled PDC apps included poor awareness of European regulations among developers and a lack of restrictions when placing PDCs in app stores. Conclusions: The compliance of PDCs with European standards on medical devices is poor. This puts clinicians and their patients at risk of medical errors resulting from the largely unrestricted use of these apps. %M 34898456 %R 10.2196/31333 %U https://www.jmir.org/2021/12/e31333 %U https://doi.org/10.2196/31333 %U http://www.ncbi.nlm.nih.gov/pubmed/34898456 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 12 %P e29286 %T Leveraging National Claims and Hospital Big Data: Cohort Study on a Statin-Drug Interaction Use Case %A Bannay,Aurélie %A Bories,Mathilde %A Le Corre,Pascal %A Riou,Christine %A Lemordant,Pierre %A Van Hille,Pascal %A Chazard,Emmanuel %A Dode,Xavier %A Cuggia,Marc %A Bouzillé,Guillaume %+ Inserm, Laboratoire Traitement du Signal et de l'Image - UMR 1099, Centre Hospitalier Universitaire de Rennes, Université de Rennes 1, UFR Santé, laboratoire d'informatique médicale, 2 avenue du Professeur Léon Bernard, Rennes, 35000, France, 33 615711230, guillaume.bouzille@gmail.com %K drug interactions %K statins %K administrative claims %K health care %K big data %K data linking %K data warehousing %D 2021 %7 13.12.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Linking different sources of medical data is a promising approach to analyze care trajectories. The aim of the INSHARE (Integrating and Sharing Health Big Data for Research) project was to provide the blueprint for a technological platform that facilitates integration, sharing, and reuse of data from 2 sources: the clinical data warehouse (CDW) of the Rennes academic hospital, called eHOP (entrepôt Hôpital), and a data set extracted from the French national claim data warehouse (Système National des Données de Santé [SNDS]). Objective: This study aims to demonstrate how the INSHARE platform can support big data analytic tasks in the health field using a pharmacovigilance use case based on statin consumption and statin-drug interactions. Methods: A Spark distributed cluster-computing framework was used for the record linkage procedure and all analyses. A semideterministic record linkage method based on the common variables between the chosen data sources was developed to identify all patients discharged after at least one hospital stay at the Rennes academic hospital between 2015 and 2017. The use-case study focused on a cohort of patients treated with statins prescribed by their general practitioner or during their hospital stay. Results: The whole process (record linkage procedure and use-case analyses) required 88 minutes. Of the 161,532 and 164,316 patients from the SNDS and eHOP CDW data sets, respectively, 159,495 patients were successfully linked (98.74% and 97.07% of patients from SNDS and eHOP CDW, respectively). Of the 16,806 patients with at least one statin delivery, 8293 patients started the consumption before and continued during the hospital stay, 6382 patients stopped statin consumption at hospital admission, and 2131 patients initiated statins in hospital. Statin-drug interactions occurred more frequently during hospitalization than in the community (3800/10,424, 36.45% and 3253/14,675, 22.17%, respectively; P<.001). Only 121 patients had the most severe level of statin-drug interaction. Hospital stay burden (length of stay and in-hospital mortality) was more severe in patients with statin-drug interactions during hospitalization. Conclusions: This study demonstrates the added value of combining and reusing clinical and claim data to provide large-scale measures of drug-drug interaction prevalence and care pathways outside hospitals. It builds a path to move the current health care system toward a Learning Health System using knowledge generated from research on real-world health data. %M 34898457 %R 10.2196/29286 %U https://medinform.jmir.org/2021/12/e29286 %U https://doi.org/10.2196/29286 %U http://www.ncbi.nlm.nih.gov/pubmed/34898457 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 4 %P e27171 %T Communicating Guideline Recommendations Using Graphic Narrative Versus Text-Based Broadcast Screensavers: Design and Implementation Study %A Sinnenberg,Lauren %A Umscheid,Craig A %A Shofer,Frances S %A Leri,Damien %A Meisel,Zachary F %+ Center for Emergency Care Policy and Research, University of Pennsylvania, Ravdin Ground, 3400 Spruce Street, Philadelphia, PA, 19104, United States, 1 215 746 5618, zfm@pennmedicine.upenn.edu %K medical informatics %K screensaver %K guideline dissemination %K graphic narratives %K health communication %K workstation %K clinical workstation %K guidelines %K medical education %K education %D 2021 %7 13.12.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: The use of graphic narratives, defined as stories that use images for narration, is growing in health communication. Objective: The aim of this study was to describe the design and implementation of a graphic narrative screensaver (GNS) to communicate a guideline recommendation (ie, avoiding low-value acid suppressive therapy [AST] use in hospital inpatients) and examine the comparative effectiveness of the GNS versus a text-based screensaver (TBS) on clinical practice (ie, low-value AST prescriptions) and clinician recall. Methods: During a 2-year period, the GNS and the TBS were displayed on inpatient clinical workstations. The numbers of new AST prescriptions were examined in the four quarters before, the three quarters during, and the one quarter after screensavers were implemented. Additionally, an electronic survey was sent to resident physicians 1 year after the intervention to assess screensaver recall. Results: Designing an aesthetically engaging graphic that could be rapidly understood was critical in the development of the GNS. The odds of receiving an AST prescription on medicine and medicine subspecialty services after the screensavers were implemented were lower for all four quarters (ie, GNS and TBS broadcast together, only TBS broadcast, only GNS broadcast, and no AST screensavers broadcast) compared to the quarter prior to implementation (odds ratio [OR] 0.85, 95% CI 0.78-0.92; OR 0.89, 95% CI 0.82-0.97; OR 0.87, 95% CI 0.80-0.95; and OR 0.81, 95% CI 0.75-0.89, respectively; P<.001 for all comparisons). There were no statistically significant decreases for other high-volume services, such as the surgical services. These declines appear to have begun prior to screensaver implementation. When surveyed about the screensaver content 1 year later, resident physicians recalled both the GNS and TBS (43/70, 61%, vs 54/70, 77%; P=.07) and those who recalled the screensaver were more likely to recall the main message of the GNS compared to the TBS (30/43, 70%, vs 1/54, 2%; P<.001). Conclusions: It is feasible to use a graphic narrative embedded in a broadcast screensaver to communicate a guideline recommendation, but further study is needed to determine the impact of graphic narratives on clinical practice. %M 34264197 %R 10.2196/27171 %U https://humanfactors.jmir.org/2021/4/e27171 %U https://doi.org/10.2196/27171 %U http://www.ncbi.nlm.nih.gov/pubmed/34264197 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 12 %P e27188 %T The Utility of Different Data Standards to Document Adverse Drug Event Symptoms and Diagnoses: Mixed Methods Study %A Chan,Erina %A Small,Serena S %A Wickham,Maeve E %A Cheng,Vicki %A Balka,Ellen %A Hohl,Corinne M %+ Centre for Clinical Epidemiology and Evaluation, Vancouver Coastal Health Research Institute, 828 West 10th Avenue, Vancouver, BC, V5Z 1M9, Canada, 1 604 875 4111 ext 55219, Serena.Small@ubc.ca %K adverse drug events %K health information technology %K data standards %D 2021 %7 10.12.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Existing systems to document adverse drug events often use free text data entry, which produces nonstandardized and unstructured data that are prone to misinterpretation. Standardized terminology may improve data quality; however, it is unclear which data standard is most appropriate for documenting adverse drug event symptoms and diagnoses. Objective: This study aims to compare the utility, strengths, and weaknesses of different data standards for documenting adverse drug event symptoms and diagnoses. Methods: We performed a mixed methods substudy of a multicenter retrospective chart review. We reviewed the research records of prospectively diagnosed adverse drug events at 5 Canadian hospitals. A total of 2 pharmacy research assistants independently entered the symptoms and diagnoses for the adverse drug events using four standards: Medical Dictionary for Regulatory Activities (MedDRA), Systematized Nomenclature of Medicine (SNOMED) Clinical Terms, SNOMED Adverse Reaction (SNOMED ADR), and International Classification of Diseases (ICD) 11th Revision. Disagreements between research assistants regarding the case-specific utility of data standards were discussed until a consensus was reached. We used consensus ratings to determine the proportion of adverse drug events covered by a data standard and coded and analyzed field notes from the consensus sessions. Results: We reviewed 573 adverse drug events and found that MedDRA and ICD-11 had excellent coverage of adverse drug event symptoms and diagnoses. MedDRA had the highest number of matches between the research assistants, whereas ICD-11 had the fewest. SNOMED ADR had the lowest proportion of adverse drug event coverage. The research assistants were most likely to encounter terminological challenges with SNOMED ADR and usability challenges with ICD-11, whereas least likely to encounter challenges with MedDRA. Conclusions: Usability, comprehensiveness, and accuracy are important features of data standards for documenting adverse drug event symptoms and diagnoses. On the basis of our results, we recommend the use of MedDRA. %M 34890351 %R 10.2196/27188 %U https://www.jmir.org/2021/12/e27188 %U https://doi.org/10.2196/27188 %U http://www.ncbi.nlm.nih.gov/pubmed/34890351 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 10 %N 12 %P e27984 %T Exploring the Intersection Between Health Professionals’ Learning and eHealth Data: Protocol for a Comprehensive Research Program in Practice Analytics in Health Care %A Janssen,Anna %A Talic,Stella %A Gasevic,Dragan %A Kay,Judy %A Shaw,Tim %+ Faculty of Medicine and Health, The University of Sydney, Level 2, Charles Perkins Centre, Sydney, 2006, Australia, 61 9036 9406, anna.janssen@sydney.edu.au %K digital health %K health informatics %K practice analytics in health care %K health professions education %K continuing professional development %D 2021 %7 9.12.2021 %9 Protocol %J JMIR Res Protoc %G English %X Background: There is an increasing amount of electronic data sitting within the health system. These data have untapped potential to improve clinical practice if extracted efficiently and harnessed to change the behavior of health professionals. Furthermore, there is an increasing expectation from the government and peak bodies that both individual health professionals and health care organizations will use electronic data for a range of applications, including improving health service delivery and informing clinical practice and professional accreditation. Objective: The aim of this research program is to make eHealth data captured within tertiary health care organizations more actionable to health professionals for use in practice reflection, professional development, and other quality improvement activities. Methods: A multidisciplinary approach was used to connect academic experts from core disciplines of health and medicine, education and learning sciences, and engineering and information communication technology with government and health service partners to identify key problems preventing the health care industry from using electronic data to support health professional learning. This multidisciplinary approach was used to design a large-scale research program to solve the problem of making eHealth data more accessible to health professionals for practice reflection. The program will be delivered over 5 years by doctoral candidates undertaking research projects with discrete aims that run in parallel to achieving this program’s objectives. Results: The process used to develop the research program identified 7 doctoral research projects to answer the program objectives, split across 3 streams. Conclusions: This research program has the potential to successfully unpack electronic data siloed within clinical sites and enable health professionals to use them to reflect on their practice and deliver informed and improved care. The program will contribute to current practices by fostering stronger connections between industry and academia, interlinking doctoral research projects to solve complex problems, and creating new knowledge for clinical sites on how data can be used to understand and improve performance. Furthermore, the program aims to affect policy by developing insights on how professional development programs may be strengthened to enhance their alignment with clinical practice. The key contributions of this paper include the introduction of a new conceptualized research program, Practice Analytics in Health care, by describing the foundational academic disciplines that the program is formed of and presenting scientific methods for its design and development. International Registered Report Identifier (IRRID): PRR1-10.2196/27984 %M 34889768 %R 10.2196/27984 %U https://www.researchprotocols.org/2021/12/e27984 %U https://doi.org/10.2196/27984 %U http://www.ncbi.nlm.nih.gov/pubmed/34889768 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 12 %P e32698 %T A BERT-Based Generation Model to Transform Medical Texts to SQL Queries for Electronic Medical Records: Model Development and Validation %A Pan,Youcheng %A Wang,Chenghao %A Hu,Baotian %A Xiang,Yang %A Wang,Xiaolong %A Chen,Qingcai %A Chen,Junjie %A Du,Jingcheng %+ Intelligent Computing Research Center, Harbin Institute of Technology, No. 6, Pingshan 1st Road, Shenzhen, 518055, China, 86 136 9164 0856, hubaotian@hit.edu.cn %K electronic medical record %K text-to-SQL generation %K BERT %K grammar-based decoding %K tree-structured intermediate representation %D 2021 %7 8.12.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Electronic medical records (EMRs) are usually stored in relational databases that require SQL queries to retrieve information of interest. Effectively completing such queries can be a challenging task for medical experts due to the barriers in expertise. Existing text-to-SQL generation studies have not been fully embraced in the medical domain. Objective: The objective of this study was to propose a neural generation model that can jointly consider the characteristics of medical text and the SQL structure to automatically transform medical texts to SQL queries for EMRs. Methods: We proposed a medical text–to-SQL model (MedTS), which employed a pretrained Bidirectional Encoder Representations From Transformers model as the encoder and leveraged a grammar-based long short-term memory network as the decoder to predict the intermediate representation that can easily be transformed into the final SQL query. We adopted the syntax tree as the intermediate representation rather than directly regarding the SQL query as an ordinary word sequence, which is more in line with the tree-structure nature of SQL and can also effectively reduce the search space during generation. Experiments were conducted on the MIMICSQL dataset, and 5 competitor methods were compared. Results: Experimental results demonstrated that MedTS achieved the accuracy of 0.784 and 0.899 on the test set in terms of logic form and execution, respectively, which significantly outperformed the existing state-of-the-art methods. Further analyses proved that the performance on each component of the generated SQL was relatively balanced and offered substantial improvements. Conclusions: The proposed MedTS was effective and robust for improving the performance of medical text–to-SQL generation, indicating strong potential to be applied in the real medical scenario. %M 34889749 %R 10.2196/32698 %U https://medinform.jmir.org/2021/12/e32698 %U https://doi.org/10.2196/32698 %U http://www.ncbi.nlm.nih.gov/pubmed/34889749 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 12 %P e25022 %T On Missingness Features in Machine Learning Models for Critical Care: Observational Study %A Singh,Janmajay %A Sato,Masahiro %A Ohkuma,Tomoko %+ Fuji Xerox Co, Ltd, 6 Chome-1-1 Minatomirai, Nishi Ward, Yokohama, 220-0012, Japan, 81 7041120526, janmajaysingh14@gmail.com %K electronic health records %K informative missingness %K machine learning %K missing data %K hospital mortality %K sepsis %D 2021 %7 8.12.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Missing data in electronic health records is inevitable and considered to be nonrandom. Several studies have found that features indicating missing patterns (missingness) encode useful information about a patient’s health and advocate for their inclusion in clinical prediction models. But their effectiveness has not been comprehensively evaluated. Objective: The goal of the research is to study the effect of including informative missingness features in machine learning models for various clinically relevant outcomes and explore robustness of these features across patient subgroups and task settings. Methods: A total of 48,336 electronic health records from the 2012 and 2019 PhysioNet Challenges were used, and mortality, length of stay, and sepsis outcomes were chosen. The latter dataset was multicenter, allowing external validation. Gated recurrent units were used to learn sequential patterns in the data and classify or predict labels of interest. Models were evaluated on various criteria and across population subgroups evaluating discriminative ability and calibration. Results: Generally improved model performance in retrospective tasks was observed on including missingness features. Extent of improvement depended on the outcome of interest (area under the curve of the receiver operating characteristic [AUROC] improved from 1.2% to 7.7%) and even patient subgroup. However, missingness features did not display utility in a simulated prospective setting, being outperformed (0.9% difference in AUROC) by the model relying only on pathological features. This was despite leading to earlier detection of disease (true positives), since including these features led to a concomitant rise in false positive detections. Conclusions: This study comprehensively evaluated effectiveness of missingness features on machine learning models. A detailed understanding of how these features affect model performance may lead to their informed use in clinical settings especially for administrative tasks like length of stay prediction where they present the greatest benefit. While missingness features, representative of health care processes, vary greatly due to intra- and interhospital factors, they may still be used in prediction models for clinically relevant outcomes. However, their use in prospective models producing frequent predictions needs to be explored further. %M 34889756 %R 10.2196/25022 %U https://medinform.jmir.org/2021/12/e25022 %U https://doi.org/10.2196/25022 %U http://www.ncbi.nlm.nih.gov/pubmed/34889756 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 12 %P e33049 %T Differential Biases and Variabilities of Deep Learning–Based Artificial Intelligence and Human Experts in Clinical Diagnosis: Retrospective Cohort and Survey Study %A Cha,Dongchul %A Pae,Chongwon %A Lee,Se A %A Na,Gina %A Hur,Young Kyun %A Lee,Ho Young %A Cho,A Ra %A Cho,Young Joon %A Han,Sang Gil %A Kim,Sung Huhn %A Choi,Jae Young %A Park,Hae-Jeong %+ Center for Systems and Translational Brain Sciences, Institute of Human Complexity and Systems Science, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seoul, 03722, Republic of Korea, 82 2 2228 2363, parkhj@yuhs.ac %K human-machine cooperation %K convolutional neural network %K deep learning, class imbalance problem %K otoscopy %K eardrum %K artificial intelligence %K otology %K computer-aided diagnosis %D 2021 %7 8.12.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Deep learning (DL)–based artificial intelligence may have different diagnostic characteristics than human experts in medical diagnosis. As a data-driven knowledge system, heterogeneous population incidence in the clinical world is considered to cause more bias to DL than clinicians. Conversely, by experiencing limited numbers of cases, human experts may exhibit large interindividual variability. Thus, understanding how the 2 groups classify given data differently is an essential step for the cooperative usage of DL in clinical application. Objective: This study aimed to evaluate and compare the differential effects of clinical experience in otoendoscopic image diagnosis in both computers and physicians exemplified by the class imbalance problem and guide clinicians when utilizing decision support systems. Methods: We used digital otoendoscopic images of patients who visited the outpatient clinic in the Department of Otorhinolaryngology at Severance Hospital, Seoul, South Korea, from January 2013 to June 2019, for a total of 22,707 otoendoscopic images. We excluded similar images, and 7500 otoendoscopic images were selected for labeling. We built a DL-based image classification model to classify the given image into 6 disease categories. Two test sets of 300 images were populated: balanced and imbalanced test sets. We included 14 clinicians (otolaryngologists and nonotolaryngology specialists including general practitioners) and 13 DL-based models. We used accuracy (overall and per-class) and kappa statistics to compare the results of individual physicians and the ML models. Results: Our ML models had consistently high accuracies (balanced test set: mean 77.14%, SD 1.83%; imbalanced test set: mean 82.03%, SD 3.06%), equivalent to those of otolaryngologists (balanced: mean 71.17%, SD 3.37%; imbalanced: mean 72.84%, SD 6.41%) and far better than those of nonotolaryngologists (balanced: mean 45.63%, SD 7.89%; imbalanced: mean 44.08%, SD 15.83%). However, ML models suffered from class imbalance problems (balanced test set: mean 77.14%, SD 1.83%; imbalanced test set: mean 82.03%, SD 3.06%). This was mitigated by data augmentation, particularly for low incidence classes, but rare disease classes still had low per-class accuracies. Human physicians, despite being less affected by prevalence, showed high interphysician variability (ML models: kappa=0.83, SD 0.02; otolaryngologists: kappa=0.60, SD 0.07). Conclusions: Even though ML models deliver excellent performance in classifying ear disease, physicians and ML models have their own strengths. ML models have consistent and high accuracy while considering only the given image and show bias toward prevalence, whereas human physicians have varying performance but do not show bias toward prevalence and may also consider extra information that is not images. To deliver the best patient care in the shortage of otolaryngologists, our ML model can serve a cooperative role for clinicians with diverse expertise, as long as it is kept in mind that models consider only images and could be biased toward prevalent diseases even after data augmentation. %M 34889764 %R 10.2196/33049 %U https://medinform.jmir.org/2021/12/e33049 %U https://doi.org/10.2196/33049 %U http://www.ncbi.nlm.nih.gov/pubmed/34889764 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 5 %N 12 %P e25833 %T Assessing Health Data Security Risks in Global Health Partnerships: Development of a Conceptual Framework %A Espinoza,Juan %A Sikder,Abu Taher %A Dickhoner,James %A Lee,Thomas %+ Department of Pediatrics, Children's Hospital Los Angeles, 4650 Sunset Blvd, Los Angeles, CA, 90006, United States, 1 3233612721, jespinoza@chla.usc.edu %K health information technology %K low- and middle-income countries %K low income %K conceptual framework analysis %K framework method %K data security %K decision-making %K database %K information use %K misuse %K global health %K security %D 2021 %7 8.12.2021 %9 Original Paper %J JMIR Form Res %G English %X Background: Health care databases contain a wealth of information that can be used to develop programs and mature health care systems. There is concern that the sensitive nature of health data (eg, ethnicity, reproductive health, sexually transmitted infections, and lifestyle information) can have significant impact on individuals if misused, particularly among vulnerable and marginalized populations. As academic institutions, nongovernmental organizations, and international agencies begin to collaborate with low- and middle-income countries to develop and deploy health information technology (HIT), it is important to understand the technical and practical security implications of these initiatives. Objective: Our aim is to develop a conceptual framework for risk stratification of global health data partnerships and HIT projects. In addition to identifying key conceptual domains, we map each domain to a variety of publicly available indices that could be used to inform a quantitative model. Methods: We conducted an overview of the literature to identify relevant publications, position statements, white papers, and reports. The research team reviewed all sources and used the framework method and conceptual framework analysis to name and categorize key concepts, integrate them into domains, and synthesize them into an overarching conceptual framework. Once key domains were identified, public international data sources were searched for relevant structured indices to generate quantitative counterparts. Results: We identified 5 key domains to inform our conceptual framework: State of HIT, Economics of Health Care, Demographics and Equity, Societal Freedom and Safety, and Partnership and Trust. Each of these domains was mapped to a number of structured indices. Conclusions: There is a complex relationship among the legal, economic, and social domains of health care, which affects the state of HIT in low- and middle-income countries and associated data security risks. The strength of partnership and trust among collaborating organizations is an important moderating factor. Additional work is needed to formalize the assessment of partnership and trust and to develop a quantitative model of the conceptual framework that can help support organizational decision-making. %M 34889752 %R 10.2196/25833 %U https://formative.jmir.org/2021/12/e25833 %U https://doi.org/10.2196/25833 %U http://www.ncbi.nlm.nih.gov/pubmed/34889752 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 12 %P e27072 %T The Effect of Automated Mammogram Orders Paired With Electronic Invitations to Self-schedule on Mammogram Scheduling Outcomes: Observational Cohort Comparison %A North,Frederick %A Nelson,Elissa M %A Buss,Rebecca J %A Majerus,Rebecca J %A Thompson,Matthew C %A Crum,Brian A %+ Division of Community Internal Medicine, Department of Internal Medicine, Mayo Clinic, 200 First Street SW, Rochester, MN, 55905, United States, 1 507 284 2511, north.frederick@mayo.edu %K electronic health record %K schedule %K patient appointment %K preventive health service %K office visit %K outpatient care %K mammogram %K software tool %K computer software application %K mobile applications %K self-schedule %K app %K EHR %K screening %K diagnostic %K cancer %D 2021 %7 7.12.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Screening mammography is recommended for the early detection of breast cancer. The processes for ordering screening mammography often rely on a health care provider order and a scheduler to arrange the time and location of breast imaging. Self-scheduling after automated ordering of screening mammograms may offer a more efficient and convenient way to schedule screening mammograms. Objective: The aim of this study was to determine the use, outcomes, and efficiency of an automated mammogram ordering and invitation process paired with self-scheduling. Methods: We examined appointment data from 12 months of scheduled mammogram appointments, starting in September 2019 when a web and mobile app self-scheduling process for screening mammograms was made available for the Mayo Clinic primary care practice. Patients registered to the Mayo Clinic Patient Online Services could view the schedules and book their mammogram appointment via the web or a mobile app. Self-scheduling required no telephone calls or staff appointment schedulers. We examined uptake (count and percentage of patients utilizing self-scheduling), number of appointment actions taken by self-schedulers and by those using staff schedulers, no-show outcomes, scheduling efficiency, and weekend and after-hours use of self-scheduling. Results: For patients who were registered to patient online services and had screening mammogram appointment activity, 15.3% (14,387/93,901) used the web or mobile app to do either some mammogram self-scheduling or self-cancelling appointment actions. Approximately 24.4% (3285/13,454) of self-scheduling occurred after normal business hours/on weekends. Approximately 9.3% (8736/93,901) of the patients used self-scheduling/cancelling exclusively. For self-scheduled mammograms, there were 5.7% (536/9433) no-shows compared to 4.6% (3590/77,531) no-shows in staff-scheduled mammograms (unadjusted odds ratio 1.24, 95% CI 1.13-1.36; P<.001). The odds ratio of no-shows for self-scheduled mammograms to staff-scheduled mammograms decreased to 1.12 (95% CI 1.02-1.23; P=.02) when adjusted for age, race, and ethnicity. On average, since there were only 0.197 staff-scheduler actions for each finalized self-scheduled appointment, staff schedulers were rarely used to redo or “clean up” self-scheduled appointments. Exclusively self-scheduled appointments were significantly more efficient than staff-scheduled appointments. Self-schedulers experienced a single appointment step process (one and done) for 93.5% (7553/8079) of their finalized appointments; only 74.5% (52,804/70,839) of staff-scheduled finalized appointments had a similar one-step appointment process (P<.001). For staff-scheduled appointments, 25.5% (18,035/70,839) of the finalized appointments took multiple appointment steps. For finalized appointments that were exclusively self-scheduled, only 6.5% (526/8079) took multiple appointment steps. The staff-scheduled to self-scheduled odds ratio of taking multiple steps for a finalized screening mammogram appointment was 4.9 (95% CI 4.48-5.37; P<.001). Conclusions: Screening mammograms can be efficiently self-scheduled but may be associated with a slight increase in no-shows. Self-scheduling can decrease staff scheduler work and can be convenient for patients who want to manage their appointment scheduling activity after business hours or on weekends. %M 34878997 %R 10.2196/27072 %U https://medinform.jmir.org/2021/12/e27072 %U https://doi.org/10.2196/27072 %U http://www.ncbi.nlm.nih.gov/pubmed/34878997 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 12 %P e28021 %T Recruitment of Patients With Amyotrophic Lateral Sclerosis for Clinical Trials and Epidemiological Studies: Descriptive Study of the National ALS Registry’s Research Notification Mechanism %A Mehta,Paul %A Raymond,Jaime %A Han,Moon Kwon %A Larson,Theodore %A Berry,James D %A Paganoni,Sabrina %A Mitsumoto,Hiroshi %A Bedlack,Richard Stanley %A Horton,D Kevin %+ Agency for Toxic Substances and Disease Registry, Centers for Disease Control and Prevention, 4770 Buford Hwy NE, Atlanta, GA, 30341, United States, 1 770 488 0556, pum4@cdc.gov %K amyotrophic lateral sclerosis %K Lou Gehrig disease %K motor neuron disease %K clinical trials %K patient recruitment %K National ALS Registry %K research notification mechanism %D 2021 %7 7.12.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Researchers face challenges in patient recruitment, especially for rare, fatal diseases such as amyotrophic lateral sclerosis (ALS). These challenges include obtaining sufficient statistical power as well as meeting eligibility requirements such as age, sex, and study proximity. Similarly, persons with ALS (PALS) face difficulty finding and enrolling in research studies for which they are eligible. Objective: The aim of this study was to describe how the federal Agency for Toxic Substances and Disease Registry’s (ATSDR) National ALS Registry is linking PALS to scientists who are conducting research, clinical trials, and epidemiological studies. Methods: Through the Registry’s online research notification mechanism (RNM), PALS can elect to be notified about new research opportunities. This mechanism allows researchers to upload a standardized application outlining their study design and objectives, and proof of Institutional Review Board approval. If the application is approved, ATSDR queries the Registry for PALS meeting the study’s specific eligibility criteria, and then distributes the researcher’s study material and contact information to PALS via email. PALS then need to contact the researcher directly to take part in any research. Such an approach allows ATSDR to protect the confidentiality of Registry enrollees. Results: From 2013 to 2019, a total of 46 institutions around the United States and abroad have leveraged this tool and over 600,000 emails have been sent, resulting in over 2000 patients conservatively recruited for clinical trials and epidemiological studies. Patients between the ages of 60 and 69 had the highest level of participation, whereas those between the ages of 18 and 39 and aged over 80 had the lowest. More males participated (4170/7030, 59.32%) than females (2860/7030, 40.68%). Conclusions: The National ALS Registry’s RNM benefits PALS by connecting them to appropriate ALS research. Simultaneously, the system benefits researchers by expediting recruitment, increasing sample size, and efficiently identifying PALS meeting specific eligibility requirements. As more researchers learn about and use this mechanism, both PALS and researchers can hasten research and expand trial options for PALS. %M 34878988 %R 10.2196/28021 %U https://www.jmir.org/2021/12/e28021 %U https://doi.org/10.2196/28021 %U http://www.ncbi.nlm.nih.gov/pubmed/34878988 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 12 %P e29225 %T Machine Learning Methodologies for Prediction of Rhythm-Control Strategy in Patients Diagnosed With Atrial Fibrillation: Observational, Retrospective, Case-Control Study %A Kim,Rachel S %A Simon,Steven %A Powers,Brett %A Sandhu,Amneet %A Sanchez,Jose %A Borne,Ryan T %A Tumolo,Alexis %A Zipse,Matthew %A West,J Jason %A Aleong,Ryan %A Tzou,Wendy %A Rosenberg,Michael A %+ Clinical Cardiac Electrophysiology Section, Division of Cardiology, University of Colorado School of Medicine, 12631 East 17th Avenue, Mail Stop B130, Aurora, CO, 80045, United States, 1 (303) 724 8391, michael.a.rosenberg@cuanschutz.edu %K atrial fibrillation %K rhythm-control %K machine learning %K ablation %K antiarrhythmia agents %K data science %K biostatistics %K artificial intelligence %D 2021 %7 6.12.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: The identification of an appropriate rhythm management strategy for patients diagnosed with atrial fibrillation (AF) remains a major challenge for providers. Although clinical trials have identified subgroups of patients in whom a rate- or rhythm-control strategy might be indicated to improve outcomes, the wide range of presentations and risk factors among patients presenting with AF makes such approaches challenging. The strength of electronic health records is the ability to build in logic to guide management decisions, such that the system can automatically identify patients in whom a rhythm-control strategy is more likely and can promote efficient referrals to specialists. However, like any clinical decision support tool, there is a balance between interpretability and accurate prediction. Objective: This study aims to create an electronic health record–based prediction tool to guide patient referral to specialists for rhythm-control management by comparing different machine learning algorithms. Methods: We compared machine learning models of increasing complexity and used up to 50,845 variables to predict the rhythm-control strategy in 42,022 patients within the University of Colorado Health system at the time of AF diagnosis. Models were evaluated on the basis of their classification accuracy, defined by the F1 score and other metrics, and interpretability, captured by inspection of the relative importance of each predictor. Results: We found that age was by far the strongest single predictor of a rhythm-control strategy but that greater accuracy could be achieved with more complex models incorporating neural networks and more predictors for each participant. We determined that the impact of better prediction models was notable primarily in the rate of inappropriate referrals for rhythm-control, in which more complex models provided an average of 20% fewer inappropriate referrals than simpler, more interpretable models. Conclusions: We conclude that any health care system seeking to incorporate algorithms to guide rhythm management for patients with AF will need to address this trade-off between prediction accuracy and model interpretability. %M 34874889 %R 10.2196/29225 %U https://medinform.jmir.org/2021/12/e29225 %U https://doi.org/10.2196/29225 %U http://www.ncbi.nlm.nih.gov/pubmed/34874889 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 12 %P e23571 %T A Risk-Based Clinical Decision Support System for Patient-Specific Antimicrobial Therapy (iBiogram): Design and Retrospective Analysis %A Müller,Lars %A Srinivasan,Aditya %A Abeles,Shira R %A Rajagopal,Amutha %A Torriani,Francesca J %A Aronoff-Spencer,Eliah %+ Design Lab, University of California San Diego, 9500 Gilman Drive, MC0436 Atkinson Hall, La Jolla, CA, 92093, United States, 1 8582462639, lmueller@tandemdiabetes.com %K antimicrobial resistance %K clinical decision support %K antibiotic stewardship %K data visualization %D 2021 %7 3.12.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: There is a pressing need for digital tools that can leverage big data to help clinicians select effective antibiotic treatments in the absence of timely susceptibility data. Clinical presentation and local epidemiology can inform therapy selection to balance the risk of antimicrobial resistance and patient risk. However, data and clinical expertise must be appropriately integrated into clinical workflows. Objective: The aim of this study is to leverage available data in electronic health records, to develop a data-driven, user-centered, clinical decision support system to navigate patient safety and population health. Methods: We analyzed 5 years of susceptibility testing (1,078,510 isolates) and patient data (30,761 patients) across a large academic medical center. After curating the data according to the Clinical and Laboratory Standards Institute guidelines, we analyzed and visualized the impact of risk factors on clinical outcomes. On the basis of this data-driven understanding, we developed a probabilistic algorithm that maps these data to individual cases and implemented iBiogram, a prototype digital empiric antimicrobial clinical decision support system, which we evaluated against actual prescribing outcomes. Results: We determined patient-specific factors across syndromes and contexts and identified relevant local patterns of antimicrobial resistance by clinical syndrome. Mortality and length of stay differed significantly depending on these factors and could be used to generate heuristic targets for an acceptable risk of underprescription. Combined with the developed remaining risk algorithm, these factors can be used to inform clinicians’ reasoning. A retrospective comparison of the iBiogram-suggested therapies versus the actual prescription by physicians showed similar performance for low-risk diseases such as urinary tract infections, whereas iBiogram recognized risk and recommended more appropriate coverage in high mortality conditions such as sepsis. Conclusions: The application of such data-driven, patient-centered tools may guide empirical prescription for clinicians to balance morbidity and mortality with antimicrobial stewardship. %M 34870601 %R 10.2196/23571 %U https://www.jmir.org/2021/12/e23571 %U https://doi.org/10.2196/23571 %U http://www.ncbi.nlm.nih.gov/pubmed/34870601 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 12 %P e29812 %T Analyzing Patient Trajectories With Artificial Intelligence %A Allam,Ahmed %A Feuerriegel,Stefan %A Rebhan,Michael %A Krauthammer,Michael %+ Ludwig Maximilian University of Munich, Geschwister-Scholl-Platz 1, Munich, 80539, Germany, 49 8921806790, feuerriegel@lmu.de %K patient trajectories %K longitudinal data %K digital medicine %K artificial intelligence %K machine learning %D 2021 %7 3.12.2021 %9 Viewpoint %J J Med Internet Res %G English %X In digital medicine, patient data typically record health events over time (eg, through electronic health records, wearables, or other sensing technologies) and thus form unique patient trajectories. Patient trajectories are highly predictive of the future course of diseases and therefore facilitate effective care. However, digital medicine often uses only limited patient data, consisting of health events from only a single or small number of time points while ignoring additional information encoded in patient trajectories. To analyze such rich longitudinal data, new artificial intelligence (AI) solutions are needed. In this paper, we provide an overview of the recent efforts to develop trajectory-aware AI solutions and provide suggestions for future directions. Specifically, we examine the implications for developing disease models from patient trajectories along the typical workflow in AI: problem definition, data processing, modeling, evaluation, and interpretation. We conclude with a discussion of how such AI solutions will allow the field to build robust models for personalized risk scoring, subtyping, and disease pathway discovery. %M 34870606 %R 10.2196/29812 %U https://www.jmir.org/2021/12/e29812 %U https://doi.org/10.2196/29812 %U http://www.ncbi.nlm.nih.gov/pubmed/34870606 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 5 %N 12 %P e23440 %T Predicting Risk of Stroke From Lab Tests Using Machine Learning Algorithms: Development and Evaluation of Prediction Models %A Alanazi,Eman M %A Abdou,Aalaa %A Luo,Jake %+ Department of Health Informatics, College of Health Sciences, Saudi Electronic University, Abi Bakr As Siddiq Branch Rd, Riyadh, 13323, Saudi Arabia, 966 112613500, e.alanazi@seu.edu.sa %K stroke %K lab tests %K machine learning technology %K predictive analytics %D 2021 %7 2.12.2021 %9 Original Paper %J JMIR Form Res %G English %X Background: Stroke, a cerebrovascular disease, is one of the major causes of death. It causes significant health and financial burdens for both patients and health care systems. One of the important risk factors for stroke is health-related behavior, which is becoming an increasingly important focus of prevention. Many machine learning models have been built to predict the risk of stroke or to automatically diagnose stroke, using predictors such as lifestyle factors or radiological imaging. However, there have been no models built using data from lab tests. Objective: The aim of this study was to apply computational methods using machine learning techniques to predict stroke from lab test data. Methods: We used the National Health and Nutrition Examination Survey data sets with three different data selection methods (ie, without data resampling, with data imputation, and with data resampling) to develop predictive models. We used four machine learning classifiers and six performance measures to evaluate the performance of the models. Results: We found that accurate and sensitive machine learning models can be created to predict stroke from lab test data. Our results show that the data resampling approach performed the best compared to the other two data selection techniques. Prediction with the random forest algorithm, which was the best algorithm tested, achieved an accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and area under the curve of 0.96, 0.97, 0.96, 0.75, 0.99, and 0.97, respectively, when all of the attributes were used. Conclusions: The predictive model, built using data from lab tests, was easy to use and had high accuracy. In future studies, we aim to use data that reflect different types of stroke and to explore the data to build a prediction model for each type. %M 34860663 %R 10.2196/23440 %U https://formative.jmir.org/2021/12/e23440 %U https://doi.org/10.2196/23440 %U http://www.ncbi.nlm.nih.gov/pubmed/34860663 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 12 %P e26407 %T Chinese-Named Entity Recognition From Adverse Drug Event Records: Radical Embedding-Combined Dynamic Embedding–Based BERT in a Bidirectional Long Short-term Conditional Random Field (Bi-LSTM-CRF) Model %A Wu,Hong %A Ji,Jiatong %A Tian,Haimei %A Chen,Yao %A Ge,Weihong %A Zhang,Haixia %A Yu,Feng %A Zou,Jianjun %A Nakamura,Mitsuhiro %A Liao,Jun %+ School of Science, China Pharmaceutical University, #639 Longmian Avenue, Jiangning District, Nanjing, 211198, China, 86 13952040425, liaojun@cpu.edu.cn %K deep learning %K BERT %K adverse drug reaction %K named entity recognition %K electronic medical records %D 2021 %7 1.12.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: With the increasing variety of drugs, the incidence of adverse drug events (ADEs) is increasing year by year. Massive numbers of ADEs are recorded in electronic medical records and adverse drug reaction (ADR) reports, which are important sources of potential ADR information. Meanwhile, it is essential to make latent ADR information automatically available for better postmarketing drug safety reevaluation and pharmacovigilance. Objective: This study describes how to identify ADR-related information from Chinese ADE reports. Methods: Our study established an efficient automated tool, named BBC-Radical. BBC-Radical is a model that consists of 3 components: Bidirectional Encoder Representations from Transformers (BERT), bidirectional long short-term memory (bi-LSTM), and conditional random field (CRF). The model identifies ADR-related information from Chinese ADR reports. Token features and radical features of Chinese characters were used to represent the common meaning of a group of words. BERT and Bi-LSTM-CRF were novel models that combined these features to conduct named entity recognition (NER) tasks in the free-text section of 24,890 ADR reports from the Jiangsu Province Adverse Drug Reaction Monitoring Center from 2010 to 2016. Moreover, the man-machine comparison experiment on the ADE records from Drum Tower Hospital was designed to compare the NER performance between the BBC-Radical model and a manual method. Results: The NER model achieved relatively high performance, with a precision of 96.4%, recall of 96.0%, and F1 score of 96.2%. This indicates that the performance of the BBC-Radical model (precision 87.2%, recall 85.7%, and F1 score 86.4%) is much better than that of the manual method (precision 86.1%, recall 73.8%, and F1 score 79.5%) in the recognition task of each kind of entity. Conclusions: The proposed model was competitive in extracting ADR-related information from ADE reports, and the results suggest that the application of our method to extract ADR-related information is of great significance in improving the quality of ADR reports and postmarketing drug safety evaluation. %M 34855616 %R 10.2196/26407 %U https://medinform.jmir.org/2021/12/e26407 %U https://doi.org/10.2196/26407 %U http://www.ncbi.nlm.nih.gov/pubmed/34855616 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e32507 %T Assessing the Performance of a New Artificial Intelligence–Driven Diagnostic Support Tool Using Medical Board Exam Simulations: Clinical Vignette Study %A Ben-Shabat,Niv %A Sloma,Ariel %A Weizman,Tomer %A Kiderman,David %A Amital,Howard %+ Department of Medicine ‘B’, Sheba Medical Center, Sheba Road 2, Ramat Gan, 52621, Israel, 972 3 530 2652, nivben7@gmail.com %K diagnostic decision support systems %K diagnostic support %K medical decision-making %K medical informatics %K artificial intelligence %K Kahun %K decision support %D 2021 %7 30.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Diagnostic decision support systems (DDSS) are computer programs aimed to improve health care by supporting clinicians in the process of diagnostic decision-making. Previous studies on DDSS demonstrated their ability to enhance clinicians’ diagnostic skills, prevent diagnostic errors, and reduce hospitalization costs. Despite the potential benefits, their utilization in clinical practice is limited, emphasizing the need for new and improved products. Objective: The aim of this study was to conduct a preliminary analysis of the diagnostic performance of “Kahun,” a new artificial intelligence-driven diagnostic tool. Methods: Diagnostic performance was evaluated based on the program’s ability to “solve” clinical cases from the United States Medical Licensing Examination Step 2 Clinical Skills board exam simulations that were drawn from the case banks of 3 leading preparation companies. Each case included 3 expected differential diagnoses. The cases were entered into the Kahun platform by 3 blinded junior physicians. For each case, the presence and the rank of the correct diagnoses within the generated differential diagnoses list were recorded. Each diagnostic performance was measured in two ways: first, as diagnostic sensitivity, and second, as case-specific success rates that represent diagnostic comprehensiveness. Results: The study included 91 clinical cases with 78 different chief complaints and a mean number of 38 (SD 8) findings for each case. The total number of expected diagnoses was 272, of which 174 were different (some appeared more than once). Of the 272 expected diagnoses, 231 (87.5%; 95% CI 76-99) diagnoses were suggested within the top 20 listed diagnoses, 209 (76.8%; 95% CI 66-87) were suggested within the top 10, and 168 (61.8%; 95% CI 52-71) within the top 5. The median rank of correct diagnoses was 3 (IQR 2-6). Of the 91 expected diagnoses, 62 (68%; 95% CI 59-78) of the cases were suggested within the top 20 listed diagnoses, 44 (48%; 95% CI 38-59) within the top 10, and 24 (26%; 95% CI 17-35) within the top 5. Of the 91 expected diagnoses, in 87 (96%; 95% CI 91-100), at least 2 out of 3 of the cases’ expected diagnoses were suggested within the top 20 listed diagnoses; 78 (86%; 95% CI 79-93) were suggested within the top 10; and 61 (67%; 95% CI 57-77) within the top 5. Conclusions: The diagnostic support tool evaluated in this study demonstrated good diagnostic accuracy and comprehensiveness; it also had the ability to manage a wide range of clinical findings. %M 34672262 %R 10.2196/32507 %U https://medinform.jmir.org/2021/11/e32507 %U https://doi.org/10.2196/32507 %U http://www.ncbi.nlm.nih.gov/pubmed/34672262 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e32180 %T Health Professionals’ Perspectives on Electronic Medical Record Infusion and Individual Performance: Model Development and Questionnaire Survey Study %A Chen,Rai-Fu %A Hsiao,Ju-Ling %+ Department of Pharmacy, Chia-Nan University of Pharmacy and Science, Number 60, Sec 1, Erren Road, Rende District, Tainan City, 71710, Taiwan, 886 6 2664911 ext 5106, mayo5012@gmail.com %K health care professional %K electronic medical records %K IS infusion %K individual performance %K EHR %K electronic health record %K performance %K perspective %K information system %K integration %K decision-making %K health information exchange %K questionnaire %D 2021 %7 30.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Electronic medical records (EMRs) are integrated information sources generated by health care professionals (HCPs) from various health care information systems. EMRs play crucial roles in improving the quality of care and medical decision-making and in facilitating cross-hospital health information exchange. Although many hospitals have invested considerable resources and efforts to develop EMRs for several years, the factors affecting the long-term success of EMRs, particularly in the EMR infusion stage, remain unclear. Objective: The aim of this study was to investigate the effects of technology, user, and task characteristics on EMR infusion to determine the factors that largely affect EMR infusion. In addition, we examined the effect of EMR infusion on individual HCP performance. Methods: A questionnaire survey was used to collect data from HCPs with >6 months experience in using EMRs in a Taiwanese teaching hospital. A total of 316 questionnaires were distributed and 211 complete copies were returned, yielding a valid response rate of 66.8%. The collected data were further analyzed using WarpPLS 5.0. Results: EMR infusion (R2=0.771) was mainly affected by user habits (β=.411), portability (β=.217), personal innovativeness (β=.198), technostress (β=.169), and time criticality (β=.168), and individual performance (R2=0.541) was affected by EMR infusion (β=.735). This finding indicated that user (habit, personal innovativeness, and technostress), technology (portability), and task (mobility and time criticality) characteristics have major effects on EMR infusion. Furthermore, the results indicated that EMR infusion positively affects individual performance. Conclusions: The factors identified in this study can extend information systems infusion theory and provide useful insights for the further improvement of EMR development in hospitals and by the government, specifically in its infusion stage. In addition, the developed instrument can be used as an assessment tool to identify the key factors for EMR infusion, and to evaluate the extent of EMR infusion and the individual performance of hospitals that have implemented EMR systems. Moreover, the results can help governments to understand the urgent needs of hospitals in implementing EMR systems, provide sufficient resources and support to improve the incentives of EMR development, and develop adequate EMR policies for the meaningful use of electronic health records among hospitals and clinics. %M 34851297 %R 10.2196/32180 %U https://medinform.jmir.org/2021/11/e32180 %U https://doi.org/10.2196/32180 %U http://www.ncbi.nlm.nih.gov/pubmed/34851297 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 11 %P e26123 %T Examination of a Canada-Wide Collaboration Platform for Order Sets: Retrospective Analysis %A Javidan,Arshia Pedram %A Brand,Allan %A Cameron,Andrew %A D'Ovidio,Tommaso %A Persaud,Martin %A Lewis,Kirsten %A O'Connor,Chris %+ Division of Vascular Surgery, Department of Surgery, University of Toronto, Toronto General Hospital, 200 Elizabeth St, Toronto, ON, M5G 2C4, Canada, 1 416 340 3131, arshia.javidan@mail.utoronto.ca %K evidence-based medicine %K health informatics %K knowledge translation %K order sets %K Web 2.0 %D 2021 %7 29.11.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Knowledge translation and dissemination are some of the main challenges that affect evidence-based medicine. Web 2.0 platforms promote the sharing and collaborative development of content. Executable knowledge tools, such as order sets, are a knowledge translation tool whose localization is critical to its effectiveness but a challenge for organizations to develop independently. Objective: This paper describes a Web 2.0 resource, referred to as the collaborative network (TCN), for order set development designed to share executable knowledge (order sets). This paper also analyzes the scope of its use, describes its use through network analysis, and examines the provision and use of order sets in the platform by organizational size. Methods: Data were collected from Think Research’s TxConnect platform. We measured interorganization sharing across Canadian hospitals using descriptive statistics. A weighted chi-square analysis was used to evaluate institutional size to share volumes based on institution size, with post hoc Cramer V score to measure the strength of association. Results: TCN consisted of 12,495 order sets across 683 diagnoses or processes. Between January 2010 and March 2015, a total of 131 health care organizations representing 360 hospitals in Canada downloaded order sets 105,496 times. Order sets related to acute coronary syndrome, analgesia, and venous thromboembolism were most commonly shared. COVID-19 order sets were among the most actively shared, adjusting for order set lifetime. A weighted chi-square analysis showed nonrandom downloading behavior (P<.001), with medium-sized institutions downloading content from larger institutions acting as the most significant driver of this variance (chi-gram=124.70). Conclusions: In this paper, we have described and analyzed a Web 2.0 platform for the sharing of order set content with significant network activity. The robust use of TCN to access customized order sets reflects its value as a resource for health care organizations when they develop or update their own order sets. %M 34847055 %R 10.2196/26123 %U https://www.jmir.org/2021/11/e26123 %U https://doi.org/10.2196/26123 %U http://www.ncbi.nlm.nih.gov/pubmed/34847055 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 11 %P e32900 %T Developing the Total Health Profile, a Generalizable Unified Set of Multimorbidity Risk Scores Derived From Machine Learning for Broad Patient Populations: Retrospective Cohort Study %A Mahajan,Abhishaike %A Deonarine,Andrew %A Bernal,Axel %A Lyons,Genevieve %A Norgeot,Beau %+ Anthem Inc, 661 Bryant St, Palo Alto, CA, 94301, United States, 1 650 272 7314, beaunorgeot@gmail.com %K multimorbidity %K clinical risk score %K outcome research %K machine learning %K electronic health record %K clinical informatics %K morbidity %K risk %K outcome %K population data %K diagnostic %K demographic %K decision making %K cohort %K prediction %D 2021 %7 26.11.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Multimorbidity clinical risk scores allow clinicians to quickly assess their patients' health for decision making, often for recommendation to care management programs. However, these scores are limited by several issues: existing multimorbidity scores (1) are generally limited to one data group (eg, diagnoses, labs) and may be missing vital information, (2) are usually limited to specific demographic groups (eg, age), and (3) do not formally provide any granularity in the form of more nuanced multimorbidity risk scores to direct clinician attention. Objective: Using diagnosis, lab, prescription, procedure, and demographic data from electronic health records (EHRs), we developed a physiologically diverse and generalizable set of multimorbidity risk scores. Methods: Using EHR data from a nationwide cohort of patients, we developed the total health profile, a set of six integrated risk scores reflecting five distinct organ systems and overall health. We selected the occurrence of an inpatient hospital visitation over a 2-year follow-up window, attributable to specific organ systems, as our risk endpoint. Using a physician-curated set of features, we trained six machine learning models on 794,294 patients to predict the calibrated probability of the aforementioned endpoint, producing risk scores for heart, lung, neuro, kidney, and digestive functions and a sixth score for combined risk. We evaluated the scores using a held-out test cohort of 198,574 patients. Results: Study patients closely matched national census averages, with a median age of 41 years, a median income of $66,829, and racial averages by zip code of 73.8% White, 5.9% Asian, and 11.9% African American. All models were well calibrated and demonstrated strong performance with areas under the receiver operating curve (AUROCs) of 0.83 for the total health score (THS), 0.89 for heart, 0.86 for lung, 0.84 for neuro, 0.90 for kidney, and 0.83 for digestive functions. There was consistent performance of this scoring system across sexes, diverse patient ages, and zip code income levels. Each model learned to generate predictions by focusing on appropriate clinically relevant patient features, such as heart-related hospitalizations and chronic hypertension diagnosis for the heart model. The THS outperformed the other commonly used multimorbidity scoring systems, specifically the Charlson Comorbidity Index (CCI) and the Elixhauser Comorbidity Index (ECI) overall (AUROCs: THS=0.823, CCI=0.735, ECI=0.649) as well as for every age, sex, and income bracket. Performance improvements were most pronounced for middle-aged and lower-income subgroups. Ablation tests using only diagnosis, prescription, social determinants of health, and lab feature groups, while retaining procedure-related features, showed that the combination of feature groups has the best predictive performance, though only marginally better than the diagnosis-only model on at-risk groups. Conclusions: Massive retrospective EHR data sets have made it possible to use machine learning to build practical multimorbidity risk scores that are highly predictive, personalizable, intuitive to explain, and generalizable across diverse patient populations. %M 34842542 %R 10.2196/32900 %U https://www.jmir.org/2021/11/e32900 %U https://doi.org/10.2196/32900 %U http://www.ncbi.nlm.nih.gov/pubmed/34842542 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e23101 %T Incorporating Domain Knowledge Into Language Models by Using Graph Convolutional Networks for Assessing Semantic Textual Similarity: Model Development and Performance Comparison %A Chang,David %A Lin,Eric %A Brandt,Cynthia %A Taylor,Richard Andrew %+ Yale Center for Medical Informatics, Yale University, Suite 501, 300 George St, New Haven, CT, United States, 1 2037854058, richard.taylor@yale.edu %K natural language processing %K graph neural networks %K National NLP Clinical Challenges %K bidirectional encoder representation from transformers %D 2021 %7 26.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Although electronic health record systems have facilitated clinical documentation in health care, they have also introduced new challenges, such as the proliferation of redundant information through the use of copy and paste commands or templates. One approach to trimming down bloated clinical documentation and improving clinical summarization is to identify highly similar text snippets with the goal of removing such text. Objective: We developed a natural language processing system for the task of assessing clinical semantic textual similarity. The system assigns scores to pairs of clinical text snippets based on their clinical semantic similarity. Methods: We leveraged recent advances in natural language processing and graph representation learning to create a model that combines linguistic and domain knowledge information from the MedSTS data set to assess clinical semantic textual similarity. We used bidirectional encoder representation from transformers (BERT)–based models as text encoders for the sentence pairs in the data set and graph convolutional networks (GCNs) as graph encoders for corresponding concept graphs that were constructed based on the sentences. We also explored techniques, including data augmentation, ensembling, and knowledge distillation, to improve the model’s performance, as measured by the Pearson correlation coefficient (r). Results: Fine-tuning the BERT_base and ClinicalBERT models on the MedSTS data set provided a strong baseline (Pearson correlation coefficients: 0.842 and 0.848, respectively) compared to those of the previous year’s submissions. Our data augmentation techniques yielded moderate gains in performance, and adding a GCN-based graph encoder to incorporate the concept graphs also boosted performance, especially when the node features were initialized with pretrained knowledge graph embeddings of the concepts (r=0.868). As expected, ensembling improved performance, and performing multisource ensembling by using different language model variants, conducting knowledge distillation with the multisource ensemble model, and taking a final ensemble of the distilled models further improved the system’s performance (Pearson correlation coefficients: 0.875, 0.878, and 0.882, respectively). Conclusions: This study presents a system for the MedSTS clinical semantic textual similarity benchmark task, which was created by combining BERT-based text encoders and GCN-based graph encoders in order to incorporate domain knowledge into the natural language processing pipeline. We also experimented with other techniques involving data augmentation, pretrained concept embeddings, ensembling, and knowledge distillation to further increase our system’s performance. Although the task and its benchmark data set are in the early stages of development, this study, as well as the results of the competition, demonstrates the potential of modern language model–based systems to detect redundant information in clinical notes. %M 34842531 %R 10.2196/23101 %U https://medinform.jmir.org/2021/11/e23101 %U https://doi.org/10.2196/23101 %U http://www.ncbi.nlm.nih.gov/pubmed/34842531 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 4 %P e31214 %T Stakeholder Perspectives on an Inpatient Hypoglycemia Informatics Alert: Mixed Methods Study %A Mathioudakis,Nestoras %A Aboabdo,Moeen %A Abusamaan,Mohammed S %A Yuan,Christina %A Lewis Boyer,LaPricia %A Pilla,Scott J %A Johnson,Erica %A Desai,Sanjay %A Knight,Amy %A Greene,Peter %A Golden,Sherita H %+ Division of Endocrinology, Diabetes & Metabolism, Department of Medicine, Johns Hopkins University, 1830 E. Monument Street, Suite 333, Baltimore, MD, 21287, United States, 1 410 502 8089, nmathio1@jhmi.edu %K informatics alert %K clinical decision support %K hypoglycemia %K hospital %K inpatient %D 2021 %7 26.11.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Iatrogenic hypoglycemia is a common occurrence among hospitalized patients and is associated with poor clinical outcomes and increased mortality. Clinical decision support systems can be used to reduce the incidence of this potentially avoidable adverse event. Objective: This study aims to determine the desired features and functionality of a real-time informatics alert to prevent iatrogenic hypoglycemia in a hospital setting. Methods: Using the Agency for Healthcare Research and Quality Five Rights of Effective Clinical Decision Support Framework, we conducted a mixed methods study using an electronic survey and focus group sessions of hospital-based providers. The goal was to elicit stakeholder input to inform the future development of a real-time informatics alert to target iatrogenic hypoglycemia. In addition to perceptions about the importance of the problem and existing barriers, we sought input regarding the content, format, channel, timing, and recipient for the alert (ie, the Five Rights). Thematic analysis of focus group sessions was conducted using deductive and inductive approaches. Results: A 21-item electronic survey was completed by 102 inpatient-based providers, followed by 2 focus group sessions (6 providers per session). Respondents universally agreed or strongly agreed that inpatient iatrogenic hypoglycemia is an important problem that can be addressed with an informatics alert. Stakeholders expressed a preference for an alert that is nonintrusive, accurate, communicated in near real time to the ordering provider, and provides actionable treatment recommendations. Several electronic medical record tools, including alert indicators in the patient header, glucose management report, and laboratory results section, were deemed acceptable formats for consideration. Concerns regarding alert fatigue were prevalent among both survey respondents and focus group participants. Conclusions: The design preferences identified in this study will provide the framework needed for an informatics team to develop a prototype alert for pilot testing and evaluation. This alert will help meet the needs of hospital-based clinicians caring for patients with diabetes who are at a high risk of treatment-related hypoglycemia. %M 34842544 %R 10.2196/31214 %U https://humanfactors.jmir.org/2021/4/e31214 %U https://doi.org/10.2196/31214 %U http://www.ncbi.nlm.nih.gov/pubmed/34842544 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e26456 %T Clinical Impact of an Analytic Tool for Predicting the Fall Risk in Inpatients: Controlled Interrupted Time Series %A Cho,Insook %A Jin,In sun %A Park,Hyunchul %A Dykes,Patricia C %+ Nursing Department, College of Medicine, Inha University, 100 Inha-ro, namu-gu, Incheon, 22212, Republic of Korea, 82 01042323943, insook.cho@inha.ac.kr %K clinical effectiveness %K data analytics %K event prediction %K inpatient falls %K process metrics %D 2021 %7 25.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Patient falls are a common cause of harm in acute-care hospitals worldwide. They are a difficult, complex, and common problem requiring a great deal of nurses’ time, attention, and effort in practice. The recent rapid expansion of health care predictive analytic applications and the growing availability of electronic health record (EHR) data have resulted in the development of machine learning models that predict adverse events. However, the clinical impact of these models in terms of patient outcomes and clinicians’ responses is undetermined. Objective: The purpose of this study was to determine the impact of an electronic analytic tool for predicting fall risk on patient outcomes and nurses’ responses. Methods: A controlled interrupted time series (ITS) experiment was conducted in 12 medical-surgical nursing units at a public hospital between May 2017 and April 2019. In six of the units, the patients’ fall risk was assessed using the St. Thomas’ Risk Assessment Tool in Falling Elderly Inpatients (STRATIFY) system (control units), while in the other six, a predictive model for inpatient fall risks was implemented using routinely obtained data from the hospital’s EHR system (intervention units). The primary outcome was the rate of patient falls; secondary outcomes included the rate of falls with injury and analysis of process metrics (nursing interventions that are designed to mitigate the risk of fall). Results: During the study period, there were 42,476 admissions, of which 707 were for falls and 134 for fall injuries. Allowing for differences in the patients’ characteristics and baseline process metrics, the number of patients with falls differed between the control (n=382) and intervention (n=325) units. The mean fall rate increased from 1.95 to 2.11 in control units and decreased from 1.92 to 1.79 in intervention units. A separate ITS analysis revealed that the immediate reduction was 29.73% in the intervention group (z=–2.06, P=.039) and 16.58% in the control group (z=–1.28, P=.20), but there was no ongoing effect. The injury rate did not differ significantly between the two groups (0.42 vs 0.31, z=1.50, P=.134). Among the process metrics, the risk-targeted interventions increased significantly over time in the intervention group. Conclusions: This early-stage clinical evaluation revealed that implementation of an analytic tool for predicting fall risk may to contribute to an awareness of fall risk, leading to positive changes in nurses’ interventions over time. Trial Registration: Clinical Research Information Service (CRIS), Republic of Korea KCT0005286; https://cris.nih.go.kr/cris/search/detailSearch.do/16984 %M 34626168 %R 10.2196/26456 %U https://medinform.jmir.org/2021/11/e26456 %U https://doi.org/10.2196/26456 %U http://www.ncbi.nlm.nih.gov/pubmed/34626168 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 11 %P e25856 %T Patients’ Perceptions Toward Human–Artificial Intelligence Interaction in Health Care: Experimental Study %A Esmaeilzadeh,Pouyan %A Mirzaei,Tala %A Dharanikota,Spurthy %+ Department of Information Systems and Business Analytics, College of Business, Florida International University, Modesto A Maidique Campus, 11200 SW 8th Street, Miami, FL, 33199, United States, 1 305 348 330, pesmaeil@fiu.edu %K AI clinical applications %K collective intelligence %K in-person examinations %K perceived benefits %K perceived risks %D 2021 %7 25.11.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: It is believed that artificial intelligence (AI) will be an integral part of health care services in the near future and will be incorporated into several aspects of clinical care such as prognosis, diagnostics, and care planning. Thus, many technology companies have invested in producing AI clinical applications. Patients are one of the most important beneficiaries who potentially interact with these technologies and applications; thus, patients’ perceptions may affect the widespread use of clinical AI. Patients should be ensured that AI clinical applications will not harm them, and that they will instead benefit from using AI technology for health care purposes. Although human-AI interaction can enhance health care outcomes, possible dimensions of concerns and risks should be addressed before its integration with routine clinical care. Objective: The main objective of this study was to examine how potential users (patients) perceive the benefits, risks, and use of AI clinical applications for their health care purposes and how their perceptions may be different if faced with three health care service encounter scenarios. Methods: We designed a 2×3 experiment that crossed a type of health condition (ie, acute or chronic) with three different types of clinical encounters between patients and physicians (ie, AI clinical applications as substituting technology, AI clinical applications as augmenting technology, and no AI as a traditional in-person visit). We used an online survey to collect data from 634 individuals in the United States. Results: The interactions between the types of health care service encounters and health conditions significantly influenced individuals’ perceptions of privacy concerns, trust issues, communication barriers, concerns about transparency in regulatory standards, liability risks, benefits, and intention to use across the six scenarios. We found no significant differences among scenarios regarding perceptions of performance risk and social biases. Conclusions: The results imply that incompatibility with instrumental, technical, ethical, or regulatory values can be a reason for rejecting AI applications in health care. Thus, there are still various risks associated with implementing AI applications in diagnostics and treatment recommendations for patients with both acute and chronic illnesses. The concerns are also evident if the AI applications are used as a recommendation system under physician experience, wisdom, and control. Prior to the widespread rollout of AI, more studies are needed to identify the challenges that may raise concerns for implementing and using AI applications. This study could provide researchers and managers with critical insights into the determinants of individuals’ intention to use AI clinical applications. Regulatory agencies should establish normative standards and evaluation guidelines for implementing AI in health care in cooperation with health care institutions. Regular audits and ongoing monitoring and reporting systems can be used to continuously evaluate the safety, quality, transparency, and ethical factors of AI clinical applications. %M 34842535 %R 10.2196/25856 %U https://www.jmir.org/2021/11/e25856 %U https://doi.org/10.2196/25856 %U http://www.ncbi.nlm.nih.gov/pubmed/34842535 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e31442 %T Assessing the Value of Unsupervised Clustering in Predicting Persistent High Health Care Utilizers: Retrospective Analysis of Insurance Claims Data %A Ramachandran,Raghav %A McShea,Michael J %A Howson,Stephanie N %A Burkom,Howard S %A Chang,Hsien-Yen %A Weiner,Jonathan P %A Kharrazi,Hadi %+ Center for Population Health Information Technology, Department of Health Policy and Management, Johns Hopkins School of Public Health, , Baltimore, MD, United States, 1 4432878264, kharrazi@jhu.edu %K persistent high users %K persistent high utilizers %K latent class analysis %K comorbidity patterns %K utilization prediction %K unsupervised clustering %K population health analytics %K health care %K prediction models %K health care services %K health care costs %D 2021 %7 25.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: A high proportion of health care services are persistently utilized by a small subpopulation of patients. To improve clinical outcomes while reducing costs and utilization, population health management programs often provide targeted interventions to patients who may become persistent high users/utilizers (PHUs). Enhanced prediction and management of PHUs can improve health care system efficiencies and improve the overall quality of patient care. Objective: The aim of this study was to detect key classes of diseases and medications among the study population and to assess the predictive value of these classes in identifying PHUs. Methods: This study was a retrospective analysis of insurance claims data of patients from the Johns Hopkins Health Care system. We defined a PHU as a patient incurring health care costs in the top 20% of all patients’ costs for 4 consecutive 6-month periods. We used 2013 claims data to predict PHU status in 2014-2015. We applied latent class analysis (LCA), an unsupervised clustering approach, to identify patient subgroups with similar diagnostic and medication patterns to differentiate variations in health care utilization across PHUs. Logistic regression models were then built to predict PHUs in the full population and in select subpopulations. Predictors included LCA membership probabilities, demographic covariates, and health utilization covariates. Predictive powers of the regression models were assessed and compared using standard metrics. Results: We identified 164,221 patients with continuous enrollment between 2013 and 2015. The mean study population age was 19.7 years, 55.9% were women, 3.3% had ≥1 hospitalization, and 19.1% had 10+ outpatient visits in 2013. A total of 8359 (5.09%) patients were identified as PHUs in both 2014 and 2015. The LCA performed optimally when assigning patients to four probability disease/medication classes. Given the feedback provided by clinical experts, we further divided the population into four diagnostic groups for sensitivity analysis: acute upper respiratory infection (URI) (n=53,232; 4.6% PHUs), mental health (n=34,456; 12.8% PHUs), otitis media (n=24,992; 4.5% PHUs), and musculoskeletal (n=24,799; 15.5% PHUs). For the regression models predicting PHUs in the full population, the F1-score classification metric was lower using a parsimonious model that included LCA categories (F1=38.62%) compared to that of a complex risk stratification model with a full set of predictors (F1=48.20%). However, the LCA-enabled simple models were comparable to the complex model when predicting PHUs in the mental health and musculoskeletal subpopulations (F1-scores of 48.69% and 48.15%, respectively). F1-scores were lower than that of the complex model when the LCA-enabled models were limited to the otitis media and acute URI subpopulations (45.77% and 43.05%, respectively). Conclusions: Our study illustrates the value of LCA in identifying subgroups of patients with similar patterns of diagnoses and medications. Our results show that LCA-derived classes can simplify predictive models of PHUs without compromising predictive accuracy. Future studies should investigate the value of LCA-derived classes for predicting PHUs in other health care settings. %M 34592712 %R 10.2196/31442 %U https://medinform.jmir.org/2021/11/e31442 %U https://doi.org/10.2196/31442 %U http://www.ncbi.nlm.nih.gov/pubmed/34592712 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 4 %P e22325 %T Including the Reason for Use on Prescriptions Sent to Pharmacists: Scoping Review %A Mercer,Kathryn %A Carter,Caitlin %A Burns,Catherine %A Tennant,Ryan %A Guirguis,Lisa %A Grindrod,Kelly %+ Library, University of Waterloo, 200 University Avenue West, DC 1555, Waterloo, ON, N2L 3G1, Canada, 1 519 888 4567 ext 42659, kmercer@uwaterloo.ca %K patient safety %K human factors %K patient engagement %K multidisciplinary %D 2021 %7 25.11.2021 %9 Review %J JMIR Hum Factors %G English %X Background: In North America, although pharmacists are obligated to ensure prescribed medications are appropriate, information about a patient’s reason for use is not a required component of a legal prescription. The benefits of prescribers including the reason for use on prescriptions is evident in the current literature. However, it is not standard practice to share this information with pharmacists. Objective: Our aim was to characterize the research on how including the reason for use on a prescription impacts pharmacists. Methods: We performed an interdisciplinary scoping review, searching literature in the fields of health care, informatics, and engineering. The following databases were searched between December 2018 and January 2019: PubMed, Institute of Electrical and Electronics Engineers (IEEE), Association for Computing Machinery (ACM), International Pharmaceutical Abstracts (IPA), and EMBASE. Results: A total of 3912 potentially relevant articles were identified, with 9 papers meeting the inclusion criteria. The studies used different terminology (eg, indication, reason for use) and a wide variety of study methodologies, including prospective and retrospective observational studies, randomized controlled trials, and qualitative interviews and focus groups. The results suggest that including the reason for use on a prescription can help the pharmacist catch more errors, reduce the need to contact prescribers, support patient counseling, impact communication, and improve patient safety. Reasons that may prevent prescribers from adding the reason for use information are concerns about workflow and patient privacy. Conclusions: More research is needed to understand how the reason for use information should be provided to pharmacists. In the limited literature to date, there is a consensus that the addition of this information to prescriptions benefits patient safety and enables pharmacists to be more effective. Future research should use an implementation science or theory-based approach to improve prescriber buy-in and, consequently, adoption. %M 34842545 %R 10.2196/22325 %U https://humanfactors.jmir.org/2021/4/e22325 %U https://doi.org/10.2196/22325 %U http://www.ncbi.nlm.nih.gov/pubmed/34842545 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 11 %P e28854 %T Analysis of a Web-Based Dashboard to Support the Use of National Audit Data in Quality Improvement: Realist Evaluation %A Alvarado,Natasha %A McVey,Lynn %A Elshehaly,Mai %A Greenhalgh,Joanne %A Dowding,Dawn %A Ruddle,Roy %A Gale,Chris P %A Mamas,Mamas %A Doherty,Patrick %A West,Robert %A Feltbower,Richard %A Randell,Rebecca %+ Faculty of Health Studies, University of Bradford, Great Horton Road, Bradford, BD7 1DP, United Kingdom, 44 07715433565, n.alvarado@bradford.ac.uk %K data %K QualDash %K audit %K dashboards %K support %K quality %D 2021 %7 23.11.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Dashboards can support data-driven quality improvements in health care. They visualize data in ways intended to ease cognitive load and support data comprehension, but how they are best integrated into working practices needs further investigation. Objective: This paper reports the findings of a realist evaluation of a web-based quality dashboard (QualDash) developed to support the use of national audit data in quality improvement. Methods: QualDash was co-designed with data users and installed in 8 clinical services (3 pediatric intensive care units and 5 cardiology services) across 5 health care organizations (sites A-E) in England between July and December 2019. Champions were identified to support adoption. Data to evaluate QualDash were collected between July 2019 and August 2021 and consisted of 148.5 hours of observations including hospital wards and clinical governance meetings, log files that captured the extent of use of QualDash over 12 months, and a questionnaire designed to assess the dashboard’s perceived usefulness and ease of use. Guided by the principles of realist evaluation, data were analyzed to understand how, why, and in what circumstances QualDash supported the use of national audit data in quality improvement. Results: The observations revealed that variation across sites in the amount and type of resources available to support data use, alongside staff interactions with QualDash, shaped its use and impact. Sites resourced with skilled audit support staff and established reporting systems (sites A and C) continued to use existing processes to report data. A number of constraints influenced use of QualDash in these sites including that some dashboard metrics were not configured in line with user expectations and staff were not fully aware how QualDash could be used to facilitate their work. In less well-resourced services, QualDash automated parts of their reporting process, streamlining the work of audit support staff (site B), and, in some cases, highlighted issues with data completeness that the service worked to address (site E). Questionnaire responses received from 23 participants indicated that QualDash was perceived as useful and easy to use despite its variable use in practice. Conclusions: Web-based dashboards have the potential to support data-driven improvement, providing access to visualizations that can help users address key questions about care quality. Findings from this study point to ways in which dashboard design might be improved to optimize use and impact in different contexts; this includes using data meaningful to stakeholders in the co-design process and actively engaging staff knowledgeable about current data use and routines in the scrutiny of the dashboard metrics and functions. In addition, consideration should be given to the processes of data collection and upload that underpin the quality of the data visualized and consequently its potential to stimulate quality improvement. International Registered Report Identifier (IRRID): RR2-10.1136/bmjopen-2019-033208 %M 34817384 %R 10.2196/28854 %U https://www.jmir.org/2021/11/e28854 %U https://doi.org/10.2196/28854 %U http://www.ncbi.nlm.nih.gov/pubmed/34817384 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e29532 %T Introduction of Systematized Nomenclature of Medicine–Clinical Terms Coding Into an Electronic Health Record and Evaluation of its Impact: Qualitative and Quantitative Study %A Pankhurst,Tanya %A Evison,Felicity %A Atia,Jolene %A Gallier,Suzy %A Coleman,Jamie %A Ball,Simon %A McKee,Deborah %A Ryan,Steven %A Black,Ruth %+ NHS Foundation Trust, University Hospitals Birmingham, Mindelsohn Way, Birmingham, B15 2TG, United Kingdom, 44 7811357984, pankhurst.tanya@gmail.com %K coding standards %K clinical decision support %K Clinician led design %K clinician reported experience %K clinical usability %K data sharing %K diagnoses %K electronic health records %K electronic health record standards %K health data exchange %K health data research %K International Classification of Diseases version 10 (ICD-10) %K National Health Service Blueprint %K patient diagnoses %K population health %K problem list %K research %K Systematized Nomenclature Of Medicine–Clinical Terms (SNOMED-CT) %K use of electronic health data %K user-led design %D 2021 %7 23.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: This study describes the conversion within an existing electronic health record (EHR) from the International Classification of Diseases, Tenth Revision coding system to the SNOMED-CT (Systematized Nomenclature of Medicine–Clinical Terms) for the collection of patient histories and diagnoses. The setting is a large acute hospital that is designing and building its own EHR. Well-designed EHRs create opportunities for continuous data collection, which can be used in clinical decision support rules to drive patient safety. Collected data can be exchanged across health care systems to support patients in all health care settings. Data can be used for research to prevent diseases and protect future populations. Objective: The aim of this study was to migrate a current EHR, with all relevant patient data, to the SNOMED-CT coding system to optimize clinical use and clinical decision support, facilitate data sharing across organizational boundaries for national programs, and enable remodeling of medical pathways. Methods: The study used qualitative and quantitative data to understand the successes and gaps in the project, clinician attitudes toward the new tool, and the future use of the tool. Results: The new coding system (tool) was well received and immediately widely used in all specialties. This resulted in increased, accurate, and clinically relevant data collection. Clinicians appreciated the increased depth and detail of the new coding, welcomed the potential for both data sharing and research, and provided extensive feedback for further development. Conclusions: Successful implementation of the new system aligned the University Hospitals Birmingham NHS Foundation Trust with national strategy and can be used as a blueprint for similar projects in other health care settings. %M 34817387 %R 10.2196/29532 %U https://medinform.jmir.org/2021/11/e29532 %U https://doi.org/10.2196/29532 %U http://www.ncbi.nlm.nih.gov/pubmed/34817387 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 9 %N 11 %P e33012 %T Physicians’ Attitudes Toward Prescribable mHealth Apps and Implications for Adoption in Germany: Mixed Methods Study %A Dahlhausen,Florian %A Zinner,Maximillian %A Bieske,Linn %A Ehlers,Jan P %A Boehme,Philip %A Fehring,Leonard %+ Faculty of Health, School of Medicine, Witten/Herdecke University, Alfred-Herrhausen-Strasse 50, Witten, 58448, Germany, 49 2302 926 78608, leonard.fehring@uni-wh.de %K mobile health %K mHealth %K digital health %K apps %K physicians %K general practitioners %K technology acceptance %K adoption %D 2021 %7 23.11.2021 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: In October 2020, Germany became the first country, worldwide, to approve certain mobile health (mHealth) apps, referred to as DiGA (Digitale Gesundheitsanwendungen, in German, meaning digital health applications), for prescription with costs covered by standard statutory health insurance. Yet, this option has only been used to a limited extent so far. Objective: The aim of this study was to investigate physicians’ and psychotherapists’ current attitudes toward mHealth apps, barriers to adoption, and potential remedies. Methods: We conducted a two-stage sequential mixed methods study. In phase one, semistructured interviews were conducted with physicians and psychotherapists for questionnaire design. In phase two, an online survey was conducted among general practitioners, physicians, and psychotherapists. Results: A total of 1308 survey responses by mostly outpatient-care general practitioners, physicians, and psychotherapists from across Germany who could prescribe DiGA were recorded, making this the largest study on mHealth prescriptions to date. A total of 62.1% (807/1299) of respondents supported the opportunity to prescribe DiGA. Improved adherence (997/1294, 77.0%), health literacy (842/1294, 65.1%), and disease management (783/1294, 60.5%) were most frequently seen as benefits of DiGA. However, only 30.3% (393/1299) of respondents planned to prescribe DiGA, varying greatly by medical specialty. Professionals are still facing substantial barriers, such as insufficient information (1135/1295, 87.6%), reimbursement for DiGA-related medical services (716/1299, 55.1%), medical evidence (712/1298, 54.9%), legal uncertainties (680/1299, 52.3%), and technological uncertainties (658/1299, 50.7%). To support professionals who are unsure of prescribing DiGA, extended information campaigns (1104/1297, 85.1%) as well as recommendations from medical associations (1041/1297, 80.3%) and medical colleagues (1024/1297, 79.0%) were seen as the most impactful remedies. Conclusions: To realize the benefits from DiGA through increased adoption, additional information sharing about DiGA from trusted bodies, reimbursement for DiGA-related medical services, and further medical evidence are recommended. %M 34817385 %R 10.2196/33012 %U https://mhealth.jmir.org/2021/11/e33012 %U https://doi.org/10.2196/33012 %U http://www.ncbi.nlm.nih.gov/pubmed/34817385 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 10 %N 11 %P e31750 %T Approaches and Criteria for Provenance in Biomedical Data Sets and Workflows: Protocol for a Scoping Review %A Gierend,Kerstin %A Krüger,Frank %A Waltemath,Dagmar %A Fünfgeld,Maximilian %A Ganslandt,Thomas %A Zeleke,Atinkut Alamirrew %+ Department of Biomedical Informatics at the Center for Preventive Medicine and Digital Health, Medical Faculty Mannheim, Heidelberg University, Theodor-Kutzer-Ufer 1-3, Mannheim, 68167, Germany, 49 0621 383 ext 8087, kerstin.gierend@medma.uni-heidelberg.de %K provenance %K biomedical %K workflow %K data sharing %K lineage %K scoping review %K data genesis %K scientific data %K digital objects %K healthcare data %D 2021 %7 22.11.2021 %9 Protocol %J JMIR Res Protoc %G English %X Background: Provenance supports the understanding of data genesis, and it is a key factor to ensure the trustworthiness of digital objects containing (sensitive) scientific data. Provenance information contributes to a better understanding of scientific results and fosters collaboration on existing data as well as data sharing. This encompasses defining comprehensive concepts and standards for transparency and traceability, reproducibility, validity, and quality assurance during clinical and scientific data workflows and research. Objective: The aim of this scoping review is to investigate existing evidence regarding approaches and criteria for provenance tracking as well as disclosing current knowledge gaps in the biomedical domain. This review covers modeling aspects as well as metadata frameworks for meaningful and usable provenance information during creation, collection, and processing of (sensitive) scientific biomedical data. This review also covers the examination of quality aspects of provenance criteria. Methods: This scoping review will follow the methodological framework by Arksey and O'Malley. Relevant publications will be obtained by querying PubMed and Web of Science. All papers in English language will be included, published between January 1, 2006 and March 23, 2021. Data retrieval will be accompanied by manual search for grey literature. Potential publications will then be exported into a reference management software, and duplicates will be removed. Afterwards, the obtained set of papers will be transferred into a systematic review management tool. All publications will be screened, extracted, and analyzed: title and abstract screening will be carried out by 4 independent reviewers. Majority vote is required for consent to eligibility of papers based on the defined inclusion and exclusion criteria. Full-text reading will be performed independently by 2 reviewers and in the last step, key information will be extracted on a pretested template. If agreement cannot be reached, the conflict will be resolved by a domain expert. Charted data will be analyzed by categorizing and summarizing the individual data items based on the research questions. Tabular or graphical overviews will be given, if applicable. Results: The reporting follows the extension of the Preferred Reporting Items for Systematic reviews and Meta-Analyses statements for Scoping Reviews. Electronic database searches in PubMed and Web of Science resulted in 469 matches after deduplication. As of September 2021, the scoping review is in the full-text screening stage. The data extraction using the pretested charting template will follow the full-text screening stage. We expect the scoping review report to be completed by February 2022. Conclusions: Information about the origin of healthcare data has a major impact on the quality and the reusability of scientific results as well as follow-up activities. This protocol outlines plans for a scoping review that will provide information about current approaches, challenges, or knowledge gaps with provenance tracking in biomedical sciences. International Registered Report Identifier (IRRID): DERR1-10.2196/31750 %M 34813494 %R 10.2196/31750 %U https://www.researchprotocols.org/2021/11/e31750 %U https://doi.org/10.2196/31750 %U http://www.ncbi.nlm.nih.gov/pubmed/34813494 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e30079 %T Prediction Model of Osteonecrosis of the Femoral Head After Femoral Neck Fracture: Machine Learning–Based Development and Validation Study %A Wang,Huan %A Wu,Wei %A Han,Chunxia %A Zheng,Jiaqi %A Cai,Xinyu %A Chang,Shimin %A Shi,Junlong %A Xu,Nan %A Ai,Zisheng %+ Department of Medical Statistics, Tongji University School of Medicine, No. 1239 Singping Road, Shanghai, 200092, China, 86 1 377 438 0743, azs1966@126.com %K femoral neck fracture %K osteonecrosis of the femoral head %K machine learning %K interpretability %D 2021 %7 19.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: The absolute number of femoral neck fractures (FNFs) is increasing; however, the prediction of traumatic femoral head necrosis remains difficult. Machine learning algorithms have the potential to be superior to traditional prediction methods for the prediction of traumatic femoral head necrosis. Objective: The aim of this study is to use machine learning to construct a model for the analysis of risk factors and prediction of osteonecrosis of the femoral head (ONFH) in patients with FNF after internal fixation. Methods: We retrospectively collected preoperative, intraoperative, and postoperative clinical data of patients with FNF in 4 hospitals in Shanghai and followed up the patients for more than 2.5 years. A total of 259 patients with 43 variables were included in the study. The data were randomly divided into a training set (181/259, 69.8%) and a validation set (78/259, 30.1%). External data (n=376) were obtained from a retrospective cohort study of patients with FNF in 3 other hospitals. Least absolute shrinkage and selection operator regression and the support vector machine algorithm were used for variable selection. Logistic regression, random forest, support vector machine, and eXtreme Gradient Boosting (XGBoost) were used to develop the model on the training set. The validation set was used to tune the model hyperparameters to determine the final prediction model, and the external data were used to compare and evaluate the model performance. We compared the accuracy, discrimination, and calibration of the models to identify the best machine learning algorithm for predicting ONFH. Shapley additive explanations and local interpretable model-agnostic explanations were used to determine the interpretability of the black box model. Results: A total of 11 variables were selected for the models. The XGBoost model performed best on the validation set and external data. The accuracy, sensitivity, and area under the receiver operating characteristic curve of the model on the validation set were 0.987, 0.929, and 0.992, respectively. The accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve of the model on the external data were 0.907, 0.807, 0.935, and 0.933, respectively, and the log-loss was 0.279. The calibration curve demonstrated good agreement between the predicted probability and actual risk. The interpretability of the features and individual predictions were realized using the Shapley additive explanations and local interpretable model-agnostic explanations algorithms. In addition, the XGBoost model was translated into a self-made web-based risk calculator to estimate an individual’s probability of ONFH. Conclusions: Machine learning performs well in predicting ONFH after internal fixation of FNF. The 6-variable XGBoost model predicted the risk of ONFH well and had good generalization ability on the external data, which can be used for the clinical prediction of ONFH after internal fixation of FNF. %M 34806984 %R 10.2196/30079 %U https://medinform.jmir.org/2021/11/e30079 %U https://doi.org/10.2196/30079 %U http://www.ncbi.nlm.nih.gov/pubmed/34806984 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e29176 %T An Open-Source, Standard-Compliant, and Mobile Electronic Data Capture System for Medical Research (OpenEDC): Design and Evaluation Study %A Greulich,Leonard %A Hegselmann,Stefan %A Dugas,Martin %+ Institute of Medical Informatics, University of Münster, Albert-Schweitzer-Campus 1, Building A11, Münster, 48149, Germany, 49 15905368729, leonard.greulich@uni-muenster.de %K electronic data capture %K open science %K data interoperability %K metadata reuse %K mobile health %K data standard %K mobile phone %D 2021 %7 19.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Medical research and machine learning for health care depend on high-quality data. Electronic data capture (EDC) systems have been widely adopted for metadata-driven digital data collection. However, many systems use proprietary and incompatible formats that inhibit clinical data exchange and metadata reuse. In addition, the configuration and financial requirements of typical EDC systems frequently prevent small-scale studies from benefiting from their inherent advantages. Objective: The aim of this study is to develop and publish an open-source EDC system that addresses these issues. We aim to plan a system that is applicable to a wide range of research projects. Methods: We conducted a literature-based requirements analysis to identify the academic and regulatory demands for digital data collection. After designing and implementing OpenEDC, we performed a usability evaluation to obtain feedback from users. Results: We identified 20 frequently stated requirements for EDC. According to the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) 25010 norm, we categorized the requirements into functional suitability, availability, compatibility, usability, and security. We developed OpenEDC based on the regulatory-compliant Clinical Data Interchange Standards Consortium Operational Data Model (CDISC ODM) standard. Mobile device support enables the collection of patient-reported outcomes. OpenEDC is publicly available and released under the MIT open-source license. Conclusions: Adopting an established standard without modifications supports metadata reuse and clinical data exchange, but it limits item layouts. OpenEDC is a stand-alone web app that can be used without a setup or configuration. This should foster compatibility between medical research and open science. OpenEDC is targeted at observational and translational research studies by clinicians. %M 34806987 %R 10.2196/29176 %U https://medinform.jmir.org/2021/11/e29176 %U https://doi.org/10.2196/29176 %U http://www.ncbi.nlm.nih.gov/pubmed/34806987 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e30066 %T Deep Learning Techniques for Fatty Liver Using Multi-View Ultrasound Images Scanned by Different Scanners: Development and Validation Study %A Kim,Taewoo %A Lee,Dong Hyun %A Park,Eun-Kee %A Choi,Sanghun %+ School of Mechanical Engineering, Kyungpook National University, 80 Daehak-ro, Buk-gu, Daegu, 41566, Republic of Korea, 82 53 950 5578, s-choi@knu.ac.kr %K fatty liver %K deep learning %K transfer learning %K classification %K regression %K magnetic resonance imaging–proton density fat fraction %K multi-view ultrasound images %K artificial intelligence %K machine imaging %K imaging %K informatics %K fatty liver disease %K detection %K diagnosis %D 2021 %7 18.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Fat fraction values obtained from magnetic resonance imaging (MRI) can be used to obtain an accurate diagnosis of fatty liver diseases. However, MRI is expensive and cannot be performed for everyone. Objective: In this study, we aim to develop multi-view ultrasound image–based convolutional deep learning models to detect fatty liver disease and yield fat fraction values. Methods: We extracted 90 ultrasound images of the right intercostal view and 90 ultrasound images of the right intercostal view containing the right renal cortex from 39 cases of fatty liver (MRI–proton density fat fraction [MRI–PDFF] ≥ 5%) and 51 normal subjects (MRI–PDFF < 5%), with MRI–PDFF values obtained from Good Gang-An Hospital. We obtained combined liver and kidney-liver (CLKL) images to train the deep learning models and developed classification and regression models based on the VGG19 model to classify fatty liver disease and yield fat fraction values. We employed the data augmentation techniques such as flip and rotation to prevent the deep learning model from overfitting. We determined the deep learning model with performance metrics such as accuracy, sensitivity, specificity, and coefficient of determination (R2). Results: In demographic information, all metrics such as age and sex were similar between the two groups—fatty liver disease and normal subjects. In classification, the model trained on CLKL images achieved 80.1% accuracy, 86.2% precision, and 80.5% specificity to detect fatty liver disease. In regression, the predicted fat fraction values of the regression model trained on CLKL images correlated with MRI–PDFF values (R2=0.633), indicating that the predicted fat fraction values were moderately estimated. Conclusions: With deep learning techniques and multi-view ultrasound images, it is potentially possible to replace MRI–PDFF values with deep learning predictions for detecting fatty liver disease and estimating fat fraction values. %M 34792476 %R 10.2196/30066 %U https://medinform.jmir.org/2021/11/e30066 %U https://doi.org/10.2196/30066 %U http://www.ncbi.nlm.nih.gov/pubmed/34792476 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e30743 %T Implementation of an Anticoagulation Practice Guideline for COVID-19 via a Clinical Decision Support System in a Large Academic Health System and Its Evaluation: Observational Study %A Shah,Surbhi %A Switzer,Sean %A Shippee,Nathan D %A Wogensen,Pamela %A Kosednar,Kathryn %A Jones,Emma %A Pestka,Deborah L %A Badlani,Sameer %A Butler,Mary %A Wagner,Brittin %A White,Katie %A Rhein,Joshua %A Benson,Bradley %A Reding,Mark %A Usher,Michael %A Melton,Genevieve B %A Tignanelli,Christopher James %+ Department of Surgery, University of Minnesota, 420 Delaware St SE, MMC 195, Minneapolis, MN, 55455, United States, 1 6126261968, ctignane@umn.edu %K COVID-19 %K anticoagulation %K clinical practice guideline %K evidence-based practice %K clinical decision support %K implementation science %K RE-AIM %D 2021 %7 18.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Studies evaluating strategies for the rapid development, implementation, and evaluation of clinical decision support (CDS) systems supporting guidelines for diseases with a poor knowledge base, such as COVID-19, are limited. Objective: We developed an anticoagulation clinical practice guideline (CPG) for COVID-19, which was delivered and scaled via CDS across a 12-hospital Midwest health care system. This study represents a preplanned 6-month postimplementation evaluation guided by the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework. Methods: The implementation outcomes evaluated were reach, adoption, implementation, and maintenance. To evaluate effectiveness, the association of CPG adherence on hospital admission with clinical outcomes was assessed via multivariable logistic regression and nearest neighbor propensity score matching. A time-to-event analysis was conducted. Sensitivity analyses were also conducted to evaluate the competing risk of death prior to intensive care unit (ICU) admission. The models were risk adjusted to account for age, gender, race/ethnicity, non-English speaking status, area deprivation index, month of admission, remdesivir treatment, tocilizumab treatment, steroid treatment, BMI, Elixhauser comorbidity index, oxygen saturation/fraction of inspired oxygen ratio, systolic blood pressure, respiratory rate, treating hospital, and source of admission. A preplanned subgroup analysis was also conducted in patients who had laboratory values (D-dimer, C-reactive protein, creatinine, and absolute neutrophil to absolute lymphocyte ratio) present. The primary effectiveness endpoint was the need for ICU admission within 48 hours of hospital admission. Results: A total of 2503 patients were included in this study. CDS reach approached 95% during implementation. Adherence achieved a peak of 72% during implementation. Variation was noted in adoption across sites and nursing units. Adoption was the highest at hospitals that were specifically transformed to only provide care to patients with COVID-19 (COVID-19 cohorted hospitals; 74%-82%) and the lowest in academic settings (47%-55%). CPG delivery via the CDS system was associated with improved adherence (odds ratio [OR] 1.43, 95% CI 1.2-1.7; P<.001). Adherence with the anticoagulation CPG was associated with a significant reduction in the need for ICU admission within 48 hours (OR 0.39, 95% CI 0.30-0.51; P<.001) on multivariable logistic regression analysis. Similar findings were noted following 1:1 propensity score matching for patients who received adherent versus nonadherent care (21.5% vs 34.3% incidence of ICU admission within 48 hours; log-rank test P<.001). Conclusions: Our institutional experience demonstrated that adherence with the institutional CPG delivered via the CDS system resulted in improved clinical outcomes for patients with COVID-19. CDS systems are an effective means to rapidly scale a CPG across a heterogeneous health care system. Further research is needed to investigate factors associated with adherence at low and high adopting sites and nursing units. %M 34550900 %R 10.2196/30743 %U https://medinform.jmir.org/2021/11/e30743 %U https://doi.org/10.2196/30743 %U http://www.ncbi.nlm.nih.gov/pubmed/34550900 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 11 %P e30042 %T Use of Patient-Reported Outcome Measures and Patient-Reported Experience Measures Within Evaluation Studies of Telemedicine Applications: Systematic Review %A Knapp,Andreas %A Harst,Lorenz %A Hager,Stefan %A Schmitt,Jochen %A Scheibe,Madlen %+ Center for Evidence-Based Healthcare, University Hospital Carl Gustav Carus, Carl Gustav Carus Faculty of Medicine, Technische Universität Dresden, Fetscherstrasse 74, Dresden, 01307, Germany, 49 3514585665, andreas.knapp@uniklinikum-dresden.de %K telemedicine %K telehealth %K evaluation %K outcome %K patient-reported outcome measures %K patient-reported outcome %K patient-reported experience measures %K patient-reported experience %K measurement instrument %K questionnaire %D 2021 %7 17.11.2021 %9 Review %J J Med Internet Res %G English %X Background: With the rise of digital health technologies and telemedicine, the need for evidence-based evaluation is growing. Patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) are recommended as an essential part of the evaluation of telemedicine. For the first time, a systematic review has been conducted to investigate the use of PROMs and PREMs in the evaluation studies of telemedicine covering all application types and medical purposes. Objective: This study investigates the following research questions: in which scenarios are PROMs and PREMs collected for evaluation purposes, which PROM and PREM outcome domains have been covered and how often, which outcome measurement instruments have been used and how often, does the selection and quantity of PROMs and PREMs differ between study types and application types, and has the use of PROMs and PREMs changed over time. Methods: We conducted a systematic literature search of the MEDLINE and Embase databases and included studies published from inception until April 2, 2020. We included studies evaluating telemedicine with patients as the main users; these studies reported PROMs and PREMs within randomized controlled trials, controlled trials, noncontrolled trials, and feasibility trials in English and German. Results: Of the identified 2671 studies, 303 (11.34%) were included; of the 303 studies, 67 (22.1%) were feasibility studies, 70 (23.1%) were noncontrolled trials, 20 (6.6%) were controlled trials, and 146 (48.2%) were randomized controlled trials. Health-related quality of life (n=310; mean 1.02, SD 1.05), emotional function (n=244; mean 0.81, SD 1.18), and adherence (n=103; mean 0.34, SD 0.53) were the most frequently assessed outcome domains. Self-developed PROMs were used in 21.4% (65/303) of the studies, and self-developed PREMs were used in 22.3% (68/303). PROMs (n=884) were assessed more frequently than PREMs (n=234). As the evidence level of the studies increased, the number of PROMs also increased (τ=−0.45), and the number of PREMs decreased (τ=0.35). Since 2000, not only has the number of studies using PROMs and PREMs increased, but the level of evidence and the number of outcome measurement instruments used have also increased, with the number of PREMs permanently remaining at a lower level. Conclusions: There have been increasingly more studies, particularly high-evidence studies, which use PROMs and PREMs to evaluate telemedicine. PROMs have been used more frequently than PREMs. With the increasing maturity stage of telemedicine applications and higher evidence level, the use of PROMs increased in line with the recommendations of evaluation guidelines. Health-related quality of life and emotional function were measured in almost all the studies. Simultaneously, health literacy as a precondition for using the application adequately, alongside proper training and guidance, has rarely been reported. Further efforts should be pursued to standardize PROM and PREM collection in evaluation studies of telemedicine. %M 34523604 %R 10.2196/30042 %U https://www.jmir.org/2021/11/e30042 %U https://doi.org/10.2196/30042 %U http://www.ncbi.nlm.nih.gov/pubmed/34523604 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 11 %P e25770 %T Dangers and Benefits of Social Media on E-Professionalism of Health Care Professionals: Scoping Review %A Vukušić Rukavina,Tea %A Viskić,Joško %A Machala Poplašen,Lovela %A Relić,Danko %A Marelić,Marko %A Jokic,Drazen %A Sedak,Kristijan %+ Department of Fixed Prosthodontics, School of Dental Medicine, University of Zagreb, Gundulićeva 5, Zagreb, 10000, Croatia, 385 98769245, viskic@sfzg.hr %K e-professionalism %K social media %K internet %K health care professionals %K physicians %K nurses %K students %K medicine %K dental medicine %K nursing %D 2021 %7 17.11.2021 %9 Review %J J Med Internet Res %G English %X Background: As we are witnessing the evolution of social media (SM) use worldwide among the general population, the popularity of SM has also been embraced by health care professionals (HCPs). In the context of SM evolution and exponential growth of users, this scoping review summarizes recent findings of the e-professionalism of HCPs. Objective: The purpose of this scoping review is to characterize the recent original peer-reviewed research studies published between November 1, 2014, to December 31, 2020, on e-professionalism of HCPs; to assess the quality of the methodologies and approaches used; to explore the impact of SM on e-professionalism of HCPs; to recognize the benefits and dangers of SM; and to provide insights to guide future research in this area. Methods: A search of the literature published from November 1, 2014, to December 31, 2020, was performed in January 2021 using 3 databases (PubMed, CINAHL, and Scopus). The searches were conducted using the following defined search terms: “professionalism” AND “social media” OR “social networks” OR “Internet” OR “Facebook” OR “Twitter” OR “Instagram” OR “TikTok.” The search strategy was limited to studies published in English. This scoping review follows the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines. Results: Of the 1632 retrieved papers, a total of 88 studies were finally included in this review. Overall, the quality of the studies was satisfactory. Participants in the reviewed studies were from diverse health care professions. Medical health professionals were involved in about three-quarters of the studies. Three key benefits of SM on e-professionalism of HCPs were identified: (1) professional networking and collaboration, (2) professional education and training, and (3) patient education and health promotion. For the selected studies, there were five recognized dangers of SM on e-professionalism of HCPs: (1) loosening accountability, (2) compromising confidentiality, (3) blurred professional boundaries, (4) depiction of unprofessional behavior, and (5) legal issues and disciplinary consequences. This scoping review also recognizes recommendations for changes in educational curricula regarding e-professionalism as opportunities for improvement and barriers that influence HCPs use of SM in the context of e-professionalism. Conclusions: Findings in the reviewed studies indicate the existence of both benefits and dangers of SM on e-professionalism of HCPs. Even though there are some barriers recognized, this review has highlighted existing recommendations for including e-professionalism in the educational curricula of HCPs. Based on all evidence provided, this review provided new insights and guides for future research on this area. There is a clear need for robust research to investigate new emerging SM platforms, the efficiency of guidelines and educational interventions, and the specifics of each profession regarding their SM potential and use. %M 34662284 %R 10.2196/25770 %U https://www.jmir.org/2021/11/e25770 %U https://doi.org/10.2196/25770 %U http://www.ncbi.nlm.nih.gov/pubmed/34662284 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e32662 %T Machine Learning–Based Hospital Discharge Prediction for Patients With Cardiovascular Diseases: Development and Usability Study %A Ahn,Imjin %A Gwon,Hansle %A Kang,Heejun %A Kim,Yunha %A Seo,Hyeram %A Choi,Heejung %A Cho,Ha Na %A Kim,Minkyoung %A Jun,Tae Joon %A Kim,Young-Hak %+ Division of Cardiology, Department of Internal Medicine, Asan Medical Center, University of Ulsan College of Medicine, 88, Olympic-ro 43-gil, Seoul, 05505, Republic of Korea, 82 2 3010 3955, mdyhkim@amc.seoul.kr %K electronic health records %K cardiovascular diseases %K discharge prediction %K bed management %K explainable artificial intelligence %D 2021 %7 17.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Effective resource management in hospitals can improve the quality of medical services by reducing labor-intensive burdens on staff, decreasing inpatient waiting time, and securing the optimal treatment time. The use of hospital processes requires effective bed management; a stay in the hospital that is longer than the optimal treatment time hinders bed management. Therefore, predicting a patient’s hospitalization period may support the making of judicious decisions regarding bed management. Objective: First, this study aims to develop a machine learning (ML)–based predictive model for predicting the discharge probability of inpatients with cardiovascular diseases (CVDs). Second, we aim to assess the outcome of the predictive model and explain the primary risk factors of inpatients for patient-specific care. Finally, we aim to evaluate whether our ML-based predictive model helps manage bed scheduling efficiently and detects long-term inpatients in advance to improve the use of hospital processes and enhance the quality of medical services. Methods: We set up the cohort criteria and extracted the data from CardioNet, a manually curated database that specializes in CVDs. We processed the data to create a suitable data set by reindexing the date-index, integrating the present features with past features from the previous 3 years, and imputing missing values. Subsequently, we trained the ML-based predictive models and evaluated them to find an elaborate model. Finally, we predicted the discharge probability within 3 days and explained the outcomes of the model by identifying, quantifying, and visualizing its features. Results: We experimented with 5 ML-based models using 5 cross-validations. Extreme gradient boosting, which was selected as the final model, accomplished an average area under the receiver operating characteristic curve score that was 0.865 higher than that of the other models (ie, logistic regression, random forest, support vector machine, and multilayer perceptron). Furthermore, we performed feature reduction, represented the feature importance, and assessed prediction outcomes. One of the outcomes, the individual explainer, provides a discharge score during hospitalization and a daily feature influence score to the medical team and patients. Finally, we visualized simulated bed management to use the outcomes. Conclusions: In this study, we propose an individual explainer based on an ML-based predictive model, which provides the discharge probability and relative contributions of individual features. Our model can assist medical teams and patients in identifying individual and common risk factors in CVDs and can support hospital administrators in improving the management of hospital beds and other resources. %M 34787584 %R 10.2196/32662 %U https://medinform.jmir.org/2021/11/e32662 %U https://doi.org/10.2196/32662 %U http://www.ncbi.nlm.nih.gov/pubmed/34787584 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e30432 %T The Role of Electronic Medical Records in Reducing Unwarranted Clinical Variation in Acute Health Care: Systematic Review %A Hodgson,Tobias %A Burton-Jones,Andrew %A Donovan,Raelene %A Sullivan,Clair %+ The University of Queensland Business School, The University of Queensland, 39 Blair Drive, St Lucia, 4067, Australia, 61 733468100, t.hodgson@business.uq.edu.au %K clinical variation %K unwarranted clinical variation %K electronic health record %K EHR %K electronic medical record %K EMR %K PowerPlan %K SmartSet %K acute care %K eHealth %K digital health %K health care %K health care outcomes %K outcome %K review %K standard of care %K hospital %K research %K literature %K variation %K intervention %D 2021 %7 17.11.2021 %9 Review %J JMIR Med Inform %G English %X Background: The use of electronic medical records (EMRs)/electronic health records (EHRs) provides potential to reduce unwarranted clinical variation and thereby improve patient health care outcomes. Minimization of unwarranted clinical variation may raise and refine the standard of patient care provided and satisfy the quadruple aim of health care. Objective: A systematic review of the impact of EMRs and specific subcomponents (PowerPlans/SmartSets) on variation in clinical care processes in hospital settings was undertaken to summarize the existing literature on the effects of EMRs on clinical variation and patient outcomes. Methods: Articles from January 2000 to November 2020 were identified through a comprehensive search that examined EMRs/EHRs and clinical variation or PowerPlans/SmartSets. Thirty-six articles met the inclusion criteria. Articles were examined for evidence for EMR-induced changes in variation and effects on health care outcomes and mapped to the quadruple aim of health care. Results: Most of the studies reported positive effects of EMR-related interventions (30/36, 83%). All of the 36 included studies discussed clinical variation, but only half measured it (18/36, 50%). Those studies that measured variation generally examined how changes to variation affected individual patient care (11/36, 31%) or costs (9/36, 25%), while other outcomes (population health and clinician experience) were seldom studied. High-quality study designs were rare. Conclusions: The literature provides some evidence that EMRs can help reduce unwarranted clinical variation and thereby improve health care outcomes. However, the evidence is surprisingly thin because of insufficient attention to the measurement of clinical variation, and to the chain of evidence from EMRs to variation in clinical practices to health care outcomes. %M 34787585 %R 10.2196/30432 %U https://medinform.jmir.org/2021/11/e30432 %U https://doi.org/10.2196/30432 %U http://www.ncbi.nlm.nih.gov/pubmed/34787585 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e30485 %T Assimilation of Medical Appointment Scheduling Systems and Their Impact on the Accessibility of Primary Care: Mixed Methods Study %A Paré,Guy %A Raymond,Louis %A Castonguay,Alexandre %A Grenier Ouimet,Antoine %A Trudel,Marie-Claude %+ Department of Information Technologies, HEC Montréal, 3000, chemin de la Côte-Sainte-Catherine, Montréal, QC, H3T 2A7, Canada, 1 514 340 6812, alexandre.castonguay@hec.ca %K medical appointment scheduling system %K electronic booking %K e-booking %K primary care %K accessibility of care %K availability of care %K advance access %K electronic medical record %D 2021 %7 16.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: The COVID-19 pandemic has prompted the adoption of digital health technologies to maximize the accessibility of medical care in primary care settings. Medical appointment scheduling (MAS) systems are among the most essential technologies. Prior studies on MAS systems have taken either a user-oriented perspective, focusing on perceived outcomes such as patient satisfaction, or a technical perspective, focusing on optimizing medical scheduling algorithms. Less attention has been given to the extent to which family medicine practices have assimilated these systems into their daily operations and achieved impacts. Objective: This study aimed to fill this gap and provide answers to the following questions: (1) to what extent have primary care practices assimilated MAS systems into their daily operations? (2) what are the impacts of assimilating MAS systems on the accessibility and availability of primary care? and (3) what are the organizational and managerial factors associated with greater assimilation of MAS systems in family medicine clinics? Methods: A survey study targeting all family medicine clinics in Quebec, Canada, was conducted. The questionnaire was addressed to the individual responsible for managing medical schedules and appointments at these clinics. Following basic descriptive statistics, component-based structural equation modeling was used to empirically explore the causal paths implied in the conceptual framework. A cluster analysis was also performed to complement the causal analysis. As a final step, 6 experts in MAS systems were interviewed. Qualitative data were then coded and extracted using standard content analysis methods. Results: A total of 70 valid questionnaires were collected and analyzed. A large majority of the surveyed clinics had implemented MAS systems, with an average use of 1 or 2 functionalities, mainly “automated appointment confirmation and reminders” and “online appointment confirmation, modification, or cancellation by the patient.” More extensive use of MAS systems appears to contribute to improved availability of medical care in these clinics, notwithstanding the effect of their application of advanced access principles. Also, greater integration of MAS systems into the clinic’s electronic medical record system led to more extensive use. Our study further indicated that smaller clinics were less likely to undertake such integration and therefore showed less availability of medical care for their patients. Finally, our findings indicated that those clinics that showed a greater adoption rate and that used the provincial MAS system tended to be the highest-performing ones in terms of accessibility and availability of care. Conclusions: The main contribution of this study lies in the empirical demonstration that greater integration and assimilation of MAS systems in family medicine clinics lead to greater accessibility and availability of care for their patients and the general population. Valuable insight has also been provided on how to identify the clinics that would benefit most from such digital health solutions. %M 34783670 %R 10.2196/30485 %U https://medinform.jmir.org/2021/11/e30485 %U https://doi.org/10.2196/30485 %U http://www.ncbi.nlm.nih.gov/pubmed/34783670 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 8 %N 11 %P e25455 %T Clinician Perspectives on Using Computational Mental Health Insights From Patients’ Social Media Activities: Design and Qualitative Evaluation of a Prototype %A Yoo,Dong Whi %A Ernala,Sindhu Kiranmai %A Saket,Bahador %A Weir,Domino %A Arenare,Elizabeth %A Ali,Asra F %A Van Meter,Anna R %A Birnbaum,Michael L %A Abowd,Gregory D %A De Choudhury,Munmun %+ School of Interactive Computing, Georgia Institute of Technology, 756 W Peachtree St NW, Atlanta, GA, 30308, United States, 1 4043858603, yoo@gatech.edu %K mental health %K social media %K information technology %D 2021 %7 16.11.2021 %9 Original Paper %J JMIR Ment Health %G English %X Background: Previous studies have suggested that social media data, along with machine learning algorithms, can be used to generate computational mental health insights. These computational insights have the potential to support clinician-patient communication during psychotherapy consultations. However, how clinicians perceive and envision using computational insights during consultations has been underexplored. Objective: The aim of this study is to understand clinician perspectives regarding computational mental health insights from patients’ social media activities. We focus on the opportunities and challenges of using these insights during psychotherapy consultations. Methods: We developed a prototype that can analyze consented patients’ Facebook data and visually represent these computational insights. We incorporated the insights into existing clinician-facing assessment tools, the Hamilton Depression Rating Scale and Global Functioning: Social Scale. The design intent is that a clinician will verbally interview a patient (eg, How was your mood in the past week?) while they reviewed relevant insights from the patient’s social media activities (eg, number of depression-indicative posts). Using the prototype, we conducted interviews (n=15) and 3 focus groups (n=13) with mental health clinicians: psychiatrists, clinical psychologists, and licensed clinical social workers. The transcribed qualitative data were analyzed using thematic analysis. Results: Clinicians reported that the prototype can support clinician-patient collaboration in agenda-setting, communicating symptoms, and navigating patients’ verbal reports. They suggested potential use scenarios, such as reviewing the prototype before consultations and using the prototype when patients missed their consultations. They also speculated potential negative consequences: patients may feel like they are being monitored, which may yield negative effects, and the use of the prototype may increase the workload of clinicians, which is already difficult to manage. Finally, our participants expressed concerns regarding the prototype: they were unsure whether patients’ social media accounts represented their actual behaviors; they wanted to learn how and when the machine learning algorithm can fail to meet their expectations of trust; and they were worried about situations where they could not properly respond to the insights, especially emergency situations outside of clinical settings. Conclusions: Our findings support the touted potential of computational mental health insights from patients’ social media account data, especially in the context of psychotherapy consultations. However, sociotechnical issues, such as transparent algorithmic information and institutional support, should be addressed in future endeavors to design implementable and sustainable technology. %M 34783667 %R 10.2196/25455 %U https://mental.jmir.org/2021/11/e25455 %U https://doi.org/10.2196/25455 %U http://www.ncbi.nlm.nih.gov/pubmed/34783667 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 11 %P e31337 %T Predicting COVID-19–Related Health Care Resource Utilization Across a Statewide Patient Population: Model Development Study %A Kasturi,Suranga N %A Park,Jeremy %A Wild,David %A Khan,Babar %A Haggstrom,David A %A Grannis,Shaun %+ Regenstrief Institute, 1101 W 10th St, Indianapolis, IN, 46202, United States, 1 (317) 274 9000, snkasthu@iu.edu %K COVID-19 %K machine learning %K population health %K health care utilization %K health disparities %K health information %K epidemiology %K public health %K digital health %K health data %K pandemic %K decision models %K health informatics %K healthcare resources %D 2021 %7 15.11.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: The COVID-19 pandemic has highlighted the inability of health systems to leverage existing system infrastructure in order to rapidly develop and apply broad analytical tools that could inform state- and national-level policymaking, as well as patient care delivery in hospital settings. The COVID-19 pandemic has also led to highlighted systemic disparities in health outcomes and access to care based on race or ethnicity, gender, income-level, and urban-rural divide. Although the United States seems to be recovering from the COVID-19 pandemic owing to widespread vaccination efforts and increased public awareness, there is an urgent need to address the aforementioned challenges. Objective: This study aims to inform the feasibility of leveraging broad, statewide datasets for population health–driven decision-making by developing robust analytical models that predict COVID-19–related health care resource utilization across patients served by Indiana’s statewide Health Information Exchange. Methods: We leveraged comprehensive datasets obtained from the Indiana Network for Patient Care to train decision forest-based models that can predict patient-level need of health care resource utilization. To assess these models for potential biases, we tested model performance against subpopulations stratified by age, race or ethnicity, gender, and residence (urban vs rural). Results: For model development, we identified a cohort of 96,026 patients from across 957 zip codes in Indiana, United States. We trained the decision models that predicted health care resource utilization by using approximately 100 of the most impactful features from a total of 1172 features created. Each model and stratified subpopulation under test reported precision scores >70%, accuracy and area under the receiver operating curve scores >80%, and sensitivity scores approximately >90%. We noted statistically significant variations in model performance across stratified subpopulations identified by age, race or ethnicity, gender, and residence (urban vs rural). Conclusions: This study presents the possibility of developing decision models capable of predicting patient-level health care resource utilization across a broad, statewide region with considerable predictive performance. However, our models present statistically significant variations in performance across stratified subpopulations of interest. Further efforts are necessary to identify root causes of these biases and to rectify them. %M 34581671 %R 10.2196/31337 %U https://www.jmir.org/2021/11/e31337 %U https://doi.org/10.2196/31337 %U http://www.ncbi.nlm.nih.gov/pubmed/34581671 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 7 %N 11 %P e29504 %T Algorithm for Individual Prediction of COVID-19–Related Hospitalization Based on Symptoms: Development and Implementation Study %A Murtas,Rossella %A Morici,Nuccia %A Cogliati,Chiara %A Puoti,Massimo %A Omazzi,Barbara %A Bergamaschi,Walter %A Voza,Antonio %A Rovere Querini,Patrizia %A Stefanini,Giulio %A Manfredi,Maria Grazia %A Zocchi,Maria Teresa %A Mangiagalli,Andrea %A Brambilla,Carla Vittoria %A Bosio,Marco %A Corradin,Matteo %A Cortellaro,Francesca %A Trivelli,Marco %A Savonitto,Stefano %A Russo,Antonio Giampiero %+ Epidemiology Unit, Agency for the Protection of Health of the Metropolitan Area of Milan, Via Conca del Naviglio 45, Milan, 20123, Italy, 39 0285782111, agrusso@ats-milano.it %K COVID-19 %K severe outcome %K prediction %K monitoring system %K symptoms %K risk prediction %K risk %K algorithms %K prediction models %K pandemic %K digital data %K health records %D 2021 %7 15.11.2021 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: The COVID-19 pandemic has placed a huge strain on the health care system globally. The metropolitan area of Milan, Italy, was one of the regions most impacted by the COVID-19 pandemic worldwide. Risk prediction models developed by combining administrative databases and basic clinical data are needed to stratify individual patient risk for public health purposes. Objective: This study aims to develop a stratification tool aimed at improving COVID-19 patient management and health care organization. Methods: A predictive algorithm was developed and applied to 36,834 patients with COVID-19 in Italy between March 8 and the October 9, 2020, in order to foresee their risk of hospitalization. Exposures considered were age, sex, comorbidities, and symptoms associated with COVID-19 (eg, vomiting, cough, fever, diarrhea, myalgia, asthenia, headache, anosmia, ageusia, and dyspnea). The outcome was hospitalizations and emergency department admissions for COVID-19. Discrimination and calibration of the model were also assessed. Results: The predictive model showed a good fit for predicting COVID-19 hospitalization (C-index 0.79) and a good overall prediction accuracy (Brier score 0.14). The model was well calibrated (intercept –0.0028, slope 0.9970). Based on these results, 118,804 patients diagnosed with COVID-19 from October 25 to December 11, 2020, were stratified into low, medium, and high risk for COVID-19 severity. Among the overall study population, 67,030 (56.42%) were classified as low-risk patients; 43,886 (36.94%), as medium-risk patients; and 7888 (6.64%), as high-risk patients. In all, 89.37% (106,179/118,804) of the overall study population was being assisted at home, 9% (10,695/118,804) was hospitalized, and 1.62% (1930/118,804) died. Among those assisted at home, most people (63,983/106,179, 60.26%) were classified as low risk, whereas only 3.63% (3858/106,179) were classified at high risk. According to ordinal logistic regression, the odds ratio (OR) of being hospitalized or dead was 5.0 (95% CI 4.6-5.4) among high-risk patients and 2.7 (95% CI 2.6-2.9) among medium-risk patients, as compared to low-risk patients. Conclusions: A simple monitoring system, based on primary care data sets linked to COVID-19 testing results, hospital admissions data, and death records may assist in the proper planning and allocation of patients and resources during the ongoing COVID-19 pandemic. %M 34543227 %R 10.2196/29504 %U https://publichealth.jmir.org/2021/11/e29504 %U https://doi.org/10.2196/29504 %U http://www.ncbi.nlm.nih.gov/pubmed/34543227 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e31186 %T The Relationship Between Electronic Health Record System and Performance on Quality Measures in the American College of Rheumatology’s Rheumatology Informatics System for Effectiveness (RISE) Registry: Observational Study %A Hammam,Nevin %A Izadi,Zara %A Li,Jing %A Evans,Michael %A Kay,Julia %A Shiboski,Stephen %A Schmajuk,Gabriela %A Yazdany,Jinoos %+ Division of Rheumatology, Department of Medicine, University of California, P O Box 0811, Floor 03, Room 3301, San Francisco, CA, 94110, United States, 1 628 206 8618, jinoos.yazdany@ucsf.edu %K rheumatoid arthritis %K electronic health record %K patient-reported outcomes %K quality measures %K electronic health record %K disease activity %K quality of care %K performance reporting %K medical informatics %K clinical informatics %D 2021 %7 12.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Routine collection of disease activity (DA) and patient-reported outcomes (PROs) in rheumatoid arthritis (RA) are nationally endorsed quality measures and critical components of a treat-to-target approach. However, little is known about the role electronic health record (EHR) systems play in facilitating performance on these measures. Objective: Using the American College Rheumatology’s (ACR’s) RISE registry, we analyzed the relationship between EHR system and performance on DA and functional status (FS) quality measures. Methods: We analyzed data collected in 2018 from practices enrolled in RISE. We assessed practice-level performance on quality measures that require DA and FS documentation. Multivariable linear regression and zero-inflated negative binomial models were used to examine the independent effect of EHR system on practice-level quality measure performance, adjusting for practice characteristics and patient case-mix. Results: In total, 220 included practices cared for 314,793 patients with RA. NextGen was the most commonly used EHR system (34.1%). We found wide variation in performance on DA and FS quality measures by EHR system (median 30.1, IQR 0-74.8, and median 9.0, IQR 0-74.2), respectively). Even after adjustment, NextGen practices performed significantly better than Allscripts on the DA measure (51.4% vs 5.0%; P<.05) and significantly better than eClinicalWorks and eMDs on the FS measure (49.3% vs 29.0% and 10.9%; P<.05). Conclusions: Performance on national RA quality measures was associated with the EHR system, even after adjusting for practice and patient characteristics. These findings suggest that future efforts to improve quality of care in RA should focus not only on provider performance reporting but also on developing and implementing rheumatology-specific standards across EHRs. %M 34766910 %R 10.2196/31186 %U https://medinform.jmir.org/2021/11/e31186 %U https://doi.org/10.2196/31186 %U http://www.ncbi.nlm.nih.gov/pubmed/34766910 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 11 %P e23481 %T Determinants of Use of the Care Information Exchange Portal: Cross-sectional Study %A Neves,Ana Luisa %A Smalley,Katelyn R %A Freise,Lisa %A Harrison,Paul %A Darzi,Ara %A Mayer,Erik K %+ Patient Safety Translational Research Centre, Institute of Global Health Innovation, Imperial College, St Mary’s Campus, Queen Elizabeth Queen Mother Wing, London, W2 1NY, United Kingdom, 44 (0)20 7589 5111, ana.luisa.neves@gmail.com %K patient portals %K electronic health records %K patient participation %D 2021 %7 11.11.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Sharing electronic health records with patients has been shown to improve patient safety and quality of care. Patient portals represent a convenient tool to enhance patient access to their own health care data. However, the success of portals will only be possible through sustained adoption by its end users: the patients. A better understanding of the characteristics of users and nonusers is critical for understanding which groups remain excluded from using such tools. Objective: This study aims to identify the determinants of the use of the Care Information Exchange, a shared patient portal program in the United Kingdom. Methods: A cross-sectional study was conducted using a web-based questionnaire. Information collected included age, gender, ethnicity, educational level, health status, postcode, and digital literacy. Registered individuals were defined as having had an account created in the portal, independent of their actual use of the platform; users were defined as having ever used the portal. Multivariate logistic regression was used to model the probability of being a user. Statistical analysis was performed in R and Tableau was used to create maps of the proportion of Care Information Exchange users by postcode area. Results: A total of 1083 participants replied to the survey (186% of the estimated minimum target sample). The proportion of users was 61.58% (667/1083). Among these, most (385/667, 57.7%) used the portal at least once a month. To characterize the system’s users and nonusers, we performed a subanalysis of the sample, including only participants who had provided at least information regarding gender and age. The subanalysis included 650 individuals (389/650, 59.8% women; 551/650, 84.8% >40 years). Most participants were White (498/650, 76.6%) and resided in London (420/650, 64.6%). Individuals with a higher educational degree (undergraduate and professional, or postgraduate and higher) had higher odds of being a portal user (adjusted odds ratio [OR] 1.58, 95% CI 1.04-2.39 and OR 2.38, 95% CI 1.42-4.02, respectively) compared with those with a secondary degree or below. Higher digital literacy scores (≥30) were associated with higher odds of being a user (adjusted OR 2.96, 95% CI 2.02-4.35). Those with a good overall health status had lower odds of being a user (adjusted OR 0.58, 95% CI 0.37-0.91). Conclusions: This work adds to the growing body of evidence highlighting the importance of educational aspects (educational level and digital literacy) in the adoption of patient portals. Further research should not only describe but also systematically address these inequalities through patient-centered interventions aimed at reducing the digital divide. Health care providers and policy makers must partner in investing and delivering strategic programs that improve access to technology and digital literacy in an effort to improve digital inclusion and reduce inequities in the delivery of care. %M 34762063 %R 10.2196/23481 %U https://www.jmir.org/2021/11/e23481 %U https://doi.org/10.2196/23481 %U http://www.ncbi.nlm.nih.gov/pubmed/34762063 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e31527 %T The Role of Physicians in Digitalizing Health Care Provision: Web-Based Survey Study %A Burmann,Anja %A Tischler,Max %A Faßbach,Mira %A Schneitler,Sophie %A Meister,Sven %+ Fraunhofer Institute for Software and Systems Engineering, Emil-Figge-Str 91, Dortmund, 44227, Germany, 49 2319 7677435, anja.burmann@isst.fraunhofer.de %K digitalization %K digital transformation %K health care %K human factor %K physicians %K digital natives %K web-based survey %K digital health %D 2021 %7 11.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Digitalization affects all areas of society, including the health care sector. However, the digitalization of health care provision is progressing slowly compared to other sectors. In the professional and political literature, physicians are partially portrayed as digitalization sceptics. Thus, the role of physicians in this process requires further investigation. The theory of “digital natives” suggests a lower hurdle for younger generations to engage with digital technologies. Objective: The objective of this study was to investigate the role of physicians in the process of digitalizing health care provision in Germany and to assess the age factor. Methods: We conducted a large-scale study to assess the role of this professional group in the progress of the digital transformation of the German health care sector. Therefore, in an anonymous online survey, we inquired about the current digital penetration of the personal working environment, expectations, attitude toward, and concerns regarding digitalization. Based on these data, we studied associations with the nominal variable age and variations across 2 age groups. Results: The 1274 participants included in the study generally showed a high affinity towards digitalization with a mean of 3.88 on a 5-point Likert scale; 723 respondents (56.75%) stated they personally use mobile apps in their everyday working life, with a weak tendency to be associated with the respondents’ age (η=0.26). Participants saw the most noticeable existing benefits through digitalization in data quality and readability (882/1274, 69.23%) and the least in patient engagement (213/1274, 16.72%). Medical practitioners preponderantly expect further improvements through increased digitalization across almost all queried areas but the most in access to medical knowledge (1136/1274, 89.17%), treatment of orphan diseases (1016/1274, 79.75%), and medical research (1023/1274, 80.30%). Conclusions: Respondents defined their role in the digitalization of health care provision as ambivalent: “scrutinizing” on the one hand but “active” and “open” on the other. A gap between willingness to participate and digital sovereignty was indicated. Thus, education on digitalization as a means to support health care provision should not only be included in the course of study but also in the continuing process of further and advanced training. %M 34545813 %R 10.2196/31527 %U https://medinform.jmir.org/2021/11/e31527 %U https://doi.org/10.2196/31527 %U http://www.ncbi.nlm.nih.gov/pubmed/34545813 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 5 %N 11 %P e27748 %T A Web-Based, Population-Based Cirrhosis Identification and Management System for Improving Cirrhosis Care: Qualitative Formative Evaluation %A Javier,Sarah J %A Wu,Justina %A Smith,Donna L %A Kanwal,Fasiha %A Martin,Lindsey A %A Clark,Jack %A Midboe,Amanda M %+ Center for Innovation to Implementation (Ci2i), VA Palo Alto Health Care System, 795 Willow Road (152-MPD), Menlo Park, CA, 94025, United States, 1 6504935000, Sarah.Javier@va.gov %K cirrhosis %K informatics %K care coordination %K implementation %K Consolidated Framework for Implementation Research (CFIR) %K quality improvement %D 2021 %7 9.11.2021 %9 Original Paper %J JMIR Form Res %G English %X Background: Cirrhosis, or scarring of the liver, is a debilitating condition that affects millions of US adults. Early identification, linkage to care, and retention of care are critical for preventing severe complications and death from cirrhosis. Objective: The purpose of this study is to conduct a preimplementation formative evaluation to identify factors that could impact implementation of the Population-Based Cirrhosis Identification and Management System (P-CIMS) in clinics serving patients with cirrhosis. P-CIMS is a web-based informatics tool designed to facilitate patient outreach and cirrhosis care management. Methods: Semistructured interviews were conducted between January and May 2016 with frontline providers in liver disease and primary care clinics at 3 Veterans Health Administration medical centers. A total of 10 providers were interviewed, including 8 physicians and midlevel providers from liver-related specialty clinics and 2 primary care providers who managed patients with cirrhosis. The Consolidated Framework for Implementation Research guided the development of the interview guides. Inductive consensus coding and content analysis were used to analyze transcribed interviews and abstracted coded passages, elucidated themes, and insights. Results: The following themes and subthemes emerged from the analyses: outer setting: needs and resources for patients with cirrhosis; inner setting: readiness for implementation (subthemes: lack of resources, lack of leadership support), and implementation climate (subtheme: competing priorities); characteristics of individuals: role within clinic; knowledge and beliefs about P-CIMS (subtheme: perceived and realized benefits; useful features; suggestions for improvement); and perceptions of current practices in managing cirrhosis cases (subthemes: preimplementation process for identifying and linking patients to cirrhosis care; structural and social barriers to follow-up). Overall, P-CIMS was viewed as a powerful tool for improving linkage and retention, but its integration in the clinical workflow required leadership support, time, and staffing. Providers also cited the need for more intuitive interface elements to enhance usability. Conclusions: P-CIMS shows promise as a powerful tool for identifying, linking, and retaining care in patients living with cirrhosis. The current evaluation identified several improvements and advantages of P-CIMS over current care processes and provides lessons for others implementing similar population-based identification and management tools in populations with chronic disease. %M 34751653 %R 10.2196/27748 %U https://formative.jmir.org/2021/11/e27748 %U https://doi.org/10.2196/27748 %U http://www.ncbi.nlm.nih.gov/pubmed/34751653 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e26914 %T Local Differential Privacy in the Medical Domain to Protect Sensitive Information: Algorithm Development and Real-World Validation %A Sung,MinDong %A Cha,Dongchul %A Park,Yu Rang %+ Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Yonsei-ro 50-1, Seoul, 03722, Republic of Korea, 82 2 228 2363, yurangpark@yuhs.ac %K privacy-preserving %K differential privacy %K medical informatics %K medical data %K privacy %K electronic health record %K algorithm %K development %K validation %K big data %K medical data %K feasibility %K machine learning %K synthetic data %D 2021 %7 8.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Privacy is of increasing interest in the present big data era, particularly the privacy of medical data. Specifically, differential privacy has emerged as the standard method for preservation of privacy during data analysis and publishing. Objective: Using machine learning techniques, we applied differential privacy to medical data with diverse parameters and checked the feasibility of our algorithms with synthetic data as well as the balance between data privacy and utility. Methods: All data were normalized to a range between –1 and 1, and the bounded Laplacian method was applied to prevent the generation of out-of-bound values after applying the differential privacy algorithm. To preserve the cardinality of the categorical variables, we performed postprocessing via discretization. The algorithm was evaluated using both synthetic and real-world data (from the eICU Collaborative Research Database). We evaluated the difference between the original data and the perturbated data using misclassification rates and the mean squared error for categorical data and continuous data, respectively. Further, we compared the performance of classification models that predict in-hospital mortality using real-world data. Results: The misclassification rate of categorical variables ranged between 0.49 and 0.85 when the value of ε was 0.1, and it converged to 0 as ε increased. When ε was between 102 and 103, the misclassification rate rapidly dropped to 0. Similarly, the mean squared error of the continuous variables decreased as ε increased. The performance of the model developed from perturbed data converged to that of the model developed from original data as ε increased. In particular, the accuracy of a random forest model developed from the original data was 0.801, and this value ranged from 0.757 to 0.81 when ε was 10-1 and 104, respectively. Conclusions: We applied local differential privacy to medical domain data, which are diverse and high dimensional. Higher noise may offer enhanced privacy, but it simultaneously hinders utility. We should choose an appropriate degree of noise for data perturbation to balance privacy and utility depending on specific situations. %M 34747711 %R 10.2196/26914 %U https://medinform.jmir.org/2021/11/e26914 %U https://doi.org/10.2196/26914 %U http://www.ncbi.nlm.nih.gov/pubmed/34747711 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 11 %P e29951 %T Patients Contributing to Visit Notes: Mixed Methods Evaluation of OurNotes %A Walker,Jan %A Leveille,Suzanne %A Kriegel,Gila %A Lin,Chen-Tan %A Liu,Stephen K %A Payne,Thomas H %A Harcourt,Kendall %A Dong,Zhiyong %A Fitzgerald,Patricia %A Germak,Matthew %A Markson,Lawrence %A Jackson,Sara L %A Shucard,Hannah %A Elmore,Joann G %A Delbanco,Tom %+ Division of General Medicine, Beth Israel Deaconess Medical Center, 331 Brookline Avenue, Boston, MA, 02215, United States, 1 6173208733, jwalker1@bidmc.harvard.edu %K electronic health record %K previsit information %K physician-patient relations %K patient portal %K mobile phone %D 2021 %7 8.11.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Secure patient portals are widely available, and patients use them to view their electronic health records, including their clinical notes. We conducted experiments asking them to cogenerate notes with their clinicians, an intervention called OurNotes. Objective: This study aims to assess patient and provider experiences and attitudes after 12 months of a pilot intervention. Methods: Before scheduled primary care visits, patients were asked to submit a word-constrained, unstructured interval history and an agenda for what they would like to discuss at the visit. Using site-specific methods, their providers were invited to incorporate the submissions into notes documenting the visits. Sites served urban, suburban, and rural patients in primary care practices in 4 academic health centers in Boston (Massachusetts), Lebanon (New Hampshire), Denver (Colorado), and Seattle (Washington). Each practice offered electronic access to visit notes (open notes) to its patients for several years. A mixed methods evaluation used tracking data and electronic survey responses from patients and clinicians. Participants were 174 providers and 1962 patients who submitted at least 1 previsit form. We asked providers about the usefulness of the submissions, effects on workflow, and ideas for the future. We asked patients about difficulties and benefits of providing the requested information and ideas for future improvements. Results: Forms were submitted before 9.15% (5365/58,652) eligible visits, and 43.7% (76/174) providers and 26.76% (525/1962) patients responded to the postintervention evaluation surveys; 74 providers and 321 patients remembered receiving and completing the forms and answered the survey questions. Most clinicians thought interim patient histories (69/74, 93%) and patient agendas (72/74, 97%) as good ideas, 70% (52/74) usually or always incorporated them into visit notes, 54% (40/74) reported no change in visit length, and 35% (26/74) thought they saved time. Their most common suggestions related to improving notifications when patient forms were received, making it easier to find the form and insert it into the note, and educating patients about how best to prepare their submissions. Patient respondents were generally well educated, most found the history (259/321, 80.7%) and agenda (286/321, 89.1%) questions not difficult to answer; more than 92.2% (296/321) thought sending answers before the visit a good idea; 68.8% (221/321) thought the questions helped them prepare for the visit. Common suggestions by patients included learning to write better answers and wanting to know that their submissions were read by their clinicians. At the end of the pilot, all participating providers chose to continue the OurNotes previsit form, and sites considered expanding the intervention to more clinicians and adapting it for telemedicine visits. Conclusions: OurNotes interests patients, and providers experience it as a positive intervention. Participation by patients, care partners, clinicians, and electronic health record experts will facilitate further development. %M 34747710 %R 10.2196/29951 %U https://www.jmir.org/2021/11/e29951 %U https://doi.org/10.2196/29951 %U http://www.ncbi.nlm.nih.gov/pubmed/34747710 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 4 %P e27568 %T The Effectiveness of a Multidisciplinary Electronic Discharge Readiness Tool: Prospective, Single-Center, Pre-Post Study %A Keniston,Angela %A McBeth,Lauren %A Pell,Jonathan %A Bowden,Kasey %A Metzger,Anna %A Nordhagen,Jamie %A Anthony,Amanda %A Rice,John %A Burden,Marisha %+ Anschutz Medical Campus, Division of Hospital Medicine, University of Colorado, 12401 E. 17th Avenue, Mail Stop F782, Aurora, CO, 80045, United States, 1 7202401431, Angela.Keniston@cuanschutz.edu %K discharge planning %K health information technology %K quasi-experimental design %K multidisciplinary %K teamwork %D 2021 %7 8.11.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: In the face of hospital capacity strain, hospitals have developed multifaceted plans to try to improve patient flow. Many of these initiatives have focused on the timing of discharges and on lowering lengths of stay, and they have met with variable success. We deployed a novel tool in the electronic health record to enhance discharge communication. Objective: The aim of this study is to evaluate the effectiveness of a discharge communication tool. Methods: This was a prospective, single-center, pre-post study. Hospitalist physicians and advanced practice providers (APPs) used the Discharge Today Tool to update patient discharge readiness every morning and at any time the patient status changed throughout the day. Primary outcomes were tool use, time of day the clinician entered the discharge order, time of day the patient left the hospital, and hospital length of stay. We used linear mixed modeling and generalized linear mixed modeling, with team and discharging provider included in all the models to account for patients cared for by the same team and the same provider. Results: During the pilot implementation period from March 5, 2019, to July 31, 2019, a total of 4707 patients were discharged (compared with 4558 patients discharged during the preimplementation period). A total of 352 clinical staff had used the tool, and 84.85% (3994/4707) of the patients during the pilot period had a discharge status assigned at least once. In a survey, most respondents reported that the tool was helpful (32/34, 94% of clinical staff) and either saved time or did not add additional time to their workflow (21/24, 88% of providers, and 34/34, 100% of clinical staff). Although improvements were not observed in either unadjusted or adjusted analyses, after including starting morning census per team as an effect modifier, there was a reduction in the time of day the discharge order was entered into the electronic health record by the discharging physician and in the time of day the patient left the hospital (decrease of 2.9 minutes per additional patient, P=.07, and 3 minutes per additional patient, P=.07, respectively). As an effect modifier, for teams that included an APP, there was a significant reduction in the time of day the patient left the hospital beyond the reduction seen for teams without an APP (decrease of 19.1 minutes per patient, P=.04). Finally, in the adjusted analysis, hospital length of stay decreased by an average of 3.7% (P=.06). Conclusions: The Discharge Today tool allows for real time documentation and sharing of discharge status. Our results suggest an overall positive response by care team members and that the tool may be useful for improving discharge time and length of stay if a team is staffed with an APP or in higher-census situations. %M 34747702 %R 10.2196/27568 %U https://humanfactors.jmir.org/2021/4/e27568 %U https://doi.org/10.2196/27568 %U http://www.ncbi.nlm.nih.gov/pubmed/34747702 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e26426 %T Event Prediction Model Considering Time and Input Error Using Electronic Medical Records in the Intensive Care Unit: Retrospective Study %A Sung,MinDong %A Hahn,Sangchul %A Han,Chang Hoon %A Lee,Jung Mo %A Lee,Jayoung %A Yoo,Jinkyu %A Heo,Jay %A Kim,Young Sam %A Chung,Kyung Soo %+ Division of Pulmonology, Department of Internal Medicine, Yonsei University Health System, 50-1 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea, 82 2227 8308, chungks78@gmail.com %K machine learning %K critical care %K prediction model %K intensive care unit %K mortality %K AKI %K sepsis %D 2021 %7 4.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: In the era of artificial intelligence, event prediction models are abundant. However, considering the limitation of the electronic medical record–based model, including the temporally skewed prediction and the record itself, these models could be delayed or could yield errors. Objective: In this study, we aim to develop multiple event prediction models in intensive care units to overcome their temporal skewness and evaluate their robustness against delayed and erroneous input. Methods: A total of 21,738 patients were included in the development cohort. Three events—death, sepsis, and acute kidney injury—were predicted. To overcome the temporal skewness, we developed three models for each event, which predicted the events in advance of three prespecified timepoints. Additionally, to evaluate the robustness against input error and delays, we added simulated errors and delayed input and calculated changes in the area under the receiver operating characteristic curve (AUROC) values. Results: Most of the AUROC and area under the precision-recall curve values of each model were higher than those of the conventional scores, as well as other machine learning models previously used. In the error input experiment, except for our proposed model, an increase in the noise added to the model lowered the resulting AUROC value. However, the delayed input did not show the performance decreased in this experiment. Conclusions: For a prediction model that was applicable in the real world, we considered not only performance but also temporal skewness, delayed input, and input error. %M 34734837 %R 10.2196/26426 %U https://medinform.jmir.org/2021/11/e26426 %U https://doi.org/10.2196/26426 %U http://www.ncbi.nlm.nih.gov/pubmed/34734837 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e29120 %T Stroke Outcome Measurements From Electronic Medical Records: Cross-sectional Study on the Effectiveness of Neural and Nonneural Classifiers %A Zanotto,Bruna Stella %A Beck da Silva Etges,Ana Paula %A dal Bosco,Avner %A Cortes,Eduardo Gabriel %A Ruschel,Renata %A De Souza,Ana Claudia %A Andrade,Claudio M V %A Viegas,Felipe %A Canuto,Sergio %A Luiz,Washington %A Ouriques Martins,Sheila %A Vieira,Renata %A Polanczyk,Carisi %A André Gonçalves,Marcos %+ Computer Science Department, Universidade Federal de Minas Gerais, Avenue Antônio Carlos, 6627, Belo Horizonte, 31270-901, Brazil, 55 3134095860, mgoncalv@dcc.ufmg.br %K natural language processing %K stroke %K outcomes %K electronic medical records %K EHR %K electronic health records %K text processing %K data mining %K text classification %K patient outcomes %D 2021 %7 1.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: With the rapid adoption of electronic medical records (EMRs), there is an ever-increasing opportunity to collect data and extract knowledge from EMRs to support patient-centered stroke management. Objective: This study aims to compare the effectiveness of state-of-the-art automatic text classification methods in classifying data to support the prediction of clinical patient outcomes and the extraction of patient characteristics from EMRs. Methods: Our study addressed the computational problems of information extraction and automatic text classification. We identified essential tasks to be considered in an ischemic stroke value-based program. The 30 selected tasks were classified (manually labeled by specialists) according to the following value agenda: tier 1 (achieved health care status), tier 2 (recovery process), care related (clinical management and risk scores), and baseline characteristics. The analyzed data set was retrospectively extracted from the EMRs of patients with stroke from a private Brazilian hospital between 2018 and 2019. A total of 44,206 sentences from free-text medical records in Portuguese were used to train and develop 10 supervised computational machine learning methods, including state-of-the-art neural and nonneural methods, along with ontological rules. As an experimental protocol, we used a 5-fold cross-validation procedure repeated 6 times, along with subject-wise sampling. A heatmap was used to display comparative result analyses according to the best algorithmic effectiveness (F1 score), supported by statistical significance tests. A feature importance analysis was conducted to provide insights into the results. Results: The top-performing models were support vector machines trained with lexical and semantic textual features, showing the importance of dealing with noise in EMR textual representations. The support vector machine models produced statistically superior results in 71% (17/24) of tasks, with an F1 score >80% regarding care-related tasks (patient treatment location, fall risk, thrombolytic therapy, and pressure ulcer risk), the process of recovery (ability to feed orally or ambulate and communicate), health care status achieved (mortality), and baseline characteristics (diabetes, obesity, dyslipidemia, and smoking status). Neural methods were largely outperformed by more traditional nonneural methods, given the characteristics of the data set. Ontological rules were also effective in tasks such as baseline characteristics (alcoholism, atrial fibrillation, and coronary artery disease) and the Rankin scale. The complementarity in effectiveness among models suggests that a combination of models could enhance the results and cover more tasks in the future. Conclusions: Advances in information technology capacity are essential for scalability and agility in measuring health status outcomes. This study allowed us to measure effectiveness and identify opportunities for automating the classification of outcomes of specific tasks related to clinical conditions of stroke victims, and thus ultimately assess the possibility of proactively using these machine learning techniques in real-world situations. %M 34723829 %R 10.2196/29120 %U https://medinform.jmir.org/2021/11/e29120 %U https://doi.org/10.2196/29120 %U http://www.ncbi.nlm.nih.gov/pubmed/34723829 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 11 %P e28763 %T Detecting Adverse Drug Events Through the Chronological Relationship Between the Medication Period and the Presence of Adverse Reactions From Electronic Medical Record Systems: Observational Study %A Teramoto,Kei %A Takeda,Toshihiro %A Mihara,Naoki %A Shimai,Yoshie %A Manabe,Shirou %A Kuwata,Shigeki %A Kondoh,Hiroshi %A Matsumura,Yasushi %+ Department of Medical Informatics, Graduate School of Medicine, Osaka University, 2-2 Yamada-oka, Suita, 565 0871, Japan, 81 6 6879 5900, ttakeda@hp-info.med.osaka-u.ac.jp %K real world data %K electronic medical record %K adverse drug event %D 2021 %7 1.11.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Medicines may cause various adverse reactions. An enormous amount of money and effort is spent investigating adverse drug events (ADEs) in clinical trials and postmarketing surveillance. Real-world data from multiple electronic medical records (EMRs) can make it easy to understand the ADEs that occur in actual patients. Objective: In this study, we generated a patient medication history database from physician orders recorded in EMRs, which allowed the period of medication to be clearly identified. Methods: We developed a method for detecting ADEs based on the chronological relationship between the presence of an adverse event and the medication period. To verify our method, we detected ADEs with alanine aminotransferase elevation in patients receiving aspirin, clopidogrel, and ticlopidine. The accuracy of the detection was evaluated with a chart review and by comparison with the Roussel Uclaf Causality Assessment Method (RUCAM), which is a standard method for detecting drug-induced liver injury. Results: The calculated rates of ADE with ALT elevation in patients receiving aspirin, clopidogrel, and ticlopidine were 3.33% (868/26,059 patients), 3.70% (188/5076 patients), and 5.69% (226/3974 patients), respectively, which were in line with the rates of previous reports. We reviewed the medical records of the patients in whom ADEs were detected. Our method accurately predicted ADEs in 90% (27/30patients) treated with aspirin, 100% (9/9 patients) treated with clopidogrel, and 100% (4/4 patients) treated with ticlopidine. Only 3 ADEs that were detected by the RUCAM were not detected by our method. Conclusions: These findings demonstrate that the present method is effective for detecting ADEs based on EMR data. %M 33993103 %R 10.2196/28763 %U https://medinform.jmir.org/2021/11/e28763 %U https://doi.org/10.2196/28763 %U http://www.ncbi.nlm.nih.gov/pubmed/33993103 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 10 %P e31294 %T Public Attitudes to Digital Health Research Repositories: Cross-sectional International Survey %A Nunes Vilaza,Giovanna %A Coyle,David %A Bardram,Jakob Eyvind %+ Department of Health Technology, Technical University of Denmark, Ørsteds Plads 345B, Kongens Lyngby, 2800, Denmark, 45 45253724, gnvi@dtu.dk %K digital medicine %K health informatics %K health data repositories %K personal sensing %K technology acceptance %K willingness to share data %K human-centered computing %K ethics %D 2021 %7 29.10.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Digital health research repositories propose sharing longitudinal streams of health records and personal sensing data between multiple projects and researchers. Motivated by the prospect of personalizing patient care (precision medicine), these initiatives demand broad public acceptance and large numbers of data contributors, both of which are challenging. Objective: This study investigates public attitudes toward possibly contributing to digital health research repositories to identify factors for their acceptance and to inform future developments. Methods: A cross-sectional online survey was conducted from March 2020 to December 2020. Because of the funded project scope and a multicenter collaboration, study recruitment targeted young adults in Denmark and Brazil, allowing an analysis of the differences between 2 very contrasting national contexts. Through closed-ended questions, the survey examined participants’ willingness to share different data types, data access preferences, reasons for concern, and motivations to contribute. The survey also collected information about participants’ demographics, level of interest in health topics, previous participation in health research, awareness of examples of existing research data repositories, and current attitudes about digital health research repositories. Data analysis consisted of descriptive frequency measures and statistical inferences (bivariate associations and logistic regressions). Results: The sample comprises 1017 respondents living in Brazil (1017/1600, 63.56%) and 583 in Denmark (583/1600, 36.44%). The demographics do not differ substantially between participants of these countries. The majority is aged between 18 and 27 years (933/1600, 58.31%), is highly educated (992/1600, 62.00%), uses smartphones (1562/1600, 97.63%), and is in good health (1407/1600, 87.94%). The analysis shows a vast majority were very motivated by helping future patients (1366/1600, 85.38%) and researchers (1253/1600, 78.31%), yet very concerned about unethical projects (1219/1600, 76.19%), profit making without consent (1096/1600, 68.50%), and cyberattacks (1055/1600, 65.94%). Participants’ willingness to share data is lower when sharing personal sensing data, such as the content of calls and texts (1206/1600, 75.38%), in contrast to more traditional health research information. Only 13.44% (215/1600) find it desirable to grant data access to private companies, and most would like to stay informed about which projects use their data (1334/1600, 83.38%) and control future data access (1181/1600, 73.81%). Findings indicate that favorable attitudes toward digital health research repositories are related to a personal interest in health topics (odds ratio [OR] 1.49, 95% CI 1.10-2.02; P=.01), previous participation in health research studies (OR 1.70, 95% CI 1.24-2.35; P=.001), and awareness of examples of research repositories (OR 2.78, 95% CI 1.83-4.38; P<.001). Conclusions: This study reveals essential factors for acceptance and willingness to share personal data with digital health research repositories. Implications include the importance of being more transparent about the goals and beneficiaries of research projects using and re-using data from repositories, providing participants with greater autonomy for choosing who gets access to which parts of their data, and raising public awareness of the benefits of data sharing for research. In addition, future developments should engage with and reduce risks for those unwilling to participate. %M 34714253 %R 10.2196/31294 %U https://www.jmir.org/2021/10/e31294 %U https://doi.org/10.2196/31294 %U http://www.ncbi.nlm.nih.gov/pubmed/34714253 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 10 %P e29259 %T Transforming Anesthesia Data Into the Observational Medical Outcomes Partnership Common Data Model: Development and Usability Study %A Lamer,Antoine %A Abou-Arab,Osama %A Bourgeois,Alexandre %A Parrot,Adrien %A Popoff,Benjamin %A Beuscart,Jean-Baptiste %A Tavernier,Benoît %A Moussa,Mouhamed Djahoum %+ Univ. Lille, CHU Lille, ULR 2694 - METRICS: Évaluation des technologies de santé et des pratiques médicales, 2 place de Verdun, Lille, F-59000, France, 33 320626969, antoine.lamer@chru-lille.fr %K data reuse %K common data model %K Observational Medical Outcomes Partnership %K anesthesia %K data warehouse %K reproducible research %D 2021 %7 29.10.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Electronic health records (EHRs, such as those created by an anesthesia management system) generate a large amount of data that can notably be reused for clinical audits and scientific research. The sharing of these data and tools is generally affected by the lack of system interoperability. To overcome these issues, Observational Health Data Sciences and Informatics (OHDSI) developed the Observational Medical Outcomes Partnership (OMOP) common data model (CDM) to standardize EHR data and promote large-scale observational and longitudinal research. Anesthesia data have not previously been mapped into the OMOP CDM. Objective: The primary objective was to transform anesthesia data into the OMOP CDM. The secondary objective was to provide vocabularies, queries, and dashboards that might promote the exploitation and sharing of anesthesia data through the CDM. Methods: Using our local anesthesia data warehouse, a group of 5 experts from 5 different medical centers identified local concepts related to anesthesia. The concepts were then matched with standard concepts in the OHDSI vocabularies. We performed structural mapping between the design of our local anesthesia data warehouse and the OMOP CDM tables and fields. To validate the implementation of anesthesia data into the OMOP CDM, we developed a set of queries and dashboards. Results: We identified 522 concepts related to anesthesia care. They were classified as demographics, units, measurements, operating room steps, drugs, periods of interest, and features. After semantic mapping, 353 (67.7%) of these anesthesia concepts were mapped to OHDSI concepts. Further, 169 (32.3%) concepts related to periods and features were added to the OHDSI vocabularies. Then, 8 OMOP CDM tables were implemented with anesthesia data and 2 new tables (EPISODE and FEATURE) were added to store secondarily computed data. We integrated data from 5,72,609 operations and provided the code for a set of 8 queries and 4 dashboards related to anesthesia care. Conclusions: Generic data concerning demographics, drugs, units, measurements, and operating room steps were already available in OHDSI vocabularies. However, most of the intraoperative concepts (the duration of specific steps, an episode of hypotension, etc) were not present in OHDSI vocabularies. The OMOP mapping provided here enables anesthesia data reuse. %M 34714250 %R 10.2196/29259 %U https://www.jmir.org/2021/10/e29259 %U https://doi.org/10.2196/29259 %U http://www.ncbi.nlm.nih.gov/pubmed/34714250 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 10 %P e25378 %T Developing a RadLex-Based Named Entity Recognition Tool for Mining Textual Radiology Reports: Development and Performance Evaluation Study %A Tsuji,Shintaro %A Wen,Andrew %A Takahashi,Naoki %A Zhang,Hongjian %A Ogasawara,Katsuhiko %A Jiang,Gouqian %+ Department of Health Sciences Research, Department of Radiology, 200 First Street, SW, Rochester, MN , United States, 1 507 266 1327, Jiang.Guoqian@mayo.edu %K named entity recognition (NER) %K natural language processing (NLP) %K RadLex %K ontology %K stem term %D 2021 %7 29.10.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Named entity recognition (NER) plays an important role in extracting the features of descriptions such as the name and location of a disease for mining free-text radiology reports. However, the performance of existing NER tools is limited because the number of entities that can be extracted depends on the dictionary lookup. In particular, the recognition of compound terms is very complicated because of the variety of patterns. Objective: The aim of this study is to develop and evaluate an NER tool concerned with compound terms using RadLex for mining free-text radiology reports. Methods: We leveraged the clinical Text Analysis and Knowledge Extraction System (cTAKES) to develop customized pipelines using both RadLex and SentiWordNet (a general purpose dictionary). We manually annotated 400 radiology reports for compound terms in noun phrases and used them as the gold standard for performance evaluation (precision, recall, and F-measure). In addition, we created a compound terms–enhanced dictionary (CtED) by analyzing false negatives and false positives and applied it to another 100 radiology reports for validation. We also evaluated the stem terms of compound terms by defining two measures: occurrence ratio (OR) and matching ratio (MR). Results: The F-measure of cTAKES+RadLex+general purpose dictionary was 30.9% (precision 73.3% and recall 19.6%) and that of the combined CtED was 63.1% (precision 82.8% and recall 51%). The OR indicated that the stem terms of effusion, node, tube, and disease were used frequently, but it still lacks capturing compound terms. The MR showed that 71.85% (9411/13,098) of the stem terms matched with that of the ontologies, and RadLex improved approximately 22% of the MR from the cTAKES default dictionary. The OR and MR revealed that the characteristics of stem terms would have the potential to help generate synonymous phrases using the ontologies. Conclusions: We developed a RadLex-based customized pipeline for parsing radiology reports and demonstrated that CtED and stem term analysis has the potential to improve dictionary-based NER performance with regard to expanding vocabularies. %M 34714247 %R 10.2196/25378 %U https://www.jmir.org/2021/10/e25378 %U https://doi.org/10.2196/25378 %U http://www.ncbi.nlm.nih.gov/pubmed/34714247 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 10 %P e26890 %T ClinicalTrials.gov as a Source of Information About Expanded Access Programs: Cohort Study %A Borysowski,Jan %A Górski,Andrzej %+ Department of Clinical Immunology, Medical University of Warsaw, Nowogrodzka Str 59, Warsaw, 02-006, Poland, 48 22 502 10 58, jborysowski@interia.pl %K ClinicalTrials.gov %K expanded access %K expanded access program %K compassionate use %K unapproved drug %K investigational drug %D 2021 %7 28.10.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: ClinicalTrials.gov (CT.gov) is the most comprehensive internet-based register of different types of clinical studies. Expanded access is the use of unapproved drugs, biologics, or medical devices outside of clinical trials. One of the key problems in expanded access is the availability to both health care providers and patients of information about unapproved treatments. Objective: We aimed to evaluate CT.gov as a potential source of information about expanded access programs. Methods: We assessed the completeness of information in the records of 228 expanded access programs registered with CT.gov from February 2017 through May 2020. Moreover, we examined what percentage of published expanded access studies has been registered with CT.gov. Logistic regression (univariate and multivariate) and mediation analyses were used to identify the predictors of the absence of some information and a study’s nonregistration. Results: We found that some important data were missing from the records of many programs. Information that was missing most often included a detailed study description, facility information, central contact person, and eligibility criteria (55.3%, 54.0%, 41.7%, and 17.5% of the programs, respectively). Multivariate analysis showed that information about central contact person was more likely to be missing from records of studies registered in 2017 (adjusted OR 21.93; 95% CI 4.42-172.29; P<.001). This finding was confirmed by mediation analysis (P=.02). Furthermore, 14% of the programs were registered retrospectively. We also showed that only 33 of 77 (42.9%) expanded access studies performed in the United States and published from 2014 through 2019 were registered with CT.gov. However, multivariate logistic regression analysis showed no significant association between any of the variables related to the studies and the odds of study nonregistration (P>.01). Conclusions: Currently, CT.gov is a quite fragmentary source of data on expanded access programs. This problem is important because CT.gov is the only publicly available primary source of information about specific programs. We suggest the actions that should be taken by different stakeholders to fully exploit this register as a source of information about expanded access. %M 34709189 %R 10.2196/26890 %U https://www.jmir.org/2021/10/e26890 %U https://doi.org/10.2196/26890 %U http://www.ncbi.nlm.nih.gov/pubmed/34709189 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 10 %P e28752 %T Privacy-Preserving Anonymity for Periodical Releases of Spontaneous Adverse Drug Event Reporting Data: Algorithm Development and Validation %A Wang,Jie-Teng %A Lin,Wen-Yang %+ Department of Computer Science and Information Engineering, National University of Kaohsiung, 700 Kaohsiung Univ. Rd, Nanzih District, Kaohsiung, 811, Taiwan, 886 7 5919517, wylin@nuk.edu.tw %K adverse drug reaction %K data anonymization %K incremental data publishing %K privacy preserving data publishing %K spontaneous reporting system %K drug %K data set %K anonymous %K privacy %K security %K algorithm %K development %K validation %K data %D 2021 %7 28.10.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Spontaneous reporting systems (SRSs) have been increasingly established to collect adverse drug events for fostering adverse drug reaction (ADR) detection and analysis research. SRS data contain personal information, and so their publication requires data anonymization to prevent the disclosure of individuals’ privacy. We have previously proposed a privacy model called MS(k, θ*)-bounding and the associated MS-Anonymization algorithm to fulfill the anonymization of SRS data. In the real world, the SRS data usually are released periodically (eg, FDA Adverse Event Reporting System [FAERS]) to accommodate newly collected adverse drug events. Different anonymized releases of SRS data available to the attacker may thwart our single-release-focus method, that is, MS(k, θ*)-bounding. Objective: We investigate the privacy threat caused by periodical releases of SRS data and propose anonymization methods to prevent the disclosure of personal privacy information while maintaining the utility of published data. Methods: We identify potential attacks on periodical releases of SRS data, namely, BFL-attacks, mainly caused by follow-up cases. We present a new privacy model called PPMS(k, θ*)-bounding, and propose the associated PPMS-Anonymization algorithm and 2 improvements: PPMS+-Anonymization and PPMS++-Anonymization. Empirical evaluations were performed using 32 selected FAERS quarter data sets from 2004Q1 to 2011Q4. The performance of the proposed versions of PPMS-Anonymization was inspected against MS-Anonymization from some aspects, including data distortion, measured by normalized information loss; privacy risk of anonymized data, measured by dangerous identity ratio and dangerous sensitivity ratio; and data utility, measured by the bias of signal counting and strength (proportional reporting ratio). Results: The best version of PPMS-Anonymization, PPMS++-Anonymization, achieves nearly the same quality as MS-Anonymization in both privacy protection and data utility. Overall, PPMS++-Anonymization ensures zero privacy risk on record and attribute linkage, and exhibits 51%-78% and 59%-82% improvements on information loss over PPMS+-Anonymization and PPMS-Anonymization, respectively, and significantly reduces the bias of ADR signal. Conclusions: The proposed PPMS(k, θ*)-bounding model and PPMS-Anonymization algorithm are effective in anonymizing SRS data sets in the periodical data publishing scenario, preventing the series of releases from disclosing personal sensitive information caused by BFL-attacks while maintaining the data utility for ADR signal detection. %M 34709197 %R 10.2196/28752 %U https://medinform.jmir.org/2021/10/e28752 %U https://doi.org/10.2196/28752 %U http://www.ncbi.nlm.nih.gov/pubmed/34709197 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 10 %P e32730 %T Adverse Drug Event Prediction Using Noisy Literature-Derived Knowledge Graphs: Algorithm Development and Validation %A Dasgupta,Soham %A Jayagopal,Aishwarya %A Jun Hong,Abel Lim %A Mariappan,Ragunathan %A Rajan,Vaibhav %+ Department of Information Systems and Analytics, National University of Singapore, School of Computing, Singapore, 117417, Singapore, 65 65166737, vaibhav.rajan@nus.edu.sg %K adverse drug event %K knowledge graph %K Embedding of Semantic Predications %K biomedical literature %D 2021 %7 25.10.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Adverse drug events (ADEs) are unintended side effects of drugs that cause substantial clinical and economic burdens globally. Not all ADEs are discovered during clinical trials; therefore, postmarketing surveillance, called pharmacovigilance, is routinely conducted to find unknown ADEs. A wealth of information, which facilitates ADE discovery, lies in the growing body of biomedical literature. Knowledge graphs (KGs) encode information from the literature, where the vertices and the edges represent clinical concepts and their relations, respectively. The scale and unstructured form of the literature necessitates the use of natural language processing (NLP) to automatically create such KGs. Previous studies have demonstrated the utility of such literature-derived KGs in ADE prediction. Through unsupervised learning of the representations (features) of clinical concepts from the KG, which are used in machine learning models, state-of-the-art results for ADE prediction were obtained on benchmark data sets. Objective: Due to the use of NLP to infer literature-derived KGs, there is noise in the form of false positive (erroneous) and false negative (absent) nodes and edges. Previous representation learning methods do not account for such inaccuracies in the graph. NLP algorithms can quantify the confidence in their inference of extracted concepts and relations from the literature. Our hypothesis, which motivates this work, is that by using such confidence scores during representation learning, the learned embeddings would yield better features for ADE prediction models. Methods: We developed methods to use these confidence scores on two well-known representation learning methods—DeepWalk and Translating Embeddings for Modeling Multi-relational Data (TransE)—to develop their weighted versions: Weighted DeepWalk and Weighted TransE. These methods were used to learn representations from a large literature-derived KG, the Semantic MEDLINE Database, which contains more than 93 million clinical relations. They were compared with Embedding of Semantic Predications, which, to our knowledge, is the best reported representation learning method using the Semantic MEDLINE Database with state-of-the-art results for ADE prediction. Representations learned from different methods were used (separately) as features of drugs and diseases to build classification models for ADE prediction using benchmark data sets. The methods were compared rigorously over multiple cross-validation settings. Results: The weighted versions we designed were able to learn representations that yielded more accurate predictive models than the corresponding unweighted versions of both DeepWalk and TransE, as well as Embedding of Semantic Predications, in our experiments. There were performance improvements of up to 5.75% in the F1-score and 8.4% in the area under the receiver operating characteristic curve value, thus advancing the state of the art in ADE prediction from literature-derived KGs. Conclusions: Our classification models can be used to aid pharmacovigilance teams in detecting potentially new ADEs. Our experiments demonstrate the importance of modeling inaccuracies in the inferred KGs for representation learning. %M 34694230 %R 10.2196/32730 %U https://medinform.jmir.org/2021/10/e32730 %U https://doi.org/10.2196/32730 %U http://www.ncbi.nlm.nih.gov/pubmed/34694230 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 10 %P e28039 %T Ensemble Learning-Based Pulse Signal Recognition: Classification Model Development Study %A Yan,Jianjun %A Cai,Xianglei %A Chen,Songye %A Guo,Rui %A Yan,Haixia %A Wang,Yiqin %+ Institute of Intelligent Perception and Diagnosis, School of Mechanical and Power Engineering, East China University of Science and Technology, 130 Meilong Road, Shanghai, 200237, China, 86 21 64252074, jjyan@ecust.edu.cn %K wrist pulse %K ensemble learning %K support vector machine %K deep convolutional neural network %K pulse signal %K machine learning %K traditional Chinese medicine %K pulse classification %K pulse analysis %K fully connected neural network %K synthetic minority oversampling technique %K feature extraction %D 2021 %7 21.10.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: In pulse signal analysis and identification, time domain and time frequency domain analysis methods can obtain interpretable structured data and build classification models using traditional machine learning methods. Unstructured data, such as pulse signals, contain rich information about the state of the cardiovascular system, and local features of unstructured data can be extracted and classified using deep learning. Objective: The objective of this paper was to comprehensively use machine learning and deep learning classification methods to fully exploit the information about pulse signals. Methods: Structured data were obtained by using time domain and time frequency domain analysis methods. A classification model was built using a support vector machine (SVM), a deep convolutional neural network (DCNN) kernel was used to extract local features of the unstructured data, and the stacking method was used to fuse the above classification results for decision making. Results: The highest average accuracy of 0.7914 was obtained using only a single classifier, while the average accuracy obtained using the ensemble learning approach was 0.8330. Conclusions: Ensemble learning can effectively use information from structured and unstructured data to improve classification accuracy through decision-level fusion. This study provides a new idea and method for pulse signal classification, which is of practical value for pulse diagnosis objectification. %M 34673537 %R 10.2196/28039 %U https://medinform.jmir.org/2021/10/e28039 %U https://doi.org/10.2196/28039 %U http://www.ncbi.nlm.nih.gov/pubmed/34673537 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 10 %P e33192 %T A Patient-Screening Tool for Clinical Research Based on Electronic Health Records Using OpenEHR: Development Study %A Li,Mengyang %A Cai,Hailing %A Nan,Shan %A Li,Jialin %A Lu,Xudong %A Duan,Huilong %+ College of Biomedical Engineering and Instrument Science, Zhejiang University, Yuquan Campus, 38 Zheda Road, Hangzhou, 310027, China, 86 13957118891, lvxd@zju.edu.cn %K openEHR %K patient screening %K electronic health record %K clinical research %D 2021 %7 21.10.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: The widespread adoption of electronic health records (EHRs) has facilitated the secondary use of EHR data for clinical research. However, screening eligible patients from EHRs is a challenging task. The concepts in eligibility criteria are not completely matched with EHRs, especially derived concepts. The lack of high-level expression of Structured Query Language (SQL) makes it difficult and time consuming to express them. The openEHR Expression Language (EL) as a domain-specific language based on clinical information models shows promise to represent complex eligibility criteria. Objective: The study aims to develop a patient-screening tool based on EHRs for clinical research using openEHR to solve concept mismatch and improve query performance. Methods: A patient-screening tool based on EHRs using openEHR was proposed. It uses the advantages of information models and EL in openEHR to provide high-level expressions and improve query performance. First, openEHR archetypes and templates were chosen to define concepts called simple concepts directly from EHRs. Second, openEHR EL was used to generate derived concepts by combining simple concepts and constraints. Third, a hierarchical index corresponding to archetypes in Elasticsearch (ES) was generated to improve query performance for subqueries and join queries related to the derived concepts. Finally, we realized a patient-screening tool for clinical research. Results: In total, 500 sentences randomly selected from 4691 eligibility criteria in 389 clinical trials on stroke from the Chinese Clinical Trial Registry (ChiCTR) were evaluated. An openEHR-based clinical data repository (CDR) in a grade A tertiary hospital in China was considered as an experimental environment. Based on these, 589 medical concepts were found in the 500 sentences. Of them, 513 (87.1%) concepts could be represented, while the others could not be, because of a lack of information models and coarse-grained requirements. In addition, our case study on 6 queries demonstrated that our tool shows better query performance among 4 cases (66.67%). Conclusions: We developed a patient-screening tool using openEHR. It not only helps solve concept mismatch but also improves query performance to reduce the burden on researchers. In addition, we demonstrated a promising solution for secondary use of EHR data using openEHR, which can be referenced by other researchers. %M 34673526 %R 10.2196/33192 %U https://medinform.jmir.org/2021/10/e33192 %U https://doi.org/10.2196/33192 %U http://www.ncbi.nlm.nih.gov/pubmed/34673526 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 10 %P e27261 %T Assessing Neonatal Intensive Care Unit Structures and Outcomes Before and During the COVID-19 Pandemic: Network Analysis Study %A Mannering,Hannah %A Yan,Chao %A Gong,Yang %A Alrifai,Mhd Wael %A France,Daniel %A Chen,You %+ Department of Biomedical Informatics, Vanderbilt University Medical Center, 2525 West End Ave, Suite 1475, Nashville, TN, 37203, United States, 1 6153431939, you.chen@vanderbilt.edu %K neonatal intensive care unit %K collaboration %K health care organization structures %K intensive care %K length of stay %K discharge dispositions %K electronic health records %K network analysis %K COVID-19 %K temporal network analysis %D 2021 %7 20.10.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Health care organizations (HCOs) adopt strategies (eg. physical distancing) to protect clinicians and patients in intensive care units (ICUs) during the COVID-19 pandemic. Many care activities physically performed before the COVID-19 pandemic have transitioned to virtual systems during the pandemic. These transitions can interfere with collaboration structures in the ICU, which may impact clinical outcomes. Understanding the differences can help HCOs identify challenges when transitioning physical collaboration to the virtual setting in the post–COVID-19 era. Objective: This study aims to leverage network analysis to determine the changes in neonatal ICU (NICU) collaboration structures from the pre– to the intra–COVID-19 era. Methods: In this retrospective study, we applied network analysis to the utilization of electronic health records (EHRs) of 712 critically ill neonates (pre–COVID-19, n=386; intra–COVID-19, n=326, excluding those with COVID-19) admitted to the NICU of Vanderbilt University Medical Center between September 1, 2019, and June 30, 2020, to assess collaboration between clinicians. We characterized pre–COVID-19 as the period of September-December 2019 and intra–COVID-19 as the period of March-June 2020. These 2 groups were compared using patients’ clinical characteristics, including age, sex, race, length of stay (LOS), and discharge dispositions. We leveraged the clinicians’ actions committed to the patients’ EHRs to measure clinician-clinician connections. We characterized a collaboration relationship (tie) between 2 clinicians as actioning EHRs of the same patient within the same day. On defining collaboration relationship, we built pre– and intra–COVID-19 networks. We used 3 sociometric measurements, including eigenvector centrality, eccentricity, and betweenness, to quantify a clinician’s leadership, collaboration difficulty, and broad skill sets in a network, respectively. We assessed the extent to which the eigenvector centrality, eccentricity, and betweenness of clinicians in the 2 networks are statistically different, using Mann-Whitney U tests (95% CI). Results: Collaboration difficulty increased from the pre– to intra–COVID-19 periods (median eccentricity: 3 vs 4; P<.001). Nurses had reduced leadership (median eigenvector centrality: 0.183 vs 0.087; P<.001), and neonatologists with broader skill sets cared for more patients in the NICU structure during the pandemic (median betweenness centrality: 0.0001 vs 0.005; P<.001). The pre– and intra–COVID-19 patient groups shared similar distributions in sex (~0 difference), race (4% difference in White, and 3% difference in African American), LOS (interquartile range difference in 1.5 days), and discharge dispositions (~0 difference in home, 2% difference in expired, and 2% difference in others). There were no significant differences in the patient demographics and outcomes between the 2 groups. Conclusions: Management of NICU-admitted patients typically requires multidisciplinary care teams. Understanding collaboration structures can provide fine-grained evidence to potentially refine or optimize existing teamwork in the NICU. %M 34637393 %R 10.2196/27261 %U https://www.jmir.org/2021/10/e27261 %U https://doi.org/10.2196/27261 %U http://www.ncbi.nlm.nih.gov/pubmed/34637393 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 10 %P e31288 %T Verifying the Feasibility of Implementing Semantic Interoperability in Different Countries Based on the OpenEHR Approach: Comparative Study of Acute Coronary Syndrome Registries %A Min,Lingtong %A Atalag,Koray %A Tian,Qi %A Chen,Yani %A Lu,Xudong %+ College of Biomedical Engineering & Instrument Science, Zhejiang University, Room 512, Zhouyiqing Building, 38 Zheda Road, Hangzhou, China, 86 13957118891, lvxd@zju.edu.cn %K semantic interoperability %K openEHR %K archetype %K registry %K acute coronary syndrome %D 2021 %7 19.10.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: The semantic interoperability of health care information has been a critical challenge in medical informatics and has influenced the integration, sharing, analysis, and use of medical big data. International standard organizations have developed standards, approaches, and models to improve and implement semantic interoperability. The openEHR approach—one of the standout semantic interoperability approaches—has been implemented worldwide to improve semantic interoperability based on reused archetypes. Objective: This study aimed to verify the feasibility of implementing semantic interoperability in different countries by comparing the openEHR-based information models of 2 acute coronary syndrome (ACS) registries from China and New Zealand. Methods: A semantic archetype comparison method was proposed to determine the semantics reuse degree of reused archetypes in 2 ACS-related clinical registries from 2 countries. This method involved (1) determining the scope of reused archetypes; (2) identifying corresponding data items within corresponding archetypes; (3) comparing the semantics of corresponding data items; and (4) calculating the number of mappings in corresponding data items and analyzing results. Results: Among the related archetypes in the two ACS-related, openEHR-based clinical registries from China and New Zealand, there were 8 pairs of reusable archetypes, which included 89 pairs of corresponding data items and 120 noncorresponding data items. Of the 89 corresponding data item pairs, 87 pairs (98%) were mappable and therefore supported semantic interoperability, and 71 pairs (80%) were labeled as “direct mapping” data items. Of the 120 noncorresponding data items, 114 (95%) data items were generated via archetype evolution, and 6 (5%) data items were generated via archetype localization. Conclusions: The results of the semantic comparison between the two ACS-related clinical registries prove the feasibility of establishing the semantic interoperability of health care data from different countries based on the openEHR approach. Archetype reuse provides data on the degree to which semantic interoperability exists when using the openEHR approach. Although the openEHR community has effectively promoted archetype reuse and semantic interoperability by providing archetype modeling methods, tools, model repositories, and archetype design patterns, the uncontrolled evolution of archetypes and inconsistent localization have resulted in major challenges for achieving higher levels of semantic interoperability. %M 34665150 %R 10.2196/31288 %U https://medinform.jmir.org/2021/10/e31288 %U https://doi.org/10.2196/31288 %U http://www.ncbi.nlm.nih.gov/pubmed/34665150 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 10 %P e32303 %T Harnessing the Electronic Health Record and Computerized Provider Order Entry Data for Resource Management During the COVID-19 Pandemic: Development of a Decision Tree %A Luu,Hung S %A Filkins,Laura M %A Park,Jason Y %A Rakheja,Dinesh %A Tweed,Jefferson %A Menzies,Christopher %A Wang,Vincent J %A Mittal,Vineeta %A Lehmann,Christoph U %A Sebert,Michael E %+ Department of Pathology, University of Texas Southwestern Medical Center, 1935 Medical District Drive, Dallas, TX, 75235, United States, 1 2144562168, hung.luu@childrens.com %K COVID-19 %K computerized provider order entry %K electronic health record %K resource utilization %K personal protective equipment %K SARS-CoV-2 testing %K clinical decision support %D 2021 %7 18.10.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: The COVID-19 pandemic has resulted in shortages of diagnostic tests, personal protective equipment, hospital beds, and other critical resources. Objective: We sought to improve the management of scarce resources by leveraging electronic health record (EHR) functionality, computerized provider order entry, clinical decision support (CDS), and data analytics. Methods: Due to the complex eligibility criteria for COVID-19 tests and the EHR implementation–related challenges of ordering these tests, care providers have faced obstacles in selecting the appropriate test modality. As test choice is dependent upon specific patient criteria, we built a decision tree within the EHR to automate the test selection process by using a branching series of questions that linked clinical criteria to the appropriate SARS-CoV-2 test and triggered an EHR flag for patients who met our institutional persons under investigation criteria. Results: The percentage of tests that had to be canceled and reordered due to errors in selecting the correct testing modality was 3.8% (23/608) before CDS implementation and 1% (262/26,643) after CDS implementation (P<.001). Patients for whom multiple tests were ordered during a 24-hour period accounted for 0.8% (5/608) and 0.3% (76/26,643) of pre- and post-CDS implementation orders, respectively (P=.03). Nasopharyngeal molecular assay results were positive in 3.4% (826/24,170) of patients who were classified as asymptomatic and 10.9% (1421/13,074) of symptomatic patients (P<.001). Positive tests were more frequent among asymptomatic patients with a history of exposure to COVID-19 (36/283, 12.7%) than among asymptomatic patients without such a history (790/23,887, 3.3%; P<.001). Conclusions: The leveraging of EHRs and our CDS algorithm resulted in a decreased incidence of order entry errors and the appropriate flagging of persons under investigation. These interventions optimized reagent and personal protective equipment usage. Data regarding symptoms and COVID-19 exposure status that were collected by using the decision tree correlated with the likelihood of positive test results, suggesting that clinicians appropriately used the questions in the decision tree algorithm. %M 34546942 %R 10.2196/32303 %U https://medinform.jmir.org/2021/10/e32303 %U https://doi.org/10.2196/32303 %U http://www.ncbi.nlm.nih.gov/pubmed/34546942 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 10 %P e29871 %T Data Anonymization for Pervasive Health Care: Systematic Literature Mapping Study %A Zuo,Zheming %A Watson,Matthew %A Budgen,David %A Hall,Robert %A Kennelly,Chris %A Al Moubayed,Noura %+ Department of Computer Science, Durham University, Lower Mountjoy, South Rd, Durham, DH1 3LE, United Kingdom, 44 1913341749, Noura.al-moubayed@durham.ac.uk %K healthcare %K privacy-preserving %K GDPR %K DPA 2018 %K EHR %K SLM %K data science %K anonymization %K reidentification risk %K usability %D 2021 %7 15.10.2021 %9 Review %J JMIR Med Inform %G English %X Background: Data science offers an unparalleled opportunity to identify new insights into many aspects of human life with recent advances in health care. Using data science in digital health raises significant challenges regarding data privacy, transparency, and trustworthiness. Recent regulations enforce the need for a clear legal basis for collecting, processing, and sharing data, for example, the European Union’s General Data Protection Regulation (2016) and the United Kingdom’s Data Protection Act (2018). For health care providers, legal use of the electronic health record (EHR) is permitted only in clinical care cases. Any other use of the data requires thoughtful considerations of the legal context and direct patient consent. Identifiable personal and sensitive information must be sufficiently anonymized. Raw data are commonly anonymized to be used for research purposes, with risk assessment for reidentification and utility. Although health care organizations have internal policies defined for information governance, there is a significant lack of practical tools and intuitive guidance about the use of data for research and modeling. Off-the-shelf data anonymization tools are developed frequently, but privacy-related functionalities are often incomparable with regard to use in different problem domains. In addition, tools to support measuring the risk of the anonymized data with regard to reidentification against the usefulness of the data exist, but there are question marks over their efficacy. Objective: In this systematic literature mapping study, we aim to alleviate the aforementioned issues by reviewing the landscape of data anonymization for digital health care. Methods: We used Google Scholar, Web of Science, Elsevier Scopus, and PubMed to retrieve academic studies published in English up to June 2020. Noteworthy gray literature was also used to initialize the search. We focused on review questions covering 5 bottom-up aspects: basic anonymization operations, privacy models, reidentification risk and usability metrics, off-the-shelf anonymization tools, and the lawful basis for EHR data anonymization. Results: We identified 239 eligible studies, of which 60 were chosen for general background information; 16 were selected for 7 basic anonymization operations; 104 covered 72 conventional and machine learning–based privacy models; four and 19 papers included seven and 15 metrics, respectively, for measuring the reidentification risk and degree of usability; and 36 explored 20 data anonymization software tools. In addition, we also evaluated the practical feasibility of performing anonymization on EHR data with reference to their usability in medical decision-making. Furthermore, we summarized the lawful basis for delivering guidance on practical EHR data anonymization. Conclusions: This systematic literature mapping study indicates that anonymization of EHR data is theoretically achievable; yet, it requires more research efforts in practical implementations to balance privacy preservation and usability to ensure more reliable health care applications. %M 34652278 %R 10.2196/29871 %U https://medinform.jmir.org/2021/10/e29871 %U https://doi.org/10.2196/29871 %U http://www.ncbi.nlm.nih.gov/pubmed/34652278 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 10 %P e29174 %T Building a Shared, Scalable, and Sustainable Source for the Problem-Oriented Medical Record: Developmental Study %A Gaudet-Blavignac,Christophe %A Rudaz,Andrea %A Lovis,Christian %+ Division of Medical Information Sciences, Geneva University Hospitals, Rue Gabrielle-Perret-Gentil 4, Geneva, 1205, Switzerland, 41 223726201, christophe.gaudet-blavignac@hcuge.ch %K medical records %K problem-oriented %K electronic health records %K semantics %D 2021 %7 13.10.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Since the creation of the problem-oriented medical record, the building of problem lists has been the focus of many studies. To date, this issue is not well resolved, and building an appropriate contextualized problem list is still a challenge. Objective: This paper aims to present the process of building a shared multipurpose common problem list at the Geneva University Hospitals. This list aims to bridge the gap between clinicians’ language expressed in free text and secondary uses requiring structured information. Methods: We focused on the needs of clinicians by building a list of uniquely identified expressions to support their daily activities. In the second stage, these expressions were connected to additional information to build a complex graph of information. A list of 45,946 expressions manually extracted from clinical documents was manually curated and encoded in multiple semantic dimensions, such as International Classification of Diseases, 10th revision; International Classification of Primary Care 2nd edition; Systematized Nomenclature of Medicine Clinical Terms; or dimensions dictated by specific usages, such as identifying expressions specific to a domain, a gender, or an intervention. The list was progressively deployed for clinicians with an iterative process of quality control, maintenance, and improvements, including the addition of new expressions or dimensions for specific needs. The problem management of the electronic health record allowed the measurement and correction of encoding based on real-world use. Results: The list was deployed in production in January 2017 and was regularly updated and deployed in new divisions of the hospital. Over 4 years, 684,102 problems were created using the list. The proportion of free-text entries decreased progressively from 37.47% (8321/22,206) in December 2017 to 18.38% (4547/24,738) in December 2020. In the last version of the list, over 14 dimensions were mapped to expressions, among which 5 were international classifications and 8 were other classifications for specific uses. The list became a central axis in the electronic health record, being used for many different purposes linked to care, such as surgical planning or emergency wards, or in research, for various predictions using machine learning techniques. Conclusions: This study breaks with common approaches primarily by focusing on real clinicians’ language when expressing patients’ problems and secondarily by mapping whatever is required, including controlled vocabularies to answer specific needs. This approach improves the quality of the expression of patients’ problems while allowing the building of as many structured dimensions as needed to convey semantics according to specific contexts. The method is shown to be scalable, sustainable, and efficient at hiding the complexity of semantics or the burden of constraint-structured problem list entry for clinicians. Ongoing work is analyzing the impact of this approach on how clinicians express patients’ problems. %M 34643542 %R 10.2196/29174 %U https://medinform.jmir.org/2021/10/e29174 %U https://doi.org/10.2196/29174 %U http://www.ncbi.nlm.nih.gov/pubmed/34643542 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 5 %N 10 %P e26314 %T Using a Constraint-Based Method to Identify Chronic Disease Patients Who Are Apt to Obtain Care Mostly Within a Given Health Care System: Retrospective Cohort Study %A Tong,Yao %A Liao,Zachary C %A Tarczy-Hornoch,Peter %A Luo,Gang %+ Department of Biomedical Informatics and Medical Education, University of Washington, UW Medicine South Lake Union, 850 Republican Street, Building C, Box 358047, Seattle, WA, 98195, United States, 1 206 221 4596, gangluo@cs.wisc.edu %K asthma %K chronic kidney disease %K chronic obstructive pulmonary disease %K data analysis %K diabetes mellitus %K emergency department %K health care system %K inpatients %K patient care management %D 2021 %7 7.10.2021 %9 Original Paper %J JMIR Form Res %G English %X Background: For several major chronic diseases including asthma, chronic obstructive pulmonary disease, chronic kidney disease, and diabetes, a state-of-the-art method to avert poor outcomes is to use predictive models to identify future high-cost patients for preemptive care management interventions. Frequently, an American patient obtains care from multiple health care systems, each managed by a distinct institution. As the patient’s medical data are spread across these health care systems, none has complete medical data for the patient. The task of building models to predict an individual patient’s cost is currently thought to be impractical with incomplete data, which limits the use of care management to improve outcomes. Recently, we developed a constraint-based method to identify patients who are apt to obtain care mostly within a given health care system. Our method was shown to work well for the cohort of all adult patients at the University of Washington Medicine for a 6-month follow-up period. It is unknown how well our method works for patients with various chronic diseases and over follow-up periods of different lengths, and subsequently, whether it is reasonable to perform this predictive modeling task on the subset of patients pinpointed by our method. Objective: To understand our method’s potential to enable this predictive modeling task on incomplete medical data, this study assesses our method’s performance at the University of Washington Medicine on 5 subgroups of adult patients with major chronic diseases and over follow-up periods of 2 different lengths. Methods: We used University of Washington Medicine data for all adult patients who obtained care at the University of Washington Medicine in 2018 and PreManage data containing usage information from all hospitals in Washington state in 2019. We evaluated our method’s performance over the follow-up periods of 6 months and 12 months on 5 patient subgroups separately—asthma, chronic kidney disease, type 1 diabetes, type 2 diabetes, and chronic obstructive pulmonary disease. Results: Our method identified 21.81% (3194/14,644) of University of Washington Medicine adult patients with asthma. Around 66.75% (797/1194) and 67.13% (1997/2975) of their emergency department visits and inpatient stays took place within the University of Washington Medicine system in the subsequent 6 months and in the subsequent 12 months, respectively, approximately double the corresponding percentage for all University of Washington Medicine adult patients with asthma. The performance for adult patients with chronic kidney disease, adult patients with chronic obstructive pulmonary disease, adult patients with type 1 diabetes, and adult patients with type 2 diabetes was reasonably similar to that for adult patients with asthma. Conclusions: For each of the 5 chronic diseases most relevant to care management, our method can pinpoint a reasonably large subset of patients who are apt to obtain care mostly within the University of Washington Medicine system. This opens the door to building models to predict an individual patient’s cost on incomplete data, which was formerly deemed impractical. International Registered Report Identifier (IRRID): RR2-10.2196/13783 %M 34617906 %R 10.2196/26314 %U https://formative.jmir.org/2021/10/e26314 %U https://doi.org/10.2196/26314 %U http://www.ncbi.nlm.nih.gov/pubmed/34617906 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 10 %P e30165 %T Writing Practices Associated With Electronic Progress Notes and the Preferences of Those Who Read Them: Descriptive Study %A Payne,Thomas H %A Keller,Carolyn %A Arora,Pallavi %A Brusati,Allison %A Levin,Jesse %A Salgaonkar,Monica %A Li,Xi %A Zech,Jennifer %A Lees,A Fischer %+ Department of Medicine, University of Washington School of Medicine, Box 359780, 325 Ninth Ave, Seattle, WA, 98104-2499, United States, 1 206 744 1824, tpayne@uw.edu %K electronic documentation %K electronic health records %K hospital progress notes %K copy-paste %K EHR %K patient records %K workflow %K human factors %K clinical communication %K physician communication %K hospital %D 2021 %7 6.10.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Hospital progress notes can serve as an important communication tool. However, they are criticized for their length, preserved content, and for the time physicians spend writing them. Objective: We aimed to describe hospital progress note content, writing and reading practices, and the preferences of those who create and read them prior to the implementation of a new electronic health record system. Methods: Using a sample of hospital progress notes from 1000 randomly selected admissions, we measured note length, similarity of content in successive daily notes for the same patient, the time notes were signed and read, and who read them. We conducted focus group sessions with note writers, readers, and clinical leaders to understand their preferences. Results: We analyzed 4938 inpatient progress notes from 418 authors. The average length was 886 words, and most were in the Assessment & Plan note section. A total of 29% of notes (n=1432) were signed after 4 PM. Notes signed later in the day were read less often. Notes were highly similar from one day to the next, and 26% (23/88) had clinical risk associated with the preserved content. Note content of the highest value varied according to the reader’s professional role. Conclusions: Progress note length varied widely. Notes were often signed late in the day when they were read less often and were highly similar to the note from the previous day. Measuring note length, signing time, when and by whom notes are read, and the amount and safety of preserved content will be useful metrics for measuring how the new electronic health record system is used, and can aid improvements. %M 34612825 %R 10.2196/30165 %U https://www.jmir.org/2021/10/e30165 %U https://doi.org/10.2196/30165 %U http://www.ncbi.nlm.nih.gov/pubmed/34612825 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 10 %P e31980 %T Expressiveness of an International Semantic Standard for Wound Care: Mapping a Standardized Item Set for Leg Ulcers to the Systematized Nomenclature of Medicine–Clinical Terms %A Hüsers,Jens %A Przysucha,Mareike %A Esdar,Moritz %A John,Swen Malte %A Hübner,Ursula Hertha %+ University of Applied Sciences Osnabrück, Albrechtstr 30, Osnabrück, 49076, Germany, 49 5419692012, u.huebner@hs-osnabrueck.de %K wound care %K chronic wound %K chronic leg ulcer %K SNOMED CT %K health information exchange %K semantic interoperability %K terminology mapping %D 2021 %7 6.10.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Chronic health conditions are on the rise and are putting high economic pressure on health systems, as they require well-coordinated prevention and treatment. Among chronic conditions, chronic wounds such as cardiovascular leg ulcers have a high prevalence. Their treatment is highly interdisciplinary and regularly spans multiple care settings and organizations; this places particularly high demands on interoperable information exchange that can be achieved using international semantic standards, such as Systematized Nomenclature of Medicine–Clinical Terms (SNOMED CT). Objective: This study aims to investigate the expressiveness of SNOMED CT in the domain of wound care, and thereby its clinical usefulness and the potential need for extensions. Methods: A clinically consented and profession-independent wound care item set, the German National Consensus for the Documentation of Leg Wounds (NKDUC), was mapped onto the precoordinated concepts of the international reference terminology SNOMED CT. Before the mapping took place, the NKDUC was transformed into an information model that served to systematically identify relevant items. The mapping process was carried out in accordance with the ISO/TR 12300 formalism. As a result, the reliability, equivalence, and coverage rate were determined for all NKDUC items and sections. Results: The developed information model revealed 268 items to be mapped. Conducted by 3 health care professionals, the mapping resulted in moderate reliability (κ=0.512). Regarding the two best equivalence categories (symmetrical equivalence of meaning), the coverage rate of SNOMED CT was 67.2% (180/268) overall and 64.3% (108/168) specifically for wounds. The sections general medical condition (55/66, 83%), wound assessment (18/24, 75%), and wound status (37/57, 65%), showed higher coverage rates compared with the sections therapy (45/73, 62%), wound diagnostics (8/14, 57%), and patient demographics (17/34, 50%). Conclusions: The results yielded acceptable reliability values for the mapping procedure. The overall coverage rate shows that two-thirds of the items could be mapped symmetrically, which is a substantial portion of the source item set. Some wound care sections, such as general medical conditions and wound assessment, were covered better than other sections (wound status, diagnostics, and therapy). These deficiencies can be mitigated either by postcoordination or by the inclusion of new concepts in SNOMED CT. This study contributes to pushing interoperability in the domain of wound care, thereby responding to the high demand for information exchange in this field. Overall, this study adds another puzzle piece to the general knowledge about SNOMED CT in terms of its clinical usefulness and its need for further extensions. %M 34428171 %R 10.2196/31980 %U https://medinform.jmir.org/2021/10/e31980 %U https://doi.org/10.2196/31980 %U http://www.ncbi.nlm.nih.gov/pubmed/34428171 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 10 %P e30697 %T The National COVID Cohort Collaborative: Analyses of Original and Computationally Derived Electronic Health Record Data %A Foraker,Randi %A Guo,Aixia %A Thomas,Jason %A Zamstein,Noa %A Payne,Philip RO %A Wilcox,Adam %A , %+ Division of General Medical Sciences, School of Medicine, Washington University in St. Louis, 600 S. Taylor Avenue, Suite 102, Campus Box 8102, St. Louis, MO, 63110, United States, 1 314 273 2211, randi.foraker@wustl.edu %K synthetic data %K protected health information %K COVID-19 %K electronic health records and systems %K data analysis %D 2021 %7 4.10.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Computationally derived (“synthetic”) data can enable the creation and analysis of clinical, laboratory, and diagnostic data as if they were the original electronic health record data. Synthetic data can support data sharing to answer critical research questions to address the COVID-19 pandemic. Objective: We aim to compare the results from analyses of synthetic data to those from original data and assess the strengths and limitations of leveraging computationally derived data for research purposes. Methods: We used the National COVID Cohort Collaborative’s instance of MDClone, a big data platform with data-synthesizing capabilities (MDClone Ltd). We downloaded electronic health record data from 34 National COVID Cohort Collaborative institutional partners and tested three use cases, including (1) exploring the distributions of key features of the COVID-19–positive cohort; (2) training and testing predictive models for assessing the risk of admission among these patients; and (3) determining geospatial and temporal COVID-19–related measures and outcomes, and constructing their epidemic curves. We compared the results from synthetic data to those from original data using traditional statistics, machine learning approaches, and temporal and spatial representations of the data. Results: For each use case, the results of the synthetic data analyses successfully mimicked those of the original data such that the distributions of the data were similar and the predictive models demonstrated comparable performance. Although the synthetic and original data yielded overall nearly the same results, there were exceptions that included an odds ratio on either side of the null in multivariable analyses (0.97 vs 1.01) and differences in the magnitude of epidemic curves constructed for zip codes with low population counts. Conclusions: This paper presents the results of each use case and outlines key considerations for the use of synthetic data, examining their role in collaborative research for faster insights. %M 34559671 %R 10.2196/30697 %U https://www.jmir.org/2021/10/e30697 %U https://doi.org/10.2196/30697 %U http://www.ncbi.nlm.nih.gov/pubmed/34559671 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 10 %P e27177 %T Use of Deep Learning to Predict Acute Kidney Injury After Intravenous Contrast Media Administration: Prediction Model Development Study %A Yun,Donghwan %A Cho,Semin %A Kim,Yong Chul %A Kim,Dong Ki %A Oh,Kook-Hwan %A Joo,Kwon Wook %A Kim,Yon Su %A Han,Seung Seok %+ Department of Biomedical Sciences, Seoul National University College of Medicine, 103 Daehakro, Jongno-gu, Seoul, 03080, Republic of Korea, 82 2 2072 4785 ext 8095, hansway80@gmail.com %K acute kidney injury %K artificial intelligence %K contrast media %K deep learning %K machine learning %K kidney injury %K computed tomography %D 2021 %7 1.10.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Precise prediction of contrast media–induced acute kidney injury (CIAKI) is an important issue because of its relationship with poor outcomes. Objective: Herein, we examined whether a deep learning algorithm could predict the risk of intravenous CIAKI better than other machine learning and logistic regression models in patients undergoing computed tomography (CT). Methods: A total of 14,185 patients who were administered intravenous contrast media for CT at the preventive and monitoring facility in Seoul National University Hospital were reviewed. CIAKI was defined as an increase in serum creatinine of ≥0.3 mg/dL within 2 days or ≥50% within 7 days. Using both time-varying and time-invariant features, machine learning models, such as the recurrent neural network (RNN), light gradient boosting machine (LGM), extreme gradient boosting machine (XGB), random forest (RF), decision tree (DT), support vector machine (SVM), κ-nearest neighbors, and logistic regression, were developed using a training set, and their performance was compared using the area under the receiver operating characteristic curve (AUROC) in a test set. Results: CIAKI developed in 261 cases (1.8%). The RNN model had the highest AUROC of 0.755 (0.708-0.802) for predicting CIAKI, which was superior to that obtained from other machine learning models. Although CIAKI was defined as an increase in serum creatinine of ≥0.5 mg/dL or ≥25% within 3 days, the highest performance was achieved in the RNN model with an AUROC of 0.716 (95% confidence interval [CI] 0.664-0.768). In feature ranking analysis, the albumin level was the most highly contributing factor to RNN performance, followed by time-varying kidney function. Conclusions: Application of a deep learning algorithm improves the predictability of intravenous CIAKI after CT, representing a basis for future clinical alarming and preventive systems. %M 34596574 %R 10.2196/27177 %U https://medinform.jmir.org/2021/10/e27177 %U https://doi.org/10.2196/27177 %U http://www.ncbi.nlm.nih.gov/pubmed/34596574 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 9 %P e31122 %T Columbia Open Health Data for COVID-19 Research: Database Analysis %A Lee,Junghwan %A Kim,Jae Hyun %A Liu,Cong %A Hripcsak,George %A Natarajan,Karthik %A Ta,Casey %A Weng,Chunhua %+ Columbia University, Ph-20, 622 W 168 ST, New York, NY, United States, 1 212 304 7907, cw2384@cumc.columbia.edu %K COVID-19 %K open data %K electronic health record %K data science %K research %K data %K access %K database %K symptom %K cohort %K prevalence %D 2021 %7 30.9.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: COVID-19 has threatened the health of tens of millions of people all over the world. Massive research efforts have been made in response to the COVID-19 pandemic. Utilization of clinical data can accelerate these research efforts to combat the pandemic since important characteristics of the patients are often found by examining the clinical data. Publicly accessible clinical data on COVID-19, however, remain limited despite the immediate need. Objective: To provide shareable clinical data to catalyze COVID-19 research, we present Columbia Open Health Data for COVID-19 Research (COHD-COVID), a publicly accessible database providing clinical concept prevalence, clinical concept co-occurrence, and clinical symptom prevalence for hospitalized patients with COVID-19. COHD-COVID also provides data on hospitalized patients with influenza and general hospitalized patients as comparator cohorts. Methods: The data used in COHD-COVID were obtained from NewYork-Presbyterian/Columbia University Irving Medical Center’s electronic health records database. Condition, drug, and procedure concepts were obtained from the visits of identified patients from the cohorts. Rare concepts were excluded, and the true concept counts were perturbed using Poisson randomization to protect patient privacy. Concept prevalence, concept prevalence ratio, concept co-occurrence, and symptom prevalence were calculated using the obtained concepts. Results: Concept prevalence and concept prevalence ratio analyses showed the clinical characteristics of the COVID-19 cohorts, confirming the well-known characteristics of COVID-19 (eg, acute lower respiratory tract infection and cough). The concepts related to the well-known characteristics of COVID-19 recorded high prevalence and high prevalence ratio in the COVID-19 cohort compared to the hospitalized influenza cohort and general hospitalized cohort. Concept co-occurrence analyses showed potential associations between specific concepts. In case of acute lower respiratory tract infection in the COVID-19 cohort, a high co-occurrence ratio was obtained with COVID-19–related concepts and commonly used drugs (eg, disease due to coronavirus and acetaminophen). Symptom prevalence analysis indicated symptom-level characteristics of the cohorts and confirmed that well-known symptoms of COVID-19 (eg, fever, cough, and dyspnea) showed higher prevalence than the hospitalized influenza cohort and the general hospitalized cohort. Conclusions: We present COHD-COVID, a publicly accessible database providing useful clinical data for hospitalized patients with COVID-19, hospitalized patients with influenza, and general hospitalized patients. We expect COHD-COVID to provide researchers and clinicians quantitative measures of COVID-19–related clinical features to better understand and combat the pandemic. %M 34543225 %R 10.2196/31122 %U https://www.jmir.org/2021/9/e31122 %U https://doi.org/10.2196/31122 %U http://www.ncbi.nlm.nih.gov/pubmed/34543225 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 9 %P e21990 %T Forecasting the Requirement for Nonelective Hospital Beds in the National Health Service of the United Kingdom: Model Development Study %A Shah,Kanan %A Sharma,Akarsh %A Moulton,Chris %A Swift,Simon %A Mann,Clifford %A Jones,Simon %+ Division of Healthcare Delivery Science, Department of Population Health, NYU Grossman School of Medicine, 227 E 30th St, New York, NY, 10016, United States, 1 646 501 2905, simon.jones@nyulangone.org %K bed occupancy %K clinical decision-making %K forecasting %K health care delivery %K models %K time-series analysis %D 2021 %7 30.9.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Over the last decade, increasing numbers of emergency department attendances and an even greater increase in emergency admissions have placed severe strain on the bed capacity of the National Health Service (NHS) of the United Kingdom. The result has been overcrowded emergency departments with patients experiencing long wait times for admission to an appropriate hospital bed. Nevertheless, scheduling issues can still result in significant underutilization of bed capacity. Bed occupancy rates may not correlate well with bed availability. More accurate and reliable long-term prediction of bed requirements will help anticipate the future needs of a hospital’s catchment population, thus resulting in greater efficiencies and better patient care. Objective: This study aimed to evaluate widely used automated time-series forecasting techniques to predict short-term daily nonelective bed occupancy at all trusts in the NHS. These techniques were used to develop a simple yet accurate national health system–level forecasting framework that can be utilized at a low cost and by health care administrators who do not have statistical modeling expertise. Methods: Bed occupancy models that accounted for patterns in occupancy were created for each trust in the NHS. Daily nonelective midnight trust occupancy data from April 2011 to March 2017 for 121 NHS trusts were utilized to generate these models. Forecasts were generated using the three most widely used automated forecasting techniques: exponential smoothing; Seasonal Autoregressive Integrated Moving Average; and Trigonometric, Box-Cox transform, autoregressive moving average errors, and Trend and Seasonal components. The NHS Modernisation Agency’s recommended forecasting method prior to 2020 was also replicated. Results: The accuracy of the models varied on the basis of the season during which occupancy was forecasted. For the summer season, percent root-mean-square error values for each model remained relatively stable across the 6 forecasted weeks. However, only the trend and seasonal components model (median error=2.45% for 6 weeks) outperformed the NHS Modernisation Agency’s recommended method (median error=2.63% for 6 weeks). In contrast, during the winter season, the percent root-mean-square error values increased as we forecasted further into the future. Exponential smoothing generated the most accurate forecasts (median error=4.91% over 4 weeks), but all models outperformed the NHS Modernisation Agency’s recommended method prior to 2020 (median error=8.5% over 4 weeks). Conclusions: It is possible to create automated models, similar to those recently published by the NHS, which can be used at a hospital level for a large national health care system to predict nonelective bed admissions and thus schedule elective procedures. %M 34591020 %R 10.2196/21990 %U https://medinform.jmir.org/2021/9/e21990 %U https://doi.org/10.2196/21990 %U http://www.ncbi.nlm.nih.gov/pubmed/34591020 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 9 %P e30157 %T COVID-19 Mortality Prediction From Deep Learning in a Large Multistate Electronic Health Record and Laboratory Information System Data Set: Algorithm Development and Validation %A Sankaranarayanan,Saranya %A Balan,Jagadheshwar %A Walsh,Jesse R %A Wu,Yanhong %A Minnich,Sara %A Piazza,Amy %A Osborne,Collin %A Oliver,Gavin R %A Lesko,Jessica %A Bates,Kathy L %A Khezeli,Kia %A Block,Darci R %A DiGuardo,Margaret %A Kreuter,Justin %A O’Horo,John C %A Kalantari,John %A Klee,Eric W %A Salama,Mohamed E %A Kipp,Benjamin %A Morice,William G %A Jenkinson,Garrett %+ Mayo Clinic, 200 1st St SW, Rochester, MN, 55905, United States, 1 507 293 9457, Jenkinson.William@mayo.edu %K COVID-19 %K mortality %K prediction %K recurrent neural networks %K missing data %K time series %K deep learning %K machine learning %K neural network %K electronic health record %K EHR %K algorithm %K development %K validation %D 2021 %7 28.9.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: COVID-19 is caused by the SARS-CoV-2 virus and has strikingly heterogeneous clinical manifestations, with most individuals contracting mild disease but a substantial minority experiencing fulminant cardiopulmonary symptoms or death. The clinical covariates and the laboratory tests performed on a patient provide robust statistics to guide clinical treatment. Deep learning approaches on a data set of this nature enable patient stratification and provide methods to guide clinical treatment. Objective: Here, we report on the development and prospective validation of a state-of-the-art machine learning model to provide mortality prediction shortly after confirmation of SARS-CoV-2 infection in the Mayo Clinic patient population. Methods: We retrospectively constructed one of the largest reported and most geographically diverse laboratory information system and electronic health record of COVID-19 data sets in the published literature, which included 11,807 patients residing in 41 states of the United States of America and treated at medical sites across 5 states in 3 time zones. Traditional machine learning models were evaluated independently as well as in a stacked learner approach by using AutoGluon, and various recurrent neural network architectures were considered. The traditional machine learning models were implemented using the AutoGluon-Tabular framework, whereas the recurrent neural networks utilized the TensorFlow Keras framework. We trained these models to operate solely using routine laboratory measurements and clinical covariates available within 72 hours of a patient’s first positive COVID-19 nucleic acid test result. Results: The GRU-D recurrent neural network achieved peak cross-validation performance with 0.938 (SE 0.004) as the area under the receiver operating characteristic (AUROC) curve. This model retained strong performance by reducing the follow-up time to 12 hours (0.916 [SE 0.005] AUROC), and the leave-one-out feature importance analysis indicated that the most independently valuable features were age, Charlson comorbidity index, minimum oxygen saturation, fibrinogen level, and serum iron level. In the prospective testing cohort, this model provided an AUROC of 0.901 and a statistically significant difference in survival (P<.001, hazard ratio for those predicted to survive, 95% CI 0.043-0.106). Conclusions: Our deep learning approach using GRU-D provides an alert system to flag mortality for COVID-19–positive patients by using clinical covariates and laboratory values within a 72-hour window after the first positive nucleic acid test result. %M 34449401 %R 10.2196/30157 %U https://www.jmir.org/2021/9/e30157 %U https://doi.org/10.2196/30157 %U http://www.ncbi.nlm.nih.gov/pubmed/34449401 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 9 %P e15739 %T Exploring the Use of Genomic and Routinely Collected Data: Narrative Literature Review and Interview Study %A Daniels,Helen %A Jones,Kerina Helen %A Heys,Sharon %A Ford,David Vincent %+ Population Data Science, Swansea University, Singleton Park, Swansea, SA2 8PP, United Kingdom, 44 01792606572, h.daniels@swansea.ac.uk %K genomic data %K routine data %K electronic health records %K health data science %K genome %K data regulation %K case study %K eHealth %D 2021 %7 24.9.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Advancing the use of genomic data with routinely collected health data holds great promise for health care and research. Increasing the use of these data is a high priority to understand and address the causes of disease. Objective: This study aims to provide an outline of the use of genomic data alongside routinely collected data in health research to date. As this field prepares to move forward, it is important to take stock of the current state of play in order to highlight new avenues for development, identify challenges, and ensure that adequate data governance models are in place for safe and socially acceptable progress. Methods: We conducted a literature review to draw information from past studies that have used genomic and routinely collected data and conducted interviews with individuals who use these data for health research. We collected data on the following: the rationale of using genomic data in conjunction with routinely collected data, types of genomic and routinely collected data used, data sources, project approvals, governance and access models, and challenges encountered. Results: The main purpose of using genomic and routinely collected data was to conduct genome-wide and phenome-wide association studies. Routine data sources included electronic health records, disease and death registries, health insurance systems, and deprivation indices. The types of genomic data included polygenic risk scores, single nucleotide polymorphisms, and measures of genetic activity, and biobanks generally provided these data. Although the literature search showed that biobanks released data to researchers, the case studies revealed a growing tendency for use within a data safe haven. Challenges of working with these data revolved around data collection, data storage, technical, and data privacy issues. Conclusions: Using genomic and routinely collected data holds great promise for progressing health research. Several challenges are involved, particularly in terms of privacy. Overcoming these barriers will ensure that the use of these data to progress health research can be exploited to its full potential. %M 34559060 %R 10.2196/15739 %U https://www.jmir.org/2021/9/e15739 %U https://doi.org/10.2196/15739 %U http://www.ncbi.nlm.nih.gov/pubmed/34559060 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 9 %P e27602 %T Versatile GCH Control Software for Correction of Loads Applied to Forearm Crutches During Gait Recovery Through Technological Feedback: Development and Implementation Study %A Chamorro-Moriana,Gema %A Sevillano,Jose Luis %A Perez-Cabezas,V %+ Department of Physiotherapy, Area of Physiotherapy Research Group CTS-305, University of Seville, Calle Avicena, S/N, Seville, 41009, Spain, 34 954486554, gchamorro@us.es %K control and monitoring software %K feedback technology %K motor control %K gait %K crutches %K assisted gait for partial weight-bearing %K functional recovery of the gait %K unloading of lower limb musculoskeletal injury %K rehabilitation %K physical therapy %K lower limb %K injury %K injuries %K feedback technology %K crutches %D 2021 %7 22.9.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Measuring weight bearing is an essential aspect of clinical care for lower limb injuries such as sprains or meniscopathy surgeries. This care often involves the use of forearm crutches for partial loads progressing to full loads. Therefore, feasible methods of load monitoring for daily clinical use are needed. Objective: The main objective of this study was to design an innovative multifunctional desktop load-measuring software that complements GCH System 2.0–instrumented forearm crutches and monitors the applied loads, displaying real-time graphical and numerical information, and enabling the correction of inaccuracies through feedback technology during assisted gait. The secondary objective was to perform a preliminary implementation trial. Methods: The software was designed for indoor use (clinics/laboratories). This software translates the crutch sensor signal in millivolts into force units, records and analyzes data (10-80 Hz), and provides real-time effective curves of the loads exerted on crutches. It covers numerous types of extrinsic feedback, including visual, acoustic (verbal/beeps), concurrent, terminal, and descriptive feedback, and includes a clinical and research use database. An observational descriptive pilot study was performed with 10 healthy subjects experienced in bilateral assisted gait. The Wilcoxon matched-pairs signed-rank test was used to evaluate the load accuracy evolution of each subject (ie, changes in the loads exerted on crutches for each support) among various walks, which was interpreted at the 95% confidence level. Results: GCH Control Software was developed as a multifunctional desktop tool complementing GCH System 2.0–instrumented forearm crutches. The pilot implementation of the feedback mechanism observed 96/100 load errors at baseline (walk 0, no feedback) with 7/10 subjects exhibiting crutch overloading. Errors ranged from 61.09% to 203.98%, demonstrating heterogeneity. The double-bar feedback found 54/100 errors in walk 1, 28/100 in walk 2, and 14/100 in walk 3. The first walk with double-bar feedback (walk 1) began with errors similar to the baseline walk, generally followed by attempts at correction. The Wilcoxon matched-pairs signed-rank test used to evaluate each subject’s progress showed that all participants steadily improved the accuracy of the loads applied to the crutches. In particular, Subject 9 required extra feedback with two single-bar walks to focus on the total load. The participants also corrected the load balance between crutches and fluency errors. Three subjects made one error of load balance and one subject made six fluctuation errors during the three double-bar walks. The latter subject performed additional feedback with two balance-bar walks to focus on the load balance. Conclusions: GCH Control Software proved to be useful for monitoring the loads exerted on forearm crutches, providing a variety of feedback for correcting load accuracy, load balance between crutches, and fluency. The findings of the complementary implementation were satisfactory, although clinical trials with larger samples are needed to assess the efficacy of the different feedback mechanisms and to select the best alternatives in each case. %M 34550073 %R 10.2196/27602 %U https://www.jmir.org/2021/9/e27602 %U https://doi.org/10.2196/27602 %U http://www.ncbi.nlm.nih.gov/pubmed/34550073 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 9 %P e29678 %T A Fully Automated Analytic System for Measuring Endolymphatic Hydrops Ratios in Patients With Ménière Disease via Magnetic Resonance Imaging: Deep Learning Model Development Study %A Park,Chae Jung %A Cho,Young Sang %A Chung,Myung Jin %A Kim,Yi-Kyung %A Kim,Hyung-Jin %A Kim,Kyunga %A Ko,Jae-Wook %A Chung,Won-Ho %A Cho,Baek Hwan %+ Department of Medical Device Management and Research, Samsung Advanced Institute for Health Sciences & Technology, Sungkyunkwan University, 81 Irwon-ro, Gangnam-gu, Seoul, 06355, Republic of Korea, 82 234100885, baekhwan.cho@samsung.com %K deep learning %K magnetic resonance imaging %K medical image segmentation %K Ménière disease %K inner ear %K endolymphatic hydrops %K artificial intelligence %K machine learning %K multi-class segmentation %K convolutional neural network %K end-to-end system %K clinician support %K clinical decision support system %K image selection %K clinical usability %K automation %D 2021 %7 21.9.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Recently, the analysis of endolymphatic hydropses (EHs) via inner ear magnetic resonance imaging (MRI) for patients with Ménière disease has been attempted in various studies. In addition, artificial intelligence has rapidly been incorporated into the medical field. In our previous studies, an automated algorithm for EH analysis was developed by using a convolutional neural network. However, several limitations existed, and further studies were conducted to compensate for these limitations. Objective: The aim of this study is to develop a fully automated analytic system for measuring EH ratios that enhances EH analysis accuracy and clinical usability when studying Ménière disease via MRI. Methods: We proposed the 3into3Inception and 3intoUNet networks. Their network architectures were based on those of the Inception-v3 and U-Net networks, respectively. The developed networks were trained for inner ear segmentation by using the magnetic resonance images of 124 people and were embedded in a new, automated EH analysis system—inner-ear hydrops estimation via artificial intelligence (INHEARIT)-version 2 (INHEARIT-v2). After fivefold cross-validation, an additional test was performed by using 60 new, unseen magnetic resonance images to evaluate the performance of our system. The INHEARIT-v2 system has a new function that automatically selects representative images from a full MRI stack. Results: The average segmentation performance of the fivefold cross-validation was measured via the intersection of union method, resulting in performance values of 0.743 (SD 0.030) for the 3into3Inception network and 0.811 (SD 0.032) for the 3intoUNet network. The representative magnetic resonance slices (ie, from a data set of unseen magnetic resonance images) that were automatically selected by the INHEARIT-v2 system only differed from a maximum of 2 expert-selected slices. After comparing the ratios calculated by experienced physicians and those calculated by the INHEARIT-v2 system, we found that the average intraclass correlation coefficient for all cases was 0.941; the average intraclass correlation coefficient of the vestibules was 0.968, and that of the cochleae was 0.914. The time required for the fully automated system to accurately analyze EH ratios based on a patient's MRI stack was approximately 3.5 seconds. Conclusions: In this study, a fully automated full-stack magnetic resonance analysis system for measuring EH ratios was developed (named INHEARIT-v2), and the results showed that there was a high correlation between the expert-calculated EH ratio values and those calculated by the INHEARIT-v2 system. The system is an upgraded version of the INHEARIT system; it has higher segmentation performance and automatically selects representative images from an MRI stack. The new model can help clinicians by providing objective analysis results and reducing the workload for interpreting magnetic resonance images. %M 34546181 %R 10.2196/29678 %U https://www.jmir.org/2021/9/e29678 %U https://doi.org/10.2196/29678 %U http://www.ncbi.nlm.nih.gov/pubmed/34546181 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 3 %P e28501 %T Examining How Internet Users Trust and Access Electronic Health Record Patient Portals: Survey Study %A Yin,Rong %A Law,Katherine %A Neyens,David %+ Department of Industrial Engineering, Clemson University, 100 Freeman Hall, Clemson, SC, United States, 1 8646564719, dneyens@clemson.edu %K internet %K consumer health informatics %K patient portal %K participatory medicine %K electronic health records %K logistic model %K surveys %K questionnaires %D 2021 %7 21.9.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Electronic health record (EHR) patient portals are designed to provide medical health records to patients. Using an EHR portal is expected to contribute to positive health outcomes and facilitate patient-provider communication. Objective: Our objective was to examine how portal users report using their portals and the factors associated with obtaining health information from the internet. We also examined the desired portal features, factors impacting users’ trust in portals, and barriers to using portals. Methods: An internet-based survey study was conducted using Amazon Mechanical Turk. All the participants were adults in the United States who used patient portals. The survey included questions about how the participants used their portals, what factors acted as barriers to using their portals, and how they used and how much they trusted other web-based health information sources as well as their portals. A logistic regression model was used to examine the factors influencing the participants’ trust in their portals. Additionally, the desired features and design characteristics were identified to support the design of future portals. Results: A total of 394 participants completed the survey. Most of the participants were less than 35 years old (212/394, 53.8%), with 36.3% (143/394) aged between 35 and 55 years, and 9.9% (39/394) aged above 55 years. Women accounted for 48.5% (191/394) of the survey participants. More than 78% (307/394) of the participants reported using portals at least monthly. The most common portal features used were viewing lab results, making appointments, and paying bills. Participants reported some barriers to portal use including data security and limited access to the internet. The results of a logistic regression model used to predict the trust in their portals suggest that those comfortable using their portals (odds ratio [OR] 7.97, 95% CI 1.11-57.32) thought that their portals were easy to use (OR 7.4, 95% CI 1.12-48.84), and frequent internet users (OR 43.72, 95% CI 1.83-1046.43) were more likely to trust their portals. Participants reporting that the portals were important in managing their health (OR 28.13, 95% CI 5.31-148.85) and that their portals were a valuable part of their health care (OR 6.75, 95% CI 1.51-30.11) were also more likely to trust their portals. Conclusions: There are several factors that impact the trust of EHR patient portal users in their portals. Designing easily usable portals and considering these factors may be the most effective approach to improving trust in patient portals. The desired features and usability of portals are critical factors that contribute to users’ trust in EHR portals. %M 34546182 %R 10.2196/28501 %U https://humanfactors.jmir.org/2021/3/e28501 %U https://doi.org/10.2196/28501 %U http://www.ncbi.nlm.nih.gov/pubmed/34546182 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 9 %P e30223 %T Automatic Classification of Thyroid Findings Using Static and Contextualized Ensemble Natural Language Processing Systems: Development Study %A Shin,Dongyup %A Kam,Hye Jin %A Jeon,Min-Seok %A Kim,Ha Young %+ Graduate School of Information, Yonsei University, New millennium hall 420, Yonsei-ro 50, Seodaemun-gu, Seoul, 03722, Republic of Korea, 82 10 4094 2392, hayoung.kim@yonsei.ac.kr %K deep learning %K natural language processing %K word embedding %K convolution neural network %K long short-term memory %K transformer %K ensemble %K thyroid %K electronic medical records %D 2021 %7 21.9.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: In the case of Korean institutions and enterprises that collect nonstandardized and nonunified formats of electronic medical examination results from multiple medical institutions, a group of experienced nurses who can understand the results and related contexts initially classified the reports manually. The classification guidelines were established by years of workers’ clinical experiences and there were attempts to automate the classification work. However, there have been problems in which rule-based algorithms or human labor–intensive efforts can be time-consuming or limited owing to high potential errors. We investigated natural language processing (NLP) architectures and proposed ensemble models to create automated classifiers. Objective: This study aimed to develop practical deep learning models with electronic medical records from 284 health care institutions and open-source corpus data sets for automatically classifying 3 thyroid conditions: healthy, caution required, and critical. The primary goal is to increase the overall accuracy of the classification, yet there are practical and industrial needs to correctly predict healthy (negative) thyroid condition data, which are mostly medical examination results, and minimize false-negative rates under the prediction of healthy thyroid conditions. Methods: The data sets included thyroid and comprehensive medical examination reports. The textual data are not only documented in fully complete sentences but also written in lists of words or phrases. Therefore, we propose static and contextualized ensemble NLP network (SCENT) systems to successfully reflect static and contextual information and handle incomplete sentences. We prepared each convolution neural network (CNN)-, long short-term memory (LSTM)-, and efficiently learning an encoder that classifies token replacements accurately (ELECTRA)-based ensemble model by training or fine-tuning them multiple times. Through comprehensive experiments, we propose 2 versions of ensemble models, SCENT-v1 and SCENT-v2, with the single-architecture–based CNN, LSTM, and ELECTRA ensemble models for the best classification performance and practical use, respectively. SCENT-v1 is an ensemble of CNN and ELECTRA ensemble models, and SCENT-v2 is a hierarchical ensemble of CNN, LSTM, and ELECTRA ensemble models. SCENT-v2 first classifies the 3 labels using an ELECTRA ensemble model and then reclassifies them using an ensemble model of CNN and LSTM if the ELECTRA ensemble model predicted them as “healthy” labels. Results: SCENT-v1 outperformed all the suggested models, with the highest F1 score (92.56%). SCENT-v2 had the second-highest recall value (94.44%) and the fewest misclassifications for caution-required thyroid condition while maintaining 0 classification error for the critical thyroid condition under the prediction of the healthy thyroid condition. Conclusions: The proposed SCENT demonstrates good classification performance despite the unique characteristics of the Korean language and problems of data lack and imbalance, especially for the extremely low amount of critical condition data. The result of SCENT-v1 indicates that different perspectives of static and contextual input token representations can enhance classification performance. SCENT-v2 has a strong impact on the prediction of healthy thyroid conditions. %M 34546183 %R 10.2196/30223 %U https://medinform.jmir.org/2021/9/e30223 %U https://doi.org/10.2196/30223 %U http://www.ncbi.nlm.nih.gov/pubmed/34546183 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 9 %P e29374 %T Using a New Model of Electronic Health Record Training to Reduce Physician Burnout: A Plan for Action %A Mohan,Vishnu %A Garrison,Cort %A Gold,Jeffrey A %+ Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, 3181 SW Sam Jackson Park Rd, Mail Code BICC, Portland, OR, 97239-3098, United States, 1 5034944469, mohanv@ohsu.edu %K electronic health records %K clinician burnout %K EHR training %K clinician wellness %K after-hours EHR use %K EHR %K patient data %K burnout %K simulation %K efficiency %K optimization %K well-being %D 2021 %7 20.9.2021 %9 Viewpoint %J JMIR Med Inform %G English %X Physician burnout in the United States has been growing at an alarming rate, and health care organizations are beginning to invest significant resources in combating this phenomenon. Although the causes for burnout are multifactorial, a key issue that affects physicians is that they spend a significant proportion of their time interacting with their electronic health record (EHR) system, primarily because of the need to sift through increasing amounts of patient data, coupled with a significant documentation burden. This has led to physicians spending increasing amounts of time with the EHR outside working hours trying to catch up on paperwork (“pajama time”), which is a factor linked to burnout. In this paper, we propose an innovative model of EHR training using high-fidelity EHR simulations designed to facilitate efficient optimization of EHR use by clinicians and emphasize the importance of both lifelong learning and physician well-being. %M 34325400 %R 10.2196/29374 %U https://medinform.jmir.org/2021/9/e29374 %U https://doi.org/10.2196/29374 %U http://www.ncbi.nlm.nih.gov/pubmed/34325400 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 7 %N 9 %P e30010 %T Assessing COVID-19 Vaccine Uptake and Effectiveness Through the North West London Vaccination Program: Retrospective Cohort Study %A Glampson,Ben %A Brittain,James %A Kaura,Amit %A Mulla,Abdulrahim %A Mercuri,Luca %A Brett,Stephen J %A Aylin,Paul %A Sandall,Tessa %A Goodman,Ian %A Redhead,Julian %A Saravanakumar,Kavitha %A Mayer,Erik K %+ Department of Surgery and Cancer, Imperial College London, 10th Floor QEQM building, St Mary's Hospital Campus, Praed Street, London, W2 1NY, United Kingdom, 44 2078082076, e.mayer@imperial.ac.uk %K health informatics %K real-word evidence %K COVID-19 %K medical informatics %K vaccine %K vaccination %D 2021 %7 17.9.2021 %9 Original Paper %J JMIR Public Health Surveill %G English %X Background: On March 11, 2020, the World Health Organization declared SARS-CoV-2, causing COVID-19, as a pandemic. The UK mass vaccination program commenced on December 8, 2020, vaccinating groups of the population deemed to be most vulnerable to severe COVID-19 infection. Objective: This study aims to assess the early vaccine administration coverage and outcome data across an integrated care system in North West London, leveraging a unique population-level care data set. Vaccine effectiveness of a single dose of the Oxford/AstraZeneca and Pfizer/BioNTech vaccines were compared. Methods: A retrospective cohort study identified 2,183,939 individuals eligible for COVID-19 vaccination between December 8, 2020, and February 24, 2021, within a primary, secondary, and community care integrated care data set. These data were used to assess vaccination hesitancy across ethnicity, gender, and socioeconomic deprivation measures (Pearson product-moment correlations); investigate COVID-19 transmission related to vaccination hubs; and assess the early effectiveness of COVID-19 vaccination (after a single dose) using time-to-event analyses with multivariable Cox regression analysis to investigate if vaccination independently predicted positive SARS-CoV-2 in those vaccinated compared to those unvaccinated. Results: In this study, 5.88% (24,332/413,919) of individuals declined and did not receive a vaccination. Black or Black British individuals had the highest rate of declining a vaccine at 16.14% (4337/26,870). There was a strong negative association between socioeconomic deprivation and rate of declining vaccination (r=–0.94; P=.002) with 13.5% (1980/14,571) of individuals declining vaccination in the most deprived areas compared to 0.98% (869/9609) in the least. In the first 6 days after vaccination, 344 of 389,587 (0.09%) individuals tested positive for SARS-CoV-2. The rate increased to 0.13% (525/389,243) between days 7 and 13, before then gradually falling week on week. At 28 days post vaccination, there was a 74% (hazard ratio 0.26, 95% CI 0.19-0.35) and 78% (hazard ratio 0.22, 95% CI 0.18-0.27) reduction in risk of testing positive for SARS-CoV-2 for individuals that received the Oxford/AstraZeneca and Pfizer/BioNTech vaccines, respectively, when compared with unvaccinated individuals. A very low proportion of hospital admissions were seen in vaccinated individuals who tested positive for SARS-CoV-2 (288/389,587, 0.07% of all patients vaccinated) providing evidence for vaccination effectiveness after a single dose. Conclusions: There was no definitive evidence to suggest COVID-19 was transmitted as a result of vaccination hubs during the vaccine administration rollout in North West London, and the risk of contracting COVID-19 or becoming hospitalized after vaccination has been demonstrated to be low in the vaccinated population. This study provides further evidence that a single dose of either the Pfizer/BioNTech vaccine or the Oxford/AstraZeneca vaccine is effective at reducing the risk of testing positive for COVID-19 up to 60 days across all age groups, ethnic groups, and risk categories in an urban UK population. %M 34265740 %R 10.2196/30010 %U https://publichealth.jmir.org/2021/9/e30010 %U https://doi.org/10.2196/30010 %U http://www.ncbi.nlm.nih.gov/pubmed/34265740 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 9 %P e18307 %T Running an Internet Hospital in China: Perspective Based on a Case Study %A Zhi,Lihua %A Yin,Pei %A Ren,Jingjing %A Wei,Guoqing %A Zhou,Jun %A Wu,Jun %A Shen,Qun %+ Department of Medical Administration, The First Affiliated Hospital, Zhejiang University School of Medicine, 3rd Fl, No. 17 Bldg, 79 Qingchun Rd, Shangcheng District, Hangzhou, China, 86 057187231515, weiguoqing2018@zju.edu.cn %K internet hospitals %K telemedicine %K medical service %K medical procedures %K operation management %K network security %D 2021 %7 16.9.2021 %9 Viewpoint %J J Med Internet Res %G English %X Internet hospitals, as a new forum for doctors to conduct diagnosis and treatment activities based on the internet, are emerging in China and have become integral to the development of the medical field in conjunction with increasing reforms and policies in China’s medical and health system. Here, we take the Internet Hospital of the First Affiliated Hospital, Zhejiang University (FAHZU Internet Hospital) as an example to discuss the operations and functional positioning of developing internet hospital medical services in relation to physical hospitals. This viewpoint considers the platform operation, management, and network security of FAHZU Internet Hospital, and summarizes the advantages and limitations in the operation to provide a reference for other areas with interest in developing internet hospitals. %M 34342267 %R 10.2196/18307 %U https://www.jmir.org/2021/9/e18307 %U https://doi.org/10.2196/18307 %U http://www.ncbi.nlm.nih.gov/pubmed/34342267 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 9 %P e29622 %T Characterization and Identification of Variations in Types of Primary Care Visits Before and During the COVID-19 Pandemic in Catalonia: Big Data Analysis Study %A Lopez Segui,Francesc %A Hernandez Guillamet,Guillem %A Pifarré Arolas,Héctor %A Marin-Gomez,Francesc X %A Ruiz Comellas,Anna %A Ramirez Morros,Anna Maria %A Adroher Mas,Cristina %A Vidal-Alaball,Josep %+ Health Promotion in Rural Areas Research Group, Gerència Territorial de la Catalunya Central, Institut Català de la Salut, Pica d'Estats 36, Sant Fruitós de Bages, Spain, 34 93 693 0040, jvidal.cc.ics@gencat.cat %K COVID-19 %K primary care %K diagnose variations %K big data %K ICD10 %K health system %K big data %K primary care %K healthcare system %D 2021 %7 14.9.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: The COVID-19 pandemic has turned the care model of health systems around the world upside down, causing the abrupt cancellation of face-to-face visits and redirection of the model toward telemedicine. Digital transformation boosts information systems—the more robust they are, the easier it is to monitor the health care system in a highly complex state and allow for more agile and reliable analysis. Objective: The purpose of this study was to analyze diagnoses from primary care visits and distinguish between those that had higher and lower variations, relative to the 2019 and 2020 periods (roughly before and during COVID-19), to identify clinical profiles that may have been most impaired from the least-used diagnostic codes for visits during the pandemic. Methods: We used a database from the Primary Care Services Information Technologies Information System of Catalonia. We analyzed the register of visits (n=2,824,185) and their International Classification of Diseases (ICD-10) diagnostic codes (n=3,921,974; mean 1.38 per visit), as approximations of the reasons for consultations, at 3 different grouping levels. The data were represented by a term frequency matrix and analyzed recursively in different partitions aggregated according to date. Results: The increase in non–face-to-face visits (+267%) did not counterbalance the decrease in face-to-face visits (–47%), with an overall reduction in the total number of visits of 1.36%, despite the notable increase in nursing visits (10.54%). The largest increases in 2020 were visits with diagnoses related to COVID-19 (ICD-10 codes Z20-Z29: 2.540%), along with codes related to economic and housing problems (ICD-10 codes Z55-Z65: 44.40%). Visits with most of the other diagnostic codes decreased in 2020 relative to those in 2019. The largest reductions were chronic pathologies such as arterial hypertension (ICD-10 codes I10-I16: –32.73%) or diabetes (ICD-10 codes E08-E13: –21.13%), but also obesity (E65-E68: –48.58%) and bodily injuries (ICD-10 code T14: –33.70%). Visits with mental health–related diagnostic codes decreased, but the decrease was less than the average decrease. There was a decrease in consultations—for children, adolescents, and adults—for respiratory infections (ICD-10 codes J00-J06: –40.96%). The results show large year-on-year variations (in absolute terms, an average of 12%), which is representative of the strong shock to the health system. Conclusions: The disruption in the primary care model in Catalonia has led to an explosive increase in the number of non–face-to-face visits. There has been a reduction in the number of visits for diagnoses related to chronic pathologies, respiratory infections, obesity, and bodily injuries. Instead, visits for diagnoses related to socioeconomic and housing problems have increased, which emphasizes the importance of social determinants of health in the context of this pandemic. Big data analytics with routine care data yield findings that are consistent with those derived from intuition in everyday clinical practice and can help inform decision making by health planners in order to use the next few years to focus on the least-treated diseases during the COVID-19 pandemic. %M 34313600 %R 10.2196/29622 %U https://www.jmir.org/2021/9/e29622 %U https://doi.org/10.2196/29622 %U http://www.ncbi.nlm.nih.gov/pubmed/34313600 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 9 %P e26231 %T Understanding Pediatric Surgery Cancellation: Geospatial Analysis %A Liu,Lei %A Ni,Yizhao %A Beck,Andrew F %A Brokamp,Cole %A Ramphul,Ryan C %A Highfield,Linda D %A Kanjia,Megha Karkera %A Pratap,J “Nick” %+ Department of Anesthesia, Cincinnati Children's Hospital Medical Center, MLC 2001, 3333 Burnet Avenue, Cincinnati, OH, 45229-3039, United States, 1 513 636 4408, jnpratap@pratap.co.uk %K surgery cancellation %K socioeconomic factors %K spatial regression models %K machine learning %D 2021 %7 10.9.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Day-of-surgery cancellation (DoSC) represents a substantial wastage of hospital resources and can cause significant inconvenience to patients and families. Cancellation is reported to impact between 2% and 20% of the 50 million procedures performed annually in American hospitals. Up to 85% of cancellations may be amenable to the modification of patients’ and families’ behaviors. However, the factors underlying DoSC and the barriers experienced by families are not well understood. Objective: This study aims to conduct a geospatial analysis of patient-specific variables from electronic health records (EHRs) of Cincinnati Children’s Hospital Medical Center (CCHMC) and of Texas Children’s Hospital (TCH), as well as linked socioeconomic factors measured at the census tract level, to understand potential underlying contributors to disparities in DoSC rates across neighborhoods. Methods: The study population included pediatric patients who underwent scheduled surgeries at CCHMC and TCH. A 5-year data set was extracted from the CCHMC EHR, and addresses were geocoded. An equivalent set of data >5.7 years was extracted from the TCH EHR. Case-based data related to patients’ health care use were aggregated at the census tract level. Community-level variables were extracted from the American Community Survey as surrogates for patients’ socioeconomic and minority status as well as markers of the surrounding context. Leveraging the selected variables, we built spatial models to understand the variation in DoSC rates across census tracts. The findings were compared to those of the nonspatial regression and deep learning models. Model performance was evaluated from the root mean squared error (RMSE) using nested 10-fold cross-validation. Feature importance was evaluated by computing the increment of the RMSE when a single variable was shuffled within the data set. Results: Data collection yielded sets of 463 census tracts at CCHMC (DoSC rates 1.2%-12.5%) and 1024 census tracts at TCH (DoSC rates 3%-12.2%). For CCHMC, an L2-normalized generalized linear regression model achieved the best performance in predicting all-cause DoSC rate (RMSE 1.299%, 95% CI 1.21%-1.387%); however, its improvement over others was marginal. For TCH, an L2-normalized generalized linear regression model also performed best (RMSE 1.305%, 95% CI 1.257%-1.352%). All-cause DoSC rate at CCHMC was predicted most strongly by previous no show. As for community-level data, the proportion of African American inhabitants per census tract was consistently an important predictor. In the Texas area, the proportion of overcrowded households was salient to DoSC rate. Conclusions: Our findings suggest that geospatial analysis offers potential for use in targeting interventions for census tracts at a higher risk of cancellation. Our study also demonstrates the importance of home location, socioeconomic disadvantage, and racial minority status on the DoSC of children’s surgery. The success of future efforts to reduce cancellation may benefit from taking social, economic, and cultural issues into account. %M 34505837 %R 10.2196/26231 %U https://www.jmir.org/2021/9/e26231 %U https://doi.org/10.2196/26231 %U http://www.ncbi.nlm.nih.gov/pubmed/34505837 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 8 %N 9 %P e30564 %T Implementation of Electronic Medical Records in Mental Health Settings: Scoping Review %A Zurynski,Yvonne %A Ellis,Louise A %A Tong,Huong Ly %A Laranjo,Liliana %A Clay-Williams,Robyn %A Testa,Luke %A Meulenbroeks,Isabelle %A Turton,Charmaine %A Sara,Grant %+ Centre for Healthcare Resilience and Implementation Science, Australian Institute of Health Innovation, Macquarie University, 75 Talavera Road, Sydney, Australia, 61 9850 ext 2414, Yvonne.Zurynski@mq.edu.au %K electronic medical records %K health information technology %K implementation %K mental health %D 2021 %7 7.9.2021 %9 Review %J JMIR Ment Health %G English %X Background: The success of electronic medical records (EMRs) is dependent on implementation features, such as usability and fit with clinical processes. The use of EMRs in mental health settings brings additional and specific challenges owing to the personal, detailed, narrative, and exploratory nature of the assessment, diagnosis, and treatment in this field. Understanding the determinants of successful EMR implementation is imperative to guide the future design, implementation, and investment of EMRs in the mental health field. Objective: We intended to explore evidence on effective EMR implementation for mental health settings and provide recommendations to support the design, adoption, usability, and outcomes. Methods: The scoping review combined two search strategies that focused on clinician-facing EMRs, one for primary studies in mental health settings and one for reviews of peer-reviewed literature in any health setting. Three databases (Medline, EMBASE, and PsycINFO) were searched from January 2010 to June 2020 using keywords to describe EMRs, settings, and impacts. The Proctor framework for implementation outcomes was used to guide data extraction and synthesis. Constructs in this framework include adoption, acceptability, appropriateness, feasibility, fidelity, cost, penetration, and sustainability. Quality assessment was conducted using a modified Hawker appraisal tool and the Joanna Briggs Institute Critical Appraisal Checklist for Systematic Reviews and Research Syntheses. Results: This review included 23 studies, namely 12 primary studies in mental health settings and 11 reviews. Overall, the results suggested that adoption of EMRs was impacted by financial, technical, and organizational factors, as well as clinician perceptions of appropriateness and acceptability. EMRs were perceived as acceptable and appropriate by clinicians if the system did not interrupt workflow and improved documentation completeness and accuracy. Clinicians were more likely to value EMRs if they supported quality of care, were fit for purpose, did not interfere with the clinician-patient relationship, and were operated with readily available technical support. Evidence on the feasibility of the implemented EMRs was mixed; the primary studies and reviews found mixed impacts on documentation quality and time; one primary study found downward trends in adverse events, whereas a review found improvements in care quality. Five papers provided information on implementation outcomes such as cost and fidelity, and none reported on the penetration and sustainability of EMRs. Conclusions: The body of evidence relating to EMR implementation in mental health settings is limited. Implementation of EMRs could benefit from methods used in general health settings such as co-designing the software and tailoring EMRs to clinical needs and workflows to improve usability and acceptance. Studies in mental health and general health settings rarely focused on long-term implementation outcomes such as penetration and sustainability. Future evaluations of EMRs in all settings should consider long-term impacts to address current knowledge gaps. %M 34491208 %R 10.2196/30564 %U https://mental.jmir.org/2021/9/e30564 %U https://doi.org/10.2196/30564 %U http://www.ncbi.nlm.nih.gov/pubmed/34491208 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 9 %P e28998 %T Measuring Collaboration Through Concurrent Electronic Health Record Usage: Network Analysis Study %A Li,Patrick %A Chen,Bob %A Rhodes,Evan %A Slagle,Jason %A Alrifai,Mhd Wael %A France,Daniel %A Chen,You %+ Department of Biomedical Informatics, Vanderbilt University Medical Center, 2525 West End Ave, Nashville, TN, United States, 1 6153431939, you.chen@vanderbilt.edu %K collaboration %K electronic health records %K audit logs %K health care workers %K neonatal intensive care unit %K network analysis %K clustering %K visualization %K concurrent interaction %K human-computer interaction %K survey instrument %K informatics framework %K secondary data analysis %D 2021 %7 3.9.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Collaboration is vital within health care institutions, and it allows for the effective use of collective health care worker (HCW) expertise. Human-computer interactions involving electronic health records (EHRs) have become pervasive and act as an avenue for quantifying these collaborations using statistical and network analysis methods. Objective: We aimed to measure HCW collaboration and its characteristics by analyzing concurrent EHR usage. Methods: By extracting concurrent EHR usage events from audit log data, we defined concurrent sessions. For each HCW, we established a metric called concurrent intensity, which was the proportion of EHR activities in concurrent sessions over all EHR activities. Statistical models were used to test the differences in the concurrent intensity between HCWs. For each patient visit, starting from admission to discharge, we measured concurrent EHR usage across all HCWs, which we called temporal patterns. Again, we applied statistical models to test the differences in temporal patterns of the admission, discharge, and intermediate days of hospital stay between weekdays and weekends. Network analysis was leveraged to measure collaborative relationships among HCWs. We surveyed experts to determine if they could distinguish collaborative relationships between high and low likelihood categories derived from concurrent EHR usage. Clustering was used to aggregate concurrent activities to describe concurrent sessions. We gathered 4 months of EHR audit log data from a large academic medical center’s neonatal intensive care unit (NICU) to validate the effectiveness of our framework. Results: There was a significant difference (P<.001) in the concurrent intensity (proportion of concurrent activities: ranging from mean 0.07, 95% CI 0.06-0.08, to mean 0.36, 95% CI 0.18-0.54; proportion of time spent on concurrent activities: ranging from mean 0.32, 95% CI 0.20-0.44, to mean 0.76, 95% CI 0.51-1.00) between the top 13 HCW specialties who had the largest amount of time spent in EHRs. Temporal patterns between weekday and weekend periods were significantly different on admission (number of concurrent intervals per hour: 11.60 vs 0.54; P<.001) and discharge days (4.72 vs 1.54; P<.001), but not during intermediate days of hospital stay. Neonatal nurses, fellows, frontline providers, neonatologists, consultants, respiratory therapists, and ancillary and support staff had collaborative relationships. NICU professionals could distinguish high likelihood collaborative relationships from low ones at significant rates (3.54, 95% CI 3.31-4.37 vs 2.64, 95% CI 2.46-3.29; P<.001). We identified 50 clusters of concurrent activities. Over 87% of concurrent sessions could be described by a single cluster, with the remaining 13% of sessions comprising multiple clusters. Conclusions: Leveraging concurrent EHR usage workflow through audit logs to analyze HCW collaboration may improve our understanding of collaborative patient care. HCW collaboration using EHRs could potentially influence the quality of patient care, discharge timeliness, and clinician workload, stress, or burnout. %M 34477566 %R 10.2196/28998 %U https://medinform.jmir.org/2021/9/e28998 %U https://doi.org/10.2196/28998 %U http://www.ncbi.nlm.nih.gov/pubmed/34477566 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 8 %P e30470 %T Classification of Electronic Health Record–Related Patient Safety Incidents: Development and Validation Study %A Palojoki,Sari %A Saranto,Kaija %A Reponen,Elina %A Skants,Noora %A Vakkuri,Anne %A Vuokko,Riikka %+ Department of Steering of Healthcare and Social Welfare, Ministry of Social Affairs and Health, P.O. Box 33, Helsinki, 00023, Finland, 358 29516001, sari.palojoki@gmail.com %K classification %K electronic health records %K hospitals %K medical informatics %K patient safety %K risk %D 2021 %7 31.8.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: It is assumed that the implementation of health information technology introduces new vulnerabilities within a complex sociotechnical health care system, but no international consensus exists on a standardized format for enhancing the collection, analysis, and interpretation of technology-induced errors. Objective: This study aims to develop a classification for patient safety incident reporting associated with the use of mature electronic health records (EHRs). It also aims to validate the classification by using a data set of incidents during a 6-month period immediately after the implementation of a new EHR system. Methods: The starting point of the classification development was the Finnish Technology-Induced Error Risk Assessment Scale tool, based on research on commonly recognized error types. A multiprofessional research team used iterative tests on consensus building to develop a classification system. The final classification, with preliminary descriptions of classes, was validated by applying it to analyze EHR-related error incidents (n=428) during the implementation phase of a new EHR system and also to evaluate this classification’s characteristics and applicability for reporting incidents. Interrater agreement was applied. Results: The number of EHR-related patient safety incidents during the implementation period (n=501) was five-fold when compared with the preimplementation period (n=82). The literature identified new error types that were added to the emerging classification. Error types were adapted iteratively after several test rounds to develop a classification for reporting patient safety incidents in the clinical use of a high-maturity EHR system. Of the 427 classified patient safety incidents, interface problems accounted for 96 (22.5%) incident reports, usability problems for 73 (17.1%), documentation problems for 60 (14.1%), and clinical workflow problems for 33 (7.7%). Altogether, 20.8% (89/427) of reports were related to medication section problems, and downtime problems were rare (n=8). During the classification work, 14.8% (74/501) of reports of the original sample were rejected because of insufficient information, even though the reports were deemed to be related to EHRs. The interrater agreement during the blinded review was 97.7%. Conclusions: This study presents a new classification for EHR-related patient safety incidents applicable to mature EHRs. The number of EHR-related patient safety incidents during the implementation period may reflect patient safety challenges during the implementation of a new type of high-maturity EHR system. The results indicate that the types of errors previously identified in the literature change with the EHR development cycle. %M 34245558 %R 10.2196/30470 %U https://medinform.jmir.org/2021/8/e30470 %U https://doi.org/10.2196/30470 %U http://www.ncbi.nlm.nih.gov/pubmed/34245558 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 8 %P e29807 %T Patient-Level Cancer Prediction Models From a Nationwide Patient Cohort: Model Development and Validation %A Lee,Eunsaem %A Jung,Se Young %A Hwang,Hyung Ju %A Jung,Jaewoo %+ Department of Mathematics, Pohang University of Science and Technology, 77 Cheongam-ro, Nam-gu, Pohang-si, 37673, Republic of Korea, 82 054 279 2056, hjhwang@postech.ac.kr %K prediction %K model %K claim data %K cancer %K machine learning %K development %K cohort %K validation %K database %K algorithm %D 2021 %7 30.8.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Nationwide population-based cohorts provide a new opportunity to build automated risk prediction models at the patient level, and claim data are one of the more useful resources to this end. To avoid unnecessary diagnostic intervention after cancer screening tests, patient-level prediction models should be developed. Objective: We aimed to develop cancer prediction models using nationwide claim databases with machine learning algorithms, which are explainable and easily applicable in real-world environments. Methods: As source data, we used the Korean National Insurance System Database. Every Korean in ≥40 years old undergoes a national health checkup every 2 years. We gathered all variables from the database including demographic information, basic laboratory values, anthropometric values, and previous medical history. We applied conventional logistic regression methods, light gradient boosting methods, neural networks, survival analysis, and one-class embedding classifier methods to effectively analyze high dimension data based on deep learning–based anomaly detection. Performance was measured with area under the curve and area under precision recall curve. We validated our models externally with a health checkup database from a tertiary hospital. Results: The one-class embedding classifier model received the highest area under the curve scores with values of 0.868, 0.849, 0.798, 0.746, 0.800, 0.749, and 0.790 for liver, lung, colorectal, pancreatic, gastric, breast, and cervical cancers, respectively. For area under precision recall curve, the light gradient boosting models had the highest score with values of 0.383, 0.401, 0.387, 0.300, 0.385, 0.357, and 0.296 for liver, lung, colorectal, pancreatic, gastric, breast, and cervical cancers, respectively. Conclusions: Our results show that it is possible to easily develop applicable cancer prediction models with nationwide claim data using machine learning. The 7 models showed acceptable performances and explainability, and thus can be distributed easily in real-world environments. %M 34459743 %R 10.2196/29807 %U https://medinform.jmir.org/2021/8/e29807 %U https://doi.org/10.2196/29807 %U http://www.ncbi.nlm.nih.gov/pubmed/34459743 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 8 %P e23219 %T A System to Support Diverse Social Program Management %A McKillop,Mollie %A Snowdon,Jane %A Willis,Van C %A Alevy,Shira %A Rizvi,Rubina %A Rewalt,Karen %A Lefebvre-Paillé,Charlyne %A Kassler,William %A Purcell Jackson,Gretchen %+ IBM Watson Health, 75 Binney Street, Cambridge, MA, 02142, United States, 1 3322073519, mollie.mckillop@ibm.com %K other clinical informatics applications %K process management tools %K requirements analysis and design %K consumer health informatics %K public health %D 2021 %7 30.8.2021 %9 Viewpoint %J JMIR Med Inform %G English %X Background: Social programs are services provided by governments, nonprofits, and other organizations to help improve the health and well-being of individuals, families, and communities. Social programs aim to deliver services effectively and efficiently, but they are challenged by information silos, limited resources, and the need to deliver frequently changing mandated benefits. Objective: We aim to explore how an information system designed for social programs helps deliver services effectively and efficiently across diverse programs. Methods: This viewpoint describes the configurable and modular architecture of Social Program Management (SPM), a system to support efficient and effective delivery of services through a wide range of social programs and lessons learned from implementing SPM across diverse settings. We explored usage data to inform the engagement and impact of SPM on the efficient and effective delivery of services. Results: The features and functionalities of SPM seem to support the goals of social programs. We found that SPM provides fundamental management processes and configurable program-specific components to support social program administration; has been used by more than 280,000 caseworkers serving more than 30 million people in 13 countries; contains features designed to meet specific user requirements; supports secure information sharing and collaboration through data standardization and aggregation; and offers configurability and flexibility, which are important for digital transformation and organizational change. Conclusions: SPM is a user-centered, configurable, and flexible system for managing social program workflows. %M 34459741 %R 10.2196/23219 %U https://medinform.jmir.org/2021/8/e23219 %U https://doi.org/10.2196/23219 %U http://www.ncbi.nlm.nih.gov/pubmed/34459741 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 8 %P e16293 %T Potential Uses of Blockchain Technology for Outcomes Research on Opioids %A Gonzales,Aldren %A Smith,Scott R %A Dullabh,Prashila %A Hovey,Lauren %A Heaney-Huls,Krysta %A Robichaud,Meagan %A Boodoo,Roger %+ US Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation, Office of Health Policy, 200 Independence Ave SW, Washington, DC, 20201, United States, 1 2028707414, aldren.gonzales@hhs.gov %K blockchain %K distributed ledger %K opioid crisis %K outcomes research %K patient-centered outcomes research %K mobile phone %D 2021 %7 27.8.2021 %9 Viewpoint %J JMIR Med Inform %G English %X The scale and severity of the opioid epidemic call for innovative, multipronged solutions. Research and development is key to accelerate the discovery and evaluation of interventions that support pain and substance use disorder management. In parallel, the use and integration of blockchain technology within research networks holds the potential to address some of the unique challenges facing opioid research. This paper discusses the applications of blockchain technology and illustrates potential ways in which it could be applied to strengthen the validity of outcomes research on the opioid epidemic. We reviewed published and gray literature to identify useful applications of blockchain, specifically those that address the challenges faced by opioid research networks and programs. We then convened a panel of experts to discuss the strengths, limitations, and feasibility of each application. Blockchain has the potential to address some of the issues surrounding health data management, including data availability, data sharing and interoperability, and privacy and security. We identified five primary applications of blockchain to opioids: clinical trials and pharmaceutical research, incentivizing data donation and behavior change, secure exchange and management of e-prescriptions, supply chain management, and secondary use of clinical data for research and public health surveillance. The published literature was limited, leading us to rely on gray literature, which was also limited in its discussion of the technical aspects of implementation. The technical expert panel provided additional context and an assessment of feasibility that was lacking in the literature. Research on opioid use and misuse is challenging because of disparate data stored across different systems, data and system interoperability issues, and legal requirements. These areas must be navigated to make data accessible, timely, and useful to researchers. Blockchain technologies have the potential to act as a facilitator in this process, offering a more efficient, secure, and privacy-preserving solution for data exchange. Among the 5 primary applications, we found that clinical trial research, supply chain management, and secondary use of data had the most examples in practice and the potential effectiveness of blockchain. More discussions and studies should focus on addressing technical questions concerning scalability and tackling practical concerns such as cost, standards, and governance around the implementation of blockchain in health care. Policy concerns related to balancing the need for data accessibility that also protects patient privacy and autonomy in revoking consent should also be examined. %M 34448721 %R 10.2196/16293 %U https://medinform.jmir.org/2021/8/e16293 %U https://doi.org/10.2196/16293 %U http://www.ncbi.nlm.nih.gov/pubmed/34448721 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 8 %P e24890 %T Alignment of Key Stakeholders’ Priorities for Patient-Facing Tools in Digital Health: Mixed Methods Study %A Lyles,Courtney Rees %A Adler-Milstein,Julia %A Thao,Crishyashi %A Lisker,Sarah %A Nouri,Sarah %A Sarkar,Urmimala %+ Division of General Internal Medicine, Department of Medicine, University of California San Francisco, 1001 Potrero Avenue, Box 1364, San Francisco, CA, 94143, United States, 1 628 206 6483, courtney.lyles@ucsf.edu %K medical informatics %K medical informatics apps %K information technology %K implementation science %K mixed methods %D 2021 %7 26.8.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: There is widespread agreement on the promise of patient-facing digital health tools to transform health care. Yet, few tools are in widespread use or have documented clinical effectiveness. Objective: The aim of this study was to gain insight into the gap between the potential of patient-facing digital health tools and real-world uptake. Methods: We interviewed and surveyed experts (in total, n=24) across key digital health stakeholder groups—venture capitalists, digital health companies, payers, and health care system providers or leaders—guided by the Consolidated Framework for Implementation Research. Results: Our findings revealed that external policy, regulatory demands, internal organizational workflow, and integration needs often take priority over patient needs and patient preferences for digital health tools, which lowers patient acceptance rates. We discovered alignment, across all 4 stakeholder groups, in the desire to engage both patients and frontline health care providers in broader dissemination and evaluation of digital health tools. However, major areas of misalignment between stakeholder groups have stymied the progress of digital health tool uptake—venture capitalists and companies focused on external policy and regulatory demands, while payers and providers focused on internal organizational workflow and integration needs. Conclusions: Misalignment of the priorities of digital health companies and their funders with those of providers and payers requires direct attention to improve uptake of patient-facing digital health tools and platforms. %M 34435966 %R 10.2196/24890 %U https://www.jmir.org/2021/8/e24890 %U https://doi.org/10.2196/24890 %U http://www.ncbi.nlm.nih.gov/pubmed/34435966 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 8 %P e27571 %T Hospital Investment Decisions in Healthcare 4.0 Technologies: Scoping Review and Framework for Exploring Challenges, Trends, and Research Directions %A Vassolo,Roberto Santiago %A Mac Cawley,Alejandro Francisco %A Tortorella,Guilherme Luz %A Fogliatto,Flavio Sanson %A Tlapa,Diego %A Narayanamurthy,Gopalakrishnan %+ IAE Business School, Universidad Austral, Mariano Acosta s/n y Ruta Prov 8, Pilar, B1629WWA, Argentina, 54 2304481000, rvassolo@iae.edu.ar %K healthcare 4.0 %K scoping review %K investments %K real options %K health technology assessment %K technological bundles %K decision-makers %K hospital %K public health %K technology %K health technology %K smart technology %K hospital management %K health care investment %K decision making %K new technologies %D 2021 %7 26.8.2021 %9 Review %J J Med Internet Res %G English %X Background: Alternative approaches to analyzing and evaluating health care investments in state-of-the-art technologies are being increasingly discussed in the literature, especially with the advent of Healthcare 4.0 (H4.0) technologies or eHealth. Such investments generally involve computer hardware and software that deal with the storage, retrieval, sharing, and use of health care information, data, and knowledge for communication and decision-making. Besides, the use of these technologies significantly increases when addressed in bundles. However, a structured and holistic approach to analyzing investments in H4.0 technologies is not available in the literature. Objective: This study aims to analyze previous research related to the evaluation of H4.0 technologies in hospitals and characterize the most common investment approaches used. We propose a framework that organizes the research associated with hospitals’ H4.0 technology investment decisions and suggest five main research directions on the topic. Methods: To achieve our goal, we followed the standard procedure for scoping reviews. We performed a search in the Crossref, PubMed, Scopus, and Web of Science databases with the keywords investment, health, industry 4.0, investment, health technology assessment, healthcare 4.0, and smart in the title, abstract, and keywords of research papers. We retrieved 5701 publications from all the databases. After removing papers published before 2011 as well as duplicates and performing further screening, we were left with 244 articles, from which 33 were selected after in-depth analysis to compose the final publication portfolio. Results: Our findings show the multidisciplinary nature of the research related to evaluating hospital investments in H4.0 technologies. We found that the most common investment approaches focused on cost analysis, single technology, and single decision-maker involvement, which dominate bundle analysis, H4.0 technology value considerations, and multiple decision-maker involvement. Conclusions: Some of our findings were unexpected, given the interrelated nature of H4.0 technologies and their multidimensional impact. Owing to the absence of a more holistic approach to H4.0 technology investment decisions, we identified five promising research directions for the topic: development of economic valuation methodologies tailored for H4.0 technologies; accounting for technology interrelations in the form of bundles; accounting for uncertainties in the process of evaluating such technologies; integration of administrative, medical, and patient perspectives into the evaluation process; and balancing and handling complexity in the decision-making process. %M 34435967 %R 10.2196/27571 %U https://www.jmir.org/2021/8/e27571 %U https://doi.org/10.2196/27571 %U http://www.ncbi.nlm.nih.gov/pubmed/34435967 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 8 %P e27163 %T Variation of Daily Care Demand in Swiss General Hospitals: Longitudinal Study on Capacity Utilization, Patient Turnover and Clinical Complexity Levels %A Sharma,Narayan %A Schwendimann,René %A Endrich,Olga %A Ausserhofer,Dietmar %A Simon,Michael %+ Institute of Nursing Science, Department Public Health, Faculty of Medicine, University of Basel, Bernoullistrasse 28, Basel, 4056, Switzerland, 41 61 207 09 12, m.simon@unibas.ch %K inpatient population %K routine data %K general hospitals %K capacity utilization %K clinical complexity %K patient data %K hospital system %K complexity algorithm %D 2021 %7 19.8.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Variations in hospitals’ care demand relies not only on the patient volume but also on the disease severity. Understanding both daily severity and patient volume in hospitals could help to identify hospital pressure zones to improve hospital-capacity planning and policy-making. Objective: This longitudinal study explored daily care demand dynamics in Swiss general hospitals for 3 measures: (1) capacity utilization, (2) patient turnover, and (3) patient clinical complexity level. Methods: A retrospective population-based analysis was conducted with 1 year of routine data of 1.2 million inpatients from 102 Swiss general hospitals. Capacity utilization was measured as a percentage of the daily maximum number of inpatients. Patient turnover was measured as a percentage of the daily sum of admissions and discharges per hospital. Patient clinical complexity level was measured as the average daily patient disease severity per hospital from the clinical complexity algorithm. Results: There was a pronounced variability of care demand in Swiss general hospitals. Among hospitals, the average daily capacity utilization ranged from 57.8% (95% CI 57.3-58.4) to 87.7% (95% CI 87.3-88.0), patient turnover ranged from 22.5% (95% CI 22.1-22.8) to 34.5% (95% CI 34.3-34.7), and the mean patient clinical complexity level ranged from 1.26 (95% CI 1.25-1.27) to 2.06 (95% CI 2.05-2.07). Moreover, both within and between hospitals, all 3 measures varied distinctly between days of the year, between days of the week, between weekdays and weekends, and between seasons. Conclusions: While admissions and discharges drive capacity utilization and patient turnover variation, disease severity of each patient drives patient clinical complexity level. Monitoring—and, if possible, anticipating—daily care demand fluctuations is key to managing hospital pressure zones. This study provides a pathway for identifying patients’ daily exposure to strained hospital systems for a time-varying causal model. %M 34420926 %R 10.2196/27163 %U https://www.jmir.org/2021/8/e27163 %U https://doi.org/10.2196/27163 %U http://www.ncbi.nlm.nih.gov/pubmed/34420926 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 8 %P e26650 %T Patient and Clinician Characteristics Associated With Secure Message Content: Retrospective Cohort Study %A Heisey-Grove,Dawn %A Rathert,Cheryl %A McClelland,Laura E %A Jackson,Kevin %A DeShazo,Jonathan P %+ The MITRE Corporation, 7525 Colshire Dr, McLean, VA, 22102, United States, 1 7035477389, heiseygroved@mitre.org %K patient-provider communication %K electronic messaging %K hypertension %K diabetes %D 2021 %7 19.8.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Good communication has been shown to affect patient outcomes; however, the effect varies according to patient and clinician characteristics. To date, no research has explored the differences in the content of secure messages based on these characteristics. Objective: This study aims to explore characteristics of patients and clinic staff associated with the content exchanged in secure messages. Methods: We coded 18,309 messages that were part of threads initiated by 1031 patients with hypertension, diabetes, or both conditions, in communication with 711 staff members. We conducted four sets of analyses to identify associations between patient characteristics and the types of messages they sent, staff characteristics and the types of messages they sent, staff characteristics and the types of messages patients sent to them, and patient characteristics and the types of messages they received from staff. Logistic regression was used to estimate the strength of the associations. Results: We found that younger patients had reduced odds of sharing clinical updates (odds ratio [OR] 0.77, 95% CI 0.65-0.91) and requesting prescription refills (OR 0.77, 95% CI 0.65-0.90). Women had reduced odds of self-reporting biometrics (OR 0.78, 95% CI 0.62-0.98) but greater odds of responding to a clinician (OR 1.20, 95% CI 1.02-1.42) and seeking medical guidance (OR 1.19, 95% CI 1.01-1.40). Compared with White patients, Black patients had greater odds of requesting preventive care (OR 2.68, 95% CI 1.30-5.51) but reduced odds of requesting a new or changed prescription (OR 0.72, 95% CI 0.53-0.98) or laboratory or other diagnostic procedures (OR 0.66, 95% CI 0.46-0.95). Staff had lower odds of sharing medical guidance with younger patients (OR 0.83, 95% CI 0.69-1.00) and uninsured patients (OR 0.21, 95% CI 0.06-0.73) but had greater odds of sharing medical guidance with patients with public payers (OR 2.03, 95% CI 1.26-3.25) compared with patients with private payers. Staff had reduced odds of confirming to women that their requests were fulfilled (OR 0.82, 95% CI 0.69-0.98). Compared with physicians, nurse practitioners had greater odds of sharing medical guidance with patients (OR 2.74, 95% CI 1.12-6.68) and receiving prescription refill requests (OR 3.39, 95% CI 1.49-7.71). Registered nurses had greater odds of deferred information sharing (OR 1.61, 95% CI 1.04-2.49) and receiving responses to messages (OR 3.93, 95% CI 2.18-7.11) than physicians. Conclusions: The differences we found in content use based on patient characteristics could lead to the exacerbation of health disparities when content is associated with health outcomes. Disparities in the content of secure messages could exacerbate disparities in patient outcomes, such as satisfaction, trust in the system, self-care, and health outcomes. Staff and administrators should evaluate how secure messaging is used to ensure that disparities in care are not perpetuated via this communication modality. %M 34420923 %R 10.2196/26650 %U https://www.jmir.org/2021/8/e26650 %U https://doi.org/10.2196/26650 %U http://www.ncbi.nlm.nih.gov/pubmed/34420923 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 8 %P e30453 %T Digital Orientation of Health Systems in the Post–COVID-19 “New Normal” in the United States: Cross-sectional Survey %A Khuntia,Jiban %A Ning,Xue %A Stacey,Rulon %+ CU Business School, University of Colorado Denver, 1475 Lawrence Street, Denver, CO, 80202, United States, 1 3038548024, jiban.khuntia@ucdenver.edu %K post–COVID-19 %K digital orientation %K health systems %K digital transformation %K digital health %K telehealth %K telemedicine %K COVID-19 %K impact %K insight %K cross-sectional %K survey %K United States %K electronic health record %K EHR %D 2021 %7 16.8.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Almost all health systems have developed some form of customer-facing digital technologies and have worked to align these systems to their existing electronic health records to accommodate the surge in remote and virtual care deliveries during the COVID-19 pandemic. Others have developed analytics-driven decision-making capabilities. However, it is not clear how health systems in the United States are embracing digital technologies and there is a gap in health systems’ abilities to integrate workflows with expanding technologies to spur innovation and futuristic growth. There is a lack of reliable and reported estimates of the current and futuristic digital orientations of health systems. Periodic assessments will provide imperatives to policy formulation and align efforts to yield the transformative power of emerging digital technologies. Objective: The aim of this study was to explore and examine differences in US health systems with respect to digital orientations in the post–COVID-19 “new normal” in 2021. Differences were assessed in four dimensions: (1) analytics-oriented digital technologies (AODT), (2) customer-oriented digital technologies (CODT), (3) growth and innovation–oriented digital technologies (GODT), and (4) futuristic and experimental digital technologies (FEDT). The former two dimensions are foundational to health systems’ digital orientation, whereas the latter two will prepare for future disruptions. Methods: We surveyed a robust group of health system chief executive officers (CEOs) across the United States from February to March 2021. Among the 625 CEOs, 135 (22%) responded to our survey. We considered the above four broad digital technology orientations, which were ratified with expert consensus. Secondary data were collected from the Agency for Healthcare Research and Quality Hospital Compendium, leading to a matched usable dataset of 124 health systems for analysis. We examined the relationship of adopting the four digital orientations to specific hospital characteristics and earlier reported factors as barriers or facilitators to technology adoption. Results: Health systems showed a lower level of CODT (mean 4.70) or GODT (mean 4.54) orientations compared with AODT (mean 5.03), and showed the lowest level of FEDT orientation (mean 4.31). The ordered logistic estimation results provided nuanced insights. Medium-sized (P<.001) health systems, major teaching health systems (P<.001), and systems with high-burden hospitals (P<.001) appear to be doing worse with respect to AODT orientations, raising some concerns. Health systems of medium (P<.001) and large (P=.02) sizes, major teaching health systems (P=.07), those with a high revenue (P=.05), and systems with high-burden hospitals (P<.001) have less CODT orientation. Health systems in the midwest (P=.05) and southern (P=.04) states are more likely to adopt GODT, whereas high-revenue (P=.004) and investor-ownership (P=.01) health systems are deterred from GODT. Health systems of a medium size, and those that are in the midwest (P<.001), south (P<.001), and west (P=.01) are more adept to FEDT, whereas medium (P<.001) and high-revenue (P<.001) health systems, and those with a high discharge rate (P=.04) or high burden (P=.003, P=.005) have subdued FEDT orientations. Conclusions: Almost all health systems have some current foundational digital technological orientations to glean intelligence or service delivery to customers, with some notable exceptions. Comparatively, fewer health systems have growth or futuristic digital orientations. The transformative power of digital technologies can only be leveraged by adopting futuristic digital technologies. Thus, the disparities across these orientations suggest that a holistic, consistent, and well-articulated direction across the United States remains elusive. Accordingly, we suggest that a policy strategy and financial incentives are necessary to spur a well-visioned and articulated digital orientation for all health systems across the United States. In the absence of such a policy to collectively leverage digital transformations, differences in care across the country will continue to be a concern. %M 34254947 %R 10.2196/30453 %U https://www.jmir.org/2021/8/e30453 %U https://doi.org/10.2196/30453 %U http://www.ncbi.nlm.nih.gov/pubmed/34254947 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 8 %P e22391 %T Communicating the Implementation of Open Notes to Health Care Professionals: Mixed Methods Study %A Jonnergård,Karin %A Petersson,Lena %A Erlingsdóttir,Gudbjörg %+ Department of Design Sciences, Lund University, Box 118, Lund, SE-221 00, Sweden, 46 46 222 05 33, Lena.Petersson@design.lth.se %K implementation %K health care %K electronic health records %K communication strategy %K eHealth %K telemedicine %K PAEHRs %K Open Notes %K professions %K EHR %D 2021 %7 16.8.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: The literature on how to communicate reform in organizations has mainly focused on levels of hierarchy and has largely ignored the variety of professions that may be found within an organization. In this study, we focus on the relationship between media type and professional responses. Objective: The objective of this study was to investigate whether and how belonging to a profession influences the choice of communication media and the perception of information when a technical innovation is implemented in a health care setting. Methods: This study followed a mixed methods design based on observations and participant studies, as well as a survey of professionals in psychiatric health care in Sweden. The χ2 test was used to detect differences in perceptions between professional groups. Results: The use of available communication media differed among professions. These differences seem to be related to the status attached to each profession. The sense-making of the information appears to be similar among the professions, but is based on their traditional professional norms rather than on reflection on the reform at hand. Conclusions: When communicating about the implementation of a new technology, the choice of media and the message need to be attuned to the employees in both hierarchical and professional terms. This also applies to situations where professional employees are only indirectly affected by the implementation. A differentiated communication strategy is preferred over a downward cascade of information. %M 34398794 %R 10.2196/22391 %U https://medinform.jmir.org/2021/8/e22391 %U https://doi.org/10.2196/22391 %U http://www.ncbi.nlm.nih.gov/pubmed/34398794 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 8 %P e28287 %T Ranking Rule-Based Automatic Explanations for Machine Learning Predictions on Asthma Hospital Encounters in Patients With Asthma: Retrospective Cohort Study %A Zhang,Xiaoyi %A Luo,Gang %+ Department of Biomedical Informatics and Medical Education, University of Washington, UW Medicine South Lake Union, 850 Republican Street, Building C, Box 358047, Seattle, WA, 98195, United States, 1 206 221 4596, gangluo@cs.wisc.edu %K asthma %K clinical decision support %K machine learning %K patient care management %K forecasting %D 2021 %7 11.8.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Asthma hospital encounters impose a heavy burden on the health care system. To improve preventive care and outcomes for patients with asthma, we recently developed a black-box machine learning model to predict whether a patient with asthma will have one or more asthma hospital encounters in the succeeding 12 months. Our model is more accurate than previous models. However, black-box machine learning models do not explain their predictions, which forms a barrier to widespread clinical adoption. To solve this issue, we previously developed a method to automatically provide rule-based explanations for the model’s predictions and to suggest tailored interventions without sacrificing model performance. For an average patient correctly predicted by our model to have future asthma hospital encounters, our explanation method generated over 5000 rule-based explanations, if any. However, the user of the automated explanation function, often a busy clinician, will want to quickly obtain the most useful information for a patient by viewing only the top few explanations. Therefore, a methodology is required to appropriately rank the explanations generated for a patient. However, this is currently an open problem. Objective: The aim of this study is to develop a method to appropriately rank the rule-based explanations that our automated explanation method generates for a patient. Methods: We developed a ranking method that struck a balance among multiple factors. Through a secondary analysis of 82,888 data instances of adults with asthma from the University of Washington Medicine between 2011 and 2018, we demonstrated our ranking method on the test case of predicting asthma hospital encounters in patients with asthma. Results: For each patient predicted to have asthma hospital encounters in the succeeding 12 months, the top few explanations returned by our ranking method typically have high quality and low redundancy. Many top-ranked explanations provide useful insights on the various aspects of the patient’s situation, which cannot be easily obtained by viewing the patient’s data in the current electronic health record system. Conclusions: The explanation ranking module is an essential component of the automated explanation function, and it addresses the interpretability issue that deters the widespread adoption of machine learning predictive models in clinical practice. In the next few years, we plan to test our explanation ranking method on predictive modeling problems addressing other diseases as well as on data from other health care systems. International Registered Report Identifier (IRRID): RR2-10.2196/5039 %M 34383673 %R 10.2196/28287 %U https://medinform.jmir.org/2021/8/e28287 %U https://doi.org/10.2196/28287 %U http://www.ncbi.nlm.nih.gov/pubmed/34383673 %0 Journal Article %@ 2563-6316 %I JMIR Publications %V 2 %N 3 %P e27017 %T Finding Potential Adverse Events in the Unstructured Text of Electronic Health Care Records: Development of the Shakespeare Method %A Bright,Roselie A %A Rankin,Summer K %A Dowdy,Katherine %A Blok,Sergey V %A Bright,Susan J %A Palmer,Lee Anne M %+ Booz Allen Hamilton, 8283 Greensboro Dr, McLean, VA, 22102, United States, 1 808 594 5975, rankin_summer@bah.com %K epidemiology %K electronic health record %K electronic health care record %K big data %K patient harm %K patient safety %K public health %K product surveillance, postmarketing %K natural language processing %K proof-of-concept study %K critical care %D 2021 %7 11.8.2021 %9 Original Paper %J JMIRx Med %G English %X Background: Big data tools provide opportunities to monitor adverse events (patient harm associated with medical care) (AEs) in the unstructured text of electronic health care records (EHRs). Writers may explicitly state an apparent association between treatment and adverse outcome (“attributed”) or state the simple treatment and outcome without an association (“unattributed”). Many methods for finding AEs in text rely on predefining possible AEs before searching for prespecified words and phrases or manual labeling (standardization) by investigators. We developed a method to identify possible AEs, even if unknown or unattributed, without any prespecifications or standardization of notes. Our method was inspired by word-frequency analysis methods used to uncover the true authorship of disputed works credited to William Shakespeare. We chose two use cases, “transfusion” and “time-based.” Transfusion was chosen because new transfusion AE types were becoming recognized during the study data period; therefore, we anticipated an opportunity to find unattributed potential AEs (PAEs) in the notes. With the time-based case, we wanted to simulate near real-time surveillance. We chose time periods in the hope of detecting PAEs due to contaminated heparin from mid-2007 to mid-2008 that were announced in early 2008. We hypothesized that the prevalence of contaminated heparin may have been widespread enough to manifest in EHRs through symptoms related to heparin AEs, independent of clinicians’ documentation of attributed AEs. Objective: We aimed to develop a new method to identify attributed and unattributed PAEs using the unstructured text of EHRs. Methods: We used EHRs for adult critical care admissions at a major teaching hospital (2001-2012). For each case, we formed a group of interest and a comparison group. We concatenated the text notes for each admission into one document sorted by date, and deleted replicate sentences and lists. We identified statistically significant words in the group of interest versus the comparison group. Documents in the group of interest were filtered to those words, followed by topic modeling on the filtered documents to produce topics. For each topic, the three documents with the maximum topic scores were manually reviewed to identify PAEs. Results: Topics centered around medical conditions that were unique to or more common in the group of interest, including PAEs. In each use case, most PAEs were unattributed in the notes. Among the transfusion PAEs was unattributed evidence of transfusion-associated cardiac overload and transfusion-related acute lung injury. Some of the PAEs from mid-2007 to mid-2008 were increased unattributed events consistent with AEs related to heparin contamination. Conclusions: The Shakespeare method could be a useful supplement to AE reporting and surveillance of structured EHR data. Future improvements should include automation of the manual review process. %M 37725533 %R 10.2196/27017 %U https://med.jmirx.org/2021/3/e27017 %U https://doi.org/10.2196/27017 %U http://www.ncbi.nlm.nih.gov/pubmed/37725533 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 3 %P e25046 %T Barriers to the Use of Clinical Decision Support for the Evaluation of Pulmonary Embolism: Qualitative Interview Study %A Richardson,Safiya %A Dauber-Decker,Katherine L %A McGinn,Thomas %A Barnaby,Douglas P %A Cattamanchi,Adithya %A Pekmezaris,Renee %+ Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, 600 Community Drive, Suite 403, Manhasset, NY, 11030, United States, 1 5166001411, srichard12@northwell.edu %K medical informatics %K pulmonary embolism %K electronic health records %K quality improvement %K clinical decision support systems %D 2021 %7 4.8.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Clinicians often disregard potentially beneficial clinical decision support (CDS). Objective: In this study, we sought to explore the psychological and behavioral barriers to the use of a CDS tool. Methods: We conducted a qualitative study involving emergency medicine physicians and physician assistants. A semistructured interview guide was created based on the Capability, Opportunity, and Motivation-Behavior model. Interviews focused on the barriers to the use of a CDS tool built based on Wells’ criteria for pulmonary embolism to assist clinicians in establishing pretest probability of pulmonary embolism before imaging. Results: Interviews were conducted with 12 clinicians. Six barriers were identified, including (1) Bayesian reasoning, (2) fear of missing a pulmonary embolism, (3) time pressure or cognitive load, (4) gestalt includes Wells’ criteria, (5) missed risk factors, and (6) social pressure. Conclusions: Clinicians highlighted several important psychological and behavioral barriers to CDS use. Addressing these barriers will be paramount in developing CDS that can meet its potential to transform clinical care. %M 34346901 %R 10.2196/25046 %U https://humanfactors.jmir.org/2021/3/e25046 %U https://doi.org/10.2196/25046 %U http://www.ncbi.nlm.nih.gov/pubmed/34346901 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 8 %P e29331 %T A Deep Neural Network for Estimating Low-Density Lipoprotein Cholesterol From Electronic Health Records: Real-Time Routine Clinical Application %A Hwang,Sangwon %A Gwon,Chanwoo %A Seo,Dong Min %A Cho,Jooyoung %A Kim,Jang-Young %A Uh,Young %+ Department of Laboratory Medicine, Yonsei University Wonju College of Medicine, 20, Ilsan- ro, Wonju, Gangwon-do, Wonju, 26426, Republic of Korea, 82 33 741 1592, u931018@yonsei.ac.kr %K low-density lipoprotein cholesterol %K deep neural network %K transfer learning %K real-time clinical application %D 2021 %7 3.8.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Previously, we constructed a deep neural network (DNN) model to estimate low-density lipoprotein cholesterol (LDL-C). Objective: To routinely provide estimated LDL-C levels, we applied the aforementioned DNN model to an electronic health record (EHR) system in real time (deep LDL-EHR). Methods: The Korea National Health and Nutrition Examination Survey and the Wonju Severance Christian Hospital (WSCH) datasets were used as training and testing datasets, respectively. We measured our proposed model’s performance by using 5 indices, including bias, root mean-square error, P10-P30, concordance, and correlation coefficient. For transfer learning (TL), we pretrained the DNN model using a training dataset and fine-tuned it using 30% of the testing dataset. Results: Based on 5 accuracy criteria, deep LDL-EHR generated inaccurate results compared with other methods for LDL-C estimation. By comparing the training and testing datasets, we found an overfitting problem. We then revised the DNN model using the TL algorithms and randomly selected subdata from the WSCH dataset. Therefore, the revised model (DNN+TL) exhibited the best performance among all methods. Conclusions: Our DNN+TL is expected to be suitable for routine real-time clinical application for LDL-C estimation in a clinical laboratory. %M 34342586 %R 10.2196/29331 %U https://medinform.jmir.org/2021/8/e29331 %U https://doi.org/10.2196/29331 %U http://www.ncbi.nlm.nih.gov/pubmed/34342586 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 8 %P e24405 %T Using Electronic Medical Record Data for Research in a Healthcare Information and Management Systems Society (HIMSS) Analytics Electronic Medical Record Adoption Model (EMRAM) Stage 7 Hospital in Beijing: Cross-sectional Study %A Li,Rui %A Niu,Yue %A Scott,Sarah Robbins %A Zhou,Chu %A Lan,Lan %A Liang,Zhigang %A Li,Jia %+ Information Center, Xuanwu Hospital, Capital Medical University, 45 Changchun Street, Beijing, 100053, China, 86 10 83929211, lij@xwhosp.org %K electronic medical records %K data utilization %K medical research %K China %D 2021 %7 3.8.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: With the proliferation of electronic medical record (EMR) systems, there is an increasing interest in utilizing EMR data for medical research; yet, there is no quantitative research on EMR data utilization for medical research purposes in China. Objective: This study aimed to understand how and to what extent EMR data are utilized for medical research purposes in a Healthcare Information and Management Systems Society (HIMSS) Analytics Electronic Medical Record Adoption Model (EMRAM) Stage 7 hospital in Beijing, China. Obstacles and issues in the utilization of EMR data were also explored to provide a foundation for the improved utilization of such data. Methods: For this descriptive cross-sectional study, cluster sampling from Xuanwu Hospital, one of two Stage 7 hospitals in Beijing, was conducted from 2016 to 2019. The utilization of EMR data was described as the number of requests, the proportion of requesters, and the frequency of requests per capita. Comparisons by year, professional title, and age were conducted by double-sided chi-square tests. Results: From 2016 to 2019, EMR data utilization was poor, as the proportion of requesters was 5.8% and the frequency was 0.1 times per person per year. The frequency per capita gradually slowed and older senior-level staff more frequently used EMR data compared with younger staff. Conclusions: The value of using EMR data for research purposes is not well studied in China. More research is needed to quantify to what extent EMR data are utilized across all hospitals in Beijing and how these systems can enhance future studies. The results of this study also suggest that young doctors may be less exposed or have less reason to access such research methods. %M 34342589 %R 10.2196/24405 %U https://medinform.jmir.org/2021/8/e24405 %U https://doi.org/10.2196/24405 %U http://www.ncbi.nlm.nih.gov/pubmed/34342589 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 8 %P e28266 %T Usage Patterns of Web-Based Stroke Calculators in Clinical Decision Support: Retrospective Analysis %A Kummer,Benjamin %A Shakir,Lubaina %A Kwon,Rachel %A Habboushe,Joseph %A Jetté,Nathalie %+ Department of Neurology, Icahn School of Medicine at Mount Sinai, One Gustave Levy Pl, Box 1137, New York, NY, 10029, United States, 1 2122415050, benjamin.kummer@mountsinai.org %K medical informatics %K clinical informatics %K mhealth %K digital health %K cerebrovascular disease %K medical calculators %K health information %K health information technology %K information technology %K economic health %K clinical health %K electronic health records %D 2021 %7 2.8.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Clinical scores are frequently used in the diagnosis and management of stroke. While medical calculators are increasingly important support tools for clinical decisions, the uptake and use of common medical calculators for stroke remain poorly characterized. Objective: We aimed to describe use patterns in frequently used stroke-related medical calculators for clinical decisions from a web-based support system. Methods: We conducted a retrospective study of calculators from MDCalc, a web-based and mobile app–based medical calculator platform based in the United States. We analyzed metadata tags from MDCalc’s calculator use data to identify all calculators related to stroke. Using relative page views as a measure of calculator use, we determined the 5 most frequently used stroke-related calculators between January 2016 and December 2018. For all 5 calculators, we determined cumulative and quarterly use, mode of access (eg, app or web browser), and both US and international distributions of use. We compared cumulative use in the 2016-2018 period with use from January 2011 to December 2015. Results: Over the study period, we identified 454 MDCalc calculators, of which 48 (10.6%) were related to stroke. Of these, the 5 most frequently used calculators were the CHA2DS2-VASc score for atrial fibrillation stroke risk calculator (5.5% of total and 32% of stroke-related page views), the Mean Arterial Pressure calculator (2.4% of total and 14.0% of stroke-related page views), the HAS-BLED score for major bleeding risk (1.9% of total and 11.4% of stroke-related page views), the National Institutes of Health Stroke Scale (NIHSS) score calculator (1.7% of total and 10.1% of stroke-related page views), and the CHADS2 score for atrial fibrillation stroke risk calculator (1.4% of total and 8.1% of stroke-related page views). Web browser was the most common mode of access, accounting for 82.7%-91.2% of individual stroke calculator page views. Access originated most frequently from the most populated regions within the United States. Internationally, use originated mostly from English-language countries. The NIHSS score calculator demonstrated the greatest increase in page views (238.1% increase) between the first and last quarters of the study period. Conclusions: The most frequently used stroke calculators were the CHA2DS2-VASc, Mean Arterial Pressure, HAS-BLED, NIHSS, and CHADS2. These were mainly accessed by web browser, from English-speaking countries, and from highly populated areas. Further studies should investigate barriers to stroke calculator adoption and the effect of calculator use on the application of best practices in cerebrovascular disease. %M 34338647 %R 10.2196/28266 %U https://medinform.jmir.org/2021/8/e28266 %U https://doi.org/10.2196/28266 %U http://www.ncbi.nlm.nih.gov/pubmed/34338647 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 7 %P e28496 %T Blockchain for Increased Trust in Virtual Health Care: Proof-of-Concept Study %A Hasselgren,Anton %A Hanssen Rensaa,Jens-Andreas %A Kralevska,Katina %A Gligoroski,Danilo %A Faxvaag,Arild %+ Department of Neuromedicine and Movement Science, Norwegian University of Science and Technology, Mellomila 71, Trondheim, Norway, 47 46948498, anton.hasselgren@ntnu.no %K blockchain %K ethereum %K decentralization %K Healthcare 4.0 %K virtualization %K trust %D 2021 %7 30.7.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Health care systems are currently undergoing a digital transformation that has been primarily triggered by emerging technologies, such as artificial intelligence, the Internet of Things, 5G, blockchain, and the digital representation of patients using (mobile) sensor devices. One of the results of this transformation is the gradual virtualization of care. Irrespective of the care environment, trust between caregivers and patients is essential for achieving favorable health outcomes. Given the many breaches of information security and patient safety, today’s health information system portfolios do not suffice as infrastructure for establishing and maintaining trust in virtual care environments. Objective: This study aims to establish a theoretical foundation for a complex health care system intervention that aims to exploit a cryptographically secured infrastructure for establishing and maintaining trust in virtualized care environments and, based on this theoretical foundation, present a proof of concept that fulfills the necessary requirements. Methods: This work applies the following framework for the design and evaluation of complex intervention research within health care: a review of the literature and expert consultation for technology forecasting. A proof of concept was developed by following the principles of design science and requirements engineering. Results: This study determined and defined the crucial functional and nonfunctional requirements and principles for enhancing trust between caregivers and patients within a virtualized health care environment. The cornerstone of our architecture is an approach that uses blockchain technology. The proposed decentralized system offers an innovative governance structure for a novel trust model. The presented theoretical design principles are supported by a concrete implementation of an Ethereum-based platform called VerifyMed. Conclusions: A service for enhancing trust in a virtualized health care environment that is built on a public blockchain has a high fit for purpose in Healthcare 4.0. %M 34328437 %R 10.2196/28496 %U https://www.jmir.org/2021/7/e28496 %U https://doi.org/10.2196/28496 %U http://www.ncbi.nlm.nih.gov/pubmed/34328437 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 7 %P e26823 %T Predicting Biologic Therapy Outcome of Patients With Spondyloarthritis: Joint Models for Longitudinal and Survival Analysis %A Barata,Carolina %A Rodrigues,Ana Maria %A Canhão,Helena %A Vinga,Susana %A Carvalho,Alexandra M %+ Instituto de Telecomunicações, Instituto Superior Técnico, Universidade de Lisboa, Lisbon, Portugal, 351 218 418 454, alexandra.carvalho@tecnico.ulisboa.pt %K data mining %K survival analysis %K joint models %K spondyloarthritis %K drug survival %K rheumatic disease %K electronic medical records %K medical records %D 2021 %7 30.7.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Rheumatic diseases are one of the most common chronic diseases worldwide. Among them, spondyloarthritis (SpA) is a group of highly debilitating diseases, with an early onset age, which significantly impacts patients’ quality of life, health care systems, and society in general. Recent treatment options consist of using biologic therapies, and establishing the most beneficial option according to the patients’ characteristics is a challenge that needs to be overcome. Meanwhile, the emerging availability of electronic medical records has made necessary the development of methods that can extract insightful information while handling all the challenges of dealing with complex, real-world data. Objective: The aim of this study was to achieve a better understanding of SpA patients’ therapy responses and identify the predictors that affect them, thereby enabling the prognosis of therapy success or failure. Methods: A data mining approach based on joint models for the survival analysis of the biologic therapy failure is proposed, which considers the information of both baseline and time-varying variables extracted from the electronic medical records of SpA patients from the database, Reuma.pt. Results: Our results show that being a male, starting biologic therapy at an older age, having a larger time interval between disease start and initiation of the first biologic drug, and being human leukocyte antigen (HLA)–B27 positive are indicators of a good prognosis for the biological drug survival; meanwhile, having disease onset or biologic therapy initiation occur in more recent years, a larger number of education years, and higher values of C-reactive protein or Bath Ankylosing Spondylitis Functional Index (BASFI) at baseline are all predictors of a greater risk of failure of the first biologic therapy. Conclusions: Among this Portuguese subpopulation of SpA patients, those who were male, HLA-B27 positive, and with a later biologic therapy starting date or a larger time interval between disease start and initiation of the first biologic therapy showed longer therapy adherence. Joint models proved to be a valuable tool for the analysis of electronic medical records in the field of rheumatic diseases and may allow for the identification of potential predictors of biologic therapy failure. %M 34328435 %R 10.2196/26823 %U https://medinform.jmir.org/2021/7/e26823 %U https://doi.org/10.2196/26823 %U http://www.ncbi.nlm.nih.gov/pubmed/34328435 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 7 %P e21929 %T The Fast Health Interoperability Resources (FHIR) Standard: Systematic Literature Review of Implementations, Applications, Challenges and Opportunities %A Ayaz,Muhammad %A Pasha,Muhammad F %A Alzahrani,Mohammed Y %A Budiarto,Rahmat %A Stiawan,Deris %+ Malaysia School of Information Technology, Monash University, Jalan Lagoon Selatan, Bandar Sunway, 47500, Malaysia, 60 0355146224, Muhammad.ayaz@monash.edu %K Fast Health Interoperability Resources %K FHIR %K electronic health record %K EHR %K clinical document architecture %K CDA %K Substitutable Medical Applications Reusable Technologies %K SMART %K HL7 %K health standard %K systematic literature review %D 2021 %7 30.7.2021 %9 Review %J JMIR Med Inform %G English %X Background: Information technology has shifted paper-based documentation in the health care sector into a digital form, in which patient information is transferred electronically from one place to another. However, there remain challenges and issues to resolve in this domain owing to the lack of proper standards, the growth of new technologies (mobile devices, tablets, ubiquitous computing), and health care providers who are reluctant to share patient information. Therefore, a solid systematic literature review was performed to understand the use of this new technology in the health care sector. To the best of our knowledge, there is a lack of comprehensive systematic literature reviews that focus on Fast Health Interoperability Resources (FHIR)-based electronic health records (EHRs). In addition, FHIR is the latest standard, which is in an infancy stage of development. Therefore, this is a hot research topic with great potential for further research in this domain. Objective: The main aim of this study was to explore and perform a systematic review of the literature related to FHIR, including the challenges, implementation, opportunities, and future FHIR applications. Methods: In January 2020, we searched articles published from January 2012 to December 2019 via all major digital databases in the field of computer science and health care, including ACM, IEEE Explorer, Springer, Google Scholar, PubMed, and ScienceDirect. We identified 8181 scientific articles published in this field, 80 of which met our inclusion criteria for further consideration. Results: The selected 80 scientific articles were reviewed systematically, and we identified open questions, challenges, implementation models, used resources, beneficiary applications, data migration approaches, and goals of FHIR. Conclusions: The literature analysis performed in this systematic review highlights the important role of FHIR in the health care domain in the near future. %M 34328424 %R 10.2196/21929 %U https://medinform.jmir.org/2021/7/e21929 %U https://doi.org/10.2196/21929 %U http://www.ncbi.nlm.nih.gov/pubmed/34328424 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 7 %P e22491 %T Predicting Unscheduled Emergency Department Return Visits Among Older Adults: Population-Based Retrospective Study %A Chen,Rai-Fu %A Cheng,Kuei-Chen %A Lin,Yu-Yin %A Chang,I-Chiu %A Tsai,Cheng-Han %+ Department of Information Management, National Chung Cheng University, No168, Sec 1, University Rd, Minhsiung, Chiayi County, 621301, Taiwan, 886 5 2720411 ext 16850, misicc@mis.ccu.edu.tw %K classification model %K decision tree %K emergency department %K older adult patients %K unscheduled return visits %D 2021 %7 28.7.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Unscheduled emergency department return visits (EDRVs) are key indicators for monitoring the quality of emergency medical care. A high return rate implies that the medical services provided by the emergency department (ED) failed to achieve the expected results of accurate diagnosis and effective treatment. Older adults are more susceptible to diseases and comorbidities than younger adults, and they exhibit unique and complex clinical characteristics that increase the difficulty of clinical diagnosis and treatment. Older adults also use more emergency medical resources than people in other age groups. Many studies have reviewed the causes of EDRVs among general ED patients; however, few have focused on older adults, although this is the age group with the highest rate of EDRVs. Objective: This aim of this study is to establish a model for predicting unscheduled EDRVs within a 72-hour period among patients aged 65 years and older. In addition, we aim to investigate the effects of the influencing factors on their unscheduled EDRVs. Methods: We used stratified and randomized data from Taiwan’s National Health Insurance Research Database and applied data mining techniques to construct a prediction model consisting of patient, disease, hospital, and physician characteristics. Records of ED visits by patients aged 65 years and older from 1996 to 2010 in the National Health Insurance Research Database were selected, and the final sample size was 49,252 records. Results: The decision tree of the prediction model achieved an acceptable overall accuracy of 76.80%. Economic status, chronic illness, and length of stay in the ED were the top three variables influencing unscheduled EDRVs. Those who stayed in the ED overnight or longer on their first visit were less likely to return. This study confirms the results of prior studies, which found that economically underprivileged older adults with chronic illness and comorbidities were more likely to return to the ED. Conclusions: Medical institutions can use our prediction model as a reference to improve medical management and clinical services by understanding the reasons for 72-hour unscheduled EDRVs in older adult patients. A possible solution is to create mechanisms that incorporate our prediction model and develop a support system with customized medical education for older patients and their family members before discharge. Meanwhile, a reasonably longer length of stay in the ED may help evaluate treatments and guide prognosis for older adult patients, and it may further reduce the rate of their unscheduled EDRVs. %M 34319244 %R 10.2196/22491 %U https://medinform.jmir.org/2021/7/e22491 %U https://doi.org/10.2196/22491 %U http://www.ncbi.nlm.nih.gov/pubmed/34319244 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 7 %P e20492 %T Assessing the Performance of Clinical Natural Language Processing Systems: Development of an Evaluation Methodology %A Canales,Lea %A Menke,Sebastian %A Marchesseau,Stephanie %A D’Agostino,Ariel %A del Rio-Bermudez,Carlos %A Taberna,Miren %A Tello,Jorge %+ MedSavana SL, Calle Gran Vía 30, Planta 10, Madrid, 28013, Spain, 34 627906138, jtello@savanamed.com %K natural language processing %K clinical natural language processing %K electronic health records %K gold standard %K reference standard %K sample size %D 2021 %7 23.7.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Clinical natural language processing (cNLP) systems are of crucial importance due to their increasing capability in extracting clinically important information from free text contained in electronic health records (EHRs). The conversion of a nonstructured representation of a patient’s clinical history into a structured format enables medical doctors to generate clinical knowledge at a level that was not possible before. Finally, the interpretation of the insights gained provided by cNLP systems has a great potential in driving decisions about clinical practice. However, carrying out robust evaluations of those cNLP systems is a complex task that is hindered by a lack of standard guidance on how to systematically approach them. Objective: Our objective was to offer natural language processing (NLP) experts a methodology for the evaluation of cNLP systems to assist them in carrying out this task. By following the proposed phases, the robustness and representativeness of the performance metrics of their own cNLP systems can be assured. Methods: The proposed evaluation methodology comprised five phases: (1) the definition of the target population, (2) the statistical document collection, (3) the design of the annotation guidelines and annotation project, (4) the external annotations, and (5) the cNLP system performance evaluation. We presented the application of all phases to evaluate the performance of a cNLP system called “EHRead Technology” (developed by Savana, an international medical company), applied in a study on patients with asthma. As part of the evaluation methodology, we introduced the Sample Size Calculator for Evaluations (SLiCE), a software tool that calculates the number of documents needed to achieve a statistically useful and resourceful gold standard. Results: The application of the proposed evaluation methodology on a real use-case study of patients with asthma revealed the benefit of the different phases for cNLP system evaluations. By using SLiCE to adjust the number of documents needed, a meaningful and resourceful gold standard was created. In the presented use-case, using as little as 519 EHRs, it was possible to evaluate the performance of the cNLP system and obtain performance metrics for the primary variable within the expected CIs. Conclusions: We showed that our evaluation methodology can offer guidance to NLP experts on how to approach the evaluation of their cNLP systems. By following the five phases, NLP experts can assure the robustness of their evaluation and avoid unnecessary investment of human and financial resources. Besides the theoretical guidance, we offer SLiCE as an easy-to-use, open-source Python library. %M 34297002 %R 10.2196/20492 %U https://medinform.jmir.org/2021/7/e20492 %U https://doi.org/10.2196/20492 %U http://www.ncbi.nlm.nih.gov/pubmed/34297002 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 7 %P e29226 %T Predicting Antituberculosis Drug–Induced Liver Injury Using an Interpretable Machine Learning Method: Model Development and Validation Study %A Zhong,Tao %A Zhuang,Zian %A Dong,Xiaoli %A Wong,Ka Hing %A Wong,Wing Tak %A Wang,Jian %A He,Daihai %A Liu,Shengyuan %+ Department of Tuberculosis Control, Shenzhen Nanshan Center for Chronic Disease Control, Hua Ming Road No 7, Nanshan District, Shenzhen, 518000, China, 86 13543301395, jfk@sznsmby.com %K accuracy %K drug %K drug-induced liver injury %K high accuracy %K injury %K interpretability %K interpretation %K liver %K machine learning %K model %K prediction %K treatment %K tuberculosis %K XGBoost algorithm %D 2021 %7 20.7.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Tuberculosis (TB) is a pandemic, being one of the top 10 causes of death and the main cause of death from a single source of infection. Drug-induced liver injury (DILI) is the most common and serious side effect during the treatment of TB. Objective: We aim to predict the status of liver injury in patients with TB at the clinical treatment stage. Methods: We designed an interpretable prediction model based on the XGBoost algorithm and identified the most robust and meaningful predictors of the risk of TB-DILI on the basis of clinical data extracted from the Hospital Information System of Shenzhen Nanshan Center for Chronic Disease Control from 2014 to 2019. Results: In total, 757 patients were included, and 287 (38%) had developed TB-DILI. Based on values of relative importance and area under the receiver operating characteristic curve, machine learning tools selected patients’ most recent alanine transaminase levels, average rate of change of patients’ last 2 measures of alanine transaminase levels, cumulative dose of pyrazinamide, and cumulative dose of ethambutol as the best predictors for assessing the risk of TB-DILI. In the validation data set, the model had a precision of 90%, recall of 74%, classification accuracy of 76%, and balanced error rate of 77% in predicting cases of TB-DILI. The area under the receiver operating characteristic curve score upon 10-fold cross-validation was 0.912 (95% CI 0.890-0.935). In addition, the model provided warnings of high risk for patients in advance of DILI onset for a median of 15 (IQR 7.3-27.5) days. Conclusions: Our model shows high accuracy and interpretability in predicting cases of TB-DILI, which can provide useful information to clinicians to adjust the medication regimen and avoid more serious liver injury in patients. %M 34283036 %R 10.2196/29226 %U https://medinform.jmir.org/2021/7/e29226 %U https://doi.org/10.2196/29226 %U http://www.ncbi.nlm.nih.gov/pubmed/34283036 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 7 %P e16750 %T Barriers to Dissemination of Local Health Data Faced by US State Agencies: Survey Study of Behavioral Risk Factor Surveillance System Coordinators %A Ahuja,Manik %A Aseltine Jr,Robert %+ Department of Health Services Management and Policy, College of Public Health, East Tennessee State University, 41B Lamb Hall, Johnson City, TN, 37604, United States, 1 4234396637, ahujam@etsu.edu %K web-based data query systems, WDQS %K health data %K population health %K dissemination of local health data %D 2021 %7 13.7.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Advances in information technology have paved the way to facilitate accessibility to population-level health data through web-based data query systems (WDQSs). Despite these advances in technology, US state agencies face many challenges related to the dissemination of their local health data. It is essential for the public to have access to high-quality data that are easy to interpret, reliable, and trusted. These challenges have been at the forefront throughout the COVID-19 pandemic. Objective: The purpose of this study is to identify the most significant challenges faced by state agencies, from the perspective of the Behavioral Risk Factor Surveillance System (BRFSS) coordinator from each state, and to assess if the coordinators from states with a WDQS perceive these challenges differently. Methods: We surveyed BRFSS coordinators (N=43) across all 50 US states and the District of Columbia. We surveyed the participants about contextual factors and asked them to rate system aspects and challenges they faced with their health data system on a Likert scale. We used two-sample t tests to compare the means of the ratings by participants from states with and without a WDQS. Results: Overall, 41/43 states (95%) make health data available over the internet, while 65% (28/43) employ a WDQS. States with a WDQS reported greater challenges (P=.01) related to the cost of hardware and software (mean score 3.44/4, 95% CI 3.09-3.78) than states without a WDQS (mean score 2.63/4, 95% CI 2.25-3.00). The system aspect of standardization of vocabulary scored more favorably (P=.01) in states with a WDQS (mean score 3.32/5, 95% CI 2.94-3.69) than in states without a WDQS (mean score 2.85/5, 95% CI 2.47-3.22). Conclusions: Securing of adequate resources and commitment to standardization are vital in the dissemination of local-level health data. Factors such as receiving data in a timely manner, privacy, and political opposition are less significant barriers than anticipated. %M 34255650 %R 10.2196/16750 %U https://www.jmir.org/2021/7/e16750 %U https://doi.org/10.2196/16750 %U http://www.ncbi.nlm.nih.gov/pubmed/34255650 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 7 %P e26151 %T Clinically Applicable Segmentation of Head and Neck Anatomy for Radiotherapy: Deep Learning Algorithm Development and Validation Study %A Nikolov,Stanislav %A Blackwell,Sam %A Zverovitch,Alexei %A Mendes,Ruheena %A Livne,Michelle %A De Fauw,Jeffrey %A Patel,Yojan %A Meyer,Clemens %A Askham,Harry %A Romera-Paredes,Bernadino %A Kelly,Christopher %A Karthikesalingam,Alan %A Chu,Carlton %A Carnell,Dawn %A Boon,Cheng %A D'Souza,Derek %A Moinuddin,Syed Ali %A Garie,Bethany %A McQuinlan,Yasmin %A Ireland,Sarah %A Hampton,Kiarna %A Fuller,Krystle %A Montgomery,Hugh %A Rees,Geraint %A Suleyman,Mustafa %A Back,Trevor %A Hughes,Cían Owen %A Ledsam,Joseph R %A Ronneberger,Olaf %+ Google Health, 6 Pancras Square, London, N1C 4AG, United Kingdom, 1 650 253 0000, cianh@google.com %K radiotherapy %K segmentation %K contouring %K machine learning %K artificial intelligence %K UNet %K convolutional neural networks %K surface DSC %D 2021 %7 12.7.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Over half a million individuals are diagnosed with head and neck cancer each year globally. Radiotherapy is an important curative treatment for this disease, but it requires manual time to delineate radiosensitive organs at risk. This planning process can delay treatment while also introducing interoperator variability, resulting in downstream radiation dose differences. Although auto-segmentation algorithms offer a potentially time-saving solution, the challenges in defining, quantifying, and achieving expert performance remain. Objective: Adopting a deep learning approach, we aim to demonstrate a 3D U-Net architecture that achieves expert-level performance in delineating 21 distinct head and neck organs at risk commonly segmented in clinical practice. Methods: The model was trained on a data set of 663 deidentified computed tomography scans acquired in routine clinical practice and with both segmentations taken from clinical practice and segmentations created by experienced radiographers as part of this research, all in accordance with consensus organ at risk definitions. Results: We demonstrated the model’s clinical applicability by assessing its performance on a test set of 21 computed tomography scans from clinical practice, each with 21 organs at risk segmented by 2 independent experts. We also introduced surface Dice similarity coefficient, a new metric for the comparison of organ delineation, to quantify the deviation between organ at risk surface contours rather than volumes, better reflecting the clinical task of correcting errors in automated organ segmentations. The model’s generalizability was then demonstrated on 2 distinct open-source data sets, reflecting different centers and countries to model training. Conclusions: Deep learning is an effective and clinically applicable technique for the segmentation of the head and neck anatomy for radiotherapy. With appropriate validation studies and regulatory approvals, this system could improve the efficiency, consistency, and safety of radiotherapy pathways. %M 34255661 %R 10.2196/26151 %U https://www.jmir.org/2021/7/e26151 %U https://doi.org/10.2196/26151 %U http://www.ncbi.nlm.nih.gov/pubmed/34255661 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 7 %P e28009 %T Application of Telemedicine Services Based on a Regional Telemedicine Platform in China From 2014 to 2020: Longitudinal Trend Analysis %A Cui,Fangfang %A He,Xianying %A Zhai,Yunkai %A Lyu,Minzhao %A Shi,Jinming %A Sun,Dongxu %A Jiang,Shuai %A Li,Chenchen %A Zhao,Jie %+ National Engineering Laboratory for Internet Medical Systems and Applications, The First Affiliated Hospital of Zhengzhou University, 1 Jianshe Road, Erqi District, Zhengzhou, China, 86 371 67966215, zhaojie@zzu.edu.cn %K telemedicine %K regional telemedicine service platform %K remote consultation %K efficiency %K satisfaction degree %K telehealth %K mobile health %K mHealth %K remote %K China %D 2021 %7 12.7.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Telemedicine that combines information technology and health care augments the operational model of traditional medical services and brings new opportunities to the medical field. China promotes telemedicine with great efforts, and its practices in the deployment of telemedicine platforms and delivery of services have become important references for the research and development in this field. Objective: Our work described in this paper focuses on a regional telemedicine platform that was built in 2014. We analyzed the system design scheme and remote consultations that were conducted via the system to understand the deployment and service delivery processes of a representative telemedicine platform in China. Methods: We collected information on remote consultations conducted from 2015 to 2020 via the regional telemedicine platform that employs a centralized architectural system model. We used graphs and statistical methods to describe the changing trends of service volume of remote consultation, geographical and demographic distribution of patients, and waiting time and duration of consultations. The factors that affect consultation duration and patient referral were analyzed by multivariable linear regression models and binary logistic regression models, respectively. The attitudes toward telemedicine of 225 medical practitioners and 225 patients were collected using the snowball sampling method. Results: The regional telemedicine platform covers all levels of medical institutions and hospitals in all 18 cities of Henan Province as well as some interprovince hospitals. From 2015 to 2020, 103,957 remote medical consultations were conducted via the platform with an annual increasing rate of 0.64%. A total of 86.64% (90,069/103,957) of medical institutions (as clients) that applied for remote consultations were tier 1 or 2 and from less-developed regions; 65.65% (68,243/103,945) of patients who applied for remote consultations were aged over 50 years. The numbers of consultations were high for departments focusing in the treatment of chronic diseases such as neurology, respiratory medicine, and oncology. The invited experts were mainly experienced doctors with senior professional titles. Year of consultation, tier of hospital, consultation department, and necessity of patient referral were the main factors affecting the duration of consultations. In surveys, we found that 60.4% (136/225) of medical practitioners and 53.8% (121/225) of patients had high satisfaction and believed that telemedicine is of vital importance for the treatment of illness. Conclusions: The development of telemedicine in China shows a growing trend and provides great benefits especially to medical institutions located in less developed regions and senior citizens who have less mobility. Cases of remote consultations are mainly for chronic diseases. At present, the importance and necessity of telemedicine are well recognized by both patients and medical practitioners. However, the waiting time needs to be further reduced to improve the efficiency of remote medical services. %M 34255686 %R 10.2196/28009 %U https://www.jmir.org/2021/7/e28009 %U https://doi.org/10.2196/28009 %U http://www.ncbi.nlm.nih.gov/pubmed/34255686 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 7 %P e29986 %T Ambulatory Risk Models for the Long-Term Prevention of Sepsis: Retrospective Study %A Lee,Jewel Y %A Molani,Sevda %A Fang,Chen %A Jade,Kathleen %A O'Mahony,D Shane %A Kornilov,Sergey A %A Mico,Lindsay T %A Hadlock,Jennifer J %+ Institute for Systems Biology, 401 Terry Ave N, Seattle, WA, 98109, United States, jhadlock@isbscience.org %K sepsis %K machine learning %K electronic health records %K risk prediction %K clinical decision making %K prevention %K risk factors %D 2021 %7 8.7.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Sepsis is a life-threatening condition that can rapidly lead to organ damage and death. Existing risk scores predict outcomes for patients who have already become acutely ill. Objective: We aimed to develop a model for identifying patients at risk of getting sepsis within 2 years in order to support the reduction of sepsis morbidity and mortality. Methods: Machine learning was applied to 2,683,049 electronic health records (EHRs) with over 64 million encounters across five states to develop models for predicting a patient’s risk of getting sepsis within 2 years. Features were selected to be easily obtainable from a patient’s chart in real time during ambulatory encounters. Results: The models showed consistent prediction scores, with the highest area under the receiver operating characteristic curve of 0.82 and a positive likelihood ratio of 2.9 achieved with gradient boosting on all features combined. Predictive features included age, sex, ethnicity, average ambulatory heart rate, standard deviation of BMI, and the number of prior medical conditions and procedures. The findings identified both known and potential new risk factors for long-term sepsis. Model variations also illustrated trade-offs between incrementally higher accuracy, implementability, and interpretability. Conclusions: Accurate implementable models were developed to predict the 2-year risk of sepsis, using EHR data that is easy to obtain from ambulatory encounters. These results help advance the understanding of sepsis and provide a foundation for future trials of risk-informed preventive care. %M 34086596 %R 10.2196/29986 %U https://medinform.jmir.org/2021/7/e29986 %U https://doi.org/10.2196/29986 %U http://www.ncbi.nlm.nih.gov/pubmed/34086596 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 10 %N 7 %P e27532 %T Predicting and Responding to Clinical Deterioration in Hospitalized Patients by Using Artificial Intelligence: Protocol for a Mixed Methods, Stepped Wedge Study %A Holdsworth,Laura M %A Kling,Samantha M R %A Smith,Margaret %A Safaeinili,Nadia %A Shieh,Lisa %A Vilendrer,Stacie %A Garvert,Donn W %A Winget,Marcy %A Asch,Steven M %A Li,Ron C %+ Department of Medicine, School of Medicine, Stanford University, 1265 Welch Rd, Stanford, CA, , United States, 1 650 736 3391, lmh1@stanford.edu %K artificial intelligence %K clinical deterioration %K rapid response team %K mixed methods %K workflow %K predictive models, SEIPS 2.0 %D 2021 %7 7.7.2021 %9 Protocol %J JMIR Res Protoc %G English %X Background: The early identification of clinical deterioration in patients in hospital units can decrease mortality rates and improve other patient outcomes; yet, this remains a challenge in busy hospital settings. Artificial intelligence (AI), in the form of predictive models, is increasingly being explored for its potential to assist clinicians in predicting clinical deterioration. Objective: Using the Systems Engineering Initiative for Patient Safety (SEIPS) 2.0 model, this study aims to assess whether an AI-enabled work system improves clinical outcomes, describe how the clinical deterioration index (CDI) predictive model and associated work processes are implemented, and define the emergent properties of the AI-enabled work system that mediate the observed clinical outcomes. Methods: This study will use a mixed methods approach that is informed by the SEIPS 2.0 model to assess both processes and outcomes and focus on how physician-nurse clinical teams are affected by the presence of AI. The intervention will be implemented in hospital medicine units based on a modified stepped wedge design featuring three stages over 11 months—stage 0 represents a baseline period 10 months before the implementation of the intervention; stage 1 introduces the CDI predictions to physicians only and triggers a physician-driven workflow; and stage 2 introduces the CDI predictions to the multidisciplinary team, which includes physicians and nurses, and triggers a nurse-driven workflow. Quantitative data will be collected from the electronic health record for the clinical processes and outcomes. Interviews will be conducted with members of the multidisciplinary team to understand how the intervention changes the existing work system and processes. The SEIPS 2.0 model will provide an analytic framework for a mixed methods analysis. Results: A pilot period for the study began in December 2020, and the results are expected in mid-2022. Conclusions: This protocol paper proposes an approach to evaluation that recognizes the importance of assessing both processes and outcomes to understand how a multifaceted AI-enabled intervention affects the complex team-based work of identifying and managing clinical deterioration. International Registered Report Identifier (IRRID): PRR1-10.2196/27532 %M 34255728 %R 10.2196/27532 %U https://www.researchprotocols.org/2021/7/e27532 %U https://doi.org/10.2196/27532 %U http://www.ncbi.nlm.nih.gov/pubmed/34255728 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 7 %P e26817 %T Frustration With Technology and its Relation to Emotional Exhaustion Among Health Care Workers: Cross-sectional Observational Study %A Tawfik,Daniel S %A Sinha,Amrita %A Bayati,Mohsen %A Adair,Kathryn C %A Shanafelt,Tait D %A Sexton,J Bryan %A Profit,Jochen %+ Department of Pediatrics, Stanford University School of Medicine, 770 Welch Road, Suite 435, Palo Alto, CA, 94304, United States, 1 650 723 9902, dtawfik@stanford.edu %K frustration with technology %K emotional exhaustion %K professional burnout %K work-life integration %K biomedical technology %K work-life balance %K user-centered design %K electronic health records %K medical informatics applications %K health information systems %D 2021 %7 6.7.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: New technology adoption is common in health care, but it may elicit frustration if end users are not sufficiently considered in their design or trained in their use. These frustrations may contribute to burnout. Objective: This study aimed to evaluate and quantify health care workers’ frustration with technology and its relationship with emotional exhaustion, after controlling for measures of work-life integration that may indicate excessive job demands. Methods: This was a cross-sectional, observational study of health care workers across 31 Michigan hospitals. We used the Safety, Communication, Operational Reliability, and Engagement (SCORE) survey to measure work-life integration and emotional exhaustion among the survey respondents. We used mixed-effects hierarchical linear regression to evaluate the relationship among frustration with technology, other components of work-life integration, and emotional exhaustion, with adjustment for unit and health care worker characteristics. Results: Of 15,505 respondents, 5065 (32.7%) reported that they experienced frustration with technology on at least 3-5 days per week. Frustration with technology was associated with higher scores for the composite Emotional Exhaustion scale (r=0.35, P<.001) and each individual item on the Emotional Exhaustion scale (r=0.29-0.36, P<.001 for all). Each 10-point increase in the frustration with technology score was associated with a 1.2-point increase (95% CI 1.1-1.4) in emotional exhaustion (both measured on 100-point scales), after adjustment for other work-life integration items and unit and health care worker characteristics. Conclusions: This study found that frustration with technology and several other markers of work-life integration are independently associated with emotional exhaustion among health care workers. Frustration with technology is common but not ubiquitous among health care workers, and it is one of several work-life integration factors associated with emotional exhaustion. Minimizing frustration with health care technology may be an effective approach in reducing burnout among health care workers. %M 34255674 %R 10.2196/26817 %U https://www.jmir.org/2021/7/e26817 %U https://doi.org/10.2196/26817 %U http://www.ncbi.nlm.nih.gov/pubmed/34255674 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 7 %P e28729 %T The Association Between Using a Mobile Version of an Electronic Health Record and the Well-Being of Nurses: Cross-sectional Survey Study %A Heponiemi,Tarja %A Kaihlanen,Anu-Marja %A Gluschkoff,Kia %A Saranto,Kaija %A Nissinen,Sari %A Laukka,Elina %A Vehko,Tuulikki %+ Department of Public Health and Welfare, Finnish Institute for Health and Welfare, PO Box 30, Helsinki, 00271, Finland, 358 295247434, tarja.heponiemi@thl.fi %K stress related to information systems %K time pressure %K usability %K stress %K health and social care %D 2021 %7 6.7.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Mobile devices such as tablets and smartphones are increasingly being used in health care in many developed countries. Nurses form the largest group in health care that uses electronic health records (EHRs) and their mobile versions. Mobile devices are suggested to promote nurses’ workflow, constant updating of patient information, and improve the communication within the health care team. However, little is known about their effect on nurses’ well-being. Objective: This study aimed to examine the association between using a mobile version of the EHR and nurses’ perceived time pressure, stress related to information systems, and self-rated stress. Moreover, we examined whether mobile device use modifies the associations of EHR usability (ease of use and technical quality), experience in using EHRs, and number of systems in daily use with these well-being indicators. Methods: This was a cross-sectional population-based survey study among 3610 Finnish registered nurses gathered in 2020. The aforesaid associations were examined using analyses of covariance and logistic regression adjusted for age, gender, and employment sector (hospital, primary care, social service, and other). Results: Nurses who used the mobile version of their EHR had higher levels of time pressure (F1,3537=14.96, P<.001) and stress related to information systems (F1,3537=6.11, P=.01), compared with those who did not use mobile versions. Moreover, the interactions of mobile device use with experience in using EHRs (F1,3581=14.93, P<.001), ease of use (F1,3577=10.16, P=.001), and technical quality (F1,3577=6.45, P=.01) were significant for stress related to information systems. Inexperience in using EHRs, low levels of ease of use, and technical quality were associated with higher stress related to information systems and this association was more pronounced among those who used mobile devices. That is, the highest levels of stress related to information systems were perceived among those who used mobile devices as well as among inexperienced EHR users or those who perceived usability problems in their EHRs. Conclusions: According to our results, it seems that at present mobile device use is not beneficial for the nurses’ well-being. In addition, mobile device use seems to intensify the negative effects of usability issues related to EHRs. In particular, inexperienced users of EHRs seem to be at a disadvantage when using mobile devices. Thus, we suggest that EHRs and their mobile versions should be improved such that they would be easier to use and would better support the nurses’ workflow (eg, improvements to problems related to small display, user interface, and data entry). Moreover, additional training on EHRs, their mobile versions, and workflow related to these should be provided to nurses. %M 34255704 %R 10.2196/28729 %U https://medinform.jmir.org/2021/7/e28729 %U https://doi.org/10.2196/28729 %U http://www.ncbi.nlm.nih.gov/pubmed/34255704 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 7 %P e24796 %T Machine Learning Methods for the Diagnosis of Chronic Obstructive Pulmonary Disease in Healthy Subjects: Retrospective Observational Cohort Study %A Muro,Shigeo %A Ishida,Masato %A Horie,Yoshiharu %A Takeuchi,Wataru %A Nakagawa,Shunki %A Ban,Hideyuki %A Nakagawa,Tohru %A Kitamura,Tetsuhisa %+ Department of Data Science, Medical, AstraZeneca KK, 3-1, Ofuka-cho, Kita-ku, Osaka, 5300011, Japan, 81 81 6 4802 3600, yoshiharu.horie@astrazeneca.com %K chronic obstructive pulmonary disease %K airflow limitation %K medical check-up %K Gradient Boosting Decision Tree %K logistic regression %D 2021 %7 6.7.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Airflow limitation is a critical physiological feature in chronic obstructive pulmonary disease (COPD), for which long-term exposure to noxious substances, including tobacco smoke, is an established risk. However, not all long-term smokers develop COPD, meaning that other risk factors exist. Objective: This study aimed to predict the risk factors for COPD diagnosis using machine learning in an annual medical check-up database. Methods: In this retrospective observational cohort study (ARTDECO [Analysis of Risk Factors to Detect COPD]), annual medical check-up records for all Hitachi Ltd employees in Japan collected from April 1998 to March 2019 were analyzed. Employees who provided informed consent via an opt-out model were screened and those aged 30 to 75 years without a prior diagnosis of COPD/asthma or a history of cancer were included. The database included clinical measurements (eg, pulmonary function tests) and questionnaire responses. To predict the risk factors for COPD diagnosis within a 3-year period, the Gradient Boosting Decision Tree machine learning (XGBoost) method was applied as a primary approach, with logistic regression as a secondary method. A diagnosis of COPD was made when the ratio of the prebronchodilator forced expiratory volume in 1 second (FEV1) to prebronchodilator forced vital capacity (FVC) was <0.7 during two consecutive examinations. Results: Of the 26,101 individuals screened, 1213 met the exclusion criteria, and thus, 24,815 individuals were included in the analysis. The top 10 predictors for COPD diagnosis were FEV1/FVC, smoking status, allergic symptoms, cough, pack years, hemoglobin A1c, serum albumin, mean corpuscular volume, percent predicted vital capacity, and percent predicted value of FEV1. The areas under the receiver operating characteristic curves of the XGBoost model and the logistic regression model were 0.956 and 0.943, respectively. Conclusions: Using a machine learning model in this longitudinal database, we identified a number of parameters as risk factors other than smoking exposure or lung function to support general practitioners and occupational health physicians to predict the development of COPD. Further research to confirm our results is warranted, as our analysis involved a database used only in Japan. %M 34255684 %R 10.2196/24796 %U https://medinform.jmir.org/2021/7/e24796 %U https://doi.org/10.2196/24796 %U http://www.ncbi.nlm.nih.gov/pubmed/34255684 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 7 %P e27527 %T Relation Classification for Bleeding Events From Electronic Health Records Using Deep Learning Systems: An Empirical Study %A Mitra,Avijit %A Rawat,Bhanu Pratap Singh %A McManus,David D %A Yu,Hong %+ Department of Computer Science, University of Massachusetts Lowell, 1 University Avenue, Lowell, MA, , United States, 1 508 612 7292, Hong_Yu@uml.edu %K bleeding %K relation classification %K electronic health records %K CNN %K GCN %K BERT %D 2021 %7 2.7.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Accurate detection of bleeding events from electronic health records (EHRs) is crucial for identifying and characterizing different common and serious medical problems. To extract such information from EHRs, it is essential to identify the relations between bleeding events and related clinical entities (eg, bleeding anatomic sites and lab tests). With the advent of natural language processing (NLP) and deep learning (DL)-based techniques, many studies have focused on their applicability for various clinical applications. However, no prior work has utilized DL to extract relations between bleeding events and relevant entities. Objective: In this study, we aimed to evaluate multiple DL systems on a novel EHR data set for bleeding event–related relation classification. Methods: We first expert annotated a new data set of 1046 deidentified EHR notes for bleeding events and their attributes. On this data set, we evaluated three state-of-the-art DL architectures for the bleeding event relation classification task, namely, convolutional neural network (CNN), attention-guided graph convolutional network (AGGCN), and Bidirectional Encoder Representations from Transformers (BERT). We used three BERT-based models, namely, BERT pretrained on biomedical data (BioBERT), BioBERT pretrained on clinical text (Bio+Clinical BERT), and BioBERT pretrained on EHR notes (EhrBERT). Results: Our experiments showed that the BERT-based models significantly outperformed the CNN and AGGCN models. Specifically, BioBERT achieved a macro F1 score of 0.842, outperforming both the AGGCN (macro F1 score, 0.828) and CNN models (macro F1 score, 0.763) by 1.4% (P<.001) and 7.9% (P<.001), respectively. Conclusions: In this comprehensive study, we explored and compared different DL systems to classify relations between bleeding events and other medical concepts. On our corpus, BERT-based models outperformed other DL models for identifying the relations of bleeding-related entities. In addition to pretrained contextualized word representation, BERT-based models benefited from the use of target entity representation over traditional sequence representation %M 34255697 %R 10.2196/27527 %U https://medinform.jmir.org/2021/7/e27527 %U https://doi.org/10.2196/27527 %U http://www.ncbi.nlm.nih.gov/pubmed/34255697 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 6 %P e27591 %T A National, Semantic-Driven, Three-Pillar Strategy to Enable Health Data Secondary Usage Interoperability for Research Within the Swiss Personalized Health Network: Methodological Study %A Gaudet-Blavignac,Christophe %A Raisaro,Jean Louis %A Touré,Vasundra %A Österle,Sabine %A Crameri,Katrin %A Lovis,Christian %+ Division of Medical Information Sciences, Geneva University Hospitals, Rue Gabrielle-Perret-Gentil 4, Geneva, 1205, Switzerland, 41 223726201, christophe.gaudet-blavignac@hcuge.ch %K interoperability %K clinical data reuse %K personalized medicine %D 2021 %7 24.6.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Interoperability is a well-known challenge in medical informatics. Current trends in interoperability have moved from a data model technocentric approach to sustainable semantics, formal descriptive languages, and processes. Despite many initiatives and investments for decades, the interoperability challenge remains crucial. The need for data sharing for most purposes ranging from patient care to secondary uses, such as public health, research, and quality assessment, faces unmet problems. Objective: This work was performed in the context of a large Swiss Federal initiative aiming at building a national infrastructure for reusing consented data acquired in the health care and research system to enable research in the field of personalized medicine in Switzerland. The initiative is the Swiss Personalized Health Network (SPHN). This initiative is providing funding to foster use and exchange of health-related data for research. As part of the initiative, a national strategy to enable a semantically interoperable clinical data landscape was developed and implemented. Methods: A deep analysis of various approaches to address interoperability was performed at the start, including large frameworks in health care, such as Health Level Seven (HL7) and Integrating Healthcare Enterprise (IHE), and in several domains, such as regulatory agencies (eg, Clinical Data Interchange Standards Consortium [CDISC]) and research communities (eg, Observational Medical Outcome Partnership [OMOP]), to identify bottlenecks and assess sustainability. Based on this research, a strategy composed of three pillars was designed. It has strong multidimensional semantics, descriptive formal language for exchanges, and as many data models as needed to comply with the needs of various communities. Results: This strategy has been implemented stepwise in Switzerland since the middle of 2019 and has been adopted by all university hospitals and high research organizations. The initiative is coordinated by a central organization, the SPHN Data Coordination Center of the SIB Swiss Institute of Bioinformatics. The semantics is mapped by domain experts on various existing standards, such as Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT), Logical Observation Identifiers Names and Codes (LOINC), and International Classification of Diseases (ICD). The resource description framework (RDF) is used for storing and transporting data, and to integrate information from different sources and standards. Data transformers based on SPARQL query language are implemented to convert RDF representations to the numerous data models required by the research community or bridge with other systems, such as electronic case report forms. Conclusions: The SPHN strategy successfully implemented existing standards in a pragmatic and applicable way. It did not try to build any new standards but used existing ones in a nondogmatic way. It has now been funded for another 4 years, bringing the Swiss landscape into a new dimension to support research in the field of personalized medicine and large interoperable clinical data. %M 34185008 %R 10.2196/27591 %U https://medinform.jmir.org/2021/6/e27591/ %U https://doi.org/10.2196/27591 %U http://www.ncbi.nlm.nih.gov/pubmed/34185008 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 6 %P e25124 %T Enhancing Obstructive Sleep Apnea Diagnosis With Screening Through Disease Phenotypes: Algorithm Development and Validation %A Ferreira-Santos,Daniela %A Rodrigues,Pedro Pereira %+ MEDCIDS-FMUP – Community Medicine, Information and Decision Sciences, Faculty of Medicine of the University of Porto, Rua Dr. Plácido da Costa, s/n, Porto, 4200-450, Portugal, 351 22 551 3622, danielasantos@med.up.pt %K obstructive sleep apnea %K screening %K risk factors %K phenotypes %K Bayesian network classifiers %D 2021 %7 22.6.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: The American Academy of Sleep Medicine guidelines suggest that clinical prediction algorithms can be used in patients with obstructive sleep apnea (OSA) without replacing polysomnography, which is the gold standard. Objective: This study aims to develop a clinical decision support system for OSA diagnosis according to its standard definition (apnea-hypopnea index plus symptoms), identifying individuals with high pretest probability based on risk and diagnostic factors. Methods: A total of 47 predictive variables were extracted from a cohort of patients who underwent polysomnography. A total of 14 variables that were univariately significant were then used to compute the distance between patients with OSA, defining a hierarchical clustering structure from which patient phenotypes were derived and described. Affinity from individuals at risk of OSA phenotypes was later computed, and cluster membership was used as an additional predictor in a Bayesian network classifier (model B). Results: A total of 318 patients at risk were included, of whom 207 (65.1%) individuals were diagnosed with OSA (111, 53.6% with mild; 50, 24.2% with moderate; and 46, 22.2% with severe). On the basis of predictive variables, 3 phenotypes were defined (74/207, 35.7% low; 104/207, 50.2% medium; and 29/207, 14.1% high), with an increasing prevalence of symptoms and comorbidities, the latter describing older and obese patients, and a substantial increase in some comorbidities, suggesting their beneficial use as combined predictors (median apnea-hypopnea indices of 10, 14, and 31, respectively). Cross-validation results demonstrated that the inclusion of OSA phenotypes as an adjusting predictor in a Bayesian classifier improved screening specificity (26%, 95% CI 24-29, to 38%, 95% CI 35-40) while maintaining a high sensitivity (93%, 95% CI 91-95), with model B doubling the diagnostic model effectiveness (diagnostic odds ratio of 8.14). Conclusions: Defined OSA phenotypes are a sensitive tool that enhances our understanding of the disease and allows the derivation of a predictive algorithm that can clearly outperform symptom-based guideline recommendations as a rule-out approach for screening. %M 34156340 %R 10.2196/25124 %U https://medinform.jmir.org/2021/6/e25124 %U https://doi.org/10.2196/25124 %U http://www.ncbi.nlm.nih.gov/pubmed/34156340 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 6 %P e28247 %T A Novel Metric to Quantify the Effect of Pathway Enrichment Evaluation With Respect to Biomedical Text-Mined Terms: Development and Feasibility Study %A Qin,Xuan %A Yao,Xinzhi %A Xia,Jingbo %+ Hubei Key Lab of Agricultural Bioinformatics, College of Informatics, Huazhong Agricultural University, 1#, Lion Rock Street, Hongshan District, Hubei Province, Wuhan, 430070, China, 86 02787288509, xiajingbo.math@gmail.com %K pathway enrichment %K metric %K evaluation %K text mining %D 2021 %7 18.6.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Natural language processing has long been applied in various applications for biomedical knowledge inference and discovery. Enrichment analysis based on named entity recognition is a classic application for inferring enriched associations in terms of specific biomedical entities such as gene, chemical, and mutation. Objective: The aim of this study was to investigate the effect of pathway enrichment evaluation with respect to biomedical text-mining results and to develop a novel metric to quantify the effect. Methods: Four biomedical text mining methods were selected to represent natural language processing methods on drug-related gene mining. Subsequently, a pathway enrichment experiment was performed by using the mined genes, and a series of inverse pathway frequency (IPF) metrics was proposed accordingly to evaluate the effect of pathway enrichment. Thereafter, 7 IPF metrics and traditional P value metrics were compared in simulation experiments to test the robustness of the proposed metrics. Results: IPF metrics were evaluated in a case study of rapamycin-related gene set. By applying the best IPF metrics in a pathway enrichment simulation test, a novel discovery of drug efficacy of rapamycin for breast cancer was replicated from the data chosen prior to the year 2000. Our findings show the effectiveness of the best IPF metric in support of knowledge discovery in new drug use. Further, the mechanism underlying the drug-disease association was visualized by Cytoscape. Conclusions: The results of this study suggest the effectiveness of the proposed IPF metrics in pathway enrichment evaluation as well as its application in drug use discovery. %M 34142969 %R 10.2196/28247 %U https://medinform.jmir.org/2021/6/e28247 %U https://doi.org/10.2196/28247 %U http://www.ncbi.nlm.nih.gov/pubmed/34142969 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 6 %P e27345 %T Development, Status Quo, and Challenges to China’s Health Informatization During COVID-19: Evaluation and Recommendations %A Huang,Mian %A Wang,Jian %A Nicholas,Stephen %A Maitland,Elizabeth %A Guo,Ziyue %+ Dong Fureng Institute of Economic and Social Development, Wuhan University, No 54 Dongsi Lishi Hutong, Dongcheng District, Beijing, 100010, China, 86 13864157135, wangjian993@whu.edu.cn %K health informatization %K COVID-19 %K health policy %K digital health %K health information technology %K China %D 2021 %7 17.6.2021 %9 Viewpoint %J J Med Internet Res %G English %X By applying advanced health information technology to the health care field, health informatization helps optimize health resource allocation, improve health care services, and realize universal health coverage. COVID-19 has tested the status quo of China’s health informatization, revealing challenges to the health care system. This viewpoint evaluates the development, status quo, and practice of China’s health informatization, especially during COVID-19, and makes recommendations to address the health informatization challenges. We collected, assessed, and evaluated data on the development of China’s health informatization from five perspectives—health information infrastructure, information technology (IT) applications, financial and intellectual investment, health resource allocation, and standard system—and discussed the status quo of the internet plus health care service pattern during COVID-19. The main data sources included China’s policy documents and national plans on health informatization, commercial and public welfare sources and websites, public reports, institutional reports, and academic papers. In particular, we extracted data from the 2019 National Health Informatization Survey released by the National Health Commission in China. We found that China developed its health information infrastructure and IT applications, made significant financial and intellectual informatization investments, and improved health resource allocations. Tested during COVID-19, China’s current health informatization system, especially the internet plus health care system, has played a crucial role in monitoring and controlling the pandemic and allocating medical resources. However, an uneven distribution of health resources and insufficient financial and intellectual investment continue to challenge China’s health informatization. China’s rapid development of health informatization played a crucial role during COVID-19, providing a reference point for global pandemic prevention and control. To further promote health informatization, China’s health informatization needs to strengthen top-level design, increase investment and training, upgrade the health infrastructure and IT applications, and improve internet plus health care services. %M 34061761 %R 10.2196/27345 %U https://www.jmir.org/2021/6/e27345 %U https://doi.org/10.2196/27345 %U http://www.ncbi.nlm.nih.gov/pubmed/34061761 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 6 %P e17095 %T Rating Hospital Performance in China: Review of Publicly Available Measures and Development of a Ranking System %A Dong,Shengjie %A Millar,Ross %A Shi,Chenshu %A Dong,Minye %A Xiao,Yuyin %A Shen,Jie %A Li,Guohong %+ China Hospital Development Institute, Shanghai Jiao Tong University School of Medicine, 227 South Chong Qing Road, Shanghai, 200025, China, 86 21 63846590, guohongli@sjtu.edu.cn %K hospital ranking %K performance measurement %K health care quality %K China health care reform %D 2021 %7 17.6.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: In China, significant emphasis and investment in health care reform since 2009 has brought with it increasing scrutiny of its public hospitals. Calls for greater accountability in the quality of hospital care have led to increasing attention toward performance measurement and the development of hospital ratings. Despite such interest, there has yet to be a comprehensive analysis of what performance information is publicly available to understand the performance of hospitals in China. Objective: This study aims to review the publicly available performance information about hospitals in China to assess options for ranking hospital performance. Methods: A review was undertaken to identify performance measures based on publicly available data. Following several rounds of expert consultation regarding the utility of these measures, we clustered the available options into three key areas: research and development, academic reputation, and quality and safety. Following the identification and clustering of the available performance measures, we set out to translate these into a practical performance ranking system to assess variation in hospital performance. Results: A new hospital ranking system termed the China Hospital Development Index (CHDI) is thus presented. Furthermore, we used CHDI for ranking well-known tertiary hospitals in China. Conclusions: Despite notable limitations, our assessment of available measures and the development of a new ranking system break new ground in understanding hospital performance in China. In doing so, CHDI has the potential to contribute to wider discussions and debates about assessing hospital performance across global health care systems. %M 34137724 %R 10.2196/17095 %U https://www.jmir.org/2021/6/e17095 %U https://doi.org/10.2196/17095 %U http://www.ncbi.nlm.nih.gov/pubmed/34137724 %0 Journal Article %@ 2292-9495 %I JMIR Publications %V 8 %N 2 %P e26012 %T Usability Evaluation of a Tablet-Based Intervention to Prevent Intradialytic Hypotension in Dialysis Patients During In-Clinic Dialysis: Mixed Methods Study %A Willis,Matthew %A Brand Hein,Leah %A Hu,Zhaoxian %A Saran,Rajiv %A Argentina,Marissa %A Bragg-Gresham,Jennifer %A Krein,Sarah L %A Gillespie,Brenda %A Zheng,Kai %A Veinot,Tiffany C %+ School of Information, University of Michigan, 105 South State Street, Ann Arbor, MI, 48109, United States, 1 7347632285, mandwill@umich.edu %K user interaction %K dialysis %K usability %K informatics intervention %D 2021 %7 14.6.2021 %9 Original Paper %J JMIR Hum Factors %G English %X Background: Patients on hemodialysis receive dialysis thrice weekly for about 4 hours per session. Intradialytic hypotension (IDH)—low blood pressure during hemodialysis—is a serious but common complication of hemodialysis. Although patients on dialysis already participate in their care, activating patients toward IDH prevention may reduce their risk of IDH. Interactive, technology-based interventions hold promise as a platform for patient activation. However, little is known about the usability challenges that patients undergoing hemodialysis may face when using tablet-based informatics interventions, especially while dialyzing. Objective: This study aims to test the usability of a patient-facing, tablet-based intervention that includes theory-informed educational modules and motivational interviewing–based mentoring from patient peers via videoconferencing. Methods: We conducted a cross-sectional, mixed methods usability evaluation of the tablet-based intervention by using think-aloud methods, field notes, and structured observations. These qualitative data were evaluated by trained researchers using a structured data collection instrument to capture objective observational data. We calculated descriptive statistics for the quantitative data and conducted inductive content analysis using the qualitative data. Results: Findings from 14 patients cluster around general constraints such as the use of one arm, dexterity issues, impaired vision, and lack of experience with touch screen devices. Our task-by-task usability results showed that specific sections with the greatest difficulty for users were logging into the intervention (difficulty score: 2.08), interacting with the quizzes (difficulty score: 1.92), goal setting (difficulty score: 2.28), and entering and exiting videoconference rooms (difficulty score: 2.07) that are used to engage with peers during motivational interviewing sessions. Conclusions: In this paper, we present implications for designing informatics interventions for patients on dialysis and detail resulting changes to be implemented in the next version of this intervention. We frame these implications first through the context of the role the patients’ physical body plays when interacting with the intervention and then through the digital considerations for software and interface interaction. %M 34121664 %R 10.2196/26012 %U https://humanfactors.jmir.org/2021/2/e26012 %U https://doi.org/10.2196/26012 %U http://www.ncbi.nlm.nih.gov/pubmed/34121664 %0 Journal Article %@ 2563-6316 %I JMIR Publications %V 2 %N 2 %P e25560 %T Machine Learning for Risk Group Identification and User Data Collection in a Herpes Simplex Virus Patient Registry: Algorithm Development and Validation Study %A Surodina,Svitlana %A Lam,Ching %A Grbich,Svetislav %A Milne-Ives,Madison %A van Velthoven,Michelle %A Meinert,Edward %+ Centre for Health Technology, University of Plymouth, 6 Kirkby Place, Room 2, Plymouth, PL4 6DN, United Kingdom, 44 1752600600, edward.meinert@plymouth.ac.uk %K data collection %K herpes simplex virus %K registries %K machine learning %K risk assessment %K artificial intelligence %K medical information system %K user-centered design %K predictor %K risk %D 2021 %7 11.6.2021 %9 Original Paper %J JMIRx Med %G English %X Background: Researching people with herpes simplex virus (HSV) is challenging because of poor data quality, low user engagement, and concerns around stigma and anonymity. Objective: This project aimed to improve data collection for a real-world HSV registry by identifying predictors of HSV infection and selecting a limited number of relevant questions to ask new registry users to determine their level of HSV infection risk. Methods: The US National Health and Nutrition Examination Survey (NHANES, 2015-2016) database includes the confirmed HSV type 1 and type 2 (HSV-1 and HSV-2, respectively) status of American participants (14-49 years) and a wealth of demographic and health-related data. The questionnaires and data sets from this survey were used to form two data sets: one for HSV-1 and one for HSV-2. These data sets were used to train and test a model that used a random forest algorithm (devised using Python) to minimize the number of anonymous lifestyle-based questions needed to identify risk groups for HSV. Results: The model selected a reduced number of questions from the NHANES questionnaire that predicted HSV infection risk with high accuracy scores of 0.91 and 0.96 and high recall scores of 0.88 and 0.98 for the HSV-1 and HSV-2 data sets, respectively. The number of questions was reduced from 150 to an average of 40, depending on age and gender. The model, therefore, provided high predictability of risk of infection with minimal required input. Conclusions: This machine learning algorithm can be used in a real-world evidence registry to collect relevant lifestyle data and identify individuals’ levels of risk of HSV infection. A limitation is the absence of real user data and integration with electronic medical records, which would enable model learning and improvement. Future work will explore model adjustments, anonymization options, explicit permissions, and a standardized data schema that meet the General Data Protection Regulation, Health Insurance Portability and Accountability Act, and third-party interface connectivity requirements. %M 37725536 %R 10.2196/25560 %U https://xmed.jmir.org/2021/2/e25560 %U https://doi.org/10.2196/25560 %U http://www.ncbi.nlm.nih.gov/pubmed/37725536 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 8 %N 6 %P e26681 %T Design and Implementation of an Informatics Infrastructure for Standardized Data Acquisition, Transfer, Storage, and Export in Psychiatric Clinical Routine: Feasibility Study %A Blitz,Rogério %A Storck,Michael %A Baune,Bernhard T %A Dugas,Martin %A Opel,Nils %+ Department of Psychiatry, University of Münster, Albert-Schweitzer-Str. 11, Münster, 48149, Germany, 49 2518358160, n_opel01@uni-muenster.de %K medical informatics %K digital mental health %K digital data collection %K psychiatry %K single-source metadata architecture transformation %K mental health %K design %K implementation %K feasibility %K informatics %K infrastructure %K data %D 2021 %7 9.6.2021 %9 Original Paper %J JMIR Ment Health %G English %X Background: Empirically driven personalized diagnostic applications and treatment stratification is widely perceived as a major hallmark in psychiatry. However, databased personalized decision making requires standardized data acquisition and data access, which are currently absent in psychiatric clinical routine. Objective: Here, we describe the informatics infrastructure implemented at the psychiatric Münster University Hospital, which allows standardized acquisition, transfer, storage, and export of clinical data for future real-time predictive modelling in psychiatric routine. Methods: We designed and implemented a technical architecture that includes an extension of the electronic health record (EHR) via scalable standardized data collection and data transfer between EHRs and research databases, thus allowing the pooling of EHRs and research data in a unified database and technical solutions for the visual presentation of collected data and analyses results in the EHR. The Single-source Metadata ARchitecture Transformation (SMA:T) was used as the software architecture. SMA:T is an extension of the EHR system and uses module-driven engineering to generate standardized applications and interfaces. The operational data model was used as the standard. Standardized data were entered on iPads via the Mobile Patient Survey (MoPat) and the web application Mopat@home, and the standardized transmission, processing, display, and export of data were realized via SMA:T. Results: The technical feasibility of the informatics infrastructure was demonstrated in the course of this study. We created 19 standardized documentation forms with 241 items. For 317 patients, 6451 instances were automatically transferred to the EHR system without errors. Moreover, 96,323 instances were automatically transferred from the EHR system to the research database for further analyses. Conclusions: In this study, we present the successful implementation of the informatics infrastructure enabling standardized data acquisition and data access for future real-time predictive modelling in clinical routine in psychiatry. The technical solution presented here might guide similar initiatives at other sites and thus help to pave the way toward future application of predictive models in psychiatric clinical routine. %M 34106072 %R 10.2196/26681 %U https://mental.jmir.org/2021/6/e26681 %U https://doi.org/10.2196/26681 %U http://www.ncbi.nlm.nih.gov/pubmed/34106072 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 6 %P e26230 %T Smart Decentralization of Personal Health Records with Physician Apps and Helper Agents on Blockchain: Platform Design and Implementation Study %A Kim,Hyeong-Joon %A Kim,Hye Hyeon %A Ku,Hosuk %A Yoo,Kyung Don %A Lee,Suehyun %A Park,Ji In %A Kim,Hyo Jin %A Kim,Kyeongmin %A Chung,Moon Kyung %A Lee,Kye Hwa %A Kim,Ju Han %+ Division of Biomedical Informatics, College of Medicine, Seoul National University, 103, Daehak-ro, Jongno-gu, Seoul, 03080, Republic of Korea, 82 27408320, juhan@snu.ac.kr %K personal health records %K blockchain %K mobile health %K semantic interoperatbility %K decentralized system %K patient-centered system %D 2021 %7 7.6.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: The Health Avatar Platform provides a mobile health environment with interconnected patient Avatars, physician apps, and intelligent agents (termed IoA3) for data privacy and participatory medicine; however, its fully decentralized architecture has come at the expense of decentralized data management and data provenance. Objective: The introduction of blockchain and smart contract technologies to the legacy Health Avatar Platform with a clinical metadata registry remarkably strengthens decentralized health data integrity and immutable transaction traceability at the corresponding data-element level in a privacy-preserving fashion. A crypto-economy ecosystem was built to facilitate secure and traceable exchanges of sensitive health data. Methods: The Health Avatar Platform decentralizes patient data in appropriate locations (ie, on patients’ smartphones and on physicians’ smart devices). We implemented an Ethereum-based hash chain for all transactions and smart contract–based processes to guarantee decentralized data integrity and to generate block data containing transaction metadata on-chain. Parameters of all types of data communications were enumerated and incorporated into 3 smart contracts, in this case, a health data transaction manager, a transaction status manager, and an application programming interface transaction manager. The actual decentralized health data are managed in an off-chain manner on appropriate smart devices and authenticated by hashed metadata on-chain. Results: Metadata of each data transaction are captured in a Health Avatar Platform blockchain node by the smart contracts. We provide workflow diagrams each of the 3 use cases of data push (from a physician app or an intelligent agents to a patient Avatar), data pull (request to a patient Avatar by other entities), and data backup transactions. Each transaction can be finely managed at the corresponding data-element level rather than at the resource or document levels. Hash-chained metadata support data element–level verification of data integrity in subsequent transactions. Smart contracts can incentivize transactions for data sharing and intelligent digital health care services. Conclusions: Health Avatar Platform and interconnected patient Avatars, physician apps, and intelligent agents provide a decentralized blockchain ecosystem for health data that enables trusted and finely tuned data sharing and facilitates health value-creating transactions with smart contracts. %M 34096877 %R 10.2196/26230 %U https://medinform.jmir.org/2021/6/e26230 %U https://doi.org/10.2196/26230 %U http://www.ncbi.nlm.nih.gov/pubmed/34096877 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 8 %N 5 %P e26746 %T Intention to Use Behavioral Health Data From a Health Information Exchange: Mixed Methods Study %A Cochran,Randyl A %A Feldman,Sue S %A Ivankova,Nataliya V %A Hall,Allyson G %A Opoku-Agyeman,William %+ Department of Health Sciences, College of Health Professions, Towson University, 8000 York Road, Linthicum Hall 121C, Towson, MD, 21252, United States, 1 410 704 2345, rcochran@towson.edu %K behavioral health %K integrated care %K health information exchange %K behavioral intention %K patient care %K mixed methods research %D 2021 %7 27.5.2021 %9 Original Paper %J JMIR Ment Health %G English %X Background: Patients with co-occurring behavioral health and chronic medical conditions frequently overuse inpatient hospital services. This pattern of overuse contributes to inefficient health care spending. These patients require coordinated care to achieve optimal health outcomes. However, the poor exchange of health-related information between various clinicians renders the delivery of coordinated care challenging. Health information exchanges (HIEs) facilitate health-related information sharing and have been shown to be effective in chronic disease management; however, their effectiveness in the delivery of integrated care is less clear. It is prudent to consider new approaches to sharing both general medical and behavioral health information. Objective: This study aims to identify and describe factors influencing the intention to use behavioral health information that is shared through HIEs. Methods: We used a mixed methods design consisting of two sequential phases. A validated survey instrument was emailed to clinical and nonclinical staff in Alabama and Oklahoma. The survey captured information about the impact of predictors on the intention to use behavioral health data in clinical decision making. Follow-up interviews were conducted with a subsample of participants to elaborate on the survey results. Partial least squares structural equation modeling was used to analyze survey data. Thematic analysis was used to identify themes from the interviews. Results: A total of 62 participants completed the survey. In total, 63% (n=39) of the participants were clinicians. Performance expectancy (β=.382; P=.01) and trust (β=.539; P<.001) predicted intention to use behavioral health information shared via HIEs. The interviewees (n=5) expressed that behavioral health information could be useful in clinical decision making. However, privacy and confidentiality concerns discourage sharing this information, which is generally missing from patient records altogether. The interviewees also stated that training for HIE use was not mandatory; the training that was provided did not focus specifically on the exchange of behavioral health information. Conclusions: Despite barriers, individuals are willing to use behavioral health information from HIEs if they believe that it will enhance job performance and if the information being transmitted is trustworthy. The findings contribute to our understanding of the role HIEs can play in delivering integrated care, particularly to vulnerable patients. %M 34042606 %R 10.2196/26746 %U https://mental.jmir.org/2021/5/e26746 %U https://doi.org/10.2196/26746 %U http://www.ncbi.nlm.nih.gov/pubmed/34042606 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 5 %P e25656 %T Digitization of Measurement-Based Care Pathways in Mental Health Through REDCap and Electronic Health Record Integration: Development and Usability Study %A Hawley,Steve %A Yu,Joanna %A Bogetic,Nikola %A Potapova,Natalia %A Wakefield,Chris %A Thompson,Mike %A Kloiber,Stefan %A Hill,Sean %A Jankowicz,Damian %A Rotenberg,David %+ Krembil Centre for Neuroinformatics, Centre for Addiction and Mental Health, 250 College St., Toronto, ON, M5T 1L8, Canada, 1 416 535 8501, steve.hawley@camh.ca %K REDCap %K electronic health record %K systems integration %K measurement-based care %K hospital information systems %D 2021 %7 20.5.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: The delivery of standardized self-report assessments is essential for measurement-based care in mental health. Paper-based methods of measurement-based care data collection may result in transcription errors, missing data, and other data quality issues when entered into patient electronic health records (EHRs). Objective: This study aims to help address these issues by using a dedicated instance of REDCap (Research Electronic Data Capture; Vanderbilt University)—a free, widely used electronic data capture platform—that was established to enable the deployment of digitized self-assessments in clinical care pathways to inform clinical decision making. Methods: REDCap was integrated with the primary clinical information system to facilitate the real-time transfer of discrete data and PDF reports from REDCap into the EHR. Both technical and administrative components were required for complete implementation. A technology acceptance survey was also administered to capture physicians’ and clinicians’ attitudes toward the new system. Results: The integration of REDCap with the EHR transitioned clinical workflows from paper-based methods of data collection to electronic data collection. This resulted in significant time savings, improved data quality, and valuable real-time information delivery. The digitization of self-report assessments at each appointment contributed to the clinic-wide implementation of the major depressive disorder integrated care pathway. This digital transformation facilitated a 4-fold increase in the physician adoption of this integrated care pathway workflow and a 3-fold increase in patient enrollment, resulting in an overall significant increase in major depressive disorder integrated care pathway capacity. Physicians’ and clinicians’ attitudes were overall positive, with almost all respondents agreeing that the system was useful to their work. Conclusions: REDCap provided an intuitive patient interface for collecting self-report measures and accessing results in real time to inform clinical decisions and an extensible backend for system integration. The approach scaled effectively and expanded to high-impact clinics throughout the hospital, allowing for the broad deployment of complex workflows and standardized assessments, which led to the accumulation of harmonized data across clinics and care pathways. REDCap is a flexible tool that can be effectively leveraged to facilitate the automatic transfer of self-report data to the EHR; however, thoughtful governance is required to complement the technical implementation to ensure that data standardization, data quality, patient safety, and privacy are maintained. %M 34014169 %R 10.2196/25656 %U https://www.jmir.org/2021/5/e25656 %U https://doi.org/10.2196/25656 %U http://www.ncbi.nlm.nih.gov/pubmed/34014169 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 5 %P e25869 %T Federated Learning for Thyroid Ultrasound Image Analysis to Protect Personal Information: Validation Study in a Real Health Care Environment %A Lee,Haeyun %A Chai,Young Jun %A Joo,Hyunjin %A Lee,Kyungsu %A Hwang,Jae Youn %A Kim,Seok-Mo %A Kim,Kwangsoon %A Nam,Inn-Chul %A Choi,June Young %A Yu,Hyeong Won %A Lee,Myung-Chul %A Masuoka,Hiroo %A Miyauchi,Akira %A Lee,Kyu Eun %A Kim,Sungwan %A Kong,Hyoun-Joong %+ Transdisciplinary Department of Medicine and Advanced Technology, Seoul National University Hospital, Daehak-ro 101, Jongno-gu, Seoul, Republic of Korea, 82 2 2072 4492, gongcop@gmail.com %K deep learning %K federated learning %K thyroid nodules %K ultrasound image %D 2021 %7 18.5.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Federated learning is a decentralized approach to machine learning; it is a training strategy that overcomes medical data privacy regulations and generalizes deep learning algorithms. Federated learning mitigates many systemic privacy risks by sharing only the model and parameters for training, without the need to export existing medical data sets. In this study, we performed ultrasound image analysis using federated learning to predict whether thyroid nodules were benign or malignant. Objective: The goal of this study was to evaluate whether the performance of federated learning was comparable with that of conventional deep learning. Methods: A total of 8457 (5375 malignant, 3082 benign) ultrasound images were collected from 6 institutions and used for federated learning and conventional deep learning. Five deep learning networks (VGG19, ResNet50, ResNext50, SE-ResNet50, and SE-ResNext50) were used. Using stratified random sampling, we selected 20% (1075 malignant, 616 benign) of the total images for internal validation. For external validation, we used 100 ultrasound images (50 malignant, 50 benign) from another institution. Results: For internal validation, the area under the receiver operating characteristic (AUROC) curve for federated learning was between 78.88% and 87.56%, and the AUROC for conventional deep learning was between 82.61% and 91.57%. For external validation, the AUROC for federated learning was between 75.20% and 86.72%, and the AUROC curve for conventional deep learning was between 73.04% and 91.04%. Conclusions: We demonstrated that the performance of federated learning using decentralized data was comparable to that of conventional deep learning using pooled data. Federated learning might be potentially useful for analyzing medical images while protecting patients’ personal information. %M 33858817 %R 10.2196/25869 %U https://medinform.jmir.org/2021/5/e25869 %U https://doi.org/10.2196/25869 %U http://www.ncbi.nlm.nih.gov/pubmed/33858817 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 5 %P e24803 %T An Attention Model With Transfer Embeddings to Classify Pneumonia-Related Bilingual Imaging Reports: Algorithm Development and Validation %A Park,Hyung %A Song,Min %A Lee,Eun Byul %A Seo,Bo Kyung %A Choi,Chang Min %+ Department of Pulmonary and Critical Care Medicine, Asan Medical Center, Olympic-ro 43-gil, Seoul, 05505, Republic of Korea, 82 2 3010 5902, ccm9607@gmail.com %K deep learning %K natural language process %K attention %K clinical data %K pneumonia %K classification %K medical imaging %K electronic health record %K machine learning %K model %D 2021 %7 17.5.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: In the analysis of electronic health records, proper labeling of outcomes is mandatory. To obtain proper information from radiologic reports, several studies were conducted to classify radiologic reports using deep learning. However, the classification of pneumonia in bilingual radiologic reports has not been conducted previously. Objective: The aim of this research was to classify radiologic reports into pneumonia or no pneumonia using a deep learning method. Methods: A data set of radiology reports for chest computed tomography and chest x-rays of surgical patients from January 2008 to January 2018 in the Asan Medical Center in Korea was retrospectively analyzed. The classification performance of our long short-term memory (LSTM)–Attention model was compared with various deep learning and machine learning methods. The area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve, sensitivity, specificity, accuracy, and F1 score for the models were compared. Results: A total of 5450 radiologic reports were included that contained at least one pneumonia-related word. In the test set (n=1090), our proposed model showed 91.01% (992/1090) accuracy (AUROCs for negative, positive, and obscure were 0.98, 0.97, and 0.90, respectively). The top 3 performances of the models were based on FastText or LSTM. The convolutional neural network–based model showed a lower accuracy 73.03% (796/1090) than the other 2 algorithms. The classification of negative results had an F1 score of 0.96, whereas the classification of positive and uncertain results showed a lower performance (positive F1 score 0.83; uncertain F1 score 0.62). In the extra-validation set, our model showed 80.0% (642/803) accuracy (AUROCs for negative, positive, and obscure were 0.92, 0.96, and 0.84, respectively). Conclusions: Our method showed excellent performance in classifying pneumonia in bilingual radiologic reports. The method could enrich the research on pneumonia by obtaining exact outcomes from electronic health data. %M 33820755 %R 10.2196/24803 %U https://medinform.jmir.org/2021/5/e24803 %U https://doi.org/10.2196/24803 %U http://www.ncbi.nlm.nih.gov/pubmed/33820755 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 5 %P e23479 %T Information Quality Frameworks for Digital Health Technologies: Systematic Review %A Fadahunsi,Kayode Philip %A O'Connor,Siobhan %A Akinlua,James Tosin %A Wark,Petra A %A Gallagher,Joseph %A Carroll,Christopher %A Car,Josip %A Majeed,Azeem %A O'Donoghue,John %+ Department of Public Health and Primary Care, Imperial College London, The Reynolds Building, St. Dunstan’s Road, London, W6 8RP, United Kingdom, 44 07477854209, K.fadahunsi14@imperial.ac.uk %K digital health %K patient safety %K information quality %D 2021 %7 17.5.2021 %9 Review %J J Med Internet Res %G English %X Background: Digital health technologies (DHTs) generate a large volume of information used in health care for administrative, educational, research, and clinical purposes. The clinical use of digital information for diagnostic, therapeutic, and prognostic purposes has multiple patient safety problems, some of which result from poor information quality (IQ). Objective: This systematic review aims to synthesize an IQ framework that could be used to evaluate the extent to which digital health information is fit for clinical purposes. Methods: The review was conducted according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) guidelines. We searched Embase, MEDLINE, PubMed, CINAHL, Maternity and Infant Care, PsycINFO, Global Health, ProQuest Dissertations and Theses Global, Scopus, and HMIC (the Health Management Information Consortium) from inception until October 2019. Multidimensional IQ frameworks for assessing DHTs used in the clinical context by health care professionals were included. A thematic synthesis approach was used to synthesize the Clinical Information Quality (CLIQ) framework for digital health. Results: We identified 10 existing IQ frameworks from which we developed the CLIQ framework for digital health with 13 unique dimensions: accessibility, completeness, portability, security, timeliness, accuracy, interpretability, plausibility, provenance, relevance, conformance, consistency, and maintainability, which were categorized into 3 meaningful categories: availability, informativeness, and usability. Conclusions: This systematic review highlights the importance of the IQ of DHTs and its relevance to patient safety. The CLIQ framework for digital health will be useful in evaluating and conceptualizing IQ issues associated with digital health, thus forestalling potential patient safety problems. Trial Registration: PROSPERO International Prospective Register of Systematic Reviews CRD42018097142; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=97142 International Registered Report Identifier (IRRID): RR2-10.1136/bmjopen-2018-024722 %M 33835034 %R 10.2196/23479 %U https://www.jmir.org/2021/5/e23479 %U https://doi.org/10.2196/23479 %U http://www.ncbi.nlm.nih.gov/pubmed/33835034 %0 Journal Article %@ 2368-7959 %I JMIR Publications %V 8 %N 5 %P e20865 %T Knowledge-Infused Abstractive Summarization of Clinical Diagnostic Interviews: Framework Development Study %A Manas,Gaur %A Aribandi,Vamsi %A Kursuncu,Ugur %A Alambo,Amanuel %A Shalin,Valerie L %A Thirunarayan,Krishnaprasad %A Beich,Jonathan %A Narasimhan,Meera %A Sheth,Amit %+ Artificial Intelligence Institute, University of South Carolina, Room 513, 1112 Greene St, Columbia, SC, 29208, United States, 1 5593879476, mgaur@email.sc.edu %K knowledge-infusion %K abstractive summarization %K distress clinical diagnostic interviews %K Patient Health Questionnaire-9 %K healthcare informatics %K interpretable evaluations %D 2021 %7 10.5.2021 %9 Original Paper %J JMIR Ment Health %G English %X Background: In clinical diagnostic interviews, mental health professionals (MHPs) implement a care practice that involves asking open questions (eg, “What do you want from your life?” “What have you tried before to bring change in your life?”) while listening empathetically to patients. During these interviews, MHPs attempted to build a trusting human-centered relationship while collecting data necessary for professional medical and psychiatric care. Often, because of the social stigma of mental health disorders, patient discomfort in discussing their presenting problem may add additional complexities and nuances to the language they use, that is, hidden signals among noisy content. Therefore, a focused, well-formed, and elaborative summary of clinical interviews is critical to MHPs in making informed decisions by enabling a more profound exploration of a patient’s behavior, especially when it endangers life. Objective: The aim of this study is to propose an unsupervised, knowledge-infused abstractive summarization (KiAS) approach that generates summaries to enable MHPs to perform a well-informed follow-up with patients to improve the existing summarization methods built on frequency heuristics by creating more informative summaries. Methods: Our approach incorporated domain knowledge from the Patient Health Questionnaire-9 lexicon into an integer linear programming framework that optimizes linguistic quality and informativeness. We used 3 baseline approaches: extractive summarization using the SumBasic algorithm, abstractive summarization using integer linear programming without the infusion of knowledge, and abstraction over extractive summarization to evaluate the performance of KiAS. The capability of KiAS on the Distress Analysis Interview Corpus-Wizard of Oz data set was demonstrated through interpretable qualitative and quantitative evaluations. Results: KiAS generates summaries (7 sentences on average) that capture informative questions and responses exchanged during long (58 sentences on average), ambiguous, and sparse clinical diagnostic interviews. The summaries generated using KiAS improved upon the 3 baselines by 23.3%, 4.4%, 2.5%, and 2.2% for thematic overlap, Flesch Reading Ease, contextual similarity, and Jensen Shannon divergence, respectively. On the Recall-Oriented Understudy for Gisting Evaluation-2 and Recall-Oriented Understudy for Gisting Evaluation-L metrics, KiAS showed an improvement of 61% and 49%, respectively. We validated the quality of the generated summaries through visual inspection and substantial interrater agreement from MHPs. Conclusions: Our collaborator MHPs observed the potential utility and significant impact of KiAS in leveraging valuable but voluminous communications that take place outside of normally scheduled clinical appointments. This study shows promise in generating semantically relevant summaries that will help MHPs make informed decisions about patient status. %M 33970116 %R 10.2196/20865 %U https://mental.jmir.org/2021/5/e20865 %U https://doi.org/10.2196/20865 %U http://www.ncbi.nlm.nih.gov/pubmed/33970116 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 4 %P e26877 %T Utility, Value, and Benefits of Contemporary Personal Health Records: Integrative Review and Conceptual Synthesis %A Ruhi,Umar %A Chugh,Ritesh %+ Telfer School of Management, University of Ottawa, 55 Laurier Ave East, Ottawa, ON, K1N 6N5, Canada, 1 6135625800 ext 1990, umar.ruhi@uottawa.ca %K electronic personal health records %K PHR %K functionality synopsis %K value analysis %K consumer health informatics %D 2021 %7 29.4.2021 %9 Review %J J Med Internet Res %G English %X Background: Contemporary personal health record (PHR) technologies offer a useful platform for individuals to maintain a lifelong record of personally reported and clinically sourced data from various points of medical care. Objective: This paper presents an integrative review and synthesis of the extant literature on PHRs. This review draws upon multiple lenses of analysis and deliberates value perspectives of PHRs at the product, consumer, and industry levels. Methods: Academic databases were searched using multiple keywords related to PHRs for the years 2001-2020. Three research questions were formulated and used as selection criteria in our review of the extant literature relevant to our study. Results: We offer a high-level functional utility model of PHR features and functions. We also conceptualize a consumer value framework of PHRs, highlighting the applications of these technologies across various health care delivery activities. Finally, we provide a summary of the benefits of PHRs for various health care constituents, including consumers, providers, payors, and public health agencies. Conclusions: PHR products offer a myriad of content-, connectivity-, and collaboration-based features and functions for their users. Although consumers benefit from the tools provided by PHR technologies, their overall value extends across the constituents of the health care delivery chain. Despite advances in technology, our literature review identifies a shortfall in the research addressing consumer value enabled by PHR tools. In addition to scholars and researchers, our literature review and proposed framework may be especially helpful for value analysis committees in the health care sector that are commissioned for the appraisal of innovative health information technologies such as PHRs. %M 33866308 %R 10.2196/26877 %U https://www.jmir.org/2021/4/e26877 %U https://doi.org/10.2196/26877 %U http://www.ncbi.nlm.nih.gov/pubmed/33866308 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 4 %P e24014 %T Physician Stress During Electronic Health Record Inbox Work: In Situ Measurement With Wearable Sensors %A Akbar,Fatema %A Mark,Gloria %A Prausnitz,Stephanie %A Warton,E Margaret %A East,Jeffrey A %A Moeller,Mark F %A Reed,Mary E %A Lieu,Tracy A %+ Division of Research, Kaiser Permanente Northern California, 2000 Broadway, Oakland, CA, 94611, United States, 1 510 891 3407, tracy.lieu@kp.org %K electronic health records %K stress %K wearables %K HRV %K inbox %K EHR alerts %K after-hours work %K electronic mail %K physician well-being %K Inbasket %D 2021 %7 28.4.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Increased work through electronic health record (EHR) messaging is frequently cited as a factor of physician burnout. However, studies to date have relied on anecdotal or self-reported measures, which limit the ability to match EHR use patterns with continuous stress patterns throughout the day. Objective: The aim of this study is to collect EHR use and physiologic stress data through unobtrusive means that provide objective and continuous measures, cluster distinct patterns of EHR inbox work, identify physicians’ daily physiologic stress patterns, and evaluate the association between EHR inbox work patterns and physician physiologic stress. Methods: Physicians were recruited from 5 medical centers. Participants (N=47) were given wrist-worn devices (Garmin Vivosmart 3) with heart rate sensors to wear for 7 days. The devices measured physiological stress throughout the day based on heart rate variability (HRV). Perceived stress was also measured with self-reports through experience sampling and a one-time survey. From the EHR system logs, the time attributed to different activities was quantified. By using a clustering algorithm, distinct inbox work patterns were identified and their associated stress measures were compared. The effects of EHR use on physician stress were examined using a generalized linear mixed effects model. Results: Physicians spent an average of 1.08 hours doing EHR inbox work out of an average total EHR time of 3.5 hours. Patient messages accounted for most of the inbox work time (mean 37%, SD 11%). A total of 3 patterns of inbox work emerged: inbox work mostly outside work hours, inbox work mostly during work hours, and inbox work extending after hours that were mostly contiguous to work hours. Across these 3 groups, physiologic stress patterns showed 3 periods in which stress increased: in the first hour of work, early in the afternoon, and in the evening. Physicians in group 1 had the longest average stress duration during work hours (80 out of 243 min of valid HRV data; P=.02), as measured by physiological sensors. Inbox work duration, the rate of EHR window switching (moving from one screen to another), the proportion of inbox work done outside of work hours, inbox work batching, and the day of the week were each independently associated with daily stress duration (marginal R2=15%). Individual-level random effects were significant and explained most of the variation in stress (conditional R2=98%). Conclusions: This study is among the first to demonstrate associations between electronic inbox work and physiological stress. We identified 3 potentially modifiable factors associated with stress: EHR window switching, inbox work duration, and inbox work outside work hours. Organizations seeking to reduce physician stress may consider system-based changes to reduce EHR window switching or inbox work duration or the incorporation of inbox management time into work hours. %M 33908888 %R 10.2196/24014 %U https://medinform.jmir.org/2021/4/e24014 %U https://doi.org/10.2196/24014 %U http://www.ncbi.nlm.nih.gov/pubmed/33908888 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 4 %P e25066 %T Predicting Intensive Care Transfers and Other Unforeseen Events: Analytic Model Validation Study and Comparison to Existing Methods %A Cummings,Brandon C %A Ansari,Sardar %A Motyka,Jonathan R %A Wang,Guan %A Medlin Jr,Richard P %A Kronick,Steven L %A Singh,Karandeep %A Park,Pauline K %A Napolitano,Lena M %A Dickson,Robert P %A Mathis,Michael R %A Sjoding,Michael W %A Admon,Andrew J %A Blank,Ross %A McSparron,Jakob I %A Ward,Kevin R %A Gillies,Christopher E %+ Michigan Center for Integrative Research In Critical Care, Department of Emergency Medicine, University of Michigan, 2800 N Plymouth Road, NCRC 10-A112, Ann Arbor, MI, 48109, United States, 1 (734) 647 7436, cummingb@med.umich.edu %K COVID-19 %K biomedical informatics %K critical care %K machine learning %K deterioration %K predictive analytics %K informatics %K prediction %K intensive care unit %K ICU %K mortality %D 2021 %7 21.4.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: COVID-19 has led to an unprecedented strain on health care facilities across the United States. Accurately identifying patients at an increased risk of deterioration may help hospitals manage their resources while improving the quality of patient care. Here, we present the results of an analytical model, Predicting Intensive Care Transfers and Other Unforeseen Events (PICTURE), to identify patients at high risk for imminent intensive care unit transfer, respiratory failure, or death, with the intention to improve the prediction of deterioration due to COVID-19. Objective: This study aims to validate the PICTURE model’s ability to predict unexpected deterioration in general ward and COVID-19 patients, and to compare its performance with the Epic Deterioration Index (EDI), an existing model that has recently been assessed for use in patients with COVID-19. Methods: The PICTURE model was trained and validated on a cohort of hospitalized non–COVID-19 patients using electronic health record data from 2014 to 2018. It was then applied to two holdout test sets: non–COVID-19 patients from 2019 and patients testing positive for COVID-19 in 2020. PICTURE results were aligned to EDI and NEWS scores for head-to-head comparison via area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve. We compared the models’ ability to predict an adverse event (defined as intensive care unit transfer, mechanical ventilation use, or death). Shapley values were used to provide explanations for PICTURE predictions. Results: In non–COVID-19 general ward patients, PICTURE achieved an AUROC of 0.819 (95% CI 0.805-0.834) per observation, compared to the EDI’s AUROC of 0.763 (95% CI 0.746-0.781; n=21,740; P<.001). In patients testing positive for COVID-19, PICTURE again outperformed the EDI with an AUROC of 0.849 (95% CI 0.820-0.878) compared to the EDI’s AUROC of 0.803 (95% CI 0.772-0.838; n=607; P<.001). The most important variables influencing PICTURE predictions in the COVID-19 cohort were a rapid respiratory rate, a high level of oxygen support, low oxygen saturation, and impaired mental status (Glasgow Coma Scale). Conclusions: The PICTURE model is more accurate in predicting adverse patient outcomes for both general ward patients and COVID-19 positive patients in our cohorts compared to the EDI. The ability to consistently anticipate these events may be especially valuable when considering potential incipient waves of COVID-19 infections. The generalizability of the model will require testing in other health care systems for validation. %M 33818393 %R 10.2196/25066 %U https://medinform.jmir.org/2021/4/e25066 %U https://doi.org/10.2196/25066 %U http://www.ncbi.nlm.nih.gov/pubmed/33818393 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 4 %P e23587 %T Novel Graph-Based Model With Biaffine Attention for Family History Extraction From Clinical Text: Modeling Study %A Zhan,Kecheng %A Peng,Weihua %A Xiong,Ying %A Fu,Huhao %A Chen,Qingcai %A Wang,Xiaolong %A Tang,Buzhou %+ Key Laboratory of Network Oriented Intelligent Computation, Harbin Institute of Technology, University Town, Shenzhen, 518055, China, 86 13725525983, tangbuzhou@hit.edu.cn %K family history information %K named entity recognition %K relation extraction %K deep biaffine attention %D 2021 %7 21.4.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Family history information, including information on family members, side of the family of family members, living status of family members, and observations of family members, plays an important role in disease diagnosis and treatment. Family member information extraction aims to extract family history information from semistructured/unstructured text in electronic health records (EHRs), which is a challenging task regarding named entity recognition (NER) and relation extraction (RE), where named entities refer to family members, living status, and observations, and relations refer to relations between family members and living status, and relations between family members and observations. Objective: This study aimed to introduce the system we developed for the 2019 n2c2/OHNLP track on family history extraction, which can jointly extract entities and relations about family history information from clinical text. Methods: We proposed a novel graph-based model with biaffine attention for family history extraction from clinical text. In this model, we first designed a graph to represent family history information, that is, representing NER and RE regarding family history in a unified way, and then introduced a biaffine attention mechanism to extract family history information in clinical text. Convolution neural network (CNN)-Bidirectional Long Short Term Memory network (BiLSTM) and Bidirectional Encoder Representation from Transformers (BERT) were used to encode the input sentence, and a biaffine classifier was used to extract family history information. In addition, we developed a postprocessing module to adjust the results. A system based on the proposed method was developed for the 2019 n2c2/OHNLP shared task track on family history information extraction. Results: Our system ranked first in the challenge, and the F1 scores of the best system on the NER subtask and RE subtask were 0.8745 and 0.6810, respectively. After the challenge, we further fine tuned the parameters and improved the F1 scores of the two subtasks to 0.8823 and 0.7048, respectively. Conclusions: The experimental results showed that the system based on the proposed method can extract family history information from clinical text effectively. %M 33881405 %R 10.2196/23587 %U https://medinform.jmir.org/2021/4/e23587 %U https://doi.org/10.2196/23587 %U http://www.ncbi.nlm.nih.gov/pubmed/33881405 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 4 %P e24179 %T Beyond Notes: Why It Is Time to Abandon an Outdated Documentation Paradigm %A Steinkamp,Jackson %A Kantrowitz,Jacob %A Sharma,Abhinav %A Bala,Wasif %+ Department of Family and Community Medicine, University of Toronto, 500 University Avenue, 5th Floor, Toronto, ON, M5G 1V7, Canada, 1 7789387714, abhinavarun@gmail.com %K electronic medical records %K health informatics %K information chaos %K medical documentation %K clinicians %K medical notes %K electronic medical notes %K medical team %D 2021 %7 20.4.2021 %9 Viewpoint %J J Med Internet Res %G English %X Clinicians spend a substantial part of their workday reviewing and writing electronic medical notes. Here we describe how the current, widely accepted paradigm for electronic medical notes represents a poor organizational framework for both the individual clinician and the broader medical team. As described in this viewpoint, the medical chart—including notes, labs, and imaging results—can be reconceptualized as a dynamic, fully collaborative workspace organized by topic rather than time, writer, or data type. This revised framework enables a more accurate and complete assessment of the current state of the patient and easy historical review, saving clinicians substantial time on both data input and retrieval. Collectively, this approach has the potential to improve health care delivery effectiveness and efficiency. %M 33877053 %R 10.2196/24179 %U https://www.jmir.org/2021/4/e24179 %U https://doi.org/10.2196/24179 %U http://www.ncbi.nlm.nih.gov/pubmed/33877053 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 10 %N 4 %P e24818 %T Electronic Health Record–Embedded Individualized Pain Plans for Emergency Department Treatment of Vaso-occlusive Episodes in Adults With Sickle Cell Disease: Protocol for a Preimplementation and Postimplementation Study %A Luo,Lingzi %A King,Allison A %A Carroll,Yvonne %A Baumann,Ana A %A Brambilla,Donald %A Carpenter,Christopher R %A Colla,Joseph %A Gibson,Robert W %A Gollan,Siera %A Hall,Greg %A Klesges,Lisa %A Kutlar,Abdullah %A Lyon,Matthew %A Melvin,Cathy L %A Norell,Sarah %A Mueller,Martina %A Potter,Michael B %A Richesson,Rachel %A Richardson,Lynne D %A Ryan,Gery %A Siewny,Lauren %A Treadwell,Marsha %A Zun,Leslie %A Armstrong-Brown,Janelle %A Cox,Lisa %A Tanabe,Paula %+ Washington University School of Medicine, 660 S Euclid Ave, St Louis, MO, 63108, United States, 1 3143323108, lingzi.luo@wustl.edu %K sickle cell disease %K RE-AIM %K emergency department care %K pain treatment %K digital medicine %K health innovation %K implementation science %K patient portal %K electronic health record %D 2021 %7 16.4.2021 %9 Protocol %J JMIR Res Protoc %G English %X Background: Individuals living with sickle cell disease often require aggressive treatment of pain associated with vaso-occlusive episodes in the emergency department. Frequently, pain relief is poor. The 2014 National Heart, Lung, and Blood Institute evidence-based guidelines recommended an individualized treatment and monitoring protocol to improve pain management of vaso-occlusive episodes. Objective: This study will implement an electronic health record–embedded individualized pain plan with provider and patient access in the emergency departments of 8 US academic centers to improve pain treatment for adult patients with sickle cell disease. This study will assess the overall effects of electronic health record–embedded individualized pain plans on improving patient and provider outcomes associated with pain treatment in the emergency department setting and explore barriers and facilitators to the implementation process. Methods: A preimplementation and postimplementation study is being conducted by all 8 sites that are members of the National Heart, Lung, and Blood Institute–funded Sickle Cell Disease Implementation Consortium. Adults with sickle cell disease aged 18 to 45 years who had a visit to a participating emergency department for vaso-occlusive episodes within 90 days prior to enrollment will be eligible for inclusion. Patients will be enrolled in the clinic or remotely. The target analytical sample size of this study is 160 patient participants (20 per site) who have had an emergency department visit for vaso-occlusive episode treatment at participating emergency departments during the study period. Each site is expected to enroll approximately 40 participants to reach the analytical sample size. The electronic health record–embedded individualized pain plans will be written by the patient’s sickle cell disease provider, and sites will work with the local informatics team to identify the best method to build the electronic health record–embedded individualized pain plan with patient and provider access. Each site will adopt required patient and provider implementation strategies and can choose to adopt optional strategies to improve the uptake and sustainability of the intervention. The study is informed by the Technology Acceptance Model 2 and the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework. Provider and patient baseline survey, follow-up survey within 96 hours of an emergency department vaso-occlusive episode visit, and selected qualitative interviews within 2 weeks of an emergency department visit will be performed to assess the primary outcome, patient-perceived quality of emergency department pain treatment, and additional implementation and intervention outcomes. Electronic health record data will be used to analyze individualized pain plan adherence and additional secondary outcomes, such as hospital admission and readmission rates. Results: The study is currently enrolling study participants. The active implementation period is 18 months. Conclusions: This study proposes a structured, framework-informed approach to implement electronic health record–embedded individualized pain plans with both patient and provider access in routine emergency department practice. The results of the study will inform the implementation of electronic health record–embedded individualized pain plans at a larger scale outside of Sickle Cell Disease Implementation Consortium centers. Trial Registration: ClinicalTrials.gov NCT04584528; https://clinicaltrials.gov/ct2/show/NCT04584528. International Registered Report Identifier (IRRID): DERR1-10.2196/24818 %M 33861209 %R 10.2196/24818 %U https://www.researchprotocols.org/2021/4/e24818 %U https://doi.org/10.2196/24818 %U http://www.ncbi.nlm.nih.gov/pubmed/33861209 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 4 %P e22796 %T Forecasting Future Asthma Hospital Encounters of Patients With Asthma in an Academic Health Care System: Predictive Model Development and Secondary Analysis Study %A Tong,Yao %A Messinger,Amanda I %A Wilcox,Adam B %A Mooney,Sean D %A Davidson,Giana H %A Suri,Pradeep %A Luo,Gang %+ Department of Biomedical Informatics and Medical Education, University of Washington, UW Medicine South Lake Union, 850 Republican Street, Building C, Box 358047, Seattle, WA, 98109, United States, 1 206 221 4596, gangluo@cs.wisc.edu %K asthma %K forecasting %K machine learning %K patient care management %K risk factors %D 2021 %7 16.4.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Asthma affects a large proportion of the population and leads to many hospital encounters involving both hospitalizations and emergency department visits every year. To lower the number of such encounters, many health care systems and health plans deploy predictive models to prospectively identify patients at high risk and offer them care management services for preventive care. However, the previous models do not have sufficient accuracy for serving this purpose well. Embracing the modeling strategy of examining many candidate features, we built a new machine learning model to forecast future asthma hospital encounters of patients with asthma at Intermountain Healthcare, a nonacademic health care system. This model is more accurate than the previously published models. However, it is unclear how well our modeling strategy generalizes to academic health care systems, whose patient composition differs from that of Intermountain Healthcare. Objective: This study aims to evaluate the generalizability of our modeling strategy to the University of Washington Medicine (UWM), an academic health care system. Methods: All adult patients with asthma who visited UWM facilities between 2011 and 2018 served as the patient cohort. We considered 234 candidate features. Through a secondary analysis of 82,888 UWM data instances from 2011 to 2018, we built a machine learning model to forecast asthma hospital encounters of patients with asthma in the subsequent 12 months. Results: Our UWM model yielded an area under the receiver operating characteristic curve (AUC) of 0.902. When placing the cutoff point for making binary classification at the top 10% (1464/14,644) of patients with asthma with the largest forecasted risk, our UWM model yielded an accuracy of 90.6% (13,268/14,644), a sensitivity of 70.2% (153/218), and a specificity of 90.91% (13,115/14,426). Conclusions: Our modeling strategy showed excellent generalizability to the UWM, leading to a model with an AUC that is higher than all of the AUCs previously reported in the literature for forecasting asthma hospital encounters. After further optimization, our model could be used to facilitate the efficient and effective allocation of asthma care management resources to improve outcomes. International Registered Report Identifier (IRRID): RR2-10.2196/resprot.5039 %M 33861206 %R 10.2196/22796 %U https://www.jmir.org/2021/4/e22796 %U https://doi.org/10.2196/22796 %U http://www.ncbi.nlm.nih.gov/pubmed/33861206 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 4 %P e24120 %T Real-Time Clinical Decision Support Based on Recurrent Neural Networks for In-Hospital Acute Kidney Injury: External Validation and Model Interpretation %A Kim,Kipyo %A Yang,Hyeonsik %A Yi,Jinyeong %A Son,Hyung-Eun %A Ryu,Ji-Young %A Kim,Yong Chul %A Jeong,Jong Cheol %A Chin,Ho Jun %A Na,Ki Young %A Chae,Dong-Wan %A Han,Seung Seok %A Kim,Sejoong %+ Department of Internal Medicine, Seoul National University Bundang Hospital, 82 Gumi-ro 173-beon-gil Bundang-gu, Seongnam, 13620, Republic of Korea, 82 31 787 7051, sejoong@snubh.org %K acute kidney injury %K recurrent neural network %K prediction model %K external validation %K internal validation %K kidney %K neural networks %D 2021 %7 16.4.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Acute kidney injury (AKI) is commonly encountered in clinical practice and is associated with poor patient outcomes and increased health care costs. Despite it posing significant challenges for clinicians, effective measures for AKI prediction and prevention are lacking. Previously published AKI prediction models mostly have a simple design without external validation. Furthermore, little is known about the process of linking model output and clinical decisions due to the black-box nature of neural network models. Objective: We aimed to present an externally validated recurrent neural network (RNN)–based continuous prediction model for in-hospital AKI and show applicable model interpretations in relation to clinical decision support. Methods: Study populations were all patients aged 18 years or older who were hospitalized for more than 48 hours between 2013 and 2017 in 2 tertiary hospitals in Korea (Seoul National University Bundang Hospital and Seoul National University Hospital). All demographic data, laboratory values, vital signs, and clinical conditions of patients were obtained from electronic health records of each hospital. We developed 2-stage hierarchical prediction models (model 1 and model 2) using RNN algorithms. The outcome variable for model 1 was the occurrence of AKI within 7 days from the present. Model 2 predicted the future trajectory of creatinine values up to 72 hours. The performance of each developed model was evaluated using the internal and external validation data sets. For the explainability of our models, different model-agnostic interpretation methods were used, including Shapley Additive Explanations, partial dependence plots, individual conditional expectation, and accumulated local effects plots. Results: We included 69,081 patients in the training, 7675 in the internal validation, and 72,352 in the external validation cohorts for model development after excluding cases with missing data and those with an estimated glomerular filtration rate less than 15 mL/min/1.73 m2 or end-stage kidney disease. Model 1 predicted any AKI development with an area under the receiver operating characteristic curve (AUC) of 0.88 (internal validation) and 0.84 (external validation), and stage 2 or higher AKI development with an AUC of 0.93 (internal validation) and 0.90 (external validation). Model 2 predicted the future creatinine values within 3 days with mean-squared errors of 0.04-0.09 for patients with higher risks of AKI and 0.03-0.08 for those with lower risks. Based on the developed models, we showed AKI probability according to feature values in total patients and each individual with partial dependence, accumulated local effects, and individual conditional expectation plots. We also estimated the effects of feature modifications such as nephrotoxic drug discontinuation on future creatinine levels. Conclusions: We developed and externally validated a continuous AKI prediction model using RNN algorithms. Our model could provide real-time assessment of future AKI occurrences and individualized risk factors for AKI in general inpatient cohorts; thus, we suggest approaches to support clinical decisions based on prediction models for in-hospital AKI. %M 33861200 %R 10.2196/24120 %U https://www.jmir.org/2021/4/e24120 %U https://doi.org/10.2196/24120 %U http://www.ncbi.nlm.nih.gov/pubmed/33861200 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 4 %P e25657 %T Usability of Electronic Health Record–Generated Discharge Summaries: Heuristic Evaluation %A Tremoulet,Patrice D %A Shah,Priyanka D %A Acosta,Alisha A %A Grant,Christian W %A Kurtz,Jon T %A Mounas,Peter %A Kirchhoff,Michael %A Wade,Elizabeth %+ Department of Psychology, Rowan University, 201 Mullica Hill Rd, Robinson Hall Room 115K, Glassboro, NJ, 08028, United States, 1 8562564500 ext 53777, tremoulet@rowan.edu %K discharge summary %K usability %K electronic health record (EHR) %K care coordination %K elderly patients %K patient safety %K heuristic evaluation %K human factors %D 2021 %7 15.4.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Obtaining accurate clinical information about recent acute care visits is extremely important for outpatient providers. However, documents used to communicate this information are often difficult to use. This puts patients at risk of adverse events. Elderly patients who are seen by more providers and have more care transitions are especially vulnerable. Objective: This study aimed to (1) identify the information about elderly patients’ recent acute care visits needed to coordinate their care, (2) use this information to assess discharge summaries, and (3) provide recommendations to help improve the quality of electronic health record (EHR)–generated discharge summaries, thereby increasing patient safety. Methods: A literature review, clinician interviews, and a survey of outpatient providers were used to identify and categorize data needed to coordinate care for recently discharged elderly patients. Based upon those data, 2 guidelines for creating useful discharge summaries were created. The new guidelines, along with 17 previously developed medical documentation usability heuristics, were applied to assess 4 simulated elderly patient discharge summaries. Results: The initial research effort yielded a list of 29 items that should always be included in elderly patient discharge summaries and a list of 7 “helpful, but not always necessary” items. Evaluation of 4 deidentified elderly patient discharge summaries revealed that none of the documents contained all 36 necessary items; between 14 and 18 were missing. The documents each had several other issues, and they differed significantly in organization, layout, and formatting. Conclusions: Variations in content and structure of discharge summaries in the United States make them unnecessarily difficult to use. Standardization would benefit both patients, by lowering the risk of care transition–related adverse events, and outpatient providers, by helping reduce frustration that can contribute to burnout. In the short term, acute care providers can help improve the quality of their discharge summaries by working with EHR vendors to follow recommendations based upon this study. Meanwhile, additional human factors work should determine the most effective way to organize and present information in discharge summaries, to facilitate effective standardization. %M 33856353 %R 10.2196/25657 %U https://www.jmir.org/2021/4/e25657 %U https://doi.org/10.2196/25657 %U http://www.ncbi.nlm.nih.gov/pubmed/33856353 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 4 %P e23961 %T Association of Electronic Health Record Vendors With Hospital Financial and Quality Performance: Retrospective Data Analysis %A Beauvais,Bradley %A Kruse,Clemens Scott %A Fulton,Lawrence %A Shanmugam,Ramalingam %A Ramamonjiarivelo,Zo %A Brooks,Matthew %+ School of Health Administration, College of Health Professions, Texas State University, 601 University Dr, San Marcos, TX, 78666, United States, 1 2103554742, scottkruse@txstate.edu %K electronic health records %K medical informatics %K hospitals %K delivery of health care %K financial management %K quality of health care %K treatment outcome %D 2021 %7 14.4.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Electronic health records (EHRs) are a central feature of care delivery in acute care hospitals; however, the financial and quality outcomes associated with system performance remain unclear. Objective: In this study, we aimed to evaluate the association between the top 3 EHR vendors and measures of hospital financial and quality performance. Methods: This study evaluated 2667 hospitals with Cerner, Epic, or Meditech as their primary EHR and considered their performance with regard to net income, Hospital Value–Based Purchasing Total Performance Score (TPS), and the unweighted subdomains of efficiency and cost reduction; clinical care; patient- and caregiver-centered experience; and patient safety. We hypothesized that there would be a difference among the 3 vendors for each measure. Results: None of the EHR systems were associated with a statistically significant financial relationship in our study. Epic was positively associated with TPS outcomes (R2=23.6%; β=.0159, SE 0.0079; P=.04) and higher patient perceptions of quality (R2=29.3%; β=.0292, SE 0.0099; P=.003) but was negatively associated with patient safety quality scores (R2=24.3%; β=−.0221, SE 0.0102; P=.03). Cerner and Epic were positively associated with improved efficiency (R2=31.9%; Cerner: β=.0330, SE 0.0135, P=.01; Epic: β=.0465, SE 0.0133, P<.001). Finally, all 3 vendors were associated with positive performance in the clinical care domain (Epic: β=.0388, SE 0.0122, P=.002; Cerner: β=.0283, SE 0.0124, P=.02; Meditech: β=.0273, SE 0.0123, P=.03) but with low explanatory power (R2=4.2%). Conclusions: The results of this study provide evidence of a difference in clinical outcome performance among the top 3 EHR vendors and may serve as supportive evidence for health care leaders to target future capital investments to improve health care delivery. %M 33851924 %R 10.2196/23961 %U https://www.jmir.org/2021/4/e23961 %U https://doi.org/10.2196/23961 %U http://www.ncbi.nlm.nih.gov/pubmed/33851924 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 4 %P e24360 %T Reduction in Hospital System Opioid Prescribing for Acute Pain Through Default Prescription Preference Settings: Pre–Post Study %A Slovis,Benjamin Heritier %A Riggio,Jeffrey M %A Girondo,Melanie %A Martino,Cara %A Babula,Bracken %A Roke,Lindsey M %A Kairys,John C %+ Office of Clinical Informatics, Thomas Jefferson University, 833 Chestnut Street Floor 10, Philadelphia, PA, 19107, United States, 1 (215) 955 6844, bxs088@jefferson.edu %K informatics %K electronic health record %K opioids %K prescriptions %K oxycodone %D 2021 %7 14.4.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: The United States is in an opioid epidemic. Passive decision support in the electronic health record (EHR) through opioid prescription presets may aid in curbing opioid dependence. Objective: The objective of this study is to determine whether modification of opioid prescribing presets in the EHR could change prescribing patterns for an entire hospital system. Methods: We performed a quasi-experimental retrospective pre–post analysis of a 24-month period before and after modifications to our EHR’s opioid prescription presets to match Centers for Disease Control and Prevention guidelines. We included all opioid prescriptions prescribed at our institution for nonchronic pain. Our modifications to the EHR include (1) making duration of treatment for an opioid prescription mandatory, (2) adding a quick button for 3 days’ duration while removing others, and (3) setting the default quantity of all oral opioid formulations to 10 tablets. We examined the quantity in tablets, duration in days, and proportion of prescriptions greater than 90 morphine milligram equivalents/day for our hospital system, and compared these values before and after our intervention for effect. Results: There were 78,246 prescriptions included in our study written on 30,975 unique patients. There was a significant reduction for all opioid prescriptions pre versus post in (1) the overall median quantity of tablets dispensed (54 [IQR 40-120] vs 42 [IQR 18-90]; P<.001), (2) median duration of treatment (10.5 days [IQR 5.0-30] vs 7.5 days [IQR 3.0-30]; P<.001), and (3) proportion of prescriptions greater than 90 morphine milligram equivalents/day (27.46% [10,704/38,976; 95% CI 27.02%-27.91%] vs 22.86% [8979/39,270; 95% CI 22.45%-23.28%]; P<.001). Conclusions: Modifications of opioid prescribing presets in the EHR can improve prescribing practice patterns. Reducing duration and quantity of opioid prescriptions could reduce the risk of dependence and overdose. %M 33851922 %R 10.2196/24360 %U https://www.jmir.org/2021/4/e24360 %U https://doi.org/10.2196/24360 %U http://www.ncbi.nlm.nih.gov/pubmed/33851922 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 4 %P e24754 %T Diagnostic Classification and Prognostic Prediction Using Common Genetic Variants in Autism Spectrum Disorder: Genotype-Based Deep Learning %A Wang,Haishuai %A Avillach,Paul %+ Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA, 02115, United States, 1 617 432 2144, Paul_Avillach@hms.harvard.edu %K deep learning %K autism spectrum disorder %K common genetic variants, diagnostic classification %D 2021 %7 7.4.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: In the United States, about 3 million people have autism spectrum disorder (ASD), and around 1 out of 59 children are diagnosed with ASD. People with ASD have characteristic social communication deficits and repetitive behaviors. The causes of this disorder remain unknown; however, in up to 25% of cases, a genetic cause can be identified. Detecting ASD as early as possible is desirable because early detection of ASD enables timely interventions in children with ASD. Identification of ASD based on objective pathogenic mutation screening is the major first step toward early intervention and effective treatment of affected children. Objective: Recent investigation interrogated genomics data for detecting and treating autism disorders, in addition to the conventional clinical interview as a diagnostic test. Since deep neural networks perform better than shallow machine learning models on complex and high-dimensional data, in this study, we sought to apply deep learning to genetic data obtained across thousands of simplex families at risk for ASD to identify contributory mutations and to create an advanced diagnostic classifier for autism screening. Methods: After preprocessing the genomics data from the Simons Simplex Collection, we extracted top ranking common variants that may be protective or pathogenic for autism based on a chi-square test. A convolutional neural network–based diagnostic classifier was then designed using the identified significant common variants to predict autism. The performance was then compared with shallow machine learning–based classifiers and randomly selected common variants. Results: The selected contributory common variants were significantly enriched in chromosome X while chromosome Y was also discriminatory in determining the identification of autistic individuals from nonautistic individuals. The ARSD, MAGEB16, and MXRA5 genes had the largest effect in the contributory variants. Thus, screening algorithms were adapted to include these common variants. The deep learning model yielded an area under the receiver operating characteristic curve of 0.955 and an accuracy of 88% for identifying autistic individuals from nonautistic individuals. Our classifier demonstrated a considerable improvement of ~13% in terms of classification accuracy compared to standard autism screening tools. Conclusions: Common variants are informative for autism identification. Our findings also suggest that the deep learning process is a reliable method for distinguishing the diseased group from the control group based on the common variants of autism. %M 33714937 %R 10.2196/24754 %U https://medinform.jmir.org/2021/4/e24754 %U https://doi.org/10.2196/24754 %U http://www.ncbi.nlm.nih.gov/pubmed/33714937 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 4 %P e24192 %T An Agent-Based Model of the Local Spread of SARS-CoV-2: Modeling Study %A Staffini,Alessio %A Svensson,Akiko Kishi %A Chung,Ung-Il %A Svensson,Thomas %+ Precision Health, Department of Bioengineering, Graduate School of Engineering, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8655, Japan, 81 080 7058 1309, alessio.staffini@bocconialumni.it %K computational epidemiology %K COVID-19 %K SARS-CoV-2 %K agent-based modeling %K public health %K computational models %K modeling %K agent %K spread %K computation %K epidemiology %K policy %D 2021 %7 6.4.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: The spread of SARS-CoV-2, originating in Wuhan, China, was classified as a pandemic by the World Health Organization on March 11, 2020. The governments of affected countries have implemented various measures to limit the spread of the virus. The starting point of this paper is the different government approaches, in terms of promulgating new legislative regulations to limit the virus diffusion and to contain negative effects on the populations. Objective: This paper aims to study how the spread of SARS-CoV-2 is linked to government policies and to analyze how different policies have produced different results on public health. Methods: Considering the official data provided by 4 countries (Italy, Germany, Sweden, and Brazil) and from the measures implemented by each government, we built an agent-based model to study the effects that these measures will have over time on different variables such as the total number of COVID-19 cases, intensive care unit (ICU) bed occupancy rates, and recovery and case-fatality rates. The model we implemented provides the possibility of modifying some starting variables, and it was thus possible to study the effects that some policies (eg, keeping the national borders closed or increasing the ICU beds) would have had on the spread of the infection. Results: The 4 considered countries have adopted different containment measures for COVID-19, and the forecasts provided by the model for the considered variables have given different results. Italy and Germany seem to be able to limit the spread of the infection and any eventual second wave, while Sweden and Brazil do not seem to have the situation under control. This situation is also reflected in the forecasts of pressure on the National Health Services, which see Sweden and Brazil with a high occupancy rate of ICU beds in the coming months, with a consequent high number of deaths. Conclusions: In line with what we expected, the obtained results showed that the countries that have taken restrictive measures in terms of limiting the population mobility have managed more successfully than others to contain the spread of COVID-19. Moreover, the model demonstrated that herd immunity cannot be reached even in countries that have relied on a strategy without strict containment measures. %M 33750735 %R 10.2196/24192 %U https://medinform.jmir.org/2021/4/e24192 %U https://doi.org/10.2196/24192 %U http://www.ncbi.nlm.nih.gov/pubmed/33750735 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 4 %P e21547 %T Implementation of the COVID-19 Vulnerability Index Across an International Network of Health Care Data Sets: Collaborative External Validation Study %A Reps,Jenna M %A Kim,Chungsoo %A Williams,Ross D %A Markus,Aniek F %A Yang,Cynthia %A Duarte-Salles,Talita %A Falconer,Thomas %A Jonnagaddala,Jitendra %A Williams,Andrew %A Fernández-Bertolín,Sergio %A DuVall,Scott L %A Kostka,Kristin %A Rao,Gowtham %A Shoaibi,Azza %A Ostropolets,Anna %A Spotnitz,Matthew E %A Zhang,Lin %A Casajust,Paula %A Steyerberg,Ewout W %A Nyberg,Fredrik %A Kaas-Hansen,Benjamin Skov %A Choi,Young Hwa %A Morales,Daniel %A Liaw,Siaw-Teng %A Abrahão,Maria Tereza Fernandes %A Areia,Carlos %A Matheny,Michael E %A Lynch,Kristine E %A Aragón,María %A Park,Rae Woong %A Hripcsak,George %A Reich,Christian G %A Suchard,Marc A %A You,Seng Chan %A Ryan,Patrick B %A Prieto-Alhambra,Daniel %A Rijnbeek,Peter R %+ Janssen Research & Development, 1125 Trenton Harbourton Rd, Titusville, NJ, United States, 1 732 715 6300, jreps@its.jnj.com %K external validation %K transportability %K COVID-19 %K prognostic model %K prediction %K C-19 %K modeling %K datasets %K observation %K hospitalization %K bias %K risk %K decision-making %D 2021 %7 5.4.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: SARS-CoV-2 is straining health care systems globally. The burden on hospitals during the pandemic could be reduced by implementing prediction models that can discriminate patients who require hospitalization from those who do not. The COVID-19 vulnerability (C-19) index, a model that predicts which patients will be admitted to hospital for treatment of pneumonia or pneumonia proxies, has been developed and proposed as a valuable tool for decision-making during the pandemic. However, the model is at high risk of bias according to the “prediction model risk of bias assessment” criteria, and it has not been externally validated. Objective: The aim of this study was to externally validate the C-19 index across a range of health care settings to determine how well it broadly predicts hospitalization due to pneumonia in COVID-19 cases. Methods: We followed the Observational Health Data Sciences and Informatics (OHDSI) framework for external validation to assess the reliability of the C-19 index. We evaluated the model on two different target populations, 41,381 patients who presented with SARS-CoV-2 at an outpatient or emergency department visit and 9,429,285 patients who presented with influenza or related symptoms during an outpatient or emergency department visit, to predict their risk of hospitalization with pneumonia during the following 0-30 days. In total, we validated the model across a network of 14 databases spanning the United States, Europe, Australia, and Asia. Results: The internal validation performance of the C-19 index had a C statistic of 0.73, and the calibration was not reported by the authors. When we externally validated it by transporting it to SARS-CoV-2 data, the model obtained C statistics of 0.36, 0.53 (0.473-0.584) and 0.56 (0.488-0.636) on Spanish, US, and South Korean data sets, respectively. The calibration was poor, with the model underestimating risk. When validated on 12 data sets containing influenza patients across the OHDSI network, the C statistics ranged between 0.40 and 0.68. Conclusions: Our results show that the discriminative performance of the C-19 index model is low for influenza cohorts and even worse among patients with COVID-19 in the United States, Spain, and South Korea. These results suggest that C-19 should not be used to aid decision-making during the COVID-19 pandemic. Our findings highlight the importance of performing external validation across a range of settings, especially when a prediction model is being extrapolated to a different population. In the field of prediction, extensive validation is required to create appropriate trust in a model. %M 33661754 %R 10.2196/21547 %U https://medinform.jmir.org/2021/4/e21547 %U https://doi.org/10.2196/21547 %U http://www.ncbi.nlm.nih.gov/pubmed/33661754 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 4 %P e21109 %T Factors Affecting General Practitioners’ Readiness to Accept and Use an Electronic Health Record System in the Republic of North Macedonia: A National Survey of General Practitioners %A Dimitrovski,Tomi %A Bath,Peter A %A Ketikidis,Panayiotis %A Lazuras,Lambros %+ CITY College, University of York Europe Campus, 24 Proxenou Koromila Street, Thessaloniki, 54622, Greece, 30 6979222130, tdimitrovski@citycollege.sheffield.eu %K general practitioner %K eHealth %K technology acceptance %K electronic health record %D 2021 %7 5.4.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Electronic health records (EHRs) represent an important aspect of digital health care, and to promote their use further, we need to better understand the drivers of their acceptance among health care professionals. EHRs are not simple computer applications; they should be considered as a highly integrated set of systems. Technology acceptance theories can be used to better understand users’ intentions to use EHRs. It is recommended to assess factors that determine the future acceptance of a system before it is implemented. Objective: This study uses a modified version of the Unified Theory of Acceptance and Use of Technology with the aim of examining the factors associated with intentions to use an EHR application among general practitioners (GPs) in the Republic of North Macedonia, a country that has been underrepresented in extant literature. More specifically, this study aims to assess the role of technology acceptance predictors such as performance expectancy, effort expectancy, social influence, facilitating conditions, job relevance, descriptive norms, and satisfaction with existing eHealth systems already implemented in the country. Methods: A web-based invitation was sent to 1174 GPs, of whom 458 completed the questionnaire (response rate=40.2%). The research instrument assessed performance expectancy, effort expectancy, facilitating conditions, and social influence in relation to the GPs’ intentions to use future EHR systems. Job relevance, descriptive norms, satisfaction with currently used eHealth systems in the country, and computer/internet use were also measured. Results: Hierarchical linear regression analysis showed that effort expectancy, descriptive norms, social influence, facilitating conditions, and job relevance were significantly associated with intentions to use the future EHR system, but performance expectance was not. Multiple mediation modeling analyses further showed that social influence (z=2.64; P<.001), facilitating conditions (z=4.54; P<.001), descriptive norms (z=4.91; P<.001), and effort expectancy (z=5.81; P=.008) mediated the association between job relevance and intentions. Finally, moderated regression analysis showed that the association between social influence and usage intention was significantly moderated (P=.02) by experience (Bexperience×social influence=.005; 95% CI 0.001 to 0.010; β=.080). In addition, the association between social influence and intentions was significantly moderated (P=.02) by age (Bage×social influence=.005; 95% CI 0.001 to 0.010; β=.077). Conclusions: Expectations of less effort in using EHRs and perceptions on supportive infrastructures for enabling EHR use were significantly associated with the greater acceptance of EHRs among GPs. Social norms were also associated with intentions, even more so among older GPs and those with less work experience. The theoretical and practical implications of these findings are also discussed. %M 33818399 %R 10.2196/21109 %U https://medinform.jmir.org/2021/4/e21109 %U https://doi.org/10.2196/21109 %U http://www.ncbi.nlm.nih.gov/pubmed/33818399 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 3 %P e23983 %T A Framework (SOCRATex) for Hierarchical Annotation of Unstructured Electronic Health Records and Integration Into a Standardized Medical Database: Development and Usability Study %A Park,Jimyung %A You,Seng Chan %A Jeong,Eugene %A Weng,Chunhua %A Park,Dongsu %A Roh,Jin %A Lee,Dong Yun %A Cheong,Jae Youn %A Choi,Jin Wook %A Kang,Mira %A Park,Rae Woong %+ Department of Biomedical Informatics, Ajou University School of Medicine, 164, World cup-ro, Yeongtong-gu, Suwon-si, Gyeonggi-do, Suwon, 16499, Republic of Korea, 82 31 219 4471, veritas@ajou.ac.kr %K natural language processing %K search engine %K data curation %K data management %K common data model %D 2021 %7 30.3.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Although electronic health records (EHRs) have been widely used in secondary assessments, clinical documents are relatively less utilized owing to the lack of standardized clinical text frameworks across different institutions. Objective: This study aimed to develop a framework for processing unstructured clinical documents of EHRs and integration with standardized structured data. Methods: We developed a framework known as Staged Optimization of Curation, Regularization, and Annotation of clinical text (SOCRATex). SOCRATex has the following four aspects: (1) extracting clinical notes for the target population and preprocessing the data, (2) defining the annotation schema with a hierarchical structure, (3) performing document-level hierarchical annotation using the annotation schema, and (4) indexing annotations for a search engine system. To test the usability of the proposed framework, proof-of-concept studies were performed on EHRs. We defined three distinctive patient groups and extracted their clinical documents (ie, pathology reports, radiology reports, and admission notes). The documents were annotated and integrated into the Observational Medical Outcomes Partnership (OMOP)-common data model (CDM) database. The annotations were used for creating Cox proportional hazard models with different settings of clinical analyses to measure (1) all-cause mortality, (2) thyroid cancer recurrence, and (3) 30-day hospital readmission. Results: Overall, 1055 clinical documents of 953 patients were extracted and annotated using the defined annotation schemas. The generated annotations were indexed into an unstructured textual data repository. Using the annotations of pathology reports, we identified that node metastasis and lymphovascular tumor invasion were associated with all-cause mortality among colon and rectum cancer patients (both P=.02). The other analyses involving measuring thyroid cancer recurrence using radiology reports and 30-day hospital readmission using admission notes in depressive disorder patients also showed results consistent with previous findings. Conclusions: We propose a framework for hierarchical annotation of textual data and integration into a standardized OMOP-CDM medical database. The proof-of-concept studies demonstrated that our framework can effectively process and integrate diverse clinical documents with standardized structured data for clinical research. %M 33783361 %R 10.2196/23983 %U https://medinform.jmir.org/2021/3/e23983 %U https://doi.org/10.2196/23983 %U http://www.ncbi.nlm.nih.gov/pubmed/33783361 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 3 %P e16306 %T Early Prediction of Unplanned 30-Day Hospital Readmission: Model Development and Retrospective Data Analysis %A Zhao,Peng %A Yoo,Illhoi %A Naqvi,Syed H %+ Department of Health Management and Informatics, School of Medicine, University of Missouri, Five Hospital Drive, CE718 Clinical Support and Education Building (DC006.00), Columbia, MO, 65212, United States, 1 5738827642, YooIL@health.missouri.edu %K patient readmission %K risk factors %K unplanned %K early detection %K all-cause %K predictive model %K 30-day %K machine learning %D 2021 %7 23.3.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Existing readmission reduction solutions tend to focus on complementing inpatient care with enhanced care transition and postdischarge interventions. These solutions are initiated near or after discharge, when clinicians’ impact on inpatient care is ending. Preventive intervention during hospitalization is an underexplored area that holds potential for reducing readmission risk. However, it is challenging to predict readmission risk at the early stage of hospitalization because few data are available. Objective: The objective of this study was to build an early prediction model of unplanned 30-day hospital readmission using a large and diverse sample. We were also interested in identifying novel readmission risk factors and protective factors. Methods: We extracted the medical records of 96,550 patients in 205 participating Cerner client hospitals across four US census regions in 2016 from the Health Facts database. The model was built with index admission data that can become available within 24 hours and data from previous encounters up to 1 year before the index admission. The candidate models were evaluated for performance, timeliness, and generalizability. Multivariate logistic regression analysis was used to identify readmission risk factors and protective factors. Results: We developed six candidate readmission models with different machine learning algorithms. The best performing model of extreme gradient boosting (XGBoost) achieved an area under the receiver operating characteristic curve of 0.753 on the development data set and 0.742 on the validation data set. By multivariate logistic regression analysis, we identified 14 risk factors and 2 protective factors of readmission that have never been reported. Conclusions: The performance of our model is better than that of the most widely used models in US health care settings. This model can help clinicians identify readmission risk at the early stage of hospitalization so that they can pay extra attention during the care process of high-risk patients. The 14 novel risk factors and 2 novel protective factors can aid understanding of the factors associated with readmission. %M 33755027 %R 10.2196/16306 %U https://medinform.jmir.org/2021/3/e16306 %U https://doi.org/10.2196/16306 %U http://www.ncbi.nlm.nih.gov/pubmed/33755027 %0 Journal Article %@ 1929-0748 %I JMIR Publications %V 10 %N 3 %P e25148 %T Beyond Getting Rid of Stupid Stuff in the Electronic Health Record (Beyond-GROSS): Protocol for a User-Centered, Mixed-Method Intervention to Improve the Electronic Health Record System %A Otokiti,Ahmed Umar %A Craven,Catherine K %A Shetreat-Klein,Avniel %A Cohen,Stacey %A Darrow,Bruce %+ Department of Medicine, Icahn School of Medicine at Mount Sinai, Mount Sinai Health System, 1 Gustave L Levy Place, New York, NY, 10029, United States, 1 212 289 6393, ahmedotoks@yahoo.com %K electronic health records %K burnout, psychological %K user-centered design %K usability %K EHR optimization %D 2021 %7 16.3.2021 %9 Protocol %J JMIR Res Protoc %G English %X Background: Up to 60% of health care providers experience one or more symptoms of burnout. Perceived clinician burden resulting in burnout arises from factors such as electronic health record (EHR) usability or lack thereof, perceived loss of autonomy, and documentation burden leading to less clinical time with patients. Burnout can have detrimental effects on health care quality and contributes to increased medical errors, decreased patient satisfaction, substance use, workforce attrition, and suicide. Objective: This project aims to improve the user-centered design of the EHR by obtaining direct input from clinicians about deficiencies. Fixing identified deficiencies via user-centered design has the potential to improve usability, thereby increasing satisfaction by reducing EHR-induced burnout. Methods: Quantitative and qualitative data will be obtained from clinician EHR users. The input will be received through a form built in a REDCap database via a link embedded in the home page of the EHR. The REDCap data will be analyzed in 2 main dimensions, based on nature of the input, what section of the EHR is affected, and what is required to fix the issue(s). Identified issues will be escalated to relevant stakeholders responsible for rectifying the problems identified. Data analysis, project evaluation, and lessons learned from the evaluation will be incorporated in a Plan-Do-Study-Act (PDSA) manner every 4-6 weeks. Results: The pilot phase of the study began in October 2020 in the Gastroenterology Division at Mount Sinai Hospital, New York City, NY, which includes 39 physicians and 15 nurses. The pilot is expected to run over a 4-6–month period. The results of the REDCap data analysis will be reported within 1 month of completing the pilot phase. We will analyze the nature of requests received and the impact of rectified issues on the clinician EHR user. We expect that the results will reveal which sections of the EHR have the highest deficiencies while also highlighting issues about workflow difficulties. Perceived impact of the project on provider engagement, patient safety, and workflow efficiency will also be captured by evaluation survey and other qualitative methods where possible. Conclusions: The project aims to improve user-centered design of the EHR by soliciting direct input from clinician EHR users. The ultimate goal is to improve efficiency, reduce EHR inefficiencies with the possibility of improving staff engagement, and lessen EHR-induced clinician burnout. Our project implementation includes using informatics expertise to achieve the desired state of a learning health system as recommended by the National Academy of Medicine as we facilitate feedback loops and rapid cycles of improvement. International Registered Report Identifier (IRRID): PRR1-10.2196/25148 %M 33724202 %R 10.2196/25148 %U https://www.researchprotocols.org/2021/3/e25148 %U https://doi.org/10.2196/25148 %U http://www.ncbi.nlm.nih.gov/pubmed/33724202 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 3 %P e23456 %T Natural Language Processing of Clinical Notes to Identify Mental Illness and Substance Use Among People Living with HIV: Retrospective Cohort Study %A Ridgway,Jessica P %A Uvin,Arno %A Schmitt,Jessica %A Oliwa,Tomasz %A Almirol,Ellen %A Devlin,Samantha %A Schneider,John %+ Department of Medicine, University of Chicago, 5841 S Maryland Ave, MC 5065, Chicago, IL, 60637, United States, 1 7737029185, jessica.ridgway@uchospitals.edu %K natural language processing %K HIV %K substance use %K mental illness %K electronic medical records %D 2021 %7 10.3.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Mental illness and substance use are prevalent among people living with HIV and often lead to poor health outcomes. Electronic medical record (EMR) data are increasingly being utilized for HIV-related clinical research and care, but mental illness and substance use are often underdocumented in structured EMR fields. Natural language processing (NLP) of unstructured text of clinical notes in the EMR may more accurately identify mental illness and substance use among people living with HIV than structured EMR fields alone. Objective: The aim of this study was to utilize NLP of clinical notes to detect mental illness and substance use among people living with HIV and to determine how often these factors are documented in structured EMR fields. Methods: We collected both structured EMR data (diagnosis codes, social history, Problem List) as well as the unstructured text of clinical HIV care notes for adults living with HIV. We developed NLP algorithms to identify words and phrases associated with mental illness and substance use in the clinical notes. The algorithms were validated based on chart review. We compared numbers of patients with documentation of mental illness or substance use identified by structured EMR fields with those identified by the NLP algorithms. Results: The NLP algorithm for detecting mental illness had a positive predictive value (PPV) of 98% and a negative predictive value (NPV) of 98%. The NLP algorithm for detecting substance use had a PPV of 92% and an NPV of 98%. The NLP algorithm for mental illness identified 54.0% (420/778) of patients as having documentation of mental illness in the text of clinical notes. Among the patients with mental illness detected by NLP, 58.6% (246/420) had documentation of mental illness in at least one structured EMR field. Sixty-three patients had documentation of mental illness in structured EMR fields that was not detected by NLP of clinical notes. The NLP algorithm for substance use detected substance use in the text of clinical notes in 18.1% (141/778) of patients. Among patients with substance use detected by NLP, 73.8% (104/141) had documentation of substance use in at least one structured EMR field. Seventy-six patients had documentation of substance use in structured EMR fields that was not detected by NLP of clinical notes. Conclusions: Among patients in an urban HIV care clinic, NLP of clinical notes identified high rates of mental illness and substance use that were often not documented in structured EMR fields. This finding has important implications for epidemiologic research and clinical care for people living with HIV. %M 33688848 %R 10.2196/23456 %U https://medinform.jmir.org/2021/3/e23456 %U https://doi.org/10.2196/23456 %U http://www.ncbi.nlm.nih.gov/pubmed/33688848 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 3 %P e22806 %T Contribution of Free-Text Comments to the Burden of Documentation: Assessment and Analysis of Vital Sign Comments in Flowsheets %A Yin,Zhijun %A Liu,Yongtai %A McCoy,Allison B %A Malin,Bradley A %A Sengstack,Patricia R %+ Department of Biomedical Informatics, Vanderbilt University Medical Center, 2525 West End Ave., Suite 1475, Nashville, TN, 37203, United States, 1 615 936 3690, zhijun.yin@vanderbilt.edu %K electronic health system %K documentation burden %K flowsheets %K content analysis %K vital sign comments %K free text %D 2021 %7 4.3.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Documentation burden is a common problem with modern electronic health record (EHR) systems. To reduce this burden, various recording methods (eg, voice recorders or motion sensors) have been proposed. However, these solutions are in an early prototype phase and are unlikely to transition into practice in the near future. A more pragmatic alternative is to directly modify the implementation of the existing functionalities of an EHR system. Objective: This study aims to assess the nature of free-text comments entered into EHR flowsheets that supplement quantitative vital sign values and examine opportunities to simplify functionality and reduce documentation burden. Methods: We evaluated 209,055 vital sign comments in flowsheets that were generated in the Epic EHR system at the Vanderbilt University Medical Center in 2018. We applied topic modeling, as well as the natural language processing Clinical Language Annotation, Modeling, and Processing software system, to extract generally discussed topics and detailed medical terms (expressed as probability distribution) to investigate the stories communicated in these comments. Results: Our analysis showed that 63.33% (6053/9557) of the users who entered vital signs made at least one free-text comment in vital sign flowsheet entries. The user roles that were most likely to compose comments were registered nurse, technician, and licensed nurse. The most frequently identified topics were the notification of a result to health care providers (0.347), the context of a measurement (0.307), and an inability to obtain a vital sign (0.224). There were 4187 unique medical terms that were extracted from 46,029 (0.220) comments, including many symptom-related terms such as “pain,” “upset,” “dizziness,” “coughing,” “anxiety,” “distress,” and “fever” and drug-related terms such as “tylenol,” “anesthesia,” “cannula,” “oxygen,” “motrin,” “rituxan,” and “labetalol.” Conclusions: Considering that flowsheet comments are generally not displayed or automatically pulled into any clinical notes, our findings suggest that the flowsheet comment functionality can be simplified (eg, via structured response fields instead of a text input dialog) to reduce health care provider effort. Moreover, rich and clinically important medical terms such as medications and symptoms should be explicitly recorded in clinical notes for better visibility. %M 33661128 %R 10.2196/22806 %U https://www.jmir.org/2021/3/e22806 %U https://doi.org/10.2196/22806 %U http://www.ncbi.nlm.nih.gov/pubmed/33661128 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 3 %P e23951 %T Commitment Levels of Health Care Providers in Using the District Health Information System and the Associated Factors for Decision Making in Resource-Limited Settings: Cross-sectional Survey Study %A Kanfe,Shuma G %A Endehabtu,Berhanu F %A Ahmed,Mohammedjud H %A Mengestie,Nebyu D %A Tilahun,Binyam %+ Health Informatics, Mettu University, Metu Zuria, Ethiopia, 251 0935054730, shumagosha33@gmail.com %K commitment %K district health information system %K decision making %K performance monitoring %K health facilities %K information use %D 2021 %7 4.3.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Changing the culture of information use, which is one of the transformation agendas of the Ministry of Health of Ethiopia, cannot become real unless health care providers are committed to using locally collected data for evidence-based decision making. The commitment of health care providers has paramount influence on district health information system 2 (DHIS2) data utilization for decision making. Evidence is limited on health care providers’ level of commitment to using DHIS2 data in Ethiopia. Therefore, this study aims to fill this evidence gap. Objective: This study aimed to assess the levels of commitment of health care providers and the factors influencing their commitment levels in using DHIS2 data for decision making at public health care facilities in the Ilu Aba Bora zone of the Oromia national regional state, Ethiopia in 2020. Methods: The cross-sectional quantitative study supplemented by qualitative methods was conducted from February 26, 2020 to April 17, 2020. A total of 264 participants were approached. SPSS version 20 software was used for data entry and analysis. Descriptive and analytical statistics, including bivariable and multivariable analyses, were performed. Thematic analysis was conducted for the qualitative data. Results: Of the 264 respondents, 121 (45.8%, 95% CI 40.0%-52.8%) respondents showed high commitment levels to use DHIS2 data. The variables associated with the level of commitment to use DHIS2 data were found to be provision of feedback for DHIS2 data use (adjusted odds ratio [AOR] 1.85, 95% CI 1.02-3.33), regular supervision and managerial support (AOR 2.84, 95% CI 1.50-5.37), information use culture (AOR 1.92, 95% CI 1.03-3.59), motivation to use DHIS2 data (AOR 1.80, 95% CI 1.00-3.25), health needs (AOR 3.96, 95% CI 2.11-7.41), and competency in DHIS2 tasks (AOR 2.41, 95% CI 1.27-4.55). Conclusions: In general, less than half of the study participants showed high commitment levels to use DHIS2 data for decision making in health care. Providing regular supportive supervision and feedback and increasing the motivation and competency of the health care providers in performing DHIS2 data tasks will help in promoting their levels of commitment that can result in the cultural transformation of data use for evidence-based decision making in health care. %M 33661133 %R 10.2196/23951 %U https://medinform.jmir.org/2021/3/e23951 %U https://doi.org/10.2196/23951 %U http://www.ncbi.nlm.nih.gov/pubmed/33661133 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 3 %P e22923 %T Physicians’ Use of the Computerized Physician Order Entry System for Medication Prescribing: Systematic Review %A Mogharbel,Asra %A Dowding,Dawn %A Ainsworth,John %+ Division of Informatics Imaging and Data Sciences, School of Health Sciences, Faculty of Biology, Medicine and Health, Manchester Academic Health Science Centre, Centre for Health Informatics, The University of Manchester, Vaughan House, Portsmouth St, Manchester, M13 9GB, United Kingdom, 44 161 275 1274, asra.mogharbel@postgrad.manchester.ac.uk %K computerized physician order entry %K CPOE %K e-prescribing %K system use %K actual usage %K systematic review %D 2021 %7 4.3.2021 %9 Review %J JMIR Med Inform %G English %X Background: Computerized physician order entry (CPOE) systems in health care settings have many benefits for prescribing medication, such as improved quality of patient care and patient safety. However, to achieve their full potential, the factors influencing the usage of CPOE systems by physicians must be identified and understood. Objective: The aim of this study is to identify the factors influencing the usage of CPOE systems by physicians for medication prescribing in their clinical practice. Methods: We conducted a systematic search of the literature on this topic using four databases: PubMed, CINAHL, Ovid MEDLINE, and Embase. Searches were performed from September 2019 to December 2019. The retrieved papers were screened by examining the titles and abstracts of relevant studies; two reviewers screened the full text of potentially relevant papers for inclusion in the review. Qualitative, quantitative, and mixed methods studies with the aim of conducting assessments or investigations of factors influencing the use of CPOE for medication prescribing among physicians were included. The identified factors were grouped based on constructs from two models: the unified theory of acceptance and use of technology model and the Delone and McLean Information System Success Model. We used the Mixed Method Appraisal Tool to assess the quality of the included studies and narrative synthesis to report the results. Results: A total of 11 articles were included in the review, and 37 factors related to the usage of CPOE systems were identified as the factors influencing how physicians used CPOE for medication prescribing. These factors represented three main themes: individual, technological, and organizational. Conclusions: This study identified the common factors that influenced the usage of CPOE systems by physicians for medication prescribing regardless of the type of setting or the duration of the use of a system by participants. Our findings can be used to inform implementation and support the usage of the CPOE system by physicians. %M 33661126 %R 10.2196/22923 %U https://medinform.jmir.org/2021/3/e22923 %U https://doi.org/10.2196/22923 %U http://www.ncbi.nlm.nih.gov/pubmed/33661126 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 3 %P e22219 %T What Every Reader Should Know About Studies Using Electronic Health Record Data but May Be Afraid to Ask %A Kohane,Isaac S %A Aronow,Bruce J %A Avillach,Paul %A Beaulieu-Jones,Brett K %A Bellazzi,Riccardo %A Bradford,Robert L %A Brat,Gabriel A %A Cannataro,Mario %A Cimino,James J %A García-Barrio,Noelia %A Gehlenborg,Nils %A Ghassemi,Marzyeh %A Gutiérrez-Sacristán,Alba %A Hanauer,David A %A Holmes,John H %A Hong,Chuan %A Klann,Jeffrey G %A Loh,Ne Hooi Will %A Luo,Yuan %A Mandl,Kenneth D %A Daniar,Mohamad %A Moore,Jason H %A Murphy,Shawn N %A Neuraz,Antoine %A Ngiam,Kee Yuan %A Omenn,Gilbert S %A Palmer,Nathan %A Patel,Lav P %A Pedrera-Jiménez,Miguel %A Sliz,Piotr %A South,Andrew M %A Tan,Amelia Li Min %A Taylor,Deanne M %A Taylor,Bradley W %A Torti,Carlo %A Vallejos,Andrew K %A Wagholikar,Kavishwar B %A , %A Weber,Griffin M %A Cai,Tianxi %+ Department of Biomedical Informatics, Harvard Medical School, 10 Shattuck Street, Boston, MA, 02115, United States, 1 617 432 3226, isaac_kohane@harvard.edu %K COVID-19 %K electronic health records %K real-world data %K literature %K publishing %K quality %K data quality %K reporting standards %K reporting checklist %K review %K statistics %D 2021 %7 2.3.2021 %9 Viewpoint %J J Med Internet Res %G English %X Coincident with the tsunami of COVID-19–related publications, there has been a surge of studies using real-world data, including those obtained from the electronic health record (EHR). Unfortunately, several of these high-profile publications were retracted because of concerns regarding the soundness and quality of the studies and the EHR data they purported to analyze. These retractions highlight that although a small community of EHR informatics experts can readily identify strengths and flaws in EHR-derived studies, many medical editorial teams and otherwise sophisticated medical readers lack the framework to fully critically appraise these studies. In addition, conventional statistical analyses cannot overcome the need for an understanding of the opportunities and limitations of EHR-derived studies. We distill here from the broader informatics literature six key considerations that are crucial for appraising studies utilizing EHR data: data completeness, data collection and handling (eg, transformation), data type (ie, codified, textual), robustness of methods against EHR variability (within and across institutions, countries, and time), transparency of data and analytic code, and the multidisciplinary approach. These considerations will inform researchers, clinicians, and other stakeholders as to the recommended best practices in reviewing manuscripts, grants, and other outputs from EHR-data derived studies, and thereby promote and foster rigor, quality, and reliability of this rapidly growing field. %M 33600347 %R 10.2196/22219 %U https://www.jmir.org/2021/3/e22219 %U https://doi.org/10.2196/22219 %U http://www.ncbi.nlm.nih.gov/pubmed/33600347 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 3 %P e22974 %T Electronic Health Record Use in Swiss Nursing Homes and Its Association With Implicit Rationing of Nursing Care Documentation: Multicenter Cross-sectional Survey Study %A Ausserhofer,Dietmar %A Favez,Lauriane %A Simon,Michael %A Zúñiga,Franziska %+ Nursing Science, Department of Public Health, University of Basel, Bernoullistrasse 28, Basel, 4056, Switzerland, 41 61 207 09 13, franziska.zuniga@unibas.ch %K electronic health records %K nursing homes %K nursing care %K health care rationing %K rationing of nursing care %K unfinished care %K documentation %K patient care planning %K mobile phone %D 2021 %7 2.3.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Nursing homes (NHs) are increasingly implementing electronic health records (EHRs); however, little information is available on EHR use in NH settings. It remains unclear how care workers perceive its safety, quality, and efficiency, and whether EHR use might ease the burden of documentation, thereby reducing its implicit rationing. Objective: This study aims to describe nurses’ perceptions regarding the usefulness of the EHR system and whether sufficient numbers of computers are available in Swiss NHs, and to explore the system’s association with implicit rationing of nursing care documentation. Methods: This was a multicenter cross-sectional study using survey data from the Swiss Nursing Homes Human Resources Project 2018. It includes a convenience sample of 107 NHs, 302 care units, and 1975 care workers (ie, registered nurses and licensed practical nurses) from Switzerland’s German- and French-speaking regions. Care workers completed questionnaires assessing the level of implicit rationing of nursing care documentation, their perceptions of the EHR system’s usefulness and of how sufficient the number of available computers was, staffing and resource adequacy, leadership ability, and teamwork and safety climate. For analysis, we applied generalized linear mixed models, including individual-level nurse survey data and data on unit and facility characteristics. Results: Overall, the care workers perceived the EHR systems as useful; ratings ranged from 69.42% (1362/1962; guarantees safe care and treatment) to 78.32% (1535/1960; allows quick access to relevant information on the residents). However, less than half (914/1961, 46.61%) of the care workers reported sufficient computers on their unit to allow timely documentation. Half of the care workers responded that they sometimes or often had to ration the documentation of care. After adjusting for work environment factors and safety and teamwork climate, both higher care worker ratings of the EHR system’s usefulness (β=−.12; 95% CI −0.17 to −0.06) and sufficient numbers of computers (β=−.09; 95% CI −0.12 to −0.06) were consistently associated with lower implicit rationing of nursing care documentation. Conclusions: Both the usefulness of the EHR system and the number of computers available were important explanatory factors for care workers leaving care activities (eg, developing or updating nursing care plans) unfinished. NH managers should carefully select and implement their information technology infrastructure with greater involvement and attention to the needs of their care workers and residents. Further research is needed to develop and implement user-friendly information technology infrastructure in NHs and to evaluate their impact on care processes as well as resident and care worker outcomes. %M 33650983 %R 10.2196/22974 %U https://medinform.jmir.org/2021/3/e22974 %U https://doi.org/10.2196/22974 %U http://www.ncbi.nlm.nih.gov/pubmed/33650983 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 3 %P e24188 %T Human–Computer Agreement of Electrocardiogram Interpretation for Patients Referred to and Declined for Primary Percutaneous Coronary Intervention: Retrospective Data Analysis Study %A Iftikhar,Aleeha %A Bond,Raymond %A Mcgilligan,Victoria %A Leslie,Stephen J %A Knoery,Charles %A Shand,James %A Ramsewak,Adesh %A Sharma,Divyesh %A McShane,Anne %A Rjoob,Khaled %A Peace,Aaron %+ Computing Engineering and Build Environment, Ulster University, Jordanstown, Belfast, BT37 0QB, United Kingdom, 44 7496635353, Iftikhar-a1@ulster.ac.uk %K ECG interpretation %K agreement between human and computer %K primary percutaneous coronary intervention service %K acute myocardial infarction %K scan %K electrocardiogram %K heart %K intervention %K infarction %K human-computer %K diagnostic %D 2021 %7 2.3.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: When a patient is suspected of having an acute myocardial infarction, they are accepted or declined for primary percutaneous coronary intervention partly based on clinical assessment of their 12-lead electrocardiogram (ECG) and ST-elevation myocardial infarction criteria. Objective: We retrospectively determined the agreement rate between human (specialists called activator nurses) and computer interpretations of ECGs of patients who were declined for primary percutaneous coronary intervention. Methods: Various features of patients who were referred for primary percutaneous coronary intervention were analyzed. Both the human and computer ECG interpretations were simplified to either “suggesting” or “not suggesting” acute myocardial infarction to avoid analysis of complex heterogeneous and synonymous diagnostic terms. Analyses, to measure agreement, and logistic regression, to determine if these ECG interpretations (and other variables such as patient age, chest pain) could predict patient mortality, were carried out. Results: Of a total of 1464 patients referred to and declined for primary percutaneous coronary intervention, 722 (49.3%) computer diagnoses suggested acute myocardial infarction, whereas 634 (43.3%) of the human interpretations suggested acute myocardial infarction (P<.001). The human and computer agreed that there was a possible acute myocardial infarction for 342 out of 1464 (23.3%) patients. However, there was a higher rate of human–computer agreement for patients not having acute myocardial infarctions (450/1464, 30.7%). The overall agreement rate was 54.1% (792/1464). Cohen κ showed poor agreement (κ=0.08, P=.001). Only the age (odds ratio [OR] 1.07, 95% CI 1.05-1.09) and chest pain (OR 0.59, 95% CI 0.39-0.89) independent variables were statistically significant (P=.008) in predicting mortality after 30 days and 1 year. The odds for mortality within 1 year of referral were lower in patients with chest pain compared to those patients without chest pain. A referral being out of hours was a trending variable (OR 1.41, 95% CI 0.95-2.11, P=.09) for predicting the odds of 1-year mortality. Conclusions: Mortality in patients who were declined for primary percutaneous coronary intervention was higher than the reported mortality for ST-elevation myocardial infarction patients at 1 year. Agreement between computerized and human ECG interpretation is poor, perhaps leading to a high rate of inappropriate referrals. Work is needed to improve computer and human decision making when reading ECGs to ensure that patients are referred to the correct treatment facility for time-critical therapy. %M 33650984 %R 10.2196/24188 %U https://medinform.jmir.org/2021/3/e24188 %U https://doi.org/10.2196/24188 %U http://www.ncbi.nlm.nih.gov/pubmed/33650984 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 2 %P e24813 %T Adoption of Electronic Health Records (EHRs) in China During the Past 10 Years: Consecutive Survey Data Analysis and Comparison of Sino-American Challenges and Experiences %A Liang,Jun %A Li,Ying %A Zhang,Zhongan %A Shen,Dongxia %A Xu,Jie %A Zheng,Xu %A Wang,Tong %A Tang,Buzhou %A Lei,Jianbo %A Zhang,Jiajie %+ Institute of Medical Technology, Health Science Center, Peking University, 38 Xueyuan Rd, Haidian District, Beijing, 100191, China, 86 (10) 8280 59, jblei@hsc.pku.edu.cn %K medical informatics %K health information technologies %K electronic health records %K hospitals %K Sino-American %D 2021 %7 18.2.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: The adoption rate of electronic health records (EHRs) in hospitals has become a main index to measure digitalization in medicine in each country. Objective: This study summarizes and shares the experiences with EHR adoption in China and in the United States. Methods: Using the 2007-2018 annual hospital survey data from the Chinese Health Information Management Association (CHIMA) and the 2008-2017 United States American Hospital Association Information Technology Supplement survey data, we compared the trends in EHR adoption rates in China and the United States. We then used the Bass model to fit these data and to analyze the modes of diffusion of EHRs in these 2 countries. Finally, using the 2007, 2010, and 2014 CHIMA and Healthcare Information and Management Systems Services survey data, we analyzed the major challenges faced by hospitals in China and the United States in developing health information technology. Results: From 2007 to 2018, the average adoption rates of the sampled hospitals in China increased from 18.6% to 85.3%, compared to the increase from 9.4% to 96% in US hospitals from 2008 to 2017. The annual average adoption rates in Chinese and US hospitals were 6.1% and 9.6%, respectively. However, the annual average number of hospitals adopting EHRs was 1500 in China and 534 in the US, indicating that the former might require more effort. Both countries faced similar major challenges for hospital digitalization. Conclusions: The adoption rates of hospital EHRs in China and the United States have both increased significantly in the past 10 years. The number of hospitals that adopted EHRs in China exceeded 16,000, which was 3.3 times that of the 4814 nonfederal US hospitals. This faster adoption outcome may have been a benefit of top-level design and government-led policies, particularly the inclusion of EHR adoption as an important indicator for performance evaluation and the appointment of public hospitals. %M 33599615 %R 10.2196/24813 %U http://www.jmir.org/2021/2/e24813/ %U https://doi.org/10.2196/24813 %U http://www.ncbi.nlm.nih.gov/pubmed/33599615 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 2 %P e23606 %T Electronic Health Record–Based Prediction of 1-Year Risk of Incident Cardiac Dysrhythmia: Prospective Case-Finding Algorithm Development and Validation Study %A Zhang,Yaqi %A Han,Yongxia %A Gao,Peng %A Mo,Yifu %A Hao,Shiying %A Huang,Jia %A Ye,Fangfan %A Li,Zhen %A Zheng,Le %A Yao,Xiaoming %A Li,Zhen %A Li,Xiaodong %A Wang,Xiaofang %A Huang,Chao-Jung %A Jin,Bo %A Zhang,Yani %A Yang,Gabriel %A Alfreds,Shaun T %A Kanov,Laura %A Sylvester,Karl G %A Widen,Eric %A Li,Licheng %A Ling,Xuefeng %+ Department of Surgery, Stanford University, S370 Grant Building, 300 Pasteur Drive, Stanford, CA, 94305, United States, 1 6504279198, bxling@stanford.edu %K cardiac dysrhythmia %K prospective case finding %K risk stratification %K electronic health records %D 2021 %7 17.2.2021 %9 Original Paper %J JMIR Med Inform %G English %X Background: Cardiac dysrhythmia is currently an extremely common disease. Severe arrhythmias often cause a series of complications, including congestive heart failure, fainting or syncope, stroke, and sudden death. Objective: The aim of this study was to predict incident arrhythmia prospectively within a 1-year period to provide early warning of impending arrhythmia. Methods: Retrospective (1,033,856 individuals enrolled between October 1, 2016, and October 1, 2017) and prospective (1,040,767 individuals enrolled between October 1, 2017, and October 1, 2018) cohorts were constructed from integrated electronic health records in Maine, United States. An ensemble learning workflow was built through multiple machine learning algorithms. Differentiating features, including acute and chronic diseases, procedures, health status, laboratory tests, prescriptions, clinical utilization indicators, and socioeconomic determinants, were compiled for incident arrhythmia assessment. The predictive model was retrospectively trained and calibrated using an isotonic regression method and was prospectively validated. Model performance was evaluated using the area under the receiver operating characteristic curve (AUROC). Results: The cardiac dysrhythmia case-finding algorithm (retrospective: AUROC 0.854; prospective: AUROC 0.827) stratified the population into 5 risk groups: 53.35% (555,233/1,040,767), 44.83% (466,594/1,040,767), 1.76% (18,290/1,040,767), 0.06% (623/1,040,767), and 0.003% (27/1,040,767) were in the very low-risk, low-risk, medium-risk, high-risk, and very high-risk groups, respectively; 51.85% (14/27) patients in the very high-risk subgroup were confirmed to have incident cardiac dysrhythmia within the subsequent 1 year. Conclusions: Our case-finding algorithm is promising for prospectively predicting 1-year incident cardiac dysrhythmias in a general population, and we believe that our case-finding algorithm can serve as an early warning system to allow statewide population-level screening and surveillance to improve cardiac dysrhythmia care. %M 33595452 %R 10.2196/23606 %U http://medinform.jmir.org/2021/2/e23606/ %U https://doi.org/10.2196/23606 %U http://www.ncbi.nlm.nih.gov/pubmed/33595452 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 2 %P e22939 %T A Bespoke Electronic Health Record for Epilepsy Care (EpiToMe): Development and Qualitative Evaluation %A Tao,Shiqiang %A Lhatoo,Samden %A Hampson,Johnson %A Cui,Licong %A Zhang,Guo-Qiang %+ Department of Neurology, The University of Texas Health Science Center at Houston, 1133 John Freeman Blvd, JJL 430, Houston, TX, 77030, United States, 1 7135007117, guo-qiang.zhang@uth.tmc.edu %K specialty-specific EHR %K physician-centered design %K clinical workflow %K patient care management %K clinical care documentation %K physician burnout %K interoperability %D 2021 %7 12.2.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: While electronic health records (EHR) bring various benefits to health care, EHR systems are often criticized as cumbersome to use, failing to fulfill the promise of improved health care delivery with little more than a means of meeting regulatory and billing requirements. EHR has also been recognized as one of the contributing factors for physician burnout. Objective: Specialty-specific EHR systems have been suggested as an alternative approach that can potentially address challenges associated with general-purpose EHRs. We introduce the Epilepsy Tracking and optimized Management engine (EpiToMe), an exemplar bespoke EHR system for epilepsy care. EpiToMe uses an agile, physician-centered development strategy to optimize clinical workflow and patient care documentation. We present the design and implementation of EpiToMe and report the initial feedback on its utility for physician burnout. Methods: Using collaborative, asynchronous data capturing interfaces anchored to a domain ontology, EpiToMe distributes reporting and documentation workload among technicians, clinical fellows, and attending physicians. Results of documentation are transmitted to the parent EHR to meet patient care requirements with a push of a button. An HL7 (version 2.3) messaging engine exchanges information between EpiToMe and the parent EHR to optimize clinical workflow tasks without redundant data entry. EpiToMe also provides live, interactive patient tracking interfaces to ease the burden of care management. Results: Since February 2019, 15,417 electroencephalogram reports, 2635 Epilepsy Monitoring Unit daily reports, and 1369 Epilepsy Monitoring Unit phase reports have been completed in EpiToMe for 6593 unique patients. A 10-question survey was completed by 11 (among 16 invited) senior clinical attending physicians. Consensus was found that EpiToMe eased the burden of care documentation for patient management, a contributing factor to physician burnout. Conclusions: EpiToMe offers an exemplar bespoke EHR system developed using a physician-centered design and latest advancements in information technology. The bespoke approach has the potential to ease the burden of care management in epilepsy. This approach is applicable to other clinical specialties. %M 33576745 %R 10.2196/22939 %U http://www.jmir.org/2021/2/e22939/ %U https://doi.org/10.2196/22939 %U http://www.ncbi.nlm.nih.gov/pubmed/33576745 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 2 %P e24246 %T A Machine Learning Prediction Model of Respiratory Failure Within 48 Hours of Patient Admission for COVID-19: Model Development and Validation %A Bolourani,Siavash %A Brenner,Max %A Wang,Ping %A McGinn,Thomas %A Hirsch,Jamie S %A Barnaby,Douglas %A Zanos,Theodoros P %A , %+ Feinstein Institutes for Medical Research, Northwell Health, 350 Community Dr, Room 1257, Manhasset, NY, 11030, United States, 1 5165620484, tzanos@northwell.edu %K artificial intelligence %K prognostic %K model %K pandemic %K severe acute respiratory syndrome coronavirus 2 %K modeling %K development %K validation %K COVID-19 %K machine learning %D 2021 %7 10.2.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Predicting early respiratory failure due to COVID-19 can help triage patients to higher levels of care, allocate scarce resources, and reduce morbidity and mortality by appropriately monitoring and treating the patients at greatest risk for deterioration. Given the complexity of COVID-19, machine learning approaches may support clinical decision making for patients with this disease. Objective: Our objective is to derive a machine learning model that predicts respiratory failure within 48 hours of admission based on data from the emergency department. Methods: Data were collected from patients with COVID-19 who were admitted to Northwell Health acute care hospitals and were discharged, died, or spent a minimum of 48 hours in the hospital between March 1 and May 11, 2020. Of 11,525 patients, 933 (8.1%) were placed on invasive mechanical ventilation within 48 hours of admission. Variables used by the models included clinical and laboratory data commonly collected in the emergency department. We trained and validated three predictive models (two based on XGBoost and one that used logistic regression) using cross-hospital validation. We compared model performance among all three models as well as an established early warning score (Modified Early Warning Score) using receiver operating characteristic curves, precision-recall curves, and other metrics. Results: The XGBoost model had the highest mean accuracy (0.919; area under the curve=0.77), outperforming the other two models as well as the Modified Early Warning Score. Important predictor variables included the type of oxygen delivery used in the emergency department, patient age, Emergency Severity Index level, respiratory rate, serum lactate, and demographic characteristics. Conclusions: The XGBoost model had high predictive accuracy, outperforming other early warning scores. The clinical plausibility and predictive ability of XGBoost suggest that the model could be used to predict 48-hour respiratory failure in admitted patients with COVID-19. %M 33476281 %R 10.2196/24246 %U http://www.jmir.org/2021/2/e24246/ %U https://doi.org/10.2196/24246 %U http://www.ncbi.nlm.nih.gov/pubmed/33476281 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 12 %P e18418 %T Limitations of Deep Learning Attention Mechanisms in Clinical Research: Empirical Case Study Based on the Korean Diabetic Disease Setting %A Kim,Junetae %A Lee,Sangwon %A Hwang,Eugene %A Ryu,Kwang Sun %A Jeong,Hanseok %A Lee,Jae Wook %A Hwangbo,Yul %A Choi,Kui Son %A Cha,Hyo Soung %+ Cancer Data Center, National Cancer Control Institute, National Cancer Center, 809 Madu 1(il)-dong, Ilsandong-gu, Goyang-si, Gyeonggi-do, 10408, Republic of Korea, 82 31 920 1892, kkido@ncc.re.kr %K attention %K deep learning %K explainable artificial intelligence %K uncertainty awareness %K Bayesian deep learning %K artificial intelligence %K health data %D 2020 %7 16.12.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Despite excellent prediction performance, noninterpretability has undermined the value of applying deep-learning algorithms in clinical practice. To overcome this limitation, attention mechanism has been introduced to clinical research as an explanatory modeling method. However, potential limitations of using this attractive method have not been clarified to clinical researchers. Furthermore, there has been a lack of introductory information explaining attention mechanisms to clinical researchers. Objective: The aim of this study was to introduce the basic concepts and design approaches of attention mechanisms. In addition, we aimed to empirically assess the potential limitations of current attention mechanisms in terms of prediction and interpretability performance. Methods: First, the basic concepts and several key considerations regarding attention mechanisms were identified. Second, four approaches to attention mechanisms were suggested according to a two-dimensional framework based on the degrees of freedom and uncertainty awareness. Third, the prediction performance, probability reliability, concentration of variable importance, consistency of attention results, and generalizability of attention results to conventional statistics were assessed in the diabetic classification modeling setting. Fourth, the potential limitations of attention mechanisms were considered. Results: Prediction performance was very high for all models. Probability reliability was high in models with uncertainty awareness. Variable importance was concentrated in several variables when uncertainty awareness was not considered. The consistency of attention results was high when uncertainty awareness was considered. The generalizability of attention results to conventional statistics was poor regardless of the modeling approach. Conclusions: The attention mechanism is an attractive technique with potential to be very promising in the future. However, it may not yet be desirable to rely on this method to assess variable importance in clinical settings. Therefore, along with theoretical studies enhancing attention mechanisms, more empirical studies investigating potential limitations should be encouraged. %M 33325832 %R 10.2196/18418 %U http://www.jmir.org/2020/12/e18418/ %U https://doi.org/10.2196/18418 %U http://www.ncbi.nlm.nih.gov/pubmed/33325832 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 12 %P e18001 %T Development of Phenotyping Algorithms for the Identification of Organ Transplant Recipients: Cohort Study %A Wheless,Lee %A Baker,Laura %A Edwards,LaVar %A Anand,Nimay %A Birdwell,Kelly %A Hanlon,Allison %A Chren,Mary-Margaret %+ Department of Dermatology, Vanderbilt University Medical Center, 719 Thompson Lane, Suite 26300, Nashville, TN, 37204, United States, 1 6153226485, lee.e.wheless@vumc.org %K phenotyping %K electronic health record %K organ transplant recipients %D 2020 %7 10.12.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: Studies involving organ transplant recipients (OTRs) are often limited to the variables collected in the national Scientific Registry of Transplant Recipients database. Electronic health records contain additional variables that can augment this data source if OTRs can be identified accurately. Objective: The aim of this study was to develop phenotyping algorithms to identify OTRs from electronic health records. Methods: We used Vanderbilt’s deidentified version of its electronic health record database, which contains nearly 3 million subjects, to develop algorithms to identify OTRs. We identified all 19,817 individuals with at least one International Classification of Diseases (ICD) or Current Procedural Terminology (CPT) code for organ transplantation. We performed a chart review on 1350 randomly selected individuals to determine the transplant status. We constructed machine learning models to calculate positive predictive values and sensitivity for combinations of codes by using classification and regression trees, random forest, and extreme gradient boosting algorithms. Results: Of the 1350 reviewed patient charts, 827 were organ transplant recipients while 511 had no record of a transplant, and 12 were equivocal. Most patients with only 1 or 2 transplant codes did not have a transplant. The most common reasons for being labeled a nontransplant patient were the lack of data (229/511, 44.8%) or the patient being evaluated for an organ transplant (174/511, 34.1%). All 3 machine learning algorithms identified OTRs with overall >90% positive predictive value and >88% sensitivity. Conclusions: Electronic health records linked to biobanks are increasingly used to conduct large-scale studies but have not been well-utilized in organ transplantation research. We present rigorously evaluated methods for phenotyping OTRs from electronic health records that will enable the use of the full spectrum of clinical data in transplant research. Using several different machine learning algorithms, we were able to identify transplant cases with high accuracy by using only ICD and CPT codes. %M 33156808 %R 10.2196/18001 %U http://medinform.jmir.org/2020/12/e18001/ %U https://doi.org/10.2196/18001 %U http://www.ncbi.nlm.nih.gov/pubmed/33156808 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 12 %P e18526 %T Transformation of Pathology Reports Into the Common Data Model With Oncology Module: Use Case for Colon Cancer %A Ryu,Borim %A Yoon,Eunsil %A Kim,Seok %A Lee,Sejoon %A Baek,Hyunyoung %A Yi,Soyoung %A Na,Hee Young %A Kim,Ji-Won %A Baek,Rong-Min %A Hwang,Hee %A Yoo,Sooyoung %+ Office of eHealth Research and Business, Seoul National University Bundang Hospital, 82 173rd Street, Gumi-ro, Bundang-gu, Seongnam, 13620, Republic of Korea, 82 317878980, yoosoo0@gmail.com %K common data model %K natural language processing %K oncology module %K colon cancer %K electronic health record %K oncology %K pathology %K clinical data %D 2020 %7 9.12.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Common data models (CDMs) help standardize electronic health record data and facilitate outcome analysis for observational and longitudinal research. An analysis of pathology reports is required to establish fundamental information infrastructure for data-driven colon cancer research. The Observational Medical Outcomes Partnership (OMOP) CDM is used in distributed research networks for clinical data; however, it requires conversion of free text–based pathology reports into the CDM’s format. There are few use cases of representing cancer data in CDM. Objective: In this study, we aimed to construct a CDM database of colon cancer–related pathology with natural language processing (NLP) for a research platform that can utilize both clinical and omics data. The essential text entities from the pathology reports are extracted, standardized, and converted to the OMOP CDM format in order to utilize the pathology data in cancer research. Methods: We extracted clinical text entities, mapped them to the standard concepts in the Observational Health Data Sciences and Informatics vocabularies, and built databases and defined relations for the CDM tables. Major clinical entities were extracted through NLP on pathology reports of surgical specimens, immunohistochemical studies, and molecular studies of colon cancer patients at a tertiary general hospital in South Korea. Items were extracted from each report using regular expressions in Python. Unstructured data, such as text that does not have a pattern, were handled with expert advice by adding regular expression rules. Our own dictionary was used for normalization and standardization to deal with biomarker and gene names and other ungrammatical expressions. The extracted clinical and genetic information was mapped to the Logical Observation Identifiers Names and Codes databases and the Systematized Nomenclature of Medicine (SNOMED) standard terminologies recommended by the OMOP CDM. The database-table relationships were newly defined through SNOMED standard terminology concepts. The standardized data were inserted into the CDM tables. For evaluation, 100 reports were randomly selected and independently annotated by a medical informatics expert and a nurse. Results: We examined and standardized 1848 immunohistochemical study reports, 3890 molecular study reports, and 12,352 pathology reports of surgical specimens (from 2017 to 2018). The constructed and updated database contained the following extracted colorectal entities: (1) NOTE_NLP, (2) MEASUREMENT, (3) CONDITION_OCCURRENCE, (4) SPECIMEN, and (5) FACT_RELATIONSHIP of specimen with condition and measurement. Conclusions: This study aimed to prepare CDM data for a research platform to take advantage of all omics clinical and patient data at Seoul National University Bundang Hospital for colon cancer pathology. A more sophisticated preparation of the pathology data is needed for further research on cancer genomics, and various types of text narratives are the next target for additional research on the use of data in the CDM. %M 33295294 %R 10.2196/18526 %U https://www.jmir.org/2020/12/e18526 %U https://doi.org/10.2196/18526 %U http://www.ncbi.nlm.nih.gov/pubmed/33295294 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 12 %P e21750 %T Family History Information Extraction With Neural Attention and an Enhanced Relation-Side Scheme: Algorithm Development and Validation %A Dai,Hong-Jie %A Lee,You-Qian %A Nekkantti,Chandini %A Jonnagaddala,Jitendra %+ College of Electrical Engineering and Computer Science, Department of Electrical Engineering, National Kaohsiung University of Science and Technology, No. 415, Jiangong Road, Sanmin District, Kaohsiung City, 807, Taiwan, 886 7 3814526 ext 15510, hjdai@nkust.edu.tw %K family history information %K natural language processing %K deep learning %K electronic health record %D 2020 %7 1.12.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: Identifying and extracting family history information (FHI) from clinical reports are significant for recognizing disease susceptibility. However, FHI is usually described in a narrative manner within patients’ electronic health records, which requires the application of natural language processing technologies to automatically extract such information to provide more comprehensive patient-centered information to physicians. Objective: This study aimed to overcome the 2 main challenges observed in previous research focusing on FHI extraction. One is the requirement to develop postprocessing rules to infer the member and side information of family mentions. The other is to efficiently utilize intrasentence and intersentence information to assist FHI extraction. Methods: We formulated the task as a sequential labeling problem and propose an enhanced relation-side scheme that encodes the required family member properties to not only eliminate the need for postprocessing rules but also relieve the insufficient training instance issues. Moreover, an attention-based neural network structure was proposed to exploit cross-sentence information to identify FHI and its attributes requiring cross-sentence inference. Results: The dataset released by the 2019 n2c2/OHNLP family history extraction task was used to evaluate the performance of the proposed methods. We started by comparing the performance of the traditional neural sequence models with the ordinary scheme and enhanced scheme. Next, we studied the effectiveness of the proposed attention-enhanced neural networks by comparing their performance with that of the traditional networks. It was observed that, with the enhanced scheme, the recall of the neural network can be improved, leading to an increase in the F score of 0.024. The proposed neural attention mechanism enhanced both the recall and precision and resulted in an improved F score of 0.807, which was ranked fourth in the shared task. Conclusions: We presented an attention-based neural network along with an enhanced tag scheme that enables the neural network model to learn and interpret the implicit relationship and side information of the recognized family members across sentences without relying on heuristic rules. %M 33258777 %R 10.2196/21750 %U https://medinform.jmir.org/2020/12/e21750 %U https://doi.org/10.2196/21750 %U http://www.ncbi.nlm.nih.gov/pubmed/33258777 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 11 %P e17150 %T Influence of Scanner Precision and Analysis Software in Quantifying Three-Dimensional Intraoral Changes: Two-Factor Factorial Experimental Design %A O'Toole,Saoirse %A Bartlett,David %A Keeling,Andrew %A McBride,John %A Bernabe,Eduardo %A Crins,Luuk %A Loomans,Bas %+ Centre for Clinical, Oral and Translational Sciences, King’s College London, London, United Kingdom, 44 2071887462, saoirse.otoole@kcl.ac.uk %K diagnostic systems %K digital imaging/radiology %K engineering %K imaging %K outcomes research %K tooth wear %D 2020 %7 27.11.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Three-dimensional scans are increasingly used to quantify biological topographical changes and clinical health outcomes. Traditionally, the use of 3D scans has been limited to specialized centers owing to the high cost of the scanning equipment and the necessity for complex analysis software. Technological advances have made cheaper, more accessible methods of data capture and analysis available in the field of dentistry, potentially facilitating a primary care system to quantify disease progression. However, this system has yet to be compared with previous high-precision methods in university hospital settings. Objective: The aim of this study was to compare a dental primary care method of data capture (intraoral scanner) with a precision hospital-based method (laser profilometer) in addition to comparing open source and commercial software available for data analysis. Methods: Longitudinal dental wear data from 30 patients were analyzed using a two-factor factorial experimental design. Bimaxillary intraoral digital scans (TrueDefinition, 3M, UK) and conventional silicone impressions, poured in type-4 dental stone, were made at both baseline and follow-up appointments (mean 36 months, SD 10.9). Stone models were scanned using precision laser profilometry (Taicaan, Southampton, UK). Three-dimensional changes in both forms of digital scans of the first molars (n=76) were quantitatively analyzed using the engineering software Geomagic Control (3D Systems, Germany) and freeware WearCompare (Leeds Digital Dentistry, UK). Volume change (mm3) was the primary measurement outcome. The maximum point loss (μm) and the average profile loss (μm) were also recorded. Data were paired and skewed, and were therefore compared using Wilcoxon signed-rank tests with Bonferroni correction. Results: The median (IQR) volume change for Geomagic using profilometry and using the intraoral scan was –0.37 mm3 (–3.75-2.30) and +0.51 mm3 (–2.17-4.26), respectively (P<.001). Using WearCompare, the median (IQR) volume change for profilometry and intraoral scanning was –1.21 mm3 (–3.48-0.56) and –0.39 mm3 (–3.96-2.76), respectively (P=.04). WearCompare detected significantly greater volume loss than Geomagic regardless of scanner type. No differences were observed between groups with respect to the maximum point loss or average profile loss. Conclusions: As expected, the method of data capture, software used, and measurement metric all significantly influenced the measurement outcome. However, when appropriate analysis was used, the primary care system was able to quantify the degree of change and can be recommended depending on the accuracy needed to diagnose a condition. Lower-resolution scanners may underestimate complex changes when measuring at the micron level. %M 33245280 %R 10.2196/17150 %U https://www.jmir.org/2020/11/e17150 %U https://doi.org/10.2196/17150 %U http://www.ncbi.nlm.nih.gov/pubmed/33245280 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 11 %P e19761 %T Predicting Unplanned Readmissions Following a Hip or Knee Arthroplasty: Retrospective Observational Study %A Mohammadi,Ramin %A Jain,Sarthak %A Namin,Amir T %A Scholem Heller,Melissa %A Palacholla,Ramya %A Kamarthi,Sagar %A Wallace,Byron %+ Northeastern University, 2208, 177 Huntington Ave, Boston, MA, , United States, 1 6173732402, b.wallace@northeastern.edu %K deep learning %K natural language processing %K electronic health records %K auto ML %K 30-days readmission %K hip arthroplasty %K knee arthroplasty %D 2020 %7 27.11.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: Total joint replacements are high-volume and high-cost procedures that should be monitored for cost and quality control. Models that can identify patients at high risk of readmission might help reduce costs by suggesting who should be enrolled in preventive care programs. Previous models for risk prediction have relied on structured data of patients rather than clinical notes in electronic health records (EHRs). The former approach requires manual feature extraction by domain experts, which may limit the applicability of these models. Objective: This study aims to develop and evaluate a machine learning model for predicting the risk of 30-day readmission following knee and hip arthroplasty procedures. The input data for these models come from raw EHRs. We empirically demonstrate that unstructured free-text notes contain a reasonably predictive signal for this task. Methods: We performed a retrospective analysis of data from 7174 patients at Partners Healthcare collected between 2006 and 2016. These data were split into train, validation, and test sets. These data sets were used to build, validate, and test models to predict unplanned readmission within 30 days of hospital discharge. The proposed models made predictions on the basis of clinical notes, obviating the need for performing manual feature extraction by domain and machine learning experts. The notes that served as model inputs were written by physicians, nurses, pathologists, and others who diagnose and treat patients and may have their own predictions, even if these are not recorded. Results: The proposed models output readmission risk scores (propensities) for each patient. The best models (as selected on a development set) yielded an area under the receiver operating characteristic curve of 0.846 (95% CI 82.75-87.11) for hip and 0.822 (95% CI 80.94-86.22) for knee surgery, indicating reasonable discriminative ability. Conclusions: Machine learning models can predict which patients are at a high risk of readmission within 30 days following hip and knee arthroplasty procedures on the basis of notes in EHRs with reasonable discriminative power. Following further validation and empirical demonstration that the models realize predictive performance above that which clinical judgment may provide, such models may be used to build an automated decision support tool to help caretakers identify at-risk patients. %M 33245283 %R 10.2196/19761 %U https://medinform.jmir.org/2020/11/e19761 %U https://doi.org/10.2196/19761 %U http://www.ncbi.nlm.nih.gov/pubmed/33245283 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 11 %P e19597 %T Proposal and Assessment of a De-Identification Strategy to Enhance Anonymity of the Observational Medical Outcomes Partnership Common Data Model (OMOP-CDM) in a Public Cloud-Computing Environment: Anonymization of Medical Data Using Privacy Models %A Jeon,Seungho %A Seo,Jeongeun %A Kim,Sukyoung %A Lee,Jeongmoon %A Kim,Jong-Ho %A Sohn,Jang Wook %A Moon,Jongsub %A Joo,Hyung Joon %+ Department of Internal Medicine, Korea University College of Medicine, Korea University, 145 Anam-ro, Seongbuk-gu, Seoul, 02841, Republic of Korea, 82 10 3476 0525, drjoohj@gmail.com %K de-identification %K privacy %K anonymization %K common data model %K Observational Health Data Sciences and Informatics %D 2020 %7 26.11.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: De-identifying personal information is critical when using personal health data for secondary research. The Observational Medical Outcomes Partnership Common Data Model (CDM), defined by the nonprofit organization Observational Health Data Sciences and Informatics, has been gaining attention for its use in the analysis of patient-level clinical data obtained from various medical institutions. When analyzing such data in a public environment such as a cloud-computing system, an appropriate de-identification strategy is required to protect patient privacy. Objective: This study proposes and evaluates a de-identification strategy that is comprised of several rules along with privacy models such as k-anonymity, l-diversity, and t-closeness. The proposed strategy was evaluated using the actual CDM database. Methods: The CDM database used in this study was constructed by the Anam Hospital of Korea University. Analysis and evaluation were performed using the ARX anonymizing framework in combination with the k-anonymity, l-diversity, and t-closeness privacy models. Results: The CDM database, which was constructed according to the rules established by Observational Health Data Sciences and Informatics, exhibited a low risk of re-identification: The highest re-identifiable record rate (11.3%) in the dataset was exhibited by the DRUG_EXPOSURE table, with a re-identification success rate of 0.03%. However, because all tables include at least one “highest risk” value of 100%, suitable anonymizing techniques are required; moreover, the CDM database preserves the “source values” (raw data), a combination of which could increase the risk of re-identification. Therefore, this study proposes an enhanced strategy to de-identify the source values to significantly reduce not only the highest risk in the k-anonymity, l-diversity, and t-closeness privacy models but also the overall possibility of re-identification. Conclusions: Our proposed de-identification strategy effectively enhanced the privacy of the CDM database, thereby encouraging clinical research involving multiple centers. %M 33177037 %R 10.2196/19597 %U http://www.jmir.org/2020/11/e19597/ %U https://doi.org/10.2196/19597 %U http://www.ncbi.nlm.nih.gov/pubmed/33177037 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 11 %P e6924 %T Temporal Design Patterns for Digital Phenotype Cohort Selection in Critical Care: Systematic Literature Assessment and Qualitative Synthesis %A Capurro,Daniel %A Barbe,Mario %A Daza,Claudio %A Santa Maria,Josefa %A Trincado,Javier %+ School of Computing and Information Systems, Centre for Digital Transformation of Health, University of Melbourne, Room 3.24, Level 3, Doug McDonnel (Building 168), Parkville Campus, Melbourne, 3010, Australia, 61 8344 4504, dcapurro@unimelb.edu.au %K digital phenotyping %K clinical data %K temporal abstraction %D 2020 %7 24.11.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: Inclusion criteria for observational studies frequently contain temporal entities and relations. The use of digital phenotypes to create cohorts in electronic health record–based observational studies requires rich functionality to capture these temporal entities and relations. However, such functionality is not usually available or requires complex database queries and specialized expertise to build them. Objective: The purpose of this study is to systematically assess observational studies reported in critical care literature to capture design requirements and functionalities for a graphical temporal abstraction-based digital phenotyping tool. Methods: We iteratively extracted attributes describing patients, interventions, and clinical outcomes. We qualitatively synthesized studies, identifying all temporal and nontemporal entities and relations. Results: We extracted data from 28 primary studies and 367 temporal and nontemporal entities. We generated a synthesis of entities, relations, and design patterns. Conclusions: We report on the observed types of clinical temporal entities and their relations as well as design requirements for a temporal abstraction-based digital phenotyping system. The results can be used to inform the development of such a system. %M 33231554 %R 10.2196/medinform.6924 %U http://medinform.jmir.org/2020/11/e6924/ %U https://doi.org/10.2196/medinform.6924 %U http://www.ncbi.nlm.nih.gov/pubmed/33231554 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 11 %P e20050 %T An Environmental Scan of Sex and Gender in Electronic Health Records: Analysis of Public Information Sources %A Lau,Francis %A Antonio,Marcy %A Davison,Kelly %A Queen,Roz %A Bryski,Katie %+ School of Health Information Science, University of Victoria, P.O. Box 1700 STN CSC, University of Victoria, Victoria, BC, V8W2Y2, Canada, 1 2504725131, fylau@uvic.ca %K sex %K gender %K electronic health records %K standards %K transgender persons %D 2020 %7 11.11.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Historically, the terms sex and gender have been used interchangeably as a binary attribute to describe a person as male or female, even though there is growing recognition that sex and gender are distinct concepts. The lack of sex and gender delineation in electronic health records (EHRs) may be perpetuating the inequities experienced by the transgender and gender nonbinary (TGNB) populations. Objective: This study aims to conduct an environmental scan to understand how sex and gender are defined and implemented in existing Canadian EHRs and current international health information standards. Methods: We examined public information sources on sex and gender definitions in existing Canadian EHRs and international standards communities. Definitions refer to data element names, code systems, and value sets in the descriptions of EHRs and standards. The study was built on an earlier environment scan by Canada Health Infoway, supplemented with sex and gender definitions from international standards communities. For the analysis, we examined the definitions for clarity, consistency, and accuracy. We also received feedback from a virtual community interested in sex-gender EHR issues. Results: The information sources consisted of public website descriptions of 52 databases and 55 data standards from 12 Canadian entities and 10 standards communities. There are variations in the definition and implementation of sex and gender in Canadian EHRs and international health information standards. There is a lack of clarity in some sex and gender concepts. There is inconsistency in the data element names, code systems, and value sets used to represent sex and gender concepts across EHRs. The appropriateness and adequacy of some value options are questioned as our societal understanding of sexual health evolves. Outdated value options raise concerns about current EHRs supporting the provision of culturally competent, safe, and affirmative health care. The limited options also perpetuate the inequities faced by the TGNB populations. The expanded sex and gender definitions from leading Canadian organizations and international standards communities have brought challenges in how to migrate these definitions into existing EHRs. We proposed 6 high-level actions, which are to articulate the need for this work, reach consensus on sex and gender concepts, reach consensus on expanded sex and gender definitions in EHRs, develop a coordinated action plan, embrace EHR change from socio-organizational and technical aspects to ensure success, and demonstrate the benefits in tangible terms. Conclusions: There are variations in sex and gender concepts across Canadian EHRs and the health information standards that support them. Although there are efforts to modernize sex and gender concept definitions, we need decisive and coordinated actions to ensure clarity, consistency, and competency in the definition and implementation of sex and gender concepts in EHRs. This work has implications for addressing the inequities of TGNB populations in Canada. %M 33174858 %R 10.2196/20050 %U http://www.jmir.org/2020/11/e20050/ %U https://doi.org/10.2196/20050 %U http://www.ncbi.nlm.nih.gov/pubmed/33174858 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 11 %P e20765 %T Pharmacists’ Perceptions of the Benefits and Challenges of Electronic Product Information System Implementation in Hong Kong: Mixed-Method Study %A Fung,Eunice Wing To %A Au-Yeung,Gordon Tsz Fung %A Tsoi,Lo Mei %A Qu,Lili %A Cheng,Tommy Kwan Wa %A Chong,Donald Wing-Kit %A Lam,Teddy Tai Ning %A Cheung,Yin Ting %+ School of Pharmacy, Faculty of Medicine, The Chinese University of Hong Kong, 8th Floor, Lo Kwee-Seong Integrated Biomedical Sciences Building, Area 39, Shatin, New Territory, Hong Kong, China (Hong Kong), 852 39436833, yinting.cheung@cuhk.edu.hk %K electronic product information %K drug information system %K electronic health information %K health care professionals %K retrieval of health information %D 2020 %7 10.11.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: With the advancement of technology, more countries are now adopting the use of electronic product information (ePI), which refer to an electronic version of physical product inserts in a semistructured format optimized for electronic manipulation. The successful implementation of ePI has led to advantages and convenience to patients, health care professionals, and pharmaceutical companies in many regions and countries. In the Hong Kong Special Administrative Region (SAR), there is currently no citywide implementation of ePI. The SAR exhibits conditions that would favor the implementation of an ePI system, as well as existing barriers hindering its implementation. However, no study has been performed to examine the specific situation in Hong Kong. Objective: The objective of this study is to explore working pharmacists’ overall perception of ePI and to identify potential challenges to the implementation of an ePI system in Hong Kong. Methods: This mixed-method study involved a structured survey and interview with practicing pharmacists in Hong Kong. Pharmacists were eligible if they were licensed to practice in Hong Kong, and currently working locally in any pharmacy-related sectors and institutions. Respondents completed a survey to indicate their level of agreement with statements regarding the potential advantages of ePI over paper PI. A structured interview was conducted to gather respondents’ perceived advantages of ePI over paper PI in different aspects, such as professionalism, usability, presentation, and environment, as well as challenges of citywide ePI implementation in Hong Kong. Thematic analysis was adopted to analyze the qualitative data. Grounded theory was used to generate themes and identify specific outcomes. Results: A total of 16 pharmacists were recruited, comprising 4 community pharmacists, 5 hospital pharmacists, and 7 industrial pharmacists. All of them used electronic platforms at least once per month on average. Respondents identified many flaws in physical package inserts that can potentially be mitigated using ePI. The speed with which drug information can be retrieved and the degree to which the drug information can be readily updated and disseminated were considered the greatest strengths of ePI. The clarity with which ePI present drug information to patients was considered as the weakest aspect of ePI. Many respondents highlighted concerns about the security risks and high cost associated with system maintenance and that certain subpopulations may not be sufficiently computer literate to navigate the ePI system. Respondents also voiced many concerns about the implementation and maintenance of a local ePI system. Conclusions: We conclude that an ePI system is generally supported by pharmacists but concerns about implementation process and maintenance of the system has been raised. The perceived benefits of ePI gathered from this study, as well as collective evidence from other countries with mature ePI systems, confirm that more efforts should be made to promote optimized development and implementation of an ePI system in Hong Kong. %M 33170130 %R 10.2196/20765 %U http://www.jmir.org/2020/11/e20765/ %U https://doi.org/10.2196/20765 %U http://www.ncbi.nlm.nih.gov/pubmed/33170130 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 11 %P e23361 %T openEHR Archetype Use and Reuse Within Multilingual Clinical Data Sets: Case Study %A Leslie,Heather %+ Atomica Informatics, Fitzroy, Australia, 61 418966670, heather.leslie@atomicainformatics.com %K openEHR %K archetype %K template %K reuse %K clinical informatics %K COVID-19 %K standard %K crowd sourced %K data set %K data quality %K multilingual %K EHR %K electronic health record %K SARS-CoV-2 %D 2020 %7 2.11.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Despite electronic health records being in existence for over 50 years, our ability to exchange health data remains frustratingly limited. Commonly used clinical content standards, and the information models that underpin them, are primarily related to health data exchange, and so are usually document- or message-focused. In contrast, over the past 12 years, the Clinical Models program at openEHR International has gradually established a governed, coordinated, and coherent ecosystem of clinical information models, known as openEHR archetypes. Each archetype is designed as a maximal data set for a universal use-case, intended for reuse across various health data sets, known as openEHR templates. To date, only anecdotal evidence has been available to indicate if the hypothesis of archetype reuse across templates is feasible and scalable. As a response to the COVID-19 pandemic, between February and July 2020, 7 openEHR templates were independently created to represent COVID-19–related data sets for symptom screening, confirmed infection reporting, clinical decision support, and research. Each of the templates prioritized reuse of existing use-case agnostic archetypes found in openEHR International's online Clinical Knowledge Manager tool as much as possible. This study is the first opportunity to investigate archetype reuse within a range of diverse, multilingual openEHR templates. Objective: This study aims to investigate the use and reuse of openEHR archetypes across the 7 openEHR templates as an initial investigation about the reuse of information models across data sets used for a variety of clinical purposes. Methods: Analysis of both the number of occurrences of archetypes and patterns of occurrence within 7 discrete templates was carried out at the archetype or clinical concept level. Results: Across all 7 templates collectively, 203 instances of 58 unique archetypes were used. The most frequently used archetype occurred 24 times across 4 of the 7 templates. Total data points per template ranged from 40 to 179. Archetype instances per template ranged from 10 to 62. Unique archetype occurrences ranged from 10 to 28. Existing archetype reuse of use-case agnostic archetypes ranged from 40% to 90%. Total reuse of use-case agnostic archetypes ranged from 40% to 100%. Conclusions: Investigation of the amount of archetype reuse across the 7 openEHR templates in this initial study has demonstrated significant reuse of archetypes, even across unanticipated, novel modeling challenges and multilingual deployments. While the trigger for the development of each of these templates was the COVID-19 pandemic, the templates represented a variety of types of data sets: symptom screening, infection report, clinical decision support for diagnosis and treatment, and secondary use or research. The findings support the openEHR hypothesis that it is possible to create a shared, public library of standards-based, vendor-neutral clinical information models that can be reused across a diverse range of health data sets. %M 33035176 %R 10.2196/23361 %U https://www.jmir.org/2020/11/e23361 %U https://doi.org/10.2196/23361 %U http://www.ncbi.nlm.nih.gov/pubmed/33035176 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 10 %P e19810 %T Clinical Context–Aware Biomedical Text Summarization Using Deep Neural Network: Model Development and Validation %A Afzal,Muhammad %A Alam,Fakhare %A Malik,Khalid Mahmood %A Malik,Ghaus M %+ Department of Computer Science & Engineering, School of Engineering and Computer Science, Oakland University, Engineering Center 501, 115 Library Drive, Rochester, MI, 48309, United States, 1 2483703542, mahmood@oakland.edu %K biomedical informatics %K automatic text summarization %K deep neural network %K word embedding %K semantic similarity %K brain aneurysm %D 2020 %7 23.10.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Automatic text summarization (ATS) enables users to retrieve meaningful evidence from big data of biomedical repositories to make complex clinical decisions. Deep neural and recurrent networks outperform traditional machine-learning techniques in areas of natural language processing and computer vision; however, they are yet to be explored in the ATS domain, particularly for medical text summarization. Objective: Traditional approaches in ATS for biomedical text suffer from fundamental issues such as an inability to capture clinical context, quality of evidence, and purpose-driven selection of passages for the summary. We aimed to circumvent these limitations through achieving precise, succinct, and coherent information extraction from credible published biomedical resources, and to construct a simplified summary containing the most informative content that can offer a review particular to clinical needs. Methods: In our proposed approach, we introduce a novel framework, termed Biomed-Summarizer, that provides quality-aware Patient/Problem, Intervention, Comparison, and Outcome (PICO)-based intelligent and context-enabled summarization of biomedical text. Biomed-Summarizer integrates the prognosis quality recognition model with a clinical context–aware model to locate text sequences in the body of a biomedical article for use in the final summary. First, we developed a deep neural network binary classifier for quality recognition to acquire scientifically sound studies and filter out others. Second, we developed a bidirectional long-short term memory recurrent neural network as a clinical context–aware classifier, which was trained on semantically enriched features generated using a word-embedding tokenizer for identification of meaningful sentences representing PICO text sequences. Third, we calculated the similarity between query and PICO text sequences using Jaccard similarity with semantic enrichments, where the semantic enrichments are obtained using medical ontologies. Last, we generated a representative summary from the high-scoring PICO sequences aggregated by study type, publication credibility, and freshness score. Results: Evaluation of the prognosis quality recognition model using a large dataset of biomedical literature related to intracranial aneurysm showed an accuracy of 95.41% (2562/2686) in terms of recognizing quality articles. The clinical context–aware multiclass classifier outperformed the traditional machine-learning algorithms, including support vector machine, gradient boosted tree, linear regression, K-nearest neighbor, and naïve Bayes, by achieving 93% (16127/17341) accuracy for classifying five categories: aim, population, intervention, results, and outcome. The semantic similarity algorithm achieved a significant Pearson correlation coefficient of 0.61 (0-1 scale) on a well-known BIOSSES dataset (with 100 pair sentences) after semantic enrichment, representing an improvement of 8.9% over baseline Jaccard similarity. Finally, we found a highly positive correlation among the evaluations performed by three domain experts concerning different metrics, suggesting that the automated summarization is satisfactory. Conclusions: By employing the proposed method Biomed-Summarizer, high accuracy in ATS was achieved, enabling seamless curation of research evidence from the biomedical literature to use for clinical decision-making. %M 33095174 %R 10.2196/19810 %U http://www.jmir.org/2020/10/e19810/ %U https://doi.org/10.2196/19810 %U http://www.ncbi.nlm.nih.gov/pubmed/33095174 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 10 %P e23049 %T Assessment of Myosteatosis on Computed Tomography by Automatic Generation of a Muscle Quality Map Using a Web-Based Toolkit: Feasibility Study %A Kim,Dong Wook %A Kim,Kyung Won %A Ko,Yousun %A Park,Taeyong %A Khang,Seungwoo %A Jeong,Heeryeol %A Koo,Kyoyeong %A Lee,Jeongjin %A Kim,Hong-Kyu %A Ha,Jiyeon %A Sung,Yu Sub %A Shin,Youngbin %+ Department of Radiology and Research Institute of Radiology, Asan Medical Center, 88 Olympic-ro, 43-gil, Songpa-gu, Seoul, 05505, Republic of Korea, 82 2 3010 4377, medimash@gmail.com %K body composition %K muscle %K skeletal %K sarcopenia %K computed tomography %K x-ray %K scan %K web-based tool %K feasibility %K automated %K CT %D 2020 %7 19.10.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: Muscle quality is associated with fatty degeneration or infiltration of the muscle, which may be associated with decreased muscle function and increased disability. Objective: The aim of this study is to evaluate the feasibility of automated quantitative measurements of the skeletal muscle on computed tomography (CT) images to assess normal-attenuation muscle and myosteatosis. Methods: We developed a web-based toolkit to generate a muscle quality map by categorizing muscle components. First, automatic segmentation of the total abdominal muscle area (TAMA), visceral fat area, and subcutaneous fat area was performed using a predeveloped deep learning model on a single axial CT image at the L3 vertebral level. Second, the Hounsfield unit of each pixel in the TAMA was measured and categorized into 3 components: normal-attenuation muscle area (NAMA), low-attenuation muscle area (LAMA), and inter/intramuscular adipose tissue (IMAT) area. The myosteatosis area was derived by adding the LAMA and IMAT area. We tested the feasibility of the toolkit using randomly selected healthy participants, comprising 6 different age groups (20 to 79 years). With stratification by sex, these indices were compared between age groups using 1-way analysis of variance (ANOVA). Correlations between the myosteatosis area or muscle densities and fat areas were analyzed using Pearson correlation coefficient r. Results: A total of 240 healthy participants (135 men and 105 women) with 40 participants per age group were included in the study. In the 1-way ANOVA, the NAMA, LAMA, and IMAT were significantly different between the age groups in both male and female participants (P≤.004), whereas the TAMA showed a significant difference only in male participants (male, P<.001; female, P=.88). The myosteatosis area had a strong negative correlation with muscle densities (r=–0.833 to –0.894), a moderate positive correlation with visceral fat areas (r=0.607 to 0.669), and a weak positive correlation with the subcutaneous fat areas (r=0.305 to 0.441). Conclusions: The automated web-based toolkit is feasible and enables quantitative CT assessment of myosteatosis, which can be a potential quantitative biomarker for evaluating structural and functional changes brought on by aging in the skeletal muscle. %M 33074159 %R 10.2196/23049 %U http://medinform.jmir.org/2020/10/e23049/ %U https://doi.org/10.2196/23049 %U http://www.ncbi.nlm.nih.gov/pubmed/33074159 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 10 %P e17962 %T Fueling Clinical and Translational Research in Appalachia: Informatics Platform Approach %A Cecchetti,Alfred A %A Bhardwaj,Niharika %A Murughiyan,Usha %A Kothakapu,Gouthami %A Sundaram,Uma %+ Department of Clinical and Translational Science, Joan C Edwards School of Medicine, Marshall University, 1600 Medical Center Drive, Huntington, WV, 25701, United States, 1 304 691 1585, cecchetti@marshall.edu %K Appalachian region %K medical informatics %K health care disparities %K electronic health records %K data warehousing %K data mining %K data visualization %K machine learning %K data science %D 2020 %7 14.10.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: The Appalachian population is distinct, not just culturally and geographically but also in its health care needs, facing the most health care disparities in the United States. To meet these unique demands, Appalachian medical centers need an arsenal of analytics and data science tools with the foundation of a centralized data warehouse to transform health care data into actionable clinical interventions. However, this is an especially challenging task given the fragmented state of medical data within Appalachia and the need for integration of other types of data such as environmental, social, and economic with medical data. Objective: This paper aims to present the structure and process of the development of an integrated platform at a midlevel Appalachian academic medical center along with its initial uses. Methods: The Appalachian Informatics Platform was developed by the Appalachian Clinical and Translational Science Institute’s Division of Clinical Informatics and consists of 4 major components: a centralized clinical data warehouse, modeling (statistical and machine learning), visualization, and model evaluation. Data from different clinical systems, billing systems, and state- or national-level data sets were integrated into a centralized data warehouse. The platform supports research efforts by enabling curation and analysis of data using the different components, as appropriate. Results: The Appalachian Informatics Platform is functional and has supported several research efforts since its implementation for a variety of purposes, such as increasing knowledge of the pathophysiology of diseases, risk identification, risk prediction, and health care resource utilization research and estimation of the economic impact of diseases. Conclusions: The platform provides an inexpensive yet seamless way to translate clinical and translational research ideas into clinical applications for regions similar to Appalachia that have limited resources and a largely rural population. %M 33052114 %R 10.2196/17962 %U http://medinform.jmir.org/2020/10/e17962/ %U https://doi.org/10.2196/17962 %U http://www.ncbi.nlm.nih.gov/pubmed/33052114 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 9 %P e20645 %T Marrying Medical Domain Knowledge With Deep Learning on Electronic Health Records: A Deep Visual Analytics Approach %A Li,Rui %A Yin,Changchang %A Yang,Samuel %A Qian,Buyue %A Zhang,Ping %+ The Ohio State University, Lincoln Tower 310A, 1800 Cannon Drive, Columbus, OH, 43210, United States, 1 614 293 9286, zhang.10631@osu.edu %K electronic health records %K interpretable deep learning %K knowledge graph %K visual analytics %D 2020 %7 28.9.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Deep learning models have attracted significant interest from health care researchers during the last few decades. There have been many studies that apply deep learning to medical applications and achieve promising results. However, there are three limitations to the existing models: (1) most clinicians are unable to interpret the results from the existing models, (2) existing models cannot incorporate complicated medical domain knowledge (eg, a disease causes another disease), and (3) most existing models lack visual exploration and interaction. Both the electronic health record (EHR) data set and the deep model results are complex and abstract, which impedes clinicians from exploring and communicating with the model directly. Objective: The objective of this study is to develop an interpretable and accurate risk prediction model as well as an interactive clinical prediction system to support EHR data exploration, knowledge graph demonstration, and model interpretation. Methods: A domain-knowledge–guided recurrent neural network (DG-RNN) model is proposed to predict clinical risks. The model takes medical event sequences as input and incorporates medical domain knowledge by attending to a subgraph of the whole medical knowledge graph. A global pooling operation and a fully connected layer are used to output the clinical outcomes. The middle results and the parameters of the fully connected layer are helpful in identifying which medical events cause clinical risks. DG-Viz is also designed to support EHR data exploration, knowledge graph demonstration, and model interpretation. Results: We conducted both risk prediction experiments and a case study on a real-world data set. A total of 554 patients with heart failure and 1662 control patients without heart failure were selected from the data set. The experimental results show that the proposed DG-RNN outperforms the state-of-the-art approaches by approximately 1.5%. The case study demonstrates how our medical physician collaborator can effectively explore the data and interpret the prediction results using DG-Viz. Conclusions: In this study, we present DG-Viz, an interactive clinical prediction system, which brings together the power of deep learning (ie, a DG-RNN–based model) and visual analytics to predict clinical risks and visually interpret the EHR prediction results. Experimental results and a case study on heart failure risk prediction tasks demonstrate the effectiveness and usefulness of the DG-Viz system. This study will pave the way for interactive, interpretable, and accurate clinical risk predictions. %M 32985996 %R 10.2196/20645 %U http://www.jmir.org/2020/9/e20645/ %U https://doi.org/10.2196/20645 %U http://www.ncbi.nlm.nih.gov/pubmed/32985996 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 9 %P e23565 %T Clinical Mortality in a Large COVID-19 Cohort: Observational Study %A Jarrett,Mark %A Schultz,Susanne %A Lyall,Julie %A Wang,Jason %A Stier,Lori %A De Geronimo,Marcella %A Nelson,Karen %+ Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hofstra University, 500 Hofstra University, Hempstead, NY, 11549, United States, 1 5163216044, MJarrett@northwell.edu %K COVID-19 %K mortality %K respiratory failure %K hypoxemia %K observational %K review %K cohort %K ICU %K intensive care unit %D 2020 %7 25.9.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Northwell Health, an integrated health system in New York, has treated more than 15,000 inpatients with COVID-19 at the US epicenter of the SARS-CoV-2 pandemic. Objective: We describe the demographic characteristics of patients who died of COVID-19, observation of frequent rapid response team/cardiac arrest (RRT/CA) calls for non–intensive care unit (ICU) patients, and factors that contributed to RRT/CA calls. Methods: A team of registered nurses reviewed the medical records of inpatients who tested positive for SARS-CoV-2 via polymerase chain reaction before or on admission and who died between March 13 (first Northwell Health inpatient expiration) and April 30, 2020, at 15 Northwell Health hospitals. The findings for these patients were abstracted into a database and statistically analyzed. Results: Of 2634 patients who died of COVID-19, 1478 (56.1%) had oxygen saturation levels ≥90% on presentation and required no respiratory support. At least one RRT/CA was called on 1112/2634 patients (42.2%) at a non-ICU level of care. Before the RRT/CA call, the most recent oxygen saturation levels for 852/1112 (76.6%) of these non-ICU patients were at least 90%. At the time the RRT/CA was called, 479/1112 patients (43.1%) had an oxygen saturation of <80%. Conclusions: This study represents one of the largest reviewed cohorts of mortality that also captures data in nonstructured fields. Approximately 50% of deaths occurred at a non-ICU level of care despite admission to the appropriate care setting with normal staffing. The data imply a sudden, unexpected deterioration in respiratory status requiring RRT/CA in a large number of non-ICU patients. Patients admitted at a non-ICU level of care suffered rapid clinical deterioration, often with a sudden decrease in oxygen saturation. These patients could benefit from additional monitoring (eg, continuous central oxygenation saturation), although this approach warrants further study. %M 32930099 %R 10.2196/23565 %U http://www.jmir.org/2020/9/e23565/ %U https://doi.org/10.2196/23565 %U http://www.ncbi.nlm.nih.gov/pubmed/32930099 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 9 %P e17978 %T Understanding the Information Needs and Context of Trauma Handoffs to Design Automated Sensing Clinical Documentation Technologies: Qualitative Mixed-Method Study of Military and Civilian Cases %A Novak,Laurie Lovett %A Simpson,Christopher L %A Coco,Joseph %A McNaughton,Candace D %A Ehrenfeld,Jesse M %A Bloos,Sean M %A Fabbri,Daniel %+ Department of Biomedical Informatics, Vanderbilt University Medical Center, 2525 West End Avenue, Suite 1475, Nashville, TN, 37203, United States, 1 615 936 6497, laurie.l.novak@vumc.org %K trauma handoffs %K military field medicine %K documentation %K trauma %K health records %K hospital %K emergency %D 2020 %7 25.9.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Current methods of communication between the point of injury and receiving medical facilities rely on verbal communication, supported by brief notes and the memory of the field medic. This communication can be made more complete and reliable with technologies that automatically document the actions of field medics. However, designing state-of-the-art technology for military field personnel and civilian first responders is challenging due to the barriers researchers face in accessing the environment and understanding situated actions and cognitive models employed in the field. Objective: To identify design insights for an automated sensing clinical documentation (ASCD) system, we sought to understand what information is transferred in trauma cases between prehospital and hospital personnel, and what contextual factors influence the collection, management, and handover of information in trauma cases, in both military and civilian cases. Methods: Using a multi-method approach including video review and focus groups, we developed an understanding of the information needs of trauma handoffs and the context of field documentation to inform the design of an automated sensing documentation system that uses wearables, cameras, and environmental sensors to passively infer clinical activity and automatically produce documentation. Results: Comparing military and civilian trauma documentation and handoff, we found similarities in the types of data collected and the prioritization of information. We found that military environments involved many more contextual factors that have implications for design, such as the physical environment (eg, heat, lack of lighting, lack of power) and the potential for active combat and triage, creating additional complexity. Conclusions: An ineffectiveness of communication is evident in both the civilian and military worlds. We used multiple methods of inquiry to study the information needs of trauma care and handoff, and the context of medical work in the field. Our findings informed the design and evaluation of an automated documentation tool. The data illustrated the need for more accurate recordkeeping, specifically temporal aspects, during transportation, and characterized the environment in which field testing of the developed tool will take place. The employment of a systems perspective in this project produced design insights that our team would not have identified otherwise. These insights created exciting and interesting challenges for the technical team to resolve. %M 32975522 %R 10.2196/17978 %U http://www.jmir.org/2020/9/e17978/ %U https://doi.org/10.2196/17978 %U http://www.ncbi.nlm.nih.gov/pubmed/32975522 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 9 %P e16224 %T Web-Based Technology for Remote Viewing of Radiological Images: App Validation %A Min,Qiusha %A Wang,Xin %A Huang,Bo %A Xu,Liangzhou %+ School of Educational Information Technology, Central China Normal University, No.152 Luoyu Road, Wuhan, Hubei, 430079, China, 86 27 67867597, qiusham@mail.ccnu.edu.cn %K internet access %K medical informatics applications %K computer-assisted image analyses %K computer-assisted three-dimensional imaging %K medical imaging %K radiology %K application %D 2020 %7 25.9.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Internet technologies can create advanced and rich web-based apps that allow radiologists to easily access teleradiology systems and remotely view medical images. However, each technology has its own drawbacks. It is difficult to balance the advantages and disadvantages of these internet technologies and identify an optimal solution for the development of medical imaging apps. Objective: This study aimed to compare different internet platform technologies for remotely viewing radiological images and analyze their advantages and disadvantages. Methods: Oracle Java, Adobe Flash, and HTML5 were each used to develop a comprehensive web-based medical imaging app that connected to a medical image server and provided several required functions for radiological interpretation (eg, navigation, magnification, windowing, and fly-through). Java-, Flash-, and HTML5-based medical imaging apps were tested on different operating systems over a local area network and a wide area network. Three computed tomography colonography data sets and 2 ordinary personal computers were used in the experiment. Results: The experimental results demonstrated that Java-, Flash-, and HTML5-based apps had the ability to provide real-time 2D functions. However, for 3D, performances differed between the 3 apps. The Java-based app had the highest frame rate of volume rendering. However, it required the longest time for surface rendering and failed to run surface rendering in macOS. The HTML5-based app had the fastest surface rendering and the highest speed for fly-through without platform dependence. Volume rendering, surface rendering, and fly-through performances of the Flash-based app were significantly worse than those of the other 2 apps. Conclusions: Oracle Java, Adobe Flash, and HTML5 have individual strengths in the development of remote access medical imaging apps. However, HTML5 is a promising technology for remote viewing of radiological images and can provide excellent performance without requiring any plug-ins. %M 32975520 %R 10.2196/16224 %U http://www.jmir.org/2020/9/e16224/ %U https://doi.org/10.2196/16224 %U http://www.ncbi.nlm.nih.gov/pubmed/32975520 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 8 %P e18542 %T Model-Based Algorithms for Detecting Peripheral Artery Disease Using Administrative Data From an Electronic Health Record Data System: Algorithm Development Study %A Weissler,Elizabeth Hope %A Lippmann,Steven J %A Smerek,Michelle M %A Ward,Rachael A %A Kansal,Aman %A Brock,Adam %A Sullivan,Robert C %A Long,Chandler %A Patel,Manesh R %A Greiner,Melissa A %A Hardy,N Chantelle %A Curtis,Lesley H %A Jones,W Schuyler %+ Department of Medicine, Duke University School of Medicine, DUMC Box 3330, Durham, NC, 27710, United States, 1 919 668 8917, schuyler.jones@duke.edu %K peripheral artery disease %K patient selection %K electronic health records %K cardiology %K health data %D 2020 %7 19.8.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: Peripheral artery disease (PAD) affects 8 to 10 million Americans, who face significantly elevated risks of both mortality and major limb events such as amputation. Unfortunately, PAD is relatively underdiagnosed, undertreated, and underresearched, leading to wide variations in treatment patterns and outcomes. Efforts to improve PAD care and outcomes have been hampered by persistent difficulties identifying patients with PAD for clinical and investigatory purposes. Objective: The aim of this study is to develop and validate a model-based algorithm to detect patients with peripheral artery disease (PAD) using data from an electronic health record (EHR) system. Methods: An initial query of the EHR in a large health system identified all patients with PAD-related diagnosis codes for any encounter during the study period. Clinical adjudication of PAD diagnosis was performed by chart review on a random subgroup. A binary logistic regression to predict PAD was built and validated using a least absolute shrinkage and selection operator (LASSO) approach in the adjudicated patients. The algorithm was then applied to the nonsampled records to further evaluate its performance. Results: The initial EHR data query using 406 diagnostic codes yielded 15,406 patients. Overall, 2500 patients were randomly selected for ground truth PAD status adjudication. In the end, 108 code flags remained after removing rarely- and never-used codes. We entered these code flags plus administrative encounter, imaging, procedure, and specialist flags into a LASSO model. The area under the curve for this model was 0.862. Conclusions: The algorithm we constructed has two main advantages over other approaches to the identification of patients with PAD. First, it was derived from a broad population of patients with many different PAD manifestations and treatment pathways across a large health system. Second, our model does not rely on clinical notes and can be applied in situations in which only administrative billing data (eg, large administrative data sets) are available. A combination of diagnosis codes and administrative flags can accurately identify patients with PAD in large cohorts. %M 32663152 %R 10.2196/18542 %U http://medinform.jmir.org/2020/8/e18542/ %U https://doi.org/10.2196/18542 %U http://www.ncbi.nlm.nih.gov/pubmed/32663152 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 8 %P e17022 %T Technological Capabilities to Assess Digital Excellence in Hospitals in High Performing Health Care Systems: International eDelphi Exercise %A Krasuska,Marta %A Williams,Robin %A Sheikh,Aziz %A Franklin,Bryony Dean %A Heeney,Catherine %A Lane,Wendy %A Mozaffar,Hajar %A Mason,Kathy %A Eason,Sally %A Hinder,Susan %A Dunscombe,Rachel %A Potts,Henry W W %A Cresswell,Kathrin %+ Usher Institute, University of Edinburgh, Old Medical School, Teviot Place, Edinburgh, EH8 9AG, United Kingdom, 44 131 651 7878, Kathrin.Cresswell@ed.ac.uk %K digital excellence %K digital maturity %K Delphi technique %K hospitals, eHealth %D 2020 %7 18.8.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Hospitals worldwide are developing ambitious digital transformation programs as part of broader efforts to create digitally advanced health care systems. However, there is as yet no consensus on how best to characterize and assess digital excellence in hospitals. Objective: Our aim was to develop an international agreement on a defined set of technological capabilities to assess digital excellence in hospitals. Methods: We conducted a two-stage international modified electronic Delphi (eDelphi) consensus-building exercise, which included a qualitative analysis of free-text responses. In total, 31 international health informatics experts participated, representing clinical, academic, public, and vendor organizations. Results: We identified 35 technological capabilities that indicate digital excellence in hospitals. These are divided into two categories: (a) capabilities within a hospital (n=20) and (b) capabilities enabling communication with other parts of the health and social care system, and with patients and carers (n=15). The analysis of free-text responses pointed to the importance of nontechnological aspects of digitally enabled change, including social and organizational factors. Examples included an institutional culture characterized by a willingness to transform established ways of working and openness to risk-taking. The availability of a range of skills within digitization teams, including technological, project management and business expertise, and availability of resources to support hospital staff, were also highlighted. Conclusions: We have identified a set of criteria for assessing digital excellence in hospitals. Our findings highlight the need to broaden the focus from technical functionalities to wider digital transformation capabilities. %M 32808938 %R 10.2196/17022 %U https://www.jmir.org/2020/8/e17022 %U https://doi.org/10.2196/17022 %U http://www.ncbi.nlm.nih.gov/pubmed/32808938 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 8 %P e15630 %T Identification of Factors Influencing the Adoption of Health Information Technology by Nurses Who Are Digitally Lagging: In-Depth Interview Study %A De Leeuw,Jacqueline A %A Woltjer,Hetty %A Kool,Rudolf B %+ Department of Information Management, Radboud University Medical Center, PO Box 9100, Nijmegen, 6500HB, Netherlands, 31 643914595, jacqueline.deleeuw@radboudumc.nl %K qualitative research %K semi-structured interview %K purposive sampling %K health information systems %K computer user training %K professional education %K professional competence %K registered nurses %K nursing informatics %D 2020 %7 14.8.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: The introduction of health information technology (HIT) has drastically changed health care organizations and the way health care professionals work. Some health care professionals have trouble coping efficiently with the demands of HIT and the personal and professional changes it requires. Lagging in digital knowledge and skills hampers health care professionals from adhering to professional standards regarding the use of HIT and may cause professional performance problems, especially in the older professional population. It is important to gain more insight into the reasons and motivations behind the technology issues experienced by these professionals, as well as to explore what could be done to solve them. Objective: Our primary research objective was to identify factors that influence the adoption of HIT in a sample of nurses who describe themselves as digitally lagging behind the majority of their colleagues in their workplaces. Furthermore, we aimed to formulate recommendations for practice and leadership on how to help and guide these nurses through ongoing digital transformations in their health care work settings. Methods: In a Dutch university medical center, 10 face-to-face semi-structured interviews were performed with registered nurses (RN). Ammenwerth’s FITT-framework (fit between the Individual, Task, and Technology) was used to guide the interview topic list and to formulate themes to explore. Thematic analysis was used to analyze the interview data. The FITT-framework was also used to further interpret and clarify the interview findings. Results: Analyses of the interview data uncovered 5 main categories and 12 subthemes. The main categories were: (1) experience with digital working, (2) perception and meaning, (3) barriers, (4) facilitators, and (5) future perspectives. All participants used electronic devices and digital systems, including the electronic health record. The latter was experienced by some as user-unfriendly, time-consuming, and not supportive in daily professional practice. Most of the interviewees described digital working as “no fun at all,” “working in a fake world,” “stressful,” and “annoying.” There was a lack of general digital knowledge and little or no formal basic digital training or education. A negative attitude toward computer use and a lack of digital skills contributed to feelings of increased incompetency and postponement or avoidance of the use of HIT, both privately and professionally. Learning conditions of digital training and education did not meet personal learning needs and learning styles. A positive impact was seen in the work environment when colleagues and nurse managers were aware and sensitive to the difficulties participants experienced in developing digital skills, and when there was continuous training on the job and peer support from digitally savvy colleagues. The availability of a digital play environment combined with learning on the job and support of knowledgeable peers was experienced as helpful and motivating by participants. Conclusions: Nurses who are digitally lagging often have had insufficient and ineffective digital education. This leads to stress, frustration, feelings of incompetency, and postponement or avoidance of HIT use. A digital training approach tailored to the learning needs and styles of these nurses is needed, as well as an on-the-job training structure and adequate peer support. Hospital management and nurse leadership should be informed about the importance of the fit between technology, task, and the individual for adequate adoption of HIT. %M 32663142 %R 10.2196/15630 %U http://www.jmir.org/2020/8/e15630/ %U https://doi.org/10.2196/15630 %U http://www.ncbi.nlm.nih.gov/pubmed/32663142 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 8 %P e18078 %T Cultural Impact on the Intention to Use Nursing Information Systems of Nurses in Taiwan and China: Survey and Analysis %A Chang,I-Chiu %A Lin,Po-Jin %A Chen,Ting-Hung %A Chang,Chia-Hui %+ Department of Nursing, Taichung Veterans General Hospital, 1650 Taiwan Boulevard Sect 4, Taichung, 407204, Taiwan, 886 4 2359 2525, cjhsnd@gmail.com %K Nursing information system %K intention to use %K cultural differences %K information literacy %D 2020 %7 12.8.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Nursing workforce shortage has emerged as a global problem. Foreign nurse importation is a popular strategy to address the shortage. The interactions between nursing staff on either side of the Taiwan Strait continue to increase. Since both nurses in Taiwan and nurses in China have adopted nursing information systems to improve health care processes and quality, it is necessary to investigate factors influencing nursing information system usage in nursing practice. Objective: This study examined the effects of cultural and other related factors on nurses’ intentions to use nursing information systems. The findings were expected to serve as an empirical base for further benchmarking and management of cross-strait nurses. Methods: This survey was conducted in two case hospitals (one in Taiwan and one in China). A total of 880 questionnaires were distributed (n=440 in each hospital). Results: The results showed effort expectancy had a significant effect on the intention to use nursing information systems of nurses in China (P=.003) but not nurses in Taiwan (P=.16). Conclusions: Findings suggest nursing managers should adopt different strategies to motivate cross-strait nurses to use nursing information systems. Promoting effort expectancy is more likely to motivate nurses in China than in Taiwan. This discrepancy is probably due to the less hierarchical and more feminine society in Taiwan. %M 32784174 %R 10.2196/18078 %U https://www.jmir.org/2020/8/e18078 %U https://doi.org/10.2196/18078 %U http://www.ncbi.nlm.nih.gov/pubmed/32784174 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 8 %P e16903 %T Machine Learning Model for Risk Prediction of Community-Acquired Acute Kidney Injury Hospitalization From Electronic Health Records: Development and Validation Study %A Hsu,Chien-Ning %A Liu,Chien-Liang %A Tain,You-Lin %A Kuo,Chin-Yu %A Lin,Yun-Chun %+ Department of Industrial Engineering and Management, National Chiao Tung University, 1001 University Road, Hsinchu, 300, Taiwan, 886 +886975368975, jacky168@gmail.com %K community-acquired acute kidney injury (CA-AKI) %K hospitalization %K treatment decision making %K clinical decision support system %K machine learning %K feature selection with extreme gradient boost (XGBoost) %K least absolute shrinkage and selection operator (LASSO) %K risk prediction %D 2020 %7 4.8.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Community-acquired acute kidney injury (CA-AKI)-associated hospitalizations impose significant health care needs and contribute to in-hospital mortality. However, most risk prediction models developed to date have focused on AKI in a specific group of patients during hospitalization, and there is limited knowledge on the baseline risk in the general population for preventing CA-AKI-associated hospitalization. Objective: To gain further insight into risk exploration, the aim of this study was to develop, validate, and establish a scoring system to facilitate health professionals in enabling early recognition and intervention of CA-AKI to prevent permanent kidney damage using different machine-learning techniques. Methods: A nested case-control study design was employed using electronic health records derived from a group of Chang Gung Memorial Hospitals in Taiwan from 2010 to 2017 to identify 234,867 adults with at least two measures of serum creatinine at hospital admission. Patients were classified into a derivation cohort (2010-2016) and a temporal validation cohort (2017). Patients with the first episode of CA-AKI at hospital admission were classified into the case group and those without CA-AKI were classified in the control group. A total of 47 potential candidate variables, including age, gender, prior use of nephrotoxic medications, Charlson comorbid conditions, commonly measured laboratory results, and recent use of health services, were tested to develop a CA-AKI hospitalization risk model. Permutation-based selection with both the extreme gradient boost (XGBoost) and least absolute shrinkage and selection operator (LASSO) algorithms was performed to determine the top 10 important features for scoring function development. Results: The discriminative ability of the risk model was assessed by the area under the receiver operating characteristic curve (AUC), and the predictive CA-AKI risk model derived by the logistic regression algorithm achieved an AUC of 0.767 (95% CI 0.764-0.770) on derivation and 0.761 on validation for any stage of AKI, with positive and negative predictive values of 19.2% and 96.1%, respectively. The risk model for prediction of CA-AKI stages 2 and 3 had an AUC value of 0.818 for the validation cohort with positive and negative predictive values of 13.3% and 98.4%, respectively. These metrics were evaluated at a cut-off value of 7.993, which was determined as the threshold to discriminate the risk of AKI. Conclusions: A machine learning–generated risk score model can identify patients at risk of developing CA-AKI-related hospitalization through a routine care data-driven approach. The validated multivariate risk assessment tool could help clinicians to stratify patients in primary care, and to provide monitoring and early intervention for preventing AKI while improving the quality of AKI care in the general population. %M 32749223 %R 10.2196/16903 %U https://www.jmir.org/2020/8/e16903 %U https://doi.org/10.2196/16903 %U http://www.ncbi.nlm.nih.gov/pubmed/32749223 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 8 %P e18389 %T Analysis of Benzodiazepine Prescription Practices in Elderly Appalachians with Dementia via the Appalachian Informatics Platform: Longitudinal Study %A Bhardwaj,Niharika %A Cecchetti,Alfred A %A Murughiyan,Usha %A Neitch,Shirley %+ Department of Clinical and Translational Science, Joan C Edwards School of Medicine, Marshall University, 1600 Medical Center Drive, Suite 265, Huntington, WV, 25701, United States, 1 304 691 5397, bhardwaj1@marshall.edu %K dementia %K Alzheimer disease %K benzodiazepines %K Appalachia %K geriatrics %K informatics platform %K interactive visualization %K eHealth %K clinical data %D 2020 %7 4.8.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: Caring for the growing dementia population with complex health care needs in West Virginia has been challenging due to its large, sizably rural-dwelling geriatric population and limited resource availability. Objective: This paper aims to illustrate the application of an informatics platform to drive dementia research and quality care through a preliminary study of benzodiazepine (BZD) prescription patterns and its effects on health care use by geriatric patients. Methods: The Maier Institute Data Mart, which contains clinical and billing data on patients aged 65 years and older (N=98,970) seen within our clinics and hospital, was created. Relevant variables were analyzed to identify BZD prescription patterns and calculate related charges and emergency department (ED) use. Results: Nearly one-third (4346/13,910, 31.24%) of patients with dementia received at least one BZD prescription, 20% more than those without dementia. More women than men received at least one BZD prescription. On average, patients with dementia and at least one BZD prescription sustained higher charges and visited the ED more often than those without one. Conclusions: The Appalachian Informatics Platform has the potential to enhance dementia care and research through a deeper understanding of dementia, data enrichment, risk identification, and care gap analysis. %M 32749226 %R 10.2196/18389 %U https://medinform.jmir.org/2020/8/e18389 %U https://doi.org/10.2196/18389 %U http://www.ncbi.nlm.nih.gov/pubmed/32749226 %0 Journal Article %@ 2561-3278 %I JMIR Publications %V 5 %N 1 %P e18139 %T Video Cloud Services for Hospitals: Designing an End-to-End Cloud Service Platform for Medical Video Storage and Secure Access %A Pawałowski,Piotr %A Mazurek,Cezary %A Leszczuk,Mikołaj %A Moureaux,Jean-Marie %A Chaabouni,Amine %+ Poznan Supercomputing and Networking Center, ul Z Noskowskiego 12/14, Poznań, 61-704, Poland, 48 693919937, astagor@man.poznan.pl %K medical video %K telemedicine %K medical cloud platforms %D 2020 %7 29.7.2020 %9 Viewpoint %J JMIR Biomed Eng %G English %X The amount of medical video data that has to be securely stored has been growing exponentially. This rapid expansion is mainly caused by the introduction of higher video resolution such as 4K and 8K to medical devices and the growing usage of telemedicine services, along with a general trend toward increasing transparency with respect to medical treatment, resulting in more and more medical procedures being recorded. Such video data, as medical data, must be maintained for many years, resulting in datasets at the exabytes scale that each hospital must be able to store in the future. Currently, hospitals do not have the required information and communications technology infrastructure to handle such large amounts of data in the long run. In this paper, we discuss the challenges and possible solutions to this problem. We propose a generic architecture for a holistic, end-to-end recording and storage platform for hospitals, define crucial components, and identify existing and future solutions to address all parts of the system. This paper focuses mostly on the recording part of the system by introducing the major challenges in the area of bioinformatics, with particular focus on three major areas: video encoding, video quality, and video metadata. %R 10.2196/18139 %U https://biomedeng.jmir.org/2020/1/e18139 %U https://doi.org/10.2196/18139 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 7 %P e19126 %T Information Overload in Emergency Medicine Physicians: A Multisite Case Study Exploring the Causes, Impact, and Solutions in Four North England National Health Service Trusts %A Sbaffi,Laura %A Walton,James %A Blenkinsopp,John %A Walton,Graham %+ Information School, University of Sheffield, 211 Portobello Street, Regent Court, Sheffield, S1 4DP, United Kingdom, 44 114 2222686, L.Sbaffi@sheffield.ac.uk %K emergency medicine %K information overload %K physicians %K national health care system %D 2020 %7 27.7.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Information overload is affecting modern society now more than ever because of the wide and increasing distribution of digital technologies. Social media, emails, and online communications among others infuse a sense of urgency as information must be read, produced, and exchanged almost instantaneously. Emergency medicine is a medical specialty that is particularly affected by information overload with consequences on patient care that are difficult to quantify and address. Understanding the current causes of medical information overload, their impact on patient care, and strategies to handle the inflow of constant information is crucial to alleviating stress and anxiety that is already crippling the profession. Objective: This study aims to identify and evaluate the main causes and sources of medical information overload, as experienced by emergency medicine physicians in selected National Health Service (NHS) trusts in the United Kingdom. Methods: This study used a quantitative, survey-based data collection approach including close- and open-ended questions. A web-based survey was distributed to emergency physicians to assess the impact of medical information overload on their jobs. In total, 101 valid responses were collected from 4 NHS trusts in north England. Descriptive statistics, principal component analysis, independent sample two-tailed t tests, and one-way between-group analysis of variance with post hoc tests were performed on the data. Open-ended questions were analyzed using thematic analysis to identify key topics. Results: The vast majority of respondents agreed that information overload is a serious issue in emergency medicine, and it increases with time. The always available culture (mean 5.40, SD 1.56), email handling (mean 4.86, SD 1.80), and multidisciplinary communications (mean 4.51, SD 1.61) are the 3 main reasons leading to information overload. Due to this, emergency physicians experience guideline fatigue, stress and tension, longer working hours, and impaired decision making, among other issues. Aspects of information overload are also reported to have different impacts on physicians depending on demographic factors such as age, years spent in emergency medicine, and level of employment. Conclusions: There is a serious concern regarding information overload in emergency medicine. Participants identified a considerable number of daily causes affecting their job, particularly the traditional culture of emergency departments being always available on the ward, exacerbated by email and other forms of communication necessary to maintain optimal, evidence-based practice standards. However, not all information is unwelcome, as physicians also need to stay updated with the latest guidelines on conditions and treatment, and communicate with larger medical teams to provide quality care. %M 32716313 %R 10.2196/19126 %U http://www.jmir.org/2020/7/e19126/ %U https://doi.org/10.2196/19126 %U http://www.ncbi.nlm.nih.gov/pubmed/32716313 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 7 %P e16300 %T Adoption and Performance of Complementary Clinical Information Technologies: Analysis of a Survey of General Practitioners %A Poba-Nzaou,Placide %A Uwizeyemungu,Sylvestre %A Liu,Xuecheng %+ Department of Organization and Human Resources, University of Quebec in Montreal, 315, Sainte-Catherine East, Montreal, QC, H2X 3X2, Canada, 1 514 987 3000 ext 7744, poba-nzaou.placide@uqam.ca %K electronic health record %K personal health record %K health information exchange %K telehealth %K general practitioners %K quality of care %K efficiency %K organizational productivity %D 2020 %7 23.7.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: The benefits from the combination of 4 clinical information systems (CISs)—electronic health records (EHRs), health information exchange (HIE), personal health records (PHRs), and telehealth—in primary care depend on the configuration of their functional capabilities available to clinicians. However, our empirical knowledge of these configurations and their associated performance implications is very limited because they have mostly been studied in isolation. Objective: This study aims to pursue 3 objectives: (1) characterize general practitioners (GPs) by uncovering the typical profiles of combinations of 4 major CIS capabilities, (2) identify physician and practice characteristics that predict cluster membership, and (3) assess the variation in the levels of performance associated with each configuration. Methods: We used data from a survey of GPs conducted throughout the European Union (N=5793). First, 4 factors, that is, EHRs, HIE, PHRs, and Telehealth, were created. Second, a cluster analysis helps uncover clusters of GPs based on the 4 factors. Third, we compared the clusters according to five performance outcomes using an analysis of variance (ANOVA) and a Tamhane T2 post hoc test. Fourth, univariate and multivariate multinomial logistic regressions were used to identify predictors of the clusters. Finally, with a multivariate multinomial logistic regression, among the clusters, we compared performance in terms of the number of patients treated (3 levels) over the last 2 years. Results: We unveiled 3 clusters of GPs with different levels of CIS capability profiles: strong (1956/5793, 37.36%), medium (2764/5793, 47.71%), and weak (524/5793, 9.04%). The logistic regression analysis indicates that physicians (younger, female, and less experienced) and practice (solo) characteristics are significantly associated with a weak profile. The ANOVAs revealed a strong cluster associated with significantly high practice performance outcomes in terms of the quality of care, efficiency, productivity, and improvement of working processes, and two noncomprehensive medium and weak profiles associated with medium (equifinal) practice performance outcomes. The logistic regression analysis also revealed that physicians in the weak profile are associated with a decrease in the number of patients treated over the last 2 years. Conclusions: Different CIS capability profiles may lead to similar equifinal performance outcomes. This underlines the importance of looking beyond the adoption of 1 CIS capability versus a cluster of capabilities when studying CISs. GPs in the strong cluster exhibit a comprehensive CIS capability profile and outperform the other two clusters with noncomprehensive profiles, leading to significantly high performance in terms of the quality of care provided to patients, efficiency of the practice, productivity of the practice, and improvement of working processes. Our findings indicate that medical practices should develop high capabilities in all 4 CISs if they have to maximize their performance outcomes because efforts to develop high capabilities selectively may only be in vain. %M 32706715 %R 10.2196/16300 %U http://www.jmir.org/2020/7/e16300/ %U https://doi.org/10.2196/16300 %U http://www.ncbi.nlm.nih.gov/pubmed/32706715 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 7 %P e20443 %T Ontological Organization and Bioinformatic Analysis of Adverse Drug Reactions From Package Inserts: Development and Usability Study %A Li,Xiaoying %A Lin,Xin %A Ren,Huiling %A Guo,Jinjing %+ Institute of Medical Information, Chinese Academy of Medical Sciences, 69 Dongdan N St, Beijing, 100005, China, 86 10 52328911, ren.huiling@imicams.ac.cn %K ontology %K adverse drug reactions %K package inserts %K information retrieval %K natural language processing %K bioinformatics %K drug %K adverse events %K machine-understandable knowledge %K clinical applications %D 2020 %7 20.7.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Licensed drugs may cause unexpected adverse reactions in patients, resulting in morbidity, risk of mortality, therapy disruptions, and prolonged hospital stays. Officially approved drug package inserts list the adverse reactions identified from randomized controlled clinical trials with high evidence levels and worldwide postmarketing surveillance. Formal representation of the adverse drug reaction (ADR) enclosed in semistructured package inserts will enable deep recognition of side effects and rational drug use, substantially reduce morbidity, and decrease societal costs. Objective: This paper aims to present an ontological organization of traceable ADR information extracted from licensed package inserts. In addition, it will provide machine-understandable knowledge for bioinformatics analysis, semantic retrieval, and intelligent clinical applications. Methods: Based on the essential content of package inserts, a generic ADR ontology model is proposed from two dimensions (and nine subdimensions), covering the ADR information and medication instructions. This is followed by a customized natural language processing method programmed with Python to retrieve the relevant information enclosed in package inserts. After the biocuration and identification of retrieved data from the package insert, an ADR ontology is automatically built for further bioinformatic analysis. Results: We collected 165 package inserts of quinolone drugs from the National Medical Products Administration and other drug databases in China, and built a specialized ADR ontology containing 2879 classes and 15,711 semantic relations. For each quinolone drug, the reported ADR information and medication instructions have been logically represented and formally organized in an ADR ontology. To demonstrate its usage, the source data were further bioinformatically analyzed. For example, the number of drug-ADR triples and major ADRs associated with each active ingredient were recorded. The 10 ADRs most frequently observed among quinolones were identified and categorized based on the 18 categories defined in the proposal. The occurrence frequency, severity, and ADR mitigation method explicitly stated in package inserts were also analyzed, as well as the top 5 specific populations with contraindications for quinolone drugs. Conclusions: Ontological representation and organization using officially approved information from drug package inserts enables the identification and bioinformatic analysis of adverse reactions caused by a specific drug with regard to predefined ADR ontology classes and semantic relations. The resulting ontology-based ADR knowledge source classifies drug-specific adverse reactions, and supports a better understanding of ADRs and safer prescription of medications. %M 32706718 %R 10.2196/20443 %U https://www.jmir.org/2020/7/e20443 %U https://doi.org/10.2196/20443 %U http://www.ncbi.nlm.nih.gov/pubmed/32706718 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 7 %P e19274 %T The Influence of Electronic Health Record Use on Physician Burnout: Cross-Sectional Survey %A Tajirian,Tania %A Stergiopoulos,Vicky %A Strudwick,Gillian %A Sequeira,Lydia %A Sanches,Marcos %A Kemp,Jessica %A Ramamoorthi,Karishini %A Zhang,Timothy %A Jankowicz,Damian %+ Centre for Addiction and Mental Health, 6168F, 100 Stokes St., Toronto, ON, M6J1H4, Canada, 1 4165358501 ext 30515, Tania.Tajirian@camh.ca %K electronic health record %K physician %K burnout %K psychiatry %K medical informatics %D 2020 %7 15.7.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Physician burnout has a direct impact on the delivery of high-quality health care, with health information technology tools such as electronic health records (EHRs) adding to the burden of practice inefficiencies. Objective: The aim of this study was to determine the extent of burnout among physicians and learners (residents and fellows); identify significant EHR-related contributors of physician burnout; and explore the differences between physicians and learners with regard to EHR-related factors such as time spent in EHR, documentation styles, proficiency, training, and perceived usefulness. In addition, the study aimed to address gaps in the EHR-related burnout research methodologies by determining physicians’ patterns of EHR use through usage logs. Methods: This study used a cross-sectional survey methodology and a review of administrative data for back-end log measures of survey respondents’ EHR use, which was conducted at a large Canadian academic mental health hospital. Chi-square and Fisher exact tests were used to examine the association of EHR-related factors with general physician burnout. The survey was sent out to 474 individuals between May and June 2019, including physicians (n=407), residents (n=53), and fellows (n=14), and we measured physician burnout and perceptions of EHR stressors (along with demographic and practice characteristics). Results: Our survey included 208 respondents, including physicians (n=176) and learners (n=32). The response rate was 43.2% for physicians (full-time: 156/208, 75.0%; part-time: 20/199, 10.1%), and 48% (32/67) for learners. A total of 25.6% (45/176) of practicing physicians and 19% (6/32) of learners reported having one or more symptoms of burnout, and 74.5% (155/208) of all respondents who reported burnout symptoms identified the EHR as a contributor. Lower satisfaction and higher frustration with the EHRs were significantly associated with perceptions of EHR contributing toward burnout. Physicians’ and learners’ experiences with the EHR, gathered through open-ended survey responses, identified challenges around the intuitiveness and usability of the technology as well as workflow issues. Metrics gathered from back-end usage logs demonstrated a 13.6-min overestimation in time spent on EHRs per patient and a 5.63-hour overestimation of after-hours EHR time, when compared with self-reported survey data. Conclusions: This study suggests that the use of EHRs is a perceived contributor to physician burnout. There should be a focus on combating physician burnout by reducing the unnecessary administrative burdens of EHRs through efficient implementation of systems and effective postimplementation strategies. %M 32673234 %R 10.2196/19274 %U https://www.jmir.org/2020/7/e19274 %U https://doi.org/10.2196/19274 %U http://www.ncbi.nlm.nih.gov/pubmed/32673234 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 7 %P e17508 %T Requirements of Health Data Management Systems for Biomedical Care and Research: Scoping Review %A Ismail,Leila %A Materwala,Huned %A Karduck,Achim P %A Adem,Abdu %+ Department of Computer Science and Software Engineering, College of Information Technology, United Arab Emirates University, Al Maqam Campus, Al Ain, Abu Dhabi, 15551, United Arab Emirates, 971 37673333 ext 5530, leila@uaeu.ac.ae %K big data %K blockchain %K data analytics %K eHealth %K electronic medical records %K health care %K health information management %K Internet of Things %K medical research %K mHealth %D 2020 %7 7.7.2020 %9 Review %J J Med Internet Res %G English %X Background: Over the last century, disruptive incidents in the fields of clinical and biomedical research have yielded a tremendous change in health data management systems. This is due to a number of breakthroughs in the medical field and the need for big data analytics and the Internet of Things (IoT) to be incorporated in a real-time smart health information management system. In addition, the requirements of patient care have evolved over time, allowing for more accurate prognoses and diagnoses. In this paper, we discuss the temporal evolution of health data management systems and capture the requirements that led to the development of a given system over a certain period of time. Consequently, we provide insights into those systems and give suggestions and research directions on how they can be improved for a better health care system. Objective: This study aimed to show that there is a need for a secure and efficient health data management system that will allow physicians and patients to update decentralized medical records and to analyze the medical data for supporting more precise diagnoses, prognoses, and public insights. Limitations of existing health data management systems were analyzed. Methods: To study the evolution and requirements of health data management systems over the years, a search was conducted to obtain research articles and information on medical lawsuits, health regulations, and acts. These materials were obtained from the Institute of Electrical and Electronics Engineers, the Association for Computing Machinery, Elsevier, MEDLINE, PubMed, Scopus, and Web of Science databases. Results: Health data management systems have undergone a disruptive transformation over the years from paper to computer, web, cloud, IoT, big data analytics, and finally to blockchain. The requirements of a health data management system revealed from the evolving definitions of medical records and their management are (1) medical record data, (2) real-time data access, (3) patient participation, (4) data sharing, (5) data security, (6) patient identity privacy, and (7) public insights. This paper reviewed health data management systems based on these 7 requirements across studies conducted over the years. To our knowledge, this is the first analysis of the temporal evolution of health data management systems giving insights into the system requirements for better health care. Conclusions: There is a need for a comprehensive real-time health data management system that allows physicians, patients, and external users to input their medical and lifestyle data into the system. The incorporation of big data analytics will aid in better prognosis or diagnosis of the diseases and the prediction of diseases. The prediction results will help in the development of an effective prevention plan. %M 32348265 %R 10.2196/17508 %U https://www.jmir.org/2020/7/e17508 %U https://doi.org/10.2196/17508 %U http://www.ncbi.nlm.nih.gov/pubmed/32348265 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 7 %P e16312 %T A Multiview Model for Detecting the Inappropriate Use of Prescription Medication: Machine Learning Approach %A Zhuo,Lin %A Cheng,Yinchu %A Liu,Shaoqin %A Yang,Yu %A Tang,Shuang %A Zhen,Jiancun %A Zhao,Junfeng %A Zhan,Siyan %+ Research Center of Clinical Epidemiology, Peking University Third Hospital, 49 North Garden Rd, Haidian District, Beijing, 100191, China, 86 1082805162, siyan-zhan@bjmu.edu.cn %K inappropriate use of prescription medication %K topic model %K latent Dirichlet allocation %K multiview learning %K prescription review %D 2020 %7 6.7.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: The inappropriate use of prescription medication has recently garnered worldwide attention, but most national policies do not effectively provide for early detection or timely intervention. Objective: This study aimed to develop and assess the validity of a model that can detect the inappropriate use of prescription medication. This effort combines a multiview and topic matching method. The study also assessed the validity of this approach. Methods: A multiview extension of the latent Dirichlet allocation algorithm for topic modeling was chosen to generate diagnosis-medication topics, with data obtained from the Chinese Monitoring Network for Rational Use of Drugs (CMNRUD) database. Topic mapping allowed for calculating the degree to which diagnoses and medications were similarly distributed and, by setting a threshold, for identifying prescription misuse. The Beijing Regional Prescription Review Database (BRPRD) database was used as the gold standard to assess the model’s validity. We also conducted a sensitivity analysis using random samples of validated prescriptions and evaluated the model’s performance. Results: A total of 44 million prescriptions were used to generate topics using the diagnoses and medications from the CMNRUD database. A random sample (15,000 prescriptions) from the BRPRD was used for validation, and it was found that the model had a sensitivity of 81.8%, specificity of 47.4%, positive-predictive value of 14.5%, and negative-predictive value of 96.0%. The model showed superior stability under different sampling proportions. Conclusions: A method that combines multiview topic modeling and topic matching can detect the inappropriate use of prescription medication. This model, which has mediocre specificity and moderate sensitivity, can be used as a primary screening tool and will likely complement and improve the process of manually reviewing prescriptions. %M 32209527 %R 10.2196/16312 %U https://medinform.jmir.org/2020/7/e16312 %U https://doi.org/10.2196/16312 %U http://www.ncbi.nlm.nih.gov/pubmed/32209527 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 7 %P e18963 %T Predicting Current Glycated Hemoglobin Levels in Adults From Electronic Health Records: Validation of Multiple Logistic Regression Algorithm %A Alhassan,Zakhriya %A Budgen,David %A Alshammari,Riyad %A Al Moubayed,Noura %+ Department of Computer Science, Durham University, Mountjoy Centre, Stockton Road, Durham, DH1 3LE, United Kingdom, 44 191 3341724, zakhriya.n.alhassan@durham.ac.uk %K glycated hemoglobin %K HbA1c %K prediction %K electronic health records %K diabetes %K differentiated replication %K EHR %K hemoglobin %K logistic regression %K medical informatics %D 2020 %7 3.7.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: Electronic health record (EHR) systems generate large datasets that can significantly enrich the development of medical predictive models. Several attempts have been made to investigate the effect of glycated hemoglobin (HbA1c) elevation on the prediction of diabetes onset. However, there is still a need for validation of these models using EHR data collected from different populations. Objective: The aim of this study is to perform a replication study to validate, evaluate, and identify the strengths and weaknesses of replicating a predictive model that employed multiple logistic regression with EHR data to forecast the levels of HbA1c. The original study used data from a population in the United States and this differentiated replication used a population in Saudi Arabia. Methods: A total of 3 models were developed and compared with the model created in the original study. The models were trained and tested using a larger dataset from Saudi Arabia with 36,378 records. The 10-fold cross-validation approach was used for measuring the performance of the models. Results: Applying the method employed in the original study achieved an accuracy of 74% to 75% when using the dataset collected from Saudi Arabia, compared with 77% obtained from using the population from the United States. The results also show a different ranking of importance for the predictors between the original study and the replication. The order of importance for the predictors with our population, from the most to the least importance, is age, random blood sugar, estimated glomerular filtration rate, total cholesterol, non–high-density lipoprotein, and body mass index. Conclusions: This replication study shows that direct use of the models (calculators) created using multiple logistic regression to predict the level of HbA1c may not be appropriate for all populations. This study reveals that the weighting of the predictors needs to be calibrated to the population used. However, the study does confirm that replicating the original study using a different population can help with predicting the levels of HbA1c by using the predictors that are routinely collected and stored in hospital EHR systems. %M 32618575 %R 10.2196/18963 %U https://medinform.jmir.org/2020/7/e18963 %U https://doi.org/10.2196/18963 %U http://www.ncbi.nlm.nih.gov/pubmed/32618575 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 6 %P e19938 %T Information Technology–Based Management of Clinically Healthy COVID-19 Patients: Lessons From a Living and Treatment Support Center Operated by Seoul National University Hospital %A Bae,Ye Seul %A Kim,Kyung Hwan %A Choi,Sae Won %A Ko,Taehoon %A Jeong,Chang Wook %A Cho,BeLong %A Kim,Min Sun %A Kang,EunKyo %+ Office of Hospital Information, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul, 03080, Republic of Korea, 82 2 2072 7600, kkh726@snu.ac.kr %K COVID-19 %K clinical informatics %K mobile app %K telemedicine %K hospital information system %K app %K health information technology %D 2020 %7 12.6.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: South Korea took preemptive action against coronavirus disease (COVID-19) by implementing extensive testing, thorough epidemiological investigation, strict social distancing, and rapid treatment of patients according to disease severity. The Korean government entrusted large-scale hospitals with the operation of living and treatment support centers (LTSCs) for the management for clinically healthy COVID-19 patients. Objective: The aim of this paper is to introduce our experience implementing information and communications technology (ICT)-based remote patient management systems at a COVID-19 LTSC. Methods: We adopted new electronic health record templates, hospital information system (HIS) dashboards, cloud-based medical image sharing, a mobile app, and smart vital sign monitoring devices. Results: Enhancements were made to the HIS to assist in the workflow and care of patients in the LTSC. A dashboard was created for the medical staff to view the vital signs and symptoms of all patients. Patients used a mobile app to consult with their physician or nurse, answer questionnaires, and input self-measured vital signs; the results were uploaded to the hospital information system in real time. Cloud-based image sharing enabled interoperability between medical institutions. Korea’s strategy of aggressive mitigation has “flattened the curve” of the rate of infection. A multidisciplinary approach was integral to develop systems supporting patient care and management at the living and treatment support center as quickly as possible. Conclusions: Faced with a novel infectious disease, we describe the implementation and experience of applying an ICT-based patient management system in the LTSC affiliated with Seoul National University Hospital. ICT-based tools and applications are increasingly important in health care, and we hope that our experience will provide insight into future technology-based infectious disease responses. %M 32490843 %R 10.2196/19938 %U http://www.jmir.org/2020/6/e19938/ %U https://doi.org/10.2196/19938 %U http://www.ncbi.nlm.nih.gov/pubmed/32490843 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 6 %P e15068 %T Developing and Applying a Formative Evaluation Framework for Health Information Technology Implementations: Qualitative Investigation %A Cresswell,Kathrin %A Williams,Robin %A Sheikh,Aziz %+ Usher Institute, The University of Edinburgh, Teviot Place, Edinburgh, , United Kingdom, 44 1316508102, kathrin.beyer@ed.ac.uk %K health information technology %K evaluation %K sociotechnical %D 2020 %7 10.6.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: There is currently a lack of comprehensive, intuitive, and usable formative evaluation frameworks for health information technology (HIT) implementations. We therefore sought to develop and apply such a framework. This study describes the Technology, People, Organizations, and Macroenvironmental factors (TPOM) framework we developed. Objective: The aim was to develop and apply a formative evaluation framework for HIT implementations, highlighting interrelationships between identified dimensions and offering guidance for implementers. Methods: We drew on an initial prototype framework developed as part of a literature review exploring factors for the effective implementation of HIT. In addition, we used qualitative data from three national formative evaluations of different HIT interventions (electronic health record, electronic prescribing, and clinical decision support functionality). The combined data set comprised 19 case studies of primarily hospital settings, and included 703 semistructured interviews, 663 hours of observations, and 864 documents gathered from a range of care settings across National Health Service (NHS) England and NHS Scotland. Data analysis took place over a period of 10 years and was guided by a framework informed by the existing evidence base. Results: TPOM dimensions are intimately related and each include a number of subthemes that evaluators need to consider. Although technological functionalities are crucial in getting an initiative off the ground, system design needs to be cognizant of the accompanying social and organizational transformations required to ensure that technologies deliver the desired value for a variety of stakeholders. Wider structural changes, characterized by shifting policy landscapes and markets, influence technologies and the ways they are used by organizations and staff. Conclusions: The TPOM framework supports formative evaluations of HIT implementation and digitally enabled transformation efforts. There is now a need for prospective application of the TPOM framework to determine its value. %M 32519968 %R 10.2196/15068 %U http://www.jmir.org/2020/6/e15068/ %U https://doi.org/10.2196/15068 %U http://www.ncbi.nlm.nih.gov/pubmed/32519968 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 5 %P e18707 %T Agile Health Care Analytics: Enabling Real-Time Disease Surveillance With a Computational Health Platform %A Schulz,Wade L %A Durant,Thomas J S %A Torre Jr,Charles J %A Hsiao,Allen L %A Krumholz,Harlan M %+ Department of Laboratory Medicine, Yale School of Medicine, 55 Park St, PS215, New Haven, CT, 06511, United States, 1 (203) 688 2286, wade.schulz@yale.edu %K real-time analytics %K real-world data %K disease surveillance %K computational health %K surveillance %K public health %K COVID-19 %K outbreak %K health information technology %K HIT %K interface %K monitoring %K pandemic %D 2020 %7 28.5.2020 %9 Viewpoint %J J Med Internet Res %G English %X The ongoing coronavirus disease outbreak demonstrates the need for novel applications of real-time data to produce timely information about incident cases. Using health information technology (HIT) and real-world data, we sought to produce an interface that could, in near real time, identify patients presenting with suspected respiratory tract infection and enable monitoring of test results related to specific pathogens, including severe acute respiratory syndrome coronavirus 2. This tool was built upon our computational health platform, which provides access to near real-time data from disparate HIT sources across our health system. This combination of technology allowed us to rapidly prototype, iterate, and deploy a platform to support a cohesive organizational response to a rapidly evolving outbreak. Platforms that allow for agile analytics are needed to keep pace with evolving needs within the health care system. %M 32442130 %R 10.2196/18707 %U http://www.jmir.org/2020/5/e18707/ %U https://doi.org/10.2196/18707 %U http://www.ncbi.nlm.nih.gov/pubmed/32442130 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 5 %P e16452 %T Challenges of Clustering Multimodal Clinical Data: Review of Applications in Asthma Subtyping %A Horne,Elsie %A Tibble,Holly %A Sheikh,Aziz %A Tsanas,Athanasios %+ Usher Institute, Edinburgh Medical School, University of Edinburgh, Nine Edinburgh Bio Quarter, 9 Little France Road, Edinburgh, EH16 4UX, United Kingdom, 44 1316517887, Elsie.Horne@ed.ac.uk %K asthma %K cluster analysis %K data mining %K machine learning %K unsupervised machine learning %D 2020 %7 28.5.2020 %9 Review %J JMIR Med Inform %G English %X Background: In the current era of personalized medicine, there is increasing interest in understanding the heterogeneity in disease populations. Cluster analysis is a method commonly used to identify subtypes in heterogeneous disease populations. The clinical data used in such applications are typically multimodal, which can make the application of traditional cluster analysis methods challenging. Objective: This study aimed to review the research literature on the application of clustering multimodal clinical data to identify asthma subtypes. We assessed common problems and shortcomings in the application of cluster analysis methods in determining asthma subtypes, such that they can be brought to the attention of the research community and avoided in future studies. Methods: We searched PubMed and Scopus bibliographic databases with terms related to cluster analysis and asthma to identify studies that applied dissimilarity-based cluster analysis methods. We recorded the analytic methods used in each study at each step of the cluster analysis process. Results: Our literature search identified 63 studies that applied cluster analysis to multimodal clinical data to identify asthma subtypes. The features fed into the cluster algorithms were of a mixed type in 47 (75%) studies and continuous in 12 (19%), and the feature type was unclear in the remaining 4 (6%) studies. A total of 23 (37%) studies used hierarchical clustering with Ward linkage, and 22 (35%) studies used k-means clustering. Of these 45 studies, 39 had mixed-type features, but only 5 specified dissimilarity measures that could handle mixed-type features. A further 9 (14%) studies used a preclustering step to create small clusters to feed on a hierarchical method. The original sample sizes in these 9 studies ranged from 84 to 349. The remaining studies used hierarchical clustering with other linkages (n=3), medoid-based methods (n=3), spectral clustering (n=1), and multiple kernel k-means clustering (n=1), and in 1 study, the methods were unclear. Of 63 studies, 54 (86%) explained the methods used to determine the number of clusters, 24 (38%) studies tested the quality of their cluster solution, and 11 (17%) studies tested the stability of their solution. Reporting of the cluster analysis was generally poor in terms of the methods employed and their justification. Conclusions: This review highlights common issues in the application of cluster analysis to multimodal clinical data to identify asthma subtypes. Some of these issues were related to the multimodal nature of the data, but many were more general issues in the application of cluster analysis. Although cluster analysis may be a useful tool for investigating disease subtypes, we recommend that future studies carefully consider the implications of clustering multimodal data, the cluster analysis process itself, and the reporting of methods to facilitate replication and interpretation of findings. %M 32463370 %R 10.2196/16452 %U http://medinform.jmir.org/2020/5/e16452/ %U https://doi.org/10.2196/16452 %U http://www.ncbi.nlm.nih.gov/pubmed/32463370 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 5 %P e14693 %T Using Machine Learning to Predict Early Onset Acute Organ Failure in Critically Ill Intensive Care Unit Patients With Sickle Cell Disease: Retrospective Study %A Mohammed,Akram %A Podila,Pradeep S B %A Davis,Robert L %A Ataga,Kenneth I %A Hankins,Jane S %A Kamaleswaran,Rishikesan %+ Department of Biomedical Informatics, Emory University School of Medicine, WMB, 101 Woodruff Circle, Suite 4127, Atlanta, GA, 30322, United States, 1 (901) 462 6908, rkamaleswaran@emory.edu %K multiple organ failure %K sickle cell disease %K machine learning %K electronic medical record %K hematology %D 2020 %7 13.5.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Sickle cell disease (SCD) is a genetic disorder of the red blood cells, resulting in multiple acute and chronic complications, including pain episodes, stroke, and kidney disease. Patients with SCD develop chronic organ dysfunction, which may progress to organ failure during disease exacerbations. Early detection of acute physiological deterioration leading to organ failure is not always attainable. Machine learning techniques that allow for prediction of organ failure may enable early identification and treatment and potentially reduce mortality. Objective: The aim of this study was to test the hypothesis that machine learning physiomarkers can predict the development of organ dysfunction in a sample of adult patients with SCD admitted to intensive care units (ICUs). Methods: We applied diverse machine learning methods, statistical methods, and data visualization techniques to develop classification models to distinguish SCD from controls. Results: We studied 63 sequential SCD patients admitted to ICUs with 163 patient encounters (mean age 30.7 years, SD 9.8 years). A subset of these patient encounters, 22.7% (37/163), met the sequential organ failure assessment criteria. The other 126 SCD patient encounters served as controls. A set of signal processing features (such as fast Fourier transform, energy, and continuous wavelet transform) derived from heart rate, blood pressure, and respiratory rate was identified to distinguish patients with SCD who developed acute physiological deterioration leading to organ failure from patients with SCD who did not meet the criteria. A multilayer perceptron model accurately predicted organ failure up to 6 hours before onset, with an average sensitivity and specificity of 96% and 98%, respectively. Conclusions: This retrospective study demonstrated the viability of using machine learning to predict acute organ failure among hospitalized adults with SCD. The discovery of salient physiomarkers through machine learning techniques has the potential to further accelerate the development and implementation of innovative care delivery protocols and strategies for medically vulnerable patients. %M 32401216 %R 10.2196/14693 %U https://www.jmir.org/2020/5/e14693 %U https://doi.org/10.2196/14693 %U http://www.ncbi.nlm.nih.gov/pubmed/32401216 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 5 %P e16084 %T A User-Friendly, Web-Based Integrative Tool (ESurv) for Survival Analysis: Development and Validation Study %A Pak,Kyoungjune %A Oh,Sae-Ock %A Goh,Tae Sik %A Heo,Hye Jin %A Han,Myoung-Eun %A Jeong,Dae Cheon %A Lee,Chi-Seung %A Sun,Hokeun %A Kang,Junho %A Choi,Suji %A Lee,Soohwan %A Kwon,Eun Jung %A Kang,Ji Wan %A Kim,Yun Hak %+ Department of Anatomy, School of Medicine, Pusan National University, 49 Busandaehak-ro, Yangsan, 50612, Republic of Korea, 82 515108091, yunhak10510@pusan.ac.kr %K survival analysis %K grouped variable selection %K The Cancer Genome Atlas %K web-based tool %K user service %D 2020 %7 5.5.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Prognostic genes or gene signatures have been widely used to predict patient survival and aid in making decisions pertaining to therapeutic actions. Although some web-based survival analysis tools have been developed, they have several limitations. Objective: Taking these limitations into account, we developed ESurv (Easy, Effective, and Excellent Survival analysis tool), a web-based tool that can perform advanced survival analyses using user-derived data or data from The Cancer Genome Atlas (TCGA). Users can conduct univariate analyses and grouped variable selections using multiomics data from TCGA. Methods: We used R to code survival analyses based on multiomics data from TCGA. To perform these analyses, we excluded patients and genes that had insufficient information. Clinical variables were classified as 0 and 1 when there were two categories (for example, chemotherapy: no or yes), and dummy variables were used where features had 3 or more outcomes (for example, with respect to laterality: right, left, or bilateral). Results: Through univariate analyses, ESurv can identify the prognostic significance for single genes using the survival curve (median or optimal cutoff), area under the curve (AUC) with C statistics, and receiver operating characteristics (ROC). Users can obtain prognostic variable signatures based on multiomics data from clinical variables or grouped variable selections (lasso, elastic net regularization, and network-regularized high-dimensional Cox-regression) and select the same outputs as above. In addition, users can create custom gene signatures for specific cancers using various genes of interest. One of the most important functions of ESurv is that users can perform all survival analyses using their own data. Conclusions: Using advanced statistical techniques suitable for high-dimensional data, including genetic data, and integrated survival analysis, ESurv overcomes the limitations of previous web-based tools and will help biomedical researchers easily perform complex survival analyses. %M 32369034 %R 10.2196/16084 %U https://www.jmir.org/2020/5/e16084 %U https://doi.org/10.2196/16084 %U http://www.ncbi.nlm.nih.gov/pubmed/32369034 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 5 %P e15686 %T Determining Factors Affecting Nurses’ Acceptance of a Care Plan System Using a Modified Technology Acceptance Model 3: Structural Equation Model With Cross-Sectional Data %A Ho,Kuei-Fang %A Chang,Pi-Chen %A Kurniasari,Maria Dyah %A Susanty,Sri %A Chung,Min-Huey %+ School of Nursing, College of Nursing, Taipei Medical University, 250 Wuxing Street, Taipei, , Taiwan, 886 2736 1661 ext 6317, minhuey300@tmu.edu.tw %K care plan system %K technology acceptance model 3 %K behavioral intention %D 2020 %7 5.5.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: Health information technology is used in nursing practice worldwide, and holistic patient care planning can serve as a guide for nursing practice to ensure quality in patient-centered care. However, few studies have thoroughly analyzed users’ acceptance of care plan systems to establish individual plans. Objective: Based on the technology acceptance model 3 (TAM3), a user technology acceptance model was established to explore what determines the acceptance of care plan systems by users in clinical settings. Methods: Cross-sectional quantitative data were obtained from 222 nurses at eight hospitals affiliated with public organizations in Taiwan. Using the modified TAM3, the collected data were employed to analyze the determinants of user acceptance of a care plan system through structural equation modeling (SEM). We also employed moderated multiple regression analysis and partial least squares–SEM to test the moderating effects. Results: We verified all significant effects from the use of a care plan system among bivariate patterns in the modified TAM3, except for moderating effects. Our results revealed that the determinants of perceived usefulness and perceived ease of use significantly influenced perceived usefulness and perceived ease of use, respectively. The results also indicated that nurses’ perceptions of subjective norm (path coefficient=.25, P<.001), perceived ease of use (path coefficient=.32, P<.001), and perceived usefulness (path coefficient=.31, P<.001) had significantly positive effects on their behavioral intention to use the care plan system, accounting for 69% of the total explained variance. Conclusions: By exploring nurses’ acceptance of a care plan system, this study revealed relationships among the variables in TAM3. Our results confirm that the modified TAM3 is an innovative assessment instrument that can help managers understand nurses’ acceptance of health information technology in nursing practice to enhance the adoption of health information technology. %M 32369033 %R 10.2196/15686 %U https://medinform.jmir.org/2020/5/e15686 %U https://doi.org/10.2196/15686 %U http://www.ncbi.nlm.nih.gov/pubmed/32369033 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 5 %P e14330 %T Use of Machine Learning Techniques for Case-Detection of Varicella Zoster Using Routinely Collected Textual Ambulatory Records: Pilot Observational Study %A Lanera,Corrado %A Berchialla,Paola %A Baldi,Ileana %A Lorenzoni,Giulia %A Tramontan,Lara %A Scamarcia,Antonio %A Cantarutti,Luigi %A Giaquinto,Carlo %A Gregori,Dario %+ Department of Cardiac Thoracic Vascular Sciences and Public Health, University of Padova, Unit of Biostatistics, Epidemiology and Public Health, Via Leonardo Loredan 18, Padova, 35121, Italy, 39 049 827 5384, dario.gregori@unipd.it %K machine learning technique %K text mining %K electronic health report %K varicella zoster %K pediatric infectious disease %D 2020 %7 5.5.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: The detection of infectious diseases through the analysis of free text on electronic health reports (EHRs) can provide prompt and accurate background information for the implementation of preventative measures, such as advertising and monitoring the effectiveness of vaccination campaigns. Objective: The purpose of this paper is to compare machine learning techniques in their application to EHR analysis for disease detection. Methods: The Pedianet database was used as a data source for a real-world scenario on the identification of cases of varicella. The models’ training and test sets were based on two different Italian regions’ (Veneto and Sicilia) data sets of 7631 patients and 1,230,355 records, and 2347 patients and 569,926 records, respectively, for whom a gold standard of varicella diagnosis was available. Elastic-net regularized generalized linear model (GLMNet), maximum entropy (MAXENT), and LogitBoost (boosting) algorithms were implemented in a supervised environment and 5-fold cross-validated. The document-term matrix generated by the training set involves a dictionary of 1,871,532 tokens. The analysis was conducted on a subset of 29,096 tokens, corresponding to a matrix with no more than a 99% sparsity ratio. Results: The highest predictive values were achieved through boosting (positive predicative value [PPV] 63.1, 95% CI 42.7-83.5 and negative predicative value [NPV] 98.8, 95% CI 98.3-99.3). GLMNet delivered superior predictive capability compared to MAXENT (PPV 24.5% and NPV 98.3% vs PPV 11.0% and NPV 98.0%). MAXENT and GLMNet predictions weakly agree with each other (agreement coefficient 1 [AC1]=0.60, 95% CI 0.58-0.62), as well as with LogitBoost (MAXENT: AC1=0.64, 95% CI 0.63-0.66 and GLMNet: AC1=0.53, 95% CI 0.51-0.55). Conclusions: Boosting has demonstrated promising performance in large-scale EHR-based infectious disease identification. %M 32369038 %R 10.2196/14330 %U https://medinform.jmir.org/2020/5/e14330 %U https://doi.org/10.2196/14330 %U http://www.ncbi.nlm.nih.gov/pubmed/32369038 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 4 %P e17787 %T Modified Bidirectional Encoder Representations From Transformers Extractive Summarization Model for Hospital Information Systems Based on Character-Level Tokens (AlphaBERT): Development and Performance Evaluation %A Chen,Yen-Pin %A Chen,Yi-Ying %A Lin,Jr-Jiun %A Huang,Chien-Hua %A Lai,Feipei %+ Graduate Institute of Biomedical Electronics and Bioinformatics, National Taiwan University, Room 410, Barry Lam Hall, No 1, Sec 4, Roosevelt Road, Taipei City, , Taiwan, 886 2 3366 3754, f06945029@g.ntu.edu.tw %K transformer %K BERT %K deep learning %K emergency medicine %K automatic summarization %D 2020 %7 29.4.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: Doctors must care for many patients simultaneously, and it is time-consuming to find and examine all patients’ medical histories. Discharge diagnoses provide hospital staff with sufficient information to enable handling multiple patients; however, the excessive amount of words in the diagnostic sentences poses problems. Deep learning may be an effective solution to overcome this problem, but the use of such a heavy model may also add another obstacle to systems with limited computing resources. Objective: We aimed to build a diagnoses-extractive summarization model for hospital information systems and provide a service that can be operated even with limited computing resources. Methods: We used a Bidirectional Encoder Representations from Transformers (BERT)-based structure with a two-stage training method based on 258,050 discharge diagnoses obtained from the National Taiwan University Hospital Integrated Medical Database, and the highlighted extractive summaries written by experienced doctors were labeled. The model size was reduced using a character-level token, the number of parameters was decreased from 108,523,714 to 963,496, and the model was pretrained using random mask characters in the discharge diagnoses and International Statistical Classification of Diseases and Related Health Problems sets. We then fine-tuned the model using summary labels and cleaned up the prediction results by averaging all probabilities for entire words to prevent character level–induced fragment words. Model performance was evaluated against existing models BERT, BioBERT, and Long Short-Term Memory (LSTM) using the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) L score, and a questionnaire website was built to collect feedback from more doctors for each summary proposal. Results: The area under the receiver operating characteristic curve values of the summary proposals were 0.928, 0.941, 0.899, and 0.947 for BERT, BioBERT, LSTM, and the proposed model (AlphaBERT), respectively. The ROUGE-L scores were 0.697, 0.711, 0.648, and 0.693 for BERT, BioBERT, LSTM, and AlphaBERT, respectively. The mean (SD) critique scores from doctors were 2.232 (0.832), 2.134 (0.877), 2.207 (0.844), 1.927 (0.910), and 2.126 (0.874) for reference-by-doctor labels, BERT, BioBERT, LSTM, and AlphaBERT, respectively. Based on the paired t test, there was a statistically significant difference in LSTM compared to the reference (P<.001), BERT (P=.001), BioBERT (P<.001), and AlphaBERT (P=.002), but not in the other models. Conclusions: Use of character-level tokens in a BERT model can greatly decrease the model size without significantly reducing performance for diagnoses summarization. A well-developed deep-learning model will enhance doctors’ abilities to manage patients and promote medical studies by providing the capability to use extensive unstructured free-text notes. %M 32347806 %R 10.2196/17787 %U http://medinform.jmir.org/2020/4/e17787/ %U https://doi.org/10.2196/17787 %U http://www.ncbi.nlm.nih.gov/pubmed/32347806 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 4 %P e15196 %T Epidemiological Characterization of a Directed and Weighted Disease Network Using Data From a Cohort of One Million Patients: Network Analysis %A Ko,Kyungmin %A Lee,Chae Won %A Nam,Sangmin %A Ahn,Song Vogue %A Bae,Jung Ho %A Ban,Chi Yong %A Yoo,Jongman %A Park,Jungmin %A Han,Hyun Wook %+ Department of Biomedical Informatics, CHA University of Medicine, Pangyo-ro, 335, Seongnam, KS009, Republic of Korea, 82 31 881 7109, stepano7@gmail.com %K cohort studies %K data science %K longitudinal studies %K statistical data interpretation %K medical informatics %D 2020 %7 9.4.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: In the past 20 years, various methods have been introduced to construct disease networks. However, established disease networks have not been clinically useful to date because of differences among demographic factors, as well as the temporal order and intensity among disease-disease associations. Objective: This study sought to investigate the overall patterns of the associations among diseases; network properties, such as clustering, degree, and strength; and the relationship between the structure of disease networks and demographic factors. Methods: We used National Health Insurance Service-National Sample Cohort (NHIS-NSC) data from the Republic of Korea, which included the time series insurance information of 1 million out of 50 million Korean (approximately 2%) patients obtained between 2002 and 2013. After setting the observation and outcome periods, we selected only 520 common Korean Classification of Disease, sixth revision codes that were the most prevalent diagnoses, making up approximately 80% of the cases, for statistical validity. Using these data, we constructed a directional and weighted temporal network that considered both demographic factors and network properties. Results: Our disease network contained 294 nodes and 3085 edges, a relative risk value of more than 4, and a false discovery rate-adjusted P value of <.001. Interestingly, our network presented four large clusters. Analysis of the network topology revealed a stronger correlation between in-strength and out-strength than between in-degree and out-degree. Further, the mean age of each disease population was related to the position along the regression line of the out/in-strength plot. Conversely, clustering analysis suggested that our network boasted four large clusters with different sex, age, and disease categories. Conclusions: We constructed a directional and weighted disease network visualizing demographic factors. Our proposed disease network model is expected to be a valuable tool for use by early clinical researchers seeking to explore the relationships among diseases in the future. %M 32271154 %R 10.2196/15196 %U https://www.jmir.org/2020/4/e15196 %U https://doi.org/10.2196/15196 %U http://www.ncbi.nlm.nih.gov/pubmed/32271154 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 4 %P e13761 %T Adoption of an Electronic Patient Record Sharing Pilot Project: Cross-Sectional Survey %A Wang,Jingxuan %A Huang,Junjie %A Cheung,Clement Shek Kei %A Wong,Wing Nam %A Cheung,Ngai Tseung %A Wong,Martin CS %+ The Jockey Club School of Public Health and Primary Care, Faculty of Medicine, Chinese University of Hong Kong, New Territories, Hong Kong, 852 2252 8782, wong_martin@cuhk.edu.hk %K health information exchange %K shared electronic health record %K online platform %K public-private partnership %D 2020 %7 6.4.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: The Public Private Interface–Electronic Patient Record (PPI-ePR) system was implemented as a new electronic platform to facilitate collaboration between the public and private sectors in Hong Kong. However, its barriers to participate and benefits have not been comprehensively assessed. Objective: This study aimed to evaluate the awareness, acceptance, perceived benefits, and obstacles to participation among private doctors and the general public. Methods: From December 2012 to January 2013, 2435 telephone interviews were performed by trained interviewers to survey randomly selected patients who were enrolled or not enrolled in the PPI-ePR system. In addition, self-administered surveys were sent by postal mail to 4229 registered doctors in Hong Kong. The questionnaires for both patients and doctors contained questions on subjects’ awareness, acceptance, and perceptions of the PPI-ePR, perceived benefits and obstacles of participating in the program, reasons for not using the system after enrolling, and perceived areas for service improvement of the system. Results: More than 53.1% (266/501) of enrolled patients believed that the PPI-ePR system would improve health care quality by reducing duplicate tests and treatments, while more than 76.8% (314/409) of enrolled doctors emphasized timely access to patients’ medical records as the biggest benefit of their enrollment. Among nonenrolled patients, unawareness of the project was the most popular obstacle to enrolling in the PPI-ePR system (483/1200, 40.3%). Regarding nonenrolled doctors, the complicated registration process hindered them from participating in the program the most (95/198, 48.0%). Television, newspaper, and magazine advertisements and medical profession newsletters or journals were suggested as the most effective means to encourage participation in the program among surveyed patients (1297/1701, 76.2%) and doctors (428/610, 70.2%), respectively. Lack of clinical indication requiring data extraction from other hospitals was the main reason for low level of PPI-ePR use. Conclusions: This study comprehensively assessed the popularity, perceived benefits, and hindering factors of enrolling in the PPI-ePR system in Hong Kong. Low levels of awareness, few privacy concerns, and inactive use of the PPI-ePR system were among the key features for patients and physicians. Public promotions, simplified logistics, and a user-friendly online interface were suggested to improve the coverage and effectiveness of health information exchange between private and public health care sectors. %M 32250279 %R 10.2196/13761 %U https://www.jmir.org/2020/4/e13761 %U https://doi.org/10.2196/13761 %U http://www.ncbi.nlm.nih.gov/pubmed/32250279 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 4 %P e15876 %T Leveraging Eye Tracking to Prioritize Relevant Medical Record Data: Comparative Machine Learning Study %A King,Andrew J %A Cooper,Gregory F %A Clermont,Gilles %A Hochheiser,Harry %A Hauskrecht,Milos %A Sittig,Dean F %A Visweswaran,Shyam %+ Department of Biomedical Informatics, University of Pittsburgh, The Offices at Baum, 5607 Baum Blvd., Suite 523, Pittsburgh, PA, United States, 1 412 648 7119, shv3@pitt.edu %K electronic medical record system %K eye tracking %K machine learning %K intensive care unit %K information-seeking behavior %D 2020 %7 2.4.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Electronic medical record (EMR) systems capture large amounts of data per patient and present that data to physicians with little prioritization. Without prioritization, physicians must mentally identify and collate relevant data, an activity that can lead to cognitive overload. To mitigate cognitive overload, a Learning EMR (LEMR) system prioritizes the display of relevant medical record data. Relevant data are those that are pertinent to a context—defined as the combination of the user, clinical task, and patient case. To determine which data are relevant in a specific context, a LEMR system uses supervised machine learning models of physician information-seeking behavior. Since obtaining information-seeking behavior data via manual annotation is slow and expensive, automatic methods for capturing such data are needed. Objective: The goal of the research was to propose and evaluate eye tracking as a high-throughput method to automatically acquire physician information-seeking behavior useful for training models for a LEMR system. Methods: Critical care medicine physicians reviewed intensive care unit patient cases in an EMR interface developed for the study. Participants manually identified patient data that were relevant in the context of a clinical task: preparing a patient summary to present at morning rounds. We used eye tracking to capture each physician’s gaze dwell time on each data item (eg, blood glucose measurements). Manual annotations and gaze dwell times were used to define target variables for developing supervised machine learning models of physician information-seeking behavior. We compared the performance of manual selection and gaze-derived models on an independent set of patient cases. Results: A total of 68 pairs of manual selection and gaze-derived machine learning models were developed from training data and evaluated on an independent evaluation data set. A paired Wilcoxon signed-rank test showed similar performance of manual selection and gaze-derived models on area under the receiver operating characteristic curve (P=.40). Conclusions: We used eye tracking to automatically capture physician information-seeking behavior and used it to train models for a LEMR system. The models that were trained using eye tracking performed like models that were trained using manual annotations. These results support further development of eye tracking as a high-throughput method for training clinical decision support systems that prioritize the display of relevant medical record data. %M 32238342 %R 10.2196/15876 %U https://www.jmir.org/2020/4/e15876 %U https://doi.org/10.2196/15876 %U http://www.ncbi.nlm.nih.gov/pubmed/32238342 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 4 %P e15554 %T Longitudinal Study of the Variation in Patient Turnover and Patient-to-Nurse Ratio: Descriptive Analysis of a Swiss University Hospital %A Musy,Sarah N %A Endrich,Olga %A Leichtle,Alexander B %A Griffiths,Peter %A Nakas,Christos T %A Simon,Michael %+ Institute of Nursing Science, University of Basel, Bernoullistrasse 28, Basel, 4057, Switzerland, 41 61 267 09 12, m.simon@unibas.ch %K patient safety %K electronic health records %K nurse staffing %K workload %K routine data %D 2020 %7 2.4.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Variations in patient demand increase the challenge of balancing high-quality nursing skill mixes against budgetary constraints. Developing staffing guidelines that allow high-quality care at minimal cost requires first exploring the dynamic changes in nursing workload over the course of a day. Objective: Accordingly, this longitudinal study analyzed nursing care supply and demand in 30-minute increments over a period of 3 years. We assessed 5 care factors: patient count (care demand), nurse count (care supply), the patient-to-nurse ratio for each nurse group, extreme supply-demand mismatches, and patient turnover (ie, number of admissions, discharges, and transfers). Methods: Our retrospective analysis of data from the Inselspital University Hospital Bern, Switzerland included all inpatients and nurses working in their units from January 1, 2015 to December 31, 2017. Two data sources were used. The nurse staffing system (tacs) provided information about nurses and all the care they provided to patients, their working time, and admission, discharge, and transfer dates and times. The medical discharge data included patient demographics, further admission and discharge details, and diagnoses. Based on several identifiers, these two data sources were linked. Results: Our final dataset included more than 58 million data points for 128,484 patients and 4633 nurses across 70 units. Compared with patient turnover, fluctuations in the number of nurses were less pronounced. The differences mainly coincided with shifts (night, morning, evening). While the percentage of shifts with extreme staffing fluctuations ranged from fewer than 3% (mornings) to 30% (evenings and nights), the percentage within “normal” ranges ranged from fewer than 50% to more than 80%. Patient turnover occurred throughout the measurement period but was lowest at night. Conclusions: Based on measurements of patient-to-nurse ratio and patient turnover at 30-minute intervals, our findings indicate that the patient count, which varies considerably throughout the day, is the key driver of changes in the patient-to-nurse ratio. This demand-side variability challenges the supply-side mandate to provide safe and reliable care. Detecting and describing patterns in variability such as these are key to appropriate staffing planning. This descriptive analysis was a first step towards identifying time-related variables to be considered for a predictive nurse staffing model. %M 32238331 %R 10.2196/15554 %U https://www.jmir.org/2020/4/e15554 %U https://doi.org/10.2196/15554 %U http://www.ncbi.nlm.nih.gov/pubmed/32238331 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 3 %P e13075 %T Peak Outpatient and Emergency Department Visit Forecasting for Patients With Chronic Respiratory Diseases Using Machine Learning Methods: Retrospective Cohort Study %A Peng,Junfeng %A Chen,Chuan %A Zhou,Mi %A Xie,Xiaohua %A Zhou,Yuqi %A Luo,Ching-Hsing %+ School of Data and Computer Science, Sun Yat-sen University, National Supercomputing Guangzhou Center Bldg, 5th Fl, Guangzhou, 510006, China, 86 13265384134, luojinx5@mail.sysu.edu.cn %K chronic respiratory diseases %K ensemble machine learning %K health forecasting %K outpatient and emergency departments management %D 2020 %7 30.3.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: The overcrowding of hospital outpatient and emergency departments (OEDs) due to chronic respiratory diseases in certain weather or under certain environmental pollution conditions results in the degradation in quality of medical care, and even limits its availability. Objective: To help OED managers to schedule medical resource allocation during times of excessive health care demands after short-term fluctuations in air pollution and weather, we employed machine learning (ML) methods to predict the peak OED arrivals of patients with chronic respiratory diseases. Methods: In this paper, we first identified 13,218 visits from patients with chronic respiratory diseases to OEDs in hospitals from January 1, 2016, to December 31, 2017. Then, we divided the data into three datasets: weather-based visits, air quality-based visits, and weather air quality-based visits. Finally, we developed ML methods to predict the peak event (peak demand days) of patients with chronic respiratory diseases (eg, asthma, respiratory infection, and chronic obstructive pulmonary disease) visiting OEDs on the three weather data and environmental pollution datasets in Guangzhou, China. Results: The adaptive boosting-based neural networks, tree bag, and random forest achieved the biggest receiver operating characteristic area under the curve, 0.698, 0.714, and 0.809, on the air quality dataset, the weather dataset, and weather air quality dataset, respectively. Overall, random forests reached the best classification prediction performance. Conclusions: The proposed ML methods may act as a useful tool to adapt medical services in advance by predicting the peak of OED arrivals. Further, the developed ML methods are generic enough to cope with similar medical scenarios, provided that the data is available. %M 32224488 %R 10.2196/13075 %U http://medinform.jmir.org/2020/3/e13075/ %U https://doi.org/10.2196/13075 %U http://www.ncbi.nlm.nih.gov/pubmed/32224488 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 3 %P e16117 %T Predicting Adverse Outcomes for Febrile Patients in the Emergency Department Using Sparse Laboratory Data: Development of a Time Adaptive Model %A Lee,Sungjoo %A Hong,Sungjun %A Cha,Won Chul %A Kim,Kyunga %+ Department of Emergency Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, 81, Irwon-ro, Gangnam-gu, Seoul, , Republic of Korea, 82 10 5386 6597, wc.cha@samsung.com %K order status %K sparse laboratory data %K time adaptive model %K emergency department %K adverse outcome %K machine learning %K imbalanced data %D 2020 %7 26.3.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: A timely decision in the initial stages for patients with an acute illness is important. However, only a few studies have determined the prognosis of patients based on insufficient laboratory data during the initial stages of treatment. Objective: This study aimed to develop and validate time adaptive prediction models to predict the severity of illness in the emergency department (ED) using highly sparse laboratory test data (test order status and test results) and a machine learning approach. Methods: This retrospective study used ED data from a tertiary academic hospital in Seoul, Korea. Two different models were developed based on laboratory test data: order status only (OSO) and order status and results (OSR) models. A binary composite adverse outcome was used, including mortality or hospitalization in the intensive care unit. Both models were evaluated using various performance criteria, including the area under the receiver operating characteristic curve (AUC) and balanced accuracy (BA). Clinical usefulness was examined by determining the positive likelihood ratio (PLR) and negative likelihood ratio (NLR). Results: Of 9491 eligible patients in the ED (mean age, 55.2 years, SD 17.7 years; 4839/9491, 51.0% women), the model development cohort and validation cohort included 6645 and 2846 patients, respectively. The OSR model generally exhibited better performance (AUC=0.88, BA=0.81) than the OSO model (AUC=0.80, BA=0.74). The OSR model was more informative than the OSO model to predict patients at low or high risk of adverse outcomes (P<.001 for differences in both PLR and NLR). Conclusions: Early-stage adverse outcomes for febrile patients could be predicted using machine learning models of highly sparse data including test order status and laboratory test results. This prediction tool could help medical professionals who are simultaneously treating the same patient share information, lead dynamic communication, and consequently prevent medical errors. %M 32213477 %R 10.2196/16117 %U http://medinform.jmir.org/2020/3/e16117/ %U https://doi.org/10.2196/16117 %U http://www.ncbi.nlm.nih.gov/pubmed/32213477 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 8 %N 3 %P e16381 %T Performance Evaluation of an Information Technology Intervention Regarding Charging for Inpatient Medical Materials at a Regional Teaching Hospital in Taiwan: Empirical Study %A Liao,Min-Chi %A Lin,I-Chun %+ Department of Industrial Engineering and Management, National Yunlin University of Science and Technology, Yunlin, Taiwan, 886 05 5342601 ext 5126, ichunlin@yuntech.edu.tw %K Information System Success Model %K information technology intervention %K charging %K medical materials %K work performance %D 2020 %7 25.3.2020 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: The process of manually recording the consumption of medical materials can be time consuming and prone to omission owing to its detailed and complicated nature. Implementing an information system will better improve work performance. Objective: The Information System Success Model was adopted as the theoretical foundation. The opinions of nursing staff were collected to verify the impact of the system intervention on their work performance. Methods: This cross-sectional study was conducted at a regional teaching hospital. Nursing staff were invited to participate in the field survey. A total of 296 questionnaires were collected, and of these, 284 (95.9%) were valid and returned. Results: The key findings showed that two critical factors (“subjective norm” and “system quality”) had significant positive effects (both P<.001) on user satisfaction (R2=0.709). The path of “service quality” to “user satisfaction” showed marginal significance (P=.08) under the 92% CI. Finally, the explanatory power of the model reached 68.9%. Conclusions: Support from the top management, appointment of a nurse supervisor as the change agent, recruitment of seed members to establish a pioneer team, and promotion of the system through the influence of opinion leaders in small groups were critical success factors needed for implementing the system in the case hospital. The target system was proven to be able to improve work performance, and the time saved could be further used for patient care, thereby increasing the value of nursing work. The positive experiences gained from this study could lay the foundation for the further promotion of the new system, and this is for future studies to replicate. The example of the successful experience of the case hospital could also serve as a reference for other hospitals in developing countries like Taiwan with regard to the promotion of nursing informatization. %M 32209534 %R 10.2196/16381 %U http://mhealth.jmir.org/2020/3/e16381/ %U https://doi.org/10.2196/16381 %U http://www.ncbi.nlm.nih.gov/pubmed/32209534 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 3 %P e14855 %T A Late Attempt to Involve End Users in the Design of Medication-Related Alerts: Survey Study %A Baysari,Melissa Therese %A Zheng,Wu Yi %A Van Dort,Bethany %A Reid-Anderson,Hannah %A Gronski,Mihaela %A Kenny,Eliza %+ Faculty of Health Sciences, The University of Sydney, Charles Perkins Centre D17, Sydney, 2006, Australia, 61 28627 9245, melissa.baysari@sydney.edu.au %K alert fatigue %K alerting %K medication alert systems %K clinical decision support %K hospital information systems %D 2020 %7 13.3.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: When users of electronic medical records (EMRs) are presented with large numbers of irrelevant computerized alerts, they experience alert fatigue, begin to ignore alert information, and override alerts without processing or heeding alert recommendations. Anecdotally, doctors at our study site were dissatisfied with the medication-related alerts being generated, both in terms of volume being experienced and clinical relevance. Objective: This study aimed to involve end users in the redesign of medication-related alerts in a hospital EMR, 4 years post implementation. Methods: This work was undertaken at a private not-for-profit teaching hospital in Sydney, Australia. Since EMR implementation in 2015, the organization elected to implement all medication-related alert types available in the system for prescribers: allergy and intolerance alerts, therapeutic duplication alerts, pregnancy alerts, and drug-drug interaction alerts. The EMR included no medication administration alerts for nurses. To obtain feedback on current alerts and suggestions for redesign, a Web-based survey was distributed to all doctors and nurses at the site via hospital mailing lists. Results: Despite a general dissatisfaction with alerts, very few end users completed the survey. In total, only 3.37% (36/1066) of doctors and 14.5% (60/411) of nurses took part. Approximately 90% (30/33) of doctors who responded held the view that too many alerts were triggered in the EMR. Doctors suggested that most alerts be removed and that alerts be more specific and less sensitive. In contrast, 97% (58/60) of the nurse respondents indicated that they would like to receive medication administration alerts in the EMR. Most nurses indicated that they would like to receive all the alert types available at all severity levels. Conclusions: Attempting to engage with end users several years post implementation was challenging. Involving users so late in the implementation process may lead to clinicians viewing the provision of feedback to be futile. Seeking user feedback on usefulness, volume, and design of alerts is extremely valuable; however, we suggest this is undertaken early, preferably before system implementation. %M 32167479 %R 10.2196/14855 %U https://www.jmir.org/2020/3/e14855 %U https://doi.org/10.2196/14855 %U http://www.ncbi.nlm.nih.gov/pubmed/32167479 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 3 %P e15002 %T Improvement of the Efficiency and Completeness of Neuro-Oncology Patient Referrals to a Tertiary Center Through the Implementation of an Electronic Referral System: Retrospective Cohort Study %A Fernández-Méndez,Rocío %A Wong,Mei Yin %A Rastall,Rebecca J %A Rebollo-Díaz,Samuel %A Oberg,Ingela %A Price,Stephen J %A Joannides,Alexis J %+ Department of Clinical Neurosciences, University of Cambridge, Box 165, A Block, Level 3, B Spur, Cambridge Biomedical Campus, Cambridge, CB2 0QQ, United Kingdom, 44 1223746466, rociofmendez.inv@gmail.com %K quality improvement %K electronic health records %K hospital referral %K hospital oncology services %D 2020 %7 5.3.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Quality referrals to specialist care are key for prompt, optimal decisions about the management of patients with brain tumors. Objective: This study aimed to determine the impact of introducing a Web-based, electronic referral (eReferral) system to a specialized neuro-oncology center, using a service-developed proforma, in terms of waiting times and information completeness. Methods: We carried out a retrospective cohort study based on the review of medical records of referred adult patients, excluding follow-ups. Primary outcome measures were durations of three key phases within the referral pathway and completion rates of six referral fields. Results: A total of 248 patients were referred to the specialist center during the study period. Median (IQR) diagnostic imaging to referral intervals were 3 (1-5) days with eReferrals, and 9 (4-19), 19 (14-49), and 8 (4-23) days with paper proforma, paper letter, and internal referrals, respectively (P<.001). Median (IQR) referral to multidisciplinary team decision intervals were 3 (2-7), 2 (1-3), 8 (2-24), and 3 (2-6) days respectively (P=.01). For patients having surgery, median (IQR) diagnostic imaging to surgery intervals were 28 (21-41), 34 (27-51), 104 (69-143), and 32 (15-89) days, respectively (P<.001). Proportions of complete fields differed significantly by referral type in all study fields (all with Ps <.001) except for details of presentation, which were present in all referrals. All study fields were always present in eReferrals, as these are compulsory for referral submission. Depending on the data field, level of completeness in the remaining referral types ranged within 69% (65/94) to 87% (82/94), 15% (3/20) to 65% (13/20), and 22% (8/41) to 63% (26/41) in paper proforma, paper letter, and internal referrals, respectively. Conclusions: An electronic, Web-based, service-developed specific proforma for neuro-oncology referrals performs significantly better, with shorter waiting times and greater completeness of information than other referral types. A wider application of eReferrals is an important first step to streamlining specialist care pathways and providing excellent care. International Registered Report Identifier (IRRID): RR2-10.2196/10.2196/15002 %M 32134389 %R 10.2196/15002 %U https://www.jmir.org/2020/3/e15002 %U https://doi.org/10.2196/15002 %U http://www.ncbi.nlm.nih.gov/pubmed/32134389 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 2 %P e16932 %T A Physician-Completed Digital Tool for Evaluating Disease Progression (Multiple Sclerosis Progression Discussion Tool): Validation Study %A Ziemssen,Tjalf %A Piani-Meier,Daniela %A Bennett,Bryan %A Johnson,Chloe %A Tinsley,Katie %A Trigg,Andrew %A Hach,Thomas %A Dahlke,Frank %A Tomic,Davorka %A Tolley,Chloe %A Freedman,Mark S %+ Center of Clinical Neuroscience, Neurological University Clinic Carl Gustav Carus, TU Dresden, Fetscherstraße 74, Dresden, Germany, 49 351 458 4465, ziemssen@web.de %K multiple sclerosis %K relapsing-remitting multiple sclerosis %K secondary progressive multiple sclerosis %K transition %K progression %K digital %K validation %D 2020 %7 12.2.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Defining the transition from relapsing-remitting multiple sclerosis (RRMS) to secondary progressive multiple sclerosis (SPMS) can be challenging and delayed. A digital tool (MSProDiscuss) was developed to facilitate physician-patient discussion in evaluating early, subtle signs of multiple sclerosis (MS) disease progression representing this transition. Objective: This study aimed to determine cut-off values and corresponding sensitivity and specificity for predefined scoring algorithms, with or without including Expanded Disability Status Scale (EDSS) scores, to differentiate between RRMS and SPMS patients and to evaluate psychometric properties. Methods: Experienced neurologists completed the tool for patients with confirmed RRMS or SPMS and those suspected to be transitioning to SPMS. In addition to age and EDSS score, each patient’s current disease status (disease activity, symptoms, and its impacts on daily life) was collected while completing the draft tool. Receiver operating characteristic (ROC) curves determined optimal cut-off values (sensitivity and specificity) for the classification of RRMS and SPMS. Results: Twenty neurologists completed the draft tool for 198 patients. Mean scores for patients with RRMS (n=89), transitioning to SPMS (n=47), and SPMS (n=62) were 38.1 (SD 12.5), 55.2 (SD 11.1), and 69.6 (SD 12.0), respectively (P<.001, each between-groups comparison). Area under the ROC curve (AUC) including and excluding EDSS were for RRMS (including) AUC 0.91, 95% CI 0.87-0.95, RRMS (excluding) AUC 0.88, 95% CI 0.84-0.93, SPMS (including) AUC 0.91, 95% CI 0.86-0.95, and SPMS (excluding) AUC 0.86, 95% CI 0.81-0.91. In the algorithm with EDSS, the optimal cut-off values were ≤51.6 for RRMS patients (sensitivity=0.83; specificity=0.82) and ≥58.9 for SPMS patients (sensitivity=0.82; specificity=0.84). The optimal cut-offs without EDSS were ≤46.3 and ≥57.8 and resulted in similar high sensitivity and specificity (0.76-0.86). The draft tool showed excellent interrater reliability (intraclass correlation coefficient=.95). Conclusions: The MSProDiscuss tool differentiated RRMS patients from SPMS patients with high sensitivity and specificity. In clinical practice, it may be a useful tool to evaluate early, subtle signs of MS disease progression indicating the evolution of RRMS to SPMS. MSProDiscuss will help assess the current level of progression in an individual patient and facilitate a more informed physician-patient discussion. %M 32049062 %R 10.2196/16932 %U https://www.jmir.org/2020/2/e16932 %U https://doi.org/10.2196/16932 %U http://www.ncbi.nlm.nih.gov/pubmed/32049062 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 2 %P e17194 %T A Virtual Multidisciplinary Care Program for Management of Advanced Chronic Kidney Disease: Matched Cohort Study %A Kaiser,Paulina %A Pipitone,Olivia %A Franklin,Anthony %A Jackson,Dixie R %A Moore,Elizabeth A %A Dubuque,Christopher R %A Peralta,Carmen A %A De Mory,Anthony C %+ Cricket Health, 251 Kearny St, Floor 7, San Francisco, CA, 94108, United States, 1 (888) 780 0253, carmen@crickethealth.com %K chronic kidney disease %K end-stage renal disease %K online social networking %K patient education %K renal dialysis %D 2020 %7 12.2.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: It is not well established whether a virtual multidisciplinary care program for persons with advanced chronic kidney disease (CKD) can improve their knowledge about their disease, increase their interest in home dialysis therapies, and result in more planned outpatient (versus inpatient) dialysis starts. Objective: We aimed to evaluate the feasibility and preliminary associations of program participation with disease knowledge, home dialysis modality preference, and outpatient dialysis initiation among persons with advanced CKD in a community-based nephrology practice. Methods: In a matched prospective cohort, we enrolled adults aged 18 to 85 years with at least two estimated glomerular filtration rates (eGFRs) of less than 30 mL/min/1.73 m2 into the Cricket Health program and compared them with controls receiving care at the same clinic, matched on age, gender, eGFR, and presence of heart failure and diabetes. The intervention included online education materials, a virtual multidisciplinary team (nurse, pharmacist, social worker, dietician), and patient mentors. Prespecified follow-up time was nine months with extended follow-up to allow adequate time to determine the dialysis start setting. CKD knowledge and dialysis modality choice were evaluated in a pre-post survey among intervention participants. Results: Thirty-seven participants were matched to 61 controls by age (mean 67.2, SD 10.4 versus mean 68.8, SD 9.5), prevalence of diabetes (54%, 20/37 versus 57%, 35/61), congestive heart failure (22%, 8/37 versus 25%, 15/61), and baseline eGFR (mean 19, SD 6 versus mean 21, SD 5 mL/min/1.73 m2), respectively. At nine-month follow-up, five patients in each group started dialysis (P=.62). Among program participants, 80% (4/5) started dialysis as an outpatient compared with 20% (1/5) of controls (OR 6.28, 95% CI 0.69-57.22). In extended follow-up (median 15.7, range 11.7 to 18.1 months), 19 of 98 patients started dialysis; 80% (8/10) of the intervention group patients started dialysis in the outpatient setting versus 22% (2/9) of control patients (hazard ratio 6.89, 95% CI 1.46-32.66). Compared to before participation, patients who completed the program had higher disease knowledge levels (mean 52%, SD 29% versus mean 94%, SD 14% of questions correct on knowledge-based survey, P<.001) and were more likely to choose a home modality as their first dialysis choice (36%, 7/22 versus 68%, 15/22, P=.047) after program completion. Conclusions: The Cricket Health program can improve patient knowledge about CKD and increase interest in home dialysis modalities, and may increase the proportion of dialysis starts in the outpatient setting. %M 32049061 %R 10.2196/17194 %U http://www.jmir.org/2020/2/e17194/ %U https://doi.org/10.2196/17194 %U http://www.ncbi.nlm.nih.gov/pubmed/32049061 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 12 %P e15166 %T Digital Alerting and Outcomes in Patients With Sepsis: Systematic Review and Meta-Analysis %A Joshi,Meera %A Ashrafian,Hutan %A Arora,Sonal %A Khan,Sadia %A Cooke,Graham %A Darzi,Ara %+ Chelsea and Westminster Hospital, NHS Foundation Trust, 369 Fulham Rd, Chelsea, London, SW10 9NH, United Kingdom, 44 2033158000, meera.joshi03@imperial.ac.uk %K diagnosis %K electronic health records, sepsis %K medical order entry systems, outcome assessment (health care) %D 2019 %7 20.12.2019 %9 Review %J J Med Internet Res %G English %X Background: The diagnosis and management of sepsis remain a global health care challenge. Digital technologies have the potential to improve sepsis care. Objective: The aim of this paper was to systematically review the evidence on the impact of digital alerting systems on sepsis related outcomes. Methods: The following databases were searched for studies published from April 1964 to February 12, 2019, with no language restriction: EMBASE, MEDLINE, HMIC, PsycINFO, and Cochrane. All full-text reports of studies identified as potentially eligible after title and abstract reviews were obtained for further review. The search was limited to adult inpatients. Relevant articles were hand searched for other studies. Only studies with clear pre- and postalerting phases were included. Primary outcomes were hospital length of stay (LOS) and intensive care LOS, whereas secondary outcomes were time to antibiotics and mortality. Studies based solely on intensive care, case reports, narrative reviews, editorials, and commentaries were excluded. All other trial designs were included. A qualitative assessment and meta-analysis were performed. Results: This review identified 72 full-text articles. From these, 16 studies met the inclusion criteria and were included in the final analysis. Of these, 8 studies reviewed hospital LOS, 12 reviewed mortality outcomes, 5 studies explored time to antibiotics, and 5 studies investigated intensive care unit (ICU) LOS. Both quantitative and qualitative assessments of the studies were performed. There was evidence of a significant benefit of digital alerting in hospital LOS, which reduced by 1.31 days (P=.014), and ICU LOS, which reduced by 0.766 days (P=.007). There was no significant association between digital alerts and mortality (mean decrease 11.4%; P=.77) or time to antibiotics (mean decrease 126 min; P=.13). Conclusions: This review highlights that digital alerts can considerably reduce hospital and ICU stay for patients with sepsis. Further studies including randomized controlled trials are necessary to confirm these findings and identify the choice of alerting system according to the patient status and pathological cohort. %M 31859672 %R 10.2196/15166 %U http://www.jmir.org/2019/12/e15166/ %U https://doi.org/10.2196/15166 %U http://www.ncbi.nlm.nih.gov/pubmed/31859672 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 4 %P e15794 %T Identifying Opioid Use Disorder in the Emergency Department: Multi-System Electronic Health Record–Based Computable Phenotype Derivation and Validation Study %A Chartash,David %A Paek,Hyung %A Dziura,James D %A Ross,Bill K %A Nogee,Daniel P %A Boccio,Eric %A Hines,Cory %A Schott,Aaron M %A Jeffery,Molly M %A Patel,Mehul D %A Platts-Mills,Timothy F %A Ahmed,Osama %A Brandt,Cynthia %A Couturier,Katherine %A Melnick,Edward %+ Department of Emergency Medicine, Yale University School of Medicine, 464 Congress Ave, Suite 260, New Haven, CT, 06519, United States, 1 2037855174, edward.melnick@yale.edu %K electronic health records %K emergency medicine %K algorithms %K phenotype %K opioid-related disorders %D 2019 %7 31.10.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: Deploying accurate computable phenotypes in pragmatic trials requires a trade-off between precise and clinically sensical variable selection. In particular, evaluating the medical encounter to assess a pattern leading to clinically significant impairment or distress indicative of disease is a difficult modeling challenge for the emergency department. Objective: This study aimed to derive and validate an electronic health record–based computable phenotype to identify emergency department patients with opioid use disorder using physician chart review as a reference standard. Methods: A two-algorithm computable phenotype was developed and evaluated using structured clinical data across 13 emergency departments in two large health care systems. Algorithm 1 combined clinician and billing codes. Algorithm 2 used chief complaint structured data suggestive of opioid use disorder. To evaluate the algorithms in both internal and external validation phases, two emergency medicine physicians, with a third acting as adjudicator, reviewed a pragmatic sample of 231 charts: 125 internal validation (75 positive and 50 negative), 106 external validation (56 positive and 50 negative). Results: Cohen kappa, measuring agreement between reviewers, for the internal and external validation cohorts was 0.95 and 0.93, respectively. In the internal validation phase, Algorithm 1 had a positive predictive value (PPV) of 0.96 (95% CI 0.863-0.995) and a negative predictive value (NPV) of 0.98 (95% CI 0.893-0.999), and Algorithm 2 had a PPV of 0.8 (95% CI 0.593-0.932) and an NPV of 1.0 (one-sided 97.5% CI 0.863-1). In the external validation phase, the phenotype had a PPV of 0.95 (95% CI 0.851-0.989) and an NPV of 0.92 (95% CI 0.807-0.978). Conclusions: This phenotype detected emergency department patients with opioid use disorder with high predictive values and reliability. Its algorithms were transportable across health care systems and have potential value for both clinical and research purposes. %M 31674913 %R 10.2196/15794 %U http://medinform.jmir.org/2019/4/e15794/ %U https://doi.org/10.2196/15794 %U http://www.ncbi.nlm.nih.gov/pubmed/31674913 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 4 %P e13085 %T Fast Prediction of Deterioration and Death Risk in Patients With Acute Exacerbation of Chronic Obstructive Pulmonary Disease Using Vital Signs and Admission History: Retrospective Cohort Study %A Zhou,Mi %A Chen,Chuan %A Peng,Junfeng %A Luo,Ching-Hsing %A Feng,Ding Yun %A Yang,Hailing %A Xie,Xiaohua %A Zhou,Yuqi %+ Department of Respiratory and Critical Care Medicine, The Third Affiliated Hospital of Sun Yat-sen University, No 600 Tianhe Road, Tianhe District, Guangzhou, 510640, China, 86 13533489943, zhouyuqi@mail.sysu.edu.cn %K chronic obstructive pulmonary disease %K clinical decision support systems %K health risk assessment %D 2019 %7 21.10.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: Chronic obstructive pulmonary disease (COPD) has 2 courses with different options for medical treatment: the acute exacerbation phase and the stable phase. Stable patients can use the Global Initiative for Chronic Obstructive Lung Disease (GOLD) to guide treatment strategies. However, GOLD could not classify and guide the treatment of acute exacerbation as acute exacerbation of COPD (AECOPD) is a complex process. Objective: This paper aimed to propose a fast severity assessment and risk prediction approach in order to strengthen monitoring and medical interventions in advance. Methods: The proposed method uses a classification and regression tree (CART) and had been validated using the AECOPD inpatient’s medical history and first measured vital signs at admission that can be collected within minutes. We identified 552 inpatients with AECOPD from February 2011 to June 2018 retrospectively and used the classifier to predict the outcome and prognosis of this hospitalization. Results: The overall accuracy of the proposed CART classifier was 76.2% (83/109 participants) with 95% CI 0.67-0.84. The precision, recall, and F-measure for the mild AECOPD were 76% (50/65 participants), 82% (50/61 participants), and 0.79, respectively, and those with severe AECOPD were 75% (33/44 participants), 68% (33/48 participants), and 0.72, respectively. Conclusions: This fast prediction CART classifier for early exacerbation detection could trigger the initiation of timely treatment, thereby potentially reducing exacerbation severity and recovery time and improving the patients’ health. %M 31638595 %R 10.2196/13085 %U http://medinform.jmir.org/2019/4/e13085/ %U https://doi.org/10.2196/13085 %U http://www.ncbi.nlm.nih.gov/pubmed/31638595 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 9 %P e14135 %T Health Care Professionals’ Perspectives on the Secondary Use of Health Records to Improve Quality and Safety of Care in England: Qualitative Study %A Neves,Ana Luísa %A Poovendran,Dilkushi %A Freise,Lisa %A Ghafur,Saira %A Flott,Kelsey %A Darzi,Ara %A Mayer,Erik K %+ Patient Safety Translational Research Centre, Institute of Global Health Innovation, Imperial College London, St Mary’s Campus, Queen Elizabeth Queen Mother Wing, London, W2 1NY, United Kingdom, 44 (0)20 7589 5111, ana.luisa.neves14@ic.ac.uk %K electronic health records %K information technology %K health policy %K safety culture %D 2019 %7 26.9.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Health care professionals (HCPs) are often patients’ first point of contact in what concerns the communication of the purposes, benefits, and risks of sharing electronic health records (EHRs) for nondirect care purposes. Their engagement is fundamental to ensure patients’ buy-in and a successful implementation of health care data sharing schemes. However, their views on this subject are seldom evaluated. Objective: This study aimed to explore HCPs’ perspectives on the secondary uses of health care data in England. Specifically, we aimed to assess their knowledge on its purposes and the main concerns about data sharing processes. Methods: A total of 30 interviews were conducted between March 27, 2017, and April 7, 2017, using a Web-based interview platform and following a topic guide with open-ended questions. The participants represented a variety of geographic locations across England (London, West Midlands, East of England, North East England, and Yorkshire and the Humber), covering both primary and secondary care services. The transcripts were compiled verbatim and systematically reviewed by 2 independent reviewers using the framework analysis method to identify emerging themes. Results: HCPs were knowledgeable about the possible secondary uses of data and highlighted its importance for patient profiling and tailored care, research, quality assurance, public health, and service delivery planning purposes. Main concerns toward data sharing included data accuracy, patients’ willingness to share their records, challenges on obtaining free and informed consent, data security, lack of adequacy or understanding of current policies, and potential patient exposure and exploitation. Conclusions: These results suggest a high level of HCPs’ understanding about the purposes of data sharing for secondary purposes; however, some concerns still remain. A better understanding of HCPs’ knowledge and concerns could inform national communication policies and improve tailoring to maximize efficiency and improve patients’ buy-in. %M 31573898 %R 10.2196/14135 %U https://www.jmir.org/2019/9/e14135 %U https://doi.org/10.2196/14135 %U http://www.ncbi.nlm.nih.gov/pubmed/31573898 %0 Journal Article %@ 2561-9128 %I JMIR Publications %V 2 %N 2 %P e14501 %T Impact of an Intensive Care Information System on the Length of Stay of Surgical Intensive Care Unit Patients: Observational Study %A Havel,Camille %A Selim,Jean %A Besnier,Emmanuel %A Gouin,Philippe %A Veber,Benoit %A Clavier,Thomas %+ Department of Anesthesiology and Critical Care, Rouen University Hospital, 1 rue de Germont, Rouen, 76000, France, 33 2 32 88 89 90, camille.havel@chu-rouen.fr %K intensive care unit %K length of stay %K software %K critically ill patient %D 2019 %7 04.09.2019 %9 Original Paper %J JMIR Perioper Med %G English %X Background: The implementation of computerized monitoring and prescription systems in intensive care has proven to be reliable in reducing the rate of medical error and increasing patient care time. They also showed a benefit in reducing the length of stay in the intensive care unit (ICU). However, this benefit has been poorly studied, with conflicting results. Objective: This study aimed to show the impact of computerization on the length of stay in ICUs. Methods: This was a before-after retrospective observational study. All patients admitted in the surgical ICU at the Rouen University Hospital were included, from June 1, 2015, to June 1, 2016, for the before period and from August 1, 2016, to August 1, 2017, for the after period. The data were extracted from the hospitalization report and included the following: epidemiological data (age, sex, weight, height, and body mass index), reason for ICU admission, severity score at admission, length of stay and mortality in ICU, mortality in hospital, use of life support during the stay, and ICU readmission during the same hospital stay. The consumption of antibiotics, biological analyses, and the number of chest x-rays during the stay were also analyzed. Results: A total of 1600 patients were included: 839 in the before period and 761 in the after period. Only the severity score Simplified Acute Physiology Score II was significantly higher in the postcomputerization period (38 [SD 20] vs 40 [SD 21]; P<.05). There was no significant difference in terms of length of stay in ICU, mortality, or readmission during the stay. There was a significant increase in the volume of prescribed biological analyses (5416 [5192-5956] biological exams prescribed in the period before Intellispace Critical Care and Anesthesia [ICCA] vs 6374 [6013-6986] biological exams prescribed in the period after ICCA; P=.002), with an increase in the total cost of biological analyses, to the detriment of hematological and biochemical blood tests. There was also a trend toward reduction in the average number of chest x-rays, but this was not significant (0.55 [SD 0.39] chest x-rays per day per patient before computerization vs 0.51 [SD 0.37] chest x-rays per day per patient after computerization; P=.05). On the other hand, there was a decrease in antibiotic prescribing in terms of cost per patient after the implementation of computerization (€149.50 [$164 USD] per patient before computerization vs €105.40 [$155 USD] per patient after computerization). Conclusions: Implementation of an intensive care information system at the Rouen University Hospital in June 2016 did not have an impact on reducing the length of stay. %M 33393935 %R 10.2196/14501 %U http://periop.jmir.org/2019/2/e14501/ %U https://doi.org/10.2196/14501 %U http://www.ncbi.nlm.nih.gov/pubmed/33393935 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 7 %P e13147 %T Implementation of a Digitally Enabled Care Pathway (Part 1): Impact on Clinical Outcomes and Associated Health Care Costs %A Connell,Alistair %A Raine,Rosalind %A Martin,Peter %A Barbosa,Estela Capelas %A Morris,Stephen %A Nightingale,Claire %A Sadeghi-Alavijeh,Omid %A King,Dominic %A Karthikesalingam,Alan %A Hughes,Cían %A Back,Trevor %A Ayoub,Kareem %A Suleyman,Mustafa %A Jones,Gareth %A Cross,Jennifer %A Stanley,Sarah %A Emerson,Mary %A Merrick,Charles %A Rees,Geraint %A Montgomery,Hugh %A Laing,Christopher %+ Royal Free London NHS Foundation Trust, Pond Street, London,, United Kingdom, 44 020 7794 0500 ext 33322, chris.laing@nhs.net %K nephrology %K acute kidney injury %D 2019 %7 15.7.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: The development of acute kidney injury (AKI) in hospitalized patients is associated with adverse outcomes and increased health care costs. Simple automated e-alerts indicating its presence do not appear to improve outcomes, perhaps because of a lack of explicitly defined integration with a clinical response. Objective: We sought to test this hypothesis by evaluating the impact of a digitally enabled intervention on clinical outcomes and health care costs associated with AKI in hospitalized patients. Methods: We developed a care pathway comprising automated AKI detection, mobile clinician notification, in-app triage, and a protocolized specialist clinical response. We evaluated its impact by comparing data from pre- and postimplementation phases (May 2016 to January 2017 and May to September 2017, respectively) at the intervention site and another site not receiving the intervention. Clinical outcomes were analyzed using segmented regression analysis. The primary outcome was recovery of renal function to ≤120% of baseline by hospital discharge. Secondary clinical outcomes were mortality within 30 days of alert, progression of AKI stage, transfer to renal/intensive care units, hospital re-admission within 30 days of discharge, dependence on renal replacement therapy 30 days after discharge, and hospital-wide cardiac arrest rate. Time taken for specialist review of AKI alerts was measured. Impact on health care costs as defined by Patient-Level Information and Costing System data was evaluated using difference-in-differences (DID) analysis. Results: The median time to AKI alert review by a specialist was 14.0 min (interquartile range 1.0-60.0 min). There was no impact on the primary outcome (estimated odds ratio [OR] 1.00, 95% CI 0.58-1.71; P=.99). Although the hospital-wide cardiac arrest rate fell significantly at the intervention site (OR 0.55, 95% CI 0.38-0.76; P<.001), DID analysis with the comparator site was not significant (OR 1.13, 95% CI 0.63-1.99; P=.69). There was no impact on other secondary clinical outcomes. Mean health care costs per patient were reduced by £2123 (95% CI −£4024 to −£222; P=.03), not including costs of providing the technology. Conclusions: The digitally enabled clinical intervention to detect and treat AKI in hospitalized patients reduced health care costs and possibly reduced cardiac arrest rates. Its impact on other clinical outcomes and identification of the active components of the pathway requires clarification through evaluation across multiple sites. %M 31368447 %R 10.2196/13147 %U http://www.jmir.org/2019/7/e13147/ %U https://doi.org/10.2196/13147 %U http://www.ncbi.nlm.nih.gov/pubmed/31368447 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 7 %P e13143 %T Implementation of a Digitally Enabled Care Pathway (Part 2): Qualitative Analysis of Experiences of Health Care Professionals %A Connell,Alistair %A Black,Georgia %A Montgomery,Hugh %A Martin,Peter %A Nightingale,Claire %A King,Dominic %A Karthikesalingam,Alan %A Hughes,Cían %A Back,Trevor %A Ayoub,Kareem %A Suleyman,Mustafa %A Jones,Gareth %A Cross,Jennifer %A Stanley,Sarah %A Emerson,Mary %A Merrick,Charles %A Rees,Geraint %A Laing,Christopher %A Raine,Rosalind %+ Department of Applied Health Research, University College London, 1-19 Torrington Place, London, WC1E 7HB, United Kingdom, 44 (0) 20 7679 1713, r.raine@ucl.ac.uk %K nephrology %K acute kidney injury %D 2019 %7 15.7.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: One reason for the introduction of digital technologies into health care has been to try to improve safety and patient outcomes by providing real-time access to patient data and enhancing communication among health care professionals. However, the adoption of such technologies into clinical pathways has been less examined, and the impacts on users and the broader health system are poorly understood. We sought to address this by studying the impacts of introducing a digitally enabled care pathway for patients with acute kidney injury (AKI) at a tertiary referral hospital in the United Kingdom. A dedicated clinical response team—comprising existing nephrology and patient-at-risk and resuscitation teams—received AKI alerts in real time via Streams, a mobile app. Here, we present a qualitative evaluation of the experiences of users and other health care professionals whose work was affected by the implementation of the care pathway. Objective: The aim of this study was to qualitatively evaluate the impact of mobile results viewing and automated alerting as part of a digitally enabled care pathway on the working practices of users and their interprofessional relationships. Methods: A total of 19 semistructured interviews were conducted with members of the AKI response team and clinicians with whom they interacted across the hospital. Interviews were analyzed using inductive and deductive thematic analysis. Results: The digitally enabled care pathway improved access to patient information and expedited early specialist care. Opportunities were identified for more constructive planning of end-of-life care due to the earlier detection and alerting of deterioration. However, the shift toward early detection also highlighted resource constraints and some clinical uncertainty about the value of intervening at this stage. The real-time availability of information altered communication flows within and between clinical teams and across professional groups. Conclusions: Digital technologies allow early detection of adverse events and of patients at risk of deterioration, with the potential to improve outcomes. They may also increase the efficiency of health care professionals’ working practices. However, when planning and implementing digital information innovations in health care, the following factors should also be considered: the provision of clinical training to effectively manage early detection, resources to cope with additional workload, support to manage perceived information overload, and the optimization of algorithms to minimize unnecessary alerts. %M 31368443 %R 10.2196/13143 %U http://www.jmir.org/2019/7/e13143/ %U https://doi.org/10.2196/13143 %U http://www.ncbi.nlm.nih.gov/pubmed/31368443 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 3 %P e14499 %T Projection Word Embedding Model With Hybrid Sampling Training for Classifying ICD-10-CM Codes: Longitudinal Observational Study %A Lin,Chin %A Lou,Yu-Sheng %A Tsai,Dung-Jang %A Lee,Chia-Cheng %A Hsu,Chia-Jung %A Wu,Ding-Chung %A Wang,Mei-Chuen %A Fang,Wen-Hui %+ Department of Family and Community Medicine, Tri-Service General Hospital, National Defense Medical Center, No. 325, Section 2, Chenggong Road, Neihu District, Taipei, 11490, Taiwan, 886 02 87923100 ext 18448, rumaf.fang@gmail.com %K word embedding %K convolutional neural network %K artificial intelligence %K natural language processing %K electronic health records %D 2019 %7 23.7.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: Most current state-of-the-art models for searching the International Classification of Diseases, Tenth Revision Clinical Modification (ICD-10-CM) codes use word embedding technology to capture useful semantic properties. However, they are limited by the quality of initial word embeddings. Word embedding trained by electronic health records (EHRs) is considered the best, but the vocabulary diversity is limited by previous medical records. Thus, we require a word embedding model that maintains the vocabulary diversity of open internet databases and the medical terminology understanding of EHRs. Moreover, we need to consider the particularity of the disease classification, wherein discharge notes present only positive disease descriptions. Objective: We aimed to propose a projection word2vec model and a hybrid sampling method. In addition, we aimed to conduct a series of experiments to validate the effectiveness of these methods. Methods: We compared the projection word2vec model and traditional word2vec model using two corpora sources: English Wikipedia and PubMed journal abstracts. We used seven published datasets to measure the medical semantic understanding of the word2vec models and used these embeddings to identify the three–character-level ICD-10-CM diagnostic codes in a set of discharge notes. On the basis of embedding technology improvement, we also tried to apply the hybrid sampling method to improve accuracy. The 94,483 labeled discharge notes from the Tri-Service General Hospital of Taipei, Taiwan, from June 1, 2015, to June 30, 2017, were used. To evaluate the model performance, 24,762 discharge notes from July 1, 2017, to December 31, 2017, from the same hospital were used. Moreover, 74,324 additional discharge notes collected from seven other hospitals were tested. The F-measure, which is the major global measure of effectiveness, was adopted. Results: In medical semantic understanding, the original EHR embeddings and PubMed embeddings exhibited superior performance to the original Wikipedia embeddings. After projection training technology was applied, the projection Wikipedia embeddings exhibited an obvious improvement but did not reach the level of original EHR embeddings or PubMed embeddings. In the subsequent ICD-10-CM coding experiment, the model that used both projection PubMed and Wikipedia embeddings had the highest testing mean F-measure (0.7362 and 0.6693 in Tri-Service General Hospital and the seven other hospitals, respectively). Moreover, the hybrid sampling method was found to improve the model performance (F-measure=0.7371/0.6698). Conclusions: The word embeddings trained using EHR and PubMed could understand medical semantics better, and the proposed projection word2vec model improved the ability of medical semantics extraction in Wikipedia embeddings. Although the improvement from the projection word2vec model in the real ICD-10-CM coding task was not substantial, the models could effectively handle emerging diseases. The proposed hybrid sampling method enables the model to behave like a human expert. %R 10.2196/14499 %U http://medinform.jmir.org/2019/3/e14499/ %U https://doi.org/10.2196/14499 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 7 %P e13719 %T A Real-Time Early Warning System for Monitoring Inpatient Mortality Risk: Prospective Study Using Electronic Medical Record Data %A Ye,Chengyin %A Wang,Oliver %A Liu,Modi %A Zheng,Le %A Xia,Minjie %A Hao,Shiying %A Jin,Bo %A Jin,Hua %A Zhu,Chunqing %A Huang,Chao Jung %A Gao,Peng %A Ellrodt,Gray %A Brennan,Denny %A Stearns,Frank %A Sylvester,Karl G %A Widen,Eric %A McElhinney,Doff B %A Ling,Xuefeng %+ Department of Surgery, Stanford University, S370 Grant Bldg, 300 Pasteur Drive, Stanford, CA, 94305, United States, 1 6504279198, bxling@stanford.edu %K inpatients %K mortality %K risk assessment %K electronic health records %K machine learning %D 2019 %7 05.07.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: The rapid deterioration observed in the condition of some hospitalized patients can be attributed to either disease progression or imperfect triage and level of care assignment after their admission. An early warning system (EWS) to identify patients at high risk of subsequent intrahospital death can be an effective tool for ensuring patient safety and quality of care and reducing avoidable harm and costs. Objective: The aim of this study was to prospectively validate a real-time EWS designed to predict patients at high risk of inpatient mortality during their hospital episodes. Methods: Data were collected from the system-wide electronic medical record (EMR) of two acute Berkshire Health System hospitals, comprising 54,246 inpatient admissions from January 1, 2015, to September 30, 2017, of which 2.30% (1248/54,246) resulted in intrahospital deaths. Multiple machine learning methods (linear and nonlinear) were explored and compared. The tree-based random forest method was selected to develop the predictive application for the intrahospital mortality assessment. After constructing the model, we prospectively validated the algorithms as a real-time inpatient EWS for mortality. Results: The EWS algorithm scored patients’ daily and long-term risk of inpatient mortality probability after admission and stratified them into distinct risk groups. In the prospective validation, the EWS prospectively attained a c-statistic of 0.884, where 99 encounters were captured in the highest risk group, 69% (68/99) of whom died during the episodes. It accurately predicted the possibility of death for the top 13.3% (34/255) of the patients at least 40.8 hours before death. Important clinical utilization features, together with coded diagnoses, vital signs, and laboratory test results were recognized as impactful predictors in the final EWS. Conclusions: In this study, we prospectively demonstrated the capability of the newly-designed EWS to monitor and alert clinicians about patients at high risk of in-hospital death in real time, thereby providing opportunities for timely interventions. This real-time EWS is able to assist clinical decision making and enable more actionable and effective individualized care for patients’ better health outcomes in target medical facilities. %M 31278734 %R 10.2196/13719 %U https://www.jmir.org/2019/7/e13719/ %U https://doi.org/10.2196/13719 %U http://www.ncbi.nlm.nih.gov/pubmed/31278734 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 6 %P e12847 %T Visibility of Community Nursing Within an Administrative Health Classification System: Evaluation of Content Coverage %A Block,Lorraine J %A Currie,Leanne M %A Hardiker,Nicholas R %A Strudwick,Gillian %+ School of Nursing, University of British Columbia, T201-2211 Wesbrook Mall, Vancouver, BC, V6T 2B5, Canada, 1 604 822 7417, lori.block@ubc.ca %K World Health Organization %K classification %K nursing informatics %K medical informatics %K data collection %K terminology %K community health services %K standardized nursing terminology %D 2019 %7 26.06.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: The World Health Organization is in the process of developing an international administrative classification for health called the International Classification of Health Interventions (ICHI). The purpose of ICHI is to provide a tool for supporting intervention reporting and analysis at a global level for policy development and beyond. Nurses represent the largest resource carrying out clinical interventions in any health system. With the shift in nursing care from hospital to community settings in many countries, it is important to ensure that community nursing interventions are present in any international health information system. Thus, an investigation into the extent to which community nursing interventions were covered in ICHI was needed. Objective: The objectives of this study were to examine the extent to which International Classification for Nursing Practice (ICNP) community nursing interventions were represented in the ICHI administrative classification system, to identify themes related to gaps in coverage, and to support continued advancements in understanding the complexities of knowledge representation in standardized clinical terminologies and classifications. Methods: This descriptive study used a content mapping approach in 2 phases in 2018. A total of 187 nursing intervention codes were extracted from the ICNP Community Nursing Catalogue and mapped to ICHI. In phase 1, 2 coders completed independent mapping activities. In phase 2, the 2 coders compared each list and discussed concept matches until consensus on ICNP-ICHI match and on mapping relationship was reached. Results: The initial percentage agreement between the 2 coders was 47% (n=88), but reached 100% with consensus processes. After consensus was reached, 151 (81%) of the community nursing interventions resulted in an ICHI match. A total of 36 (19%) of community nursing interventions had no match to ICHI content. A total of 100 (53%) community nursing interventions resulted in a broader ICHI code, 9 (5%) resulted in a narrower ICHI code, and 42 (23%) were considered equivalent. ICNP concepts that were not represented in ICHI were thematically grouped into the categories family and caregivers, death and dying, and case management. Conclusions: Overall, the content mapping yielded similar results to other content mapping studies in nursing. However, it also found areas of missing concept coverage, difficulties with interterminology mapping, and further need to develop mapping methods. %M 31244480 %R 10.2196/12847 %U https://www.jmir.org/2019/6/e12847/ %U https://doi.org/10.2196/12847 %U http://www.ncbi.nlm.nih.gov/pubmed/31244480 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 5 %P e13504 %T Discovering Clinical Information Models Online to Promote Interoperability of Electronic Health Records: A Feasibility Study of OpenEHR %A Yang,Lin %A Huang,Xiaoshuo %A Li,Jiao %+ Institute of Medical Information / Medical Library, Chinese Academy of Medical Sciences & Peking Union Medical College, No 3 Yabao Road, Chaoyang District, Beijing, 100020, China, 86 18618461596, li.jiao@imicams.ac.cn %K openEHR %K clinical information model %K health information interoperability %K information retrieval %K probabilistic graphical model %D 2019 %7 28.05.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Clinical information models (CIMs) enabling semantic interoperability are crucial for electronic health record (EHR) data use and reuse. Dual model methodology, which distinguishes the CIMs from the technical domain, could help enable the interoperability of EHRs at the knowledge level. How to help clinicians and domain experts discover CIMs from an open repository online to represent EHR data in a standard manner becomes important. Objective: This study aimed to develop a retrieval method to identify CIMs online to represent EHR data. Methods: We proposed a graphical retrieval method and validated its feasibility using an online CIM repository: openEHR Clinical Knowledge Manager (CKM). First, we represented CIMs (archetypes) using an extended Bayesian network. Then, an inference process was run in the network to discover relevant archetypes. In the evaluation, we defined three retrieval tasks (medication, laboratory test, and diagnosis) and compared our method with three typical retrieval methods (BM25F, simple Bayesian network, and CKM), using mean average precision (MAP), average precision (AP), and precision at 10 (P@10) as evaluation metrics. Results: We downloaded all available archetypes from the CKM. Then, the graphical model was applied to represent the archetypes as a four-level clinical resources network. The network consisted of 5513 nodes, including 3982 data element nodes, 504 concept nodes, 504 duplicated concept nodes, and 523 archetype nodes, as well as 9867 edges. The results showed that our method achieved the best MAP (MAP=0.32), and the AP was almost equal across different retrieval tasks (AP=0.35, 0.31, and 0.30, respectively). In the diagnosis retrieval task, our method could successfully identify the models covering “diagnostic reports,” “problem list,” “patients background,” “clinical decision,” etc, as well as models that other retrieval methods could not find, such as “problems and diagnoses.” Conclusions: The graphical retrieval method we propose is an effective approach to meet the uncertainty of finding CIMs. Our method can help clinicians and domain experts identify CIMs to represent EHR data in a standard manner, enabling EHR data to be exchangeable and interoperable. %M 31140433 %R 10.2196/13504 %U http://www.jmir.org/2019/5/e13504/ %U https://doi.org/10.2196/13504 %U http://www.ncbi.nlm.nih.gov/pubmed/31140433 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 5 %P e12875 %T Developing the National Usability-Focused Health Information System Scale for Physicians: Validation Study %A Hyppönen,Hannele %A Kaipio,Johanna %A Heponiemi,Tarja %A Lääveri,Tinja %A Aalto,Anna-Mari %A Vänskä,Jukka %A Elovainio,Marko %+ National Institute for Health and Welfare, PO Box 30, Helsinki, 00271, Finland, 358 295247434, tarja.heponiemi@thl.fi %K physicians %K health information systems %K questionnaire %K validation studies %D 2019 %7 16.05.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Problems in the usability of health information systems (HISs) are well acknowledged, but research still lacks a validated questionnaire for measuring and monitoring different dimensions of usability of HISs. Such questionnaires are needed not only for research but also for developing usability of HISs from the viewpoint of end-user experiences. Objective: This study aimed to develop and test the validity of the questionnaire measuring the National Usability-Focused HIS-Scale (NuHISS) among a nationally representative sample of Finnish physicians. Methods: We utilized 2 cross-sectional data collected from a random sample of Finnish physicians in 2014 (N=3781; of which 2340 [61.9%] were women) and 2017 (N=4018; of which 2604 [64.8%] were women). Exploratory and confirmatory factor analyses (structural equation modeling [SEM]) were applied to test the structural validity of the NuHISS. As the concurrent validity measure, we used the self-reported overall quality of the electronic health record system (school grade) provided by the participants using marginal structural models. Results: The exploratory factor analyses with Varimax rotation suggested that the 7-factor solution did offer a good fit to the data in both samples (C2=2136.14 in 2014 and C2=2109.83 in 2017, both P<.001). Moreover, structural equation modelling analyses, using comparative fit index (CFI), Tucker-Lewis Index (TLI), Normed Fit Index (NFI), root mean squared error of approximation (RMSEA), and Standardized Root Mean square Residual (SRMR), showed that the 7-factor solution provided an acceptable fit in both samples (CFI=0.92/0.91, TLI=0.92/0.91, NFI=0.92/0.91, RMSEA=0.048/0.049, and SRMR=0.040/0.039). In addition, concurrent validity of this solution was shown to be acceptable. Ease of use, but also all other dimensions, was especially associated with overall quality reports independent of measured confounders. The 7-factor solution included dimensions of technical quality, information quality, feedback, ease of use, benefits, internal collaboration, and cross-organizational collaboration. Conclusions: NuHISS provides a useful tool for measuring usability of HISs among physicians and offers a valid measure for monitoring the long-term development of HISs on a large scale. The relative importance of items needs to be assessed against national electronic health policy goals and complemented with items that have remained outside the NuHISS from the questionnaire when appropriate. %M 31099336 %R 10.2196/12875 %U https://www.jmir.org/2019/5/e12875/ %U https://doi.org/10.2196/12875 %U http://www.ncbi.nlm.nih.gov/pubmed/31099336 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 2 %P e11499 %T Adapting State-of-the-Art Deep Language Models to Clinical Information Extraction Systems: Potentials, Challenges, and Solutions %A Zhou,Liyuan %A Suominen,Hanna %A Gedeon,Tom %+ Research School of Computer Science, College of Engineering and Computer Science, The Australian National University, 108 North Road, Canberra, 2600, Australia, 61 (02) 6125 5111, annjouno@gmail.com %K computer systems %K artificial intelligence %K deep learning %K information storage and retrieval %K medical informatics %K nursing records %K patient handoff %D 2019 %7 25.04.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: Deep learning (DL) has been widely used to solve problems with success in speech recognition, visual object recognition, and object detection for drug discovery and genomics. Natural language processing has achieved noticeable progress in artificial intelligence. This gives an opportunity to improve on the accuracy and human-computer interaction of clinical informatics. However, due to difference of vocabularies and context between a clinical environment and generic English, transplanting language models directly from up-to-date methods to real-world health care settings is not always satisfactory. Moreover, the legal restriction on using privacy-sensitive patient records hinders the progress in applying machine learning (ML) to clinical language processing. Objective: The aim of this study was to investigate 2 ways to adapt state-of-the-art language models to extracting patient information from free-form clinical narratives to populate a handover form at a nursing shift change automatically for proofing and revising by hand: first, by using domain-specific word representations and second, by using transfer learning models to adapt knowledge from general to clinical English. We have described the practical problem, composed it as an ML task known as information extraction, proposed methods for solving the task, and evaluated their performance. Methods: First, word representations trained from different domains served as the input of a DL system for information extraction. Second, the transfer learning model was applied as a way to adapt the knowledge learned from general text sources to the task domain. The goal was to gain improvements in the extraction performance, especially for the classes that were topically related but did not have a sufficient amount of model solutions available for ML directly from the target domain. A total of 3 independent datasets were generated for this task, and they were used as the training (101 patient reports), validation (100 patient reports), and test (100 patient reports) sets in our experiments. Results: Our system is now the state-of-the-art in this task. Domain-specific word representations improved the macroaveraged F1 by 3.4%. Transferring the knowledge from general English corpora to the task-specific domain contributed a further 7.1% improvement. The best performance in populating the handover form with 37 headings was the macroaveraged F1 of 41.6% and F1 of 81.1% for filtering out irrelevant information. Performance differences between this system and its baseline were statistically significant (P<.001; Wilcoxon test). Conclusions: To our knowledge, our study is the first attempt to transfer models from general deep models to specific tasks in health care and gain a significant improvement. As transfer learning shows its advantage over other methods, especially on classes with a limited amount of training data, less experts’ time is needed to annotate data for ML, which may enable good results even in resource-poor domains. %M 31021325 %R 10.2196/11499 %U http://medinform.jmir.org/2019/2/e11499/ %U https://doi.org/10.2196/11499 %U http://www.ncbi.nlm.nih.gov/pubmed/31021325 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 2 %P e12172 %T Patient-Sharing Relations in the Treatment of Diabetes and Their Implications for Health Information Exchange: Claims-Based Analysis %A Duftschmid,Georg %A Rinner,Christoph %A Sauter,Simone Katja %A Endel,Gottfried %A Klimek,Peter %A Mitsch,Christoph %A Heinzl,Harald %+ Section for Medical Information Management, Center for Medical Statistics, Informatics, and Intelligent Systems, Medical University of Vienna, Spitalgasse 23, Vienna, A-1090, Austria, 43 1 40400 ext 66960, georg.duftschmid@meduniwien.ac.at %K health information exchange %K professional-patient relations %K diabetes mellitus %K Austria %D 2019 %7 12.04.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: Health information exchange (HIE) among care providers who cooperate in the treatment of patients with diabetes mellitus (DM) has been rated as an important aspect of successful care. Patient-sharing relations among care providers permit inferences about corresponding information-sharing relations. Objectives: This study aimed to obtain information for an effective HIE platform design to be used in DM care by analyzing patient-sharing relations among various types of care providers (ToCPs), such as hospitals, pharmacies, and different outpatient specialists, within a nationwide claims dataset of Austrian DM patients. We focus on 2 parameters derived from patient-sharing networks: (1) the principal HIE partners of the different ToCPs involved in the treatment of DM and (2) the required participation rate of ToCPs in HIE platforms for the purpose of effective communication. Methods: The claims data of 7.9 million Austrian patients from 2006 to 2007 served as our data source. DM patients were identified by their medication. We established metrics for the quantification of our 2 parameters of interest. The principal HIE partners were derived from the portions of a care provider’s patient-sharing relations with different ToCPs. For the required participation rate of ToCPs in an HIE platform, we determine the concentration of patient-sharing relations among ToCPs. Our corresponding metrics are derived in analogy from existing work for the quantification of the continuity of care. Results: We identified 324,703 DM patients treated by 12,226 care providers; the latter were members of 16 ToCPs. On the basis of their score for 2 of our parameters, we categorized the ToCPs into low, medium, and high. For the most important HIE partner parameter, pharmacies, general practitioners (GPs), and laboratories were the representatives of the top group, that is, our care providers shared the highest numbers of DM patients with these ToCPs. For the required participation rate of type of care provide (ToCP) in HIE platform parameter, the concentration of DM patient-sharing relations with a ToCP tended to be inversely related to the ToCPs member count. Conclusions: We conclude that GPs, pharmacies, and laboratories should be core members of any HIE platform that supports DM care, as they are the most important DM patient-sharing partners. We further conclude that, for implementing HIE with ToCPs who have many members (in Austria, particularly GPs and pharmacies), an HIE solution with high participation rates from these ToCPs (ideally a nationwide HIE platform with obligatory participation of the concerned ToCPs) seems essential. This will raise the probability of HIE being achieved with any care provider of these ToCPs. As chronic diseases are rising because of aging societies, we believe that our quantification of HIE requirements in the treatment of DM can provide valuable insights for many industrial countries. %M 30977733 %R 10.2196/12172 %U http://medinform.jmir.org/2019/2/e12172/ %U https://doi.org/10.2196/12172 %U http://www.ncbi.nlm.nih.gov/pubmed/30977733 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 3 %P e13249 %T Genomic Common Data Model for Seamless Interoperation of Biomedical Data in Clinical Practice: Retrospective Study %A Shin,Seo Jeong %A You,Seng Chan %A Park,Yu Rang %A Roh,Jin %A Kim,Jang-Hee %A Haam,Seokjin %A Reich,Christian G %A Blacketer,Clair %A Son,Dae-Soon %A Oh,Seungbin %A Park,Rae Woong %+ Department of Biomedical Informatics, Ajou University School of Medicine, 206 World cup-ro, Yeongtong-gu, Suwon, 16499, Republic of Korea, 82 312194471, veritas@ajou.ac.kr %K high-throughput nucleotide sequencing %K databases, genetic %K multicenter study %K patient privacy %K data visualization %D 2019 %7 26.03.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Clinical sequencing data should be shared in order to achieve the sufficient scale and diversity required to provide strong evidence for improving patient care. A distributed research network allows researchers to share this evidence rather than the patient-level data across centers, thereby avoiding privacy issues. The Observational Medical Outcomes Partnership (OMOP) common data model (CDM) used in distributed research networks has low coverage of sequencing data and does not reflect the latest trends of precision medicine. Objective: The aim of this study was to develop and evaluate the feasibility of a genomic CDM (G-CDM), as an extension of the OMOP-CDM, for application of genomic data in clinical practice. Methods: Existing genomic data models and sequencing reports were reviewed to extend the OMOP-CDM to cover genomic data. The Human Genome Organisation Gene Nomenclature Committee and Human Genome Variation Society nomenclature were adopted to standardize the terminology in the model. Sequencing data of 114 and 1060 patients with lung cancer were obtained from the Ajou University School of Medicine database of Ajou University Hospital and The Cancer Genome Atlas, respectively, which were transformed to a format appropriate for the G-CDM. The data were compared with respect to gene name, variant type, and actionable mutations. Results: The G-CDM was extended into four tables linked to tables of the OMOP-CDM. Upon comparison with The Cancer Genome Atlas data, a clinically actionable mutation, p.Leu858Arg, in the EGFR gene was 6.64 times more frequent in the Ajou University School of Medicine database, while the p.Gly12Xaa mutation in the KRAS gene was 2.02 times more frequent in The Cancer Genome Atlas dataset. The data-exploring tool GeneProfiler was further developed to conduct descriptive analyses automatically using the G-CDM, which provides the proportions of genes, variant types, and actionable mutations. GeneProfiler also allows for querying the specific gene name and Human Genome Variation Society nomenclature to calculate the proportion of patients with a given mutation. Conclusions: We developed the G-CDM for effective integration of genomic data with standardized clinical data, allowing for data sharing across institutes. The feasibility of the G-CDM was validated by assessing the differences in data characteristics between two different genomic databases through the proposed data-exploring tool GeneProfiler. The G-CDM may facilitate analyses of interoperating clinical and genomic datasets across multiple institutions, minimizing privacy issues and enabling researchers to better understand the characteristics of patients and promote personalized medicine in clinical practice. %M 30912749 %R 10.2196/13249 %U http://www.jmir.org/2019/3/e13249/ %U https://doi.org/10.2196/13249 %U http://www.ncbi.nlm.nih.gov/pubmed/30912749 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 2 %P e12902 %T Beyond One-Off Integrations: A Commercial, Substitutable, Reusable, Standards-Based, Electronic Health Record–Connected App %A Mandl,Kenneth D %A Gottlieb,Daniel %A Ellis,Alyssa %+ Computational Health Informatics Program, Boston Children's Hospital, 300 Longwood Avenue, Boston, MA, 02115, United States, 1 6173554145, kenneth_mandl@harvard.edu %K electronic medical records %K application programming interfaces %D 2019 %7 01.02.2019 %9 Viewpoint %J J Med Internet Res %G English %X The Substitutable Medical Apps and Reusable Technology (SMART) Health IT project launched in 2010 to facilitate the development of medical apps that are scalable and substitutable. SMART defines an open application programming interface (API) specification that enables apps to connect to electronic health record systems and data warehouses without custom integration efforts. The SMART-enabled version of the Meducation app, developed by Polyglot, has been implemented at scores of hospitals and clinics in the United States, nation-wide. After expanding their product’s reach by relying on a universal, open API for integrations, the team estimates that one project manager can handle up to 20 simultaneous implementations. The app is made available through the SMART App Gallery, an open app store that supports discovery of apps and, because the apps are substitutable, market competition. This case illustrates how a universal open API for patient and clinician-facing health IT systems supported and accelerated commercial success for a start-up company. Giving end users a wide and ever-growing choice of apps that leverage data generated by the health care system and patients at home through a universal, open API is a promising and generalizable approach for rapid diffusion of innovation across health systems. %M 30707097 %R 10.2196/12902 %U http://www.jmir.org/2019/2/e12902/ %U https://doi.org/10.2196/12902 %U http://www.ncbi.nlm.nih.gov/pubmed/30707097 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 1 %P e10929 %T Measuring the Impact of an Open Web-Based Prescribing Data Analysis Service on Clinical Practice: Cohort Study on NHS England Data %A Walker,Alex J %A Curtis,Helen J %A Croker,Richard %A Bacon,Seb %A Goldacre,Ben %+ Evidence Based Medicine DataLab, Nuffield Department of Primary Care Health Sciences, University of Oxford, Radcliffe Observatory Quarter, Woodstock Road, Oxford, OX2 6GG, United Kingdom, 44 1865289313, ben.goldacre@phc.ox.ac.uk %K drug prescribing %K cost control %K patient safety %K treatment efficacy %D 2019 %7 16.01.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: OpenPrescribing is a freely accessible service that enables any user to view and analyze the National Health Service (NHS) primary care prescribing data at the level of individual practices. This tool is intended to improve the quality, safety, and cost-effectiveness of prescribing. Objective: We aimed to measure the impact of OpenPrescribing being viewed on subsequent prescribing. Methods: Having preregistered our protocol and code, we measured three different metrics of prescribing quality (mean percentile across 34 existing OpenPrescribing quality measures, available “price-per-unit” savings, and total “low-priority prescribing” spend) to see whether they changed after the viewing of Clinical Commissioning Group (CCG) and practice pages. We also measured whether practices whose data were viewed on OpenPrescribing differed in prescribing, prior to viewing, compared with those who were not. We used fixed-effects and between-effects linear panel regression to isolate change over time and differences between practices, respectively. We adjusted for the month of prescribing in the fixed-effects model to remove underlying trends in outcome measures. Results: We found a reduction in available price-per-unit savings for both practices and CCGs after their pages were viewed. The saving was greater at practice level (−£40.42 per thousand patients per month; 95% CI −54.04 to −26.81) than at CCG level (−£14.70 per thousand patients per month; 95% CI −25.56 to −3.84). We estimate a total saving since launch of £243 thosand at practice level and £1.47 million at CCG level between the feature launch and end of follow-up (August to November 2017) among practices viewed. If the observed savings from practices viewed were extrapolated to all practices, this would generate £26.8 million in annual savings for the NHS, approximately 20% of the total possible savings from this method. The other two measures were not different after CCGs or practices were viewed. Practices that were viewed had worse prescribing quality scores overall prior to viewing. Conclusions: We found a positive impact from the use of OpenPrescribing, specifically for the class of savings opportunities that can only be identified by using this tool. Furthermore, we show that it is possible to conduct a robust analysis of the impact of such a Web-based service on clinical practice. %M 30664459 %R 10.2196/10929 %U http://www.jmir.org/2019/1/e10929/ %U https://doi.org/10.2196/10929 %U http://www.ncbi.nlm.nih.gov/pubmed/30664459 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 1 %P e10013 %T Application of Efficient Data Cleaning Using Text Clustering for Semistructured Medical Reports to Large-Scale Stool Examination Reports: Methodology Study %A Woo,Hyunki %A Kim,Kyunga %A Cha,KyeongMin %A Lee,Jin-Young %A Mun,Hansong %A Cho,Soo Jin %A Chung,Ji In %A Pyo,Jeung Hui %A Lee,Kun-Chul %A Kang,Mira %+ Center for Health Promotion, Samsung Medical Center, Sungkyunkwan University School of Medicine, 81 Irwon-ro, Gangnam-gu, Seoul, 06351, Republic of Korea, 82 2 3410 3882, mira90.kang@samsung.com %K data cleaning %K text clustering %K key collision %K nearest neighbor methods %K OpenRefine %D 2019 %7 08.01.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Since medical research based on big data has become more common, the community’s interest and effort to analyze a large amount of semistructured or unstructured text data, such as examination reports, have rapidly increased. However, these large-scale text data are often not readily applicable to analysis owing to typographical errors, inconsistencies, or data entry problems. Therefore, an efficient data cleaning process is required to ensure the veracity of such data. Objective: In this paper, we proposed an efficient data cleaning process for large-scale medical text data, which employs text clustering methods and value-converting technique, and evaluated its performance with medical examination text data. Methods: The proposed data cleaning process consists of text clustering and value-merging. In the text clustering step, we suggested the use of key collision and nearest neighbor methods in a complementary manner. Words (called values) in the same cluster would be expected as a correct value and its wrong representations. In the value-converting step, wrong values for each identified cluster would be converted into their correct value. We applied these data cleaning process to 574,266 stool examination reports produced for parasite analysis at Samsung Medical Center from 1995 to 2015. The performance of the proposed process was examined and compared with data cleaning processes based on a single clustering method. We used OpenRefine 2.7, an open source application that provides various text clustering methods and an efficient user interface for value-converting with common-value suggestion. Results: A total of 1,167,104 words in stool examination reports were surveyed. In the data cleaning process, we discovered 30 correct words and 45 patterns of typographical errors and duplicates. We observed high correction rates for words with typographical errors (98.61%) and typographical error patterns (97.78%). The resulting data accuracy was nearly 100% based on the number of total words. Conclusions: Our data cleaning process based on the combinatorial use of key collision and nearest neighbor methods provides an efficient cleaning of large-scale text data and hence improves data accuracy. %M 30622098 %R 10.2196/10013 %U https://www.jmir.org/2019/1/e10013/ %U https://doi.org/10.2196/10013 %U http://www.ncbi.nlm.nih.gov/pubmed/30622098 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 12 %P e11293 %T Primary Care Patient Records in the United Kingdom: Past, Present, and Future Research Priorities %A McMillan,Brian %A Eastham,Robert %A Brown,Benjamin %A Fitton,Richard %A Dickinson,David %+ Centre for Primary Care and Health Services Research, Division of Population Health, Health Services Research and Primary Care, School of Health Sciences, Faculty of Biology, Medicine and Health, University of Manchester, Williamson Building, Oxford Road, Manchester, M13 9PL, United Kingdom, 44 0161 2757662, brian.mcmillan@manchester.ac.uk %K primary care %K access to records %K medical records %K computerized records %D 2018 %7 19.12.2018 %9 Viewpoint %J J Med Internet Res %G English %X This paper briefly outlines the history of the medical record and the factors contributing to the adoption of computerized records in primary care in the United Kingdom. It discusses how both paper-based and electronic health records have traditionally been used in the past and goes on to examine how enabling patients to access their own primary care record online is changing the form and function of the patient record. In addition, it looks at the evidence for the benefits of Web-based access and discusses some of the challenges faced in this transition. Finally, some suggestions are made regarding the future of the patient record and research questions that need to be addressed to help deepen our understanding of how they can be used more beneficially by both patients and clinicians. %M 30567695 %R 10.2196/11293 %U http://www.jmir.org/2018/12/e11293/ %U https://doi.org/10.2196/11293 %U http://www.ncbi.nlm.nih.gov/pubmed/30567695 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 10 %P e279 %T Modeling and Predicting Outcomes of eHealth Usage by European Physicians: Multidimensional Approach from a Survey of 9196 General Practitioners %A Torrent-Sellens,Joan %A Díaz-Chao,Ángel %A Soler-Ramos,Ivan %A Saigí-Rubió,Francesc %+ Faculty of Health Sciences, Universitat Oberta de Catalunya, Avinguda del Tibidabo 39-43, Barcelona, 08035, Spain, 34 933263622, fsaigi@uoc.edu %K internet %K eHealth %K health care %K health drivers %K health barriers %K health attitude %K health information %K health empowerment %K information and communication technologies %K structural equation modeling %K Europe %D 2018 %7 22.10.2018 %9 Original Paper %J J Med Internet Res %G English %X Background: The literature has noted the need to use more advanced methods and models to evaluate physicians’ outcomes in the shared health care model that electronic health (eHealth) proposes. Objective: The goal of our study was to design and evaluate a predictive multidimensional model of the outcomes of eHealth usage by European physicians. Methods: We used 2012-2013 survey data from a sample of 9196 European physicians (general practitioners). We proposed and tested two composite indicators of eHealth usage outcomes (internal practices and practices with patients) through 2-stage structural equation modeling. Logistic regression (odds ratios, ORs) to model the predictors of eHealth usage outcomes indicators were also calculated. Results: European general practitioners who were female (internal practices OR 1.15, 95% CI 1.10-1.20; practices with patients OR 1.19, 95% CI 1.14-1.24) and younger—aged <35 years (internal practices OR 1.14, 95% CI 1.02-1.26; practices with patients OR 1.32, 95% CI 1.13-1.54) and aged 36-45 years (internal practices OR 1.16, 95% CI 1.06-1.28; practices with patients OR 1.21, 95% CI 1.10-1.33)—had a greater propensity toward favorable eHealth usage outcomes in internal practices and practices with patients. European general practitioners who positively valued information and communication technology (ICT) impact on their personal working processes (internal practices OR 5.30, 95% CI 4.73-5.93; practices with patients OR 4.83, 95% CI 4.32-5.40), teamwork processes (internal practices OR 4.19, 95% CI 3.78-4.65; practices with patients OR 3.38, 95% CI 3.05-3.74), and the doctor-patient relationship (internal practices OR 3.97, 95% CI 3.60-4.37; practices with patients OR 6.02, 95% CI 5.43-6.67) had a high propensity toward favorable effects of eHealth usage on internal practices and practices with patients. More favorable eHealth outcomes were also observed for self-employed European general practitioners (internal practices OR 1.33, 95% CI 1.22-1.45; practices with patients OR 1.10, 95% CI 1.03-1.28). Finally, general practitioners who reported that the number of patients treated in the last 2 years had remained constant (internal practices OR 1.08, 95% CI 1.01-1.17) or increased (practices with patients OR 1.12, 95% CI 1.03-1.22) had a higher propensity toward favorable eHealth usage outcomes. Conclusions: We provide new evidence of predictors (sociodemographic issues, attitudes toward ICT impacts, and working conditions) that explain favorable eHealth usage outcomes. The results highlight the need to develop more specific policies for eHealth usage to address different realities. %M 30348628 %R 10.2196/jmir.9253 %U http://www.jmir.org/2018/10/e279/ %U https://doi.org/10.2196/jmir.9253 %U http://www.ncbi.nlm.nih.gov/pubmed/30348628 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 10 %P e274 %T Web-Based Information Infrastructure Increases the Interrater Reliability of Medical Coders: Quasi-Experimental Study %A Varghese,Julian %A Sandmann,Sarah %A Dugas,Martin %+ Institute of Medical Informatics, University of Münster, Albert-Schweitzer-Campus 1, Gebäude A11, Münster, 48149, Germany, 49 2518354714, julian.varghese@ukmuenster.de %K clinical coding %K health information interoperability %K Unified Medical Language System %K eligibility criteria %D 2018 %7 15.10.2018 %9 Original Paper %J J Med Internet Res %G English %X Background: Medical coding is essential for standardized communication and integration of clinical data. The Unified Medical Language System by the National Library of Medicine is the largest clinical terminology system for medical coders and Natural Language Processing tools. However, the abundance of ambiguous codes leads to low rates of uniform coding among different coders. Objective: The objective of our study was to measure uniform coding among different medical experts in terms of interrater reliability and analyze the effect on interrater reliability using an expert- and Web-based code suggestion system. Methods: We conducted a quasi-experimental study in which 6 medical experts coded 602 medical items from structured quality assurance forms or free-text eligibility criteria of 20 different clinical trials. The medical item content was selected on the basis of mortality-leading diseases according to World Health Organization data. The intervention comprised using a semiautomatic code suggestion tool that is linked to a European information infrastructure providing a large medical text corpus of >300,000 medical form items with expert-assigned semantic codes. Krippendorff alpha (Kalpha) with bootstrap analysis was used for the interrater reliability analysis, and coding times were measured before and after the intervention. Results: The intervention improved interrater reliability in structured quality assurance form items (from Kalpha=0.50, 95% CI 0.43-0.57 to Kalpha=0.62 95% CI 0.55-0.69) and free-text eligibility criteria (from Kalpha=0.19, 95% CI 0.14-0.24 to Kalpha=0.43, 95% CI 0.37-0.50) while preserving or slightly reducing the mean coding time per item for all 6 coders. Regardless of the intervention, precoordination and structured items were associated with significantly high interrater reliability, but the proportion of items that were precoordinated significantly increased after intervention (eligibility criteria: OR 4.92, 95% CI 2.78-8.72; quality assurance: OR 1.96, 95% CI 1.19-3.25). Conclusions: The Web-based code suggestion mechanism improved interrater reliability toward moderate or even substantial intercoder agreement. Precoordination and the use of structured versus free-text data elements are key drivers of higher interrater reliability. %M 30322834 %R 10.2196/jmir.9644 %U http://www.jmir.org/2018/10/e274/ %U https://doi.org/10.2196/jmir.9644 %U http://www.ncbi.nlm.nih.gov/pubmed/30322834 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 10 %P e265 %T Investigating the Perceptions of Primary Care Dietitians on the Potential for Information Technology in the Workplace: Qualitative Study %A Jones,Aimee %A Mitchell,Lana J %A O'Connor,Rochelle %A Rollo,Megan E %A Slater,Katherine %A Williams,Lauren T %A Ball,Lauren %+ School of Allied Health Sciences, Griffith University, G01 2.05A, Gold Coast,, Australia, 61 0413031470, l.ball@griffith.edu.au %K dietetics %K information technology %K mobile phone %K primary health care %K private practice %D 2018 %7 15.10.2018 %9 Original Paper %J J Med Internet Res %G English %X Background: Chronic diseases are the leading cause of morbidity and mortality worldwide. The primary health care setting is an effective avenue for the management and prevention of chronic diseases. Dietitians working in this setting assist with the management of modifiable risk factors of chronic diseases. However, health care professionals report challenges in providing care in this setting because of time and financial constraints. Information technology offers the potential to improve health care quality, safety, efficiency, and cost-efficiency, but there exists limited understanding of dietitians’ application of technology in this setting. Objective: The objective of this study was to explore the perceptions of primary care dietitians about using information technology in their workplace. Methods: We recruited 20 Australian primary care dietitians using purposive and snowball sampling for semistructured telephonic interviews. Interview questions aimed to gain an understanding of dietitians’ perceptions about sharing patient outcomes through a national database and the benefits, disadvantages, feasibility, and barriers of using information technology. Interviews were audiorecorded, transcribed verbatim, and thematically analyzed for emerging themes and subthemes. Finally, the technologies used by participants were collated by name and researched for their key attributes. Results: The following 4 distinct themes emerged from the data: information technology improving the efficiency of practice tasks, experiencing barriers to using information technology in practice, information technology enhancing outcomes through education and monitoring, and information technology for sharing information with others. Participants identified several advantages and disadvantages of using technology and expressed willingness to share patient outcomes using a Web-based database. Conclusions: This study suggests that information technology is perceived to have benefits to dietitians and patients in primary health care. However, to achieve the optimal benefit, support is required to overcome barriers to integrate information technology into practice better. Further development of patient management systems and standardized Web-based data collection systems are needed to support better usage by dietitians. %M 30322837 %R 10.2196/jmir.9568 %U https://www.jmir.org/2018/10/e265/ %U https://doi.org/10.2196/jmir.9568 %U http://www.ncbi.nlm.nih.gov/pubmed/30322837 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 8 %P e251 %T Internet of Things Buttons for Real-Time Notifications in Hospital Operations: Proposal for Hospital Implementation %A Chai,Peter Ray %A Zhang,Haipeng %A Baugh,Christopher W %A Jambaulikar,Guruprasad D %A McCabe,Jonathan C %A Gorman,Janet M %A Boyer,Edward W %A Landman,Adam %+ Division of Medical Toxicology, Department of Emergency Medicine, Brigham and Women's Hospital, 75 Francis street, Boston, MA,, United States, 1 6177325640, pchai@bwh.harvard.edu %K Internet of Things %K operations %K hospital systems %K health care %D 2018 %7 10.08.2018 %9 Proposal %J J Med Internet Res %G English %X Background: Hospital staff frequently performs the same process hundreds to thousands of times a day. Customizable Internet of Things buttons are small, wirelessly-enabled devices that trigger specific actions with the press of an integrated button and have the potential to automate some of these repetitive tasks. In addition, IoT buttons generate logs of triggered events that can be used for future process improvements. Although Internet of Things buttons have seen some success as consumer products, little has been reported on their application in hospital systems. Objective: We discuss potential hospital applications categorized by the intended user group (patient or hospital staff). In addition, we examine key technological considerations, including network connectivity, security, and button management systems. Methods: In order to meaningfully deploy Internet of Things buttons in a hospital system, we propose an implementation framework grounded in the Plan-Do-Study-Act method. Results: We plan to deploy Internet of Things buttons within our hospital system to deliver real-time notifications in public-facing tasks such as restroom cleanliness and critical supply restocking. We expect results from this pilot in the next year. Conclusions: Overall, Internet of Things buttons have significant promise; future rigorous evaluations are needed to determine the impact of Internet of Things buttons in real-world health care settings. %M 30097420 %R 10.2196/jmir.9454 %U http://www.jmir.org/2018/8/e251/ %U https://doi.org/10.2196/jmir.9454 %U http://www.ncbi.nlm.nih.gov/pubmed/30097420 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 8 %P e10458 %T Forecasting the Maturation of Electronic Health Record Functions Among US Hospitals: Retrospective Analysis and Predictive Model %A Kharrazi,Hadi %A Gonzalez,Claudia P %A Lowe,Kevin B %A Huerta,Timothy R %A Ford,Eric W %+ Center for Population Health IT, Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, 624 N Broadway, Office 606, Baltimore, MD, 21205-2103, United States, 1 443 287 8264, kharrazi@jhu.edu %K electronic health records %K United States %K hospitals %K HIMSS EMRAM %K Bass diffusion model %D 2018 %7 07.08.2018 %9 Original Paper %J J Med Internet Res %G English %X Background: The Meaningful Use (MU) program has promoted electronic health record adoption among US hospitals. Studies have shown that electronic health record adoption has been slower than desired in certain types of hospitals; but generally, the overall adoption rate has increased among hospitals. However, these studies have neither evaluated the adoption of advanced functionalities of electronic health records (beyond MU) nor forecasted electronic health record maturation over an extended period in a holistic fashion. Additional research is needed to prospectively assess US hospitals’ electronic health record technology adoption and advancement patterns. Objective: This study forecasts the maturation of electronic health record functionality adoption among US hospitals through 2035. Methods: The Healthcare Information and Management Systems Society (HIMSS) Analytics’ Electronic Medical Record Adoption Model (EMRAM) dataset was used to track historic uptakes of various electronic health record functionalities considered critical to improving health care quality and efficiency in hospitals. The Bass model was used to predict the technological diffusion rates for repeated electronic health record adoptions where upgrades undergo rapid technological improvements. The forecast used EMRAM data from 2006 to 2014 to estimate adoption levels to the year 2035. Results: In 2014, over 5400 hospitals completed HIMSS’ annual EMRAM survey (86%+ of total US hospitals). In 2006, the majority of the US hospitals were in EMRAM Stages 0, 1, and 2. By 2014, most hospitals had achieved Stages 3, 4, and 5. The overall technology diffusion model (ie, the Bass model) reached an adjusted R-squared of .91. The final forecast depicted differing trends for each of the EMRAM stages. In 2006, the first year of observation, peaks of Stages 0 and 1 were shown as electronic health record adoption predates HIMSS’ EMRAM. By 2007, Stage 2 reached its peak. Stage 3 reached its full height by 2011, while Stage 4 peaked by 2014. The first three stages created a graph that exhibits the expected “S-curve” for technology diffusion, with inflection point being the peak diffusion rate. This forecast indicates that Stage 5 should peak by 2019 and Stage 6 by 2026. Although this forecast extends to the year 2035, no peak was readily observed for Stage 7. Overall, most hospitals will achieve Stages 5, 6, or 7 of EMRAM by 2020; however, a considerable number of hospitals will not achieve Stage 7 by 2035. Conclusions: We forecasted the adoption of electronic health record capabilities from a paper-based environment (Stage 0) to an environment where only electronic information is used to document and direct care delivery (Stage 7). According to our forecasts, the majority of hospitals will not reach Stage 7 until 2035, absent major policy changes or leaps in technological capabilities. These results indicate that US hospitals are decades away from fully implementing sophisticated decision support applications and interoperability functionalities in electronic health records as defined by EMRAM’s Stage 7. %M 30087090 %R 10.2196/10458 %U http://www.jmir.org/2018/8/e10458/ %U https://doi.org/10.2196/10458 %U http://www.ncbi.nlm.nih.gov/pubmed/30087090 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 7 %P e10725 %T Reimagining Health Data Exchange: An Application Programming Interface–Enabled Roadmap for India %A Balsari,Satchit %A Fortenko,Alexander %A Blaya,Joaquín A %A Gropper,Adrian %A Jayaram,Malavika %A Matthan,Rahul %A Sahasranam,Ram %A Shankar,Mark %A Sarbadhikari,Suptendra N %A Bierer,Barbara E %A Mandl,Kenneth D %A Mehendale,Sanjay %A Khanna,Tarun %+ Harvard FXB Center for Health and Human Rights, 651 Huntington Avenue, 703C, Boston, MA,, United States, 1 6174320011, sbalsari@bidmc.harvard.edu %K health information exchange %K India %K health APIs %D 2018 %7 13.07.2018 %9 Policy Proposal %J J Med Internet Res %G English %X In February 2018, the Government of India announced a massive public health insurance scheme extending coverage to 500 million citizens, in effect making it the world’s largest insurance program. To meet this target, the government will rely on technology to effectively scale services, monitor quality, and ensure accountability. While India has seen great strides in informational technology development and outsourcing, cellular phone penetration, cloud computing, and financial technology, the digital health ecosystem is in its nascent stages and has been waiting for a catalyst to seed the system. This National Health Protection Scheme is expected to provide just this impetus for widespread adoption. However, health data in India are mostly not digitized. In the few instances that they are, the data are not standardized, not interoperable, and not readily accessible to clinicians, researchers, or policymakers. While such barriers to easy health information exchange are hardly unique to India, the greenfield nature of India’s digital health infrastructure presents an excellent opportunity to avoid the pitfalls of complex, restrictive, digital health systems that have evolved elsewhere. We propose here a federated, patient-centric, application programming interface (API)–enabled health information ecosystem that leverages India’s near-universal mobile phone penetration, universal availability of unique ID systems, and evolving privacy and data protection laws. It builds on global best practices and promotes the adoption of human-centered design principles, data minimization, and open standard APIs. The recommendations are the result of 18 months of deliberations with multiple stakeholders in India and the United States, including from academia, industry, and government. %M 30006325 %R 10.2196/10725 %U http://www.jmir.org/2018/7/e10725/ %U https://doi.org/10.2196/10725 %U http://www.ncbi.nlm.nih.gov/pubmed/30006325 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 7 %P e10041 %T Rethinking the Meaning of Cloud Computing for Health Care: A Taxonomic Perspective and Future Research Directions %A Gao,Fangjian %A Thiebes,Scott %A Sunyaev,Ali %+ Department of Economics and Management, Karlsruhe Institute of Technology, Kaiserstraße 89, Karlsruhe,, Germany, 49 721 608 46037, sunyaev@kit.edu %K cloud computing %K taxonomy %K health IT innovation %D 2018 %7 11.07.2018 %9 Original Paper %J J Med Internet Res %G English %X Background: Cloud computing is an innovative paradigm that provides users with on-demand access to a shared pool of configurable computing resources such as servers, storage, and applications. Researchers claim that information technology (IT) services delivered via the cloud computing paradigm (ie, cloud computing services) provide major benefits for health care. However, due to a mismatch between our conceptual understanding of cloud computing for health care and the actual phenomenon in practice, the meaningful use of it for the health care industry cannot always be ensured. Although some studies have tried to conceptualize cloud computing or interpret this phenomenon for health care settings, they have mainly relied on its interpretation in a common context or have been heavily based on a general understanding of traditional health IT artifacts, leading to an insufficient or unspecific conceptual understanding of cloud computing for health care. Objective: We aim to generate insights into the concept of cloud computing for health IT research. We propose a taxonomy that can serve as a fundamental mechanism for organizing knowledge about cloud computing services in health care organizations to gain a deepened, specific understanding of cloud computing in health care. With the taxonomy, we focus on conceptualizing the relevant properties of cloud computing for service delivery to health care organizations and highlighting their specific meanings for health care. Methods: We employed a 2-stage approach in developing a taxonomy of cloud computing services for health care organizations. We conducted a structured literature review and 24 semistructured expert interviews in stage 1, drawing on data from theory and practice. In stage 2, we applied a systematic approach and relied on data from stage 1 to develop and evaluate the taxonomy using 14 iterations. Results: Our taxonomy is composed of 8 dimensions and 28 characteristics that are relevant for cloud computing services in health care organizations. By applying the taxonomy to classify existing cloud computing services identified from the literature and expert interviews, which also serves as a part of the taxonomy, we identified 7 specificities of cloud computing in health care. These specificities challenge what we have learned about cloud computing in general contexts or in traditional health IT from the previous literature. The summarized specificities suggest research opportunities and exemplary research questions for future health IT research on cloud computing. Conclusions: By relying on perspectives from a taxonomy for cloud computing services for health care organizations, this study provides a solid conceptual cornerstone for cloud computing in health care. Moreover, the identified specificities of cloud computing and the related future research opportunities will serve as a valuable roadmap to facilitate more research into cloud computing in health care. %M 29997108 %R 10.2196/10041 %U http://www.jmir.org/2018/7/e10041/ %U https://doi.org/10.2196/10041 %U http://www.ncbi.nlm.nih.gov/pubmed/29997108 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 5 %P e185 %T Possible Sources of Bias in Primary Care Electronic Health Record Data Use and Reuse %A Verheij,Robert A %A Curcin,Vasa %A Delaney,Brendan C %A McGilchrist,Mark M %+ Netherlands Institute for Health Services Research, PO Box 1568, Utrecht, 3500BN, Netherlands, 31 641242229, r.verheij@nivel.nl %K electronic health record %K data accuracy %K data sharing %K health information interoperability %K health care systems %K health information systems %K medical informatics %D 2018 %7 29.05.2018 %9 Original Paper %J J Med Internet Res %G English %X Background: Enormous amounts of data are recorded routinely in health care as part of the care process, primarily for managing individual patient care. There are significant opportunities to use these data for other purposes, many of which would contribute to establishing a learning health system. This is particularly true for data recorded in primary care settings, as in many countries, these are the first place patients turn to for most health problems. Objective: In this paper, we discuss whether data that are recorded routinely as part of the health care process in primary care are actually fit to use for other purposes such as research and quality of health care indicators, how the original purpose may affect the extent to which the data are fit for another purpose, and the mechanisms behind these effects. In doing so, we want to identify possible sources of bias that are relevant for the use and reuse of these type of data. Methods: This paper is based on the authors’ experience as users of electronic health records data, as general practitioners, health informatics experts, and health services researchers. It is a product of the discussions they had during the Translational Research and Patient Safety in Europe (TRANSFoRm) project, which was funded by the European Commission and sought to develop, pilot, and evaluate a core information architecture for the learning health system in Europe, based on primary care electronic health records. Results: We first describe the different stages in the processing of electronic health record data, as well as the different purposes for which these data are used. Given the different data processing steps and purposes, we then discuss the possible mechanisms for each individual data processing step that can generate biased outcomes. We identified 13 possible sources of bias. Four of them are related to the organization of a health care system, whereas some are of a more technical nature. Conclusions: There are a substantial number of possible sources of bias; very little is known about the size and direction of their impact. However, anyone that uses or reuses data that were recorded as part of the health care process (such as researchers and clinicians) should be aware of the associated data collection process and environmental influences that can affect the quality of the data. Our stepwise, actor- and purpose-oriented approach may help to identify these possible sources of bias. Unless data quality issues are better understood and unless adequate controls are embedded throughout the data lifecycle, data-driven health care will not live up to its expectations. We need a data quality research agenda to devise the appropriate instruments needed to assess the magnitude of each of the possible sources of bias, and then start measuring their impact. The possible sources of bias described in this paper serve as a starting point for this research agenda. %R 10.2196/jmir.9134 %U http://www.jmir.org/2018/5/e185/ %U https://doi.org/10.2196/jmir.9134 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 6 %N 2 %P e26 %T A Neuroimaging Web Services Interface as a Cyber Physical System for Medical Imaging and Data Management in Brain Research: Design Study %A Lizarraga,Gabriel %A Li,Chunfei %A Cabrerizo,Mercedes %A Barker,Warren %A Loewenstein,David A %A Duara,Ranjan %A Adjouadi,Malek %+ Center for Advanced Technology and Education, Computer Science, Florida International University, 10555 West Flagler Street, EC 2220, Miami, FL, 33174, United States, 1 3053484106, gliza002@fiu.edu %K neuroimaging %K multimodal imaging %K magnetic resonance imaging %K image processing %K positron-emission tomography %K diffusion tensor imaging %K information storage and retrieval %K diagnostic imaging %D 2018 %7 26.04.2018 %9 Original Paper %J JMIR Med Inform %G English %X Background: Structural and functional brain images are essential imaging modalities for medical experts to study brain anatomy. These images are typically visually inspected by experts. To analyze images without any bias, they must be first converted to numeric values. Many software packages are available to process the images, but they are complex and difficult to use. The software packages are also hardware intensive. The results obtained after processing vary depending on the native operating system used and its associated software libraries; data processed in one system cannot typically be combined with data on another system. Objective: The aim of this study was to fulfill the neuroimaging community’s need for a common platform to store, process, explore, and visualize their neuroimaging data and results using Neuroimaging Web Services Interface: a series of processing pipelines designed as a cyber physical system for neuroimaging and clinical data in brain research. Methods: Neuroimaging Web Services Interface accepts magnetic resonance imaging, positron emission tomography, diffusion tensor imaging, and functional magnetic resonance imaging. These images are processed using existing and custom software packages. The output is then stored as image files, tabulated files, and MySQL tables. The system, made up of a series of interconnected servers, is password-protected and is securely accessible through a Web interface and allows (1) visualization of results and (2) downloading of tabulated data. Results: All results were obtained using our processing servers in order to maintain data validity and consistency. The design is responsive and scalable. The processing pipeline started from a FreeSurfer reconstruction of Structural magnetic resonance imaging images. The FreeSurfer and regional standardized uptake value ratio calculations were validated using Alzheimer’s Disease Neuroimaging Initiative input images, and the results were posted at the Laboratory of Neuro Imaging data archive. Notable leading researchers in the field of Alzheimer’s Disease and epilepsy have used the interface to access and process the data and visualize the results. Tabulated results with unique visualization mechanisms help guide more informed diagnosis and expert rating, providing a truly unique multimodal imaging platform that combines magnetic resonance imaging, positron emission tomography, diffusion tensor imaging, and resting state functional magnetic resonance imaging. A quality control component was reinforced through expert visual rating involving at least 2 experts. Conclusions: To our knowledge, there is no validated Web-based system offering all the services that Neuroimaging Web Services Interface offers. The intent of Neuroimaging Web Services Interface is to create a tool for clinicians and researchers with keen interest on multimodal neuroimaging. More importantly, Neuroimaging Web Services Interface significantly augments the Alzheimer’s Disease Neuroimaging Initiative data, especially since our data contain a large cohort of Hispanic normal controls and Alzheimer’s Disease patients. The obtained results could be scrutinized visually or through the tabulated forms, informing researchers on subtle changes that characterize the different stages of the disease. %M 29699962 %R 10.2196/medinform.9063 %U http://medinform.jmir.org/2018/2/e26/ %U https://doi.org/10.2196/medinform.9063 %U http://www.ncbi.nlm.nih.gov/pubmed/29699962 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 4 %P e124 %T Optimizing Electronic Consultation Between Primary Care Providers and Psychiatrists: Mixed-Methods Study %A Hensel,Jennifer M %A Yang,Rebecca %A Rai,Minnie %A Taylor,Valerie H %+ Women's College Hospital Institute for Healthcare Solutions and Virtual Care, Women's College Hospital, 76 Grenville St, Toronto, ON, M5S 1B2, Canada, 1 416 323 6230, jennifer.hensel@wchospital.ca %K eHealth %K psychiatry %K primary care %K consultation %K health services %D 2018 %7 06.04.2018 %9 Original Paper %J J Med Internet Res %G English %X Background: The use of electronic consultation (e-consult) between primary care providers (PCPs) and psychiatrists has potential, given the high prevalence of mental health issues in primary care and problematic access to specialist care. Utilization and uptake, however, appears to be lower than would be expected. Objective: This study aimed to examine actual utilization of e-consult between PCPs and psychiatrists and investigate the perceptions of PCPs about this form of psychiatric advice to inform how to optimize the utility and thereby the uptake of this service. Methods: In this mixed-methods study, we conducted a chart review of psychiatry e-consults (N=37) over 2 platforms during early implementation in Ontario, Canada, as well as 3 group interviews and 1 individual interview with PCPs (N=10) with variable experience levels and from a range of practice settings. The chart review assessed response times and referral content including the type of request, referral attachments, and consultant responses. Interviews explored the perceptions of the PCPs about the uses and barriers of psychiatry e-consult. Thematic content analysis of interview data identified common themes as well as themes unique to different provider profiles (eg, experienced PCPs vs new PCPs and rural vs urban practice). On the basis of interpretation of the quantitative and qualitative findings, we developed recommendations for the optimization of psychiatry e-consultation services. Results: During the study period, psychiatry e-consults comprised 3.66% (49/1339) of all e-consults submitted on the studied platforms. Among the e-consults reviewed, different psychiatric diagnoses were represented: 70% of requests (26/37) queried about medication safety or side effects, whereas 59% (22/37) asked about psychiatric symptom management. Moreover, 81% (30/37) of e-consults were answered within 24 hours, and 65% (24/37) were addressed in a single exchange. Themes from the interview data included psychiatry having a complexity that differentiates it from other specialties and may limit the utility of e-consult, other than for psychopharmacology advice. Variability in awareness exists in the way e-consultation could be used in psychiatry, with new PCPs feeling unsure about the appropriateness of a question. In general, new PCPs and PCPs practicing in rural areas were more receptive to psychiatry e-consult. PCPs viewed e-consult as an opportunity to collaborate and desired that it be integrated with other available services. Recommendations include the need for appropriate specialist staffing to address a wide range of requests, adequate education to referrers regarding the use of psychiatry e-consult, and the need to integrate psychiatry e-consult with other geographically relevant services, given the complexity of psychiatric issues. Conclusions: E-consult is a viable and timely way for PCPs to get much-needed psychiatric advice. For optimizing its utility and uptake, e-consult needs to be integrated into reliable care pathways with adequate referrer and consultant preparation. %M 29625949 %R 10.2196/jmir.8943 %U http://www.jmir.org/2018/4/e124/ %U https://doi.org/10.2196/jmir.8943 %U http://www.ncbi.nlm.nih.gov/pubmed/29625949 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 2 %P e41 %T Health Information Technology Continues to Show Positive Effect on Medical Outcomes: Systematic Review %A Kruse,Clemens Scott %A Beane,Amanda %+ School of Health Administration, Texas State University, 601 University Drive, Encino Hall, Rm 250, San Marcos, TX, 78666, United States, 1 512 245 4462, scottkruse@txstate.edu %K health information technology %K treatment outcome %K electronic health record %D 2018 %7 05.02.2018 %9 Review %J J Med Internet Res %G English %X Background: Health information technology (HIT) has been introduced into the health care industry since the 1960s when mainframes assisted with financial transactions, but questions remained about HIT’s contribution to medical outcomes. Several systematic reviews since the 1990s have focused on this relationship. This review updates the literature. Objective: The purpose of this review was to analyze the current literature for the impact of HIT on medical outcomes. We hypothesized that there is a positive association between the adoption of HIT and medical outcomes. Methods: We queried the Cumulative Index of Nursing and Allied Health Literature (CINAHL) and Medical Literature Analysis and Retrieval System Online (MEDLINE) by PubMed databases for peer-reviewed publications in the last 5 years that defined an HIT intervention and an effect on medical outcomes in terms of efficiency or effectiveness. We structured the review from the Primary Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA), and we conducted the review in accordance with the Assessment for Multiple Systematic Reviews (AMSTAR). Results: We narrowed our search from 3636 papers to 37 for final analysis. At least one improved medical outcome as a result of HIT adoption was identified in 81% (25/37) of research studies that met inclusion criteria, thus strongly supporting our hypothesis. No statistical difference in outcomes was identified as a result of HIT in 19% of included studies. Twelve categories of HIT and three categories of outcomes occurred 38 and 65 times, respectively. Conclusions: A strong majority of the literature shows positive effects of HIT on the effectiveness of medical outcomes, which positively supports efforts that prepare for stage 3 of meaningful use. This aligns with previous reviews in other time frames. %M 29402759 %R 10.2196/jmir.8793 %U http://www.jmir.org/2018/2/e41/ %U https://doi.org/10.2196/jmir.8793 %U http://www.ncbi.nlm.nih.gov/pubmed/29402759 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 19 %N 11 %P e380 %T Artificial Intelligence Learning Semantics via External Resources for Classifying Diagnosis Codes in Discharge Notes %A Lin,Chin %A Hsu,Chia-Jung %A Lou,Yu-Sheng %A Yeh,Shih-Jen %A Lee,Chia-Cheng %A Su,Sui-Lung %A Chen,Hsiang-Cheng %+ Division of Rheumatology/Immunology/Allergy, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, No.161, Min-Chun E. Rd., Sec. 6, Neihu, Taipei, 114, Taiwan, 886 2 7927135, hccheng@ndmctsgh.edu.tw %K word embedding %K convolutional neural network %K neural networks (computer) %K natural language processing %K text mining %K data mining %K machine learning %K electronic medical records %K electronic health records %D 2017 %7 06.11.2017 %9 Original Paper %J J Med Internet Res %G English %X Background: Automated disease code classification using free-text medical information is important for public health surveillance. However, traditional natural language processing (NLP) pipelines are limited, so we propose a method combining word embedding with a convolutional neural network (CNN). Objective: Our objective was to compare the performance of traditional pipelines (NLP plus supervised machine learning models) with that of word embedding combined with a CNN in conducting a classification task identifying International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) diagnosis codes in discharge notes. Methods: We used 2 classification methods: (1) extracting from discharge notes some features (terms, n-gram phrases, and SNOMED CT categories) that we used to train a set of supervised machine learning models (support vector machine, random forests, and gradient boosting machine), and (2) building a feature matrix, by a pretrained word embedding model, that we used to train a CNN. We used these methods to identify the chapter-level ICD-10-CM diagnosis codes in a set of discharge notes. We conducted the evaluation using 103,390 discharge notes covering patients hospitalized from June 1, 2015 to January 31, 2017 in the Tri-Service General Hospital in Taipei, Taiwan. We used the receiver operating characteristic curve as an evaluation measure, and calculated the area under the curve (AUC) and F-measure as the global measure of effectiveness. Results: In 5-fold cross-validation tests, our method had a higher testing accuracy (mean AUC 0.9696; mean F-measure 0.9086) than traditional NLP-based approaches (mean AUC range 0.8183-0.9571; mean F-measure range 0.5050-0.8739). A real-world simulation that split the training sample and the testing sample by date verified this result (mean AUC 0.9645; mean F-measure 0.9003 using the proposed method). Further analysis showed that the convolutional layers of the CNN effectively identified a large number of keywords and automatically extracted enough concepts to predict the diagnosis codes. Conclusions: Word embedding combined with a CNN showed outstanding performance compared with traditional methods, needing very little data preprocessing. This shows that future studies will not be limited by incomplete dictionaries. A large amount of unstructured information from free-text medical writing will be extracted by automated approaches in the future, and we believe that the health care field is about to enter the age of big data. %M 29109070 %R 10.2196/jmir.8344 %U http://www.jmir.org/2017/11/e380/ %U https://doi.org/10.2196/jmir.8344 %U http://www.ncbi.nlm.nih.gov/pubmed/29109070 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 19 %N 10 %P e340 %T What Clinical Information Is Valuable to Doctors Using Mobile Electronic Medical Records and When? %A Kim,Junetae %A Lee,Yura %A Lim,Sanghee %A Kim,Jeong Hoon %A Lee,Byungtae %A Lee,Jae-Ho %+ Department of Biomedical Informatics, Asan Medical Center, University of Ulsan College of Medicine, Asan Medical Center, 88 Olympic-ro 43-gil, Songpa-gu, Seoul, 05505, Republic Of Korea, 82 23010 3350, rufiji@gmail.com %K mobile health %K electronic medical records %K clinical information %K rounding %K timeliness %K accessibility %K smartphone %D 2017 %7 18.10.2017 %9 Original Paper %J J Med Internet Res %G English %X Background: There has been a lack of understanding on what types of specific clinical information are most valuable for doctors to access through mobile-based electronic medical records (m-EMRs) and when they access such information. Furthermore, it has not been clearly discussed why the value of such information is high. Objective: The goal of this study was to investigate the types of clinical information that are most valuable to doctors to access through an m-EMR and when such information is accessed. Methods: Since 2010, an m-EMR has been used in a tertiary hospital in Seoul, South Korea. The usage logs of the m-EMR by doctors were gathered from March to December 2015. Descriptive analyses were conducted to explore the overall usage patterns of the m-EMR. To assess the value of the clinical information provided, the usage patterns of both the m-EMR and a hospital information system (HIS) were compared on an hourly basis. The peak usage times of the m-EMR were defined as continuous intervals having normalized usage values that are greater than 0.5. The usage logs were processed as an indicator representing specific clinical information using factor analysis. Random intercept logistic regression was used to explore the type of clinical information that is frequently accessed during the peak usage times. Results: A total of 524,929 usage logs from 653 doctors (229 professors, 161 fellows, and 263 residents; mean age: 37.55 years; males: 415 [63.6%]) were analyzed. The highest average number of m-EMR usage logs (897) was by medical residents, whereas the lowest (292) was by surgical residents. The usage amount for three menus, namely inpatient list (47,096), lab results (38,508), and investigation list (25,336), accounted for 60.1% of the peak time usage. The HIS was used most frequently during regular hours (9:00 AM to 5:00 PM). The peak usage time of the m-EMR was early in the morning (6:00 AM to 10:00 AM), and the use of the m-EMR from early evening (5:00 PM) to midnight was higher than during regular business hours. Four factors representing the types of clinical information were extracted through factor analysis. Factors related to patient investigation status and patient conditions were associated with the peak usage times of the m-EMR (P<.01). Conclusions: Access to information regarding patient investigation status and patient conditions is crucial for decision making during morning activities, including ward rounds. The m-EMRs allow doctors to maintain the continuity of their clinical information regardless of the time and location constraints. Thus, m-EMRs will best evolve in a manner that enhances the accessibility of clinical information helpful to the decision-making process under such constraints. %M 29046269 %R 10.2196/jmir.8128 %U http://www.jmir.org/2017/10/e340/ %U https://doi.org/10.2196/jmir.8128 %U http://www.ncbi.nlm.nih.gov/pubmed/29046269 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 5 %N 3 %P e26 %T Is There Evidence of Cost Benefits of Electronic Medical Records, Standards, or Interoperability in Hospital Information Systems? Overview of Systematic Reviews %A Reis,Zilma Silveira Nogueira %A Maia,Thais Abreu %A Marcolino,Milena Soriano %A Becerra-Posada,Francisco %A Novillo-Ortiz,David %A Ribeiro,Antonio Luiz Pinho %+ Informatics Center in Health, Obstetrics and Gynecology Department, Universidade Federal de Minas Gerais, Av. Prof. Alfredo Balena, 190, Belo Horizonte, Minas Gerais, 30130100, Brazil, 55 3134099648, zilma.medicina@gmail.com %K electronic medical records %K standards %K medical information exchange %K health information exchange %K cost %K benefits and costs %D 2017 %7 29.08.2017 %9 Original Paper %J JMIR Med Inform %G English %X Background: Electronic health (eHealth) interventions may improve the quality of care by providing timely, accessible information about one patient or an entire population. Electronic patient care information forms the nucleus of computerized health information systems. However, interoperability among systems depends on the adoption of information standards. Additionally, investing in technology systems requires cost-effectiveness studies to ensure the sustainability of processes for stakeholders. Objective: The objective of this study was to assess cost-effectiveness of the use of electronically available inpatient data systems, health information exchange, or standards to support interoperability among systems. Methods: An overview of systematic reviews was conducted, assessing the MEDLINE, Cochrane Library, LILACS, and IEEE Library databases to identify relevant studies published through February 2016. The search was supplemented by citations from the selected papers. The primary outcome sought the cost-effectiveness, and the secondary outcome was the impact on quality of care. Independent reviewers selected studies, and disagreement was resolved by consensus. The quality of the included studies was evaluated using a measurement tool to assess systematic reviews (AMSTAR). Results: The primary search identified 286 papers, and two papers were manually included. A total of 211 were systematic reviews. From the 20 studies that were selected after screening the title and abstract, 14 were deemed ineligible, and six met the inclusion criteria. The interventions did not show a measurable effect on cost-effectiveness. Despite the limited number of studies, the heterogeneity of electronic systems reported, and the types of intervention in hospital routines, it was possible to identify some preliminary benefits in quality of care. Hospital information systems, along with information sharing, had the potential to improve clinical practice by reducing staff errors or incidents, improving automated harm detection, monitoring infections more effectively, and enhancing the continuity of care during physician handoffs. Conclusions: This review identified some benefits in the quality of care but did not provide evidence that the implementation of eHealth interventions had a measurable impact on cost-effectiveness in hospital settings. However, further evidence is needed to infer the impact of standards adoption or interoperability in cost benefits of health care; this in turn requires further research. %M 28851681 %R 10.2196/medinform.7400 %U http://medinform.jmir.org/2017/3/e26/ %U https://doi.org/10.2196/medinform.7400 %U http://www.ncbi.nlm.nih.gov/pubmed/28851681 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 19 %N 8 %P e294 %T Development and Deployment of the OpenMRS-Ebola Electronic Health Record System for an Ebola Treatment Center in Sierra Leone %A Oza,Shefali %A Jazayeri,Darius %A Teich,Jonathan M %A Ball,Ellen %A Nankubuge,Patricia Alexandra %A Rwebembera,Job %A Wing,Kevin %A Sesay,Alieu Amara %A Kanter,Andrew S %A Ramos,Glauber D %A Walton,David %A Cummings,Rachael %A Checchi,Francesco %A Fraser,Hamish S %+ Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, Keppel Street, London, WC1E 7HT, United Kingdom, 44 020 7636 863, shefali@alum.mit.edu %K Ebola virus disease %K electronic health records %K eHealth %K health information systems %K disease outbreaks %K disasters %K West Africa %K Sierra Leone %D 2017 %7 21.08.2017 %9 Original Paper %J J Med Internet Res %G English %X Background: Stringent infection control requirements at Ebola treatment centers (ETCs), which are specialized facilities for isolating and treating Ebola patients, create substantial challenges for recording and reviewing patient information. During the 2014-2016 West African Ebola epidemic, paper-based data collection systems at ETCs compromised the quality, quantity, and confidentiality of patient data. Electronic health record (EHR) systems have the potential to address such problems, with benefits for patient care, surveillance, and research. However, no suitable software was available for deployment when large-scale ETCs opened as the epidemic escalated in 2014. Objective: We present our work on rapidly developing and deploying OpenMRS-Ebola, an EHR system for the Kerry Town ETC in Sierra Leone. We describe our experience, lessons learned, and recommendations for future health emergencies. Methods: We used the OpenMRS platform and Agile software development approaches to build OpenMRS-Ebola. Key features of our work included daily communications between the development team and ground-based operations team, iterative processes, and phased development and implementation. We made design decisions based on the restrictions of the ETC environment and regular user feedback. To evaluate the system, we conducted predeployment user questionnaires and compared the EHR records with duplicate paper records. Results: We successfully built OpenMRS-Ebola, a modular stand-alone EHR system with a tablet-based application for infectious patient wards and a desktop-based application for noninfectious areas. OpenMRS-Ebola supports patient tracking (registration, bed allocation, and discharge); recording of vital signs and symptoms; medication and intravenous fluid ordering and monitoring; laboratory results; clinician notes; and data export. It displays relevant patient information to clinicians in infectious and noninfectious zones. We implemented phase 1 (patient tracking; drug ordering and monitoring) after 2.5 months of full-time development. OpenMRS-Ebola was used for 112 patient registrations, 569 prescription orders, and 971 medication administration recordings. We were unable to fully implement phases 2 and 3 as the ETC closed because of a decrease in new Ebola cases. The phase 1 evaluation suggested that OpenMRS-Ebola worked well in the context of the rollout, and the user feedback was positive. Conclusions: To our knowledge, OpenMRS-Ebola is the most comprehensive adaptable clinical EHR built for a low-resource setting health emergency. It is designed to address the main challenges of data collection in highly infectious environments that require robust infection prevention and control measures and it is interoperable with other electronic health systems. Although we built and deployed OpenMRS-Ebola more rapidly than typical software, our work highlights the challenges of having to develop an appropriate system during an emergency rather than being able to rapidly adapt an existing one. Lessons learned from this and previous emergencies should be used to ensure that a set of well-designed, easy-to-use, pretested health software is ready for quick deployment in future. %M 28827211 %R 10.2196/jmir.7881 %U http://www.jmir.org/2017/8/e294/ %U https://doi.org/10.2196/jmir.7881 %U http://www.ncbi.nlm.nih.gov/pubmed/28827211 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 19 %N 8 %P e283 %T An Internet-Based Method for Extracting Nursing Home Resident Sedative Medication Data From Pharmacy Packing Systems: Descriptive Evaluation %A Ling,Tristan %A Gee,Peter %A Westbury,Juanita %A Bindoff,Ivan %A Peterson,Gregory %+ Unit for Medication Outcomes Research and Education, Division of Pharmacy, School of Medicine, University of Tasmania, Private Bag 26, University of Tasmania, Hobart, 7001, Australia, 61 362267396, Tristan.Ling@utas.edu.au %K electronic health records %K information storage and retrieval %K inappropriate prescribing %K antipsychotic agents %K benzodiazepines %K nursing homes %K systematized nomenclature of medicine %K health information systems %D 2017 %7 03.08.2017 %9 Original Paper %J J Med Internet Res %G English %X Background: Inappropriate use of sedating medication has been reported in nursing homes for several decades. The Reducing Use of Sedatives (RedUSe) project was designed to address this issue through a combination of audit, feedback, staff education, and medication review. The project significantly reduced sedative use in a controlled trial of 25 Tasmanian nursing homes. To expand the project to 150 nursing homes across Australia, an improved and scalable method of data collection was required. This paper describes and evaluates a method for remotely extracting, transforming, and validating electronic resident and medication data from community pharmacies supplying medications to nursing homes. Objective: The aim of this study was to develop and evaluate an electronic method for extracting and enriching data on psychotropic medication use in nursing homes, on a national scale. Methods: An application uploaded resident details and medication data from computerized medication packing systems in the pharmacies supplying participating nursing homes. The server converted medication codes used by the packing systems to Australian Medicines Terminology coding and subsequently to Anatomical Therapeutic Chemical (ATC) codes for grouping. Medications of interest, in this case antipsychotics and benzodiazepines, were automatically identified and quantified during the upload. This data was then validated on the Web by project staff and a “champion nurse” at the participating home. Results: Of participating nursing homes, 94.6% (142/150) had resident and medication records uploaded. Facilitating an upload for one pharmacy took an average of 15 min. A total of 17,722 resident profiles were extracted, representing 95.6% (17,722/18,537) of the homes’ residents. For these, 546,535 medication records were extracted, of which, 28,053 were identified as antipsychotics or benzodiazepines. Of these, 8.17% (2291/28,053) were modified during validation and verification stages, and 4.75% (1398/29,451) were added. The champion nurse required a mean of 33 min website interaction to verify data, compared with 60 min for manual data entry. Conclusions: The results show that the electronic data collection process is accurate: 95.25% (28,053/29,451) of sedative medications being taken by residents were identified and, of those, 91.83% (25,762/28,053) were correct without any manual intervention. The process worked effectively for nearly all homes. Although the pharmacy packing systems contain some invalid patient records, and data is sometimes incorrectly recorded, validation steps can overcome these problems and provide sufficiently accurate data for the purposes of reporting medication use in individual nursing homes. %M 28778844 %R 10.2196/jmir.6938 %U http://www.jmir.org/2017/8/e283/ %U https://doi.org/10.2196/jmir.6938 %U http://www.ncbi.nlm.nih.gov/pubmed/28778844 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 19 %N 6 %P e221 %T The Effectiveness of Information Technology-Supported Shared Care for Patients With Chronic Disease: A Systematic Review %A Kooij,Laura %A Groen,Wim G %A van Harten,Wim H %+ The Netherlands Cancer Institute, Division of Psychosocial Research and Epidemiology, Plesmanlaan 121, Amsterdam, 1066CX, Netherlands, 31 88 005 75, w.v.harten@nki.nl %K review %K integrated healthcare systems %K health information systems %K chronic disease %D 2017 %7 22.06.2017 %9 Original Paper %J J Med Internet Res %G English %X Background: In patients with chronic disease, many health care professionals are involved during treatment and follow-up. This leads to fragmentation that in turn may lead to suboptimal care. Shared care is a means to improve the integration of care delivered by various providers, specifically primary care physicians (PCPs) and specialty care professionals, for patients with chronic disease. The use of information technology (IT) in this field seems promising. Objective: Our aim was to systematically review the literature regarding the effectiveness of IT-supported shared care interventions in chronic disease in terms of provider or professional, process, health or clinical and financial outcomes. Additionally, our aim was to provide an inventory of the IT applications' characteristics that support such interventions. Methods: PubMed, Scopus, and EMBASE were searched from 2006 to 2015 to identify relevant studies using search terms related to shared care, chronic disease, and IT. Eligible studies were in the English language, and the randomized controlled trials (RCTs), controlled trials, or single group pre-post studies used reported on the effects of IT-supported shared care in patients with chronic disease and cancer. The interventions had to involve providers from both primary and specialty health care. Intervention and IT characteristics and effectiveness—in terms of provider or professional (proximal), process (intermediate), health or clinical and financial (distal) outcomes—were extracted. Risk of bias of (cluster) RCTs was assessed using the Cochrane tool. Results: The initial search yielded 4167 results. Thirteen publications were used, including 11 (cluster) RCTs, a controlled trial, and a pre-post feasibility study. Four main categories of IT applications were identified: (1) electronic decision support tools, (2) electronic platform with a call-center, (3) electronic health records, and (4) electronic communication applications. Positive effects were found for decision support-based interventions on financial and health outcomes, such as physical activity. Electronic health record use improved PCP visits and reduced rehospitalization. Electronic platform use resulted in fewer readmissions and better clinical outcomes—for example, in terms of body mass index (BMI) and dyspnea. The use of electronic communication applications using text-based information transfer between professionals had a positive effect on the number of PCPs contacting hospitals, PCPs’ satisfaction, and confidence. Conclusions: IT-supported shared care can improve proximal outcomes, such as confidence and satisfaction of PCPs, especially in using electronic communication applications. Positive effects on intermediate and distal outcomes were also reported but were mixed. Surprisingly, few studies were found that substantiated these anticipated benefits. Studies showed a large heterogeneity in the included populations, outcome measures, and IT applications used. Therefore, a firm conclusion cannot be drawn. As IT applications are developed and implemented rapidly, evidence is needed to test the specific added value of IT in shared care interventions. This is expected to require innovative research methods. %M 28642218 %R 10.2196/jmir.7405 %U http://www.jmir.org/2017/6/e221/ %U https://doi.org/10.2196/jmir.7405 %U http://www.ncbi.nlm.nih.gov/pubmed/28642218 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 19 %N 6 %P e224 %T The Gap in Medical Informatics and Continuing Education Between the United States and China: A Comparison of Conferences in 2016 %A Liang,Jun %A Wei,Kunyan %A Meng,Qun %A Chen,Zhenying %A Zhang,Jiajie %A Lei,Jianbo %+ Center for Medical Informatics, Peking University, 38 Xueyuan Rd., Haidian District, Beijing, 100191, China, 86 (10) 8280 5901, jblei@hsc.pku.edu.cn %K medical informatics %K conferences %K continuing education %K Sino-American comparison %D 2017 %7 21.06.2017 %9 Original Paper %J J Med Internet Res %G English %X Background: China launched its second health reform in 2010 with considerable investments in medical informatics (MI). However, to the best of our knowledge, research on the outcomes of this ambitious undertaking has been limited. Objective: Our aim was to understand the development of MI and the state of continuing education in China and the United States from the perspective of conferences. Methods: We conducted a quantitative and qualitative analysis of four MI conferences in China and two in the United States: China Medical Information Association Annual Symposium (CMIAAS), China Hospital Information Network Annual Conference (CHINC), China Health Information Technology Exchange Annual Conference (CHITEC), China Annual Proceeding of Medical Informatics (CPMI) versus the American Medical Informatics Association (AMIA) and Healthcare Information and Management Systems Society (HIMSS). The scale, composition, and regional distribution of attendees, topics, and research fields for each conference were summarized and compared. Results: CMIAAS and CPMI are mainstream academic conferences, while CHINC and CHITEC are industry conferences in China. Compared to HIMSS 2016, the meeting duration of CHITEC was 3 versus 5 days, the number of conference sessions was 132 versus 950+, the number of attendees was 5000 versus 40,000+, the number of vendors was 152 versus 1400+, the number of subforums was 12 versus 230, the number of preconference education symposiums and workshops was 0 versus 12, and the duration of preconference educational symposiums and workshops was 0 versus 1 day. Compared to AMIA, the meeting duration of Chinese CMIAAS was 2 versus 5 days, the number of conference sessions was 42 versus 110, the number of attendees was 200 versus 2500+, the number of vendors was 5 versus 75+, and the number of subforums was 4 versus 10. The number of preconference tutorials and working groups was 0 versus 29, and the duration of tutorials and working group was 0 versus 1.5 days. Conclusions: Given the size of the Chinese economy and the substantial investment in MI, the output in terms of conferences remains low. The impact of conferences on continuing education to professionals is not significant. Chinese researchers and professionals should approach MI with greater rigor, including validated research methods, formal training, and effective continuing education, in order to utilize knowledge gained by other countries and to expand collaboration. %M 28637638 %R 10.2196/jmir.8014 %U http://www.jmir.org/2017/6/e224/ %U https://doi.org/10.2196/jmir.8014 %U http://www.ncbi.nlm.nih.gov/pubmed/28637638 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 5 %N 2 %P e14 %T Computerized Childbirth Monitoring Tools for Health Care Providers Managing Labor: A Scoping Review %A Balikuddembe,Michael S %A Tumwesigye,Nazarius M %A Wakholi,Peter K %A Tylleskär,Thorkild %+ Department of Epidemiology and Biostatistics, Makerere University, SPH Bldg. Mulago, Kampala,, Uganda, 256 99876602, balikuddembem@gmail.com %K childbirth %K obstetric labor %K fetal monitoring %K medical informatics applications %K systematic review %D 2017 %7 15.06.2017 %9 Original Paper %J JMIR Med Inform %G English %X Background: Proper monitoring of labor and childbirth prevents many pregnancy-related complications. However, monitoring is still poor in many places partly due to the usability concerns of support tools such as the partograph. In 2011, the World Health Organization (WHO) called for the development and evaluation of context-adaptable electronic health solutions to health challenges. Computerized tools have penetrated many areas of health care, but their influence in supporting health staff with childbirth seems limited. Objective: The objective of this scoping review was to determine the scope and trends of research on computerized labor monitoring tools that could be used by health care providers in childbirth management. Methods: We used key terms to search the Web for eligible peer-reviewed and gray literature. Eligibility criteria were a computerized labor monitoring tool for maternity service providers and dated 2006 to mid-2016. Retrieved papers were screened to eliminate ineligible papers, and consensus was reached on the papers included in the final analysis. Results: We started with about 380,000 papers, of which 14 papers qualified for the final analysis. Most tools were at the design and implementation stages of development. Three papers addressed post-implementation evaluations of two tools. No documentation on clinical outcome studies was retrieved. The parameters targeted with the tools varied, but they included fetal heart (10 of 11 tools), labor progress (8 of 11), and maternal status (7 of 11). Most tools were designed for use in personal computers in low-resource settings and could be customized for different user needs. Conclusions: Research on computerized labor monitoring tools is inadequate. Compared with other labor parameters, there was preponderance to fetal heart monitoring and hardly any summative evaluation of the available tools. More research, including clinical outcomes evaluation of computerized childbirth monitoring tools, is needed. %M 28619702 %R 10.2196/medinform.6959 %U http://medinform.jmir.org/2017/2/e14/ %U https://doi.org/10.2196/medinform.6959 %U http://www.ncbi.nlm.nih.gov/pubmed/28619702 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 19 %N 4 %P e134 %T Web-Based Medical Appointment Systems: A Systematic Review %A Zhao,Peng %A Yoo,Illhoi %A Lavoie,Jaie %A Lavoie,Beau James %A Simoes,Eduardo %+ Department of Health Management and Informatics, School of Medicine, University of Missouri, Clinical Support and Education Building (DC006.00), Five Hospital Dr, Columbia, MO, 65212, United States, 1 5738827642, yooil@health.missouri.edu %K appointments and schedules %K Internet %K smartphone %K patient-centered care %K no-show patients %K hospital information systems %D 2017 %7 26.04.2017 %9 Original Paper %J J Med Internet Res %G English %X Background: Health care is changing with a new emphasis on patient-centeredness. Fundamental to this transformation is the increasing recognition of patients' role in health care delivery and design. Medical appointment scheduling, as the starting point of most non-urgent health care services, is undergoing major developments to support active involvement of patients. By using the Internet as a medium, patients are given more freedom in decision making about their preferences for the appointments and have improved access. Objective: The purpose of this study was to identify the benefits and barriers to implement Web-based medical scheduling discussed in the literature as well as the unmet needs under the current health care environment. Methods: In February 2017, MEDLINE was searched through PubMed to identify articles relating to the impacts of Web-based appointment scheduling. Results: A total of 36 articles discussing 21 Web-based appointment systems were selected for this review. Most of the practices have positive changes in some metrics after adopting Web-based scheduling, such as reduced no-show rate, decreased staff labor, decreased waiting time, and improved satisfaction, and so on. Cost, flexibility, safety, and integrity are major reasons discouraging providers from switching to Web-based scheduling. Patients’ reluctance to adopt Web-based appointment scheduling is mainly influenced by their past experiences using computers and the Internet as well as their communication preferences. Conclusions: Overall, the literature suggests a growing trend for the adoption of Web-based appointment systems. The findings of this review suggest that there are benefits to a variety of patient outcomes from Web-based scheduling interventions with the need for further studies. %M 28446422 %R 10.2196/jmir.6747 %U http://www.jmir.org/2017/4/e134/ %U https://doi.org/10.2196/jmir.6747 %U http://www.ncbi.nlm.nih.gov/pubmed/28446422 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 19 %N 4 %P e122 %T Impact of Information and Communication Technologies on Nursing Care: Results of an Overview of Systematic Reviews %A Rouleau,Geneviève %A Gagnon,Marie-Pierre %A Côté,José %A Payne-Gagnon,Julie %A Hudson,Emilie %A Dubois,Carl-Ardy %+ Faculty of Nursing Sciences, Université Laval, Pavillon Ferdinand-Vandry, 1050 Avenue de la Médecine, Quebec, QC, G1V 0A6, Canada, 1 418 525 4444 ext 53169, marie-pierre.gagnon@fsi.ulaval.ca %K information and communication technology %K eHealth %K telehealth %K nursing care %K review, overview of systematic review %D 2017 %7 25.04.2017 %9 Original Paper %J J Med Internet Res %G English %X Background: Information and communication technologies (ICTs) are becoming an impetus for quality health care delivery by nurses. The use of ICTs by nurses can impact their practice, modifying the ways in which they plan, provide, document, and review clinical care. Objective: An overview of systematic reviews was conducted to develop a broad picture of the dimensions and indicators of nursing care that have the potential to be influenced by the use of ICTs. Methods: Quantitative, mixed-method, and qualitative reviews that aimed to evaluate the influence of four eHealth domains (eg, management, computerized decision support systems [CDSSs], communication, and information systems) on nursing care were included. We used the nursing care performance framework (NCPF) as an extraction grid and analytical tool. This model illustrates how the interplay between nursing resources and the nursing services can produce changes in patient conditions. The primary outcomes included nurses’ practice environment, nursing processes, professional satisfaction, and nursing-sensitive outcomes. The secondary outcomes included satisfaction or dissatisfaction with ICTs according to nurses’ and patients’ perspectives. Reviews published in English, French, or Spanish from January 1, 1995 to January 15, 2015, were considered. Results: A total of 5515 titles or abstracts were assessed for eligibility and full-text papers of 72 articles were retrieved for detailed evaluation. It was found that 22 reviews published between 2002 and 2015 met the eligibility criteria. Many nursing care themes (ie, indicators) were influenced by the use of ICTs, including time management; time spent on patient care; documentation time; information quality and access; quality of documentation; knowledge updating and utilization; nurse autonomy; intra and interprofessional collaboration; nurses’ competencies and skills; nurse-patient relationship; assessment, care planning, and evaluation; teaching of patients and families; communication and care coordination; perspectives of the quality of care provided; nurses and patients satisfaction or dissatisfaction with ICTs; patient comfort and quality of life related to care; empowerment; and functional status. Conclusions: The findings led to the identification of 19 indicators related to nursing care that are impacted by the use of ICTs. To the best of our knowledge, this was the first attempt to apply NCPF in the ICTs’ context. This broad representation could be kept in mind when it will be the time to plan and to implement emerging ICTs in health care settings. Trial Registration: PROSPERO International Prospective Register of Systematic Reviews: CRD42014014762; http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42014014762 (Archived by WebCite at http://www.webcitation.org/6pIhMLBZh) %M 28442454 %R 10.2196/jmir.6686 %U http://www.jmir.org/2017/4/e122/ %U https://doi.org/10.2196/jmir.6686 %U http://www.ncbi.nlm.nih.gov/pubmed/28442454 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 19 %N 2 %P e35 %T Virtual Visits for Acute, Nonurgent Care: A Claims Analysis of Episode-Level Utilization %A Gordon,Aliza S %A Adamson,Wallace C %A DeVries,Andrea R %+ HealthCore, Inc, 123 Justison Street, Suite 200, Wilmington, DE, 19801, United States, 1 302 230 2007, agordon@healthcore.com %K virtual visit %K health care utilization %K claims analysis %D 2017 %7 17.02.2017 %9 Original Paper %J J Med Internet Res %G English %X Background: Expansion of virtual health care—real-time video consultation with a physician via the Internet—will continue as use of mobile devices and patient demand for immediate, convenient access to care grow. Objective: The objective of the study is to analyze the care provided and the cost of virtual visits over a 3-week episode compared with in-person visits to retail health clinics (RHC), urgent care centers (UCC), emergency departments (ED), or primary care physicians (PCP) for acute, nonurgent conditions. Methods: A cross-sectional, retrospective analysis of claims from a large commercial health insurer was performed to compare care and cost of patients receiving care via virtual visits for a condition of interest (sinusitis, upper respiratory infection, urinary tract infection, conjunctivitis, bronchitis, pharyngitis, influenza, cough, dermatitis, digestive symptom, or ear pain) matched to those receiving care for similar conditions in other settings. An episode was defined as the index visit plus 3 weeks following. Patients were children and adults younger than 65 years of age without serious chronic conditions. Visits were classified according to the setting where the visit occurred. Care provided was assessed by follow-up outpatient visits, ED visits, or hospitalizations; laboratory tests or imaging performed; and antibiotic use after the initial visit. Episode costs included the cost of the initial visit, subsequent medical care, and pharmacy. Results: A total of 59,945 visits were included in the analysis (4635 virtual visits and 55,310 nonvirtual visits). Virtual visit episodes had similar follow-up outpatient visit rates (28.09%) as PCP (28.10%, P=.99) and RHC visits (28.59%, P=.51). During the episode, lab rates for virtual visits (12.56%) were lower than in-person locations (RHC: 36.79%, P<.001; UCC: 39.01%, P<.001; ED: 53.15%, P<.001; PCP: 37.40%, P<.001), and imaging rates for virtual visits (6.62%) were typically lower than in-person locations (RHC: 5.97%, P=.11; UCC: 8.77%, P<.001; ED: 43.06%, P<.001; PCP: 11.26%, P<.001). RHC, UCC, ED, and PCP were estimated to be $36, $153, $1735, and $162 more expensive than virtual visit episodes, respectively, including medical and pharmacy costs. Conclusions: Virtual care appears to be a low-cost alternative to care administered in other settings with lower testing rates. The similar follow-up rate suggests adequate clinical resolution and that patients are not using virtual visits as a first step before seeking in-person care. %M 28213342 %R 10.2196/jmir.6783 %U http://www.jmir.org/2017/2/e35/ %U https://doi.org/10.2196/jmir.6783 %U http://www.ncbi.nlm.nih.gov/pubmed/28213342 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 19 %N 2 %P e42 %T Readiness for Delivering Digital Health at Scale: Lessons From a Longitudinal Qualitative Evaluation of a National Digital Health Innovation Program in the United Kingdom %A Lennon,Marilyn R %A Bouamrane,Matt-Mouley %A Devlin,Alison M %A O'Connor,Siobhan %A O'Donnell,Catherine %A Chetty,Ula %A Agbakoba,Ruth %A Bikker,Annemieke %A Grieve,Eleanor %A Finch,Tracy %A Watson,Nicholas %A Wyke,Sally %A Mair,Frances S %+ General Practice and Primary Care, Institute of Health and Wellbeing, University of Glasgow, Horselethill Road, University of Glasgow, Glasgow, G12 9LX, UK, United Kingdom, 44 01413308317, Frances.Mair@glasgow.ac.uk %K telemedicine %K health plan implementation %K community health services %K health services research %K electronic health records %K instrumentation %K qualitative research %K diffusion of innovation %K medical informatics %D 2017 %7 16.02.2017 %9 Original Paper %J J Med Internet Res %G English %X Background: Digital health has the potential to support care delivery for chronic illness. Despite positive evidence from localized implementations, new technologies have proven slow to become accepted, integrated, and routinized at scale. Objective: The aim of our study was to examine barriers and facilitators to implementation of digital health at scale through the evaluation of a £37m national digital health program: ‟Delivering Assisted Living Lifestyles at Scale” (dallas) from 2012-2015. Methods: The study was a longitudinal qualitative, multi-stakeholder, implementation study. The methods included interviews (n=125) with key implementers, focus groups with consumers and patients (n=7), project meetings (n=12), field work or observation in the communities (n=16), health professional survey responses (n=48), and cross program documentary evidence on implementation (n=215). We used a sociological theory called normalization process theory (NPT) and a longitudinal (3 years) qualitative framework analysis approach. This work did not study a single intervention or population. Instead, we evaluated the processes (of designing and delivering digital health), and our outcomes were the identified barriers and facilitators to delivering and mainstreaming services and products within the mixed sector digital health ecosystem. Results: We identified three main levels of issues influencing readiness for digital health: macro (market, infrastructure, policy), meso (organizational), and micro (professional or public). Factors hindering implementation included: lack of information technology (IT) infrastructure, uncertainty around information governance, lack of incentives to prioritize interoperability, lack of precedence on accountability within the commercial sector, and a market perceived as difficult to navigate. Factors enabling implementation were: clinical endorsement, champions who promoted digital health, and public and professional willingness. Conclusions: Although there is receptiveness to digital health, barriers to mainstreaming remain. Our findings suggest greater investment in national and local infrastructure, implementation of guidelines for the safe and transparent use and assessment of digital health, incentivization of interoperability, and investment in upskilling of professionals and the public would help support the normalization of digital health. These findings will enable researchers, health care practitioners, and policy makers to understand the current landscape and the actions required in order to prepare the market and accelerate uptake, and use of digital health and wellness services in context and at scale. %M 28209558 %R 10.2196/jmir.6900 %U http://www.jmir.org/2017/2/e42/ %U https://doi.org/10.2196/jmir.6900 %U http://www.ncbi.nlm.nih.gov/pubmed/28209558 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 5 %N 1 %P e3 %T Patient-Specific Predictive Modeling Using Random Forests: An Observational Study for the Critically Ill %A Lee,Joon %+ Health Data Science Lab, School of Public Health and Health Systems, University of Waterloo, 200 University Ave W, Waterloo, ON, N2L 3G1, Canada, 1 519 888 4567 ext 31567, joon.lee@uwaterloo.ca %K forecasting %K critical care %K predictive analytics %K patient similarity %K random forest %D 2017 %7 17.01.2017 %9 Original Paper %J JMIR Med Inform %G English %X Background: With a large-scale electronic health record repository, it is feasible to build a customized patient outcome prediction model specifically for a given patient. This approach involves identifying past patients who are similar to the present patient and using their data to train a personalized predictive model. Our previous work investigated a cosine-similarity patient similarity metric (PSM) for such patient-specific predictive modeling. Objective: The objective of the study is to investigate the random forest (RF) proximity measure as a PSM in the context of personalized mortality prediction for intensive care unit (ICU) patients. Methods: A total of 17,152 ICU admissions were extracted from the Multiparameter Intelligent Monitoring in Intensive Care II database. A number of predictor variables were extracted from the first 24 hours in the ICU. Outcome to be predicted was 30-day mortality. A patient-specific predictive model was trained for each ICU admission using an RF PSM inspired by the RF proximity measure. Death counting, logistic regression, decision tree, and RF models were studied with a hard threshold applied to RF PSM values to only include the M most similar patients in model training, where M was varied. In addition, case-specific random forests (CSRFs), which uses RF proximity for weighted bootstrapping, were trained. Results: Compared to our previous study that investigated a cosine similarity PSM, the RF PSM resulted in superior or comparable predictive performance. RF and CSRF exhibited the best performances (in terms of mean area under the receiver operating characteristic curve [95% confidence interval], RF: 0.839 [0.835-0.844]; CSRF: 0.832 [0.821-0.843]). RF and CSRF did not benefit from personalization via the use of the RF PSM, while the other models did. Conclusions: The RF PSM led to good mortality prediction performance for several predictive models, although it failed to induce improved performance in RF and CSRF. The distinction between predictor and similarity variables is an important issue arising from the present study. RFs present a promising method for patient-specific outcome prediction. %M 28096065 %R 10.2196/medinform.6690 %U http://medinform.jmir.org/2017/1/e3/ %U https://doi.org/10.2196/medinform.6690 %U http://www.ncbi.nlm.nih.gov/pubmed/28096065 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 4 %N 2 %P e21 %T Users’ Perspectives on a Picture Archiving and Communication System (PACS): An In-Depth Study in a Teaching Hospital in Kuwait %A Buabbas,Ali Jassem %A Al-Shamali,Dawood Ameer %A Sharma,Prem %A Haidar,Salwa %A Al-Shawaf,Hamza %+ Faculty of Medicine, Community Medicine and Behavioral Sciences, Kuwait University, Al-Jabreya, Hawally,, Kuwait, 965 25319504, ali.buabbas@hsc.edu.kw %K PACS evaluation %K user perspective %K IS success %K imaging informatics %K radiology %D 2016 %7 15.06.2016 %9 Original Paper %J JMIR Med Inform %G English %X Background: Picture archiving and communication system (PACS) is a well-known imaging informatics application in health care organizations, specifically designed for the radiology department. Health care providers have exhibited willingness toward evaluating PACS in hospitals to ascertain the critical success and failure of the technology, considering that evaluation is a basic requirement. Objective: This study aimed at evaluating the success of a PACS in a regional teaching hospital of Kuwait, from users’ perspectives, using information systems success criteria. Methods: An in-depth study was conducted by using quantitative and qualitative methods. This mixed-method study was based on: (1) questionnaires, distributed to all radiologists and technologists and (2) interviews, conducted with PACS administrators. Results: In all, 60 questionnaires were received from the respondents. These included 39 radiologists (75% response rate) and 21 technologists (62% response rate), with the results showing almost three-quarters (74%, 44 of 59) of the respondents rating PACS positively and as user friendly. This study’s findings revealed that the demographic data, including computer experience, was an insignificant factor, having no influence on the users’ responses. The findings were further substantiated by the administrators’ interview responses, which supported the benefits of PACS, indicating the need for developing a unified policy aimed at streamlining and improving the departmental workflow. Conclusions: The PACS had a positive and productive impact on the radiologists’ and technologists’ work performance. They were endeavoring to resolve current problems while keeping abreast of advances in PACS technology, including teleradiology and mobile image viewer, which is steadily increasing in usage in the Kuwaiti health system. %M 27307046 %R 10.2196/medinform.5703 %U http://medinform.jmir.org/2016/2/e21/ %U https://doi.org/10.2196/medinform.5703 %U http://www.ncbi.nlm.nih.gov/pubmed/27307046 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 4 %N 2 %P e13 %T Electronic Health Record-Related Safety Concerns: A Cross-Sectional Survey of Electronic Health Record Users %A Palojoki,Sari %A Pajunen,Tuuli %A Saranto,Kaija %A Lehtonen,Lasse %+ Helsinki and Uusimaa Hospital District, Helsinki University Hospital, Group Administration, P.O. Box 100, Stenbäckinkatu 9, Helsinki, 00029, Finland, 358 504284179, sari.palojoki@hus.fi %K Electronic Health Records %K Health Information Technology %K Patient Safety %K Risk Assessment %K Questionnaire %D 2016 %7 06.05.2016 %9 Original Paper %J JMIR Med Inform %G English %X Background: The rapid expansion in the use of electronic health records (EHR) has increased the number of medical errors originating in health information systems (HIS). The sociotechnical approach helps in understanding risks in the development, implementation, and use of EHR and health information technology (HIT) while accounting for complex interactions of technology within the health care system. Objective: This study addresses two important questions: (1) “which of the common EHR error types are associated with perceived high- and extreme-risk severity ratings among EHR users?”, and (2) “which variables are associated with high- and extreme-risk severity ratings?” Methods: This study was a quantitative, non-experimental, descriptive study of EHR users. We conducted a cross-sectional web-based questionnaire study at the largest hospital district in Finland. Statistical tests included the reliability of the summative scales tested with Cronbach’s alpha. Logistic regression served to assess the association of the independent variables to each of the eight risk factors examined. Results: A total of 2864 eligible respondents provided the final data. Almost half of the respondents reported a high level of risk related to the error type “extended EHR unavailability”. The lowest overall risk level was associated with “selecting incorrectly from a list of items”. In multivariate analyses, profession and clinical unit proved to be the strongest predictors for high perceived risk. Physicians perceived risk levels to be the highest (P<.001 in six of eight error types), while emergency departments, operating rooms, and procedure units were associated with higher perceived risk levels (P<.001 in four of eight error types). Previous participation in eLearning courses on EHR-use was associated with lower risk for some of the risk factors. Conclusions: Based on a large number of Finnish EHR users in hospitals, this study indicates that HIT safety hazards should be taken very seriously, particularly in operating rooms, procedure units, emergency departments, and intensive care units/critical care units. Health care organizations should use proactive and systematic assessments of EHR risks before harmful events occur. An EHR training program should be compulsory for all EHR users in order to address EHR safety concerns resulting from the failure to use HIT appropriately. %M 27154599 %R 10.2196/medinform.5238 %U http://medinform.jmir.org/2016/2/e13/ %U https://doi.org/10.2196/medinform.5238 %U http://www.ncbi.nlm.nih.gov/pubmed/27154599 %0 Journal Article %@ 1438-8871 %I JMIR Publications Inc. %V 18 %N 4 %P e75 %T A Patient-Centered Framework for Evaluating Digital Maturity of Health Services: A Systematic Review %A Flott,Kelsey %A Callahan,Ryan %A Darzi,Ara %A Mayer,Erik %+ Centre for Health Policy, Institute of Global Health Innovation, Imperial College London, St Mary’s Campus, Praed Street, London, W2 1NY, United Kingdom, 44 7909248515, k.flott14@imperial.ac.uk %K digital maturity %K evaluation %K health information exchange %K patient-centered care %D 2016 %7 14.04.2016 %9 Original Paper %J J Med Internet Res %G English %X Background: Digital maturity is the extent to which digital technologies are used as enablers to deliver a high-quality health service. Extensive literature exists about how to assess the components of digital maturity, but it has not been used to design a comprehensive framework for evaluation. Consequently, the measurement systems that do exist are limited to evaluating digital programs within one service or care setting, meaning that digital maturity evaluation is not accounting for the needs of patients across their care pathways. Objective: The objective of our study was to identify the best methods and metrics for evaluating digital maturity and to create a novel, evidence-based tool for evaluating digital maturity across patient care pathways. Methods: We systematically reviewed the literature to find the best methods and metrics for evaluating digital maturity. We searched the PubMed database for all papers relevant to digital maturity evaluation. Papers were selected if they provided insight into how to appraise digital systems within the health service and if they indicated the factors that constitute or facilitate digital maturity. Papers were analyzed to identify methodology for evaluating digital maturity and indicators of digitally mature systems. We then used the resulting information about methodology to design an evaluation framework. Following that, the indicators of digital maturity were extracted and grouped into increasing levels of maturity and operationalized as metrics within the evaluation framework. Results: We identified 28 papers as relevant to evaluating digital maturity, from which we derived 5 themes. The first theme concerned general evaluation methodology for constructing the framework (7 papers). The following 4 themes were the increasing levels of digital maturity: resources and ability (6 papers), usage (7 papers), interoperability (3 papers), and impact (5 papers). The framework includes metrics for each of these levels at each stage of the typical patient care pathway. Conclusions: The framework uses a patient-centric model that departs from traditional service-specific measurements and allows for novel insights into how digital programs benefit patients across the health system. Trial Registration: N/A %M 27080852 %R 10.2196/jmir.5047 %U http://www.jmir.org/2016/4/e75/ %U https://doi.org/10.2196/jmir.5047 %U http://www.ncbi.nlm.nih.gov/pubmed/27080852 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 4 %N 1 %P e6 %T Disruptive Innovation: Implementation of Electronic Consultations in a Veterans Affairs Health Care System %A Gupte,Gouri %A Vimalananda,Varsha %A Simon,Steven R %A DeVito,Katerina %A Clark,Justice %A Orlander,Jay D %+ School of Public Health, Department of Health Policy and Management, Boston University, Room 264, 715 Albany Street, Boston, MA, 02118, United States, 1 617 414 1426, gourig@bu.edu %K remote consultations %K clinical communication %K electronic consultation %K telehealth %K clinical information %K decision making, telemonitoring %K eHealth infrastructures %D 2016 %7 12.02.2016 %9 Original Paper %J JMIR Med Inform %G English %X Background: Electronic consultations (e-consults) offer rapid access to specialist input without the need for a patient visit. E-consult implementation began in 2011 at VA Boston Healthcare System (VABHS). By early 2013, e-consults were available for all clinical services. In this implementation, the requesting clinician selects the desired consultation within the electronic health record (EHR) ordering menu, which creates an electronic form that is pre-populated with patient demographic information and allows free-text entry of the reason for consult. This triggers a message to the requesting clinician and requested specialty, thereby enabling bidirectional clinician-clinician communication. Objective: The aim of this study is to examine the utilization of e-consults in a large Veterans Affairs (VA) health care system. Methods: Data from the electronic health record was used to measure frequency of e-consult use by provider type (physician or nurse practitioner (NP) and/or physician assistant), and by the requesting and responding specialty from January 2012 to December 2013. We conducted chart reviews for a purposive sample of e-consults and semi-structured interviews with a purposive sample of clinicians and hospital leaders to better characterize the process, challenges, and usability of e-consults. Results: A total of 7097 e-consults were identified, 1998 from 2012 and 5099 from 2013. More than one quarter (27.56%, 1956/7097) of the e-consult requests originated from VA facilities in New England other than VABHS and were excluded from subsequent analysis. Within the VABHS e-consults (72.44%, 5141/7097), variability in frequency and use of e-consults across provider types and specialties was found. A total of 64 NPs requested 2407 e-consults (median 12.5, range 1-415). In contrast, 448 physicians (including residents and fellows) requested 2349 e-consults (median 2, range 1-116). More than one third (37.35%, 1920/5141) of e-consults were sent from primary care to specialists. While most e-consults reflected a request for specialist input to a generalist’s question in diagnosis or management in the ambulatory setting, we identified creative uses of e-consults, including requests for face-to-face appointments and documentation of pre-operative chart reviews; moreover, 7.00% (360/5141) of the e-consults originated from our sub-acute and chronic care inpatient units. In interviews, requesting providers reported high utility and usability. Specialists recognized the value of e-consults but expressed concerns about additional workload. Conclusions: The e-consult mechanism is frequently utilized for its initial intended purpose. It has also been adopted for unexpected clinical and administrative uses, developing into a “disruptive innovation” and highlighting existing gaps in mechanisms for provider communication. Further investigation is needed to characterize optimal utilization of e-consults within specialty and the medical center, and what features of the e-consult program, other than volume, represent valid measures of access and quality care. %M 26872820 %R 10.2196/medinform.4801 %U http://medinform.jmir.org/2016/1/e6/ %U https://doi.org/10.2196/medinform.4801 %U http://www.ncbi.nlm.nih.gov/pubmed/26872820 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 4 %N 1 %P e4 %T Integration of Provider, Pharmacy, and Patient-Reported Data to Improve Medication Adherence for Type 2 Diabetes: A Controlled Before-After Pilot Study %A Dixon,Brian E %A Alzeer,Abdullah H %A Phillips,Erin O'Kelly %A Marrero,David G %+ Indiana University Richard M. Fairbanks School of Public Health, Department of Epidemiology, 1101 West 10th Street, RF 304, Indianapolis, IN, 46202, United States, 1 3172780046, bedixon@regenstrief.org %K medication adherence %K barriers to medication use %K diabetes mellitus %K type 2 %K medical records systems %K computerized %K health records %K personal %K physician-patient relations %K drug monitoring %K patient-centered care %D 2016 %7 08.02.2016 %9 Original Paper %J JMIR Med Inform %G English %X Background: Patients with diabetes often have poor adherence to using medications as prescribed. The reasons why, however, are not well understood. Furthermore, most health care delivery processes do not routinely assess medication adherence or the factors that contribute to poor adherence. Objective: The objective of the study was to assess the feasibility of an integrated informatics approach to aggregating and displaying clinically relevant data with the potential to identify issues that may interfere with appropriate medication utilization and facilitate patient-provider communication during clinical encounters about strategies to improve medication use. Methods: We developed a clinical dashboard within an electronic health record (EHR) system that uses data from three sources: the medical record, pharmacy claims, and a patient portal. Next, we implemented the dashboard into three community health centers. Health care providers (n=15) and patients with diabetes (n=96) were enrolled in a before-after pilot to test the system’s impact on medication adherence and clinical outcomes. To measure adherence, we calculated the proportion of days covered using pharmacy claims. Demographic, laboratory, and visit data from the EHR were analyzed using pairwise t tests. Perceived barriers to adherence were self-reported by patients. Providers were surveyed about their use and perceptions of the clinical dashboard. Results: Adherence significantly and meaningfully improved (improvements ranged from 6%-20%) consistently across diabetes as well as cardiovascular drug classes. Clinical outcomes, including HbA1c, blood pressure, lipid control, and emergency department utilization remained unchanged. Only a quarter of patients (n=24) logged into the patient portal and completed psychosocial questionnaires about their barriers to taking medications. Conclusions: Integrated approaches using advanced EHR, clinical decision support, and patient-controlled technologies show promise for improving appropriate medication use and supporting better management of chronic conditions. Future research and development is necessary to design, implement, and integrate the myriad of EHR and clinical decision support systems as well as patient-focused information systems into routine care and patient processes that together support health and well-being. %M 26858218 %R 10.2196/medinform.4739 %U http://medinform.jmir.org/2016/1/e4/ %U https://doi.org/10.2196/medinform.4739 %U http://www.ncbi.nlm.nih.gov/pubmed/26858218 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 3 %N 4 %P e39 %T Outcomes From Health Information Exchange: Systematic Review and Future Research Needs %A Hersh,William R %A Totten,Annette M %A Eden,Karen B %A Devine,Beth %A Gorman,Paul %A Kassakian,Steven Z %A Woods,Susan S %A Daeges,Monica %A Pappas,Miranda %A McDonagh,Marian S %+ Pacific Northwest Evidence-Based Practice Center, Department of Medical Informatics & Clinical Epidemiology, Oregon Health & Science University, 3181 SW Sam Jackson Park Rd., BICC, Portland, OR, , United States, 1 503 494 4563, hersh@ohsu.edu %K diagnostic tests %K health information exchange %K outcome assessment (health care) %K patient readmission %K routine %K systematic review %D 2015 %7 15.12.2015 %9 Original Paper %J JMIR Med Inform %G English %X Background: Health information exchange (HIE), the electronic sharing of clinical information across the boundaries of health care organizations, has been promoted to improve the efficiency, cost-effectiveness, quality, and safety of health care delivery. Objective: To systematically review the available research on HIE outcomes and analyze future research needs. Methods: Data sources included citations from selected databases from January 1990 to February 2015. We included English-language studies of HIE in clinical or public health settings in any country. Data were extracted using dual review with adjudication of disagreements. Results: We identified 34 studies on outcomes of HIE. No studies reported on clinical outcomes (eg, mortality and morbidity) or identified harms. Low-quality evidence generally finds that HIE reduces duplicative laboratory and radiology testing, emergency department costs, hospital admissions (less so for readmissions), and improves public health reporting, ambulatory quality of care, and disability claims processing. Most clinicians attributed positive changes in care coordination, communication, and knowledge about patients to HIE. Conclusions: Although the evidence supports benefits of HIE in reducing the use of specific resources and improving the quality of care, the full impact of HIE on clinical outcomes and potential harms are inadequately studied. Future studies must address comprehensive questions, use more rigorous designs, and employ a standard for describing types of HIE. Trial Registration: PROSPERO Registry No CRD42014013285; http://www.crd.york.ac.uk/PROSPERO/ display_record.asp?ID=CRD42014013285 (Archived by WebCite at http://www.webcitation.org/6dZhqDM8t). %M 26678413 %R 10.2196/medinform.5215 %U http://medinform.jmir.org/2015/4/e39/ %U https://doi.org/10.2196/medinform.5215 %U http://www.ncbi.nlm.nih.gov/pubmed/26678413 %0 Journal Article %@ 1438-8871 %I JMIR Publications Inc. %V 17 %N 11 %P e260 %T Bringing Health and Fitness Data Together for Connected Health Care: Mobile Apps as Enablers of Interoperability %A Gay,Valerie %A Leijdekkers,Peter %+ Faculty of Engineering and Information Technology, University of Technology Sydney, PO box 123, Broadway NSW, 2007, Australia, 61 2 9514 4645, Valerie.Gay@uts.edu.au %K health informatics %K connected health %K pervasive and mobile computing %K ubiquitous and mobile devices %D 2015 %7 18.11.2015 %9 Viewpoint %J J Med Internet Res %G English %X Background: A transformation is underway regarding how we deal with our health. Mobile devices make it possible to have continuous access to personal health information. Wearable devices, such as Fitbit and Apple’s smartwatch, can collect data continuously and provide insights into our health and fitness. However, lack of interoperability and the presence of data silos prevent users and health professionals from getting an integrated view of health and fitness data. To provide better health outcomes, a complete picture is needed which combines informal health and fitness data collected by the user together with official health records collected by health professionals. Mobile apps are well positioned to play an important role in the aggregation since they can tap into these official and informal health and data silos. Objective: The objective of this paper is to demonstrate that a mobile app can be used to aggregate health and fitness data and can enable interoperability. It discusses various technical interoperability challenges encountered while integrating data into one place. Methods: For 8 years, we have worked with third-party partners, including wearable device manufacturers, electronic health record providers, and app developers, to connect an Android app to their (wearable) devices, back-end servers, and systems. Results: The result of this research is a health and fitness app called myFitnessCompanion, which enables users to aggregate their data in one place. Over 6000 users use the app worldwide to aggregate their health and fitness data. It demonstrates that mobile apps can be used to enable interoperability. Challenges encountered in the research process included the different wireless protocols and standards used to communicate with wireless devices, the diversity of security and authorization protocols used to be able to exchange data with servers, and lack of standards usage, such as Health Level Seven, for medical information exchange. Conclusions: By limiting the negative effects of health data silos, mobile apps can offer a better holistic view of health and fitness data. Data can then be analyzed to offer better and more personalized advice and care. %M 26581920 %R 10.2196/jmir.5094 %U http://www.jmir.org/2015/11/e260/ %U https://doi.org/10.2196/jmir.5094 %U http://www.ncbi.nlm.nih.gov/pubmed/26581920 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 3 %N 4 %P e36 %T Real-Time and Retrospective Health-Analytics-as-a-Service: A Novel Framework %A Khazaei,Hamzeh %A McGregor,Carolyn %A Eklund,J Mikael %A El-Khatib,Khalil %+ IBM, Canada Research and Development Center, IBM Canada, 3600 Steeles Avenue East, Markham, Toronto, ON, L3R 1H5, Canada, 1 905 721 8668 ext 3697, hamzeh.k.h@ieee.org %K premature babies %K physiological data %K decision support system %K analytics-as-a-service %K cloud computing %K big data, health informatics %K real-time analytics %K retrospective analysis %K performance modeling %D 2015 %7 18.11.2015 %9 Original Paper %J JMIR Med Inform %G English %X Background: Analytics-as-a-service (AaaS) is one of the latest provisions emerging from the cloud services family. Utilizing this paradigm of computing in health informatics will benefit patients, care providers, and governments significantly. This work is a novel approach to realize health analytics as services in critical care units in particular. Objective: To design, implement, evaluate, and deploy an extendable big-data compatible framework for health-analytics-as-a-service that offers both real-time and retrospective analysis. Methods: We present a novel framework that can realize health data analytics-as-a-service. The framework is flexible and configurable for different scenarios by utilizing the latest technologies and best practices for data acquisition, transformation, storage, analytics, knowledge extraction, and visualization. We have instantiated the proposed method, through the Artemis project, that is, a customization of the framework for live monitoring and retrospective research on premature babies and ill term infants in neonatal intensive care units (NICUs). Results: We demonstrated the proposed framework in this paper for monitoring NICUs and refer to it as the Artemis-In-Cloud (Artemis-IC) project. A pilot of Artemis has been deployed in the SickKids hospital NICU. By infusing the output of this pilot set up to an analytical model, we predict important performance measures for the final deployment of Artemis-IC. This process can be carried out for other hospitals following the same steps with minimal effort. SickKids’ NICU has 36 beds and can classify the patients generally into 5 different types including surgical and premature babies. The arrival rate is estimated as 4.5 patients per day, and the average length of stay was calculated as 16 days. Mean number of medical monitoring algorithms per patient is 9, which renders 311 live algorithms for the whole NICU running on the framework. The memory and computation power required for Artemis-IC to handle the SickKids NICU will be 32 GB and 16 CPU cores, respectively. The required amount of storage was estimated as 8.6 TB per year. There will always be 34.9 patients in SickKids NICU on average. Currently, 46% of patients cannot get admitted to SickKids NICU due to lack of resources. By increasing the capacity to 90 beds, all patients can be accommodated. For such a provisioning, Artemis-IC will need 16 TB of storage per year, 55 GB of memory, and 28 CPU cores. Conclusions: Our contributions in this work relate to a cloud architecture for the analysis of physiological data for clinical decisions support for tertiary care use. We demonstrate how to size the equipment needed in the cloud for that architecture based on a very realistic assessment of the patient characteristics and the associated clinical decision support algorithms that would be required to run for those patients. We show the principle of how this could be performed and furthermore that it can be replicated for any critical care setting within a tertiary institution. %M 26582268 %R 10.2196/medinform.4640 %U http://medinform.jmir.org/2015/4/e36/ %U https://doi.org/10.2196/medinform.4640 %U http://www.ncbi.nlm.nih.gov/pubmed/26582268 %0 Journal Article %@ 1438-8871 %I JMIR Publications Inc. %V 17 %N 11 %P e247 %T Analysis of Documentation Speed Using Web-Based Medical Speech Recognition Technology: Randomized Controlled Trial %A Vogel,Markus %A Kaisers,Wolfgang %A Wassmuth,Ralf %A Mayatepek,Ertan %+ University Children’s Hospital Düsseldorf, Department of General Pediatrics, Neonatology and Pediatric Cardiology, Heinrich-Heine-University, Moorenstrasse 5, Düsseldorf, 40225, Germany, 49 21181 ext 16984, markus.vogel@med.uni-duesseldorf.de %K electronic health record %K automatic speech recognition %K randomized controlled trial %D 2015 %7 03.11.2015 %9 Original Paper %J J Med Internet Res %G English %X Background: Clinical documentation has undergone a change due to the usage of electronic health records. The core element is to capture clinical findings and document therapy electronically. Health care personnel spend a significant portion of their time on the computer. Alternatives to self-typing, such as speech recognition, are currently believed to increase documentation efficiency and quality, as well as satisfaction of health professionals while accomplishing clinical documentation, but few studies in this area have been published to date. Objective: This study describes the effects of using a Web-based medical speech recognition system for clinical documentation in a university hospital on (1) documentation speed, (2) document length, and (3) physician satisfaction. Methods: Reports of 28 physicians were randomized to be created with (intervention) or without (control) the assistance of a Web-based system of medical automatic speech recognition (ASR) in the German language. The documentation was entered into a browser’s text area and the time to complete the documentation including all necessary corrections, correction effort, number of characters, and mood of participant were stored in a database. The underlying time comprised text entering, text correction, and finalization of the documentation event. Participants self-assessed their moods on a scale of 1-3 (1=good, 2=moderate, 3=bad). Statistical analysis was done using permutation tests. Results: The number of clinical reports eligible for further analysis stood at 1455. Out of 1455 reports, 718 (49.35%) were assisted by ASR and 737 (50.65%) were not assisted by ASR. Average documentation speed without ASR was 173 (SD 101) characters per minute, while it was 217 (SD 120) characters per minute using ASR. The overall increase in documentation speed through Web-based ASR assistance was 26% (P=.04). Participants documented an average of 356 (SD 388) characters per report when not assisted by ASR and 649 (SD 561) characters per report when assisted by ASR. Participants' average mood rating was 1.3 (SD 0.6) using ASR assistance compared to 1.6 (SD 0.7) without ASR assistance (P<.001). Conclusions: We conclude that medical documentation with the assistance of Web-based speech recognition leads to an increase in documentation speed, document length, and participant mood when compared to self-typing. Speech recognition is a meaningful and effective tool for the clinical documentation process. %M 26531850 %R 10.2196/jmir.5072 %U http://www.jmir.org/2015/11/e247/ %U https://doi.org/10.2196/jmir.5072 %U http://www.ncbi.nlm.nih.gov/pubmed/26531850 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 3 %N 3 %P e30 %T Meaningful Use of Electronic Health Records: Experiences From the Field and Future Opportunities %A Slight,Sarah Patricia %A Berner,Eta S %A Galanter,William %A Huff,Stanley %A Lambert,Bruce L %A Lannon,Carole %A Lehmann,Christoph U %A McCourt,Brian J %A McNamara,Michael %A Menachemi,Nir %A Payne,Thomas H %A Spooner,S Andrew %A Schiff,Gordon D %A Wang,Tracy Y %A Akincigil,Ayse %A Crystal,Stephen %A Fortmann,Stephen P %A Vandermeer,Meredith L %A Bates,David W %+ The Centre for Patient Safety Research and Practice, Division of General Internal Medicine, Brigham and Womens Hospital, 1620 Tremont Street, Boston, MA, MA 02120-1613, United States, 1 617 732 5650, dbates@partners.org %K medical informatics %K health policy %K electronic health records %K meaningful use %D 2015 %7 18.9.2015 %9 Original Paper %J JMIR Med Inform %G English %X Background: With the aim of improving health care processes through health information technology (HIT), the US government has promulgated requirements for “meaningful use” (MU) of electronic health records (EHRs) as a condition for providers receiving financial incentives for the adoption and use of these systems. Considerable uncertainty remains about the impact of these requirements on the effective application of EHR systems. Objective: The Agency for Healthcare Research and Quality (AHRQ)-sponsored Centers for Education and Research in Therapeutics (CERTs) critically examined the impact of the MU policy relating to the use of medications and jointly developed recommendations to help inform future HIT policy. Methods: We gathered perspectives from a wide range of stakeholders (N=35) who had experience with MU requirements, including academicians, practitioners, and policy makers from different health care organizations including and beyond the CERTs. Specific issues and recommendations were discussed and agreed on as a group. Results: Stakeholders’ knowledge and experiences from implementing MU requirements fell into 6 domains: (1) accuracy of medication lists and medication reconciliation, (2) problem list accuracy and the shift in HIT priorities, (3) accuracy of allergy lists and allergy-related standards development, (4) support of safer and effective prescribing for children, (5) considerations for rural communities, and (6) general issues with achieving MU. Standards are needed to better facilitate the exchange of data elements between health care settings. Several organizations felt that their preoccupation with fulfilling MU requirements stifled innovation. Greater emphasis should be placed on local HIT configurations that better address population health care needs. Conclusions: Although MU has stimulated adoption of EHRs, its effects on quality and safety remain uncertain. Stakeholders felt that MU requirements should be more flexible and recognize that integrated models may achieve information-sharing goals in alternate ways. Future certification rules and requirements should enhance EHR functionalities critical for safer prescribing of medications in children. %R 10.2196/medinform.4457 %U http://medinform.jmir.org/2015/3/e30/ %U https://doi.org/10.2196/medinform.4457 %0 Journal Article %@ 1438-8871 %I JMIR Publications Inc. %V 17 %N 7 %P e157 %T Secure Cloud-Based Solutions for Different eHealth Services in Spanish Rural Health Centers %A de la Torre- Díez,Isabel %A Lopez-Coronado,Miguel %A Garcia-Zapirain Soto,Begonya %A Mendez-Zorrilla,Amaia %+ DeustoTech Institute of Technology, DeustoTech-Life Lab, University of Deusto, Avda Universidades, 24, Bilbao, 48007, Spain, 34 944139000, mbgarciazapi@deusto.es %K cloud %K eHealth services %K rural %K security %D 2015 %7 27.07.2015 %9 Original Paper %J J Med Internet Res %G English %X Background: The combination of eHealth applications and/or services with cloud technology provides health care staff—with sufficient mobility and accessibility for them—to be able to transparently check any data they may need without having to worry about its physical location. Objective: The main aim of this paper is to put forward secure cloud-based solutions for a range of eHealth services such as electronic health records (EHRs), telecardiology, teleconsultation, and telediagnosis. Methods: The scenario chosen for introducing the services is a set of four rural health centers located within the same Spanish region. iCanCloud software was used to perform simulations in the proposed scenario. We chose online traffic and the cost per unit in terms of time as the parameters for choosing the secure solution on the most optimum cloud for each service. Results: We suggest that load balancers always be fitted for all solutions in communication together with several Internet service providers and that smartcards be used to maintain identity to an appropriate extent. The solutions offered via private cloud for EHRs, teleconsultation, and telediagnosis services require a volume of online traffic calculated at being able to reach 2 Gbps per consultation. This may entail an average cost of €500/month. Conclusions: The security solutions put forward for each eHealth service constitute an attempt to centralize all information on the cloud, thus offering greater accessibility to medical information in the case of EHRs alongside more reliable diagnoses and treatment for telecardiology, telediagnosis, and teleconsultation services. Therefore, better health care for the rural patient can be obtained at a reasonable cost. %M 26215155 %R 10.2196/jmir.4422 %U http://www.jmir.org/2015/7/e157/ %U https://doi.org/10.2196/jmir.4422 %U http://www.ncbi.nlm.nih.gov/pubmed/26215155 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 3 %N 3 %P e26 %T Building Data-Driven Pathways From Routinely Collected Hospital Data: A Case Study on Prostate Cancer %A Bettencourt-Silva,Joao H %A Clark,Jeremy %A Cooper,Colin S %A Mills,Robert %A Rayward-Smith,Victor J %A de la Iglesia,Beatriz %+ School of Computing Sciences, University of East Anglia, Norwich Research Park, Earlham Road, Norwich, NR4 7TJ, United Kingdom, 44 1603592300, jhbs@cmp.uea.ac.uk %K hospital information systems %K data summarization %K clinical pathways %K data quality %K visualization %K prostate cancer %K electronic medical records %D 2015 %7 10.07.2015 %9 Original Paper %J JMIR Med Inform %G English %X Background: Routinely collected data in hospitals is complex, typically heterogeneous, and scattered across multiple Hospital Information Systems (HIS). This big data, created as a byproduct of health care activities, has the potential to provide a better understanding of diseases, unearth hidden patterns, and improve services and cost. The extent and uses of such data rely on its quality, which is not consistently checked, nor fully understood. Nevertheless, using routine data for the construction of data-driven clinical pathways, describing processes and trends, is a key topic receiving increasing attention in the literature. Traditional algorithms do not cope well with unstructured processes or data, and do not produce clinically meaningful visualizations. Supporting systems that provide additional information, context, and quality assurance inspection are needed. Objective: The objective of the study is to explore how routine hospital data can be used to develop data-driven pathways that describe the journeys that patients take through care, and their potential uses in biomedical research; it proposes a framework for the construction, quality assessment, and visualization of patient pathways for clinical studies and decision support using a case study on prostate cancer. Methods: Data pertaining to prostate cancer patients were extracted from a large UK hospital from eight different HIS, validated, and complemented with information from the local cancer registry. Data-driven pathways were built for each of the 1904 patients and an expert knowledge base, containing rules on the prostate cancer biomarker, was used to assess the completeness and utility of the pathways for a specific clinical study. Software components were built to provide meaningful visualizations for the constructed pathways. Results: The proposed framework and pathway formalism enable the summarization, visualization, and querying of complex patient-centric clinical information, as well as the computation of quality indicators and dimensions. A novel graphical representation of the pathways allows the synthesis of such information. Conclusions: Clinical pathways built from routinely collected hospital data can unearth information about patients and diseases that may otherwise be unavailable or overlooked in hospitals. Data-driven clinical pathways allow for heterogeneous data (ie, semistructured and unstructured data) to be collated over a unified data model and for data quality dimensions to be assessed. This work has enabled further research on prostate cancer and its biomarkers, and on the development and application of methods to mine, compare, analyze, and visualize pathways constructed from routine data. This is an important development for the reuse of big data in hospitals. %M 26162314 %R 10.2196/medinform.4221 %U http://medinform.jmir.org/2015/3/e26/ %U https://doi.org/10.2196/medinform.4221 %U http://www.ncbi.nlm.nih.gov/pubmed/26162314 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 3 %N 2 %P e19 %T Benchmarking Clinical Speech Recognition and Information Extraction: New Data, Methods, and Evaluations %A Suominen,Hanna %A Zhou,Liyuan %A Hanlen,Leif %A Ferraro,Gabriela %+ Canberra Research Laboratory, Machine Learning Research Group, NICTA, Locked Bag 8001, Canberra, ACT, 2601, Australia, 61 431913826, hanna.suominen@nicta.com.au %K computer systems evaluation %K data collection %K information extraction %K nursing records %K patient handoff %K records as topic %K speech recognition software %D 2015 %7 27.04.2015 %9 Original Paper %J JMIR Med Inform %G English %X Background: Over a tenth of preventable adverse events in health care are caused by failures in information flow. These failures are tangible in clinical handover; regardless of good verbal handover, from two-thirds to all of this information is lost after 3-5 shifts if notes are taken by hand, or not at all. Speech recognition and information extraction provide a way to fill out a handover form for clinical proofing and sign-off. Objective: The objective of the study was to provide a recorded spoken handover, annotated verbatim transcriptions, and evaluations to support research in spoken and written natural language processing for filling out a clinical handover form. This dataset is based on synthetic patient profiles, thereby avoiding ethical and legal restrictions, while maintaining efficacy for research in speech-to-text conversion and information extraction, based on realistic clinical scenarios. We also introduce a Web app to demonstrate the system design and workflow. Methods: We experiment with Dragon Medical 11.0 for speech recognition and CRF++ for information extraction. To compute features for information extraction, we also apply CoreNLP, MetaMap, and Ontoserver. Our evaluation uses cross-validation techniques to measure processing correctness. Results: The data provided were a simulation of nursing handover, as recorded using a mobile device, built from simulated patient records and handover scripts, spoken by an Australian registered nurse. Speech recognition recognized 5276 of 7277 words in our 100 test documents correctly. We considered 50 mutually exclusive categories in information extraction and achieved the F1 (ie, the harmonic mean of Precision and Recall) of 0.86 in the category for irrelevant text and the macro-averaged F1 of 0.70 over the remaining 35 nonempty categories of the form in our 101 test documents. Conclusions: The significance of this study hinges on opening our data, together with the related performance benchmarks and some processing software, to the research and development community for studying clinical documentation and language-processing. The data are used in the CLEFeHealth 2015 evaluation laboratory for a shared task on speech recognition. %M 25917752 %R 10.2196/medinform.4321 %U http://medinform.jmir.org/2015/2/e19/ %U https://doi.org/10.2196/medinform.4321 %U http://www.ncbi.nlm.nih.gov/pubmed/25917752 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 3 %N 1 %P e1 %T Impact of Information Technology on Information Gaps in Canadian Ambulatory Care Encounters %A Korosec,Lauren %A Balenko,Krista %A Hagens,Simon %+ Canada Health Infoway, 150 King Street West, #1300, Toronto, ON, M5H 1J9, Canada, 1 888 733 6462, kbalenko@infoway-inforoute.ca %K digital health %K information gaps %K ambulatory %K outpatient %D 2015 %7 08.01.2015 %9 Original Paper %J JMIR Med Inform %G English %X Background: Specialist physicians require clinical information for patient visits in ambulatory encounters, some of which they may access via digital health solutions. Objective: This study explored the completeness of information for patient care and the consequences of gaps for ambulatory specialist services provided in ambulatory settings in Canada. Methods: A sample of specialist physicians practising in outpatient clinics was recruited from a health care provider research panel. The study (n=1800 patient encounters) looked at the completeness of patient information experienced by physicians who work in environments with rich health information exchange (Connected) and a comparison cohort with less information available electronically (Unconnected). Results: Unconnected physicians were significantly more likely to be missing information they needed for patient encounters (13% of encounters for Unconnected physicians vs 7% for Connected physicians). Unconnected physicians were also more likely to report that missing information had consequences (23% vs 13% of encounters). Lab results were the most common type of patient information missing for both Unconnected and Connected specialists (25% for Unconnected physicians vs 11% Connected physicians). Conclusions: The results from this study indicate that Canadian physicians commonly experience information gaps in ambulatory encounters, and that many of these gaps are of consequence to themselves, their patients, and the healthcare system. Wasting physician and patient time, as well as being forced to proceed with incomplete information, were the most common consequences of information gaps reported. %M 25595130 %R 10.2196/medinform.4066 %U http://medinform.jmir.org/2015/1/e1/ %U https://doi.org/10.2196/medinform.4066 %U http://www.ncbi.nlm.nih.gov/pubmed/25595130 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 2 %N 2 %P e29 %T Clinical Decision Support System to Enhance Quality Control of Spirometry Using Information and Communication Technologies %A Burgos,Felip %A Melia,Umberto %A Vallverdú,Montserrat %A Velickovski,Filip %A Lluch-Ariet,Magí %A Caminal,Pere %A Roca,Josep %+ Hospital Clinic - IDIBAPS - Ciberes, Respiratory Diagnostic Center, University of Barcelona, Sotano porta 6, Villarroel, 170, Barcelona, 08036, Spain, 34 932275540, fburgos@ub.edu %K spirometry %K telemedicine %K information communication technologies %K primary care %K quality control %D 2014 %7 21.10.2014 %9 Original Paper %J JMIR Med Inform %G English %X Background: We recently demonstrated that quality of spirometry in primary care could markedly improve with remote offline support from specialized professionals. It is hypothesized that implementation of automatic online assessment of quality of spirometry using information and communication technologies may significantly enhance the potential for extensive deployment of a high quality spirometry program in integrated care settings. Objective: The objective of the study was to elaborate and validate a Clinical Decision Support System (CDSS) for automatic online quality assessment of spirometry. Methods: The CDSS was done through a three step process including: (1) identification of optimal sampling frequency; (2) iterations to build-up an initial version using the 24 standard spirometry curves recommended by the American Thoracic Society; and (3) iterations to refine the CDSS using 270 curves from 90 patients. In each of these steps the results were checked against one expert. Finally, 778 spirometry curves from 291 patients were analyzed for validation purposes. Results: The CDSS generated appropriate online classification and certification in 685/778 (88.1%) of spirometry testing, with 96% sensitivity and 95% specificity. Conclusions: Consequently, only 93/778 (11.9%) of spirometry testing required offline remote classification by an expert, indicating a potential positive role of the CDSS in the deployment of a high quality spirometry program in an integrated care setting. %M 25600957 %R 10.2196/medinform.3179 %U http://medinform.jmir.org/2014/2/e29/ %U https://doi.org/10.2196/medinform.3179 %U http://www.ncbi.nlm.nih.gov/pubmed/25600957 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 2 %N 2 %P e28 %T Clinical Data Miner: An Electronic Case Report Form System With Integrated Data Preprocessing and Machine-Learning Libraries Supporting Clinical Diagnostic Model Research %A Installé,Arnaud JF %A Van den Bosch,Thierry %A De Moor,Bart %A Timmerman,Dirk %+ Department of Electrical Engineering ESAT, STADIUS Center for Dynamical Systems, Signal Processing and Data Analytics, KU Leuven, Kasteelpark Arenberg 10 - box 2446, Leuven, B-3001, Belgium, 32 16 328646, arnaud.installe@esat.kuleuven.be %K data collection %K machine-learning %K clinical decision support systems %K data analysis %D 2014 %7 20.10.2014 %9 Original Paper %J JMIR Med Inform %G English %X Background: Using machine-learning techniques, clinical diagnostic model research extracts diagnostic models from patient data. Traditionally, patient data are often collected using electronic Case Report Form (eCRF) systems, while mathematical software is used for analyzing these data using machine-learning techniques. Due to the lack of integration between eCRF systems and mathematical software, extracting diagnostic models is a complex, error-prone process. Moreover, due to the complexity of this process, it is usually only performed once, after a predetermined number of data points have been collected, without insight into the predictive performance of the resulting models. Objective: The objective of the study of Clinical Data Miner (CDM) software framework is to offer an eCRF system with integrated data preprocessing and machine-learning libraries, improving efficiency of the clinical diagnostic model research workflow, and to enable optimization of patient inclusion numbers through study performance monitoring. Methods: The CDM software framework was developed using a test-driven development (TDD) approach, to ensure high software quality. Architecturally, CDM’s design is split over a number of modules, to ensure future extendability. Results: The TDD approach has enabled us to deliver high software quality. CDM’s eCRF Web interface is in active use by the studies of the International Endometrial Tumor Analysis consortium, with over 4000 enrolled patients, and more studies planned. Additionally, a derived user interface has been used in six separate interrater agreement studies. CDM's integrated data preprocessing and machine-learning libraries simplify some otherwise manual and error-prone steps in the clinical diagnostic model research workflow. Furthermore, CDM's libraries provide study coordinators with a method to monitor a study's predictive performance as patient inclusions increase. Conclusions: To our knowledge, CDM is the only eCRF system integrating data preprocessing and machine-learning libraries. This integration improves the efficiency of the clinical diagnostic model research workflow. Moreover, by simplifying the generation of learning curves, CDM enables study coordinators to assess more accurately when data collection can be terminated, resulting in better models or lower patient recruitment costs. %M 25600863 %R 10.2196/medinform.3251 %U http://medinform.jmir.org/2014/2/e28/ %U https://doi.org/10.2196/medinform.3251 %U http://www.ncbi.nlm.nih.gov/pubmed/25600863 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 2 %N 2 %P e25 %T Return on Investment in Electronic Health Records in Primary Care Practices: A Mixed-Methods Study %A Jang,Yeona %A Lortie,Michel A %A Sanche,Steven %+ McGill University, Desautels Faculty of Management, 1001 Rue Sherbrooke Ouest, Montreal, QC, H3A 1G5, Canada, 1 514 398 8489, yeona.jang@mcgill.ca %K return on investment in electronic health records %K cost recovery from EHR implementation %K ROI indicator %K physician satisfaction with EHR %K primary care practices %D 2014 %7 29.09.2014 %9 Original Paper %J JMIR Med Inform %G English %X Background: The use of electronic health records (EHR) in clinical settings is considered pivotal to a patient-centered health care delivery system. However, uncertainty in cost recovery from EHR investments remains a significant concern in primary care practices. Objective: Guided by the question of “When implemented in primary care practices, what will be the return on investment (ROI) from an EHR implementation?”, the objectives of this study are two-fold: (1) to assess ROI from EHR in primary care practices and (2) to identify principal factors affecting the realization of positive ROI from EHR. We used a break-even point, that is, the time required to achieve cost recovery from an EHR investment, as an ROI indicator of an EHR investment. Methods: Given the complexity exhibited by most EHR implementation projects, this study adopted a retrospective mixed-method research approach, particularly a multiphase study design approach. For this study, data were collected from community-based primary care clinics using EHR systems. Results: We collected data from 17 primary care clinics using EHR systems. Our data show that the sampled primary care clinics recovered their EHR investments within an average period of 10 months (95% CI 6.2-17.4 months), seeing more patients with an average increase of 27% in the active-patients-to-clinician-FTE (full time equivalent) ratio and an average increase of 10% in the active-patients-to-clinical-support-staff-FTE ratio after an EHR implementation. Our analysis suggests, with a 95% confidence level, that the increase in the number of active patients (P=.006), the increase in the active-patients-to-clinician-FTE ratio (P<.001), and the increase in the clinic net revenue (P<.001) are positively associated with the EHR implementation, likely contributing substantially to an average break-even point of 10 months. Conclusions: We found that primary care clinics can realize a positive ROI with EHR. Our analysis of the variances in the time required to achieve cost recovery from EHR investments suggests that a positive ROI does not appear automatically upon implementing an EHR and that a clinic’s ability to leverage EHR for process changes seems to play a role. Policies that provide support to help primary care practices successfully make EHR-enabled changes, such as support of clinic workflow optimization with an EHR system, could facilitate the realization of positive ROI from EHR in primary care practices. %M 25600508 %R 10.2196/medinform.3631 %U http://medinform.jmir.org/2014/2/e25/ %U https://doi.org/10.2196/medinform.3631 %U http://www.ncbi.nlm.nih.gov/pubmed/25600508 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 2 %N 2 %P e21 %T Incorporation of Personal Single Nucleotide Polymorphism (SNP) Data into a National Level Electronic Health Record for Disease Risk Assessment, Part 3: An Evaluation of SNP Incorporated National Health Information System of Turkey for Prostate Cancer %A Beyan,Timur %A Aydın Son,Yeşim %+ Informatics Institute, Department of Health Informatics, Middle East Technical University, Üniversiteler Mahallesi Dumlupınar Bulvarı No:1, ODTÜ Enformatik Enstitüsü B-207, Çankaya, Ankara, 06800, Turkey, 90 312 210 7708, yesim@metu.edu.tr %K health information systems %K clinical decision support systems %K disease risk model %K electronic health record %K epigenetics %K personalized medicine %K single nucleotide polymorphism %D 2014 %7 19.08.2014 %9 Original Paper %J JMIR Med Inform %G English %X Background: A personalized medicine approach provides opportunities for predictive and preventive medicine. Using genomic, clinical, environmental, and behavioral data, the tracking and management of individual wellness is possible. A prolific way to carry this personalized approach into routine practices can be accomplished by integrating clinical interpretations of genomic variations into electronic medical records (EMRs)/electronic health records (EHRs). Today, various central EHR infrastructures have been constituted in many countries of the world, including Turkey. Objective: As an initial attempt to develop a sophisticated infrastructure, we have concentrated on incorporating the personal single nucleotide polymorphism (SNP) data into the National Health Information System of Turkey (NHIS-T) for disease risk assessment, and evaluated the performance of various predictive models for prostate cancer cases. We present our work as a three part miniseries: (1) an overview of requirements, (2) the incorporation of SNP data into the NHIS-T, and (3) an evaluation of SNP data incorporated into the NHIS-T for prostate cancer. Methods: In the third article of this miniseries, we have evaluated the proposed complementary capabilities (ie, knowledge base and end-user application) with real data. Before the evaluation phase, clinicogenomic associations about increased prostate cancer risk were extracted from knowledge sources, and published predictive genomic models assessing individual prostate cancer risk were collected. To evaluate complementary capabilities, we also gathered personal SNP data of four prostate cancer cases and fifteen controls. Using these data files, we compared various independent and model-based, prostate cancer risk assessment approaches. Results: Through the extraction and selection processes of SNP-prostate cancer risk associations, we collected 209 independent associations for increased risk of prostate cancer from the studied knowledge sources. Also, we gathered six cumulative models and two probabilistic models. Cumulative models and assessment of independent associations did not have impressive results. There was one of the probabilistic, model-based interpretation that was successful compared to the others. In envirobehavioral and clinical evaluations, we found that some of the comorbidities, especially, would be useful to evaluate disease risk. Even though we had a very limited dataset, a comparison of performances of different disease models and their implementation with real data as use case scenarios helped us to gain deeper insight into the proposed architecture. Conclusions: In order to benefit from genomic variation data, existing EHR/EMR systems must be constructed with the capability of tracking and monitoring all aspects of personal health status (genomic, clinical, environmental, etc) in 24/7 situations, and also with the capability of suggesting evidence-based recommendations. A national-level, accredited knowledge base is a top requirement for improved end-user systems interpreting these parameters. Finally, categorization using similar, individual characteristics (SNP patterns, exposure history, etc) may be an effective way to predict disease risks, but this approach needs to be concretized and supported with new studies. %M 25600087 %R 10.2196/medinform.3560 %U http://medinform.jmir.org/2014/2/e21/ %U https://doi.org/10.2196/medinform.3560 %U http://www.ncbi.nlm.nih.gov/pubmed/25600087 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 2 %N 2 %P e17 %T Incorporation of Personal Single Nucleotide Polymorphism (SNP) Data into a National Level Electronic Health Record for Disease Risk Assessment, Part 2: The Incorporation of SNP into the National Health Information System of Turkey %A Beyan,Timur %A Aydın Son,Yeşim %+ Informatics Institute, Department of Health Informatics, Middle East Technical University, Üniversiteler Mahallesi Dumlupınar Bulvarı No:1, ODTÜ Enformatik Enstitüsü B-207, Çankaya, Ankara, 06800, Turkey, 90 3122107708, yesim@metu.edu.tr %K health information systems %K clinical decision support systems %K disease risk model %K electronic health record %K epigenetics %K personalized medicine %K single nucleotide polymorphism %D 2014 %7 11.08.2014 %9 Original Paper %J JMIR Med Inform %G English %X Background: A personalized medicine approach provides opportunities for predictive and preventive medicine. Using genomic, clinical, environmental, and behavioral data, the tracking and management of individual wellness is possible. A prolific way to carry this personalized approach into routine practices can be accomplished by integrating clinical interpretations of genomic variations into electronic medical record (EMR)s/electronic health record (EHR)s systems. Today, various central EHR infrastructures have been constituted in many countries of the world, including Turkey. Objective: As an initial attempt to develop a sophisticated infrastructure, we have concentrated on incorporating the personal single nucleotide polymorphism (SNP) data into the National Health Information System of Turkey (NHIS-T) for disease risk assessment, and evaluated the performance of various predictive models for prostate cancer cases. We present our work as a miniseries containing three parts: (1) an overview of requirements, (2) the incorporation of SNP into the NHIS-T, and (3) an evaluation of SNP data incorporated into the NHIS-T for prostate cancer. Methods: For the second article of this miniseries, we have analyzed the existing NHIS-T and proposed the possible extensional architectures. In light of the literature survey and characteristics of NHIS-T, we have proposed and argued opportunities and obstacles for a SNP incorporated NHIS-T. A prototype with complementary capabilities (knowledge base and end-user applications) for these architectures has been designed and developed. Results: In the proposed architectures, the clinically relevant personal SNP (CR-SNP) and clinicogenomic associations are shared between central repositories and end-users via the NHIS-T infrastructure. To produce these files, we need to develop a national level clinicogenomic knowledge base. Regarding clinicogenomic decision support, we planned to complete interpretation of these associations on the end-user applications. This approach gives us the flexibility to add/update envirobehavioral parameters and family health history that will be monitored or collected by end users. Conclusions: Our results emphasized that even though the existing NHIS-T messaging infrastructure supports the integration of SNP data and clinicogenomic association, it is critical to develop a national level, accredited knowledge base and better end-user systems for the interpretation of genomic, clinical, and envirobehavioral parameters. %M 25599817 %R 10.2196/medinform.3555 %U http://medinform.jmir.org/2014/2/e17/ %U https://doi.org/10.2196/medinform.3555 %U http://www.ncbi.nlm.nih.gov/pubmed/25599817 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 2 %N 2 %P e14 %T OWLing Clinical Data Repositories With the Ontology Web Language %A Lozano-Rubí,Raimundo %A Pastor,Xavier %A Lozano,Esther %+ Hospital Clínic, Unit of Medical Informatics, University of Barcelona, Villarroel 170, Barcelona, 08036, Spain, 34 932279206, rlozano@clinic.ub.es %K biomedical ontologies %K data storage and retrieval %K knowledge management %K data sharing %K electronic health records %D 2014 %7 01.08.2014 %9 Original Paper %J JMIR Med Inform %G English %X Background: The health sciences are based upon information. Clinical information is usually stored and managed by physicians with precarious tools, such as spreadsheets. The biomedical domain is more complex than other domains that have adopted information and communication technologies as pervasive business tools. Moreover, medicine continuously changes its corpus of knowledge because of new discoveries and the rearrangements in the relationships among concepts. This scenario makes it especially difficult to offer good tools to answer the professional needs of researchers and constitutes a barrier that needs innovation to discover useful solutions. Objective: The objective was to design and implement a framework for the development of clinical data repositories, capable of facing the continuous change in the biomedicine domain and minimizing the technical knowledge required from final users. Methods: We combined knowledge management tools and methodologies with relational technology. We present an ontology-based approach that is flexible and efficient for dealing with complexity and change, integrated with a solid relational storage and a Web graphical user interface. Results: Onto Clinical Research Forms (OntoCRF) is a framework for the definition, modeling, and instantiation of data repositories. It does not need any database design or programming. All required information to define a new project is explicitly stated in ontologies. Moreover, the user interface is built automatically on the fly as Web pages, whereas data are stored in a generic repository. This allows for immediate deployment and population of the database as well as instant online availability of any modification. Conclusions: OntoCRF is a complete framework to build data repositories with a solid relational storage. Driven by ontologies, OntoCRF is more flexible and efficient to deal with complexity and change than traditional systems and does not require very skilled technical people facilitating the engineering of clinical software systems. %M 25599697 %R 10.2196/medinform.3023 %U http://medinform.jmir.org/2014/2/e14/ %U https://doi.org/10.2196/medinform.3023 %U http://www.ncbi.nlm.nih.gov/pubmed/25599697 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 2 %N 2 %P e15 %T Incorporation of Personal Single Nucleotide Polymorphism (SNP) Data into a National Level Electronic Health Record for Disease Risk Assessment, Part 1: An Overview of Requirements %A Beyan,Timur %A Aydın Son,Yeşim %+ Informatics Institute, Department of Health Informatics, Middle East Technical University, Üniversiteler Mahallesi Dumlupınar Bulvarı No:1, ODTÜ Enformatik Enstitüsü B-207, Ankara, 06800, Turkey, 90 312 210 7708, yesim@metu.edu.tr %K health information systems %K clinical decision support systems %K disease risk model %K electronic health record %K epigenetics %K personalized medicine %K single nucleotide polymorphism %D 2014 %7 24.07.2014 %9 Original Paper %J JMIR Med Inform %G English %X Background: Personalized medicine approaches provide opportunities for predictive and preventive medicine. Using genomic, clinical, environmental, and behavioral data, tracking and management of individual wellness is possible. A prolific way to carry this personalized approach into routine practices can be accomplished by integrating clinical interpretations of genomic variations into electronic medical records (EMRs)/electronic health records (EHRs). Today, various central EHR infrastructures have been constituted in many countries of the world including Turkey. Objective: The objective of this study was to concentrate on incorporating the personal single nucleotide polymorphism (SNP) data into the National Health Information System of Turkey (NHIS-T) for disease risk assessment, and evaluate the performance of various predictive models for prostate cancer cases. We present our work as a miniseries containing three parts: (1) an overview of requirements, (2) the incorporation of SNP into the NHIS-T, and (3) an evaluation of SNP incorporated NHIS-T for prostate cancer. Methods: For the first article of this miniseries, the scientific literature is reviewed and the requirements of SNP data integration into EMRs/EHRs are extracted and presented. Results: In the literature, basic requirements of genomic-enabled EMRs/EHRs are listed as incorporating genotype data and its clinical interpretation into EMRs/EHRs, developing accurate and accessible clinicogenomic interpretation resources (knowledge bases), interpreting and reinterpreting of variant data, and immersing of clinicogenomic information into the medical decision processes. In this section, we have analyzed these requirements under the subtitles of terminology standards, interoperability standards, clinicogenomic knowledge bases, defining clinical significance, and clinicogenomic decision support. Conclusions: In order to integrate structured genotype and phenotype data into any system, there is a need to determine data components, terminology standards, and identifiers of clinicogenomic information. Also, we need to determine interoperability standards to share information between different information systems of stakeholders, and develop decision support capability to interpret genomic variations based on the knowledge bases via different assessment approaches. %M 25599712 %R 10.2196/medinform.3169 %U http://medinform.jmir.org/2014/2/e15/ %U https://doi.org/10.2196/medinform.3169 %U http://www.ncbi.nlm.nih.gov/pubmed/25599712 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 2 %N 1 %P e12 %T Electronic Clinical Safety Reporting System: A Benefits Evaluation %A Elliott,Pamela %A Martin,Desmond %A Neville,Doreen %+ Elliott's Enterprises, 320 Torbay Road, Suite 204, St John's, NL, A1A 4E1, Canada, 1 709 754 6706, pelliott@nl.rogers.com %K electronic occurrence reporting %K electronic clinical safety reporting %K adverse event reporting in health care %K evaluating electronic reporting systems in health care %K health information technology evaluations %D 2014 %7 11.06.2014 %9 Original Paper %J JMIR Med Inform %G English %X Background: Eastern Health, a large health care organization in Newfoundland and Labrador (NL), started a staged implementation of an electronic occurrence reporting system (used interchangeably with “clinical safety reporting system”) in 2008, completing Phase One in 2009. The electronic clinical safety reporting system (CSRS) was designed to replace a paper-based system. The CSRS involves reporting on occurrences such as falls, safety/security issues, medication errors, treatment and procedural mishaps, medical equipment malfunctions, and close calls. The electronic system was purchased from a vendor in the United Kingdom that had implemented the system in the United Kingdom and other places, such as British Columbia. The main objective of the new system was to improve the reporting process with the goal of improving clinical safety. The project was funded jointly by Eastern Health and Canada Health Infoway. Objective: The objectives of the evaluation were to: (1) assess the CSRS on achieving its stated objectives (particularly, the benefits realized and lessons learned), and (2) identify contributions, if any, that can be made to the emerging field of electronic clinical safety reporting. Methods: The evaluation involved mixed methods, including extensive stakeholder participation, pre/post comparative study design, and triangulation of data where possible. The data were collected from several sources, such as project documentation, occurrence reporting records, stakeholder workshops, surveys, focus groups, and key informant interviews. Results: The findings provided evidence that frontline staff and managers support the CSRS, identifying both benefits and areas for improvement. Many benefits were realized, such as increases in the number of occurrences reported, in occurrences reported within 48 hours, in occurrences reported by staff other than registered nurses, in close calls reported, and improved timelines for notification. There was also user satisfaction with the tool regarding ease of use, accessibility, and consistency. The implementation process encountered challenges related to customizing the software and the development of the classification system for coding occurrences. This impacted on the ability of the managers to close-out files in a timely fashion. The issues that were identified, and suggestions for improvements to the form itself, were shared with the Project Team as soon as they were noted. Changes were made to the system before the rollout. Conclusions: There were many benefits realized from the new system that can contribute to improved clinical safety. The participants preferred the electronic system over the paper-based system. The lessons learned during the implementation process resulted in recommendations that informed the rollout of the system in Eastern Health, and in other health care organizations in the province of Newfoundland and Labrador. This study also informed the evaluation of other health organizations in the province, which was completed in 2013. %M 25600569 %R 10.2196/medinform.3316 %U http://medinform.jmir.org/2014/1/e12/ %U https://doi.org/10.2196/medinform.3316 %U http://www.ncbi.nlm.nih.gov/pubmed/25600569 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 2 %N 1 %P e2 %T Designing an Algorithm to Preserve Privacy for Medical Record Linkage With Error-Prone Data %A Pal,Doyel %A Chen,Tingting %A Zhong,Sheng %A Khethavath,Praveen %+ State Key Laboratory for Novel Software Technology, Nanjing University, Department of Computer Science and Technology, Nanjing University, Nanjing, 210023, China, 86 25 89680686, sheng.zhong@gmail.com %K privacy %K medical record linkage %K error-prone data %D 2014 %7 20.01.2014 %9 Original Paper %J JMIR Med Inform %G English %X Background: Linking medical records across different medical service providers is important to the enhancement of health care quality and public health surveillance. In records linkage, protecting the patients’ privacy is a primary requirement. In real-world health care databases, records may well contain errors due to various reasons such as typos. Linking the error-prone data and preserving data privacy at the same time are very difficult. Existing privacy preserving solutions for this problem are only restricted to textual data. Objective: To enable different medical service providers to link their error-prone data in a private way, our aim was to provide a holistic solution by designing and developing a medical record linkage system for medical service providers. Methods: To initiate a record linkage, one provider selects one of its collaborators in the Connection Management Module, chooses some attributes of the database to be matched, and establishes the connection with the collaborator after the negotiation. In the Data Matching Module, for error-free data, our solution offered two different choices for cryptographic schemes. For error-prone numerical data, we proposed a newly designed privacy preserving linking algorithm named the Error-Tolerant Linking Algorithm, that allows the error-prone data to be correctly matched if the distance between the two records is below a threshold. Results: We designed and developed a comprehensive and user-friendly software system that provides privacy preserving record linkage functions for medical service providers, which meets the regulation of Health Insurance Portability and Accountability Act. It does not require a third party and it is secure in that neither entity can learn the records in the other’s database. Moreover, our novel Error-Tolerant Linking Algorithm implemented in this software can work well with error-prone numerical data. We theoretically proved the correctness and security of our Error-Tolerant Linking Algorithm. We have also fully implemented the software. The experimental results showed that it is reliable and efficient. The design of our software is open so that the existing textual matching methods can be easily integrated into the system. Conclusions: Designing algorithms to enable medical records linkage for error-prone numerical data and protect data privacy at the same time is difficult. Our proposed solution does not need a trusted third party and is secure in that in the linking process, neither entity can learn the records in the other’s database. %M 25600786 %R 10.2196/medinform.3090 %U http://medinform.jmir.org/2014/1/e2/ %U https://doi.org/10.2196/medinform.3090 %U http://www.ncbi.nlm.nih.gov/pubmed/25600786 %0 Journal Article %@ 2291-9694 %I Gunther Eysenbach %V 1 %N 1 %P e1 %T Factors Associated With Adoption of the Electronic Health Record System Among Primary Care Physicians %A Cheung,Clement SK %A Tong,Ellen LH %A Cheung,Ngai Tseung %A Chan,Wai Man %A Wang,Harry HX %A Kwan,Mandy WM %A Fan,Carmen KM %A Liu,Kirin QL %A Wong,Martin CS %+ School of Public Health and Primary Care, The Chinese University of Hong Kong, 4/F, The Jockey Club School of Public Health and Primary Care, Prince of Wales Hospital, Shatin, New Territories, Hong Kong SAR, China, Shatin, NT, Hong Kong, , China (Hong Kong), 852 22528782, drwong_martin@yahoo.com.hk %K electronic medical record %K physicians %K adoption %K associated factors %K medical informatics %D 2013 %7 26.08.2013 %9 Original Paper %J JMIR Med Inform %G English %X Background: A territory-wide Internet-based electronic patient record allows better patient care in different sectors. The engagement of private physicians is one of the major facilitators for implementation, but there is limited information about the current adoption level of electronic medical record (eMR) among private primary care physicians. Objective: This survey measured the adoption level, enabling factors, and hindering factors of eMR, among private physicians in Hong Kong. It also evaluated the key functions and the popularity of electronic systems and vendors used by these private practitioners. Methods: A central registry consisting of 4324 private practitioners was set up. Invitations for self-administered surveys and the completed questionnaires were sent and returned via fax, email, postal mail, and on-site clinic visits. Current users and non-users of eMR system were compared according to their demographic and practice characteristics. Student’s t tests and chi-square tests were used for continuous and categorical variables, respectively. Results: A total of 524 completed surveys (response rate 524/4405 11.90%) were collected. The proportion of using eMR in private clinics was 79.6% (417/524). When compared with non-users, the eMR users were younger (users: 48.4 years SD 10.6 years vs non-users: 61.7 years SD 10.2 years, P<.001); more were female physicians (users: 80/417, 19.2% vs non-users: 14/107, 13.1%, P=.013); possessed less clinical experience (with more than20 years of practice: users: 261/417, 62.6% vs non-user: 93/107, 86.9%, P<.001); fewer worked under a Health Maintenance Organization (users: 347/417, 83.2% vs non-users: 97/107, 90.7%, P<.001) and more worked with practice partners (users: 126/417, 30.2% vs non-users: 4/107, 3.7%, P<.001). Efficiency (379/417, 90.9%) and reduction of medical errors (229/417, 54.9%) were the major enabling factors, while patient-unfriendliness (58/107, 54.2%) and limited consultation time (54/107, 50.5%) were the most commonly reported hindering factors. The key functions of computer software among eMR users consisted of electronic patient registration system (376/417, 90.2%), drug dispensing system (328/417, 78.7%) and electronic drug labels (296/417, 71.0%). SoftLink Clinic Solution was the most popular vendor (160/417, 38.4%). Conclusions: These findings identified several physician groups who should be targeted for more assistance on eMR installation and its adoption. Future studies should address the barriers of using Internet-based eMR to enhance its adoption. %M 25599989 %R 10.2196/medinform.2766 %U http://medinform.jmir.org/2013/1/e1/ %U https://doi.org/10.2196/medinform.2766 %U http://www.ncbi.nlm.nih.gov/pubmed/25599989 %0 Journal Article %@ 1929-073X %I JMIR Publications Inc. %V 2 %N 2 %P e26 %T The Computerized Medical Record as a Tool for Clinical Governance in Australian Primary Care %A Pearce,Christopher Martin %A de Lusignan,Simon %A Phillips,Christine %A Hall,Sally %A Travaglia,Joanne %+ Inner East Melbourne Medicare Local, 6 Lakeside Drive, Burwood East, 3151, Australia, 61 3 8822 8444, chris.pearce@monash.edu %K clinical governance %K electronic health records %K general practice %K realist evaluation %K quality assurance %K health care %D 2013 %7 12.08.2013 %9 Review %J Interact J Med Res %G English %X Background: Computerized medical records (CMR) are used in most Australian general practices. Although CMRs have the capacity to amalgamate and provide data to the clinician about their standard of care, there is little research on the way in which they may be used to support clinical governance: the process of ensuring quality and accountability that incorporates the obligation that patients are treated according to best evidence. Objective: The objective of this study was to explore the capability, capacity, and acceptability of CMRs to support clinical governance. Methods: We conducted a realist review of the role of seven CMR systems in implementing clinical governance, developing a four-level maturity model for the CMR. We took Australian primary care as the context, CMR to be the mechanism, and looked at outcomes for individual patients, localities, and for the population in terms of known evidence-based surrogates or true outcome measures. Results: The lack of standardization of CMRs makes national and international benchmarking challenging. The use of the CMR was largely at level two of our maturity model, indicating a relatively simple system in which most of the process takes place outside of the CMR, and which has little capacity to support benchmarking, practice comparisons, and population-level activities. Although national standards for coding and projects for record access are proposed, they are not operationalized. Conclusions: The current CMR systems can support clinical governance activities; however, unless the standardization and data quality issues are addressed, it will not be possible for current systems to work at higher levels. %M 23939340 %R 10.2196/ijmr.2700 %U http://www.i-jmr.org/2013/2/e26/ %U https://doi.org/10.2196/ijmr.2700 %U http://www.ncbi.nlm.nih.gov/pubmed/23939340 %0 Journal Article %@ 14388871 %I JMIR Publications Inc. %V 15 %N 4 %P e75 %T Computing Health Quality Measures Using Informatics for Integrating Biology and the Bedside %A Klann,Jeffrey G %A Murphy,Shawn N %+ Laboratory of Computer Science, Massachusetts General Hospital, One Constitution Center, Boston, MA, 02129, United States, 1 6176435879, jklann@partners.org %K medical informatics %K healthcare quality assessment %K reimbursement %K incentive %K systems integration %K database management systems %D 2013 %7 19.04.2013 %9 Original Paper %J J Med Internet Res %G English %X Background: The Health Quality Measures Format (HQMF) is a Health Level 7 (HL7) standard for expressing computable Clinical Quality Measures (CQMs). Creating tools to process HQMF queries in clinical databases will become increasingly important as the United States moves forward with its Health Information Technology Strategic Plan to Stages 2 and 3 of the Meaningful Use incentive program (MU2 and MU3). Informatics for Integrating Biology and the Bedside (i2b2) is one of the analytical databases used as part of the Office of the National Coordinator (ONC)’s Query Health platform to move toward this goal. Objective: Our goal is to integrate i2b2 with the Query Health HQMF architecture, to prepare for other HQMF use-cases (such as MU2 and MU3), and to articulate the functional overlap between i2b2 and HQMF. Therefore, we analyze the structure of HQMF, and then we apply this understanding to HQMF computation on the i2b2 clinical analytical database platform. Specifically, we develop a translator between two query languages, HQMF and i2b2, so that the i2b2 platform can compute HQMF queries. Methods: We use the HQMF structure of queries for aggregate reporting, which define clinical data elements and the temporal and logical relationships between them. We use the i2b2 XML format, which allows flexible querying of a complex clinical data repository in an easy-to-understand domain-specific language. Results: The translator can represent nearly any i2b2-XML query as HQMF and execute in i2b2 nearly any HQMF query expressible in i2b2-XML. This translator is part of the freely available reference implementation of the QueryHealth initiative. We analyze limitations of the conversion and find it covers many, but not all, of the complex temporal and logical operators required by quality measures. Conclusions: HQMF is an expressive language for defining quality measures, and it will be important to understand and implement for CQM computation, in both meaningful use and population health. However, its current form might allow complexity that is intractable for current database systems (both in terms of implementation and computation). Our translator, which supports the subset of HQMF currently expressible in i2b2-XML, may represent the beginnings of a practical compromise. It is being pilot-tested in two Query Health demonstration projects, and it can be further expanded to balance computational tractability with the advanced features needed by measure developers. %M 23603227 %R 10.2196/jmir.2493 %U http://www.jmir.org/2013/4/e75/ %U https://doi.org/10.2196/jmir.2493 %U http://www.ncbi.nlm.nih.gov/pubmed/23603227 %0 Journal Article %@ 1438-8871 %I Gunther Eysenbach %V 15 %N 2 %P e36 %T What Do Electronic Health Record Vendors Reveal About Their Products: An Analysis of Vendor Websites %A Yeung,Natalie K %A Jadad,Alejandro R %A Shachak,Aviv %+ University of Toronto, Institute of Health Policy, Management and Evaluation, 155 College St., Toronto, ON, M5T 3M6, Canada, 1 416 978 0998, aviv.shachak@utoronto.ca %K Electronic health record %K Vendors %K Diffusion of Innovations %K Websites %D 2013 %7 19.02.2013 %9 Original Paper %J J Med Internet Res %G English %X Background: Purchasing electronic health records (EHRs) typically follows a process in which potential adopters actively seek information, compare alternatives, and form attitudes towards the product. A potential source of information on EHRs that can be used in the process is vendor websites. It is unclear how much product information is presented on EHR vendor websites or the extent of its value during EHR purchasing decisions. Objective: To explore what features of EHR systems are presented by vendors in Ontario, Canada, on their websites, and the persuasive means they use to market such systems; to compare the online information available about primary care EHR systems with that about hospital EHR systems, and with data compiled by OntarioMD, a regional certifying agency. Methods: A list of EHR systems available in Ontario was created. The contents of vendor websites were analyzed. A template for data collection and organization was developed and used to collect and organize information on the vendor, website content, and EHR features. First, we mapped information on system features to categories based on a framework from the Institute of Medicine (IOM). Second, we used a grounded theory–like approach to explore information for building consumer confidence in the vendor and product, and the various persuasive strategies employed on vendor websites. All data were first coded by one researcher. A peer reviewer independently analyzed a randomly chosen subset of the websites (10 of 21; 48%) and provided feedback towards a unified coding scheme. All data were then re-coded and categorized into themes. Finally, we compared information from vendor websites and data gathered by OntarioMD. Results: Vendors provided little specific product information on their websites. Only two of five acute care EHR websites (40%) and nine of 16 websites for primary care systems (56%) featured seven or all eight of the IOM components. Several vendor websites included system interface demonstrations: screenshots (six websites), public videos or slideshows (four websites), or for registered viewers only (three websites). Persuasive means used by vendors included testimonials on 14/21 (67%) websites, and directional language. Except for one free system, trial EHR versions were not available. OntarioMD provided more comprehensive information about primary care systems than the vendors’ websites. Of 14 points of comparison, only the inclusion of templates and bilingual interfaces were fully represented in both data sources. For all other categories, the vendor websites were less complete than the OntarioMD site. Conclusions: EHR vendor websites employ various persuasive means, but lack product-specific information and do not provide options for trying systems on a limited basis. This may impede the ability of potential adopters to form perceptions and compare various offerings. Both vendors and clients could benefit from greater transparency and more specific product information on the Web. Trial Registration: N/A %M 23422722 %R 10.2196/jmir.2312 %U http://www.jmir.org/2013/2/e36/ %U https://doi.org/10.2196/jmir.2312 %U http://www.ncbi.nlm.nih.gov/pubmed/23422722 %0 Journal Article %@ 1929-073X %I JMIR Publications Inc. %V 2 %N 1 %P e1 %T Does Socioeconomic Status Affect Patients’ Ease of Use of a Touch-Screen (iPad) Patient Survey? %A Zarghom,Saman %A Di Fonzo,David %A Leung,Fok-Han %+ University of Toronto, Department of Family and Community Medicine, St. Michael's Hospital, Health Centre at 80 Bond, 80 Bond Street, Toronto, ON, M5B 1X2, Canada, 1 4168643011, leungf@smh.ca %K socioeconomic factors %K age factors %K medical informatics %K computer-user interface %D 2013 %7 11.01.2013 %9 Brief Report %J Interact J Med Res %G English %X Socioeconomic disparities influence the usage rate of advanced communication technologies in Canada. It is important to assess all patient interactions with computers and electronic devices based on these socioeconomic differences. This project studied the ease of use of a touch-screen interface program for collecting patient feedback. The interface collected feedback on physicians’ communication skills, an important health concern that has been garnering more and more attention. A concurrent paper survey was used to gather information on the socioeconomic status and the usability of the touchscreen device. As expected, patients who were older, had lower annual household income, and had lower educational attainment were associated with more difficulty using the devices. Surprisingly, 94% of all users (representing a wide range of socioeconomic status backgrounds) rated the device as easy to use. %M 23612116 %R 10.2196/ijmr.2314 %U http://www.i-jmr.org/2013/1/e1/ %U https://doi.org/10.2196/ijmr.2314 %U http://www.ncbi.nlm.nih.gov/pubmed/23612116 %0 Journal Article %@ 1438-8871 %I Gunther Eysenbach %V 14 %N 1 %P e21 %T Development and Implementation of a Web-Enabled 3D Consultation Tool for Breast Augmentation Surgery Based on 3D-Image Reconstruction of 2D Pictures %A de Heras Ciechomski,Pablo %A Constantinescu,Mihai %A Garcia,Jaime %A Olariu,Radu %A Dindoyal,Irving %A Le Huu,Serge %A Reyes,Mauricio %+ Institute for Surgical Technology and Biomechanics, University of Bern, Staufacherstrasse 78, Bern, 3014, Switzerland, 41 316315950, mauricio.reyes@istb.unibe.ch %K Medical informatics computing %K computer-assisted surgery %K imaging, three-dimensional %D 2012 %7 03.02.2012 %9 Original Paper %J J Med Internet Res %G English %X Background: Producing a rich, personalized Web-based consultation tool for plastic surgeons and patients is challenging. Objective: (1) To develop a computer tool that allows individual reconstruction and simulation of 3-dimensional (3D) soft tissue from ordinary digital photos of breasts, (2) to implement a Web-based, worldwide-accessible preoperative surgical planning platform for plastic surgeons, and (3) to validate this tool through a quality control analysis by comparing 3D laser scans of the patients with the 3D reconstructions with this tool from original 2-dimensional (2D) pictures of the same patients. Methods: The proposed system uses well-established 2D digital photos for reconstruction into a 3D torso, which is then available to the user for interactive planning. The simulation is performed on dedicated servers, accessible via Internet. It allows the surgeon, together with the patient, to previsualize the impact of the proposed breast augmentation directly during the consultation before a surgery is decided upon. We retrospectively conduced a quality control assessment of available anonymized pre- and postoperative 2D digital photographs of patients undergoing breast augmentation procedures. The method presented above was used to reconstruct 3D pictures from 2D digital pictures. We used a laser scanner capable of generating a highly accurate surface model of the patient’s anatomy to acquire ground truth data. The quality of the computed 3D reconstructions was compared with the ground truth data used to perform both qualitative and quantitative evaluations. Results: We evaluated the system on 11 clinical cases for surface reconstructions and 4 clinical cases of postoperative simulations, using laser surface scan technologies showing a mean reconstruction error between 2 and 4 mm and a maximum outlier error of 16 mm. Qualitative and quantitative analyses from plastic surgeons demonstrate the potential of these new emerging technologies. Conclusions: We tested our tool for 3D, Web-based, patient-specific consultation in the clinical scenario of breast augmentation. This example shows that the current state of development allows for creation of responsive and effective Web-based, 3D medical tools, even with highly complex and time-consuming computation, by off-loading them to a dedicated high-performance data center. The efficient combination of advanced technologies, based on analysis and understanding of human anatomy and physiology, will allow the development of further Web-based reconstruction and predictive interfaces at different scales of the human body. The consultation tool presented herein exemplifies the potential of combining advancements in the core areas of computer science and biomedical engineering with the evolving areas of Web technologies. We are confident that future developments based on a multidisciplinary approach will further pave the way toward personalized Web-enabled medicine. %M 22306688 %R 10.2196/jmir.1903 %U http://www.jmir.org/2012/1/e21/ %U https://doi.org/10.2196/jmir.1903 %U http://www.ncbi.nlm.nih.gov/pubmed/22306688 %0 Journal Article %@ 1438-8871 %I Gunther Eysenbach %V 13 %N 3 %P e57 %T A Comparison of Physician Pre-Adoption and Adoption Views on Electronic Health Records in Canadian Medical Practices %A Archer,Norm %A Cocosila,Mihail %+ DeGroote School of Business, McMaster University, 1280 Main St. West, Hamilton, ON, L8S 4M4, Canada, 1 905 525 9140 ext 23944, archer@mcmaster.ca %K Electronic health record %K information technology %K medical practice %K Canada %D 2011 %7 12.08.2011 %9 Original Paper %J J Med Internet Res %G English %X Background: There is a major campaign involving large expenditures of public money to increase the adoption rate of electronic health record (EHR) systems in Canada. To maximize the chances of success in this effort, physician views on EHRs must be addressed, since user perceptions are key to successful implementation of technology innovations. Objective: We propose a theoretical model comprising behavioral factors either favoring or against EHR adoption and use in Canadian medical practices, from the physicians’ point of view. EHR perceptions of physicians already using EHR systems are compared with those not using one, through the lens of this model. Methods: We conducted an online cross-sectional survey in both English and French among medical practitioners across Canada. Data were collected both from physicians using EHRs and those not using EHRs, and analyzed with structural equation modeling (SEM) techniques. Results: We collected 119 responses from EHR users and 100 from nonusers, resulting in 2 valid samples of 102 and 83 participants, respectively. The theoretical adoption model explained 55.8% of the variance in behavioral intention to continue using EHRs for physicians already using them, and 66.8% of the variance in nonuser intention to adopt such systems. Perception of ease of use was found to be the strongest motivator for EHR users (total effect .525), while perceptions of usefulness and of ease of use were the key determinants for nonusers (total effect .538 and .519, respectively) to adopt the system. Users see perceived overall risk associated with EHR adoption as a major obstacle (total effect –.371), while nonusers perceive risk only as a weak indirect demotivator. Of the 13 paths of the SEM model, 5 showed significant differences between the 2 samples (at the .05 level): general doubts about using the system (P = .02), the necessity for the system to be relevant for their job (P < .001), and the necessity for the system to be useful (P = .049) are more important for EHR nonusers than for users, while perceptions of overall obstacles to adoption (P = .03) and system ease of use (P = .042) count more for EHR users than for nonusers. Conclusions: Relatively few differences in perceptions about EHR system adoption and use exist between physicians already using such systems and those not yet using the systems. To maximize the chances of success for new EHR implementations from a behavioral point of view, general doubts about the rationale for such systems must be mitigated through improving design, stressing how EHRs are relevant to physician jobs, and providing substantiating evidence that EHRs are easier to use and more effective than nonusers might expect. %M 21840835 %R 10.2196/jmir.1726 %U http://www.jmir.org/2011/3/e57/ %U https://doi.org/10.2196/jmir.1726 %U http://www.ncbi.nlm.nih.gov/pubmed/21840835 %0 Journal Article %@ 1438-8871 %I Gunther Eysenbach %V 13 %N 2 %P e40 %T Parent Satisfaction With the Electronic Medical Record in an Academic Pediatric Rheumatology Practice %A Rosen,Paul %A Spalding,Steven J %A Hannon,Michael J %A Boudreau,Robert M %A Kwoh,C Kent %+ Children's Hospital of Pittsburgh of UPMC, University of Pittsburgh School of Medicine, One Children's Hosptial Drive, 4401 Penn Avenue, Pittsburgh, PA, 15224, United States, 1 412 692 3294, Paul.Rosen@chp.edu %K Electronic medical record %K pediatric rheumatology %K ambulatory care %D 2011 %7 27.05.2011 %9 Original Paper %J J Med Internet Res %G English %X Background: Patient satisfaction has not been widely studied with respect to implementation of the electronic medical record (EMR). There are few reports of the impact of the EMR in pediatrics. Objective: The objective of this study was to assess the impact of implementation of an electronic medical record system on families in an academic pediatric rheumatology practice. Methods: Families were surveyed 1 month pre-EMR implementation and 3 months post-EMR implementation. Results: Overall, EMR was well received by families. Compared with the paper chart, parents agreed the EMR improved the quality of doctor care (55% or 59/107 vs 26% or 26/99, P < .001). More parents indicated they would prefer their pediatric physicians to use an EMR (68% or 73/107 vs 51% or 50/99, P = .01). Conclusions: Transitioning an academic pediatric rheumatology practice to an EMR can increase family satisfaction with the office visit. %M 21622292 %R 10.2196/jmir.1525 %U http://www.jmir.org/2011/2/e40/ %U https://doi.org/10.2196/jmir.1525 %U http://www.ncbi.nlm.nih.gov/pubmed/21622292