@Article{info:doi/10.2196/60148, author="Maru, Shoko and Kuwatsuru, Ryohei and Matthias, D. Michael and Simpson Jr, J. Ross", title="Public Disclosure of Results From Artificial Intelligence/Machine Learning Research in Health Care: Comprehensive Analysis of ClinicalTrials.gov, PubMed, and Scopus Data (2010-2023)", journal="J Med Internet Res", year="2025", month="Mar", day="21", volume="27", pages="e60148", keywords="machine learning", keywords="ML", keywords="artificial intelligence", keywords="AI", keywords="algorithm", keywords="model", keywords="analytics", keywords="deep learning", keywords="health care", keywords="health disparities", keywords="disparity", keywords="social disparity", keywords="social inequality", keywords="social inequity", keywords="data-source disparities", keywords="ClinicalTrials.gov", keywords="clinical trial", keywords="database", keywords="PubMed", keywords="Scopus", keywords="public disclosure of results", keywords="public disclosure", keywords="dissemination", abstract="Background: Despite the rapid growth of research in artificial intelligence/machine learning (AI/ML), little is known about how often study results are disclosed years after study completion. Objective: We aimed to estimate the proportion of AI/ML research that reported results through ClinicalTrials.gov or peer-reviewed publications indexed in PubMed or Scopus. Methods: Using data from the Clinical Trials Transformation Initiative Aggregate Analysis of ClinicalTrials.gov, we identified studies initiated and completed between January 2010 and December 2023 that contained AI/ML-specific terms in the official title, brief summary, interventions, conditions, detailed descriptions, primary outcomes, or keywords. For 842 completed studies, we searched PubMed and Scopus for publications containing study identifiers and AI/ML-specific terms in relevant fields, such as the title, abstract, and keywords. We calculated disclosure rates within 3 years of study completion and median times to disclosure---from the ``primary completion date'' to the ``results first posted date'' on ClinicalTrials.gov or the earliest date of journal publication. Results: When restricted to studies completed before 2021, ensuring at least 3 years of follow-up in which to report results, 7.0\% (22/316) disclosed results on ClinicalTrials.gov, 16.5\% (52/316) in journal publications, and 20.6\% (65/316) through either route within 3 years of completion. Higher disclosure rates were observed for trials: 11.0\% (15/136) on ClinicalTrials.gov, 25.0\% (34/136) in journal publications, and 30.1\% (41/136) through either route. Randomized controlled trials had even higher disclosure rates: 12.2\% (9/74) on ClinicalTrials.gov, 31.1\% (23/74) in journal publications, and 36.5\% (27/74) through either route. Nevertheless, most study findings (79.4\%; 251/316) remained undisclosed 3 years after study completion. Trials using randomization (vs nonrandomized) or masking (vs open label) had higher disclosure rates and shorter times to disclosure. Most trials (85\%; 305/357) had sample sizes of ?1000, yet larger trials (n>1000) had higher publication rates (30.8\%; 16/52) than smaller trials (n?1000) (17.4\%; 53/305). Hospitals (12.4\%; 42/340), academia (15.1\%; 39/259), and industry (13.7\%; 20/146) published the most. High-income countries accounted for 82.4\% (89/108) of all published studies. Of studies with disclosed results, the median times to report through ClinicalTrials.gov and in journal publications were 505 days (IQR 399-676) and 407 days (IQR 257-674), respectively. Open-label trials were common (60\%; 214/357). Single-center designs were prevalent in both trials (83.3\%; 290/348) and observational studies (82.3\%; 377/458). Conclusions: For nearly 80\% of completed studies, findings remained undisclosed within the 3 years of follow-up, raising questions about the representativeness of publicly available evidence. While methodological rigor was generally associated with higher publication rates, the predominance of single-center designs and high-income countries may limit the generalizability of the results currently accessible. ", doi="10.2196/60148", url="https://www.jmir.org/2025/1/e60148" } @Article{info:doi/10.2196/62750, author="Wallraf, Simon and K{\"o}themann, Sara and Wiesemann, Claudia and W{\"o}hlke, Sabine and Dierks, Marie-Luise and Schmidt, Andrea Marion and van Gils-Schmidt, Jasper Henk and Lander, Jonas", title="Digital Transformation in Patient Organizations: Interview and Focus Group Study", journal="J Med Internet Res", year="2025", month="Feb", day="13", volume="27", pages="e62750", keywords="patient organization", keywords="patient support", keywords="digitalization", keywords="digital transformation", keywords="health research", abstract="Background: Patient organizations (POs) are an integral part of the health care landscape, serving as advocates and support systems for patients and their families. As the digitalization of health care accelerates, POs are challenged to adapt their diverse roles to digital formats. However, the extent and form of POs' digital adaptation and the challenges POs encounter in their digital transformation remain unexplored. Objective: This study aims to investigate the digital transformation processes within POs. We examined the types of digital activities and processes implemented, people involved in respective tasks, challenges encountered, and attitudes toward the digitalization of POs. Methods: The study was carried out by the multicenter interdisciplinary research network Pandora. We adopted a qualitative exploratory approach by conducting 37 semistructured interviews and 2 focus groups with representatives and members of POs in Germany. Results were obtained using a deductive-inductive approach based on a qualitative content analysis. Methods and results were reported in accordance with the COREQ (Consolidated Criteria for Reporting Qualitative Research) checklist. Results: POs primarily apply basic digital tools to engage in communication, health education, and information dissemination. Some also develop specific mobile apps and collect health data through patient registries. Volunteers cover a considerable part of the workload. Sometimes, POs collaborate with external partners, such as health professionals or other nonprofit organizations. Furthermore, many (13/46, 28\%) interviewees referred to the importance of involving members in digitalization efforts to better meet their needs. However, they described the actual practices used to involve members in, for example, developing digital services as limited, passive, or implicit. When evaluating digital transformation processes, representatives and members of POs expressed generally positive attitudes and acknowledged their potential to improve the accessibility of support services, management efficiency, and outreach. Still, resource constraints; the complexity of digital initiatives; and accessibility issues for certain demographic groups, especially older persons, were frequently mentioned as challenges. Several (15/46, 33\%) interviewees highlighted POs' increasing responsibility to support their members' digital competencies and digital health literacy. Conclusions: POs are actively involved in the digital transformation of health services. To navigate challenges and further shape and sustain digital activities and processes, POs may benefit from governance frameworks, that is, a clear plan outlining with whom, how, and with what objectives digital projects are being realized. Support from public, scientific, and policy institutions to enhance the process through training, mentorship, and fostering collaborative networks seems warranted. ", doi="10.2196/62750", url="https://www.jmir.org/2025/1/e62750", url="http://www.ncbi.nlm.nih.gov/pubmed/39946181" } @Article{info:doi/10.2196/64069, author="Liu, Yingxin and Zhang, Jingyi and Thabane, Lehana and Bai, Xuerui and Kang, Lili and Lip, H. Gregory Y. and Van Spall, C. Harriette G. and Xia, Min and Li, Guowei", title="Data-Sharing Statements Requested from Clinical Trials by Public, Environmental, and Occupational Health Journals: Cross-Sectional Study", journal="J Med Internet Res", year="2025", month="Feb", day="7", volume="27", pages="e64069", keywords="data sharing", keywords="clinical trial", keywords="public health", keywords="International Committee of Medical Journal Editors", keywords="ICMJE", keywords="journal request", keywords="clinical trials", keywords="decision-making", keywords="occupational health", keywords="health informatics", keywords="patient data", abstract="Background: Data sharing plays a crucial role in health informatics, contributing to improving health information systems, enhancing operational efficiency, informing policy and decision-making, and advancing public health surveillance including disease tracking. Sharing individual participant data in public, environmental, and occupational health trials can help improve public trust and support by enhancing transparent reporting and reproducibility of research findings. The International Committee of Medical Journal Editors (ICMJE) requires all papers to include a data-sharing statement. However, it is unclear whether journals in the field of public, environmental, and occupational health adhere to this requirement. Objective: This study aims to investigate whether public, environmental, and occupational health journals requested data-sharing statements from clinical trials submitted for publication. Methods: In this bibliometric survey of ``Public, Environmental, and Occupational Health'' journals, defined by the Journal Citation Reports (as of June 2023), we included 202 journals with clinical trial reports published between 2019 and 2022. The primary outcome was a journal request for a data-sharing statement, as identified in the paper submission instructions. Multivariable logistic regression analysis was conducted to evaluate the relationship between journal characteristics and journal requests for data-sharing statements, with results presented as odds ratios (ORs) and corresponding 95\% CIs. We also investigated whether the journals included a data-sharing statement in their published trial reports. Results: Among the 202 public, environmental, and occupational health journals included, there were 68 (33.7\%) journals that did not request data-sharing statements. Factors significantly associated with journal requests for data-sharing statements included open access status (OR 0.43, 95\% CI 0.19-0.97), high journal impact factor (OR 2.31, 95\% CI 1.15-4.78), endorsement of Consolidated Standards of Reporting Trials (OR 2.43, 95\% CI 1.25-4.79), and publication in the United Kingdom (OR 7.18, 95\% CI 2.61-23.4). Among the 134 journals requesting data-sharing statements, 26.9\% (36/134) did not have statements in their published trial reports. Conclusions: Over one-third of the public, environmental, and occupational health journals did not request data-sharing statements in clinical trial reports. Among those journals that requested data-sharing statements in their submission guidance pages, more than one quarter published trial reports with no data-sharing statements. These results revealed an inadequate practice of requesting data-sharing statements by public, environmental, and occupational health journals, requiring more effort at the journal level to implement ICJME recommendations on data-sharing statements. ", doi="10.2196/64069", url="https://www.jmir.org/2025/1/e64069" } @Article{info:doi/10.2196/51955, author="Song, Shanshan and Ashton, Micaela and Yoo, Hahn Rebecca and Lkhagvajav, Zoljargal and Wright, Robert and Mathews, H. Debra J. and Taylor, Overby Casey", title="Participant Contributions to Person-Generated Health Data Research Using Mobile Devices: Scoping Review", journal="J Med Internet Res", year="2025", month="Jan", day="20", volume="27", pages="e51955", keywords="scoping review", keywords="person-generated health data", keywords="PGHD", keywords="mHealth", keywords="mobile device", keywords="smartphone", keywords="mobile phone", keywords="wearable", keywords="fitness tracker", keywords="smartwatch", keywords="BYOD", keywords="crowdsourcing", keywords="reporting deficiency", abstract="Background: Mobile devices offer an emerging opportunity for research participants to contribute person-generated health data (PGHD). There is little guidance, however, on how to best report findings from studies leveraging those data. Thus, there is a need to characterize current reporting practices so as to better understand the potential implications for producing reproducible results. Objective: The primary objective of this scoping review was to characterize publications' reporting practices for research that collects PGHD using mobile devices. Methods: We comprehensively searched PubMed and screened the results. Qualifying publications were classified according to 6 dimensions---1 covering key bibliographic details (for all articles) and 5 covering reporting criteria considered necessary for reproducible and responsible research (ie, ``participant,'' ``data,'' ``device,'' ``study,'' and ``ethics,'' for original research). For each of the 5 reporting dimensions, we also assessed reporting completeness. Results: Out of 3602 publications screened, 100 were included in this review. We observed a rapid increase in all publications from 2016 to 2021, with the largest contribution from US authors, with 1 exception, review articles. Few original research publications used crowdsourcing platforms (7\%, 3/45). Among the original research publications that reported device ownership, most (75\%, 21/28) reported using participant-owned devices for data collection (ie, a Bring-Your-Own-Device [BYOD] strategy). A significant deficiency in reporting completeness was observed for the ``data'' and ``ethics'' dimensions (5 reporting factors were missing in over half of the research publications). Reporting completeness for data ownership and participants' access to data after contribution worsened over time. Conclusions: Our work depicts the reporting practices in publications about research involving PGHD from mobile devices. We found that very few papers reported crowdsourcing platforms for data collection. BYOD strategies are increasingly popular; this creates an opportunity for improved mechanisms to transfer data from device owners to researchers on crowdsourcing platforms. Given substantial reporting deficiencies, we recommend reaching a consensus on best practices for research collecting PGHD from mobile devices. Drawing from the 5 reporting dimensions in this scoping review, we share our recommendations and justifications for 9 items. These items require improved reporting to enhance data representativeness and quality and empower participants. ", doi="10.2196/51955", url="https://www.jmir.org/2025/1/e51955" } @Article{info:doi/10.2196/63875, author="Dorosan, Michael and Chen, Ya-Lin and Zhuang, Qingyuan and Lam, Sean Shao Wei", title="In Silico Evaluation of Algorithm-Based Clinical Decision Support Systems: Protocol for a Scoping Review", journal="JMIR Res Protoc", year="2025", month="Jan", day="16", volume="14", pages="e63875", keywords="clinical decision support algorithms", keywords="in silico evaluation", keywords="clinical workflow simulation", keywords="health care modeling", keywords="digital twin", keywords="quadruple aims", keywords="clinical decision", keywords="decision-making", keywords="decision support", keywords="workflow", keywords="support system", keywords="protocol", keywords="scoping review", keywords="algorithm-based", keywords="screening", keywords="thematic analysis", keywords="descriptive analysis", keywords="clinical decision-making", abstract="Background: Integrating algorithm-based clinical decision support (CDS) systems poses significant challenges in evaluating their actual clinical value. Such CDS systems are traditionally assessed via controlled but resource-intensive clinical trials. Objective: This paper presents a review protocol for preimplementation in silico evaluation methods to enable broadened impact analysis under simulated environments before clinical trials. Methods: We propose a scoping review protocol that follows an enhanced Arksey and O'Malley framework and PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines to investigate the scope and research gaps in the in silico evaluation of algorithm-based CDS models---specifically CDS decision-making end points and objectives, evaluation metrics used, and simulation paradigms used to assess potential impacts. The databases searched are PubMed, Embase, CINAHL, PsycINFO, Cochrane, IEEEXplore, Web of Science, and arXiv. A 2-stage screening process identified pertinent articles. The information extracted from articles was iteratively refined. The review will use thematic, trend, and descriptive analyses to meet scoping aims. Results: We conducted an automated search of the databases above in May 2023, with most title and abstract screenings completed by November 2023 and full-text screening extended from December 2023 to May 2024. Concurrent charting and full-text analysis were carried out, with the final analysis and manuscript preparation set for completion in July 2024. Publication of the review results is targeted from July 2024 to February 2025. As of April 2024, a total of 21 articles have been selected following a 2-stage screening process; these will proceed to data extraction and analysis. Conclusions: We refined our data extraction strategy through a collaborative, multidisciplinary approach, planning to analyze results using thematic analyses to identify approaches to in silico evaluation. Anticipated findings aim to contribute to developing a unified in silico evaluation framework adaptable to various clinical workflows, detailing clinical decision-making characteristics, impact measures, and reusability of methods. The study's findings will be published and presented in forums combining artificial intelligence and machine learning, clinical decision-making, and health technology impact analysis. Ultimately, we aim to bridge the development-deployment gap through in silico evaluation-based potential impact assessments. International Registered Report Identifier (IRRID): DERR1-10.2196/63875 ", doi="10.2196/63875", url="https://www.researchprotocols.org/2025/1/e63875" } @Article{info:doi/10.2196/59598, author="Jiang, Yuyan and Liu, Xue-li and Wang, Liyun", title="Evaluation and Comparison of the Academic Quality of Open-Access Mega Journals and Authoritative Journals: Disruptive Innovation Evaluation", journal="J Med Internet Res", year="2025", month="Jan", day="15", volume="27", pages="e59598", keywords="innovative evaluation", keywords="disruption index", keywords="open-access mega journals", keywords="paper evaluation", keywords="open citation data", abstract="Background: Some scholars who are skeptical about open-access mega journals (OAMJs) have argued that low-quality papers are often difficult to publish in more prestigious and authoritative journals, and OAMJs may be their main destination. Objective: This study aims to evaluate the academic quality of OAMJs and highlight their important role in clinical medicine. To achieve this aim, authoritative journals and representative OAMJs in this field were selected as research objects. The differences between the two were compared and analyzed in terms of their level of disruptive innovation. Additionally, this paper explored the countries and research directions for which OAMJs serve as publication channels for disruptive innovations. Methods: In this study, the journal information, literature data, and open citation relationship data were sourced from Journal Citation Reports (JCR), Web of Science (WoS), InCites, and the OpenCitations Index of PubMed Open PMID-to-PMID citations (POCI). Then, we calculated the disruptive innovation level of the focus paper based on the local POCI database. Results: The mean Journal Disruption Index (JDI) values for the selected authoritative journals and OAMJs were 0.5866 (SD 0.26933) and 0.0255 (SD 0.01689), respectively, showing a significant difference. Only 1.48\% (861/58,181) of the OAMJ papers reached the median level of disruptive innovation of authoritative journal papers (MDAJ). However, the absolute number was roughly equal to that of authoritative journals. OAMJs surpassed authoritative journals in publishing innovative papers in 24 research directions (eg, Allergy), accounting for 40.68\% of all research directions in clinical medicine. Among research topics with at least 10 authoritative papers, OAMJs matched or exceeded MDAJ in 35.71\% of cases. The number of papers published in authoritative journals and the average level of disruptive innovation in each country showed a linear relationship after logarithmic treatment, with a correlation coefficient of --0.891 (P<.001). However, the number of papers published in OAMJs in each country and the average level of disruptive innovation did not show a linear relationship after logarithmic treatment. Conclusions: While the average disruptive innovation level of papers published by OAMJs is significantly lower than that of authoritative journals, OAMJs have become an important publication channel for innovative research in various research directions. They also provide fairer opportunities for the publication of innovative results from limited-income countries. Therefore, the academic community should recognize the contribution and value of OAMJs to advancing scientific research. ", doi="10.2196/59598", url="https://www.jmir.org/2025/1/e59598" } @Article{info:doi/10.2196/50235, author="Jefferson, Emily and Milligan, Gordon and Johnston, Jenny and Mumtaz, Shahzad and Cole, Christian and Best, Joseph and Giles, Charles Thomas and Cox, Samuel and Masood, Erum and Horban, Scott and Urwin, Esmond and Beggs, Jillian and Chuter, Antony and Reilly, Gerry and Morris, Andrew and Seymour, David and Hopkins, Susan and Sheikh, Aziz and Quinlan, Philip", title="The Challenges and Lessons Learned Building a New UK Infrastructure for Finding and Accessing Population-Wide COVID-19 Data for Research and Public Health Analysis: The CO-CONNECT Project", journal="J Med Internet Res", year="2024", month="Nov", day="20", volume="26", pages="e50235", keywords="COVID-19", keywords="infrastructure", keywords="trusted research environments", keywords="safe havens", keywords="feasibility analysis", keywords="cohort discovery", keywords="federated analytics", keywords="federated discovery", keywords="lessons learned", keywords="population wide", keywords="data", keywords="public health", keywords="analysis", keywords="CO-CONNECT", keywords="challenges", keywords="data transformation", doi="10.2196/50235", url="https://www.jmir.org/2024/1/e50235" } @Article{info:doi/10.2196/60057, author="Kaczmarczyk, Robert and Wilhelm, Isabelle Theresa and Roos, Jonas and Martin, Ron", title="Decoding the Digital Pulse: Bibliometric Analysis of 25 Years in Digital Health Research Through the Journal of Medical Internet Research", journal="J Med Internet Res", year="2024", month="Nov", day="15", volume="26", pages="e60057", keywords="digital health", keywords="JMIR publication analysis", keywords="network analysis", keywords="artificial intelligence", keywords="AI", keywords="large language models", keywords="eHealth", keywords="Claude 3 Opus", keywords="digital", keywords="digital technology", keywords="digital intervention", keywords="machine learning", keywords="natural language processing", keywords="NLP", keywords="deep learning", keywords="algorithm", keywords="model", keywords="analytics", keywords="practical model", keywords="pandemic", keywords="postpandemic era", keywords="mobile phone", abstract="Background: As the digital health landscape continues to evolve, analyzing the progress and direction of the field can yield valuable insights. The Journal of Medical Internet Research (JMIR) has been at the forefront of disseminating digital health research since 1999. A comprehensive network analysis of JMIR publications can help illuminate the evolution and trends in digital medicine over the past 25 years. Objective: This study aims to conduct a detailed network analysis of JMIR's publications to uncover the growth patterns, dominant themes, and potential future trajectories in digital health research. Methods: We retrieved 8068 JMIR papers from PubMed using the Biopython library. Keyword metrics were assessed using accuracy, recall, and F1-scores to evaluate the effectiveness of keyword identification from Claude 3 Opus and Gemini 1.5 Pro in addition to 2 conventional natural language processing methods using key bidirectional encoder representations from transformers. Future trends for 2024-2026 were predicted using Claude 3 Opus, Google's Time Series Foundation Model, autoregressive integrated moving average, exponential smoothing, and Prophet. Network visualization techniques were used to represent and analyze the complex relationships between collaborating countries, paper types, and keyword co-occurrence. Results: JMIR's publication volume showed consistent growth, with a peak in 2020. The United States dominated country contributions, with China showing a notable increase in recent years. Keyword analysis from 1999 to 2023 showed significant thematic shifts, from an early internet and digital health focus to the dominance of COVID-19 and advanced technologies such as machine learning. Predictions for 2024-2026 suggest an increased focus on artificial intelligence, digital health, and mental health. Conclusions: Network analysis of JMIR publications provides a macroscopic view of the evolution of the digital health field. The journal's trajectory reflects broader technological advances and shifting research priorities, including the impact of the COVID-19 pandemic. The predicted trends underscore the growing importance of computational technology in future health care research and practice. The findings from JMIR provide a glimpse into the future of digital medicine, suggesting a robust integration of artificial intelligence and continued emphasis on mental health in the postpandemic era. ", doi="10.2196/60057", url="https://www.jmir.org/2024/1/e60057" } @Article{info:doi/10.2196/58130, author="Penev, P. Yordan and Buchanan, R. Timothy and Ruppert, M. Matthew and Liu, Michelle and Shekouhi, Ramin and Guan, Ziyuan and Balch, Jeremy and Ozrazgat-Baslanti, Tezcan and Shickel, Benjamin and Loftus, J. Tyler and Bihorac, Azra", title="Electronic Health Record Data Quality and Performance Assessments: Scoping Review", journal="JMIR Med Inform", year="2024", month="Nov", day="6", volume="12", pages="e58130", keywords="electronic health record", keywords="EHR", keywords="record", keywords="data quality", keywords="data performance", keywords="clinical informatics", keywords="performance", keywords="data science", keywords="synthesis", keywords="review methods", keywords="review methodology", keywords="search", keywords="scoping", abstract="Background: Electronic health records (EHRs) have an enormous potential to advance medical research and practice through easily accessible and interpretable EHR-derived databases. Attainability of this potential is limited by issues with data quality (DQ) and performance assessment. Objective: This review aims to streamline the current best practices on EHR DQ and performance assessments as a replicable standard for researchers in the field. Methods: PubMed was systematically searched for original research articles assessing EHR DQ and performance from inception until May 7, 2023. Results: Our search yielded 26 original research articles. Most articles had 1 or more significant limitations, including incomplete or inconsistent reporting (n=6, 30\%), poor replicability (n=5, 25\%), and limited generalizability of results (n=5, 25\%). Completeness (n=21, 81\%), conformance (n=18, 69\%), and plausibility (n=16, 62\%) were the most cited indicators of DQ, while correctness or accuracy (n=14, 54\%) was most cited for data performance, with context-specific supplementation by recency (n=7, 27\%), fairness (n=6, 23\%), stability (n=4, 15\%), and shareability (n=2, 8\%) assessments. Artificial intelligence--based techniques, including natural language data extraction, data imputation, and fairness algorithms, were demonstrated to play a rising role in improving both dataset quality and performance. Conclusions: This review highlights the need for incentivizing DQ and performance assessments and their standardization. The results suggest the usefulness of artificial intelligence--based techniques for enhancing DQ and performance to unlock the full potential of EHRs to improve medical research and practice. ", doi="10.2196/58130", url="https://medinform.jmir.org/2024/1/e58130" } @Article{info:doi/10.2196/60025, author="Raman, Raghu and Singhania, Monica and Nedungadi, Prema", title="Advancing the United Nations Sustainable Development Goals Through Digital Health Research: 25 Years of Contributions From the Journal of Medical Internet Research", journal="J Med Internet Res", year="2024", month="Nov", day="4", volume="26", pages="e60025", keywords="sustainable development goal", keywords="topic modeling", keywords="public health", keywords="surveillance", keywords="gender equality", keywords="non-communicable disease", keywords="social media", keywords="COVID-19", keywords="SARS-CoV-2", keywords="coronavirus", keywords="machine learning", keywords="artificial intelligence", keywords="AI", keywords="digital health", doi="10.2196/60025", url="https://www.jmir.org/2024/1/e60025" } @Article{info:doi/10.2196/58987, author="Hu, Jing and Li, Chong and Ge, Yanlei and Yang, Jingyi and Zhu, Siyi and He, Chengqi", title="Mapping the Evolution of Digital Health Research: Bibliometric Overview of Research Hotspots, Trends, and Collaboration of Publications in JMIR (1999-2024)", journal="J Med Internet Res", year="2024", month="Oct", day="17", volume="26", pages="e58987", keywords="JMIR", keywords="bibliometric analysis", keywords="ehealth", keywords="digital health", keywords="medical informatics", keywords="health informatics", keywords="open science", keywords="publishing", abstract="Background: While bibliometric studies of individual journals have been conducted, to the best of our knowledge, bibliometric mapping has not yet been utilized to analyze the literature published by the Journal of Medical Internet Research (JMIR). Objective: In celebration of the journal's 25th anniversary, this study aimed to review the entire collection of JMIR publications from 1999 to 2024 and provide a comprehensive overview of the main publication characteristics. Methods: This study included papers published in JMIR during the 25-year period from 1999 to 2024. The data were analyzed using CiteSpace, VOSviewer, and the ``Bibliometrix'' package in R. Through descriptive bibliometrics, we examined the dynamics and trend patterns of JMIR literature production and identified the most prolific authors, papers, institutions, and countries. Bibliometric maps were used to visualize the content of published articles and to identify the most prominent research terms and topics, along with their evolution. A bibliometric network map was constructed to determine the hot research topics over the past 25 years. Results: This study revealed positive trends in literature production, with both the total number of publications and the average number of citations increasing over the years. And the global COVID-19 pandemic induced an explosive rise in the number of publications in JMIR. The most productive institutions were predominantly from the United States, which ranked highest in successful publications within the journal. The editor-in-chief of JMIR was identified as a pioneer in this field. The thematic analysis indicated that the most prolific topics aligned with the primary aims and scope of the journal. Currently and in the foreseeable future, the main themes of JMIR include ``artificial intelligence,'' ``patient empowerment,'' and ``victimization.'' Conclusions: This bibliometric study highlighted significant contributions to digital health by identifying key research trends, themes, influential authors, and collaborations. The findings underscore the necessity to enhance publications from developing countries, improve gender diversity among authors, and expand the range of research topics explored in the journal. ", doi="10.2196/58987", url="https://www.jmir.org/2024/1/e58987" } @Article{info:doi/10.2196/57148, author="Mesk{\'o}, Bertalan and Krist{\'o}f, Tam{\'a}s and Dhunnoo, Pranavsingh and {\'A}rvai, N{\'o}ra and Katonai, Gell{\'e}rt", title="Exploring the Need for Medical Futures Studies: Insights From a Scoping Review of Health Care Foresight", journal="J Med Internet Res", year="2024", month="Oct", day="9", volume="26", pages="e57148", keywords="foresight", keywords="futures studies", keywords="health care future", keywords="medical futures", keywords="technology foresight", abstract="Background: The historical development and contemporary instances of futures studies, an interdisciplinary field that focuses on exploring and formulating alternative futures, exemplify the increasing significance of using futures methods in shaping the health care domain. Despite the wide array of these methodologies, there have been limited endeavors to employ them within the medical community thus far. Objective: We undertook the first scoping review to date about the application of futures methodologies and published foresight projects in health care. Methods: Through the use of the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) method, we identified 59 studies that were subsequently categorized into the following 5 distinct themes: national strategies (n=19), strategic health care foresight (n=15), health care policy and workforce dynamics (n=6), pandemic preparedness and response (n=7), and specialized medical domains (n=12). Results: Our scoping review revealed that the application of futures methods and foresight has been successfully demonstrated in a wide range of fields, including national strategies, policy formulation, global threat preparedness, and technological advancements. The results of our review indicate that a total of 8 futures methods have already been used in medicine and health care, while there are more than 50 futures methods available. It may underscore the notion that the field is unexploited. Furthermore, the absence of structured methodologies and principles for employing foresight and futures techniques in the health care domain warrants the creation of medical futures studies as a separate scientific subfield within the broad domains of health care, medicine, and life sciences. This subfield would focus on the analysis of emerging technological trends, the evaluation of policy implications, and the proactive anticipation and mitigation of potential challenges. Conclusions: Futures studies can significantly enhance medical science by addressing a crucial deficiency in the promotion of democratic participation, facilitating interdisciplinary dialogue, and shaping alternative futures. To further contribute to the development of a new research community in medical futures studies, it is recommended to establish a specialized scientific journal. Additionally, appointing dedicated futurists in decision-making and national strategy, and incorporating futures methods into the medical curriculum could be beneficial. ", doi="10.2196/57148", url="https://www.jmir.org/2024/1/e57148" } @Article{info:doi/10.2196/46800, author="Chow, Chi Julie and Cheng, Yun Teng and Chien, Tsair-Wei and Chou, Willy", title="Assessing ChatGPT's Capability for Multiple Choice Questions Using RaschOnline: Observational Study", journal="JMIR Form Res", year="2024", month="Aug", day="8", volume="8", pages="e46800", keywords="RaschOnline", keywords="ChatGPT", keywords="multiple choice questions", keywords="differential item functioning", keywords="Wright map", keywords="KIDMAP", keywords="website tool", keywords="evaluation tool", keywords="tool", keywords="application", keywords="artificial intelligence", keywords="scoring", keywords="testing", keywords="college", keywords="students", abstract="Background: ChatGPT (OpenAI), a state-of-the-art large language model, has exhibited remarkable performance in various specialized applications. Despite the growing popularity and efficacy of artificial intelligence, there is a scarcity of studies that assess ChatGPT's competence in addressing multiple-choice questions (MCQs) using KIDMAP of Rasch analysis---a website tool used to evaluate ChatGPT's performance in MCQ answering. Objective: This study aims to (1) showcase the utility of the website (Rasch analysis, specifically RaschOnline), and (2) determine the grade achieved by ChatGPT when compared to a normal sample. Methods: The capability of ChatGPT was evaluated using 10 items from the English tests conducted for Taiwan college entrance examinations in 2023. Under a Rasch model, 300 simulated students with normal distributions were simulated to compete with ChatGPT's responses. RaschOnline was used to generate 5 visual presentations, including item difficulties, differential item functioning, item characteristic curve, Wright map, and KIDMAP, to address the research objectives. Results: The findings revealed the following: (1) the difficulty of the 10 items increased in a monotonous pattern from easier to harder, represented by logits (--2.43, --1.78, --1.48, --0.64, --0.1, 0.33, 0.59, 1.34, 1.7, and 2.47); (2) evidence of differential item functioning was observed between gender groups for item 5 (P=.04); (3) item 5 displayed a good fit to the Rasch model (P=.61); (4) all items demonstrated a satisfactory fit to the Rasch model, indicated by Infit mean square errors below the threshold of 1.5; (5) no significant difference was found in the measures obtained between gender groups (P=.83); (6) a significant difference was observed among ability grades (P<.001); and (7) ChatGPT's capability was graded as A, surpassing grades B to E. Conclusions: By using RaschOnline, this study provides evidence that ChatGPT possesses the ability to achieve a grade A when compared to a normal sample. It exhibits excellent proficiency in answering MCQs from the English tests conducted in 2023 for the Taiwan college entrance examinations. ", doi="10.2196/46800", url="https://formative.jmir.org/2024/1/e46800", url="http://www.ncbi.nlm.nih.gov/pubmed/39115919" } @Article{info:doi/10.2196/45242, author="Bostan, Sarah and Johnson, A. Owen and Jaspersen, J. Lena and Randell, Rebecca", title="Contextual Barriers to Implementing Open-Source Electronic Health Record Systems for Low- and Lower-Middle-Income Countries: Scoping Review", journal="J Med Internet Res", year="2024", month="Aug", day="1", volume="26", pages="e45242", keywords="implementation", keywords="open source", keywords="electronic health records", keywords="digital health", keywords="low- and lower-middle-income countries", keywords="barriers", keywords="global health care", keywords="scoping", keywords="review", abstract="Background: Low- and lower-middle-income countries account for a higher percentage of global epidemics and chronic diseases. In most low- and lower-middle-income countries, there is limited access to health care. The implementation of open-source electronic health records (EHRs) can be understood as a powerful enabler for low- and lower-middle-income countries because it can transform the way health care technology is delivered. Open-source EHRs can enhance health care delivery in low- and lower-middle-income countries by improving the collection, management, and analysis of health data needed to inform health care delivery, policy, and planning. While open-source EHR systems are cost-effective and adaptable, they have not proliferated rapidly in low- and lower-middle-income countries. Implementation barriers slow adoption, with existing research focusing predominantly on technical issues preventing successful implementation. Objective: This interdisciplinary scoping review aims to provide an overview of contextual barriers affecting the adaptation and implementation of open-source EHR systems in low- and lower-middle-income countries and to identify areas for future research. Methods: We conducted a scoping literature review following a systematic methodological framework. A total of 7 databases were selected from 3 disciplines: medicine and health sciences, computing, and social sciences. The findings were reported in accordance with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist. The Mixed Methods Appraisal Tool and the Critical Appraisal Skills Programme checklists were used to assess the quality of relevant studies. Data were collated and summarized, and results were reported qualitatively, adopting a narrative synthesis approach. Results: This review included 13 studies that examined open-source EHRs' adaptation and implementation in low- and lower-middle-income countries from 3 interrelated perspectives: socioenvironmental, technological, and organizational barriers. The studies identified key issues such as limited funding, sustainability, organizational and management challenges, infrastructure, data privacy and protection, and ownership. Data protection, confidentiality, ownership, and ethics emerged as important issues, often overshadowed by technical processes. Conclusions: While open-source EHRs have the potential to enhance health care delivery in low- and lower-middle-income-country settings, implementation is fraught with difficulty. This scoping review shows that depending on the adopted perspective to implementation, different implementation barriers come into view. A dominant focus on technology distracts from socioenvironmental and organizational barriers impacting the proliferation of open-source EHRs. The role of local implementing organizations in addressing implementation barriers in low- and lower-middle-income countries remains unclear. A holistic understanding of implementers' experiences of implementation processes is needed. This could help characterize and solve implementation problems, including those related to ethics and the management of data protection. Nevertheless, this scoping review provides a meaningful contribution to the global health informatics discipline. ", doi="10.2196/45242", url="https://www.jmir.org/2024/1/e45242" } @Article{info:doi/10.2196/55496, author="Pirmani, Ashkan and Oldenhof, Martijn and Peeters, M. Liesbet and De Brouwer, Edward and Moreau, Yves", title="Accessible Ecosystem for Clinical Research (Federated Learning for Everyone): Development and Usability Study", journal="JMIR Form Res", year="2024", month="Jul", day="17", volume="8", pages="e55496", keywords="federated learning", keywords="multistakeholder collaboration", keywords="real-world data", keywords="integrity", keywords="reliability", keywords="clinical research", keywords="implementation", keywords="inclusivity", keywords="inclusive", keywords="accessible", keywords="ecosystem", keywords="design effectiveness", abstract="Background: The integrity and reliability of clinical research outcomes rely heavily on access to vast amounts of data. However, the fragmented distribution of these data across multiple institutions, along with ethical and regulatory barriers, presents significant challenges to accessing relevant data. While federated learning offers a promising solution to leverage insights from fragmented data sets, its adoption faces hurdles due to implementation complexities, scalability issues, and inclusivity challenges. Objective: This paper introduces Federated Learning for Everyone (FL4E), an accessible framework facilitating multistakeholder collaboration in clinical research. It focuses on simplifying federated learning through an innovative ecosystem-based approach. Methods: The ``degree of federation'' is a fundamental concept of FL4E, allowing for flexible integration of federated and centralized learning models. This feature provides a customizable solution by enabling users to choose the level of data decentralization based on specific health care settings or project needs, making federated learning more adaptable and efficient. By using an ecosystem-based collaborative learning strategy, FL4E encourages a comprehensive platform for managing real-world data, enhancing collaboration and knowledge sharing among its stakeholders. Results: Evaluating FL4E's effectiveness using real-world health care data sets has highlighted its ecosystem-oriented and inclusive design. By applying hybrid models to 2 distinct analytical tasks---classification and survival analysis---within real-world settings, we have effectively measured the ``degree of federation'' across various contexts. These evaluations show that FL4E's hybrid models not only match the performance of fully federated models but also avoid the substantial overhead usually linked with these models. Achieving this balance greatly enhances collaborative initiatives and broadens the scope of analytical possibilities within the ecosystem. Conclusions: FL4E represents a significant step forward in collaborative clinical research by merging the benefits of centralized and federated learning. Its modular ecosystem-based design and the ``degree of federation'' feature make it an inclusive, customizable framework suitable for a wide array of clinical research scenarios, promising to revolutionize the field through improved collaboration and data use. Detailed implementation and analyses are available on the associated GitHub repository. ", doi="10.2196/55496", url="https://formative.jmir.org/2024/1/e55496" } @Article{info:doi/10.2196/54265, author="Soares, Andrey and Schilling, M. Lisa and Richardson, Joshua and Kommadi, Bhagvan and Subbian, Vignesh and Dehnbostel, Joanne and Shahin, Khalid and Robinson, A. Karen and Afzal, Muhammad and Lehmann, P. Harold and Kunnamo, Ilkka and Alper, S. Brian", title="Making Science Computable Using Evidence-Based Medicine on Fast Healthcare Interoperability Resources: Standards Development Project", journal="J Med Internet Res", year="2024", month="Jun", day="25", volume="26", pages="e54265", keywords="evidence-based medicine", keywords="FHIR", keywords="Fast Healthcare Interoperability Resources", keywords="computable evidence", keywords="EBMonFHIR", keywords="evidence-based medicine on Fast Healthcare Interoperability Resources", abstract="Background: Evidence-based medicine (EBM) has the potential to improve health outcomes, but EBM has not been widely integrated into the systems used for research or clinical decision-making. There has not been a scalable and reusable computer-readable standard for distributing research results and synthesized evidence among creators, implementers, and the ultimate users of that evidence. Evidence that is more rapidly updated, synthesized, disseminated, and implemented would improve both the delivery of EBM and evidence-based health care policy. Objective: This study aimed to introduce the EBM on Fast Healthcare Interoperability Resources (FHIR) project (EBMonFHIR), which is extending the methods and infrastructure of Health Level Seven (HL7) FHIR to provide an interoperability standard for the electronic exchange of health-related scientific knowledge. Methods: As an ongoing process, the project creates and refines FHIR resources to represent evidence from clinical studies and syntheses of those studies and develops tools to assist with the creation and visualization of FHIR resources. Results: The EBMonFHIR project created FHIR resources (ie, ArtifactAssessment, Citation, Evidence, EvidenceReport, and EvidenceVariable) for representing evidence. The COVID-19 Knowledge Accelerator (COKA) project, now Health Evidence Knowledge Accelerator (HEvKA), took this work further and created FHIR resources that express EvidenceReport, Citation, and ArtifactAssessment concepts. The group is (1) continually refining FHIR resources to support the representation of EBM; (2) developing controlled terminology related to EBM (ie, study design, statistic type, statistical model, and risk of bias); and (3) developing tools to facilitate the visualization and data entry of EBM information into FHIR resources, including human-readable interfaces and JSON viewers. Conclusions: EBMonFHIR resources in conjunction with other FHIR resources can support relaying EBM components in a manner that is interoperable and consumable by downstream tools and health information technology systems to support the users of evidence. ", doi="10.2196/54265", url="https://www.jmir.org/2024/1/e54265", url="http://www.ncbi.nlm.nih.gov/pubmed/38916936" } @Article{info:doi/10.2196/50698, author="Stupnicki, Aleksander and Suresh, Basil and Jain, Saurabh", title="Online Visibility and Scientific Relevance of Strabismus Research: Bibliometric Analysis", journal="Interact J Med Res", year="2024", month="Jun", day="12", volume="13", pages="e50698", keywords="strabismus research", keywords="squint", keywords="social media", keywords="scientific relevance", keywords="altmetrics", keywords="accuracy", keywords="medical knowledge", keywords="metric", keywords="bibliometric analysis", keywords="research", keywords="strabismus", keywords="online visibility", keywords="platform", keywords="evidence-based information", keywords="accessibility", abstract="Background: Quality and accuracy of online scientific data are crucial, given that the internet and social media serve nowadays as primary sources of medical knowledge. Objective: This study aims to analyze the relationship between scientific relevance and online visibility of strabismus research to answer the following questions: (1) Are the most popular strabismus papers scientifically relevant? (2) Are the most high-impact strabismus studies shared enough online? Methods: The Altmetric Attention Score (AAS) was used as a proxy for online visibility, whereas citations and the journal's impact factor (IF) served as a metric for scientific relevance. Using ``strabismus'' as a keyword, 100 papers with the highest AAS and 100 papers with the highest number of citations were identified. Statistical analyses, including the Spearman rank test, linear regression, and factor analysis, were performed to assess the relationship between AAS, citations, a journal's IF, and mentions across 18 individual Web 2.0 platforms. Results: A weak, positive, statistically significant correlation was observed between normalized AAS and normalized citations (P<.001; r=0.27) for papers with high visibility. Only Twitter mentions and Mendeley readers correlated significantly with normalized citations (P=.02 and P<.001, respectively) and IF (P=.04 and P=.009, respectively), with Twitter being the strongest significant predictor of citation numbers (r=0.53). For high-impact papers, no correlation was found between normalized citations and normalized AAS (P=.12) or the IF of the journal (P=.55). Conclusions: While clinical relevance influences online attention, most high-impact research related to strabismus is not sufficiently shared on the web. Therefore, researchers should make a greater effort to share high-impact papers related to strabismus on online media platforms to improve accessibility and quality of evidence-based knowledge for patients. ", doi="10.2196/50698", url="https://www.i-jmr.org/2024/1/e50698", url="http://www.ncbi.nlm.nih.gov/pubmed/38865170" } @Article{info:doi/10.2196/53855, author="Burns, Kara and Bloom, Shoshana and Gilbert, Cecily and Merner, Bronwen and Kalla, Mahima and Sheri, Sreshta and Villanueva, Cleva and Matenga Ikihele, Amio and Nazer, Lama and Sarmiento, Francis Raymond and Stevens, Lindsay and Blow, Ngaree and Chapman, Wendy", title="Centering Digital Health Equity During Technology Innovation: Protocol for a Comprehensive Scoping Review of Evidence-Based Tools and Approaches", journal="JMIR Res Protoc", year="2024", month="Jun", day="5", volume="13", pages="e53855", keywords="digital health technology", keywords="eHealth", keywords="mHealth", keywords="health informatics", keywords="equity", keywords="inclusion", keywords="participatory design", keywords="universal design", keywords="Validitron", keywords="digital health", keywords="cost", keywords="technology", keywords="technology innovation", keywords="innovation", keywords="evidence-based tools", keywords="evidence-based", keywords="tools", keywords="digital innovation", keywords="cost-effective", keywords="accessibility", keywords="digital inequity", keywords="digital health equity", abstract="Background: In the rush to develop health technologies for the COVID-19 pandemic, the unintended consequence of digital health inequity or the inability of priority communities to access, use, and receive equal benefits from digital health technologies was not well examined. Objective: This scoping review will examine tools and approaches that can be used during digital technology innovation to improve equitable inclusion of priority communities in the development of digital health technologies. The results from this study will provide actionable insights for professionals in health care, health informatics, digital health, and technology development to proactively center equity during innovation. Methods: Based on the Arksey and O'Malley framework, this scoping review will consider priority communities' equitable involvement in digital technology innovation. Bibliographic databases in health, medicine, computing, and information sciences will be searched. Retrieved citations will be double screened against the inclusion and exclusion criteria using Covidence (Veritas Health Innovation). Data will be charted using a tailored extraction tool and mapped to a digital health innovation pathway defined by the Centre for eHealth Research roadmap for eHealth technologies. An accompanying narrative synthesis will describe the outcomes in relation to the review's objectives. Results: This scoping review is currently in progress. The search of databases and other sources returned a total of 4868 records. After the initial screening of titles and abstracts, 426 studies are undergoing dual full-text review. We are aiming to complete the full-text review stage by May 30, 2024, data extraction in October 2024, and subsequent synthesis in December 2024. Funding was received on October 1, 2023, from the Centre for Health Equity Incubator Grant Scheme, University of Melbourne, Australia. Conclusions: This paper will identify and recommend a series of validated tools and approaches that can be used by health care stakeholders and IT developers to produce equitable digital health technology across the Centre for eHealth Research roadmap. Identified evidence gaps, possible implications, and further research will be discussed. International Registered Report Identifier (IRRID): DERR1-10.2196/53855 ", doi="10.2196/53855", url="https://www.researchprotocols.org/2024/1/e53855", url="http://www.ncbi.nlm.nih.gov/pubmed/38838333" } @Article{info:doi/10.2196/55121, author="Jiang, Yuyan and Liu, Xue-li and Zhang, Zixuan and Yang, Xinru", title="Evaluation and Comparison of Academic Impact and Disruptive Innovation Level of Medical Journals: Bibliometric Analysis and Disruptive Evaluation", journal="J Med Internet Res", year="2024", month="May", day="31", volume="26", pages="e55121", keywords="medical journals", keywords="journal evaluation", keywords="innovative evaluation", keywords="journal disruption index", keywords="disruptive innovation", keywords="academic impact", keywords="peer review", abstract="Background: As an important platform for researchers to present their academic findings, medical journals have a close relationship between their evaluation orientation and the value orientation of their published research results. However, the differences between the academic impact and level of disruptive innovation of medical journals have not been examined by any study yet. Objective: This study aims to compare the relationships and differences between the academic impact, disruptive innovation levels, and peer review results of medical journals and published research papers. We also analyzed the similarities and differences in the impact evaluations, disruptive innovations, and peer reviews for different types of medical research papers and the underlying reasons. Methods: The general and internal medicine Science Citation Index Expanded (SCIE) journals in 2018 were chosen as the study object to explore the differences in the academic impact and level of disruptive innovation of medical journals based on the OpenCitations Index of PubMed open PMID-to-PMID citations (POCI) and H1Connect databases, respectively, and we compared them with the results of peer review. Results: First, the correlation coefficients of the Journal Disruption Index (JDI) with the Journal Cumulative Citation for 5 years (JCC5), Journal Impact Factor (JIF), and Journal Citation Indicator (JCI) were 0.677, 0.585, and 0.621, respectively. The correlation coefficient of the absolute disruption index (Dz) with the Cumulative Citation for 5 years (CC5) was 0.635. However, the average difference in the disruptive innovation and academic influence rankings of journals reached 20 places (about 17.5\%). The average difference in the disruptive innovation and influence rankings of research papers reached about 2700 places (about 17.7\%). The differences reflect the essential difference between the two evaluation systems. Second, the top 7 journals selected based on JDI, JCC5, JIF, and JCI were the same, and all of them were H-journals. Although 8 (8/15, 53\%), 96 (96/150, 64\%), and 880 (880/1500, 58.67\%) of the top 0.1\%, top 1\%, and top 10\% papers selected based on Dz and CC5, respectively, were the same. Third, research papers with the ``changes clinical practice'' tag showed only moderate innovation (4.96) and impact (241.67) levels but had high levels of peer-reviewed recognition (6.00) and attention (2.83). Conclusions: The results of the study show that research evaluation based on innovative indicators is detached from the traditional impact evaluation system. The 3 evaluation systems (impact evaluation, disruptive innovation evaluation, and peer review) only have high consistency for authoritative journals and top papers. Neither a single impact indicator nor an innovative indicator can directly reflect the impact of medical research for clinical practice. How to establish an integrated, comprehensive, scientific, and reasonable journal evaluation system to improve the existing evaluation system of medical journals still needs further research. ", doi="10.2196/55121", url="https://www.jmir.org/2024/1/e55121", url="http://www.ncbi.nlm.nih.gov/pubmed/38820583" } @Article{info:doi/10.2196/52655, author="Invernici, Francesco and Bernasconi, Anna and Ceri, Stefano", title="Searching COVID-19 Clinical Research Using Graph Queries: Algorithm Development and Validation", journal="J Med Internet Res", year="2024", month="May", day="30", volume="26", pages="e52655", keywords="big data corpus", keywords="clinical research", keywords="co-occurrence network", keywords="COVID-19 Open Research Dataset", keywords="CORD-19", keywords="graph search", keywords="Named Entity Recognition", keywords="Neo4j", keywords="text mining", abstract="Background: Since the beginning of the COVID-19 pandemic, >1 million studies have been collected within the COVID-19 Open Research Dataset, a corpus of manuscripts created to accelerate research against the disease. Their related abstracts hold a wealth of information that remains largely unexplored and difficult to search due to its unstructured nature. Keyword-based search is the standard approach, which allows users to retrieve the documents of a corpus that contain (all or some of) the words in a target list. This type of search, however, does not provide visual support to the task and is not suited to expressing complex queries or compensating for missing specifications. Objective: This study aims to consider small graphs of concepts and exploit them for expressing graph searches over existing COVID-19--related literature, leveraging the increasing use of graphs to represent and query scientific knowledge and providing a user-friendly search and exploration experience. Methods: We considered the COVID-19 Open Research Dataset corpus and summarized its content by annotating the publications' abstracts using terms selected from the Unified Medical Language System and the Ontology of Coronavirus Infectious Disease. Then, we built a co-occurrence network that includes all relevant concepts mentioned in the corpus, establishing connections when their mutual information is relevant. A sophisticated graph query engine was built to allow the identification of the best matches of graph queries on the network. It also supports partial matches and suggests potential query completions using shortest paths. Results: We built a large co-occurrence network, consisting of 128,249 entities and 47,198,965 relationships; the GRAPH-SEARCH interface allows users to explore the network by formulating or adapting graph queries; it produces a bibliography of publications, which are globally ranked; and each publication is further associated with the specific parts of the query that it explains, thereby allowing the user to understand each aspect of the matching. Conclusions: Our approach supports the process of query formulation and evidence search upon a large text corpus; it can be reapplied to any scientific domain where documents corpora and curated ontologies are made available. ", doi="10.2196/52655", url="https://www.jmir.org/2024/1/e52655", url="http://www.ncbi.nlm.nih.gov/pubmed/38814687" } @Article{info:doi/10.2196/53790, author="Sanchez, Jasmin and Trofholz, Amanda and Berge, M. Jerica", title="Best Practices and Recommendations for Research Using Virtual Real-Time Data Collection: Protocol for Virtual Data Collection Studies", journal="JMIR Res Protoc", year="2024", month="May", day="14", volume="13", pages="e53790", keywords="real-time data collection", keywords="remote research", keywords="virtual data collection", keywords="virtual research protocol", keywords="virtual research visits", abstract="Background: The COVID-19 pandemic and the subsequent need for social distancing required the immediate pivoting of research modalities. Research that had previously been conducted in person had to pivot to remote data collection. Researchers had to develop data collection protocols that could be conducted remotely with limited or no evidence to guide the process. Therefore, the use of web-based platforms to conduct real-time research visits surged despite the lack of evidence backing these novel approaches. Objective: This paper aims to review the remote or virtual research protocols that have been used in the past 10 years, gather existing best practices, and propose recommendations for continuing to use virtual real-time methods when appropriate. Methods: Articles (n=22) published from 2013 to June 2023 were reviewed and analyzed to understand how researchers conducted virtual research that implemented real-time protocols. ``Real-time'' was defined as data collection with a participant through a live medium where a participant and research staff could talk to each other back and forth in the moment. We excluded studies for the following reasons: (1) studies that collected participant or patient measures for the sole purpose of engaging in a clinical encounter; (2) studies that solely conducted qualitative interview data collection; (3) studies that conducted virtual data collection such as surveys or self-report measures that had no interaction with research staff; (4) studies that described research interventions but did not involve the collection of data through a web-based platform; (5) studies that were reviews or not original research; (6) studies that described research protocols and did not include actual data collection; and (7) studies that did not collect data in real time, focused on telehealth or telemedicine, and were exclusively intended for medical and not research purposes. Results: Findings from studies conducted both before and during the COVID-19 pandemic suggest that many types of data can be collected virtually in real time. Results and best practice recommendations from the current protocol review will be used in the design and implementation of a substudy to provide more evidence for virtual real-time data collection over the next year. Conclusions: Our findings suggest that virtual real-time visits are doable across a range of participant populations and can answer a range of research questions. Recommended best practices for virtual real-time data collection include (1) providing adequate equipment for real-time data collection, (2) creating protocols and materials for research staff to facilitate or guide participants through data collection, (3) piloting data collection, (4) iteratively accepting feedback, and (5) providing instructions in multiple forms. The implementation of these best practices and recommendations for future research are further discussed in the paper. International Registered Report Identifier (IRRID): DERR1-10.2196/53790 ", doi="10.2196/53790", url="https://www.researchprotocols.org/2024/1/e53790", url="http://www.ncbi.nlm.nih.gov/pubmed/38743477" } @Article{info:doi/10.2196/52508, author="El Emam, Khaled and Leung, I. Tiffany and Malin, Bradley and Klement, William and Eysenbach, Gunther", title="Consolidated Reporting Guidelines for Prognostic and Diagnostic Machine Learning Models (CREMLS)", journal="J Med Internet Res", year="2024", month="May", day="2", volume="26", pages="e52508", keywords="reporting guidelines", keywords="machine learning", keywords="predictive models", keywords="diagnostic models", keywords="prognostic models", keywords="artificial intelligence", keywords="editorial policy", doi="10.2196/52508", url="https://www.jmir.org/2024/1/e52508", url="http://www.ncbi.nlm.nih.gov/pubmed/38696776" } @Article{info:doi/10.2196/48330, author="Ke, Yuhe and Yang, Rui and Liu, Nan", title="Comparing Open-Access Database and Traditional Intensive Care Studies Using Machine Learning: Bibliometric Analysis Study", journal="J Med Internet Res", year="2024", month="Apr", day="17", volume="26", pages="e48330", keywords="BERTopic", keywords="critical care", keywords="eICU", keywords="machine learning", keywords="MIMIC", keywords="Medical Information Mart for Intensive Care", keywords="natural language processing", abstract="Background: Intensive care research has predominantly relied on conventional methods like randomized controlled trials. However, the increasing popularity of open-access, free databases in the past decade has opened new avenues for research, offering fresh insights. Leveraging machine learning (ML) techniques enables the analysis of trends in a vast number of studies. Objective: This study aims to conduct a comprehensive bibliometric analysis using ML to compare trends and research topics in traditional intensive care unit (ICU) studies and those done with open-access databases (OADs). Methods: We used ML for the analysis of publications in the Web of Science database in this study. Articles were categorized into ``OAD'' and ``traditional intensive care'' (TIC) studies. OAD studies were included in the Medical Information Mart for Intensive Care (MIMIC), eICU Collaborative Research Database (eICU-CRD), Amsterdam University Medical Centers Database (AmsterdamUMCdb), High Time Resolution ICU Dataset (HiRID), and Pediatric Intensive Care database. TIC studies included all other intensive care studies. Uniform manifold approximation and projection was used to visualize the corpus distribution. The BERTopic technique was used to generate 30 topic-unique identification numbers and to categorize topics into 22 topic families. Results: A total of 227,893 records were extracted. After exclusions, 145,426 articles were identified as TIC and 1301 articles as OAD studies. TIC studies experienced exponential growth over the last 2 decades, culminating in a peak of 16,378 articles in 2021, while OAD studies demonstrated a consistent upsurge since 2018. Sepsis, ventilation-related research, and pediatric intensive care were the most frequently discussed topics. TIC studies exhibited broader coverage than OAD studies, suggesting a more extensive research scope. Conclusions: This study analyzed ICU research, providing valuable insights from a large number of publications. OAD studies complement TIC studies, focusing on predictive modeling, while TIC studies capture essential qualitative information. Integrating both approaches in a complementary manner is the future direction for ICU research. Additionally, natural language processing techniques offer a transformative alternative for literature review and bibliometric analysis. ", doi="10.2196/48330", url="https://www.jmir.org/2024/1/e48330", url="http://www.ncbi.nlm.nih.gov/pubmed/38630522" } @Article{info:doi/10.2196/54490, author="Wu, MeiJung and Islam, Mohaimenul Md and Poly, Nasrin Tahmina and Lin, Ming-Chin", title="Application of AI in Sepsis: Citation Network Analysis and Evidence Synthesis", journal="Interact J Med Res", year="2024", month="Apr", day="15", volume="13", pages="e54490", keywords="AI", keywords="artificial intelligence", keywords="bibliometric analysis", keywords="bibliometric", keywords="citation", keywords="deep learning", keywords="machine learning", keywords="network analysis", keywords="publication", keywords="sepsis", keywords="trend", keywords="visualization", keywords="VOSviewer", keywords="Web of Science", keywords="WoS", abstract="Background: Artificial intelligence (AI) has garnered considerable attention in the context of sepsis research, particularly in personalized diagnosis and treatment. Conducting a bibliometric analysis of existing publications can offer a broad overview of the field and identify current research trends and future research directions. Objective: The objective of this study is to leverage bibliometric data to provide a comprehensive overview of the application of AI in sepsis. Methods: We conducted a search in the Web of Science Core Collection database to identify relevant articles published in English until August 31, 2023. A predefined search strategy was used, evaluating titles, abstracts, and full texts as needed. We used the Bibliometrix and VOSviewer tools to visualize networks showcasing the co-occurrence of authors, research institutions, countries, citations, and keywords. Results: A total of 259 relevant articles published between 2014 and 2023 (until August) were identified. Over the past decade, the annual publication count has consistently risen. Leading journals in this domain include Critical Care Medicine (17/259, 6.6\%), Frontiers in Medicine (17/259, 6.6\%), and Scientific Reports (11/259, 4.2\%). The United States (103/259, 39.8\%), China (83/259, 32\%), United Kingdom (14/259, 5.4\%), and Taiwan (12/259, 4.6\%) emerged as the most prolific countries in terms of publications. Notable institutions in this field include the University of California System, Emory University, and Harvard University. The key researchers working in this area include Ritankar Das, Chris Barton, and Rishikesan Kamaleswaran. Although the initial period witnessed a relatively low number of articles focused on AI applications for sepsis, there has been a significant surge in research within this area in recent years (2014-2023). Conclusions: This comprehensive analysis provides valuable insights into AI-related research conducted in the field of sepsis, aiding health care policy makers and researchers in understanding the potential of AI and formulating effective research plans. Such analysis serves as a valuable resource for determining the advantages, sustainability, scope, and potential impact of AI models in sepsis. ", doi="10.2196/54490", url="https://www.i-jmr.org/2024/1/e54490", url="http://www.ncbi.nlm.nih.gov/pubmed/38621231" } @Article{info:doi/10.2196/49411, author="Abd-alrazaq, Alaa and Nashwan, J. Abdulqadir and Shah, Zubair and Abujaber, Ahmad and Alhuwail, Dari and Schneider, Jens and AlSaad, Rawan and Ali, Hazrat and Alomoush, Waleed and Ahmed, Arfan and Aziz, Sarah", title="Machine Learning--Based Approach for Identifying Research Gaps: COVID-19 as a Case Study", journal="JMIR Form Res", year="2024", month="Mar", day="5", volume="8", pages="e49411", keywords="research gaps", keywords="research gap", keywords="research topic", keywords="research topics", keywords="scientific literature", keywords="literature review", keywords="machine learning", keywords="COVID-19", keywords="BERTopic", keywords="topic clustering", keywords="text analysis", keywords="BERT", keywords="NLP", keywords="natural language processing", keywords="review methods", keywords="review methodology", keywords="SARS-CoV-2", keywords="coronavirus", keywords="COVID", abstract="Background: Research gaps refer to unanswered questions in the existing body of knowledge, either due to a lack of studies or inconclusive results. Research gaps are essential starting points and motivation in scientific research. Traditional methods for identifying research gaps, such as literature reviews and expert opinions, can be time consuming, labor intensive, and prone to bias. They may also fall short when dealing with rapidly evolving or time-sensitive subjects. Thus, innovative scalable approaches are needed to identify research gaps, systematically assess the literature, and prioritize areas for further study in the topic of interest. Objective: In this paper, we propose a machine learning--based approach for identifying research gaps through the analysis of scientific literature. We used the COVID-19 pandemic as a case study. Methods: We conducted an analysis to identify research gaps in COVID-19 literature using the COVID-19 Open Research (CORD-19) data set, which comprises 1,121,433 papers related to the COVID-19 pandemic. Our approach is based on the BERTopic topic modeling technique, which leverages transformers and class-based term frequency-inverse document frequency to create dense clusters allowing for easily interpretable topics. Our BERTopic-based approach involves 3 stages: embedding documents, clustering documents (dimension reduction and clustering), and representing topics (generating candidates and maximizing candidate relevance). Results: After applying the study selection criteria, we included 33,206 abstracts in the analysis of this study. The final list of research gaps identified 21 different areas, which were grouped into 6 principal topics. These topics were: ``virus of COVID-19,'' ``risk factors of COVID-19,'' ``prevention of COVID-19,'' ``treatment of COVID-19,'' ``health care delivery during COVID-19,'' ``and impact of COVID-19.'' The most prominent topic, observed in over half of the analyzed studies, was ``the impact of COVID-19.'' Conclusions: The proposed machine learning--based approach has the potential to identify research gaps in scientific literature. This study is not intended to replace individual literature research within a selected topic. Instead, it can serve as a guide to formulate precise literature search queries in specific areas associated with research questions that previous publications have earmarked for future exploration. Future research should leverage an up-to-date list of studies that are retrieved from the most common databases in the target area. When feasible, full texts or, at minimum, discussion sections should be analyzed rather than limiting their analysis to abstracts. Furthermore, future studies could evaluate more efficient modeling algorithms, especially those combining topic modeling with statistical uncertainty quantification, such as conformal prediction. ", doi="10.2196/49411", url="https://formative.jmir.org/2024/1/e49411", url="http://www.ncbi.nlm.nih.gov/pubmed/38441952" } @Article{info:doi/10.2196/57779, author="Graham, Scott S. and Shiva, Jade and Sharma, Nandini and Barbour, B. Joshua and Majdik, P. Zoltan and Rousseau, F. Justin", title="Conflicts of Interest Publication Disclosures: Descriptive Study", journal="JMIR Data", year="2024", month="Oct", day="31", volume="5", pages="e57779", keywords="conflicts of interest", keywords="biomedical publishing", keywords="research integrity", keywords="dataset", keywords="COI", keywords="ethical", keywords="ethics", keywords="publishing", keywords="drugs", keywords="pharmacies", keywords="pharmacology", keywords="pharmacotherapy", keywords="pharmaceuticals", keywords="medication", keywords="disclosure", keywords="information science", keywords="library science", keywords="open data", abstract="Background: Multiple lines of previous research have documented that author conflicts of interest (COI) can compromise the integrity of the biomedical research enterprise. However, continuing research that would investigate why, how, and in what circumstances COI is most risky is stymied by the difficulty in accessing disclosure statements, which are not widely represented in available databases. Objective: In this study, we describe a new open access dataset of COI disclosures extracted from published biomedical journal papers. Methods: To develop the dataset, we used ClinCalc's Top 300 drugs lists for 2017 and 2018 to identify 319 of the most commonly used drugs. Search strategies for each product were developed using the National Library of Medicine's and MeSH (Medical Subject Headings) browser and deployed using the eUtilities application programming interface in April 2021. We identified the 150 most relevant papers for each product and extracted COI disclosure statements from PubMed, PubMed Central, or retrieved papers as necessary. Results: Conflicts of Interest Publication Disclosures (COIPonD) is a new dataset that captures author-reported COI disclosures for biomedical research papers published in a wide range of journals and subspecialties. COIPonD captures author-reported disclosure information (including lack of disclosure) for over 38,000 PubMed-indexed papers published between 1949 and 2022. The collected papers are indexed by discussed drug products with a focus on the 319 most commonly used drugs in the United States. Conclusions: COIPonD should accelerate research efforts to understand the effects of COI on the biomedical research enterprise. In particular, this dataset should facilitate new studies of COI effects across disciplines and subspecialties. ", doi="10.2196/57779", url="https://data.jmir.org/2024/1/e57779" } @Article{info:doi/10.2196/43089, author="Delardas, Orestis and Giannos, Panagiotis", title="How COVID-19 Affected the Journal Impact Factor of High Impact Medical Journals: Bibliometric Analysis", journal="J Med Internet Res", year="2022", month="Dec", day="21", volume="24", number="12", pages="e43089", keywords="COVID-19", keywords="journal impact factor", keywords="scientometrics", keywords="bibliometrics", keywords="infometrics", keywords="journal", keywords="assessment", keywords="research", keywords="resources", keywords="medical journal", keywords="literature", keywords="database", keywords="community", keywords="behavior", abstract="Background: Journal impact factor (IF) is the leading method of scholarly assessment in today's research world, influencing where scholars submit their research and where funders distribute their resources. COVID-19, one of the most serious health crises, resulted in an unprecedented surge of publications across all areas of knowledge. An important question is whether COVID-19 affected the gold standard of scholarly assessment. Objective: In this paper, we aimed to comprehensively compare the productivity trends of COVID-19 and non--COVID-19 literature as well as track their evolution and scholarly impact across 3 consecutive calendar years. Methods: We took as an example 6 high-impact medical journals (Annals of Internal Medicine [Annals], The British Medical Journal [The BMJ], Journal of the American Medical Association [JAMA], The Lancet, Nature Medicine [NatMed], and The New England Journal of Medicine [NEJM]) and searched the literature using the Web of Science database for manuscripts published between January 1, 2019, and December 31, 2021. To assess the effect of COVID-19 and non--COVID-19 literature in their scholarly impact, we calculated their annual IFs and percentage changes. Thereafter, we estimated the citation probability of COVID-19 and non--COVID-19 publications along with their rates of publication and citation by journal. Results: A significant increase in IF change for manuscripts including COVID-19 published from 2019 to 2020 (P=.002; Annals: 283\%; The BMJ: 199\%; JAMA: 208\%; The Lancet: 392\%; NatMed: 111\%; and NEJM: 196\%) and to 2021 (P=.007; Annals: 41\%; The BMJ: 90\%; JAMA: 6\%; The Lancet: 22\%; NatMed: 53\%; and NEJM: 72\%) was seen, against non--COVID-19 ones. The likelihood of highly cited publications was significantly increased in COVID-19 manuscripts between 2019 and 2021 (Annals: z=3.4, P<.001; The BMJ: z=4.0, P<.001; JAMA: z=3.8, P<.001; The Lancet: z=3.5, P<.001; NatMed: z=5.2, P<.001; and NEJM: z=4.7, P<.001). The publication and citation rates of COVID-19 publications followed a positive trajectory, as opposed to non--COVID-19. The citation rate for COVID-19 publications peaked by the second quarter of 2020 while that of the publication rate approximately a year later. Conclusions: The rapid surge of COVID-19 publications emphasized the capacity of scientific communities to respond against a global health emergency, yet inflated IFs create ambiguity as benchmark tools for assessing scholarly impact. The immediate implication is a loss in value of and trust in journal IFs as metrics of research and scientific rigor perceived by academia and society. Loss of confidence toward procedures employed by highly reputable publishers may incentivize authors to exploit the publication process by monopolizing their research on COVID-19 and encourage them to publish in journals of predatory behavior. ", doi="10.2196/43089", url="https://www.jmir.org/2022/12/e43089", url="http://www.ncbi.nlm.nih.gov/pubmed/36454727" } @Article{info:doi/10.2196/37331, author="Luo, Kai and Yang, Yang and Teo, Hai Hock", title="The Asymmetric Influence of Emotion in the Sharing of COVID-19 Science on Social Media: Observational Study", journal="JMIR Infodemiology", year="2022", month="Dec", day="8", volume="2", number="2", pages="e37331", keywords="COVID-19", keywords="science communication", keywords="emotion", keywords="COVID-19 science", keywords="online social networks", keywords="computational social science", keywords="social media", abstract="Background: Unlike past pandemics, COVID-19 is different to the extent that there is an unprecedented surge in both peer-reviewed and preprint research publications, and important scientific conversations about it are rampant on online social networks, even among laypeople. Clearly, this new phenomenon of scientific discourse is not well understood in that we do not know the diffusion patterns of peer-reviewed publications vis-{\`a}-vis preprints and what makes them viral. Objective: This paper aimed to examine how the emotionality of messages about preprint and peer-reviewed publications shapes their diffusion through online social networks in order to inform health science communicators' and policy makers' decisions on how to promote reliable sharing of crucial pandemic science on social media. Methods: We collected a large sample of Twitter discussions of early (January to May 2020) COVID-19 medical research outputs, which were tracked by Altmetric, in both preprint servers and peer-reviewed journals, and conducted statistical analyses to examine emotional valence, specific emotions, and the role of scientists as content creators in influencing the retweet rate. Results: Our large-scale analyses (n=243,567) revealed that scientific publication tweets with positive emotions were transmitted faster than those with negative emotions, especially for messages about preprints. Our results also showed that scientists' participation in social media as content creators could accentuate the positive emotion effects on the sharing of peer-reviewed publications. Conclusions: Clear communication of critical science is crucial in the nascent stage of a pandemic. By revealing the emotional dynamics in the social media sharing of COVID-19 scientific outputs, our study offers scientists and policy makers an avenue to shape the discussion and diffusion of emerging scientific publications through manipulation of the emotionality of tweets. Scientists could use emotional language to promote the diffusion of more reliable peer-reviewed articles, while avoiding using too much positive emotional language in social media messages about preprints if they think that it is too early to widely communicate the preprint (not peer reviewed) data to the public. ", doi="10.2196/37331", url="https://infodemiology.jmir.org/2022/2/e37331", url="http://www.ncbi.nlm.nih.gov/pubmed/36536762" } @Article{info:doi/10.2196/40905, author="Ahmad, Areebah and Alhanshali, Lina and Jefferson, S. Itisha and Dellavalle, Robert", title="Cochrane Skin Group's Global Social Media Reach: Content Analysis of Facebook, Instagram, and Twitter Posts", journal="JMIR Dermatol", year="2022", month="Nov", day="30", volume="5", number="4", pages="e40905", keywords="social media", keywords="Cochrane Skin", keywords="dermatology", keywords="content engagement", keywords="Facebook", keywords="Cochrane", keywords="Twitter", keywords="social media analysis", keywords="content analysis", keywords="skin disease", keywords="dermatologist", abstract="Background: Researchers in all medical specialties increasingly use social media to educate the public, share new publications with peers, and diversify their audiences. Objective: Given Cochrane Skin Group's expanded use of social media in the past years, we aimed to characterize Cochrane Skin Group's international social media audience and identify themes that result in increased content engagement. Methods: Cochrane Skin Group's Facebook, Instagram, and Twitter analytics data were extracted for follower demographics and the most viewed posts within a 3-year span (June 2019 to June 2022). Results: Overall, Cochrane Skin Group had the highest number of followers on Facebook (n=1037). The number of Instagram and Twitter followers reached 214 and 352, respectively. The greatest numbers of Facebook followers were from Brazil, Egypt, and India, with 271, 299, and 463 followers, respectively. Facebook's most viewed post about Cochrane Skin Group's annual meeting received 1041 views. The top post on Instagram, which introduced Cochrane Skin Group's social media editors, received 2522 views. Conclusions: Each of the social media platforms used by Cochrane Skin Group reached varying audiences all over the world. Across social media platforms, posts regarding Cochrane Skin Group meetings, members, and professional opportunities received the most views. Overall, Cochrane Skin Group's multiplatform social media approach will continue to grow an international audience, connecting people interested in skin disease. ", doi="10.2196/40905", url="https://derma.jmir.org/2022/4/e40905", url="http://www.ncbi.nlm.nih.gov/pubmed/37632904" } @Article{info:doi/10.2196/41456, author="Shubina, Ivanna", title="Scientific Publication Patterns of Systematic Reviews on Psychosocial Interventions Improving Well-being: Bibliometric Analysis", journal="Interact J Med Res", year="2022", month="Nov", day="11", volume="11", number="2", pages="e41456", keywords="psychosocial intervention", keywords="well-being", keywords="systematic review", keywords="bibliometric analysis", keywords="bibliometrics", keywords="scientific research", keywords="medical research", keywords="publication", keywords="publish", keywords="citation", keywords="scientometrics", keywords="mental health", abstract="Background: Despite numerous empirical studies and systematic reviews conducted on the effectiveness of interventions improving psychological well-being, there is no holistic overview of published systematic reviews in this field. Objective: This bibliometric study explored the scientific patterns of the effectiveness of different psychosocial interventions improving well-being among various categories of individuals with mental and physical diseases, to synthesize well-being intervention studies, and to suggest gaps and further studies in this emerging field. Methods: The bibliometric analysis included identifying the most productive authors, institutions, and countries; most explored fields and subjects of study; most active journals and publishers; and performing citation analysis and analyzing publication trends between 2014 and 2022. We focused on data retrieved from known databases, and the study was conducted with a proven bibliometric approach. Results: In total, 156 studies were found concerning the research domains and retrieved using LENS software from high-ranking databases (Crossref, Microsoft Academic, PubMed, and Core). These papers were written in English by 100 authors from 24 countries, among which, the leading country was the United Kingdom. Descriptive characteristics of the publications involved an increased number of publications in 2017 (n=35) and 2019 (n=34) and a decreased number in 2021 (n=4). The top 2 leading authors by citation score are James Thomas (3 papers and 260 citations) and Chris Dickens (3 papers and 182 citations). However, the most cited study had 592 citations. BMJ Open (n=6 articles) is the leading journal in the field of medicine; Clinical Psychology Review (n=5), in psychology; and Frontiers in Psychology, in psychological intervention (n=5) and psychology (n=5). The top 2 publishers were Wiley (n=28) and Elsevier (n=25). Conclusions: This study indicates an overall interest in the declared domains within the last decade. Our findings primarily indicate that psychosocial interventions (PIs) were evaluated as being effective in managing mental and physical problems and enhancing well-being. Cognitive behavioral therapy was assessed as being effective in treating anxiety, psychoeducation in relapse prevention, and gratitude interventions in improving overall health, and the mindfulness approach had a positive impact on decreasing distress and depression. Moreover, all these intervention types resulted in an overall increase in an individuals' well-being and resilience. Integrating social and cultural factors while considering individual differences increases the efficiency of PIs. Furthermore, PIs were evaluated as being effective in managing symptoms of eating disorders, dementia, and cancer. Our findings could help provide researchers an overview of the publication trends on research domains of focus for further studies, since it shows current findings and potential research needs in these fields, and would also benefit practitioners working on increasing their own and their patients' well-being. ", doi="10.2196/41456", url="https://www.i-jmr.org/2022/2/e41456", url="http://www.ncbi.nlm.nih.gov/pubmed/36367767" } @Article{info:doi/10.2196/37532, author="Zhao, Junqiang and Lu, Yi and Qian, Yong and Luo, Yuxin and Yang, Weihua", title="Emerging Trends and Research Foci in Artificial Intelligence for Retinal Diseases: Bibliometric and Visualization Study", journal="J Med Internet Res", year="2022", month="Jun", day="14", volume="24", number="6", pages="e37532", keywords="artificial intelligence", keywords="retinal disease", keywords="data visualization", keywords="bibliometric", keywords="citespace, VOSviewer", keywords="retinal", keywords="eye", keywords="visual impairment", abstract="Background: Patients with retinal diseases may exhibit serious complications that cause severe visual impairment owing to a lack of awareness of retinal diseases and limited medical resources. Understanding how artificial intelligence (AI) is used to make predictions and perform relevant analyses is a very active area of research on retinal diseases. In this study, the relevant Science Citation Index (SCI) literature on the AI of retinal diseases published from 2012 to 2021 was integrated and analyzed. Objective: The aim of this study was to gain insights into the overall application of AI technology to the research of retinal diseases from set time and space dimensions. Methods: Citation data downloaded from the Web of Science Core Collection database for AI in retinal disease publications from January 1, 2012, to December 31, 2021, were considered for this analysis. Information retrieval was analyzed using the online analysis platforms of literature metrology: Bibliometrc, CiteSpace V, and VOSviewer. Results: A total of 197 institutions from 86 countries contributed to relevant publications; China had the largest number and researchers from University College London had the highest H-index. The reference clusters of SCI papers were clustered into 12 categories. ``Deep learning'' was the cluster with the widest range of cocited references. The burst keywords represented the research frontiers in 2018-2021, which were ``eye disease'' and ``enhancement.'' Conclusions: This study provides a systematic analysis method on the literature regarding AI in retinal diseases. Bibliometric analysis enabled obtaining results that were objective and comprehensive. In the future, high-quality retinal image--forming AI technology with strong stability and clinical applicability will continue to be encouraged. ", doi="10.2196/37532", url="https://www.jmir.org/2022/6/e37532", url="http://www.ncbi.nlm.nih.gov/pubmed/35700021" } @Article{info:doi/10.2196/35307, author="Alvarez-Romero, Celia and Martinez-Garcia, Alicia and Ternero Vega, Jara and D{\'i}az-Jim{\`e}nez, Pablo and Jim{\`e}nez-Juan, Carlos and Nieto-Mart{\'i}n, Dolores Mar{\'i}a and Rom{\'a}n Villar{\'a}n, Esther and Kovacevic, Tomi and Bokan, Darijo and Hromis, Sanja and Djekic Malbasa, Jelena and Besla{\'c}, Suzana and Zaric, Bojan and Gencturk, Mert and Sinaci, Anil A. and Ollero Baturone, Manuel and Parra Calder{\'o}n, Luis Carlos", title="Predicting 30-Day Readmission Risk for Patients With Chronic Obstructive Pulmonary Disease Through a Federated Machine Learning Architecture on Findable, Accessible, Interoperable, and Reusable (FAIR) Data: Development and Validation Study", journal="JMIR Med Inform", year="2022", month="Jun", day="2", volume="10", number="6", pages="e35307", keywords="FAIR principles", keywords="research data management", keywords="clinical validation", keywords="chronic obstructive pulmonary disease", keywords="privacy-preserving distributed data mining", keywords="early predictive model", abstract="Background: Owing to the nature of health data, their sharing and reuse for research are limited by legal, technical, and ethical implications. In this sense, to address that challenge and facilitate and promote the discovery of scientific knowledge, the Findable, Accessible, Interoperable, and Reusable (FAIR) principles help organizations to share research data in a secure, appropriate, and useful way for other researchers. Objective: The objective of this study was the FAIRification of existing health research data sets and applying a federated machine learning architecture on top of the FAIRified data sets of different health research performing organizations. The entire FAIR4Health solution was validated through the assessment of a federated model for real-time prediction of 30-day readmission risk in patients with chronic obstructive pulmonary disease (COPD). Methods: The application of the FAIR principles on health research data sets in 3 different health care settings enabled a retrospective multicenter study for the development of specific federated machine learning models for the early prediction of 30-day readmission risk in patients with COPD. This predictive model was generated upon the FAIR4Health platform. Finally, an observational prospective study with 30 days follow-up was conducted in 2 health care centers from different countries. The same inclusion and exclusion criteria were used in both retrospective and prospective studies. Results: Clinical validation was demonstrated through the implementation of federated machine learning models on top of the FAIRified data sets from different health research performing organizations. The federated model for predicting the 30-day hospital readmission risk was trained using retrospective data from 4.944 patients with COPD. The assessment of the predictive model was performed using the data of 100 recruited (22 from Spain and 78 from Serbia) out of 2070 observed (records viewed) patients during the observational prospective study, which was executed from April 2021 to September 2021. Significant accuracy (0.98) and precision (0.25) of the predictive model generated upon the FAIR4Health platform were observed. Therefore, the generated prediction of 30-day readmission risk was confirmed in 87\% (87/100) of cases. Conclusions: Implementing a FAIR data policy in health research performing organizations to facilitate data sharing and reuse is relevant and needed, following the discovery, access, integration, and analysis of health research data. The FAIR4Health project proposes a technological solution in the health domain to facilitate alignment with the FAIR principles. ", doi="10.2196/35307", url="https://medinform.jmir.org/2022/6/e35307", url="http://www.ncbi.nlm.nih.gov/pubmed/35653170" } @Article{info:doi/10.2196/36086, author="Yeung, Kan Andy Wai and Kulnik, Tino Stefan and Parvanov, D. Emil and Fassl, Anna and Eibensteiner, Fabian and V{\"o}lkl-Kernstock, Sabine and Kletecka-Pulker, Maria and Crutzen, Rik and Gutenberg, Johanna and H{\"o}ppchen, Isabel and Niebauer, Josef and Smeddinck, David Jan and Willschke, Harald and Atanasov, G. Atanas", title="Research on Digital Technology Use in Cardiology: Bibliometric Analysis", journal="J Med Internet Res", year="2022", month="May", day="11", volume="24", number="5", pages="e36086", keywords="cardiovascular", keywords="heart", keywords="hypertension", keywords="atrial fibrillation", keywords="cardiopulmonary resuscitation", keywords="electrocardiography", keywords="photoplethysmography", keywords="wearable device, digital health, mHealth", keywords="cardiology", keywords="cardiac", keywords="health application", abstract="Background: Digital technology uses in cardiology have become a popular research focus in recent years. However, there has been no published bibliometric report that analyzed the corresponding academic literature in order to derive key publishing trends and characteristics of this scientific area. Objective: We used a bibliometric approach to identify and analyze the academic literature on digital technology uses in cardiology, and to unveil popular research topics, key authors, institutions, countries, and journals. We further captured the cardiovascular conditions and diagnostic tools most commonly investigated within this field. Methods: The Web of Science electronic database was queried to identify relevant papers on digital technology uses in cardiology. Publication and citation data were acquired directly from the database. Complete bibliographic data were exported to VOSviewer, a dedicated bibliometric software package, and related to the semantic content of titles, abstracts, and keywords. A term map was constructed for findings visualization. Results: The analysis was based on data from 12,529 papers. Of the top 5 most productive institutions, 4 were based in the United States. The United States was the most productive country (4224/12,529, 33.7\%), followed by United Kingdom (1136/12,529, 9.1\%), Germany (1067/12,529, 8.5\%), China (682/12,529, 5.4\%), and Italy (622/12,529, 5.0\%). Cardiovascular diseases that had been frequently investigated included hypertension (152/12,529, 1.2\%), atrial fibrillation (122/12,529, 1.0\%), atherosclerosis (116/12,529, 0.9\%), heart failure (106/12,529, 0.8\%), and arterial stiffness (80/12,529, 0.6\%). Recurring modalities were electrocardiography (170/12,529, 1.4\%), angiography (127/12,529, 1.0\%), echocardiography (127/12,529, 1.0\%), digital subtraction angiography (111/12,529, 0.9\%), and photoplethysmography (80/12,529, 0.6\%). For a literature subset on smartphone apps and wearable devices, the Journal of Medical Internet Research (20/632, 3.2\%) and other JMIR portfolio journals (51/632, 8.0\%) were the major publishing venues. Conclusions: Digital technology uses in cardiology target physicians, patients, and the general public. Their functions range from assisting diagnosis, recording cardiovascular parameters, and patient education, to teaching laypersons about cardiopulmonary resuscitation. This field already has had a great impact in health care, and we anticipate continued growth. ", doi="10.2196/36086", url="https://www.jmir.org/2022/5/e36086", url="http://www.ncbi.nlm.nih.gov/pubmed/35544307" } @Article{info:doi/10.2196/32357, author="Fecho, Karamarie and Ahalt, C. Stanley and Appold, Stephen and Arunachalam, Saravanan and Pfaff, Emily and Stillwell, Lisa and Valencia, Alejandro and Xu, Hao and Peden, B. David", title="Development and Application of an Open Tool for Sharing and Analyzing Integrated Clinical and Environmental Exposures Data: Asthma Use Case", journal="JMIR Form Res", year="2022", month="Apr", day="1", volume="6", number="4", pages="e32357", keywords="open patient data", keywords="electronic health records", keywords="airborne pollutant exposures", keywords="socioeconomic exposures", keywords="medication exposures", keywords="asthma exacerbation", abstract="Background: The Integrated Clinical and Environmental Exposures Service (ICEES) serves as an open-source, disease-agnostic, regulatory-compliant framework and approach for openly exposing and exploring clinical data that have been integrated at the patient level with a variety of environmental exposures data. ICEES is equipped with tools to support basic statistical exploration of the integrated data in a completely open manner. Objective: This study aims to further develop and apply ICEES as a novel tool for openly exposing and exploring integrated clinical and environmental data. We focus on an asthma use case. Methods: We queried the ICEES open application programming interface (OpenAPI) using a functionality that supports chi-square tests between feature variables and a primary outcome measure, with a Bonferroni correction for multiple comparisons ($\alpha$=.001). We focused on 2 primary outcomes that are indicative of asthma exacerbations: annual emergency department (ED) or inpatient visits for respiratory issues; and annual prescriptions for prednisone. Results: Of the 157,410 patients within the asthma cohort, 26,332 (16.73\%) had 1 or more annual ED or inpatient visits for respiratory issues, and 17,056 (10.84\%) had 1 or more annual prescriptions for prednisone. We found that close proximity to a major roadway or highway, exposure to high levels of particulate matter ?2.5 $\mu$m (PM2.5) or ozone, female sex, Caucasian race, low residential density, lack of health insurance, and low household income were significantly associated with asthma exacerbations (P<.001). Asthma exacerbations did not vary by rural versus urban residence. Moreover, the results were largely consistent across outcome measures. Conclusions: Our results demonstrate that the open-source ICEES can be used to replicate and extend published findings on factors that influence asthma exacerbations. As a disease-agnostic, open-source approach for integrating, exposing, and exploring patient-level clinical and environmental exposures data, we believe that ICEES will have broad adoption by other institutions and application in environmental health and other biomedical fields. ", doi="10.2196/32357", url="https://formative.jmir.org/2022/4/e32357", url="http://www.ncbi.nlm.nih.gov/pubmed/35363149" } @Article{info:doi/10.2196/30258, author="Douze, Laura and Pelayo, Sylvia and Messaadi, Nassir and Grosjean, Julien and Kerdelhu{\'e}, Ga{\'e}tan and Marcilly, Romaric", title="Designing Formulae for Ranking Search Results: Mixed Methods Evaluation Study", journal="JMIR Hum Factors", year="2022", month="Mar", day="25", volume="9", number="1", pages="e30258", keywords="information retrieval", keywords="search engine", keywords="topical relevance", keywords="search result ranking", keywords="user testing", keywords="human factors", abstract="Background: A major factor in the success of any search engine is the relevance of the search results; a tool should sort the search results to present the most relevant documents first. Assessing the performance of the ranking formula is an important part of search engine evaluation. However, the methods currently used to evaluate ranking formulae mainly collect quantitative data and do not gather qualitative data, which help to understand what needs to be improved to tailor the formulae to their end users. Objective: This study aims to evaluate 2 different parameter settings of the ranking formula of LiSSa (the French acronym for scientific literature in health care; Department of Medical Informatics and Information), a tool that provides access to health scientific literature in French, to adapt the formula to the needs of the end users. Methods: To collect quantitative and qualitative data, user tests were carried out with representative end users of LiSSa: 10 general practitioners and 10 registrars. Participants first assessed the relevance of the search results and then rated the ranking criteria used in the 2 formulae. Verbalizations were analyzed to characterize each criterion. Results: A formula that prioritized articles representing a consensus in the field was preferred. When users assess an article's relevance, they judge its topic, methods, and value in clinical practice. Conclusions: Following the evaluation, several improvements were implemented to give more weight to articles that match the search topic and to downgrade articles that have less informative or scientific value for the reader. Applying a qualitative methodology generates valuable user inputs to improve the ranking formula and move toward a highly usable search engine. ", doi="10.2196/30258", url="https://humanfactors.jmir.org/2022/1/e30258", url="http://www.ncbi.nlm.nih.gov/pubmed/35333180" } @Article{info:doi/10.2196/25243, author="Rivera, M. Yonaira and Moran, B. Meghan and Thrul, Johannes and Joshu, Corinne and Smith, C. Katherine", title="Contextualizing Engagement With Health Information on Facebook: Using the Social Media Content and Context Elicitation Method", journal="J Med Internet Res", year="2022", month="Mar", day="4", volume="24", number="3", pages="e25243", keywords="mixed methods", keywords="data collection", keywords="social media", keywords="cancer", keywords="health information", keywords="Facebook", keywords="digital health", abstract="Background: Most of what is known regarding health information engagement on social media stems from quantitative methodologies. Public health literature often quantifies engagement by measuring likes, comments, and/or shares of posts within health organizations' Facebook pages. However, this content may not represent the health information (and misinformation) generally available to and consumed by platform users. Furthermore, some individuals may prefer to engage with information without leaving quantifiable digital traces. Mixed methods approaches may provide a way of surpassing the constraints of assessing engagement with health information by using only currently available social media metrics. Objective: This study aims to discuss the limitations of current approaches in assessing health information engagement on Facebook and presents the social media content and context elicitation method, a qualitatively driven, mixed methods approach to understanding engagement with health information and how engagement may lead to subsequent actions. Methods: Data collection, management, and analysis using the social media content and context elicitation method are presented. This method was developed for a broader study exploring how and why US Latinos and Latinas engage with cancer prevention and screening information on Facebook. The study included 20 participants aged between 40 and 75 years without cancer who participated in semistructured, in-depth interviews to discuss their Facebook use and engagement with cancer information on the platform. Participants accessed their Facebook account alongside the researcher, typed cancer in the search bar, and discussed cancer-related posts they engaged with during the previous 12 months. Engagement was defined as liking, commenting, and/or sharing a post; clicking on a post link; reading an article in a post; and/or watching a video within a post. Content engagement prompted questions regarding the reasons for engagement and whether engagement triggered further action. Data were managed using MAXQDA (VERBI GmbH) and analyzed using thematic and content analyses. Results: Data emerging from the social media content and context elicitation method demonstrated that participants mainly engaged with cancer prevention and screening information by viewing and/or reading content (48/66, 73\%) without liking, commenting, or sharing it. This method provided rich content regarding how US Latinos and Latinas engage with and act upon cancer prevention and screening information on Facebook. We present 2 emblematic cases from the main study to exemplify the additional information and context elicited from this methodology, which is currently lacking from quantitative approaches. Conclusions: The social media content and context elicitation method allows a better representation and deeper contextualization of how people engage with and act upon health information and misinformation encountered on social media. This method may be applied to future studies regarding how to best communicate health information on social media, including how these affect assessments of message credibility and accuracy, which can influence health outcomes. ", doi="10.2196/25243", url="https://www.jmir.org/2022/3/e25243", url="http://www.ncbi.nlm.nih.gov/pubmed/35254266" } @Article{info:doi/10.2196/33996, author="Nowlin, Ross and Wirtz, Alexis and Wenger, David and Ottwell, Ryan and Cook, Courtney and Arthur, Wade and Sallee, Brigitte and Levin, Jarad and Hartwell, Micah and Wright, Drew and Sealey, Meghan and Zhu, Lan and Vassar, Matt", title="Spin in Abstracts of Systematic Reviews and Meta-analyses of Melanoma Therapies: Cross-sectional Analysis", journal="JMIR Dermatol", year="2022", month="Feb", day="24", volume="5", number="1", pages="e33996", keywords="melanoma", keywords="spin", keywords="melanoma treatment", keywords="skin conditions", keywords="skin", keywords="misinterpreting data", keywords="misinterpretation", keywords="skin cancer", abstract="Background: Spin is defined as the misrepresentation of a study's results, which may lead to misperceptions or misinterpretation of the findings. Spin has previously been found in randomized controlled trials and systematic reviews of acne vulgaris treatments and treatments of various nondermatological conditions. Objective: The purpose of this study was to quantify the presence of spin in abstracts of systematic reviews and meta-analyses of melanoma therapies and identify any related secondary characteristics of these articles. Methods: We used a cross-sectional approach on June 2, 2020, to search the MEDLINE and Embase databases from their inception. To meet inclusion criteria, a study was required to be a systematic review or meta-analysis pertaining to the treatment of melanoma in human subjects, and reported in English. We used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) definition of systematic reviews and meta-analyses. Data were extracted in a masked, duplicate fashion. We conducted a powered bivariate linear regression and calculated odds ratios for each study characteristic. Results: A total of 200 systematic reviews met the inclusion criteria. We identified spin in 38\% (n=76) of the abstracts. The most common type of spin found was type 3 (selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention), occurring 40 times; the least common was type 2 (title claims or suggests a beneficial effect of the experimental intervention not supported by the findings), which was not present in any included abstracts. We found that abstracts pertaining to pharmacologic interventions were 3.84 times more likely to contain spin. The likelihood of an article containing spin has decreased annually (adjusted odds ratio 0.91, 95\% CI 0.84-0.99). No significant correlation between funding source or other study characteristics and the presence of spin was identified. Conclusions: We have found that spin is fairly common in the abstracts of systematic reviews of melanoma treatments, but the prevalence of spin in these abstracts has been declining from 1992-2020. ", doi="10.2196/33996", url="https://derma.jmir.org/2022/1/e33996", url="http://www.ncbi.nlm.nih.gov/pubmed/37632865" } @Article{info:doi/10.2196/32125, author="Masselot, Camille and Greshake Tzovaras, Bastian and Graham, B. Chris L. and Finnegan, Gary and Jeyaram, Rathin and Vitali, Isabelle and Landrain, Thomas and Santolini, Marc", title="Implementing the Co-Immune Open Innovation Program to Address Vaccination Hesitancy and Access to Vaccines: Retrospective Study", journal="J Particip Med", year="2022", month="Jan", day="21", volume="14", number="1", pages="e32125", keywords="open science", keywords="open innovation", keywords="programmatic research", keywords="collective intelligence", keywords="web based", keywords="immunization", keywords="vaccination access", keywords="vaccine hesitancy", keywords="innovation", keywords="vaccine", keywords="public health", keywords="access", keywords="framework", keywords="participatory", keywords="design", keywords="implementation", abstract="Background: The rise of major complex public health problems, such as vaccination hesitancy and access to vaccination, requires innovative, open, and transdisciplinary approaches. Yet, institutional silos and lack of participation on the part of nonacademic citizens in the design of solutions hamper efforts to meet these challenges. Against this background, new solutions have been explored, with participatory research, citizen science, hackathons, and challenge-based approaches being applied in the context of public health. Objective: Our aim was to develop a program for creating citizen science and open innovation projects that address the contemporary challenges of vaccination in France and around the globe. Methods: We designed and implemented Co-Immune, a program created to tackle the question of vaccination hesitancy and access to vaccination through an online and offline challenge-based open innovation approach. The program was run on the open science platform Just One Giant Lab. Results: Over a 6-month period, the Co-Immune program gathered 234 participants of diverse backgrounds and 13 partners from the public and private sectors. The program comprised 10 events to facilitate the creation of 20 new projects, as well as the continuation of two existing projects, to address the issues of vaccination hesitancy and access, ranging from app development and data mining to analysis and game design. In an open framework, the projects made their data, code, and solutions publicly available. Conclusions: Co-Immune highlights how open innovation approaches and online platforms can help to gather and coordinate noninstitutional communities in a rapid, distributed, and global way toward solving public health issues. Such initiatives can lead to the production and transfer of knowledge, creating novel solutions in the public health sector. The example of Co-Immune contributes to paving the way for organizations and individuals to collaboratively tackle future global challenges. ", doi="10.2196/32125", url="https://jopm.jmir.org/2022/1/e32125", url="http://www.ncbi.nlm.nih.gov/pubmed/35060917" } @Article{info:doi/10.2196/41446, author="Bernardi, Andrade Filipe and Alves, Domingos and Crepaldi, Nathalia and Yamada, Bettiol Diego and Lima, Costa Vin{\'i}cius and Rijo, Rui", title="Data Quality in Health Research: Integrative Literature Review", journal="J Med Internet Res", year="2023", month="Oct", day="31", volume="25", pages="e41446", keywords="data quality", keywords="research", keywords="digital health", keywords="review", keywords="decision-making", keywords="health data", keywords="research network", keywords="artificial intelligence", keywords="e-management", keywords="digital governance", keywords="reliability", keywords="database", keywords="health system", keywords="health services", keywords="health stakeholders", abstract="Background: Decision-making and strategies to improve service delivery must be supported by reliable health data to generate consistent evidence on health status. The data quality management process must ensure the reliability of collected data. Consequently, various methodologies to improve the quality of services are applied in the health field. At the same time, scientific research is constantly evolving to improve data quality through better reproducibility and empowerment of researchers and offers patient groups tools for secured data sharing and privacy compliance. Objective: Through an integrative literature review, the aim of this work was to identify and evaluate digital health technology interventions designed to support the conducting of health research based on data quality. Methods: A search was conducted in 6 electronic scientific databases in January 2022: PubMed, SCOPUS, Web of Science, Institute of Electrical and Electronics Engineers Digital Library, Cumulative Index of Nursing and Allied Health Literature, and Latin American and Caribbean Health Sciences Literature. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist and flowchart were used to visualize the search strategy results in the databases. Results: After analyzing and extracting the outcomes of interest, 33 papers were included in the review. The studies covered the period of 2017-2021 and were conducted in 22 countries. Key findings revealed variability and a lack of consensus in assessing data quality domains and metrics. Data quality factors included the research environment, application time, and development steps. Strategies for improving data quality involved using business intelligence models, statistical analyses, data mining techniques, and qualitative approaches. Conclusions: The main barriers to health data quality are technical, motivational, economical, political, legal, ethical, organizational, human resources, and methodological. The data quality process and techniques, from precollection to gathering, postcollection, and analysis, are critical for the final result of a study or the quality of processes and decision-making in a health care organization. The findings highlight the need for standardized practices and collaborative efforts to enhance data quality in health research. Finally, context guides decisions regarding data quality strategies and techniques. International Registered Report Identifier (IRRID): RR2-10.1101/2022.05.31.22275804 ", doi="10.2196/41446", url="https://www.jmir.org/2023/1/e41446", url="http://www.ncbi.nlm.nih.gov/pubmed/37906223" } @Article{info:doi/10.2196/48529, author="Sebo, Paul and Schwarz, Jo{\"e}lle and Achtari, Margaux and Clair, Carole", title="Women Are Underrepresented Among Authors of Retracted Publications: Retrospective Study of 134 Medical Journals", journal="J Med Internet Res", year="2023", month="Oct", day="6", volume="25", pages="e48529", keywords="error", keywords="gender", keywords="misconduct", keywords="publication", keywords="research", keywords="retraction", keywords="scientific integrity", keywords="woman", keywords="women", keywords="publish", keywords="publishing", keywords="inequality", keywords="retractions", keywords="integrity", keywords="fraud", keywords="plagiarism", keywords="research study", keywords="research article", keywords="scientific research", keywords="journal", keywords="retrospective", doi="10.2196/48529", url="https://www.jmir.org/2023/1/e48529", url="http://www.ncbi.nlm.nih.gov/pubmed/37801343" } @Article{info:doi/10.2196/45322, author="Ide, Kazuki", title="The Skewed Impact of Highly Cited Articles on Journal Impact Factor", journal="J Med Internet Res", year="2023", month="Sep", day="18", volume="25", pages="e45322", keywords="COVID-19", keywords="journal impact factor", keywords="JIF", keywords="scientometrics", keywords="bibliometrics", keywords="infometrics", keywords="journal", keywords="assessment", keywords="research", keywords="resources", keywords="medical journal", keywords="literature", keywords="database", keywords="community", keywords="behavior", doi="10.2196/45322", url="https://www.jmir.org/2023/1/e45322", url="http://www.ncbi.nlm.nih.gov/pubmed/37721788" } @Article{info:doi/10.2196/50844, author="M{\'a}jovsk{\'y}, Martin and Mikolov, Tomas and Netuka, David", title="AI Is Changing the Landscape of Academic Writing: What Can Be Done? Authors' Reply to: AI Increases the Pressure to Overhaul the Scientific Peer Review Process. Comment on ``Artificial Intelligence Can Generate Fraudulent but Authentic-Looking Scientific Medical Articles: Pandora's Box Has Been Opened''", journal="J Med Internet Res", year="2023", month="Aug", day="31", volume="25", pages="e50844", keywords="artificial intelligence", keywords="AI", keywords="publications", keywords="ethics", keywords="neurosurgery", keywords="ChatGPT", keywords="Chat Generative Pre-trained Transformer", keywords="language models", keywords="fraudulent medical articles", doi="10.2196/50844", url="https://www.jmir.org/2023/1/e50844", url="http://www.ncbi.nlm.nih.gov/pubmed/37651175" } @Article{info:doi/10.2196/50591, author="Liu, Nicholas and Brown, Amy", title="AI Increases the Pressure to Overhaul the Scientific Peer Review Process. Comment on ``Artificial Intelligence Can Generate Fraudulent but Authentic-Looking Scientific Medical Articles: Pandora's Box Has Been Opened''", journal="J Med Internet Res", year="2023", month="Aug", day="31", volume="25", pages="e50591", keywords="artificial intelligence", keywords="AI", keywords="publications", keywords="ethics", keywords="neurosurgery", keywords="ChatGPT", keywords="Chat Generative Pre-trained Transformer", keywords="language models", keywords="fraudulent medical articles", doi="10.2196/50591", url="https://www.jmir.org/2023/1/e50591", url="http://www.ncbi.nlm.nih.gov/pubmed/37651167" } @Article{info:doi/10.2196/51584, author="Leung, I. Tiffany and de Azevedo Cardoso, Taiane and Mavragani, Amaryllis and Eysenbach, Gunther", title="Best Practices for Using AI Tools as an Author, Peer Reviewer, or Editor", journal="J Med Internet Res", year="2023", month="Aug", day="31", volume="25", pages="e51584", keywords="publishing", keywords="open access publishing", keywords="open science", keywords="publication policy", keywords="science editing", keywords="scholarly publishing", keywords="scientific publishing", keywords="research", keywords="scientific research", keywords="editorial", keywords="artificial intelligence", keywords="AI", doi="10.2196/51584", url="https://www.jmir.org/2023/1/e51584", url="http://www.ncbi.nlm.nih.gov/pubmed/37651164" } @Article{info:doi/10.2196/48763, author="Klement, William and El Emam, Khaled", title="Consolidated Reporting Guidelines for Prognostic and Diagnostic Machine Learning Modeling Studies: Development and Validation", journal="J Med Internet Res", year="2023", month="Aug", day="31", volume="25", pages="e48763", keywords="machine learning", keywords="prognostic models", keywords="prediction models", keywords="reporting guidelines", keywords="reproducibility guidelines", keywords="diagnostic", keywords="prognostic", keywords="model evaluation", keywords="model training", abstract="Background: The reporting of machine learning (ML) prognostic and diagnostic modeling studies is often inadequate, making it difficult to understand and replicate such studies. To address this issue, multiple consensus and expert reporting guidelines for ML studies have been published. However, these guidelines cover different parts of the analytics lifecycle, and individually, none of them provide a complete set of reporting requirements. Objective: We aimed to consolidate the ML reporting guidelines and checklists in the literature to provide reporting items for prognostic and diagnostic ML in in-silico and shadow mode studies. Methods: We conducted a literature search that identified 192 unique peer-reviewed English articles that provide guidance and checklists for reporting ML studies. The articles were screened by their title and abstract against a set of 9 inclusion and exclusion criteria. Articles that were filtered through had their quality evaluated by 2 raters using a 9-point checklist constructed from guideline development good practices. The average $\kappa$ was 0.71 across all quality criteria. The resulting 17 high-quality source papers were defined as having a quality score equal to or higher than the median. The reporting items in these 17 articles were consolidated and screened against a set of 6 inclusion and exclusion criteria. The resulting reporting items were sent to an external group of 11 ML experts for review and updated accordingly. The updated checklist was used to assess the reporting in 6 recent modeling papers in JMIR AI. Feedback from the external review and initial validation efforts was used to improve the reporting items. Results: In total, 37 reporting items were identified and grouped into 5 categories based on the stage of the ML project: defining the study details, defining and collecting the data, modeling methodology, model evaluation, and explainability. None of the 17 source articles covered all the reporting items. The study details and data description reporting items were the most common in the source literature, with explainability and methodology guidance (ie, data preparation and model training) having the least coverage. For instance, a median of 75\% of the data description reporting items appeared in each of the 17 high-quality source guidelines, but only a median of 33\% of the data explainability reporting items appeared. The highest-quality source articles tended to have more items on reporting study details. Other categories of reporting items were not related to the source article quality. We converted the reporting items into a checklist to support more complete reporting. Conclusions: Our findings supported the need for a set of consolidated reporting items, given that existing high-quality guidelines and checklists do not individually provide complete coverage. The consolidated set of reporting items is expected to improve the quality and reproducibility of ML modeling studies. ", doi="10.2196/48763", url="https://www.jmir.org/2023/1/e48763", url="http://www.ncbi.nlm.nih.gov/pubmed/37651179" } @Article{info:doi/10.2196/46620, author="Zhu, Harrison and Narayana, Vishnu and Zhou, Kelvin and Patel, B. Anisha", title="Altmetric Analysis of Dermatology Manuscript Dissemination During the COVID-19 Era: Cross-Sectional Study", journal="JMIR Dermatol", year="2023", month="Aug", day="16", volume="6", pages="e46620", keywords="altmetric", keywords="media dissemination", keywords="citation number", keywords="bibliometric", keywords="attention score", keywords="social media", abstract="Background: Alternative bibliometrics or altmetrics, is a measure of an academic article's impact on social media outlets, which is quantified by the Altmetric Attention score (AAS). Given a lack of data for altmetric trends during the COVID-19 pandemic, we conducted a comprehensive, multivariable analysis of top dermatology manuscripts published during this time period. Objective: We aim to assess (1) the relationship between traditional bibiliometrics and Altmetrics and (2) factors associated with high AAS. Methods: All abstracted articles published in the top-5 (ranked by SCImago Journal Rankings) peer-reviewed dermatology journals published in 2021 were included in our study. We collected AAS as the dependent variable and categorical predictor variables included journal title, whether a conflict of interest existed, open access status, whether the article was related to COVID-19 or skin-of-color research, and the type of research (eg, clinical, basic science, review, etc). Numerical predictor variables consisted of the impact factor of journal, total citations, and number of authors. Multivariable linear or logistic regression models were used. Results: The relationship between AAS and citation number was significant by multivariable analysis during the COVID-19 pandemic (P<.001). Numerous factors, including studies related to COVID-19, whether the article was open access, title of the journal, and journal impact factor were also independently related to higher AAS (P<.002). Conclusions: Our results validate the use of altmetrics as a complement to traditional bibliometrics, especially in times of widespread scientific interest. Despite existing in a complex realm of bibliometrics, there are also discernable patterns associated with higher AAS. This is especially relevant in the era of growing technologic importance and utility to assess the impact of scientific works within the general public. ", doi="10.2196/46620", url="https://derma.jmir.org/2023/1/e46620", url="http://www.ncbi.nlm.nih.gov/pubmed/37585241" } @Article{info:doi/10.2196/45059, author="Agnello, Marie Danielle and Loisel, Armand Quentin Emile and An, Qingfan and Balaskas, George and Chrifou, Rabab and Dall, Philippa and de Boer, Janneke and Delfmann, Rahel Lea and Gin{\'e}-Garriga, Maria and Goh, Kunshan and Longworth, Raffaella Giuliana and Messiha, Katrina and McCaffrey, Lauren and Smith, Niamh and Steiner, Artur and Vogelsang, Mira and Chastin, Sebastien", title="Establishing a Health CASCADE--Curated Open-Access Database to Consolidate Knowledge About Co-Creation: Novel Artificial Intelligence--Assisted Methodology Based on Systematic Reviews", journal="J Med Internet Res", year="2023", month="Jul", day="18", volume="25", pages="e45059", keywords="co-creation", keywords="co-production", keywords="co-design", keywords="database", keywords="participatory", keywords="methodology", keywords="artificial intelligence", abstract="Background: Co-creation is an approach that aims to democratize research and bridge the gap between research and practice, but the potential fragmentation of knowledge about co-creation has hindered progress. A comprehensive database of published literature from multidisciplinary sources can address this fragmentation through the integration of diverse perspectives, identification and dissemination of best practices, and increase clarity about co-creation. However, two considerable challenges exist. First, there is uncertainty about co-creation terminology, making it difficult to identify relevant literature. Second, the exponential growth of scientific publications has led to an overwhelming amount of literature that surpasses the human capacity for a comprehensive review. These challenges hinder progress in co-creation research and underscore the need for a novel methodology to consolidate and investigate the literature. Objective: This study aimed to synthesize knowledge about co-creation across various fields through the development and application of an artificial intelligence (AI)--assisted selection process. The ultimate goal of this database was to provide stakeholders interested in co-creation with relevant literature. Methods: We created a novel methodology for establishing a curated database. To accommodate the variation in terminology, we used a broad definition of co-creation that encompassed the essence of existing definitions. To filter out irrelevant information, an AI-assisted selection process was used. In addition, we conducted bibliometric analyses and quality control procedures to assess content and accuracy. Overall, this approach allowed us to develop a robust and reliable database that serves as a valuable resource for stakeholders interested in co-creation. Results: The final version of the database included 13,501 papers, which are indexed in Zenodo and accessible in an open-access downloadable format. The quality assessment revealed that 20.3\% (140/688) of the database likely contained irrelevant material, whereas the methodology captured 91\% (58/64) of the relevant literature. Participatory and variations of the term co-creation were the most frequent terms in the title and abstracts of included literature. The predominant source journals included health sciences, sustainability, environmental sciences, medical research, and health services research. Conclusions: This study produced a high-quality, open-access database about co-creation. The study demonstrates that it is possible to perform a systematic review selection process on a fragmented concept using human-AI collaboration. Our unified concept of co-creation includes the co-approaches (co-creation, co-design, and co-production), forms of participatory research, and user involvement. Our analysis of authorship, citations, and source landscape highlights the potential lack of collaboration among co-creation researchers and underscores the need for future investigation into the different research methodologies. The database provides a resource for relevant literature and can support rapid literature reviews about co-creation. It also offers clarity about the current co-creation landscape and helps to address barriers that researchers may face when seeking evidence about co-creation. ", doi="10.2196/45059", url="https://www.jmir.org/2023/1/e45059", url="http://www.ncbi.nlm.nih.gov/pubmed/37463024" } @Article{info:doi/10.2196/49323, author="Ballester, L. Pedro", title="Open Science and Software Assistance: Commentary on ``Artificial Intelligence Can Generate Fraudulent but Authentic-Looking Scientific Medical Articles: Pandora's Box Has Been Opened''", journal="J Med Internet Res", year="2023", month="May", day="31", volume="25", pages="e49323", keywords="artificial intelligence", keywords="AI", keywords="ChatGPT", keywords="open science", keywords="reproducibility", keywords="software assistance", doi="10.2196/49323", url="https://www.jmir.org/2023/1/e49323", url="http://www.ncbi.nlm.nih.gov/pubmed/37256656" } @Article{info:doi/10.2196/46924, author="M{\'a}jovsk{\'y}, Martin and ?ern{\'y}, Martin and Kasal, Mat?j and Komarc, Martin and Netuka, David", title="Artificial Intelligence Can Generate Fraudulent but Authentic-Looking Scientific Medical Articles: Pandora's Box Has Been Opened", journal="J Med Internet Res", year="2023", month="May", day="31", volume="25", pages="e46924", keywords="artificial intelligence", keywords="publications", keywords="ethics", keywords="neurosurgery", keywords="ChatGPT", keywords="language models", keywords="fraudulent medical articles", abstract="Background: Artificial intelligence (AI) has advanced substantially in recent years, transforming many industries and improving the way people live and work. In scientific research, AI can enhance the quality and efficiency of data analysis and publication. However, AI has also opened up the possibility of generating high-quality fraudulent papers that are difficult to detect, raising important questions about the integrity of scientific research and the trustworthiness of published papers. Objective: The aim of this study was to investigate the capabilities of current AI language models in generating high-quality fraudulent medical articles. We hypothesized that modern AI models can create highly convincing fraudulent papers that can easily deceive readers and even experienced researchers. Methods: This proof-of-concept study used ChatGPT (Chat Generative Pre-trained Transformer) powered by the GPT-3 (Generative Pre-trained Transformer 3) language model to generate a fraudulent scientific article related to neurosurgery. GPT-3 is a large language model developed by OpenAI that uses deep learning algorithms to generate human-like text in response to prompts given by users. The model was trained on a massive corpus of text from the internet and is capable of generating high-quality text in a variety of languages and on various topics. The authors posed questions and prompts to the model and refined them iteratively as the model generated the responses. The goal was to create a completely fabricated article including the abstract, introduction, material and methods, discussion, references, charts, etc. Once the article was generated, it was reviewed for accuracy and coherence by experts in the fields of neurosurgery, psychiatry, and statistics and compared to existing similar articles. Results: The study found that the AI language model can create a highly convincing fraudulent article that resembled a genuine scientific paper in terms of word usage, sentence structure, and overall composition. The AI-generated article included standard sections such as introduction, material and methods, results, and discussion, as well a data sheet. It consisted of 1992 words and 17 citations, and the whole process of article creation took approximately 1 hour without any special training of the human user. However, there were some concerns and specific mistakes identified in the generated article, specifically in the references. Conclusions: The study demonstrates the potential of current AI language models to generate completely fabricated scientific articles. Although the papers look sophisticated and seemingly flawless, expert readers may identify semantic inaccuracies and errors upon closer inspection. We highlight the need for increased vigilance and better detection methods to combat the potential misuse of AI in scientific research. At the same time, it is important to recognize the potential benefits of using AI language models in genuine scientific writing and research, such as manuscript preparation and language editing. ", doi="10.2196/46924", url="https://www.jmir.org/2023/1/e46924", url="http://www.ncbi.nlm.nih.gov/pubmed/37256685" } @Article{info:doi/10.2196/42289, author="Johns, Marco and Meurers, Thierry and Wirth, N. Felix and Haber, C. Anna and M{\"u}ller, Armin and Halilovic, Mehmed and Balzer, Felix and Prasser, Fabian", title="Data Provenance in Biomedical Research: Scoping Review", journal="J Med Internet Res", year="2023", month="Mar", day="27", volume="25", pages="e42289", keywords="data provenance", keywords="biomedical research", keywords="scoping review", keywords="systematization", keywords="comparison", abstract="Background: Data provenance refers to the origin, processing, and movement of data. Reliable and precise knowledge about data provenance has great potential to improve reproducibility as well as quality in biomedical research and, therefore, to foster good scientific practice. However, despite the increasing interest on data provenance technologies in the literature and their implementation in other disciplines, these technologies have not yet been widely adopted in biomedical research. Objective: The aim of this scoping review was to provide a structured overview of the body of knowledge on provenance methods in biomedical research by systematizing articles covering data provenance technologies developed for or used in this application area; describing and comparing the functionalities as well as the design of the provenance technologies used; and identifying gaps in the literature, which could provide opportunities for future research on technologies that could receive more widespread adoption. Methods: Following a methodological framework for scoping studies and the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines, articles were identified by searching the PubMed, IEEE Xplore, and Web of Science databases and subsequently screened for eligibility. We included original articles covering software-based provenance management for scientific research published between 2010 and 2021. A set of data items was defined along the following five axes: publication metadata, application scope, provenance aspects covered, data representation, and functionalities. The data items were extracted from the articles, stored in a charting spreadsheet, and summarized in tables and figures. Results: We identified 44 original articles published between 2010 and 2021. We found that the solutions described were heterogeneous along all axes. We also identified relationships among motivations for the use of provenance information, feature sets (capture, storage, retrieval, visualization, and analysis), and implementation details such as the data models and technologies used. The important gap that we identified is that only a few publications address the analysis of provenance data or use established provenance standards, such as PROV. Conclusions: The heterogeneity of provenance methods, models, and implementations found in the literature points to the lack of a unified understanding of provenance concepts for biomedical data. Providing a common framework, a biomedical reference, and benchmarking data sets could foster the development of more comprehensive provenance solutions. ", doi="10.2196/42289", url="https://www.jmir.org/2023/1/e42289", url="http://www.ncbi.nlm.nih.gov/pubmed/36972116" } @Article{info:doi/10.2196/35568, author="{\vS}uster, Simon and Baldwin, Timothy and Lau, Han Jey and Jimeno Yepes, Antonio and Martinez Iraola, David and Otmakhova, Yulia and Verspoor, Karin", title="Automating Quality Assessment of Medical Evidence in Systematic Reviews: Model Development and Validation Study", journal="J Med Internet Res", year="2023", month="Mar", day="13", volume="25", pages="e35568", keywords="critical appraisal", keywords="evidence synthesis", keywords="systematic reviews", keywords="bias detection", keywords="automated quality assessment", abstract="Background: Assessment of the quality of medical evidence available on the web is a critical step in the preparation of systematic reviews. Existing tools that automate parts of this task validate the quality of individual studies but not of entire bodies of evidence and focus on a restricted set of quality criteria. Objective: We proposed a quality assessment task that provides an overall quality rating for each body of evidence (BoE), as well as finer-grained justification for different quality criteria according to the Grading of Recommendation, Assessment, Development, and Evaluation formalization framework. For this purpose, we constructed a new data set and developed a machine learning baseline system (EvidenceGRADEr). Methods: We algorithmically extracted quality-related data from all summaries of findings found in the Cochrane Database of Systematic Reviews. Each BoE was defined by a set of population, intervention, comparison, and outcome criteria and assigned a quality grade (high, moderate, low, or very low) together with quality criteria (justification) that influenced that decision. Different statistical data, metadata about the review, and parts of the review text were extracted as support for grading each BoE. After pruning the resulting data set with various quality checks, we used it to train several neural-model variants. The predictions were compared against the labels originally assigned by the authors of the systematic reviews. Results: Our quality assessment data set, Cochrane Database of Systematic Reviews Quality of Evidence, contains 13,440 instances, or BoEs labeled for quality, originating from 2252 systematic reviews published on the internet from 2002 to 2020. On the basis of a 10-fold cross-validation, the best neural binary classifiers for quality criteria detected risk of bias at 0.78 F1 (P=.68; R=0.92) and imprecision at 0.75 F1 (P=.66; R=0.86), while the performance on inconsistency, indirectness, and publication bias criteria was lower (F1 in the range of 0.3-0.4). The prediction of the overall quality grade into 1 of the 4 levels resulted in 0.5 F1. When casting the task as a binary problem by merging the Grading of Recommendation, Assessment, Development, and Evaluation classes (high+moderate vs low+very low-quality evidence), we attained 0.74 F1. We also found that the results varied depending on the supporting information that is provided as an input to the models. Conclusions: Different factors affect the quality of evidence in the context of systematic reviews of medical evidence. Some of these (risk of bias and imprecision) can be automated with reasonable accuracy. Other quality dimensions such as indirectness, inconsistency, and publication bias prove more challenging for machine learning, largely because they are much rarer. This technology could substantially reduce reviewer workload in the future and expedite quality assessment as part of evidence synthesis. ", doi="10.2196/35568", url="https://www.jmir.org/2023/1/e35568", url="http://www.ncbi.nlm.nih.gov/pubmed/36722350" } @Article{info:doi/10.2196/44217, author="Kiene, Julianne and Minion, Sarah and Rodriguez, Ramiro and Dellavalle, Robert", title="Diversity, Equity, and Inclusion of Dermatology Journals and Their Editorial Board Members", journal="JMIR Dermatol", year="2023", month="Mar", day="10", volume="6", pages="e44217", keywords="diversity", keywords="equity", keywords="inclusion", keywords="dermatology", doi="10.2196/44217", url="https://derma.jmir.org/2023/1/e44217", url="http://www.ncbi.nlm.nih.gov/pubmed/37632920" } @Article{info:doi/10.2196/34051, author="Kudlow, Paul and Brown, Tashauna and Eysenbach, Gunther", title="Citation Advantage of Promoted Articles in a Cross-Publisher Distribution Platform: 36-Month Follow-up to a Randomized Controlled Trial", journal="J Med Internet Res", year="2021", month="Dec", day="10", volume="23", number="12", pages="e34051", keywords="knowledge translation", keywords="knowledge", keywords="dissemination", keywords="digital knowledge translation", keywords="digital publishing", keywords="e-publishing", keywords="open access", keywords="scientometrics", keywords="infometrics", abstract="Background: There are limited evidence-based strategies that have been shown to increase the rate at which peer-reviewed articles are cited. In a previously reported randomized controlled trial, we demonstrated that promotion of article links in an online cross-publisher distribution platform (TrendMD) persistently augments citation rates after 12 months, leading to a statistically significant 50\% increase in citations relative to the control. Objective: This study aims to investigate if the citation advantage of promoted articles upholds after 36 months. Methods: A total of 3200 published articles in 64 peer-reviewed journals across 8 subject areas were block randomized at the subject level to either the TrendMD group (n=1600) or the control group (n=1600) of the study. Articles were promoted in the TrendMD Network for 6 months. We compared the citation rates in both groups after 36 months. Results: At 36 months, we found the citation advantage endured; articles randomized to TrendMD showed a 28\% increase in mean citations relative to the control. The difference in mean citations at 36 months for articles randomized to TrendMD versus the control was 10.52 (95\% CI 3.79-17.25) and was statistically significant (P=.001). Conclusions: To our knowledge, this is the first randomized controlled trial to demonstrate how a postpublication article promotion intervention can be used to persistently augment citations of peer-reviewed articles. TrendMD is an efficient digital tool for knowledge translation and dissemination to targeted audiences to facilitate the uptake of research. ", doi="10.2196/34051", url="https://www.jmir.org/2021/12/e34051", url="http://www.ncbi.nlm.nih.gov/pubmed/34890350" } @Article{info:doi/10.2196/26995, author="Moran, Addy and Hampton, Shawn and Dowson, Scott and Dagdelen, John and Trewartha, Amalie and Ceder, Gerbrand and Persson, Kristin and Saxon, Elise and Barker, Andrew and Charles, Lauren and Webb-Robertson, Bobbie-Jo", title="Online Interactive Platform for COVID-19 Literature Visual Analytics: Platform Development Study", journal="J Med Internet Res", year="2021", month="Jul", day="16", volume="23", number="7", pages="e26995", keywords="COVID-19", keywords="visual analytics", keywords="natural language processing", keywords="scientific literature", keywords="software", keywords="online platform", keywords="literature", keywords="interactive", keywords="publish", keywords="research", keywords="tool", keywords="pattern", keywords="usability", abstract="Background: Papers on COVID-19 are being published at a high rate and concern many different topics. Innovative tools are needed to aid researchers to find patterns in this vast amount of literature to identify subsets of interest in an automated fashion. Objective: We present a new online software resource with a friendly user interface that allows users to query and interact with visual representations of relationships between publications. Methods: We publicly released an application called PLATIPUS (Publication Literature Analysis and Text Interaction Platform for User Studies) that allows researchers to interact with literature supplied by COVIDScholar via a visual analytics platform. This tool contains standard filtering capabilities based on authors, journals, high-level categories, and various research-specific details via natural language processing and dozens of customizable visualizations that dynamically update from a researcher's query. Results: PLATIPUS is available online and currently links to over 100,000 publications and is still growing. This application has the potential to transform how COVID-19 researchers use public literature to enable their research. Conclusions: The PLATIPUS application provides the end user with a variety of ways to search, filter, and visualize over 100,00 COVID-19 publications. ", doi="10.2196/26995", url="https://www.jmir.org/2021/7/e26995", url="http://www.ncbi.nlm.nih.gov/pubmed/34138726" } @Article{info:doi/10.2196/26956, author="Taneja, L. Sonia and Passi, Monica and Bhattacharya, Sumona and Schueler, A. Samuel and Gurram, Sandeep and Koh, Christopher", title="Social Media and Research Publication Activity During Early Stages of the COVID-19 Pandemic: Longitudinal Trend Analysis", journal="J Med Internet Res", year="2021", month="Jun", day="17", volume="23", number="6", pages="e26956", keywords="coronavirus", keywords="COVID-19", keywords="social media", keywords="gastroenterology", keywords="SARS-CoV-2", keywords="research", keywords="literature", keywords="dissemination", keywords="Twitter", keywords="preprint", abstract="Background: The COVID-19 pandemic has highlighted the importance of rapid dissemination of scientific and medical discoveries. Current platforms available for the distribution of scientific and clinical research data and information include preprint repositories and traditional peer-reviewed journals. In recent times, social media has emerged as a helpful platform to share scientific and medical discoveries. Objective: This study aimed to comparatively analyze activity on social media (specifically, Twitter) and that related to publications in the form of preprint and peer-reviewed journal articles in the context of COVID-19 and gastroenterology during the early stages of the COVID-19 pandemic. Methods: COVID-19--related data from Twitter (tweets and user data) and articles published in preprint servers (bioRxiv and medRxiv) as well as in the PubMed database were collected and analyzed during the first 6 months of the pandemic, from December 2019 through May 2020. Global and regional geographic and gastrointestinal organ--specific social media trends were compared to preprint and publication activity. Any relationship between Twitter activity and preprint articles published and that between Twitter activity and PubMed articles published overall, by organ system, and by geographic location were identified using Spearman's rank-order correlation. Results: Over the 6-month period, 73,079 tweets from 44,609 users, 7164 journal publications, and 4702 preprint publications were retrieved. Twitter activity (ie, number of tweets) peaked in March 2020, whereas preprint and publication activity (ie, number of articles published) peaked in April 2020. Overall, strong correlations were identified between trends in Twitter activity and preprint and publication activity (P<.001 for both). COVID-19 data across the three platforms mainly concentrated on pulmonology or critical care, but when analyzing the field of gastroenterology specifically, most tweets pertained to pancreatology, most publications focused on hepatology, and most preprints covered hepatology and luminal gastroenterology. Furthermore, there were significant positive associations between trends in Twitter and publication activity for all gastroenterology topics (luminal gastroenterology: P=.009; hepatology and inflammatory bowel disease: P=.006; gastrointestinal endoscopy: P=.007), except pancreatology (P=.20), suggesting that Twitter activity did not correlate with publication activity for this topic. Finally, Twitter activity was the highest in the United States (7331 tweets), whereas PubMed activity was the highest in China (1768 publications). Conclusions: The COVID-19 pandemic has highlighted the potential of social media as a vehicle for disseminating scientific information during a public health crisis. Sharing and spreading information on COVID-19 in a timely manner during the pandemic has been paramount; this was achieved at a much faster pace on social media, particularly on Twitter. Future investigation could demonstrate how social media can be used to augment and promote scholarly activity, especially as the world begins to increasingly rely on digital or virtual platforms. Scientists and clinicians should consider the use of social media in augmenting public awareness regarding their scholarly pursuits. ", doi="10.2196/26956", url="https://www.jmir.org/2021/6/e26956", url="http://www.ncbi.nlm.nih.gov/pubmed/33974550" } @Article{info:doi/10.2196/29156, author="{\"A}lg{\aa}, Andreas and Eriksson, Oskar and Nordberg, Martin", title="Authors' Reply to: COVID-19 as a ``Force Majeure'' for Non--COVID-19 Clinical and Translational Research. Comment on ``Analysis of Scientific Publications During the Early Phase of the COVID-19 Pandemic: Topic Modeling Study''", journal="J Med Internet Res", year="2021", month="May", day="20", volume="23", number="5", pages="e29156", keywords="COVID-19", keywords="SARS-CoV-2", keywords="coronavirus", keywords="pandemic", keywords="topic modeling", keywords="research", keywords="literature", keywords="medical research", keywords="publishing", keywords="force majeure", doi="10.2196/29156", url="https://www.jmir.org/2021/5/e29156", url="http://www.ncbi.nlm.nih.gov/pubmed/33989170" } @Article{info:doi/10.2196/27937, author="Milovanovic, Petar and Dumic, Igor", title="COVID-19 as a ``Force Majeure'' for Non--COVID-19 Clinical and Translational Research. Comment on ``Analysis of Scientific Publications During the Early Phase of the COVID-19 Pandemic: Topic Modeling Study''", journal="J Med Internet Res", year="2021", month="May", day="20", volume="23", number="5", pages="e27937", keywords="COVID-19", keywords="SARS-CoV-2", keywords="coronavirus", keywords="pandemic", keywords="topic modeling", keywords="research", keywords="literature", keywords="medical research", keywords="publishing", keywords="force majeure", doi="10.2196/27937", url="https://www.jmir.org/2021/5/e27937", url="http://www.ncbi.nlm.nih.gov/pubmed/33989167" } @Article{info:doi/10.2196/25714, author="Vaghela, Uddhav and Rabinowicz, Simon and Bratsos, Paris and Martin, Guy and Fritzilas, Epameinondas and Markar, Sheraz and Purkayastha, Sanjay and Stringer, Karl and Singh, Harshdeep and Llewellyn, Charlie and Dutta, Debabrata and Clarke, M. Jonathan and Howard, Matthew and and Serban, Ovidiu and Kinross, James", title="Using a Secure, Continually Updating, Web Source Processing Pipeline to Support the Real-Time Data Synthesis and Analysis of Scientific Literature: Development and Validation Study", journal="J Med Internet Res", year="2021", month="May", day="6", volume="23", number="5", pages="e25714", keywords="structured data synthesis", keywords="data science", keywords="critical analysis", keywords="web crawl data", keywords="pipeline", keywords="database", keywords="literature", keywords="research", keywords="COVID-19", keywords="infodemic", keywords="decision making", keywords="data", keywords="data synthesis", keywords="misinformation", keywords="infrastructure", keywords="methodology", abstract="Background: The scale and quality of the global scientific response to the COVID-19 pandemic have unquestionably saved lives. However, the COVID-19 pandemic has also triggered an unprecedented ``infodemic''; the velocity and volume of data production have overwhelmed many key stakeholders such as clinicians and policy makers, as they have been unable to process structured and unstructured data for evidence-based decision making. Solutions that aim to alleviate this data synthesis--related challenge are unable to capture heterogeneous web data in real time for the production of concomitant answers and are not based on the high-quality information in responses to a free-text query. Objective: The main objective of this project is to build a generic, real-time, continuously updating curation platform that can support the data synthesis and analysis of a scientific literature framework. Our secondary objective is to validate this platform and the curation methodology for COVID-19--related medical literature by expanding the COVID-19 Open Research Dataset via the addition of new, unstructured data. Methods: To create an infrastructure that addresses our objectives, the PanSurg Collaborative at Imperial College London has developed a unique data pipeline based on a web crawler extraction methodology. This data pipeline uses a novel curation methodology that adopts a human-in-the-loop approach for the characterization of quality, relevance, and key evidence across a range of scientific literature sources. Results: REDASA (Realtime Data Synthesis and Analysis) is now one of the world's largest and most up-to-date sources of COVID-19--related evidence; it consists of 104,000 documents. By capturing curators' critical appraisal methodologies through the discrete labeling and rating of information, REDASA rapidly developed a foundational, pooled, data science data set of over 1400 articles in under 2 weeks. These articles provide COVID-19--related information and represent around 10\% of all papers about COVID-19. Conclusions: This data set can act as ground truth for the future implementation of a live, automated systematic review. The three benefits of REDASA's design are as follows: (1) it adopts a user-friendly, human-in-the-loop methodology by embedding an efficient, user-friendly curation platform into a natural language processing search engine; (2) it provides a curated data set in the JavaScript Object Notation format for experienced academic reviewers' critical appraisal choices and decision-making methodologies; and (3) due to the wide scope and depth of its web crawling method, REDASA has already captured one of the world's largest COVID-19--related data corpora for searches and curation. ", doi="10.2196/25714", url="https://www.jmir.org/2021/5/e25714", url="http://www.ncbi.nlm.nih.gov/pubmed/33835932" } @Article{info:doi/10.2196/24288, author="Ossom-Williamson, Peace and Williams, Maximilian Isaac and Kim, Kukhyoung and Kindratt, B. Tiffany", title="Reporting and Availability of COVID-19 Demographic Data by US Health Departments (April to October 2020): Observational Study", journal="JMIR Public Health Surveill", year="2021", month="Apr", day="6", volume="7", number="4", pages="e24288", keywords="coronavirus disease 2019", keywords="COVID-19", keywords="SARS-CoV-2", keywords="race", keywords="ethnicity", keywords="age", keywords="sex", keywords="health equity", keywords="open data", keywords="dashboards", abstract="Background: There is an urgent need for consistent collection of demographic data on COVID-19 morbidity and mortality and sharing it with the public in open and accessible ways. Due to the lack of consistency in data reporting during the initial spread of COVID-19, the Equitable Data Collection and Disclosure on COVID-19 Act was introduced into the Congress that mandates collection and reporting of demographic COVID-19 data on testing, treatments, and deaths by age, sex, race and ethnicity, primary language, socioeconomic status, disability, and county. To our knowledge, no studies have evaluated how COVID-19 demographic data have been collected before and after the introduction of this legislation. Objective: This study aimed to evaluate differences in reporting and public availability of COVID-19 demographic data by US state health departments and Washington, District of Columbia (DC) before (pre-Act), immediately after (post-Act), and 6 months after (6-month follow-up) the introduction of the Equitable Data Collection and Disclosure on COVID-19 Act in the Congress on April 21, 2020. Methods: We reviewed health department websites of all 50 US states and Washington, DC (N=51). We evaluated how each state reported age, sex, and race and ethnicity data for all confirmed COVID-19 cases and deaths and how they made this data available (ie, charts and tables only or combined with dashboards and machine-actionable downloadable formats) at the three timepoints. Results: We found statistically significant increases in the number of health departments reporting age-specific data for COVID-19 cases (P=.045) and resulting deaths (P=.002), sex-specific data for COVID-19 deaths (P=.003), and race- and ethnicity-specific data for confirmed cases (P=.003) and deaths (P=.005) post-Act and at the 6-month follow-up (P<.05 for all). The largest increases were race and ethnicity state data for confirmed cases (pre-Act: 18/51, 35\%; post-Act: 31/51, 61\%; 6-month follow-up: 46/51, 90\%) and deaths due to COVID-19 (pre-Act: 13/51, 25\%; post-Act: 25/51, 49\%; and 6-month follow-up: 39/51, 76\%). Although more health departments reported race and ethnicity data based on federal requirements (P<.001), over half (29/51, 56.9\%) still did not report all racial and ethnic groups as per the Office of Management and Budget guidelines (pre-Act: 5/51, 10\%; post-Act: 21/51, 41\%; and 6-month follow-up: 27/51, 53\%). The number of health departments that made COVID-19 data available for download significantly increased from 7 to 23 (P<.001) from our initial data collection (April 2020) to the 6-month follow-up, (October 2020). Conclusions: Although the increased demand for disaggregation has improved public reporting of demographics across health departments, an urgent need persists for the introduced legislation to be passed by the Congress for the US states to consistently collect and make characteristics of COVID-19 cases, deaths, and vaccinations available in order to allocate resources to mitigate disease spread. ", doi="10.2196/24288", url="https://publichealth.jmir.org/2021/4/e24288", url="http://www.ncbi.nlm.nih.gov/pubmed/33821804" } @Article{info:doi/10.2196/23011, author="Coetzee, Timothy and Ball, Price Mad and Boutin, Marc and Bronson, Abby and Dexter, T. David and English, A. Rebecca and Furlong, Patricia and Goodman, D. Andrew and Grossman, Cynthia and Hernandez, F. Adrian and Hinners, E. Jennifer and Hudson, Lynn and Kennedy, Annie and Marchisotto, Jane Mary and Matrisian, Lynn and Myers, Elizabeth and Nowell, Benjamin W. and Nosek, A. Brian and Sherer, Todd and Shore, Carolyn and Sim, Ida and Smolensky, Luba and Williams, Christopher and Wood, Julie and Terry, F. Sharon", title="Data Sharing Goals for Nonprofit Funders of Clinical Trials", journal="J Participat Med", year="2021", month="Mar", day="29", volume="13", number="1", pages="e23011", keywords="clinical trial", keywords="biomedical research", keywords="data sharing", keywords="patients", doi="10.2196/23011", url="https://jopm.jmir.org/2021/1/e23011", url="http://www.ncbi.nlm.nih.gov/pubmed/33779573" } @Article{info:doi/10.2196/26718, author="Dron, Louis and Dillman, Alison and Zoratti, J. Michael and Haggstrom, Jonas and Mills, J. Edward and Park, H. Jay J.", title="Clinical Trial Data Sharing for COVID-19--Related Research", journal="J Med Internet Res", year="2021", month="Mar", day="12", volume="23", number="3", pages="e26718", keywords="COVID-19", keywords="data-sharing", keywords="clinical trials", keywords="data", keywords="research", keywords="privacy", keywords="security", keywords="registry", keywords="feasibility", keywords="challenge", keywords="recruitment", keywords="error", keywords="bias", keywords="assessment", keywords="interoperability", keywords="dataset", keywords="intervention", keywords="cooperation", doi="10.2196/26718", url="https://www.jmir.org/2021/3/e26718", url="http://www.ncbi.nlm.nih.gov/pubmed/33684053" } @Article{info:doi/10.2196/23703, author="Abd-Alrazaq, Alaa and Schneider, Jens and Mifsud, Borbala and Alam, Tanvir and Househ, Mowafa and Hamdi, Mounir and Shah, Zubair", title="A Comprehensive Overview of the COVID-19 Literature: Machine Learning--Based Bibliometric Analysis", journal="J Med Internet Res", year="2021", month="Mar", day="8", volume="23", number="3", pages="e23703", keywords="novel coronavirus disease", keywords="COVID-19", keywords="SARS-CoV-2", keywords="2019-nCoV", keywords="bibliometric analysis", keywords="literature", keywords="machine learning", keywords="research", keywords="review", abstract="Background: Shortly after the emergence of COVID-19, researchers rapidly mobilized to study numerous aspects of the disease such as its evolution, clinical manifestations, effects, treatments, and vaccinations. This led to a rapid increase in the number of COVID-19--related publications. Identifying trends and areas of interest using traditional review methods (eg, scoping and systematic reviews) for such a large domain area is challenging. Objective: We aimed to conduct an extensive bibliometric analysis to provide a comprehensive overview of the COVID-19 literature. Methods: We used the COVID-19 Open Research Dataset (CORD-19) that consists of a large number of research articles related to all coronaviruses. We used a machine learning--based method to analyze the most relevant COVID-19--related articles and extracted the most prominent topics. Specifically, we used a clustering algorithm to group published articles based on the similarity of their abstracts to identify research hotspots and current research directions. We have made our software accessible to the community via GitHub. Results: Of the 196,630 publications retrieved from the database, we included 28,904 in our analysis. The mean number of weekly publications was 990 (SD 789.3). The country that published the highest number of COVID-19--related articles was China (2950/17,270, 17.08\%). The highest number of articles were published in bioRxiv. Lei Liu affiliated with the Southern University of Science and Technology in China published the highest number of articles (n=46). Based on titles and abstracts alone, we were able to identify 1515 surveys, 733 systematic reviews, 512 cohort studies, 480 meta-analyses, and 362 randomized control trials. We identified 19 different topics covered among the publications reviewed. The most dominant topic was public health response, followed by clinical care practices during the COVID-19 pandemic, clinical characteristics and risk factors, and epidemic models for its spread. Conclusions: We provide an overview of the COVID-19 literature and have identified current hotspots and research directions. Our findings can be useful for the research community to help prioritize research needs and recognize leading COVID-19 researchers, institutes, countries, and publishers. Our study shows that an AI-based bibliometric analysis has the potential to rapidly explore a large corpus of academic publications during a public health crisis. We believe that this work can be used to analyze other eHealth-related literature to help clinicians, administrators, and policy makers to obtain a holistic view of the literature and be able to categorize different topics of the existing research for further analyses. It can be further scaled (for instance, in time) to clinical summary documentation. Publishers should avoid noise in the data by developing a way to trace the evolution of individual publications and unique authors. ", doi="10.2196/23703", url="https://www.jmir.org/2021/3/e23703", url="http://www.ncbi.nlm.nih.gov/pubmed/33600346" } @Article{info:doi/10.2196/25935, author="Bernardo, Theresa and Sobkowich, Edward Kurtis and Forrest, Othmer Russell and Stewart, Silva Luke and D'Agostino, Marcelo and Perez Gutierrez, Enrique and Gillis, Daniel", title="Collaborating in the Time of COVID-19: The Scope and Scale of Innovative Responses to a Global Pandemic", journal="JMIR Public Health Surveill", year="2021", month="Feb", day="9", volume="7", number="2", pages="e25935", keywords="crowdsourcing", keywords="artificial intelligence", keywords="collaboration", keywords="personal protective equipment", keywords="big data", keywords="AI", keywords="COVID-19", keywords="innovation", keywords="information sharing", keywords="communication", keywords="teamwork", keywords="knowledge", keywords="dissemination", doi="10.2196/25935", url="http://publichealth.jmir.org/2021/2/e25935/", url="http://www.ncbi.nlm.nih.gov/pubmed/33503001" } @Article{info:doi/10.2196/22327, author="Oska, Sandra and Lerma, Edgar and Topf, Joel", title="A Picture Is Worth a Thousand Views: A Triple Crossover Trial of Visual Abstracts to Examine Their Impact on Research Dissemination", journal="J Med Internet Res", year="2020", month="Dec", day="4", volume="22", number="12", pages="e22327", keywords="social media", keywords="science communication", keywords="visual abstract", keywords="Twitter", keywords="dissemination", abstract="Background: A visual abstract is a graphic summary of a research article's question, methods, and major findings. Although they have a number of uses, visual abstracts are chiefly used to promote research articles on social media. Objective: This study aimed to determine if the use of visual abstracts increases the visibility of nephrology research shared on Twitter. Methods: A prospective case-control crossover study was conducted using 40 research articles published in the American Journal of Nephrology (AJN). Each article was shared by the AJN Twitter account in 3 formats: (1) the article citation, (2) the citation with a key figure from the article, and (3) the citation with a visual abstract. Tweets were spaced 2 weeks apart to allow washout of the previous tweet, and the order of the tweets was randomized. Dissemination was measured via retweets, views, number of link clicks, and Altmetric scores. Results: Tweets that contained a visual abstract had more than twice as many views as citation-only tweets (1351, SD 1053 vs 639, SD 343) and nearly twice as many views as key figure tweets (1351, SD 1053 vs 732, SD 464). Visual abstract tweets had 5 times the engagements of citation-only tweets and more than 3.5 times the engagements of key figure tweets. Visual abstract tweets were also associated with greater increases in Altmetric scores as compared to citation-only tweets (2.20 vs 1.05). Conclusions: The use of visual abstracts increased visibility of research articles on Twitter, resulting in a greater number of views, engagements, and retweets. Visual abstracts were also associated with increased Altmetric scores as compared to citation-only tweets. These findings support the broader use of visual abstracts in the scientific community. Journals should consider visual abstracts as valuable tools for research dissemination. ", doi="10.2196/22327", url="https://www.jmir.org/2020/12/e22327", url="http://www.ncbi.nlm.nih.gov/pubmed/33275112" } @Article{info:doi/10.2196/21559, author="{\"A}lg{\aa}, Andreas and Eriksson, Oskar and Nordberg, Martin", title="Analysis of Scientific Publications During the Early Phase of the COVID-19 Pandemic: Topic Modeling Study", journal="J Med Internet Res", year="2020", month="Nov", day="10", volume="22", number="11", pages="e21559", keywords="COVID-19", keywords="SARS-CoV-2", keywords="coronavirus", keywords="pandemic", keywords="topic modeling", keywords="research", keywords="literature", abstract="Background: The COVID-19 pandemic has spread at an alarming speed, and effective treatment for the disease is still lacking. The body of evidence on COVID-19 has been increasing at an impressive pace, creating the need for a method to rapidly assess the current knowledge and identify key information. Gold standard methods such as systematic reviews and meta-analyses are regarded unsuitable because they have a narrow scope and are very time consuming. Objective: This study aimed to explore the published scientific literature on COVID-19 and map the research evolution during the early phase of the COVID-19 pandemic. Methods: We performed a PubMed search to analyze the titles, keywords, and abstracts of published papers on COVID-19. We used latent Dirichlet allocation modeling to extract topics and conducted a trend analysis to understand the temporal changes in research for each topic, journal impact factor (JIF), and geographic origin. Results: Based on our search, we identified 16,670 relevant articles dated between February 14, 2020, and June 1, 2020. Of these, 6 articles were reports from peer-reviewed randomized trials on patients with COVID-19. We identified 14 main research topics, of which the most common topics were health care responses (2812/16,670, 16.86\%) and clinical manifestations (1828/16,670, 10.91\%). We found an increasing trend for research on clinical manifestations and protective measures and a decreasing trend for research on disease transmission, epidemiology, health care response, and radiology. Publications on protective measures, immunology, and clinical manifestations were associated with the highest JIF. The overall median JIF was 3.7 (IQR 2.6-5.9), and we found that the JIF for these publications declined over time. The top countries producing research were the United States, China, Italy, and the United Kingdom. Conclusions: In less than 6 months since the novel coronavirus was first detected, a remarkably high number of research articles on COVID-19 have been published. Here, we discuss and present the temporal changes in the available COVID-19 research during the early phase of the pandemic. Our findings may aid researchers and policy makers to form a structured view of the current COVID-19 evidence base and provide further research directions. ", doi="10.2196/21559", url="http://www.jmir.org/2020/11/e21559/", url="http://www.ncbi.nlm.nih.gov/pubmed/33031049" } @Article{info:doi/10.2196/21648, author="Khan, Younus Junaed and Khondaker, Islam Md Tawkat and Hoque, Tazim Iram and Al-Absi, H. Hamada R. and Rahman, Saifur Mohammad and Guler, Reto and Alam, Tanvir and Rahman, Sohel M.", title="Toward Preparing a Knowledge Base to Explore Potential Drugs and Biomedical Entities Related to COVID-19: Automated Computational Approach", journal="JMIR Med Inform", year="2020", month="Nov", day="10", volume="8", number="11", pages="e21648", keywords="COVID-19", keywords="2019-nCoV", keywords="coronavirus", keywords="SARS-CoV-2", keywords="SARS", keywords="remdesivir", keywords="statin", keywords="statins", keywords="dexamethasone", keywords="ivermectin", keywords="hydroxychloroquine", abstract="Background: Novel coronavirus disease 2019 (COVID-19) is taking a huge toll on public health. Along with the non-therapeutic preventive measurements, scientific efforts are currently focused, mainly, on the development of vaccines and pharmacological treatment with existing drugs. Summarizing evidences from scientific literatures on the discovery of treatment plan of COVID-19 under a platform would help the scientific community to explore the opportunities in a systematic fashion. Objective: The aim of this study is to explore the potential drugs and biomedical entities related to coronavirus related diseases, including COVID-19, that are mentioned on scientific literature through an automated computational approach. Methods: We mined the information from publicly available scientific literature and related public resources. Six topic-specific dictionaries, including human genes, human miRNAs, diseases, Protein Databank, drugs, and drug side effects, were integrated to mine all scientific evidence related to COVID-19. We employed an automated literature mining and labeling system through a novel approach to measure the effectiveness of drugs against diseases based on natural language processing, sentiment analysis, and deep learning. We also applied the concept of cosine similarity to confidently infer the associations between diseases and genes. Results: Based on the literature mining, we identified 1805 diseases, 2454 drugs, 1910 genes that are related to coronavirus related diseases including COVID-19. Integrating the extracted information, we developed the first knowledgebase platform dedicated to COVID-19, which highlights potential list of drugs and related biomedical entities. For COVID-19, we highlighted multiple case studies on existing drugs along with a confidence score for their applicability in the treatment plan. Based on our computational method, we found Remdesivir, Statins, Dexamethasone, and Ivermectin could be considered as potential effective drugs to improve clinical status and lower mortality in patients hospitalized with COVID-19. We also found that Hydroxychloroquine could not be considered as an effective drug for COVID-19. The resulting knowledgebase is made available as an open source tool, named COVID-19Base. Conclusions: Proper investigation of the mined biomedical entities along with the identified interactions among those would help the research community to discover possible ways for the therapeutic treatment of COVID-19. ", doi="10.2196/21648", url="http://medinform.jmir.org/2020/11/e21648/", url="http://www.ncbi.nlm.nih.gov/pubmed/33055059" } @Article{info:doi/10.2196/21169, author="Gates, Elaine Lyndsey and Hamed, Abdeen Ahmed", title="The Anatomy of the SARS-CoV-2 Biomedical Literature: Introducing the CovidX Network Algorithm for Drug Repurposing Recommendation", journal="J Med Internet Res", year="2020", month="Aug", day="20", volume="22", number="8", pages="e21169", keywords="health", keywords="informatics", keywords="COVID-19 treatment", keywords="drug repurposing", keywords="network algorithm", keywords="ranking", keywords="drug", keywords="biomedical", keywords="antiviral", keywords="COVID-19", abstract="Background: Driven by the COVID-19 pandemic and the dire need to discover an antiviral drug, we explored the landscape of the SARS-CoV-2 biomedical publications to identify potential treatments. Objective: The aims of this study are to identify off-label drugs that may have benefits for the coronavirus disease pandemic, present a novel ranking algorithm called CovidX to recommend existing drugs for potential repurposing, and validate the literature-based outcome with drug knowledge available in clinical trials. Methods: To achieve such objectives, we applied natural language processing techniques to identify drugs and linked entities (eg, disease, gene, protein, chemical compounds). When such entities are linked, they form a map that can be further explored using network science tools. The CovidX algorithm was based upon a notion that we called ``diversity.'' A diversity score for a given drug was calculated by measuring how ``diverse'' a drug is calculated using various biological entities (regardless of the cardinality of actual instances in each category). The algorithm validates the ranking and awards those drugs that are currently being investigated in open clinical trials. The rationale behind the open clinical trial is to provide a validating mechanism of the PubMed results. This ensures providing up to date evidence of the fast development of this disease. Results: From the analyzed biomedical literature, the algorithm identified 30 possible drug candidates for repurposing, ranked them accordingly, and validated the ranking outcomes against evidence from clinical trials. The top 10 candidates according to our algorithm are hydroxychloroquine, azithromycin, chloroquine, ritonavir, losartan, remdesivir, favipiravir, methylprednisolone, rapamycin, and tilorone dihydrochloride. Conclusions: The ranking shows both consistency and promise in identifying drugs that can be repurposed. We believe, however, the full treatment to be a multifaceted, adjuvant approach where multiple drugs may need to be taken at the same time. ", doi="10.2196/21169", url="http://www.jmir.org/2020/8/e21169/", url="http://www.ncbi.nlm.nih.gov/pubmed/32735546" } @Article{info:doi/10.2196/20007, author="Michelson, Matthew and Chow, Tiffany and Martin, A. Neil and Ross, Mike and Tee Qiao Ying, Amelia and Minton, Steven", title="Artificial Intelligence for Rapid Meta-Analysis: Case Study on Ocular Toxicity of Hydroxychloroquine", journal="J Med Internet Res", year="2020", month="Aug", day="17", volume="22", number="8", pages="e20007", keywords="meta-analysis", keywords="rapid meta-analysis", keywords="artificial intelligence", keywords="drug", keywords="analysis", keywords="hydroxychloroquine", keywords="toxic", keywords="COVID-19", keywords="treatment", keywords="side effect", keywords="ocular", keywords="eye", abstract="Background: Rapid access to evidence is crucial in times of an evolving clinical crisis. To that end, we propose a novel approach to answer clinical queries, termed rapid meta-analysis (RMA). Unlike traditional meta-analysis, RMA balances a quick time to production with reasonable data quality assurances, leveraging artificial intelligence (AI) to strike this balance. Objective: We aimed to evaluate whether RMA can generate meaningful clinical insights, but crucially, in a much faster processing time than traditional meta-analysis, using a relevant, real-world example. Methods: The development of our RMA approach was motivated by a currently relevant clinical question: is ocular toxicity and vision compromise a side effect of hydroxychloroquine therapy? At the time of designing this study, hydroxychloroquine was a leading candidate in the treatment of coronavirus disease (COVID-19). We then leveraged AI to pull and screen articles, automatically extract their results, review the studies, and analyze the data with standard statistical methods. Results: By combining AI with human analysis in our RMA, we generated a meaningful, clinical result in less than 30 minutes. The RMA identified 11 studies considering ocular toxicity as a side effect of hydroxychloroquine and estimated the incidence to be 3.4\% (95\% CI 1.11\%-9.96\%). The heterogeneity across individual study findings was high, which should be taken into account in interpretation of the result. Conclusions: We demonstrate that a novel approach to meta-analysis using AI can generate meaningful clinical insights in a much shorter time period than traditional meta-analysis. ", doi="10.2196/20007", url="http://www.jmir.org/2020/8/e20007/", url="http://www.ncbi.nlm.nih.gov/pubmed/32804086" } @Article{info:doi/10.2196/18087, author="Suver, Christine and Thorogood, Adrian and Doerr, Megan and Wilbanks, John and Knoppers, Bartha", title="Bringing Code to Data: Do Not Forget Governance", journal="J Med Internet Res", year="2020", month="Jul", day="28", volume="22", number="7", pages="e18087", keywords="data management", keywords="privacy", keywords="ethics, research", keywords="data science", keywords="machine learning", doi="10.2196/18087", url="http://www.jmir.org/2020/7/e18087/", url="http://www.ncbi.nlm.nih.gov/pubmed/32540846" } @Article{info:doi/10.2196/18212, author="Peng, Cheng and He, Miao and Cutrona, L. Sarah and Kiefe, I. Catarina and Liu, Feifan and Wang, Zhongqing", title="Theme Trends and Knowledge Structure on Mobile Health Apps: Bibliometric Analysis", journal="JMIR Mhealth Uhealth", year="2020", month="Jul", day="27", volume="8", number="7", pages="e18212", keywords="mobile app", keywords="mobile health", keywords="mhealth", keywords="digital health", keywords="digital medicine", keywords="bibliometrics", keywords="co-word analysis", keywords="mobile phone", keywords="VOSviewer", abstract="Background: Due to the widespread and unprecedented popularity of mobile phones, the use of digital medicine and mobile health apps has seen significant growth. Mobile health apps have tremendous potential for monitoring and treating diseases, improving patient care, and promoting health. Objective: This paper aims to explore research trends, coauthorship networks, and the research hot spots of mobile health app research. Methods: Publications related to mobile health apps were retrieved and extracted from the Web of Science database with no language restrictions. Bibliographic Item Co-Occurrence Matrix Builder was employed to extract bibliographic information (publication year and journal source) and perform a descriptive analysis. We then used the VOSviewer (Leiden University) tool to construct and visualize the co-occurrence networks of researchers, research institutions, countries/regions, citations, and keywords. Results: We retrieved 2802 research papers on mobile health apps published from 2000 to 2019. The number of annual publications increased over the past 19 years. JMIR mHealth and uHealth (323/2802, 11.53\%), Journal of Medical Internet Research (106/2802, 3.78\%), and JMIR Research Protocols (82/2802, 2.93\%) were the most common journals for these publications. The United States (1186/2802, 42.33\%), England (235/2802, 8.39\%), Australia (215/2802, 7.67\%), and Canada (112/2802, 4.00\%) were the most productive countries of origin. The University of California San Francisco, the University of Washington, and the University of Toronto were the most productive institutions. As for the authors' contributions, Schnall R, Kuhn E, Lopez-Coronado M, and Kim J were the most active researchers. The co-occurrence cluster analysis of the top 100 keywords forms 5 clusters: (1) the technology and system development of mobile health apps; (2) mobile health apps for mental health; (3) mobile health apps in telemedicine, chronic disease, and medication adherence management; (4) mobile health apps in health behavior and health promotion; and (5) mobile health apps in disease prevention via the internet. Conclusions: We summarize the recent advances in mobile health app research and shed light on their research frontier, trends, and hot topics through bibliometric analysis and network visualization. These findings may provide valuable guidance on future research directions and perspectives in this rapidly developing field. ", doi="10.2196/18212", url="https://mhealth.jmir.org/2020/7/e18212", url="http://www.ncbi.nlm.nih.gov/pubmed/32716312" } @Article{info:doi/10.2196/15607, author="Bardus, Marco and El Rassi, Rola and Chahrour, Mohamad and Akl, W. Elie and Raslan, Sattar Abdul and Meho, I. Lokman and Akl, A. Elie", title="The Use of Social Media to Increase the Impact of Health Research: Systematic Review", journal="J Med Internet Res", year="2020", month="Jul", day="6", volume="22", number="7", pages="e15607", keywords="social media", keywords="research", keywords="bibliometrics", keywords="Altmetrics", keywords="journal impact factor", keywords="translational medical research", abstract="Background: Academics in all disciplines increasingly use social media to share their publications on the internet, reaching out to different audiences. In the last few years, specific indicators of social media impact have been developed (eg, Altmetrics), to complement traditional bibliometric indicators (eg, citation count and h-index). In health research, it is unclear whether social media impact also translates into research impact. Objective: The primary aim of this study was to systematically review the literature on the impact of using social media on the dissemination of health research. The secondary aim was to assess the correlation between Altmetrics and traditional citation-based metrics. Methods: We conducted a systematic review to identify studies that evaluated the use of social media to disseminate research published in health-related journals. We specifically looked at studies that described experimental or correlational studies linking the use of social media with outcomes related to bibliometrics. We searched the Medical Literature Analysis and Retrieval System Online (MEDLINE), Excerpta Medica dataBASE (EMBASE), and Cumulative Index to Nursing and Allied Health Literature (CINAHL) databases using a predefined search strategy (International Prospective Register of Systematic Reviews: CRD42017057709). We conducted independent and duplicate study selection and data extraction. Given the heterogeneity of the included studies, we summarized the findings through a narrative synthesis. Results: Of a total of 18,624 retrieved citations, we included 51 studies: 7 (14\%) impact studies (answering the primary aim) and 44 (86\%) correlational studies (answering the secondary aim). Impact studies reported mixed results with several limitations, including the use of interventions of inappropriately low intensity and short duration. The majority of correlational studies suggested a positive association between traditional bibliometrics and social media metrics (eg, number of mentions) in health research. Conclusions: We have identified suggestive yet inconclusive evidence on the impact of using social media to increase the number of citations in health research. Further studies with better design are needed to assess the causal link between social media impact and bibliometrics. ", doi="10.2196/15607", url="https://www.jmir.org/2020/7/e15607", url="http://www.ncbi.nlm.nih.gov/pubmed/32628113" } @Article{info:doi/10.2196/19170, author="Mavian, Carla and Marini, Simone and Prosperi, Mattia and Salemi, Marco", title="A Snapshot of SARS-CoV-2 Genome Availability up to April 2020 and its Implications: Data Analysis", journal="JMIR Public Health Surveill", year="2020", month="Jun", day="1", volume="6", number="2", pages="e19170", keywords="covid-19", keywords="sars-cov-2", keywords="phylogenetics", keywords="genome", keywords="evolution", keywords="genetics", keywords="pandemic", keywords="infectious disease", keywords="virus", keywords="sequence", keywords="transmission", keywords="tracing", keywords="tracking", abstract="Background: The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic has been growing exponentially, affecting over 4 million people and causing enormous distress to economies and societies worldwide. A plethora of analyses based on viral sequences has already been published both in scientific journals and through non--peer-reviewed channels to investigate the genetic heterogeneity and spatiotemporal dissemination of SARS-CoV-2. However, a systematic investigation of phylogenetic information and sampling bias in the available data is lacking. Although the number of available genome sequences of SARS-CoV-2 is growing daily and the sequences show increasing phylogenetic information, country-specific data still present severe limitations and should be interpreted with caution. Objective: The objective of this study was to determine the quality of the currently available SARS-CoV-2 full genome data in terms of sampling bias as well as phylogenetic and temporal signals to inform and guide the scientific community. Methods: We used maximum likelihood--based methods to assess the presence of sufficient information for robust phylogenetic and phylogeographic studies in several SARS-CoV-2 sequence alignments assembled from GISAID (Global Initiative on Sharing All Influenza Data) data released between March and April 2020. Results: Although the number of high-quality full genomes is growing daily, and sequence data released in April 2020 contain sufficient phylogenetic information to allow reliable inference of phylogenetic relationships, country-specific SARS-CoV-2 data sets still present severe limitations. Conclusions: At the present time, studies assessing within-country spread or transmission clusters should be considered preliminary or hypothesis-generating at best. Hence, current reports should be interpreted with caution, and concerted efforts should continue to increase the number and quality of sequences required for robust tracing of the epidemic. ", doi="10.2196/19170", url="http://publichealth.jmir.org/2020/2/e19170/", url="http://www.ncbi.nlm.nih.gov/pubmed/32412415" } @Article{info:doi/10.2196/11567, author="Kan, Wei-Chih and Chou, Willy and Chien, Tsair-Wei and Yeh, Yu-Tsen and Chou, Po-Hsin", title="The Most-Cited Authors Who Published Papers in JMIR mHealth and uHealth Using the Authorship-Weighted Scheme: Bibliometric Analysis", journal="JMIR Mhealth Uhealth", year="2020", month="May", day="7", volume="8", number="5", pages="e11567", keywords="betweenness centrality", keywords="authorship collaboration", keywords="Google Maps", keywords="social network analysis", keywords="knowledge concept map", keywords="the author-weighted scheme", abstract="Background: Many previous papers have investigated most-cited articles or most productive authors in academics, but few have studied most-cited authors. Two challenges are faced in doing so, one of which is that some different authors will have the same name in the bibliometric data, and the second is that coauthors' contributions are different in the article byline. No study has dealt with the matter of duplicate names in bibliometric data. Although betweenness centrality (BC) is one of the most popular degrees of density in social network analysis (SNA), few have applied the BC algorithm to interpret a network's characteristics. A quantitative scheme must be used for calculating weighted author credits and then applying the metrics in comparison. Objective: This study aimed to apply the BC algorithm to examine possible identical names in a network and report the most-cited authors for a journal related to international mobile health (mHealth) research. Methods: We obtained 676 abstracts from Medline based on the keywords ``JMIR mHealth and uHealth'' (Journal) on June 30, 2018. The author names, countries/areas, and author-defined keywords were recorded. The BCs were then calculated for the following: (1) the most-cited authors displayed on Google Maps; (2) the geographical distribution of countries/areas for the first author; and (3) the keywords dispersed by BC and related to article topics in comparison on citation indices. Pajek software was used to yield the BC for each entity (or node). Bibliometric indices, including h-, g-, and x-indexes, the mean of core articles on g(Ag)=sum (citations on g-core/publications on g-core), and author impact factor (AIF), were applied. Results: We found that the most-cited author was Sherif M Badawy (from the United States), who had published six articles on JMIR mHealth and uHealth with high bibliometric indices (h=3; AIF=8.47; x=4.68; Ag=5.26). We also found that the two countries with the highest BC were the United States and the United Kingdom and that the two keyword clusters of mHealth and telemedicine earned the highest indices in comparison to other counterparts. All visual representations were successfully displayed on Google Maps. Conclusions: The most cited authors were selected using the authorship-weighted scheme (AWS), and the keywords of mHealth and telemedicine were more highly cited than other counterparts. The results on Google Maps are novel and unique as knowledge concept maps for understanding the feature of a journal. The research approaches used in this study (ie, BC and AWS) can be applied to other bibliometric analyses in the future. ", doi="10.2196/11567", url="https://mhealth.jmir.org/2020/5/e11567", url="http://www.ncbi.nlm.nih.gov/pubmed/32379053" } @Article{info:doi/10.2196/16606, author="Schoeb, Dominik and Suarez-Ibarrola, Rodrigo and Hein, Simon and Dressler, Friedrich Franz and Adams, Fabian and Schlager, Daniel and Miernik, Arkadiusz", title="Use of Artificial Intelligence for Medical Literature Search: Randomized Controlled Trial Using the Hackathon Format", journal="Interact J Med Res", year="2020", month="Mar", day="30", volume="9", number="1", pages="e16606", keywords="artificial intelligence", keywords="literature review", keywords="medical information technology", abstract="Background: Mapping out the research landscape around a project is often time consuming and difficult. Objective: This study evaluates a commercial artificial intelligence (AI) search engine (IRIS.AI) for its applicability in an automated literature search on a specific medical topic. Methods: To evaluate the AI search engine in a standardized manner, the concept of a science hackathon was applied. Three groups of researchers were tasked with performing a literature search on a clearly defined scientific project. All participants had a high level of expertise for this specific field of research. Two groups were given access to the AI search engine IRIS.AI. All groups were given the same amount of time for their search and were instructed to document their results. Search results were summarized and ranked according to a predetermined scoring system. Results: The final scoring awarded 49 and 39 points out of 60 to AI groups 1 and 2, respectively, and the control group received 46 points. A total of 20 scientific studies with high relevance were identified, and 5 highly relevant studies (``spot on'') were reported by each group. Conclusions: AI technology is a promising approach to facilitate literature searches and the management of medical libraries. In this study, however, the application of AI technology lead to a more focused literature search without a significant improvement in the number of results. ", doi="10.2196/16606", url="http://www.i-jmr.org/2020/1/e16606/", url="http://www.ncbi.nlm.nih.gov/pubmed/32224481" } @Article{info:doi/10.2196/16102, author="Grundstrom, Casandra and Korhonen, Olli and V{\"a}yrynen, Karin and Isomursu, Minna", title="Insurance Customers' Expectations for Sharing Health Data: Qualitative Survey Study", journal="JMIR Med Inform", year="2020", month="Mar", day="26", volume="8", number="3", pages="e16102", keywords="data sharing", keywords="qualitative research", keywords="survey", keywords="health insurance", keywords="insurance", keywords="medical informatics", keywords="health services", abstract="Background: Insurance organizations are essential stakeholders in health care ecosystems. For addressing future health care needs, insurance companies require access to health data to deliver preventative and proactive digital health services to customers. However, extant research is limited in examining the conditions that incentivize health data sharing. Objective: This study aimed to (1) identify the expectations of insurance customers when sharing health data, (2) determine the perceived intrinsic value of health data, and (3) explore the conditions that aid in incentivizing health data sharing in the relationship between an insurance organization and its customer. Methods: A Web-based survey was distributed to randomly selected customers from a Finnish insurance organization through email. A single open-text answer was used for a qualitative data analysis through inductive coding, followed by a thematic analysis. Furthermore, the 4 constructs of commitment, power, reciprocity, and trust from the social exchange theory (SET) were applied as a framework. Results: From the 5000 customers invited to participate, we received 452 surveys (response rate: 9.0\%). Customer characteristics were found to reflect customer demographics. Of the 452 surveys, 48 (10.6\%) open-text responses were skipped by the customer, 57 (12.6\%) customers had no expectations from sharing health data, and 44 (9.7\%) customers preferred to abstain from a data sharing relationship. Using the SET framework, we found that customers expected different conditions to be fulfilled by their insurance provider based on the commitment, power, reciprocity, and trust constructs. Of the 452 customers who completed the surveys, 64 (14.2\%) customers required that the insurance organization meets their data treatment expectations (commitment). Overall, 4.9\% (22/452) of customers were concerned about their health data being used against them to profile their health, to increase insurance prices, or to deny health insurance claims (power). A total of 28.5\% (129/452) of customers expected some form of benefit, such as personalized digital health services, and 29.9\% (135/452) of customers expected finance-related compensation (reciprocity). Furthermore, 7.5\% (34/452) of customers expected some form of empathy from the insurance organization through enhanced transparency or an emotional connection (trust). Conclusions: To aid in the design and development of digital health services, insurance organizations need to address the customers' expectations when sharing their health data. We established the expectations of customers in the social exchange of health data and explored the perceived values of data as intangible goods. Actions by the insurance organization should aim to increase trust through a culture of transparency, commitment to treat health data in a prescribed manner, provide reciprocal benefits through digital health services that customers deem valuable, and assuage fears of health data being used to prevent providing insurance coverage or increase costs. ", doi="10.2196/16102", url="http://medinform.jmir.org/2020/3/e16102/", url="http://www.ncbi.nlm.nih.gov/pubmed/32213467" } @Article{info:doi/10.2196/16810, author="Glicksberg, Scott Benjamin and Burns, Shohei and Currie, Rob and Griffin, Ann and Wang, Jane Zhen and Haussler, David and Goldstein, Theodore and Collisson, Eric", title="Blockchain-Authenticated Sharing of Genomic and Clinical Outcomes Data of Patients With Cancer: A Prospective Cohort Study", journal="J Med Internet Res", year="2020", month="Mar", day="20", volume="22", number="3", pages="e16810", keywords="data sharing", keywords="electronic health records", keywords="genomics", keywords="medicine", keywords="blockchain", keywords="neoplasms", abstract="Background: Efficiently sharing health data produced during standard care could dramatically accelerate progress in cancer treatments, but various barriers make this difficult. Not sharing these data to ensure patient privacy is at the cost of little to no learning from real-world data produced during cancer care. Furthermore, recent research has demonstrated a willingness of patients with cancer to share their treatment experiences to fuel research, despite potential risks to privacy. Objective: The objective of this study was to design, pilot, and release a decentralized, scalable, efficient, economical, and secure strategy for the dissemination of deidentified clinical and genomic data with a focus on late-stage cancer. Methods: We created and piloted a blockchain-authenticated system to enable secure sharing of deidentified patient data derived from standard of care imaging, genomic testing, and electronic health records (EHRs), called the Cancer Gene Trust (CGT). We prospectively consented and collected data for a pilot cohort (N=18), which we uploaded to the CGT. EHR data were extracted from both a hospital cancer registry and a common data model (CDM) format to identify optimal data extraction and dissemination practices. Specifically, we scored and compared the level of completeness between two EHR data extraction formats against the gold standard source documentation for patients with available data (n=17). Results: Although the total completeness scores were greater for the registry reports than those for the CDM, this difference was not statistically significant. We did find that some specific data fields, such as histology site, were better captured using the registry reports, which can be used to improve the continually adapting CDM. In terms of the overall pilot study, we found that CGT enables rapid integration of real-world data of patients with cancer in a more clinically useful time frame. We also developed an open-source Web application to allow users to seamlessly search, browse, explore, and download CGT data. Conclusions: Our pilot demonstrates the willingness of patients with cancer to participate in data sharing and how blockchain-enabled structures can maintain relationships between individual data elements while preserving patient privacy, empowering findings by third-party researchers and clinicians. We demonstrate the feasibility of CGT as a framework to share health data trapped in silos to further cancer research. Further studies to optimize data representation, stream, and integrity are required. ", doi="10.2196/16810", url="http://www.jmir.org/2020/3/e16810/", url="http://www.ncbi.nlm.nih.gov/pubmed/32196460" } @Article{info:doi/10.2196/15603, author="Bacon, Seb and Goldacre, Ben", title="Barriers to Working With National Health Service England's Open Data", journal="J Med Internet Res", year="2020", month="Jan", day="13", volume="22", number="1", pages="e15603", keywords="informatics", keywords="health services", keywords="software", keywords="access to information", doi="10.2196/15603", url="https://www.jmir.org/2020/1/e15603", url="http://www.ncbi.nlm.nih.gov/pubmed/31929101" } @Article{info:doi/10.2196/17578, author="Eysenbach, Gunther", title="Celebrating 20 Years of Open Access and Innovation at JMIR Publications", journal="J Med Internet Res", year="2019", month="Dec", day="23", volume="21", number="12", pages="e17578", keywords="JMIR", keywords="internet", keywords="medical informatics", keywords="ehealth", keywords="digital health", keywords="participatory medicine", keywords="open access", keywords="electronic publishing", keywords="scholarly publishing", keywords="science communication", keywords="journalogy", keywords="history of science", keywords="overlay journal", keywords="preprints", keywords="open science", doi="10.2196/17578", url="http://www.jmir.org/2019/12/e17578/", url="http://www.ncbi.nlm.nih.gov/pubmed/31868653" } @Article{info:doi/10.2196/16532, author="Wyatt, C. Jeremy", title="Preserving the Open Access Benefits Pioneered by the Journal of Medical Internet Research and Discouraging Fraudulent Journals", journal="J Med Internet Res", year="2019", month="Dec", day="23", volume="21", number="12", pages="e16532", keywords="open access", keywords="predatory journals", keywords="knowledge management", keywords="scientific journals", keywords="mobilizing computable knowledge", keywords="fraudulent journals", doi="10.2196/16532", url="http://www.jmir.org/2019/12/e16532/" } @Article{info:doi/10.2196/16078, author="Anderson, Michael J. and Niemann, Andrew and Johnson, L. Austin and Cook, Courtney and Tritz, Daniel and Vassar, Matt", title="Transparent, Reproducible, and Open Science Practices of Published Literature in Dermatology Journals: Cross-Sectional Analysis", journal="JMIR Dermatol", year="2019", month="Nov", day="7", volume="2", number="1", pages="e16078", keywords="reproducibility of findings", keywords="data sharing", keywords="publishing, open access", keywords="dermatology", abstract="Background: Reproducible research is a foundational component for scientific advancements, yet little is known regarding the extent of reproducible research within the dermatology literature. Objective: This study aimed to determine the quality and transparency of the literature in dermatology journals by evaluating for the presence of 8 indicators of reproducible and transparent research practices. Methods: By implementing a cross-sectional study design, we conducted an advanced search of publications in dermatology journals from the National Library of Medicine catalog. Our search included articles published between January 1, 2014, and December 31, 2018. After generating a list of eligible dermatology publications, we then searched for full text PDF versions by using Open Access Button, Google Scholar, and PubMed. Publications were analyzed for 8 indicators of reproducibility and transparency---availability of materials, data, analysis scripts, protocol, preregistration, conflict of interest statement, funding statement, and open access---using a pilot-tested Google Form. Results: After exclusion, 127 studies with empirical data were included in our analysis. Certain indicators were more poorly reported than others. We found that most publications (113, 88.9\%) did not provide unmodified, raw data used to make computations, 124 (97.6\%) failed to make the complete protocol available, and 126 (99.2\%) did not include step-by-step analysis scripts. Conclusions: Our sample of studies published in dermatology journals do not appear to include sufficient detail to be accurately and successfully reproduced in their entirety. Solutions to increase the quality, reproducibility, and transparency of dermatology research are warranted. More robust reporting of key methodological details, open data sharing, and stricter standards journals impose on authors regarding disclosure of study materials might help to better the climate of reproducible research in dermatology. ", doi="10.2196/16078", url="http://derma.jmir.org/2019/1/e16078/" } @Article{info:doi/10.2196/10131, author="Boonstra, W. Tjeerd and Nicholas, Jennifer and Wong, JJ Quincy and Shaw, Frances and Townsend, Samuel and Christensen, Helen", title="Using Mobile Phone Sensor Technology for Mental Health Research: Integrated Analysis to Identify Hidden Challenges and Potential Solutions", journal="J Med Internet Res", year="2018", month="Jul", day="30", volume="20", number="7", pages="e10131", keywords="passive sensing", keywords="mental health", keywords="ubiquitous computing", keywords="ethics", keywords="depression", keywords="mobile health", keywords="smartphone", keywords="wearable sensors", abstract="Background: Mobile phone sensor technology has great potential in providing behavioral markers of mental health. However, this promise has not yet been brought to fruition. Objective: The objective of our study was to examine challenges involved in developing an app to extract behavioral markers of mental health from passive sensor data. Methods: Both technical challenges and acceptability of passive data collection for mental health research were assessed based on literature review and results obtained from a feasibility study. Socialise, a mobile phone app developed at the Black Dog Institute, was used to collect sensor data (Bluetooth, location, and battery status) and investigate views and experiences of a group of people with lived experience of mental health challenges (N=32). Results: On average, sensor data were obtained for 55\% (Android) and 45\% (iOS) of scheduled scans. Battery life was reduced from 21.3 hours to 18.8 hours when scanning every 5 minutes with a reduction of 2.5 hours or 12\%. Despite this relatively small reduction, most participants reported that the app had a noticeable effect on their battery life. In addition to battery life, the purpose of data collection, trust in the organization that collects data, and perceived impact on privacy were identified as main factors for acceptability. Conclusions: Based on the findings of the feasibility study and literature review, we recommend a commitment to open science and transparent reporting and stronger partnerships and communication with users. Sensing technology has the potential to greatly enhance the delivery and impact of mental health care. Realizing this requires all aspects of mobile phone sensor technology to be rigorously assessed. ", doi="10.2196/10131", url="http://www.jmir.org/2018/7/e10131/", url="http://www.ncbi.nlm.nih.gov/pubmed/30061092" } @Article{info:doi/10.2196/jmir.7513, author="Andriesen, Jessica and Bull, Sheana and Dietrich, Janan and Haberer, E. Jessica and Van Der Pol, Barbara and Voronin, Yegor and Wall, M. Kristin and Whalen, Christopher and Priddy, Frances", title="Using Digital Technologies in Clinical HIV Research: Real-World Applications and Considerations for Future Work", journal="J Med Internet Res", year="2017", month="Jul", day="31", volume="19", number="7", pages="e274", keywords="clinical trial", keywords="HIV", keywords="mobile phone", keywords="text messaging", keywords="biometric identification", keywords="observational study privacy", keywords="data collection", abstract="Background: Digital technologies, especially if used in novel ways, provide a number of potential advantages to clinical research in trials related to human immunodeficiency virus (HIV) and acquired immune deficiency syndrome (AIDS) and may greatly facilitate operations as well as data collection and analysis. These technologies may even allow answering questions that are not answerable with older technologies. However, they come with a variety of potential concerns for both the participants and the trial sponsors. The exact challenges and means for alleviation depend on the technology and on the population in which it is deployed, and the rapidly changing landscape of digital technologies presents a challenge for creating future-proof guidelines for technology application. Objective: The aim of this study was to identify and summarize some common themes that are frequently encountered by researchers in this context and highlight those that should be carefully considered before making a decision to include these technologies in their research. Methods: In April 2016, the Global HIV Vaccine Enterprise surveyed the field for research groups with recent experience in novel applications of digital technologies in HIV clinical research and convened these groups for a 1-day meeting. Real-world uses of various technologies were presented and discussed by 46 attendees, most of whom were researchers involved in the design and conduct of clinical trials of biomedical HIV prevention and treatment approaches. After the meeting, a small group of organizers reviewed the presentations and feedback obtained during the meeting and categorized various lessons-learned to identify common themes. A group of 9 experts developed a draft summary of the findings that was circulated via email to all 46 attendees for review. Taking into account the feedback received, the group finalized the considerations that are presented here. Results: Meeting presenters and attendees discussed the many successful applications of digital technologies to improve research outcomes, such as those for recruitment and enrollment, participant identification, informed consent, data collection, data quality, and protocol or treatment adherence. These discussions also revealed unintended consequence of technology usage, including risks to study participants and risks to study integrity. Conclusions: Key lessons learned from these discussions included the need to thoroughly evaluate systems to be used, the idea that early success may not be sustained throughout the study, that some failures will occur, and considerations for study-provided devices. Additionally, taking these key lessons into account, the group generated recommendations on how to move forward with the use of technology in HIV vaccine and biomedical prevention trials. ", doi="10.2196/jmir.7513", url="http://www.jmir.org/2017/7/e274/", url="http://www.ncbi.nlm.nih.gov/pubmed/28760729" } @Article{info:doi/10.2196/jmir.6937, author="Payne, Philip and Lele, Omkar and Johnson, Beth and Holve, Erin", title="Enabling Open Science for Health Research: Collaborative Informatics Environment for Learning on Health Outcomes (CIELO)", journal="J Med Internet Res", year="2017", month="Jul", day="31", volume="19", number="7", pages="e276", keywords="healthcare research", keywords="information dissemination", keywords="open access to information", keywords="social networking", keywords="reproducibility of results", abstract="Background: There is an emergent and intensive dialogue in the United States with regard to the accessibility, reproducibility, and rigor of health research. This discussion is also closely aligned with the need to identify sustainable ways to expand the national research enterprise and to generate actionable results that can be applied to improve the nation's health. The principles and practices of Open Science offer a promising path to address both goals by facilitating (1) increased transparency of data and methods, which promotes research reproducibility and rigor; and (2) cumulative efficiencies wherein research tools and the output of research are combined to accelerate the delivery of new knowledge in proximal domains, thereby resulting in greater productivity and a reduction in redundant research investments. Objectives: AcademyHealth's Electronic Data Methods (EDM) Forum implemented a proof-of-concept open science platform for health research called the Collaborative Informatics Environment for Learning on Health Outcomes (CIELO). Methods: The EDM Forum conducted a user-centered design process to elucidate important and high-level requirements for creating and sustaining an open science paradigm. Results: By implementing CIELO and engaging a variety of potential users in its public beta testing, the EDM Forum has been able to elucidate a broad range of stakeholder needs and requirements related to the use of an open science platform focused on health research in a variety of ``real world'' settings. Conclusions: Our initial design and development experience over the course of the CIELO project has provided the basis for a vigorous dialogue between stakeholder community members regarding the capabilities that will add the greatest value to an open science platform for the health research community. A number of important questions around user incentives, sustainability, and scalability will require further community dialogue and agreement. ", doi="10.2196/jmir.6937", url="http://www.jmir.org/2017/7/e276/", url="http://www.ncbi.nlm.nih.gov/pubmed/28760728" }