%0 Journal Article %@ 2373-6658 %I JMIR Publications %V 9 %N %P e63769 %T COVID-19–Related Racism and Mental Health Among Asian Americans: Integrative Review %A Von Visger,Tania %A Lyons,Amy %A Zhou,Yanjun %A Wardlaw,Kayla %A Park,Eunhee %A Chang,Yu-Ping %K racism %K anti-Asian sentiment %K integrative review %K psychological distress %K mental health %K review %K Asian American %K Asian %K wellness %K psychological %K distress %K COVID-19 %K pandemic %K cross-sectional survey %K survey %K depression %K anxiety %D 2025 %7 2.4.2025 %9 %J Asian Pac Isl Nurs J %G English %X Background: Racism against Asian Americans escalated during the COVID-19 pandemic. About 31%‐91% of Asian American adults and children reported experiencing various types of racism during the pandemic. According to the Federal Bureau of Investigation hate crime statistics, anti-Asian hate crime incidents increased from 158 in 2019 to 279 in 2020 and 746 in 2021. In 2022, the incidents decreased to 499, corresponding to the downward trend of the pandemic. The degree of impact racism has on mental health and wellness among Asian Americans requires investigation, specifically during the COVID-19 pandemic. Objective: We aim to describe racism-related mental health problems experienced by Asian Americans living in the United States and propose implementation strategies for mitigating their consequences. Methods: We conducted an integrative review of peer-reviewed publications in English reporting anti-Asian sentiments and racism’s impacts on mental health among Asian Americans in the United States. Results: The 29 eligible articles report on studies that utilized cross-sectional survey designs with various sample sizes. Racism is directly correlated with the prevalence of depression and anxiety experienced by victims of racist acts. The prevalence of in-person direct racism (racist expression aimed directly at the victim) is lower than in-person indirect racism (racist expression aimed at the ethnic group the victim belongs to). During the COVID-19 pandemic, the incidence of explicit online racism was lower than online indirect racism. Conclusions: COVID-19–related racism exacerbated preexisting racism, contributing to worse depression and anxiety among Asian Americans. To address this issue, we propose 2 main approaches: increase public awareness and education about recognizable racist sentiments/acts and systematized reporting of racially motivated crimes to guide political action. At an individual level, culturally responsive, trauma-informed interventions promoting cultural support and cohesion for various Asian American groups will foster this empowerment. These proposed actions will help alleviate racism by reducing stereotypes, empowering victims, and chipping away at the systemic racism structure. %R 10.2196/63769 %U https://apinj.jmir.org/2025/1/e63769 %U https://doi.org/10.2196/63769 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e59970 %T Publishing Identifiable Patient Photographs in the Digital Age: Focus Group Study of Patients, Doctors, and Medical Students %A Roguljić,Marija %A Šimunović,Dina %A Buljan,Ivan %A Žuljević,Marija Franka %A Turić,Antonela %A Marušić,Ana %+ Department of Periodontology, University of Split, Šoltanska 2A, Split, 21000, Croatia, 385 21557800, marija.roguljic@mefst.hr %K patient photographs %K patient privacy, confidentiality %K data protection %K ethical publishing %K informed consent %K open access %K scientific journals %K focus group. %D 2025 %7 5.3.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: The publication of patient photographs in scientific journals continues to pose challenges regarding privacy and confidentiality, despite existing ethical guidelines. Recent studies indicate that key stakeholders—including health care professionals and patients—lack sufficient awareness of the ethical considerations surrounding patient photographs, particularly in the context of digital scientific publishing. Objective: This qualitative study aims to explore how different stakeholders—patients, medical students, and doctors—understand the challenges of patient privacy and confidentiality in scientific publications. Additionally, it sought to identify key areas for future research, particularly in the context of online, open-access articles. Methods: We conducted 4 online focus groups due to COVID-19 restrictions: 1 with patients, 2 with final-year medical students, and 1 with head and neck physicians and dentists who regularly handle patient photographs. Participants were invited via email, and those who accepted took part in discussions lasting approximately 1 hour. All interviews were recorded and transcribed for analysis. All 4 focus groups were asked the same set of questions, covering the following topics: (1) consent for publishing patient photographs, (2) information on guidelines and standards for consent to publish patient photographs, (3) the importance of informed consent for various purposes, (4) methods for deidentifying patient photographs, and (5) the use of patient photographs in online, open-access publishing. Results: Three key themes emerged from the focus group discussions: (1) no definitive resources or practical recommendations available, (2) online publishing of patient images makes them more open to misuse, and (3) anonymization techniques have limitations. All stakeholder groups expressed a lack of knowledge about online publishing in general and concerns about the fate of patient photographs in the digital environment after publication. They emphasized the need for increased awareness among all relevant stakeholders and more stringent procedures for obtaining informed patient consent before publishing photographs. While they recognized the usefulness of image anonymization techniques in protecting patient identity, they were also aware that current methods remain insufficient to ensure complete anonymity. Conclusions: This qualitative study highlights that publishing patient photographs in open-access scientific journals is an important, serious, and largely unexplored issue, with all stakeholders still uncertain about the best ways to protect patient privacy. Clinicians, publishers, and journal editors should not only implement best practices to ensure fully informed patient consent for publishing identifiable photographs but also develop technical and governance safeguards. Future quantitative studies are needed to identify the most effective ways to enhance stakeholders’ knowledge, policies, and procedures, ultimately guiding the development of practical recommendations for the ethical publication of patient photographs in scientific journals. %M 40053737 %R 10.2196/59970 %U https://www.jmir.org/2025/1/e59970 %U https://doi.org/10.2196/59970 %U http://www.ncbi.nlm.nih.gov/pubmed/40053737 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 9 %N %P e60465 %T Assessment of the Sensitivity of a Smartphone App to Assist Patients in the Identification of Stroke and Myocardial Infarction: Cross-Sectional Study %A Dhand,Amar %A Mangipudi,Rama %A Varshney,Anubodh S %A Crowe,Jonathan R %A Ford,Andria L %A Sweitzer,Nancy K %A Shin,Min %A Tate,Samuel %A Haddad,Haissam %A Kelly,Michael E %A Muller,James %A Shavadia,Jay S %K time-to-treatment %K digital health %K patient-centered care %K eHealth apps %K smartphones %K apps %K cross-sectional study %K stroke %K myocardial infarction %K heart attack %K neurology %K neurological assessment %K accuracy %K emergency department %K finger-tapping test %K hospital admission %K physicians %K medical records %K mobile health %K mhealth %K symptom recognition %K mobile phone %D 2025 %7 3.3.2025 %9 %J JMIR Form Res %G English %X Background: Most people do not recognize symptoms of neurological and cardiac emergencies in a timely manner. This leads to delays in hospital arrival and reduced access to therapies that can open arteries. We created a smartphone app to help patients and families evaluate if symptoms may be high risk for stroke or heart attack (myocardial infarction, MI). The ECHAS (Emergency Call for Heart Attack and Stroke) app guides users to assess their risk through evidence-based questions and a test of weakness in one arm by evaluating finger-tapping on the smartphone. Objective: This study is an initial step in the accuracy evaluation of the app focused on sensitivity. We evaluated whether the app provides appropriate triage advice for patients with known stroke or MI symptoms in the Emergency Department. We designed this study to evaluate the sensitivity of the app, since the most dangerous output of the app would be failure to recognize the need for emergency evaluation. Specificity is also important, but the consequences of low specificity are less dangerous than those of low sensitivity. Methods: In this single-center cross-sectional study, we enrolled patients presenting with symptoms of possible stroke or MI. The ECHAS app assessment consisted of a series of evidence-based questions regarding symptoms and a test of finger-tapping speed and accuracy on the phone’s screen to detect unilateral arm weakness. The primary outcome was the sensitivity of the ECHAS app in detecting the need for ED evaluation. The secondary outcome was the sensitivity of the ECHAS app in detecting the need for hospital admission. Two independent and blinded board-certified physicians reviewed the medical record and adjudicated the appropriateness of the ED visit based on a 5-point score (ground truth). Finally, we asked patients semistructured questions about the app’s ease of use, drawbacks, and benefits. Results: We enrolled 202 patients (57 with stroke and 145 with MI). The ECHAS score was strongly correlated with the ground truth appropriateness score (Spearman correlation 0.41, P<.001). The ECHAS app had a sensitivity of 0.98 for identifying patients in whom ED evaluation was appropriate. The app had a sensitivity of 1.0 for identifying patients who were admitted to the hospital because of their ED evaluation. Patients completed an app session in an average of 111 (SD 60) seconds for the stroke pathway and 60 (SD 33) seconds for the MI pathway. Patients reported that the app was easy to use and valuable for personal emergency situations at home. Conclusions: The ECHAS app demonstrated a high sensitivity for the detection of patients who required emergency evaluation for symptoms of stroke or MI. This study supports the need for a study of specificity of the app, and then a prospective trial of the app in patients at increased risk of MI and stroke. %R 10.2196/60465 %U https://formative.jmir.org/2025/1/e60465 %U https://doi.org/10.2196/60465 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 9 %N %P e71664 %T Correction: Evaluating Bard Gemini Pro and GPT-4 Vision Against Student Performance in Medical Visual Question Answering: Comparative Case Study %A Roos,Jonas %A Martin,Ron %A Kaczmarczyk,Robert %D 2025 %7 11.2.2025 %9 %J JMIR Form Res %G English %X %R 10.2196/71664 %U https://formative.jmir.org/2025/1/e71664 %U https://doi.org/10.2196/71664 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e64069 %T Data-Sharing Statements Requested from Clinical Trials by Public, Environmental, and Occupational Health Journals: Cross-Sectional Study %A Liu,Yingxin %A Zhang,Jingyi %A Thabane,Lehana %A Bai,Xuerui %A Kang,Lili %A Lip,Gregory Y H %A Van Spall,Harriette G C %A Xia,Min %A Li,Guowei %+ Center for Clinical Epidemiology and Methodology, The Affiliated Guangdong Second Provincial General Hospital of Jinan University, 466 Newport Middle Road, Haizhu District, Guangzhou, 510317, China, 86 02089169546, ligw@gd2h.org.cn %K data sharing %K clinical trial %K public health %K International Committee of Medical Journal Editors %K ICMJE %K journal request %K clinical trials %K decision-making %K occupational health %K health informatics %K patient data %D 2025 %7 7.2.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Data sharing plays a crucial role in health informatics, contributing to improving health information systems, enhancing operational efficiency, informing policy and decision-making, and advancing public health surveillance including disease tracking. Sharing individual participant data in public, environmental, and occupational health trials can help improve public trust and support by enhancing transparent reporting and reproducibility of research findings. The International Committee of Medical Journal Editors (ICMJE) requires all papers to include a data-sharing statement. However, it is unclear whether journals in the field of public, environmental, and occupational health adhere to this requirement. Objective: This study aims to investigate whether public, environmental, and occupational health journals requested data-sharing statements from clinical trials submitted for publication. Methods: In this bibliometric survey of “Public, Environmental, and Occupational Health” journals, defined by the Journal Citation Reports (as of June 2023), we included 202 journals with clinical trial reports published between 2019 and 2022. The primary outcome was a journal request for a data-sharing statement, as identified in the paper submission instructions. Multivariable logistic regression analysis was conducted to evaluate the relationship between journal characteristics and journal requests for data-sharing statements, with results presented as odds ratios (ORs) and corresponding 95% CIs. We also investigated whether the journals included a data-sharing statement in their published trial reports. Results: Among the 202 public, environmental, and occupational health journals included, there were 68 (33.7%) journals that did not request data-sharing statements. Factors significantly associated with journal requests for data-sharing statements included open access status (OR 0.43, 95% CI 0.19-0.97), high journal impact factor (OR 2.31, 95% CI 1.15-4.78), endorsement of Consolidated Standards of Reporting Trials (OR 2.43, 95% CI 1.25-4.79), and publication in the United Kingdom (OR 7.18, 95% CI 2.61-23.4). Among the 134 journals requesting data-sharing statements, 26.9% (36/134) did not have statements in their published trial reports. Conclusions: Over one-third of the public, environmental, and occupational health journals did not request data-sharing statements in clinical trial reports. Among those journals that requested data-sharing statements in their submission guidance pages, more than one quarter published trial reports with no data-sharing statements. These results revealed an inadequate practice of requesting data-sharing statements by public, environmental, and occupational health journals, requiring more effort at the journal level to implement ICJME recommendations on data-sharing statements. %M 39919275 %R 10.2196/64069 %U https://www.jmir.org/2025/1/e64069 %U https://doi.org/10.2196/64069 %U http://www.ncbi.nlm.nih.gov/pubmed/39919275 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e59598 %T Evaluation and Comparison of the Academic Quality of Open-Access Mega Journals and Authoritative Journals: Disruptive Innovation Evaluation %A Jiang,Yuyan %A Liu,Xue-li %A Wang,Liyun %+ , Faculty of Humanities & Social Sciences, Xinxiang Medical University, Library and Information Building, 2nd floor, No. 601 Jinsui Avenue, Hongqi District, Xinxiang, , China, 86 1 383 736 0965, liueditor03@163.com %K innovative evaluation %K disruption index %K open-access mega journals %K paper evaluation %K open citation data %D 2025 %7 15.1.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: Some scholars who are skeptical about open-access mega journals (OAMJs) have argued that low-quality papers are often difficult to publish in more prestigious and authoritative journals, and OAMJs may be their main destination. Objective: This study aims to evaluate the academic quality of OAMJs and highlight their important role in clinical medicine. To achieve this aim, authoritative journals and representative OAMJs in this field were selected as research objects. The differences between the two were compared and analyzed in terms of their level of disruptive innovation. Additionally, this paper explored the countries and research directions for which OAMJs serve as publication channels for disruptive innovations. Methods: In this study, the journal information, literature data, and open citation relationship data were sourced from Journal Citation Reports (JCR), Web of Science (WoS), InCites, and the OpenCitations Index of PubMed Open PMID-to-PMID citations (POCI). Then, we calculated the disruptive innovation level of the focus paper based on the local POCI database. Results: The mean Journal Disruption Index (JDI) values for the selected authoritative journals and OAMJs were 0.5866 (SD 0.26933) and 0.0255 (SD 0.01689), respectively, showing a significant difference. Only 1.48% (861/58,181) of the OAMJ papers reached the median level of disruptive innovation of authoritative journal papers (MDAJ). However, the absolute number was roughly equal to that of authoritative journals. OAMJs surpassed authoritative journals in publishing innovative papers in 24 research directions (eg, Allergy), accounting for 40.68% of all research directions in clinical medicine. Among research topics with at least 10 authoritative papers, OAMJs matched or exceeded MDAJ in 35.71% of cases. The number of papers published in authoritative journals and the average level of disruptive innovation in each country showed a linear relationship after logarithmic treatment, with a correlation coefficient of –0.891 (P<.001). However, the number of papers published in OAMJs in each country and the average level of disruptive innovation did not show a linear relationship after logarithmic treatment. Conclusions: While the average disruptive innovation level of papers published by OAMJs is significantly lower than that of authoritative journals, OAMJs have become an important publication channel for innovative research in various research directions. They also provide fairer opportunities for the publication of innovative results from limited-income countries. Therefore, the academic community should recognize the contribution and value of OAMJs to advancing scientific research. %R 10.2196/59598 %U https://www.jmir.org/2025/1/e59598 %U https://doi.org/10.2196/59598 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e65775 %T Geographical Disparities in Research Misconduct: Analyzing Retraction Patterns by Country %A Sebo,Paul %A Sebo,Melissa %+ University Institute for Primary Care, University of Geneva, Rue Michel-Servet 1, Geneva, 1211, Switzerland, 41 223794390, paul.seboe@unige.ch %K affiliation %K country %K fraud %K integrity %K misconduct %K plagiarism %K publication %K research %K retraction %K ethical standards %K ethics %K research misconduct %K literature %D 2025 %7 14.1.2025 %9 Research Letter %J J Med Internet Res %G English %X This study examines disparities in research retractions due to misconduct, identifying countries with the highest retraction counts and those disproportionately represented relative to population and publication output. The findings emphasize the need for improved research integrity measures. %M 39808480 %R 10.2196/65775 %U https://www.jmir.org/2025/1/e65775 %U https://doi.org/10.2196/65775 %U http://www.ncbi.nlm.nih.gov/pubmed/39808480 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e50862 %T Jargon and Readability in Plain Language Summaries of Health Research: Cross-Sectional Observational Study %A Lang,Iain A %A King,Angela %A Boddy,Kate %A Stein,Ken %A Asare,Lauren %A Day,Jo %A Liabo,Kristin %+ Department of Health and Community Sciences, University of Exeter Medical School, University of Exeter, South Cloisters, St Luke's Campus, Exeter, , United Kingdom, 44 7500 786180, i.lang@exeter.ac.uk %K readability %K jargon %K reading %K accessibility %K health research %K science communication %K public understanding of science %K open science %K patient and public involvement %K health literacy %K plain language summary %K health communication %D 2025 %7 13.1.2025 %9 Original Paper %J J Med Internet Res %G English %X Background: The idea of making science more accessible to nonscientists has prompted health researchers to involve patients and the public more actively in their research. This sometimes involves writing a plain language summary (PLS), a short summary intended to make research findings accessible to nonspecialists. However, whether PLSs satisfy the basic requirements of accessible language is unclear. Objective: We aimed to assess the readability and level of jargon in the PLSs of research funded by the largest national clinical research funder in Europe, the United Kingdom’s National Institute for Health and Care Research (NIHR). We also aimed to assess whether readability and jargon were influenced by internal and external characteristics of research projects. Methods: We downloaded the PLSs of all NIHR National Journals Library reports from mid-2014 to mid-2022 (N=1241) and analyzed them using the Flesch Reading Ease (FRE) formula and a jargon calculator (the De-Jargonizer). In our analysis, we included the following study characteristics of each PLS: research topic, funding program, project size, length, publication year, and readability and jargon scores of the original funding proposal. Results: Readability scores ranged from 1.1 to 70.8, with an average FRE score of 39.0 (95% CI 38.4-39.7). Moreover, 2.8% (35/1241) of the PLSs had an FRE score classified as “plain English” or better; none had readability scores in line with the average reading age of the UK population. Jargon scores ranged from 76.4 to 99.3, with an average score of 91.7 (95% CI 91.5-91.9) and 21.7% (269/1241) of the PLSs had a jargon score suitable for general comprehension. Variables such as research topic, funding program, and project size significantly influenced readability and jargon scores. The biggest differences related to the original proposals: proposals with a PLS in their application that were in the 20% most readable were almost 3 times more likely to have a more readable final PLS (incidence rate ratio 2.88, 95% CI 1.86-4.45). Those with the 20% least jargon in the original application were more than 10 times as likely to have low levels of jargon in the final PLS (incidence rate ratio 13.87, 95% CI 5.17-37.2). There was no observable trend over time. Conclusions: Most of the PLSs published in the NIHR’s National Journals Library have poor readability due to their complexity and use of jargon. None were readable at a level in keeping with the average reading age of the UK population. There were significant variations in readability and jargon scores depending on the research topic, funding program, and other factors. Notably, the readability of the original funding proposal seemed to significantly impact the final report’s readability. Ways of improving the accessibility of PLSs are needed, as is greater clarity over who and what they are for. %M 39805102 %R 10.2196/50862 %U https://www.jmir.org/2025/1/e50862 %U https://doi.org/10.2196/50862 %U http://www.ncbi.nlm.nih.gov/pubmed/39805102 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 27 %N %P e57263 %T Contribution of Open Access Databases to Intensive Care Medicine Research: Scoping Review %A Kallout,Julien %A Lamer,Antoine %A Grosjean,Julien %A Kerdelhué,Gaétan %A Bouzillé,Guillaume %A Clavier,Thomas %A Popoff,Benjamin %+ Department of Anesthesiology and Critical Care, CHU Rouen, 37 Boulevard Gambetta, Rouen, 76000, France, 33 2 32 88 89 90, julien.kallout@chu-rouen.fr %K intensive care unit %K ICU %K big data %K databases %K open access %K Amsterdam University Medical Centers Database %K AmsterdamUMCdb %K eICU Collaborative Research Database %K eICU-CRD %K database %K screening %K descriptive analysis %D 2025 %7 9.1.2025 %9 Review %J J Med Internet Res %G English %X Background: Intensive care units (ICUs) handle the most critical patients with a high risk of mortality. Due to those conditions, close monitoring is necessary and therefore, a large volume of data is collected. Collaborative ventures have enabled the emergence of large open access databases, leading to numerous publications in the field. Objective: The aim of this scoping review is to identify the characteristics of studies using open access intensive care databases and to describe the contribution of these studies to intensive care research. Methods: The research was conducted using 3 databases (PubMed–MEDLINE, Embase, and Web of Science) from the inception of each database to August 1, 2022. We included original articles based on 4 open databases of patients admitted to ICUs: Amsterdam University Medical Centers Database, eICU Collaborative Research Database, High time resolution ICU dataset, Medical Information Mart for Intensive Care (II to IV). A double-blinded screening for eligibility was performed, first on the title and abstract and subsequently on the full-text articles. Characteristics relating to publication journals, study design, and statistical analyses were extracted and analyzed. Results: We observed a consistent increase in the number of publications from these databases since 2016. The Medical Information Mart for Intensive Care databases were the most frequently used. The highest contributions came from China and the United States, with 689 (52.7%) and 370 (28.3%) publications respectively. The median impact factor of publications was 3.8 (IQR 2.8-5.8). Topics related to cardiovascular and infectious diseases were predominant, accounting for 333 (25.5%) and 324 (24.8%) articles, respectively. Logistic regression emerged as the most commonly used statistical model for both inference and prediction questions, featuring in 396 (55.5%) and 281 (47.5%) studies, respectively. A majority of the inference studies yielded statistically significant results (84.0%). In prediction studies, area under the curve was the most frequent performance measure, with a median value of 0.840 (IQR 0.780-0.890). Conclusions: The abundance of scientific outputs resulting from these databases, coupled with the diversity of topics addressed, highlight the importance of these databases as valuable resources for clinical research. This suggests their potential impact on clinical practice within intensive care settings. However, the quality and clinical relevance of these studies remains highly heterogeneous, with a majority of articles being published in low–impact factor journals. %M 39787600 %R 10.2196/57263 %U https://www.jmir.org/2025/1/e57263 %U https://doi.org/10.2196/57263 %U http://www.ncbi.nlm.nih.gov/pubmed/39787600 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e60057 %T Decoding the Digital Pulse: Bibliometric Analysis of 25 Years in Digital Health Research Through the Journal of Medical Internet Research %A Kaczmarczyk,Robert %A Wilhelm,Theresa Isabelle %A Roos,Jonas %A Martin,Ron %+ Eye Center—Medical Center, Faculty of Medicine, Albert-Ludwigs-University of Freiburg, Killianstraße 5, Freiburg, 79106, Germany, 49 76127040020, theresa.wilhelm@uniklinik-freiburg.de %K digital health %K JMIR publication analysis %K network analysis %K artificial intelligence %K AI %K large language models %K eHealth %K Claude 3 Opus %K digital %K digital technology %K digital intervention %K machine learning %K natural language processing %K NLP %K deep learning %K algorithm %K model %K analytics %K practical model %K pandemic %K postpandemic era %K mobile phone %D 2024 %7 15.11.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: As the digital health landscape continues to evolve, analyzing the progress and direction of the field can yield valuable insights. The Journal of Medical Internet Research (JMIR) has been at the forefront of disseminating digital health research since 1999. A comprehensive network analysis of JMIR publications can help illuminate the evolution and trends in digital medicine over the past 25 years. Objective: This study aims to conduct a detailed network analysis of JMIR’s publications to uncover the growth patterns, dominant themes, and potential future trajectories in digital health research. Methods: We retrieved 8068 JMIR papers from PubMed using the Biopython library. Keyword metrics were assessed using accuracy, recall, and F1-scores to evaluate the effectiveness of keyword identification from Claude 3 Opus and Gemini 1.5 Pro in addition to 2 conventional natural language processing methods using key bidirectional encoder representations from transformers. Future trends for 2024-2026 were predicted using Claude 3 Opus, Google’s Time Series Foundation Model, autoregressive integrated moving average, exponential smoothing, and Prophet. Network visualization techniques were used to represent and analyze the complex relationships between collaborating countries, paper types, and keyword co-occurrence. Results: JMIR’s publication volume showed consistent growth, with a peak in 2020. The United States dominated country contributions, with China showing a notable increase in recent years. Keyword analysis from 1999 to 2023 showed significant thematic shifts, from an early internet and digital health focus to the dominance of COVID-19 and advanced technologies such as machine learning. Predictions for 2024-2026 suggest an increased focus on artificial intelligence, digital health, and mental health. Conclusions: Network analysis of JMIR publications provides a macroscopic view of the evolution of the digital health field. The journal’s trajectory reflects broader technological advances and shifting research priorities, including the impact of the COVID-19 pandemic. The predicted trends underscore the growing importance of computational technology in future health care research and practice. The findings from JMIR provide a glimpse into the future of digital medicine, suggesting a robust integration of artificial intelligence and continued emphasis on mental health in the postpandemic era. %M 39546778 %R 10.2196/60057 %U https://www.jmir.org/2024/1/e60057 %U https://doi.org/10.2196/60057 %U http://www.ncbi.nlm.nih.gov/pubmed/39546778 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e60025 %T Advancing the United Nations Sustainable Development Goals Through Digital Health Research: 25 Years of Contributions From the Journal of Medical Internet Research %A Raman,Raghu %A Singhania,Monica %A Nedungadi,Prema %+ Amrita School of Business, Amrita School of Computing, Amrita Vishwa Vidyapeetham, Amritapuri, Kerala, 690525, India, 91 9895028779, raghu@amrita.edu %K sustainable development goal %K topic modeling %K public health %K surveillance %K gender equality %K non-communicable disease %K social media %K COVID-19 %K SARS-CoV-2 %K coronavirus %K machine learning %K artificial intelligence %K AI %K digital health %D 2024 %7 4.11.2024 %9 Research Letter %J J Med Internet Res %G English %X %M 39496147 %R 10.2196/60025 %U https://www.jmir.org/2024/1/e60025 %U https://doi.org/10.2196/60025 %U http://www.ncbi.nlm.nih.gov/pubmed/39496147 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e64221 %T Video Abstracts in Research %A Nachman,Sophie %A Ortiz-Prado,Esteban %A Tucker,Joseph D %+ Institute for Global Health and Infectious Diseases, 130 Mason Farm Road, 2nd Floor, Chapel Hill, NC, 27599, United States, 1 9199662536, jdtucker@med.unc.edu %K video abstract %K abstract %K dissemination %K public engagement %K online %K videos %K public audience %K communication %K infographics %K health literacy %K patient education %K public health %D 2024 %7 4.11.2024 %9 Viewpoint %J J Med Internet Res %G English %X Video abstracts can be useful in health research. A video abstract provides key messages about a research article and can increase public engagement, spark conversations, and may increase academic attention. A growing number of open source software programs make it easier to develop a video abstract. This viewpoint provides practical tips for creating a video abstract for health research. %M 39496154 %R 10.2196/64221 %U https://www.jmir.org/2024/1/e64221 %U https://doi.org/10.2196/64221 %U http://www.ncbi.nlm.nih.gov/pubmed/39496154 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 12 %N %P e54653 %T Accelerating Evidence Synthesis in Observational Studies: Development of a Living Natural Language Processing–Assisted Intelligent Systematic Literature Review System %A Manion,Frank J %A Du,Jingcheng %A Wang,Dong %A He,Long %A Lin,Bin %A Wang,Jingqi %A Wang,Siwei %A Eckels,David %A Cervenka,Jan %A Fiduccia,Peter C %A Cossrow,Nicole %A Yao,Lixia %K machine learning %K deep learning %K natural language processing %K systematic literature review %K artificial intelligence %K software development %K data extraction %K epidemiology %D 2024 %7 23.10.2024 %9 %J JMIR Med Inform %G English %X Background: Systematic literature review (SLR), a robust method to identify and summarize evidence from published sources, is considered to be a complex, time-consuming, labor-intensive, and expensive task. Objective: This study aimed to present a solution based on natural language processing (NLP) that accelerates and streamlines the SLR process for observational studies using real-world data. Methods: We followed an agile software development and iterative software engineering methodology to build a customized intelligent end-to-end living NLP-assisted solution for observational SLR tasks. Multiple machine learning–based NLP algorithms were adopted to automate article screening and data element extraction processes. The NLP prediction results can be further reviewed and verified by domain experts, following the human-in-the-loop design. The system integrates explainable articificial intelligence to provide evidence for NLP algorithms and add transparency to extracted literature data elements. The system was developed based on 3 existing SLR projects of observational studies, including the epidemiology studies of human papillomavirus–associated diseases, the disease burden of pneumococcal diseases, and cost-effectiveness studies on pneumococcal vaccines. Results: Our Intelligent SLR Platform covers major SLR steps, including study protocol setting, literature retrieval, abstract screening, full-text screening, data element extraction from full-text articles, results summary, and data visualization. The NLP algorithms achieved accuracy scores of 0.86-0.90 on article screening tasks (framed as text classification tasks) and macroaverage F1 scores of 0.57-0.89 on data element extraction tasks (framed as named entity recognition tasks). Conclusions: Cutting-edge NLP algorithms expedite SLR for observational studies, thus allowing scientists to have more time to focus on the quality of data and the synthesis of evidence in observational studies. Aligning the living SLR concept, the system has the potential to update literature data and enable scientists to easily stay current with the literature related to observational studies prospectively and continuously. %R 10.2196/54653 %U https://medinform.jmir.org/2024/1/e54653 %U https://doi.org/10.2196/54653 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e58987 %T Mapping the Evolution of Digital Health Research: Bibliometric Overview of Research Hotspots, Trends, and Collaboration of Publications in JMIR (1999-2024) %A Hu,Jing %A Li,Chong %A Ge,Yanlei %A Yang,Jingyi %A Zhu,Siyi %A He,Chengqi %+ Rehabilitation Medicine Center, West China Hospital, Sichuan University, #37 Guoxue Alley, Wuhou District, Chengdu, 610041, China, 86 28 8542 2847, hxkfhcq2015@126.com %K JMIR %K bibliometric analysis %K ehealth %K digital health %K medical informatics %K health informatics %K open science %K publishing %D 2024 %7 17.10.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: While bibliometric studies of individual journals have been conducted, to the best of our knowledge, bibliometric mapping has not yet been utilized to analyze the literature published by the Journal of Medical Internet Research (JMIR). Objective: In celebration of the journal’s 25th anniversary, this study aimed to review the entire collection of JMIR publications from 1999 to 2024 and provide a comprehensive overview of the main publication characteristics. Methods: This study included papers published in JMIR during the 25-year period from 1999 to 2024. The data were analyzed using CiteSpace, VOSviewer, and the “Bibliometrix” package in R. Through descriptive bibliometrics, we examined the dynamics and trend patterns of JMIR literature production and identified the most prolific authors, papers, institutions, and countries. Bibliometric maps were used to visualize the content of published articles and to identify the most prominent research terms and topics, along with their evolution. A bibliometric network map was constructed to determine the hot research topics over the past 25 years. Results: This study revealed positive trends in literature production, with both the total number of publications and the average number of citations increasing over the years. And the global COVID-19 pandemic induced an explosive rise in the number of publications in JMIR. The most productive institutions were predominantly from the United States, which ranked highest in successful publications within the journal. The editor-in-chief of JMIR was identified as a pioneer in this field. The thematic analysis indicated that the most prolific topics aligned with the primary aims and scope of the journal. Currently and in the foreseeable future, the main themes of JMIR include “artificial intelligence,” “patient empowerment,” and “victimization.” Conclusions: This bibliometric study highlighted significant contributions to digital health by identifying key research trends, themes, influential authors, and collaborations. The findings underscore the necessity to enhance publications from developing countries, improve gender diversity among authors, and expand the range of research topics explored in the journal. %M 39419496 %R 10.2196/58987 %U https://www.jmir.org/2024/1/e58987 %U https://doi.org/10.2196/58987 %U http://www.ncbi.nlm.nih.gov/pubmed/39419496 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e58950 %T Twenty-Five Years of Progress—Lessons Learned From JMIR Publications to Address Gender Parity in Digital Health Authorships: Bibliometric Analysis %A Meyer,Annika %A Streichert,Thomas %+ Institute of Clinical Chemistry, Faculty of Medicine and University Hospital, University Hospital Cologne, Kerpener Str 62, Cologne, 50937, Germany, annika.meyer@uk-koeln.de %K digital health %K medical informatics, authorship %K gender distribution %K diversity %K bibliometric %K scientometric %K algorithmic bias reduction %K gender gap %K JMIR Publications %K authorships %K author %K authors %K bibliometric analysis %K equality %K comparison %K gender representation %K journal %K journals %K article %K articles %K Web of Science %K control group %K comparative analysis %K statistical analysis %K gender %D 2024 %7 9.8.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Digital health research plays a vital role in advancing equitable health care. The diversity of research teams is thereby instrumental in capturing societal challenges, increasing productivity, and reducing bias in algorithms. Despite its importance, the gender distribution within digital health authorship remains largely unexplored. Objective: This study aimed to investigate the gender distribution among first and last authors in digital health research, thereby identifying predicting factors of female authorship. Methods: This bibliometric analysis examined the gender distribution across 59,980 publications from 1999 to 2023, spanning 42 digital health journals indexed in the Web of Science. To identify strategies ensuring equality in research, a detailed comparison of gender representation in JMIR journals was conducted within the field, as well as against a matched sample. Two-tailed Welch 2-sample t tests, Wilcoxon rank sum tests, and chi-square tests were used to assess differences. In addition, odds ratios were calculated to identify predictors of female authorship. Results: The analysis revealed that 37% of first authors and 30% of last authors in digital health were female. JMIR journals demonstrated a higher representation, with 49% of first authors and 38% of last authors being female, yielding odds ratios of 1.96 (95% CI 1.90-2.03; P<.001) and 1.78 (95% CI 1.71-1.84; P<.001), respectively. Since 2008, JMIR journals have consistently featured a greater proportion of female first authors than male counterparts. Other factors that predicted female authorship included having female authors in other relevant positions and gender discordance, given the higher rate of male last authors in the field. Conclusions: There was an evident shift toward gender parity across publications in digital health, particularly from the publisher JMIR Publications. The specialized focus of its sister journals, equitable editorial policies, and transparency in the review process might contribute to these achievements. Further research is imperative to establish causality, enabling the replication of these successful strategies across other scientific fields to bridge the gender gap in digital health effectively. %M 39121467 %R 10.2196/58950 %U https://www.jmir.org/2024/1/e58950 %U https://doi.org/10.2196/58950 %U http://www.ncbi.nlm.nih.gov/pubmed/39121467 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e52001 %T Assessing the Reproducibility of the Structured Abstracts Generated by ChatGPT and Bard Compared to Human-Written Abstracts in the Field of Spine Surgery: Comparative Analysis %A Kim,Hong Jin %A Yang,Jae Hyuk %A Chang,Dong-Gune %A Lenke,Lawrence G %A Pizones,Javier %A Castelein,René %A Watanabe,Kota %A Trobisch,Per D %A Mundis Jr,Gregory M %A Suh,Seung Woo %A Suk,Se-Il %+ Department of Orthopedic Surgery, Inje University Sanggye Paik Hospital, College of Medicine, Inje University, 1342, Dongil-Ro, Nowon-Gu, Seoul, 01757, Republic of Korea, 82 2 950 1284, dgchangmd@gmail.com %K artificial intelligence %K AI %K ChatGPT %K Bard %K scientific abstract %K orthopedic surgery %K spine %K journal guidelines %K plagiarism %K ethics %K spine surgery %K surgery %K language model %K chatbot %K formatting guidelines %K abstract %D 2024 %7 26.6.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Due to recent advances in artificial intelligence (AI), language model applications can generate logical text output that is difficult to distinguish from human writing. ChatGPT (OpenAI) and Bard (subsequently rebranded as “Gemini”; Google AI) were developed using distinct approaches, but little has been studied about the difference in their capability to generate the abstract. The use of AI to write scientific abstracts in the field of spine surgery is the center of much debate and controversy. Objective: The objective of this study is to assess the reproducibility of the structured abstracts generated by ChatGPT and Bard compared to human-written abstracts in the field of spine surgery. Methods: In total, 60 abstracts dealing with spine sections were randomly selected from 7 reputable journals and used as ChatGPT and Bard input statements to generate abstracts based on supplied paper titles. A total of 174 abstracts, divided into human-written abstracts, ChatGPT-generated abstracts, and Bard-generated abstracts, were evaluated for compliance with the structured format of journal guidelines and consistency of content. The likelihood of plagiarism and AI output was assessed using the iThenticate and ZeroGPT programs, respectively. A total of 8 reviewers in the spinal field evaluated 30 randomly extracted abstracts to determine whether they were produced by AI or human authors. Results: The proportion of abstracts that met journal formatting guidelines was greater among ChatGPT abstracts (34/60, 56.6%) compared with those generated by Bard (6/54, 11.1%; P<.001). However, a higher proportion of Bard abstracts (49/54, 90.7%) had word counts that met journal guidelines compared with ChatGPT abstracts (30/60, 50%; P<.001). The similarity index was significantly lower among ChatGPT-generated abstracts (20.7%) compared with Bard-generated abstracts (32.1%; P<.001). The AI-detection program predicted that 21.7% (13/60) of the human group, 63.3% (38/60) of the ChatGPT group, and 87% (47/54) of the Bard group were possibly generated by AI, with an area under the curve value of 0.863 (P<.001). The mean detection rate by human reviewers was 53.8% (SD 11.2%), achieving a sensitivity of 56.3% and a specificity of 48.4%. A total of 56.3% (63/112) of the actual human-written abstracts and 55.9% (62/128) of AI-generated abstracts were recognized as human-written and AI-generated by human reviewers, respectively. Conclusions: Both ChatGPT and Bard can be used to help write abstracts, but most AI-generated abstracts are currently considered unethical due to high plagiarism and AI-detection rates. ChatGPT-generated abstracts appear to be superior to Bard-generated abstracts in meeting journal formatting guidelines. Because humans are unable to accurately distinguish abstracts written by humans from those produced by AI programs, it is crucial to exercise special caution and examine the ethical boundaries of using AI programs, including ChatGPT and Bard. %M 38924787 %R 10.2196/52001 %U https://www.jmir.org/2024/1/e52001 %U https://doi.org/10.2196/52001 %U http://www.ncbi.nlm.nih.gov/pubmed/38924787 %0 Journal Article %@ 1929-073X %I JMIR Publications %V 13 %N %P e50698 %T Online Visibility and Scientific Relevance of Strabismus Research: Bibliometric Analysis %A Stupnicki,Aleksander %A Suresh,Basil %A Jain,Saurabh %+ University College London Medical School, 4 Huntley Street, London, WC1E 6DE, United Kingdom, 44 02031088235, zchatup@ucl.ac.uk %K strabismus research %K squint %K social media %K scientific relevance %K altmetrics %K accuracy %K medical knowledge %K metric %K bibliometric analysis %K research %K strabismus %K online visibility %K platform %K evidence-based information %K accessibility %D 2024 %7 12.6.2024 %9 Original Paper %J Interact J Med Res %G English %X Background: Quality and accuracy of online scientific data are crucial, given that the internet and social media serve nowadays as primary sources of medical knowledge. Objective: This study aims to analyze the relationship between scientific relevance and online visibility of strabismus research to answer the following questions: (1) Are the most popular strabismus papers scientifically relevant? (2) Are the most high-impact strabismus studies shared enough online? Methods: The Altmetric Attention Score (AAS) was used as a proxy for online visibility, whereas citations and the journal’s impact factor (IF) served as a metric for scientific relevance. Using “strabismus” as a keyword, 100 papers with the highest AAS and 100 papers with the highest number of citations were identified. Statistical analyses, including the Spearman rank test, linear regression, and factor analysis, were performed to assess the relationship between AAS, citations, a journal’s IF, and mentions across 18 individual Web 2.0 platforms. Results: A weak, positive, statistically significant correlation was observed between normalized AAS and normalized citations (P<.001; r=0.27) for papers with high visibility. Only Twitter mentions and Mendeley readers correlated significantly with normalized citations (P=.02 and P<.001, respectively) and IF (P=.04 and P=.009, respectively), with Twitter being the strongest significant predictor of citation numbers (r=0.53). For high-impact papers, no correlation was found between normalized citations and normalized AAS (P=.12) or the IF of the journal (P=.55). Conclusions: While clinical relevance influences online attention, most high-impact research related to strabismus is not sufficiently shared on the web. Therefore, researchers should make a greater effort to share high-impact papers related to strabismus on online media platforms to improve accessibility and quality of evidence-based knowledge for patients. %M 38865170 %R 10.2196/50698 %U https://www.i-jmr.org/2024/1/e50698 %U https://doi.org/10.2196/50698 %U http://www.ncbi.nlm.nih.gov/pubmed/38865170 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e55121 %T Evaluation and Comparison of Academic Impact and Disruptive Innovation Level of Medical Journals: Bibliometric Analysis and Disruptive Evaluation %A Jiang,Yuyan %A Liu,Xue-li %A Zhang,Zixuan %A Yang,Xinru %+ Faculty of Humanities & Social Sciences, Xinxiang Medical University, Library and Information Building, 2nd Fl., No. 601, Jinsui Avenue, Hongqi District, Xinxiang, 450003, China, 86 1 383 736 0965, liueditor03@163.com %K medical journals %K journal evaluation %K innovative evaluation %K journal disruption index %K disruptive innovation %K academic impact %K peer review %D 2024 %7 31.5.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: As an important platform for researchers to present their academic findings, medical journals have a close relationship between their evaluation orientation and the value orientation of their published research results. However, the differences between the academic impact and level of disruptive innovation of medical journals have not been examined by any study yet. Objective: This study aims to compare the relationships and differences between the academic impact, disruptive innovation levels, and peer review results of medical journals and published research papers. We also analyzed the similarities and differences in the impact evaluations, disruptive innovations, and peer reviews for different types of medical research papers and the underlying reasons. Methods: The general and internal medicine Science Citation Index Expanded (SCIE) journals in 2018 were chosen as the study object to explore the differences in the academic impact and level of disruptive innovation of medical journals based on the OpenCitations Index of PubMed open PMID-to-PMID citations (POCI) and H1Connect databases, respectively, and we compared them with the results of peer review. Results: First, the correlation coefficients of the Journal Disruption Index (JDI) with the Journal Cumulative Citation for 5 years (JCC5), Journal Impact Factor (JIF), and Journal Citation Indicator (JCI) were 0.677, 0.585, and 0.621, respectively. The correlation coefficient of the absolute disruption index (Dz) with the Cumulative Citation for 5 years (CC5) was 0.635. However, the average difference in the disruptive innovation and academic influence rankings of journals reached 20 places (about 17.5%). The average difference in the disruptive innovation and influence rankings of research papers reached about 2700 places (about 17.7%). The differences reflect the essential difference between the two evaluation systems. Second, the top 7 journals selected based on JDI, JCC5, JIF, and JCI were the same, and all of them were H-journals. Although 8 (8/15, 53%), 96 (96/150, 64%), and 880 (880/1500, 58.67%) of the top 0.1%, top 1%, and top 10% papers selected based on Dz and CC5, respectively, were the same. Third, research papers with the “changes clinical practice” tag showed only moderate innovation (4.96) and impact (241.67) levels but had high levels of peer-reviewed recognition (6.00) and attention (2.83). Conclusions: The results of the study show that research evaluation based on innovative indicators is detached from the traditional impact evaluation system. The 3 evaluation systems (impact evaluation, disruptive innovation evaluation, and peer review) only have high consistency for authoritative journals and top papers. Neither a single impact indicator nor an innovative indicator can directly reflect the impact of medical research for clinical practice. How to establish an integrated, comprehensive, scientific, and reasonable journal evaluation system to improve the existing evaluation system of medical journals still needs further research. %M 38820583 %R 10.2196/55121 %U https://www.jmir.org/2024/1/e55121 %U https://doi.org/10.2196/55121 %U http://www.ncbi.nlm.nih.gov/pubmed/38820583 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e53164 %T Hallucination Rates and Reference Accuracy of ChatGPT and Bard for Systematic Reviews: Comparative Analysis %A Chelli,Mikaël %A Descamps,Jules %A Lavoué,Vincent %A Trojani,Christophe %A Azar,Michel %A Deckert,Marcel %A Raynier,Jean-Luc %A Clowez,Gilles %A Boileau,Pascal %A Ruetsch-Chelli,Caroline %+ Institute for Sports and Reconstructive Bone and Joint Surgery, Groupe Kantys, 7 Avenue, Durante, Nice, 06000, France, 33 4 93 16 76 40, mikael.chelli@gmail.com %K artificial intelligence %K large language models %K ChatGPT %K Bard %K rotator cuff %K systematic reviews %K literature search %K hallucinated %K human conducted %D 2024 %7 22.5.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Large language models (LLMs) have raised both interest and concern in the academic community. They offer the potential for automating literature search and synthesis for systematic reviews but raise concerns regarding their reliability, as the tendency to generate unsupported (hallucinated) content persist. Objective: The aim of the study is to assess the performance of LLMs such as ChatGPT and Bard (subsequently rebranded Gemini) to produce references in the context of scientific writing. Methods: The performance of ChatGPT and Bard in replicating the results of human-conducted systematic reviews was assessed. Using systematic reviews pertaining to shoulder rotator cuff pathology, these LLMs were tested by providing the same inclusion criteria and comparing the results with original systematic review references, serving as gold standards. The study used 3 key performance metrics: recall, precision, and F1-score, alongside the hallucination rate. Papers were considered “hallucinated” if any 2 of the following information were wrong: title, first author, or year of publication. Results: In total, 11 systematic reviews across 4 fields yielded 33 prompts to LLMs (3 LLMs×11 reviews), with 471 references analyzed. Precision rates for GPT-3.5, GPT-4, and Bard were 9.4% (13/139), 13.4% (16/119), and 0% (0/104) respectively (P<.001). Recall rates were 11.9% (13/109) for GPT-3.5 and 13.7% (15/109) for GPT-4, with Bard failing to retrieve any relevant papers (P<.001). Hallucination rates stood at 39.6% (55/139) for GPT-3.5, 28.6% (34/119) for GPT-4, and 91.4% (95/104) for Bard (P<.001). Further analysis of nonhallucinated papers retrieved by GPT models revealed significant differences in identifying various criteria, such as randomized studies, participant criteria, and intervention criteria. The study also noted the geographical and open-access biases in the papers retrieved by the LLMs. Conclusions: Given their current performance, it is not recommended for LLMs to be deployed as the primary or exclusive tool for conducting systematic reviews. Any references generated by such models warrant thorough validation by researchers. The high occurrence of hallucinations in LLMs highlights the necessity for refining their training and functionality before confidently using them for rigorous academic purposes. %M 38776130 %R 10.2196/53164 %U https://www.jmir.org/2024/1/e53164 %U https://doi.org/10.2196/53164 %U http://www.ncbi.nlm.nih.gov/pubmed/38776130 %0 Journal Article %@ 2562-0959 %I JMIR Publications %V 7 %N %P e40819 %T Gender Parity Analysis of the Editorial Boards of Influential Dermatology Journals: Cross-Sectional Study %A Szeto,Mindy D %A Sivesind,Torunn E %A Kim,Lori S %A O’Connell,Katie A %A Sprague,Kathryn A %A Nong,Yvonne %A Strock,Daniel M %A Cao,Annie L %A Wu,Jieying %A Toledo,Lauren M %A Wolfe,Sophia M %A Boothby-Shoemaker,Wyatt %A Dellavalle,Robert P %+ Department of Dermatology, University of Minnesota Medical School, 1-411 Phillips-Wangensteen Building, 516 Delaware St SE, MMC 98, Minneapolis, MN, 55455, United States, 1 612 625 8625, della056@umn.edu %K diversity %K equity %K inclusion %K editors %K journals %K publications %K editorial board %K women %K gender %K underrepresentation %D 2024 %7 21.5.2024 %9 Research Letter %J JMIR Dermatol %G English %X This study underscores the persistent underrepresentation of women in academic dermatology leadership positions by examining the gender composition of editorial boards across top dermatology journals, emphasizing the urgent need for proactive strategies to promote diversity, equity, and inclusion. %M 38772024 %R 10.2196/40819 %U https://derma.jmir.org/2024/1/e40819 %U https://doi.org/10.2196/40819 %U http://www.ncbi.nlm.nih.gov/pubmed/38772024 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e48330 %T Comparing Open-Access Database and Traditional Intensive Care Studies Using Machine Learning: Bibliometric Analysis Study %A Ke,Yuhe %A Yang,Rui %A Liu,Nan %+ Centre for Quantitative Medicine, Duke-NUS Medical School, National University of Singapore, 8 College Road, Singapore, 169857, Singapore, 65 66016503, liu.nan@duke-nus.edu.sg %K BERTopic %K critical care %K eICU %K machine learning %K MIMIC %K Medical Information Mart for Intensive Care %K natural language processing %D 2024 %7 17.4.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Intensive care research has predominantly relied on conventional methods like randomized controlled trials. However, the increasing popularity of open-access, free databases in the past decade has opened new avenues for research, offering fresh insights. Leveraging machine learning (ML) techniques enables the analysis of trends in a vast number of studies. Objective: This study aims to conduct a comprehensive bibliometric analysis using ML to compare trends and research topics in traditional intensive care unit (ICU) studies and those done with open-access databases (OADs). Methods: We used ML for the analysis of publications in the Web of Science database in this study. Articles were categorized into “OAD” and “traditional intensive care” (TIC) studies. OAD studies were included in the Medical Information Mart for Intensive Care (MIMIC), eICU Collaborative Research Database (eICU-CRD), Amsterdam University Medical Centers Database (AmsterdamUMCdb), High Time Resolution ICU Dataset (HiRID), and Pediatric Intensive Care database. TIC studies included all other intensive care studies. Uniform manifold approximation and projection was used to visualize the corpus distribution. The BERTopic technique was used to generate 30 topic-unique identification numbers and to categorize topics into 22 topic families. Results: A total of 227,893 records were extracted. After exclusions, 145,426 articles were identified as TIC and 1301 articles as OAD studies. TIC studies experienced exponential growth over the last 2 decades, culminating in a peak of 16,378 articles in 2021, while OAD studies demonstrated a consistent upsurge since 2018. Sepsis, ventilation-related research, and pediatric intensive care were the most frequently discussed topics. TIC studies exhibited broader coverage than OAD studies, suggesting a more extensive research scope. Conclusions: This study analyzed ICU research, providing valuable insights from a large number of publications. OAD studies complement TIC studies, focusing on predictive modeling, while TIC studies capture essential qualitative information. Integrating both approaches in a complementary manner is the future direction for ICU research. Additionally, natural language processing techniques offer a transformative alternative for literature review and bibliometric analysis. %M 38630522 %R 10.2196/48330 %U https://www.jmir.org/2024/1/e48330 %U https://doi.org/10.2196/48330 %U http://www.ncbi.nlm.nih.gov/pubmed/38630522 %0 Journal Article %@ 1929-073X %I JMIR Publications %V 13 %N %P e54490 %T Application of AI in Sepsis: Citation Network Analysis and Evidence Synthesis %A Wu,MeiJung %A Islam,Md Mohaimenul %A Poly,Tahmina Nasrin %A Lin,Ming-Chin %+ Graduate Institute of Biomedical Informatics, College of Medical Science and Technology, Taipei Medical University, 250 Wuxing St, Xinyi District, Taipei, 110, Taiwan, 886 66202589, arbiter@tmu.edu.tw %K AI %K artificial intelligence %K bibliometric analysis %K bibliometric %K citation %K deep learning %K machine learning %K network analysis %K publication %K sepsis %K trend %K visualization %K VOSviewer %K Web of Science %K WoS %D 2024 %7 15.4.2024 %9 Original %J Interact J Med Res %G English %X Background: Artificial intelligence (AI) has garnered considerable attention in the context of sepsis research, particularly in personalized diagnosis and treatment. Conducting a bibliometric analysis of existing publications can offer a broad overview of the field and identify current research trends and future research directions. Objective: The objective of this study is to leverage bibliometric data to provide a comprehensive overview of the application of AI in sepsis. Methods: We conducted a search in the Web of Science Core Collection database to identify relevant articles published in English until August 31, 2023. A predefined search strategy was used, evaluating titles, abstracts, and full texts as needed. We used the Bibliometrix and VOSviewer tools to visualize networks showcasing the co-occurrence of authors, research institutions, countries, citations, and keywords. Results: A total of 259 relevant articles published between 2014 and 2023 (until August) were identified. Over the past decade, the annual publication count has consistently risen. Leading journals in this domain include Critical Care Medicine (17/259, 6.6%), Frontiers in Medicine (17/259, 6.6%), and Scientific Reports (11/259, 4.2%). The United States (103/259, 39.8%), China (83/259, 32%), United Kingdom (14/259, 5.4%), and Taiwan (12/259, 4.6%) emerged as the most prolific countries in terms of publications. Notable institutions in this field include the University of California System, Emory University, and Harvard University. The key researchers working in this area include Ritankar Das, Chris Barton, and Rishikesan Kamaleswaran. Although the initial period witnessed a relatively low number of articles focused on AI applications for sepsis, there has been a significant surge in research within this area in recent years (2014-2023). Conclusions: This comprehensive analysis provides valuable insights into AI-related research conducted in the field of sepsis, aiding health care policy makers and researchers in understanding the potential of AI and formulating effective research plans. Such analysis serves as a valuable resource for determining the advantages, sustainability, scope, and potential impact of AI models in sepsis. %M 38621231 %R 10.2196/54490 %U https://www.i-jmr.org/2024/1/e54490 %U https://doi.org/10.2196/54490 %U http://www.ncbi.nlm.nih.gov/pubmed/38621231 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 26 %N %P e52935 %T Evaluation of Large Language Model Performance and Reliability for Citations and References in Scholarly Writing: Cross-Disciplinary Study %A Mugaanyi,Joseph %A Cai,Liuying %A Cheng,Sumei %A Lu,Caide %A Huang,Jing %+ Department of Hepato-Pancreato-Biliary Surgery, Ningbo Medical Center Lihuili Hospital, Health Science Center, Ningbo University, No 1111 Jiangnan Road, Ningbo, 315000, China, 86 13819803591, huangjingonline@163.com %K large language models %K accuracy %K academic writing %K AI %K cross-disciplinary evaluation %K scholarly writing %K ChatGPT %K GPT-3.5 %K writing tool %K scholarly %K academic discourse %K LLMs %K machine learning algorithms %K NLP %K natural language processing %K citations %K references %K natural science %K humanities %K chatbot %K artificial intelligence %D 2024 %7 5.4.2024 %9 Original Paper %J J Med Internet Res %G English %X Background: Large language models (LLMs) have gained prominence since the release of ChatGPT in late 2022. Objective: The aim of this study was to assess the accuracy of citations and references generated by ChatGPT (GPT-3.5) in two distinct academic domains: the natural sciences and humanities. Methods: Two researchers independently prompted ChatGPT to write an introduction section for a manuscript and include citations; they then evaluated the accuracy of the citations and Digital Object Identifiers (DOIs). Results were compared between the two disciplines. Results: Ten topics were included, including 5 in the natural sciences and 5 in the humanities. A total of 102 citations were generated, with 55 in the natural sciences and 47 in the humanities. Among these, 40 citations (72.7%) in the natural sciences and 36 citations (76.6%) in the humanities were confirmed to exist (P=.42). There were significant disparities found in DOI presence in the natural sciences (39/55, 70.9%) and the humanities (18/47, 38.3%), along with significant differences in accuracy between the two disciplines (18/55, 32.7% vs 4/47, 8.5%). DOI hallucination was more prevalent in the humanities (42/55, 89.4%). The Levenshtein distance was significantly higher in the humanities than in the natural sciences, reflecting the lower DOI accuracy. Conclusions: ChatGPT’s performance in generating citations and references varies across disciplines. Differences in DOI standards and disciplinary nuances contribute to performance variations. Researchers should consider the strengths and limitations of artificial intelligence writing tools with respect to citation accuracy. The use of domain-specific models may enhance accuracy. %M 38578685 %R 10.2196/52935 %U https://www.jmir.org/2024/1/e52935 %U https://doi.org/10.2196/52935 %U http://www.ncbi.nlm.nih.gov/pubmed/38578685 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 8 %N %P e49411 %T Machine Learning–Based Approach for Identifying Research Gaps: COVID-19 as a Case Study %A Abd-alrazaq,Alaa %A Nashwan,Abdulqadir J %A Shah,Zubair %A Abujaber,Ahmad %A Alhuwail,Dari %A Schneider,Jens %A AlSaad,Rawan %A Ali,Hazrat %A Alomoush,Waleed %A Ahmed,Arfan %A Aziz,Sarah %+ AI Center for Precision Health, Weill Cornell Medicine-Qatar, A031, Weill Cornell Medicine-Qatar, Education City, Al Luqta St, Doha, 23435, Qatar, 974 55708599, aaa4027@qatar-med.cornell.edu %K research gaps %K research gap %K research topic %K research topics %K scientific literature %K literature review %K machine learning %K COVID-19 %K BERTopic %K topic clustering %K text analysis %K BERT %K NLP %K natural language processing %K review methods %K review methodology %K SARS-CoV-2 %K coronavirus %K COVID %D 2024 %7 5.3.2024 %9 Original Paper %J JMIR Form Res %G English %X Background: Research gaps refer to unanswered questions in the existing body of knowledge, either due to a lack of studies or inconclusive results. Research gaps are essential starting points and motivation in scientific research. Traditional methods for identifying research gaps, such as literature reviews and expert opinions, can be time consuming, labor intensive, and prone to bias. They may also fall short when dealing with rapidly evolving or time-sensitive subjects. Thus, innovative scalable approaches are needed to identify research gaps, systematically assess the literature, and prioritize areas for further study in the topic of interest. Objective: In this paper, we propose a machine learning–based approach for identifying research gaps through the analysis of scientific literature. We used the COVID-19 pandemic as a case study. Methods: We conducted an analysis to identify research gaps in COVID-19 literature using the COVID-19 Open Research (CORD-19) data set, which comprises 1,121,433 papers related to the COVID-19 pandemic. Our approach is based on the BERTopic topic modeling technique, which leverages transformers and class-based term frequency-inverse document frequency to create dense clusters allowing for easily interpretable topics. Our BERTopic-based approach involves 3 stages: embedding documents, clustering documents (dimension reduction and clustering), and representing topics (generating candidates and maximizing candidate relevance). Results: After applying the study selection criteria, we included 33,206 abstracts in the analysis of this study. The final list of research gaps identified 21 different areas, which were grouped into 6 principal topics. These topics were: “virus of COVID-19,” “risk factors of COVID-19,” “prevention of COVID-19,” “treatment of COVID-19,” “health care delivery during COVID-19,” “and impact of COVID-19.” The most prominent topic, observed in over half of the analyzed studies, was “the impact of COVID-19.” Conclusions: The proposed machine learning–based approach has the potential to identify research gaps in scientific literature. This study is not intended to replace individual literature research within a selected topic. Instead, it can serve as a guide to formulate precise literature search queries in specific areas associated with research questions that previous publications have earmarked for future exploration. Future research should leverage an up-to-date list of studies that are retrieved from the most common databases in the target area. When feasible, full texts or, at minimum, discussion sections should be analyzed rather than limiting their analysis to abstracts. Furthermore, future studies could evaluate more efficient modeling algorithms, especially those combining topic modeling with statistical uncertainty quantification, such as conformal prediction. %M 38441952 %R 10.2196/49411 %U https://formative.jmir.org/2024/1/e49411 %U https://doi.org/10.2196/49411 %U http://www.ncbi.nlm.nih.gov/pubmed/38441952 %0 Journal Article %I JMIR Publications %V 5 %N %P e57779 %T Conflicts of Interest Publication Disclosures: Descriptive Study %A Graham,S Scott %A Shiva,Jade %A Sharma,Nandini %A Barbour,Joshua B %A Majdik,Zoltan P %A Rousseau,Justin F %+ The Department of Rhetoric & Writing, The University of Texas at Austin, Parlin Hall 29, Mail Code: B5500, Austin, TX, 78712, United States, 1 5124759507, ssg@utexas.edu %K conflicts of interest %K biomedical publishing %K research integrity %K dataset %K COI %K ethical %K ethics %K publishing %K drugs %K pharmacies %K pharmacology %K pharmacotherapy %K pharmaceuticals %K medication %K disclosure %K information science %K library science %K open data %D 2024 %7 31.10.2024 %9 Original Paper %J JMIR Data %G English %X Background: Multiple lines of previous research have documented that author conflicts of interest (COI) can compromise the integrity of the biomedical research enterprise. However, continuing research that would investigate why, how, and in what circumstances COI is most risky is stymied by the difficulty in accessing disclosure statements, which are not widely represented in available databases. Objective: In this study, we describe a new open access dataset of COI disclosures extracted from published biomedical journal papers. Methods: To develop the dataset, we used ClinCalc’s Top 300 drugs lists for 2017 and 2018 to identify 319 of the most commonly used drugs. Search strategies for each product were developed using the National Library of Medicine’s and MeSH (Medical Subject Headings) browser and deployed using the eUtilities application programming interface in April 2021. We identified the 150 most relevant papers for each product and extracted COI disclosure statements from PubMed, PubMed Central, or retrieved papers as necessary. Results: Conflicts of Interest Publication Disclosures (COIPonD) is a new dataset that captures author-reported COI disclosures for biomedical research papers published in a wide range of journals and subspecialties. COIPonD captures author-reported disclosure information (including lack of disclosure) for over 38,000 PubMed-indexed papers published between 1949 and 2022. The collected papers are indexed by discussed drug products with a focus on the 319 most commonly used drugs in the United States. Conclusions: COIPonD should accelerate research efforts to understand the effects of COI on the biomedical research enterprise. In particular, this dataset should facilitate new studies of COI effects across disciplines and subspecialties. %R 10.2196/57779 %U https://data.jmir.org/2024/1/e57779 %U https://doi.org/10.2196/57779 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 10 %P e40011 %T Tracking Openness and Topic Evolution of COVID-19 Publications January 2020-March 2021: Comprehensive Bibliometric and Topic Modeling Analysis %A San Torcuato,Maider %A Bautista-Puig,Núria %A Arrizabalaga,Olatz %A Méndez,Eva %+ Innovation Unit, Biodonostia Health Research Institute, Paseo Dr Beguiristain s/n, 20014, San Sebastián, Spain, 34 943006001, olatz.arrizabalaga@biodonostia.org %K COVID-19 %K open access %K OA %K SARS-CoV-2 %K scholarly communication %K topic modeling %K research %K dissemination %K accessibility %K scientometry %K publications %K communication %K research topics %D 2022 %7 3.10.2022 %9 Review %J J Med Internet Res %G English %X Background: The COVID-19 outbreak highlighted the importance of rapid access to research. Objective: The aim of this study was to investigate research communication related to COVID-19, the level of openness of papers, and the main topics of research into this disease. Methods: Open access (OA) uptake (typologies, license use) and the topic evolution of publications were analyzed from the start of the pandemic (January 1, 2020) until the end of a year of widespread lockdown (March 1, 2021). Results: The sample included 95,605 publications; 94.1% were published in an OA form, 44% of which were published as Bronze OA. Among these OA publications, 42% do not have a license, which can limit the number of citations and thus the impact. Using a topic modeling approach, we found that articles in Hybrid and Green OA publications are more focused on patients and their effects, whereas the strategy to combat the pandemic adopted by different countries was the main topic of articles selecting publication via the Gold OA route. Conclusions: Although OA scientific production has increased, some weaknesses in OA practice, such as lack of licensing or under-researched topics, still hold back its effective use for further research. %M 36190742 %R 10.2196/40011 %U https://www.jmir.org/2022/10/e40011 %U https://doi.org/10.2196/40011 %U http://www.ncbi.nlm.nih.gov/pubmed/36190742 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 24 %N 7 %P e41046 %T Brilliant Ideas Can Come in All Sizes: Research Letters %A Kukafka,Rita %A Leung,Tiffany I %A Eysenbach,Gunther %+ JMIR Publications, Inc, 130 Queens Quay East, Unit 1100, Toronto, ON, M5A 0P6, Canada, 1 416 583 2040, tiffany.leung@jmir.org %K open science %K open access publishing %K publishing %K scholarly publishing %K scientific publishing %K research %K scientific research %K research letter %D 2022 %7 26.7.2022 %9 Editorial %J J Med Internet Res %G English %X The Journal of Medical Internet Research is pleased to offer “Research Letter” as a new article type. Research Letters are similar to original and short paper types in that they report the original results of studies in a peer-reviewed, structured scientific communication. The Research Letter article type is optimal for presenting new, early, or sometimes preliminary research findings, including interesting observations from ongoing research with significant implications that justify concise and rapid communication. %M 35881444 %R 10.2196/41046 %U https://www.jmir.org/2022/7/e41046 %U https://doi.org/10.2196/41046 %U http://www.ncbi.nlm.nih.gov/pubmed/35881444 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e45815 %T Mapping the Bibliometrics Landscape of AI in Medicine: Methodological Study %A Shi,Jin %A Bendig,David %A Vollmar,Horst Christian %A Rasche,Peter %+ Institute for Entrepreneurship, University of Münster, Geiststraße 24 - 26, Münster, 48149, Germany, 49 2518323176, jshi1@uni-muenster.de %K artificial intelligence %K AI %K AI in medicine %K medical AI taxonomy %K Python %K latent Dirichlet allocation %K LDA %K topic modeling %K unsupervised machine learning %D 2023 %7 8.12.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Artificial intelligence (AI), conceived in the 1950s, has permeated numerous industries, intensifying in tandem with advancements in computing power. Despite the widespread adoption of AI, its integration into medicine trails other sectors. However, medical AI research has experienced substantial growth, attracting considerable attention from researchers and practitioners. Objective: In the absence of an existing framework, this study aims to outline the current landscape of medical AI research and provide insights into its future developments by examining all AI-related studies within PubMed over the past 2 decades. We also propose potential data acquisition and analysis methods, developed using Python (version 3.11) and to be executed in Spyder IDE (version 5.4.3), for future analogous research. Methods: Our dual-pronged approach involved (1) retrieving publication metadata related to AI from PubMed (spanning 2000-2022) via Python, including titles, abstracts, authors, journals, country, and publishing years, followed by keyword frequency analysis and (2) classifying relevant topics using latent Dirichlet allocation, an unsupervised machine learning approach, and defining the research scope of AI in medicine. In the absence of a universal medical AI taxonomy, we used an AI dictionary based on the European Commission Joint Research Centre AI Watch report, which emphasizes 8 domains: reasoning, planning, learning, perception, communication, integration and interaction, service, and AI ethics and philosophy. Results: From 2000 to 2022, a comprehensive analysis of 307,701 AI-related publications from PubMed highlighted a 36-fold increase. The United States emerged as a clear frontrunner, producing 68,502 of these articles. Despite its substantial contribution in terms of volume, China lagged in terms of citation impact. Diving into specific AI domains, as the Joint Research Centre AI Watch report categorized, the learning domain emerged dominant. Our classification analysis meticulously traced the nuanced research trajectories across each domain, revealing the multifaceted and evolving nature of AI’s application in the realm of medicine. Conclusions: The research topics have evolved as the volume of AI studies increases annually. Machine learning remains central to medical AI research, with deep learning expected to maintain its fundamental role. Empowered by predictive algorithms, pattern recognition, and imaging analysis capabilities, the future of AI research in medicine is anticipated to concentrate on medical diagnosis, robotic intervention, and disease management. Our topic modeling outcomes provide a clear insight into the focus of AI research in medicine over the past decades and lay the groundwork for predicting future directions. The domains that have attracted considerable research attention, primarily the learning domain, will continue to shape the trajectory of AI in medicine. Given the observed growing interest, the domain of AI ethics and philosophy also stands out as a prospective area of increased focus. %M 38064255 %R 10.2196/45815 %U https://www.jmir.org/2023/1/e45815 %U https://doi.org/10.2196/45815 %U http://www.ncbi.nlm.nih.gov/pubmed/38064255 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e48529 %T Women Are Underrepresented Among Authors of Retracted Publications: Retrospective Study of 134 Medical Journals %A Sebo,Paul %A Schwarz,Joëlle %A Achtari,Margaux %A Clair,Carole %+ University Institute for Primary Care, University of Geneva, Rue Michel-Servet 1, Geneva, 1211, Switzerland, 41 223795900, paulsebo@hotmail.com %K error %K gender %K misconduct %K publication %K research %K retraction %K scientific integrity %K woman %K women %K publish %K publishing %K inequality %K retractions %K integrity %K fraud %K plagiarism %K research study %K research article %K scientific research %K journal %K retrospective %D 2023 %7 6.10.2023 %9 Research Letter %J J Med Internet Res %G English %X We examined the gender distribution of authors of retracted articles in 134 medical journals across 10 disciplines, compared it with the gender distribution of authors of all published articles, and found that women were underrepresented among authors of retracted articles, and, in particular, of articles retracted for misconduct. %M 37801343 %R 10.2196/48529 %U https://www.jmir.org/2023/1/e48529 %U https://doi.org/10.2196/48529 %U http://www.ncbi.nlm.nih.gov/pubmed/37801343 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e51584 %T Best Practices for Using AI Tools as an Author, Peer Reviewer, or Editor %A Leung,Tiffany I %A de Azevedo Cardoso,Taiane %A Mavragani,Amaryllis %A Eysenbach,Gunther %+ JMIR Publications, Inc, 130 Queens Quay East, Unit 1100, Toronto, ON, M5A 0P6, Canada, 1 416 583 2040, tiffany.leung@jmir.org %K publishing %K open access publishing %K open science %K publication policy %K science editing %K scholarly publishing %K scientific publishing %K research %K scientific research %K editorial %K artificial intelligence %K AI %D 2023 %7 31.8.2023 %9 Editorial %J J Med Internet Res %G English %X The ethics of generative artificial intelligence (AI) use in scientific manuscript content creation has become a serious matter of concern in the scientific publishing community. Generative AI has computationally become capable of elaborating research questions; refining programming code; generating text in scientific language; and generating images, graphics, or figures. However, this technology should be used with caution. In this editorial, we outline the current state of editorial policies on generative AI or chatbot use in authorship, peer review, and editorial processing of scientific and scholarly manuscripts. Additionally, we provide JMIR Publications’ editorial policies on these issues. We further detail JMIR Publications’ approach to the applications of AI in the editorial process for manuscripts in review in a JMIR Publications journal. %M 37651164 %R 10.2196/51584 %U https://www.jmir.org/2023/1/e51584 %U https://doi.org/10.2196/51584 %U http://www.ncbi.nlm.nih.gov/pubmed/37651164 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e49323 %T Open Science and Software Assistance: Commentary on “Artificial Intelligence Can Generate Fraudulent but Authentic-Looking Scientific Medical Articles: Pandora’s Box Has Been Opened” %A Ballester,Pedro L %+ Neuroscience Graduate Program, McMaster University, 1280 Main Street West, Hamilton, ON, L8S 4L8, Canada, 1 905 525 9140, pedballester@gmail.com %K artificial intelligence %K AI %K ChatGPT %K open science %K reproducibility %K software assistance %D 2023 %7 31.5.2023 %9 Commentary %J J Med Internet Res %G English %X Májovský and colleagues have investigated the important issue of ChatGPT being used for the complete generation of scientific works, including fake data and tables. The issues behind why ChatGPT poses a significant concern to research reach far beyond the model itself. Once again, the lack of reproducibility and visibility of scientific works creates an environment where fraudulent or inaccurate work can thrive. What are some of the ways in which we can handle this new situation? %M 37256656 %R 10.2196/49323 %U https://www.jmir.org/2023/1/e49323 %U https://doi.org/10.2196/49323 %U http://www.ncbi.nlm.nih.gov/pubmed/37256656 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e46924 %T Artificial Intelligence Can Generate Fraudulent but Authentic-Looking Scientific Medical Articles: Pandora’s Box Has Been Opened %A Májovský,Martin %A Černý,Martin %A Kasal,Matěj %A Komarc,Martin %A Netuka,David %+ Department of Neurosurgery and Neurooncology, First Faculty of Medicine, Charles University, U Vojenské nemocnice 1200, Prague, 16000, Czech Republic, 420 973202963, majovmar@uvn.cz %K artificial intelligence %K publications %K ethics %K neurosurgery %K ChatGPT %K language models %K fraudulent medical articles %D 2023 %7 31.5.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Artificial intelligence (AI) has advanced substantially in recent years, transforming many industries and improving the way people live and work. In scientific research, AI can enhance the quality and efficiency of data analysis and publication. However, AI has also opened up the possibility of generating high-quality fraudulent papers that are difficult to detect, raising important questions about the integrity of scientific research and the trustworthiness of published papers. Objective: The aim of this study was to investigate the capabilities of current AI language models in generating high-quality fraudulent medical articles. We hypothesized that modern AI models can create highly convincing fraudulent papers that can easily deceive readers and even experienced researchers. Methods: This proof-of-concept study used ChatGPT (Chat Generative Pre-trained Transformer) powered by the GPT-3 (Generative Pre-trained Transformer 3) language model to generate a fraudulent scientific article related to neurosurgery. GPT-3 is a large language model developed by OpenAI that uses deep learning algorithms to generate human-like text in response to prompts given by users. The model was trained on a massive corpus of text from the internet and is capable of generating high-quality text in a variety of languages and on various topics. The authors posed questions and prompts to the model and refined them iteratively as the model generated the responses. The goal was to create a completely fabricated article including the abstract, introduction, material and methods, discussion, references, charts, etc. Once the article was generated, it was reviewed for accuracy and coherence by experts in the fields of neurosurgery, psychiatry, and statistics and compared to existing similar articles. Results: The study found that the AI language model can create a highly convincing fraudulent article that resembled a genuine scientific paper in terms of word usage, sentence structure, and overall composition. The AI-generated article included standard sections such as introduction, material and methods, results, and discussion, as well a data sheet. It consisted of 1992 words and 17 citations, and the whole process of article creation took approximately 1 hour without any special training of the human user. However, there were some concerns and specific mistakes identified in the generated article, specifically in the references. Conclusions: The study demonstrates the potential of current AI language models to generate completely fabricated scientific articles. Although the papers look sophisticated and seemingly flawless, expert readers may identify semantic inaccuracies and errors upon closer inspection. We highlight the need for increased vigilance and better detection methods to combat the potential misuse of AI in scientific research. At the same time, it is important to recognize the potential benefits of using AI language models in genuine scientific writing and research, such as manuscript preparation and language editing. %M 37256685 %R 10.2196/46924 %U https://www.jmir.org/2023/1/e46924 %U https://doi.org/10.2196/46924 %U http://www.ncbi.nlm.nih.gov/pubmed/37256685 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e45482 %T Using Normative Language When Describing Scientific Findings: Randomized Controlled Trial of Effects on Trust and Credibility %A Agley,Jon %A Xiao,Yunyu %A Thompson,Esi E %A Golzarri-Arroyo,Lilian %+ Prevention Insights, Department of Applied Health Science, School of Public Health - Bloomington, Indiana University Bloomington, 809 East 9th Street, Bloomington, IN, 47405, United States, 1 812 855 3123, jagley@indiana.edu %K trust %K trust in science %K scientific communication %K meta-science %K RCT %D 2023 %7 30.3.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Scientists often make cognitive claims (eg, the results of their work) and normative claims (eg, what should be done based on those results). Yet, these types of statements contain very different information and implications. This randomized controlled trial sought to characterize the granular effects of using normative language in science communication. Objective: Our study examined whether viewing a social media post containing scientific claims about face masks for COVID-19 using both normative and cognitive language (intervention arm) would reduce perceptions of trust and credibility in science and scientists compared with an identical post using only cognitive language (control arm). We also examined whether effects were mediated by political orientation. Methods: This was a 2-arm, parallel group, randomized controlled trial. We aimed to recruit 1500 US adults (age 18+) from the Prolific platform who were representative of the US population census by cross sections of age, race/ethnicity, and gender. Participants were randomly assigned to view 1 of 2 images of a social media post about face masks to prevent COVID-19. The control image described the results of a real study (cognitive language), and the intervention image was identical, but also included recommendations from the same study about what people should do based on the results (normative language). Primary outcomes were trust in science and scientists (21-item scale) and 4 individual items related to trust and credibility; 9 additional covariates (eg, sociodemographics, political orientation) were measured and included in analyses. Results: From September 4, 2022, to September 6, 2022, 1526 individuals completed the study. For the sample as a whole (eg, without interaction terms), there was no evidence that a single exposure to normative language affected perceptions of trust or credibility in science or scientists. When including the interaction term (study arm × political orientation), there was some evidence of differential effects, such that individuals with liberal political orientation were more likely to trust scientific information from the social media post’s author if the post included normative language, and political conservatives were more likely to trust scientific information from the post’s author if the post included only cognitive language (β=0.05, 95% CI 0.00 to 0.10; P=.04). Conclusions: This study does not support the authors’ original hypotheses that single exposures to normative language can reduce perceptions of trust or credibility in science or scientists for all people. However, the secondary preregistered analyses indicate the possibility that political orientation may differentially mediate the effect of normative and cognitive language from scientists on people’s perceptions. We do not submit this paper as definitive evidence thereof but do believe that there is sufficient evidence to support additional research into this topic, which may have implications for effective scientific communication. Trial Registration: OSF Registries osf.io/kb3yh; https://osf.io/kb3yh International Registered Report Identifier (IRRID): RR2-10.2196/41747 %M 36995753 %R 10.2196/45482 %U https://www.jmir.org/2023/1/e45482 %U https://doi.org/10.2196/45482 %U http://www.ncbi.nlm.nih.gov/pubmed/36995753 %0 Journal Article %@ 2561-326X %I JMIR Publications %V 7 %N %P e44633 %T Factors Associated With Open Access Publishing Costs in Oncology Journals: Cross-sectional Observational Study %A Koong,Alex %A Gardner,Ulysses Grant %A Burton,Jason %A Stewart,Caleb %A Thompson,Petria %A Fuller,Clifton David %A Ludmir,Ethan Bernard %A Rooney,Michael Kevin %+ Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX, 77045, United States, 1 346 212 1728, mkrooney@mdanderson.org %K academic publishing %K article processing charge %K open access %K oncology %K open access publishing %K scholarly communication %D 2023 %7 16.3.2023 %9 Original Paper %J JMIR Form Res %G English %X Background: Open access (OA) publishing represents an exciting opportunity to facilitate the dissemination of scientific information to global audiences. However, OA publishing is often associated with significant article processing charges (APCs) for authors, which may thus serve as a barrier to publication. Objective: In this observational cohort study, we aimed to characterize the landscape of OA publishing in oncology and, further, identify characteristics of oncology journals that are predictive of APCs. Methods: We identified oncology journals using the SCImago Journal & Country Rank database. All journals with an OA publication option and APC data openly available were included. We searched journal websites and tabulated journal characteristics, including APC amount (in US dollars), OA model (hybrid vs full), 2-year impact factor (IF), H-index, number of citable documents, modality/treatment specific (if applicable), and continent of origin. All APCs were converted to US-dollar equivalents for final analyses. Selecting variables with significant associations in the univariable analysis, we generated a multiple regression model to identify journal characteristics independently associated with OA APC amount. An audit of a random 10% sample of the data was independently performed by 2 authors to ensure data accuracy, precision, and reproducibility. Results: Of 367 oncology journals screened, 251 met the final inclusion criteria. The median APC was US $2957 (IQR 1958-3450). The majority of journals (n=156, 62%) adopted the hybrid OA publication model and were based in Europe (n=119, 47%) or North America (n=87, 35%). The median (IQR) APC for all journals was US $2957 (1958-3540). Twenty-five (10%) journals had APCs greater than US $4000. There were 10 (4%) journals that offered OA publication with no publication charge. Univariable testing showed that journals with a greater number of citable documents (P<.001), higher 2-year IF (P<.001), higher H-index (P<.001), and those using the hybrid OA model (P<.001), or originating in Europe or North America (P<.001) tended to have higher APCs. In our multivariable model, the number of citable documents (β=US $367, SD US $133; P=.006), 2-year IF (US $1144, SD US $177; P<.001), hybrid OA publishing model (US $991, SD US $189; P<.001), and North American origin (US $838, SD US $186; P<.001) persisted as significant predictors of processing charges. Conclusions: OA publication costs are greater in oncology journals that publish more citable articles, use the hybrid OA model, have a higher IF, and are based in North America or Europe. These findings may inform targeted action to help the oncology community fully appreciate the benefits of open science. %M 36927553 %R 10.2196/44633 %U https://formative.jmir.org/2023/1/e44633 %U https://doi.org/10.2196/44633 %U http://www.ncbi.nlm.nih.gov/pubmed/36927553 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e35568 %T Automating Quality Assessment of Medical Evidence in Systematic Reviews: Model Development and Validation Study %A Šuster,Simon %A Baldwin,Timothy %A Lau,Jey Han %A Jimeno Yepes,Antonio %A Martinez Iraola,David %A Otmakhova,Yulia %A Verspoor,Karin %+ School of Computing and Information Systems, University of Melbourne, Parkville, Melbourne, 3000, Australia, 61 (03) 9035 4422, simon.suster@unimelb.edu.au %K critical appraisal %K evidence synthesis %K systematic reviews %K bias detection %K automated quality assessment %D 2023 %7 13.3.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Assessment of the quality of medical evidence available on the web is a critical step in the preparation of systematic reviews. Existing tools that automate parts of this task validate the quality of individual studies but not of entire bodies of evidence and focus on a restricted set of quality criteria. Objective: We proposed a quality assessment task that provides an overall quality rating for each body of evidence (BoE), as well as finer-grained justification for different quality criteria according to the Grading of Recommendation, Assessment, Development, and Evaluation formalization framework. For this purpose, we constructed a new data set and developed a machine learning baseline system (EvidenceGRADEr). Methods: We algorithmically extracted quality-related data from all summaries of findings found in the Cochrane Database of Systematic Reviews. Each BoE was defined by a set of population, intervention, comparison, and outcome criteria and assigned a quality grade (high, moderate, low, or very low) together with quality criteria (justification) that influenced that decision. Different statistical data, metadata about the review, and parts of the review text were extracted as support for grading each BoE. After pruning the resulting data set with various quality checks, we used it to train several neural-model variants. The predictions were compared against the labels originally assigned by the authors of the systematic reviews. Results: Our quality assessment data set, Cochrane Database of Systematic Reviews Quality of Evidence, contains 13,440 instances, or BoEs labeled for quality, originating from 2252 systematic reviews published on the internet from 2002 to 2020. On the basis of a 10-fold cross-validation, the best neural binary classifiers for quality criteria detected risk of bias at 0.78 F1 (P=.68; R=0.92) and imprecision at 0.75 F1 (P=.66; R=0.86), while the performance on inconsistency, indirectness, and publication bias criteria was lower (F1 in the range of 0.3-0.4). The prediction of the overall quality grade into 1 of the 4 levels resulted in 0.5 F1. When casting the task as a binary problem by merging the Grading of Recommendation, Assessment, Development, and Evaluation classes (high+moderate vs low+very low-quality evidence), we attained 0.74 F1. We also found that the results varied depending on the supporting information that is provided as an input to the models. Conclusions: Different factors affect the quality of evidence in the context of systematic reviews of medical evidence. Some of these (risk of bias and imprecision) can be automated with reasonable accuracy. Other quality dimensions such as indirectness, inconsistency, and publication bias prove more challenging for machine learning, largely because they are much rarer. This technology could substantially reduce reviewer workload in the future and expedite quality assessment as part of evidence synthesis. %M 36722350 %R 10.2196/35568 %U https://www.jmir.org/2023/1/e35568 %U https://doi.org/10.2196/35568 %U http://www.ncbi.nlm.nih.gov/pubmed/36722350 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 25 %N %P e42789 %T Evaluating the Ability of Open-Source Artificial Intelligence to Predict Accepting-Journal Impact Factor and Eigenfactor Score Using Academic Article Abstracts: Cross-sectional Machine Learning Analysis %A Macri,Carmelo %A Bacchi,Stephen %A Teoh,Sheng Chieh %A Lim,Wan Yin %A Lam,Lydia %A Patel,Sandy %A Slee,Mark %A Casson,Robert %A Chan,WengOnn %+ Discipline of Ophthalmology and Visual Sciences, The University of Adelaide, North Terrace, Adelaide, 5000, Australia, 61 468951763, carmelo.macri@adelaide.edu.au %K journal impact factor %K artificial intelligence %K ophthalmology %K radiology %K neurology %K eye %K neuroscience %K impact factor %K research quality %K journal recommender %K publish %K open source %K predict %K machine learning %K academic journal %K scientometric %K scholarly literature %D 2023 %7 7.3.2023 %9 Original Paper %J J Med Internet Res %G English %X Background: Strategies to improve the selection of appropriate target journals may reduce delays in disseminating research results. Machine learning is increasingly used in content-based recommender algorithms to guide journal submissions for academic articles. Objective: We sought to evaluate the performance of open-source artificial intelligence to predict the impact factor or Eigenfactor score tertile using academic article abstracts. Methods: PubMed-indexed articles published between 2016 and 2021 were identified with the Medical Subject Headings (MeSH) terms “ophthalmology,” “radiology,” and “neurology.” Journals, titles, abstracts, author lists, and MeSH terms were collected. Journal impact factor and Eigenfactor scores were sourced from the 2020 Clarivate Journal Citation Report. The journals included in the study were allocated percentile ranks based on impact factor and Eigenfactor scores, compared with other journals that released publications in the same year. All abstracts were preprocessed, which included the removal of the abstract structure, and combined with titles, authors, and MeSH terms as a single input. The input data underwent preprocessing with the inbuilt ktrain Bidirectional Encoder Representations from Transformers (BERT) preprocessing library before analysis with BERT. Before use for logistic regression and XGBoost models, the input data underwent punctuation removal, negation detection, stemming, and conversion into a term frequency-inverse document frequency array. Following this preprocessing, data were randomly split into training and testing data sets with a 3:1 train:test ratio. Models were developed to predict whether a given article would be published in a first, second, or third tertile journal (0-33rd centile, 34th-66th centile, or 67th-100th centile), as ranked either by impact factor or Eigenfactor score. BERT, XGBoost, and logistic regression models were developed on the training data set before evaluation on the hold-out test data set. The primary outcome was overall classification accuracy for the best-performing model in the prediction of accepting journal impact factor tertile. Results: There were 10,813 articles from 382 unique journals. The median impact factor and Eigenfactor score were 2.117 (IQR 1.102-2.622) and 0.00247 (IQR 0.00105-0.03), respectively. The BERT model achieved the highest impact factor tertile classification accuracy of 75.0%, followed by an accuracy of 71.6% for XGBoost and 65.4% for logistic regression. Similarly, BERT achieved the highest Eigenfactor score tertile classification accuracy of 73.6%, followed by an accuracy of 71.8% for XGBoost and 65.3% for logistic regression. Conclusions: Open-source artificial intelligence can predict the impact factor and Eigenfactor score of accepting peer-reviewed journals. Further studies are required to examine the effect on publication success and the time-to-publication of such recommender systems. %M 36881455 %R 10.2196/42789 %U https://www.jmir.org/2023/1/e42789 %U https://doi.org/10.2196/42789 %U http://www.ncbi.nlm.nih.gov/pubmed/36881455 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 12 %P e34051 %T Citation Advantage of Promoted Articles in a Cross-Publisher Distribution Platform: 36-Month Follow-up to a Randomized Controlled Trial %A Kudlow,Paul %A Brown,Tashauna %A Eysenbach,Gunther %+ Department of Psychiatry, University of Toronto, 250 College Street, Toronto, ON, M5T 1R8, Canada, 1 416 480 6100, paul.kudlow@gmail.com %K knowledge translation %K knowledge %K dissemination %K digital knowledge translation %K digital publishing %K e-publishing %K open access %K scientometrics %K infometrics %D 2021 %7 10.12.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: There are limited evidence-based strategies that have been shown to increase the rate at which peer-reviewed articles are cited. In a previously reported randomized controlled trial, we demonstrated that promotion of article links in an online cross-publisher distribution platform (TrendMD) persistently augments citation rates after 12 months, leading to a statistically significant 50% increase in citations relative to the control. Objective: This study aims to investigate if the citation advantage of promoted articles upholds after 36 months. Methods: A total of 3200 published articles in 64 peer-reviewed journals across 8 subject areas were block randomized at the subject level to either the TrendMD group (n=1600) or the control group (n=1600) of the study. Articles were promoted in the TrendMD Network for 6 months. We compared the citation rates in both groups after 36 months. Results: At 36 months, we found the citation advantage endured; articles randomized to TrendMD showed a 28% increase in mean citations relative to the control. The difference in mean citations at 36 months for articles randomized to TrendMD versus the control was 10.52 (95% CI 3.79-17.25) and was statistically significant (P=.001). Conclusions: To our knowledge, this is the first randomized controlled trial to demonstrate how a postpublication article promotion intervention can be used to persistently augment citations of peer-reviewed articles. TrendMD is an efficient digital tool for knowledge translation and dissemination to targeted audiences to facilitate the uptake of research. %M 34890350 %R 10.2196/34051 %U https://www.jmir.org/2021/12/e34051 %U https://doi.org/10.2196/34051 %U http://www.ncbi.nlm.nih.gov/pubmed/34890350 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 11 %P e25394 %T Information and Scientific Impact of Advanced Therapies in the Age of Mass Media: Altmetrics-Based Analysis of Tissue Engineering %A Santisteban-Espejo,Antonio %A Martin-Piedra,Miguel Angel %A Campos,Antonio %A Moran-Sanchez,Julia %A Cobo,Manuel J %A Pacheco-Serrano,Ana I %A Moral-Munoz,Jose A %+ Department of Histology, Tissue Engineering Group, University of Granada, School of Medicine, Avda de la Ilustración, 11, Granada, 18016, Spain, 34 958241000 ext 40703, mmartin@ugr.es %K advanced therapies %K tissue engineering %K scientometrics %K altmetrics %K online %K web %K communication of science %D 2021 %7 26.11.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Tissue engineering (TE) constitutes a multidisciplinary field aiming to construct artificial tissues to regenerate end-stage organs. Its development has taken place since the last decade of the 20th century, entailing a clinical revolution. TE research groups have worked and shared relevant information in the mass media era. Thus, it would be interesting to study the online dimension of TE research and to compare it with traditional measures of scientific impact. Objective: The objective of this study was to evaluate the online dimension of TE documents from 2012 to 2018 using metadata obtained from the Web of Science (WoS) and Altmetric and to develop a prediction equation for the impact of TE documents from altmetric scores. Methods: We analyzed 10,112 TE documents through descriptive and statistical methods. First, the TE temporal evolution was exposed for WoS and 15 online platforms (news, blogs, policy, Twitter, patents, peer review, Weibo, Facebook, Wikipedia, Google, Reddit, F1000, Q&A, video, and Mendeley Readers). The 10 most cited TE original articles were ranked according to the normalized WoS citations and the normalized Altmetric Attention Score. Second, to better comprehend the TE online framework, correlation and factor analyses were performed based on the suitable results previously obtained for the Bartlett sphericity and Kaiser–Meyer–Olkin tests. Finally, the linear regression model was applied to elucidate the relation between academics and online media and to construct a prediction equation for TE from altmetrics data. Results: TE dynamic shows an upward trend in WoS citations, Twitter, Mendeley Readers, and Altmetric Scores. However, WoS and Altmetric rankings for the most cited documents clearly differ. When compared, the best correlation results were obtained for Mendeley Readers and WoS (ρ=0.71). In addition, the factor analysis identified 6 factors that could explain the previously observed differences between academic institutions and the online platforms evaluated. At this point, the mathematical model constructed is able to predict and explain more than 40% of TE WoS citations from Altmetric scores. Conclusions: Scientific information related to the construction of bioartificial tissues increasingly reaches society through different online media. Because the focus of TE research importantly differs when the academic institutions and online platforms are compared, basic and clinical research groups, academic institutions, and health politicians should make a coordinated effort toward the design and implementation of adequate strategies for information diffusion and population health education. %M 34842548 %R 10.2196/25394 %U https://www.jmir.org/2021/11/e25394 %U https://doi.org/10.2196/25394 %U http://www.ncbi.nlm.nih.gov/pubmed/34842548 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 9 %P e21974 %T Evolutionary Overview of Consumer Health Informatics: Bibliometric Study on the Web of Science from 1999 to 2019 %A Ouyang,Wei %A Xie,Wenzhao %A Xin,Zirui %A He,Haiyan %A Wen,Tingxiao %A Peng,Xiaoqing %A Dai,Pingping %A Yuan,Yifeng %A Liu,Fei %A Chen,Yang %A Luo,Aijing %+ The Second Xiangya Hospital, Central South University, 139 Renmin Middle Road, Changsha, 410011, China, 86 073188618316, luoaijing@163.com %K consumer health informatics %K consumer health information %K thematic evaluation %K co-word analysis %K informatics %K SciMAT %D 2021 %7 9.9.2021 %9 Review %J J Med Internet Res %G English %X Background: Consumer health informatics (CHI) originated in the 1990s. With the rapid development of computer and information technology for health decision making, an increasing number of consumers have obtained health-related information through the internet, and CHI has also attracted the attention of an increasing number of scholars. Objective: The aim of this study was to analyze the research themes and evolution characteristics of different study periods and to discuss the dynamic evolution path and research theme rules in a time-series framework from the perspective of a strategy map and a data flow in CHI. Methods: The Web of Science core collection database of the Institute for Scientific Information was used as the data source to retrieve relevant articles in the field of CHI. SciMAT was used to preprocess the literature data and construct the overlapping map, evolution map, strategic diagram, and cluster network characterized by keywords. Besides, a bibliometric analysis of the general characteristics, the evolutionary characteristics of the theme, and the evolutionary path of the theme was conducted. Results: A total of 986 articles were obtained after the retrieval, and 931 articles met the document-type requirement. In the past 21 years, the number of articles increased every year, with a remarkable growth after 2015. The research content in 4 different study periods formed the following 38 themes: patient education, medicine, needs, and bibliographic database in the 1999-2003 study period; world wide web, patient education, eHealth, patients, medication, terminology, behavior, technology, and disease in the 2004-2008 study period; websites, information seeking, physicians, attitudes, technology, risk, food labeling, patient, strategies, patient education, and eHealth in the 2009-2014 study period; and electronic medical records, health information seeking, attitudes, health communication, breast cancer, health literacy, technology, natural language processing, user-centered design, pharmacy, academic libraries, costs, internet utilization, and online health information in the 2015-2019 study period. Besides, these themes formed 10 evolution paths in 3 research directions: patient education and intervention, consumer demand attitude and behavior, and internet information technology application. Conclusions: Averaging 93 publications every year since 2015, CHI research is in a rapid growth period. The research themes mainly focus on patient education, health information needs, health information search behavior, health behavior intervention, health literacy, health information technology, eHealth, and other aspects. Patient education and intervention research, consumer demand, attitude, and behavior research comprise the main theme evolution path, whose evolution process has been relatively stable. This evolution path will continue to become the research hotspot in this field. Research on the internet and information technology application is a secondary theme evolution path with development potential. %M 34499042 %R 10.2196/21974 %U https://www.jmir.org/2021/9/e21974 %U https://doi.org/10.2196/21974 %U http://www.ncbi.nlm.nih.gov/pubmed/34499042 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 9 %N 9 %P e30401 %T Machine Learning Approaches to Retrieve High-Quality, Clinically Relevant Evidence From the Biomedical Literature: Systematic Review %A Abdelkader,Wael %A Navarro,Tamara %A Parrish,Rick %A Cotoi,Chris %A Germini,Federico %A Iorio,Alfonso %A Haynes,R Brian %A Lokker,Cynthia %+ Health Information Research Unit, Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main St W, CRL Building, First Floor, Hamilton, ON, L8S 4K1, Canada, 1 647 563 5732, Abdelkaw@mcmaster.ca %K machine learning %K bioinformatics %K information retrieval %K evidence-based medicine %K literature databases %K systematic review %K accuracy %K medical literature %K clinical support %K clinical care %D 2021 %7 9.9.2021 %9 Review %J JMIR Med Inform %G English %X Background: The rapid growth of the biomedical literature makes identifying strong evidence a time-consuming task. Applying machine learning to the process could be a viable solution that limits effort while maintaining accuracy. Objective: The goal of the research was to summarize the nature and comparative performance of machine learning approaches that have been applied to retrieve high-quality evidence for clinical consideration from the biomedical literature. Methods: We conducted a systematic review of studies that applied machine learning techniques to identify high-quality clinical articles in the biomedical literature. Multiple databases were searched to July 2020. Extracted data focused on the applied machine learning model, steps in the development of the models, and model performance. Results: From 3918 retrieved studies, 10 met our inclusion criteria. All followed a supervised machine learning approach and applied, from a limited range of options, a high-quality standard for the training of their model. The results show that machine learning can achieve a sensitivity of 95% while maintaining a high precision of 86%. Conclusions: Machine learning approaches perform well in retrieving high-quality clinical studies. Performance may improve by applying more sophisticated approaches such as active learning and unsupervised machine learning approaches. %M 34499041 %R 10.2196/30401 %U https://medinform.jmir.org/2021/9/e30401 %U https://doi.org/10.2196/30401 %U http://www.ncbi.nlm.nih.gov/pubmed/34499041 %0 Journal Article %@ 2562-0959 %I JMIR Publications %V 4 %N 2 %P e29282 %T Global Burden of Skin Disease Representation in the Literature: Bibliometric Analysis %A Pulsipher,Kayd J %A Szeto,Mindy D %A Rundle,Chandler W %A Presley,Colby L %A Laughter,Melissa R %A Dellavalle,Robert P %+ Dermatology Service, Rocky Mountain Regional Medical Center, US Department of Veteran Affairs, 1700 N Wheeling St, Rm E1-342, Aurora, CO, 80045, United States, 1 720 857 5562, robert.dellavalle@cuanschutz.edu %K global burden of disease %K global health %K global dermatology %K disability-adjusted life years %K GBD %K DALYs %K journalology %K dermatology %K skin disorders %D 2021 %7 31.8.2021 %9 Original Paper %J JMIR Dermatol %G English %X Background: The global burden of skin disease may be reduced through research efforts focused on skin diseases with the highest reported disability-adjusted life years. Objective: This study evaluates the representation of dermatologic conditions comprising the highest disability-adjusted life years in dermatology literature to identify areas that could benefit from greater research focus. Methods: The top 10 skin disorders according to their respective disability-adjusted life years as per the 2013 Global Burden of Disease were identified using previous studies. The top 5 dermatology journals ranked by the 2019 h-index were also identified. A PubMed search of each journal was performed using individual skin disease terms. From 2015 to 2020, all indexed publications pertaining to each disease were recorded and compared to the total number of publications for each journal surveyed. Results: A total of 19,727 papers were published in the 5 journals over the span of 2015-2020. Although melanoma ranked as the eighth highest in disability-adjusted life years, it had the highest representation in the literature (1995/19,727, 10.11%). Melanoma was followed in representation by psoriasis (1936/19,727, 9.81%) and dermatitis (1927/19,727, 9.77%). These 3 conditions comprised a total of 29.69% (5858/19,727) of the total publications, while the remaining 7 skin conditions were represented by a combined 6.79% (1341/19,727) of the total publications. Conclusions: This research identifies gaps in the literature related to the top skin diseases contributing to the global burden of disease. Our study provides insight into future opportunities of focused research on less-studied skin diseases to potentially aid in reducing the global burden of skin disease. %M 37632830 %R 10.2196/29282 %U https://derma.jmir.org/2021/2/e29282 %U https://doi.org/10.2196/29282 %U http://www.ncbi.nlm.nih.gov/pubmed/37632830 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 7 %P e26378 %T The Use of Twitter by Medical Journals: Systematic Review of the Literature %A Erskine,Natalie %A Hendricks,Sharief %+ Division of Physiological Sciences, Department of Human Biology, Faculty of Health Sciences, University of Cape Town, Boundary Road, Newlands, Cape Town, 7725, South Africa, 27 0728716033, natserskine@mweb.co.za %K Twitter %K social media %K medical journals %K impact %D 2021 %7 28.7.2021 %9 Review %J J Med Internet Res %G English %X Background: Medical journals use Twitter to engage and disseminate their research articles and implement a range of strategies to maximize reach and impact. Objective: This study aims to systematically review the literature to synthesize and describe the different Twitter strategies used by medical journals and their effectiveness on journal impact and readership metrics. Methods: A systematic search of the literature before February 2020 in four electronic databases (PubMed, Web of Science, Scopus, and ScienceDirect) was conducted. Articles were reviewed using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines. Results: The search identified 44 original research studies that evaluated Twitter strategies implemented by medical journals and analyzed the relationship between Twitter metrics and alternative and citation-based metrics. The key findings suggest that promoting publications on Twitter improves citation-based and alternative metrics for academic medical journals. Moreover, implementing different Twitter strategies maximizes the amount of attention that publications and journals receive. The four key Twitter strategies implemented by many medical journals are tweeting the title and link of the article, infographics, podcasts, and hosting monthly internet-based journal clubs. Each strategy was successful in promoting the publications. However, different metrics were used to measure success. Conclusions: Four key Twitter strategies are implemented by medical journals: tweeting the title and link of the article, infographics, podcasts, and hosting monthly internet-based journal clubs. In this review, each strategy was successful in promoting publications but used different metrics to measure success. Thus, it is difficult to conclude which strategy is most effective. In addition, the four strategies have different costs and effects on dissemination and readership. We recommend that journals and researchers incorporate a combination of Twitter strategies to maximize research impact and capture audiences with a variety of learning methods. %M 34319238 %R 10.2196/26378 %U https://www.jmir.org/2021/7/e26378 %U https://doi.org/10.2196/26378 %U http://www.ncbi.nlm.nih.gov/pubmed/34319238 %0 Journal Article %@ 2562-0959 %I JMIR Publications %V 4 %N 2 %P e30126 %T Conflicts of Interest in “Throwaway” Dermatology Publications: Analysis of the Open Payments Database %A Roman,Jorge %A Elpern,David J %+ The Ronald O. Perelman Department of Dermatology, New York University Grossman School of Medicine, 11th Fl, 240 East 38th St, New York, NY, 10016, United States, 1 2122635889, jorge.roman@nyulangone.org %K pharmaceutical industry %K continuing medical education %K dermatology %K influence %K payments %K Open Payments database %K publications %K medical education %K compensation %K consulting %K dermatologists %D 2021 %7 22.7.2021 %9 Original Paper %J JMIR Dermatol %G English %X Background: Dermatology journals, periodicals, editorials, and news magazines are influential resources that are not uniformly regulated and subject to influence from the pharmaceutical industry. This study evaluates industry payments to physician editorial board members of common dermatology publications, including “throwaway” publications. Objective: The aim of this study was to characterize the extent and nature of industry payments to editorial board members of different dermatologic publications in order to ascertain differences in payments between different types of publications. Methods: A list of editorial board members was compiled from a collection of clinical dermatology publications received over a 3-month period. Data from the Open Payments database from 2013 to 2019 were collected, and analysis of payments data was performed. Results: Ten publications were evaluated, and payments data for 466 physicians were analyzed. The total compensation across all years was US $75,622,369.64. Consulting, services other than consulting, and travel or lodging payments constituted most of the payments. A fraction of dermatologists received the majority of payments. The top payers were manufacturers of biologic medications. Payment amounts were higher for throwaway publications compared to peer-reviewed journals. Conclusions: Editorial board members of dermatology publications received substantial payments from the pharmaceutical industry. A minority of physicians receive the lion’s share of payments from industry. “Throwaway” publications have more financial conflict of interest than do peer-reviewed journals. The impact of these conflicts of interest on patient care, physicians' practice patterns, and patient perception of physicians is noteworthy. %M 37632829 %R 10.2196/30126 %U https://derma.jmir.org/2021/2/e30126 %U https://doi.org/10.2196/30126 %U http://www.ncbi.nlm.nih.gov/pubmed/37632829 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 7 %P e26995 %T Online Interactive Platform for COVID-19 Literature Visual Analytics: Platform Development Study %A Moran,Addy %A Hampton,Shawn %A Dowson,Scott %A Dagdelen,John %A Trewartha,Amalie %A Ceder,Gerbrand %A Persson,Kristin %A Saxon,Elise %A Barker,Andrew %A Charles,Lauren %A Webb-Robertson,Bobbie-Jo %+ Pacific Northwest National Laboratory, 902 Battelle Blvd, J4-18, Richland, WA, 99352, United States, 1 5093752292, bj@pnnl.gov %K COVID-19 %K visual analytics %K natural language processing %K scientific literature %K software %K online platform %K literature %K interactive %K publish %K research %K tool %K pattern %K usability %D 2021 %7 16.7.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Papers on COVID-19 are being published at a high rate and concern many different topics. Innovative tools are needed to aid researchers to find patterns in this vast amount of literature to identify subsets of interest in an automated fashion. Objective: We present a new online software resource with a friendly user interface that allows users to query and interact with visual representations of relationships between publications. Methods: We publicly released an application called PLATIPUS (Publication Literature Analysis and Text Interaction Platform for User Studies) that allows researchers to interact with literature supplied by COVIDScholar via a visual analytics platform. This tool contains standard filtering capabilities based on authors, journals, high-level categories, and various research-specific details via natural language processing and dozens of customizable visualizations that dynamically update from a researcher’s query. Results: PLATIPUS is available online and currently links to over 100,000 publications and is still growing. This application has the potential to transform how COVID-19 researchers use public literature to enable their research. Conclusions: The PLATIPUS application provides the end user with a variety of ways to search, filter, and visualize over 100,00 COVID-19 publications. %M 34138726 %R 10.2196/26995 %U https://www.jmir.org/2021/7/e26995 %U https://doi.org/10.2196/26995 %U http://www.ncbi.nlm.nih.gov/pubmed/34138726 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 6 %P e26956 %T Social Media and Research Publication Activity During Early Stages of the COVID-19 Pandemic: Longitudinal Trend Analysis %A Taneja,Sonia L %A Passi,Monica %A Bhattacharya,Sumona %A Schueler,Samuel A %A Gurram,Sandeep %A Koh,Christopher %+ Liver Diseases Branch, National Institutes of Diabetes and Digestive and Kidney Diseases, 10 Center Drive, Clinical Research Center, 5-2740, Bethesda, MD, 20892, United States, 1 301 443 9402, christopher.koh@nih.gov %K coronavirus %K COVID-19 %K social media %K gastroenterology %K SARS-CoV-2 %K research %K literature %K dissemination %K Twitter %K preprint %D 2021 %7 17.6.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: The COVID-19 pandemic has highlighted the importance of rapid dissemination of scientific and medical discoveries. Current platforms available for the distribution of scientific and clinical research data and information include preprint repositories and traditional peer-reviewed journals. In recent times, social media has emerged as a helpful platform to share scientific and medical discoveries. Objective: This study aimed to comparatively analyze activity on social media (specifically, Twitter) and that related to publications in the form of preprint and peer-reviewed journal articles in the context of COVID-19 and gastroenterology during the early stages of the COVID-19 pandemic. Methods: COVID-19–related data from Twitter (tweets and user data) and articles published in preprint servers (bioRxiv and medRxiv) as well as in the PubMed database were collected and analyzed during the first 6 months of the pandemic, from December 2019 through May 2020. Global and regional geographic and gastrointestinal organ–specific social media trends were compared to preprint and publication activity. Any relationship between Twitter activity and preprint articles published and that between Twitter activity and PubMed articles published overall, by organ system, and by geographic location were identified using Spearman’s rank-order correlation. Results: Over the 6-month period, 73,079 tweets from 44,609 users, 7164 journal publications, and 4702 preprint publications were retrieved. Twitter activity (ie, number of tweets) peaked in March 2020, whereas preprint and publication activity (ie, number of articles published) peaked in April 2020. Overall, strong correlations were identified between trends in Twitter activity and preprint and publication activity (P<.001 for both). COVID-19 data across the three platforms mainly concentrated on pulmonology or critical care, but when analyzing the field of gastroenterology specifically, most tweets pertained to pancreatology, most publications focused on hepatology, and most preprints covered hepatology and luminal gastroenterology. Furthermore, there were significant positive associations between trends in Twitter and publication activity for all gastroenterology topics (luminal gastroenterology: P=.009; hepatology and inflammatory bowel disease: P=.006; gastrointestinal endoscopy: P=.007), except pancreatology (P=.20), suggesting that Twitter activity did not correlate with publication activity for this topic. Finally, Twitter activity was the highest in the United States (7331 tweets), whereas PubMed activity was the highest in China (1768 publications). Conclusions: The COVID-19 pandemic has highlighted the potential of social media as a vehicle for disseminating scientific information during a public health crisis. Sharing and spreading information on COVID-19 in a timely manner during the pandemic has been paramount; this was achieved at a much faster pace on social media, particularly on Twitter. Future investigation could demonstrate how social media can be used to augment and promote scholarly activity, especially as the world begins to increasingly rely on digital or virtual platforms. Scientists and clinicians should consider the use of social media in augmenting public awareness regarding their scholarly pursuits. %M 33974550 %R 10.2196/26956 %U https://www.jmir.org/2021/6/e26956 %U https://doi.org/10.2196/26956 %U http://www.ncbi.nlm.nih.gov/pubmed/33974550 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 6 %P e17137 %T Team Science in Precision Medicine: Study of Coleadership and Coauthorship Across Health Organizations %A An,Ning %A Mattison,John %A Chen,Xinyu %A Alterovitz,Gil %+ Brigham and Women's Hospital, Harvard Medical School, 75 Francis Street, Boston, MA, 02115, United States, 1 617 329 1445, gil_alterovitz@hms.harvard.edu %K precision medicine %K team science %D 2021 %7 14.6.2021 %9 Viewpoint %J J Med Internet Res %G English %X Background: Interdisciplinary collaborations bring lots of benefits to researchers in multiple areas, including precision medicine. Objective: This viewpoint aims at studying how cross-institution team science would affect the development of precision medicine. Methods: Publications of organizations on the eHealth Catalogue of Activities were collected in 2015 and 2017. The significance of the correlation between coleadership and coauthorship among different organizations was calculated using the Pearson chi-square test of independence. Other nonparametric tests examined whether organizations with coleaders publish more and better papers than organizations without coleaders. Results: A total of 374 publications from 69 organizations were analyzed in 2015, and 7064 papers from 87 organizations were analyzed in 2017. Organizations with coleadership published more papers (P<.001, 2015 and 2017), which received higher citations (Z=–13.547, P<.001, 2017), compared to those without coleadership. Organizations with coleaders tended to publish papers together (P<.001, 2015 and 2017). Conclusions: Our findings suggest that organizations in the field of precision medicine could greatly benefit from institutional-level team science. As a result, stronger collaboration is recommended. %M 34125070 %R 10.2196/17137 %U https://www.jmir.org/2021/6/e17137 %U https://doi.org/10.2196/17137 %U http://www.ncbi.nlm.nih.gov/pubmed/34125070 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 5 %P e28859 %T Evaluating Scholars’ Impact and Influence: Cross-sectional Study of the Correlation Between a Novel Social Media–Based Score and an Author-Level Citation Metric %A Oliveira J e Silva,Lucas %A Maldonado,Graciela %A Brigham,Tara %A Mullan,Aidan F %A Utengen,Audun %A Cabrera,Daniel %+ Department of Emergency Medicine, Mayo Clinic Rochester, 200 First Street SW, Rochester, MN, 55905, United States, 1 507 255 4399, cabrera.daniel@mayo.edu %K social media %K Twitter %K journal impact factor %K h-index %K digital scholarship %K digital platform %K Scopus %K metrics %K scientometrics %K altmetrics %K metrics %K stakeholders %K health care %K digital health care %D 2021 %7 31.5.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: The development of an author-level complementary metric could play a role in the process of academic promotion through objective evaluation of scholars’ influence and impact. Objective: The objective of this study was to evaluate the correlation between the Healthcare Social Graph (HSG) score, a novel social media influence and impact metric, and the h-index, a traditional author-level metric. Methods: This was a cross-sectional study of health care stakeholders with a social media presence randomly sampled from the Symplur database in May 2020. We performed stratified random sampling to obtain a representative sample with all strata of HSG scores. We manually queried the h-index in two reference-based databases (Scopus and Google Scholar). Continuous features (HSG score and h-index) from the included profiles were summarized as the median and IQR. We calculated the Spearman correlation coefficients (ρ) to evaluate the correlation between the HSG scores and h-indexes obtained from Google Scholar and Scopus. Results: A total of 286 (31.2%) of the 917 stakeholders had a Google Scholar h-index available. The median HSG score for these profiles was 61.1 (IQR 48.2), and the median h-index was 14.5 (IQR 26.0). For the 286 subjects with the HSG score and Google Scholar h-index available, the Spearman correlation coefficient ρ was 0.1979 (P<.001), indicating a weak positive correlation between these two metrics. A total of 715 (78%) of 917 stakeholders had a Scopus h-index available. The median HSG score for these profiles was 57.6 (IQR 46.4), and the median h-index was 7 (IQR 16). For the 715 subjects with the HSG score and Scopus h-index available, ρ was 0.2173 (P<.001), also indicating a weak positive correlation. Conclusions: We found a weak positive correlation between a novel author-level complementary metric and the h-index. More than a chiasm between traditional citation metrics and novel social media–based metrics, our findings point toward a bridge between the two domains. %M 34057413 %R 10.2196/28859 %U https://www.jmir.org/2021/5/e28859 %U https://doi.org/10.2196/28859 %U http://www.ncbi.nlm.nih.gov/pubmed/34057413 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 5 %P e26666 %T Characterization of an Open-Access Medical News Platform’s Readership During the COVID-19 Pandemic: Retrospective Observational Study %A Chan,Alex K %A Wu,Constance %A Cheung,Andrew %A Succi,Marc D %+ Department of Radiology, Massachusetts General Hospital, 55 Fruit Street, Boston, MA, 02114, United States, 1 6179359144, msucci@mgh.harvard.edu %K COVID-19 %K internet %K medical news %K text summaries %K readership trends %K news %K media %K open access %K literature %K web-based health information %K survey %K cross-sectional %K trend %D 2021 %7 25.5.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: There are many alternatives to direct journal access, such as podcasts, blogs, and news sites, that allow physicians and the general public to stay up to date with medical literature. However, there is a scarcity of literature that investigates the readership characteristics of open-access medical news sites and how these characteristics may have shifted during the COVID-19 pandemic. Objective: This study aimed to assess readership and survey data to characterize open-access medical news readership trends related to the COVID-19 pandemic and overall readership trends regarding pandemic-related information delivery. Methods: Anonymous, aggregate readership data were obtained from 2 Minute Medicine, an open-access, physician-run medical news organization that has published over 8000 original, physician-written texts and visual summaries of new medical research since 2013. In this retrospective observational study, the average number of article views, number of actions (defined as the sum of the number of views, shares, and outbound link clicks), read times, and bounce rates (probability of leaving a page in <30 s) were compared between COVID-19 articles published from January 1 to May 31, 2020 (n=40) and non–COVID-19 articles (n=145) published in the same time period. A voluntary survey was also sent to subscribed 2 Minute Medicine readers to further characterize readership demographics and preferences, which were scored on a Likert scale. Results: COVID-19 articles had a significantly higher median number of views than non–COVID-19 articles (296 vs 110; U=748.5; P<.001). There were no significant differences in average read times (P=.12) or bounce rates (P=.12). Non–COVID-19 articles had a higher median number of actions than COVID-19 articles (2.9 vs 2.5; U=2070.5; P=.02). On a Likert scale of 1 (strongly disagree) to 5 (strongly agree), our survey data revealed that 65.5% (78/119) of readers agreed or strongly agreed that they preferred staying up to date with emerging literature about COVID-19 by using sources such as 2 Minute Medicine instead of journals. A greater proportion of survey respondents also indicated that open-access news sources were one of their primary sources for staying informed (86/120, 71.7%) compared to the proportion who preferred direct journal article access (61/120, 50.8%). The proportion of readers indicating they were reading one or less full-length medical studies a month were lower following introduction to 2 Minute Medicine compared to prior (21/120, 17.5% vs 38/120, 31.6%; P=.005). Conclusions: The readership significantly increased for one open-access medical literature platform during the pandemic. This reinforces the idea that open-access, physician-written sources of medical news represent an important alternative to direct journal access for readers who want to stay up to date with medical literature. %M 33866307 %R 10.2196/26666 %U https://www.jmir.org/2021/5/e26666 %U https://doi.org/10.2196/26666 %U http://www.ncbi.nlm.nih.gov/pubmed/33866307 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 5 %P e29156 %T Authors' Reply to: COVID-19 as a “Force Majeure” for Non–COVID-19 Clinical and Translational Research. Comment on “Analysis of Scientific Publications During the Early Phase of the COVID-19 Pandemic: Topic Modeling Study” %A Älgå,Andreas %A Eriksson,Oskar %A Nordberg,Martin %+ Department of Clinical Science and Education, Södersjukhuset, Karolinska Institutet, Sjukhusbacken 10, Stockholm, 11883, Sweden, 46 86161000, andreas.alga@ki.se %K COVID-19 %K SARS-CoV-2 %K coronavirus %K pandemic %K topic modeling %K research %K literature %K medical research %K publishing %K force majeure %D 2021 %7 20.5.2021 %9 Letter to the Editor %J J Med Internet Res %G English %X %M 33989170 %R 10.2196/29156 %U https://www.jmir.org/2021/5/e29156 %U https://doi.org/10.2196/29156 %U http://www.ncbi.nlm.nih.gov/pubmed/33989170 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 5 %P e27937 %T COVID-19 as a “Force Majeure” for Non–COVID-19 Clinical and Translational Research. Comment on “Analysis of Scientific Publications During the Early Phase of the COVID-19 Pandemic: Topic Modeling Study” %A Milovanovic,Petar %A Dumic,Igor %+ Faculty of Medicine, University of Belgrade, Dr Subotica 4/2, Belgrade, 11000, 381 66413225, drpmilovanovic@gmail.com %K COVID-19 %K SARS-CoV-2 %K coronavirus %K pandemic %K topic modeling %K research %K literature %K medical research %K publishing %K force majeure %D 2021 %7 20.5.2021 %9 Letter to the Editor %J J Med Internet Res %G English %X %M 33989167 %R 10.2196/27937 %U https://www.jmir.org/2021/5/e27937 %U https://doi.org/10.2196/27937 %U http://www.ncbi.nlm.nih.gov/pubmed/33989167 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 5 %P e25077 %T Establishing and Facilitating Large-Scale Manuscript Collaborations via Social Media: Novel Method and Tools for Replication %A Acquaviva,Kimberly D %+ School of Nursing, University of Virginia, 4005 McLeod Hall, Charlottesville, VA, 22903, United States, 1 202 423 0984, kda8xj@virginia.edu %K social media %K crowdsourcing %K collaboration %K health professions %K medicine %K scholarship %K literature %K research %D 2021 %7 17.5.2021 %9 Tutorial %J J Med Internet Res %G English %X Background: Authorship teams in the health professions are typically composed of scholars who are acquainted with one another before a manuscript is written. Even if a scholar has identified a diverse group of collaborators outside their usual network, writing an article with a large number of co-authors poses significant logistical challenges. Objective: This paper describes a novel method for establishing and facilitating large-scale manuscript collaborations via social media. Methods: On September 11, 2020, I used the social media platform Twitter to invite people to collaborate on an article I had drafted. Anyone who wanted to collaborate was welcome, regardless of discipline, specialty, title, country of residence, or degree completion. During the 25 days that followed, I used Google Docs, Google Sheets, and Google Forms to manage all aspects of the collaboration. Results: The collaboration resulted in the completion of 2 manuscripts in a 25-day period. The International Council of Medical Journal Editors authorship criteria were met by 40 collaborators for the first article (“Documenting Social Media Engagement as Scholarship: A New Model for Assessing Academic Accomplishment for the Health Professions”) and 35 collaborators for the second article (“The Benefits of Using Social Media as a Health Professional in Academia”). The authorship teams for both articles were notably diverse, with 17%-18% (7/40 and 6/35, respectively) of authors identifying as a person of color and/or underrepresented minority, 37%-38% (15/40 and 13/35, respectively) identifying as LGBTQ+ (lesbian, gay, bisexual, transgender, gender non-conforming, queer and/or questioning), 73%-74% (29/40 and 26/35, respectively) using she/her pronouns, and 20%-23% (9/40 and 7/35, respectively) identifying as a person with a disability. Conclusions: Scholars in the health professions can use this paper in conjunction with the tools provided to replicate this process in carrying out their own large-scale manuscript collaborations. %M 33999002 %R 10.2196/25077 %U https://www.jmir.org/2021/5/e25077 %U https://doi.org/10.2196/25077 %U http://www.ncbi.nlm.nih.gov/pubmed/33999002 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 5 %P e25252 %T The Patterns and Impact of Social Media Exposure of Journal Publications in Gastroenterology: Retrospective Cohort Study %A Chiang,Austin Lee %A Rabinowitz,Loren Galler %A Alakbarli,Javid %A Chan,Walter W %+ Brigham and Women's Hospital, 75 Francis St, Boston, MA, United States, 1 617 732 4840, austinleechiang@gmail.com %K social media %K gastroenterology journals %K gastroenterology research %K journal citations %D 2021 %7 14.5.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Medical journals increasingly promote published content through social media platforms such as Twitter. However, gastroenterology journals still rank below average in social media engagement. Objective: We aimed to determine the engagement patterns of publications in gastroenterology journals on Twitter and evaluate the impact of tweets on citations. Methods: This was a retrospective cohort study comparing the 3-year citations of all full-length articles published in five major gastroenterology journals from January 1, 2012, to December 31, 2012, tweeted by official journal accounts with those that were not. Multivariate analysis using linear regression was performed to control for journal impact factor, time since publication, article type, frequency of reposting by other users (“retweets”), and media addition to tweets. Secondary analyses were performed to assess the associations between article type or subtopic and the likelihood of social media promotion/engagement. Results: A total of 1666 articles were reviewed, with 477 tweeted by the official journal account. Tweeting an article independently predicted increased citations after controlling for potential confounders (β coefficient=13.09; P=.007). There was significant association between article type and number of retweets on analysis of variance (ANOVA) (P<.001), with guidelines/technical reviews (mean difference 1.04, 95% CI 0.22-1.87; P<.001) and meta-analyses/systemic reviews (mean difference 1.03, 95% CI 0.35-1.70; P<.001) being retweeted more than basic science articles. The manuscript subtopics most frequently promoted included motility/functional bowel disease (odds ratio [OR] 3.84, 95% CI 1.93-7.64; P<.001) and education (OR 4.69, 95% CI 1.62-13.58; P=.004), while basic science papers were less likely tweeted (OR 0.154, 95% CI 0.07-0.34; P<.001). Conclusions: Tweeting of gastroenterology journal articles independently predicted higher 3-year citations. Wider adoption of social media to increase reach and measure uptake of published research should be considered. %M 33707166 %R 10.2196/25252 %U https://www.jmir.org/2021/5/e25252 %U https://doi.org/10.2196/25252 %U http://www.ncbi.nlm.nih.gov/pubmed/33707166 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 5 %P e25714 %T Using a Secure, Continually Updating, Web Source Processing Pipeline to Support the Real-Time Data Synthesis and Analysis of Scientific Literature: Development and Validation Study %A Vaghela,Uddhav %A Rabinowicz,Simon %A Bratsos,Paris %A Martin,Guy %A Fritzilas,Epameinondas %A Markar,Sheraz %A Purkayastha,Sanjay %A Stringer,Karl %A Singh,Harshdeep %A Llewellyn,Charlie %A Dutta,Debabrata %A Clarke,Jonathan M %A Howard,Matthew %A , %A Serban,Ovidiu %A Kinross,James %+ Data Science Institute, Imperial College London, William Penney Laboratory, South Kensington Campus, London, United Kingdom, o.serban@imperial.ac.uk %K structured data synthesis %K data science %K critical analysis %K web crawl data %K pipeline %K database %K literature %K research %K COVID-19 %K infodemic %K decision making %K data %K data synthesis %K misinformation %K infrastructure %K methodology %D 2021 %7 6.5.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: The scale and quality of the global scientific response to the COVID-19 pandemic have unquestionably saved lives. However, the COVID-19 pandemic has also triggered an unprecedented “infodemic”; the velocity and volume of data production have overwhelmed many key stakeholders such as clinicians and policy makers, as they have been unable to process structured and unstructured data for evidence-based decision making. Solutions that aim to alleviate this data synthesis–related challenge are unable to capture heterogeneous web data in real time for the production of concomitant answers and are not based on the high-quality information in responses to a free-text query. Objective: The main objective of this project is to build a generic, real-time, continuously updating curation platform that can support the data synthesis and analysis of a scientific literature framework. Our secondary objective is to validate this platform and the curation methodology for COVID-19–related medical literature by expanding the COVID-19 Open Research Dataset via the addition of new, unstructured data. Methods: To create an infrastructure that addresses our objectives, the PanSurg Collaborative at Imperial College London has developed a unique data pipeline based on a web crawler extraction methodology. This data pipeline uses a novel curation methodology that adopts a human-in-the-loop approach for the characterization of quality, relevance, and key evidence across a range of scientific literature sources. Results: REDASA (Realtime Data Synthesis and Analysis) is now one of the world’s largest and most up-to-date sources of COVID-19–related evidence; it consists of 104,000 documents. By capturing curators’ critical appraisal methodologies through the discrete labeling and rating of information, REDASA rapidly developed a foundational, pooled, data science data set of over 1400 articles in under 2 weeks. These articles provide COVID-19–related information and represent around 10% of all papers about COVID-19. Conclusions: This data set can act as ground truth for the future implementation of a live, automated systematic review. The three benefits of REDASA’s design are as follows: (1) it adopts a user-friendly, human-in-the-loop methodology by embedding an efficient, user-friendly curation platform into a natural language processing search engine; (2) it provides a curated data set in the JavaScript Object Notation format for experienced academic reviewers’ critical appraisal choices and decision-making methodologies; and (3) due to the wide scope and depth of its web crawling method, REDASA has already captured one of the world’s largest COVID-19–related data corpora for searches and curation. %M 33835932 %R 10.2196/25714 %U https://www.jmir.org/2021/5/e25714 %U https://doi.org/10.2196/25714 %U http://www.ncbi.nlm.nih.gov/pubmed/33835932 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 3 %P e22860 %T Revealing Opinions for COVID-19 Questions Using a Context Retriever, Opinion Aggregator, and Question-Answering Model: Model Development Study %A Lu,Zhao-Hua %A Wang,Jade Xiaoqing %A Li,Xintong %+ Department of Biostatistics, St. Jude Children’s Research Hospital, MS 768, Room R6006, 262 Danny Thomas Place, Memphis, TN, 38105-3678, United States, 1 901 595 2714, zhaohua.lu@stjude.org %K natural language processing %K question-answering systems %K language summarization %K machine learning %K life and medical sciences %K COVID-19 %K public health %K coronavirus literature %D 2021 %7 19.3.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: COVID-19 has challenged global public health because it is highly contagious and can be lethal. Numerous ongoing and recently published studies about the disease have emerged. However, the research regarding COVID-19 is largely ongoing and inconclusive. Objective: A potential way to accelerate COVID-19 research is to use existing information gleaned from research into other viruses that belong to the coronavirus family. Our objective is to develop a natural language processing method for answering factoid questions related to COVID-19 using published articles as knowledge sources. Methods: Given a question, first, a BM25-based context retriever model is implemented to select the most relevant passages from previously published articles. Second, for each selected context passage, an answer is obtained using a pretrained bidirectional encoder representations from transformers (BERT) question-answering model. Third, an opinion aggregator, which is a combination of a biterm topic model and k-means clustering, is applied to the task of aggregating all answers into several opinions. Results: We applied the proposed pipeline to extract answers, opinions, and the most frequent words related to six questions from the COVID-19 Open Research Dataset Challenge. By showing the longitudinal distributions of the opinions, we uncovered the trends of opinions and popular words in the articles published in the five time periods assessed: before 1990, 1990-1999, 2000-2009, 2010-2018, and since 2019. The changes in opinions and popular words agree with several distinct characteristics and challenges of COVID-19, including a higher risk for senior people and people with pre-existing medical conditions; high contagion and rapid transmission; and a more urgent need for screening and testing. The opinions and popular words also provide additional insights for the COVID-19–related questions. Conclusions: Compared with other methods of literature retrieval and answer generation, opinion aggregation using our method leads to more interpretable, robust, and comprehensive question-specific literature reviews. The results demonstrate the usefulness of the proposed method in answering COVID-19–related questions with main opinions and capturing the trends of research about COVID-19 and other relevant strains of coronavirus in recent years. %M 33739287 %R 10.2196/22860 %U https://www.jmir.org/2021/3/e22860 %U https://doi.org/10.2196/22860 %U http://www.ncbi.nlm.nih.gov/pubmed/33739287 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 23 %N 3 %P e23703 %T A Comprehensive Overview of the COVID-19 Literature: Machine Learning–Based Bibliometric Analysis %A Abd-Alrazaq,Alaa %A Schneider,Jens %A Mifsud,Borbala %A Alam,Tanvir %A Househ,Mowafa %A Hamdi,Mounir %A Shah,Zubair %+ Division of Information and Computing Technology, College of Science and Engineering, Hamad Bin Khalifa University, Qatar Foundation, P.O. Box 5825, Doha Al Luqta St, Ar-Rayyan, Doha, 00000, Qatar, 974 55708549, zshah@hbku.edu.qa %K novel coronavirus disease %K COVID-19 %K SARS-CoV-2 %K 2019-nCoV %K bibliometric analysis %K literature %K machine learning %K research %K review %D 2021 %7 8.3.2021 %9 Original Paper %J J Med Internet Res %G English %X Background: Shortly after the emergence of COVID-19, researchers rapidly mobilized to study numerous aspects of the disease such as its evolution, clinical manifestations, effects, treatments, and vaccinations. This led to a rapid increase in the number of COVID-19–related publications. Identifying trends and areas of interest using traditional review methods (eg, scoping and systematic reviews) for such a large domain area is challenging. Objective: We aimed to conduct an extensive bibliometric analysis to provide a comprehensive overview of the COVID-19 literature. Methods: We used the COVID-19 Open Research Dataset (CORD-19) that consists of a large number of research articles related to all coronaviruses. We used a machine learning–based method to analyze the most relevant COVID-19–related articles and extracted the most prominent topics. Specifically, we used a clustering algorithm to group published articles based on the similarity of their abstracts to identify research hotspots and current research directions. We have made our software accessible to the community via GitHub. Results: Of the 196,630 publications retrieved from the database, we included 28,904 in our analysis. The mean number of weekly publications was 990 (SD 789.3). The country that published the highest number of COVID-19–related articles was China (2950/17,270, 17.08%). The highest number of articles were published in bioRxiv. Lei Liu affiliated with the Southern University of Science and Technology in China published the highest number of articles (n=46). Based on titles and abstracts alone, we were able to identify 1515 surveys, 733 systematic reviews, 512 cohort studies, 480 meta-analyses, and 362 randomized control trials. We identified 19 different topics covered among the publications reviewed. The most dominant topic was public health response, followed by clinical care practices during the COVID-19 pandemic, clinical characteristics and risk factors, and epidemic models for its spread. Conclusions: We provide an overview of the COVID-19 literature and have identified current hotspots and research directions. Our findings can be useful for the research community to help prioritize research needs and recognize leading COVID-19 researchers, institutes, countries, and publishers. Our study shows that an AI-based bibliometric analysis has the potential to rapidly explore a large corpus of academic publications during a public health crisis. We believe that this work can be used to analyze other eHealth-related literature to help clinicians, administrators, and policy makers to obtain a holistic view of the literature and be able to categorize different topics of the existing research for further analyses. It can be further scaled (for instance, in time) to clinical summary documentation. Publishers should avoid noise in the data by developing a way to trace the evolution of individual publications and unique authors. %M 33600346 %R 10.2196/23703 %U https://www.jmir.org/2021/3/e23703 %U https://doi.org/10.2196/23703 %U http://www.ncbi.nlm.nih.gov/pubmed/33600346 %0 Journal Article %@ 2369-2960 %I JMIR Publications %V 7 %N 2 %P e25935 %T Collaborating in the Time of COVID-19: The Scope and Scale of Innovative Responses to a Global Pandemic %A Bernardo,Theresa %A Sobkowich,Kurtis Edward %A Forrest,Russell Othmer %A Stewart,Luke Silva %A D'Agostino,Marcelo %A Perez Gutierrez,Enrique %A Gillis,Daniel %+ Department of Population Medicine, University of Guelph, 50 Stone Rd E, Guelph, ON, N1G 2W1, Canada, 1 519 824 4120 ext 54184, theresabernardo@gmail.com %K crowdsourcing %K artificial intelligence %K collaboration %K personal protective equipment %K big data %K AI %K COVID-19 %K innovation %K information sharing %K communication %K teamwork %K knowledge %K dissemination %D 2021 %7 9.2.2021 %9 Viewpoint %J JMIR Public Health Surveill %G English %X The emergence of COVID-19 spurred the formation of myriad teams to tackle every conceivable aspect of the virus and thwart its spread. Enabled by global digital connectedness, collaboration has become a constant theme throughout the pandemic, resulting in the expedition of the scientific process (including vaccine development), rapid consolidation of global outbreak data and statistics, and experimentation with novel partnerships. To document the evolution of these collaborative efforts, the authors collected illustrative examples as the pandemic unfolded, supplemented with publications from the JMIR COVID-19 Special Issue. Over 60 projects rooted in collaboration are categorized into five main themes: knowledge dissemination, data propagation, crowdsourcing, artificial intelligence, and hardware design and development. They highlight the numerous ways that citizens, industry professionals, researchers, and academics have come together worldwide to consolidate information and produce products to combat the COVID-19 pandemic. Initially, researchers and citizen scientists scrambled to access quality data within an overwhelming quantity of information. As global curated data sets emerged, derivative works such as visualizations or models were developed that depended on consistent data and would fail when there were unanticipated changes. Crowdsourcing was used to collect and analyze data, aid in contact tracing, and produce personal protective equipment by sharing open designs for 3D printing. An international consortium of entrepreneurs and researchers created a ventilator based on an open-source design. A coalition of nongovernmental organizations and governmental organizations, led by the White House Office of Science and Technology Policy, created a shared open resource of over 200,000 research publications about COVID-19 and subsequently offered cash prizes for the best solutions to 17 key questions involving artificial intelligence. A thread of collaboration weaved throughout the pandemic response, which will shape future efforts. Novel partnerships will cross boundaries to create better processes, products, and solutions to consequential societal challenges. %M 33503001 %R 10.2196/25935 %U http://publichealth.jmir.org/2021/2/e25935/ %U https://doi.org/10.2196/25935 %U http://www.ncbi.nlm.nih.gov/pubmed/33503001 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 12 %P e22327 %T A Picture Is Worth a Thousand Views: A Triple Crossover Trial of Visual Abstracts to Examine Their Impact on Research Dissemination %A Oska,Sandra %A Lerma,Edgar %A Topf,Joel %+ William Beaumont School of Medicine, Oakland University, 586 Pioneer Dr, Rochester, MI, 48309, United States, 1 248 470 8163, joel.topf@gmail.com %K social media %K science communication %K visual abstract %K Twitter %K dissemination %D 2020 %7 4.12.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: A visual abstract is a graphic summary of a research article’s question, methods, and major findings. Although they have a number of uses, visual abstracts are chiefly used to promote research articles on social media. Objective: This study aimed to determine if the use of visual abstracts increases the visibility of nephrology research shared on Twitter. Methods: A prospective case-control crossover study was conducted using 40 research articles published in the American Journal of Nephrology (AJN). Each article was shared by the AJN Twitter account in 3 formats: (1) the article citation, (2) the citation with a key figure from the article, and (3) the citation with a visual abstract. Tweets were spaced 2 weeks apart to allow washout of the previous tweet, and the order of the tweets was randomized. Dissemination was measured via retweets, views, number of link clicks, and Altmetric scores. Results: Tweets that contained a visual abstract had more than twice as many views as citation-only tweets (1351, SD 1053 vs 639, SD 343) and nearly twice as many views as key figure tweets (1351, SD 1053 vs 732, SD 464). Visual abstract tweets had 5 times the engagements of citation-only tweets and more than 3.5 times the engagements of key figure tweets. Visual abstract tweets were also associated with greater increases in Altmetric scores as compared to citation-only tweets (2.20 vs 1.05). Conclusions: The use of visual abstracts increased visibility of research articles on Twitter, resulting in a greater number of views, engagements, and retweets. Visual abstracts were also associated with increased Altmetric scores as compared to citation-only tweets. These findings support the broader use of visual abstracts in the scientific community. Journals should consider visual abstracts as valuable tools for research dissemination. %M 33275112 %R 10.2196/22327 %U https://www.jmir.org/2020/12/e22327 %U https://doi.org/10.2196/22327 %U http://www.ncbi.nlm.nih.gov/pubmed/33275112 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 11 %P e21931 %T Analysis of the Trends in Publications on Clinical Cancer Research in Mainland China from the Surveillance, Epidemiology, and End Results (SEER) Database: Bibliometric Study %A Lin,Min-Qiang %A Lian,Chen-Lu %A Zhou,Ping %A Lei,Jian %A Wang,Jun %A Hua,Li %A Zhou,Juan %A Wu,San-Gang %+ Department of Radiation Oncology, The First Affiliated Hospital of Xiamen University, 55 Zhenhai Road, Xiamen, China, 86 5922139531, wusg@xmu.edu.cn %K cancer %K China %K data collection %K bibliometrics %K PubMed %K SEER program %D 2020 %7 17.11.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: The application of China’s big data sector in cancer research is just the beginning. In recent decades, more and more Chinese scholars have used the Surveillance, Epidemiology, and End Results (SEER) database for clinical cancer research. A comprehensive bibliometric study is required to analyze the tendency of Chinese scholars to utilize the SEER database for clinical cancer research and provide a reference for the future of big data analytics. Objective: Our study aimed to assess the trend of publications on clinical cancer research in mainland China from the SEER database. Methods: We performed a PubMed search to identify papers published with data from the SEER database in mainland China until August 31, 2020. Results: A total of 1566 papers utilizing the SEER database that were authored by investigators in mainland China were identified. Over the past years, significant growth in studies based on the SEER database was observed (P<.001). The top 5 research topics were breast cancer (213/1566, 13.6%), followed by colorectal cancer (185/1566, 11.8%), lung cancer (179/1566, 11.4%), gastrointestinal cancer (excluding colorectal cancer; 149/1566, 9.5%), and genital system cancer (93/1566, 5.9%). Approximately 75.2% (1178/1566) of papers were published from the eastern coastal region of China, and Fudan University Shanghai Cancer Center (Shanghai, China) was the most active organization. Overall, 267 journals were analyzed in this study, of which Oncotarget was the most contributing journal (136/267, 50.9%). Of the 1566 papers studied, 585 (37.4%) were published in the second quartile, 489 (31.2%) in the third quartile, 312 (19.9%) in the first quartile, and 80 (5.1%) in the fourth quartile, with 100 (6.4%) having an unknown Journal Citation Reports ranking. Conclusions: Clinical cancer research based on the SEER database in mainland China underwent constant and rapid growth during recent years. High-quality and comprehensive cancer databases based on Chinese demographic data are urgently needed. %M 33200992 %R 10.2196/21931 %U http://medinform.jmir.org/2020/11/e21931/ %U https://doi.org/10.2196/21931 %U http://www.ncbi.nlm.nih.gov/pubmed/33200992 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 11 %P e21559 %T Analysis of Scientific Publications During the Early Phase of the COVID-19 Pandemic: Topic Modeling Study %A Älgå,Andreas %A Eriksson,Oskar %A Nordberg,Martin %+ Department of Clinical Science and Education, Södersjukhuset, Karolinska Institutet, Sjukhusbacken 10, Stockholm, 118 83, Sweden, 46 8 616 10 00, andreas.alga@ki.se %K COVID-19 %K SARS-CoV-2 %K coronavirus %K pandemic %K topic modeling %K research %K literature %D 2020 %7 10.11.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: The COVID-19 pandemic has spread at an alarming speed, and effective treatment for the disease is still lacking. The body of evidence on COVID-19 has been increasing at an impressive pace, creating the need for a method to rapidly assess the current knowledge and identify key information. Gold standard methods such as systematic reviews and meta-analyses are regarded unsuitable because they have a narrow scope and are very time consuming. Objective: This study aimed to explore the published scientific literature on COVID-19 and map the research evolution during the early phase of the COVID-19 pandemic. Methods: We performed a PubMed search to analyze the titles, keywords, and abstracts of published papers on COVID-19. We used latent Dirichlet allocation modeling to extract topics and conducted a trend analysis to understand the temporal changes in research for each topic, journal impact factor (JIF), and geographic origin. Results: Based on our search, we identified 16,670 relevant articles dated between February 14, 2020, and June 1, 2020. Of these, 6 articles were reports from peer-reviewed randomized trials on patients with COVID-19. We identified 14 main research topics, of which the most common topics were health care responses (2812/16,670, 16.86%) and clinical manifestations (1828/16,670, 10.91%). We found an increasing trend for research on clinical manifestations and protective measures and a decreasing trend for research on disease transmission, epidemiology, health care response, and radiology. Publications on protective measures, immunology, and clinical manifestations were associated with the highest JIF. The overall median JIF was 3.7 (IQR 2.6-5.9), and we found that the JIF for these publications declined over time. The top countries producing research were the United States, China, Italy, and the United Kingdom. Conclusions: In less than 6 months since the novel coronavirus was first detected, a remarkably high number of research articles on COVID-19 have been published. Here, we discuss and present the temporal changes in the available COVID-19 research during the early phase of the pandemic. Our findings may aid researchers and policy makers to form a structured view of the current COVID-19 evidence base and provide further research directions. %M 33031049 %R 10.2196/21559 %U http://www.jmir.org/2020/11/e21559/ %U https://doi.org/10.2196/21559 %U http://www.ncbi.nlm.nih.gov/pubmed/33031049 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 11 %P e21648 %T Toward Preparing a Knowledge Base to Explore Potential Drugs and Biomedical Entities Related to COVID-19: Automated Computational Approach %A Khan,Junaed Younus %A Khondaker,Md Tawkat Islam %A Hoque,Iram Tazim %A Al-Absi,Hamada R H %A Rahman,Mohammad Saifur %A Guler,Reto %A Alam,Tanvir %A Rahman,M Sohel %+ College of Science and Engineering, Hamad Bin Khalifa University, PO Box 34110, Education City, Doha, Qatar, 974 44542277, talam@hbku.edu.qa %K COVID-19 %K 2019-nCoV %K coronavirus %K SARS-CoV-2 %K SARS %K remdesivir %K statin %K statins %K dexamethasone %K ivermectin %K hydroxychloroquine %D 2020 %7 10.11.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: Novel coronavirus disease 2019 (COVID-19) is taking a huge toll on public health. Along with the non-therapeutic preventive measurements, scientific efforts are currently focused, mainly, on the development of vaccines and pharmacological treatment with existing drugs. Summarizing evidences from scientific literatures on the discovery of treatment plan of COVID-19 under a platform would help the scientific community to explore the opportunities in a systematic fashion. Objective: The aim of this study is to explore the potential drugs and biomedical entities related to coronavirus related diseases, including COVID-19, that are mentioned on scientific literature through an automated computational approach. Methods: We mined the information from publicly available scientific literature and related public resources. Six topic-specific dictionaries, including human genes, human miRNAs, diseases, Protein Databank, drugs, and drug side effects, were integrated to mine all scientific evidence related to COVID-19. We employed an automated literature mining and labeling system through a novel approach to measure the effectiveness of drugs against diseases based on natural language processing, sentiment analysis, and deep learning. We also applied the concept of cosine similarity to confidently infer the associations between diseases and genes. Results: Based on the literature mining, we identified 1805 diseases, 2454 drugs, 1910 genes that are related to coronavirus related diseases including COVID-19. Integrating the extracted information, we developed the first knowledgebase platform dedicated to COVID-19, which highlights potential list of drugs and related biomedical entities. For COVID-19, we highlighted multiple case studies on existing drugs along with a confidence score for their applicability in the treatment plan. Based on our computational method, we found Remdesivir, Statins, Dexamethasone, and Ivermectin could be considered as potential effective drugs to improve clinical status and lower mortality in patients hospitalized with COVID-19. We also found that Hydroxychloroquine could not be considered as an effective drug for COVID-19. The resulting knowledgebase is made available as an open source tool, named COVID-19Base. Conclusions: Proper investigation of the mined biomedical entities along with the identified interactions among those would help the research community to discover possible ways for the therapeutic treatment of COVID-19. %M 33055059 %R 10.2196/21648 %U http://medinform.jmir.org/2020/11/e21648/ %U https://doi.org/10.2196/21648 %U http://www.ncbi.nlm.nih.gov/pubmed/33055059 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 8 %P e20007 %T Artificial Intelligence for Rapid Meta-Analysis: Case Study on Ocular Toxicity of Hydroxychloroquine %A Michelson,Matthew %A Chow,Tiffany %A Martin,Neil A %A Ross,Mike %A Tee Qiao Ying,Amelia %A Minton,Steven %+ Evid Science, 2361 Rosencrans Ave Ste 348, El Segundo, CA, 90245-4929, United States, 1 626 765 1903, mmichelson@evidscience.com %K meta-analysis %K rapid meta-analysis %K artificial intelligence %K drug %K analysis %K hydroxychloroquine %K toxic %K COVID-19 %K treatment %K side effect %K ocular %K eye %D 2020 %7 17.8.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Rapid access to evidence is crucial in times of an evolving clinical crisis. To that end, we propose a novel approach to answer clinical queries, termed rapid meta-analysis (RMA). Unlike traditional meta-analysis, RMA balances a quick time to production with reasonable data quality assurances, leveraging artificial intelligence (AI) to strike this balance. Objective: We aimed to evaluate whether RMA can generate meaningful clinical insights, but crucially, in a much faster processing time than traditional meta-analysis, using a relevant, real-world example. Methods: The development of our RMA approach was motivated by a currently relevant clinical question: is ocular toxicity and vision compromise a side effect of hydroxychloroquine therapy? At the time of designing this study, hydroxychloroquine was a leading candidate in the treatment of coronavirus disease (COVID-19). We then leveraged AI to pull and screen articles, automatically extract their results, review the studies, and analyze the data with standard statistical methods. Results: By combining AI with human analysis in our RMA, we generated a meaningful, clinical result in less than 30 minutes. The RMA identified 11 studies considering ocular toxicity as a side effect of hydroxychloroquine and estimated the incidence to be 3.4% (95% CI 1.11%-9.96%). The heterogeneity across individual study findings was high, which should be taken into account in interpretation of the result. Conclusions: We demonstrate that a novel approach to meta-analysis using AI can generate meaningful clinical insights in a much shorter time period than traditional meta-analysis. %M 32804086 %R 10.2196/20007 %U http://www.jmir.org/2020/8/e20007/ %U https://doi.org/10.2196/20007 %U http://www.ncbi.nlm.nih.gov/pubmed/32804086 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 8 %P e18747 %T Social, Behavioral, and Cultural factors of HIV in Malawi: Semi-Automated Systematic Review %A Thiabaud,Amaury %A Triulzi,Isotta %A Orel,Erol %A Tal,Kali %A Keiser,Olivia %+ Institut de Santé Globale, Université de Genève, Chemin des mines 9, Genève, 1202, Switzerland, 41 22 379 81 30, amaury.thiabaud@unige.ch %K HIV/AIDS %K topic modelling %K text mining %K Malawi %K risk factors %K machine learning %D 2020 %7 14.8.2020 %9 Review %J J Med Internet Res %G English %X Background: Demographic and sociobehavioral factors are strong drivers of HIV infection rates in sub-Saharan Africa. These factors are often studied in qualitative research but ignored in quantitative analyses. However, they provide in-depth insight into the local behavior and may help to improve HIV prevention. Objective: To obtain a comprehensive overview of the sociobehavioral factors influencing HIV prevalence and incidence in Malawi, we systematically reviewed the literature using a newly programmed tool for automatizing part of the systematic review process. Methods: Due to the choice of broad search terms (“HIV AND Malawi”), our preliminary search revealed many thousands of articles. We, therefore, developed a Python tool to automatically extract, process, and categorize open-access articles published from January 1, 1987 to October 1, 2019 in the PubMed, PubMed Central, JSTOR, Paperity, and arXiV databases. We then used a topic modelling algorithm to classify and identify publications of interest. Results: Our tool extracted 22,709 unique articles; 16,942 could be further processed. After topic modelling, 519 of these were clustered into relevant topics, of which 20 were kept after manual screening. We retrieved 7 more publications after examining the references so that 27 publications were finally included in the review. Reducing the 16,942 articles to 519 potentially relevant articles using the software took 5 days. Several factors contributing to the risk of HIV infection were identified, including religion, gender and relationship dynamics, beliefs, and sociobehavioral attitudes. Conclusions: Our software does not replace traditional systematic reviews, but it returns useful results to broad queries of open-access literature in under a week, without a priori knowledge. This produces a “seed dataset” of relevance that could be further developed. It identified known factors and factors that may be specific to Malawi. In the future, we aim to expand the tool by adding more social science databases and applying it to other sub-Saharan African countries. %M 32795992 %R 10.2196/18747 %U http://www.jmir.org/2020/8/e18747/ %U https://doi.org/10.2196/18747 %U http://www.ncbi.nlm.nih.gov/pubmed/32795992 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 8 %N 7 %P e18212 %T Theme Trends and Knowledge Structure on Mobile Health Apps: Bibliometric Analysis %A Peng,Cheng %A He,Miao %A Cutrona,Sarah L %A Kiefe,Catarina I %A Liu,Feifan %A Wang,Zhongqing %+ Department of Information Center, The First Hospital of China Medical University, 155 Nanjinbei Street, Shenyang, China, 86 15940082159, wangzhongqing@cmu.edu.cn %K mobile app %K mobile health %K mhealth %K digital health %K digital medicine %K bibliometrics %K co-word analysis %K mobile phone %K VOSviewer %D 2020 %7 27.7.2020 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: Due to the widespread and unprecedented popularity of mobile phones, the use of digital medicine and mobile health apps has seen significant growth. Mobile health apps have tremendous potential for monitoring and treating diseases, improving patient care, and promoting health. Objective: This paper aims to explore research trends, coauthorship networks, and the research hot spots of mobile health app research. Methods: Publications related to mobile health apps were retrieved and extracted from the Web of Science database with no language restrictions. Bibliographic Item Co-Occurrence Matrix Builder was employed to extract bibliographic information (publication year and journal source) and perform a descriptive analysis. We then used the VOSviewer (Leiden University) tool to construct and visualize the co-occurrence networks of researchers, research institutions, countries/regions, citations, and keywords. Results: We retrieved 2802 research papers on mobile health apps published from 2000 to 2019. The number of annual publications increased over the past 19 years. JMIR mHealth and uHealth (323/2802, 11.53%), Journal of Medical Internet Research (106/2802, 3.78%), and JMIR Research Protocols (82/2802, 2.93%) were the most common journals for these publications. The United States (1186/2802, 42.33%), England (235/2802, 8.39%), Australia (215/2802, 7.67%), and Canada (112/2802, 4.00%) were the most productive countries of origin. The University of California San Francisco, the University of Washington, and the University of Toronto were the most productive institutions. As for the authors’ contributions, Schnall R, Kuhn E, Lopez-Coronado M, and Kim J were the most active researchers. The co-occurrence cluster analysis of the top 100 keywords forms 5 clusters: (1) the technology and system development of mobile health apps; (2) mobile health apps for mental health; (3) mobile health apps in telemedicine, chronic disease, and medication adherence management; (4) mobile health apps in health behavior and health promotion; and (5) mobile health apps in disease prevention via the internet. Conclusions: We summarize the recent advances in mobile health app research and shed light on their research frontier, trends, and hot topics through bibliometric analysis and network visualization. These findings may provide valuable guidance on future research directions and perspectives in this rapidly developing field. %M 32716312 %R 10.2196/18212 %U https://mhealth.jmir.org/2020/7/e18212 %U https://doi.org/10.2196/18212 %U http://www.ncbi.nlm.nih.gov/pubmed/32716312 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 7 %P e15607 %T The Use of Social Media to Increase the Impact of Health Research: Systematic Review %A Bardus,Marco %A El Rassi,Rola %A Chahrour,Mohamad %A Akl,Elie W %A Raslan,Abdul Sattar %A Meho,Lokman I %A Akl,Elie A %+ Department of Internal Medicine, American University of Beirut Medical Center, PO Box 11-0236, Riad-El-Solh, Beirut, 1107 2020, Lebanon, 961 1 374 374, ea32@aub.edu.lb %K social media %K research %K bibliometrics %K Altmetrics %K journal impact factor %K translational medical research %D 2020 %7 6.7.2020 %9 Review %J J Med Internet Res %G English %X Background: Academics in all disciplines increasingly use social media to share their publications on the internet, reaching out to different audiences. In the last few years, specific indicators of social media impact have been developed (eg, Altmetrics), to complement traditional bibliometric indicators (eg, citation count and h-index). In health research, it is unclear whether social media impact also translates into research impact. Objective: The primary aim of this study was to systematically review the literature on the impact of using social media on the dissemination of health research. The secondary aim was to assess the correlation between Altmetrics and traditional citation-based metrics. Methods: We conducted a systematic review to identify studies that evaluated the use of social media to disseminate research published in health-related journals. We specifically looked at studies that described experimental or correlational studies linking the use of social media with outcomes related to bibliometrics. We searched the Medical Literature Analysis and Retrieval System Online (MEDLINE), Excerpta Medica dataBASE (EMBASE), and Cumulative Index to Nursing and Allied Health Literature (CINAHL) databases using a predefined search strategy (International Prospective Register of Systematic Reviews: CRD42017057709). We conducted independent and duplicate study selection and data extraction. Given the heterogeneity of the included studies, we summarized the findings through a narrative synthesis. Results: Of a total of 18,624 retrieved citations, we included 51 studies: 7 (14%) impact studies (answering the primary aim) and 44 (86%) correlational studies (answering the secondary aim). Impact studies reported mixed results with several limitations, including the use of interventions of inappropriately low intensity and short duration. The majority of correlational studies suggested a positive association between traditional bibliometrics and social media metrics (eg, number of mentions) in health research. Conclusions: We have identified suggestive yet inconclusive evidence on the impact of using social media to increase the number of citations in health research. Further studies with better design are needed to assess the causal link between social media impact and bibliometrics. %M 32628113 %R 10.2196/15607 %U https://www.jmir.org/2020/7/e15607 %U https://doi.org/10.2196/15607 %U http://www.ncbi.nlm.nih.gov/pubmed/32628113 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 6 %P e16739 %T Understanding Drug Repurposing From the Perspective of Biomedical Entities and Their Evolution: Bibliographic Research Using Aspirin %A Li,Xin %A Rousseau,Justin F %A Ding,Ying %A Song,Min %A Lu,Wei %+ Information Retrieval and Knowledge Mining Laboratory, School of Information Management, Wuhan University, 299 Bayi DR, Wuchang District, Wuhan, 430072, China, 86 02768752757, weilu@whu.edu.cn %K drug repurposing %K biomedical entities %K entitymetrics %K bibliometrics %K aspirin %K acetylsalicylic acid %D 2020 %7 16.6.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: Drug development is still a costly and time-consuming process with a low rate of success. Drug repurposing (DR) has attracted significant attention because of its significant advantages over traditional approaches in terms of development time, cost, and safety. Entitymetrics, defined as bibliometric indicators based on biomedical entities (eg, diseases, drugs, and genes) studied in the biomedical literature, make it possible for researchers to measure knowledge evolution and the transfer of drug research. Objective: The purpose of this study was to understand DR from the perspective of biomedical entities (diseases, drugs, and genes) and their evolution. Methods: In the work reported in this paper, we extended the bibliometric indicators of biomedical entities mentioned in PubMed to detect potential patterns of biomedical entities in various phases of drug research and investigate the factors driving DR. We used aspirin (acetylsalicylic acid) as the subject of the study since it can be repurposed for many applications. We propose 4 easy, transparent measures based on entitymetrics to investigate DR for aspirin: Popularity Index (P1), Promising Index (P2), Prestige Index (P3), and Collaboration Index (CI). Results: We found that the maxima of P1, P3, and CI are closely associated with the different repurposing phases of aspirin. These metrics enabled us to observe the way in which biomedical entities interacted with the drug during the various phases of DR and to analyze the potential driving factors for DR at the entity level. P1 and CI were indicative of the dynamic trends of a specific biomedical entity over a long time period, while P2 was more sensitive to immediate changes. P3 reflected the early signs of the practical value of biomedical entities and could be valuable for tracking the research frontiers of a drug. Conclusions: In-depth studies of side effects and mechanisms, fierce market competition, and advanced life science technologies are driving factors for DR. This study showcases the way in which researchers can examine the evolution of DR using entitymetrics, an approach that can be valuable for enhancing decision making in the field of drug discovery and development. %M 32543442 %R 10.2196/16739 %U https://medinform.jmir.org/2020/6/e16739 %U https://doi.org/10.2196/16739 %U http://www.ncbi.nlm.nih.gov/pubmed/32543442 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 22 %N 5 %P e17741 %T Online Impact and Presence of a Specialized Social Media Team for the Journal of Neurosurgery: Descriptive Analysis %A Linzey,Joseph R %A Robertson,Faith %A Haider,Ali S %A Graffeo,Christopher Salvatore %A Wang,Justin Z %A Shasby,Gillian %A Alotaibi,Naif M %A Cohen-Gadol,Aaron A %A Rutka,James T %+ Department of Neurosurgery, Massachusetts General Hospital and Harvard Medical School, 55 Fruit St, Boston, MA, 02114, United States, 1 8579489955, nalotaibi@mgh.harvard.edu %K social media %K Twitter %K Facebook %K research dissemination %D 2020 %7 19.5.2020 %9 Original Paper %J J Med Internet Res %G English %X Background: Social media use continues to gain momentum in academic neurosurgery. To increase journal impact and broaden engagement, many scholarly publications have turned to social media to disseminate research. The Journal of Neurosurgery Publishing Group (JNSPG) established a dedicated, specialized social media team (SMT) in November 2016 to provide targeted improvement in digital outreach. Objective: The goal of this study was to examine the impact of the JNSPG SMT as measured by increased engagement. Methods: We analyzed various metrics, including impressions, engagements, retweets, likes, profile clicks, and URL clicks, from consecutive social media posts from the JNSPG’s Twitter and Facebook platforms between February 1, 2015 and February 28, 2019. Standard descriptive statistics were utilized. Results: Between February 2015 and October 2016, when a specialized SMT was created, 170 tweets (8.1 tweets/month) were posted compared to 3220 tweets (115.0 tweets/month) between November 2016 and February 2019. All metrics significantly increased, including the impressions per tweet (mean 1646.3, SD 934.9 vs mean 4605.6, SD 65,546.5; P=.01), engagements per tweet (mean 35.2, SD 40.6 vs mean 198.2, SD 1037.2; P<.001), retweets (mean 2.5, SD 2.8 vs mean 10.5, SD 15.3; P<.001), likes (mean 2.5, SD 4.0 vs mean 18.0, SD 37.9; P<.001), profile clicks (mean 1.5, SD 2.0 vs mean 5.2, SD 43.3; P<.001), and URL clicks (mean 13.1, SD 14.9 vs mean 38.3, SD 67.9; P<.001). Tweets that were posted on the weekend compared to weekdays had significantly more retweets (mean 9.2, SD 9.8 vs mean 13.4, SD 25.6; P<.001), likes (mean 15.3, SD 17.9 vs mean 23.7, SD 70.4; P=.001), and URL clicks (mean 33.4, SD 40.5 vs mean 49.5, SD 117.3; P<.001). Between November 2015 and October 2016, 49 Facebook posts (2.3 posts/month) were sent compared to 2282 posts (81.5 posts/month) sent between November 2016 and February 2019. All Facebook metrics significantly increased, including impressions (mean 5475.9, SD 5483.0 vs mean 8506.1, SD 13,113.9; P<.001), engagements (mean 119.3, SD 194.8 vs mean 283.8, SD 733.8; P<.001), and reach (mean 2266.6, SD 2388.3 vs mean 5344.1, SD 8399.2; P<.001). Weekend Facebook posts had significantly more impressions per post (mean 7967.9, SD 9901.0 vs mean 9737.8, SD 19,013.4; P=.03) and a higher total reach (mean 4975.8, SD 6309.8 vs mean 6108.2, SD 12,219.7; P=.03) than weekday posts. Conclusions: Social media has been established as a crucial tool for the propagation of neurosurgical research and education. Implementation of the JNSPG specialized SMT had a demonstrable impact on increasing the online visibility of social media content. %M 32163371 %R 10.2196/17741 %U http://www.jmir.org/2020/5/e17741/ %U https://doi.org/10.2196/17741 %U http://www.ncbi.nlm.nih.gov/pubmed/32163371 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 4 %P e18323 %T A Knowledge Graph of Combined Drug Therapies Using Semantic Predications From Biomedical Literature: Algorithm Development %A Du,Jian %A Li,Xiaoying %+ Institute of Medical Information, Chinese Academy of Medical Sciences, No 69, Dongdan North Street, Dongcheng District, Beijing, 100005, China, 86 10 52328792, lixiaoying@imicams.ac.cn %K combined drug therapy %K knowledge graph %K knowledge discovery %K semantic predications %D 2020 %7 28.4.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: Combination therapy plays an important role in the effective treatment of malignant neoplasms and precision medicine. Numerous clinical studies have been carried out to investigate combination drug therapies. Automated knowledge discovery of these combinations and their graphic representation in knowledge graphs will enable pattern recognition and identification of drug combinations used to treat a specific type of cancer, improve drug efficacy and treatment of human disorders. Objective: This paper aims to develop an automated, visual approach to discover knowledge about combination therapies from biomedical literature, especially from those studies with high-level evidence such as clinical trial reports and clinical practice guidelines. Methods: Based on semantic predications, which consist of a triple structure of subject-predicate-object (SPO), we proposed an automated algorithm to discover knowledge of combination drug therapies using the following rules: 1) two or more semantic predications (S1-P-O and Si-P-O, i = 2, 3…) can be extracted from one conclusive claim (sentence) in the abstract of a given publication, and 2) these predications have an identical predicate (that closely relates to human disease treatment, eg, “treat”) and object (eg, disease name) but different subjects (eg, drug names). A customized knowledge graph organizes and visualizes these combinations, improving the traditional semantic triples. After automatic filtering of broad concepts such as “pharmacologic actions” and generic disease names, a set of combination drug therapies were identified and characterized through manual interpretation. Results: We retrieved 22,263 clinical trial reports and 31 clinical practice guidelines from PubMed abstracts by searching “antineoplastic agents” for drug restriction (published between Jan 2009 and Oct 2019). There were 15,603 conclusive claims locally parsed using the search terms “conclusion*” and “conclude*” ready for semantic predications extraction by SemRep, and 325 candidate groups of semantic predications about combined medications were automatically discovered within 316 conclusive claims. Based on manual analysis, we determined that 255/316 claims (78.46%) were accurately identified as describing combination therapies and adopted these to construct the customized knowledge graph. We also identified two categories (and 4 subcategories) to characterize the inaccurate results: limitations of SemRep and limitations of proposal. We further learned the predominant patterns of drug combinations based on mechanism of action for new combined medication studies and discovered 4 obvious markers (“combin*,” “coadministration,” “co-administered,” and “regimen”) to identify potential combination therapies to enable development of a machine learning algorithm. Conclusions: Semantic predications from conclusive claims in the biomedical literature can be used to support automated knowledge discovery and knowledge graph construction for combination therapies. A machine learning approach is warranted to take full advantage of the identified markers and other contextual features. %M 32343247 %R 10.2196/18323 %U http://medinform.jmir.org/2020/4/e18323/ %U https://doi.org/10.2196/18323 %U http://www.ncbi.nlm.nih.gov/pubmed/32343247 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 8 %N 2 %P e11287 %T Intellectual Structure and Evolutionary Trends of Precision Medicine Research: Coword Analysis %A Lyu,Xiaoguang %A Hu,Jiming %A Dong,Weiguo %A Xu,Xin %+ School of Information Management, Wuhan University, Wuchang, Wuhan, 430072, China, 86 18995621959, hujiming@whu.edu.cn %K precision medicine %K topics distribution %K correlation structure %K evolution patterns %K coword analysis %D 2020 %7 4.2.2020 %9 Original Paper %J JMIR Med Inform %G English %X Background: Precision medicine (PM) is playing a more and more important role in clinical practice. In recent years, the scale of PM research has been growing rapidly. Many reviews have been published to facilitate a better understanding of the status of PM research. However, there is still a lack of research on the intellectual structure in terms of topics. Objective: This study aimed to identify the intellectual structure and evolutionary trends of PM research through the application of various social network analysis and visualization methods. Methods: The bibliographies of papers published between 2009 and 2018 were extracted from the Web of Science database. Based on the statistics of keywords in the papers, a coword network was generated and used to calculate network indicators of both the entire network and local networks. Communities were then detected to identify subdirections of PM research. Topological maps of networks, including networks between communities and within each community, were drawn to reveal the correlation structure. An evolutionary graph and a strategic graph were finally produced to reveal research venation and trends in discipline communities. Results: The results showed that PM research involves extensive themes and, overall, is not balanced. A minority of themes with a high frequency and network indicators, such as Biomarkers, Genomics, Cancer, Therapy, Genetics, Drug, Target Therapy, Pharmacogenomics, Pharmacogenetics, and Molecular, can be considered the core areas of PM research. However, there were five balanced theme directions with distinguished status and tendencies: Cancer, Biomarkers, Genomics, Drug, and Therapy. These were shown to be the main branches that were both focused and well developed. Therapy, though, was shown to be isolated and undeveloped. Conclusions: The hotspots, structures, evolutions, and development trends of PM research in the past ten years were revealed using social network analysis and visualization. In general, PM research is unbalanced, but its subdirections are balanced. The clear evolutionary and developmental trend indicates that PM research has matured in recent years. The implications of this study involving PM research will provide reasonable and effective support for researchers, funders, policymakers, and clinicians. %M 32014844 %R 10.2196/11287 %U https://medinform.jmir.org/2020/2/e11287 %U https://doi.org/10.2196/11287 %U http://www.ncbi.nlm.nih.gov/pubmed/32014844 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 12 %P e17578 %T Celebrating 20 Years of Open Access and Innovation at JMIR Publications %A Eysenbach,Gunther %+ JMIR Publications, 130 Queens Quay E, Suite 1100, Toronto, ON, Canada, 1 416 583 2040, editor@jmir.org %K JMIR %K internet %K medical informatics %K ehealth %K digital health %K participatory medicine %K open access %K electronic publishing %K scholarly publishing %K science communication %K journalogy %K history of science %K overlay journal %K preprints %K open science %D 2019 %7 23.12.2019 %9 Editorial %J J Med Internet Res %G English %X In this 20th anniversary theme issue, we are celebrating how JMIR Publications, an innovative publisher deeply rooted in academia and created by scientists for scientists, pioneered the open access model, is advancing digital health research, is disrupting the scholarly publishing world, and is helping to empower patients. All this has been made possible by the disintermediating power of the internet. And we are not done innovating: Our new series of “superjournals,” called JMIRx, will provide a glimpse into what we see as the future and end goal in scholarly publishing: open science. In this model, the vast majority of papers will be published on preprint servers first, with “overlay” journals then competing to peer review and publish peer-reviewed “versions of record” of the best papers. %M 31868653 %R 10.2196/17578 %U http://www.jmir.org/2019/12/e17578/ %U https://doi.org/10.2196/17578 %U http://www.ncbi.nlm.nih.gov/pubmed/31868653 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 12 %P e16532 %T Preserving the Open Access Benefits Pioneered by the Journal of Medical Internet Research and Discouraging Fraudulent Journals %A Wyatt,Jeremy C %+ Department of Public Health and Primary Care, University of Southampton, Aldermoor Health Centre, Aldermoor Close, Southampton, SO16 7NS, United Kingdom, 44 2380597551, j.c.wyatt@soton.ac.uk %K open access %K predatory journals %K knowledge management %K scientific journals %K mobilizing computable knowledge %K fraudulent journals %D 2019 %7 23.12.2019 %9 Viewpoint %J J Med Internet Res %G English %X The Journal of Medical Internet Research (JMIR) was an early pioneer of open access online publishing, and two decades later, some readers and authors may have forgotten the challenges of previous scientific publishing models. This commentary summarizes the many advantages of open access publishing for each of the main stakeholders in scientific publishing and reminds us that, like every innovation, there are disadvantages that we need to guard against, such as the problem of fraudulent journals. This paper then reviews the potential impact of some current initiatives, such as Plan S and JMIRx, concluding with some suggestions to help new open-access publishers ensure that the advantages of open access publishing outweigh the challenges. %R 10.2196/16532 %U http://www.jmir.org/2019/12/e16532/ %U https://doi.org/10.2196/16532 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 12 %P e10477 %T Trends and Visibility of “Digital Health” as a Keyword in Articles by JMIR Publications in the New Millennium: Bibliographic-Bibliometric Analysis %A Ahmadvand,Alireza %A Kavanagh,David %A Clark,Michele %A Drennan,Judy %A Nissen,Lisa %+ School of Clinical Sciences, Faculty of Health, Queensland University of Technology, 2 George Street, Brisbane, 4000, Australia, 61 731384404, l.nissen@qut.edu.au %K bibliometrics %K review literature %K JMIR Publications %K telemedicine %D 2019 %7 19.12.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Digital health has become an advancing phenomenon in the health care systems of modern societies. Over the past two decades, various digital health options, technologies, and innovations have been introduced; many of them are still being investigated and evaluated by researchers all around the globe. However, the actual trends and visibility of peer-reviewed publications using “digital health” as a keyword to reflect the topic, published by major relevant journals, still remain to be quantified. Objective: This study aimed to conduct a bibliographic-bibliometric analysis on articles published in JMIR Publications journals that used “digital health” as a keyword. We evaluated the trends, topics, and citations of these research publications to identify the important share and contribution of JMIR Publications journals in publishing articles on digital health. Methods: All JMIR Publications journals were searched to find articles in English, published between January 2000 and August 2019, in which the authors focused on, utilized, or discussed digital health in their study and used “digital health” as a keyword. In addition, a bibliographic-bibliometric analysis was conducted using the freely available Profiles Research Networking Software by the Harvard Clinical and Translational Science Center. Results: Out of 1797 articles having “digital health” as a keyword, published mostly between 2016 and 2019, 277 articles (32.3%) were published by JMIR Publications journals, mainly in the Journal of Medical Internet Research. The most frequently used keyword for the topic was “mHealth.” The average number of times an article had been cited, including self-citations, was above 2.8. Conclusions: The reflection of “digital health” as a keyword in JMIR Publications journals has increased noticeably over the past few years. To maintain this momentum, more regular bibliographic and bibliometric analyses will be needed. This would encourage authors to consider publishing their articles in relevant, high-visibility journals and help these journals expand their supportive publication policies and become more inclusive of digital health. %M 31855190 %R 10.2196/10477 %U http://www.jmir.org/2019/12/e10477/ %U https://doi.org/10.2196/10477 %U http://www.ncbi.nlm.nih.gov/pubmed/31855190 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 12 %P e16368 %T Open Access as a Revolution: Knowledge Alters Power %A deBronkart,Dave %+ e-Patient Dave, LLC, 17 Grasmere Lane, Nashua, NH, United States, 1 6034595119, dave@epatientdave.com %K patient engagement %K empowerment %K patient empowerment %K participatory medicine %K open access %K patient portals %K EMRs %K EHRs %K Patient-clinician relationship %D 2019 %7 11.12.2019 %9 Viewpoint %J J Med Internet Res %G English %X The slogan “Gimme My Damn Data” has become a hallmark of a patient movement whose goal is to gain access to data in their medical records. Its first conference appearance was ten years ago, in September 2009. In the decade since there have been enormous changes in both the technology and sociology of medicine as well as in their synthesis. As the patient movement has made strides, it has been met with opposition and obstacles. It has also become clear that the availability of Open Access information is just as empowering (or disabling) as access to electronic medical records and device data. Knowledge truly is power, and to withhold knowledge is to disempower patients. This essay lays out many examples of how this shows up as we strive for the best future of care. %M 31825321 %R 10.2196/16368 %U http://www.jmir.org/2019/12/e16368/ %U https://doi.org/10.2196/16368 %U http://www.ncbi.nlm.nih.gov/pubmed/31825321 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 7 %N 4 %P e13430 %T Impact of Automatic Query Generation and Quality Recognition Using Deep Learning to Curate Evidence From Biomedical Literature: Empirical Study %A Afzal,Muhammad %A Hussain,Maqbool %A Malik,Khalid Mahmood %A Lee,Sungyoung %+ Department of Computer Science and Engineering, Kyung Hee University, Room 313, Yongin, 446-701, Republic of Korea, 82 312012514, sylee@oslab.khu.ac.kr %K data curation %K evidence-based medicine %K clinical decision support systems %K precision medicine %K biomedical research %K machine learning %K deep learning %D 2019 %7 9.12.2019 %9 Original Paper %J JMIR Med Inform %G English %X Background: The quality of health care is continuously improving and is expected to improve further because of the advancement of machine learning and knowledge-based techniques along with innovation and availability of wearable sensors. With these advancements, health care professionals are now becoming more interested and involved in seeking scientific research evidence from external sources for decision making relevant to medical diagnosis, treatments, and prognosis. Not much work has been done to develop methods for unobtrusive and seamless curation of data from the biomedical literature. Objective: This study aimed to design a framework that can enable bringing quality publications intelligently to the users’ desk to assist medical practitioners in answering clinical questions and fulfilling their informational needs. Methods: The proposed framework consists of methods for efficient biomedical literature curation, including the automatic construction of a well-built question, the recognition of evidence quality by proposing extended quality recognition model (E-QRM), and the ranking and summarization of the extracted evidence. Results: Unlike previous works, the proposed framework systematically integrates the echelons of biomedical literature curation by including methods for searching queries, content quality assessments, and ranking and summarization. Using an ensemble approach, our high-impact classifier E-QRM obtained significantly improved accuracy than the existing quality recognition model (1723/1894, 90.97% vs 1462/1894, 77.21%). Conclusions: Our proposed methods and evaluation demonstrate the validity and rigorousness of the results, which can be used in different applications, including evidence-based medicine, precision medicine, and medical education. %M 31815673 %R 10.2196/13430 %U http://medinform.jmir.org/2019/4/e13430/ %U https://doi.org/10.2196/13430 %U http://www.ncbi.nlm.nih.gov/pubmed/31815673 %0 Journal Article %@ 2562-0959 %I JMIR Publications %V 2 %N 1 %P e16078 %T Transparent, Reproducible, and Open Science Practices of Published Literature in Dermatology Journals: Cross-Sectional Analysis %A Anderson,J Michael %A Niemann,Andrew %A Johnson,Austin L %A Cook,Courtney %A Tritz,Daniel %A Vassar,Matt %+ Oklahoma State University Center for Health Sciences, 1111 W 17th St, Tulsa, OK, 74107, United States, 1 918 582 1972, jande31@okstate.edu %K reproducibility of findings %K data sharing %K publishing, open access %K dermatology %D 2019 %7 7.11.2019 %9 Original Paper %J JMIR Dermatol %G English %X Background: Reproducible research is a foundational component for scientific advancements, yet little is known regarding the extent of reproducible research within the dermatology literature. Objective: This study aimed to determine the quality and transparency of the literature in dermatology journals by evaluating for the presence of 8 indicators of reproducible and transparent research practices. Methods: By implementing a cross-sectional study design, we conducted an advanced search of publications in dermatology journals from the National Library of Medicine catalog. Our search included articles published between January 1, 2014, and December 31, 2018. After generating a list of eligible dermatology publications, we then searched for full text PDF versions by using Open Access Button, Google Scholar, and PubMed. Publications were analyzed for 8 indicators of reproducibility and transparency—availability of materials, data, analysis scripts, protocol, preregistration, conflict of interest statement, funding statement, and open access—using a pilot-tested Google Form. Results: After exclusion, 127 studies with empirical data were included in our analysis. Certain indicators were more poorly reported than others. We found that most publications (113, 88.9%) did not provide unmodified, raw data used to make computations, 124 (97.6%) failed to make the complete protocol available, and 126 (99.2%) did not include step-by-step analysis scripts. Conclusions: Our sample of studies published in dermatology journals do not appear to include sufficient detail to be accurately and successfully reproduced in their entirety. Solutions to increase the quality, reproducibility, and transparency of dermatology research are warranted. More robust reporting of key methodological details, open data sharing, and stricter standards journals impose on authors regarding disclosure of study materials might help to better the climate of reproducible research in dermatology. %R 10.2196/16078 %U http://derma.jmir.org/2019/1/e16078/ %U https://doi.org/10.2196/16078 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 10 %P e16390 %T Beyond the Impact Factor: Reflecting on Twenty Years of Leading Efforts in Research, Innovation in Publishing, and Investment in People %A Torous,John %+ Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, 75 Fenwood Rd, Boston, MA, 02115, United States, 1 682 7822, jtorous@bidmc.harvard.edu %K JMIR %K publishing %K eHealth %K digital health %K digital medicine %K knowledge dissemination %D 2019 %7 31.10.2019 %9 Viewpoint %J J Med Internet Res %G English %X This viewpoint celebrates the accomplishments of the Journal of Medical Internet Research (JMIR) on its twentieth anniversary and reviews accomplishments around research publications, journal innovation, and supporting people. %M 31674922 %R 10.2196/16390 %U http://www.jmir.org/2019/10/e16390/ %U https://doi.org/10.2196/16390 %U http://www.ncbi.nlm.nih.gov/pubmed/31674922 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 8 %P e13769 %T Perspectives From Authors and Editors in the Biomedical Disciplines on Predatory Journals: Survey Study %A Cohen,Andrew J %A Patino,German %A Kamal,Puneet %A Ndoye,Medina %A Tresh,Anas %A Mena,Jorge %A Butler,Christi %A Washington,Samuel %A Breyer,Benjamin N %+ Department of Urology, University of California-San Francisco, 400 Parnassus Avenue, Suite A610, San Francisco, CA, 94110, United States, 1 415 476 3372, Benjamin.Breyer@ucsf.edu %K predatory journals %K open access publication %K global %K citation %K literature %D 2019 %7 30.08.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Predatory journals fail to fulfill the tenets of biomedical publication: peer review, circulation, and access in perpetuity. Despite increasing attention in the lay and scientific press, no studies have directly assessed the perceptions of the authors or editors involved. Objective: Our objective was to understand the motivation of authors in sending their work to potentially predatory journals. Moreover, we aimed to understand the perspective of journal editors at journals cited as potentially predatory. Methods: Potential online predatory journals were randomly selected among 350 publishers and their 2204 biomedical journals. Author and editor email information was valid for 2227 total potential participants. A survey for authors and editors was created in an iterative fashion and distributed. Surveys assessed attitudes and knowledge about predatory publishing. Narrative comments were invited. Results: A total of 249 complete survey responses were analyzed. A total of 40% of editors (17/43) surveyed were not aware that they were listed as an editor for the particular journal in question. A total of 21.8% of authors (45/206) confirmed a lack of peer review. Whereas 77% (33/43) of all surveyed editors were at least somewhat familiar with predatory journals, only 33.0% of authors (68/206) were somewhat familiar with them (P<.001). Only 26.2% of authors (54/206) were aware of Beall’s list of predatory journals versus 49% (21/43) of editors (P<.001). A total of 30.1% of authors (62/206) believed their publication was published in a predatory journal. After defining predatory publishing, 87.9% of authors (181/206) surveyed would not publish in the same journal in the future. Conclusions: Authors publishing in suspected predatory journals are alarmingly uninformed in terms of predatory journal quality and practices. Editors’ increased familiarity with predatory publishing did little to prevent their unwitting listing as editors. Some suspected predatory journals did provide services akin to open access publication. Education, research mentorship, and a realignment of research incentives may decrease the impact of predatory publishing. %M 31471960 %R 10.2196/13769 %U http://www.jmir.org/2019/8/e13769/ %U https://doi.org/10.2196/13769 %U http://www.ncbi.nlm.nih.gov/pubmed/31471960 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 5 %P e12957 %T The Deep Learning–Based Recommender System “Pubmender” for Choosing a Biomedical Publication Venue: Development and Validation Study %A Feng,Xiaoyue %A Zhang,Hao %A Ren,Yijie %A Shang,Penghui %A Zhu,Yi %A Liang,Yanchun %A Guan,Renchu %A Xu,Dong %+ Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education, College of Computer Science and Technology, Jilin University, 2699 Qinjin Street, Changchun, 130012, China, 86 13944088266, xudong@missouri.edu %K recommender system %K deep learning %K convolutional neural network %K biomedical literature %K PubMed %D 2019 %7 24.05.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: It is of great importance for researchers to publish research results in high-quality journals. However, it is often challenging to choose the most suitable publication venue, given the exponential growth of journals and conferences. Although recommender systems have achieved success in promoting movies, music, and products, very few studies have explored recommendation of publication venues, especially for biomedical research. No recommender system exists that can specifically recommend journals in PubMed, the largest collection of biomedical literature. Objective: We aimed to propose a publication recommender system, named Pubmender, to suggest suitable PubMed journals based on a paper’s abstract. Methods: In Pubmender, pretrained word2vec was first used to construct the start-up feature space. Subsequently, a deep convolutional neural network was constructed to achieve a high-level representation of abstracts, and a fully connected softmax model was adopted to recommend the best journals. Results: We collected 880,165 papers from 1130 journals in PubMed Central and extracted abstracts from these papers as an empirical dataset. We compared different recommendation models such as Cavnar-Trenkle on the Microsoft Academic Search (MAS) engine, a collaborative filtering–based recommender system for the digital library of the Association for Computing Machinery (ACM) and CiteSeer. We found the accuracy of our system for the top 10 recommendations to be 87.0%, 22.9%, and 196.0% higher than that of MAS, ACM, and CiteSeer, respectively. In addition, we compared our system with Journal Finder and Journal Suggester, which are tools of Elsevier and Springer, respectively, that help authors find suitable journals in their series. The results revealed that the accuracy of our system was 329% higher than that of Journal Finder and 406% higher than that of Journal Suggester for the top 10 recommendations. Our web service is freely available at https://www.keaml.cn:8081/. Conclusions: Our deep learning–based recommender system can suggest an appropriate journal list to help biomedical scientists and clinicians choose suitable venues for their papers. %M 31127715 %R 10.2196/12957 %U http://www.jmir.org/2019/5/e12957/ %U https://doi.org/10.2196/12957 %U http://www.ncbi.nlm.nih.gov/pubmed/31127715 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 21 %N 1 %P e11429 %T The Most Influential Medical Journals According to Wikipedia: Quantitative Analysis %A Jemielniak,Dariusz %A Masukume,Gwinyai %A Wilamowski,Maciej %+ Department of Management in Networked and Digital Societies, Kozminski University, Jagiellonska 59, Warszawa, 03301, Poland, 48 604901352, darekj@kozminski.edu.pl %K citizen science %K medical journals %K open knowledge %K Wikipedia %K knowledge translation %K journalology %K medical publishing %K scholarly publishing %D 2019 %7 18.01.2019 %9 Original Paper %J J Med Internet Res %G English %X Background: Wikipedia, the multilingual encyclopedia, was founded in 2001 and is the world’s largest and most visited online general reference website. It is widely used by health care professionals and students. The inclusion of journal articles in Wikipedia is of scholarly interest, but the time taken for a journal article to be included in Wikipedia, from the moment of its publication to its incorporation into Wikipedia, is unclear. Objective: We aimed to determine the ranking of the most cited journals by their representation in the English-language medical pages of Wikipedia. In addition, we evaluated the number of days between publication of journal articles and their citation in Wikipedia medical pages, treating this measure as a proxy for the information-diffusion rate. Methods: We retrieved the dates when articles were included in Wikipedia and the date of journal publication from Crossref by using an application programming interface. Results: From 11,325 Wikipedia medical articles, we identified citations to 137,889 journal articles from over 15,000 journals. There was a large spike in the number of journal articles published in or after 2002 that were cited by Wikipedia. The higher the importance of a Wikipedia article, the higher was the mean number of journal citations it contained (top article, 48.13 [SD 33.67]; lowest article, 6.44 [SD 9.33]). However, the importance of the Wikipedia article did not affect the speed of reference addition. The Cochrane Database of Systematic Reviews was the most cited journal by Wikipedia, followed by The New England Journal of Medicine and The Lancet. The multidisciplinary journals Nature, Science, and the Proceedings of the National Academy of Sciences were among the top 10 journals with the highest Wikipedia medical article citations. For the top biomedical journal papers cited in Wikipedia's medical pages in 2016-2017, it took about 90 days (3 months) for the citation to be used in Wikipedia. Conclusions: We found evidence of “recentism,” which refers to preferential citation of recently published journal articles in Wikipedia. Traditional high-impact medical and multidisciplinary journals were extensively cited by Wikipedia, suggesting that Wikipedia medical articles have robust underpinnings. In keeping with the Wikipedia policy of citing reviews/secondary sources in preference to primary sources, the Cochrane Database of Systematic Reviews was the most referenced journal. %M 30664451 %R 10.2196/11429 %U http://www.jmir.org/2019/1/e11429/ %U https://doi.org/10.2196/11429 %U http://www.ncbi.nlm.nih.gov/pubmed/30664451 %0 Journal Article %@ 2291-9694 %I JMIR Publications %V 6 %N 3 %P e10295 %T Three-Dimensional Portable Document Format (3D PDF) in Clinical Communication and Biomedical Sciences: Systematic Review of Applications, Tools, and Protocols %A Newe,Axel %A Becker,Linda %+ Chair of Medical Informatics, Friedrich-Alexander University Erlangen-Nürnberg, Wetterkreuz 13, Erlangen, 91058, Germany, 49 913185 ext 26720, axel.newe@fau.de %K 3D PDF %K 3D visualization %K interactive %K clinical communication %K biomedical science %K tools %K protocols %K apps %K online data sharing %K scholarly publishing %K electronic publishing %D 2018 %7 07.08.2018 %9 Review %J JMIR Med Inform %G English %X Background: The Portable Document Format (PDF) is the standard file format for the communication of biomedical information via the internet and for electronic scholarly publishing. Although PDF allows for the embedding of three-dimensional (3D) objects and although this technology has great potential for the communication of such data, it is not broadly used by the scientific community or by clinicians. Objective: The objective of this review was to provide an overview of existing publications that apply 3D PDF technology and the protocols and tools for the creation of model files and 3D PDFs for scholarly purposes to demonstrate the possibilities and the ways to use this technology. Methods: A systematic literature review was performed using PubMed and Google Scholar. Articles searched for were in English, peer-reviewed with biomedical reference, published since 2005 in a journal or presented at a conference or scientific meeting. Ineligible articles were removed after screening. The found literature was categorized into articles that (1) applied 3D PDF for visualization, (2) showed ways to use 3D PDF, and (3) provided tools or protocols for the creation of 3D PDFs or necessary models. Finally, the latter category was analyzed in detail to provide an overview of the state of the art. Results: The search retrieved a total of 902 items. Screening identified 200 in-scope publications, 13 covering the use of 3D PDF for medical purposes. Only one article described a clinical routine use case; all others were pure research articles. The disciplines that were covered beside medicine were many. In most cases, either animal or human anatomies were visualized. A method, protocol, software, library, or other tool for the creation of 3D PDFs or model files was described in 19 articles. Most of these tools required advanced programming skills and/or the installation of further software packages. Only one software application presented an all-in-one solution with a graphical user interface. Conclusions: The use of 3D PDF for visualization purposes in clinical communication and in biomedical publications is still not in common use, although both the necessary technique and suitable tools are available, and there are many arguments in favor of this technique. The potential of 3D PDF usage should be disseminated in the clinical and biomedical community. Furthermore, easy-to-use, standalone, and free-of-charge software tools for the creation of 3D PDFs should be developed. %M 30087092 %R 10.2196/10295 %U http://medinform.jmir.org/2018/3/e10295/ %U https://doi.org/10.2196/10295 %U http://www.ncbi.nlm.nih.gov/pubmed/30087092 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 6 %P e10281 %T A Deep Learning Method to Automatically Identify Reports of Scientifically Rigorous Clinical Research from the Biomedical Literature: Comparative Analytic Study %A Del Fiol,Guilherme %A Michelson,Matthew %A Iorio,Alfonso %A Cotoi,Chris %A Haynes,R Brian %+ University of Utah, Department of Biomedical Informatics, 421 Wakara Way, Suite 140, Salt Lake City, UT, 84108, United States, 1 8015814080, guilherme.delfiol@utah.edu %K information retrieval %K evidence-based medicine %K deep learning %K machine learning %K literature databases %D 2018 %7 25.06.2018 %9 Original Paper %J J Med Internet Res %G English %X Background: A major barrier to the practice of evidence-based medicine is efficiently finding scientifically sound studies on a given clinical topic. Objective: To investigate a deep learning approach to retrieve scientifically sound treatment studies from the biomedical literature. Methods: We trained a Convolutional Neural Network using a noisy dataset of 403,216 PubMed citations with title and abstract as features. The deep learning model was compared with state-of-the-art search filters, such as PubMed’s Clinical Query Broad treatment filter, McMaster’s textword search strategy (no Medical Subject Heading, MeSH, terms), and Clinical Query Balanced treatment filter. A previously annotated dataset (Clinical Hedges) was used as the gold standard. Results: The deep learning model obtained significantly lower recall than the Clinical Queries Broad treatment filter (96.9% vs 98.4%; P<.001); and equivalent recall to McMaster’s textword search (96.9% vs 97.1%; P=.57) and Clinical Queries Balanced filter (96.9% vs 97.0%; P=.63). Deep learning obtained significantly higher precision than the Clinical Queries Broad filter (34.6% vs 22.4%; P<.001) and McMaster’s textword search (34.6% vs 11.8%; P<.001), but was significantly lower than the Clinical Queries Balanced filter (34.6% vs 40.9%; P<.001). Conclusions: Deep learning performed well compared to state-of-the-art search filters, especially when citations were not indexed. Unlike previous machine learning approaches, the proposed deep learning model does not require feature engineering, or time-sensitive or proprietary features, such as MeSH terms and bibliometrics. Deep learning is a promising approach to identifying reports of scientifically rigorous clinical research. Further work is needed to optimize the deep learning model and to assess generalizability to other areas, such as diagnosis, etiology, and prognosis. %M 29941415 %R 10.2196/10281 %U http://www.jmir.org/2018/6/e10281/ %U https://doi.org/10.2196/10281 %U http://www.ncbi.nlm.nih.gov/pubmed/29941415 %0 Journal Article %@ 2291-5222 %I JMIR Publications %V 6 %N 6 %P e135 %T Visualizing Collaboration Characteristics and Topic Burst on International Mobile Health Research: Bibliometric Analysis %A Shen,Lining %A Xiong,Bing %A Li,Wei %A Lan,Fuqiang %A Evans,Richard %A Zhang,Wei %+ School of Medicine and Health Management, Tongji Medical College, Huazhong University of Science & Technology, No.13 Hangkong Road, Wuhan, 430030, China, 86 027 83692730, shenln@163.com %K collaboration characteristics %K topic bursts %K international mobile health %K mHealth %K telemedicine %K bibliometric analysis %K bibliometrics %K research trends %D 2018 %7 05.06.2018 %9 Original Paper %J JMIR Mhealth Uhealth %G English %X Background: In the last few decades, mobile technologies have been widely adopted in the field of health care services to improve the accessibility to and the quality of health services received. Mobile health (mHealth) has emerged as a field of research with increasing attention being paid to it by scientific researchers and a rapid increase in related literature being reported. Objective: The purpose of this study was to analyze the current state of research, including publication outputs, in the field of mHealth to uncover in-depth collaboration characteristics and topic burst of international mHealth research. Methods: The authors collected literature that has been published in the last 20 years and indexed by Thomson Reuters Web of Science Core Collection (WoSCC). Various statistical techniques and bibliometric measures were employed, including publication growth analysis; journal distribution; and collaboration network analysis at the author, institution, and country collaboration level. The temporal visualization map of burst terms was drawn, and the co-occurrence matrix of these burst terms was analyzed by hierarchical cluster analysis and social network analysis. Results: A total of 2704 bibliographic records on mHealth were collected. The earliest paper centered on mHealth was published in 1997, with the number of papers rising continuously since then. A total of 21.28% (2318/10,895) of authors publishing mHealth research were first author, whereas only 1.29% (141/10,895) of authors had published one paper. The total degree of author collaboration was 4.42 (11,958/2704) and there are 266 core authors who have collectively published 53.07% (1435/2704) of the total number of publications, which means that the core group of authors has fundamentally been formed based on the Law of Price. The University of Michigan published the highest number of mHealth-related publications, but less collaboration among institutions exits. The United States is the most productive country in the field and plays a leading role in collaborative research on mHealth. There are 5543 different identified keywords in the cleaned records. The temporal bar graph clearly presents overall topic evolutionary process over time. There are 12 important research directions identified, which are in the imbalanced development. Moreover, the density of the network was 0.007, a relatively low level. These 12 topics can be categorized into 4 areas: (1) patient engagement and patient intervention, (2) health monitoring and self-care, (3) mobile device and mobile computing, and (4) security and privacy. Conclusions: The collaboration of core authors on mHealth research is not tight and stable. Furthermore, collaboration between institutions mainly occurs in the United States, although country collaboration is seen as relatively scarce. The focus of research topics on mHealth is decentralized. Our study might provide a potential guide for future research in mHealth. %R 10.2196/mhealth.9581 %U http://mhealth.jmir.org/2018/6/e135/ %U https://doi.org/10.2196/mhealth.9581 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 4 %P e86 %T Impact of Low Back Pain Clinical Trials Measured by the Altmetric Score: Cross-Sectional Study %A Araujo,Amanda Costa %A Nascimento,Dafne Port %A Gonzalez,Gabrielle Zoldan %A Maher,Christopher G %A Costa,Leonardo Oliveira Pena %+ Masters and Doctoral Programs in Physical Therapy, Universidade Cidade de São Paulo, Tatuapé, 448 Cesario Galeno St, São Paulo, 03070-000, Brazil, 55 11 2476 5749, mandaa_costa@hotmail.com %K Altmetric %K social impact %K clinical trials %K low back pain %D 2018 %7 05.04.2018 %9 Original Paper %J J Med Internet Res %G English %X Background: There is interest from authors and publishers in sharing the results of their studies over the Internet in order to increase their readership. In this way, articles tend to be discussed and the impact of these articles tends to be increased. In order to measure this type of impact, a new score (named Altmetric) was created. Altmetric aims to understand the individual impact of each article through the attention attracted online. Objective: The primary objective of this study was to analyze potential factors related with the publishing journal and the publishing trial that could be associated with Altmetric scores on a random sample of low back pain randomized controlled trials (RCTs). The secondary objective of this study was to describe the characteristics of these trials and their Altmetric scores. Methods: We searched for all low back pain RCTs indexed on the Physiotherapy Evidence Database (PEDro; www.pedro.org.au) published between 2010 and 2015. A total of 200 articles were randomly selected, and we extracted data related to the publishing trial, the publishing journal, methodological quality of the trials (measured by the 0-10 item PEDro scale), and total and individual scores of Altmetric mentioned and Altmetric reader. The study was a cross-sectional study, and multivariate regression models and descriptive statistics were used. Results: A total of four variables were associated with Altmetric mentioned score: impact factor (β-coefficient=3.4 points), number of years since publication (β-coefficient=–4.9 points), number of citations divided by years since publication (β-coefficient=5.2 points), and descriptive title (β-coefficient=–29.4 points). Only one independent variable was associated with Altmetric reader score: number of citations divided by years since publication (β-coefficient=10.1 points, 95% CI 7.74-12.46). We also found that the majority of articles were published in English, with a descriptive title, and published in open access journals endorsing the Consolidated Standards of Reporting Trials (CONSORT) statement. Conclusions: Researchers should preferably select high impact factor journals for submission and use declarative or interrogative titles, as these factors are likely to increase the visibility of their studies in social media. %M 29622526 %R 10.2196/jmir.9368 %U http://www.jmir.org/2018/4/e86/ %U https://doi.org/10.2196/jmir.9368 %U http://www.ncbi.nlm.nih.gov/pubmed/29622526 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 20 %N 2 %P e70 %T Open Availability of Patient Medical Photographs in Google Images Search Results: Cross-Sectional Study of Transgender Research %A Marshall,Zack %A Brunger,Fern %A Welch,Vivian %A Asghari,Shabnam %A Kaposy,Chris %+ School of Social Work, McGill University, Room 300, 3506 University Street, Montreal, QC, H3A 2A7, Canada, 1 514 398 5178, zack.marshall@mcgill.ca %K informed consent %K photography %K open access publishing %K confidentiality %K image repositories %K big data %K publication ethics %K transgender persons %D 2018 %7 26.02.2018 %9 Original Paper %J J Med Internet Res %G English %X Background: This paper focuses on the collision of three factors: a growing emphasis on sharing research through open access publication, an increasing awareness of big data and its potential uses, and an engaged public interested in the privacy and confidentiality of their personal health information. One conceptual space where this collision is brought into sharp relief is with the open availability of patient medical photographs from peer-reviewed journal articles in the search results of online image databases such as Google Images. Objective: The aim of this study was to assess the availability of patient medical photographs from published journal articles in Google Images search results and the factors impacting this availability. Methods: We conducted a cross-sectional study using data from an evidence map of research with transgender, gender non-binary, and other gender diverse (trans) participants. For the original evidence map, a comprehensive search of 15 academic databases was developed in collaboration with a health sciences librarian. Initial search results produced 25,230 references after duplicates were removed. Eligibility criteria were established to include empirical research of any design that included trans participants or their personal information and that was published in English in peer-reviewed journals. We identified all articles published between 2008 and 2015 with medical photographs of trans participants. For each reference, images were individually numbered in order to track the total number of medical photographs. We used odds ratios (OR) to assess the association between availability of the clinical photograph on Google Images and the following factors: whether the article was openly available online (open access, Researchgate.net, or Academia.edu), whether the article included genital images, if the photographs were published in color, and whether the photographs were located on the journal article landing page. Results: We identified 94 articles with medical photographs of trans participants, including a total of 605 photographs. Of the 94 publications, 35 (37%) included at least one medical photograph that was found on Google Images. The ability to locate the article freely online contributes to the availability of at least one image from the article on Google Images (OR 2.99, 95% CI 1.20-7.45). Conclusions: This is the first study to document the existence of medical photographs from peer-reviewed journals appearing in Google Images search results. Almost all of the images we searched for included sensitive photographs of patient genitals, chests, or breasts. Given that it is unlikely that patients consented to sharing their personal health information in these ways, this constitutes a risk to patient privacy. Based on the impact of current practices, revisions to informed consent policies and guidelines are required. %M 29483069 %R 10.2196/jmir.8787 %U http://www.jmir.org/2018/2/e70/ %U https://doi.org/10.2196/jmir.8787 %U http://www.ncbi.nlm.nih.gov/pubmed/29483069 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 19 %N 4 %P e49 %T An Analysis of 2.3 Million Participations in the Continuing Medical Education Program of a General Medical Journal: Suitability, User Characteristics, and Evaluation by Readers %A Christ,Hildegard %A Franklin,Jeremy %A Griebenow,Reinhard %A Baethge,Christopher %+ Institute of Medical Statistics, Informatics and Epidemiology, University of Cologne, Kerpener Str 62, Cologne, 50937, Germany, 49 221 478 6502, Hildegard.Christ@uni-koeln.de %K education %K medical %K continuing interactive tutorial %K journal article %D 2017 %7 03.04.2017 %9 Original Paper %J J Med Internet Res %G English %X Background: Physicians frequently use continuing medical education (CME) in journals. However, little is known of the evaluation of journal CME by readers and also user and participation characteristics. Deutsches Ärzteblatt, the journal of the German Medical Association, is distributed to every physician in Germany and regularly offers its readers CME articles. Therefore, it provides a unique opportunity to analyze a journal CME program directed at an entire population of physicians. Objective: The aim is to show key sociodemographic characteristics of participants, frequency and temporal distributions of participations, and to analyze whether the articles are suitable for a general medical audience, how physicians rate the CME articles, how successful they were in answering simple multiple-choice questions, and to detect distinct clusters of participants. Methods: Using obligatory online evaluation forms and multiple-choice questions, we analyzed all participations of the entire 142 CME articles published between September 2004 and February 2014. We compared demographic characteristics of participants with official figures on those characteristics as provided by the German Medical Association. Results: A total of 128,398 physicians and therapists (male: 54.64%, 70,155/128,393; median age class 40 to 49 years) participated 2,339,802 times (mean 16,478, SD 6436 participations/article). Depending on the year, between 12.33% (44,064/357,252) and 16.15% (50,259/311,230) of all physicians in the country participated at least once. The CME program was disproportionally popular with physicians in private practice, and many participations took place in the early mornings and evenings (4544.53%, 1,041,931/2,339,802) as well as over the weekend (28.70%, 671,563/2,339,802). Participation by specialty (ranked in descending order) was internal medicine (18.25%, 23,434/128,392), general medicine (16.38%, 21,033/128,392), anesthesiology (10.00%, 12,840/128,392), and surgery (7.06%, 9059/128,392). Participants rated the CME articles as intelligible to a wider medical audience and filling clinically relevant knowledge gaps; 78.57% (1,838,358/2,339,781) of the sample gave the CME articles very good or good marks. Cluster analysis revealed three groups, one comprised of only women, with two-thirds working in private practice. Conclusions: The CME article series of Deutsches Ärzteblatt is used on a regular basis by a considerable proportion of all physicians in Germany; its multidisciplinary articles are suitable to a broad spectrum of medical specialties. The program seems to be particularly attractive for physicians in private practice and those who want to participate from their homes and on weekends. Although many physicians emphasize that the articles address gaps in knowledge, it remains to be investigated how this impacts professional performance and patient outcomes. %M 28373156 %R 10.2196/jmir.6052 %U http://www.jmir.org/2017/4/e49/ %U https://doi.org/10.2196/jmir.6052 %U http://www.ncbi.nlm.nih.gov/pubmed/28373156 %0 Journal Article %@ 1438-8871 %I JMIR Publications %V 19 %N 2 %P e52 %T Public Funding and Open Access to Research: A Review of Canadian Multiple Sclerosis Research %A Bakker,Caitlin %A Stephenson,Carol %A Stephenson,Erin %A Chaves,Debbie %+ Health Sciences Libraries, University of Minnesota, 303 Diehl Hall, 505 Essex Street SE, Minneapolis, MN,, United States, 1 612 301 1353, cjbakker@umn.edu %K multiple sclerosis %K open access publishing %K research support as topic %D 2017 %7 27.02.2017 %9 Original Paper %J J Med Internet Res %G English %X Background: Multiple sclerosis (MS), a progressive demyelinating disease of the brain and spinal cord, is the leading cause of nontraumatic neurological damage in young adults. Canada has one of the highest reported incidents of MS, with estimates between 55 and 240 per 100,000 individuals. Between 2009 and 2014, the MS Society of Canada provided over Can $90 million to researchers and, since 2013, has encouraged researchers to make both current and previous research products openly available. Objective: The goal of the study was to determine the open access (OA) cost implications and repository policies of journals frequently used by a sample of MS researchers. This study benchmarked current publishing preferences by MS Society of Canada researchers by examining the OA full-text availability of journal articles written by researchers funded between 2009 and 2014. Methods: Researchers were identified from the 2009 to 2014 annual MS Society of Canada Research Summaries. Articles were identified through searches in Web of Science, Scopus, Medline and Embase (both via OVID). Journal level analysis included comparison of OA policies, including article processing charges (APCs) and repository policies. Data were analyzed using descriptive statistics. Results: There were 758 articles analyzed in this study, of which 288 (38.0%) were OA articles. The majority of authors were still relying on journal policies for deposit in PubMed Central or availability on publisher websites for OA. Gold OA journals accounted for 10.2% of the journals in this study and were associated with significantly lower APCs (US $1900) than in hybrid journals (US $3000). Review of the journal self-archiving options highlighted the complexity of stipulations that authors would have to navigate to legally deposit a version of their article. Conclusions: This study found that there are currently researcher- and publisher-imposed barriers to both the gold and green roads to OA. These results provide a current benchmark against which efforts to enhance openness can be measured and can serve as a reference point in future assessments of the impact of OA policies within this field. With funding agencies worldwide releasing OA mandates, future success in compliance will require changes to how researchers and publishers approach production and dissemination of research. %M 28242594 %R 10.2196/jmir.6250 %U http://www.jmir.org/2017/2/e52/ %U https://doi.org/10.2196/jmir.6250 %U http://www.ncbi.nlm.nih.gov/pubmed/28242594 %0 Journal Article %@ 1438-8871 %I JMIR Publications Inc. %V 16 %N 11 %P e259 %T Beyond Open Big Data: Addressing Unreliable Research %A Moseley,Edward T %A Hsu,Douglas J %A Stone,David J %A Celi,Leo Anthony %+ Institute for Medical Engineering & Science, Massachusetts Institute of Technology, 77 Massachusetts Avenue, E25-505, Cambridge, MA, 02139, United States, 1 6172537937, lceli@mit.edu %K open data %K unreliable research %K collaborative learning %K knowledge discovery %K peer review %K research culture %D 2014 %7 11.11.2014 %9 Original Paper %J J Med Internet Res %G English %X The National Institute of Health invests US $30.9 billion annually in medical research. However, the subsequent impact of this research output on society and the economy is amplified dramatically as a result of the actual medical treatments, biomedical innovations, and various commercial enterprises that emanate from and depend on these findings. It is therefore a great concern to discover that much of published research is unreliable. We propose extending the open data concept to the culture of the scientific research community. By dialing down unproductive features of secrecy and competition, while ramping up cooperation and transparency, we make a case that what is published would then be less susceptible to the sometimes corrupting and confounding pressures to be first or journalistically attractive, which can compromise the more fundamental need to be robustly correct. %M 25405277 %R 10.2196/jmir.3871 %U http://www.jmir.org/2014/11/e259/ %U https://doi.org/10.2196/jmir.3871 %U http://www.ncbi.nlm.nih.gov/pubmed/25405277 %0 Journal Article %@ 14388871 %I JMIR Publications Inc. %V 16 %N 4 %P e112 %T Is Biblioleaks Inevitable? %A Dunn,Adam G %A Coiera,Enrico %A Mandl,Kenneth D %+ Centre for Health Informatics, Australian Institute of Health Innovation, The University of New South Wales, Sydney, 2052, Australia, 61 9385 8699, a.dunn@unsw.edu.au %K bibliographic databases %K compromising of data %K open access %K public access to information %K peer-to-peer architectures %D 2014 %7 22.04.2014 %9 Viewpoint %J J Med Internet Res %G English %X In 2014, the vast majority of published biomedical research is still hidden behind paywalls rather than open access. For more than a decade, similar restrictions over other digitally available content have engendered illegal activity. Music file sharing became rampant in the late 1990s as communities formed around new ways to share. The frequency and scale of cyber-attacks against commercial and government interests has increased dramatically. Massive troves of classified government documents have become public through the actions of a few. Yet we have not seen significant growth in the illegal sharing of peer-reviewed academic articles. Should we truly expect that biomedical publishing is somehow at less risk than other content-generating industries? What of the larger threat—a “Biblioleaks” event—a database breach and public leak of the substantial archives of biomedical literature? As the expectation that all research should be available to everyone becomes the norm for a younger generation of researchers and the broader community, the motivations for such a leak are likely to grow. We explore the feasibility and consequences of a Biblioleaks event for researchers, journals, publishers, and the broader communities of doctors and the patients they serve. %M 24755534 %R 10.2196/jmir.3331 %U http://www.jmir.org/2014/4/e112/ %U https://doi.org/10.2196/jmir.3331 %U http://www.ncbi.nlm.nih.gov/pubmed/24755534 %0 Journal Article %@ 14388871 %I JMIR Publications Inc. %V 15 %N 11 %P e259 %T Correlation and Interaction Visualization of Altmetric Indicators Extracted From Scholarly Social Network Activities: Dimensions and Structure %A Liu,Chun Li %A Xu,Yue Quan %A Wu,Hui %A Chen,Si Si %A Guo,Ji Jun %+ Information Office, Library, China Medical University, No.92 of Beier Road, Heping District, Shenyang, Liaoning, Shenyang, Liaoning, 110001, China, 86 (024)23256666 ext 5434, liuchunliliangxu@163.com %K altmetrics %K article-level metrics %K scholarly social network tools %K indicator %K dimension %K structure %D 2013 %7 25.11.2013 %9 Original Paper %J J Med Internet Res %G English %X Background: Citation counts for peer-reviewed articles and the impact factor of journals have long been indicators of article importance or quality. In the Web 2.0 era, growing numbers of scholars are using scholarly social network tools to communicate scientific ideas with colleagues, thereby making traditional indicators less sufficient, immediate, and comprehensive. In these new situations, the altmetric indicators offer alternative measures that reflect the multidimensional nature of scholarly impact in an immediate, open, and individualized way. In this direction of research, some studies have demonstrated the correlation between altmetrics and traditional metrics with different samples. However, up to now, there has been relatively little research done on the dimension and interaction structure of altmetrics. Objective: Our goal was to reveal the number of dimensions that altmetric indicators should be divided into and the structure in which altmetric indicators interact with each other. Methods: Because an article-level metrics dataset is collected from scholarly social media and open access platforms, it is one of the most robust samples available to study altmetric indicators. Therefore, we downloaded a large dataset containing activity data in 20 types of metrics present in 33,128 academic articles from the application programming interface website. First, we analyzed the correlation among altmetric indicators using Spearman rank correlation. Second, we visualized the multiple correlation coefficient matrixes with graduated colors. Third, inputting the correlation matrix, we drew an MDS diagram to demonstrate the dimension for altmetric indicators. For correlation structure, we used a social network map to represent the social relationships and the strength of relations. Results: We found that the distribution of altmetric indicators is significantly non-normal and positively skewed. The distribution of downloads and page views follows the Pareto law. Moreover, we found that the Spearman coefficients from 91.58% of the pairs of variables indicate statistical significance at the .01 level. The non-metric MDS map divided the 20 altmetric indicators into three clusters: traditional metrics, active altmetrics, and inactive altmetrics. The social network diagram showed two subgroups that are tied to each other but not to other groups, thus indicating an intersection between altmetrics and traditional metric indicators. Conclusions: Altmetrics complement, and most correlate significantly with, traditional measures. Therefore, in future evaluations of the social impact of articles, we should consider not only traditional metrics but also active altmetrics. There may also be a transfer phenomenon for the social impact of academic articles. The impact transfer path has transfer, or intermediate, stations that transport and accelerate article social impact from active altmetrics to traditional metrics and vice versa. This discovery will be helpful to explain the impact transfer mechanism of articles in the Web 2.0 era. Hence, altmetrics are in fact superior to traditional filters for assessing scholarly impact in multiple dimensions and in terms of social structure. %M 24275693 %R 10.2196/jmir.2707 %U http://www.jmir.org/2013/11/e259/ %U https://doi.org/10.2196/jmir.2707 %U http://www.ncbi.nlm.nih.gov/pubmed/24275693 %0 Journal Article %@ 14388871 %I JMIR Publications Inc. %V 15 %N 11 %P e245 %T The Virtual Skeleton Database: An Open Access Repository for Biomedical Research and Collaboration %A Kistler,Michael %A Bonaretti,Serena %A Pfahrer,Marcel %A Niklaus,Roman %A Büchler,Philippe %+ Institute for Surgical Technology and Biomechanics, University of Bern, Stauffacherstrasse 78, Bern, 3014, Switzerland, 41 316315959, michael.kistler@istb.unibe.ch %K medical informatics %K Internet %K image processing %K computer-assisted %K demographic analysis %K statistical models %D 2013 %7 12.11.2013 %9 Original Paper %J J Med Internet Res %G English %X Background: Statistical shape models are widely used in biomedical research. They are routinely implemented for automatic image segmentation or object identification in medical images. In these fields, however, the acquisition of the large training datasets, required to develop these models, is usually a time-consuming process. Even after this effort, the collections of datasets are often lost or mishandled resulting in replication of work. Objective: To solve these problems, the Virtual Skeleton Database (VSD) is proposed as a centralized storage system where the data necessary to build statistical shape models can be stored and shared. Methods: The VSD provides an online repository system tailored to the needs of the medical research community. The processing of the most common image file types, a statistical shape model framework, and an ontology-based search provide the generic tools to store, exchange, and retrieve digital medical datasets. The hosted data are accessible to the community, and collaborative research catalyzes their productivity. Results: To illustrate the need for an online repository for medical research, three exemplary projects of the VSD are presented: (1) an international collaboration to achieve improvement in cochlear surgery and implant optimization, (2) a population-based analysis of femoral fracture risk between genders, and (3) an online application developed for the evaluation and comparison of the segmentation of brain tumors. Conclusions: The VSD is a novel system for scientific collaboration for the medical image community with a data-centric concept and semantically driven search option for anatomical structures. The repository has been proven to be a useful tool for collaborative model building, as a resource for biomechanical population studies, or to enhance segmentation algorithms. %M 24220210 %R 10.2196/jmir.2930 %U http://www.jmir.org/2013/11/e245/ %U https://doi.org/10.2196/jmir.2930 %U http://www.ncbi.nlm.nih.gov/pubmed/24220210 %0 Journal Article %@ 14388871 %I JMIR Publications Inc. %V 15 %N 10 %P e210 %T Wikis and Collaborative Writing Applications in Health Care: A Scoping Review %A Archambault,Patrick M %A van de Belt,Tom H %A Grajales III,Francisco J %A Faber,Marjan J %A Kuziemsky,Craig E %A Gagnon,Susie %A Bilodeau,Andrea %A Rioux,Simon %A Nelen,Willianne LDM %A Gagnon,Marie-Pierre %A Turgeon,Alexis F %A Aubin,Karine %A Gold,Irving %A Poitras,Julien %A Eysenbach,Gunther %A Kremer,Jan AM %A Légaré,France %+ Centre de recherche du Centre hospitalier affilié universitaire de Lévis, Centre de santé et de services sociaux Alphonse-Desjardins (Centre hospitalier affilié universitaire de Lévis), 143 rue Wolfe, Lévis, QC, G6V 3Z1, Canada, 1 418 835 7121 ext 3905, patrick.m.archambault@gmail.com %K collaborative writing applications %K collaborative authoring %K knowledge management %K crowdsourcing %K medical informatics %K ehealth %K Internet %K Wiki %K Wikipedia %K Google Docs %K Google Knol %K Web 2.0 %K knowledge translation %K evidence-based medicine %K participatory med %D 2013 %7 08.10.2013 %9 Review %J J Med Internet Res %G English %X Background: Collaborative writing applications (eg, wikis and Google Documents) hold the potential to improve the use of evidence in both public health and health care. The rapid rise in their use has created the need for a systematic synthesis of the evidence of their impact as knowledge translation (KT) tools in the health care sector and for an inventory of the factors that affect their use. Objective: Through the Levac six-stage methodology, a scoping review was undertaken to explore the depth and breadth of evidence about the effective, safe, and ethical use of wikis and collaborative writing applications (CWAs) in health care. Methods: Multiple strategies were used to locate studies. Seven scientific databases and 6 grey literature sources were queried for articles on wikis and CWAs published between 2001 and September 16, 2011. In total, 4436 citations and 1921 grey literature items were screened. Two reviewers independently reviewed citations, selected eligible studies, and extracted data using a standardized form. We included any paper presenting qualitative or quantitative empirical evidence concerning health care and CWAs. We defined a CWA as any technology that enables the joint and simultaneous editing of a webpage or an online document by many end users. We performed qualitative content analysis to identify the factors that affect the use of CWAs using the Gagnon framework and their effects on health care using the Donabedian framework. Results: Of the 111 studies included, 4 were experimental, 5 quasi-experimental, 5 observational, 52 case studies, 23 surveys about wiki use, and 22 descriptive studies about the quality of information in wikis. We classified them by theme: patterns of use of CWAs (n=26), quality of information in existing CWAs (n=25), and CWAs as KT tools (n=73). A high prevalence of CWA use (ie, more than 50%) is reported in 58% (7/12) of surveys conducted with health care professionals and students. However, we found only one longitudinal study showing that CWA use is increasing in health care. Moreover, contribution rates remain low and the quality of information contained in different CWAs needs improvement. We identified 48 barriers and 91 facilitators in 4 major themes (factors related to the CWA, users’ knowledge and attitude towards CWAs, human environment, and organizational environment). We also found 57 positive and 23 negative effects that we classified into processes and outcomes. Conclusions: Although we found some experimental and quasi-experimental studies of the effectiveness and safety of CWAs as educational and KT interventions, the vast majority of included studies were observational case studies about CWAs being used by health professionals and patients. More primary research is needed to find ways to address the different barriers to their use and to make these applications more useful for different stakeholders. %M 24103318 %R 10.2196/jmir.2787 %U http://www.jmir.org/2013/10/e210/ %U https://doi.org/10.2196/jmir.2787 %U http://www.ncbi.nlm.nih.gov/pubmed/24103318 %0 Journal Article %@ 1438-8871 %I Gunther Eysenbach %V 13 %N 4 %P e123 %T Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact %A Eysenbach,Gunther %+ University Health Network, Centre for Global eHealth Innovation & Techna Institute, 190 Elizabeth St, Toronto, ON, M4L3Y7, Canada, 1 416 7866970, geysenba@uhnres.utoronto.ca %K bibliometrics %K blogging %K periodicals as topic %K peer-review %K publishing %K social media analytics %K scientometrics %K infodemiology %K infometrics %K reproducibility of results %K medicine 2.0 %K power law %K Twitter %D 2011 %7 16.12.2011 %9 Editorial %J J Med Internet Res %G English %X Background: Citations in peer-reviewed articles and the impact factor are generally accepted measures of scientific impact. Web 2.0 tools such as Twitter, blogs or social bookmarking tools provide the possibility to construct innovative article-level or journal-level metrics to gauge impact and influence. However, the relationship of the these new metrics to traditional metrics such as citations is not known. Objective: (1) To explore the feasibility of measuring social impact of and public attention to scholarly articles by analyzing buzz in social media, (2) to explore the dynamics, content, and timing of tweets relative to the publication of a scholarly article, and (3) to explore whether these metrics are sensitive and specific enough to predict highly cited articles. Methods: Between July 2008 and November 2011, all tweets containing links to articles in the Journal of Medical Internet Research (JMIR) were mined. For a subset of 1573 tweets about 55 articles published between issues 3/2009 and 2/2010, different metrics of social media impact were calculated and compared against subsequent citation data from Scopus and Google Scholar 17 to 29 months later. A heuristic to predict the top-cited articles in each issue through tweet metrics was validated. Results: A total of 4208 tweets cited 286 distinct JMIR articles. The distribution of tweets over the first 30 days after article publication followed a power law (Zipf, Bradford, or Pareto distribution), with most tweets sent on the day when an article was published (1458/3318, 43.94% of all tweets in a 60-day period) or on the following day (528/3318, 15.9%), followed by a rapid decay. The Pearson correlations between tweetations and citations were moderate and statistically significant, with correlation coefficients ranging from .42 to .72 for the log-transformed Google Scholar citations, but were less clear for Scopus citations and rank correlations. A linear multivariate model with time and tweets as significant predictors (P < .001) could explain 27% of the variation of citations. Highly tweeted articles were 11 times more likely to be highly cited than less-tweeted articles (9/12 or 75% of highly tweeted article were highly cited, while only 3/43 or 7% of less-tweeted articles were highly cited; rate ratio 0.75/0.07 = 10.75, 95% confidence interval, 3.4–33.6). Top-cited articles can be predicted from top-tweeted articles with 93% specificity and 75% sensitivity. Conclusions: Tweets can predict highly cited articles within the first 3 days of article publication. Social media activity either increases citations or reflects the underlying qualities of the article that also predict citations, but the true use of these metrics is to measure the distinct concept of social impact. Social impact measures based on tweets are proposed to complement traditional citation metrics. The proposed twimpact factor may be a useful and timely metric to measure uptake of research findings and to filter research findings resonating with the public in real time. %M 22173204 %R 10.2196/jmir.2012 %U http://www.jmir.org/2011/4/e123/ %U https://doi.org/10.2196/jmir.2012 %U http://www.ncbi.nlm.nih.gov/pubmed/22173204 %0 Journal Article %@ 1438-8871 %I Gunther Eysenbach %V 13 %N 4 %P e115 %T A Study of Innovative Features in Scholarly Open Access Journals %A Björk,Bo-Christer %+ Hanken School of Economics, P.O. Box 479, Helsinki, 00101, Finland, 358 50 3553425, Bo-Christer.Bjork@hanken.fi %K Scholarly publishing %K open access %K Internet %K peer review %D 2011 %7 16.12.2011 %9 Original Paper %J J Med Internet Res %G English %X Background: The emergence of the Internet has triggered tremendous changes in the publication of scientific peer-reviewed journals. Today, journals are usually available in parallel electronic versions, but the way the peer-review process works, the look of articles and journals, and the rigid and slow publication schedules have remained largely unchanged, at least for the vast majority of subscription-based journals. Those publishing firms and scholarly publishers who have chosen the more radical option of open access (OA), in which the content of journals is freely accessible to anybody with Internet connectivity, have had a much bigger degree of freedom to experiment with innovations. Objective: The objective was to study how open access journals have experimented with innovations concerning ways of organizing the peer review, the format of journals and articles, new interactive and media formats, and novel publishing revenue models. Methods: The features of 24 open access journals were studied. The journals were chosen in a nonrandom manner from the approximately 7000 existing OA journals based on available information about interesting journals and include both representative cases and highly innovative outlier cases. Results: Most early OA journals in the 1990s were founded by individual scholars and used a business model based on voluntary work close in spirit to open-source development of software. In the next wave, many long-established journals, in particular society journals and journals from regions such as Latin America, made their articles OA when they started publishing parallel electronic versions. From about 2002 on, newly founded professional OA publishing firms using article-processing charges to fund their operations have emerged. Over the years, there have been several experiments with new forms of peer review, media enhancements, and the inclusion of structured data sets with articles. In recent years, the growth of OA publishing has also been facilitated by the availability of open-source software for journal publishing. Conclusions: The case studies illustrate how a new technology and a business model enabled by new technology can be harnessed to find new innovative ways for the organization and content of scholarly publishing. Several recent launches of OA journals by major subscription publishers demonstrate that OA is rapidly gaining acceptance as a sustainable alternative to subscription-based scholarly publishing. %M 22173122 %R 10.2196/jmir.1802 %U http://www.jmir.org/2011/4/e115/ %U https://doi.org/10.2196/jmir.1802 %U http://www.ncbi.nlm.nih.gov/pubmed/22173122 %0 Journal Article %@ 1438-8871 %I Gunther Eysenbach %V 13 %N 4 %P e97 %T Public Access and Use of Health Research: An Exploratory Study of the National Institutes of Health (NIH) Public Access Policy Using Interviews and Surveys of Health Personnel %A O'Keeffe,Jamie %A Willinsky,John %A Maggio,Lauren %+ Stanford University School of Education, 485 Lasuen Mall, Stanford, CA, CA, 94305, United States, 1 650 862 9093, jamie.okeeffe@stanford.edu %K Health policy %K evidence-based practice %K information storage and retrieval %K access to information %K information literacy %K health personnel %D 2011 %7 21.11.2011 %9 Original Paper %J J Med Internet Res %G English %X Background: In 2008, the National Institutes of Health (NIH) Public Access Policy mandated open access for publications resulting from NIH funding (following a 12-month embargo). The large increase in access to research that will take place in the years to come has potential implications for evidence-based practice (EBP) and lifelong learning for health personnel. Objective: This study assesses health personnel’s current use of research to establish whether grounds exist for expecting, preparing for, and further measuring the impact of the NIH Public Access Policy on health care quality and outcomes in light of time constraints and existing information resources. Methods: In all, 14 interviews and 90 surveys of health personnel were conducted at a community-based clinic and an independent teaching hospital in 2010. Health personnel were asked about the research sources they consulted and the frequency with which they consulted these sources, as well as motivation and search strategies used to locate articles, perceived level of access to research, and knowledge of the NIH Public Access Policy. Results: In terms of current access to health information, 65% (57/88) of the health personnel reported being satisfied, while 32% (28/88) reported feeling underserved. Among the sources health personnel reported that they relied upon and consulted weekly, 83% (73/88) reported turning to colleagues, 77% (67/87) reported using synthesized information resources (eg, UpToDate and Cochrane Systematic Reviews), while 32% (28/88) reported that they consulted primary research literature. The dominant resources health personnel consulted when actively searching for health information were Google and Wikipedia, while 27% (24/89) reported using PubMed weekly. The most prevalent reason given for accessing research on a weekly basis, reported by 35% (31/88) of survey respondents, was to help a specific patient, while 31% (26/84) were motivated by general interest in research. Conclusions: The results provide grounds for expecting the NIH Public Access Policy to have a positive impact on EBP and health care more generally given that between a quarter and a third of participants in this study (1) frequently accessed research literature, (2) expressed an interest in having greater access, and (3) were aware of the policy and expect it to have an impact on their accessing research literature in the future. Results also indicate the value of promoting a greater awareness of the NIH policy, providing training and education in the location and use of the literature, and continuing improvements in the organization of biomedical research for health personnel use. %M 22106169 %R 10.2196/jmir.1827 %U http://www.jmir.org/2011/4/e97/ %U https://doi.org/10.2196/jmir.1827 %U http://www.ncbi.nlm.nih.gov/pubmed/22106169 %0 Journal Article %@ 1438-8871 %I Gunther Eysenbach %V 13 %N 1 %P e14 %T Wikipedia: A Key Tool for Global Public Health Promotion %A Heilman,James M %A Kemmann,Eckhard %A Bonert,Michael %A Chatterjee,Anwesh %A Ragar,Brent %A Beards,Graham M %A Iberri,David J %A Harvey,Matthew %A Thomas,Brendan %A Stomp,Wouter %A Martone,Michael F %A Lodge,Daniel J %A Vondracek,Andrea %A de Wolff,Jacob F %A Liber,Casimir %A Grover,Samir C %A Vickers,Tim J %A Meskó,Bertalan %A Laurent,Michaël R %+ Department of Internal Medicine, University Hospitals Leuven, Herestraat 49, Leuven, B-3000, Belgium, 32 485 143267, michael.laurent@gmail.com %K Internet %K Wikipedia %K public health %K health information %K knowledge dissemination %K patient education %K medical education %D 2011 %7 31.01.2011 %9 Viewpoint %J J Med Internet Res %G English %X The Internet has become an important health information resource for patients and the general public. Wikipedia, a collaboratively written Web-based encyclopedia, has become the dominant online reference work. It is usually among the top results of search engine queries, including when medical information is sought. Since April 2004, editors have formed a group called WikiProject Medicine to coordinate and discuss the English-language Wikipedia’s medical content. This paper, written by members of the WikiProject Medicine, discusses the intricacies, strengths, and weaknesses of Wikipedia as a source of health information and compares it with other medical wikis. Medical professionals, their societies, patient groups, and institutions can help improve Wikipedia’s health-related entries. Several examples of partnerships already show that there is enthusiasm to strengthen Wikipedia’s biomedical content. Given its unique global reach, we believe its possibilities for use as a tool for worldwide health promotion are underestimated. We invite the medical community to join in editing Wikipedia, with the goal of providing people with free access to reliable, understandable, and up-to-date health information. %M 21282098 %R 10.2196/jmir.1589 %U http://www.jmir.org/2011/1/e14/ %U https://doi.org/10.2196/jmir.1589 %U http://www.ncbi.nlm.nih.gov/pubmed/21282098