This paper is in the following e-collection/theme issue:

Engagement with and Adherence to Digital Health Interventions, Law of Attrition (491) Diabetes Self-Management (484) Web-based and Mobile Health Interventions (3297) Apps, Mobile, Wearables for Diabetes (298) Glucose Tracking and Self-Monitoring of Blood Glucose (162) mHealth for Symptom and Disease Monitoring, Chronic Disease Management (1399) Mobile Health (mhealth) (2939)

Published on in Vol 23, No 7 (2021): July

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/23227, first published .
The Effects of Continuous Usage of a Diabetes Management App on Glycemic Control in Real-world Clinical Practice: Retrospective Analysis

The Effects of Continuous Usage of a Diabetes Management App on Glycemic Control in Real-world Clinical Practice: Retrospective Analysis

The Effects of Continuous Usage of a Diabetes Management App on Glycemic Control in Real-world Clinical Practice: Retrospective Analysis

Authors of this article:

Yu-Zhen Tu1 Author Orcid Image ;   Ya-Ting Chang1 Author Orcid Image ;   Hung-Yi Chiou2 Author Orcid Image ;   Ken Lai1 Author Orcid Image
Article Authors Cited by (12) Tweetations (2) Metrics

    Original Paper

    1H2 Inc, Taipei, Taiwan

    2School of Public Health, Taipei Medical University, Taipei, Taiwan

    Corresponding Author:

    Ken Lai, MSc

    H2 Inc

    4F, No 32, Aly 18, Ln 478

    Ruiguang Rd, Neihu Dist

    Taipei

    Taiwan

    Phone: 886 287976661 ext 105

    Email: klai@health2sync.com


    Background: The efficacy of digital technology in improving diabetes management has typically been demonstrated through studies such as randomized controlled trials, which have reported a steeper reduction in hemoglobin A1c (HbA1c) values for patients who adopted a digital solution. However, evidence from real-world clinical practice is still limited.

    Objective: This study aimed to evaluate the effectiveness of digital interventions by tracking HbA1c improvements over 1 year in real-world clinical settings.

    Methods: Patients used the Health2Sync mobile app to track self-measured outcomes and communicate with health care professionals (HCPs). HCPs used the web-based Patient Management Platform to monitor patient data, view test results from clinical laboratories, and communicate with patients. Patients who have been onboarded for at least 13 months and have consecutive HbA1c findings for 5 quarters were included in the analysis. They were then stratified into 3 groups (high, mid, and low retention) based on their level of use of Health2Sync in the first 6 months of onboarding. A mixed model was built to compare the slopes of the rate of reduction in HbA1c among the groups. In addition, these patients’ retention on the app from the seventh to the 12th month was verified through multiple comparisons.

    Results: A sample of 2036 users was included in the analysis. With the mixed model coefficient estimates, we found that app users had significant HbA1c percentage reductions as the passed quarter count increased (t=–9.869; P<.001), and that effectiveness increased in the high (t=–5.173) and mid retention (t=–6.620) groups as the interaction effects were significantly negative compared to that in the low retention group (P<.001) in the passed quarter count. The low retention group also had the highest average HbA1c value at the end of 13 months (high: 7.01%, SD 1.02%; mid: 6.99%, SD 1.00%; low: 7.17%, SD 1.14%) (Bonferroni correction: high vs low, P=.07; mid vs low, P=.02; high vs mid, P>.99). The level of use of the app remained consistent in the seventh to the 12th month after onboarding (high: 5.23 [SD 1.37] months, mid: 2.43 [SD 1.68] months, low: 0.41 [SD 0.97] months) (P<.001).

    Conclusions: Our analysis shows that continuous usage of the diabetes management app is associated with better glycemic control in real-world clinical practice. Further studies are required to reveal the efficacy for specific diabetes types and to observe effects beyond 1 year.

    J Med Internet Res 2021;23(7):e23227

    doi:10.2196/23227

    Keywords

    app; diabetes care; diabetes; digital intervention; digital therapeutics; glycemic control; mobile app; mHealth; real-world data; therapy 


    Global diabetes prevalence in the adult population is estimated to have grown from 8.5% in 2014 to 9.3% in 2019 and is projected to be over 10% by 2030 [1,2]. In Taiwan, the prevalence of diabetes approached 9.8% in 2016 [3]. Caring for patients with diabetes and a couple of its associated complications—chronic kidney disease and acute kidney failure—costed more than US $2.5 billion in Taiwan in 2019, which accounted for 11.4% of the total health care expenditure in that year [4]. As the cost of diabetes care continues to increase in the foreseeable future, health care providers will need to consider adopting digital solutions to manage patients with diabetes in an effective and scalable manner.

    Increasing evidence suggests that interventions with digital technology enhance the effect of conventional care practices on patients with diabetes. For example, patients who received coaching and decision support from the WellDoc app presented greater reductions in hemoglobin A1c (HbA1c) levels compared to those in the control group [5,6]. A cohort study using the FareWell app suggested the same effect on adults with type 2 diabetes, and the users reported greater confidence in managing the disorder [7]. Furthermore, multiple meta-analyses on randomized controlled trials (RCTs) have shown that interventions including mobile apps help patients, especially those with type 2 diabetes, to lower their HbA1c levels significantly without notable adverse effects [8-14]. These apps usually allow patients with diabetes to log and visualize their self-monitored data, provide education and feedback, or provide communication channels between patients and their caregivers or peers [12,13,15]. From the patient’s perspective, these features were also deemed as characteristics of a great app [16].

    Although the aforementioned mobile apps are intended to be used by patients in their daily lives, few studies have verified the effects of mobile apps with real-world data. A previous study [17] using the mobile version of the One Drop diabetes management platform reported a substantial reduction in HbA1c values of users for the first 2 entries in the app regardless of diabetes type, but the study did not compare effects between app users and nonusers and did not examine the long-term effects of app engagement [17].

    This study aims to address the aforementioned issues through a retrospective analysis. We evaluated the difference in HbA1c improvement between active app users and those who drop out from the app in a real-world context, and monitored their glycemic control status for 1 year.


    Health2Sync App and Patient Management Platform

    The Health2Sync mobile app and web-based Patient Management Platform were launched in 2014 to support patients with chronic diseases such as diabetes. The Health2Sync solution was developed to support patients to make behavioral changes through a do-track-learn cycle and enable those around the patient to care for the patient. The mobile app, intended for patients, is free to download for iOS and Android users worldwide. The Patient Management Platform is designed for health care professionals (HCPs) to care for patients in clinics or hospitals remotely.

    The Health2Sync app has numerous features that support diabetes management (Figure 1A). To start with, users can record their self-measured outcomes manually or by synchronizing with glucose meters, sphygmomanometers, and weight scales. They can also log their daily behaviors such as diet, medication, and exercise. Users can review their past records to remind themselves of how their behaviors are linked to the outcomes. On the dashboard, users can visualize their recent progress and trends by referring to simple charts and statistics. Health2Sync’s most differentiating feature is its “Partners” function, which enables users to connect with HCPs, family members, or peers so that they can collaborate with the patient to manage his/her condition. The “Partners” feature allows hospitals or clinics that use the Patient Management Platform to view the patient’s data and communicate with the patient through a messaging feature on the platform. Every user of the app also has the Health2Sync bot as a default partner that provides automated analyses, alerts, encouragements, and educational content. In addition, the app has a peer functionality that lets users learn and interact with other patients with diabetes who use the app.

    ‎
    Figure 1. Screenshots of the Health2Sync app and Patient Management Platform.
    View this figure

    The recorded outcomes and behavioral data from patients will be sent in real time to the Patient Management Platform to be reviewed by HCPs (Figure 1B). On the platform, HCPs can also send messages to patients to provide care and answer questions. From the Patient Management Platform’s dashboard, HCPs can quickly scan their pool of patients to check their latest status and identify any patients requiring immediate care. The alert functionality will notify HCPs of critical patient trends or events. The Patient Management Platform can also be integrated with clinical laboratories so that HCPs can view the results of laboratory tests directly on the platform. Test results including those for HbA1c, blood glucose, lipid profile, and renal and liver function are supported.

    Data Collection

    Users’ HbA1c records on the Patient Management Platform were included. All the HbA1c values were either inputted by HCPs or synchronized from clinical laboratories. To assess the effects of the intervention at different time points, we looked at when each patient added a clinic or hospital as a partner in Health2Sync and determined how many months after adding the partner the HbA1c values were recorded. The calculation method considered the difference in the number of days between the date when users added the partner and that of each HbA1c recording, divided by 30. According to the calculated numbers, these records were then categorized into five time buckets from B0 to B4: [–2, 1), [1, 4), [4, 7), [7, 10), and [10, 13). The buckets were designed to reflect the common practice of determining HbA1c levels once every 3 months in Taiwan, and that the first HbA1c record can be in –2~1 month from the time when the partnership was created. We believe that as glycated hemoglobin levels typically reflect blood glucose values in the past 8-12 weeks [18], HbA1c levels in the first month of the partnership would only be minimally affected by the intervention.

    Subjects

    Health2Sync users who were connected to clinics or hospitals via the Health2Sync platform for at least 13 months were included, thus yielding an initial pool of 14,386 users. Those with missing data in any of the 5-bucket HbA1c observation periods were excluded owing to the requirement of a complete data set for subsequent modeling. At this stage, we excluded 12,350 users and retained 2036 users. These users were then separated into 3 groups in accordance with their app retention. In the first 6 months after adding the partner, users who opened the Health2Sync app every month were categorized as the “high retention” group, those who opened the app in only the first month were categorized as the “low retention” group, and the remaining patients were categorized as the “mid retention” group. Eventually, there were 569, 1108, and 359 users in the high, mid, and low retention groups, respectively. Demographic data of the included 2036 users and their distribution across the 3 retention groups are depicted in Table 1, although the majority of these included users preferred not to disclose their demographic status. Throughout this study, all users continued to receive diabetes care from the same clinic or hospital, otherwise there would be no HbA1c records on the Patient Management Platform. Figure 2 shows the inclusion flow chart described above.

    ‎
    Figure 2. Inclusion flow chart of this study. The final 3 groups were separated by their retention in the first 6 months. Sample sizes of each stage are noted. PMP: Patient Management Platform.
    View this figure

    Analyses

    All the analyses in this study were performed with R (version 3.6.1, The R Foundation) [19].

    User Characteristics

    All users filled an onboarding form in the Health2Sync app where they self-reported their age, gender, and diabetes type. One-way analysis of variance (ANOVA) and the Pearson chi-square test were used for continuous and categorical variables, respectively, to assess the homogeneity of demographics across the 3 retention groups.

    HbA1c Changes

    Mean (SD) HbA1c levels of each group in B0 and B4 were calculated to compare the glycemic control status by app retention levels. One-way ANOVA was used to examine the heterogeneity across groups, and the pairwise t test with Bonferroni correction was applied for post hoc analysis.

    We used the rate of change in HbA1c levels, generated using the formula (HbA1c value - mean HbA1c in B0) / mean HbA1c in B0, to assess the improvement in glycemic status across the groups. The rate of change was used instead of the absolute magnitude of change because users who had higher HbA1c levels in B0 also had more room for a reduction in its value; hence, the comparison of changes in absolute HbA1c values across groups would be biased. In addition, considering that the reduction in HbA1c levels was nonlinear with time, all the changes in rates were added by a constant so that the minimum value is 1, and all these values were log-transformed. These log-transformed values were incorporated in the mixed model as the dependent variable, where fixed effects included the time bucket numbers (ie, quarters passed, from 0 to 4), app retention groups, and their interaction. Each user’s intercept was controlled in the random effects model [20,21].

    Monitoring of App Retention

    To follow the users’ app retention in the subsequent 6 months (ie, months 7-12 after partnership creation), we enumerated the months after which they had opened the app. The purpose of this analysis is to verify whether these 3 groups of users maintained their usage behavior in the following months of the retrospective observation period. The differences across these groups were assessed using 1-way ANOVA, and a pairwise t test with Bonferroni correction was used for post hoc analysis.


    Table 1 presents user characteristics stratified by the 3 user groups based on different levels of app retention. Users were not required to provide their demographic profiles; hence, sample sizes may differ across demographic characteristics. We found no significant difference in age (F2,806=1.441; P=.24) and gender (χ222=0.3637; P=.83) distributions across the 3 groups; however, the diabetes type distribution was not even in these groups (χ224=27.489; P<.001).

    Table 1. User characteristics stratified by app retention (N=2036).
    CharacteristicsTotalHigh retention (n=569)Mid retention (n=1108)Low retention (n=359)P value
    Age (years), mean (SD); n57.7 (13.4); 80957.1 (12.3); 29758.7 (13.7); 32757.1 (14.4); 185.24
    Gender, n.83

    		Male506226175105

    		Female46121415790
    Diabetes type, n<.001

    		Type 153211022

    		Type 2859384311164

    		Others332463
    HbA1c in time bucket B0 (%), mean (SD)7.90 (1.74)7.99 (1.86)7.92 (1.72)7.70 (1.60).04
    HbA1c in time bucket B4 (%), mean (SD)7.03 (1.03)7.01 (1.02)6.99 (1.00)7.17 (1.14).02

    Users with higher app retention presented a greater reduction in the rate of change in HbA1c levels (Figure 3). The group with the lowest retention presented the smallest reduction in HbA1c levels in time bucket B1 (–4.8%, SD 13.6%) and remained constant afterward up to time bucket B4 (–4.7%, SD 15.9%), while the other groups presented a steeper reduction in time bucket B1 (mid: –7.3%, SD 13.6%; high: –6.8%, SD 14.4%), and the decreasing trends continued up to time bucket B4 (mid: –9.2%, 15.5%; high: –9.3%, SD 16.4%). The mixed model further verified the significance of this effect (Figure 4). Users presented significant HbA1c percentage reductions as the passed quarter count increased (β estimate=–9.626×10-3; t=–9.869; P<.001). Furthermore, being in mid and high retention groups further augmented this reduction with time as the interactions were significantly negative when the low retention group was set as the baseline for the past quarter (mid: β estimate=–7.360×10-3; t=–6.620; P<.001; high: β estimate=–6.419×10-3; t=–5.173; P<.001). When the high retention group was set as the baseline for comparison, no significant difference was found between the augmentation effects between the mid and high retention groups for the past quarter (β estimate=–9.410×10-4; t=–1.007; P=.31). No main effect was observed for the retention groups when the low retention group was set as the baseline for comparison (mid: β estimate=–3.294×10-3; t=–0.691; P=.49; high: β estimate=–8.951×10-3; t=–1.687; P=.09).

    ‎
    Figure 3. Users' averaged percentage change in HbA1c levels in each time bucket. The error bars represent the SE among users in the group.
    View this figure
    ‎
    Figure 4. Jittered scatter plot depicting the relationship between HbA1c percentage changes and time periods after joining the Patient Management Platform. Each dot represents 1 user's data at the time. The overlaid regression lines are based on the estimated coefficients from the mixed model.
    View this figure

    The high and mid retention groups also presented a greater reduction in raw HbA1c values and lower HbA1c levels at the end of the observation period, although their raw HbA1c values were higher in time bucket B0 (Table 1 and Figure 5A). We observed significant differences in HbA1c levels between the high and low retention groups in time bucket B0 (high: 7.99%, SD 1.86%; 95% CI 7.83%-8.14%; mid: 7.92%, SD 1.72%; 95% CI 7.82%-8.02%; low: 7.70%, SD 1.60%; 95% CI 7.54%-7.87%) (Bonferroni correction: high vs low, P=.045; mid vs low, P=.11; high vs mid, P>.99) and between mid and low retention groups in time bucket B4 (high: 7.01%, SD 1.02%; 95% CI 6.93%-7.09%; mid: 6.99%, SD 1.00%; 95% CI 6.93%-7.05%; low: 7.17%, SD 1.14%; 95% CI 7.05%-7.29%) (Bonferroni correction: high vs low, P=.07; mid vs low, P=.02; high vs mid, P>.99). In a stratified analysis, users with HbA1c values of ≥8% in time bucket B0 presented no significant difference in glycemic status at the beginning (high: 9.79%, SD 1.66%; 95% CI 9.58%-10.01%; mid: 9.68%, SD 1.54%; 95% CI 9.53%-9.83%; low: 9.48%, SD 1.41%; 95% CI 9.23%-9.73%) (F2=1.632; P=.20), but the difference in HbA1c levels between the mid and low retention groups in time bucket B4 was still significant (high: 7.45%, SD 1.23%; 95% CI 7.29%-7.61%; mid: 7.44%, SD 1.21%; 95% CI 7.33%-7.56%; low: 7.76%, SD 1.36%; 95% CI 7.52%-8.00%) (Bonferroni correction: high vs low, P=.08; mid vs low, P=.04; high vs mid, P>.99) (Figure 5B). Similar results were obtained among users with HbA1c values of <8% in time bucket B0 (high: 6.81%, SD 0.62%; 95% CI 6.75%-6.88%; mid: 6.86%, SD 0.57%; 95% CI 6.81%-6.90%; low: 6.79%, SD 0.61%; 95% CI 6.71%-6.86%) (F2,1269=1.425; P=.24) and in time bucket B4 (high: 6.72%, SD 0.74%; 95% CI 6.65%-6.80%; mid: 6.72%, SD 0.72%; 95% CI 6.67%-6.77%; low: 6.86%, SD 0.86%; 95% CI 6.75%-6.97%) (Bonferroni correction: high vs low, P=.08; mid vs low, P=.03; high vs mid, P>.99) (Figure 5C).

    ‎
    Figure 5. Users' mean HbA1c in each time bucket. The error bars represent the standard errors among users in the group.
    View this figure

    Users’ retention behavior patterns remained the same in the second half of the tracking period (Figure 6). One-way ANOVA revealed significant differences in app retention across the 3 groups (F2,1797=1082; P<.001), and subsequent post hoc analyses using Bonferroni correction revealed significant differences among the groups (high: 5.23 [SD 1.37] months, mid: 2.43 [SD 1.68] months, low: 0.41 [SD 0.97] months) (P<.001) for all comparison pairs.

    ‎
    Figure 6. Month count when the app was opened during the seventh to 12th month. The heights of the bars and the ranges of error bars represent the mean (SE) values in each group, respectively.
    View this figure

    Principal Findings

    In this study, we found that patients with diabetes, who continuously used the Health2Sync app and were supported by HCPs who used the Health2Sync Patient Management Platform, presented a steeper reduction in HbA1c levels and achieved lower HbA1c levels after 1 year, compared to patients who dropped out and received traditional care for 1 year. Although we only used the app usage behavior data in the first 6 months to stratify the patients, their app usage patterns remained consistent in the following months. Additionally, our study shows positive results with regard to diabetes management from real-world data without prior RCT settings. While an RCT has an advantage over real-world evidence in providing homogeneous study groups for comparisons with designed inclusion and exclusion criteria, it is incapable of reflecting actual clinical practice where heterogeneous scenarios exist [22,23]. As digital interventions are to be applied to all patients, we believe that our study with real-world evidence is more convincing in demonstrating efficacy.

    In our study, both the mid and high retention groups presented a greater reduction in HbA1c levels than their counterparts in the low retention group. In other words, only users who stopped using the app in the first month after adding a clinic or hospital as a partner saw poorer HbA1c improvements than others. The beneficial effects of the Health2Sync app and Patient Management Platform might result from the in-app structured display and the general education or personalized feedback that further enhance users’ health awareness, which were reported to be associated with better HbA1c outcomes [13,24].

    Although the majority of included subjects chose not to disclose their diabetes type, and the sample size was unbalanced across known diabetes types and retention groups, we performed a subgroup analysis to investigate the effect of app retention on different diabetes types, specifically type 1 and type 2. Based on changes in the raw HbA1c values, we found that the reduction in the HbA1c levels of only the patients with type 2 diabetes was related to app retention, while that of patients with type 1 diabetes seemed unrelated to app adherence (Multimedia Appendices 1 and 2 show the trend in app retention for patients with type 1 and type 2 diabetes, respectively). Many studies have focused on the efficacy of digital interventions for patients with type 2 diabetes or provided inadequate evidence of the efficacy of such interventions among patients with type 1 diabetes [7,9,14]. Some studies or analyses reported positive outcomes among patients with type 1 diabetics, but the effects were not as large as those among patients with type 2 diabetes [8,13,17].

    Limitations

    Our data also suggest such differences between the patients with type 1 and those with type 2 diabetes. However, considering the limited sample, future studies are needed to investigate whether the benefits from digital tools can also be applied to patients with type 1 diabetes, especially in real-world settings. Another study limitation is that we did not consider the differences in medication and daily behaviors such as exercise across patients, and these factors could have vital impacts on glycemic control. Future studies should also include these features in analyses.

    With the growing prevalence of diabetes, it may become increasingly difficult to care for patients without increasing the number of HCPs. As such, digital solutions can play a critical role to enhance and scale diabetes care [25]. Moreover, during the COVID-19 pandemic, diabetes was found to be a common comorbidity, and patients with diabetes were found to be more vulnerable to COVID-19 [26-28]. A mobile app and platform that can enable HCPs to practice remote care should be helpful for glycemic control and can reduce the risk of people with diabetes being infected during visits to clinics or hospitals [29].

    Conclusions

    In conclusion, our retrospective analyses show that continuous usage of the Health2Sync app and Patient Management Platform was helpful for improving HbA1c levels. Further studies are needed to reveal the efficacy of such interventions for specific diabetes types and to observe the effects beyond 1 year.

    Authors' Contributions

    YZT was responsible for the design of the analysis protocol, conducting statistical analyses, and drafting the manuscript. YTC, HYC, and KL assisted with the statistical analyses. All authors reviewed the final manuscript.

    Conflicts of Interest

    The Health2Sync mobile app and web-based Patient Management Platform are products of H2 Inc. YZT, YTC, and KL are full-time employees at H2 Inc, and KL supervises YZT and YTC. HYC received a consulting fee to assist with the analyses but otherwise declared no conflict of interest.

    ‎
    Multimedia Appendix 1

    Diabetes type 1 users' mean HbA<sub>1c</sub> in each time bucket. The error bars represent the standard errors among users in the group.

    PNG File , 129 KB
    ‎
    Multimedia Appendix 2

    Diabetes type 2 users' mean HbA<sub>1c</sub> in each time bucket. The error bars represent the standard errors among users in the group.

    PNG File , 137 KB

    References

    1. Global Report on Diabetes. World Health Organization. 2016.   URL: https://apps.who.int/iris/handle/10665/204871 [accessed 2021-06-25]
    2. Saeedi P, Petersohn I, Salpea P, Malanda B, Karuranga S, Unwin N, IDF Diabetes Atlas Committee. Global and regional diabetes prevalence estimates for 2019 and projections for 2030 and 2045: Results from the International Diabetes Federation Diabetes Atlas, 9 edition. Diabetes Res Clin Pract 2019 Nov;157:107843. [CrossRef] [Medline]
    3. Health Promotion Administration, Ministry of Health and Welfare. 國民營養健康狀況變遷調查2013-2016年成果報告. 2013.   URL: https://www.hpa.gov.tw/Pages/ashx/File.ashx?FilePath=~/File/Attach/11145/File_12788.pdf [accessed 2021-06-25]
    4. National Health Insurance Administration, Ministry of Health and Welfare. 2018年全民健康保險醫療費用前二十大疾病. 2019.   URL: https://tinyurl.com/46yphwpw [accessed 2021-06-25]
    5. Quinn CC, Clough SS, Minor JM, Lender D, Okafor MC, Gruber-Baldini A. WellDoc mobile diabetes management randomized controlled trial: change in clinical and behavioral outcomes and patient and physician satisfaction. Diabetes Technol Ther 2008 Jun;10(3):160-168. [CrossRef] [Medline]
    6. Quinn CC, Shardell MD, Terrin ML, Barr EA, Ballew SH, Gruber-Baldini AL. Cluster-randomized trial of a mobile phone personalized behavioral intervention for blood glucose control. Diabetes Care 2011 Sep;34(9):1934-1942 [FREE Full text] [CrossRef] [Medline]
    7. Berman MA, Guthrie NL, Edwards KL, Appelbaum KJ, Njike VY, Eisenberg DM, et al. Change in Glycemic Control With Use of a Digital Therapeutic in Adults With Type 2 Diabetes: Cohort Study. JMIR Diabetes 2018 Feb 14;3(1):e4 [FREE Full text] [CrossRef] [Medline]
    8. Liang X, Wang Q, Yang X, Cao J, Chen J, Mo X, et al. Effect of mobile phone intervention for diabetes on glycaemic control: a meta-analysis. Diabet Med 2011 Apr;28(4):455-463. [CrossRef] [Medline]
    9. Hou C, Carter B, Hewitt J, Francisa T, Mayor S. Do Mobile Phone Applications Improve Glycemic Control (HbA1c) in the Self-management of Diabetes? A Systematic Review, Meta-analysis, and GRADE of 14 Randomized Trials. Diabetes Care 2016 Nov;39(11):2089-2095. [CrossRef] [Medline]
    10. Bonoto BC, de Araújo VE, Godói IP, de Lemos LLP, Godman B, Bennie M, et al. Efficacy of Mobile Apps to Support the Care of Patients With Diabetes Mellitus: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. JMIR Mhealth Uhealth 2017 Mar 01;5(3):e4 [FREE Full text] [CrossRef] [Medline]
    11. Faruque LI, Wiebe N, Ehteshami-Afshar A, Liu Y, Dianati-Maleki N, Hemmelgarn BR, Alberta Kidney Disease Network. Effect of telemedicine on glycated hemoglobin in diabetes: a systematic review and meta-analysis of randomized trials. CMAJ 2017 Mar 06;189(9):E341-E364 [FREE Full text] [CrossRef] [Medline]
    12. Greenwood DA, Gee PM, Fatkin KJ, Peeples M. A Systematic Review of Reviews Evaluating Technology-Enabled Diabetes Self-Management Education and Support. J Diabetes Sci Technol 2017 Sep;11(5):1015-1027 [FREE Full text] [CrossRef] [Medline]
    13. Wu Y, Yao X, Vespasiani G, Nicolucci A, Dong Y, Kwong J, et al. Mobile App-Based Interventions to Support Diabetes Self-Management: A Systematic Review of Randomized Controlled Trials to Identify Functions Associated with Glycemic Efficacy. JMIR Mhealth Uhealth 2017 Mar 14;5(3):e35 [FREE Full text] [CrossRef] [Medline]
    14. Kebede MM, Zeeb H, Peters M, Heise TL, Pischke CR. Effectiveness of Digital Interventions for Improving Glycemic Control in Persons with Poorly Controlled Type 2 Diabetes: A Systematic Review, Meta-analysis, and Meta-regression Analysis. Diabetes Technol Ther 2018 Nov;20(11):767-782. [CrossRef] [Medline]
    15. Payne HE, Lister C, West JH, Bernhardt JM. Behavioral functionality of mobile apps in health interventions: a systematic review of the literature. JMIR Mhealth Uhealth 2015 Feb 26;3(1):e20 [FREE Full text] [CrossRef] [Medline]
    16. Baptista S, Trawley S, Pouwer F, Oldenburg B, Wadley G, Speight J. What Do Adults with Type 2 Diabetes Want from the "Perfect" App? Results from the Second Diabetes MILES: Australia (MILES-2) Study. Diabetes Technol Ther 2019 Jul;21(7):393-399. [CrossRef] [Medline]
    17. Osborn CY, van Ginkel JR, Rodbard D, Heyman M, Marrero DG, Huddleston B, et al. One Drop | Mobile: An Evaluation of Hemoglobin A1c Improvement Linked to App Engagement. JMIR Diabetes 2017 Aug 24;2(2):e21 [FREE Full text] [CrossRef] [Medline]
    18. Nathan DM, Turgeon H, Regan S. Relationship between glycated haemoglobin levels and mean glucose levels over time. Diabetologia 2007 Nov;50(11):2239-2244. [CrossRef] [Medline]
    19. R: a language and environment for statistical computing. Global Biodiversity Information Facility.   URL: https://www.R-project.org/ [accessed 2021-06-25]
    20. Bates D, Mächler M, Bolker B, Walker S. Fitting Linear Mixed-Effects Models Using. J Stat Soft 2015;67(1). [CrossRef]
    21. Kuznetsova A, Brockhoff PB, Christensen RHB. Package: Tests in Linear Mixed Effects Models. J Stat Soft 2017;82(13). [CrossRef]
    22. Makady A, de Boer A, Hillege H, Klungel O, Goettsch W. What Is Real-World Data? A Review of Definitions Based on Literature and Stakeholder Interviews. Value Health 2017;20(7):858-865 [FREE Full text] [CrossRef] [Medline]
    23. Kim H, Lee S, Kim JH. Real-world Evidence versus Randomized Controlled Trial: Clinical Research Based on Electronic Medical Records. J Korean Med Sci 2018 Aug 20;33(34):e213 [FREE Full text] [CrossRef] [Medline]
    24. Tang YH, Pang SM, Chan MF, Yeung GS, Yeung VT. Health literacy, complication awareness, and diabetic control in patients with type 2 diabetes mellitus. J Adv Nurs 2008 Apr;62(1):74-83. [CrossRef] [Medline]
    25. Ramchandani N. Virtual Coaching to Enhance Diabetes Care. Diabetes Technol Ther 2019 Jun;21(S2):S248-S251. [CrossRef] [Medline]
    26. Fang L, Karakiulakis G, Roth M. Are patients with hypertension and diabetes mellitus at increased risk for COVID-19 infection? Lancet Respir Med 2020 Apr;8(4):e21 [FREE Full text] [CrossRef] [Medline]
    27. Zhou F, Yu T, Du R, Fan G, Liu Y, Liu Z, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet 2020 Mar 28;395(10229):1054-1062 [FREE Full text] [CrossRef] [Medline]
    28. Yang J, Zheng Y, Gou X, Pu K, Chen Z, Guo Q, et al. Prevalence of comorbidities and its effects in patients infected with SARS-CoV-2: a systematic review and meta-analysis. Int J Infect Dis 2020 May;94:91-95 [FREE Full text] [CrossRef] [Medline]
    29. Gupta R, Ghosh A, Singh AK, Misra A. Clinical considerations for patients with diabetes in times of COVID-19 epidemic. Diabetes Metab Syndr 2020;14(3):211-212 [FREE Full text] [CrossRef] [Medline]

    ‎
    ANOVA: analysis of variance
    HbA1c: hemoglobin A1c
    HCP: health care professional
    RCT: randomized controlled trial

    Edited by R Kukafka; submitted 05.08.20; peer-reviewed by B Carter, G Darville; comments to author 27.08.20; revised version received 22.10.20; accepted 24.05.21; published 15.07.21

    Copyright

    ©Yu-Zhen Tu, Ya-Ting Chang, Hung-Yi Chiou, Ken Lai. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 15.07.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.