This paper is in the following e-collection/theme issue:

Digital Health Reviews (1289)

Published on in Vol 23, No 1 (2021): January

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/18901, first published .
Effect of Interactive eHealth Interventions on Improving Medication Adherence in Adults With Long-Term Medication: Systematic Review

Effect of Interactive eHealth Interventions on Improving Medication Adherence in Adults With Long-Term Medication: Systematic Review

Effect of Interactive eHealth Interventions on Improving Medication Adherence in Adults With Long-Term Medication: Systematic Review

Authors of this article:

Bart P H Pouls1, 2 Author Orcid Image ;   Johanna E Vriezekolk1 Author Orcid Image ;   Charlotte L Bekker2 Author Orcid Image ;   Annemiek J Linn3 Author Orcid Image ;   Hein A W van Onzenoort4, 5 Author Orcid Image ;   Marcia Vervloet6 Author Orcid Image ;   Sandra van Dulmen6, 7, 8 Author Orcid Image ;   Bart J F van den Bemt1, 2 Author Orcid Image
Article Authors Cited by (51) Tweetations (21) Metrics

    Review

    1Department of Rheumatology Research, Sint Maartenskliniek, Nijmegen, Netherlands

    2Department of Pharmacy, Radboud Institute for Health Sciences, Radboud University Medical Centre, Nijmegen, Netherlands

    3Amsterdam School of Communication Research, University of Amsterdam,, Amsterdam, Netherlands

    4Department of Clinical Pharmacy, Amphia Hospital, Breda, Netherlands

    5Department of Clinical Pharmacy and Toxicology, Maastricht University Medical Centre, Maastricht, Netherlands

    6Nivel (Netherlands Institute for Health Services Research), Utrecht, Netherlands

    7Department of Primary and Community Care, Radboud Institute for Health Sciences, Radboud University Medical Centre, Nijmegen, Netherlands

    8Faculty of Health and Social Sciences, University of South-Eastern Norway, Drammen, Norway

    Corresponding Author:

    Bart P H Pouls, PharmD

    Department of Rheumatology Research

    Sint Maartenskliniek

    Hengstdal 3

    Nijmegen, 6574NA

    Netherlands

    Phone: 31 0243272518

    Email: b.pouls@maartenskliniek.nl


    Background: Medication nonadherence leads to suboptimal treatment outcomes, making it a major priority in health care. eHealth provides an opportunity to offer medication adherence interventions with minimal effort from health care providers whose time and resources are limited.

    Objective: The aim of this systematic review is twofold: (1) to evaluate effectiveness of recently developed and tested interactive eHealth (including mHealth) interventions on medication adherence in adult patients using long-term medication and (2) to describe strategies among effective interventions.

    Methods: MEDLINE, EMBASE, Cochrane Library, PsycINFO, and Web of Science were systematically searched from January 2014 to July 2019 as well as reference lists and citations of included articles. Eligible studies fulfilled the following inclusion criteria: (1) randomized controlled trial with a usual care control group; (2) a total sample size of at least 50 adult patients using long-term medication; (3) applying an interactive eHealth intervention aimed at the patient or patient’s caregiver; and (4) medication adherence as primary outcome. Methodologic quality was assessed using the Cochrane risk of bias tool. Selection and quality assessment of studies were performed by 2 researchers (BP and BvdB or JV) independently. A best evidence synthesis was performed according to the Cochrane Back Review Group.

    Results: Of the 9047 records screened, 22 randomized clinical trials were included reporting on 29 interventions. Most (21/29, 72%) interventions specified using a (mobile) phone for calling, SMS text messaging, or mobile apps. A majority of all interactive interventions (17/29) had a statistically significant effect on medication adherence (P<.05). Of these interventions, 9 had at least a small effect size (Cohen d ≥ 0.2) and 3 showed strong odds for becoming adherent in the intervention group (odds ratio > 2.0). Our best evidence synthesis provided strong evidence for a positive effect of interventions using SMS text messages or interactive voice response, mobile app, and calls as mode of providing adherence tele-feedback. Intervention strategies “to teach medication management skills,” “to improve health care quality by coordinating medication adherence care between professionals,” and “to facilitate communication or decision making between patients and health care providers” also showed strong evidence for a positive effect.

    Conclusions: Overall, this review supports the hypothesis that interactive eHealth interventions can be effective in improving medication adherence. Intervention strategies that improve patients’ treatment involvement and their medication management skills are most promising and should be considered for implementation in practice.

    J Med Internet Res 2021;23(1):e18901

    doi:10.2196/18901

    Keywords

    eHealth; mHealth; medication adherence; interventions; long-term conditions 


    Long-term medication aims to reduce the risk of disease progression, comorbidity, and mortality [1]. These outcomes will only be reached when patients adhere to their medication. Presumably 50% of all patients with long-term medication are nonadherent, leading to suboptimal treatment outcomes [2,3]. Although improvement on clinical outcomes is the ultimate treatment goal, measuring adherence to long-term medication allows for comparison across chronic conditions.

    Medication adherence is defined as the extent to which medication taking behavior corresponds with the medication regimen agreed upon with the health care professional [4]. Medication-taking behavior is influenced by different factors such as experience, beliefs, and culture, making it multifaceted. Moreover, medication-taking behavior can differ between each drug and may change over time. Targeted, timely interventions enhancing medication adherence have therefore become one of the major priorities in health care. Despite efforts, randomized controlled trials have demonstrated limited effectiveness of medication-enhancing interventions [5-9]. Besides, effective interventions differed markedly and did not apply similar intervention strategies, making comparisons or meta-analysis difficult [3,5-7].

    eHealth might provide an opportunity to offer accessible, interactive, timely, and feasible medication adherence interventions that require minimal effort from health care providers whose time and resources are limited. eHealth or telemedicine—these words are used interchangeably—is defined as the use of information and communication technology in health care [10]. These technologies can facilitate tailored and interactive solutions such as targeted education, consistent support, and skill acquisition. Thus, the multifaceted and versatile medication-taking behavior can well be targeted by eHealth interventions.

    eHealth seems a promising way forward but recent systematic reviews showed conflicting results for eHealth interventions on improving medication adherence [11-14]. These reviews focused on single long-term conditions and have led to fragmented knowledge on the effectiveness and strategies of eHealth interventions. Evidence on eHealth interventions should be clustered to comprehensively investigate effectiveness of eHealth interventions and facilitate generalizability of study findings. Sieben et al [5] and Linn et al [15] found promising results across long-term conditions but the fast developments in eHealth render those results outdated, their definition of eHealth as “internet” was too narrow, and included studies had methodological limitations. Therefore the aim of our systematic review is twofold: (1) to evaluate effectiveness of recent interactive eHealth interventions on medication adherence in adult patients using long-term medication and (2) to describe applied strategies within effective interventions.


    This systematic review adheres to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) statement [16] and was completed according to the registered protocol PROSPERO 2019 CRD42019088873 [17].

    Search Strategy and Study Selection

    Searches were undertaken in MEDLINE, EMBASE, Cochrane Library, PsycINFO, and Web of Science to identify eligible studies. The search strategy comprised 3 blocks: eHealth, medication adherence, and randomized clinical trial (see Multimedia Appendix 1 for the MEDLINE search). Reference lists and citations of included studies were checked to ensure literature saturation. Titles and abstracts were screened and full-text articles were assessed by 2 researchers (BP and BvdB or BP and JV) independently based on the inclusion criteria below. Discrepancies between researchers were resolved through discussion or by reaching consensus with the third researcher (BvdB or JV).

    Eligibility Criteria

    Eligible studies fulfilled the following inclusion criteria: (1) randomized controlled trial with a usual care control group; (2) applying an interactive eHealth intervention aimed at the patient or patient’s caregiver; (3) medication adherence as primary outcome; (4) a total sample size of at least 50 adult patients using long-term medication as determined by Zwikker et al [18]; and (5) published between 2014 and July 2019. Only peer-reviewed English full-text articles were included. We considered all interventions solely applied over distance as eHealth interventions (eg, online portals, telephone calls). Blended care interventions, where face-to-face contact is combined with online components, were excluded. Interventions were considered interactive if there was tele-feedback regardless by whom on medication adherence (eg, bidirectional text messaging, sending adherence reports). Only validated medication adherence outcomes (ie, objective measures or subjective measures that have been compared to objective measures) were taken into account.

    Quality of Evidence

    Two researchers (BP and JV) independently assessed the internal validity of included studies using the Cochrane Collaboration’s tool for assessing risk of bias [19]. Seven domains were scored as having low (+), high (–), or unclear (?) risk of bias. Because blinding of participants and personnel is hardly feasible in studies evaluating interventions aimed at adherence, this domain was considered high risk (–) for all studies. Studies with a positive score (+) on at least five domains were considered high-quality studies. If relevant information was not reported, the corresponding author was contacted to request additional information. When no additional relevant information was provided, the risk of bias domain was scored as unclear (?).

    Data Extraction

    A standardized template was made to extract data on study characteristics, eHealth interventions, and medication adherence outcomes. Details of the eHealth interventions were extracted according to the Template for Intervention Description and Replication (TIDieR) checklist [20]. Additionally, the mode of adherence tele-feedback of each eHealth intervention was extracted. We distinguished the following modes of adherence tele-feedback: monitoring device, SMS text messaging, interactive voice response (IVR), mobile app, call, or e-training. Intervention strategies were categorized based on the strategies defined by Lowe et al [21] to support behavior change (ie, strategies focusing on adopting treatment behaviors); to inform and educate; to support (ie, strategies assisting consumers with their medicines use such as peer support); to teach skills; to minimize risk and harms (ie, strategies focusing on preventing or managing adverse events); to facilitate communication or decision making; and to improve health care quality (ie, strategies improving, coordinating, or integrating the delivered care). Only the adherence outcome at study endpoint was extracted where magnitude of the intervention effect would be most apparent. For continuous outcomes Cohen d for assessing effect size was calculated if means and SDs were provided [22]. Dichotomous outcomes were recalculated to odds ratio (OR) where OR ≥2.0 is deemed to be a strong OR [23]. Additionally, if the authors reported a statistical significance favoring the intervention group compared to the control group, this was scored positive (+). A negative score (–) means there was no statistically significant difference to report. Data were extracted by one researcher (BP) and checked for accuracy by a second researcher (JV).

    Data Analysis

    Statistical data pooling was not feasible due to heterogeneity between studies and interventions. Therefore a best evidence synthesis was performed to examine the effectiveness of interactive eHealth interventions on medication adherence. The Cochrane Back Review Group defines 4 levels of evidence: strong, moderate, limited, and conflicting evidence [24]. Strong evidence reflects consistent (ie, 75% or more of the studies report results in the same direction) results among 2 or more high quality studies. Moderate evidence reflects consistent results of 1 high-quality study and 2 or more lower-quality studies. Limited evidence reflects the result of 1 lower-quality study. Conflicting evidence reflects inconsistent results among 2 or more studies. If there were 2 or more high-quality studies, the lower-quality studies were disregarded in the best evidence synthesis. A post-hoc sensitivity analysis was performed to examine the robustness of the best evidence synthesis using a more stringent cut-off score (6 out of 7 instead of 5 out of 7 domains graded as low risk of bias) for determining the quality of the studies.


    Search Results

    Figure 1 shows a flow diagram of the literature search which provided a total of 9047 publications for screening and yielded 21 articles reporting on 29 interactive eHealth interventions [25-45]. One article, by Reese et al [34], reported on 2 studies. Five studies reported on more than 1 intervention [28,34,35,38,45].

    ‎
    Figure 1. PRISMA flow diagram of study search and selection.
    View this figure

    Risk of Bias Assessment

    Fifteen studies had a positive score on at least five domains and were regarded high-quality studies as shown in Figure 2. Two studies had the lowest score with 2 out of 7 domains scored as positive.

    ‎
    Figure 2. Summary of risk of bias assessment using the Cochrane Collaboration’s tool for assessing risk of bias.
    View this figure

    Description of Study Population

    Over half of the studies (13/22) included long-term medication for cardiovascular disease, diabetes, or both. Seven studies focused on other, single long-term conditions, leaving 2 studies that looked at any long-term conditions where long-term medication was in use.

    The smallest study reported on 70 participants at baseline and the largest study involved 21,752 participants. Because all studies were randomized, baseline characteristics of the different groups were generally the same.

    Description of Study Design

    Follow-up was short (ie, less than 6 months) in 11 studies and long (at least six months) in 11 studies. The primary medication adherence outcome of each of the studies was mainly assessed objectively using medication monitoring devices, pharmacy prescription data, and serum levels. The remaining 6 studies measured adherence subjectively with validated self-report questionnaires (eg, Immunosuppressant Therapy Adherence Instrument).

    Description of eHealth Interventions and Intervention Strategies

    Twenty-nine different interactive eHealth interventions were evaluated as shown in Table 1. Most (21/29, 72%) interventions specified using a (mobile) phone for calling, SMS text messaging, or mobile apps.

    Most (25/29) interventions were aimed at the patient, 3 interventions were aimed at the caregiver, and another was aimed at either patient or caregiver.

    Sixteen interventions were provided through automated software without involvement of a health care professional: 6 mobile apps, 5 monitoring devices, 3 SMS text messages or IVR interventions, and 2 e-training modules through an online portal. Another 7 interventions were provided through automated software in combination with tele-feedback by a health care professional or caregiver: 4 monitoring devices, 2 IVR or SMS text message interventions, 1 e-training. The 6 remaining interventions were telephone calls performed by health care professionals.

    Regarding intervention strategies, nearly all (23/29, 79%) interventions aimed at informing and educating patients and just over half (15/29, 52%) sought to support patients by providing assistance and encouragement. All other strategies (eg, teaching skills, facilitating communication or decision making) were less frequently applied (see Multimedia Appendix 2).

    Effectiveness of eHealth Interventions on Medication Adherence

    Overall, 17 interventions yielded a statistically significant improvement of medication adherence compared to the control group (Table 2). For 14 of these interventions an effect size (Cohen d) could also be calculated; 2 interventions reported a large effect size (Cohen d ≥ 0.8) [25,29], 4 had a medium effect size (Cohen d ≥ 0.5 < 0.8) [35,43,45], 3 had a small effect size (Cohen d ≥ 0.2 < 0.5) [26,32,35], and 5 interventions had a negligible effect size (Cohen d < 0.2) [27,30,37,38]. For the remaining 3 interventions an OR could be calculated which showed strong odds for becoming adherent in the intervention group (OR ≥ 2.0) [36,39,42].

    Table 1. Characteristics of the eHealth interventions.
    Study and medicationAdherence inclusion criterionIntervention arm (n)Control arm (n)Follow-up (in weeks)Mode of adherence tele-feedbackDescription of the intervention
    Levine et al [28]






    		TacrolimusNone385013AppTransplant Hero is an interactive alarm to remind patients to take their medications as well as providing educational content.

    		TacrolimusNone205013App and smart-watchTransplant Hero (see above) combined with a smartwatch that displayed the reminder notifications.
    Cizmic et al [39]






    		BisphosphonatesNone1271184IVRaAn IVR call focusing on known reasons for not initiating therapy. If the medication was not picked up 7 days after receiving the call, a reminder letter was sent.
    O’Connoret al [33]






    		Antihypertensives or medication for lowering blood glucose or cholesterolNone122011589CallA single-protocol-structured telephone call from an interventionist using positive reinforcement and probing for reasons of nonadherence.
    Kessler et al [45]






    		Statins<80%513426DeviceA wireless pill bottle generated an alert message, sent to the participant, if medication was missed the previous day and at least once in the 2 prior days.

    		Statins<80%463426DeviceA wireless pill bottle generated an automated alert message (see above), sent to the participant and a designated caregiver.
    Márquez Contreraset al [29]






    		AntihypertensivesNone737552AppThe AlerHTA app aimed to promote health education in hypertension and remind for both appointments and medication intake time.
    Montalescotet al [31]






    		ApixabanNone57958324e-TrainingAn education program consisting of an education booklet, one or more reminder tools chosen by the participant, and access to a telephone clinic.
    Reese et al [34]






    		Statins<80%676713DevicePROMOTE-1: a wireless pill bottle generated a weekly adherence report in which the patient’s adherence was compared to other patients.

    		Statins<80%676713DevicePROMOTE-2: a wireless pill bottle generated a weekly adherence report.

    		Statins<80%505013DeviceSUPPORT-1: a wireless pill bottle generated a daily adherence report.

    		Statins<80%505013DeviceSUPPORT-2: a wireless pill bottle generated a weekly adherence report.

    		Statins<80%505013DeviceSUPPORT-3: a wireless pill bottle generated an email alert if the patient missed a dose the previous day.
    Reese et al [35]






    		TacrolimusNone404026DeviceA wireless pill bottle generated an alert when medication was due and patients could select additional reminders such as SMS text messages, calls, or emails.

    		TacrolimusNone404026DeviceA wireless pill bottle generated an alert (see above). If adherence decreased to <90% in a 14-day period, the study coordinator would call the patient and notify the involved HCPsc.
    Svendsen et al [36]






    		Calcipotriol/betamethasone foamnone68664AppAn app which provided once-daily reminders and information on number of treatment applications and amount of prescribed foam applied.
    Abughosh et al [25]






    		RASb inhibitors<80%879926CallA brief telephone intervention by pharmacists to remind the patients of their overdue refill and to identify potential adherence barriers.
    Abughosh et al [26]






    		RAS inhibitors<80%24849526CallSix motivational interviewing phone calls by pharmacy students to identify potential adherence barriers and provide guidance to address these barriers.
    Choudhry et al [37]






    		Antihypertensives or medication for lowering blood glucose or cholesterol<80%2038204052CallTailored telephone consultation to develop a shared plan to improve adherence and disease control. At 6 and 9 months progress reports were mailed.
    Ducoulombier et al [42]






    		Bisphosphonates and strontium ranelateNone798552CallBimonthly telephone follow-up to motivate patients to maintain good adherence, detect difficulties in compliance, and recall the importance of treatment continuation.
    Kooij et al [27]






    		Bisphosphonates, RAS inhibitors, and statinsNone2008291452CallTelephone counselling 7-21 days after the start of therapy assessing practical and perceptual barriers and providing information and motivation.
    Vollmer et al [38]






    		RAS inhibitors and statins<90%7247725552IVRAn IVR call when (over)due for a refill providing patient education and refill support.

    		RAS inhibitors and statins<90%7250725552IVRIn addition to IVR calls (see above), a reminder letter was sent if they were 60-89 days overdue, a call was made if they were ≥90 days overdue, and primary care provider informed. Patients also received a personalized health report, a pill organizer, and bimonthly mailings.
    Claborn et al [40]






    		Highly active antiretroviral therapy<95%47504e-TrainingeLifeSteps: a single-session, self-paced multimedia intervention tackling practical and psychological adherence barriers accompanied with a workbook.
    Côtéet al [41]






    		ImmunosuppressantsNone353526e-TrainingTransplant-TAVIE was composed of 3 interactive Web-based sessions by a virtual nurse aimed at developing and reinforcing self-management skills required for medication intake.
    Kamal et al [43]






    		Preventive medication for strokeNone1001008SMS text messagesSMS4stroke sent automated customized SMS text message reminders to either patient or caregiver.
    Kamal et al [44]






    		Statins and antiplateletsNone999813IVR and SMSDaily IVR call services, daily prescription-tailored medication reminders, and once weekly life style modification messages.
    Mira et al [30]






    		All medication allowed, >2None514913AppA tablet-based medication self-management app (ALICE) with medication reminders and medication information such as pictures, interactions, storage instructions, and common errors in medication use.
    Morawski et al [32]






    		AntihypertensivesNone20920212AppThe MediSafe app is a medication reminder app with additional functions such as adherence reports, tracking of measurements, and peer support.

    aIVR: interactive voice response.

    bRAS: renin–angiotensin system.

    cHCP: health care professional.

    Table 2. Adherence measure and medication adherence results of the studies reviewed.
    Adherence measure, study, and results on medication adherenceStatistically significanta
    Serum level (0-100)

    		Levine et al [28]

    		The coefficient of variability (SD/mean × 100) of tacrolimus levels was 33.0 for the intervention group and 32.8 for the control group (Cohen d = 0.01).–
    The coefficient of variability was 33.8 for the intervention group and 32.8 for the control group (Cohen d = 0.07).–
    Fill first prescription (0%-100%)

    		Cizmic et al [39]


    		49% of the intervention group filled their first prescription compared to 31% of the control group (ORb 2.17; 95% CI 1.29-3.67).+

    		O’Connoret al [33]–


    		84% of the intervention group filled their first prescription compared to 84% of the control group (OR 0.94; 95% CI 0.79-1.11).–
    Bottle openings (0%-100%)

    		Kessler et al [45]


    		Average daily adherence was 53% for the intervention group and 36% for the control group (Cohen d = 0.70).+


    		Average daily adherence was 55% for the intervention group and 36% for the control group (Cohen d = 0.70).+

    		Márquez Contreraset al [29]


    		Average daily adherence was 86% for the intervention group and 63% for the control group (Cohen d = 4.72).+

    		Montalescotet al [31]


    		Average daily adherence was 92% for the intervention group and 92% for the control group (Cohen d = 0.02).–

    		Reese et al [34]


    		Average daily adherence was 77% for the intervention group and 75% for the control group.–


    		Average daily adherence was 71% for the intervention group and 75% for the control group.–

    		Reese et al [34]


    		Average daily adherence was 73% for the intervention group and 79% for the control group.–


    		Average daily adherence was 75% for the intervention group and 79% for the control group.–


    		Average daily adherence was 75% for the intervention group and 79% for the control group.–

    		Reese et al [35]


    		Average daily adherence (during the final 90 days) was 78% for the intervention group and 55% for the control group (Cohen d = 0.37).+


    		Average daily adherence (during final 90 days) was 88% for the intervention group and 55% for the control group (Cohen d = 0.57).+

    		Svendsen et al [36]


    		66% of the intervention group was considered adherent compared to 38% of the control group (OR 3.22; 95% CI 1.53-6.80).+
    PDCc,d (0%-100%)

    		Abughosh et al [25]


    		PDC was 58% for the intervention group and 29% for the control group (Cohen d = 1.32).+

    		Abughosh et al [26]


    		PDC was 66% for the intervention group and 57% for the control group (Cohen d = 0.26).+

    		Choudhry et al [37]


    		PDC was 46% for the intervention group and 42% for the control group (Cohen d = 0.12).+

    		Ducoulombier et al [42]+


    		65% of the intervention group was considered adherent compared to 33% of the control group (OR 3.71; 95% CI 1.94-7.07).+

    		Kooij et al [27]


    		PDC was 81% for the intervention group and 76% for the control group (Cohen d = 1.34).+

    		Vollmer et al [38]


    		PDC was 58% for the intervention group and 56% for the control group (Cohen d = 2.09).+


    		PDC was 59% for the intervention group and 56% for the control group (Cohen d = 2.14).+
    AACTGAIe (0%-100%)

    		Claborn et al [40]


    		Adherence was 81% for the intervention group and 81% for the control group (Cohen d = –0.03).–
    ITASf(0-12)

    		Côtéet al [41]


    		Mean ITAS score was 11.7 in the intervention group and 11.3 in the control group (Cohen d = 0.30).–
    MMAS g (0-8)

    		Kamal et al [43]


    		Mean MMAS score was 7.4 in the intervention group and 6.7 in the control group (Cohen d = 0.62).+
    MMAS (0-8)

    		Kamal et al [44]


    		Mean MMAS score was 7.3 in the intervention group and 7.1 in the control group (Cohen d = 0.03).–
    MMAS-4 (0-8)

    		Mira et al [30]


    		Mean MMAS score was 7.4 in the intervention group and 7.3 in the control group (Cohen d = 0.12; not corrected for baseline).+
    MMAS (0-8)

    		Morawski et al [32]


    		Mean MMAS score was 6.3 in the intervention group and 5.7 in the control group (Cohen d = 0.35).+

    aAs reported by the authors. + indicates P<.05 favoring intervention and – indicates P>.05 (no significant difference between groups).

    bOR: odds ratio.

    cPDC: percentage of days covered.

    dAll PDC outcomes were based on refill data; pill counts were considered separately.

    eAACTGAI: Adult AIDS Clinical Trials Group Adherence Instrument.

    fITAS: Immunosuppressant Therapy Adherence Instrument.

    gMMAS: Morisky Medication Adherence Scale.

    Further details of the study, population, intervention, and outcomes can be found in the extraction database provided as Multimedia Appendix 5.

    The best evidence synthesis (Table 3) showed strong evidence for a positive effect for SMS text messages or IVR, mobile apps, and calls as mode of adherence tele-feedback. The evidence for e-training was weak and for monitoring devices conflicting.

    In the post hoc sensitivity analysis the criteria for a high-quality study were more stringent (6 out of 7 domains graded as low risk of bias). The sensitivity analysis showed that the strong evidence for a positive effect for SMS or IVR as mode of adherence tele-feedback remained, whereas the evidence turned to conflicting for interventions delivered through mobile apps and calls (see Multimedia Appendix 3).

    The level of evidence of the intervention strategies was also assessed. There was strong evidence for a positive effect of strategies to teach skills, to facilitate communication or decision making, and to improve health care quality. For all other intervention strategies (eg, to support, to inform and educate) there was conflicting evidence (see Multimedia Appendix 4).

    Table 3. Results of the best evidence synthesis.
    Mode of adherence tele-feedback and qualityStatistically significantaLevel of evidence
    Monitoring device
    		Conflicting evidence

    		9 HQb interventions+, +, +, +, –, –, –, –, –
    0 LQc interventions
    SMS text messaging or IVRd
    		Strong evidence for a positive effect

    		5 HQ interventions+, +, +, +, –
    0 LQ interventions
    Mobile app
    		Strong evidence for a positive effect
    3 HQ interventions+, +, +
    3 LQ interventions+, –, –
    Call
    		Strong evidence for a positive effect

    		4 HQ interventions+, +, +, –
    2 LQ interventions+, +
    e-Training
    		Moderate evidence for no effect

    		1 HQ intervention–
    2 LQ interventions–, –

    a+ indicates P<.05 favoring intervention; – indicates P>.05 (no significant difference between groups). In grading the level of evidence low-quality studies were disregarded when there were 2 or more high-quality studies.

    bHQ: high quality.

    cLQ: lower quality.

    dIVR: interactive voice response.


    Principal Findings

    This systematic review examined the effectiveness of interactive eHealth interventions to improve medication adherence in patients using long-term medication published between 2014 and 2019. A majority, 17/29 interactive interventions, had a statistically significant (P<.05) effect on medication adherence. There was strong evidence for a positive effect for interventions using SMS or IVR, mobile apps, and calls as mode of adherence tele-feedback. Intervention strategies to teach skills, to improve health care quality, and to facilitate communication or decision making showed a strong evidence for a positive effect. Overall, this review shows that interactive eHealth interventions can be effective in improving medication adherence.

    Interactive eHealth Interventions

    This study showed strong evidence for a positive effect on medication adherence of eHealth interventions across various channels, including SMS, IVR, mobile apps, and calls. Our findings add robustness to the positive effect of eHealth interventions provided by previous systematic reviews and meta-analyses [46-51]. Where those authors were cautious with interpreting their findings because of low-quality studies, small sample sizes, and short follow-up, many studies we included were of high quality (22/29), had sample sizes of 100 patients or more (19/29), and follow-up of at least six months (14/29). IVR interventions that included information about health consequences suggest a stronger behavioral change, including medication-taking behavior [51]. This review confirms these findings as the included IVR interventions all contained information on the consequences of (not) taking medication as prescribed. For call, mobile app, and SMS text messaging interventions it remains unclear whether there are intervention elements (eg, content, intervention design, or extent of tailoring) that contribute to increased medication adherence because most eHealth interventions are multicomponent and elements vary widely across interventions [46,48,50].

    We found a lack of convincing evidence for interventions applying an electronic monitoring device or e-training. By contrast, van Heuckelum et al [52] found a positive effect for interventions using monitoring device feedback. In our review all interventions coupled their electronic monitoring devices to the same (Way To Health) communication platform which could be a possible explanation. Yet, van Heuckelum et al [52] also included interventions that gave face-to-face feedback on adherence data collected by monitoring devices. They showed that these interventions were effective, whereas those that applied tele-feedback were not. This suggests that feedback on tele-monitoring of medication adherence is best given face-to-face.

    Intervention Strategies

    To describe intervention strategies among effective interactive eHealth interventions we used Lowe’s taxonomy as it is specific for adherence interventions with clear examples for each strategy. Although other taxonomies (eg, Abraham and Michie [53], Demonceau et al [54], Kini and Ho [7]) could have been used, they show many conceptual similarities with Lowe’s taxonomy. Following Lowe’s taxonomy, we provide evidence for interactive adherence interventions aimed at teaching skills such as self-management programs, improving health care quality by coordinating medication adherence care between professionals, and facilitating communication or decision making between patient and health care professional. These results should be interpreted with caution because interventions were multifaceted and combined different strategies. Furthermore, the strategies with the highest level of evidence were also those that were less used. It is not possible to assign success to a single strategy within a multifaceted intervention. Nonetheless, the effective strategies we identified in this review suggest to be good starting points for development or selection of interventions.

    Patient Populations

    Noteworthy, the included studies in our review using eHealth interventions to address medication adherence reflect 2 distinct patient populations, namely, the large patient population (eg, metabolic and cardiovascular disease) and the population where optimal medication adherence is critical (eg, immunosuppressants, antiretroviral therapy). Applying eHealth to address medication adherence can be advantageous for both populations albeit for different reasons. eHealth interventions can be accessible for large patient populations, giving health care professionals a large outreach with limited resources. For populations where optimal medication adherence is critical, eHealth interventions can be tailored to patients’ specific needs and provide continuous support.

    Study Quality

    Where others found a lack of high-quality studies and stressed the importance of improving study quality [3,9,15], this review included 15 (out of 23) high-quality studies. The increase in quality presumably is a direct consequence of better reporting and study designs. We chose the Cochrane risk of bias tool (version 1) to assess study quality. This tool focusses on internal validity and does not cover all aspects of study design. We found flaws in study design that were not covered by the Cochrane risk of bias tool such as absence of sample size calculation, selection bias, and disputable (adherence) outcomes. This could have (negatively) affected the implications of the results.

    Clinical Implications and Future Research

    We clustered evidence of various long-term conditions in our best evidence synthesis to provide a comprehensive overview. This overview is based on the statistically significant effects (P<.05) found by the authors and supported by the effect sizes we calculated. Of the 17 statistically significant (P<.05) interventions, 9 showed at least a small effect size (Cohen d ≥ 0.2) and 3 interventions showed strong odds (OR > 2.0) for effect in the intervention group.

    The synthesis was limited to medication adherence and did not consider other clinical outcomes. As a result, our findings may not be applicable one-on-one to specific conditions. Next step is to study the identified effective interventions/strategies in specific long-term conditions to ascertain that this may lead to improved medication adherence and other clinical outcomes.

    Only postintervention effectiveness on medication adherence was assessed in this review. Whether the found beneficial effects will be maintained over a longer period (>12 months) remains unclear. However, 12/17 effective interventions in our review had a follow-up of at least six months which is considered the shortest period to accurately assess long-term medication adherence [4].

    We were surprised to find many interactive eHealth interventions that use technologies published in the 20th century. Although technology changes, applied techniques are very similar. To be able to build upon data and lessons learnt from older technologies, crosslinks between similar techniques need to be made (eg, between SMS text messaging and chat services such as WhatsApp or WeChat).

    Technological developments are very fast paced and eHealth interventions continuously change. This high turnaround speed creates a need for study designs that allow continuous evaluation of interventions over a period of at least six months.

    In this review intervention exposure ranged from a single call to daily messages for months. To establish a relation between exposure and medication taking behavior change, dose–response studies are called for.

    Conclusion

    We found that a majority of interactive eHealth interventions are effective in improving adherence to long-term medication. Intervention strategies that improve patient’s treatment involvement and their medication management skills are most promising. While most interactive eHealth interventions were multifaceted, even simple eHealth technologies such as SMS text messaging and telephone calls can be effective in promoting medication adherence in a wide variety of patient populations.

    Authors' Contributions

    BP developed the PROSPERO protocol which was reviewed by JV, AL, HvO, MV, SvD, and BvdB. BP developed and conducted search strategy and screening and inclusion of the studies, designed extraction template, extracted data, and drafted the manuscript. Rating study quality and revision of subsequent drafts of the manuscript were done by BP and JV. JV and BvdB also performed article screening. CB, AL, HvO, MV, SvD, and BvdB critically reviewed the manuscript. All authors read and approved the final manuscript.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    MEDLINE search strategy.

    DOC File , 30 KB
    ‎
    Multimedia Appendix 2

    Overview of the intervention strategies present in each e-Health intervention.

    PNG File , 27 KB
    ‎
    Multimedia Appendix 3

    Sensitivity analysis.

    DOC File , 33 KB
    ‎
    Multimedia Appendix 4

    Level of evidence of intervention strategies.

    DOC File , 37 KB
    ‎
    Multimedia Appendix 5

    Extraction database.

    XLSX File (Microsoft Excel File), 121 KB
    1. Christensen A. Patient Adherence to Medical Treatment Regimens. New Haven, CT: Yale University Press; 2004.
    2. Corrao G, Parodi A, Nicotra F, Zambon A, Merlino L, Cesana G, et al. Better compliance to antihypertensive medications reduces cardiovascular risk. J Hypertens 2011 Mar;29(3):610-618. [CrossRef] [Medline]
    3. Nieuwlaat R, Wilczynski N, Navarro T, Hobson N, Jeffery R, Keepanasseril A, et al. Interventions for enhancing medication adherence. Cochrane Database Syst Rev 2014;11:CD000011. [CrossRef] [Medline]
    4. Przemyslaw K, Andrzejczyk A, Clyne W, De GS, Demonceau J, Dobbels F. Ascertaining Barriers for Compliance: policies for safe, effective and cost-effective use of medicines in Europe - Final Report of The ABC Project. 2012.   URL: http://abcproject.eu/img/ABC%20Final.pdf [accessed 2020-12-16]
    5. Sieben A, Bredie SJHB, van Laarhoven CJHMK, Schoonhoven L, Burger DM, van Onzenoort HAW. [e-Health interventions and improvement in treatment adherence]. Ned Tijdschr Geneeskd 2014;158:A8460. [Medline]
    6. Viswanathan M, Golin CE, Jones CD, Ashok M, Blalock SJ, Wines RCM, et al. Interventions to improve adherence to self-administered medications for chronic diseases in the United States: a systematic review. Ann Intern Med 2012 Dec 4;157(11):785-795. [CrossRef] [Medline]
    7. Kini V, Ho PM. Interventions to Improve Medication Adherence: A Review. JAMA 2018 Dec 18;320(23):2461-2473. [CrossRef] [Medline]
    8. Wilhelmsen NC, Eriksson T. Medication adherence interventions and outcomes: an overview of systematic reviews. Eur J Hosp Pharm 2019 Jul;26(4):187-192. [CrossRef] [Medline]
    9. Conn VS, Ruppar TM. Medication adherence outcomes of 771 intervention trials: Systematic review and meta-analysis. Prev Med 2017 Jun;99:269-276. [CrossRef] [Medline]
    10. Ryu S. Telemedicine: Opportunities and Developments in Member States: Report on the Second Global Survey on eHealth 2009 (Global Observatory for eHealth Series, Volume 2). Healthc Inform Res 2012;18(2):153. [CrossRef]
    11. Hamine S, Gerth-Guyette E, Faulx D, Green BB, Ginsburg AS. Impact of mHealth chronic disease management on treatment adherence and patient outcomes: a systematic review. J Med Internet Res 2015;17(2):e52 [FREE Full text] [CrossRef] [Medline]
    12. Wang Z, Zhu Y, Cui L, Qu B. Electronic Health Interventions to Improve Adherence to Antiretroviral Therapy in People Living With HIV: Systematic Review and Meta-Analysis. JMIR Mhealth Uhealth 2019 Oct 16;7(10):e14404 [FREE Full text] [CrossRef] [Medline]
    13. Stevenson JK, Campbell ZC, Webster AC, Chow CK, Tong A, Craig JC, et al. eHealth interventions for people with chronic kidney disease. Cochrane Database Syst Rev 2019 Aug 06;8:CD012379. [CrossRef] [Medline]
    14. Jeminiwa R, Hohmann L, Qian J, Garza K, Hansen R, Fox BI. Impact of eHealth on medication adherence among patients with asthma: A systematic review and meta-analysis. Respir Med 2019 Mar;149:59-68. [CrossRef] [Medline]
    15. Linn AJ, Vervloet M, van DL, Smit EG, Van WJCM. Effects of eHealth interventions on medication adherence: a systematic review of the literature. J Med Internet Res 2011;13(4):e103 [FREE Full text] [CrossRef] [Medline]
    16. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement. Open Med 2009;3(3):e123-e130 [FREE Full text] [Medline]
    17. Pouls B, Van den Bemt BJF, Vriezekolk J. Effectiveness of eHealth interventions on medication adherence in adults with chronic medication: a systematic review. 2019 Jun 19.   URL: https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=88873 [accessed 2020-12-16]
    18. Zwikker HE, van den Bemt BJ, Vriezekolk JE, van den Ende CH, van Dulmen S. Psychosocial predictors of non-adherence to chronic medication: systematic review of longitudinal studies. Patient Prefer Adherence 2014;8:519-563 [FREE Full text] [CrossRef] [Medline]
    19. Higgins JPT, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928 [FREE Full text] [Medline]
    20. Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ 2014;348:g1687 [FREE Full text] [Medline]
    21. Lowe D, Ryan R, Santesso N, Hill S. Development of a taxonomy of interventions to organise the evidence on consumers' medicines use. Patient Educ Couns 2011 Nov;85(2):e101-e107. [CrossRef] [Medline]
    22. Cohen J. A power primer. Psychol Bull 1992 Jul;112(1):155-159. [Medline]
    23. van der Windt DA, Thomas E, Pope DP, de Winter AF, Macfarlane GJ, Bouter LM, et al. Occupational risk factors for shoulder pain: a systematic review. Occup Environ Med 2000 Jul;57(7):433-442 [FREE Full text] [CrossRef] [Medline]
    24. Furlan AD, Pennick V, Bombardier C, van Tulder M. 2009 Updated Method Guidelines for Systematic Reviews in the Cochrane Back Review Group. Spine 2009;34(18):1929-1941. [CrossRef]
    25. Abughosh SM, Wang X, Serna O, Henges C, Masilamani S, Essien EJ, et al. A Pharmacist Telephone Intervention to Identify Adherence Barriers and Improve Adherence Among Nonadherent Patients with Comorbid Hypertension and Diabetes in a Medicare Advantage Plan. J Manag Care Spec Pharm 2016 Jan;22(1):63-73 [FREE Full text] [CrossRef] [Medline]
    26. Abughosh S, Wang X, Serna O, Esse T, Mann A, Masilamani S, et al. A Motivational Interviewing Intervention by Pharmacy Students to Improve Medication Adherence. J Manag Care Spec Pharm 2017 May;23(5):549-560 [FREE Full text] [CrossRef] [Medline]
    27. Kooij MJ, Heerdink ER, van Dijk L, van Geffen ECG, Belitser SV, Bouvy ML. Effects of Telephone Counseling Intervention by Pharmacists (TelCIP) on Medication Adherence; Results of a Cluster Randomized Trial. Front Pharmacol 2016;7:269 [FREE Full text] [CrossRef] [Medline]
    28. Levine D, Torabi J, Choinski K, Rocca JP, Graham JA. Transplant surgery enters a new era: Increasing immunosuppressive medication adherence through mobile apps and smart watches. Am J Surg 2019 Jul;218(1):18-20. [CrossRef] [Medline]
    29. Márquez Contreras E, Márquez Rivero S, Rodríguez García E, López-García-Ramos L, Pastoriza Vilas JC, Baldonedo Suárez A, Compliance Group of Spanish Society of Hypertension (SEH-LELHA). Specific hypertension smartphone app to improve medication adherence in hypertension: a cluster-randomized trial. Curr Med Res Opin 2018 Nov 15;35(1):1-15. [CrossRef] [Medline]
    30. Mira JJ, Navarro I, Botella F, Borrás F, Nuño-Solinís R, Orozco D, et al. A Spanish pillbox app for elderly patients taking multiple medications: randomized controlled trial. J Med Internet Res 2014;16(4):e99 [FREE Full text] [CrossRef] [Medline]
    31. Montalescot G, Brotons C, Cosyns B, Crijns HJ, D'Angelo A, Drouet L, AEGEAN Study Investigators. Educational Impact on Apixaban Adherence in Atrial Fibrillation (the AEGEAN STUDY): A Randomized Clinical Trial. Am J Cardiovasc Drugs 2020 Feb;20(1):61-71 [FREE Full text] [CrossRef] [Medline]
    32. Morawski K, Ghazinouri R, Krumme A, Lauffenburger JC, Lu Z, Durfee E, et al. Association of a Smartphone Application With Medication Adherence and Blood Pressure Control: The MedISAFE-BP Randomized Clinical Trial. JAMA Intern Med 2018 Jun 01;178(6):802-809. [CrossRef] [Medline]
    33. O'Connor PJ, Schmittdiel JA, Pathak RD, Harris RI, Newton KM, Ohnsorg KA, et al. Randomized trial of telephone outreach to improve medication adherence and metabolic control in adults with diabetes. Diabetes Care 2014 Dec;37(12):3317-3324 [FREE Full text] [CrossRef] [Medline]
    34. Reese PP, Kessler JB, Doshi JA, Friedman J, Mussell AS, Carney C, et al. Two Randomized Controlled Pilot Trials of Social Forces to Improve Statin Adherence among Patients with Diabetes. J Gen Intern Med 2016 Apr;31(4):402-410 [FREE Full text] [CrossRef] [Medline]
    35. Reese PP, Bloom RD, Trofe-Clark J, Mussell A, Leidy D, Levsky S, et al. Automated Reminders and Physician Notification to Promote Immunosuppression Adherence Among Kidney Transplant Recipients: A Randomized Trial. Am J Kidney Dis 2017 Mar;69(3):400-409. [CrossRef] [Medline]
    36. Svendsen MT, Andersen F, Andersen KH, Pottegård A, Johannessen H, Möller S, et al. A smartphone application supporting patients with psoriasis improves adherence to topical treatment: a randomized controlled trial. Br J Dermatol 2018 Nov;179(5):1062-1071. [CrossRef] [Medline]
    37. Choudhry NK, Isaac T, Lauffenburger JC, Gopalakrishnan C, Lee M, Vachon A, et al. Effect of a Remotely Delivered Tailored Multicomponent Approach to Enhance Medication Taking for Patients With Hyperlipidemia, Hypertension, and Diabetes: The STIC2IT Cluster Randomized Clinical Trial. JAMA Intern Med 2018 Sep 01;178(9):1182-1189 [FREE Full text] [CrossRef] [Medline]
    38. Vollmer WM, Owen-Smith AA, Tom JO, Laws R, Ditmer DG, Smith DH, et al. Improving adherence to cardiovascular disease medications with information technology. Am J Manag Care 2014 Nov;20(11 Spec No. 17):SP502-SP510 [FREE Full text] [Medline]
    39. Cizmic AD, Heilmann RMF, Milchak JL, Riggs CS, Billups SJ. Impact of interactive voice response technology on primary adherence to bisphosphonate therapy: a randomized controlled trial. Osteoporos Int 2015 Aug;26(8):2131-2136. [CrossRef] [Medline]
    40. Claborn KR, Leffingwell TR, Miller MB, Meier E, Stephens JR. Pilot study examining the efficacy of an electronic intervention to promote HIV medication adherence. AIDS Care 2014;26(3):404-409. [CrossRef] [Medline]
    41. Côté J, Fortin M, Auger P, Rouleau G, Dubois S, Boudreau N, et al. Web-Based Tailored Intervention to Support Optimal Medication Adherence Among Kidney Transplant Recipients: Pilot Parallel-Group Randomized Controlled Trial. JMIR Form Res 2018 Jul 19;2(2):e14 [FREE Full text] [CrossRef] [Medline]
    42. Ducoulombier V, Luraschi H, Forzy G, Vandecandelaere M, Houvenagel E. Contribution of phone follow-up to improved adherence to oral osteoporosis treatment. Am J Pharm Benefits 2015;7(3):e81-e89.
    43. Kamal AK, Shaikh Q, Pasha O, Azam I, Islam M, Memon AA, et al. A randomized controlled behavioral intervention trial to improve medication adherence in adult stroke patients with prescription tailored Short Messaging Service (SMS)-SMS4Stroke study. BMC Neurol 2015 Oct 21;15:212 [FREE Full text] [CrossRef] [Medline]
    44. Kamal AK, Khalid W, Muqeet A, Jamil A, Farhat K, Gillani SRA, et al. Making prescriptions. PLoS One 2018;13(12):e0197671 [FREE Full text] [CrossRef] [Medline]
    45. Kessler JB, Troxel AB, Asch DA, Mehta SJ, Marcus N, Lim R, et al. Partners and Alerts in Medication Adherence: A Randomized Clinical Trial. J Gen Intern Med 2018 Sep;33(9):1536-1542 [FREE Full text] [CrossRef] [Medline]
    46. DeKoekkoek T, Given B, Given CW, Ridenour K, Schueller M, Spoelstra SL. mHealth SMS text messaging interventions and to promote medication adherence: an integrative review. J Clin Nurs 2015 Oct;24(19-20):2722-2735. [CrossRef] [Medline]
    47. Ng R, Carter SR, El-Den S. The impact of mobile applications on medication adherence: a systematic review. Transl Behav Med 2019 Aug 06:ibz125. [CrossRef] [Medline]
    48. Palacio A, Garay D, Langer B, Taylor J, Wood BA, Tamariz L. Motivational Interviewing Improves Medication Adherence: a Systematic Review and Meta-analysis. J Gen Intern Med 2016 Aug 9;31(8):929-940 [FREE Full text] [CrossRef] [Medline]
    49. Pérez-Jover V, Sala-González M, Guilabert M, Mira JJ. Mobile Apps for Increasing Treatment Adherence: Systematic Review. J Med Internet Res 2019 Jun 18;21(6):e12505 [FREE Full text] [CrossRef] [Medline]
    50. Thakkar J, Kurup R, Laba T, Santo K, Thiagalingam A, Rodgers A, et al. Mobile Telephone Text Messaging for Medication Adherence in Chronic Disease: A Meta-analysis. JAMA Intern Med 2016 Mar;176(3):340-349. [CrossRef] [Medline]
    51. Tsoli S, Sutton S, Kassavou A. Interactive voice response interventions targeting behaviour change: a systematic literature review with meta-analysis and meta-regression. BMJ Open 2018 Feb 24;8(2):e018974. [CrossRef] [Medline]
    52. van Heuckelum M, van den Ende CHM, Houterman AEJ, Heemskerk CPM, van Dulmen S, van den Bemt BJF. The effect of electronic monitoring feedback on medication adherence and clinical outcomes: A systematic review. PLoS One 2017;12(10):e0185453 [FREE Full text] [CrossRef] [Medline]
    53. Abraham C, Michie S. A taxonomy of behavior change techniques used in interventions. Health Psychol 2008 May;27(3):379-387. [CrossRef] [Medline]
    54. Demonceau J, Ruppar T, Kristanto P, Hughes DA, Fargher E, Kardas P, et al. Identification and assessment of adherence-enhancing interventions in studies assessing medication adherence through electronically compiled drug dosing histories: a systematic literature review and meta-analysis. Drugs 2013 May;73(6):545-562. [CrossRef] [Medline]

    ‎
    AACTG: Adult AIDS Clinical Trials Group Adherence Instrument
    HQ: high quality
    ITAS: Immunosuppressant Therapy Adherence Instrument
    IVR: interactive voice response
    LQ: lower quality
    MMAS: Morisky Medication Adherence Scale
    OR: odds ratio
    PDC: percentage of days covered
    RAS: renin–angiotensin system
    TIDieR: Template for Intervention Description and Replication

    Edited by G Eysenbach; submitted 27.03.20; peer-reviewed by L Carmona, R Ramos, M Fleming, T Patel, JK Low, M Taitel; comments to author 12.06.20; revised version received 06.08.20; accepted 30.10.20; published 08.01.21

    Copyright

    ©Bart P H Pouls, Johanna E Vriezekolk, Charlotte L Bekker, Annemiek J Linn, Hein A W van Onzenoort, Marcia Vervloet, Sandra van Dulmen, Bart J F van den Bemt. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 08.01.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.