This paper is in the following e-collection/theme issue:

Web-based and Mobile Health Interventions (3297)

Published on in Vol 13, No 3 (2011): Jul-Sep

A Systematic Review of the Impact of Adherence on the Effectiveness of e-Therapies

A Systematic Review of the Impact of Adherence on the Effectiveness of e-Therapies

A Systematic Review of the Impact of Adherence on the Effectiveness of e-Therapies

Authors of this article:

Liesje Donkin1 Author Orcid Image ;   Helen Christensen2 ;   Sharon L Naismith1 ;   Bruce Neal3 Author Orcid Image ;   Ian B Hickie1 ;   Nick Glozier1, 4 Author Orcid Image
Article Authors Cited by (596) Tweetations Metrics

    Original Paper

    1Brain & Mind Research Institute, The University of Sydney, Camperdown, Australia

    2Centre for Mental Health Research, Australian National University, Canberra, Australia

    3The George Institute for Global Health, The University of Sydney, Sydney, Australia

    4Disciplines of Psychiatry and Sleep Medicine, Sydney Medical School, The University of Sydney, Sydney, Australia

    Corresponding Author:

    Liesje Donkin, BSC, MSc

    Brain & Mind Research Institute

    The University of Sydney

    100 Mallet Street

    Camperdown, NSW 2050

    Australia

    Phone: 61 293510520

    Fax:61 293510855

    Email: Liesje.Donkin@sydney.edu.au


    Background: As the popularity of e-therapies grows, so too has the body of literature supporting their effectiveness. However, these interventions are often plagued by high attrition rates and varying levels of user adherence. Understanding the role of adherence may be crucial to understanding how program usage influences the effectiveness of e-therapy interventions.

    Objective: The aim of this study was to systematically review the e-therapy literature to (1) describe the methods used to assess adherence and (2) evaluate the association of adherence with outcome of these interventions.

    Methods: A systematic review of e-therapy interventions was conducted across disease states and behavioral targets. Data were collected on adherence measures, outcomes, and analyses exploring the relationship between adherence measures and outcomes.

    Results: Of 69 studies that reported an adherence measure, only 33 (48%) examined the relationship between adherence and outcomes. The number of logins was the most commonly reported measure of adherence, followed by the number of modules completed. The heterogeneity of adherence and outcome measures limited analysis. However, logins appeared to be the measure of adherence most consistently related to outcomes in physical health interventions, while module completion was found to be most related to outcomes in psychological health interventions.

    Conclusions: There is large variation in the reporting of adherence and the association of adherence with outcomes. A lack of agreement about how best to measure adherence is likely to contribute to the variation in findings. Physical and psychological outcomes seem influenced by different types of adherence. A composite measure encompassing time online, activity completion, and active engagements with the intervention may be the best measure of adherence. Further research is required to establish a consensus for measuring adherence and to understand the role of adherence in influencing outcomes.

    J Med Internet Res 2011;13(3):e52

    doi:10.2196/jmir.1772

    Keywords

    Adherence; persistence; online therapy; e-therapy; systematic review 


    The past two decades have seen a shift from traditional face-to-face consultations to technology-driven interventions or e-therapies. Recent reviews have shown that e-therapies are both effective [1-3] and growing in popularity. This is supported by the level of publications relating to e-therapies: Medline and PsycINFO citations in the subject group “online therapy” rose from 12 citations between 1991 and 2000 to 709 citations from 2001 to September 2010.

    A potential difficulty in evaluating these programs is adherence (see [4]). Little is known about the degree to which users’ engagement matches the usage pattern for which the websites are designed. Also, little is known about the influence of program adherence on outcomes. Within the medication literature, adherence, “the extent to which a person’s behaviour – taking medication, following a diet and/or executing lifestyle changes, corresponds with agreed recommendations from a health care provider” [5], and persistence, the act of adhering to treatment recommendations for the prescribed duration of time [6], are widely studied. These behavioral variables significantly influence medical [5] and psychotherapy [7-9] outcomes. In pharmaceutical trials, a dose–response curve is often plotted to understand the optimal level of medication to reach a desired response, and adherence is considered highly influential within this. In e-therapy, adherence may be just as important a consideration.

    The eHealth equivalent of failing to persist with therapy is treatment dropout. Treatment dropout refers to when a user prematurely stops using the intervention. Some of these users may remain in the trial, completing the trial assessments, while others may choose to leave the trial. Those who choose to leave the trial early are said to have discontinued and are reflected in trial attrition rates. Such attrition can affect the ability of results to be generalized [10,11] and it undermines the statistical power of the trial. Many authors note that dropout rates are high, particularly in open-access trials [12-14] where the intervention is made available to the public with minimal or no cost. Entry into these trials is therefore open, with users being able to join at any time. Despite each user’s choice to engage with the site, only a small proportion of users persist with the trial and associated follow-ups. However, with some programs, these figures may still be comparable to [15], or even lower than [16], the dropout rates found in traditional face-to-face therapies. Little is known about the impact of the degree of persistence on outcome in those who complete trial ratings and outcomes.

    While an appreciation of persistence is important in evaluating e-therapy, an understanding of adherence to the program content, such as the completion of program modules or online activities, may be more so. As the field of e-therapy has been growing, so has the interest in potentially modifiable user factors that may influence adherence. Clearly, program content evolves from empirically supported research, but only recently have the exploration and manipulation of program factors become foci. Program usability and feasibility testing is increasingly common. Recent findings have indicated that greater use of computer relational skills, such as the use of empathy and social dialogue in the computer program, leads to increased program usage [17]. Many trials use pilots and usability studies to ensure the program functions as planned [18-22]. Several authors have begun to explore the impact of reminders and trial factors on dropout. Clarke and colleagues’ [23] comparison of the Overcoming Depression on the Internet websites found that participants were more likely to use the program as recommended if they received reminders, and that this increased use was present regardless of the type of reminder (telephone versus postcard) that they received. Alternatively, Christensen et al [24] demonstrated that weekly tracking and reminders reduced attrition in a cognitive behavior therapy e-therapy intervention for depression. Christensen et al [25] also found that shorter cognitive behavior therapy e-therapy interventions were not as effective as longer interventions but that attrition rates were lower, potentially indicating an important trade-off between acceptability and effectiveness. It is thought that the variation in adherence and persistence may be due to the participant’s discretion in using e-therapy [26], rather than therapy that is prescribed, as in a drug trial [13]. Engagement in an e-therapy trial tends to require more physical, cognitive, and time investments by the participant, compared to the relative ease of taking a medication daily. This may explain why e-therapy users are more prone to dropout and nonadherence.

    E-therapy trials have an advantage over traditional trials when measuring persistence and adherence, in that more objective measures of intervention usage are readily available to researchers. Objective measures of persistence include metrics such as the number of times the participant accesses the program, and adherence measures include the time spent online, number of completed activities, and patterns of usage. Subjective measures, such as estimated time spent online, reporting on the completion of behaviorally based homework activities, and use of skills, can also be incorporated. Despite the relative ease of capturing these data in online interventions, few studies report these statistics. Merely reporting on adherence and dropout provides limited insight about the impact of adherence on program outcomes or the translation of program behaviors into daily life. Even fewer studies examine the role of adherence on outcomes. The few that do, by publishing only significant results, may result in publication bias.

    Recent studies have begun to address this through the exploration of the relationship between program exposure and outcomes [27-30]. Although more data can often be collected in e-therapy trials, there is a need to consider what data are collected, how they are collected, and how adherence is defined. It is likely that the influence of design, application, and logistics of e-therapies on outcomes can become informed in the same way medication dosing affects outcomes. For instance, single daily dosing or polypills have been suggested as methods of improving adherence and outcomes in medication treatments.

    To inform this development, we conducted a systematic review of the literature evaluating adherence in e-therapies. Within the context of this review, adherence is defined as the degree to which the user followed the program as it was designed. The two aims were (1) to review the methods used to assess adherence and (2) to evaluate the association between adherence measures and outcome.


    Systematic Selection Criteria and Search Strategy

    A systematic database search was used to identify articles relevant to the aims of the review. Articles published up to and including April 2010, as indexed by Medline, PsycINFO, and the Cochrane Central Register of Controlled Trials databases, were included. Broad keyword search terms were used, favoring sensitivity over specificity, given the relative youth of the field. Search terms used were the keywords “Internet” OR “web” combined with “therapy” OR “self-help” OR “intervention.” Following this initial search strategy, we identified 8300 articles (see Figure 1 for a flow diagram of article selection).

    ‎
    Figure 1. Flow diagram of the process of article selection for systematic review
    View this figure

    The abstracts of the selected articles were reviewed and selected for inclusion by the first author using the following criteria.

    Inclusion Criteria

    An article was suitable for inclusion if it was published in a peer-reviewed journal and the subjects of the intervention were aged 18 years or older. The article also needed to consist of the evaluation of a “self-help,” user-directed online intervention where the Internet was the primary therapeutic delivery modality. In addition, the intervention also required the user to engage with the material on at least 2 occasions in a structured format (therefore, not a website with tools or information, but requiring a progression through the program). When reporting results, the study needed to report 1 or more adherence measures (defined as a measure of program usage by the participant) as well as the outcome of the intervention.

    Exclusion Criteria

    Articles that were not written in the English Language were excluded. Design characteristics that led to exclusion were the participant concurrently receiving a psychotherapeutic intervention in addition to the intervention being studied, or that the program involved synchronous communication as part of the program (eg, online chat, teleconferencing, or personalized therapeutic telephone contact from the therapist during the intervention). These studies were excluded to remove the potentially confounding nature of instantaneous feedback, extrinsic motivation, social desirability, and uncontrolled social contact. Studies with telephone contact were included if the calls were reported to be scripted, described as nontherapeutic in nature, and for the purpose of tracking participants.

    Additionally, further studies were excluded because the individual completing the program was not the individual displaying the target behavior (eg, teachers of children with behavioral issues, or parents of children with enuresis), the program was not delivered over the Internet (eg, the program was delivered by CD-ROM via a computer), or the program was delivered at a specific location or time (eg, at a hospital clinic or at a specific time each week), therefore requiring the participant’s presence online at that time or place.

    Coding of Study Characteristics

    Key article characteristics were recorded using a data extraction template designed for this review. Key data captured for analysis consisted of variables that would allow the articles to be reviewed and factors thought to be important in adherence, based on a review of the literature.

    These factors consisted of study sample size; the study design, including the nature of the control and intervention; the behavior or state that was the target of the intervention; and a description of the intervention, its therapeutic underpinnings, its intervention style, and its requirements from users.

    In addition to the study characteristics, adherence data were captured. These consisted of a measure of the degree to which the individual engaged with the website as determined by program logins, engagement in online activities, time spent onsite, and the number of modules completed. Data were extracted for each adherence measure, including the type of measure and data reported relating to this. Any statistical analyses that examined the role of adherence in outcome were also recorded during data extraction.

    The data collected on adherence variables were aggregated and evaluated for strength of association with outcome. Due to the methodological challenges associated with Internet trials (eg, high attrition rates), it was difficult to use a widely used rating system. Given these challenges, an alternative rating system was developed, informed by the Scottish Intercollegiate Guidelines Network rating system [31]. Based on this system, the strength of adherence measure rating was determined, using a 5-category scale based on the consistency of findings from this review (note that only 4 categories are reported here as no measures met the criteria for the 5th category). Details of this system can be found in the footnotes of the corresponding tables.


    We initially found a total of 8300 studies by using the above combination of search terms. Once duplicates were removed, the articles were further limited to those that were published in an English-language, peer-reviewed journal (n = 1095). The remaining articles were systematically reviewed (by LD) to ensure they met the inclusion criteria. This resulted in a total of 74 articles describing 69 individual studies. The heterogeneity of adherence and outcome measures precluded a formal meta-analysis.

    Sample size varied significantly between the trials, ranging between 20 and 3176 (median 146), with many of the open-access trials having more participants (Table 1). Structured, discrete-period clinical trials tended to have fewer participants. This may be due to the seemingly more intensive researcher input in the form of feedback provided by emails, monitoring of activity completion, and moderation of activity, requiring greater resourcing for smaller numbers of participants. Larger trials tended to evaluate unstructured websites where participants were able to use the website in whatever manner they pleased, rather than using a structured preplanned program.

    Mean study discontinuation rate was approximately 23% of all trial participants (range 0%–83%). Studies of physical and psychological target behaviors had similar attrition rates.

    Table 1. Descriptive statistics for participant randomization sample size in studies that reported adherence included within systematic review (n = 69)
    Descriptive statisticTotal sample size (n)Physical health sample size (n)Psychological health sample size (n)
    Median146190103
    Minimum206220
    Maximum317625233176
    First quartile779156
    Third quartile400958272
    Total number of studies692940
    Total number of participants34,46519,14715,318

    Methods Used to Measure Adherence

    Adherence data that were captured varied across studies. This included reporting the number of times the participant accessed or logged into the program, completed modules or activities, visits made to forums, posts made to the forum, and pages viewed and printed, as well as self-reported completion of activities away from the program or offline. Despite the commonality and functionality of being able to capture or measure participant logins across trials, only half (33/69) of the studies presented these adherence data in relation to outcome measures. See table 2 for a breakdown of ways in which adherence was measured in the included studies.

    Table 2. Methods for measuring adherence to e-therapy as reported by included studies (n = 69)
    Measure of adherenceNumber of times reported
    Logins to program36
    Module completion31
    Time spent online18
    Completion of a predefined activity such or use of an online tool16
    Posts made9
    Pages viewed5
    Replies to emails6
    Forum visits1
    Use of online tools1
    Self-reported completion of offline activities1
    Print requests made1

    Of the 69 studies that reported measuring adherence, approximately half did so by measuring logins and/or completion of modules. Only a quarter of the 69 included studies reported 1 or more of the other potential measures of adherence. The reporting of module completion was more common in studies where the target behavior was psychological health or well-being (25/40, 63%) rather than physical health (6/29, 21%) (n = 69, χ21 = 11.9, P <.001). Conversely, login reporting was more common in studies where the target behavior was related to physical health (23/29, 79%) rather than psychological health (13/40, 33%) (n = 69, χ21 = 14.8, P ≤.001).

    Effect of Adherence on Outcomes

    Of the 69 studies included in the review, 33 (48%) analyzed the impact of 1 or more measure of adherence on outcome variables. Complete results of the review can be found in Multimedia Appendix 1.

    Logins

    A total of 9 studies correlated logins as a measure of adherence to outcomes. Using the previously described rating system, logins were found to be positively associated with outcome in studies targeting food and vegetable consumption [32], physical activity [33-37], and weight management [38-40]. Number of logins was not found to be related to outcomes in studies targeting depression [23,41].

    Activities Completed

    In 6 studies, activities completed were correlated with outcomes. The number of self-reported activities completed (eg, completing a diary, engaging in online tests, or making forum posts) was found to correlate with outcomes in interventions that targeted cigarette smoking [42], weight management [39,40], and body dissatisfaction [43,44]. In studies targeting physical activity [33,34] or depression [45], the number of activities completed was not found to correlate with outcomes.

    Modules Completed

    Completed modules were the most commonly reported measure of adherence, and they correlated with outcomes in 16 studies. The number of modules completed was found to correlate with outcomes in interventions that targeted cigarette smoking [42,46,47], depression [45,48-51], and anxiety disorders [52-57].

    Time Spent Online

    The relationship between time spent online and outcomes was correlated in 4 studies. Time spent online was not correlated to outcomes in studies that targeted depression [58] and anxiety [59], but was correlated to outcomes in studies of infertility related distress [60] and body dissatisfaction [61].

    Pages Opened

    The number of pages opened was explored in 3 studies. The number of pages opened was negatively correlated to outcome in 1 study targeting depression [58] but positively related to outcomes in 2 studies targeting body dissatisfaction [62,63].

    Website Exposure

    The relationship between website exposure and outcomes was explored in 3 studies. Website exposure and program usage was positively correlated with outcomes in interventions targeting smokeless tobacco [64], depression [65], and physical activity [66].

    Table 3 summarizes adherence measures by intervention target, reported by target behavior. (For a more in-depth table, please see Multimedia Appendix 1.)

    Table 3. Summary of the strength of aggregated adherence measure data by target behaviora
    Strength of adherence–outcome association (number of studies)
    Target behaviorLoginsActivities
    completed    		Modules
    completed    		Time spent
    online    		Pages openedWebsite exposure
    Physical health
    Fruit and vegetable consumption+b (n = 1)
    Physical activity+b (n = 3)0c (n = 1)+b (n = 1)
    Weight management++d (n = 2)++ d (n = 3)
    Smoking+b (n = 1)+b (n = 4)
    Smokeless tobacco+b (n = 1)
    Psychological health
    Depression0c (n = 2)0c (n = 1)+b (n = 7)–e (n = 1)–e (n = 1)+b (n = 1)
    Anxiety+d (n = 6)0c (n = 1)
    Body dissatisfaction+b (n = 2)+b (n = 1)+b (n = 2)
    Fertility-related distress+b (n = 1)

    a The number of studies that were aggregated to form the strength rating in the review is indicated in parentheses following the rating indicator (+ = positive; – = negative; 0 = null). (For a complete breakdown of the studies that contributed to the aggregate results see Multimedia Appendix 1.)

    b 1 study or mixed evidence with predominantly positive relationships found between adherence measures and outcome.

    c No relationship found between adherence and outcome measures.

    d At least 2 studies finding a positive correlation between increased adherence and outcome measures.

    e 1 study or mixed evidence with predominantly negative relationships found between adherence measures and outcome.


    Understanding adherence to e-therapies is important in understanding how these therapies may benefit individuals who need intervention. The impact of adherence on outcome appears to vary. The review demonstrates that however it is measured, adherence is associated positively when reported with intervention outcomes targeting physical health. However, in the most commonly targeted outcome of e-therapy, depression, the number of logins, self-reported activities, time online, and pages opened showed no such association. Only 2 measures of adherence—the degree of completion of the modules within the program and a summative “website exposure” outcome—were associated with better depression outcomes.

    The association between the number of logins and outcomes in e-therapies targeting physical health is similar to the positive relationship observed between session attendance and outcome in physical rehabilitation [67-70] and to the dose–response curve seen in medication therapy. This association may largely be due to the number of logins being representative of the participant’s willingness to use the program through their return to the website. The number of logins may be more indicative of program usage than are self-reported activities completed or forums posted. This is particularly the case in programs where there is structured program use (therefore needing completion of an activity or module prior to moving to another) but no time restrictions placed on progression. In this style of program, a participant may complete several modules or activities during a single login. Therefore, the benefits received in completing a module, processing it over a time period, and using its skills before building on this with the next module may be lost.

    For participants involved in trials targeting weight changes as an outcome [38-40], all measures of adherence were correlated with outcomes. Therefore, more adherent participants had a higher level of weight loss. This is consistent with the current literature, which shows that the more closely people follow dietary plans, the better their outcomes [71]. In the case of physical outcomes, one interpretation of the results could be that adherence is a marker of an unmeasured factor such as personality or self-efficacy that predisposes participants to a better outcome. A more extreme interpretations is that the content of the intervention is irrelevant; it is the application of the participant to “something” that produces the outcomes.

    The most common use of e-therapy is to intervene in depression and anxiety. However, measures of logins, self-reported activity completions, and time online were not associated with outcome—only evidence of actual module completion either on its own or in the composite “program exposure” measure appeared to be related to outcome. This suggests a more nuanced interpretation, in that a participant’s interaction with the module content leads to change, rather than improvement merely reflecting a greater propensity to adhere. This reflects the face-to-face literature, in that the structured psychotherapies, with module (task) completion and review, are generally more effective than supportive psychotherapy [72-74]. The lack of association of many adherence measures with outcome might be explained by lack of power in these studies. This is particularly supported by more consistent findings in physical health interventions, which had a median sample size nearly twice that of psychological interventions. However, it is of note that many psychological health studies did have large sample sizes and, for some adherence measures, the results were not null, but negative.

    The definition of adherence and its consequent analysis also varied considerably across studies. Several studies categorized participants into adherent or not [49,54,57,75], or usage categories such as low, medium, and high [60], while others used continuous variables [23,24,33-36,38-41,43-45,50-52,55,56,58,59,61,76] or a combination of these [42,46,53,64-66,77]. Such variation is likely to produce mixed findings. It is also worth noting that several studies reported collecting multiple measures of adherence but reported on the outcome of only a few selected items. In many, it is unclear why some measures of adherence were reported over others, though this may presumably reflect a reporting bias and may therefore inaccurately portray the role of adherence. It is possible that unreported relationships may be nonsignificant, but may also have been useful in understanding the important components of program engagement and may build understanding of the user-engagement factors of e-therapy.

    The variation in measurement of adherence in online interventions makes it difficult to accurately determine the impact of adherence on outcome. Despite having several seemingly objective measures such as number of logins, time spent online, and activities completed, there are still difficulties in determining the “dose” a participant receives. Specifically, time spent online is likely to be influenced by factors such as processing speed, cognitive ability, reading aptitude, and familiarly with using computers, several of which are likely to be influenced by psychological health. Likewise, a person who is unfamiliar with using a computer may write out their activities, rather than engage in the functionality presented in the online environment. This needs to be explored further by studies that randomly assign participants to receive different doses of programs, such as that by Christensen et al [25], which explored exposure to modules against change in outcome measures. Future interventions that intend to measure the impact of adherence on outcomes would benefit from clearly defining their adherence variables and exploring all relationships between the potential adherence variables and outcomes. In addition to this, measures of inactivity need to be included where temporal measures are used. Therefore, when a measure of adherence is the amount of time spent online, there needs to be a time-out function of need to engage readily with the program, to show that it is being used. This will allow the exclusion of individuals who have left their computer on and who are no longer engaging with the program material.

    From this review, the mechanism of how adherence influences Internet treatment outcomes is unclear. Medical studies have shown that adherence is associated with better outcomes, regardless of whether patients received placebo or active interventions [78,79]. Within this review, it is unclear whether adherent patients generally do better, regardless of condition. It has been suggested that the mechanisms underlying a generalized adherence effect may be similar to the mechanisms that underlie the placebo effect [80], whereby the positive outcomes achieved from taking a placebo are the result of internalized beliefs about therapy, such as the expectations that the individual holds or the belief that the therapy will be effective. The placebo effect may also indicate a phenomenon of regression to the mean, in that outlying behavior returns closer to the mean over time. Alternatively, it has been suggested that adherence may be a more general indicator of orientation to healthy behaviors. Therefore, individuals who are adherent may be more likely to follow healthy lifestyle practices and therefore have improved outcomes [81,82]. Further exploration within the area of e-therapy needs to determine whether adherence influences outcomes through expectation beliefs or through participants being generally adherent to treatment recommendations. To do this, control group adherence behaviors need to be reported more.

    Limitations

    We recognize that several key studies that were included in the original sample of 8300 were excluded as a result of selection criteria. Specifically, some trials that potentially would have met the review criteria were excluded before analysis as a result of database coding and indexing. While Cochrane-style systematic reviews have used rigorous data selection and extraction templates to select trials, the varied nature of the design and high attrition rate of e-therapies made it difficult to use such templates. Given this, the limits within the databases were used to ensure that the methodology of the review was as systematic and replicable as possible.

    As the studies contained within this review were heterogeneous in terms of the populations studied, nature of the interventions used, lengths of follow-up, and outcome measures, we summarized the findings by adherence measure only, rather than attempting a meta-analysis. This has limited the ability to combine and report the data beyond the format used. Similarly, the large number of small studies included in the review may have been underpowered for finding significant results.

    Implications for Future Work

    There are many difficulties in determining the role of adherence on outcomes in e-therapies. While objective data can be captured with relative ease, this may not truly reflect the user’s experience and dose. This is particularly pertinent when the user is required to complete activities such as homework exercises between online sessions, or when the user can print off material to review. In both of these scenarios, “objective measures” may underestimate the usage of the program. Similarly, user aspects such as processing speed, and familiarity with Web-based platforms and user interfaces are likely to influence the time spent online. Therefore, understanding the contributing factors of adherence is likely to be as important as understanding adherence per se.

    From the findings of this review, it appears that the number of logins is the measure of adherence best correlated with physical outcomes, while module completion correlated with psychological outcomes. This suggests that program persistence and adherence may be important for physical and psychological interventions in different ways. However, these results need to be considered with caution, given the limitations of this review and the potential biases in the data. We therefore recommend that this be explored in future studies where adherence is the focus. It may mean that an aggregated adherence measure such as program exposure could provide a more meaningful measure of adherence by incorporating logins, time spent online, and number of activities and modules completed. However, it is not always clear how such a composite measure was derived in the studies and how each factor contributes to such a score. Similarly, variations existed within composite measure across each trial, making the comparison difficult. Furthermore, the design of interventions could take this into account by maximizing the likelihood of a participant undertaking the behavior most likely to enhance outcomes. For instance, maximizing module completion for depression and anxiety interventions, even at the expense of number of logins, may be a good trade-off. Understanding the differential effects of different measures of adherence will be important in future content and platform development, as well as in evaluating applicability and health service issues.

    Acknowledgments

    Liesje Donkin receives a scholarship from the University of Sydney to complete her PhD with additional support from the Cardiovascular Disease and Depression Strategic Research Program (Award Reference No. G08S 4048) and beyondblue. Thanks also to Sun Bin who retrieved articles to be included in this review.

    Conflicts of Interest

    None declared

    ‎

    Multimedia Appendix 1

    Association of each type of adherence measure with outcome grouped by target of intervention

    PDF file (Adobe PDF File), 274 KB
    1. Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther 2009 Dec;38(4):196-205. [CrossRef] [Medline]
    2. Amstadter AB, Broman-Fulks J, Zinzow H, Ruggiero KJ, Cercone J. Internet-based interventions for traumatic stress-related mental health problems: a review and suggestion for future research. Clin Psychol Rev 2009 Jul;29(5):410-420. [CrossRef] [Medline]
    3. Barak A, Hen L, Boniel-Nissim M, Shapira NA. A comprehensive review and a meta-analysis of the effectiveness of Internet-based psychotherapeutic interventions. J Technol Hum Serv 2008;26(2-4):109-160. [CrossRef]
    4. Eysenbach G. Issues in evaluating health websites in an Internet-based randomized controlled trial. J Med Internet Res 2002 Dec;4(3):E17 [FREE Full text] [CrossRef] [Medline]
    5. Sabate E. Adherence to Long-Term Therapies: Evidence for Action. Geneva: World Health Organization; 2003.
    6. Cramer JA, Roy A, Burrell A, Fairchild CJ, Fuldeore MJ, Ollendorf DA, et al. Medication compliance and persistence: terminology and definitions. Value Health 2008;11(1):44-47. [CrossRef] [Medline]
    7. Coon DW, Thompson LW. The relationship between homework compliance and treatment outcomes among older adult outpatients with mild-to-moderate depression. Am J Geriatr Psychiatry 2003;11(1):53-61. [Medline]
    8. Neimeyer RA, Kazantzis N, Kassler DM, Baker KD, Fletcher R. Group cognitive behavioural therapy for depression outcomes predicted by willingness to engage in homework, compliance with homework, and cognitive restructuring skill acquisition. Cogn Behav Ther 2008;37(4):199-215. [CrossRef] [Medline]
    9. Thase ME, Callan JA. The role of homework in cognitive behavior therapy of depression. J Psychother Integr 2006;16(2):162-177. [CrossRef]
    10. Tierney JF, Stewart LA. Investigating patient exclusion bias in meta-analysis. Int J Epidemiol 2005 Feb;34(1):79-87 [FREE Full text] [CrossRef] [Medline]
    11. Schulz KF, Grimes DA. Sample size slippages in randomised trials: exclusions and the lost and wayward. Lancet 2002 Mar 2;359(9308):781-785. [CrossRef] [Medline]
    12. Batterham PJ, Neil AL, Bennett K, Griffiths KM, Christensen H. Predictors of adherence among community users of a cognitive behavior therapy website. Patient Prefer Adherence 2008;2:97-105 [FREE Full text] [Medline]
    13. Eysenbach G. The law of attrition. J Med Internet Res 2005;7(1):e11 [FREE Full text] [CrossRef] [Medline]
    14. Glasgow RE, Nelson CC, Kearney KA, Reid R, Ritzwoller DP, Strecher VJ, et al. Reach, engagement, and retention in an Internet-based weight loss program in a multi-site randomized controlled trial. J Med Internet Res 2007;9(2):e11 [FREE Full text] [CrossRef] [Medline]
    15. Wierzbicki M, Pekarik G. A meta-analysis of psychotherapy dropout. Prof Psychol Res Pr 1993;24(2):190-195. [CrossRef]
    16. Strauss JL, Guerra VS, Marx CE, Meade Eggleston A, Calhoun PS. Adherence and psychotherapy. In: Bosworth HB, Oddone EZ, Weinberger M, editors. Improving Patient Treatment Adherence: Concepts, Interventions, and Measurement. New York, NY: Springer Verlag; 2010:215-240.
    17. Bickmore T, Gruber A, Picard R. Establishing the computer-patient working alliance in automated health behavior change interventions. Patient Educ Couns 2005 Oct;59(1):21-30. [CrossRef] [Medline]
    18. Bewick BM, Trusler K, Mulhern B, Barkham M, Hill AJ. The feasibility and effectiveness of a web-based personalised feedback and social norms alcohol intervention in UK university students: a randomised control trial. Addict Behav 2008 Sep;33(9):1192-1198. [CrossRef] [Medline]
    19. Bosak KA, Yates B, Pozehl B. Feasibility of an internet physical activity intervention. West J Nurs Res 2009 Aug;31(5):648-661. [CrossRef] [Medline]
    20. Goessens BM, Visseren FL, de Nooijer J, van den Borne HW, Algra A, Wierdsma J, SMART Study Group. A pilot-study to identify the feasibility of an Internet-based coaching programme for changing the vascular risk profile of high-risk patients. Patient Educ Couns 2008 Oct;73(1):67-72. [CrossRef] [Medline]
    21. Harvey-Berino J, Pintauro SJ, Gold EC. The feasibility of using Internet support for the maintenance of weight loss. Behav Modif 2002 Jan;26(1):103-116. [Medline]
    22. Slootmaker SM, Chinapaw MJ, Schuit AJ, Seidell JC, Van Mechelen W. Feasibility and effectiveness of online physical activity advice based on a personal activity monitor: randomized controlled trial. J Med Internet Res 2009;11(3):e27 [FREE Full text] [CrossRef] [Medline]
    23. Clarke G, Eubanks D, Reid E, Kelleher C, O'Connor E, DeBar LL, et al. Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders. J Med Internet Res 2005;7(2):e16 [FREE Full text] [CrossRef] [Medline]
    24. Christensen H, Griffiths KM, Korten AE, Brittliffe K, Groves C. A comparison of changes in anxiety and depression symptoms of spontaneous users and trial participants of a cognitive behavior therapy website. J Med Internet Res 2004 Dec 22;6(4):e46 [FREE Full text] [CrossRef] [Medline]
    25. Christensen H, Griffiths KM, Mackinnon AJ, Brittliffe K. Online randomized controlled trial of brief and full cognitive behaviour therapy for depression. Psychol Med 2006 Dec;36(12):1737-1746. [CrossRef] [Medline]
    26. Advocat J, Lindsay J. Internet-based trials and the creation of health consumers. Soc Sci Med 2010 Feb;70(3):485-492. [CrossRef] [Medline]
    27. Christensen H, Griffiths KM, Farrer L. Adherence in internet interventions for anxiety and depression. J Med Internet Res 2009;11(2):e13 [FREE Full text] [CrossRef] [Medline]
    28. Kaltenthaler E, Sutcliffe P, Parry G, Beverley C, Rees A, Ferriter M. The acceptability to patients of computerized cognitive behaviour therapy for depression: a systematic review. Psychol Med 2008 Nov;38(11):1521-1530. [CrossRef] [Medline]
    29. Manwaring JL, Bryson SW, Goldschmidt AB, Winzelberg AJ, Luce KH, Cunning D, et al. Do adherence variables predict outcome in an online program for the prevention of eating disorders? J Consult Clin Psychol 2008 Apr;76(2):341-346. [CrossRef] [Medline]
    30. Waller R, Gilbody S. Barriers to the uptake of computerized cognitive behavioural therapy: a systematic review of the quantitative and qualitative evidence. Psychol Med 2009 May;39(5):705-712. [CrossRef] [Medline]
    31. Scottish Intercollegiate Guidelines Network. SIGN 50: A Guideline Developers Handbook. Edinburgh: NHS Quality Improvement Scotland (NHS QIS); 2008.
    32. Alexander GL, McClure JB, Calvi JH, Divine GW, Stopponi MA, Rolnick SJ, MENU Choices Team. A randomized clinical trial evaluating online interventions to improve fruit and vegetable consumption. Am J Public Health 2010 Feb;100(2):319-326. [CrossRef] [Medline]
    33. Carr LJ, Bartee RT, Dorozynski C, Broomfield JF, Smith ML, Smith DT. Internet-delivered behavior change program increases physical activity and improves cardiometabolic disease risk factors in sedentary adults: results of a randomized controlled trial. Prev Med 2008 May;46(5):431-438. [CrossRef] [Medline]
    34. Carr LJ, Bartee RT, Dorozynski CM, Broomfield JF, Smith ML, Smith DT. Eight-month follow-up of physical activity and central adiposity: results from an Internet-delivered randomized control trial intervention. J Phys Act Health 2009 Jul;6(4):444-455. [Medline]
    35. Lewis B, Williams D, Dunsiger S, Sciamanna C, Whiteley J, Napolitano M, et al. User attitudes towards physical activity websites in a randomized controlled trial. Prev Med 2008 Nov;47(5):508-513. [CrossRef] [Medline]
    36. Marcus BH, Lewis BA, Williams DM, Dunsiger S, Jakicic JM, Whiteley JA, et al. A comparison of Internet and print-based physical activity interventions. Arch Intern Med 2007 May 14;167(9):944-949 [FREE Full text] [CrossRef] [Medline]
    37. McKay HG, Danaher BG, Seeley JR, Lichtenstein E, Gau JM. Comparing two web-based smoking cessation programs: randomized controlled trial. J Med Internet Res 2008;10(5):e40 [FREE Full text] [CrossRef] [Medline]
    38. Tate DF, Jackvony EH, Wing RR. Effects of Internet behavioral counseling on weight loss in adults at risk for type 2 diabetes: a randomized trial. JAMA 2003 Apr 9;289(14):1833-1836 [FREE Full text] [CrossRef] [Medline]
    39. Tate DF, Wing RR, Winett RA. Using Internet technology to deliver a behavioral weight loss program. JAMA 2001 Mar 7;285(9):1172-1177 [FREE Full text] [Medline]
    40. Webber KH, Tate DF, Michael Bowling J. A randomized comparison of two motivationally enhanced Internet behavioral weight loss programs. Behav Res Ther 2008 Sep;46(9):1090-1095. [CrossRef] [Medline]
    41. Clarke G, Reid E, Eubanks D, O'Connor E, DeBar LL, Kelleher C, et al. Overcoming depression on the Internet (ODIN): a randomized controlled trial of an Internet depression skills intervention program. J Med Internet Res 2002 Dec;4(3):e14 [FREE Full text] [CrossRef] [Medline]
    42. Muñoz RF, Barrera AZ, Delucchi K, Penilla C, Torres LD, Pérez-Stable EJ. International Spanish/English Internet smoking cessation trial yields 20% abstinence rates at 1 year. Nicotine Tob Res 2009 Sep;11(9):1025-1034. [CrossRef] [Medline]
    43. Zabinski MF, Wilfley DE, Pung MA, Winzelberg AJ, Eldredge K, Taylor CB. An interactive internet-based intervention for women at risk of eating disorders: a pilot study. Int J Eat Disord 2001 Sep;30(2):129-137. [Medline]
    44. Winzelberg AJ, Eppstein D, Eldredge KL, Wilfley D, Dasmahapatra R, Dev P, et al. Effectiveness of an Internet-based program for reducing risk factors for eating disorders. J Consult Clin Psychol 2000 Apr;68(2):346-350. [Medline]
    45. Andersson G, Bergström J, Holländare F, Carlbring P, Kaldo V, Ekselius L. Internet-based self-help for depression: randomised controlled trial. Br J Psychiatry 2005 Nov;187:456-461 [FREE Full text] [CrossRef] [Medline]
    46. Strecher VJ, McClure J, Alexander G, Chakraborty B, Nair V, Konkel J, et al. The role of engagement in a tailored web-based smoking cessation program: randomized controlled trial. J Med Internet Res 2008;10(5):e36 [FREE Full text] [CrossRef] [Medline]
    47. Seidman DF, Westmaas JL, Goldband S, Rabius V, Katkin ES, Pike KJ, et al. Randomized controlled trial of an interactive internet smoking cessation program with long-term follow-up. Ann Behav Med 2010 Feb;39(1):48-60. [CrossRef] [Medline]
    48. Spek V, Nyklícek I, Smits N, Cuijpers P, Riper H, Keyzer J, et al. Internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years old: a randomized controlled clinical trial. Psychol Med 2007 Dec;37(12):1797-1806. [CrossRef] [Medline]
    49. de Graaf LE, Gerhards SA, Arntz A, Riper H, Metsemakers JF, Evers SM, et al. Clinical effectiveness of online computerised cognitive-behavioural therapy without support for depression in primary care: randomised trial. Br J Psychiatry 2009 Jul;195(1):73-80 [FREE Full text] [CrossRef] [Medline]
    50. Meyer B, Berger T, Caspar F, Beevers CG, Andersson G, Weiss M. Effectiveness of a novel integrative online treatment for depression (Deprexis): randomized controlled trial. J Med Internet Res 2009;11(2):e15 [FREE Full text] [CrossRef] [Medline]
    51. Spek V, Cuijpers P, Nyklícek I, Smits N, Riper H, Keyzer J, et al. One-year follow-up results of a randomized controlled clinical trial on internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years. Psychol Med 2008 May;38(5):635-639. [CrossRef] [Medline]
    52. Tillfors M, Carlbring P, Furmark T, Lewenhaupt S, Spak M, Eriksson A, et al. Treating university students with social phobia and public speaking fears: Internet delivered self-help with or without live group exposure sessions. Depress Anxiety 2008;25(8):708-717. [CrossRef] [Medline]
    53. Titov N, Andrews G, Choi I, Schwencke G, Mahoney A. Shyness 3: randomized controlled trial of guided versus unguided Internet-based CBT for social phobia. Aust N Z J Psychiatry 2008 Dec;42(12):1030-1040. [CrossRef] [Medline]
    54. Titov N, Andrews G, Schwencke G. Shyness 2: treating social phobia online: replication and extension. Aust N Z J Psychiatry 2008 Jul;42(7):595-605. [CrossRef] [Medline]
    55. Carlbring P, Nilsson-Ihrfelt E, Waara J, Kollenstam C, Buhrman M, Kaldo V, et al. Treatment of panic disorder: live therapy vs. self-help via the Internet. Behav Res Ther 2005 Oct;43(10):1321-1333. [CrossRef] [Medline]
    56. Furmark T, Carlbring P, Hedman E, Sonnenstein A, Clevberger P, Bohman B, et al. Guided and unguided self-help for social anxiety disorder: randomised controlled trial. Br J Psychiatry 2009 Nov;195(5):440-447 [FREE Full text] [CrossRef] [Medline]
    57. van Straten A, Cuijpers P, Smits N. Effectiveness of a web-based self-help intervention for symptoms of depression, anxiety, and stress: randomized controlled trial. J Med Internet Res 2008;10(1):e7 [FREE Full text] [CrossRef] [Medline]
    58. Clarke G, Kelleher C, Hornbrook M, Debar L, Dickerson J, Gullion C. Randomized effectiveness trial of an Internet, pure self-help, cognitive behavioral intervention for depressive symptoms in young adults. Cogn Behav Ther 2009 Dec;38(4):222-234. [CrossRef] [Medline]
    59. Kenardy J, McCafferty K, Rosa V. Internet-delivered indicated prevention for anxiety disorders: a randomized controlled trial. Behav Cogn Psychother 2003;31(3):279-289. [CrossRef]
    60. Cousineau TM, Green TC, Corsini E, Seibring A, Showstack MT, Applegarth L, et al. Online psychoeducational support for infertile women: a randomized controlled trial. Hum Reprod 2008 Mar;23(3):554-566 [FREE Full text] [CrossRef] [Medline]
    61. Low KG, Charanasomboon S, Lesser J, Reinhalter K, Martin R, Jones H, et al. Effectiveness of a computer-based interactive eating disorders prevention program at long-term follow-up. Eat Disord 2006;14(1):17-30. [CrossRef] [Medline]
    62. Taylor CB, Bryson S, Luce KH, Cunning D, Doyle AC, Abascal LB, et al. Prevention of eating disorders in at-risk college-age women. Arch Gen Psychiatry 2006 Aug;63(8):881-888 [FREE Full text] [CrossRef] [Medline]
    63. Zabinski MF, Wilfley DE, Calfas KJ, Winzelberg AJ, Taylor CB. An interactive psychoeducational intervention for women at risk of developing an eating disorder. J Consult Clin Psychol 2004 Oct;72(5):914-919. [CrossRef] [Medline]
    64. Severson HH, Gordon JS, Danaher BG, Akers L. ChewFree.com: evaluation of a Web-based cessation program for smokeless tobacco users. Nicotine Tob Res 2008 Feb;10(2):381-391. [CrossRef] [Medline]
    65. de Graaf LE, Huibers MJ, Riper H, Gerhards SA, Arntz A. Use and acceptability of unsupported online computerized cognitive behavioral therapy for depression and associations with clinical outcome. J Affect Disord 2009 Aug;116(3):227-231. [CrossRef] [Medline]
    66. McKay HG, King D, Eakin EG, Seeley JR, Glasgow RE. The diabetes network internet-based physical activity intervention: a randomized pilot study. Diabetes Care 2001 Aug;24(8):1328-1334 [FREE Full text] [Medline]
    67. Nathan PA, Keniston RC, Meadows KD. Outcome study of ulnar nerve compression at the elbow treated with simple decompression and an early programme of physical therapy. J Hand Surg Br 1995 Oct;20(5):628-637. [Medline]
    68. Rives K, Gelberman R, Smith B, Carney K. Severe contractures of the proximal interphalangeal joint in Dupuytren's disease: results of a prospective trial of operative correction and dynamic extension splinting. J Hand Surg Am 1992 Nov;17(6):1153-1159. [Medline]
    69. Lyngcoln A, Taylor N, Pizzari T, Baskus K. The relationship between adherence to hand therapy and short-term outcome after distal radius fracture. J Hand Ther 2005;18(1):2-8. [CrossRef] [Medline]
    70. Groth GN, Wilder DM, Young VL. The impact of compliance on the rehabilitation of patients with mallet finger injuries. J Hand Ther 1994;7(1):21-24. [Medline]
    71. Sacks FM, Bray GA, Carey VJ, Smith SR, Ryan DH, Anton SD, et al. Comparison of weight-loss diets with different compositions of fat, protein, and carbohydrates. N Engl J Med 2009 Feb 26;360(9):859-873 [FREE Full text] [CrossRef] [Medline]
    72. Cottraux J, Note I, Albuisson E, Yao SN, Note B, Mollard E, et al. Cognitive behavior therapy versus supportive therapy in social phobia: a randomized controlled trial. Psychother Psychosom 2000;69(3):137-146. [Medline]
    73. Tolin DF. Is cognitive-behavioral therapy more effective than other therapies? A meta-analytic review. Clin Psychol Rev 2010 Aug;30(6):710-720. [CrossRef] [Medline]
    74. Rowa K, Antony MM. Psychological treatments for social phobia. Can J Psychiatry 2005 May;50(6):308-316. [Medline]
    75. Warmerdam L, van Straten A, Twisk J, Riper H, Cuijpers P. Internet-based treatment for adults with depressive symptoms: randomized controlled trial. J Med Internet Res 2008;10(4):e44 [FREE Full text] [CrossRef] [Medline]
    76. Taylor Y, Eliasson A, Andrada T, Kristo D, Howard R. The role of telemedicine in CPAP compliance for patients with obstructive sleep apnea syndrome. Sleep Breath 2006 Sep;10(3):132-138. [CrossRef] [Medline]
    77. Strecher VJ, McClure JB, Alexander GL, Chakraborty B, Nair VN, Konkel JM, et al. Web-based smoking-cessation programs: results of a randomized trial. Am J Prev Med 2008 May;34(5):373-381. [CrossRef] [Medline]
    78. Avins AL, Pressman A, Ackerson L, Rudd P, Neuhaus J, Vittinghoff E. Placebo adherence and its association with morbidity and mortality in the studies of left ventricular dysfunction. J Gen Intern Med 2010 Dec;25(12):1275-1281. [CrossRef] [Medline]
    79. Granger AL, Fehnel SE, Hogue SL, Bennett L, Edin HM. An assessment of patient preference and adherence to treatment with Wellbutrin SR: a web-based survey. J Affect Disord 2006 Feb;90(2-3):217-221. [CrossRef] [Medline]
    80. Czajkowski SM, Chesney MA, Smith AW. Adherence and the placebo effect. In: Shumaker SA, Ockene JK, Riekert KA, editors. The Handbook of Health Behavior Change. New York, NY: Springer; 2009:713-734.
    81. Coronary Drug Project Research Group. Influence of adherence to treatment and response of cholesterol on mortality in the coronary drug project. N Engl J Med 1980 Oct 30;303(18):1038-1041. [CrossRef] [Medline]
    82. Horwitz RI, Viscoli CM, Berkman L, Donaldson RM, Horwitz SM, Murray CJ, et al. Treatment adherence and risk of death after a myocardial infarction. Lancet 1990 Sep 1;336(8714):542-545. [Medline]

    Edited by G Eysenbach; submitted 14.02.11; peer-reviewed by B Cugelman, W Brouwer; comments to author 10.03.11; revised version received 24.05.11; accepted 05.05.11; published 05.08.11

    Copyright

    ©Liesje Donkin, Helen Christensen, Sharon L Naismith, Bruce Neal, Ian B Hickie, Nick Glozier. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 05.08.2011.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.