This paper is in the following e-collection/theme issue:

Digital Health Reviews (1289) Adoption and Change Management of eHealth Systems (712) Theoretical Frameworks and Concepts (252) Engagement with and Adherence to Digital Health Interventions, Law of Attrition (491)
Clarifying the Concept of Adherence to eHealth Technology: Systematic Review on When Usage Becomes Adherence

Clarifying the Concept of Adherence to eHealth Technology: Systematic Review on When Usage Becomes Adherence

Clarifying the Concept of Adherence to eHealth Technology: Systematic Review on When Usage Becomes Adherence

Authors of this article:

Floor Sieverink1 Author Orcid Image ;   Saskia M Kelders1, 2 Author Orcid Image ;   Julia EWC van Gemert-Pijnen1 Author Orcid Image
Article Authors Cited by (216) Tweetations (50) Metrics

    Review

    1Centre for eHealth and Wellbeing Research, Department of Psychology, Health and Technology, University of Twente, Enschede, Netherlands

    2Optentia Research Focus Area, North-West University, Vanderbijlpark, South Africa

    Corresponding Author:

    Floor Sieverink, MSc, MA

    Centre for eHealth and Wellbeing Research

    Department of Psychology, Health and Technology

    University of Twente

    P.O. Box 217

    Enschede, 7500AE

    Netherlands

    Phone: 31 534896631

    Email: f.sieverink@utwente.nl


    Background: In electronic health (eHealth) evaluations, there is increasing attention for studying the actual usage of a technology in relation to the outcomes found, often by studying the adherence to the technology. On the basis of the definition of adherence, we suggest that the following three elements are necessary to determine adherence to eHealth technology: (1) the ability to measure the usage behavior of individuals; (2) an operationalization of intended use; and (3) an empirical, theoretical, or rational justification of the intended use. However, to date, little is known on how to operationalize the intended usage of and the adherence to different types of eHealth technology.

    Objective: The study aimed to improve eHealth evaluations by gaining insight into when, how, and by whom the concept of adherence has been used in previous eHealth evaluations and finding a concise way to operationalize adherence to and intended use of different eHealth technologies.

    Methods: A systematic review of eHealth evaluations was conducted to gain insight into how the use of the technology was measured, how adherence to different types of technologies was operationalized, and if and how the intended use of the technology was justified. Differences in variables between the use of the technology and the operationalization of adherence were calculated using a chi-square test of independence.

    Results: In total, 62 studies were included in this review. In 34 studies, adherence was operationalized as “the more use, the better,” whereas 28 studies described a threshold for intended use of the technology as well. Out of these 28, only 6 reported a justification for the intended use. The proportion of evaluations of mental health technologies reporting a justified operationalization of intended use is lagging behind compared with evaluations of lifestyle and chronic care technologies. The results indicated that a justification of intended use does not require extra measurements to determine adherence to the technology.

    Conclusions: The results of this review showed that to date, justifications for intended use are often missing in evaluations of adherence. Evidently, it is not always possible to estimate the intended use of a technology. However, such measures do not meet the definition of adherence and should therefore be referred to as the actual usage of the technology. Therefore, it can be concluded that adherence to eHealth technology is an underdeveloped and often improperly used concept in the existing body of literature. When defining the intended use of a technology and selecting valid measures for adherence, the goal or the assumed working mechanisms should be leading. Adherence can then be standardized, which will improve the comparison of adherence rates to different technologies with the same goal and will provide insight into how adherence to different elements contributed to the outcomes.

    J Med Internet Res 2017;19(12):e402

    doi:10.2196/jmir.8578

    Keywords

    adherence; eHealth; systematic review 


    Adherence and Attrition

    One of the main goals of electronic health (eHealth) evaluations is to gain insight into the effects of technology on outcomes such as quality of life, health-related outcomes (eg, glycemic control, weight loss), or psychological outcomes (eg, depressive complaints, anxiety). However, many eHealth evaluations report no or limited positive effects [1-5]. There is strong evidence that this is often related to participants not using technologies in the desired way. For every technology, a proportion of the users will not use the intervention at all, will stop using the technology after a period, or will not use the available elements of the technology as intended [1,6-8].

    To gain more insight into this phenomenon, Eysenbach made a plea back in 2005 for reporting the levels of nonusage attrition, or the extent to which individuals stop using the technology [9]. On the other hand, understanding adherence, or how actual usage of the technology may have influenced the outcomes, might be just as important [6]. The term adherence is rooted in the pharmaceutical industry, and according to the World Health Organization’s (WHO) definition, it refers to “the extent to which a person’s behaviour – taking medication, following a diet, and/or executing lifestyle changes, corresponds with agreed recommendations from a health care provider ” [10].

    Adherence to eHealth Technologies

    For eHealth technologies, several definitions for adherence can be identified in the existing literature. For example, Christensen et al defined adherence as “the degree to which individuals experience the content of the Internet intervention” [11]. However, the concept of “following the prescribed recommendations” (as implied by the WHO’s definition) is missing from this definition. Therefore, Donkin et al referred to adherence as “the degree to which the user followed the program as it was designed” [6]. In accordance with the WHO definition of adherence, this definition contains the concept of intended use, or “the extent to which individuals should experience the content to derive maximum benefit from the intervention, as defined or implied by its creators” [1]. According to these definitions, the intended use is thus the minimum use to establish adherence.

    Although adherence is related to other measures such as engagement or nonusage attrition, these terms do not refer to the same or inverse concepts. After all, not using the technology as defined or implied by its creators does not necessarily mean that a participant is not using the technology at all (as implied by the definition of nonusage attrition) [9]. Moreover, definitions of engagement usually incorporate the more subjective attributes of challenge, positive affect, endurability, and aesthetic and sensory appeals [12], whereas adherence is mostly based on measures for usage behavior.

    Determining Adherence

    There is now increasing attention for studying the adherence rates and reasons for nonadherence in eHealth evaluations. However, it still can be a challenge to operationalize the intended use for individual eHealth technologies in a certain context. In the pharmaceutical industry, the intended use (ie, agreed recommendations) is mostly based on the observed or rationalized working mechanisms and the dose-response curves of the medication for a certain condition. As a result, the dosage of one particular medication can vary depending on (the severity of) the condition and the patient’s characteristics (eg, age, gender, or weight).

    This is in contrast with many prior eHealth studies, which often assume that all users should experience all of the elements of a technology to obtain effects, and in which adherence is thus often operationalized as using everything the technology offers. However, a technology can be designed for multiple target groups and, depending on the individual user goals and the desired outcomes, technology can be used in many different ways in terms of the features that are used, as well as the frequency, time, and place of use [13,14]. Furthermore, the amount of use that is needed to obtain the desired outcomes may vary a lot across different user groups [6]. This implies that users do not always have to experience all of the available elements of a technology or have to use the same elements because usage goals may differ across users as well. Moreover, individuals may also stop using the technology because they have reached their personal goals (early completers or e-attainers) [11,15], and nonusage dropout is thus not always a consequence of losing interest (as stated by Eysenbach) [9].

    To summarize, based on the definition of adherence, we suggest that the following three elements are necessary to determine adherence to eHealth technology: (1) the ability to measure the usage behavior of individuals; (2) an operationalization of intended use; and (3) an empirical, theoretical, or rational justification of the intended use. However, to date, little is known about how to operationalize the intended usage of and thus the adherence to different types of eHealth technology. Many systematic reviews gaining insight into adherence to eHealth technology focus on the extent to which individuals use different types of technology and what the reasons for nonadherence are, without a proper operationalization of intended use and adherence [1,6,11,16,17]. These reviews therefore fail to provide insight into how adherence and intended use have been operationalized.

    The Goal of the Review

    The goal of this systematic review was to improve evaluations of eHealth technologies by gaining insight into when, by whom, and how the concept of adherence has been used in previous eHealth evaluations, and finding a concise way to operationalize the adherence to and the intended use of different eHealth technologies. We do this by providing insight into how the usage of the technology was measured across previous studies; how adherence to different types of technologies (eg, structured interventions, patients platforms) was operationalized; and if and how the intended use of eHealth technologies has been justified with theory, evidence, or rationale.


    Search Strategy

    A literature search was conducted using the Scopus, Web of Science, ScienceDirect, and PsycINFO databases. A combination of the constructs “technology,” “intervention,” “adherence,” and “health” was used. To ensure sufficient coverage of each construct, we used different keywords for every construct (see Multimedia Appendix 1). We excluded other usage-related concepts (eg, nonusage attrition or engagement) because these do not refer to the same concept.

    Eligibility Criteria

    All articles that met the following criteria were included in the review: (1) it involved health-related technology (Web-based technologies, apps, wearables, or technologies provided via other devices); (2) the technology was intended to be used more than once by the patient or client; (3) the article described a primary study or a protocol for a primary study that included objective, quantifiable measurements, and an operationalization of adherence to the technology; (4) the study was published in English; and (5) the study was peer-reviewed and published.

    Articles were excluded in the following situations: (1) adherence was defined as adhering to offline treatment or as a measure for following a study protocol, (2) the technology studied was only used as a tool for exchanging information without the possibility for further interaction with the system (eg, telemonitoring only, sending or receiving messages like SMS [short message service] interventions, or in chat rooms), and (3) the article was a conference abstract or full text was not available.

    Study Selection

    The selection of studies was completed in three steps. First, all titles were screened by two authors (FS and SK) to exclude the records that clearly indicated a study outside the scope of this review (eg, medication adherence). Second, the abstracts of the articles initially deemed relevant were screened for eligibility by the same authors. During this process of title and abstract screening, studies were included in the next step if they were deemed eligible by at least one of the reviewers.

    Third, the full texts of all remaining publications were checked for inclusion by FS, and the final selection was discussed by FS, SK, and LvG. Disagreements regarding the inclusion of full texts were discussed until consensus was reached.

    Data Collection and Analysis

    The required information for all included technologies and studies was coded by FS using a data extraction form. The information that was extracted from each article is presented in Textbox 1.

    On the basis of the extracted information, the operationalizations for adherence in every study were categorized. An overview of these categories is provided in Table 1.

    All the data on each study were entered in SPSS version 24.0 (IBM Corporation, Somers, NY, USA). Each was treated as a separate case. The results are categorized based on the use of the technology (structured, hybrid, and unstructured) and the categorization of adherence operationalizations (Category A, B, and C). Descriptive data for the different categories were calculated using SPSS. Differences in variables between the use of the technology and the operationalization of adherence were calculated using a chi-square test of independence. When the observed counts were below the expected counts, a Monte Carlo correction was applied. We used an alpha level of .05 for all statistical tests.

    Information extracted from the included articles.
    1. General information regarding the authors, affiliation, country, year, and journal of publication.
    2. The name of the technology: when no name was reported, the name was indicated as “N/A.”
    3. The type of technology or the device: for example, Web-based, mobile phone apps, wearable, or other devices for monitoring.
    4. Type of use (structured, hybrid, or unstructured): “Structured use” was assigned to technologies consisting entirely of separate modules or lessons that users had to complete before moving on to the next [6]. “Free use” was assigned to technologies that consisted of different elements that users could then use at their own convenience (eg, a personal health record containing a diary, educational material, and a messaging function; or a wearable connected to a mobile phone app to gain insight into something like activity levels). “Hybrid use” was assigned to technologies with a fixed core, supplemented with other components for free use.
    5. The health care field targeted with the technology, distinguishing between mental health (eg, targeting depressive symptoms or anxiety), chronic conditions (eg, self-management support for patients with type 1 diabetes mellitus), or lifestyle technologies (eg, losing weight, improving physical activity, or quitting smoking). These categories were assigned depending on the technology’s goal, meaning that an intervention to support patients with chronic conditions maintaining a healthy lifestyle is seen as a lifestyle technology.
    6. The variables that were used to assess adherence, such as the number of logins, the number of different days that users used the technology, the time spent on the technology, the number of modules or lessons started or completed, and the number of different elements that were accessed or used.
    7. The intended use of the technology.
    8. Whether the described intended use was justified, for example, using theory, evidence, or rationale.
    Textbox 1. Information extracted from the included articles.
    Table 1. Categorization of adherence operationalizations.
    CategoryExplanation
    Category AAssigned when adherence was operationalized in terms of “the more usage, the better.” Category A operationalizations do not include an operationalization of intended use, and therefore do not comply with the definition of adherence.
    Category BAssigned when the intended use of a technology was provided without justification (eg, “a user is adherent when logging in at least once a week for three subsequent weeks”).
    Category CAssigned when the intended use of the technology was provided and justified using theory, evidence, or rationale (eg, “we know from previous research that users benefit the most from the technology when finishing module 4, so a user is adherent once module 4 is completed”).

    Study Selection

    A total of 7005 studies were identified via the search. After screening of the titles, abstracts, and full texts, 62 full texts were included in this review. An overview of these articles is presented in Multimedia Appendix 2.

    In total, 36 articles were excluded during the full-text screening phase (Figure 1). Most full texts (n=18) were excluded because they did not include objective, quantifiable measurements, and an operationalization of adherence to the technology (12 primary studies and 6 viewpoint papers). Other reasons for exclusion are presented in Figure 1.

    All included articles were published in 2006 or later, and more articles published in recent years were included overall (Figure 2). The first authors are mostly affiliated in the United States of America (n=15), Australia (n=10), and the Netherlands (n=8) (Table 2). In total, 24 of the studies were published in the Journal of Medical Internet Research or its sister journals.

    Technology Characteristics

    Table 3 provides an overview of the technologies that are subjects of the included studies. The technologies described in most of the articles are Web-based (51/62). Furthermore, five are smartphone apps and five are Web-based or smartphone technologies combined with other devices such as wearables. Almost half of the technologies (29/62) were structured technologies, 18 were unstructured technologies, and 15 had a hybrid nature.

    Half of all included articles reported adherence to mental health technologies. Most of these technologies targeted depression (n=8) and anxiety disorders (n=5), some of the latter also in combination with depression (n=3). Other mental health technologies targeted postdisaster mental health distress (n=3); cancer-related distress (n=3); general stress management (n=2); eating pathology (n=2); or insomnia, erectile dysfunction, bipolar disorders, mindfulness, and cognitive training (all n=1). Eighteen of these technologies were based on cognitive behavioral therapy. Most of the structured technologies (17/29) and almost all hybrid technologies (12/15) were aimed at improving mental health (P<.001).

    A total of 25 technologies were aimed at supporting a healthy lifestyle, more specifically smoking cessation (n=7); improving physical activity (n=7); weight loss (n=5); alcohol cessation (n=3); general health promotion (n=2); or healthy eating (n=1). Six technologies were aimed at self-management support for patients with chronic diseases (diabetes [n=3], inflammatory bowel disease [n=1], hypertension [n=1], or surgical site infections [n=1]). Most of the unstructured technologies were aimed at lifestyle support (13/18).

    For all technologies, adherence was mostly operationalized using measures regarding the number of modules or lessons completed and the number of different days, weeks, or months that people used the technology. Adherence to unstructured technologies was mostly operationalized using the number of logins or sessions (P=.03), the number of features accessed or used (P<.001), and the time spent using the technology (P<.001). Adherence to structured technologies was most often operationalized using the number of modules or lessons completed (P<.001).

    Operationalization of the Adherence Definition

    Out of the 62 included articles, 34 reported adherence only in terms of how often the technology was used (Category A operationalization) [7,18-50]. In 23 studies, the intended usage was described as well (Category B operationalization) [51-73], and 5 studies reported the intended usage with a justification for this threshold (Category C operationalization) [74-78]. The number of publications reporting Category C operationalizations has increased since 2015 (Figure 2).

    Table 4 provides an overview of the characteristics of the studies for adherence category. Although no significant differences were found, we were still able to identify some interesting patterns. Overall, the data show that 20 out of 31 technologies for mental health contain a Category A operationalization, whereas Category C operationalizations are more equally distributed over the three health care fields. Furthermore, 48 out of 56 Category A and Category B operationalizations are for Web-based technologies (whether or not in combination with other devices), whereas a third of the Category C operationalizations are also for smartphone technologies.

    Most Category A operationalizations contain a measure for the number of modules that the users accessed or completed (19/34) and the time spent on the technology and number of features accessed and used (both 11/34). Most Category B operationalizations contain the number of days, weeks, or months that people used the technology (11/23), the number of accessed or completed modules (8/23), and the number of logins or sessions (4/23).

    Category C definitions are mostly based on the number of accessed or completed modules (3/5 operationalizations). The number of logins and the number of days, weeks, or months that people used the technology were used in 2 out of 5 operationalizations.

    Most operationalizations of adherence are based on a maximum of two different measures, regardless of the category (49/62). Ten out of 13 operationalizations that feature 3 or more measures are categorized as Category A.

    The included Category C operationalizations provided justification in various ways. Reinwand et al asked all participants to complete a questionnaire to assess to what extent their lifestyle met the Dutch guidelines for healthy eating, drinking alcohol, physical activity, etc [74]. Recommendations for the use of corresponding elements of the technology were then made based on the outcomes of the assessment and, in turn, adherence was defined as using the technology in accordance with these recommendations. In the study by Zeng et al, the technology consisted of different elements that were all evaluated as effective in other studies [75]. Users were considered adherent if they used all elements. Beatty et al considered a user to be highly adherent when a therapeutic dose of 66% of the intervention was received [76]. This threshold for a therapeutic dose was obtained from previous studies. In the study by Mertens et al, technology use was represented in relation to medication use, and users were therefore seen as adherent when the technology was used in accordance with the recommendations for medication use [77]. Carolan et al describe a protocol for a study to be conducted to understand the optimum adherence to the technology in relation to the outcomes [78].

    ‎
    Figure 1. Flowchart of full text selection.
    View this figure
    ‎
    Figure 2. The number of studies reporting Category A, B, and C operationalizations per publication year.
    View this figure
    Table 2. Country of affiliation of the first authors of all included articles.
    CountryNumber of included articles
    United States of America15
    Australia10
    The Netherlands8
    Sweden5
    United Kingdom5
    Canada3
    Germany3
    Switzerland3
    China2
    Norway2
    Austria1
    Denmark1
    Finland1
    Ireland1
    Portugal1
    Spain1
    Table 3. Characteristics of structured, unstructured, and hybrid technologies and their operationalizations of adherence.
    CharacteristicsStructured (n=29)Unstructured (n=18)Hybrid (n=15)

    		n (%)n (%)n (%)
    Health care fielda



    		Mental health (n=31)17 (59)2 (11)12 (80)

    		Lifestyle (n=25)9 (31)13 (72)3 (20)

    		Chronic care (n=6)3 (10)3 (17)0 (0)
    Device



    		Web-based (n=51)25 (86)13 (72)13 (87)

    		Smartphone app (n=6)3 (10)2 (11)1 (7)

    		Web-based or smartphone with wearable (n=2)0 (0)1 (6)1 (7)

    		Web-based or smartphone with wearable and monitoring device (n=3)1 (3)2 (11)0 (0)
    Level of adherence definition



    		Category A (n=34)16 (55)10 (56)8 (53)

    		Category B (n=23)11 (38)7 (39)5 (33)

    		Category C (n=5)2 (7)1 (6)2 (13)
    Number of measuresa



    		1 (n=32)19 (66)4 (22)9 (60)

    		2 (n=17)8 (28)7 (39)2 (13)

    		3 (n=7)1 (3)3 (17)3 (20)

    		4 or more (n=6)1 (3)4 (22)1 (7)
    Measures of adherence



    		Number of logins/number of sessions (n=14)a3 (10)8 (44)3 (20)

    		Number of modules/number of lessons completed (n=30)a17 (59)2 (11)11 (73)

    		Number of features accessed/used (n=16)a5 (17)9 (50)2 (13)

    		Number of exercises completed (n=9)6 (21)2 (11)1 (7)

    		Number of pages viewed (n=11)2 (7)6 (33)3 (20)

    		Number of days/weeks/months (n=19)a8 (28)7 (39)4 (27)

    		Time spent (n=14)a2 (7)9 (50)3 (20)

    aP<.05.

    Table 4. Characteristics of Category A, B, and C definitions.
    CharacteristicsCategory A (n=34)Category B (n=23)Category C (n=5)

    		n (%)n (%)n (%)
    Health care field



    		Mental health (n=31)20 (59)9 (39)2 (40)

    		Lifestyle (n=25)13 (38)10 (44)2 (40)

    		Chronic care (n=6)1 (3)4 (17)1 (40)
    Device



    		Web-based (n=51)31 (91)17 (74)3 (60)

    		Smartphone app (n=6)1 (3)3 (13)2 (40)

    		Web-based or smartphone with wearable (n=2)1 (3)1 (4)0 (0)

    		Web-based or smartphone with wearable and monitoring device (n=3)1 (3)2 (9)0 (0)
    Type of use



    		Structured (n=29)16 (47)11 (48)2 (40)

    		Unstructured (n=18)10 (29)7 (30)1 (20)

    		Hybrid (n=15)8 (24)5 (22)2 (40)
    Measures of adherence



    		Number of logins/number of sessions (n=14)8 (24)4 (17)2 (40)

    		Number of modules/number of lessons completed (n=30)19 (56)8 (35)3 (60)

    		Number of features accessed/used (n=16)11 (32)4 (17)1 (20)

    		Number of exercises completed (n=9)6 (18)3 (13)0 (0)

    		Number of pages viewed (n=11)9 (27)1 (4)1 (20)

    		Number of days/weeks/months (n=19)a6 (18)11 (48)2 (40)

    		Time spent (n=14)11 (32)2 (9)1 (17)
    Number of measures



    		1 (n=32)13 (38)16 (70)3 (60)

    		2 (n=17)11 (32)5 (22)1 (20)

    		3 (n=7)6 (18)1 (4)0 (0)

    		4 or more (n=6)4 (12)1 (4)1 (20)

    aP<.05.


    Aim of this Review

    In this systematic review, we have sought to gain insight into how the concept of adherence has been used in previous eHealth evaluations. In line with the definitions for adherence and intended use maintained by the WHO [10] and Kelders [1], we reviewed not only how usage was measured but also if and how intended use of eHealth technologies was operationalized and justified using theory, evidence, or rationale.

    Principal Findings

    We included 62 studies in this review, all published after 2005. The majority of the technologies described in these studies were structured or hybrid Web-based interventions targeting mental health (mostly cognitive behavioral therapy [CBT] interventions) or unstructured technologies for lifestyle support.

    We observed a growing number of studies that studied adherence to eHealth technologies since Eysenbach’s plea for reporting the levels of nonusage attrition with eHealth technology in 2005 [9]. Although the “prescribed recommendations” or the intended use of a technology form an important element of the definition of adherence [10], and although there is evidence that users do not always have to complete an intervention to experience effects [6,11,13-15], half of all operationalizations are based on the assumption of “the more use, the better” and do not include a threshold for intended use. Sometimes, we do not know (yet) what the intended use of a technology is, or defining the intended use is not necessary to answer the research question(s) of the study. Then, a Category A operationalization suffices for answering the research questions. However, a Category A operationalization only refers to the actual usage of a technology without comparing it with its intended usage. According to the definitions used, they should therefore not be referred to as adherence.

    When the intended use for the technology was reported, only a minority of all included studies featured justified Category C operationalizations, making the comparison of adherence across different eHealth technologies more complicated. However, we were still able to observe a small increase in Category C operationalizations since 2015.

    Remarkably, the proportion of evaluations of mental health technologies reporting a justified (Category C) operationalization of intended use is lagging behind compared with evaluations of lifestyle and chronic care technologies. This is unexpected because the majority of mental health technologies is based on (principles of) CBT, which is the most studied treatment for depression and has proven effective in many studies [79]. However, a meta-analysis of Van Ballegooijen et al revealed that participants complete approximately 84% of their CBT program in both offline and online treatment. Although a longer treatment duration is associated with better effects [80], this still implies that users do not necessarily need to finish the complete program to experience reduction of their complaints, and that there should be a threshold for intended use. As such, it seems that this knowledge could be used to define and substantiate the intended use of a CBT technology, but in the studies included in this review, this notion has not been put into practice yet.

    We were also able to observe some interesting patterns in the composition of measures for adherence and intended use. When the operationalization of adherence consists of a combination of four or more measures, it is most likely a Category A operationalization, whereas most Category C operationalizations consist of one or two measures. This implies that for many Category A operationalizations, a scattershot approach was used when it comes to measuring adherence. In contrast, the results indicated that justifications of intended use are often based on the goal of the technology and/or the assumed working mechanisms, leading to more focused operationalizations that do not require additional measurements to evaluate adherence to the technology. In other words, more measures are not necessarily the key to knowledge if they are not sufficiently specific.

    No significant differences could be found between the kinds of measures that are used for all three levels of operationalizations. Category A operationalizations most often contain the number of modules that a user started or completed, the number of features accessed or used, and the time spent online. This seems obvious, as this level of operationalization is mostly used for structured or hybrid mental health interventions consisting of different modules that users have to follow. Category C or justified operationalizations are more often defined by the number of days, weeks, or months that the technology is used by people. This can be explained by way of the finding that Category C operationalizations are used for a large proportion of unstructured and hybrid technologies. As people are able to use the features of these technologies more or less at their own convenience, the development of use over time would probably provide more information regarding adherence than use of the technology’s content at fixed points in time only.

    Implications and Recommendations

    An important reason for the lack of justifications for the intended use of eHealth technologies might be that there is a lack of knowledge regarding the working mechanisms of technology-based applications [8]. However, the included Category C operationalizations did show that knowledge of the working mechanisms of the technology is not a prerequisite for defining the intended use. After all, the intended use or the “therapeutic dose” can be justified just as well using existing guidelines for healthy living and medication use [74,77] or using previous research regarding other technologies [75,76]. Moreover, the intended use has also been operationalized by linking the (positive) outcomes of individual users to their usage patterns to find the most effective patterns [78].

    The fact that we did not find a justification of intended use based on existing models for behavior change was unexpected. For instance, Kaushal and Rhodes found that exercising for at least four times per week for 6 weeks is a minimal requirement for establishing exercise habits [81]. These kind of findings can also be used for determining intended use; for example, a user of a technology to improve physical activity is adherent when using (specific elements of) the technology at least four times a week and 6 weeks in a row.

    Another example comes from Kelders et al who found that a group of users dropped out from an intervention for reducing depressive complaints after a lesson that focused on applying newly acquired skills in practice, as doing so can be confrontational [56]. However, following this lesson can also be seen as an important precondition for gaining effects from using the intervention. The intended use of this intervention could thus be operationalized as following the intervention until that critical lesson is completed at the very least. An important aspect of operationalizing the intended use of a technology is therefore to keep the goal of the technology and the desired outcomes in mind. What use is necessary at minimum to reach that goal (eg, to experience certain effects or establish new skills and habits), and how can we translate this into measures for adherence?

    Although it has previously been suggested that a combination of a range of different variables for technology usage provides a more meaningful measure of adherence [6], the results of this review show that a limited but deliberate set of only one or two different measures in accordance with the goal of the technology can also be used for operationalizing intended use. At the moment, eHealth evaluations often fail to demonstrate the dose-response relationship (the usage that is minimally needed to experience certain effects) or simply define it as “the more use, the better.” However, the results of this review indicate that Category A and B operationalizations of adherence often do not take the characteristics of the technology (eg, goal, persuasiveness, and user-friendliness) into account. It is thus very possible that dose-response relationships might become more apparent when the measures used to operationalize adherence match the goal of the technology [82,83].

    All of the measures for adherence in this review are based on data regarding technology usage. However, the results of a recent literature review of theoretical perspectives on adherence showed that adherence is a multidimensional concept, influenced by a range of technological, environmental, and individual factors altogether that cannot be evaluated by technology usage alone [8,84]. Therefore, additional measures are needed to determine whether and why users are or have been adherent to technology. For example, a mixed-methods approach that combines usage data with questionnaires, health measurements, and/or interviews could provide important knowledge regarding why people do or do not use the technology, how people learn from using the technology, the minimal use that is needed for users to experience certain effects or to reach certain goals, and how the skills acquired while using the technology are then applied in daily life. These outcomes could then in turn be used for determining the intended use of the technology and translating that concept into concrete measures for evaluating adherence.

    In their review, Donkin et al state that it is difficult to compare adherence with different technologies when the measures that are used across the different trials vary [6]. However, this statement is based on Category A operationalizations of adherence where more use is better. When using Category B or (preferably) Category C operationalizations, the actual usage of each individual can be compared with the technology’s intended usage. In turn, the percentage of people who adhered to the intervention can be calculated, making adherence a more objective and standardized concept [1]. This approach simplifies the comparison of adherence across technologies with the same goal (eg, improving physical activity) but different technology characteristics (eg, features of the technology, persuasiveness, and user-friendliness) and a different operationalization of intended use. At the same time, this approach also simplifies the comparison of adherence of different users of a specific technology. Ideally, when individuals have different goals for using a technology, they should also have an individual intended usage, which could be used to get more fine-grained, personalized measures of adherence. This will be of added value for both developers and researchers, as this approach will provide better insight into how adherence to the various elements of different technologies contributed to the outcomes that are found, and for whom.

    Limitations

    As the goal of this review was to gain insight into how the concept of adherence has been used in previous eHealth evaluations, we only included studies that used adherence as an outcome measure or studies that explicitly stated how other outcome measures are used as a proxy for adherence. As such, we may have missed relevant studies that formulated the intended use of a technology, but used other related terms for adherence (eg, nonusage attrition, engagement, drop-out, or [non]usage). In future research, added value might be obtained by reviewing these studies to find directions on how to operationalize the intended use for different eHealth technologies.

    Furthermore, we defined the categories of adherence operationalizations for every study, instead of every type of technology. After all, adherence and intended use can be defined in many different ways, and it is very possible for different operationalizations to be used for different studies regarding the same technology. Even so, we feel that we have included a large body of studies in this review, providing valuable insight into the concept of adherence and intended use.

    Conclusions

    Previous research has shown that users do not always have to experience all of the elements of a technology and that effective usage patterns might differ across users. However, the results of this review show that the operationalization of intended use is mostly based on the assumption of “the more use, the better” and that when a threshold of intended use is provided, justification is often missing. Therefore, it can be concluded that adherence to eHealth technology is an underdeveloped and often improperly used concept in the existing body of literature.

    When the intended use of a technology was defined, the goal of the technology and/or the assumed working mechanisms often formed the starting point for selecting valid measures (eg, number of logins, number of completed modules). A justified threshold for intended use in accordance with the goal of the technology provides information for a concise evaluation of adherence and the working mechanisms of a technology. Subsequently, justified operationalization (comprising multidimensional measures or not) can be used to standardize adherence to different eHealth technologies, making it easier to compare the adherence rates of different technologies.

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    Keywords literature search.

    PDF File (Adobe PDF File), 323KB
    ‎

    Multimedia Appendix 2

    Included studies and technologies, levels of adherence operationalization, and measures.

    PDF File (Adobe PDF File), 75KB
    1. Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JE. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res 2012;14(6):e152 [FREE Full text] [CrossRef] [Medline]
    2. Elbert NJ, Van Os-Medendorp H, Van Renselaar W, Ekeland AG, Hakkaart-van Roijen L, Raat H, et al. Effectiveness and cost-effectiveness of ehealth interventions in somatic diseases: a systematic review of systematic reviews and meta-analyses. J Med Internet Res 2014;16(4):e110 [FREE Full text] [CrossRef] [Medline]
    3. Beishuizen CR, Stephan BC, Van Gool WA, Brayne C, Peters RJ, Andrieu S, et al. Web-based interventions targeting cardiovascular risk factors in middle-aged and older people: a systematic review and meta-analysis. J Med Internet Res 2016 Mar 11;18(3):e55 [FREE Full text] [CrossRef] [Medline]
    4. Hadjiconstantinou M, Byrne J, Bodicoat DH, Robertson N, Eborall H, Khunti K, et al. Do web-based interventions improve well-being in type 2 diabetes? A systematic review and meta-analysis. J Med Internet Res 2016 Oct 21;18(10):e270 [FREE Full text] [CrossRef] [Medline]
    5. Kooij L, Groen WG, Van Harten WH. The effectiveness of information technology-supported shared care for patients with chronic disease: a systematic review. J Med Internet Res 2017 Jun 22;19(6):e221 [FREE Full text] [CrossRef] [Medline]
    6. Donkin L, Christensen H, Naismith SL, Neal B, Hickie IB, Glozier N. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res 2011;13(3):e52 [FREE Full text] [CrossRef] [Medline]
    7. Manwaring JL, Bryson SW, Goldschmidt AB, Winzelberg AJ, Luce KH, Cunning D, et al. Do adherence variables predict outcome in an online program for the prevention of eating disorders? J Consult Clin Psychol 2008 Apr;76(2):341-346. [CrossRef] [Medline]
    8. Michie S, Yardley L, West R, Patrick K, Greaves F. Developing and evaluating digital interventions to promote behavior change in health and health care: recommendations resulting from an international workshop. J Med Internet Res 2017 Jun 29;19(6):e232 [FREE Full text] [CrossRef] [Medline]
    9. Eysenbach G. The law of attrition. J Med Internet Res 2005;7(1):e11 [FREE Full text] [CrossRef] [Medline]
    10. Sabaté E. Adherence to long-term therapies: evidence for action. Geneva: World Health Organization; 2003.
    11. Christensen H, Griffiths KM, Farrer L. Adherence in internet interventions for anxiety and depression. J Med Internet Res 2009;11(2):e13 [FREE Full text] [CrossRef] [Medline]
    12. Smith W, Ploderer B, Wadley G, Webber S, Borland R. Trajectories of Engagement and Disengagement with a Story-Based Smoking Cessation App. : ACM; 2017 Presented at: 2017 CHI Conference on Human Factors in Computing Systems; May 6-11, 2017; Denver, Colorado p. 3045-3056.
    13. Hekler EB, Klasnja P, Riley WT, Buman MP, Huberty J, Rivera DE, et al. Agile science: creating useful products for behavior change in the real world. Transl Behav Med 2016 Jun;6(2):317-328 [FREE Full text] [CrossRef] [Medline]
    14. Moller AC, Merchant G, Conroy DE, West R, Hekler E, Kugler KC, et al. Applying and advancing behavior change theories and techniques in the context of a digital health revolution: proposals for more effectively realizing untapped potential. J Behav Med 2017 Feb;40(1):85-98. [CrossRef] [Medline]
    15. Christensen H, Mackinnon A. The law of attrition revisited. J Med Internet Res 2006;8(3):e20 [FREE Full text] [CrossRef]
    16. van Ballegooijen W, Cuijpers P, van Straten A, Karyotaki E, Andersson G, Smit JH, et al. Adherence to Internet-based and face-to-face cognitive behavioural therapy for depression: a meta-analysis. PLoS One 2014;9(7):e100674 [FREE Full text] [CrossRef] [Medline]
    17. Beatty L, Binnion C. A systematic review of predictors of, and reasons for, adherence to online psychological interventions. Int J Behav Med 2016 Dec;23(6):776-794. [CrossRef] [Medline]
    18. El Alaoui S, Ljótsson B, Hedman E, Kaldo V, Andersson E, Rück C, et al. Predictors of symptomatic change and adherence in Internet-based cognitive behaviour therapy for social anxiety disorder in routine psychiatric care. PLoS One 2015;10(4):e0124258 [FREE Full text] [CrossRef] [Medline]
    19. Alfonsson S, Olsson E, Hursti T. Motivation and treatment credibility predicts dropout, treatment adherence, and clinical outcomes in an internet-based cognitive behavioral relaxation program: a randomized controlled trial. J Med Internet Res 2016;18(3):e52 [FREE Full text] [CrossRef] [Medline]
    20. Alfonsson S, Olsson E, Linderman S, Winnerhed S, Hursti T. Is online treatment adherence affected by presentation and therapist support? A randomized controlled trial. Comput Human Behav 2016 Jul;60(3):550-558. [CrossRef]
    21. Batterham PJ, Neil AL, Bennett K, Griffiths KM, Christensen H. Predictors of adherence among community users of a cognitive behavior therapy website. Patient Prefer Adherence 2008;2:97-105 [FREE Full text] [Medline]
    22. Beintner I, Jacobi C, Taylor CB. Participant adherence to the Internet-based prevention program StudentBodies™ for eating disorders — A review. Internet Interv 2014 Mar;1(1):26-32. [CrossRef]
    23. Christensen H, Griffiths K, Groves C, Korten A. Free range users and one hit wonders: community users of an Internet-based cognitive behaviour therapy program. Aust N Z J Psychiatry 2006 Jan;40(1):59-62. [CrossRef] [Medline]
    24. Graham AL, Cha S, Cobb NK, Fang Y, Niaura RS, Mushro A. Impact of seasonality on recruitment, retention, adherence, and outcomes in a web-based smoking cessation intervention: randomized controlled trial. J Med Internet Res 2013 Nov 07;15(11):e249 [FREE Full text] [CrossRef] [Medline]
    25. Graham AL, Cha S, Papandonatos GD, Cobb NK, Mushro A, Fang Y, et al. Improving adherence to web-based cessation programs: a randomized controlled trial study protocol. Trials 2013;14:48 [FREE Full text] [CrossRef] [Medline]
    26. Graham AL, Jacobs MA, Cohn AM, Cha S, Abroms LC, Papandonatos GD, et al. Optimising text messaging to improve adherence to web-based smoking cessation treatment: a randomised control trial protocol. BMJ Open 2016 Mar 30;6(3):e010687 [FREE Full text] [CrossRef] [Medline]
    27. Ho FY, Chung K, Yeung W, Ng TH, Cheng SK. Weekly brief phone support in self-help cognitive behavioral therapy for insomnia disorder: relevance to adherence and efficacy. Behav Res Ther 2014 Dec;63:147-156. [CrossRef] [Medline]
    28. Johansson O, Michel T, Andersson G, Paxling B. Experiences of non-adherence to Internet-delivered cognitive behavior therapy: a qualitative study. Internet Interv 2015 May;2(2):137-142. [CrossRef]
    29. Mañanes G, Vallejo MA. Usage and effectiveness of a fully automated, open-access, Spanish Web-based smoking cessation program: randomized controlled trial. J Med Internet Res 2014;16(4):e111 [FREE Full text] [CrossRef] [Medline]
    30. Lillevoll KR, Vangberg HCB, Griffiths KM, Waterloo K, Eisemann MR. Uptake and adherence of a self-directed internet-based mental health intervention with tailored e-mail reminders in senior high schools in Norway. BMC Psychiatry 2014;14:14 [FREE Full text] [CrossRef] [Medline]
    31. McCabe MP, Price E. Attrition from an internet-based psychological intervention for erectile dysfunction: who is likely to drop out? J Sex Marital Ther 2009;35(5):391-401. [CrossRef] [Medline]
    32. Mohr DC, Duffecy J, Ho J, Kwasny M, Cai X, Burns MN, et al. A randomized controlled trial evaluating a manualized TeleCoaching protocol for improving adherence to a web-based intervention for the treatment of depression. PLoS One 2013 Aug;8(8):e70086 [FREE Full text] [CrossRef] [Medline]
    33. Murray E, White IR, Varagunam M, Godfrey C, Khadjesari Z, McCambridge J. Attrition revisited: adherence and retention in a web-based alcohol trial. J Med Internet Res 2013;15(8):e162 [FREE Full text] [CrossRef] [Medline]
    34. Neil AL, Batterham P, Christensen H, Bennett K, Griffiths KM. Predictors of adherence by adolescents to a cognitive behavior therapy website in school and community-based settings. J Med Internet Res 2009;11(1):e6 [FREE Full text] [CrossRef] [Medline]
    35. Neve MJ, Collins CE, Morgan PJ. Dropout, nonusage attrition, and pretreatment predictors of nonusage attrition in a commercial Web-based weight loss program. J Med Internet Res 2010;12(4):e69 [FREE Full text] [CrossRef] [Medline]
    36. Price M, Davidson TM, Andrews JO, Ruggiero KJ. Access, use and completion of a brief disaster mental health intervention among Hispanics, African-Americans and Whites affected by Hurricane Ike. J Telemed Telecare 2013 Feb;19(2):70-74 [FREE Full text] [CrossRef] [Medline]
    37. Price M, Gros DF, McCauley JL, Gros KS, Ruggiero KJ. Nonuse and dropout attrition for a web-based mental health intervention delivered in a post-disaster context. Psychiatry 2012;75(3):267-284 [FREE Full text] [CrossRef] [Medline]
    38. Wanner M, Martin-Diener E, Bauer G, Braun-Fahrländer C, Martin BW. Comparison of trial participants and open access users of a web-based physical activity intervention regarding adherence, attrition, and repeated participation. J Med Internet Res 2010;12(1):e3 [FREE Full text] [CrossRef] [Medline]
    39. Ware LJ, Hurling R, Bataveljic O, Fairley BW, Hurst TL, Murray P, et al. Rates and determinants of uptake and use of an internet physical activity and weight management program in office and manufacturing work sites in England: cohort study. J Med Internet Res 2008;10(4):e56 [FREE Full text] [CrossRef] [Medline]
    40. Blake H, Quirk H, Leighton P, Randell T, Greening J, Guo B, et al. Feasibility of an online intervention (STAK-D) to promote physical activity in children with type 1 diabetes: protocol for a randomised controlled trial. Trials 2016 Dec 08;17(1):583 [FREE Full text] [CrossRef] [Medline]
    41. Graham AL, Papandonatos GD, Cha S, Erar B, Amato MS, Cobb NK, et al. Improving adherence to smoking cessation treatment: intervention effects in a web-based randomized trial. Nicotine Tob Res 2017 Mar 01;19(3):324-332. [CrossRef] [Medline]
    42. Grossert A, Urech C, Alder J, Gaab J, Berger T, Hess V. Web-based stress management for newly diagnosed cancer patients (STREAM-1): a randomized, wait-list controlled intervention study. BMC Cancer 2016 Nov 03;16(1):838 [FREE Full text] [CrossRef] [Medline]
    43. Hayman M, Reaburn P, Browne M, Vandelanotte C, Alley S, Short CE. Feasibility, acceptability and efficacy of a web-based computer-tailored physical activity intervention for pregnant women - the Fit4Two randomised controlled trial. BMC Pregnancy Childbirth 2017 Mar 23;17(1):96 [FREE Full text] [CrossRef] [Medline]
    44. Jander A, Crutzen R, Mercken L, Candel M, de Vries H. Effects of a web-based computer-tailored game to reduce binge drinking among Dutch adolescents: a cluster randomized controlled trial. J Med Internet Res 2016 Feb 03;18(2):e29 [FREE Full text] [CrossRef] [Medline]
    45. Kooistra LC, Ruwaard J, Wiersma JE, van Oppen P, van der Vaart R, van Gemert-Pijnen JE, et al. Development and initial evaluation of blended cognitive behavioural treatment for major depression in routine specialized mental health care. Internet Interventions 2016 May;4:61-71. [CrossRef]
    46. Noone C, Hogan MJ. A protocol for a randomised active-controlled trial to evaluate the effects of an online mindfulness intervention on executive control, critical thinking and key thinking dispositions in a university student sample. BMC Psychol 2016 Apr 12;4:17 [FREE Full text] [CrossRef] [Medline]
    47. Nordin CA, Michaelson P, Gard G, Eriksson MK. Effects of the web behavior change program for activity and multimodal pain rehabilitation: randomized controlled trial. J Med Internet Res 2016 Oct 05;18(10):e265 [FREE Full text] [CrossRef] [Medline]
    48. Schulz A, Stolz T, Vincent A, Krieger T, Andersson G, Berger T. A sorrow shared is a sorrow halved? A three-arm randomized controlled trial comparing internet-based clinician-guided individual versus group treatment for social anxiety disorder. Behav Res Ther 2016 Sep;84:14-26. [CrossRef] [Medline]
    49. Tu AW, Watts AW, Chanoine J, Panagiotopoulos C, Geller J, Brant R, et al. Does parental and adolescent participation in an e-health lifestyle modification intervention improves weight outcomes? BMC Public Health 2017 Apr 24;17(1):352 [FREE Full text] [CrossRef] [Medline]
    50. Wang Z, Wang J, Maercker A. Program use and outcome change in a Web-based trauma intervention: individual and social factors. J Med Internet Res 2016 Sep 09;18(9):e243 [FREE Full text] [CrossRef] [Medline]
    51. An LC, Perry CL, Lein EB, Klatt C, Farley DM, Bliss RL, et al. Strategies for increasing adherence to an online smoking cessation intervention for college students. Nicotine Tob Res 2006 Dec;8(Suppl 1):S7-12. [Medline]
    52. Carter MC, Burley VJ, Nykjaer C, Cade JE. Adherence to a smartphone application for weight loss compared to website and paper diary: pilot randomized controlled trial. J Med Internet Res 2013;15(4):e32 [FREE Full text] [CrossRef] [Medline]
    53. Tedim Cruz V, Pais J, Alves I, Ruano L, Mateus C, Barreto R, et al. Web-based cognitive training: patient adherence and intensity of treatment in an outpatient memory clinic. J Med Internet Res 2014 May 07;16(5):e122. [CrossRef]
    54. Farrer LM, Griffiths KM, Christensen H, Mackinnon AJ, Batterham PJ. Predictors of adherence and outcome in internet-based cognitive behavior therapy delivered in a telephone counseling setting. Cogn Ther Res 2013 Oct 27;38(3):358-367. [CrossRef]
    55. Joseph RP, Dutton GR, Cherrington A, Fontaine K, Baskin M, Casazza K, et al. Feasibility, acceptability, and characteristics associated with adherence and completion of a culturally relevant internet-enhanced physical activity pilot intervention for overweight and obese young adult African American women enrolled in college. BMC Res Notes 2015 Jun 02;8:209 [FREE Full text] [CrossRef] [Medline]
    56. Kelders SM, Bohlmeijer ET, van Gemert-Pijnen JE. Participants, usage, and use patterns of a web-based intervention for the prevention of depression within a randomized controlled trial. J Med Internet Res 2013;15(8):e172 [FREE Full text] [CrossRef] [Medline]
    57. Mâsse LC, Watts AW, Barr SI, Tu AW, Panagiotopoulos C, Geller J, et al. Individual and household predictors of adolescents' adherence to a web-based intervention. Ann Behav Med 2015 Jun;49(3):371-383. [CrossRef] [Medline]
    58. Mattila E, Orsama A, Ahtinen A, Hopsu L, Leino T, Korhonen I. Personal health technologies in employee health promotion: usage activity, usefulness, and health-related outcomes in a 1-year randomized controlled trial. JMIR Mhealth Uhealth 2013;1(2):e16 [FREE Full text] [CrossRef] [Medline]
    59. Nicholas J, Proudfoot J, Parker G, Gillis I, Burckhardt R, Manicavasagar V, et al. The ins and outs of an online bipolar education program: a study of program attrition. J Med Internet Res 2010;12(5):e57 [FREE Full text] [CrossRef] [Medline]
    60. Postel MG, de Haan HA, ter Huurne ED, van der Palen JA, Becker ES, de Jong CA. Attrition in web-based treatment for problem drinkers. J Med Internet Res 2011;13(4):e117 [FREE Full text] [CrossRef] [Medline]
    61. Richardson CR, Buis LR, Janney AW, Goodrich DE, Sen A, Hess ML, et al. An online community improves adherence in an internet-mediated walking program. Part 1: results of a randomized controlled trial. J Med Internet Res 2010;12(4):e71 [FREE Full text] [CrossRef] [Medline]
    62. Rodgers RF, Franko DL, Shiyko M, Intille S, Wilson K, O'Carroll D, et al. Exploring healthy eating among ethnic minority students using mobile technology: feasibility and adherence. Health Informatics J 2016 Sep;22(3):440-450. [CrossRef] [Medline]
    63. Titov N, Dear BF, Johnston L, Lorian C, Zou J, Wootton B, et al. Improving adherence and clinical outcomes in self-guided internet treatment for anxiety and depression: randomised controlled trial. PLoS One 2013;8(7):e62873 [FREE Full text] [CrossRef] [Medline]
    64. van den Berg S, Peters EJ, Kraaijeveld JF, Gielissen MF, Prins JB. Usage of a generic web-based self-management intervention for breast cancer survivors: substudy analysis of the BREATH trial. J Med Internet Res 2013;15(8):e170 [FREE Full text] [CrossRef] [Medline]
    65. Van der Zanden R, Curie K, Van Londen M, Kramer J, Steen G, Cuijpers P. Web-based depression treatment: associations of clients' word use with adherence and outcome. J Affect Disord 2014 May;160:10-13 [FREE Full text] [CrossRef] [Medline]
    66. Wangberg SC, Bergmo TS, Johnsen JK. Adherence in Internet-based interventions. Patient Prefer Adherence 2008;2:57-65 [FREE Full text] [Medline]
    67. Alley S, Jennings C, Plotnikoff RC, Vandelanotte C. Web-based video-coaching to assist an automated computer-tailored physical activity intervention for inactive adults: a randomized controlled trial. J Med Internet Res 2016 Aug 12;18(8):e223 [FREE Full text] [CrossRef] [Medline]
    68. Carlsen K, Jakobsen C, Houen G, Kallemose T, Paerregaard A, Riis LB, et al. Self-managed eHealth disease monitoring in children and adolescents with inflammatory bowel disease: a randomized controlled trial. Inflamm Bowel Dis 2017 Feb 13:357-365 Epub ahead of print. [CrossRef] [Medline]
    69. Fernandes-Taylor S, Gunter RL, Bennett KM, Awoyinka L, Rahman S, Greenberg CC, et al. Feasibility of implementing a patient-centered postoperative wound monitoring program using smartphone images: a pilot protocol. JMIR Res Protoc 2017 Feb 22;6(2):e26 [FREE Full text] [CrossRef] [Medline]
    70. Rutledge T, Skoyen JA, Wiese JA, Ober KM, Woods GN. A comparison of MOVE! versus TeleMOVE programs for weight loss in Veterans with obesity. Obes Res Clin Pract 2017;11(3):344-351. [CrossRef] [Medline]
    71. Schuster R, Leitner I, Carlbring P, Laireiter A. Exploring blended group interventions for depression: randomised controlled feasibility study of a blended computer- and multimedia-supported psychoeducational group intervention for adults with depressive symptoms. Internet Interv 2017 Jun;8:63-71. [CrossRef]
    72. Zarski A, Lehr D, Berking M, Riper H, Cuijpers P, Ebert DD. Adherence to internet-based mobile-supported stress management: a pooled analysis of individual participant data from three randomized controlled trials. J Med Internet Res 2016;18(6):e146 [FREE Full text] [CrossRef] [Medline]
    73. Nijland N, van Gemert-Pijnen JE, Kelders SM, Brandenburg BJ, Seydel ER. Factors influencing the use of a Web-based application for supporting the self-care of patients with type 2 diabetes: a longitudinal study. J Med Internet Res 2011;13(3):e71 [FREE Full text] [CrossRef] [Medline]
    74. Reinwand DA, Schulz DN, Crutzen R, Kremers SP, de Vries H. Who follows eHealth interventions as recommended? A study of participants' personal characteristics from the experimental arm of a randomized controlled trial. J Med Internet Res 2015;17(5):e115 [FREE Full text] [CrossRef] [Medline]
    75. Zeng EY, Heffner JL, Copeland WK, Mull KE, Bricker JB. Get with the program: adherence to a smartphone app for smoking cessation. Addict Behav 2016 Dec;63:120-124. [CrossRef] [Medline]
    76. Beatty L, Kemp E, Binnion C, Turner J, Milne D, Butow P, et al. Uptake and adherence to an online intervention for cancer-related distress: older age is not a barrier to adherence but may be a barrier to uptake. Support Care Cancer 2017 Jun;25(6):1905-1914. [CrossRef] [Medline]
    77. Mertens A, Becker S, Theis S, Rasche P, Wille M, Brohl C. Mobile Technology Improves Therapy-Adherence Rates in Elderly Patients Undergoing Rehabilitation-A Crossover Design Study. In: Duffy V, Lightner N, editors. Advances in Human Factors and Ergonomics in Healthcare. Cham: Springer Int Publishing Ag; 2017:295-308.
    78. Carolan S, Harris PR, Greenwood K, Cavanagh K. Increasing engagement with, and effectiveness of, an online CBT-based stress management intervention for employees through the use of an online facilitated bulletin board: study protocol for a pilot randomised controlled trial. Trials 2016 Dec 15;17(1):598 [FREE Full text] [CrossRef] [Medline]
    79. Cuijpers P, Berking M, Andersson G, Quigley L, Kleiboer A, Dobson KS. A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. Can J Psychiatry 2013 Jul;58(7):376-385. [Medline]
    80. Zachariae R, Lyby MS, Ritterband LM, O'Toole MS. Efficacy of internet-delivered cognitive-behavioral therapy for insomnia - a systematic review and meta-analysis of randomized controlled trials. Sleep Med Rev 2016 Dec;30:1-10. [CrossRef] [Medline]
    81. Kaushal N, Rhodes RE. Exercise habit formation in new gym members: a longitudinal study. J Behav Med 2015 Aug;38(4):652-663. [CrossRef] [Medline]
    82. Sieverink F, Kelders S, Poel M, van Gemert-Pijnen JE. Opening the black box of electronic health: collecting, analyzing, and interpreting log data. JMIR Res Protoc 2017 Aug 07;6(8):e156 [FREE Full text] [CrossRef] [Medline]
    83. van Gemert-Pijnen JE, Kelders SM, Bohlmeijer ET. Understanding the usage of content in a mental health intervention for depression: an analysis of log data. J Med Internet Res 2014;16(1):e27 [FREE Full text] [CrossRef] [Medline]
    84. Ryan C, Bergin M, Wells JS. Theoretical perspectives of adherence to web-based interventions: a scoping review. Int J Behav Med 2017 Jul 20:1-13 Epub ahead of print. [CrossRef] [Medline]

    ‎
    eHealth: electronic health
    SMS: short message service
    WHO: World Health Organization
    CBT: cognitive behavioral therapy

    Edited by G Eysenbach; submitted 27.07.17; peer-reviewed by S Schueller, E Hekler, T V, J Heffner; comments to author 25.08.17; revised version received 18.10.17; accepted 03.11.17; published 06.12.17

    Copyright

    ©Floor Sieverink, Saskia M Kelders, Julia EWC van Gemert-Pijnen. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 06.12.2017.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.