This paper is in the following e-collection/theme issue:

Web-based and Mobile Health Interventions (3092)

Published on in Vol 13, No 3 (2011): Jul-Sep

Bringing Loyalty to E-health: Theory Validation Using Three Internet-Delivered Interventions

Bringing Loyalty to E-health: Theory Validation Using Three Internet-Delivered Interventions

Bringing Loyalty to E-health: Theory Validation Using Three Internet-Delivered Interventions

Authors of this article:

Rik Crutzen1 Author Orcid Image ;   Dianne Cyr2 ;   Nanne K de Vries1
Article Authors Cited by (44) Tweetations (2) Metrics

    Original Paper

    1Maastricht University/CAPHRI, Maastricht, Netherlands

    2Simon Fraser University, Surrey, BC, Canada

    Corresponding Author:

    Rik Crutzen, PhD

    Maastricht University/CAPHRI

    P.O. Box 616

    Maastricht, 6200MD

    Netherlands

    Phone: 31 433882828

    Fax:31 433671032

    Email: rik.crutzen@maastrichtuniversity.nl


    Background: Internet-delivered interventions can effectively change health risk behaviors, but the actual use of these interventions by the target group once they access the website is often very low (high attrition, low adherence). Therefore, it is relevant and necessary to focus on factors related to use of an intervention once people arrive at the intervention website. We focused on user perceptions resulting in e-loyalty (ie, intention to visit an intervention again and to recommend it to others). A background theory for e-loyalty, however, is still lacking for Internet-delivered interventions.

    Objective: The objective of our study was to propose and validate a conceptual model regarding user perceptions and e-loyalty within the field of eHealth.

    Methods: We presented at random 3 primary prevention interventions aimed at the general public and, subsequently, participants completed validated measures regarding user perceptions and e-loyalty. Time on each intervention website was assessed by means of server registrations.

    Results: Of the 592 people who were invited to participate, 397 initiated the study (response rate: 67%) and 351 (48% female, mean age 43 years, varying in educational level) finished the study (retention rate: 88%). Internal consistency of all measures was high (Cronbach alpha > .87). The findings demonstrate that the user perceptions regarding effectiveness (betarange .21–.41) and enjoyment (betarange .14–.24) both had a positive effect on e-loyalty, which was mediated by active trust (betarange .27–.60). User perceptions and e-loyalty had low correlations with time on the website (rrange .04–.18).

    Conclusions: The consistent pattern of findings speaks in favor of their robustness and contributes to theory validation regarding e-loyalty. The importance of a theory-driven solution to a practice-based problem (ie, low actual use) needs to be stressed in view of the importance of the Internet in terms of intervention development. Longitudinal studies are needed to investigate whether people will actually revisit intervention websites and whether this leads to changes in health risk behaviors.

    J Med Internet Res 2011;13(3):e73

    doi:10.2196/jmir.1837

    Keywords

    e-Loyalty; adherence; attrition; user perceptions; theory; Internet; interventions 


    Internet-delivered interventions can effectively change health risk behaviors (eg, lack of physical activity, low consumption of fruit, cigarette smoking, and excessive alcohol consumption) [1]. However, the actual use of these interventions by the target group once they access the website is very low [2,3]. For example, server statistics of an intervention promoting heart-healthy behaviors showed that 285,146 visitors from unique internet protocol (IP) addresses landed on the home page in a 36-month period, but 56.3% of these left the intervention website within 30 seconds [4]. This finding touches on the critical issue in Internet-delivered interventions: how can behavior ever be changed if people are not exposed or are only briefly exposed to the actual intervention? Therefore, it is relevant and necessary to focus on factors related to use of an intervention once people arrive at the intervention website. These factors relate to the visitor (eg, people’s motivation to be healthy [5,6]) as well as the intervention website (eg, visual complexity of the homepage). Two recently published systematic reviews provide a detailed overview of factors used by current interventions to stimulate use of intervention websites [7,8]. Our study focused on perceptions of visitors resulting in a user experience [9,10].

    User experience refers to what a person thinks and feels during and after exposure to a website [11]. The main idea is that a positive user experience leads to increased website use. User experience consists of cognitive and affective perceptions [12]. Cognitive perceptions are rational in nature and are induced by utilitarian or cognitive motives. Affective perceptions are emotional in nature and are induced by hedonic or affective motives [13]. Previous studies demonstrated the importance of these perceptions regarding intention to use a technology [14] and to visit a website again [12]. We designed our study on the basis of these findings and applied them to loyalty regarding intervention websites in the field of eHealth (ie, e-loyalty). Besides visiting an Internet-delivered intervention again, e-loyalty also consists of recommending an Internet-delivered intervention to others. The latter is based on previous research indicating that word-of-mouth is an effective strategy to improve use of Internet-delivered interventions [15,16]. A background theory for e-loyalty is still lacking for Internet-delivered interventions. Although previous studies did explicitly describe the theory used to develop the content of Internet-delivered interventions, these theories primarily related to behavior determinants or behavior change [8,17]. Theory development regarding e-loyalty is highly needed to increase the public health impact of Internet-delivered interventions. Therefore, in this study we propose and validate a conceptual model.

    To systematically constitute the proposed conceptual model, we describe conceptual definitions and their relationship with e-loyalty [18]. Terminology that is used within the conceptual model (ie, key user perceptions) is derived from other fields such as e-commerce. Although these terms can have a different meaning within public health, we chose to use the same terminology as in previous studies in other fields to avoid further confusion. The key user perceptions in the conceptual model are efficiency, effectiveness, trustworthiness, enjoyment, and active trust [12]. Efficiency refers to easy search of and access to the information provided, and effectiveness refers to the quality of that information (eg, in terms of relevance) [19]. These cognitive perceptions have parallels with perceived ease of use and perceived usefulness in the technology acceptance model, but are applicable in a broader context [20]. The positive effect of these cognitive perceptions on e-loyalty has been demonstrated in, for example, e-service environments [21]. The idea that a positive user experience leads to e-loyalty applies not only to cognitive perceptions, but also to affective perceptions [5,22]. These affective perceptions are often referred to as enjoyment [23] and have been demonstrated to have a positive effect on e-loyalty in, for example, e-commerce [24]. Trustworthiness is defined as the believability of the provided information and refers to both cognitive and affective perceptions: it is based on a cognitive process (eg, rational reasons) and an emotional base (eg, a strong positive affect for the trustee) [25]. It has been demonstrated to have a positive effect on e-loyalty in, for example, online shopping [26,27]. Active trust might be a working mechanism leading to e-loyalty [28]. Whereas trustworthiness refers to the believability (eg, “I trust the information presented on this website”), active trust refers to the confidence in acting on the provided information (eg, “I would act on the information presented on this website”). Active trust has been proven to be the primary intermediate associated with e-loyalty [26,29]. In line with the study of Cugelman and colleagues [28], we expected active trust to mediate the impact of trustworthiness and effectiveness on e-loyalty. This resulted in the following hypotheses to be tested in a new context lacking a background theory: the field of eHealth (Figure 1).

    ‎
    Figure 1. Conceptual model.
    View this figure

    Hypothesis 1

    H1a Efficiency has a positive effect on e-loyalty.
    H1b Effectiveness has a positive effect on e-loyalty.
    H1c Enjoyment has a positive effect on e-loyalty.

    Hypothesis 2

    H2a Active trust mediates the relationship between effectiveness and e-loyalty.
    H2b Active trust mediates the relationship between trustworthiness and e-loyalty.

    To improve the external and ecological validity, we included 3 generally available, Internet-delivered interventions. The interventions were certified according to the guidelines of the Dutch recognition system for health promotion interventions [30]. The quality assessment of health promotion interventions is supervised by the Netherlands Institute for Public Health and the Environment (interventions aimed at adults) and the Netherlands Youth Institute (interventions aimed at youth) [31]. We included interventions from all levels of recognition (theoretically sound, probable effectiveness, and established effectiveness; inspired by the UK Medical Research Council’s evaluation framework for complex interventions) in this study. The first intervention, registered by the Consumer and Safety Foundation (Netherlands), was theoretically sound and is concerned with prevention of sports injuries (intervention 1 [32]). The second intervention, registered by the Netherlands Institute of Mental Health and Addiction, was probably effective and is concerned with drinking less alcohol (intervention 2 [33]). The third intervention, registered by the Netherlands Institute of Mental Health and Addiction, was effective and is intended for people feeling gloomy or having mild depressive complaints (intervention 3 [34]). We must stress that these were all primary prevention interventions aimed at the general public. In other words, these interventions were not targeted at diagnosing (secondary prevention) or treating (tertiary prevention) health problems related to health risk behaviors, but at people who did not yet have these problems. Hence, these interventions were deemed of interest to the general public.

    Participants

    Participants were recruited through a research panel of a Dutch Internet research agency [35]. From this panel, we invited through email a stratified sample of 592 potential participants to take part in this study. This sample was representative of the Dutch population above 18 years, taking into account gender, age, and level of education. Of those invited, 397 clicked on the link in the invitation email to start the study (response rate: 67%) and 351 finished the study (retention rate: 88%). There was no selective dropout regarding gender (n = 592, χ21 = 3.2, P = .08), but those who dropped out were somewhat younger (40 vs 43 years, t590 = 2.86, P = .004) and differed in terms of level of education (n = 592, χ21 = 10.9, P = .004). The final sample consisted of 48% (169/351) women; the average age was 43 (SD 13) years. In terms of level of education, 30% (107/351) of the participants had a low level of highest completed education, 35% (122/351) an intermediate level, and 35% (122/351) a high level (according to the definitions of Statistics Netherlands).

    Procedure

    The study consisted of 3 blocks (ie, 3 intervention websites and related measurements) that were presented at random to each participant. In each block participants were exposed to 1 of the 3 intervention websites described above, and subsequently participants completed the measures described in the measurements section. Participants were asked to assess several websites. It was stressed that there were no right or wrong answers and they could base their opinion on their first impression. The reason behind this was to prevent participants from thoroughly studying the intervention website, and to mimic a real-life situation in which the time being exposed to and willing to investigate an intervention website is often limited [4]. On average, participants took 17 minutes to complete the full study (eg, exploring the intervention websites and completing related measurements). Participants received credit points for participating in the study, for a value of €1.95.

    Measurements

    Directly after exposure to each intervention website, participants indicated whether they had seen the website before. For each intervention website, data from participants who indicated that they had seen website before were removed, because their perceptions and loyalty might have been based on the previous exposure to the intervention website. This concerned 8 (different) participants per intervention website and results did not differ if their data were included. Subsequently, participants completed the following validated measures after being exposed to each intervention website.

    E-loyalty: intention to visit the website again (eg, “It is likely that I will visit the website again in the future”) and whether participants would recommend the website to others (“It is likely that I will recommend this website to others “) were assessed by 3 items each [36]. Items were answered on a 7-point Likert scale ranging from “strongly agree” to “strongly disagree.”

    User perceptions: efficiency (eg, “I was able to access the information quickly on this website”), effectiveness (eg, “The website provided me with relevant information about...”), trustworthiness (eg, “I trust the information presented on this website”), enjoyment (eg, “I found my visit to this website enjoyable”), and active trust (eg, “I would act on the information presented on this website if needed”) were assessed by 3 items each [21,37]. Items were answered on a 7-point Likert scale ranging from “strongly agree” to “strongly disagree.”

    Two native speakers (RC and an assistant) translated all items into Dutch and discussed semantic similarity until reaching a consensus. Besides these self-reported measures, time on each intervention website was assessed by means of server registrations.

    Analyses

    First, using Predictive Analytics SoftWare Statistics (version 18.0; IBM Corporation, Somers, NY, USA), we conducted correlation and reliability analyses for each intervention website separately. Subsequently, using Mplus (version 5; Muthén & Muthén, Los Angeles, CA, USA), we constructed structural equation models to test the hypotheses per intervention website. First of all, we tested the hypothesized conceptual model: intention to visit again and recommending to others were regressed on efficiency, effectiveness, and enjoyment; active trust was regressed on effectiveness and trustworthiness. Subsequently, we added paths to the conceptual model based on modification indices, which are chi-square distributed, implying that a modification index larger than 3.84 indicates that adding the suggested path will significantly improve model fit. The reason to include paths beyond the hypotheses was to explore whether unanticipated relationships might explain variance in e-loyalty and, hence, contribute to theory development. The criterion for accepting or rejecting a hypothesis was a significant pattern across all 3 models. A level of significance of .05 was used for the relationships within the model.

    Comparative fit index (CFI), Tucker-Lewis index (TLI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR) were used as fit indices for each model. CFI and TLI are goodness-of-fit indices, where larger values signal better fit. Values over .95 indicate close fit. RMSEA and SRMR are goodness-of-fit indices, where larger values signal worse fit. Rules of thumb for close fit are RMSEA ≤ .05 and SRMR ≤ .09 [38,39].


    Tables 1–3 show the results of correlation and reliability analyses. Internal consistency of all measures was high (Cronbach alpha > .87). Overall, correlations between user perceptions and e-loyalty were high (rrange .44–.84). User perceptions and e-loyalty have low correlations with time on the website (rrange .04–.18).

    Table 1. Correlation matrix intervention 1 (N = 343)
    AlphaMeanSD12345678
    1. Efficiency.984.71.6–.68.72.71.61.58.60.04
    2. Effectiveness.954.41.6–.79.77.82.75.76.13
    3. Trustworthiness.974.71.4–.73.72.64.66.16
    4. Enjoyment.994.11.7–.79.74.75.09
    5. Active trust.944.21.7–.76.76.10
    6. Intention to visit again.893.61.7–.84.09
    7. Recommend to others.954.01.7–.14
    8. Time on website (minutes)–3:06–
    Table 2. Correlation matrix intervention 2 (N = 343)
    AlphaMeanSD12345678
    1. Efficiency.974.81.5–.62.63.60.57.49.58.12
    2. Effectiveness.914.31.5–.71.67.76.63.70.16
    3. Trustworthiness.964.61.4–.66.68.50.61.14
    4. Enjoyment.984.01.5–.71.62.67.09
    5. Active trust.914.11.6–.71.74.16
    6. Intention to visit again.873.31.7–.77.18
    7. Recommend to others.944.01.7–.16
    8. Time on website (minutes)–1:28–
    Table 3. Correlation matrix intervention 3 (N = 343)
    AlphaMeanSD12345678
    1. Efficiency.984.71.5–.63.61.57.57.44.57.09
    2. Effectiveness.953.91.7–.73.72.77.72.78.14
    3. Trustworthiness.974.41.4–.70.70.56.69.07
    4. Enjoyment.984.01.6–.79.70.75.11
    5. Active trust.943.81.7–.7477.09
    6. Intention to visit again.913.31.7–.82.10
    7. Recommend to others.963.71.8–.14
    8. Time on website (minutes)–3:10–

    Table 4 shows the results of the structural equation models when testing the conceptual model. H1a was rejected; efficiency did not have a positive effect on e-loyalty. H1b and H1c were confirmed; both effectiveness and enjoyment had a positive effect on e-loyalty. H2a was also confirmed; active trust mediated the relationship between effectiveness and e-loyalty. Results for H2b were mixed, because the relationship between trustworthiness and active trust differed in terms of being significant and standardized betas [40,41]. Therefore, the relationship between trustworthiness and active trust was included when adding paths to the conceptual model based on modification indices. The only path that was added to the conceptual model was the relationship between enjoyment and active trust. Modification indices (respective values of 50.27, 39.15, and 72.62) suggested the addition of this path to each model representing an intervention website. Table 5 shows the results of the structural equation models when testing this extended model. The results were similar to the conceptual model: H1a was rejected and H1b, H1c, and H2a were confirmed. H2b, however, was rejected; active trust did not mediate the relationship between trustworthiness and e-loyalty, because there was no relationship between trustworthiness and active trust. Unanticipatedly, but consistently, the positive effect of enjoyment was mediated by active trust. All fit indices indicated good fit for the extended model. Figure 2 shows the extended model resulting from the analyses for all 3 intervention websites.

    Table 4. Results of conceptual model (figures are standardized betas of paths within the model)
    Intervention
    123
    PathaINTbRECcINTRECINTREC
    EFI → e-loyaltynsdnsnsns–.16ns
    EFE → e-loyalty.28.25ns.22.42.43
    ENJ → e-loyalty.26.29.19.21.20.25
    ACT → e-loyalty.39.33.58.40.36.26
    EFE → ACT.81.71.66
    TRU → ACTns.16.21
    R2.73.68.56.63.65.70
    CFIe.95.95.95
    TLIf.94.94.94
    RMSEAg.10.09.09
    SRMRh.05.06.06

    a EFI = efficiency; EFE = effectiveness; TRU = trustworthiness; ENJ = enjoyment; ACT = active trust.

    b Intention to visit again.

    c Recommend to others.

    d Not significant; all other paths are significant at the P = .05 level.

    e Comparative fit index.

    f Tucker-Lewis index.

    g Root mean square error of approximation.

    h Standardized root mean square residual.

    Table 5. Results of extended model (figures are standardized betas of paths within the model)
    Intervention
    123
    PathaINTbRECcINTRECINTREC
    EFI → e-loyaltynsdnsnsns–.15ns
    EFE → e-loyalty.27.24ns.21.40.41
    ENJ → e-loyalty.23.27.14.18.17.24
    ACT → e-loyalty.41.35.60.42.39.27
    EFE → ACT.57.51.40
    TRU → ACTnsnsns
    ENJ → ACT.37.34.47
    R2.73.68.57.64.65.70
    CFIe.95.96.96
    TLIf.94.95.95
    RMSEAg.09.09.09
    SRMRh.05.05.05

    a EFI = efficiency; EFE = effectiveness; TRU = trustworthiness; ENJ = enjoyment; ACT = active trust.

    b Intention to visit again.

    c Recommend to others.

    d Not significant; all other paths are significant at the P = .05 level.

    e Comparative fit index.

    f Tucker-Lewis index.

    g Root mean square error of approximation.

    h Standardized root mean square residual.

    ‎
    Figure 2. Extended Model.
    View this figure

    Our findings consistently demonstrate that effectiveness and enjoyment both had a positive effect on e-loyalty, which was mediated by active trust. The findings regarding effectiveness were anticipated and in line with previous research [28]. Mediation of the positive effect of enjoyment by active trust, however, was unanticipated. An explanation can be based on previous research demonstrating that enjoyment is related to cognitive perceptions [42]. Thus, cognitive perceptions might be a working mechanism for the positive effect of enjoyment on e-loyalty. Future research is needed to shed more light on the plausibility of this explanation, since this relationship can also be reversed: affective perceptions as a working mechanism leading to e-loyalty [43].

    Rejection of the hypothesis regarding the positive effect of efficiency on e-loyalty can be explained by the procedure used in this study. Efficiency refers to easy search of and access to the information provided. Participants, however, were not necessarily looking for information regarding the topic of the intervention websites to which they were exposed. Although participants could fill out the items regarding efficiency based on whether the intervention website at hand would be easy to search and access if they were looking for information at that intervention website, the lack of a need for information might explain the absence of evidence for a positive effect of efficiency. This could be solved by giving participants an assignment for which they have to study the intervention website thoroughly. The reason why we did not do this in the current study was to mimic a real-life situation in which people might review an intervention website when time limitations prevail [4]. This was reflected in this study as well, given the average time on website (range 1:28–3:10 minutes).

    The lack of a relationship between trustworthiness and active trust in the structural equation models is puzzling. It might be that active trust by itself captures all the variance in e-loyalty that could be explained by trustworthiness. Since previous research demonstrated that active trust is the primary intermediate associated with e-loyalty [26,29], it might be that active trust reduces the possible impact of trustworthiness. This is contrary to previous research [28], however, and still does not explain the absence of a relationship between trustworthiness and active trust.

    A final finding that deserves attention is that user perceptions and e-loyalty had low correlation with time on website. This can be explained by a confirmation bias [44]: since participants were told that they had to assess several websites, they might have been looking for evidence in line with their first impression, regardless of whether their impression was negative or positive. So, in the current setting the time spent on an intervention website is independent of user perceptions. Time on website may be related to user perceptions and e-loyalty if people explore an intervention website without any instructions.

    In sum, although not all hypotheses were confirmed, this study clearly demonstrates that user perceptions (ie, effectiveness, enjoyment, and active trust) regarding e-loyalty are not important just in fields such as e-commerce, but also in the context of eHealth. The next question is how to improve user perceptions of intervention websites. To answer this question, characteristics of intervention websites need to be systematically manipulated, and these manipulations should be linked to user perceptions, and subsequently to e-loyalty. A possible variable to be manipulated is user control, defined as the voluntary and instrumental actions of a website visitor that influence the user experience [45,46]. The ability to control information flow increases one’s ability to explore and understand the structure of a website [47]. Nevertheless, one of the most common issues faced by visitors of websites is lack of user control [48]. This is awkward, given the wealth of literature (eg, McMillan and Hwang provide an overview [49]) documenting the importance of user control in shaping user experience [45,50]. Furthermore, previous research identified the role of user control (ie, freedom of choice) in attitude change [51] and intention to use [12,14]. The effect of user control on e-loyalty is in line with previous studies and is expected to be mediated through user perceptions [12,52]. Another characteristic to be manipulated in future research might be the use of tailoring strategies (eg, personalization, feedback) that have been shown to have a positive effect on intervention outcomes (in terms of health behaviors), which is related to intervention use [53].

    Acknowledgments

    This study was supported by the Innovational Research Incentives Scheme Veni from NWO-MaGW (Netherlands Organisation for Scientific Research - Division for the Social Sciences) accredited to the first author. There has been no involvement by the funding body in the review or approval of the manuscript for publication. The authors would like to thank Nienke Beekers for her help in translating items into Dutch and the members of “het voedingsoverleg” for their valuable comments on an earlier version of the manuscript.

    Conflicts of Interest

    None declared

    1. Portnoy DB, Scott-Sheldon LA, Johnson BT, Carey MP. Computer-delivered interventions for health promotion and behavioral risk reduction: a meta-analysis of 75 randomized controlled trials, 1988-2007. Prev Med 2008 Jul;47(1):3-16. [CrossRef] [Medline]
    2. Bennett GG, Glasgow RE. The delivery of public health interventions via the Internet: actualizing their potential. Annu Rev Public Health 2009 Apr 29;30:273-292. [CrossRef] [Medline]
    3. Eysenbach G. The law of attrition. J Med Internet Res 2005;7(1):e11 [FREE Full text] [CrossRef] [Medline]
    4. Brouwer W, Oenema A, Raat H, Crutzen R, de Nooijer J, de Vries NK, et al. Characteristics of visitors and revisitors to an Internet-delivered computer-tailored lifestyle intervention implemented for use by the general public. Health Educ Res 2010 Aug;25(4):585-595. [CrossRef] [Medline]
    5. Van 't Riet J, Crutzen R, De Vries H. Investigating predictors of visiting, using, and revisiting an online health-communication program: a longitudinal study. J Med Internet Res 2010;12(3):e37 [FREE Full text] [CrossRef] [Medline]
    6. Crutzen R, de Nooijer J, Candel MJ, de Vries NK. Adolescents who intend to change multiple health behaviours choose greater exposure to an internet-delivered intervention. J Health Psychol 2008 Oct;13(7):906-911. [CrossRef] [Medline]
    7. Brouwer W, Kroeze W, Crutzen R, de Nooijer J, de Vries NK, Brug J, et al. Which intervention characteristics are related to more exposure to internet-delivered healthy lifestyle promotion interventions? A systematic review. J Med Internet Res 2011;13(1):e2 [FREE Full text] [CrossRef] [Medline]
    8. Crutzen R, de Nooijer J, Brouwer W, Oenema A, Brug J, de Vries NK. Strategies to facilitate exposure to internet-delivered health behavior change interventions aimed at adolescents or young adults: a systematic review. Health Educ Behav 2011 Feb;38(1):49-62. [CrossRef] [Medline]
    9. Van Oppen CAML. From Rags to Richness. Maastricht: Maastricht University; 2007.
    10. Sundar SS. Theorizing interactivity's effects. Inf Soc 2004;20:385-389. [CrossRef]
    11. Crutzen R, de Nooijer J, Brouwer W, Oenema A, Brug J, de Vries NK. A conceptual framework for understanding and improving adolescents' exposure to Internet-delivered interventions. Health Promot Int 2009 Sep;24(3):277-284 [FREE Full text] [CrossRef] [Medline]
    12. Cyr D, Head M, Ivanov A. Perceived interactivity leading to e-loyalty: development of a model for cognitive-affective user responses. Int J Hum Comput Stud 2009;67:850-869. [CrossRef]
    13. Park CW, Young SM. Consumer response to television commercials: the impact of involvement and background music on brand attitude formation. J Marketing Res 1986;23:11-24.
    14. Lee T. The impact of perceptions of interactivity on customer trust and transaction intentions in mobile commerce. J Electron Commer Res 2005;6(3):165-180.
    15. Crutzen R, de Nooijer J, Brouwer W, Oenema A, Brug J, de Vries N. Effectiveness of online word of mouth on exposure to an Internet-delivered intervention. Psychol Health 2009 Jul;24(6):651-661. [CrossRef] [Medline]
    16. Crutzen R, de Nooijer J, Brouwer W, Oenema A, Brug J, de Vries NK. Internet-delivered interventions aimed at adolescents: a Delphi study on dissemination and exposure. Health Educ Res 2008 Jun;23(3):427-439 [FREE Full text] [CrossRef] [Medline]
    17. Christensen H, Griffiths KM, Farrer L. Adherence in internet interventions for anxiety and depression. J Med Internet Res 2009;11(2):e13 [FREE Full text] [CrossRef] [Medline]
    18. Wacker JG. A definition of theory: research guidelines for different theory-building research methods in operations management. Journal of Operations Management 1998;16:361-385. [CrossRef]
    19. Keeney R. The value of Internet commerce to the customer. Manag Sci 1999;45:533-542. [CrossRef]
    20. Benbasat I, Barki H. Quo vadis, TAM? J Assoc Inf Syst 2007;8(4):211-218.
    21. Cyr D, Hassanein K, Head M, Ivanov A. The role of social presence in establishing loyalty in e-service environments. Interact Comput 2007;19:43-56. [CrossRef]
    22. Sun Y, Wang N, Peng Z. Working for one penny: understanding why people would like to participate in online tasks with low payment. Comput Hum Behav 2011;27:1033-1041. [CrossRef]
    23. Van der Heijden H. Factors influencing the usage of websites: the case of a generic portal in the Netherlands. Inf Manag 2003;40:541-549. [CrossRef]
    24. Cyr D, Head M, Ivanov A. Design aesthetics leading to m-loyalty in mobile commerce. Inf Manag 2006;43:950-963. [CrossRef]
    25. Komiak S, Benbasat I. Understanding customer trust in agent-mediated electronic commerce, web-mediated electronic commerce, and traditional commerce. Inf Technol Manag 2004;5:181-207. [CrossRef]
    26. Flavián C, Guinalíu M, Gurrea R. The role played by perceived usability, satisfaction and consumer trust on website loyalty. Inf Manag 2006;43:1-14. [CrossRef]
    27. Gefen D, Karahanna E, Straub D. Trust and TAM in online shopping: an integrated model. MIS Q 2003;27(1):51-90.
    28. Cugelman B, Thelwall M, Dawes P. The dimensions of web site credibility and their relation to active trust and behavioural impact. Commun Assoc Inf Syst 2009;24:455-472.
    29. Bart Y, Shankar V, Fareena S, Urban G. Are the drivers and roles of online trust the same for all web sites and consumers? A large-scale exploratory empirical study. J Marketing 2005;69:133-152. [CrossRef]
    30. National Institute for Public Health and the Environment (Netherlands). Centre for Healthy Living (Centrum Gezond Leven). 2011. I-database   URL: http://www.loketgezondleven.nl/interventies/i-database [accessed 2011-04-22] [WebCite Cache]
    31. Brug J, van Dale D, Lanting L, Kremers S, Veenhof C, Leurs M, et al. Towards evidence-based, quality-controlled health promotion: the Dutch recognition system for health promotion interventions. Health Educ Res 2010 Dec;25(6):1100-1106 [FREE Full text] [CrossRef] [Medline]
    32. Voorkom Blessures met Gezondverstand. Stichting Consument en Veiligheid. 2011   URL: http://www.voorkomblessures.nl/csi/websitesportblessure.nsf/ [accessed 2011-04-22] [WebCite Cache]
    33. Netherlands Institute of Mental Health and Addiction. Trimbos Instituut. 2011. MinderDrinken   URL: http://www.minderdrinken.nl/ [accessed 2011-04-22] [WebCite Cache]
    34. Netherlands Institute of Mental Health and Addiction. Mentalshare. 2011. 'Kleur je leven' Online Zelfhulpcursus   URL: https://www.kleurjeleven.nl/ [accessed 2011-04-22] [WebCite Cache]
    35. FlyCatcher. FlyCatcher Internet Research. 2011   URL: http://www.flycatcher.eu/ [accessed 2011-04-22] [WebCite Cache]
    36. Cyr D, Bonanni C, Bowes J, Ilsever J. Beyond trust: website design preferences across cultures. J Glob Inf Manag 2005;13:25-52. [CrossRef]
    37. Teo H, Oh L, Liu C, Wei K. An empirical study of the effects of interactivity on web user attitude. Int J Hum Comput Stud 2003;58:281-305. [CrossRef]
    38. Iacobucci D. Structural equations modeling: fit indices, sample size, and advanced topics. J Consum Psychol 2010;20:90-98. [CrossRef]
    39. Kline RB. Principles and Practice of Structural Equation Modeling. New York, NY: Guilford Press; 2005.
    40. Crutzen R. A systematic review on computer-based education for patients with hypertension: what about effect sizes? Health Educ J 2010;69:365-366. [CrossRef]
    41. Crutzen R. Adding effect sizes to a systematic review on interventions for promoting physical activity among European teenagers. Int J Behav Nutr Phys Act 2010;7:29 [FREE Full text] [CrossRef] [Medline]
    42. Kim J, Fiore AM. Lee H-H. Influences of online store perception, shopping enjoyment, and shopping involvement on consumer patronage behavior towards an online retailer. J Retailing Consum Serv 2007;14:95-107. [CrossRef]
    43. De Wulf K, Schillewaert N, Muylle S, Rangarajan D. The role of pleasure in web site success. Inf Manag 2006;43:434-446. [CrossRef]
    44. Nickerson RS. Confirmation bias: a ubiquitous phenomenon in many guises. Rev Gen Psychol 1998;2:175-220. [CrossRef]
    45. Liu Y, Schrum LJ. What is interactivity and is it always such a good thing? Implications of definition, person, and situation for the influence of interactivity on advertising effectiveness. J Advertising 2002;31:53-64.
    46. Lowry PB, Spaulding T, Wells T, Moody G, Moffit K, Madariaga S. A theoretical model and empirical results linking website interactivity and usability satisfaction. Syst Sci 2006;123. [CrossRef]
    47. Ariely D. Controlling the information flow: effects on consumers' decision making and preferences. J Consum Res 2000;27:233-248. [CrossRef]
    48. Srinivasan SS, Anderson R, Ponnavolu K. Customer loyalty in e-commerce: an exploration of its antecedents and consequences. J Retailing 2002;78:41. [CrossRef]
    49. McMillan SJ. Hwang J-S. Measures of perceived interactivity: an exploration of communication, user control, and time in shaping perceptions of interactivity. J Advertising 2002;31:29-42.
    50. Liu Y, Schrum LJ. A dual-process model of interactivity effects. J Advertising 2009;38:53-68.
    51. Chiou WB. Induced attitude change on online gaming among adolescents: an application of the less-leads-to-more effect. Cyberpsychol Behav 2008 Apr;11(2):212-216. [CrossRef] [Medline]
    52. Hassenzahl M, Tractinsky N. User experience: a research agenda. Behav Inf Technol 2006;25:91-97. [CrossRef]
    53. Cugelman B, Thelwall M, Dawes P. Online interventions for social marketing health behavior change campaigns: a meta-analysis of psychological architectures and adherence factors. J Med Internet Res 2011;13(1):e17 [FREE Full text] [CrossRef] [Medline]

    Edited by G Eysenbach; submitted 22.04.11; peer-reviewed by J Brown, L Yardley; comments to author 12.05.11; revised version received 25.05.11; accepted 26.05.11; published 24.09.11

    Copyright

    ©Rik Crutzen, Dianne Cyr, Nanne K de Vries. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 24.09.2011.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.