This paper is in the following e-collection/theme issue:

Web-based and Mobile Health Interventions (3092) Diabetes Education and Elearning for Patients (104)

Published on in Vol 19, No 1 (2017): January

Effects on Engagement and Health Literacy Outcomes of Web-Based Materials Promoting Physical Activity in People With Diabetes: An International Randomized Trial

Effects on Engagement and Health Literacy Outcomes of Web-Based Materials Promoting Physical Activity in People With Diabetes: An International Randomized Trial

Effects on Engagement and Health Literacy Outcomes of Web-Based Materials Promoting Physical Activity in People With Diabetes: An International Randomized Trial

Authors of this article:

Ingrid Muller1 Author Orcid Image ;   Alison Rowsell1, 2 Author Orcid Image ;   Beth Stuart3 Author Orcid Image ;   Victoria Hayter1 Author Orcid Image ;   Paul Little3 Author Orcid Image ;   Kristin Ganahl4 Author Orcid Image ;   Gabriele Müller5 Author Orcid Image ;   Gerardine Doyle6 Author Orcid Image ;   Peter Chang7 Author Orcid Image ;   Courtney R Lyles8 Author Orcid Image ;   Don Nutbeam9 Author Orcid Image ;   Lucy Yardley1 Author Orcid Image
Article Authors Cited by (30) Tweetations (25) Metrics

    Original Paper

    1Department of Psychology, University of Southampton, Southampton, United Kingdom

    2University of Southampton, Department of Health Sciences, Southampton, United Kingdom

    3Primary Care and Population Sciences, University of Southampton, Southampton, United Kingdom

    4Gesundheit Österreich GmbH (Austrian Public Health Institute), Vienna, Austria

    5Centre for Evidence-based Healthcare, University Hospital Carl Gustav Carus, Technical University Dresden, Dresden, Germany

    6UCD College of Business, University College Dublin, Dublin, Ireland

    7Taipei Medical University and National Taipei Hospital, Taipei, Taiwan

    8Division of General Internal Medicine, Zuckerberg San Francisco General Hospital, University of California, San Francisco, San Francisco, CA, United States

    9School of Public Health, University of Sydney, Sydney, Australia

    ‡The Diabetes Literacy Study Group

    Corresponding Author:

    Ingrid Muller, PhD

    Department of Psychology

    University of Southampton

    Highfield campus

    Southampton, SO17 1BJ

    United Kingdom

    Phone: 44 02380 592581

    Fax:44 0000000

    Email: I.Muller@soton.ac.uk


    Background: Developing accessible Web-based materials to support diabetes self-management in people with lower levels of health literacy is a continuing challenge.

    Objective: The objective of this international study was to develop a Web-based intervention promoting physical activity among people with type 2 diabetes to determine whether audiovisual presentation and interactivity (quizzes, planners, tailoring) could help to overcome the digital divide by making digital interventions accessible and effective for people with all levels of health literacy. This study also aimed to determine whether these materials can improve health literacy outcomes for people with lower levels of health literacy and also be effective for people with higher levels of health literacy.

    Methods: To assess the impact of interactivity and audiovisual features on usage, engagement, and health literacy outcomes, we designed two versions of a Web-based intervention (one interactive and one plain-text version of the same content) to promote physical activity in people with type 2 diabetes. We randomly assigned participants from the United Kingdom, Austria, Germany, Ireland, and Taiwan to either an interactive or plain-text version of the intervention in English, German, or Mandarin. Intervention usage was objectively recorded by the intervention software. Self-report measures were taken at baseline and follow-up (immediately after participants viewed the intervention) and included measures of health literacy, engagement (website satisfaction and willingness to recommend the intervention to others), and health literacy outcomes (diabetes knowledge, enablement, attitude, perceived behavioral control, and intention to undertake physical activity).

    Results: In total, 1041 people took part in this study. Of the 1005 who completed health literacy information, 268 (26.67%) had intermediate or low levels of health literacy. The interactive intervention overall did not produce better outcomes than did the plain-text version. Participants in the plain-text intervention group looked at significantly more sections of the intervention (mean difference –0.47, 95% CI –0.64 to –0.30, P<.001), but this did not lead to better outcomes. Health literacy outcomes, including attitudes and intentions to engage in physical activity, significantly improved following the intervention for participants in both intervention groups. These improvements were similar across higher and lower health literacy levels and in all countries. Participants in the interactive intervention group had acquired more diabetes knowledge (mean difference 0.80, 95% CI 0.65-0.94, P<.001). Participants from both groups reported high levels of website satisfaction and would recommend the website to others.

    Conclusions: Following established practice for simple, clear design and presentation and using a person-based approach to intervention development, with in-depth iterative feedback from users, may be more important than interactivity and audiovisual presentations when developing accessible digital health interventions to improve health literacy outcomes.

    ClinicalTrial: International Standard Randomized Controlled Trial Number (ISRCTN): 43587048; http://www.isrctn.com/ISRCTN43587048. (Archived by WebCite at http://www.webcitation.org/6nGhaP9bv)

    J Med Internet Res 2017;19(1):e21

    doi:10.2196/jmir.6601

    Keywords

    health literacy; digital intervention; diabetes; quantitative trial; physical activity 


    Health literacy has been defined as “knowledge, motivation and competences to access, understand, appraise, and apply health information” [1]. The capacity to understand and apply health information depends not only on the capabilities of the individual, but also on the way in which health information is presented. Well-designed materials to support self-management of health can help to improve health literacy outcomes such as knowledge, motivation, confidence, and adherence [2,3]. Lower levels of health literacy are associated with poor illness management, health knowledge, health service use, and health and with higher mortality. Addressing the challenges posed by low health literacy in populations has been highlighted as an urgent priority in many countries [4].

    Barriers to accessing support for self-management of chronic health problems include disability, cost, work or family responsibilities, and lack of transport [5]. Studies have shown that these barriers are more common among people with lower levels of education [6]. Web-based health interventions may help address this problem, as they can be conveniently accessed in the home and reach large numbers of people at low cost, thereby having the potential to reduce health disparities [7]. Access to and use of the Internet through a personal computer or mobile phone is rapidly becoming common among more sections of the population, with over 80% of the adult population now using the Internet in the countries participating in this study [8]. However, low health literacy levels may present barriers to understanding and applying health information obtained from the Internet [9-11]. Lower levels of eHealth literacy are also associated with lower levels of healthy behavior, such as physical activity [12]. Therefore, reducing the “literacy burden” of online health information is an important strategy in making support for self-management of chronic conditions more accessible.

    To date, interventions to reduce the literacy burden and improve health literacy have included using simple language, audiovisual or pictorial formats, interactivity, and tailoring of content to individuals’ needs (if the intervention is Web based). Reviews of the effectiveness of such interventions for the general public and mixed-patient populations [13-17] and for diabetes [18-20] suggest that these techniques show promise for some outcomes, but that overall the evidence for improving health literacy or reducing the literacy burden is weak and inconclusive, and it remains unclear exactly which elements of such interventions improve which outcomes.

    This study addressed the evidence gap regarding how best to design Web-based materials for the growing population of patients with basic literacy and computer skills but lower levels of health literacy. We developed a Web-based intervention to promote physical activity in people with type 2 diabetes, following established best practice for designing accessible Web-based written content. We included a range of interactive elements (quiz, tailoring, a planner) and audiovisual modes of presentation, so that we could evaluate whether these improved usage and health literacy outcomes, particularly in those with lower levels of health literacy. We used our person-based approach to intervention development [21], carrying out iterative qualitative research with people with high and low levels of health literacy to gain feedback to improve accessibility and engagement [22].

    This paper reports on a subsequent large international quantitative study comparing this Web-based intervention with a static, plain-text presentation of identical content. The study evaluated engagement and heath literacy outcomes in people with varied levels of health literacy. We measured engagement by objectively recorded intervention usage and self-reported user experience (website satisfaction and whether participants would recommend the website to others) [23]. The primary research question asked whether an interactive, tailored, and audiovisual Web-based intervention would lead to better engagement than a plain-text version of the same content. Secondary research questions asked (1) whether we could design a Web-based intervention that people with lower and higher levels of health literacy find engaging, (2) whether these materials could improve health literacy outcomes for people with lower levels of health literacy, and (3) whether the materials would also be effective for people with higher levels of health literacy.


    Intervention Development

    Healthy Living with Diabetes is a tailored Web-based intervention to motivate people with type 2 diabetes to increase their physical activity. The intervention was developed by a team of health researchers at the University of Southampton, United Kingdom, in collaboration with the Diabetes Literacy research consortium [24], patient representatives, and an international expert panel.

    We developed 2 Web-based interventions using the LifeGuide software, an open access platform for developing Web-based behavior change interventions [25]. The first was a plain-text version of the intervention, and the second was an interactive version of the intervention. Both versions included the same content, which was written and designed to be accessible for people with lower levels of health literacy and to be engaging and novel. To enhance engagement, the intervention content contained novel and compelling information about the benefits of physical activity for people with type 2 diabetes. To enhance accessibility, we followed good practice guidelines for accessible Web-based design and presentation of written content [26-31] in both interventions.

    We designed the interactive version to assess the additional impact that interactivity, audiovisual features, and tailoring may have on engagement with the intervention and health literacy outcomes in people with varied health literacy levels. Audiovisual aspects of the interactive intervention were positive images throughout, and a series of audiovisual sequences demonstrating lifestyle and physical activities (tailored to age and sex). The interactive features of the website consisted of a quiz, a physical activity planner, and tailored advice, feedback, and images based on user responses to questions (such as current physical activity levels, attitudes to physical activity, age, and sex).

    We first developed the intervention in English for testing in the United Kingdom, and then adapted and translated it for testing in Austria, Germany, Ireland, Taiwan, and the United States. Researchers in the United States did not take part in this subsequent trial. We followed our person-based approach to intervention development [21,32] to enhance acceptability and feasibility from the earliest stages of intervention development through an in-depth understanding of the views and perspectives of our target users. Full details of the development and qualitative evaluation of the intervention, including screenshots of the intervention, have previously been published [22].

    Design

    We carried out a multisite randomized trial in the United Kingdom, Austria, Germany, Ireland, and Taiwan to compare the interactive Web-based materials versus a plain-text Web-based version of the intervention. The plain-text intervention contained the same content and structure as the interactive version, but all tailoring, interactivity, and audiovisual features were removed. Ethics and research governance approvals were granted by the University of Southampton and UK National Health Service (NHS) research ethics committees (number 13/LO/0316).

    Participants and Procedure

    Participants were invited to take part in the study if they were over 18 years old with a diagnosis of type 2 diabetes, had access to the Internet, were able to read the intervention language (English, German, or Mandarin), and give informed consent. We recruited participants from the United Kingdom, Austria, Germany, Ireland, and Taiwan between July 2014 and March 2015. Minor country differences in recruitment procedures were permitted to allow for differing health care systems and patient access. UK participants were recruited through 43 primary care practices specifically selected for being in areas of high deprivation in order to reach more people with low health literacy. Participants in Ireland and Taiwan were recruited opportunistically by health care professionals in diabetes outpatient clinics, and participants in Austria and Germany were recruited through national diabetes support group newsletters and advertisements placed on the Internet. Health care professionals in the United Kingdom, Ireland, and Taiwan screened potential participants to exclude patients with potential difficulties, including severe mental health problems, palliative care, recent bereavement, and inability to complete research measures (eg, learning disability, inability to read or speak an intervention language) before they were invited to the study.

    Participants from all countries were presented with details of the study, research team contact details for more information, and a website URL where they could log in to the Web-based intervention on their own time. Participant information stated that we were comparing two types of webpages to see which was more helpful; it did not mention website features such as interactivity or audiovisual features. Participants were therefore blinded to what the differences between the 2 arms were. Consent was given online, and participants completed a very brief baseline questionnaire before being randomly assigned to 1 of the 2 groups (with a 50% ratio). Participants were then presented with either the interactive or plain-text Web-based materials, depending on randomization assignment. Participants were asked follow-up questions immediately after using the intervention. All recruitment and follow-up procedures (including full study information, obtaining informed consent, baseline and follow-up data collection, and randomization) were Web based using automated procedures carried out by the LifeGuide software [25].

    Sample Size

    We calculated the sample size a priori using the G*Power 3 (version 3.1.9.2) sample size calculation program [33]. We calculated that a minimum sample size of 676 participants in total would be required to detect a small difference (effect size, Cohen d=0.25) between the 2 groups on our primary outcome measure of objective intervention usage, with alpha=.05 and beta=.1.

    Measures

    Participants completed Web-based assessments at baseline (immediately before) and follow-up (immediately after using the intervention materials). We collected demographic variables at baseline, consisting of age, sex, time since diabetes diagnosis, and age they left full-time education. Participants’ levels of physical activity during the previous week were measured at follow-up using the International Physical Activity Questionnaire-Short Form (IPAQ-SF) self-administered questionnaire assessing the minutes spent doing vigorous and moderate activity and walking in the last 7 days [34]. We scored the IPAQ-SF using the recommended categorical scoring system [35], where participants are categorized as being either (1) inactive, (2) minimally active, or (3) highly active.

    We measured engagement with the Web-based intervention by intervention usage and self-reported measures of engagement. Intervention usage was measured by the number of intervention sections completed, as total time spent on the intervention was likely to be confounded with format (plain text vs interactive). Both the interactive and plain-text intervention were designed to comprise 5 distinct sections: knowledge of physical activity benefits (with/without interactive quiz); advice on selecting physical activities (with/without tailoring); advice on planning physical activity (with/without interactive planner); success stories (with/without audiovisual presentation); access to further information about undertaking physical activity. All intervention usage data was automatically recorded by the LifeGuide software [25]. Self-reported measures of engagement at follow-up were a previously validated 3-item measure of satisfaction with Web-delivered advice [36], and a single item measuring whether participants would recommend the website to friends and family with diabetes, based on the NHS Friends and Family Test [37].

    Health literacy outcomes were (1) diabetes knowledge, (2) patient enablement, and (3) attitude, behavioral control, and intention to undertake physical activity. Diabetes knowledge was measured by a 9-item knowledge quiz based on the intervention content. Patient enablement was measured by 3 items from the Patient Enablement Instrument [38] assessing participants’ perceptions of their understanding of the benefits of physical activity for people with diabetes, their ability to cope with diabetes, and confidence in managing their health. Participants completed these measures immediately after viewing the intervention. Attitude, behavioral control, and intention to undertake physical activity were measured by 3 items drawn from the theory of planned behavior [39]. Participants completed these items at baseline (immediately before viewing the intervention) and follow-up (immediately after completing the intervention) in order to assess change. Responses were given on a 7-point Likert scale (ranging from disagree to agree). These 3 items were (1) “Increasing my level of physical activity would be good for me” (physical activity attitude), (2) “I would find it easy to increase my level of physical activity” (perceived behavioral control), and (3) “I plan to increase my level of physical activity” (physical activity intentions).

    We measured health literacy at baseline by a validated single item: “How often do you have problems learning about your condition because of difficulty understanding written information?” [40]. On the basis of this measure, we identified participants as having high, intermediate, or low levels of health literacy. Measures were translated from English to German and Mandarin and checked by each country’s research team for accuracy. All measures were optional apart from age and sex, which were essential for tailoring.

    Analysis

    We analyzed the data using IBM SPSS for Windows version 14.0 (IBM Corporation) and Stata statistical software Special Edition Release 2007 (version 13; StataCorp LP), following a prespecified data analysis plan developed with our statistician (BS) and approved by the whole Diabetes Literacy consortium. All comparisons of the plain-text and interactive versions of the website controlled for potential confounding effects of the covariates health literacy, education, age, sex, and illness duration. We allowed for clustering by country by including country as a random effect in the model.

    Due to the small numbers of participants with low health literacy levels, we categorized health literacy as low/intermediate compared with high health literacy. To avoid undertaking too many between-country comparisons, analyses by country compared UK data with a pooled sample of all other countries, as the UK sample was the largest and the intervention materials were originally developed for testing in the United Kingdom, and then translated and adapted for other countries and cultures.

    The primary research question asked whether an interactive, tailored, and audiovisual Web-based intervention can lead to better engagement than a plain-text version of the same content can. The primary analysis compared the number of intervention sections completed by participants randomly assigned to the interactive intervention versus the number completed by participants randomly assigned to the plain-text intervention to test the prediction that more sections of the interactive version of the Web-based intervention would be completed. We used linear regression to compare the mean difference between intervention groups. We then examined whether intervention usage was moderated by health literacy level or by country. For these analyses, we carried out linear regressions to look for group differences by health literacy level and country. Post hoc exploratory analyses of Web usage were carried out using visualization analyses to examine patterns of intervention usage. Intervention usage data were analyzed using the LifeGuide visualization tool [41] to explore patterns of intervention use. This tool enables researchers to visualize and compare which intervention features were viewed, for how long, and in what order, across all participants.

    Secondary research questions asked whether people with high and low health literacy found the materials engaging, and whether the intervention improved health literacy outcomes in people with lower and high levels of health literacy. In order to answer these questions, we analyzed self-report measures of engagement (website satisfaction; recommending the website to others) and health literacy outcomes (diabetes knowledge; patient enablement; and change in attitude, behavioral control, and intention to undertake physical activity) using linear regression models and then assessed for potential moderator effects by heath literacy level and country.

    Missing Data

    The main outcome for this study was intervention usage, which was automatically recorded by the intervention software for all participants and therefore had no missing data. We investigated levels of missing data for baseline and follow-up measures and compared the frequency of missing data between the 2 intervention groups. Levels of missing data were high for the diabetes knowledge quiz score (459/1041, 44.09% missing) and the single item measuring whether participants would recommend the intervention to others (231/1041, 22.19% missing data). We assumed that these were at random and applied a multiple imputation model of 100 imputations for missing secondary outcomes and key covariates. We present this analysis as a sensitivity analysis alongside the main analysis on complete cases.


    Participants

    In total, 1045 participants from the United Kingdom, Austria, Germany, Ireland, and Taiwan participated in the study and were randomly assigned to view either the interactive intervention or the plain-text intervention. Of these, 4 participants used the Back button on their Internet browsers to be rerandomized and were consequently excluded, resulting in 1041 participants in the final analysis. We successfully measured the primary outcome, intervention usage, for 100% of randomly assigned participants. See Figure 1 for the Consolidated Standards of Reporting Trials (CONSORT) flow diagram.

    ‎
    Figure 1. Consolidated Standards of Reporting Trials (CONSORT) flow diagram.
    View this figure

    Participant Characteristics

    Participants in this study were predominantly male (662/1041, 63.59%), with a mean age of 62 years. On average, participants left full-time education before the age of 18 years and had been diagnosed with type 2 diabetes for 9.2 years (this ranged from just a few months to 50 years). The majority of participants (737/1005, 73.33%) had high levels of health literacy, while 268/1005 (26.67%) had intermediate or low levels of health literacy. A total of 835/1041 (80.21%) of participants completed the IPAQ-SF physical activity questionnaire. Most of these participants reported being inactive (561/835, 67.2%), while some reported being minimally active (190/835, 22.8%) and a minority reported being highly active (84/835, 10.1%). Participant characteristics were similar across both groups at baseline, with the only slight difference being higher health literacy levels in the interactive group. See Table 1 for participant characteristics by intervention group. Participant characteristics were similar by country (see Multimedia Appendix 1 for details).

    Table 1. Participant characteristics in the 2 arms of the Web-based intervention promoting physical activity among people with type 2 diabetes.
    CharacteristicGroup

    		Plain-text (n=497)Interactive (n=544)
    Female, n (%)182 (36.6)197 (36.2)
    Age in years, mean (SD)61.5 (11.2)62.4 (11.4)
    Years since diagnosis, mean (SD)9.1 (9.1)9.5 (9.3)
    Age when left full-time education, mean (SD)17.8 (3.0)17.8 (3.0)
    Health literacy level (single-item measure), n (%)

    		Low37/478 (7.7)30/527 (5.7)

    		Intermediate105/478 (22.0)96/527 (18.2)

    		High336/478 (70.3)401/527 (76.1)
    Physical activity attitudes and intentions

    		IPAQ-SFa, mean (SD)15.1 (3.5)15.0 (3.7)

    		Highly active, n (%)35/431 (8.1)49/404 (12.1)

    		Minimally active, n (%)106/431 (24.6)84/404 (20.8)

    		Inactive, n (%)290/431 (67.3)271/404 (67.1)

    aIPAQ-SF: International Physical Activity Questionnaire-Short Form.

    Intervention Usage

    The primary outcome in this study was Web-based intervention usage to test whether the interactive intervention led to better engagement than the plain-text version. Analysis of usage data found a significant difference in intervention usage between the 2 groups, with participants in the interactive intervention group being likely to complete fewer of the 5 intervention sections than were participants in the plain-text intervention group (mean difference –0.47, 95% CI –0.64 to –0.30, P<.001). Table 2 gives the results of intervention usage analyses.

    Moderator analysis examined intervention usage by health literacy level. Participants with higher levels of health literacy were significantly more likely to complete more sections of the intervention (mean difference 0.25, 95% CI 0.05-0.45, P=.02; Table 3).

    Table 2. Results of analysis of intervention usage as determined by number of sections completed, and results of self-reported measures of engagement and moderator analyses of self-reported engagement, by intervention group.
    AnalysisIntervention groupUnivariate differenceMultivariate differenceaMultivariate differencea based on 100 imputations

    		Plain textInteractiveMean (95% CI)P valueMean (95% CI)P valueMean (95% CI)P value
    Intervention usage

    		No. of sections completed, mean (SD)4.5 (1.3)4.0 (1.5)–0.47 (–0.64 to –0.30)<.001–0.49 (–0.67 to –0.31)<.001N/AbN/A
    Measures of engagement

    		Satisfied with website, mean (SD)4.1 (2.0)4.1 (1.9)0.03 (–0.24 to 0.30).820.05 (–0.22 to 0.33).700.08 (–0.19 to 0.35).54

    		Would recommend to others, n (%)281/419 (67.1)248/391 (63.4)0.85 (0.64 to 1.14).280.85 (0.62 to 1.15).290.78 (0.58 to 1.05).10

    aAll analyses controlled for possible confounding by age, sex, time since diagnosis, age when the participant left education, health literacy, and for clustering by country.

    bN/A: not applicable.

    Table 3. Results of analysis of intervention usage as determined by number of sections completed, and results of self-reported measures of engagement and moderator analyses of self-reported engagement, by health literacy level.
    AnalysisHealth literacy levelUnivariate differenceMultivariate differenceaMultivariate differencea based on 100 imputations

    		LowerHighMean (95% CI)P valueMean (95% CI)P valueMean (95% CI)P value
    Intervention usage

    		No. of sections completed, mean (SD)4.1 (1.5)4.3 (1.4)0.25 (0.05 to 0.45).020.28 (0.08 to 0.48).01N/AbN/A
    Measures of engagement

    		Satisfied with website, mean (SD)4.1 (2.0)4.1 (2.0)–0.03 (–0.34 to 0.29).870.05 (–0.27 to 0.37).760.04 (–0.28 to 0.35).82

    		Would recommend to others, n (%)139/195 (71.3)372/591 (62.9)0.70 (0.48 to 0.97).040.64 (0.44 to 0.93).020.69 (0.48 to 1.01).05

    aAll analyses controlled for possible confounding by age, sex, time since diagnosis, age when the participant left education, health literacy, and for clustering by country.

    bN/A: not applicable.

    ‎
    Figure 2. Visualization of intervention usage by health literacy level and intervention. Blue: time spent on quiz; yellow: time spent on physical activity planner; red: time spent on reading personal tips; green: time spent on audiovisual sequences.
    View this figure

    We carried out exploratory analyses to examine whether intervention usage differed by country (comparing the United Kingdom versus the other participating countries). Patterns of usage were similar in the United Kingdom and the other countries. See Multimedia Appendix 2 for details of intervention usage by country. Extensive visualization analyses of usage in the whole sample revealed no difference in how the intervention was used by health literacy level, age, sex, time since diagnosis, physical activity level, or change in physical activity attitude. Visualization analyses outputs illustrate intervention usage for the selected sample over time (in 30-second intervals). Visualization analyses comparing usage of the interactive intervention and the static intervention revealed that the interactive group spent more time on the quiz (represented by blue in Figure 2) and the physical activity planner (represented by yellow), while the static intervention group spent more time reading personal tips (represented by red). These differences could be explained by the interactive nature of the quiz and planner adding to the time taken to complete them, while the lack of tailoring in the static intervention increased the reading burden for the personal tips section (since all tips were presented rather than just those tailored to the users). Similarly, we did not include audiovisual sequences (represented by green) in the static intervention and these are therefore represented only in the visualization of usage of the interactive intervention. See Figure 2 for visualization of intervention usage by health literacy level and intervention.

    Self-Reported Measures of Engagement

    The self-reported measures of engagement were website satisfaction and a single item measuring whether participants would recommend the website to others. We used these items to address the secondary research question asking whether Web-based materials can be developed to be engaging to people with low and high levels of health literacy. There were no significant group differences, with participants in both groups reporting high levels of website satisfaction and the majority of participants in both groups reporting that they would be likely to recommend the website to others. Table 2 and Table 3 give details of these results.

    We carried out exploratory analyses to evaluate whether self-reported measures of engagement varied by health literacy level. Participants with lower health literacy were significantly more likely to recommend the website to friends or family with diabetes (mean difference –0.70, 95% CI 0.48-0.97, P=.04), although this difference was no longer significant following 100 imputations (mean difference 0.69, 95% CI 0.48-1.01, P=.05). There were no significant differences in website satisfaction, with participants with all levels of health literacy reporting high levels of satisfaction. See Table 3 for details. Moderator analysis found that the same pattern of results occurred in the UK data compared with other countries. See Multimedia Appendix 3 for details of moderator analyses by country.

    Health Literacy Outcomes

    Secondary research questions asked whether the Web-based materials could improve health literacy outcomes in people with low health literacy and be effective for people with higher levels of health literacy. The health literacy outcomes in this study were (1) diabetes knowledge, (2) patient enablement, and (3) change in attitude, behavioral control, and intention to undertake physical activity. There was a significant group difference in participants’ diabetes knowledge, with participants in the interactive group scoring significantly higher than the plain-text intervention group (mean difference 0.80, 95% CI 0.65-0.94, P<.001). The diabetes knowledge measure had a ceiling effect with a large proportion of participants from both groups scoring highly. When comparing participants who answered all the questions correctly with those who got 1 or more wrong, the group difference was maintained and the interactive group was nearly 7 times more likely than the plain-text group to have answered all the questions correctly (mean difference 6.5, P<.001, 95% CI 4.4-9.4). There were no significant group differences in patient enablement, with participants in both groups reporting feeling more enabled as a result of using the intervention materials. Details of these results are given in Table 4.

    Table 4. Health literacy outcomes by intervention group.
    OutcomeIntervention groupUnivariate differenceMultivariate differenceaMultivariate differencea based on 100 imputations

    		Plain textInteractiveMean (95% CI)P valueMean (95% CI)P valueMean (95% CI)P value
    Diabetes knowledge, mean (SD)8.0 (1.1)8.8 (0.5)0.80 (0.65 to 0.94)<.0010.78 (0.63 to 0.92)<.0010.74 (0.50 to 0.88)<.001
    Diabetes knowledge score of 9 vs lower score, n (%)124/303 (40.9)228/279 (81.7)6.5 (4.4 to 9.4)<.0016.9 (4.6 to 10.3)<.0014.90 (3.35 to 7.17)<.001
    Patient Enablement Instrument, mean (SD)7.5 (3.1)7.6 (3.0)0.08 (–0.33 to 0.49).700.02 (–0.40 to 0.43).930.17 (–0.25 to 0.58).44

    aAll analyses controlled for possible confounding by age, sex, time since diagnosis, age when the participant left education, health literacy, and for clustering by country.

    Moderator analyses explored these results by health literacy level. There was a trend for people with higher levels of health literacy to score higher on the Patient Enablement Instrument (multivariate mean difference 0.53, 95% CI 0.04-1.02, P<.03), although this was no longer significant following 100 imputations (mean difference 0.40, 95% CI –0.09 to 0.88, P<.11). There were no significant health literacy differences in diabetes knowledge acquired, with both groups scoring highly. See Table 5 for details. Moderation analyses by country showed a similar pattern of results for the United Kingdom compared with other countries; see Multimedia Appendix 4 for details.

    Participants were asked about their attitudes and intentions toward physical activity at baseline and again at follow-up, enabling an analysis to establish whether the score had changed within each group. In both intervention groups, and across all health literacy levels, the score at follow-up was significantly higher than at baseline, indicating that participants from all groups had more positive attitudes and intentions toward physical activity after viewing the intervention materials. Table 6 shows the results of this analysis.

    Table 5. Moderator analyses of health literacy outcomes by health literacy levels.
    OutcomeHealth literacy levelUnivariate differenceMultivariate differenceaMultivariate differencea based on 100 imputations

    		LowerHighMean (95% CI)P valueMean (95% CI)P valueMean (95% CI)P value
    Diabetes knowledge, mean (SD)8.2 (1.1)8.4 (0.9)0.16 (–0.02 to 0.35).090.13 (–0.05 to 0.30).160.13 (–0.06 to 0.32).18
    Diabetes knowledge score of 9 vs lower score, n (%)73/132 (55.3)270/434 (62.2)1.33 (0.90 to 1.97).161.31 (0.83 to 2.07).251.27 (0.84 to 1.92).26
    Patient Enablement Instrument, mean (SD)7.3 (2.8)7.7 (3.1)0.39 (–0.09 to 0.87).110.53 (0.04 to 1.02).030.40 (–0.09 to 0.88).11

    aAll analyses controlled for possible confounding by age, sex, time since diagnosis, age when the participant left education, and for clustering by country.

    Table 6. Change in attitude behavioral control and physical activity intentions from baseline to follow-up across all groups and literacy levels.
    OutcomePlain text groupInteractive groupLower health literacyHigh health literacy

    		Mean (95% CI)P valueMean (95% CI)P valueMean (95% CI)P valueMean (95% CI)P value
    Physical activity attitude0.10 (0.02-0.18).010.22 (0.11-0.34)<.0010.15 (0.02-0.27).020.15 (0.07-0.23)<.001
    Perceived behavioral control0.34 (0.24-0.45).0010.35 (0.22-0.47)<.0010.33 (0.17-0.49)<.0010.34 (0.24-0.43)<.001
    Physical activity intention0.35 (0.24-0.45)<.0010.49 (0.35-0.63)<.0010.27 (0.10-0.44).0020.46 (0.35-0.56)<.001

    Principal Findings

    The main finding of this study was that the interactive intervention overall did not produce better outcomes than those obtained by a plain-text version of the intervention. Participants in the plain-text intervention group showed higher levels of engagement by completing more sections of the intervention, although this did not lead to better health literacy outcomes, and participants in the interactive intervention group had better diabetes knowledge.

    Health literacy outcomes significantly improved following the intervention to a very similar extent in both groups. These significant changes were reflected across all health literacy levels and all countries, with participants reporting increased beliefs in the benefits of physical activity, greater confidence in undertaking physical activity, and a stronger intention to increase physical activity as a result of the intervention. Given the low levels of physical activity reported by our sample, these changes in attitude to physical activity are positive, and it is encouraging that we observed these changes in those with lower levels of health literacy, since low self-confidence for physical activity has been shown to be a key mediator of the association between low health literacy and inactivity [42]. Diabetes knowledge was higher in the interactive group, suggesting that the interactive quiz format may have been useful for learning new information. Both interactive and plain-text intervention groups reported high levels of enablement as a result of viewing the intervention materials, and both intervention groups were likely to recommend the intervention to friends or family with diabetes.

    Analysis by health literacy level revealed few differences. Participants with high levels of health literacy completed more sections of the intervention, but this did not lead to better health literacy outcomes. Participants with high health literacy reported higher levels of enablement, and participants with lower health literacy were more likely to recommend the intervention to others, but these differences were not significant after correcting for missing data. Despite these minor group differences, there are encouraging signs that the intervention design was accessible and helpful for people with all health literacy levels. These findings are consistent with evidence from previous research that interventions designed to be accessible for people with lower health literacy can be suitable for people with higher health literacy [15,22,43]. Participants with all health literacy levels reported high levels of enablement and were likely to recommend the intervention to friends or family members with diabetes. We observed similar patterns of results in the United Kingdom compared with other countries, suggesting the translated and adapted materials were equally effective. A detailed description and illustrations of the intervention have previously been published [22].

    However, more work is needed to engage hard-to-reach populations in Web-based interventions. Despite deliberately sampling in socially deprived populations, we attracted surprisingly few people with lower levels of health literacy.

    Limitations

    This study did not succeed in recruiting many participants with very low levels of health literacy, and the results can therefore not be generalized to this group. It is also important to note that the results only refer to our version of interactivity, and others may be able to produce more engaging interactive materials. This study was not powered for examining interactions, and all subgroup analyses were exploratory and should be interpreted with caution. There were minor recruitment differences between countries, which should be taken into account when interpreting response rates. We did not undertake longer-term follow-up and therefore do not know the extent to which the immediate intervention effect will endure in this population. Since this study did not include a control group, we cannot draw firm conclusions regarding the effectiveness of the Web-based intervention content, since changes in attitudes before and after viewing the content could in theory have been due to other factors.

    Conclusion

    In this study, a good, clear design and person-based intervention development [21,32] to establish an in-depth understanding of the views and perspectives of target users appears to have been more important than interactivity and audiovisual presentation when developing accessible digital health interventions to improve health literacy outcomes. This approach also seems able to be adapted for successful use in different counties and cultures. The finding that the same materials can be equally engaging for people with high and lower levels of health literacy is important, since the need to tailor or target interventions for different sectors of the population increases the complexity of interventions and could reduce their cost effectiveness. Consequently, well-designed digital communication materials that have been developed and evaluated for accessibility with a range of users may be sufficient as a means of filling unmet needs for improving health literacy. Looking to the future, more needs to be done to encourage and support intervention providers to develop Web-based materials that can benefit people with limited health literacy.

    Acknowledgments

    This research was part of the Diabetes Literacy project supported by grant FP7-Health-2012-Innovation-1/306186 of the European Commission. We would like to thank the members of the Diabetes Literacy Study Group who assisted with data collection, in particular Peter Schwarz, Jürgen Pelikan, Sarah Gibney, and Becky Sun. We would especially like to thank our 2 lay advisors, Andy Hamson and Wendy Heath. We would also like to acknowledge the support of the National Institute of Health Research Clinical Research Network.

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    Participant characteristics by country.

    PDF File (Adobe PDF File), 31KB
    ‎

    Multimedia Appendix 2

    Moderator analysis of intervention usage by country.

    PDF File (Adobe PDF File), 19KB
    ‎

    Multimedia Appendix 3

    Moderator analyses of self-reported engagement by country.

    PDF File (Adobe PDF File), 20KB
    ‎

    Multimedia Appendix 4

    Moderator analyses of health literacy outcomes by country.

    PDF File (Adobe PDF File), 21KB
    ‎

    Multimedia Appendix 5

    CONSORT-EHEALTH checklist V1.6.2 [44].

    PDF File (Adobe PDF File), 678KB
    1. Sørensen K, Van den Broucke S, Fullam J, Doyle G, Pelikan J, Slonska Z, (HLS-EU) Consortium Health Literacy Project European. Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health 2012;12:80 [FREE Full text] [CrossRef] [Medline]
    2. Nutbeam D. The evolving concept of health literacy. Soc Sci Med 2008 Dec;67(12):2072-2078. [CrossRef] [Medline]
    3. Geboers B, Brainard JS, Loke YK, Jansen CJ, Salter C, Reijneveld SA, et al. The association of health literacy with adherence in older adults, and its role in interventions: a systematic meta-review. BMC Public Health 2015 Sep 17;15:903 [FREE Full text] [CrossRef] [Medline]
    4. Greenhalgh T. Health literacy: towards system level solutions. BMJ 2015 Feb 24;350:h1026. [Medline]
    5. Pereira K, Phillips B, Johnson C, Vorderstrasse A. Internet delivered diabetes self-management education: a review. Diabetes Technol Ther 2015 Jan;17(1):55-63. [CrossRef] [Medline]
    6. Thoolen B, de Ridder RD, Bensing J, Gorter K, Rutten G. Who participates in diabetes self-management interventions?: Issues of recruitment and retainment. Diabetes Educ 2007;33(3):465-474. [CrossRef] [Medline]
    7. Muñoz RF. Using evidence-based internet interventions to reduce health disparities worldwide. J Med Internet Res 2010;12(5):e60 [FREE Full text] [CrossRef] [Medline]
    8. The World Bank. World Bank Open Data. Washington, DC: World Bank Group; 2016.   URL: http://data.worldbank.org/ [accessed 2017-01-11] [WebCite Cache]
    9. Kusec S, Brborovic O, Schillinger D. Diabetes websites accredited by the Health On the Net Foundation Code of Conduct: readable or not? Stud Health Technol Inform 2003;95:655-660. [Medline]
    10. Glasgow RE, Strycker LA, Kurz D, Faber A, Bell H, Dickman JM, et al. Recruitment for an internet-based diabetes self-management program: scientific and ethical implications. Ann Behav Med 2010 Aug;40(1):40-48. [CrossRef] [Medline]
    11. Sarkar U, Karter AJ, Liu JY, Adler NE, Nguyen R, Lopez A, et al. The literacy divide: health literacy and the use of an internet-based patient portal in an integrated health system-results from the diabetes study of northern California (DISTANCE). J Health Commun 2010;15 Suppl 2:183-196 [FREE Full text] [CrossRef] [Medline]
    12. Mitsutake S, Shibata A, Ishii K, Oka K. Associations of eHealth literacy with health behavior among adult internet users. J Med Internet Res 2016 Jul;18(7):e192 [FREE Full text] [CrossRef] [Medline]
    13. Clement S, Ibrahim S, Crichton N, Wolf M, Rowlands G. Complex interventions to improve the health of people with limited literacy: A systematic review. Patient Educ Couns 2009 Jun;75(3):340-351. [CrossRef] [Medline]
    14. Synnot A, Ryan R, Prictor M, Fetherstonhaugh D, Parker B. Audio-visual presentation of information for informed consent for participation in clinical trials. Cochrane Database Syst Rev 2014 May 09(5):CD003717. [CrossRef] [Medline]
    15. Mackert M, Champlin SE, Holton A, Muñoz II, Damásio MJ. eHealth and health literacy: a research methodology review. J Comput-Mediat Comm 2014 Apr 12;19(3):516-528. [CrossRef]
    16. Jacobs RJ, Lou JQ, Ownby RL, Caballero J. A systematic review of eHealth interventions to improve health literacy. Health Informatics J 2014 Jun 10:81-98. [CrossRef] [Medline]
    17. Barry MM, D'Eath M, Sixsmith J. Interventions for improving population health literacy: insights from a rapid review of the evidence. J Health Commun 2013;18(12):1507-1522. [CrossRef] [Medline]
    18. Welch G, Shayne R. Interactive behavioral technologies and diabetes self-management support: recent research findings from clinical trials. Curr Diab Rep 2006 Apr;6(2):130-136. [Medline]
    19. Glasgow RE. Interactive media for diabetes self-management: issues in maximizing public health impact. Med Decis Making 2010;30(6):745-758. [CrossRef] [Medline]
    20. Fransen MP, von Wagner C, Essink-Bot M. Diabetes self-management in patients with low health literacy: ordering findings from literature in a health literacy framework. Patient Educ Couns 2012 Jul;88(1):44-53. [CrossRef] [Medline]
    21. Yardley L, Morrison L, Bradbury K, Muller I. The person-based approach to intervention development: application to digital health-related behavior change interventions. J Med Internet Res 2015;17(1):e30 [FREE Full text] [CrossRef] [Medline]
    22. Rowsell A, Muller I, Murray E, Little P, Byrne CD, Ganahl K, et al. Views of people with high and low levels of health literacy about a digital intervention to promote physical activity for diabetes: a qualitative study in five countries. J Med Internet Res 2015 Oct 12;17(10):e230 [FREE Full text] [CrossRef] [Medline]
    23. Yardley L, Spring BJ, Riper H, Morrison LG, Crane DH, Curtis K, et al. Understanding and promoting engagement with digital behavior change interventions. Am J Prev Med 2016 Nov;51(5):833-842. [CrossRef] [Medline]
    24. Van den Broucke S, Van der Zanden G, Chang P, Doyle G, Levin D, Pelikan J, et al. Enhancing the effectiveness of diabetes self-management education: the diabetes literacy project. Horm Metab Res 2014 Dec;46(13):933-938. [CrossRef] [Medline]
    25. LifeGuide online. Southampton, UK: University of Southampton; 2016.   URL: https://www.lifeguideonline.org/ [accessed 2017-01-04] [WebCite Cache]
    26. Maher T. Plain English campaign. High Peak, UK: Plain English Campaign; 2014.   URL: http://www.plainenglish.co.uk [accessed 2014-11-07] [WebCite Cache]
    27. Rotondi AJ, Sinkule J, Haas GL, Spring MB, Litschge CM, Newhill CE, et al. Designing websites for persons with cognitive deficits: Design and usability of a psychoeducational intervention for persons with severe mental illness. Psychol Serv 2007 Aug;4(3):202-224 [FREE Full text] [CrossRef] [Medline]
    28. Krug S. Don't Make Me Think!: A Common Sense Approach to Web Usability. 2nd edition. Berkeley, CA: New Riders; 2005.
    29. U.S. Dept. of Health and Human Services. usability.gov: improving the user experience. 2014.   URL: http://www.usability.gov [accessed 2014-11-07] [WebCite Cache]
    30. Hirsh D, Clerehan R, Staples M, Osborne RH, Buchbinder R. Patient assessment of medication information leaflets and validation of the Evaluative Linguistic Framework (ELF). Patient Educ Couns 2009 Nov;77(2):248-254. [CrossRef] [Medline]
    31. Rotondi AJ, Eack SM, Hanusa BH, Spring MB, Haas GL. Critical design elements of e-health applications for users with severe mental illness: singular focus, simple architecture, prominent contents, explicit navigation, and inclusive hyperlinks. Schizophr Bull 2015 Mar;41(2):440-448. [CrossRef] [Medline]
    32. Yardley L, Ainsworth B, Arden-Close E, Muller I. The person-based approach to enhancing the acceptability and feasibility of interventions. Pilot Feasibility Stud 2015 Oct 26;1(1):1-6. [CrossRef]
    33. Faul F, Erdfelder E, Lang A, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 2007 May;39(2):175-191. [Medline]
    34. Ainsworth BE, Bassett DR, Strath SJ, Swartz AM, O'Brien WL, Thompson RW, et al. Comparison of three methods for measuring the time spent in physical activity. Med Sci Sports Exerc 2000 Sep;32(9 Suppl):S457-S464. [Medline]
    35. Guidelines for data processing and analysis of the International Physical Activity Questionnaire (IPAQ)-Short Form: Version 2.0. 2004 Apr. IPAQ scoring protocol   URL: http://www.institutferran.org/documentos/scoring_short_ipaq_april04.pdf [accessed 2017-01-08] [WebCite Cache]
    36. Yardley L, Joseph J, Michie S, Weal M, Wills G, Little P. Evaluation of a Web-based intervention providing tailored advice for self-management of minor respiratory symptoms: exploratory randomized controlled trial. J Med Internet Res 2010 Dec 15;12(4):e66 [FREE Full text] [CrossRef] [Medline]
    37. NHS England. NHS Choices. Friends and Family Test   URL: https://www.england.nhs.uk/ourwork/pe/fft/ [accessed 2017-01-08] [WebCite Cache]
    38. Howie JG, Heaney DJ, Maxwell M, Walker JJ. A comparison of a Patient Enablement Instrument (PEI) against two established satisfaction scales as an outcome measure of primary care consultations. Fam Pract 1998 Apr;15(2):165-171 [FREE Full text] [Medline]
    39. Ajzen I. The theory of planned behavior. Organ Behav Hum Dec Processes 1991 Dec;50(2):179-211. [CrossRef]
    40. Chew LD, Bradley KA, Boyko EJ. Brief questions to identify patients with inadequate health literacy. Fam Med 2004 Sep;36(8):588-594 [FREE Full text] [Medline]
    41. Arden-Close EJ, Smith E, Bradbury K, Morrison L, Dennison L, Michaelides D, et al. A visualization tool to analyse usage of web-based interventions: the example of positive online weight reduction (POWeR). JMIR Hum Factors 2015 May 19;2(1):e8 [FREE Full text] [CrossRef] [Medline]
    42. Geboers B, de Winter AF, Luten KA, Jansen CJ, Reijneveld SA. The association of health literacy with physical activity and nutritional behavior in older adults, and its social cognitive mediators. J Health Commun 2014;19 Suppl 2:61-76. [CrossRef] [Medline]
    43. Yardley L, Morrison LG, Andreou P, Joseph J, Little P. Understanding reactions to an internet-delivered health-care intervention: accommodating user preferences for information provision. BMC Med Inform Decis Mak 2010 Sep 17;10:52 [FREE Full text] [CrossRef] [Medline]
    44. Eysenbach G, Consort-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res 2011 Dec 31;13(4):e126 [FREE Full text] [CrossRef] [Medline]

    ‎
    CONSORT: Consolidated Standards of Reporting Trials
    IPAQ-SF: International Physical Activity Questionnaire-Short Form
    NHS: National Health Service

    Edited by G Eysenbach; submitted 06.09.16; peer-reviewed by B Davis, S Li; comments to author 04.10.16; revised version received 11.11.16; accepted 23.11.16; published 23.01.17

    Copyright

    ©Ingrid Muller, Alison Rowsell, Beth Stuart, Victoria Hayter, Paul Little, Kristin Ganahl, Gabriele Müller, Gerardine Doyle, Peter Chang, Courtney R Lyles, Don Nutbeam, Lucy Yardley. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 23.01.2017.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.