Web-Based Intervention for Women With Type 1 Diabetes in Pregnancy and Early Motherhood: Critical Analysis of Adherence to Technological Elements and Study Design

Introduction

Background

One important component of eHealth research is the evaluation of the use of information and communication technology. This was stated by Eysenbach more than 15 years ago [1] and further critically discussed, focusing on emerging Web-based interventions for patients with chronic illnesses. The issue of using randomized controlled trials (RCTs) as the “gold-standard” research methodology was also highlighted, as was the risk that the control group could become involved with other Web-based applications with similar objectives, thus “contaminating” the design [2]. Since then, the global use of mobile phones, the Internet, and a plethora of related technologies and applications has increased exponentially, and numerous interventions for promoting health and health-related behaviors have been carried out.

Thus, applying a critical perspective to Web-based interventions for persons with long-term and chronic illnesses is of specific importance. However, several methodological challenges exist, for example, issues of recruitment, randomization, fidelity, retention data quality, and degree of adherence [3]. Another matter is how the used technology and its design influence a Web-based intervention. To investigate whether intervention characteristics and persuasive design affect adherence, 83 Web-based health interventions were reviewed by Kelders et al [4]. Through the use of a framework for Persuasive System Design (PSD) [5] and its design principles, they coded persuasive technology elements and analyzed intervention characteristics and adherence. In a multiple regression analysis, they found that users’ adherence to an intervention was predicted by differences in technology and interaction with a counselor. Providing persuasive technology for social support did not affect adherence, although the need to further explore this was stressed [4].

Objectives

In this paper, we further critically analyze and discuss the challenges of conducting an RCT using Web-based technology, including devices for social support, with special focus on adherence. According to Kelders et al, adherence is defined as “the intended usage in line with the therapeutic regime” [4] in which social support is a prominent component. The basis is a recently finalized RCT in which Web-based support was offered to women with type 1 diabetes mellitus (T1DM) in pregnancy and early motherhood: the MODIAB-Web (MOtherhood and DIABetes) study.

The following research questions have been elaborated upon in this paper:

• What persuasive content and technological elements were used in the design of the Web-based support?
• How was the Web-based support used?
• What was the content of the social peer support and how did the peers support each other?
• What were the main challenges in relation to adherence to the RCT?

Methods

Research Design

Using an exploratory case study methodology [6] and the framework for analysis offered through the PSD model [5], we have critically examined the MODIAB-Web study. The analytical focus was on technology and its design, as well as the usage of the different parts of the Web-based support with special focus on the forum for peer support.

The Motherhood and Diabetes (MODIAB)-Web Study

The MODIAB-Web RCT study was directed at women with T1DM registered at 6 hospital-based antenatal care units in Sweden. Complementary Web-based support was offered to the intervention group, with support starting in early pregnancy and persisting up to 6 months after the child was born [7,8]. The rationale for the study was that T1DM women face particular demands in relation to pregnancy, childbirth, and early motherhood. During pregnancy, they struggle to achieve normal blood glucose levels, which optimize the probability of giving birth to a healthy child [9-11]. In the early months after childbirth, the women have to deal with unstable blood glucose levels, whereas at the same time having to meet all the challenges of early motherhood, and especially those related to breastfeeding [9,12,13]. Their need of Web-based support including social peer support has been identified in earlier studies [14-16].

The Web-based support was developed in line with the ideal of participatory design [17] and includes three parts: (1) information based on scientific evidence, (2) a self-care diary, and (3) a forum for peer support divided into three topics: pregnancy, childbirth, and life as a new mother. In addition, the Web-based support comprised a frequently asked question section where participants could ask questions and receive answers from experts in the field, and a collection of links to other recommended resources. The intention was to offer complementary Web-based support as an add-on to regular pregnancy care and use it especially after childbirth to fill the gap of healthcare support between the different healthcare providers, that is, the maternity care professionals and the diabetes care professionals. If they wished, the women could share their self-care diaries with healthcare professionals during their visits, as the diaries were part of the Web-support structure [7]. The hypothesis was that the Web-based support should strengthen the women’s personal capacity and autonomy, thereby leading to improved self-management of diabetes and overall well-being. Ethical approval was attained from the Ethics Committee of Gothenburg, Sweden (No. 659-09), and the trial was registered at clinicaltrials.gov (ID: NCT015665824). Eligible study participants were successively recruited by an appointed study midwife in early pregnancy. The study was performed over a period of more than 4 years. The first study participant was included on November 22, 2011, and the last participant ended participation on January 25, 2016, that is, 6 months after the last childbirth. Further details about the study design are described elsewhere [7,8].

The findings showed that the Web-based support plus standard care was not superior to standard care alone in terms of general well-being and self-efficacy of diabetes management. Details on this are reported elsewhere [18].

Analysis

Research question 1, “what persuasive content and technological elements were used in the design of the Web-based support?,” has been answered using the PSD model for coding design principles in different categories, each one comprising 7 design principles. The categories are: primary task support, dialogue support, system credibility support, and social support [5]. A deductive analysis consisted of identifying and describing which PSD principles and technological elements were used in the MODIAB-Web intervention. Research question 2, “how was the Web-based support used?”, was answered using descriptive statistics and qualitative content analysis [19]. Research question 3, “the content of the social peer support and how the peers supported each other”, was answered using descriptive statistics and qualitative content analysis [19]. Initially, the forum posts were read as a whole several times to get an overall sense of the data. In the next step, new readings followed in which the data were organized. This process included open coding, in which notes and headings were written in the margins. The headings were transferred to a separate coding sheet and were grouped into categories. The categories were presented under the preset topics of pregnancy, childbirth, and life as a new mother. The quotations used to illustrate the dialogue in the forum for peer support were professionally translated into English. The analysis was performed by MB in close collaboration with AA. Research question 4, “what were the main challenges in relation to adherence to the RCT?”, was answered through a critical analysis based on the results of the first three research questions and the intervention as a whole.

Results

Persuasive Content and Technological Elements in the Design of the Motherhood and Diabetes -Web Support

The implemented categories with used design principles [5] in the MODIAB-Web study are presented in Table 1. In the category Primary Task Support the focus is on providing technological elements to manage the targeted behavior [5]. The MODIAB-Web study implemented 3 out of 7 design principles: tailoring, reduction, and self-monitoring, whereas tunneling, personalization, simulation, and rehearsal were not used. In Dialogue Support the focus is on providing various kinds of feedback between the human and the system [5]. In the MODIAB-Web study reminders, liking and social role were implemented, whereas praise, rewards, suggestions, and similarity were not. In System Credibility Support, the design principles aim to increase credibility and consequently to be more persuasive [5]. Five design principles were used in the MODIAB-Web study: trustworthiness, expertise, surface credibility, real-world feel, and authority, whereas third-party endorsement and variability were not. The category Social Support motivates usage by leveraging social influence [5]. The MODIAB-Web study implemented the principles social learning and social comparison while the principles normative influence, social facilitation, cooperation, competition, and recognition were not used.

Use of the Motherhood and Diabetes Web-Based Support

In total, 83 women were randomized to the MODIAB Web-based support, and of these, 81 received a log-in. As the women were successively randomized to either control group or intervention, and as the intervention group could use the Web-based support from randomization to 6 months after birth of the child, a maximum of 36 women could be active simultaneously. This is illustrated in Figure 1.

Of the 81 women with log-in, 69 women were classified as “active users,” that is, they had logged in at least once after the introduction session. How the participants used the Web-based support is presented in Table 2.

The peer support forum was used by 63 women of whom one was a facilitator appointed by the researchers in the first year of the intervention. This facilitator was an experienced mother with T1DM. After that period, it was presumed that the included women could collectively run their activities. The forum was moderated by a member of the research team to have some kind of control over what was discussed and to make sure it did not include advice that was contrary to current scientific evidence. Almost all forum activity occurred in the first 3 years (2011-2014). Of the 63 forum users, 39 participants were readers and 24 were active writers including the study facilitator.

Forum Content and Type of Peer Support

There were 109 written posts (range: 1-20) divided into 19 threads, of which 84 posts (16 threads) were related to the topic pregnancy, 15 posts (1 thread) to childbirth, and 10 posts (2 threads) to life as a new mother. No inappropriate advice was given during the study period, so the moderator did not have to act. Some women who asked questions in their posts had to wait some time before receiving an answer, while others did not receive an answer at all. This happened after the study facilitator had left the forum.

Forum Content

The design of the forum for peer support was intended to help users sort experiences in relation to the different phases of pregnancy, childbirth, and life as a new mother. To some extent, it became apparent that the topics overlapped in each category.

‎View this figure

Pregnancy

This topic had the largest number of posts. Its content related to overly focusing on diabetes as a disease; the first ultrasound; how to handle glycemia and specifically hypoglycemic episodes; insulin dosages, and the estimated weight of the expectant child. Minor discussions concerned managing attendance at the many checkups combined with working and how to plan for parenthood after the birth of the child, including how to get in contact with the Swedish social insurance agency for the parental benefit.

Childbirth

The posts related to inducing labor and labor routines, including how the insulin was administered and if an insulin pump was permitted during labor. Other posts discussed wounds and wound healing. One woman who had given birth 3 times shared her story of all these childbirths, one having been vaginal, one vaginal vacuum extraction, and one elective cesarean section.

Life as a New Mother

Participants shared posts on this topic for a period of only 6 months (December 2011 to July 2012). Content mostly concerned postpartum care at the hospital—that there was insufficient professional support or insufficient knowledge about diabetes. Another main concern was breastfeeding-related issues. One woman expressed her disappointment that the ward ignored her wish to avoid giving her baby cows’ milk-based nutrition supplements. Another woman concluded that in her situation with diabetes, it was best to combine breastfeeding and a milk supplement. One woman asked for experiences of breastfeeding and how to integrate it in daily life as a mother with diabetes. Unfortunately, no one responded. In addition, there were many diverse posts, for example, about a baby who had got an infection, and another baby who had a clavicle fracture. Several questions were asked, for example, did you get preeclampsia? What type of pain relief did you choose? One woman, who was active at the beginning of the intervention period (December 2012), advertised her own blog and its weblink.

Type of Support

The analysis of the peer support in the forum shows that, although there were few participants there was a rich exchange between them. The dialogues comprised sharing of personal experiences, giving concrete advice, affirmations, and words of reassurance. Examples of two dialogues between the participating women are given in Textbox 1.

What Were the Main Challenges in Relation to Adherence to the Web-Based Randomized Controlled Trial?

This section comprises a critical analysis of both significant and minor issues in relation to adherence (defined as “the intended usage in line with the therapeutic regime” [4]) to the Web-based intervention. It is also based on the fact that the presumed effect of the RCT, in terms of increased well-being and diabetes management, was not achieved [18]. The MODIAB-Web study was typically designed according to the gold standard of an RCT [22]. How did this come about and why?

Adherence in Relation to Study Design

There were few women available to include in this study. In Sweden, around 0.5% of all pregnant women have T1DM, corresponding to 500 per year. At the beginning, two study centers were included based on an estimate of few eligible women. It became apparent that the recruitment rate of study participants was slower than expected. In addition, four study centers were included to avoid a more prolonged data collection period. Furthermore, the contact frequency between the researchers and the healthcare professionals at the study centers was increased. Prolonging clinical trials because of slow recruitment pace and retention of study participants is a common issue. One reason for this might be poor engagement from the healthcare professionals who are inviting the patients to take part in the study [23].

The slow inclusion pace prolonged the study duration, which in turn meant only a few women (about 30) could be active at the same time (Figure 1). Thus, the critical mass of simultaneous active users was not sufficient and undermined the extent of interactions in the Forum. It has been concluded that many of the participants in a Web-based study may be lost to follow-up or end up not adhering to the intervention [2]. This was not taken into account when calculating the sample size. One way to get around this could have been to have started the study at more study centers from the beginning to gain a greater critical mass faster.

The use of a facilitator who was active in the peer support forum the first year was fruitful, as it increased the dialogues in the peer support forum. A shortfall in the study design was that the facilitator was not engaged during the whole study period, which led to the activity level in the forum dropping.

The quality of the support shared between the participants in the MODIAB-Web study was reasonably good, as exemplified in the dialogues (Textbox 1). The active writers in the forum for peer support coached each other and shared their experiences in a generous way. The majority of participants were “readers”; they did not actively participate in the dialogues. However, in line with recent research, these are probably also “passive actors”—meaning only readers can receive social support related to their specific needs [24].

The forum user who advertised the weblink to her blog in the MODIAB-Web study at least potentially created an alternative Web-based support. This makes it hard to truly evaluate the usefulness of the intervention, as meaningful interaction might happen outside the study platform. It also proposes a threat to adherence to the intervention, if participants choose to communicate in alternative ways, such as through social media. It is almost impossible to control for this, and it is a methodological weakness in studies evaluating Web-based social support.

A main problem arising due to the prolonged duration of the MODIAB-Web RCT study was the simultaneous explosion of more advanced general mobile phone technology [24] as well as similar technology in diabetes care contexts. The developed Web-support serving as a platform for self-management consequently became outdated with time as few available alternatives grew.

Adherence in Relation to Technology

With regard to the PSD model [5], four categories were identified in the MODIAB-Web intervention: Primary Task Support, Dialogue Support, System Credibility Support, and Social Support (see Table 1). This shows that several but not all of the design principles were implemented.

To accomplish adherence, Kelder et al argue that it is essential to actually plan for adherence when designing Web-based interventions [4]. The MODIAB-Web intervention was developed using participatory design [17] and grounded on previous empirical and theoretical investigations of the needs of the user groups, as well as ongoing user participation of lay and professional experts [7]. However, there was no theoretical basis for the technological design or the use of design principles of a more general kind, such as the PSD model. A theoretical grounding would have generated several advantages. One prominent advantage is insights into a greater repertoire of design elements than those available through participating lay and professional experts. In this manner, design is based on practical and theoretical knowledge outside of the local project.

Moreover, a theoretical grounding such as the PSD model was launched in 2009, meaning that it existed well before the investigations of general user needs [14] and the design of the MODIAB-Web intervention in particular [7]. However, it must be recognized that the recent increasing importance and subtleties of research processes focusing on design to accomplish behavior change are still treated in specific research fields such as design studies [25]. A theoretical grounding when it comes to behavior change itself would enable a repertoire of more passive and active strategies for influencing people that, in turn, should be brought into the technology design part of the intervention. Such strategies might involve using models of how to accomplish a feeling of social connectedness [26], which is relevant in interventions aiming to increase the social support between peers. A theoretically more conscious design might be an alternative to the recent suggestion of generating predictive models of potential dropouts in online health communities by Big Data analysis to inform design [27].

The use of design theory in the analysis of the MODIAB-Web intervention enabled a discussion of the rationale behind its emphasis on System Credibility Support and its significantly less emphasis on Dialogue Support and Social Support. Quite likely, potential was lost in unutilized design principles. For example, in the Dialogue Support category, the design principle “Rewards” that aims to give credit for performing the target behavior [5], that is, using the Web-support, could have been implemented to increase adherence. For instance, this could entail rewarding participants once they had used the support 5 times, or when at least 5 other participants had read a person’s posts in the forum.

A further technical element related to Social Support, not used in the MODIAB-Web forum for peer support, was “flagging” of new posts. Such flagging had probably increased the participants’ “adherence,” in terms of active communication. Another issue that probably reduced adherence was that the forum was divided into 3 parts. This division meant, there was a risk, that new posts were not seen by the other participants. Another shortfall was that the participants were not obliged to write a post to introduce themselves upon first entering the peer support forum. Admittedly, certain PSD design principles related to Social Support would have been positive in relation to adherence, for example, “Recognition,” which would include success stories. Other design principles, such as “Normative influence,” might be less relevant in this context due to the extreme demand for optimal diabetes control for childbearing women.

Discussion

In this paper, we used case study methodology to enable a critical discussion of RCTs involving Web-based support and adherence in general, and social support in particular. In other words, this methodology has been used as a basis for our critical endeavor in line with Baxter and Jack, who state: “it enables the researcher to answer ‘how’ and ‘why’ type questions, while taking into consideration how a phenomenon is influenced by the context within which it is situated” [6].

In this paper adherence is, in line with Kelders et al [4], defined as “use in accordance with identified intentions,” and in which the need for relevant information about the specific condition as well as social support from peers is emphasized. Our analysis supports the conclusion that PSD does matter in general, by positively influencing adherence [4]. We argue that there is a particular value in basing technology design on theoretical design principles, as it increases the repertoire of options, thus enabling adaption to suit the target population for the intervention and creating opportunities to utilize the full design potential of social support. The value of applying theory ties in with recent acknowledgment of theoretical consciousness regarding adaptation and implementation of health information technology innovations [28]. When it comes to health information technology such as Web-based support, we argue that theories related to technology use and design, such as the ones discussed above, are equally important. This notwithstanding, the essential rationale of Web-based health interventions remains the same but becomes significantly better theoretically informed. A systematic review published post completion of our analysis showed that justifications for intended use, and adherence to eHealth technology is often underdeveloped and improperly used. The authors believe that adherence can be standardized, and this will improve comparison of adherence rates to different technologies with the same goals [29]. Our critical analysis thus contributes to ongoing discussion about adherence.

Moreover, we conclude that study design matters and that technology and study design might mutually influence each other. With a limited target group, the comparative element that constitutes an RCT design becomes a barrier to achieving a critical mass of user interactions. Extended project periods mean that the used technology must either be changed or tailored according to new technical options that become available during the process, which in turn might jeopardize study validity. Not doing so implies a risk that the technology will become somewhat outdated in the world of the users. In relation to providing social support, a further conclusion is that using theory enables a multifaceted repertoire in technology design. However, we also note that apparently low user activity might still provide social support for the individual, irrespective of whether they are an active or passive user.

On the basis of critical analysis of how study design and technology interacts, we recommend fellow researchers to consider the following aspects when designing and evaluating Web-based health interventions:

• When designing a Web-based intervention, use existing design theories to utilize the full potential of the technology and increase adherence, especially with regard to social support.
• Be realistic when calculating your sample size. Take the risk of losing participants and poor adherence to the intervention into account.
• Be aware of limited target populations. If the number of possible participants is low, consider starting the intervention in multiple study centers simultaneously to avoid prolonged study periods and thereby outdated technology.
• Bear in mind that the majority of participants will not actively engage in discussions in peer support forums. Therefore, a larger critical mass of participants is needed in interventions based on effects of social support.
• Keep the design of the forum for peer support simple and avoid divisions. Web-design is often costly if the researchers do not possess the skills themselves or team up with researchers from other fields. Consider integrating your intervention into existing social media (but be aware of ethical pitfalls if you cannot guarantee the security and confidentiality of data).
• If your intervention consists of a forum for peer support, consider engaging a study facilitator from the target group for the entire study period to boost activity.
• Health interventions are often complex in their nature. RCT design is one way of evaluating the effects of an intervention but it is in itself inadequate for truly capturing complex interactions. Design your study with multiple methods of analysis and consider the contamination of the control group as technology evolves during the study period.
• Adjust per-protocol criteria after intended usage or conduct a dose-response analysis to properly evaluate the effect of the intervention within the RCT design.

Finally, we want to return to the initial critical questioning of whether the RCT design really is appropriate as the gold standard for Web-based interventions [2]. Our analysis does not fully answer this question, nor does it reveal what could be a better research design, but we would like to encourage further constructive discussion on these issues within the scientific community.