Evaluation of an eHealth Intervention in Chronic Care for Frail Older People: Why Adherence is the First Target

Introduction

Chronic care for frail older people is fragmented, with involvement from a large and constantly changing group of professionals who are frequently unaware that they provide care to the same patient [1]. Such professionals include home care professionals, general practitioners (GPs), clinicians, physiotherapists, and case managers dedicated to long-term care of the patients in the community. Frail elderly often suffer from comorbidities, which results in care by multiple health care professionals [2]. Therefore lack of communication between professionals leads to a fragmented and ineffective health care delivery for frail elderly [3]. To reduce fragmentation and promote continuity of care, better coordination and communication between professionals and with patients is necessary. Online health communities (OHCs) have been recognized as an effective mechanism for supporting continuous care for frail older people [4], allowing better coordination and more efficient communication with patients and among professionals. OHCs consist of Internet-based platforms that unite groups of individuals with a shared goal or similar interest, including both professionals and patients [5]. The main strength of OHCs is that they allow communication between people who would not have met each other otherwise [5]. Thus, OHCs are particularly suited for improving the coordination of care for frail elderly who have multiple professional caregivers. For this purpose, we developed and evaluated the Health and Welfare Information Portal (Zorg en Welzijns Informatie Portaal, ZWIP, in Dutch) [1,6] on its effectiveness.

Methods

Intervention

ZWIP is an OHC [5] that aims to facilitate communication for patients, their informal caregivers, and their professionals. ZWIP contains a secure messaging system supplemented by a shared electronic health record. All messages shared in a patient’s ZWIP are visible for all users, thus stimulating involvement of and discussion between patients and a team of health professionals. All informal caregivers and health care professionals have access to the electronic health record. To ensure confidentiality, professionals can participate in a patient’s personal care network in ZWIP only at the invitation of the patient. Patients who were not able to manage their own ZWIP account could appoint an informal caregiver to act on their behalf. Figure 1 demonstrates the conceptual model underlying ZWIP, and the video in Multimedia Appendix 1 illustrates the use of ZWIP by a patient and an informal caregiver.

‎View this figure

Development and Implementation

The development of ZWIP and the process of implementation have been described elsewhere [1,6]. In brief, ZWIP was developed using intervention mapping [7], a stepwise approach for the systematic development of interventions informed by both evidence and theory [1]. Main steps of intervention mapping for ZWIP were (1) needs assessment in frail elderly, (2) developing program objectives, (3) selecting theory informed intervention methods and strategies, (4) creating and pilot testing program components, (5) planning program adoption and implementation, and (6) planning for evaluation [1]. Theoretically, ZWIP was based on social cognitive theory [8], with special attention paid to improving self-efficacy, the belief people have in their ability to complete tasks and achieve specific goals [9]. Following the steps of intervention mapping and as suggested in the guideline on development and evaluation of complex interventions [8], the ZWIP was piloted by 2 frail elderly and 7 professionals, including one GP. Furthermore, newly developed elements of ZWIP were regularly piloted by similar user panels.

To enhance implementation of the ZWIP, we used several strategies for professionals such as a continuing medical education (CME) accredited education program based on active learning theory [10], direct experience, and modeling [1]. Additionally, drawing from organization theory, we installed a telephonic helpdesk and provided e-coaching and financial compensation to support the uptake of ZWIP by professionals [7,11]. To facilitate the use of ZWIP among elderly patients, a number of approaches were used: flyers were distributed in the primary care centers, a hard-copy version of ZWIP was provided in order for patients to familiarize themselves with ZWIP, coaching on the use of ZWIP was made available, involvement of informal caregivers was encouraged, and the GPs actively advocated the use of ZWIP, thus drawing on modeling, guided practice, and tailoring support for use of the intervention [1,6]. During the implementation phase, we designated one key person in each family practice who coordinated implementation activities and helped colleagues with questions [6].

Inclusion and Design

Between July 2010 and July 2011, frail older patients were included in an observational, controlled before-after study with 12 months follow-up to investigate ZWIP’s effects on patient outcomes. Participating primary care centers were recruited from the university primary care network around the city of Nijmegen, the Netherlands. These centers identified their frail older people using the EASYcare Two-step Older person Screening (TOS) instrument [12]. Therefore, both intervention and control practices had to implement an identification scheme and redesign care for their frail elderly. Interventions centers were selected based on willingness to participate in ZWIP, whereas control practices were selected from a separate project: the EASYcare-TOS validation study [13]{van Kempen, 2013 #7718}. Frail status as determined by the EASYcare-TOS was the only inclusion criterion for included patients. Patients in the intervention group patients needed to agree to the creation of a ZWIP account. No exclusion criteria were specified.

All measurements were performed by trained nurses in the patients’ homes, using a face-to-face questionnaire at baseline and at follow-up. The study was exempt from ethics review by the local ethics committee because of its observational nature and nonintrusive data collection. Nevertheless, oral informed consent was obtained to analyze the data during data collection.

Outcomes

Primary outcomes were Activities of Daily Living (ADL) as measured by the Katz index [14], combined ADL and Instrumental Activities of Daily Living (IADL) as measured by the Katz-15, a combined measure of the ADL and Lawton-index [14,15], SF-36 mental health and social activity limitation dimensions [16]. The Katz index consists of yes or no responses on ADL items such as bathing or dressing. ADL scores range from 0-6 with higher scores indicating higher dependency. The Katz-15 consists of yes or no responses on ADL and additional IADL items such as using the telephone and managing money [14]. The Katz-15 scores range from 0-15 with higher scores indicating more limitations. Both scales are established in the literature and have adequate reliability and validity [17]. The SF-36 mental health dimension, consisting of the following subscales: happy, calm, blue, down, nervous scoring from 0-5 with higher numbers indicating a higher score. The scores were summed into a summary score ranging from 0-100, with 100 indicating full mental health, and 0 low mental health [16]. To assess differences in social activity limitations, the social activity limitation item from the SF-36 was used [16]. This item measures the frequency in which respondents experienced social activity limitations due to health. The item used in this current study is scored from 0 (none of the time) to 5 (all of the time). The various SF-36 subscales have excellent reliability and validity [17]. Secondary outcomes were several self-developed scales of patient satisfaction and GPs’ subjective experience with care coordination. Patient satisfaction items were scored on a 5-point Likert scale ranging from 1 (way too little or way too much) to 5 (optimal), similar to this article [18]. GP experience with coordination of care was scored between 1 (uncoordinated) to 10 (optimal coordination). Important covariates were measured including a frailty index based on the accumulation of deficits concept [19,20]. The frailty index is the number of deficits present divided by a total possible number of deficits [2]. As such, the frailty index can account for all kinds of health-related imbalances between the intervention and control group and provides an accurate measure of individuals’ frailty.

Analysis

For comparing baseline characteristics, chi-square tests were used to compare nominal variables, and t tests were used for normally distributed continuous variables. Effects were determined using linear mixed models within a highly efficient analysis of covariance (ANCOVA) framework [21] to allow for clustering within a primary care center. Adjustments were made for frailty status and centered baseline status of the outcome variable and additional covariates with baseline imbalance. All analyses were performed with SAS 9.2.

Results

Overall, 290/622 (46.6%) of all frail persons identified within 11 practices participated in the intervention group. From 6 practices 392 frail older people participated in the control group. At 12-month follow-up, in the intervention group 179/290 (61.7% of original) patients provided data at follow-up, versus 270/392 (68.8% of original) patients in the control group. At baseline, participants in the intervention group were more likely to have completed primary education only, have more informal caregivers, and have higher complexity of care compared to the participants in the control group. Further, participants in the intervention groups also had a higher average frailty index score, and GPs had lower experience with coordination of care (Table 1).

One quarter 45/117 (25%) of all patients in the intervention group used ZWIP at least once a month during a period of 12 months. Controlling for frailty and other unbalanced baseline characteristics, we found no significant differences in primary patient outcomes (Table 2). Change in coordination of care as reported by GPs improved in the control group.

Discussion

Summary of Results

There were 290 patients who participated in the intervention group and 392 in the control group. In the intervention group 76/290 (26.2%) of the patients actively used ZWIP. After a follow-up of 12 months, we observed no significant improvement on primary patient outcomes, ADL, IADL, and mental health.

Strengths and Limitations

The online ZWIP platform was specifically developed for reducing fragmentation of care delivery in older people. Almost half of a frail elderly population without exclusion criteria could be included in the intervention group for using the online ZWIP tool [6]. This is modestly higher than what can be expected in the Dutch context, where 39% persons older than 75 years report having Internet access [22]. This study has two important limitations that can impact results. First, due to the observational nature of the study, comparability between the intervention and the control groups was limited. Despite adjusting for a range of covariates, there may be residual confounding.

Observational, controlled before-after designs are common for complex interventions, where randomized controlled trials (RCTs) are often not appropriate or feasible for evaluation [23]. In the case of ZWIP, contamination between patients would have made individual-level randomization inappropriate. Cluster randomization was not feasible because the level of commitment required from a number of local stakeholders could not be sustained in the control group.

A second limitation was the fact that actual usage of ZWIP was low, even though the implementation of ZWIP was prepared systematically during the development of ZWIP, as this is a structural part of intervention mapping [1,6,7,24,25]. Additionally, implementation strategies were added or adapted when needed during the actual implementation phase. A wide range of implementation strategies were used to encourage uptake; for example, a training program was developed for professionals and an active recruitment phase led to a high participation of older persons. Therefore, low levels of use were attained not because of the lack of, but despite using state of the art implementation techniques. Failure to integrate eHealth interventions in health care is widespread [26], and therefore the low levels of use of these frail older subjects is not surprising. This is especially true for sustained usage of an eHealth intervention [27]. As in other studies [26], further efforts should be focused on improving usability of the intervention, in terms of compatibility for frail older people in chronic disease trajectories [28].

Future Directions

In addition to further refinement, it is essential to identify those who benefit most from ZWIP and eHealth applications in general. The use of eHealth applications in frail populations could be increased by first identifying frail people with a high likelihood of early adoption of the eHealth intervention, such as people with high computer literacy. Which frail elderly are likely adopters requires further research [26]. Therefore, we plan to perform a quantitative and qualitative evaluation of ZWIP usage as well, going beyond the scope of this paper. We must recognize that in the early stages of evaluation, we take more of an efficacy approach to the evaluation, rather than a pragmatic trial approach. Although the efficacy approach limits generalizability, it allows a thorough investigation of the intervention’s working mechanisms under more controlled, laboratory-like conditions. Such work may also reveal ideal levels of use of ZWIP, as it is possible that communication was already adequate in the case of some patients, making ZWIP usage superfluous. Using both quantitative and qualitative methods in this development phase may elicit remaining barriers and reveal more effective implementation strategies. Only after adapting to this group and proven efficacy is large-scale implementation warranted. Successful wide-scale implementation is a precondition for investigating the effectiveness of eHealth interventions. Otherwise finding no differences between treatment arms cannot be interpreted as a lack of effectiveness. These arguments show that, sufficient time and resources are required to develop, test, and retest new eHealth interventions before finally evaluating their effectiveness in pragmatic trials [29,30].

Conclusions

Overall, the study confirmed that introducing eHealth interventions in the elderly is a difficult task. Despite using a theory-driven intervention design and state of the art implementation techniques, usage remained low and effectiveness was not observed. Performing a thorough proof of principle study in early adopters may be crucial to improving the use of eHealth interventions in the elderly before evaluating effects on a larger scale.