What Is eHealth (5): A Research Agenda for eHealth Through Stakeholder Consultation and Policy Context Review

Introduction

In 2002, the National Health Service (NHS) in England and Wales planned to invest over £2 billion in information and communication technology (ICT) [1]. This includes initiatives such as electronic patient records, electronic prescribing, the NHS Direct Telephone and Internet Service, and the National Electronic Library for Health [1,2]. Researchers from multiple disciplines in the UK and elsewhere had been investigating health informatics, but the NHS, despite its large investment in ICT, had not set a national research agenda for ICT.

A one-day conference in 2002 on Health Informatics Research and Development, sponsored by the research councils, Department of Health, and Department of Trade and Industry, concluded that the “lack of national strategy, capacity and career paths in health informatics have been weaknesses and remain threats to realising the informatics potential of the National Health Service…. [L]arge investment in the National Health Service and e-Science is unlikely to achieve its objectives without radical improvement in support for academic health informatics…. [This emphasizes the] importance of...clarifying the academic agenda for health informatics” [3]. The short- and medium-term challenges were seen as the following: (1) establishing the foundations of a knowledge infrastructure, (2) innovations in the clinician computer interface, (3) workable privacy protection, (4) more creation of knowledge from routinely collected data, and (5) finding the metrics of success for health informatics.

The NHS has three main national NHS research and development programs: Health Technology Assessment, New and Emerging Applications of Technology, and Service Delivery and Organisation (SDO). The SDO was launched on March 30, 2000, to consolidate and develop the evidence base on the organization, management, and delivery of health care services [4]. To respond to the needs of the “stakeholders” (service users, health professionals, and policy makers), the NHS, through its SDO research program, undertook an initial “listening exercise”[5] to produce a document outlining its overall priorities for research. It has continued to use this approach to develop and commission research [6]. It commissions a “scoping exercise” (normally a literature review and a stakeholder consultation) and then uses that in subsequent calls for proposals.

This study explored the concerns of professional and lay stakeholders regarding future developments of eHealth and reviewed relevant policy to produce recommendations for eHealth research. A parallel literature review was carried out by others and has been reported elsewhere [7-9].

Methods

Study Design

The study was reviewed and approved by the SouthWest Multi-Centre Research Ethics Committee and the University of Plymouth Faculty of Health and Social Work Ethics Committee. Data collection was carried out between November 2003 and June 2004. There were two parallel strands (Figure 1):

1. Stakeholder consultation: Focus groups and interviews with “professional” stakeholders generated themes that were subsequently prioritized by lay participants.
2. Policy context review: Policy makers were interviewed and policy documents were reviewed in order to produce recommendations that were subsequently reviewed in a conference workshop.

The themes and research questions arising from the stakeholder consultation and policy context review were compared, and recommendations from policy context were adapted to take account of stakeholder concerns. Diagrams were developed to bring together stakeholder and policy maker views of the scope of eHealth research.

‎View this figure

Stakeholder Consultation

In all, 12 groups (30 in each group) of professional stakeholders were contacted via email. Potential participants were identified by Web searches and by “snowballing” from existing contacts, trying to get geographical coverage within England. Stakeholders were defined as the following:

• NHS eHealth innovators and implementers
• University researchers in health informatics
• NHS staff in primary care
• NHS staff in secondary care
• NHS primary care trust managerial staff
• NHS acute trust managerial staff
• Suppliers of ICT to the NHS
• Professional organizations and royal colleges
• Informatics trainers
• Governance and other regulators
• Charities and other information providers
• Other NHS managers

These 360 people were sent an email inviting them to take part in the study, with a consent form to return to the researcher via email or post if they agreed to take part in the study. We asked professional participants to rank themselves on a four-point scale (from “I am pretty sceptical that eHealth will have any benefits at all” through to “I am very positive that eHealth can help improve the NHS if used appropriately”) as a rough guide to attitudes toward eHealth (Table 1).

A convenience sample of lay participants was recruited via snowballing from contacts in a local children’s nursery and from older friends of the research team. Potential participants were sent study information sheets and consent forms either via email or post. Two groups of older people and one group of parents took part in the study.

Scenarios depicting the current or future use of eHealth technologies were constructed to prompt discussion among the professional groups of the relevant themes regarding the use of eHealth technologies. Subject matter of the scenarios was developed from the content of news reports, informatics conference proceedings, and general Web searches. Both patient- and professional-centered scenarios were developed in order to achieve a balance of perspectives. The research team developed and discussed 32 scenarios: 15 were omitted, 7 were added, and a number were reworded to ensure neutrality in presentation.

In total, 24 scenarios (see Multimedia Appendix 1) were allocated to the 12 professional groups using a balanced incomplete block design [10]. Each group had four scenarios, and each scenario was used twice. A semistructured schedule based on the scenarios was constructed for use either as focus group or interview prompts. Some scenarios described patients being monitored by an implanted device that sent physiological information to hospital, a family doctor booking a hospital appointment during consultation in primary care, and a woman having an antenatal ultrasound in the community with expert diagnosis from abroad. Textbox 1 provides four examples of scenarios used as prompts in telephone focus groups. The topics covered in the scenarios included patients’ use of the Internet to order prescriptions, arrange doctor’s appointments, or join patient discussion forums. Other topics were about patients accessing their own health record, assessing the quality of a website, using a digital interactive television for a program on multiple sclerosis, or using a public access touch screen health information point.

Policy Context

A parallel, two-stage process was used to review the policy context. First, policy makers were interviewed and policy documents were identified and reviewed. The contents of documents and interview notes were categorized under English policies on ICT specific to health, English health policies influencing eHealth, nonhealth policies influencing eHealth, and European Union policies influencing eHealth. The reviewers focused on seven specific topics: (1) birth to death records, (2) country-wide access to quality health advice, (3) application of ICT to pharmacy, (4) telemedicine, (5) reduction of adverse incidents, (6) confidentiality, and (7) health data cards. In a second stage, 28 recommendations from the policy makers and documents were reviewed in a health informatics conference workshop by 60 participants using discussion and an interactive voting system. Participants scored each recommendation for relevance to the needs of the NHS (using a nine-point scale from “not relevant” to “highly relevant”). Recommendations with “middle scores” were discussed with the audience in more detail to obtain their views and decide if the recommendation needed to be worded more clearly.

Synthesis and Conceptual Mapping

Two members of the research team (RJ and LC) independently ranked the correspondence between the stakeholder concerns and the policy context recommendations on a scale of 1 (no correspondence) to 3 (strong correspondence). Analysis showed there to be strong agreement between the researchers. Stakeholder concerns, particularly of “technology meeting needs and improving health and quality of life,” were not consistently addressed by the policy context recommendations. Recommendations were adapted to take stakeholder concerns into account and were regrouped from “source-oriented” to “research-oriented” groupings. The new policy context recommendations were agreed upon between team members by telephone conference discussion. The two lists, one from stakeholder consultation and the other from the policy context review, were then reviewed again and similar areas of research were grouped. Reference was also made to the work of the Scottish Consumer Health Informatics Network [11]. We concluded that the scope of eHealth research could be described by a simple block diagram with four elements with linked areas of “best practice.” The recommendations were regrouped according to this “conceptual map.”

Results

Subjects and Sources

In total, 37 (10%) professional stakeholders consented to take part in the study and were consulted either via telephone focus group (25), telephone interview (6), videoconference (4), or in-person interview (2). There were 17 lay people (12 older people and 5 parents of young children) that took part in in-person focus groups. We interviewed 26 policy makers and identified and reviewed 95 policy documents.

Validity of Methods

One of the limitations of stakeholder consultation is that participants have to be sufficiently interested in the topic to take part. None of the 37 professional participants were “sceptical that eHealth will have any benefits at all,” and none thought “there could be some possible benefits to eHealth methods but on balance think that it is unlikely that the benefits will outweigh the costs.” On the other hand, 27 thought “that there are definitely benefits to eHealth but that we need to choose and develop methods carefully,” and 10 were “very positive that eHealth can help improve the NHS if used appropriately.”

Scope of eHealth

The research questions identified by stakeholders and policy review fell into six groups. Four of these (using, controlling, processing, sharing information) were used to describe the “scope” of eHealth. The other two groups of research questions fall under principles of research and development and stakeholder hierarchy (Figure 2).

‎View this figure

Principles of Research and Development

Both stakeholders and policy makers referred to many examples where, before innovative approaches are introduced, best practice procedures and barriers to implementation should be identified, and where professional and public stakeholders should be involved in research and development. A number of areas were suggested (Textbox 2).

Scope of New Research in eHealth

Using Information

Information is used in decision support, in the organization of services, for reassurance of professionals and patients, and in information-based therapies. Four research aims (Textbox 3) from the policy context review concerned the use of information in decision support.

Sharing Information

Both stakeholders and the policy context review identified a large number of research questions related to sharing information (Textbox 4). These included how information should be shared across sites (eg, between hospital and home), across sectors (eg, between social services and NHS), and between different professional (and patient) groups (eg, between doctors, nurses, dentists, patients).

Controlling Information

This group of concerns was ranked second most important by stakeholders. Ten research aims (Textbox 5) incorporated issues of control, accessibility, reliability, confidentiality, security, ownership, and regulation.

Processing Information

This covers a range of issues, including how best to present information (eg, should it be tailored for different users) and where it might be presented (eg, should it be sent to the user [push], or should it wait until the user seeks it [pull]). It also includes the coordinated integration of information derived from a variety of sources, as demonstrated in the electronic ordering and home delivery of medicines (Textbox 6).

Hierarchy of Stakeholder Concerns

The overriding concern of stakeholders was that spending money on eHealth should be worthwhile and should lead to improved health and quality of life. Particular research aims suggested by the data included “to review the costs and benefits of a range of recent eHealth applications, including the modelling of new forms of care made possible by ICT support,” and “to present those examples of eHealth applications, shown to have a demonstrable effect on improved health and quality of life, to professional and public stakeholders to obtain their views as to the nature of the most appropriate investment in eHealth.” In addition, the stakeholders identified themes concerned with controlling information (responsibility; reliability; regulation; accessibility; confidentiality, security, and ownership) as being particularly important, and so placed them on the second level of a hierarchy of concerns.

Discussion

In the context of the British health service, which is mostly free at the point of delivery, the overriding concern of stakeholders was that spending money on eHealth should be worthwhile and should lead to improved health and quality of life. At first this appears an unremarkable finding. However, although such an aim is part of the political rhetoric and may be an unstated assumption of policy documents in the United Kingdom, it is not often explicitly addressed in service development and use of information and communication technology. It is significant, for example, that the NHS recently funded a program originally called the National Programme for Information Technology, rather than a program for eHealth. (Subsequently, the program has been renamed Connecting for Health).

On the other hand, we know that doing research that can show a difference in health as a result of an eHealth intervention is difficult, partly because partial implementation of an e-booking system or a hospital information system is difficult. Gold standard randomized trials whose results can be generalized for widespread implementation are very difficult. In addition, to see changes in health or to measure cost benefit is slow and expensive, made more so now in the United Kingdom by the time needed for increased levels of ethical and research control and approval [12,13]. Thirty years ago, Blum noted that half the papers about computer applications concerned systems that were no longer operational [14]. We all know that ICT changes become ever more rapid. Research, therefore, has to be more about human behavior and how we use information and less about specific organizational or technological environments. It is essential to recognize the difficulties of addressing stakeholders’ needs by measuring change in health outcomes.

In some cases, modelling the longer-term costs and benefits (in terms of health) may be desirable. As systems continue to evolve, the health benefits may be seen not in the immediate change, but in a future evolution made possible by the initial change [15-17].

Many of the recommendations derived from both the stakeholder consultation and policy review confirmed the need to identify best practice and the barriers to implementation of that best practice. There are many examples of medical informatics research with demonstrable benefit which decades later still wait to be implemented more widely. For example, computers have been used successfully for patient interviews for nearly 30 years. Slack et al first reported on a computer-based medical history system in the New England Journal of Medicine in 1966 [18]. Yet, despite numerous research reports [19], the method has not been routinely adopted.

Rogers’ description of the diffusion of innovations [20] is well known, but stakeholders and policy makers want to see this process accelerated. The recommendations included, for example, identifying and exploring examples of effective informatics training for health professionals to see how these can be disseminated, or investigating working practices in other sectors (eg, e-business) to identify best practice and barriers to similar uses of ICT in the health sector.

In discussion, professional stakeholders often reverted to their role as patient or consumer rather than, for example, speaking as a supplier of ICT or from the point of view of primary care. Thus, although we sampled from all segments of the stakeholder population, there were no obvious differences between the different types of stakeholder. (Our sample was small, however, making our power to detect differences limited.)

The four categories which emerged as the grouped themes from the data are similar to classifications and descriptions found in textbooks of health informatics (eg, [21]). Blum, in a personal review of Medical Informatics in the United States, 1950-1975, presents a historical table of the Scope of Medical Computing from 1950 to 1980 using the three headings Data, Information, and Knowledge Applications [14]. He said, “Within a category, research begins only after the supporting technology is mature enough to support it beyond the conceptual level.” His table showed that he viewed data applications in the 1980s as refined, information applications as mature, and knowledge applications as prototype.

Four limitations of our study are the following: (1) We were not able to recruit people who were very sceptical about the potential of eHealth. However, we think that a more sceptical sample would be likely to have expressed similar concerns about improved health and value for money. (2) We did not have equal representation for the 12 predefined groups. However, we achieved coverage (although, in some cases, only one member; see Table 1) for all groups. Furthermore, as most participants often “reverted” to their role as patient in the discussion, our original idea that different professional groups might have certain biases and try to put forward ideas to their advantage seemed wrong (although numbers were small to detect any differences between groups). (3) Our lay sample was recruited only in the South East and the South West of England and did not include any people from ethnic minorities. We have no knowledge or hypotheses about how views may differ in groups not represented. (4) An eHealth agenda derived from policy review and stakeholder consultation in a country with a state-run health service may not transfer well to countries with private or insurance-based systems.

The purpose of our study was to produce a conceptual map and research agenda for eHealth based on stakeholder views and policy review. In a parallel SDO project, Pagliari et al [7-9] developed recommendations based on a review of existing academic and wider evidence sources, indicating the scope of the eHealth concept, the effectiveness of eHealth innovations, issues for implementation, and future directions for eHealth. Their research evidence is grouped by four broad technological categories:

1. Decision support tools for patients and clinicians
2. Networked digital technologies (Internet) used by patients (eg, for information, self-management, or peer support) and professionals (eg, for interprofessional communication, education, or communication with patients)
3. Computerized patient records, including issues relating to patient access and confidentiality and influences on health care delivery
4. Telemedicine and telecare

Their results are also interpreted with respect to the following broad content areas:

1. Specific research needs (evidence of effectiveness in specific areas)
2. Generic research needs (eg, methodological challenges to eHealth research, factors affecting implementation, effects on behavior and relationships, educational interventions, health inequalities, alternative delivery media, risks to the health service and society, role in self-care, and consumer empowerment)
3. Challenges for implementation (demonstrating impact, high-level support, strong project management, stakeholder engagement, the digital divide, ensuring credibility and quality, ethical, security and privacy issues, standards)
4. Emerging trends and future directions (eg. personalized and tailored systems, new technological advances for the information management and care facilitation, and delivery modes)

While differences in emphasis were expected due to the methodology of each study, the clear parallels between the results offer support for our recommendations. Our conceptual map, which has come from stakeholder discussion and policy review, also helps to put both our own and other detailed recommendations into a framework concerned with information and how we use it.