Review
Abstract
Background: Theory-guided approaches to implementation science have informed translation efforts and the acceptance of eHealth (digital health) interventions in clinical care. However, there is scarce evidence on which theories are best suited to addressing the inherent complexity of eHealth implementation.
Objective: The objectives of this systematic review are to identify theories that inform and explain eHealth implementation and to classify these theories using the typology by Sovacool and Hess for theories of sociotechnical change.
Methods: An electronic search was conducted in the PsycINFO, MEDLINE, Embase, CINAHL, Scopus, Sociological Source Ultimate, Web of Science, ABI/INFORM, EBSCO, and ProQuest databases in June 2019. Studies were included if they were published between 2009 and June 2019; were written in English; reported on empirical research, regardless of study or publication type; reported on one or more theories in the context of eHealth implementation; and were published in a peer-reviewed journal. A total of 2 reviewers independently assessed the titles, abstracts, and full texts. Theories identified were classified using a typology for theories of sociotechnical change, which was considered a useful tool for ordering and analyzing the diverse theoretical approaches as a basis for future theory building.
Results: Of the 13,101 potentially relevant titles, 119 studies were included. The review identified 36 theories used to explain implementation approaches in eHealth. The most commonly used approaches were the Technology Acceptance Model (TAM) (n=33) and the Unified Theory of Acceptance and Use of Technology (UTAUT) (n=32). These theories were primarily concerned with individual and interpersonal elements of eHealth acceptance. Less common were theories that reflect the various disorderly social processes and structural dimensions of implementation, such as the normalization process theory (n=17) and the structuration theory (n=6).
Conclusions: Theories currently informing the implementation of eHealth interventions predominantly focus on predicting or explaining end-user acceptance. Theoretical perspectives that capture the dense and intricate relationships and structures required to enact sustainable change are less well represented in the eHealth literature. Given the growing acknowledgment of the inherent complexity of eHealth implementation, future research should develop and test models that recognize and reflect the multidimensional, dynamic, and relational nature of this process.
doi:10.2196/18500
Keywords
Introduction
In recent years, technological innovation in health care has developed exponentially, and eHealth is now widely viewed as a significant potential contributor to improved quality of care [
, ]. However, despite much policy-level and scholarly discussions of triggering a revolution in health service delivery, problems of implementation and uptake of eHealth among both patients and service providers persist [ , , ].Poor uptake of eHealth (a term with contested definitions [
] but, broadly speaking, “health services and information delivered or enhanced through the internet and related technologies” [ ]) is often explained in terms of barriers and facilitators [ ]. In a recent study, Schreiweis et al [ ] identified 77 barriers and 292 facilitators in implementing eHealth services. Similarly, a systematic review by Granja et al [ ] identified 27 factors that determine the success or failure of eHealth interventions. Although studies about barriers and facilitators are important, they tend to fall short of capturing the complexity of the implementation process and the multiple interrelated factors that determine the translation and uptake of eHealth [ , ].Evidence suggests that theory-informed approaches to implementation science can enhance the translation and acceptance of eHealth into clinical care [
, - ]. Theories offer explanatory frameworks and formal heuristic devices that have the potential to move beyond the basic listing of individual facilitators and barriers to implementation, to capture the dynamic interaction between them [ ]. As Damschroder [ ] notes, theory “enables knowledge to emerge out of seeming chaos,” facilitating exploration of complex relationships and interdependencies between variables that unfold in diverse and changing contexts [ ]. This is of paramount importance in eHealth settings [ , ], which are characterized by a complicated interplay between patients, clinicians, the health care system, and the eHealth technology.Many theories and models have been articulated to inform and explain eHealth implementation [
]. Despite this abundance, findings from several reviews show that only a small number of select theories have been used repeatedly across multiple publications and by several authors [ - ]. For example, a recent review by Harst et al [ ] of 24 studies of end-user acceptance of telemedicine found that 2 theories accounted for 20 instances of theory use: the technology acceptance model (TAM) and the unified theory of acceptance and use of technology (UTAUT). Similarly, a review on the use of theory in eHealth weight management interventions by Willmott [ ] identified 18 studies referencing a theory, of which 16 mentioned either the social cognitive theory or the transtheoretical model.Theories most commonly used in the literature tend to emphasize individual factors, such as motivation, attitudes, and behavior, rather than the broader social and environmental factors impacting implementation [
, , ]. This is despite evidence highlighting the multilevel nature of technology implementation in health care and the importance of targeting variables at different levels [ , ]. As Glanz and Bishop [ ] noted, social and environmental factors may constrain individuals’ behavior even when they are highly motivated. Therefore, the authors recommend complementing individually oriented theories with theories of social, policy, or organizational change [ ].One hindrance to this is that the current eHealth implementation literature is fragmented across multiple specialty areas and disciplines, making it difficult to locate the range of theories available [
]. To improve the selection and application of theory, it is necessary to identify an array of theories, across diverse disciplines, that have the potential to inform eHealth implementation. A further issue is that many theories contain overlapping constructs but use different terms to describe them [ ]. Synthesizing theories according to their similarities would facilitate their selection and application at different levels [ ].To address these issues, we conducted a systematic review and classification of eHealth implementation theories. The review aims to address the following question: “What theories exist across disciplines that have been used to inform or explain eHealth implementation?” Theories identified by our review were classified using the typology by Sovacool and Hess [
] for theories of sociotechnical change. This typology provides an accessible and useful framework for organizing and selecting diverse theoretical options that target variables at different levels. Its use also allows the identification of areas where further theoretical development is required.Methods
Overview
This systematic review was conducted by members of the review team in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [
]. A PRISMA checklist is available in [ ]. The authors adopted a flexible approach by continuing to apply the core principles of systematic review methodology but tailoring the PRISMA guidelines to the needs of this review [ ]. As such, formal quality assessment was not conducted for this review, as the perceived validity or trustworthiness of the included studies did not address the overall research question, which sought to identify the existence of theories across a broad and varied body of literature.Search Strategy
Electronic searches of PsycINFO, MEDLINE, Embase, CINAHL, Scopus, Sociological Source Ultimate, Web of Science, ABI/INOFORM, EBSCO, and ProQuest Databases were conducted by the review team in June 2019 to identify studies that applied one or more theoretical frameworks to inform eHealth implementation. For this review, implementation was defined as “the scientific study of methods to promote the systematic uptake of research findings and other EBPs (evidence-based practices) into routine practice, and, hence, to improve the quality and effectiveness of health services.” [
]. These databases were chosen because they were deemed to be likely to catalog studies and disciplines relevant to the eHealth context and the specific research question. The search was limited to studies published in the last 10 years (from 2009 to June 2019) and yielded 21,704 abstracts for initial consideration. A full list of key search terms used can be found in . All records were converted into an EndNote library and reduced to 13,101 following deduplication. Papers were then title-checked for relevance to the topic and research questions and aims before further screening by 2 independent reviewers (MH and HW) in accordance with the detailed inclusion and exclusion criteria outlined below.Eligibility Criteria
Individual studies were included in the review if they were (1) published in the last 10 years (from 2009 to June 2019), (2) published in English, (3) outputs of empirical research or theoretical papers reporting on one or more theories in the context of eHealth implementation (this included all study types and populations), or (4) published in a peer-reviewed journal. Studies were excluded if they were (1) published before 2009, (2) not written in English, (3) studies that did not report on one or more theories applied in the context of eHealth implementation, (4) gray literature not published in a peer-reviewed journal, (5) dissertations, theses, conference proceedings, or abstracts, or (6) any form of literature review. The full eligibility criteria for this review are provided in
.Eligibility criteria for the review.
Inclusion criteria
- Publication date from 2009 (inclusive) to June 2019
- Australian and international literature in English language
- Papers reporting on one or more theories in the context of eHealth implementation (any study type and population)
- Empirical studies (both quantitative and qualitative)
- Position, discussion, or theoretical papers
- Peer-reviewed articles
Exclusion criteria
- Publication before 2009
- Literature in non-English language
- Papers not reporting on one or more theories in the context of eHealth implementation
- Gray literature or not published in a peer-reviewed journal
- Dissertations or theses or conference proceedings or abstracts
- Literature reviews (narrative, scoping, and systematic)
Identification and Selection of Studies
A total of 2 reviewers (MH and HW) independently applied the predefined inclusion and exclusion criteria to screen for relevant studies from those obtained through database searching. To ensure accuracy, record titles and abstracts were screened manually in EndNote, and documents that did not meet the selection criteria outlined above were excluded by the reviewers. Following 2 rigorous rounds of title and abstract screening, full texts of all potentially eligible studies were examined and further screened by the 2 independent reviewers (MH and HW) using the Covidence web-based software (Veritas Health Innovation Ltd), an effective tool for assisting research teams when performing systematic reviews or meta-analyses [
]. Articles that failed to meet the selection criteria were excluded and then cross-checked to ensure transparency and accuracy surrounding the reasons given for exclusion. Any conflicts in decision making during the screening phase were resolved via discussion between reviewers or, if needed, with the research coordinator (FKL) until consensus was reached.Data Extraction and Presentation
As the standardized extraction tool in Covidence did not meet the specific needs of this review, a modified extraction form was developed and piloted by the 2 reviewers (MH and HW) with 10 included studies tabulated and refined accordingly. The modified extraction form was tailored to include characteristics relevant to the research question. The characteristics extracted by the reviewers included (1) name of theory, (2) description, (3) instances of theory use, (4) examples of theory application, and (5) theory type. Instances of theory use refer to the number of occurrences in which a theory was used. As several studies used more than one theory, the total number of theory instances exceeded the number of papers included in the review. Examples of theory application were drawn from the literature to specify how each theory informed eHealth implementation. The reviewers then determined each theory type by drawing on the typology by Sovacool and Hess [
] for theories of sociotechnical change. This typology categorizes theories according to where they tend to center their analysis. The term center is intended to convey that a theory may involve elements of multiple types but that it approximates one ideal type above all. This typology was considered a useful tool for ordering and analyzing the diverse theoretical approaches identified, as a basis for future theory building [ ].The typology includes 5 categories: agency, structure, relations, meaning, and norms. Agency-centered theories relate to people’s individual actions, beliefs, and attitudes, and assume that these can be explained without deeper consideration of broader social and systemic elements [
, , ]. In contrast, structural theories propose that people are influenced largely by external forces beyond their control, such as their organizational, political, or macrosocial environments [ , ]. Relational theories attempt to interpret the interactional processes that influence the circulation of knowledge throughout different social networks. They view technology and society as coproduced and coconstructed, with no single dimension creating change by itself [ , ]. Meaning-centered theories focus on language, ideas, symbolism, narratives, rhetorical visions, and other cognitive dimensions that both orient action and are changed by it. Normative theories offer criteria by which to assess the positive or negative impact of technology on society or on a specific group. A sixth category, combined theories, was added to these 5 categories. This included meta-theories that explored a combination of individual, structural, or relational frameworks. All authors (MH, JW, JC, HW, CT, and FKL) reviewed and agreed upon the classification of theories using this typology.Results
Search Results
The electronic search of key databases resulted in 21,704 potentially eligible articles (
). This number was reduced to 13,101, following deduplication. Of these, 12,001 papers were excluded based on title screening and application of the eligibility criteria previously outlined. Key reasons for exclusion of papers at title screening included eliminating those that were in non-English language or those that reported on an irrelevant topic to the research question, for example, non-eHealth or theory-related papers. The abstracts of the remaining 1100 papers were then independently screened by reapplying the inclusion and exclusion criteria, and a further 935 papers were excluded. Key reasons for exclusion at abstract screening included nonempirical or gray literature and papers that reported abstracts or protocols only. Following a full-text review of the remaining 165 articles, an additional 46 articles were excluded because of insufficient reporting on or mention of theories related to eHealth implementation. In total, 119 articles met the full, predefined eligibility criteria and were included for data extraction and synthesis of findings. The PRISMA flowchart in details the process of eligibility and study selection.Theory Summary and Classification
The summary (including theory name, description, instances of theory use, and examples of application to implementation) and classification of all theories used to inform and explain eHealth implementation is provided in
. In total, 36 distinct theories were identified. Classification of these theories using the typology by Sovacool and Hess [ ] showed that the theories used in the literature were predominantly agency centered (19/36, 53%), followed by relational (7/36, 20%), structural (6/36, 16%), meaning (3/36, 8%), and combined theory types (1/36, 4%). No normative theories were identified.Theory | Description | Instances of theory use | Examples of theory application | Type |
TAMa | Proposes that technology acceptance and use are affected by an individual’s perceived ease of use, perceived usefulness, and subjective norms | 33 |
| Agency centered |
UTAUTb | Proposes that behavioral intention to use eHealth interventions is affected by individual effort expectancy, performance expectancy, social influence, facilitating conditions, and habit | 32 |
| Agency centered |
NPTc | Explains the social processes (eg, coherence, participation, collective action, and reflexive monitoring) through which eHealth interventions are operationalized | 17 |
| Relational |
DOId theory | Explains how an eHealth innovation gains momentum and diffuses through a specific population. This process is affected by the innovation itself, time, channels of communication, and an individual’s social system | 16 |
| Agency centered |
Structuration theory | Models the relationship between agency and structure. In eHealth implementation, interventions are configured and coconstructed over time and can be adapted or revised to better accommodate different settings and needs | 6 |
| Relational |
CFIRf | A meta-theoretical framework that provides an overarching typology of constructs relating to implementation, including intervention characteristics, outer setting, inner setting, and individual characteristics | 6 |
| Combined |
ANTg | Posits that objects have agency and that a combination of network components (both human and nonhuman) helps create and/or influence social effects (such as implementation) | 4 |
| Relational |
TPBh | Extends the theory of reasoned action to incorporate actors’ perceived control over the outcomes of their behaviors | 3 |
| Agency centered |
Institutional theory | Posits that an organization’s environment is capable of strongly influencing the development, acceptance, and use of eHealth interventions | 3 |
| Structural |
SCTj | Posits that the acquisition of new knowledge and perceived self-efficacy in using interventions is influenced by observing others in the context of social interactions and experiences (including the media) | 3 |
| Agency centered |
Theory of reasoned action | A social-psychological attitude-behavior model that examines normative social influences on behavioral intention | 2 |
| Agency centered |
ITk | Posits that the identity associated with a particular role is likely to drive individual decision making about eHealth interventions | 2 |
| Agency centered |
PMTl | Explains how the perceived threat of a new eHealth intervention affects individual adoption and implementation attitudes and behaviors | 2 |
| Agency centered |
Health belief model | Suggests that a personal threat, together with belief in the effectiveness of the proposed behavior, predicts the likelihood of engaging in that behavior | 2 |
| Agency centered |
BMHSUm | A model for predicting and explaining factors that lead to the use of eHealth services. These can include predisposing, enabling, and needs factors | 2 |
| Agency centered |
Sociotechnical systems theory | Proposes an interdependent relationship between the social and technical aspects of an organization (eg, human-computer interface). It suggests that these must be viewed congruently to optimize the implementation of new eHealth interventions | 1 |
| Relational |
ECTn | Posits that expectations, coupled with perceived performance of an eHealth intervention, lead to postpurchase satisfaction | 1 |
| Meaning centered |
PCTo | Posits that past experiences of using eHealth technologies and individual assumptions in relation to the design, impact, ownership, and value of these technologies can strongly influence acceptance or reluctance toward an eHealth intervention | 1 |
| Meaning centered |
Resource dependence theory | Asserts that acquiring and maintaining resources (eg, eHealth interventions) is key to organizational survival. Scarce resource availability or uncertainty about the environment motivates managers to act in ways to secure more resources and reduce their uncertainty | 1 |
| Structural |
Theory of middle managers’ role | Hypothesizes that middle managers promote implementation by fulfilling 4 roles: diffusing information, synthesizing information, mediating between strategy and day-to-day activities, and selling intervention implementation | 1 |
| Structural |
PADp emotional state theory | Asserts that all emotional responses to physical and social environmental stimuli can be captured in 3 dimensions: pleasure (enjoyment), arousal (alertness), and dominance (control), which subsequently influence human behavior | 1 |
| Agency centered |
Information behavior theory | Posits that information systems serve as a bridge between users and information resources. They consist of mediators (people who help users seek information and share the same social norms) and technologies (techniques and tools) that help users with the search | 1 |
| Agency centered |
JDRMq | Suggests that strain is a response to imbalance between demands on individuals and the resources they have to deal with these demands | 1 |
| Agency centered |
Cultural dimension theory | Shows the effects of a society’s culture on the values of its members and how these values relate to implementation behaviors | 1 |
| Meaning centered |
Affect theory | Claims that there are 3 primary types of affect or emotion, including positive (joy, interest, and excitement), neutral (surprise), and negative (anger, terror, and disgust). These affective states may be advantageous or disadvantageous in users’ acceptance of new information systems | 1 |
| Agency centered |
Activity theory | Focuses on understanding the mental capabilities of individuals by analyzing the cultural and technical aspects of human actions | 1 |
| Agency centered |
Social capital theory | Contends that social relationships are resources that can lead to the development and accumulation of human capital | 1 |
| Agency centered |
Contingency theory | Claims that there is no best way to organize or lead an organization or to make decisions. Instead, the optimal course of action is dependent upon the internal and external situation | 1 |
| Structural |
Social information processing theory | Predicts that technology-related attitudes and behaviors are not individually laden but socially constructed | 1 |
| Relational |
Consequence of modernity | Suggests that use of technologies is influenced by trust and sense of security in the absence of complete information from face-to-face interactions | 1 |
| Agency centered |
Social worlds theory | Proposes that social worlds are self-organizing units in which people share resources, information, and assumptions about what is important and ideas about what types of activities are desirable | 1 |
| Relational |
Boundary objects | Boundary objects serve as interfaces between multiple social worlds and facilitate the interaction; communication; and flow of information, concepts, skills, and materials between diverse social actors | 1 |
| Relational |
Adult learning theory | Posits that one’s learning context influences learning outcomes. Thus, knowledge evolves not only through formal learning activities, such as training programs, but also through the context and culture in which they are delivered | 1 |
| Agency centered |
Social contagion theory | A theory of collective behavior that explains how ideas and opinions spread in a social network. It holds that actors’ behaviors are a function of their exposure to others’ behaviors | 1 |
| Agency centered |
TTFr theory | Explains how technology interacts with the tasks or activities of an organization and impacts their performance | 1 |
| Structural |
Technology organization environment theory | Predicts that technology adoption is influenced by factors relating to technological, organizational, and environmental dimensions | 1 |
| Structural |
aTAM: technology acceptance model.
bUTAUT: unified theory of acceptance and use of technology.
cNPT: normalization process theory.
dDOI: diffusion of innovations.
eAST: adaptive structuration theory.
fCFIR: Consolidated Framework for Implementation Research.
gANT: actor-network theory.
hTPB: theory of planned behavior.
iEHR: electronical health record.
jSCT: social cognitive theory.
kIT: identity theory.
lPMT: protection motivation theory.
mBMHSU: behavioral model of health service utilization.
nECT: expectation confirmation theory.
oPCT: personal construct theory.
pPAD: pleasure, arousal and dominance.
qJDRM: job demands resource model.
rTTF: Task Technology Fit.
sHIS: hospital information system.
In total, 53% (19/36) of theories were classified as agency centered. Individual theories that occurred most frequently in the literature were the TAM by Davis and Venkatesh [
] (33 instances), UTAUT by Venkatesh [ ] (32 instances), and Diffusion of Innovations Theory by Rogers [ ] (16 instances). These theories were found to be primarily concerned with the individual and interpersonal elements of eHealth implementation. Although they did, to some extent, appear to consider the influence of organizational and social factors on eHealth adoption, individual attitudes, behaviors, and motivations remained the core focus of theoretical analysis. Theories classified as individual examined the adoption of eHealth either before or soon after the implementation of an intervention. However, they did not emphasize any form of user involvement in the development of an intervention. These theories tended to depict adoption as a temporally discrete and relatively immediate event, rather than as one stage in a larger multistage process. They often focused on what people were going to do soon, a decision they are about to make, or a behavior they need to alter. The diffusion of innovations theory provides an exception, as this theory considers time to be an essential factor influencing adoption [ ].A total of 20% (7/36) of theories identified in the literature were classified as relational. Of these, the normalization process theory (NPT) by May et al [
] occurred most frequently in the literature (17 instances), followed by structuration theory (ST) [ ] (6 instances) and actor-network theory (ANT) [ , ] (4 instances). Sociotechnical systems theory, social information processing theory, social worlds theory, and boundary object theory occurred only once each in the literature. Relational theories emphasize social relations and interactions at the human-technology interface. They highlighted the complex networks of social structure and meaning in which people are embedded, proposing that the translation of knowledge is facilitated by processes of circulation both within and across different social worlds. Some relational theories, such as ANT and ST, emphasized the role of nonhuman actors, such as computer software or programs, in transforming and mediating social relationships. These theories tended to view technology and society as coconstructed or coproduced, with no single dimension dictating change by itself. Within these theories, coproduction and implementation were often described as continuous processes, in which eHealth interventions were adapted to better accommodate different end-user settings and needs.A total of 16% (6/36) of theories were classified as structural. The most common structural theory was institutional theory (IT) [
] (3 instances). Resource dependence theory, theory of middle managers’ role, contingency theory, task technology fit theory, and technology organization environment theory occurred only once each in the literature. These theories conceptualized structure as including institutional or organizational systems as well as political, cultural, and other macrosocial environments. They often assumed that people are constrained or influenced by external forces frequently beyond their comprehension or control. For example, IT posits that organizational structures and cultural norms drive eHealth implementation, despite strong political influence.A total of 8% (3/36) of theories were classified as meaning centered: expectation confirmation theory, personal construct theory, and cultural dimension theory. Each of these theories occurred only once in the literature. These theories tended to focus on the cognitive dimensions (expectations, perceptions, and beliefs) that explain people’s willingness to accept the use of new health technologies. Although some meaning-centered theories, such as cultural dimension theory, have considered the influence of cultural values on the adoption and use of eHealth, these theories nonetheless centered their analysis at the individual level and were often used in combination with agency-level theories.
The Consolidated Framework for Implementation Research (CFIR) [
] was the only theory to be classified as a combined theory type. This theory is a meta-theoretical framework that provides a comprehensive listing of individual, social, and organizational constructs thought to influence eHealth implementation. However, it does not consider how these factors might be interrelated or how changes occur.Discussion
Principal Findings
Evidence from a range of disciplines suggests that theory-informed approaches to implementation science are integral to the translation and implementation of eHealth into clinical care [
, - ]. Analysis of the 119 studies included in this review identified 36 distinct theories that inform or explain eHealth implementation. However, only a few selected theories (UTAUT and TAM) were dominant, which is consistent with the findings from previous reviews [ - ]. Although these theories have been empirically proven to explain or predict certain aspects of implementation, Willmott et al [ ] and Davis et al [ ] caution that overreliance on common or favorite theories without direct questioning of their underlying assumptions limits progress in the field.The typology by Sovacool and Hess [
] facilitated a closer examination of the assumptions underlying eHealth implementation theories. The findings revealed that the majority of theories were agency centered, emphasizing individual factors rather than the broader social and environmental factors impacting implementation. Although these findings were consistent with previous reviews [ , , , ], the wider net cast for this review provided the needed validation that this trend can be observed across multiple specialty areas and disciplines [ ]. This calls into question whether theories currently being used to inform and explain the eHealth implementation adequately address the multiple and complex factors that influence the implementation process, and highlights the need for more dynamic, multilevel models of eHealth implementation [ , , ].This review identified a number of theories classified as relational or structural, which, to varying degrees, capture the complexity and multilevel nature of eHealth implementation. The most commonly cited relational theories were NPT, ST, and ANT. These theories recognize the important role of actors, relationships, and networks in mobilizing knowledge and embedding interventions into everyday practice. For ANT, networks are made up of both human and nonhuman actors, and technologies are understood to have agency and the potential to transform human interactions [
, ]. From this perspective, it may be a particularly useful theory for examining the implementation of eHealth technologies and the impacts these technologies have on human behavior. A criticism of ANT is that it has a flat ontology and refuses to consider institutional sources of power and inequality. Here, NPT and ST offer a possible extension, as both theories recognize the inseparable intersection between individual agents and wider social and organizational structures and norms. Structural theories also consider the influence of external forces on individual behavior and decision making. For example, IT, the most commonly used structural theory in this review, posits that an organization’s environment is capable of strongly influencing the development, acceptance, and use of eHealth interventions. This theory is considered particularly relevant for application in eHealth environments, which are highly institutionalized and subject to multiple regulatory forces, high levels of professionalism, and growing network externalities that can influence adoption decisions [ ].Of particular interest was the lacuna of normative theories identified in this review. Normative theories attempt to answer whether a technology is a net positive or negative for society and individuals [
]. To do so, they often rely on evaluative criteria determined by ethics, moral studies, political ecology, or social justice. Social justice theory and sustainable development are 2 common examples of normative theories. The absence of normative theories in eHealth implementation studies is emblematic of the broader tendency of implementation science to overlook the importance of contextual factors, such as economic, social, historical, and political forces, that perpetuate inequalities in the delivery of health care services [ ]. This omission is concerning in the context of eHealth, as digital technologies have been found to exacerbate inequalities associated with older age, lower level of educational attainment, and lower socioeconomic status [ ]. Future research should not shy away from normative questions of equity, justice, and sustainability and should find ways to incorporate theoretical approaches that enable exactly that.When incorporating or combining theories, Sovacool and Hess [
] highlight the need for careful consideration of the epistemological baggage of different approaches. Combining multiple theoretical approaches may offer a more complete understanding or explanation, yet such combinations may mask contrasting assumptions regarding key issues [ ]. For instance, are people driven primarily by their individual attitudes and motivation or do pervasive organizational cultures and social systems impose norms and values that shape people’s behavior, making individual characteristics relatively unimportant? These challenges may account for the tendency of theories to target variables at the same level. One exception was the CFIR framework, which was the sole theory that provided a menu of constructs at different levels for researchers to choose from. However, although CFIR recognizes the multilevel nature of eHealth implementation, it does not consider the relationship between constructs or how change takes place, leading Nilsen [ ] to contend that it should not be considered a theory at all. Further research is needed to explore how diverse theoretical perspectives can be brought together in ways that capture the dynamic interaction between constructs [ ], while avoiding disconnects and incompatibilities [ ].Limitations
This study has several limitations. First, papers not published in English were excluded, which may indicate a selection bias. The decision to keep the research question and inclusion criteria for this review broad resulted in a high yield of papers and, to some extent, reduced the specificity of search results. This decision was made to ensure the identification of the full spectrum of theories being used to inform and explain eHealth implementation. Restriction of inclusion criteria in previous systematic reviews [
] led to the omission of a number of key theories that provide a more comprehensive explanation of the various constituents of the implementation processes. A further limitation is that the protocol for this systematic review was not registered. However, every care was taken to ensure compliance with the core principles of the systematic review methodology. As Mallett [ ] noted, systematic reviews do not constitute a homogenous approach, and researchers may adopt a more flexible approach that better suits their research purpose while continuing to comply with the principles for conducting a systematic review. Finally, the literature search for this review was conducted in June 2019. Given the rapid rate of publication in the field of eHealth, it is likely that recent relevant articles have not been included. As completing an updated search was not feasible for the research team, we suggest that future studies must continue to identify theories used to inform and explain the implementation of eHealth interventions.Conclusions
This systematic review identified 36 theories that are being used to inform and explain eHealth implementation and classified these theories using the categories adapted from the typology by Sovacool and Hess [
] for theories of sociotechnical change. The results highlight the dominance of theories that focus mainly on individual readiness to accept health technologies rather than the various disorderly social processes or systemic dimensions of implementation. This calls into question whether theories currently being used to inform and explain eHealth implementation adequately address the multiple and multilevel factors that influence the implementation process. Nonetheless, this review identified a number of theories classified as relational, structural, or combined, which, to varying degrees, capture the complex interactions within a wider organization and policy system. Although less prominent in the literature, these theories may be particularly applicable to the implementation of eHealth in health settings and services.Acknowledgments
This systematic review is part of the eCliPSE (Electronic Clinical Pathways to Service Excellence) project, which is funded by a National Health and Medical Research Council partnership project grant (APP117181) and Beyond Blue.
Conflicts of Interest
None declared.
PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist.
DOC File , 80 KB
Key search terms used.
DOCX File , 16 KBReferences
- Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A'Court C, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res 2017 Nov 01;19(11):e367 [FREE Full text] [CrossRef] [Medline]
- Ossebaard HC, Van Gemert-Pijnen L. eHealth and quality in health care: implementation time. Int J Qual Health Care 2016 Jun;28(3):415-419. [CrossRef] [Medline]
- Huang F, Blaschke S, Lucas H. Beyond pilotitis: taking digital health interventions to the national level in China and Uganda. Global Health 2017 Jul 31;13(1):49 [FREE Full text] [CrossRef] [Medline]
- Morrison LG. Theory-based strategies for enhancing the impact and usage of digital health behaviour change interventions: a review. Digit Health 2015;1:1-10 [FREE Full text] [CrossRef] [Medline]
- Shaw T, McGregor D, Brunner M, Keep M, Janssen A, Barnet S. What is eHealth (6)? Development of a conceptual model for eHealth: qualitative study with key informants. J Med Internet Res 2017 Oct 24;19(10):e324 [FREE Full text] [CrossRef] [Medline]
- Eysenbach G. What is e-health? J Med Internet Res 2001 Jun;3(2):E20. [CrossRef] [Medline]
- Schreiweis B, Pobiruchin M, Strotbaum V, Suleder J, Wiesner M, Bergh B. Barriers and facilitators to the implementation of eHealth services: systematic literature analysis. J Med Internet Res 2019 Nov 22;21(11):e14197 [FREE Full text] [CrossRef] [Medline]
- Granja C, Janssen W, Johansen MA. Factors determining the success and failure of ehealth interventions: systematic review of the literature. J Med Internet Res 2018 May 01;20(5):e10235 [FREE Full text] [CrossRef] [Medline]
- Heinsch M. Exploring the potential of interaction models of research use in social work. Br J Soc Work 2018;48(2):468-486. [CrossRef]
- Jennings HM, Morrison J, Akter K, Kuddus A, Ahmed N, Shaha SK, et al. Developing a theory-driven contextually relevant mHealth intervention. Glob Health Action 2019;12(1):1550736 [FREE Full text] [CrossRef] [Medline]
- Pingree S, Hawkins R, Baker T, duBenske L, Roberts LJ, Gustafson DH. The value of theory for enhancing and understanding e-health interventions. Am J Prev Med 2010 Jan;38(1):103-109 [FREE Full text] [CrossRef] [Medline]
- Cotter AP, Durant N, Agne AA, Cherrington AL. Internet interventions to support lifestyle modification for diabetes management: a systematic review of the evidence. J Diabetes Complications 2014;28(2):243-251 [FREE Full text] [CrossRef] [Medline]
- Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci 2015;10(53):1-13 [FREE Full text] [CrossRef] [Medline]
- Ramachandran A, Snehalatha C, Ram J, Selvam S, Simon M, Nanditha A, et al. Effectiveness of mobile phone messaging in prevention of type 2 diabetes by lifestyle modification in men in India: a prospective, parallel-group, randomised controlled trial. Lancet Diabetes Endocrinol 2013 Nov;1(3):191-198. [CrossRef] [Medline]
- Kitson A. Moving on…. J Clin Nurs 2018 Jun 10;27(11-12):2175-2176. [CrossRef] [Medline]
- Michie S, Prestwich A. Are interventions theory-based? Development of a theory coding scheme. Health Psychol 2010 Jan;29(1):1-8. [CrossRef] [Medline]
- Ridde V. Need for more and better implementation science in global health. BMJ Glob Health 2016;1(2):e000115 [FREE Full text] [CrossRef] [Medline]
- Van Belle S, van de Pas R, Marchal B. Towards an agenda for implementation science in global health: there is nothing more practical than good (social science) theories. BMJ Glob Health 2017;2(2):e000181 [FREE Full text] [CrossRef] [Medline]
- Damschroder LJ. Clarity out of chaos: use of theory in implementation research. Psychiatry Res 2020 Jan;283:112461 [FREE Full text] [CrossRef] [Medline]
- May CR, Johnson M, Finch T. Implementation, context and complexity. Implement Sci 2016 Dec 19;11(1):141 [FREE Full text] [CrossRef] [Medline]
- Davis R, Campbell R, Hildon Z, Hobbs L, Michie S. Theories of behaviour and behaviour change across the social and behavioural sciences: a scoping review. Health Psychol Rev 2015 Sep;9(3):323-344 [FREE Full text] [CrossRef] [Medline]
- Glanz K, Bishop DB. The role of behavioral science theory in development and implementation of public health interventions. Annu Rev Public Health 2010;31:399-418. [CrossRef] [Medline]
- Harst L, Lantzsch H, Scheibe M. Theories predicting end-user acceptance of telemedicine use: systematic review. J Med Internet Res 2019 May 21;21(5):e13117 [FREE Full text] [CrossRef] [Medline]
- Willmott T, Pang B, Rundle-Thiele S, Badejo A. Reported theory use in electronic health weight management interventions targeting young adults: a systematic review. Health Psychol Rev 2019 Sep;13(3):295-317. [CrossRef] [Medline]
- Brennan L, Previte J, Fry M. Social marketing’s consumer myopia. J Soc Mark 2016 Jul 11;6(3):219-239. [CrossRef]
- Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care 2005 Feb;14(1):26-33 [FREE Full text] [CrossRef] [Medline]
- Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci 2016 Oct 26;11(1):146 [FREE Full text] [CrossRef] [Medline]
- Sovacool BK, Hess DJ. Ordering theories: typologies and conceptual frameworks for sociotechnical change. Soc Stud Sci 2017 Oct 23;47(5):703-750 [FREE Full text] [CrossRef] [Medline]
- Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
- Mallett R, Hagen-Zanker J, Slater R, Duvendack M. The benefits and challenges of using systematic reviews in international development research. J Dev Effect 2012 Sep;4(3):445-455. [CrossRef]
- Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC Psychol 2015 Sep 16;3:32 [FREE Full text] [CrossRef] [Medline]
- Helfer B, Samara M, Leucht S. Experiences with covidence in preparing a comprehensive systematic review. In: Filtering the Information Overload for Better Decisions. 2015 Presented at: 23rd Cochrane Colloquium; Oct 3-7, 2015; Vienna, Austria.
- O'Raghallaigh P, Sammon D, Murphy C. Map-making and theory-building: principles for achieving presentation and content affectiveness. In: Proceedings of the International Conference on Information Systems, ICIS 2010. 2010 Presented at: International Conference on Information Systems, ICIS 2010; December 12-15, 2010; Saint Louis, Missouri, USA,.
- Hess DJ. Neoliberalism and the history of STS theory: toward a reflexive sociology. Soc Epistemol 2013 Apr;27(2):177-193. [CrossRef]
- Jackson T. Motivating sustainable consumption: a review of evidence on consumer behaviour and behavioural change. The Sustainable Development Research Network, Surrey. 2005. URL: https://timjackson.org.uk/wp-content/uploads/2018/04/Jackson.-2005.-Motivating-Sustainable-Consumption.pdf [accessed 2019-06-30]
- Rutherford J, Coutard O. Urban energy transitions: places, processes and politics of socio-technical change. Urban Stud 2014 Apr 22;51(7):1353-1377. [CrossRef]
- de Veer AJ, Francke AL. Attitudes of nursing staff towards electronic patient records: a questionnaire survey. Int J Nurs Stud 2010 Jul;47(7):846-854. [CrossRef] [Medline]
- Zayyad MA, Toycan M. Factors affecting sustainable adoption of e-health technology in developing countries: an exploratory survey of Nigerian hospitals from the perspective of healthcare professionals. PeerJ 2018;6:1-15 [FREE Full text] [CrossRef] [Medline]
- Hennemann S, Beutel ME, Zwerenz R. Ready for eHealth? Health professionals' acceptance and adoption of eHealth interventions in inpatient routine care. J Health Commun 2017 Mar;22(3):274-284. [CrossRef] [Medline]
- Markam H, Hochheiser H, Kuntoro K, Notobroto HB. Exploring midwives' need and intention to adopt electronic integrated antenatal care. Perspect Health Inf Manag 2018(Winter):1-21 [FREE Full text] [Medline]
- Bagot KL, Cadilhac DA, Bladin CF, Watkins CL, Vu M, Donnan GA, VSTASTUTE investigators. Integrating acute stroke telemedicine consultations into specialists' usual practice: a qualitative analysis comparing the experience of Australia and the United Kingdom. BMC Health Serv Res 2017 Nov 21;17(1):1-8 [FREE Full text] [CrossRef] [Medline]
- May CR, Finch T, Ballini L, MacFarlane A, Mair F, Murray E, et al. Evaluating complex interventions and health technologies using normalization process theory: development of a simplified approach and web-enabled toolkit. BMC Health Serv Res 2011 Sep 30;11:245 [FREE Full text] [CrossRef] [Medline]
- Brooks E, Manson SM, Bair B, Dailey N, Shore JH. The diffusion of telehealth in rural American Indian communities: a retrospective survey of key stakeholders. Telemed J E Health 2012;18(1):60-66 [FREE Full text] [CrossRef] [Medline]
- Lin CP, Guirguis-Blake J, Keppel GA, Dobie S, Osborn J, Cole AM, et al. Using the diffusion of innovations theory to assess socio-technical factors in planning the implementation of an electronic health record alert across multiple primary care clinics. J Innov Health Inform 2016 Apr 15;23(1):450-458 [FREE Full text] [CrossRef] [Medline]
- Barrett AK, Stephens KK. The pivotal role of change appropriation in the implementation of health care technology. Manag Commun Q 2016 Dec 18;31(2):163-193. [CrossRef]
- Varsi C, Ekstedt M, Gammon D, Ruland CM. Using the consolidated framework for implementation research to identify barriers and facilitators for the implementation of an internet-based patient-provider communication service in five settings: a qualitative study. J Med Internet Res 2015 Nov 18;17(11):e262. [CrossRef] [Medline]
- Ware P, Ross HJ, Cafazzo JA, Laporte A, Gordon K, Seto E. Evaluating the implementation of a mobile phone-based telemonitoring program: longitudinal study guided by the consolidated framework for implementation research. JMIR Mhealth Uhealth 2018 Jul 31;6(7):13 [FREE Full text] [CrossRef] [Medline]
- Cavalcante RB, Esteves CJ, Gontijo TL, Brito MJ, Guimarães EA, Barbosa SP. Computerization of primary health care in Brazil: the network of actors. Rev Bras Enferm 2019;72(2):337-344 [FREE Full text] [CrossRef] [Medline]
- AlBar AM, Hoque MR. Patient acceptance of e-health services in Saudi-Arabia: an integrative perspective. Telemed J E Health 2019 Sep;25(9):847-852. [CrossRef] [Medline]
- Sherer SA, Meyerhoefer CD, Peng L. Applying institutional theory to the adoption of electronic health records in the U.S. Inf Manag 2016 Jul;53(5):570-580. [CrossRef]
- Weeger A, Gewald H. Acceptance and use of electronic medical records: an exploratory study of hospital physicians’ salient beliefs about HIT systems. Health Syst 2015;4(1):64-81. [CrossRef]
- Moon BC, Chang H. Technology acceptance and adoption of innovative smartphone uses among hospital employees. Healthc Inform Res 2014 Oct;20(4):304-312 [FREE Full text] [CrossRef] [Medline]
- Mishra AN, Anderson C, Angst CM, Agarwal R. Electronic health records assimilation and physician identity evolution: an identity theory perspective. Inf Syst Res 2012 Sep;23(3-part-1):738-760. [CrossRef]
- Guo X, Han X, Zhang X, Dang Y, Chen C. Investigating m-Health acceptance from a protection motivation theory perspective: gender and age differences. Telemed J E Health 2015 Aug;21(8):661-669. [CrossRef] [Medline]
- Tavares J, Oliveira T. New integrated model approach to understand the factors that drive electronic health record portal adoption: Cross-Sectional National Survey. J Med Internet Res 2018 Nov 19;20(11):e11032 [FREE Full text] [CrossRef] [Medline]
- Chiu TM, Eysenbach G. Stages of use: consideration, initiation, utilization, and outcomes of an internet-mediated intervention. BMC Med Inform Decis Mak 2010 Nov 23;10:73 [FREE Full text] [CrossRef] [Medline]
- Ahmad F, Norman C, O'Campo P. What is needed to implement a computer-assisted health risk assessment tool? An exploratory concept mapping study. BMC Med Inform Decis Mak 2012 Dec 19;12:149 [FREE Full text] [CrossRef] [Medline]
- Chou HK, Lin IC, Woung LC, Tsai MT. Engagement in e-learning opportunities: an empirical study on patient education using expectation confirmation theory. J Med Syst 2012 Jun;36(3):1697-1706. [CrossRef] [Medline]
- Fernando S, Choudrie J, Lycett M, de Cesare S. Hidden assumptions and their influence on clinicians’ acceptance of new IT systems in the NHS. Inf Syst Front 2012;14(2):279-299. [CrossRef]
- Tarver WL, Menachemi N. Environmental market factors associated with electronic health record adoption among cancer hospitals. Health Care Manage Rev 2018;43(4):303-314. [CrossRef] [Medline]
- Birken SA, DiMartino LD, Kirk MA, Lee SY, McClelland M, Albert NM. Elaborating on theory with middle managers' experience implementing healthcare innovations in practice. Implement Sci 2016;11(2):1-15 [FREE Full text] [CrossRef] [Medline]
- Alaiad A, Zhou L. Patients' adoption of WSN-based smart home healthcare systems: an integrated model of facilitators and barriers. IEEE Trans Profess Commun 2017 Mar;60(1):4-23. [CrossRef]
- Chiu TM, Eysenbach G. Theorizing the health service usage behavior of family caregivers: a qualitative study of an internet-based intervention. Int J Med Inform 2011 Nov;80(11):754-764. [CrossRef] [Medline]
- Hysong SJ, Spitzmuller C, Espadas D, Sittig DF, Singh H. Electronic alerts and clinician turnover: the influence of user acceptance. Am J Manag Care 2014 Nov;20(11 Spec No. 17):520-530 [FREE Full text] [Medline]
- Hoque MR, Bao Y. Cultural influence on adoption and use of e-Health: evidence in Bangladesh. Telemed J E Health 2015 Oct;21(10):845-851. [CrossRef] [Medline]
- Djamasbi S, Fruhling AL, Loiacono ET. The influence of affect, attitude and usefulness in the acceptance of telemedicine systems. Information Systems and Quantitative Analysis Faculty Publications. 2009. URL: https://core.ac.uk/download/pdf/232755549.pdf [accessed 2021-05-19]
- Tsai CH. Integrating social capital theory, social cognitive theory, and the technology acceptance model to explore a behavioral model of telehealth systems. Int J Environ Res Public Health 2014 May 07;11(5):4905-4925 [FREE Full text] [CrossRef] [Medline]
- Hsiao JL, Chen RF. Critical factors influencing physicians' intention to use computerized clinical practice guidelines: an integrative model of activity theory and the technology acceptance model. BMC Med Inform Decis Mak 2016;16:15 [FREE Full text] [CrossRef] [Medline]
- Baird A, Furukawa MF, Raghu TS. Understanding contingencies associated with the early adoption of customer-facing web portals. J Manag Inf Syst 2014 Dec 08;29(2):293-324. [CrossRef]
- Sharma U, Barnett J, Clarke M. Stud Health Technol Inform 2010;160(Pt 1):545-549. [Medline]
- Barrett AK, Stephens KK. Making Electronic Health Records (EHRs) work: informal talk and workarounds in healthcare organizations. Health Commun 2017 Aug;32(8):1004-1013. [CrossRef] [Medline]
- Nielsen JA, Mengiste SA. Analysing the diffusion and adoption of mobile IT across social worlds. Health Informatics J 2014 Jun;20(2):87-103 [FREE Full text] [CrossRef] [Medline]
- McAlearney AS, Robbins J, Kowalczyk N, Chisolm DJ, Song PH. The role of cognitive and learning theories in supporting successful EHR system implementation training: a qualitative study. Med Care Res Rev 2012 Jun;69(3):294-315. [CrossRef] [Medline]
- Gan Q. Is the adoption of electronic health record system “contagious”? Health Policy Technol 2015 Jun;4(2):107-112. [CrossRef]
- Ahmadi H, Nilashi M, Shahmoradi L, Ibrahim O. Hospital Information System adoption: expert perspectives on an adoption framework for Malaysian public hospitals. Comput Hum Beh 2017 Feb;67:161-189. [CrossRef]
- Davis F, Venkatesh V. Toward preprototype user acceptance testing of new information systems: implications for software project management. IEEE Trans Eng Manage 2004 Feb;51(1):31-46. [CrossRef]
- Venkatesh V. Determinants of perceived ease of use: integrating control, intrinsic motivation, and emotion into the technology acceptance model. Inf Syst Res 2000 Dec;11(4):342-365. [CrossRef]
- Rogers E. Diffusion of Innovations. New York: Free Press; 2003:1-576.
- Anttila M, Välimäki M, Koivunen M, Luukkaala T, Kaila M, Pitkänen A, et al. Adoption of an internet-based patient education programme in psychiatric hospitals. J Psychiatr Ment Health Nurs 2011 Dec;18(10):914-923. [CrossRef] [Medline]
- May CR, Mair F, Finch T, MacFarlane A, Dowrick C, Treweek S, et al. Development of a theory of implementation and integration: Normalization Process Theory. Implement Sci 2009 May 21;4:29 [FREE Full text] [CrossRef] [Medline]
- Giddens A. The Constitution of Society: Outline of the Theory of Structuration. Oakland, California: University of California Press; 1986:1-417.
- Latour B. Science in Action – How to Follow Scientists & Engineers Through Society. Cambridge, Massachusetts, United States: Harvard University Press; 1988:1-282.
- Latour B, Woolgar S, Salk J. Laboratory Life: The Construction of Scientific Facts. Princeton, NJ: Princeton University Press; 1986:1-296.
- Meyer JW, Rowan B. Institutionalized organizations: formal structure as myth and ceremony. Am J Sociol 1977 Sep;83(2):340-363. [CrossRef]
- Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50 [FREE Full text] [CrossRef] [Medline]
- McIsaac JD, Penney TL, Storey KE, Sigfridson L, Cunningham J, Kuhle S, et al. Integrated knowledge translation in population health intervention research: a case study of implementation and outcomes from a school-based project. Health Res Policy Syst 2018 Aug 02;16(1):72 [FREE Full text] [CrossRef] [Medline]
- Hekler EB, Michie S, Pavel M, Rivera DE, Collins LM, Jimison HB, et al. Advancing models and theories for digital behavior change interventions. Am J Prev Med 2016 Nov;51(5):825-832. [CrossRef] [Medline]
- Callon M. The sociology of an actor-network: the case of the electric vehicle. In: Callon M, Law J, Rip A, editors. Mapping the Dynamics of Science and Technology. London: Palgrave Macmillan; 1986:19-34.
- Latour B. Where are the missing masses? The sociology of a few mundane artifacts. In: Bijker W, Law J, editors. Shaping Technology-Building Society: Studies in Sociotechnical Change. Cambridge, MA: The MIT Press; 1992:225-259.
- Baumann AA, Cabassa LJ. Reframing implementation science to address inequities in healthcare delivery. BMC Health Serv Res 2020 Mar 12;20(1):190 [FREE Full text] [CrossRef] [Medline]
- Azzopardi-Muscat N, Sørensen K. Towards an equitable digital public health era: promoting equity through a health literacy perspective. Eur J Public Health 2019 Oct 01;29(3):13-17 [FREE Full text] [CrossRef] [Medline]
- Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci 2015;10:53 [FREE Full text] [CrossRef] [Medline]
Abbreviations
ANT: actor-network theory |
CFIR: Consolidated Framework for Implementation Research |
eCliPSE: Electronic Clinical Pathways to Service Excellence |
IT: institutional theory |
NPT: normalization process theory |
PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses |
ST: structuration theory |
TAM: technology acceptance model |
UTAUT: unified theory of acceptance and use of technology |
Edited by G Eysenbach, Q Zeng; submitted 01.03.20; peer-reviewed by DA Hepworth, P Chow, S Nepal, CJ( Wu, T Willmott; comments to author 28.08.20; revised version received 22.10.20; accepted 15.04.21; published 31.05.21
Copyright©Milena Heinsch, Jessica Wyllie, Jamie Carlson, Hannah Wells, Campbell Tickner, Frances Kay-Lambkin. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 31.05.2021.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.