Published on in Vol 21, No 6 (2019): June

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/10896, first published .
Context-Aware Systems for Chronic Disease Patients: Scoping Review

Context-Aware Systems for Chronic Disease Patients: Scoping Review

Context-Aware Systems for Chronic Disease Patients: Scoping Review

Review

1Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, Australia

2Macquarie University Health Sciences Centre, Macquarie University, Sydney, Australia

*these authors contributed equally

Corresponding Author:

Kathleen Yin, PhD

Centre for Health Informatics

Australian Institute of Health Innovation

Macquarie University

75 Talavera Rd

Sydney,

Australia

Phone: 61 2 98502477

Email: kathleen.yin@mq.edu.au


Background: Context-aware systems, also known as context-sensitive systems, are computing applications designed to capture, interpret, and use contextual information and provide adaptive services according to the current context of use. Context-aware systems have the potential to support patients with chronic conditions; however, little is known about how such systems have been utilized to facilitate patient work.

Objective: This study aimed to characterize the different tasks and contexts in which context-aware systems for patient work were used as well as to assess any existing evidence about the impact of such systems on health-related process or outcome measures.

Methods: A total of 6 databases (MEDLINE, EMBASE, CINAHL, ACM Digital, Web of Science, and Scopus) were scanned using a predefined search strategy. Studies were included in the review if they focused on patients with chronic conditions, involved the use of a context-aware system to support patients’ health-related activities, and reported the evaluation of the systems by the users. Studies were screened by independent reviewers, and a narrative synthesis of included studies was conducted.

Results: The database search retrieved 1478 citations; 6 papers were included, all published from 2009 onwards. The majority of the papers were quasi-experimental and involved pilot and usability testing with a small number of users; there were no randomized controlled trials (RCTs) to evaluate the efficacy of a context-aware system. In the included studies, context was captured using sensors or self-reports, sometimes involving both. Most studies used a combination of sensor technology and mobile apps to deliver personalized feedback. A total of 3 studies examined the impact of interventions on health-related measures, showing positive results.

Conclusions: The use of context-aware systems to support patient work is an emerging area of research. RCTs are needed to evaluate the effectiveness of context-aware systems in improving patient work, self-management practices, and health outcomes in chronic disease patients.

J Med Internet Res 2019;21(6):e10896

doi:10.2196/10896

Keywords



Background

As health care moves from the traditional hospital setting to the personal sphere of home and community, individuals are increasingly being encouraged to engage in self-care [1,2]. Sociologists refer to this act of self-care as patient work, which involves effort and investment of time on the part of patients or family members to accomplish a health goal [3]. Patient work extends beyond strictly health-related tasks and is shaped by the context of patients’ lives and their daily routines [1]. It has been suggested that the use of context-aware technologies may thus better support patient work and improve self-care, as contextual information could trigger more personalized and relevant services or information [4].

Context-aware systems, also known as context-sensitive systems, are computing applications designed to capture, interpret, and use contextual information and provide adaptive services according to the current context of use [4,5]. Context-aware systems may thus harness everything from sensors that capture data indicative of context (such as time, location, and light intensity) to inference mechanisms that interpret and action such data [5]. Even though context-aware systems have been piloted in some health care settings, their impact on health care outcomes remains unclear. Specifically, context-aware systems have mainly been piloted in the hospital setting [4] and for primary prevention [6-8], rarely addressing the context of chronic disease patients’ health-related activities in everyday life.

Objectives

The aim of this study was to examine existing literature on interventions using context-aware technologies that support patient work. Specifically, we sought to characterize the different tasks and contexts in which such systems were used, as well as assess any existing evidence about their impact on health-related process and outcome measures.


Search Strategy

A systematic search of the literature was performed in September 2016 and updated in October 2017 on MEDLINE, EMBASE, CINAHL, ACM Digital, Web of Science, and Scopus using search terms regarding patient work, context awareness, and consumer health informatics. The complete search strategy is available in Multimedia Appendix 1. The reference lists of relevant articles were also screened to ensure that all eligible studies were captured. A grey literature search was performed using Google Scholar to capture dissertations, theses, and conference proceedings that met the inclusion criteria.

Study Selection Criteria

In the scope of our study, we focused on context-aware systems that were capable of (1) capturing and processing contextual information (eg, environmental data and user-related features) and (2) using the captured contextual information to provide adaptive services and support patient work tasks in everyday life, either at home or in the community.

Studies were included in the review if they focused on patients with chronic conditions, involved the use of a context-aware system to support patients’ health-related activities, and reported the evaluation of the systems by the users.

Studies were excluded if they were not in English or if they focused on health care providers instead of consumers. We also excluded interventions that merely gathered and displayed context information, without using it to adapt system behavior (passive context awareness), as this was outside the scope of this study.

Paper Screening Process

We conducted a 2-phase screening process, initially excluding papers based on their titles and abstracts using a standard screening form, and then rescreening the remaining papers based on the full-text article.

Both phases were conducted by teams of 2 independent reviewers (2 teams in the first phase and 1 in the second). Cohen kappa was used in the full-text paper screening to measure intercoder agreement. Any disagreements in the screening were resolved through discussion and consensus.

Data Extraction and Synthesis Strategy

One reviewer extracted information from the eligible studies into a data extraction form, whereas 2 other reviewers examined the completed form for consistency and accuracy. The following information was collected: first author, year, health domain, study type, participants’ characteristics (number, age, and sex ratio), intervention characteristics, health activities (tasks undertaken by patients to achieve health goals), and main findings. To explore how context was utilized by the included studies, we analyzed the related contextual elements based on previous work [4,9]. Context information was grouped into the following dimensions: setting (indoor or outdoor), environmental features (indoor and outdoor attributes, eg, room temperature, humidity, and air pollen), and user features (user-related data captured by the system, eg, physical activity, physiological measurements, and mental state). Time is also considered an important element, but it is often coupled with other dimensions, so it was not analyzed separately in this review. Finally, we characterized the utilization of context for each study (ie, the adaptive services provided by the system based on contextual data).

Our study design follows the guidelines for a scoping review proposed by Arksey and O’Malley (2005) and follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement [10], where applicable.


Retrieved Studies

The database search retrieved 1478 citations; 607 duplicates were removed (Figure 1). After screening the abstracts and titles, 768 articles were excluded for not meeting the eligibility criteria. Full-text screening eliminated 36 articles (a list of excluded articles is available in Multimedia Appendix 2). An additional 2 articles were found via hand-search, yielding 6 included studies in total. The kappa statistic measuring inter-rater agreement for full-text screening was 0.6 (moderate agreement) [11].

Description of Included Studies

The number of participants in studies ranged from 4 to 47 (Table 1). A total of 1 study was conducted in Canada [12], 2 in the United States [13,14], and 2 in Europe [15,16]. Most articles were published after 2010 except 1 [17]. The health domains covered in the studies included asthma [13], cardiovascular disease [16], kidney disease [12], Parkinson disease [17], diabetes [15], and mental health [14]. All included studies used quasi-experimental study designs to pilot test different context-aware systems. Demographic information on participants was often missing and inconsistently reported. Specifically, age data were reported by 3 studies [12,14,17], and 4 studies reported sex data [12,14,15,17].

Figure 1. Flow diagram of included studies.
View this figure
Table 1. Characteristics of included studies and context-aware systems.
Study author, year, locationHealth domainStudy type; durationN total (mean age, % female)Health activitiesPatient-facing technologiesFunctions
Bächlin et al, 2009, Israel [17]Parkinson diseaseQuasi-experimental; not reported (NR)10 patients (66.4, 30)Self-management of gait deficits in Parkinson patientsAcceleration sensorsDetect movement and freezing of gait
Earphones and wearable computerProduce sound when freezing of gait occurs (continuous external rhythmic auditory cues improve gait performance)>
Ong et al, 2016, Canada [12]Chronic kidney diseaseQuasi-experimental; 6 months47 patients with chronic kidney disease (59, 47)Self-management of chronic kidney disease (self-monitor blood pressure [BP] and symptoms, manage medications, track lab test results)Wireless BP monitorMeasure BP
Mobile appPersonalized real-time feedback on BP; reminders (eg, reconcile medication, and measure BP); self-monitor symptoms; access to lab test results and medication list
Lamprinos et al, 2016, Germany and Turkey [15]DiabetesQuasi-experimental; 6 weeksIn Germany: 21 patients (NR, 24); In Turkey: 39 patients (NR, 46)Self-management of diabetes (self-monitor physiological measures; manage medications and lifestyle behaviors)Mobile app and websiteSelf-monitor (eg, blood glucose, weight, BP, medication, physical activity, diet, and sleep); personalized feedback (decision making and action planning)
Zhang et al, 2016, Germany [16]Cardiovascular diseaseQuasi-experimental; NR5 healthy young adults (NR)Self-management of cardiovascular disease (self-monitor heart rate and identify abnormalities)Wearable sensorsTrack physical activity, heart rate, skin temperature, cardiac and pulmonary function, posture
Environmental sensorsDetect room temperature
Mobile appRetrieve sensor data; trigger an alarm when an abnormal heartbeat is detected
Anantharam et al, 2015, United States [13]AsthmaQuasi-experimental; 10 days4 children (NR)Self-management of asthma (self-monitor symptoms and identify triggers)Indoor sensorMonitor environmental and air quality observations (eg, pollen levels, carbon monoxide, temperature, and humidity)
Exhaled air sensorMonitor exhaled nitric oxide (indicator of inflammation)
Mobile appGather and display sensor data; record users’ observations (eg, asthma-related symptoms) via questionnaires; personalized feedback
Burns et al, 2011, United States [14]Major depressive disorderQuasi-experimental; 8 weeks8 patients (37.4, 88)Self-management of depression (self-monitor symptoms and identify triggers)Mobile phone sensorsCollect data on location, ambient light, phone usage
WebsiteProvide behavioral therapy; display data collected from the mobile phone
Mobile appCollect self-reported data on social context, activity, location, and internal states (ie, mood) via ecological momentary assessment; integrate self-reports with sensor data; personalized feedback; predict patient states based on self-reports and sensor data

Context Elements and Technologies of Current Interventions

The contextual elements of each included study are summarized in Table 2. A total of 2 studies focused on indoor settings [13,17], and 4 studies involved both indoor and outdoor settings [12,14-16]. Context was captured using sensors [12-14,16,17] and self-reports [12-15]. A total of 1 particular study used ecological momentary assessment to capture self-reported data on social context, activity, and internal states (ie, mood) [14]. Sensors captured data on location, ambient light [14], air quality [13], room temperature [16], and physiological measures (blood pressure [BP], heart rate, skin temperature, cardiac function, exhaled nitric oxide) [12,13,16]. Acceleration sensors were used to track movement and physical activity [16,17].

Table 2. Context elements present in included studies.
Study, yearSettingsEnvironmental featuresUser featuresUtilization of context
Bächlin, 2009 [14]IndoorNoneMovement trackingReal-time movement tracking system triggering cueing sound upon detection of freezing of gait.
Ong, 2016 [12]Indoor and outdoorNoneBlood pressure (BP)Provide real-time personalized feedback on BP (eg, uncontrolled BP triggered reminder messages recommending an increase in frequency of self-monitoring).
Lamprinos, 2016 [15]Indoor and outdoorNonePhysical activity tracking (step counts), sleep tracking, blood glucose, BP, weight, mood, nutritionCreates a personalized action plan based on patient-recorded data and generates self-management recommendations.
Zhang, 2016 [16]Indoor and outdoorRoom temperaturePhysical activity tracking (standing, walking, running, jumping, walking upstairs or downstairs), heart rate, skin temperature, cardiac function, pulmonary function, postureTrigger an alarm when an abnormal heartbeat is detected.
Anantharam, 2015 [13]IndoorCarbon monoxide, temperature, humidity, pollen levelsExhaled nitric oxide, asthma-related symptoms (eg, coughing and chest tightness)Provide personalized actionable recommendations based on sensor data and patient-reported information (eg, identify and alert patients regarding triggers).
Burns, 2011 [13]Indoor and outdoorLocation sensing, ambient lightPhysical activity tracking, social context (eg, interactions with other people), and internal states (mood, intensity of discrete emotions, fatigue, sense of accomplishment, concentration and engagement, and perceived control over current activities); manually self-reported via ecological momentary assessmentPredict patient states based on self-reported and sensor data (using machine learning), displaying them on the mobile app. Future iterations will involve the use of predicted states to provide real-time interventions.

Most studies used a combination of sensor technology and mobile apps, where the sensors collected context information, and the apps utilized those data to deliver personalized feedback [12-14,16]. Only 1 study did not involve the use of sensors, collecting contextual information solely through user self-reports [15]. In another intervention, a sensor was used without a mobile app, where context was harnessed with the help of a wearable computer, and earphones delivered auditory cues to improve gait performance in Parkinson patients [17].

Health Activities and Health-Related Measures

Self-monitoring was the most frequent health activity supported by context-aware systems in the included studies, where the collected data were then used to provide personalized feedback. Self-monitored data included physiological measures (eg, BP) [12,13,15,16], symptoms [12-14], and lifestyle behaviors [15]. Other health activities included tracking lab test results [12], managing medications [12,15], and practicing specific behaviors (eg, overcoming freezing of gait) [17].

Only 3 studies reported the impact of the intervention on health-related measures [12,14]. Specifically, Bächlin [17] found that the intervention had a sensitivity of 73.1% and a specificity of 81.6% in detecting freezing of gait events. Ong et al [12] found statistically significant reductions in home BP readings between baseline and after intervention (systolic BP: –3.4 mmHg; 95% CI –5.0 to –1.8; diastolic BP: –2.1 mmHg; 95% CI –2.9 to –1.2). Burns et al [14] found a significant decrease in self-reported depressive symptoms (P<.001; per-­protocol Cohen d=3.43) and comorbid anxiety symptoms (P<.001, per-­protocol Cohen d=2.58) [14]).

No studies mentioned a thorough evaluation of patient safety problems. A total of 4 studies highlighted technical issues such as system downtime [15], battery drainage problems [13,14], and wearable sensor issues in activity detection [16].


Principal Findings

To the best of our knowledge, this is the first systematic scoping review to examine context-aware interventions to support patient work. The emerging nature of the field is reflected in the small number of included studies, their recent time of publication (all after 2010), and the predominance of quasi-experimental study designs. The majority of the papers involved pilot and usability testing with a small number of users; there were no randomized controlled trials (RCTs) to evaluate the efficacy of a context-aware system. In the included studies, context was captured using sensors or self-reports, sometimes involving both. Most studies used a combination of sensor technology and mobile apps to deliver personalized feedback. A total of 3 studies examined the impact of interventions on health-related measures, showing moderate-to-good sensitivity and specificity in detecting freezing of gait events in Parkinson patients [17], as well as significant improvements in BP [12] and reductions in depression symptoms and comorbid anxiety symptoms [14].

Comparison With Previous Literature

Other reviews have looked at the use of context awareness in health care [4,18]. Bricon-Souf (2007) found that there was a large gap between the requirements expressed by users and the context-aware prototypes developed. In addition, they reported that there was no consensus in the research community on how to model context and architectures to support its use. Similarly, Orwat et al revealed that most systems were described in their prototype stage and that implementation issues were rarely mentioned.

Our study, though only focused on patient work, also revealed comparable findings. Most studies described prototypes, and only 3 studies examined the impact of interventions on health-related measures, showing promising results in detecting freezing of gait events in Parkinson patients [17], as well as in improving BP [12] and depression and anxiety symptoms [14]. The use of context-awareness systems in patient work interventions has the potential to facilitate self-monitoring and improve the relevance and quality of the feedback provided, personalizing it to better fit participants’ context [12-14,16]. This sort of “just-in-time” support [19] has the potential to facilitate patient work and improve the self-management of chronic conditions, by providing the advice patients need to make health management decisions at the right time, on a daily basis. Ameliorating self-management practices is a cornerstone of quality improvement efforts in chronic disease care and is associated with better health outcomes in several conditions such as type 2 diabetes [20].

The costs and risks of using context-aware systems for patient work were rarely reported in the included studies. A total of 4 studies highlighted technical issues such as system downtime [15], battery drainage problems [13,14], and wearable sensor issues in activity detection [16]. No studies mentioned a thorough evaluation of patient safety problems. Future studies should consistently report unintended effects and possible harms of the systems, such as privacy, technical issues, or any other unanticipated incidents [21].

Strengths and Limitations

This systematic scoping review has several strengths in terms of study design. First, an extensive search was performed across multiple databases to ensure that all relevant studies were captured. Second, the screening form was pretested and piloted before screening. Third, all full-text papers were screened by 2 independent reviewers. Finally, the kappa score of 0.6 for the full-text screening phase revealed an acceptable level of agreement.

The results of our study need to be interpreted in light of some limitations. Given that this is an emerging field in health informatics, there is a lack of longitudinal and experimental studies, which hampers the evaluation of the impact of these interventions. This is the reason why a systematic scoping review was conducted instead of a systematic review.

Another limitation was the exclusion of non-English papers. Even though this was conducted to ensure that all the authors could adequately understand and make an informed decision based on the abstracts, we might have missed important papers on patient work.

Implications for Research and Health Care

The use of context-aware systems to support patient work is a promising area of research, as these interventions have the potential to facilitate self-monitoring and provide personalized just-in-time feedback based on users’ characteristics and environmental features, with the aim of improving disease management and clinical outcomes. Specifically, the increasing use of sensors to automatically collect context information could eliminate the need for self-reporting and manual data entry, streamlining the task of self-monitoring for chronic disease patients [22]. Furthermore, future applications of artificial intelligence have the potential to expand on the current capacity of these systems to provide personalized and relevant services to individuals [23], better supporting users with their health-related tasks, and decreasing the burden of patient work.

A common issue in context-aware systems research is the challenge of evaluating their real-world implementation [4,18]. Implementation fidelity is “the degree to which programs are implemented as intended by the program developers” [24]. It is known that implementation settings play a crucial role in the effectiveness of interventions, an issue that is at the core of implementation science’s efforts to model the impact of context on outcomes [25]. To allow for implementation fidelity and replicability, studies of context-aware systems should describe the setting explicitly as well as provide sufficient details about the intervention and any potential adaptations for it to fit a different setting [26,27].

Conclusions

The use of context-aware systems to support patient work is an emerging area of research. RCTs are needed to evaluate the effectiveness of context-aware systems in improving patient work, self-management practices, and health outcomes in chronic disease patients. Future studies should consistently report the intervention and the settings in which the intervention is being implemented.

Acknowledgments

The authors would like to thank Mr Duy Hung Nguyen and Ms Keyu Wang for their assistance in the manuscript format. This research was supported by the National Health and Medical Research Council grant APP1134919 (Centre for Research Excellence in Digital Health) and Program Grant APP1054146.

Authors' Contributions

AYSL, SV, and EC contributed to the study design. HLT, LL, and AYSL contributed to the search strategy. HLT, LL, KY, and PM contributed to the screening. HLT, LL, and ABK were responsible for data extraction. HLT, LL, AYSL, KY, ABK, and EC wrote the first draft. All authors contributed to the revision and subsequent drafts.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Search strategy.

PDF File (Adobe PDF File), 69KB

Multimedia Appendix 2

Excluded articles.

PDF File (Adobe PDF File), 75KB

  1. Valdez RS, Holden RJ, Novak LL, Veinot TC. Transforming consumer health informatics through a patient work framework: connecting patients to context. J Am Med Inform Assoc 2015 Jan;22(1):2-10. [CrossRef] [Medline]
  2. Hibbard JH. Moving toward a more patient-centered health care delivery system. Health Aff (Millwood) 2004;Suppl Variation:VAR133-VAR135. [CrossRef] [Medline]
  3. Strauss A. Continual Permutations Of Action. New York: Aldine Transaction; 1993.
  4. Bricon-Souf N, Newman CR. Context awareness in health care: a review. Int J Med Inform 2007 Jan;76(1):2-12. [CrossRef] [Medline]
  5. Baldauf M, Dustdar S, Rosenberg F. A survey on context-aware systems. Int J Ad Hoc Ubiq Co 2007;2(4):263-277 [FREE Full text] [CrossRef]
  6. Naughton F, Hopewell S, Lathia N, Schalbroeck R, Brown C, Mascolo C, et al. A context-sensing mobile phone app (Q sense) for smoking cessation: a mixed-methods study. JMIR Mhealth Uhealth 2016 Sep 16;4(3):e106 [FREE Full text] [CrossRef] [Medline]
  7. Businelle MS, Ma P, Kendzor DE, Frank SG, Vidrine DJ, Wetter DW. An ecological momentary intervention for smoking cessation: evaluation of feasibility and effectiveness. J Med Internet Res 2016 Dec 12;18(12):e321 [FREE Full text] [CrossRef] [Medline]
  8. Hébert ET, Stevens EM, Frank SG, Kendzor DE, Wetter DW, Zvolensky MJ, et al. An ecological momentary intervention for smoking cessation: the associations of just-in-time, tailored messages with lapse risk factors. Addict Behav 2018 Mar;78:30-35. [CrossRef] [Medline]
  9. Dey AK, Abowd GD, Salber D. A conceptual framework and a toolkit for supporting the rapid prototyping of context-aware applications. Hum Comput Interact 2009 Nov 12;16(2-4):97-166. [CrossRef]
  10. Liberati A, Altman D, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 2009 Jul 21;6(7):e1000100 [FREE Full text] [CrossRef] [Medline]
  11. Higgens JP, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. London: The Cochrane Collaboration; 2011.
  12. Ong SW, Jassal SV, Miller JA, Porter EC, Cafazzo JA, Seto E, et al. Integrating a smartphone-based self-management system into usual care of advanced CKD. Clin J Am Soc Nephrol 2016 Jun 6;11(6):1054-1062 [FREE Full text] [CrossRef] [Medline]
  13. Anantharam P, Banerjee T, Sheth A, Thirunarayan K, Marupudi S, Sridharan V. Knowledge-Driven Personalized Contextual mHealth Service for Asthma Management in Children. 2015 Presented at: IEEE International Conference on Mobile Services (MS); June 27-July 2, 2015; New York, NY, USA.
  14. Burns MN, Begale M, Duffecy J, Gergle D, Karr CJ, Giangrande E, et al. Harnessing context sensing to develop a mobile intervention for depression. J Med Internet Res 2011;13(3):e55 [FREE Full text] [CrossRef] [Medline]
  15. Lamprinos I, Demski H, Mantwill S, Kabak Y, Hildebrand C, Ploessnig M. Modular ICT-based patient empowerment framework for self-management of diabetes: design perspectives and validation results. Int J Med Inform 2016 Jul;91:31-43. [CrossRef] [Medline]
  16. Zhang W, Thurow K, Stoll R. A context-aware mhealth system for online physiological monitoring in remote healthcare. Int J Comput Commun Contr 2016;11(1):142-156.
  17. Bächlin M, Plotnik M, Roggen D, Giladi N, Hausdorff JM, Tröster G. A wearable system to assist walking of Parkinson s disease patients. Methods Inf Med 2010;49(1):88-95. [CrossRef] [Medline]
  18. Orwat C, Graefe A, Faulwasser T. Towards pervasive computing in health care - a literature review. BMC Med Inform Decis Mak 2008;8:26 [FREE Full text] [CrossRef] [Medline]
  19. Nahum-Shani I, Smith SN, Spring BJ, Collins LM, Witkiewitz K, Tewari A, et al. Just-in-time adaptive interventions (JITAIs) in mobile health: key components and design principles for ongoing health behavior support. Ann Behav Med 2016 Sep 23:446-462. [CrossRef] [Medline]
  20. Tricco AC, Ivers NM, Grimshaw JM, Moher D, Turner L, Galipeau J, et al. Effectiveness of quality improvement strategies on the management of diabetes: a systematic review and meta-analysis. Lancet 2012 Jun 16;379(9833):2252-2261. [CrossRef] [Medline]
  21. Coiera E, Ash J, Berg M. The unintended consequences of health information technology revisited. Yearb Med Inform 2016 Nov 10(1):163-169. [CrossRef] [Medline]
  22. Kennedy CM, Powell J, Payne TH, Ainsworth J, Boyd A, Buchan I. Active assistance technology for health-related behavior change: an interdisciplinary review. J Med Internet Res 2012;14(3):e80 [FREE Full text] [CrossRef] [Medline]
  23. Isern D, Moreno A. A systematic literature review of agents applied in healthcare. J Med Syst 2016 Feb;40(2):43 [FREE Full text] [CrossRef] [Medline]
  24. Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implement Sci 2007 Nov 30;2:40 [FREE Full text] [CrossRef] [Medline]
  25. Helfrich C, Damschroder L, Hagedorn H, Daggett G, Sahay A, Ritchie M, et al. A critical synthesis of literature on the promoting action on research implementation in health services (PARIHS) framework. Implement Sci 2010 Oct 25;5:82 [FREE Full text] [CrossRef] [Medline]
  26. Cotterill S, Knowles S, Martindale A, Elvey R, Howard S, Coupe N, et al. Getting messier with TIDieR: embracing context and complexity in intervention reporting. BMC Med Res Methodol 2018 Jan 18;18(1):12 [FREE Full text] [CrossRef] [Medline]
  27. Coiera E, Ammenwerth E, Georgiou A, Magrabi F. Does health informatics have a replication crisis? J Am Med Inform Assoc 2018 Apr 13:963-968. [CrossRef] [Medline]


BP: blood pressure
RCT: randomized controlled trial


Edited by G Eysenbach; submitted 27.04.18; peer-reviewed by N Archer, C Granja; comments to author 25.08.18; revised version received 09.04.19; accepted 26.04.19; published 17.06.19

Copyright

©Kathleen Yin, Liliana Laranjo, Huong Ly Tong, Annie YS Lau, A Baki Kocaballi, Paige Martin, Sanjyot Vagholkar, Enrico Coiera. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 17.06.2019.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.