The Influence of Context on Implementation and Improvement: Protocol for a Mixed Methods, Secondary Analyses Study

Background Caring for the well-being of older adults is one of the greatest challenges in modern societies. Improving the quality of care and life for older adults and the work lives of their care providers calls for effective knowledge translation of evidence-based best practices. Objective This study’s purpose is to contribute to knowledge translation by better understanding the roles of organizational context (workplace environment) and facilitation (process or role) in implementation and improvement success. Our study has 2 goals: (1) to advance knowledge translation science by further developing and testing the Promoting Action on Research Implementation in Health Services framework (which outlines how implementation relies on the interplay of context, facilitation, and evidence) and (2) to advance research by optimizing implementation success via tailoring of modifiable elements of organizational context and facilitation. Methods This is secondary analyses of 15 years of longitudinal data from the Translating Research in Elder Care (TREC) program’s multiple data sources. This research is ongoing in long-term care (LTC) homes in western Canada. TREC data include the following: 5 waves of survey collection, 2 clinical trials, and regular ongoing outcome data for LTC residents. We will use a sequential exploratory and confirmatory mixed methods design. We will analyze qualitative and quantitative data holdings in an iterative process: (1) comprehensive reanalysis of qualitative data to derive hypotheses, (2) quantitative modeling to test hypotheses, and (3) action cycles to further refine and integrate qualitative and quantitative analyses. The research team includes 4 stakeholder panels: (1) system decision- and policy makers, (2) care home managers, (3) direct care staff, and (4) a citizen engagement group of people living with dementia and family members of LTC residents. A fifth group is our panel of external scientific advisors. Each panel will engage periodically, providing their perspectives on project direction and findings. Results This study is funded by the Canadian Institutes of Health Research. Ethics approval was obtained from the University of Alberta (Pro00096541). The results of the secondary analyses are expected by the end of 2023. Conclusions The project will advance knowledge translation science by deepening our understanding of the roles of context, the interactions between context and facilitation, and their influence on resident and staff quality outcomes. Importantly, findings will inform understanding of the mechanisms by which context and facilitation affect the success of implementation and offer insights into factors that influence the implementation success of interventions in nursing homes. International Registered Report Identifier (IRRID) DERR1-10.2196/40611

The team will explore sex and gender in their analysis. The key consideration here is that their existing dataset includes information on sex, allowing them to investigate the sex composition of teams on implementation success, for example.

Review Type / Type d'évaluation:
Reviewer 1 / Évaluateur 1 Name of Applicant / Nom du chercheur: Estabrooks, Carole Anne This IKT project will explore context and facilitation related to uptake of research, organizational performance, and health outcomes. The project is structured as a sequential exploratory and confirmatory mixed-methods design, taking place over 42 months. At the end of the project, the ways in which context can influence success of implementation of quality improve initiatives, and the conditions under which facilitation affects staff quality of worklife outcomes as well as resident outcomes. This project focuses on two critical factors that influence the success of implementation and improvement efforts, as well as, resident and care staff outcomes. These are: a) modifiable features of organizational context, and b) modifiable aspects of facilitation roles and processes (argued to improve uptake of research, organizational performance and health outcomes).
Our specific aims are to: 1. Identify how context influences success of implementation and of quality improvement initiatives (focusing on our successful trials).
2. Identify and map context conditions in which facilitation affects staff quality of worklife outcomes (e.g., burnout, satisfaction, work engagement).
3. Identify and map context conditions in which facilitation affects resident outcomes (e.g., care quality).
A mixed methods project involving secondary data analysis is proposed. TREC data from multiple sources, in multiple forms, collected from 2007-2019 are available. Data holdings include quantitative data (continuously collected administrative data using the RAI-MDS 2.0 instrument, 5 waves of primary surveys from multiple care provider groups in the residential long term care sector (15,000 cases to date)) and extensive qualitative data (case studies, extensive process evaluations from 2 large trials). We will use a variety of analytic methods that account for our data clustering (quantitative) and a variety of qualitative techniques commonly used in mixed methods research (e.g., constant comparison).
Expected outcomes include: 1)New knowledge about how context and facilitation influence the success of implementation and improvement successes, and about how they interact to do this; 2) advances to the PARIHS framework specifically and to the broader literature on the mechanisms by which context and facilitation influence implementation and improvement processes; 3) a stronger evidence base for designing interventions in large pragmatic implementation and quality improvement trials; 4) advances in the utility and practical application of this work in complex health systems. The Influence of Context on Implementation and Improvement

Strengths and Weaknesses/Forces et faiblesses:
25% Significance and Impact of the Research:

Strengths
•Key terms, concept and facilitation, defined.
•iKT approach taken, longstanding partnership with KUs from LTC setting.
•Evidence supporting the link between context and facilitation and 3 categories of outcomes: success of improvement initiatives, quality of care for LTC residents, and quality of worklife for paid staff, provide a solid rationale for the proposed research.
•Strong rationale for targeting the LTC setting given that residents experience a high rate of burdensome symptoms and inappropriate care at end of life, and unregulated care aides are at high risk for negative mental health outcomes.
•An overview of the PARIHS framework is provided and its weaknesses identified. Theories and frameworks used by the research team to complement the guidance provided by PARIHS are listed.

Weaknesses
•A rationale for selecting the PARIHS framework over other frameworks for examining context and facilitation would strengthen the rationale for this research.
•A brief description of the data that are available for the proposed research in the background section would enable the reader to better understand the proposed project goals and aims which are somewhat non-specific. For example what types of quality improvement initiatives are the focus of specific research aim 1? In aim 2, does "staff" refer to all staff in the LTC setting? Or just unregulated care aides? Aim 3 would be improved by specifying the outcomes of interest.
•Presumably all project goals and aims relate the LTC setting but this is not stated.

Strengths
•Feasibility supported by availability of data in this secondary data analysis.
•Data are from 94 LTC facilities across 4 western provinces which support the external validity of the results.
• Table 1 indicates that extensive data are available which suggests the project will yield robust results.
•Central housing of the data in the Health Research Data Repository (University of Alberta Faculty of Nursing) facilitates the proposed research.
•The proposed statistical analysis accounts for the clustered nature of the data at multiple levels.
•Use of panels of internal and external stakeholders, including a citizen engagement group of people with dementia, caregivers of LTC residents, and international scientists, throughout the research provides strong evidence of the iKT approach and will help ensure the relevance of the research.
•Some potential risks and mitigation strategies are identified. The Influence of Context on Implementation and Improvement •The proposed methods are difficult to follow. A high level overview of the TREC data available is first provided, and limited detail as to how the outcomes (staff quality of worklife, quality of resident care) were measured is not given until page 8. A comprehensive description of the quantitative data (ie constructs/variables), the quantitative measurement tools used, and the measurement properties of those tools is not provided making it difficult to judge the quality of the data.
•No description is given of the aims of the interviews that produced the qualitative data that would enable a judgment of the feasibility of the qualitative data matching onto the quantitative data as proposed in figure 1. The RAI-MDS 2.0 is the only measure named in the methods section.
•A description of variables and how they were measured is needed prior to the quantitative data analysis section to evaluate the appropriateness of the analysis proposed.
•The "panels" mentioned in Figure 1 on page 5 are not clarified until page 7 making the proposed methodology difficult to follow.

Strengths
•NPA Estabrooks holds a Tier 1 Canada Research Chair and has strong expertise and experience related to the focus of the proposed research (examining the influence of context in LTC settings, use of PARIHS) as supported by numerous publications and grants.
•The team has extensive experience with the proposed quantitative statistical analyses proposed.
•This team has depth and breadth in disciplinary composition, the methodological expertise, and a rich history of partner engagement. The researchers include senior, mid-career and junior investigators and will enable the introduction of postdoctoral trainees into the project, and other trainees as research assistants.
•The team has senior decision making partners and end users and the methodological approach reflects extensive engagement of policy makers, system managers, residents, their caregivers, and an external panel of substantive experts.

Weaknesses
•Whitney Berta is mentioned as a co-applicant but she is not listed under project details.