Effectiveness of an Electronic Communication Tool on Transitions in Care From the Intensive Care Unit: Protocol for a Cluster-Specific Pre-Post Trial

Background Transitions in care are vulnerable periods in health care that can expose patients to preventable errors due to incomplete or delayed communication between health care providers. Transitioning critically ill patients from intensive care units (ICUs) to other patient care units (PCUs) is particularly risky, due to the high acuity of the patients and the diversity of health care providers involved in their care. Instituting structured documentation to standardize written communication between health care providers during transitions has been identified as a promising means to reduce communication breakdowns. We developed an evidence-informed, computer-enabled, ICU-specific structured tool—an electronic transfer (e-transfer) tool—to facilitate and standardize the composition of written transfer summaries in the ICUs of one Canadian city. The tool consisted of 10 primary sections with a user interface combination of structured, automated, and free-text fields. Objective Our overarching goal is to evaluate whether implementation of our e-transfer tool will improve the completeness and timeliness of transfer summaries and streamline communications between health care providers during high-risk transitions. Methods This study is a cluster-specific pre-post trial, with randomized and staggered implementation of the e-transfer tool in four hospitals in Calgary, Alberta. Hospitals (ie, clusters) were allocated randomly to cross over every 2 months from control (ie, dictation only) to intervention (ie, e-transfer tool). Implementation at each site was facilitated with user education, point-of-care support, and audit and feedback. We will compare transfer summaries randomly sampled over 6 months postimplementation to summaries randomly sampled over 6 months preimplementation. The primary outcome will be a binary composite measure of the timeliness and completeness of transfer summaries. Secondary measures will include overall completeness, timeliness, and provider ratings of transfer summaries; hospital and ICU lengths of stay; and post-ICU patient outcomes, including ICU readmission, adverse events, cardiac arrest, rapid response team activation, and mortality. We will use descriptive statistics (ie, medians and means) to describe demographic characteristics. The primary outcome will be compared within each hospital pre- and postimplementation using separate logistic regression models for each hospital, with adjustment for patient characteristics. Results Participating hospitals were cluster randomized to the intervention between July 2018 and January 2019. Preliminary extraction of ICU patient admission lists was completed in September 2019. We anticipate that evaluation data collection will be completed by early 2021, with first results ready for publication in spring or summer 2021. Conclusions This study will report the impact of implementing an evidence-informed, computer-enabled, ICU-specific structured transfer tool on communication and preventable medical errors among patients transferred from the ICU to other hospital care units. Trial Registration ClinicalTrials.gov NCT03590002; https://www.clinicaltrials.gov/ct2/show/NCT03590002 International Registered Report Identifier (IRRID) DERR1-10.2196/18675

year two. The environment is suitable to support the project, and I do not see any issues with the feasibility of completing the project according to the proposed timeline.

Impact of Research
The dissemination plan for the project is good with respect to engaging stakeholders and preparing for provincial/national scale. However, the dissemination plan academically to the scientific community is weak. The proposed work, if successful, may have a significant impact on transitions in care provincially and nationally as well.

SUMMARY
The proposed research is designed to address a key gap in the transition of patients from the ICU to hospital ward or to home. With the support of key stakeholders and previous funding obtained from CIHR and the Canadian Frailty Network, the team has developed a nonproprietary electronic ICU Transfer Tool. On the basis of pilot testing that has been completed with this tool, there are 2 phases of research that are proposed. In Phase 1, the team will conduct an adapted non-blinded pre-post randomized cluster (hospital-level) stepped-wedge design to evaluate the quality of ICU transfer. Atrained auditor will review the presence /absence of 8 medical information fields on the transfer record and score these on 5 dimensions: organization, completeness, pertinence, effectiveness and understanding of the patient care plan. A composite measure of completeness and timeliness will be derived as the outcome variable. In Phase 2, a KT Tool Kit will be developed to facilitate dissemination of this ICU transfer tool across Alberta. This phase will follow input from focus groups with key stakeholders. Surveys will also be conducted using purposive sampling. Findings from Phases 1 and 2 will be disseminated to key stakeholders to support subsequent roll out in Alberta and beyond.
Scientific rationale. The literature review provides a clear rationale as to the need for this transition in care proposal. It highlights that there is a gap in the transition from ICU to the hospital ward or home. It notes that impaired communication is a "root cause" for this gap which results in preventable medical errors, redundant testing, readmissions to the ICU or hospital, and dissatisfaction with quality of care. The team has consulted with key stakeholders and secured previous funding. Moreover, pilot testing has been completed. Positive results were reported for the benefit of the electronic ICU Transfer Tool vs. a "dictated cohort", which is the current standard of care-although this pilot study was not referenced as a presentation or publication.

RESEARCH APPROACH Strengths.
This proposal was a pleasure to read. The literature review presents evidence to support the need for this proposed research in a succinct manner, while effectively demonstrating the need for this proposed transition in care research. The research design for Phase 1 and 2 is clearly presented, appropriate, and well documented with regard to objectives, procedures, design models, outcomes, and analyses. Additional the fact that ICU and frontline providers blinded to the study details of the procedures that stand out include will rate the quality of ICU transfer summaries, and their satisfaction with documentation practices (Data Collection/Sources section). In addition, the inclusion of all patients admitted to the ICU for this study will ensure the generalizability of the findings-which is important for scaling this procedure to other hospitals outside of Alberta. The feasibility of this project is extremely well documented in terms of the background range of inter-professional and multidisciplinary expertise of the team, previous engagement with stakeholders, completion of previous pilot research, and the KT plan for disseminating findings.

Weaknesses.
Outcomes: The efficacy of the ICU Transfer Tool will be evaluated using a "trained auditor who will review the presence/absence of 8 medical information fields on the transfer record and score these on 5 dimensions: organization, completeness, pertinence, effectiveness and understanding of the patient care plan. Given the importance of this evaluation for the Phase 1 project, the strength of this study would have been increased with evaluations being conducted by at least 2 independent auditors, where their ratings could be reviewed for level of agreement, and where any differences in ratings could be resolved by consensus involving a 3 rd party. This would improve the actual and perceived objectivity of this assessment, while providing information on the reliability of the ratings about how the device is used in practice.
The proposal notes that the outcome will be derived from 8 medical information fields on the transfer record and scored on 5 dimensions: organization, completeness, pertinence, effectiveness and understanding of the patient care plan. A composite measure of completeness and timeliness will be derived and then recoded into a binary variable to reflect the presence/absence of four key information fields: goals of care, diagnosis, active problem list and medications to continue. This assessment will also reflect whether the record is available at time of transfer from the ICU. This process represents several steps in the merging and recoding of outcome data. This section would have been strengthened with a clear definition of the criteria by which each of the final 4 information fields would receive a positive score using the binary code. Further, I wondered whether a scoring index on a continuous (vs binary) scale would be more informative about clinical practice.
It is also noted that the research team will engage stakeholders and opinion leaders at each of the 4 participating hospitals (unit managers, clinicians and nursing staff) to facilitate adoption of the ICU Transfer Tool by physicians, nurse practitioners, and nurses: "…critical care leaders…will encourage clinicians to use the ICU Transfer Tool as the default system." Adherence of staff to use this tool is a potential issue -which (in fairness) is addressed in this proposal. Nevertheless, in preparation for scaling the adoption of this electronic tool, it would seem advisable to collect and analyze usability data from the Phase 1 study. This would provide requisite information on the rate at which the ICU Transfer Tool was utilized by staff.
The analyses for this investigation are not well described, which was surprising given the pilot work that has already gone into this initiative. It was surprising to see that the outcomes for Phase 1 of this research will be evaluated using univariable analyses: chi-squared test, t-tests, and Wilcoxon rank sum test. The data analysis section states that sensitivity analyses will adjust for patient and hospital characteristics using logistic or linear regress models, but how this will be conducted remains vague. This section would have been markedly improved if the primary outcome analysis was described in terms of a multivariable analysis (e.g. the regression models) with an indication how confounding factors (moderators or mediators) would be incorporated into the model. Further, given the complexity of the ICU population, it is likely that the primary outcome will be analysed using several regression models. A brief summary of the key models to be used would have been helpful.

Applicants:
The PI (Dr. Parsons Leigh) is an career investigator with expertise in sociology, early new knowledge acquisition and collaboration, critical care, and knowledge translation.Over the past 7 years, peer reviewed funding for which she has been a Co-Applicant is approximately $4,630,000. Within this total, she has had success as a PI, with approximately $530,000. Over the past 7 years, Dr. Parsons Leigh has 22 peer-reviewed publications, with 8 as first or last author. She is Supervisor or Academic Advisor for 2 MSc currently students. Other members of the team have demonstrated expertise in patient and family engagement, critical care medicine, nursing, palliative care, geriatrics, primary care, rehabilitation, health economics, implementation science, and biostatistics. They hold senior positions with key stakeholder groups and they have an impressive record of publications (e.g. NEJM, JAMA, Lancet) for studies conducted at the local, national and international level. They report approximately $119 million in funding as PI's.

Budget
No issues.

Impact of Research and KT.
The knowledge translation plan is clearly presented. There is strong support from key stakeholders (Alberta Health Services, Canadian Critical Care Trials Group/Society and participating hospitals). KT plans for the ICU transfer Tool include making it available via open access as a non-proprietary, standardized, evidenceinformed electronic ICU suitable for national and international application. Further, the team will engage knowledge users and knowledge brokers through professional and organizational methods (e.g. newsletters), and they will host a national forum ontransitions of patient care to link researchers with knowledge users. The ultimate goal is to influence the standard of practice by rolling out this ICU Transfer Tool and the KT Tool Kit to national and international settings.   Research Questions: The applicant proposes a clearly formulated research question related to the ICU transfer tool evaluation. There is strong evidence supporting the pursuit of this particular question which is relevant in the context of the funding application. The 2nd topic is not a research question but rather the proposed solution to be developed based on findings in phase 1. Research Approach: for the phase 1 of the proposed project, the proposed methodology is appropriate and supports the objectives of the research. Approach to data collection and available outcomes for evaluation are feasible. The significance threshold for decision makers appears rather modest (15% absolute increase). Phase 2 of the project consists of the development of KT tool kit to support spread and scale -not a research proposal but the development of the proposed scalable solution enabled by results in phase 1. For the stated purpose of phase 2, the proposed approach is reasonable and achievable. However it is not clear what happens if the pre-defined significance threshold is not achieved. The phase 2 should be conditioned by phase 1 outcomes. Potential confounders of phase 1 results as well as other limitations that can be created by e.g. center selection bias are not described. Applicants: qualified team with appropriate representation of key stakeholders for the proposed transition in care Feasibility: proposed budget is reasonable vs proposed activities, environment for research is appropriate and ensures access to key data required for the research to be completed, risks identified and risk mitigation strategies proposed, timelines are achievable. Phase 2 of the project should be conditioned by outcomes of phase 1. Impact of Research: publication dissemination plan beyond communication to key stakeholders and development of the KT tool in phase 2 is not detailed. However, the applicant is committed to develop and roll-out, based on findings in phase 1, the KT tool at provincial and national level with the clear aim to create actionable health system changes to improve care transitions for ICU admitted patients in Canada.
This proposal evaluates an electronic tool aimed at improving communication between ICU and other medical staff for individuals being discharged from the ICU. Strengths noted by the committee included the rationale for the proposed work, the previous pilot work that this proposal builds on, the good fit with the CIHR call, the engagement of KUs in the dissemination plan, and the theoretically informed and evidencebased approach. The fact that this tool is non-proprietary and the framing of the ICU as a survivable environment were also noted as pluses.
Committee discussion included whether more nuanced outcomes measures should be considered, whether treating the three sites as separate RCTs was consistent with a stepped-wedge design, the need for a more fulsome description of how the progress from focus group to surveys would occur, and the need for KUs to be involved in the evaluation phase as well. The value of adding another trained auditor was discussed as was the value of having a template to support the extensive data reduction process described. More information on the adherence of staff to using the tool would have been helpful. How potential mental health impacts of the ICU would be captured and addressed and also how medication discontinuation postdischarge might be handled were also raised.
Finally, the strength and thoroughness of the phase 1 description was not matched by the phase 2 description. For example, how the Donabedian framework or the Triple Aim are actually operationalized could use more description. Also, more clarity on how phase 1 would influence phase 2 would have been helpful.
Attention to these concerns would strengthen this already strong proposal.