Improving Transplant Medication Safety Through a Technology and Pharmacist Intervention (ISTEP): Protocol for a Cluster Randomized Controlled Trial

Background Medication errors, adverse drug events, and nonadherence lead to increased health care utilization and increased risk of adverse clinical outcomes, including graft loss, in solid organ transplant recipients. Veterans living with organ transplants represent a population that is at substantial risk for medication safety events and fragmented care coordination issues. To improve medication safety and long-term clinical outcomes in veteran transplant patients, interventions should address interorganizational system failures and provider-level and patient-level factors. Objective This study aims to measure the clinical and economic effectiveness of a pharmacist-led, technology-enabled intervention, compared with usual care, in veteran organ transplant recipients. Methods This is a 24-month prospective, parallel-arm, cluster-randomized, controlled multicenter study. The pharmacist-led intervention uses an innovative dashboard system to improve medication safety and health outcomes, compared with usual posttransplant care. Pharmacists at 10 study sites will be consented into this study before undergoing randomization, and 5 sites will then be randomized to each study arm. Approximately, 1600 veteran transplant patients will be included in the assessment of the primary outcome across the 10 sites. Results This study is ongoing. Institutional review board approval was received in October 2018 and the study opened in March 2019. To date there are no findings from this study, as the delivery of the intervention is scheduled to occur over a 24-month period. The first results are expected to be submitted for publication in August 2021. Conclusions With this report, we describe the study design, methods, and outcome measures that will be used in this ongoing clinical trial. Successful completion of the Improving Transplant Medication Safety through a Technology and Pharmacist Intervention study will provide empirical evidence of the effectiveness of a feasible and scalable technology-enabled intervention on improving medication safety and costs. Clinical Trial ClinicalTrials.gov NCT03860818; https://clinicaltrials.gov/ct2/show/NCT03860818 International Registered Report Identifier (IRRID) PRR1-10.2196/13821

Background: Organ transplant is the gold-standard treatment for patients with end organ diseases of the kidney, liver, heart and lungs, as it substantially improves survival and quality of life. Over the past 20 years, the use of contemporary immunosuppression has reduced the risk of acute rejection rates by upwards of 80%; yet long-term allograft survival remains suboptimal. Studies have demonstrated that causes of late graft loss is predominantly driven by immunosuppression adverse events and late allograft rejection episodes from medication errors and non-adherence, which encompass issues directly related to medication safety. Our research demonstrates that medication errors occur in nearly two-thirds of transplant recipients, leading to hospitalization in 1 in 8 recipients. Recipients that develop significant medication errors are at considerably higher risk of graft loss, leading to higher costs and mortality. Thus, in order to improve medication safety and long-term outcomes in transplant recipients, enhancements in immunosuppressant therapy management is needed.
Objectives: The central hypothesis for the ISTEP study (Improving Transplant Medication Safety through a TEchnology and Pharmacist Intervention) is that pharmacist-led immunosuppressant therapy management, facilitated through the use of innovative technology, will significantly improve immunosuppressant safety and clinical outcomes in Veteran transplant recipients.
Methods: This is a 24-month, prospective, multicenter, cluster-randomized controlled clinical trial at 10 sites, randomizing 5 sites to standard clinical care and 5 to standard care and the technology-enabled pharmacist intervention. The technology component of this intervention consists of the use of an expanded dashboard system that has already demonstrated effectiveness in improving immunosuppression monitoring. The dashboard performs population-level surveillance of transplant recipients and identifies those with potential drug-related problems, including non-adherence, drug interactions, missing and worrisome trends in labs; then providing a real-time alert to the pharmacist, who will determine its relevance and intervene in an appropriate protocol-guided manner. Effectiveness will be determined by comparing the rates of hospitalizations and ER visits between groups, while 1 I01 HX002130-01A2 3 HSR1 TABER, D adjusting for baseline patient, provider and facility characteristics. Secondary measures include comparing healthcare costs and determining dashboard functionality, dashboard actionability and pharmacist intervention types and acceptance rates. We will also assess the overall incidence and severity of drug-related problems and graft and patient survival rates and compare these between the intervention and control sites.
The investigators make a strong case for the needed improvement in care for 12,000 patients receiving immunosuppression. Their prior work shows significant medication "issues" and the high costs and health concerns associated with these issues. The 10 study sites treated 1,689 patients, roughly 14% of the target population (7% in the study group); if this project works, they will have made significant headway in providing appropriate care to affected patients.

Approach.
This two-part study seeks to 1) refine the dashboard tool and train the personnel; 2) evaluate the utility of the tool (Aim 3); and 3) assess the consequences of its use (Aim 1) and its cost-effectiveness (Aim 3).
The investigators propose a randomized design for five study sites and five control sites led by pharmacists to follow patients for two years, with a final year for analyses and dissemination of resultsincluding planning for broader implementation. The approach towards development and training is reasonable. The randomized study is important, recognizing that n=10 and n=1,689; this is both a test of whether pharmacists will use the dashboard as it is whether use of the dashboard and the pharmacists' subsequent intervention makes a difference in use of hospital and emergency department (ED) services.
The cost-benefit analysis will have all investment costs (first year) and operational costs and be limited in benefits to two years, with benefits expected only in the second year of intervention (third study year). Analysis plan for count data of costs (hospitalizations and ED visits) is complete. Analysis plan for cost data include propensity scoring (with unstated criteria) and multivariate analyses that include socio-demographic, donor and transplant characteristics. It is acknowledged that sample size calculations for counts are not likely to hold for cost data.

Innovation.
The innovation is a new dashboard that identifies key medication measures and threshold values (or trend values) for these measures to lead to alerts. Presumably, there is reasonable science behind the expert assessment of values leading to alerts so the development process will be one of compiling information and not requiring new research. The prior development of a laboratory tests dashboard indicates the investigators' skills and awareness of structuring dashboards.
They propose critical evaluation of the dashboard -something that is not often done, but clarity on how this evaluation will be conducted is needed. The "Overhage" method was developed in 1999 and has been cited 103 times, with an unknown number of uses in practice. Its aim is to assess good use of pharmacists' time and activities. This is slightly different from an engineering evaluation of false positives/negatives, timeliness, appropriateness of the threshold and trend values, etc. Still, the 1 I01 HX002130-01A2 4 HSR1 TABER, D evaluation is important to the overall study as the appropriate use of pharmacists' time is a key cost driver to the study. There are many evaluations of computer prompts in healthcare that were not cited.
Some of the dashboard elements are not innovative. Every pharmacy has a set of drug interaction alerts. The innovation is the use of threshold / trend values and whatever intuition or science has been applied to interactions that arise for combinations or dose-dependent concerns.
The pharmacist intervention -acting on the dashboard may or may not be innovative. Abnormal values, drug interactions and medical visits lead to deliberate actions. Expired or lack of refills is a starting point for identifying adherence issues which appear to account for a substantial portion of adverse events, require pharmacists' "social" interventions. The social interventions include motivational interviewing and other educational tools that will be providing at a training session. The evidence is weak on the effectiveness of social interventions by medical professionals and not well presented in the submission.
These are highly qualified investigators with the appropriate facilities and resources. The team is possibly missing a trainer for social interventions. The investigators have no reported record of training on social interventions and have no experts in social work/psychology/education for support.

Multiple PI Leadership Plan.
Not applicable.

Adequacy of Response to Previous Feedback Provided by HSR&D Regarding the Proposed Study.
The prior feedback was very detailed and the investigators provided point-by-point responses that address the feedback.

Protection of Human Subjects from Research Risk.
The protection of human subjects appears to be fine.

Inclusion of Women and Minorities in Research.
The inclusion of women and minorities is fine and consistent with the study population.

Budget.
The budget is fine and consistent with study activity, and much early work has made the dashboard development feasible.

Data Management and Access Plan (DMAP).
The investigators have demonstrated their ability to collect CMS data for these Medicare eligible patients (qualified by ESRD) and VA data. Collection of pharmacists' actions, efforts, and time will be input into CPRS and a REDCap hosted tool. There are challenges associated with reporting that appear to have been reasonably anticipated by the investigators.

Overall Impression.
1 I01 HX002130-01A2 5 HSR1 TABER, D The investigators seek to address medication issues that are an important concern for patients on immunosuppression. A dashboard that highlights issues and directs pharmacist intervention -in an environment where pharmacists can intervene -is a seemingly good tool. The medication and/or system related issues resulting in alerts can be directly addressed by the pharmacist. The nonadherence (on refills, an indirect measure of medication use) issues require pharmacists' use of social interventions, for which the evidence is weak and the investigators may not be fully prepared to deliver. The analysis plan is reasonable, though propensity scoring and multivariate analysis plans are indicated, but not specified. If all goes well, the investigators have provided for good dissemination of the tools.

Key Strengths.
1. The study addresses a high cost, fragile health population, potentially aided in a meaningful way by a new-age intervention.
2. Electronic medical records-based alerts are increasingly common and lessons on use could benefit efforts beyond the study population.
3. Pharmacists in the VA are in a position to undertake direct actions and communications based on alerts -reducing the complexity of the care process for these patients.

A good extension of the work plan of the investigators.
5. Prior work is suggestive that Aim 1 will show significant results and Aims 2 and 3 will show suggestive results.

Key Weaknesses.
1. Dashboards are increasing in frequency and complexity, perhaps leading to alert burnout, though they are taking many efforts to minimize pharmacist burden.
2. Motivational interviewing and educational efforts require substantial training that does not appear to be in the skill sets of the investigations, and exhibit low fidelity in practice even among programs provided by skilled trainers.
3. The cost-benefit analysis will likely only be suggestive, as the variations in costs will likely be very large relative to the sample size. Analysis plan for cost data is not well specified.

Significance.
This proposal builds an extremely strong argument for the importance of addressing medication safety issues among the 12,000 transplant recipients receiving care in the VA with immunosuppressant medication. The potential for both death and other negative clinical outcomes and increased healthcare utilization is significant. Much of the evidence around this topic in the VA comes from these investigators. Further, the proposal recognizes that efforts to address the medication related issues facing transplant patients require a multi-component process. In this case, they are testing having a technology innovation that alerts the pharmacists to potential problems and enhances the clinical pharmacist role and the availability of expert consultation.
This proposal builds an extremely strong argument for the importance of addressing medication safety issues among the 12,000 transplant recipients receiving care in the VA with immunosuppressant medication. The potential for both death and other negative clinical outcomes and increased healthcare utilization is significant. Much of the evidence around this topic in the VA comes from these investigators. This proposal also builds on important evidence for the role of clinical pharmacists and the need to have real-time electronic monitoring of safety issues to aid the role of the pharmacist.
There are important positive aspects to the study design that came out of the previous review.
 The parallel cluster-randomized trial across 10 sites is very appropriate.
 The sites have been specifically identified and heavily involved in planning the grant.
 While the process of making change to the dashboard is very ambitious, the proposal includes a strong argument that the team can accomplish this effort because they are making changes to an already used dashboard and that the investigative team has the ability to quickly agree upon changes.
 The application specifically indicates the differences between the intervention and control arms. The intervention includes three important components: 1) the dashboard; 2) enhances clinical pharmacist role; and 3) availability of expert consultation. It also recognizes that patient adherence and access to the services (e.g., nutrition) needs to be addressed.
 The proposal specifically addresses the extensive training needs of the involved pharmacist.
 Outcomes are quite appropriate. The utilization outcomes are appropriate and quite important. The application includes a strong argument that these should be primary outcomes and that clinical endpoint assessment should be exploratory.
 The analysis section is well written.
There are a couple of issues not really addressed that the investigators that would need to be considered as the application processes are finalized.
 The training session does not indicate that the pharmacists will be trained on the process of addressing patient adherence. It is not clear if the pharmacist will do this with individual patients or if they will provide referrals for this process.
 It is very interesting that this proposal can do a true cost-benefit analysis. While the patient care costs are well described, the process of measuring intervention costs is not described.

Innovation.
The investigators recognize that efforts to address the medication related issues facing transplant patients require a multi-component process. In this case, they are testing having a technology innovation that alerts the pharmacists to potential problems, enhance clinical pharmacist role, and availability of expert consultation. The dashboard goes beyond something that could be used for quality 1 I01 HX002130-01A2 7 HSR1 TABER, D improvement to a tool focused on helping with real time clinical decision making. It is also quite helpful that Aim 3 will focus on the usability and feasibility of the process, including whether the dashboard is providing clinically relevant information.

Investigator Qualifications; Facilities and Resources.
This is a highly qualified team that covers the range of expertise needed to successfully complete this research.

Multiple PI Leadership Plan.
Not applicable.

Adequacy of Response to Previous Feedback Provided by HSR&D Regarding the Proposed Study.
The investigators continue to be exceptionally responsive to reviewer feedback. They specifically address all of the questions and made substantial changes to the proposal that make is easy to follow and more methodologically strong.