Developing Strategies to Reduce Unnecessary Services in Primary Care: Protocol for User-Centered Design Charrettes

Background Overtreatment and overtesting expose patients to unnecessary, wasteful, and potentially harmful care. Reducing overtreatment or overtesting that has become ingrained in current clinical practices and is being delivered on a routine basis will require solutions that incorporate a deep understanding of multiple perspectives, particularly those on the front lines of clinical care: patients and their clinicians. Design approaches are a promising and innovative way to incorporate stakeholder needs, desires, and challenges to develop solutions to complex problems. Objective This study aimed (1) to engage patients in a design process to develop high-level deintensification strategies for primary care (ie, strategies for scaling back or stopping routine medical services that more recent evidence reveals are not beneficial) and (2) to engage both patients and primary care providers in further co-design to develop and refine the broad deintensification strategies identified in phase 1. Methods We engaged stakeholders in design charrettes—intensive workshops in which key stakeholders are brought together to develop creative solutions to a specific problem—focused on deintensification of routine overuse in primary care. We conducted the study in 2 phases: a 6.5-hour design charrette with 2 different groups of patients (phase 1) and a subsequent 4-hour charrette with clinicians and a subgroup of phase 1 patients (phase 2). Both phases included surveys and educational presentations related to deintensification. Phase 1 involved several design activities (mind mapping, business origami, and empathy mapping) to help patients gain a deeper understanding of the individuals involved in deintensification. Following that, we asked participants to review hypothetical scenarios where patients, clinicians, or the broader health system context posed a barrier to deintensification and then to brainstorm solutions. The deintensification themes identified in phase 1 were used to guide phase 2. This second phase primarily involved 1 design activity (WhoDo). In this activity, patients and clinicians worked together to develop concrete actions that specific stakeholders could take to support deintensification efforts. This activity included identifying barriers to the actions and approaches to overcoming those barriers. Results A total of 35 patients participated in phase 1, and 9 patients and 7 clinicians participated in phase 2. The analysis of the deintensification strategies and survey data is currently underway. The results are expected to be submitted for publication in early 2020. Conclusions Health care interventions are frequently developed without input from the people who are most affected. The exclusion of these stakeholders in the design process often influences and limits the impact of the intervention. This study employed design charrettes, guided by a flexible user-centered design model, to bring clinicians and patients with differing backgrounds and with different expectations together to cocreate real-world solutions to the complex issue of deintensifying medical services. International Registered Report Identifier (IRRID) RR1-10.2196/15618


KEY SUMMARY POINTS:
The area of focus is on de-intensification in primary care is of high concern to the VA. Strong investigative team with excellent partnership with VA clinical leaders Very responsive to the RFA and to prior critiques. This is a high-risk and high reward study. They have provided many details on which measures of treatment will be operationalized. Multiple-PI is justified due to the distinct scope of work outlined. Comment: Sample size for some of the proposed measures may be problematic, especially in terms of evaluating individual providers. However the reviewers acknowledge that it's hard to do a power calculation because the measures are not yet firm

DESCRIPTION (provided by applicant):
Providing appropriate health care means ensuring that patients get the care they need while avoiding care that is unnecessary or harmful. In the 1990s, VHA's first transformation focused on increasing the use of necessary services through a performance management system that has become a panoply of guidelines, performance measures, reminders, registries, and incentives. As a result, the underuse of necessary services in VHA decreased dramatically. Today, there is growing interest in identifying overuse -care that exposes patients to services that are not beneficial or may cause harm and which may take scarce resources away from those who would benefit from them. The national Choosing Wisely campaign is one manifestation of this trend. While this campaign has garnered attention, it has not produced systematic or reproducible approaches to identify overuse. Within the general realm of overuse, a more focused target is promoting de-intensification when good quality practice calls for decreasing the intensity or frequency of medical services that are currently part of a patient's ongoing management. Increasingly, we recognize that care is too frequent or too intensive in cases where the marginal benefit is absent or there is potential for patient harm, and that both providers and patients have particular difficulty stopping routine testing and treatments once they have successfully become part of a patient's regular care. One example is intensive glycemic management among diabetic patients who are elderly or have major comorbidities. Yet, most quality monitoring and improvement initiatives continue to provide largely unopposed incentives to escalate care intensity. The proposed study will: 1) identify and validate clinical indications for de-intensification in primary care (Aim 1); 2) assess the prevalence and reliability of measures of de-intensification in VHA (Aim 2); and 3) develop multi-level strategies to disseminate and implement de-intensification measures (Aim 3). Aim 1 will involve an environmental scan to identify a preliminary list of potential de-intensification indications; a rapid evidence synthesis of existing evidence supporting candidate de-intensification indications; a high level prevalence estimate of opportunities for de-intensification in VHA; and, expert panels to rate each of the potential indications on improvement opportunity, validity, and feasibility of measurement and implementation. Aim 2 will include constructing measures of de-intensification indications and conducting an analysis to examine prevalence, variance, and reliability of measuring de-intensification in VHA. AIM 3 will involve engaging providers, clinical managers, and patients in collaborative decisionmaking to develop an operational measurement procedure for disseminating and implementing the identified de-intensification measures. Findings from each of the Aims will be synthesized into practical intervention strategies. An advisory council, comprised of VHA and national policy, content and measurement experts, will work with the research team in all stages to review results and provide guidance on indication measurement prioritization, feasibility, implementation, and dissemination. Identifying, measuring, and understanding how to promote de-intensification -to complement the many measures promoting appropriate intensification -is critical to restoring balance to VHA's efforts to improve care quality. This study will develop a systematic method to identify when de-intensification is called for, assess how frequently de-intensification occurs in VHA, and provide approaches to reliably measure and deploy measures of de-intensification. The project is unique by involving all stakeholders, including clinical and policy experts, front-line providers, clinical leaders, and patients in developing an operational measurement procedure and implementation approach that can ensure reliable measurement, promote quality improvement, and avoid organizational dysfunction. Responsiveness to the Priority Areas Outlined in this RFA.
The proposal fits squarely within the goals of the RFA; namely, to develop novel measures that address overuse of low-value health care services. The proposal also characterizes sources of variation in the proposed measures, which is another goal of the RFA. However, the RFA also states: "It is expected that this RFA would fund initiatives that develop innovative approaches through use of inter-disciplinary research teams comprised of expertise in computer science, statistics, engineering, mathematics, clinical epidemiology/decision-making, social sciences, and health economics with the aim to turn existing data into knowledge for improved healthcare and health." I do not see many of these disciplines other than a biostatistician and clinical psychologist, the latter at 7.5% effort.

Approach.
The approach is systematic and well-conceptualized. Strengths include the use of an evidence synthesis, involved operational partners, a modified Delphi process, and extensive patient and provider engagement.
One main concern with the approach is whether it will generate a substantial number of valid and important performance measures. Will these be a large enough denominator for overuse measures to assess facility, practice and provider variation? This has been a challenge with developing such measures in the past. For instance, while a measure of underuse of colorectal screening would apply to a large population of most adults age 50 -79, the measure of overuse would apply to a much more selected population of adults with limited life expectancy. Presumably the variation across sites would be much lower for such a measure. Many of the Choosing Wisely measures apply to very small and highly selected populationswhich may not yield the necessary sample size for reliable reporting at all the different levels proposed in this study. Power and sample size calculations for a few candidate measures (even if these are not ultimately selected for prioritization in the study) would greatly enhance the confidence that this project will be actionable for measurement and reporting in the VA. Based on the analyses reported in Appendix 1, the diabetes measure is likely to have sufficient sample size, but to this reviewer, it is unclear how many additional indicators of overuse will be valid for assessment at the provider, practice, or even VAMC level.
It is unclear how the proposed examples of overuse on page 27 (Table 2) could be developed into performance indicators. For example, the USPTF recommends a shared decision-making approach regarding PSA testing. How would this be operationalized into a performance metric? Some of the other exampleslipid testing, frequency of a1c testinginvolve tests with extremely low costs; or have equivocal evidence about when to discontinue screening. Again, this raises concerns about just how many meaningful measures of de-intensification will arise from this study.

Innovation and impact.
There are a number of features in this proposal that are innovative. Most notably, the development of a comprehensive set of measures to assess de-intensification and profile variation across facilities has not been done. Although focused in the VA, the project would have impact in other health systems given the recent policy focus on developing measure of overuse and the Choosing Wisely campaign. The advisory board of VA operational leaders and representative from KP will also enhance the potential impact of the project. The PIs are exceptionally well-qualified to carry out the proposed research. Also, this work builds on the previous work of the PIs and the Ann Arbor COIN. My one concern is that there are 6-7 co-investigators all funded in the 5-10% range, several of whom appear to have overlapping skill sets. It is not clear why each of these individuals are involved and what all of their specific role on the project will be. This could lead to a diffusion of responsibility.
The multiple PI plan seems appropriate given the discrete set of tasks each PI will oversee. The two PIs have an extensive track record of collaboration

Adequacy of Response to Previous Feedback Provided by HSR&D Regarding the Proposed Study.
The authors thoroughly responded to the prior critiques from the previous review.

Protection of Human Subjects from Research Risk.
I see no concerns

Inclusion of Women and Minorities in Research.
Appropriate; the investigators will use random selection, but they may want to consider oversampling women and minority patients if the participation rates are low.

Budget.
This project has a very large budget beyond the typical limit for an IIR. However, there is a large scope of work.

Overall Impression.
This is an innovative project that addresses an significant issue for VA and other health care systems. The approach seems likely to have an impact given the involvement of patient and provider stakeholders. The enthusiasm is somewhat diminished by concern that there will be a large number of measures with adequate sample size to profile meaningful variation across providers and facilities.

Key Strengths.
1. Area of high concern to VA and other health systems Significance (including Importance of the Problem Addressed).
The goal of de-intensifying care when appropriate is very current and has achieved broad consensus as a priority, especially within VA. In large part due to work by the applicants and others, we now have pretty clear evidence that our pursuit of control targets and quality measures has the unintended consequences of overtreatment for a fairly large number of our patients. VA is already working on several initiatives to address de-intensifying care when appropriate, such as the hypoglycemia safety initiative. In this proposal, the applicants plan to do foundational/conceptual work to better understand the context of de-intensification and also how such efforts may translate into performance measures. They make the point, correctly, that we have focused so single-mindedly on doing more for patients that little is known or understood about how we would even go about doing less. While VA can and should proceed with efforts to promote de-intensification when appropriate, it seems like a good idea to have this sort of foundational work going on as well. To use somewhat casual language, while firing from the hip may hit the target, if it does not, VA would do well to have funded a group to already be looking into where exactly we should be aiming in the first place.

Approach (including Feasibility).
There are 3 aims 1) to identify and validate indications for de-intensification 2) to assess the prevalence and reliability of measures of de-intensification, and 3) to develop strategies to implement deintensification measures. Aim 1 will include a "scan" to locate all likely candidate targets for deintensification, rapid evidence synthesis about each, and a modified Delphi panel to help prioritize the targets with the highest evidence support and perceived importance. The use of ThinkTank to help facilitate meaningful conversations while avoiding the hassles of VA travel makes sense. Aim 2 will include efforts to actually specify performance measures based on the Aim 1 work, including comparing to a gold standard of EMR review and routine measures of prevalence, variance, and reliability. This is important work to understand how such measures will work in practice before deploying them. In Aim 3, the investigators will pivot to planning for effective implementation of the measures, incorporating views gathered from patients, VA leaders, and clinicians to help inform the effort.
One important critique which I have of the general direction of this research effort. The investigators seem to imagine, above all, that the VA needs opposing sets of measuressuch as patients with too high a blood glucose, and patients with too low a blood glucose. This can work OK, and is logical. But I wonder if VA needs twice as many performance measures as we have now. It may also add to a sense among already-beleaguered clinicians that they cannot win, as they face "failure" on either side (too high/too low). While I do not have an obvious answer, I wonder if opposing sets of measures is really the best approach to take. Isn't there a way to produce fewer, more integrated measures? I realize this risks eroding the actionability of the measure, if you can't easily tell which patients are too high vs. too low, for instance. But maybe there is a better way than just doubling the number of measures we have.

Impact and Innovation.
The proposal details a plan for careful and at times heavily conceptually-informed and foundational work. While HSR&D has a growing demand to always produce things quickly, sometimes careful work takes time to do. The investigators do make the point that they will make sure to work on a few high priority measures early (as assessed by their advisory panel of VA leaders), to have these measures ready for roll-out in 2.5 years if possible. The proposal is not "innovative" in the sense that well-1 I01 HX001895-01A1 6 HS5A HOFER, T; KERR, E established methods will be used. However, these methods have not been systematically applied to this important question, and if VA does not invest in such an effort now, they may regret it in 3 years when they are begging for the kind of information this project is likely to deliver. As our recent experience with performance metrics can attest, deploying performance metrics without careful forethought can have unforeseen consequences for our VA system.

4.
Investigator Qualifications, and Facilities and Resources.
As was stated in the previous review, this group of researchers has a superb track record doing just this kind of research, of working together, and of producing really high-quality products. The facilities and resources are well-matched to the goals of the project. I have a high degree of confidence that they can successfully complete this work.
The multiple PI plan between Drs. Kerr and Hofer is appropriate and credible.

Adequacy of Response to Previous Feedback Provided by HSR&D Regarding the Proposed Study.
The investigators have been responsive to the critiques. They appropriately grasped the major underlying issues which were raised and have addressed them directly. There are limits to how much they can change the project, but they have made small moves in the direction of the reviewer comments where appropriate. While the project remains large (in budget) and long (in time frame), they have nodded to the need to produce some products as early as possible, and I believe they will do this. They demonstrate meaningful linkages with VA leaders, through their advisory panel and by having recently joined some key efforts like the VA Choosing Wisely group (Dr. Kerr). These connections will ensure that the findings will have an impact on VA practice.

Responsiveness to Research Priorities or Special Solicitations.
This proposal is highly responsive to the RFA HX15-023. The potential of this proposal to add overtreatment measures to our regime of performance measurement has the potential to help correct a one-sided and unbalanced system of measurement. The contributions to assessing the operating characteristics (reliability etc.) of the new measures also will help ensure that they are properly specified and minimize unintended consequences. These would appear to be important contributions that help address the third point of the RFA, namely "A third challenge is that our current performance measure system, and the underlying data, need substantial revisions if we are to accurately measure quality of care and promote improvement where it is most needed."

Protection of Human Subjects from Research Risk.
Adequate/no concerns.