Development of a Patient-Reported Outcome Instrument for Patients With Severe Lower Extremity Trauma (LIMB-Q): Protocol for a Multiphase Mixed Methods Study

Background A current limitation in the care of patients with severe lower extremity traumatic injuries is the lack of a rigorously developed patient-reported outcome (PRO) instrument specific to lower extremity trauma patients. Objective This mixed methods protocol aims to describe phases I and II of the development of a PRO instrument for lower extremity trauma patients, following international PRO development guidelines. Methods The phase I study follows an interpretive description approach. Development of the PRO instrument begins with identifying the concepts that are important to patients, after which a preliminary conceptual framework is devised from a systematic literature review and used to generate an interview guide. Patients aged 18 years or above with limb-threatening lower extremity traumatic injuries resulting in reconstruction, amputation, or amputation after failed reconstruction will be recruited. The subjects will participate in semistructured, in-depth qualitative interviews to identify all important concepts of interest. The qualitative interview data will be coded with top-level domains, themes, and subthemes. The codes will then be utilized to refine the conceptual framework and generate preliminary items and a set of scales. The preliminary scales will be further refined via a process of conducting cognitive debriefing interviews with lower extremity trauma patients and soliciting expert opinions. Phase III will include a large-scale field test, using Rasch measurement theory to analyze the psychometric properties of the instrument; shortening and finalizing the scales; and determining the reliability, validity, and responsiveness of the instrument. Results Phases I and II of this study have been funded. Phase I of this study has been completed, and phase II began in January 2019 and is expected to be completed in November 2019. Phase III will begin following the completion of phase II. Conclusions This protocol describes the initial phases of development of a novel PRO instrument for use in lower extremity trauma patients. International Registered Report Identifier (IRRID) DERR1-10.2196/14397

Of equal importance it is concerning that a qualitative researcher was not engaged as a coinvestigator. I'm not clear why there is a statistician on the project since there is no data analysis.
The investigators seek to develop a patient-reported outcome instrument to assess concepts of interest and assess success of reconstructive/limb salvage techniques for patients with severe lower extremity trauma. The study will address an area with need, and the team members are well poised to identify potential study subjects. There are, however, some areas of weakness that could be better defined in the project description.
The decision to perform an amputation or embark on reconstruction is often aided by clinical impressions and are not supported by national guidelines. Moreover, none of the extant research accounts for either patient preference or patient related outcomes. In this study the investigators seek to develop a validated model to answer this question and aide clinicians when managing these injuries.
The goals of the project as well as the approach are rationally proposed and thoughtfully articulated. Potential challenges and alternate approaches have been considered. The project is ambitious, but feasible in the timeframe of the grant at the level of support the grant would provide.
The research team is highly multidisciplinary with both surgery and MSK representation, and includes trainees who are well positioned to benefit from this project, as well external consultants who will provide valuable input. The next steps leading naturally from this proposal are clearly thought through.
My enthusiasm for this project is very high.
In this proposal, the authors propose to develop a patient-reported outcome (PRO) measure for quality of life related to lower extremity trauma. This is an unmet need in the field, in which the ultimate outcomes for patients has a critical lack of a patient-reported metric to report satisfaction and quality of life for lower extremity trauma procedures. Such a PRO, which does not currently exist, would provide a patient-centered, objective metric that would be of significant value to patients and physicians forced with difficult decisions regarding dramatic surgeries related to lower extremity trauma (e.g. amputations vs. reconstruction). The proposed team has expertise in PRO development, lower limb kinetics, and reconstructive and orthopedic surgery and involves both experienced faculty and trainees. The proposed approach will provide the critical preliminary data to validate the new PRO and the budget is adequate to support this endeavor, going to support the cost of these interviews. With the data generated by this project, the team will be in an excellent position to apply for future extramural research funds.

SCORED REVIEW CRITERIA
Reviewers will consider each of the five review criteria below in the determination of scientific and technical merit, and give a separate score for each.

Strengths
• Tries to answer an important question about patient outcomes • Has potential to lead to future funding • Fulfills an as-yet unmet need (no PRO instrument for this patient population) • (From the proposal) Severe lower extremity traumatic injuries are life changing for patients. However, there is a not a clear consensus on the best treatment modality. Treatment options include immediate amputation, delayed amputation after a series of reconstructive attempts, or successful reconstructive limb-salvage. Large prospective trials have shown that successful limb salvage may involve numerous surgeries with a high rate of complications and long-term disability. However, in patients who undergo amputation, the loss of limb is permanent and leads to a life-long reliance on a prosthetic for ambulation. In the setting of modern reconstructive microsurgery, the superior treatment, from the patient perspective, has not yet been established.
• Outcomes in lower extremity trauma patients have traditionally been evaluated using clinician-determined measures, evaluating patients based on themes determined to be important by the researcher. However, engaging patients in the assessment is essential, as appearance, function and psychosocial well-being are outcomes best evaluated by patients. There are currently no patient-reported outcome (PRO) instruments that ask patients to report treatment outcomes on their own behalf. PRO instruments are developed using qualitative patient data, and assess concepts of interest (COI) determined to be relevant to patients, such as satisfaction with appearance, body image, function and psychosocial wellbeing.
• The aim for this study is to begin the development of a well-defined, valid, reliable and responsive PRO instrument for patients treated for lower extremity limbthreatening injuries. The team has designed the project to be the foundation to develop a patient-reported outcome instrument for use in patients with limb-threatening lower extremity injuries to allow for a more patient-focused, evidence-based evaluation of this patient population in research and clinical care. The grant will be used to support initial instrument develop by performing semi-structured qualitative interviews to develop a conceptual framework.
• In this proposal, the authors propose to develop a patient-reported outcome (PRO) measure for quality of life related to lower extremity trauma. This is an unmet need in the field, in which the ultimate outcomes for patients has a critical lack of a patient-reported metric to report satisfaction and quality of life for lower extremity trauma procedures. Such a PRO, which does not currently exist, would provide a patient-centered, objective metric that would be of significant value to patients and physicians forced with difficult decisions regarding dramatic surgeries related to lower extremity trauma (e.g. amputations vs. reconstruction). Weaknesses • The study does not have strong appeal if it doesn't receive additional funding.
• The scope is very specific and not a lot of potential for broadening.
• How will it be used by clinicians?
• What weigh(s) do patient preferences have in decision-making for limb salvage/reconstruction vs. amputation? Are there other clinical factors that would override these preferences?

Strengths
• Should have the right clinical team to address the question • Plans to collaborate with nationally recognized PRO experts (second phase) • Multi-disciplinary team • This project is a joint collaboration between the Division of Plastic, Maxillofacial, and Oral Surgery and the Department of Orthopaedic Surgery, with significant involvement by Plastic and Orthopaedic surgical trainees • The strengths of individual team members are highly complementary and likely to lead to an effective collaboration. • The external consultants, Drs. Pusic and Klassen, are international leaders in successful development of PRO instruments including the BREAST-Q, FASCE-Q, and others, and will be collaborating with this research team to develop and validate the final instrument • The proposed team has expertise in PRO development, lower limb kinetics, and reconstructive and orthopedic surgery and involves both experienced faculty and trainees. Weaknesses • Unclear why a statistician is part of the team • Should partner with someone from SSRI who has done qualitative interviews. Having someone with psychometrics background would be useful • Should include someone with expertise in behavior health and survey research • Qualitative research analyst (DOCR) not yet identified. This person will be critical to the study 3. Innovation

Strengths
• seems that this is addressing an unmet need in the field

• No instrument currently in use
• (from the proposal) There is currently no measure capturing the patient's voice without influence from a healthcare provider in patients treated for limb-threatening lower extremity injuries. This study will generate the preliminary data for a larger project to complete the development and validation of a PRO instrument for use in lower extremity trauma patients. In the development and validation of the final instrument, our multidisciplinary team will utilize Rasch Measurement Theory (RMT) to develop this PRO instrument, comprised of a comprehensive set of clinically meaningful scales, measuring COI important to patients with limb-threatening lower extremity injuries. The scales will be designed for use in research and direct clinical care. Measuring the patient's voice, they will allow PROs to be a component of lower extremity trauma research.
• This proposal addresses an unmet need in the field, in which the ultimate outcomes for patients has a critical lack of a patient-reported metric to report satisfaction and quality of life for lower extremity trauma procedures. No such PRO currently exists. Weaknesses • By itself this study won't push the field further 4. Approach

Strengths
• Have thought through their strategy for this project and how it will lead to future work • Using previously-described methods, which have been successfully adopted • The study will be designed to capture all the distinct health domains relevant to this population of patients. A qualitative research method will be used, guided by state-of-theart PRO instrument development methods to maximize both the clinical meaning and scientific quality. Qualitative interviews of a heterogeneous sample of 40 patients will be audio-recorded, transcribed and then coded line-by-line to identify COI. These COI will guide the development of a conceptual framework and a set of scales. The scales will contain items that map out the clinical hierarchy for each COI. It is presumed that some COI will be relevant to all patients and some may be relevant to only a subgroup of patients, such as those with a successful limb-salvage. If needed, given the heterogeneity of this patient population, future studies will collect additional qualitative interview data and subsequently field test this PRO instrument, allowing for item reduction and a test of psychometric properties (e.g., reliability, validity and responsiveness to change). Our longterm goal is to make this instrument available for widespread use in research and clinical care.
• This is a qualitative study with the following steps: 1) semi-structured patient interviews to collect data, and 2) development of a hypothesized conceptual framework and set of scales with items reflective of COI identified from qualitative data gathered in step 1. Future research will include steps: 3) cognitive interviews refining the preliminary set of scales and items to ensure they contain maximum content validity and are easy to comprehend, 4) expert opinion refining the scales, and 5) field testing of the instrument to allow for item reduction and testing of psychometric properties.
• Patients will be recruited for qualitative interviews who are age 18 years and older and are receiving or have received care at Duke University for a limb-threatening lower extremity injury. The patient population has been clearly defined, including the level of amputation, associated injuries, and exclusion criteria (e.g. spinal cord deficit, non-ambulatory state or amputation prior to traumatic injury, psychiatric disorder, etc).
• Plan for conduct and analysis for the qualitative interviews is clearly described.
• The proposed approach will provide the critical preliminary data to validate the new PRO and the budget is adequate to support this endeavor, going to support the cost of these interviews. With the data generated by this project, the team will be in an excellent position to apply for future extramural research funds.

Weaknesses
• The applicants mention wanting to interview 40 people but no mention of how many such procedures are performed so hard to evaluate whether this is feasible or not.
• Unclear as to what timepoint in the clinical course that potential study subjects will be contacted to participate. Would variation in this potentially alter the study results?
• The sample size of 40 survey participants may be difficult to achieve; however the PI presents several alternative recruitment strategies to overcome this challenge. Given that recruitment may be an important and possible issue, the team should consider many methods of patient recruitment from study outset.

Strengths
• Good Clinical environment • The preliminary data the team will generate in this project has high likelihood of success in forming the foundation for a externally funded research grant to complete the patientreported outcome instrument development and validation. The PI plans to complete this second stage with Drs. Andrea Pusic, MD at Memorial Sloan Kettering Cancer Center and Anne Klassen, DPhil at McMaster University. Drs. Pusic and Klassen are international experts in the development of patient-reported outcome instruments and have collaborated with the applicants on multiple previous research endeavors.
• Ideal collaborative effort including both reconstructive and orthopedic surgeons.

Weaknesses
• Would have liked to seen more partnership with individuals doing qualitative research across campus. Not sure if DOCR is really the right target for this.
• Is there enough clinical volume to accrue 40 patients to the study within 8 months?