Feasibility of a Career Development Intervention for Veterans in Vocational Rehabilitation: Protocol for a Pilot Randomized Controlled Trial

Background Veterans with psychiatric disorders want additional career development services to support their recovery and pursuit of meaningful employment. However, no career counseling programs have been designed for this specific population. We developed the Purposeful Pathways intervention to fill this need. Objective This study protocol aims to (1) evaluate the feasibility and acceptability of the Purposeful Pathways intervention for veterans living with psychiatric disorders and (2) explore preliminary clinical outcome data. Methods A total of 50 veterans who are participating in transitional work vocational rehabilitation services at a Veterans Affairs hospital will be randomized to either treatment as usual or the augmented treatment condition (treatment as usual plus Purposeful Pathways). Feasibility will be assessed via recruitment rates, clinician fidelity to treatment, retention rates, and acceptability of randomization procedures. Acceptability will be assessed via client satisfaction at treatment termination using quantitative and qualitative data collection. Preliminary clinical and vocational outcomes will be assessed at baseline, 6 weeks, 12 weeks (treatment termination), and a 3-month follow-up via quantitative measures assessing vocational functioning, vocational process, and mental and physical functioning. Results This pilot randomized controlled trial is beginning recruitment in June 2023 and is expected to continue through November 2025. Data collection is expected to be completed by February 2026, with full data analysis completed by March 2026. Conclusions Findings from this study will provide information on the feasibility and acceptability of the Purposeful Pathways intervention, as well as secondary outcomes related to vocational functioning, vocational process, and mental and physical functioning. Trial Registration ClinicalTrials.gov NCT04698967; https://clinicaltrials.gov/ct2/show/NCT04698967 International Registered Report Identifier (IRRID) PRR1-10.2196/47986


SUMMARY OF DISCUSSION:
A Subcommittee of the Rehabilitation Research and Development Scientific Merit Review Board met in Plenary Session and reviewed the above proposal considering all internal and external reviews. This document summarizes the major points of the discussion concerning the proposed project. In any further development of this project, the investigator should consider carefully all the issues reflected in this Summary of Discussion as well as the more detailed comments in the individual critiques.

GENERAL COMMENTS:
The applicant has been highly responsive to prior reviews, and has produced an extremely compelling application. Discussion focused on strengths of the application, and did include a few recommendations.
For the research portion of the proposal, the investigator proposes to use video for fidelity checks that will be developed over the course of the CDA. However, unless visual coding is required, Dr Stevenson may want to revisit the choice of video due to confidentiality issues. Many Veterans may be leery of being videotaped, and may want the option to restrict recording to audio. In addition, video recording can result in substantial audio quality problems due to location of camera microphone which can hinder fidelity monitoring of in-session conversation. If video is not needed, then audio recording should be offered and will likely be acceptable to the Veteran give the prevalence of audiotaping in VA treatment provision and research.
Second, this intervention has potential to effect improvement in a variety of areas, and so inclusion of a measure of general improvement in mental health functioning such as the Sheehan Disability Scale or other frequently used general mental health indices would allow further comparison of this intervention's effects with other, more focused interventions.

SUGGESTIONS:
The reviewers felt that, as mentioned in prior reviews, it would serve the investigator well to clearly distinguish this intervention from existing interventions in terms of description and implementation.

COMMENTS ON THE BUDGET:
None.

DESCRIPTION (provided by applicant):
The Veterans Health Administration (VHA) offers robust vocational programming that have helped countless Veterans obtain competitive employment; however, these services are not uniformly effective as recent data suggests that only 35 to 43% of Veterans are competitively employed at time of discharge. For those who become competitively employed, job tenure may be brief, which is often attenuated by underemployment or poor person-job fit. Moreover, only 3.5% of Veterans experiencing vocational problems engage in vocational services offered by the VHA. On average, it takes Veterans more than four years to utilize vocational services. These Veterans are at high risk of acquiring multiple functional losses and developing chronic disabilities as their vocational needs go unmet for years. Research suggests that intrinsic factors like lacking clear vocational goals, perceiving barriers to employment, and negative beliefs about one's ability to work contribute to low engagement, outcomes, and tenure of some consumers of vocational rehabilitation services. Thus, the VA may be able to improve vocational engagement, outcomes, and tenure of Veterans with psychiatric disorders by enhancing vocational services with added interventions targeting unhelpful psychological factors. Career counseling and development services have been shown to be effective in helping civilian populations clarify vocational goals and identity, enhance vocational self-efficacy, and increase proactive vocational behaviors in the face of obstacles. Additionally, career counseling and RRD9 STEVENSON, B development services help facilitate greater "match" between a person and their job, and person-job match is a key determinant of long-term career tenure among individual with psychiatric disorders. The researchers of this project propose a three-aim study to develop a career counseling and development intervention for Veterans with psychiatric disorders (CCD-V). The first aim will focus on the design and development of the CCD-V intervention with Veteran and provider input (n=16). The second aim will pilot test the intervention in an open trial (n=10) to gather Veteran input on the initial intervention. The third and final aim will consist of a feasibility pilot randomized controlled trial (n=50) to examine acceptability and feasibility outcomes and to explore the impact of the CCD-V intervention in terms of functional improvement and other vocational outcomes. The proposed CCD-V intervention will consist of approximately eight individual sessions that will be offered concurrently with existing VHA vocational rehabilitation services, (e.g., transitional work experience [TWE]).The final product of this study is to produce a manualized CCD-V intervention, and corresponding fidelity monitoring checklist, to be tested later in a larger efficacy trial.

PUBLIC HEALTH RELEVANCE:
Meaningful employment is consistently associated with the health and well-being of individuals living with psychiatric disorders, yet few Veterans reap this benefit due to high levels of un-and underemployment. Thus, there is a critical need for interventions to help Veterans with psychiatric disorders find and maintain meaningful employment. The Veterans Health Administration (VHA) is committed to improving employment outcomes among Veterans and is particularly interested in meaningful employment as important for suicide prevention. This research proposal seeks to enhance vocational rehabilitation services by developing and testing an evidence-informed career counseling and development protocol. This work will fill a substantial gap in VHA vocational services as empirically informed career counseling and development interventions for Veterans living with psychiatric disorders does not currently exist.

CRITIQUE 1
Applicant: The applicant continues to be an excellent applicant for the CDA program. Dr. Stevenson will develop and conduct a small feasibility based RCT of a VA based vocational intervention that addresses psychological factors that may impede successful employment. The VA has transitional and supported work employment programs, but the applicant's program provides relatively more focus on addressing mental health aspects of vocational success. As was the case during the last two submissions, he is currently conducting both clinical and research work in this area, with vocational leaders of the VA. He remains well positioned to contribute to, and remains committed to a career in the VA in this area. As mentioned in the prior review, the applicant's background shows a history of work in the area of the proposed study, meshing well with the refined proposed training plan. This is a strength.
As mentioned in prior reviews, the applicant's connection to the VA begins in graduate school, with his first position after graduate school with the Bedford VA in 2015. Since then, he has worked in the area of his proposed study at the VA. Since the first submission, he has increased his publication track record with his mentors each round. Of note, he is already involved in program innovation and evaluation, having developed a MIRECC funded vocational program in VA space with his primary mentor. He remains an excellent applicant for the CDA mechanism.

Mentor(s):
The mentorship team remains unchanged from the prior application, with additional clarification given to how primary mentors and the applicant will interact. Once again, all prior comments regarding the team remain relevant and accurate: This team is longstanding and successful RRD9 STEVENSON, B in terms of publication, grant acquisition, and grant implementation. The primary mentor, Dr. Lisa Mueller is a leader in the area of vocational rehabilitation in the VA, and notably, has both developed best practices programs with the applicant (e.g., their current Vocational Evaluation Center at the Bedford VA, on which they both offering clinical services and collecting research data) and engaged in research programs with the applicant (e.g., assisted applicant in obtaining pilot award funding through the VISN 1 Mental illness research, education, and clinical center (MIRECC); collaborating with the applicant on Dr. Sandra Resnick's national CWT dataset analysis) and other research endeavors.
Dr. Kelly is a co-mentor and has also worked with the applicant, and was instrumental in helping the applicant with his MIRECC award. She is a funded researcher with expertise in mental health treatment. Dr Bakken is a national level vocational rehabilitation specialist and will assure that programs developed by the applicant are VA ready and relevant. Dr. Ellison is a research health scientist and associate director of reintegration services at the Bedford VA, and is thus both an excellent clinical and research mentor in this area. Dr. Blustein is a co-mentor and an expert in the theory of career development among marginalized populations, and will directly assist in intervention development. In addition, the applicant has also already worked with the consultant, Dr. Uma Millner to examine relationships between career development variables and employment behaviors. Finally, the applicant and the consultant have participated in Boston University's Opening Doors project, a vocational rehabilitation project upon which much of the applicant's intervention is based. The mentors' letters of support are very strong.
Overall, it is clear that the mentorship team is strong, committed to the applicant, and engaged in research and clinical work relevant to the applicant, the applicant's area of proposed research, and the VA. The mentoring team is large, but there is a clear distinction between primary and secondary mentors, and a nice integration of roles across the training plan.
Training Program: There were very few questions regarding the revised training plan submitted the last round as the applicant added needed detail regarding mentorship interaction, research goals, coursework, and logistics. The training program is well matched to the applicant's needs and the proposal's research goals and methods. As mentioned in the past, this synchrony is a strength. Needed detail has been added since the last submission. A summary list of gaps to be addressed by training is complemented by clear and pragmatic attention to the logistics associated with addressing these gaps. information on his statistical training and future training intentions are given and justified. Clear schedules of training and mentorship events have been given and the details of the proposed learning plan are clear. All prior concerns have been addressed.

Scientific Merit:
Significance: As mentioned, the applicant proposes to create and preliminarily evaluate feasibility of a vocational intervention to address factors related to employment engagement that have been found to predict vocational success, including negative beliefs about one's ability to work, lack of clear vocational goals, perceived barriers to employment, etc., among individuals with severe mental illness The applicant provides a clear conceptual linkage of that final characteristic to the intervention.
Approach: Three phases of the research plan correspond to the 3 aims and are clearly presented, and the content of the treatment and the origination of data to drive these decisions has been greatly specified. In the first aim the applicant will develop a career counseling treatment manual for Veterans with psychiatric disorders and a fidelity monitoring checklist (n=16). Specifically, through focus groups with Veterans (and responsive to prior reviews, a good mix is used:…half who obtain employment and half who were not successful) and providers (from outside his VA, also responsive to reviews), the RRD9 STEVENSON, B manual will be developed using existing programs as a guiding starting point for discussion. Using such a variety of Veterans and outside providers will give rich information, particularly from Veterans for whom programs were not successful. Extensive detail is given as to how these qualitative data will be collected and analyzed, and then used to refine the treatment itself. Specifically, similar interview guides will be used for clinicians and Veterans who do and do not succeed in obtaining employment, which will result in focused, albeit rich information directly relevant to intervention modification. Clarification of how the proposed treatment differs from the opening doors program has also been provided.
In aim 2, the manual will be pilot tested with 10 Veterans. With this revision, much more information is given as to how exit interviews and other data that could inform refinement will be used at this stage to refine the treatment, which is now encouraging.
In aim 3, a small RCT will be conducted with 50 Veterans comparing the manualized treatment to a control group. An iterative approach to manual and treatment development will be used, and, as mentioned previously that there are several 'onramps' for incorporating information from clinicians and Veterans for intervention refinement. The RCT sample size of 50 is appropriate.
Addressing concerns about comparator heterogeneity, the new intervention will be compared to the most popular VA vocational program, TWE transitional work experience. Excellent justification is given for this selection (rather than simply making the selection by fiat). Measures selection is well done and complete, and a more global measure of psychological functioning/quality of life are included. With this revision, much more detail is given regarding how qualitative data will be obtained and how they will be used to inform intervention refinement.
A post treatment and a three-month follow-up assessment are specified. Fidelity checks for treatment adherence are planned, and clinicians will be actual VA clinical staff. Data analyses are focused on very basic measures of feasibility, and seem appropriate.

Resources:
Bedford VA resources remain unchanged. The site has ongoing research infrastructure to support this CDA applicant and affiliations with nearby academic universities among the mentoring team are strong. Moreover, some educational resources of Universities with which the mentoring team is affiliated are available to the applicant. Resources therefore seem strong.

Protection of Human Subjects: No concerns.
Inclusion of Women, Minorities, and Children: Children are appropriately excluded. Women and minorities are appropriately included.

Yes
No Research with vertebrate animals?
x Biohazards and Radioisotopes: No comment.

STEVENSON, B
Data Management and Access Plan (for data sharing, unscored): Data management and access seem appropriate.
Overall Strengths: Strengths remain from the original and revised applications, including a very clear conceptualization of the reciprocal relevance of work to mental health. The training plan and the research plan jibe well with the approach for intervention development, refinement, and RCT feasibility testing. The applicant has been highly responsive to reviews. The proposed work fits with the applicant's pathway of research focus and development. The research plan is logical, built on an excellent conceptual base and pilot data collected, in part, by the applicant and his mentoring team, and fits perfectly with the training plan. The training plan is on target, and mixes coursework, observation, and experiential learning. • Expertise and leadership in the integration of career counseling and development and vocational rehabilitation. A variety of training experiences are proposed, including formal coursework, directed readings, discussions and meetings, site visits, and an NIH summer institute.
The proposed training activities seem largely appropriate. In addition to advanced statistics, rather than having aim 2 focus on qualitative analysis, this reviewer would reframe this aim to encompass qualitative methods more broadly (including the development and use of semi-structured interview guides and focus group prompts; sampling strategies for studies using qualitative and/or mixed methods; etc.). Though the application was revised to clarify, in response to reviewer feedback, that qualitative methods are not a statistical method, the text following this training aim still describes proposed coursework in qualitative analysis as a course in statistics.

Scientific Merit:
Significance: The proposed work has is highly significant within VA. Employment is a critical functional outcome for Veterans with psychiatric disorders; yet, this group -despite having access to robust vocational services -has significant challenges finding, securing, and maintaining employment. Adding career counseling and development services to TWE has strong potential to enhance vocational rehabilitation outcomes for vulnerable Veterans with mental illness.
Approach: Three aims are outlined: • To develop a career counseling and development treatment manual for Veterans with psychiatric disorders (CCD-V) along with a fidelity monitoring checklist based on Veteran and provider needs assessment. • Pilot test the manual by treating a small number of Veterans in an open trial.
• Conduct a feasibility pilot randomized controlled trial comparing CCD-V to TAU.
In general, the applicant was very responsive to prior reviews. In particular, using CCD-V as an adjunct to TWE (as opposed to TWE and SE) was a responsive and very positive change. A few suggestions: • There could have been a bit more clarity as to if Aim 1 intended to develop a novel CCD-V intervention or rather to adapt and tailor existing interventions of this nature which have evidence in prior literature. • The VR-12 may not be an optimal functional measure for Veterans with SMI, this is traditionally conceptualized as a HRQOL measure. • This reviewer appreciates the appendices that are semi-structured interview guides, but investigator may consider some edits to the interview guides before they are used. There is no "grand tour" question to start either qualitative interview. Some of the nomenclature in the Veteran interview might benefit from pilot testing to ensure understanding, tailoring the interview as needed in response to this pilot testing. In addition, the qualitative interview guide for providers included some questions about content and some about implementation. The research plan does not maximally capitalize on the valuable implementation data collection that would be collected in the interview guides, so the applicant may benefit from working through this with his qualitative expert mentor.

Resources:
No concerns about resources. Dr. Stevenson is embedded in a well-recognized academic environment and has the needed support for his proposed research and training plans. Overall Strengths:

Protection of Human
• Excellent applicant with commitment to recovery-oriented interventions in the area of vocational services. • Very strong mentorship team.
• Important topic with strong significance for VA.

Overall Weaknesses:
• Consider an alternate functional measure to the VR-12.
• Pilot testing of the qualitative interview guides will be useful, as well as thinking about how to capitalize on the implementation data gathered from the qualitative data collection.

CRITIQUE 3
Applicant: Dr. Stevenson is a strong applicant, very committed to a career as a VA researcher. He has fostered collaborative and productive relationships with his mentors as well as incorporated himself into the services provided in the VA as well as community clinical settings. He also has established a record of publications and presentations in the area of interest (although difficult to determine exactly how many publications he has from his biosketch). These relationships and experiences will inform the development of the CCD-V intervention.

Mentor(s):
This is a strong mentorship team, and the letters of support from all of them are detailed and highly complementary of the applicant. The mentors also have an extensive background in relevant clinical research that will enhance the training of the applicant.
The mentorship team also has a very strong record of both NIH and VA RR&D funding, which will provide the applicant many potential avenues for funding his independent line of research following the completion of the CDA2.
Coordination among the mentors is explicitly described, and each of the mentors have extensive experience in working with individuals early in their career, further enhancing confidence in the team.
Training Program: The training program is very detailed regarding the coursework and training that the applicant will receive in his proposed 6 training goals: • Training Goal 1: Develop expertise in clinical trials methodology. RRD9

STEVENSON, B
• Training Goal 2: Increase applicant's knowledge in advanced statistics and qualitative analysis relevant to the analyzing of treatment research. • Training Goal 3: Improve knowledge and skill in conducting community reintegration research focused on functional outcomes. • Training Goal 4: Strengthen scientific writing and grant writing.
• Training Goal 5: Increase expertise in research ethics.
• Training Goal 6: Strengthen applicant's expertise and leadership in the integration of career counseling and development and vocational rehabilitation.
All of the mentors have datasets and expertise, over overlapping, in these 6 areas and the ambitious training plan does appear to be feasible. Provision of tables regarding the specific aspects of the goals and timeline, as well as a proposed schedule, was very helpful to this reviewer. These skills will also inform the applicant's proposed research plan.

Scientific Merit:
Significance: The proposed intervention, CCD-V, addresses an important gap in current vocational services provided by the VA. The findings from this study will inform a Merit or R level proposal, to be written in the fourth year of the award period. This intervention could be widely disseminated to the 177 CWT programs across the nation to improve the placement and subsequent retention of employment by Veterans or all eras.

Approach:
The CDA2 award will develop and evaluate a career counseling and development protocol for Veterans (CCD-V) living with psychiatric disorders. The following aims will be completed: Aim 1. Develop a career counseling and development treatment manual for Veterans with psychiatric disorders (CCD-V) along with a fidelity monitoring checklist based on Veteran and provider needs assessment (n=16).
Aim 2. Pilot test the manual by treating a small number of Veterans (n=10) in an open trial.
Aim 3. Conduct a feasibility pilot randomized controlled trial comparing CCD-V (n=25) to treatment as usual (n=25).
3a. Evaluate feasibility and acceptability of the treatment.
3b. Evaluate the preliminary efficacy of the treatment.
Overall, the approach has been well thought out and has incorporated the suggestions of two rounds of reviewers. The shift to comparing the CCD-V to treatment as usual, rather than a time and attention control, was wise and will enhance the relevance of the findings. A time and attention control may be more appropriate for a Merit RTC.
One concern of this reviewer is that the fidelity monitoring seems under developed, and it is not clear exactly how the interventions will be delivered -using CBT? Motivational Interviewing? A didactic approach? The monitoring of the process of the sessions, as well as the content, may be important to monitor and assist in interpreting the outcomes, especially in a Veteran population. The decision to videotape sessions, rather than audiotape them, also seems unnecessarily invasive and may influence recruitment and the sessions, as well as increase the chances of a breach of confidentiality if the videos are lost or viewed by non-project individuals (as opposed to voice). RRD9

STEVENSON, B
Resources: The site of the proposed research is at an excellent VA with resources to permit the training and research plan of the applicant: • The Edith Nourse Rogers Memorial Veterans Hospital (Bedford VA).
• VISN 1 New England Mental Illness Research, Education, and Clinical Center (VISN 1 New England MIRECC). • Social and Community Reintegration Research Program (SoCRR).
• Center for Healthcare Organization and Implementation Research (CHOIR).
The applicant is well integrated into the VA and surrounding community, which further ensures that he will take advantage of these resources.