Advancing Intersectional Discrimination Measures for Health Disparities Research: Protocol for a Bilingual Mixed Methods Measurement Study

Background Guided by intersectionality frameworks, researchers have documented health disparities at the intersection of multiple axes of social status and position, particularly race and ethnicity, gender, and sexual orientation. To advance from identifying to intervening in such intersectional health disparities, studies that examine the underlying mechanisms are required. Although much research demonstrates the negative health impacts of perceived discrimination along single axes, quantitative approaches to assessing the role of discrimination in generating intersectional health disparities remain in their infancy. Members of our team recently introduced the Intersectional Discrimination Index (InDI) to address this gap. The InDI comprises three measures of enacted (day-to-day and major) and anticipated discrimination. These attribution-free measures ask about experiences of mistreatment because of who you are. These measures show promise for intersectional health disparities research but require further validation across intersectional groups and languages. In addition, the proposal to remove attributions is controversial, and no direct comparison has ever been conducted. Objective This study aims to cognitively and psychometrically evaluate the InDI in English and Spanish and determine whether attributions should be included. Methods The study will draw on a preliminary validation data set and three original sequentially collected sources of data: qualitative cognitive interviews in English and Spanish with a sample purposively recruited across intersecting social status and position (gender, sexual orientation, race and ethnicity, socioeconomic status, age, and nativity); a Spanish quantitative survey (n=500; 250/500, 50% sexual and gender minorities); and an English quantitative survey (n=3000), with quota sampling by race and ethnicity (Black, Latino/a/x, and White), sexual or gender minority status, and gender. Results The study was funded by the National Institute on Minority Health and Health Disparities in May 2021, and data collection began in July 2021. Conclusions The key deliverables of the study will be bilingual measures of anticipated, day-to-day, and major discrimination validated for multiple health disparity populations using qualitative, quantitative, and mixed methods. International Registered Report Identifier (IRRID) PRR1-10.2196/30987

HDEP SCHEIM, A 1R21MD016177-01 Scheim, Ayden RESUME AND SUMMARY OF DISCUSSION: This application proposes to evaluate and test improvements in English and Spanish for the novel Intersectional Discrimination Index (InDI). Given the evidence about the impact of multiple forms of discrimination on health among racial/ethnic and sexual and gender minority populations, this is a highly significant area of inquiry. The panel noted a number of additional strengths including the rigorous examination of prior research to support the study's significance, strong investigative team, well described recruitment plan, and rigorous approach appropriately guided by a framework of intersectionality. The ability to test attributions for experiences of discrimination and the development of a Spanish version of the instrument are major strengths of the study. Minor weaknesses identified by reviewers include limited justification for the number of cognitive interviews and limitations in the sampling plan given the reliance on Facebook and social media. Some reviewers questioned the incremental value added of the study given the novel InDI recently developed by the investigative team. Others found the ability to test for comparative validity of the instrument and the Spanish language version as appropriate given noted gaps in the literature. Overall, the panel was enthusiastic about the study's high potential to advance intersectional health disparities research with validated measures of discrimination.

DESCRIPTION (provided by applicant):
Guided by intersectionality frameworks, health disparities researchers have documented health disparities at the intersection of multiple axes of social status and position (SSP), particularly race/ethnicity, gender, and sexual orientation. To advance from identifying to intervening upon such intersectional health disparities, studies that examine underlying mechanisms are required. Much research demonstrates the negative health impacts of perceived discrimination within single health disparity populations. Quantitative approaches to assessing the role of discrimination in generating intersectional health disparities remain in their infancy, however. Members of our team recently introduced the Intersectional Discrimination Index (InDI) to address this gap. The InDI comprises three measures of enacted (day-to-day and major) and anticipated discrimination; these attribution-free measures ask about experiences of mistreatment "because of who you are." These measures show promise for intersectional health disparities research but require further validation across intersectional groups and languages. Additionally, the proposal to remove attributions is controversial and no direct comparison has been conducted. Therefore, this study aims to (1) cognitively and (2) psychometrically evaluate the Intersectional Discrimination Index (InDI) in English and Spanish and (3) determine whether attributions should be included. Study aims will draw on three original sequentially collected sources of data: (a) Qualitative cognitive interviews in English and Spanish (n=50) with a sample purposively recruited across intersecting SSP (gender, sexual orientation, race/ethnicity, socio-economic status, age, nativity); (b) a Spanish quantitative survey (n=500; 50% SGM); and (c) an English quantitative survey (n=3000), with quota sampling by race/ethnicity (Black, Latinx, White), SGM status, and gender. The study's key deliverable will be bilingual measures of anticipated, day-to-day, and major discrimination validated for multiple health disparity populations using rigorous qualitative, quantitative, and mixed methods. This expected outcome will support NIMHD priorities for Health Disparities Science by strengthening measurement of discrimination in population health research, thereby improving understanding of how it contributes to health disparities.

PUBLIC HEALTH RELEVANCE:
The proposed study seeks to further develop instruments used to measure the impacts of multiple and intersectional discrimination on health and health disparities, in alignment with NIMHD's prioritization of research to improve measurement of racism and other forms of discrimination. The study will deliver English and Spanish-language measures of anticipated, day-today, and major forms of discrimination validated for multiple health disparity populations. These measures, in turn, will be used to advance knowledge about how discrimination impacts health HDEP SCHEIM, A disparities across intersecting axes of social status and position including race/ethnicity, gender, and sexual orientation.

CRITIQUE 1
Significance: 2 Investigator(s): 2 Innovation: 2 Approach: 2 Environment: 1 Overall Impact: The proposed study is likely to have high impact. Many researchers recognize the relevance of intersectionality theory for health equity research, but methods for assessing intersectional forms of discrimination are not well developed. This well-designed study will test improvements to a newly published instrument that made a splash when it appeared in 2019, resulting in the first bilingual, cognitively tested intersectional discrimination measure.

Significance: Strengths
• Relevance of intersectionality theory to health equity is clear, but existing measures and analytical approaches are underdeveloped.
• Multidimensional approach to measurement of discrimination-including anticipatory, everyday, and major discrimination-builds on best available evidence.
• Relevance of attributions for discrimination remains a contested area of research in need of attention.
• Developing measure for Spanish speakers is important because of known differences in response patterns to other measures of discrimination among language groups. Weaknesses • It would be helpful to have a stronger conceptual framing of how cultural differences may relate to interpretation of and response to discrimination measures.

Investigator(s): Strengths
• This team developed the original InDI, so is eminently qualified for the work.
• The investigators invited the author of a commentary on their original publication of InDI to join the team, which is exemplary and likely to enhance impact.
• Strong track record of collaboration and publication on research tightly linked to the aims of the current study.
• Good expertise on transgender health, relevant to intersectional framework.

Innovation: Strengths
• This study will have a significant impact on innovating measurement of discrimination in health research.
• Surprisingly, few studies have combined cognitive interviewing and psychometric evaluation of discrimination measures. HDEP

SCHEIM, A
• The study will advance our understanding of whether and how to measure attributions of discrimination. Weaknesses • None noted.

Approach: Strengths
• Solid plan for cognitive interviews in terms of data collection and analysis. • Feasible recruiting plan, based on prior research. • Good use of item response theory techniques.

Weaknesses
• Inadequate justification provided for sample size in cognitive interviews. Each subgroup (English and Spanish speakers) will encompass significant heterogeneity for gender, sexual orientation, race/ethnicity, socio-economic status, age, and nativity. Even if each of these criteria were treated a binary (an oversimplification), there would be 64 possible combinations. The consequence is that most intersections of categorical social status and position indicators will not be represented. The researchers appeal to the criterion of saturation for determining sample size, but given that more than half of intersectional identities will be represented, it is unlikely they will be able to determine whether they have reached saturation of not (see here the emerging literature critiquing and trying to operationalize saturation). This is the major scoredriving weakness in my evaluation.

Environment: Strengths
• Good support for research. • Adequate facilities.

Weaknesses
• None noted. Overall Impact: This proposal seeks to generate the first bilingual, cognitively evaluated intersectional discrimination measure and generate psychometric properties across intersectional subgroups. The proposal will test the already developed Intersectional Discrimination Index (InDI). The study's key deliverable will be a bilingual measure of anticipated, day-to-day, and major discrimination validated for multiple health disparity populations using a systematic and cumulative approach using qualitative, quantitative, and mixed methods. The weaknesses of the application are in the area of justification for the qualitative sample size in phase 1 and the need for more information on how qualitative data will be analyzed and synthesized with the data collected in phase 3, and the sample to be collected. The strengths of the project outweigh the weaknesses and the impact is positive for the outcomes of the proposed psychometric evaluation of the IDI.

Significance: Strengths
• Intersectionality of race/ethnicity, gender, and sexual orientation is related to significant health disparities.
• Perceived discrimination is a preventable contributor to poor health outcomes.
• There is an inherent weakness in using within-in group analyses to demonstrate between group differences.

Weaknesses
• The measurement tool has already been developed and cognitively tested in previously funded efforts and is being used in other NIH funded projects.

Investigator(s): Strengths
• The investigators have experience in the methods and content area to be addressed in this proposal. Weaknesses • None noted.

Strengths
• A measurement tool that is a bilingual measure of discrimination (day to day and major) that has been validated using a mixed methods approach.
• Investigation of attribution of discrimination is a significant innovation to the measurement.

Approach: Strengths
• The incorporation of cognitive interviews as a means to evaluate the IDI constructs. • Relative lack of justification of 50 cognitive interviews in Phase 1.
• Need for more information on how qualitative data will be analyzed and synthesized with the data collected in phase 3.
• Sample recruitment carries a significant selection bias.

Environment: Strengths
• The environment is supportive of the project and will serve to ensure successful completion.

Study Timeline: Strengths
• The timeline is feasible and realistic for the completion of the study aims Weaknesses • None noted. Overall Impact: There is a need for a more robust, efficient, and valid instrument to evaluate intersectional discrimination and its impacts on health. The investigative team has developed and tested the InDI instrument for 'face' validity. The team is transdisciplinary, experienced, and have demonstrated their ability to complete this type of research. The proposed methods for refinement (cognitive interviews), adapting to Spanish, and conducting a intersectional comparative validation study to the approach currently and commonly used in this field (attributed discrimination) is welldeveloped and rigorous. The team has excellent experience in conducting such studies with recruitment from the Web and other areas. Sex as a biologic variable is addressed. This is a strong proposal which could move measurement and attribution in the field of intersectional discrimination forward.

Significance: Strengths
• Perceived, self-reported discrimination is associated with worse mental and physical health outcomes.
• Current instruments to assess discrimination are grounded in the type of discrimination being examined such as racial/ethnic, sexual gender minority, etc.. However, this is problematic when trying to assess intersectionality as individuals are not monolithic and may be experiencing perceived discrimination based on multiple individual characteristics. Currently, this requires multiple instruments, one for each type of discrimination and each with its own battery of questions.
• A new approach to address this was developed by the investigative team whereby overall perceived discrimination is examined in the Intersectional Discrimination Index. Types of discrimination are based on anticipated, day-to-day (i.e. acute) and major. These constructs are then crossed with demographic and characteristic information to examine social status and position discrimination. HDEP

SCHEIM, A
• Prior work suggest this scale has excellent utility but it needs greater validation and needs to be designed for Spanish-speaking communities. Weaknesses • None noted.

Investigator(s): Strengths
• Strong transdisciplinary investigative team with prior collaborative experience in developing, implementing, and analyzing the InDI. Team members have expertise in psychometrics, methods to evaluate intersectionality, Latin culture and Spanish intersectionality, cardiovascular health, mediation decomposition methods, and cognitive and psychometric evaluation. Weaknesses • None noted.

Innovation: Strengths
• Development of a 'universal' discrimination instruments, adapting it to Spanish, and validating it against existing methods to evaluate intersectional discrimination is important and novel. Weaknesses • None noted.

Approach: Strengths
• The investigators are building from strong prior research in developing the InDI. • Recruitment methodology is appropriate for this type of large survey studies and the investigators have a track record with this approach but … see below. Incentives are appropriate.
• The proposed sample with all the stratifications is ambitious for 2 years but appears doable based on prior experience.
• Cognitive interview approach as part of refining existing InDI and for adapting to Spanish is good.
• Survey randomization and implementation design is good. Stratifying type of survey (unattributed, item-level attributed, and overall attributed) within only English-speaking participants is reasonable at this time of the instrument development. Quota sampling is also good to ensure balance in racial and ethnic minority by sexual gender minority categories.
• Analysis plans for Aims 1, 2, and 3 are appropriate. The mixed methods analysis of Aim 3 is well described.
• Sample size estimates are good at approximately 200 participants per intersectional category.

Weaknesses
• (Minor/Moderate) It would be helpful to provide some description of potential biases within the participants based on the recruitment methods.
• (Moderate) How do you determine 'meaningful' comparison between attribution instruments and within groups.

Environment: Strengths
• While the team is from across multiple institutions they have good experience working together and support from their institutions. Weaknesses HDEP SCHEIM, A + Derived from the range of percentile values calculated for the study section that reviewed this application.
NIH has modified its policy regarding the receipt of resubmissions (amended applications).See Guide Notice NOT-OD-18-197 at https://grants.nih.gov/grants/guide/notice-files/NOT-OD-18-197.html. The impact/priority score is calculated after discussion of an application by averaging the overall scores (1-9) given by all voting reviewers on the committee and multiplying by 10. The criterion scores are submitted prior to the meeting by the individual reviewers assigned to an application, and are not discussed specifically at the review meeting or calculated into the overall impact score. Some applications also receive a percentile ranking. For details on the review process, see http://grants.nih.gov/grants/peer_review_process.htm#scoring.

Health Disparities and Equity Promotion Study Section Healthcare Delivery and Methodologies Integrated Review Group
CENTER FOR SCIENTIFIC REVIEW HDEP 10/28/2020 -10/30/2020 Notice of NIH Policy to All Applicants: Meeting rosters are provided for information purposes only. Applicant investigators and institutional officials must not communicate directly with study section members about an application before or after the review. Failure to observe this policy will create a serious breach of integrity in the peer review process, and may lead to actions outlined in NOT-OD-14-073 at https://grants.nih.gov/grants/guide/notice-files/NOT-OD-14-073.html and NOT-OD-15-106 at https://grants.nih.gov/grants/guide/notice-files/NOT-OD-15-106.html, including removal of the application from immediate review. Consultants are required to absent themselves from the room during the review of any application if their presence would constitute or appear to constitute a conflict of interest.