Novel Interactive Tool for Breast and Ovarian Cancer Risk Assessment (Bright Pink Assess Your Risk): Development and Usability Study

Introduction

In the United States, there are nearly 250,000 cases of breast cancer and 25,000 cases of ovarian cancer diagnosed annually [1,2]. Breast cancer is the leading cause of cancer-related deaths and the most commonly occurring cancer among women globally [3-5]. In contrast, ovarian cancer is only the 10th most common cancer but the fifth leading cause of cancer-related death among women [1,6] and the leading cause of death from gynecological cancer [7]. Studies have estimated that 5% to 10% of breast cancer and 10% to 18% of ovarian cancer are due to hereditary susceptibility, in particular from breast cancer gene (BRCA) mutations or a strong family history [8-10]. Among BRCA1 carriers, specifically, estimates suggest a 40% to 87% cumulative breast cancer risk by age 70 years, whereas for ovarian cancer risk, estimates range from 16% to 68% [11]. However, studies have also demonstrated that current risk assessment and clinical practices only identify a small proportion of the at-risk population.

A central component of breast and ovarian cancer prevention programs is education on personal risk, as determined by factors such as family history, genetic susceptibility, hormonal risk factors, and modifiable health behaviors [12-14]. However, studies have estimated that current practices identify at most 10% of asymptomatic BRCA 1/2 carriers [8,10,15-17]. Moreover, among those with a strong family history of breast or ovarian cancer, but who have not received genetic testing, an estimated 20% to 30% have a pathogenic mutation in a breast cancer susceptibility gene [10]. Bright Pink is a nonprofit organization devoted to improving breast and ovarian cancer prevention through the belief that empowering educated patients leads to (1) lower distress associated with cancer risk assessment and (2) improved cancer prevention outcomes [18,19].

To achieve these goals, Bright Pink developed the Assess Your Risk (AYR) tool that provides a user-friendly, web-based survey to determine lifetime breast and ovarian cancer risk at the population level. Bright Pink was developed in collaboration with genetic counselors, public health practitioners, and clinicians to ensure alignment with standards of breast and ovarian cancer risk assessment but with a modern, consumer-facing interface that would be accessible to the general population. Since 2014, the Bright Pink AYR has been used by more than 1.2 million women to assess breast and ovarian cancer risk. The AYR tool offers a streamlined, educational, and actionable user experience with the goal of substantially increasing the number of women nationwide, as well as internationally, learning about breast and ovarian cancer risk.

However, the AYR tool has not yet been compared with the gold standard clinical guidelines for identifying women at increased risk for breast and ovarian cancer who are eligible for consultation with a genetic counselor. The aims of this study are to (1) describe the AYR tool and its content, (2) describe the AYR tool user population, and (3) compare the AYR tool classification against the National Comprehensive Cancer Network (NCCN) guidelines for genetic and family high-risk assessment in breast and ovarian cancer [2,17]. Moreover, improving health equity and early identification of young adult women at increased risk are the primary goals of Bright Pink and public health. Thus, this analysis is also the first to examine AYR classification compared with the NCCN criteria by race, ethnicity, and age group. Demonstration that Bright Pink AYR is an accurate tool for risk assessment at the population level is a critical step in ensuring that the general population, as well as patients and providers, have a reliable tool for the identification and education of women at risk for breast and ovarian cancer.

Methods

Study Population

Bright Pink, a national nonprofit organization based in Chicago, Illinois, was started in 2007 with the mission of empowering women to know their breast and ovarian cancer risk to manage their health proactively. For this study, eligible participants included Bright Pink AYR 3.0 users from October 2018 to March 2019, who were women aged ≥18 years. We included women of White non-Hispanic or Latinx, African American, Asian, or Hispanic or Latina race and ethnicity and all age groups. We excluded participants with a personal history of breast or ovarian cancer (excluding nonmelanoma skin cancer) or no knowledge of family history from calculations of agreement between the NCCN and the AYR. The goal of the NCCN criteria is to identify individuals who may be eligible for genetic testing and counseling for breast or ovarian cancer; therefore, we included women at increased risk for either breast or ovarian cancer.

Ethics Approval

This study was approved by the Northwestern University Human Subjects Protection Program (review number STU00207333).

Bright Pink AYR: History

Bright Pink developed the original AYR tool in 2014 to support the goals of modernizing and increasing access to risk assessment for breast and ovarian cancer at the population level. The AYR tool estimates breast and ovarian cancer risk through a user-friendly, informative web-based quiz with superior-rated readability [20]. In late 2018, Bright Pink introduced a new version of AYR with an updated user experience and the goal of significantly increasing the number of women in the general population learning about breast and ovarian cancer risk. Specifically, the new version of the AYR tool includes a user-friendly quiz experience and updated mobile-first design features based on feedback gathered from surveys and focus groups of key target demographics, including African American women. Bright Pink also updated the educational content throughout the AYR tool to improve the user’s understanding of how their responses affect risk calculations and understand personalized risk management recommendations. The National Society of Genetic Counselors has reviewed and approved the AYR tool. The AYR tool not only provides accessible risk assessment to the general population but also links women to additional resources, including Bright Pink’s Explore Your Genetics website, breast health mobile messaging program, and an online peer support forum.

Bright Pink AYR: Design and Content

The AYR 3.0 tool includes 26 questions (Figure 1) based on the NCCN criteria for Genetic/Familial High-Risk Assessment: Breast and Ovarian [2,17]. However, the AYR tool is unique compared with other existing risk assessment tools designed primarily for use in clinical settings. The NCCN criteria that trigger consideration of genetic testing among asymptomatic individuals include (1) a family history of BRCA1/2 or other gene variants, (2) a family history of high-risk cancers such as triple-negative or male breast cancer, or (3) a family history of more than 3 cases of cancers on either side.

On the basis of these NCCN criteria, Bright Pink’s AYR tool categorizes women into the high category when they report already having a mutation in a breast cancer susceptibility gene identified through genetic testing. Specifically, AYR high risk is triggered by a personal history of positive genetic testing for genes associated with hereditary breast and ovarian cancer syndrome (BRCA1 and BRCA2), Lynch syndrome (MSH2, MLH1, or EPCAM), or Peutz-Jeghers Syndrome (STK11) [2]. In contrast, increased risk is triggered by a personal history of positive genetic testing for other moderate to lower penetrance mutations (eg, BRIP1, RAD51C, RAD51D, PMS2, and MSH6) or a relative with a positive gene mutation test combined with no personal history of genetic testing. Moreover, a strong family history of high-risk breast or ovarian cancer, as well as a history of three or more cancers within the family, will trigger an increased risk in the AYR tool. Finally, AYR also uses components from the Gail and Tyrer-Cusick models, including a personal history of childhood radiation to the chest, abnormal breast biopsy, or polycystic ovary syndrome (PCOS) [21-24]. Women with no family history of these genes, and none of the other criteria listed in Figure 1, are categorized as average risk and provided with educational messaging focused on appropriate clinical screening guidelines and current evidence related to the impact of health behaviors on cancer prevention.

‎View this figure

Statistical Methods

The primary goal of the AYR is to estimate the lifetime risk of breast and ovarian cancer among women with no history of breast or ovarian cancer. We thus excluded women reporting a previous history of breast or ovarian cancer from the analysis because the focus of AYR is on cancer prevention and early detection. In this subset, we calculated descriptive statistics for the AYR participants overall then by average, increased, and high AYR breast and ovarian categories. We then examined measures of agreement between the NCCN and AYR criteria.

We used the AYR quiz questions to create our own average and increased categories based on the NCCN criteria for breast and ovarian cancer genetic testing. We then compared these with the categories assigned by the AYR algorithm when the quiz was taken. We assessed measures including sensitivity and specificity to estimate the ability of AYR to distinguish between women who are eligible for genetic testing (AYR increased risk) versus those who are at average risk and thus not eligible for genetic testing, considering our NCCN criteria variables as the gold standard. We also examined the positive predictive value (PPV) and negative predictive value (NPV) to determine whether the AYR tool correctly identifies eligible or ineligible women based on NCCN criteria among those in the AYR increased or average categories, respectively. A true positive was when AYR risk category matched with the independently created NCCN categories. We calculated McNemar test of agreement, with an α of .05.

Results

AYR User Population

There were 143,657 AYR completions between October 2018 and March 2019, with complete data on health behaviors and family history. We examined the results for both breast (Table 1) and ovarian (Multimedia Appendix Table S1) cancer AYR risk categories. Overall, most AYR participants reported White race, and the mean age was 30.0 years (SD 10.7), with over half of participants in the 18-29 years age group. The mean BMI was 28.2 (SD 7.6) kg/m², and most women reported that they limit alcohol intake to less than 2 drinks per day and did not smoke. However, 60.54% (86,969/143,657) of women reported that they did not meet the physical activity guidelines for 150 minutes of moderate-intensity activity per week.

We also examined participant characteristics by the risk category assigned by the AYR algorithm, which is shown for breast (Table 1) and ovarian (Multimedia Appendix 1, Table S1) cancers. Most participants were classified as average or increased risk for breast cancer, with only 4.41% (6342/143,657) reporting high risk. White race was the most commonly reported (97,227/143,657, 67.67%) compared with 3.48% (5001/143,657) for African American, 3.11% (4469/143,657) for Asian, and 9.62% (13,833/143,657) for Hispanic or Latina ethnicity. After stratifying for the AYR-assigned risk category, a greater proportion of high-risk women tended to be aged >50 years and reported Ashkenazi Jewish ethnicity. For health behaviors, we observed a trend of higher alcohol intake and current smoking among women with increased and high breast cancer risk AYR categories. However, women in the high-risk category were least likely to report that they did not meet the physical activity guidelines (378/683, 55.3%) compared with 61.89% (49,566/80,091) and 58.53% (36,358/62,122) for increased and average risk, respectively) and reported the lowest mean BMI of 27.1 kg/m². High-risk women were also more likely to have a personal history of dense breast, PCOS, or abnormal breast biopsy as well as a family history of high-risk cancers. For ovarian cancer (Multimedia Appendix 1, Table S1), we observed similar trends overall and by risk category. This is supported by Table S2 in Multimedia Appendix 1, which demonstrates that most participants matched in the AYR risk category for breast and ovarian cancer.

Agreement Between AYR-Defined Categories Versus Independently Assessed NCCN Risk Categories

In examining the agreement between the AYR tool classifications and our classification based on NCCN criteria for breast and ovarian cancer risk, we calculated the sensitivity, specificity, PPV, and NPV. As shown in Table 2, the sensitivity and NPV were 100% for AYR compared with NCCN for all categories. However, overall, the specificity of the AYR tool was 89.9%, whereas the PPV was 94%. We also examined variation in measures of validity between the AYR and NCCN criteria by race, ethnicity, and age group (Table 3). We found the lowest estimates of specificity and PPV for Asian race, followed by Ashkenazi Jewish ancestry. Overall, specificity ranged from 85.9% for the Asian race to 91.3% for the African American race. For age group, there was also some variation in specificity, and the values ranged from 87.6% for the 30-39 years age group to 93.7% for those aged ≥65 years.

To explore the reasons for the observed variation in specificity and sensitivity, we also examined the frequency of the additional characteristics evaluated by the AYR tool, including personal history of chest radiation, PCOS, or abnormal breast biopsy by race, ethnicity, and age group (Multimedia Appendix 1, Table S3). Women with Ashkenazi Jewish ethnicity reported the highest frequency of previous chest radiation, PCOS, and abnormal breast biopsy. However, Asian women and women reporting for other races also reported 14.88% (638/4288) and 13.93% (2434/17,474) relatively high frequency, respectively, of a history of PCOS. Older women were more likely to report a history of chest radiation, whereas younger women were more likely to report PCOS. We also observed a trend of higher frequency of abnormal breast biopsy for the over 40-49, 50-64, and ≥65 years age groups (905/18,671, 4.85%; 644/7997, 8.05%; and 109/1151, 9.47%, respectively) compared with younger age groups.

Discussion

Principal Findings

We present the AYR tool as a valuable, new risk assessment tool with great potential to impact breast and ovarian cancer prevention in the general population. Overall, the AYR tool is highly accurate in identifying women at increased risk for breast and ovarian cancer in comparison with the gold standard NCCN criteria. We also identified differences by race, ethnicity, and age group that highlight opportunities for improving the reach of the AYR tool to additional populations as well as objective for future studies to validate the AYR tool among diverse populations. Overall, the Bright Pink AYR tool provides reliable, evidence-based risk assessment to the general as well as clinical populations for the identification of women at increased risk for breast and ovarian cancer.

Existing Risk Assessment Tools Compared With Bright Pink AYR

Bright Pink AYR is unique compared with other web-based risk assessment tools for breast and ovarian cancer, primarily because of its nature as a participant or patient-facing web-based tool designed for use by the general population. This includes, but is not limited to, clinical risk assessment tools such as the Breast Cancer Risk Assessment Tool (BCRAT; also known as the Gail model), the International Breast Cancer Intervention Study (IBIS), the Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm (BOADICEA) model, and BCRAPRO [25-29]. There is much overlap between these validated, clinically oriented tools with AYR but also important differences. For example, the BCRAT model uses 7 questions including age, race, ethnicity, health history (age at menarche and history of abnormal breast biopsy), and family history of breast cancer to estimate the 5-year risk and lifetime risk of invasive breast cancer based on the probability of cancer incidence during a defined age range [21,22]. The BCRAT model, however, does not include the risk of genetic mutations but instead refers participants to alternative tools for testing [21,22]. In contrast, although software such as IBIS using the Tyrer-Cuzick model also estimates 5-year and lifetime risk, the model incorporates genetic mutations and breast density into the most recent versions [23,24]. Moreover, both IBIS and BOADICEA were recently updated to include polygenetic risk scores, which is a limitation that other models such as the AYR should examine in future iterations. BRCAPRO is a similar model that relies primarily on basic demographics, family history, and genetic testing results, if available [30]. However, one major difference between AYR and these other tools is that, other than BOADICEA, none include lifestyle or health behavior risk factors in the models [27]. Overall, although there are many similarities between AYR and existing breast and ovarian cancer clinical risk assessment tools, the major difference is that AYR was designed as a resource for the general population.

Opportunities for Improving Risk Assessment Through Bright Pink AYR

Bright Pink built the AYR for the general population as a tool to help inform women of their risk and to provide education on cancer prevention activities appropriate to their risk level. We believe that the patient-oriented interface, developed with the help of marketing experts, makes AYR a particularly user-friendly risk assessment tool with the potential to reach populations that existing clinical tools may not reach because of issues such as health care access. The Bright Pink AYR tool represents an advance in the field of population-level cancer prevention as a screening tool with the potential to improve the reach and use of web-based breast and ovarian risk assessment among diverse populations.

The Bright Pink AYR tool has also demonstrated a strong reach to potentially underserved populations. Epidemiological data from the Surveillance, Epidemiology, and End Results Program demonstrate disparities in breast and ovarian cancer incidence and mortality by race [31-33]. However, studies show that much of the differences in breast and ovarian cancer incidence and mortality rates by race and ethnicity are driven by factors such as access to recommended screening and treatment [34,35]. To improve early cancer detection among high-risk groups and younger women not yet eligible for cancer screening, it is critical to increase access to and use of risk assessment tools in the general population. The Bright Pink AYR was designed to bridge these gaps. However, studies have demonstrated that other current methods for breast and ovarian cancer risk assessment may not currently be acceptable to diverse populations. Studies by Cragun [36] and Sheppard et al [37] demonstrated the disparities by race in the acceptability and use of breast and ovarian cancer risk assessment. The results demonstrate that breast and ovarian cancer risk assessment in a clinical setting is less likely to be acceptable or used by African American and Hispanic or Latina women compared with White women [36,37]. The observed disparities by race and ethnicity in breast and ovarian cancer risk as well as use of risk assessment tools justify the need to modernize risk assessment to make it more broadly accessible to the general population. We have demonstrated that the Bright Pink AYR reaches these diverse populations in the general population and provides accurate risk assessment; however, the educational follow-up emails also make AYR unique.

The AYR tool is also unique compared with other tools in that health behaviors are collected and used to tailor email messages to promote cancer prevention behaviors. Previous studies have demonstrated the impact of education on health behaviors among high-risk populations. For example, Quach et al [38] examined baseline and 6-month reporting of health behaviors among Ashkenazi Jewish individuals (n=120) who underwent genetic testing for BRCA1/2. The study found that women and those with higher education were more likely to report a healthy diet. The results showed no change in diet, vitamin use, or physical activity over time. However, there was no education component for health behaviors incorporated into the genetic counseling sessions. Another study examined health behaviors among participants of self-reported genetic testing (n=3016; with n=136 reporting genetic testing) using data from the Health Information National Trends Survey 4 [39]. The results demonstrated that lifestyle factors were not statistically significantly different between those who reported genetic testing and those who did not. However, the sample size was relatively small, and the study did not examine multivariate models. The AYR tool collects data on health behaviors, unlike many other clinician-oriented risk assessment tools. However, the AYR educational component has a strong potential to impact population-level participation in cancer prevention activities.

Studies have examined the impact of risk assessment programs for breast or ovarian cancer on adherence to cancer prevention guidelines or recommendations for lifestyle risk factors, and most have identified opportunities for improvement [38-43]. For women with a strong family history, Price et al [44] reported that among 748 women in a breast cancer family registry, between baseline and 3-year surveys, 16% were underscreened for mammography and 55% were underscreened for clinical breast examinations. Moreover, a study by Loescher et al [45] examined cancer surveillance behaviors among 107 women aged ≥18 years who presented for genetic risk assessment for breast and ovarian cancer. The results showed that 60% engaged in the minimal level of recommended breast cancer prevention activities, but 70% reported behaviors below optimal guidelines for cancer prevention. The study also found that a lack of physician recommendation was the most commonly reported reason for not engaging in breast or cancer prevention activities [45]. Similarly, Botkin et al [46] studied the impact of BRCA1 genetic testing on preventative cancer screening behavior over 2 years (n=408). Among women aged ≥40 years, 82% of mutation carriers followed guidelines for screening mammography in the first year and 67% in the second year, which was significantly increased from baseline and greater than the levels observed among noncarriers [46]. These studies provide evidence that knowledge of risk for breast and ovarian cancer may not only impact participation in cancer prevention activities but also that reported levels are not optimal and expanded adoption of combined risk assessment and educational modalities such as the AYR tool may improve cancer prevention in the general population.

There are strengths and limitations of the current analysis. The strengths include the very large sample size of the completions of the AYR tool in a short period and a diverse population. However, although the population was diverse, the frequency by race and ethnicity was lower than expected for some populations. This reflects the opportunity to improve access and use of the AYR tool in diverse populations. Moreover, the questions asked by the Bright Pink AYR tool are categorical and often binomial, such as health or family history and physical activity questions. This limits the level of detailed data collected on high-risk cancers and the ability to evaluate factors such as age at cancer diagnosis and pathology or subtype of breast cancer. Finally, these data were cross-sectional. Longitudinal studies will be necessary to validate the accuracy of AYR risk assessment in breast and ovarian cancer risk prevention. However, overall, the results demonstrate that AYR accurately classifies women according to breast and ovarian cancer risk using existing gold standard criteria.

These results demonstrate that Bright Pink’s AYR accurately classifies women at an increased risk of breast or ovarian cancer. The variation observed by race, ethnicity, and age group demonstrates the need to improve access and use of risk assessment tools in diverse populations. Overall, Bright Pink AYR is a valuable tool for use by the general population as well as patients and clinical providers for early detection and education, which will improve the prevention of breast and ovarian cancer.