Patient-Reported Outcome and Experience Measures in Perinatal Care to Guide Clinical Practice: Prospective Observational Study

Introduction

Background

Currently, health care systems are moving toward high-value care, adapted to each individual patient [1,2]. These health care systems prioritize patients’ health goals in care decisions and quality improvement, above processes and clinical parameters. The transformation into a patient-centered, value-driven system is dependent on access to data that capture what matters most to patients [3-5]. Patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) provide standardized assessment of patients’ health status or experience with health care directly from the patient [6]. Integrated into routine care, these measures can facilitate patient-provider communication, improve patients’ experiences, and enhance detection and management of their health status [7-9]. When aggregated, PROMs and PREMs foster inclusion of patients’ perspective in continuous quality improvement, along with clinical measures that have already been captured for quality performance [10].

Just as in other disciplines, perinatal care may benefit from systematic PROM and PREM assessment to enhance quality of care. Moreover, patient-reported outcomes of perinatal care, such as depression or incontinence, may have serious long-term consequences for the health of the mother and child and might currently be undervalued. The interest in, and use of, PROMs and PREMs has grown in perinatal care, but most PROMs and PREMs in this field are assessed anonymously for quality improvement or research purposes only [11], whereas PROMs and PREMs, if integrated in clinical care on an individual level, could provide perinatal caregivers an opportunity to detect symptoms and adapt care appropriately, as well as encourage patients to think, and speak, about their current well-being and experiences [12]. Nevertheless, clinical integration of PROMs and PREMs has many challenges such as selecting relevant topics, valid assessment instruments, measurement moments, and threshold values that require action [3,13,14].

The International Consortium for Health Outcomes Measurement (ICHOM) has published a core set of patient-centered outcome measures for pregnancy and childbirth (PCB set), proposing standardized measures of clinical outcomes as well as patient outcomes and experiences over the full cycle of care [15]. For its patient-reported domains, the PCB set includes measurement instruments (ie, questionnaires) and a timeline for assessment: at 5 time points throughout pregnancy and post partum until 6 months after birth (Figure 1 [16]). Recently, the feasibility and acceptability of the PCB set were studied in clinic and its patient-reported domains collected for research purposes [17-19]. In addition, some of its measurement instruments were evaluated for validity and reliability in a maternity population [20-22]. However, little is known regarding compliance with the PROM and PREM questionnaires of the PCB set and the clinical performance of threshold values that require action throughout pregnancy and the postpartum period.

‎View this figure

Study Rationale

In an implementation project across the Netherlands, 7 regions incorporated the PCB set in clinic over the full cycle of perinatal care with all care professionals involved. In the journey toward value-based perinatal care, the primary goal was to discuss individual PROMs and PREMs as part of regular care and use them for shared decision-making to personalize care accordingly (level 1 of value-based health care). Furthermore, aggregated PROM and PREM results could be used for patient-centered quality improvement (level 2 of value-based health care). During the project, we closely monitored first experiences and practice insights of the regions’ incorporation of patient-reported measures into routine perinatal care at an individual level. This study aimed to report compliance with the PROM and PREM questionnaires, the outcomes per domain throughout pregnancy and post partum, and the clinical use of threshold values. Our findings can support clinical implementation of value-based health care with the PCB set, accelerate the transformation toward personalized care, and contribute to governance of the PCB set to retain its international comparability.

Methods

Study Design

An observational study was conducted to report and gain insight into PROMs and PREMs as part of clinical routine for personalized perinatal care. This paper is written following the Strengthening the Reporting of Observational Studies in Epidemiology checklist [23].

Setting

This study was carried out as part of a project involving the implementation of the PCB set in Dutch perinatal care called the Dutch abbreviation of Discuss Outcomes of Pregnancy with the Pregnant Woman (BUZZ) project. In total, 7 regions across the Netherlands joined forces to implement the PROM and PREM domains of the PCB set in routine clinical practice. The implementation was supported by Zorginstituut Nederland and coincided with a nationwide ministry program to enhance value-based health care and shared decision-making [24]. Each participating region consisted of 1 or 2 hospitals and 2 to 18 community midwifery practices (Table 1) collaborating in local obstetric care networks (OCNs; refer to Textbox 1 for an explanation of Dutch perinatal care organization). Data were collected from February 2020 to September 2021.

Participants

Women receiving perinatal care at a participating organization were invited to complete PROM and PREM questionnaires as part of usual care. Women who additionally gave informed consent to use their answers for research were included in this study. Informed consent was obtained in the PROM and PREM questionnaire itself. As this study aimed to report outcomes of the PCB set as is, we report the results of all PROM and PREM questionnaires collected within the project period; no target size was predetermined.

Implementation in Clinical Practice

The primary purpose of the BUZZ project was to use PROM and PREM questionnaires to guide individual perinatal care. Pregnant and postpartum women were invited to fill out questionnaires as part of routine care and their obstetric care professional discussed the answers in their next regular visit. The BUZZ project was explicitly organized within OCNs to ensure continuity of care over the full cycle of care for pregnancy and childbirth. The project team of each OCN made local decisions to enhance implementation in their practice on several key points (Table 1):

• Mode of administering questionnaires: some sites could capture questionnaires through their electronic health record (EHR), others used a stand-alone data capture tool, and 1 site used paper questionnaires (whatever at that moment was considered the most optimal to use the responses in their clinical setting).
• Population and time points: most sites chose to start small by either selecting a few time points for PROM and PREM assessment or a specific patient group.
• Site-specific adaptations: some sites made minor adaptations to the questionnaire content. For example, 1 site dismissed the screening questions for depression and used the full questionnaire in all women.

Outcome Measures

The PCB set’s PROM and PREM domains were captured as proposed by ICHOM with questionnaires at 5 time points during pregnancy and post partum (Figure 1) [16]. Each domain is assessed with its own measurement instrument, consisting of one or more questions (Multimedia Appendix 1). At every time point only relevant domains are assessed. In some domains, one or more screening questions can either rule in or rule out further questions for that domain. To fit Dutch perinatal care, a few domains have been added to the original PCB set (Figure 1) [17]. Before implementation, the translated Dutch questionnaires were tested among 4 women with low health literacy by the Dutch center of expertise on health disparities (Pharos). Minor adaptations were carried out where possible; questionnaires already validated in Dutch were not adapted. For each measurement instrument a clinical threshold value (alert) was defined according to existing literature or, if not available, determined by the multidisciplinary national BUZZ project team, informed by expert opinion (Multimedia Appendix 1). The alerts supported care professionals interpreting the answers, indicating worrisome outcomes through a color-coded dashboard (or calculated by hand in case of paper questionnaires). As clinical data could not yet be merged (digitally), a few casemix variables were collected through the questionnaires: age, gravidity, parity, postal code, and ethnicity.

Data Analysis

Only the data of women who gave informed consent were uploaded by project leaders to a central and highly secure digital research environment. Data merging and analysis was performed on this secured server using R software (version 4.0.2; The R Foundation for Statistical Computing) [26]. Duplicate and blank questionnaires resulting from technical problems were removed. In addition, questionnaires with only the first item filled out, requesting informed consent or social support, were excluded because we could not determine whether this resulted from a technical problem. A new option to answer a question was added by 1 site (ie, not applicable): these answers have been considered missing in analysis because they were not included in the national (validated) scoring systems. Secondary analysis of these data was considered, but the numbers were too small. Questions that were answered unintentionally, for example, a full depression questionnaire filled out despite having scored a negative screening, were removed. The casemix variables gravidity and parity are reported as state in current pregnancy: if parity and gravidity were equal, parity was corrected to gravidity–1. Completion rates were calculated per question and per measurement instrument. If applicable, sum scores were calculated according to a predefined scoring system. Missing items were excluded from this calculation; therefore, sum scores with one or more missing items are lower by definition. Alerts were calculated according to the thresholds provided in Multimedia Appendix 1. In an additional sensitivity analysis of domains with multiple questions, results with >25% missing items were removed, and their mean sum scores and alert rates were compared with the complete analysis.

Ethics Approval

The Medical Ethics Review Committee of the Erasmus Medical Center (MEC-2020-0129) declared that the Medical Research Involving Human Subjects Act (WMO) does not apply to this study. Therefore, it was exempt from formal medical ethics assessment. For each site, local approval was obtained from the regional ethics board.

Results

Overall

In total, 1923 unique questionnaires were collected, most of them during pregnancy (Table 2). The median moments of completion corresponded well with the proposed time points (Figure 1). Some T2 and T4 questionnaires were completed earlier than the proposed window, whereas a few T1 questionnaires were filled out too late. The questionnaires were filled out by 1318 individual women, of whom 838 (63.58%) completed 1 questionnaire, and the remaining 480 (36.41%) completed up to 4 questionnaires. Their baseline characteristics are presented in Table 3. Sum scores and alerts per domain and time point are presented in Tables 4 and 5. Multimedia Appendix 2 contains figures that show each domain’s scores and alerts.

PROM per Domain

Social Support

Of the 1092 women who were asked the social support question, administered at the first time point in pregnancy that each site had implemented, 44 (4.03%) scored an alert, meaning that they had 1 or no person near them to count on in time of difficulty. A comparison of T1 and T2 showed a slightly higher alert rate at T2 (17/25, 6.8%) than at T1 (26/815, 3.19%).

Quality of Life

The quality-of-health domain, assessed with the Patient-Reported Outcomes Measurement Information System–Global Health Short Form, had few alerts at all time points. The alerts were based on the sum score; no alerts came from a high pain score. In additional analysis, calculation of subscores for mental and physical health showed no variation across time points.

Mental Health

In 3.52% (61/1731) of the women completing the 2-item depression screening (Patient Health Questionnaire-2 [PHQ-2]) an alert was scored, without variations over time. Women with an alert on the PHQ-2 filled out the full depression questionnaire (ie, Edinburgh Postnatal Depression Scale-10 [EPDS-10]). As 1 region dismissed the PHQ-2 screening questions, 29 women filled out the EPDS-10 directly. The EPDS-10 exceeded the clinical threshold in 52% (47/90) of the women, meaning that 2.67% (47/1760) of the women in the whole population screened positive for depression. The numbers with regard to the EPDS-10 results were too small to allow for interpreting variations over time.

Incontinence and Dyspareunia

The screening question for urine and flatus incontinence was positive in 1 of 4 women. This proportion was lower at T1 than at the other time points. Screening for stool incontinence was positive in 0.86% (15/1741) of the cases, mostly at T4 (6/162, 3.7%). The full questionnaires in case of a positive incontinence screening resulted in alert rates of 39.4% (185/469) on urine incontinence (International Consultation on Incontinence Questionnaire, Short Form) and 97.96% (385/393) on flatus or stool incontinence or both (Wexner scale). Women who screened positive for flatus incontinence but not to stool incontinence scored lower on the Wexner scale (median 3; range 0-11) than women who screened positive for stool incontinence with or without flatus incontinence (median 6; range 1-17). In 24.7% (229/929) of the women, an alert was scored on dyspareunia, with a lower alert rate at T1 than at the other time points.

Breastfeeding

During pregnancy, 77.6% (596/768) of the women intended to breastfeed their baby. After giving birth, 64% (80/125) of the women indicated that they would breastfeed their baby (fully or combined with formula) in the first week post partum, which decreased over time: 59% (89/150) at 6 weeks and 47% (9/19) at 6 months post partum. Of the 761 women who were breastfeeding (T3 or T4) or intended to (T2), 175 (23%) scored an alert on the screening question for confidence in breastfeeding. This alert rate was higher during pregnancy than during the postpartum period. After a positive screening question, the full breastfeeding self-efficacy questionnaire (ie, Breastfeeding Self-Efficacy Scale-10) gave an alert in 72.9% (124/170) of the cases.

Role Transition

The mother-child bonding questionnaire (Mother-to-Infant Bonding Scale) had a median score of 2 (range 0-11) and 44.9% (122/272) alert values. No difference was seen over time. The single question about confidence in the role as mother scored almost no alerts, and the median score was equal to the maximum score.

PREM per Domain

Individual Insight Into PREMs

Before answering PREM questionnaires at T2 (early third trimester), the women could choose whether to give their care professional direct insight into their answers because the answers could affect the dependent relationship with their care professional. The answer to this question was not reported by all participating sites. We received data of 175 women, of whom 26 (14.9%) did not agree to share the answers of their PREM questionnaire directly with their caregiver.

Satisfaction With Care

This single-question domain, filled out by 924 women, scored almost no alerts, and the median score was 3 out of 4 (range 1-4).

Health Care Responsiveness and Shared Decision-making

Total scores were high, with a median of 16 (range 2-16) without variation over time. Still, the alert rate for this domain was 10.7% (101/947), based on a negative answer to one or more questions. Of the 101 women scoring an alert, 59 (58.4%) answered in the negative to just 1 of 8 questions. The alerts per question provided insight into direction for improvement, such as information provision about care decisions.

Birth Experience

Assessed with the 10-item Birth Satisfaction Scale, Revised, at T4, this domain gave an alert in 23.3% (37/159) of the women and had a median total score of 30 (range 8-40). The Birth Satisfaction Scale, Revised, subscales scored a median of 11 (range 2-16) for stress, 14 (range 4-16) for quality of care, and 5 (range 0-8) for women’s attributes. Comparing women with and without an alert on the sum score, the subscales stress and women’s attributes decreased by 50%, whereas the subscale quality of care decreased by 21%.

Pain Relief

During pregnancy, at T2, 41.3% (310/750) of the women indicated that the options for pain relief had not been discussed with their care professional yet. Post partum, most women were satisfied with the options for pain relief that were offered during childbirth.

Partner Role

Women were asked whether care professionals had engaged their partner enough in their care. This was insufficient for 7.4% (56/762) of the women during pregnancy and for 0.7% (1/152) during labor.

Continuity of Care

In total, 6.1% (55/905) of the women answered in the negative to one or more questions about continuity of care, with a median score of 11 (range 4-12). This domain had a slightly higher alert rate in pregnancy than during the postpartum period. In 96% (53/55) of the alerts, the women scored only 1 of the 3 questions negatively. Most alerts resulted from a negative answer to the question about knowing who their principal care provider was. In 23.5% (213/905) of the cases, the women had received perinatal care from just 1 care professional. Excluding these, the overall alert rate was 7.9% (55/692) and the median score 10 (range 4-12).

Adherence to the Questionnaires

Overall, 84% (1615/1923) of the questionnaires were filled out completely. Per domain, the percentage of completely missing answers ranged between 0% and 13%, as presented in Tables 4 and 5. Certain domains were skipped more often, such as the EPDS-10 (depression) and the Patient-Reported Outcomes Measurement Information System–Sexual Function and Satisfaction (PROMIS-SFFAC102; pain with intercourse). Missing rates per question are listed in Multimedia Appendix 3 and ranged from 0% to 16%. Evaluated per question, no remarkable missing patterns were found that could not be explained by site-specific adaptations to the questions. In Multimedia Appendix 4, missing patterns per domain are visualized. In additional sensitivity analysis of domains with multiple questions, sum scores and alert rates did not significantly change after ruling out the questionnaires with >25% missing items. Here, we chose to report the complete case analysis, best reflecting clinical use, because these results were not ruled out from individual reports to care professionals.

Discussion

Findings and Recommendations

This study reports the results of an innovation in perinatal care in the Netherlands: implementation of ICHOM’s PROM and PREM domains for pregnancy and childbirth to guide individual patient care in 7 OCNs. The large cohort resulting from this project showed good adherence to the questionnaires. In several domains, such as incontinence and breastfeeding, the high alert rates revealed opportunities to improve and personalize perinatal care for individual women on outcomes that matter to them. In addition, our results indicate that some measurement instruments and their timing as proposed by ICHOM are less suitable for clinical use. On the basis of these findings, we present several recommendations regarding the methods and timelines of PROM and PREM assessment in clinical practice.

Overall, adherence to the questionnaires was good, similar to PROM adherence when used for routine oncologic care [7]. High missing rates per instrument could be explained by technical issues, site-specific adaptation to the questionnaires, or questions addressing a relatively taboo subject, such as those included in the EPDS-10 and PROMIS-SFFAC102 (depression and pain with intercourse, respectively). In preimplementation tests, the PROMIS-SFFAC102 question also seemed difficult to understand despite language adjustments. Adapting the answer options might help, or an alternative instrument should be selected. Although they may be imperfect, the questions on these taboo subjects were answered by most women. Especially, these taboo subjects create more awareness at both patient and care professional levels, thereby increasing the likelihood of problems being recognized and addressed in clinic.

Median moments of completion corresponded well with the timeline of data collection as proposed by ICHOM. In contrast to the provider expectations described by Chen et al [27], the questionnaire administered shortly after childbirth (T3) resulted in a large group of respondents in this study who completed them mostly within 2 weeks post partum. At this point, there is an excellent opportunity to improve breastfeeding outcomes and mother-child bonding. As final maternal checkup with an obstetric care professional is at 6 weeks post partum in the Netherlands, the questionnaire at 6 months post partum (T5) is practically difficult to arrange for care providers. As a result, most OCNs chose to skip T5 to enhance feasibility; thus, few questionnaires were collected. Although practically challenging, patient views on this timing should be considered because this moment previously has been shown to be valuable to reflect on long-term recovery after pregnancy and childbirth [17,28].

Our findings in the mental health domain indicate that the first instrument of the 2-step screening (PHQ-2) is missing an unacceptable proportion of women at risk for depression, in line with the findings of Slavin et al [21]. The prevalence of perinatal depression has been reported at a rate of 7% to 20% during pregnancy and up to 22% in the first year post partum [29]. In our cohort, the prevalence of depressive symptoms was only 2.7% over the whole period of pregnancy and childbirth up until 6 months post partum. As the main purpose in clinical care is to identify women at high risk for depression, we strongly recommend removing the PHQ-2 and screening all women for depressive complaints with the EPDS-10, despite an increased response burden. The EPDS-10 has been thoroughly validated and has been shown to be acceptable to women in pregnancy and post partum [30,31]. Furthermore, 2 PREM domains showed striking results. Women answered almost always in the positive to the PREM satisfaction with results of care, despite multiple PROM alerts suggesting that their results were not as positive. This might be explained by women expecting incontinence to be a normal result of pregnancy and childbirth. Either way, this single question did not differentiate between women who were satisfied and those who were unsatisfied with their care and does not add value to shared decision-making or quality improvement. The PREM on information provision about pain relief options gave unexpected high alerts: 41.3% (310/750) of the care professionals had not discussed this yet with their patient. This might indicate that the timing of the assessment does not fit clinical practice because the T2 questionnaire was completed at 28 weeks of pregnancy on average and regular pathways plan to discuss pain relief later. Overall, each domain in need of adjustment based on our results is listed in Textbox 2, along with proposed adaptations to enhance their use in clinical practice.

In several domains, high alert rates revealed opportunities to adapt care accordingly and improve individual outcomes. For example, a high prevalence of incontinence and pain with intercourse was found over the course of pregnancy, as expected from previous research on these topics [32]. Breastfeeding success rates were low, which corresponds to provider-reported breastfeeding numbers in the Netherlands from 2018 [33]. Strikingly, many alerts were scored on breastfeeding confidence and self-efficacy during pregnancy. This provides important opportunities for all perinatal care professionals involved to improve breastfeeding outcomes. At the same time, threshold values for alerts on several instruments must be evaluated for clinical use to determine whether women scoring an alert want help and whether clinicians have the instruments to provide this help. For example, the threshold for the Mother-to-Infant Bonding Scale was set quite low based on the literature [34,35], resulting in many alerts on mother-child bonding. At this moment, it is unknown whether women want their care professional to address these alerts, and clinical guidelines on when and how to act are lacking [36]. However, in perinatal care too, structural PROM monitoring did create openings for dialogue between patients and care professionals to personalize and improve care on these themes [2].

Regarding experience domains, 85.1% (149/175) of the women in this study agreed to making their individual answers to PREMs visible to their care professionals, but the remaining 14.9% (26/175) disagreed. These numbers both affirm the acceptability of individual PREM use and underline the importance of providing women an opportunity to choose, considering their dependent relationship with care professionals. In general, evaluating results of all women, the sum scores of the PREM instruments often did not differentiate very much, but separate answers gave valuable information about directions for improvement. For example, most alerts in the domains continuity and health care responsiveness resulted from negative answers to specific items: about knowing their principal care professional and information provision, respectively. In birth experience, the PREM with the highest alert rate, the subscales most affected in women with an alert on the sum score were stress and women’s attributes. Until now, the literature on individual PREM use to guide clinical practice has been scarce because anonymous use is mostly advocated, for quality improvement only [17,37].

Strengths and Limitations

To our knowledge, this project was one of the first experiences with incorporating the complete PCB set into clinical practice to guide individual perinatal care. Although it was challenging, each participating site collaborated with a multidisciplinary transmural team of care professionals (part of an OCN) for implementation to ensure continuity of care over the whole cycle of care in a patient-centered approach. For this study, we have performed thorough additional analyses such as sensitivity analysis and appraisal of the use of screening questions, leading to practice implications for several domains. The sample size was large, and our results reflect the true clinical use of all patient-reported domains in the PCB set in various settings across the Netherlands. Nevertheless, because of this practical and local approach, nonresponders were not registered; therefore, we cannot report any response rates. In addition, variation over time in our results should be interpreted with caution because of different numbers of results per time point—especially, the numbers at 6 months post partum were too small to enable drawing any conclusions. Another limitation was the absence of questionnaire translations, restricting the participants to Dutch-speaking women only. Moreover, because no resources were available to support completion of the questionnaires, women with low (digital) health literacy are likely to be underrepresented, although women with language barriers or low health literacy probably have higher prevalence of pregnancy-related issues and thus greater opportunities to improve their outcomes [38]. This reveals an important concern regarding the transformation to value-based care: it could worsen existing health inequities even further. Therefore, efforts should be made to standardize the questionnaires to facilitate translation into multiple languages. Furthermore, when implementing PROMs and PREMs as part of value-based care, all stakeholders involved should be well informed about their purpose and supported with multiple solutions to embed the PCB set structurally in clinic; for example, through group consultations [39].

Implications for Practice

On the basis of the first efforts to incorporate the PCB set into clinical practice, we have proposed several adaptations to its content and structure to better fit routine perinatal care (Textbox 2). At the same time, international governance of the PCB set is essential to maintain comparability for care improvement purposes. In addition, although we tested their clinical usefulness, further validation is needed of all the measurement instruments and their clinical thresholds during pregnancy and post partum, which has been started successfully in another cohort [20-22]. Although the numbers per region could not be compared because of differences in pilot setup (eg, patient group selection), data capture was more feasible when PROMs could be embedded in their own EHR. When used in performance management, PROM and PREM results would preferably be merged with clinical outcomes, ideally through the EHR. Although beyond our main scope, merging patient-reported data with clinical outcomes from EHRs was explored in this project. In concordance with previous findings [40], this seemed very challenging, depending on the software systems available. This study focused on the content of the PCB set; future work should investigate other factors influencing implementation in the patient, care professional, and organization contexts [41].

Conclusions

This study shows that the PCB set is a useful tool to capture and discuss patient-reported outcomes and experiences that need attention during pregnancy, childbirth, and post partum. These are promising findings in the journey toward patient-centered, personalized, and value-based perinatal care. In the future, merging patient-reported data with clinical outcomes and casemix factors would be even more valuable to improve quality of health care both at an individual level and an aggregated level.