An official website of the United States government
NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Kimberly D. Gregory, MD, MPH, Lisa M. Korst, MD, PhD, Moshe Fridman, PhD, Samia Saeb, MPH, CCRP, and Arlene Fink, PhD.
Author Information and AffiliationsPatient-reported experiences and outcomes (PROs) are an important component of health care quality assessment. Current PRO item banks do not include childbirth, the number 1 reason for hospital admission in the United States.
Develop a conceptual framework and preliminary item bank for childbirth-specific PRO domains, limited to the delivery and immediate postpartum period.
Using PROMIS® methodology, we conducted a comprehensive literature review to identify self-reported survey items eliciting childbirth patient-reported values and preferences (V&P) measured in pregnancy and associated experiences and outcomes (PROs) measured immediately postpartum. The V&P/PRO domains largely overlapped and were validated and complemented by focus groups. In collaboration with our community partners, we used a modified Delphi approach to select domains and items that were included in the survey. We conducted an observational study using national survey response panels organized through The Nielsen Company to identify women's V&P in childbirth. Eligible participants were US pregnant women (English or Spanish speaking) ≥18 years old, and ≥20 weeks pregnant. We used bivariate analyses to test whether key predisposing conditions (eg, demographics, prior experiences, beliefs) were associated with V&P items using data weighted to reflect the US pregnant population. We also fitted a multivariable logistic regression model to each V&P item to describe “who” wanted each item. Women participated in a postpartum follow-up survey to collect information about their childbirth experiences and outcomes (PROs). In bivariate analyses, we tested whether predisposing conditions, V&P, PROs, and the “gaps” between V&P and PROs were predictors of women's satisfaction with hospital childbirth services, which was measured using an ordinal scale of 1 to 10. Multivariable logistic regression models confirmed the results. We used PROMIS guidelines to finalize the conceptual framework and preliminary item bank for childbirth-specific V&P/PROs and key predisposing conditions.
We identified 5902 PRO items that mapped to 19 domains and 58 subdomains within an empirical conceptual framework. Of 2757 respondents to the antepartum survey, 81.6% (N = 2250) anticipated a vaginal delivery in a hospital and are reported on in detail here. Maternal characteristics that were associated with each V&P item varied (eg, hospital services desired by nulliparas versus multiparas differed, with nulliparas more likely to want to avoid medical interventions and to receive information regarding baby care and feeding). Predisposing conditions, such as maternal confidence and ability to cope well with pain, appeared frequently as predictors in the models. Of 500 laboring women who answered the postpartum survey, key findings included the following: (1) The strongest predictors of women's satisfaction with hospital childbirth services were items in the domains of staff communication, compassion, empathy, and respect; and (2) 23 PROs, including being told about progress in labor and adequate pain relief in labor, appeared especially relevant to women experiencing childbirth. A final model predicting women's satisfaction with hospital childbirth services included a total of 8 items that could be optimized by doctors, midwives, and hospitals. Variables that were eligible for the model were selected in a hierarchical fashion, in the order of predisposing conditions, V&P, PRO, and gap items.
We developed a conceptual framework and preliminary item bank for childbirth experiences and outcomes. The preliminary item bank consisted of 60 key predisposing conditions and 100 V&P/PRO items, forming the foundation for the Childbirth Experiences and Outcomes Survey and providing a tool for patient-reported data collection and benchmarking efforts.
Detailed results were limited to the subpopulation of women who planned for vaginal birth in a hospital. Additional analyses will need to be conducted for women who planned for cesarean delivery or delivery at home or in a birth center. Further, the use of national online panels included the potential for recruitment bias.
With nearly 4 million births annually in the United States,1 childbirth is the number 1 reason for hospital admission,2 and women rely on the medical system to provide them with safe and appropriate care. Childbirth clinical outcomes are a top public health challenge because rates of severe maternal morbidity (eg, renal failure, pulmonary embolism, blood transfusion)3 and mortality4 have been rising and racial/ethnic disparities have been widening in recent years.5-7 Safety concerns are real. One in 5 low-risk women experiences maternal or newborn morbidity during vaginal birth, and composite hospital morbidity rates exhibit wide variation (range, 3%-58%), in addition to cesarean morbidity.8,9
Numerous organizations are developing national strategies to make childbirth safer.10,11 However, because medical interventions in childbirth (eg, continuous fetal heart rate monitoring, increasing use of cesarean delivery) have been linked to decreasing childbirth satisfaction,12 these efforts may have contributed to a gap between what hospitals believe is needed for safety and what women believe is an optimal childbirth experience.13-20 The Institute of Medicine (now the National Academy of Medicine) defines patient-centered outcomes as distinct from clinical outcomes, and includes dimensions such as respect, communication, and physical comfort.21,22 Patient-centered outcomes have received less attention than safety issues but are a complementary component of health care quality measurement.23,24 Details regarding which patient-reported data are most meaningful require development.25,26
The National Institutes for Health funded PROMIS® in 2004 to develop standardized methods for measuring patient-reported outcomes (PROs), including the production of banks of standardized and validated survey items that correspond to various health domains.27-29 To date, PROs have largely been used for clinical research purposes and to guide clinical care,30 although PROs are now being integrated into the “performance measurement” of hospitals and physicians.31-33 PCORI,34,35 the PROMIS group,30 and the National Quality Forum (NQF)26 have published their perspectives regarding the uses of PROs in such endeavors.
PROs include not only measures of clinical outcomes from the patient perspective but also measures of the patient experience36,37 of the process of care. Our project was funded through an award from PCORI that required the use of PROMIS methodology (current award). Our project's principal goal was to develop a conceptual framework and preliminary item bank of PROs as a foundation for the development of childbirth hospital performance measures.
Given the resources available for this project, we anticipated that this approach would meet not only PCORI and PROMIS requirements but also the NQF guidelines for the development of performance measures38 and the Agency for Healthcare Research and Quality (AHRQ) guidelines for measures of the patient experience.39
The financial incentive of the federal Value-based Purchasing Program,40 which stipulates that Medicare reimbursement dollars be withheld from hospitals with poor satisfaction scores, creates a strong business case for childbirth hospitals to collect and utilize patient-reported data. As measured through the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey,41 the hospital satisfaction scores include the aggregate response from medical, surgical, and maternity care service lines. These scores provide feedback for hospitals to improve their services.
Because of the generic nature of the HCAHPS survey, hospitals do not know how to improve their scores in the maternity service line. This fact was emphasized at the expanded stakeholders meeting that was conducted for the current project. These circumstances argue for the development and implementation of a “maternity HCAHPS” so that hospitals can develop strategies to improve satisfaction with the childbirth experience and increase their revenue.42
Our specific objectives comprised the following:
The PROMIS Instrument Maturity Model describes the stages of instrument scientific development from conceptualization through evidence of psychometric properties in multiple diverse populations. The model assists developers in meeting the progressive scientific standard criteria from item pool or scale development to fully validated instruments ready for use in clinical research and practice.43 We were funded by PCORI to complete stage 1: developmental—conceptualization and preliminary item bank development. The subsequent stages, (2) developmental—calibration phase, (3) public release—calibrated and preliminary validation completed, (4) maturing—responsiveness and expansion, and (5) fully mature—user support, remain to be developed.
This report will primarily be useful to the research community seeking to advance the use of PROs in hospital performance monitoring. This report provides a firm foundation for continued development of the PROs into PRO-performance measures (PRO-PMs), patient-reported experiences, and patient-reported experience performance measures.36 Additionally, this report (both the conceptual framework and preliminary item bank) may also assist those in clinical settings (hospital administrators and maternity care providers) who aim to improve the childbirth experience.
The Childbirth PRO Partnership, a group of community partners, health services researchers, maternity care providers, and advocates for pregnant women, convened before the research activities. The Partnership conceptualized the initial project and participated in the formulation and submission of the initial project proposal.
The study team recruited partners based on clinical or health policy expertise, access to diverse groups of patients, experience with health advocacy, and whether they were currently or recently pregnant. Including Nielsen panel members, 15 community partners were engaged in the project. At the end of the project, this group expanded to include external stakeholders who provided guidance on disseminating and implementing the results. The study team invited hospital quality experts; senior administrators such as patient care managers, nurse managers, department chairmen, state regional directors, and representatives from health insurers; and other health service researchers to participate in the summary meetings. We emailed invitations that included letters introducing and explaining the project. See Table 1 for a list of the Childbirth PRO Partnership members.
List of Community Partners and Their Associated Organizations.
Each partner completed a memorandum of understanding that established clear and formalized goals, work processes, roles, responsibilities, and decision-making processes. Each agreed to participate at least once a month and to provide guidance on all research activities. We held weekly meetings to advance the work and monthly meetings to vote on final decisions and project direction. We used a modified Delphi method44 to ensure team representation in decision-making throughout the research process. Half of our community partners directly recruited women to (1) participate in focus groups, (2) pilot the survey, (3) assist with face and construct validity, and (4) resolve survey or focus group translation subtleties.
In addition to recruiting, the community partners hosted focus group sessions at their facilities or online via videoconference and served as cofacilitators for all sessions. Participant familiarity with the location and personnel established a comfortable environment for the participants to speak candidly about their experiences. Working collaboratively with the community partners and their constituents afforded us the opportunity to hear directly from pregnant and recently pregnant women regarding their values and preferences in childbirth.
All investigators participated in a standing weekly meeting for the project's duration. Community partners attended these meetings (in person or by phone or videoconference) to contribute to study planning and implementation, and to monitor study conduct and progress. In addition, a standing monthly Partnership meeting convened on the third Thursday of the month to update all community partners of activities to date, ensure feedback, and make plans for focus group recruitment. We posted meeting minutes in a Box account. We compensated community partners for their expertise and participation in direct proportion to their involvement, if they desired it.
The community partners helped develop the study proposal and formulate the relevant study questions. They also provided input into the study design and initial pilot data submitted with the application. Although they did not directly affect the study's rigor or quality, they did ensure the transparency of the research process. In the spirit of a true 2-way learning environment, we held several mini-lectures on statistical techniques to help ensure that the partners understood in layman's terms what factor analysis is, what logistic regression is, and how these techniques help reduce data and provide prediction estimates.
Importantly, when evaluating items that measure the same domain or subdomains, the Partners participated in “binning” and “winnowing” (ie, providing input regarding how to map the items to the conceptual framework domains and determining which items to keep or discard—especially when trying to nuance subtleties between different communities). For example, focus groups identified newborn feeding as an important PRO domain. Our literature review confirmed this and specified 2 subdomains: (1) receiving breastfeeding information, and receiving practical support about what and how to feed the newborn. Input from the community partners helped the research team appreciate that some women did not want breastfeeding information and were offended or made to feel guilty if they decided not to breastfeed. Importantly, many of these women could benefit from receiving practical support about feeding the newborn. The researchers learned about the public perceptions and preferences for terms related to newborn feeding that distinguish “breastfeeding,” “bottle-feeding breast milk,” and “bottle-feeding formula.” The team used to finalize the final survey items for this topic.
Our community partners have continued to participate in quarterly conference calls. Hospital partners have agreed to participate as clinical sites to develop hospital performance measures for childbirth PROs if we obtain subsequent funding for dissemination and implementation. We invited additional multidisciplinary stakeholders to participate in separate “expanded” partnership meetings to discuss dissemination and implementation opportunities (see Table 1).
This study complied with Cedars-Sinai Medical Center IRB stipulations under protocol #Pro00037750. The team used PROMIS methodology for the development of PRO item banks as the basis for the research approach.43,45 The first steps of PROMIS methodology are foundational to PRO development and include (1) a comprehensive literature search for potential PRO items, (2) use of this literature to empirically develop PRO domains and a conceptual framework that details the hypothesized relationships between women's values and preferences and satisfaction with hospital childbirth services, (3) the binning and winnowing of the items retrieved, and (4) iteratively eliciting feedback from the target population throughout the process. This basic process is intended to develop the conceptual framework and domains of the PROs, which can then serve as a foundation for further development through the PROMIS PRO pathway, the NQF PRO-PM pathway, or the AHRQ methodology for developing measures of the patient experience.
The Childbirth PRO Partnership (described in the preceding section) is a group of community partners that include health services providers, health and policy advocates for pregnant women, and currently or recently pregnant women; the Partnership participated in all research activities. Throughout this report, we have addressed the relevant methodology standards as required by PCORI (Appendix A).
As noted above, this study has 4 objectives: (1) Develop a conceptual framework for PROs and map relevant PRO items to the framework domains; (2) conduct a national antepartum survey to test the prevalence, distribution, and statistical significance of PRO items in the framework domains; (3) conduct a follow-up postpartum survey to (a) describe women's experiences and outcomes of childbirth (PROs), and (b) determine the statistical significance of these various predictors in women's satisfaction with their hospital childbirth services; and (4) using the study data, finalize the conceptual model and preliminary item bank.
Because a childbirth-specific PRO item bank did not exist, we advanced a conceptual framework that we built on empirically using the PROMIS® guidelines.43,45 Our conceptual framework for this study appears in Figure 1. The framework follows Andersen's Behavioral Model of Health Services Use46 with the addition of multiple theoretical guidelines regarding health expectations and service preferences, health information seeking, satisfaction, and patient-centered and childbirth outcomes.13,47-54
Conceptual Framework for Determining PROs in Childbirth.
As depicted in Figure 1, we posited that predisposing conditions (ie, women's personal characteristics, prior childbirth experience, clinical risk) generate V&P for the services desired. Upon giving birth, women assess whether these V&P were fulfilled. Last, women provide summary measures of their satisfaction with their birth and hospital services.
We hypothesized satisfaction to be dependent on (1) predisposing conditions, (2) V&P, and (3) PROs. V&P capture the concept of “value expectations” (ie, patients' desires, hopes, or wishes concerning clinical events).55,56 For brevity, we refer to all value expectations as V&P.
This framework implies that, although quality improvement efforts focus on PROs, V&P may be equally or more important in predicting overall patient experiences and outcomes for childbirth. Therefore, for childbirth, the QI program analysis plan must consider V&P. The simplest example is mode of delivery. If a pregnant woman desired a vaginal birth (V&P item), postpartum follow-up would indicate whether she had a vaginal or cesarean birth (PRO item). Satisfaction may depend on the V&P item or the PRO item, or a combination of both. For the example, satisfaction may depend most strongly on wanting a vaginal birth, getting a vaginal birth, wanting and getting a vaginal birth, or wanting and not getting a vaginal birth. All these possibilities must be tested in the analysis plan.
Working with a medical librarian, we performed a comprehensive literature search for English-language V&P and PROs associated with childbirth and the immediate postpartum period. Relying on Figure 1, we set up standardized search strategies of the English-language publications in PubMed from January 1975 through December 2014 (Appendix A).35,57-60 Because our goal was to capture items that reflect the breadth of childbirth experiences and outcomes important to US women, and not to evaluate the efficacy of any intervention, we did not assess individual studies for quality or synthesize study results.
The title and abstract (TIAB) of the first 1700 articles were read by 2 investigators who finalized the inclusion and exclusion criteria. Criteria explicitly required for the inclusion of studies were (1) questionnaires that included patient-reported items, (2) publication in English, a focus on women's assessment of the childbirth experience or on the consequences of childbirth occurring during the hospital experience, and (4) relevance to US health care. Criteria for the specific exclusion of studies were (1) editorials, letters, news, or opinion pieces; (2) a primary focus not related to patient assessment of her experience (ie, no trials regarding drugs or specific clinical interventions); (3) a discussion of questionnaires in languages other than English or Spanish; (4) case studies of individuals, natural disasters, or epidemics; (5) investigations of factors that affect conception or a desire for pregnancy; and (6) a lack of results or questionnaire items (eg, no qualitative studies). In addition, relying on our conceptual model, we abstracted items related to patient-specific conditions, such as personal characteristics, pregnancy/delivery history, clinical risk factors, and prior experiences with childbirth services, for potential inclusion in the conceptual framework. The librarian reran the search using the expanded criteria.
Two investigators reviewed all TIAB from all retrieved studies, retaining articles that met relevance criteria. The investigators retrieved and reviewed the full text of all potentially relevant studies. We retained all articles found to be relevant by at least 1 reviewer for inclusion in a study database.
Starting with the articles found in the literature search, we developed a list of PRO domains or “bins” relevant to childbirth.46 From these articles, we abstracted potentially relevant survey items, mapping each item to its appropriate bin. At the framework level, these domains generally housed both V&P and PRO items. For example, if an item asked a pregnant woman her preference for route of delivery, we mapped this item to the delivery route domain. If an item asked a postpartum woman the route of her delivery, we also mapped this item to the delivery route domain. Some domains, such as pain assessment or satisfaction, housed only postpartum items because we could ask these items only after the delivery.
We modified bins and added new bins for items that did not easily fit into an existing bin. We also created sub-bins within each domain. This resulted in a series of bins and sub-bins for categorizing the retrieved items and a list of individual items within each bin. These bins became synonymous with “domains” of the conceptual framework.
At the end of this binning process, we had created domains of the conceptual framework. Most domains included both V&P and PRO items. The PRO items also included patient-reported experiences and outcomes.
“Winnowing” is the elimination of items that do not have face validity or are redundant.46 Our goal in winnowing was to identify a limited set of items representative of the domains identified in the literature and ranked as important using a modified Delphi method by the Childbirth PRO Partnership. We divided the bins among 4 teams, each consisting of at least 1 investigator and up to 3 community partners. All the community partners and investigators had an opportunity to weigh in on the domains and items.
The process generated a final set of potential survey item bank members. We also identified survey items that reflected predisposing conditions so that the data collected could describe “who wants what,” with “what” representing the V&P/PRO items and “who” representing women's predisposing conditions (eg, personal characteristics, beliefs, clinical risks) that might vary in association with these PROs.
We organized focus groups to understand women's experiences in depth and to identify additional important outcome domains (Table 2). Focus group participants were at least 18 years old, pregnant or recently pregnant (less than 1 year postpartum) and living in the United States. Eligible participants recruited by our community partners served as diverse sociodemographic and socioeconomic populations. We deliberately selected participants representative of specific childbirth communities (ie, Hispanic, Spanish speaking, African American, Asian, low income, or college educated).
PCORI Focus Groups: Sites and Number of Participants.
We organized and facilitated our focus groups in collaboration with The Childbirth PRO Partnership, conducting sessions in English and Spanish. We prospectively determined our focus group sample size using qualitative saturation methods.61 A community partner (or designee) cofacilitated all focus groups in a community partner facility, utilizing a standardized script and guide. The script ensured that all participants received the same disclosure information and rules of conduct. The guide specified the objectives and research questions, provided a general timeline, and outlined probes, to maximize group participation.
The focus groups were conducted in person between June and November 2015 and lasted approximately 60 minutes. Each participant received a $50 Target gift card for attending the session. With the participants' permission, we recorded, transcribed, and entered the sessions into Atlas.Ti, a computer-assisted qualitative data analysis and research software (Version 7.1.1).
We used a grounded theory approach, whereby several investigators and members of The Childbirth PRO Partnership debriefed after each focus group session and collaboratively identified emerging themes.62 Two independent reviewers mapped participant responses to the domains identified in the literature search (code-by-list) and used the Atlas.Ti code manager to identify the most referenced domains. We categorized major and minor themes under the bins previously described, created additional bins as needed, and modified the conceptual framework and domain definitions with respect to the themes that surfaced in the qualitative data analysis.
We developed a survey using a subset of the predisposing conditions and V&P identified in objective 1. Before national administration, we piloted the instrument among 30 English-speaking women,63 assessing content and construct validity, interpretability, and respondent and administrative burden for use in online administration (Table 2). We administered the pilot survey either in person using individual laptops or online via videoconferencing with participants responding on their own computers. Community partners cofacilitated all sessions. We edited or removed survey questions per participant feedback.
Using similar methods, we created a Spanish version using a professional translation service and piloted it among women who identified Spanish as their primary language.
Survey completion time was <30 minutes. Responses for items in the predisposing condition domains were formatted as categorical or dichotomous variables. We used a Likert scale for the items in the V&P domains (eg, “not at all important” to “extremely important”; “strongly disagree” to “strongly agree”).
We conducted a national survey of pregnant women ≥18 years old who had completed at least 20 weeks of gestation. Nielsen recruited women through its online panels (Critical Mix, Survey Sampling, Market Cube, Peanut Labs, and Prodege), and it developed quotas based on anticipated demographic characteristics.64
Nielsen sent potential participants from these panels an email invitation that contained a unique URL, and then screened respondents to determine their eligibility. Eligible respondents proceeded with the survey and received weekly reminders if they did not respond. Nielsen administered the survey from secure servers using digital fingerprint technology to prevent duplicate entries. Nielsen designated all eligible participants who completed a subset of mandatory items as having “completed” the survey. Nielsen applied specific protocols to ensure survey completeness and the distribution of incentive payments (approximately $15 cash equivalent in Nielsen points). Nielsen monitored survey completeness on a weekly basis and left the survey open until the goal of at least 2700 completed surveys was reached. Incomplete surveys were not analyzed (Figure 2).
Nielsen Survey Administration Flow Diagram.
We weighted the national survey data to replicate the distribution of demographic variables from the 2011-2013 National Survey of Family Growth65 and the 2014 Current Population Survey,66 to improve generalizability to the US population (Appendix B). Data were also weighted by Nielsen's proprietary propensity score to mitigate potential selection bias owing to online recruitment methods.
Women who planned to have a cesarean delivery or planned to deliver at home or in a birth center are not described here in detail because of small sample sizes that did not allow for factor analysis or modeling. We used subpopulation analysis methods for weighted data to compute statistics for the women anticipating or considering vaginal delivery in a hospital—the most prevalent delivery expectation for American women. Subpopulation analysis methods were needed because we derived the data weights for the full sample, not for sample subsets. We performed statistical analysis of the survey data using SAS, Version 9.3. All analytical tests were 2 sided. Means are reported with SDs.
We performed an exploratory factor analysis to achieve data reduction, to confirm the domains of predisposing conditions and V&P, and to establish construct validity. Exploratory factor analysis is a statistical technique that is used to reduce data to a smaller set of summary variables and to explore the underlying relationships between measured variables. We used the national antepartum sample from Round 1 (see Figure 2) to conduct this analysis. However, the factors extracted were applied in all subsequent analyses of antepartum and postpartum data. We performed a factor analysis for the V&P items using both a segmented analysis (to validate the anticipated domains) and an overall analysis (to allow for potential shifting of closely related items from one domain to another).
We used standard criteria to determine both the number of factors and which items loaded to a factor.67 We applied distinct oblique rotations and selected the rotation that provided better factorization in terms of separation of loadings for continued evaluation. We also tested Cronbach α correlation as a measure of internal validity for each factor.36 The team selected the final set of factors based on empirical fit and confirmed face validity with the Childbirth PRO Partners, retaining factor-based scores (total score for items included divided by the number of items in the factor) for subsequent analyses and items that did not load on any factors.
We examined the continuous distribution of each V&P item and factor. To simplify the analysis and interpretation of results, the investigators determined whether the ordinal or interval scale responses could be categorized as either 2-level or 3-level variables. We used 2-level variables when the V&P exhibited a monotonic preference or trend, and 3-level variables for V&P that had a U-shape or mound-shape distribution that prevented binary collapsing. In general, and if possible for the 2 principal ordinal response scales, scores 1 to 3 were collapsed versus scores 4 to 5.
For example, the PRO item “It is important that providers respect my spiritual/religious/cultural beliefs” had the following distribution: 1 = “not at all important” (9.3%); 2 = “slightly important” (9.1%); 3 = “moderately important” (18.3%); 4 = “very important” (26.6%); 5 = “extremely important” (36.2%). Consequently, scores 1 to 3 were collapsed versus scores 4 to 5 (very to extremely important), creating a 2-level variable. Rating of the “encouragement for breastfeeding from providers” had the following distribution: “far too little” (6.8%), “too little” (9.1%), “about right” (67.0%), “too much” (10.9%), and “far too much” (6.3%). Consequently, the categories “far too little” and “too little” were collapsed. “Too much” and “far too much” were also collapsed, and “about right” remained its own category, hence creating a 3-level variable. For each predisposing condition, we measured the frequency of each V&P item.
After completing the bivariate analyses, we developed a multivariable logistic regression model for each V&P item to identify the independent predisposing conditions that were associated with that item. For 3-level items, we used generalized logistic models. The dependent variable was the V&P item and the independent variables used in each model were maternal age, race/ethnicity, education level, US region, parity (nulliparity/multiparity with no prior cesarean/prior cesarean), any medical/pregnancy-related complications, gestational age at the time of the survey, and multiple gestation. Other potential predictors of the V&P item were also assessed. To limit the number of additional predisposing conditions assessed in each model, we entered only those conditions associated with the V&P item resulting in a P < .05 in bivariate analysis.
For each model, we report the proportion of respondents who indicated a preference for the V&P item, the C statistic and the max-rescaled generalized R2. These are both measures of the model's predictive power. The C statistic is a measure between 0.5 and 1 of the classification accuracy of the model predictions of the outcome based on the model's covariates (also calculated as the area under the receiver operating characteristic curve).68 The generalized R2 for nonlinear models is similar to the coefficient of determination (known as R2) for linear models but based on the ratio of the likelihood function value under the null hypothesis that all covariate coefficients are equal to 0 (null model) relative to the unrestricted maximum value using the model covariates (full model). Because the upper bound of this statistic is not 1, it is rescaled by dividing the original value by its upper bound.69,70
We received supplemental funding to conduct a follow-up postpartum survey. In collaboration with the Childbirth PRO Partnership, we revised the items in the antepartum national survey to make it appropriate for postpartum administration. For most items, this primarily involved changing the tense of the verb associated with the item. All PRO items retained the same response scales used for the V&P items so that, when possible, they could be compared. The postpartum survey updated women's pregnancy complications after the completion of the antepartum survey and before delivery and added items for the following domains that could not be assessed antepartum: maternal/newborn clinical outcomes, pain assessment, communication with physicians/midwives and nurses, following of the birth plan, and measures of satisfaction.
Satisfaction measures included several items from the HCAHPS.41 We selected a specific item, rating the hospital on a scale of 0 to 10, as the outcome measure for objective 3. We chose this measure because it is currently used by the federal Value-based Purchasing Program, and therefore impacts hospital reimbursement,71 and is also a target for hospital-based quality improvement efforts. We piloted the postpartum survey among 10 women, using methods similar to those described for the antepartum survey. Survey completion time was <30 minutes.
Nielsen conducted the postpartum survey in both English and Spanish as a follow-up to the antepartum survey, using methods similar to those previously described. Nielsen contacted women who completed the antepartum survey approximately 3 weeks after their due date. Nonresponders received weekly reminders on a rolling basis until they completed the survey or until the field period ended.
The postpartum response rate estimate was 30%, a number based on similar surveys done by Nielsen for postpartum women.72 We used this baseline to perform a sensitivity analysis and determined that given n = 2757 women who had taken the antepartum survey, and anticipating 825 (30%) postpartum responses, with the exclusion of 214 (26%) nonlaboring patients, (3%) home and freestanding birth center births, and 59 (10% of remaining 586) for missing data, yielding N = 527. In a linear equation that uses the outcome of a summary measure for hospital rating (0-10), a sample of N = 527 women achieves 80% power to detect an R2 of 0.01 attributed to 1 independent variable with a significance level (α) of .05 and adjusted for an additional 25 independent variables with an R2 of 0.30. However, the response rate varied between 15% to 20% per week. Approximately 2 months after starting the postpartum survey in the field, Nielsen employed efforts to increase participation. It upgraded the incentive from $10 to $15 in reward points redeemable for gift cards or merchandise; improved the survey-taking experience on mobile devices; revised invitation and reminder language; sent alert emails 5 weeks before the survey's due date; and, where possible, made phone calls to nonresponders.
To collect the contracted number of postpartum responses, Nielsen initiated a second round of antepartum and postpartum surveys, using the same online panels except for Peanut Labs. It used the same methodology as in the first round but maintained incentives at $15 for completion of both the antepartum and postpartum surveys (see Figure 2).
Nielsen provided a deidentified data set that linked the antepartum and postpartum survey responses, tabulating predisposing characteristics, V&P, PROs, and satisfaction data per respondent. The investigative team derived “gap data” to reflect differences between the V&P and PRO data items as follows. We dichotomized V&P items, with few exceptions, reflecting whether a respondent did or did not want an item. For example, the V&P item regarding whether the woman wanted the baby placed skin to skin immediately following delivery was originally a 5-point ordinal score that was dichotomized into “very” or “extremely” important versus the other responses. We also dichotomized most PRO items, reflecting whether a respondent did or did not get a service item or outcome. For example, in the postpartum survey we asked whether a woman “got” the service, in this case, the baby was placed skin to skin upon delivery. We defined “gap data” in 4 categories: (1) respondent did not want the item and did not get it; (2) respondent did not want the item but got it; (3) respondent wanted the item and got it; and (4) respondent wanted the item but did not get it.
Objective 3 focused on participants who answered both the antepartum and postpartum survey. We further restricted this group to those women who noted on the antepartum survey that they anticipated having a vaginal delivery in a hospital and on the postpartum survey stated that they labored and delivered in a hospital (either by cesarean or vaginal birth). We chose this group of women as the denominator to maximize data interpretability because the antepartum survey had different items for women who anticipated an elective (scheduled) cesarean delivery and those who anticipated a vaginal delivery. Specifically, items related to V&P about labor (most items) were not relevant for those who planned a cesarean.
We tested data from respondents who met these criteria for bivariate association with hospital satisfaction, an HCAHPS item that asks respondents to rate their hospital on a scale of 0 to 10. We operationalized this measure of satisfaction by dichotomizing it at a score of 9 to 10 (satisfied) versus 0 to 8 (unsatisfied). We chose this outcome because our principal goal for this study was the development of a foundation for childbirth hospital performance measurement. Furthermore, hospitals are familiar with this item, dividing it as above, using it to track hospital satisfaction, and relying on the premise that any score lower than 9 is meaningful.71
We did not weight analyses for postpartum data because weights were not developed for the postpartum sample. Means are reported ± the standard deviation. We adjusted odds ratios (ORs) for maternal age, race/ethnicity, education level, parity/prior cesarean birth, US region, pregnancy complications before admission (by either antepartum or postpartum survey), overall health (antepartum survey), and overall mental/emotional health (antepartum survey) and included 95% CIs.
To investigate the relationship between predisposing conditions, V&P, PROs, and gap data with women's satisfaction with hospital childbirth services, we used information from the bivariate frequency tables to build the final multivariate models. In these models, women's satisfaction with hospital childbirth services was the dependent variable, and independent variables were chosen from the items for predisposing conditions, V&P, PROs, and gaps. We built all models using backward, stepwise, and forward multiple logistic regression techniques. We considered any differences in covariates selected to each particular model and made final model decisions based on face validity as evaluated by the team and by improvement in the C statistic. We chose 9 variables—(1) maternal age, (2) race/ethnicity, (3) education level, (4) multiple gestation, (5) delivery category (combination of multiparity and prior cesarean delivery), (6) US region, (7) complicated pregnancy (based on a positive response to either the antepartum or postpartum items regarding clinical risk), (8) antepartum overall health, and (9) antepartum mental/emotional health—to be forced into all models and excluded all variables missing 20 or more responses.
We hypothesized that there were 4 categories of potential predictors of overall women's satisfaction with hospital childbirth services: (1) predisposing conditions, (2) V&P items, (3) PROs, and (4) gap data (eg, wanted but did not get). The study team further evaluated V&P items and PROs for their close relation to women's satisfaction with hospital childbirth services and for their specification of actionable or mutable services, practices, or policies.
To better understand the data and to limit the number of predictors, we tested predisposing conditions against the dichotomized variable for women's satisfaction with hospital childbirth services as described previously. In addition to the variables forced into all models, any predisposing condition with a P < .10 for bivariate analysis was eligible to be entered in the model. This P value allowed for a slightly more liberal inclusion criterion than normally used (P < .05) and created an alternative to forcing more predisposing conditions in the models. The predictors identified here were eligible to be used in the final models.
We performed multivariate logistic regression modeling using the predisposing conditions that reached statistical significance in the previous step (P < .10), as well as all V&P, PRO, and gap items that had a P < .05 in bivariate analysis with women's satisfaction with hospital childbirth services. Gap items with a nonsignificant 10% difference in categories were also eligible for inclusion, given that a 10% difference might be clinically relevant. Because 3 potential entries existed for a similar item (the V&P item, the PRO item, or the gap item), any significant one was eligible for the model. In the case of competing similar items, we first ranked items in order of the chi-square of the bivariate association with satisfaction, and sequentially tested in the models to determine which, if any, contributed to the model with the highest C statistic. We retained the model with the highest C statistic.
In accordance with PROMIS guidelines, our next step was to format the selected items (listed in Table 3) in a uniform style (uniform instructions and response options)45 and perform cognitive debriefing for the items' content validity (Table 2).43 On the basis of additional discussion with the community partners as well as interviews with pregnant and postpartum women, we crafted a final iteration of the item bank, specifying the relevant domains in the conceptual framework. The final childbirth-specific preliminary item bank included items that specified predisposing conditions, V&P, PROs, and gaps.
Initial Domains and Subdomains Identified Through Literature Review and Focus Groups, and the Number of Items per Domain.
The following sections present the results of objective 1 (literature search and domain mapping), objective 2 (the antepartum survey), objective 3 (the postpartum survey), and objective 4 (the conceptual framework and final preliminary item bank). Figure 3 provides a flow diagram of the steps in the item selection process.
Flow Diagram of Steps in the Item Selection Process.
Building on the initial conceptual framework, the search strategies identified 5102 unique titles; from these, we identified 5902 relevant PRO items. In collaboration with the Childbirth PRO Partnership participants, we categorized these items into 19 domains and 58 subdomains (Table 3). We conducted 8 focus groups with 45 women of varying age, race/ethnicity, socioeconomic background, and region. Each focus group included 3 to 10 women. One focus group (n = 8) was facilitated in Spanish. We captured the value expectations of women who anticipated delivering or had delivered at a hospital, freestanding birth center, or home.
Focus group data confirmed the importance of these 19 priority domains; the 3 most frequently discussed domains were communication, involvement in decision-making, and the need for respect and empathy. Only 1 new subdomain that was not part of the literature search emerged from the focus groups—health insurance concerns. This included the nuances of different types of services, hospitals, and deductibles in different types of networks. While this concern arose in only 1 focus group, all women within that group thought it was important, each raising her own individual coverage issues. As a result, we added insurance/cost of care as a subdomain under decision-making. After the winnowing process, 68 V&P items and 64 items describing predisposing conditions remained.
We administered the survey in November 2015 over a 2-week period. Of 22 503 logins to the survey, 2757 fully qualified respondents completed it. Twenty-nine surveys (1.1%) were in Spanish. Of these respondents, 2033 (73.7%) anticipated a vaginal birth in a hospital; 217 (7.9%) anticipated a hospital birth but were uncertain regarding the planned delivery route; 393 (14.3%) anticipated a cesarean delivery; 23 (0.8%) anticipated delivery in a freestanding birth center; 47 (1.7%) anticipated delivering at home; 17 (0.6%) anticipated a vaginal delivery but were unsure of location; and 27 (1.0%) gave inconsistent or incomplete responses.
All geographic regions were represented. Most (55%) respondents were White, had at least some college (64%), and planned to be delivered by an obstetrician (69%). A third of the respondents (33.1%) were 30 to 34 years old, and 41% made at least $35 000 per year (Appendix B). Although 17% were Hispanic, only 7% indicated they needed an interpreter (language not specified); approximately 2% of women took the Spanish version of the survey. Table 4 lists the frequency distribution of the 37 predisposing conditions tested in the national sample.
Frequency of 37 Predisposing Conditions in the National Sample.
For the predisposing conditions, as part of the factor analysis, we extracted 2 factors: discrimination (6 items; α = .89) and confidence (8 items; α = .76). Both factors used the 5-point response scale 1 = strongly disagree to 5 = strongly agree. We retained all other predisposing conditions as independent items. For the V&P, we extracted 4 factors. Overall and segmented factor analyses were consistent. These factors were (1) choice of labor environment (6 items; α .72); (2) communication regarding the newborn (8 items; α .89); (3) option to use labor tub, ball, or stool (3 items; α .90); and (4) desire to avoid interventions (6 items; α .80). All the involved items used the “importance” response scale. We calculated factor-based scores and collapsed to produce binary items for all the above factors. All remaining V&P remained as independent items.
The results of the multiple logistic regression models for those who anticipated a vaginal delivery appear in Table 5, which details the predisposing conditions associated (positively or negatively) with each V&P item, organized by the conceptual framework domain. In general, women who had high confidence, those who prepared a birth plan, and those who anticipated coping well with labor pain expressed preference for a more physiological birth and willingness to being more involved and in control of their childbirth.
Results of Multiple Logistic Regression Models for Women Considering Having a Vaginal Delivery, by Domain (Total N, Weighted = 2218; all Ns Are Weighted).
For Round 1, we collected antepartum survey data in November 2015 and postpartum data from December 2015 through June 2016. Of 2757 antepartum respondents, 399 (14.5%) also responded postpartum. For Round 2, we collected antepartum data from February through April 2016 and postpartum data from April through October 2016. Of 2098 antepartum respondents, 439 (20.9%) also responded postpartum. Of the total 838 respondents who answered both surveys, 500 (59.7%) met inclusion criteria (anticipated vaginal delivery and labored/delivered in a hospital), and 58 (11.6%) of these had a cesarean delivery. The mean number of weeks for completion of the postpartum response was 6.7 (5.0).
The mean rate for women's satisfaction with hospital childbirth services for this group of 500 women who answered both surveys was 8.6 ± 1.6, with a median of 9.0. Approximately 50% (59.6%; n = 298) had a “high” satisfaction score (≥9). We describe predisposing conditions and their association with women's satisfaction with hospital childbirth services separately in Table 6. Good overall health, good mental health, high confidence, and confidence filling out forms were the predisposing conditions most significantly associated with women's satisfaction with hospital childbirth services.
Frequencies of Key Predisposing Conditions in the Postpartum Population and Their Association With Women's Satisfaction With Hospital Childbirth Services (N [Unweighted] = 500).
We tested the association of each V&P, PRO, and gap item with women's satisfaction with hospital childbirth services and reported those variables to have an association with a statistical significance of P < .05 or a nonsignificant 10% difference between any of the gap data categories, as shown in Table 7. The only V&P item significantly associated with women's satisfaction with hospital childbirth services was “wanted partner/support person in the room.” Several gap variables and numerous postpartum PROs reached statistical significance.
V&P, PROs, and Gap Data Statistically Significantly Associated With Women's Satisfaction With Hospital Childbirth Services; N (unweighted) = 500.
Variables that describe intrapartum and postpartum clinical complications appear in Table 8. None of the clinical complication items achieved statistical significance at the P < .05 level with respect to women's satisfaction with hospital childbirth services after adjustment.
Clinical variables and women's satisfaction with hospital childbirth services.
Because of the large number of potential predictors, we built the models of women's satisfaction with hospital childbirth services in steps. For the first step, we determined the predisposing conditions associated with women's satisfaction with hospital childbirth services. Table 9 describes the modeling of the predisposing conditions with women's satisfaction with hospital childbirth services, yielding “high confidence” as the only predictor of women's satisfaction with hospital childbirth services (in addition to the covariates used for model adjustment) retained in subsequent models (N = 489; C statistic = 0.637).
Multiple Logistic Regression Model Results for Women's Satisfaction With Hospital Childbirth Services Using Predisposing Conditions Only (N = 489 With C Statistic = 0.637).
Table 10 describes the variables eligible for inclusion in the final model of women's satisfaction with hospital childbirth services.
Final List of Items That Were Eligible for the Model of Women's Satisfaction With Hospital Childbirth Services (in Addition to the 9 Forced Covariates).
Table 11 describes the final model of women's satisfaction with hospital childbirth services considering all the variables in Table 8 as potential covariates (N = 479; C statistic = 0.845).
Model for Predictors of Women's Satisfaction With Hospital Childbirth Services, Including Childbirth-specific and Nonspecific High-level Items (N = 479 With C Statistic = 0.845).
Upon building the model described in Table 11, we realized that most (7 of 9) of the retained items apart from the forced covariates did not explicitly suggest actions hospital staff could take to improve the patient experience. For example, 1 of these items is that the patient felt the staff was compassionate, a quality that may not be easily or consistently translated into a prescribed set of staff behaviors. On the other hand, use of a birthing stool, a specific piece of equipment included in the model and highly associated with women's satisfaction with hospital childbirth services, could easily be accommodated by staff. Distinguishing this difference is important as it provides an opportunity to improve patient satisfaction by providing or performing these actionable items. This result prompted us to create an alternative model that excluded predictors of women's satisfaction with hospital childbirth services that we felt were difficult to act on. We refer to these as “high-level” items. We empirically selected 17 items from Table 10 that we labeled as high-level items and therefore not directly actionable:
Table 12 describes this alternative model of women's satisfaction with hospital childbirth services based on all the actionable items in Table 10 and excluding the high-level items (N = 465; C statistic = 0.762). This model had a lower C statistic compared with the previous model in Table 11 because of the exclusion of the high-level items. However, the items in the alternative model were more explicit, informative, and/or actionable, as follows: coped well with labor pain (postpartum), continuous electronic fetal monitoring, adequate space/food for support person, debriefed regarding events of labor, practical support for breastfeeding, was told about progress in labor, wanted massage and got it (gap), and wanted partner/support person in the room (V&P).
Model for Predictors of Women's Satisfaction With Hospital Childbirth Services.
Upon completion of the above aims, we performed cognitive debriefing (see Table 2) to test for (1) comprehension (What did the patient believe the question was trying to ask?); (2) memory retrieval process (What strategy was employed to retrieve information to answer the question?); (3) social desirability (Was the patient motivated by social desirability [or pressure] in answering the question?); and (4) response processing (Did the patient's internal response metric for an item match those of the question?). We then finalized the domains and items (Table 13).
Framework for Childbirth V&P, PROs, and Predisposing Conditions.
Table 14 highlights those items that had the largest statistically significant odds ratios with respect to women's satisfaction with hospital childbirth services. We propose to include these items in a Childbirth Experiences and Outcomes Survey to be disseminated and implemented through an antepartum and postpartum patient-reported data collection process as we take the most meaningful and logical next steps. The items used in the antepartum and postpartum surveys described in this report appear in Appendices C and D, respectively. We anticipate that the next version of the Childbirth Experiences and Outcomes Survey will be a shortened version of these 2 surveys.
Key Items Needed for Collection in Childbirth Experiences and Outcomes Survey.
This work provides a foundation for assessing what is important to women during their childbirth experience and emphasizes the need for both antepartum and postpartum data collection to ensure the reporting of predisposing factors, services valued and preferred, services received, and clinical outcomes. Our conceptual framework suggests that, for childbirth, measurement of both V&P and PROs is important. Using PROMIS methodology and a community-based research approach, we developed a conceptual framework, a preliminary item bank of predisposing conditions, and items relevant to women's V&P and PROs for childbirth in a hospital.
Our research addresses a long-standing evidence gap regarding the drivers of women's assessment of their childbirth experience. Although physicians and hospitals have focused on improving the safety of childbirth, women's V&P—including, but not limited to, safety—remain unexplored.
An early attempt to address this void was first published in a national survey, Listening to Mothers, in 2002. The report described childbirth experiences but did not systematically address V&P or PROs.74 Since the funding of our current PCORI project, the International Consortium for Health Outcomes Measurement (ICHOM) has developed a much broader and less specific set of standards for measuring pregnancy and childbirth outcomes that include several maternity patient self-reports.75 Furthermore, Gartner et al. developed core domains for women's birth-specific priorities that were largely consistent with our work.76 Neither this effort, nor that of ICHOM, measured the statistical significance of these domains with respect to the overall childbirth experience.
Our work narrows this long-standing evidence gap, offers a tool to assess women's V&P, and identifies the childbirth services that should be optimized to have the greatest impact on women's satisfaction. Many of our findings identify specific, actionable items that hospitals could readily address.
Our results are consistent with multiple studies13-15, 77-79 demonstrating that fulfillment of women's antenatal V&P (ie, what they desire, prefer, expect, think is important, or think should be important55,56) is a strong determinant of women's satisfaction with hospital childbirth services. In addition, components of the childbirth process, including not only labor and pain management but also the supportive services provided and quality of communication, appear to be as relevant as some clinical outcomes. These results are consistent with satisfaction studies for patients hospitalized for other health conditions.80-84
Additionally, our results confirmed the importance of the domains covered in the HCAHPS survey, a generic assessment of the following dimensions of the patient experience: communication with nurses, communication with doctors, responsiveness of staff, pain management, cleanliness and quietness of hospital environment, communication about medicines, and adequacy of discharge information.85 As suggested by the results of the focus group analyses, those PROs with the strongest associations were in our framework domains of (1) communication and decision-making, and (2) empathy and respect (Table 13). In the postpartum analyses, “high-level” items such as “staff was compassionate” and “the doctors explained things in a way I could understand” demonstrated strong associations with women's satisfaction with hospital childbirth services.
These findings firmly ground our results for pregnant women in the existing body of work regarding the elements of the patient experience that predict women's satisfaction with hospital childbirth services. However, our results go beyond this confirmation. We were also able to identify 23 PROs in Table 10 describing explicit, childbirth-related services and experiences that ultimately had an important association with women's satisfaction with hospital childbirth services. This set of items is a key result of our work because it gives childbirth providers and hospitals specific avenues for improving the childbirth hospital experience and for developing paths toward improving the more broad-based need for compassion, respect, empathy, and communication with staff.
The communication and decision-making domain included this set of explicit items: “was told about my progress in labor” and “was debriefed regarding events during labor.” The empathy and respect domain included these items specifically relevant to childbirth: “had adequate space and food for my support partner” and “was able to choose who was in the room during procedures.” Because these postpartum PROs were independent predictors of satisfaction (ie, not strongly associated with antepartum V&P), they are candidates for a menu of “universally desired” components of the childbirth experience.
The final conceptual framework had 15 domains and 46 subdomains, and the preliminary item bank had 100 V&P/PROs and 60 personal characteristics that were important predictors of these V&P/PROs. We developed a preliminary draft (English and Spanish versions) of a Childbirth Experience and Outcome Survey consisting of 2 parts: antepartum (documenting predisposing conditions and evaluating V&P) and postpartum (evaluating self-reported experiences and outcome). Each survey took approximately 30 minutes to complete.
The results reported here focused on the immediate hospital experience of women who anticipated vaginal hospital births. Of the 37 V&P tested as either single items or factors in the antepartum survey, some were desired by nearly all respondents (eg, having reassurance/comfort from the nurse [96.1%]), some by a moderate proportion of the respondents (eg, wanting to eat/drink during labor [56.0%]), and some by relatively few respondents (eg, wanting acupuncture/acupressure as a pain treatment option [6.5%]). These results confirmed that childbirth is a highly preference-sensitive condition73 and suggest that childbirth services preferences must be elicited and not inferred.
We confirmed our hypothesis that the desire for specific childbirth services and outcomes (V&P) varied not only across demographic groups but also across women with different levels of confidence, different levels of pain coping ability, and different attitudes toward childbirth preparedness. Some models performed better than others, with C statistics ranging from about 0.6 to 0.8. These results will guide us in determining which items should be retained for further refinement of the Childbirth Experiences and Outcomes Survey.
Our findings suggest the necessity for new data collection efforts if providers want the ability to predict “who” wants “what,” because much of this information (eg, levels of confidence, pain coping ability) is not routinely asked of pregnant women.
Of note is that women's reports of pregnancy complications rarely contributed to the V&P models. Nearly half (41.4%) reported having a complicated pregnancy, yet this perception did not appear to affect their desired outcomes.
The postpartum data analysis yielded several important results. First, items from each of the potential predictor categories (ie, predisposing conditions, V&P, gaps, and PROs) were independently associated with women's satisfaction with hospital childbirth services. This confirms our hypothesis that all these predictor categories include important items; it also raises the potential for identifying and possibly mitigating some of these items in the predisposing conditions and V&P categories in advance.
Second, we found few predisposing conditions independently associated with women's satisfaction with hospital childbirth services. Demographics, parity, and reported pregnancy complications had no demonstrable association with women's satisfaction with hospital childbirth services in bivariate or multivariate analysis. Some bivariate associations did occur for reports of overall health and overall mental health, a result that is well described in satisfaction literature for other patient populations. However, only mental health reported as poor or fair remained consistently (and negatively) associated with satisfaction in all postpartum models. Other predisposing conditions, including high maternal confidence and literacy (confidence in filling out medical/health forms), were generally more important, particularly in bivariate analysis. Both had strong positive associations with satisfaction.
Third, although fewer of those who reported clinical complications, such as transfusion or intensive care unit admission, appeared highly satisfied with the hospital, these differences rarely reached statistical significance. Cesarean delivery, defined as emergent in this population, was, in fact, positively associated with women's satisfaction with hospital childbirth services, although this did not reach statistical significance (OR 1.79 [95% CI, 0.94-3.41]; P = .0785).
Fourth, “high-level” items (eg, “the staff was compassionate,” “doctors explained things in a way I could understand”) dominated the full model for women's satisfaction with childbirth hospital services (Table 9) and confirms the importance of such items, present in the original HCAHPS model. Most of the more explicit items regarding childbirth, found in Table 8, were not retained by the model. Because this project's goal of was to focus on “actionable” items that could be addressed to improve women's satisfaction with hospital childbirth services, our alternative modeling attempt (Table 10) excluded these high-level variables and resulted in a more explicit model featuring 6 PRO items (“coped well with labor pain,” “had continuous electronic fetal monitoring,” “had adequate space/food for support person,” “got debriefed regarding events during labor,” “received practical support for feeding the newborn,” “and was told about progress in labor”), 1 gap item (“wanted and got a massage”), and 1 V&P item (“wanted the spouse/partner in the room”). The differences between the results in Tables 9 and 10 suggest the need to further explore these high-level variables.
We have developed an early version of the Childbirth Experiences and Outcomes Survey for use before birth, to allow the opportunity for discussion between patients and providers, and after birth, to determine whether women received the services and outcomes they wanted. A survey instrument that identifies women's V&P for childbirth has not previously been available. The development of this instrument fills an existing gap, bringing our work to Step 4 in the NQF pathway for the development of performance measures.72 The next step is implementation in multiple hospitals. A draft of the preliminary Childbirth Experiences and Outcomes Survey is in Appendices C (antepartum) and D (postpartum).
The implementation of a data collection and reporting process for childbirth-specific V&P/PROs has the potential to inform the health care decisions made by hospitals, by providers, and by pregnant women themselves. Hospitals determine their policies and patient services and can evaluate the availability of those services against what their patients want. For example, most hospitals do not support vaginal delivery of twins or vaginal birth after cesarean (VBAC) because of concerns about liability, limited expertise, and/or limited resources. In response, some women may have undergone labor outside the hospital to avoid automatic cesarean delivery or opted to deliver at birth centers or at home with lay midwives.86 If meeting these childbirth preferences is a high priority for some women, hospitals may want to further examine the potential for offering these services, thereby increasing the safety of childbirth and patient satisfaction.
Hospitals can also prioritize improvements in general patient–staff interactions, ensuring women's participation in labor and pain management decisions. For example, staff training, performance/quality monitoring, condition-specific toolkits, order sets, policies, and protocols are all tools that can support the dynamic interactions between staff and laboring patients.
The integration of childbirth-specific PROs into the hospital setting and the development of performance measures with the potential for public release could provide families with valuable information in choosing a childbirth hospital that fits their personal and clinical needs. Such performance measures would also be of interest to employers and insurers who negotiate benefit packages with childbirth hospitals.
We confirmed our hypothesis that the desire for specific childbirth services and outcomes varied not only across demographic groups but also across women with different levels of confidence, different levels of pain coping ability, and different attitudes toward childbirth preparedness. First, we identified items from each of the potential predictor categories (ie, predisposing conditions, V&P, gaps, and PROs) independently associated with women's satisfaction with hospital childbirth services. This confirms our hypothesis that all the categories may include important items, and raises the potential identifying and possibly mitigating in advance some of these items in the predisposing conditions and V&P categories.
Further validating and testing the Childbirth Experiences and Outcomes Survey in a multihospital environment is the obvious next step, and participants in the expanded stakeholders meeting have volunteered their hospitals for dissemination and implementation feasibility testing.
Strengths of this work include the use of PROMIS methodology to develop and build on the conceptual framework and the community-based research approach. This foundational effort can be expanded on and serve as a basis for continued advancement using the methodologies promoted by PROMIS, NQF, PCORI, or AHRQ.
We developed 3 products: (1) a conceptual framework and childbirth V&P/PRO preliminary item bank; (2) an antepartum survey that demonstrates the variation in V&P by different predisposing conditions; and (3) a postpartum survey that demonstrates the relationship between various potential categories of predictors (ie, predisposing conditions, V&P, gaps, and PROs) and women's satisfaction with hospital childbirth services. Given the extensive literature review and item search, we believe the framework and preliminary item bank should serve as a solid foundation for further development of childbirth V&P and PROs for US women, although further domains and items may continue to be developed.
The antepartum survey confirmed our hypothesis that the desire for specific childbirth services and outcomes varied not only across demographic groups but also across women with different levels of confidence, different levels of pain coping ability, and different attitudes toward childbirth preparedness. This variation is likely to be found in most test settings, including hospital populations, and supports the concept that women's childbirth services preferences cannot necessarily be inferred.
A sample recruited through Nielsen produced data for the antepartum survey regarding patient V&P. These data were weighted by relevant demographic characteristics to maximize generalizability to the US population of reproductive-age women. Thus, our results describing “who wants what” should generally describe the preferences of pregnant women in the United States. Further exploration and confirmation of the statistical significance of these predisposing conditions will take place in the future in the hospital setting, where different hospitals will likely have different base populations that vary by both demographics and predisposing conditions.
The third set of results, which relates to the postpartum data, confirms our hypothesis that antepartum V&P and gap data may also contribute to women's satisfaction with hospital childbirth services. However, we found many items associated with women's satisfaction with hospital childbirth services, and we anticipate that as we further develop the Childbirth Experiences and Outcomes Survey, new test settings may alter what items contribute to the final satisfaction models. Also, as we employ a wider variety of satisfaction measures (eg, birth satisfaction, hospital loyalty), we may also find that certain items or categories of items are most important. The postpartum results were not weighted and are less likely to be generalizable. We recognize the potential for recruitment bias based on the online panels used for recruitment and the low response rate for the postpartum survey. However, we did have women of all age groups, racial/ethnic groups, and geographic regions. Furthermore, our postpartum findings are consistent with findings from the literature, as noted previously.
The findings presented here are specific to the services women received while in labor and in the immediate postpartum period (before hospital discharge). For interpretability, we limited our analysis to women who planned a vaginal birth in a hospital. We have data on women planning a cesarean and women planning births at home or at a birth center (approximately 2%, consistent with national estimates), but the numbers of these women are too limited to be explored sufficiently in adjusted analyses. Several of our community partners have expressed an interest in participating in analyses of these cohorts, but conclusions will be limited by the small sample sizes.87 Additional data should be collected in these patient populations to confirm similarities and differences from women planning hospital births.
This study is an early effort in the development of an approach to evaluating women's childbirth experiences and outcomes, so there are many limitations to this work. Most of these limitations can be addressed through continued development using the documented PROMIS, NQF, or AHRQ methodologies and will depend on funding opportunities. These limitations and potential efforts to mitigate them appear in Table 15.
Study Limitations and Future Mitigation Efforts.
We envision at least 6 opportunities for future research, which are outlined below.
This work brings us to Step 4 of the NQF Pathway for the development of PROs as hospital performance measures.72 The NQF outlines a clear research path proceeding through implementation of the PROs in the hospital environment and comparisons across hospitals to determine PRO variation in this environment and the potential for quality improvement. The further development of the Childbirth Experiences and Outcomes Survey may also lead to integration in AHRQ's CAHPS suite of patient experience surveys. We are proposing that the next meaningful and logical step is to demonstrate the feasibility of such an implementation in a limited number of hospitals.
Continued development under the PROMIS methodology could lead to a better understanding of women's assessments of their own health and their newborn's health after the childbirth experience. Comparison of clinical events (eg, through the medical record) with self-assessments would advance our understanding of the extent to which self-assessments are clinically accurate and reliable in the absence of electronic medical record linkage.
Once an infrastructure is in place to measure women's V&P/PROs, this information can help providers and hospitals identify vulnerable patients (ie, those unlikely to be satisfied with their care) and develop strategies to address anticipated gaps (ie, unfulfilled preferences) before delivery. Types of strategies and their effectiveness in improving the patient experience remain to be explored.
Based on Figure 1 and supported by the results of the national survey, hospitals that address women's childbirth priorities (V&P) should improve their patient experience. It remains unknown whether improved attention to women's V&P will impact maternal or neonatal clinical outcomes. Because to date there have been no formal mechanisms for identifying what women want in childbirth, research in this area is limited. To elicit and document women's priorities would encourage caregivers to respond to these priorities, and would begin to define the following: (1) the domains in which women have choices, (2) under which circumstances and to what extent those choices exist, (3) the potential for mutability of these choices, and (4) methods to help clarify choices (eg, through education of both patients and providers).88,89
Birth satisfaction is an important patient-centered outcome. The proposed conceptual framework can facilitate further study. Further exploration of V&P/PROs with respect to subpopulations such as those anticipating a scheduled cesarean birth and those planning to deliver outside the hospital Some women schedule a cesarean birth for clinical indications, and others do so out of preference. Providers and hospitals need a deeper understanding of women's preferences regarding route and location of delivery. We have begun to explore some of the subpopulations (women who want home births or VBACs) with our community partners as lead authors.
In conclusion, we have developed a conceptual framework and preliminary item bank for childbirth-specific patient-reported V&P and experiences and outcomes. We have explored the statistical significance of these V&P/PROs with respect to their association with women's satisfaction with hospital childbirth services and have developed a Childbirth Experiences and Outcomes Survey based on these results. Throughout this process we have relied on community-based participatory research techniques and the PROMIS guidelines for item bank development. We have also adhered to the PCORI Methodology Standards. Our work is consistent with prior work in both the childbirth satisfaction literature and the general patient satisfaction literature, and it specifically identifies domains of care and actionable items providers and hospitals can address to improve the patient experience. Our study findings will be useful to hospital administrators and maternity care providers who want to improve the patient care experience and their hospital satisfaction scores.
The next meaningful and logical step for the further development of this framework and preliminary item bank is to implement a data collection system for the childbirth predisposing conditions, V&P, PRO, and gap items in a multiple-hospital setting, thereby making V&P/PRO data available to providers for clinical decision-making and to researchers for the development of childbirth hospital performance monitoring.
The NQF,90 the US national clearinghouse for the assessment and endorsement of health care performance measures, has published standards for the design and selection of PROs that relate to the performance of health care organizations. NQF has emphasized that the incorporation of the patient perspective into health care services quality monitoring must ensure that the infrastructure is in place to document and respond to that perspective, and that valid and comprehensive measures of that perspective are in place.25 The work described here lays a foundation for further development of childbirth V&P and PROs as hospital performance measures of the childbirth experience and outcomes.
We are grateful to PCORI for the opportunity to conduct this study and for our project officer, Jason Gerson, for his support and guidance. We would also like to acknowledge the members of The Childbirth Patient-Reported Outcomes (PRO) Partnership, who provided guidance and insight throughout the development of the project. The Childbirth PRO Partnership authors include (listed alphabetically by organization): Adriana Lozada and Jeanette McCulloch (Birthswell); Jennifer Anger, MD; Yalda Afshar, MD, PhD; Mykel LeCheminant, RNC, MSN, ICCE; Naomi Greene, PhD; Caroline Marshall, MLS, AHIP; Katy Sharma, MD; and Brennan Spiegel, MD, MSHS (Cedars-Sinai Medical Center); Lisa Bollman, RNC, MSN, CPHQ (Community Perinatal Network); Hindi Stohl, MD, JD (Harbor-UCLA Medical Center); Cordelia Hanna Cheruiyot, MPH, CHES, ICCE, CLE, CBA (The Association for Wholistic Maternal and Newborn Health); Sandra Applebaum; Peg Jaynes; and Roz Pierson, PhD (The Nielsen Company); Geraldine Perry-Williams, MSN, PHN (Pasadena Department of Public Health–Pasadena Black Infant Health Program); Diana Ramos, MD, MPH; Leslie Lopez, MPH, CHES; Joanne Roberts, PHN (Los Angeles County Department of Public Health); Janice French, CNM, MS (Los Angeles Best Babies Network); Nathana Lurvey, MD (South Bay Family Healthcare Center); Priya Batra, MD, MS (UCLA); Gerson Hernandez, MD (University of Southern California); and Minerva Pineda, MD, MPH (UCLA).
Research reported in this report was [partially] funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (#ME-1402-10249) Further information available at: https://www.pcori.org/research-results/2014/developing-item-bank-survey-questions-measure-womens-experiences-childbirth
Literature Search Strategy (PDF, 179K)
Unweighted and Weighted Demographic Profile of Antepartum Survey Participants (N= 2757) (PDF, 145K)
Antepartum Survey (PDF, 507K)
Postpartum Survey (PDF, 365K)
Gregory KD, Korst LM, Fridman K, et al. (2019). Developing an Item Bank of Survey Questions to Measure Women's Experiences with Childbirth in Hospitals. Patient-Centered Outcomes Research Institute (PCORI). https://doi.org/10.25302/3.2019.ME.140210249
The [views, statements, opinions] presented in this report are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee.
This book is distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License which permits noncommercial use and distribution provided the original author(s) and source are credited. (See https://creativecommons.org/licenses/by-nc-nd/4.0/
Your browsing activity is empty.
Activity recording is turned off.
See more...
