Advertisement

Development and Psychometric Evaluation of a Caregiver Survey to Assess Family-Centered Care in the Emergency Department

Open AccessPublished:October 22, 2022DOI:https://doi.org/10.1016/j.acap.2022.10.018

      Abstract

      Objective

      To develop and evaluate the psychometric properties of a family caregiver-reported survey that assesses family-centeredness of care in the context of pediatric emergency department (ED) encounters.

      Methods

      We created a caregiver-reported scale, incorporated content expert feedback, and iteratively revised it based on cognitive interviews with caregivers. We then field tested the scale in a survey with caregivers. We dichotomized items using top-box scoring and obtained a summary score per respondent. Using a sample of 191 caregivers recruited from 9 EDs, we analyzed internal consistency reliability, dimensionality via item response theory modeling, and convergent validity with the ED Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey.

      Results

      Feedback from the 9 experts led us to remove 4 items. We conducted 16 cognitive interviews and revised the survey in 4 rounds. An 11-item survey was field tested. Mean (standard deviation) respondent 11-item summary score was 77.2 (26.6). We removed 2 items given inconsistent response patterns, poor variability, and poor internal consistency, which increased coefficient alpha from 0.85 to 0.88 for the final scale. A multidimensional model fit the data best, but factor scores correlated strongly with summary scores, suggesting the latter are sufficient for quality improvement and future research. Regarding convergent validity, adjusted partial correlation between our scale's 9-item summary score and the ED CAHPS summary score was 0.75 (95% confidence interval 0.67–0.81).

      Conclusions

      Psychometric analyses demonstrated strong item performance, reliability, and convergent validity for the 9-item scale. This survey can be used to assess family-centered care in the ED for research and quality improvement purposes.

      Keywords

      What's New
      No family-centered care assessment tool is designed for a pediatric emergency department encounter. This survey demonstrated good item performance, reliability, and validity, supporting its use to assess family-centeredness of care in this setting for research and quality improvement purposes.
      The importance of family-centered care, which places the family as the focus and unit of care,
      • Coyne I
      • Holmström I
      • Söderbäck M.
      Centeredness in healthcare: a concept synthesis of family-centered care, person-centered care and child-centered care.
      is widely recognized among researchers, national health professional associations, and public health policy organizations.
      • Dudley N
      • Ackerman A
      • Brown KM
      • et al.
      Patient-and family-centered care of children in the emergency department.
      ,
      Institute of Medicine
      Crossing the Quality Chasm: A New Healthcare for the 21st Century.
      Evidence demonstrates that family-centered care is associated with improvements in care quality, trust, medical goal achievement, patient and family satisfaction, patient and family anxiety, and costs.
      • Coyne I
      • Holmström I
      • Söderbäck M.
      Centeredness in healthcare: a concept synthesis of family-centered care, person-centered care and child-centered care.
      ,
      • Kuo DZ
      • Houtrow AJ
      • Arango P
      • et al.
      Family-centered care: current applications and future directions in pediatric health care.
      ,
      • Goldfarb MJ
      • Bibas L
      • Bartlett V
      • et al.
      Outcomes of patient-and family-centered care interventions in the ICU: a systematic review and meta-analysis.
      Key attributes of family-centered care include collaboration among patients, families, and health care providers; empowerment; respect; communication; individualized care; and support.
      • Coyne I
      • Holmström I
      • Söderbäck M.
      Centeredness in healthcare: a concept synthesis of family-centered care, person-centered care and child-centered care.
      ,
      Committee on Hospital Care and Institute for Patient- and Family-Centered Care
      Patient-and family-centered care and the pediatrician's role.
      Challenges to successful implementation of family-centered care include family and care team stress and competing demands, power imbalances, and organizational limitations.
      • Mirlashari J
      • Brown H
      • Fomani FK
      • et al.
      The challenges of implementing family-centered care in NICU from the perspectives of physicians and nurses.
      ,
      • Oude Maatman SM
      • Bohlin K
      • Lilliesköld S
      • et al.
      Factors influencing implementation of family-centered care in a neonatal intensive care unit.
      These challenges are especially evident in the emergency department (ED) setting, where family caregivers are navigating a new or serious illness in a new setting with unfamiliar care providers.
      • Dudley N
      • Ackerman A
      • Brown KM
      • et al.
      Patient-and family-centered care of children in the emergency department.
      To improve the delivery of family-centered care, the health care field needs evaluation mechanisms to assess whether care is family-centered. While the Consumer Assessment of Healthcare Providers and Systems (CAHPS)

      Agency for Healthcare Research and Quality. CAHPS patient experience surveys and guidance. Available at: https://www.ahrq.gov/cahps/surveys-guidance/index.html. 2021. Accessed December 9, 2021.

      surveys measure patient experience, there is no national standard to measure patient- and family-centered care.

      National Quality Forum Leadership Consortium. National quality forum leadership consortium 2022 priorities for action. Available at:https://www.qualityforum.org/Publications/2021/12/2022_NQF_Priorities_for_Action.aspx. 2021. Accessed March 2, 2022.

      For this reason, the National Quality Forum identified “measurement of person-centered care” as 1 of 3 priority initiatives.

      National Quality Forum Leadership Consortium. National quality forum leadership consortium 2022 priorities for action. Available at:https://www.qualityforum.org/Publications/2021/12/2022_NQF_Priorities_for_Action.aspx. 2021. Accessed March 2, 2022.

      No published family-centered care measure was designed for pediatric ED encounters. Existing family-centered care measures assess the physician's approach based on multiple encounters over time (eg, “In the last 12 months…”)
      • Lindly OJ
      • Geldhof GJ
      • Acock AC
      • et al.
      Family-centered care measurement and associations with unmet health care need among US children.
      or include items that assume established relationships (eg, “My doctor and I have been through a lot together”).
      • Etz RS
      • Zyzanski SJ
      • Gonzalez MM
      • et al.
      A new comprehensive measure of high-value aspects of primary care.
      Such measures that were designed for the primary care or clinic setting are not appropriate for a single ED encounter. Other family-centered care measures were designed for patients with specific chronic diseases.
      • King S
      • King G
      • Rosenbaum P.
      Evaluating health service delivery to children with chronic conditions and their families: development of a refined measure of processes of care (MPOC-20).
      Furthermore, the body of literature includes patient-reported surveys to assess patient-centered care.
      • Little P
      • Everitt H
      • Williamson I
      • et al.
      Observational study of effect of patient centredness and positive approach on outcomes of general practice consultations.
      Our team therefore wanted to address this gap by providing a caregiver-reported measurement that is applicable for pediatric ED visits. This research focused on developing and evaluating the psychometric properties of a caregiver-reported survey to assess family-centeredness of care in the context of pediatric ED encounters.

      Methods

      We used standard procedures for instrument development
      • Frost MH
      • Reeve BB
      • Liepa AM
      • et al.
      What is sufficient evidence for the reliability and validity of patient-reported outcome measures?.
      (Fig. 1) to draft the ED Family Centered Care Experience (ED-FACCE) survey, a caregiver-reported measure of family-centered care provided in the ED, to be used for research and quality improvement purposes. We solicited and incorporated feedback on the preliminary instrument from content experts. The instrument was then iteratively revised through cognitive interviews with caregivers, whereby caregivers completed the instrument and then participated in a semi-structured interview. Finally, we field tested the instrument with caregivers and analyzed the scale's psychometric properties. We determined the reading grade level of the instrument using the Flesch-Kincaid scale (Microsoft, Redmond, Wash) throughout development. We included the entire text of the instrument (ie, instructions, items, response options) when assessing reading level. The University of California Institutional Review Board approved this study.
      Figure 1
      Figure 1Overview of instrument development and evaluation process.

      The Preliminary Instrument

      The initial instrument draft was informed by Little et al,
      • Little P
      • Everitt H
      • Williamson I
      • et al.
      Observational study of effect of patient centredness and positive approach on outcomes of general practice consultations.
      who developed a survey to assess patient-centered care using the patient as the rater, which is more accurate at predicting outcomes than using a provider or research observer.
      • Hudon C
      • Fortin M
      • Haggerty JL
      • et al.
      Measuring patients’ perceptions of patient-centered care: a systematic review of tools for family medicine.
      ,
      • Stewart M
      • Brown JB
      • Donner A
      • et al.
      The impact of patient-centered care on outcomes.
      Our instrument was developed to assess the care provided by physicians, because prior research suggests that physicians’ family-centeredness of care in the ED can be improved.
      • Rosenthal JL
      • Li S-TT
      • Hernandez L
      • et al.
      Familial caregiver and physician perceptions of the family-physician interactions during interfacility transfers.
      To adapt the Little et al survey to assess family-centered care for pediatric ED encounters from the caregiver's perspective, we identified content domains based on relevant published research and policy statements.
      • Coyne I
      • Holmström I
      • Söderbäck M.
      Centeredness in healthcare: a concept synthesis of family-centered care, person-centered care and child-centered care.
      ,
      Committee on Hospital Care and Institute for Patient- and Family-Centered Care
      Patient-and family-centered care and the pediatrician's role.
      ,
      • Byczkowski TL
      • Gillespie GL
      • Kennebeck SS
      • et al.
      Family-centered pediatric emergency care: a framework for measuring what parents want and value.
      ,
      • Klassen A
      • Dix D
      • Cano S
      • et al.
      Evaluating family-centred service in paediatric oncology with the measure of processes of care (MPOC-20).
      Individual items within these domains were adapted or written de novo using published guidelines for survey development.
      • Bradburn NM
      • Sudman S
      • Wansink B.
      Asking Questions: The Definitive Guide to Questionnaire Design—For Market Research, Political Polls, and Social and Health Questionnaires.
      Additional items were also adapted from the 20-item Measure of Processes of Care,
      • King S
      • King G
      • Rosenbaum P.
      Evaluating health service delivery to children with chronic conditions and their families: development of a refined measure of processes of care (MPOC-20).
      a survey used to measure caregivers' perceptions of family-centered behaviors of health care professionals for children with disabilities. The resulting initial instrument draft had 17 items in 4 domains: 1) listening to and respecting experiences and preferences, 2) sharing information, 3) providing support, and 4) collaborating and encouraging participation.
      Consistent with the CAHPS

      Agency for Healthcare Research and Quality. CAHPS patient experience surveys and guidance. Available at: https://www.ahrq.gov/cahps/surveys-guidance/index.html. 2021. Accessed December 9, 2021.

      surveys, the response format for the preliminary instrument items was a 4-point scale (never, sometimes, usually, always) for rating the frequency of experiences with doctors (eg, “During this emergency department visit, how often did your child's doctors encourage you to ask questions?”).

      Content Expert Assessment

      We convened a panel of 9 family-centered care experts, including 3 researchers, 1 leader of a family-centered care organization, 1 bioethicist and family-centered care author, 1 hospital-based family engagement director, 2 hospital-based family engagement program managers, and 1 pediatric emergency medicine chief, to evaluate content validity and potential augmentation of the item pool. We asked panelists to identify missing items or domains and to provide feedback on instrument wording and duplicative items. We also asked panelists to rate each item's relevance to assessing family-centeredness of care (coded as not relevant = 1, somewhat relevant = 2, quite relevant = 3, highly relevant = 4).
      • Davis LL.
      Instrument review: getting the most from a panel of experts.
      For each item, we computed the item-level content validity index (I-CVI), which was the number of experts giving a rating of 3 or 4 divided by the total number of experts.
      • Davis LL.
      Instrument review: getting the most from a panel of experts.
      Items with I-CVI below the thresholds established by Lynn
      • Lynn MR.
      Determination and quantification of content validity.
      (eg, I-CVI < 0.778 with 9–10 experts) were flagged as needing to be revised or removed. We calculated the scale-level content validity index (S-CVI/Ave) as the average I-CVI value.
      • Polit DF
      • Beck CT.
      The content validity index: are you sure you know what's being reported? Critique and recommendations.

      Pretesting: Cognitive Interviews

      Participants

      We conducted one-on-one cognitive interviews with English-speaking, adult parents or guardians who were at their child's bedside during an eligible encounter in 1 of 9 northern California EDs. Eligible encounters were for children aged 0 to 18 years, identified by care team providers (eg, physicians and nurses). We used convenience sampling to recruit participants to complete the instrument within 30 days of the ED encounter. The instrument was designed to be self-administered (electronically or on paper) or interviewer-administered (face-to-face). Participants anonymously completed the instrument and then completed an interview within 3 weeks.

      Data Collection

      We used 2 main cognitive interview techniques: think-aloud interviewing and probing.
      • Collins D.
      Pretesting survey instruments: an overview of cognitive methods.
      A researcher trained in cognitive interviewing (J.L.R) asked the participants to “think aloud” as they completed the instrument, whereby the caregiver read each question out loud and described the thought process used to get to their answer. Next, the interviewer used the probing method with a semi-structured question guide to assess participants’ comprehension of each question, response, retrieval, judgment, and usability. Interviews were conducted in-person, by videoconference, or by phone. Participants received a $50 gift card. Interviews were audio recorded, transcribed, and reviewed for accuracy by the interviewer. The interviewer maintained field notes and prepared a written summary of each interview, including potential problems identified during the interview. Summaries were combined into a report that collated the identified problems under each individual item.

      Analysis

      Data were analyzed concurrently with data collection, using deductive and inductive strategies. We used an initial codebook of a priori codes developed from the question-and-answer model from cognitive psychology.
      • Tourangeau R.
      Cognitive sciences and survey methods.
      Three researchers with qualitative expertise (J.L.R, S.L.P., and H.M.Y.) reviewed completed instruments and transcriptions. We began by independently memo-writing and coding the first 3 transcripts with the a priori codes. We also identified emergent codes. Our team met to compare codes and discuss discrepancies to ensure consensus on application of codes, refine codes dimensions, add new codes, and develop tentative categories. We reviewed and discussed the report of written summaries, deleting, adding, or modifying items to improve face validity, understandability, and usability. The revised instrument and codebook were used for subsequent interviews and transcripts. We revisited earlier transcripts as new codes were identified. This process was repeated for every 3 to 5 transcripts until no further areas for instrument improvement were identified.

      Field Testing

      Participants and Data Collection

      Eligible caregivers, using the same eligibility criteria, completed the ED-FACCE instrument. One caregiver per patient was eligible to complete the instrument within 30 days of the ED visit. Each participant received a $25 gift card. We documented the number of participants invited to take the survey to monitor the proportion of respondents among all approached individuals. Participants were additionally invited via flyer advertisements.
      To examine similarity between our family-centeredness measure and a theoretically related outcome, we used 15 questions from the ED CAHPS Survey
      • Weinick RM
      • Becker K
      • Parast L
      • et al.
      Emergency department patient experience of care survey: development and field test.
      that solicit information on going to the ED, family-provider communication, and overall experience. Questions about being discharged home and nurse-specific questions were omitted. Demographic questions addressed participants’ age, gender, race, ethnicity, education, child's age, child's ED use, relationship to the child, amount of time spent at the ED bedside, and child's ED disposition.

      Item and Scale Scoring

      We assigned a numeric value to each item response to develop item raw scores (ie, No = 0, Yes, somewhat = 1, Yes, definitely = 2 for the 9 positively worded items and the reverse for the 2 negative-worded items). These item raw scores were used in item analyses, but all subsequent analyses were conducted with “top-box” scoring (ie “1” for most favorable response, “0” otherwise). We used top-box scoring to highlight the fact that family-centered care should result in caregivers reporting definitively positive experiences rather than experiences that are somewhat positive.
      • Toomey SL
      • Elliott MN
      • Zaslavsky AM
      • et al.
      Variation in family experience of pediatric inpatient care as measured by child HCAHPS.
      CAHPS research has also demonstrated that top-box scoring, compared to linear means, has higher hospital-level reliability
      • Toomey SL
      • Zaslavsky AM
      • Elliott MN
      • et al.
      The development of a pediatric inpatient experience of care measure: child HCAHPS.
      and greater patient and family understandability.
      • Giordano LA
      • Elliott MN
      • Goldstein E
      • et al.
      Development, implementation, and public reporting of the HCAHPS survey.
      We examined response patterns across both the raw scores and the top-box binary indicators to identify inconsistent responses and item missingness. Summary tox-box scores for each respondent were obtained by computing the proportion of top-box scores among the answered items and multiplying it by 100. We used a linear regression model to examine whether mean summary scores varied by mode of administration (in person, paper, electronic).

      Reliability

      Internal consistency of the scale was assessed using coefficients alpha (tau-equivalent reliability) and omega (congeneric reliability).
      • McNeish D
      Thanks coefficient alpha, we’ll take it from here.
      ,
      • McDonald RP.
      Test Theory: A Unified Treatment.

      Item Selection and Assessment of Scale Dimensionality

      After removing items with low variability and/or internal consistency, we assessed dimensionality using item response theory modeling, including models with item difficulty/location parameters (Rasch model), item discrimination (2-parameter logistic or 2PL model), and multiple dimensions/factors. Relative model fit was evaluated using Akaike and Bayes Information Criteria (AIC and BIC, respectively). We also considered root mean square error of approximation (RMSEA) as an absolute measure of model fit. Models were fit using the mirt R package.
      • Chalmers RP
      mirt: a multidimensional item response theory package for the R environment.
      ,
      R Core Team
      R: A Language and Environment for Statistical Computing.
      The aim of the dimensionality analysis was to identify the best fitting model, to summarize the underlying structure of the instrument and provide evidence of construct validity. This aim was balanced against finding a scoring process that would be interpretable operationally, where parsimony is preferred. We compared factor scores with simple summary scores. Strong positive correlations would justify the use of summary scores in reporting.

      Convergent Validity

      We estimated the partial correlation between the ED-FACCE summary top-box score and the 15-item ED CAHPS summary top-box score, adjusting for family caregiver demographics, relying on the Fisher Z transformation to calculate 95% confidence interval (CI).
      • Hotelling H.
      New light on the correlation coefficient and its transforms.

      Results

      Content Expert Assessment

      Content Validity

      The 17-item preliminary instrument had a reading grade level of 6.7. Expert assessment I-CVI for each item ranged from 0.44 to 1. The S-CVI/Ave was 0.83. No experts identified missing domains. Three experts each suggested adding 1 item, but no suggestion was made by multiple experts. We judged the suggested items to be duplicative of existing items and therefore did not incorporate them, but we adopted minor working suggestions.
      Four items were removed based on perceptions by experts that they were confusing, not relevant, or duplicative. These items included 3 of the 4 items with an I-CVI below the 0.778 threshold plus 1 item with an I-CVI 0.778. The 1 remaining item rated with low relevance (I-CVI 0.667) was revised. Following these changes, the resulting instrument had 13 items and a reading grade level of 7.0.

      Pretesting: Cognitive Interviews

      Sixteen interviews were conducted between August and November 2020: 9 in person, 2 by telephone, and 5 by videoconference. Ten caregivers completed a self-administered instrument; 6 caregivers completed an interviewer-administered instrument. Twelve of the participants were female; 4 were male. Participant ages ranged from 23 to 53 years (mean = 37.4). Ten participants were White; 2 were Black; 2 were Asian; and 2 described their race as “other.” Four participants were Hispanic or Latinx. Highest education attained among the participants ranged from less than high school to more than a 4-year degree; the modal category (n = 5) was having a high school degree or equivalent credential. Four of the children had a chronic medical condition. Seven of the children were discharged home from the ED; 4 were admitted locally; and 5 were transferred to another hospital.
      The 13-item instrument was revised in 4 rounds. Four categories of instrument problems were identified as presented in Appendix 1. In brief, participants expressed concerns regarding the phrasing of the questions, duplicative constructs, unclear terms, and a missing construct. As a result of the cognitive testing, 10 items were revised, 3 were deleted, and 1 novel item was added. Following these revisions, no further problems with the instrument were identified by participants. The reading grade level was 6.9.

      Field Testing

      Field testing of the 11-item instrument (Appendix 2) was conducted from March to August 2021. Among the participants directly invited to take the ED-FACCE survey, the response rate was 92.7% (190 of 205). One participant completed the survey via flyer advertisement. Among the 191 respondents, the mode of administration was 54 (28.3%) in person, 60 (31.4%) paper, and 77 (40.3%) electronic. Table 1 shows respondent characteristics. Most participants (70%) were mothers of the child; 23% were fathers; 6% were grandparents; and 2% were other types of guardians. Compared to the overall representation of California residents,

      United States Census Bureau. Quick facts California. Available at https://www.census.gov/quickfacts/CA. 2021. Accessed June 3, 2022.

      the participants in this study had a greater proportion of individuals reporting Black or African American race, a greater proportion of Native Hawaiian or Other Pacific Islander race, a smaller proportion of Hispanic or Latinx ethnicity, a greater proportion of high school graduates or higher, and a smaller proportion of those with a bachelor's degree or higher. Most participants (88%) completed the survey about their child's ED visit in an urban level I trauma center with a 121-bed quaternary care children's hospital. This teaching hospital is a referral center for children across a 33-county region covering 65,000 square miles. It receives pediatric transfers from ∼30 hospitals in the region; 12% of participants completed the survey about a visit in 1 of those referring ED sites.
      Table 1Field Testing Participant Characteristics
      N (%)
      About the Participant
      Age, in years
       ≤2957 (29.8)
       30–3988 (46.1)
       40–4931 (16.2)
       50+15 (7.8)
      Gender
       Female143 (74.9)
       Male47 (24.6)
       Nonbinary1 (0.5)
      Race
       White100 (52.6)
       Black or African American23 (12.1)
       Asian20 (10.5)
       Native Hawaiian or other Pacific Islander4 (2.1)
       American Indian or Alaska Native3 (1.6)
       Other40 (21.0)
      Ethnicity
       Hispanic or Latinx50 (26.2)
       Not Hispanic or Latinx141 (73.8)
      Highest education attained
       Some high school, but did not graduate5 (2.6)
       High school graduate or GED106 (56.1)
       Some college or 2-year degree26 (13.8)
       4-year college graduate38 (20.1)
       More than 4-year college degree14 (7.4)
      About the participant's child
      Child's age, in years
       <127 (14.1)
       1–592 (48.2)
       6–1252 (27.2)
       13–1820 (10.5)
      Relationship to the child
       Mother134 (70.2)
       Father43 (22.5)
       Grandmother9 (4.7)
       Grandfather2 (1.0)
       Other3 (1.6)
      Number of ED visits for the child in last 6 months
       1 time145 (75.9)
       2 times31 (16.2)
       3 times13 (6.8)
       4 times2 (1.0)
       5 or more times0 (0.0)
      About the emergency department visit
      Main reason for the child's ED visit
       An accident of injury114 (59.7)
       A new health problem53 (27.8)
       An ongoing health condition or concern24 (12.6)
      Child arrived to the ED by ambulance
       Yes, by ambulance115 (60.2)
      Amount of time spent at the child's ED bedside
       A little of the time0 (0.0)
       Some of the time3 (1.6)
       Most of the time21 (11.0)
       All or nearly all of the time167 (87.4)
      Child's ED disposition
       Discharged home94 (49.2)
       Transfer to different hospital23 (12.0)
       Admitted for hospitalization74 (38.7)
      GED indicates General Educational Development; ED, emergency department.

      Summary Score

      The mean (standard deviation) ED-FACCE 11-item summary score was 77.2 (26.6); the median (25%–75% interquartile range) was 81.8 (54.5–100). In a linear regression model, the summary score did not vary by the mode of administration (P = .50). There were no missing responses for any item from any respondent.

      Response Patterns and Item Variability

      Four of 191 respondents selected “Yes, definitely” for all 11 items. This pattern suggests inconsistent responding for these participants, at least with the last 2 items, since the first 9 items were positively worded attributes and the last 2 were negatively worded. Figure 2 shows the mean (95% CI) top-box score for each item, which ranged from 61.2% to 95.3%. Two “low-variability” items had top-box scores greater than 95% (items 10 and 11).
      Figure 2
      Figure 2Variability of item top-box scores. Top-box scores calculated as the proportion of respondents answering the most favorable response option among the number of respondents answering the item.

      Reliability

      Coefficients alpha and omega for the ED-FACCE scale (raw scores, all 11 items) were both 0.85. Items 10 and 11 demonstrated corrected item-total correlations (a measure of item discrimination) of 0.08 and 0.14, respectively. Restricting the scale to items 1 through 9, the correct item-total correlations ranged from 0.46 to 0.79, with 7 items above 0.60. Deleting any of these 9 items would not have improved reliability; however, deleting either item 10 or 11 would increase alpha and omega to 0.86. In support of top-box scoring, coefficients alpha and omega for the 9-item subset using the original raw scoring were 0.88 and 0.89, respectively. For top-box scoring, these values were both 0.90.

      Item Selection

      We removed items 10 and 11 given their poor variability and adverse effects on internal consistency reliability. Removing these 2 items also addressed the unbalanced inclusion of 2 negatively worded items in a scale of 11 total items. Items 1 through 9 were used as a scale for the subsequent analyses. Table 2 presents the text of the final ED-FACCE survey items. Table 3 contains item analysis results.
      Table 2Final Survey Instructions and Items
      Mark one answer for each question. Answer the questions about your child's recent EMERGENCY DEPARTMENT visit. If your child recently visited more than one emergency department, answer the questions about only one of those visits. Do not include any other visits in your answers.
      During your child's recent EMERGENCY DEPARTMENT visit…
      Item 1Did your child's doctors seem interested in your worries about your child's problem?
      Item 2Did you feel comfortable sharing your ideas with your child's doctors?
      Item 3Did your child's doctors focus on what matters to you the most?
      Item 4Did your child's doctors clearly explain to you their ideas about your child's medical care?
      Item 5Did your child's doctors give you information in ways that you understood?
      Item 6Did your child's doctors encourage you to ask questions?
      Item 7Did your child's doctors involve you in making decisions about your child's medical care?
      Item 8Did your child's doctors propose next steps for your child's medical care that were acceptable to you?
      Item 9Did your child's doctors address your child's needs?
      Response options for each item: Yes, definitely; Yes, somewhat; No. The contents of the instrument are distributed under a Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/).
      Table 3Item Analysis and Model Results
      Factors Labels and ItemsMSDAIDF1F2A1A2B
      Communication/interaction
      1Interested in your worries0.730.440.880.89-0.012.992.13
      2Comfortable sharing your ideas0.610.490.870.950.037.032.30
      3Focus on what matters0.660.480.890.85-0.022.521.26
      4Clearly explain their ideas0.700.460.880.770.153.351.94
      5Give information you understood0.640.480.870.800.218.753.43
      6Encourage you to ask questions0.700.460.880.780.173.842.18
      Decision-making/delivery of medical care
      7Involve you in making decisions0.790.410.880.240.745.244.27
      8Propose next steps for care0.850.360.890.070.904.735.18
      9Address your child's needs0.910.290.900.000.883.034.64
      SD indicates standard deviation.
      M – proportion of respondents selecting the highest rating category “yes, definitely.”
      AID – alpha-if-item-deleted; what coefficient alpha would be for the scale if the item were deleted.
      F1 and F2 – exploratory 2PL IRT loadings on factors 1 and 2.
      A1 and A2 – discrimination parameters from the final confirmatory multidimensional 2PL model.
      B – item location parameter.

      Item Response Theory

      An exploratory 2PL model (all items free to load on 2 factors, promax rotation) produced factors correlating at 0.77 and sums of squared loadings of 4.32 and 2.22. A trend was evident in the factor loadings, with the first 6 items loading more on 1 factor and the last 3 items loading on the other (Table 3). A multidimensional confirmatory 2PL model (AIC 1285, BIC 1346) included loadings as specified in the exploratory model, and a bifactor model (AIC 1293, BIC 1381) included a general factor and 2 specific factors, again following the structure from above. The multidimensional 2PL produced the best fit, with lower AIC and BIC than any other model, including unidimensional Rasch (AIC 1316, BIC 1348) and 2PL (AIC 1300, BIC 1359) models. All models had RMSEA below 0.05. Item parameters for the final multidimensional 2PL model are given in Table 3. Factor scores from the multidimensional 2PL correlated at 0.98 and 0.96 with summary scores.

      Convergent Validity

      The mean (standard deviation) ED CAHPS summary score for each respondent was 68.3 (25.3); median (25%–75% interquartile range) was 70 (50–90). Adjusted partial correlation between the 9-item ED-FACCE summary core and the 15-item ED CAHPS summary score was 0.75 (95% CI 0.67–0.81).

      Discussion

      This study developed and evaluated the ED-FACCE survey, which measures family-centered care provided by physicians during pediatric ED encounters. The multiphased development and evaluation process resulted in a survey with sufficient evidence supporting its reliability and validity for research and quality improvement purposes. The usability of the instrument is supported by positive feedback from cognitive interviews as well as the lack of missing values during field testing. Psychometric analyses demonstrated good item performance, reliability, and convergent validity of a 9-item scale.
      Content validity is deeply connected to the definition of the measured construct. Unfortunately, no consensus definition exists for family-centered care.
      • Kuo DZ
      • Houtrow AJ
      • Arango P
      • et al.
      Family-centered care: current applications and future directions in pediatric health care.
      ,
      Committee on Hospital Care and Institute for Patient- and Family-Centered Care
      Patient-and family-centered care and the pediatrician's role.
      ,
      • Kokorelias KM
      • Gignac MA
      • Naglie G
      • et al.
      Towards a universal model of family centered care: a scoping review.
      ,
      • Lor M
      • Crooks N
      • Tluczek A.
      A proposed model of person-, family-, and culture-centered nursing care.
      We began each cognitive interview by reading a family-centered care definition that used the 4 attributes described in the concept analysis by Coyne et al.
      • Coyne I
      • Holmström I
      • Söderbäck M.
      Centeredness in healthcare: a concept synthesis of family-centered care, person-centered care and child-centered care.
      This definition stated that the core concepts of family-centered care include: 1) Listening to and Respecting Experiences and Preferences, 2) Sharing Information, 3) Providing Support, and 4) Collaborating and Encouraging Participation. Should others seek to measure a differently defined construct, our present instrument might not be applicable. Of note, although the adjusted partial correlation between our scale's 9-item summary score and the ED CAHPS summary score supports convergent validity, it also suggests that these 2 scales give different results.
      Family-centered care has been the gold standard approach to pediatric care delivery for decades.
      • Jolley J
      • Shields L.
      The evolution of family-centered care.
      Although less recognized, there is growing emphasis on an alternative concept: child-centered care.
      • Coyne I
      • Holmström I
      • Söderbäck M.
      Centeredness in healthcare: a concept synthesis of family-centered care, person-centered care and child-centered care.
      ,
      • Carter B
      • Ford K.
      Researching children's health experiences: the place for participatory, child-centered, arts-based approaches.
      The foundation of child-centered care is that the child's perspective is heard and valued.
      • Coyne I
      • Holmström I
      • Söderbäck M.
      Centeredness in healthcare: a concept synthesis of family-centered care, person-centered care and child-centered care.
      Future iterations of the ED-FACCE tool can incorporate child-centered care attributes, while acknowledging that a construct that values the child's perspective and participation in decision-making would lack applicability for neonatal patients.
      Future research should include testing versions of the ED-FACCE survey in languages other than English. Limited English-proficiency parents are more likely to report low satisfaction, poor comprehension, and non-family-centered care.
      • Seltz LB
      • Zimmer L
      • Ochoa-Nunez L
      • et al.
      Latino families’ experiences with family-centered rounds at an academic children's hospital.
      ,
      • Eneriz-Wiemer M
      • Sanders LM
      • Barr DA
      • et al.
      Parental limited English proficiency and health outcomes for children with special health care needs: a systematic review.
      Their children are twice as likely to experience adverse events while hospitalized.
      • Khan A
      • Yin HS
      • Brach C
      • et al.
      Association between parent comfort with English and adverse events among hospitalized children.
      A language appropriate ED-FACCE survey is needed for research and quality improvement efforts targeting family-centered care for limited English-proficiency families.
      Item response theory analyses identified that the ED-FACCE survey consists of 2 factors, which we labeled as 1) quality of communication and 2) clinical decision-making. Consistent with existing literature, these 2 topics encompass various attributes used to describe family-centered care.
      • Kuo DZ
      • Houtrow AJ
      • Arango P
      • et al.
      Family-centered care: current applications and future directions in pediatric health care.
      ,
      Committee on Hospital Care and Institute for Patient- and Family-Centered Care
      Patient-and family-centered care and the pediatrician's role.
      ,
      • Kokorelias KM
      • Gignac MA
      • Naglie G
      • et al.
      Towards a universal model of family centered care: a scoping review.
      ,
      • Lor M
      • Crooks N
      • Tluczek A.
      A proposed model of person-, family-, and culture-centered nursing care.
      These 2 dimensions also align with the 4 domains of the initial instrument draft. Future confirmatory factor analyses are needed and can begin by considering the multidimensional model. Although the multidimensional 2PL model provided the best fit, factor scores correlated strongly with summary scores, so we recommend the simpler summary scores for routine use.
      This study has several limitations, to be addressed in future studies. The ED-FACCE survey was designed to measure physicians’ family-centeredness of care. It does not assess care delivered by other provider groups (eg, nurses, nurse practitioners, physician assistants) or providers more broadly. Future research should adapt and evaluate the ED-FACCE survey for other types of providers, such as nurses. While the instrument measures family-centered care provided by physicians, respondents might not have known which ED providers were physicians. Additionally, the care provided by other health care professionals might have influenced their reporting about their child's physicians. Furthermore, our field testing participants largely responded about their experiences in the same ED, which was an urban level I trauma center and teaching hospital. Most field testing participants also reported being in the ED for a new injury or health problem; few respondents accompanied a child presenting to the ED for an ongoing health condition or concern. While the sample of field testing participants was not representative of California residents, this study included a diverse sample. Additional psychometric analyses are also needed of administration mode effects, which were only explored here at the scale level, as well as measurement invariance across participant subgroups, including for individual items. Finally, recruitment and testing occurred during the coronavirus disease 2019 pandemic; participants’ varied experiences with the pandemic may have impacted their responses. Given the targeted nature of the questions, we do not expect the impact to be significant. Finally, we did not readminister the final 9 items on their own, and the presence versus absence of items 10 and 11 could potentially influence responses to items 1 through 9.
      Despite these limitations, this study provides validity evidence supporting use of the ED-FACCE to assess family-centeredness of care for pediatric ED encounters. Future research should explore additional psychometric analyses, including with larger and more diverse samples. Future research should also test alternative iterations of the ED-FACCE survey, such as versions in additional languages and surveys measuring family-centered care provided by other provider groups and in other settings.

      Acknowledgments

      Financial statement: This work was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development , National Institutes of Health ( K23HD101550 to Dr. Rosenthal). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

      References

        • Coyne I
        • Holmström I
        • Söderbäck M.
        Centeredness in healthcare: a concept synthesis of family-centered care, person-centered care and child-centered care.
        J Pediatr Nurs. 2018; 42: 45-56
        • Dudley N
        • Ackerman A
        • Brown KM
        • et al.
        Patient-and family-centered care of children in the emergency department.
        Pediatrics. 2015; 135: e255-e272
        • Institute of Medicine
        Crossing the Quality Chasm: A New Healthcare for the 21st Century.
        U.S. Department of Health and Human Services, National Academy of Sciences, Washington, DC2001
        • Kuo DZ
        • Houtrow AJ
        • Arango P
        • et al.
        Family-centered care: current applications and future directions in pediatric health care.
        Matern Child Health J. 2012; 16: 297-305
        • Goldfarb MJ
        • Bibas L
        • Bartlett V
        • et al.
        Outcomes of patient-and family-centered care interventions in the ICU: a systematic review and meta-analysis.
        Crit Care Med. 2017; 45: 1751-1761
        • Committee on Hospital Care and Institute for Patient- and Family-Centered Care
        Patient-and family-centered care and the pediatrician's role.
        Pediatrics. 2012; 129: 394
        • Mirlashari J
        • Brown H
        • Fomani FK
        • et al.
        The challenges of implementing family-centered care in NICU from the perspectives of physicians and nurses.
        J Pediatr Nurs. 2020; 50: e91-e98
        • Oude Maatman SM
        • Bohlin K
        • Lilliesköld S
        • et al.
        Factors influencing implementation of family-centered care in a neonatal intensive care unit.
        Front Pediatr. 2020; 8: 222
      1. Agency for Healthcare Research and Quality. CAHPS patient experience surveys and guidance. Available at: https://www.ahrq.gov/cahps/surveys-guidance/index.html. 2021. Accessed December 9, 2021.

      2. National Quality Forum Leadership Consortium. National quality forum leadership consortium 2022 priorities for action. Available at:https://www.qualityforum.org/Publications/2021/12/2022_NQF_Priorities_for_Action.aspx. 2021. Accessed March 2, 2022.

        • Lindly OJ
        • Geldhof GJ
        • Acock AC
        • et al.
        Family-centered care measurement and associations with unmet health care need among US children.
        Acad Pediatr. 2017; 17: 656-664
        • Etz RS
        • Zyzanski SJ
        • Gonzalez MM
        • et al.
        A new comprehensive measure of high-value aspects of primary care.
        Ann Fam Med. 2019; 17: 221-230
        • King S
        • King G
        • Rosenbaum P.
        Evaluating health service delivery to children with chronic conditions and their families: development of a refined measure of processes of care (MPOC-20).
        Children Health Care. 2004; 33: 35-57
        • Little P
        • Everitt H
        • Williamson I
        • et al.
        Observational study of effect of patient centredness and positive approach on outcomes of general practice consultations.
        BMJ. 2001; 323: 908-911
        • Frost MH
        • Reeve BB
        • Liepa AM
        • et al.
        What is sufficient evidence for the reliability and validity of patient-reported outcome measures?.
        Value Health. 2007; 10: S94-S105
        • Hudon C
        • Fortin M
        • Haggerty JL
        • et al.
        Measuring patients’ perceptions of patient-centered care: a systematic review of tools for family medicine.
        Ann Fam Med. 2011; 9: 155-164
        • Stewart M
        • Brown JB
        • Donner A
        • et al.
        The impact of patient-centered care on outcomes.
        Fam Pract. 2000; 49: 796-804
        • Rosenthal JL
        • Li S-TT
        • Hernandez L
        • et al.
        Familial caregiver and physician perceptions of the family-physician interactions during interfacility transfers.
        Hosp Pediatr. 2017; 7: 344-351
        • Byczkowski TL
        • Gillespie GL
        • Kennebeck SS
        • et al.
        Family-centered pediatric emergency care: a framework for measuring what parents want and value.
        Acad Pediatr. 2016; 16: 327-335
        • Klassen A
        • Dix D
        • Cano S
        • et al.
        Evaluating family-centred service in paediatric oncology with the measure of processes of care (MPOC-20).
        Child Care Health Dev. 2009; 35: 16-22
        • Bradburn NM
        • Sudman S
        • Wansink B.
        Asking Questions: The Definitive Guide to Questionnaire Design—For Market Research, Political Polls, and Social and Health Questionnaires.
        John Wiley & Sons, 2004
        • Davis LL.
        Instrument review: getting the most from a panel of experts.
        Appl Nurs Res. 1992; 5: 194-197
        • Lynn MR.
        Determination and quantification of content validity.
        Nurs Res. 1986; 35: 382-385
        • Polit DF
        • Beck CT.
        The content validity index: are you sure you know what's being reported? Critique and recommendations.
        Res Nurs Health. 2006; 29: 489-497
        • Collins D.
        Pretesting survey instruments: an overview of cognitive methods.
        Qual Life Res. 2003; 12: 229-238
        • Tourangeau R.
        Cognitive sciences and survey methods.
        Cognitive Aspects of Survey Methodology: Building a Bridge Between Disciplines. National Academy Press, Washington, DC1984
        • Weinick RM
        • Becker K
        • Parast L
        • et al.
        Emergency department patient experience of care survey: development and field test.
        Rand Health Q. 2014; 45
        • Toomey SL
        • Elliott MN
        • Zaslavsky AM
        • et al.
        Variation in family experience of pediatric inpatient care as measured by child HCAHPS.
        Pediatrics. 2017; 139e20163372https://doi.org/10.1542/peds.2016-3372
        • Toomey SL
        • Zaslavsky AM
        • Elliott MN
        • et al.
        The development of a pediatric inpatient experience of care measure: child HCAHPS.
        Pediatrics. 2015; 136: 360-369
        • Giordano LA
        • Elliott MN
        • Goldstein E
        • et al.
        Development, implementation, and public reporting of the HCAHPS survey.
        Med Care Res Rev. 2010; 67: 27-37
        • McNeish D
        Thanks coefficient alpha, we’ll take it from here.
        Psychol Methods. 2018; 23: 412-433
        • McDonald RP.
        Test Theory: A Unified Treatment.
        Lawrence Erlbaum Associates, Inc, New York, NY1999
        • Chalmers RP
        mirt: a multidimensional item response theory package for the R environment.
        J Stat Softw. 2012; 48: 1-29
        • R Core Team
        R: A Language and Environment for Statistical Computing.
        R Foundation for Statistical Computing, Vienna, Austria2017 (Available at:) (Accessed June 16, 2022)
        • Hotelling H.
        New light on the correlation coefficient and its transforms.
        J R Stat Soc Series B (Methodological). 1953; 15: 193-232
      3. United States Census Bureau. Quick facts California. Available at https://www.census.gov/quickfacts/CA. 2021. Accessed June 3, 2022.

        • Kokorelias KM
        • Gignac MA
        • Naglie G
        • et al.
        Towards a universal model of family centered care: a scoping review.
        BMC Health Serv Res. 2019; 19: 1-11
        • Lor M
        • Crooks N
        • Tluczek A.
        A proposed model of person-, family-, and culture-centered nursing care.
        Nurs Outlook. 2016; 64: 352-366
        • Jolley J
        • Shields L.
        The evolution of family-centered care.
        J Pediatr Nurs. 2009; 24: 164-170
        • Carter B
        • Ford K.
        Researching children's health experiences: the place for participatory, child-centered, arts-based approaches.
        Res Nurs Health. 2013; 36: 95-107
        • Seltz LB
        • Zimmer L
        • Ochoa-Nunez L
        • et al.
        Latino families’ experiences with family-centered rounds at an academic children's hospital.
        Acad Pediatr. 2011; 11: 432-438
        • Eneriz-Wiemer M
        • Sanders LM
        • Barr DA
        • et al.
        Parental limited English proficiency and health outcomes for children with special health care needs: a systematic review.
        Acad Pediatr. 2014; 14: 128-136
        • Khan A
        • Yin HS
        • Brach C
        • et al.
        Association between parent comfort with English and adverse events among hospitalized children.
        JAMA Pediatr. 2020; 174e203215