Advertisement

The Impact of Telehealth Adoption During COVID-19 Pandemic on Patterns of Pediatric Subspecialty Care Utilization

  • Eli M. Cahan
    Correspondence
    Address correspondence to Eli M. Cahan, MS, Department of Pediatrics, Stanford School of Medicine, 770 Welch Road, #100, Stanford, CA 94305
    Affiliations
    Department of Pediatrics (EM Cahan, J Maturi, P Bailey, S Fernandes, A Addala, S Kibrom, JR Krissberg, SM Smith, S Shah, E Wang, PH Wise, and LJ Chamberlain), Stanford School of Medicine, Stanford, Calif

    New York University School of Medicine (EM Cahan), New York, NY
    Search for articles by this author
  • Jay Maturi
    Affiliations
    Department of Pediatrics (EM Cahan, J Maturi, P Bailey, S Fernandes, A Addala, S Kibrom, JR Krissberg, SM Smith, S Shah, E Wang, PH Wise, and LJ Chamberlain), Stanford School of Medicine, Stanford, Calif
    Search for articles by this author
  • Paige Bailey
    Affiliations
    Department of Pediatrics (EM Cahan, J Maturi, P Bailey, S Fernandes, A Addala, S Kibrom, JR Krissberg, SM Smith, S Shah, E Wang, PH Wise, and LJ Chamberlain), Stanford School of Medicine, Stanford, Calif
    Search for articles by this author
  • Susan Fernandes
    Affiliations
    Department of Pediatrics (EM Cahan, J Maturi, P Bailey, S Fernandes, A Addala, S Kibrom, JR Krissberg, SM Smith, S Shah, E Wang, PH Wise, and LJ Chamberlain), Stanford School of Medicine, Stanford, Calif

    Department of Medicine (S Fernandes), Stanford School of Medicine, Stanford, Calif
    Search for articles by this author
  • Ananta Addala
    Affiliations
    Department of Pediatrics (EM Cahan, J Maturi, P Bailey, S Fernandes, A Addala, S Kibrom, JR Krissberg, SM Smith, S Shah, E Wang, PH Wise, and LJ Chamberlain), Stanford School of Medicine, Stanford, Calif
    Search for articles by this author
  • Sara Kibrom
    Affiliations
    Department of Pediatrics (EM Cahan, J Maturi, P Bailey, S Fernandes, A Addala, S Kibrom, JR Krissberg, SM Smith, S Shah, E Wang, PH Wise, and LJ Chamberlain), Stanford School of Medicine, Stanford, Calif
    Search for articles by this author
  • Jill R. Krissberg
    Affiliations
    Department of Pediatrics (EM Cahan, J Maturi, P Bailey, S Fernandes, A Addala, S Kibrom, JR Krissberg, SM Smith, S Shah, E Wang, PH Wise, and LJ Chamberlain), Stanford School of Medicine, Stanford, Calif
    Search for articles by this author
  • Stephanie M. Smith
    Affiliations
    Department of Pediatrics (EM Cahan, J Maturi, P Bailey, S Fernandes, A Addala, S Kibrom, JR Krissberg, SM Smith, S Shah, E Wang, PH Wise, and LJ Chamberlain), Stanford School of Medicine, Stanford, Calif
    Search for articles by this author
  • Sejal Shah
    Affiliations
    Department of Pediatrics (EM Cahan, J Maturi, P Bailey, S Fernandes, A Addala, S Kibrom, JR Krissberg, SM Smith, S Shah, E Wang, PH Wise, and LJ Chamberlain), Stanford School of Medicine, Stanford, Calif
    Search for articles by this author
  • Ewen Wang
    Affiliations
    Department of Pediatrics (EM Cahan, J Maturi, P Bailey, S Fernandes, A Addala, S Kibrom, JR Krissberg, SM Smith, S Shah, E Wang, PH Wise, and LJ Chamberlain), Stanford School of Medicine, Stanford, Calif

    Center for Policy, Outcomes, and Prevention (E Wang, O Saynina, PH Wise, and LJ Chamberlain), Stanford School of Medicine, Stanford, Calif

    Department of Emergency Medicine (E Wang), Stanford School of Medicine, Stanford, Calif
    Search for articles by this author
  • Olga Saynina
    Affiliations
    Center for Policy, Outcomes, and Prevention (E Wang, O Saynina, PH Wise, and LJ Chamberlain), Stanford School of Medicine, Stanford, Calif
    Search for articles by this author
  • Author Footnotes
    # Contributed equally as co-senior authors.
    Paul H. Wise
    Footnotes
    # Contributed equally as co-senior authors.
    Affiliations
    Department of Pediatrics (EM Cahan, J Maturi, P Bailey, S Fernandes, A Addala, S Kibrom, JR Krissberg, SM Smith, S Shah, E Wang, PH Wise, and LJ Chamberlain), Stanford School of Medicine, Stanford, Calif

    Center for Policy, Outcomes, and Prevention (E Wang, O Saynina, PH Wise, and LJ Chamberlain), Stanford School of Medicine, Stanford, Calif
    Search for articles by this author
  • Author Footnotes
    # Contributed equally as co-senior authors.
    Lisa J. Chamberlain
    Footnotes
    # Contributed equally as co-senior authors.
    Affiliations
    Department of Pediatrics (EM Cahan, J Maturi, P Bailey, S Fernandes, A Addala, S Kibrom, JR Krissberg, SM Smith, S Shah, E Wang, PH Wise, and LJ Chamberlain), Stanford School of Medicine, Stanford, Calif

    Center for Policy, Outcomes, and Prevention (E Wang, O Saynina, PH Wise, and LJ Chamberlain), Stanford School of Medicine, Stanford, Calif
    Search for articles by this author
  • Author Footnotes
    # Contributed equally as co-senior authors.
Published:March 19, 2022DOI:https://doi.org/10.1016/j.acap.2022.03.010

      Abstract

      Objective

      The COVID-19 pandemic prompted health systems to rapidly adopt telehealth for clinical care. We examined the impact of demography, subspecialty characteristics, and broadband availability on the utilization of telehealth in pediatric populations before and after the early period of the COVID-19 pandemic.

      Methods

      Outpatients scheduled for subspecialty visits at sites affiliated with a single quaternary academic medical center between March–June 2019 and March–June 2020 were included. The contribution of demographic, socioeconomic, and broadband availability to visit completion and telehealth utilization were examined in multivariable regression analyses.

      Results

      Among visits scheduled in 2020 compared to 2019, in-person visits fell from 23,318 to 11,209, while telehealth visits increased from 150 to 7,675. Visits among established patients fell by 15% and new patients by 36% (P < .0001). Multivariable analysis revealed that completed visits were reduced for Hispanic patients and those with reduced broadband; high income, private non-HMO insurance, and those requesting an interpreter were more likely to complete visits. Those with visits scheduled in 2020, established patients, those with reduced broadband, and patients older than 1 year were more likely to complete TH appointments. Cardiology, oncology, and pulmonology patients were less likely to complete scheduled TH appointments.

      Conclusions

      Following COVID-19 onset, outpatient pediatric subspecialty visits shifted rapidly to telehealth. However, the impact of this shift on social disparities in outpatient utilization was mixed with variation among subspecialties. A growing reliance on telehealth will necessitate insights from other healthcare settings serving populations of diverse social and technological character.

      Keywords

      What's New
      Telehealth's rapid adoption amid the pandemic impacted the utilization of pediatric subspecialty care, potentially worsening pre-existing social disparities. In this analysis from one quaternary children's hospital, the shift was associated with complex effects on outpatient care among diverse social groups.
      The COVID-19 pandemic has profoundly disrupted the utilization of health systems. The extent to which these disruptions are exacerbating or mitigating social disparities in health care utilization and outcomes requires continued examination. Pioneering studies of adult systems found marked reductions in office visits during the pandemic's early months. The decline in visits has been accompanied by a dramatic increase in the use of telehealth, supported in part by an array of communication technologies, including video, telephone, email, and text messaging.
      • Whaley CM
      • Pera MF
      • Cantor J
      • et al.
      Changes in health services use among commercially insured US populations during the COVID-19 pandemic.
      However, disparities in telehealth use could potentially be substantial, particularly among older, non-English speaking, publicly insured, lower income, and patients from specific racial/ethnic groups.
      • Ye S
      • Kronish I
      • Fleck E
      • et al.
      Telemedicine expansion during the COVID-19 pandemic and the potential for technology-driven disparities.
      Recently, the American Academy of Pediatrics emphasized the growing importance of telehealth services and the need to monitor telehealth's impact on social patterns of healthcare utilization.
      • Curfman AL
      • Hackell JM
      • Herendeen NE
      • et al.
      Telehealth: improving access to and quality of pediatric health care.
      The early phase of the pandemic was associated with reduced in-person pediatric visits.
      • Brown CL
      • Montez K
      • Amati JB
      • et al.
      Impact of COVID-19 on pediatric primary care visits at four academic institutions in the Carolinas.
      Accordingly, telehealth strategies could generate new opportunities to overcome, or exacerbate, longstanding obstacles to care provision for medically and socially marginalized populations.
      • Veinot TC
      • Mitchell H
      • Ancker JS.
      Good intentions are not enough: how informatics interventions can worsen inequality.
      • Nguyen OT
      • Watson AK
      • Motwani K
      • et al.
      Patient-level factors associated with utilization of telemedicine services from a free clinic during COVID-19.
      • Pierce RP
      • Stevermer JJ.
      Disparities in use of telehealth at the onset of the COVID-19 public health emergency.
      Both federal and state programs seek to expand infrastructure technologies in support of telehealth. However, it remains unclear whether these investments will advance the care of traditionally underserved groups.
      The White House
      UPDATED FACT SHEET: Bipartisan Infrastructure Investment and Jobs Act.
      United States Senate
      H.R.3684 - Infrastructure Investment and Jobs Act.
      California State Assembly
      AB-34 Broadband for All Act of 2022.
      The literature examining the digital divide for pediatric populations is rapidly expanding. Various components impacting telehealth access include a reliance on cellular networks, broadband availability, broadband data cost, device (hardware) capabilities, technological fluency, and language accessibility.
      • Cahan EM
      • Mittal V
      • Shah NR
      • et al.
      Achieving a quintuple aim for telehealth in pediatrics.
      However, the relative influence of these factors on telehealth access in pediatric populations remains unclear. Also unexplored is if, and how, telehealth visit completion varies by pediatric subspecialty, given different patient populations and practice paradigms.
      • Tilden DR
      • Datye KA
      • Moore DJ
      • et al.
      The rapid transition to telemedicine and its effect on access to care for patients with type 1 diabetes during the COVID-19 pandemic.
      ,
      • Chang JE
      • Lai AY
      • Gupta A
      • et al.
      Rapid transition to telehealth and the digital divide: implications for primary care access and equity in a post-COVID era.
      Therefore, we examined telehealth use and visit completion among patients requiring pediatric subspecialty management before and after the COVID-19 pandemic at a single academic health system. Specifically, we evaluated the social patterns of telehealth use and visit completion in the earliest months of the pandemic. Although there was no facility-wide or divisional prohibition on in-person visits or the acceptance of new patients, we hypothesized that the enhanced reliance on telehealth may have affected access to care for traditionally underserved populations.

      Methods

      Data Sources

      We conducted a retrospective analysis of 6 high-volume subspecialties’ outpatient visits seen at Stanford Children's Health (SCH) between March to June 2019 and March to June 2020. SCH is a pediatric healthcare system in the San Francisco Bay Area providing over 500,000 visits annually. The system is anchored by an academic, quaternary, free-standing children's hospital which serves as a regional care center for children with medical complexity. Data were abstracted from the Stanford Research Repository Tool, or STARR: a resource providing anonymized, aggregated clinical data generated from health system encounters (satellite sites and main campus) via a formalized encryption and extraction process. The 2 compared time periods represent the 4 earliest months of the COVID-19 pandemic and the same 4 month period of the prior year.

      Subjects

      Outpatients scheduled to be seen by pediatric cardiology, nephrology, oncology, neurology, pulmonary and endocrinology between March–June 2019 and March–June 2020 were included in the analyses. These services were chosen to assess visitation patterns among children with potentially serious conditions who would likely be most sensitive to disruptions in care. These specialty services account for the largest portion of outpatient visits. Telehealth visits were defined as those conducted through the patient portal or through third-party teleconferencing software; a negligible number of these were conducted by phone, and the remainder were via video. Each visit was considered an independent event; therefore, individual patients could account for more than one visit. Demographic information included age, gender, interpreter usage (which was available throughout both the 2019 and 2020 periods), and race/ethnicity, which included Other (referring to unlisted race/ethnicity, including multiracial) and Unknown (missing data). Insurance coverage was defined as Public which included Medicaid in California, and California Children's Services (Title V in California); private insurance was divided into HMO (Health Maintenance Organization) or Non-HMO categories. The insurance category of Other included military-affiliated (such as TriCare) and other governmental insurers (such as COVID-19 uninsurance funds). Distance traveled to access care was calculated as the straight-line distance between the patient's zip code and the clinic, and then grouped in ≤50 miles and >50 miles to access care. Approximately three-fourth of all visits fell within the ≤50 miles category which permitted comparisons with the farthest quartile, those in the >50 miles category. Family income was approximated from zip code-level annual data obtained from the 2018 American Community Survey conducted by the US Census Bureau and was grouped into: ≤3 times Federal Poverty Level (FPL), 3-4 times FPL, and >4 times FPL.
      Patient visit was characterized as established or new patient, depending on whether they had a previous visit with the provider. We characterized the appointment status as completed or not completed; a not completed visit includes visits cancelled in advance and “no-shows.” The analysis was concerned with patterns of visits and multiple visits by individual patients during the study period were counted separately.
      The broadband variable was derived from the Federal Communication Commission's (FCC) Fixed Broadband Deployment database. The assessment of broadband internet capability was based on the speed with which the connection can download and upload bits of data. While the FCC considers 25 Megabits per second (Mbps) download speed and 3 Mbps upload speed as meeting its “statutory definition of advanced telecommunications capability,” it is unclear whether this speed is adequate for a modern household.

      Federal Communication Commission. 2020 Broadband Deployment Report. Available at: https://docs.fcc.gov/public/attachments/FCC-20-50A1.pdf. Accessed March 9, 2022.

      In addition, some common video platforms generally require higher speeds and households in which more than one user is simultaneously employing the internet may require closer to 100/10 Mbps capabilities to reliably support high-quality telehealth visits. Therefore, we used the 100/10 Mbps standard to assess the availability of adequate internet broadband capability in any given zip code. Because the reported presence of only 1 high speed internet provider in a zip code may be associated with intermittent service or inadequate infrastructure for all households in the zip code, we used the availability of 2 or more 100/10 Mbps broadband providers as the indicator of adequate internet capability for high-quality video conferencing. This permitted the authors to rank zip codes from highest to lowest levels of broadband accessibility. Each patient was assigned a zip code percentile ranking, with the 100th percentile representing the highest broadband access relative to other patients. Visit completion and the use of telehealth for patients in zip codes with the lowest access to high-speed broadband, those in the lowest 25th percentile, were compared with those in the highest quartile.

      Analysis

      Descriptive statistics (frequencies and percentages) were reported for categorical variables, while means and standard deviations (SDs) or medians and interquartile ranges (IQRs) were reported for continuous variables. Chi-square test analysis was used to assess differences between proportions and a value of P< .05 was deemed statistically significant. Univariable and multivariable regression was used to assess odds ratios and 95% Wald confidence limits for the dependent variables of completed visits and for the study period in 2020, of the use of telehealth. SAS version 9.4 (SAS Inc, Cary, NC) was used. To assess changes over time associated with progression of the pandemic, a month variable was evaluated which revealed no change, thus it was dropped in the final model. To assess collinearity between covariates, variance inflation factors (VIFs) were analyzed. The Stanford University Institutional Review Board approved this study.

      Results

      The sample included 42,352 total visits: 23,468 scheduled to be seen between March-June 2019 and 18,884 scheduled for March–June 2020 (Table 1). The patients were diverse, with Hispanics representing 31% of the study population, followed by non-Hispanic white (27%) and Asians (13%). Approximately one in five visits had a request for interpreter support. Public insurance covered 40% of patients, while private HMO and non-HMO covered 43% and 14% of patients, respectively. Three out of four families resided ≤50 miles of the specialty clinic and 21% lived in areas served by the lowest 25% of broadband capacity. The distribution of patient volume among the subspecialty clinics was similar for both studied periods, with the highest noted for endocrinology (23% of examined population of patients) and lowest for nephrology (5% of examined population of patients). Approximately 80% of those included in the sample were established patients.
      Table 1.Demographic Characteristics of Pediatric Outpatients Scheduled in six Subspecialty Clinics, March – June 2019 and March – June 2020
      20192020
      #%#%P-value
      P values reflect Chi square analyses.
      Total23,468100%18,884100%
      Age, years
       <115687%11896%.4078
       1-4415418%331718%
       5-9556924%445424%
       10-14662628%544829%
       15-18555124%447624%
      Gender
       Female10,90546%896447%.0676
       Male12,55954%991953%
       Unknown40%10%
      Race / Ethnicity
       Asian304613%228412%<.0001
       Non-Hispanic Black4452%4252%
       Hispanic714630%608332%
       Other20669%17769%
       Unknown450519%331718%
       Non-Hispanic White626027%499926%
      Interpreter Requested
       No19,68484%15,63083%<.0023
       Yes378416%325417%
      Insurance
       Public921539%772541%.0001
       Private non-HMO313013%262314%
       Private HMO10,40444%793342%
       Other7193%6033%
      Distance, miles
       <=5017,10473%13,41471%<.0001
       >50636427%547029%
      Broadband availability at 100/10 mbps
       Quartile 1 (Lowest)481521%399021%.2321
       Quartile 2612226%524028%
       Quartile 3585725%484026%
       Quartile 4 (Highest)636327%455024%
      Income status
       <3 x FPL816835%673336%0.1351
       3-4 x FPL651328%507427%
       >4 x FPL852236%687636%
       n/a2651%2011%
      Patient type
       Established18,10277%15,44882%<.0001
       New536623%343618%
      Visit category
       Office23,31899%11,20959%<.0001
       Telehealth1501%767541%
      Sub-specialty
       Cardiology477320%331318%<.0001
       Endocrinology550423%437423%
       Nephrology11155%8474%
       Neurology580125%514627%
       Oncology22049%210711%
       Pulmonology407117%309716%
      Appointment status
       Completed17,60875%13,58272%<.0001
       Not Completed586025%530228%
      HMO indicates Health Maintenance Organization; FPL, Federal Poverty Level; and Mbps, megabits per second.
      low asterisk P values reflect Chi square analyses.
      Between 2019 and 2020 there was a notable decline of in-person appointments from 23,318 scheduled to 11,209 during the same period in 2020 (Table 1). During the studied 2020 period, 59% were scheduled in-person, office visits while 41% were scheduled for telehealth. Asian, non-Hispanic White, and patients of Unknown race (P= .0001), new patients (P< .0001), privately-insured HMO patients(P< .0001), those residing ≤50 miles away (P< .0001), and those requesting an interpreter (P< .0023) were less likely to be scheduled in 2020. All subspecialties experienced reductions in scheduled appointments of any type during 2020 period.
      When we examined visit types we noted a 20% overall decrease in scheduled clinic visits was dominated by a drop in new patients, which fell from 5,366 to 3,436 (−36%). There was a smaller drop in established patients, from 18,102 to 15,448 (−15%). The setting of care was also dynamic in the wake of COVID-19, as also described above: scheduled in-person office visits decreased from 23,318 to 11,209 (−52%), while telehealth visits scheduled increased markedly from 150 (0.6% of total visits) to 7,675 (a 5017% increase). While the number of completed visits dropped 23% between years, the rate of not completed visits rose slightly to 28%.
      We also assessed demographic characteristics associated with completing an in person or telehealth visit (Table 2). Established patients were no more likely to complete a visit (OR 1.01, CI 0.95–1.07) than were new patients, after adjusting for other characteristics. Those who utilized telehealth (OR 4.21, CI 3.90–4.54) were much more likely to complete visits than were those who scheduled in-person visits. Additional characteristics associated with completing a visit included requesting an interpreter (OR 1.25, CI 1.16–1.34) having private non-HMO insurance (OR 1.19, CI 1.10–1.29), or living in a high-income zip code (OR 1.14, CI 1.08–1.21). Characteristics associated with a decreased likelihood of completing a visit included all age groups >1 year old, Hispanic race/ethnicity (OR 0.90, CI 0.84–0.96) and lower broadband availability (OR 0.86, CI 0.80–0.92). Although scheduled visits declined for all subspecialties during the 2020 period, there was substantial variation in completed visits among the subspecialties. Endocrinology was chosen as the reference as it experienced among the highest visit volume and utilization of telehealth prior to the pandemic. The likelihood of visit completion varied, ranging from neurology (OR 0.91, CI 0.85–0.96) to oncology (OR 5.17, CI 4.60–5.82).
      Table 2.Univariable and Multivariable Regression Analysis of Characteristics Associated With Completed Office or Telehealth Pediatric Subspecialty Visits. (N = 41,324)
      UnivariableMultivariable
      Odds Ratio Estimates
      VariablePoint Estimate with 95% Wald confidence intervalPoint Estimate with 95% Wald confidence interval
      Year, 20200.85 (0.81–0.88)0.49 (0.47–0.52)
      Visit type, Telehealth2.12 (1.98–2.26)4.21 (3.90–4.54)
      Patient Type
       NewReferenceReference
       Established1.11 (1.06–1.17)1.01 (0.95–1.07)
      Age, Years
       <1ReferenceReference
       1-4 years0.82 (1.06–1.17)0.77 (0.69–0.86)
       5-9 years0.72 (0.65–0.80)0.66 (0.59–0.73)
       10-14 years0.65 (0.59–0.72)0.60 (0.54–0.67)
       15-18 years0.59 (0.53–0.65)0.52 (0.47–0.59)
      Sex
       FemaleReferenceReference
       Male1.05 (1.01–1.10)1.03 (0.99–1.08)
      Race/Ethnicity
       Non-Hispanic WhiteReferenceReference
       Asian1.10 (1.02–1.18)0.92 (0.85–1.00)
       Non-Hispanic Black1.04 (0.88–1.22)1.04 (0.88–1.23)
       Hispanic0.98 (0.93–1.04)0.90 (0.84–0.96)
       Other1.05 (0.97–1.15)0.95 (0.87–1.04)
       Unknown0.85 (0.80–0.91)0.89 (0.83–0.95)
      Interpreter requested
       NoReferenceReference
       Yes1.15 (1.08–1.22)1.25 (1.16–1.34)
      Insurance type
       PublicReferenceReference
       Private non-HMO1.14 (1.07–1.23)1.19 (1.10–1.29)
       Private HMO0.98 (0.93–1.03)1.07 (1.00–1.13)
       Other0.83 (0.73–0.94)0.89 (0.78–1.02)
      Distance from clinic, miles
       <50 miReferenceReference
       >50 mi0.96 (0.91–1.01)1.04 (0.97–1.11)
      Income status
       >4 x FPLReferenceReference
       <3 x FPL1.04 (0.98–1.09)1.05 (0.98–1.13)
       3-4 x FPL1.14 (1.08–1.21)1.14 (1.08–1.21)
      Broadband availability at 100mbps
       Highest 75 percentileReferenceReference
       Lowest 25th percentile0.88 (0.84–0.93)0.86 (0.80–0.92)
      Subspecialty
       EndocrinologyReferenceReference
       Cardiology1.25 (1.17–1.34)1.49 (1.38–1.60)
       Nephrology1.60 (1.42–1.80)1.56 (1.38–1.77)
       Neurology0.91 (0.85–0.96)0.86 (0.81–0.92)
       Oncology3.96 (3.54–4.43)5.17 (4.60–5.82)
       Pulmonology0.93 (0.87–1.00)0.92 (0.85–0.99)
      All variables included in multivariable models.
      We also examined the demographic characteristics associated with whether a completed visit was done through an office visit or through telehealth. This analysis was conducted for the March–June 2020 period, as the number of telehealth visits in 2019 was low. (Table 3). Established patients (OR 1.53, CI 1.37–1.70) and families with the lowest 25% percentile of broadband access (OR 1.26, CI 1.12–1.43) were more likely to complete telehealth appointments compared to those in the highest quartiles of broadband access. Patients older than 5 years were also more likely to complete telehealth appointments. In contrast, families requesting an interpreter (OR 0.68, CI 0.60–0.78) were less likely to complete telehealth visits than those without one, as were those of Hispanic race/ethnicity (OR 0.85, CI 0.75–0.95). Insurance type had no impact on telehealth visit completion. The pattern of subspecialty telehealth visit completion varied: relative to endocrinology, patients in cardiology, (OR 0.06, CI 0.05–0.07), oncology (OR 0.06, CI 0.05–0.07), and pulmonology (OR 0.52, CI 0.46–0.59) were less likely to complete scheduled visits, while patients in neurology (OR 1.16, CI 1.04–1.30) was more likely to complete scheduled visits.
      Table 3.Univariable and Multivariable Regression Analysis of Characteristics Associated With Completed Pediatric Subspecialty Visits That Utilized Telehealth, March–June 2020. (N = 13,232)
      UnivariableMultivariable
      Odds Ratio Estimates
      VariablePoint Estimate With 95% Wald Confidence IntervalPoint Estimate With 95% Wald Confidence Interval
      Patient type
       NewReferenceReference
       Established1.46 (1.33–1.60)1.53 (1.37–1.70)
      Age, years
       <1ReferenceReference
       1–41.80 (1.52–2.14)1.15 (0.94–1.40)
       5–92.31 (1.96–2.72)1.34 (1.10–1.63)
       10–142.98 (2.54–3.50)1.45 (1.19–1.76)
       15–183.02 (2.57–3.56)1.56 (1.28–1.90)
      Sex
       FemaleReferenceReference
       Male0.95 (0.89–1.01)0.98 (0.91–1.07)
      Race/ethnicity
       Non–Hispanic WhiteReferenceReference
       Asian0.69 (0.61–0.77)1.01 (0.87–1.16)
       Non-Hispanic Black0.94 (0.74–1.18)1.19 (0.90–1.58)
       Hispanic0.59 (0.54–0.65)0.85 (0.75–0.95)
       Other0.68 (0.60–0.77)0.90 (0.77–1.05)
       Unknown0.89 (0.80–0.99)0.90 (0.79–1.02)
      Interpreter preference
       No interpreterReferenceReference
       Interpreter requested0.56 (0.51–0.61)0.68 (0.60–0.78)
      Insurance type
       PublicReferenceReference
       Private non-HMO1.38 (1.24–1.53)1.08 (0.94–1.24)
       Private HMO1.50 (1.39–1.62)1.09 (0.98–1.21)
       Other insurance1.49 (1.22–1.83)1.21 (0.94–1.56)
      Distance from clinic, miles
       <50ReferenceReference
       >500.82 (0.76–0.89)0.92 (0.82–1.04)
      Income status
       >4 x FPLReferenceReference
       <3 x FPL0.70 (0.64–0.76)0.93 (0.82–1.05)
       3–4 x FPL0.92 (0.84–1.00)1.07 (0.97–1.20)
      Broadband availability at 100 mbps
       Highest 75 percentileReferenceReference
       Lowest 25th percentile1.07 (0.98–1.16)1.26 (1.12–1.43)
      Subspecialty
       EndocrinologyReferenceReference
       Cardiology0.05 (0.05–0.06)0.06 (0.05–0.07)
       Nephrology0.85 (0.71–1.02)0.91 (0.75–1.09)
       Neurology1.10 (0.99–1.23)1.16 (1.04–1.30)
       Oncology0.06 (0.05–0.07)0.06 (0.05–0.07)
       Pulmonology0.46 (0.41–0.52)0.52 (0.46–0.59)
      HMO indicates Health Maintenance Organization; FPL, Federal Poverty Level; and Mbps, megabits per second.
      All variables included in multivariable models.

      Discussion

      The early stages of the pandemic were associated with a major shift from in-person appointments to telehealth encounters. However, the nature and scale of pandemic effects were complex and varied by pediatric subspecialty type, social demographics, and established patient status. In many respects the studied outpatient systems appeared resilient without generating major disparities in utilization during the early months of the pandemic. These findings underscore the need to assess the impact of the pandemic on patterns of pediatric utilization and the potential that differences in health facility practices, patient population characteristics, local technological capacities, and subspecialty requirements may generate new forms of socially disparate care.
      • Cahan EM
      • Mittal V
      • Shah NR
      • et al.
      Achieving a quintuple aim for telehealth in pediatrics.
      ,
      • Zur J
      • Jones E.
      Racial and ethnic disparities among pediatric patients at community health centers.
      • Lee S
      • Martinez G
      • Ma GX
      • et al.
      Barriers to health care access in 13 Asian American communities.
      • Ray KN
      • Kahn JM.
      Connected subspecialty care: applying telehealth strategies to specific referral barriers.
      • Russo JE
      • McCool RR
      • Davies L.
      VA telemedicine: an analysis of cost and time savings.
      Interestingly, established patients were more likely to complete telehealth appointments, a finding consistent with other studies demonstrating that—among diverse patient populations during the pandemic—new patients were less likely to use the technology.
      • Nguyen OT
      • Watson AK
      • Motwani K
      • et al.
      Patient-level factors associated with utilization of telemedicine services from a free clinic during COVID-19.
      There was no policy barring new patients from being seen in person; variation in new patient appointments may have depended on acuity and whether laboratory or imaging studies were needed. The experience of non-White groups was mixed, as no significant disparity in visitation patterns and telehealth use was observed for non-Hispanic Black families. Interestingly, families who requested an interpreter were more likely to complete an appointment but less likely to utilize telehealth. It is unclear if requiring an interpreter was a logistical barrier to utilizing telehealth or if factors associated with requiring an interpreter, such as inadequate technical devices or digital fluency, contributed to this observed disparity. In settings with adequate digital capabilities, telehealth could eliminate obstacles to in-person visitation, such as transportation needs and parental workplace absenteeism.
      • Russo JE
      • McCool RR
      • Davies L.
      VA telemedicine: an analysis of cost and time savings.
      ,
      • McConnochie KM
      • Wood NE
      • Kitzman HJ
      Telemedicine reduces absence resulting from illness in urban child care: evaluation of an innovation.
      One study in adult populations found that no-show rates in follow-up care diminished in the setting of COVID-19, a finding that requires examination in pediatrics.
      • Dekker PK
      • Bhardwaj P
      • Singh T
      • et al.
      Telemedicine in the wake of the COVID-19 pandemic: increasing access to surgical care.
      Examining how language, immigration, generational status, disability status, and cultural factors affect telehealth use may be critical to understanding the mechanisms by which telehealth widens disparities.
      • Feiring E
      • Westdahl S.
      Factors influencing the use of video interpretation compared to in-person interpretation in hospitals: a qualitative study.
      • Rodriguez JA
      • Saadi A
      • Schwamm LH
      Disparities in telehealth use among california patients with limited English proficiency.
      • Valdez RS
      • Rogers CC
      • Claypool H
      • et al.
      Ensuring full participation of people with disabilities in an era of telehealth.
      Leveraging the promise of telehealth will require further study and ongoing vigilance to ensure all demographic groups benefit equitably.
      Overall visit completion, including both in-person and telehealth visits, was more likely among the privately insured and those living in high-income zip codes. Interestingly, these factors did not significantly affect telehealth visit completion, which suggests social stratification in the pandemic's impact on in-person utilization. Moreover, there were no significant differences in the use of telehealth visits by payer status, which is reassuring as nearly half of US children with medical complexity are covered by public insurance programs.
      Health Insurance Coverage for Children with Special Health Care Needs, by Type of Insurance.
      Rapid policy accomodations for telehealth reimbursement likely promoted the absence of payer status effects for children seeking subspecialty care. However, it is not clear whether these relationships extend to pediatric primary care settings, as studies have already observed stratification of access by payer type in adult populations.
      • Powers BWAD
      • Zhao Y
      • Haugh GS
      • et al.
      Association between primary care payment model and telemedicine use for Medicare advantage enrollees during the covid-19 pandemic.
      It is also unclear whether these relationships will persist in a postpandemic period when patient preferences, health facility practices, and reimbursement policies may evolve.
      Our analysis suggests broadband availability, at least as it is currently measured, played only a partial role in shaping pediatric subspecialty care utilization during the studied periods. Families with the lowest 25% percentile of broadband access completed telehealth appointments with greater success. We do not have a clear explanation for this counterintuitive result but it may reflect a greater reliance on a cellular-enabled connection for telehealth appointments among households with relatively poor broadband availability. Previous studies have found that only in “fully rural” counties does broadband access play a significant role in determining telehealth utilization, potentially due to a paucity of alternative internet-enabled sites such as libraries, religious centers, or community centers.
      • Wilcock AD
      • Rose S
      • Busch AB
      • et al.
      Association between broadband internet availability and telemedicine use.
      ,
      • DeGuzman PB
      • Siegfried Z
      • Leimkuhler ME
      Evaluation of rural public libraries to address telemedicine inequities.
      Further, questions have been raised regarding the reliability of the FCC's broadband mapping data to identify deficits in broadband availability at the hyperlocal level,
      Observations on Past and Ongoing Efforts to Expand Access and Improve Mapping Data.
      an issue to be addressed by recent federal legislation directed at expanding broadband access to underserved communities.
      The White House
      UPDATED FACT SHEET: Bipartisan Infrastructure Investment and Jobs Act.
      ,
      United States Senate
      H.R.3684 - Infrastructure Investment and Jobs Act.
      , The COVID-19 pandemic has reinforced the importance of broadband access and helped generate state-led initiatives, such as California's Broadband For All Act of 2022, which have emphasized the “public health imperative” of expanding broadband technology.
      United States Senate
      H.R.3684 - Infrastructure Investment and Jobs Act.
      Our analyses lacked sufficient granularity to assess socially differential rates of technical difficulties during telehealth appointments.
      Nonetheless, even with more accurate broadband data, factors such as internet costs, device adequacy, technology familiarity and literacy, and interface navigability may all be important contributors to telehealth access.
      • Struminger BB
      • Arora S.
      Leveraging telehealth to improve health care access in rural america: it takes more than bandwidth.
      For example, a study of Medicare beneficiaries conducted during summer and fall 2020 reported large portions of beneficiaries without smartphone/tablet devices and home internet; non-Hispanic Black and Hispanic individuals were twice as likely to face such barriers as non-Hispanic Whites.
      • Benjenk I
      • Franzini L
      • Roby D
      • et al.
      Disparities in audio-only telemedicine use among Medicare beneficiaries during the coronavirus disease 2019 pandemic.
      Thus, these populations may rely more on cellular-enabled, smartphone-based connections to complete telehealth appointments, supplanting the need for high-speed broadband. The use of cellular systems for telehealth visits deserves further examination, particularly if telehealth visits are increasingly integrated into complex electronic medical record systems that may require substantial broadband capabilities.
      Established patients were more likely to complete appointments than were new patients, possibly due to increased healthcare system familiarity and therapeutic rapport with the provider and team.
      • Ruggeri K
      • Folke T
      • Benzerga A
      • et al.
      Nudging New York: adaptive models and the limits of behavioral interventions to reduce no-shows and health inequalities.
      While in-person office visits declined for all subspecialties, the completion of telehealth visits was lower in pulmonology, oncology, and cardiology compared to endocrinology; neurology demonstrated higher rates of telehealth visit completion. These patterns may correspond to different interspecialty practice requirements, such as the need for a physical exam or imaging study.
      • Ray KN
      • Demirci JR
      • Bogen DL
      • et al.
      Optimizing telehealth strategies for subspecialty care: recommendations from rural pediatricians.
      It may also be that more acute patients, such as those enrolled in chemotherapy protocols or undergoing cardiac repair, experience enhanced clinical continuity as a result of highly protocolized care which determine visit frequency.
      • Vernacchio L
      • Muto JM
      • Young G
      • et al.
      Ambulatory subspecialty visits in a large pediatric primary care network.
      Objective or perceived disease severity may also dictate the frequency of clinical appointments, and these factors often vary between subspecialties.
      • Zuckerman KE
      • Nelson K
      • Bryant TK
      • et al.
      Specialty referral communication and completion in the community health center setting.
      The ability to distinguish patient subpopulations with elevated clinical needs will be important to assess the appropriate use of telehealth services during the pandemic as well as in a post-COVID-19 era. Additional research and guidelines may prove helpful going forward to stratify patients requiring in-person attention versus those who can be appropriately cared for through virtual means.

      Limitations

      This study describes the experience of a single, quaternary pediatric academic medical center and satellite clinics which is unlikely to be representative of a wide range of healthcare settings. In addition, the nature of our study population was socially diverse and largely restricted to relatively urban and suburban areas of northern California, which also may not be readily comparable to other regions. Accordingly, this study's finding that the pandemic-generated shift to telehealth did not result in pervasive disparities in utilization needs to be assessed in relation to other studies. Indeed, insight into the potential impact of the pandemic and a new reliance on telecommunications for outpatient care will rely on the comparative integration of findings from a number of diverse institutions. Of particular interest will be how the pandemic has affected inequalities in pediatric healthcare utilization in populations of diverse social and technological character.
      There are several other limitations to this study. The data utilized in this study were extracted retrospectively from electronic medical records may include some inaccuracies in some appointment outcomes. Our analysis did not assess disease severity which could be important in assessing the significance of disrupted patterns of care. In addition, the studied dataset does not provide information on telehealth visits that were terminated because of technical problems. Data regarding dropped visits, interrupted and restarted visits, visit duration, utilization of screen sharing, and patient satisfaction deserve future analysis. The analysis was concerned with visits and did not account for the influence of individual patients with more than one visit. It is also possible that selected covariates (such as income status, distance from the clinic, or broadband availability) were defined through geographic residential areas or categorized in ways that did not capture adequately the thresholds at which diminished access to the healthcare system becomes problematic. In addition, our data could not evaluate if the pandemic altered referral patterns for new patients, including their acuity, severity, and need for in-person visitation. Finally, it is possible that utilization patterns may have changed as the pandemic's impact on daily life evolved between March and June 2020. However, a detailed examination of changes over this 4 month period did not reveal clinically meaningful changes. This appears to reflect the fact that through the end of May, the seven Bay Area counties maintained strict lockdowns limiting in-person activities

      Procter R. Remember when? Timeline marks key events in California's year-long pandemic grind. CalMatters. 2021. Available at: https://calmatters.org/health/coronavirus/2021/03/timeline-california-pandemic-year-key-points/. Accessed March 9, 2022.

      . However, it is likely that the findings for this initial phase of the pandemic have likely undergone considerable change since June 2020.

      Conclusion

      The onset of the COVID-19 pandemic was associated with an increased reliance on telehealth for the provision of outpatient pediatric subspecialty care. While this shift in care modalities was associated with considerable variability in completed visits and the use of telehealth among the subspecialties, the impact of the enhanced reliance on telehealth on social disparities in outpatient utilization was complex. In this study, demographic characteristics and the residential availability of highspeed internet had a mixed influence on utilization. While these findings reflect the experience of only one pediatric referral facility, the complexity of the observed relationships underscores the need for additional analyses of increased telehealth usage in different clinical, technological, and social settings.

      Funding

      All phases of this study was supported by a Health Resources and Services Administration “Center of Excellence COVID” grant, T-1NHP39160. All authors are affiliates of Stanford School of Medicine. The funding committee had no role in the design or conduct of this study.

      References

        • Whaley CM
        • Pera MF
        • Cantor J
        • et al.
        Changes in health services use among commercially insured US populations during the COVID-19 pandemic.
        JAMA Netw Open. 2020; 3e2024984
        • Ye S
        • Kronish I
        • Fleck E
        • et al.
        Telemedicine expansion during the COVID-19 pandemic and the potential for technology-driven disparities.
        J Gen Intern Med. 2021; 36: 256-258
        • Curfman AL
        • Hackell JM
        • Herendeen NE
        • et al.
        Telehealth: improving access to and quality of pediatric health care.
        Pediatrics. 2021; 148e2021053129
        • Brown CL
        • Montez K
        • Amati JB
        • et al.
        Impact of COVID-19 on pediatric primary care visits at four academic institutions in the Carolinas.
        Int J Environ Res Public Health. 2021; 18
        • Veinot TC
        • Mitchell H
        • Ancker JS.
        Good intentions are not enough: how informatics interventions can worsen inequality.
        J Am Med Inform Assoc. 2018; 25: 1080-1088
        • Nguyen OT
        • Watson AK
        • Motwani K
        • et al.
        Patient-level factors associated with utilization of telemedicine services from a free clinic during COVID-19.
        Telemed J E Health. 2021; ([e-pub ahead of print])https://doi.org/10.1089/tmj.2021.0102
        • Pierce RP
        • Stevermer JJ.
        Disparities in use of telehealth at the onset of the COVID-19 public health emergency.
        J Telemed Telecare. 2020; ([e-pub ahead of print]. 1357633X20963893)https://doi.org/10.1177/1357633X20963893
        • The White House
        UPDATED FACT SHEET: Bipartisan Infrastructure Investment and Jobs Act.
        The White House, 2021 (Available at:) (Accessed March 9, 2022)
        • United States Senate
        H.R.3684 - Infrastructure Investment and Jobs Act.
        US Senate, 2021 (Available at:) (Accessed March 9, 2022)
        • California State Assembly
        AB-34 Broadband for All Act of 2022.
        California State Assembly, 2021 (Available at:) (Accessed March 9, 2022)
        • Cahan EM
        • Mittal V
        • Shah NR
        • et al.
        Achieving a quintuple aim for telehealth in pediatrics.
        Pediatr Clin North Am. 2020; 67: 683-705
        • Tilden DR
        • Datye KA
        • Moore DJ
        • et al.
        The rapid transition to telemedicine and its effect on access to care for patients with type 1 diabetes during the COVID-19 pandemic.
        Diabetes Care. 2021; 44: 1447-1450
        • Chang JE
        • Lai AY
        • Gupta A
        • et al.
        Rapid transition to telehealth and the digital divide: implications for primary care access and equity in a post-COVID era.
        Milbank Q. 2021; 99: 340-368
      1. Federal Communication Commission. 2020 Broadband Deployment Report. Available at: https://docs.fcc.gov/public/attachments/FCC-20-50A1.pdf. Accessed March 9, 2022.

        • Zur J
        • Jones E.
        Racial and ethnic disparities among pediatric patients at community health centers.
        J Pediatr. 2015; 167: 845-850
        • Lee S
        • Martinez G
        • Ma GX
        • et al.
        Barriers to health care access in 13 Asian American communities.
        Am J Health Behav. 2010; 34: 21-30
        • Ray KN
        • Kahn JM.
        Connected subspecialty care: applying telehealth strategies to specific referral barriers.
        Acad Peds. 2020; 20: 16-22
        • Russo JE
        • McCool RR
        • Davies L.
        VA telemedicine: an analysis of cost and time savings.
        Telemed J E Health. 2016; 22: 209-215
        • McConnochie KM
        • Wood NE
        • Kitzman HJ
        Telemedicine reduces absence resulting from illness in urban child care: evaluation of an innovation.
        Pediatrics. 2005; 115: 1273-1282
        • Dekker PK
        • Bhardwaj P
        • Singh T
        • et al.
        Telemedicine in the wake of the COVID-19 pandemic: increasing access to surgical care.
        Plast Reconstr Surg Glob Open. 2021; 9: e3228
        • Feiring E
        • Westdahl S.
        Factors influencing the use of video interpretation compared to in-person interpretation in hospitals: a qualitative study.
        BMC Health Serv Res. 2020; 20: 856
        • Rodriguez JA
        • Saadi A
        • Schwamm LH
        Disparities in telehealth use among california patients with limited English proficiency.
        Health Aff (Millwood). 2021; 40: 487-495
        • Valdez RS
        • Rogers CC
        • Claypool H
        • et al.
        Ensuring full participation of people with disabilities in an era of telehealth.
        J Am Med Inform Assoc. 2021; 28: 389-392
      2. Health Insurance Coverage for Children with Special Health Care Needs, by Type of Insurance.
        KidsData, 2022 (Available at:) (Accessed March 9, 2022)
        • Powers BWAD
        • Zhao Y
        • Haugh GS
        • et al.
        Association between primary care payment model and telemedicine use for Medicare advantage enrollees during the covid-19 pandemic.
        JAMA Health Forum. 2021; 2e211597
        • Wilcock AD
        • Rose S
        • Busch AB
        • et al.
        Association between broadband internet availability and telemedicine use.
        JAMA Intern Med. 2019; 179: 1580-1582
        • DeGuzman PB
        • Siegfried Z
        • Leimkuhler ME
        Evaluation of rural public libraries to address telemedicine inequities.
        Public Health Nurs. 2020; 37: 806-811
      3. Observations on Past and Ongoing Efforts to Expand Access and Improve Mapping Data.
        Government Accountability Office, 2022 (Available at:) (Accessed March 9, 2022)
        • United States Congress
        S.1822 - Broadband DATA Act.
        Congress.gov, 2019 (Available at:) (Accessed March 9, 2022)
        • Struminger BB
        • Arora S.
        Leveraging telehealth to improve health care access in rural america: it takes more than bandwidth.
        Ann Intern Med. 2019; 171: 376-377
        • Benjenk I
        • Franzini L
        • Roby D
        • et al.
        Disparities in audio-only telemedicine use among Medicare beneficiaries during the coronavirus disease 2019 pandemic.
        Med Care. 2021; 59: 1014-1022
        • Ruggeri K
        • Folke T
        • Benzerga A
        • et al.
        Nudging New York: adaptive models and the limits of behavioral interventions to reduce no-shows and health inequalities.
        BMC Health Serv Res. 2020; 20: 363
        • Ray KN
        • Demirci JR
        • Bogen DL
        • et al.
        Optimizing telehealth strategies for subspecialty care: recommendations from rural pediatricians.
        Telemed J E Health. 2015; 21: 622-629
        • Vernacchio L
        • Muto JM
        • Young G
        • et al.
        Ambulatory subspecialty visits in a large pediatric primary care network.
        Health Serv Res. 2012; 47: 1755-1769
        • Zuckerman KE
        • Nelson K
        • Bryant TK
        • et al.
        Specialty referral communication and completion in the community health center setting.
        Acad Pediatr. 2011; 11: 288-296
      4. Procter R. Remember when? Timeline marks key events in California's year-long pandemic grind. CalMatters. 2021. Available at: https://calmatters.org/health/coronavirus/2021/03/timeline-california-pandemic-year-key-points/. Accessed March 9, 2022.