Free access
Original Research
9 January 2024

Real-World Effectiveness of BNT162b2 Against Infection and Severe Diseases in Children and AdolescentsFREE

Publication: Annals of Internal Medicine
Volume 177, Number 2
Visual Abstract. Real-World Effectiveness of BNT162b2 Against Infection and Severe Diseases in Children and Adolescents
Using electronic health record data from a national collaboration of pediatric health systems, this study examined the effectiveness of BNT162b2 vaccines for preventing COVID-19 infections and severe disease among previously uninfected children and adolescents. It found moderate effectiveness of the vaccine against the SARS-CoV-2 Omicron variant and high effectiveness against the Delta variant.

Abstract

Background:

The efficacy of the BNT162b2 vaccine in pediatrics was assessed by randomized trials before the Omicron variant’s emergence. The long-term durability of vaccine protection in this population during the Omicron period remains limited.

Objective:

To assess the effectiveness of BNT162b2 in preventing infection and severe diseases with various strains of the SARS-CoV-2 virus in previously uninfected children and adolescents.

Design:

Comparative effectiveness research accounting for underreported vaccination in 3 study cohorts: adolescents (12 to 20 years) during the Delta phase and children (5 to 11 years) and adolescents (12 to 20 years) during the Omicron phase.

Setting:

A national collaboration of pediatric health systems (PEDSnet).

Participants:

77 392 adolescents (45 007 vaccinated) during the Delta phase and 111 539 children (50 398 vaccinated) and 56 080 adolescents (21 180 vaccinated) during the Omicron phase.

Intervention:

First dose of the BNT162b2 vaccine versus no receipt of COVID-19 vaccine.

Measurements:

Outcomes of interest include documented infection, COVID-19 illness severity, admission to an intensive care unit (ICU), and cardiac complications. The effectiveness was reported as (1-relative risk)*100, with confounders balanced via propensity score stratification.

Results:

During the Delta period, the estimated effectiveness of the BNT162b2 vaccine was 98.4% (95% CI, 98.1% to 98.7%) against documented infection among adolescents, with no statistically significant waning after receipt of the first dose. An analysis of cardiac complications did not suggest a statistically significant difference between vaccinated and unvaccinated groups. During the Omicron period, the effectiveness against documented infection among children was estimated to be 74.3% (CI, 72.2% to 76.2%). Higher levels of effectiveness were seen against moderate or severe COVID-19 (75.5% [CI, 69.0% to 81.0%]) and ICU admission with COVID-19 (84.9% [CI, 64.8% to 93.5%]). Among adolescents, the effectiveness against documented Omicron infection was 85.5% (CI, 83.8% to 87.1%), with 84.8% (CI, 77.3% to 89.9%) against moderate or severe COVID-19, and 91.5% (CI, 69.5% to 97.6%) against ICU admission with COVID-19. The effectiveness of the BNT162b2 vaccine against the Omicron variant declined 4 months after the first dose and then stabilized. The analysis showed a lower risk for cardiac complications in the vaccinated group during the Omicron variant period.

Limitation:

Observational study design and potentially undocumented infection.

Conclusion:

This study suggests that BNT162b2 was effective for various COVID-19–related outcomes in children and adolescents during the Delta and Omicron periods, and there is some evidence of waning effectiveness over time.

Primary Funding Source:

National Institutes of Health.
The U.S. Food and Drug Administration expanded the emergency use authorization of the BNT162b2 messenger RNA COVID-19 vaccine (Pfizer–BioNTech) to 12 to 15 year olds on 10 May 2021 and to 5 to 11 year olds on 29 October 2021. As of 5 April 2023, the Centers for Disease Control and Prevention reports indicate that 40% of U.S. children aged 5 to 11 years and 72% of adolescents aged 12 to 18 years had received at least 1 dose of the vaccine. The emergence of the Omicron variant (B.1.1.529) and its subvariants in early 2022 led to a new surge in COVID-19 cases worldwide (1). The randomized trials of the BNT162b2 vaccine, which demonstrated high efficacy of 2 doses against COVID-19 (100% and 91% among those aged 12 to 15 years and 5 to 11 years, respectively), were done before the emergence of the Omicron variant (2, 3).
Several observational studies have been done to investigate the effectiveness of vaccination in real-world settings (4–8). However, prior studies have had limited follow-up periods, covering the Delta variant or earlier subvariants of Omicron periods only. Studies evaluating the Omicron variant have only assessed the short-term effects of the vaccine, with only 1 study involving children evaluating the effect beyond 3 months (9). There is limited information on the long-term durability of vaccine protection during the Omicron period. Few existing studies on U.S. pediatric populations have covered both hospitalized patients and those with mild or asymptomatic conditions. Furthermore, although studies have acknowledged limitations due to misclassification in vaccination status in real-world effectiveness studies, none have rigorously evaluated the effects of such misclassification nor accounted for the potential bias it may introduce.
To address these gaps in our knowledge of the pediatric effectiveness of SARS-CoV-2 vaccination, we designed this study to assess the real-world effectiveness of BNT162b2 among children and adolescents during the periods when the Delta and Omicron variants were predominant using electronic health record (EHR) data from a national network of U.S. pediatric medical centers. Our study used a novel comparative effectiveness research (CER) method and adjusted for underreporting issues in vaccination status and has several attractive features that strengthen the generalizability and reliability of our inference. First, it is the largest study to date in the United States estimating vaccine effectiveness in children and adolescents, covering a broad spectrum of the U.S. pediatric population. Second, the study examined the effectiveness against infection over a longer follow-up than any previous study, enabling evaluation of the durability of vaccine protection. Third, the study included a diverse representation of U.S. pediatric populations from primary care, specialty care, emergency department (ED), testing centers, and inpatient settings. Fourth, the study was the first to account for the incomplete capture of vaccination status by health systems in the United States. Finally, in addition to infection and severe diseases end points, we also studied the effect of vaccination on the incidence of myocarditis, pericarditis, or multisystem inflammatory syndrome (MIS) to assess the effect of vaccination relative to the potential risk for cardiac complications.

Methods

Data Sources

This study used EHR data from PEDSnet (10), which is a national collaboration of pediatric health systems that share EHR data, conduct research, and improve outcomes together. Participating institutions in this study included the Children’s Hospital of Philadelphia, Cincinnati Children’s Hospital Medical Center, Children’s Hospital Colorado, Ann & Robert H. Lurie Children’s Hospital of Chicago, Nationwide Children’s Hospital, Nemours Children’s Health System (inclusive of the Delaware and Florida health system), Seattle Children’s Hospital, and Stanford Children’s Health. Data were extracted from the PEDSnet COVID-19 Database Version Week 141 (11). A detailed description of EHR data is available in section 1 of the Supplement.

Specification of Hypothetical Trials and CER Studies

Hypothetical randomized controlled trials were specified to guide the design of observational studies to assess the real-world effectiveness of treatments (12). We designed and conducted CER studies to investigate the effectiveness of the BNT162b2 vaccine in preventing infection with various strains of the SARS-CoV-2 virus in children and adolescents in the United States. The 3 study cohorts focused on documented SARS-CoV-2 infection and outcomes: study cohort 1 (Delta study in adolescents): adolescents aged 12 to 20 years during the period when the Delta variant was prevalent from 1 July to 30 November 2021; study cohort 2 (Omicron study in children): children aged 5 to 11 years during the period when the Omicron variant was prevalent from 1 January to 30 November 2022; and study cohort 3 (Omicron study in adolescents): adolescents aged 12 to 20 years during the period when the Omicron variant was prevalent from 1 January to 30 November 2022.
The design of hypothetical trials and implementation procedures in real-world data are summarized in Supplement Table 1.
Eligibility criteria included age of 5 to 11 years for children or 12 to 20 years for adolescents at the start of the study period and no previous COVID-19 vaccination or documented SARS-CoV-2 infection. In addition, participants were required to have a prior encounter (including telephone or telehealth encounters) within 18 months of cohort entrance to ensure that they had an ongoing interaction with the health system.
The intervention of interest was vaccination, in comparison with no receipt of any type of COVID-19 vaccine. Given that the BNT162b2 vaccine covered more than 85% of documented vaccinations among children and adolescents in the PEDSnet database, in this study we focused primarily on studying the effectiveness of this vaccine, although the Supplement reports a sensitivity analysis investigating all types of reported COVID-19 vaccines assessed in the United States, with 85.7% BNT162b2, 1.9% mRNA-1273 (Moderna), and 12.3% unspecified COVID-19 vaccine.
In this CER study using real-world data, the cohort entrance date for the intervention group was defined as the date of the first dose of the BNT162b2 vaccine, whereas for the comparator group, it was a randomly selected date from visits, chosen to ensure that the distribution of index dates for the control group matched the distribution of index dates for the vaccination group to control for time effects. The risk period for the study began 28 days after the index date, such that infections within 28 days were excluded.
Randomized trials achieve balance across potential confounders by randomly allocating the treatment to intervention and comparator groups. In our study, we attempted to balance the intervention and comparator groups by adjusting for a large number of measured confounders collected before cohort entry using propensity score stratification (13). We built the propensity score model on the basis of demographic factors, including age, sex, race, and ethnicity; clinical factors, including obesity status, a baseline chronic condition indicator as defined by the Pediatric Medical Complexity Algorithm (14), and a list of preexisting chronic conditions; and health care utilization factors collected before the cohort entry, including the number of inpatients, outpatients, ED visits, unique mediations, and negative COVID-19 tests. We stratified the patients into propensity score quintiles on the basis of these factors. See Supplement Table 3 for detailed definitions of study variables.
The 4 COVID-19 outcomes of interest were documented SARS-CoV-2 infection, mild COVID-19, moderate or severe COVID-19, and ICU admission with COVID-19. We did not evaluate death from COVID-19 because it was too rare among children and adolescents to study quantitatively. In our study, SARS-CoV-2 infections were defined by positive polymerase chain reaction, serology, or antigen tests or diagnoses of COVID-19 or by occurrence of postacute sequelae of SARS-CoV-2 or MIS in children regardless of the presence of symptoms. Classification of mild, moderate, or severe COVID-19 illness was defined on the basis of the symptoms and health conditions diagnosed from 7 days before to 13 days after the date of a documented COVID-19 infection, as in Forrest and colleagues study (15). Admission to an ICU with COVID-19 was defined by any ICU visit 7 days before and 13 days after documented SARS-CoV-2 infection. In addition, we considered the clinical outcomes of cardiac complications identified as incidence of myocarditis, pericarditis, or MIS to allow for a comprehensive capture of potential cardiac complications after infection and evaluate the effect of the BNT162b2 vaccine in terms of cardiac risks.

Statistical Analysis

We evaluated covariate balancing after propensity score stratification by plotting the standardized mean differences (SMDs) between variable values for the vaccinated and unvaccinated groups, with a difference of 0.1 or less indicating an acceptable balance. We used Poisson regression to estimate the relative risk between 2 treatment groups for the risk for each outcome while adjusting for different follow-up lengths among participants. Because immunization records are often captured and stored across multiple disconnected sources, resulting in underreported vaccination records in patients’ EHRs, we mitigated the potential bias arising from this underreporting issue by incorporating an integration likelihood (16, 17) of the Poisson regression with a prespecified range of misclassification rates. The vaccine effectiveness was defined as (1-relative risk)*100. The details of statistical methods are described in section 2 of the Supplement.
We conducted secondary analyses stratified by 2-month intervals since receipt of vaccination to investigate the durability of vaccine protection. Subgroup analyses were also done to investigate differences in vaccine protection according to age groups (5 to 8, 9 to 11, 12 to 15, and 16 to 20 years).

Sensitivity Analyses

Extensive sensitivity analyses were done to evaluate the robustness of the research findings; see sections 6 to 15 and section 19 of the Supplement for the effects of the cohort design. In scenarios in which any categorical covariates were unbalanced (with an SMD >0.1), we included a sensitivity analysis excluding participants in that category. A sensitivity analysis defining the risk period with no waiting period after the index date was done. To assess whether population heterogeneity, established by applying eligibility criteria that required a prior encounter within 18 months of cohort entry, could influence our vaccine effectiveness estimates, we conducted sensitivity analyses within a more restricted time window. We conducted a dose-response study of BNT162b2 by defining a single-dose vaccination as 1 dose 28 days before infection and a 2-dose vaccination as the receipt of a second dose 7 days before infection. A sensitivity analysis was done to compare the proposed method to a sequential target trial emulation pipeline not accounting for underreporting issues to assess the robustness of findings (18). Because the proportion of patients entered with ED visits is relatively low in the vaccinated cohort compared with the unvaccinated, we conducted a sensitivity analysis excluding all participants who entered the cohort during an ED visit. Residual study bias from unmeasured and systematic sources can still exist in observational studies after controlling for measured confounders; thus, we conducted negative control outcome experiments (13, 19, 20) in which the null hypothesis of no effect was believed to be true using 40 negative control outcomes prespecified by pediatric physicians. The empirical null distribution and calibrated effectiveness were reported as sensitivity analyses. The relative risks for cardiac complications defined by myocarditis or pericarditis (excluding MIS) were estimated. We also reported the estimated vaccine effectiveness from all brands of COVID-19 vaccines. Given the prolonged presence of the Omicron variant and the emergence of several subvariants, we conducted a secondary analysis to assess the vaccine effectiveness related to subvariants of Omicron.

Missingness in Vaccine Records

Vaccine status may be missing for persons whose vaccine doses were administered by a site outside of the PEDSnet network care delivery sites. It is likely that patients recorded as vaccinated in the EHR are true positives, so specificity could be very high, but sensitivity would be reduced by undocumented vaccinations (false negatives). To account for potential bias from the underreporting issue in vaccination status, a range of possible sensitivities based on our prior study was prespecified for each study. The sensitivity range was considered to be 0.8 to 1 for the study involving children and 0.7 to 0.9 for the studies involving adolescents. By accounting for the underreporting and specifying a range of sensitivity, the study aimed to minimize the effect of bias caused by the underreporting in the estimation of the effectiveness of the COVID-19 vaccine among children and adolescents. The details of statistical methods are described in section 2 of the Supplement.
To further evaluate the robustness of statistical methods we used to account for underreporting, we varied our CER method by considering alternative methods for bias correction, including the naive method (without adjusting for underreporting), using different ranges of misclassification rates, and using a fully Bayesian method (21). To evaluate the effect of differential misclassification on effectiveness estimates, we conducted sensitivity analyses simulating vaccination status according to various differential misclassification scenarios. Results from these sensitivity analyses are summarized in sections 16 to 18 of the Supplement.

Role of the Funding Source

The funders had no role in the design or conduct of the study; collection, management, analysis, or interpretation of the data; approval of the manuscript; or the decision to submit the manuscript for publication.

Results

Study Population

Figure 1 shows the process of selection of 3 study cohorts. A total of 77 392 adolescents (45 007 vaccinated) within the PEDSnet network were identified to study the effectiveness of vaccination against infection with the Delta variant and severe outcomes (see Table 1 for baseline characteristics). A total of 111 539 children (50 398 vaccinated) and 56 080 adolescents (21 180 vaccinated) were included in the cohort to study the effectiveness of vaccination against infection with the Omicron variant (see Table 2 for baseline characteristics). The vaccinated and unvaccinated groups had a slightly unbalanced distribution of testing rates before cohort entry across all 3 cohorts. After propensity score stratification, all covariates were well balanced between vaccinated and unvaccinated groups with an SMD smaller than 0.1 in the Omicron study involving children (Supplement Figure 12) and involving adolescents (Supplement Figure 13). In the study evaluating vaccine effectiveness for adolescents during the Delta period, 1 site remained unbalanced after propensity score stratification with an SMD larger than 0.1, and thus a sensitivity analysis was done by excluding participants from this site, which gave consistent results with the primary analysis (see Supplement Figure 11 and section 6 in the Supplement). Additional characteristics of the study cohorts, including additional medical conditions, deaths, and follow-up durations, are summarized in section 5 of the Supplement.
Figure 1. Selection of participants for the 3 study cohorts evaluating the effectiveness of the BNT162b2 vaccine in preventing infection and severe diseases with SARS-CoV-2 in adolescents aged 12 to 20 years during the period when the Delta variant was prevalent (study cohort 1) and children aged 5 to 11 years and adolescents aged 12 to 20 years during the period when the Omicron variant was prevalent (study cohorts 2 and 3, respectively).
Table 1. Baseline Characteristics of Adolescents Aged 12 to 20 Years in the Study of the Effectiveness of the BNT162b2 Vaccine in Preventing Infection and Severe Diseases With SARS-CoV-2 During the Period When the Delta Variant Was Prevalent
CharacteristicVaccinated (n = 45 007)Unvaccinated (n = 32 385)Overall (n = 77 392)
Median age (Q1–Q3), y 14 (13–16)15 (13–17)15 (13–16)
    
Distribution, n (%)   
 12 y8922 (19.8)4926 (15.2)13 848 (17.9)
 13 y8099 (18.0)4864 (15.0)12 963 (16.7)
 14 y8037 (17.9)4923 (15.2)12 960 (16.7)
 15 y7311 (16.2)4749 (14.7)12 060 (15.6)
 16 y5450 (12.1)4546 (14.0)9996 (12.9)
 17 y4076 (9.1)3991 (12.3)8067 (10.4)
 18 y1674 (3.7)2075 (6.4)3749 (4.8)
 19 y942 (2.1)1461 (4.5)2403 (3.1)
 20 y496 (1.1)850 (2.6)1346 (1.7)
    
Sex, n (%)   
 Female23 589 (52.4)16 500 (50.9)40 089 (51.8)
 Male21 416 (47.6)15 880 (49.0)37 296 (48.2)
    
Ethnicity, n (%)   
 White16 446 (50.8)17 964 (39.9)34 410 (44.5)
 Black/African American6019 (18.6)12 012 (26.7)18 031 (23.3)
 Hispanic4925 (15.2)9629 (21.4)14 554 (18.8)
 Other/unknown4995 (15.4)5402 (12.0)10 397 (13.4)
    
Hospital, n (%)   
 A5424 (12.1)7385 (22.8)12 809 (16.6)
 B12 884 (28.6)5216 (16.1)18 100 (23.4)
 C6333 (14.1)3457 (10.7)9790 (12.6)
 D1723 (3.8)914 (2.8)2637 (3.4)
 E4369 (9.7)6063 (18.7)10 432 (13.5)
 F12 831 (28.5)3409 (10.5)16 240 (21.0)
 G1430 (3.2)1457 (4.5)2887 (3.7)
 H13 (0.0)4484 (13.8)4497 (5.8)
    
Entry time, n (%)   
 July–September 202138 335 (85.2)24 509 (75.7)62 844 (81.2)
 October–November 20216672 (14.8)7876 (24.3)14 548 (18.8)
    
Obesity, n (%)   
 028 029 (62.3)22 479 (69.4)50 508 (65.3)
 116 978 (37.7)9906 (30.6)26 884 (34.7)
    
Pediatric Medical Complexity Algorithm, n (%)   
 025 634 (57.0)19 916 (61.5)45 550 (58.9)
 110 915 (24.3)6417 (19.8)17 332 (22.4)
 28458 (18.8)6052 (18.7)14 510 (18.7)
    
Negative tests before entry, n (%)   
 01330 (3.0)2888 (8.9)4218 (5.5)
 134 272 (76.1)16 299 (50.3)50 571 (65.3)
 27388 (16.4)9739 (30.1)17 127 (22.1)
 ≥32017 (4.5)3459 (10.7)5476 (7.1)
Q = quartile.
Table 2. Baseline Characteristics of Children Aged 5 to 11 Years and Adolescents Aged 12 to 20 Years in the Study of the Effectiveness of the BNT162b2 Vaccine in Preventing Infection and Severe Diseases With SARS-CoV-2 During Periods When the Omicron Variant Was Prevalent
CharacteristicOmicron Study in ChildrenOmicron Study in Adolescents
 Vaccinated (n = 50 398)Unvaccinated (n = 61 141)Overall (n = 111 539)Vaccinated (n = 21 180)Unvaccinated (n = 34 900)Overall (n = 56 080)
Median age (Q1–Q3), y 8 (6–10)7 (6–9)8 (6–10)14 (13–16)15 (13–17)15 (13–17)
       
Distribution, n (%)      
 5 y8165 (16.2)13 321 (21.8)21 486 (19.3)
 6 y7447 (14.8)11 314 (18.5)18 761 (16.8)
 7 y7090 (14.1)9151 (15.0)16 241 (14.6)
 8 y7028 (13.9)7922 (13.0)14 950 (13.4)
 9 y6773 (13.4)7085 (11.6)13 858 (12.4)
 10 y7011 (13.9)6434 (10.5)13 445 (12.1)
 11 y6884 (13.7)5914 (9.7)12 798 (11.5)
 12 y4754 (22.4)5760 (16.5)10 514 (18.7)
 13 y3421 (16.2)5520 (15.8)8941 (15.9)
 14 y3338 (15.8)5299 (15.2)8637 (15.4)
 15 y3123 (14.7)5315 (15.2)8438 (15.0)
 16 y2634 (12.4)4944 (14.2)7578 (13.5)
 17 y2201 (10.4)3836 (11.0)6037 (10.8)
 18 y986 (4.7)2114 (6.1)3100 (5.5)
 19 y475 (2.2)1400 (4.0)1875 (3.3)
 20 y248 (1.2)712 (2.0)960 (1.7)
       
Sex, n (%)      
 Female23 962 (47.5)28 669 (46.9)52 631 (47.2)11 402 (53.8)17 954 (51.4)29 356 (52.3)
 Male26 436 (52.5)32 468 (53.1)58 904 (52.8)9775 (46.2)16 939 (48.5)26 714 (47.6)
       
Ethnicity, n (%)      
 White14 399 (28.6)24 644 (40.3)39 043 (35.0)16 240 (46.5)6836 (32.3)23 076 (41.1)
 Black/African American13 711 (27.2)13 733 (22.5)27 444 (24.6)6154 (17.6)6157 (29.1)12 311 (22.0)
 Hispanic12 119 (24.0)12 781 (20.9)24 900 (22.3)6287 (18.0)3784 (17.9)10 071 (18.0)
 Other/unknown10 169 (20.2)9983 (16.3)20 152 (18.1)6219 (17.8)4403 (20.8)10 622 (18.9)
       
Hospital, n (%)      
 A5019 (10.0)9266 (15.2)14 285 (12.8)2131 (10.1)5183 (14.9)7314 (13.0)
 B15 229 (30.2)13 168 (21.5)28 397 (25.5)6397 (30.2)6556 (18.8)12 953 (23.1)
 C5482 (10.9)7409 (12.1)12 891 (11.6)1719 (8.1)4075 (11.7)5794 (10.3)
 D4766 (9.5)2878 (4.7)7644 (6.9)678 (3.2)1337 (3.8)2015 (3.6)
 E5843 (11.6)10 551 (17.3)16 394 (14.7)2047 (9.7)4263 (12.2)6310 (11.3)
 F9786 (19.4)11 348 (18.6)21 134 (18.9)3563 (16.8)5186 (14.9)8749 (15.6)
 G1250 (2.5)3239 (5.3)4489 (4.0)622 (2.9)2353 (6.7)2975 (5.3)
 H3023 (6.0)3282 (5.4)6305 (5.7)4023 (19.0)5947 (17.0)9970 (17.8)
       
Entry time, n (%)      
 January–March 202237 970 (75.3)32 523 (53.2)70 493 (63.2)14 684 (69.3)19 032 (54.5)33 716 (60.1)
 April–June 20225882 (11.7)11 919 (19.5)17 801 (16.0)3344 (15.8)7087 (20.3)10 431 (18.6)
 July–September 20224994 (9.9)10 329 (16.9)15 323 (13.7)2206 (10.4)5479 (15.7)7685 (13.7)
 October–November 20221552 (3.1)6370 (10.4)7922 (7.1)946 (4.5)3302 (9.5)4248 (7.6)
       
Obesity, n (%)      
 033 381 (66.2)42 165 (69.0)75 546 (67.7)13 832 (65.3)23 895 (68.5)37 727 (67.3)
 117 017 (33.8)18 976 (31.0)35 993 (32.3)7348 (34.7)11 005 (31.5)18 353 (32.7)
       
Pediatric Medical Complexity Algorithm, n (%)      
 033 870 (67.2)40 976 (67.0)74 846 (67.1)13 482 (63.7%)21 079 (60.4)34 561 (61.6)
 110 000 (19.8)11 189 (18.3)21 189 (19.0)4382 (20.7%)6764 (19.4)11 146 (19.9)
 26528 (13.0)8976 (14.7)15 504 (13.9)3316 (15.7%)7057 (20.2)10 373 (18.5)
       
Negative tests before entry, n (%)      
 02337 (4.6)5640 (9.2)7977 (7.2)768 (3.6)2966 (8.5)3734 (6.7)
 134 077 (67.6)28 417 (46.5)62 494 (56.0)15 707 (74.2)17 303 (49.6)33 010 (58.9)
 210 514 (20.9)19 816 (32.4)30 330 (27.2)3654 (17.3)11 012 (31.6)14 666 (26.2)
 ≥33470 (6.9)7268 (11.9)10 738 (9.6)1051 (5.0)3619 (10.4)4670 (8.3)
Q = quartile.

Vaccine Effectiveness

Table 3 summarizes the estimated vaccine effectiveness in 3 study cohorts, and Figure 2 shows the durability of protection. The vaccine effectiveness was estimated to be 98.4% (95% CI, 98.1% to 98.7%) among adolescents in the Delta period, 74.3% (CI, 72.2% to 76.2%) against documented infection among children in the Omicron period, and 85.5% (CI, 83.8% to 87.1%) among adolescents in the Omicron period. During the Delta period, the vaccine effectiveness against documented infection remained stable throughout the follow-up period of the study. Four months after the first dose, vaccine effectiveness against documented infection with Omicron declined from 82.3% (CI, 77.9% to 85.8%) to 70.6% (CI, 65.9% to 74.6%) among children, and from 91.3% (CI, 87.6% to 94.0%) to 82.9% (CI, 79.0% to 86.1%) among adolescents. Although vaccine effectiveness against documented infection stabilized after this initial decline, the corresponding CIs were much wider, indicating higher levels of uncertainty.
Figure 2. Stratified effectiveness of the BNT162b2 vaccine in preventing infection with SARS-CoV-2 in children and adolescents by 2-month intervals since receipt of the first dose.
Table 3. Estimated Effectiveness of the BNT162b2 Vaccine in Preventing Infection and Severe Diseases With SARS-CoV-2 in Children and Adolescents
Epidemiologic MeasureVaccinatedUnvaccinatedOverallVaccine Effectiveness (95% CI), %
Delta study in adolescents
 Follow-up    
  Total follow-up, n of person-weeks644 162398 9061 043 068
  Median (Q1–Q3)16 (12–18)13 (9–17)15 (10–18)
 Absolute risk, %    
  Documented infection0.355.262.4198.4 (98.1–98.7)
  Mild COVID-190.061.430.6399.0 (98.5–99.3)
  Moderate or severe COVID-190.030.490.2298.7 (97.4–99.3)
  ICU admission with COVID-19≤0.010.05≤0.0399.0 (92.5–99.9)
  Cardiac complication0.020.030.021.22 (0.34–4.35)*
 Age 12–15 y    
  Total follow-up, n of person-weeks458 981229 083688 064
  Documented infection, %0.345.522.2899.0 (98.6–99.3)
 Age 16–21 y    
  Total follow-up, n of person-weeks185 181169 823355 003
  Documented infection, %0.364.912.6497.0 (95.9–97.8)
     
Omicron study in children
 Follow-up    
  Total follow-up, n of person-weeks1 925 6861 911 5993 837 285
  Median (Q1–Q3)44 (35–46)36 (19–44)40 (25–45)
 Absolute risk, %    
  Documented infection1.895.463.8574.3 (72.2–76.2)
  Mild COVID-190.541.551.0973.5 (69.2–77.1)
  Moderate or severe COVID-190.190.670.4575.5 (69.0–81.0)
  ICU admission with COVID-190.020.080.0584.9 (64.8–93.5)
  Cardiac complication≤0.010.03≤0.020.28 (0.08–0.95)*
 Age 5–8 y    
  Total follow-up, n of person-weeks1 101 4181 254 8192 356 236
  Documented infection, %1.965.103.7871.3 (68.2–74.1)
 Age 9–11 y    
  Total follow-up, n of person-weeks824 268656 7801 481 049
  Documented infection, %1.806.113.9577.9 (75.1–80.4)
     
Omicron study in adolescents
 Follow-up    
  Total follow-up, n of person-weeks772 1761 113 5611 885 736
  Median (Q1–Q3)42 (30–45)37 (22–44)39 (25–45)
 Absolute risk, %    
  Documented infection1.828.175.7785.5 (83.8–87.1)
  Mild COVID-190.432.071.4587.0 (83.5–89.8)
  Moderate or severe COVID-190.150.880.6084.8 (77.3–89.9)
  ICU admission with COVID-19≤0.020.14≤0.1091.5 (69.5–97.6)
  Cardiac complication≤0.020.05≤0.040.10 (0.02–0.57)*
 Age 12–15 y    
  Total follow-up, n of person-weeks524 053654 3151 178 368
  Documented infection, %1.938.245.6785.8 (83.6–87.7)
 Age 16–21 y    
  Total follow-up, n of person-weeks248 123459 246707 368
  Documented infection, %1.608.075.9385.9 (82.7–88.5)
ICU = intensive care unit; Q = quartile.
* Relative risk in the vaccinated groups compared with the unvaccinated.

Vaccine Effectiveness: Severe Illness and Complications

During the Delta period, the vaccine was found to have high effectiveness against severe infections. The estimated relative risk of the vaccine on cardiac complications was 1.22 (CI, 0.34 to 4.35). The estimated vaccine effectiveness against the Omicron variant in children was 73.5% (CI, 69.2% to 77.1%) against mild COVID-19, 75.5% (CI, 69.0% to 81.0%) against moderate or severe COVID-19, and 84.9% (CI, 64.8% to 93.5%) against ICU admission with COVID-19. The estimated relative risk of the vaccine on cardiac complications was 0.28 (CI, 0.08 to 0.95). In the Omicron study in adolescents, the vaccine effectiveness was estimated to be 87.0% (CI, 83.5% to 89.8%) against mild COVID-19, 84.8% (CI, 77.3% to 89.9%) against moderate or severe COVID-19, and 91.5% (CI, 69.5% to 97.6%) against ICU admission with COVID-19. The estimated relative risk of the vaccine on cardiac complications was 0.10 (CI, 0.02 to 0.57) in this cohort.

Sensitivity Analyses and Addressing Misclassification Bias

Section 9 of the Supplement presents the vaccine effectiveness against various Omicron subvariants. The effectiveness against subvariants BA.1, BA.2, BA.4, and BA.5 aligns with our primary findings, whereas the vaccine effectiveness seems to be lower against the BQ.1, XBB, and subsequent subvariants. Section 11 of the Supplement presents the results for the dose-response analysis. Section 14 of the Supplement presents negative control experiments of 3 study cohorts using 40 negative control outcomes. After accounting for systematic error through calibration using negative control outcomes, our findings indicate a slight shift in point estimates accompanied by wider CIs. This suggests the presence of a minor degree of systematic error, as well as additional uncertainty characterized by the estimated distribution derived from the negative control outcomes. Section 19 of the Supplement summarizes the comparative results to a sequential target trial emulation not accounting for underreporting issues of vaccination, which indicated reasonably consistent findings.
Section 16 of the Supplement shows effectiveness estimated from the naive method and proposed CER method with different ranges of sensitivity of vaccination status captured by EHR. The comparison results indicated that the vaccine effectiveness was reasonably consistent across different sensitivity ranges, suggesting that our primary analysis was robust to changes in the range of sensitivity considered. Section 17 of the Supplement shows the comparative results to a fully Bayesian method indicating nearly identical results. Section 18 of the Supplement shows sensitivity analyses on differential misclassifications, which shows that the novel CER method corrects the bias even when the nondifferential misclassification assumption does not hold.

Discussion

We estimated the effectiveness of BNT162b2 vaccines for the prevention of documented SARS-CoV-2 infections and severe diseases in a national network of pediatric health systems in the United States for 3 study cohorts. During the period when the Delta variant was dominant, the BNT162b2 vaccine was associated with strong protection in adolescents, with effectiveness higher than 95%, and with little evidence of waning during the follow-up period. Our findings against the Delta infection among adolescents are consistent with vaccine efficacy seen in the BNT162b2 clinical trial involving adolescents aged between 12 and 15 years, which demonstrated vaccine efficacy of 100% (CI, 75.3% to 100%) against documented SARS-CoV-2 infection (2). Our estimates of vaccine effectiveness against severe diseases are consistent with a case–control study based on test-negative design, which found an effectiveness of 94% (CI, 90% to 96%) against hospitalization and 98% (CI, 93% to 99%) against ICU admission (7).
In our study, during the period when the Omicron variant was dominant, the estimated vaccine effectiveness was approximately 70% for children and 85% for adolescents. The estimated protection decreased by roughly 10% around 4 months from the first dose and slightly waned over time. Previous studies have shown vaccine effectiveness against Omicron infection, ranging from 20% to 80% among children and adolescents. An analysis using a test-negative design in Scotland during the Omicron period found a vaccine effectiveness of 81.2% (CI, 77.7% to 84.2%) for adolescents aged 12 to 15 years and 65.5% (CI, 56.0% to 73.0%) for those aged 16 to 17 years 2 to 5 weeks after full vaccination. This effectiveness decreased to 43.3% (CI, 30.0% to 54.2%) and 8.9% (CI, −19.1% to 30.3%) after 10 to 13 weeks, respectively (22). With a test-negative design, data from U.S. pharmacy-based, drive-through SARS-CoV-2 testing sites indicated that the estimated vaccine effectiveness for children aged 5 to 11 years was 60.1% within 2 to 4 weeks after the second dose and declined to 28.9% during the second month after vaccination. For adolescents aged 12 to 15 years, the effectiveness was 59.5% within 2 to 4 weeks after the second dose and decreased to 16.6% during the subsequent month (23). Another test-negative design analysis in the United States indicated that in children aged 5 to 11 years, the effectiveness was 59.9% (CI, 58.5% to 61.2%) at 1 month, 33.7% (CI, 32.6% to 34.8%) at 4 months, and 14.9% (CI, 12.3% to 17.5%) at 10 months after the first dose (24). A retrospective study among children aged 5 to 11 years in Italy found vaccine effectiveness to be 29.4% (CI, 28.5% to 30.2%) against documented infection and 41.1% (CI, 22.2% to 55.4%) against severe COVID-19 (8). A Singapore study found that in fully vaccinated children aged 5 to 11 years, vaccine effectiveness was 65.3% (CI, 62.0% to 68.3%) against polymerase chain reaction–confirmed SARS-CoV-2 infection and 82.7% (CI, 74.8% to 88.2%) against COVID-19–related hospitalization (23). In addition, we found that vaccine effectiveness is substantially lower against the later subvariants of Omicron, including BQ.1 and XBB. However, it remains unclear whether this is a true subvariant effect or evidence of further waning over time. Continued research is desirable to understand vaccine effectiveness on future subvariants and its potential waning effects.
This study did not identify a statistically significant elevated risk for cardiac complications among vaccinated adolescents during the Delta variant period, and it even demonstrated a lower risk in the vaccinated group during the Omicron variant period. This finding may seem unexpected given that cases of myocarditis and pericarditis after messenger RNA COVID-19 vaccines received substantial attention (26–28). However, it is essential to note that previous studies indicated a much higher risk for myocarditis or pericarditis after a documented SARS-CoV-2 infection in the pediatric population (29), with an article finding 36.8 times higher risk (CI, 25.0 to 48.6) in persons younger than 16 years after COVID-19 (30), and that myocarditis is a common symptom for patients diagnosed with MIS (31, 32). It is possible that cases of myocarditis, pericarditis, or MIS occurred after undocumented SARS-CoV-2 infections. For example, 31.5% of occurrences of MIS did not have SARS-CoV-2 infections documented in the PEDSnet EHR database. Also note that during this period, especially the Omicron period, positive tests may have been from at-home tests or otherwise outside the system. Further assessment of cardiac complications after vaccination and COVID-19 is warranted to help provide a more complete picture of risk or benefit during a changing pandemic.
Our study has several strengths. First, we used a national network of academic medical centers that covered a diverse cohort being more representative of the general pediatric population, provided a robust sample size, and allowed for multiple subgroup analyses and detection of rare outcomes. Second, the richness of these EHR data allowed us to investigate the effectiveness against infection of different levels of severity as well as adjust for a broad set of confounders. Third, we conducted the negative control outcome experiments to assess the potential residual bias due to unmeasured confounders and other potential sources of systematic bias in the data. These experiments revealed a small amount of systematic error but with excessive uncertainty across different negative control outcomes, leading to wider CIs of our estimated effectiveness that honestly reflect the effects of unmeasured confounding and other potential sources of residual biases (33). Finally, to the best of our knowledge, this is the first real-world effectiveness study evaluating COVID-19 vaccines against infection and severe outcomes that explicitly handle the underreporting in vaccination.
Our study also has several potential limitations. First, effectiveness was investigated against documented infection in a cohort without previous infection, whereas the potential inclusion of patients with undocumented infections exists. Nevertheless, if this potential lack of data is evenly distributed across treatment groups, it could potentially attenuate the true vaccine effectiveness, thereby making our analysis more conservative in its estimations. Our inclusion of previous negative COVID-19 tests as a confounder aims to balance the probability of testing between treatment groups, which could partially adjust for this factor. Moreover, the increasing availability of at-home rapid antigen testing kits over time could have further reduced the testing frequency captured by the EHR. However, persons with severe illness who test positive through home kits typically seek medical care and report their results to hospitals. This would lead to a better capture of severe infection in our database and more reliable vaccine effectiveness estimates. Baseline confounders were balanced between vaccinated and unvaccinated groups, which should adjust for between-cohort differences in exposure risks and risks for severe infections. Second, as in any observational study, assignment to the vaccine group was nonrandom and the validity of the results could be affected by unmeasured confounders. To evaluate the effect of unmeasured confounders and residual bias, we conducted negative control experiments that quantified the robustness of our results.
Third, in this study, patients who had received vaccinations before the start of the study period were excluded. Due to missing vaccine records, some patients who had previously been vaccinated may have still entered the cohort, particularly in the unvaccinated group. However, the CER method used in this study adjusted for potential bias resulting from unrecorded vaccinated patients, which could also reduce the bias resulting from this issue. Finally, in the Omicron study involving adolescents, the cohort included adolescents who had their first vaccine after 1 January 2022. Because the use of BNT162b2 vaccines was authorized in adolescents aged 12 to 15 years on 10 May 2021, this cohort may represent a population with late vaccines, which reduces the generalizability of the findings.
Although this study provides evidence of a slight waning of vaccine effectiveness 4 months after the first dose against Omicron infection and the effectiveness is stabilized after 4 months, waning can be affected by vaccines during the follow-up period and other factors. Patients who received boosters during the follow-up period were not excluded from the study. A sensitivity analysis evaluating the durability of 2-dose vaccine effectiveness considering the third dose as censoring did not suggest a statistically significant different conclusion. A future study is warranted to investigate the effect of booster vaccination among children and adolescents. Furthermore, despite the recognized risk for myocarditis associated with COVID-19 vaccines in young men and teen boys, the study shows a lower relative risk for myocarditis, pericarditis, or MIS in vaccinated groups, which may be partially explained by reduced likelihood of infection during the study period (34).
In conclusion, this study involving national pediatric cohorts in the United States estimates moderate effectiveness of the BNT162b2 vaccine for preventing infection and severe diseases of the SARS-CoV-2 Omicron variant and high effectiveness against the Delta variant. The study shows a lower risk for cardiac complications among children and adolescents who were vaccinated during the Omicron period and no significant difference between vaccinated and unvaccinated groups during the Delta period. Our assessment of vaccine effectiveness across diverse outcomes underscores the vaccine's pivotal role in reducing SARS-CoV-2 transmission, minimizing COVID-19–related sick leaves, and alleviating economic burdens during the pandemic. This study substantially contributes to our knowledge of the BNT162b2 vaccine in the U.S. pediatric population using a rigorously designed CER method accounting for the incomplete capture of vaccination status in EHR data in the United States.

Supplemental Material

Supplementary Material
Study Protocol

References

1.
Flaxman S, Whittaker C, Semenova E, et al. Assessment of COVID-19 as the underlying cause of death among children and young people aged 0 to 19 years in the US. JAMA Netw Open. 2023;6:e2253590. [PMID: 36716029] doi: 10.1001/jamanetworkopen.2022.53590
2.
Frenck RW Jr, Klein NP, Kitchin N, et al; C4591001 Clinical Trial Group. Safety, immunogenicity, and efficacy of the BNT162b2 Covid-19 vaccine in adolescents. N Engl J Med. 2021;385:239-250. [PMID: 34043894] doi: 10.1056/NEJMoa2107456
3.
Walter EB, Talaat KR, Sabharwal C, et al; C4591007 Clinical Trial Group. Evaluation of the BNT162b2 Covid-19 vaccine in children 5 to 11 years of age. N Engl J Med. 2022;386:35-46. [PMID: 34752019] doi: 10.1056/NEJMoa2116298
4.
Chemaitelly H, AlMukdad S, Ayoub HH, et al. Covid-19 vaccine protection among children and adolescents in Qatar. N Engl J Med. 2022;387:1865-1876. [PMID: 36322837] doi: 10.1056/NEJMoa2210058
5.
Cohen-Stavi CJ, Magen O, Barda N, et al. BNT162b2 vaccine effectiveness against Omicron in children 5 to 11 years of age. N Engl J Med. 2022;387:227-236. [PMID: 35767475] doi: 10.1056/NEJMoa2205011
6.
Florentino PTV, Alves FJO, Cerqueira-Silva T, et al. Vaccine effectiveness of CoronaVac against COVID-19 among children in Brazil during the Omicron period. Nat Commun. 2022;13:4756. [PMID: 35963844] doi: 10.1038/s41467-022-32524-5
7.
Olson SM, Newhams MM, Halasa NB, et al; Overcoming Covid-19 Investigators. Effectiveness of BNT162b2 vaccine against critical Covid-19 in adolescents. N Engl J Med. 2022;386:713-723. [PMID: 35021004] doi: 10.1056/NEJMoa2117995
8.
Sacco C, Del Manso M, Mateo-Urdiales A, et al; Italian National COVID-19 Integrated Surveillance System and the Italian COVID-19 vaccines registry. Effectiveness of BNT162b2 vaccine against SARS-CoV-2 infection and severe COVID-19 in children aged 5-11 years in Italy: a retrospective analysis of January-April, 2022. Lancet. 2022;400:97-103. [PMID: 35780801] doi: 10.1016/S0140-6736(22)01185-0
9.
Khan FL, Nguyen JL, Singh TG, et al. Estimated BNT162b2 vaccine effectiveness against infection with Delta and Omicron variants among US children 5 to 11 years of age. JAMA Netw Open. 2022;5:e2246915. [PMID: 36515946] doi: 10.1001/jamanetworkopen.2022.46915
10.
PEDSnet. Accessed at https://pedsnet.org/ on 21 May 2023.
11.
Data_Models_Public/PEDSnet. Accessed at https://github.com/PEDSnet/Data_Models_Public/tree/master/PEDSnet on 21 May 2023.
12.
Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. Am J Epidemiol. 2016;183:758-764. [PMID: 26994063] doi: 10.1093/aje/kwv254
13.
Suchard MA, Schuemie MJ, Krumholz HM, et al. Comprehensive comparative effectiveness and safety of first-line antihypertensive drug classes: a systematic, multinational, large-scale analysis. Lancet. 2019;394:1816-1826. [PMID: 31668726] doi: 10.1016/S0140-6736(19)32317-7
14.
Simon TD, Cawthon ML, Popalisky J, et al; Center of Excellence on Quality of Care Measures for Children with Complex Needs (COE4CCN). Development and validation of the Pediatric Medical Complexity Algorithm (PMCA) version 2.0. Hosp Pediatr. 2017;7:373-377. [PMID: 28634166] doi: 10.1542/hpeds.2016-0173
15.
Forrest CB, Burrows EK, Mejias A, et al. Severity of acute COVID-19 in children <18 years old March 2020 to December 2021. Pediatrics. 2022;149:e2021055765. [PMID: 35322270] doi: 10.1542/peds.2021-055765
16.
Berger JO, Liseo B, Wolpert RL, et al. Integrated likelihood methods for eliminating nuisance parameters. Stat Sci. 1999;14:1-28. doi: 10.1214/ss/1009211804
17.
Huang J, Duan R, Hubbard RA, et al. PIE: a prior knowledge guided integrated likelihood estimation method for bias reduction in association studies using electronic health records data. J Am Med Inform Assoc. 2018;25:345-352. [PMID: 29206922] doi: 10.1093/jamia/ocx137
18.
Dickerman BA, Gerlovin H, Madenci AL, et al. Comparative effectiveness of BNT162b2 and mRNA-1273 vaccines in U.S. veterans. N Engl J Med. 2022;386:105-115. [PMID: 34942066] doi: 10.1056/NEJMoa2115463
19.
Schuemie MJ, Hripcsak G, Ryan PB, et al. Empirical confidence interval calibration for population-level effect estimation studies in observational healthcare data. Proc Natl Acad Sci U S A. 2018;115:2571-2577. [PMID: 29531023] doi: 10.1073/pnas.1708282114
20.
Schuemie MJ, Ryan PB, DuMouchel W, et al. Interpreting observational studies: why empirical calibration is needed to correct p-values. Stat Med. 2014;33:209-218. [PMID: 23900808] doi: 10.1002/sim.5925
21.
Carlin BP, Louis TA. Bayesian Methods for Data Analysis. 3rd ed. CRC Pr; 2008.
22.
Rudan I, Millington T, Antal K, et al. BNT162b2 COVID-19 vaccination uptake, safety, effectiveness and waning in children and young people aged 12-17 years in Scotland. Lancet Reg Health Eur. 2022;23:100513. [PMID: 36189425] doi: 10.1016/j.lanepe.2022.100513
23.
Fleming-Dutra KE, Britton A, Shang N, et al. Association of prior BNT162b2 COVID-19 vaccination with symptomatic SARS-CoV-2 infection in children and adolescents during Omicron predominance. JAMA. 2022;327:2210-2219. [PMID: 35560036] doi: 10.1001/jama.2022.7493
24.
Lin D-Y, Xu Y, Gu Y, et al. Effects of COVID-19 vaccination and previous SARS-CoV-2 infection on Omicron infection and severe outcomes in children under 12 years of age in the USA: an observational cohort study. Lancet Infect Dis. 2023;23:1257-1265. [PMID: 37336222] doi: 10.1016/S1473-3099(23)00272-4
25.
Tan SHX, Cook AR, Heng D, et al. Effectiveness of BNT162b2 vaccine against Omicron in children 5 to 11 years of age. N Engl J Med. 2022;387:525-532. [PMID: 35857701] doi: 10.1056/NEJMoa2203209
26.
Block JP, Boehmer TK, Forrest CB, et al. Cardiac complications after SARS-CoV-2 infection and mRNA COVID-19 vaccination—PCORnet, United States, January 2021–January 2022. MMWR Morb Mortal Wkly Rep. 2022;71:517-523. [PMID: 35389977] doi: 10.15585/mmwr.mm7114e
27.
Patone M, Mei XW, Handunnetthi L, et al. Risks of myocarditis, pericarditis, and cardiac arrhythmias associated with COVID-19 vaccination or SARS-CoV-2 infection. Nat Med. 2022;28:410-422. [PMID: 34907393] doi: 10.1038/s41591-021-01630-0
28.
Barda N, Dagan N, Ben-Shlomo Y, et al. Safety of the BNT162b2 mRNA Covid-19 vaccine in a nationwide setting. N Engl J Med. 2021;385:1078-1090. [PMID: 34432976] doi: 10.1056/NEJMoa2110475
29.
Chin SE, Bhavsar SM, Corson A, et al. Cardiac complications associated with COVID-19, MIS-C, and mRNA COVID-19 vaccination. Pediatr Cardiol. 2022;43:483-488. [PMID: 35258639] doi: 10.1007/s00246-022-02851-x
30.
Boehmer TK, Kompaniyets L, Lavery AM, et al. Association between COVID-19 and myocarditis using hospital-based administrative data – United States, March 2020-January 2021. MMWR Morb Mortal Wkly Rep. 2021;70:1228-1232. [PMID: 34473684] doi: 10.15585/mmwr.mm7035e5
31.
Dufort EM, Koumans EH, Chow EJ, et al; New York State and Centers for Disease Control and Prevention Multisystem Inflammatory Syndrome in Children Investigation Team. Multisystem inflammatory syndrome in children in New York State. N Engl J Med. 2020;383:347-358. [PMID: 32598830] doi: 10.1056/NEJMoa2021756
32.
Belot A, Antona D, Renolleau S, et al. SARS-CoV-2-related paediatric inflammatory multisystem syndrome, an epidemiological study, France, 1 March to 17 May 2020. Euro Surveill. 2020;25:2001010. [PMID: 32524957] doi: 10.2807/1560-7917.ES.2020.25.22.2001010
33.
Schuemie MJ, Soledad Cepede M, Suchard MA, et al. How confident are we about observational findings in health care: a benchmark study. Harv Data Sci Rev. 2020;2. [PMID: 33367288] doi: 10.1162/99608f92.147cc28e
34.
Block JP, Boehmer TK, Forrest CB, et al. Cardiac complications after SARS-CoV-2 infection and mRNA COVID-19 vaccination - PCORnet, United States, January 2021-January 2022. MMWR Morb Mortal Wkly Rep. 2022;71:517-523. [PMID: 35389977] doi: 10.15585/mmwr.mm7114e1

Comments

0 Comments
Sign In to Submit A Comment
Peter Yim10 January 2024
Unjustified assumptions about vaccination status

In this study, positive COVID vaccination status was determined from the EHR and is expected to be nearly 100% accurate. However, determination of negative COVID vaccination status was defined to be the absence of an EHR record of COVID vaccination. As such, the authors correctly observe that negative COVID vaccination status may be less than 100% accurate. False negative vaccination status occurs, for example, when a study subject receives COVID vaccination in a pharmacy not associated with one of the study institutions. The authors provide evidence that the effect of uncertainty of vaccination status on estimates of vaccine efficacy are limited provided that the "sensitivity" of the EHR to positive vaccination is 0.8 or greater for children and 0.7 - 0.9 for adolescents. However, the authors do not justify those ranges, only stating: "a range of possible sensitivities based on our prior study was prespecified for each study." No reference to that study is provided. These unjustified assumptions raise doubts about the study methodology and should limit confidence in the study conclusions.

Qiong Wu (1), Jiayi Tong, Jeffrey S. Morris (1), Christopher B. Forrest(2), and Yong Chen(1, 3) on the authors’ behalf30 January 2024
Author Response to Yim

Note that some of the EHR data partners have augmented their data quality by including the vaccine records from a Health Information Exchange (HIE) program specifically focused on vaccine registries. The range of possible sensitivities was prespecified based on a pilot study, which compared the vaccination rates estimated from vaccine data obtained from health systems against the Centers for Disease Control and Prevention (CDC) reports. Specifically, a reference vaccination rate was calculated by weighting the CDC's county-level vaccination statistics by the patients’ residential addresses. This comparison between the health system-derived vaccination rates and the CDC's reference rates allowed us to predefine a range of possible values of sensitivity. The research paper detailing this pilot study is being finalized and will be submitted soon. For the pediatric population who receive their care during the pandemic from the participating institutions (all of which provided vaccination programs to the communities as well as the primary care physician units). A sensitivity of lower than 70% would be found uncommon.

In addition, to evaluate the robustness of findings from the statistical methods we used to account for the underreporting, we have conducted sensitivity analyses using alternative methods, including the naive method (without adjusting for underreporting) and using different ranges of possible sensitivity values. Please refer to Section 16 of the Supplement for details.

Disclosures:

Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M23-1754

Peter Yim31 January 2024
Response to authors

One question that may arise is the extent to which the EHR data partners with the augmented data sets overlapped with the study of the accuracy of COVID vaccination status classification.

Information & Authors

Information

Published In

cover image Annals of Internal Medicine
Annals of Internal Medicine
Volume 177Number 2February 2024
Pages: 165 - 176

History

Published online: 9 January 2024
Published in issue: February 2024

Keywords

Authors

Affiliations

The Center for Health Analytics and Synthesis of Evidence (CHASE), Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania (Q.W., J.T., D.Z., J.C., Y.Lei, Y.W.)
The Center for Health Analytics and Synthesis of Evidence (CHASE), Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania (Q.W., J.T., D.Z., J.C., Y.Lei, Y.W.)
The Center for Health Analytics and Synthesis of Evidence (CHASE), The Graduate Group in Applied Mathematics and Computational Science, School of Arts and Sciences, University of Pennsylvania, Philadelphia, Pennsylvania (B.Z., Y.Lu, L.L., Y.S.)
The Center for Health Analytics and Synthesis of Evidence (CHASE), Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania (Q.W., J.T., D.Z., J.C., Y.Lei, Y.W.)
The Center for Health Analytics and Synthesis of Evidence (CHASE), Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania (Q.W., J.T., D.Z., J.C., Y.Lei, Y.W.)
The Center for Health Analytics and Synthesis of Evidence (CHASE), Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania (Q.W., J.T., D.Z., J.C., Y.Lei, Y.W.)
The Center for Health Analytics and Synthesis of Evidence (CHASE), The Graduate Group in Applied Mathematics and Computational Science, School of Arts and Sciences, University of Pennsylvania, Philadelphia, Pennsylvania (B.Z., Y.Lu, L.L., Y.S.)
The Center for Health Analytics and Synthesis of Evidence (CHASE), Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania (Q.W., J.T., D.Z., J.C., Y.Lei, Y.W.)
The Center for Health Analytics and Synthesis of Evidence (CHASE), The Graduate Group in Applied Mathematics and Computational Science, School of Arts and Sciences, University of Pennsylvania, Philadelphia, Pennsylvania (B.Z., Y.Lu, L.L., Y.S.)
The Center for Health Analytics and Synthesis of Evidence (CHASE), The Graduate Group in Applied Mathematics and Computational Science, School of Arts and Sciences, University of Pennsylvania, Philadelphia, Pennsylvania (B.Z., Y.Lu, L.L., Y.S.)
Jie Xu, PhD
Department of Health Outcomes Biomedical Informatics, University of Florida, Gainesville, Florida (J.X., J.B., T.L.)
L. Charles Bailey, MD, PhD https://orcid.org/0000-0002-8967-0662
Applied Clinical Research Center, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania (L.C.B., K.H., H.R., C.B.F.)
Department of Health Outcomes Biomedical Informatics, University of Florida, Gainesville, Florida (J.X., J.B., T.L.)
Dimitri A. Christakis, MD, MPH
Center for Child Health, Behavior, and Development, Seattle Children’s Research Institute, Seattle, Washington (D.A.C.)
Megan L. Fitzgerald, PhD
Department of Medicine, Grossman School of Medicine, New York University, New York, New York (M.L.F.)
Kathryn Hirabayashi, MPH https://orcid.org/0000-0003-0647-5579
Applied Clinical Research Center, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania (L.C.B., K.H., H.R., C.B.F.)
Division of Pediatric Infectious Diseases, Ann & Robert H. Lurie Children’s Hospital of Chicago, Chicago, Illinois (R.J.)
Alka Khaitan, MD
Department of Pediatrics, Ryan White Center for Pediatric Infectious Diseases and Global Health, Indiana University School of Medicine, Indianapolis, Indiana (A.K.)
Tianchen Lyu, MS
Department of Health Outcomes Biomedical Informatics, University of Florida, Gainesville, Florida (J.X., J.B., T.L.)
Suchitra Rao, MBBS, MSCS https://orcid.org/0000-0002-0334-6301
Department of Pediatrics, University of Colorado School of Medicine and Children’s Hospital Colorado, Aurora, Colorado (S.R.)
Hanieh Razzaghi, PhD, MPH https://orcid.org/0000-0001-5001-1590
Applied Clinical Research Center, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania (L.C.B., K.H., H.R., C.B.F.)
Hayden T. Schwenk, MD, MPH https://orcid.org/0000-0002-8164-8655
Department of Pediatrics, Stanford School of Medicine, Stanford, California (H.T.S.)
Department of Population Health Sciences, Weill Cornell Medicine, New York, New York (F.W.)
Margot I. Gage Witvliet, PhD
Department of Sociology, Social Work and Criminal Justice, Lamar University, Beaumont, Texas (M.I.G.W.)
Eric J. Tchetgen Tchetgen, PhD
Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania (E.J.T.T., J.S.M.)
Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania (E.J.T.T., J.S.M.)
Christopher B. Forrest, MD, PhD https://orcid.org/0000-0003-1252-068X
Applied Clinical Research Center, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania (L.C.B., K.H., H.R., C.B.F.)
The Center for Health Analytics and Synthesis of Evidence (CHASE), Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, and The Graduate Group in Applied Mathematics and Computational Science, School of Arts and Sciences, University of Pennsylvania, Leonard Davis Institute of Health Economics, Penn Medicine Center for Evidence-based Practice (CEP), and Penn Institute for Biomedical Informatics (IBI), Philadelphia, Pennsylvania (Y.C.).
Note: Dr. Yong Chen is a statistical consultant with Annals.
Disclaimer: All statements in this report, including its findings and conclusions, are solely those of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology Committee.
Financial Support: By the National Institutes of Health (OT2HL161847-01, 1R01LM012607, 1R01AI130460, 1R01AG073435, 1R56AG074604, 1R01LM013519, 1R56AG069880, 1R01AG077820, 1U01TR003709) and Patient-Centered Outcomes Research Institute (PCORI) Project Program Awards (ME-2019C3-18315 and ME-2018C3-14899).
Reproducible Research Statement: Study protocol: Available at Annals.org. Statistical code: Available to interested readers by contacting Dr. Chen (e-mail, ychen123@pennmedicine.upenn.edu). Data set: The data from this study is not available due to patient privacy regulations. For further inquiries, please contact the corresponding author.
Corresponding Authors: Yong Chen, PhD, The Center for Health Analytics and Synthesis of Evidence (CHASE), Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Blockley Hall 602, 423 Guardian Drive, Philadelphia, PA 19104; e-mail, ychen123@pennmedicine.upenn.edu; Christopher B. Forrest, MD, PhD, Applied Clinical Research Center, Children’s Hospital of Philadelphia, 2716 South Street, Suite 11-473, Philadelphia, PA 19146; e-mail, forrestc@chop.edu; and Jeffrey S. Morris, PhD, Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, 600 Blockley Hall, 423 Guardian Drive, Philadelphia PA 19104-6021; e-mail, jeffrey.morris@pennmedicine.upenn.edu.
Correction: This article was amended on 19 March 2024 to correct errors in Figure 1 and to revise the Supplement. A correction has been published (doi:10.7326/L24-0049).
Previous Posting: This manuscript was posted as a preprint on medRxiv on 21 June 2023. doi:10.1101/2023.06.16.23291515
Author Contributions: Conception and design: Y. Chen, C.B. Forrest, J.S. Morris, J. Tong, Q. Wu.
Analysis and interpretation of the data: L.C. Bailey, J. Chen, Y. Chen, C.B. Forrest, R. Jhaveri, Y. Lei, L. Li, Y. Lu, J.S. Morris, H. Razzaghi, Y. Shen, J. Tong, Y. Wang, Q. Wu, B. Zhang, D. Zhang.
Drafting of the article: Y. Chen, C.B. Forrest, J.S. Morris, S. Rao, J. Tong, Q. Wu.
Critical revision of the article for important intellectual content: L.C. Bailey, J. Bian, J. Chen, Y. Chen, D.A. Christaki, M.L. Fitzgerald, C.B. Forrest, K. Hirabayashi, R. Jhaveri, A. Khaitan, Y. Lei, L. Li, Y. Lu, T. Lyu, J.S. Morris, S. Rao, H. Razzaghi, H.T. Schwenk, Y. Shen, E.J. Tchetgen Tchetgen, J. Tong, F. Wang, Y. Wang, M.I. Gage Witvliet, Q. Wu, J. Xu, B. Zhang, D. Zhang.
Final approval of the article: L.C. Bailey, J. Bian, J. Chen, Y. Chen, D.A. Christakis, M.L. Fitzgerald, C.B. Forrest, M.I. Gage Witvliet, K. Hirabayashi, R. Jhaveri, A. Khaitan, Y. Lei, L. Li, Y. Lu, T. Lyu, J.S. Morris, S. Rao, H. Razzaghi, H.T. Schwenk, Y. Shen, E.J. Tchetgen Tchetgen, J. Tong, F. Wang, Y. Wang, Q. Wu, J. Xu, B. Zhang, D. Zhang.
Statistical expertise: J. Chen, Y. Chen, Y. Lei, L. Li, Y. Lu, J.S. Morris, Y. Shen, E.J. Tchetgen Tchetgen, J. Tong, Y. Wang, Q. Wu, B. Zhang, D. Zhang.
Obtaining of funding: L.C. Bailey, Y. Chen, C.B. Forrest.
Administrative, technical, or logistic support: Y. Chen, L.C. Bailey, C.B. Forrest, J. Tong, Q. Wu.
Collection and assembly of data: L.C. Bailey, D.A. Christaki, C.B. Forrest, K. Hirabayashi, R. Jhaveri, A. Khaitan, S. Rao, H. Razzaghi, H.T. Schwenk.
This article was published at Annals.org on 9 January 2024.
*
Dr. Wu and Ms. Tong are co-first authors.
Drs. Morris, Forrest, and Chen are co-senior authors.

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. For an editable text file, please select Medlars format which will download as a .txt file. Simply select your manager software from the list below and click Download.

For more information or tips please see 'Downloading to a citation manager' in the Help menu.

Format





Download article citation data for:
Qiong Wu, Jiayi Tong, Bingyu Zhang, et al. Real-World Effectiveness of BNT162b2 Against Infection and Severe Diseases in Children and Adolescents. Ann Intern Med.2024;177:165-176. [Epub 9 January 2024]. doi:10.7326/M23-1754

View More

Get Access

Login Options:
Purchase

You will be redirected to acponline.org to sign-in to Annals to complete your purchase.

Access to EPUBs and PDFs for FREE Annals content requires users to be registered and logged in. A subscription is not required. You can create a free account below or from the following link. You will be redirected to acponline.org to create an account that will provide access to Annals. If you are accessing the Free Annals content via your institution's access, registration is not required.

Create your Free Account

You will be redirected to acponline.org to create an account that will provide access to Annals.

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share on social media