Skip to main content

Effectiveness of bivalent HPV vaccination against genital HPV DNA-positivity of a catch-up campaign at age 13–16 years compared to routine vaccination at age 12 years: a biennial repeated cross-sectional study

Abstract

Background

The Netherlands is one of few countries worldwide which has used the bivalent HPV vaccine for girls-only for over a decade. This allows assessment of vaccine effectiveness (VE) against female genital HPV DNA-positivity of this vaccine in an observational post-licencing real-world setting. Additionally, it is unclear whether catch-up vaccination campaigns result in similar VE as routine vaccination. Therefore, type-specific and grouped VE were assessed and compared for women who had been eligible for catch-up vaccination at 13–16 years with those who had been eligible for routine vaccination at 12 years.

Methods

PASSYON is a Dutch biennial repeated cross-sectional (2011–2021) study among sexual health clinic clients aged 16–24 years old. Women provided self-collected vaginal samples, questionnaires on demographics and sexual behaviour were administered, and women self-reported HPV vaccination status. Samples were analysed using a PCR-based assay (SPF10-LiPA25). Type-specific and grouped VE estimates, adjusted with propensity score stratification, were assessed against genital positivity for 14 HPV types. VE for targeted and non-targeted genotypes were compared between women who had been eligible for the catch-up and those who had been eligible for routine vaccination.

Results

The study included 4488 female participants who had been eligible for HPV vaccination and provided genital swabs (1561 eligible for catch-up, 2927 for routine vaccination). Very high VE against genital HPV-16 and HPV-18 was observed (resp. 93.5% and 89.5%) and significant cross-protection against six other genotypes (HPV-31/33/35/45/52/58), varying from 18.0% (HPV-52) to 79.6% (HPV-45). VE estimates were comparable between women who had been eligible for the catch-up campaign and those eligible for routine vaccination: VE HPV-16/HPV-18: 92.2% (95%CI: 87.9–94.9) vs. 91.8% (95%CI: 86.0–95.2).

Conclusions

In real-world settings, the VE of bivalent vaccine is high against targeted genotypes, with cross-protection against 6 other genotypes. Catch-up campaigns up to age 16 years can be as effective as routine vaccination at age 12, although it is recommendable to provide HPV vaccination at an age at which most are likely not sexually active yet. This may inform countries considering catch-up campaigns when introducing or extending the use of HPV vaccination within their national immunisation programmes.

Peer Review reports

Background

Persistent human papillomavirus (HPV) infections can lead to anogenital warts (AGWs) or cancer [1]. Approximately 5% of new cancer cases worldwide are attributable to HPV, with approximately 3–3.5% caused by genotype HPV-16 alone [2]. Several prophylactic vaccines have been proven to be effective against persistent HPV infections and CIN2 + and are available across the world. All protect against the two most oncogenic genotypes (HPV-16/18), responsible for about 71% of cervical cancers [3], and some vaccines offer additional protection against two low-risk HPV (lrHPV) genotypes and to five additional high-risk HPV (hrHPV) genotypes. The bivalent vaccine Cervarix® is targeted against HPV-16 and HPV-18 [4] and has shown cross-protection effectiveness against other hrHPV genotypes [5].

Post-licencing observational vaccine effectiveness (VE) studies are important after the introduction of a vaccine, to understand its effectiveness in real-world settings. To date, several observational studies have been conducted to assess the VE for the bivalent vaccine [5]. However, some of those studies did not assess VE type-specifically [6, 7] or were from countries that offered more than one vaccine type at the same time [8, 9]. Herd-immunity provided by another HPV vaccine may influence observed VE of the bivalent vaccine. Other observational studies (including from our own group) used mostly data of women vaccinated within a catch-up campaign [10,11,12,13,14]. In catch-up campaigns, it is more likely that some girls have become sexually active before receiving a vaccine than in routine vaccination; therefore, VE estimates based on catch-up data might not represent VE in routine vaccination settings. Studies comparing catch-up campaign VE with routine vaccination VE of the bivalent HPV vaccine are scarce and are mostly modelling studies rather than observational studies [15,16,17]. As more countries start HPV vaccination or start gender-neutral vaccination, mostly introduced in combination with catch-up campaigns, this comparison could inform catch-up strategies. Real-world comparisons of effectiveness between catch-up campaigns and routine vaccination are therefore needed.

Since the introduction of HPV vaccination in the National Immunisation Programme (NIP) of the Netherlands, the bivalent HPV vaccine has exclusively been used. This started with a catch-up campaign in 2009 targeting girls born in 1993–1996 [18] and became part of routine vaccination in 2010, starting with birth cohort 1997. The Netherlands is one of the few countries which started with bivalent HPV vaccination and has continued doing so for girls-only until 2022. This makes the Dutch setting unique and suitable for assessing VE of the bivalent vaccine in real-world post-licencing settings. It additionally provides an opportunity to compare VE of the bivalent HPV vaccine between birth cohorts vaccinated in the catch-up campaign and those vaccinated in the routine vaccination programme.

In this study, we aimed to (1) assess the type-specific and grouped VE of the bivalent HPV vaccine against female genital HPV DNA-positivity during the 12-year girls-only vaccination period. Additionally, (2) we aimed to compare VE between women offered vaccination during the catch-up campaign to those offered vaccination during routine HPV vaccination.

Methods

Study setting

The PApillomavirus Surveillance among Sti clinic YOungsters in the Netherlands (PASSYON) study is a biennial repeated cross-sectional study among young adult sexual health clinic (SHC) clients [19]. During each study round, for approximately 2 months, SHC clients from 10 to 14 SHCs aged 16–24 years were asked to participate. After signing a written consent, participants were asked to self-collect a genital swab sample for HPV testing. Women were instructed to collect the cervicovaginal swab sample, by inserting a swab (Copan Diagnostics, Italy) about 4 cm into the vagina, until resistance was felt, and to turn the swab around along the walls of the vagina for about 15 s. All swabs were placed in a tube with 1 ml universal transport medium (Copan Diagnostics, Italy) immediately after the swab was taken.

Additionally, participants were asked to complete a written questionnaire on demographics and sexual behaviour (such as number of sexual partners lifetime and in preceding 6 months, age of sexual debut, and history of STI). Participants also reported whether they had been vaccinated against HPV. Finally, diagnoses from routine STI testing were available for analyses.

For the current analyses, data from the second through seventh study round of PASSYON (2011–2021) were used, as these study rounds cover the era in which only girls were invited for HPV vaccination. Only women who had been eligible for HPV vaccination were included, based on their year of birth. Having been eligible for HPV vaccination was defined as being born in 1993 or later, as this was the first birth cohort which had been invited for the catch-up vaccination. Other inclusion criteria were provision of a genital swab and having reported HPV vaccination status in the questionnaire.

This study was approved by Medical Ethics Committee of the University of Utrecht, the Netherlands (reference number: 08/397).

Laboratory methods

All cervicovaginal swabs were stored at − 20 °C until processing. Samples were analysed within 12 months of collection, to ensure the quality of the sample. After thawing and vortexing, 200 μl of the material was used for DNA extraction using the MagnaPure platform (Total Nucleic Acid Isolation Kit, Roche, the Netherlands). Total DNA was eluted in 100 μl elution buffer, and 10 μl was used to amplify HPV-DNA with the SPF10 primer set. HPV-specific amplicons were detected using enzyme-linked immunoassay (HPV-DEIA, DDL Diagnostic Laboratory, the Netherlands). Positive samples were subsequently genotyped with the line probe assay (HPV-LiPA, DDL Diagnostic Laboratory, the Netherlands), which is able to identify 25 genotypes (6, 11, 16, 18, 31, 33–35, 39, 40, 42–45, 51–54, 56, 58, 59, 66, 68, 70, 74).

Statistical analyses

The current report is an updated and elaborated follow-up of previous VE reports with data of the PASSYON study [11, 20,21,22].

First, study population characteristics were described and compared by vaccination status. Type-specific and grouped genital HPV DNA-positivity prevalence was calculated for unvaccinated and vaccinated (≥ 1 dose) women, separately. Characteristics were also compared between women of birth cohort 1993–1996 (these women had been eligible for catch-up vaccination) and women of birth cohort 1997 and later (these women had been eligible for routine vaccination). Finally, characteristics were compared between women who were excluded from the analyses because of missing genital sample or vaccination status and those included in the analyses.

Type-specific and grouped VE estimates were calculated using the relative risk (RR) for genital HPV DNA-positivity comparing vaccinated and unvaccinated women. VE was calculated as 1 minus the RR times 100%. First, unadjusted, crude VE estimates were calculated. Next, VE was calculated based on propensity score (PS) adjusted (see further) RRs. Type-specific estimates were made for the high-risk genotypes 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, and 59 and for the low-risk genotypes 6 and 11. We also assessed prevalences of five different HPV groupings, defined as being DNA-positive for at least one of (1) the bivalent vaccine genotypes (HPV-16/18), (2) the cross-protected genotypes as observed previously [23] and widely reported in other studies (HPV-31/33/45), (3) the cross-protected types found in the current analysis, (4) the hrHPV genotypes covered by the nonavalent vaccine (HPV-16/18/31/33/45/52/58), and finally (5) all hrHPV genotypes (HPV-16/18/31/33/35/39/45/51/52/56/58/59). These groupings were also used for other estimates in this study.

To adjust for differences in study population characteristics over the study rounds, and differences between unvaccinated and vaccinated women, PS stratification (also called subclassification) was used [24,25,26]. PS was calculated as the propensity of having been vaccinated based on study round, geographical location of the SHC, demographics (age, migration background and education), sexual behaviour (age at sexual debut, currently having a steady partner, history of hormonal anticonception use, number of sexual partners lifetime and in preceding 6 months), and having a history of other STIs. For each VE calculation, strata were created of the total study population based on quintiles of the PS distribution. Having defined strata, we assessed balance in co-variates in each stratum between the vaccinated and unvaccinated groups. If for each covariate the standardised mean difference between vaccinated and unvaccinated was smaller than 0.10, the balance was considered adequate. If this was not reached, a smaller number of strata was used. Within each stratum, the RR risk was established using generalised linear regression models with binomial distribution and log link function [27]. In these models, HPV DNA-positivity was the dependent variable and self-reported vaccination status the independent variable. The overall effect was obtained by pooling the stratum-specific RRs using stratum specific weights, which are based on the total number of included participants in that stratum.

Stratified analyses

Four stratified VE analyses were conducted. Stratified analyses were conducted by (1) birth cohort, where we distinguished women who had been eligible for the catch-up campaign (birth cohort 1993–1996, 13–16 years old at eligibility) from women who had been eligible for routine vaccination in the NIP (birth cohort ≥ 1997, 12 years old at eligibility).

For women who had been eligible for the catch-up campaign, an additional stratified analysis was conducted: (2) we compared the VE for women who were, to women who were not, sexually active before, or at the same age the vaccine was offered, indicating being potentially sexually active around the time of vaccination. This was based on self-reported age at sexual debut and the age women had been eligible for vaccination based on their year of birth.

For assessing generalisability of the VE estimates, VE analyses were conducted (3) stratified by number of sexual partners in the preceding 6 months (two categories, based on median) and (4) by years sexually active (age at inclusion minus age at sexual debut; three categories, based on tertiles). This was done as our study population might have been more sexually active than the general population.

To test whether VE across the assessed strata differed, an interaction term between the stratifying variable and vaccination variable was added to the main VE model, to test for effect modification. All stratified analyses were done type-specifically for HPV-16 and HPV-18 and for all HPV groupings as mentioned above.

For each stratum, propensity scores were calculated and PS stratification was executed for that specific stratum. The stratum variable was not used in calculating the PS for the strata. For example, in the stratified analysis for catch-up campaign vs. routine vaccination, variable birth cohort was not used in the PS calculation.

Sensitivity analyses

Two sensitivity analyses were conducted. First, instead of adjusting for confounding using PS stratification, we adjusted VE estimated with multivariable logistic regression, adjusting for variables associated with vaccination (being study round, age, education, migration background, age at sexual debut, history of STI).

All women offered vaccination during the catch-up campaign (birth cohort 1993–1996) were offered a three-dose schedule (0, 1, and 6 months). All women of birth cohort 1997–2000 were offered HPV vaccination at age 12 years, in the same three-dose schedule. Women of birth cohort 2001 and later were offered vaccination in a two-dose schedule (0 and 6 months) [18]. In order to eliminate any potential differences in VE due to different dosing schedules, we performed a sensitivity analysis restricted to women who had been eligible for 3 doses.

All analyses were performed using R RStudio (Version 1.3.959) with help of the cobalt and survey packages. For all analyses, statistical significance was defined as p < 0.05.

Results

Study population

In the PASSYON study, a total 13,535 adolescents participated, of whom 9099 were female. Of those, 5006 women had been eligible for vaccination. As 518 women had not indicated their vaccination status and/or did not provide a genital swab (422 women without reporting vaccination status, 96 without genital swab, eight missed both), these women were excluded (Additional file 1: Table S1). Therefore, 4488 women were eligible for the current analysis.

The majority participated in the three study rounds 2017, 2019, and 2021 (75.7%). 34.8% reported to be unvaccinated (n = 1561) and 65.2% to be vaccinated (n = 2927) (Table 1). Eligible for the catch-up campaign (birth cohort 1993–1996) in our study population had been 2345 women (54.2%) and eligible for routine HPV vaccination within the NIP (birth cohort ≥ 1997) had been 1988 women (45.8%) (Table 2). The median number of sex partners in the preceding 6 months was 2 (IQR 1–4), 29.9% had a history of an STI (n = 1332), and 17.4% had a current Chlamydia trachomatis infection (n = 775). More information on study population characteristics can be found in Tables 1 and 2.

Table 1 Characteristics of the female study population of the PASSYON study who had been eligible for HPV vaccination in the Netherlands, by self-reported vaccination status, the Netherlands, 2011–2021
Table 2 Characteristics of the female study population of the PASSYON study who had been eligible for HPV vaccination in the Netherlands, by eligibility for catch-up or routine HPV vaccination based on birth cohort, the Netherlands, 2011–2021

Overall, 6.7% of the participants tested positive for HPV-16 and/or HPV-18 (n = 299) and 56.4% for at least one hrHPV genotype (n = 2532) (Table 1). The type-specific and grouped HPV prevalences for the twelve hrHPV genotypes and the lrHPV genotypes 6/11 by vaccination status can be found in Fig. 1 and Additional file 1: Table S2–S3.

Fig. 1
figure 1

Type-specific and grouped prevalences and 95% confidence intervals of genital HPV DNA-positivity among female participants who had been eligible for HPV vaccination in the Netherlands from the PASSYON study (2011–2021), by vaccination status. 2vHPV is DNA-positive for at least one of HPV-16/HPV-18; crosstypes 1 is DNA-positive for at least one of HPV-31/HPV-33/HPV-45; crosstypes 2 is DNA-positive for at least one of HPV-31/HPV-33/HPV-35/HPV-45/HPV-52/HPV-58; 9vhrHPV is DNA positive for at least one of HPV-16/HPV-18/HPV-31/HPV-33/HPV-45/HPV-52/HPV-58; hrHPV is DNA-positive for at least one of 16/18/31/33/35/39/45/51/52/56/58/59. Abbreviations: 2vHPV, bivalent HPV; 9vhrHPV, nonavalent high-risk HPV; HPV, human papillomavirus; hrHPV, high-risk HPV

There were important differences between women who had been vaccinated and those who had not been vaccinated (Table 1). Compared to unvaccinated women, vaccinated women were more likely to have no migration background, to have done (pre)university education, to have an older age at sexual debut, and to have no history of STI. Compared to women who had been eligible for routine vaccination, those who had been eligible for catch-up HPV vaccination were more likely to be of older age at inclusion, to be unvaccinated, to have an older age at sexual debut, and higher number of sexual partners lifetime and preceding 6 months (Table 2).

VE estimates

Unadjusted VE against at least one of the bivalent genotypes was 91.2% (95% confidence interval (CI): (87.9; 93.7). VE against HPV-16 was 92.7% (95%CI: (88.9; 95.4), and 88.5% (95%CI: (82.0; 93.0) against HPV-18. More details on other type-specific and grouped VE can be found in Additional file 1: Table S4.

The PS stratification adjusted VE against genital HPV-16 DNA positivity was 93.5% (95%CI: 89.8; 95.9); for HPV-18, this was 89.5 (95%CI: 83.0; 93.6) (Fig. 2). Next to the vaccine targeted genotypes, statistically significant protection was found against HPV-31 (63.9%; 95%CI: 54.1; 71.6), HPV-33 (51.5%; 95%CI: 32.6; 65.0), HPV-35 (48.5%; 95%CI: 23.0; 65.5), HPV-45 (79.6%; 95%CI: 64.7; 88.2), HPV-52 (18.0%; 95%CI: 6.2; 28.4), and HPV-58 (31.4%; 95%CI: 8.7; 48.5) (Fig. 2). Significant negative VE was found against HPV-59: − 42.6% (95%CI: − 92.9; − 5.4).

Fig. 2
figure 2

Type-specific and grouped vaccine effectiveness of the bivalent HPV vaccine against genital HPV DNA-positivity for female participants who had been eligible for HPV vaccination included in the PASSYON study, the Netherlands, 2011–2021. 2vHPV is DNA-positive for at least one of HPV-16/HPV-18; crosstypes 1 is DNA-positive for at least one of HPV-31/HPV-33/HPV-45; crosstypes 2 is DNA-positive for at least one of HPV-31/HPV-33/HPV-35/HPV-45/HPV-52/HPV-58; 9vhrHPV is DNA positive for at least one of HPV-16/HPV-18/HPV-31/HPV-33/HPV-45/HPV-52/HPV-58; hrHPV is DNA-positive for at least one of 16/18/31/33/35/39/45/51/52/56/58/59. Abbreviations: 2vHPV, bivalent HPV; 9vhrHPV, nonavalent high-risk HPV; CI, confidence interval; HPV, human papillomavirus; hrHPV, high-risk HPV; VE, vaccine effectiveness

The VE against grouped genital DNA positivity of at least one of the bivalent genotypes was 92.1% (95%CI: 88.9; 94.3). Grouped VE against any hrHPV was 12.8% (95%CI: 7.9; 17.4). More grouped estimates can be found in Fig. 2.

Stratified VE estimates

No statistically significant differences were found in VE estimates of the birth cohort that had been eligible for the catch-up campaign (birth cohort 1993–1996, eligible at age 13–16 years) and VE estimates of the birth cohort which had been eligible for routine HPV vaccination within the NIP (birth cohort ≥ 1997, eligible at age 12) (Fig. 3A, Additional file 1: Table S5). VE against genital HPV-16 was 92.1% (95%CI: 86.6; 95.4) for the birth cohort of the catch-up campaign and 94.9% (95%CI: 87.8; 97.8) for the birth cohort of routine vaccination (p for interaction = 0.54). For HPV-18, this was 92.2% (95%CI: 84.1; 96.2) and 86.8 (95%CI: 74.0; 93.3) (p for interaction = 0.37). More details can be found in Fig. 3A and Additional file 1: Table S5.

Fig. 3
figure 3

Stratified analyses of type-specific and grouped vaccine effectiveness of the bivalent HPV vaccine against genital HPV DNA-positivity for female participants who had been eligible for HPV vaccination within the catch-up campaign and routine vaccination included in the PASSYON study, the Netherlands, 2011–2021. A presents stratified VE analyses by birth cohort, where birth cohort 1993–1996 had been eligible for the catch-up campaign and birth cohort ≥ 1997 for routine vaccination. B presents stratified analyses by having been sexually active before or at the same age as having been eligibility for HPV vaccination for women of the catch-up cohort. 2vHPV is DNA-positive for at least one of HPV-16/HPV-18; crosstypes 1 is DNA-positive for at least one of HPV-31/HPV-33/HPV-45; crosstypes 2 is DNA-positive for at least one of HPV-31/HPV-33/HPV-35/HPV-45/HPV-52/HPV-58; 9vhrHPV is DNA positive for at least one of HPV-16/HPV-18/HPV-31/HPV-33/HPV-45/HPV-52/HPV-58; hrHPV is DNA-positive for at least one of 16/18/31/33/35/39/45/51/52/56/58/59. Abbreviations: 2vHPV, bivalent HPV; 9vhrHPV, nonavalent high-risk HPV; HPV, human papillomavirus; hrHPV, high-risk HPV; VE, vaccine effectiveness

For women who had been eligible for catch-up vaccination, an additional stratified analyses were conducted. These were stratified analyses by age of sexual debut relative to the year they had been offered HPV vaccination (i.e. by having been potentially sexually active at time of vaccination eligibility). VE estimates were slightly lower for five of the seven VE estimates for women who were sexually active before or in the same year as being eligible for vaccination, although none of these differences were statistically significant (Fig. 3B, Additional file 1: Table S6). The VE against genital HPV-16 and/or HPV-18 was 94.5% (95%CI: 89.9; 97.0) for women who were not sexually active before having been eligible for vaccination and 87.5% (95%CI: 76.1; 93.4) for those who were sexually active before or at the same age as being eligible for vaccination (p for interaction 0.07).

In the VE analyses stratified by number of sex partners in the preceding 6 months, and by years sexually active, also no significant differences were observed in type-specific and grouped VE between the strata (Additional file 1: Fig. S1).

Sensitivity analyses

In the sensitivity analyses, no differences were found in VE estimates between the two methods used for adjustment for differences in characteristic between vaccinated and unvaccinated women and differences in study population over the study rounds. Type-specific and grouped VE were comparable for unadjusted estimates, estimates adjustments with multivariable logistic regression, and estimates with PS stratification (Additional file 1: Fig. S2).

Four thousand two women were included in the sensitivity analysis restricted to women who were offered a 3-dose vaccination schedule (all born in 1993–2000), and 331 were excluded as they were born in 2001 or later. No statistically significant differences were observed between VE of the catch-up cohort and the routine vaccination cohort (Additional file 1: Fig. S3).

Discussion

This study demonstrated a very high VE of the bivalent HPV vaccine against female genital HPV positivity for the targeted HPV genotypes HPV-16 and HPV-18 during the girls-only HPV vaccination period in the Netherlands. Significant cross-protection was found against 6 other hrHPV genotypes, being HPV-31, HPV-33, HPV-35, HPV-45, HPV-52, and HPV-58. No significant differences were found in VE between women in our study who had been eligible for the 2009 catch-up campaign for girls aged 13–16 and women who had been eligible for routine vaccination at age 12 as part of the Dutch NIP.

A strength of our study was the large study population, of whom 4488 were included in the main analyses. This is substantially more than the preceding VE analysis based on PASSYON data of 1087 women [11]. Additionally, data from multiple geographical locations were used, resulting in a high coverage and representativity for the country. Furthermore, the Dutch setting is uniquely suited to assess VE of the bivalent vaccine post-licencing, as the Netherlands is one of the few high-income countries that uses only the bivalent vaccine [28] and has done so since the start of HPV vaccination within the NIP. By using SCH clients as our study population, which are individuals having experienced high HPV exposure shortly before the time of their visit, our study had power to estimate effectiveness against HPV types that are not so prevalent in the general population. As women attending SHCs are likely more sexually active than their peers not visiting SHCs, it could be suggested that this limits the generalizability of the results. The stratified analyses showed that VE estimates were comparable across strata of sexual behaviour, which favours generalizability. Lastly, the study population was aged 16–24 years old, the age span in which the incidence of female genital HPV infections is at its peak [29].

Some limitations should be mentioned. First, several variables were self-reported, such as sexual behaviour and vaccination status, possibly leading to recall or social desirability bias. However, sexual behaviour is hard to measure otherwise, and previous analyses from the PASSYON study showed that self-reported vaccination status of women correlated very well with high antibody levels [11]. Second, vaccination status was not reported, or genital swab not provided by 518 women, who were otherwise eligible for analyses. This might have led to selection bias. Based on variables such as age, age at sexual debut and number of sexual partners, there is no indication to expect this. Third, having SHC visitors in the study group might not impact the type-specific VE estimates, but the grouped estimate might be diluted. The high prevalence of non-protected HPV types in both the numerator and denominator may lead to an overestimation of the relative risks, which could result in an underestimation of the VE. Fourth, to minimise confounding caused by differences in study populations between study rounds and vaccination status of women, we used PS stratification to adjust. Nevertheless, this only adjusts for measured variables. We had no information on type of sexual practices and smoking behaviour, so we were not able to adjust for these variables. Fifth, as unvaccinated women might have benefitted from second-order herd-immunity over time [30], unvaccinated women from younger birth cohorts might be less likely to be HPV infected. This might have resulted in an underestimation in the VE estimates for the routine vaccination cohort. Additionally, the routine vaccination cohort as a whole might have benefitted more from herd-immunity than the catch-up cohort, resulting in a potential bias in the comparison between routine and catch-up vaccination. Finally, we were unable to conduct a comparison of VE of number of doses. Due the pseudonymized study design, we could not link our data to a national vaccination database. Considering evidence suggesting that a single dose provides comparable efficacy to three doses of bivalent HPV vaccination against HPV infection [31,32,33], we would not expect large differences in VE between different doses groups. We therefore defined being vaccinated as ‘having received at least one dose’. It must be mentioned that research on effectiveness of different doses schedules is still ongoing and that cross-protection effectiveness on one-dose vaccination has not been looked at in-depth.

The VE estimates of type-specific and grouped VE against genital HPV-16 and/or HPV-18 of this study are in agreement with previous literature; the estimates are in line with previous vaccine efficacy trials of the bivalent HPV vaccine [4, 34] and with post-licencing observational studies among young women [6, 8, 10]. Some of these studies assessed persistent infection instead of HPV positivity but reported comparable VE estimates against female genital HPV.

We found significant cross-protection against 6 genotypes, being HPV-31, HPV-33, HPV-35, HPV-45, HPV-52, and HPV-58. These genotypes are associated with an additional 18% of cervical cancers worldwide [3]. Cross-protection against these genotypes had been observed before in HPV vaccine studies, although never before all these 6 genotypes within one study. The PATRICIA trial, the largest phase III trial, observed significant cross-protection against persistent infection with HPV-31, HPV-33, HPV-45, and HPV-52 with comparable estimates and against incident infection with HPV-35 [35, 36]. Another large clinical trial, the Costa Rica HPV vaccine Trial, assessed cross-protection of the bivalent HPV vaccine against incident infection [37]. They observed not only cross-protection against previously mentioned genotypes (HPV-31/33/35/45) but also against HPV-58 of 21.2% (95%CI: 4.2; 35.3), somewhat lower than our estimate. In a systematic review on cross-protective effects of the bivalent HPV vaccine, outcomes of 17 observational studies were compared [5], either as VE or prevalence over time. Of these studies, being from Japan, Luxembourg, Scotland, Finland, and the Netherlands, 11 observed cross-protection: nine against HPV-31, eight against HPV-45, five against HPV-31, and two against HPV-52 [5]. Finally, previous analyses of the PASSYON study [11, 22] did not observe significant cross-protection against HPV-58 as observed in the current analysis, which is probably explained by the greater power of the current analysis due to the larger study population.

A statistically significant negative VE for HPV-59 was observed. The SPF10-DEIA-LiPA25 is known to detect HPV-59 less well in the presence with co-infection of other hrHPV genotypes [38]. Effective vaccination leads to a decrease in co-infections with other genotypes, making it more likely for HPV-59 to be detected in vaccinated compared to unvaccinated women [38]. This might lead to a biased VE estimate as shown previously [38].

In this study, VE estimates were similar for women who had been eligible for the catch-up campaign in 2009 at the age 13–16 years and for women who had been eligible from 2010 onwards for routine HPV vaccination at 12 years as part of the Dutch NIP. Even for women whom were potentially sexually active at the time they had been offered HPV vaccination, the VE was very high. Only one other study, also conducted in the Netherlands, compared routine vaccination to the catch-up but did not report type-specific HPV-16/HPV-18 VE estimates [39]. That study did not find statistically significant different hrHPV positivity rates for women vaccinated in the catch-up campaign compared to those vaccinated in routine vaccination, although the odds for hrHPV was slightly lower in the routine vaccination cohort than the catch-up cohort [39]. These results are in line with findings of the current study. Our results are suggestive that catch-up campaigns up to the age of 16 is probably as effective, in terms of VE, as routine vaccination at an earlier age. This is in line with other studies that found that vaccination up to age 16 years was associated with reduced risk of cervical cancer protection; protection was much lower for females vaccinated at the age of 17 years and above [40,41,42,43]. It seems that catch-up campaigns up to the age of 16 years might be effective in terms of protection and who are reached. With a median age of sexual debut in our study population of 16 years (IQR: 15–16), most girls were not yet, or just recently, sexually active when they would have been eligible for the catch-up vaccination at the age of 13–16 years. The older people are when being vaccinated, the more likely they are already sexually active. Therefore, higher age of vaccination is related to an increased odds for HPV infection [44]. We recommend conducting similar analyses in countries with catch-up campaigns targeting older age groups, to further assess the relation between age of vaccination and the VE for HPV vaccination. In the Netherlands, a gender-neutral catch-up campaign up to the age of 26 was conducted starting 2023. Future analyses are recommended to assess the effectiveness of this catch-up campaign.

Conclusions

In our study, a high VE of bivalent HPV vaccination was demonstrated against genital HPV-16 and HPV-18 DNA positivity for young women in the Netherlands. Additionally, significant cross-protection was found for six other hrHPV genotypes. Together, these eight genotypes are responsible for 89% of cervical cancers worldwide. These findings should be used in cost-effectiveness studies when deciding about the choice of vaccine in NIPs. Moreover, this study demonstrated that catch-up campaigns focussed on girls up to age 16 years might be as protective as routine vaccination at an earlier age. However, as most estimates were slightly lower for women who were sexually active before or at the same age as they were eligible for vaccination, it is recommendable to provide HPV vaccination at an age at which most are likely not sexually active yet.

Data Availability

This study collected data which has been pseudonymised. Pseudonymised individual participant data can be requested for scientific use with a methodologically sound proposal. In case of data sharing, receivers of data will need to sign a data transfer agreement.

Abbreviations

CI:

Confidence interval

hrHPV:

High-risk HPV

HPV:

Human papillomavirus

IPW:

Inverse probability weighting

IQR:

Interquartile range

lrHPV:

Low-risk HPV

NIP:

National immunisation programme

PS:

Propensity score

RR:

Relative risk

SHC:

Sexual health clinic

STI:

Sexually transmitted infection

VE:

Vaccine effectiveness

References

  1. de Sanjose S, Brotons M, Pavon MA. The natural history of human papillomavirus infection. Best Pract Res Clin Obstet Gynaecol. 2018;47:2–13.

    Article  PubMed  Google Scholar 

  2. De Martel C, Plummer M, Vignat J, Franceschi S. Worldwide burden of cancer attributable to HPV by site, country and HPV type. Int J Cancer. 2017;141(4):664–70.

    Article  PubMed  PubMed Central  Google Scholar 

  3. De Sanjose S, Quint WG, Alemany L, Geraets DT, Klaustermeier JE, Lloveras B, Tous S, Felix A, Bravo LE, Shin H-R. Human papillomavirus genotype attribution in invasive cervical cancer: a retrospective cross-sectional worldwide study. Lancet Oncol. 2010;11(11):1048–56.

    Article  PubMed  Google Scholar 

  4. Paavonen J, Naud P, Salmerón J, Wheeler CM, Chow S-N, Apter D, Kitchener H, Castellsague X, Teixeira JC, Skinner SR. Efficacy of human papillomavirus (HPV)-16/18 AS04-adjuvanted vaccine against cervical infection and precancer caused by oncogenic HPV types (PATRICIA): final analysis of a double-blind, randomised study in young women. Lancet. 2009;374(9686):301–14.

    Article  PubMed  Google Scholar 

  5. Brown DR, Joura EA, Yen GP, Kothari S, Luxembourg A, Saah A, Walia A, Perez G, Khoury H, Badgley D. Systematic literature review of cross-protective effect of HPV vaccines based on data from randomized clinical trials and real-world evidence. Vaccine. 2021;39(16):2224–36.

    Article  PubMed  Google Scholar 

  6. Mesher D, Panwar K, Thomas SL, Edmundson C, Choi YH, Beddows S, Soldan K. The impact of the national HPV vaccination program in England using the bivalent HPV vaccine: surveillance of type-specific HPV in young females, 2010–2016. J Infect Dis. 2018;218(6):911–21.

    Article  PubMed  Google Scholar 

  7. Purriños-Hermida MJ, Santiago-Pérez MI, Treviño M, Dopazo R, Cañizares A, Bonacho I, Trigo M, Fernández ME, Cid A, Gómez D. Direct, indirect and total effectiveness of bivalent HPV vaccine in women in Galicia, Spain. PLoS ONE. 2018;13(8): e0201653.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Kudo R, Yamaguchi M, Sekine M, Adachi S, Ueda Y, Miyagi E, Hara M, Hanley SJ, Enomoto T. Bivalent human papillomavirus vaccine effectiveness in a japanese population: high vaccine-type–specific effectiveness and evidence of cross-protection. J Infect Dis. 2019;219(3):382–90.

    Article  PubMed  Google Scholar 

  9. Latsuzbaia A, Arbyn M, Tapp J, Fischer M, Weyers S, Pesch P, Mossong J. Effectiveness of bivalent and quadrivalent human papillomavirus vaccination in Luxembourg. Cancer Epidemiol. 2019;63: 101593.

    Article  PubMed  Google Scholar 

  10. Donken R, King AJ, Bogaards JA, Woestenberg PJ, Meijer CJLM, de Melker HE. High effectiveness of the bivalent human papillomavirus (HPV) vaccine against incident and persistent HPV infections up to 6 years after vaccination in young Dutch women. J Infect Dis. 2018;217(10):1579–89.

    Article  PubMed  Google Scholar 

  11. Woestenberg PJ, King AJ, van Benthem BH, Donken R, Leussink S, van der Klis FR, de Melker HE, van der Sande MA, Hoebe CJ, Bogaards JA. Bivalent vaccine effectiveness against type-specific HPV positivity: evidence for cross-protection against oncogenic types among Dutch STI clinic visitors. J Infect Dis. 2018;217(2):213–22.

    Article  PubMed  Google Scholar 

  12. Mollers M, King AJ, Knol MJ, Scherpenisse M, Meijer CJ, van der Klis FR, de Melker HE. Effectiveness of human papillomavirus vaccine against incident and persistent infections among young girls: results from a longitudinal Dutch cohort study. Vaccine. 2015;33(23):2678–83.

    Article  PubMed  Google Scholar 

  13. Cameron RL, Kavanagh K, Pan J, Love J, Cuschieri K, Robertson C, Ahmed S, Palmer T, Pollock KG. Human papillomavirus prevalence and herd immunity after introduction of vaccination program, Scotland, 2009–2013. Emerg Infect Dis. 2016;22(1):56.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Cuschieri K, Kavanagh K, Moore C, Bhatia R, Love J, Pollock KG. Impact of partial bivalent HPV vaccination on vaccine-type infection: a population-based analysis. Br J Cancer. 2016;114(11):1261–4.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Cody P, Tobe K, Abe M, Elbasha EH. Public health impact and cost effectiveness of routine and catch-up vaccination of girls and women with a nine-valent HPV vaccine in Japan: a model-based study. BMC Infect Dis. 2021;21(1):1–13.

    Article  Google Scholar 

  16. Drolet M, Laprise J-F, Brotherton JM, Donovan B, Fairley CK, Ali H, Bénard É, Martin D, Brisson M. The impact of human papillomavirus catch-up vaccination in Australia: implications for introduction of multiple age cohort vaccination and postvaccination data interpretation. J Infect Dis. 2017;216(10):1205–9.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Daniels V, Prabhu VS, Palmer C, Samant S, Kothari S, Roberts C, Elbasha E. Public health impact and cost-effectiveness of catch-up 9-valent HPV vaccination of individuals through age 45 years in the United States. Hum Vaccin Immunother. 2021;17(7):1943–51.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Hoes J, Schurink T. van’t Klooster HP, van der Weele P, van Eer K, Woestenberg P, van Benthem B, Boogaards H, van der Klis F, King A: HPV- vaccinatie in Nederland: 10 jaar in het Rijksvaccinatieprogramma, een overzicht. Nederlands tijdschrift voor medische microbiologie. 2019;27(4):226–35.

    Google Scholar 

  19. Vriend HJ, Boot HJ, van der Sande MA. Type-specific human papillomavirus infections among young heterosexual male and female STI clinic attendees. Sex Transm Dis. 2012:72–8.

  20. Woestenberg PJ, King AJ, Van Benthem BH, Leussink S, Van der Sande MA, Hoebe CJ, Bogaards JA, Laboratories MM. Services tPH: Bivalent vaccine effectiveness against anal human papillomavirus positivity among female sexually transmitted infection clinic visitors in the Netherlands. J Infect Dis. 2020;221(8):1280–5.

    PubMed  Google Scholar 

  21. Woestenberg PJ, King AJ, van der Sande MA, Donken R, Leussink S, van der Klis FR, Hoebe CJ, Bogaards JA, van Benthem BH, Adema D. No evidence for cross-protection of the HPV-16/18 vaccine against HPV-6/11 positivity in female STI clinic visitors. J Infect. 2017;74(4):393–400.

    Article  PubMed  Google Scholar 

  22. Bogaards JA, van der Weele P, Woestenberg PJ, van Benthem BHB, King AJ. Bivalent human papillomavirus (HPV) vaccine effectiveness correlates with phylogenetic distance from HPV vaccine types 16 and 18. J Infect Dis. 2019;220(7):1141–6.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Malagón T, Drolet M, Boily M-C, Franco EL, Jit M, Brisson J, Brisson M. Cross-protective efficacy of two human papillomavirus vaccines: a systematic review and meta-analysis. Lancet Infect Dis. 2012;12(10):781–9.

    Article  PubMed  Google Scholar 

  24. Rubin DB. Propensity score methods. Am J Ophthalmol. 2010;149(1):7–9.

    Article  PubMed  Google Scholar 

  25. Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41–55.

    Article  Google Scholar 

  26. Chen JW, Maldonado DR, Kowalski BL, Miecznikowski KB, Kyin C, Gornbein JA, Domb BG. Best practice guidelines for propensity score methods in medical research: consideration on theory, implementation, and reporting. A review. Arthroscopy: J Arthrosc Related Surg. 2022;38(2):632–42.

    Article  Google Scholar 

  27. Diaz-Quijano FA. A simple method for estimating relative risk using logistic regression. BMC Med Res Methodol. 2012;12:1–6.

    Article  Google Scholar 

  28. Elfström KM, Dillner J, Arnheim-Dahlström L. Organization and quality of HPV vaccination programs in Europe. Vaccine. 2015;33(14):1673–81.

    Article  PubMed  Google Scholar 

  29. Serrano B, Brotons M, Bosch FX, Bruni L. Epidemiology and burden of HPV-related disease. Best Pract Res Clin Obstet Gynaecol. 2018;47:14–26.

    Article  PubMed  Google Scholar 

  30. Hoes J, Woestenberg PJ, Bogaards JA, King AJ, de Melker HE, Berkhof J, Hoebe CJ, van der Sande MA, van Benthem BH. Population impact of girls-only human papillomavirus 16/18 vaccination in the Netherlands: cross-protective and second-order herd effects. Clin Infect Dis. 2021;72(5):e103–11.

    Article  PubMed  Google Scholar 

  31. Kreimer AR, Herrero R, Sampson JN, Porras C, Lowy DR, Schiller JT, Schiffman M, Rodriguez AC, Chanock S, Jimenez S. Evidence for single-dose protection by the bivalent HPV vaccine—review of the Costa Rica HPV vaccine trial and future research studies. Vaccine. 2018;36(32):4774–82.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Barnabas RV, Brown ER, Onono MA, Bukusi EA, Njoroge B, Winer RL, Galloway DA, Pinder LF, Donnell DN, Wakhungu I. Durability of single-dose HPV vaccination in young Kenyan women: randomized controlled trial 3-year results. Nat Med. 2023;29(12):3224–32.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Watson-Jones D, Changalucha J, Whitworth H, Pinto L, Mutani P, Indangasi J, Kemp T, Hashim R, Kamala B, Wiggins R. Immunogenicity and safety of one-dose human papillomavirus vaccine compared with two or three doses in Tanzanian girls (DoRIS): an open-label, randomised, non-inferiority trial. Lancet Glob Health. 2022;10(10):e1473–84.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Harper DM, Franco EL, Wheeler CM, Moscicki A-B, Romanowski B, Roteli-Martins CM, Jenkins D, Schuind A, Clemens SAC, Dubin G. Sustained efficacy up to 4· 5 years of a bivalent L1 virus-like particle vaccine against human papillomavirus types 16 and 18: follow-up from a randomised control trial. The Lancet. 2006;367(9518):1247–55.

    Article  Google Scholar 

  35. Wheeler CM, Castellsagué X, Garland SM, Szarewski A, Paavonen J, Naud P, Salmerón J, Chow S-N, Apter D, Kitchener H. Cross-protective efficacy of HPV-16/18 AS04-adjuvanted vaccine against cervical infection and precancer caused by non-vaccine oncogenic HPV types: 4-year end-of-study analysis of the randomised, double-blind PATRICIA trial. Lancet Oncol. 2012;13(1):100–10.

    Article  PubMed  Google Scholar 

  36. Struyf F, Colau B, Wheeler CM, Naud P, Garland S, Quint W, Chow S-N, Salmerón J, Lehtinen M, Del Rosario-Raymundo MR. Post hoc analysis of the PATRICIA randomized trial of the efficacy of human papillomavirus type 16 (HPV-16)/HPV-18 AS04-adjuvanted vaccine against incident and persistent infection with nonvaccine oncogenic HPV types using an alternative multiplex type-specific PCR assay for HPV DNA. Clin Vaccine Immunol. 2015;22(2):235–44.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Tsang SH, Sampson JN, Schussler J, Porras C, Wagner S, Boland J, Cortes B, Lowy DR, Schiller JT, Schiffman M. Durability of cross-protection by different schedules of the bivalent HPV vaccine: the CVT trial. J Natl Cancer Inst. 2020;112(10):1030–7.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Van Eer K, Leussink S, Severs TT, van Marm-Wattimena N, Woestenberg PJ, Bogaards JA, King AJ: Evidence for missing positive results for human papilloma virus 45 (HPV-45) and HPV-59 with the SPF10-DEIA-LiPA25 (version 1) platform compared to type-specific real-time quantitative PCR assays and impact on vaccine effectiveness estimates. J Clin Microbiol. 2020;58(11): https://doi.org/10.1128/jcm.01626-01620.

  39. Schurink-van’t Klooster TM, Siebers AG, Hoes J, van Kemenade FJ, Berkhof J, Bogaards JA, de Melker HE. Early effect of bivalent human papillomavirus vaccination on cytology outcomes in cervical samples among young women in the Netherlands. Cancer Med. 2023;12(10):11786–94.

    Article  PubMed  Google Scholar 

  40. Kjaer SK, Dehlendorff C, Belmonte F, Baandrup L. Real-world effectiveness of human papillomavirus vaccination against cervical cancer. JNCI: J Natl Cancer Inst. 2021;113(10):1329–35.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Lei J, Ploner A, Elfström KM, Wang J, Roth A, Fang F, Sundström K, Dillner J, Sparén P. HPV vaccination and the risk of invasive cervical cancer. N Engl J Med. 2020;383(14):1340–8.

    Article  PubMed  Google Scholar 

  42. Palmer T, Wallace L, Pollock KG, Cuschieri K, Robertson C, Kavanagh K, Cruickshank M. Prevalence of cervical disease at age 20 after immunisation with bivalent HPV vaccine at age 12–13 in Scotland: retrospective population study. BMJ. 2019;365:l1161.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Palmer TJ, Kavanagh K, Cuschieri K, Cameron R, Graham C, Wilson A, Roy K. Invasive cervical cancer incidence following bivalent human papillomavirus vaccination: a population-based observational study of age at immunization, dose, and deprivation. JNCI: Journal of the National Cancer Institute. 2024:djad263.

  44. Kavanagh K, Pollock KG, Cuschieri K, Palmer T, Cameron RL, Watt C, Bhatia R, Moore C, Cubie H, Cruickshank M. Changes in the prevalence of human papillomavirus following a national bivalent human papillomavirus vaccination programme in Scotland: a 7-year cross-sectional study. Lancet Infect Dis. 2017;17(12):1293–302.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the following for their valuable contributions to the design or execution of the study: Hein Boot (deceased), Elske van Logchem, Naomi van Marm, Suzan Leussink, Tim Severs, Rutger Schepp, Kahren van Eer, Thomas Langenburg, Najima Lamkaraf, Maarten Schipper, Petra Woestenberg, Rianne Vriend and Marianne van der Sande.

Furthermore, the centres for sexual health, including all nurses and physicians, within the Public Health Services and the hospitals are acknowledged for their permission to collect data from their clients and for their efforts. The authors acknowledge the medical microbiologic laboratories and their staff for storage and testing of the samples.

Funding

This work was supported by the Ministry of Health, Welfare and Sport, the Netherlands. The funders had no role in the study design, data collection and analysis, interpretation of data, decision to publish, or preparation of the manuscript.

Author information

Authors and Affiliations

Authors

Consortia

Contributions

BHvB was the principal investigator of the PASSYON study, and together with HEdM responsible for funding acquisition. AJK was responsible for supervising the genital sample analyses and the interpretation of the HPV DNA-sample analyses. JMAK, AJK and BHvB were responsible for validation and verification of the data. JMAK, MFSvdLoeff, JCMH and BHBvB were responsible for the conception, methodology and design of the current study. JMAK was responsible for conducting the formal analyses. All authors participated in the interpretation of the results. The original manuscript draft, including figures and tables, was created by JMAK, and all authors contributed to editing of the manuscript. Finally, BHvB, JCMH and MFSvdL were responsible for supervising JMAK. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Johannes M. A. Kusters.

Ethics declarations

Ethics approval and consent to participate

This study was approved by Medical Ethics Committee of the University of Utrecht, the Netherlands (reference number: 08/397). Participants gave informed consent to participate in the study before taking part.

Consent for publication

Not applicable.

Competing interests

The institution of M.F. Schim van der Loeff and J.C.M Heijne received study funding for an investigator-initiated study from GSK. M.F. Schim van der Loeff served on advisory boards of MSD and Novosanis. All other authors report no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

12916_2024_3686_MOESM1_ESM.docx

Additional file 1: Tables S1-S6, Figs. S1–S3. Table S1. Characteristics of the study population by eligibility for vaccination cohort. Table S2. Genital HPV DNA-positivity prevalences among unvaccinated females. Table S3. Genital HPV DNA-positivity prevalences among vaccinated females. Table S4. Crude and propensity score stratification adjusted vaccine effectiveness against genital HPV. Table S5. Vaccine effectiveness against genital HPV by eligibility for vaccination cohort. Table S6. Vaccine effectiveness against genital HPV of birth cohort 1993–1996 by potential sexual activity at eligibility for HPV vaccination. Fig. S1. Stratified analyses of vaccine effectiveness, by sexual behaviour. Fig. S2–S3. Sensitivity analyses for vaccine effectiveness.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kusters, J.M.A., van der Loeff, M.F.S., van Benthem, B.H.B. et al. Effectiveness of bivalent HPV vaccination against genital HPV DNA-positivity of a catch-up campaign at age 13–16 years compared to routine vaccination at age 12 years: a biennial repeated cross-sectional study. BMC Med 22, 469 (2024). https://doi.org/10.1186/s12916-024-03686-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12916-024-03686-4

Keywords