Application of the screening method for estimating COVID-19 vaccine effectiveness using routine surveillance data: Germany’s experience during the COVID-19 pandemic, July 2021 to March 2023

The screening method represents a simple, quick, and practical tool for estimating vaccine effectiveness (VE) using routine disease surveillance and vaccine coverage data, even if these data cannot be linked. In Germany, where notification data, laboratory testing data, and vaccine coverage data cannot be linked due to strict data protection requirements, the screening method was used to assess COVID-19 VE continuously between July 2021 and March 2023. During this period, when Delta and Omicron variants circulated, VE estimates were produced in real-time for different age groups and clinical outcomes. Here we describe the country’s overall positive experience using the screening method, including its strengths and limitations, and provide practical guidance regarding a few issues, such as case definition stringency, testing behaviour, and data stratification, that require careful consideration during data analysis and the interpretation of the results.

The screening method represents a simple, quick, and practical tool for estimating vaccine effectiveness (VE) using routine disease surveillance and vaccine coverage data, even if these data cannot be linked.In Germany, where notification data, laboratory testing data, and vaccine coverage data cannot be linked due to strict data protection requirements, the screening method was used to assess COVID-19 VE continuously between July 2021 and March 2023.During this period, when Delta and Omicron variants circulated, VE estimates were produced in real-time for different age groups and clinical outcomes.Here we describe the country's overall positive experience using the screening method, including its strengths and limitations, and provide practical guidance regarding a few issues, such as case definition stringency, testing behaviour, and data stratification, that require careful consideration during data analysis and the interpretation of the results.

Background
Since the start of the worldwide COVID-19 vaccination campaign [1], it has become abundantly clear that continuous monitoring of novel COVID-19 vaccines in the real world is needed to estimate their effectiveness in the general population and specific subgroups, understand waning of vaccine-induced immunity, and assess the impact of new severe acute respiratory coronavirus 2 (SARS-CoV-2) variants on vaccine effectiveness (VE).To guide policy decision-making, such data need be available in a timely manner.Use of routine surveillance data to estimate VE represents a fast, practical, and resource-saving approach to achieve this goal.The 'screening' or 'case coverage' method (hereafter referred to as screening method) has been successfully applied to estimate effectiveness of influenza [2][3][4][5] or pneumococcal [6] vaccines by using routine disease surveillance and vaccine coverage data.While at the beginning of the pandemic it was unclear how well suited this method would be for estimating COVID-19 VE, a number of countries [7][8][9] successfully implemented its use early on and the World Health Organization (WHO) provided guidance for its use in June 2022 [10].
In Germany, real-world evidence on the effectiveness of COVID-19 vaccines was assessed using a threepronged approach: a prospective hospital-based case-control study [11,12], the screening method using routine surveillance data, and a living systematic review to synthesise results from studies conducted worldwide [13][14][15].While the other two approaches delivered reliable results with a time lag of up to a few months, the screening method was used to estimate VE continuously for different age groups and clinical outcomes with only minimal time lag during circulation of the Delta (Phylogenetic Assignment of Named Global Outbreak (Pango) lineage designation B.1.617.2) and Omicron (B.1.1.529)variants.The results were published in weekly (from July 2022 onwards, monthly) reports to inform health professionals, policy, and government decisionmakers, the media, and the public [16].Here we describe using the screening method to estimate and report COVID-19 VE in Germany, a country where the notification data, laboratory testing data, and vaccine coverage data cannot be linked.We also discuss strengths and limitations of the method in this context.

Screening method
The screening method estimates VE by comparing the vaccine coverage among identified cases with the vaccine coverage among the source population from which the cases arose using the following formula: where PCV is proportion of cases who are fully vaccinated, and PPV is the proportion of the population that is fully vaccinated [17,18].Implementing the screening method, therefore, requires data on cases, their vaccination status, as well as on the vaccination status of the target population.In Germany, these data were collected as part of routine public health surveillance during the COVID-19 pandemic.

German COVID-19 surveillance data COVID-19 case notification data
Data on cases were derived from the German national infectious disease case surveillance system (SurvNet) [19].As with other notifiable diseases in Germany, every SARS-CoV-2 infection confirmed by culture or nucleic acid testing fulfils the COVID-19 case definition and must be notified by physicians, hospitals, laboratories, and other testing sites to local public health authorities.The local public health authorities collect information on demographics (age, sex, residence), symptoms, medical risk factors, vaccination status, and disease outcome and document this information in pseudonymised electronic case notification files, which they transmit via the federal state health authorities to the Robert Koch Institute (RKI), Germany's national public health institute.

COVID-19 Vaccination Coverage database
Germany began its national immunisation campaign against COVID-19 on 27 December 2020, immunising its population in order of age and risk prioritisation, with older age groups and those at higher risk of severe disease or exposure being vaccinated first [20].Vaccine coverage data were derived from Germany's anonymised electronic COVID-19 Vaccination Coverage database (Digitales Impfquotenmonitoring, DIM) [21].The goal of DIM was to monitor in real-time the progress of Germany's COVID-19 vaccination campaignas there is no routine vaccination registry in Germany, providers were required to notify daily all COVID-19 vaccinations administered to the RKI, which maintains the DIM database.The Table provides a brief overview of the parameters of the screening method formula, as derived from Germany's COVID-19 surveillance data.

Calculation of COVID-19 vaccine effectiveness
COVID-19 cases were classified into vaccine 'exposure' categories by age group (5-11, 12-17, 18-59, and ≥ 60 years) and vaccination status; the age groups were selected to be consistent with the information collected in DIM.Unvaccinated cases were those who were notified as not having received any COVID-19 vaccine dose at disease onset.Except for children (5-11 years) for whom one COVID-19 vaccine dose was recommended (as further described), fully vaccinated individuals were defined as having received two doses of a COVID-19 vaccine ≥ 14 days before disease onset, and boosted individuals were defined as having received three or more doses of a COVID-19 vaccine ≥ 7 days before disease onset.If date of disease Although information on specific variants was largely missing among case data, once a circulating variant was dominant, we attributed all new cases to that variant and interpreted the VE results accordingly.For example, after the emergence and dominance of the Omicron variant, we saw a continuous and steep decrease of VE for a two-dose scheme among adults aged ≥ 60 years, whereas VE for booster vaccinations (≥ 3 doses) exhibited a slower decrease over a period of a few weeks in this age group, thereafter stabilising and remaining high (Figure 1).The aggregated nature of the vaccine coverage data did not allow for regular assessment of VE in relation to time since vaccination (i.e.waning of vaccine-induced immunity); nonetheless, when individual vaccination histories within the target population group are available, the screening method provides a simple means of examining duration of protection.As our VE analysis sought to encompass the entire German population, with no exclusion criteria or indication of a strong selection bias, our estimates were generalisable and included even population groups that would often be excluded from clinical trials or observational studies.

Challenges and considerations
While we were able to calculate and report comprehensive, population-based VE in a timely manner, we also encountered some challenges that should be considered when applying the screening method.First, with the emergence of the Omicron variant, severity of the COVID-19 disease course decreased among both, vaccinated and unvaccinated individuals [22,23].As a consequence, SARS-CoV-2 positive patients admitted to a hospital were often not admitted as a result of complications due to COVID-19, but increasingly for other diseases, while presenting only with mild COVID-19 symptoms.Due to routine screening for COVID-19 before all hospital admissions, however, these patients continued to be notified as hospitalised COVID-19 cases first and foremost.Using the endpoint 'hospitalisation with COVID-19' for our analyses led to underestimation of VE, as also seen in VE figures from England estimated using a different methodology [24].
For this reason, we retroactively narrowed the case definition for hospitalisation and ICU admission and included only cases that were 'admitted due to COVID-19' in our analyses, as per the recommendations of the WHO [25].The effect of this change in case definition is shown in Figure 2. A trade-off of this decision was that it led to a decrease in the total number of cases who could be included in the VE calculations, due to a high proportion of cases without information on reason for hospitalisation.Second, we also learned that testing behaviour was associated with vaccination status.Vaccinated individuals were more likely to test themselves (at home or in test centres) than those who were unvaccinated [26].When mandatory testing (needed, for example, for access to schools, workplaces, restaurants, and cultural events) ended for most of the population in Summer 2022, the potential for confounding increased as vaccinated individuals continued testing themselves at a higher rate than unvaccinated individuals, leading to a sharp decline in VE for symptomatic illness.However, since routine testing for every patient at hospital admission was established and continues to this day, we were confident that this selection bias did not affect cases presenting with more severe clinical outcomes.Therefore, we decided to focus on VE against severe disease and discontinued the reporting of VE estimates for (mild) symptomatic disease.
Third, when using the screening method, it is imperative to adjust for confounding factors, such as by stratifying the data.As the vaccine coverage data in Germany was mostly aggregated and no or little information on comorbidities, sex, vaccine product, or narrower age groups was collected, we could only stratify our analyses for broad age groups and calendar weeks.
Among the case data, information on prior infections with SARS-CoV-2 or with detail on the SARS-CoV-2 variant was largely unavailable.Thus, we observed some bias in our results and were unable to calculate VE by specific and narrower categories.Population groups at risk for severe COVID-19 were also more likely to have higher vaccine Without being able to adjust for medical risk factors, we could not rule out underestimation of VE in certain instances.A concrete example is the VE of booster doses among 18-59 year-olds in Germany, which continues to be low in general, but is likely to be higher among high risk groups within this age group.Assuming that the immune response in persons with underlying disease (especially those affecting the immune system) is lower than in healthy persons, this may have led to lower VE estimates of booster doses among younger adults in our analyses in comparison to older adults, who generally have much higher vaccine coverage due to their age itself being an indication for vaccination (Figure 3).
Fourth, we were unable to calculate reliable VE estimates among certain age groups and clinical outcome categories, such as ICU admissions or deaths among children, due to the small number of cases notified (sometimes none) for each 4-week interval.In such instances, VE fluctuated wildly over time depending on the vaccination status of the (often) one case notified for each time interval.This limitation is not specific to the screening method, however, as low sample sizes in population sub-groups is an issue faced in many study designs.
Fifth, as the data were not collected for purposes of VE estimation, a high proportion of missing values for vaccination status among hospitalised cases might have led to a non-representative sample, although we have no indication that case notification was associated with vaccination status in our data.Still, under-reporting, data incompleteness, misclassification of vaccination status or of clinical outcome, particularly during periods of high incidence, could have influenced our VE estimates.
Sixth, interpretation of results was not always simple: we regularly and retroactively compared our results with the results of other studies identified, among others, in the living systematic review [13][14][15], such as case-control studies with test-negative design [11,12,24], and surveillance reports published in other countries [27], in order to ensure that our screening method approach was delivering reliable results.It speaks to the overall strength of the screening method that the VE we reported out remained comparable and valid.Furthermore, it was necessary to consider the changing landscape with regards to national vaccination guidelines, emerging variants and variant-specific vaccines when analysing and interpreting the VE results for different age groups and clinical endpoints.

Conclusions
The screening method represents a practical tool for estimating VE when surveillance data on cases and vaccine coverage among the target population are already collected, but not necessarily linked or linkable.Depending on the frequency of data collection, as well as the clinical outcome of interest, VE estimates can be calculated and reported quickly, allowing for monitoring of trends in VE over time among different population groups with only minimal time lag, as was the case in Germany during the COVID-19 pandemic.Other methods that were used to estimate VE in Germany, such as case-control studies and cohort studies, could report results with long-time lags only and required significantly greater resources; they, therefore, focused on specific clinical outcomes or population groups [11,28].This is in contrast to countries such as England and Israel that could utilise already available national health register or health insurance data to rapidly assess population-wide VE without requiring significant resources [27,29].Publishing the COVID-19 VE estimates, as well as the methodology of the screening method, in a transparent manner in regular reports for decisionmakers, medical experts, the media, and the public allowed for an open dialogue and showcased, through its usage, the importance of collecting reliable vaccination data.The reports also helped to demonstrate that a continuous monitoring of the COVID-19 vaccines' effects was being conducted by national authorities and that the vaccines were and are very effective in preventing severe disease based on local, German data.Nonetheless, the screening result must be implemented within the context of its requirements and limitations and results must be interpreted with caution and used as rough guidance only.

Ethical statement
Ethical approval and informed consent were deemed not necessary.All data were collected as part of routine surveillance under public health law and deidentified prior to transmission to RKI.

Table
Summary and comparison of data sources, namely the COVID-19 case notification system and the COVID-19 Vaccination Coverage database, from which the parameters required for the screening method were extracted, Germany, July 2021-March 2023 groups, different vaccination schemes (primary series, booster vaccinations, as well as single dose VE for 5-11 year-olds as per the national immunisation recommendations), and multiple clinical endpoints.By applying the method early in the pandemic and continuing monitoring over more than 20 months, we could follow VE over time and assess trends.
onset was missing, date of diagnosis or date of notification was used, in that order.Percentage of the population by the same age and vaccination categories were derived from DIM. VE for primary vaccination series (one dose for 5-11 year-olds and two doses for others) and booster doses were calculated for consecutive intervals of 4 weeks for different clinical outcomes, which were categorised according to severity of the COVID-19 course -symptomatic infection (concomitant with SARS-CoV-2 infection), hospitalisation (concomitant with SARS-CoV-2 infection), admission to an intensive care unit (ICU) (concomitant with SARS-CoVfor more severe symptoms to develop, as well as due to follow-up and reporting delays.Owing to the extremely large number of reported COVID-19 cases (1.6 million symptomatic COVID-19 cases with information on vaccination status notified to SurvNet in the first 6 months of VE reporting from July to December 2021), we were able to stratify our analyses and report VE estimates for different age