Effectiveness of influenza vaccination programme in preventing hospital admissions , Valencia , 2014 / 15 early results

J Puig-Barberà (puig_joa@gva.es)1,2, A Mira-Iglesias1, M Tortajada-Girbés3, F X López-Labrador1,4, A Belenguer-Varea5, M Carballido-Fernández6, E Carbonell-Franco7, C Carratalá-Munuera8,9, R Limón-Ramírez10, J Mollar-Maseres11, M del Carmen Otero-Reigada11, G Schwarz-Chavarri12, J Tuells13, V Gil-Guillén9,14, for the Valencia Hospital Network for the Study of Influenza and Respiratory Viruses Disease15 1. Fundación para el Fomento de la Investigación Sanitaria y Biomédica de la Comunitat Valenciana (FISABIO), Valencia, Spain 2. Centro de Salud Pública de Castellón, Castellón, Spain 3. Hospital Doctor Peset, Valencia, Spain 4. Consorcio de Investigación Biomédica de Epidemiología y Salud Pública (CIBERESP), Instituto de Salud Carlos III, Madrid, Spain 5. Hospital de La Ribera, Alzira, Spain 6. Hospital General de Castellón, Spain 7. Hospital Arnau de Vilanova, Valencia, Spain 8. Hospital San Juan, Alicante, Spain 9. Cátedra de Medicina de Familia. Departamento de Medicina Clínica, Universidad Miguel Hernández, San Juan, Alicante, Spain 10. Hospital de la Plana, Vila-real, Spain 11. Hospital Universitario y Politécnico La Fe, Valencia, Spain 12. Hospital General de Alicante, Alicante, Spain 13. Hospital Universitario del Vinalopó, Elche, Spain 14. Hospital de Elda, Elda, Spain 15. The Valencia Hospital Network for the Study of Influenza and Respiratory Virus Disease (VAHNSI) members are listed at the end of the article

An active annual surveillance scheme in the Valencia Region in Spain monitors IVE in preventing laboratoryconfirmed influenza requiring hospitalisation [6].In the current season, a split trivalent vaccine (Vaxigrip; SANOFI PASTEUR MSD, S.A. Madrid, Spain) was acquired by public tender and offered free of charge to non-institutionalised people targeted for influenza vaccination because of age or the presence of comorbidity [7].
We report 2014/15 influenza IVE in preventing laboratory-confirmed influenza admissions in patients admitted during the first eight weeks of the 2014/15 influenza season, from 7 December 2014 to 25 January 2015.

Methods
We performed a test-negative study in 10 hospitals that provide healthcare to 48% of the 4,937,044 inhabitants of Valencia.The influenza season began in week 50 2014 (Figure 1), as defined by two or more positive influenza hospitalisations identified in two consecutive weeks.
Study procedures have been published [6].Study staff screened emergency admissions for eligible subjects.Patients were included after written consent if they reported symptoms of influenza-like illness (ILI) [8] within seven days of admission.We collected a combined nasopharyngeal and pharyngeal flocked swab and sociodemographic and clinical information.A patient was considered immunised if they had received influenza vaccine more than 14 days before the onset of ILI (as recorded by the Vaccine Information System or by recall).
Laboratory confirmation was by semiquantitative reverse-transcription polymerase chain reaction.The specimens were tested in a centralised virology laboratory at Fundación para el Fomento de la Investigación Sanitaria y Biomédica (the Foundation for the Promotion of Health and Biomedical Research; FISABIO) following the World Health Organization (WHO) protocol [9].
Influenza-positive admissions were compared (vaccination odds ratio (OR)) with influenza-negative admissions [10,11].Adjusted IVE was defined as 100 x (1 -adjusted OR).We defined two groups for IVE estimation: (i) all subjects 18 years and older targeted for vaccination, and (ii) subjects aged 65 and older.Other subgroups were not considered to avoid a sparse-numbers bias [2,12].The sample size sufficient to provide 80% statistical power to detect an IVE of at least 50%, for a vaccine coverage of 50 to 60%, with a delta of 10%, was 130 to 150 influenza cases.
The adjusted OR were obtained by logistic regression.Previous knowledge and directed acyclic graphs (DAG) [13,14] were used to define the variables finally used to improve comparability and exchangeability between vaccinated and non-vaccinated subjects and to clarify the minimum set of variables to be included in the regression logistics models and in the inverse probability-weighted regression adjustment models.The roles of 'previous vaccination' as an instrumental variable highly correlated with current vaccination and 'time to swab', which cannot be considered a confounder but is clearly related to outcome, were made explicit with the use of DAG.We assessed departure from linearity in categorical ordered variables, interaction between potential confounders and vaccination, and clustering by enrolment site or epidemiological week.We used inverse probability-weighted regression adjustment to estimate the vaccination average effect from observed data [15,16] after conditioning influenza admissions on indicator variables: age in deciles, epidemiological week, number of comorbidities, smoking habits, hospital admission in the last 12 months, number of outpatient contacts in the last three months, time to swab, and recruitment hospital.We conditioned vaccination on the same covariates, excluding time to swab and adding influenza vaccination in the past two seasons.
Sensitivity analyses were performed according to time to swab and vaccination ascertainment in the vaccine information system.All probabilities were two-tailed; p < 0.05 was considered significant.Analyses were performed with Stata 13.1 (StataCorp, College Station, TX).
The Ethics Research Committee of the Dirección General de Salud Pública-Centro Superior de Investigación en Salud Pública (DGSP-CSISP) approved the protocol of the study.
Vaccination reduced by 44% (28-68%) the overall probability of hospital admission with influenza (Table 2).These results were not altered in the sensitivity analysis (Table 2).

Discussion
Our estimate suggests that the 2014/15 influenza vaccine was moderately effective in preventing hospital admissions related to influenza in a season in which a Adjusted by indicator variables: sex, age in deciles, smoking (never, ex-smoker, current smoker), number of outpatient contacts in the past three months (0, 1, 2, > 2), number of comorbidities (0, 1, 2, ≥ 2), hospital admissions in the past 12 months (yes, no), recruitment hospital, epidemiological week of admission, days from onset of symptoms to swab (≤ 2, 3-4, 5-7, > 7).b Average vaccination effect (percentage of reduction) on the probability of admission with confirmed influenza.Admission with influenza was conditioned on age in deciles, smoking (never, ex-smoker, current smoker), number of outpatient contacts in the past three months (0, 1, 2, > 2), number of comorbidities (0, 1, 2, ≥ 2), hospital admissions in the past 12 months (yes, no), recruitment hospital, epidemiological week of admission, days from onset of symptoms to swab (≤ 2, 3-4, 5-7, > 7).Vaccination was conditioned on the same indicator variables, but days to swab was omitted and record of influenza vaccination in 2012 and 2013 were added to the model.c Patients included are those with records of any vaccination in the VIS and swabbed ≤ 4 days after symptoms onset.
Source: Valencia Hospital Network for the Study of Influenza.
most influenza A(H3N2) viruses were different from the component in the 2014/15 influenza vaccine [17].We can only provide information regarding the genetic characteristics of strains analysed at the Centro Nacional de Microbiología, Instituto de Salud Carlos III in Madrid [17].According to their data, 67% of A(H3N2) clades could be considered antigenically and genetically different from the vaccine strain.We cannot make inferences regarding the impact on IVE of the type of vaccine used in Valencia as only one brand of vaccine was used throughout the region (99% of doses according to the vaccination registry).
Previous reviews and reports have shown that the trivalent inactivated vaccine can confer substantial protection against mismatched influenza A [ [18][19][20].Unfortunately, data from mismatched seasons on IVE in people 65 years and older are scarce [21].
There can be considerable variation in reported IVE estimates due to differences in strain circulation among countries, strain proportion within one region, the vaccines used, age-specific vaccine coverage, the population studied, season definition, case definition, ascertainment of vaccination status, differences in the period of surveillance, the variables included or omitted in the statistical model, how they are modelled, and measured outcome (admission, outpatient contact or infection) [11].A major caveat are sparse numbers.The absence of statistical significance should be expected in studies with low vaccine coverage, IVE or a limited sample size [2,12], as reflected in our sensitivity analysis (Table 2).
Influenza VE estimates generated from surveillance data using the test-negative design have already been presented at the WHO's annual strain selection meeting [11] as a way to improve the vaccine composition.However, variation in the estimates may undermine their credibility and usefulness, particularly early in the season.It is important to focus on sufficient sample size, robustness of the design, representativeness of the population, and validity of adjustment to inform vaccine reformulation and vaccination policies based on epidemiological data.
Our results support the substantial benefit of vaccination in preventing hospital admissions with laboratoryconfirmed influenza in the first weeks of the 2014/15 season in Valencia.

Research articles
Trivalent inactivated seasonal influenza vaccine effectiveness for the prevention of laboratory-confirmed influenza in a Scottish population 2000 to 2009

Introduction
Each year, influenza causes substantial morbidity and mortality, particularly in people aged 65 years and older and those with underlying serious comorbidities [1].Globally, for example, it is estimated that influenza is responsible for 5 million cases of severe illness and 250,000 to 500,000 deaths per year; the 186,000 excess hospitalisations and 44,000 excess deaths in the United States (US) have been estimated to cost USD 87 billion (EUR 77 billion) per year [2][3][4].Annual costs of influenza epidemics for the European Union are estimated to be EUR 27 billion [5].National vaccination strategies represent a potentially important approach to reduce both influenza-related illness and death, hence the considerable investment in this preventive approach in many parts of the world.In Scotland, the influenza vaccination programme has been successful with high rates of uptake for targeted individuals such as adults aged over 65 years and those clinically at risk of serious influenza-like illness [6].There is evidence of the benefits of the seasonal influenza vaccine in healthy children and younger adults (16 to 65 years) [7,8].However, in populations at highest risk of developing influenza-related complications (e.g.adults 65 years and older, people with medical conditions such as diabetes, heart or respiratory disease, and people with immunodeficiency), the populations particularly targeted by many countries' vaccination programmes including in Scotland, there is a paucity of reliable estimates of efficacy from randomised controlled trials [9].This is of concern, as it has been suggested that influenza vaccine is less effective in older people due to immunosenescence [10].Given that influenza vaccination programmes now exist in most developed countries, randomised controlled trials of the vaccine are impractical; these are also viewed as unethical by some sections of the medical community [11].Observational studies are a study design that can be used to investigate the effectiveness of vaccine programmes.
Since 2005, the test-negative study design, using realtime RT-PCR testing, has become more commonly used for evaluating influenza vaccine effectiveness (VE) [12].Most, however, have been carried out on single influenza seasons [13] and the three which have pooled data from multiple seasons only reported VE for limited age groups [14][15][16].Building on related work [17,18], we undertook a data linkage study and used detailed electronic health record data over nine consecutive seasons 2000/01 to 2008/09 to determine VE of the trivalent inactivated influenza vaccine in reducing laboratory-confirmed influenza.

Study databases and population characteristics
Almost all individuals resident in Scotland are registered with primary care, which provides a comprehensive array of healthcare services (free at the point of care), including prescriptions for medications.We used the Practice Team Information network, which covers a 5% representative sample of Scottish practices [19].Using the unique Community Health Index (CHI) number, specific patient-level data approved for use in this project were extracted and then linked to the Health Protection Scotland virology dataset, which consists of all laboratory-confirmed cases of influenza in Scotland.Once linkage had been completed, the analysis file was anonymised by replacing the unique CHI number with a study identifier.
We established key population characteristics: sex, age (0-4, 5-14, 15-44, 45-64, 65-74, and ≥ 75 years), socioeconomic status (Scottish Index of Multiple Deprivation scores [20] expressed as quintiles: 1 = most deprived to 5 = most affluent), receipt of pneumococcal and influenza vaccination in the previous year, smoking status (current, ex, non, not recorded), urban/rural residence (1 = large urban to 6 = remote rural), whether a patient was in a clinical group at risk of serious illness from influenza (i.e.chronic respiratory disease, chronic heart disease, chronic kidney disease, chronic liver disease, chronic neurological disease, immunosuppression or diabetes), Charlson co-morbidity index [21], number of previous primary care consultations, prescribed drugs and hospital admissions (in the year before 1 September).

Study design
In order to estimate VE derived from linked virological swab data, we carried out a test-negative study similar to that used by I-MOVE [22] ).We carried out an individual patient-level pooled analysis and adjusted for year.The influenza season was defined as the period from the date of the first influenza isolate reported by Health Protection Scotland for each year, in or after week 40 and the date of the last influenza isolate before or in week 20 (during the period of peak influenza).Vaccination was used to define exposure status if it was given at a time point between the start of the pre-influenza season (i.e. 1 September) and the end of the influenza season.An individual was defined as vaccinated 14 days after the seasonal influenza vaccine was administered.The time period from the first day of the influenza season to day 14 post vaccination was defined as 'unexposed' and the period from day 14 post vaccination until the end of the influenza season was defined as 'exposed'.The earliest influenza season began on 26 September and the latest began on 25 November, and all seasons finished in May (Table 1).A protocol of the study methods has been previously published [23].

Study outcomes
General practitioners in this study were also involved in the Health Protection Scotland sentinel-swabbing scheme, whereby practices are encouraged to obtain nasal or throat swabs from patients of all ages who have presented with symptoms suggestive of influenza.This is independent of whether or not the patient has been vaccinated.Each general practice is requested to submit five swab samples per week to the West of Scotland Specialist Virology Centre, Glasgow, UK for PCR testing for a range of respiratory pathogens.We also included results from swabbing carried out in primary and secondary care for routine diagnostic purposes in symptomatic patients outside the sentinel scheme.As a post-hoc sensitivity analysis, we excluded patients recruited from non-sentinel sources (n = 542; 16.3%) and those presenting symptoms less than 14 days after vaccination (n = 47; 1.4%).The West of Scotland Specialist Virology Centre is a World Health Organization-accredited National Influenza Center, which participates in the Quality Assurance programme to maintain this status.To calculate VE, patient swab data were linked with the unique patient identifier CHI, allowing characteristics of patients such as vaccination status to be established from general practice and hospital admission data.In 2005/06 when influenza B (B/ Malaysia/2506/2004) was the predominant circulating virus type, tests were performed in sufficient numbers to estimate VE against influenza B in that season.The Privacy Advisory Committee of the Information Services Division, National Services Scotland, approved the linkage and analysis of the anonymised datasets for this project.

Statistical methods
A generalised additive logistic regression model [24] was fitted, adjusting for the effects of week during season (through a separate spline model each season) and age, sex, deprivation, smoking status, number of primary care and hospital consultations in the previous 12 months, influenza vaccination in the previous season, and being in a clinical group at risk of serious complications from influenza.Some of these patients did not receive the influenza vaccine; some received the vaccine, but after they were tested; and others had received the vaccine before they were tested.We therefore measured VE by comparing swabs taken after vaccination from individuals who were vaccinated, with swabs taken from all those who were not vaccinated at the time the swab was taken (people who were unvaccinated at the time of swab and who were then subsequently vaccinated counted as unvaccinated in our analysis as did people who were never vaccinated).When two doses were given we used the date of the first vaccine dose in our analysis.We stratified our analysis by people 65 years and older vs people younger than 65 years and at risk, and also tested for any heterogeneity between seasons.We also tested for any heterogeneity or collinearity between receipt of current and previous season's influenza vaccination.
Using data from previous studies, we estimated that with 400 swabs per year, an effectiveness of 20% would be detected with 79% power for our primary outcome of PCR-confirmed influenza (assuming that 15% of the population would be vaccinated, 30% swab-positive and adjusting for clustering within each primary care practice [25,26]).All statistical analysis was conducted using R (version 2.14.1).

Results
A total of 3,323 swabs were taken from 3,016 patients with influenza symptoms over the nine seasons (of a total registered primary care population of 1,767,705 person-seasons) and then tested with RT-PCR for evidence of influenza infection.Some 489 swabs (14.7%) were performed on individuals who were vaccinated at the time of the swab.Although all subgroups were represented, proportionately more young, female, and socioeconomically deprived patients were swabbed (Table 2).Furthermore, a large proportion of the virological tests (42.3%) were carried out on patients that had presented more than five times to primary care in the previous year.During our study, 13.9% of swabs were positive for RT-PCR-confirmed influenza, with male patients and the socioeconomically affluent being more likely to test positive for influenza (Table 2).One quarter of the swabs from school-aged children (5-14 years) tested positive for RT-PCR-confirmed influenza.Pooled over nine seasons, VE for the trivalent inactivated influenza vaccine in the whole population was 57% (95% confidence interval (CI): 31-73) (Table 3).VE for at-risk patients under 65 years was 60% (95% CI: 22-79) and 19% (95% CI: −104 to 68) for 65 years and older.Although there was variability between seasons, no significant heterogeneity was found (p < 0.05); there were no positive tests among vaccinated people in 2000/01 and the highest VE was found in season 2007/08 (Table 4).In 2005/06 for influenza B, there were 44 positive tests in 426 unvaccinated and three in 137 vaccinated individuals.In that season, VE against influenza B was 79% (95% CI: 32-96).When including influenza vaccination in the previous season (for all nine seasons), the vaccine effect in the current year was 55% (95% CI: 22-74) and there was no significant effect of the vaccination in the previous year (OR = 0.89; 95% CI: 0.53-1.52).A post-hoc interaction test showed a significant (p = 0.01) interaction between receipt of the previous season's and the current season influenza vaccination and VE but no major collinearity.Table 5 presents the VEs of three possible combinations of vaccinated or unvaccinated in the current or previous season (combining all the seasons studied) compared with people with no vaccination in either season.Significant positive VEs were found for subgroups vaccinated in the current and previous season and those vaccinated in the current but not the previous season.A non-significant positive VE was found for people vaccinated in the previous season but not the current.VE was similar to our primary analysis when excluding virological tests from non-sentinel sources (60%; 95% CI: 31-77) or patients with onset of symptoms less than 14 days after vaccination (VE = 61%; 95% CI: 36-76).

Discussion
Our trivalent influenza VE using RT-PCR in symptomatic patients presenting over nine seasons was similar to the efficacy found in healthy adults younger than 65 years in controlled trials (66% vs 75%) [8].Our findings were also similar to other observational studies which pooled data across several seasons and estimated a VE of 61% for adults 50 years and older [15] Table 3 Proportion of vaccinated by case/control status and adjusted vaccine effectiveness for laboratory-confirmed influenza, Scotland, 2000-09 (n = 3,323)  and 62% for 20-64 year-olds [14].In a single season (2012/13) in which genetic drift of the predominant influenza A(H3N2) strain had occurred, the VE estimate for people 65 years and older was −11% against influenza A [27].This is lower than our pooled estimate of 19% VE for this age group over nine seasons with different circulating strains.In the 2007/08 season when vaccine and circulating virus were well matched and influenza A/Solomon Island/3/2006 (H1N1) was the main circulating virus, our VE was higher for laboratory-confirmed influenza than in a US study in the same season which used a study design similar to ours (80% vs 52%) [28].In the 2005/06 season when influenza B/Malaysia/2506/2004 was the predominant strain, our estimate of 29% was similar to a US study (21%) [29], but lower than reported in Canada (63%) [30].These differences in VE are likely to be due to between-country variation in the distribution of vaccine types, dominant circulating influenza types, subtypes, and lineages, and antigenic (mis)match between vaccine virus and circulating virus [31].Although there was poor precision, we found variations in VE over the seasons.In the two seasons when influenza A(H1N1) co-dominated or dominated and the vaccine was well matched (2000/01 and 2007/08, respectively), VE was high (≥ 80%).In 2003/04 and 2004/05 when vaccine mismatch occurred in the A(H3N2) component of the vaccine, VEs of 49% and 44% were found.In 2005/06 when there was vaccine mismatch for influenza B, a 79% VE for influenza B was found.This was similar to findings in a well-powered study in the same season on influenza B in England (67%; 95% CI: 31-85).In all other seasons, influenza A(H3N2) was the predominant influenza A subtype and VE varied from 22% (2006/07) to 77% (2001/02) [32].
Our finding of an interaction, whereby prior influenza vaccination interfered with current vaccine effectiveness, has been described previously in a communitybased study [33].Similar to that study, we were limited by a relatively small number of cases and were only able to dichotomise prior and current season vaccination status (yes or no).However, this simplified approach has been criticised and a more in-depth analysis has been suggested which includes the number, nature and antigenic distance specified by virus mutations across sequential circulating variants and vaccine components [34].This is a potential avenue for further work.
Clinical data collected by these sentinel practices are of high quality (90% completeness and accuracy [25]) and their value for epidemiological research has been repeatedly demonstrated [26].Observational studies can be used to assess the effects of healthcare interventions without influencing the care provided or the patients who receive it.When used in the assessment of vaccination programmes they therefore have high external validity and can be broadly generalised.Furthermore, by pooling data from nine seasons from the same population, we were able to generate sufficient power to provide a precise VE estimate.The test-negative design offers an elegant way to deal with selection bias that may arise if there is a strong association between vaccination status and subject recruitment.However, this design only measures the protection provided by the vaccine to individuals seeking medical attention, rather than VE against influenza, because for some persons (e.g.people with co-morbidities and at risk of serious complications from influenza), vaccination may not truly prevent influenza, but may reduce illness severity, preventing death or hospitalisation or reducing severity below their care-seeking threshold [35].If possible, one should therefore assess the likely impact of VE on disease severity [36] and the influence of non-influenza acute respiratory infections by restricting controls to those who tested negative for influenza and positive for a different respiratory pathogen (e.g.parainfluenza or respiratory syncytial viruses) [35,37].Swabs from symptomatic patients outside the systematically collected subset were included in our study, and this may have led to some selection bias, although physicians swabbing in secondary care (where the majority of non-sentinel swabbing took place) were unlikely to know the patient's vaccine status unless self-reported and a sensitivity analysis found no change to our VE estimates (but decreased their precision).However, even with the inclusion of these additional tests from non-sentinel sources, there was an over-representation of swabs from working-age adults and therefore we had lower power to measure VE among children and older people.There was also inadequate power to measure pooled estimates of VE for types or subtypes of influenza (e.g.A(H3N2), A(H1N1) and B), most individual seasons, patients with chronic diseases (e.g.asthma) or pregnancy (which was not included as a risk factor for this analysis) or for those given a second dose of the vaccine.A much larger study is therefore required to perform these stratified analyses.In our primary analysis, we considered that the vaccine effect was random over seasons rather than the seasons having random effect.In this pooled model we found that there was already a different intercept and seasonal trend each year and that this permitted more differences among the seasons compared with a random effects model.The random effects meta-analysis estimate was 51%, close to the pooled estimate reported in this paper (57%).Furthermore, treating each season equally gave a VE estimate of 65%.Some of the patients were found to have contributed with more than one swab in different seasons, with 231 people with swabs in two seasons, 27 with swabs in three seasons and six with swabs in more than three seasons.We therefore performed post-hoc sensitivity analyses using a generalised estimating equation (GEE) model and a clustered regression model.Both of these models were found to inflate the variance of the vaccine effect, but did not have a major impact on the conclusions.
Our primary objective was to make use of the best integrated and accessible Scottish data available to us to evaluate a new national influenza vaccination programme introduced in Scotland in September 2000.
During the period 2000 to 2009, seasonal influenza vaccination was provided to at-risk groups (at no cost to the patient) through primary care.This targeted approach resulted in high vaccine uptake rates of 66 to 76% in older people and 38 to 49% in at-risk groups [6].We found that during the period when the programme was implemented (and before pandemic influenza), which included seasons with poor vaccination match and severe influenza, there was strong evidence for the effectiveness of vaccination in preventing laboratoryconfirmed influenza, particularly for younger people and people susceptible to severe influenza-like illness.This information should reassure countries considering the implementation of a similar programme.However, while work is being undertaken to produce better vaccines and new vaccines are introduced, the continued development of a strong international evidence base is required to monitor the effectiveness of seasonal influenza vaccination programmes, particularly among subgroups of patients at risk of serious complications from influenza such as older people.

Introduction
In

Method
The sample size for the pilot study was calculated to produce validation results overall for the European PPS rather than at individual country specific level.A pilot ECDC PPS had indicated a prevalence of 7.1% and an average hospital size of 300 beds [2].The sample size for the pilot validation study was calculated at approximately 2,000 patients for an estimated sensitivity of reporting HAIs of 80% with a precision of +/-5% and a prevalence of 7% [3,4].These 2,000 patients were specified by the ECDC validation pilot protocol as approximately 200 patients per country within the 10 participating countries, sampled in at least two hospitals per country.
Two approaches, including a validation method by an external validation team (method 1) and an on-site assessment of the IRR of different hospital PPS data collectors (method 2), were taken in order to address the objectives.The methods are summarised here and a full description is available in the ECDC pilot validation protocol [4].

Validation
A standard ECDC protocol was used by all countries [3].Each country collected data on 100 patient records from each of two hospitals.The hospitals and the patient records were chosen by the national coordinators from each country and not randomly allocated at a country level.A number of approaches were taken including retrospective, simultaneous same day, simultaneous same time, blind and unblind data collection.
The approach undertaken by each country was purposively selected dependent on timing of the primary PPS and availability of resources.Countries also had an option of oversampling within the protocol, whereby the number of HAIs in the validation sample was increased on purpose to increase the precision of the specificity estimation, by selecting wards with higher prevalence (e.g.intensive care units) in blind validation or by including all HAI cases detected in the primary PPS in unblind validation.
The validation findings were considered the 'gold standard' (true positives and true negatives) as the validation team consisted of at least one trained expert from (and/or acting on behalf of) the national/regional PPS coordinating centre (external to the validated hospital), using the ECDC-PPS protocol and codebook [5] and accompanied by a hospital staff member for the purposes of access and orientation.
Identical data to the primary data collector were collected by the validator using one or more of the approaches outlined above.Patient notes, nursing notes, hospital information systems and clinical ward personal were the data sources used.
From the validation dataset, the positive predictive value (PPV) was calculated as the percentage of patients with true HAIs (or patients receiving antimicrobials as appropriate) among all positive patients in the primary dataset, and the negative predictive value (NPV) as the percentage of true negative cases among all patients identified as negative in the primary sample.The results of the validation were applied to the aggregated primary data by multiplying the number of all positive cases in the primary sample by the PPV to obtain an approximation of the number of true positives to account for potential differences in prevalence due to oversampling.The same procedure was performed for negative cases with the NPV.This allowed determination of the sensitivity and specificity for the primary sample [4,6].95% confidence intervals (CI) were calculated using a continuity-corrected version of the Wilson's score method.They were evaluated as 'worst case' instances using a combination method.
The effects of omitting these adjustments, in most cases did not result in major differences to the results presented here.

On site assessment of inter-rater reliability
Five HAI-positive and 10 HAI-negative patient records were selected from a single setting, i.e. intensive care unit (ICU), where the prevalence of infection was the highest, or, if access to the ICU was restricted, in a limited number of other wards with expected high HAI prevalence, such as high dependency units.
Between two and five hospital primary PPS data collectors gathered data at an agreed time in the selected ward/setting in turn with the national contact point (validator).A procedure was followed to minimise any potential bias inclusive of the other rater(s) waiting in another room or at a distance where the reproducibility process could not be heard (e.g.use music in the waiting room/area).Data items were collected as detailed in method 1 and agreement between the data collections was analysed using kappa (κ) statistics (0.81-1.00 is very good, 0.61-0.80 is good, 0.41-0.60 is moderate, 0.21-0.40 is fair/marginal, < 0.2 is poor; negative values are possible and also denote 'poor' agreement [7][8][9].κ statistics were also reported for certain variables of the validation approach (method 1), as it can be argued that the external validation team does not truly represent a gold standard for HAIs and variables such as the McCabe score.

Validation
The primary data set that originated from the 20 hospitals in the 10 participating countries comprised 3,958 patient records.Among these, the prevalence of HAIs was 9% (367 patients) and the prevalence of antimicrobial use was 38% (1,504 patients).Validation data were collected from October to December 2011.Of the 3,958 primary patient records, a total of 1,950 were selected for validation in accordance with the calculated study sample size.Of those, 1,912 were matched to the primary dataset, since it was not possible to link all patient records due to errors in data entry or missing data.The reported prevalence of HAIs in the matched validation dataset was 12% (233 patients) and the prevalence of antimicrobial use was 46% (878 patients).Due to oversampling in the validation dataset, the prevalence of HAIs in this dataset was significantly higher than in the 'primary' set, Χ 2 (1) = 7.7, p = 0.005.The prevalence of antimicrobial use was also higher in the validation dataset, Χ 2 (1) = 27.5, p < 0.001.Four of the ten countries included oversampling.
There was 97% agreement (κ = 0.81) between primary records and the validation records for the presence of an HAI (Table 1).The level of agreement was very good across all the most important types of HAIs, ranging from κ = 0.83 for bloodstream infections to κ =1.00 for lower respiratory tract infections (Table 2).Specificity of reporting HAIs was 98% (95% CI: 98-99%) with sensitivity comparatively lower at 83% (95% CI: 79-87%) (Table 1).The sensitivity by type of HAI ranged from 83% for bloodstream infections to 100% for lower respiratory tract infections, with specificity values higher than 99% for all types of HAI (Table 2).
Very good results (96% agreement, k=0.91) were achieved with respect to the recording of overall antimicrobial use (Table 3).Sensitivity and specificity were both very high at 94% (95% CI: 92.9-95.3%)and 97% (95% CI: 96.1-97.5%)respectively.Validation of the route of antimicrobial administration oral, parenteral showed that oral antimicrobials were frequently reported as parenteral, resulting in a lower specificity for the parental route and a lower sensitivity for the oral route (Table 4).
At the individual variable level, some variation was noted.Agreement on basic demographic variables was very good: age (κ=0.95), sex (κ=0.93),specialty of physician (κ=0.87).A high level of agreement was also found with respect to the presence of invasive devices, although specificity for the presence of peripheral vascular catheters (93%; 95% CI: 91-95%) was noted to be significantly lower than that of central venous catheters (99%; 95% CI: 98-99%).Variables which required more interpretation such as McCabe score had a moderate score (κ = 0.57).

Inter-rater reliability
Eight of ten countries participated in the IRR component of the pilot study with a total of 44 raters across all the participating hospitals, rating 195 patient records.An analysis of IRR by selected variables was undertaken on the dataset.Variables were selected on the basis of their importance and the frequency of reporting in the dataset.Analysis of IRR overall showed a very good level of agreement (κ = 0.92) for both the presence of HAIs (96%) and antimicrobial use (97%) (Table 5).
There was very good IRR (κ > 0.8) for most of the PPS variables (with the exception of HAI origin) (κ =0.31).

Discussion
Studies on validation of national HAI surveillance are rarely published, and when they are, a variety of approaches are described, according to a recent review in the United States (US) [10].In that review, of those that included either a validation or an IRR study, the results were varied, underscoring the need for PPS to include validation studies to add confidence to    The validation component identified an overall sensitivity of 83% (95% CI: 79-87%) and specificity of 98% (95% CI: 98-99%) for the presence of HAI.The level of agreement between the primary analysis and the validation assessment was very good across all the types of HAI.Previous studies indicated some variation at the level of individual types of HAI.In these studies respiratory tract infections had lower sensitivity, specificity and inter-rater reliability than other types of HAI [11][12][13].However, the results of this pilot study indicated a high level of specificity and a high level of agreement for these types of HAIs.It is likely that the training given to support ECDC PPS has had an impact on the good validity results in our pilot study, however, this is difficult to assess and, to our knowledge, no study has been published to date assessing the effect of training on data validity.Moreover, the relatively good sensitivity and specificity results found in our pilot study may have been influenced by the 'experimental' conditions (e.g.selection of two hospitals per country willing to participate), which may have resulted in higher sensitivity and specificity than would have been found in validation across a non-selected group of hospitals.Indeed, in four national validation surveys carried out in 2012 during the second phase of the ECDC PPS, the average sensitivity of reporting HAIs was 71.9%, considerably lower in our pilot study [1].The sensitivity (83%) in our study indicates potential underreporting of HAIs in the ECDC PPS.This underreporting of HAIs may have resulted from difficulties with application of definitions or availability of patient record information at the time of data collection.To the authors' knowledge, this was the first study which formally validated the reporting of antimicrobial use within a PPS study.Antimicrobial use had a high sensitivity of 94% (95% CI: 93-95%) and specificity of 97% (95% CI: 96-98%) with a very good level of agreement.Validation of the route of antimicrobial showed that oral antimicrobials were frequently reported as parental antimicrobials.
Other variables within the validation dataset were well recorded.A complex patient records review study by Yawn and Wollan (2005) [14] found that demographic data that required copying explicit information (e.g.sex, birth date), 'free-text' data that required identifying and copying (e.g.chief complaints and diagnoses), and data that required abstractor judgment in determining what to record (e.g.whether heart disease was considered) differed in terms of rates of agreement.In our study, agreement between the validation and the primary data collectors on more basic demographic variables ranged from moderate for the McCabe score to very good for sex and age.This finding was in line with the scarce literature published to date [11][12][15][16][17][18][19], wherein basic demographic variables such as age and sex tend to have very good levels of agreement compared to those variables where interpretation is required, such as the McCabe score or other markers of co-morbidity.The variables requiring abstractor judgment in this pilot validation study usually involved verification with a clinician present on the ward, which may account for the higher than expected validity.
The IRR component showed that the in-hospital IRR reliability was very good, for HAIs and antimicrobial use.This level of agreement was also found for other variables with the notable exception of HAI origin, which had a fair/marginal κappa.No studies that have looked formally at IRR in more than one country were identified in the literature.One study [17] did examine the difference between teams of data collectors within an Indonesian PPS and indicated that interobserver variation differed significantly between the teams of data collectors in terms of completeness of data, and most importantly in the number of detected HAIs.Differences of note in this previous study were with respect to surgical site infection, urinary tract infection and septicaemia (p = 0.01) and the reported agreement (κ) did not exceed 0.59 for any type of HAI [17].Their evaluation indicated that ascertainment was affected by underreporting in patient records, and the retrospective nature of data collection for validation purposes.While the issue of record keeping with respect to device use, infection criteria and antimicrobial use, had been identified as a potential limitation at the outset of this study, the outcomes in the validation aspects of this pilot study were better than expected in this regard.
The IRR results overall in this pilot study were better than those published previously in the literature with respect to presence of HAIs and types of HAI.
As with all observation studies of this nature there are a number of potential biases which are acknowledged herein.The first of these is the potential for selection bias as participating countries chose the hospitals and patient records; these were not randomly allocated at a country level.Observer bias potential is also acknowledged as not all the validators were blinded to the primary results although the high levels of IRR indicate minimal risk of this.
In summary, this pilot study suggested that the ECDC PPS dataset of HAIs and antimicrobial use was valid and reliable.Basic demographic data and antimicrobial use data had very good levels of validity and reliability and may not need to be routinely collected in future validation studies.The high specificity and IRR are an indication that the training on the case definitions organised during preparation of the ECDC PPS was effective.The lower sensitivity findings show the potential for underreporting of HAIs in the ECDC PPS and highlight the importance of validation studies for future surveillance activities in order for the burden of HAI to be better estimated.

Introduction
Surgical site infections (SSIs) are a major complication following surgery, causing an important increase in both postoperative morbidity and mortality and healthcare-associated costs [1].In the Netherlands, SSIs account for about 25% of healthcare-related infections [2], making them one of the most common nosocomial infections.Cumulative incidences of SSIs (commonly referred to as SSI rates) are considered the primary indicator of the quality of surgical and postoperative care.They are, therefore, an important measure in surveillance systems for healthcare-associated infections.
Identifying SSIs is multidimensional.Case finding using inpatient data may be homogeneous across hospitals: however, focusing only on inpatient data from the initial surgical admission is insufficient [3][4][5].As hospital stays have become increasingly shorter, a growing proportion of SSIs is recognised after discharge.Therefore, for measuring SSI incidence, post-discharge surveillance (PDS) has become inevitable.If no PDS is performed, the incidence of SSIs will be greatly underestimated [4][5][6] and comparisons between hospitals may be flawed.When PDS is performed, two important aspects influence the incidence of SSIs: duration of follow-up and method of follow-up.
Until recently, international consensus was that PDS should be performed up to 30 days, or, if an implant is inserted, one year after the operation [7,8].For reasons of simplicity and to reduce the burden of performing PDS, the United States Centers for Disease Control and Prevention (CDC) in summer 2013 decided to link the duration of PDS to the type of surgery instead of the presence of implants, and to reduce the maximum duration of PDS from one year to 90 days [9,10].
Although not officially published yet, for similar reasons, the European Centre for Disease Prevention and Control (ECDC) adopted the 90-day PDS for implant surgeries in 2014 (C.Suetens, personal communication, 15 December 2014).By making these changes, the international consensus on the recommended duration of PDS has been lost and is currently subject to research.
Whereas consensus on the duration of PDS is being sought, there is, however, still no international agreement about the preferred method of PDS.As a result, there is widespread use of various methods that may cause an underestimation of the incidence of SSIs [3,6,11,12].In the Dutch national nosocomial surveillance network (PREZIES), several methods of PDS were used until 2009.By 2009, however, two methods for PDS found to be superior, but labour intensive [3,11], became mandatory, as did the duration of PDS: 30 days (non-implant) or one year (implant).These two commitments have led to problems of delayed feedback and increased workload for healthcare professionals.
As swift communication of surveillance results after the surgery will help to stimulate healthcare professionals to act and improve, and as the previous international consensus on duration of PDS was lost, the main goal of our study was to investigate the effect of shorter PDS durations on incidence of SSIs, in order to determine a justifiable and advisable duration of PDS.We also aimed to quantify the impact of the mandatory PDS methods and 'other' PDS methods that were acceptable before 2009, in detecting more or less SSIs, and then we compared this impact with the effect of shorter PDS durations on SSI incidences.

Design, definitions and data selection
We used data from PREZIES, the Dutch national nosocomial surveillance network [3].Hospitals in the Netherlands participate voluntarily in this network and may select surgical procedures for SSI surveillance.PREZIES distinguishes superficial SSIs from deep SSIs, the latter being an umbrella term for so-called deep incisional and organ-space SSIs.In accordance with international guidelines, SSIs were defined as infections that originated within 30 days after surgery (deep and superficial SSIs) or one year after implant surgery (only deep SSIs) [7,8].An implant is defined as a non-human-derived, implantable foreign body that is permanently placed in a patient during surgery.All SSI surveillance data are collected locally by the hospitals.Further details on PREZIES and data collection, validation, and monitoring quality and reliability have been described previously [3].
SSIs occurring after discharge from hospital were detected by PDS.The operations in the PREZIES database were divided into two groups: those followed up using so-called 'mandatory' PDS methods and all other operations ('other' PDS) (Table 1).The mandatory PDS methods comprise two methods considered superior [11]: (i) use of a registration card; and/or (ii) retrospective examination of medical records for all operated patients.Both have a high sensitivity for capturing SSIs [11] and meet the following five requirements: they detect SSIs during the initial admission, readmission or outpatient time, as well as those treated at a different facility (except for surgeries exclusively performed by referral hospitals) and have a mandatory duration of either 30 days (implant free) or one year (implant used).These methods of PDS were recommended from 1998 to 2008 and became mandatory in 2009.In contrast to operations in the mandatory PDS group, those in the other PDS group were followed up using a variety of PDS methods not fulfilling the mandatory PDS requirements, including using no PDS at all.As such, they differ from the mandatory PDS methods in their way of case finding and/or in their duration.Although their follow-up ranges from no PDS to PDS methods similar to (but not qualifying for) the mandatory methods, the majority of the other PDS group consists of PDS performed by checking records of readmitted patients.

Table 1
Methods of post-discharge surveillance for surgical site infections, the Netherlands, 1999-2008 The date of last follow-up was not available from PREZIES data, but it was not necessary as we calculated duration of PDS according to mandatory PDS definitions: 30 days (implant free) or one year (implant used).In case of an SSI, the duration of PDS was the date of infection minus the date of surgery.As we wanted to compare the incidences for the group of other PDS methods from a 30-day or one-year perspective too, the PDS durations were calculated the same way for this group.
We selected data from a 10-year period (1999-2008), with the upper limit chosen to include data from both PDS groups (mandatory and other) for each selected year.Six surgical procedures were chosen for investigation: two procedures with implants (total hip replacement and knee replacement) and four implant-free procedures (breast amputation, colon resection, laparoscopic cholecystectomy and Caesarean section).SSI incidence was determined for each surgical procedure.

Statistical analysis
We analysed the effect of PDS duration on cumulative incidences of SSIs within both PDS groups using crude SSI incidences that combined deep and superficial SSIs.We plotted cumulative SSI incidences over time for both PDS groups (Kaplan-Meier survival techniques).To be able to compare the effect of better (mandatory) methods with the effect of shorter durations, we calculated crude SSI incidences for each surgical procedure after 90 days, 180 days and one year of PDS for implant procedures, or after 21 days and 30 days of PDS for implant-free surgical procedures.Durations of 21 days and 180 days were arbitrarily chosen but give insight into the timing of infections after surgery.Using the longest PDS duration as a gold standard, we quantified the proportion of SSIs that would be missed by shortening the PDS duration.Finally, for specific time intervals, we calculated the number of detected SSIs as a percentage of the total number at the end of the PDS.
We compared SSI incidences of both PDS groups for each type of surgery by calculating the relative risk of detecting an SSI (hazard ratio (HR)) for other PDS methods compared with mandatory PDS methods, while taking into account varying durations of the PDS (Cox regression analyses).To account for potential confounding (factors associated with the PDS group influencing the HR), we also performed multivariable Cox regression analyses.Several patient-specific and procedurespecific confounders were considered: the American Society of Anesthesiologists score (ASA, a physical status classification system) [13], wound contamination class [14], sex, age, year of surgery (accounting for yearly differences in hospitals' participation, with their differences in methods of PDS used) and duration of surgery.The potential confounders (determinants) were tested for their impact on the regression coefficient.A determinant altering the coefficient of the (univariate) analysis by 10% or more was considered a confounder and was included in the multivariable model by manual forward inclusion.This procedure was repeated for other potential confounders until the final model was constructed.For each type of surgery, we converted the resulting HRs into the proportion of SSIs that would be missed when choosing other instead of mandatory, methods of PDS (proportion missed = (1 − HR) × 100).Again, all analyses were performed at 21 days and 30 days (implant-free procedures) or at 90 days, 180 days and one year (implant procedures).Most Cox regression analyses were performed for deep and superficial SSIs, both in combination and separately.
Statistical significance was defined at 0.05, and a power of 80% was chosen.All statistical procedures were performed with SAS software, version 9.3 (SAS institute).

Data selection
From 1999 to 2008, PREZIES collected data on 234,841 surgical procedures.For the six surgical

Table 2
Details of patients whose operations (n=105,607) were included in the analysis of post-discharge surveillance of surgical site infections, the Netherlands, 1999-2008

Figure 1
Cumulative incidence of surgical site infections, the Netherlands, 1999-2008 (n=105,607) SSI: surgical site infection; PDS: post-discharge surveillance; PREZIES: Dutch national nosocomial surveillance network.Crude cumulative SSI incidence is shown over time for the 'mandatory' methods of PDS (blue line) and 'other' methods of PDS (green line) for the six surgical procedures.Mandatory PDS methods are considered superior and have been mandatory in PREZIES since 2009.'Other' PDS methods are all other methods used in the PREZIES surveillance of SSIs not meeting the criteria for mandatory PDS methods.Deep and superficial SSIs were combined.For the two procedures with implants (knee and total hip replacement), the maximum PDS duration is one year.For the other four procedures, the maximum PDS duration is 30 days.For both groups of PDS methods, the percentage of SSIs detected for several durations of PDS are presented.These percentage are crude, uncorrected percentages (cumulative incidences) and do not take into account any differences between both groups of methods.To directly compare both sets of PDS methods, see the results in Table 4. a The proportion of SSIs detected using the shortest PDS duration for that specific type of surgery (21 days or 90 days) compared with the longest PDS duration (30 days or 1 year).

Table 3
Effect of duration of post-discharge surveillance on the detection of surgical site infections, the Netherlands, 1999-2008 (n=105,607) procedures under investigation, data on 127,705 surgeries was available; 7,000 were excluded because it was unknown whether an implant was used.Another 11,819 records were excluded because either no use of implants was registered (knee replacement and total hip replacement) or use of implants was registered in a predominantly implant-free procedure (breast amputation, colon resection, laparoscopic cholecystectomy or Caesarean section).Finally, 3,279 surgical procedures were excluded because the method of PDS was unknown.Therefore, our results are based on data from 105,607 operations, which were collected at 87 hospital sites in the Netherlands.The patient characteristics are described in Table 2.

Effect of post-discharge surveillance duration on incidence of surgical site infections
Cumulative SSI incidences over time for both PDS groups for the six surgical procedures show that most SSIs were detected in the first weeks and months of follow-up (Figure 1).
The distribution of SSIs detected per time interval as a percentage of the total number of SSIs detected with the mandatory methods of PDS is shown in Figure 2.An additional 6% (3/53, cholecystectomy) to 17% (27/161, breast amputation) of all SSIs were detected in the final nine days of a 30-day follow-up (Figure 2, Table 3).Furthermore, 94% (700/748, hip) and 86% (292/340, knee) of all SSIs were already detected after 90 days of a one-year follow-up.After 180 days of follow-up, 97% (726/748) and 92% (313/340), respectively, were detected.The same analyses for the other methods of PDS yielded comparable results (Table 3).

Impact of post-discharge surveillance method on incidence of surgical site infections
Our results show an important difference between both PDS groups regarding the percentage of SSIs detected (Figure 1, Table 3).Crude SSI incidences at the end of PDS were lower for the other PDS methods (0.60% (12/1,984) to 11.17% (272/2,435)) than for the mandatory PDS methods (1.55% (93/6,007) to 14.35% (470/3,275)), except for hip replacements.
The HRs comparing the SSI incidences of other PDS methods with those of mandatory PDS methods, while accounting for differences between both groups, confirm a lower chance of detecting SSIs by other methods of PDS in five of the six types of surgery (statistically significant for four of the procedures) (Table 4).Analyses for a PDS duration of 21 days (implant-free procedures) and 180 days (knee replacement and total hip replacements) yielded similar results (data not shown).
Up to 62% of all SSIs (for cholecystectomy) may be missed during the surveillance when other methods of PDS are used instead of the mandatory methods.

Duration of post-discharge surveillance
Although the cumulative incidence of SSIs varied greatly between procedures, the number of new SSIs detected decreased during the PDS for all types of surgery; SSIs were detected primarily in the first weeks or months of the surveillance.A reduction in the PDS duration for implant procedures from one year to 90 days would potentially miss only 6% (hip replacement) and 14% (knee replacement) of SSIs.This would result in a decrease of the SSI incidence of 0.14% (from 2.26% to 2.12%) and 0.22% (from 1.58% to 1.36%) respectively, meaning that for every 714 hip or 455 knee replacements, one SSI would be missed.In other words, shortening the duration of PDS by nine months would not cause a substantial drop in SSI incidence.When the aim is to report SSIs for clinical purposes, missing even a small proportion of SSIs might be unacceptable.For surveillance purposes, however, not only the reliability of the SSI incidence but also the workload involved in PDS and the speed of feedback to the healthcare professional must be considered.It is acceptable for surveillance of SSIs to underestimate actual SSI incidence, as long there are other important advantages of the surveillance.The advantages of a shorter PDS seem not to outweigh the effect on the SSI incidence for implant-free surgeries.After all, up to 17% (breast amputation) of the identified SSIs were detected in the final period (days 22-30), and these additional nine days of PDS would not have a considerable impact on workload or swiftness of the feedback.For implant procedures, however, considering all the effort required to perform a one-year-long PDS and the advisability of returning surveillance results to healthcare professionals sooner rather than later, it would be worthwhile to shorten the recommended PDS duration.

Methods of post-discharge surveillance
SSI incidence varied between the types of procedure, but there was also a great variation in SSI incidence between the two groups of PDS methods.We found that the chances of detecting SSIs in implant-free procedures were lower (HRs varying from 0.38 (95% CI: 0.23-0.63) to 0.91 (95% CI: 0.77-1.07)when other, instead of mandatory, PDS methods were used.For implant surgery, the results were less straightforward.The crude SSI incidence did not differ significantly between the other and mandatory methods of PDS.
When adjusted for available confounders, however, the mandatory methods of PDS again resulted in significantly improved detection of SSIs for knee-replacement surgery (HR: 0.79 (95% CI: 0.64-0.98)).There may be several reasons why the mandatory PDS methods were not more sensitive for hip replacements.Firstly, due to the severe complications of a deep SSI in the hip joint, patients with deep SSIs following hip replacement are always readmitted.This makes other methods of PDS such as 'only checking readmitted patients' more sensitive for hip replacement surgery than for procedures with less severe SSIs.Another explanation could be our observation that the other PDS methods used for the surveillance of hip-(and to a lesser degree, knee-) replacement surgery were in general more similar to the mandatory methods of PDS than the other PDS methods used following other types of surgery.
In general, the decrease in detection of superficial SSIs by other methods was more noticeable than that of deep SSIs.The better detection of SSIs using mandatory methods of PDS can be explained by the fact that the these methods obviously aim at finding cases during more stages (during initial admission, readmission, outpatient time, and for those treated in another hospital) and does so for a mandatory period (30 days or one year).In addition, since superficial SSIs do not always require readmission and thus are more easily missed using 'other' PDS methods, the better detection of SSIs by mandatory PDS was logically more marked for superficial SSIs than for deep SSIs.
The Dutch mandatory methods of PDS are considered labour intensive due to the use of specific case-finding methods for several stages during the (mandatory) long period.Especially for PDS durations of one year, the surveillance work is generally carried out twice for each operated patient (after one to three months, and again after a year) or more frequently.Nevertheless, when compared with other methods of PDS, the increased detection of SSIs (up to 62% for cholecystectomy) justifies the use of the mandatory methods.We are convinced that the costs and time saved by shortening the mandatory PDS durations from one year to 90 days can be applied to improve and intensify existing methods of PDS.

Comparison with literature
Our study focused on the effect of both PDS duration and PDS method on SSI incidence and on the timing (accumulation) of SSIs.To the best of our knowledge, no European studies and only a few American and Canadian studies have analysed timing of SSIs to quantify the impact of a shorter PDS duration [10,[15][16][17][18]. None of these studies used survival techniques to visually demonstrate the accumulation of SSIs over time.Also, we are not aware of any studies in which multivariable Cox regression models were used to analyse the impact of method of PDS on incidence of SSIs.
Our results are in line with other studies investigating the effect of duration of PDS on SSI incidence.The vast majority of SSIs were detected within a 90-day window [10,[15][16][17][18], varying from about 70% [10] to 100% [17].After 90 days, only a few more SSIs were detected, triggering discussion about whether those late SSIs are truly due to preventable issues during the operation [16].Three of the five studies mentioned [15][16][17] confirm our finding that more SSIs are missed after knee-replacement surgery than after hip replacement surgery when the follow-up is reduced to 90 days.
Regarding the impact of different methods of PDS on SSI incidence, some studies have tried to compare the results of different methods of PDS [3,4,6,12,19,20].Similar to our results, most conclude that enhanced efforts to perform PDS result in an improved detection of deep and (more markedly) superficial SSIs [3,4,19,20].Thus, hospitals using improved surveillance methods will be 'penalised' with a higher incidence of SSIs, especially superficial SSIs.
To prevent hospitals being penalised in this way, Wilson et al. propose that the use of in-hospital incidence density (number of SSIs per 1,000 post-operative inpatient days) might be a more reliable indicator than cumulative incidence for comparison between hospitals or countries [21].After all, in-hospital case finding is probably more homogeneous across hospitals than post-discharge case finding, and, by focusing on inpatient data alone, differences in methods of PDS do not influence the indicator.By focusing on inpatient data, however, the differences in post-operative hospital stay largely influence the number of SSIs detected.
When the cumulative incidence is linearly related to length of time after surgery, the incidence density adequately adjusts for these differences in post-operative hospital stay.However, as we have shown in our analysis, the cumulative incidence is not a linear function of time after surgery, but has an S-shaped curve (Figure 1).As a result, calculations of in-hospital incidence density depend on the duration of post-operative hospital stay.During the first 10-14 days after surgery, a longer hospital stay leads to a higher incidence density (slope of line drawn from origin O to a point on the steep part of the curve).After that, however, the incidence density will decrease again (slope of a line drawn from origin O to a point on the flattened right part of the curve).Thus, even with perfect detection of SSIs, the in-hospital incidence density depends on the average duration of admission after surgery, and will be progressively underestimated as the time after surgery increases beyond two weeks.Therefore, we are convinced that a fixed, mandatory duration of following patients after surgery should be recommended to keep data comparable.The second-best option, if durations of follow-up do differ, the use of incidence densities could be considered but certainly has many limitations.

Strengths and limitations of our study
The strength of our study is that we gained insight into cumulative SSI incidences over time using survival techniques.Additionally, besides comparing crude SSI incidences, we used multivariable regression techniques to compare both groups of PDS methods, which allowed for correction for possible confounders.The correction, however, was probably not complete, resulting in some residual confounding in our analyses.
For instance, since method of PDS was obviously not randomised within the hospitals, we corrected for differences in PDS by using 'year of surgery' as a proxy (to account for yearly differences in hospitals' participation with their differences in methods of PDS used).Since we used a proxy, this correction is probably incomplete.On the other hand, it seems unlikely that our results can be explained by hospitals with lower 'true' SSI incidences systematically choosing other methods of PDS instead of mandatory methods.We would rather expect the opposite, which would result in an underestimation of the effect found in this study.Finally, we are convinced that by using surveillance data, we studied daily practice; therefore, the study itself caused no selection or changes in professional behaviour for the detection of SSIs.
A potential weakness is the fact that sometimes there was not enough power to significantly identify (small) differences, especially for surgical procedures with a low SSI incidence, or for procedures with a relatively small number of operations, or both.Another point is that we included two orthopaedic procedures for the implant surgeries; we could not include other implant procedures (for example, breast-enlargement surgery) because not enough data were available.Also, the Dutch mandatory methods of PDS are not an international gold standard for detecting SSIs, and the analysed group of other PDS methods is a collection of several different methods of PDS.And finally, since durations for mandatory PDS were a mandatory 30 days or one year, we accordingly assumed all followup durations in absence of an SSI to be 30 days or one year.If there were any records in the mandatory PDS group that did not completely comply with the protocol for mandatory PDS, this assumption may have been incorrect.This may have caused an underestimation of SSI incidence for the mandatory PDS group, making the detected differences between both PDS groups smaller.However, using the experience from our visits to the hospitals (as part of our quality assurance system) [22], we are of the opinion that this effect is either non-existent or negligible.
In the Dutch surveillance network, many hospitals chose knee-or hip-replacement surgeries for their surveillance of SSIs.From our visits to the hospitals [22], we noticed that the other PDS methods for these surgeries were often more similar to the mandatory PDS methods (and hence of higher quality regarding detection of SSIs) than those for procedures less frequently included in the surveillance.Therefore, the results of comparing both PDS groups for hip replacements and knee replacements may not simply be generalised to all implant surgeries.However, the effect of shortening the PDS duration did not differ between both groups of PDS methods.We, therefore, hypothesise that our conclusions and advice regarding shortening the PDS duration for knee replacements and total hip replacements may be generalised to other implant procedures.
Nevertheless, we consider it prudent to perform similar analyses for other implant procedures to confirm our results.

Recent developments and implications
In July 2013, the United States CDC's National Healthcare Safety Network (NHSN) reduced the maximum PDS duration from one year to 90 days [9,10].This change was made on the recommendation of an SSI surveillance working group (CDC working with clinical partners) that was supported by the Healthcare Infection Control Practices Advisory Committee (HICPAC).Reasons for changing the PDS duration were that 'The benefits include simplicity, a shorter followup time for many procedures that will reduce burden, and an opportunity to intensify post-discharge surveillance efforts for a shorter follow-up period' [10] and the data presented to support the NHSN decision [10,[16][17][18] are in line with our results.Another advantage of a shorter PDS duration could be that those hospitals and countries currently investing more time and energy than others in the final nine months of surveillance will no longer be 'penalised' with higher SSI incidences for their efforts.This will make inter-hospital or intercountry comparisons of SSI incidences more valuable, although ranking of hospitals or countries based on SSI incidence should be avoided [23].

Conclusion and recommendation
A one-year PDS for hip-and knee-replacement surgery no longer seems justified, since a 90-day PDS would capture the majority of the SSIs equally as well.Maintaining a PDS duration of 30 days for implant-free surgeries, however, is still recommended.
Although a small proportion of the SSIs would be missed for implant procedures, shortening the duration of PDS to 90 days would substantially facilitate prompt feedback to healthcare professionals and reduce workload for those performing the surveillance.Also, we conclude that choosing a method of PDS superior in detecting SSIs (such as the Dutch mandatory methods) is at least as important as choosing a sensible duration of PDS, because inferior methods of PDS may lead to greater underestimation of SSI incidence than shorter PDS durations do.
Our data validate international recommendations to limit the duration of PDS for implant surgeries to 90 days for surveillance purposes, as this provides robust insight into trends.Costs and time saved by shortening the duration of PDS can be applied to improve the methods of PDS.

Figure 2
Figure 2Flowchart showing exclusion criteria, study of mid-season influenza vaccine effectiveness in preventing hospital admissions related to influenza, Valencia, 7 December 2014-28 January 2015 (n = 3,988 )

Table 2
Number of swabs vs laboratory-confirmed influenza, by population group, Scotland, 2000-09 (n = 3,323) AOR: adjusted odds ratio; CI: confidence interval; NA: not applicable.a Adjusted for season, week during season (through a separate spline model each season), age, sex, previous season's influenza vaccination, consultations and socioeconomic deprivation.b Deprivation score only available for 3,313 swabs.c 1 = most socioeconomically deprived.
Adjusted for season, week during season, sex, number of hospital and primary care consultations, socioeconomic deprivation and being in a clinical at-risk group (where appropriate).
a b All patients including clinically at risk.

Table 4
Vaccine effectiveness for laboratory-confirmed influenza and predominant circulating influenza by season, Scotland, 2000-09 (n = 3,323) CI: confidence interval; NA: not applicable.a Poorly matched vaccine.

Table 5
Vaccine effectiveness for the combined influenza vaccinations in the previous and current season, Scotland, for all seasons (n = 3,323)

Table 1
Validation of the point prevalence survey for assessing healthcare-associated infections, 10 European Union Member States, 2011 A. Validation of the point prevalence survey (n=1,912 patient records) The 10 European Union Member States that took part in the validation part of the study were Bulgaria, Finland, Germany, Hungary, Italy, Latvia, Lithuania, Poland, Spain, United Kingdom.a 304 = PPV*367.b 3,529 = NPV*3,591.

Table 2
Validation of the point prevalence survey for healthcare-associated infections (HAIs), by type of HAI, 10 European Union Member States, 2011 (n=1,912 patient records) a Number of patient records used for validation, which could be matched to those reported in the primary point prevalence survey data.b 95% CIs have been adjusted to the overall prevalence among the primary cases (9%). a

Table 3
Validation of the point prevalence survey for assessing antimicrobial use, 10 European Union Member States, 2011 A. Validation of the point prevalence survey (n=1,912 patient records) B. Results of the validation study applied to the total primary point prevalence survey (n=3,958 patient records) CI: confidence interval; NPV: negative predictive value; PPV: positive predictive value.The 10 European Union Member States that took part in the validation part of the study were Bulgaria, Finland, Germany, Hungary, Italy, Latvia, Lithuania, Poland, Spain, United Kingdom.a Number of patient records used for validation, which could be matched to those reported in the primary point prevalence survey data.b 95% CIs have been adjusted to the overall prevalence among the primary cases (38%).

Table 4
Validation of the point prevalence survey for antimicrobial use, by administration route, 10 European Union Member States, 2011 (n=1,912 patient records) ainterpretation of the data.This study is the first multicountry validation study undertaken on the first ECDC PPS dataset.Based on the findings, a revised protocol for data validation of the PPS of HAIs and antimicrobial use in European acute care hospitals was made available in 2012.This protocol might be helpful to other countries considering similar studies in the future[4].

Table 5
Inter-rater reliability results by selected variables, pilot validation study of a point prevalence survey on healthcare-associated infections and antimicrobial use, eight European Union Member States, 2011 (n=44 raters and 195 patient records) Methods of PDS considered superior and being used mandatorily in PREZIES since 2009.For instance, registration card in medical records of a selection of patients, retrospective examination of the medical records of a selection of patients, questionnaires of the patient and/or surgeon, interview of the patient by phone, etc.
a b Method of case finding not suitable for surgeries exclusively performed by referral hospitals.c All other ways of follow-up used in PREZIES not meeting the criteria for mandatory PDS methods, including performing no PDS.d Not always reliable.e Distribution of site infections detected by post-discharge surveillance, by time interval, the Netherlands, 1999-2008 (n=68,635) A. Surgery without implants: Caesarean section, breast amputation, laparoscopic cholecystectomy and colon resection B. Surgery with implants: total hip replacement and knee replacement SSI: surgical site infection; PDS: post-discharge surveillance; PREZIES: Dutch national nosocomial surveillance network.The proportion of SSIs detected during each PDS time interval is displayed as a percentage of the total number of SSIs detected.Only SSIs detected by the 'mandatory' methods of PDS are presented; deep and superficial SSIs combined.For the four procedures without implants (panel A) the PDS duration (30 days) is divided into four intervals; for the two procedures with implants (panel B) the PDS duration of one year) is divided into five intervals.Mandatory PDS methods are considered superior and have been mandatory in PREZIES since 2009.
Dutch national nosocomial surveillance network.Mandatory PDS methods are considered superior and have been mandatory in PREZIES since 2009.'Other' PDS methods are all other methods used in the PREZIES surveillance of SSIs not meeting the criteria for mandatory PDS methods.
Mandatory PDS methods are considered superior and have been mandatory in PREZIES since 2009.'Other' PDS methods are all other methods used in the PREZIES surveillance of SSIs not meeting the criteria for mandatory PDS methods.
96).For the remaining four types of surgery showing a decreased SSI incidence when other PDS methods were used, the incidence of superficial SSIs as well as deep SSIs dropped, although the latter especially was not always statistically significant.HRs and 95% CIs for superficial and deep SSIs respectively were: Caesarean section 0.35 (0.21-0.70, superficial) a No confounding detected; univariate hazard ratio presented.