Late season interim estimates of influenza vaccine effectiveness reliably predict end of season estimates in Victoria , Australia , 2007 to 2012

Twice each year the World Health Organization makes a recommendation for the composition of the influenza vaccine, based on circulating strains of influenza A(H3N2), A(H1N1) and B. Strain selection has always been based on immunogenicity studies with limited human data. Immunogenicity can be considered as a proxy for vaccine effectiveness (VE). However, only interim VE estimates for the target hemisphere can be considered in time for the strain selection meeting. Using surveillance data from Victoria, Australia, we retrospectively estimated and compared interim and final VE estimates for 2007 to 2012. In general, interim estimates were within five percentage points of final estimates. However, estimates made too early or in years of low influenza activity may be unreliable.


Introduction
Twice every year, the World Health Organization (WHO) hosts an influenza vaccine strain selection meeting where data gathered by members of the Global Influenza Surveillance and Response System (GISRS) are reviewed and used to generate formal recommendations for the composition of seasonal influenza vaccines [1,2].Recommendations for the northern hemisphere vaccine are made in February and for the southern hemisphere in September.Strain selection is based on serological data, with human data used to estimate immunogenicity, generally considered as a proxy for vaccine effectiveness (VE).At the February 2013 meeting, epidemiological data were submitted reporting interim VE estimates from surveillance systems in Canada, Europe and the United States (US).These estimates were published [3][4][5][6][7][8] and included for the first time with the package reviewed by GISRS meeting members.Final season estimates for the northern hemisphere were recently presented at the September 2013 meeting, as well as interim estimates for the southern hemisphere.
Interim estimates are vulnerable to change.First, as the season reaches its peak more data become available.For example, early US interim estimates released for the period from 3 December to 2 January 2013 [9] suggested a crude VE of 62% (95% confidence interval (CI): 51 to 71), while a second, lower interim estimate of 51% (95% CI: 43 to 58) was released later in February, with an adjusted estimate of 56% (95% CI: 47 to 63) [6].The second interim estimate was made after the peak of influenza circulation had been reached, included more than double the sample size and had more complete information on covariates for adjustment and/or exclusion.Interim and final season estimates might also be expected to differ when the predominant type or subtype shifts within a season.For example, the early adjusted VE estimate from pooled European Influenza Monitoring Vaccine Effectiveness (I-MOVE) data in 2010/11 was 42% (95% CI: −7 to 69) [10] while the final estimate was 52% (95% CI: 30 to 67) [11].In the interim analysis, 77% of viruses were influenza A(H1)pdm09 and 21% were influenza B, whereas the final analysis included 58% influenza A(H1)pdm09 and 38% influenza B viruses [10,11].In the 2011/12 season final European estimates against influenza A(H3) were all revised down [12][13][14].This was attributed to waning immunity against A(H3), a phenomenon that is being further investigated.
In the 2007/08 season, Belongia and colleagues compared interim and final estimates from US data, observing a difference of about seven percentage points (44%; 95% CI: 11 to 65 versus 37%; 95% CI: 22 to 49) [15].They concluded that interim estimates were a useful indicator of VE mid-season.Systematic comparison of interim and final estimates has not been done since and has not been done for multiple seasons.To assess whether interim estimates reliably predicted final estimates, we compared retrospective interim and final VE estimates against influenza A and B for six influenza seasons, from 2007 to 2012, in Victoria, Australia.

Methods
We used data collected as part of the Victorian General Practice Sentinel Surveillance network for the years 2007 to 2012 to calculate retrospective interim and final estimates.This network has been described in detail elsewhere [16].Briefly, recruitment follows the case test-negative design [17][18][19]: a subset of patients seeing their general practitioner (GP) for influenzalike-illness (ILI; combination fever (measured or history of), cough and fatigue [20]) during the southern hemisphere influenza surveillance period are recruited at the GP's discretion, swabbed and tested for influenza by real-time reverse transcription-polymerase chain reaction (RT-PCR).Those testing positive for influenza A or B are cases and those testing negative are non-cases.The GPs collect demographic data (age, sex), symptom onset date, vaccination status, vaccination date and, since 2011, the presence of conditions predisposing the patient to severe influenza (chronic heart disease, chronic respiratory disease, diabetes, impaired immunity, obesity, pregnancy).Surveillance generally begins in epidemiological week 18 in April/ May and ends in week 44 in October/November.VE estimates from the sentinel network have been reported for all years from to 2012 [21][22][23][24].The   present analysis compared interim with end-of-surveillance period estimates, where the interim period ended to coincide with the WHO vaccine strain selection meeting in September.This meeting usually falls around week 38, so the interim period was restricted to weeks 18 to 36, to hypothetically allow two weeks to generate estimates and submit the report to the WHO.Final estimates were calculated using the entire surveillance period (weeks 18-44).For simplicity, we used all available data for weeks 18 to 36 or weeks 18 to 44, without considering other markers for influenza activity (such as ILI or laboratory indicators), which we have previously shown can influence VE estimates in seasons when the VE estimates are not robust [25].
VE was estimated as (1 − OR) using logistic regression.Patients were considered vaccinated if they had received the vaccine ≥14 days prior to the onset of symptoms and excluded if vaccination took place <14 days.Patients were considered influenza-positive if they tested positive to any of influenza A(H1), A(H3) or influenza B viruses by real-time RT-PCR, but no separate analyses were conducted for type or subtype.Models were adjusted for age group (<18, 18-64, ≥65 years) and week of presentation.The sensitivity of the estimates was tested in four ways: (i) the end of the interim period was brought forward by one and two weeks; (ii) final estimates excluded patients presenting more than eight days after symptom onset to reduce the possibility of false negative results, an exclusion which may not be possible in an interim analysis; (iii) estimates were restricted to people in a target group for vaccination (people with predisposing conditions or aged ≥65 years); and (iv) different variables for adjustment were used in the interim and final models.The fourth sensitivity analysis was based on the likely scenario where some information would be missing for the interim analysis, such as complete data on the presence of a condition predisposing to severe influenza and the date of onset, and where a decision was made to change the age groups used to increase comparability with other studies.Thus, the model for the final VE estimate included a variable representing the presence of at least one comorbid condition and a variable indicating the time between onset and consultation, an additional age group was added (<18, 18-44, 45-64, ≥65 years) and month was used instead of week to denote calendar time.The third and fourth sensitivity analyses were restricted to the years 2011 and 2012 because data on predisposing conditions were only available for these two years.

Results
The data available for each year are shown in Figure 1 with lines indicating the end of the interim period.For all years, the peak of the season preceded the end of the interim period.The characteristics of participants were not different for interim and final estimates (Table ).
Interim and final estimates were determined using the same model adjusting only for age group and week.There were no statistical differences between estimates, with point estimates varying by up to five percentage points (Figure 2).For 2007, 2008, 2010 and 2011 interim point estimates were lower than final estimates, while for other years interim estimates were higher.In the first sensitivity analysis, when the interim period was shortened by one week, there continued to be little difference in estimates for all years except 2008, where estimates differed by more than 10 percentage points.Shortened by a further week, estimates for 2008 continued to show great variability, and the direction of effect was reversed.
The second sensitivity analysis excluded people presenting more than eight days since symptoms onset (or for whom the onset date was not recorded), resulting in the exclusion of 437 people from the analysis (

Discussion
We found that interim VE estimates over six influenza seasons closely approximated final estimates when the interim period was limited to week 36.When the interim period was shortened, estimates for 2008 were different by more than ten percentage points.Estimates for 2008 showed the greatest instability, which may be explained by that year's smaller sample size and the timing of the season, the peak of which fell later than in other years in week 35 (Figure 1).Only one interim estimate has previously been reported for this surveillance network, for the 2009 pandemic [26], a season    (18) 88 (19) 60 (20) 70 (18) 195 (20) 205 (20) 78 (21) 92 (20) 63 (12) 84 (13) 152 (24) 27) 180 ( 28 which started and peaked earlier than in other years.The final estimate for that year used a more complete model with data collected over a much longer period but the VE point estimate was the same, and differed by only four percentage points when the analysis was restricted to the weeks of maximum influenza activity [22].Thus, interim estimates may be their most reliable when made after the peak, which is more likely in seasons which start early.For weeks 18 to 36 of the 2013 season, our interim estimate at the time of writing was 43% (95% CI: −30 to 75).However, like 2008, the 2013 season has been characterised by a late start and relatively low activity.In addition the VE estimate was unusually sensitive to the model used.Consequently, we expect the interim estimate for 2013 to be less reliable than in other years.
We expected interim estimates would be their least reliable when made for specific, smaller groups, such as people in a target group for vaccination.Moreover in the presence of waning VE within a season, it would be expected that the distribution of vaccinated cases would be skewed towards the end of the season, resulting in a lower final VE estimate.For example, in the 2011/12 European season, pooled estimates against A(H3) for people in a target group for vaccination were 43% (95% CI: −0.4 to 68) in the interim [27] and 25% (95% CI: −6 to 46) at the end of the season [12].Similarly, our final estimates for people in a target group for vaccination in 2011 declined 13 percentage points, from 61% to 48%.Conversely, estimates increased two percentage points in 2012 but the sample size was larger in 2012 and the season peaked earlier.Meaningful differences between interim and final estimates might also be expected when making estimates separately for each type/subtype, as the distribution of cases may be skewed or the number of exposed cases may be small.

Figure 2
Interim and final vaccine effectiveness estimates against influenza A and B for six influenza seasons, Victoria, Australia, 2007-2012 CI: confidence interval; ID: identity.All models adjusted for age group (<18, 18-64, ≥65 years) and week of presentation.Oldest age group dropped from all models in 2010 due to perfect prediction (no unvaccinated cases).Final refers to the period defined by weeks 18 Estimates varied little when the model used for final estimates was altered, as was done in the second and fourth sensitivity analyses.The restriction of patients to include only those presenting within eight days was done to reduce the possibility of false negatives.However, this modification to the model may not be expected to alter estimates, as imperfect, nondifferential sensitivity in the presence of perfect specificity will not usually bias estimates [19,28].In contrast, modifying the covariates in the model might be expected to have a greater impact on both point estimates (by removing or introducing bias) and precision.By comparison with the principle analysis, we observed modest changes using a larger model, suggesting the more parsimonious model would have sufficed for these data.However, this may not always be the case.In some circumstances, larger models can reduce the effective sample size used due to complete or quasi-complete separation (also known as perfect prediction), which can inflate estimates and reduce precision [29].For the data reported here, perfect prediction led to the loss of the oldest age group in the 2010 analyses.This problem has also been reported by the I-MOVE investigators when including calendar time (week) as a categorical variable, due to perfect prediction within a week or weeks [11].In such cases, it may be preferable to employ methods such as exact logistic regression or penalised likelihood estimation to avoid generating biased estimates [29].
The relatively recent adoption of the test-negative study design [17][18][19] has permitted rapid dissemination of VE estimates on a yearly and interim basis, and consideration of these estimates in vaccine strain selection meetings has been suggested for some time [30].However at this early stage, VE estimates are unlikely to influence strain selection; VE studies are less developed than the immunogenicity studies that have been used for decades to guide strain selection and should not yet be expected to provide reliable estimates of VE by type, sub-type, age-group and target group.However, our results illustrate the likely range of protection afforded by trivalent influenza vaccines (the only vaccines licensed in Australia) and support the use of late interim estimates as a proxy for final estimates.As VE studies evolve, their usefulness for strain selection should improve.The results presented here are hypothetical comparisons using cleaned data.It would be instructive to see a similar retrospective comparison for countries in the northern hemisphere.

Figure 1
Figure 1Number of influenza-like-illness consultations per week by case status for the Victorian general practice sentinel surveillance network, Victoria, Australia, 2007-2012

Final
Figures may not sum to 'Total included' due to missing data.Complete information on sex was not a criterion for inclusion in the analysis.