+ Author affiliations
Citation style for this article: Leval A, Hergens MP, Persson K, Örtqvist Å. Real-time real-world analysis of seasonal influenza vaccine effectiveness: method development and assessment of a population-based cohort in Stockholm County, Sweden, seasons 2011/12 to 2014/15. Euro Surveill. 2016;21(43):pii=30381. DOI: http://dx.doi.org/10.2807/1560-7917.ES.2016.21.43.30381
Received:20 May 2016; Accepted:12 September 2016
Annual vaccination against circulating influenza viruses remains the best strategy for preventing illness from influenza. A clear challenge, however, is that vaccine effectiveness (VE) varies from year to year . These variations may be due to differences in antigenic match between the vaccine and the circulating strain, the immune status of those who are being vaccinated, or the time interval between vaccination and influenza outbreak.
Influenza outcome specificity is an important factor affecting VE estimates, since outcomes with low specificity will either underestimate or overestimate influenza VE [2,3]. Seasonal influenza VE uncertainty is an important reason for obtaining estimates for in-season vaccine effectiveness (IVE) as early as possible [2,4,5]. Such estimates may help guide the outbreak response, especially if there are signs of an antigenic mismatch that might require complementary public health measures.
There are controversies concerning the overall influenza VE, especially in elderly people, in most studies defined as adults ≥ 65 years of age [6,7]. Real-world evidence of vaccine effectiveness is therefore imperative for future influenza vaccine development and programme evaluation. The seasonal influenza vaccination programme in Stockholm offers vaccination at no out-of-pocket cost to individuals aged 65 years and older, pregnant women, and people of any age with certain underlying risk factors (chronic diseases of the heart, lungs, kidneys or liver, diabetes mellitus, neurological disease affecting the patient’s lung function, obesity with a body mass index of > 40, and immunosuppression caused by a disease or treatment). The actual benefit to these targeted groups is largely unknown and the aim of this study was therefore to develop methods for evaluating IVE and the overall seasonal vaccine effectiveness (VE) in all persons, irrespective of underlying risk factors, with medically attended influenza and pneumonia hospitalisations and primary care cases in Stockholm County, Sweden.
Study population and period
This study was based on four annual closed cohorts each comprising all individuals registered in Stockholm at the start of each season. The influenza season was defined as starting on 1 October and ending on 31 May the following year.
Data were collected using Stockholm County’s central database for healthcare utilisation, consultations and diagnoses, VAL. VAL has comprehensive inpatient, hospital outpatient, and primary care data and is used by the County Council to update the national patient register (PR) . Multiple register linkages are possible due to unique personal identification numbers (PIN). Age and sex were retrieved from the primary care listing register in VAL. Immigration and death dates were not available in VAL, necessitating the design of a closed cohort for each season. We used the Stockholm Mosaic system as a proxy for living conditions and socioeconomic status . The Mosaic system is based on eleven mutually exclusive categories (e.g. living in a low-income urban apartment block, multicultural suburb, affluent inner city, countryside, etc.) and involves 120 smaller urban agglomerations. Data on vaccine exposures were retrieved from the vaccination register, Vaccinera, which contains all data on seasonal influenza, pandemic influenza and pneumococcal vaccination of persons belonging to medical risk groups from the region, since 2009. Regional coverage in this database is assumed to be 100% as high-risk persons are vaccinated free of charge within the programme and registration is mandatory and required for reimbursements to the healthcare provider. Data on influenza status and comorbidities were obtained from the inpatient, hospital outpatient, and primary care databases.
Cases were defined as a clinical diagnosis of influenza during the season. International Classification of Diseases, 10th revision (ICD-10) codes J09 (influenza due to certain identified influenza viruses), J10 (influenza due to other identified influenza virus) and J11 (influenza due to unidentified influenza virus with pneumonia) were used to identify influenza diagnoses from inpatient, hospital outpatient, and primary care registers in VAL . In a recent study VAL had over 99% coverage for inpatient care, 90% coverage for hospital outpatient care, and estimated 85% coverage for primary care . National-level reporting estimates a validity of 85–95% for inpatient care, depending on the ICD-10 diagnosis . Influenza cases were classified as inpatient cases if they came from the inpatient register and as outpatient cases if they came from the hospital outpatient or primary care registers. The inpatient register defined the case if an individual existed in multiple registers.
For the purpose of subanalysis, inpatient or outpatient non-influenza pneumonia, using ICD-10 codes J12-J18, was allowed.
Comorbidities were extracted from VAL using ICD-10 codes registered for a period of up to three years before the start of the respective season. ICD-10 codes for tumours (C00-D48), diabetes (E10–14) and circulatory (I00-I99) and non-acute respiratory illness (J40-J99) were extracted.
Vaccination dates and seasonal vaccine type were derived from Vaccinera. Three different trivalent inactivated vaccines, Vaxigrip (Sanofi Pasteur MSD, Lyon, France), Fluarix (GSK, Brentford, United Kingdom), and Inflexal V (Crucell, Janssen Vaccines, Leiden, The Netherlands), were used during the seasons covered. No high-dose or adjuvanted vaccines were available in Sweden during the four seasons. Individuals with influenza infection before vaccination, or up to 13 days post-vaccination, were considered to be unvaccinated as were those who did not receive the seasonal vaccine. Those with influenza infection ≥ 14 days post vaccination were considered to be vaccinated. Pandemic influenza (Pandemrix, GSK) vaccination status from 2009/10 was included as a covariate as was pneumococcal vaccination (in the current season or previous seasons since 2009). Vaccination against seasonal influenza in the previous season was also included as a covariate.
According to the Public Health Agency of Sweden, when compared with previous seasons, influenza activity was high during the most recent of the four seasons (2014/15), moderate during the 2011/12 and 2012/13 seasons, and low during the 2013/14 season  (Figure). Influenza A(H3N2) dominated in the 2011/12 and 2014/15 seasons, influenza A(H1N1)pdm09 dominated in 2013/14, whereas both these and influenza B viruses circulated in the 2012/13 season. There was also a significant amount of influenza B cases (approximately one-third of the cases) in 2014/15. In all four seasons influenza peaked during the second half of February.
Number and incidence of laboratory-confirmed influenza cases, and number of patients hospitalised with influenza diagnosis in Stockholm County, influenza seasons 2011/12–2014/15
ICD: International Classification of Diseases.
Unadjusted incidence calculated by number of laboratory-confirmed cases per 100,000 inhabitants. Numbers reported by calendar week each season.
No data were available on number of laboratory-confirmed hospitalised influenza cases due to anonymous data in the central database for healthcare utilisation, making linkage impossible.
a Data obtained from the Public Health Agency of Sweden
b Data collected in Stockholm County’s central database for healthcare utilisation using ICD-10 codes J09-J11
c (Number of laboratory-confirmed cases/number of inhabitants in Stockholm during season) × 100,000.
Hazard rate ratios (HRR) comparing influenza inpatient and outpatient incidence among vaccinated and unvaccinated individuals were calculated using Cox regression analyses. Models were adjusted for age (grouped into 10-year intervals), sex, comorbidity status, socioeconomic status, pandemic vaccination, previous season influenza vaccination and pneumococcal vaccination. Stratified analysis of elderly people, aged 65 years or older, and individuals with underlying chronic illnesses was also performed, including age as a linear variable. Vaccination status was included as a time-varying exposure in the model, so individuals could contribute both vaccinated and unvaccinated risk time. In the final model comorbidity was adjusted for as a dichotomous variable as yes or no. The overall seasonal influenza vaccine effectiveness (VE) was calculated as (1 − HRR) x 100%. Both HRR and VE were reported with 95% confidence intervals (CI).
Additional regression analyses modelled VE on inpatient and outpatient pneumonia (ICD-10 J12-J18), adjusting for age (grouped into 10-year intervals), sex, comorbidity status, pandemic vaccination, previous season influenza vaccination and pneumococcal vaccination.
Regression analyses for the pre-influenza periods, 1 June to 30 September of the four seasons under investigation were performed to assess whether there was a healthy-vaccinee bias present in the cohort. Previous studies have reported on such a bias, which would augment VE estimates [13,14]. Pre-season analyses modelling influenza among those vaccinated later during the season were adjusted for age, sex and comorbidity status.
Data management and analyses were carried out using SAS Enterprise software (SAS Institute Inc., Cary, NC).
This analysis was part of ongoing programme evaluations required at the Department of Communicable Disease Control and Prevention, Stockholm County Council, Stockholm, Sweden. As this evaluation was a requisite part of Stockholm County Council work processes, it falls outside the mandate for the Regional Ethics committee. PINs have been anonymised in VAL and no data making individual identification possible is retained.
In total, 2–2.2 million individuals were included per season in the study (Tables 1A and 1B). A slightly higher proportion of women were vaccinated compared to men. (Tables 1A and 1B). The number of patients with a clinical diagnosis of influenza was highest in 2011/12 and in 2014/15, seasons dominated by influenza A(H3N2), but the need for hospital treatment was about three times higher in 2014/15 than in 2011/12 (Table 2). The number of people hospitalised with a diagnosis of influenza during the influenza seasons followed the curve of laboratory-confirmed cases in the county (Figure).
Baseline characteristics of the cohorts in influenza analysis, Stockholm County, influenza seasons 2011/12 and 2012/13
|Mosaic income/education categories
|Highest income and education
|Middle income and education
|Lowest income and education
|Previous seasonal vaccinationb
Baseline characteristics of cohorts, Stockholm County, influenza seasons 2013/14 and 2014/15
|Highest income and education
|Middle income and education
|Lowest income and education
|Previous seasonal vaccinationb
Hazard ratios with 95% confidence intervals and vaccine effectiveness estimates for seasonal influenza vaccination on influenza outcome including inpatient and outpatient cases, Stockholm County, influenza seasons 2011/12–2014/15
In 2011/12, more than 99% of all those vaccinated received Vaxigrip, while in the remaining seasons more than 99% were vaccinated with Flurarix. Almost 30% of the individuals included in the analysis had a documented comorbidity and of these ca 25% were vaccinated. There were no differences in vaccination rates among those with high or low socioeconomic status (Tables 1A and 1B).
For the 2011/12 season, overall VE for inpatient and outpatient care was 19% (95% CI: 6–31), driven primarily by outpatient effects in those younger than 65 years of age (Table 2). For the 2012/13 season, overall VE was higher, 40% (95% CI: 23–52), with stronger VE seen among inpatients, particularly those 65 years of age or older (VE: 52%; 95% CI: 31–66). For the 2013/14 season, overall VE was 37% (95% CI: 17–52), with elderly inpatient care driving the effects (VE: 49%; 95% CI: 17–69 for those 65 years or older). In 2014/15, the study season with the highest burden of hospital treatment of influenza, the VE was again lower and the vaccine effect was strongest for those 65 years, or older, 18% (95 CI: 7–29) overall and 21% (95% CI: 7–32) for inpatient care.
For the two seasons with moderately high VEs, inpatient VE for patients with comorbidities was similar to that of the whole population (Table 3). Stratified analyses on comorbidity showed 48–55% effectiveness against inpatient care in the seasons 2012/13 and 2013/14 for those with underlying chronic illness, both overall and among those 65 years of age or older. VE in outpatient care was not as strongly affected by comorbidity status.
Stratified analyses presenting hazard ratios with 95% confidence intervals and vaccine effectiveness estimates for seasonal influenza vaccination on influenza outcome among individuals with comorbidity, Stockholm County, influenza seasons 2011/12–2014/15
|All cases among those with comorbidity
|Outpatient among those with comorbidity
|Inpatient among those with comorbidity
Stratified analyses on previous season influenza vaccination among those 65 years of age or older, showed no clear effects, either protective or negative, against the risk of being hospitalised with a diagnosis of influenza in the current season (data not shown).
The pre-influenza season analyses, 1 June to 30 September, were all statistically insignificant, with HRs of 1.71 (95% CI: 0.80–3.66), 0.87 (95% CI: 0.29–2.56), 1.09 (95% CI: 0.45–2.65), and 0.83 (95% CI: 0.34–2.01), respectively, indicating that vaccination was not associated with either a decreased or increased risk of receiving a diagnosis of influenza in any of these four pre-influenza season periods.
VE for inpatient non-influenza pneumonia in persons aged 65 years or older ranged from 11% to 18% during the four seasons. No effectiveness could be demonstrated against non-hospitalised pneumonia (Table 4).
Hazard ratios with 95% confidence intervals and vaccine effectiveness estimates for seasonal influenza vaccination on pneumonia outcome including inpatient and outpatient cases, Stockholm County, influenza seasons 2011/12–2014/15
In this study we used influenza and pneumonia diagnosis codes linked with vaccination status from the entire population of a large metropolitan area to evaluate seasonal influenza vaccine effectiveness on inpatient hospitalisations and primary care visits. Our results thus provide important real-world vaccination programme effects in individuals of varying ages and health statuses. Vaccine effects were moderately good both in adults <65 years of age and in elderly people (≥ 65 years of age), including those with comorbidities, during two of the four seasons. Small but significant VE against non-influenza pneumonias was found in persons 65 years or older in all four seasons. However, since the proportion of pneumonia caused by influenza in most studies is less than 20%, a VE of 11–18% for pneumonia hospitalisation in persons aged 65 years or older, of whom about half were vaccinated, could indicate a VE for influenza-related pneumonia as high as 50–75% .
Seasonal influenza programme vaccination is typically recommended to prevent severe outcomes in highly vulnerable groups. What constitutes optimal outcome measures for seasonal influenza VE is debatable, however. Commonly used outcome measures are influenza-like-illness (ILI), acute respiratory infection (ARI), or hospitalisation for influenza or pneumonia [6,15,16]. Effectiveness against laboratory-confirmed influenza vaccine type is the most specific outcome measure, although often available for relatively limited populations, such as healthy adults, and as such not fully generalisable to populations targeted for influenza programmes [2,4].
The four pre-influenza season period analyses did not show any difference in the risk of receiving a clinical diagnosis of influenza in vaccinated vs non-vaccinated persons, indicating that there was no healthy-vaccinee bias in the current study. This is in contrast to most studies, including an earlier study from Stockholm [13,14,17,18]. The former Stockholm study was performed in 1998–2001 when the yearly seasonal influenza vaccination campaigns were new and included only adults aged 65 years or older. Vaccines were not offered free of charge as they are today, which may also explain the healthy-vaccinee bias found in that study . In addition, during the last few years, Stockholm’s influenza vaccine campaign has been developed specifically to target the chronically ill, irrespective of age.
Randomised control trials (RCTs) measuring influenza VE among elderly people are rare and the only one of high quality showed a 50% effect against serologically confirmed influenza . Pooled observational studies have shown nominal effects among the elderly in nursing homes (ILI VE 23%; hospitalisation for pneumonia VE 45%), but non-significant effects on elderly people living in the community in terms of ILI or influenza . Overall, observational VE estimates range from 25% to 60% in protecting against hospitalisation for influenza or pneumonia among the elderly [6,16,20]. Observational studies are often not able to account for specific effects among the chronically ill, which is a major limitation . When treatment choice, or in this case vaccination status, is driven by an individual’s disease status, it is referred to as confounding by indication and is another type of selection bias. The influenza vaccination programme promotes this population selection bias by targeting those with underlying comorbidities. A major strength in our study is that these effect results have accounted for this major bias by linking with patient records and adjusting for comorbidity status. Other strengths were that we adjusted for potential differences stemming from socioeconomic status and controlled for residual effects in seasonal VE estimates due to previous seasonal vaccinations [21,22], pandemic influenza and pneumococcal vaccinations.
The European network Influenza - Monitoring Vaccine Effectiveness (I-MOVE) has monitored VE in a number of countries since 2008 by observational studies using the ‘test-negative’ or ‘screening’ designs . Our results among persons with comorbidity showing a very low VE in 2011/12, but a moderately good VE around 50% for prevention of hospitalisation for influenza among persons aged 65 years or older in 2012/13 and 2013/14, are in accordance with those presented by I-MOVE. They found a very low VE during the 2011/12 season, from 43% during the early part of the season down to less than 10% in risk groups when the whole season was analysed [23,24]. The reason for this low VE late in the 2011/12 season may have been a waning vaccine effect in older persons, since the peak came late in the season, or an antigenic drift . During the 2012/13 season, when all three influenza types circulated, I-MOVE reported a moderately high VE in Europe (43–63% depending on influenza type), and also in 2013/14 with a VE for the dominating influenza A(H1N1)pdm09 of 48% [23,25]. Reports from the 2014/15 season from North America and Europe are in accordance with our findings that VE was lower than during the two preceding seasons [26-28]. A possible reason for this lower VE is that circulation of newly emerged A(H3N2) clades 3C.3a and 3C.2a viruses, to which antibodies in humans to the A/Texas/50/2012 antigens contained in the seasonal vaccine, reacted less well [28,29].
Effects among adults under 65 years of age, particularly healthy individuals, should theoretically be higher than in elderly people, as they have a better immune response to vaccination. In contrast, VE among healthy adults below 65 years in our study was similar to, or lower than among the elderly. A possible reason for this finding is a potential misclassification of exposure, since entering influenza vaccination of healthy adults below 65 years in the vaccination register is not requisite, as Stockholm neither recommends nor subsidises influenza vaccinations for these individuals. If healthy individuals aged under 65 years obtain vaccinations via mobile clinics at their workplace or via a healthcare provider, they may not be entered in the vaccination register. As such, some may be inappropriately classified as unvaccinated in our study, and hence weaken the effect measures of VE. In contrast, persons belonging to risk groups according to the programme will most likely have been registered in the vaccination register, since they are offered the vaccine free of charge and have easy access to caregivers included in the programme. In addition, caregivers are reimbursed only when they adhere to the reporting requirements.
Although we did not see any evidence of a healthy-vaccinee bias in pre-season analyses, the power of this analysis was low since the few cases with influenza diagnoses off-season resulted in wide confidence intervals. Another limitation is that VAL experienced a technical problem while merging primary care data for 2013, and thus it appears as if there are a reduced number of primary care cases for this year. This technical problem is non-differential and, if anything, would generate diluted VEs. Inpatient care is complete and not affected by these technicalities. We could not control for the severity of comorbidity or the severity of the acute disease in order to identify patients in need of intensive care treatment, nor could we analyse mortality outcomes, since these data are not included in the County’s surveillance. Negative controls were not included in these analyses, although pneumonia was included as a subanalysis, and while significant VE was found, it was very low because of the diluting effect of such a non-specific diagnosis.
Our study found robust VE against influenza hospitalisation, a proxy for severe disease. This VE was most substantial among adults and the elderly having underlying chronic conditions. Therefore, we believe that public health officials should focus resources also on attaining high coverage in people with underlying diseases, irrespective of age, in addition to the WHO/EU goal of a 75% for coverage among all people 65 years of age or older .
The need for additional effectiveness studies for the influenza vaccine with non-specific outcomes such as pneumonia or influenza-like illness has been questioned since the potential for overestimation or underestimation of vaccine effectiveness is too great . Although the influenza diagnoses were not laboratory-confirmed, our study demonstrates that comprehensive population-based patient register data on influenza-specific outcomes, which allow for adjustments of multiple confounders and assessments of potential biases, can and should be used for routine estimates of seasonal influenza IVE and VE. The VEs in our study were in accordance with those from European multicentre studies using the much more laborious test-negative design [25,31,32]. International sentinel surveillance efforts remain vital to gauge circulating types, but are not needed to accurately assess VE across broad populations. In addition, large and expensive RCTs to estimate effects of seasonal influenza vaccines are neither fiscally nor ethically justifiable in the era of reliable electronic medical record data.
Since the beginning of 2016 we have had a regular weekly linkage between Stockholm’s central database for healthcare diagnoses, VAL, and the vaccine register . These real-time data showed that the 55–68% IVE seen in persons aged 65 years or older during January and February, when A(H1N1)pdm09 dominated, declined when influenza B (Victoria) took over and was only 43–44% from the end of March, an observation which lead us to take action and recommend that doctors prescribe early antiviral therapy for ILI in this patient group.
In conclusion, results from this population-based evaluation of multiple vaccine seasons show substantial protective VE against being hospitalised with a diagnosis of influenza among elderly and chronically ill persons in all age groups during two of four seasons and lower, but still significant, VE in another. Programmes that target these vulnerable populations can anticipate ca 50% reductions in influenza-specific inpatient care, in seasons with a good antigenic match. We also demonstrate that the use of population-based patient register data on influenza-specific outcomes enables valuable real-time estimates of seasonal influenza vaccine effectiveness.