On 6 June 2017, the World Health Organization (WHO) published updates to its ‘Essential Medicines List’ (EML). Read more here.

Eurosurveillance is on the updated list of the Directory of Open Access Journals and in the SHERPA/RoMEO database. Read more here.

Follow Eurosurveillance on Twitter: @Eurosurveillanc

In this issue

Home Eurosurveillance Edition  2013: Volume 18/ Issue 7 Article 1
Back to Table of Contents
Download (pdf)

Eurosurveillance, Volume 18, Issue 7, 14 February 2013
Complexities in assessing the effectiveness of inactivated influenza vaccines
  1. Victorian Infectious Diseases Reference Laboratory, Melbourne, Australia
  2. National Centre for Epidemiology and Population Health, Australian National University, Canberra, Australia
  3. European Centre for Disease Prevention and Control (ECDC), Stockholm, Sweden

Citation style for this article: Kelly H, Steffens I. Complexities in assessing the effectiveness of inactivated influenza vaccines. Euro Surveill. 2013;18(7):pii=20403. Available online:
Date of submission: 12 February 2013

For many years it has been generally accepted that well-matched vaccines are the most effective single measure to protect people who are predisposed to a more severe outcome following infection with the influenza virus [1]. This group includes those aged at least 65 years, pregnant women and those who suffer from specific chronic conditions and/or are immunocompromised. 

There is nonetheless widespread professional and public interest in, and some debate about, the level of protection afforded by annual influenza vaccination [2,3]. The level of such protection is assessed in two main ways: as estimates of efficacy from randomised controlled trials and as estimates of effectiveness from observational studies. Both are estimates of the proportion of vaccinated people, compared to the proportion of unvaccinated people, who are protected from a specified influenza outcome. Efficacy estimates (from trials) may be higher than effectiveness estimates (from observational studies) because trials are conducted in a controlled environment. A recent meta-analysis of trials using laboratory-confirmed influenza based on culture or PCR testing as the study endpoint estimated influenza vaccine efficacy for healthy adults at 59% (95% confidence interval (CI): 51 to 67) for vaccines licensed in the United States (US) [4]. The majority of those included in the studies were younger than 40 years. Unfortunately there were too few methodologically acceptable observational studies for a pooled analysis of vaccine effectiveness (VE). 

A number of observational studies published recently in Eurosurveillance [5-10] have reported estimates of influenza VE less than 60%, and thus below that from the meta-analysis. Possible reasons for this may include (i) the study design, (ii) different age or risk groups being studied, (iii) waning immunity, within a season or between seasons, (iv) the match between circulating viruses and vaccine strains, and (v) the effect of repeated annual vaccination.

In the following each of these factors is considered separately to determine whether they may explain VE estimates below 60% reported recently in this journal.

To overcome potential differences in VE estimates due to variations in observational study design, a European collaboration, the Influenza Monitoring of Vaccine Effectiveness (I-MOVE), has produced pooled estimates of influenza VE from multiple countries using shared protocols over a number of influenza seasons [11]. Most contemporary observational studies of trivalent influenza vaccine (TIV), in Europe or elsewhere, have used PCR (with or without culture)-confirmed symptomatic influenza infection as their endpoints. This facilitates comparison with the meta-analysis, where culture was most often used as an endpoint for studies of live attenuated vaccine and PCR for studies of TIV. Two main study designs have been used in the I-MOVE collaboration. The first is a prospective test-negative variant of the case–control study, where cases have a clinical illness consistent with a diagnosis of influenza and are test-positive for influenza, while controls have the same clinical illness but test negative. This is generally referred to as the test-negative design. The second is a cohort, usually assembled from administrative and/or surveillance databases. Some cohort study endpoints are not laboratory-confirmed infection, but when the same endpoint is used, VE estimates from both study designs have generally been similar for patients in the community or those admitted to hospital [11].

In the pooled test-negative design studies in Europe, the seasonal influenza VE estimate in 2010/11 for community patients was 52% (95% CI: 30 to 67), while the point estimate for adults aged 65 years and over was 59% in 2008/09 and 56% in 2010/11, with missing values imputed in the latter estimate [11]. The European estimates for 2010/11 are supported by a point estimate of 56% from Australia in 2011 where the circulating viruses and vaccine strains were the same as in Europe [12]. In Canada in 2010/11 VE among adults aged 20–49 years was 65% (95% CI: 8 to 87) for A(H1N1)pdm09, 66% (95%CI: 10 to 87) for influenza B and 39% (95% CI: 0 to 63) for influenza A(H3N2) but VE for all ages against any influenza strain was estimated to be only 37% (95% CI: 17 to 52) [13].

Given the wide confidence intervals, the evidence suggests that, while point estimates may sometimes be lower in the observational studies in Europe and elsewhere [5-13], they are not always substantially different from the meta-analysis of VE from trials of vaccines licensed in the US [4].  This is despite the fact that the meta-analysis included only healthy adults recruited into trials while the observational studies comprised unselected adults, including those in risk groups.

When restricted to risk groups in the pooled European test-negative design studies, VE estimates reported in 2010/11 were similar to those from the meta-analysis. Among those targeted for vaccination, the point estimate of VE was 56% (95% CI: 34 to 71) [11]. The studies were performed in the community but similar VE estimates were obtained when hospitalised patients were studied in the same season in Valencia region, Spain.  For all patients hospitalised with laboratory-confirmed influenza, the VE was estimated at 54% (95%CI: 11 to 76), with little difference when estimates were restricted by age or risk group [14]. VE estimates from observational studies in 2010/11 were similar to the pooled estimate from trials [4], but the effect of study design on VE is not yet clear [15,16].

In the northern hemisphere  2011/12 influenza season, considerably lower estimates of VE were reported from the Navarre region, Spain [5], the United Kingdom (UK) [6] and from pooled studies in eight European countries [7] compared with those from previous seasons using the same test-negative study design. In a mild season in the UK in 2011/12, for instance, protection against influenza A(H3N2) infection in primary care was described as poor. Adjusted VE against confirmed influenza A(H3N2) infection was only 23% (95% CI -10 to 47), with VE of 43% (95% CI -34 to 75) for October 2011 to January 2012 and 17% (95% CI -24 to 45) for February to April 2012 [6]. Researchers speculated that a late season in which influenza A(H3N2) was the dominant circulating virus might have resulted in lower VE because antibody-derived immunity waned, which  resulted in reduced protection, especially evident among older people who are more likely to suffer an adverse outcome from influenza A(H3N2) virus infection. The influenza A(H3N2) strain in the vaccine was assessed as being not well matched to the circulating strain by the World Health Organization (WHO) which consequently recommended a change in the A(H3N2) strain for the vaccine in the following season [17].

Early estimates of VE, published in Eurosurveillance, against the matched influenza A(H3N2) strain in 2012/13 have shown improved protection of around 50% in community settings. In the Canadian sentinel surveillance scheme, VE estimates were driven by the dominant influenza A(H3N2) strain among adults aged 20–49 years, an age group often over represented in similar schemes. The adjusted VE against influenza A(H3N2) was 45% (95%CI: 13 to 66) [8]. In the UK, adjusted VE against laboratory-confirmed influenza in primary care was 51% (95%CI: 27 to 68); against influenza A, 49% (95%CI: -2 to 75) and against influenza B, 52% (95%CI: 23 to 70) [9]. In the pooled estimate from five European countries, adjusted VE was 62% (95% CI: 21 to 82) and 42% (95% CI: -67 to 80) against A(H3N2) for all ages [18]. The similar point estimates from Canada and the UK were also in the range of the interim point estimate of 55% protection against influenza A reported for the US, although the US estimate included children and was adjusted only for study site [19].

However, VE against influenza for the same season was not as encouraging in a study from Denmark, where influenza A(H3N2) was also circulating. The study used linked databases to construct a test-negative design for patients aged at least 65 years [10]. Eligible patients could be tested in hospitals or in the community, but 95% were tested in hospitals. VE against influenza A was estimated as -11% (95% CI: -41 to 14) and 69% (95% CI: 26 to 87) against influenza B.  The investigators identified antigenic changes that may have contributed to the low VE estimate for influenza A in the hospitalised elderly.

Thus, despite the improved interim VE estimates in community settings from the UK, Canada and the US after the change of the influenza A(H3N2) vaccine antigen for the 2012/13 season, the VE estimate from predominantly hospitalised elderly patients in Denmark indicates the relationship between vaccine effectiveness and vaccine strain match may not be as clear as we had assumed. Indeed after several years of observational studies, it is apparent that a better match between circulating virus and vaccine strains does not necessarily translate into improved VE. For instance, in a recent study from Australian sentinel practices, where the four-year point estimate of VE was 62% for adults aged 20–64 years, VE was estimated at 58% (95% CI: 17 to 79) in 2007 when all strains were assessed as predominantly mismatched, but 59% (95% CI: 4 to 82), in 2010 when strains predominantly matched [20].

The apparent anomaly of VE not correlating with strain match may relate to the way match is currently evaluated or to the fact that antibody immunity measured by haemagglutinin inhibition (HI) is not the only (or even the best) correlate with protection from clinical infection. Indeed it has been suggested that HI assays may not always detect drift of the haemagglutinin antigen [8,13]. VE may also vary with the antigenic distance between circulating wild virus and the vaccine virus contained in current and previous vaccines [21]. 

A number of observations, many of which have been made previously and which may affect estimates of VE, have become evident again in the last few years, but are still not well understood. These include the possibility of intra-seasonal waning of immunity as described in papers published in this journal [5-7], the predictable effect of vaccine strain match or mismatch on VE estimates [22], and the effect of repeated annual vaccination on VE [21,23].

The recent VE estimates summarised above indicate that inactivated influenza vaccine may provide levels of protection below the range of 70–90% [1] and it is becoming more widely accepted that VE is a complex measure, with variation by year, influenza type and sub-type, and by age and health status [2,3,22,24]. When reporting VE estimates in a range from 40–60% in Eurosurveillance, authors have described protection as ‘sub-optimal’, ‘moderate to low’ or ‘moderate’. While it could be useful for messaging if researchers agreed on a common approach to describing estimates of VE, it is important to bear in mind that ’moderate’ protection, in the order of 40 to 60%, is nonetheless protection. If vaccination with a safe vaccine decreases the risk of an adverse outcome by approximately half, vaccination can be confidently recommended. We note that all the interim VE estimates for influenza B in the 2012/13 season are generally at the top end of this range, or above, particularly evident in the study from the Navarre region, Spain, where influenza B was dominant and the vaccine and circulating strains were of the same lineage [25].

Of course it is desirable to offer better than 50% protection and this is achieved in some years for some influenza types or sub-types. We therefore continue to support the use of existing vaccines, especially in populations at risk of a severe outcome following infection. We hope that influenza vaccine researchers continue to refine study methods to estimate the benefit that can be expected from inactivated vaccines in routine practice, and agree on how best to describe this benefit. This is important for a consistent and credible public health message.

  1. Vaccines against influenza WHO position paper – November 2012. Week Epidemiol Record 2012; 87:461-76.
  2. Osterholm MT, Kelley NS, Manske JM, Ballering KS, Leighton TR, Moore KA. The compelling need for game-changing influenza vaccines. University of Minnesota Center for Infectious Disease Research and Policy, Oct 2012. Available from:  
  3. Nicoll A, Sprenger M. Low effectiveness undermines promotion of seasonal influenza vaccine. Lancet Inf Dis 2013; 13:7-9.  
  4. Osterholm MT, Kelley NS, Sommer A, Belongia EA. Efficacy and effectiveness of influenza vaccines: a systematic review and meta-analysis. Lancet Infect Dis. 2012 Jan;12(1):36-44.  
  5. Castilla J, Martínez-Baz I, Martínez-Artola V, Reina G, Pozo F, García Cenoz M, Guevara M, Morán J, Irisarri F, Arriazu M, Albéniz E, Ezpeleta C, Barricarte A, Primary Health Care Sentinel Network, Network for Influenza Surveillance in Hospitals of Navarre. Decline in influenza vaccine effectiveness with time after vaccination, Navarre, Spain, season 2011/12. Euro Surveill. 2013;18(5):pii=20388. Available from: PMid:23399423 
  6. Pebody RG, Andrews N, McMenamin J, Durnall H, Ellis J, Thompson CI, Robertson C, Cottrell S, Smyth B, Zambon M, Moore C, Fleming DM, Watson JM. Vaccine effectiveness of 2011/12 trivalent seasonal influenza vaccine in preventing laboratory-confirmed influenza in primary care in the United Kingdom: evidence of waning intra-seasonal protection . Euro Surveill. 2013;18(5):pii=20389. Available from:
  7. Kissling E, Valenciano M, Larrauri A, Oroszi B, Cohen JM, Nunes B, Pitigoi D, Rizzo C, Rebolledo J, Paradowska-Stankiewicz I, Jiménez-Jorge S, Horváth JK, Daviaud I, Guiomar R, Necula G, Bella A, O'Donnell J, Głuchowska M, Ciancio BC, Nicoll A, Moren A. Low and decreasing vaccine effectiveness against influenza A(H3) in 2011/12 among vaccination target groups in Europe: results from the I-MOVE multicentre case–control study . Euro Surveill. 2013;18(5):pii=20390. Available from: PMid:23399425 
  8. Skowronski DM, Janjua NZ, De Serres G, Dickinson JA, Winter A, Mahmud SM, et al. Interim estimates of influenza vaccine effectiveness in 2012/13 from Canada's sentinel surveillance network, January 2013. Euro Surveill. 2013;18(5):pii=20394 Available from: PMid:23399422   
  9. McMenamin J, Andrews N, Robertson C, Fleming DM, Durnall H, von Wissmann B, et al. Effectiveness of seasonal 2012/13 vaccine in preventing laboratory-confirmed influenza infection in primary care in the United Kingdom: mid-season analysis 2012/13. Euro Surveill. 2013;18(5):pii=20393.  Available from:
  10. Bragstad K, Emborg HD, Kølsen Fischer T, Voldstedlund M, Gubbels S, et al. Low vaccine effectiveness against influenza A(H3N2) virus among elderly people in Denmark in 2012/13 – a rapid epidemiological and virological assessment. Euro Surveill. 2013;18(6):pii=20397.   Available from:
  11. Valenciano M, Ciancio BC, on behalf of the I-MOVE study team. I-MOVE: a European network to measure the effectiveness of influenza vaccines. Euro Surveill. 2012;17(39):pii=20281.  Available from:
  12. Fielding JE, Grant KA, Tran T, Kelly HA. Moderate influenza vaccine effectiveness in Victoria, Australia, 2011. Euro Surveill. 2012;17(11):pii=20115. Available from: PMid:22449867   
  13. Skowronski DM, Janjua NZ, De Serres G, Winter AL, Dickinson JA, Gardy JL, et al. A sentinel platform to evaluate influenza vaccine effectiveness and new variant circulation, Canada 2010-2011 season. Clin Infect Dis 2012; 55:332-42. PMid:22539661   
  14. Puig-Barberà J, Díez-Domingo J, Arnedo-Pena A, Ruiz-García M, Pérez-Vilar S, Micó-Esparza JL et al. Effectiveness of the 2010–2011 seasonal influenza vaccine in preventing confirmed influenza hospitalizations in adults: A case–case comparison, case–control study. Vaccine 2012; 30:5174-20. PMid:22819720   
  15. van der Hoek W, Dijkstra F, de Lange MM, Donker GA, Meijer A, van der Sande MA. Letter to the editor: Influenza vaccine effectiveness: heterogeneity in estimates for the 2012/13 season. Euro Surveill. 2013;18(7):pii=20399. Available from:
  16. Mølbak K, Emborg HD, Gubbels S, Bragstad K, Krause TG, Kølsen Fischer T. Authors’ reply: Influenza vaccine effectiveness: heterogeneity in estimates for the 2012/13 season . Euro Surveill. 2013;18(7):pii=20401. Available from: 
  17. World Health Organization. Recommended composition of influenza virus vaccines for use in the 2012-2013 northern hemisphere influenza season. Geneva: WHO; 2012. Available from:  
  18. Valenciano M, Kissling E, I-MOVE case–control study team. Early estimates of seasonal influenza vaccine effectiveness in Europe: results from the I-MOVE multicentre case–control study, 2012/13 . Euro Surveill. 2013;18(7):pii=20400. Available from:  
  19. Centers for Disease Control and Prevention (CDC). Early estimates of seasonal influenza vaccine effectiveness — United States, January 2013. MMWR Morb Mortal Wkly Rep. 2013;62:32-5. PMid:23325354 
  20. Kelly H, Sullivan S, Grant K, Fielding J. Moderate influenza vaccine effectiveness with variable effectiveness by match between circulating and vaccine strains in Australian adults aged 20-64 years, 2007-11. Influenza Other Respi Viruses. 2012 Oct 19.  
  21. Smith DJ, Forrest S, Ackley DH, Perelson AS. Variable efficacy of repeated annual influenza vaccination. Proc Natl Acad Sci U S A. 1999 Nov 23;96(24):140016. PMid:10570188 PMCid:24180  
  22. Carrat F, Flahault A. Influenza vaccine: the challenge of antigenic drift. Vaccine. 2007 Sep 28;25(39-40):6852-62. PMid:17719149 
  23. Keitel WA, Catet TR, Couch RB. Efficacy of sequential annual vaccination with inactivated influenza virus vaccine. Am J Epidemiol. 1988 Feb;127(2):353-64. PMid:3337087  
  24. Centers for Disease Control and Prevention (CDC). Update: Flu Season and Vaccine Effectiveness. Atlanta: CDC; 2013. Available from: Accessed 5 February 2013.  
  25. Castilla J, Martínez-Baz I, Martínez-Artola V, Fernandez-Alonso M, Reina G, Guevara M, García Cenoz M, Elía F, Alvarez N. Early estimates of influenza vaccine effectiveness in Navarre, Spain: 2012/13 mid-season analysis. Euro Surveill. 2013;18(7):pii=20404. Available from:

Back to Table of Contents
Download (pdf)

The publisher’s policy on data collection and use of cookies.

Disclaimer: The opinions expressed by authors contributing to Eurosurveillance do not necessarily reflect the opinions of the European Centre for Disease Prevention and Control (ECDC) or the editorial team or the institutions with which the authors are affiliated. Neither ECDC nor any person acting on behalf of ECDC is responsible for the use that might be made of the information in this journal. The information provided on the Eurosurveillance site is designed to support, not replace, the relationship that exists between a patient/site visitor and his/her physician. Our website does not host any form of commercial advertisement. Except where otherwise stated, all manuscripts published after 1 January 2016 will be published under the Creative Commons Attribution (CC BY) licence. You are free to share and adapt the material, but you must give appropriate credit, provide a link to the licence, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.

Eurosurveillance [ISSN] - ©2007-2016. All rights reserved

This website is certified by Health On the Net Foundation. Click to verify. This site complies with the HONcode standard for trustworthy health information:
verify here.