Since 2007, the European Centre for Disease Prevention and Control (ECDC) has supported I-MOVE (influenza monitoring vaccine effectiveness), a network to monitor seasonal and pandemic influenza vaccine effectiveness (IVE) in the European Union (EU) and European Economic Area (EEA). To set up I-MOVE, we conducted a literature review and a survey on methods used in the EU/EEA to measure IVE and held expert consultations to guide the development of generic protocols to estimate IVE in the EU/EEA. On the basis of these protocols, from the 2008/09 season, I-MOVE teams have conducted multicentre case–control, cohort and screening method studies, undertaken within existing sentinel influenza surveillance systems. The estimates obtained include effectiveness against medically attended laboratory-confirmed influenza and are adjusted for the main confounding factors described in the literature. I-MOVE studies are methodologically sound and feasible: the availability of various study designs, settings and outcomes provides complementary evidence, facilitating the interpretation of the results. The IVE estimates have been useful in helping to guide influenza vaccine policy at national and European level. I-MOVE is a unique platform for exchanging views on methods to estimate IVE. The scientific knowledge and experience in practical, managerial and logistic issues can be adapted to monitor surveillance of the effectiveness of other vaccines.
Human influenza viruses are subject to frequent antigenic changes. For this reason, the influenza vaccine is the only vaccine reformulated each year to optimise antigenic match between the vaccine and circulating virus strains. The seasonal influenza vaccine is a trivalent vaccine, which currently includes strains of the A subtypes H3N2 and H1N1 and one strain of B virus . The World Health Organization (WHO) issues recommendations in February for which strains should be included in the seasonal vaccine for the northern hemisphere. Once these recommendations have been made, vaccine producers need at least six months to manufacture and distribute the seasonal vaccine. In a pandemic situation, pandemic strain-specific vaccines become available four to six months after the beginning of the vaccine development. During the 2009 pandemic, the influenza A(H1N1)pdm09 strain was identified in April 2009 but the first pandemic vaccines started to become available in Europe only at the end of September 2009. Consequently, antigenic changes in circulating viruses may occur before the start of the vaccination campaigns and can result in a poor match between vaccine (seasonal and pandemic) and circulating strains.
In Europe, seasonal influenza viruses circulate in the cold months, generally between October and April. National influenza surveillance networks have been established since the 1950s based on sentinel practitioner networks. In 1995, the European Influenza Surveillance Scheme was established . Since 2008, the European Centre for Disease Prevention and Control (ECDC) has coordinated the European Influenza Surveillance Network (EISN) . Sentinel practitioners include general practitioners (GPs), paediatricians or other physicians, depending on the European Union (EU) Member State.
Influenza vaccination is the most effective preventive measure available against influenza infection. In May 2003, the World Health Assembly recommended vaccination for all people at high risk, defined as the elderly and persons with underlying diseases. WHO Member States committed to attain a vaccination coverage in the elderly population of at least 50% by 2006 and 75% by 2010 . In December 2009, the Council of the EU issued a recommendation encouraging EU Member States to take action to reach the target of 75% vaccine coverage of the older age groups recommended by WHO and if possible of other risk groups, preferably by 2014–2015 .
Influenza vaccination campaigns are conducted every year in the EU Member States, targeting a high number of individuals . As with any public health intervention, it is important to evaluate their effectiveness. The existence of robust systems to monitor the safety and effectiveness of vaccines is a major determinant of the success of vaccination programmes. Because of antigenic drift, vaccine effectiveness cannot be inferred from estimates from previous seasons. In order to evaluate influenza vaccine effectiveness (IVE) in Europe, ECDC developed a network to monitor seasonal and pandemic IVE in the EU and European Economic Area (EEA). In this article, we describe the phases undertaken to establish the network, its organisation and the main lessons learnt in the first four influenza seasons.
I-MOVE preparatory phase
In 2007, under the ECDC umbrella, the network – composed initially of 18 European public health institutes and EpiConcept, the coordinating hub – was set up. Named I-MOVE (influenza monitoring vaccine effectiveness) in Europe, it aimed to measure IVE on a routine basis. The study methods had to be simple, sustainable in a context of limited budget, adapted to the EU/EEA context and scientifically robust.
The objectives of the first preparatory phase of I-MOVE were to identify the most appropriate observational study designs to measure IVE routinely in the EU/EEA and to identify key methodological issues to be considered in the study protocols. To achieve these objectives, EpiConcept conducted the following: (i) a survey among EU/EEA Member States to identify IVE studies performed in Europe and potential data sources for future studies; (ii) a literature review on methods used to estimate IVE; and (iii) three expert consultations. The methods and results of this phase have been described elsewhere . In brief, the main conclusions were that EU/EEA influenza sentinel surveillance systems seemed to provide a sustainable platform suitable for case–control studies and screening method studies monitoring IVE. In Member States or regions with computerised primary care databases, cohort studies could be conducted to measure IVE against different outcomes.
To control for positive and negative confounding, a minimum set of variables had to be collected in all studies . In addition, as using a specific outcome reduces bias, it was recommended to measure IVE against laboratory-confirmed influenza.. To minimise selection bias, sentinel practitioners were to select systematically the patients to swab.
On the basis of these conclusions, EpiConcept and ECDC developed and published generic case–control, cohort, screening method and cluster investigation protocols for IVE studies [9-11] to be adapted by each potential study site. An expert panel selected seven protocols for the 2008/09 pilot season, and eleven for the2009/10 (pandemic season), 2010/11 and 2011/12 seasons (Table 1).
Table 1. Sites conducting influenza vaccine effectiveness studies as part of I-MOVE, influenza seasons 2008/09 to 2011/12
I-MOVE organisational aspects
In the first four seasons, 26 partner institutions from 17 EU/EEA Member States participated in I-MOVE (Figure 1). Each institution designated an I-MOVE focal point, most of them being influenza experts coordinating the national influenza surveillance system. A total of 13 study site teams have conducted IVE studies: some use several study designs in the same site during the same season (Table 1). The functioning of the I-MOVE network has to date been funded by ECDC and the IVE studies co-funded by ECDC and the study sites. EpiConcept coordinates the I-MOVE activities. The network collaborates with teams conducting IVE studies in Canada, USA and Australia.
Figure 1. European Union and European Economic Area Member States with I-MOVE partner institutions, influenza seasons 2007/08 to 2011/12
Technical workshops are organised during the influenza season among I-MOVE study sites to discuss the preliminary results, to plan the final analysis and to define the publication strategy. Periodically, follow-up videoconferences are organised between study sites. The whole network meets annually at the end of the influenza season to share the IVE estimates and discuss practical and methodological issues related to the studies. Since the 2009/10 season, the last day of the meeting has been open to decision-makers (European Medicines Agency (EMA), European Commission, WHO) and vaccine manufacturers. An I-MOVE website is in place with three different levels of access: unrestricted, restricted to I-MOVE partner institutions, restricted to I-MOVE study sites .
Each study site defines its strategy to communicate its results. EpiConcept, in close collaboration with ECDC, coordinates the publication of the multicentre case–control pooled results. Since 2008, I-MOVE IVE results have been published in peer-reviewed journals [13-32].
I-MOVE implementation phase (2008/9 to 2011/12)
The study designs used within I-MOVE are case–control studies, cohort studies using primary care databases and screening method studies.
Case–control studies, including multicentre case–control study
Nine EU sites have contributed to the multicentre case–control study (Table 1). The methods used for the individual and multicentre case–control studies have been described elsewhere [21,22,25,27-30,32]. In summary, each season the study site coordinators invite sentinel primary care practitioners belonging to the national sentinel surveillance systems to participate in the study. In Portugal, Italy, and Hungary, practitioners other than those participating in the sentinel surveillance system have been also invited to participate.
The study population in each case–control study consists of non-institutionalised patients consulting a participating practitioner for ILI or ARI (France) within eight days after symptom onset. The age groups and covariates included in the study have varied from one season to another (Table 2). Practitioners take nasal or throat swabs from all or a sample of ILI/ARI patients. From the third season, all study sites selected systematically ILI/ARI patients to swab.
Table 2. Characteristics of I-MOVE multicentre case–control study, influenza seasons 2008/09 (5 study sites), 2009/10 (7 study sites), 2010/11 (8 study sites), 2011/12 (8 study sites)
Study sites have progressively adopted the EU ILI case definition : four sites in 2008/09 and seven from 2009/10.
We defined a case of influenza as an ILI patient who tests positive for influenza using reverse transcription polymerase chain reaction (RT-PCR) or culture. Controls are ILI patients testing negative for influenza (test-negative controls). Depending on the study site, testing is performed at national or regional reference laboratories. All laboratories testing sentinel specimens within the EISN scheme are part of a community network of reference laboratories (CNRL), which undergo periodic external quality assessments for virus detection and characterisation methods .
The sentinel practitioners interview the ILI/ARI patients face-to-face, collect information on a set of predefined variables common to all study sites (Table 1) and send the completed questionnaires to each of the I-MOVE study site coordinators.
National study teams send to the EpiConcept coordination team anonymised databases of recruited ILI cases. We evaluate heterogeneity between studies qualitatively and quantitatively [35,36]. We estimate the pooled IVE using a one-stage method, with the study site included as fixed effect in the model. To estimate adjusted IVE, we use a logistic regression model including all potential confounding factors. In 2009/10 and 2010/11 seasons, we estimated missing data for vaccination status and covariates using the multiple multivariate imputation by chained equations procedure in STATA .
The number of participating primary care practitioners/practices was 343 in 2008/09, 1,114 in 2009/10, 1,035 in 2010/11, 942 in 2010/11, and 1,056 in 2011/12. The sample size increased in the first three seasons: in 2008/09, the pilot season, the study was restricted to individuals aged 65 years or more and 327 ILI cases were included in the pooled analysis. In 2009/10, 2010/11 and 2011/12, the number of ILI patients recruited were 2,902, 4,410 and 4,747, respectively.
All cases included in the 2009/10 study were laboratory confirmed as influenza A(H1N1)pdm09. Therefore, the effectiveness of the monovalent pandemic vaccine (72%; 95% CI: 48 to 85) was pandemic strain specific. Estimates of the trivalent 2010/11 seasonal vaccine effectiveness were lower than the pandemic IVE: 52% (95% CI: 30 to 67) overall, 51% (95% CI: 17 to 71) against influenza A(H1N1)pdm09 and 56% (95% CI: 34 to 71) when restricting the analysis to the target group for vaccination (Figure 2). In the age group 15–54 years, the point estimate for the pandemic vaccine effectiveness (73%) was higher than the point estimate for the effectiveness of the trivalent seasonal vaccines in 2009/10 (65%) and 2010/11 (47%). In the 2011/12 season, preliminary results (April 2012) suggested an overall low adjusted effectiveness (27 %) against influenza A(H3N2) among persons targeted for vaccination . In Spain, early (25 December 2011 to 19 February 2012) IVE in the target population was 54% .
Figure 2. Adjusted overall and stratified influenza vaccine effectiveness estimates against medically attended laboratory-confirmed influenza, I-MOVE multicentre case–control study, 2008/09 (5 study sites), 2009/10 (7 study sites), 2010/11 (8 study sites)
Four study sites have conducted cohort studies (Table 1). These studies are based on electronic primary care databases that, using a unique identifier, can be linked to other databases such as a vaccination registry, hospital admissions or laboratory databases. The linkage of the databases provides information on exposure, various outcomes and potential confounding factors or effect modifiers (Table 3). Consequently, using a person-time analysis, cohort studies estimate adjusted IVE against various clinical outcomes (ILI, ARI, lower respiratory tract infection, hospital admission and death). In the 2010/11 season, the size of the cohorts varied from 93,380 individuals in Scotland to 1,005,132 in England.
Table 3. Variables collected, data sources and timing of data extraction in I-MOVE cohort studies in Navarre (Spain), Royal College of General Practitioners (England and Wales) and Scotland, influenza season 2011/12
Within the cohort studies, nested test-negative case–control studies are conducted to estimate IVE against medically attended laboratory-confirmed influenza [14,16,17,20,26]. During the 2009/10 season, the cohort in Scotland gave an estimated adjusted IVE of 49% (95% CI: 19 to 67) for ILI, of 40% (95% CI: 18 to 56) for overall mortality and of 60% (95% CI: −38 to 89) for virologically confirmed symptomatic individuals . During the same season, the cohort study in England and Wales estimated an adjusted IVE of 21% (95% CI: 5.3 to 34.0) in preventing ILI and of 64% (95% CI: −6 to 88.6) in preventing PCR-confirmed influenza A(H1N1)pdm09 . The Navarra cohort (Spain) results were similar: vaccination with the 2009 pandemic vaccine was associated with an adjusted 32% (95% CI: 8 to 50) reduction in the overall incidence of medically attended ILI and an adjusted 89% (95% CI: 36 to 100) reduction in the incidence of PCR-confirmed influenza .
Screening method studies
In addition to estimating IVE using a cohort or a case–control study, some study sites use the screening method (Table 1). In the screening method, IVE is estimated by comparing the vaccination coverage between ILI patients positive for influenza and a reference group. The reference groups used in I-MOVE studies vary by study site and include the vaccination coverage in the practitioners´ catchment area (e.g. Spain, Scotland, England), the vaccination coverage in a random sample of the population (Portugal ) or in the general population (Italy ). The Farrington method  is used to adjust IVE for age group (Spain, Italy, Portugal, Scotland), risk group (Portugal, Scotland), GP practice (Scotland) and socio-economic status (Scotland). During the pandemic and in line with results using other study designs, the I-MOVE screening method studies estimated a high pandemic IVE against medically attended laboratory-confirmed influenza A(H1N1)pdm09: the IVE was 78% (95% CI: 61 to 88) in the Spanish study  and 92% (95% CI: 46 to 99) in the Italian .
The I-MOVE network is well established and has provided seasonal and pandemic IVE for four consecutive influenza seasons (2011/12 results have been submitted for publication). The I-MOVE results are timely: since the 2009/10 season, preliminary results have been communicated early in the season to the decision-makers and published in peer-reviewed journals [15,19,23,24,29].
I-MOVE results have assisted in guiding public vaccination policy at national and European level. In particular, during the 2009 pandemic they contributed to the risk–benefit analysis process coordinated in the EU by the EMA  and globally by WHO  by providing regular updates of IVE estimates. In 2012, the low observed IVE against influenza A(H3N2) prompted a discussion on the respective role of antigenic drift and early waning immunity [19,24]. In addition, the European regulatory authority (EMA) incorporated I-MOVE estimates as a component of post-licensure surveillance for the 2009 pandemic vaccines . As the I-MOVE IVE studies are conducted by an independent scientific research network, this adds weight to the integrity of their results and to how they are perceived professionally and by the public.
Using a sound methodology, I-MOVE studies have shown that seasonal IVE is moderate against medically attended laboratory-confirmed influenza. This is triggering a number of initiatives including a possible revision of EU regulatory criteria for annual vaccine re-licensure that include results of IVE studies . Given the timely provision of in-season estimates of IVE from I-MOVE and similar networks elsewhere in the world, discussions are ongoing with WHO to consider how such estimates can contribute to the annual vaccine strain selection process .
During the 2009/10 pandemic season, all the I-MOVE network participants (practitioners, laboratories, national and regional public health institutes) were overwhelmed with response activities. Having the I-MOVE coordinating hub based in a structure not directly involved in the response was an advantage: the studies were not disrupted and the coordination hub could focus on facilitating exchanges between study sites, on rapidly analysing the multicentre case–control study and on coordinating the communication of IVE results to ECDC.
I-MOVE is a unique platform for exchanging views on and experience of methods to estimate IVE. During the I-MOVE technical workshops and annual meetings, the discussions around the epidemiological and logistic challenges allowed improvement of standard methods, good scientific practices to be followed and have strengthened EU expertise on IVE. The network has contributed to strengthening influenza surveillance in the EU. Currently, most of the sentinel practitioners conducting I-MOVE studies use the same EU ILI case definition and select patients for swabbing in a systematic way. As most I-MOVE practitioners are part of the national sentinel surveillance systems, any improvement and standardisation of methods should have a positive impact on national and European influenza surveillance systems.
The I-MOVE network takes into account the operational and methodological aspects required to building a sustainable system: studies are methodologically sound but at the same time feasible within existing surveillance systems and with limited resources. The estimates include effectiveness against medically attended laboratory-confirmed influenza as outcome and are adjusted for the main confounding factors described in the literature (e.g. presence of underlying chronic diseases, health-seeking behaviour, age group, etc.). In addition, the availability of various study designs, settings and outcomes gives a combination of different sets of evidence, facilitating better interpretation of the results.
The challenges for monitoring IVE in Europe include the variety of influenza vaccines available and in use and differences in vaccination coverage and groups targeted for vaccination and in health-seeking behaviours between Member States . Having estimates by vaccine type and among target groups represents a major challenge and requires large sample sizes. The constant increase in sample size observed throughout the four influenza seasons and the precise information collected on vaccine type and brand could allow estimation of IVE by vaccine type in the near future. Time, especially during a pandemic, needs to be accounted for in the analysis (e.g. adjustment for week/month of symptom onset, person-time analysis in the cohort studies). The main limitation in reaching a large sample size is the low influenza vaccination coverage in some groups . Pooling data from the various I-MOVE case–control studies is one of the I-MOVE strengths that allows IVE to be estimated early in the season and for different subgroups. In 2010/11, we had for the first time IVE estimates for the target groups for vaccination . However, results still lack sufficient precision and efforts should be made to increase the sample size in each study site.
In I-MOVE, the cohort study in the Navarre region of Spain is the only study able to provide early and repeated estimates of IVE against hospital admission of persons with laboratory-confirmed influenza . Therefore, one of the limitations of I-MOVE is that it does not yet provide early estimates of IVE against severe outcomes at European level. The main challenge is to attain a sufficient sample size enabling precise adjusted estimates and stratification by effect modifiers. A European hospital network with multiple sites using the same protocol would allow a multicentre study to be conducted with a sample size sufficient to rapidly estimate IVE against severe influenza outcomes. As a first step, the I-MOVE network has developed a generic protocol for IVE hospital case–control studies. From the 2011/12 season, hospitals in the Valencia and Navarre regions (Spain), France and Italy are conducting studies based on this protocol and are providing pooled estimates of IVE against hospital admissions with laboratory-confirmed influenza.
Some countries considered that the screening method, in addition to case–control and cohort studies, was convenient because of its inexpensiveness, reliance on already available data and ability to provide an early indication of IVE.
Influenza sentinel surveillance networks have shown to be an excellent framework in which IVE observational studies can be conducted using different study designs (cohort, case–control and screening method) not only in Europe but also in other countries such as Canada or Australia [47,48]. The scientific knowledge and experience in practical, managerial and logistic issues gained by the I-MOVE network can be used in other regions of the world to estimate IVE. The I-MOVE model can also be adapted to establish similar monitoring systems in Europe for vaccines that may change their effectiveness over time due to, for example, serotype replacement or to changes in vaccination schedules (e.g. rotavirus, pneumococcal conjugate vaccine). I-MOVE experience, protocols and some of the study sites infrastructures are already contributing to a recent ECDC project for assessing vaccination impact and effectiveness studies for pneumococcal conjugate vaccine .
There is a strong case for use of the I-MOVE methodology for monitoring IVE as a routine part of post-licensure monitoring. The approach described in this article is ethical and practical. While it cannot have the accuracy of randomised controlled trials, the results achieve their objective in detecting changes in effectiveness over time and with changes in vaccine.
The major challenge is how to make these studies sustainable. While they are not as expensive as randomised controlled trials, they are not as inexpensive as the sentinel surveillance undertaken by the same practitioners. Nevertheless, they require accurate virus testing and careful coordination to retain quality. What has yet to be resolved is how to attract co-sponsorship from industry and public sectors while retaining independence. Some manufacturers appreciate the advantages to them of having such validation as do regulators. However, a way of combining monies in a share scheme has yet to be achieved. The recent breakthrough of agreement for sustaining WHO’s essential influenza surveillance work may show the way forward .
All participating general practitioners, paediatricians and patients in Denmark, England (Royal College of General Practitioners), France (Réseau des GROG), Hungary, Ireland, Italy, Navarre region of Spain, Portugal (‘Médicos-Sentinela’ and general practitioners participating in the EuroEVA project), Romania, Scotland, Spain (cycEVA), the Netherlands (IPCI project). Acknowledgements by country can be found at https://sites.google.com/site/epiflu/about-i-move/study-sites/acknowledgements
I-MOVE study team (participation during influenza seasons 2008/9 to 2011/12)
European Centre for Disease Prevention and Control (ECDC), Sweden: B.C. Ciancio, P. Kramarz, A. Nicoll; EpiConcept, France: E. Kissling, A. Moren, C. Savulescu, T. Seyler, M. Valenciano; Statens Serum Institut, Denmark: A. Mazick, K. Widgren; Open Rome, France: T. Bui, J.M. Cohen, I. Daviaud; Réseau des GROG, France: A. Mosnier; Office of the Chief Medical Officer, Hungary: B. Oroszi; National Center for Epidemiology, Hungary: S. Caini, A. Csohán, J.K. Horváth, M. Rózsa; Health Protection Surveillance Centre, Ireland: A.S. Barret, L. Domegan, J. O´Donnell, D. O´Flanagan, J. Rebolledo; Istituto Superiore di Sanità, Italy: A. Bella, I. Donatelli, S. Declich, S. Puzelli, C. Rizzo, M.C. Rota; Erasmus University Medical Center, the Netherlands: J. Dieleman, M. Sturkenboom, L. Wijnans; Dutch Medicines Agency, the Netherlands: B. Voordouw; National Institute of Public Health - National Institute of Hygiene, Poland: M. Głuchowska, I. Paradowska-Stankiewicz, P. Stefanoff; Instituto Nacional de Saúde Dr Ricardo Jorge, Portugal: I. Batista, M. Barreto, P. Conde, J.M. Falcão, P. Gonçalves, R. Guiomar, A. Machado, B. Nunes, P. Pechirra, H. Rebelo de Andrade, L. Santos; General Directorate of Health, Portugal: I Falcão; Cantacuzino Institute, National Centre for Research and Development in Microbiology and Immunology, Romania: V. Alexandrescu, A. Ivanciuc, E. Lupulescu, D. Pitigoi; Royal College of General Practitioners, Research and Surveillance Centre, England: H. Durnall, D. Fleming; Health Protection Agency, United Kingdom: N. Andrews, P. Hardelid, G. Kafatos, R. Pebody, J. Watson, M. Zambon; Strathclyde University, Scotland: K. Kavanagh, C. Robertson; Health Protection Scotland, Scotland: J. McMenamin, A. Reynolds; Institute of Health Carlos III, Spain: A. Larrauri, S. Jiménez-Jorge, S. De Mateo, F. Pozo; Instituto de Salud Pública de Navarra, CIBERESP, Spain: A. Barricarte, J. Castilla, M. García Cenoz, I. Martínez-Baz, M. Guevara.
- World Health Organization (WHO). Recommended composition of influenza virus vaccines for use in the 2012-2013 northern hemisphere influenza season. Geneva: WHO; Feb 2012. Available from: http://www.who.int/influenza/vaccines/virus/recommendations/201202_recommendation.pdf
- Snacken R, Manuguerra JC, Taylor P. European Influenza Surveillance Scheme on the Internet. Methods Inf Med. 1998;37(3):266-70.
- European Centre for Disease Prevention and Control (ECDC). European Influenza Surveillance Network (EISN). Stockholm: ECDC. [Accessed 26 Sep 2012]. Available from: http://ecdc.europa.eu/en/activities/surveillance/EISN/Pages/index.aspx
- World Health Organization (WHO). Prevention and control of influenza pandemics and annual epidemics. Fifty-sixth World Health Assembly, Resolution WHA56.19.28. Geneva: WHO: 2003. Available from: http://apps.who.int/gb/archive/pdf_files/WHA56/ea56r19.pdf
- European Commission (EC). Proposal for a Council recommendation on seasonal influenza vaccination. Brussels: EC; 2009. COM(2009) 353/final/2. Available from: http://ec.europa.eu/health/ph_threats/com/Influenza/docs/seasonflu_rec2009_en.pdf
- Nicoll A, Ciancio B, Tsolova S, Blank P, Yilmaz C. The scientific basis for offering seasonal influenza immunisation to risk groups in Europe. Euro Surveill. 2008;13(43):pii=19018. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=19018
- Valenciano M, Ciancio B, Moren A, the influenza vaccine effectiveness working group. First steps in the design of a system to monitor vaccine effectiveness during seasonal and pandemic influenza in EU/EEA Member States. Euro Surveill. 2008;13(43):19015. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=19015
- Valenciano M, Kissling E, Ciancio BC, Moren A. Study designs for timely estimation of influenza vaccine effectiveness using European sentinel practitioner networks. Vaccine. 2010;28(46):7381-8.
- European Centre for Disease Prevention and Control (ECDC). Protocol for cluster investigations to measure influenza vaccine effectiveness in the EU/EEA. Stockholm: ECDC; 2009. Avalaible from: http://www.ecdc.europa.eu/en/publications/Publications/0912_TED_Protocol_for_Cluster_Investigations_to_Measure_Influenza_Vaccine_Effectiveness.pdf
- European Centre for Disease Prevention and Control (ECDC). Protocol for case-control studies to measure influenza vaccine effectiveness in the European Union and European Economic Area Member States. Stockholm: ECDC; 2009. Available from: http://www.ecdc.europa.eu/en/publications/Publications/0907_TED_Influenza_AH1N1_Measuring_Influenza_Vaccine_Effectiveness_Protocol_Case_Control_Studies.pdf
- European Centre for Disease Prevention and Control (ECDC). Protocol for cohort database studies to measure influenza vaccine effectiveness in the European Union and European Economic Area Member States. Stockholm: ECDC; 2009. Available from: http://www.ecdc.europa.eu/en/publications/Publications/0907_TER_Influenza_AH1N1_Measuring_Influenza_Vaccine_Effectiveness_Protocol_Cohort_Database_Studies.pdf
- I-MOVE in Europe. Paris: EpiConcept. [Accessed 26 Sep 2012]. Available from: https://sites.google.com/site/epiflu/Home
- Castilla J, Etxeberria J, Ardanaz E, Floristan Y, Lopez Escudero R, Guevara M. Estimating the impact of the 2009 influenza A(H1N1) pandemic on mortality in the elderly in Navarre, Spain. Euro Surveill. 2010;15(5):pii=19481. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=19481
- Castilla J, Morán J, Martínez-Artola V, Fernández-Alonso M, Guevara M, Cenoz MG, et al. Effectiveness of the monovalent influenza A(H1N1)2009 vaccine in Navarre, Spain, 2009-2010: cohort and case-control study. Vaccine. 2011;29(35):5919-24.
- Castilla J, Moran J, Martinez-Artola V, Reina G, Martinez-Baz I, Garcia Cenoz M, et al. Effectiveness of trivalent seasonal and monovalent influenza A(H1N1)2009 vaccines in population with major chronic conditions of Navarre, Spain: 2010/11 mid-season analysis. Euro Surveill. 2011;16(7):pii=19799. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=19799
- Castilla J, Martínez-Artola V, Salcedo E, Martínez-Baz I, Cenoz MG, Guevara M, et al. Vaccine effectiveness in preventing influenza hospitalizations in Navarre, Spain, 2010-2011: cohort and case-control study. Vaccine. 2012;30(2):195-200.
- Hardelid P, Fleming DM, McMenamin J, Andrews N, Robertson C, SebastianPillai P, et al. Effectiveness of pandemic and seasonal influenza vaccine in preventing pandemic influenza A(H1N1)2009 infection in England and Scotland 2009-2010. Euro Surveill. 2011;16(2):pii=19763. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=19763
- Hardelid P, Fleming DM, Andrews N, Barley M, Durnall H, Mangtani P, et al. Effectiveness of trivalent and pandemic influenza vaccines in England and Wales 2008-2010: results from a cohort study in general practice. Vaccine. 2012;30(7):1371-8.
- Jimenez-Jorge S, de Mateo S, Pozo F, Casas I, Garcia Cenoz M, Castilla J, et al. Early estimates of the effectiveness of the 2011/12 influenza vaccine in the population targeted for vaccination in Spain, 25 December 2011 to 19 February 2012. Euro Surveill. 2012;17(12):pii=20129. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=20129
- Kavanagh K, Robertson C, McMenamin J. Assessment of the variability in influenza A(H1N1) vaccine effectiveness estimates dependent on outcome and methodological approach. PLoS One. 2011;6(12):e28743.
- Kissling E, Valenciano M, Falcao J, Larrauri A, Widgren K, Pitigoi D, et al. "I-MOVE" towards monitoring seasonal and pandemic influenza vaccine effectiveness: lessons learnt from a pilot multi-centric case-control study in Europe, 2008-9. Euro Surveill. 2009;14(44):pii=19388. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=19388
- Kissling E, Valenciano M, Cohen JM, Oroszi B, Barret AS, Rizzo C, et al. I-MOVE multi-centre case control study 2010-11: overall and stratified estimates of influenza vaccine effectiveness in Europe. PLoS One. 2011;6(11):e27622.
- Kissling E, Valenciano M; I-MOVE case-control studies team. Early estimates of seasonal influenza vaccine effectiveness in Europe, 2010/11: I-MOVE, a multicentre case-control study. Euro Surveill. 2011;16(11):pii=19818. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=19818
- Kissling E, Valenciano M; I-MOVE Case-Control Studies Team. Early estimates of seasonal influenza vaccine effectiveness in Europe among target groups for vaccination: results from the I-MOVE multicentre case-control study, 2011/12. Euro Surveill. 2012;17(15):pii=20146. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=20146
- Nunes B, Machado A, Pechirra P, Falcão I, Gonçalves P, Conde P, et al. Efetividade da vacina antigripal na época 2010-2011 em Portugal: resultados do projeto EuroEVA. [Influenza vaccine effectiveness during the 2010-2011 season in Portugal: results of the EuroEVA project]. Revista Portuguesa de Medicina Geral e Familiar . 2012;28:271-84. Portuguese.
- Pebody R, Hardelid P, Fleming D, McMenamin J, Andrews N, Robertson C, et al. Effectiveness of seasonal 2010/11 and pandemic influenza A(H1N1)2009 vaccines in preventing influenza infection in the United Kingdom: mid-season analysis 2010/11. Euro Surveill. 2011;16(6):pii=19791. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=19791
- Pitigoi D, Lupulescu E, Alexandrescu V, Băetel A, Sbârcea C, Ionită E, et al. [Efficiency of seasonal influenza vaccine in persons older than 65 in Romania. Pilot case-control study I-MOVE, 2008-2009]. Bacteriol Virusol Parazitol Epidemiol. 2009;54(2):109-16. Romanian.
- Savulescu C, Valenciano M, de Mateo S, Larrauri A; cycEVA Study Team. Estimating the influenza vaccine effectiveness in elderly on a yearly basis using the Spanish influenza surveillance network--pilot case-control studies using different control groups, 2008-2009 season, Spain. Vaccine. 2010;28(16):2903-7.
- Savulescu C, Jiménez-Jorge S, de Mateo S, Ledesma J, Pozo F, Casas I, et al. Effectiveness of the 2010/11 seasonal trivalent influenza vaccine in Spain: preliminary results of a case-control study. Euro Surveill. 2011;16(11):pii=19820. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=19820
- Savulescu C, Jiménez-Jorge S, de Mateo S, Pozo F, Casas I, Breña PP, et al. Using surveillance data to estimate pandemic vaccine effectiveness against laboratory confirmed influenza A(H1N1)2009 infection: two case-control studies, Spain, season 2009-2010. BMC Public Health. 2011;11:899.
- Seyler T, Bella A, Puzelli S, Donatelli I, Rizzo C; screening method working group. Estimating pandemic vaccine effectiveness in two Italian regions using the screening method, 2009-2010. Vaccine. 2012;30(2):109-11.
- Valenciano M, Kissling E, Cohen JM, Oroszi B, Barret AS, Rizzo C, et al. Estimates of pandemic influenza vaccine effectiveness in Europe, 2009-2010: results of Influenza Monitoring Vaccine Effectiveness in Europe (I-MOVE) multicentre case-control study. PLoS Med. 2011;8(1):e1000388.
- European Commission. Commission Decision of 28 April 2008 amending Decision 2002/253/EC laying down case definitions for reporting communicable diseases to the Community network under Decision No 2119/98/EC of the European Parliament and of the Council. Official Journal of the European Union. Luxembourg: Publications Office of the European Union. 18.6.2008:L 159. European Commission, (2008). Available from: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2008:159:0046:0090:EN:PDF
- European Centre for Disease Prevention and Control (ECDC). External quality assessment scheme for influenza virus detection and culture for the Community Network of Reference Laboratories for Human Influenza in Europe 2010/2011. Stockholm: ECDC; 2012. Available from: http://ecdc.europa.eu/en/publications/Publications/1205-TER-CNRL-EQA-Influenza.pdf
- Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539-58.
- Huedo-Medina TB, Sánchez-Meca J, Marín-Martínez F, Botella J. Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychol Methods. 2006;11(2):193-206.
- Sterne JA, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ. 2009;338:b2393.
- Farrington CP. Estimation of vaccine effectiveness using the screening method. Int J Epidemiol. 1993;22(4):742-6.
- Larrauri A, Savulescu C, Jiménez-Jorge S, de Mateo S, Spanish Influenza Sentinel Surveillance System. Pandemic influenza vaccine effectiveness estimates using the screening method in the frame of Spanish Influenza Surveillance Sentinel System, season 2009-2010. Presentation at the 2010 European Scientific Conference on Applied Infectious Disease Epidemiology (ESCAIDE), 11-13 November 2010, Lisbon, Portugal. Available from: http://ecdc.europa.eu/en/ESCAIDE/Materials/Presentations%202010/ESCAIDE2010_Parallel_Session14_03_Larrauri.pdf
- European Medicines Agency (EMA), European Centre for Disease Prevention and Control (ECDC), Heads of Medicines Agencies (HMA). European strategy for influenza A/H1N1 vaccine benefit-risk monitoring. London: EMA; 2009. Available from: http://www.ema.europa.eu/docs/en_GB/document_library/Report/2010/01/WC500044933.pdf
- Girard MP, Katz JM, Pervikov Y, Hombach J, Tam JS. Report of the 7th meeting on Evaluation of Pandemic Influenza Vaccines in Clinical Trials, World Health Organization, Geneva, 17-18 February 2011. Vaccine. 2011;29(44):7579-86.
- European Medicines Agency (EMA). CHMP recommendations for the pharmacovigilance plan as part of the risk management plan to be submitted with the marketing authorisation application for a pandemic influenza vaccine. London: EMA; 2009. Available from: http://www.ema.europa.eu/docs/en_GB/document_library/Report/2010/01/WC500051739.pdf
- European Medicines Agency (EMA). Concept paper on the revision of guidelines for influenza vaccines. London: EMA; 2011. Available from: http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2011/10/WC500115612.pdf
- Skowronski DM, Janjua NZ, De Serres G, Winter AL, Dickinson JA, Gardy JL, et al. A sentinel platform to evaluate influenza vaccine effectiveness and new variant circulation, Canada 2010-2011 season. Clin Infect Dis. 2012;55(3):332-42.
- Mereckiene J, Cotter S, D'Ancona F, Giambi C, Nicoll A, Levy-Bruhl D, et al. Differences in national influenza vaccination policies across the European Union, Norway and Iceland 2008-2009. Euro Surveill. 2010;15(44):pii=19700. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=19700
- Mereckiene J, Cotter S, Weber JT, Nicoll A, Lévy-Bruhl D, Ferro A, et al. Low coverage of seasonal influenza vaccination in the elderly in many European countries. Euro Surveill. 2008;13(41):pii=19001. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=19001
- Kelly H, Carville K, Grant K, Jacoby P, Tran T, Barr I. Estimation of influenza vaccine effectiveness from routine surveillance data. PLoS One. 2009;4(3):e5079.
- Skowronski DM, Masaro C, Kwindt TL, Mak A, Petric M, Li Y, et al. Estimating vaccine effectiveness against laboratory-confirmed influenza using a sentinel physician network: results from the 2005-2006 season of dual A and B vaccine mismatch in Canada. Vaccine. 2007;25(15):2842-51.
- European Centre for Disease Prevention and Control (ECDC). Assessing the impact of vaccination with conjugate vaccines on the epidemiology of the invasive pneumococcal disease in Europe. Stockholm: ECDC. OJ/04/04/2012-PROC/2012/031. [Accessed 11 Sep 2012]. Available from: http://www.ecdc.europa.eu/en/aboutus/calls/Lists/Calls%20for%20tender/ECDC_DispForm.aspx?List=a70e951a%2D9260%2D4909%2Dbc27%2Dcefd2af6e9a4&ID=636&RootFolder=%2Fen%2Faboutus%2Fcalls%2FLists%2FCalls%20for%20tender
- Fidler DP, Gostin LO. The WHO pandemic influenza preparedness framework: a milestone in global governance for health. JAMA. 2011;306(2):200-1.