23 October 2008
First steps in the design of a system to monitor vaccine effectiveness during seasonal and pandemic influenza in EU/EEA Member States
Estimating influenza vaccine effectiveness (IVE) early in the season helps measuring the consequences of a mismatch between the vaccine and the circulating strain and guiding alternative or complementary interventions. The European Centre for Disease Prevention and Control is funding a project to develop pilot studies to monitor IVE in the Member States (MS) of the European Union and European Economic Area (EU/EEA) during seasonal and pandemic influenza. To identify key methodological and practical issues in developing protocols for pilot studies, we conducted a survey among EU/EEA MS, a literature review on IVE methods, and consultations of experts. The survey and literature review highlighted the variety of the data sources used to estimate IVE and the difficulty to interpret data on IVE, which varies with age, risk group, outcome specificity and virus-vaccine mismatch. We also found that negative and positive confounding can bias IVE. The experts consultations lead to the following recommendations: to measure IVE in the same population in various seasons; to control for positive/negative confounding (including pre- and post-influenza season IVE estimates); and to include laboratory confirmation as outcome in various study designs. In the 2008-9 influenza season, two cohort studies using general practitioners’ databases and six case control studies will be piloted in EU/EEA MS and will adhere to the above recommendations. The pilot studies will be the basis for the development of robust methods to monitor IVE in EU/EEA MS.
Because influenza viruses are constantly changing and vaccines are reformulated every year, the influenza vaccine effectiveness (IVE) estimates from previous years cannot be used to estimate IVE in the subsequent years. Having annual IVE estimates at European level available as soon as possible after the start of a seasonal influenza epidemic or pandemic and monitoring it along the course of the epidemic/pandemic is essential in order to:
- decide on recommendations for the use of the vaccine by specific age and risk groups,
- target complementary or alternative public health measures (e.g. antivirals) to population segments in which the vaccine is less effective,
- estimate more precisely the impact of current vaccination strategies on the burden of disease with a view to supporting vaccination campaigns,
- provide some quantification to the current virological system of comparing antigenic matches of vaccine and circulating viruses,
- trigger further investigations on seasonal and pandemic vaccines (improving their composition, use of adjuvants, need for booster doses),
- better manage and respond to eports of vaccine failures (especially during a pandemic),
- counterbalance the reports of adverse events following immunisation by providing a basis for adequate risk management and cost-effectiveness analysis.
In addition, in order to be able to measure IVE for the pandemic vaccine it is necessary to develop already now a robust method that provides early estimates of IVE.
As the vaccine is recommended for risk groups, clinical trials to estimate IVE in Europe would not be ethical. Only observational studies can be considered when trying to obtain IVE estimates early in the season . It is therefore necessary to define which observational study designs can be adopted in the Member States (MS) of the European Union and European Economic Area (EU/EEA) that would provide IVE estimates during an ongoing influenza season and allow monitoring it through consecutive seasons. These methods need to take into account the specific situation of each MS in terms of resources and available data.
The European Centre for Disease Prevention and Control (ECDC) is funding the development and piloting of study protocols for monitoring IVE in EU/EEA MS in the context of seasonal and pandemic influenza. A consortium of 18 European public health institutes coordinated by EpiConcept is carrying out this project. The first phase (January-July 2008) consisted of the development of protocols for the pilot studies. To identify key methodological and practical issues to be considered in the study protocols, we conducted a survey among EU/EEA MS, a literature review on methods used to estimate IVE, and three consultations of experts. These three approaches are described in the following sections of this article.
We carried out a survey among EU/EEA Member States to identify, in each MS, observational IVE studies and available data sources that could be used for real-time IVE studies.
We contacted 29 experts from 29 EU/EEA MS involved in influenza surveillance. The experts were the representatives of the institutions included in the consortium and, for MS not participating in the consortium, the epidemiologist focal point of the European Influenza Surveillance Scheme (EISS) or the gatekeeper of the Vaccine European New Integrated Collaboration Effort (VENICE). The experts were given the options either to provide information through a self-completed questionnaire or during a telephone or face to face interview. In addition, we reviewed available reports from EISS and VENICE, web pages from European institutions involved in influenza surveillance and articles on IVE studies conducted in EU/EEA MS.
We collected data on IVE studies conducted in the MS, available data sources for case identification (identification of influenza cases, death registries, hospital registries, general practitioners' (GP) databases, other) and for documenting influenza vaccination status, as well as potential interest in conducting a pilot study during the season 2008-9.
Among the 29 MS we contacted, 24 (83%) accepted to participate in the survey. In four MS, we interviewed the experts face to face, in 12 by telephone and in eight MS, the experts self-completed the questionnaire we sent them.
Of the participating 24 MS, ten had conducted IVE studies in the past. We identified 43 published articles reporting results of case control studies (12 articles), of cohort studies (28 articles) and of studies using a screening method (three articles). Additional details on the studies including data sources and study outcomes are reported in the Table. A complete survey report is also planned to be published on the ECDC website.
Table. Influenza vaccine effectiveness studies conducted in EU/EEA Member States, by study design and country
In most of these studies, the study population and data sources had been identified through health delivery services. In the Czech Republic, Italy and Portugal, other data sources had been used for IVE studies as reported in the Table.
Malta, Norway and Sweden have population registries including an individual unique identifier which allows linking existing databases (e.g. death registers, in-patient registers, vaccination registers if available). The linkage of the various databases is not immediate and an ethical or a personal protection approval is needed.
In Finland, France, Ireland, the Netherlands, Norway, and the United Kingdom (UK), various GP networks have computerised databases. Computerised GP databases are also available in some regions in Spain and in some counties in Sweden.
Computerised GP databases allow evaluating various outcomes: influenza-like illness (ILI)/acute respiratory infections (ARI), death, hospitalisation, vaccine status and some confounding factors (e.g. co-morbidities). However, certain issues need to be considered when using computerised databases for IVE studies, such as the representativeness, completeness, timeliness and quality of the data. For some of the databases, ad hoc studies may be necessary to further evaluate data quality.
Computerised databases have been used in the Netherlands, Spain, Sweden and the UK to conduct IVE cohort studies. They can provide rapid estimates for some outcomes (e.g. ARI/ILI) and more solid estimates at the end of the season (e.g. estimates adjusted for confounding factors, estimates for severe clinical outcomes).
In all 24 responding MS, the main source to identify clinical cases of influenza on a real-time basis was the virological or epidemiological sentinel influenza surveillance system. Case definitions vary from MS to MS but most sentinel networks report cases of ILI symptoms or ARI .
Laboratory confirmation of influenza cases is usually done in a subset of patients consulting the sentinel practitioners. In most MS, the decision of which patients to collect laboratory specimens from is based on clinical criteria. Thus, patients with laboratory tests are not a representative sample of all patients consulting a GP because of influenza symptoms . In Denmark and France, the patients to be sampled are selected in a systematic way. Following EISS recommendations, laboratory request forms include the patients´ vaccination status.
Sentinel surveillance systems have been used to conduct case control studies of IVE in Denmark, France, Germany, the Netherlands, and the UK (Table).
Hospitalisation discharge databases
In most MS, cases with severe clinical influenza outcome (hospitalisations, deaths) are not identified in real time. Hospitalisation discharge databases are available with delays varying from three months to two years. In France, hospitals report on a daily basis to the Institut de Veille Sanitaire individual data from in-patients and out-patients consulting emergency rooms.
Various MS have developed or are developing real-time mortality monitoring . Mortality has not yet been used in Europe to estimate real-time IVE.
Influenza vaccination status
Sources to document influenza vaccination status include medical records, computerised medical records, immunisation registries, surveys, and pharmaceutical data . Vaccination registries allowing the extraction of real-time vaccination status are currently available at regional level in Finland, in some counties in Sweden and in some regions in Spain. In 2008-9, Spain plans to estimate real-time vaccination coverage using vaccination coverage reported by the sentinel practitioners.
In addition to the survey described above, a literature review was conducted to identify the key elements to be considered in the design of the pilot studies. In particular, we focused on factors affecting IVE estimates and on methods described to control them. In the following paragraphs, we summarise factors that will have an influence on the choices made when developing the pilot study protocols: outcomes and confounding factors.
Literature review methods
To identify relevant papers, we searched the Cochrane database and consulted Cochrane reviews on influenza vaccine effectiveness [48,49]. Additionally, we reviewed the Health Technology Assessment report "Systematic review and economic decision modelling for the prevention and treatment of influenza A and B" . We also included a recent Sanofi Pasteur-MSD review . Finally, we also reviewed references from each of the selected articles.
We selected studies providing IVE estimates. We also included studies addressing methodological aspects of IVE estimates and certain studies addressing the methodology of VE measurements for infectious diseases.
Literature review results
Overall, we reviewed 284 scientific articles and of them selected 93 descriptive observational studies (34 cohort studies, 26 outbreak investigations, 31 case control studies and two studies using the screening method). In addition we consulted 23 articles focusing on methodological issues.
The main clinical outcomes reported in the literature were hospitalisations for all or specific causes (e.g. pneumonia and influenza), deaths from all or specific causes (e.g. pneumonia and influenza), ILI, ARI and laboratory-confirmed cases of influenza.
IVE studies using non-specific clinical outcomes will include as cases individuals with clinical symptoms unrelated to influenza, leading to an underestimation of the IVE [52,53]. The influenza case definition combined with laboratory confirmation results has the highest specificity for influenza, and laboratory confirmation is therefore essential to estimate the true IVE . Due to the costs involved, some authors have suggested to perform laboratory tests only in a small proportion of the study participants (validation set) .
Comparing the crude IVE estimates and the IVE estimates adjusted for confounding factors reported in the literature provides an overview of the magnitude of confounding in IVE studies. We found a difference in percentage between crude and adjusted IVE in case control studies (Figure 1) and cohort studies (Figure 2) that ranged from -220% to 21%.
Figure 1. Difference between crude and adjusted influenza vaccine effectiveness estimated in case control studies, by study outcome
Figure 2. Difference between crude and adjusted influenza vaccine effectiveness in cohort studies, by study outcome
The list of potential confounding factors reported in the literature is very long (Box).
Box. List of potential confounding factors in influenza vaccine effectiveness studies reported in the literature
The main confounding factors discussed in the literature are factors resulting either in an underestimation of the IVE (negative confounding) or in an overestimation of the IVE (positive confounding factors). Negative confounding is the result of 'confounding by indication': Individuals that are at high risk of influenza are more likely to be vaccinated than individuals that are at low risk, and consequently, IVE is underestimated. Positive confounding is the consequence of healthier individuals being more conscious about their health, more motivated to accept vaccination and therefore more likely to be vaccinated than unhealthier individuals. An alternative explanation for positive confounding is the fact that critically ill patients are not offered (or refuse) to be vaccinated. Therefore, vaccinated individuals have a better baseline health status than the unvaccinated group leading to an overestimation of the IVE ('healthy vaccinee' effect).
Different alternatives have been proposed to adjust for the 'healthy vaccinee' and 'confounding by indication' effects. Some authors restricted the study population to groups that were more homogeneous with regard to the potential confounding factor. Others stratified the results according to risk groups. A majority of the studies reviewed included the potential confounders as covariates in a regression model. Some authors controlled for confounding using propensity scores, the conditional probability of being vaccinated given observed covariates [11,18,39,56-58]. They are used to group individuals at levels of the propensity score or as a covariate in the regression model.
Comparison with non-influenza season data
Some authors considered those adjustment methods insufficient to adjust for the 'healthy vaccinee' effect and suggest that residual confounding may persist. They proposed to compare the IVE estimates in the influenza season with estimates from periods with no influenza. The rationale behind this is that the vaccine should not have an effect in non-influenza seasons.
Several studies using this approach compared IVE during and after the influenza season. Most of the results showed a lower IVE after the season suggesting that there was no positive confounding [21,24,59-61]. Other authors, however, found a greater reduction in the risk of death and pneumonia hospitalisation in the period before the influenza season compared to the time during the influenza season, suggesting positive confounding . They argue that studies that did not find an association between vaccine and disease outcome (low IVE) after the influenza season had assumed the difference in underlying characteristics to be constant over time. They suggest that the differences between vaccinated and unvaccinated individuals may diminish over time and the data should therefore be compared not only with the post-influenza season, but also with the pre-influenza season.
During the first phase of the project, we organised several workshops for experts participating in the consortium and additional invited influenza experts.
The first workshop was held in April 2008. The aim was to present and discuss the results of the literature review and survey as described above and to consider the feasibility of the various observational methods to estimate real-time IVE at EU/EEA level. The participants included 25 experts from institutions participating in the consortium, four external influenza experts (London School of Hygiene and Tropical Medicine, Instituto de Salud Pública de Castellón, Sanofi Pasteur MSD, United States-Centers for Disease Control and Prevention Influenza division), four staff members from the ECDC Scientific Advice Unit and two EpiConcept epidemiologists.
The participants worked in three groups to discuss cohort studies, case control studies, and screening method studies. For each study design, the groups made recommendations to be considered in the development of generic protocols for the pilot studies. The experts' recommendations were to determine IVE in various population subgroups, to control for positive and negative confounding and to use laboratory-confirmed influenza as outcome. The group recommended measuring IVE in a homogenous population for a period of several years, using the same design each year. The participating MS and ECDC expressed their interest in supporting this project in the long term.
Following the first workshop, we developed two generic protocols (see below) for case control and cohort studies to be adapted to the situation of each MS.
The second set of consultations was held in June 2008 with the MS that were interested in conducting pilot studies in the season 2008-9. The objective was to further discuss methodological issues related to the two generic protocols for measuring IVE. Specific sessions were held for each study design.
The group agreed that, during the first season of the pilot phase, 2008-9, the following study designs were to be considered:
- Case-control studies based on influenza sentinel surveillance systems with laboratory-confirmed influenza-positive ILI as cases and influenza-negative ILI as controls.
- Prospective cohort studies using computerised databases and providing IVE estimates for different periods (pre-/during/post-influenza season). At least a subset of the cases would be laboratory-confirmed.
The survey showed that data sources to conduct IVE studies vary from MS to MS and in some MS from region to region. Computerised databases are available in few countries and, where available, are a good basis for cohort studies as they include large populations. Sentinel GP networks are present in all 24 EU/EEA MS that participated in the survey; they include laboratory confirmation of influenza cases and data on vaccination status for a subset of the population.
The literature review underlined the difficulty to interpret IVE estimates. IVE estimates vary with age, risk group and the specificity of the disease outcome. In addition, IVE estimates can be heavily biased by positive or negative confounding.
The expert consultations led to specific recommendations to be applied in the next phase of the project. Eight studies will be piloted in the 2008-9 season: two cohort studies, one case control nested in one of the cohorts, and five case control studies.
The two cohort studies will be conducted in England and Scotland, and in the Comunidad Autónoma de Navarra, Spain, using GP databases. These two studies will provide IVE for the pre- and post-influenza season and will allow to further analyse confounding factors included in the GP database. IVE will be estimated against ILI (both studies), all respiratory infections (England and Scotland), pneumonia and influenza hospitalisations (Navarra), all respiratory hospitalisations (Navarra), and all deaths (Navarra). In Navarra, a subset of patients will be laboratory-confirmed.
A case control study with laboratory-confirmed outcome will be nested in the England and Scotland cohort.
In addition, five case control studies among the elderly population will be conducted during the influenza season in Denmark, Hungary, Portugal, Romania and Spain. The vaccine status of ILI cases that are laboratory-confirmed for influenza will be compared to various sets of controls including influenza-negative ILI cases, controls from GP patients and controls from GP catchment areas.
The five studies will use the recommended European Commission case definition for ILI and a common definition for potential confounding factors such as functional status, underlying diseases, severity, smoking, previous influenza vaccination and pneumococcal vaccination. Therefore, the possibility of pooling the results from those five studies to have a multicentre IVE estimate will be explored.
Results of the 2008-9 pilot studies will be presented in an expert meeting in June 2009. Based on those results, amendments to the protocols will be proposed and implemented in the next round of pilot studies in the same eight countries in the season 2009-10. Subject to available resources, at least two additional pilot studies will start in 2009.
The results of the pilot studies will guide the establishment of a system capable to provide and share rapid and reliable information on IVE on an annual basis. The intention is for this information to be integrated as an essential part of the routine influenza surveillance outputs/data. In order to achieve the successful inclusion of IVE data in regular influenza surveillance, sustained commitment from all partners as well as secured funding is fundamental.
Working group influenza vaccine effectiveness:
AAVITSLAND, Preben, Norwegian Institute of Public Health, Oslo, Norway; ALEXANDRESCU, Viorel, Cantacuzino Institute, Bucharest, Romania; BONMARIN, Isabelle, Institut de Veille Sanitaire, Paris, France; BUCHHOLZ, Udo, Robert Koch Institute, Berlin, Germany; CASTILLA, Jesús, Instituto de Salud Pública de Navarra, Spain; CIANCIO, Bruno, European Centre for Disease Prevention and Control, Stockholm, Sweden; COHEN, Jean-Marie, Open Rome, Paris, France; CZUMBEL, Ida, National Centre for Epidemiology, Budapest, Hungary; DANCONA, Fortunato, Istituto Superiore di Sanità, Rome, Italy; DE MATEO, Salvador, Instituto de Salud Carlos III, Madrid, Spain; DIJKSTRA, Frederika, Rijksinstituut voor Volksgezondheid en Milieu, Bilthoven, The Netherlands; ELLIOT, Alex, Birmingham Research Unit of the Royal College of General Practitioners, Birmingham, United-Kingdom; ENSTONE, Joanne, Nottingham University, Nottingham, United-Kingdom; FALCAO, José Marinho, Instituto Nacional de Saúde Dr Ricardo Jorge, Lisbon, Portugal; FLEMING, Douglas, Birmingham Reserach Unit of the Royal College of General Practitioners, Birmingham, United-Kingdom; GERGONNE, Bernadette, EpiLife, Stockholm, Sweden; GIESECKE, Johan, European Centre for Disease Prevention and Control, Stockholm, Sweden; LARRAURI, Amparo,Instituto de Salud Carlos III, Madrid, Spain; LEINO, Tuija, National Public Health Institute, Helsinki, Finland; LEVY BRUHL, Daniel, Institut de Veille Sanitaire, Paris, France; LINDE, Annika, Smittskyddsinstitutet, Stockholm, Sweden; MOLBAK, Kare, Statens Serum Institut, Copenhagen, Denmark; MALTEZOU, Helen, Hellenic Centre for Infectious Disease Control, Athens Greece; MANGTANI, Punam, London School of Hygiene and Tropical Medicine, London, United-Kingdom; MAZICK, Anne, Statens Serum Institut, Copenhagen Denmark; MC MENAMIN, Jim, Health Protection Scotland, Glasgow, United-Kingdom; MOSNIER, Anne, Open Rome, Paris, France; MOREN, Alain, EpiConcept, Paris, France; NICOLL, Angus, European Centre for Disease Prevention and Control, Stockholm, Sweden; O'DONNELL, Joan, Health Protection Surveillance Centre, Dublin, Ireland; O’FLANAGAN, Darina, Health Protection Surveillance Centre, Dublin, Ireland; OROSZI, Beatrix, National Centre for Epidemiology, Budapest, Hungary; ORTQVIST, Ake, Karolinska Institute, Stockholm, Sweden; PAGET, John, European Influenza Surveillance Scheme, Amsterdam, The Netherlands; PEBODY, Richard, Centre for Infections, Health Protection Agency, London, United Kingdom; PUIG-BARBERA, Joan, Centre de Salut Pública de Castelló, Valencia, Spain; QUOLIN, Sophie, Scientific Institute of Public Health, Louis Pasteur, Brussels, Belgium; RIZZO, Caterina, Istituto Superiore di Sanità, Rome, Italy; SAVULESCU, Camelia, Instituto de Salud Carlos III, Madrid, Spain; SHAY, David K, United States Centers for Disease Control and Prevention, Atlanta, USA; SIMONDON, François, Sanofi Pasteur MSD, Lyon,France; UPHOFF, Helmut, Centre for Health Protection, State of Hesse, Germany; VALENCIANO, Marta, EpiConcept, Paris, France; VAN DER SANDE, Marianne, Rijksinstituut voor Volksgezondheid en Milieu, Bilthoven, The Netherlands; SBARCEA, Claudius, Cantacuzino Institute, Bucharest, Romania;VAN TAM, Jonathan , Nottingham University, Nottingham, United Kingdom; WEBER, Todd, European Centre for Disease Prevention and Control, Stockholm, Sweden;WUILLAUME, Françoise, Scientific Institute of Public Health, Louis Pasteur, Brussels, Belgium.
- Mangtani P, Shay DK, Valenciano M, Ciancio BC, Nicoll A, Moren A. An assessment of the literature on the evidence for the protective effect of seasonal influenza vaccine among adults. Euro Surveill., submitted.
- Chlibek R, Beran J, Splino M. [Effectiveness of influenza vaccination in healthy adults--a fourfold decrease in influenza morbidity during one influenza season]. [In Czech]. Epidemiol Mikrobiol Imunol. 2002;51(2):47-51.
- Beran J, Moravik J. Effectiveness of vaccination against influenza in SkodaAuto Company employees during the influenza season 2000-2001. Central European journal of public health. 2003;11(4):209-12.
- Comeri L, Tinella M, Croce E, Arzese M. [Protective efficacy of anti Influenza vaccination in the elderly]. [In Italian]. L'igiene Moderna. 1995;103(6):651-6.
- Consonni S, Sandrini C, Segato E, Perucchini E, Bergamaschini L, Vergani C. Tolerability and efficacy of anti-influenza vaccination alone and associated with antipneumococcal vaccination in an elderly ambulatory population and adherence to the vaccination campaign. J Prev Med Hyg. 2004;45(3):45-50.
- Pregliasco F, Giardini G, Mandrini MG, et al. [Protective efficacy of Inflexal V in the elderly patient.] [In Italian]. Vaccine Glance. 2002;1:2-5.
- Rizzuto E, Prete AM, Virtuani L, Pompa MG. Effectiveness of influenza vaccination: a survey within the Italian Ministry of Health personnel. Vaccine. 2006;24(44-46):6612-4.
- Landi F, Onder G, Cesari M, Gravina EM, Lattanzio F, Russo A, et al. Effects of influenza vaccination on mortality among frail, community-living elderly patients: an observational study. Aging Clin Exp Res. 2003 06;15(3):254-8.
- Landi F, Onder G, Cesari M, Russo A, Barillaro C, Bernabei R, et al. In a prospective observational study, influenza vaccination prevented hospitalization among older home care patients. J Clin Epidemiol. 2006;59(10):1072-7.
- Smits AJ, Hak E, Stalman WA, van Essen GA, Hoes AW, Verheij TJ. Clinical effectiveness of conventional influenza vaccination in asthmatic children. Epidemiol Infect. 2002;128(2):205-11.
- Tacken MA, Braspenning JC, Berende A, Hak E, de Bakker DH, Groenewegen PP, et al. Vaccination of high-risk patients against influenza: impact on primary care contact rates during epidemics. Analysis of routinely collected data. Vaccine. 2004;22(23-24):2985-92.
- Voordouw BC, van der Linden PD, Simonian S, van der LJ, Sturkenboom MC, Stricker BH. Influenza vaccination in community-dwelling elderly: impact on mortality and influenza-associated morbidity. Arch Intern Med. 2003;163(9):1089-94.
- Voordouw BC, Sturkenboom MC, Dieleman JP, Stijnen T, van der LJ, Stricker BH. Annual influenza vaccination in community-dwelling elderly individuals and the risk of lower respiratory tract infections or pneumonia. ArchIntern Med. 2006;166(18):1980-5.
- Castilla J, Arregui L, Baleztena J, Barricarte A, Brugos A, Carpintero M, et al. [Incidence of influenza and influenza vaccine effectiveness in the 2004-2005 season]. [In Spanish]. An Sist Sanit Navar. 2006;29(1):97-106.
- Gené Badia J, Calero Muñóz S, Castañera Ribé C, Gran Rovireta A. [Effectiveness of an anti-influenza vaccination program in 4 primary care centers]. [In Spanish]. Gac Sanit. 1991;5(26):203-8.
- López Hernández B, Vázquez J, Fernández E, Martínez B, Romero J, Arribas L. [Effectiveness of anti-flu vaccine in the elderly]. [In Spanish]. Aten Primaria. 1994;14(1):532-6.
- Salleras L, Dominguez A, Pumarola T, Prat A, Marcos MA, Garrido P, et al. Effectiveness of virosomal subunit influenza vaccine in preventing influenza-related illnesses and its social and economic consequences in children aged 3-14 years: a prospective cohort study. Vaccine. 2006;24(44-46):6638-42.
- Vila-Corcoles A, Rodriguez T, de Diego C, Ochoa O, Valdivieso A, Salsench E, et al. Effect of influenza vaccine status on winter mortality in Spanish community-dwelling elderly people during 2002-2005 influenza periods. Vaccine. 2007;25(37-38):6699-707.
- Christenson B, Lundbergh P, Hedlund J, Ortqvist A. Effects of a large-scale intervention with influenza and 23-valent pneumococcal vaccines in adults aged 65 years or older: a prospective study. Lancet. 2001;357(9261):1008-11.
- Christenson B, Hedlund J, Lundbergh P, Ortqvist A. Additive preventive effect of influenza and pneumococcal vaccines in elderly persons. Eur Respir J. 2004;23(3):363-8.
- Ortqvist A, Granath F, Askling J, Hedlund J. Influenza vaccination and mortality: prospective cohort study of the elderly in a large geographical area. Eur Respir J. 2007;30(3):414-22.
- Fleming DM, Watson JM, Nicholas S, Smith GE, Swan AV. Study of the effectiveness of influenza vaccination in the elderly in the epidemic of 1989-90 using a general practice database. Epidemiol Infect. 1995;115(3):581-9.
- Armstrong BG, Mangtani P, Fletcher A, Kovats S, McMichael A, Pattenden S, et al. Effect of influenza vaccination on excess deaths occurring during periods of high circulation of influenza: cohort study in elderly people. BMJ. 2004;329(7467):660.
- Mangtani P, Cumberland P, Hodgson CR, Roberts JA, Cutts FT, Hall AJ. A cohort study of the effectiveness of influenza vaccine in older people, performed using the United Kingdom general practice research database. J Infect Dis. 2004;190(1):1-10.
- Aymard M, Bentejac MC, Larbaigt G, Michaut D, Triau R. Efficacy of the antiinfluenza A vaccination during epidemics due to A/VIC/3/75 and A/Texas/1/77 viruses. Dev Biol Stand. 1979;43:231-9.
- Caminiti C, Ricco D, Tanzi ML, Borrini B, Corsello A, Biasio LR. Field evaluation of influenza vaccine efficacy in a population of institutionalized elderly. L'igiene Moderna. 1994;101(2):163-75.
- Arroyo JC, Postic B, Brown A, Harrison K, Birgenheier R, Dowda H. Influenza A/Philippines/2/82 outbreak in a nursing home: limitations of influenza vaccination in the aged. AmJ InfectControl. 1984;12(6):329-34.
- Mukerjee A. Spread of influenza: a study of risk factors in homes for the elderly in Wales. J Epidemiol Community Health. 1994;48(6):602-3.
- Nicholls S, Carroll K, Crofts J, Ben-Eliezer E, Paul J, Zambon M, et al. Outbreak of influenza A (H3N2) in a highly-vaccinated religious community: a retrospective cohort study. Commun Dis Public Health. 2004;7(4):272-7.
- Mazick A, Christiansen AH, Samuelsson S, Mølbak K. Using sentinel surveillance to monitor effectiveness of influenza vaccine is feasible: A pilot study in Denmark. Euro Surveill. 2006;11(10):pii=654. Available online: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=654.
- Carrat F, Flahault A, Boussard E, Farran N, Dangoumau L, Valleron AJ. Surveillance of influenza-like illness in France. The example of the 1995/1996 epidemic. J Epidemiol Community Health. 1998;52 Suppl 1:32S-8S.
- Lavallee P, Perchaud V, Gautier-Bertrand M, Grabli D, Amarenco P. Association between influenza vaccination and reduced risk of brain infarction. Stroke. 2002;33(2):513-8.
- Grau AJ, Fischer B, Barth C, Ling P, Lichy C, Buggle F. Influenza vaccination is associated with a reduced risk of stroke. Stroke. 2005;36(7):1501-6.
- Uphoff H, Hauri AM, Schweiger B, Heckler R, Haas W, Gruber A, et al. [Estimation of influenza vaccine effectiveness using routine surveillance data]. [In German]. Bundesgesundheitsblatt, Gesundheitsforschung, Gesundheitsschutz. 2006;49(3):287-95.
- Crocetti E, Arniani S, Bordoni F, Maciocco G, Zappa M, Buiatti E. Effectiveness of influenza vaccination in the elderly in a community in Italy. Eur J Epidemiol. 2001;17(2):163-8.
- Hak E, Nordin J, Wei F, Mullooly J, Poblete S, Strikas R, et al. Influence of high-risk medical conditions on the effectiveness of influenza vaccination among elderly members of 3 large managed-care organizations. Clin Infect Dis. 2002;35(4):370-7.
- Puig-Barbera J, Marquez-Calderon S, Masoliver-Fores A, Lloria-Paes F, Ortega-Dicha A, Gil-Martin M, et al. Reduction in hospital admissions for pneumonia in non-institutionalised elderly people as a result of influenza vaccination: a case-control study in Spain. J Epidemiol Community Health. 1997;51(5):526-30.
- Puig-Barbera J, Diez-Domingo J, Perez Hoyos S, Belenguer Varea A, Gonzalez VD. Effectiveness of the MF59-adjuvanted influenza vaccine in preventing emergency admissions for pneumonia in the elderly over 64 years of age. Vaccine. 2004;23(3):283-9.
- Puig-Barbera J, Diez-Domingo J, Varea AB, Chavarri GS, Rodrigo JA, Hoyos SP, et al. Effectiveness of MF59-adjuvanted subunit influenza vaccine in preventing hospitalisations for cardiovascular disease, cerebrovascular disease and pneumonia in the elderly. Vaccine. 2007;25(42):7313-21.
- Ahmed AE, Nicholson KG, Nguyen-Van-Tam JS. Reduction in mortality associated with influenza vaccine during 1989-90 epidemic. Lancet. 1995;346(8975):591-5.
- Jordan RE, Hawker JI, Ayres JG, Tunnicliffe W, Adab P, Olowokure B, et al. A case-control study of elderly patients with acute respiratory illness: effect of influenza vaccination on admission to hospital in winter 2003-2004. Vaccine. 2007;25(46):7909-13.
- Carrat F, Tachet A, Rouzioux C, Housset B, Valleron AJ. Field investigation of influenza vaccine effectiveness on morbidity. Vaccine. 1998;16(9-10):893-8.
- Legrand J, Vergu E, Flahault A. Real-time monitoring of the influenza vaccine field effectiveness. Vaccine. 2006;24(44-46):6605-11.
- European Influenza Surveillance Scheme. Case definitions. 12 December 2005. Available from: URL: http://www.eiss.org/html/case_definitions.html
- Larrauri A, De Mateo S. Characterisation of swabbing for virological analysis in the Spanish Influenza Sentinel Surveillance System during four influenza seasons in the period 2002-2006. Euro Surveill. 2007;12(5):pii=706. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=706
- Mazick A, Participants of a workshop on mortality monitoring in Europe. Monitoring excess mortality for public health action: potential for a future European network. Euro Surveill. 2007;12(1):pii=3107. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=3107
- Mereckiene J, Cotter S, Weber JT, Nicoll A, Levy-Bruhl D, Ferro A, et al. National Seasonal Influenza Vaccination Survey in Europe. Euro Surveill. 2008;13(43):pii=19017. http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=19017
- Jefferson T, Rivetti D, Rivetti A, Rudin M, Di Pietrantonj C, Demicheli V. Efficacy and effectiveness of influenza vaccines in elderly people: a systematic review. Lancet. 2005;366(9492): 1165-74.
- Jefferson T, Smith S, Demicheli V, Harnden A, Rivetti A, Di Pietrantonj C. Assessment of the efficacy and effectiveness of influenza vaccines in healthy children: systematic review. Lancet. 2005;365(9461):773-80.
- Turner D, Wailoo A, Nicholson K, Cooper N, Sutton A, Abrams K. Systematic review and economic decision modelling for the prevention and treatment of influenza A and B. Health Technol Assess. 2003;7(35): iii-iv, xi-xiii, 1-170.
- Gerbier S, Barret B, Sanofi Pasteur Epidemiology working group. Assessment of Influenza Vaccine Efficacy/Effectiveness in the elderly. Methods and influencing factors: a literature review. 2007 Sep. [Unpublished results].
- Nichol KL, Mendelman P. Influence of clinical case definitions with differing levels of sensitivity and specificity on estimates of the relative and absolute health benefits of influenza vaccination among healthy working adults and implications for economic analyses. Virus Res. 2004;103(1-2):3-8.
- Nichol KL. Heterogeneity of influenza case definitions and implications for interpreting and comparing study results. Vaccine. 2006;24(44-46):6726-8.
- Orenstein EW, De Serres G, Haber MJ, Shay DK, Bridges CB, Gargiullo P, et al. Methodologic issues regarding the use of three observational study designs to assess influenza vaccine effectiveness. Int J Epidemiol. 2007;36(3):623-1.
- Halloran ME, Longini IM Jr. Using validation sets for outcomes and exposure to infection in vaccine field studies. Am J Epidemiol. 2001;154(5):391-8.
- Nichol KL, Nordin JD, Nelson DB, Mullooly JP, Hak E. Effectiveness of influenza vaccine in the community-dwelling elderly. N Engl J Med. 2007;357(14):1373-81.
- Ozasa K, Kawahito Y, Doi T, Watanabe Y, Washio M, Mori M, et al. Retrospective assessment of influenza vaccine effectiveness among the non-institutionalized elderly population in Japan. Vaccine. 2006;24(14):2537-43.
- Hak E, Verheij TJ, Grobbee DE, Nichol KL, Hoes AW. Confounding by indication in non-experimental evaluation of vaccine effectiveness: the example of prevention of influenza complications. J Epidemiol Community Health. 2002;56(12):951-5.
- Davis JW, Lee E, Taira DA, Chung RS. Influenza vaccination, hospitalizations, and costs among members of a Medicare managed care plan. Med Care. 2001;39(12):1273-80.
- Nichol KL, Nordin JD, Nelson DB, Mullooly JP, Hak E. Effectiveness of influenza vaccine in the community-dwelling elderly. N Engl J Med. 2007;357(14):1373-81.
- Vila-Corcoles A, Ochoa-Gondar O, Ansa-Echeverria X, Gomez-Sorribes A, Espelt-Aluja P, Pascual-Moron I. [Influenza vaccination and mortality in the elderly]. [In Spanish]. Med Clin (Barc). 2005;125(18):689-91.
- Jackson ML, Weiss NS, Nelson JC, Jackson LA. To rule out confounding, observational studies of influenza vaccine need to include analyses during the "preinfluenza period". Arch Intern Med. 2007;167(14):1553-4.