27 June 2013
Hospital-based cluster randomised controlled trial to assess effects of a multi-faceted programme on influenza vaccine coverage among hospital healthcare workers and nosocomial influenza in the Netherlands, 2009 to 2011
Nosocomial influenza is a large burden in hospitals. Despite recommendations from the World Health Organization to vaccinate healthcare workers against influenza, vaccine uptake remains low in most European countries. We performed a pragmatic cluster randomised controlled trial in order to assess the effects of implementing a multi-faceted influenza immunisation programme on vaccine coverage in hospital healthcare workers (HCWs) and on in-patient morbidity. We included hospital HCWs of three intervention and three control University Medical Centers (UMCs), and 3,367 patients. An implementation programme was offered to the intervention UMCs to assess the effects on both vaccine uptake among hospital staff and patient morbidity. In 2009/10, the coverage of seasonal, the first and second dose of pandemic influenza vaccine as well as seasonal vaccine in 2010/11 was higher in intervention UMCs than control UMCs (all p<0.05). At the internal medicine departments of the intervention group with higher vaccine coverage compared to the control group, nosocomial influenza and/or pneumonia was recorded in 3.9% and 9.7% of patients of intervention and control UMCs, respectively (p=0.015). Though potential bias could not be completely ruled out, an increase in vaccine coverage was associated with decreased patient in-hospital morbidity from influenza and/or pneumonia.
The value of vaccinating healthcare workers (HCWs) against influenza has been subject of debate over decades. In the United States (US), despite respective immunisation recommendations since 1981, vaccine coverage among HCWs was only 63.5% in 2010-/11 . In the United Kingdom (UK), the Netherlands and other European countries, coverage is even lower [2,3]. Several arguments support influenza vaccination of HCWs. First, each year, influenza causes substantial morbidity and mortality among vulnerable patients in hospitals and nursing homes [4-6]. Since contacts between patients, visitors and HCWs are frequent in such settings, and HCWs who are infected with mild symptoms often continue to work , epidemics can easily develop and can be large . Second, prophylaxis with neuraminidase inhibitors can be effective, but viral resistance may develop rendering these drugs less effective during influenza infections and such a strategy has not been routinely implemented in healthcare settings. Third, immunisation with the inactivated influenza vaccine has been shown in a large meta-analysis of randomised controlled trials among healthy adults representative of the HCWs population to be 59% effective in preventing laboratory-confirmed influenza infection . Fourth, a mathematical model for a 30-bed hospital predicted that seven HCWs need to be vaccinated to prevent one influenza infection in a patient . Finally, despite some methodological constraints, a meta-analysis of four large randomised controlled trials in long-term care institutions showed significant reductions in patients presenting influenza-like illness and patient mortality in settings with high vaccine coverage among HCWs versus control settings with low coverage .
In the Netherlands, a high influenza vaccine uptake is reached among those belonging to risk groups for influenza. Each year, in October/November, general practitioners immunise patients aged 60 years or older and patients with risk-elevating diseases with stable high vaccination uptake rates above 71% across most parts of the Netherlands . However, if younger than 60 years and admitted for the first time with a high-risk diagnosis, patients are mostly not immunised since they did not belong to a risk group before. Also they are infrequently vaccinated in the hospital since there is no vaccination programme for hospitalised patients in the Netherlands.
In contrast, in both the Netherlands and most other European countries, vaccine uptake among HCWs remains low and influenza vaccination programmes have been voluntary. To be effective in reaching high vaccine coverage against influenza, a large variety of behavioural and organisational factors has to be targeted  and a setting- and culture- specific quantitative need assessment is essential to focus the programme on the most influential factors .
We applied the Intervention Mapping (IM) method  to structure the development of an influenza vaccination programme targeted at hospital staff. We here report the results of an evaluation of this programme. In the study, University Medical Centers (UMCs) from the Netherlands participated during the 2009/10 and 2010/11 influenza seasons. We primarily set out to determine the effects of the programme on vaccine coverage among HCWs using a pragmatic cluster randomised controlled trial. As clinical assessments from hospital settings are lacking, we also set out to determine the effects on patient outcomes during the studied influenza seasons.
Design, setting and participants
We aimed to assess the clustered effects of a multi-faceted influenza vaccination programme on influenza vaccine coverage in HCWs as well as the effect on influenza morbidity in hospitalised patients in UMCs in the Netherlands. In our trial, a cluster is the unit of randomisation defined as one UMC. In this study, we consider HCWs to be all employees working in the hospital. The study period included the influenza seasons 2009/10 and 2010/11.
To reach the objectives we conducted a pragmatic cluster randomised trial because the developed influenza vaccine implementation programme was best applied at hospital level rather than at individual level. All eight UMCs (Erasmus Medical Center, Rotterdam; Academic Medical Center, Amsterdam; University Medical Center, Groningen; University Medical Center, Utrecht; University Medical Center, Maastricht; Free University Medical Center, Amsterdam; University Medical Center, Nijmegen; Leiden University Medical Center, Leiden) were invited to participate in the trial. After permission from the Dutch Federation of UMCs, the board of directors of six of the eight UMCs agreed to randomisation at cluster level. The board of directors of the two remaining UMCs refused to be randomised because their institutions had already undertaken considerable efforts to raise influenza vaccine coverage among staff, but they agreed to act as external controls. Unfortunately, the two UMCs did not give permission to collect patient data.
At baseline, policies for the randomised UMCs were either to offer influenza vaccination to selected healthcare workers or not to vaccinate at all, and the highest vaccine coverage in any UMC was estimated at just below 27%. The baseline vaccine coverage in the external UMCs was somewhat higher reaching levels as high as estimated at 37%, and there was more experience with immunistion campaigns.
UMCs are tertiary referral centers each taking care of special hospitalised patient populations in the eight geographical regions of the Netherlands where they are placed. Acute care is delivered for a large number of patients who are admitted for a wide variety of indications.
In May 2009, prior to the upcoming 2009/10 influenza season, six UMCs were randomly allocated by computer (using the procedure Random in SPSS version 18.0) into two clusters, either the intervention or the control group, by a researcher blinded to the identity of the UMCs. Since the UMCs were about similar in size, number of HCWs and annual number of hospitalisations, we did not match before randomisation. Since we conducted a pragmatic study, the outcome of randomisation was neither blinded for the research group nor for the lead contacts of the UMCs. Although most HCWs were aware that they were targeted for vaccination, they did not know to which arm their UMC was randomly allocated. The study period covered the period from the first influenza vaccination campaign in September/October 2009 to the end of the influenza season 2010/11. The protocol of the trial was waived by the medical ethical committee of the University Medical Center Groningen for ethical approval according to the Dutch Law of Research with Humans (No. 2009.267). The study was conducted in accordance with the Dutch Law for the Protection of Personal Data (Wet Bescherming Persoonsgegevens) and the Declaration of Helsinki .
In November and December 2008, prior to the trial start in 2009, we conducted a survey to assess which behavioural and organisational factors were associated with vaccine uptake among hospital staff of the UMCs . An 11-item prediction model with nine behavioural and two demographic predictors could be developed that was highly accurate in discriminating vaccinated from non-vaccinated staff in approximately 95% of the study population. Subsequently, we used the Intervention Mapping (IM) method to thoroughly plan, develop and evaluate a programme that was directed at HCWs in order to influence their behaviour towards immunisation [15,18]. This IM method is a theoretical framework to systematically develop health education interventions and can be used as part of the dynamic process of planning intervention strategies in health education. It contains six consecutive steps: (i) a needs assessment, (ii) creating a matrix of proximal programme objectives, (iii) selecting theory-based intervention methods and practical strategies, (iv) programme planning, (v) adopting and implementing the programme, and (vi) monitoring and programme evaluation.
Various educational tools were developed following the proximal objectives based on the needs assessment (Box). Prior to the immunisation campaign in September 2009 and 2010, the programme educational tools were offered to the lead contact persons from the departments of occupational health of each UMC in the intervention and external group. These departments, in close collaboration with the communication units, are responsible for the influenza vaccination campaign. Information on the methods was provided to them by communication experts within the research group and they were encouraged to communicate the methods at various levels including the board of directors, heads of departments and staff members. The intervention and external group were allowed to make their own choices and decisions regarding the implementation of programme elements. An evaluation of the process showed that intervention and external UMCs targeted most of the behavioural determinants and choose to implement a variety of the developed methods, whereas the control UMCs targeted less determinants , Figure. However, actual exposure of HCWs to these methods was variable and in 2009 largely affected by the pandemic preparedness plans. Lead contacts from the control group did not receive the developed methods and were encouraged to follow their usual influenza vaccination policy. We did not seek to influence vaccine coverage among patients.
Box. Behavioural determinants associated with vaccine uptake and developed health education methods to increase influenza vaccine uptake, the Netherlands, 2009
The primary outcome measure of this trial was the influenza vaccine uptake among all HCWs at UMC level. Vaccine uptake was expressed as percentage calculated through dividing the number of all vaccinated HCWs by the total number of HCWs multiplied by 100. For financial administrative reasons all immunisations are accurately recorded at the hospital level, hence this information was regarded most valid.
Secondary outcome measures were absenteeism rates among HCWs during December of each study year as this is normally the month in which influenza circulates at epidemic levels . The cumulative absenteeism rates for the month December were provided by each department of occupational health of all UMCs after the influenza seasons. Vaccine uptake and absenteeism among HCWs were both analysed at cluster level.
As further secondary outcome, patient outcome data from two selected high risk departments i.e. paediatrics and internal medicine, were collected retrospectively for all patients hospitalised three days or more, to ensure nosocomial exposure during both study seasons. In the 2009/10 influenza season, a lower number of patients could be included after vaccination of HCWs, since the campaign had begun late in the epidemic, whereas we could observe a high number of patients during the complete season of 2010/11. The outcomes collected were laboratory-confirmed influenza and/or pneumonia, length of hospital stay in days, admittance to intensive care and duration. They were compiled by scrutinising computerised discharge letters from the patients’ medical files and information from the microbiology laboratories by two reviewers. Influenza was defined as laboratory-confirmed influenza A (all subtypes) or influenza B during hospital stay. Pneumonia was defined as any pneumonia which was clinically diagnosed during hospital stay. Since vaccination coverage was different between departments, patient data were analysed at department level. Since pneumonia is a common complication following influenza, influenza remains often undiagnosed and the combined outcome is regarded most accurate and specific. In accordance with previous studies among seniors we combined this outcome .
We were able to obtain patient outcome data on a large number of patients in two departments during the influenza seasons.
We aimed to include all HCWs from the eight UMCs prior to conducting the study. Sample size calculations for cluster randomised studies were applied. Based on the high vaccine uptake among patients (around 70%) we expected that we could raise the vaccine coverage of staff in the intervention group from 37%, the highest vaccination rate in all UMCs as estimated by questionnaire  to at least 70% and that the control group would remain at 37% coverage. We assumed that all eight UMCs would participate. A minimum of 32 participants per UMC (128 per cluster) were needed to provide more than 80% power if the intra-class correlation (ICC) was estimated at 10% and significance level was set at 5%. Given the much higher numbers of HCWs per UMC, smaller effects could be detected with adequate power.
Data were analysed using SPSS for Windows, version 18.0 and SAS statistical package 9.1. All outcomes were analysed at cluster level. In addition, patient outcomes were analysed at departmental level. For the primary outcome influenza vaccine coverage and absenteeism rates, we calculated risk differences (RD) and relative risks (RR) with their corresponding 95% confidence intervals (95% CI) and the levels of statistical significance in the different clusters for both influenza seasons combined. This was done by a specifically designed bootstrap program in R statistical software  to account for clustering. To account for dependencies of individual observations within hospitals and possible heterogeneity between hospitals we addressed our research questions within the generalised linear mixed model framework. To estimate RR, the binomial distribution was used employing the logarithmic function as link between the mean of the response and the linear part of the model using SAS statistical package. RD were obtained using the identity link function and the normal distribution. We calculated RR and corresponding 95% CI as well as levels of statistical significance for the patient outcomes pooled over both years after adjustments for small baseline differences of sex (see results). We chose to pool the data to obtain a more precise estimate of the effect because both seasons were dominated by influenza A(H1N1)pdm09 and vaccines matched the circulating strain in both seasons. Adjusted differences in duration of hospitalisation and intensive care admission between clusters were compared after transformation of extreme values to a clinically relevant maximum (30 days for hospital and seven days for intensive care stay). Results were similar as for the non-transformed values.
At the beginning of the measurements in 2009, the baseline characteristics at the level of the whole UMC were determined per group (Table 1). On average, the intervention UMCs were somewhat larger than control and external UMCs with more staff full time equivalents and a higher number of clinical admissions each year. However, the mean HCW/patient ratio was comparable for all three groups. The age and sex distribution of staff as estimated from a web-based survey in 2009 was similar as well (response rate 30.1%) (data not presented). The pooled baseline characteristics of patients from the selected departments of the intervention and control groups showed similar mean age and percentage of men in the intervention and control group (Table 2). The percentage of patients from the internal medicine department and study year 2010/11 was also similar between both groups.
Table 1. Baseline characteristics of University Medical Centers, randomised controlled trial in the Netherlands, 2009 (n=8)
Table 2. Baseline characteristics of patients in eight University Medical Centers by intervention/control and department, randomised controlled trial in the Netherlands, 2009–2011 (n=3,367)
Influenza vaccine uptake
In both study seasons, influenza vaccine coverage among HCWs was significantly higher in the intervention group compared with the control group (Table 2). In 2009 three influenza vaccination rounds were offered because of the emergence of the influenza A(H1N1)pdm09 pandemic virus. In all three groups coverage was highest for the first dose of the pandemic vaccine. In the intervention group the absolute difference in vaccine coverage compared with the control group, for the first dose of the pandemic vaccine was 23.7% (95% CI 4.3% to 47.8%, p<0.05). For the second pandemic vaccine dose, coverage was lower in all groups than for the first one, but still 21.4% higher in the intervention than in the control group (95% CI: 3.6% to 40.3%; p<0.05). The external UMCs, which were already more active in their vaccination campaign prior to the study than the randomised UMCs, reached even higher influenza vaccine uptake rates compared to the control UMCs in all vaccination rounds with an outstanding 44.0% absolute higher uptake of the first pandemic vaccine dose from 38.0% to 82.0% (95% CI: 30.0% to 53.7%; p<0.05). In 2010/11, when the pandemic threat was no longer an issue, coverage of the seasonal influenza vaccine was much lower than the pandemic vaccine coverage in the year before for each group. The absolute RD was the intervention and external group, respectively, compared with the control group (both p-levels <0.05).
To obtain more insights into exposure to different programme methods and the vaccine uptake, we related the number of targeted determinants to vaccine uptake (Figure). There was a clear trend towards increased vaccine coverage if more methods were applied. There was a significant correlation between the number of applied methods and vaccine coverage for both pandemic vaccines (first pandemic vaccine dose Spearman r=0.79, P=0.021; second pandemic vaccine dose Spearman r=0.90, P=0.003). Correlation estimates were not significant for the seasonal vaccines (2009/10: Spearman r=0.41, P=0.317; 2010/11: Spearman r=0.27, P=0.51).
Figure. Number of targeted behavioural determinants in the influenza vaccination programme and vaccine uptake in healthcare workers in University Medical Centers by vaccine, randomised controlled trial in the Netherlands, 2009–2011
Work absenteeism rates among HCWs were recorded for December 2009 and December 2010 (Table 3). For both seasons, absenteeism rates were 0.7% to 1.2% higher (absolute RD) on average in both the intervention and external cluster compared with the control group (all p<0.05 except for comparison between external and control UMCs in 2010 where p>0.05).
Table 3. Influenza vaccine uptake rates and work absenteeism rates for the month of December among healthcare workers in eight University Medical Centers, randomised controlled trial in the Netherlands, 2009–2011
Self-reported vaccine coverage in 2009/10 and 2010/11 influenza seasons among HCWs differed between the two studied departments. In 2009/10 coverage of a pandemic vaccine in the internal medicine and pediatric departments of intervention UMCs was 100% and 50%, and 92% and 81% in control UMCs, respectively. In 2010/11, corresponding vaccine coverage were 57% and 50%, and 51% and 44%, respectively. Over the two study years, the probability of being tested for the presence of influenza virus during the influenza epidemics was nearly twice as high in the intervention cluster compared with the control group, though not statistically significant (Table 4). Despite higher diagnostic testing rates, a diagnosis of influenza and/or pneumonia during hospitalisation was made in half as many cases in the internal medicine department of intervention UMCs compared with the control UMCs (RR=0.5; 95% CI: 0.3-0.9; p=0.015). Nosocomial pneumonia was reduced by a relative reduction of 76% (p=0.028). Other characteristics did not significantly differ between groups and no statistically significant differences were observed in the paediatric departments.
Table 4. Pooled analysis of patient outcomes by department for intervention and control of eight University Medical Centers, randomised controlled trial in the Netherlands, 2009-2011 (n=3,367)
In a 2008 publication, Nicoll et al. stated that there is strong evidence for immunising HCWs against influenza that take care of the elderly and the chronically ill in long-term care facilities. However, they did not find strong data on whether or not to vaccinate HCWs in other healthcare settings, such as hospitals .
Our study is the first hospital-based trial that showed that adopting a multi-faceted influenza vaccination programme was associated with improved vaccine coverage among HCWs. We also observed a lower risk for nosocomial influenza and/or pneumonia in hospitalised patients at the internal medicine departments during two consecutive influenza seasons, but we did not observe this effect in the studied paediatric departments.
It is surprising that only a small self-reported higher vaccine uptake in the departments of internal medicine led to our observation of a 50% reduction of the RR in patient outcomes. There may be several explanations for this finding. Actual vaccine coverage differences might have been higher than our self-reported estimates given that we observed an absolute higher difference of 23.7% (from 38.0% to 61.7%) and 11.9% (from 20.4% to 32.3%) respectively at group level in both seasons. Other explanations might be that not only vaccine uptake was higher in the intervention UMCs but that the programme led to more hygienic measures such as earlier diagnosis of influenza and isolation or better compliance with hand hygiene. This agrees with the fact that the number of influenza tests was twice higher in the intervention clusters than in the control clusters. Alternatively, baseline risks of patient outcomes might by chance have been different between the departments. For example, we did not have pre-intervention patient outcome prevalences of nosocomial influenza for both clusters. Potential of confounding bias cannot be completely ruled out, but is unlikely given similar age and sex distributions between the two groups.
Further, vaccine uptake was measured at the level of the UMCs and could not be obtained from all individual departments because of the centralisation of the immunisation in most UMCs. Of note, at baseline prior to the trial start, vaccine coverage might have been higher in departments of intervention UMCs than in control UMCs. Self-reported data from HCWs showed, however, that the seasonal influenza vaccine coverage in 2008/09 was 44% and 14% among HCWs of the internal medicine and paediatric departments in intervention UMCs and 54% and 58% in control UMCs, respectively, hence baseline differences cannot explain the improved coverage. The uptake at UMC level most probably accurately reflects the coverage in most but not all departments as observed for the departments of paediatrics and internal medicine. The self-reported coverage was almost twice higher than the overall UMC level data because of the high-risk residents of these departments and longer tradition of taking hygienic preventive measures against infectious diseases in internal medicine and paediatric departments, as compared with most other departments.
The lead contacts and researchers were not blinded for the allocated strategy; hence this may have caused information bias. However, since the numbers of administered vaccines is a marker of quality of care in the UMCs and administration has financial consequences, it is highly unlikely that such bias has occurred.
A major strength of the study includes the randomised design which resulted in largely comparable HCWs and patient populations over the study years. Also, the presence of a control group accounted for natural fluctuation in vaccine coverage as well as external factors at a national level, and the presence of an external group confirming the positive correlation between a targeted campaign and influenza vaccine uptake among HCWs was a major strength. Moreover, the size of the trial HCWs population and patient population was more than adequate to obtain highly precise estimates of the main effects. Finally, in day-to-day practice swabbing is not routinely done and can therefore not have affected differentially the intervention and control UMCs.
The work absenteeism rate was 1.2 HCWs per 100 HCWs higher in the whole month of December 2009 in the intervention than in control clusters. Since testing for influenza appeared to be more frequent in intervention than control UMCs, if anything, it is likely a marker of stricter working rules applied during influenza seasons in the intervention compared with control UMCs. Obviously, routine swabbing of all patients suspected of influenza would have been the ideal study outcome. Because the pandemic threat was over in 2010 , the absolute risk difference for the trial population was down to 0.7 per 100 HCWs during the latter study season. One participating UMC from the control group could not reliably obtain absenteeism data at their UMC level. However, department specific data that could be obtained showed similar rates as within similar departments of the other control UMCs.
The participating hospitals were tertiary centers and the observed effects may not necessarily be applicable to all types of hospitals. In a survey among administrators of all hospitals in the Netherlands in 2010 with a response rate of over 53%, we observed that the average vaccine coverage of staff reported by the administrators was comparable with the coverage in control UMCs (17.7% versus 17.8% in our study) . Interestingly, in that survey we observed a clear association between economic spending on the immunisation programme in these hospitals and vaccine coverage, with higher programme spending (>1,250 Euro versus ≤1,250 euro) leading to 9% improved coverage (24% versus 15%; 95% CI for the difference: 0.7% to 17%). We also observed in our trial that the higher the number of determinants targeted, the higher vaccine uptake in both study seasons (Figure). Although evidence is scarce, the introduction of a thoroughly developed programme likely leads to improved coverage in any type of hospital.
In 2009, the influenza A(H1N1)pdm09 pandemic also affected the Netherlands, starting in early October and ending in December 2009. Following the advice from the World Health Organization and the Dutch Health Council, the Ministry of Health decided that risk patient groups should be prioritised for pandemic vaccination against this new influenza variant. HCWs were considered both as an important potential transmitter of influenza to risk patients and essential in the care of patients during a pandemic and were considered a target group for pandemic vaccination. As in most other countries, the pandemic was associated with enormous media attention and fear in the community. Therefore, in summer of 2009, all UMCs installed their pandemic response team and prepared for a worst case scenario . The installed preventive measures were very costly, reaching hundred thousands of Euros per UMC, and led to pressure on both management and HCWs. It was therefore unexpected to see that despite general circumstances, both the intervention and external cluster reached higher vaccine coverage than the controls.
After the pandemic was declared over and it appeared to be much less severe than had initially been feared , we hypothesized that many HCWs were displeased about the pressure on them and the measures taken. In 2010/11, therefore, seasonal vaccine coverage was half the coverage of the first dose of pandemic vaccine, and despite higher coverage in the intervention than the control cluster, it remained below a staggering low of 30%.
In conclusion, our results suggest that a multi-faceted influenza vaccination programme for hospital HCWs is effective in raising vaccine uptake among HCWs. Although bias cannot be completely ruled out, an increase in vaccine coverage was associated with a decrease in influenza and/or pneumonia among patients during hospitalisation. Given the current evidence for annual risks of influenza complications in hospital and benefits of vaccination, and the low voluntary coverage, mandatory programmes should be seriously considered.
This study was funded by the Netherlands Organisation for Health Research and Development (ZonMw), grant number 125030012. The funding organisation had neither a role in conduct and analysis of the data-analysis nor in the report of the study.
Conflict of interest
JRD conducted the study, collected and analysed the data and drafted the manuscript. JB re-analysed the data and commented on the final version of the manuscript. GF, AGB, MDP, HJ, AB, ES, MV and PG contributed to the design of the study, were lead contacts during the study and critically reviewed the manuscript. EH obtained funding, supervised the conduct and report of the study and critically commented on the manuscript. JRD and EH had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. All authors read and approved the final version of a manuscript.
Trial registration number
- Centers for Disease Control and Prevention (CDC). Influenza vaccination coverage among health-care personnel - United States, 2010-11 influenza season. MMWR Morb Mortal Wkly Rep. 2011;60(32):1073-7. PMid:21849963.
- Kroneman M, Paget WJ, van Essen GA. Influenza vaccination in Europe: An inventory of strategies to reach target populations and optimise vaccination uptake. Euro Surveill. 2003;8(6):pii=418. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=418. PMid:12878803.
- Mereckiene J, Cotter S, Nicoll A, Lévy-Bruhl D, Ferro A, Tridente G, et al. National Seasonal Influenza Vaccination Survey in Europe, 2008. Euro Surveill. 2008;13(43):pii=19017. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=19017
- Lee N, Choi KW, Chan PK, Hui DS, Lui GC, Wong BC, et al. Outcomes of adults hospitalised with severe influenza. Thorax. 2010;65(6):510-5. http://dx.doi.org/10.1136/thx.2009.130799. PMid:20522848.
- Hak E, Hoes AW, Verheij TJ. Influenza vaccinations: Who needs them and when? Drugs. 2002;62(17):2413-20. http://dx.doi.org/10.2165/00003495-200262170-00001. PMid:12421099.
- Everts RJ, Hanger HC, Jennings LC, Hawkins A, Sainsbury R. Outbreaks of influenza A among elderly hospital inpatients. N Z Med J. 1996;109(1026):272-4. PMid:8769048.
- Weingarten S, Riedinger M, Bolton LB, Miles P, Ault M. Barriers to influenza vaccine acceptance. A survey of physicians and nurses. Am J Infect Control. 1989;17(4):202-7. http://dx.doi.org/10.1016/0196-6553(89)90129-6
- van den Dool C, Bonten MJ, Hak E, Wallinga J. Modeling the effects of influenza vaccination of health care workers in hospital departments. Vaccine. 2009;27(44):6261-7. http://dx.doi.org/10.1016/j.vaccine.2009.07.104. PMid:19686690.
- Osterholm MT, Kelley NS, Sommer A, Belongia EA. Efficacy and effectiveness of influenza vaccines: A systematic review and meta-analysis. Lancet Infect Dis. 2012;12(1):36-44. http://dx.doi.org/10.1016/S1473-3099(11)70295-X
- van den Dool C, Bonten MJ, Hak E, Heijne JC, Wallinga J. The effects of influenza vaccination of health care workers in nursing homes: Insights from a mathematical model. PLoS Med. 2008;5(10):e200. http://dx.doi.org/10.1371/journal.pmed.0050200. PMid:18959470. PMCid:2573905.
- Thomas RE, Jefferson T, Lasserson TJ. Influenza vaccination for healthcare workers who work with the elderly. Cochrane Database Syst Rev. 2010;(2):CD005187. PMid:20166073.
- Tacken M, Mulder J, van den Hoogen H, Tiersma W, Donkers J, Verheij R, et al. Monitoring Nationaal Programma Grieppreventie 2008. [Monitoring the National Program for Influenza Prevention 2008]. Netherlands Information Network of General Practice (LINH). 2009; Dutch. Available from: http://www.nivel.nl/sites/default/files/bestanden/grieprap2008%20met%20erratum_.pdf
- Riphagen-Dalhuisen J, Gefenaite G, Hak E. Predictors of seasonal influenza vaccination among healthcare workers in hospitals: A descriptive meta-analysis. Occup Environ Med. 2012;69(4):230-5. http://dx.doi.org/10.1136/oemed-2011-100134 PMid:22172951.
- Looijmans-van den Akker I, Hulscher ME, Verheij TJ, Riphagen-Dalhuisen J, van Delden JJ, Hak E. How to develop a program to increase influenza vaccine uptake among workers in health care settings? Implement Sci. 2011;6:47. http://dx.doi.org/10.1186/1748-5908-6-47. PMid:21595877. PMCid:3115899.
- Bartholomew LK, Parcel GS, Kok G. Intervention mapping: a process for developing theory- and evidence -based health education programs. Health Educ Behav. 1998;25(5):545-63. http://dx.doi.org/10.1177/109019819802500502. PMid:9768376.
- World Medical Association (WMA). WMA Declaration of Helsinki: Ehtical Principles for Medical Research Involving Human Subjects. Seoul: WMA; Oct 2008; Available from: www.wma.net/en/30publications/10policies/b3/
- Hopman CE, Riphagen-Dalhuisen J, Looijmans-van den Akker I, Frijstein G, Van der Geest-Blankert AD, Danhof-Pont MB, et al. Determination of factors required to increase uptake of influenza vaccination among hospital-based healthcare workers. J Hosp Infect. 2011;77(4):327-31. http://dx.doi.org/10.1016/j.jhin.2010.10.009. PMid:21316803.
- Riphagen-Dalhuisen J, Frijstein G, van der Geest-Blankert N, Danhof-Pont M, de Jager H, Bos N, et al. Planning and process evaluation of a multi-faceted influenza vaccination implementation strategy for health care workers in acute health care settings. BMC Infect Dis. 2013;13:235 http://dx.doi.org/10.1186/1471-2334-13-235. PMid:23701921. PMCid:3680164.
- Netherlands institute for health services research (NIVEL). Wekelijkse griepcijfers Nederland. [Weekly influenza report, the Netherlands]. Utrecht: NIVEL. 19 Jun 2013. Dutch. Available from: http://www.nivel.nl/dossier/wekelijkse-griepcijfers-nederland
- R Development Core Team. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2010.
- Nicoll A, Ciancio BC, Tsolova S, Blank PR, Yilmaz C. The scientific basis for offering seasonal influenza immunisation to risk groups in Europe. Euro Surveill. 2008;13(43):pii=19018. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=19018
- Dieleman J, Romio S, Johansen K, Weibel D, Bonhoeffer J, Sturkenboom M, et al. Guillain-Barre syndrome and adjuvanted pandemic influenza A (H1N1) 2009 vaccine: multinational case-control study in Europe. BMJ. 2011;343:d3908 http://dx.doi.org/10.1136/bmj.d3908. PMid:21750072. PMCid:3134565.
- Riphagen-Dalhuisen J, Kuiphuis JC, Procé AR, Luytjes W, Postma MJ, Hak E. Contributing factors to influenza vaccine uptake in general hospitals: an explorative management questionnaire study from the Netherlands. BMC Public Health. 2012;12:1101. http://dx.doi.org/10.1186/1471-2458-12-1101. PMid:23259743. PMCid:3545720.
- van der Sande MA, Jacobi A, Meijer A, Wallinga J, van der Hoek W, van der Lubben M. The 2009 influenza A (H1N1) pandemic. Management and vaccination strategies in The Netherlands. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2013;56(1):67-75. http://dx.doi.org/10.1007/s00103-012-1582-4. PMid:23275958.