Applicability of evidence-based practice in public health : risk assessment on Q fever under an ongoing outbreak

With reference to the Q fever outbreak in the Netherlands in 2009–10, we tested if an evidencebased approach, comparable to the methodology used in clinical medicine, was appropriate for giving public health advice under time constrains. According to the principles of evidence-based methodologies, articles were retrieved from bibliographic databases and categorised by type and size, outcome, strengths and limitations. The risk assessment was conducted in two months and involved six staff members. We retrieved and read 559 abstracts and selected approximately 150 full text articles. The most striking finding was the lack of sound scientific evidence behind standard treatment regimes for Q fever in pregnancy. Difficulties in applying existing evidence rating systems and in expressing uncertainties were identified as problems during the process. By systematically assessing the evidence on several questions about Q fever, we were able to draw new conclusions and specify earlier statements. We found it difficult to grade the mostly observational studies with the known evidence-based grading systems. There is need to develop new methods for grading evidence from different sources in the field of public health. We conclude that an evidencebased approach is feasible for providing a risk assessment within two to three months.


Introduction
The European Centre for Disease Prevention and Control (ECDC) may be requested by the European Commission, the Member States of the European Union (EU), third countries and international organisations to provide scientific or technical assistance in any field within its mandate.Regarding the Q fever outbreak in the Netherlands in 2009 and 2010 [1], ECDC was asked by the European Commission to assess the following questions: (i) What is the risk and safety of blood transfusions, especially from donors who are asymptomatic or still in the incubation phase of the disease?(ii) What is known on the impact on health of chronic Q fever disease?(iii) What is the impact on health for risk groups like pregnant women?(iv) Is it advisable to strengthen the surveillance of new cases?After a short-term risk assessment had been conducted within a few days, we tested if an evidencebased approach, comparable to the methodology used in clinical medicine, was appropriate for giving more in-depth public health advice on Q fever to policy makers and public health practitioners.Evidence-based methodologies are increasingly discussed and applied in public health practice and health promotion.There is a growing consensus that scientific and technical advice in the field of public health should rely on evidence-based science and technology and should aim to support evidence-based decision making [2].During this process, we addressed two questions: Does an evidence-based approach work when advice has to be given in an outbreak situation, i.e. under time constraints?And if so, does change the conclusions compared with more traditional, expert-based approaches?
In this paper we summarise the risk assessment and discuss our experiences with applying evidence-based methodology in its production.

Background
Q fever is a zoonotic disease caused by the intracellular bacterium Coxiella burnetii.A wide range of wild and domestic animals (including arthropods, birds, rodents, cats, and livestock) serve as a natural reservoir for the pathogen [3].Acute Q fever most often presents with non-specific influenza-like symptoms, and the infection is asymptomatic in approximately 50% of cases.A subset of the patients develops chronic Q fever, a potentially life-threatening condition.Since 2007, the Netherlands has been experiencing the largest Q fever outbreak ever reported in the literature.As of the end of 2010 approximately 4,000 people have been affected and at least 14 of these patients, nearly all of them with severe underlying conditions, have died.

Methods
On the basis of a rapid risk assessment in the beginning of 2010, a more comprehensive risk assessment was performed according to the principles of evidencebased medicine (EBM) [4].In March 2010, a working group was established at the ECDC including one medical librarian and five reviewers with broad epidemiological experience.Reviews and original research articles were retrieved from PubMed and Embase bibliographic databases.The search strategies covered different aspects of Q fever: blood, pregnancy, chronic diseases, occupational exposure, transmission and surveillance of the disease.The concepts used in the search were taken from the controlled vocabulary available in the bibliographic databases (i.e.MeSH and Emtree terms).These were complemented with multiple field search combinations by using natural vocabulary (i.e.keywords).The results were limited to records published from 1970 onwards.The search was not restricted to articles written in English.Studies were selected according to relevance for the different questions, using inclusion criteria agreed upon before the review process started.When in doubt about inclusion of a paper, it was discussed with the group of reviewers.We included only studies reporting on outbreaks and having primary results from research.Excluding commentaries, editorials, single case reports.
The studies were categorised according to the following study designs: reviews, trials and observational studies.The observational studies were sub-classified into the following categories: cohort studies, case series, case-control studies, case studies, cross-sectional studies, time series, 'before and after' studies.The following sections were included in the evidence table: bibliographic citation, type of study, number of patients or size of population, study outcome, strengths of study and limitations of study.The results were presented to, and discussed with, an expert panel with 18 representatives from the Netherlands, France, Germany, the United Kingdom, the United States (US), the European Food Safety Authority and the European Commission.
The applicability of the EBM methods was assessed during the process of preparing the risk assessment in discussions with the panel of experts and the advisory forum of ECDC, and after publication of the risk assessment in discussion among the team of reviewers.

Results
The risk assessment was conducted within two months (mid-March to mid-May 2010), and involved six staff members (at approximately half of their working time).A total of 559 abstracts were retrieved and read, and approximately 150 full text articles were selected for inclusion in the evidence base.A meeting with experts from Europe and the US was held in Paris in April 2010.The full report describing the exact methodology including search strategies, evidence tables and recommendations has been published [5].
The following results regarding the four questions were obtained by the risk assessment: Blood Q fever can be transmitted through direct contact to blood, and cases have been reported among laboratory personnel and pathologists [6].The exact duration of bacteraemia is unknown.To date there has been only one documented case of human-to-human transmission via blood transfusion [7].One case of transmission from a bone marrow transplant in an immunosuppressed patient has also been reported [8].Q fever has also been transmitted via organ transplantation in animals [9].Donors of organs, cells or tissues are not routinely screened for C. burnetii [10].Blood donors have been examined for Q fever mainly in epidemic settings [11].
The following recommendations were made, based on the evidence as described in the full report [5], and bearing the precautionary principle in mind: • During an outbreak, the affected area should be defined and safety precautions should be considered, such as screening of blood and tissue products, active surveillance among blood and tissue recipients, and screening of donors.• It should be considered to defer travellers returning from an epidemic area from donating blood for six weeks after their arrival in a low-prevalence area.• An antibiotic course could be considered for blood transfusion recipients at particularly high risk of chronic disease, such as patients with heart valve defects, in an epidemic area.• Donors who have had an acute Q fever infection should be deferred from giving blood for two years following the date of confirmed cure from acute infection (absence of phase 1 antibodies).

Chronic Q fever
A cumulative point estimate calculated from all the studies included in this assessment, gave an overall average prevalence for chronic Q fever of 1.9% of acute cases.Chronic Q fever can develop after, or appear as an asymptomatic infection [12,13].The fatality rate for chronic Q fever may vary from 5% to 60% [14].Risk factors for developing chronic disease are mainly connected to the host and include heart valve defect, heart valve prosthesis or arterial graft, aneurisms, malignancies, and immunosuppression.Medical treatment for chronic Q fever should be at least one year with more than one drug.The optimal treatment of chronic Q fever is still debated and the recommended duration of treatment varies from one year up to a lifespan [15].Most authors today recommend broad-spectrum tetracyclines, preferably doxycycline in combination with hydroxychloroquine for at least 18 months [16].During an outbreak, three possible strategies are described in the risk assessment for population-wide, targeted case finding and individual follow-up to identify patients at risk in the outbreak area: (i) Serological testing, during an outbreak, of all patients with known heart valve disease or vascular grafts, in order to identify them early and refer them for treatment.(ii) Testing of all patients with acute Q fever with echocardiography for heart valve lesions.(iii) Individual serological follow-up after acute Q fever infection and raising awareness among the general population and physicians.An effective whole-cell vaccine is used for defined risk groups in Australia but is not licensed or used in any other country [3].
The recommendations below are based mainly on evidence from observational studies and the judgements from the expert panel: • Acute and chronic cases need to be followed up individually by primary and secondary healthcare services.• Special attention should be paid to risk groups, i.e.
people with valvular heart disease, vascular diseases, cancer or a compromised immune system.• Among these risk groups, targeted case-finding should be considered as an option.• People with known risk factors should not visit farms infested with Q fever.• The formalin-inactivated whole-cell Q fever vaccine is effective, but pre-vaccination testing is necessary due to high reactogenicity in persons who have earlier been infected with C. burnetii, making the vaccine more suitable for defined risk groups than for general vaccination.• Making the vaccine available for defined risk groups should be considered.• There is need to initiate good prospective cohort studies and trials with control groups when ethically feasible, to obtain more robust evidence on how to prevent and control outbreaks of Q fever, and on how to diagnose and treat acute and chronic disease at the clinical level.

Pregnancy
The available evidence with regard to effects of Q fever infection in pregnant women is limited [17].There are indications for severe disease and progress towards chronic infection/disease in pregnant women.To what extent the risk of pregnant women for severe Q fever outcomes differs from the risk of the general (female) population and in comparison to other well-known risk groups cannot be quantified based on the current available evidence.The presence of C. burnetii in fetal tissue after abortion or intrauterine fetal death has been reported, but also in healthy children delivered from infected mothers with placentitis.Transplacental transmission seems to be possible but its association with adverse obstetrical outcomes remains incompletely understood as well as the consequences for the child in case of live birth.Several case reports on adverse pregnancy outcomes associated with maternal Q fever exist [15,16,18,19].The largest published case series summarising the serological profiles and pregnancy outcomes of 53 women during a period of 15 years in southern France, found obstetric complications in 70% of all observed pregnancies, and in 81% of the nontreated pregnancies [17].So far, this case series also provides some indication that long-term antibiotic therapy with co-trimoxazole has the potential to prevent the most severe pregnancy outcomes [17].
The evidence led to the following conclusions and recommendations: • There is some indication that long-term antibiotic therapy with co-trimoxazole has the potential to prevent severe pregnancy outcomes associated with Q fever, but the evidence is based on a case series without randomisation and without controlling for potential biases.• As long as no further evidence from high quality treatment studies is available, pregnant women with diagnosed Q fever infection should be treated with antibiotics until the end of the pregnancy.However, the scientific basis for this recommendation is weak, and ECDC would strongly recommend that randomised controlled trials are performed to obtain more reliable evidence.Pregnant women should be advised not to visit farms in affected areas.• ECDC does not recommend against breastfeeding by mothers with proven C. burnetii infection, except in cases of chronic disease that need long-term treatment of the mother.

Transmission and surveillance
There is scientific evidence (experimentally, epidemiologically and by use of statistical models) that airborne transmission of C. burnetii is the principal mode of transmission to humans [1][2][3].Airborne transmission includes long-distance (indirect) transmission of the aerosolised bacteria and direct transmission through inhalation of droplets, aerosols and dust during contact with infected animals, contaminated animal products (e.g.wool or straw) and contaminated clothing [20][21][22][23].
An association between transmission to humans and environmental factors, i.e. wind speed, dry weather conditions and vegetation density, has also been established [21,24,25].The distance infectious particles can spread by air is a point of controversy.Several estimates ranging from 400 m to 40 km are provided in the literature from different outbreak investigations [21,26,27].More sound data was provided from a Dutch study on a Geographical Information System, which demonstrated that the risk of infection is highest within a 5 km radius from the source [28].
There have only been a few studies that describe foodborne transmission of C. burnetii.These indicated that consumption of contaminated food may lead to seroconversion, but not to clinical disease [29].Data from experiments in which contaminated milk was fed to healthy volunteers gave no clear evidence about transmission [30].Single case reports indicate a low rate of human-to-human transmission during birth or through breastfeeding, sexual transmission, transplacental transmission and spread after autopsies [31][32][33].Active surveillance (i.e.active serological targeted case finding for Q fever independent of clinical symptoms) helped to detect cases of acute Q fever in the general population, in patients with valvular heart diseases or vascular grafts, and in pregnant women [34][35][36][37].In epidemic situations, awareness campaigns addressing both the general public and medical care providers were successfully used to enhance case finding [27,36,38].
We derived the following conclusions and recommendations from the reviewed evidence: • Available evidence suggests an effective range of airborne spread of C. burnetii from infested farms in the Netherlands of less than 5 km.The risk of airborne spread is therefore limited to areas close to outbreak sources.• Active surveillance or case finding for acute Q fever in risk groups on a local level and for a defined period of time is reported feasible and an efficient method for detecting acute infections.• In areas adjacent to epidemic settings (≤5 km from the source), awareness campaigns among healthcare providers should be initiated.• If the area also affects other Member States, the responsible public health authorities need to inform their cross-border counterparts.• Sharing of information between public health and veterinary authorities would facilitate early recognition of an outbreak.

Discussion
By systematically assessing the evidence for the four questions from the European Commission related to the Q fever epidemic in the Netherlands, we explored the applicability of an evidence based methodology in a medium-term (i.e. two to three months) public health risk assessment.When compared with the earlier short-term risk assessment which had been conducted within a few days by the ECDC, the use of EBM allowed us not only to refine some of the previous statements, but also to draw some new conclusions.The most remarkable finding was the lack of sound evidence behind some standard treatment regimes (e.g.longterm co-trimoxazole treatment for pregnant women).This should be an incentive for the research community to initiate high quality studies on the effects of different clinical and public health interventions on Q fever and pregnancy.This knowledge gap has also been recognised by research institutes in the Netherlands, and a first well designed study about screening strategies for Q fever among pregnant women in risk areas has recently been launched [39].We were also able to provide more accurate information on the risk for chronic disease, and on the risk for possible spread of C. burnetii to neighbouring countries.
While conducting this risk assessment, we identified several potential problems that could make it difficult to conduct an EBM approach in a public health setting, including logistical and managerial problems, difficulties in applying existing evidence rating systems, and difficulties in expressing uncertainties.
After reviewing the process of developing the risk assessment, we found that endorsement by the top management is essential to promote EBM as a core part of public health practice, and several steps might be considered by the management to foster EBM as part of daily working routine.To work on a medium-term evidence-based risk assessment within an organisation where everybody is preoccupied with other assignments, turned out to be logistically difficult.The Q fever risk assessment was developed within a time frame of approximately two months, and six experts were actively involved in the process.A group of experienced people should be clearly assigned to the task and share the work to be able to deliver in the short time frames.We found that discussions with a panel of experts are mandatory, but the questions to be addressed and the evidence should be prepared by the review team.Experts should be selected in a transparent way, i.e. by using an existing database of experts with well defined profiles and conflict of interest declaration.
An evidence-based approach normally includes grading of the quality of the studies and thereafter grading of recommendations.In many settings of infectious disease epidemiology, however, observational studies or natural experiments are the only feasible study designs, i.e. evidence at the lower level of the evidence hierarchy when referring to the GRADE system [40].
That was also the case in this situation.Nevertheless, we found that such studies can still be judged according to their quality.A study can be of high quality even if its design does not fulfil the strict criteria for 'high quality evidence'.Existing grading systems, however, were perceived as not appropriate since almost all studies which were included for our risk assessment would have been graded very low.To enhance the information level with regard to study quality the group decided instead to indicate strengths and limitations.We found that there is a need to develop new tools and methods for grading evidence from different sources (especially from observational studies) in the field of public health and infectious diseases.
To conduct comprehensive, evidence-based risk assessments is time and resource consuming and may not be feasible for all the assessments required when threats emerge.A rapid assessment, conducted within few days of the occurrence of an event, is often needed to provide immediate guidance.It relies on review of easily assessable evidence from different sources, including review articles, websites of internationally recognised organisations and textbooks, which might be outdated, not transparent on conclusions and presenting diverging views.It is hardly possible, however, to apply the classical evidence-based methodology on a two-day risk assessment, and EBM was not designed to do so.On the other hand, these constraints are no justification for disregarding the principles of EBM when conducting rapid risk assessments: transparency, reproducibility and validity of all scientific advice given to the public, to professionals or to other stakeholders.Following these principles under pressure of time will probably reveal a higher level of uncertainty about the conclusions and recommendations when compared to medium-or long-term risk assessments.
We are aware that it is difficult, especially for public health agencies, to translate scientific uncertainty into policy advice.Stakeholders expect certainty and clear answers.However, we also believe that public health advice and policy is most consistent if scientific uncertainty is included in the assessment and the decisionmaking process as information, not ignorance.The decision of starting a full assessment should balance the expected benefits against the resources needed and the time it will take to produce it.There is need to define indications for doing evidence-based risk assessments under different time constraints.
In this assessment we tested whether an evidencebased approach, comparable to the methodology used in clinical medicine is appropriate for giving public health advice under an ongoing outbreak.We found that an evidence-based approach is feasible for providing an intermediate-term risk assessment within two to three months.Working explicitly and transparently with methods, evidence and experts will result in higher quality of public health advice.
It should be expected that recommendations and decisions for any scientific advice are based on the best available evidence and that appropriate methods are employed to search and analyse the evidence.We think there is a need to incorporate EBM as part of the goals and objectives for project managers and programme leaders in public health, and continuous EBM training should be established in organisations and institutes which are involved in producing general public health recommendations and assessments.To support the use of EBM in public health, ECDC has established a one-week training course, held for the first time in November 2010, which has been open also to external participants since May 2011.