Perspectives Evidence-Based Medicine

Control of acute communicable disease incidents demands rapid risk assessment, often with minimal peer-reviewed literature available but conducted in the public's view. This paper explores how methods of evidence-based medicine (EBM) can be applied in this scenario to improve decision making and risk communication. A working group with members from EBM organisations, public health institutions and the European Centre for Disease Prevention and Control used a six-stage framework for rapid risk assessments: preparation, risk detection/verification, risk assessment, development of advice, implementation, and evaluation. It concluded that data from observational studies, surveillance and modelling play a vital role in the evidence base. However, there is a need to further develop protocols and standards, to perform, report and register outbreak investigations more systematically and rigorously, and to allow rapid retrieval of the evidence in emergencies. Lack of evidence for risk assessment and advice (usual for new and emerging diseases) should be made explicit to policy makers and the public. Priorities are to improve templates for reporting and assessing the quality of case and outbreak reports, apply grading systems to evidence generated from field investigations, improve retrieval systems for incident reports internationally, and assess how to communicate uncertainties of scientific evidence more explicitly.


Introduction
Public health agencies responsible for the control of public health emergencies are expected to work according to the best standards of scientific evidence.They need to be explicit about the source, type, quality, scope and completeness of the evidence, so that policy makers, politicians and the public can understand the evolving nature of evidence, its strengths and limitations [1].Even in the acute situation of infectious disease emergencies such as an influenza pandemic, agreed protocols for developing policy and advice should be followed.However, there are two important challenges: reliance upon limited field investigations and population surveillance data, and the speed with which evidence has to be identified and synthesised.
In 2010 the European Centre for Disease Prevention and Control (ECDC) set up a working group to review the potential utility of currently used evidence-based medicine (EBM) tools and risk assessment tools in realistic communicable disease outbreak scenarios, and to propose new tools [2].A group of experts from 12 countries working in EBM and public health institutions or at ECDC, with a broad range of experience in public health methodology and infectious diseases, were appointed to give guidance on how to strengthen the scientific work at ECDC by adapting and applying EBM methods that were practical and applicable in the environment of infectious diseases and public health.
In this paper we report the conclusions on how to apply the principles of EBM in situations where rapid risk assessment is needed.

Methods and results
The working group presented the experiences of Member States in providing evidence-based guidance in circumstances when time was short, including the influenza pandemic in 2009 [3] and the Q fever epidemic in the Netherlands [4].Consensus within the group was reached through informal group processes, through plenary and smaller group discussions, and by review of draft texts by the members and work colleagues in their institutions.The group members are listed at the end of the article.
The development of evidence for control of any incident, outbreak or pandemic was conceptualised as a knowledge cycle in which data are collated from surveillance and field investigation reports and peerreviewed literature, rapidly appraised and used to assess risks, develop advice and implement control measures.Continued surveillance, monitoring and auditing further consolidate the evidence base and allow refinement of risk assessment and evaluation of the effectiveness of interventions (Figure 1).Usually in the acute incident the knowledge cycle is entered at the risk assessment stage, when a report of an incident has to be verified, evidence collated and synthesised, and the risk assessed.
We identified six stages that need to be considered when preparing a rapid risk assessment under time constraints, and the need for improvement in each.They are summarised in the Table and described in detail below.

Stage 1: Preparatory phase
Alerting and surveillance systems should be set up that are regularly reviewed for fitness for purpose [5].For newly emerging infections, the published data available to carry out systematic reviews will necessarily be very limited.It is therefore vital that critical summaries of evidence about epidemiology and control of these diseases are kept up to date and accessible internationally, including specifying key gaps in knowledge and suggesting appropriate models for risk assessment.Outbreak investigations are vital for defining epidemiological characteristics of specific pathogens (e.g.reproduction number) and can be used to evaluate the success of interventions [6,7].However, to the best of our knowledge there are no agreed international standards for outbreak investigation and reporting.The value of field investigations would be greatly improved if a standardised framework for conducting, reporting, and synthesising data from outbreak investigations was used.Such standards exist for strengthening the reporting of observational studies in epidemiology (STROBE) [8], for the transparent reporting of evaluations of non-randomised designs (TREND) [9], and for meta-analysis of observational studies (MOOSE) [10].Fine-tuning and evaluation for their application to outbreak situations has been undertaken for hospital Ongoing field investigations Guidelines outbreaks (the outbreak reports and intervention studies of nosocomial infection (ORION) statement) [11].For outbreak reports to be useful to others in a timely way, there needs to be an international repository of such reports and international agreement to make data rapidly available to investigators.
We identified tools and decision aids that we think would greatly improve public health decision making in acute outbreak situations.
• Up-to-date critical summaries of evidence from epidemiology and control of infectious diseases; • Quality standards for performance and reporting of surveillance and field investigations; • An international database of outbreak reports, accessible for all and with a user-friendly search function.

Stage 2: Incident verification
The critical step at this stage is to recognise the signal among the background noise of information.The agreed terminology outlining the epidemic intelligence process is the following: • A signal needs to be filtered; • An event needs to be validated; • A validated event needs to be analysed.
In order to reduce the risk of bias, reproducible, transparent and explicit incident verification protocols should be followed.The process of verification requires rapid international communication networks of communicable diseases units.Algorithms should include trigger levels for upscaling, and stopping rules, to allow control agencies to agree that further investigation or more detailed risk assessment are not considered appropriate so that resources can be prioritised efficiently [12].Tools required for this stage: • International alerting and verification systems (e.g. the European Union's Early Warning and Response System [13]), • Effective communication platforms (e.g.The European Union's Epidemic Intelligence Information System [14]).

Stage 3: Assessment of risk
This stage follows the verification of a threat and should address specific population groups at risk of more severe disease/outcome (e.g.pregnant women, the elderly, young children and immune-compromised individuals), and those at increased risk of exposure (e.g.healthcare workers).For rare, new and emerging infections there may be little or no peer-reviewed literature, and assessments will depend on field investigations, data from ongoing surveillance, and communication with experts in other countries.A comprehensive international database of outbreaks does currently not exist.Systematic methods for rapid searching and appraisal need to be developed that are appropriate to the time scales involved.
In order to reduce bias and to provide transparent quality assurance, risk assessment protocols and algorithms should be followed, and these should explicitly include frameworks for the synthesis of different types of evidence in relation to public health questions (e.g.risk of influenza A(H1N1) infection to pregnant women at different stages of pregnancy), admit to gaps and uncertainties in the evidence and possible alternative explanations of findings.Evidence should be classified by type (e.g.case report, population surveillance, field investigation) and study quality assessed through evidence-based checklists or tools such as the graphic approach to epidemiology (GATE) instrument for critical appraisal [15] and rapid risk assessment algorithms [16].Tools required for this stage: • A protocol for rapid searching for relevant peerreviewed and grey literature, • Checklists and templates for rapid appraisal of the evidence, • An international database on disease incidents and outbreak reports, • A rapid risk assessment procedure and tool.

Stage 4: Developing advice
Guidance will need to recognise explicitly the situational context and the population groups to which it is applied, but should seek to follow agreed EBM principles as embodied in, for example, the guidelines evaluation tool AGREE II (appraisal of guidelines for research and evaluation) [17].The grading of recommendations applicability, development and evaluation (GRADE) instrument was developed to evaluate and make explicit the steps from evidence to recommendations about treatments of diseases, but these principles also apply when a public health decision is to be made under time constraints [18].An essential part of developing advice is to state clearly what are the options for interventions and the expected relative merits of different options, as well as openness in dealing with uncertainty [19].Following the principles of EBM under pressure of time will usually reveal a higher level of uncertainty about the conclusions and recommendations than medium-or long-term risk assessments.We are aware that it is difficult, especially for public health agencies, to translate scientific uncertainty into policy advice [20].Stakeholders expect certainty and clear answers.However, we also believe that scientific uncertainty should be included in the assessment and the decision-making process as information, not ignored [21].
The working group considered the added value to communicable disease incident control of integrating principles from the discipline of risk analysis, as embodied, for example, in the Codex Alimentarius [22].If we consider the Public health decision making process as a predictive model, uncertainties can arise both from the potential errors associated with the structure of the model (such as the context of the outbreak, modes of transmission and potential control measures for new infections) and from uncertainties in the values of the model parameters (incomplete data or measurement errors) [23].These uncertainties are an integral part of scientific judgment and should be reflected in communication with policy makers and the public.

Tools required:
• Guidance on developing advice, including assessment of the quality of evidence; • Uncertainty tables addressing uncertainties arising directly from the data and from the model/ process used to capture and interpret the data.

Stage 5: Implementation
For effective implementation, advice must be framed by requirements of the target groups.Public perception and communication of risk must therefore be considered.Various governments and international organisations have published guidelines on risk communication which embrace the need for consistent, credible and high-quality information to be shared with the public [24,25].In acute scenarios, the rapidly changing picture and accumulation of intelligence needs to be explained, and caveats about interim advice clearly admitted.

Tools required:
• A checklist of key points to address in risk communication.

Stage 6: Monitoring and evaluation
The last stage is monitoring the implementation of control measures.It is increasingly recognised by public health agencies that they should have in place systems for learning lessons from incidents and continuously improving performance [26].Therefore, incidents should be reviewed systematically to identify the lessons for better management of future incidents, and to identify new knowledge about the causative agent and the risks to the population.This would be aided by the use of standardised audit tools [27] and protocols [28] that should be followed to give a rapid but systematic approach to identifying lessons within a framework of organisational accountability.

Tools required:
• Protocols for review and audit of lessons to be learned from of incidents.

Discussion
The validity, credibility and success of public policy and risk management of public health threats are increasingly being seen as dependent upon the use of the best available scientific evidence developed through a transparent and open process [1].To this end, a working group set up by ECDC has assessed the potential value of a more widespread use of strategies from evidence-based medicine in communicable disease control.
The EBM movement started as an application of epidemiological and public health principles in clinical practice; the application to public health threats is a more recent trend [28].We recognise that there are important distinctions between evidence-based strategies applied to the review and appraisal in clinical medicine and the reality of public health policy making and communicable disease control, not least the lack of a strong evidence base and the pressure of time.In the sister discipline of risk analysis it is also increasingly being recognised that public health decision making is generally a result of a more complex interaction of the best available evidence from research and other epidemiological sources, with judgements made on needs, resources, local circumstances, and ethical, legal and societal implications [29].
We see considerable merit in an integrative approach bringing risk analysis methods together with the epidemiological principles of EBM.For example, the EU Scientific Committee for New and Emerging Health Risks uses the expression 'lines of evidence' to characterise different sources and levels of evidence and information [29].They consider lines of evidence that lie at the bottom of the EBM hierarchy.The highest levels of evidence from systematic reviews of randomised trials are seldom available in acute communicable disease incidents and advice has to be derived from observational studies underpinned by microbiological and virological principles.Sometimes advice has to be based on analogy and modelling, using laboratory research, animal experiments and mathematical modelling of outbreak data.When empirical data in an outbreak emerge, they first appear in expert committee papers and conference presentations, well before peer-reviewed publication, making it difficult to identify that knowledge systematically and quickly.But as with higher-level forms of evidence, the quality of such studies, their collation and interpretation should be guided by EBM methods.This demands the application of rigorous, standardised and systematic ways of handling evidence so that the risk of bias is minimised and assumptions are made explicit.
The application of risk analysis methods is particularly important when dealing with the uncertainties implicit in rapid decision making.It is important to acknowledge that the level of confidence in the conclusions reached is typically inversely related to the time that has passed since the start of the event (Figure 2).
The confidence level which can be achieved for shortterm risk assessments is largely dependent upon the preparatory work done."Constraints, uncertainties and assumptions having an impact on the risk assessment should be explicitly considered at each step in the risk assessment and documented in a transparent manner.Expressions of uncertainty or variability in risk estimates may be qualitative or quantitative, but should be quantified to the extent that is scientifically achievable" [22].The applicability and relevance of standard EBM methods increases with time as the outbreak investigations proceed, but at any particular time there is also the necessity to consider the application of the Areas of knowledge/evidence precautionary principle, and to be clear that lack of evidence of harm is not interpreted as evidence for no harm [30].The principles of EBM, working rigorously, systematically and transparently and according to best available evidence, should apply at all times.

Next steps
In order to improve the management of outbreaks of communicable disease across Europe, the working group developed a conceptual framework and a potential set of tools and checklists that need to be developed to deal with the twin pressures of timeliness of risk assessment and lack of evidence.We hypothesise that these tools would improve outbreak management and thereby reduce the human and resource costs of outbreaks.They would also provide a clear auditable trail of decision making that would allow continuous learning from outbreaks.We envisage that the tools described above, collected together with worked examples in the format of a work book, could provide a uniform, consistent methodology for health protection practitioners.The international health protection community should work together to take this agenda forward and in particular identify leadership and responsibilities for developing the tools and for setting up and managing the archives and databases identified as a necessary part of EBM applied to outbreak control.Led by the Robert Koch Institute and based on a tender from ECDC, a multidisciplinary team has started to develop and pilot a systematic, transparent and comprehensive evidence assessment framework for rating the evidence and strength of recommendations in the area of infectious disease prevention and control.

Figure 1
Figure 1Evidence cycle in outbreak recognition, investigation, control and review

Figure 2
Figure 2Conceptual model of the relationship between uncertainty and time in risk assessments

Table
Conceptual stages in rapid risk assessment and proposed evidence-based medicine tools