PRECEPT: an evidence assessment framework for infectious disease epidemiology, prevention and control

Decisions in public health should be based on the best available evidence, reviewed and appraised using a rigorous and transparent methodology. The Project on a Framework for Rating Evidence in Public Health (PRECEPT) defined a methodology for evaluating and grading evidence in infectious disease epidemiology, prevention and control that takes different domains and question types into consideration. The methodology rates evidence in four domains: disease burden, risk factors, diagnostics and intervention. The framework guiding it has four steps going from overarching questions to an evidence statement. In step 1, approaches for identifying relevant key areas and developing specific questions to guide systematic evidence searches are described. In step 2, methodological guidance for conducting systematic reviews is provided; 15 study quality appraisal tools are proposed and an algorithm is given for matching a given study design with a tool. In step 3, a standardised evidence-grading scheme using the Grading of Recommendations Assessment, Development and Evaluation Working Group (GRADE) methodology is provided, whereby findings are documented in evidence profiles. Step 4 consists of preparing a narrative evidence summary. Users of this framework should be able to evaluate and grade scientific evidence from the four domains in a transparent and reproducible way.

Decisions in public health should be based on the best available evidence, reviewed and appraised using a rigorous and transparent methodology. The Project on a Framework for Rating Evidence in Public Health (PRECEPT) defined a methodology for evaluating and grading evidence in infectious disease epidemiology, prevention and control that takes different domains and question types into consideration. The methodology rates evidence in four domains: disease burden, risk factors, diagnostics and intervention. The framework guiding it has four steps going from overarching questions to an evidence statement. In step 1, approaches for identifying relevant key areas and developing specific questions to guide systematic evidence searches are described. In step 2, methodological guidance for conducting systematic reviews is provided; 15 study quality appraisal tools are proposed and an algorithm is given for matching a given study design with a tool. In step 3, a standardised evidence-grading scheme using the Grading of Recommendations Assessment, Development and Evaluation Working Group (GRADE) methodology is provided, whereby findings are documented in evidence profiles.
Step 4 consists of preparing a narrative evidence summary. Users of this framework should be able to evaluate and grade scientific evidence from the four domains in a transparent and reproducible way.

Introduction
The potential fallacies of relying solely on expert opinion to establish best practice in clinical decision-making and public health policies are well known globally [1]. In guideline development, it is standard practice to draw on systematic reviews of the available evidence. For evidence of benefits and harms, well conducted randomised controlled trials (RCTs) minimise bias and systematic reviews of these are commonly used in decision-making. However, observational studies are important for answering public health questions, not least because in many cases they are the only available or feasible source of empirical evidence [2].
Judging the effectiveness of infectious disease prevention and control interventions creates challenges related to the population-level effects and long-term aspects of the intervention. In addition, a variety of other elements need to be considered in decision-making, including disease burden, risk factors for infection or mode of transmission. In adopting the concept of 'best available evidence' [3], evaluating the benefits and harms along the full causal chain from intervention to outcomes within a given context requires a variety of fit-for-purpose methods from multiple disciplines.
Although considerable progress has been made regarding the use of systematic reviews for public health decision-making, the approaches currently used for conducting and appraising systematic reviews in public health have their limits. They regularly report effect estimates and risk of bias, but often do not assess the certainty of the evidence for the entire body of such across outcomes. Moreover, traditional approaches mainly focus on intervention effectiveness and safety, but do not provide a generalised approach that addresses all factors relevant to decision-making in infectious disease prevention and control, such as the epidemiology, interventions as well as diagnostics and risk factors.
New evidence appraisal and grading system approaches that incorporate information from studies with different designs have been developed. The most prominent system, developed by the Grading of Recommendations Assessment, Development and Evaluation Working Group (GRADE) [4], has been widely applied in clinical medicine as well as public health. A working group established by the European Centre for Disease Prevention and Control (ECDC) examined the application of GRADE to infectious disease prevention and control [5] and initiated the Project on a Framework for Rating Evidence in Public Health (PRECEPT). The PRECEPT consortium was established in 2012 with representatives from European public health agencies, academic institutions and ECDC. The first results of this project were published earlier [6,7].
An approach to infectious disease prevention and control that adheres to the principles of evidencebased public health, using a defined framework for the assessment of the certainty in the evidence, has a number of advantages over conventional approaches. In particular, such an approach: • Helps to improve the quality of the resulting public health recommendations.
• Reduces anticipated or actual arbitrary decisions.
• Builds trust and supports the acceptance of recommendations by professionals and the public.
• Helps to compare recommendations endorsed by different countries or institutions. Step 1 Step 2 Step 3 Step 4

Figure 2
Overview of the systematic review process Appraisal of methodological quality Step 2.2 Step 2.1 Step 2.3 Step 2.4 Step 2.5 Data synthesis Step 2.6

Figure 3
PRECEPT algorithm for identifying quality appraisal tools according to study design   The approach presented here applies the most advanced methodology for assessing certainty of the evidence, the GRADE methodology. Even though the individual elements (e.g. the GRADE methodology, PICO question framing and systematic review) of the proposed approach are not new, they have never been integrated into one comprehensive framework that guides users from identification of the relevant research questions to preparation of a final evidence assessment summary for the area of infectious disease prevention and control. Moreover, in contrast to other approaches, the framework puts particular emphasis on question framing and the selection of appropriate risk of bias tools. The framework provides evidence assessment guidance in infectious disease epidemiology, prevention and control, but is not designed to conduct a rapid assessment for the purpose of answering urgent questions in public health crises or emergencies.
PRECEPT intends to provide methodological guidance for public health agencies, scientists working in the field of evidence-based public health, and other institutions and individuals involved in appraising evidence and developing public health guidance with a focus on infectious diseases. Here we present an overview of the PRECEPT approach (see also Supplementary Material [8]).

Domains used in the PRECEPT framework
The PRECEPT framework focuses on the following four domains: i. Disease burden (significance of the problem), which encompasses studies on the incidence, prevalence and severity of diseases and complications, as well as studies on the perception of diseases in target populations. For example, what is the incidence of hepatitis B in sex workers in eastern European countries?
ii. Infection and disease risk factors (causes of the problem), which encompasses studies on preventable and non-preventable risk factors for infection, disease and complications. For example, is sepsis acquired in the neonatal intensive care unit a risk factor for cerebral palsy?   iii. Diagnostics (detection of the problem), which encompasses studies on diagnostic accuracy (sensitivity and/ or specificity of diagnostic tests/measures). For example, what is the sensitivity and specificity of tests for tuberculosis in children?
iv. Interventions (consequences of action against the problem), which encompasses studies examining efficacy, effectiveness and adverse effects. For example, what is the effectiveness of vaccination of infants against rotavirus for the prevention of hospitalisation?

Four-steps to assessing evidence
The general approach to this process of assessing evidence within these four domains consists of four steps (Figure 1).

Step 1: Identify the relevant questions
PRECEPT proposes applying the extended Patient/ Population, Intervention, Comparator/Comparison/ Control, and Outcome (PICO) format and integrating other questions to the extent possible.
PICO is considered to be the most appropriate way of framing questions related to interventions (domain (iv)) [2,9]. However, for studies on disease burden (domain (i)), it can be changed to focus on population, condition (i.e. disease) and context [10]. For risk factors (domain (ii)), the PICO format can be easily modified by replacing 'intervention' with 'exposure' and 'comparator' with 'absence of risk factor'. Furthermore, different risk factors can be compared with each other (risk factor x vs risk factor y) and different risk factor levels to establish an exposure-response relationship. For diagnostic accuracy studies (domain (iii)), PICO can be interpreted as population, index test, comparator test and outcome (target condition) [11]. Examples of PICO questions for all domains are shown in Table 1.
In infectious disease prevention and control, researchers are regularly confronted with complex of questions that have to be addressed in combination. For example, the question, 'Should vaccination against rotavirus be recommended for all infants?' comprises multiple questions from different fields that need to be addressed e.g.: • What is the incidence of rotavirus infection among children < 5 years of age?
• Is age a risk factor for rotavirus infection among children < 5 years of age?
• What is the effectiveness of vaccination against rotavirus?
• What is the risk of intussusception associated with the vaccine?
In such situations, developing a logic model (conceptual diagram) tends to be helpful [12] to identify and prioritise all relevant questions, and to place these in context. In systematic reviews, a logic model is a graphical representation that helps in scoping the review, defining and conducting the review, and making results from the review relevant to policy and practice [13].

Step 2: Perform the systematic review
Evidence should be identified and synthesised using a rigorous systematic review process. A systematic review usually includes six steps (Step 2.1. to 2.6. as parts of step 2 of the PRECEPT workflow, see Figure 2). For more extensive information on conducting a systematic review, readers are referred to the literature [9,14].

Assessment of methodological quality (risk of bias)
For appraising the methodological quality (risk of bias) of each study identified during the systematic review, the PRECEPT framework proposes using specific quality appraisal tools (QATs) according to study design [6]. A selection of 15 QATs, identified and selected during the first phase of the project by applying a systematic review-based approach [6], are proposed ( Figure  3, Table 2). For each study design, the algorithm leads to the identification of a single QAT or a group of QATs. Risk of bias should be assessed in the form of a judgment rather than a score. It is suggested that the Cochrane classification scheme for bias is used: (i) high risk of bias; (ii) low risk of bias, and (iii) unclear risk of bias [14].

Synthesis of data
In the case of quantitative data, data synthesis can be conducted using meta-analytic techniques. If statistical pooling appears to be inappropriate, e.g. if interventions are too heterogeneous to be grouped in a meaningful way, if data are highly heterogeneous or if study designs differ considerably, a tabular, graphical or narrative synthesis might be more useful [15].

Use of existing systematic reviews
It is estimated that a new full systematic review takes between six to 24 months, but using existing systematic reviews for the development of new evidence assessments can shorten this by one to two thirds [16]. The use of existing systematic reviews might therefore reduce efforts and costs, making the evidence assessment process more efficient. Existing systematic reviews might complement the PRECEPT framework in various ways, e.g. by identifying studies, by answering full questions or by providing search strategies. Before using an existing systematic review, the need for an update should be evaluated. The process of identifying, assessing and applying existing systematic reviews should follow the steps proposed by Robinson et al. [17], and tools such as AMSTAR [18] or ROBIS [19] should be used to assess the methodological quality of existing systematic reviews.
Step 3: Apply the evidence-grading system and document the results PRECEPT recommends a standard approach that uses the GRADE methodology to evidence-grading across all four types of domains.

Quantitative evidence
According to GRADE, the certainty in the evidence indicates the extent to which one can be confident that the estimate of effect is correct [4]. The units of analysis of GRADE are outcomes, meaning all assessments focus on the outcome of the intervention. At the beginning of the evidence review process, each outcome is rated from 1 to 9 regarding its importance to the decision, where outcomes rated 7 to 9 are regarded as 'critical', 4 to 6 as 'important', and 1 to 3 as 'of less importance'.  (Figure 4), which is consistent with the current GRADE approach [4,[20][21][22]: i. For each body of evidence related to an outcome, an initial rating of the certainty in the evidence is performed. For some of the domains, this initial rating depends on study design.
ii. Risk of bias is assessed using the appropriate QAT for the individual studies (see Step 2). A judgment about the risk of bias is made for the body of evidence, and evidence certainty can be downgraded, if necessary.
iii. Thereafter, the other GRADE criteria for downgrading the certainty in the evidence (inconsistency, indirectness, imprecision, publication bias) are applied.
iv. For the domains of 'intervention studies' and 'risk factor studies', upgrading of the certainty in the evidence is possible, according to the criteria introduced by GRADE. Evidence certainty should usually not be up graded after having been downgraded. It is currently unclear whether and how upgrading criteria are applicable to bodies of evidence on prevalence and diagnostics.

Qualitative evidence
For rating the evidence certainty of qualitative studies, PRECEPT proposes to assess limitations in the individual studies using the appropriate QATs. In addition, users of PRECEPT might consider the GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative research) approach for assessing the confidence of evidence from reviews of qualitative research [23].

Step 4: Prepare an evidence summary
At the end of the evidence appraisal process, a narrative evidence summary for communication of the results should be prepared. The following points should be captured: (i) the overall question, which describes the key question of the review, (ii) detailed questions, which lists the questions addressed by the review, and (iii) the volume of evidence, which describes the studies identified during the review, and (iv) the evidence statement and grading, which summarises the evidence which was identified by the review and the results of the grading process.

Conclusion
Over the past three years, the PRECEPT team has developed this framework for the assessment of evidence in the field of infectious disease epidemiology, prevention and control. Currently, the application of the framework is being tested in other systematic reviews and projects [16,24]. Furthermore, a consultation process will be carried out to obtain feedback and collect suggestions for improvement. This process might lead to further refinements and adaptations of the framework.