On 6 June 2017, the World Health Organization (WHO) published updates to its ‘Essential Medicines List’ (EML). Read more here.

Eurosurveillance is on the updated list of the Directory of Open Access Journals and in the SHERPA/RoMEO database. Read more here.

Follow Eurosurveillance on Twitter: @Eurosurveillanc

In this issue

Home Eurosurveillance Edition  2011: Volume 16/ Issue 27 Article 4
Back to Table of Contents
Previous Download (pdf)

Eurosurveillance, Volume 16, Issue 27, 07 July 2011
Letter to the editor: Prioritisation of infectious diseases in public health: feedback on the prioritisation methodology, 15 July 2008 to 15 January 2009
  1. Bechtel International Systems Inc. Tashkent, Uzbekistan
  2. Department of Management, London School of Economics, London, United Kingdom
  3. Warwick Business School, University of Warwick, Coventry, United Kingdom

Citation style for this article: Del Rio Vilas VJ, Montibeller G, Franco LA. Letter to the editor: Prioritisation of infectious diseases in public health: feedback on the prioritisation methodology, 15 July 2008 to 15 January 2009. Euro Surveill. 2011;16(27):pii=19911. Available online:
Date of submission: 29 June 2011

To the editor: We read with great interest the results presented by Gilsdorf and Krause (2011) [1] of a survey to experts on the methodology used by Krause et al. (2008) [2] to prioritise 85 pathogens of public health importance. Their work deals with a very relevant subject, given current pressure on health budgets: the allocation of finite disease surveillance and control resources among competing alternatives, infectious diseases in this case. The authors correctly identify the evaluation as being multi-dimensional and compensatory. Unfortunately, they appear to have overlooked findings and principles of well-established methodologies for assessing the impact of multiple effects on non-tradable goods, such as multi-criteria decision analysis (MCDA) [5]. It is within this perspective that we make our comments.

First, we noted that some of the criteria considered by the authors do not exhibit certain essential properties of evaluation criteria, particularly with regards to preferential independence. For example, it seems difficult to assess the “treatability” of a pathogen without considering simultaneously the “evidence for pathogenesis”. We observed that lack of preferential independence may also exist between other criteria within the groups “Information needed” and “Health gain opportunity”. When criteria are not preferentially independent, the use of linear additive models of weighted pathogen-specific scores, as in Krause et al. (2008), should not be used, as the overall impact cannot be assessed by simply adding up partial impacts. The criteria set should be redefined to make sure the essential properties hold and a simple weighted sum may be employed [4].

Second, the score’s scale, -1, 0 or 1, lacks granularity and discretises continuous variables unnecessarily. Under the current model, a disease with an incidence of 20/100,000 would score “0” whereas as a disease with incidence of 20.1/100,000 would score ”1”. As suggested by some of the respondents to the survey, a continuous score is better suited. This issue is easily dealt with in MCDA, with the assessment of value functions, which map out and normalise different levels of impact into 0-100 scales.  

The score “0” holds a double label: “average importance” and “lack of knowledge”. The ambiguity of criteria labels is something to avoid in all prioritisation exercises. In this particular application, two diseases, one well known but “average” and the other suffering from lack of evidence, could score similarly. This would not help in the ranking of diseases and the subsequent distribution of resources that would probably allocate greater relevance to the unknown than to the average known. The survey respondents were rightly concerned about the need to incorporate uncertainty in their assessment against a number of criteria. To this request, the authors argued that the complexity of such addition to their model may outweigh the benefits. We would just like to add that simple approaches to handling uncertainty in decision frameworks similar to this are already available and widely used and that uncertainty about impacts should not increase the ambiguity in assessments.

Third, when allocating the weights to the criteria, the authors failed to recognise that weights are scaling constants, which aggregate partial impacts into overall impacts, and not direct measurements of importance. Indeed, a survey respondent correctly identified the limitations associated with this approach to weighting “that the difference in importance between each criterion is always equal”. This could lead to misleading conclusions. Weights should reflect explicitly public value trade-offs of the group involved in model building and their assessment has to follow careful elicitation procedures to avoid well-known biases [4].

Fourth, a number of survey respondents raised the need for a time frame for some of the criteria. This is a very genuine concern that should be expanded to include all the characteristics that define the context (e.g. geographical location). In our experience, prioritisation of diseases is a valid exercise that allows systematic comparisons to support strategic resource allocation. Like any other general strategy, it will fail to capture all possible presentations and heterogeneities that will surely be present depending on the risk pathways involved. Alternative methods to reactively measure the impact of such variability are required to feed into the regular strategic prioritisation of diseases. MCDA has been successfully used in these contexts and, in our view, provides a robust methodological framework for such evaluations [3]. 

We would like to finish our note with a comment on the composition of the expert group for prioritisation and congratulate the authors for engaging with a wide group of technical experts. If we may, we would like to suggest that the authors consider the incorporation of experts on MCDA to this group. This would follow common practice in other scientific fields such as nuclear waste management and drugs risk-benefit assessments.

  1. Gilsdorf A, Krause G. Prioritisation of infectious diseases in public health: feedback on the prioritisation methodology, 15 July 2008 to 15 January 2009. Euro Surveill. 2011; 16(18):pii=19861. Available from:
  2. Krause G; Working group on prioritisation at the Robert Koch Institute. Prioritisation of infectious disease in public health – call for comments. Euro Surveill. 2008; 13(40):pii=18996. Available from:
  3. Montibeller G, Franco LA, Del Rio Vilas VJ. Prioritising emerging threats with multi-criteria decision analysis. Proceedings of the 21st International Conference on Multi-Criteria Decision Making; 2011 Jun 13-17, Jyvaskyla, Finland. Forthcoming.
  4. Keeney RL. Common mistakes in making value trade-offs. Operations Research 2002;50(6):935-45.
  5. Keeney RL, Raiffa H. Decisions with multiple objectives: preferences and value tradeoffs, 2nd ed. Cambridge: Cambridge University Press; 1993.


Back to Table of Contents
Previous Download (pdf)

The publisher’s policy on data collection and use of cookies.

Disclaimer: The opinions expressed by authors contributing to Eurosurveillance do not necessarily reflect the opinions of the European Centre for Disease Prevention and Control (ECDC) or the editorial team or the institutions with which the authors are affiliated. Neither ECDC nor any person acting on behalf of ECDC is responsible for the use that might be made of the information in this journal. The information provided on the Eurosurveillance site is designed to support, not replace, the relationship that exists between a patient/site visitor and his/her physician. Our website does not host any form of commercial advertisement. Except where otherwise stated, all manuscripts published after 1 January 2016 will be published under the Creative Commons Attribution (CC BY) licence. You are free to share and adapt the material, but you must give appropriate credit, provide a link to the licence, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.

Eurosurveillance [ISSN] - ©2007-2016. All rights reserved

This website is certified by Health On the Net Foundation. Click to verify. This site complies with the HONcode standard for trustworthy health information:
verify here.