Introduction

T he European Influenza Surveillance Scheme (EISS) aims to facilitate collaboration between European countries on influenza surveillance (see article p. 147-57). EISS has developed two protocols to evaluate the quality of clinical and virological data collected by surveillance networks based on sentinel practitionners (sp) reporting in Europe. The rationale for developing these protocols was to: a) move towards standardisation of data collection, b) improve the surveillance of influenza activity and data quality, c) facilitate the sharing of data at a European level, and d) encourage and facilitate access to new member states wanting to join the Scheme (1,2). This paper briefly describes the protocol to evaluate the quality of clinical data collected by SPs and the resulting experiences during the 2001–02 influenza season.

Methods

The EISS evaluation protocol is based on the Centers for Disease Control and Prevention (CDC Atlanta), and the World Health Organization (WHO) guidelines (2,3). We designed and pre-tested in the Netherlands and France two questionnaires to be administered: (a) face to face to the networks’ coordinators at regional and/or national level, and (b) either face to face or by telephone, depending on logistic constraints, to a representative sample of sentinel practitioners -in terms of either or both geographical location, and medical speciality (general practitioners/paediatricians).

The questionnaires addressed the following networks’ attributes, at the coordination and / or the sentinel practitioners’ levels:

• Sensitivity and specificity of the network, including information on the number of SPs and the use of : 1) a case definition for influenza and the adherence of SPs to it, 2) an epidemic threshold to estimate the magnitude of morbidity related to influenza, 3) incidence of ‘influenza-like illness’ (ILI) or ‘acute respiratory infection’ (ARI) as a numerator for measuring influenza activity, 4) a population-based or encounters-based denominator, 5) age groups for the classification of cases, and 6) criteria for collection of clinical specimens;

• Representativeness of sentinel practitioners to all national/regional practices, and of the population covered by sentinel practitioners to the general population;

• Timeliness for reporting weekly data, and ability to detect changes in trends in a timely way;

• Network output including reports, publications, communication, training to sentinel practitioners or staff, and activities leading to improved practice;

• Resources for the network.

We implemented the evaluation protocol during on site visits, at both coordination and sentinel practitioners levels, in the Belgian and three regional Spanish networks (Aragón, Castilla y León, and the Madrid Community). These networks were selected according to their EISS membership, the willingness of their coordinators to participate in the evaluation, logistic constraints, and because the EISS coordination centre wanted to evaluate a national network made up of regional networks (in Spain). The standardised evaluation protocol includes a report format to facilitate the formulation of conclusions and recommendations based on generally accepted surveillance practices, obtained from a literature review, WHO/CDC standards, current consensus reached within EISS, and surveillance practices in Europe (4).

Results

All networks fulfilled either international standards or EISS requirements in terms of : 1) simplicity and acceptability of influenza surveillance activities, 2) types of numerator (ILI in all networks as well as ARI in Belgium), age groups, denominators (population-based), case definitions and criteria for swabbing, 3) completeness of reports (over 95%), 4) use of epidemic thresholds (not used in Aragón), 5) laboratory support, 6) timeliness of reporting data and detecting trends, and 7) resources (see table).

Table. Preliminary results of the evaluation protocol

*Estimée en janvier 2001 / Estimated in January 2001.
Le protocole d’évaluation a été mis en place par les réseaux de surveillance de la grippe en Belgique, et en Espagne (Castilla y León, Aragón et Madrid). Saison de la grippe 2001 – 2002 / The evaluation protocol was implemented in influenza surveillance networks in Belgium, and Spain (Castilla y León, Aragón and Madrid). Influenza season 2001–02.

Each network coordinator has received an evaluation report that is based on a standard format and includes a description of the network’s attributes, conclusions and recommendations. For all networks, the recommendations made to the coordinators included improving sentinel practitioners’ adherence to the national/regional case definitions and criteria for swabbing. Additionally, the following network-specific recommendations were made:

• In Belgium, to increase the number of sentinel practitioners, to improve their geographical representativeness, to organise regular meetings to discuss epidemiological and virological issues of the previous season, and to harmonise the methods used to establish the epidemic thresholds for ARI and ILI;

• In Spain, to increase the number of regional networks collecting and reporting clinical and virological data to the national level (currently eight out of 17 Spanish regions), and to improve the standardisation of methods used to establish epidemic thresholds in regional networks. At a regional level, to reduce the delay of reporting swab results to sentinel practitioners.

Conclusion

The EISS group has developed an evaluation protocol to assess the quality of clinical data collected by sentinel practitioners within EISS. This protocol allows an in-depth description of the surveillance network, and the sharing of national/regional experiences with other networks in Europe and elsewhere. It should facilitate the improvement and harmonisation of influenza surveillance activities in Europe, and be a valuable reference for new networks wanting to join EISS.

Acknowledgements

The Health and Consumer Protection Directorate-General of the European Commission funded this evaluation project. We would like to thank the following persons for testing and implementing the evaluation protocol:

Aad Bartelds (Netherlands Institute for Health Services Research, NIVEL, Utrecht): Anne Mosnier (France GROG, Paris); Fernande Yane and Frederic Pierquin (Institut Scientifique de Santé Publique, Brussels); Tomas Vega and Raul Ortiz de Lejarazu (Consejería de Sanidad y Bienestar Social y Centro de Gripe de Valladolid, Castilla y León); Maria Ordobas and Silvia Fernandes (Consejería de Sanidad, Comunidad de Madrid), Alberto Larrosa and Marta Fajó (Servicio Aragonés de Salud, Zaragoza); Manuel Omeñaca Teres (Laboratorio de Microbiología, Hospital Universitario Miguel Servet, Zaragoza); Pilar Perez (Laboratorio de Referencia de Virología, Centro Nacional de Microbiología, Instituto Carlos III); Salvador de Mateo Ontañón (Servicio de Vigilancia Epidemiológica, Centro Nacional de Epidemiología, Madrid); and all sentinel practitioners who participated in the evaluation in Aragón, Belgium, Castilla y León, France, Madrid Community, and the Netherlands.

A copy of the evaluation protocol can be obtained by contacting Jean-Francois Aguilera (jaguilera@phls.org.uk). It will be posted on the EISS website (www.eiss.org) by the end of 2002.