Evaluation of the surveillance system for undiagnosed serious infectious illness ( USII ) in intensive care units , England , 2011 to 2013

G Dabrera (zoonoses@phe.gov.uk)1,2,3, B Said3, H Kirkbride3, On behalf of the USII Collaborating Group4 1. UK Field Epidemiology Training Programme, Public Health England, London, United Kingdom 2. European Programme for Intervention Epidemiology Training (EPIET), European Centre for Disease Prevention and Control (ECDC), Stockholm, Sweden 3. Centre for Infectious Disease Surveillance and Control, Public Health England, London, United Kingdom 4. Members of the group are listed at the end of the article


Introduction
Mass gatherings have been described as 'a stress test for public health' by the Director General of World Health Organization (WHO) [1]; one of the challenges to public health is the potential for introduction of emerging infectious diseases due to the international movement of large numbers of people [2].
Emerging infections are a particular concern, as they can place a significant burden on public health and acute medical services within short time periods.A recent example is the Middle East respiratory syndrome coronavirus (MERS-CoV), which emerged in 2012 in the Arabian Peninsula [3].The consecutive importation of MERS-CoV cases to the United Kingdom (UK) [4,5], Germany [6], France [7], Italy [8], Greece [9], the Netherlands [10], United States (US) [11] and Malaysia [12] required considerable public health resources with wide public health follow-up of contacts, extensive virological testing and international risk assessments.Clinicians are accustomed to recognising and reporting specific diagnoses to public health surveillance systems.However, identification and reporting of emerging infections is problematic as these infections may not fit a recognisable clinical presentation and routine laboratory tests will not positively diagnose such cases.They are therefore less likely to be captured by traditional public health surveillance systems, instead requiring novel surveillance systems that aim to detect such cases of undiagnosed serious infectious illness (USII).The most severe emerging infections are likely to present to clinicians as USII in an acute medical setting such as intensive care units (ICUs).The ability to detect clusters of USII, related by common exposures, demographic or clinical characteristics could help to identify the first few cases of an emerging infection.This is especially important in complex health systems, where individual cases may be admitted to different hospitals.
To address these difficulties the Health Protection Agency (HPA, now part of Public Health England) developed a new surveillance system to detect cases and clusters of USII, as part of public health planning for the London 2012 Olympic and Paralympic Games between 27 July and 9 September [13,14].The structure of this USII surveillance system is based on a sentinel network of ICUs and has been described in detail previously [15].A sentinel structure was chosen to develop a proactive network of ICUs which reported regularly and provided good coverage in the areas of interest for a limited period related to the London 2012 Games.It was considered that this would have the potential to detect a proportion of any related cases of an emerging infection.These ICUs were chosen to provide good coverage in the areas of interest for a limited period related to the London 2012 Games.More comprehensive coverage would be necessary to develop an ongoing USII surveillance system for the whole country.The system was initially piloted in six ICUs for a period of six months starting from January 2011.It was then progressively expanded between July 2011 and February 2012, to a total of 19 units (including 12 adult units and 7 paediatric units), as part of the preparedness for the London 2012 Games [15].All 19 units were enrolled by 27 February 2012.The surveillance system was operational for 18 months from 11 July 2011.
Immediately after the 18 months of operation and until March 2013, the USII surveillance system was reviewed and evaluated, using the Centers for Disease Control and Prevention (CDC) guidelines for the evaluation of surveillance systems [16].The purpose of this report is to describe the cases reported to the system and to assess this surveillance system for future mass gatherings, in terms of its sensitivity, acceptability and simplicity to participating clinicians, positive predictive value (PPV), and timeliness of case reporting.

Description of undiagnosed serious infectious illness surveillance
The USII surveillance system operated in 19 adult and paediatric ICUs (PICUs).These units were approached to participate on the basis of their proximity to games venues in the London region (13 units, of which eight were adult and five were paediatric units), or their role as major intensive care centres in the surrounding areas (4 in South East and 2 in East of England).These units participated on a voluntary basis and represented 48% and 59% of London ICU and PICU beds, respectively.The hospital represented by the units comprised a mix of large teaching hospitals and local acute hospitals, each with their own internal medical microbiology service and their own standard range of investigations.The majority of London 2012 Olympic and Paralympic Games activities were located in London and the South East regions.The USII case definition is shown in the Box.The USII diagnosis was made by clinicians in participating units on the basis of clinical opinion and hospital microbiology results; the precise microbiology tests used for cases varied between units as each participating hospital had its own specialist microbiology service which operated independently.One or more lead clinicians from each participating ICU reported cases primarily through a dedicated online reporting tool but cases could also be notified by email or telephone.Information was collected on patient demographics, clinical presentation, travel history (including travel within the UK and abroad) and other relevant exposures.If no cases were identified, clinicians were asked to provide a nil report every two weeks (or weekly during the London 2012 Games period).Participating clinicians were able to update information for reported cases, such as new alternative diagnoses, via the web-based tool, email or telephone.

Evaluation of undiagnosed serious infectious illness surveillance
The evaluation involved a number of different approaches:

Retrospective analysis of cases and sensitivity of surveillance system
Data on cases reported during the 18 month period between 11 July 2011 and 10 January 2013, inclusive, were extracted from the secure web-based tool and added to a password-protected excel spreadsheet (Microsoft Excel 2007, Microsoft, Redmond, WA) which also contained data on cases reported by telephone and email.The cases which remained undiagnosed at the time of the evaluation (i.e.USII cases) were identified.The status of these cases was reviewed by the surveillance team in conjunction with the reporting clinicians, following the initial report and also during the evaluation, to ensure that they fulfilled the USII case definition.Cases were reviewed and described to provide an understanding of the role and function of the surveillance system.
The population coverage for the system was calculated as previously described [15] based on the proportion of all ICU beds in the local geographical area (as provided by each participating unit) and assuming a binomial distribution.This was used as the denominator, and the number of USII cases as the numerator, to calculate an annualised rate of USII assuming a Poisson distribution.

Box
Case definition, surveillance system for undiagnosed serious infectious illness, England, 2011-2013 Cases were defined as any child (aged ≤ 16 years) or adult admitted to an intensive care unit with a serious illness suggestive of an infectious process, where the clinical presentation did not fit with any recognisable clinical picture or there was no improvement in response to standard therapy and initial laboratory investigations for infectious agents were negative or did not establish a diagnosis.
A 95% Bonferroni-type confidence interval (CI) was calculated to reflect the variability in the population covered over time.This was produced by calculating CIs using the lower and upper limits of the annualised rate of USII; the lowest of the lower limits and the highest of the upper limits, formed the lower and upper limits of the 95% Bonferroni-type CI, respectively.These annualised rates were calculated overall for all ages and separately for adult and for paediatric (aged less than or equal to 16 years) cases and compared to the published rate from the pilot period [15], to assess the sensitivity of the surveillance system.

Acceptability and simplicity
This was assessed during site visits and meetings with clinicians at each participating trust by the authors (GD, BS, HK).Clinicians were asked to complete a short paper-based questionnaire using a five point Likert scale during these visits.The questionnaire asked about their understanding of the role of the USII surveillance system, the acceptability of sending a fortnightly nil report and their willingness to continue reporting USII cases in the future.To assess simplicity, participants were specifically asked how easy the case definition and the web-tool were to use.Completed paper questionnaires were double-entered using Epidata Entry v3.This was defined as the proportion of all cases reported to the surveillance system, which remained USII (i.e. which were not subsequently diagnosed) at the time of last report.Counts of cases notified to the USII surveillance system and of those which were subsequently diagnosed at the date of last report or outcome notification, were identified from the previously described password-protected excel spreadsheet holding case data.

Timeliness
The reporting time was defined as the number of days elapsed between a case being admitted to a participating unit and reported to the USII surveillance system.The following data fields were extracted from the online reporting tool for each case: date of admission and date of reporting for cases notified using the reporting tool between 11 July 2011 and 10 January 2013, inclusive.The median and range of reporting times was calculated for these cases.Timeliness was similarly calculated for those cases reported by email or telephone which had a recorded date of admission in surveillance records.

Retrospective analysis of cases and sensitivity of the surveillance system
During the evaluation period, 34 cases were reported to USII surveillance by participating units (Table 1).Of these, 27 were notified online, six by email and one by telephone.A total of 12 cases were subsequently diagnosed and were no longer classified as USII cases, leaving 22 cases that remained USII (20 reported online and two by email) (Table 1 and 2).At the time of the evaluation, none of the 22 cases that remained USII were hospitalised, as 12 had been discharged and the remainder had died.Of the 22 USII cases, 11 were male and 11 were female with ages ranging between four and 69 years.The principal presenting syndrome was respiratory illness (seven cases), followed by presumed sepsis/bacteraemia (five cases), neurological (four cases) and cardiac (three cases).Nineteen cases were adults and three were paediatric cases (less than 16 years-old).Ten of the adult cases had a history of travel outside the UK in the preceding six months.None of the paediatric cases had a travel history.There were no relevant exposures identified for four cases and no exposure information was given for the remaining five cases.Ten of the USII cases were fatal, giving a case fatality rate of 10/22 (45%).
There was no clustering of USII cases by clinical presentation, exposure or demographic characteristics identified during the evaluation period, including the period of the London 2012 Olympic and Paralympic Games (Figure ).The 22 USII cases detected had no association with the Olympic or Paralympic Games.
Of the subsequently denotified cases most were female (9 cases), the age range was 11 to 77 years, and the main presenting syndrome was presumed sepsis/ bacteraemia (5 cases).The infectious diagnoses for 10 cases or reasons for exclusion (one immunocompromised and one non-infectious and therefore outside the case definition) of these cases are shown in Table 2.The median time from ICU admission to denotification was 10 days (range: 8-41 days).
The USII rate reported in this study was lower than that found during the initial USII pilot (an estimated annual rate of 1.2 per 100,000 persons, with a range of 0.4-3.1 per 100,000 [15]).

Acceptability and simplicity
Twenty-two participating clinicians completed questionnaires during site visits by the surveillance team.
The majority (n=20) of respondents either agreed (n=8) or strongly agreed (n=12) that they understood the role of the USII surveillance system in identifying unknown infective syndromes.In addition 17 respondents agreed (n=10) or strongly agreed (n=7) that the request for a nil report every two weeks was convenient.Furthermore, 19 respondents stated that they would be prepared to continue reporting to the USII system.
Most respondents either strongly agreed (n=8) or agreed (n=11) with the statements that the USII case definition was easy to use and apply to cases.
Approximately half of the respondents reported that

Figure
Distribution of undiagnosed serious infectious illness cases, England, 11 July 2011-10 January 2013 (n=22) Week 32 in 2011 starts on 8 August.On the X axis label, the period corresponding to the Olympic and Paralympic Games is surrounded by a square box.
a Starting from January 2011, in a period lasting six months, which does not figure on the graph, a pilot sentinel surveillance of undiagnosed serious infectious illness cases, including six intensive care units (ICUs) was developed.From July 2011, the number of ICUs in the surveillance was expanded, reaching 19 units in February 2012.
the reporting tool was easy to use (4 strongly agreed and 8 agreed).

Positive predictive value
Of the 34 cases reported to the USII surveillance system, 22 remained USII.The positive predictive value for a case reported to the surveillance system remaining a USII case was 65%.

Timeliness
Data were available to calculate median reporting time for 27 cases reported via the online reporting tool.The median reporting time for USII cases was 11 days following admission to ICU (range: 3-52 days).The median reporting time for three cases reported by email/telephone was 16 days (range: 2-49 days).

Discussion
The USII system is a unique surveillance system developed as part of a range of new and enhanced surveillance systems for the London 2012 Games.To our knowledge, there have been only two similar surveillance systems [17,18] described previously in the literature although these were not specifically developed for mass gatherings.
During 18 months of full operation a total of 22 cases of USII were identified through 19 participating ICUs.This is equivalent to a rate of 0.39 cases per 100,000 persons (95% CI: 0.23-0.64)and along with the absence of any clusters, indicates that as expected, USII cases are rare.However, these cases were associated with a case fatality of 45% which was considered to be high by many of the participating clinicians.None of the cases were associated with the London 2012 Games suggesting that observed cases represent the background level of USII in the area covered by the surveillance.
One of the difficulties in undertaking surveillance for USII is that the diagnosis is based on exclusion of known infections, and therefore depends on the extent of laboratory investigation.This may vary between different clinicians depending on local protocols and clinical experience.It is, in addition, difficult to distinguish between an unknown serious infectious illness and an unknown serious illness (which may not be infectious).
The lower USII rate reported in this study compared to a previous USII pilot was accompanied by an increase in the PPV between the evaluation (65%) and pilot periods (50%).We hypothesise that the pilot provided a period of initial learning, where clinicians were becoming accustomed to the case definition.Subsequently, the experience from the pilot period may have led to a higher threshold for reporting cases and therefore lower USII rates, during the evaluation period.
The results of the evaluation show that the USII system was acceptable to clinicians and that the system was simple to use both in terms of applying the case definition and reporting via the web-based tool.The simplicity of a reporting procedure has previously been reported as a key factor for increasing participants' willingness to report cases of infectious diseases [19].These factors, in combination with the rarity of cases, may explain the willingness of clinicians to continue reporting.
The USII surveillance was an integral part of the surveillance initiatives introduced during the 2012 Olympics and Paralympics to monitor the potential threat of emerging infections.For instance, the HPA and European Centre for Disease Prevention and Control (ECDC) worked jointly to identify and risk assess infectious disease hazards occurring outside the UK which may have had an impact on the Olympic and Paralympic Games [14,20].In addition, syndromic surveillance was expanded to detect signals of uncommon illnesses [13,21].However, the USII system was unique in having the capability to collect case-level information on potential cases of new and emerging infections.
The length of the reporting times identified by this evaluation can be explained by the need to investigate suspected cases following admission, before considering a USII diagnosis.It is difficult to make an informed decision about whether these reporting times are adequate or too short, in relation to characteristics such as incubation period, as these are by definition, unknown for USII.However, this does reinforce the advantages for timeliness of using electronic reporting systems as demonstrated by other authors [22].
Data from similar surveillance have only been published from systems in Taiwan (2000-2005) and the US (1995)(1996)(1997)(1998).These show similar proportions in the presentations of reported cases.As in this study, where the majority (7/22) of USII cases had respiratory illness, surveillance from Taiwan also found that respiratory syndromes were most common, accounting for 59% of cases [17].Respiratory syndromes were also the second most common presentation for the US system, accounting for 26% of cases, although this was closely preceded by neurological presentations (29%) [18].The latter were less common among our cases.
The USII surveillance approach may therefore be useful in addressing new and emerging respiratory infections such as influenza A(H7N9) in China [23,24].
One of the strengths of this evaluation is that we checked reported cases for diagnoses up to the end of the evaluation, maximising the length of follow-up of these cases.This allowed the exclusion of those cases which were diagnosed at later stage, which may be due to results from specialist testing, therefore retaining true USII cases only and making USII case rates more accurate.
As the study was undertaken in an acute health system in England, we anticipate that countries with similar health systems may also benefit from implementing USII surveillance during other mass gatherings.However, the challenge of emerging infections is not limited to just mass gatherings but is a persistent issue occurring in many settings, as evidenced by MERS-CoV [25] and influenza A(H7N9) in China [23,24].We argue that USII surveillance may be useful for such rapidly evolving situations where the capability to detect emerging infections is required.This surveillance is practical to operate, requiring only half a full-time epidemiological scientist and support from a consultant epidemiologist.
The official report into public health activities during the London 2012 Games highlighted the importance of surveillance systems such as USII to public health services during mass gatherings and advocated maintaining the USII system or being able to reactivate it in the future [26].More general, population-wide surveillance would require a remodelling of the system and its processes in terms of increasing the number and distribution of participating ICUs, to ensure sufficient coverage across England and to improve sustainability.We will investigate the feasibility of developing USII beyond its original mass gathering function, in order to address the general, ongoing threat from emerging infections.

Conclusion
Emerging infections pose a constant challenge globally and the USII surveillance model could also be a useful public health tool in other countries, seeking to identify clusters of USII.Prospective surveillance within the ICU setting is simple and acceptable to clinicians and provides a valuable opportunity for the identification of clusters of emerging infections.
2012_52 Number of cases Year, week of symptom onset Paediatric (≤16 years-old) Adult (>16 years-old) Unit recruitment complete a 1 (The Epidata Association, Odense Denmark, 2008) and exported to Microsoft Excel 2007 (Microsoft, Redmond, WA) for analysis.The responses to each point of the Likert scale were summarised as counts in relation to the total number of completed questionnaires.

Table 2
Initial cases reported to the surveillance system for undiagnosed serious infectious illness, that were subsequently denotified due to a diagnosis, England, 11 July 2011-10 January 2013 (n=12)