Performance of case definitions for influenza surveillance

L Jiang1, V J Lee1,2, W Y Lim1, M I Chen (mark.chen.ic@gmail.com)1,3,4, Y Chen1, L Tan1, R T Lin5, Y S Leo1,3, I Barr6, A R Cook1,7,8,9 1. Saw Swee Hock School of Public Health, National University of Singapore and National University Health System, Singapore 2. Biodefence Centre, Ministry of Defence, Singapore 3. Communicable Disease Centre, Institute of Infectious Disease and Epidemiology, Tan Tock Seng Hospital, Singapore 4. Emerging Infectious Diseases Program, Duke-NUS Graduate Medical School, Singapore 5. National Public Health Laboratory, Ministry of Health, Singapore 6. WHO Collaborating Centre for Reference and Research for Influenza, Victorian Infectious Diseases Reference Laboratory, Melbourne, Australia 7. Yale-NUS College, National University of Singapore, Singapore 8. Department of Statistics and Applied Probability, Faculty of Science, National University of Singapore, Singapore 9. Program in Health Services and Systems Research, Duke-NUS Graduate Medical School, Singapore


Introduction
The emergence of influenza A(H1N1)pdm09 virus, a novel strain of influenza virus A(H1N1), in April 2009 and its subsequent rapid global spread have focused attention on influenza surveillance capabilities worldwide [1].As one of the main components of influenza surveillance, surveillance for influenza-like illness (ILI) among outpatients or patients of emergency departments can provide early warning of increased influenza virus circulation and information on where influenza activity is occurring, track the course of influenza activity during the season, and serve as a source of samples for virus isolation [2].It is hence important to have a reliable case definition for ILI.While the presence of fever and selected symptoms of acute respiratory tract infection are common features, case definitions used for ILI differ slightly from country to country and have also changed over time [3][4][5][6][7][8].Notably, it remains unclear if any of these specific combinations of symptoms or temperature cut-off point is better than others for influenza surveillance.
There have been a number of studies evaluating the performance of ILI case definitions, but most of these studies were based on outpatients visiting EDs and general practices or hospitalised patients [7][8][9][10][11][12], which may be subject to biases arising from how different individuals and populations access medical care.Moreover, there have recently been initiatives to use self-reported symptoms based on telephone or Internet surveys for influenza surveillance [13], and it is unclear if ILI case definitions validated on cases seeking medical care are appropriate in such situations.Furthermore, the pattern of medical care consultations in influenza as well as non-influenza related acute illness episodes may also affect the performance of ILI case definitions for influenza surveillance.What is hence needed is a study that can capture data on symptoms in the community, verify which of these might be associated with influenza, and clarify how individual and episode level characteristics associated with medical care consultation might affect the quality of ILI surveillance data [2,14,15].Singapore, a tropical south-east Asian city-state and global travel hub, detected its first imported case of influenza A(H1N1)pdm09 in late May 2009.Sustained community transmission started in late June 2009, with epidemic activity peaking in early August and subsiding in September 2009 [16].In this study, we exploited the combination of serological investigations and self-reported data on symptoms and medical consultations from a cohort established in the run-up to the pandemic to compare the ILI case definitions used by the European Centre for Disease Prevention and Control (as mandated by the European Union for communicable disease reporting, and henceforth abbreviated as EU-ECDC) [3,17], the United States Centers for Disease Control and Prevention (US-CDC) [6] and the World Health Organization (WHO) (previously used ('old') [4] and the January 2014 revised ('new') [5]) with regard to sensitivity, specificity, predictive value and likelihood ratio.We also illustrate the performance of various ILI-based approaches to estimating influenza incidence during the first wave of influenza A(H1N1) pdm09 in Singapore.Finally, we assessed factors associated with primary care consultation, to highlight how consultation patterns vary by population subgroups, disease, symptoms and timing, as these factors may additionally complicate the interpretation of ILI surveillance data.

Study design and recruitment
This prospective community cohort study was part of a larger study to determine serological conversion to influenza A(H1N1)pdm09 virus in different populations [18][19][20].We recruited 838 community-dwelling adults aged 21-75 years and aimed to get three blood samples from each participant.Banked samples were used for the baseline sample (sample 1); these were obtained during 29 June 2005 to 27 June 2009, before widespread community transmission of influenza A(H1N1) pdm09 virus in Singapore.Two additional blood samples were obtained: an intra-epidemic sample (sample 2) collected about four weeks after the peak of the epidemic (20-29 August 2009) and a post-epidemic sample (sample 3) collected about four weeks after the epidemic had subsided (6-11 October 2009).Only individuals who contributed at least two blood samples were included in this analysis.
A baseline telephone interview with a standardised questionnaire was conducted at recruitment, followed by fortnightly interviews throughout the epidemic period for new onset of the following symptoms, with dates of onset for each of the individual symptoms collected: • respiratory symptoms -cough, sore throat, runny or blocked nose and breathlessness; • constitutional symptoms -fever, myalgia and headache; • gastrointestinal symptoms -abdominal pain, nausea, vomiting and diarrhoea.
All participants provided written consent, and the study was approved by the Institutional Review Board of the National University of Singapore.

Laboratory methods and definition of seroconversion
Haemagglutination-inhibition assay was performed according to standard protocols of the WHO Collaborating Centre for Reference and Research on Influenza in Melbourne, Australia, as previously described [18].We defined seroconversion as a fourfold or greater increase in antibody titres to influenza A/ California/7/2009 (H1N1) pandemic virus between any successive pairs of blood samples.For each subject, we assigned a 'seroconversion date', against which the timing of various illness episodes was assessed.This would be the date of collection of sample 2 if seroconversion had occurred by then, regardless of additional changes in titre in sample 3. We also recorded the date of sample 3 collection if seroconversion was detected then but not in sample 2.

Illness episodes and case definitions
We defined an illness episode as acute onset of at least one of the symptoms listed in the earlier section on study design and recruitment.As symptoms due to an infection could start on different days, we defined onset of an illness episode as a cluster of symptoms starting within seven days of each other, and defined the onset date of that illness episode based on the earliest symptom in the cluster.Any symptoms starting after these seven days were then considered as part of a new illness episode.Since infection with influenza A(H1N1)pdm09 and the resultant illness episode had to have an onset date consistent with the seroconversion date, we therefore defined a probable influenza A(H1N1)pdm09-related episode as an episode in which the earliest symptom occurred at least one day before the seroconversion date.Illness episodes occurring on or after the day when seroconversion was detected, or in individuals who did not seroconvert, could then be classified with some certainty as being unrelated to influenza A(H1N1)pdm09.All combinations of episode occurrences relative to seroconversion, and how these are classified, are given in Figure 1.In some seroconverting individuals, there was more than one probable influenza A(H1N1)pdm09-related episode, based on the timing of symptoms.Since we cannot definitively identify which of these episodes was caused by influenza A(H1N1)pdm09, we opted to exclude such observations, and thus defined a presumptive influenza A(H1N1)pdm09 episode as those episodes that occurred in individuals who had only a single probable influenza A(H1N1)pdm09-related episode.
Several specific case definitions commonly used for influenza surveillance were analysed: • acute respiratory illness (ARI) -we defined an ARI episode as acute onset of any of the following respiratory symptoms: cough, shortness of breath, Permutations of illness episode occurrence relative to seroconversion and classification of illness episode NA: not applicable.Algorithm for classifying illness episodes as probable influenza A(H1N1)pdm09-related (triangles) and unrelated to influenza A(H1N1)pdm09 (boxes), based on when seroconversion (arrows, which denote a fourfold or greater increase in antibody titre) occurred.We defined a probable influenza A(H1N1)pdm09-related episode as an episode in which the earliest symptom occurred at least one day before the seroconversion date.The vertical axis and horizontal axis denote antibody titre and time respectively using an arbitrary scale.The right-hand part of the horizontal axis is not applicable if only the first two blood samples are collected.Episodes that occur in the corresponding time intervals with different permutations for seroconversion are classified as shown in panels A to J.

Statistical analysis
To assess the performance of different case definitions, we compared how well different symptoms, as well as commonly used case definitions, could distinguish presumptive influenza A(H1N1)pdm09 episodes from those classified as unrelated to influenza A(H1N1) pdm09.Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), odds ratio (OR), positive likelihood ratio (LR+), negative likelihood ratio (LR−) and 95% confidence intervals (CIs) for OR, LR+ and LR− were calculated.
For each case definition, we also used the total number of episodes per 100 individuals, the number of episodes involving primary care consultation per 100 individuals, and the proportion of individuals who ever had such an episode, to illustrate how these measures might perform in monitoring the progress of an epidemic compared with virological surveillance of ILI based on positive samples from sentinel general practices submitting respiratory samples to the National Public Health Laboratory.We also compared the respective estimates of influenza attack rates with serological data.
Finally, we investigated if demographics, timing of episode onset by day of the week and nature of symptoms affected presentation to primary care, with the outcome of interest being whether that illness episode was associated with a primary care consultation within seven days after onset.
Because each individual could have had more than one illness episode during the whole study period, a multilevel model was used in all logistic regression analyses to control for the effects of multiple episodes in the same individual.All statistical analysis was done with

Results
Our analysis included 727 individuals who provided two or more blood samples (Figure 2), of whom 98 (13.5%) seroconverted.Those who seroconverted were significantly younger (42. a Number of presumptive influenza A(H1N1)pdm09 episodes where n = 36 for all episodes, and n = 18 for episodes with primary care consultation.b Number of episodes unrelated to influenza A(H1N1)pdm09 where n = 664 for all episodes, and n = 201 for episodes with primary care consultation.c Self-reported feeling of having a fever.
Of 788 reported illness episodes, there were 36 (4.6%) presumptive influenza A(H1N1)pdm09 episodes and 664 (84.3%) episodes unrelated to influenza A(H1N1) pdm09; the corresponding numbers were 18 (7.2%)and 201 (80.7%) if restricting to those episodes with primary care consultation (n = 249).Reported symptoms for presumptive influenza A(H1N1)pdm09 episodes and episodes unrelated to influenza A(H1N1) pdm09 are shown in Table 1.Among the 36 presumptive influenza A(H1N1)pdm09 episodes, cough (n = 30) and fever (n = 18) were the most common respiratory and constitutional symptoms respectively, while gastrointestinal symptoms were rare.Cough, sore throat, fever and headache were significantly more common among presumptive influenza A(H1N1)pdm09 episodes in the univariate analysis; with the exception of sore throat, these were also significant in the multivariate analysis.The ORs for cough, fever and headache in the multivariate model were 8.8, 4.3 and 3.1 respectively.Results for the small number of presumptive influenza A(H1N1)pdm09 episodes for which there had been primary care consultation were similar.
The performance of different case definitions in distinguishing presumptive influenza A(H1N1)pdm09 from unrelated illness episodes is shown in Table 2. Of the 36 presumptive influenza A(H1N1) pdm09 episodes identified, only three did not fulfil any of the case definitions we assessed (one episode where only headache was reported, and two episodes which both had nausea and vomiting only).Among all the case definitions we looked at, ARI gave the highest sensitivity (92%) but the lowest specificity (35%).Using FRI improved the specificity to 90% but reduced the sensitivity to 50%.Among the four ILI case definitions, the modified EU-ECDC ILI case definition was more sensitive (61%) than for FRI but less specific (85%).The other three ILI case definitions had similar sensitivity (36-39%) and specificity (95-97%).On comparing predictive values and likelihood ratios, all four case definitions gave similar NPVs (95-97%), but the WHO new ILI case definition had the highest PPV (42%), followed by the WHO old ILI case definition (39%), the US-CDC ILI case definition (31%) and the modified EU-ECDC ILI case definition (18%).Similarly, while all case definitions gave similar LR−, the WHO new ILI case definition had the highest LR+ (13.32), followed by the WHO old ILI case definition (11.99),US-CDC ILI case definition (8.33) and the modified EU-ECDC case definition (4.10).The performance of the four ILI case definitions relative to a ARI: we defined an ARI episode as acute onset of any of the following respiratory symptoms: cough, shortness of breath, sore throat or nasal congestion (runny nose or blocked nose).FRI: ARI with self-reported fever, regardless of body temperature measurement.Modified EU-ECDC ILI: one or more respiratory symptoms (cough, sore throat and/or shortness of breath) plus one or more systemic symptoms (selfreported fever, headache and/or myalgia); this is an approximation of the EU-ECDC ILI case definition [3,17] (which additionally has malaise as one of the systemic symptoms).US-CDC ILI: fever defined as body temperature ≥ 37.8 °C together plus cough and/or sore throat in the absence of a known cause other than influenza [6].WHO old ILI: fever defined as body temperature > 38 °C plus either cough or sore throat [4].WHO new ILI: fever defined as body temperature ≥ 38 °C plus cough [5].b Number of presumptive influenza A(H1N1)pdm09 episodes where n = 36 for all episodes, and n = 18 for episodes with primary care consultation.c Number of episodes unrelated to influenza A(H1N1)pdm09 where n = 664 for all episodes, and n = 201 for episodes with primary care consultation.

Figure 3
Illness episode rates using different case definitions for all episodes and episodes involving primary care consultation compared with laboratory surveillance and serological attack rates, Singapore, June-October 2009 Right-hand graphs: rates per 100 persons from serological infections (midline with bars giving 95% CIs) and estimated rates from the case definitions (lines).Panels A to F are for ARI, FRI, EU-ECDC ILI, US-CDC ILI, WHO old ILI, and WHO new ILI case definitions.ARI: we defined an ARI episode as acute onset of any of the following respiratory symptoms: cough, shortness of breath, sore throat or nasal congestion (runny nose or blocked nose).FRI: ARI with self-reported fever, regardless of body temperature measurement.Modified EU-ECDC ILI: one or more respiratory symptoms (cough, sore throat and/or shortness of breath) plus one or more systemic symptoms (self-reported fever, headache and/or myalgia); this is an approximation of the EU-ECDC ILI case definition [3,17] (which additionally has malaise as one of the systemic symptoms).US-CDC ILI: fever defined as body temperature ≥ 37.8 °C together plus cough and/or sore throat in the absence of a known cause other than influenza [6].WHO old ILI: fever defined as body temperature > 38 °C plus either cough or sore throat [4].WHO new ILI: fever defined as body temperature ≥ 38 °C plus cough [5].
30 Jul to 28 Aug 149 62 ( 29 Aug to 8 Oct 184 55 each other was unchanged when repeating the analysis using episodes in which the individuals had presented to primary care, but the sensitivities were slightly higher while the specificities and LR+s, PPVs and LR+s were lower. We further explored how the respective case definitions would have performed in capturing influenza A(H1N1)pdm09 episodes in our cohort.Since only a minority of ARI was influenza A(H1N1)pdm09, ARI performed poorly in monitoring epidemic progression (Figure 3, panel A), where we were unable to discern a clear epidemic pattern as compared with the pattern in influenza A(H1N1)pdm09-positive samples from sentinel general practices.There were, however, no substantive differences in the epidemic curves when using the other five case definitions (Figures 3, panels B to F).For attack rates, we present estimates derived from all self-reported episodes, ever having had an episode, and episodes having a consultation at a general practice; these would reflect what might be obtained through repeated Internet or telephone surveys, retrospective surveys after the epidemic for ever having had an episode, and surveillance based on primary care episodes respectively.Estimates using all self-reported ARI episodes and having ever had an ARI episode gave much higher rates than serological attack rates, although overall general practice clinic consultation rates were (perhaps coincidentally) similar.The estimates numerically closest to serological attack rates for all episodes and ever having an episode were those for FRI; estimates from the modified EU-ECDC ILI were slightly higher, while those from the US-CDC ILI, WHO old ILI, and WHO new ILI were substantially lower.
The influence of participant characteristics and the type and timing of symptoms on determining presentation to primary care is shown Table 3.Having self-reported fever, cough or sore throat was associated with an increase in likelihood to seek primary care both in univariate and multivariate analysis.On the other hand, episodes that were presumptive influenza A(H1N1) pdm09, while significantly associated with seeking primary care in univariate analysis, were not more likely to result in primary care consultation after adjusting for the type of symptoms experienced.For participant characteristics, in the multivariate analysis, those aged 25-34 years and those employed in work outside the home were significantly more likely to seek care.The timing of episodes also affected the probability of seeking care: individuals with episodes in which onset was on a Saturday were significantly less likely to seek care.Individuals with episodes occurring from 9 July to 28 August were also marginally more likely to seek care.
When considering all illness episodes for which primary care was sought, consultations occurred more frequently on Mondays and less frequently on Saturdays and Sundays as compared with other days of the week (Figure 4).Illness episodes with onset on weekdays (Figure 4, panels B to F) were most likely to consult on the day of onset and the day after (Figures 4, panels B to E), while those with onset on weekends would often delay consultation until Monday (Figure 4, panels G and H); this consultation pattern was consistent regardless of the case definition used (data not shown).

Discussion
Cough, sore throat and fever are commonly included in ILI case definitions [4,6,8].In our study, cough, sore throat and fever were significantly more common among presumptive influenza A(H1N1)pdm09 episodes in univariate analysis, although sore throat was not in the multivariate model.This is because most influenza episodes with sore throat would also have a cough [21]; in our data, more than 90% of presumptive influenza A(H1N1)pdm09 episodes with sore throat also reported having a cough.Headache is another symptom that physicians associate with influenza [22], and headache was significantly more common among presumptive influenza A(H1N1)pdm09 episodes in both univariate and multivariate analysis in our study.A similar study among schoolchildren in Taiwan also indicated that headache had a significant association with influenza infection [23], although this was not found to be the case in several other studies [7,9,11].
Previous work assessing the appropriateness of influenza case definitions have focused only on patients who seek care for respiratory symptoms or those presenting with a fever [8,10,11], whereas our study captures all events with serological evidence of influenza A(H1N1)pdm09 virus infection.Our results show that the modified EU-ECDC ILI performed rather differently from the other three ILI case definitions, with a much higher sensitivity and a relatively lower specificity, mainly because the EU-ECDC ILI case definition does not require a measured fever, in contrast to the other case definitions (body temperature in US-CDC ILI: ≥ 37.8 °C, WHO old ILI: > 38 °C, WHO new ILI: ≥ 38 °C).One of the objectives of influenza surveillance is to signal the start of an influenza season or influenza epidemic, and an ILI case definition with higher sensitivity integrated with laboratory surveillance can be better at detecting the start of sustained community circulation of influenza, although it may also be more resource intensive in terms of requirements for laboratory testing [4].The sensitivities and specificities of the other three ILI case definitions were very similar, but the PPVs and LR+s were different, with the US-CDC ILI having the lowest and the WHO new ILI having the highest PPV and LR+.In the analysis using all reported episodes as well as episodes with primary care consultation, the LR+ calculated for the WHO new ILI case definition was higher (13.32 and 8.12 respectively) and provides support for the simplification of removing sore throat as a component of the case definition.Interestingly, although the number of presumptive influenza A(H1N1)pdm09 episodes with primary care consultation was small, the LR+ of 5.29 for the US-CDC ILI and 6.87 for the WHO old ILI case definition was fairly similar to the previous LR+ estimates reported by the studies of Govaert et al. [24] and Stein et al. [25] in individuals who sought care for symptoms suggestive of influenza.
When comparing the accuracy of estimates obtained from the various case definitions against serological attack rates, the use of ARI is clearly inadequate due to the substantial noise from episodes unrelated to influenza A(H1N1)pdm09.While the FRI estimates for all episodes and ever having an episode are numerically closest to serological attack rates, we suspect this is an artefact due to causes of FRI unrelated to influenza A(H1N1)pdm09 compensating for the imperfect sensitivity of the case definition, an effect that cannot be expected to be consistent in future epidemics or across regions.The US-CDC ILI, WHO old ILI, and WHO new ILI definitions all underestimate infection rates, as not all cases present with the requisite symptoms, and therefore data on the proportion of infections not meeting these case definitions would be needed as a multiplier to estimate overall attack rates [15].Feasibly, this proportion could be obtained from serological studies performed with similar influenza strains elsewhere, and may not need to be repeated in different locations and time points, though it may be preferable to use external data only provisionally and later validate estimates with serological data from the same setting and outbreak.
Our previous data also illustrate, however, that how individuals seek care is an important determinant of the performance characteristics of surveillance data collected from primary care doctors [15].When exploring how various factors influenced consultations, presumptive influenza A(H1N1)pdm09 was not independently associated with seeking care after adjusting for the type of symptoms experienced (specifically, fever, cough and sore throat) and as such the clinical presentation of the particular strain of influenza virus would affect the performance of ILI surveillance based on primary care consultations.Not surprisingly, those employed in work outside the home were significantly more likely to seek care, probably due to the need for time off work, which in Singapore usually requires a physician's certificate.An Israeli study also found that obtaining a sick-leave note was the main reason to seek medical care for patients aged 30-65 years with influenza-like symptoms [26].It is also noteworthy that consultations more frequently occurred on Mondays and much less frequently on Saturdays and Sundays, and individuals with episodes with onset on Saturday were significantly less likely to seek care, which is possibly related to the need for certifying absenteeism from work, although it could also reflect heterogeneities in the infection risk across the week, as has been found for dengue [27].Estimates of infection rates derived from primary care consultations are therefore most stable if performed on a weekly basis to smooth out these fluctuations, or if daily fluctuations are explicitly accounted for in the analysis [28].In addition, the slightly increased probability of consultation during some weeks of the pandemic (in particular the period 9 July to 29 July (Table 3)), also shows the importance of early and repeated surveys during the course of an epidemic as failure to account for changes in healthseeking behaviour may lead to misleading estimates.This study has several limitations that should be addressed in future work.Attack rates and clinical presentations of influenza virus infection often vary by age [29,30].The adult-only cohort (21-75 years) may not be representative of all age groups, and we only identified 36 presumptive influenza A(H1N1)pdm09 episodes with which to assess various case definitions.Moreover, differences in clinical symptoms between influenza virus subtypes [31,32] mean that the case definition may have to be adjusted depending on the circulating strain.Other limitations include the possibility of non-seroconversion upon infection [33], which may lead to systematic errors in the estimates.Recall bias, in spite of the short interval between surveys, may be present, and we did not consider the timing of symptoms relative to presentation (e.g.sore throat may precede cough).Knowing about the study objectives or that there was a pandemic may also have led participants to seek healthcare more, or over-report certain symptoms.Finally, multiple general practice consultations were not considered as it was not possible to ascertain if individuals fulfilled the case definitions across all consultations.
In conclusion, we evaluated the performance of the modified EU-ECDC ILI, US-CDC ILI, WHO old ILI and WHO new ILI case definitions in detecting illness episodes due to influenza A(H1N1)pdm09.The recently revised WHO ILI case definition was found to be an improvement over the others, with a higher PPV and LR+.Regardless of the performance of the different case definitions, health-seeking behaviour was strongly associated with several factors independent of the symptoms and disease under consideration, and population-and episode-level characteristics, such as the proportion who work and day-of-week effects respectively, would need to be accounted for when interpreting surveillance data based on ILI case definitions.

Figure 2
Figure 2 Characteristics of participants, by seroconversion status, included in study of case definition performance for influenza surveillance, Singapore, June-October 2009 (n = 727) A Age in years, median (IQR) Seroconverted Not seroconverted 20 30 40 50 60 70 80 years ARI: acute respiratory illness; CI: confidence interval; ECDC: European Centre for Disease Prevention and Control; EU: European Union; FRI: febrile respiratory illness; ILI: influenza-like illness; NPHL: National Public Health Laboratory; US-CDC: United States Centers for Disease Control and Prevention; WHO: World Health Organization.Left-hand graphs: number of episodes per 100 persons obtained from the case definition (lines) and number of influenza A(H1N1)pdm09 samples obtained from sentinel general practices (bars}, by epidemiological week of the episode onset date, from week 23 (7 June 2009 to 13 June 2009) to week 40 (4 October 2009 to 10 October 2009).

Table 2
Performance of case definitions in distinguishing presumptive influenza A(H1N1)pdm09 from unrelated illness episodes, Singapore, June-October 2009 (n = 700) ARI: acute respiratory illness; CI: confidence interval; ECDC: European Centre for Disease Prevention and Control; EU: European Union; FRI: febrile respiratory illness; ILI: influenza-like illness; LR+: positive likelihood ratio; LR−: negative likelihood ratio; NPV: negative predictive value; PPV: positive predictive value; US-CDC: United States Centers for Disease Control and Prevention; WHO: World Health Organization.

Table 3
Participant and illness episode-level characteristics associated with seeking primary care, Singapore, June-October 2009 (n = 700 episodes)