26 March 2015
Evaluation of a point-of-care blood test for identification of Ebola virus disease at Ebola holding units, Western Area, Sierra Leone, January to February 2015
Current Ebola virus disease (EVD) diagnosis relies on reverse transcription-PCR (RT-PCR) technology, requiring skilled laboratory personnel and technical infrastructure. Lack of laboratory diagnostic capacity has led to diagnostic delays in the current West African EVD outbreak of 2014 and 2015, compromising outbreak control. We evaluated the diagnostic accuracy of the EVD bedside rapid diagnostic antigen test (RDT) developed by the United Kingdom’s Defence Science and Technology Laboratory, compared with Ebola virus RT-PCR, in an operational setting for EVD diagnosis of suspected cases admitted to Ebola holding units in the Western Area of Sierra Leone. From 22 January to 16 February 2015, 138 participants were enrolled. EVD prevalence was 11.5%. All EVD cases were identified by a positive RDT with a test line score of 6 or more, giving a sensitivity of 100% (95% confidence interval (CI): 78.2–100). The corresponding specificity was high (96.6%, 95% CI: 91.3–99.1). The positive and negative predictive values for the population prevalence were 79.0% (95% CI: 54.4–93.8) and 100% (95% CI: 96.7–100), respectively. These results, if confirmed in a larger study, suggest that this RDT could be used as a ‘rule-out’ screening test for EVD to improve rapid case identification and resource allocation.
More than one year after the first human-to-human transmission, the largest Ebola virus disease (EVD) outbreak continues in West Africa, with an estimated 24,701 cases reported and 10,194 deaths by 15 March 2015 . To date, Sierra Leone is the most severely affected country.
Case identification is essential for effective EVD control and rapid case detection is critical for rationalisation of resources and implementation of early treatment interventions . A locally adapted EVD clinical case definition allows suspected cases to be identified and isolated in Ebola holding units (EHU), but this alone is inadequate to reliably differentiate EVD cases from patients with other conditions that mimic EVD presentation . A confirmed EVD diagnosis is a prerequisite for transfer of a patient to an EVD treatment centre (ETC) to access EVD-specific care. All patients meeting the suspected case definition require isolation, laboratory sampling and diagnostic testing. For such patients, a negative EVD result is required before admission into general healthcare.
Current EVD diagnosis relies on reverse transcription-PCR (RT-PCR) technology . This test is highly sensitive and specific but requires skilled laboratory personnel and technical infrastructure . In the early months of the current West African outbreak, these personnel and infrastructure were largely absent locally. As the EVD response has grown, laboratory infrastructure in the region has improved, but this may not be sustainable in the long term or available at the onset of future outbreaks. In addition, the current EVD diagnostic pathway has cost, resource and safety implications relating to venous blood sampling inside the EHU, timely transport of samples to the EVD diagnostic laboratory, potential for labelling error, and rapid relay of results .
One immunofiltration antigen-based assay developed in the mid-2000s has been tested on field specimens from 2003, but is not yet available in routine clinical practice and requires a photometer for analysis .
A rapid diagnostic test for EVD, performed at the bedside in EHUs or other isolation facilities would be of great benefit . The Defence Science and Technology Laboratory (DSTL) in the United Kingdom (UK) has developed a rapid antigen diagnostic test (RDT) for EVD diagnosis. The DSTL EVD RDT is a bedside lateral flow assay using capillary blood rather than venous blood to detect presence of an undisclosed Ebola virus antigen. The test can be conducted and interpreted with minimal training and the result is obtained within 20 min. A semi-quantitative result is obtained by scoring a test line on colour intensity.
In this study, we evaluate the diagnostic accuracy of the DSTL EVD RDT compared with the gold standard Ebola virus (EBOV) RT-PCR in an operational setting for EVD diagnosis of suspected EVD cases admitted to EHUs in the Western Area of Sierra Leone.
The study was conducted at four EHUs in the Western Area of Sierra Leone that routinely isolated suspected EVD cases and collected diagnostic blood samples for EVD testing: Connaught Government Hospital (the national adult referral hospital), Macauley Street Government Hospital, Rokupa Government Hospital, and Newton Health Centre. These sites belong to a network of holding units supported by King’s Sierra Leone Partnership (KSLP) and managed by the Ministry of Health and Sanitation (MOHS) and use a screening tool based on the national case definition for suspected EVD cases, combining exposure risk evaluation and a symptom checklist for identification of suspected EVD cases. Each centre had trained phlebotomists and local healthcare workers who routinely provided patient care. Clinical staff were invited to training in the use of the RDT and study protocol which was undertaken in three one-hour sessions.
Staff who completed the training were approved to enrol patients and perform the RDT. Trained clinical staff obtained verbal informed consent from consecutive patients newly admitted to the EHU during the study period, wearing appropriate personal protective equipment . Patients who could not give informed consent (e.g. due to young age, cognitive impairment or confusion) and patients who withheld consent were not enrolled.
Enrolment occurred on the day of admission or on the following day when patients were admitted during the night. In all cases, enrolment occurred before the results of routine EVD diagnostic testing were available, i.e. only suspected cases were enrolled. The RDT was performed at the bedside. All equipment for the RDT was provided in individually packaged test kits. Capillary blood for the RDT was obtained using a sterile lancet to prick a finger. Blood was applied to the well of the lateral flow device with a small pipette, followed by three drops of buffer. After 20 min, the RDTs were read in designated areas with good lighting and scores were obtained with the aid of a scorecard. RDTs were scored C when a single control (C) line was visible and CT when the C line and the test (T) line were visible. If visible, the T line was scored [2-10] on colour intensity by matching the T line to samples on the scorecard. Clinical staff performing RDTs were blind to RDT score interpretation.
Venepuncture for routine EVD diagnostic testing was performed as per routine clinical care, usually on the same day as the RDT. Venous blood samples were transported to the Public Health England (PHE) laboratory at Port Loko for EVD RT-PCR testing with the Altona RealStar Filovirus screen kit for real-time PCR (Altona Diagnostics Gmbh, Germany). Extraction was performed using a manual method with the Qiagen QIAamp Viral RNA kit (Qiagen). Altona quote a detection limit of 1.39 copies/µL of eluate (range: 0.69 to 5.32) for Zaire EBOV and 100% specificity against a range of viruses. In a small number of cases, routine EVD diagnostic testing by RT-PCR on venous blood was performed at other local diagnostic laboratories. Laboratory personnel were blind to the RDT result. The Altona assay has been selected by the World Health Organization (WHO) as the reference standard for this outbreak.
Study enrolment and results were recorded in a password protected spreadsheet and matched to EVD RT-PCR results for analysis by the study coordinator (NFW). Analysis was performed in Excel (Microsoft Corporation), Prism 6 (GraphPad Software, Inc), and Medcalc version 15.2.2 (Medcalc Software, Ostend Belgium). As the DSTL EVD RDT provides a quantitative result, analysis was performed to establish the diagnostic accuracy of the test for the range of CT scores, in comparison with the gold standard result. Results were anonymised before dissemination. Reporting of results follows the STARD (Standards for Reporting Diagnostic Accuracy Studies) statement .
The study was approved by the Sierra Leone Ethics and Scientific Review Committee (SLESRC, 16/01/2015).
Participants and enrolment
Participants were recruited consecutively at study sites, from 22 January to 16 February 2015. A total of 138 participants were enrolled. At Connaught Hospital, 112 patients were enrolled. This constituted 83% of 135 total admissions at Connaught Hospital EHU during the study period. Seven enrolled participants were excluded at the analysis stage because insufficient information was available (Figure 1). Of these patients, four had RDT tests performed but did not have corresponding EVD RT-PCR results available. The RDT result was negative in each of these cases. One patient had a negative EVD RT-PCR result but did not have an RDT result recorded. One patient had neither RDT nor EVD RT-PCR result available. One patient with a negative RDT had no corresponding EVD RT-PCR result available but similar clinical details to a subsequent participant, suggesting that this was an error in documentation and possibly a double entry. Finally, 131 participants were included in the analysis. Of those, 90 (68.7%) were male, and the median age was 32 years (interquartile range (IQR): 24–47 years).
Figure 1. DSTL rapid diagnostic antigen test for Ebola virus disease, study enrolment and inclusion, Sierra Leone, January–February 2015 (n = 138)
Ebola virus disease diagnosis
Fifteen of 131 patients tested positive for EVD by EVD RT-PCR, giving a study EVD prevalence of 11.5% (Figure 2). Data on duration of symptoms before presentation for EVD-positive patients was available for seven of 15 (47%) cases. In these patients, median symptom duration before date of EVD diagnostic testing was four days (IQR: 3–5 days). The PHE Port Loko laboratory processed 125 of the laboratory samples (95%). Three samples were processed at the PHE Kerry Town laboratory using the same diagnostic assay and standard operating procedure as PHE Port Loko, and three samples were processed at other laboratories.
Figure 2. Diagnosis by gold standard (Ebola virus PCR) in study participants for the DSTL rapid diagnostic antigen test for Ebola virus disease, Sierra Leone, January–February 2015 (n = 131)*
Performance of the rapid diagnostic antigen test
Twenty-four patients had RDT results with both C and T line visible (CT). In 15 of these patients, an EVD diagnosis was made by positive EVD RT-PCR and in nine cases, EVD RT-PCR results were negative (Table 1). In all confirmed cases of EVD, a T line was present on the RDT (Table 1 and Figure 3). Higher CT scores were found in patients with EVD than those without EVD (Table 1 and Figure 3).
Table 1. Results of DSTL rapid diagnostic antigen test for Ebola virus disease, Sierra Leone, January–February 2015 (n = 131)
Figure 3. CT scores for the DSTL rapid diagnostic antigen test for Ebola virus disease, Sierra Leone, January–February 2015 (n = 131)
Table 2 details the sensitivity and specificity of the RDT with increasing CT score. If any test with a visible T line (corresponding to CT score of CT2 and above) was classified as positive, the RDT had a sensitivity of 100% (95% confidence interval (CI): 78.2–100) and a specificity of 92% (95% CI: 85.8–96.4) compared with the gold standard RT-PCR. If any test with a T line score above 4 (corresponding to a CT score of CT6 and above) was classified as positive, the RDT remained 100% sensitive (95% CI: 78.2–100), but had a higher specificity of 97% (95% CI: 91.4–99.1). The specificity of the test increased with higher CT score threshold for a positive result, but the corresponding sensitivity was reduced for a CT score of 8 or above. A specificity of 99% (95% CI: 95.3–100.0) was achievable if an RDT score above CT8 was classified as positive, but the corresponding sensitivity was low at 40% (95% CI: 16.3–67.7).
Table 2. Diagnostic accuracy of DSTL rapid diagnostic antigen test for Ebola virus disease compared with gold standard PCR, by CT score, Sierra Leone, January–February 2015 (n = 131)
The positive predictive value (PPV) of the DSTL EVD RDT, for the study population EVD prevalence of 11.5%, was 79.0% (95% CI: 54.4–93.8) for a CT score of 6 and above and increased at higher CT score thresholds for a positive result (Table 3). A negative predictive value of 100% was achievable if a CT score 2 and above, a CT score 4 and above, or a CT score 6 and above, were classified as a positive result.
Table 3. Positive and negative predictive values of DSTL rapid diagnostic antigen test for an Ebola virus disease prevalence of 11.5%, by CT score, Sierra Leone, January–February 2015 (n = 131)
Our data suggest that the DSTL EVD RDT is highly sensitive, specific and performs well in an operational setting. A high sensitivity is critical to EVD diagnostic test acceptability. A highly sensitive screening test such as this would allow high-risk suspected EVD cases to be prioritised for isolation and confirmatory diagnostic testing with RT-PCR, reducing non-EVD admissions in EHUs. If the sensitivity was lower, EVD-positive cases could be inappropriately discharged to inpatient wards, with risks of onward nosocomial transmission.
Although the specificity was high, a small number of non-EVD patients tested positive with the RDT at all T Line scores. Using the DSTL EVD RDT as a ‘rule-in’ test for EVD would result in some EVD-negative patients being inappropriately referred to ETCs and exposed to nosocomial risk, unless confirmatory testing by RT-PCR was undertaken.
Therefore the RDT may be best used as a ‘rule-out’ screening test. If the high sensitivity of the RDT is confirmed by further evaluation, this would allow RDT-negative patients to be discharged, reducing pressure on isolation unit beds and diagnostic laboratories. It would allow safe and rapid referral of sick, RDT-negative patients to general wards to receive appropriate healthcare, or for patients with milder illness to be discharged. In addition, emergency surgical procedures and obstetric deliveries could be performed without EVD transmission risk, following a negative RDT. This would allow healthcare workers to confidently and safely treat non-EVD conditions without being exposed to potentially infectious patients and may allow normal healthcare services to be maintained in future epidemics. This has been a significant challenge during the current epidemic . Those with a positive RDT should be considered high-probability suspected EVD cases, prioritised for isolation in the appropriate risk-stratified area of the EHU, with confirmatory diagnostic testing performed by RT-PCR.
Our results, particularly if confirmed by larger studies on stored samples, support the use of this test for screening purposes.
The number of admissions in the study period was lower than expected and the EVD prevalence lower than that observed in late 2014, reducing the power of the study. In addition, it was intended that all consecutive EHU admissions should be recruited at study sites. This was not always possible as a limited number of trained staff were available to enter the EHUs to enrol patients and some patients were unable to give informed consent. However, at Connaught Hospital EHU, the main site of enrolment, the majority (83%) of admissions were enrolled. The wide confidence intervals around sensitivity will need further confirmatory work before routine clinical use.
The prevalence of EVD was low in our study compared with earlier in the outbreak, when up to 75% of admissions to the Connaught Hospital EHU were EVD-positive. This has resulted in a relatively low PPV for the RDT. As the PPV only applies for a particular population prevalence, the performance of the test should be confirmed at a higher population prevalence. It is likely that the PPV would be higher at a higher EVD prevalence.
We compared the RDT result to gold-standard EVD diagnosis with RT-PCR. The WHO recommends repeat testing of symptomatic patients who test negative for EVD by RT-PCR less than three days after the onset of their illness, as the sensitivity of EVD RT-PCR may be lower early in the clinical course of EVD . Our routine practice complied with this policy. However, it remains possible that we have classified some patients as false-positive RDTs who were infected with Ebola virus but had RT-PCR results below the assay detection limits. If this was the case, our study underestimates the diagnostic accuracy of the DSTL EVD RDT. PHE has now moved to an alternative in-house assay which his more sensitive than the Altona RT-PCR and may verify the DSTL test results in any future work. Further study is required to assess the performance of the RDT early in the clinical course of EVD and in the EVD incubation period.
Relationship to other studies
The WHO approved the first RDT for use as a screening test for EVD (ReEBOV Antigen Rapid Test) on the basis of a reported sensitivity of 91.8% (95% CI: 84.5–96.8) and a specificity of 84.6% (95% CI: 78.8–89.4). This RDT was evaluated on 147 fresh venous blood and 146 frozen plasma samples in a laboratory setting in Sierra Leone . Performance of this test in an operational setting has not been reported. Our findings suggest that the DSTL EVD RDT performs well against this benchmark, exceeding these reported findings in an operational setting.
The performance of the DSTL EVD RDT in this study strongly supports its use as a ‘rule-out’ screening test for EVD. Further laboratory and operational data are required to improve confidence and inform further on sensitivity and specificity in a broader setting.
We would like to acknowledge the invaluable role of the clinical staff at Connaught Hospital, Macauley Street Hospital, Newton Health Centre and Rokupa Government Hospital, who were instrumental in completion of the study, especially those who enrolled patients into the study: Muhammed Ali, William F Bangura, Ann-Marie B Condor, Yealie Conteh, Patricia James, Lovetta Jawara, Brima Kamara, Amie Koroma, Bilkisu A Koroma, Melrose Koroma, Mamie Kuyateh, Catherine Lonnie, Esther Ngaejia, Benjamin S Samura, Abu Bakar Sanu, Lansana Sesay, Emelia Sunchsu, and Sahr B Yoki. We are also grateful for operational support from Nandini Shetty at Public Health England. The Rapid Antigen Test kits were donated by the United Kingdom’s Defense Science and Technology Laboratory. The study was supported by the Response to the Ebola Outbreak in Sierra Leone, UK aid from the British people, implemented by the Ebola Response Consortium. KSLP acknowledges support from numerous voluntary donations for work on the Ebola virus disease outbreak response. NF Walker has been supported by the Wellcome Trust (094000).
Conflict of interest
NF Walker, CS Brown, D Youkee, K Russell, N Bentley, T Boyles, A Simpson, T Brooks, A Kamara, TB Kamara, M Lado, O Johnson designed the research; NF Walker, P Baker, D Youkee, N Williams, A Kalawa, B Healey and the RDT study team performed the research; AF Samba, F Koroma, MB King, BE Parker, M Thompson, B Kargbo, D Bash-Taqi, TB Kamara, A Kamara contributed vital technical support; NF Walker, CS Brown, P Baker analysed the data; NF Walker, CS Brown, P Baker, T Boyles wrote the draft manuscript; all authors have seen and approved the final manuscript.
*Erratum: On 12 June 2015, the x axis label in Figure 2 was corrected to read ‘Site of enrolment’.
- World Health Organization (WHO). Ebola situation report - 18th March 2015. Geneva: WHO; 2015. Available from: http://apps.who.int/ebola/current-situation/ebola-situation-report-18-march-2015
- WHO Ebola Response Team. Ebola virus disease in West Africa--the first 9 months of the epidemic and forward projections. N Engl J Med. 2014;371(16):1481-95. http://dx.doi.org/10.1056/NEJMoa1411100 PMID:25244186
- Youkee D, Johnson O, Amardeep K, Hall A, Lado M. King’s Sierra Leone Partnership Ebola holding units standard operating procedure. Freetown: KSLP; 2014.
- World Health Organization (WHO). Laboratory diagnosis of Ebola virus disease. Geneva: WHO; 2014. Available from: http://apps.who.int/iris/bitstream/10665/134009/1/WHO_EVD_GUIDANCE_LAB_14.1_eng.pdf
- Towner JS, Rollin PE, Bausch DG, Sanchez A, Crary SM, Vincent M, et al. Rapid diagnosis of Ebola hemorrhagic fever by reverse transcription-PCR in an outbreak setting and assessment of patient viral load as a predictor of outcome. J Virol. 2004;78(8):4330-41. http://dx.doi.org/10.1128/JVI.78.8.4330-4341.2004 PMID:15047846
- Pathmanathan I, O’Connor KA, Adams ML, Rao CY, Kilmarx PH, Park BJ, et al.; Centers for Disease Control and Prevention (CDC). Rapid assessment of Ebola infection prevention and control needs--six districts, Sierra Leone, October 2014. MMWR Morb Mortal Wkly Rep. 2014;63(49):1172-4. PMID:25503922
- Lucht A, Formenty P, Feldmann H, Gotz M, Leroy E, Bataboukila P, et al. Development of an immunofiltration-based antigen-detection assay for rapid diagnosis of Ebola virus infection. J Infect Dis. 2007;196(s2) Suppl 2;S184-92. http://dx.doi.org/10.1086/520593 PMID:17940948
- World Health Organization (WHO). Urgently needed: rapid, sensitive, safe and simple Ebola diagnostic tests. Geneva: WHO; 2014. Available from: http://www.who.int/mediacentre/news/ebola/18-november-2014-diagnostics/en/
- World Health Organization (WHO). Clinical management of patients in the Ebola Treatment Centres and other care centres in Sierra Leone. December 2014. Interim emergency guidelines. Geveva: WHO; 2014. Available from: https://extranet.who.int/ebolafmt/content/clinical-management-patients-ebola-treatment-centres-and-other-care-centres-sierra-leone
- University of Amsterdam (UOA). STARD Statement. Standards for the reporting of diagnostic accuracy studies. Amsterdam: UOA. [Accessed: 24 Feb 2015]. Available from: http://www.stard-statement.org/
- Jain V, Brown CS, Johnson O. Sierra Leone: the forgotten mortality. Glob Health Action. 2015;8(0):26757. http://dx.doi.org/10.3402/gha.v8.26757 PMID:25669239
- World Health Organization (WHO). WHO emergency use assessment and listing for Ebola virus disease IVDs. Public report. Product: ReEBOV™ Antigen Rapid Test Kit EUAL Number: EA 0011-011-00. Geneva: WHO; 2015. Available from: http://www.who.int/diagnostics_laboratory/procurement/150219_reebov_antigen_rapid_test_public_report.pdf