17 June 2010
Using tests for recent infection to estimate incidence: problems and prospects for HIV
Tests for recent infection (TRIs), such as the BED assay, provide a convenient way to estimate HIV incidence rates from cross-sectional survey data. Controversy has arisen over how the imperfect performance of a TRI should be characterised and taken into account. Recent theoretical work is providing a unified framework within which to work with a variety of TRI- and epidemic-specific assumptions in order to estimate incidence using imperfect TRIs, but suggests that larger survey sample sizes will be required than previously thought. This paper reviews the framework qualitatively and provides examples of estimator performance, identifying the characteristics required by a TRI to estimate incidence reliably that should guide the future development of TRIs.
When monitoring HIV epidemics it is vital to estimate incidence in order to plan and evaluate HIV programmes . Prospective cohort studies are the most direct way to achieve this. They are, however, expensive, prone to recruitment and retention bias, and potentially rendered unrepresentative by ethical obligations. The use of prevalence data in conjunction with mathematical modelling is an alternative approach [2,3], but is indirect and requires accurate knowledge of mortality and migration. The disadvantages of these methods have focused attention on estimating incidence from cross-sectional surveys [4-8], with the result that a number of assays and algorithms that test for recent infection have been developed [9,10]. In the context of HIV, such an assay or algorithm has sometimes been termed a STARHS (Serological Testing Algorithm for Recent HIV Seroconversion) [9,10], but we prefer to use the generic term ‘test for recent infection’ (TRI), because it does not specify a particular disease and method of testing. Recently, the World Health Organization (WHO) Technical Working Group on Statistical Approaches for Development, Validation and Use of HIV Incidence Assays has proposed using the term ‘recent infection testing algorithm’ (RITA). The term has not, however, gained universal acceptance.
TRIs identify HIV-positive individuals who have been infected recently. By using a TRI in a serosurvey, incidence (I) can be estimated by applying the epidemiological relationship (based on ‘Prevalence = Incidence x Duration’):
where R and S are the counts of ‘recently infected’ and ‘susceptible’ (HIV-uninfected) individuals observed in the cross-sectional survey and D is the mean duration spent in the ‘recently infected’ state, often called the (mean) window period. This incidence estimate is an average of the instantaneous incidence over a period of approximately D prior to the survey. The problem of incidence estimation then reduces to measuring the prevalence of ‘recent infection’, given knowledge of its duration.
TRIs usually discriminate recent from established infections by measuring specific aspects of the immune system which evolve during the course of initial infection. For HIV, this is typically the antibody response, with the titre, proportion of HIV-specific IgG, or antibody avidity (or a combination of these) providing quantitative output . Laboratory-defined thresholds are chosen to convert these outputs into categorical results. These results may be augmented with other clinical information, such as CD4 lymphocyte counts and antiretroviral therapy (ART) status, to classify individuals as either TRI-positive (P i.e. recent) or TRI-negative (N i.e. non-recent). Positive and negative in this context should not be confused with HIV-positive and HIV-negative.
The interaction between the virus and the immune system is complex, and individuals vary in their response to infection as assessed by a particular TRI. Modest variation is not intrinsically problematic, but serious complications arise if, in some individuals, the immune response is such that they remain indefinitely classified as TRI-positive or if individuals revert back to a TRI-positive classification as a result of advanced disease or in the presence of antiretroviral therapy. Unfortunately, both these complications arise for TRIs currently in use. This not only limits the applicability of the simple incidence estimator above, but also makes it difficult to define and estimate the mean duration spent in the recently infected state (i.e. to evaluate D). Methods for ‘adjusting’ estimates of incidence have been proposed [7,8] and adopted by the United States Centers for Disease Control and Prevention  but are currently under debate [12-15]. Recently, a formally rigorous framework has been developed [16,17]. We provide a summary of the framework and explore its implications for the analysis of surveys and development of new TRIs.
We now briefly describe the theoretical framework and how it can be generalised. The key results that emerge from the analysis are:
- A TRI is ideal if all individuals eventually progress permanently out of the TRI-positive state before there is any disease-related mortality. In this case, the TRI-positive category directly corresponds to a useful definition of ‘recently infected’ [16,18], which means that an estimate for the number of recent infections is:
- For a non-ideal TRI (i.e. when some individuals never progress out of the TRI-positive state), it is in principle still possible to estimate the number of individuals in a well-defined ‘recently infected’ state, even though this state is not directly observable in all individuals. If Pnp is the proportion of the HIV-positive individuals who never progress on the TRI under consideration, then an estimate for the number of recent infections is :
When the TRI is ideal, then Pnp = 0, and this formula reduces to the previous expression.
- For all applications (including determination of a trend without regard to the absolute level of incidence), an estimate of Pnp is required.
- To determine the absolute level of incidence, it is also necessary to estimate the mean time spent TRI-positive in the subset of individuals who eventually do progress to become TRI-negative. This quantity, which we denote by ω, is analogous to the duration D in the simple estimator, but differs in the requirement that it should be estimated in the subset of individuals that progress on the TRI.
- As Pnp increases (i.e. a larger fraction of individuals fail to progress on the TRI) and as ω decreases (i.e. individuals spend less time in the TRI-positive state) statistical power is lost. This means that estimates of incidence will have more uncertainty (i.e. wider confidence intervals), and it is less likely that a true change in incidence will be detected.
Previous work by McDougal et al.  used terminology usually employed to characterise the performance of diagnostic tests, such as sensitivity and specificity, to characterise TRI performance. ‘Recent infection’ was defined as being infected for less than a particular time (chosen to be the mean window period). A sensitivity and two specificity parameters were introduced to characterise imperfect classification. No procedure incorporating the effect of parameter uncertainty has thus far been proposed to estimate statistical error or power for the McDougal approach. It has recently been shown that use of sensitivity and specificity parameters is a redundant description of the TRI characteristics [17,19]. In contrast, the new framework defines the condition of being ‘recently infected’ directly in terms of the TRI result. This approach is applicable under less restrictive assumptions, is less prone to bias, and admits an equally informative description of TRI performance using only ω and Pnp .
In deriving the results outlined above, two assumptions were made. Firstly, it was assumed that individuals who do not progress on the TRI have the same survival outcomes as TRI progressors. There is, however, evidence for some TRIs that individuals that fail to progress on the test have a survival advantage. For example, in Baltimore, USA, 60% of elite suppressors (individuals with naturally suppressed virus below 50 copies per ml) failed to progress on the BED assay , and elite suppressors have been observed to survive for longer than others . Secondly, it was assumed that TRI progressors never regress back to the TRI-positive state, but there are indications that this is not true for some TRIs. For example, the rate of misclassification by the BED assay is observed to be higher in individuals with advanced infection  and individuals on ART [22-24]. When these assumptions are true, Pnp is always equal to the proportion of non-recently infected individuals who are classified TRI-positive. When the assumptions are violated, this proportion, or false-recent rate, denoted by ε, varies according to the historic trajectory of the epidemic [17,25]. This would be consistent with the apparently higher BED assay false-recent rate in Uganda  (an older, declining epidemic) than in South Africa  (a younger, growing epidemic) . It is, however, still possible to estimate the number of recent infections by replacing Pnp, in the expression (1) above, with an estimate of ε applicable to the time and place of an incidence survey  (see eAppendix http://www0.sun.ac.za/sacema/publications/eAppendix.pdf for justification). The incidence estimator can then be written as:
The inputs to this estimator are of two types: survey counts (P, N and S), which need to be estimated in every incidence survey, and parameters that describe the characteristics of the TRI (ω and ε), which ideally are estimated in a smaller number of parameter estimation studies.
When ε and ω are known with sufficient accuracy, there are no theoretical reasons why an imperfect TRI should not allow the accurate estimation of incidence. However, two distinct types of practical problems arise – counting error and TRI parameter error. An important component of recent developments is the first consistent analysis of incidence uncertainty accounting for both counting and parameter error (see eAppendix http://www0.sun.ac.za/sacema/publications/eAppendix.pdf for a description of the uncertainty expression). We now illustrate this uncertainty with a somewhat idealised model of the BED assay, which has received much attention and application .
Even in the largest HIV epidemics, infection events are relatively rare (about 2% of the population per year) and ‘recent’ infections (infections in the last 155 days or so, for the BED assay [7,29]) are even less common (about 0.85% in a cross-section of the population). Thus, estimates of incidence are associated with substantial uncertainty since there are few recent infections to be counted. Figure 1 shows the coefficient of variation (CV, which is the ratio of the standard deviation to the estimate) for the estimator (2) calculated under various survey sample sizes and steady-state HIV incidence rates (see eAppendix http://www0.sun.ac.za/sacema/publications/eAppendix.pdf for a description of the uncertainty and steady-state calculations). The TRI parameters (ω and ε) are assumed to be known with absolute certainty. Low values of CV are desirable and indicate that estimates of incidence have small confidence bounds, while high values indicate that incidence estimates will be less certain. For example, in a cross-sectional survey of 5,000 individuals from a population with a steady-state incidence of 2.0 per 100 person years at risk (pyar) the CV is 25.8% – i.e. the 95% likelihood interval for an incidence estimate is 1.0 to 3.0 per 100 pyar.
To explore the ability to detect a change in incidence, a substantial reduction (halving) in incidence is simulated (initially in a steady-state epidemic, with prevalence remaining constant between the two surveys), and a two-tailed test of the null hypothesis that incidence is the same in the two surveys is performed. The possible outcomes are: sustaining the null hypothesis, or concluding that incidence has either increased or decreased. Figure 2 shows the probability of correctly inferring a reduction in incidence, when testing the null hypothesis at a significance level of α=5%. A probability close to 100% indicates that reductions in incidence will be reliably detected, with a probability of less than 90% indicating that results will be unreliable. The South African National Strategic plan for HIV AIDS  has ambitiously set a target of halving incidence between 2007 and 2012. Our calculations suggest that the sample size of each of two surveys (in 2007 and 2012) required to reliably conclude that incidence has decreased, at the 5% significance level, is approximately 25,000.
Figure 1. Uncertainty of incidence point estimates as a result of sample size and background incidence
Figure 2. Probability of correctly inferring a reduction in incidence
TRI parameter error
In the previous section, it was assumed that the correct TRI parameters were known with certainty. The incidence estimates are very sensitive to changes in the values of ω and ε, however, and small differences between the values used in the calculation and the true values can lead to large errors. These parameters have to be estimated in separate studies, usually using cohorts of individuals whose infection time is known approximately. Such cohorts are rare, however, and the numbers of individuals in them are typically small, resulting in substantial uncertainty for the values of ω and ε. In Figure 3 we explore the uncertainty of the estimator (expressed as a CV), as a function of the uncertainty in the TRI parameters. For example, when the BED-like parameters are known with a CV of only 15.0% (reference to a CV for ω relates to the uncertainty of the estimate of ω, not the variation associated with progression times), at a sample size of 5,000 and a steady-state incidence of 2.0 per 100 pyar the CV, as a result of both counting error and parameter uncertainty, is 35.7% – i.e. the 95% likelihood interval for an incidence estimate is 0.6 to 3.4 per 100 pyar.
Since the TRI parameter estimation study may be conducted in a separate population, it is possible to introduce systematic bias if the true values of the TRI parameters vary between populations or over time. The few estimates of ε that have been published vary widely. For example, the false-recent rate is estimated at 1.7% in a South African survey  and 26.7% in Rwanda and Zambia  presumably due to population differences in the historic courses of the epidemics, viral subtypes, host immune profiles, and uptake of antiretroviral therapy. This undermines confidence in the ability to use an estimate for ε obtained in a different population to the one in which incidence is to be estimated, and could contribute to the apparently inflated estimates of incidence reported recently [34,35]. There is also currently no general theoretically unbiased procedure for estimating ε – work on this problem is in progress . In Figure 4 we explore the systematic error in the incidence estimate, expressed as a percentage of the correct value, introduced by systematic errors in the TRI parameters, also expressed as percentages. There is a region in which bias may be small due to cancellation of systematic errors (see the zero error contour).
Figure 3. Uncertainty of incidence point estimates as a result of TRI parameter uncertainty
Figure 4. Systematic error in incidence point estimates as a result of systematic error in TRI parameters
In the short term, reports from early studies using BED should be interpreted with caution , given the substantial uncertainties identified above. Analysis of TRI data should be performed within a more general theoretical framework [16,17], rather than using earlier methods. Most importantly, incidence surveillance should not currently rely on any single methodology, but make use of multiple methods for estimating incidence , such as interpretation of prevalence trends and epidemiological and demographic modelling [3,38].
The search for robust means of estimating incidence from cross-sectional surveys is at a crucial juncture. Although an imperfect TRI can be used to estimate HIV incidence reliably, the reliance on having accurate and precise values of two key aspects of TRI performance (ω and ε) can undermine the use of this technology. The effect of ω and ε on statistical power is shown in Figure 5. While larger values of ω provide sufficient numbers of TRI-positive individuals to ensure statistical power, ω should not be so large that the estimated incidence is not representative of the recent past. On this basis, a value of approximately six months to a year is desirable. It is also essential that ε be small (progress in this regard is being made, for instance using TRIs consisting of an assay in combination with clinical information ). Ideally, to ensure that the fraction of misclassifications is independent of time and epidemic state, inter-individual variability in TRI progression should be unrelated to survival outcomes, and there should be no regression to the TRI-positive state. These form the core requirements for the development of new TRI assays and algorithms used to estimate incidence.
Figure 5. Uncertainty of incidence point estimates as a result of TRI performance
In the next phase of TRI development, it will be essential to be guided by these insights into the key determinants of test performance, and to focus on characterising the performance of the test within a systematic framework.
A Welte and TA McWalter thank the Canadian International Development Agency (CIDA) for funding support. TB Hallett thanks The Wellcome Trust for funding support. This research was partly supported by the Division of Intramural Research, National Institute of Allergy and Infectious Diseases (NIAID), United States National Institutes for Health (NIH). In addition this research was sponsored in part by The HIV Prevention Trials Network (HPTN) sponsored by NIAID, National Institute on Drug Abuse (NIDA), National Institute of Mental Health (NIMH), and Office of AIDS Research, of the NIH, United States Department of Health and Human Services (U01-AI-068613).
- Low-Beer D, Afkhami H, Komatsu R, Banati P, Sempala M, Katz I, et al. Making performance-based funding work for health. PLoS Med. 2007;4(8):e219.
- Hallett TB, Zaba B, Todd J, Lopman B, Mwita W, Biraro S, et al. Estimating incidence from prevalence in generalised HIV epidemics: methods and validation. PLoS Med 2008;5(4):e80.
- Stover J, Johnson P, Zaba B, Zwahlen M, Dabis F, Ekpini RE. The Spectrum projection package: improvements in estimating mortality, ART needs, PMTCT impact and uncertainty bounds. Sex Transm Infect. 2008;84 Suppl 1:i24-i30.
- Brookmeyer R, Quinn TC. Estimation of current Human Immunodeficiency Virus incidence rates from a cross-sectional survey using early diagnostic tests. Am J Epidemiol. 1995;141(2):166-72.
- Janssen RS, Satten GA, Stramer SL, Rawal BD, O'Brien TR, Weiblen BJ, et al. New testing strategy to detect early HIV-1 infection for use in incidence estimates and for clinical and prevention purposes. JAMA. 1998;280(1):42-8.
- Parekh BS, McDougal JS. New approaches for detecting recent HIV-1 infection. AIDS Rev. 2001;3:183-93.
- McDougal JS, Parekh BS, Peterson ML, Branson BM, Dobbs T, Ackers M, et al. Comparison of HIV type 1 incidence observed during longitudinal follow-up with incidence estimated by cross-sectional analysis using the BED capture enzyme immunoassay. AIDS Res Hum Retroviruses. 2006;22(10):945-52.
- Hargrove JW, Humphrey JH, Mutasa K, Parekh BS, McDougal JS, Ntozini R, et al. Improved HIV-1 incidence estimates using the BED capture enzyme immunoassay. AIDS. 2008;22(4):511-8.
- McDougal JS, Pilcher CD, Parekh BS, Gershy-Damet G, Branson BM, Marsh K, et al. Surveillance for HIV-1 incidence using tests for recent infection in resource-constrained countries. AIDS. 2005;19 Suppl 2:S25-30.
- Murphy G, Parry JV. Assays for the detection of recent infections with Human Immunodeficiency Virus Type 1. Euro Surveill. 2008;13(36). pii=18966. Available from: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=18966
- Centers for Disease Control and Prevention (CDC). Interim recommendations for the use of the BED capture enzyme immunoassay for incidence estimation and surveillance. Statement from the Surveillance and Survey and the Laboratory Working Groups to the Office of the Global AIDS Coordinator. CDC. 2006. Available from: http://www.cdc.gov/globalAIDS/docs/surveillance/Interim%20Recommendations%20for%20the%20use%20of%20the%20BED%20capture%20enzyme%20immunoassay%20for%20incidence%20estimation%20and%20surveillance%20Approved%20November%2021%202006%20(2).pdf
- Brookmeyer R. Should biomarker estimates of HIV incidence be adjusted? AIDS. 2009;23(4):485-91.
- Hargrove JW. BED estimates of HIV incidence must be adjusted. AIDS. 2009;23(15):2061-2.
- McDougal JS. BED estimates of HIV incidence must be adjusted. AIDS. 2009;23(15):2064-5.
- Welte A, McWalter TA, Bärnighausen T. Reply to 'Should biomarker estimates of HIV incidence be adjusted?'. AIDS. 2009;23(15):2062-3.
- McWalter TA, Welte A. Relating recent infection prevalence to incidence with a sub-population of assay non-progressors. J Math Biol. 2010;60(5):687-710.
- McWalter TA, Welte A. A comparison of biomarker based incidence estimators. PLoS ONE. 2009;4(10):e7368.
- Kaplan EH, Brookmeyer R. Snapshot estimators of recent HIV incidence rates. Operations Research. 1999;47(1):29-37.
- Welte A, McWalter TA, Bärnighausen T. A simplified formula for inferring HIV incidence from cross-sectional surveys using a test for recent infection. AIDS Res Hum Retroviruses. 2009;25(1):125-6.
- Laeyendecker O, Rothman RE, Henson C, Horne BJ, Ketlogetswe KS, Kraus CK, et al. The effect of viral suppression on cross-sectional incidence testing in the Johns Hopkins Hospital Emergency Department. J Acquir Immune Defic Syndr. 2008;48(2):211-5.
- Hubert JB, Burgard M, Dussaix E, Tamalet C, Deveau C, Le Chenadec J, et al. Natural history of serum HIV-1 RNA levels in 330 patients with a known date of infection. The SEROCO Study Group. AIDS. 2000;14(2):123-31.
- Marinda ET, Hargrove JW, Preiser W, Slabbert H, van Zyl G, Levin J, et al. Significantly diminished long-term specificity of the BED capture enzyme immunoassay among patients with HIV-1 with very low CD4 counts and those on antiretroviral therapy. J Acquir Immune Defic Syndr. 2010;53(4):496-9.
- Hayashida T, Gatanaga H, Tanuma J, Oka S. Effects of low HIV type 1 load and antiretroviral treatment on IgG-capture BED-enzyme immunoassay. AIDS Res Hum Retroviruses. 2008;24(3):495-8.
- Hladik W, Olara D, Were W, Mermin J, Downing R. The effect of antiretroviral treatment on the specificity of the serological BED HIV-1 incidence assay (Abstract 998). The 2007 HIV/AIDS PEPFAR Implementers' Meeting. Kigali, Rwanda, 2007.
- Hallett TB, Ghys P, Bärnighausen T, Yan P, Garnett GP. Errors in ‘BED’-derived estimates of HIV incidence will vary by place, time and age. PLoS One. 2009;4(5):e5720.
- Karita E, Price M, Hunter E, Chomba E, Allen S, Fei L, et al. Investigating the utility of the HIV-1 BED capture enzyme immunoassay using cross-sectional and longitudinal seroconverter specimens from Africa. AIDS. 2007;21(4):403-8.
- Bärnighausen T, Wallrauch C, Welte A, McWalter TA, Mbizana N, Viljoen J, et al. HIV incidence in rural South Africa: comparison of estimates from longitudinal surveillance and cross-sectional cBED assay testing. PLoS One. 2008;3(11):e3640.
- Bärnighausen T, McWalter TA, Rosner Z, Newell M-L, Welte A. HIV incidence estimation using the BED capture enzyme immunoassay: systematic review and sensitivity analysis. Epidemiology. Forthcoming 2010.
- Calypte Biomedical Corporation. Aware(TM) BED(TM) EIA HIV-1 Incidence Test (IgG-Capture HIV-EIA). Enzyme Immunoassay for Population Estimates of HIV-1 Incidence. Cat No. 98003. Portland, OR, USA, 2008.
- South African Department of Health [Internet]. Cape Town. Ministry of Health. Department of Health. HIV and AIDS and STI Strategic Plan for South Africa, 2007-2011. [Accessed: 16 June 2010]. Available from: http://www.doh.gov.za/docs/misc/stratplan-f.html
- Ghys PD, Zaba B, Prins M. Survival and mortality of people infected with HIV in low and middle income countries: results from the extended ALPHA network. AIDS. 2007;21 Suppl 6:S1-4.
- Laeyendecker O, Oliver A, Gamiel J, Neal J, Kraus C, Eshleman S, et al. Decreasing HIV incidence and prevalence at the Johns Hopkins Emergency Department with a concurrent increase of virally suppressed HIV-infected individuals (Poster # 1045). Conference on Retrovirus and Opportunistic Infection (CROI), 2009.
- Todd J, Glynn JR, Marston M, Lutalo T, Biraro S, Mwita W, et al. Time from HIV seroconversion to death: a collaborative analysis of eight studies in six low and middle-income countries before highly active antiretroviral therapy. AIDS. 2007;21(Suppl 6):S55-S63.
- Mermin J, Musinguzi J, Opio A, Kirungi W, Ekwaru JP, Hladik W, et al. Risk factors for recent HIV infection in Uganda. JAMA. 2008;300(5):540-9.
- National AIDS and STI Control Programme, Ministry of Health - Kenya. Kenya AIDS Indicator Survey, 2007: Preliminary Report; KAIS presentation, 7th October, George Washington University, Washington D.C., USA. 2008.
- McWalter TA, Kassanjee R, Welte A. Incidence from cross-sectional surveys: Improved characterization of tests for recent infection. XVIII International AIDS Conference. Vienna, Austria, 2010; (Abstract CDC0473). Forthcoming.
- Ghys PD, Kufa E, George MV. Measuring trends in prevalence and incidence of HIV infection in countries with generalised epidemics. Sex Transm Infect. 2006;82(suppl_1):i52-6.
- Hallett TB, Gregson S, Gonese E, Mugurungi O, Garnett GP. Assessing evidence for behaviour change affecting the course of HIV epidemics: A new mathematical modelling approach and application to data from Zimbabwe. Epidemics. 2009;1(2):108-17
- Laeyendecker O. Session II: Measuring new HIV infections; where we are with new technologies and approaches measuring HIV incidence. BED + avidity testing algorithm for incidence estimates in Uganda. The 2nd Global HIV/AIDS Surveillance Meeting. Bangkok, Thailand, 2009. Available from: http://www.hivsurveillance2009.org/docs/session_ii/pres4.ppt