Research articles Evaluation of a risk assessment questionnaire to assist hepatitis C screening in the general population

Many individuals with hepatitis C virus (HCV) infection are undiagnosed. This study evaluates a risk assessment questionnaire, developed for use online to target blood-screening for HCV. Two hundred and eighty-nine patients with known HCV status completed a written questionnaire on prominent HCV risk factors. Questionnaires generated advice to seek testing if at least one risk factor was reported. Agreement of the testing advice with the HCV status of respondents was evaluated. Subsequently, we validated our questionnaire among 985 patients of an outpatient clinic for sexually transmitted infections. The post-test-probability-of-disease (PTPD) and diagnostic gain (PTPD minus prior probability of disease) were calculated. The questionnaire's sensitivity and specificity were 84.6% and 63.8%, respectively, and higher in the STI clinic patients. The PTPD of positive testing advice was 72.5% given HCV prevalence of 53.0%, yielding a diagnostic gain of 19.5%. Applying the estimated prevalence in the general Dutch population (0.1-0.4%), and the anticipated prevalence in the online project (1.0-6.0%), yielded diagnostic gains of 0.13-0.53% and 1.3-7.0%, respectively. We conclude that our questionnaire succeeded in selecting at-risk individuals as its testing advice agreed well with the HCV status. We suggest that the questionnaire be used online as a selection tool for HCV blood-screening in the general population.


Introduction
Hepatitis C virus (HCV) infection, first identified in 1989, is caused by a bloodborne virus and affects an estimated 120 million individuals worldwide [1]. Almost 75% of HCV infections become chronic [2].
Twenty to 30 years after infection, chronic HCV leads to liver cirrhosis in 20%-30% of patients, 2%-5% of whom each year will progress to liver failure or liver carcinoma [3]. Since the onset of the infection itself and the development of cirrhosis in chronically infected patients are usually asymptomatic [3,4], many cases are undetected. Earlier diagnosis of HCV enables patients to start timely treatment, adopt a healthy lifestyle (e.g., avoiding alcohol [5]), and prevent possible transmission to others. Treatment options for HCV have improved substantially since 2001 [5,6], and the Dutch Health Council has recommended that more education and tracing be focused on groups at risk for HCV infection [7]. In most western European countries, the prevalence of HCV infection is low, estimated at 0.1%-0.4% in the Netherlands [8], 0.8% in France [9], and 0.6%-1.1% in the UK [10]. For low-prevalence countries, it is worth considering whether selective screening (i.e., establishing individual risk for HCV infection as a condition for screening) may be more cost-effective than mass screening (i.e. every inhabitant is advised to test for HCV) [11,12]. Therefore, as a pilot project in the Netherlands, an HCV internet programme was set up to identify HCV-infected individuals in the general population by testing individuals at risk for HCV. The programme's strategy consists of a public media campaign to refer individuals from the general population who are potentially at risk of HCV, to an online interactive risk assessment questionnaire at www.heptest. nl. The questionnaire determines whether or not individuals are at risk for HCV and offers an opportunity for anonymous blood testing, free of charge.
This study describes the development and evaluation of the HCV risk assessment questionnaire before its use online. We determined the questionnaire's discriminative value for diagnosing HCV. Furthermore, we evaluated its relevance in clinical practice. This paper discusses implications for use of the questionnaire online.

Development of the HCV risk assessment questionnaire
The questionnaire was developed in three stages. Firstly, the content was developed. Secondly, the questions were formulated and tested on members of the public for comprehensibility. This resulted in a core questionnaire, which was sent out to patients for the evaluation study. Meanwhile, however, new data on risk factors had become available. Thirdly, therefore, an extended questionnaire was developed. The following paragraphs describe these three stages in the developmental process.

Content
Development of the core questionnaire was based on a literature review of risk factors for HCV, followed by a meeting of experts, in which the risk factors from the literature were discussed for inclusion in the questionnaire. The expert group consisted of eight health care professionals (professor in hepatology, professor in epidemiology and prevention of infectious diseases, senior epidemiologist, two medical doctors who specialised in infectious diseases and public health, coordinator of the National Hepatitis Centre, senior social psychologist specialising in online research, health communication expert). The expert group decided to include risk factors/groups either if the expected prevalence in the specific group was considered to be high (e.g. injecting drug users (IDUs)) or if not informing a specific group was considered to be unethical (e.g. individuals who were administered blood products before 1992 as these individuals have never been informed in the Netherlands and have the right to know about their risk). Some risk factors described in literature (e.g. dental care [13]) were not included, or included only when they occurred in countries with a medium to high Being born in a HCV-endemic country HCV endemic countries: Egypt (18%), Bolivia (11%), Rwanda (17%), Burundi (11%), Cameroon (13%), Guinea (11%), Mongolia (11%) [16] Having received blood (products) before 1992 0.02%-0.2% [8] HCV-infected mother Mother HIV-neg: ~4% Mother HIV-pos: ~20% [17] Mother is/was IDU Prevalence may be slightly lower than the above (4%-20%) as the HCV prevalence among IDU is high but not 100% Living together for >1 year and sharing bathroom items with HCV-infected individuals 0%-11% [15] Living together for >1 year and sharing bathroom items with IDU Prevalence may be slightly lower than the above (0%-11%), as the HCV prevalence among IDU is high but not 100% Needlestick injury: needle exposed to high-risk person (IDU, haemophiliac, dialysis patient, HCV-infected individual) Prevalence unknown. Transmission rate with HCV-contaminated needle: 1%-10% [18,19] Needlestick injury in HCV-endemic country Prevalence data of HCV-endemic countries: see above. Transmission rate with HCV-contaminated needle: 1%-10% [18,19] Haemophilia patient~70% [20,21] Haemodialysis patient 2.6%-22.9% [22] Organ recipient Prevalence unknown prevalence of HCV infection, as including these risks would be tantamount to advising almost everyone to be tested for HCV, yielding low discriminative power to the questionnaire. The experts reached consensus for all risk factors . The upper panel of table 1 shows the risk factors selected for inclusion in the core questionnaire, and the prevalence of HCV infection associated with each risk factor. For study purposes, we also included questions on demographics (age, sex, educational level) and whether or not individuals were infected with hepatitis B virus (HBV).

Pre-testing
To improve its comprehensibility, the core questionnaire was pre-tested on 20 people (11 male) recruited at a popular Amsterdam street market that attracts a demographically diverse population and at the liver outpatient clinic of the Academic Medical Center of Amsterdam. All questions were read by the participants, and comprehension was examined by asking them to comment if they did not fully understand any detail. If concepts thought likely to be difficult were not queried by a participant, the interviewer asked him/her to describe their meaning. Terminology found difficult to comprehend was altered according to suggestions by participants. After pre-testing, the core questionnaire was ready for evaluation.

Development of the extended HCV risk assessment questionnaire
After the initial development of the core questionnaire, data were published that indicated a relatively high prevalence of HCV infection in non-injecting illicit drug users (NIDU) and HIV-infected patients [25,26]. We therefore extended the core questionnaire with these two risk factors. Furthermore, in this extended questionnaire, we asked patients how they thought they had become infected, seeking risks for HCV infection that were not covered by the core questionnaire. The lower panel of table 1 shows the risk factors that were added in the extended questionnaire.

Evaluation study
To evaluate both the core and the extended questionnaire, individuals whose HCV infection status was known (i.e. liver disease patients) were approached and asked to fill out the questionnaire. Firstly, the sensitivity and specificity of both the core and the extended questionnaire were determined. Secondly, clinical relevance was evaluated by determining the diagnostic gain (i.e. the improvement in knowledge/certainty as to whether or not an individual was infected with HCV, resulting from the use of the questionnaire). Thirdly, a validation study was performed using data from patients attending a clinic for sexually transmitted infections (STI).

Recruitment of the liver disease patients
Between October 2006 and October 2007, Dutch speaking patients suffering from liver-related diseases (such as HCV or HBV infection) were recruited at various locations. These people were selected because they were presumed to have been tested for HCV and to know their HCV status.
From October 2006 to June 2007 the core questionnaire was distributed at two liver outpatient clinics in Amsterdam, and was handed out during the National Hepatitis Week's patient symposium 2007. From July to October 2007 the extended questionnaire was sent to 459 members of the Dutch liver patient organisation (Nederlandse Leverpatiëntenvereniging), with an explanation about the evaluation study and a request to cooperate by filling out and returning the questionnaires by post.

Validation study in STI clinic patients
In order to validate the questionnaire in a population more representative of the general Dutch population with respect to liver disease prevalence, data from an anonymous survey conducted from April to May 2007 among 985 patients at the outpatient clinic for STI of the Public Health Service of Amsterdam were used retrospectively. This survey collected detailed data about sexual risk behaviour and risk factors for HCV and blood tests for HIV, HCV, and other STI. HCV antibody screening was performed by means of a third-generation commercial microparticle EIA system (AxSym HCV version 3.0), and positive test results were confirmed by Immunoblot (Chiron RIBA HCV 3.0 SIA). The prevalence of HCV infection among the STI clinic patients was 1.0%. The data collected on HCV risk factors were used to assess whether an individual would have been advised to test for HCV according to the extended risk assessment questionnaire.

Statistical methods
All participants who reported at least one risk factor were advised to be tested for HCV infection ('positive testing advice'; PTA), and those who reported no risk factors were advised that testing was unnecessary ('negative testing advice'; NTA). Where answers to questions were missing or inconclusive (i.e., the answer 'don't know'), we assumed that the risk was not present. Differences in risk factor prevalence between the HCV-positive and the HCV-negative group were tested using Pearson chi-square test or, when numbers were small, Fisher's Exact two-tailed test. For testing differences in age, the Mann-Whitney-U test was used. We calculated Likelihood Ratio-based 95% confidence intervals (CI) for sensitivity and specificity.
To examine whether sensitivity and specificity differed with sex and age, we performed stratified sensitivity and specificity analyses for sex and age (≤50 and >50 years, cut-off based on median age). Furthermore, we performed two multivariate logistic regression analyses, separately for HCV positives and for HCV negatives/unknown, with sex and age (continuous variable) as predictors of testing advice (outcome variable).
The sensitivity of the core and extended questionnaires -i.e., the percentage of HCV-positive patients being correctly identified as HCV-positive -was calculated as True PTA/(True PTA+False NTA). The specificity -i.e. the percentage of HCV-negative patients being correctly identified as HCV negative -was calculated as True NTA/(True NTA+False PTA).
For the validation study, we calculated sensitivity and specificity of the extended questionnaire in the STI clinic patients. Some minor risk details had not been questioned in the STI clinic survey (e.g. living together for >1 year and sharing bathroom items with HCV-infected individuals or IDU). Data from the liver disease patients were restricted to the same risk factors to calculate a comparable sensitivity and specificity. Differences between sensitivity and specificity from liver disease patients and STI clinic patients were evaluated using Newcombe's method 10 for independent proportions [27].
Sensitivity and specificity represent the diagnostic accuracy of a screening questionnaire, but they do not reflect the individual likelihood of disease associated with a certain questionnaire result and are therefore less useful in clinical practice. The clinical relevance of the questionnaire was assessed by calculating the post-test probability of disease (PTPD; i.e. the likelihood of being HCV-positive when given a positive or negative HCV testing advice [28]) using the formulas: As the PTPD depends largely on the pre-test probability of disease (i.e. the HCV prevalence in the population), Fagan's nomogram [29] was used to visualise the diagnostic gain after a PTA. This graphical calculation of Bayes' theorem describes how the result of a test (positive or negative) changes the perception of disease probability by combining the pre-test probability of disease with the likelihood ratio of the test (which is calculated from sensitivity and specificity) [28]. Fagan's nomogram converts pre-test probabilities into pre-test odds, then multiplies the odds by the likelihood ratios and converts post-test odds back to posttest probabilities. The PTPD was plotted for a range of HCV prevalences, including the prevalence in the liver disease patients, the estimated prevalence for the general Dutch population, and the prevalence expected to be revealed by the HCV internet programme.

Results
At the liver outpatient clinics, 99 patients filled out the core questionnaires anonymously while waiting for their consultation. In addition, 20 visitors at the National Hepatitis Week's patient symposium 2007 took part. Data on non-response for these two groups were not collected. Of the 459 members of the Dutch Liver Patient Organisation to whom the extended questionnaire was sent, 249 (54%) responded; 72 returned blank questionnaires, (some said they had not been tested for HCV and therefore could not participate; some did not want to); and 177 were willing to cooperate, yielding a response rate of 39% (177/459). In total, 296 patients took part: 99 and 20 filled out the core questionnaire (total 119), and 177 responded to the extended questionnaire.
One hundred and thirty-eight of the 296 participants (47%) reported that they were HCV-positive, 132 (45%) said they were HCV-negative, and 19 (6%) were unaware of their HCV status. An additional 7 (2%) did not give their HCV status and were therefore excluded, leaving 289 liver disease patients. Those unaware of their HCV status were assumed to be HCV-negative. Table 2 shows characteristics and HCV risk factors of the liver disease patients by HCV status. As expected, prevalence of IDU, having received blood products before 1992, living together for >1 year and sharing bathroom items with HCV-infected individuals or IDU, having experienced a needlestick injury from a needle exposed to a high risk person, and non-injecting illicit drug use on regular basis were significantly higher among HCV-positives than among HCV-negatives.
Being an organ recipient achieved borderline significance in the opposite direction (p=0.05). Prevalence of other risk factors did not differ significantly between HCV-positives and HCV-negatives, but the numbers of individuals with these exposures were often very small.  In the extended questionnaire, HCV-positive patients were asked to describe their perceived route of infection. Fourteen HCV-positive participants (15.4%) had reported no risks and were therefore assigned to NTA. Nine of these 14 did not know how they acquired HCV; four presumed they had been infected due to:  Post-test probability (%) Pre-test probability (%) dentistry, vaccination during military service, health care work without gloves, and travel vaccination in the mid 1970s. One HCV infection was officially recognised as occupational, resulting from police work related to traffic accidents.

Post-test probability of disease
The post-test probability of disease (PTPD) was calculated using sensitivity and specificity of the extended HCV risk assessment questionnaire in the liver disease patients. Fagan's nomogram ( figure) shows the PTPD of a PTA and gives a precise view of diagnostic gain, specifically for low-prevalence populations. The line that starts at the left y-axis shows the pre-test probability of disease (i.e. the HCV prevalence), crosses the likelihood ratio for PTA (+LR, i.e. sensitivity/(1-specificity)), then points to the post-test probability of disease at the right y-axis. The diagnostic gain is the difference between the chance of disease for an individual before filling out the questionnaire (i.e. the prevalence) and the chance of disease for an individual after being assigned to PTA according to the questionnaire (i.e. the PTPD). For example, the diagnostic gain after PTA in the liver disease patients with a prevalence of 53.0% (n=171) is 19.5% (72.5% minus 53.0%), as shown by the striped line.
For the estimated prevalence in the general Dutch population (0.1%-0.4% [8]), the PTPD of a PTA is 0.23% to 0.93% (see dotted lines). The diagnostic gain varies from 0.13% (0.23% minus 0.1%) to 0.53% (0.93% minus 0.4%). In the HCV Internet programme, the media campaign, targeted at the general population, addresses risk factors for HCV and aims to refer those potentially at risk to the questionnaire. Therefore, we anticipate a prevalence of 1.0% to 6.0% in the population filling out the online questionnaire, yielding a PTPD of a PTA of 2.3% to 13.0% (vertically hatched area), which would lead to a diagnostic gain of 1.3% to 7.0%.

Discussion
Sensitivity was relatively high in this study. The HCV risk assessment questionnaire identified 84.3% of the HCV-infected individuals, and almost 90% when patients with missing values were excluded from analyses. In the STI clinic patients both sensitivity and specificity reached almost 90%. The fact that the risk assessment was based on self-reported risk factors, relying on the participant's memory instead of biological markers, strengthened the findings. Of the 14 HCV-infected individuals not identified by the questionnaire, only one mentioned a confirmed transmission route (police work related to traffic accidents). The others either did not know the route or mentioned various possibilities, such as dentistry, vaccinations, and health care work without gloves. Although all these possibilities include blood-blood contact and therefore could be sources of HCV infection, their probability of transmitting infection in low prevalence areas is likely to be very low. Furthermore, adding such risk factors to the questionnaire would decrease its discriminative value as it would lead to almost everyone in the Netherlands (or other low prevalence areas) being advised to seek testing.
The extended questionnaire performed better than the core questionnaire. It includes HIV as a risk factor for HCV. Recently, outbreaks of sexually acquired HCV infection have been reported among HIV-infected men who have sex with men [25]. Based largely on case studies, sexually-acquired HCV infection has been associated with HIV infection, the presence of ulcerative sexually transmitted diseases (STD), sexual practices that cause mucosal damage, and sex under the influence of drugs [25]. As the prevalence of HCV infection among HIV-infected individuals is high, partly because of shared bloodborne routes, and HCV/HIV coinfection accelerates HCV disease progression [30;31], HIV infection should be included in a HCV risk assessment questionnaire.
A few other studies have used or evaluated a risk assessment questionnaire for HCV infection [32][33][34][35]. For example, Lapane et al. found sensitivity and specificity of 69% and 74%, respectively, for risk factor based screening using a questionnaire including socially intrusive questions (e.g. IDU). Using this model, the costs per case detected were lower than when a questionnaire was used omitting socially intrusive questions, or when screening was based on elevated alanine transaminase levels [32]. However not all studies evaluated sensitivity, specificity, and feasibility in clinical practice. The feasibility of a prescreening selection questionnaire, as opposed to mass screening, requires a balance between sensitivity and specificity, to ensure validity of the advice, diagnostic value, and cost-effectiveness of the selection method. The diagnostic value is largely dependent upon the disease prevalence. When the estimated prevalence in the general Dutch population (0.1%-0.4%) was used as a pre-test probability of disease, PTPD after PTA more than doubled but still remained small. This means that a large proportion of those who receive PTA will test HCV-negative, because of the relatively low risk of HCV infection associated with risk factors such as having received a blood transfusion. Nevertheless, false NTA is more problematic than false PTA because of the potentially severe long-term consequences of HCV infection. On the other hand, a large proportion of HCV-negative individuals receive NTA and avoid the invasive and costly blood-screening procedure.
The following scenario illustrates the diagnostic value of the risk assessment questionnaire. If there is a population of 100,000 individuals, 2,000 of whom have HCV infection (prevalence 2.0%) and the aim is to trace them, one could simply test everyone, yielding one infected individual per 50 tested. Using a pre-screening selection questionnaire, however, 37,266 (84.6% of 2,000 HCV-infected plus 36.3% of 98,000 HCVnegative) individuals would be tested for HCV antibodies to trace 1,692 infected individuals, yielding a ratio of 1:22 instead of 1:50. Three hundred and eight (15.4% of 2,000) HCV-infected individuals would not be tested and therefore not traced, but 62,524 (63.8% of 98,000) HCV-negative individuals would not have to undergo testing. As the validation study showed a higher specificity in non-liver disease patients, the number of screened HCV-negative individuals may decrease when the questionnaire is applied to the general population.
Online use of the risk assessment questionnaire in the HCV internet programme appears feasible, and may be more cost-effective than other screening strategies, such as mass screening. Firstly, as the internet programme's public media campaign and website information will address risk factors (e.g. receiving a blood transfusion), the online questionnaire will be likely to attract groups at increased risk of HCV infection in the general population, leading to a higher PTPD after PTA. Secondly, the possible anxiety of HCV-negative participants who are concerned about their potential risk of HCV infection could be reduced by incorporating an internet-mediated, low-threshold, anonymous blood testing procedure (i.e. a service in which individuals print their laboratory forms from the website, visit a laboratory for blood sampling, and obtain their blood test results online). Thirdly, internet-mediated blood testing may reduce health care costs (e.g. GP consultations).
The internet may provide easy availability and anonymity, but certain factors must be considered when using the internet for offering an HCV risk assessment. Firstly, although internet uptake is high in the Netherlands, not all individuals have access to it or possess sufficient literacy or skills to use it. Secondly, it is a challenge to attract individuals to a website. Developing an HCV screening programme through the internet without marketing it properly would probably fail to identify HCV-infected individuals.
Our study has several limitations. We used selfreported HCV status of the liver disease patients to calculate sensitivity and specificity. Although unlikely in this population, it could be that some patients did not report their true HCV status. We did not collect data on non-response for the liver disease patients at the hospitals and at the symposium and were thus unable to evaluate whether selection bias had occurred. Our validation study made use of previously collected survey data. We cannot exclude the possibility that individuals who fill out a risk assessment questionnaire knowing its purpose (like the liver disease patients in our study) recall relevant information differently from those who take part in a survey without knowing why the data are being collected. A potential difference might result in an under-or overestimation of the sensitivity and specificity in our validation study. In general, we do not know whether our study population is representative for the population as a whole.
In conclusion, although our study population might not be representative for the population as a whole, the questionnaire's validity is high, as the testing advice agrees well with the HCV status in this study. The diagnostic gain, however, depends largely on HCV prevalence and is therefore lower when the questionnaire is used in low-prevalence populations.
We encourage the use of our questionnaire, especially in European countries where the prevalence is somewhat higher than in the Netherlands. A future study should assess the cost-effectiveness of a risk-based screening strategy in internet-based and alternative programmes compared with other strategies, such as mass screening or screening of easy-to-target-risk groups only (e.g. drug users who participate in care programmes). The cost-effectiveness analysis should take into account not only the prevention of future health care costs of identified HCV-infected individuals but also the health care costs associated with HCVinfected individuals who would not be detected using one of these screening strategies.