Skip to main content
Download PDF

Development and validation of a risk score to assist screening for acute HIV-1 infection among men who have sex with men

BMC Infectious Diseases volume 17, Article number: 425 (2017) Cite this article

A Correction to this article was published on 24 October 2017

This article has been updated

Abstract

Background

Early treatment of acute HIV-1 infection (AHI) is beneficial for patients and could reduce onward transmission. However, guidelines on whom to test for AHI with HIV-1 RNA testing are lacking.

Methods

A risk score for possible AHI based on literature and expert opinion – including symptoms associated with AHI and early HIV-1 – was evaluated using data from the Amsterdam Cohort Studies among men who have sex with men (MSM). Subsequently, we optimized the risk score by constructing two multivariable logistic regression models: one including only symptoms and one combining symptoms with known risk factors for HIV-1 seroconversion, using generalized estimating equations. Several risk scores were generated from these models and the optimal risk score was validated using data from the Multicenter AIDS Cohort Study.

Results

Using data from 1562 MSM with 175 HIV-1 seroconversion visits and 17,271 seronegative visits in the Amsterdam Cohort Studies, the optimal risk score included four symptoms (oral thrush, fever, lymphadenopathy, weight loss) and three risk factors (self-reported gonorrhea, receptive condomless anal intercourse, more than five sexual partners, all in the preceding six months) and yielded an AUC of 0.82. Sensitivity was 76.3% and specificity 76.3%. Validation in the Multicenter AIDS Cohort Study resulted in an AUC of 0.78, sensitivity of 56.2% and specificity of 88.8%.

Conclusions

The optimal risk score had good overall performance in the Amsterdam Cohort Studies and performed comparable (but showed lower sensitivity) in the validation study. Screening for AHI with four symptoms and three risk factors would increase the efficiency of AHI testing and potentially enhance early diagnosis and immediate treatment.

Peer Review reports

Background

Acute HIV-1 infection is the phase of HIV-1 disease immediately after infection that is characterized by an initial burst of viremia; although anti-HIV-1 antibodies are undetectable, HIV-1 RNA or p24 antigen are present [1, 2]. Identifying patients with AHI is important for three reasons. First, patients benefit from immediate start of combination antiretroviral therapy (cART) during the early phase of infection [3,4,5]. Second, starting cART soon after infection could have a significant impact on the ongoing HIV-1 epidemic [6,7,8,9]. Very high viral loads during AHI and continued high-risk behavior before diagnosis are major determinants of transmission [1, 10, 11]. Third, patients who start cART during AHI may offer insight into the potential for post treatment control of HIV-1 infection [12,13,14,15].

The US Centers for Disease Control & Prevention included HIV-1 RNA testing in their guidelines for detecting AHI [16]. The recent development of point-of-care HIV-1 RNA tests have made the prompt diagnosis of AHI possible at the point-of-care [17] and offers a unique opportunity to increase the number of HIV-1 infections diagnosed during the acute phase, potentially even before p24 antigen is detectable. However, these tests are expensive and guidelines on whom to test for AHI are lacking.

Criteria for testing might include symptoms of AHI. Forty to 90% of patients with AHI experience symptoms, but these symptoms are transient and often nonspecific [2, 18, 19]. Timely recognition of individuals in the early weeks of infection with HIV-1 RNA or p24 antigen tests may change the course of infection [20]. A suitable screening tool used to select patients for AHI testing would restrict testing to a limited number of individuals, while detecting a large proportion of all AHI cases. Recent studies from sub-Saharan Africa reported the development of risk scores for AHI screening [21,22,23] with good performance among men who have sex with men (MSM) [21]. It remains to be elucidated whether risk scores developed in recourse-limited settings could be translated into a different context, where different risk behavior and HIV-1 subtype may influence factors included in the risk scores [21, 24]. Risk scores for AHI detection in high-income countries have been previously developed [25, 26], but a risk score for use for the symptomatic patient at the point-of-care is lacking.

As part of a recently launched initiative aiming to curb the epidemic at the city level, the HIV Transmission Elimination AMsterdam initiative (H-TEAM) [27], an AHI testing strategy was developed in order to identify individuals with AHI and start immediate treatment. This strategy included a risk score for possible AHI that weighs symptoms in combination with sexual risk behavior. This risk score was implemented as an interactive online screening tool for MSM through a dedicated website [28], used by MSM and health providers at the sexually transmitted infections (STI) clinic of the Public Health Service of Amsterdam (GGD). In the present study, we aimed to assess whether such a risk score could be useful for AHI screening in a public health setting. We aimed to evaluate the performance of the AHI risk score using data from the Amsterdam Cohort Studies (ACS) and to assess whether the H-TEAM risk score could be optimized in the Amsterdam setting. We validated the optimal risk score using data from the Multicenter AIDS Cohort Study (MACS) in the USA.

Methods

Initial development of a risk score for AHI

A recent meta-analysis reviewed 21 studies on acute and early HIV-1 infection and showed that seventeen symptoms and seven signs were associated with acute or early HIV-1 infection [18]. The symptoms are summarized in Table 1. A pragmatic risk score for possible AHI was developed based on peer-reviewed literature and comprehensive consultations with experts. This resulted in an individual risk score for MSM which is equal to the sum of scores for a list of seventeen symptoms, whereby presence of a symptom either yields a score of one (fatigue, genital warts, headache, myalgia, nausea, night sweats, pharyngitis, or rash) or a score of two (arthralgia, diarrhea, fever, genital ulcers, lymphadenopathy, oral thrush, oral ulcers, vomiting, or weight loss). These scores were based on the positive likelihood ratio for each symptom in association with acute or early HIV-1 infection, as reported by Wood et al. [18]. We translated this list of symptoms into a self-administered online screening tool by which patients could refer themselves to the STI clinic of the Public Health Service of Amsterdam (GGD) for AHI testing, including a point-of-care HIV-1 RNA test. The initial pragmatic risk score (risk score A) was: possible AHI is present if an individual reports condomless anal intercourse (CLAI) in the preceding three months and has a risk score of at least two using the above mentioned symptom list.

Table 1 Symptoms associated with AHI and questions in ACS and MACS
Full size table

Performance of the initial risk score in Amsterdam cohort studies

To validate our initial risk score (risk score A) we used data from the ACS. This is an open, prospective cohort study among MSM in Amsterdam, initiated in 1984 aiming to investigate the epidemiology, natural history, and pathogenesis of HIV-1, and to evaluate the effect of interventions [29]. Until 1995, men of all age groups were eligible to participate if they had had at least two male sexual partners in the preceding six months. From 1995 to 2004, men aged thirty or less with at least one male sexual partner in the preceding six months could enter the study. From 2005 onwards, men of all age groups with at least one male sexual partner in the preceding six months could participate. Participation is voluntary, and written informed consent is obtained from every participant at enrollment.

Study participants visited the GGD every six months to complete a self-administered questionnaire on risk behavior in the preceding six months [30]. A face-to-face standardized interview with a study nurse captured symptoms experienced since last study visit. All except four of the symptoms in our initial risk score were included in the standardized interview; there was no question on arthralgia, and questions on oral ulcers and genital warts were only included in the interview from 2006 onwards. Furthermore, there was no question on pharyngitis, but sore throat was included. Physical examination was not performed. Each visit, participants provided a blood sample for HIV-1 testing by consecutive generations of commercially available screening assays to detect HIV-1 antibodies and for storage. HIV-1 test results were confirmed by Western blot analysis. A seroconversion visit was defined as the visit with the first positive HIV-1 test result after a visit with an HIV-1 seronegative result. For this study, we included MSM who were HIV-1 negative at study entry and had had at least one follow-up visit between October 1984 and December 2009. Any visit with an interval of more than twelve months following the preceding visit was excluded from the analyses.

Points were assigned to each of the symptoms included in the ACS questionnaire according to the scoring procedure of the initial risk score (A). Oral ulcers and genital warts were excluded from the analyses due to large numbers of missing data (questions only asked from 2006 onwards). In total, fourteen symptoms were included in the analysis of this risk score. Reported CLAI was a condition in this risk score and a pre-specified cutoff of two was used, both consistent with the initial risk score (A).

Optimizing the risk score

We aimed to improve the initial risk score by analyzing the association of symptoms with HIV-1 seroconversion in the ACS. Analysis was done with logistic regression using generalized estimating equations, with an exchangeable covariance matrix and robust standard errors to adjust for multiple observations per participant. We calculated odds ratios (OR) for HIV-1 seroconversion for each symptom in our initial risk score, using univariable analysis. A multivariable logistic regression model including the fourteen symptoms was constructed. We created a risk score in which each symptom received points equal to its beta coefficient (β, natural log of the adjusted odds ratio) in the multivariable model, rounded to the nearest decimal [23]. CLAI was a condition in this risk score (risk score B), consistent with the initial risk score. Because not all seroconverters reported CLAI, we evaluated performance of this risk score also without CLAI as a condition (risk score C).

Next, we developed a risk score in which we combined symptoms with risk factors reported at a seroconversion visit. An earlier analysis of the ACS data found that self-reported gonorrhea, receptive CLAI, more than five sexual partners (all in the preceding six months) and lower educational level were associated with HIV-1 seroconversion in this study population [30]. After consultation with clinical experts we decided not to incorporate lower educational level in the risk score, as this was considered inappropriate. The three remaining risk factors were included in a multivariable model, thus including fourteen symptoms and three risk factors, and a risk score with all variables scored as the β rounded to the nearest decimal was generated (risk score D). In order to simplify the risk score, performance of a risk score including only variables that were statistically significant (p < 0.05) in the combined symptom and risk factor multivariable model was also evaluated (risk score E) [23].

We compared seroconversion visits with seronegative visits to evaluate the performance of all risk scores generated. We used receiver operating characteristics (ROC) curves to calculate the area under the curve (AUC). To evaluate sensitivity and specificity at a consistent cutoff for all risk scores, the Youden-index [31] was calculated for each risk score, and sensitivity and specificity were defined at this cutoff. The optimal risk score was defined as good overall performance (i.e. high AUC) and suitable to use as a screening tool (i.e. a limited number of symptoms and risk factors).

To study differences in symptoms and risk factors by age and over time we categorized age at time of the study visit (≤24, 25–34, 35–44, and ≥45 years of age) and calendar year of visit (<1990, 1990–1994, 1995–1999, 2000–2004, 2005–2009). We added these categorical age and calendar year variables to the multivariable model including fourteen symptoms and three risk factors. The HIV-1 epidemic was over its peak after 1990 [30], therefore we performed a sensitivity analysis by including only visits from 1990 onwards in the multivariable model. In another sensitivity analysis, we included educational level in the multivariable model.

It is possible that infection occurred at the visit before the seroconversion visit, therefore we explored the prevalence of reported symptoms at the last HIV-1 negative visit prior to seroconversion. Lastly, we evaluated the interval between the seroconversion visit and the previous visit for all seroconverters.

The ACS dataset had missing values for several variables, varying between zero and 2546 out of a total of 17,446 observations. For six variables no data were missing, for fourteen variables less than 1% of records had missing data, and for four variables data were missing from more records, the largest missing number being 2546 for the variable gonorrhea. We used the last observation carried forward method to impute missing values. We only imputed values of a variable if the value of the variable of the visit immediately preceding was available. We also conducted complete case analyses using only records without missing values.

Validation of the risk score

In order to validate the optimal risk score data from the MACS were used. The MACS is an ongoing prospective study in homosexual and bisexual men located in Baltimore, Chicago, Pittsburgh and Los Angeles (USA) [32]. Study visits were scheduled every six months. We used the Public Data Sets containing data from 1984 to October 2010 (version P23, released on 15 October 2014 [33]). Participants who were HIV-1 negative at baseline and had at least one follow-up visit were included in the analyses. Participants who seroconverted for HIV-1 infection during follow-up, but did not have a registered seroconversion visit or date of seroconversion (n = 39) were excluded from the analyses. The MACS Public Data Sets did not include date of visit, so visits were excluded when a participant had skipped two or more previous visits (any visit above twelve months following the previous visit, assuming that the protocol of semiannual visits was precisely followed). The remaining dataset, including 63,618 visits, contained large numbers of missing values for all variables, in part because the questionnaires varied by study visit: questions on fever (5883 missings) and lymphadenopathy (5881 missings) were asked from 1986 onwards, oral thrush (24,013 missings) was asked from 1990 onwards, and weight loss (525 missings), gonorrhea (562 missings), number of male sex partners (8850 missings) and receptive CLAI (21,344 missings) were included in the questionnaire through all years. We used the last observation carried forward method to impute missing values, but only if the value of the variable of the visit immediately preceding was available.

Post-test probability of disease

To evaluate the clinical relevance of the optimal risk score, the post-test probability of disease (PTPD; i.e. the likelihood of having AHI when having a score above the risk score cutoff [34]) was estimated. Prevalence of AHI has not been evaluated in the Netherlands, therefore AHI prevalence from a recent study in the USA among sexually active MSM was used (AHI prevalence: 151/44955, 0.34% [35]).

Statistical analyses were conducted using Stata version 13.1 (StataCorp, College Station, Texas, USA).

Results

In total, 1562 MSM who were HIV-1 negative at enrollment in the ACS were included in the analyses. They accounted for 175 HIV-1 seroconversion visits and 17,217 seronegative visits for the period 1984 through 2009. The median number of visits was 7 (interquartile range (IQR) 4–13). Median age at study entry was 34.4 years (IQR 29.4–41.4).

Performance of the initial risk score in Amsterdam cohort studies

In the ACS, the overall AUC for the initial risk score was 0.70 (risk score A). With the pre-defined risk score cutoff of two, sensitivity was 45.7% (95% confidence interval (CI) 38.2–53.4) and specificity 89.5% (95%CI 89.1–90.0). By applying this risk score to ACS study participants, 10.8% would be indicated for AHI testing, identifying 45.7% of cases (Table 2 and Fig. 1a).

Table 2 Performance of several risk scores for AHIa
Full size table
Fig. 1
figure 1

a-e. ROC curves of risk scores in ACS and validation in MACS. ACS, Amsterdam Cohort Studies; MACS, Multicenter AIDS Cohort Study; ROC, receiver operating characteristics

Full size image

Optimizing the risk score

Prevalence of self-reported symptoms in the preceding six months is shown in Table 3. In univariable analysis, all fourteen symptoms were significantly associated with HIV-1 seroconversion. Self-reported oral thrush (OR 10.8, 95%CI 3.8–30.7), fever (OR 7.2, 95%CI 5.4–9.8), and lymphadenopathy (OR 7.2, 95%CI 4.7–10.9) were most strongly correlated with seroconversion. In a multivariable logistic regression model including all fourteen symptoms, oral thrush (adjusted odds ratio (aOR) 5.0, 95%CI 1.5–16.3), fever (aOR 5.0, 95%CI 3.5–7.1), lymphadenopathy (aOR 3.3, 95%CI 2.1–5.4) and genital ulcers (aOR 2.6, 95%CI 1.3–5.0) remained statistically significant.

Table 3 Prevalence of symptoms and risk factors and their association with HIV-1 seroconversiona
Full size table

Three risk factors (self-reported gonorrhea, receptive CLAI, more than five sexual partners, all in the previous six months) for HIV-1 seroconversion previously described in the ACS [30] were significantly associated with seroconversion in univariable analysis (Table 3, column 4). The results of the combined multivariable model with fourteen symptoms and three risk factors are shown in Table 3, column 7. In this combined multivariable model, oral thrush (aOR 5.6, 95%CI 1.4–22.3), fever (aOR 4.7, 95%CI 3.2–7.0), lymphadenopathy (aOR 4.4, 95%CI 2.6–7.3), weight loss (aOR 2.4, 95%CI 1.1–5.2), self-reported gonorrhea (aOR 4.9, 95%CI 2.3–10.2), receptive CLAI (aOR 3.0, 95%CI 2.1–4.2), and more than five sexual partners (aOR 2.5, 95%CI 1.7–3.6) (all in the preceding six months) remained statistically significant.

The overall AUC was 0.71 for a risk score with points assigned to each symptom according to the β in a symptom based model in which CLAI was a condition (risk score B, Fig. 1b). The same risk score, but without CLAI as a condition (risk score C) resulted in an overall AUC of 0.74 (Fig. 1c).

In a risk score including fourteen symptoms and also three risk factors, the overall AUC was 0.83 (risk score D, Fig. 1d). A simplified risk score including only variables statistically significantly associated with HIV-1 seroconversion in the multivariable model yielded an AUC of 0.82 (risk score E). At the risk score cutoff of 1.5 (Youden index), sensitivity was 76.3% (95%CI 68.2–83.2) and specificity was 76.3% (95%CI 75.6–77.0). Applying this risk score to ACS participants, 24.2% would be indicated for AHI testing, identifying 76.3% of cases. Choosing a less stringent cutoff of 1.1 would result in sensitivity of 85.9% but lower the sensitivity to 59.3% (Fig. 1e). Risk score E was defined as the optimal risk score because of a relatively high AUC in combination with simplicity of the risk score (based on only seven variables, all statistically significant in the multivariable model) and therefore suitable for implementation as a screening tool.

In a multivariable logistic regression model that also included age (categorized) and calendar year of visit (categorized), the β of symptoms and risk factors were comparable to the original model (data not shown). An analysis limited to visits from 1990 onwards resulted in an increase of the aOR of myalgia (aOR 2.6, 95%CI 1.4–4.8). Applying the optimal risk score (E) to data from 1990 onwards did not change the performance of the risk score (AUC 0.82), and very similar outcomes (sensitivity, specificity and AUC) were observed when myalgia was added to this risk score (data not shown). Adding educational level to the multivariable model did not result in major changes in the β of symptoms and risk factors, but lower educational level itself was significantly associated with HIV-1 seroconversion (β 0.7, data not shown). A risk score including the variables from the optimal risk score (E) and also educational level showed similar performance (AUC 0.83) compared to the optimal risk score (data not shown). Complete case analyses did not show important differences in outcomes compared to the analyses based on imputed data, but weight loss was no longer significantly associated with HIV-1 seroconversion in the combined symptom and risk factor model (Additional file 1: Table S1 and S2).

For participants who seroconverted during follow-up, prevalence of reported symptoms was consistently lower at the last HIV-1 negative visit preceding seroconversion. There was one participant who seroconverted in which the time interval between the first seropositive visit and the last seronegative visit was more than nine months. Additionally, retrospective HIV-1 RNA testing of stored blood samples among seroconverters showed that 2/175 seroconverters in this study had a positive RNA test result at the visit preceding their seroconversion visit (data not shown).

Validation of the risk score

The optimal risk score (E) was validated in the MACS, with data on 3751 MSM with 491 seroconversion visits and 63,127 seronegative visits. Due to a large number of missings for all variables, the evaluation of the optimal risk score was based on 137 seroconversion visits and 32,916 seronegative visits. The overall AUC was 0.78 (Fig. 1e). Using 1.5 as a cutoff, sensitivity was 56.2% (95%CI 47.5–64.7) and specificity 88.8% (95% CI 88.4–89.1). 11.4% would be indicated for RNA testing, which would identify 56.0% of cases. Table 4 shows sensitivity and specificity at several cutoff levels for this risk score in the MACS.

Table 4 Validation of optimal risk score E.a in MACS, 1984–2010
Full size table

Post-test probability of disease

Risk score E was further used to estimate the PTPD in the ACS. Using an AHI prevalence of 0.34% from a similar MSM population [35] (i.e. the probability of AHI for an individual before applying the risk score) and a risk score cutoff of 1.5, the PTPD (i.e. the probability of AHI after having an AHI testing advice) was 1.09%. The positive likelihood ratio at this cutoff was 3.2.

Discussion

We aimed to assess whether a risk score could be useful for AHI screening. Several risk scores were generated and performance was assessed using data from MSM cohorts in Amsterdam (ACS) and the USA (MACS). The initial risk score (A) – generated by expert opinion, including fourteen symptoms and CLAI as a condition – had a weak overall performance in the ACS. Other risk scores including only symptoms also had a weak performance. The addition of three risk factors for HIV-1 seroconversion to the risk score resulted in good overall performance in the ACS.

The optimal risk score included oral thrush (β 1.7), fever (β 1.6), lymphadenopathy (β 1.5), weight loss (β 1.9), self-reported gonorrhea (β 1.6), receptive CLAI (β 1.1), and more than five sexual partners (β 0.9) (all in the preceding six months) (risk score E). Using this risk score in the ACS, with a cutoff of 1.5, 24% of men would be assigned as having possible AHI and indicated for HIV-1 RNA testing. This would identify 76% of seroconverters, with a PTPD of 1.09%. The overall AUC was 0.82. Validation of this risk score in the MACS resulted in a similar AUC (0.78) but lower sensitivity (56%). These outcomes are comparable with previous studies [21,22,23, 36]. A study including data of AHI patients from four African sites described a risk score (including age 18–29 years, fever, diarrhea, fatigue, body pains, sore throat and genital ulcer disease) resulting in an overall AUC of 0.78, and among Kenyan MSM specifically, the AUC was 0.89, sensitivity 90% and specificity 74% [21].

This study has several limitations. First, in both ACS and MACS the interval of six months between study visits was relatively long. Questionnaires in both cohorts included questions on symptoms and risk behavior in the preceding six months. These were reported at seroconversion visits, but it is unclear whether symptoms occurred during AHI. Twenty-nine percent of seroconverters in the ACS did not report any symptom in the preceding six months (data not shown). AHI might have been asymptomatic [24], associated with nonspecific symptoms [18, 37] or symptoms might have been underreported due to recall bias. A multicenter study from Africa showed that prevalence of reported AHI symptoms of patients evaluated within six weeks following infection was higher as compared to patients who were evaluated later [38]. This may also explain the relatively low sensitivity observed in the present study. Thus, this relatively long interval between study visits may have led to an underestimation of the performance of the risk scores. Furthermore, the period of acute infection might have been occurring at the previous study visit, when antibodies were not yet detectable, rather than at the seroconversion visit. However, in the ACS, fewer symptoms were reported at last HIV-1 negative visits compared to seroconversion visits and additional HIV-1 RNA testing showed positive test results for only two seroconverters at the pre-seroconversion visit. Moreover, a previous sensitivity analyses of ACS data using risk behavior reported at the visit preceding the seroconversion visit did not result in a different pattern of behavioral variables associated with HIV-1 seroconversion [30]. Although we excluded visits above twelve months following the previous visit from the analyses, theoretically some seroconversions may have occurred up to eleven months prior to the first HIV-1 positive visit. However, evaluating the time interval between seroconversion visits and last seronegative visits showed that there was only one seroconverter in the ACS in which this time interval was more than nine months. These findings suggest that the results reported in the present study are expected to be robust.

A second limitation is the imputation method (last observation carried forward for missing values if the variable of the visit immediately preceding was available), being potentially inappropriate for time-sensitive risk behavior prior to seroconversion. However, complete case analyses (Additional file 1: Table S1 and S2) did not show important differences in outcomes, suggesting that the imputation approach did not introduce bias into the results.

Third, it was decided in this study to report sensitivity and specificity for every risk score at the cutoff calculated by the Youden-Index, to have a consistent approach for defining a threshold. However, the Youden-index may not be the ideal cutoff for every context. The optimal cutoff would depend on the AHI prevalence of the target population, the clinical setting and the resources available. To increase sensitivity, a lower cutoff could be used (Fig. 1a-e and Table 4). This would lower specificity, therefore a larger number of individuals would get an AHI testing advice but have a negative test result, resulting in an increase of costs.

Finally, only 112 of 175 (64%) seroconverters in the ACS reported CLAI (insertive or receptive) in the preceding six months. It is well established that CLAI is the main transmission route for acquiring HIV-1 among MSM [39] and not reporting CLAI could be due to social desirability bias, again potentially lowering the performance of the risk score. Similar rates of CLAI were described for AHI patients in a recent prospective study from the USA among MSM [40] and the data collected in both cohorts are likely consistent with clinical settings in which the risk score could be implemented as a screening tool, using clients’ self-reported behavior.

Despite these limitations, we believe that our data show that early recognition of symptoms and risk behavior in MSM could potentially enhance early diagnosis and linkage to care. Targeted screening for AHI with risk score algorithms was recently recommended to be included in clinical evaluation of high HIV-1 incidence groups [41]. Considering AHI, the Dutch HIV guidelines refer to the US guidelines [2, 42], in which it is recommended that clinicians consider a diagnosis of AHI in patients with acute retroviral syndrome and recent high risk of exposure to HIV-1. This study shows that using a risk score for AHI as a screening tool could reduce the number of individuals needing AHI testing if MSM could be targeted for AHI evaluation at the point-of-care. During a first encounter with the health care system, less than half of AHI patients were correctly diagnosed at their first visit [19, 43]. Correct and direct identification of individuals with AHI would result in the opportunity to start immediate treatment with cART. This has numerous benefits, from both a patient and a public health perspective [9, 13, 44]. Furthermore, it would offer the opportunity to provide pre-exposure prophylaxis (PrEP) to MSM in established partnerships until the index patient is virologically suppressed.

In this study, risk score E (including oral thrush, fever, lymphadenopathy, weight loss, gonorrhea, receptive CLAI, and more than five sexual partners) was identified as the optimal screening tool. It had good overall performance and could be incorporated in both a self-assessment (online) tool and assist clinicians with AHI screening. The online screening tool could be used by risk groups, general practitioners working with MSM, and clinicians working at STI clinics. For example, in a public health setting or STI clinic, in which STI and HIV testing are largely based on standardized protocols, the items of the risk score can be incorporated into the routine assessment of MSM. A computer-based algorithm could identify men above the threshold of 1.5 and AHI testing could be automatically offered to them. Furthermore, a self-assessment, publicly available as an online tool, could be easily completed by MSM potentially at risk for AHI or worried about acute infection; after completion of this tool they would receive a tailored advice whether or not to attend for AHI testing. Using 1.5 as a cutoff, MSM reporting oral thrush, fever, lymphadenopathy, or gonorrhea would immediately qualify for AHI screening. Summation would be required for reporting weight loss, receptive CLAI or more than five sexual partners.

Conclusion

In summary, while we initially established a risk score based on symptoms alone, we improved this algorithm by adding known predictors for HIV-1 acquisition in MSM. Screening for AHI with four symptoms (oral thrush, fever, lymphadenopathy, weight loss) and three risk factors (self-reported gonorrhea, receptive CLAI and more than five sexual partners, all in the previous six months) would lower the number of individuals requiring AHI testing and potentially enhance early diagnosis and immediate treatment.

Change history

  • 24 October 2017

    A correction to this article has been published.

Abbreviations

ACS:

Amsterdam Cohort Studies

AHI:

Acute HIV-1 infection

aOR:

Adjusted odds ratio

AUC:

Area under the curve

cART:

Combination antiretroviral therapy

CI:

Confidence interval

CLAI:

Condomless anal intercourse

GGD:

Public Health Service of Amsterdam

H-TEAM:

HIV Transmission Elimination AMsterdam Initiative

IQR:

Interquartile range

MACS:

Multicenter AIDS Cohort Study

MSM:

Men who have sex with men

OR:

Odd ratio

PrEP:

Pre-exposure prophylaxis

ROC:

Receiver operating characteristics

STI:

Sexually transmitted infection

β:

Beta

References

  1. Cohen M, Shaw G, McMichael A, Haynes B. Acute HIV-1 infection. N Engl J Med. 2011;364:1943–54.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Panel on antiretroviral guidelines for adults and adolescents. Guidelines for the use of antiretroviral agents in HIV-1-infected adults and adolescents. Department of Health and Human Services. 2014. http://aidsinfo.Nih.Gov/ContentFiles/AdultandAdolescentGL.Pdf. Accessed 17 December 2016.

  3. Grijsen ML, Steingrover R, Wit FW, Jurriaans S, Verbon A, Brinkman K, et al. No treatment versus 24 or 60 weeks of antiretroviral treatment during primary HIV infection: the randomized Primo-SHM trial. PLoS Med. 2012;9(3):e1001196.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Le T, Wright EJ, Smith DM, He W, Catano G, Okulicz JF, et al. Enhanced CD4+ T-cell recovery with earlier HIV-1 antiretroviral therapy. N Engl J Med. 2013;368(3):218–30.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Oxenius A, Price D, Easterbrook P, O’Callaghan C, Kelleher A, Whelan J, et al. Early highly active antiretroviral therapy for acute HIV-1 infection preserves immune function of CD81 and CD41 T lymphocytes. PNAS. 2000;97(7):33823387.

    Article  Google Scholar 

  6. Powers KA, Ghani AC, Miller WC, Hoffman IF, Pettifor AE, Kamanga G, et al. The role of acute and early HIV infection in the spread of HIV and implications for transmission prevention strategies in Lilongwe, Malawi: a modelling study. Lancet. 2011;378(9787):256–68.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Cohen MS, Chen YQ, McCauley M, Gamble T, Hosseinipour MC, Kumarasamy N, et al. Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med. 2011;365(6):493–505.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Ratmann O, van Sighem A, Bezemer D, Gavryushkina A, Jurriaans S, Wensing A, et al. Sources of HIV infection among men having sex with men and implications for prevention. Sci Transl Med. 2016;8(320):320ra322.

    Article  Google Scholar 

  9. Fidler S, Porter K, Ewings F, Frater J, Ramjee G, Cooper D, et al. Short-course antiretroviral therapy in primary HIV infection. N Engl J Med. 2013;368(3):207–17.

    Article  PubMed  Google Scholar 

  10. Heijman T, Geskus RB, Davidovich U, Coutinho RA, Prins M, Stolte IG. Less decrease in risk behaviour from pre-HIV to post-HIV seroconversion among MSM in the combination antiretroviral therapy era compared with the pre-combination antiretroviral therapy era. AIDS. 2012;26(4):489–95.

    Article  PubMed  Google Scholar 

  11. Pettifor A, MacPhail C, Corneli A, Sibeko J, Kamanga G, Rosenberg N, et al. Continued high risk sexual behavior following diagnosis with acute HIV infection in South Africa and Malawi: implications for prevention. AIDS Behav. 2011;15(6):1243–50.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Chun TW, Justement JS, Moir S, Hallahan CW, Maenza J, Mullins JI, et al. Decay of the HIV reservoir in patients receiving antiretroviral therapy for extended periods: implications for eradication of virus. J Infect Dis. 2007;195(12):1762–4.

    Article  CAS  PubMed  Google Scholar 

  13. Ananworanich J, Schuetz A, Vandergeeten C, Sereti I, de Souza M, Rerknimitr R, et al. Impact of multi-targeted antiretroviral treatment on gut T cell depletion and HIV reservoir seeding during acute HIV infection. PLoS One. 2012;7(3):e33948.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Buzon MJ, Martin-Gayo E, Pereyra F, Ouyang Z, Sun H, Li JZ, et al. Long-term antiretroviral treatment initiated at primary HIV-1 infection affects the size, composition, and decay kinetics of the reservoir of HIV-1-infected CD4 T cells. J Virol. 2014;88(17):10056–65.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Saez-Cirion A, Bacchus C, Hocqueloux L, Avettand-Fenoel V, Girault I, Lecuroux C, et al. Post-treatment HIV-1 controllers with a long-term virological remission after the interruption of early initiated antiretroviral therapy ANRS VISCONTI study. PLoS Pathog. 2013;9(3):e1003211.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Branson B, Owen S, Wesolowski L, Bennett B, Werner B, Wroblewski K, Pentella M. Laboratory testing for the diagnosis of HIV infection: updated recommendations. Centers for Disease Control and Prevention. 2014. https://stacks.Cdc.Gov/view/cdc/23447. Accessed 17 December 2016.

  17. Garrett NJ, Drain PK, Werner L, Samsunder N, Abdool Karim SS. Diagnostic accuracy of the point-of-care xpert HIV-1 viral load assay in a South African HIV clinic. J Acquir Immune Defic Syndr. 2016;72(2):e45–8.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Wood E, Kerr T, Rowell G, Montaner JS, Phillips P, Korthuis PT, et al. Does this adult patient have early HIV infection? The rational clinical examination systematic review. JAMA. 2014;312(3):278–85.

    Article  CAS  PubMed  Google Scholar 

  19. Braun DL, Kouyos RD, Balmer B, Grube C, Weber R, Gunthard HF. Frequency and Spectrum of unexpected clinical manifestations of primary HIV-1 infection. Clin Infect Dis. 2015;61(6):1013–21.

    Article  CAS  PubMed  Google Scholar 

  20. Robb ML, Ananworanich J. Lessons from acute HIV infection. Curr Opin HIV AIDS. 2016;11(6):555–60.

    Article  CAS  PubMed  Google Scholar 

  21. Sanders EJ, Wahome E, Powers KA, Werner L, Fegan G, Lavreys L, et al. targeted screening of at-risk adults for acute HIV-1 infection in sub-Saharan Africa. AIDS. 2015;29(suppl 3):S221–30.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Mlisana K, Sobieszczyk M, Werner L, Feinstein A, van Loggerenberg F, Naicker N, et al. Challenges of diagnosing acute HIV-1 subtype c infection in African women: performance of a clinical algorithm and the need for point-of-care nucleic-acid based testing. PLoS One. 2013;8(4):e62928.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Powers KA, Millera WC, Pilcher CD, Mapanje C, Martinsond FEA, Fiscusc SA, et al. improved detection of acute HIV-1 infection in sub-Saharan Africa: development of a risk score algorithm. AIDS. 2007;21:2237–42.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Robb ML, Eller LA, Kibuuka H, Rono K, Maganga L, Nitayaphan S, et al. Prospective study of acute HIV-1 infection in adults in East Africa and Thailand. N Engl J Med. 2016;374(22):2120–30.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Miller WC, Leone PA, McCoy S, Nguyen TQ, Williams DE, Pilcher CD. Targeted testing for acute HIV infection in North Carolina. AIDS. 2009;23(7):835–43.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Hoenigl M, Weibel N, Mehta SR, Anderson CM, Jenks J, Green N, et al. Development and validation of the San Diego early test score to predict acute and early HIV infection risk in men who have sex with men. Clin Infect Dis. 2015;61(3):468–75.

    Article  PubMed  PubMed Central  Google Scholar 

  27. HIV tranmission elimination AMsterdam initiative (h-TEAM). http://www.Hteam.Nl/en. Accessed 17 December 2016.

  28. Heb ik hiv? https://hebikhiv.Nl/. Accessed 17 December 2016.

  29. Van Griensven GJP, de Vroome EMM, Goudsmit J, Coutinho RA. Changes in sexual behaviour and the fall in incidence of HIV infection among homosexual men. BMJ. 1989;298:218–2.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Jansen IA, Geskus RB, Davidovich U, Jurriaans S, Coutinho RA, Prins M, et al. Ongoing HIV-1 transmission among men who have sex with men in Amsterdam: a 25-year prospective cohort study. AIDS. 2011;25(4):493–501.

    Article  PubMed  Google Scholar 

  31. Youden W. Index for rating diagnostic tests. Cancer. 1950;3:32–5.

    Article  CAS  PubMed  Google Scholar 

  32. Kaslow R, Friedman H, Lyter D, Solomon R, Dudley J, Polk B, et al. Infection with the human immunodeficiency virus: clinical manifestations and their relationship to immune deficiency. A report from the multicenter AIDS cohort study. Ann Intern Med. 1987;107:474–80.

    Article  CAS  PubMed  Google Scholar 

  33. Multicenter AIDS cohort study public dataset. https://statepi.Jhsph.Edu/macs/pdt.Html. Accessed 17 December 2016.

  34. Sackett DL, Brian Haynes R, Guyat GH, Tugwell P. clinical epidemiology. A basic science for clinical medicine. USA: Lippincott Williams&Wilkins; 1991.

    Google Scholar 

  35. Peters PJ, Westheimer E, Cohen S, Hightow-Weidman LB, Moss N, Tsoi B, et al. Screening yield of HIV antigen/antibody combination and pooled HIV RNA testing for acute HIV infection in a high-prevalence population. JAMA. 2016;315(7):682–90.

    Article  CAS  PubMed  Google Scholar 

  36. Rutstein SE, Pettifor AE, Phiri S, Kamanga G, Hoffman IF, Hosseinipour MC, et al. Incorporating acute HIV screening into routine HIV testing at sexually transmitted infection clinics, and HIV testing and counseling centers in Lilongwe, Malawi. J Acquir Immune Defic Syndr. 2016;71(3):272–80.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Veugelers P, Kaldor J, Strathdee S, Page-Shafer K, Schechter M, Coutinho R, et al. Incidence and prognostic significance of symptomatic primary human immunodeficiency virus type 1 infection in homosexual men. J Infect Dis. 1997;176(1):112–7.

    Article  CAS  PubMed  Google Scholar 

  38. Sanders E, Powers K, Karita E, Kamali A, Kilemba W, Allen S. Recall of acute retroviral symptoms in a multicentre cohort study in Africa. AIDS Res Hum Retrovir. 2014;30(Suppl 1):A93.

    Article  Google Scholar 

  39. Vittinghoff E, Douglas J, Judson F, McKirnan D, MacQueen K, Buchbinder S. Per-contact risk of human immunodeficiency virus transmission between male sexual partners. Am J Epidemiol. 1999;150:306.

    Article  CAS  PubMed  Google Scholar 

  40. Davey J, Beymer M, Roberts C, Bolan R, Klausner J. Sexual behavior during acute HIV infection among men who have sex with men in Los Angeles, California. J Infect Dis. 2016;213(12):2020–1.

    Article  Google Scholar 

  41. Rutstein SE, Sellers CJ, Ananworanich J, Cohen MS. The HIV treatment cascade in acutely infected people: informing global guidelines. Curr Opin HIV AIDS. 2015;10(6):395–402.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Nederlandse vereniging van HIV-behandelaren. Hoofdstuk 5. Diagnostiek en behandeling primaire HIV-infectie. 2015. http://www.Nvhb.Nl/richtlijnhiv/index.Php/Hoofdstuk_5._Diagnostiek_en_behandeling_primaire_HIV-infectie. Accessed 17 December 2016.

  43. McKellar MS, Cope AB, Gay CL, McGee KS, Kuruc JD, Kerkau MG, Hurt CB, fiscus SA, Ferrari G, Margolis DM et al. Acute HIV-1 infection in the Southeastern United States: a cohort study. AIDS Res Hum Retrovir 2013, 29(1):121-128.

  44. Brenner BG, Roger M, Routy JP, Moisi D, Ntemgwa M, Matte C, et al. High rates of forward transmission events after acute/early HIV-1 infection. J Infect Dis. 2007;195(7):951–9.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank all the personnel of the Amsterdam Cohort Studies for data collection and management. We gratefully acknowledge all the study participants for their co-operation and participation. We further acknowledge E. Hoornenborg, H.J.C. de Vries, W. Zuilhof, J.E.A.M. van Bergen, and J.M. Prins in their help and advice for the development of the initial risk score. We thank the investigators of the Multicenter AIDS Cohort Study for sharing their data set, which made the validation study possible.

H-TEAM initiative study group members:

H-TEAM Steering Committee: J.E.A.M. van Bergen4,5, F. Deug6, M. Heidenrijk1, J.M. Prins2, M. Prins3, P.J. Smit8, M. van der Valk2, P. Reiss1,7 (chair)

H-TEAM Core Project Group: G.J. de Bree1,2 (chair), E. Hoornenborg3, J.E.A.M. van Bergen4,5,6, U. Davidovich3,10, S.E. Geerlings2, A. Oomen6, J.M. Prins2, A. van Sighem7, P.J. Smit8, W. Zuilhof6

H-TEAM Project Management: M. Groot Bruinderink1, W. Zikkenheiner1, F.R. Janssen1

H-TEAM additional collaborators: R.C.A. Achterbergh3, M. van Agtmael25, J. Ananworanich23, D. Van de Beek17, G.E.L. van den Berk11, B. Berkhout20, D. Bezemer7, A. van Bijnen6, W.L. Blok11, M. Bomers25, C.A.B. Boucher13, W. Brokking28, D. Burger21, K. Brinkman11, M. de Bruin12, B O’Dell1, L. Dellemann6, M. Dijkstra3, Y.T. van Duijnhoven3, A. van Eeden28, S. van Elsen6, L. Elsenburg28, C. Ester7, P.H.J. Frissen11, T.B.H. Geijtenbeek18, M.H. Godfried2, A. Goorhuis2, M. Groot28, R.A. Gruters24, C.A. Hankins1, A. Heijnen4, J.J. van der Helm3, M. Hillebregt7, A. Hogewoning3, J.W. Hovius2, I.K. Joore4, N. Kodde1, N.A. Kootstra19, R.A. Koup22, F.P. Kroon16, F. Lauw26, K. Lettinga29, T. Mahmoudi24, E. Meddens6, J.T. van der Meer2, S. van Meeteren3, T. Mouhebati6, J. Mulder26, F.J. Nellen2, A. Nijsters6, H. Nobel2, A.O. Pasternak20, E. Peters25, T. van der Poll2, A. van Puffelen1, W.E.M. Schoute11, C. Rokx14, M.S. van Rooijen3, M.F. Schim van der Loeff3,10, G.J. Sonder3, M.G. van Veen3, J. Veenstra29, A. Verbon14, F. Verdult8, D. Verhagen27, G. Visser7, J. de Vocht25, H.J. de Vries3,9,10, S. Vrouenraets26, M. van Vugt2, W.J. Wiersinga2, F.W. Wit2, L.R. Woittiez2, S. Zaheri7, P. Zandkuijl6, M.C. van Zelm24, F.R. Zuure3

1Department of Global Health, Academic Medical Center, and Amsterdam Institute for Global Health and Development, Amsterdam, the Netherlands

2Department of Internal Medicine, Division of Infectious Diseases, Academic Medical Center Amsterdam, the Netherlands

3Department of Infectious Diseases, Public Health Service of Amsterdam, Amsterdam, the Netherlands

4Department of General Practice, Academic Medical Center, University of Amsterdam, the Netherlands

5Epidemiology and Surveillance Unit, Centre for Infectious Disease Control, National Institute of Public Health and the Environment, the Netherlands

6Soa Aids Nederland, Amsterdam, the Netherlands

7Stichting HIV Monitoring, Amsterdam, the Netherlands

8Dutch HIV Association, Amsterdam, the Netherlands

9Department of Dermatology, Academic Medical Center (AMC), University of

Amsterdam, Amsterdam, the Netherlands

10Center for Infection and Immunology, Amsterdam (CINIMA), Academic Medical Center (AMC), University of Amsterdam, Amsterdam, the Netherlands

11Department of internal medicine, Onze Lieve Vrouwe Gasthuis, Amsterdam, the Netherlands

12Aberdeen Health Psychology Group, Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK.

13Department of viro-science, Erasmus Medical Center Rotterdam, the Netherlands

14Department of Internal Medicine and Infectious Diseases, Erasmus University Medical Center, Rotterdam, the Netherlands

15University of Amsterdam, the Netherlands

16Department of Infectious Diseases, Leiden University Medical Centre, Leiden, the Netherlands

17Center of Infection and Immunity Amsterdam (CINIMA), Department of Neurology, Academic Medical Center, Amsterdam, The Netherlands

18Laboratory of Experimental Immunology, Academic Medical Center Amsterdam, the Netherlands

19Laboratory for Viral Immune Pathogenesis, Academic Medical Center Amsterdam, the Netherlands

20Laboratory of Experimental Virology, Academic Medical Center Amsterdam, the Netherlands

21Department of Pharmacy, Radboud University Nijmegen Medical Center, the Netherlands

22Immunology Laboratory, Vaccine Research Center, NIAID, National Institutes of Health

23US Military HIV Research Program and the Henry M. Jackson Foundation for the Advancement of Military Medicine, Bethesda, United States

24Department of Virology, Erasmus Medical Center, Rotterdam, the Netherlands

25Department of Internal Medicine, Free University Medical Center, Amsterdam, the Netherlands

26Department of Internal Medicine, Slotervaart Hospital Amsterdam, the Netherlands

27Department of Internal Medicine, Jan van Goyen Institute, Amsterdam, the Netherlands

28DC Clinics, Amsterdam, the Netherlands

29Department of Internal Medicine, Lukas Andreas Hospital, Amsterdam, the Netherlands

Funding

The Amsterdam Cohort Studies on HIV infection and AIDS, a collaboration between the Public Health Service of Amsterdam, the Academic Medical Center of the University of Amsterdam, and the Sanquin Blood Supply Foundation, are part of the Netherlands HIV Monitoring Foundation and financially supported by the Netherlands National Institute for Public Health and the Environment. E.J.S. receives funding from the Wellcome Trust (grant: 107,752). The H-TEAM initiative is supported through grants by the Dutch AIDS foundation (grant: 2,013,169), ZonMW (grant: 522,002,003), and unrestricted grants from Gilead Sciences, Janssen Pharmaceutics, ViiV Healthcare, Bristol-Meyers Squibb and The Amsterdam Dinner Foundation.

Availability of data and materials

The Amsterdam Cohort Studies dataset used and analysed during the current study available from the corresponding author on reasonable request. The Multicenter AIDS Cohort Study Public Dataset used and analysed during the current study is available from: https://statepi.jhsph.edu/macs/pdt.html [33].

Authors’ contributions

M.D. and G.J.B. contributed equally to this study. M.D. conducted data analyses and drafted the manuscript. G.J.B. designed the overall study and helped draft and revise the manuscript. I.G.S. helped design the study, contributed to the analyses and interpretation of data and reviewed the manuscript. U.D., E.J.S, and M.P. contributed to the analyses and interpretation of data and reviewed the manuscript. M.S.L. designed and supervised the overall study, contributed substantially to the analyses and interpretation of data, and helped draft and revise the manuscript. All authors read and approved the final version of the manuscript.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

The Amsterdam Cohort Studies have been conducted in accordance with the ethical principles set out in the declaration of Helsinki, and participation in the Amsterdam Cohort Studies is voluntary; written informed consent is obtained from each participant. The study was approved by the Medical Ethics Committee of the Academic Medical Center (reference number: MEC 07/182).

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Author notes

    Authors and Affiliations

    1. Academic Medical Center, Department of Infectious Diseases, University of Amsterdam, P.O. Box 22700, 1100DE, Amsterdam, the Netherlands

      Maartje Dijkstra, Godelieve J. de Bree, Udi Davidovich, Maria Prins & Maarten F. Schim van der Loeff

    2. Public Health Service of Amsterdam, Department of Infectious Diseases, Research and Prevention, P.O. Box 2200, 1000CE, Amsterdam, the Netherlands

      Maartje Dijkstra, Ineke G. Stolte, Udi Davidovich, Maria Prins & Maarten F. Schim van der Loeff

    3. Amsterdam Institute for Global Health and Development, University of Amsterdam, P.O. Box 22700, 1100DE, Amsterdam, the Netherlands

      Godelieve J. de Bree & Eduard J. Sanders

    4. Kenya Medical Research Institute, Centre for Geographic Medicine Research –Coast, P.O. Box 230, Kilifi, Kenya

      Eduard J. Sanders

    5. Nuffield Department of Clinical Medicine, University of Oxford, Oxford, OX3 7BN, UK

      Eduard J. Sanders

    Authors
    1. Maartje Dijkstra
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Godelieve J. de Bree
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Ineke G. Stolte
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Udi Davidovich
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. Eduard J. Sanders
      View author publications

      You can also search for this author in PubMed Google Scholar

    6. Maria Prins
      View author publications

      You can also search for this author in PubMed Google Scholar

    7. Maarten F. Schim van der Loeff
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Godelieve J. de Bree.

    Additional information

    A correction to this article is available online at https://doi.org/10.1186/s12879-017-2805-y.

    Additional file

    Additional file 1:

    Tables with results of complete case analyses. (DOCX 27.2 kb)

    Rights and permissions

    Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Dijkstra, M., de Bree, G.J., Stolte, I.G. et al. Development and validation of a risk score to assist screening for acute HIV-1 infection among men who have sex with men. BMC Infect Dis 17, 425 (2017). https://doi.org/10.1186/s12879-017-2508-4

    Download citation

    • Received:

    • Accepted:

    • Published:

    • DOI: https://doi.org/10.1186/s12879-017-2508-4

    Keywords

    BMC Infectious Diseases

    ISSN: 1471-2334

    Contact us