Brazilian Program for HIV Point-of-Care Test Evaluation : a Decade ’ s Experience

Background: The point-of-care tests (POCTs) for HIV diagnosis have been widely used in Brazil in order to expand and to allow HIV diagnosis outside health units including remote areas, such as the Amazon region. In order to guarantee the quality of HIV diagnostics based on rapid tests, the Brazilian Ministry of Health (MoH) implemented the HIV POCT Evaluation Program. This study compiles the Brazilian experience acquired over the last 13 years conducting the HIV POCT Evaluation Program.


Introduction
For most infectious diseases, prompt and accurate diagnosis is a crucial public health strategy for implementing early and more effective treatment and, consequently, interrupting transmission chains sustained by untreated unaware cases [1].
According to the 2015 Brazilian HIV/AIDS Epidemiological Bulletin, since 1980, a total of 798,366 Acquired Immunodeficiency Syndrome (AIDS) cases and 290,929 AIDS related deaths have been registered in the country.Over the last decade, AIDS detection rates have shown that HIV/AIDS epidemic in Brazil has been relatively stable (with an AIDS incidence rate of 20.5 per 100,000 inhabitants), although there were important differences among Brazilian states, ranging from 9.2 AIDS cases per 100,000 inhabitants (Acre state) to 39.2 AIDS cases per 100,000 inhabitants (Amazonas state) [2].
In Brazil, HIV diagnostic is regulated by the Ministerial Directive MS/SVS n° 29 [3], which approved the Technical Manual for the Diagnosis of HIV Infection in Adults and Children.The Manual establishes six algorithms for HIV diagnosis, two of which are based on the use of point-of-care tests (POCTs).The main reasons for employing POCTs in Brazil are to expand and to allow HIV diagnosis outside health units including remote areas, such as the Amazon region.Indeed, HIV POCTs have been widely used to test parturient and puerperal women unaware of their serological status, patients attending emergency clinics, campaigns and street interventions towards general population as well as peers testing for those at increased risk, as men who have sex with men, sex workers, drug users, homeless and incarcerated population [4].
Currently, the Brazilian Ministry of Health (MoH) distributes three different POCTs for HIV diagnosis (two based on whole blood and one on oral fluid) to any public health facility, in all the 26 states and the Federal District.POCT diagnosis is based on two sequential tests but, whenever the oral fluid is an option, it is always the first test in the algorithm.HIV positive individuals are then referred to the public health system for medical care.
Ideally every POCT commercially available should meet expected performance characteristics.Howe-HIV Multi-test (whole blood), HIV 1/2 Colloidal Gold (whole blood), OraQuick ADVANCE ® Rapid HIV-1/2 Antibody Test (whole blood and oral fluid) and TR DPP HIV-1/2 (whole blood, plasma and oral fluid).Regarding other evaluated criteria, all assays met the requirements.ver, many countries lack active regulatory agencies and problems related to poor product performance have arisen in the field.Therefore, the quality assurance is frequently based on data from the manufacturer, which may be founded on biased, inadequate and/or flawed studies [1,5].Aware of this, the Brazilian government, through the Ministerial Directive MS/SVS n°29/2013, established the accepted performance criteria for HIV POCTs.The minimum sensitivity of 99.5% and specificity of 99.0% must be met, among other criteria described below [4].
The MoH first implemented the Brazilian HIV POCT Evaluation Program in 2003 [6] in order to guarantee the quality of HIV diagnostics based on rapid tests.All manufacturers intending to participate in the MoH public tender for POCT provision must have had their product evaluated by this program.Two rounds of the Brazilian HIV POCT Evaluation Program have been previously described [6,7].
Since its implementation, the POCT evaluation protocol has undergone some modifications aiming to improve and simplify the evaluation process.It is worth to mention that even though some POCT quality evaluation programs adopt protocols based on plasma sample [8], the Brazilian program has developed a strategy to implement POCT evaluation using whole blood and/or oral fluid as sample matrix.This study compiles the Brazilian experience acquired over the last 13 years conducting the HIV POCT Evaluation Program, which allowed the optimization of time demand and resource consumption.Moreover, the results obtained for each POCT evaluated are described in details.

Study design
The Brazilian MoH in collaboration with the Federal University of Rio de Janeiro (UFRJ) developed a protocol to evaluate the performance of HIV POCTs commercially available in Brazil.Since it involves co-llection of fresh whole blood, an anonymous unlinked study was conceived and approved by the National Bioethics Commission of Brazil (Conselho Nacional de Ética -CONEP).An informed consent was signed by each participant before enrollment.All volunteers were 18 years old or older.
The HIV POCT evaluation took place at different time frames.Since the POCT Evaluation Program uses fresh whole blood and oral fluid samples, each round was performed with an unique set of samples, always including HIV positive and negative ones.After the fourth round, a brief questionnaire was incorporated to the protocol, which addresses three main issues: (i) gender; and if female, number of pregnancies; (ii) for HIV-positive individuals, the use of antiretrovirals (ARV) and, if yes, for how long the ARV therapy has been used; and (iii) for oral fluid tests only, use of antiseptics, alcohol ingestion, smoking and time of last meal before oral fluid collection.
For tests based on whole blood and plasma, 5 mL of whole blood were collected by venous puncture in tubes containing ethylenediaminetetraacetic acid.Samples were labeled and sent to the UFRJ, where the POCT was performed on the same day of the sample collection.Daily, after completion of the POCT evaluation with whole blood, samples were centrifuged in order to recover the leftover plasma.The plasma volume was split into two aliquots: (i) 1-2 mL for the plasma repository; and (ii) 0.5 mL for both POCT evaluation and gold standard testing.
For oral fluid based tests, the sample was collected using the swab provided by the manufacturer.If the oral fluid sample could be maintained on buffer solution, the swab was immersed in the buffer and transported, under the conditions and within the time frame established by the manufacturer, to the UFRJ, where the POCT was done.Otherwise, if the POCT needed to be performed shortly after oral fluid collection, the test was operated by a properly trained technician, at the collection site.
In total, six evaluation rounds were conducted in Brazil since 2003.This study encompasses the last four evaluations.All rounds have used the same sampling strategy, except one.The source for HIV positive samples (n=200) were HIV infected individuals under clinical follow up.The number of 200 positive samples was established on the premises that a POCT could misdiagnosis one sample and still remains within the acceptable range (99.5%) of sensitivity.The source of HIV negative samples were blood donors.The initial number of samples per round was 800 and accumulated data allowed the reduction of this number to 600 without compromising the target of 99.0% for the specificity.In the exceptional round mentioned above, we evaluated 1,744 samples from the following services/clinics: sexually transmitted infections (STI) outpatient clinics (n=105); antenatal clinics (n=682), voluntary counseling and testing sites (n=240), HIV clinics (n=175) and blood banks (n=542).Amongst these 1,744 samples, 191 were HIV positive and 1,553 were HIV negative.
Every test result was read by two technicians and, in case of discordance between the readers, a third independent person was called in.Also, every day, three samples from the routine (at least one negative and one positive) were randomly selected for a quality control procedure.These samples were retested and in case of discordant outcomes between the initial test and the quality control, a third additional test was performed.In this case, all samples from the original testing batch were re-tested with the assay that presented the discordant result.The final result was defined as the median among the three tests.
In order to address the analytical sensitivity of the POCTs, we have used commercial seroconversion panels PRB931 (nine samples; the last four are EIA positive), PRB940 (eight samples; the last six are EIA positive) and PRB959 (seven samples; the last five are EIA positive) (Boston Biomedica Inc.); and one in-house panel BX1798 (five EIA positive samples from seroconverting blood donors).

Reference tests
All fresh whole blood samples collected were submitted to a reference testing protocol.The screening test was an enzyme immunoassay (EIA), either Vironostika HIV Uni-Form II plus O (BioMerieux, Durham, NC, USA) or Murex HIV 1.2.0 (Murex Biotech Limited, Dartford, UK).The EIA non-reactive samples were classified as non-reactive for HIV and no additional tests were performed.EIA reactive samples and those with discordant results between the EIA and the POCT were subjected to a confirmatory Western blot using the Cambridge Biotech HIV-1 WB test (Calypte Biom.Corp., Rockville, MD, EUA).The criteria for Western blot positivity required the presence of p24 gag protein and any two proteins from the virus envelope (gp41, gp120 or gp160), regardless of band intensity.Samples with negative Western blot results were considered non-infected by HIV; samples with positive results were considered infected by HIV and samples with indeterminate results were excluded from the analysis.In some specific situations, HIV-1 RNA PCR was performed to further confirm early HIV infection.

Point-of-care tests
The selection of tests was based on the interest of manufacturers to qualify for the MoH tenders.However, to be eligible for the evaluation, the product must be previously registered by the Brazilian Health Surveillance Agency (Agência Nacional de Vigilância Sanitária -ANVISA); be able to simultaneously detect HIV-1 (groups M and O) and HIV-2; and be supplied free of charge by the manufacturer or distributor in enough quantities to perform the evaluation.
Table 1 describes the round, the sample matrix and the characteristics of the POCT analyzed by the Brazilian HIV POCT Evaluation Program.All tests were performed accordingly to the manufacturer's instructions by trained laboratory technicians.

Criteria used for ranking the point-of-care tests performance
The criteria used for ranking the POCT performance were: (i) clinical sensitivity (≥ 99.5%); (ii) clinical specificity (≥ 99.0%); (iii) assay operational characteristics; (iv) detection of HIV-2 antibodies; (v) sensitivity to subtypes panels; and (vi) sensitivity to seroconversion panels.
The operational assay performance considered the following characteristics of the assay as positive (score 1) or negative (score 0): i. Number of reagents needed to run the assay: 1, only one reagent needed; and 0, more than one reagent needed; ii. Reagent storage temperature: 1, refrigeration not required for storage; and 0, refrigeration required; iii.Total number of assay steps: 1, equal to or less than four steps; and 0, more than four steps required to complete the test (steps defined as: addition of the sample, addition of reagents and reading); iv.Total performance time: 1, equal to or less than 30 minutes; and 0, more than 30 min; and v. Technical skill needed by the operator: 1, no laboratory experience; and 0, laboratory experience.To achieve good operational performance, the RT had to score at least four points.Redmond, Washington, USA).Sensitivity and specificity were calculated accordingly to Clinical & Laboratory Standards Institute (CLSI) and were expressed with a 95% binomial confidence interval (CI).

Results
Point-of-care tests performance using fresh whole blood and plasma as matrices Concerning clinical specificity, no test was 100% specific.However, all POCTs evaluated met the threshold of 99.00% for specificity, regardless of the matrix used.The specificity for assays that used whole blood ranged from 99.50% (TR DPP HIV-1/2) to 99.94% (HIV 1/2 Colloidal Gold); and, for assays using plasma, it ranged from 99.17% (TR DPP HIV-1/2) to 99.50% (Imuno-Rápido HIV 1/2).It is interesting to note that for both assays simultaneously evaluated for plasma and whole blood samples (Imuno-Rápido HIV 1/2 and TR DPP HIV-1/2), the specificity when using plasma was lower than for whole blood.The variation of false-reactivity rates among different tests also strikes attention.Table 2 describes the clinical specificity of all POCTs evaluated.
Samples from three different individuals showed false-positive results on TR DPP HIV-1/2, for both plasma and whole blood sample matrices.One of those also presented a false-positive result when using oral fluid (Table 2).A similar observation was made for one individual that scored false-positive on Imuno-Rápido HIV 1/2, for both whole blood and plasma.
The clinical sensitivity of all POCTs evaluated is described on Table 3. Concerning clinical sensitivity, two POCTs failed to reach the criteria of 99.5%: Retrocheck HIV (98.95%), for whole blood sample, and Imuno-Rápido HIV 1/2, for both whole blood (98.00%) and plasma (97.50%).The other four tests (BD Chek™ HIV Multi-test, HIV 1/2 Colloidal Gold, OraQuick ADVANCE ® Rapid HIV-1/2 Antibody Test, and TR DPP HIV-1/2) met the sensitivity criteria.Samples from three different individuals showed falsenegative results on Imuno-Rápido HIV 1/2, for both whole blood and plasma matrices.Table 3 describes the clinical sensitivity of all POCTs evaluated.

Point-of-care tests performance using plasma panels
Different seroconversion panels were used along POCT evaluation rounds.In order to make all obtained results comparable, we standardized every panel member based on Fiebig's staging classification for primary HIV infection [9].As expected, none panel members classified on Fiebig's stages I and II were detected by any of the evaluated POCTs.Two panel members were classified as Fiebig's stage III; three POCTs (HIV 1/2 Colloidal Gold, BD Chek™ HIV Multi-test and Retrocheck HIV) misclassified one of the two members, and two POCTs (TR DPP HIV-1/2 and Imuno-Rápido HIV 1/2) misclassified both members.Three POCTs (HIV 1/2 Colloidal Gold, BD Chek™ HIV Multi-test and Retrocheck HIV) were not tested by panel members classified as Fiebig's stage IV.The remaining two POCTs (TR DPP HIV-1/2 and Imuno-Rápido HIV 1/2) correctly classified both Fiebig's stage IV panel members.Finally, with exception to Imuno-Rápido HIV 1/2, which missed two out of 12 panel members classified as Fiebig's stage V, all other POCTs classified properly the Fiebig's stage V panel members (Table 4).
Different HIV-1 subtype panels were used along POCT evaluation rounds.As shown on Table 5, the protocol was improved by adding panels containing members of HIV-1 group O and HIV-2 samples.With exception to Imuno-Rápido HIV 1/2, all other POCTs detected all HIV-1 subtype panels' members.The Imuno-Rápido HIV 1/2 (plasma) misclassified one out of seven HIV-1 subtype F panel members, the HIV-1 group O member, and four out of five HIV-2 panel members (Table 5).
Concerning the assay operational characteristics, all POCTs evaluated achieved the maximal score of 5/5, as presented on Table 6.The ongoing revision    of the HIV POCT Evaluation Program now places the assay's operational characteristics at a prequalification phase.Consequently, in future evaluations, any POCT that does not match this criterion will not be submitted to the evaluation protocol.

Results of point-of-care tests evaluation using oral fluid matrix
On the fourth and fifth rounds, two oral fluid based tests (OraQuick ADVANCE ® Rapid HIV-1/2 Antibody Test and TR DPP HIV-1/2) were evaluated.For the fifth round, sample quality was determined by its clear physical aspects.Among the 800 samples analyzed, 696 (87.0%) showed clear appearance, 5 (0.6%) had traces of blood, 33 (4.1%) had lipstick remnants, and 9 (1.1%) showed food residues.Physical aspects were not registered for fifty-one oral fluid samples.None of the samples with any kind of residues presented discordant results between TR DPP HIV-1/2 (oral fluid) and the gold standard tests used in the evaluation.
Regarding TR DPP HIV-1/2 (oral fluid) evaluation, false-negative and false-positive results were recorded for one sample only, resulting in a specificity of 99.83% and a sensitivity of 99.50%.The OraQuick ADVANCE ® Rapid HIV-1/2 Antibody Test (oral fluid), presented both specificity and sensitivity of 100% (Table 7).

Discussion
Since its implementation, the protocol used by the Brazilian HIV POCT Evaluation Program has undergone some improvements in order to optimize its running time and rationalize its financial costs.This study reports the lessons learned over the past 13 years and the results obtained.
When the program started, a costly protocol was used, based on 800 HIV-negative and 200 HIV-positive fresh whole blood samples.All these samples were submitted to the POCT and EIA.EIA reactive samples and those with discordant results between EIA and the POCT were further confirmed by Western blot.After the fourth round, the protocol was amended and, for samples originated from the HIV-positive individuals, the EIA and Western blot were only performed if the POCT was non-reactive.
Another important modification was the reduction from 800 to 600 HIV negative samples with no impact in the specificity assessment.Together, the changes in the protocol resulted in saving one quarter of time spent in each run and a cost cut of somewhat around 80%, depending on the cost of the Western blot relative to the cost of the EIA (here assumed to be 10 times greater).
In order to guarantee that the approved POCTs would be able to detect all HIV-1 subtypes circulating in Brazil, including group O, and HIV-2, the number of subtype panels was increased from one to three panels, including one panel with HIV-2 samples.The addition of the HIV-2 panel to the evaluation protocol aimed to assure that all POCTs used in the country would meet the requirement of detecting both HIV-1 and HIV-2 antibodies, imposed by ANVISA to all registered immunoassays.
The inclusion of seroconversion panel in the evaluation rounds aimed to identify the POCT with the highest sensitivity.This would help stakeholders to discriminate the most sensitive screening test, among assays with equal sensitivity score, to be used as first-line screening assay in the algorithm.
According to the POCT performance criteria established by the Brazilian Ministry of Health (Ministerial Directive MS/SVS n°29/2013), the following RT matched the minimum clinical sensitivity of 99.5%: BD Chek™ HIV Multi-test (whole blood), HIV 1/2 Colloidal Gold (whole blood), OraQuick ADVAN-CE ® Rapid HIV-1/2 Antibody Test (whole blood and plasma) and TR DPP HIV-1/2 (whole blood, plasma and oral fluid).Regarding the specificity criteria of 99.0%, all assays met this requirement.
The World Health Organization (WHO) initiated in 1988 the WHO HIV Test Kit Evaluation Programme, aiming to provide an objective evaluation of commercially available assays for HIV-1 and HIV-2 antibodies detection.In 2010, this program was superseded by the WHO Prequalification of in Vitro Diagnostics Programme, which is based on regulatory principles concerning performance, quality and safety of the assay.Briefly, such evaluations are based on serum/plasma panels comprising samples When compared to the results reported in the present study, we could observe small variations on sensitivity and specificity values for the evaluated tests.According to WHO analysis and assuming the MoH criteria for sensitivity and specificity, Retrocheck HIV would be disqualified and, on the other hand, OraQuick ADVANCE ® Rapid HIV-1/2 Antibody Test (serum/plasma) would be approved.It is worth to highlight that WHO analyses were run on serum/plasma panels whereas the present analyses were based on whole blood samples and this might explain the divergent results.
To the best of our knowledge, there is no data on literature regarding the evaluation of performance characteristics for BD Chek™ HIV Multi-test, TR DPP HIV-1/2 e Imuno-Rápido HIV 1/2.
It is important to highlight that the use of different sample matrices presents advantages and disadvantages.Theoretically, by choosing plasma as standard matrix for POCT evaluations, it is possible to compare directly the results obtained by different kits, which in turn might occur on the same or different moments.On the other hand, the use of fresh whole blood samples has the advantage of being the matrix actually used on field.In addition, it also permits the separation of plasma in order to provide POCT performance comparison based on both samples matrices obtained by the same individual.It is worth to mention that the accumulated data based on all evaluations showed that plasma matrices are more likely to report false-positive results.However, in the present study, three samples with Western blot negative results were false-positive for TR DPP HIV-1/2 on both whole blood and plasma and one of the samples was false-positive for oral fluid.A false-positive result was also found with the use of Imuno-Rápido HIV 1/2 for both whole blood and plasma sample.These results suggest that a test can present false-positive results regardless the sample matrix used.
The use of fresh whole blood samples also implied in some disadvantages, to note: (i) every sample had to be characterized by a reference assay testing algorithm, which elevates the cost of the evaluation programme; (ii) it limits the number of tests that can be done per day, once the samples are collected and tested on the same day; and (iii) every evaluation round is performed with different sample sets, making comparisons between the POCTs evaluated in different rounds more complicate.This is especially important for diagnostic algorithm determination.For instance, on round three, two samples presented false-positive results for both BD Chek™ HIV Multi-test and Retrocheck HIV.The two samples were Western blot negative, one was from a blood donor and the other from a pregnant woman.Therefore, the use of both POCTs in the same diagnostic algorithm would lead to a misdiagnosis.
Regarding the use of HIV positive samples from individuals in use of antiretroviral therapy, most manufacturers claim that treated individuals would generate a false-negative result because of the loss of HIV-specific antibodies.In addition, it is known that oral fluid presents 100 to 1,000 times less antibodies than serum/plasma.Together, such information might explain the false-negative result observed for oral fluid on round five, whereas the whole blood and plasma matrices of this same POCT and the reference assays have scored positive results.It is worth to mention that the patient, from which the sample was collected, was under antiretroviral therapy for more than six years.

Conclusions
The present study evaluated six tests along four rounds.Two tests (Retrocheck e Imuno-Rápido HIV 1/2) did not meet the MoH sensitivity criteria.Among the four approved tests -BD Chek™ HIV Multi-test (whole blood), HIV 1/2 Colloidal Gold (whole blood), OraQuick ADVANCE ® Rapid HIV-1/2 Antibody Test (whole blood and oral fluid) and TR DPP HIV-1/2 (whole blood, plasma and oral fluid)the last one (TR DPP HIV-1/2 for whole blood and oral fluid) was eventually acquired by the MoH for nationwide distribution.
For the last 13 years, the Brazilian experience on POCT evaluation has constituted an important management tool, allowing (i) to know the quality of commercially available POCTs in Brazil; (ii) to verify if the available products meet the criteria established by Brazilian legislation for sensitivity and specificity; (iii) to assist public stakeholders in choosing the tests to be acquired and countrywide distributed by the Brazilian National Health System; (iv) to assist on the design of HIV diagnostic algorithms that privilege the use of most sensitive tests for screening and more specific ones as confirmatory tests; and (v) to assist the POCT manufacturers to always improve their products, especially if those were reproved by any criteria.
The benefits of POCT use on public health can only be achieved by effective regulation, rigorous evaluation and correct use of the tests [5].The successful Brazilian policy on POCT use for HIV infection diagnosis includes the evaluation of the POCT itself in addition to appropriate selection of tests to be acquired and nationwide distributed to the public health facilities, control of each test batch distributed by the MoH, proper and easily accessible training to all health professionals involved in rapid testing through distance learning tools, and continued evaluation of POCT use through external quality assessment.Although STI POCs are generally simple and easy to perform, ensuring proper implementation, quality on POC testing and case management can help guarantee HIV testing quality in decentralized settings and identify where remedial training is required [16].

Table 1 .
Characteristics of the point-of-care rapid tests analyzed by the Brazilian HIV POCT Evaluation Program.
Daily results were transferred to an Excel spreadsheet (Microsoft Windows 2000; Microsoft Corp.,

Table 2 .
Clinical specificity of all point-of-care tests evaluated by the Brazilian HIV POCT Evaluation Program.
a : Clinical specificity; b : Lower limit of the 95% CI.C : Upper limit of the 95% CI

Table 3 .
Clinical sensitivity of all point-of-care tests evaluated by the Brazilian HIV POCT Evaluation Program.
a : Clinical sensitivity; b : Lower limit of the 95% CI.C : Upper limit of the 95% CI

Table 4 .
[9]luation of point-of-care tests against the seroconversion panels by the Brazilian HIV POCT Evaluation Program, according to Fiebig's staging classification for primary HIV infection[9].

Table 5 .
Evaluation of point-of-care tests against the subtype panels by the Brazilian HIV POCT Evaluation Program.
a : Number of positive samples detected by the POCT / total number of samples in the subtype category.

Table 6 .
Evaluation of the point-of-care test operational performance based on five characteristics.

Table 7 .
The clinical specificity and sensitivity of point-of-care tests, using oral fluid matrix, evaluated by the Brazilian HIV POCT Evaluation Program.