The objective of this study was to review the diagnostic accuracy of the Aptima Combo 2 (AC2) assay on the PantherTM system for reporting Neisseria gonorrhoeae (NG) detection from routine clinical samples. Results for samples routinely submitted for the diagnosis of NG infection from two laboratories in Australia were retrospectively reviewed from September 2011 to May 2013. Each sample was initially tested on a Panther instrument with the AC2 assay. Samples positive or equivocal for NG in AC2 were retested with the Aptima Neisseria gonorrhoeae assay (AGC) on the Panther instrument. AC2-NG positive results were considered false positive if the AGC result was negative. AC2 results were available for 91,217 samples, 1,434 of which were initially AC2 non-negative for NG and included an AGC result. The overall prevalence of NG was 1.5% (2.8% in males and 0.7% in females). Overall, 16% of all samples were from extragenital specimens. Overall, 0.02% of all samples (14/91,217) were false positives (0.05% of extragenital samples and 0.01% of urogenital samples). Without supplementary testing with the AGC assay, 14/91,217 false positive results would have been reported as positive, representing 0.02% of all samples tested and 1.0% of initially non-negative samples. This study demonstrates that the initial AC2 results from all sampling sites (urogenital and extragenital) can be accepted with high confidence.
Keywords: Aptima, Panther Instrument; Neisseria; Confirmatory testing
In Australia, the 2005 Public Health Laboratory Network (PHLN) guideline  recommends repeat testing of all initial positive Neisseria gonorrhoeae (NG) results from Nucleic Acid Amplification Tests (NAAT) obtained with a commercial assay using a second suitable alternate NAAT assay. Under the PHLN guideline, if the results from both assays are positive, the patient is considered infected. However, if the alternate assay is negative, the initial result is considered a false positive result and the patient is considered as non-infected. A second recommendation is that any assay being used should have a positive predictive value (PPV) of at least 90% in the test population. Recognizing the difficulties that laboratories may encounter in accurately calculating this value due to varying prevalence by population, the guideline recommends that a population prevalence of 1% be used in calculations to represent a low prevalence population.
Many publications have debated the necessity to confirm all initial results [2-4]. Analytical studies have also produced conflicting results. The study by Tabrizi showed false positive results with non-gonococcal Neisseria species for a number of NG assays including the Aptima assays . However, a later, very similar study documented 100% sensitivity and specificity for the Aptima assays . Current guidelines in other regions of the world differ. The United Kingdom National Guideline for Gonorrhoea Testing 2012 recommends that positive NAATs from extragenital sites and low prevalence populations be confirmed by supplementary testing that uses a different nucleic acid target . The test used also must have a PPV of at least 90%. Guidelines in the United States are similar. The recommendations from the Association of Public Health Laboratories 2009 are that “routine repeat testing of NAAT positive specimens is not recommended for N. gonorrhoeae unless there is a significant number of false‐positive test results found in clinical specimens due to cross‐reaction with non‐gonococcal Neisseria species” . However, this document does not define what constitutes “a significant number of false‐positive test results.”
In Australia, most, if not all laboratories follow the guidelines of the PHLN . The objective of the present study was to review the diagnostic accuracy of the Aptima Combo 2 (AC2) assay for reporting NG detection and the Aptima Neisseria gonorrhoeae (AGC) assay for confirmation on the PantherTM system from a large number of routine clinical samples from two laboratories in Australia.
Materials and Methods
Both laboratories (in Sydney, New South Wales, and in Adelaide, South Australia) provide diagnostic services to patients of community general practice clinics and patients of specialty health care facilities such as hospitals and sexual health clinics. The laboratory in Sydney receives most of its samples for sexually transmitted infections (STI) detection from an associated sexual health clinic while the laboratory in Adelaide receives only 25% of its samples from sexual health clinics. Samples received were from urogenital (cervical swabs, vaginal swabs, urethral swabs, and urine) and extragenital sites (including oropharyngeal swabs, rectal swabs, eye swabs, and joint fluids). Some patients considered at high risk for NG had samples collected from more than one site. All samples were submitted for routine analysis using the AC2 assay.
No ethics approval was required for this study as all samples were routine samples for the detection of NG. All data were de-identified and collected from the respective laboratory information systems of each facility. No assessment of whether patients were symptomatic or asymptomatic was made. Data management was performed using Microsoft Access, while R (Free Software Foundation’s GNU General Public Licence) was used for statistical analysis.
NG Detection Assays
The AC2  and AGC  assays are target amplification nucleic acid probe tests that utilize target capture for the in vitro qualitative detection and differentiation of ribosomal RNA (rRNA) from Chlamydia trachomatis (CT) and NG. The NG assays both target the 16S rRNA subunit for capture and detection. The capture probe is the same for both assays, but the AGC assay recognizes a different region of the 16S rRNA subunit than the AC2 assay for detection, which makes it a suitable assay for the confirmation of AC2-NG positive results. If the curve shape has the kinetic type of NG, a sample is considered AC2-NG negative if its relative light unit (RLU) value is <60, equivocal if its RLU value is 60 to <150, and positive if its RLU value is ≥150. In the AGC assay, samples with RLU values <50 are considered negative, those with 50 to <100 RLU equivocal, those with 100 to <2,000 RLU low positive, and those with 2000 to <12,000 RLU positive. Both assays can be performed from the same sample preparation on the Panther instrument (Hologic Inc., United States), a random access molecular diagnostics platform. Analysis of test results is performed automatically by the Panther assay software. The results can be viewed on the instrument, and printed and/or uploaded to the laboratory information system via a bidirectional interface.
All samples were handled and processed within the timeframe specified by the manufacturer. Although the Aptima package
insert states that “the first valid result for each analyte is the result that should be reported,” the interpretation of the NG results was made following the PHLN guideline. Samples were tested first with the AC2 assay, and if the results were not negative (i.e., positive or equivocal for NG) they were retested with the AGC assay.
Results were considered initially concordant if they were identical for both assays. Possible concordant combinations were AC2-NG positive – AGC positive (considered positive; concordant) and AC2-NG equivocal – AGC equivocal (considered indeterminate; concordant). Results were considered initially discordant if they were not identical for both assays. Discordant results were categorized into “Minor Discordant” and “Major Discordant.” Possible discordant result combinations were: AC2-NG positive – AGC equivocal (considered indeterminate; minor discordant), AC2-NG positive – AGC negative (considered negative as per PHLN guideline - initial result false positive; major discordant), AC2-NG equivocal – AGC negative (considered negative - initial result false; minor discordant), and AC2-NG equivocal – AGC positive (considered positive; minor discordant).
Data were grouped by sample type for analysis. For example, sample descriptions such as “urethral swab”, “urethral discharge”, “urethral,” and “penile swab” were all grouped as “Urethra” in the analysis; those described as “throat swab”, “mouth swab”, “oral swab,” or “oral” were all grouped as “Oropharyngeal,” and any non-descript samples such as “swab”, “wound”, “skin”, “pus”, “fluid” etc. were grouped as “Other”.
Overall, 9,321 sample results (72.3% from male patients) were available from the Sydney laboratory from September 2011 to December 2012, and 81,896 results (36.8% from male patients) were available from the Adelaide laboratory from October 2011 to May 2013. In total, 91,217 sample results were available for review, 40.5% of the samples were from male patients. Ages ranged from <1 to 97 years of age. Of the 91,217 samples, 16% (14,744) were from extragenital sites, with the majority coming from oropharyngeal (6,133) and rectal swabs (5,241), and the remainder from eye swabs (305), joint fluids , and various other miscellaneous or undocumented sites (3,053).
Prevalence of NG
The overall prevalence of NG was 1.5% (2.8% in male samples and 0.7% in female samples). Figure 1 shows the distribution of positive results and the rates of detection in each age group. NG prevalence was higher in males than in females across all age groups. The highest prevalence was observed in the 16 to 20 year age group for females, and in the 26 to 30 year age group for males.
Analysis of Discordant Results
Table 1 shows the test results by sample type: samples with manufacturer’s validation claims (urine, ThinPrep, vaginal, endocervical, and urethral swabs), and extragenital samples (including oropharyngeal, rectal, eye, and joint fluids). Of the 91,217 samples analyzed, 14 (0.02%) had major discordant results (AC2-NG positive – AGC negative); 6 (0.01%) were urogenital samples (3 cervical, 1 vaginal, and 2 urine samples) and 8 (0.05%) were extragenital samples (6 oropharyngeal, 1 rectal, and 1 other sample) (Table 1). Samples from male patients had more discordant results than those from female patients (8 versus 5, respectively).
Additional laboratory data was sought for these major discordant results, including culture for NG, AC2 results from other sampling sites on the same day, and results of cultures and AC2 from additional samples.
Of the three cervical samples with major discordant results, all 3 subjects had culture performed from vaginal swabs collected the same day, which were negative for NG. Two of these samples had low RLU values in AC2 (359 and 304). The major discordant result from a vaginal swab was from a 43 year old with a low RLU value of 472 who had a negative culture result for NG on a sample collected the same day. The remaining major discordant sample from a female patient was from a swab sample from an 82 year old with a non-descript identification – “swab”. The RLU value for this sample was 1225. This patient had a conjunctival sample cultured for NG the same day that was negative. The AGC RLU values for all of the abovementioned samples were ≤ 11.
All 6 of the oropharyngeal samples with major discordant results were from males. The RLU values for these samples ranged from 197 to 531. One of these 6 patients had positive
Figure 1. NG preva lence by age group. Large dark bars indicate the percent of positive (prevalence) within each age group (left Y axis). Small grey bars show the number of positive tests within each age group (right Y axis).
AC2=Aptima Combo 2 assay; AGC=Aptima Neisseria gonorrhoeae
assay; P=Positive; EQ=Equivocal; N=Negative.
Table 1. Assay results summarized by sex, sample site, and AC2+AGC results
AC2 results confirmed by AGC for rectal and urine samples collected the same day; culture for NG was not performed. A second patient had a positive oropharyngeal sample 2 weeks later. A third patient had negative rectal and urine samples collected the same day; culture for NG was not performed. A fourth patient had negative rectal and urine samples collected the same day, but had a positive oropharyngeal sample 2 weeks later. The fifth patient had negative results for rectal and urine samples collected the same day, and negative a NG culture from an oropharyngeal sample collected a week later. The last (sixth) patient had confirmed positive results from rectal and “other” samples on the same day.
The one rectal swab with major discordant result was from a 27 year old male. He was confirmed positive in a urethral sample collected the same day and had a negative oropharyngeal sample. Five months prior to the discordant result, the patient had a positive rectal sample along with negative oropharyngeal and urethral samples. Five months after the discordant result, the patient had a positive rectal sample and negative urethral and oropharyngeal samples.
There was one urine sample with a major discordant result. This sample had a RLU value of 428 in AC2; this sample under went 2 additional AC2 tests with RLU values of 146 and 150, and 2 additional AGC tests with RLU values of 24 and 45. Culture of this sample was negative for NG.
Nine oropharyngeal samples (8 male, 1 female) produced minor discordant results; AC2 positive – AGC equivocal. The female patient had a negative urine sample collected the same day but was positive two months later in a second oropharyngeal sample. One male patient had a positive rectal sample collected the same day. This patient had a previous positive oropharyngeal sample 5 months prior but was negative 1 week after this discordant sample. A second male patient had a negative rectal sample from the same day and was negative in an oropharyngeal sample 1 week later. None of these three patients had NG cultures performed. All of the remaining 6 male patients had negative NG cultures from oropharyngeal samples collected up to 2 weeks later.
Two urine samples from female patients had AC2 positive – AGC equivocal results. The first did not have culture performed
and had RLU values of 295 and 92 for the AC2 and AGC assays, respectively. The second sample was culture negative from an endocervical sample and had RLU values of 237 and 93 for the AC2 and AGC assays, respectively.
One urethral sample had AC2 positive – AGC equivocal results with RLU values of 184 and 94, respectively. This patient had a positive rectal sample on the same day.
A summary of the performance of the AC2 assay for validated and non-validated sample types is shown in Table 2. The relative risk of obtaining a major discordant result from a non-validated sample type compared to that from a validated sample type was not statistically significant; RR = 1.85 (95% confidence interval [CI] (0.64 to 5.29), P = 0.254). For the entire data set, the overall major discordant rate was 0.02% of all samples (14/91,217). False positive results occurred with 0.05% (8/14,744) of all extragenital samples compared with 0.01% (6/76,473) of all urogenital samples. For just the samples initially reactive with AC2, these discordant rates were 1.33% and 0.72% for extragenital and urogenital sites, respectively.
AC2=Aptima Combo 2 assay; AGC=Aptima Neisseria gonorrhoeae assay;
Table 2: Rates of AC2 – AGC concordance from validated and non-validated sites
A calculation of the PPV for the AC2 assay is possible. As previously defined, samples with AC2 positive – AGC positive or AC2 equivocal – AGC positive results are considered true positives, while samples with AC2 positive – AGC negative results are considered false positives. The samples that have AC2 equivocal – AGC negative results are considered negative as per the assay package insert [10, 11]. Twenty-two indeterminate samples (with AC2 positive – AGC equivocal or AC2 equivocal – AGC equivocal results) were excluded from the calculations. The assay package insert recommends that a new sample be collected for these patients [10, 11]. Using these definitions, AC2 PPV was 99.0% for all samples and 98.6% for extragenital samples only.
This is the first study of the performance of the Panther – Aptima assay combination for the detection of NG from routine clinical samples in Australia. This study assesses the clinical accuracy of the AC2 assay as a screening assay as per the PHLN guideline. The data set is large, from a geographically diverse population, and includes a significant number of samples from extragenital sites not validated by the manufacturer.
The definition applied for the patient infected status here is consistent with that of the PHLN guideline ; that is, if the result from a suitable second assay is not positive, the sample must be reported as negative for NG. This very conservative definition of a positive result protects against reporting potentially false positive results. However, a review of additional test data and clinical information for some of the discordant results presented here suggests that some of these “false positive results” may have in fact been true positive results. This could potentially result in an undiagnosed reservoir of NG leading to the transmission to others. The relative insensitivity of culture and the problems of the inclusion of this result into the definition of the patient infected status have been well documented .
Different guidelines exist in different regions of the world based on peer-reviewed publications of local data and the input of key opinion leaders [7, 8]. All agree that the screening assay must be fit for use in a low prevalence population.
A limitation of the present study is that, as not all samples had both tests performed, the number of true negative results may not be accurate. Some NG infections may have been missed by the AC2 assay, which would increase the population prevalence. As is true of all non-culture based detection methods, the detection of nucleic acid does not necessarily indicate a current active infection. Moreover, this study did not assess the treatment status for all patients. The persistence of rRNA following treatment has been well documented  and this may potentially influence calculations of disease prevalence. The PHLN guideline  suggests that a value of 1% be used to define a low prevalence population for the purposes of calculating the PPV. The hypothetical PPV for a population with a NG prevalence of 1% is 55.3% based on data from North America . The population in this study had a prevalence of 1.5%. It is noted that this study included mixed groups and that some of these groups (e.g., clients of sexual health clinics and patients from certain regions of Central Australia) were expected to have higher rates of NG prevalence . The specificity, PPV, and prevalence values were 99.97%, 99.2%, and 4.0 %, respectively, for the laboratory in Sydney, and 99.99%, 98.9%, and 1.3%, respectively, for the laboratory in Adelaide. The specificity, PPV and prevalence for the entire dataset is 99.98%, 99.0%, and 1.5% respectively. Thus, the AC2 assay meets the criteria for use as a screening assay listed in the various guidelines [1, 7, 8].
Another potential limitation of this study is that culture results were not available for all of the discordant AC2 results. Many studies have shown that in comparison to culture, NAATs have superior sensitivity in the detection of urogenital and extragenital STI infections [13-15, 17]. It is therefore unlikely that additional infections would have been diagnosed had culture data been included.
No assay has yet received FDA clearance for use with extragenital samples even though STI guidelines [8, 16] recommend the use of these assays for use with these sample types. Calculation of the PPV in extragenital sites meets the PPV requirement at 98.6%. Diagnostic laboratories routinely receive a significant number of oropharyngeal, rectal, and eye swab samples for STI diagnosis. Therefore, there is a need for manufacturers to gain approval for NAATs for STI diagnosis from these extragenital sites to support the end-user laboratory and provide confidence to patients and treating physicians that the diagnostic assay is accurate.
The performance of the AC2 assay as a stand-alone assay for NG screening was very good. Without testing with the alternate AGC assay (which is recommended in the PHLN guideline  ), only 14 false positive results would have been reported as positive, representing 0.02% of all 91,217 samples tested. Thirty-four samples yielded results defined as a minor discordant and should be interpreted in conjunction with other clinical and assay results and consideration of the patient’s history. Calculation of assay specificity is based on the assumption that, if significant numbers of positive samples were not missed by the AC2 assay, the specificity of the assay is 99.98%. This is a significant finding considering the numbers of extragenital samples tested that are known to harbor organisms that can potentially cross-react in NAATs. The rate of false positive results in urogenital samples and extragenital samples was not statistically significant different.
The use of the AGC assay as a secondary assay in the diagnostic algorithm for NG detection confirms the findings of others  and supports the use of the AC2 assay as an appropriate choice for primary NG screening in urogenital and extragenital specimens. The accumulation of data from extragenital testing should contribute to the approval of these sample types with the appropriate regulatory bodies. This should reduce the burden on users who are required to validate these NAAT assay–sample type combinations.
The authors thank Florence Paillard for her editorial assistance.