Clinical Performance of the Roche SARS-CoV-2 Serologic Assay

Abstract

In a study published in Clinical Chemistry, we compared the clinical performance of two serologic assays [Abbott and EUROIMMUN (EI)] for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) (1). Here, we characterize the clinical performance of the Roche Elecsys Anti-SARS-CoV-2 assay using the same specimens and validation protocol. The Roche assay measures total immunoglobulins directed toward a recombinant nucleocapsid protein from SARS-CoV-2. The assay reports a ratio of specimen electrochemiluminescent signal to calibrator. The final interpretation of positivity is defined as a cut-off index (ratio) of ≥1.0. Compared to SARS-CoV-2 PCR, the specificity of the Roche SARS-CoV-2 assay was 98.69% (95% CI 95.63–99.84) (Abbott: 99.34%; 96.41–99.98. EI: 94.77%; 89.96–97.72.) (Fig. 1, A). The sensitivity of the Roche assay at >14 d postonset of symptoms was 89.36% (76.9–96.45) (Abbott: 93.76%; 82.8–98.69. EI: 85.42%; 72.24–93.93). The sensitivity ranged from 50.0% (95% CI 34.19–65.81) at <3 d post-PCR testing to 75.0% (47.62–92.73%) at >14 d (Fig. 1, B). Relative to the Abbott assay, the total concordance was 0.89 (95% CI 0.81–0.95), the positive percentage agreement was 96.0%, and the negative percentage agreement was 95.2%. Relative to the EI assay, the total concordance was 0.84 (0.77–0.92), the positive percentage agreement was 95.8%, and the negative percentage agreement was 88.2% if borderline results were considered positive. Among patients with PCR confirmed SARS-CoV-2, at <14 d of symptom onset, the Roche assay detected 6 more true positives than the Abbott assay and 5 more true positives than the EI assay. However, at 14 d+ of symptom onset, the Roche assay detected 2 fewer positives when compared to Abbott. Three patients with immunodeficiencies were negative by all 3 assays after day 14 from symptom onset. Two patients were negative by molecular testing but positive by the Roche assay; the first was clinically adjudicated as COVID-19+ (open circle) and positive by all 3 assays while the second was negative by the Abbott and EI assays and not clinically adjudicated as COVID-19+. Clinical performance of the Roche SARS-CoV-2 Immunoassay. (A), Seropositivity in 153 expected negative specimens and 102 specimens from 48 patients with PCR-positive COVID-19 relative to days from onset of symptoms. (B), Seropositivity in 102 specimens from 48 patients with PCR-positive COVID-19 relative to days from testing positive by PCR. Pre-2019——50 specimens collected in 2015 and stored at −80 °C. Other Resp.——specimens from patients with PCR confirmed Influenza A (n = 2), influenza B (n = 2), other non-COVID-19 coronaviruses (n = 5, including Coronaviruses HKU1, NL63, and 229E). Other Int.——specimens from patients with positive CMV IgG (n = 5), EBV VCA IgG (n = 5), EBV VCA IgM (n = 3), Rheumatoid factor (n = 1). Symp. PCR——specimens from 80 patients presenting to the hospital with symptoms of respiratory infection and PCR negative for COVID-19. The large open circle represents a patient who was PCR negative but had symptoms consistent with COVID-19 and prolonged exposure to a family member with PCR confirmed COVID-19. Values in parentheses represent a 95% confidence interval. (C), Seropositivity relative to days of symptom onset. Dotted line represents the cutoff off for positivity (ratio ≥1.0). (D), Kinetics of the antibody response in 12 patients with serial samples. Time from symptom onset to positive result could not be calculated due to a lack of discernable kinetic pattern when the signal from serial specimens was examined (Fig. 1, C). The Roche assay demonstrated a wider analytic measuring range (ratio 0.089–112.0) relative to Abbott (0.01–7.6) and EI (0.2–10.0). However, the signal diminished over time in several patients with serial specimens (Fig. 1, D). Specimen pools at 3 concentrations (ratios 0.09, 1.29, and 54.42) had repeatability and total imprecision of <3.9%. No interference was noted up to a hemolysis index of 1000, lipemic index of 1000, or icteric index of 60 and no carryover was observed. The Abbott assay demonstrated the highest specificity and sensitivity for detection of antibodies to SARS-CoV-2 >14 d after onset of symptoms compared to Roche and EI, but these findings were not statistically different. Importantly, the 3 assays evaluated do not have sufficient clinical sensitivity for detection of SARS-CoV-2 prior to 14 d from symptom onset to be useful for acute diagnosis. Our results (1,) demonstrate inferior clinical performance (particularly with regards to sensitivity) of SARS-CoV-2 immunoassays than that reported by the manufacturers. Our studies used specimens from acutely ill hospitalized patients, which may have differed from the patient populations used in studies performed by the manufacturers. Nonetheless, serological assays will likely be used for both acutely ill and recovered healthy patients, and assay validations should be performed using both patient populations (2). Despite the Abbott and Roche assays both targeting the SARS-CoV-2 nucleocapsid protein, modest differences in sensitivity and specificity were noted, possibly reflecting variation in assay design. The Roche assay is unique since it measures total immunoglobulin (i.e., IgG, IgA, and IgM) as opposed to IgG (Abbott and EI). This likely explains why the Roche assay was able to detect antibodies to SARS-CoV-2 earlier than the EI and Abbott assays. Furthermore, the potential to detect multiple classes of immunoglobulins likely underlies the lack of discernable pattern in the kinetics of serial specimens. Paradoxically, fewer positive results were observed in patients with >14 d of symptoms on the Roche assay relative to the Abbott assay. All 3 specimens that were positive by Abbott and negative by Roche had a modest signal on the Abbott assay (ratio 3–4). Therefore, the antibodies detected in these patients may have lower affinity and are unable to bind to the antigen targets in the Roche assay. To this end, Roche has stated in publications distributed to users that the assay primarily targets high-affinity antibodies to increase specificity. However, the robust signal detected in several patients early in symptomatology (<7 d) and rapidly decreasing signal from days 7–28 may be more consistent with lower affinity IgM antibody binding on the Roche assay. Larger studies must be performed to confirm this finding. In conclusion, the Abbott assay demonstrated the fewest false negative results >14 d postsymptom onset and the fewest false positive results. While the Roche assay detected more positive results earlier after onset of symptoms than the other assays, none of the assays demonstrated high enough clinical sensitivity before day 14 from symptom onset to diagnose acute infection. Nonetheless, the clinical performance between the Roche, Abbott, and EI SARS-CoV-2 assays are similar and can detect antibodies to SARS-CoV-2 in a majority of patients 14 d after the onset of symptoms. All authors confirmed they have contributed to the intellectual content of this paper and have met the following 4 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; (c) final approval of the published article; and (d) agreement to be accountable for all aspects of the article thus ensuring that questions related to the accuracy or integrity of any part of the article are appropriately investigated and resolved. K.G. Hock, provision of study material or patients; N.M. Logsdon, statistical analysis, provision of study material or patients; C.W. Farnsworth, statistical analysis. Authors’ Disclosures or Potential Conflicts of Interest: Upon manuscript submission, all authors completed the author disclosure form. Disclosures and/or potential conflicts of interest: Employment or Leadership: A.M. Gronowski, Clinical Chemistry, AACC. Consultant or Advisory Role: N.W. Anderson, Diasorin Molecular. Stock Ownership: None declared. Honoraria: None declared. Research Funding: None declared. Expert Testimony: None declared. Patents: None declared.

Topics & Concepts