Results
Of 543 patients enrolled in the study, 14 did not have matching PCR/ESI-MS or standard-of-care microbiology results and were excluded from the final analysis. Table 1 shows the patient demographics, reflecting a typically heterogeneous ICU population: one third of the patients were admitted from the emergency department; 75% were exposed to one or more antibiotics prior to study enrolment. Overall mortality was 29%, with cardiac arrest, septic shock, multiple organ failure, and acute respiratory distress syndrome accounting for ~62% of deaths.
BSI Analysis
A total of 616 direct whole-blood specimens from the 529 patients were tested to assess the accuracy of organism identification. PCR/ESI-MS results from analysis of blood using the bacteria, antibiotic resistance, and Candida BSI assay were compared with results from standard clinical microbiology cultures. As shown in Table 2, there were 228 PCR/ESI-MS positive specimens (36.5%) for at least one pathogen compared with 68 positive specimens by culture (10.9%). The total number of positive tests for each method was statistically different (McNemar test statistic = 137.6; df = 1; p < 0.0001). There were 55 samples that were positive for the same organism with both techniques (Table 2), yielding an overall concordance of identification (calculated sensitivity) of 81% (95% CI, 70–89%) and a κ value of 0.25 (95% CI, 0.18–0.31). In 13 instances, culture identified an organism that was either negative by PCR/ESI-MS (6/13) or the identity of the organism reported by PCR/ESI-MS did not match the organism identified by microbiology testing (7/13) (Table S1, Supplemental Digital Content 1, http://links.lww.com/CCM/B418). In contrast, PCR/ESI-MS reported a BSI-relevant organism in 173 additional specimens that were culture negative, resulting in a calculated assay specificity of 69% (Table S2, Supplemental Digital Content 1, http://links.lww.com/CCM/B418). Finally, there were 384 concordant negative specimens, yielding a negative predictive value (NPV) of ~97% (95% CI, 94–98%).
The frequencies of organisms detected from BSI specimens are shown in Figure 1. The distributions of the top 10 species detected by microbiology and those detected by PCR/ESI-MS were similar. The largest single discrepancy between the two methods by sheer volume of detections was in the identification of Escherichia coli. Although culture and PCR/ESI-MS techniques both reported E. coli as the most abundant species, PCR/ESI-MS detection was 4-fold higher (89 vs 21). Other organisms in which blood culture performed less well included the enterococcus species, Enterococcus faecalis (1 vs 10) and Enterococcus faecium (2 vs 25), Candida albicans (2 vs 13), and Staphylococcus aureus (14 vs 31). In contrast, Pseudomonas aeruginosa detection was comparable between the two methods (6 vs 8). Additional analysis of the PCR/ESI-MS results showed that the levels (genome equivalent/mL) of organisms reported in the majority of these PCR/ESI-MS positive, but culture-negative, cases were similar to cases in which culture matched PCR/ESI-MS detections (data not shown).
(Enlarge Image)
Figure 1.
Bacteria and Candida detected in the Rapid Diagnosis of Infections in the Critically Ill (RADICAL) study. Distribution of organisms reported by polymerase chain reaction/electrospray ionization-mass spectrometry (PCR/ESI-MS) (blue bar) and culture (red bar) observed in the RADICAL study are shown, sorted by decreasing order of PCR/ESI-MS reported organisms. Both methods showed similar distribution for the top eight reportable organisms that were seen >5 times by PCR/ESI-MS, with some minor reshuffling of the order. PCR/ESI-MS showed a longer tail of reportable organisms that were infrequent (≤5 times). Normal skin flora are shown below the line were not included in further analysis by either method.
When subsets of specimens were tested in replicate using independently collected samples, 52 of 168 replicate samples were positive in both PCR/ESI-MS tests (Table S3, Supplemental Digital Content 1, http://links.lww.com/CCM/B418). In contrast, culture was positive in the two replicates in only 12 of 158 samples, all of which were also PCR/ESI-MS positive in both samples. In three instances, culture was positive in one of two replicates and was negative by PCR in both samples tested (false negative by PCR), whereas in three and four cases, respectively, when culture was positive in one replicate, PCR was positive in one or both replicates (Table S4, Supplemental Digital Content 1, http://links.lww.com/CCM/B418).
Nonbloodstream Infections
Heterogeneous samples from patients with suspected pneumonia or sterile site infections were also obtained in several cases. Overall, there were 185 LRT samples (88 BAL, 96 ETA, and 1 other) and 110 SF&T samples (36 intraperitoneal fluid, 14 pleural fluid, 11 CSF, 13 tissue, and 36 other fluid types). Results from the analysis of these specimens are shown in Table 3. LRT and SF&T specimens often had multiple detections reported by both methods in several samples. Only the primary detections by either method were included in the analysis. The overall sensitivities for concordance between standard-of-care and PCR/ESI-MS were 84% (95% CI, 74–91%) and 85% (95% CI, 72–93%), respectively. As for the bloodstream infection data, the McNemar test for both the LRT and the SF&T sample data showed that the total number of samples considered positive was significantly different for culture versus PCR/ESI-MS (McNemar test statistic = 20.9 for LRT and 15.2 for SF&T; p < 0.0001 in both cases). Also similar to the bloodstream infection data, there was more agreement in the contingency table comparing culture to PCR/ESI-MS than would be expected by chance (LRT κ = 0.35; 95% confidence limits, 0.23, 0.47 and SF&T κ = 0.27; 95% confidence limits, 0.11, 0.43). For LRT specimens, there was no statistically significant difference in sensitivity (p = 0.677) or specificity (p = 0.444) when testing the hypothesis that the BAL proportion – the ETA proportion was equal to zero.
In 151 patients, two or more specimen types (BSI plus LRT and/or SF&T) were obtained and analyzed. In 86 of these 151 cases (57%), the same organisms were reported by PCR/ESI-MS in all samples tested from an individual patient (data not shown). In comparison, culture concordance between the sample types was seen in only 19 cases (12%), driven largely by no detection reported in the BSI culture results.
Resistance Markers
There were no identified cases of Klebsiella-associated carbapenemase. There was a single report of vancomycin-resistant Enterococci, which was matched across the two methods. There were 23 reports of mecA+ staphylococcus organisms (seven in BSI samples, 13 in LRT samples, and three in SF&T samples), with the following agreement between PCR/ESI-MS and culture: for BSI samples, results were concordant in four cases, and PCR/ESI-MS was positive and culture negative in three; for LRT samples, results were concordant in three cases, PCR/ESI-MS was positive with culture negative in nine, and PCR/ESI-MS was negative with culture positive in one; and the three cases in the SF&T samples were concordant across PCR/ESI-MS and culture.
Clinical Value Assessment
To assess the clinical significance of the PCR/ESI-MS detections, each case was adjudicated by a panel of three independent clinical expert reviewers randomly selected from a pool of seven infectious disease, microbiology, and intensive care specialists not associated with the study sites. The panel was provided with clinical case summaries, and PCR/ESI-MS results and standard-of-care results from all samples tested. To identify potential changes in antimicrobial management that may have occurred if the results from the PCR/ESI-MS technology had been available for clinical use, the panel was provided with a questionnaire (Table 4). Analysis of the reviewers' independent responses was performed using a majority rule such that two of three responses for a given patient determined the outcome for that patient. Table 4 provides a breakdown of the responses by each question. Combining all responses per patient and eliminating overlapping answers so that only a single response was recorded per patient gave an overall recommendation for change based on PCR/ESI-MS compared with standard-of-care in 178 patients (41%). This percentage increased to 57% for cases in which PCR/ESI-MS results were positive and microbiology results were negative.