Browse the corpus

Cardiovascular revascularization procedural data are routinely collected to assess procedural quality, create risk adjustment tools, and assess outcomes in populations not well-studied in clinical trials. The National Cardiovascular Data Registry (NCDR) and the Society for Thoracic Surgeons (STS) data sets are widely used for these purposes,1, 2 and their data collection instruments are utilized as “off-the shelf” tools to facilitate standardized data capture. Correct identification of dialysis-dependent end-stage renal disease (ESRD) is particularly important because ESRD is a potent risk factor for cardiovascular mortality and procedural complications.3, 4, 5, 6, 7 Incorrect identification could impact risk-adjusted quality reporting for cardiac procedures as well as the retrospective analyses widely used to assess revascularization outcomes in dialysis patients. However, to our knowledge, the STS and NCDR instruments for identification of dialysis patients have not been validated. We assessed accuracy of dialysis identification by linking United States Renal Data System (USRDS) data to Massachusetts Data Analysis Center statewide data collected using the STS and NCDR instruments under a legal mandate requiring universal data capture on all patients undergoing coronary artery bypass graft (CABG) and percutaneous coronary intervention (PCI).8, 9

y linking United States Renal Data System (USRDS) data to Massachusetts Data Analysis Center statewide data collected using the STS and NCDR instruments under a legal mandate requiring universal data capture on all patients undergoing coronary artery bypass graft (CABG) and percutaneous coronary intervention (PCI).8, 9 Results Study Characteristics We identified 26,317 individuals undergoing CABG, and 99,848 undergoing PCI. The mean age was 66.8 ± 10.5 years in the CABG group, and 64.7 ± 12.6 years in the PCI group. Subjects were primarily White (CABG 89.9%; PCI 88.9%), with 55.1% of CABG and 71.7% of PCI patients admitted with acute coronary syndrome. Many procedures were performed urgently (CABG 62.7%; PCI 45.3%). Emergent or salvage procedures were rare for CABG (2.9%) but not PCI patients (23.9%). Diabetes, heart failure, and hypertension were common (Table 1).Table 1 Overall population

5.1% of CABG and 71.7% of PCI patients admitted with acute coronary syndrome. Many procedures were performed urgently (CABG 62.7%; PCI 45.3%). Emergent or salvage procedures were rare for CABG (2.9%) but not PCI patients (23.9%). Diabetes, heart failure, and hypertension were common (Table 1).Table 1 Overall population Variable CABGa (N = 26,317) PCIb (N = 99,848) N % N % Demographics and insurance Age (mean ± SD), yr 66.8 ± 10.5 64.7 ± 12.6 Male 20,018 76.06 69,090 69.20 Race White 23,666 89.93 88,733 88.87 Black 623 2.37 2770 2.77 Other 2028 7.71 8345 8.36 Insurance payorc Private 11,440 43.47 50,145 50.22 Government 14,146 53.75 46,482 46.55 Other 644 2.45 3221 3.23 Dialysis status Dialysis identified in the Mass-DAC instruments 431 1.64 1432 1.43 Dialysis identified through USRDS data 295 1.12 950 0.95 Baseline medical conditions Acute coronary syndrome 14,493 55.09 71,596 71.70 Diabetes 10,412 39.56 29,084 29.13 Heart failure 4631 17.60 11,446 11.46 Hypertension 22,145 84.15 75,354 75.47 Hypercholesterolemia 22,950 87.21 78,101 78.22 Peripheral vascular disease 4291 16.31 11,428 11.45 Prior myocardial infarction 13,183 50.09 21,615 21.65 Prior PCI 3179 12.08 15,341 15.36 Prior CABG 408 1.55 10,613 10.63 CABG or PCI performed at a teaching hospital 22,073 83.87 77,472 77.59 Urgent status 16,507 62.72 45,210 45.28 Emergent or salvage status 753 2.86 23,885 23.92 Cardiogenic shock 217 0.82 2082 2.09 Hospital characteristics Hospital procedural volume Low 5000 19.0 5043 5.05 Medium 9414 35.8 26,546 26.6 High 11,903 45.2 68,259 56.18 CABG, coronary artery bypass grafting; Mass-DAC, Massachusetts Data Analysis Center; PCI, percutaneous coronary intervention; USRDS, US Renal Data System.

genic shock 217 0.82 2082 2.09 Hospital characteristics Hospital procedural volume Low 5000 19.0 5043 5.05 Medium 9414 35.8 26,546 26.6 High 11,903 45.2 68,259 56.18 CABG, coronary artery bypass grafting; Mass-DAC, Massachusetts Data Analysis Center; PCI, percutaneous coronary intervention; USRDS, US Renal Data System. a For CABG, hospital procedural volume is defined as low ≤1418, medium >1418 to ≤2175, high >2175. b Hospital procedural volume for PCI is defined as low ≤914 cases in total during the study period, medium >914 to ≤4625, high ≥4625. c Eighty-seven (0.33%) CABG patients were missing payor. Dialysis Identification After excluding patients with kidney transplants (N = 49 for CABG; N = 193 for PCI), 295 of 26,268 (1.1%) CABG patients were identified by USRDS as having dialysis-dependent ESRD (Table 2). Of these, 278 (94.2%) were correctly identified by the STS instrument, and 17 (5.8%) were not. Conversely, 147 of 25,973 (0.6%) non–dialysis patients were incorrectly flagged as dialysis patients at the time of their procedure. Sensitivity for identification of dialysis-dependent ESRD was 94.2%, specificity 99.4%, positive predictive value 65.4%, and negative predictive value 99.9%.Table 2 Sensitivity and specificity for identification of chronic dialysis patients

s were incorrectly flagged as dialysis patients at the time of their procedure. Sensitivity for identification of dialysis-dependent ESRD was 94.2%, specificity 99.4%, positive predictive value 65.4%, and negative predictive value 99.9%.Table 2 Sensitivity and specificity for identification of chronic dialysis patients Identification of dialysis in the USRDS and Massachusetts state revascularization data Dialysis according to Mass-DAC coronary artery bypass–STS data Dialysis identified in USRDS No Yes Total No 25,826 17 25,843 Yes 147 278 425 Total 25,973 295 26,268 Sensitivity (%) Specificity (%) Positive predictive value (%) Negative predictive value (%) 94.2 99.4 65.4 99.9 Dialysis according to Mass-DAC percutaneous coronary intervention–NCDR data Dialysis identified in USRDS No Yes Total No 98,161 74 98,235 Yes 544 876 1420 Total 98,705 950 99,655 Sensitivity (%) Specificity (%) Positive predictive value (%) Negative predictive value (%) 92.2 99.5 61.7 99.9 Mass-DAC, Massachusetts Data Analysis Center; NCDR, National Cardiovascular Data Registry; STS, Society of Thoracic Surgeons; USRDS, US Renal Data System.

61 74 98,235 Yes 544 876 1420 Total 98,705 950 99,655 Sensitivity (%) Specificity (%) Positive predictive value (%) Negative predictive value (%) 92.2 99.5 61.7 99.9 Mass-DAC, Massachusetts Data Analysis Center; NCDR, National Cardiovascular Data Registry; STS, Society of Thoracic Surgeons; USRDS, US Renal Data System. There were 950 of 99,655 (0.95%) PCI patients on dialysis identified by the USRDS. Of these, 876 (92.2%) were correctly identified, and 74 (7.8%) were not identified by the NCDR instruments. Of 98,705 individuals not on dialysis, 544 (0.6%) were incorrectly flagged as receiving chronic dialysis at the time of their procedure. Sensitivity was 92.2%, specificity 99.5%, positive predictive value 61.7%, and negative predictive value 99.9%. A supplementary analysis (Supplementary Figure S1 and Supplementary Results) identified dialysis type, hospital procedural volume, and procedural urgency as characteristics common to both data sets that differed in the number of false-negative or false-positive patients compared with true-positive patients. Results for both PCI and CABG were similar following adjustment for hospital (Supplementary Table S1).

lysis type, hospital procedural volume, and procedural urgency as characteristics common to both data sets that differed in the number of false-negative or false-positive patients compared with true-positive patients. Results for both PCI and CABG were similar following adjustment for hospital (Supplementary Table S1). Change by Instrument Version and Calendar Year Analyses stratified by STS version (CABG data) did not demonstrate significant variability in sensitivity according to instrument version (Figure 1), but specificity differed (sensitivity Ptrend = 0.84; specificity Ptrend = 0.01). However, differences were marginal, with an overall change in specificity of <0.4%. For PCI, sensitivity and specificity did not vary significantly by NCDR version (Figure 2; sensitivity Ptrend = 0.86; specificity Ptrend = 0.64). Trends across calendar years were qualitatively similar for the 2 data sets (not shown).Figure 1 Sensitivity and specificity according to the version of the Society for Thoracic Surgeons data collection instrument. False-negative compared with true-positive identification (a) and true-negative compared with false-positive identification (b) of dialysis patients undergoing coronary artery bypass grafting (CABG), according to the version of the Society for Thoracic Surgeons (STS) data instrument. P values are for tests to assess differences between versions and trends across versions. The underlying data are provided at the top of the table. Mass-DAC, Massachussetts Data Analysis Center; USRDS, United States Renal Data System.

according to the version of the Society for Thoracic Surgeons (STS) data instrument. P values are for tests to assess differences between versions and trends across versions. The underlying data are provided at the top of the table. Mass-DAC, Massachussetts Data Analysis Center; USRDS, United States Renal Data System. Figure 2 Sensitivity and specificity according to the version of the National Cardiovascular Data Repository (NCDR) instrument used. False-negative compared with true-positive identification (a) and true-negative compared with false-positive identification (b) of dialysis patients undergoing percutaneous coronary intervention (PCI) according to the version of the instrument. P values are provided for tests to assess differences between versions and trends across versions. The underlying data are provided at the top of the table. Mass-DAC, Massachussetts Data Analysis Center; USRDS, United States Renal Data System.

eous coronary intervention (PCI) according to the version of the instrument. P values are provided for tests to assess differences between versions and trends across versions. The underlying data are provided at the top of the table. Mass-DAC, Massachussetts Data Analysis Center; USRDS, United States Renal Data System. Impact on Risk-Adjusted Outcomes For PCI, the area under the curve (AUC) was significantly lower (P = 0.02) for models incorporating the USRDS variable (0.899, 95% confidence interval [CI]: 0.891, 907) compared with the Massachusetts Data Analysis Center variable (0.900, 95% CI: 0.892, 0.908), but differences were marginal (Figure 3). Similarly, the continuous net reclassification index (–0.031, 95% CI: –0.058, –0.003) was consistent with weak effects on risk discrimination. For CABG, AUCs using the USRDS variable (0.765, 95% CI: 0.741, 0.790) and the Massachusetts Data Analysis Center variable (0.770, 95% CI: 0.745, 0.795) were not different (P = 0.06). The net reclassification index –0.045 (95% CI: –0.112, 0.022) was also consistent with only weak effects on reclassification. Lastly, differences in predicted risk were minimal, regardless of hospital procedural volume, for the vast majority of procedures (Supplementary Tables S2–S5 and Supplementary Figure S2).Figure 3 Receiver operating curves for prediction equations incorporating Massachussetts Data Analysis Center (Mass-DAC) and United States Renal Data System (USRDS) dialysis variables. Plots show receiver operating curves for the regression equation incorporating the USRDS variable (blue) or the Mass-DAC variable (red) for percutaneous coronary intervention data (a) and coronary artery bypass graft data (b).

ta Analysis Center (Mass-DAC) and United States Renal Data System (USRDS) dialysis variables. Plots show receiver operating curves for the regression equation incorporating the USRDS variable (blue) or the Mass-DAC variable (red) for percutaneous coronary intervention data (a) and coronary artery bypass graft data (b). Discussion We assessed dialysis identification by the NCDR and STS data instruments for 2003–2012 Massachusetts patients receiving CABG or PCI, by linking our data to the USRDS. Specificity and negative predictive values for identification of chronic dialysis patients were high, and false-positive rates were low. However, the proportion of individuals receiving maintenance dialysis was small, and positive predictive values were low (62% for PCI; 65% for CABG). The impact on overall prediction of procedural risk was small, suggesting that use of these tools to compare facility outcomes is reasonable despite the misidentification of an important risk factor like chronic dialysis status.

ce dialysis was small, and positive predictive values were low (62% for PCI; 65% for CABG). The impact on overall prediction of procedural risk was small, suggesting that use of these tools to compare facility outcomes is reasonable despite the misidentification of an important risk factor like chronic dialysis status. Prognostic risk scores derived from STS (the STS score) and NCDR data sets and instruments have been widely used to assess risk-adjusted outcomes, compare procedural results across providers, analyze outcomes of cardiac surgery, assess the impact of kidney disease and dialysis status on practice patterns, and assess postsurgical, postmyocardial infarction, and post-PCI outcomes.8, 9,S1–S11 However, to our knowledge, the current investigation is the first to assess the precision of the dialysis variables, and our results suggest that their accuracy is suboptimal. Although sensitivity and specificity are high, the overall prevalence of maintenance dialysis patients was <1.5% in each cohort. Consequently, positive predictive values were low, with more than one third of patients identified by the STS instrument, and nearly 40% of those identified by the NCDR, not actually receiving maintenance dialysis. This raises questions about use of data based on the NCDR and STS instruments to assess cardiac procedures. Although the exclusion of patients with chronic kidney disease and ESRD from cardiovascular trialsS12 makes use of these data sets to investigate cardiac treatment strategies attractive, our results suggest that identification of chronic dialysis patients within NCDR- and STS-based data sets is not sufficiently accurate to provide reliable guidance for the care of dialysis patients. Misspecification of the dialysis variable could negatively influence adjustment for confounding and reduce the accuracy of public reporting of PCI and cardiac surgery outcomes. To avoid over-reliance on any single metric for assessing prognostic value, we examined changes in AUC and net reclassification index, and compared predicted and actual risk. We detected marginal effects on AUC, and predicted risk differed significantly in a minority of individual cases.

I and cardiac surgery outcomes. To avoid over-reliance on any single metric for assessing prognostic value, we examined changes in AUC and net reclassification index, and compared predicted and actual risk. We detected marginal effects on AUC, and predicted risk differed significantly in a minority of individual cases. Thus, in aggregate, our data suggest that the overall impact of dialysis status misspecification is small and unlikely to significantly compromise analyses of procedural risk and benefits or comparative hospital scorecards, although the net impact could be important in hospitals with a combination of low procedural volume and unusually high rates of misspecification. Determining the underlying reasons for and best response to the inaccuracies we identified is necessary. Our results suggest that including explicit variables for peritoneal dialysis, targeting training efforts at low volume centers, and considering enhanced validation of data gathered during emergent or urgent procedures are steps with potential utility. Although we lacked the data needed to investigate misidentification of individuals with dialysis-dependent acute kidney injury as patients with dialysis-dependent ESRD, we also believe that clarification of the instrument fields to better discriminate between acute and chronic kidney disease should be considered.

lity. Although we lacked the data needed to investigate misidentification of individuals with dialysis-dependent acute kidney injury as patients with dialysis-dependent ESRD, we also believe that clarification of the instrument fields to better discriminate between acute and chronic kidney disease should be considered. Our analysis had several limitations. We analyzed data from a single state, and our results may not be fully generalizable. However, data collection using the NCDR and STS instruments is mandated in Massachusetts and is performed by trained staff; selected fields were audited to ensure fidelity. Nevertheless, better performance in the national data sets is theoretically possible. Additionally, state privacy regulations precluded use of social security numbers during matching to the USRDS, although we were able to utilize name, date of birth, and last known alive dates. Significant numbers of patients sharing these identifiers, within Massachusetts during the study period, is unlikely. In conclusion, we matched Massachusetts PCI and CABG patients from the USRDS to identification of chronic dialysis patients by the NCDR and STS. Neither accurately identified individuals with dialysis-dependent ESRD, suggesting that data collected using these instruments may not be useful for informing therapeutic choices in individuals requiring chronic dialysis and that efforts to improve these instruments are warranted.

dialysis patients by the NCDR and STS. Neither accurately identified individuals with dialysis-dependent ESRD, suggesting that data collected using these instruments may not be useful for informing therapeutic choices in individuals requiring chronic dialysis and that efforts to improve these instruments are warranted. Disclosure DMC received consulting fees from Amgen, Medtronic, and Lilly-Boehringer, and fees related to service on DSMBs (AstraZeneca, Allena Pharmaceuticals) and trial steering committees (Jannsen Pharmaceuticals-CREDECE Trial, Zoll Medical-WEDHEAD trial). All the other authors declared no competing interests. Supplementary Material Supplementary File (PDF) Acknowledgments Funding for this work was provided by National Institutes of Health Grant R01HL118314-01A1. Data reported here have been supplied by the United States Renal Data System (USRDS). The interpretation and reporting of these data are the responsibility of the authors and in no way should be seen as official policy or interpretation of the US government. We thank Treacy Silbaugh for support on this project. Supplementary File (PDF) Supplementary Methods. Supplementary Results. Figure S1. Characteristics associated with misidentification of dialysis patients. Figure S2. Predicted risk using US RDS and MASS-DAC variables. Table S1. Sensitivity, specificity, positive and negative predictive values in analyses corrected for hospital. Table S2. Results of mixed-effects model using MASS-DAC dialysis variable for prediction of 30-day mortality in the PCI data sets. Estimates are log-odds ratios.

Figure S2. Predicted risk using US RDS and MASS-DAC variables. Table S1. Sensitivity, specificity, positive and negative predictive values in analyses corrected for hospital. Table S2. Results of mixed-effects model using MASS-DAC dialysis variable for prediction of 30-day mortality in the PCI data sets. Estimates are log-odds ratios. Table S3. Results of mixed-effects model using USRD dialysis variable for prediction of 30-day mortality in the PCI data sets. Estimates are log-odds ratios. Table S4. Results of mixed-effects model using USRDS dialysis variable for prediction of 30-day mortality in the CABG data sets. Estimates are log-odds ratios. Table S5. Results of mixed-effects model using Mass-DAC dialysis variable for prediction of 30-day mortality in the CABG data sets. Estimates are log-odds ratios.