In this retrospective cohort study of Medicare beneficiaries receiving PCI, we compared results using conventional modeling approaches to an IV approach leveraging the rapid uptake of DES over time. Using conventional methods, DES was associated with a strong (20%) relative risk reduction (RRR) in all-cause mortality and only a modest 10% RRR effect in repeated revascularization at 46 months post-PCI. In contrast, IV methods produced estimates showing no association between DES and all-cause mortality, but a much stronger (24% RRR) effect on repeat revascularization.
The findings from this study of Medicare beneficiaries with ACS are consistent with those of other observational studies of stent outcomes. Multiple investigators have suggested a survival benefit of DES using conventional methods such as propensity scores. In contrast, others using IV and related methods have generally shown no difference in mortality and stronger impacts on repeated revascularization than is observed with propensity scores. A natural question that emerges from these disparate lines of evidence is, "which one should be believed?" Although the extensive trial data available for the DES vs BMS comparison are consistent with the IV results in this analysis, reliance on clinical trial results to validate any particular methodology (IV or PS in this case) is not necessarily appropriate in most applications. Both propensity scores and IV methods rely on untestable assumptions: for propensity scores, it is the lack of confounding between treatment assignment and outcome after adjustment for propensity score, whereas for IV, it is the lack of confounding between instrument and outcome after adjustment. We believe that in this circumstance, the latter is the more defensible assumption.
For our time-based instrument approach to be a valid, it must be plausible that potential bias from other secular trends, such as unobserved changes in patient characteristics, improvements in other medical treatments provided as part of care, and data coding quality, are modest. Although changes in unobserved characteristics are, by definition, impossible to test, we show in II that after stratification for an observed characteristic known to exhibit seasonal variation (ACS diagnosis), based on all other observed characteristics, there is no evidence that patients differ in predicted event rate over the study window. Another possibility is that trends in the characteristics of facilities using DES are correlated with differences in outcome. However, correcting for all static, unobserved facility-level characteristics through fixed-effects modeling did not alter our results, suggesting that the effect of any such bias is modest. It is possible that a time-based IV could be affected by seasonal differences in outcome, such as potentially worse outcomes for patients treated by relatively inexperienced cardiology fellows in July, but a sensitivity analyses (results not shown) restricting the cohort to only those treated in nonteaching facilities showed comparable results. Although improvements in ACS care or data coding are possible, we believe that the potential impact of these threats is very limited because of the limited time period (10 months) under study.
Instrumental variable approaches have important limitations. As with most IV techniques, the estimates we produce are local average treatment effects and are estimated from the population of "marginal patients" whose treatment is affected by the IV. The local average treatment effect will only equal the overall average treatment effect if DES work equally well in all patients (ie, if the treatment effect is homogenous) or if treatment choice is not affected by expected treatment effect. Although these assumptions are not plausible in the case of DES, we still believe the local average treatment effects to be useful estimates, given that we have a broad population of marginal patients (utilization doubled from 30% to 60% over the study period). As with most observational analyses using Medicare claims, we did not have access to patients enrolled in Medicare Advantage plans, who are, in general, younger and healthier than Medicare part A and B beneficiaries; the generalizability of our results to this subset of the Medicare population is thus also limited.
In this evaluation of drug-eluting stents for Medicare patients with ACS, IV methods suggested no mortality benefit and a significant reduction in revascularization procedures. These findings are generally consistent with randomized clinical trials and suggest that this particular subgroup may not receive additional benefit of DES therapy beyond reduction in repeated revascularization. Although observational analyses are a critically important component of outcomes research, confounding due to unmeasured factors (eg, angiographic characteristics, socioeconomic status, etc) is a major threat to validity when assessing treatment effectiveness. Analyses with administrative data are particularly vulnerable to this threat. In those circumstances in which clinical practice changes rapidly after the introduction of a new technology, IV analysis can provide a complementary mechanism to assess outcome.