Of all the structural (how care is organized) “evidence-based markers of high quality care,” perhaps the most ironclad has been the involvement of critical care physicians in the care of ICU patients. That is, until now.

In a sophisticated study in today’s Annals of Internal Medicine, Levy and colleagues mine a decade-old, 100-hospital, 123-ICU database containing detailed clinical data on more than 100,000 patients to examine the association between ICU staffing models and hospital mortality. The researchers tell us that they began the study expecting to confirm the benefit of intensivists (also called “critical care physicians”). It would have been odd to expect otherwise, since such a benefit has been seen in a number of prior, smaller studies (summarized here).

Levy et al. were really seeking answers to two different but related questions. The first: in those ICUs (n=79, or 64% of the 123 ICUs) in which non-intensivist physicians sometimes called for intensivist help, what was the effect of involving an intensivist on hospital mortality? In the other hospitals (“no choice” institutions), it appeared that patients either virtually always received intensivist consultation and/or management (n=23, 19%; particularly large teaching hospitals) or virtually never did (n=21, 17%). So in these hospitals, the authors were testing which of the two models led to lower mortality rates. Seemed like a no-brainer.

Shockingly (no pun), in the “choice” hospitals, hospital mortality rates were significantly higher when intensivists were called in. Moreover, mortality was also substantially higher in those ICUs in which intensivists managed virtually every patient than in those in which they managed few or none.

You are probably thinking that these must be apples-to-oranges comparisons – patients in intensivist-only ICUs, or those in “choice” hospitals who receive intensivist care, simply must be sicker. A reasonable concern, to be sure, and in fact such patients were more ill. But the investigators, using robust statistical methods to adjust for severity of illness and for the “propensity” to involve an intensivist, found that this case-mix difference explained only a small proportion of the increased mortality.

They also examined several other possible explanations (including that non-intensivists were more likely to discharge patients to hospice or SNF; thus patients would not appear in the “hospital deaths” column), but found that none of them made the overall 40% higher chance of death vanish. If you’re an intensivist looking for any silver lining, the “harm” associated with intensivists was greatest in the least ill ICU patients, and appeared to lessen (but not evaporate) in the sickest quartile of patients. Boo-ya.

In a well-written accompanying editorial, Gordon Rubenfeld and Derek Angus, two of the world’s top ICU researchers, ponder the possible explanations for these stunning results. One, of course, is that the tools to measure, and thus adjust for, severity of illness are imperfect, and that some unmeasured variables (perhaps captured in “the eyeball test”) are associated with both intensivist involvement and mortality, at least in “choice” hospitals. Of course, there is no way to be sure about this without a randomized trial, which would be awfully hard to do (“Sir, you could be randomly assigned to an ICU either with or without experts. Please sign the consent form here.”). In fact, the Annals study could have only been done in the U.S., since virtually every other country’s hospitals are dominated by closed ICUs.

Supporting the “unmeasured confounders” hypothesis, Rubenfeld and Angus highlight the fact that virtually every prior study found that intensivists were beneficial. They also note that Levy and colleagues did not provide data supporting a plausible causal pathway for intensivist-related harm. In a way, they are arguing for a Bayesian approach to the interpretation of clinical research (a point made previously by Browner and Newman): the “pre-test probability” of intensivist benefit is so high (based on face validity and prior studies) that there is a pretty good chance that a single study showing harm is a “false positive.”

A bit haltingly, Rubenfeld and Angus go on to consider an alternative hypothesis: namely, that intensivists kill.  

Although we believe that critical care physicians are trained and expertly skilled in the management of critically ill patients, perhaps some routine critical care practices and procedures may not be beneficial or cumulative use of more interventions may take a negative toll.

Although good on ‘em for raising this possibility, their heart isn’t really in it, quite understandably. After discussing various explanations for the Levy results, they conclude that this study is not enough to change practice or policy: 

...until someone replicates Levy and colleagues’ results in another cohort and provides evidence for a mechanism by which intensivist-staffed ICUs increase mortality, their study will remain one observation against many.

I happen to agree – if I had a closed ICU that seemed to be working well, I wouldn’t throw open the glass doors tomorrow. And if I ran the Leapfrog Group, I would not take intensivist staffing off my list of evidence-based safety practices, at least not yet. But I would question my assumptions and use this study to motivate further inquiry into the best ways to organize an individual ICU, or all ICUs. As Levy and colleagues conclude…

Although all of the possible explanatory mechanisms we have mentioned [overly aggressive care, infections from indwelling catheters] may seem to portend badly for the practice of critical care medicine, we suggest that, if true, they are amenable to correction or mitigation through such efforts as guideline development and adherence, quality improvement, and systematic efforts to reduce errors. Given the complexity of critical illness, the need for dedicated critical care physicians seems inevitable, and strategies to assure best practices will help them to guarantee the best outcomes possible. Further research is needed to explain these findings and determine whether these results may be explained by unrecognized residual confounders of illness severity.

Seems right to me.

One postscript: though neither the study nor the editorial mention hospitalists, there are two hospitalist angles worthy of reflection. First, SHM surveys tell us that the vast majority of U.S. hospitalists do see ICU patients. Our ICUs at UCSF Medical Center fall into this category – we have a hybrid model in which our hospitalists remain the physicians-of-record for medical ICU patients, with mandatory intensivist consultation (the intensivists assuming the management of the ventilators, the lines, and the sedation). Over the past decade, I have been under intense pressure to “close the unit,” even though most of our hospitalists and intensivists think our present arrangement works quite well (and our model does pass Leapfrog muster for “high intensity intensivist involvement”). The Annals study reminds us that there is much that we do not understand about best practices in ICU organization, and that a model of open ICUs with selective use of intensivists might well prove to be as good as, or better than, the hermetically sealed ICU, with its forced ICU-floor discontinuity.

Secondly, the data from which the Annals study were derived came from Project IMPACT – a study begun by the Society of Critical Care Medicine (with pharma support) in 1996. Although SCCM can’t be thrilled that their database produced these results (you can be sure that the society’s PR firm received an emergency call from the SCCM CEO this week), the study highlights the power of and need for clinical research to answer important clinical and organizational questions. It also underlines the unique ability of medical societies to help organize multicenter studies whose fruits may be harvested over many years. The Society of Hospital Medicine can, and should, be in this business.