Most of the attention given to Massachusetts health reform has focused on insuring the uninsured. That’s appropriate, since the first priority of this new law is to get as close to universal coverage as possible. Two elements of reform that deserve some attention, however, are the new Cost and Quality Council and the provision that conditions payment increases on performance reporting. This is all part of the larger national drive toward a “transparent” health care system, a concept which is the cornerstone of an executive order signed by President Bush last August that many commentators expect will finally tame healthcare costs.
Proponents of healthcare transparency contend that outcomes report cards will drive market reform via informed consumer choice – educated patients with some of their own money at risk will seek out the best value in healthcare at the lowest cost if provided with detailed cost and outcomes data along the lines of a healthcare “Consumer Reports.” A fine theory and a worthy goal: the unfortunate problem is that producing an effective consumer report is easier said than done.
Most outcomes measures do not accurately separate performance into a rank order – even with a relatively simple to measure outcome like surgical mortality. For example, a recent report from Dartmouth shows that for seven common hospital procedures, only one (bypass surgery) is done often enough to allow meaningful differentiation among hospitals. The problem is small sample size. To illustrate this point, consider the process of determining the batting title in baseball. The rules require that the winner must have a minimum number of at-bats – otherwise someone who was up twice and had one hit could win the batting title by batting .500.
It turns out that for most operations, hospitals don’t have enough “at bats” to be validly compared, even when you add several years’ worth of outcomes together. It’s even harder with the outcomes of individual doctors because there are fewer cases per doctor than there are per hospital, so differences are even less likely to be non-random. That doesn’t mean that rankings can’t be made – there are already plenty and there will undoubtedly be plenty more. The point is that if the lists are arbitrary because the differences in them are largely random, the ability of such lists to drive consumers to value is an illusion.
At present, report cards are more useful for establishing quality standards than they are for fueling consumerism. This may gradually change but there is one other caveat. Public physician report cards also have a powerful effect on physician behavior and decision making. There is a simple reason for this: the threat of being identified as a substandard performer. This is the quintessential two-edged sword that can both do good and cause harm. As an example, consider the measurement of mortality for coronary angioplasty, a very common therapy for heart disease which is still the number one cause of death in the US. New York has been publicly reporting angioplasty outcomes at the level of the individual physician for years, with the state’s most recent mortality rate for angioplasty at 0.58 percent. In Massachusetts, publicly reporting at the institution level for the first time in 2005, the mortality rate was three times higher at 1.71 percent. The equipment and technology are the same in both states. Are the hospitals and doctors in New York actually three times better?
A more likely explanation is that the mix of patients is different. If you look at just the low risk, elective cases, the outcomes are about the same. The highest risk (and benefit) of angioplasty is for patients with acute heart attacks, especially when they are in shock. There are similar rates of coronary artery disease, angioplasty and heart attacks in each state, but patients with heart attack/shock in Massachusetts are treated with angioplasty six times more frequently than in New York. The difference in angioplasty mortality is more related to who gets treated than the quality of treatment. Undoubtedly, there is judgment involved in deciding which patients should receive a given treatment and angioplasty shouldn’t be offered to everybody. But the added element of public reporting can bring with it a dangerous trade-off: excluding the relatively few highest-risk patients from treatment is a very effective strategy for improving a medical outcome – unless you’re a high-risk patient who needs the therapy.
To get the right results, public quality profiles need to be based on the best available data and methods so that they are accurate. A subtle, but equally critical element, is to design measures that are sufficiently broad so that improvement in one public measure does not come at the expense of worsening an equally important but unmeasured outcome (think of the angioplasty example). We should take on the work of creating a transparent measurement system thoughtfully – and transparently – to make sure we get the results we want. If these issues aren’t part of the discussion, there is a real problem.
David F. Torchiana, MD
Chairman and CEO
Massachusetts General Physicians Organization (MGPO)
This program aired on June 1, 2007. The audio for this program is not available.