I’ve recently encountered a number of articles questioning the usefulness of comparative effectiveness research. For example, Keith Winstein, writing in the Wall Street Journal Feb. 10, details the failure of medical practice to adapt to findings from the Courage trial — reported to great acclaim in the New England Journal of Medicine in 2007 — about the relative effectiveness of drug therapy and stenting in relieving chronic chest pain. Courage found that trying drugs alone first, and only stenting if the drugs didn’t work, was just as good as using stents as a first intervention in the defined patient group. While stent use fell markedly just after publication of the study results, it has returned to prior levels. The study’s principal investigator, Dr. William Boden, thinks he knows why. As reported by Winstein: “‘Most [cardiologists] haven’t voluntarily incorporated the Courage criteria into their practice,’ says Dr. Boden. ‘What’s going to continue to drive practice is reimbursement.'”
Dr. Jerome Groopman, writing in the Feb. 11 New York Review of Books, attacks the utility and the desirability of CER from a different angle. For Groopman the problem is that most areas of clinical practice are far too complex to be codified by the simple decision algorithms and practice standards that result from CER studies. Consequently, recommendations for CER-generated practice recommendations seem to be extremely unstable and broad application of study findings often does more harm than good. Groopman also makes a meal of the refusal of many women’s health specialists to accept the recent U.S. Preventive Services Task Force recommended revision in mammography screening for the age 40-49 cohort — conveniently misrepresenting, as did most of the media and so many critics of the recommendation, what the task force actually said.
At the risk of grossly oversimplifying arguments made by reasonably intelligent and cogent observers, I’d summarize the critiques of CER as follows: Winstein doesn’t think CER is all that great because doctors don’t pay attention to findings — for reasons that include economic self-interest; Groopman is leery of CER because clinical medicine is complex, research findings keep changing, and treatment decisions need to respond to patient-specific considerations rather than to population-derived parameters — his real worry is that the government will turn CER-based recommendations into requirements. There is some merit to both of these critiques. But ought we, as both Winstein and Groopman seem to suggest, simply abandon the field — eliminate or very closely circumscribe CER investments and efforts to use research findings to improve clinical practice — in the face of these difficulties?
Quite obviously, we should not. The advisability of studying the relative effectiveness of different interventions so that we can do more of what works and less of what doesn’t — so that doctors have a broader, deeper and stronger evidentiary basis for making treatment decisions — shouldn’t be debatable. The difficulties of implementation — unsolved problems in how to foster adoption of CER findings — are a second-order issue, not a reason not to do CER. And we shouldn’t be surprised that findings evolve over time, leading to refinement and sometimes substantial revision of practice recommendations, as we learn more and more about the underlying causes and manifestations of different clinical conditions. The first and simplest rules will be wrong for many individuals; over time, we’ll learn to discriminate better, the treatment recommendations will become more sensitive to subgroup characteristics and overall choices will improve. That is how knowledge and capability grows in every discipline, and how medical practice will improve.
In their recently published “Chaos and Organization in Health Care,” Partners Healthcare System executives Drs. Thomas H. Lee and James Mongan provide an extraordinarily straightforward and insightful framework for understanding the current state of medical care in the U.S., the obstacles to improvement of both quality and efficiency and how those obstacles can be overcome. Their analysis is worth consideration by anyone interested in improving the delivery of health care in the U.S. It also provides a clear and practical perspective on the potential for effective clinical adoption of CER findings and prospects for overcoming impediments to that adoption.
Lee and Mongan argue that the fundamental problem of U.S. healthcare is the combination of too much knowledge and too little communication, reinforced by inadequate organization, an outdated professional culture and perverse financial incentives. For them, the core issue is that no practitioner can keep up with the explosive growth of knowledge and technology developments across all of medicine, or often across a particular specialty area. Taken separately, each development is beneficial; taken together, they create chaos. When serious or multiple problems arise, an individual’s care is of necessity provided by a number of different practitioners with extremely limited ability to keep informed about each other’s findings, tests, prescriptions, etc. This leads to over-utilization of resources through duplication and to clinical decisions which may reflect only a partial understanding of the patient’s overall condition.
Effective responses to the problem of information overload and inadequate communications exist. Lee and Mongan point to two powerful tools that address those issues:
- The electronic health record (EHR), which provides the potential for organizing all of a patient’s clinical information, including test results, in a single place that’s readily accessible to any practitioner; and
- Computerized physician order entry (CPOE), which, combined with sophisticated protocol reminders and decision support tools, can eliminate many potential prescribing or treating errors and can give physicians immediate access to the most current relevant practice recommendations.
They argue that implementation of these tools at Partners has yielded enormous quality and efficiency benefits.
At the same time, Lee and Mongan recognize that there are significant impediments to the widespread implementation of these tools, including:
- High capital costs for adoption that are well beyond the resources of most physicians, who practice either independently or in small groups. Investment in EHRs and CPOE requires very substantial group size and ongoing support beyond the resources of individuals or small groups;
- Physicians’ professional culture of independence, which makes them resistant to accepting that “systems” can yield better decisions than their personal, professional knowledge and judgment — they need to be pushed to adopt these tools; and
- The dominant fee-for-service reimbursement system, which creates perverse financial incentives that reinforce the do-it-yourself instincts of individual physicians; capitation models of payment correct those perverse incentives, but are inappropriate on account of financial risk for most practices.
At bottom, Lee and Mongan argue for the superiority of large and tightly structured service delivery organizations — organizations that can employ physicians, own hospitals and other service settings, bargain effectively with insurers and/or employers, invest in technologies for EHRs and CPOE and accept full capitation for large populations of patients. Those are the circumstance that will best eliminate waste and duplication, avoid errors, provide the best road maps to guide physicians through the complexities of ever-evolving clinical knowledge and technology and thereby improve quality and efficiency. And such organizations effectively answer the objections to CER posed by Groopman and Winstein.
Lee’s and Mongan’s prescription, however, is evolutionary, not revolutionary, and that is critically important. They know that we can’t simply remake our delivery system by forcing everyone into tightly structure organizations (such as, for example, the Accountable Care Organizations so beloved by many reformers). They believe that the system can be led in that direction, and that that would be a good thing, but they understand how hard the transition will be — they’ve been working on it for decades at Partners and haven’t gotten all the way there yet. They recognize a continuum of organizations, from the independent solo practitioner or small group at one extreme to the staff-model, hospital-owning HMO at the other, each capable of assuming an organizationally appropriate level of risk. The least-organized practitioners require fee-for-service payment; the largest and most tightly structured HMOs can accept full capitation; and in the middle various levels of organization can effectively deal with quality incentives, pay for performance contracts, episode of care payment, disease management and partial capitation. Our goal, they argue, should be to move practitioners up the organizational continuum, but to make sure that each extant organizational model has access to an appropriate payment model — one they can implement effectively.
Edward Berger is a senior healthcare executive with more than 25 years of experience in medical device reimbursement analysis, planning and advocacy. He’s the founder of Larchmont Strategic Advisors and the vice president of the Medical Development Group. Check him out at Larchmont Strategic Advisors.