To improve the quality of care we need accountability, and one way to get it is documenting what clinicians do. I’ve argued that Canada needs better measurements of health outcomes and health care quality here and here. This stuff matters: Checklists have saved thousands of lives.
But you can have too much of a good thing. On EconTalk, Russ Roberts interviews Leonard Wong, a retired military officer and a professor at the US Army War College. Wong and Stephen Gerras have written a paper describing the moral challenge experienced by military officers faced with high burdens of regulatory compliance. Like medicine, the military has a complex mission with life and death stakes, using tools that change constantly and are inherently unsafe. The military cultivates intensive safety disciplines that have, for example, reduced aviation fatalities per mile flown by many orders of magnitude.
The inner workings of government
Keep track of who’s doing what to get federal policy made. In The Functionary.
The Functionary
Our newsletter about the public service.
Nominated for a Digital Publishing Award.
To build an effective military and protect servicemen and women, the military sets many requirements. Officers must ensure that their units meet these requirements, and they have to certify conformity with these requirements with a signed report or checklist.
The problem is that there are now so many requirements that it is impossible to meet them. Moreover, the US Army is in effect a zero-defects culture; failure to complete requirements disqualifies you from promotion.
Wong argues that as a result, officers are falsely documenting completion of requirements that either have not been met or that they have never inspected. This means that potentially important tasks are not getting done, and that nobody knows which things are not getting done. Perhaps more importantly, the professional norm of integrity is being undermined.
People in health care should study this in detail. My view is that the burden of documentation and compliance for health care providers is comparable to the burden on the military. Here are David Blumenthal and J. Michael McGinnis in JAMA:
The budding enthusiasm for performance measurement, however, has begun to create serious problems for public health and for health care. Not only are many measures imperfect, but they are proliferating at an astonishing rate, increasing the burden and blurring the ability to focus on issues most important to better health and health care. Measures of the same phenomenon also vary in specification and application, leading to confusion and inefficiency that make health care more expensive and undermine the very purpose of measurement, namely, to facilitate improvement. Not uncommonly, a health care organization delivering primary care to a typical population is asked to report and collect hundreds of measures aimed at dozens of conditions.
In some ways, the situation in health care is worse than the military. The military has a single chain of command, but medicine has many bodies issuing standards, many of them redundant. Blumenthal and McGinnis illustrate their point with this Figure:
So what do we do? What we shouldn’t do is give up the concepts of accountability or documentation.
The inner workings of government
Keep track of who’s doing what to get federal policy made. In The Functionary.
The Functionary
Our newsletter about the public service.
Nominated for a Digital Publishing Award.
But critics of excessive standardization have an important point. Clinicians are not automatons following recipes. They have to optimize their time by prioritizing from an indefinitely long list of tasks that could potentially benefit patients. Therefore any requirement that constrains clinician choice has an opportunity cost in terms of the other things that she might do for her patient. Ideally, before setting a standard we should compare the benefit that can be achieved by implementing the standard against that opportunity cost.
Most people engaged in quality research and improvement are aware of this tradeoff. But it isn’t addressed in any serious way.
We can’t even begin to address this problem unless the people setting standards look at the total burden of documenation and compliance. Blumenthal and McGinnis argue that what we need are ”œcore metrics”, defined as
a parsimonious set [of measures] that provides ”œa quantitative indication of current status on the most important elements in a given field, and that can be used as a standardized and accurate tool for informing, comparing, focusing, monitoring, and reporting change.” [Core metrics should be] outcomes oriented, reflective of system performance, and meaningful and have utility at multiple levels of the health care system.
We can’t measure everything about medical practice, so we need consensus on a minimum set of requirements that are feasible to measure and maximally affect practice.
Specifying that minimum set is an enormous and challenging task. There needs to be a process whereby health care providers, patients, and other stakeholders can achieve consensus about what matters. I believe that this should be a national conversation. 15% of the Canadian population lives in provinces or territories of less than 1 million people: these jurisdictions do not have the resources to do this work on their own. But the core metrics shouldn’t be dictated by the Federal government. We need to get to a set of metrics with normative force and this won’t happen unless health care providers own them.
However we do it, specification of core metrics for health care is critical for the long run success of quality improvement and the preservation of the virtue of integrity in medicine.
(An American version of this post previously appeared here.)
