Benchmarking

A Process, Not a Number

It's more complicated than it appears

More than one-quarter of all hospitals today are actively involved in benchmarking.


Typically working with a vendor, managers are handed complicated spreadsheet-style comparative data, and then told to contact their best-performing peers to find out how they can run at the same level. So what have organizations reaped after investing hundreds of thousands of dollars over the course of several years? Unfortunately, not much. Strategically, the typical benchmarking program fails on day one, tasking the wrong people with a difficult, complex project that goes way beyond spreadsheets; employing a faulty method of operational change, and lacking a sound foundation of budget discipline and analytical expertise.

Benchmarking is an analytically rigorous process of learning what superior-performing organizations do, how they do it, and how this knowledge can be adapted to work elsewhere. It should not just be a pile of reports! The sort of report that vendors provide, like the example to the left, is virtually unintelligible to department managers. No amount of PowerPoint slideshows will change that. These reports are analytical reports for analysts. 


Typically working with a vendor, hospitals produce a large number of reports and distribute them to managers, who then must call their top-performing industry peers and learn what they do to achieve their numbers. To do this task effectively, managers would need to have the expertise of a first-rate management engineer or financial analyst, possess multi-clinical skills to solve inter-departmental issues, and be strong and articulate advocates of higher efficiency. Managers from other organizations who get these phone calls and emails would need to have the same skills in order to respond effectively! Not only are those conditions unrealistic, we implicitly assume that managers will eagerly cooperate in the absence of any incentives to do so. We burden managers with an almost impossible task, and the overwhelming majority of organizations embarking on this path are destined to fail. The problem is not the data—it’s the method.

 

Presented with thousands of data points like those contained in the report above, managers will usually adopt a defensive position: first, they challenge the accuracy of their own department’s data. Second, they challenge the appropriateness of the outside comparison. Third, they remain immobile until told exactly what to do—something the organization is ill equipped to deal with. Nothing happens. 


Yet, despite these problems, benchmarking remains a popular exercise in many healthcare organizations. Senior managers like the intuitive appeal of the approach: learn from the best. Hard data seems to make the case that one’s organization is ripe with opportunities for the plucking. Commercial databases can extract numerous examples of any given department that look much more efficient than one’s own, whether labeled “best,” “better,” or “top” performers. Theoretically, moving most of a typical organization’s departments to the top performing quartile would yield millions of dollars of savings per year, and this holds out a very compelling and tantalizing prospect for the CEO or CFO. The appeal is undeniable, and health care organizations keep coming back for more. 

A better way

We have a unique three-path approach tailored to organizations of any size and complexity.

  1. Internal (historical) benchmarking
  2. Benchmarking within the system      
  3. External benchmarking (outside the system)   

Internal benchmarking is the analysis of best-demonstrated historical performance for every department. It compares current performance against past performance, using a measure of work, or unit of service. The results show what productivity levels could be achieved at current workload volumes. Since this performance had already been achieved in the past, it is highly realistic. This approach is explored at length in Superior Productivity in Healthcare Organizations, 2nd Edition


If a healthcare system is of sufficient size, benchmarking within the system compares the performance of like departments within the system. For example, the best Emergency department performers within the system are compared to all other Emergency departments within the system. The results show what could be achieved if all Emergency departments operated at the same productivity as the best of them. Being within the same system offers convenient and cooperative benchmarking partners for further study.  
 

External benchmarking, available through commercial vendors, shows where each department stands in relation to a comparison of like departments on narrowly defined metrics, almost all of which are financially-oriented. Since there is no real consideration for operational differences or service goals, external benchmarking must go beyond these surveys, requiring rigorous analysis to account for different patients, different operational and medical practices, scopes of service, clinical and service quality, and unique tasks. Financial performance and productivity is the outcome of all the tasks and activities performed for a given patient load and medical complexity. External benchmarking requires due time and the involvement of many people, as it touches many areas and may cross department boundaries. The results must be interpreted carefully for application to operations.  These three measurements not only tell us what is possible and realistic, they also guide how best to implement the opportunities presented.  

Implementation

Internal benchmarking doesn't present nearly the same concerns that must be addressed in intra-system and external benchmarking. Operational issues— departments splitting into two, departments combining, a change in workload measures, startup departments, departments winding down, grant-funded programs to be excluded—need to be understood so that the analyses can be adjusted to reflect current operations.  Meeting with individual directors to evaluate any operational changes to be considered, we would adjust the analyses and get agreement on a labor standard for each department. In almost all cases, the new labor standards would incorporate current operational practice. For example, if operating hours have been extended, or new functions added to a department, labor standards would preserve those operational changes. No reengineering would be required.  


Benchmarking within the system presents much the same issues as benchmarking outside the system, although those issues may be of lesser magnitude. The working assumption is that a healthcare system’s departments are more like each other in organization and scope than to hospitals and departments in different systems in different states.  

As intra-system and external benchmarking involves analysis that is more rigorous and possibly reengineering department operations, we implement labor standards based on internal benchmarking for all departments first. This establishes a foundation of accountability to standards based on existing operations for every department throughout the organization.  

Download the three-path approach

Click on a file to download.

EIS Productivity Strategy (pdf)

Download