Identify data-driven opportunities
How are we doing? This question is loaded with possibilities for health systems that are on a continual hunt for ways to improve operational performance. It is critical that administrators are able to understand the current status of their health system’s performance as well as recognize emerging trends to accurately identify areas that are performing well or lagging.
Traditionally, to benchmark operation performance, health systems specify criteria to measure in comparison groups based on what is important to them. The comparison group approach has proven useful because it enables health systems to evaluate progress based on key priorities. But it may also produce small sample sizes and introduce redundant criteria and measures that are unrelated to required work effort and expense. As a result, important criteria that strongly relates to work effort and expenses may be omitted.
Health systems can level the playing field by adding comparison group criterion constructed on evidence-based data. This approach is known as the empirical method. It addresses the challenge inherent in operational benchmarking by leveraging multiple years of data to identify factors that impact resource demand in a manner that is similar to the risk and severity adjustment method utilized at the patient level when benchmarking clinical performance.
Extending the reach of comparison groups
Establishing the right benchmarks to evaluate overall performance across an entire health system can be difficult and time-consuming. Services provided at hospitals can vary greatly depending on location, resources and patient mix. Each hospital within a system likely has a different mix of departments to meet patient care needs. Hospitals may also operate multiple instances of a given department, such as several surgical units or general medical unit services. Health systems need to compare performance on a fair basis that considers the varied make-up of all their facilities.
If comparison groups are defined too tightly, they may be limiting because:
- Data about the characteristics and performance of departments in other hospitals is not always readily available.
- Finding a suitable comparison group can be difficult. Key characteristics may not be obvious, or there may be few facilities or departments which match all of the desired criteria.
- Even for the most common types of hospitals or departments, specifying just a few criteria can result in a small comparison group; sometimes too small to be statistically valid.
Augmenting the comparison group method
When health systems only use self-defined criteria to identify peer hospitals or departments that have similar characteristics to set operational benchmarks, it is possible that data is left out of the evaluation that could improve confidence levels. As a result, important decisions about resource allocation are based on smaller sample sizes. By adding empirical data to the benchmarks, more factors are included that are essential to drive operational performance.
Because the empirical method leverages multiple years of data, hospital and department operational benchmarks are based only on factors and criterion that are empirically proven to impact required work effort and expense. Data from multiple years is validated, and only data that reveal consistent effects are retained.
Empirically-derived benchmarks for every hospital and department are generated based on many more factors than is possible using just the comparison group method. The database is normalized to reduce redundancy and improve the accuracy of the data.
Why fair comparisons are important
In the quest to assess performance based on operational benchmarks, results that rely on comparison groups that are too narrowly defined can result in unfair assessments. Improvement opportunities in hospitals or departments that would benefit from additional labor or other resources may not be revealed. Instead, the funds to increase staffing, make facility upgrades, provide more supplies or other steps are allocated elsewhere.
By adding an empirical method to operational benchmarking comparison groups, health systems can identify improvement opportunities in hospitals and departments based on validated data from facilities that provide similar services but have different characteristics within their own system and from a larger data set.
The result is a data-driven analysis base from which to develop strategic action plans, measure success and provide a more comprehensive answer to the “how are we doing” question.