Using Control Charts to Assess Performance Measurement Data

https://doi.org/10.1016/S1070-3241(02)28009-8Get rights and content

Tutorial-at-a-Glance

Background

In 1997 the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) announced the ORYX initiative, which integrates outcomes and other performance measurement data into the accreditation process. JCAHO uses control and comparison charts to identify performance trends and patterns that are provided to JCAHO surveyors in advance of the organization’s survey. During its survey, the health care organization (HCO) is asked to explain its rationale for its selection of performance measures, how the ORYX data have been analyzed and used to improve performance, and the outcomes of these activities.

What do control charts do?

Control charts indicate whether an HCO’s process is in statistical control (that is, stable insofar as only common cause variation exists) or out of statistical control (that is, unstable insofar as special cause variation exists). With the presence of special cause variation, the HCO should not make any change in its processes until the special cause is identified and eliminated.

Choosing the correct control chart

An HCO can use many different control charts. Selecting the correct control chart type for the type of data collected makes interpretation more sensitive for detecting special cause variation. The ORYX measures are calculated as proportions (rates), ratios, and means (continuous variables data, such as average length of stay), and this information forms the basis for selecting the correct type of control chart. In addition, the average rate (especially for rare event measures) and the average number of cases need to be considered when selecting the control chart type for small population measures.

Section snippets

Control Charts and Comparison Charts

Control charts and comparison charts are complementary to each other; each offers decision makers a different view of data. Using both charts together, leaders can determine whether they are comfortable with their organization’s level of performance or whether a performance improvement initiative should be undertaken.

Control charts present an HCO’s observed outcomes over time to permit analysis of the type of variation (that is, common cause or special cause).4 This is critical. Although they

How Do We Use Statistical Analysis to Detect a Natural Variation from a Special Variation?

Every measurement process varies—but not all necessarily vary in the same way. For example, an HCO decides to monitor fall rates to determine if its fall prevention protocols are working as designed. Suppose that the HCO’s number of patient falls averaged 20 per month and ranged between 17 and 23 per month in a single year (Figure 1, above). This would suggest a stable process because the variation is predictable within given limits. In statistical process control terminology, this type of

Elements of a Control Chart

A control chart is a line graph with the addition of a center line representing the overall process average (or mean) and upper and lower control limits. It shows the flow of a process over time, as distinguished from a distribution, which is a collection of data that are not presented in the order in which they were collected. A control chart is a dynamic presentation of data, whereas a data distribution is a static presentation. The measure of the process being monitored or evaluated appears

Choosing the Correct Control Chart

An HCO can use many different control charts. Selecting the correct control chart type for the type of data collected makes interpretation more sensitive for detecting special cause variation. The ORYX measures are calculated as proportions (rates), ratios, and means (continuous variable data, such as average length of stay), and this information forms the basis for selecting the correct type of control chart. In addition, the average rate (especially for rare event measures) and the average

JCAHO’s Control Chart Analysis

Control charts can be created when there are at least 12 data points for a given measure for an HCO. The 12 points may include missing data points. For example, for a measure having a data collection “begin date” of July 1, 2001 (third quarter 2001), a control chart is created after the second quarter of 2002 data are submitted to JCAHO, regardless of any missing data points during that 12-month period.

The charts depict a center line (overall process average), +3 and −3 sigma lines (UCL and

Small Monthly Sample Size Issues for Control Chart Use

The X-bar S chart can be used for variables data unless the monthly population size is extremely small (fewer than 10 cases per month). However, the p-chart and u-chart, which are used for attributes data, may require special statistical adjustments when the population size is too small.

When a small population size precludes the use of an X-bar S chart for continuous data, JCAHO uses the XmR (“average moving range”) chart for its control chart analysis. The XmR chart is also called an

Case Studies

Four case studies using various types of control charts follow. (Source: Joint Commission on Accreditation of Healthcare Organizations: Benchmarking in Health Care: Finding and Implementing Best Practices. Oakbrook Terrace, IL, 2000.)

References (12)

  • K.Y. Lee

    Special report: An overview of the Joint Commission’s ORYX Initiative and proposed statistical methods

    Health Services and Outcomes Research Methodology

    (2000)
  • Joint Commission: Facts about ORYX: The next evolution in accreditation. www.jcaho.org/search_frm.html; see...
  • Getting hospital core measures on track: The continued evolution of ORYX

    Jt Comm Perspect

    (2001)
  • R.G. Carey et al.

    Measuring Quality Improvement in Healthcare: A Guide to Statistical Process Control Applications

    (2001)
  • D. Montgomery

    Introduction to Statistical Quality Control

    (1996)
  • D.J. Wheeler et al.

    Understanding Statistical Process Control

    (1992)
There are more references available in the full text version of this article.

Cited by (0)

View full text