Presented to the Biennial Intergovernmental Audit Forum
It is 1998 and a grand time to be a performance auditor! Managers are emphasizing measurement and improvement. Elected officials are talking about increased accountability and obtaining assurances of results. They are investing in data systems and analysts, and incorporating measures into their reporting and budgeting systems. These efforts to measure performance are occurring across all levels of government in every corner of our country, and the world. But I fear their enthusiasm is short-lived.
You see, we have been through these periods before. It began in 1911 when F.W. Taylor showed how “scientific management” could increase productivity. He applied scientific methods to industrial operations to find improvements. For example, he studied how apprentice bricklayers were trained and also filmed the best bricklayers, but found that the training did not match the actual practice. His recommendations improved training and overall productivity. (Another interesting example of his methods applied to 1913 Portland government is included later in this book)
On the government side, the New York Bureau of Municipal Research applied his principles to advocate for civil service, better budgeting, and streamlined administrative systems. In subsequent years, other management theories became more popular than Taylor’s fundamentals.
During the Great Depression in the 1930s, managers focused on waste and inefficiency, again turning to measurement to increase productivity and reduce costs. In the 1950s the tools created during World War II—such as statistics, queuing analysis, and information theory—were used to manage better. The early 1970s brought Peter Drucker’s Management by Objectives, which started from goals and worked back to measured activities. And here we are in the 1990s with Total Quality Management and W. Edwards Deming’s methods for measurement and improvement.
I think that there are lessons to be learned in this history. The value of measurement seems to be regularly rediscovered on a twenty-year cycle. But there’s also a down cycle, when measurement inevitably loses its eminence to other management styles and theories. Each time we discover measurement we apply it everywhere with enthusiasm until we see its limitations and renew our search for other means of achieving our desired outcomes.
There’s a basic formula at work when we discover the value of measurement: the cost of the measurement must be less than the value of improvement that can be gained. The ratio of cost and benefit has shifted with new analytical tools and technologies. It started in the 1910s when motion pictures were used to analyze and streamline assembly-line activities. The recent revolution in computers and software has reduced the cost of gathering, analyzing, and presenting measurement information.
At some point in the near future we’ll run into some areas where the cost of our current measures exceed the value for management and decision-making. We’ll find that the quality of a service or the extent of human behavioral change cannot be measured without great expense. Perturbations, time lags, and outside forces will confound our measures. And finally, measures often require new measures to explain why a change has occurred. These are the methodological challenges of measurement.
Social and political pressures can also reduce the popularity of measurement. Obviously, authors and consultants can’t make money selling what everyone knows. In three or four years a senator or commissioner will ask: “We’ve invested huge amounts of resources in computers, custom software, data collection, analysts, reporting mechanisms and management training. Why aren’t we realizing the great benefits that were promised?”
As auditors we should be concerned about how that question is answered. One wrong answer is: “Bad data.” We shouldn’t poison the well of data—we still want to use it. I would suggest that we answer the question with, “No value? That’s odd, our auditors can generally find all sorts of valuable recommendations from that kind of data when we conduct performance audits.” This preserves the value of the data and also accomplishes another important task: creating an aura of superhuman auditing powers.
Rest assured that measurement doesn’t go away during the twenty-year cycle when other management styles and theories gain prominence. Each cycle of discovery creates investments in data gathering and measurement tools which continue to provide auditors with valuable audit findings, even when the enthusiasm for measurement dies down.
I don’t want to leave you with the impression that I have anything against measurement, or wish it an early demise. I think it is important for performance auditors to be involved with measurement because we have become experts at it. My office has been involved in performance measurement in Multnomah County for at least five years, and I continue to push for more and better measures to be used in decision-making. My office participated in the development and training of our program-performance budget and we reviewed the 250 performance measures developed by the programs. My office is also responsible for tracking over 85 community indicators which we call the Portland-Multnomah Benchmarks.
The survival of these measures in the future depends upon how well we can produce something of value with them. As performance auditors, we are always finding value in measurement, but we go at it a little differently. Performance measurement is the construction of a dashboard full of gauges to detect problems and inform us of conditions. Performance auditing is the observance of a problem that we then build a gauge to measure. There is more selectivity in performance auditing, which reduces the ratio of measurement costs to the value produced.
Since 1911, each measurement effort has added to the layers of management information and analytical tools available to performance auditors. Regardless of the popularity of measurement in the future, as auditors we will have more data to sift through for valuable recommendations. As a result, performance auditing will continue to be the most consequential effort we can perform, which persistently demonstrates the importance of measurement.
