Presented at the 2005 ALGA Conference
Auditing isn’t the only accountability tool. The mission, framework and methods of performance auditing have parallels in other accountability mechanisms and problem-solving strategies. As the elected auditor of the city of Portland, I have greater freedom to explore accountability strategies to accomplish the mission of my office. In addition to the ten staff conducting performance audits, two other staff work with the Portland Multnomah Progress Board to provide a different kind of accountability program in the auditor’s office.
The Progress Board was created in 1993 to track a comprehensive set of key conditions in the community related to the city and county strategic plans. The measures are called benchmarks and are based on a system developed by the state of Oregon several years earlier. (Oregon’s meaning of “benchmark” is more akin to the surveyor’s term, which is a permanent marker used as a reference point, rather than the more contemporary coinage as a standard for comparison.)
About half the local measures are the same as the state measures, including high school completion, teen pregnancies, and poverty. The other half reflect the more urban nature of the community, such as proximity to parks, air quality, and street cleanliness.
Benchmarks are the highest-level measure in a performance measurement system. Public funding provides for personnel and supplies which produce the construction and services that serve our community. We can measure many the resources and efforts, but not the extent of the outcomes we hope to achieve. Benchmarks measure conditions in the community, and they are often the most relevant and important measures for the citizens.
Yet benchmarks are often affected by multiple agencies and forces. For example, housing affordability may be affected by efforts like public subsidies, but larger economic forces such as interest rates and unemployment often have a stronger influence. There is less expectation that these issues can be solved quickly, so a greater focus is needed on long-term monitoring and reporting.
Because benchmarks are more difficult to influence, collaboration among agencies and the community are necessary. Benchmarks can focus agency efforts to produce greater impacts as well. For that reason, the Portland Multnomah Progress Board consists of sixteen community leaders, such the mayor and county chair, school superintendent, and business and non-profit leaders.
The mission of the Progress Board is to monitor the benchmarks and act as a catalyst for change.
The two staff gather fresh data from other agencies to update the benchmarks, develop supplemental information to understand the drivers of trends, and communicate the knowledge to organizations that can respond and influence the benchmark. The benchmarks are organized in seven clusters:
- Economy
- Education
- Environment
- Governance & Civic Participation
- Health & Families
- Public Safety
- Urban Vitality
One important staff task is looking for new and better benchmark data. Sometimes an agency will discontinue gathering a measure, but more often, we discover a new, more relevant data set for our community—for example, it may provide more frequent reporting, or a better geographic fit. Ten years ago, we only had data for about thirty-five of the ninety-five benchmarks we wished to monitor, and now we have data on over sixty. Where possible we also develop benchmark data for comparable jurisdictions so we can compare our conditions and trends.
Portland-Multnomah County Progress Board Home page, May 2001
Another strategy is to track the benchmarks at smaller geographic levels that are more relevant to policymakers. We use geographic information software to cut data into neighborhoods, school districts, and even elementary school areas. As a Census Affiliate we provide census data assistance to other agencies.
Because of the Progress Board, the United States Census Bureau chose Multnomah County as one of the first six sites in the country to participate in the American Community Survey (ACS)—essentially an annual census produced by sampling. This helps us spot trends we would otherwise only see every ten years. For example, the Multnomah County population is richer with college graduates because of in-migration patterns. The first chart below shows a very positive picture, whereas annual ACS data shows the trend is flattening.
An example of our supplemental analysis is some work we did to identify high-performance schools so that principles could compare teaching methods. This helped spread successful strategies for boosting reading and math skills.
Our analysis of smaller geographies allowed us to track how poverty in our community was displaced by gentrification. Our information resulted in the siting of a new facility to house state, school, community college, and county services to better serve the neediest families in our community.
Our most far-reaching effort is represented by our in-depth research of individual benchmarks. In these studies, we move from the community condition down to the contributing factors and ultimately an inventory of agency resources and efforts. We conducted work in the areas of school readiness, educational success, and salmon recovery. Below is the diagram that links a benchmark back to agency performance.
The biggest challenge our efforts faced over the past eleven years is the survival of the Progress Board. Everybody likes measures but nobody wants to pay for them. Measurement is difficult to justify and protect when budget cuts arrive. We needed to develop activities that produced visible benefits. We do not generate specific recommendations because we function more to better inform leadership about trends in the community and the influence of other factors. It is ironic that these kinds of activities related to measurement are extremely difficult to measure.
While community indicators and performance auditing both use measures to identify problems and strategies for improvement, it is easier to show clear impact with performance auditing. Benchmark results are more anecdotal, yet our educational efforts have created a constituency among agencies who rely upon the objectivity of our work.
Recently, the new mayor initiated a community-wide strategic planning effort and we are preparing a report of conditions in our community to assist in setting the context. Our benchmarks sprang from a similar effort fifteen years ago, and we expect to adapt the benchmarks to best reflect this new plan.
At the national level, the Government Accountability Office issued a report in May 2003 entitled Forum on Key National Indicators: Assessing the Nation’s Position and Progress. (GAO-03-672SP) The report recommends the development of national indicators and provides an extensive inventory of other indicator systems. We are eager to see the creation of national indicators which will encourage other communities to start tracking their conditions.
Postscript: Over the course of ten years the staffing was cut from 8 to 2, then to 0 in 2008.
