I refer to that phrase from The Big Lebowski because I think it captures a challenge we face when auditing public safety agencies. In our realm of auditing, we examine the actions of various agencies for ways to improve. And agencies in the realm of the criminal justice system examine the various actions of individuals to punish/rehabilitate them. Most often, we’re all eating the bears.
The audit profession shares some common fundamentals with the justice system: objectivity, independence, procedures, and Standards. Like the police, we look for non-compliance with rules and laws. Like detectives, we gather evidence via interviews, examination of records, and scientific analytical tools. We may find evidence of a crime and our Standards require us to notify law enforcement, with our workpapers sometimes used as evidence.
Some of the policing phrases and theories also apply to auditing, with a bit of irony as well. Police arrest miscreants for disorderly conduct, and auditors report ineffective internal controls. Police respond to 911 calls and auditors have a hotline. Community or proactive policing aligns with auditors who identify risks and lapses early enough to prevent large problems.
However, auditors encompass several criminal justice functions that are intentionally kept independent. Auditing doesn’t have adversarial prosecutors and defense attorneys to debate evidence, or a judge to apply our Standards like laws. Instead, when we produce our reports, auditors must combine all those roles, as well as the role of evidence-gathering police investigators. This mix of roles seems acceptable since our audits don’t result in forfeiture of property, incarceration, or the death penalty. Nonetheless, this combination does have risks, making it important for us to follow Standards and to regularly and successfully pass external quality control reviews. And just as important, we must be open-minded when the auditee disagrees with our draft findings. Overzealous prosecution can happen to us also.
We can’t avoid public safety audits because those services often consume a large portion of a jurisdiction’s general funds. Nonetheless, when auditors enter the criminal justice realm, the rest of the Big Lebowski quote has added meaning: “…and sometimes, well, the bar, he eats you.”
As a caution, each public safety agency has its own culture and competencies, and I only intend my discussion to flag possibilities you may encounter in your jurisdiction, not certainties. Below are some accumulated insights and strategies about auditing public safety agencies for efficiency and effectiveness.
In general, the criminal justice system is of great interest to the public, as evidenced by hours and hours of television dramas each week about police, crime labs, defense attorneys, and prosecutors. A couple of fire and prison dramas have now been added to the mix of public safety fiction that often beguiles the public. Make sure you set those assumptions and expectations aside during your audit.
Television has no 911 call center dramas yet, but this agency is often the starting point for police, fire, and ambulance services. It depends upon a robust technology operated by highly competent call-takers and dispatchers. The technology also captures a wealth of workload and response data that can be analyzed for resource deployment of all these services. Call-taking and dispatch can be highly stressful, resulting in higher turnover which results in vacancies that require overtime spending. Unnecessary overtime can also be caused by a mismatch between staffing and the variable workloads during the hours in the day and the day of week.
Overtime is a recurring theme in many criminal justice agencies, and the rich data invites audit scrutiny. Beyond the risk of fatigue-addled decisions, overtime needs to be approached differently than standard assumptions. Sometimes overtime is an avoidable expense, but for other staffing issues it can be a very sensible solution. Adding the (increasingly expensive) health benefit costs to the other fixed costs of an additional employee—hiring, training, and leave—means that the total cost of an hour for a new employee is nearing the cost of overtime. Several public safety agencies have “fixed posts” that must be staffed—911, police, fire, jails—and may be unnecessarily spending overtime. Look at the jail overtime audits of Multnomah County and the prison overtime audit for Oregon to see how this approach is applied.
As I mentioned previously, police patrol officers respond to the 911 calls; some efficiencies can be found by looking for mismatches between the workload and the staffing.
The next largest police program expense is often detectives. Look at FBI clearance data to see how your jurisdiction fares in comparison to others. If they are solving fewer crimes and arresting fewer suspects, then some scrutiny might be worthwhile. Both Portland and Toronto conducted good audits of investigations a while ago.
Fire is the other agency responding to 911 fire and medical dispatches. Fires are a diminishing workload due to better building codes, reduction in smokers, smoke detectors, and other factors. Medical calls have become the main workload for these agencies, and the workforce and traditions are adapting to this shift, though there are some obstacles to the transition which presents potential topics for audits. For example, jurisdictions are only now starting to dispatch medical vehicles, rather than fire apparatus, to medical calls.
When police build a case for convicting a suspect, prosecution and defense attorneys become engaged. Very few decisions about charges are determined in a trial. Over 90% are settled between the prosecutors and defense attorneys, then approved by a judge. Efficiencies are difficult to determine, and effectiveness findings are even more challenging. We found that some county detention facilities were underused and encouraged the prosecutors to consider these alternatives in their sentencing negotiations. One caution: auditing attorneys can be challenging because they operate in an adversarial environment and may respond aggressively.
One other caution I should mention is the sensitivity of elected officials. Elected prosecutors, judges, and sheriffs are highly concerned about criticism that could threaten their re-election chances. Be aware and be prepared for your audit to be politicized. The elected official may vehemently deny your findings or, if they have strong voter support, simply ignore your recommendations.
Judges are also challenging to audit as they are elected and attorneys and their efficiency and effectiveness is nearly impossible to evaluate. Equity issues and disparate treatment may be possible topics when examined in the aggregate of sentencing decisions, though determining cause and assigning recommendations can be difficult. Judges sign off on the 90% of cases that don’t go to trial; most of the time because judgments are negotiated between the prosecutor and defense attorney. We raised questions in the early 1990s about disparate sentencing and were chastised by the presiding judge in his chambers.
Once convicted, many offenders are sentenced to jail or prison, put on probation, or some combination. This is one of the most complex and challenging aspects of the criminal justice system to audit. We often think of the linear sequence, but offenders are often in multiple statuses. For example someone in jail awaiting trial may also be on probation for a previous crime, which adds multiple decision points and actions. Then add to the status mix whether there are concurrent and consecutive sentences, and various conditions and restrictions, and holds from other jurisdictions. Decisions about placing offenders on probation or returning them to custody are risk decisions, and particularly difficult to audit. As a result, auditing the management of these populations is complex.
One other factor that can complicate recommendations are the limits and procedures set forth in collective bargaining agreements, and the resistance to changing them that could arise from the bargaining unit.
In 1995 I conceived of an audit that might address a serious problem in Multnomah County. The courthouse facility, which was nearly a century old, was a maintenance nightmare and far exceeded its useful life. It was built of unreinforced masonry, likely to collapse in any serious earthquake, but building a new courthouse would be costly and commissioners were reluctant to ask the voters to fund it.
We researched for criteria on the ways to build a modern, secure, and cost-effective courthouse. We analyzed the current usage of the building and forecast the need for additional judges in the future. We applied the square footage costs of newly constructed courthouses to the forecasts and estimated total costs for construction, as well as proposing a parking lot site next to the city booking facility. The figure below showed that most court sessions had fewer than a dozen spectators, which could reduce the square footage needed in many courtrooms.
Multnomah County Court Space Needs: Cost saving alternatives, p.27
The plan was met with fierce resistance from the judges who would have to give up their own courtroom, and instead schedule the use of one of the courtrooms on the floor. We had observed the courtrooms and found that no more than half were in use at the busiest part of the day. The presiding judge argued against the plan because judges would have to give up their private bathrooms and share secure bathrooms with other courthouse personnel.
The audit describing the need for a new courthouse was also opposed by the sheriff, who wanted an additional jail instead. A few weeks later he successfully got county board support to seek voter approval for construction of the additional jail. I suspected the sheriff had shared some private polling that supported his jail over a new courthouse. Yet for the next twenty-plus years the additional jail was never occupied, and was recently sold off. As a postscript, a new county courthouse was recently completed, though the preferred site was no longer available. The bear ate us on that one.
Personnel in the criminal justice agencies are trained and expected to establish control over situations and people. They use their presence and their commands and even physical force to help solve problems. We control our audit process, not them, which is contrary to their practices, and you may encounter some conflict: the who-eats-who point of the engagement.
