After thirty years of auditing in three entirely different work environments—city, county, and state—I learned far more than I was taught. Continuing professional education is valuable, but there’s just so much to be learned that can’t fit in all those classes, conferences, and webinars.
I also learned that some topics should be taught but aren’t, while others simply can’t be taught except through experience.
Evidence
Evaluation of evidence is one of those essential topics. Though I’ve sought out practical audit training on evidence, t simply doesn’t exist, so I’ve sought to develop some thoughts on evidence in this guide, writing about it at length in the Evidence Throughline section.
Every audit shop seems to send rookie auditors out to gather evidence and expect on-the-job-training to establish some concept of sufficiency. When I was new, my first assignment was to perform quality control on an audit. I was challenged to think about what was persuasive evidence and the kinds of expectations I would need to meet for evidence. I also saw how audits were assembled, which helped immensely when I started gathering and organizing the needed evidence in my first audit.
Many audit shops are pushed by the “supervision” word in the Standards to think only experienced auditors understand sufficiency enough to perform this function. Everyone needs to understand sufficiency. The most important unmet training need is a conceptual structure and language to allow auditors to discuss the various considerations that comprise sufficiency.
Beyond the lack of clarity, it’s more efficient if auditors can distinguish sufficient evidence from superfluous evidence. Gathering unnecessary evidence can eat a lot of audit hours. Consistent training will help the new hire, improve everyone’s efficiency while assuring compliance with Standards, and reduce problems during quality control reviews.
We developed some training in Oregon to describe the aspects of sufficiency, which was presented at the ALGA conference in San Diego, and it’s the first I’ve seen.
Writing
I’ve already spoken at length about the importance of good writing in the section entitled “Writing a story to engage the public.” I had hours and hours of training, but until I watched a journalist craft an audit report I didn’t realize how badly I—and most auditors—wrote. In that audit, the narrative flew, the findings were accessible, and the agency’s story was alive. In contrast, my sentences were cumbersome, stale phrases with no spark of humanity. I still don’t write as well as I wished, but the much higher standard keeps me humble and striving.
This skill is in us, but our report writing smothers it. At our internal meetings on audit progress, we talk about our findings in clear and compelling ways. Then we write as if it were a technical manual for a nuclear spectroscope. Here’s my scientific explanation: our brains are close to our mouths and much farther from our fingers on a keyboard. The distance degrades the message.
In my career, I probably had at least three hundred hours of training in order to improve my writing skills. It always seemed to miss the mark. Stan Stenersen’s training was the exception. He taught how to translate complex findings into accessible concepts, which was a big leap. He also covered the second big step: changing the whole organization to address its writing and editing behavior. Senseless editing will puree the best sentences to a flavorless pulp.
The last step wasn’t covered though: expanding the vocabulary to enliven the narrative. We need good training, but organizations must empower our auditors to write stories for the public in a living language.
Audit procedures
Two perspectives are needed here. Managers need to be exposed to various ways that audits can be performed, and staff need to be trained on the procedures chosen by their agency managers.
Anyone who thinks there is only one way to conduct an audit hasn’t been on a peer review. It can be challenging for new peer reviewers to accept that their way of conducting an audit is not the only way. Once they’ve overcome that mindset, they’ll be surprised at how much they can learn about different ways to conduct audits while still complying with Standards.
Although there are multiple approaches to produce an audit, they’re not being taught. Even if they were, entire management teams need to be trained on the alternatives so they can agree upon the best steps for their organization. Once an approach is decided on and documented, the rest of the staff needs to be trained. The effort to develop training is important for new hires, because the organization will benefit from a shorter learning curve and the new hire will experience a better success rate. You may think experienced auditors would need less training but I’ve also seen generations of auditors, and managers, who never depart from their early training.
Facilitation
One of the most valued classes I took many years ago taught how to facilitate a meeting by applying a Committee of Sponsoring Organizations (COSO) framework. Most of the class’s time was spent on skills and tools for leading group meetings, with an emphasis on the responsibility of a facilitator to promote open dialogue and discussion to reach a decision without influencing it. It was not taught by an auditor, but by a facilitator.
The role of a facilitator was the big takeaway for me. Of course, in exit interviews I never declared I would play the “facilitator,” but I applied the skills. Watching the discussion from an observer’s standpoint—rather than a participant’s—gave me new insights, especially when I saw the unspoken body language. What was most valuable to me was the ability to switch from facilitator to decision-maker when needed.
All managers and staff auditors could benefit from this training.
Finding findings
I have said this before in many ways but we need to ensure that our audits address actual recurring problems more than theoretical risks. Identifying the biggest problems should be the first step of an audit.
Sure, we can train and apply the COSO model or other risk analyses to identify many things that might go wrong in an organization. Those are good textbook fundamentals, but we should also train auditors to spot the problems that impede the agency’s mission, teaching them, for example, to contact staff, stakeholders, and peers of the auditee. Reality is much richer than any deductive or theoretical description. Of the hundreds—or thousands—of decisions and actions taken each day in an agency, which could be reconsidered to produce better outcomes?
As I’ve said before, the Standards are silent on methods for audit topic selection. They are written as if setting the scope and objective was the first step of the audit, followed by developing findings within that box. It’s like your family doctor deciding, without even talking to you, that this visit will be about your spleen, just because there might be something wrong with it.
Using a more inductive audit approach is not recognized in the Standards, though it is not prohibited in any way. We need guidance on spotting and choosing potential audit findings, as well as training to compensate for this omission.
Organizational dynamics
The root causes of findings spring from the organization. Employees may not have the supervision necessary for complex tasks. Leadership may not be hearing what is actually happening on the front lines. Resources may not match the workload in the various parts of the organization. Or frontline staff may not be hearing clear direction from leadership.
Here is a sad truism that we have all seen: organizations dominated by frustrated and unhappy employees are less likely to provide good services.
Talking to managers and frontline staff can reveal these problems and auditors can then follow the causal chain to their adverse effects on public services. I know public administration classes teach how to organize people and resources to achieve an objective, which can be great criteria and help to guide recommendations. A class on common dysfunctions in organizations could also help auditors spot the deficiencies more quickly.
Untrainable topics
One of my phrases to auditors has been, “This is not your typical audit, but no audit is typical.” Auditing is jazz, not a formal composition.
To use another metaphor, each audit follows its own path through the landscape of an organization and its mission. A new hire sees that landscape as if it were a maze. The maze is a combination of audit Standards and procedures, the finding, the relationship with the auditee, and many other factors. The movement within the maze seems haphazard, without an understanding of distance or overall direction. Afterward, the new hire achieves a better understanding, getting what amounts to a bird’s eye view of the path taken and the structure of the maze. Yet the next audit is likely to be a different maze, with some factors altered.
Training by rote produces unrealistic expectations for auditors. By “unrealistic” I mean “impractical and unsuccessful when automatically applied.” Auditors need to find the underlying and undiscovered themes that impede an organization, then draw the managers onto the path to improvement. And no matter how well the plans are laid, surprises will happen..
No training can replace the experience gained from multiple audits, and the experience gained from the departures in subtle and important ways from a standard path. And multiple audits can instill an understanding of the role and conduct of an auditor in the course of an audit. We can talk about attitude and ethics and objectivity and many other important expectations, but again, the uniqueness of every situation adds another layer of real-life complexity.
Training can’t address the unique contexts of either an audit organization or the organization being audited. The attitude toward the audit organization within the jurisdiction is one determinant of the auditor’s role. The relationship with elected officials is delicate and complex, starting with the authority of the auditor, whether they’re legislatively directed or independently elected. Some general principles can be set out in training, but the political context of the audit organization plays out in agency interactions and public perceptions of the auditors as they go about their work.
If the agencies respect or distrust or trivialize auditors, then each attitude requires a different communications style that can’t be taught in a national seminar. Neither can training instill the correct behavior in an auditor who encounters angry auditees, an abused whistle blower, or the distraught victim of inappropriate agency services.
So there is no substitute for experience. And there is no substitute for a good mentor, on-the-job trainer, or communicative leader to offer guidance for new hires and experienced auditors. The field of performance auditing depends upon an apprenticeship period for its new hires because there’s so much that can’t be taught, or that is particular to the audit organization’s working environment.
A performance auditor’s learning curve will truly span an entire career because we must never stop seeking ways to be better. And, continuous learning is one of the most satisfying elements of this profession.
