Using the Balanced Scorecard
Planning and Conducting Performance Audits
Southeastern and Southwest Intergovernmental Audit Forum
Joint Conference
August 30 September 1, 2006
Using BSC for Performance Audits
Shared understanding of scorecard
Why bother?
Knowing-doing gap: why you arent
more effective
How to use a scorecard to structure
audit
How to use scorecard to manage and
evaluate audit
Every picture tells a story, dont it*
* apologies to Rod Stewart
Mission story
Customer Focus Story
Internal Process Story
Learning and Knowledge Story
Financial Story
Organizational story
Truth is truth
To the end of reckoning.
William Shakespeare
Measure for Measure, V.i.45
SAO Organizational Scorecard
Balanced Scorecard - Organizational
Actively provide government leaders with useful information that improves accountability
Perspective
Objective
- be a world leader in professional services - competitive with any private firm in quality of
services and efficiency
- identify fiscal impact that equals SAO budget, that is, deliver recommendations that help
the state use its limited resources more effectively and efficiently
- produce results that add value to the operations of the legislature and the client agency
- provide legislators and their staff with information that they find useful in policy
deliverations and decision making
Mission
Measure
Target
OPI - Useful Information Rating
Rating of 2.50: scale from 1 to 3
Org-Fiscal Impact
10 x Annual Budget = $160,000,000 per year
OPI-Customer Feedback Rating
Rating of 4.0: scale from 1 to 5
OPI-Service Delivery Status
85% of budget
Org-Employee Feedback Rating
Rating of 3.75; scale from 1 to 5
- spend taxpayer money wisely to acquire the knowledge, equipment, and other
resources needed to achieve the SAO mission and deliver high quality services
Customer Focus
- build respectful relationships based on trust, cooperation, and open communication with
our clients
- create deliverables that are easy to read, understand, and to use
- maintain effective working relationships with those we audit
- provide customers with information that helps them improve their operations and their
accountability
Internal Processes
- continuously identify better approaches to our work
- deliver high quality products and services within the fastest time frame
- focus resources on service delivery
Learning & Knowledge
- provide information and services when we say we will
- create a positive work environment that encourages creativity, innovation, high quality
results, and low turnover
- give employees the opportunity to fulfill their potential
Org-Workforce Capacity
85% of needs identified
Org- AAS Productive Workforce
98% to 102% of budget
- promote growth of knowledge and learning, risk-taking, innovation, and change
Financial
- deliver support services that help the SAO strategies to achieve their mission
- maintain the right number of people with the right knowledge at the right time to deliver
SAO services
Org-Audit Support Workforce
98% to 102% of budget
Org-Budget Compliance - Dollars
98% to 100% of budget
Org-EES-Cost Recovery
100%
Org-MAS-Cost Recovery
100%
Washington State Scorecard
North Carolina DOT
Safety
Goal 1: Develop and deliver services, products and projects
that ensure a safe and secure environment for employees,
stakeholders /partners, and customers
Objective A: Reduce employee accident and injury/illness rates
1. Reduce number of DOT vehicle and equipment accidents by 5%
from 2004 benchmark (Sanderson)
2. Achieve an overall DOT incident rate of 5.5 or less (Sanderson)
3. No work place fatalities (Sanderson)
Objective B: Reduce accidents, injuries, and fatalities among
stakeholders/partners
1. No work place fatalities (Sanderson)
2. Provide real-time information to employers regarding their
employees driving records (Tatum)
3. Increase the number of employers with access to the
Transportation (Tatum)
BSC Causal Relationships
Mission achievement
Satisfied customers
Efficient, high quality internal
processes
Skilled employees
Allocated resources
Mission
Customer
Focus
Internal
Processes
Learning &
Knowledge
Financial
Research on Scorecard Use in
Public Sector
Comprehensive range of measures covering
operations
Improved strategic planning
Helped translate strategic objectives into
improvement in operations at service level.
European Quality Award Model (similar to TQM)
compared to BSC
BSC linked better to strategy
BSC easier to adapt to customer measures
BSC easier to involve people at all levels and
to relate to specific job
BSC good link between process measures
and business strategy
The Art of Getting Things
Done
About 10% of all managers take
decisive, purposeful action.
Managers know what to do.
Managers have the knowledge.
Managers are very busy.
Managers permanently face requests.
Baruch and Ghoshal
Do not mistake activity
for achievement.
Mabel Newcomber
20th century academic and writer
Manager Actions
High
High
Detached
Detached
Managers
Managers
Purposeful
Purposeful
Managers
Managers
20%
20%
10%
10%
Procrastinators
Procrastinators
Busy
Busy
Managers
Managers
Focus
Low
Low
30%
30%
Low
Low
40%
40%
Energy
High
High
Making choices
Willpower, not motivation, drives
purposeful action
intention formation
engaging willpower
intention protection
Scorecards and Focus
Scorecards tell a story
Scorecards focus attention on critical
activity and results
Goal setting and management lead to
focus
Reticular Formation
Reticular Activating
System - RAS
Brains attention center
Filter between conscious
and subconscious
Real events and thoughts
are the same
Characteristics of a
scorecard
Strategies align with mission
Relationships among limited number of measures
are clear, provide relevant information
Scorecards tell a story about what is important and
about accountability to everyone in the organization
Scorecards balance
financial / non-financial information
short term / long term targets
lead / lag measures
Why don't you write books
people can read?
Nora Joyce to her husband James
1882-1941
Scorecards and Auditing
Scorecard as business model
Key activities in scorecard
Measures reflect key activities
Audit verifies outputs / outcomes
Using BSC to Structure an Audit
Balanced Scorecard: Audit Measures Worksheet
Perspective
Mission
Objective/Goal/Outcome
Measure
Target
Why do we exist?
Why are we doing this?
Customer Focus
To achieve our mission, what
customer needs must we meet
What must we do to satisfy,
retain and attract new
Internal Processes
To satisfy customer needs and
to achieve our mission, in what
internal business processes
Learning & Knowledge
To achieve our goals, what
must the organization know
and learn, and how must we
What must we do to develop
employee skills and technology
to continue adding value and
Financial
To satisfy our stakeholders and
to meet our mission, how will
we allocate our appropriated
How will we acquire
resources?
@work: Auditing Scorecards
City of Bedrock Balanced Scorecard
Provide responsible leadership and the services and infrastructure necessary for a safe environment and a high quality of life for our citizens
Perspective
Mission
Objective
Measure
Provide services that result in overall citizen satisfaction with
community safety
Citizens believe they have a high quality of life
City leaders are responsive to citizen needs and desires
Customer Focus
Telephone courtesy of employee - overall citizen satisfaction
Provide police services perceived as satisfactory by citizens
Phone calls directed accurately
Average score 85%
Average score 85%
Response to items on citizen survey about satisfaction with
responsiveness of leaders
Average score 80%
Response to items on citizen survey about courtesy of call
center employees
Response to items on citizen survey about satisfaction with
police services
Satisfaction with ambulance service
Response to items on citizen survey about satisfaction with
ambulance services
Number of misdirected calls per month
Internal Processes
Staff members are courteous while providing services
Phone call monitoring by supervisors
Quick response to service calls
Effectiveness in maintenance of city buildings
Target
Response to items on citizen survey about feelings of safety
Response to items on citizen survey about satisfaction with
quality of life
Average time to respond to service calls
Accuracy of response to requests
Average score 80%
Average score 80%
Average score 80%
Fewer than 5 instances of misdirected
calls in 1 month
Fewer than 5 instances of
discourteous responses by agents in 1
month
Target to be determined after baseline
established
Acquisition excellence
Learning & Knowledge
Employees received adequate training to perform efficiently
Hours of job-related training per employee annually
Employees are treated as valued customers
12 hours of job related training / yr
Employee engagement indicates overal satisfaction with the
city as an employer
Financial
Operates within budget
Budget variance measured monthly
Variance from budget is 2%
Finance reports are prepared and delivered to users timely
Number of late reports per month
No. reports delivered after due date
Finance reports are accurate and reliable
Number of errors in standard finance reports / month
No. of errors monthly
Payroll is processed accurately
Number of paychecks with errors per pay period
Payables are processed timely
There are enough employees on staff to perform services
Number of vendor payments made more than 30 days after
invoice is received
2 paycheck errors per month
% of late payments per month
@work: Auditing Scorecards
City of Bedrock Balanced Scorecard
Provide responsible leadership and the services and infrastructure necessary for a safe environment and a high quality of life for our citizens
Perspective
Audit Notes
Measure
Target
Mission
Response to items on citizen survey about feelings of safety
Average score 85%
Response to items on citizen survey about satisfaction with quality of life
Average score 85%
Response to items on citizen survey about satisfaction with responsiveness of leaders
Average score 80%
Average score 80%
Customer Focus
Response to items on citizen survey about courtesy of call center employees
Response to items on citizen survey about satisfaction with police services
Internal Processes
Response to items on citizen survey about satisfaction with ambulance services
Number of misdirected calls per month
Phone call monitoring by supervisors
Average time to respond to service calls
Learning & Knowledge
Average score 80%
Average score 80%
Fewer than 5 instances of misdirected
calls in 1 month
Fewer than 5 instances of
discourteous responses by agents in 1
Target to be determined after baseline
established
Hours of job-related training per employee annually
12 hours of job related training / yr
Employee engagement indicates overal satisfaction with the city as an employer
Financial
Budget variance measured monthly
Number of late reports per month
Number of errors in standard finance reports / month
Number of paychecks with errors per pay period
Variance from budget is 2%
No. reports delivered after due date
No. of errors monthly
2 paycheck errors per month
% of late payments per month
Number of vendor payments made more than 30 days after invoice is received
Auditing Scorecards
Review architecture and security of
e-systems
Review relevant policies and plans
Review goals and measures
Identify unintended consequences
Share information with management
Result: ensure that scorecard
processes function as intended,
emphasizing the quality and strength
of process controls and identifying
ineffective metrics
Evaluating Audit Processes &
Results
Mission
Customer Focus
Internal Processes
Learning & Knowledge
Financial
Did client/auditee receive
useful information?
Did client find the
information easy to
understand? Timely?
Project surprise free?
Was the audit work and
project management high
quality and effective?
Did audit team members
learn and grow on this
audit? Was employee
climate positive?
Did we adhere to our time
and dollar budgets?
Evaluating Audit Processes
& Results
Balanced Scorecard Measures - Audit & Review
The State Auditor's Office provides services that improve and assure the quality and accuracy of information for business decision making.
Perspective
Mission
Customer Focus
Objective
- Provide various levels of assurance
Measure
A&R-Audit Plan Completion Status
Target
85% of budget
about that: compliance, information
OPI-Useful Information Rating
Office Wide Target = 2.50, scale from 1 to 3; Project Level Target = 3.00, scale from 1 to 3
accuracy, performance achieves
Org-Fiscal Impact
Audit & Review Target = $160,000,000.00
the expected results, and systems
PL-% of Recommendations Action by Client
90% of budget
effectiveness
PL-Reports with Reviews of IT Systems
60% of budget
effectiveness
PL-Useful Information Rating
Office Wide Target = 2.50, scale from 1 to 3; Project Level Target = 3.00, scale from 1 to 3
- Balanced reporting
A&R-Reports Delivered On-Time
90% of budget
- Credible
OPI-Customer Feedback Rating
Rating of 4.0: scale from 1 to 5
- Independence
PL-Customer Feedback Rating
Average Rating of 4.00; scale from 1 to 5
- Objectivity
- Recognized expertise
- Timely service
Internal Processes
- Audit tools and technology
- Auditing
A&R-Hours Audit Staff-Assurance Services
A&R-% Projects with Complete BSC Data
- Packaging information/deliverables
OPI_A&R-Audit Duration
85% of budget
PL-Projects Addressing at least 2 of 3 SAO IT Objectives
60% of budget
PL-Projects Meeting Fieldwork Checkpoint Meeting Date
80% of budget
(i.e., production)
- Project management
74% of the budget
90% of budget
- Quality control and compliance with
standards
- Risk assessment (statewide and
project level)
Learning & Knowledge
- Positive employee climate
A&R-% of Highly Qual PMs
30% of senior staff
- Technical expertise
A&R-Attendance at Core Courses
93% of budget
- Training
A&R-Compliance w/ Training Standards - CPE Standards
100% of eligible A&R staff for the quarter
A&R-Workforce Capacity
85% of needs identified
Org-Employee Feedback Rating
Financial
Rating of 3.75; scale from 1 to 5
- Budget allocation (size of efforts)
A&R-% Audit Staff with > 100 Hours Comp Time
5% or less
- Recover cost for billable services
A&R-Audit Resources
< 3000 hours
# of productive employees (for the month) established by Direct of Audit at the beginning of each
Fiscal Year
85.00% to 100.00% of budget
A&R-Productive Workforce
F_Budget Summary
10
References Used
McAdam, Rodney and Edel ONeill. 1999. Taking a critical perspective
to the European Business Excellence Model using a balanced
scorecard approach: a case study in the service sector. Managing
Service Quality 9 (3): 191-197.
McAdam, Rodney and Timothy Walker. 2003. An Inquiry into
Balanced Scorecards Within Best Value Implementation in UK Local
Government. Public Administration 81 (4): 873892.
Nohria, Nitin, William Joyce and Bruce Roberson. 2003. What Really
Works. Harvard Business Review 81 (7).
Phillips, Jason Keith. 2004. An Application of the Balanced Scorecard
to Public Transit System Performance Assessment. Transportation
Journal 43 (1): 26-55.
Schmidt, Chris. 2005. The Drivers View. Internal Auditing 62(3): 2932.
Talbot, C. 1999. Public performance towards a new model, in
Special Issue: Quality issues in the public service. Public Policy and
Administration 14 (3): 1535.
Wong, Jeff. 2000. The Role of the Balanced Scorecard in Operational
Auditing. Internal Auditing 15 (4): 33-36.
Contact
Deborah L. Kerr, Ph.D.
deborah.kerr@heart.org
512.433.7106
11
