Performance Audit
Adding Value
ICGFM Conference May 19, 2011
Lily Bi, CIA, CGEIT, CISA
Director, Standards and Guidance
Institute of Internal Auditors
www.theiia.org/Training
Program Objectives
Understand the Landscape –
Internal Audit
Concept and Benefits of Performance Audit
Increase your ability to work with management in a positive
and constructive partnership
• The International Standards for Professional Practice of Internal
Auditing
• Analyze risks and develop a risk-based performance audit
• Learn a value-for-money approach for performance audit
• Final Thoughts – Trend of Internal Audit Profession
[2] www.theiia.org/Training
Program Topics
Unit 1 - Understand the Landscape
Unit 2 - Management Functions and Performance
Measures
Unit 3 - International Standards For Performance
Audit
Unit 4 - Risk-Based Approach (Case Study)
Unit 5 - Value-for-Money Approach (Case Study)
Unit 6 – Final Thoughts
[3] www.theiia.org/Training
Working Agreement
P = Participation
O = Openness
S = Sense of fun
E = Enthusiasm
[4] www.theiia.org/Training
Unit 1
Understand the Landscape
• The road map of internal audit profession
• The definition of internal Auditing
• The definition of performance audit
• Benefit of performance audit
www.theiia.org/Training
Road Map of
Internal Audit Profession
[6] www.theiia.org/Training
Road Map of Internal Audit
1941 - Internal Audit,
a separate and distinctive
discipline.
Complex Services
Clients – the organization
Single Service Multiple Services
•Review all critical functions in
Single Client Single Client an organization
•Review accounting and •Review accounting, financial •Play roles in governance, risk
financial reports and other operations management
•Serve the management •Serve the management •Server the organization: Audit
Committee and Management
•Increase reliance from
external stakeholders
[7] www.theiia.org/Training
About the IIA
• Established in 1941, global
headquarters in Altamonte Springs,
Florida, USA
• Nonprofit professional association
• 170,000 members worldwide
• 103 national institutes worldwide
• Key focus:
– Standards-setting body for internal
auditors
– Professional certifications
– Global research center
– Principal educator
– Global voice for the profession
[8] www.theiia.org/Training
Definition of Internal Auditing
[9] www.theiia.org/Training
Images of Internal Auditors
Which metaphor do you like?
• Magnifying glass
• Telescope
• Compass
• Hunting dogs
• Watch dogs
• Policemen
• Consultants
• Eyes and ears of the Audit Committee
[10] www.theiia.org/Training
Definition of Internal Auditing
Internal auditing is an independent, objective
assurance and consulting activity designed to add
value and improve an organization’s operations. It
helps an organization accomplish objectives by
bringing a systematic, disciplined approach to
evaluate and improve the effectiveness of risk
management, control, and governance processes.
Source: International Professional Practices Framework (IPPF)
The Institute of Internal Auditors
[11] www.theiia.org/Training
Internal Auditing Is
Assurance
Independent Add Value
Activity
designed
to
Consulting Improve
Objective Operations
Activity
[12] www.theiia.org/Training
Internal Auditing Helps
To The Effectiveness of To Help
Evaluate Risk Management
Process
Organization
Control Process accomplish it’s
Objectives
Improve
Governance
Process
[13] www.theiia.org/Training
Performance Audit
[14] www.theiia.org/Training
Definitions of PA
• INTOSAI: Performance auditing is an independent examination of
the efficiency and effectiveness of government undertakings,
programs, or organizations, with due regard to economy, and the
aim of leading to improvements.
• US Government Auditing Standards: Performance audits are
defined as engagements that provide assurance or conclusions
based on an evaluation of sufficient, appropriate evidence against
stated criteria, such as specific requirements, measures, or
defined business practices. Performance audits provide objective
analysis so that management and those charged with governance
and oversight can use the information to improve program
performance and operations, reduce costs, facilitate decision
making by parties with responsibility to oversee or initiate
corrective action, and contribute to public accountability.
[15] www.theiia.org/Training
Working Definition of PA
Performance Audit is an independent and
objective examination of a program, function,
operation or the management systems of a
governmental entity to:
– assure the entity’s objectives are carried out
in an economic, efficient and effective way,
and
– identify opportunity for improvement
[16] www.theiia.org/Training
Financial vs. Compliance vs. Performance Auditing
Financial Compliance Performance
Objective Attest to the Determine the Evaluate and improve the
fairness of financial adherence to policies, effectiveness, efficiency,
statements procedures, laws, and and economy of
regulations operations
Information Legislators Regulators Management
primarily for Stakeholders Audit Committee
Direction of Looking Back Looking back Looking at the present
Audit and to the future
Audits Financial reporting Specific laws and Mission, vision, and
based on standards such as regulations; objectives of the
IFRS Government standards organization and it’s
of business conduct; management
internal policies;
Examples Annual audits Contract audits; All other audits such as
performed by public business conduct those of departments,
accountants - may reviews; audits by processes, information
be supported by banking or other systems and other
functions
specific internal regulators
audits
[17] www.theiia.org/Training
What Makes this Performance Audit?
An Example:
“…to determine whether laws, contracts, policies
and procedures have been properly observed and
whether all business transactions were conducted
in accordance with established policies and with
success. In this connection, the auditors are to
make suggestions for the improvement of existing
facilities and procedures, criticisms of contracts
with suggestions for improvement, etc.”
[18] www.theiia.org/Training
Benefit of
Performance Audit
[19] www.theiia.org/Training
Benefit of PA – Adding Value
• Relevant
– Focus on the key initiatives
• Flexible
– Define the scope of the audit based on
risk
• Improving organizational performance
• Strengthen the governance
• Fraud prevention and detection
• Gaining public trust
[20] www.theiia.org/Training
Internal Audit Value
Assurance = Governance,
Risk Management,
Control
Insight = Catalyst,
Analyses,
Assessments
Objectivity = Integrity,
Accountability,
Independence
[21] www.theiia.org/Training
Exercise - Connect the Dots
o o o
o o o
o o o
Connect all nine dots using just 4 lines
without taking the pencil off the paper
[22] www.theiia.org/Training
Think Outside the Box
o o o
o o o
o o o
[23] www.theiia.org/Training
Unit 2
Management Functions and
Performance Measures
• Understanding the management functions
• Seeing the organization through the eyes of
management
• Understanding performance measures
[24] www.theiia.org/Training
Management Functions
[25] www.theiia.org/Training
Management
Issues and Concerns
• Cost Containment • Technological
• Human Resources Changes and
• Values and Vision Innovations
Initiatives • Communication
• Empowered • Customer
Environments vs. Satisfaction
Traditional Structures • Public Perception
[26] www.theiia.org/Training
Management’s Roles
Plan
Control Get the Job Done Organize
Direct
[27] www.theiia.org/Training
Management’s Roles
[28] www.theiia.org/Training
Performance Auditor’s Roles
• Evaluate the management processes and identify the
heart of the problem
• Alert to actual and potential changes
• Identify the opportunity for improvement
All units, programs, systems and activities are
subject to internal auditor’s evaluations
[29] www.theiia.org/Training
See though the Eyes of
Management
Almost every deviation or
deficiency results from the
violation of some principle of
management or good
administration.
See the organization and its activities
through the eyes of management
[30] www.theiia.org/Training
Three Simple Questions to
Ask Management
• What can go wrong?
• How do you it won’t go wrong?
• So what?
[31] www.theiia.org/Training
Performance Measures
[32] www.theiia.org/Training
Types of Management
Performance Measures
• INPUTS - Measures of service efforts, e.g., number of
hours, amount of materials.
• OUTPUTS - Measures of service level, e.g., number of
residences served, amount of service provided.
• OUTCOMES - Measures of service accomplishments,
e.g., measures related to program goals, including
effectiveness of quality.
• EFFICIENCY - Measures that relate service efforts to
service accomplishments, e.g., output/unit of input,
productivity indexes.
[33] www.theiia.org/Training
Principles
• Measure only what are important to the
organization
• Use of output-oriented measures
• Identify the total costs of service delivery
• Focus on continuous process improvement
• Performance measures should interconnect
throughout the organization
[34] www.theiia.org/Training
One Example –
Five Performance Categories:
• Effectiveness – the degree to which process output
conforms to requirements
• Efficiency – the degree to which the process produces
the output at a minimum cost of resources
• Quality – the degree to which the product or service
meets customer expectations
• Timeliness – the degree to which a unit of work was
done correctly and on time
• Safety – the measure of health and the working
environment of the organization
[35] www.theiia.org/Training
Unit 3
International Standards
For Performance Audit
International Professional Practices Framework
- IPPF from the IIA
www.theiia.org/Training
Why the Standards Matter
The Standards
Lead Represent
Advancement of the Profession
[37] www.theiia.org/Training
Road Map of Internal Audit
- Changes to the IIA Standards
Complex Services
Clients - the Organization
•1978 The Standards for the
Single Service Multiple Services Single Professional Practice of Internal
Single Client Client Auditing
•1947 Statement of •1957, 1971 and 1976 •1999 New Definition of Internal
Responsibilities of the Statement of Responsibilities Auditing
Internal Auditor of the Internal Auditor •1999 Professional Practice
Framework (PPF)
•2009 International Professional
practices Framework (IPPF)
[38] www.theiia.org/Training
The IIA’s IPPF
International
Professional
Practices
Framework
[39] www.theiia.org/Training
AUTHORITATIVE Guidance
Mandatory
Authoritative =
Strongly
recommended
[40] www.theiia.org/Training
Code of Ethics
• Integrity
– The integrity of internal auditors establishes trust and thus
provides the basis for reliance on their judgment.
• Objectivity
– Internal auditors exhibit the highest level of professional objectivity
in gathering, evaluating, and communicating information about the
activity or process being examined. Internal auditors make a
balanced assessment of all the relevant circumstances and are not
unduly influenced by their own interests or by others in forming
judgments.
• Confidentiality
– Internal auditors respect the value and ownership of information
they receive and do not disclose information without appropriate
authority unless there is a legal or professional obligation to do so.
• Competency
– Internal auditors apply the knowledge, skills, and experience
needed in the performance of internal auditing services.
[41] www.theiia.org/Training
International Standards for
Professional Practice of
Internal Auditing
[42] www.theiia.org/Training
Importance of the Standards
• They define the profession.
• They set the bar that every
auditor should comply with.
• They give you a reference guide
for how to conduct yourself.
• They lay the ground work, but are
not the ultimate goal.
• They give our customers peace of
mind and confidence they’re
getting a quality product.
[43] www.theiia.org/Training
The International Standards
• Mandatory requirements consisting of:
– Statements of basic requirements for
professional practice of internal
auditing
– Interpretations which clarify terms or
concepts within the Statements.
– Glossary
[44] www.theiia.org/Training
Overview of the IIA Standards
Attribute Standards:
Purpose, Authority and Responsibility……………………1000
Independence and Objectivity………………………………..1100
Proficiency and Due Professional Care……………….….1200
Quality Assurance and Improvement Program……..…1300
Performance Standards:
Managing the Internal Auditing Activity……………………2000
Nature of Work.……………………………………………….…………2100
Engagement Planning…………………………………….……..…2200
Performing the Engagement…………………………..……… 2300
Communicating Results………………………………..….………2400
Monitoring Progress………………………………………….……. 2500
Resolution of Management’s Acceptance of Risks……..2600
[45] www.theiia.org/Training
Important Knowledge for Satisfactory Performance
Of Internal Auditing
IIA CBOK 2006 - Figure 2-1
[46] www.theiia.org/Training
2010 IIA Global Internal Audit Study
Who Uses the Standards
• Mandatory requirements for 170,000 IIA members and 100,000 Certified
Internal Auditors
Translated into 21 languages
• Recognized or referenced by International Standards Setting Bodies,
such as:
INTOSAI (IIA Standards are recognized globally for public sector
audit professions)
Basel Committee on Banking Supervision
OECD Internal Audit Function
• Referenced on the mandated legislation or regulation in countries or
territories, such as
Belgium, Bosnia & Herzegovina, Canada, Chinese Taiwan, Estonia,
Poland, Romania, South Africa, Sweden, Thailand, Tunisia, Unites
States, United Kingdom, Zimbabwe, and …
[47] www.theiia.org/Training
IPPF Strongly
Recommended Guidance
• Practice Advisories (56)
Address approach, methodology and considerations, but NOT detailed
processes and procedures. Concise and timely guidance to assist internal
auditors in applying Code of Ethics and Standards and promoting good
practices.
• Position Papers (2)
IIA statement to assist a wide range of interested parties, including those
not in internal auditing profession, in understanding significant
governance, risk or control issues and delineating related roles and
responsibilities of internal auditing.
• Practice Guides (26)
Detailed guidance for conducting internal audit activities. Includes
detailed processes and procedures, such as tools and techniques,
programs, and step-by-step approaches, including examples of
deliverables.
www.theiia.org/guidance
[48] www.theiia.org/Training
Unit 4
Risk-Based Performance Audit
• Performance audit process
• The importance of clearly defined business objectives
and associated performance measures (goals) to a
performance audit
• Risk assessment using a Risk/Control Matrix
methodology
• Case Study
www.theiia.org/Training
Performance Audit Process
• Planning
• Examining and Evaluating Information
• Communicating Results
• Following Up
[50] www.theiia.org/Training
IIA Standards Related to
Performance Audit Process
[51] www.theiia.org/Training
Plan Performance Audit
• The most important part of an audit is the
planning phase.
• Standard 2010 – Planning: The chief audit
executive must establish risk-based plans to
determine the priorities of the internal audit
activity, consistent with the organization’s
goals.
[52] www.theiia.org/Training
Plan Performance Audit
• Standard 2201 – Planning Considerations: In
planning the engagement, internal auditors must
consider:
– The objectives of the activity being reviewed and the means by
which the activity controls its performance;
– The significant risks to the activity, its objectives, resources,
and operations and the means by which the potential impact of
risk is kept to an acceptable level;
– The adequacy and effectiveness of the activity’s risk
management and control processes compared to a relevant
control framework or model; and
– The opportunities for making significant improvements to the
activity’s risk management and control processes.
[53] www.theiia.org/Training
Risk-based Performance Audit
• Start with an organization’s objectives and associated
performance measures.
• Focus on an evaluation of performance risks and controls
related to those objectives.
• Help the organization achieve the desirable goals and
protect it from bad or undesirable things happening.
• Help reduce the chance of missed opportunities.
• Provide suggestions for improvement in controls designed
to mitigate the risks associated with meeting performance
objectives.
[54] www.theiia.org/Training
Risk Assessment Formula
Objective Risks Controls
[55] www.theiia.org/Training
Identification of Objectives
Objectives are the things an
organization wants to
accomplish.
Objectives should be S.M.A.R.T.
[56] www.theiia.org/Training
Objectives Cascade
Mission
Vision
Objective 1 Objective 2 Objective 3
Sub-Objective Sub-Objective Sub-Objective
Sub-Objective Sub-Objective Sub-Objective
Sub-Objective Sub-Objective Sub-Objective
[57] www.theiia.org/Training
What is Risk
• Risks are things that could prevent an
organization from meeting its objectives.
• IIA definition - Risk is the possibility of
an event occurring that will have an
impact on the achievement of objectives.
Risk is measured in terms of impact and
likelihood.
[58] www.theiia.org/Training
Business Risk Examples
1. Erroneous records and/or information
2. Business interruption (Government shutdown)
3. Public criticism or legal action
4. High costs
5. Loss or destruction of assets
6. Customer dissatisfaction due to ineffective
program/service design
7. Fraud or conflict of interest
8. Inappropriate mgmt. policy and/or decision making
process
[59] www.theiia.org/Training
Focusing on the “Real Risks”
Strategic & Business 60% Operational 20%
Financial 15% Compliance 5%
[60] www.theiia.org/Training
Risk Assessment
H
High
Risk Impact
Total Audit
Universe
Low
L Likelihood H
[61] www.theiia.org/Training
Risk Responses
Examples of risk response options:
• Acceptance
• Avoidance
• Transfer
• Mitigation
[62] www.theiia.org/Training
Risk Response Strategy
• Management identifies available risk response
options
• Considers their effect on event likelihood and
impact, in relation to risk appetite and cost
versus benefit
• Effective enterprise risk management does not
dictate which response management should
chose, but that the chosen response brings
the expected likelihood and impact within the
desired risk tolerances
[63] www.theiia.org/Training
Risk Assessment
- Two perspectives
• Inherent (Gross) - BEFORE RISK RESPONSE
• Residual (Net) - AFTER RISK REPONSE
Inherent Residual
Responses
Risk Risk
[64] www.theiia.org/Training
Exercise: Rain and Umbrella
When it rains, where are Inherent and
Residual Risk (IR and RR)?
[65] www.theiia.org/Training
When it rains, where are IR and RR?
IR IR
IR IR
IR
IR
IR
RR CR
RR RR
RR
RR
IR = All the raindrops
RR = The raindrops outside the umbrella
CR = Control Risk, possibility the umbrella leaks
Risk Appetite = How big the umbrella is
[66] www.theiia.org/Training
What is Control
• Controls are things that help meet an
organization's objectives.
• IIA Definition Control - any action taken by
management, the board, and other parties to
manage risk and increase the likelihood that
established objectives and goals will be
achieved. Management plans, organizes, and
directs the performance of sufficient actions
to provide reasonable assurance that
objectives and goals will be achieved.
[67] www.theiia.org/Training
Control to Mitigate These Risks
1. Erroneous records and/or information
2. Business interruption
3. Public criticism or legal action
4. High costs
5. Loss or destruction of assets
6. Customer dissatisfaction due to ineffective
program/service design
7. Fraud or conflict of interest
8. Inappropriate mgmt. policy and/or decision making
process
[68] www.theiia.org/Training
Risk Management and Control
• Two sides of the same coin:
– Risk is managed by having in place the right controls
to safeguard against its occurrence;
– Internal control exists only in relation to what they
do to mitigate risk.
• Risk management and internal control are
integrated parts of an entity’s overall
governance and management system.
[69] www.theiia.org/Training
Control - Who Is Responsible
• Management is responsible to design,
implement and monitor controls
• Internal auditors is responsible to
assess the adequacy and effectiveness
of controls
[70] www.theiia.org/Training
Risk Control Matrix
Objectives Risk Control
Name Likelihood Significance Ranking Name Evaluate Test
Adequacy Effectiveness
Use RCM to
• Plan an audit
• Document an audit
[71] www.theiia.org/Training
Benefits of Risk Control Matrix
• Open-ended
• Disciplined
• Risk-based
• Inclusive
Most organizations modify, delete, and
add columns on the Risk/Control Matrix
to fit their own environment.
[72] www.theiia.org/Training
Validate the Audit Plan
Special
Request Mandated
H
AUDIT RESOURCES
High
Risk Impact
Total Audit
Universe
*
Low
L Likelihood H
[73] www.theiia.org/Training
Case Study
State Department of
Fruit and Vegetable
[74] www.theiia.org/Training
Unit 5
Value for Money Approach
• Why Value-for-Money approach?
• Three E’s Performance Measures
• Difference between Risk-Based and Value-for-Money
approaches
• Twelve Attributes for Evaluating Effectiveness
• Case Study
www.theiia.org/Training
Needs for Performance Audit
To evaluate a unit or program and answer
questions like:
• Do we get value for money?
• Is it possible to spend the money better or
more wisely?
• Are the right things been done?
• If so, are things been done in the right way?
• If not, what are the causes?
[76] www.theiia.org/Training
Value-for-Money
• Definition: VFM is utility derived from every purchase
or every sum of money spent. VFM is based not only on
the minimum purchase price (economy) but also on
the maximum efficiency and effectiveness of the
purchase.
• Looks at how well an organization provides value for
money.
• Focuses on economy, efficiency, and effectiveness
• Based on the Twelve Attributes for Evaluating
Effectiveness
[77] www.theiia.org/Training
Audit Performance Measures
– 3E’s
• The principle of ECONOMY is keeping costs low. It requires that
the resources used by the audited entity for its activities shall be
made available in due time, in appropriate quantity and quality
and at the best price.
• The principle of EFFICIENCY is getting the most from available
resources. It is concerned with the best relationship between
resources employed, conditions given and results achieved.
• The principle of EFFECTIVENESS is meeting the objectives set. It
is concerned with attaining the specific aims or objectives set
and/or achieving the intended results.
[78] www.theiia.org/Training
12 Attributes For
Evaluating Effectiveness
1. Management Direction 7. Costs and Productivity
2. Relevance 8. Responsiveness
3. Appropriateness 9. Financial Results
4. Achievement of 10. Working Environment
Intended Results 11. Protection of Assets
5. Acceptance 12. Monitoring and
6. Secondary Impacts Reporting
[79] www.theiia.org/Training
Conducting Performance Audit
- Planning
• Gather background information on the audit area.
• Understand the organization’s business, objectives,
mission, etc.
• Interview management and staff.
• Use the twelve attributes to scope the audit by looking at
each attribute to choose which are most applicable.
• For the selected attributes, form questions to be
answered during the next phase.
[80] www.theiia.org/Training
Conducting Performance Audit
- Examining and Evaluating
• The questions are answered through:
- Interviews with management, employees and
others
- Industry research
- Performance measures (criteria)
- Benchmarking (criteria)
- Other management and audit reports.
- Site visits.
[81] www.theiia.org/Training
Conducting Performance Audit
- Reporting and Following Up
Communicating Results Phase
• Issues should be communicated to client throughout the
audit.
• The report is written and presented to the client.
Following Up
• Management implements action items from the report.
Audit assists as required.
[82] www.theiia.org/Training
Case Study
State Department of
Fruit and Vegetable
[83] www.theiia.org/Training
Unit 6
Final Thoughts
• Summary of What We Discussed
• Internal Audit - Today and Tomorrow
www.theiia.org/Training
Summary
• Understanding of internal audit and
performance audit
• Performance measures
• IIA’s International Professional Practices
Framework (IPPF)
• Management functions
• Risk-based performance audit
• Value-for-money performance audit
[85] www.theiia.org/Training
Modern Internal Auditing
• Client-focused, value-added service to management and
oversight bodies
• Guided by international standards and enhanced emphasis
on quality
• Adoption of risk-based methodologies
• Consulting service + assurance service
• More independence and enhanced stature
• Add value to the organization and stronger alignment
• More strategic approach to staffing: out-sourcing and co-
sourcing
• Integration of IT and non-IT audit resources
• Enhanced use of technology tools/services
• Started to be part of governance structure
[86] www.theiia.org/Training
Top 5 Internal Audit Activities
Today
• Operational auditing (89% of respondents).
• Audits of compliance with regulatory code (including
privacy) requirements (75% of respondents).
• Auditing of financial risks (72% of respondents).
• Investigations of fraud and irregularities (71% of
respondents).
• Evaluating the effectiveness of control frameworks (i.e.,
using COSO and COBIT) (69 percent of respondents).
2010 IIA Global Internal Audit Study
[87] www.theiia.org/Training
What Is Next?
Top Five Imperatives
• Assess and align with key stakeholder expectations
• “Step up to the plate” in risk management
• Enhance internal audit knowledge of the business
• Streamline internal audit processes and operations
• Coordinate and align with other risk, control and
compliance functions
[88] www.theiia.org/Training
Performance Audit
Adds Value By
• Reducing risk exposure
• Improving opportunities to achieve goals
• Identifying operational improvement
[89] www.theiia.org/Training
Questions
Guidance@theiia.org
www.theiia.org/guidance
[90] www.theiia.org/Training
90
