CUSTOMER

SATISFACTION DATA
Collecting, Analyzing and
Reporting
With information from: Surveying Clients About Outcomes – The Urban Institute, 2003
CSBG ORGANIZATIONAL
STANDARD 1.3 :
The organization has a systematic
approach for collecting, analyzing,
and reporting customer satisfaction
data to the governing board
 1.3 INTERPRETATION:
SYSTEMATIC APPROACH
Collection of data across programs – how many programs collect
customer satisfaction data? At what points are data collected
(intake, after services are delivered, and after outcomes are
achieved)?
Methods of collection – surveys, focus groups,
Structure that supports data analysis – quality improvement
committee or staff person
Tools for data collection and analysis – scorecards, dashboards
Process for reporting – clear procedure for reporting and analysis
from committee to leadership team to board
IT CAN BE RELATIVELY EASY TO
MEET THE MINIMUM REQUIRED
BY THE STANDARD:
Conduct a survey once every three years that simply asks your clients
to rate their ‘satisfaction’ with the agency on a scale of 1-5.

Report to the board the average score that the agency received, and
note whether this is a better score than last time or not.

Include the agency ‘score’ in your community assessment.

BUT WHAT
DOES ‘THE
MINIMUM’ GET
YOU?
.
 LET’S START
AT THE
VERY
BEGINNING
This Photo by Unknown Author is licensed under CC BY-NC-ND
 WHY IS CUSTOMER
SATISFACTION IMPORTANT?
Directly affects the ability of customers to access and use services
Critically important to the continuous improvement of services
(ROMA cycle)
Data is key component of the needs assessment & strategic
planning processes
Customer focus empowers the individuals and families served by
the agency
Required by the Organizational Standards (1.3, 6.4)
 WHAT IS CUSTOMER
SATISFACTION?
Highly personal assessment strongly affected by customer expectations

Influenced by (1) contact with the service provider, and (2) personal outcomes

Research shows is based on confirmation or disconfirmation of customer

expectations

Related to ability of customers to hold organization accountable

Three customer needs: security, esteem, and justice

 THREE ELEMENTS OF
CUSTOMER SATISFACTION

Experience of
Satisfaction
customer- Satisfaction
with service
worker with outcome
provided
interaction
IF YOU ARE GOING TO GATHER
CUSTOMER SATISFACTION DATA,
WHY NOT GET THE MOST VALUE
OUT OF YOUR EFFORTS?
Who are your customers?
What do your customers value?
What do you want and need to know?
SERVICE QUALITY FACTORS
Timeliness and convenience

Personal attention

Reliability and dependability

Employee competence and professionalism

SERVICE QUALITY FACTORS
Responsiveness

Tangibles such as physical facilities, equipment, and

appearance of personnel

Availability
 CUSTOMER SERVICE –
FIVE GENERAL OPERATING PRINCIPLES
(1) Successful customer service organizations listen to, understand, and
respond - often in unique and creative ways – to the evolving needs
and constantly shifting expectations of their customers

(2) These organizations establish a clear vision of what superior

service is, communicate that vision to employees at every level, and
ensure that service quality is personally and positively important to
everyone in the organization
 CUSTOMER SERVICE –
FIVE GENERAL OPERATING PRINCIPLES

3) They establish concrete standards of service quality and regularly

measure themselves against those standards.

(4) They carefully hire people, train them extensively so they have the
knowledge and skills to achieve the service standards, and then
empower them to work on behalf of customers, whether inside or
outside the organization
 CUSTOMER SERVICE –
FIVE GENERAL OPERATING PRINCIPLES
(5) They recognize and reward service
accomplishments, sometimes individually, sometimes
as a group effort, in particular celebrating the
successes of employees who go one step beyond the
expected actions for their customers
WHAT MAKES A CUSTOMER-
FOCUSED ORGANIZATION?
1. Customer-focused vision and mission statements

2. Customer-oriented organizational culture

3. Attention to the total customer experience

4. Customer service standards and accountability

 CUSTOMER FOCUSED VISION
AND MISSION STATEMENTS
Focuses and guides employee actions (core values)

Sets and manages customer expectations (performance

management system)

Contributes to instilling a culture of customer orientation

(provides all employees the necessary information to
effectively design, manage, and improve their processes)
 CUSTOMER ORIENTED
ORGANIZATIONAL CULTURE
Often entails “a massive cultural shift away from what is
convenient for the organization to what is needed by the
service users”

Requires multiple methods and strategies

Needs assessment and strategic planning should highlight

customer feedback
ATTENTION TO THE TOTAL
CUSTOMER EXPERIENCE
Conduct a “customer walkthrough” exercise

Attention to detail--Personal notes in database

Physical factors--Emphasize comfort and convenience

Organize procedures around customers (convenient,

streamlined, timely, and prepared)
CUSTOMER SERVICE STANDARDS AND
ACCOUNTABILITY
Behavioral change requires standards,
not suggestions

Develop customer service standards

FOCUS ON FRONTLINE STAFF
Golden Rule: Treat employees well
Emphasize customer satisfaction in the hiring process
Incorporate customer satisfaction in the orientation process and
provide training (instill the organizational culture)
Remove barriers to customer satisfaction
Involve employees in planning and improvement
Recognize and reward employee performance
WHO ARE OUR CUSTOMERS?
Family
Agency
Community
WHO TO SURVEY?
Customers – defined as:???
Current Customers
Former Customers?
Community Partners?
The Board?
Staff?
Volunteers?
Others?
HOW TO SURVEY??
In person interviews
Phone interviews

Self-administered by mail

Hard copy handout of questions

On-line survey (survey monkey?)

Focus group
 WHAT DOES YOUR
CUSTOMER VALUE?
Many organizations are very clear about the
value they would like to deliver, but they
often do not understand the value of what
they deliver from the customer’s point of
view.
Phillip Kotler, Northwestern University
CUSTOMER SATISFACTION DATA CAN
TELL YOU…
Are your programs/services making a difference?
What other services are needed?
What do you need to change/improve to be more effective in
pursuit of your mission?
What impact has a service or services had on customers’ lives?
How are customers treated by your staff?
What ideas do customers have for improving service quality?
What is a Systematic
Approach?
In Writing

Making
Methods
Changes
of
Based on
collection
the Data

Tools for
Reporting
data
data
collection

Analysis
of data
JUST WHAT IS ‘A SYSTEMATIC APPROACH’ FOR
COLLECTING, ANALYZING, AND REPORTING?
Needs to be in writing

Something you can repeatedly (or on an ongoing basis) conduct without

breaking the bank

Gathers information that can be compared from one time frame to

another

Provides information (data) that you can share with the board and use
to make decisions and take action to make positive changes
COLLECTIN
G DATA
METHODS FOR MEASURING
CUSTOMER SATISFACTION
Comment cards (in-person, online)

1-800 numbers

Surveys (in-person, online, mail)

Complaint monitoring and resolution

Focus groups
METHODS FOR MEASURING
CUSTOMER SATISFACTION
1-1 interviews

Community meetings

Quality improvement committees

Testers and “Secret Customers”

THE QUESTIONS
?????
 DEVELOPING YOUR
QUESTIONS
Make sure you are asking only one question at a time

Written at a literacy level consistent with your customers’

Eliminate negatively worded questions

Don’t ask leading or biased questions

Be time-specific and recent

 DEVELOPING YOUR
QUESTIONS
Use complete sentences when asking questions
Make sure questions are consistent from one time period to another so
that results can be compared

Phrase questions so that respondents have accurate, ready made answers

Include a ‘don’t know’ or ‘N/A’ option in your questions

 CUSTOMER SATISFACTION
AREAS/TOPICS TO EXPLORE
Welcomed when arrived
Clean
Timely assistance provided
Respectful treatment
Needs met
 CUSTOMER SATISFACTION
AREAS/TOPICS TO EXPLORE
Information on other agency services

Recommend the agency to others

Willing to provide additional feedback

Comments – how can we improve

Service area
GUIDELINES
FOR QUESTION
RESPONSE
CATEGORIES

This Photo by Unknown Author is licensed under CC BY-SA

GUIDELINES FOR QUESTION
RESPONSE CATEGORIES
Ask questions that provide actionable feedback

Use a scale over Yes/No options

Ask limited open-ended questions “What is the one thing we

can do to improve our service…”
GUIDELINES FOR QUESTION
RESPONSE CATEGORIES
Link methods to provide in-depth feedback (focus
group follow-up based on survey responses)

Take the surveys to the customers – ease,

convenience, and incentives are key to response rate
 GUIDELINES FOR QUESTION
RESPONSE CATEGORIES
Don’t overlap ranges
How many children do you have? 1-2, 2-4, 4-6, 6 or more

Make sure to cover all options

Use broad ranges for things like income to make it easier for
respondents
Under $10,000 $10,001 - $15,000 $15,001 - $20,000 $20,001 - $25,000
 GUIDELINES FOR ORDER
OF QUESTIONS
Put most interesting/important to customer questions first
Consider questioning sequencing logic – do questions flow from one to another?
Go from specific to general
Provide options for suggestions/recommendations
Put easy to answer questions last
GUIDELINES FOR QUESTIONNAIRE
CONSTRUCTION
If sent by mail, include a letter from ED and/or board chair

Have an introduction that explains the survey

Make the letter and survey appealing, simple, easy to follow

Include an ‘any other comments you’d like to make?’

Conclude with a friendly thank you !

GUIDELINES FOR SURVEY
IMPLEMENTATION
When do customers receive the initial survey?

What is your target response rate?

What training do you provide front line staff?

What form is the survey in?

--Comment card
--Formal paper survey
--E-survey (e.g. on an iPad)
CONDUCT A PRE-TEST(OR TWO!)
Have several people fill out the survey and provide you with feedback
– whether on-line or hard copy

If in-person or phone interview – interview several people to see if all

questions are clear, etc.

Make any changes and then perform a pre-test (or two!) again
 IN–APP SURVEYS
It’s one of the methods with the highest response rates, thanks to the
fact that the customer is asked for her opinion while she’s engaged
with your agency.

In-app surveys are especially handy to measure some of the standard

customer satisfaction metrics
 E-MAIL SURVEYS
In-App survey methods aren't suitable for in-depth insights about your
customer happiness. Why are they happy or unhappy?

Email surveys, on the other hand, are a good tool for this.
 HOW TO IMPROVE
RESPONSE RATES?
Sending advanced post card or letter – include logo, etc.
Include a letter from the Exec. Dir. with the survey
Send a reminder phone call or card
Include SASE if mailing
Make questionnaire easy to complete
Provide incentives
Use multiple modes
Ask staff to encourage folks to participate
This Photo by Unknown Author is licensed under CC BY-SA
 ENSURE CLIENT
CONFIDENTIALITY
Inform customers that their responses are voluntary and that
they can refuse to participate without any impact on services,
etc.

Make sure that you can make responses anonymous and

confidential and inform respondents that you are doing so

If surveying youth, make sure to get parent/guardian

permission
ANALYZING DATA
COLLECTING AND USING
THE DATA
Using Survey Monkey allows you to easily analyze survey results

Hard copy surveys can be manually entered into survey monkey, as

can written results of in-person or phone interviews

For open-ended questions, you need to code the data and/or use
the survey monkey ‘word cloud’ feature
 WAYS TO ANALYZE THE DATA
Comparison across client sub-groups – demographics, program
participation, number of services received, case manager,
location, etc.
Comparison across targets – where are we meeting (or failing
to meet) established goals.
Comparison with previous surveys – trend analysis.
WAYS TO ANALYZE THE DATA
What were the trends the data identified?

Were there differences in responses from different

segments of your customer population?

Did the customers offer any suggestions for

improvement?
Let’s Get Customer Service Geeky…
 CUSTOMER SATISFACTION
SCORE
This is the most standard customer satisfaction metric, asking your
customer to rate satisfaction with your agency or service.

Your CSAT score is then the average rating of your customer responses.

The scale typically ranges between 1 – 3, 1 – 5, or 1 – 10. A larger range

is not always better, due to cultural differences in how people rate their
satisfaction.
CUSTOMER SATISFACTION SCALE
 CUSTOMER SATISFACTION
The benefit of the CSAT metric comes from its
directness.

The downside, however, is that satisfaction is hard

to estimate, even for the customer.
 NET PROMOTER SCORE
(NPS)
The Net Promoter Score (NPS) measures
the likeliness of a customer referring you
to someone.
NET PROMOTER
SCORE
 NET PROMOTER
SCORE
The strength from this metric is that its question isn’t about an
emotion (“How satisfied am I?”), but about an intention (“How
likely am I to refer?”), which is easier to answer.

It cuts down to the question of whether the service is good

enough to refer it and put your own reputation on the line.
 CALCULATING YOUR NET
PROMOTER SCORE

Calculating your NPS score is quite easy. Take the percentage of

respondents who fall within the ‘promoter’ category (10 - 9)
and subtract the percentage of ‘detractors’ (0 - 6).
 CUSTOMER EFFORT SCORE
(CES)
With this method, customers rate the effort it took them to have their issue
solved — generally on a scale from 1 (very low effort) to 7 (very high effort).

According to CEB Global*, 96% of customers with a high effort score showed
reduced loyalty in the future, while that was the case with only 9% of those
who reported low effort scores.
CUSTOMER EFFORT SCORE (CES)
CUSTOMER EFFORT SCORE (CES)
Customers are much more likely to punish bad service than to reward
good service.

The costs of exceeded customer expectations are high, while the

payoffs are minimal.

Effort should be invested in making the customer experience and

problem resolution as easy as possible.
CUSTOMER EFFORT SCORE
Don’t ask, “How satisfied are you with this service?”; ask, “How easy
was it to get in contact/get a service/have your issue resolved?”

Relevance is crucial here. The time to pop the question is right after
your customer had the experience.
 THINGS GONE WRONG
(TGW)

This metric measures the number of complaints, or "Things

Gone Wrong.”

The standard approach to measure TGW is through

complaint sections in customer surveys, but you could also
maintain internal metrics.
USING THE
DATA
 USING THE DATA
Once you start measuring, you can start
optimizing or making use of the data.

And, optimizing your customer satisfaction

is one of the best investments you can
make.
 USING THE DATA
Identify where improvement has occurred and where it is needed
Identify trends – what is working? What are successful practices?
What does this data tell us about fulfilling our mission?
Encouraging innovation and learning – you can get feedback on pilot
programs or new approaches to service delivery
Informing board members and stakeholders about the quality and
effectiveness of your work
Can be used to help attract volunteers and clients
 WHAT DO YOU DO WITH THE
DATA AND YOUR ANALYSIS?
Compile a report that includes :
A summary of the highlights of your analysis; describe the limitations of
the data; provide a respondent profile (demographic and or services);
findings from your analysis
Report should be reader-friendly – charts and graphics whenever possible
Include information on the nuts and bolts – response rates; description of
analysis process
Include survey itself in the appendix
 USING THE DATA: NEXT
STEPS
Discuss results with supervisors involved in service delivery
Discuss shortcomings constructively
Top management taking this feedback seriously is extremely
important to reinforce the agency’s focus on client results
 USING THE DATA: NEXT STEPS
Meet with staff – focus on worsening outcomes and on very
good outcomes – what can we learn from this feedback?
What can we do to improve worsening or poor outcomes?
Do we need to change our expectations (targets)?
Meet with customers to discuss what we can do to improve
Plan for follow up on agreed-upon strategies for
improvement.
Subsequent surveys provide data on effectiveness of
strategies.
Using the Data
Needs
Assessment
/Strategic
Plan

Data Data Data

Analysis Reporting Program
Changes

Board/
Annual
Report
DOCUMENTING
 1.3 DOCUMENTATION
(1) Systematic Approach
Policy and procedure document describing operations of the
customer satisfaction process

(2) Collecting
Data collection instruments (surveys, focus group questions)
Summary data (reports, data collection schedules)
 1.3 DOCUMENTATION
(3) Analyzing
Policies and procedures documents describing how data is
analyzed
Reports, scorecards, dashboards
Written reports and recommendations

(4) Reporting…to the governing board

Policies and procedures documents describing reporting
procedures
Written reports, scorecards, etc. submitted to the board
Board minutes indicating reporting of data
 MODEL CUSTOMER
SATISFACTION PROCESS
Collect customer satisfaction data through multiple methods
and sources
-
Regular assessment of data by quality improvement
committee

Targeted follow-up interviews and focus groups

Quarterly leadership team reports to the board

 RESOURCES
Surveying Clients About Outcomes: Series on
Outcome Management for Nonprofit
Organizations. The Urban Institute – 2003
Technical Assistance Guide: Category 1:
Consumer Input and Involvement – Community
Action Partnership
Customer Satisfaction – Jarle Crocker and
Barbara Mooney – April 28, 2016 Region III
Webinar