15

Questionnaire Design

Questionnaire Design

©2013 Cengage Learning. All Rights Reserved. May

not be scanned, copied or duplicated, or posted to a
publicly accessible website, in whole or in part.
 The Research Design

DETAILS OF STUDY MEASUREMENT

Measurement
Purpose of the Types of Extent of researcher Study setting DATA
and
study investigation interference ANALYSIS
measures
Exploration Establishing: Minimal: Studying 1. Feel
Description – Causal relationships events as they normally Contrived for data
Operational
Hypothesis testing – Correlations occur Noncontrived
definition
– Group differences, Manipulation and/or
Scaling
ranks, etc. control and/or
simulation

2. Goodness
of data

Data-
Unit of analysis Sampling Time
collection
(population to be Design horizon
method
studied) 3. Hypothesis
Individuals testing
One-shot Observation
Dyads Probability/
Interview
Groups nonprobability (crosssectional)
Questionnaire
Organizations Sample size (n) Longitudinal
Physical
Machines
measurement
etc.

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
LEARNING
OUTCOMES

1. Explain the significance of decisions about

questionnaire design and wording
2. Define alternatives for wording open-ended and
fixed-alternative questions
3. Summarize guidelines for questions that avoid
mistakes in questionnaire design
4. Describe how the proper sequence of questions
may improve a questionnaire
5. Discuss how to design a questionnaire layout
6. Describe criteria for pretesting and revising a
questionnaire and for adapting it to global
markets

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. 15-3
 J.D. Power Asks: It’s Interesting, But Do
You Really Want It?

• Car makers have to evaluate

what features consumers
want but will actually buy.
• J.D. Power surveyed
consumers on what features
they were familiar with and
if they were willing to pay
the price.
• Knowing price information
changed consumers’
interest levels.

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. 15-4
 Questionnaire Quality and Design:
Basic Considerations
• Questionnaire design is one of the most
critical stages in the survey research
process.
• A questionnaire (survey) is only as good as the
questions it asks—ask a bad question, get bad
results.
• Composing a good questionnaire appears easy,
but it is usually the result of long, painstaking
work.
• The questions must meet the basic criteria of
relevance and accuracy.

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. 15–5
 1.Principles of
wording Content and Type and
purpose Wording & Sequencing
form of
Language
observation of question questions
Data
collection
methods Questionnaire Testing Classfcn.data
Quest. or personal
administration
goodness of
Interview data informtn

categorisation
2.Principles of coding
measurement
Scales and scaling

Reliability & Validity

Appearance
Length

Introduction to respondents
3.General getup
Instructions for completion
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 Decisions in Questionnaire Design
1. What should be asked?
2. How should questions be phrased?
3. In what sequence should the questions be
arranged?
4. What questionnaire layout will best serve
the research objectives?
5. How should the questionnaire be
pretested? Does the questionnaire need
to be revised?

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. 15–7
 1.Principles of Wording
• Appropriateness of the content of the
questions
• How questions are worded and the level of
sophistication of the language used
• The type and form of questions asked
• The sequencing of the questions
• The personal data sought from the
respondents

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 1.Principles of Wording
• Content and Purpose of the questions
• Nature of the variable tapped
• Subjective feelings (job satisfaction, involvement,
measurement of perceptions, attitude etc.) or objective
facts (etc.)
• Variables should be adequately measured and no
superfluous questions should be asked
• Language and wording of the Questionnaire
• Should approximate the level of understanding of the
respondents
• Word questions in a way that could be understood by the
respondents
• Wording should be appropriate to tap the respondents
attitudes, perceptions and feelings

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 1.Principles of Wording
• Type and form of questions
• Open-ended vs. Closed questions
• Open-ended Q allows respondents to ans. in anyway they like.
Eg.state 5 things that are interesting and challenging in the job.
• Closed Q ask respondents to make choices among a set of
alternatives given by the researcher.In the above eg.the
researcher might list 10 to 15 aspects that might seem
interesting and challenging and ask the respondents to rank
them
• All items in a Q using ordinal, nominal, Likert or ratio scale are
closed.
• Closed questions help respondents to make quick decisions to
choose from among several alternatives

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 1.Principles of Wording

• Type and form of questions

• Positively and Negatively worded questions
• Instead of phrasing all questions positively,
include some negatively worded questions as
well
• A good Q should include both positively and
negatively worded questions.
• Eg. Coming to work is no great fun; Not coming
to work is greater fun than coming to work

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 1.Principles of Wording

• Type and form of questions

• Double-Barreled questions
• A question that lends itself to different possible responses to its
subparts is called a double barreled question.
• Eg.Do you think there is a good market for the product and that
it will sell well?
• In other words put two Q’s; Do you think the product will sell
well?; Do you think there is a good market for the product?
• Ambiguous responses; hence such questions should be
eliminated

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 1.Principles of Wording
• Type and form of questions
• Leading vs. Loaded questions
• Leading questions
• Questions should not be phrased in such a way that they lead
the respondents to give the responses that the researcher
would like or want them to give
• Eg “Don’t you think in these days of escalating costs of living,
employees should be given good pay rises?”
• “To what extent do you agree that employees should be given
higher pay rises?”
• Ie.dont frame the question in a suggestive manner

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 1.Principles of Wording
• Type and form of questions
• Loaded questions
• Questions should not be phrased in an emotionally charged
manner
• Eg “To what extent do you think management is likely to be
vindictive if the union decides to go on strike?”
• The words vindictive and strike are emotionally charged terms,
polarising management and unions; which would elicit highly
biased responses
• Depending on the purpose, the above question can be framed to
ask
◗ The extent to which employees are in favour of strike
◗ The extent to which they fear adverse reactions if they go on
strike
• Then get the responses as “strongly agree or disagree”

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 1.Principles of Wording
• Type and form of questions
• Social Desirability
• Questions should not be worded such that they elicit socially
desirable responses
• Eg.“Do you think that older people should be laid off?”
• Response in most cases would be ‘no’; hence reframe
• “There are advantages and disadvantages to retaining
senior citizens in the workforce. To what extent do you think
companies should continue to keep the elderly on their
payroll?”
• Sometimes certain items that tap social desirability are
deliberately introduced at various points in the
questionnaire to calculate an index of each individual’s social
desirability tendency. This is applied to adjust for social
desirability biases
• Length of Questions
• Simple short questions are the rule
• A question or a statement should not exceed 20 words or
one full line in print
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 1.Principles of Wording
• Sequencing of questions
• Questions should be ordered in such a way that the
respondent is led from questions of a general nature to those
that are more specific
• Questions that are relatively easy to answer to those that are
progressively difficult
• This is called Funnel approach
• It is advisable not to place continuously a positively worded
and a negatively worded question tapping the same element
or dimension of a concept. For eg.
◗ 1.I have opportunities to interact with my colleagues during
work hours
◗ 2.I have few opportunities to interact with my colleagues
during work hours
• To tap consistency of the responses the two questions should
be placed in different parts of the questionnaire.
• Sequencing biases frequently referred to as ordering effects
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 1.Principles of Wording
• Classification Data or Personal Information
• Include demographic questions such as age,
marital status, educational level, income etc.
• In most cases unless extremely needed it is best
not to reccord the name of the respondent; if
the respondent has to be identified, reccord it
separately
• For age and income level, it is best to give a
number of alternatives rather than an open-
ended question

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 2.Principles of Measurement
• Categorisation
• If categories of Likert scaling are used, then see
that they follow the same positive or negative
direction, instead of mixing up
• Coding
• Each questionnaire item can be coded to
facilitate easy entry into the computer
• Scales and scaling
• Reliability and validity
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 3.General Appearance of the Q
• A good introduction
• It conveys the identity of the researcher and
purpose of the survey;
• Assurance of confidentiality
• Organizing questions, giving instructions and
guidance and good alignment
• Open-ended questions at the end
• Instructions for completion
• Conclude the Q with a thankyou note
©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 Observational Surveys

• Nonparticipant-Observer
• Participant-Observer
• Structured observational studies
• Unstructured observational studies

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 Analyze the Questionnaire

• Q: Assume the American Society of Training Directors is

studying its membership in order to enhance member
benefits and attract new members. Below is a copy of the
cover letter and mail questionnaire received by a member
of the society. Evaluate the questions & format of the
instrument.
• The ASTD is evaluating the perception of value of
membership among its members. Enclosed is a short
questionnaire and a return envelope. I hope you will take a
few minutes and fill out the questionnaire as soon as
possible.
Sincerely
Director of Membership

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 Analyze the Questionnaire
Questionnaire
Directions: Please answer as briefly as possible
1.With what company did you enter the field of training?
---------------------------
2.How long have you been in the field of training?
-----------------------
3. How long have you been in the training department of the company
with which you are presently employed?
4. How long has the training department in your company been in
existence?
------------------------
5. Is the training department a subset of another department? If so what
department?
6. For what functions other than training is your department responsible?
-----------------

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 Analyze the Questionnaire
7. How many people, including yourself, are in the training
department of your company?
-----------------
8. What degrees do you hold and from what institutions?
------------------(Degree/PG)
9. Why were you chosen for training? What special qualifications
prompted your entry into training?
------------
10. What experience would you consider necessary for an individual
to enter into the field of training with your company? Include
both educational requirements and actual experience.
------------------

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
 Types of Fixed-Alternative Questions
• Simple-dichotomy (dichotomous) Question
• Requires the respondent to choose one of two alternatives (e.g.,
yes or no).
• Determinant-choice Question
• Requires the respondent to choose one response from among
multiple alternatives (e.g., A, B, or C).
• Frequency-determination Question
• Asks for an answer about general frequency of occurrence (e.g.,
often, occasionally, or never).
• Checklist Question
• Allows the respondent to provide multiple answers to a single
question by checking off items.

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. 15–24
 Phrasing Questions for Self-Administered,
Telephone, and Personal Interview Surveys

• Influences on Question Phrasing:

• The means of data collection—telephone
interview, personal interview, self-
administered questionnaire—will influence
the question format and question phrasing.
◗ Questions for mail, Internet, and telephone
surveys must be less complex than those used in
personal interviews.
◗ Questionnaires for telephone and personal
interviews should be written in a conversational
style.

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. 15–25
 Internet Questionnaire Layout (cont’d)
• Drop-down Box
• A space saving device that reveals responses when they are
needed but otherwise hides them from view.
• Check Boxes
• Small graphic boxes, next to an answers, that a respondent clicks
on to choose an answer; typically, a check mark or an X appears in
the box when the respondent clicks on it.
• Open-ended Boxes
• Boxes where respondents can type in their own answers to open-
ended questions.
• Pop-up Boxes
• Boxes that appear at selected points and contain information or
instructions for respondents.

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. 15–26
 EXHIBIT 15.7 Question in an Online Screening Survey for
Joining a Consumer Panel

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. 15–27
 EXHIBIT 15.8

Alternative
Ways of
Displaying
Internet
Questions

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. 15–28
 Pretesting and Revising Questionnaires

• Pretesting Process
• Seeks to determine whether respondents have
any difficulty understanding the questionnaire
and whether there are any ambiguous or
biased questions.
• Preliminary Tabulation
• A tabulation of the results of a pretest to help
determine whether the questionnaire will
meet the objectives of the research.

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. 15–29
 • Design 10 questions that assess how
effective a post graduate business course
has been.

©2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. 15–30