CHAP 9

ADMINISTERING QUESTIONNAIRES

INTRODUCTION
QUESTIONNAIRE:

“A questionnaire is a preformulated written set of questions to which respondents record their answers,
usually within rather closely defined alternatives.”

▪ Questionnaires are generally designed to collect large numbers of quantitative data.

▪ Questionnaires are generally less expensive and time consuming than interviews and
observation, but they also introduce a much larger chance of nonresponse and nonresponse
error.

TYPES OF QUESTIONNAIRES

Three types of questionnaires:

✓ Administered personally questionnaires

✓ Distributed electronically and online questionnaires
✓ Mailed to the respondents questionnaires

PERSONALLY ADMINISTERED QUESTIONNAIRES:

When the survey is confined to a local area a good way to collect data is to personally administer the
questionnaires.

Advantages:

▪ The researcher or a member of the research team can collect all the completed responses within
a short period of time.
▪ Any doubts that the respondents might have on any question can be clarified on the spot.
▪ The researcher also has the opportunity to introduce the research topic and motivate the
respondents to offer their frank answers.
 ▪ Administering questionnaires to large numbers of individuals at the same time is less expensive
and it does not require as much skill to administer a questionnaire as it does to conduct
interviews.

Disadvantages:

▪ The researcher may introduce a bias by explaining questions differently to different people;
participants may be in fact answering different questions as compared to those to whom the
questionnaire was mailed.
▪ Personally administered questionnaires take time and a lot of effort.

MAIL QUESTIONNAIRES:

A mail questionnaire is a self‐administered (paper and pencil) questionnaire that is sent to respondents
via the mail.

This method has long been the backbone of business research, but with the arrival of the Internet,
mobile phones, and social networks, mail questionnaires have become redundant or even obsolete.
Instead, online questionnaires are posted on the Internet or sent via email.

ELECTRONIC AND ONLINE QUESTIONNAIRES:

The distribution of electronic or online questionnaires is easy and fast. All you have to do is to email the
invitations to complete a survey, post a link on a website or personal blog, or use social networks.

Advantages:

▪ Electronic and online questionnaires makes the most of the ability of the Internet to provide
access to groups and individuals who would be difficult, if not impossible, to reach through other
channels. Virtual communities flourish online, and hundreds of thousands of people regularly
participate in discussions about almost every conceivable issue and interest.
▪ A wide geographical area can be covered in the survey.
▪ The automatic processing of the survey saves further costs, time, and energy.

Disadvantages:

▪ When conducting online research, researchers often encounter problems with regard to
sampling.
 o For example self‐selection and extremely low response rates make it difficult to establish
the representativeness of the sample and to generalize the findings, because those
responding to the survey may not at all represent the population they are supposed to.
Indeed, the return rates of such questionnaires are typically low. A 30% response rate is
considered acceptable, and in many cases even exceptional.
▪ Posting invitations to participate in a survey on social networks, discussion groups, and chat
rooms is often perceived as rude or offensive. This is another drawback of online questionnaires.
Many people consider this type of posting to be “spam”, and the researcher may be flooded with
emails from angry members of a virtual community.
▪ Any doubts the respondents might have cannot be clarified.

Advantages and disadvantages of types of questionnaires in summarized form:

 GUIDELINES FOR QUESTIONNAIRE DESIGN

Sound questionnaire design principles should focus on three areas.

▪ The first relates to the wording of the questions.

▪ The second refers to the planning of issues with regard to how the variables will be categorized,
scaled, and coded after receipt of the responses.
▪ The third pertains to the general appearance of the questionnaire.

All three are important issues in questionnaire design because they can minimize bias in research.

➢ PRINCIPLES OF WORDING:

The principles of wording refer to such factors as:

1. The appropriateness of the content of the questions.

2. How questions are worded and the level of sophistication of the language used.
3. The type and form of questions asked.

4. The sequencing of the questions.

5. The personal data sought from the respondents.

1. Content and purpose of the questions:

The nature of the variable tapped – subjective feelings or objective facts – will determine what
kinds of questions are asked. If the variables tapped are of a subjective nature (e.g., satisfaction,
involvement), where respondents’ beliefs, perceptions, and attitudes are to be measured, the
questions should tap the dimensions and elements of the concept.
Where objective variables, such as age and educational levels of respondents, are tapped, a
single direct question – preferably one that has an ordinal scaled set of categories – is
appropriate.
Thus, the purpose of each question should be carefully considered so that the variables are
adequately measured and yet no superfluous questions are asked.

2. Language and wording of the questionnaire:

The language of the questionnaire should approximate the level of understanding of the
respondents. The choice of words will depend on their educational level, the usage of terms and
idioms in the culture, and the frames of reference of the respondents.
For example, even when English is the spoken or official language in two cultures, certain words
may be alien to one culture. Terms such as “working here is a drag” and “she is a compulsive
worker” may not be interpreted the same way in different cultures. Some blue‐collar workers
may not understand terminology such as “organizational structure.”
Thus, it is essential to word the questions in a way that can be understood by the respondent. If
some questions are either not understood or are interpreted differently by the respondent, the
researcher will obtain the wrong answers to the questions, and responses will thus be biased.
Hence, the questions asked, the language used, and the wording should be appropriate to tap
respondents’ attitudes, perceptions, and feelings.

3. Type and form of questions:

The type of question refers to whether the question is open‐ended or closed. The form of the
question refers to whether it is positively or negatively worded.

❖ Open-ended versus closed questions:

 o Open-ended questions :
Allow respondents to answer them in any way they choose.
An example of an open‐ended question is asking the respondent to state five things that
are interesting and challenging in the job.
Another example is asking what the respondents like about their supervisors or their
work environment. A third example is to invite their comments on the investment
portfolio of the firm.

o Closed question:
A closed question, in contrast, asks the respondents to make choices among a set of
alternatives given by the researcher.
For example, instead of asking the respondent to state any five aspects of the job that
she finds interesting and challenging, the researcher might list 10 or 15 aspects that
might seem interesting or challenging in jobs and ask the respondents to rank the first
five among these in the order of their preference.
All items in a questionnaire using a nominal, ordinal, Likert, or ratio scale are considered
closed.
Closed questions help the respondents to make quick decisions to choose among the
several alternatives before them.
They also help the researcher to code the information easily for subsequent analysis.

❖ Positively and negatively worded questions:

Instead of phrasing all questions positively, it is advisable to include some negatively

worded questions as well, so the tendency in respondents to mechanically circle the
points toward one end of the scale is minimized.
For example, let us say that a set of six questions is used to tap the variable “perceived
success” on a five‐point scale, with 1 being “very low” and 5 being “very high” on the
scale. A respondent who is not particularly interested in completing the questionnaire is
more likely to stay involved and remain alert while answering the questions when
positively and negatively worded questions are interspersed in it. If the respondent has
circled 5 for a positively worded question such as, “I feel I have been able to accomplish
a number of different things in my job,” he cannot circle number 5 again to the
negatively worded question, “I do not feel I am very effective in my job.” The respondent
is now shaken out of any likely tendency to mechanically respond to one end of the
scale. In case this does still happen, the researcher has an opportunity to detect such
bias.
A good questionnaire should therefore include both positively and negatively worded
questions.
 The use of double negatives and excessive use of the words “not” and “only” should be
avoided in negatively worded questions because they tend to confuse respondents.
For example it is better to say “Coming to work is not great fun” than to say
“Not coming to work is greater fun than coming to work.” Likewise, it is better to say
“The rich need no help” than to say “Only the rich do not need help.”

❖ Double-barreled questions:

A question that lends itself to different possible responses to its subparts is called a
double‐barreled question. Such questions should be avoided and two or more separate
questions asked instead.
For example, the question “Do you think there is a good market for the product and that
it will sell well?” could bring a “yes” response to the first part (i.e., there is a good
market for the product) and a “no” response to the latter part (i.e., it will not sell well for
various other reasons). In this case, it would be better to ask two questions: (1) “Do you
think there is a good market for the product?” and (2) “Do you think the product will sell
well?” The answers might be “yes” to both, “no” to both, “yes” to the first and “no” to
the second, or “yes” to the second and “no” to the first.
If we combined the two questions and asked a double‐barreled question, we would
confuse the respondents and obtain ambiguous responses.

❖ Ambiguous questions:

Even questions that are not double‐barreled might be ambiguously worded and the
respondent may not be sure what exactly they mean.
An example of such a question is “To what extent would you say you are happy?”
Respondents might find it difficult to decide whether the question refers to their state of
feelings in the workplace, or at home, or in general.
Thus, responses to ambiguous questions have built‐in bias inasmuch as different
respondents might interpret such items in the questionnaire differently.
The result is a mixed bag of ambiguous responses that do not accurately provide the
correct answer to the question.

❖ Recall-dependent questions:

Some questions might require respondents to recall experiences from the past that are
hazy in their memory. Answers to such questions might have bias.
 For example, if an employee who has had 30 years’ service in the organization is asked
to state when he first started working in a particular department and for how long, he
may not be able to give the correct answers and may be way off in his responses.

❖ Leading questions:

Questions should not be phrased in such a way that they lead the respondents to give
the responses that the researcher would like them to give.
An example of such a question is: “Don’t you think that in these days of escalating costs
of living, employees should be given good pay rises?” By asking a leading question, we
are signaling and pressuring respondents to say “yes.” Tagging the question to rising
living costs makes it difficult for most respondents to say, “No; not unless their
productivity increases too!”.

❖ Loaded questions:

Another type of bias in questions occurs when they are phrased in an emotionally
charged manner.
An example of such a loaded question is asking employees: “To what extent do you
think management is likely to be vindictive if the union decides to go on strike?” The
words “strike” and “vindictive” are emotionally charged terms, polarizing management
and unions.

❖ Social desirability:

Questions should not be worded such that they elicit socially desirable responses.
For example, a question such as “Do you think that older people should be laid off?”
would elicit a response of “no,” mainly because society would frown on a person who
said that elderly people should be fired even if they are capable of performing their jobs
satisfactorily.

❖ Length of questions:

Finally, simple, short questions are preferable to long ones. As a rule of thumb, a
question or a statement in the questionnaire should not exceed 20 words, or exceed one
full line in print.

4. Sequencing of questions:

The sequence of questions in the questionnaire should be such that the respondent is led from
questions of a general nature to those that are more specific, and from questions that are
relatively easy to answer to those that are progressively more difficult. This funnel approach, as
 it is called (Festinger & Katz, 1966), facilitates the easy and smooth progress of the respondent
through the items in the questionnaire. The progression from general to specific questions
might mean that the respondent is first asked questions of a global nature that pertain to the
issue, and then is asked more incisive questions regarding the specific topic.
For example, placing two questions such as the following, one immediately after the other, is not
only awkward but might also seem insulting to the respondent.
I have opportunities to interact with my colleagues during work hours.
I have few opportunities to interact with my colleagues during work hours.
First, there is no need to ask the very same question in both a positive and a negative way.
Second, if for some reason this is deemed necessary (e.g., to check the consistency of the
responses), the two questions should be placed in different parts of the questionnaire, as far
apart as possible. The way questions are sequenced can also introduce certain biases, frequently
referred to as ordering effects.

5. Classification data or personal information:

Classification data, also known as personal information or demographic questions, elicit such
information as age, educational level, marital status, and income. Unless absolutely necessary, it
is best not to ask for the name of the respondent. If, however, the questionnaire has to be
identified with the respondents for any reason, then the questionnaire can be numbered and
connected by the researcher to the respondent’s name, in a separately maintained, private
document. This procedure should be clearly explained to the respondent. The reason for using
the numerical system in questionnaires is to ensure the anonymity of the respondent.

➢ PRINCIPLES OF MEASUREMENT:

There some principles of measurement to be followed to ensure that the data collected are appropriate
to test our hypotheses. These refer to the scales and scaling techniques used in measuring concepts, as
well as the assessment of reliability and validity of the measures used.

➢ GENERAL APPEARANCE OR “GETUP” OF THE QUESTIONNAIRE:

Not only is it important to address issues of wording and measurement in questionnaire design,
but it is also necessary to pay attention to how the questionnaire looks.
An attractive and neat questionnaire with appropriate introduction, instructions, and well
arrayed set of questions and response alternatives will make it easier for the respondents to
answer them. A good introduction, well‐organized instructions, and neat alignment of the
questions are all important.

❖ A good introduction:
A proper introduction that clearly discloses the identity of the researcher and conveys
the purpose of the survey is absolutely necessary. It is also essential to establish some
rapport with the respondents and motivate them to respond to the questions in the
questionnaire wholeheartedly and enthusiastically.
The introduction section should end on a courteous note, thanking the respondent for
taking the time to respond to the survey.

❖ Organizing questions, giving instructions and guidance, and good alignment:

Organizing the questions logically and neatly in appropriate sections and providing
instructions on how to complete the items in each section will help the respondents to
answer them without difficulty. Questions should also be neatly aligned in a way that
allows the respondent to complete the task of reading and answering the questionnaire
by expending the least time and effort and without straining the eyes.

❖ Personal data:

Demographic or personal data could be organized as in the example that follows. Note
the ordinal scaling of the age variable.

❖ Information on income and other sensitive personal data:

 Although demographic information can be sought either at the beginning or at the end
of the questionnaire, information of a very private and personal nature such as income,
state of health, and so on, if considered at all necessary for the survey, should be asked
at the end of the questionnaire, rather than the beginning. Also, such questions should
be justified by explaining how this information might contribute to knowledge and
problem solving, so that respondents do not perceive them to be of an intrusive or
prying nature

❖ Open-ended question at the end:

The questionnaire could include an open‐ended question at the end, allowing

respondents to comment on any aspect they choose. It should end with an expression of
sincere thanks to respondents.

❖ Concluding the questionnaire:

The questionnaire should end on a courteous note, reminding the respondent to check
that all the items have been completed.

PRETESTING OF STRUCTURED QUESTIONS:

Pretesting involves the use of a small number of respondents to test the appropriateness of the
questions and their comprehension. This helps to rectify any inadequacies before administering the
instrument orally or through a questionnaire to respondents, and thus reduces bias. It would be good to
debrief the results of the pretest and obtain additional information from the small group of participants
(who serve the role of a focus group) on their general reactions to the questionnaire and how they felt
about completing the instrument.

ELECTRONIC QUESTIONNAIRE AND SURVEY DESIGN:

Online surveys are easily designed and administered. Electronic survey design systems which facilitate
the preparation and administration of questionnaires, are particularly useful for online research.

Even as the survey is in progress, descriptive summaries of the cumulative data can be obtained either
on the screen or in printed form. After data collection is complete, a data‐editing program identifies
missing or out‐of‐range data (e.g., a 6 in response to a question on a five‐point scale). The researcher can
set the parameters to either delete missing responses if there are too many of them, or compute the
mean of other responses and substitute this figure for the missing response. Such systems also include
data analytic programs such as ANOVA, multiple regression, and others. Randomization of questions and
the weighting of respondents to ensure more representative results are some of the attractive features
of survey design systems.

INTERNATIONAL DIMENSIONS OF SURVEYS

With the globalization of business operations, managers often need to compare the business
effectiveness of their subsidiaries in different countries. Researchers engaged in cross‐cultural research
may attempt to trace the similarities and differences in the behavioral and attitudinal responses of
employees, consumers, or investors in different cultures.

When data are collected through questionnaires and occasionally through interviews, one should pay
attention to the measuring instruments and how data are collected, in addition to being sensitive to
cultural differences in the use of certain terms.

Special issues in instrumentation for cross-cultural research:

▪ Certain special issues need to be addressed while designing instruments for collecting data from
multiple countries. Since different languages are spoken in different countries, it is important to
ensure that the translation of the instrument to the local language matches accurately to the
original language. For this purpose, the instrument should be first translated by a local expert.
▪ Supposing a comparative survey is to be done between Japan and the United States, and the
researcher is a US national, then the instrument has first to be translated from English to
Japanese. Then, another bilinguist should translate it back to English. This back translation, as it
is called, ensures vocabulary equivalence (i.e., that the words used have the same meaning).
▪ Idiomatic equivalence could also become an issue, where some idioms unique to one language
just do not lend themselves for translation to another language.

Issues in cross-cultural data collection:

At least three issues are important for cross‐cultural data collection – response equivalence, timing of
data collection, and the status of the individual collecting the data.

▪ Response equivalence is ensured by adopting uniform data collection procedures in the

different cultures. Identical methods of introducing the study, the researcher, task instructions,
and closing remarks, in personally administered questionnaires, provide equivalence in
motivation, goal orientation, and response attitudes.
 ▪ Timing of data collected across cultures is also critical for cross‐cultural comparison. Data
collection should be completed within acceptable time frames in the different countries – say
within three to four months. If too much time elapses in collecting data in the different
countries, much might change during the time interval in any one country or all the countries.

▪ Status of the individual collecting the data : As pointed out as early as 1969 by Mitchell, in
interview surveys, the egalitarian‐oriented interviewing style used in the West may not be
appropriate in societies that have well‐defined status and authority structures. Also, when a
foreigner comes to collect data, the responses might be biased for fear of portraying the country
to a “foreigner” in an “adverse light”.

REVIEW OF THE ADVANTAGES AND DISADVANTAGES OF

DIFFERENT DATA COLLECTION METHODS AND WHEN TO USE
EACH

Advantages Disadvantages Best suited when

FACE-TO-FACE Provide rich data, offer On the negative side, Face‐to‐face

INTERVIEWS the opportunity to face‐to‐face interviews interviews are
establish rapport with have the potential for best suited to the
the interviewees, and introducing interviewer exploratory stages
help to explore and bias and can be of research when
understand complex expensive if a large the researcher is
issues. number of subjects are trying to get an
involved. overarching view
of the concepts or
the situational
factors.
TELEPHONE INTERVIEWS Interviews help to On the negative side, the
contact subjects interviewer cannot
dispersed over various observe the nonverbal
geographic regions and responses of the
obtain immediate respondents, and the
responses from them. interviewee can block
the call.
They are, hence, an
efficient way of
collecting data when one
has specific, structured
questions to ask, needs
the responses quickly,
and has a sample spread
over a wide area.

OBSERVATIONAL Help us to comprehend On the negative side, Observational

STUDIES complex issues through they are expensive, studies are best
direct observation since long periods of suited for
(either as a participant observation (usually research requiring
or a nonparticipant encompassing several non‐self‐report
observer) and then, if weeks or even months) descriptive data;
possible, asking are required, and that is, when
questions to seek observer bias may well behaviors are to
clarification on certain be present in the data. be understood
issues. without directly
asking the
The data obtained are respondents
rich and uncontaminated themselves.
by self‐report bias.
 Personally administering Helps to (1) establish On the negative side, Personally
questionnaires rapport with the administering administered
respondents while questionnaires questionnaires
introducing the survey, personally is expensive, are best suited
(2) provide clarification especially if the sample when data are
sought by the is widely collected from
respondents on the spot, dispersed subjects that are
and geographically. located in close
(3) collect the proximity to one
questionnaires another and
immediately after they groups of
are completed. In that respondents can
sense, there is a 100% be conveniently
response rate. assembled.

ELECTRONIC Are advantageous when On the negative side, The

QUESTIONNAIRES responses to many such questionnaires electronic
questions have to be usually have a low questionnaire
obtained from a response rate and one survey is best
sample that is cannot be sure suited when
geographically if the data obtained are information is to
dispersed, or it is difficult unbiased since the be obtained on a
or not possible to nonrespondents may be substantial scale
conduct telephone different from those who through
interviews without did respond. structured
much expense. questions, at a
reasonable cost,
from a sample
that is widely
dispersed
geographically.

MULTIMETHODS OF DATA COLLECTION

Because almost all data collection methods have some bias associated with them, collecting data
through multimethods and from multiple sources lends rigor to research. For instance, if the responses
collected through interviews, questionnaires, and observation are strongly correlated with one another,
then we will have more confidence about the goodness of the collected data. If the same question
fetches discrepant answers in the questionnaire and during the interview, then an air of uncertainty
emerges and we will be inclined to discard both data as being biased.

Likewise, if data obtained from several sources bear a great degree of similarity, we will have stronger
conviction in the goodness of the data. For example, if an employee rates his performance as 4 on a five‐
point scale, and his supervisor gives him a similar rating, we may be inclined to consider him a better
than average worker. On the contrary, if he gives himself a 5 on the five‐point scale and his supervisor
gives him a rating of 2, then we will not know to what extent there is a bias and from which source.

Therefore, high correlations among data obtained on the same variable from different sources and
through different data collection methods lend more credibility to the research instrument and to the
data obtained through these instruments. Good research entails collection of data from multiple sources
and through multiple data collection methods. Such research, though, is more costly and time
consuming.

The END