Design of postal questionnaires

Theme and covering letter

The general theme of the questionnaire should be made explicit in a covering letter. You

should state who you are; why the data is required; give, if necessary, an assurance of

confidentiality and/or anonymity; and contact number and address or telephone number. This

ensures that the respondents know what they are committing themselves to, and also that they

understand the context of their replies. If possible, you should offer an estimate of the

completion time. Instructions for return should be included with the return date made obvious.

For example: ‘It would be appreciated if you could return the completed questionnaire by... if

at all possible’.

Instructions for completion

You need to provide clear and unambiguous instructions for completion. Within most

questionnaires these are general instructions and specific instructions for particular question

structures. It is usually best to separate these, supplying the general instructions as a preamble

to the questionnaire, but leaving the specific instructions until the questions to which they

apply. The response method should be indicated (circle, tick, cross, etc.). Wherever possible,

and certainly if a slightly unfamiliar response system is employed, you should give an

example.

Appearance

Appearance is usually the first feature of the questionnaire to which the recipient reacts. A

neat and professional look will encourage further consideration of your request, increasing

your response rate. In addition, careful thought to layout should help your analysis. There are

a number of simple rules to help improve questionnaire appearance:

• Liberal spacing makes the reading easier.

• Photo-reduction can produce more space without reducing content.

• Consistent positioning of response boxes, usually to the right, speeds up completion

and also avoids inadvertent omission of responses.

• Choose the font style to maximize legibility.

• Differentiate between instructions and questions. Either lower case and capitals can be

used, or responses can be boxed.

Length

There may be a strong temptation to include any vaguely interesting questions, but you should

resist this at all costs. Excessive size can only reduce response rates. If a long questionnaire is

necessary, then you must give even more thought to appearance. It is best to leave pages

unnumbered; for respondents to flick to the end and see ‘page 27’ can be very disconcerting!

Order

Probably the most crucial stage in questionnaire response is the beginning. Once the

respondents have started to complete the questions they will normally finish the task, unless it

is very long or difficult. Consequently, you need to select the opening questions with care.

Usually the best approach is to ask for biographical details first, as the respondents should

know all the answers without much thought. Another benefit is that an easy start provides

practice in answering questions.

Once the introduction has been achieved the subsequent order will depend on many

considerations. You should be aware of the varying importance of different questions.

Essential information should appear early, just in case the questionnaire is not completed. For

the same reasons, relatively unimportant questions can be placed towards the end. If questions

are likely to provoke the respondent and remain unanswered, these too are best left until the

end, in the hope of obtaining answers to everything else.

Coding

If analysis of the results is to be carried out using a statistical package or spreadsheet it is

advisable to code non-numerical responses when designing the questionnaire, rather than

trying to code the responses when they are returned. An example of coding is:

Male [ ] Female [ ]

1 2

The coded responses (1 or 2) are then used for the analysis.

Thank you

Respondents to questionnaires rarely benefit personally from their efforts and the least the

researcher can do is to thank them. Even though the covering letter will express appreciation

for the help given, it is also a nice gesture to finish the questionnaire with a further thank you.

Questions

• Keep the questions short, simple and to the point; avoid all unnecessary words.

• Use words and phrases that are unambiguous and familiar to the respondent. For

example, 'dinner' has a number of different interpretations; use an alternative

expression such as 'evening meal'.

• Only ask questions that the respondent can answer. Hypothetical questions should be

avoided. Avoid calculations and questions that require a lot of memory work, for

example, 'How many grammar lessons did you attend last year?'

• Vacuous words or phrases should be avoided. 'Generally', 'usually', or 'normally' are

imprecise terms with various meanings. They should be replaced with quantitative

statements, for example, 'at least once a week'.

• Questions should only address a single issue. For example, questions like: 'Do you take

extra grammar lessons at evening classes? 'should be broken down into two discreet

stages, firstly find out if the respondent takes extra grammar lessons, and then secondly

find out if they go to evening classes.

• Do not ask two questions in one by using 'and'. For example, 'Did you watch videos

and read newspapers in English last semester?'

• Avoid double negatives. For example, 'Is it not true that you did not read a newspaper

yesterday?' Respondents may tackle a double negative by switching both negatives and

then assuming that the same answer applies. This is not necessarily valid.

• Avoid emotive or embarrassing words usually connected with race, religion, politics,

sex, money.

Types of questions

Closed questions

A question is asked and then a number of possible answers are provided for the respondent.
The respondent selects the answer which is appropriate. Closed questions are particularly

useful in obtaining factual information:

Sex: Male [ ] Female [ ]

Did you watch television last night? Yes [ ] No [ ]

Some ‘Yes/No’ questions have a third category ‘Do not know’. Experience shows that as long

as this alternative is not mentioned people will make a choice. Also the phrase ‘Do not know’

is ambiguous:

Do you agree with the introduction of the EMU?

Yes [ ] No [ ] Do not know [ ]

What was your main way of learning a new word? Tick one box only.

Saying it aloud []

Writing it on the board []

Translating it into Vietnamese []

Making new sentences with It []

Other means, please specify

With such lists you should always include an ‘other’ category, because not all possible

responses might have been included in the list of answers.

Sometimes the respondent can select more than one from the list. However, this makes

analysis difficult:

Why have you watched videos in English? Tick the relevant answer(s). You may tick as

many as you like.

I enjoy watching movies [ ]

Watching videos in English is useful for my listening skills [ ]

My teachers advised me to do so [ ]

Other reason, please specify

Attitude questions

Frequently questions are asked to find out the respondents’ opinions or attitudes to a given

situation. A Likert scale2

provides a battery of attitude statements. The respondent then says

how much they agree or disagree with each one:

Read the following statements and then indicate by a tick whether you strongly agree,

agree, disagree or strongly disagree with the statement.

Strongly agree

Agree Disagree Strongly

disagree

My English teacher is always explains grammatical points in English.

There are many variations on this type of question. One variation is to have a ‘middle

statement’, for example, ‘Neither agree nor disagree’. However, many respondents take this

as the easy option. Only having four statements, as above, forces the respondent into making a

Attitude measurement used in research, where, in place of a numerical scale for answers, answers are
given on

a scale ranging from complete agreement on one side to complete disagreement on the other side, with
no

positive or negative choice. Another variation is to rank the various attitude statements,

however, this can cause analysis problems:

Which of these characteristics do you like about your English lesson? Indicate the best

three in order, with the best being number 1.

Varied activities [ ]

Good presentation [ ]

Opportunities for speaking [ ]

Opportunities for pair work [ ]

Learner-centered [ ]

Friendly atmosphere [ ]

A semantic differential scale attempts to see how strongly an attitude is held by the

respondent. With these scales double-ended terms are given to the respondents who are asked
to indicate where their attitude lies on the scale between the terms. The response can be

indicated by putting a cross in a particular position or circling a number:

English pronunciation is: (circle the appropriate number)

Difficult 1 2 3 4 5 6 7 Easy

Useless 1 2 3 4 5 6 7 Useful

Interesting 1 2 3 4 5 6 7 Boring

For summary and analysis purposes, a ‘score’ of 1 to 7 may be allocated to the seven points of

the scale, thus quantifying the various degrees of opinion expressed. This procedure has some

disadvantages. It is implicitly assumed that two people with the same strength of feeling will

mark the same point on the scale. This almost certainly will not be the case. When faced with

a semantic differential scale, some people will never, as a matter of principle, use the two end

indicators of 1 and 7. Effectively, therefore, they are using a five-point scale. Also scoring the

scale 1 to 7 assumes that they represent equidistant points on the continuous spectrum of

opinion. This again is probably not true. Nevertheless, within its limitations, the semantic

differential can provide a useful way of measuring and summarizing subjective opinions.

Other types of questions to determine peoples’ opinions or attitudes are:

Which one/two words best describes...?

Which of the following statements best describes...?

How much do you agree with the following statement...?

Open questions

An open question such as ‘What are the essential skills a manager should possess?’ should be

used as an adjunct to the main theme of the questionnaire and could allow the respondent to

elaborate upon an earlier more specific question. Open questions inserted at the end of major

sections, or at the end of the questionnaire, can act as safety valves, and possibly offer

additional information. However, they should not be used to introduce a section since there is

a high risk of influencing later responses. The main problem of open questions is that many

different answers have to be summarized and possibly coded.

Testing – pilot survey

Questionnaire design is fraught with difficulties and problems. A number of rewrites will be
necessary, together with refinement and rethinks on a regular basis. Do not assume that you

will write the questionnaire accurately and perfectly at the first attempt. If poorly designed,

you will collect inappropriate or inaccurate data and good analysis cannot then rectify the

situation.

To refine the questionnaire, you need to conduct a pilot survey. This is a small-scale trial prior

to the main survey that tests all your question planning. Amendments to questions can be

made. After making some amendments, the new version would be re-tested. If this re-test

produces more changes, another pilot would be undertaken and so on. For example, perhaps

responses to open-ended questions become closed; questions which are all answered the same

way can be omitted; difficult words replaced, etc.

It is usual to pilot the questionnaires personally so that the respondent can be observed and

questioned if necessary. By timing each question, you can identify any questions that appear

too difficult, and you can also obtain a reliable estimate of the anticipated completion time for

inclusion in the covering letter. The result can also be use to test the coding and analytical

procedures to be performed later.

Distribution and return

The questionnaire should be checked for completeness to ensure that all pages are present and

that none is blank or illegible. It is usual to supply a prepaid addressed envelope for the return

of the questionnaire. You need to explain this in the covering letter and reinforce it at the end

of the questionnaire, after the ‘Thank you’.

Finally, many organizations are approached continually for information. Many, as a matter of

course, will not respond in a positive way.

Interviews

Interviewing is a technique that is primarily used to gain an understanding of the underlying

reasons and motivations for people’s attitudes, preferences or behavior. Interviews can be

undertaken on a personal one-to-one basis or in a group. They can be conducted at work, at

home, in the street or in a shopping center, or some other agreed location.

Personal interview

Advantages
• Serious approach by respondent resulting in accurate information.

• Good response rate.

• Completed and immediate.

• Possible in-depth questions.

• Interviewer in control and can give help if there is a problem.

• Can investigate motives and feelings.

• Can use recording equipment.

• Characteristics of respondent assessed ฀ tone of voice, facial expression, hesitation,

etc.

• Can use props.

• If one interviewer used, uniformity of approach.

• Used to pilot other methods.

Disadvantages

• Need to set up interviews.

• Time consuming.

• Geographic limitations.

• Can be expensive.

• Normally need a set of questions.

• Respondent bias "tendency to please or impress, create false personal image, or end

interview quickly".

• Embarrassment possible if personal questions.

• Transcription and analysis can present problems ฀ subjectivity.

• If many interviewers, training required.

Types of interview

Structured

• Based on a carefully worded interview schedule.

• Frequently require short answers with the answers being ticked off.

• Useful when there are a lot of questions which are not particularly contentious or

thought provoking.
• Respondent may become irritated by having to give over-simplified answers.

Semi-structured

The interview is focused by asking certain questions but with scope for the respondent to

express him or herself at length.

Unstructured

This also called an in-depth interview. The interviewer begins by asking a general question.

The interviewer then encourages the respondent to talk freely. The interviewer uses an

unstructured format, the subsequent direction of the interview being determined by the

respondent’s initial reply. The interviewer then probes for elaboration – ‘Why do you say

that?’ or, ‘That’s interesting, tell me more’ or, ‘Would you like to add anything else?’ being

typical probes.

The following section is a step-by-step guide to conducting an interview. You should

remember that all situations are different and therefore you may need refinements to the

approach.

Planning an interview

• List the areas in which you require information.

• Decide on type of interview.

• Transform areas into actual questions.

• Try them out on a friend or relative.

• Make an appointment with respondent(s) ฀ discussing details of why and how long.

• Try and fix a venue and time when you will not be disturbed.

Conducting an interview

Personally

arrive on time be smart smile employ good manners find a balance between

friendliness and objectivity.

At the start

introduce yourself re-confirm the purpose assure confidentiality – if relevant specify

what will happen to the data.

The questions

speak slowly in a soft, yet audible tone of voice control your body language know the

questions and topic ask all the questions.

Responses

recorded as you go on questionnaire written verbatim, but slow and time-consuming

summarized by you taped – agree beforehand – have alternative method if not

acceptable consider effect on respondent’s answers proper equipment in good working

order sufficient tapes and batteries minimum of background noise.

At the end

ask if the respondent would like to give further details about anything or any questions

about the research thank them.

Telephone interview

This is an alternative form of interview to the personal, face-to-face interview.

Advantages

• Relatively cheap.

• Quick.

• Can cover reasonably large numbers of people or organizations.

• Wide geographic coverage.

• High response rate.

• No waiting.

• Spontaneous response.

• Help can be given to the respondent.

• Can tape answers.

Disadvantages

• Often connected with selling.

• Questionnaire required.

• Not everyone has a telephone.

• Repeat calls are inevitable.

• Time is wasted.

• Straightforward questions are required.