Guide to the Design of Questionnaires

Design of the questionnaire can be split in to three elements:

1. determine the questions to be asked,

2. select the question type for each question and specify the wording, and

3. design the question sequence and overall questionnaire layout.

Available software tends to focus on support for (b) and (c).

Determine the Questions to be Asked

This step is a key one that seems not to be sufficiently stressed in the literature or conducted in practice. A key link needs to be established between the research aims and the individual questions via the research issues. Issues and questions can be determined through a combined process of exploring the literature and thinking creatively. A simple illustration of the outcome of such a process is given below.

Survey aims: to explore the factors that might explain the reasons that Leeds University candidates give for undertaking a MBA programme:

Issue:	Question focus:
What reasons might candidates give for undertaking an MBA?	Is the candidate looking for: career change career advancement higher remuneration etc.
Could past experience affect the reasons?	How many years work experience does candidate have?
Could gender differences affect the reasons?	Is the candidate male or female?
Could educational background and attainment affect the reasons?	What is highest educational qualification obtained? What subject area(s) is this qualification in?
Etc.	Etc.

The above process generates the focus for individual questions that can then be designed in detail.

Decide on a Layout and Sequence

Do not clutter the form up with unnecessary headings, and numbers. However, it is good practice to ensure that the questionnaire has a title and that the revision or date of the version is printed on the questionnaire. (This will particularly help as you take the draft through a series of revisions.) A brief introductory statement is useful, especially if the introductory letter could go adrift. Contact and return information should be included on the questionnaire, irrespective of whether addressed return envelopes are provided; these can easily become separated. Similarly it is good practice to number or otherwise identify individual questions for reference purposes; this is particularly helpful to deal with queries during the data entry and analysis stage.

Lay out the questions and answer choices attractively and neatly. Try to be consistent in aspects such as wording and try to standardise by using as few question types as possible. Avoid switching between landscape and portrait for the text layout. Be careful not to overfill the page. Avoid using lots of lines, borders and boxes since these can make the page look too 'dense'. A key factor that affects the response rate is the length of the questionnaire; questionnaires perceived as long will deter respondents. Using a small font can cut down the number of pages and hence make the questionnaire look shorter; but remember that small fonts can put people off - particularly those with less than perfect eyesight. Use a good legible font; a serif font like Times is easier to read than a sanserif one like Helvetica. Make good use of italics and bold types: think of using italics consistently to give instructions, e.g. tick the relevant box. Consider using bold for the questions themselves or for headings. Symbol fonts can be useful for characters such as boxes and ticks.

If you are relying on the respondent to complete the questionnaire, begin with questions that will raise interest. However, there are different views on sequencing of questions. For example, someone might argue that the easier questions to answer should be at the beginning to get the respondent in to the swing of things. However, someone else might suggest that questions about personal data, which are easy to answer, should be left until the end when the respondent has committed themselves to answering and they are less likely to object to giving such data. Whatever approach you choose you should try to have a logical sequence, e.g. group together all questions that relate to similar areas.

You should try to keep the flow through a questionnaire logical and very simple, i.e. avoid complex branching. Although some questions may be consequent upon earlier answers, keep the number of branches to the minimum. If necessary, use two or three versions of the questionnaire for respondents in different situations.

Question Types

Different types of questions can be used, e.g. open vs. closed, single vs. multiple responses, ranking, and rating.

Open vs. Closed Questions

Many advise against using open-ended questions and advocate using closed questions. However, open questions can be useful. For example, the open question:

What do you think are the reasons for football hooliganism?

would elicit a whole range of replies of varying length and articulation. If you are interested in making very precise judgements of each individual respondent this may well be useful. If, however, you are concerned, as most surveys are, in summarising replies to produce a picture of your population, a better approach may be ...

Do you think football hooliganism is caused by: (tick if appropriate)

Lack of discipline at home
Players' behaviour on pitch
Family breakdown
Youth unemployment
Poor schooling
Violence on T.V.
Other (please specify)

Plan to make your categories exhaustive, i.e. covering all possibilities, by making fairly broad suggestions that will still satisfy your objectives. However, you could include the catch-all type option (Other) as above.

If you still feel that your questions cannot be categorised until all the replies are returned then ensure that sufficient space is included for the question and leave a space in the margin to code a numeric response.

e.g. What was the main problem you encountered
with your wheelchair?

|

Single vs. Multiple Response

When designing questions make sure you have thought through whether you want the respondent to give a single or a multiple response. For example, if you ask the question:

Which of the following means do you use to travel to college?

Bus
Car
Bike

You might get someone who thinks that only one box should be ticked while another respondent might believe they are at liberty to tick as many boxes as they like. If you intend that the respondent treat these as a series of independent dichotomous yes/no questions then the question could be clarified by inserting the text "tick all boxes that apply". Note that treating the question in this way would require three separate variables to be set up on the computer and for each variable coding might be 0 for no and 1 for yes.

Consider the following question:

What is your most usual means of travelling to college?

(Tick one box only)

Bus
Car
Bike

The above should elicit one response, i.e. the answers form mutually exclusive categories. For the computer the above would be coded as one variable with Bus represented by 1, Car by 2 and Bike by 3. If the respondent omitted to answer then this could be coded as 0 or some other missing value. However, note the problem with both the above questions if someone travels by train or just walks. Either all the possibilities have to be anticipated in advance or an additional box has to be offered for the respondent to tick and specify the mode of transport.

Never be tempted to use the following structure:

Select up to three of the options below and enter in the boxes opposite

Option A	Option B	Option C
Option D	Option E	Option F

Whilst this may be logically sound you will have problems when summarising these results. A very tedious and finally unsatisfactory search is required to determine how any given option performs.

Ranked Responses

Sometimes it is useful for the respondent to rank a set of options by numbering them in order from 1 to the maximum number you are interested in.

For example, to a question like:

Place in order of importance to you the following features of a camping holiday

(Indicate by numbering from 1-4 in order where 1 is the most important)

Open air
Mobility
Cost
People
Atmosphere

Note that each option will need to be coded as a separate variable and in the above case five variables are required (even though only four ranks are to be identified). This approach can generate a lot of data and so the number of options used should not be excessive. Apart from this respondents find it difficult to discriminate meaningfully between lots of options.

Rated Responses

A popular approach in the social sciences is to use Likert scales such as the example below:

(Circle the number under the initials that applies.
VI=Very important; I=Important; N=Neutral; U=Unimportant;
VU=Very Unimportant).

Indicate your view of the following aspects of a camping holiday

        VI I  N  U  VU
Community life     1  2  3  4  5
Low cost      1  2  3  4  5
Outdoor life      1  2  3  4  5
Ability to move around 1  2  3  4  5

Note that each of the four rows will form a separate variable that contains the appropriate numeric code from 1 to 5.

Decide on Question Wording

Some general rules can be stated on question wording:

• Be concise and unambiguous

• Avoid double questions

• Avoid questions involving negatives

• Ask for precise answers

• Avoid leading questions

Be Concise and Unambiguous

Make questions brief and clear. Avoid jargon; e.g. asking "do you believe that the UK should have upper and lower houses of parliament?" is more likely to elicit an informed response than "do you believe that the UK should have a bicameral parliament?"

Check for ambiguity and make sure that the answer may be competently answered. E.g. asking "have you been to the cinema recently" is more ambiguous than "have you been to the cinema in the last two weeks?"

Avoid Double Questions

Sometimes questions hide a dual question, for example:

Do you think the British should eat less and exercise more?

instead ask:

(Please circle relevant number)

                      Yes No
Do you think the British should eat less  1  2
Do you think the British should exercise more  1  2

Notice the consistent use of circled number responses for Yes/No questions.

Avoid Questions Involving Negatives

Don't confuse the respondent by language like this:

(Please circle relevant number)

                   Yes  No
Are you against a ban on smoking  1  2

Ask for Precise Answers

Ask for precise answers if you think the information is available and there are no other constraints (e.g. too intrusive on privacy). For example:

Give your age on lst September 2001

years

is preferable to:

Are you...

Under 18
18-65
Over 65

In this example although age groups may be all that is required, asking for the exact age will also suffice and have the added advantages that:

There is less room for error, i.e. ticking the wrong box.

Exact ages may very simply be recoded, by a computer program, into groups.

The researcher can alter these groups by keeping the exact ages. Not only could you inspect different age groups than at first envisaged, but you could backdate your age groups.

You can obtain continuous descriptive statistics, e.g. mean and standard deviation.

Using a computer you can transform dates of birth into ages as long as you have the date on which the information was given. Of course an obvious advantage of offering groups to the respondent is that it can reduce the time taken to complete the question or, in some cases, improve the response rate to that particular question.

Avoid Leading Questions

Leading questions such as "Do you agree with the majority of people that the health service is failing?" should be avoided for obvious reasons that any right-minded individual can see. Don't you agree?

Using the Computer to Design and Construct the Questionnaire

Standard applications, such as a word-processing package, can help in designing and producing professional-quality questionnaires. However, specific packages are available that support the design (and subsequent analysis) of questionnaires, e.g.

• Snap by Mercator

• SphinxSurvey by Scolari (Sage)

• Merlin Software by Merlinco

In fact there is a wide range of products and services available, many via the Internet. A good list of what is available can be obtained from www.yahoo.com on their pages: Business and Economy > Companies> Computers> Business to Business> Software> Surveys and Polling

Some further web resources that also have lists of available software are listed later in the references section. Apart from design software that you can buy to run on your computer, some companies offer software and services for free; but these are often tied to the company's proprietary data entry and analysis processes that you must pay to use. This is particularly the case where companies host the questionnaire on their web site for respondents to fill on-line. Notwithstanding these comments, such packages may also enable survey data to be imported and exported in SPSS and other file formats, e.g. survey interchange standard. If you can pay, companies will provide a full service from design through to analysis.

For the adventurous or the adept HTML author, a questionnaire can be built using HTML and CGI script to publish on a website (see e.g. chapter 11 on forms in Castro, 2000).

Confidentiality and Ethics

Firstly allow for privacy and do not ask questions which may offend, or ask for data that is not essential. Apart from anything else, your response rate will suffer.

Secondly, especially if you need to ask some personally searching questions, it helps to explain as much as you are able about your research to the respondent, both at the beginning and throughout the questionnaire.

For example having asked most of your general questions you might at the end write:

Finally could you give us a few bits of information about yourself so that we can put your other replies in greater context.

Age in years

Sex: Male
Female

and so on.

In surveys promises of confidentiality are often made to the respondents to reassure and encourage replies. The researcher should comply with any such promises. However, it might help to explain to the potential respondent what is meant rather than give a blanket assurance of confidentiality, e.g. "your responses will be treated with confidence and at all times data will presented in such a way that your identity cannot be connected with specific published data". This is still compatible with publishing, with respondents' permissions, a list of participants who have co-operated in the survey.

In connection with issues of privacy researchers should be aware that the Data Protection and Human Rights legislation has implications for privacy and confidentiality of survey data. For example, researchers should only use data for the purposes that were declared on the questionnaire. Where the data is particularly sensitive or substantial you may want to include a statement in the questionnaire granting permission for the data to be used in connection with the research and ask respondents to sign. Apart from the legal issues, it is unethical to obtain data from respondents by misleading them about the survey purpose and the method of analysis.