Question order in the questionnaire

There are certain rules regarding the ordering of questions that must always be borne in mind. These have been covered in Chapter 3 and include:

• There must be no prompting of any information before spontaneous questions on the same subject.
• The interview should normally start with the more general questions relating to the topic and work through to the more specific or detailed subject matter.
• Behavioural questions should be asked before attitudinal questions on the same topic.

These issues should have been considered when the questionnaire was planned, but still need to be thought about as the detailed questionnaire is written.

1. Funnelling

Funnelling sequences are used to take respondents from general ques­tions on a topic through to questions that are more specific without allow­ing the earlier questions to condition or bias the responses to the later ones.

Typically in the funnelling sequence, whether respondents are asked a question depends on their response to the previous one. This means that people for whom questions are irrelevant can be routed round them. Because people are routed out without knowing what the criteria are for continuing the question sequence, we can be more confident that the response that we obtain to the final question is not biased. In the example in Figure 7.6, we would have little confidence that there was no bias had we asked the one question ‘If you have seen any advertising for Bulmer’s cider on television recently, what did it say?’ This question would lead to overclaiming of having seen advertising, because there is an assumption that Bulmer’s cider has been advertised on television recently. Some respondents would then claim to have seen it, even though they had not.

Funnelling sequences can be complicated for respondents to follow on paper self-completion questionnaires because of the routeing, and are best avoided. However, they can be used with any interviewer-adminis­tered questionnaire and work very well with electronic or web-based self­completion questionnaires where the routeing is hidden.

2. Question order bias

2.1. Priming effects

Where there is a key question to be asked, such as approval of a proposal, response to a new concept or rating of an issue, the act of asking ques­tions about the respondent’s feelings regarding the proposal, concept or issue prior to the key questions can have an effect on the response to it.

This can be desirable, as the researcher will want respondents to give an answer that takes into account their considered view. However, the researcher can suggest to respondents what they should answer. McFarland (1981) reported that asking a series of specific questions about the energy crisis led to a higher rating of the severity of the crisis than when the questions were not asked.

Questionnaire writers need to be aware of the influence that prior ques­tions can have, and write the questions and interpret the responses accordingly.

2.2. Consistency effect

A particular type of priming effect is the consistency effect. This can occur because respondents are led along a particular route of responses to a conclusion to which they can only answer one way if they are to appear consistent.

Consider the sequence in Figure 7.7.

Now compare Figure 7.7 with the sequence in Figure 7.8.

It should be expected that the responses to Q2 will show significant variation between Figures 7.7 and 7.8. By using statements that reflect one side of an argument, in this case for and against the building of a new airport, respondents are led to Q2 along different paths. Most people like to appear to be consistent. If they agree with the statements in Q1, it is then very difficult not to answer ‘yes’ at Q2 in the first example or ‘no’ in the second example.

To be even-handed, the preliminary question should contain state­ments that relate to both or all sides of an argument. The researcher may want to put questions to respondents about the issues before asking the key question, in order to help them to give a considered answer to that question. However, the preliminary questions must fairly represent all the issues if they are not to bias the response to the key question.

Source: Brace Ian (2018), Questionnaire Design: How to Plan, Structure and Write Survey Material for Effective Market Research, Kogan Page; 4th edition.