Main questionnaire

The main questionnaire can now be planned.

Once into the main questionnaire, the writer must consider the order in which the various topics are presented to the respondents. As a rule, it is better to work from the most general topics through to the most specific. Thus, the interview might start with questions about the respondent’s behaviour in the market in general, before proceeding through to specific questions about the client’s product and then to reaction to a new propo­sition for the client’s product. There are two reasons for this.

First, if the questions regarding the specific product or brand of interest were asked first, then the respondents would be aware of the question writer’s interest and this would bias their answers to the more general market questions that come later. Raising the respondents’ consciousness of the product or brand in question will tend to lead to it being over-represented as a response in any questions that follow. This may include questions about consumption of products or brands in the market generally and lead to overestimation of consumption of the brand of interest.

Second, starting with general questions allows the respondents to think about their behaviour in the market before getting into the detail. Respondents are rarely as interested in the market as are the researcher and client. They may find it difficult to respond immediately to questions about the detail of a particular brand or product. Starting with questions that are more general helps the respondents to ease into the subject, recalling their overall behaviour and how they feel about brands and products before reaching the detailed questions.

There are many exceptions to this general rule when there is a good research reason for not starting with the more general questions, but the questionnaire writer should always be prepared to justify the decision.

It is important to map the questionnaire so that it flows logically from one subject area to the next. Avoid returning to a topic area previously asked about. This makes the questionnaire appear not to have been thought through, can confuse respondents who think that they have dealt with this already, and can frequently require interviewers to refer back in the questionnaire for information already given, which may lead to errors.

A flow diagram can assist in ensuring that all topics are covered and that respondents are asked the sections that are relevant to them. In the example flow chart (Figure 3.2), the objective is to determine what journey types buses are used for; to determine why the bus or other public transport is preferred to using a car; and to obtain a rating of different types of public transport. People who do not use any form of public transport are not to be asked this last section. This diagram does not tell us precisely what questions need to be asked. What it determines is how the question areas that the different categories of respondents (bus users, non-bus users who use other public transport, and people who use no public transport) need to be asked will flow.

The flow chart also demonstrates that there will be some routeing issues. Whether or not the respondent has use of a car appears three times in different paths. Complex routeing will be required if the questionnaire writer decides that this question should appear only once, in order to facilitate analysis. Alternatively, the same question can appear three times, once in the path of each respondent category. The latter approach is less likely to result in interviewer error if using paper questionnaires, or in routeing errors within electronic questionnaires.

1. Behaviour before attitude

It is generally advisable to start any section of the interview with behav­ioural questions before going on to ask attitudes and images. This is in part to allow the respondents to assess their behavioural position and then to explain their behaviour through their attitudes. Behavioural ques­tions, are usually easier to answer because they relate to fact and require only recall. If respondents find it difficult to answer behavioural ques­tions, then this is usually because the questionnaire writer has been too ambitious in the level of detail expected, and the reliability of the infor­mation that is being reported will be in doubt.

If attitudes are asked first there is a danger that respondents will take a position that is not thought through and that is contradicted by their behaviour. They may well then misreport their behaviour in order to justify their attitudes.

2. Spontaneous before prompted

It may appear obvious, but great care must be taken not to prompt respondents with possible answers before asking questions designed to obtain their spontaneous response. Thus you cannot ask ‘Which brands of instant coffee can you think of?’ if you have already asked ‘Which of the brands of instant coffee on this list do you buy?’ An example like this appears obvious, but there are many occasions where it is not so obvious that this is happening.

Sometimes it can be virtually impossible to obtain a ‘clean’ measure of spontaneous brand awareness, particularly where purchase or consump­tion of a brand is one of the screening criteria for eligibility. This is because respondents will have been exposed to a list of brands in the screening questions. Thereafter it is impossible to obtain a measure of spontaneous awareness.

This is a particular issue with certain types of surveys such as advertis­ing testing. Here respondents may be recruited based on their brand consumption in order to evaluate a new advertisement. Part of that eval­uation may be to show the test advertisement among other ads. For tele­vision ads this would be as part of a clutter reel; for press ads they would be contained within a mock-up of a newspaper or magazine. The test ad will, however, stand out from the rest if the respondents have been sensi­tized to the brand or the category through the screening questions. To ameliorate this, a series of mock screening questions are sometimes asked, which relate to the products and categories shown in the other ads. Whilst this is unlikely to reduce the sensitization of the respondents to the test ad’s category, it does raise the level of sensitization so that it is the same for all the ads, thereby cancelling out the differential effect. This type of strategy often needs to be adopted where it is essential that prompting occurs earlier than is desirable.

Prompting also extends to attitudes. A questionnaire may include a series of attitude statements to which respondents are asked to respond. If attitudes on the same subject are to be assessed spontaneously, that must be asked before the attitude statements have been shown or respon­dents will continue to play back the attitudes with which they have been prompted.

3. Sensitive sections

If the interview is to include questions of a sensitive nature, then they should not be asked right at the beginning of the interview. Where the questionnaire is interviewer administered, this allows a relationship to be built between interviewer and respondent, so that the respondent is more willing to disclose sensitive information. The trust that has hopefully been built between them reassures the respondent that the information will not be abused.

With web-based questionnaires, these questions should also be posi­tioned towards the end of the interview. Although there is no interviewer, there is still a relationship built between the respondent and ‘the survey’. Having been prepared to divulge less sensitive information in earlier questions, it may be less difficult for respondents then to disclose data that are more sensitive. Such questions at the beginning of the interview are likely to be seen as more intrusive and provoke a greater level of non­response or termination of interview.

A further reason for asking sensitive questions later in the interview is that if the interview is terminated at this point by the respondent, most of the data have already been collected and may be usable. In extreme cases where it is expected that the level of termination due to intrusiveness of the questions will be high, being able to salvage as much information as possible will be part of the questionnaire writer’s strategy, and all key questions for analysis will have been asked before the intrusive ques­tions. However, if questions are so intrusive as to cause a significant level of offence, then the questionnaire writer should consider the ethical posi­tion carefully before including them. (See Chapter 11 for what may constitute a sensitive topic.)

4. Classification questions

Partly because they can be seen as intrusive, classification questions are normally asked at the end of the interview. They are also positioned here because they are usually disconnected with the subject matter of the interview. Asking them earlier in the interview would disrupt the flow of the ‘conversation’. Information such as gender, age, income, social group­ing, final level of education, television viewing, number of children in household, etc rarely relate directly to the subject of the study. However, they are proven discriminators in many behavioural and attitudinal fields and so are invaluable for cross-analysis purposes.

The researcher should resist the temptation to ask for more classifica­tion data than are needed simply because it might be useful for cross­analysis. This is often personal information and respondents do not always understand why it is needed. The questionnaire writer should think carefully about what is and what is not required.

Source: Brace Ian (2018), Questionnaire Design: How to Plan, Structure and Write Survey Material for Effective Market Research, Kogan Page; 4th edition.