Use of language in the questionnaire

When writing the questionnaire it is the questionnaire writer’s job to ensure that the respondents will understand the questions and that the respondents will not feel intimidated, challenged or threatened by the questions.

Writing questionnaires is about helping respondents to give the best information that they can. Questions should be clear and unambiguous, and the respondent should be put at ease by the tone of the questions and

not made to feel challenged by the words and phrases used. Respondents who feel challenged because they don’t understand the questions will quickly become alienated from the interview process and make little effort to respond accurately. They may become fatigued earlier than they would have done and fail to complete the interview.

Therefore, we must ensure that the questions are phrased in everyday language to which the respondents can relate. The interview can be seen as a conversation by proxy between the researcher and the respondent. The questionnaire should be suitably conversational in tone, while not seeking to be too familiar or condescending.

Researchers are frequently given briefs by clients that are expressed in technical terms that relate to the client’s business. They may talk of ‘chan­nels of distribution’ or ‘above-the-line advertising’. It is the job of the questionnaire writer to turn this into phrases that will be part of the everyday speech of respondents, or at least readily understood by them.

Getting rid of technical terms is not always easy to achieve. They exist because they are needed. Some sympathy must be felt for the writer of the question in the box above. How do you convey to respondents precise noise levels? But equally, how usable is any response to this question? Anyone using the data generated must be concerned about how well the question was understood.

Because technical terms are often the everyday language of the commissioners of the study, they do not always appreciate that others outside their industry or profession might not understand them, or might understand something different by them.

Sometimes technical terms are used in order to describe something, or to differentiate between objects or services, with far greater subtlety than the non-specialist can appreciate. To most motorists a petrol pump is a

petrol pump, and they would not distinguish between a ‘high line fast flow’ and a ‘grouped hose blender’. Researchers must ask themselves if it is necessary for the respondent to be able to distinguish between them in the interview. If it is, then the differences must be clearly explained, if possible without reference to the technical terminology.

Some technical terms are words that have a different everyday use. Market researchers will use the terms ‘random’ and ‘significance’ with specific meanings that are different to the way that they are used by most people. The danger here is that researchers might think that respondents understand the terms in the same way that they do. The respondents, though, understand these terms differently, and so answer a different question to the one that the researcher thinks is being asked.

1. The interview as conversation

Previously in this chapter, the interview was described as a conversation by proxy between the researcher and the respondent. However, it is not the sort of conversation that two people who know each other would have.

With interviewer-administered interviews it is not unusual for respon­dents to try to enter into conversation with the interviewer, to give their views and elaborate on their responses. Only when the interviewer insists on reducing this answer to one of the pre-codes on the questionnaire does the respondent appreciate that this is not really a conversation but an interaction in which they have a specific and limited role to play (Suchman and Jordan, 1990).

The lack of conversation can mask incorrect answers. Through elabora­tion of answers such as ‘Yes, but…’ or ‘I agree, except that…’ it can become clear that the true answer is ‘No’ or ‘I disagree’. If respondents are not allowed to elaborate in this way, then their true answer may not become apparent, and an incorrect answer may be recorded. With self­completion surveys we rely on respondents to think it through, to in effect elaborate to themselves, and not necessarily give their first reaction.

Thus, whilst we conceptualize the questionnaire as the medium of conversation, we must recognize that it is not a true conversation; that this may mean that we do not acquire all of the information that respon­dents could give us or may wish to give us; and that it can, on occasion, lead to incorrect answers being recorded.

Schober (1999) points out two key differences between having a conversation with your aunt and carrying out an interview with a struc­tured questionnaire, known as ‘audience design’ and ‘grounding’.

1.1. Audience design

When one person who knows another asks the second person a question or makes a statement, it is framed to be heard specifically by that other person, and draws on the knowledge that each has of the other. This is known as ‘audience design’. The person to whom it is said is the addressee. Addressees are likely to give different interpretations and responses to the question ‘How many hours a week do you work?’ depending on whether it is asked by their aunt, their boss or someone from the tax office. Addressees will use their knowledge of the relation­ship to determine what type of response the questioner expects to hear.

In a survey questionnaire, the questions are not framed for specific respondents, but to have general applicability to as many people as possi­ble. Interviewers are specifically instructed neither to deviate from the question script nor to tailor the question to the individual. In quantitative research, as hard as questionnaire writers may try, they cannot write a questionnaire to be one side of a conversation.

1.2. Grounding

Grounding occurs in a conversation when the participants establish that each has understood what one of them has said, and that it has entered their common ground. This can come from an acknowledgement of the question or statement (‘uh-huh’, ‘okay’) or a request for elaboration as to what is meant from the addressee, or clarification volunteered by the questioner if it is clear that the addressee has not understood.

Some level of grounding is available in an interview, but interviewers are deliberately restricted in the procedures that they can use in order to avoid introducing bias. Often when asked for clarification, all the inter­viewer can do is to repeat the question, or describe the type of response that is needed, or ask for a best estimate. Elaboration of individual words in the question is to be avoided as, apart from potentially introducing bias, the interviewers themselves may not understand precisely what is meant and present a misinterpretation of the question to respondents.

These difficulties in audience design and grounding can lead to a number of response effects from prompt material, question ordering and interpretation of questions.

2. Minority languages

There are many different types of question that can be asked and in many different ways. What is common to all questions, though, is that they must be worded in a way that is understood by the respondents and to which respondents can relate. This means ensuring that there are minor­ity-language versions of the questionnaire if the sample is likely to include people who speak a language other than the majority language, or whose command of that language is unlikely to be sufficiently good to be able to complete an interview in it. By denying sections of the survey population the opportunity to participate in the study, the questionnaire writer is effectively disenfranchising them from influencing the findings.

For many studies commissioned by the public sector in many coun­tries, it is important that the interview is capable of being conducted in any language that is spoken by a significant number of people in the survey population to avoid the danger of disenfranchisement. In the UK many government studies require questionnaire versions in Welsh, Urdu, Hindi and other languages, and in the USA a Spanish-language version will often be required.

The relevance of minority-language speakers to the study will natu­rally vary by the subject of the study and the degree of accuracy required in the data. For a study of housing conditions it is likely to be important that recently arrived immigrant communities are represented in the sample in their correct proportions. If the questionnaire is not available in a language that they understand, they will be effectively excluded and hence under-represented.

For many commercial studies, the issue of minority languages can be mostly ignored in many countries, although a Spanish version of the questionnaire is frequently necessary in the USA. This is because for most commercial studies the difference that a minority of non-majority- language-speaking consumers is likely to make to the findings is small, particularly in comparison to the variation caused by sampling error, non-response rates and even interviewer error.

Source: Brace Ian (2018), Questionnaire Design: How to Plan, Structure and Write Survey Material for Effective Market Research, Kogan Page; 4th edition.