Questionnaire in Tracking studies

Consistency of question wording is important in ongoing or tracking studies, in order to ensure that changes in data over time are not due to wording changes.

To ensure data consistency, it is also important to maintain the order in which the questions are asked, so that any order bias that exists is itself consistent. Keeping the question order means that adding new questions can cause problems, and the positioning of them must be considered very carefully. If possible, new questions should be added to the end of the questionnaire so as not to affect responses to any of the earlier questions. For the sake of the interview flow, though, this is not always possible.

For example, in an ongoing customer satisfaction survey, respondents were asked to give a rating of their overall satisfaction with the service received on their most recent visit to the client company. This has then been followed with questions rating various staff and service attributes, including one on efficiency. After a while, a competitor introduces a guar­antee that all transactions will be completed within 10 minutes or customers get their money back. To measure the impact of this, the client now asks that, on the next wave of the survey, a new question is inserted between the overall satisfaction question and the service attribute ratings, on how quickly the customers perceive their transaction to have been handled and how satisfied they were with that. The introduction of these questions at this point could influence the way in which respondents rate the individual service attributes, in particular the one relating to effi­-ciency, as the speed of transaction has been raised higher in their consciousness than in previous waves of the study. Researchers must alert the client to the potential impact of such a change in the question­naire on the comparability of the data with previous waves, and endeav­our to find an alternative solution, such as a less sensitive position.

If no alternative solution can be found and the question changes are to be included for the foreseeable future, then it may be worth considering having a split run for one wave. For this, the sample is split randomly into two. One half is asked the existing questionnaire, the other the new questionnaire with the changes incorporated. Differences in results on the affected questions between the two halves of the sample can then be attributed to the changed questionnaire. An assessment of the impact of the changes can thus be made.

Source: Brace Ian (2018), Questionnaire Design: How to Plan, Structure and Write Survey Material for Effective Market Research, Kogan Page; 4th edition.