Collecting Primary Data for Quantitative Research

The most developed method of collecting primary data for quantitative research is the questionnaire. We take a particularly close look at mailed ques­tionnaires, as these are used very frequently in management research and necessitate particular techniques. We then present other methods of collecting primary data for quantitative research: observation and experimentation.

1. Surveys

A survey or questionnaire enables researchers directly to question individuals. It is a tool for collecting primary data that adapts well to quantitative research, as it allows the researcher to work with large samples and to establish statisti­cal relationships or numerical comparisons.

Collecting data by survey involves three major steps: initial crafting of the survey and choosing scales, pre-tests to check the validity and reliability of the survey, and then the actual administering of the final version. There are certain procedures that should be followed for each step, to obtain the maximum amount of relevant and usable data. Numerous articles and other published works have gone into lengthy detail about these procedures (Albreck and Settle, 1989; Fink and Kosecoff, 1998; Rossi et al., 1985; Schuman, 1996), but we have chosen here to focus on a number of fundamental points.

1.1. Choosing scales

To craft a survey for quantitative research is, in fact, to construct a measuring instrument. Before tackling the problems of wording the questions and organi­zing the structure of the questionnaire, the researcher must first choose which scales to use (the different types of scales – nominal, ordinal, interval or proportional – are presented in Chapter 4). Not only do researchers need to determine the type of scale to use, they also have to choose between using pre­existent scales or creating their own.

Using pre-existent scales Most questionnaires used in management research combine a number of scales. A researcher may choose to use scales already con­structed and validated by other researchers. These are generally published in the annex of the article or the work in which they are first employed, or they can be obtained by requesting them directly from the researcher who created them. A number of publications (Bearden et al., 1993; Robinson et al., 1991) present a large range of scales. Researcher should be aware, though, that the validity of these pre-existent scales is strongly linked to the context in which they are used. A scale designed to measure the degree of radicalness of an inno­vation in the biotechnology industry may not be able to be transposed to a study of the literary editing sector. Scales developed in a particular country (most often the USA) or socio-cultural context may need to be adapted for use in other contexts. When using scales in other contexts than that for which they were created, researchers must always verify that they are indeed valid in the new context.

Constructing new scales If appropriate scales cannot be found, researchers have to construct their own measuring instruments. Detailed interviews are a good way to obtain a better picture of the phenomenon under study, and they enable the researcher to define coherent items that will be understood by the population being studied. This should then be followed by an initial pre-test phase, to refine the list of questions and to validate the scale.

For a complete description of all the steps to be followed to create new scales, the researcher can refer to detailed works such as Aaker and Day (1990), Devellis (1991) or Tull and Hawkins (1987).

1.2. Designing and pre-testing a questionnaire

Designing the questionnaire Preparing the questions is complex work. The researcher needs to avoid errors in the formulation and organization of the questions as well as in the choice of possible responses. Specialized works on the design of questionnaires make quite precise recommendations (Converse and Presser, 1986; Fink, 1995; Rossi et al., 1985; Schuman, 1996). Here we will simply summarize several fundamental points.

A questionnaire (or survey) generally begins with relatively simple and closed questions. It is preferable to group together questions that are more involved, complex or open at the end of the document. The questions should, as far as possible, follow a logical order that uses thematic groupings and facili­tates the passage from one theme to another. There are two common sources of error that should be avoided when formulating and deciding the order of the questions. The halo effect results from associating a series of successive ques­tions that are too similar to each other. This can occur, for example, when a long series of questions uses the same scale for all modes of response. To avoid the halo effect, the researcher can introduce a change in the form of the questions, or propose an open question. The contamination effect occurs when one ques­tion influences the subsequent question or questions. To guard against this bias, one needs to be scrupulously careful about the order of the questions. The unit of analysis (the industrial sector, the organization, a product line, a particular department …) that a question or a series of questions relates to must always be made clear, and any change of unit should be systematically acknowledged. When the survey includes questions relating to different subjects, it is useful to draw the respondent’s attention to this by introducing the new subject with a short phrase separating the groups of questions.

The pre-test Once the researcher has prepared a first draft of the question­naire, he or she will need to carry out a pre-test – to test the form of the ques­tions and their order, to ensure that respondents understand the questions and to assess whether the proposed modes of reply are relevant (Hunt et al., 1982). Ideally, the draft should be given to several respondents in face-to-face inter­views, so that their non-verbal reactions can be noted. Following this, it is recom­mended that researchers carry out a pre-test using the same method they propose to use for the definitive questionnaire, and under the same conditions of interaction (or non-interaction) with the respondents. Data collected during the pre-test(s) also allows researchers to measure the internal validity of their scales. Through this phase the list of items can be refined so that only those which really measure the phenomenon being studied are retained. By the end of the pre-test phase, the questionnaire should be pertinent, efficient and clear for the researcher as well as for the respondents.

1.3. Administering a questionnaire

There are several ways of administering the questionnaire. It can be adminis­tered electronically – via e-mail, Internet or intranet – interviews can be con­ducted face to face or by video and telephone, or it can be sent out by mail. While each of these methods has a number of specific considerations, it must be remembered that administering the questionnaire harbors its own particular difficulties in each individual case, and always calls for prudence. A danger faced by all researchers is the risk of a weak response rate, which can call the whole research project into question. The issue here is the problem of managing data sources in the context of a survey conducted by questionnaire. Different researchers specializing in crafting questionnaires have proposed administra­tion methods that enable researchers to obtain high response rates (Childers and Skinner, 1979; Dillman, 1978; Linsky, 1975; Yammarino et al., 1991) and we present the most elementary points here. Researchers need to be prepared to adapt these techniques to suit the socio-cultural context of their research and the means at their disposal. Finally, different techniques are used depending on whether the survey is administered by mail, in face to face interviews, by tele­phone or using information technology.

Administering a mailed questionnaire Mailed questionnaires are somewhat particular in that they are auto-administered by the respondents. Given the importance of motivating correspondents to complete a mailed questionnaire, great care must be taken over the document’s general presentation. The ques­tionnaire should be printed on white paper,1 and should be in booklet form. There should be no questions on the first page – this page is often reserved for the title of the study and recommendations for the respondent. An illustra­tion may also be used; sufficiently neutral to avoid the questionnaire being turned into an advertising brochure. It is also preferable to leave the final page free of questions. This page is reserved for the respondent’s comments. To date, researchers have been unable to agree on the ideal length of a mailed questionnaire. Logically, subjects are more reticent about replying to a lengthy questionnaire, which requires more of their time. Certain specialists say ques­tionnaires should not exceed ten pages, while others say they should be limited to four pages.

It is generally recommended to send an accompanying letter with a mailed questionnaire.

Once the questionnaire and accompanying letter have been drawn up, there is the auto-administration of the mailed questionnaire to think about. The fact that the subjects solicited are not in direct contact with the researcher leads to certain difficulties (Bourque and Fielder, 1995). It is impossible to be insistent, to rely on one’s physical presence to help lessen a subject’s reticence. One needs to alleviate these difficulties by developing other forms of contact or conduct. In the following we present some techniques for optimizing auto-administration of a mailed questionnaire.

Administering a questionnaire face to face This procedure allows the researcher to reply directly to any queries respondents may have about the actual nature of the questions. It also makes it easier to check that the sample is representative. The main limitation is that the researcher must always guard against expressing any opinion or sign of approval, or disapproval, at the risk of influencing the respondent. Moreover, this method blocks the responses of certain people who consider them too personal to be expressed face to face. When using this technique to administer a questionnaire, researchers still need to present the study clearly to the respondents, and involve them in its aims. The researcher should have a suitable text prepared to present the question­naire, confronting the same issues as those discussed above for a letter accom­panying a mailed questionnaire.

Administering a questionnaire by telephone It is pointless to pretend that a respondent is anonymous during a telephone conversation. Yet when telephone interviews are used to complete a questionnaire, the researcher is confronted with the dilemma of choosing between personalizing the relationship and maintaining the subject’s anonymity as much as is possible. One compromise solution can be to personalize the conversation while guaranteeing to respect the subject’s anonymity. As with the preceding techniques, the researcher must begin the interaction by explaining the aims of the research and the contribu­tion it will make. Preliminary contact by mail enables the researcher to prepare potential respondents and to explain their particular importance to the study. This technique avoids the element of surprise and lessens the negative reaction that the disturbance of a telephone call can so often provoke.

Administering a questionnaire using information technology There are two pos­sible ways information technology can be used to administer a questionnaire. The researcher may send out a file, a diskette, or a CD-ROM containing a pro­gram the respondents can load into their computers so as to reply directly to the questionnaire. The respondent then sends back a completed file. It is equally possible to ask respondents to connect to a web site – a link can be included in an e-mail – where they will find the questionnaire and can reply to it directly. These two methods have the advantage of freeing the researcher from the unappetizing tasks of envelope-stuffing and mailing, and inputting replies. It can provide data that can be directly used for statistical analyses. Dillman (1999) has gone into lengthy detail on the procedures to follow in his book.

1.4. Advantages and limitations of questionnaires

The questionnaire seems to be one of the most efficient ways of collecting pri­mary data. It also offers the possibility of standardizing and comparing scales, and enables the anonymity of the data sources to be preserved. Nevertheless, data collection by questionnaire has certain limitations. It is not flexible. Once the administration phase is under way, it is no longer possible to backtrack. The researcher can no longer offset a lack of sufficient data or an error in the scale used. Furthermore, standardization of the measurement instrument has a downside: the data gathered using standardized methods is necessarily very perfunctory. Collecting data by questionnaire also exposes the researcher to the bias of the person making the statements. There is a commonly cited difference between declaratory measurements and behavioral measurements.

Some of the advantages and disadvantages inherent to the different methods of administering questionnaires are presented in Table 9.1.

2. Other Ways to Collect Data

There are other ways of collecting primary data for quantitative use. These are principally observation procedures and experimental methods.

2.1. Observation

As noted by Silverman (1993), observation is not a collection method that is used very often in quantitative research. It is difficult to observe large samples, and to obtain a sample that is statistically large enough can require mobilizing several observers. This can entail another problem, that of the reliability of the measurement – as there is a risk that the observations will not be homogeneous. When using this collection method the researcher needs to develop and vali­date a standard (or systematic) observation framework from which to uni­formly describe the types of behavior observed (Bouchard, 1976).

Taking account of the rigidity of such a system, the researcher will need to guard against possible errors of content (resulting from simplification of the observation) or context (inherent to the link between data and situations) and against instrumental bias (due to the researcher’s judgements and assump­tions) (Weick, 1968).

2.2. Experimental methods

Certain experimental methods enable us to draw out quantitative results and to make statistical use of the data collected. The quality of such experimenta­tion rests above all on creating optimum conditions for the participants (behav­ior, willingness to participate, environment, etc.). Participants should never feel obliged to adopt special behavior to suit the experimentation situation. The researcher’s job, therefore, is to create conditions that encourage participants to behave as naturally as possible. There are a number of different methods that can be employed to conduct the experimentation. The researcher can use the protocol method, where subjects are invited to reconstruct and describe ‘out loud’ their internal method of processing information when they need to make a decision. Another experimental method involves the subjects taking part in role-playing.

These experimental methods offer a wealth of information for the researcher. The variables are measurable and can be controlled, and the researcher can establish comparisons and test causal relationships between events. However, experimental methods are sometimes too simplistic and can be limited in terms of external validity. The results should always be analyzed with care, as they often give only limited scope for generalization.

Source: Thietart Raymond-Alain et al. (2001), Doing Management Research: A Comprehensive Guide, SAGE Publications Ltd; 1 edition.

Leave a Reply

Your email address will not be published. Required fields are marked *