Planning the Research

The design stage often results in a proposal stating the approach the researcher will follow and the essential methodological choices he or she has made (the data collection and analysis methods he or she will use, and the observational field). These choices should always be justified in line with the research question. Although such formalization is not essential at this stage, it will be required when it comes to publishing research results. It is important to keep in mind, however, that the structure of the completed research will be evaluated, not that of the initial design. The structure conceived during the design stage can evolve during the research, as opportunities or difficulties are encountered along the way.

Notwithstanding, it is definitely not recommended that researchers omit the design stage. The design guides the course of the research and helps avoid at least some of the obstacles that can crop up in latter stages of the research. Researchers can come up against an impasse, or at least a snarl, at late stages of their research that stem from preceding stages – in which case they can be very difficult to resolve (Selltiz et al., 1976). When such problems are serious and appear late in the piece, they can waste time and demand further effort that might have been avoided. They can even appear insurmountable and halt the research entirely. For example, when a researcher, after collecting data through 300 completed questionnaires, realizes they were not given to the right people in the first place, data collection will probably have to start all over again. More forethought at the design stage would probably have made it possible to avoid this type of incident. Many problems can stem from a defective or unsatisfactory research design. Experimental results can in some cases be unusable because of the omission of a control variable. Often, the only solution is to start the experi­ment again. A thorough knowledge of the literature or the research field may help to avoid such an oversight. Statistical results may not be significant if the sample used is too small. At best the researcher will be able to increase the size of the sample, but if that is not possible, the research may have to be redirected towards another application. A good preliminary knowledge of the selected statistical method’s application conditions generally makes it possible to deter­mine the number of observations that will be required. A detailed design, pre­senting the general procedure and justifying the research components selected, will help to avoid this type of error. Planning the research and putting together an initial design should definitely be seen as a distinct and important stage of the research.

1. When to Construct a Research Design

The design stage usually comes after defining the research question and before beginning data collection (see Figure 6.2). Constructing a research design con­sists of defining the means necessary to answer the research question: selecting data collection and analysis methods and determining data types and sources (including details of sample composition and size). This is why it is useful to define the research question beforehand – even if the formulation is still a little unclear, one should at least know what one wishes to study. Of course, within the framework of constructivist or inductive approaches, the research question cannot be specified during the design stage. In this case, only the topic is defined during the development of the design: the research question is built up progressively as data is collected and analyzed (see Chapter 2).

Moreover, we recommend establishing a coherent research design before moving on to the data collection. To start collecting data without knowing how it is to be analysed entails the risk of it being inapplicable. This can also lead to the researcher wasting an observation field that can be difficult to replace, espe­cially if the phenomena under study are sensitive or out of the ordinary.

Within the framework of a doctoral dissertation, construction of a research design begins only after some months of reading and exploratory work – through which a research question is formulated. Depending on the approach chosen, these activities may take on very different forms. A researcher might carry out an exhaustive literature review as part of a hypothetico-deductive approach – leading to a conceptual framework. But a more cursory reading involving a small number of works only might be more applicable to an inductive approach. Whichever approach is chosen, constructing a research design is generally a lengthy process. Simple logical deduction is rarely enough on its own. Instead, a process of trial and error is entered upon, which continues until the researcher is satisfied that he or she has come up with a complete, coherent, and potentially

feasible design. This process often requires new reading, in particular on the general approach chosen, but also on data analysis methods and data collection and sampling techniques. Similarly, further exploration is often required, in parti­cular to find an accessible observation field and to evaluate the feasibility of the intended data collection process. For case studies, for example, a researcher might at this stage make preliminary contact with a manager of the organization chosen: not simply to gain access to the observation field, but also to specify which sources of information are available and authorized. It is also a good time to ensure that the chosen means of data collection is acceptable to all concerned.

Constructing a design is also a way to improve the precision or the formu­lation both of the research question and of the theoretical references (Grunow, 1995). Putting the research into a practical perspective makes it easier to estimate its feasibility. For example it can lead to a researcher restricting or limiting the research question if it appears too broad to be handled in its entirety. By ques­tioning methodological choices and the types of results that might stem from them, researchers can often identify inaccuracies, even omissions, in their design concept. Such revelations can return researchers to the literature, to supplement theoretical references that have proved unsatisfactory. Consequently, research design construction is an ongoing process (see Figure 6.2). It will require more or less time depending on the approach chosen, on the level of preliminary methodological knowledge, and on the difficulties encountered by the researcher in finding an appropriate field. The precision and detail of this initial design will depend on the strictness of the research approach. For example, a research design in which the researcher intends to build-up an interpretation of a pheno­menon via an in-depth case study could be limited to an outline comprising the research topic, the general approach, the choice of field and the generic data collection and analysis methods that will be used. By definition, this approach leaves great scope for flexibility, allowing new elements to emerge. But a project in which the validity of the results is closely related to the precision and control of the system will require a far more detailed design.

2. How to Construct a Research Design

Morse (1994) suggests using the normal procedure backwards, that is, to start by imagining what one might find. Imagining the expected or even the desired result often makes it possible to refine a research question and to determine appropri­ate research methods. Similarly, it is preferable to select analysis techniques before defining the data collection process in detail, as each technique has its restrictions in terms of the type of data required as well as the appropriate mode of collection.

2.1. Data analysis

A great number of data analysis methods are available, both quantitative and qualitative. Each has its own goals (comparing, structuring, classifying, describing, etc.) and sheds light on different aspects of the problem in question. The choice of analysis method depends on the research question and the type of result desired. As we have already indicated, no method is superior to another in absolute terms. The complexity of the analysis is not a guarantee of a better quality of research. Indeed, complex analysis methods are not necessarily the best adapted to the research question. Daft (1995) warns researchers that statistics are no substitute for clearly defining concepts, and that highly sophisticated statistical processing can distance the data from reality to such a point that the results become difficult to interpret.

Each analysis method depends on hypotheses that limit its conditions of use. Each one brings its own restrictions concerning the nature of the data, the number of observations necessary, or the probability distribution of these obser­vations. Choosing an analysis method requires a perfect understanding of its conditions, so that the researcher can detect in advance any factors that could make it unusable within the framework of his or her research. It is advantageous to consider various methods before making a definitive choice. Not only can this help to identify a more appropriate method than the one initially envisaged, but it is also a good way to clarify different methods’ conditions of use and to understand their limits.

By identifying a method’s limitations beforehand, researchers can consider using a second, complementary, method from the outset, to compensate for any deficiencies of the primary method and to reinforce research results. In this case, researchers must always ensure the two methods are compatible, and take account of the particular requirements of each one.

Analysis methods are not limited to those traditionally used in management research. It is more than possible to use a technique borrowed from another dis­cipline. A new method can generate new knowledge, or extend knowledge to a wider field. However, importing methods is by no means easy (Bartunek et al., 1993). The researcher has to evaluate whether the method is adaptable to the new context: this requires a detailed understanding of its limits and underlying assumptions.

2.2. Data collection

Data collection can be divided into four principal components: the type of data collected; the data collection method used; the nature of both the observation field and the sample; and data sources. Each of these components must be appropriate to the research question and the data analysis method selected – that is, research question, data collection and data analysis must be coherent. The feasibility of these choices must also be taken into account.

Identifying the data needed to investigate a research question presupposes that the researcher has a good idea about what theories are likely to explain the phenomenon. This seems obvious for research destined to test hypothe­ses using data collected through questionnaires, but can also be useful when a researcher uses an inductive procedure to explore a phenomenon. Yin (1990) argues that encouraging a researcher to begin collecting data too early in a case study is the worst piece of advice one can give. Even for exploratory research, the relevance of the data, just like the choice of participants or of observation sites, depends to an extent on researchers’ prior familiarity with the pheno­mena they study. However, the other extreme is to refrain from starting data collection on the pretext that doubts remain. Since the main contribution of exploratory study is in gaining new insights, this presupposes that the existing literature cannot explain everything beforehand.

The data collection method used should enable the researcher to collect all information needed to answer the research question. Once again, many methods are possible: close-ended questionnaires, observation, verbal protocols, unstruc­tured interviews, etc. Some are better adapted than others to the collection of a given type of information, and all of them have their limitations. An inadequate data collection method can also invalidate the whole research. For example, a closed-mail questionnaire using a random sample of managers is unsuitable for research that proposes to study a subtle and intangible decision-making process (Daft, 1995). Sometimes several data collection methods can be used to increase the validity of the data. For example, ex post interviews are likely to prove insuf­ficient to reconstruct a chronological path of actions because of divergence in participants’ recollections. In this case, documents might be collected to sup­plement the data gathered from interviews or to validate it, according to the principle of triangulation.

Researchers should always ensure that the observational field can answer the research question. The size and the composition of the sample should also be determined during the design construction phase. It is useful to verify that the sample size is sufficient to implement the analysis method. At this stage, too, the researcher should define the structure of the sample (or samples) as this will affect the validity of research.

Data sources should also be taken into account when constructing a research design. In survey research, answers can differ greatly according to the respon­dent. There are particular biases associated with the hierarchical or functional position of respondents. For example, in a study of the characteristics of management control systems, a questionnaire sent to the heads of a large company might provide information about the company’s formal structure. This source, though, would probably be insufficient if the researcher wanted to know how managers used this structure, and whether they were satisfied with it. Researchers should always ensure their respondents can provide the infor­mation they are seeking. Similarly, when using secondary data, researchers should question its adequacy. Identical wording can hide different realities, according to who constructed the data and how it was collected. In the case of data collected over a period of time, the researcher could also question whether the definition and the data collection method did not change as time went on. For example, an apparent drop in a company’s workforce may reflect a down­sizing or a change in the definition of workforce, which may no longer include certain workers such as temporary labor.

Contrary to other research components, choices concerning data collection are not dictated solely by a need for coherence. Data collection often raises practical problems that result in revising the ideal guidelines the researcher had already established. Any research design is consequently a compromise between theoretical and practical considerations (Suchman, in Miller, 1991). It is therefore recommended at this stage to take account of the feasibility of the design in addition to its coherence.

Researchers should ensure the duration of the data collection phase is reasonable, and that they have enough funds to complete it. If, for example, a research question requires 40 case studies, the researcher should plan for a long period of investigation. Researchers can use assistants but, if funds are not sufficient, it would be better to revise the research design, or to reduce the scope of the research question. Similarly, in researching cultural differences, translation or travelling costs can be prohibitory, and lead the researcher to limit the number of countries under study.

Many other feasibility difficulties also exist. For example, the administra­tion of a questionnaire in an organization often requires obtaining permission (Selltiz et al., 1976). Very often, a study of an ongoing product development requires a contract of confidentiality with the company. Researchers should always evaluate whether the observational field is accessible, and estimate the consequences of possible constraints imposed by the field on the research. They should also decide whether the data collection system will be tolerated by the subjects under observation. For example, it is not easy to convince a busy execu­tive to agree to be closely observed for an entire day to record the time given over to various activities (reading of strategic reports, meetings, telephone calls, etc.). Similarly, executives do not necessarily agree to filling out daily questionnaires detailing all the people they met in the course of their work. In general, access to the field is easier when the members of the organization are interested in the results.

In an attempt to anticipate these various feasibility problems, Selltiz et al. (1976) strongly advise meeting other researchers who have worked in similar or close fields, to question them on the problems they may have faced, or even the pleasant surprises they had. Another suggestion is to explore the field before­hand, which could help to identify and resolve certain difficulties.

2.3. Expected results

The reiterative and recursive process involved in constructing a coherent design can very easily drift off course. It can produce a design in which the three elements (data collection, data processing and the expected result) may be perfectly coherent among themselves, whereas the expected result no longer answers the research question formulated at the outset. At this stage it is use­ful to evaluate once again whether the result will be coherent with the original research question. If they diverge, it may be more profitable to reformulate the question, or even to return to the stage of reviewing the literature, rather than recommencing to construct a new design to respond to the initial question. Modifying the research question at this stage does not cast doubts on the hypothetico-deductive principles of hypothesis testing, as data collection has not yet been carried out.

Researchers can also, at this stage, reassess the contribution they expect their work to make to their research domain. When hypothetico-deductive approaches are used, the answer to this question is known as soon as the research question is defined. Nevertheless, since the development of the design can lead to the ques­tion being reduced or altered, it may still be useful to question the expected contribution. It would obviously be a pity to realize at the end of the research that the results add little or nothing to the existing knowledge in the domain (Selltiz et al., 1976).

As researchers have many data collection and analysis methods at their disposal, a wide variety of designs are open to them – from those that follow a standard approach to far more complex solutions. It is therefore up to the individual researcher to put together a research design that can effectively answer the question he or she has chosen to consider.

Source: Thietart Raymond-Alain et al. (2001), Doing Management Research: A Comprehensive Guide, SAGE Publications Ltd; 1 edition.