The theory of causality and the research design

Now let’s turn to the second function of the research design — ensuring that the procedures undertaken are adequate to obtain valid, objective and accurate answers to the research questions. To ensure this, it is important that you select a study design that helps you to isolate, eliminate or quantify the effects of different sets of variable influencing the independent variable. To help explain this, we look at a few examples.

Suppose you want to find out the effectiveness of a marriage counselling service provided by an agency — that is, the extent to which the service has been able to resolve the marital problems of its clients. In studying such relationships you must understand that in real life there are many outside factors that can influence the outcome of your intervention. For example, during visits to your agency for counselling, your client may get a better job. If some of the marital problems came about because of economic hardship, and if the problem of money is now solved, it may be a factor in reducing the marital problems. On the other hand, if a client loses his/her job, the increase in the economic problems may either intensify or lessen the marital problems; that is, for some couples a perceived financial threat may increase marital problems, whereas, for others, it may create more closeness between partners. In some situations, an improvement in a mar­riage may have very little to do with the counselling received, coming about almost entirely because of a change in economic circumstances. Other events such as the birth of a child to a couple or a couple’s independent ‘self-realisation’, independently arrived at, may also affect the extent and nature of marital problems. Figure 7.1 lists other possible factors under the category of extraneous variables. This does not exhaust the list by any means.

Continuing the example of marriage and counselling, there are sets of factors that can affect the relationship between counselling and marriage problems, and each is a defined category of variables:

1. Counselling per se.
2. An the factors other than counselling that affect the marital problems.
3. The outcome – that is, the change or otherwise in the extent of the marital problems.
4. Sometimes, the variation in response to questions about marital problems can be accounted for by the mood of respondents or ambiguity in the questions. Some respondents may either overestimate or underestimate their marital problems because of their state of mind at the time. Or some respondents, in spite of being in exactly the same situation, may respond to non-specific or ambiguous questions differently, according to how they interpret the question.

As already explained in Chapter 5, any variable that is responsible for bringing about a change is called an independent variable. In this example, the counselling is an independent vari­able. When you study a cause-and-effect relationship, usually you study the impact of only one independent variable. Occasionally you may study the impact of two independent variables, or (very rarely) more than two, but these study designs are more complex.

For this example counselling was the assumed cause of change in the extent of marital problems; hence, the extent of marital problems is the dependent variable, as the change in the degree of marital problems was dependent upon counselling.

All other factors that affect the relationship between marital problems and counselling are called extraneous variables. In the social sciences, extraneous variables operate in every study and cannot be eliminated. However, they can be controlled to some extent. (Some of the methods
for controlling them are described later in this chapter.) Nevertheless, it is possible to find out the impact attributable to extraneous variables. This is done with the introduction of a con­trol group in the study design. The sole function of a control group is to quantify the impact of extraneous variables on the dependent variable(s).

Changes in the dependent variable, because of the respondent’s state of mood or ambiguity in the research instrument, are called random variables or chance variables. The error thus introduced is called the chance or random error. In most cases the net effect of chance variables is considered to be negligible as respondents who overreport tend to cancel out those who underreport. The same applies to responses to ambiguous questions in a research instrument.

Hence in any causal relationship, changes in the dependent variable may be attributed to three types of variable:

Let us take another example. Suppose you want to study the impact of different teaching models on the level of comprehension of students for which you adopt a comparative study design. In this study, the change in the level of comprehension, in addition to the teaching mod­els, can be attributed to a number of other factors, some of which are shown in Figure 7.2:

[change in level of comprehension] =

[change attributable to the teaching model] ±

[change attributable to extraneous variables] ±

[change attributable to chance variables]

In fact, in any study that attempts to establish a causal relationship, you will discover that there are three sets of variables operating to bring about a change in the dependent variable. This can be expressed as an equation:

[change in the outcome variable] =

[change because of the chance variable] ±

[change because of extraneous variables] ±

[change because of chance or random variables]

or in other words:

[change in the dependent variable] =

[change attributable to the independent variable] ±

[change attributable to extraneous variables] ±

[change attributable to chance variables]

or in technical terms:

[total variance] =

[variance attributable to the independent variable] ±

[variance attributable to extraneous variables] ±

[random or chance variance]

It can also be expressed graphically (Figure 7.3).

As the total change measures the combined effect of all three components it is difficult to isolate the individual impact of each of them (see Figure 7.3). Since your aim as a researcher is to determine the change that can be attributed to the independent variable, you need to design your study to ensure that the independent variable has the maximum opportunity to have its full effect on the dependent variable, while the effects that are attributed to extrane­ous and chance variables are minimised (if possible) or quantified or eliminated. This is what Kerlinger (1986: 286) calls the ‘maxmincon’ principle of variance.

One of the most important questions is: how do we minimise the effect attributable to extraneous and chance variables? The answer is that in most situations we cannot; however, it can be quantified. The sole purpose of having a control group, as mentioned earlier, is to measure the change that is a result of extraneous variables. The effect of chance variables is often assumed to be none or negligible. As discussed, chance varia­tion comes primarily from two sources: respondents and the research instrument. It is assumed that if some respondents affect the dependent variable positively, others will affect it negatively. For example, if some respondents are extremely positive in their atti­tude towards an issue, being very liberal or positively biased, there are bound to be others who are extremely negative (being very conservative or negatively biased). Hence, they tend to cancel each other out so the net effect is assumed to be zero. However, if in a study population most individuals are either negatively or positively biased, a systematic error in the findings will be introduced. Similarly, if a research instrument is not reliable (i.e. it is not measuring correctly what it is supposed to measure), a systematic bias may be introduced into the study.

In the physical sciences a researcher can control extraneous variables as experiments are usually done in a laboratory. By contrast, in the social sciences, the laboratory is society, over which the researcher lacks control. Since no researcher has control over extraneous vari­ables, their effect, as mentioned, in most situations cannot be minimised. The best option is to quantify their impact through the use of a control group, though the introduction of a control group creates the problem of ensuring that the extraneous variables have a similar effect on both control and experimental groups. In some situations their impact can be eliminated (this is possible only where one or two variables have a marked impact on the dependent variable). There are two methods used to ensure that extraneous variables have a similar effect on control and experimental groups and two methods for eliminating extraneous variables:

• Ensure that extraneous variables have a similar impact on control and experimental groups. It is assumed that if two groups are comparable, the extent to which the extraneous variables will affect the dependent variable will be similar in both groups. The following two methods ensure that the control and experimental groups are comparable with one another:
  • Randomisation – Ensures that the two groups are comparable with respect to the variable(s). It is assumed that if the groups are comparable, the extent to which extrane­ous variables are going to affect the dependent variable is the same in each group.
  • Matching – Another way of ensuring that the two groups are comparable so that the effect of extraneous variables will be the same in both groups (discussed in Chapter 8).
• Eliminate extraneous variable(s). Sometimes it is possible to eliminate the extraneous vari­able or to build it into the study design. This is usually done when there is strong evidence that the extraneous variable has a high correlation with the dependent variable, or when you want to isolate the impact of the extraneous variable. There are two methods used to achieve this:
  • Build the affecting variable into the design of the study – To explain this concept let us take an example. Suppose you want to study the impact of maternal health services on the infant mortality of a population. It can be assumed that the nutritional status of children also has a marked effect on infant mortality. To study the impact of maternal health serv­ices per se, you adopt a two-by-two factorial design as explained in Figure 7.4. In this way you can study the impact of the extraneous variables separately and interactively with the independent variable.
  • Eliminate the variable – To understand this, let us take another example. Suppose you want to study the impact of a health education programme on the attitudes towards, and beliefs about, the causation and treatment of a certain illness among non-indigenous Australians and indigenous Australians living in a particular community. As attitudes and beliefs vary markedly from culture to culture, studying non-indigenous Australians and indigenous Australians as one group will not provide an accurate picture. In such studies it is appropriate to eliminate the cultural variation in the study population by selecting and studying the populations separately or by constructing culture-specific cohorts at the time of analysis.

Source: Kumar Ranjit (2012), Research methodology: a step-by-step guide for beginners, SAGE Publications Ltd; Third edition.