The concept of validity of a research instrument

To examine the concept of validity, let us take a very simple example. Suppose you have designed a study to ascertain the health needs of a community. In doing so, you have devel­oped an interview schedule. Further suppose that most of the questions in the interview schedule relate to the attitude of the study population towards the health services being provided to them. Note that your aim was to find out about health needs but the interview schedule is finding out what attitudes respondents have to the health services; thus, the instrument is not measuring what it was designed to measure. The author has come across many similar examples among students and less skilled researchers.

In terms of measurement procedures, therefore, validity is the ability of an instrument to measure what it is designed to measure: ‘Validity is defined as the degree to which the researcher has measured what he has set out to measure’ (Smith 1991: 106). According to Kerlinger, ‘The commonest definition of validity is epitomised by the question: Are we meas­uring what we think we are measuring?’ (1973: 457). Babbie writes, ‘validity refers to the extent to which an empirical measure adequately reflects the real meaning of the concept under consideration’ (1989: 133). These definitions raise two key questions:

1. Who decides whether an instrument is measuring what it is supposed to measure?
2. How can it be established that an instrument is measuring what it is supposed to measure?

Obviously the answer to the first question is the person who designed the study, the readership of the report and experts in the field. The second question is extremely impor­tant. On what basis do you (as a researcher), a reader as a consumer or an expert make this judgement? In the social sciences there appear to be two approaches to establishing the validity of a research instrument. These approaches are based upon either logic that underpins the construction of the research tool or statistical evidence that is gathered using information generated through the use of the instrument. Establishing validity through logic implies justi­fication of each question in relation to the objectives of the study, whereas the statistical pro­cedures provide hard evidence by way of calculating the coefficient of correlations between the questions and the outcome variables.

Establishing a logical link between the questions and the objectives is both simple and dif­ficult. It is simple in the sense that you may find it easy to see a link for yourself, and difficult because your justification may lack the backing of experts and the statistical evidence to con­vince others. Establishing a logical link between questions and objectives is easier when the questions relate to tangible matters. For example, if you want to find out about age, income, height or weight, it is relatively easy to establish the validity of the questions, but to establish whether a set of questions is measuring, say, the effectiveness of a programme, the attitudes of a group of people towards an issue, or the extent of satisfaction of a group of consumers with the service provided by an organisation is more difficult. When a less tangible concept is involved, such as effectiveness, attitude or satisfaction, you need to ask several questions in order to cover different aspects of the concept and demonstrate that the questions asked are actually measuring it. Validity in such situations becomes more difficult to establish, and espe­cially in qualitative research where you are mostly exploring feelings, experiences, perceptions, motivations or stories.

It is important to remember that the concept of validity is pertinent only to a particular instrument and it is an ideal state that you as a researcher aim to achieve.

Source: Kumar Ranjit (2012), Research methodology: a step-by-step guide for beginners, SAGE Publications Ltd; Third edition.