Whose Meaning Is It? Validity and Reliability for In-depth, Phenomenological Interviewing

Whose meaning is it that an interview brings forth and that a re­searcher reports in a presentation, article, or book? That is not a simple question. Every aspect of the structure, process, and practice of inter­viewing can be directed toward the goal of minimizing the effect the interviewer and the interviewing situation have on how the participants reconstruct their experience. No matter how diligently we work to that effect, however, the fact is that interviewers are a part of the interview­ing picture. They ask questions, respond to the participant, and at times even share their own experiences. Moreover, interviewers work with the material, select from it, interpret, describe, and analyze it. Though they may be disciplined and dedicated to keeping the interviews as the par­ticipants’ meaning-making process, interviewers are also a part of that process (Ferrarotti, 1981; Kvale, 1996; Mishler, 1986).

The interaction between the data gatherers and the participants is inherent in the nature of interviewing. It is inherent, as well, in other qualitative approaches, such as participant observation. And I believe it is also inherent in most experimental and quasi-experimental method­ologies applied to human beings, despite the myriad and sophisticated measures developed to control for it (Campbell & Stanley, 1963).

One major difference, however, between qualitative and quantitative approaches is that in in-depth interviewing we recognize and affirm the role of the instrument, the human interviewer. Rather than decrying the fact that the instrument used to gather data affects this process, we say the human interviewer can be a marvelously smart, adaptable, flexible instrument who can respond to situations with skill, tact, and understand­ing (Lincoln & Guba, 1985, p. 107).

Although the interviewer can strive to have the meaning being made in the interview as much a function of the participant’s reconstruction and reflection as possible, the interviewer must nevertheless recognize that the meaning is, to some degree, a function of the participant’s inter­action with the interviewer. Only by recognizing that interaction and af­firming its possibilities can interviewers use their skills (see Chapter 6) to minimize the distortion (see Patton, 1989, p. 157) that can occur because of their role in the interview.

1. Is It Anybody’s Meaning?

How do we know that what the participant is telling us is true? And if it is true for this participant, is it true for anyone else? And if another per­son were doing the interview, would we get a different meaning? Or if we were to do the interview at a different time of year, would the participant reconstruct his or her experience differently? Or if we had picked dif­ferent participants to interview, would we get an entirely dissimilar and perhaps contradictory sense of the issue at hand? These are some of the questions underlying the issues of validity, reliability, and generalizability that researchers confront.

Many qualitative researchers disagree with the epistemological as­sumptions underlying the notion of validity. They argue for a new vocab­ulary and rhetoric with which to discuss validity and reliability (Mishler, 1986, pp. 108-110). Lincoln and Guba (1985), for example, substitute the notion of “trustworthiness” for that of validity. In a careful exposition they argue that qualitative researchers must inform what they do by concepts of “credibility,” “transferability,” “dependability,” and “confirmability” (pp. 289-332).

Others criticize the idea of objectivity that underlies notions of reli­ability and validity. Kvale (1996) sees the issue of validity as a question of the “quality of craftsmanship” of the researchers as they make defensible knowledge claims (pp. 241-244). Ferrarotti (1981) argues that the most profound knowledge can be gained only by the deepest intersubjectivity among researchers and that which they are researching. Such a discus­sion suggests that neither the vocabulary of “validity” nor “trustworthi­ness” is adequate.

Yet, in-depth interviewers can respond to the question, “Are the par­ticipant’s comments valid?” The three-interview structure incorporates features that enhance the accomplishment of validity. It places partici­pants’ comments in context. It encourages interviewing participants over the course of 1 to 3 weeks to account for idiosyncratic days and to check for the internal consistency of what they say. Furthermore, by interview­ing a number of participants, we can connect their experiences and check the comments of one participant against those of others. Finally, the goal of the process is to understand how our participants understand and make meaning of their experience. If the interview structure works to al­low them to make sense to themselves as well as to the interviewer, then it has gone a long way toward validity.

2. An Example of an Approach to Validity

One participant in our Secondary Teacher Education Program was a woman who had taught in parochial schools for a number of years but was not certified. She had enrolled in our program to get certified at the high school level in social studies. She agreed to be interviewed about her experience in our clinical site teacher education program.

The interviewer began her third interview with its basic question: “What does it mean to you to be a student teacher?” She responded:

Well, I guess-well, . . . [small laugh]-it kinda-it really kind of means that I’ve finally gotten down to actually trying to-I guess what it means is-[it] is the final passage into making a commitment to this, the profession, to teaching as-as a profession. What am I going to do with my life because I have all-all this time, going up and down and in and out of teaching. Should I or shouldn’t I? I was kind of stuck in that space where people say, you know, “Oh, those who can’t, teach. Those who can, do.” Just the whole negative status that teaching and education have. So it’s kind of fraught with that. And really resisting the fact that I had to student teach. I mean, I can remember [that] holding me back, what, 10 years ago, thinking, “Oh, no, I will actually have to be a student teacher some day,” and remembered what student teachers were like in my high school, and thinking, “Oh, I’ll never humiliate myself that way.” [small laugh] And so I guess it was the final-[pause]-biting the bullet to . . . making a commitment.

Is what she says valid? In the first interview she recounted how she had dropped out of college and taught in elementary grades in paro­chial schools because she needed money. In that interview she also talked about how she had dropped out of education courses because she didn’t think she was getting enough out of them; how she had switched to an academic field, but later realized that she really liked teaching.

The material in her third interview is internally consistent with the material in her first, which was given 2 weeks earlier. Internal consistency over a period of time leads one to trust that she is not lying to the inter­viewer. Furthermore, there is enough in the syntax, the pauses, the grop­ing for words, the self-effacing laughter, to make a reader believe that she is grappling seriously with the question of what student teaching means to her, and that what she is saying is true for her at the time she is saying it.

Moreover, in reading the transcript, we see that the interviewer has kept quiet, not interrupted her, not tried to redirect her thinking while she was developing it; so her thoughts seem to be hers and not the inter­viewer’s. These are her words, and they reflect her understanding of her experience at the time of her interview.

When I read this passage, I learned something both about this par­ticular student and about an aspect of the student-teaching experience that had not really been apparent to me. I began to think about aspects of the process we require prior to student teaching that enhance the need for students to make a commitment and about other aspects of our program that minimize that need. I began to wonder what the conditions are that encourage a person to make that commitment.

Finally, what the participant said about the status of education as a career and how that related to her personal indecision is consistent with what we know the literature says about the teaching profession and with what other participants in our study have said. I can relate this individual passage to a broader discourse on the issue.

The interview allowed me to get closer to understanding this student teacher’s experience than I would have been able to do by other methods such as questionnaires or observation. I cannot say that her understand­ing of student teaching as a commitment is valid for others, although passages in other interviews connect to what she has said. I can say that it seems valid for her at this point in her life. I cannot say that her un­derstanding of the meaningfulness of student teaching as a commitment she had heretofore not been willing to make will not change. Unlike the laws of physics, the rules governing human life and social interaction are always changing-except that we die. There is no solid, unmovable platform upon which to base our understanding of human affairs. They are in constant flux. Heisenberg’s principle of indeterminacy (Lincoln & Guba, 1985; Polanyi, 1958) speaks at least as directly to human affairs as it does to the world of physics.

The structure of the three interviews, the passage of time over which the interviews occur, the internal consistency and possible external con­sistency of the passages, the syntax, diction, and even nonverbal aspects of the passage, and the discovery and sense of learning that I get from read­ing the passage lead me to have confidence in its authenticity. Because we are concerned with the participant’s understanding of her experience, the authenticity of what she is saying makes it reasonable for me to have confidence in its validity for her.

3. Avoiding a Mechanistic Response

There is room in the universe for multiple approaches to validity. The problem is not in the multiplicity. Rather it lies in the sometimes doctrinaire ways some advocates of divergent approaches polarize the is­sue. (See Gage, 1989.) Those who advocate qualitative approaches are in danger of becoming as doctrinaire as those who once held the monopoly on educational research and advocated quantitative approaches.

On occasion I see dissertations in which doctoral candidates are as mechanical about establishing an “audit trail” or devising methods of “tri- angularization” (Lincoln & Guba, 1985, p. 283) as those in my genera­tion who dutifully devised procedures to confront “instrument decay” and “experimental mortality” (Campbell & Stanley, 1963, pp. 79, 182). What are needed are not formulaic approaches to enhancing either validity or trustworthiness but understanding of and respect for the issues that under­lie those terms. We must grapple with them, doing our best to increase our ways of knowing and of avoiding ignorance, realizing that our efforts are quite small in the larger scale of things. (For a “common sense” approach to validity, see Maxwell, 1996. For a “craftsmanship” approach to validity, see Kvale, 1996. For a highly personal view of validity, see Wolcott, 1994.)

Source: Seidman Irving (2006), Interviewing As Qualitative Research: A Guide for Researchers in Education And the Social Sciences, Teachers College Press; 3rd edition.