Six Sources of Case Study Evidence

The sources of evidence discussed here are the ones most commonly used in doing case studies: documentation, archival records, interviews, direct obser­vations, participant-observation, and physical artifacts. However, you should be aware that a complete list of sources can be quite extensive—including films, photographs, and videotapes; projective techniques and psychological testing; proxemics; kinesics; “street” ethnography; and life histories (Marshall & Rossman, 1989).

A useful overview of the six major sources considers their comparative strengths and weaknesses (see Figure 4.1). You should immediately note that no single source has a complete advantage over all the others. In fact, the var­ious sources are highly complementary, and a good case study will therefore want to use as many sources as possible (see the later discussion in this chapter on “multiple sources of evidence”).

1. Documentation

Except for studies of preliterate societies, documentary information is likely to be relevant to every case study topic.¹ This type of information can take many forms and should be the object of explicit data collection plans. For instance, consider the following variety of documents:

• letters, memoranda, e-mail correspondence, and other personal documents, such as diaries, calendars, and notes;
• agendas, announcements and minutes of meetings, and other written reports of events;
• administrative documents—proposals, progress reports, and other internal records;
• formal studies or evaluations of the same “case” that you are studying; and
• news clippings and other articles appearing in the mass media or in community newspapers.

These and other types of documents all are increasingly available through Internet searches. The documents are useful even though they are not always accurate and may not be lacking in bias. In fact, documents must be carefully used and should not be accepted as literal recordings of events that have taken place. Few people realize, for instance, that even the “verbatim” transcripts of official U.S. Congress hearings have been deliberately edited—by the con­gressional staff and others who may have testified—before being printed in final form. In another field, historians working with primary documents also must be concerned with the validity of a document.

For case studies, the most important use of documents is to corroborate and augment evidence from other sources. First, documents are helpful in verify­ing the correct spellings and titles or names of organizations that might have been mentioned in an interview. Second, documents can provide other specific details to corroborate information from other sources. If the documentary evi­dence is contradictory rather than corroboratory, you need to pursue the prob­lem by inquiring further into the topic. Third, you can make inferences from documents—for example, by observing the distribution list for a specific doc­ument, you may find new questions about communications and networking within an organization. However, you should treat inferences only as clues worthy of further investigation rather than as definitive findings because the inferences could later turn out to be false leads.

Because of their overall value, documents play an explicit role in any data collection in doing case studies. Systematic searches for relevant documents are important in any data collection plan. For example, prior to field visits, an Internet search can produce invaluable information. During field visits, you should allot time for using local libraries and other reference centers whose documents, such as back issues of periodicals, may not be available electron­ically. You should also arrange access to examine the files of any organizations being studied, including a review of documents that may have been put into cold storage. The scheduling of such retrieval activities is usually a flexible matter, independent of other data collection activities, and the search can usually be conducted at your convenience. For this reason, there is little excuse for omitting a thorough review of documentary evidence. Among such evi­dence, news accounts are excellent sources for covering certain topics, such as the two in BOXES 16 and 17.

BOX 16

Combining Personal Participation with
Extensive Newspaper Documentation

Improving educational conditions—especially for urban schools in the United States—has become one of the biggest challenges for the 21st century. How the Houston, Texas, system dealt with constrained fiscal resources, diverse student pop­ulations, and local political constituencies is the topic of an exciting and riveting case study by Donald McAdams (2000). McAdams benefits from having been a member of the system’s school board for three elected, 4-year terms. He writes as a storyteller, not a social science analyst. At the same time, the book contains numer­ous references to local news articles to corroborate events. The result is one of the most readable but also well-documented case studies that readers will encounter.

BOX 17

Comparing Evidence from Two Archival Sources to
Cover the Same Community Events

One of the most inflammatory community events in the 1990s came to be known as the “Rodney King crisis.” White police officers were serendipitously videotaped in the act of beating an African American man, but a year later, they all were acquitted of any wrongdoing. The acquittal sparked a major civil disturbance, in which 58 people were killed, 2,000 injured, and 11,000 arrested.

A case study of this crisis deliberately drew from two different newspapers—the major daily for the metropolitan area and the most significant newspaper for the area’s African American community (R. N. Jacobs, 1996). For the pertinent period surrounding the crisis, the first newspaper produced 357 articles and the second (a weekly, not daily, publication) 137 articles. The case study traces the course of events and shows how the two papers constructed different understandings of the crisis, illustrating the potential biases of documentary evidence and the need to address such biases.

At the same time, many people have been critical of the potential overre­liance on documents in case study research. This is probably because the casual investigator may mistakenly assume that all kinds of documents— including proposals for projects or programs—contain the unmitigated truth. In fact, important in reviewing any document is to understand that it was writ­ten for some specific purpose and some specific audience other than those of the case study being done. In this sense, the case study investigator is a vicar­ious observer, and the documentary evidence reflects a communication among other parties attempting to achieve some other objectives. By constantly trying to identify these objectives, you are less likely to be misled by documentary evidence and more likely to be correctly critical in interpreting the contents of such evidence.²

A newer problem has arisen because of the abundance of materials available through Internet searches. You may get lost in reviewing such materials and actually waste a lot of time on them. Note, however, that the problem is not that different from having an overabundance of numeric data about your case, as might be available from sources such as the U.S. census (also see discussion of archival records, next) if you were doing a neighborhood study. In both situa­tions, you need to have a strong sense of your case study inquiry and focus on the most pertinent information. One suggestion is to sort or triage the materials (documents or numeric data) by their apparent centrality to your inquiry. Then, spend more time reading or reviewing what appears central, and leave aside other, less important materials for later reading or review. The procedure will not be perfect, but it will permit you to keep moving to other case study tasks.

2. Archival Records

For many case studies, archival records—often taking the form of computer files and records as in the U.S. census data just mentioned—also may be rele­vant. Examples of archival records include

• “public use files” such as the U.S. census and other statistical data made available by federal, state, and local governments;
• service records, such as those showing the number of clients served over a given period of time;
• organizational records, such as budget or personnel records;
• maps and charts of the geographical characteristics of a place; and
• survey data, such as data previously collected about a site’s employees, residents, or participants.

These and other archival records can be used in conjunction with other sources of information in producing a case study. However, unlike documen­tary evidence, the usefulness of these archival records will vary from case study to case study. For some studies, the records can be so important that they can become the object of extensive retrieval and quantitative analysis (for example, see a multiple-case study of 20 universities, in Yin, 2003, chap. 9). In other studies, they may be of only passing relevance.

When archival evidence has been deemed relevant, an investigator must be careful to ascertain the conditions under which it was produced as well as its accuracy. Sometimes, the archival records can be highly quantitative, but numbers alone should not automatically be considered a sign of accuracy. Nearly every social scientist, for instance, is aware of the pitfalls of using the FBI’s Uniform Crime Reports—or any other archival records based on crimes reported by law enforcement agencies. The same general word of caution made earlier with documentary evidence therefore also applies to archival evidence: Most archival records were produced for a specific purpose and a specific audi­ence other than the case study investigation, and these conditions must be fully appreciated in interpreting the usefulness and accuracy of the records.

3. Interviews

One of the most important sources of case study information is the inter­view. Such an observation may be surprising because of the usual association between interviews and the survey method. However, interviews also are essential sources of case study information. The interviews will be guided conversations rather than structured queries. In other words, although you will be pursuing a consistent line of inquiry, your actual stream of questions in a case study interview is likely to be fluid rather than rigid (H. J. Rubin & Rubin, 1995).

Note that this means that, throughout the interview process, you have two jobs: (a) to follow your own line of inquiry, as reflected by your case study protocol, and (b) to ask your actual (conversational) questions in an unbiased manner that also serves the needs of your line of inquiry (see distinction between “Level 1” and “Level 2” questions in Chapter 3). For instance, you may want (in your line of inquiry) to know “why” a particular process occurred as it did. Becker (1998, pp. 58-60), however, has pointed to the important difference in actually posing a “why” question to an informant (which, in his view, creates defensiveness on the informant’s part) in contrast to posing a “how” question—the latter in fact being his preferred way of addressing any “why” question in an actual conversation. Thus, case study interviews require you to operate on two levels at the same time: satisfying the needs of your line of inquiry (Level 2 questions) while simultaneously putting forth “friendly” and “nonthreatening” questions in your open-ended inter­views (Level 1 questions).

One type of case study interview is an in-depth interview. You can ask key respondents about the facts of a matter as well as their opinions about events. In some situations, you may even ask the interviewee to propose her or his own insights into certain occurrences and may use such propositions as the basis for further inquiry. The “interview” may therefore take place over an extended period of time, not just a single sitting. The interviewee also can suggest other persons for you to interview, as well as other sources of evidence.

The more that an interviewee assists in this manner, the more that the role may be considered one of an “informant” rather than a respondent. Key infor­mants are often critical to the success of a case study. Such persons provide the case study investigator with insights into a matter and also can initiate access to corroboratory or contrary sources of evidence. Such a person, named “Doc,” played an essential role in the conduct of the famous case study presented in Street Comer Society (Whyte, 1943/1955; also see BOX 2A, Chapter 1, p. 7). Similar key informants have been noted in other case studies. Of course, you need to be cautious about becoming overly dependent on a key informant, especially because of the interpersonal influence—frequently subtle—that the informant may have over you. A reasonable way of dealing with this pitfall again is to rely on other sources of evidence to corroborate any insight by such infor­mants and to search for contrary evidence as carefully as possible.

A second type of case study interview is a focused interview (Merton, Fiske, & Kendall, 1990), in which a person is interviewed for a short period of time— an hour, for example. In such cases, the interviews may still remain open- ended and assume a conversational manner, but you are more likely to be following a certain set of questions derived from the case study protocol.

For example, a major purpose of such an interview might simply be to cor­roborate certain facts that you already think have been established (but not to ask about other topics of a broader, open-ended nature). In this situation, the specific questions must be carefully worded, so that you appear genuinely naive about the topic and allow the interviewee to provide a fresh commentary about it; in contrast, if you ask leading questions, the corroboratory purpose of the interview will not have been served. Even so, you need to exercise caution when different interviewees appear to be echoing the same thoughts—corrob­orating each other but in a conspiratorial way.³ Further probing is needed. One way is to test the sequence of events by deliberately checking with persons known to hold different perspectives. If one of the interviewees fails to com­ment, even though the others tend to corroborate one another’s versions of what took place, the good case study investigator will even jot this down in the case study notes, citing the fact that a person was asked but declined to com­ment, as done in good journalistic accounts.

Yet a third type of interview entails more structured questions, along the lines of a formal survey. Such a survey could be designed as part of an embed­ded case study (see Chapter 2) and produce quantitative data as part of the case study evidence (see BOX 18). This situation would be relevant, for instance, if you were doing a case study of an urban design project and surveyed a group of designers about the project (e.g., Crewe, 2001) or if you did a case study of an organization that included a survey of workers and managers. This type of survey would follow both the sampling procedures and the instruments used in regular surveys, and it would subsequently be analyzed in a similar manner. The difference would be the survey’s role in relation to other sources of evidence. For example, residents’ perceptions of neighborhood decline or improvement would not necessarily be taken as a measure of actual decline or improvement but would be considered only one component of the overall assessment of the neighborhood.

BOX 18

A Case Study Encompassing a Survey

Hanna (2000) used a variety of sources of data, including a survey, to conduct a case study of an urban-rural estuarine setting. In this setting, an integrated resource management program was established to help manage environmental and eco­nomic planning issues. The case study focused on the estuarine setting, including its description and the policies and public participation that appeared to affect it. Within the case study, participants in the policy process served as an embedded unit of analysis. Hanna surveyed these individuals, and the survey data were pre­sented with statistical tests as part of the single-case study.

Overall, interviews are an essential source of case study evidence because most case studies are about human affairs or behavioral events. Well-informed interviewees can provide important insights into such affairs or events. The interviewees also can provide shortcuts to the prior history of such situations, helping you to identify other relevant sources of evidence.

At the same time, even though your interviews may focus on behavioral events because they are the key ingredients of your case study, the interviews should always be considered verbal reports only. As such, even in reporting about such events or explaining how they occurred, the interviewees’ responses are subject to the common problems of bias, poor recall, and poor or inaccurate articulation. Again, a reasonable approach is to corroborate inter­view data with information from other sources.

Sometimes, you will be interested in an interviewee’s opinions or attitudes, apart from explaining behavioral events. Corroborating these opinions or attitudes against other sources would not be relevant, as in dealing with behav­ioral events. You still may want to get a feeling for the prevalence of the opin­ions or attitudes by comparing them with those of others, but the more you do this, the more you are moving toward a conventional survey and should follow survey procedures and precautions.

A common question about doing interviews is whether to record them. Using recording devices is a matter of personal preference. Audiotapes cer­tainly provide a more accurate rendition of any interview than any other method. However, a recording device should not be used when (a) an inter­viewee refuses permission or appears uncomfortable in its presence, (b) there is no specific plan for transcribing or systematically listening to the contents of the electronic record—a process that takes enormous time and energy, (c) the investigator is clumsy enough with mechanical devices that the recording creates distractions during the interview itself, or (d) the investigator thinks that the recording device is a substitute for “listening” closely throughout the course of an interview.

4. Direct Observation

Because a case study should take place in the natural setting of the “case,” you are creating the opportunity for direct observations. Assuming that the phenomena of interest have not been purely historical, some relevant behav­iors or environmental conditions will be available for observation. Such obser­vations serve as yet another source of evidence in a case study.

The observations can range from formal to casual data collection activities. Most formally, observational instmments can be developed as part of the case study protocol, and the fieldworker may be asked to assess the occurrence of certain types of behaviors during certain periods of time in the field (see the two examples in BOX 19). This can involve observations of meetings, side­walk activities, factory work, classrooms, and the like. Less formally, direct observations might be made throughout a field visit, including those occasions during which other evidence, such as that from interviews, is being collected. For instance, the condition of buildings or work spaces will indicate something about the climate or impoverishment of an organization; similarly, the location or the furnishings of an interviewee’s office may be one indicator of the status of the interviewee within an organization.

BOX 19

Using Observational Evidence

19A. Reporting Field Observations

“Clean rooms” are a key part of the manufacturing process for producing semicon­ductor chips. Among other features, employees wear “bunny suits” of lint-free cloth and handle extremely small components in these rooms. In their case study of high- tech working life, Silicon Valley Fever, Rogers and Larsen (1984) used observational evidence to show how employees adapted to the working conditions in these clean rooms, adding that, at the time, most of the employees were women while most of the supervisors were men.

19B. Combining Field Observations with Other Types of Case Study Evidence

Case studies need not be limited to a single source of evidence. In fact, most of the better case studies rely on a variety of sources.

One example of a case study that used such a variety is a book by Cross et al. (1971) covering events in a single school (also see BOX 7, Chapter 2, p. 48). The case study included an observational protocol for measuring the time that students spent on various tasks but also relied on a structured survey of a larger number of teachers, open-ended interviews with a smaller number of key persons, and a review of organizational documents. Both the observational and survey data led to quanti­tative information about attitudes and behavior in the school, whereas the open- ended interviews and documentary evidence led to qualitative information.

All sources of evidence were reviewed and analyzed together, so that the case study’s findings were based on the convergence of information from different sources, not quantitative or qualitative data alone.

Observational evidence is often useful in providing additional information about the topic being studied. If a case study is about a new technology or a school curriculum, for instance, observations of the technology or curriculum at work are invaluable aids for understanding the actual uses of the technology or curriculum or any potential problems being encountered. Similarly, obser­vations of a neighborhood or of an organizational unit add new dimensions for understanding either the context or the phenomenon being studied. The obser­vations can be so valuable that you may even consider taking photographs at the case study site. At a minimum, these photographs will help to convey important case characteristics to outside observers (see Dabbs, 1982). Note, however, that in some situations—such as photographing students in public schools—you will need written permission before proceeding.

A common procedure to increase the reliability of observational evidence is to have more than a single observer making an observation—whether of the formal or the casual variety. Thus, when resources permit, a case study inves­tigation should allow for the use of multiple observers.

5. Participant-Observation

Participant-observation is a special mode of observation in which you are not merely a passive observer. Instead, you may assume a variety of roles within a case study situation and may actually participate in the events being studied. In urban neighborhoods, for instance, these roles may range from hav­ing casual social interactions with various residents to undertaking specific functional activities within the neighborhood (see Yin, 1982a). The roles for different illustrative studies in neighborhoods and organizations have included

• being a resident in a neighborhood that is the subject of a case study (see BOX 20);
• taking some other functional role in a neighborhood, such as serving as a store­keeper’s assistant;
• serving as a staff member in an organizational setting; and
• being a key decision maker in an organizational setting.

The participant-observation technique has been most frequently used in anthropological studies of different cultural or social groups. The technique also can be used in more everyday settings, such as a large organization (see BOX 21; also see BOX 16, earlier) or informal small groups.

BOX 21

A Participant-Observer Study in an “Everyday” Setting

Eric Redman provides an insider’s account of how Congress works in his well- regarded case study. The Dance of Legislation (1973). The case study traces the intro­duction and passage of the legislation that created the National Health Service Corps during the 91st Congress in 1970.

Redman’s account, from the vantage point of an author who was on the staff of one of the bill’s main supporters, Senator Warren C. Magnuson, is well written and easy to read. The account also provides the reader with great insight into the daily operations of Congress—from the introduction of a bill to its eventual passage, including the politics of a lame-duck session when Richard Nixon was president.

The account is an excellent example of participant-observation in a contempo­rary setting. It contains information about insiders’ roles that few persons have been privileged to share. The subtle legislative strategies, the overlooked role of commit­tee clerks and lobbyists, and the interaction between the legislative and executive branches of government are all re-created by the case study, and all add to the reader’s general understanding of the legislative process.

Participant-observation provides certain unusual opportunities for collect­ing case study data, but it also involves major problems. The most distinctive opportunity is related to your ability to gain access to events or groups that are otherwise inaccessible to a study. In other words, for some topics, there may be no way of collecting evidence other than through participant-observation. Another distinctive opportunity is the ability to perceive reality from the view­point of someone “inside” the case study rather than external to it. Many have argued that such a perspective is invaluable in producing an “accurate” por­trayal of a case study phenomenon. Finally, other opportunities arise because you may have the ability to manipulate minor events—such as convening a meeting of a group of persons in the case. Only through participant-observation can such manipulation occur, as the use of documents, archival records, and interviews, for instance, assumes a passive investigator. The manipulations will not be as precise as those in experiments, but they can produce a greater variety of situations for the purposes of collecting data.

The major problems related to participant-observation have to do with the potential biases produced (see Becker, 1958). First, the investigator has less ability to work as an external observer and may, at times, have to assume posi­tions or advocacy roles contrary to the interests of good social science prac­tice. Second, the participant-observer is likely to follow a commonly known phenomenon and become a supporter of the group or organization being stud­ied, if such support did not already exist. Third, the participant role may simply require too much attention relative to the observer role. Thus, the par­ticipant-observer may not have sufficient time to take notes or to raise ques­tions about events from different perspectives, as a good observer might. Fourth, if the organization or social group being studied is physically dis­persed, the participant-observer may find it difficult to be at the right place at the right time, either to participate in or to observe important events.

These trade-offs between the opportunities and the problems have to be con­sidered seriously in undertaking any participant-observation study. Under some circumstances, this approach to case study evidence may be just the right approach; under other circumstances, the credibility of a whole case study project can be threatened.

6. Physical Artifacts

A final source of evidence is a physical or cultural artifact—a technological device, a tool or instrument, a work of art, or some other physical evidence. Such artifacts may be collected or observed as part of a case study and have been used extensively in anthropological research.

Physical artifacts have less potential relevance in the most typical kind of case study. However, when relevant, the artifacts can be an important com­ponent in the overall case. For example, one case study of the use of personal computers in the classroom needed to ascertain the nature of the actual use of the machines. Although use could be directly observed, an artifact—the computer printout—also was available. Students displayed these printouts as the finished product of their work and maintained notebooks of their print­outs. Each printout showed the type of schoolwork that had been done as well as the date and amount of computer time used to do the work. By exam­ining the printouts, the case study investigators were able to develop a broader perspective concerning all of the classroom applications over the length of a semester, far beyond that which could be directly observed in the limited time of a field visit.

Source: Yin K Robert (2008), Case Study Research Designs and Methods, SAGE Publications, Inc; 4th edition.