An Analytic Strategy in Case Study Research: More Than Familiarity with Analytic Tools

1. Need for an Analytic Strategy

Introduction. The analysis of case study evidence is one of the least developed and most difficult aspects of doing case studies. Too many times, investigators start case studies without having the foggiest notion about how the evidence is to be analyzed (despite Chapter 3’s recommendation that the analytic approaches be considered when developing the case study protocol). Such investigations easily become stalled at the analytic stage; this author has known colleagues who have simply ignored their case study data for month after month, not knowing what to do with the evidence.

Because of the problem, the experienced case study investigator is likely to have great advantages over the novice at the analytic stage. Unlike statistical analysis, there are few fixed formulas or cookbook recipes to guide the novice. Instead, much depends on an investigator’s own style of rigorous empirical thinking, along with the sufficient presentation of evidence and careful con­sideration of alternative interpretations.

Investigators and especially novices do continue to search for formulas, recipes, or tools, hoping that familiarity with these devices will produce the needed analytic result. The tools are important and can be useful, but they are usually most helpful if you know what to look for (i.e., have an overall ana­lytic strategy), which unfortunately returns you back to your original problem, if you hadn’t noticed.

Computer-assisted tools. For instance, computer-assisted routines with prepack­aged software such as Atlas.ti, HyperRESEARCH, NVivo, or The Ethnograph

all are examples of computer-assisted qualitative data analysis software (CAQDAS—e.g., Fielding & Lee, 1998). The software has become more diverse and functional over the past decade. Essentially, the tools can help you code and categorize large amounts of narrative text, as might have been col­lected from open-ended interviews or from large volumes of written materials, such as newspaper articles. Guidance on coding skills and techniques also has improved (e.g., Boyatzis, 1998).

Key to your understanding of the value of these packages are two words: assisted and tools. The software will not do any analysis for you, but it may serve as an able assistant and reliable tool. For instance, if you enter your textual data and then define an initial set of codes, one or another of the var­ious software packages will readily locate in the textual data all words and phrases matching these codes, count the incidence or occurrence of the words or codes, and even conduct Boolean searches to show when and where multi­ple combinations are found together. You can do this process iteratively, grad­ually building more complex categories or groups of codes. However, unlike statistical analyses, you cannot use the software’s outputs themselves as if they were the end of your analysis.

Instead, you will need to study the outputs to determine whether any mean­ingful patterns are emerging. Quite likely, any patterns—such as the frequency of codes or code combinations—will still be conceptually more primitive (lower) than the initial “how” and “why” research questions that might have led to your case study in the first place. In other words, developing a rich and full explanation or even a good description of your case, in response to your initial “how” or “why” questions, will require much post-computer thinking and analysis on your part.

Backtracking, you also will need to have clarified the reasons for defining the initial codes or subsequent codes, as well as connecting them to your original research design (you, not the software, created them). In what ways do the codes or concepts accurately reflect the meaning of the retrieved words and phrases, and why? Answering these questions requires your own analytic rationale.

Under some circumstances, the computerized functions can nevertheless be extremely helpful. The minimal conditions include when (a) the words or verbal reports represent verbatim records and are the central part of your case study evi­dence and (b) you have a large collection of such data. Such conditions com­monly occur in research using grounded theory strategies (e.g., Corbin & Strauss, 2007), where the surfacing of a new concept or theme can be highly valuable. However, even under the best of circumstances, nearly all scholars express strong caveats about any use of computer-assisted tools: You must still be prepared to be the main analyst and to direct the tools; they are the assistant, not you.

Most case studies pose a more serious challenge in efforts to use computer- assisted tools: Verbatim records such as interviewees’ responses are likely to be only part of the total array of case study evidence. The case study will typ­ically be about complex events and behavior, occurring within a possibly more complex, real-life context. Unless you convert all of your evidence—including your field notes and the archival documents you might have collected—into the needed textual form, computerized tools cannot readily handle this more diverse array of evidence. Yet, as emphasized in Chapter 4, such an array should represent an important strength of your case study. For a diverse set of evidence, you therefore need to develop your own analytic strategies.

A helpful starting point is to “play” with your data. One set of analytic manipulations has been comprehensively described and summarized by Miles and Huberman (1994) and includes

• Putting information into different arrays
• Making a matrix of categories and placing the evidence within such categories
• Creating data displays—flowcharts and other graphics—for examining the data
• Tabulating the frequency of different events
• Examining the complexity of such tabulations and their relationships by calculat­ing second-order numbers such as means and variances
• Putting information in chronological order or using some other temporal scheme

These are indeed useful and important manipulations and can put the evidence in some preliminary order. Moreover, conducting such manipulations is one way of overcoming the stalling problem mentioned earlier. Without a broader strat­egy, however, you are still likely to encounter many false starts and potentially waste large chunks of your time. Furthermore, if after playing with the data, a general strategy does not emerge (or if you are not facile in playing with the data to begin with), the entire case study analysis is likely to be in jeopardy.

Any preliminary manipulations, such as the preceding, or any use of com­puter-assisted tools therefore cannot substitute for having a general analytic strategy in the first place. Put another way, all empirical research studies, including case studies, have a “story” to tell. The story differs from a fictional account because it embraces your data, but it remains a story because it must have a beginning, end, and middle. The needed analytic strategy is your guide to crafting this story, and only rarely will your data do the crafting for you.

Once you have a strategy, the tools may turn out to be extremely useful (or irrelevant). The strategy will help you to treat the evidence fairly, produce compelling analytic conclusions, and rule out alternative interpretations. The strategy also will help you to use tools and make manipulations more effec­tively and efficiently. Four such strategies are described below, after which five specific techniques for analyzing case study data are reviewed. These strate­gies or techniques are not mutually exclusive. You can use any number of them in any combination. A continued alert is to be aware of these choices before collecting your data, so that you can be sure your data will be analyzable.

2. Four General Strategies

Relying on theoretical propositions. The first and most preferred strategy is to follow the theoretical propositions that led to your case study. The original objectives and design of the case study presumably were based on such propo­sitions, which in turn reflected a set of research questions, reviews of the lit­erature, and new hypotheses or propositions.

The propositions would have shaped your data collection plan and therefore would have given priorities to the relevant analytic strategies. One example, from a study of intergovernmental relationships, followed the proposition that federal funds have redistributive dollar effects but also create new organiza­tional changes at the local level (Yin, 1980). The basic proposition—the creation of a “counterpart bureaucracy” in the form of local planning organi­zations, citizen action groups, and other new offices within a local government itself, but all attuned to specific federal programs—was traced in case studies of several cities. For each city, the purpose of the case study was to show how the formation and modification in local organizations occurred after changes in related federal programs and how these local organizations acted on behalf of the federal programs even though they might have been components of local government.

This proposition is an example of a theoretical orientation guiding the case study analysis. Clearly, the proposition helps to focus attention on certain data and to ignore other data. (A good test is to decide what data you might cite if you had only 5 minutes to defend a proposition in your case study.) The propo­sition also helps to organize the entire case study and to define alternative explanations to be examined. Theoretical propositions stemming from “how” and “why” questions can be extremely useful in guiding case study analysis in this manner.

Developing a case description. A second general analytic strategy is to develop a descriptive framework for organizing the case study. This strategy is less preferable than relying on theoretical propositions but serves as an alternative when you are having difficulty making the first strategy work. For instance, you actually (but undesirably) may have collected a lot of data without having set­tled on an initial set of research questions or propositions. Studies started this way inevitably encounter challenges at their analytic phase.

Sometimes, the original and explicit purpose of the case study may have been a descriptive one. This was the objective of the famous sociological study Middletown (Lynd & Lynd, 1929), which was a case study of a small midwest- em city. What is interesting about Middletown, aside from its classic value as a rich and historic case, is its compositional structure, reflected by its chapters:

• Chapter I: Getting a Living
• Chapter II: Making a Home
• Chapter III: Training the Young
• Chapter IV: Using Leisure
• Chapter V: Engaging in Religious Practices
• Chapter VI: Engaging in Community Activities

These chapters cover a range of topics relevant to community life in the early 20th century, when Middletown was studied. Note how the descriptive framework organizes the case study analysis but also assumes that data were collected about each topic in the first place. In this sense, you should have thought (at least a little) about your descriptive framework before designing your data collection instruments. As usual, the ideas for your framework should have come from your initial review of literature, which may have revealed gaps or topics of interest to you, spurring your interest in doing a case study. Another suggestion is to note the structure of existing case studies (e.g., by examining the original versions of those cited in the BOXES throughout this book) and at least to observe their tables of contents as an implicit clue to different descriptive approaches.

In other situations, the original objective of the case study may not have been a descriptive one, but a descriptive approach may help to identify the appropriate causal links to be analyzed—even quantitatively. BOX 25 gives an example of a case study that was concerned with the complexity of imple­menting a local public works program in Oakland, California. Such complex­ity, the investigators realized, could be described in terms of the multiplicity of decisions, by public officials, that had to occur in order for implementation to succeed. This descriptive insight later led to the enumeration, tabulation, and hence quantification of the various decisions. In this sense, the descriptive approach was used to identify (a) an embedded unit of analysis (see Chapter 2) and (b) an overall pattern of complexity that ultimately was used in a causal sense to “explain” why implementation had failed.

BOX 25

Quantifying the Descriptive Elements of a Case Study

Pressman and Wildavsky’s (1973) book, Implementation: How Great Expectations in Washington Are Dashed in Oakland, is regarded as one of the breakthrough contri­butions to the study of implementation (Yin, 1982b). This is the process whereby some programmatic activity—an economic development project, a new curriculum in a school, or a crime prevention program, for example—is installed in a specific setting (e.g., organization or community). The process is complex and involves numerous individuals, organizational rules, social norms, and mixtures of good and bad intentions.

Can such a complex process also be the subject of quantitative inquiry and analy­sis? Pressman and Wildavsky (1973) offer one innovative solution. To the extent that successful implementation can be described as a sequence of decisions, an analyst can focus part of the case study on the number and types of such decisions or elements.

Thus, in their chapter titled “The Complexity of Joint Action,” the authors analyze the difficulties in Oakland: To implement one public works program required a total of 70 sequential decisions—project approvals, negotiation of leases, letting of con­tracts, and so on. The analysis examined the level of agreement and the time needed to reach agreement at each of the 70 decision points. Given the normal diversity of opinion and slippage in time, the analysis illustrates—in a quantitative manner—the low probability of implementation success.

Using both qualitative and quantitative data. This third strategy may be more attractive to advanced students and scholars and can yield appreciable bene­fits. Certain case studies can include substantial amounts of quantitative data. If these data are subjected to statistical analyses at the same time that qualita­tive data nevertheless remain central to the entire case study, you will have successfully followed a strong analytic strategy.

The quantitative data may have been relevant to your case study for at least two reasons. First, the data may cover the behavior or events that your case study is trying to explain—typically, the “outcomes” in an evaluative case study. Second, the data may be related to an embedded unit of analysis within your broader case study. In either situation, the qualitative data may be critical in explaining or otherwise testing your case study’s key propositions. So, imagine a case study about a school, a neighborhood, an organization, a com­munity, a medical practice, or some other common case study topic. For these topics, the outcomes of an evaluative case study might be, respectively student achievement (for the case study about the school), housing prices (for the neighborhood), employees’ salaries (for the organization), various crime rates (for the community), or the course of an illness (for the medical practice). Alternatively, the embedded units might be students (or teachers), census blocks (or single-family housing), employees (for the organization), persons arrested (for the community), or patients (for the medical practice).

All of the illustrative outcomes or embedded units can be the occasion for having collected fine-grained quantitative data. Yet, the main case study ques­tions might have been at a higher level: a single school (not its students), the neighborhood (not its housing units), a business firm (not its employees), a community (not its residents), or a new medical practice (not the patients). To explore, describe, or explain events at this higher level, you would have col­lected and used qualitative data. Thus, your case study would have deliberately used both qualitative and quantitative data.

If you attempt this third strategy, be prepared for the skills you will need Beyond knowing how to do the case study well, you may have to master cer­tain statistical techniques. Mentioned later in this chapter (but only in passing) are regression discontinuity analyses, hierarchical linear models, and struc­tural equation models. Do you believe that any of them can be part of a case study analysis?

Examining rival explanations. A fourth general analytic strategy, trying to define and test rival explanations, generally works with all of the previous three: Initial theoretical propositions (the first strategy above) might have included rival hypotheses; the contrasting perspectives of participants and stakeholders may produce rival descriptive frameworks (the second strategy); and data from comparison groups may cover rival conditions to be examined as part of using both quantitative and qualitative data (the third strategy).

For instance, the typical hypothesis in an evaluation is that the observed outcomes were the result of an intervention supported by public or founda­tion funds. The simple or direct rival explanation would be that the observed outcomes were in fact the result of some other influence besides the inter­vention and that the investment of funds may not actually have been needed. Being aware (ahead of time) of this direct rival, your case study data collec­tion should then have included attempts to collect evidence about the possi­ble “other influences.” Furthermore, you should have pursued your data collection about them vigorously—as if you were in fact trying to prove the potency of the other influences rather than rejecting them (Patton, 2002, p. 553; P. R. Rosenbaum, 2002, pp. 8-10). Then, if you had found insufficient evidence, you would less likely be accused of stacking the deck in favor of the original hypothesis.

The direct rival—that the original investment was not the reason for the observed outcomes—is but one of several types of rival explanations. Figure 5.1 classifies and lists many types of rivals (Yin, 2000). For each type, an informal and more understandable descriptor (in the parentheses and quotation marks in Figure 5.1) accompanies the formal social science categorization, making the gist of the rival thinking clearer.

The list reminds us of three “craft” rivals that underlie all of our social sci­ence research, and textbooks have given much attention to these craft rivals. However, the list also defines six “real-life” rivals, which have received virtu­ally no attention by other textbooks (nor, unfortunately, do most texts discuss the challenges and benefits of rival thinking or the use of rival explanations). These real-life rivals are the ones that you should carefully identify prior to your data collection (while not ignoring the craft rivals). Some real-life rivals also may not become apparent until you are in the midst of your data collec­tion, and attending to them at that point is acceptable and desirable. Overall, the more rivals that your analysis addresses and rejects, the more confidence you can place in your findings.

Rival explanations were a critical part of several of the case studies already contained in the BOXES cited earlier (e.g., refer to BOXES 1 and 11 in Chapters 1 and 2, respectively). The authors of these case studies used the rivals to drive their entire case study analysis. Additional examples—covering cases of university innovation and of drug abuse prevention but deliberately focusing on the essence of the evidence about rival explanations—are found in Yin (2003, chaps. 4 and 5).

Summary. The best preparation for conducting case study analysis is to have a general analytic strategy. Four have been described, relying on theoretical propositions, case descriptions, a dual use of both quantitative and qualitative data, and rival explanations. All four strategies underlie the analytic techniques to be described below. Without such strategies (or alternatives to them), case study analysis will proceed with difficulty.

The remainder of this chapter covers the specific analytic techniques, to be used as part of and along with any of the general strategies. The techniques are especially intended to deal with the previously noted problems of developing internal validity and external validity in doing case studies (see Chapter 2).

Source: Yin K Robert (2008), Case Study Research Designs and Methods, SAGE Publications, Inc; 4th edition.