The post Case Study Methodology in Management Research appeared first on HKT Consultant.
]]>A case study involves an up-close, in-depth, and detailed examination of a particular case or cases, within a real-world context. For example, case studies in medicine may focus on an individual patient or ailment; case studies in business might cover a particular firm’s strategy or a broader market; similarly, case studies in politics can range from a narrow happening over time (e.g., a specific political campaign) to an enormous undertaking (e.g., a World War).
Generally, a case study can highlight nearly any individual, group, organization, event, belief system, or action. A case study does not necessarily have to be one observation (N=1), but may include many observations (one or multiple individuals and entities across multiple time periods, all within the same case study).
Case study research has been extensively practiced in both the social and natural sciences.
Main contentsSee more from basic to advanced
Case Study Research Designs
As with other social science methods, no single research design dominates case study research. Case studies can use at least four types of designs. First, there may be a “no theory first” type of case study design, which is closely connected to Kathleen M. Eisenhardt’s methodological work. A second type of research design highlights the distinction between single- and multiple-case studies, following Robert K. Yin’s guidelines and extensive examples. A third design deals with a “social construction of reality”, represented by the work of Robert E. Stake. Finally, the design rationale for a case study may be to identify “anomalies”. A representative scholar of this design is Michael Burawoy. Each of these four designs may lead to different applications, and understanding their sometimes unique ontological and epistemological assumptions becomes important. However, although the designs can have substantial methodological differences, the designs also can be used in explicitly acknowledged combinations with each other.
While case studies can be intended to provide bounded explanations of single cases or phenomena, they are often intended to theoretical insights about the features of a broader population.
Case Selection and Structure
Case selection in case study research is generally intended to both find cases that are a representative sample and which have variations on the dimensions of theoretical interest. Using that is solely representative, such as an average or typical case is often not the richest in information. In clarifying lines of history and causation it is more useful to select subjects that offer an interesting, unusual or particularly revealing set of circumstances. A case selection that is based on representativeness will seldom be able to produce these kinds of insights.
While random selection of cases is a valid case selection strategy in large-N research, there is a consensus among scholars that it risks generating serious biases in small-N research. Cases should generally be chosen that have a high expected information gain. For example, outlier cases (those which are extreme, deviant or atypical) can reveal more information than the potentially representative case. A case may also be chosen because of the inherent interest of the case or the circumstances surrounding it. Alternatively it may be chosen because of researchers’ in-depth local knowledge; where researchers have this local knowledge they are in a position to “soak and poke” as Richard Fenno put it, and thereby to offer reasoned lines of explanation based on this rich knowledge of setting and circumstances.
Beyond decisions about case selection and the subject and object of the study, decisions need to be made about purpose, approach and process in the case study. Gary Thomas thus proposes a typology for the case study wherein purposes are first identified (evaluative or exploratory), then approaches are delineated (theory-testing, theory-building or illustrative), then processes are decided upon, with a principal choice being between whether the study is to be single or multiple, and choices also about whether the study is to be retrospective, snapshot or diachronic, and whether it is nested, parallel or sequential.
John Gerring and Jason Seawright list seven case selection strategies:
Typical cases are cases that exemplify a stable cross-case relationship. These cases are representative of the larger population of cases, and the purpose of the study is to look within the case rather than compare it with other cases.
- Diverse cases are cases that have variation on the relevant X and Y variables. Due to the range of variation on the relevant variables, these cases are representative of the full population of cases.
- Extreme cases are cases that have an extreme value on the X or Y variable relative to other cases.
- Deviant cases are cases that defy existing theories and common sense. They not only have extreme values on X or Y (like extreme cases), but defy existing knowledge about causal relations.
- Influential cases are cases that are central to a model or theory (for example, Nazi Germany in theories of fascism and the far-right).
- Most similar cases are cases that are similar on all the independent variables, except the one of interest to the researcher.
- Most different cases are cases that are different on all the independent variables, except the one of interest to the researcher.
For theoretical discovery, Jason Seawright recommends using deviant cases or extreme cases that have an extreme value on the X variable.
Arend Lijphart, and Harry Eckstein identified five types of case study research designs (depending on the research objectives), Alexander George and Andrew Bennett added a sixth category:
- In atheoretical (or configurative idiographic) case studies the goal is to describe a case very well, but not to contribute to a theory.
- In interpretative (or disciplined configurative) case studies the goal is to use established theories to explain a specific case.
- In hypothesis-generating (or heuristic) case studies the goal is to inductively identify new variables, hypotheses, causal mechanisms and causal paths.
- In theory testing case studies the goal is to assess the validity and scope conditions of existing theories.
- In plausibility probes the goal is to assess the plausibility of new hypotheses and theories.
- In building block studies of types or subtypes the goal is to identify common patterns across cases.
In terms of case selection, Gary King, Robert Keohane, and Sidney Verba warn against “selecting on the dependent variable”. They argue for example that researchers cannot make valid causal inferences about war outbreak by only looking at instances where war did happen (the researcher should also look at cases where war did not happen). Scholars of qualitative methods have disputed this claim, however. They argue that selecting on the dependent variable can be useful depending on the purposes of the research.
King, Keohane and Verba argue that there is no methodological problem in selecting on the explanatory variable, however. They do warn about multicollinearity (choosing two or more explanatory variables that perfectly correlate with each other).
Uses and Limits
Uses
Case studies have commonly been seen as a fruitful way to come up hypotheses and generate theories. Case studies are also useful for formulating concepts, which are an important aspect of theory construction. The concepts used in qualitative research will tend to have higher conceptual validity than concepts used in quantitative research (due to conceptual stretching: the unintentional comparison of dissimilar cases). Case studies add descriptive richness. Case studies are suited to explain outcomes in individual cases, which is something that quantitative methods are less equipped to do. Through fine-gained knowledge and description, case studies can fully specify the causal mechanisms in a way that may be harder in a large-N study. In terms of identifying “causal mechanisms”, some scholars distinguish between “weak” and “strong chains”. Strong chains actively connect elements of the causal chain to produce an outcome whereas weak chains are just intervening variables.
Case studies of cases that defy existing theoretical expectations may contribute knowledge by delineating why the cases violate theoretical predictions and specifying the scope conditions of the theory Case studies are useful in situations of causal complexity where there is equifinality, complex interaction effects and path dependency Case studies can identify necessary and insufficient conditions, as well as complex combinations of necessary and sufficient conditions. They argue that case studies may also be useful in identifying the scope conditions of a theory: whether variables are sufficient or necessary to bring about an outcome.
Limitations
Designing Social Inquiry, an influential 1994 book written by Gary King, Robert Keohane, and Sidney Verba, primarily applies lessons from regression-oriented analysis to qualitative research, arguing that the same logics of causal inference can be used in both types of research. The authors’ recommendation is to increase the number of observations (a recommendation that Barbara Geddes also makes in Paradigms and Sand Castles), because few observations make it harder to estimate multiple causal effects, as well as increase the risk that there is measurement error, and that an event in a single case was caused by random error or unobservable factors. KKV sees process-tracing and qualitative research as being “unable to yield strong causal inference” due to the fact that qualitative scholars would struggle with determining which of many intervening variables truly links the independent variable with a dependent variable. The primary problem is that qualitative research lacks a sufficient number of observations to properly estimate the effects of an independent variable. They write that the number of observations could be increased through various means, but that would simultaneously lead to another problem: that the number of variables would increase and thus reduce degrees of freedom.
The purported “degrees of freedom” problem that KKV identify is widely considered flawed; while quantitative scholars try to aggregate variables to reduce the number of variables and thus increase the degrees of freedom, qualitative scholars intentionally want their variables to have many different attributes and complexity. For example, James Mahoney writes, “the Bayesian nature of process tracing explains why it is inappropriate to view qualitative research as suffering from a small-N problem and certain standard causal identification problems.” By using Bayesian probability, it may be possible to makes strong causal inferences from a small sliver of data.
A commonly described limit of case studies is that they do not lend themselves to generalizability. Some scholars, such as Bent Flyvbjerg, have pushed back on that notion.
As small-N research should not rely on random sampling, scholars must be careful in avoiding selection bias when picking suitable cases. A common criticism of qualitative scholarship is that cases are chosen because they are consistent with the scholar’s preconceived notions, resulting in biased research.
Alexander George and Andrew Bennett note that a common problem in case study research is that of reconciling conflicting interpretations of the same data.
The post Case Study Methodology in Management Research appeared first on HKT Consultant.
]]>The post The Case Study as a Research Method appeared first on HKT Consultant.
]]>Do not underestimate the depth of your challenge. Although you may be ready to focus on designing and doing case study research, others may espouse and advocate other research methods. Similarly, prevailing federal or other research funds may favor other methods, but not the case study. As a result, you may need to have ready responses to some inevitable questions.
First and foremost, you should explain and show how you are devoting yourself to following a rigorous methodological path. The path begins with a thorough literature review and the careful and thoughtful posing of research questions or objectives. Equally important will be a dedication to formal and explicit procedures when doing your research. Along these lines, this book offers much guidance. It shows how case study research includes procedures central to all types of research methods, such as protecting against threats to validity, maintaining a “chain of evidence,” and investigating and testing “rival explanations.” The successful experiences of scholars and students, for over 25 years, may attest to the potential payoffs from using this book.
Second, you should understand and openly acknowledge the strengths and limitations of case study research. Such research, like any other, complements the strengths and limitations of other types of research. In the face of those who might only see the need for a single research method, this book believes that, just as different scientific methods prevail in the natural sciences, different social science research methods fill different needs and situations for investigating social science topics. For instance, in the natural sciences, astronomy is a science but does not rely on the experimental method. Similarly, much neurophysiological and neuroanatomical research does not rely on statistical methods. For social science, later portions of this chapter present more about the potential “niches” of different research methods.
As a research method, the case study is used in many situations, to contribute to our knowledge of individual, group, organizational, social, political, and related phenomena. Not surprisingly, the case study has been a common research method in psychology, sociology, political science, anthropology, social work, business, education, nursing, and community planning. Case studies are even found in economics, in which the structure of a given industry or the economy of a city or a region may be investigated. In all of these situations, the distinctive need for case studies arises out of the desire to understand complex social phenomena. In brief, the case study method allows investigators to retain the holistic and meaningful characteristics of real-life events—such as individual life cycles, small group behavior, organizational and managerial processes, neighborhood change, school performance, international relations, and the maturation of industries.
This method covers the distinctive characteristics of the case study as a research method. We will help you to deal with some of the more difficult questions still frequently neglected by available research texts. So often, for instance, the author has been confronted by a student or colleague who has asked (a) how to define the “case” being studied, (b) how to determine the relevant data to be collected, or (c) what to do with the data, once collected. This book answers these questions and more, by covering all of the phases of design, data collection, analysis, and reporting.
Main contentsSee more from basic to advanced
At the same time, the book does not cover all uses of case studies. For example, it is not intended to help those who might use case studies as a teaching tool, popularized in the fields of law, business, medicine, or public policy (see Garvin, 2003; Llewellyn, 1948; Stein, 1952; Towl, 1969; Windsor & Greanias, 1983) but now prevalent in virtually every academic field, including the natural sciences. For teaching purposes, a case study need not contain a complete or accurate rendition of actual events. Rather, the purpose of the “teaching case” is to establish a framework for discussion and debate among students. The criteria for developing good cases for teaching—usually of the single- and not multiple-case variety—are different from those for doing research (e.g., Caulley & Dowdy, 1987). Teaching case studies need not be concerned with the rigorous and fair presentation of empirical data; research case studies need to do exactly that.
Similarly, this book is not intended to cover those situations in which cases are used as a form of record keeping. Medical records, social work files, and other case records are used to facilitate some practice, such as medicine, law, or social work. Again, the criteria for developing good cases for practice differ from those for doing case study research.
In contrast, the rationale for this book is that case studies are commonly used as a research method in the social science disciplines—psychology (e.g., D. T. Campbell, 1975; Hersen & Barlow, 1976), sociology (e.g., Hamel, 1992; Platt, 1992; Ragin & Becker, 1992), political science (e.g., George & Bennett, 2004; Gerring, 2004), and anthropology—and for doing research in different professional fields, such as social work (e.g., Gilgun, 1994), business and marketing (e.g., Benbasat, Goldstein, & Mead, 1987; Bonoma, 1985; Ghauri & Grdnhaug, 2002; Gibbert & Ruigrok, 2007; Graebner & Eisenhardt, 2004; Voelpel, Leibold, Tekie, & von Krogh, 2005), public administration (e.g., Agranoff & Radin, 1991; Perry & Kraemer, 1986), public health (e.g., Pluye, Potvin, Denis, Pelletier, & Mannoni, 2005; Richard et al., 2004), education (e.g., Yin, 2006a; Yin & Davis, 2006), accounting (e.g., Brans, 1989), and evaluation (e.g., U.S. Government Accountability Office, 1990).
You as a social scientist would like to know how to design and conduct single- or multiple-case studies to investigate a research issue. You may only be doing a case study or may be using it as part of a larger mixed methods study (see Chapter 2). Whichever, this book covers the entire range of issues in designing and doing case studies, including how to start a case study, collect case study evidence, analyze case study data, and compose a case study report.
Source: Yin K Robert (2008), Case Study Research Designs and Methods, SAGE Publications, Inc; 4th edition.
The post The Case Study as a Research Method appeared first on HKT Consultant.
]]>The post Comparing Case Studies with Other Research Methods in the Social Sciences appeared first on HKT Consultant.
]]>These and other choices represent different research methods. Each is a different way of collecting and analyzing empirical evidence, following its own logic. And each method has its own advantages and disadvantages. To get the most out of using the case study method, you need to appreciate these differences.
A common misconception is that the various research methods should be arrayed hierarchically. Many social scientists still deeply believe that case studies are only appropriate for the exploratory phase of an investigation, that surveys and histories are appropriate for the descriptive phase, and that experiments are the only way of doing explanatory or causal inquiries. This hierarchical view reinforces the idea that case studies are only a preliminary research method and cannot be used to describe or test propositions.
This hierarchical view, however, may be questioned. Experiments with an exploratory motive have certainly always existed. In addition, the development of causal explanations has long been a serious concern of historians, reflected by the subfield known as historiography. Likewise, case studies are far from being only an exploratory strategy. Some of the best and most famous case studies have been explanatory case studies (e.g., see BOX 1 for a vignette on Allison and Zelikow’s Essence of Decision: Explaining the Cuban Missile Crisis, 1999). Similarly, famous descriptive case studies are found in major disciplines such as sociology and political science (e.g., see BOX 2 for two vignettes). Additional examples of explanatory case studies are presented in their entirety in a companion book cited throughout this text (Yin, 2003, chaps. 4-7). Examples of descriptive case studies are similarly found there (Yin, 2003, chaps. 2 and 3).
Distinguishing among the various research methods and their advantages and disadvantages may require going beyond the hierarchical stereotype. The more appropriate view may be an inclusive and pluralistic one: Every research method can be used for all three purposes—exploratory, descriptive, and explanatory. There may be exploratory case studies, descriptive case studies, or explanatory case studies. Similarly, there may be exploratory experiments, descriptive experiments, and explanatory experiments. What distinguishes the different methods is not a hierarchy but three important conditions discussed below. As an important caution, however, the clarification does not imply that the boundaries between the methods—or the occasions when each is to be used—are always sharp. Even though each method has its distinctive characteristics, there are large overlaps among them. The goal is to avoid gross misfits—that is, when you are planning to use one type of method but another is really more advantageous.
1. When to Use Each Method
The three conditions consist of (a) the type of research question posed, (b) the extent of control an investigator has over actual behavioral events, and (c) the degree of focus on contemporary as opposed to historical events. Figure 1.1 displays these three conditions and shows how each is related to the five major research methods being discussed: experiments, surveys, archival analyses, histories, and case studies. The importance of each condition, in distinguishing among the five methods, is as follows.
Types of research questions (Figure 1.1, column 1). The first condition covers your research question(s) (Hedrick, Bickman, & Rog, 1993). A basic categorization scheme for the types of questions is the familiar series: “who,” “what,” “where,” “how,” and “why” questions.
If research questions focus mainly on “what” questions, either of two possibilities arises. First, some types of “what” questions are exploratory, such as “What can be learned from a study of a startup business?” This type of question is a justifiable rationale for conducting an exploratory study, the goal being to develop pertinent hypotheses and propositions for further inquiry. However, as an exploratory study, any of the five research methods can be used—for example, an exploratory survey (testing, for instance, the ability to survey startups in the first place), an exploratory experiment (testing, for instance, the potential benefits of different kinds of incentives), or an exploratory case study (testing, for instance, the importance of differentiating “first-time” startups from startups by entrepreneurs who had previously started other firms).
The second type of “what” question is actually a form of a “how many” or “how much” line of inquiry—for example, “What have been the ways that communities have assimilated new immigrants?” Identifying such ways is more likely to favor survey or archival methods than others. For example, a survey can be readily designed to enumerate the “what,” whereas a case study would not be an advantageous method in this situation.
Similarly, like this second type of “what” question, “who” and “where” questions (or their derivatives—“how many” and “how much”) are likely to favor survey methods or the analysis of archival data, as in economic studies. These methods are advantageous when the research goal is to describe the incidence or prevalence of a phenomenon or when it is to be predictive about certain outcomes. The investigation of prevalent political attitudes (in which a survey or a poll might be the favored method) or of the spread of a disease like AIDS (in which an epidemiologic analysis of health statistics might be the favored method) would be typical examples.
In contrast, “how” and “why” questions are more explanatory and likely to lead to the use of case studies, histories, and experiments as the preferred research methods. This is because such questions deal with operational links needing to be traced over time, rather than mere frequencies or incidence. Thus, if you wanted to know how a community successfully overcame the negative impact of the closing of its largest employer—a military base (see Bradshaw, 1999, also presented in BOX 26, Chapter 5, p. 138)—you would be less likely to rely on a survey or an examination of archival records and might be better off doing a history or a case study. Similarly, if you wanted to know how research investigators may possibly (but unknowingly) bias their research, you could design and conduct a series of experiments (see Rosenthal, 1966).
Let us take two more examples. If you were studying “who” had suffered as a result of terrorist acts and “how much” damage had been done, you might survey residents, examine government records (an archival analysis), or conduct a “windshield survey” of the affected area. In contrast, if you wanted to know “why” the act had occurred, you would have to draw upon a wider array of documentary information, in addition to conducting interviews; if you focused on the “why” question in more than one terrorist act, you would probably be doing a multiple-case study.
Similarly, if you wanted to know “what” the outcomes of a new governmental program had been, you could answer this question by doing a survey or by examining economic data, depending upon the type of program involved. Questions—such as “How many clients did the program serve?” “What kinds of benefits were received?” “How often were different benefits produced?”— all could be answered without doing a case study. But if you needed to know “how” or “why” the program had worked (or not), you would lean toward either a case study or a field experiment.
To summarize, the first and most important condition for differentiating among the various research methods is to classify the type of research question being asked. In general, “what” questions may either be exploratory (in which case, any of the methods could be used) or about prevalence (in which surveys or the analysis of archival records would be favored). “How” and “why” questions are likely to favor the use of case studies, experiments, or histories.
EXERCISE 1.1 Defining a Case Study Question
Develop a “how” or “why” question that would be the rationale for a case study that you might conduct. Instead of doing a case study, now imagine that you only could do a history, a survey, or an experiment (but not a case study) in order to answer this question. What would be the distinctive advantage of doing a case study, compared to these other methods, in order to answer this question?
Defining the research questions is probably the most important step to be taken in a research study, so you should be patient and allow sufficient time for this task. The key is to understand that your research questions have both substance—for example, What is my study about?—and form—for example, am I asking a “who,” “what,” “where,” “why,” or “how” question? Others have focused on some of the substantively important issues (see J. P. Campbell, Daft, & Hulin, 1982); the point of the preceding discussion is that the form of the question can provide an important clue regarding the appropriate research method to be used. Remember, too, the large areas of overlap among the methods, so that, for some questions, a choice among methods might actually exist. Be aware, finally, that you (or your academic department) may be predisposed to favor a particular method regardless of the study question. If so, be sure to create the form of the study question best matching the method you were predisposed to favor in the first place.
EXERCISE 1.2 Identifying the Research Questions Covered When Other Research Methods Are Used
Locate a research study based solely on the use of survey, historical, or experimental (but not case study) methods. Identify the research question(s) addressed by the study. Does the type of question differ from those that might have appeared as part of a case study on the same topic, and if so, how?
Extent of control over behavioral events (Figure 1.1, column 2) and degree of focus on contemporary as opposed to historical events (Figure 1.1, column 3). Assuming that “how” and “why” questions are to be the focus of study, a further distinction among history, case study, and experiment is the extent of the investigator’s control over and access to actual behavioral events. Histories are the preferred method when there is virtually no access or control. The distinctive contribution of the historical method is in dealing with the “dead” past— that is, when no relevant persons are alive to report, even retrospectively, what occurred and when an investigator must rely on primary documents, secondary documents, and cultural and physical artifacts as the main sources of evidence. Histories can, of course, be done about contemporary events; in this situation, the method begins to overlap with that of the case study.
The case study is preferred in examining contemporary events, but when the relevant behaviors cannot be manipulated. The case study relies on many of the same techniques as a history, but it adds two sources of evidence not usually included in the historian’s repertoire: direct observation of the events being studied and interviews of the persons involved in the events. Again, although case studies and histories can overlap, the case study’s unique strength is its ability to deal with a full variety of evidence—documents, artifacts, interviews, and observations—beyond what might be available in a conventional historical study. Moreover, in some situations, such as participant-observation (see Chapter 4), informal manipulation can occur.
Finally, experiments are done when an investigator can manipulate behavior directly, precisely, and systematically. This can occur in a laboratory setting, in which an experiment may focus on one or two isolated variables (and presumes that the laboratory environment can “control” for all the remaining variables beyond the scope of interest), or it can be done in a field setting, where the term field or social experiment has emerged to cover research where investigators “treat” whole groups of people in different ways, such as providing them with different kinds of vouchers to purchase services (Boruch & Foley, 2000). Again, the methods overlap. The full range of experimental science also includes those situations in which the experimenter cannot manipulate behavior but in which the logic of experimental design still may be applied. These situations have been commonly regarded as “quasi-experimental” situations (e.g., D. T. Campbell & Stanley, 1966; Cook & Campbell, 1979) or “observational” studies (e.g., P. R. Rosenbaum, 2002). The quasi-experimental approach even can be used in a historical setting, where, for instance, an investigator may be interested in studying race riots or lynchings (see Spilerman, 1971) and use a quasi-experimental design because no control over the behavioral event was possible. In this case, the experimental method begins to overlap with histories.
In the field of evaluation research, Boruch and Foley (2000) have made a compelling argument for the practicality of one type of field experiment—randomized field trials. The authors maintain that the field trials design, emulating the design of laboratory experiments, can be and has been used even when evaluating complex community initiatives. However, you should be cautioned about the possible limitations of this design.
In particular, the design may work well when, within a community, individual consumers or users of services are the unit of analysis. Such a situation would exist if a community intervention consisted, say, of a health promotion campaign and the outcome of interest was the incidence of certain illnesses among the community’s residents. The random assignment might designate a few communities to have the campaign, compared to a few that did not, and the outcomes would compare the condition of the residents in both sets of communities.
In many community studies, however, the outcomes of interest and therefore the appropriate unit of analysis are at the community or collective level and not at the individual level. For instance, efforts to upgrade neighborhoods may be concerned with improving a neighborhood’s economic base (e.g., the number of jobs per residential population). Now, although the candidate communities still can be randomly assigned, the degrees of freedom in any later statistical analysis are limited by the number of communities rather than the number of residents. Most field experiments will not be able to support the participation of a sufficiently large number of communities to overcome the severity of the subsequent statistical constraints.
The limitations when communities or collective entities are the unit of analysis are extremely important because many public policy objectives focus on the collective rather than individual level. For instance, the thrust of federal education policy in the early 2000s focused on school performance. Schools were held accountable for year-to-year performance even though the composition of the students enrolled at the schools changed each year. Creating and implementing a field trial based on a large number of schools, as opposed to a large number of students, would present an imposing challenge and the need for extensive research resources. In fact, Boruch (2007) found that a good number of the randomized field trials inadvertently used the incorrect unit of analysis (individuals rather than collectives), thereby making the findings from the trials less usable.
Field experiments with a large number of collective entities (e.g., neighborhoods, schools, or organizations) also raise a number of practical challenges:
- any randomly selected control sites may adopt important components of the intervention of interest before the end of the field experiment and no longer qualify as “no-treatment” sites;
- the funded intervention may call for the experimental communities to reorganize their entire manner of providing certain services—that is, a “systems” change— thereby creating site-to-site variability in the unit of assignment (the experimental design assumes that the unit of assignment is the same at every site, both intervention and control);
- the same systems change aspect of the intervention also may mean that the organizations or entities administering the intervention may not necessarily remain stable over the course of time (the design requires such stability until the random field trials have been completed); and
- the experimental or control sites may be unable to continue using the same instruments and measures (the design, which will ultimately “group” the data to compare intervention sites as a group with comparison sites as a second group, requires common instruments and measures across sites).
The existence of any of these conditions will likely lead to the need to find alternatives to randomized field trials.
Summary. You should be able to identify some situations in which all research methods might be relevant (such as exploratory research) and other situations in which two methods might be considered equally attractive. You also can use multiple methods in any given study (for example, a survey within a case study or a case study within a survey). To this extent, the various methods are not mutually exclusive. But you should also be able to identify some situations in which a specific method has a distinct advantage. For the case study, this is when
- A “how” or “why” question is being asked about
- a contemporary set of events,
- over which the investigator has little or no control.
To determine the questions that are most significant for a topic, as well as to gain some precision in formulating these questions requires much preparation. One way is to review the literature on the topic (Cooper, 1984). Note that such a literature review is therefore a means to an end, and not—as many people have been taught to think—an end in itself. Novices may think that the purpose of a literature review is to determine the answers about what is known on a topic; in contrast, experienced investigators review previous research to develop sharper and more insightful questions about the topic.
2. Traditional Prejudices against the Case Study Method
Although the case study is a distinctive form of empirical inquiry, many research investigators nevertheless disdain the strategy. In other words, as a research endeavor, case studies have been viewed as a less desirable form of inquiry than either experiments or surveys. Why is this?
Perhaps the greatest concern has been over the lack of rigor of case study research. Too many times, the case study investigator has been sloppy, has not followed systematic procedures, or has allowed equivocal evidence or biased views to influence the direction of the findings and conclusions. Such lack of rigor is less likely to be present when using the other methods—possibly because of the existence of numerous methodological texts providing investigators with specific procedures to be followed. In contrast, only a small (though increasing) number of texts besides the present one cover the case study method in similar fashion.
The possibility also exists that people have confused case study teaching with case study research. In teaching, case study materials may be deliberately altered to demonstrate a particular point more effectively (e.g., Garvin, 2003). In research, any such step would be strictly forbidden. Every case study investigator must work hard to report all evidence fairly, and this book will help her or him to do so. What is often forgotten is that bias also can enter into the conduct of experiments (see Rosenthal, 1966) and the use of other research methods, such as designing questionnaires for surveys (Sudman & Bradbum, 1982) or conducting historical research (Gottschalk, 1968). The problems are not different, but in case study research, they may have been more frequently encountered and less frequently overcome.
EXERCISE 1.3 Examining Case Studies Used for Teaching Purposes
Obtain a copy of a case study designed for teaching purposes (e.g., a case in a textbook used in a business school course). Identify the specific ways in which this type of “teaching” case is different from research case studies.
Does the teaching case cite primary documents, contain evidence, or display data? Does the teaching case have a conclusion? What appears to be the main objective of the teaching case?
A second common concern about case studies is that they provide little basis for scientific generalization. “How can you generalize from a single case?” is a frequently heard question. The answer is not simple (Kennedy, 1976). However, consider for the moment that the same question had been asked about an experiment: “How can you generalize from a single experiment?” In fact, scientific facts are rarely based on single experiments; they are usually based on a multiple set of experiments that have replicated the same phenomenon under different conditions. The same approach can be used with multiple-case studies but requires a different concept of the appropriate research designs, discussed in detail in Chapter 2. The short answer is that case studies, like experiments, are generalizable to theoretical propositions and not to populations or universes. In this sense, the case study, like the experiment, does not represent a “sample,” and in doing a case study, your goal will be to expand and generalize theories (analytic generalization) and not to enumerate frequencies (statistical generalization). Or, as three notable social scientists describe in their single case study done years ago, the goal is to do a “generalizing” and not a “particularizing” analysis (Lipset, Trow, & Coleman, 1956, pp. 419-420).2
A third frequent complaint about case studies is that they take too long, and they result in massive, unreadable documents. This complaint may be appropriate, given the way case studies have been done in the past (e.g., Feagin, Orum, & Sjoberg, 1991), but this is not necessarily the way case studies—yours included—must be done in the future. Chapter 6 discusses alternative ways of writing the case study—including ones in which the traditional, lengthy narrative can be avoided altogether. Nor need case studies take a long time. This incorrectly confuses the case study method with a specific method of data collection, such as ethnography (e.g., Fetterman, 1989) or participant-observation (e.g., Jorgensen, 1989). Ethnographies usually require long periods of time in the “field” and emphasize detailed, observational evidence. Participant-observation may not require the same length of time but still assumes a hefty investment of field efforts. In contrast, case studies are a form of inquiry that does not depend solely on ethnographic or participant-observer data. You could even do a valid and high-quality case study without leaving the telephone or Internet, depending upon the topic being studied.
A fourth possible objection to case studies has seemingly emerged with the renewed emphasis, especially in education and related research, on randomized field trials or “true experiments.” Such studies aim to establish causal relationships—that is, whether a particular “treatment” has been efficacious in producing a particular “effect” (e.g., Jadad, 1998). In the eyes of many, the emphasis has led to a downgrading of case study research because case studies (and other types of nonexperimental methods) cannot directly address this issue.
Overlooked has been the possibility that case studies can offer important evidence to complement experiments. Some noted methodologists suggest, for instance, that experiments, though establishing the efficacy of a treatment (or intervention), are limited in their ability to explain “how” or “why” the treatment necessaiily worked, whereas case studies could investigate such issues (e.g., Shavelson & Townes, 2002, pp. 99-106).3 Case studies may therefore be valued “as adjuncts to experiments rather than as alternatives to them” (Cook & Payne, 2002). In clinical psychology, a “large series of single case studies,” confirming predicted behavioral changes after the initiation of treatment, even may provide additional evidence of efficaciousness (e.g., Veerman & van Yperen, 2007).
Despite the fact that these four common concerns can be allayed, as above, one major lesson is that good case studies are still difficult to do. The problem is that we have little way of screening for an investigator’s ability to do good case studies. People know when they cannot play music; they also know when they cannot do mathematics beyond a certain level, and they can be tested for other skills, such as the bar examination in law. Somehow, the skills for doing good case studies have not yet been formally defined. As a result, “most people feel that they can prepare a case study, and nearly all of us believe we can understand one. Since neither view is well founded, the case study receives a good deal of approbation it does not deserve” (Hoaglin, Light, McPeek, Mosteller, & Stoto, 1982, p. 134). This quotation is from a book by five prominent statisticians. Surprisingly, from another field, even they recognize the challenge of doing good case studies.
Source: Yin K Robert (2008), Case Study Research Designs and Methods, SAGE Publications, Inc; 4th edition.
The post Comparing Case Studies with Other Research Methods in the Social Sciences appeared first on HKT Consultant.
]]>The post Different Kinds of Case Studies, but a Common Definition appeared first on HKT Consultant.
]]>1. Definition of the Case Study as a Research Method
The most frequently encountered definitions of case studies have merely repeated the types of topics to which case studies have been applied. For example, in the words of one observer,
The essence of a case study, the central tendency among all types of case study, is that it tries to illuminate a decision or set of decisions: why they were taken, how they were implemented, and with what result. (Schramm, 1971, emphasis added)
This definition thus cites cases of “decisions” as the major focus of case studies. Other common cases include “individuals,” “organizations,” “processes,” “programs,” “neighborhoods,” “institutions,” and even “events.” However, citing a case topic4 is surely insufficient to establish the needed definition of case studies as a research method.
Alternatively, many of the earlier social science textbooks failed to consider the case study a formal research method at all (the major exception is the book by five statisticians from Harvard University—Hoaglin et al., 1982). As discussed previously, one common flaw was to consider the case study as the exploratory stage of some other type of research method, and the case study itself was only mentioned in a line or two of text.
Another definitional flaw has been to confuse case studies with ethnographies or with participant-observation, so that a textbook’s presumed discussion of case studies was in reality a description either of the ethnographic method or of participant-observation as a data collection technique. Many earlier methodological texts (e.g., see L. Kidder & Judd, 1986; Nachmias & Nachmias, 1992), in fact, only covered “fieldwork” as a data collection technique and omitted any further discussion of case studies.
In a historical overview of the case study in American methodological thought, Jennifer Platt (1992) explains the reasons for these treatments. She traces the practice of doing case studies back to the conduct of life histories, the work of the Chicago school of sociology, and casework in social work. She then shows how “participant-observation” emerged as a data collection technique, leaving the further definition of any distinctive case study method in suspension. Finally, she explains how the first edition of this book (1984) definitively dissociated the case study strategy from the limited perspective of only doing participant-observation (or any type of fieldwork). The case study strategy, in her words, begins with “a logic of design … a strategy to be preferred when circumstances and research problems are appropriate rather than an ideological commitment to be followed whatever the circumstances” (Platt, 1992, p. 46).
And just what is this logic of design? The critical features had been worked out prior to the first edition of this book (Yin, 1981a, 1981b) but now may be restated as part of a twofold, technical definition of case studies. The first part begins with the scope of a case study:
- A case study is an empirical inquiry that
-
- investigates a contemporary phenomenon in depth and within its real-life context, especially when
- the boundaries between phenomenon and context are not clearly evident.
In other words, you would use the case study method because you wanted to understand a real-life phenomenon in depth, but such understanding encompassed important contextual conditions—because they were highly pertinent to your phenomenon of study (e.g., Yin & Davis, 2007). This first part of the logic of design therefore helps to continue to distinguish case studies from the other research methods that have been discussed.
An experiment, for instance, deliberately divorces a phenomenon from its context, attehcling to only a few variables (typically, the context is “controlled” by the laboratory environment). A history, by comparison, does deal with the entangled situation between phenomenon and context but usually with noncontemporary events. Finally, surveys can try to deal with phenomenon and context, but their ability to investigate the context is extremely limited. The survey designer, for instance, constantly struggles to limit the number of variables to be analyzed (and hence the number of questions that can be asked) to fall safely within the number of respondents who can be surveyed.
Second, because phenomenon and context are not always distinguishable in real-life situations, other technical characteristics, including data collection and data analysis strategies, now become the second part of our technical definition of case studies:
- The case study inquiry
-
- copes with the technically distinctive situation in which there will be many more variables of interest than data points, and as one result
- relies on multiple sources of evidence, with data needing to converge in a triangulating fashion, and as another result
- benefits from the prior development of theoretical propositions to guide data collection and analysis.
In essence, the twofold definition shows how case study research comprises an all-encompassing method—covering the logic of design, data collection techniques, and specific approaches to data analysis. In this sense, the case study is not limited to being a data collection tactic alone or even a design feature alone (Stoecker, 1991). How the method is practiced is the topic of this entire book.
EXERCISE 1.4 Finding and Analyzing an Existing Case Study from the Literature
Retrieve an example of case study research from the literature. The case study can be on any topic, but it must have used some empirical method and presented some empirical (qualitative or quantitative) data. Why is this a case study? What, if anything, is distinctive about the findings that could not be learned by using some other social science method focusing on the same topic?
Certain other features of the case study method are not critical for defining the method, but they may be considered variations within case study research and also provide answers to common questions.
2. Variations within Case Studies as a Research Method
Yes, case study research includes both single- and multiple-case studies. Though some fields, such as political science and public administration, have tried to distinguish between these two approaches (and have used such terms as the comparative case method as a distinctive form of multiple-case studies; see Agranoff & Radin, 1991; Dion, 1998; Lijphart, 1975), single- and multiple-case studies are in reality but two variants of case study designs (see Chapter 2 for more).
And yes, case studies can include, and even be limited to, quantitative evidence. In fact, any contrast between quantitative and qualitative evidence does not distinguish the various research methods. Note that, as analogous examples, some experiments (such as studies of perceptions) and some survey questions (such as those seeking categorical rather than numerical responses) rely on qualitative and not quantitative evidence. Likewise, historical research can include enormous amounts of quantitative evidence.
As a related but important note, the case study method is not just a form of “qualitative research,” even though it may be recognized among the array of qualitative research choices (e.g., Creswell, 2007). Some case study research goes beyond being a type of qualitative research, by using a mix of quantitative and qualitative evidence. In addition, case studies need not always include the direct and detailed observational evidence marked by other forms of “qualitative research.”
And yes, case studies have a distinctive place in evaluation research (see Cronbach & Associates, 1980; Patton, 2002; U.S. Government Accountability Office, 1990). There are at least four different applications. The most important is to explain the presumed causal links in real-life interventions that are too complex for the survey or experimental strategies. A second application is to describe an intervention and the real-life context in which it occurred. Third, case studies can illustrate certain topics within an evaluation, again in a descriptive mode. Fourth, the case study strategy may be used to enlighten those situations in which the intervention being evaluated has no clear, single set of outcomes. Whatever the application, one constant theme is that program sponsors—rather than research investigators alone—may have the prominent role in defining the evaluation questions and desired data categories (U.S. Government Accountability Office, 1990).
And finally, yes, case studies can be conducted and written with many different motives. These motives vary from the simple presentation of individual cases to the desire to arrive at broad generalizations based on case study evidence but without presenting any of the individual case studies separately (see BOX 3).
BOX 3
Multiple-Case Studies: Case Studies Containing Multiple “Cases”
Case studies can cover multiple cases and then draw a single set of “cross-case” conclusions. The two examples below both focused on a topic of continuing public interest: identifying successful programs to improve U.S. social conditions.
3A. A Cross-Case Analysis following the Presentation of Separate, Single Cases
Jonathan Crane (1998) edited a book that had nine social programs as separate cases. Each case had a different author and was presented in its own chapter. The programs had in common strong evidence of their effectiveness, but they varied widely in their focus—from education to nutrition to drug prevention to preschool programs to drug treatment for delinquent youths. The editor then presents a crossprogram analysis in a final chapter, attempting to draw generalizable conclusions that could apply to many other programs.
3B. A Book Whose Entire Text Is Devoted to the Multiple-Case (“Cross-Case”) Analysis
Lisbeth Schorr’s (1997) book is about major strategies for improving social conditions, illustrated by four policy topics: welfare reform, strengthening the child protection system, education reform, and transforming neighborhoods. The book continually refers to specific cases of successful programs, but these programs do not appear as separate, individual chapters. Also citing data from the literature, the author develops numerous generalizations based on the case studies, including the need for successful programs to be “results oriented.” Similarly, she identifies six other attributes of highly effective programs (also see BOX 41A and 41B, Chapter 6, p. 173).
EXERCISE 1.5 Defining Different Types of Case Studies Used for Research Purposes
Define the three types of case studies used for research (but not teaching) purposes: (a) explanatory or causal case studies, (b) descriptive case studies, and (c) exploratory case studies. Compare the situations in which these different types of case studies would be most applicable. Now name a case study that you would like to conduct. Would it be explanatory, descriptive, or exploratory? Why?
Source: Yin K Robert (2008), Case Study Research Designs and Methods, SAGE Publications, Inc; 4th edition.
The post Different Kinds of Case Studies, but a Common Definition appeared first on HKT Consultant.
]]>The post General Approach to Designing Case Studies appeared first on HKT Consultant.
]]>The development of this research design is a difficult part of doing case studies. Unlike other research methods, a comprehensive “catalog” of research designs for case studies has yet to be developed. There are no textbooks, like those in the biological and psychological sciences, covering such design considerations as the assignment of subjects to different “groups,” the selection of different stimuli or experimental conditions, or the identification of various response measures (see Cochran & Cox, 1957; Fisher, 1935, cited in Cochran & Cox, 1957; Sidowski, 1966). In a laboratory experiment, each of these choices reflects an important logical connection to the issues being studied. Similarly, there are not even textbooks like the well-known volumes by Campbell and Stanley (1966) or by Cook and Campbell (1979) that summarize the various research designs for quasi-experimental situations. Nor have there emerged any common designs—for example, “panel” studies—such as those recognized in doing survey research (see L. Kidder & Judd, 1986, chap. 6).
One pitfall to be avoided, however, is to consider case study designs to be a subset or variant of the research designs used for other methods, such as experiments. For the longest time, scholars incorrectly thought that the case study was but one type of quasi-experimental design (the “one-shot post-test-only” design). This misperception has finally been corrected, with the following statement appearing in a revision on quasi-experimental designs (Cook & Campbell, 1979): “Certainly the case study as normally practiced should not be demeaned by identification with the one-group post-test-only design” (p. 96). In other words, the one-shot, post-test-only design as a quasi-experimental design still may be considered flawed, but the case study has now been recognized as something different. In fact, the case study is a separate research method that has its own research designs.
Unfortunately, case study research designs have not been codified. The following chapter therefore expands on the new methodological ground broken by earlier editions of this book and describes a basic set of research designs for doing single- and multiple-case studies. Although these designs will need to be continually modified and improved in the future, in their present form they will nevertheless help you to design more rigorous and methodologically sound case studies.
1. Definition of Research Designs
Every type of empirical research has an implicit, if not explicit, research design. In the most elementary sense, the design is the logical sequence that connects the empirical data to a study’s initial research questions and, ultimately, to its conclusions. Colloquially, a research design is a logical plan for getting from here to there, where here may be defined as the initial set of questions to be answered, and there is some set of conclusions (answers) about these questions. Between “here” and “there” may be found a number of major steps, including the collection and analysis of relevant data. As a summary definition, another textbook has described a research design as a plan that
guides the investigator in the process of collecting, analyzing, and interpreting observations. It is a logical model of proof that allows the researcher to draw inferences concerning causal relations among the variables under investigation. (Nachmias & Nachmias, 1992, pp. 77-78, emphasis added)
Another way of thinking about a research design is as a “blueprint” for your research, dealing with at least four problems: what questions to study, what data are relevant, what data to collect, and how to analyze the results (Philliber, Schwab, & Samsloss, 1980).
Note that a research design is much more than a work plan. The main purpose of the design is to help to avoid the situation in which the evidence does not address the initial research questions. In this sense, a research design deals with a logical problem and not a logistical problem. As a simple example, suppose you want to study a single organization. Your research questions, however, have to do with the organization’s relationships with other organizations—their competitive or collaborative nature, for example. Such questions can be answered only if you collect information directly from the other organizations and not merely from the one you started with. If you complete your study by examining only one organization, you cannot draw unbiased conclusions about interorganizational partnerships. This is a flaw in your research design, not in your work plan. The outcome could have been avoided if you had developed an appropriate research design in the first place.
2. Components of Research Designs
For case studies, five components of a research design are especially important:
- a study’s questions;
- its propositions, if any;
- its unit(s) of analysis;
- the logic linking the data to the propositions; and
- the criteria for interpreting the findings.
Study questions. This first component has already been described in Chapter 1, which suggested that the form of the question—in terms of “who,” “what,” “where,” “how,” and “why”—provides an important clue regarding the most relevant research method to be used. The case study method is most likely to be appropriate for “how” and “why” questions, so your initial task is to clarify precisely the nature of your study questions in this regard.
More troublesome may be coming up with the substance of the questions. Many students take an initial stab, only to be discouraged when they find the same question(s) already well covered by previous research. Other less desirable questions focus on too trivial or minor parts of an issue. A helpful hint is to move in three stages. In the first, try to use the literature to narrow your interest to a key topic or two, not worrying about any specific research questions. In the second, examine closely—even dissect—a few key studies on your topic of interest. Identify the questions in those few studies and whether they conclude with new questions or loose ends for future research. These may then stimulate your own thinking and imagination, and you may find yourself articulating some potential questions of your own. In the third stage, examine another set of studies on the same topic. They may provide support for your potential questions or even suggest ways of sharpening them.
EXERCISE 2.1 Defining the Boundaries of a Case Study
Select a topic for a case study you would like to do. Identify some research questions to be answered or propositions to be examined by your case study.
How does the naming of these questions or propositions clarify the boundaries of your case study with regard to the time period covered by the case study; the relevant social group, organization, or geographic area; the type of evidence to be collected; and the priorities for data collection and analysis?
Study propositions. As for the second component, each proposition directs attention to something that should be examined within the scope of study. For instance, assume that your research, on the topic of interorganizational partnerships, began with the following question; How and why do organizations collaborate with one another to provide joint services (for example, a manufacturer and a retail outlet collaborating to sell certain computer products)? These “how” and “why” questions, capturing what you are really interested in answering, led yOu to the case study as the appropriate method in the first place. Nevertheless, these “how” and “why” questions do not point to what you should study.
Only if you are forced to state some propositions will you move in the right direction. For instance, you might think that organizations collaborate because they derive mutual benefits. This proposition, besides reflecting an important theoretical issue (that other incentives for collaboration do not exist or are unimportant), also begins to tell you where to look for relevant evidence (to define and ascertain the extent of specific benefits to each organization).
At the same time, some studies may have a legitimate reason for not having any propositions. This is the condition—which exists in experiments, surveys, and the other research methods alike—in which a topic is the subject of “exploration.” Every exploration, however, should still have some purpose. Instead of propositions, the design for an exploratory study should state this purpose, as well as the criteria by which an exploration will be judged successful. Consider the analogy in BOX 4 for exploratory case studies. Can you imagine how you would ask for support from Queen Isabella to do your exploratory study?
BOX 4
“Exploration” as an Analogy for an Exploratory Case Study
When Christopher Columbus went to Queen Isabella to ask for support for his “exploration” of the New World, he had to have some reasons for asking for three ships (Why not one? Why not five?), and he had some rationale for going westward (Why not south? Why not south and then east?). He also had some (mistaken) criteria for recognizing the Indies when he actually encountered it. In short, his exploration began with some rationale and direction, even if his initial assumptions might later have been proved wrong (Wilford, 1992). This same degree of rationale and direction should underlie even an exploratory case study.
Unit of analysis. This third component is related to the fundamental problem of defining what the “case” is—a problem that has plagued many investigators at the outset of case studies (e.g., Ragin & Becker, 1992). For instance, in the classic case study, a “case” may be an individual. Jennifer Platt (1992) has noted how the early case studies in the Chicago school of sociology were life histories of such persons as juvenile delinquents or derelict men. You also can imagine case studies of clinical patients, of exemplary students, or of certain types of leaders. In each situation, an individual person is the case being studied, and the individual is the primary unit of analysis. Information about the relevant individual would be collected, and several such individuals or “cases” might be included in a multiple-case study.
You would still need study questions and study propositions to help identify the relevant information to be collected about this individual or individuals. Without such questions and propositions, you might be tempted to cover “everything” about the individual(s), which is impossible to do. For example, the propositions in studying these individuals might involve the influence of early childhood or the role of peer relationships. Such seemingly general topics nevertheless represent a vast narrowing of the relevant data. The more a case study contains specific questions and propositions, the more it will stay within feasible limits.
Of course, the “case” also can be some event or entity other than a single individual. Case studies have been done about decisions, programs, the implementation process, and organizational change. Feagin et al. (1991) contains some classic examples of these single cases in sociology and political science. Beware of these types of cases—none is easily defined in terms of the beginning or end points of the “case.” For example, a case study of a specific program may reveal (a) variations in program definition, depending upon the perspective of different actors, and (b) program components that preexisted the formal designation of the program. Any case study of such a program would therefore have to confront these conditions in delineating the unit of analysis.
As a general guide, your tentative definition of the unit of analysis (which is the same as the definition of the “case”) is related to the way you have defined your initial research questions. Suppose, for example, you want to study the role of the United States in the global economy. Years ago, Peter Drucker (1986) wrote a provocative essay (not a case study) about fundamental changes in the world economy, including the importance of “capital movements” independent of the flow of goods and services. Using Drucker’s work or some similar theoretical framework, the unit of analysis (or “case”) for your case study might be a country’s economy, an industry in the world marketplace, an economic policy, or the trade or capital flow between countries. Each unit of analysis and its related questions and propositions would call for a slightly different research design and data collection strategy.
Selection of the appropriate unit of analysis will start to occur when you accurately specify your primary research questions. If your questions do not lead to the favoring of one unit of analysis over another, your questions are probably either too vague or too numerous—and you may have trouble doing a case study. However, when you do eventually arrive at a definition of the unit of analysis, do not consider closure permanent. Your choice of the unit of analysis, as with other facets of your research design, can be revisited as a result of discoveries during your data collection (see discussion and cautions about flexibility throughout this book and at the end of this chapter).
Sometimes, the unit of analysis may have been defined one way, even though the phenomenon being studied actually follows a different definition. Most frequently, investigators have confused case studies of neighborhoods with case studies of small groups (as another example, confusing a new technology with the workings of an engineering team in an organization; see BOX 5A). How a geographic area such as a neighborhood copes with racial transition, upgrading, and other phenomena can be quite different from how a small group copes with these same phenomena. For instance, Street Comer Society (Whyte, 1943/1955; see BOX 2A in Chapter 1 of this book) and Tally’s Comer (Liebow, 1967; see BOX 9, this chapter) often have been mistaken for being case studies of neighborhoods when in fact they are case studies of small groups (note that in neither book is the neighborhood geography described, even though the small groups lived in a small area with clear neighborhood implications). BOX 5B, however, presents a good example of how units of analyses can be defined in a more discriminating manner—in the field of world trade.
BOX 5
Defining the Unit of Analysis 5 A. What Is the Unit of Analysis?
5A. What Is the Unit of Analysis?
The Soul of a New Machine (1981) was a Pulitzer Prize-winning book by Tracy Kidder. The book, also a best seller, is about the development of a new minicomputer, produced by Data General Corporation, intended to compete with one produced by a direct competitor, Digital Equipment Corporation (also see BOX 28, Chapter 5, p. 142).
This easy-to-read book describes how Data General’s engineering team invented and developed the new computer. The book begins with the initial conceptualization of the computer and ends when the engineering team relinquishes control of the machine to Data General’s marketing staff.
The book is an excellent example of a case study. However, the book also illustrates a fundamental problem in doing case studies—that of defining the unit of analysis. Is the “case” being studied the minicomputer, or is it about the dynamics of a small group—the engineering team? The answer is critical for understanding how the case study might relate to any broader body of knowledge—that is, whether to generalize to a technology topic or to a group dynamics topic. Because the book is not an academic study, it does not need to, nor does it, provide an answer.
5B. A Clearer Choice among Units of Analysis
Ira Magaziner and Mark Patinkin’s (1989) book, The Silent War: Inside the Global Business Battles Shaping America’s Future, presents nine individual case studies (also see BOX 35, Chapter 5, p. 161). Each case helps the reader to understand a real-life situation of international economic competition.
Two of the cases appear similar but in fact have different main units of analysis. One case, about the Korean firm Samsung, is a case study of the critical policies that make the firm competitive. Understanding Korean economic development is part of the context, and the case study also contains an embedded unit—Samsung’s development of the microwave oven as an illustrative product. The other case, about the development of an Apple computer factory in Singapore, is in fact a case study of Singapore’s critical policies that make the country competitive. The Apple computer factory experience—an embedded unit of analysis—is actually an illustrative example of how the national policies affected foreign investments.
These two cases show how the definition of the main and embedded units of analyses, as well as the definition of the contextual events surrounding these units, depends on the level of inquiry. The main unit of analysis is likely to be at the level being addressed by the main study questions.
Most investigators will encounter this type of confusion in defining the unit of analysis or “case.” To reduce the confusion, one recommended practice is to discuss the potential case with a colleague. Try to explain to that person what questions you are trying to answer and why you have chosen a specific case or group of cases as a way of answering those questions. This may help you to avoid incorrectly identifying the unit of analysis.
Once the general definition of the case has been established, other clarifications in the unit of analysis become important. If the unit of analysis is a small group, for instance, the persons to be included within the group (the immediate topic of the case study) must be distinguished from those who are outside it (the context for the case study). Similarly, if the case is about local services in a specific geographic area, you need to decide which services to cover. Also desirable, for almost any topic that might be chosen, are specific time boundaries to define the beginning and end of the case (e.g., whether to include the entire or only some part of the life cycle of the entity that is to be the case). Answering all of these types of questions will help to determine the scope of your data collection and, in particular, how you will distinguish data about the subject of your case study (the “phenomenon”) from data external to the case (the “context”).
These latter cautions regarding the need for spatial, temporal, and other concrete boundaries underlie a key but subtle aspect in defining your case. The desired case should be some real-life phenomenon, not an abstraction such as a topic, an argument, or even a hypothesis. These abstractions, absent the identification of specific examples or cases, would rightfully serve as the subjects of research studies using other kinds of methods but not case studies. To justify using the case study method, you need to go one step further: You need to define a specific, real-life “case” to represent the abstraction. (For examples of more concrete and less concrete case study topics, see Figure 2.1.)
Take the concept of “neighboring.” Alone, it could be the subject of research studies using methods other than the case study method. The other methods might include a survey of the relationships among neighbors, a history of the evolution of the sense of neighboring and the setting of boundaries, or an experiment in which young children do tasks next to each other to determine the distracting effects, if any, of their neighbors. These examples show how the abstract concept of “neighboring” does not alone produce the grounds for a case study. However, the concept could readily become a case study topic if it were accompanied by your selecting a specific neighborhood (“case”) to be studied and posing study questions and propositions about the neighborhood in relation to the concept of “neighboring.”
One final point pertains to the role of the available research literature and needs to be made about defining the case and the unit of analysis. Most researchers will want to compare their findings with previous research. For this reason, the key definitions used in your study should not be idiosyncratic. Rather, each case study and unit of analysis either should be similar to those previously studied by others or should innovate in clear, operationally defined ways. In this manner, the previous literature also can become a guide for defining the case and unit of analysis.
EXERCISE 2.2 Defining the Unit of Analysis (and the “Case”) for a Case Study
Examine Figure 2.1. Discuss each subject, which illustrates a different unit of analysis. Find a published case study on at least one of these subjects, indicating the actual “case” that was being studied. Understanding that each subject illustrates a different unit of analysis and involves the selection of different cases to be studied, do you think that the more concrete units might be easier to define than the less concrete ones? Why?
Linking data to propositions and criteria for interpreting the findings. The fourth and fifth components have been increasingly better developed in doing case studies. These components foreshadow the data analysis steps in case study research. Because the analytic techniques and choices are covered in
detail in Chapter 5, your main concern during the design phase is to be aware of the main choices and how they might suit your case study. In this way, your research design can create a more solid foundation for the later analysis.
All of the analytic techniques in Chapter 5 represent ways of linking data to propositions: pattern matching, explanation building, time-series analysis, logic models, and cross-case synthesis. The actual analyses will require that you combine or calculate your case study data as a direct reflection of your initial study propositions. For instance, knowing that some or all of your propositions cover a temporal sequence would mean that you might eventually use some type of time-series analysis. Noting this strong likelihood during the design phase would call your attention to the need to be sure you had sufficient procedures to collect time markers as part of your data collection plans.
If you have had limited experience in conducting empirical studies, you will not easily identify the likely analytic technique(s) or anticipate the needed data to use the techniques to their full advantage. More experienced researchers will note how often they have either (a) collected too much data that were not later used in any analysis or (b) collected too little data that prevented the proper use of a desired analytic technique. Sometimes, the latter situation even may force researchers to return to their data collection phase (if they can), to supplement the original data. The more you can avoid any of these situations, the better off you will be.
Criteria for interpreting a study’s findings. Statistical analyses offer some explicit criteria for such interpretations. For instance, by convention, social science considers a p level of less than .05 to demonstrate that observed differences were “statistically significant.” However, much case study analysis will not rely on the use of statistics and therefore calls attention to other ways of thinking about such criteria.
A major and important alternative strategy is to identify and address rival explanations for your findings. Again, Chapter 5 discusses this strategy and how it works more fully. At the design stage of your work, the challenge is to anticipate and enumerate the important rivals, so you will include information about them as part of your data collection. If you only think of rival explanations after data collection has been completed, you will be starting to justify and design a future study, but you will not be helping to complete your current case study. For this reason, specifying important rival explanations is a part of a case study’s research design work.
Summary. A research design should include five components. Although the current state of the art does not provide detailed guidance on the last two, the complete research design should indicate what data are to be collected—as indicated by a study’s questions, its propositions, and its units of analysis. The design also should tell you what is to be done after the data have been collected—as indicated by the logic linking the data to the propositions and the criteria for interpreting the findings.
3. The Role of Theory in Design Work
Covering these preceding five components of research designs will effectively force you to begin constructing a preliminary theory related to your topic of study. This role of theory development, prior to the conduct of any data collection, is one point of difference between case studies and related methods such as ethnography (Lincoln & Guba, 1985; Van Maanen, 1988) and “grounded theory” (Corbin & Strauss, 2007). Typically, these related methods deliberately avoid specifying any theoretical propositions at the outset of an inquiry. As a result, students confusing these methods with case studies wrongly think that, by having selected the case study method, they can proceed quickly into the data collection phase of their work, and they may have been encouraged to make their “field contacts” as quickly as possible. No guidance could be more misleading. Among other considerations, the relevant field contacts depend upon an understanding—or theory—of what is being studied.
Theory development. For case studies, theory development as part of the design phase is essential, whether the ensuing case study’s purpose is to develop or to test theory. Using a case study on the implementation of a new management information system (MIS) as an example (Markus, 1983), the simplest ingredient of a theory is a statement such as the following:
The case study will show why implementation only succeeded when the organization was able to re-structure itself, and not just overlay the new MIS on the old organizational structure. (Markus, 1983)
The statement presents the nutshell of a theory of MIS implementation— that is, that organizational restructuring is needed to make MIS implementation work.
Using the same case, an additional ingredient might be the following statement:
The case study will also show why the simple replacement of key persons was not sufficient for successful implementation. (Markus, 1983)
This second statement presents the nutshell of a rival theory—that is, that MIS implementation fails because of the resistance to change on the part of individual people and that the replacement of such people is the main requirement for implementation to succeed.
You can see that as these two initial ingredients are elaborated, the stated ideas will increasingly cover the questions, propositions, units of analysis, logic connecting data to propositions, and criteria for interpreting the findings—that is, the five components of the needed research design. In this sense, the complete research design embodies a “theory” of what is being studied.
This theory should by no means be considered with the formality of grand theory in social science, nor are you being asked to be a masterful theoretician. Rather, the simple goal is to have a sufficient blueprint for your study, and this requires theoretical propositions, usefully noted by Sutton and Staw (1995) as “a [hypothetical] story about why acts, events, structure, and thoughts occur” (p. 378). Then, the complete research design will provide surprisingly strong guidance in determining what data to collect and the strategies for analyzing the data. For this reason, theory development prior to the collection of any case study data is an essential step in doing case studies. As noted for nonexperi- mental studies more generally, a more elaborate theory desirably points to a more complex pattern of expected results (R R. Rosenbaum, 2002, pp. 5-6 and 277-279). The benefit is a stronger design and a heightened ability to interpret your eventual data.
However, theory development takes time and can be difficult (Eisenhardt, 1989). For some topics, existing works may provide a rich theoretical framework for designing a specific case study. If you are interested in international economic development, for instance, Peter Drucker’s (1986) “The Changed World Economy” is an exceptional source of theories and hypotheses. Drucker claims that the world economy has changed significantly from the past. He points to the “uncoupling” between the primary products (raw materials) economy and the industrial economy, a similar uncoupling between low labor costs and manufacturing production, and the uncoupling between financial markets and the real economy of goods and services. To test these propositions might require different studies, some focusing on the different uncouplings, others focusing on specific industries, and yet others explaining the plight of specific countries. Each different study would likely call for a different unit of analysis. Drucker’s theoretical framework would provide guidance for designing these studies and even for collecting relevant data.
In other situations, the appropriate theory may be a descriptive theory (see BOX 2A in Chapter 1 for another example), and your concern should focus on such issues as (a) the purpose of the descriptive effort, (b) the full but realistic range of topics that might be considered a “complete” description of what is to be studied, and (c) the likely topic(s) that will be the essence of the description. Good answers to these questions, including the rationales underlying the answers, will help you go a long way toward developing the needed theoretical base—and research design—for your study.
For yet other topics, the existing knowledge base may be poor, and the available literature will provide no conceptual framework or hypotheses of note. Such a knowledge base does not lend itself to the development of good theoretical statements, and any new empirical study is likely to assume the characteristic of an “exploratory” study. Nevertheless, as noted earlier with the illustrative case in BOX 4, even an exploratory case study should be preceded by statements about what is to be explored, the purpose of the exploration, and the criteria by which the exploration will be judged successful.
Overall, you may want to gain a richer understanding of how theory is used in case studies by reviewing specific case studies that have been successfully completed. For instance, Yin (2003, chap. 1) shows how theory was used in exploratory, descriptive, and explanatory situations by discussing five actual case studies.
Illustrative types of theories. In general, to overcome the barriers to theory development, you should try to prepare for your case study by doing such things as reviewing the literature related to what you would like to study (also see Cooper, 1984), discussing your topic and ideas with colleagues or teachers, and asking yourself challenging questions about what you are studying, why you are proposing to do the study, and what you hope to learn as a result of the study.
As a further reminder, you should be aware of the full range of theories that might be relevant to your study. For instance, note that the MIS example illustrates MIS “implementation” theory and that this is but one type of theory that can be the subject of study. Other types of theories for you to consider include
- individual theories—for example, theories of individual development, cognitive behavior, personality, learning and disability, individual perception, and interpersonal interactions;
- group theories—for example, theories of family functioning, informal groups, work teams, supervisory-employee relations, and interpersonal networks;
- organizational theories—for example, theories of bureaucracies, organizational structure and functions, excellence in organizational performance, and interorga- nizational partnerships; and
- societal theories—for example, theories of urban development, international behavior, cultural institutions, technological development, and marketplace functions.
Other examples cut across these illustrative types. Decision-making theory (Carroll & Johnson, 1992), for instance, can involve individuals, organizations, or social groups. As another example, a common topic of case studies is the evaluation of publicly supported programs, such as federal, state, or local programs. In this situation, the development of a theory of how a program is supposed to work is essential to the design of the evaluation. In this situation, Bickman (1987) reminds us that the theory needs to distinguish between the substance of the program (e.g., how to make education more effective) and the process of program implementation (e.g., how to install an effective program). The distinction would avoid situations where policy makers might want to know the desired substantive remedies (e.g., findings about a newly effective curriculum) but where an evaluation unfortunately focused on managerial issues (e.g., the need to hire a good project director). Such a mismatch can be avoided by giving closer attention to the substantive theory.
Generalizing from case study to theory. Theory development does not only facilitate the data collection phase of the ensuing case study. The appropriately developed theory also is the level at which the generalization of the case study results will occur. This role of theory has been characterized throughout this book as “analytic generalization” and has been contrasted with another way of generalizing results, known as “statistical generalization.” Understanding the distinction between these two types of generalization may be your most important challenge in doing case studies.
Let us first take the more commonly recognized way of generalizing—statistical generalization—although it is the less relevant one for doing case studies. In statistical generalization, an inference is made about a population (or universe) on the basis of empirical data collected about a sample from that universe. This is shown as a Level One inference in Figure 2.2.1 This method of generalizing is commonly recognized because research investigators have ready access to quantitative formulas for determining the confidence with which generalizations can be made, depending mostly upon the size and internal variation within the universe and sample. Moreover, this is the most common way of generalizing when doing surveys (e.g., Fowler, 1988; Lavrakas, 1987) or analyzing archival data.
A fatal flaw in doing case studies is to conceive of statistical generalization as the method of generalizing the results of your case study. This is because your cases are not “sampling units” and should not be chosen for this reason. Rather, individual case studies are to be selected as a laboratory investigator selects the topic of a new experiment. Multiple cases, in this sense, resemble multiple experiments. Under these circumstances, the mode of generalization is analytic generalization, in which a previously developed theory is used as a template with which to compare the empirical results of the case study.2 If two or more cases are shown to support the same theory, replication may be claimed. The empirical results may be considered yet more potent if two or more cases support the same theory but do not support an equally plausible, rival theory. Graphically, this type of generalization is shown as a Level Two inference in Figure 2.2.
Analytic generalization can be used whether your case study involves one or several cases, which shall be later referenced as single-case or multiple-case studies. Furthermore, the logic of replication and the distinction between statistical and analytic generalization will be covered in greater detail in the discussion of multiple-case study designs. The main point at this juncture is that you should try to aim toward analytic generalization in doing case studies, and you should avoid thinking in such confusing terms as “the sample of cases” or the “small sample size of cases,” as if a single-case study were like a single respondent in a survey or a single subject in an experiment. In other words, in terms of Figure 2.2, you should aim for Level Two inferences when doing case studies.
Because of the importance of this distinction between the two ways of generalizing, you will find repeated examples and discussion throughout the remainder of this chapter as well as in Chapter 5.
Source: Yin K Robert (2008), Case Study Research Designs and Methods, SAGE Publications, Inc; 4th edition.
The post General Approach to Designing Case Studies appeared first on HKT Consultant.
]]>The post Criteria for Judging the Quality of Research Designs appeared first on HKT Consultant.
]]>Four tests, however, have been commonly used to establish the quality of any empirical social research. Because case studies are one form of such research, the four tests also are relevant to case studies. An important innovation of this book is the identification of several tactics for dealing with these four tests when doing case studies. Figure 2.3 lists the four widely used tests and the recommended case study tactics, as well as a cross-reference to the phase of research when the tactic is to be used. (Each tactic is described in detail in the referenced chapter of this book.)
Because the four tests are common to all social science methods, the tests have been summarized in numerous textbooks (see L. Kidder & Judd, 1986, pp. 26-29):
- Construct validity: identifying correct operational measures for the concepts being studied
- Internal validity (for explanatory or causal studies only and not for descriptive or exploratory studies): seeking to establish a causal relationship, whereby certain conditions are believed to lead to other conditions, as distinguished from spurious relationships
- External validity: defining the domain to which a study’s findings can be generalized
- Reliability: demonstrating that the operations of a study—such as the data collection procedures—can be repeated, with the same results
Each item on this list deserves explicit attention. For case studies, an important revelation is that the several tactics to be used in dealing with these tests should be applied throughout the subsequent conduct of the case study, not just at its beginning. Thus, the “design work” for case studies may actually continue beyond the initial design plans.
1. Construct Validity
This first test is especially challenging in case study research. People who have been critical of case studies often point to the fact that a case study investigator fails to develop a sufficiently operational set of measures and that “subjective” judgments are used to collect the data.3 Take an example such as studying “neighborhood change”—a common case study topic (e.g., Bradshaw, 1999; Keating & Krumholz, 1999).
Over the years, concerns have arisen over how certain urban neighborhoods have changed their character. Any number of case studies has examined the types of changes and their consequences. However, without any prior specification of the significant, operational events that constitute “change,” a reader cannot tell whether the claimed changes in a case study genuinely reflect the events in a neighborhood or whether they happen to be based on an investigator’s impressions only.
Neighborhood change can cover a wide variety of phenomena: racial turnover, housing deterioration and abandonment, changes in the pattern of urban services, shifts in a neighborhood’s economic institutions, or the turnover from low- to middle-income residents in revitalizing neighborhoods. The choice of whether to aggregate blocks, census tracts, or larger areas also can produce different results (Hipp, 2007).
To meet the test of construct validity, an investigator must be sure to cover two steps:
- define neighborhood change in terms of specific concepts (and relate them to the original objectives of the study) and
- identify operational measures that match the concepts (preferably citing published studies that make the same matches).
For example, suppose you satisfy the first step by stating that you plan to study neighborhood change by focusing on trends in neighborhood crime. The second step now demands that you select a specific measure, such as police- reported crime (which happens to be the standard measure used in the FBI Uniform Crime Reports) as your measure of crime. The literature will indicate certain known shortcomings in this measure, mainly that unknown proportions of crimes are not reported to the police. You will then need to discuss how the shortcomings nevertheless will not bias your study of neighborhood crime and hence neighborhood change.
As Figure 2.3 shows, three tactics are available to increase construct validity when doing case studies. The first is the use of multiple sources of evidence, in a manner encouraging convergent lines of inquiry, and this tactic is relevant during data collection (see Chapter 4). A second tactic is to establish a chain of evidence, also relevant during data collection (also Chapter 4). The third tactic is to have the draft case study report reviewed by key informants (a procedure described further in Chapter 6).
2. Internal Validity
This second test has been given the greatest attention in experimental and quasi-experimental research (see Campbell & Stanley, 1966; Cook & Campbell, 1979). Numerous “threats” to validity have been identified, mainly dealing with spurious effects. However, because so many textbooks already cover this topic, only two points need to be made here.
First, internal validity is mainly a concern for explanatory case studies, when an investigator is trying to explain how and why event x led to event y. If the investigator incorrectly concludes that there is a causal relationship between x and y without knowing that some third factor—z,—may actually have caused y, the research design has failed to deal with some threat to internal validity. Note that this logic is inapplicable to descriptive or exploratory studies (whether the studies are case studies, surveys, or experiments), which are not concerned with this kind of causal situation.
Second, the concern over internal validity, for case study research, extends to the broader problem of making inferences. Basically, a case study involves an inference every time an event cannot be directly observed. An investigator will “infer” that a particular event resulted from some earlier occurrence, based on interview and documentary evidence collected as part of the case study. Is the inference correct? Have all the rival explanations and possibilities been considered? Is the evidence convergent? Does it appear to be airtight? A research design that has anticipated these questions has begun to deal with the overall problem of making inferences and therefore the specific problem of internal validity.
However, the specific tactics for achieving this result are difficult to identify. This is especially true in doing case studies. As one set of suggestions, Figure 2.3 shows that the analytic tactic of pattern matching, described further in Chapter 5, is one way of addressing internal validity. Three other analytic tactics, explanation building, addressing rival explanations, and using logic models, also are described in Chapter 5.
3. External Validity
The third test deals with the problem of knowing whether a study’s findings are generalizable beyond the immediate case study. In the simplest example, if a study of neighborhood change focused on one neighborhood, are the results applicable to another neighborhood? The external validity problem has been a major barrier in doing case studies. Critics typically state that single cases offer a poor basis for generalizing. However, such critics are implicitly contrasting the situation to survey research, in which a sample is intended to generalize to a larger universe. This analogy to samples and universes is incorrect when dealing with case studies. Survey research relies on statistical generalization, whereas case studies (as with experiments) rely on analytic generalization. In analytical generalization, the investigator is striving to generalize a particular set of results to some broader theory (see three examples in BOX 6).
For example, the theory of neighborhood change that led to a case study in the first place is the same theory that will help to identify the other cases to which the results are generalizable. If a study had focused on population transition in an urban neighborhood (e.g., Flippen, 2001), the procedure for selecting a neighborhood for study would have begun with identifying a neighborhood within which the hypothesized transitions were occurring. Theories about transition would then be the domain to which the results could later be generalized.
BOX 6
How Case Studies Can Be Generalized to Theory: Three Examples
6A. The Origins of Social Class Theory
The first example is about the uncovering and labeling of a social class structure based on a case study of a typical American city, Yankee City (Warner & Lunt, 1941). This classic case study in sociology made a critical contribution to social stratification theory and an understanding of social differences among “upper,” “upper- middle,” “middle-middle,” “upper-lower,” and “lower” classes.
6B. Contributions to Urban Planning Theory
The second example is Jane Jacobs and her famous book, The Death and Life of Great American Cities (1961). The book is based mostly on experiences from a single case, New York City. However, the chapter topics, rather than reflecting the single experiences of New York, cover broader theoretical issues in urban planning, such as the role of sidewalks, the role of neighborhood parks, the need for primary mixed uses, the need for small blocks, and the processes of slumming and unslumming. In the aggregate, these issues in fact represent Jacobs’s building of a theory of urban planning.
Jacobs’s book created heated controversy in the planning profession. As a partial result, new empirical inquiries were made in other locales, to examine one or another facet of her rich and provocative ideas. Her theory, in essence, became the vehicle for examining other cases, and the theory still stands as a significant contribution to the field of urban planning.
6C. A More Contemporary Example
A third example covers a 5-year ethnographic study of a single neighborhood at the edge of Chicago (Carr, 2003). The study shows how the neighborhood successfully thwarted undesirable youth-related crime. The experience, in the author’s view, challenged existing theories claiming that strong social ties are crucial to effective neighborhood control. Instead, the author offers newer theories of informal social control that he believes may be especially pertinent to youth crime prevention in contemporary suburban neighborhoods.
The generalization is not automatic, however. A theory must be tested by replicating the findings in a second or even a third neighborhood, where the theory has specified that the same results should occur. Once such direct replications have been made, the results might be accepted as providing strong support for the theory, even though further replications had not been performed. This replication logic is the same that underlies the use of experiments (and allows scientists to cumulate knowledge across experiments). The logic will be discussed further in this chapter in the section on multiple-case designs.
4. Reliability
Most people are probably already familiar with this final test. The objective is to be sure that, if a later investigator followed the same procedures as described by an earlier investigator and conducted the same case study all over again, the later investigator should arrive at the same findings and conclusions. (Note that the emphasis is on doing the same case over again, not on “replicating” the results of one case by doing another case study.) The goal of reliability is to minimize the errors and biases in a study.
One prerequisite for allowing this other investigator to repeat an earlier case study is the need to document the procedures followed in the earlier case. Without such documentation, you could not even repeat your own work (which is another way of dealing with reliability). In the past, case study research procedures have been poorly documented, making external reviewers suspicious of the reliability of the case study method.4 Figure 2.3 indicates two specific tactics to overcome these shortcomings—the use of a case study protocol to deal with the documentation problem in detail (discussed in Chapter 3) and the development of a case study database (discussed in Chapter 4).
The general way of approaching the reliability problem is to make as many steps as operational as possible and to conduct research as if someone were always looking over your shoulder. Accountants and bookkeepers always are aware that any calculations must be capable of being audited. In this sense, an auditor also is performing a reliability check and must be able to produce the same results if the same procedures are followed. A good guideline for doing case studies is therefore to conduct the research so that an auditor could in principle repeat the procedures and arrive at the same results.
Source: Yin K Robert (2008), Case Study Research Designs and Methods, SAGE Publications, Inc; 4th edition.
The post Criteria for Judging the Quality of Research Designs appeared first on HKT Consultant.
]]>The post Case Study Designs appeared first on HKT Consultant.
]]>
1. What Are the Potential Single-Case Designs (Types 1 and 2)?
Rationale for single-case designs. A primary distinction in designing case studies is between single- and multiple-case designs. This means the need for a decision, prior to any data collection, on whether a single case or multiple cases are going to be used to address the research questions. The single-case study is an appropriate design under several circumstances, and five rationales are given below. Recall that a single-case study is analogous to a single experiment, and many of the same conditions that justify a single experiment also justify a single-case study.
One rationale for a single case is when it represents the critical case in testing a well-formulated theory (again, note the analogy to the critical experiment). The theory has specified a clear set of propositions as well as the circumstances within which the propositions are believed to be true. A single case, meeting all of the conditions for testing the theory, can confirm, challenge, or extend the theory. The single case can then be used to determine whether a theory’s propositions are correct or whether some alternative set of explanations might be more relevant. In this manner, like Graham Allison’s comparison of three theories and the Cuban missile crisis (described in Chapter 1, BOX 2), the single case can represent a significant contribution to knowledge and theory building. Such a study can even help to refocus future investigations in an entire field. (See BOX 7 for another example, in the field of organizational innovation.)
A second rationale for a single case is where the case represents an extreme case or a unique case. Either of these situations commonly occurs in clinical psychology, where a specific injury or disorder may be so rare that any single case is worth documenting and analyzing. For instance, one rare clinical syndrome is the inability of certain clinical patients to recognize familiar faces. Given visual cues alone, such patients are unable to recognize loved ones, friends, pictures of famous people, or (in some cases) their own image in a mirror. This syndrome appears to be due to some physical injury to the brain. Yet the syndrome occurs so rarely that scientists have been unable to establish any common patterns (Yin, 1970, 1978). In such circumstances, the single-case study is an appropriate research design whenever a new person with this syndrome—known as prosopagnosia—is encountered. The case study would document the person’s abilities and disabilities, determine the precise nature of the face recognition deficit, but also ascertain whether related disorders exist.
BOX 7
The Critical Case as a Single-Case Study
One rationale for selecting a single-case rather than a multiple-case design is that the single case can represent the critical test of a significant theory. Cross, Bernstein, and Giacquinta (1971) used such a design by focusing on a single school in their book. Implementing Organizational Innovations (also see BOX 19B, Chapter 3, p. 110).
The school was selected because it had a prior history of innovation and could not be claimed to suffer from “barriers to innovation.” In the prevailing theories, such barriers had been prominently cited as the major reason that innovations failed. Cross et al. (1971) showed that, in this school, an innovation also failed but that the failure could not be attributed to any barriers. Implementation processes, rather than barriers, appeared to account for the failure.
In this manner, the book, though limited to a single case, represented a watershed in organizational innovation theory. Prior to the study, analysts had focused on the identification of barriers to innovation; since the study, the literature has been much more dominated by studies of the implementation process.
Conversely, a third rationale for a single case is the representative or typical case. Here, the objective is to capture the circumstances and conditions of an everyday or commonplace situation (see BOX 8; also see BOX 14, p. 75). The case study may represent a typical “project” among many different projects, a manufacturing firm believed to be typical of many other manufacturing firms in the same industry, a typical urban neighborhood, or a representative school, as examples. The lessons learned from these cases are assumed to be informative about the experiences of the average person or institution.
A fourth rationale for a single-case study is the revelatory case. This situation exists when an investigator has an opportunity to observe and analyze a phenomenon previously inaccessible to social science inquiry, such as Whyte’s (1943/1955) Street Comer Society, previously described in Chapter 1, BOX 2A. Another example is Elliot Liebow’s (1967) famous case study of unemployed men, Tally’s Corner (see BOX 9). Liebow had the opportunity to meet the men in an African American neighborhood in Washington, D.C. and to learn about their everyday lives. His observations of and insights into the problems of unemployment formed a significant case study, because few social scientists had previously had the opportunity to investigate these problems, even though the problems were common across the country. When other investigators have similar types of opportunities and can uncover some prevalent phenomenon previously inaccessible to social scientists, such conditions justify the use of a single-case study on the grounds of its revelatory nature.
BOX 8
The Average Case as a Single-Case Study
A famous community case study in sociology, Middletown, is about an average American city. The investigators, Robert and Helen Lynd (1929), deliberately chose to study a small town in middle America during the early 20th century (also see BOX 14, p. 75). Their purpose was to show how the transition from an agricultural to an industrial economy occurred in the average town—and thereby to provide a case ^jstudy about a significant development in all of American history.
BOX 9
The Revelatory Case as in Single-Case Study
Another rationale for selecting a single-case rather than a multiple-case design is that the investigator has access to a situation previously inaccessible to scientific observation. The case study is therefore worth conducting because the descriptive information alone will be revelatory.
Such was the situation in Elliot Liebow’s (1967) sociological classic, Tally’s Comer. The book is about a single group of African American men living in a poor, inner-city neighborhood. By befriending these men, the author was able to learn about their lifestyles, their coping behavior, and in particular their sensitivity to unemployment and failure. The book provided insights into a subculture that has prevailed in many U.S. cities for a long period of time, but one that had been only obscurely understood. The single case showed how investigations of such topics could be done, thus stimulating much further research and eventually the development of policy actions.
A fifth rationale for a single-case study is the longitudinal case: studying the same single case at two or more different points in time. The theory of interest would likely specify how certain conditions change over time, and the desired time intervals would presumably reflect the anticipated stages at which the changes should reveal themselves.
These five serve as major reasons for conducting a single-case study. There are other situations in which the single-case study may be used as a pilot case that is the first of a multiple-case study. However, in these latter instances, the single-case study cannot be regarded as a complete study on its own.
Whatever the rationale for doing single-case studies (and there may be more than the five mentioned here), a potential vulnerability of the single-case design is that a case may later turn out not to be the case it was thought to be at the outset. Single-case designs therefore require careful investigation of the potential case to minimize the chances of misrepresentation and to maximize the access needed to collect the case study evidence. A fair warning is not to commit yourself to any single-case study until all of these major concerns have been covered.
Holistic versus embedded case studies. The same single-case study may involve more than one unit of analysis. This occurs when, within a single case, attention is also given to a subunit or subunits (see BOX 10). For instance, even though a case study might be about a single organization, such as a hospital, the analysis might include outcomes about the clinical services and staff employed by the hospital (and possibly even some quantitative analyses based on the employee records of the staff). In an evaluation study, the single case might be a public program that involves large numbers of funded projects— which would then be the embedded units. In either situation, these embedded units can be selected through sampling or cluster techniques (McClintock, 1985). No matter how the units are selected, the resulting design would be called an embedded case study design (see Figure 2.4, Type 2). In contrast, if the case study examined only the global nature of an organization or of a program, a holistic design would have been used (see Figure 2.4, Type 1).
These two variants of single-case studies both have their strengths and weaknesses. The holistic design is advantageous when no logical subunits can be identified or when the relevant theory underlying the case study is itself of a holistic nature. Potential problems arise, however, when a global approach allows an investigator to avoid examining any specific phenomenon in operational detail. Thus, a typical problem with the holistic design is that the entire case study may be conducted at an unduly abstract level, lacking sufficiently clear measures or data.
BOX 10
An Embedded, Single-Case Design
Union Democracy (1956) is a highly regarded case study by three eminent academicians— Seymour Martin Lipset, Martin Trow, and James Coleman. The case study is about the inside politics of the International Typographical Union and involves several units of analysis (see “Kinds of Data” table). The main unit was the organization as a whole, the smallest unit was the individual member, and several intermediary units also were important. At each level of analysis, different data collection techniques were used, ranging from historical to survey analysis.
A further problem with the holistic design is that the entire nature of the case study may shift, unbeknownst to the researcher, during the course of study. The initial study questions may have reflected one orientation, but as the case study proceeds, a different orientation may emerge, and the evidence begins to address different research questions. Although some people have claimed such flexibility to be a strength of the case study approach, in fact the largest criticism of case studies is based on this type of shift—in which the implemented research design is no longer appropriate for the research questions being asked (see COSMOS Corporation, 1983). Because of this problem, you need to avoid such unsuspected slippage; if the relevant research questions really do change, you should simply start over again, with a new research design. One way to increase the sensitivity to such slippage is to have a set of subunits. Thus, an embedded design can serve as an important device for focusing a case study inquiry.
An embedded design, however, also has its pitfalls. A major one occurs when the case study focuses only on the subunit level and fails to return to the larger unit of analysis. For instance, an evaluation of a program consisting of multiple projects may include project characteristics as a subunit of analysis. The project-level data may even be highly quantitative if there are many projects. However, the original evaluation becomes a project study (i.e., a multiple-case study of different projects) if no investigating is done at the level of the original case—that is, the program. Similarly, a study of organizational climate may involve individual employees as a subunit of study. However, if the data focus only on individual employees, the study will in fact become an employee and not an organizational study. In both examples, what has happened is that the original phenomenon of interest (a program or organizational climate) has become the context and not the target of study.
Summary. Single cases are a common design for doing case studies, and two variants have been described: those using holistic designs and those using embedded units of analysis. Overall, the single-case design is eminently justifiable under certain conditions—where the case represents (a) a critical test of existing theory, (b) a rare or unique circumstance, or (c) a representative or typical case, or where the case serves a (d) revelatory or (e) longitudinal purpose.
A major step in designing and conducting a single case is defining the unit of analysis (or the case itself). An operational definition is needed, and some caution must be exercised—before a total commitment to the whole case study is made— to ensure that the case in fact is relevant to the issues and questions of interest.
Within the single case may still be incorporated subunits of analyses, so that a more complex—or embedded—design is developed. The subunits can often add significant opportunities for extensive analysis, enhancing the insights into the
single case. However, if too much attention is given to these subunits, and if the larger, holistic aspects of the case begin to be ignored, the case study itself will have shifted its orientation and changed its nature. If the shift is justifiable, you need to address it explicitly and indicate its relationship to the original inquiry.
2. What Are the Potential Multiple-Case Designs (Types 3 and 4)?
The same study may contain more than a single case. When this occurs, the study has used a multiple-case design, and such designs have increased in frequency in recent years. A common example is a study of school innovations (such as the use of new curricula, rearranged school schedules, or a new educational technology), in which individual schools adopt some innovation. Each school might be the subject of an individual case study, but the study as a whole covers several schools and in this way uses a multiple-case design.
Multiple- versus single-case designs. In some fields, multiple-case studies have been considered a different “methodology” from single-case studies. For example, both anthropology and political science have developed one set of rationales for doing single-case studies and a second set for doing what have been considered “comparative” (or multiple-case) studies (see Eckstein, 1975; Lijphart, 1975). This book, however, considers single- and multiple-case designs to be variants within the same methodological framework—and no broad distinction is made between the so-called classic (that is, single) case study and multiple-case studies. The choice is considered one of research design, with both being included under the case study method.
Multiple-case designs have distinct advantages and disadvantages in comparison to single-case designs. The evidence from multiple cases is often considered more compelling, and the overall study is therefore regarded as being more robust (Herriott & Firestone, 1983). At the same time, the rationale for single-case designs cannot usually be satisfied by multiple cases. By definition, the unusual or rare case, the critical case, and the revelatory case all are likely to involve only single cases. Moreover, the conduct of a multiple-case study can require extensive resources and time beyond the means of a single student or independent research investigator. Therefore, the decision to undertake multiple- case studies cannot be taken lightly.
Selecting the multiple cases also raises a new set of questions. Here, a major insight is to consider multiple cases as one would consider multiple experiments—that is, to follow a “replication” design. This is far different from a mistaken analogy in the past, which incorrectly considered multiple cases to be similar to the multiple respondents in a survey (or to the multiple subjects within an experiment)—that is, to follow a “sampling” design. The methodological differences between these two views are revealed by the different rationales underlying the replication as opposed to sampling designs.
Replication, not sampling logic, for multiple-case studies. The replication logic is analogous to that used in multiple experiments (see Hersen & Barlow, 1976). For example, upon uncovering a significant finding from a single experiment, an ensuing and pressing priority would be to replicate this finding by conducting a second, third, and even more experiments. Some of the replications might attempt to duplicate the exact conditions of the original experiment. Other replications might alter one or two experimental conditions considered unimportant to the original finding, to see whether the finding could still be duplicated. Only with such replications would the original finding be considered robust.
The logic underlying the use of multiple-case studies is the same. Each case must be carefully selected so that it either (a) predicts similar results (a literal replication) or (b) predicts contrasting results but for anticipatable reasons (a theoretical replication). The ability to conduct 6 or 10 case studies, arranged effectively within a multiple-case design, is analogous to the ability to conduct 6 to 10 experiments on related topics; a few cases (2 or 3) would be literal replications, whereas a few other cases (4 to 6) might be designed to pursue two different patterns of theoretical replications. If all the cases turn out as predicted, these 6 to 10 cases, in the aggregate, would have provided compelling support for the initial set of propositions. If the cases are in some way contradictory, the initial propositions must be revised and retested with another set of cases. Again, this logic is similar to the way scientists deal with conflicting experimental findings.
An important step in all of these replication procedures is the development of a rich, theoretical framework. The framework needs to state the conditions under which a particular phenomenon is likely to be found (a literal replication) as well as the conditions when it is not likely to be found (a theoretical replication). The theoretical framework later becomes the vehicle for generalizing to new cases, again similar to the role played in cross-experiment designs. Furthermore, just as with experimental science, if some of the empirical cases do not work as predicted, modification must be made to the theory. Remember, too, that theories can be practical and not just academic.
For example, one might consider the initial proposition that an increase in using a new technology in school districts will occur when the technology is used for both administrative and instmctional applications, but not either alone. To pursue this proposition in a multiple-case study design, 3 or 4 cases might be selected in which both types of applications are present, to determine whether, in fact, technology use did increase over a period of time (the investigation would be predicting a literal replication in these 3 or 4 cases). Three or 4 additional cases might be selected in which only administrative applications are present, with the prediction being little increase in use (predicting a theoretical replication). Finally, 3 or 4 other cases would be selected in which only instructional applications are present, with the same prediction of little increase in use, but for different reasons than the administrative-only cases (another theoretical replication). If this entire pattern of results across these multiple cases is indeed found, the 9 to 12 cases, in the aggregate, would provide substantial support for the initial proposition.
Another example of a multiple-case replication design comes from the field of urban studies (see BOX 11). You also can find examples of three entire case studies, all following a replication design but covering HIV/AIDS prevention, university administration, and the transformation of business firms, in the companion text (Yin, 2003, chaps. 8-10).
This replication logic, whether applied to experiments or to case studies, must be distinguished from the sampling logic commonly used in surveys. The sampling logic requires an operational enumeration of the entire universe or pool of potential respondents and then a statistical procedure for selecting a specific subset of respondents to be surveyed. The resulting data from the sample that is actually surveyed are assumed to reflect the entire universe or pool, with inferential statistics used to establish the confidence intervals for which this representation is presumed accurate. The entire procedure is commonly used when an investigator wishes to determine the prevalence or frequency of a particular phenomenon.
BOX 11
A Multiple-Case, Replication Design
A common problem in the 1960s and 1970s was how to get good advice to city governments. Peter Szanton’s (1981) book, Not Well Advised, reviewed the experiences of numerous attempts by university and research groups to collaborate with city officials.
The book is an excellent example of a multiple-case, replication design. Szanton starts with eight case studies, showing how different university groups all failed to help cities. The eight cases are sufficient “replications” to convince the reader of a general phenomenon. Szanton then provides five more case studies, in which nonuniversity groups also failed, concluding that failure was therefore not necessarily inherent in the academic enterprise. Yet a third group of cases shows how university groups have successfully helped business, engineering firms, and sectors other than city government. A final set of three cases shows that those few groups able to help city government were concerned with implementation and not just with the production of new ideas, leading to the major conclusion that city governments may have peculiar needs in receiving but also then putting advice into practice.
Within each of the four groups of case studies, Szanton has illustrated the principle of literal replication. Across the four groups, he has illustrated theoretical replication. This potent case study design can and should be applied to many other topics.
Any application of this sampling logic to case studies would be misplaced. First, case studies are not the best method for assessing the prevalence of phenomena. Second, a case study would have to cover both the phenomenon of interest and its context, yielding a large number of potentially relevant variables. In turn, this would require an impossibly large number of cases—too large to allow any statistical consideration of the relevant variables.
Third, if a sampling logic had to be applied to all types of research, many important topics could not be empirically investigated, such as the following problem: Your investigation deals with the role of the presidency of the United States, and you are interested in doing a multiple-case study of a (few) presidents to test your theory about presidential leadership. However, the complexity of your topic means that your choice of a small number of cases could not adequately represent all the 44 presidents since the beginning of the Republic. Critics using a sampling logic might therefore deny the acceptability of your study. In contrast, if you use a replication logic, the study is eminently feasible.
The replication approach to multiple-case studies is illustrated in Figure 2.5. The figure indicates that the initial step in designing the study must consist of theory development, and then shows that case selection and the definition of specific measures are important steps in the design and data collection process. Each individual case study consists of a “whole” study, in which convergent evidence is sought regarding the facts and conclusions for the case; each case’s conclusions are then considered to be the information needing replication by other individual cases. Both the individual cases and the multiple-case results can and should be the focus of a summary report. For each individual case, the report should indicate how and why a particular proposition was demonstrated (or not demonstrated). Across cases, the report should indicate the extent of the replication logic and why certain cases were predicted to have certain results, whereas other cases, if any, were predicted to have contrasting results.
An important part of Figure 2.5 is the dashed-line feedback loop. The loop represents the situation where important discovery occurs during the conduct of one of the individual case studies (e.g., one of the cases did not in fact suit the original design). Such a discovery even may require you to reconsider one or more of the study’s original theoretical propositions. At this point, “redesign” should take place before proceeding further. Such redesign might involve the selection of alternative cases or changes in the case study (i.e., data collection) protocol (see Chapter 3). Without such redesign, you risk being accused of distorting or ignoring the discovery, just to accommodate the original design. This condition leads quickly to a further accusation—that you have been selective in reporting your data, to suit your preconceived ideas (i.e., the original theoretical propositions).
Overall, Figure 2.5 depicts a very different logic from that of a sampling design. The logic as well as its contrast with a sampling design may be difficult to follow and is worth extensive discussion with colleagues before proceeding with any multiple case study.
When using a multiple-case design, a further question you will encounter has to do with the number of cases deemed necessary or sufficient for your study. However, because a sampling logic should not be used, the typical criteria regarding sample size also are irrelevant. Instead, you should think of this decision as a reflection of the number of case replications—both literal and theoretical—that you need or would like to have in your study.
For the number of literal replications, an appropriate analogy from statistics is the selection of the criterion for establishing the sample size desired to detect an “effect.” Designating a “p < .05” or “p < .01” likelihood of detection as part of a power analysis is not based on any formula but is a matter of discretionary, judgmental choice. Analogously, designating the number of replications depends upon the certainty you want to have about your multiple-case results (as with the higher criterion for establishing the likelihood of detection, the greater certainty lies with the larger number of cases). For example, you may want to settle for two or three literal replications when your theory is straightforward and the issue at hand does not demand an excessive degree of certainty. However, if your theory is subtle or if you want a high degree of certainty, you may press for five, six, or more replications.
For the number of theoretical replications, the important consideration is related to your sense of the importance of rival explanations. The stronger the rivals, the more additional cases you might want, each case showing a different result when some rival explanation had been taken into account. For example, your original hypothesis might be that summer reading programs improve students’ reading scores, and you already might have shown this result through several cases that served as literal replications. A rival explanation might be that parents also work more closely with their children during the summer and that this circumstance can account for improved reading scores. You would then find another case, with parent participation but no summer reading program, and in this theoretical replication you would predict that the scores would not improve.
Rationale for multiple-case designs. In short, the rationale for multiple-case designs derives directly from your understanding of literal and theoretical replications. The simplest multiple-case design would be the selection of two or more cases that are believed to be literal replications, such as a set of cases with exemplary outcomes in relation to some evaluation questions, such as “how and why a particular intervention has been implemented smoothly.” Selecting such cases requires prior knowledge of the outcomes, with the multiple-case inquiry focusing on how and why the exemplary outcomes might have occurred and hoping for literal (or direct) replications of these conditions from case to case.5
More complicated multiple-case designs would likely result from the number and types of theoretical replications you might want to cover. For example, investigators have used a “two-tail” design in which cases from both extremes (of some important theoretical condition, such as good and bad outcomes) have been deliberately chosen. Multiple-case rationales also can derive from the prior hypothesizing of different types of conditions and the desire to have subgroups of cases covering each type. These and other similar designs are more complicated because the study should still have at least two individual cases within each of the subgroups, so that the theoretical replications across subgroups are complemented by literal replications within each subgroup.
Multiple-case studies: Holistic or embedded. The fact that a design calls for multiple-case studies does not eliminate the variation identified earlier with single cases: Each individual case may still be holistic or embedded. In other words, a multiple-case study may consist of multiple holistic cases (see Figure 2.4, Type 3) or of multiple embedded cases (see Figure 2.4, Type 4).
The difference between these two variants depends upon the type of phenomenon being studied and your research questions. In an embedded design, a study even may call for the conduct of a survey at each case study site. For instance, suppose a study is concerned with the impact of the same type of curriculum adopted by different schools. Each school may be the topic of a case study, with the theoretical framework dictating that nine such schools be included as case studies, three to replicate a direct result (literal replication) and six others to deal with contrasting conditions (theoretical replications).
For all nine schools, an embedded design is used because surveys of the students (or, alternatively, examination of students’ archival records) are needed to address research questions about the performance of the schools. However, the results of each survey will not be pooled across schools. Rather, the survey data will be part of the findings for each individual school, or case. These data may be highly quantitative, focusing on the attitudes and behavior of individual students, and the data will be used along with archival information to interpret the success and operations at the given school. If, in contrast, the survey data are pooled across schools, a replication design is no longer being used. In fact, the study has now become a single-case study, in which all nine schools and their students have now become part of some larger, main unit of analysis. Such a new case study would then require a complete redefinition of the main unit of analysis, with extensive revisions to the original theories and propositions of interest also a likely need.
Summary. This section has dealt with situations in which the same investigation may call for multiple-case studies. These types of designs are becoming more prevalent, but they are more expensive and time-consuming to conduct.
Any use of multiple-case designs should follow a replication, not a sampling logic, and an investigator must choose each case carefully. The cases should serve in a manner similar to multiple experiments, with similar results (a literal replication) or contrasting results (a theoretical replication) predicted explicitly at the outset of the investigation.
The individual cases within a multiple-case study design may be either holistic or embedded. When an embedded design is used, each individual case study may in fact include the collection and analysis of quantitative data, including the use of surveys within each case.
EXERCISE 2.4 Defining a Case Study Research Design
Select one of the case studies described in the BOXES of this book, reviewing the entire case study (not just the material in the BOX). Describe the research design of this case study. How did it justify the relevant evidence to be sought, given the basic research questions to be answered? What methods were used to draw conclusions, based on the evidence? Is the design a single- or multiple-case design? Is it holistic or does it have embedded units of analysis?
Source: Yin K Robert (2008), Case Study Research Designs and Methods, SAGE Publications, Inc; 4th edition.
The post Case Study Designs appeared first on HKT Consultant.
]]>The post Modest Advice in Selecting Case Study Designs appeared first on HKT Consultant.
]]>1. Single- or Multiple-Case Designs?
The first word of advice is that, although all designs can lead to successful case studies, when you have the choice (and resources), multiple-case designs may be preferred over single-case designs. Even if you can do a “two-case”
case study, your chances of doing a good case study will be better than using a single-case design. Single-case designs are vulnerable if only because you will have put “all your eggs in one basket.” More important, the analytic benefits from having two (or more) cases may be substantial.
To begin with, even with two cases, you have the possibility of direct replication. Analytic conclusions independently arising from two cases, as with two experiments, will be more powerful than those coming from a single case (or single experiment) alone. Alternatively you may have deliberately selected your two cases because they offered contrasting situations, and you were not seeking a direct replication. In this design, if the subsequent findings support the hypothesized contrast, the results represent a strong start toward theoretical replication—again vastly strengthening your findings compared to those from a single case alone (e.g., Eilbert & Lafronza, 2005; Hanna, 2005; also see BOX 12).
BOX 12
Two, “Two-Case” Case Studies
12A. Contrasting Cases for Community Building
Chaskin (2001) used two case studies to illustrate contrasting strategies for capacity building at the neighborhood level. The author’s overall conceptual framework, which was the main topic of inquiry, claimed that there could be two approaches to building community capacity—using a collaborative organization to (a) reinforce existing networks of community organizations or (b) initiate a new organization in the neighborhood. After thoroughly airing the framework on theoretical grounds, the author presents the two case studies, showing the viability of each approach.
12B. Contrasting Strategies for Educational Accountability
In a directly complementary manner, Elmore, Abelmann, and Fuhrman (1997) chose two case studies to illustrate contrasting strategies for designing and implementing educational accountability (i.e., holding schools accountable for the academic performance of their students). One case represented a lower cost, basic version of an accountability system. The other represented a higher cost, more complex version.
In general, criticisms about single-case studies usually reflect fears about the uniqueness or artifactual conditions surrounding the case (e.g., special access to a key informant). As a result, the criticisms may turn into skepticism about your ability to do empirical work beyond having done a single-case
study. Having two cases can begin to blunt such criticism and skepticism. Having more than two cases will produce an even stronger effect. In the face of these benefits, having at least two cases should be your goal. If you do use a single-case design, you should be prepared to make an extremely strong argument in justifying your choice for the case.
EXERCISE 2.5 Establishing the Rationale for a Multiple-Case Study
Develop some preliminary ideas about a “case” for your case study. Alternatively, focus on one of the single-case studies presented in the BOXES in this book. In either situation, now think of a companion “case” that might augment the single case. In what ways might the companion case’s findings supplement those of the first case? Could the data from the second case fill a gap left by the first case or respond better to some obvious shortcoming or criticism of the first case? Would the two cases together comprise a stronger case study? Could yet a third case make the findings even more compelling?
2. Closed Designs or Flexible Designs?
Another word of advice is that, despite this chapter’s details about design choices, you should not think that a case study’s design cannot be modified by new information or discovery during data collection. Such revelations can be enormously important, leading to your altering or modifying your original design.
As examples, in a single-case study, what was thought to be a critical or unique case might have turned out not to be so, after initial data collection had started; ditto a multiple-case study, where what was thought to be parallel cases for literal replication turn out not to be so. With these revelations, you have every right to conclude that your initial design needs to be modified. However, you should undertake any alterations only given a serious caution. The caution is to understand precisely the nature of the alteration: Are you merely selecting different cases, or are you also changing your original theoretical concerns and objectives? The point is that the needed flexibility should not lessen the rigor with which case study procedures are followed.
3. Mixed Methods Designs: Mixing Case Studies with Other Methods?
Researchers have given increasing attention to “mixed methods research”— a “class of research where the researcher mixes or combines quantitative and qualitative research techniques, methods, approaches, concepts or language into a single study” (Johnson & Onwuegbuzie, 2004, p. 17, emphasis added). Confinement to a single study forces the methods being mixed into an integrated mode. The mode differs from the conventional situation whereby different methods are used in separate studies that may later be synthesized.
Mixed methods research forces the methods to share the same research questions, to collect complementary data, and to conduct counterpart analyses (e.g., Yin, 2006b)—in short, to follow a mixed methods design. As such, mixed methods research can permit investigators to address more complicated research questions and collect a richer and stronger array of evidence than can be accomplished by any single method alone. Depending upon the nature of your research questions and your ability to use different methods, mixed methods research opens a class of research designs that deserve your consideration.
The earlier discussion of embedded case study designs in fact points to the fact that certain kinds of case studies already represent a form of mixed methods research. The embedded case studies rely on more holistic data collection strategies for studying the main case but then call upon surveys or other more quantitative techniques to collect data about the embedded unit(s) of analysis. In this situation, other research methods are embedded within your case study.
The opposite relationship also can occur. Your case study may be part of a larger, mixed methods study. The main investigation may rely on a survey or other quantitative techniques, and your case study may help to investigate the conditions within one of the entities being surveyed. The contrasting relationships (survey within case or case within survey) are illustrated in Figure 2.6.
At the same time, mixed methods research need not include the use of the case study strategy at all. For instance, much historical work embraces the quantitative analysis of archival records, such as newspapers and other file material. And, in an even broader sense, mixed methods research need not be limited to combinations of quantitative and qualitative methods. For instance, a study could employ a survey to describe certain conditions, complemented by an experiment that tried to manipulate some of those conditions (e.g., Berends & Garet, 2002).
By definition, studies using mixed methods research are more difficult to execute than studies limited to single methods. However, mixed methods research can enable you to address broader or more complicated research questions than case studies alone. As a result, mixing case studies with other methods should be among the possibilities meriting your consideration.
Source: Yin K Robert (2008), Case Study Research Designs and Methods, SAGE Publications, Inc; 4th edition.
The post Modest Advice in Selecting Case Study Designs appeared first on HKT Consultant.
]]>The post The Case Study Investigator: Desired Skills appeared first on HKT Consultant.
]]>In actuality, the demands of a case study on your intellect, ego, and emotions are far greater than those of any other research method. This is because the data collection procedures are not routinized. In laboratory experiments or in surveys, for instance, the data collection phase of a research project can be largely, if not wholly, conducted by one (or more) research assistant(s). The assistant is to carry out the data collection activities with a minimum of discretionary behavior, and in this sense, the activity is routinized—and analytically boring.
Conducting case studies offers no such parallel. Rather, a well-trained and experienced investigator is needed to conduct a high-quality case study because of the continuous interaction between the theoretical issues being studied and the data being collected. During data collection, only a more experienced investigator will be able to take advantage of unexpected opportunities rather than being trapped by them—and also will exercise sufficient care against potentially biased procedures.
Unfortunately, there are no tests for distinguishing those persons likely to become good case study investigators from those who are not. Compare this situation to that in mathematics or even a profession such as law. In math, people are able to score themselves for their abilities and to screen themselves from further advancement because they simply cannot carry out higher levels of math problems. To practice law, a person must pass the bar examination in a particular state. Again, many people screen themselves out of the field by failing to pass this test.
No such gatekeepers exist for assessing case study skills. However, a basic list of commonly required skills is as follows:
- A good case study investigator should be able to ask good questions—and interpret the answers.
- An investigator should be a good “listener” and not be trapped by her or his own ideologies or preconceptions.
- An investigator should be adaptive and flexible, so that newly encountered situations can be seen as opportunities, not threats.
- An investigator must have a firm grasp of the issues being studied, even if in an exploratory mode. Such a grasp reduces the relevant events and information to be sought to manageable proportions.
- A person should be unbiased by preconceived notions, including those derived from theory. Thus, a person should be sensitive and responsive to contradictory evidence.
Each of these attributes is described below. Any absence of these attributes is remediable, as anyone missing one or more of the skills can work on developing them. But everyone must be honest in assessing her or his capabilities in the first place.
1. Asking Good Questions
More than with the other research methods discussed in Chapter 1, case studies require an inquiring mind during data collection, not just before or after the activity. The ability to pose and ask good questions is therefore a prerequisite for case study investigators. The desired result is for the investigator to create a rich dialogue with the evidence, an activity that encompasses pondering the possibilities gained from deep familiarity with some aspect of the world, systematizing those ideas in relation to kinds of information one might gather, checking the ideas in the light of that information, dealing with the inevitable discrepancies between what was expected and what was found by rethinking the possibilities of getting more data, and so on. (Becker, 1998, p. 66)
Case study data collection does follow a formal protocol, but the specific information that may become relevant to a case study is not readily predictable. As you collect case study evidence, you must quickly review the evidence and continually ask yourself why events or facts appear as they do. Your judgments may lead to the immediate need to search for additional evidence. If you are able to ask good questions throughout the data collection process, a good prediction is that you also will be mentally and emotionally exhausted at the end of each day. This depletion of analytic energy is far different from the experience in collecting experimental or survey data—that is, testing “subjects” or administering questionnaires. In these situations, data collection is highly routinized, and the data collector must complete a certain volume of work but exercise minimal discretionary behavior. Furthermore, any substantive review of the evidence does not come until some later time. The result is that such a data collector may become physically exhausted but will have been mentally untested after a day of data collection.
One insight into asking good questions is to understand that research is about questions and not necessarily about answers. If you are the type of person for whom one tentative answer immediately leads to a whole host of new questions, and if these questions eventually aggregate to some significant inquiry about how or why the world works as it does, you are likely to be a good asker of questions.
2. Being a Good “Listener”
For case studies, “listening” means receiving information through multiple modalities—for example, making keen observations or sensing what might be going on—not just using the aural modality. Being a good listener means being able to assimilate large amounts of new information without bias. As an interviewee recounts an incident, a good listener hears the exact words used by the interviewee (sometimes, the terminology reflects an important orientation), captures the mood and affective components, and understands the context from which the interviewee is perceiving the world.
The listening skill also needs to be applied to the inspection of documentary evidence, as well as to observations of real-life situations. In reviewing documents, listening takes the form of worrying whether there is any important message between the lines; any inferences, of course, would need to be corroborated with other sources of information, but important insights may be gained in this way. Poor “listeners” may not even realize that there can be information between the lines. Other listening deficiencies include having a closed mind or simply having a poor memory.
3. Exercising Adaptiveness and Flexibility
Few case studies will end up exactly as planned. Inevitably, you will have to make minor if not major changes, ranging from the need to pursue an unexpected lead (potentially minor) to the need to identify a new “case” for study (potentially major). The skilled investigator must remember the original purpose of the investigation but then must be willing to adapt procedures or plans if unanticipated events occur (see BOX 13).
BOX 13
Maintaining Flexibility in Designing a Case Study
Peter Blau’s study of behavior in large government agencies (The Dynamics of Bureaucracy, 1955) is still valued for its insights into the relationship between the formal and informal organization of work groups, even over 50 years later.
Although his study focused on two government agencies, that was not Blau’s initial design. As the author notes, he first intended to study a single organization and later switched to a plan to compare two organizations—a public one and a private one (Blau, 1955, pp. 272-273). However, his initial attempts to gain access to a private firm were unsuccessful, and in the meanwhile, he had developed a stronger rationale for comparing two government agencies but of different kinds.
These shifts in the initial plans are examples of the kinds of changes that can occur in the design of a case study. Blau’s experience shows how a skilled investigator can take advantage of changing opportunities, as well as shifts in theoretical concerns, to produce a classic case study.
When a shift is made, you must maintain an unbiased perspective and acknowledge those situations in which, in fact, you may have inadvertently begun to pursue a totally new investigation. When this occurs, many completed steps—including the initial design of the case study—must be repeated and redocumented. One of the worst complaints about the conduct of case study research is that investigators change directions without knowing that their original research design was inadequate for the revised investigation, thereby leaving unknown gaps and biases. Thus, the need to balance adaptiveness with rigor—but not rigidity—cannot be overemphasized.
4. Having a Firm Grasp of the Issues Being Studied
The main way of staying on target, of course, is to understand the purpose of the case study investigation in the first place. Each case study investigator must understand the theoretical or policy issues because analytic judgments have to be made throughout the data collection phase. Without a firm grasp of the issues, you could miss important clues and would not know when a deviation was acceptable or even desirable. The point is that case study data collection is not merely a matter of recording data in a mechanical fashion, as it is in some other types of research. You must be able to interpret the information as it is being collected and to know immediately, for instance, if several sources of information contradict one another and lead to the need for additional evidence—much like a good detective.
In fact, the detective role offers some keen insights into case study fieldwork. Note that the detective arrives on a scene after a crime has occurred and is basically being called upon to make inferences about what actually transpired. The inferences, in turn, must be based on convergent evidence from witnesses and physical evidence, as well as some unspecifiable element of common sense. Finally, the detective may have to make inferences about multiple crimes, to determine whether the same perpetrator committed them. This last step is similar to the replication logic underlying multiple- case studies.
5. Avoiding Bias
All of the preceding conditions will be negated if an investigator seeks only to use a case study to substantiate a preconceived position. Case study investigators are especially prone to this problem because they must understand the issues beforehand (see Becker, 1958, 1967). You also may have selected the case study method to enable you (wrongly) to pursue or (worse yet) advocate particular issues.1 In contrast, the traditional research assistant, though mechanistic and possibly even sloppy, is not likely to introduce a substantive bias into the research.
One test of this possible bias is the degree to which you are open to contrary findings. For example, researchers studying “nonprofit” organizations may be surprised to find that many of these organizations have entrepreneurial and capitalistic motives (even though the organizations don’t formally make profits). If such findings are based on compelling evidence, the conclusions of the case study would have to reflect these contrary findings. To test your own tolerance for contrary findings, report your preliminary findings—possibly while still in the data collection phase—to two or three critical colleagues. The colleagues should offer alternative explanations and suggestions for data collection. If the quest for contrary findings can produce documentable rebuttals, the likelihood of bias will have been reduced.
Source: Yin K Robert (2008), Case Study Research Designs and Methods, SAGE Publications, Inc; 4th edition.
The post The Case Study Investigator: Desired Skills appeared first on HKT Consultant.
]]>The post Preparation and Training for a Specific Case Study appeared first on HKT Consultant.
]]>Some time between the completion of your design and the start of your data collection, you will need to show how you plan to protect the human subjects in your case study. You will need to obtain formal approval for your plan. Such approval should not merely be viewed as an oversight process, because you should always conduct all of your research with the highest ethical standard.
The specific need for protecting human subjects comes from the fact that nearly all case studies, like those covered by this book, are about contemporary human affairs. In this single manner, you and other social scientists differ from scientists who study physical, chemical, or other nonhuman systems or from historians who may be studying the “dead past.” The study of “a contemporary phenomenon in its real-life context” obligates you to important ethical practices akin to those followed in medical research.
As part of the protection, you are responsible for conducting your case study with special care and sensitivity—going beyond the research design and other technical considerations covered throughout this book. The care usually involves
- gaining informed consent from all persons who may be part of your case study, by alerting them to the nature of your case study and formally soliciting their volunteerism in participating in the study;
- protecting those who participate in your study from any harm, including avoiding the use of any deception in your study;
- protecting the privacy and confidentiality of those who participate so that, as a result of their participation, they will not be unwittingly put in any undesirable position, even such as being on a roster to receive requests to participate in some future study, whether conducted by you or anyone else; and
- taking special precautions that might be needed to protect especially vulnerable groups (for instance, research involving children).
Exactly how you exercise the needed care and sensitivity will vary, depending on your case study. General guidance comes from your own professional ethics. Professional research associations also promulgate their own standards for doing human subjects research, not just case studies (e.g., Joint Committee on Standards for Educational Evaluation, 1981). Most important, however, your institutional setting will have its own expectations, whether you are part of a university or of an independent research organization, and you need to follow its specific guidance.
In particular, every institution now has an Institutional Review Board (IRB). The board is charged with reviewing and approving all human subjects research before such research can proceed. The board’s review will cover the objectives of your study and how you plan to protect the human subjects that may be part of the study. Note that your interactions with the specific human subjects in your study take place through both direct contact (as in interviews) and the potential use of personal records (as in client records). Case studies present a more challenging situation than when using other research methods because these interactions are not necessarily as structured as with other methods (such as in administering a closed-ended questionnaire). The board will want to know such information as how you plan to interact with those being studied, the protocols or data collection instruments you are planning to use, and how you will ensure such protections as informed consent and confidentiality.
As a result, the most imperative step before proceeding with your case study is to seek out the IRB at your institution, follow its guidance, and obtain its approval. The IRB’s concerns will vary from institution to institution and IRB to IRB. Do not hesitate to speak with a member or two of the IRB informally and ahead of time, to gain insight into the review process and its expectations.
2. Case Study Training as a Seminar Experience
Training also is a necessary step in doing case study research. The timing of the training, relative to the timing for seeking human subjects approval, will not always be linear. You need to have some data collection plans before seeking approval, but, as pointed out below, the finalization of the plans cannot occur until after the approval has been granted. The training activities described below may therefore take place over an extended period of time, as in a regular seminar.
For case study research, the key to understanding the needed training is to understand that every case study investigator must be able to operate as a “senior” investigator. Once you have started collecting data, you should think of yourself as an independent investigator who cannot rely on a rigid formula to guide your inquiry. You must be able to make intelligent decisions throughout the data collection process.
In this sense, training for a case study investigation actually begins with the definition of the questions being addressed and the development of the case study design. If these steps have been satisfactorily conducted, as described in Chapters 1 and 2, only minimal further effort may be needed, especially if there is only a single case study investigator.
However, it often happens that a case study investigation must rely on a case study team,2 for any of three reasons:
- a single case calls for intensive data collection at the same site, requiring a “team” of investigators (see BOX 14);
- a case study involves multiple cases, with different persons being needed to cover each site or to rotate among the sites (Stake, 2006, p. 21); or
- a combination of the first two conditions.
Under these circumstances, all team members should have contributed to the development of a draft case study protocol. This draft would then have been the version submitted for IRB approval, with the IRB-approved version subsequently being considered the final version of the protocol.
BOX 14
The Logistics of Field Research, Circa 1924-1925
Arranging schedules and gaining access to relevant sources of evidence are important to the management of a case study. The modern researcher may feel that these activities have only emerged with the growth of “big” social science during the 1960s and 1970s.
In a famous field study done decades ago, however, many of the same management techniques already had been practiced. The two principal investigators and their staff secretary opened a local office in the city they were studying. This office was used by other project staff for extended periods of time. From this vantage point, the research team participated in local life, examined documentary materials, compiled local statistics, conducted interviews, and distributed and collected questionnaires. This extensive fieldwork resulted 5 years later in the publication of the now-classic study of small-town America, Middletown (1929), by Robert and Helen Lynd (also see BOX 8, Chapter 2, p. 48).
When multiple investigators or team members participate in the same case study, all need to learn to be “senior” investigators. Training takes the form of a seminar rather than didactic instruction. As in a seminar, much time has to be allowed for reading, preparing for the training, and holding the training. (See Figure 3.1 for an agenda of an illustrative training session.)
Typically, the seminar will cover all phases of the planned case study investigation, including readings on the subject matter, the theoretical issues that led to the case study design, and case study methods and tactics. You might review examples of tools used in other case studies (see BOX 15), to add to the methodological portion of the training.
The goal of the training is to have all participants understand the basic concepts, terminology, and methodological issues relevant to the study. Each team member needs to know
- why the study is being done,
- what evidence is being sought,
- what variations can be anticipated (and what should be done if such variations occur), and
- what would constitute supportive or contrary evidence for any given proposition.
BOX 15
Reviewing the Tools and Methods Used in Other Case Studies, Circa the 21st Century
Web sites have provided new opportunities to access the tools and methods used in other case studies. For example, in online versions of articles, academic journals may reproduce supplementary materials that might not have appeared in the printed version of the article. For one case study, the supplementary materials included the formal case study protocol, case study coding book, evidentiary tables linking claims to sections of the case study database, and a list of documents in the case study database (Randolph & Eronen, 2007).
Discussions, rather than lectures, are the key part of the training effort, to ensure that the desired level of understanding has been achieved.
This seminar approach to case study training can be contrasted to the training for other types of data collection—for example, group training for survey interviewers. The survey training does involve discussions, but it mainly emphasizes the questionnaire items or terminology to be used and takes place over an intensive but short period of time. Moreover, the survey training may not cover the global or conceptual concerns of the study, as interviewers may not need to have any broader understanding beyond the mechanics of the survey instrument. Survey training rarely involves any outside reading about the substantive issues, and the survey interviewer generally does not know how the survey data are to be analyzed or what issues are to be investigated. Such an approach may feed the strengths of doing surveys but would be insufficient for case study training.
3. Protocol Development and Review
The next subsection will say more about the content of the case study protocol. However, a legitimate and desirable training task is the understanding of the protocol by all of the case study investigators.
To reinforce such an understanding, each investigator or team member may be assigned one portion of the substantive topics covered by the protocol. Each investigator is then responsible for reviewing the appropriate reading materials related to the assigned portion, adding any other information that may be relevant, and leading a discussion that clarifies that portion of the protocol’s questions. In this manner, such an arrangement should ensure that each team member has mastered the content of the protocol.
4. Problems to Be Addressed
The training also has the purpose of uncovering problems within the case study plan or with the research team’s capabilities. If such problems do emerge, one consolation is that they will be more troublesome if they are not recognized until later, after the data collection begins. Good case study investigators should therefore press to be certain, during the training period, that potential problems are brought into the open.
The most obvious problem is that the training may reveal flaws in the case study design or even the initial definition of the study questions. If this occurs, you must be willing to make the necessary revisions, even if more time and effort are necessary. Sometimes, the revisions will challenge the basic purpose of the investigation, as in a situation in which the original objective may have been to investigate a technological phenomenon, such as the use of personal computers, but in which the case study really turns out to be about an organizational phenomenon, such as poor supervision. Any revisions, of course, also may lead to the need to review a slightly different literature and to recast the entire study and its audience. You also should check your IRB’s procedures to see whether it will need to conduct a new human subjects review. Despite these unexpected developments, changing the basic premise of your case study is fully warranted if the training has demonstrated the unrealistic (or uninteresting) nature of the original plan.
A second problem is that the training may reveal incompatibilities among the investigating team—and in particular, the fact that some of the team members may not share the orientation of the project or its sponsors. In one multiple-case study of community organizations, for instance, team members varied in then- beliefs regarding the efficacy of such organizations (U.S. National Commission on Neighborhoods, 1979). When such biases are discovered, one way of dealing with the contrary orientations is to suggest to the team that contrary evidence will be respected if it is collected and verifiable. A team member still has the choice, of course, of continuing to participate in the study or deciding to drop out.
A third problem is that the training may reveal some impractical time deadlines or expectations regarding available resources. For instance, a case study may have assumed that 20 persons were to be contacted for open-ended interviews during a site visit, as part of the data collection. The training may have revealed, however, that the time needed for meeting with these persons is likely to be much longer than anticipated. Under such circumstances, any expectation for interviewing 20 persons would have to depend on revising the original data collection schedule.
Finally, the training may uncover some positive features, such as the fact that two or more team members have complementary skills and are able to work productively together. Such rapport and productivity during the training session may readily extend to the actual data collection period and may therefore suggest certain pairings for the fieldwork teams. In general, the training should have the effect of creating group norms for the ensuing data collection activity. This norm-building process is more than an amenity; it will help ensure supportive reactions, should unexpected problems arise during the data collection.
Source: Yin K Robert (2008), Case Study Research Designs and Methods, SAGE Publications, Inc; 4th edition.
The post Preparation and Training for a Specific Case Study appeared first on HKT Consultant.
]]>