How to Apply Content Analysis

GAO evaluators can use content analysis to articulate a program’s objectives, describe its activities, and determine its results.

1. Program Objectives

Many evaluations characterize a program’s objectives. For example, evaluators might compare a program’s legislative objectives with those of the executive branch. To do this, they might gather written or tape-recorded information from the program’s legislative history and from interviews with agency officials. In content analysis, they would then be able to compare the two kinds of documentary sources to determine whether the agency’s goals conform to its legislative intent.

2. Program Activities

To describe a program’s activities, an evaluator could perform case studies, attend agency meetings, or interview program stakeholders (for example, managers, service deliverers, or beneficiaries) and then use content analysis to examine the results. For example, GAO evaluators might ask. program stakeholders open-ended questions about a program’s activities and then describe them by simply tabulating the categories of activities the respondents have reported.

The extent to which program activities were accurately targeted could also be investigated. Evaluators could interview program beneficiaries and analyze their responses to assess their eligibility for the program’s services. The responses could then be compared with established eligibility criteria, and the evaluators could estimate the proportion of program recipients who were truly eligible.

3. Program Results

When evaluators want to estimate the results of a program, they might take sample surveys, construct case studies, or examine earlier evaluation reports. When such data are quantitative, a variety of statistical procedures can be applied. (See GAO, 1992e, and Mohr, 1988.) However, to the extent that such data are textual, the evaluator can estimate program results with the help of content analysis.

Evaluators may analyze content when they are, for example, uncertain about program effectiveness criteria or when they find many diverse criteria within the program, are engaged in exploratory work, want to ensure that structured questions did not miss
something, or want to clarify the meaning of close-ended questions.

Source: GAO (2013), Content Analysis: A Methodology for Structuring and Analyzing Written Material: PEMD-10.3.1, BiblioGov.

Leave a Reply

Your email address will not be published. Required fields are marked *