Project Evaluation

The term “evaluate” means to set the value of or appraise. A project evaluation appraises the progress and performance relative to the project’s initial or revised plan. The evalu­ation also appraises the project against the goals and objectives set for it during the selec­tion process—amended, of course, by any changes in the goals and objectives made during the project’s life. In addition, evaluations are sometimes made relative to other similar projects.

The project evaluation, however, should not be limited simply to an after-the-fact analysis. Rather, it is useful to conduct an evaluation at a number of crucial points during the project life cycle. Because the primary purpose of a project evaluation is to give feed­back to senior management for decision and control purposes, it is important for the evaluation to have credibility in the eyes of both senior management and the project team. The control purpose of evaluation is meant to improve the process of carrying out projects. The decision purpose is intended to improve the selection process. Thus an evaluation should be as carefully planned and executed as the project itself.

The use of postproject evaluation is to help the organization improve its project- management skills on future projects. PMBOK refers to this as “lessons learned,” a topic PMBOK covers in Chapter 8 on Quality. This means that considerable attention must be given to managing the process of project management. This is best accomplished and most effective if there is already a project-management guide or manual detailing stand­ardized project-management practices for the organization. Such manuals commonly cover best practices for planning, monitoring, and controlling projects and may include advice on both selecting and closing projects. An example of such a manual was that developed by Johnson Controls, described in Chapter 7, where internal benchmarking of their most successful project managers identified four sets of detailed project manage­ment procedures. These procedures are updated with each new project experience so that the learning that has occurred is captured and made available for future projects.

1. Evaluation Criteria

There are many different measures that may be applied in a project evaluation. As indicated, senior management may have particular areas they want evaluated for future planning and decisions, called “ancillary goals,” and these should be indicated in the charge to the evaluation committee. Beyond that, the original criteria for selecting and funding the project should be considered—for example, profitability, acquiring new competencies for the organization, or getting a foothold in a new market segment. Any special reasons for selection should also play a role. Was this project someone’s sacred cow? Did a scoring model identify particularly important quantitative or qualitative reasons to select this project? Was the project a competitive necessity? How the project is progressing on such criteria should be an important part of the evaluation as well.

Certainly, one of the major evaluation criteria would be the project’s apparent “success” to date. One study (Shenhar, Levy, and Dvir, 1997) identified four important dimensions of project success. The first dimension is simply the project’s efficiency in meeting the budget and schedule. Of course, efficiency does not necessarily translate into performance, or effectiveness, so the second (and most complex) dimension is customer impact/satisfaction. This dimension includes not only meeting the formal technical and operational specifications of the project but also the less tangible aspects of fulfilling the customer’s needs, whether the customer actually uses the project results—that perennial challenge of customer “satisfaction.”

The third dimension is business/direct success meaning, for external projects, factors such as the level of commercial success and market share and for internal projects, the achievement of the project’s goals such as improved yields or reduced throughput time. The final dimension, more difficult to assess, is future potential which includes establish­ing a presence in a new market, developing a new technology, and such.

To this list we would add two other criteria: the project’s contribution to the organi­zation’s ancillary goals and objectives, and the project’s contributions to the objectives of project team members. To recognize the project’s contributions, all facets of the project must be considered in order to identify and understand the project’s strengths and weaknesses. The evaluation report should include the findings regarding these two criteria, as well as some recommendations concerning the items in Table 8.1.

2. Measurement

Measuring the project’s performance against a planned budget and schedule is relatively straightforward, and it is not too difficult to determine if individual milestones have been reached. As we have noted several times, there are complications regarding measure­ment of actual expenditures and earned values, as well as with reporting on difficult technical issues that may have been deferred while progress was being made along other fronts.

If the project selection process focuses on profits, the evaluation usually includes determination of profits and costs and often assigns these among the several groups work­ing on the project. Conflict typically results. Each group wants credit for revenues. Each group wants costs assigned elsewhere. Although there are no “theoretically correct” solutions to this problem, there are politically acceptable solutions. As with budget and schedule, these are most easily addressed if they have been anticipated and decided when the project was initiated rather than at the end. If allocations are made by a predeter­mined formula, major conflicts tend to be avoided—or at least lessened.

When a multivariate model (e.g., a scoring model, such as Table D in the scoring model example in Chapter 1) has been used for project selection, measurements may raise more difficult problems. Some measures may be objective and easily measured. Others, however, are typically subjective and may require careful, standardized measure­ment techniques to attain reliable and valid evaluation results. Interview and question­naire methods for gathering data must be carefully constructed and executed if their results are to be taken seriously.

Source: Meredith Jack R., Mantel Jr. Samuel J., Shafer Scott M., Sutton Margaret M. (2017), Project Management in Practice, John Wiley & Sons, Inc. 3th Edition.