Introduction

It is increasingly common for funders to request a project evaluation. They want to know that they got some bang for their buck. This is especially true in the not-for-profit and public sectors, where the budget is tight. Lola was not immune to such requests, and we were back in the coffee shop discussing how to best evaluate her project to build a donors’ database.

Types of Project Evaluation

There are many, many methodologies for evaluating projects. A quick search on the Internet will probably provide the novice project manager with a gaggle of resources (perhaps that is how you found this article) to dissect every aspect of the project’s success or failure. A helpful tip is to realize that almost all of these resources will fall into one of three types:

1. Project management evaluation: Evaluation of the project management techniques and processes that were used to deliver the project.
2. Project impact: Evaluation of the impact that the project had on the organization or the target group.
3. Project deliverables: Evaluation of the quality of the deliverables that the project team created.

Let’s look briefly at each of these three types and see a few techniques that Lola might employ for her project.

Project Management Evaluation

Project management evaluation examines the “how” of project delivery. It examines the process of project management. Evaluating the project management allows the project team and organization to learn from its successes and mistakes, and hopefully deliver an even better project the next time. Some of the key questions that you will answer during the project management evaluation are:

• Was the project delivered on time and budget?
• Were risks to the project identified and addressed? How well did the project team respond to crises that occurred?
• How effective were the project communications? Were people adequately informed? What methods worked well? What methods were not as successful?
• How well did the team work together? Was conflict addressed and resolved? Were team members encouraged to generate ideas and contribute to project strategy?
• Was change well managed? Was the impact of change understood? Did the team effectively implement change?

I suggested to Lola that she conduct the project management evaluation by calling a meeting of her project team and other key stakeholders. This meeting would be similar to a focus group in which the team answers the questions such as the above. I further suggested that she ask an impartial person (i.e. someone not connected to the project) to facilitate and document the session. The number one rule of this meeting is no blame. The review is not intended to blame individuals or call attention to people’s professional weaknesses. It is to identify what went well and what did not so that future projects can be delivered more smoothly. Like a wise project manager once said to me, it is to understand “what went well, and what could go even better!”

Project Deliverables Evaluation

This evaluation examines the quality of the project deliverables by comparing them against standards or benchmarks. The standards can be defined by either internal or external people or organizations. Similarly, they can either be codified (i.e. written down) or simply common knowledge among people in the industry (i.e. best practice). The project team should identify in the Project Charter which standards they will meet.

Lola was somewhat flummoxed by this explanation so I gave her an example. She had just finished landscaping her backyard to make more room for the puppy she recently acquired. The deliverable is a backyard to accommodate her puppy. She would need to meet several standards in the course of completing her project. The fence cannot be more than six feet high. This is a codified standard set by the municipality in which she lives. She cannot plant the flowers less than two feet apart. This is simply common knowledge shared by gardeners. She needed to put all the plants in a straight line. This was a standard defined by herself (i.e. internal standard) to suit her sense of aesthetics. After completing the backyard, she reviewed it to ensure that it met all of those standards that she had identified at project beginning.

Let’s now look at her database project as an example. Lola’s donor database had to meet legislative requirements for information privacy. Lola compared the database against a checklist that she had made of the privacy legislation to ensure that it met the legislative requirements. She also hired a consultant to review the database to ensure that it was technically sound. The consultant confirmed that it observed best practices in the technology industry. Both the privacy checklist and the independent review evaluated the key project deliverable to certify that it was of sufficient quality.

Project Impact Evaluation

The third type of evaluation examines the impact of the project. Recall our discussion about the “why” of the project. All projects are delivered for a reason – to fulfill a need. Lola delivered her project to make communications with donors more efficient. Evaluating the impact means identifying whether the project did indeed make communications with donors more efficient.

The most important point of impact evaluation is identifying a metric by which the team can determine whether it achieved the objective. This is the project management way of saying, “How do you know you accomplished what you intended?” How would Lola know that communication was made more efficient after the database implementation? She needs to define some way of measuring the efficiency of communications, and then measure it before and after her project.

Lola talked to the project team and decided that a good metric to indicate more efficient communications would be a reduction in the amount of returned mail. The agency misaddresses about 10% of mail-outs, thus those letters are returned. If this were to reduce to 5%, Lola could assert that the database made communications more efficient. Another metric she identified was the time it takes to send a letter to a donor. The development officer spends about ten minutes finding an address in the files, typing it into the letter, and addressing the envelope. If Lola could reduce this to five minutes, she could say that she made the communications more efficient.

After deploying the database, Lola re-measured the amount of returned mail and the time required to address and send a letter. Although 10% of mail was still returned, the time required to address and send each letter was reduced to only two minutes per letter. In her evaluation summary, Lola noted that the project had achieved the intended objective of making communications more efficient, and made suggestions for potentially reducing the amount of misaddressed mail.

Summary

There are many ways to evaluate a project. Almost all of which fall into one of three categories: project management, project deliverables, or project impact. The purpose of evaluation is not only to satisfy the funding requirements, but to help the organization deliver even better projects in the future.

Lola’s project evaluation received mixed results. This is a normal course of projects. Don’t be too upset if the project evaluation is not as glowing as you had hoped. It will teach you valuable lessons for doing better the next time!

Blair Witzel (blair@mcdoane.com) is a member of the Project Management Institute and a consultant with McDonnell Doane + Associates, an information management and technology firm focusing on the not-for-profit and public sectors. His work centres on managing multi-project portfolios and working with organizations to develop project management methodologies to more effectively deliver projects.