This is the second in a series of six articles that discuss outcome measurement: what it is, how to do it, and most importantly, how it will help your organization. The content of these articles draws heavily on a framework designed by the United Way of America, in addition to the author’s experience and research. While this is not the only system for measuring outcomes, it has been proven effective by many organizations in both Canada and the United States.

Last month, we learned the basic concepts used in outcome measurement. This month, we will look at the preparation needed to implement outcome measurement in your organization. This preparation includes securing buy-in, identifying resources, defining the work, and communicating the plan for implementation. Once prepared, your organization can move forward with defining outcomes and logic models for the programs selected for measurement.

Champion the cause

Before planning can begin, you need to ensure there is support for this initiative from key levels of your organization. You’ll need buy-in from the board, senior management, program staff, and volunteer. The executive director must be willing to champion the cause, demonstrating that this is a priority by allocating the staff, time, and money needed, while publicly supporting the work and joining in as required.

You’ll also need to articulate the benefits of outcome measurement clearly in order to champion it effectively. How you position these benefits will depend on the priorities of your organization. You may find it useful to involve key stakeholders when selecting your organization’s key benefits; not only to get broader input, but broader support as well.

Some key benefits include:

• Improved programming
• Demonstrated value of service
• Empowered future strategy
• Increased morale
• Funding justification
• A culture of accountability

 

Know your limits

Make sure that you learn the limitations of outcome measurement too. When you’re clear on these limits, you will avoid frustration and wasted effort later. Here are some things outcome measurement will not do:

• Verify program impact. Outcome measurement evaluates whether or not your organization achieved an outcome. To know the extent to which your program, rather than external influences, was responsible for that achievement requires an expensive process called impact research. While impact research is impractical for most Canadian charities, measuring outcomes properly provides a decent description of a program’s effect at a fraction of the cost.
• Measure program efficiency, or identify where a program is failing. To gauge a program’s efficiency or identify improvement opportunities, use traditional measurement of inputs, activities, and outputs.
• Verify your selection of outcomes, indicators, or measurement procedures. The selection of meaningful outcomes and indicators can only be accomplished through careful deliberation; test your measurement procedures to ensure that they produce reliable data. To avoid having to redo your work, spend enough time planning, designing, and testing your measurement efforts.

 

Show me the money

While knowing the benefits and limitations of outcome measurement will help you explain why this investment is important, you will still encounter questions and concerns. Some will certainly ask, “Where will the money come from?” and you must know your response in advance. There are many ways to reduce the cost: partner with another organization, seek pro bono work from professionals, find in-kind gifts of space or communication expenses, and apply for capacity-building grants. Identifying potential funding sources before someone raises this question will help move you toward successful implementation.

The workgroup

Of course, no project can survive on money alone. You will need a small, focused workgroup consisting of paid staff, rather than participants. While participants may be passionate about a program and offer needed input, the amount of work required and responsibility assumed is usually too much for anyone other than paid staff and committed volunteers. In order to keep the workgroup a manageable size (five or less for most programs), you may find it useful to have a core group of members and call on others as required. The core group should have representation from the following areas (one person can represent more than one area):

• Program staff – both management and ground-level implementation
• Development staff
• Administrative and data support staff
• Leadership volunteer – board or committee members

 

Productive groups require certain qualities. All members must possess an understanding of the ground-level work the organization performs and why it is done. All members must understand the concept of outcome measurement, and be able to communicate its key benefits with others. Perhaps most importantly, all members must be comfortable sharing ideas and negotiating disagreements constructively. The workgroup is charged with specific project deliverables. These are:

• Selecting measurement candidates
• Proposing a timeline
• Selecting program outcomes
• Developing a program logic model
• Selecting outcome indicators
• Designing data collection procedures and tools
• Testing the measurement process
• Evaluating, refining, and implementing the measurement process

 

Over the course of the project, the workgroup will need to perform additional duties. If additional resources become needed – whether time, money or expertise – the group needs to identify and communicate these while estimating and reducing the associated cost. The group must also maintain ongoing communication, including seeking input from stakeholders and communicating planned activities broadly.

Scoping the project

The workgroup’s first task is selecting which program(s) to start with. By selecting a single program to measure, the workgroup can learn from the process while minimizing the risk involved. By selecting multiple programs, the workgroup can streamline effort, but with greater risk. In either case, start with programs that have a clearly defined purpose and client base. Success will be most meaningful for programs that represent a reasonable portion of your organizational activity, especially if funders have expressed interest in the results of those programs.

This is also an ideal time to take inventory of your current measurement practices. What about these programs are you currently measuring? Are these measurements consistently reported and tracked from year to year? If you find your current measurement of inputs, activities and outputs to be inadequate, this may be something you want to incorporate in your efforts. Remember, outcome measurement is intended to compliment, not replace, traditional measurement.

Setting the pace

Once the scope of the work is defined, chart a timeline. This timeline is not final – it could change depending on which outcome indicators are chosen and how the measurement procedures are designed. Getting the planning right is far more important than meeting an arbitrary deadline; being flexible can prevent wasted effort. Consider the effect that program cycles and other deadlines will have and be sure to leave time for delays. Once the timeline is completed, share it with relevant staff and volunteers to keep everyone on the same page. If there are any funders who have expressed interest in the project, this is a good time to provide them with an update.

Next month we will explore the exciting and challenging topic of selecting outcomes and building logic models.

Eli Bennett has been serving the Canadian philanthropic sector for seven years. A graduate of Humber’s Fundraising and Volunteer Management postgraduate program under Ken Wyman, Eli has extensive experience raising millions of dollars through various media across Canada. Currently, Eli is applying his passion for objective management to service provision and program design. If you have any questions on applied measurement in the philanthropic sector, please contact Eli at elibennett@gmail.com.