Understanding Evaluation

The evaluation of programs and services is a critical part of the day-to-day work of organizations. While we all hope that the programs we design and implement are beneficial to participants, the only way that we can truly know if we are hitting the mark is to gather credible evidence. At first, evaluation can seem overwhelming and perhaps anxiety-provoking. How do we gather evidence? What if the evidence is not what we had hoped for? How to we break programs down so we can see what components are working and what components need addressing?

The first place to start is developing a clear evaluation plan once you have your program in place. When developing a new program, based on solid theory, the ideal sequence of evaluation activities is as follows.

Process (implementation) evaluation: This type of evaluation examines whether a program is delivered in the way it was intended to be delivered. For example, if you have a social skills program that consists of ten sessions, it would be important to know the answers to questions such as: Are participants receiving all the lessons? Are the lessons being conducted as they were originally planned? Are the staff who facilitate the lessons adequately trained? Are there adequate resources being put into the program so that it can run smoothly? These are just a few of the questions that are very important because if the program is shaky in the implementation phase, the outcomes of the program will no doubt be compromised.

Outcome Evaluation: Once you have a program that is being implemented as intended, you may be ready to move onto an outcome evaluation. In this type of evaluation, we want to know if participants are achieving the intended benefits from the program – has changed occurred as a result of the program? For example, if a program is designed to encourage healthier lifestyle choices (e.g., better nutritional and exercise habits), are participants who are in the program more likely to make healthier lifestyle choices as a result of their participation? What is it about the program that helped or hindered participants in meeting program goals? There are many methods that can be used in outcome evaluations and the choice of the method(s) is heavily influenced by factors such as the type of program, program participants, evaluation questions and allocated evaluation resources.

Impact Evaluation: This type of evaluation involves taking a deeper look at a program and asks the key question – ‘If the program did not exist, would people/community still have arrived at the same outcomes over time?’ For example, if an organization runs a program to encourage smoking cessation, would participants of that program have been just as likely to quit smoking given other available supports in the community?

The following guides are helpful tools for organizations that are interested in learning more about evaluation.

Health Evaluation Guide
Produced by the Canadian Institutes of Health Research (CIHR)

Project Evaluation Guide for Nonprofit Organizations
Produced by Imagine Canada, this guide is a step-by-step guide.

Participatory Evaluation

This chapter is part of a comprehensive toolkit for communities. The guide contains a wealth of information on the promotion of community health and development.