Why and when to evaluate?
The primary purpose of evaluation is to measure the success of a project or program by the end of the grant. An evaluation can begin part way through a project or at its completion.
The Ontario Trillium Foundation considers outcome evaluation an important part of its granting process. An evaluation is intended to provide information for grantees as well as for the Foundation. Both are interested in identifying areas in a project or program that require improvement by answering questions such as the following:
- What works? What doesn't work? Why not?
- Are the needs of the target population being met? If not, why not? How can they be met better?
- Do the program materials serve the intended purpose? If not, why not? What improvements can be made to the material?
- What are the key components of the project or program that are worth replicating?
The funder is also interested in ensuring the accountability of funds by:
- collecting evidence on the effectiveness or impact of a project or program
- demonstrating to stakeholders that funds are being used both efficiently and effectively
Developing an evaluation plan
Although an evaluation may not actually be undertaken until midway through a project or at its completion, planning for an evaluation must be incorporated into the project design prior to start of the project. Developing the plan prior to the beginning of the project enables the Grant Applicant to:
- identify and put in place the process and tools necessary to collect and analyze information required to conduct the evaluation
- identify and obtain resources necessary to carry out the evaluation
An evaluation should inform you about the success of your results and activities that you developed for your project. The indicators or measures that you defined for your project results and activities serve as the basis for your evaluation.
Your evaluation plan should specify the following:
- Data collection methodology: how you will collect the information you need for your measures or indicators (e.g. through activity logs, attendance records, a review of project documents, face-to-face interviews, phone survey, focus groups).
- How you will analyze the information
- Reporting and utilizing the evaluation results: what you will report and with whom you will share your findings
- Responsibility and time frame: who will conduct and assess the information and when it will be done
Fundamentals of evaluation
- Structure your evaluation to suit the size and complexity of your project. If your grant request is for $15,000 to purchase and plant trees in a local park, your evaluation plan will be very simple since the results and activities will be both straightforward and fairly limited in scope and number. On the other hand, a three-year project involving several collaborators and funding in the $250,000 range will require a more detailed evaluation.
- Data collection: if possible, use a combination of methods. Collecting both "before" and "after" information will enable you to demonstrate change over the life of the project.
- Data analysis: review all of the information collected and group the information into emerging themes or patterns. Involving as many people as possible in interpreting the information may yield varying perspectives.
- Reporting the results: reports outlining your findings may be brief and should highlight the following:
- what you have achieved
- unexpected results
- lessons learned
- Utilizing the results: evaluation results should be fed back into project planning in order to make any necessary adjustments. Findings should be shared with staff, volunteers, funders, program participants, partner organizations and others doing similar work.