Why might a program look like it isn't working?
A program might look like it isn't working because it wasn't carried out as planned, because participants didn't receive enough of it, or because conditions changed around it. A disappointing result doesn't automatically mean the idea behind the program was wrong, and it doesn't automatically mean it was right, either. It means there is something to understand.
That's why outcomes are hard to interpret without knowing how a program ran. An evaluation that looks at implementation asks questions such as these:
- Was the program carried out as intended?
- Who participated, and who didn't?
- How much of the program did participants actually receive?
- What barriers came up, and what changed along the way?
- Did participants at different sites have the same experience?
Sometimes the answers show that the model itself needs refinement. Other times they point to something more specific, such as a recruitment problem, a schedule that kept people from attending, or one part of the program working better than another. Your evaluator can help you sort out which of these explains your result, because the answer shapes what you do next.
How do we decide what to change first?
To decide what to change first, look at what the evaluation shows about why the program fell short: who was reached, what participants experienced, and where things broke down. Then talk it through with your evaluator and start where the evidence points.
Numbers and participant feedback usually tell different parts of that story. A survey might show that participants were less satisfied with one part of the program, and interviews or focus groups might explain why. Completion records might show that one group finished at lower rates, and participant feedback might reveal barriers the numbers can't identify on their own. Read together, they help you see whether the first change belongs in how participants are recruited, how the program is delivered, or how one component is designed.
The people who run the program day to day are part of this conversation too. They may know why something played out the way it did, and they will be the ones carrying out the change. Once you make a change, keep collecting information so you can see whether it helped. Our program changed mid-grant. Can we change the evaluation plan too? covers what that means for your evaluation plan.
How do we share this with the people who support the program?
When you share a disappointing result with the people who support the program, share what you learned and what you're changing. That shows your funders, board, and partners that you're listening, adapting, and using resources responsibly.
It also gives you a stronger story than a report that says everything went well. You can show that you collected the information that revealed a problem, took the time to reflect on it honestly, and made changes that serve your participants better. That kind of account demonstrates responsible leadership and careful use of funding, which can matter when you ask for support again.
Telling a funder is its own conversation, with its own timing. How do I tell my funder the results aren't what we hoped? walks through it, and Will an evaluation report that shows our program isn't working cost us our grant? covers what is and isn't at risk.
The Grantibly perspective
Angelika's perspective
If your evaluation shows your program isn't working, I'd say you're lucky. You collected the information that revealed something to improve, and now you have the chance to reflect on it honestly, make improvements, and really change the lives of your participants.
You also get to show the people who support your program that you're listening and willing to adapt, change, and innovate. That shows them you're thinking critically about the work you do and managing the resources it takes to make an impact. And you can share your plans for even greater impact moving forward.
Evaluation isn't about looking for evidence that everything worked. The purpose is to learn. We want clients to see evaluation as a source of information rather than a judgment, and a program that needs work as an opportunity to adapt and grow.
