What parts of evaluation can a small team do on its own?
The parts of evaluation a small team can do on its own are mostly the ones that depend on knowing the program, its participants, and its context well:
- Naming what you need to learn. Decide what you need to understand, improve, or explain, and what you might change depending on the answer. This is where your team's knowledge of the program, its participants, and its day-to-day realities matters most.
- Using what you already have. Start with what exists, such as participation and attendance records, applications, intake forms, and any surveys you already run, and collect only what's missing.
- Asking participants about their experience. Short feedback forms or conversations can tell you what participants found useful and what got in their way. Explain what you're asking, why, and how the information will be used, keep it secure, and report results so no one can be identified.
- Reviewing results together and acting on them. Set aside time as a team to look at what you've learned, decide what to change, and tell participants what is changing because of what they told you.
These activities make your organization a stronger user of its own evidence, and they also make any future evaluation easier, because the records and the habits are already in place.
Will a funder or board take our own evaluation seriously?
Whether a funder or board takes your own evaluation seriously depends mostly on what you claim from it and how clearly you describe how it was done. Say who you asked, how many people responded, what you asked them, and what the results can and can't show. A careful description of what participants said and what your records show, with its limits stated, is more useful to a reader than a broad claim about impact that the evidence can't support.
Some audiences will need more than a self-evaluation. If a funder requires an independent evaluator, evaluating the program yourselves won't meet that requirement, though your team can still track routine information alongside the evaluator. Should I hire an external evaluator or handle evaluation in-house? walks through that decision. Participants may also answer differently when the person asking is the staff member who provides their services, which is one reason an outside evaluator can matter even for a small program.
Where does an evaluator's design work matter even for a small program?
An evaluator's design work matters even for a small program because the most consequential choices in an evaluation are made before anyone collects anything. Designing an evaluation means choosing the questions it will answer, deciding what evidence would answer them, planning when and from whom to gather it, and writing survey or interview questions that produce usable answers. It also means interpreting the results in light of how the program was carried out. Those are specialized skills, and many self-evaluations go wrong at this step, often by starting with a survey before anyone has agreed on what it should answer. We ran our own survey and the results were useless. What went wrong? looks at how that happens.
Help with design doesn't have to mean a full evaluation. A workshop on one skill, such as writing survey questions, or a focused planning step that names your priority questions, the evidence you already have, and the most proportionate next step can give your team a sound plan to carry out. Can a small nonprofit afford real evaluation help? describes those options, and What parts of evaluation should we never DIY? covers the work that shouldn't be done alone at all.
How much staff time does evaluating our own program take?
How much staff time evaluating your own program takes depends on how many questions you try to answer and how much new information you collect, so keeping both small is the most effective way to keep the work manageable. Every additional survey or measure creates work for the people answering it and for the staff who have to gather, organize, and make sense of it.
The harder issue is usually protecting that time. Evaluation done in-house often falls to someone who already has a full job, and it tends to slip when the program gets busy, which is often when the information would be most useful. Name the person responsible, set aside regular time for the work, and choose one or two questions you can answer well this year rather than several you can only answer partly.
The Grantibly perspective
Lauren's perspective
We want clients to understand their own data, and to become better at asking questions like these: What evidence would tell us whether this is working? How will we know if this change helped? Whose perspective are we missing? Are we collecting data we actually need?
We don't want clients dependent on us to make sense of their work. We believe evaluation should collect enough of the right data to answer the questions that matter.
