How do we know how much evaluation we need?
How much evaluation you need depends on what you need to understand, decide, improve, explain, or do, so begin by identifying those needs before anyone chooses a survey or a method. A few questions help determine the size of the work:
- What decision will this inform? For example, choosing between two versions of a workshop requires far less evidence than deciding whether to expand a program to several new sites.
- Who will use the results? Your own staff, your board, and your funders each need a different kind and amount of evidence, depending on their responsibilities.
- What do you already have? Attendance records, intake forms, previous surveys, and participant feedback may already answer part of the question.
- What can your organization take on? Every additional survey or interview requires time from your staff and from the people your organization serves.
- How much is riding on the answer? The more consequential the decision, the more you'll want to rely on careful, well-designed evidence.
An evaluator can help you work through these questions before any scope is set, because designing an evaluation that fits the organization and its decisions is part of an evaluator's expertise.
What are the alternatives to a full evaluation?
The alternatives to a full evaluation are focused pieces of work that answer one important question or set up the next step, without committing your organization to a large study. Three forms this can take:
- A review of the evidence you already have. An evaluator examines the records, surveys, and feedback your organization already collects and identifies which questions they can and can't answer.
- A focused study of one question. For example, a program might want to understand why participants stop attending after the first month, without evaluating everything else the program does.
- A planning step before any evaluation. This identifies the decisions you face, your priority questions, the evidence you have, the important gaps, and the most proportionate next step. At Grantibly, this is the purpose of our Data-to-Action Roadmap.
Each of these can stand on its own, and each can also become the first stage of a larger evaluation if the answer shows you need one. If you're deciding whether new surveys are needed at all, Do we need new surveys, or can we use what we already collect? looks at that question.
When is a full evaluation worth it even if no one requires it?
A full evaluation can be worth it even if no one requires it when a lot depends on the answer. That includes deciding whether to continue, expand, or replicate a program, making claims about outcomes to funders or the public, or seeking significant new funding that will ask what the program has accomplished.
It's also worth considering when you need to understand how a program is operating as well as whether it's producing the results you hoped for. Outcomes are hard to interpret without knowing who took part, how much of the program they received, and what changed along the way. A fuller evaluation follows both implementation and outcomes, which helps the team identify what may lie behind a disappointing result.
Whether something is required and whether it's useful are two different questions. When a funder says nothing about evaluation, the decision about scope is yours, and it's worth making on the merits.
How do we avoid investing in more evaluation than we need?
You can avoid investing in more evaluation than you need by asking what each part of a proposed scope is for and what decision it supports. A measure or a survey that won't change anything your organization does is a candidate to cut, and so is a question that would be interesting to study but isn't needed to evaluate the program well.
Ask for a smaller version, too. A thoughtful evaluator should be able to tell you what you'd learn from a focused scope, what you'd give up, and whether you could start small and expand later. You should be able to begin with one important question, a workshop, or a roadmap without committing to a large study.
If cost is the main worry, Can a small nonprofit afford real evaluation help? covers the options for smaller organizations.
The Grantibly perspective
Lauren's perspective
We believe evaluation should collect enough of the right data to answer the questions that matter. We don't assume every program needs the most sophisticated possible methodology.
We help clients distinguish between what they need to know to evaluate a project well and what would be interesting to study. Both can be worthwhile. They simply may not belong in the same scope of work.
