What do we gain when grantees report on the same outcomes?
When grantees report on the same outcomes, you gain information you can compare and combine across the portfolio. You can see how many organizations are making progress on a shared goal, where results are stronger or weaker, and which patterns hold across different kinds of organizations. That's the foundation for telling your board what the portfolio as a whole is accomplishing, which is hard to do when each report measures something different.
Shared outcomes also give grantees a reason to learn from each other. Kania and Kramer's description of collective impact makes the same point about shared measurement: collecting data consistently on a short list of indicators keeps the organizations' efforts aligned and lets them learn from each other's successes and failures. It works only when the list is short and the organizations agree on what each outcome means, which is why the design matters as much as the decision.
What does it cost to ask every grantee for the same outcomes?
Asking every grantee for the same outcomes costs grantee time, and it can cost you information about the work that doesn't fit the shared measure. Each outcome you add is one more thing every organization has to track, collect, and report, often with limited staff. An outcome that fits some grantees' programs poorly can also produce numbers that say little about what those organizations are doing, while their more meaningful results go unreported.
That second cost is why funders often worry that shared measurement will flatten local differences. Will shared measurement flatten what makes each site different? takes up that question directly, and How do I ask grantees for data without overwhelming them? covers keeping the reporting burden manageable.
Is there a middle ground between everyone the same and everyone different?
There's a middle ground between asking every grantee to report the same outcomes and letting each report differently, and most portfolio designs use some version of it. Three common designs are:
- A few shared outcomes plus local ones. Every grantee reports two or three common outcomes in the same way, and each adds outcomes that fit its own program.
- Shared questions, local measures. Every grantee answers the same evaluation questions, but each chooses the measures that fit its work, and the results are combined as themes.
- Shared outcomes for groups of similar grantees. Organizations doing similar work report on the same outcomes, and the portfolio is described group by group.
Which design is appropriate depends on how similar your grantees' work is and on what your board and your grantees need to understand. A portfolio of organizations running similar programs can share more outcomes than a portfolio that funds widely different work toward the same broad goal.
How do we decide which outcomes everyone reports?
To decide which outcomes everyone reports, start from what your foundation or network needs to learn and decide, then choose outcomes that make sense across your grantees' work and that the grantees can report without building something new. Involve the grantees in choosing them, because they know which outcomes reflect their work and what they already collect. Agree on how each outcome is defined, measured, and timed, and say how the information will be used and who will see it before anyone reports.
Check that each shared item is an outcome, meaning a change for the people served, and that it isn't simply a count of activities. What's the difference between outputs and outcomes, and why does it matter to funders? explains the distinction.
The Grantibly perspective
Lauren's perspective
We believe evaluation should collect enough of the right data to answer the questions that matter. That means making intentional choices about what needs to be measured, when it needs to be measured, who needs to be asked, and what will be done with the information.
It's easy for evaluation plans to grow. Add another survey. Add another measure. Ask another group. We would rather have a focused evaluation that produces useful evidence than an enormous evaluation plan that creates survey fatigue, burdens project staff, and generates data no one has time to use.
