Guide 10 · For funders and networks

The Funder's Guide to Portfolio Learning Without Grantee Burden

Boards want evidence, and grantees have little time to spare. This guide shows how a foundation or network can learn across its portfolio in a way that serves both, from the questions it asks to what it gives back.

The short version

A funder can learn what its portfolio is accomplishing without exhausting its grantees, as long as the learning is designed for grantees as well as for the board. That means asking only for what the foundation will use, sharing a short set of measures while leaving room for each site's own, keeping support and monitoring separate and saying which is which, and returning what was learned to the grantees who made it possible.

Lauren McClain, Ph.D.

Lauren McClain, Ph.D.

Co-founder and workforce development consultant, Grantibly

grantibly.com/about/

· 13 min read

What You Need to Know

Key takeaways about portfolio learning

What this guide covers
Where it starts

Portfolio learning starts with the questions a foundation or network will use

Portfolio learning starts with the questions a foundation or network will use, because those questions decide what's worth asking grantees for and what can be left out. A program officer often sits between two responsibilities: a board that wants to know what the grants are accomplishing, and grantees who are running programs with limited staff and time. The easiest response to a board's interest is to add to the reporting form.

Each addition can seem reasonable on its own. Together, they can produce more information than anyone at the foundation has time to read, while the organizations supplying it spend hours they could have given to their programs.

The alternative is to decide first what your organization needs to understand and what it will do with the answer. Good evaluation design starts with what you need to know and why you need to know it, and that holds for a portfolio as much as for a single program.

Three questions to answer before any request reaches a grantee. What do our board, our staff, and our grantees need to understand about this portfolio? What will we do differently depending on the answer? Which of those answers can we already find in proposals, past reports, budgets, and site visit notes?

A board question such as "Which kinds of grantees are making the most progress toward our shared goal, and what seems to help them?" points to specific information. A general wish to see the impact of every grant points to everything, which is how reporting forms grow. The example is illustrative; your board's questions will be its own, and they're worth writing down in the board's own words.

The third question matters more than it looks. Much of what a funder needs may already be in its own files, and reading what grantees have already sent is the most direct way to ask them for less. How do I ask grantees for data without overwhelming them? covers the request itself.

The rest of this guide is about the design around it: what to share across the portfolio, what to leave to each site, how to keep support apart from monitoring, and how to turn what you receive into an account your board can use and your grantees can learn from.

The cost a funder rarely sees

Grantee reporting grows one request at a time, and grantees may not say when it has grown too far

Grantee reporting grows one request at a time when each question is added for a good reason and none is reviewed for whether anyone still uses it, and grantees may not say when it has grown too far, because the funder decides whether they're funded again.

On the funder's side, a new question is one more line on a form. On the grantee's side, it's time spent tracking, collecting, and writing up information, often by staff who are also running the program, and sometimes by participants asked to complete one more survey.

Across a portfolio, that time is multiplied. A measure added to a common form is collected by every grantee that uses the form, so the work grows with the size of the portfolio, while the value of the answer depends on whether someone at the foundation will use it. The founders' view of evaluation applies directly here: more data isn't necessarily better data, and every additional measure creates work for the people asked to provide it.

The power dynamic is what keeps this cost out of view. As sociologists, the founders pay attention to who's included, who's missing, and who holds decision-making power, and in a funder's relationship with its grantees, the funder holds most of it. A grantee that's worried about its renewal may find the time somehow, even when a request takes staff away from its programs, and may describe the request as manageable when the person asking also reviews its renewal.

Some funder networks have written about this directly. The Trust-Based Philanthropy Project, named here as one example of a framework written for funders, lists "simplify and streamline paperwork" and "solicit and act on feedback" among its six practices. Adopting a particular framework isn't the point.

The idea underneath it's that a request should be worth the time it takes, and the people who spend that time are the ones who can tell you whether it's. A few practical ways to hear an honest answer:

Shared measurement

A short shared list of measures lets a funder see patterns across its grantees

A short shared list of measures, reported the same way by every grantee, lets a funder see patterns across its portfolio and give its board a credible account of the whole, and keeping that list short is what keeps it workable. When each report measures something different, a funder can describe individual grants but has little basis for saying what the portfolio accomplished together.

Shared measurement has a recognized place in the field. John Kania and Mark Kramer's 2011 article on collective impact says that developing a shared measurement system is "essential to collective impact" and describes collecting data consistently on "a short list of indicators" across the participating organizations. The same article acknowledges that it may seem impossible to evaluate many different organizations on the same set of measures, and that developing a shared measurement system takes real investment.

Many portfolio designs sit somewhere between asking every grantee for the same outcomes and letting each report differently. A design that works usually gives each kind of information its own job:

1
Shared measures

A few outcomes every grantee reports with the same definitions and timing, so results can be compared and combined.

2
Local measures

Outcomes each grantee chooses because they fit what it's trying to change in its own community.

3
Context

Who took part, how the work was carried out, and what changed along the way, so each result can be interpreted fairly.

The shared list only helps if each item means the same thing everywhere. Agreeing with grantees on definitions and timing before anyone reports is what makes the numbers combine, and involving grantees in choosing the list matters for a second reason: they know which outcomes reflect their work and what they already collect.

Each shared item should also be an outcome, meaning a change for the people served, and not only a count of activities. What's the difference between outputs and outcomes, and why does it matter to funders? explains that distinction.

How short is short depends on what grantees can reasonably report and on what your foundation will use. If you can't say what you would do with an outcome's results, it probably doesn't belong on the shared list. Should all our grantees report on the same outcomes? walks through the decision, including the hybrid designs that sit between everyone the same and everyone different.

Local difference

Shared measurement keeps local differences visible when results are read site by site

Shared measurement keeps local differences visible when results are read site by site and paired with context, and it flattens them when every site's result is combined into one number and compared without explanation. Funders worry about this for good reason. Grantees serve different people, under different conditions, with different resources, and a measure that has to fit every site tends to capture the most general part of the work.

Two habits do most of the flattening. The first is averaging: a portfolio-wide figure can look steady while some sites do much better and others struggle, and small sites carry little weight in the total even when what they're learning is important. The second is ranking: grantees that expect to be compared on a number without context have understandable reasons to report cautiously, and the funder learns less as a result.

Flattens differences

Read as one number

Results are combined before anyone looks at individual sites, and sites are compared on a figure with nothing about who they serve or how the work was carried out.

Keeps differences visible

Read site by site, with context

Each site's results are reviewed first, alongside information about implementation, and the people at each site help explain what the results mean.

A difference between two sites is a question worth asking: what's different about the people served, the way the work is carried out, or the conditions at each one? If outcomes fall short at one site, that doesn't automatically mean the approach was ineffective there. The program may not have been fully implemented, participants may not have received enough of it, or local conditions may have changed. Likewise, a strong result doesn't automatically tell you what produced it.

The people at each site can often explain a result in ways no number can, so share each site's results with the people who run it and ask what they think explains them before the portfolio account is written.

Will shared measurement flatten what makes each site different? takes up the funder's worry in full. When one initiative runs at several sites under a single design, the same principles shape a multi-site evaluation, and What does a multi-site evaluation look like in practice? describes how one runs over the life of the initiative.

Learning or surveillance

The line between supporting grantees and monitoring them depends on four things being clear

The line between supporting grantees and monitoring them depends on four things being clear before grantees are asked to take part: what the work is for, whether taking part is their choice, what stays confidential, and what information, if any, will reach the funder. Compliance monitoring is a legitimate part of a funder's responsibility to its board, and grantees can plan for a requirement they understand. The trouble starts when monitoring is presented as support.

When that happens, grantees speak openly in what they believe is a learning session and later find that what they said reached their program officer or showed up in a review. One experience like that can change how a grantee approaches every offer of help that follows.

The founders' standard for funder-sponsored work is that it's designed as support for the people taking part, with those four things made explicit, and that it never functions as compliance monitoring in another form.

One activity can serve both the grantees and the funder's own learning, as long as that's decided in advance and grantees agree to how it will work. Settle the four questions before the first session and put the answers in writing, so your staff can keep the promise as well as make it.

Follow-through matters as much as the promise. Grantees may judge the next offer of support by how the last one was handled, and if your organization later wants to use what it learned differently, ask grantees before you change anything. What's the difference between grantee support and compliance monitoring? draws the line in more detail. The same principles of privacy, power dynamic, and follow-through guide ethical evaluation with participants, which How do we evaluate students or participants ethically? covers.

Sponsored help

Sponsored evaluation help and training work best for a funder when grantees help shape them, because the need a funder sees from its side and the help grantees would use can differ.

Sponsoring help makes the most sense when several grantees share a need, such as being asked to report outcomes for the first time without the staff or skills to do it, and when the help can be designed to strengthen the grantees' own work as well as the funder's learning.

Asking first is the most reliable way to get the design right. Recent reports, site visit notes, and program officers' conversations often show challenges that come up across several organizations, and those are a reasonable starting point. A short conversation with a few grantees, a brief survey, or a listening session can then confirm what they're working toward, what's getting in the way, and how much staff time they could give to learning something new.

The format follows from what grantees need and how much time they have. A workshop series or cohort learning suits a shared topic and lets grantees learn from each other. Hands-on help for a single organization suits a need specific to its program or its information.

Scheduling matters as much as format: a session offered during grantees' own reporting deadlines or busiest program season may go unused however relevant the topic. What kind of capacity-building do grantees actually want? covers finding out what grantees want.

The aim is to leave grantees better able to understand and use their own information, without having to become evaluators themselves. That also shapes what a funder can promise its board. You can describe the work, who it reaches, what grantees report using afterward, and the common needs that come up across the group. Promising specific changes in grantee results would claim more than anyone can know in advance, because grantee outcomes depend on many things beyond one sponsored program.

Designing sponsored help draws on evaluation expertise, and an evaluator should design the approach with you and with grantees and keep any role in evaluating your initiative clearly separate from the help itself. Should we sponsor evaluation help for our grantees? covers when sponsoring makes sense and how to structure it so grantees trust it.

One account, given back

A portfolio learning cycle turns many grantee reports into one account a board can act on

A portfolio learning cycle turns many grantee reports into one account a board can act on by reading every report against the board's few questions, checking the interpretation with grantees, and returning what was learned to them before the next round of reporting is planned. The cycle is what connects the rest of this guide: the questions decide the request, the request produces the reports, and what the reports show informs the next request.

Numbers that were measured differently can't be added or averaged without producing a figure that misrepresents every organization in it, which is why the third step matters. Numbers and participants' own descriptions of their experiences tell different parts of the story, and a theme is easier to act on when both point the same way.

How do I turn a mountain of grantee reports into insight for my board? works through the synthesis itself, and How do I explain our results to a board that just wants numbers? helps when the board responds mostly to numbers.

Cross-site synthesis is specialized work. An evaluator experienced in portfolio learning can design the questions with you, read across reports that were never written to line up, facilitate the conversations where grantees interpret the results, and prepare the account for your board.

Grantibly can serve as the lead external partner for portfolio learning or as a specialist within a larger team. Either way, the evaluator's job includes making sure the grantees who supplied the information can see what came of it.

The Grantibly perspective

Lauren's perspective

Evaluation may be required by a funder, but compliance should never be its only purpose. At its best, evaluation helps organizations understand what's working, what isn't working, what they're learning, and what they should do next. For a funder, that's true of its own learning across a portfolio and of the grantees whose work it supports.

Organizations are made of people. Programs operate within relationships, culture, communication patterns, incentives, institutional conditions, and power dynamics, and results can't be understood responsibly by looking at numbers alone. We value the knowledge people have about their own experiences and organizations, and we pay attention to who's included, who's missing, and who holds decision-making power.

At Grantibly, we don't want to be an evaluator who appears once a year, asks for data, writes a report, and disappears. We see ourselves as a partner, not a vendor. We want the organizations we work with to understand their own data, and we don't want them dependent on us to make sense of their work.

We would rather have a focused evaluation that produces useful evidence than an enormous evaluation plan that creates survey fatigue and generates data no one has time to use. Across a portfolio, that choice is multiplied by every organization asked to report.

The Bottom Line

What portfolio learning without exhausting grantees comes down to

Portfolio learning without exhausting grantees comes down to designing the learning for both sides of the funding relationship. A board needs a credible account of what the portfolio is accomplishing, and grantees need their time respected, their differences understood, and a clear account of what happens to what they share. Those needs pull in different directions only when the learning is designed for one side alone.

The work begins with the few questions a foundation will use and ends with grantees seeing what came of the information they provided. In between, the funder keeps its shared measures short, reads results site by site, keeps support and monitoring separate, and says plainly which is which.

A funder learns the most from its portfolio when the organizations in it can see that the learning serves them too.

Sources
  1. John Kania and Mark Kramer, "Collective Impact," Stanford Social Innovation Review, Winter 2011
  2. Trust-Based Philanthropy Project, "The Six Trust-Based Practices"
Frequently Asked Questions

More questions about portfolio learning

Can a funder learn what its portfolio is accomplishing without adding grantee reporting?

A funder can often learn a good deal about what its portfolio is accomplishing without adding grantee reporting, by reading what grantees already send against a few clear questions. Proposals, past reports, budgets, and site visit notes may answer some of those questions outright, and the gaps that remain show what's worth asking for next time.

Is portfolio learning the same as evaluating each grantee?

Portfolio learning isn't the same as evaluating each grantee, because its purpose is to understand patterns across the organizations a funder supports and what the funder itself could do better. It can draw on grantees' own reports and evaluations, and it should be explained to grantees so they don't mistake it for a review of their performance.

Who should run portfolio learning, foundation staff or an outside evaluator?

Portfolio learning can be run by foundation staff or by an outside evaluator, and the choice often turns on trust and on the work involved. An outside evaluator can make confidentiality easier to keep, because grantees aren't speaking directly to the people who make funding decisions, and brings experience reading across reports and sites. Foundation staff bring their knowledge of the grantees.

What should a funder tell its board when grantee results differ widely?

When grantee results differ widely, a funder should tell its board where results differ, what seems to explain the differences, and what the evidence can and can't support. Differences in who a grantee serves, how its work was carried out, or local conditions often explain more than the numbers do, and they can show the board where support is most needed.

Should small grantees report the same way as large ones?

Small grantees don't always need to report the same way as large ones, because the same request takes a larger share of a small staff's time. A funder can ask its smallest grantees for less, accept information in forms they already use, or offer help with collecting it, as long as it explains how the approach was decided.

What if a funder's board wants a single number for the whole portfolio?

If a funder's board wants a single number for the whole portfolio, report the shared measures that were collected the same way everywhere, and show the results by site or by group of similar grantees alongside the total. A combined figure can hide sites doing well and sites that are struggling, so the board needs both to read it fairly.

Continue Reading

Related guides

Guide 9

From Different Views to Shared Direction: A Guide to Facilitation That Ends in Action

How to design a conversation that takes a team or a partnership from different views to shared priorities, named responsibility, and next steps.

Guide 8

Building Evaluation Capacity: A Guide for Teams That Want to Own Their Data

What a nonprofit team should build in-house, what to rely on an evaluator for, and how adults learn to use their own data in ways that last.

Guide 7

Telling Your Impact Story: A Practical Guide to Credible Data Storytelling

How evidence, context, and participants' experiences combine into an account a funder, board, leadership team, or community can trust and act on.

Explore the Insight Library

Related reading in the Insight Library

Insights & Answers 06 / Topic 6C

Supporting Grantees Without Burdening Them

Funders can learn from grantees without exhausting them, and the line between support and monitoring is a design choice the funder makes.

Insights & Answers 06 / Topic 6D

Shared Measurement and Portfolio Learning Across Sites

Measuring the same things across sites is possible without erasing what makes each one different, and the design question is which few things everyone reports and what stays local.

Insights & Answers 04 / Topic 4C

Communicating Results to Boards, Communities, and Funders

Boards, communities, and accreditors each need something different from the same results, and sharing results with a community carries obligations a board briefing doesn't.

Lauren McClain, Ph.D.

Lauren McClain, Ph.D.

Lauren McClain, Ph.D. is a co-founder of Grantibly and a workforce development consultant who develops the workforce through evaluation, training, facilitation, and strategic planning. A sociologist and former faculty member at Savannah State University and Western Kentucky University, she taught more than 50 social research methods courses, mentored more than 21 thesis and dissertation committees, and served on three Institutional Review Boards. Her work helps organizations use evidence to understand what is happening, decide what to do next, and do MORE good.

grantibly.com
Not sure where to start?

Tell us what you're trying to understand

Schedule a consult Meet Lauren and Angelika