Guide 8 · Long read

Building Evaluation Capacity: A Guide for Teams That Want to Own Their Data

A team's ability to understand and use its own evidence can outlast any workshop or outside evaluation when it is planned for. This guide covers what to keep in-house, what to rely on an evaluator for, and how people actually learn this work.

The short version

Evaluation capacity is a team's ability to understand its own data and use it in the work, continuously, and it rarely survives a training or an outside evaluation by accident. It lasts when the everyday use of evidence is built into the team's routines and the specialized design work stays with an evaluator who explains it as the work happens. It also depends on learning that is built around the team's own questions and the conditions it actually works in.

Angelika Gulbis, Ph.D.

Angelika Gulbis, Ph.D.

Co-founder and workforce development consultant, Grantibly

grantibly.com/about/

· 13 min read

What You Need to Know

Key takeaways about building evaluation capacity

What this guide covers
What Capacity Means

Evaluation capacity means a team can understand and use its own evidence

Evaluation capacity is a team's ability to understand the information it has and use it in its work on an ongoing basis. In practice, that means being able to ask a useful question, recognize what evidence would answer it, read what comes back, and decide what to change.

For a nonprofit, that might look like program staff who know why each item on an intake form is collected, and a leadership team that reviews participation records before it plans the next program cycle.

Evaluation capacity in this sense is about using evidence, and it has a clear boundary. Your staff don't need to become evaluators to use data well, and a team that takes on the specialized parts of evaluation by itself can end up with more work and results it can't use.

Designing an evaluation, deciding what evidence would support a claim about cause, and providing an independent view are an evaluator's work. What your team builds is the ability to work with that evidence, ask better questions of it, and act on what it shows.

At Grantibly, we describe that ability through the questions a team becomes better at asking over time:

Questions a team with evaluation capacity learns to ask

  • What evidence would tell us whether this is working?
  • How will we know if this change helped?
  • Whose perspective are we missing?
  • Are we collecting data we actually need?
  • What does this finding suggest we should do differently?
  • What information will we need to demonstrate the value of this work later?

None of those questions calls for statistical training to ask. Each one calls for a team that knows its program, has a reason to look at its information, and has a habit of doing so. That is why evaluation capacity can be built deliberately, and also why it so often fades when nobody plans for it.

If your staff are uneasy about data to begin with, How do I get my staff comfortable collecting and using data? starts from the reasons behind that discomfort.

Why It Fades

Why a team's data skills fade after a training or an outside evaluation ends

A team's data skills often fade after a training or an outside evaluation ends for reasons that have little to do with the people involved. Most outside help is designed to deliver something, such as a session, a report, or a set of findings, and not to leave the team able to continue on its own.

When capability isn't one of the things the work is designed to produce, whether any of it remains depends on circumstances nobody planned, such as one staff member staying in the same role.

Four patterns account for much of the fading, and each one is familiar to anyone who has sat through a training or received an evaluation report:

Each of these patterns comes from how the work was set up, so each one can be addressed before the next engagement or training begins. The sections that follow describe what to plan for: which parts of the work to build inside your team, which parts to rely on an evaluator for, how people learn the skills involved, and what conditions allow those skills to last.

We want clients to understand their own data.

Angelika Gulbis, Ph.D., Grantibly
Build In-House

What a nonprofit team should build and keep in-house

The part of evaluation a nonprofit team should build and keep in-house is the everyday use of evidence. That work depends on knowing the program, its participants, and its daily realities, which your staff know better than any outside evaluator. This work happens continuously, often at the meetings where staff already plan, and it makes any future evaluation easier, since the records and the habits are already in place.

Naming what you need to learn

Decide what you need to understand, improve, or explain, and what you might change depending on the answer.

Knowing why each item is collected

Everyone who records information understands what question it helps answer, which is what makes consistent records possible.

Using what you already have

Attendance and participation records, intake forms, and existing feedback get reviewed before anyone creates something new.

Reviewing results together

The team looks at what the information shows, discusses what it means, agrees on what to change, and tells staff what changed as a result.

Two other responsibilities belong with the team as well. The first is asking participants about their experience in ways that respect them: explaining what you are asking and why, keeping what they tell you secure, and reporting results so no one can be identified.

The second is deciding, as an organization, what to stop collecting. Plans for collecting information tend to grow one survey or one measure at a time, and every addition creates work for participants and staff. A team that regularly asks what each item is for, who uses it, and whether a funder requires it keeps that growth in check. My team sees data collection as extra work. How do I change that? walks through that review.

Done carefully and described honestly, with its limits stated, this in-house work is genuine evaluation work and worth doing, even for a small team with no evaluator. Can we evaluate our own program without hiring anyone? describes how far a small team can take it on its own and where its limits are.

Rely on an Evaluator

What a team should rely on an evaluator for, and why evaluation design belongs there

A team should rely on an evaluator for the design of an evaluation and for any part of the work whose credibility depends on specialized methods or on independence. Designing an evaluation means choosing the questions it will answer, deciding what evidence would answer them, planning when and from whom to gather that evidence, and writing survey or interview questions that produce usable answers. Those choices shape everything that follows, and they are where evaluation expertise makes the biggest difference.

Question wording shows why. Pew Research Center, which designs surveys professionally, notes that even small wording differences can substantially affect the answers people provide, and it treats testing questions before a survey goes out as an essential step.

Surveys a team writes for itself often run into those problems, or into a more basic one: nobody agreed beforehand on what the survey should answer. We ran our own survey and the results were useless. What went wrong? looks at how that happens.

Claims about cause

A positive result after a program does not by itself show what produced it, and supporting that kind of claim takes a design built for it.

Independence for an outside audience

When a funder or another outside audience expects an independent evaluation, a review a team conducts of its own program cannot supply it.

IRB and ethics review

Whether an evaluation needs IRB review, and how to follow IRB and federal guidelines if it does, is a determination to make with someone who knows those rules.

Complex designs

Comparisons between groups, across sites, or over several years produce results that are hard to interpret correctly without specialized methods.

Relying on an evaluator for this work is most useful as a shared process. An evaluator who builds capacity explains the design as it is made, involves staff in interpreting what comes back, and makes the reasoning visible, so the team understands why the evaluation asks what it asks and can use what it finds.

The CDC's 2024 program evaluation framework makes a related point: engaging the people involved in an evaluation can improve their own evaluation capacity, and taking part in the reasoning of evaluation can have lasting effects. Relying on an evaluator and building your team's capacity can happen in the same engagement. What parts of evaluation should we never DIY? sets out where the lines fall in more detail.

How People Learn

How adults actually learn to use data in their work

Adults learn to use data in their work by practicing on a question that matters to them, using their own information, while someone who knows the skill is there to give feedback. As former and current faculty members, Lauren and I pay attention to how people learn, how to design engaging learning and feedback experiences, and the real conditions that affect whether people can participate.

We design learning that helps people use what they learn, and we don't simply move through a slide deck.

A presentation on survey design or spreadsheet formulas can be easy to follow and still leave staff unsure where to begin the next morning, because the skill was never connected to anything they do. Learning designed for use starts before the session, by asking staff what they find most difficult about working with data and which upcoming decisions the data could inform. The session is then built around those answers, in five parts:

Timing belongs to the design as well. We would never schedule a focus group during finals week, and the same thinking applies to a training scheduled during a team's busiest season, when it competes with the work it is meant to improve. What makes a workshop worth pulling my team off their work for? offers questions to ask before you commit your team's time to a particular workshop.

What Makes It Last

A new data skill lasts when the organization makes room to use it

A new data skill lasts when the organization makes room to use it, because a skill with no time, reason, or support behind it can fade before it becomes part of the work, even when it was well taught.

Staff who learned to summarize participant feedback won't keep doing it if no one asks for the summary, if their workload leaves no room to try it, or if the system they use makes the new approach harder than the old one.

We're interested in both individual and organizational responsibility. People can develop new skills and better ways of working, but organizations, leaders, and supervisors also have a responsibility to create the conditions that support them. When leaders ask what the information shows before they make a decision, and give staff the time to find out, using data becomes an expected part of the job instead of an extra task on top of it.

Sometimes creating those conditions means redistributing workloads or changing a policy before asking staff to learn anything new. For a team that is already stretched thin, training can be the wrong first step: if the obstacle is time, tools that get in the way, or information collected for no clear purpose, a training adds another demand without addressing it.

Before you schedule a training, find out what is making data difficult for your staff. It may be an unfamiliar skill, too little time, forms or systems that get in the way, or uncertainty about why the information matters. Training solves the first of these. The others call for changes to the work itself.

Staff wellbeing belongs in this decision too. In our view, employee wellbeing should be valued in its own right, not simply as a means of improving retention or productivity, and capacity-building worth your staff's time should make their responsibilities easier to carry out well. Is data training worth it when my staff is already stretched thin? helps you decide when training makes sense for a stretched team and when something else should come first.

Spread It Across the Team

Evaluation capacity holds up when it is spread across a team instead of held by one person

Evaluation capacity holds up better when it is spread across a team, because an organization whose data skill sits with one person depends on that person staying, and on their having the time and energy to do the work. When a sole data champion leaves, the knowledge of how the spreadsheets work and why information is recorded the way it is often leaves with them, and their replacement starts over.

The CDC's evaluation framework describes evaluation capacity at both the individual and the organizational level, and the distinction is useful here. One skilled person is individual capacity. A team with shared routines, documented systems, and leaders who use the information is organizational capacity, and it is the kind that survives turnover. Sorting the work by who needs which skills makes the difference practical:

Everyone on the team

Shared understanding

Why each item is collected, how to record it consistently, and how to read a summary and ask what it suggests, including for leaders.

One or two trained people

Deeper day-to-day skills

Routine feedback forms, cleaning and analyzing records, regular summaries, and managing the database or spreadsheets, with a backup who can step in.

An evaluator, working with your team

Evaluation design

Designing the evaluation, including its survey and interview questions, and determining what caused a change.

Supporting a data champion is part of supporting the team. Give the role protected time, train a second person alongside them, document how the key files and reports work, and ask the champion to share one result at a regular team meeting so everyone practices reading the information. The people others rely on need support themselves if they're going to provide it sustainably. Should we train one data champion or the whole team? sorts these responsibilities in more detail.

Plan for It

How to plan evaluation capacity into a training or an engagement from the start

Planning evaluation capacity into an engagement or a training from the start means naming, before the work begins, what your team should be able to do on its own when the work ends, and then asking how the work will get it there.

Not every engagement needs to pursue every outcome, and a focused engagement can be valuable in its own right, so capability may be the main goal or one part of a larger one. Either way, it is more likely to happen when someone has said it out loud at the beginning.

A few questions make that expectation concrete, whether you are planning a training, scoping an evaluation with an evaluator, or talking with a funder about support:

The answers also point to a proportionate starting point. For a small or underfunded organization, that may be a single workshop or a free or low-cost resource, and Can a small nonprofit afford real evaluation help? describes those options. For a team that wants internal capability, a longer engagement can combine instruction tailored to the team, applied exercises, reusable tools, and a plan for putting the skills to work.

A funder that wants its grantees to build this capacity can sponsor a workshop series or a learning cohort, with the purpose, participants' choice, and anything shared with the funder made clear from the start. Should we sponsor evaluation help for our grantees? covers that decision from the funder's side.

The Grantibly perspective

Angelika's perspective

As sociologists and former and current faculty members, Lauren and I understand rigorous research methods, data collection, and analysis. We also understand what it's like to be the person responsible for actually doing the work: managing staff, serving a community, meeting grant requirements, working with partners, and trying to keep a program moving. Evaluation should help people do that work better, not simply give them another requirement to manage.

We're educators as much as consultants. We don't just want to deliver findings. We want people to understand what those findings mean and how they can use them, which is why we explain what we're doing and why, make findings accessible, and help teams develop stronger evaluation thinking over time. Data becomes valuable when people can understand it and use it.

The goal is not to turn every client into an evaluator. The goal is to help them become stronger users of evidence. We don't want clients dependent on us to make sense of their work.

At Grantibly, we help clients build knowledge, skills, systems, and practical capability that remain useful after an engagement ends.

The Bottom Line

Evaluation capacity lasts when a team plans for it and practices it

Building evaluation capacity comes down to deciding, early and on purpose, what your team should be able to do with its own evidence, and then shaping each training, engagement, and routine around that goal. Keep the everyday use of evidence inside the team. Rely on an evaluator for design, claims about cause, and independence, and expect that evaluator to explain the work as it happens. Build learning around your own questions and information, give people the time and support to use it, and spread the skills so they don't leave with one person.

None of this asks your staff to become evaluators. It asks your organization to treat its own evidence as part of the work, reviewed where decisions are made and used to change what needs changing.

Lasting capability is a meaningful form of value.

Sources
  1. CDC Program Evaluation Framework, 2024 (MMWR Recommendations and Reports, Centers for Disease Control and Prevention)
  2. Writing Survey Questions (Pew Research Center)
Frequently Asked Questions

More questions about building evaluation capacity

Can a small nonprofit build evaluation capacity without an evaluator?

A small nonprofit can build a good deal of evaluation capacity without an evaluator, because the everyday use of evidence, such as naming what you need to learn, using records you already keep, asking participants about their experience, and reviewing results together, depends mostly on knowing your own program. Where the work calls for an evaluation design, a claim about cause, or independence, involve an evaluator. Help with design can be much smaller than a full evaluation.

How long does it take a team to build evaluation capacity?

How long it takes a team to build evaluation capacity depends on where the team starts and how often it uses what it learns, so progress shows up in habits more than in hours of training. A team that reviews its own information at regular meetings and acts on it builds capability with each cycle. A team that attends one training and never uses the skill may lose it before it is needed.

What is the difference between evaluation capacity building and data training?

The difference between evaluation capacity building and data training is scope: a data training teaches a specific skill, while evaluation capacity building helps a team understand and use its own evidence over time, through skills, routines, systems, shared understanding, and conditions that let people use what they learn. A good training can be one part of building capacity, but a single session is rarely enough to change how an organization uses information.

Should capacity building be part of our grant evaluation even if the funder doesn't require it?

Capacity building can be worth including in your grant evaluation even when the funder doesn't require it, because a team that understands its own evaluation can use what it shows during the grant and after it ends. Raise it with your evaluator when you agree on scope, so the evaluator can plan to explain the design, involve staff in interpreting results, and leave tools your team keeps, scaled to the project's goals and your team's time.

Can a funder pay for evaluation capacity building for its grantees?

A funder can pay for evaluation capacity building for its grantees, and some funders sponsor workshops or learning cohorts for the organizations they fund. When a funder sponsors that work, the purpose, participants' choice, confidentiality expectations, and any information shared with the funder should be clear from the start, so the learning supports grantees instead of monitoring them.

How does Grantibly help a team build evaluation capacity?

Grantibly helps a team build evaluation capacity through skill-building workshops and the Team Insight Capability Lab, which builds a team's practical ability to gather, interpret, communicate, and use evidence. The work combines instruction tailored to your team, applied exercises on your own information, reusable tools, and a plan for putting the skills to work, and it starts with what your team needs to understand and do.

Continue Reading

Related guides

Guide 7

Telling Your Impact Story: A Practical Guide to Credible Data Storytelling

How evidence, context, and participants' experiences combine into an account a funder, board, leadership team, or community can trust and act on.

Guide 5

How to Choose an External Evaluator: An Honest Guide from Two of Them

Whether you need an external evaluator, how to find and compare one, what the cost depends on, and what the working relationship should be like.

Guide 10

The Funder's Guide to Portfolio Learning Without Grantee Burden

How a funder can understand what its portfolio is accomplishing without exhausting the organizations in it: shared measures, sponsored help, and a clear line between support and monitoring.

Explore the Insight Library

Related reading in the Insight Library

Insights & Answers 03 / Topic 3A

Deciding Between an External Evaluator and In-House Evaluation

The choice between an outside evaluator and doing it internally turns on independence, capacity, and what the funder requires, and sometimes a full evaluation is more than you need.

Insights & Answers 04 / Topic 4A

Making Sense of the Data You Already Have

Most organizations already hold enough to show whether a program is working, and the first task is telling outputs and outcomes apart.

Insights & Answers 06 / Topic 6C

Supporting Grantees Without Burdening Them

Funders can learn from grantees without exhausting them, and the line between support and monitoring is a design choice the funder makes.

Angelika Gulbis, Ph.D.

Angelika Gulbis, Ph.D.

Angelika Gulbis, Ph.D. is a co-founder of Grantibly and a workforce development consultant who develops the workforce through evaluation, training, facilitation, and strategic planning. A sociologist and faculty member at Madison Area Technical College, her areas include wellbeing, emotional intelligence, leadership development, relationship dynamics and organizational culture. Her work helps organizations understand how programs create change through the relationships and systems they build, and use that understanding to do MORE good.

grantibly.com
Not sure where to start?

Tell us what you're trying to understand

Schedule a consult Meet Lauren and Angelika