Guide 6 · Working with an evaluator

Making the Most of Your Evaluation: From Compliance to Strategic Asset

Most evaluations are planned to meet a funder's requirement. Planned well from the start, the same evaluation can also improve your program, strengthen your partnerships, tell your program's story, and build the evidence for your next proposal.

The short version

An evaluation is more than a compliance task. It becomes a strategic asset when its questions are tied to your program's goals, its findings arrive in time to change something, the people with a stake in the program are involved, and the results are used to make decisions, tell the program's story, and build the evidence for its next proposal.

Lauren McClain, Ph.D.

Lauren McClain, Ph.D.

Co-founder and workforce development consultant, Grantibly

grantibly.com/about/

· 13 min read

What You Need to Know

Key takeaways about making the most of an evaluation

What this guide covers
From compliance to strategic asset

An evaluation can be a compliance task, a learning tool, or a strategic asset for your program

Whether an evaluation ends up as a compliance task, a learning tool, or a strategic asset for your program depends on how it is planned and used, more than on what your funder requires. Most funded projects meet evaluation first as a requirement: the solicitation asks for an evaluation plan, the award terms ask for reports, and the budget includes a line for an evaluator.

Those requirements are real, and a good evaluation meets them. Meeting them is the least an evaluation can do, though, not the most. It is worth asking what comes to mind when you think about evaluation, and how thinking about it makes you feel. If the honest answer is a report and a deadline, that is how the evaluation is being treated, and it can be more than that.

We define evaluation as the systematic process of determining the strengths and opportunities for improvement of a project, policy, program, or product. It involves measurement, assessment, and learning. For a grant-funded program, that means assessing the program's effectiveness and impact in order to guide decision-making and to support both accountability and learning. The learning is where much of the value sits, and it is also the part a compliance-only evaluation leaves out.

The difference shows up in what happens to the evaluation's findings. The same evaluation can be used in three quite different ways:

Compliance-focused evaluation

Meets reporting requirements and produces end-of-year summaries, with minimal use beyond submission.

Learning-focused evaluation

Uses data on an ongoing basis for continuous improvement, so the findings inform real decisions while the program is running.

Strategic evaluation

Strengthens the program, supports its alignment with industry and other partners, and helps build its sustainability and future funding.

Moving from the first kind toward the third doesn't have to mean a bigger evaluation. It means planning, from the start, for how the findings will be used. Used that way, an evaluation supports compliance and accountability, builds engagement and trust among the people who have a stake in the program, informs decisions, and makes the program more effective.

It also demonstrates the program's impact, helps attract funding and resources, and can even serve as advertising for the program. In our words, it is evaluation to do MORE good.

Evaluation may be required by a funder, but compliance should never be its only purpose.

Lauren McClain, Ph.D., Grantibly
Four components

Four components make an evaluation useful: alignment, timing, interest holder engagement, and use of findings

The four components that make an evaluation useful are alignment, timing, interest holder engagement, and use of findings, and each one is a choice made while the evaluation is being planned. None of them depends on the size of the evaluation or the sophistication of its methods. An evaluation can be modest in scope and still have all four.

Interest holders are the people and groups who have a stake in a program and its results. For a college program, that often means faculty, students, administrators, and the employers who will hire graduates. For a community program, it can mean staff, participants, families, and partner organizations.

Funders sometimes ask for these components directly. NSF's Advanced Technological Education (ATE) program is one example of a funder that is specific about evaluation: its solicitation asks proposals to detail clear alignment between the evaluation plan and the project's goals, objectives, and intended outcomes, and to include a timeline that identifies when data will be collected. Your own funder may ask for different things, so read its solicitation for them.

Each of the four components is designed into the evaluation, and designing the evaluation is the evaluator's expertise, done together with the project team. The team brings the program's goals, its calendar, its partners, and the decisions it faces. The evaluator turns those into evaluation questions, methods, and a schedule. How that work unfolds from the first meeting onward is covered in What does the evaluation process look like from kickoff to final report?.

Alignment

Aligning an evaluation with your program means tracing each question from a goal to data to a use

Aligning an evaluation with your program means every evaluation question traces back to a program goal, every data source helps answer a question, and every answer has a planned use. Mapping goals to questions to data, and then to how the findings will be used, is practical work for an evaluator and a project team to do together early in a project.

Here is one example of what that mapping can look like for a program that prepares students for technician careers. It is an illustration of the method, not a description of any particular program.

Each link in that chain does work. The question asks what influences persistence, not only how many students persisted, because a retention rate on its own can show a pattern without explaining it.

Retention data shows whether students are staying, and surveys and focus groups help explain why some stay and others leave. The use is written down before any data is collected, so the team knows which decisions the findings are meant to inform and can plan to make them.

A logic model can support this mapping, because it lays out how the program is expected to work, which helps the evaluator decide what questions to ask and which assumptions to examine. What's a logic model, and does my proposal really need one? explains what one is for.

Deciding what to collect follows from the questions, and What data do I actually need to collect for my grant evaluation? covers that step. What a proposal needs to say about all of this is covered in What should the evaluation section of my grant proposal include?.

Alignment also runs toward what your funder prioritizes. For an NSF ATE project, for example, the evaluation should actively support technician education outcomes; workforce alignment and industry needs; student recruitment, retention, and completion; employer partnerships and feedback; and the scalability and sustainability of programs. Another funder will name its own priorities, usually in its solicitation, and an aligned evaluation speaks to those as well as to the program's own goals.

Common gaps

Five common evaluation gaps keep findings from being used, and each can be avoided when the evaluation is planned

The five common evaluation gaps that keep findings from being used are questions not aligned with goals, data collected but not used, limited input from industry partners, feedback that comes too late to act on, and findings used only for reports. They are missed opportunities more than mistakes, and each is easier to avoid while the evaluation is being designed than to fix once it is underway.

The timing gap is the one that most separates a learning-focused evaluation from a compliance-focused one. Gathering information while the program runs and sharing findings while there is still time to improve it is the formative side of evaluation, and What's the difference between formative and summative evaluation, and why does my funder want both? explains how it fits with the summative evaluation at the end.

Alignment can also slip after the evaluation starts. Programs change once implementation begins: staff change, partners change, and recruitment strategies change. An evaluation plan that doesn't change with the program ends up measuring a program that no longer exists. Our program changed mid-grant. Can we change the evaluation plan too? covers how to keep the two in step and when your funder needs to approve a change.

Interest holders

Engaging industry partners and other interest holders during the program makes the evaluation more useful

Engaging industry partners and other interest holders during the program, rather than only at the end, makes the evaluation more useful because it brings in information the project team can't supply on its own and strengthens the partnerships the program depends on. An employer can tell you whether the skills a program teaches match the work. A student or participant can tell you what got in the way of finishing.

For industry partners, the ways to gather that input during a program are familiar ones:

The project team can then use that input to adjust the curriculum, to check that the outcomes the evaluation measures are the ones employers care about, and to strengthen future proposals.

NSF's ATE program is one example of a funder that builds employers into the project itself: its solicitation says employers must be committed partners with defined project roles and responsibilities, which makes their feedback a natural part of the evaluation. If employers are telling you graduates aren't ready, Employers keep saying our graduates aren't ready. How do we find out what they actually need? looks at how to find out what they need.

Industry partners are not the only interest holders whose input belongs in an evaluation. Faculty and staff know how the program is actually delivered. Students or participants know what it is like to go through it. Community partners may see effects the program team never hears about. One of the most useful questions a team can ask about its evaluation is whose perspective is missing.

Engaging interest holders doesn't stop when the program ends. They should hear the findings and what the team is doing about them, both during and after the program, and the earlier they are involved, the better.

Using findings

Using evaluation findings for decisions starts with asking what the data is telling you to do differently

Using evaluation findings for decisions starts with one question, asked of every finding: what is this data telling us to do differently? Findings don't turn into action on their own. A percentage in a table may be technically accurate and still leave a project team asking, "So what?" The value comes when people talk through what a finding means and decide what to change.

Ask: What is this data telling us to DO differently?

Then go further. What does the finding mean? Why might we be seeing it? Does it confirm what the team expected or challenge an assumption? Is there an important difference between groups? Does it suggest something the project should change?

The decisions an evaluation can inform are practical ones. A team might use its findings to adjust recruitment strategies, improve supports for students or participants, refine the curriculum, or strengthen partnerships. For each finding, the team decides what to keep or enhance, what to change, and what to remove, and who is responsible for each change.

That works best when findings become part of project meetings, planning conversations, implementation decisions, and strategy, instead of sitting in a report until the next one is due. The evaluator's part is to explain what the data shows and what it can't establish, and to help turn the findings into priorities. The project team's part is to bring the context that explains a finding and to make the changes.

Not every finding will be welcome. Sometimes the data shows strong results, sometimes mixed ones, and often both strengths and opportunities for improvement. Finding something early enough to improve it can be one of the most valuable outcomes of evaluation, and What if the evaluation shows my program isn't working? covers what to do when a finding suggests the program isn't working as planned.

Telling the story

Evaluation findings help a program tell its story when they become clear narratives, visual summaries, and reports

Evaluation findings help a program tell its story because they give it evidence to show its impact clearly, highlight the success of its students or participants, and demonstrate its workforce outcomes. A strong evaluation also helps a program strengthen its partnerships, inform practice, make a case for future funding, and gain broader support, which keeps the program relevant.

The evidence has to be turned into something people will read. That means clear narratives, visual summaries, and compelling reports, each shaped for the people who will use it.

Telling the story also means going beyond activity counts. Instead of saying "We served 200 people," a program can explain who took part, what they experienced, what changed, why that change matters, and what happens next. What makes an impact story credible instead of just emotional? looks at how to combine numbers and people's experiences without overstating either, and How do I turn program data into a report my funder actually reads? covers the report itself.

Don't keep it to yourself. Sharing a program's successes, and how it overcame its challenges, can inspire others and extend its impact. There are several good ways to share evaluation findings and program achievements:

Present it

Host workshops for industry and for other departments or colleges, present at conferences, and use what you learned in professional development and to inform teaching.

Publish it

Publish evaluation findings in industry or trade magazines and peer-reviewed journals, or write case studies and white papers.

Summarize it

Create visually appealing one-pagers that the program's key interest holders can read in a few minutes and pass along.

Before sharing findings outside the project, check what your funder's terms and your institution's policies allow, and protect the privacy of the students or participants the data describes. Publishing in a journal deserves an extra step: a plan to publish findings or to use the data in a broader study can change whether your institution's IRB needs to review the evaluation, so raise it with the IRB early. Does my program evaluation need IRB approval? explains why.

The next proposal

An evaluation sustains a program and strengthens its next proposal when it builds evidence from the first year

An evaluation sustains a program and strengthens its next proposal when it builds evidence from the first year: evidence of impact, evidence of continuous improvement, and credible evidence a reviewer can weigh. That evidence is much easier to build as the program runs than to reconstruct when the next solicitation is open.

A team can make that a habit by asking, every year, what information it will need to demonstrate the value of this work later, and planning to collect it now.

Ask yourself: What evidence are we building TODAY for our NEXT proposal?

Sustainability is often a funder's expectation as well as the program's own goal. NSF's ATE solicitation is one example: it describes sustainability as ensuring that products and services have a life beyond ATE funding, and it expects every proposal to communicate a realistic vision and an achievable plan for sustainability. An evaluation that has documented what worked, for whom, and what the team changed along the way gives that plan something solid to stand on, whoever the next funder is.

Continuous improvement is evidence in its own right. A record of what the evaluation found, what the team changed in response, and what happened next shows a reviewer a project that pays attention to its own work. That record is also what helps a team rebuild a plan when a proposal is declined, which My proposal was declined and reviewers flagged the evaluation plan. How do I fix it? covers.

This is also why evaluation is better treated as an investment than as money carved out of a grant because a funder requires it. The question shouldn't simply be how much the project has to spend on evaluation. It should also be what better information could help the project accomplish, both in this grant and in the next one.

The Grantibly perspective

Grantibly does not want to be an evaluator who appears once a year, asks for data, writes a report, and disappears. We see ourselves as a partner, not a vendor. That doesn't mean telling clients what they want to hear. A good evaluation partner should be willing to raise questions, identify concerns, challenge assumptions, and share findings that may be uncomfortable, but we do that in service of the project.

We want clients to call us when they're considering a change, struggling with participation, trying to understand an unexpected result, developing a new strategy, preparing for a funder conversation, or wondering what data would help them make a decision. The relationship should make the evaluation stronger, and make the project stronger.

If an evaluation produces a report at the end of the year but doesn't help anyone think differently, make a decision, improve something, or communicate the value of the work, an important opportunity has been missed. We also want clients to understand their own data, so we explain what we're doing and why and make our findings accessible. The goal isn't to turn every client into an evaluator. It's to help them become stronger users of evidence.

The Bottom Line

Making the most of your evaluation comes down to one change you can make now

Making the most of your evaluation comes down to using it strategically from the very start: defining clear goals and objectives, developing an evaluation plan that follows from them, using findings in decision-making, engaging interest holders both during and after the program, communicating results widely, and planning for ongoing evaluation and continuous improvement so the program can sustain itself and earn future funding. None of that requires doing everything at once.

Start with one change to your evaluation approach. It might be a question you will refine so it ties more closely to a program goal, a data source you will add because it answers something the others can't, or a way you will use findings differently, such as bringing them to your next project meeting or your advisory board. Then act on it.

We want clients to stop seeing evaluation as something they have to do for a grant and start seeing it as something that can help them do their work better.

Sources
  1. National Science Foundation, Advanced Technological Education (ATE) Program Solicitation, NSF 24-584
  2. Centers for Disease Control and Prevention, "Step 6: Act on Findings," CDC Program Evaluation Framework Action Guide (2024)
Frequently Asked Questions

More questions about making the most of an evaluation

Is making the most of an evaluation the same as doing a bigger evaluation?

Making the most of an evaluation isn't the same as doing a bigger one, because more data isn't necessarily better data. Every added survey or measure creates work for participants, project staff, and the evaluator. A focused evaluation with questions tied to the program's goals, findings shared in time to act on them, and a plan for using them produces more useful evidence than an enormous plan that generates data no one has time to use.

When should a project team start planning how its evaluation findings will be used?

A project team should start planning how its evaluation findings will be used while the evaluation is being designed, ideally during proposal development, when the evaluation questions, methods, timeline, and budget are first set. Writing down the intended use of each finding before data collection begins tells the team which decisions the evaluation is meant to inform.

Does this evaluation guide apply only to NSF ATE projects?

This evaluation guide doesn't apply only to NSF ATE projects, although it began as material for ATE project teams and uses ATE as its example. Alignment, timing, interest holder engagement, and use of findings matter for any funded program. What changes from one funder to another is the set of priorities the evaluation should speak to, so read your own solicitation for those.

Who is responsible for using evaluation findings, the project team or the evaluator?

The project team and the evaluator share responsibility for using evaluation findings, in different ways. The evaluator explains what the findings mean, what they can and can't establish, and helps turn them into priorities. The project team brings the context, decides what to keep, change, or remove, and names who is responsible for each change.

Can we share our evaluation findings publicly before the grant ends?

Whether you can share evaluation findings publicly before the grant ends depends on your funder's terms and your institution's policies, so check both first, and protect the privacy of the students or participants the data describes. If you plan to publish the findings in a journal or use the data in a broader study, talk with your institution's IRB early. With Grantibly, your organization keeps full ownership of its primary program data.

What if our evaluation findings show the program isn't working as planned?

If your evaluation findings show the program isn't working as planned, you've found something you can act on, and finding it early enough to improve the program is one of the most valuable things an evaluation can do. A program that needs work isn't a failure or a sign to stop. It is an opportunity to do things better.

Continue Reading

Related guides

Guide 5

How to Choose an External Evaluator: An Honest Guide from Two of Them

Whether you need an external evaluator, how to find and compare one, what the cost depends on, and what the working relationship should be like.

Guide 3

The Complete Guide to Grant Evaluation, From Proposal to Final Report

The grant evaluation lifecycle, stage by stage: designing the plan with the proposal, starting after the award, findings during the grant, hard results, and the final report.

Guide 7

Telling Your Impact Story: A Practical Guide to Credible Data Storytelling

How evidence, context, and participants' experiences combine into an account a funder, board, leadership team, or community can trust and act on.

Explore the Insight Library

Related reading in the Insight Library

Insights & Answers 03 / Topic 3D

What Working with an Evaluator Is Really Like

Working with an evaluator follows a known rhythm, and a good one starts from the records you already keep without judging the state they're in.

Insights & Answers 02 / Topic 2B

Running a Grant Evaluation Across the Award Lifecycle

A grant evaluation runs across the whole award, from the first question asked to the funder's annual report, and the plan can change when the program does.

Insights & Answers 04 / Topic 4B

Turning Data into Reports People Actually Read

A report gets read when it answers the question the reader arrived with, and a credible impact story carries evidence and context alongside the human account.

Lauren McClain, Ph.D.

Lauren McClain, Ph.D.

Lauren McClain, Ph.D. is a co-founder of Grantibly and a workforce development consultant who develops the workforce through evaluation, training, facilitation, and strategic planning. A sociologist and former faculty member at Savannah State University and Western Kentucky University, she taught more than 50 social research methods courses, mentored more than 21 thesis and dissertation committees, and served on three Institutional Review Boards. Her work helps organizations use evidence to understand what is happening, decide what to do next, and do MORE good.

grantibly.com
Not sure where to start?

Tell us what you're trying to understand

Schedule a consult Meet Lauren and Angelika