What does formative evaluation look like during a grant's funding period?
During a grant's funding period, formative evaluation looks like gathering information while the program runs and sharing actionable findings while there's still time to make changes to improve the program. The information can come from participation records, surveys, interviews, focus groups, or meetings with the team or other stakeholders.
Much of it is about how the program is being carried out. Was it implemented as intended? Who is taking part, and who isn't? How much of the program are students or participants receiving? What barriers have come up, and are some parts working better than others? For example, attendance records might show that fewer participants come to one part of the program, and a few interviews might help explain why.
At Grantibly, formative findings reach the project team through regular meetings and shorter summaries between formal reports, so the team isn't waiting for the annual report to hear them.
When does summative evaluation happen?
Summative evaluation happens mostly at the end of a year or funding period, and it focuses more on outcomes or impact than on implementation and process. It asks what the project accomplished, what changed for the students or participants it served, and what the team learned.
NSF's Advanced Technological Education program is one example of how a funder describes this. Its solicitation says the evaluator's primary role is to inform the project team's decisions about changes to the project as it's carried out, which it calls formative evaluation, and that at the end of the project the evaluator should provide a summative evaluation identifying key accomplishments, their impacts, and lessons learned. Your own funder may describe it differently, so check your solicitation.
Summative results are hard to interpret without the formative record. If outcomes fall short, that doesn't automatically mean the idea behind the program was ineffective. It may not have been fully implemented, or participants may not have received enough of it. Positive outcomes don't automatically show what produced them, either. What the evaluation contributes to each year's report is covered in What does my funder's annual report actually need from the evaluation?
How does the project team use formative findings?
The project team uses formative findings to adjust the program while it's running: what to keep or enhance, what to change, and what to remove so the program can do the most good. That works best when findings become part of project meetings and planning conversations, where the team can talk through what a finding means, why it might be happening, and what to try next.
When the team does change something, the evaluation records what changed and why, so the summative evaluation can account for it later. Our program changed mid-grant. Can we change the evaluation plan too? covers how that works with your funder. If a formative finding suggests part of the program isn't working, that's a separate question, and What if the evaluation shows my program isn't working? takes it up.
The Grantibly perspective
Lauren's perspective
We don't think evaluation should operate on a separate track from the program. The cycle we look for runs from the program to the data, from the data to learning, from learning to action, and from action back to a better program, and it should happen throughout the life of a project. That's why we share useful findings while there's still time to act on them.
It's also why we pay attention to how a program operates, and not only to its outcomes. Projects are often eager to know whether their program "worked." That's an important question, but it's difficult to interpret outcomes without understanding implementation. Understanding how a program operates is part of understanding its results.
