What do reviewers look for in an evaluation plan?
What reviewers look for in an evaluation plan depends on the funder, so start with the funder's requirements. For example, the National Science Foundation's Advanced Technological Education (ATE) program requires an evaluation plan that includes:
- alignment with the project's goals, objectives, intended outcomes, activities, and deliverables
- the data sources, instruments, and methods that will be used
- a timeline for data collection, reporting, and communication between the evaluator and project team
- an evaluator who is independent of the project, with funding for the evaluation included in the budget
Other funders and programs may have different requirements. Use the requirements for your specific funding opportunity as your guide.
What should the evaluation questions look like?
Good evaluation questions are specific, answerable, connected to the project's goals and intended outcomes, and focused on information that will actually be useful. They should help you understand both how the project is working and whether it is achieving what it set out to achieve.
For example, an implementation question might ask, "Which parts of the new advising model are students using, and what affects their participation?" An outcome question might ask, "Is the new advising model increasing efficiency in students' path to graduation?"
Avoid questions that are so broad that you won't know how to answer them or so narrow that the answers won't tell you anything useful. Also be careful not to turn evaluation questions into research questions. Evaluation is primarily about understanding and improving the project being funded, while research is typically designed to generate or test knowledge that extends beyond that particular project. Each evaluation question should have a clear connection to what the project is trying to accomplish and to data you can realistically collect during the grant.
How much detail does the evaluation section need about methods and timing?
An evaluation section needs enough detail about methods and timing for a reviewer to follow each evaluation question from start to finish. For each one, say what information will answer it, where that information will come from, and when it will be gathered. A simple table like this one can do most of that work.
| For each question | Describe | Made-up example |
|---|
| The information | What will be measured or asked | Course completion and students' reasons for leaving |
| The source | Who provides it, or where it already exists | Institutional records and interviews with students who left |
| The timing | When it's gathered and reported | Each term, before the next term's planning |
| The use | What the project team will do with it | Adjust advising or course scheduling |
If your solicitation asks for a logic model, connect your evaluation questions to it. To learn more, read What's a logic model, and does my proposal really need one?
What does the evaluation section need to say about the evaluator and the budget?
The evaluation section should name the evaluator, explain why they're qualified for this project, and show in the budget what the evaluation will cost. If your funder expects an evaluator who's independent of the project, explain how the evaluator meets that requirement.
The evaluator's qualifications should fit the work you're proposing. That can include experience with the type of program being evaluated, the population or setting, and the methods the evaluation will require.
The evaluation plan and budget also need to make sense together. A plan with surveys, interviews, focus groups, administrative data analysis, site visits, and multiple reports cannot reasonably be supported by a very small evaluation budget. An unrealistically low budget can signal that the evaluation has not been adequately planned or that its scope and cost are not well understood. At the same time, don't add activities simply to make an evaluation look rigorous. Every activity takes time from participants, project staff, and the evaluator, and each should contribute information the project actually needs. On the other end, relying on too little data—such as a single survey when the evaluation needs to answer several different questions—may not provide enough evidence to understand how the project is working or what it is accomplishing.
The scope, methods, workload, and budget should align and be sufficient to answer the evaluation questions well.
Do I need to name an external evaluator before my grant is funded? and How much of my grant budget should go to evaluation? cover both questions in more detail.
The Grantibly perspective
Lauren's perspective
As sociologists and former and current faculty members, we understand both sides of evaluation. We understand research methods, data, analysis, and evidence. We also understand what it's like to be the faculty member or project director responsible for doing the work while managing other parts of the job, working with students, participants, and partners, and meeting grant requirements.
Evaluation may be required by a funder, but we don't think of evaluation as a compliance exercise alone. Done well, evaluation can help a program get better at achieving its goals, understand the difference it's making for students or participants, and tell that story to funders, partners, and others who need to know about it.
