How to Choose an External Evaluator: An Honest Guide from Two of Them

Choosing an external evaluator well starts with what your project needs to learn

Choosing an external evaluator well starts with knowing what your project needs to learn and what decisions the evaluation will support, because those answers determine whether you need an evaluator at all, what kind of evaluator fits, and how much work the evaluation should involve.

Most advice on hiring an evaluator begins with where to find one. That order skips the questions that decide whether a search is worth doing and what you should be looking for once it starts.

This guide is written by evaluators, and it follows the order the decision actually takes. Each step below has its own section, and each section links to pages in the Insight Library that answer one part of the question in more detail. Where a focused piece of work would serve your project better than a full evaluation, the guide says so.

Choosing an evaluator is also choosing who will design your evaluation. Deciding on the evaluation questions, the methods, the timeline, and the budget is part of an evaluator's expertise, so the person you choose shapes what your project will learn over the life of the grant, not only how its reports read. That is why the comparison in step three weighs how an evaluator thinks more heavily than how their credentials look.

• 1
  Decide whether you need one

  Your funder's requirements, the stakes of the decisions the results will inform, and your team's expertise and time settle this first.

• 2
  Find candidates

  Start with colleagues, grants offices, and principal investigators who have worked with an evaluator, then widen the search.

• 3
  Compare how they think

  Listen for whether each evaluator asks what your project needs to learn before proposing a method.

• 4
  Understand the cost

  The cost follows the scope of work, so compare quotes by the work each one describes.

• 5
  Choose the working relationship

  You will work with this evaluator for the life of the grant, anywhere from one to six years.

To find an external evaluator, start with people who have already worked with one

The best way to find an external evaluator is usually to ask people who have already worked with one: colleagues who have managed similar grants, your grants or sponsored programs office, and principal investigators on funded projects like yours. They can tell you who evaluated their work and, more usefully, whether that person was helpful to the project team over several years, which no website can tell you.

Professional directories can widen the search. The American Evaluation Association's Find an Evaluator listing, for example, identifies association members available for evaluation consulting. The association states that it does not endorse any consultant or firm, that the listing is not its full member directory, and that an area of practice listed there does not indicate certification or expertise. A directory gives you names, and deciding whether a person fits your project is still your decision.

Start early.

An evaluator brought in a few days before a proposal deadline may only have time to review a plan someone else wrote, while one brought in earlier can help design the evaluation questions, methods, timeline, and budget, and can often write the evaluation section. Whether you can formally name that person in the proposal depends on your funder's rules and your institution's rules for selecting an evaluator.

Involving an evaluator during proposal development is not the same as formally naming or contracting with one. Do I need to name an external evaluator before my grant is funded? covers those rules, and What can an evaluator actually do for me before I submit the proposal? describes the work in that window.

The evaluators you find will come in different kinds. None is best for every project, and the differences within each kind can be as large as the differences between them. Grantibly is a consulting firm, so it is one of the kinds described here, and the question on its card applies to it too.

Independent consultant, evaluation firm, or a university center: which kind of evaluator do we want? compares the three in more depth, and How do I find an external evaluator for my federal grant? covers the search itself.

What an external evaluator costs follows the scope of work, so compare quotes by the work each describes

What an external evaluator costs depends on the scope of work, meaning the amount of work the evaluation plan asks for, so no single price applies to every project and a figure quoted for one project tells you little about another.

The factors that move the investment are the number of sites, the number of participants, how often data is collected, whether the data comes from focus groups and interviews or from surveys and institutional records, how often there are reports, meetings, or presentations, and travel.

The length of the engagement and the condition of your existing data matter too.

Check your solicitation first, and if it says anything about evaluation costs, follow it. Under-budgeting is the more common mistake, especially when no evaluator helped plan the evaluation, and reviewers can tell when a plan promises more than its budget can pay for. A budget much larger than the planned work is less common and just as worth questioning. How much of my grant budget should go to evaluation? covers the proposal budget line.

When quotes from different firms vary widely, they usually describe different work. Firms make different assumptions about how much data will be collected and how often, how many sites and participants are involved, and what condition your data is in. They also assign people with different levels of experience and leave different things, such as data cleaning, IRB preparation, or travel, outside the scope.

To compare fairly, give every firm the same description of the project, ask each one to state what its quote covers, and then compare what each would do for the price. A bigger plan isn't automatically a stronger one.

Be honest about your data before you sign an agreement. Cleaning data takes time, and if an evaluator budgeted to start with a clean dataset and learns otherwise after the agreement is signed, the budget will need to change. What does an external evaluator actually cost? and Why do evaluation quotes vary so much between firms? go further into both.

Choosing an external evaluator comes down to need, how the evaluator thinks, and trust

Choosing an external evaluator comes down to three questions: whether your project needs an external evaluation and how much of one, whether the evaluator starts with what your project needs to learn, and whether you can trust that person to tell you what the evidence shows for the years the grant runs. Price and credentials matter, but they follow from those three questions rather than replacing them.

The order matters as well. Settle the need before the search, compare people by how they think about your project rather than by credentials alone, and read every quote as a description of work. An evaluator worth hiring will help you do each of those things, including telling you when a smaller piece of work would serve your project better.

Choose the evaluator who asks what your project needs to learn before proposing how to measure it, and who will tell you honestly what the evidence shows.