How do independent consultants, evaluation firms, and university centers differ?
Independent consultants, evaluation firms, and university centers differ mainly in who does the work, how much capacity they have, and how they're connected to institutions. The table below describes general patterns; any individual evaluator may differ from them, so use it to decide what to ask each option.
General patterns, and the question each one raises
| Independent consultant | Evaluation firm | University evaluation center |
|---|
| Who does the work | Usually the person you talk with | A team, which may include senior and less experienced staff | Center staff and researchers, sometimes with graduate students |
| What it can do well | One relationship throughout; direct access to the person doing the work | Continuity if someone is unavailable; capacity for several sites and methods | Research staff and infrastructure; experience with IRB review and data agreements |
| Question to ask | What happens if you become unavailable during the grant? | Who will do the day-to-day work on our project? | Will our funder consider you independent, and how do you fit evaluation around academic schedules? |
Size matters here too. A focused, single-site project may not need a team, and a multi-site evaluation with surveys, interviews, and focus groups at several points each year may be more than one person can carry. Grantibly is a small consulting firm, and both of its founders are former or current university faculty, so we're one of the options this page describes; the table is written to help you choose among all three.
Does a university center at our own institution count as independent?
Whether a university evaluation center at your own institution counts as independent depends on your funder's definition. NSF's Advanced Technological Education (ATE) program, for example, allows the evaluator to be employed by the project's home institution if they work in a separate organizational unit with a different reporting line from the project's home unit. Other funders and programs may set different rules, so read your solicitation or ask your program officer.
Beyond the written rule, consider how the arrangement will look to reviewers, partners, and your institution's leaders. An evaluator from outside your institution can bring more distance from institutional relationships and pressures, while a center inside it may already know your institution's data and processes. If you're weighing your institution's institutional research office rather than an evaluation center, Can our internal institutional researcher do the evaluation to save money? looks at that case.
How does contracting differ between the three?
Contracting can differ between the three because your institution may handle an outside vendor, another university, and a unit of your own institution through different processes. An independent consultant or firm is usually set up as a vendor, and some institutions require evaluators to be selected through a competitive process after the award. A center at another university may involve an agreement between the two institutions. A center at your own institution may follow an internal process instead.
Each of these has its own paperwork and timeline, and some can take weeks. Ask your grants or sponsored programs office how each option would be handled before you choose, and plan for IRB review and any data use agreement whichever kind of evaluator you select. Do I need to name an external evaluator before my grant is funded? covers how procurement rules affect what the proposal can say.
Which kind of evaluator handles uncomfortable results best?
How well an evaluator handles uncomfortable results depends on the individual evaluator, whatever kind of organization they work in, so ask each option directly. Evaluators of every kind are expected to share the limitations of their work honestly and to decline an evaluation if its integrity can't be protected, according to the American Evaluation Association's Guiding Principles.
Useful questions include how they've shared a disappointing result with a project team, what they'd do if they were asked to soften a conclusion, and how they make sure the team hears important results while there's still time to act. At Grantibly, we decline work when the conclusion is decided before the evaluation begins. What questions should I ask an evaluator before hiring them? has more questions to ask.
The Grantibly perspective
Lauren's perspective
We don't want to be an evaluator who appears once a year, asks for data, writes a report, and disappears. We see ourselves as a partner, not a vendor.
We want clients to call us when they're considering a change, struggling with participation, trying to understand an unexpected result, developing a new strategy, preparing for a funder conversation, or wondering what data would help them make a decision. Whichever kind of evaluator you choose, that's the kind of relationship worth asking about.
