Can our internal institutional researcher do the evaluation to save money?

What do institutional research offices do well?

Institutional research offices do well at providing the data and analysis that help an institution make decisions, which is the work they're built for. The Association for Institutional Research lists the core functions. IR identifies the institution's information needs and collects, analyzes, and reports data. It supports planning and program review, especially for accreditation, and it looks after the security and privacy of institutional data. It also supplies data for formative and summative evaluation.

For a grant-funded project, that means the IR office can be a valuable source of the records an evaluation often relies on, such as enrollment, course completion, retention, and graduation. The office also knows how those figures are defined and reported at your institution, which helps keep a project's numbers consistent with what the institution reports elsewhere.

Does our institutional research office count as independent?

Whether your institutional research office counts as independent depends on your funder's definition, so start with the solicitation. NSF's Advanced Technological Education (ATE) program, for example, allows the evaluator to be employed by the project's home institution. They must work in a separate organizational unit with a different reporting line from the project's home unit. Other funders and programs may set different rules, or say nothing specific.

Meeting the written requirement is one question; how the arrangement will be perceived is another. If the IR office and the project ultimately answer to the same leaders, ask your program officer whether that meets the funder's expectations. Also consider how reviewers, partners, and your own institution's leaders will read results produced by an office inside the institution that runs the program.

Where does using institutional research for the evaluation run into trouble?

Using institutional research for the evaluation most often runs into trouble over time and the type of work. An IR office serves the whole institution, including required reporting with fixed deadlines, so a single grant's evaluation competes with that work throughout the year. Evaluation activities also have their own timing, such as data collection planned around the academic calendar and reports due to the funder each year.

The work itself differs too. Institutional reporting mostly draws on records the institution already keeps. A grant evaluation usually also needs evaluation questions written for the project and surveys, interviews, or focus groups with students or participants. It needs interpretation that connects the results to how the program was carried out. Some IR offices have that experience and capacity, and some don't, so ask directly.

Can our institutional research office and an external evaluator work together?

Your institutional research office and an external evaluator can work together, and dividing the work this way can make the evaluation stronger. In one way of dividing it, the IR office provides the institutional data the evaluation needs. The external evaluator designs the evaluation, gathers any new information, interprets the results, and reports on them.

The arrangement works best when it's settled before the grant begins. Agree on which records the evaluation needs, how often they'll be shared, and how each measure is defined. Find out whether a data use agreement is required for the evaluator to receive student data, and who at your institution issues it. We've received notice that our project was funded. When do I need to get my evaluator involved? covers the steps your institution may need to complete before the evaluation can start.