What tools do we need to gather information?
The tools you need to gather information depend on what you're trying to learn and who you need to hear from, and a small program can often start with the records and software it already has. Common options include:
- Records you already keep. Attendance, participation, applications, intake forms, and case notes may already answer part of your question, and using them adds no work for participants.
- Online forms and survey tools. These are useful for short feedback forms and surveys, and many offer a free version. They work best when participants have reliable internet access and are comfortable answering online.
- Paper forms. Paper still has a place, particularly for participants who lack reliable internet access or are more comfortable writing their answers by hand. If you use paper, decide in advance who will enter the answers and when, so the forms don't sit in a folder until the end of the year.
- Interviews and small group conversations. A shared document and a consistent set of questions are often all you need, and conversations can help explain why a pattern you see in your records is occurring.
Whatever you use, ask whose perspective you might be missing. If the only people who answer are the ones who come most often, your results describe them and not everyone your program serves.
How do we organize and analyze what we collect?
You can organize and analyze what you collect in a spreadsheet for many small programs, as long as you set it up before the information starts arriving. Use one row for each person or each session, keep categories consistent (always the same label for the same site or activity), and store names and contact details separately from answers when you can, so fewer people see identifying information.
Simple analysis goes a long way: counts, percentages, and comparisons between groups, such as sites, sessions, or first-time and returning participants, can show differences that a single overall number hides. For open-ended answers, read through them, group similar responses into themes, and keep a few representative comments, with permission and without identifying details, to help explain what the numbers show. The charts in your spreadsheet software and a one-page summary are often enough to share results with your team or board.
How do we choose between tools?
You can choose between tools by asking a few questions that stay the same even as the products change:
- Does it answer our question? A tool should fit the information you need, and a feature you have no use for adds nothing to your evaluation.
- Do we already have something that works? Software your organization already uses may cover the need without additional cost, training, or another account for staff to manage.
- Is it easy for the people answering and the staff using it? A tool that participants find confusing will reduce the number of people who respond, and one that staff find tedious is unlikely to be used consistently.
- Where is the information stored, and who can see it? Check the tool's privacy and storage terms, limit access to the people who need it, and collect only what you'll use.
- Can we get our information out? Make sure you can export your data in a common format if you change tools later.
- What happens as we grow? Check what the free version includes and what changes when you collect more responses or add users.
When you ask participants for information, explain what you're asking, why, and how it will be used, and report results so no one can be identified. How do we evaluate students or participants ethically? covers those obligations in more detail.
Can a free tool show our program's impact?
A free tool can help you gather and organize evidence about your program's impact, but no tool decides what that evidence means. Whether your results show impact depends on what you asked, who answered, when you asked, and how the program was carried out, and those are design decisions made before any tool is opened. A survey tool can report that most participants rated a workshop highly; it can't tell you whether the workshop changed what they do afterward, or whether something else explains the change.
That's where an evaluator's design work matters, even for a small program: choosing the questions, deciding what evidence would answer them, and planning when to gather it. What parts of evaluation should we never DIY? explains where those lines fall, and Are we collecting the right data to show that our program is working? looks at whether the information you have can answer the question your funder is asking.
The Grantibly perspective
Lauren's perspective
Once we know what a project needs to learn, we can determine what evidence would answer that question and the best way to collect it. Sometimes that is a survey. Sometimes it is an interview, focus group, administrative data, participation records, observations, institutional data, or a combination of methods.
Good evaluation design starts with what you need to know and why you need to know it. Data don't make decisions. People do. Data become valuable when people can understand and use them.
