A Look in the Mirror

Many people would consider Idealware an expert on data maturity. Do our own practices measure up?

In our 2017 research on Data Maturity in nonprofit organizations, we studied the steps most organizations go through as they become more data-informed. We have also taught thousands of people about data in our workshops and online courses, such as Data for Beginners. Those courses cover the tools to use as well as the best practices for collecting, analyzing, and sharing data.

So we know a thing or two about being data-informed. But in spite of our knowledge, I suspected that if we turned the mirror on ourselves, we would reveal some flaws. We’re a small nonprofit. We face the same challenges that many of our trainees and readers do in applying this knowledge–resource constraints, imperfect data management systems, business processes that aren’t always consistent. Here’s our story.

Using a Theory of Change to Ask the Right Questions

Often people are eager to jump right into collecting data. But it’s important to first understand what kind of data is useful to your organization. Idealware held a board retreat two years ago to develop our Theory of Change. In other words, we came to an agreement on what we are trying to accomplish, how, and for whom.

Idealware's theory of change including our mission, whom we serve, and what they need.

We also set some strategic priorities for the organization, which included audience growth, financial sustainability, and so forth.

What To Measure?

With the theory of change as our compass, and the strategic priorities as our map, we chose a few metrics that would help us understand what progress we were making. It was tempting to list 100 or more metrics that would give us a very precise indication of how we were doing. Instead, we pushed ourselves to winnow that down to just a handful. That handful of metrics had to be easy to track, easy to understand, and provide a snapshot of the organization’s health. If we hadn’t insisted on keeping the number of metrics small, I think we would have put all of our energy into collecting and reporting on the data. Then we would have nothing left over to ACT on the results.

Here are three of the key metrics.

  • In order to find out whether we are reaching the people our theory of change says that we set out to serve, we want to measure budget size and type of organization.
  • In order to find out whether our training program results in people gaining knowledge and confidence, we want to measure percent improvement on a knowledge scale, from before and after training.
  • In order to find out whether we are delivering on our promise of impartial, research-based, accessible resources, we want to measure audience opinions.

What’s Next?

Once we had decided on what to measure, we were ready to gather some data. In Part Two of this series I will explain how we hunted that down. Meanwhile, if you would like to learn more about getting started with data, consider enrolling in the next round of Nonprofit Data for Beginners course which starts on February 8.