Beyond Spreadsheets: Give reporting its due in software planning

Database Designs isn’t just about databases but I still spend a lot of time thinking and strategizing about them. Recently, I have noticed that, with some exaggeration, you could divide up much of the population of database managers into those trying to get data out of spreadsheets and those trying to get data back into them. From dust to dust, from spreadsheets in to spreadsheets out, data collection seems like burdensome toil for many. Maybe those flying closest to the sun, with large budgets and staff, truly escape, but most of us still struggle.

On the one side, no matter how powerful one’s database or CRM, administrators find themselves regularly battling users who keep their real data separately in spreadsheets. Not the official data, yet what counts day to day. At a 2009 Nonprofit Technology Conference session on tech planning, a speaker commented that a good technique is to just walk around and see what users are actually using at their desk, regardless of the organization’s prioritized software systems. Yup, that struck a chord.

On the other side, in evaluating systems, its easy to focus on the processing workflow, data collection fields, interactive usability and get to “reporting” last. Under-budgeting for reporting and data exchange is an easy trap to fall into.

Absence enough attention to these things, your staff's data world may remain spreadsheet driven. Here’s some thoughts.

In software planning and selection, tie every feature discussion back to reporting. Reporting today is not just neat formatted lists and labels. It’s also spreadsheets, mailmerge, email list sync, mobile and beyond. One of the reasons I have continued to have affection for Microsoft Access is its powerful reporting system, to which its adherents then layer on additional utility over the years. If you have Office, you have a pretty useful tool at hand, even if your data sits in SQL Server or on the web in MySQL or other databases. It shouldn’t be that hard anywhere.

Salesforce has a great reporting tool once you get used to it. CiviCRM, long dogged by absence of output mechanisms, now is on the verge of addressing this. And it doesn’t have to be in big, complex systems. And definitely check out the truly smart “Smart Lists” component to Mission Research’s GiftWorks software. As a reporting tool builder, I'm envious.

Going further, staff fundamentally do not want to enter the same data twice. If they do, it’s more likely because they can’t make the actual lists they want than that important data collection fields are missing. Even if you have made sure you have the right tools, it is so easy to short time for customizing the reporting features or in training on reporting. Adding another custom field or two to a web page typically takes a lot less time than getting it into appropriate search pages or output templates. You have to consider all of it in planning for reporting. An easy way to protect yourself is to include reporting elements in each phase of a projects implementation, instead of having a giant reports phase at the end.

Third, there is life beyond spreadsheets. Some of the most exciting stuff at the 2009 NTC had to do with using free and low cost tools for visualizing data. Visualizing data can mean any framework that enables the information you want to organize come alive in context. It can be putting it on an interactive map. At the 2009 NTC, Peter Black showed some done for the Environmental Defense Fund (edf.org) such as this not-so-fun exploration of sea level rise: Or , check out Google motion charts and the work it is based on at gapminder.org.

Data visualization is a whole separate topic. I’ll just say that thinking creatively with today’s tools about visualization is also part of how to break out of the dust-to-dust, spreadsheet in to spreadsheet out framework for data , and to perhaps generate greater enthusiasm for and quality of data collection.