Working with Voter Files

There was a great thread recently on the ProgressiveExchange discussion list, about the challenges and options for working with voter files. I didn't know much about this area, so I found the conversation really useful.

The basic problem is that voter files are big. Really big. A statewide voter file - summarizing the registered voters for a particular state - can have millions of entries. The general consensus was that these files are too big to try to read via common office software, like Access or Excel. Excel has a cap on the actual number of rows you can have (the number depends on the version, but it's only around a million even for the most up-to-date versions), and Access is likely to be extremely slow and undependable - and at risk for catastrophic failure and data loss.

So what should you do instead? A number of people suggested outsourcing the data management process - there are folks who specialize in voter files, such as Astro or VAN (Voter Action Network). These services aren't cheap, but they provide a number of benefits. They deal with all the data, and let you just pull the reports, walk sheets, call sheets, etc that you want. They also can be accessed over the internet - a big plus for organizations working with organizers in multiple offices or locations.

It can also be useful to look for other organizations using voter files for the same geographic area - you might be able to share their file infrastructure, or go in together on one of the outsourced services.

If you do want to store the data in-house, the consensus was that you'll want a SQL database back end, optimized by a professional, and likely a specific database server - probably a several thousand dollar investment at a minimum.

The Progressive Technology Project has more info - as well a lot of great resources about technology useful in organizing in their Voter Tech Kit.