Ask most any developer what's the worst part of a software project and mostly likely you'll hear, “data conversion.” And what's most magical when working a project spec? “We're going to start fresh with new data.”
Reflecting back on the year gone by, clean or messy data migration definitely has a huge impact project success as a whole.
What makes the data migration part of a project so hard?
You look at old data on screen, in reports, or with tech specs. That's often all you get to look at before starting, and that's not enough to judge its true quality. You can't fully judge how well its going to work until you do real trials into the new system. And you can't really do that until you have that new system ready.
So here are painfully accumulated tips about data conversion. I'm writing this from the point of view of the “joint planning team”--lessons to both client and consultant/developer. And I'm not going to say which ones I have the most success sticking to. I'm doing this partly as therapy and partly as New Year's resolution. I promise to reread this myself on every new 2010 project.
Data conversion and the project plan
Do as much clean up as possible in the old system. The data will look familiar there, and staff who have to do it will be faster on the old.
Don't wait until the end. Put time in the schedule to for clean-up straight through the planning and configuration of the new. Reflect each decision about the new back to the old. Understand how you will get there and when the route can be opened.
It's often hard to reserve enough time for clean-up. Explore whether you can effectively use volunteers, interns, or temp staff to do routine clean up work instead of taking consultant project time to do it or burdening organizational staff beyond what is reasonable or essential for each side to do.
Data conversion and the budget
Here's a tough one: See if you take data conversion outside of the rest of the project estimate and work it as time and materials. That protects the consultant and focuses program manager attention on only doing as much as is necessary.
The fall-back choice is to allocate a specific, reasonable number of hours, say as a percent of the whole project, and agree that the consultant will do much and as best they can within that block of time, and no more.
Data conversion and "letting go"
A corollary to the last: Consider converting the essential top level stuff—the primary contact lists—and not all the historical details (all donation or membership history, services rendered, etc). The top level lists are often cleaner, easier to check, more essential, and less encumbered with old-to-new data structure changes.
For historical data you really do need, consider adding special new fields to summarize the old. For example, instead of importing an entire “trainings” history, can you live with aggregating “total workshops attended” for each contact? And remembering preferred training topics, how about just tagging the contact with “attended series A workshops” or “workshop B series”? You might be surprised how much headache this saves where the old data turns out not as clean as it now appears on screen.
Data conversion in stages
It frequently makes sense to plan on a data migration way station--a temporary clean-up and check-me database. It could be Excel, Access or something else that has strong query and reporting tools. Maybe all the status categories need to be remapped, or collected from three different current places. Maybe the notes field holds way more than notes and needs analysis
On the other hand, the new system may have fantastic clean up tools available, such as address correction and verification, or deduping. Maybe it specifically has importers for the old donor database you are moving form. Check them on sample data and use them where you can.
What about all the rest of the details of the old? If keeping the old system around does not incur licensing or hosting costs, why not just leave it up, frozen as an archive for historical data. Again, only convert what is really needed.
Data conversion and testing
Count on doing at least one trial conversion for staff to test before the day you go live. Once you have worked that into the plan, it makes sense to do it as soon as a trial version of the new software is up. Let folks test the new software with some real, existing data. It improves the testing and improves the data migration. And make sure you have put time on the calendar for that checking. Discovering gaps six months into a new system, say when quarterly or annual reports first have to run for real, really will mess things up.
Do have a series of benchmark queries to run on old and new system to quickly check results. For example, maybe the count of contacts by status group has to match exactly. Maybe the total dollar value of donations by year will not match 100% for this, that and the other reason. Make an agreement in advance, the new system will vary by as much as 3% earlier than the last two years.
Conversely, agree on a couple representative accounts you will use as a standard base line for checking the conversion. I remember one large scale donor management system with a huge master contact list mostly sparsely populated and just a handful with tons of details. Running overall lists just wouldn't cut it. Every step in the development process had to be run against one particular wealthy donor with a 28-page single spaced personal profile. Each new iteration of the software had to pass muster on him or it was back to the cubicles for us.
Data conversion documentation
Data conversions usually involve a lot of steps. List them out in a shared document and explain what each does. When you have a series of scripts that have to run, number them in order, like “step100_clear main contact table,” “step 200_import from spreadsheet xyz,” “step310_relink contacts table to database X_manual step,” step 315 _import from database x,” “step 400_run dedup query,” “step510_extract priority A people to excel and give to ED to check personally.” By putting that first step in there, you have already set things up for running the conversion process as a whole more than once. Get every step, manual or automated, in there. Consider counting by 10s at least so you insert more steps as you fine tune the overall process.
Use a shared Google doc or some other collaboratively editable document to hold the conversion steps. Even if your steps lean toward the technical, take the time to go through them collaboratively, understand what they mean, and sign off that this is the plan.
Data conversion therapy
Accept that data conversion has a therapeutic component. You can count on the staff person who is most impatient with the old system to miss data and procedures that don't show up exactly the same in the new. Hmm. My Tai Chi sensibility says maybe the framework is more meditation than therapy: it's all about grasping the essence firmly, remaining calm to find the path from the old to the new, and letting go of what you can.
And through it all, be reasonable and patient with each other. This means the organizational lead needs to look at the raw underbelly of the data and appreciate whatever is giving he consultant the most trouble. The consultant needs to sit with users and understand the consequences, workarounds, new ways of doing things if new data doesn't 100% line up with the old.
Data conversion sounds like such a mine field that you might ask, why would anyone ever agree to do it? Well, you often have to, to get the project at all. In addition, I have to say, data conversion does have a creative appeal. Its almost always unique. Unlocking old data is like unraveling a mystery. It requires real detective work. It requires knowledge of multiple systems, which takes time and experience to accumulate. If both sides agree to a reasonable plan, it is possible for all to find satisfaction and joy when the new reports line up with the old.
And speaking of the therapeutic and the meditative, just writing this has been both therapeutic and meditative for me. hope it helps you as well, and best wishes for the New Year.
Last week, I reported that Nonprofit assessors like Charity Navigator and Guidestar will be moving to a model of judging effectiveness (as opposed to thriftiness). The title of my post drew some criticism. People far more knowledgeable than I am on these topics questioned my description of this as a "sea change", and I certainly get their point. Sure, the intention to do a fair job of judging Nonprofits is sincere; but the task is daunting. As with many such efforts, we might well wind up with something that isn't a sea change at all, but, rather, a modified version of what we have today that includes some info about mission effectiveness, but still boils down to a financial assessment.
Why would this happen? Simple. Because metrics are numbers: ratios, averages, totals. It's easy to make metrics from financial data. It's very difficult to make them out of less quantifiable things, such as measuring how successfully one organization changed the world; protected the planet; or stopped the spread of a deadly disease.
I used to work for an org whose mission was to end poverty in the San Francisco Bay Area. And, sure enough, at the time, poverty was becoming far less prevalent in San Francisco. So could we be judged as successful? Could we grab the 2005 versus 2000 poverty statistics and claim the advances as our outcomes? Of course not. The reduction in poverty had far more to do with gentrification during the dotcom and real estate booms than our efforts. Poverty wasn't reduced at all; it was just displaced. And our mission wasn't to move all of the urban poor to the suburbs; it was to bring them out of poverty.
So the announcement that our ratings will now factor in mission effectiveness and outcomes could herald something worse than we have today. The dangerous scenario goes like this:
Charity Navigator, Guidestar, et al, determine what additional info they need to request from nonprofits in order to measure outcomes.
They make that a requirement; nonprofits now have to jump through those hoops.
The data they collect is far too generalized and subjective to mean much; they draw conclusions anyway, based more on how easy it is to call something a metric than how accurate or valuable that metric is.
NPOs now have more reporting requirements and no better representation.
So, my amended title: "We Need A Sea Change In The Way That Our Organizations Are Assessed".
I'm harping on this topic because I consider it a call to action; a chance to make sure that this self-assessment by the assessors is an opportunity for us, not a threat. We have to get the right people at the table to develop standardized outcome measurements that the assessing organizations can use. They can't develop these by themselves. And we need to use our influence in the nonprofit software development community to make sure that NPOs have software that can generate these reports.
The good news? Holly Ross of NTEN got right back to me with some ideas on how to get both of these actions going. That's a powerful start. We'll need the whole community in on this.
This is very good news. That overhead metric has hamstrung serious efforts to do bold things and have higher impact. An assessment that is based solely on annualized budgetary efficiency precludes many options to make long-term investments in major strategies. For most nonprofits, taking a year to staff up and prepare for a major initiative would generate a poor Charity Navigator score. A poor score that is prominently displayed to potential donors.
Assuming that these new metrics will be more tolerant of varying operational approaches and philosophies, justified by the outcomes, this will give organizations a chance to be recognized for their work, as opposed to their cost-cutting talents. But it puts a burden on those same organizations to effectively represent that work. I've blogged before (and will blog again) on our need to improve our outcome reporting and benchmark with our peers. Now, there's a very real danger that neglecting to represent your success stories with proper data will threaten your ability to muster financial support. You don't want to be great at what you do, but have no way to show it.
More to the point, the metrics that value social organizational effectiveness need to be developed by a broad community, not a small group or segment of that community. The move by Charity Navigator and their peers is bold, but it's also complicated. Nonprofit effectiveness is a subjective thing. When I worked for a workforce development agency, we had big questions about whether our mission was served by placing a client in a job, or if that wasn't an outcome as much as an output, and the real metric was tied to the individual's long-term sustainability and recovery from the conditions that had put them in poverty.
Certainly, a donor, a watchdog, a funder a, nonprofit executive and a nonprofit client are all going to value the work of a nonprofit differently. Whose interests will be represented in these valuations?
So here's what's clear to me:
- Developing standardized metrics, with broad input from the entire community, will benefit everyone.
- Determining what those metrics are and should be will require improvements in data management and reporting systems. It's a bit of a chicken and egg problem, as collecting the data wis a precedent to determining how to assess it, but standardizing the data will assist in developing the data systems.
- We have to share our outcomes and compare them in order to develop actual standards. And there are real opportunities available to us if we do compare our methodologies and results.
This isn't easy. This will require that NPO's who have have never had the wherewith-all to invest in technology systems to assess performance do so. But, I maintain, if the world is going to start rating your effectiveness on more than the 990, that's a threat that you need to turn into an opportunity. You can't afford not to.
And I look to my nptech community, including Idealware, NTEN, Techsoup, Aspiration and many others -- the associations, formal, informal, incorporated or not, who advocate for and support technology in the nonprofit sector -- to lead this effort. We have the data systems expertise and the aligned missions to lead the project of defining shared outcome metrics. We're looking into having initial sessions on this topic at the 2010 Nonprofit Technology Conference.
As the world starts holding nonprofits up to higher standards, we need a common language that describes those standards. It hasn't been written yet. Without it, we'll escape the limited, Form 990 assessments to something that might equally fail to reflect our best efforts and outcomes.
A few months ago, I blogged a bit about Google Wave, and how it might live up to the hype of being the successor to email. Now that I've had a month or so to play with it, I wanted to share my initial reactions. Short story: Google Wave is an odd duck, that takes getting used to. As it is today, it is not that revolutionary -- in fact, it's kind of redundant. The jury is still out.
If you haven't gotten a Wave invite and want to try it, now is the time to query your Twitter and Facebook friends, because invites are being offered and we've passed the initial, competitive "gimme" stage. They should be easier to find if you speak up. And, once you get there (or if you are there and don't know what to do), there are some excellent ways to start learning and playing, which I'll discuss below.
Awkwardness
To put Wave in perspective, I clearly remember my first exposure to email. I bought my first computer in 1987: a Compaq "portable". The thing weighed about 60 pounds, sported a tiny green on black screen, and had two 5 and 1/4 inch floppy drives for applications and storage). Along with the PC, I got a 1200 BPS modem, which allowed me o dial up local bulletin boards. And, as I poked around, I discovered the 1987 version of email: the line editor.
On those early BBSes, emails were sent by typing one line (80 characters, max) of text and hitting "enter". Once "enter" was pressed, that line was sent to the BBS. No correcting typos, no rewriting the sentence. It was a lot like early typewriters, before they added the ability to strike out previously submitted text.
But, regardless of the primitive editing capabilities, email was a revelation. It was a new medium; a form of communication that, while far more awkward than telephone communications, was much more immediate than postal mail. And it wasn't long before more sophisticated interfaces and editors made their way to the bulletin boards.
Google Wave is also, at this point, awkward. To use it, you have to be somewhat self-confident right from the start, as others are potentially watching every letter that you type. And while it's clear that the ability to co-edit and converse about a document in the same place is powerful, it's messy. Even if you get over the sprawling nature of the conversations, which are only minimally better than what you would get with ten to twenty-five people all conversing in one Word document, the lack of navigational tools within each wave is a real weakness.
Redundant?
I'm particularly aware of these faults because I just installed and began using Confluence, a sophisticated, enterprise Wiki (free for nonprofits) at my organization. While we've been told that Wave is the successor to email, Google Docs and, possibly, Sharepoint, I have to say that Confluence does pretty much all of those things and is far more capable. All wikis, at their heart, offer collaborative editing, but the good ones also allow for conversations, plug-ins and automation, just as Google Wave promises. But with a wiki, the canvas is large enough and the tools are there to organize and manage the work and conversation. With Wave, it's awfully cramped, and somewhat primitive in comparison.
Too early to tell?
Of course, we're looking at a preview. The two things that possibly differentiate Wave from a solid wiki are the "inbox" metaphor and the automation capabilities. Waves can come to you, like email, and anyone who has tried to move a group from an email list to a web forum knows how powerful that can be. And Wave's real potential is in how the "bots", server-side components that can interact with the people communicating and collaborating, will integrate the development and conversation with existing data sources. It's still hard to see all of that in this nascent stage. Until then, it's a bit chicken and egg.
Wave starting points
There are lots of good Wave resources popping up, but the best, hands down, is Gina Trapini's Complete Guide, available online for free and in book form soon. Gina's blog is a must read for people who find the types of things I write about interesting.
Once you're on wave, you'll want to find Waves to join, and exactly how you do that is anything but obvious. the trick is to search for a term "such as "nonprofit" or "fundraising" and add the phrase "with:public". A good nonprofit wave to start with is titled, appropriately, "The Nonprofit Technology Wave".
If you haven't gotten a Wave invite and want to, now is the time to query your Twitter and Facebook friends, because invites are being offered and we've passed the initial "gimme" stage. In fact, I have ten or more to share (I'm peterscampbell on most social networks and at Google's email service).
Meaningful shared data, like the Miles Per Gallon ratings on new car stickers or the calorie counts on food packaging help us make better choices;
But not all data is as easy to interpret;
Nonprofits have continually been challenged to quantify the conditions that their missions address;
Shared knowledge and metrics will facilitate far better dialog and solutions than our individual efforts have;
The web is a great vehicle for sharing, analyzing and reporting on data;
Therefore, the nonprofit sector should start defining and adopting common data formats that support shared analysis and reporting.
I've made the case before for shared outcomes reporting, which is a big piece of this. Sharing and transparency aren't traditional approaches to our work. Historically, we've siloed our efforts, even to the point where membership-based organizations are guarded about sharing with other members.
The reason that technologists like Marnie and I end up jumping on this bandwagon is that the tech industry has modeled the disfunction of a siloed approach better than most. early computing was an exercise in cognitive dissonance. If you regularly used Lotus 123, Wordperfect and dBase (three of the most popular business applications circa 1989) on your MS-DOS PC, then hitting "/", F7 or "." were the things you needed to know in order to close those applications respectively. For most of my career, I stuck with PCs for home use because I needed compatibility with work, and the Mac operating system, prior to OSX, just couldn't easily provide that.
The tech industry has slowly and painfully progressed towards a model that competes on the sales and services level, but cooperates on the platform side. Applications, across manufacturers and computing platforms, function with similar menus and command sequences. Data formats are more commonly shared. Options are available for saving in popular, often competitive formats (as in Word's "Save As" offering Wordperfect and Lotus formats). The underlying protocols that fuel modern operating systems and applications are far more standardized. Windows, Linux and MacOS all use the same technologies to manage users and directories, network systems and communicate with the world. Microsoft, Google, Apple and others in the software world are embracing open standards and interoperability. This makes me, the customer, much less of an innocent bystander who is constantly sniped by their competitive strategies.
So how does this translate to our social service, advocacy and educational organizations? Far too often, we frame cooperation as the antithesis to competition. That's a common, but crippling mistake. The two can and do coexist in almost every corner of our lives. We need to adopt a "rising tide" philosophy that values the work that we can all do together over the work that we do alone, and have some faith that the sustainable model is an open, collaborative one. Looking at each opportunity to collaborate from the perspective of how it will enhance our ability to accomplish our public-serving goals. And trusting that this won't result in the similarly-focused NGO down the street siphoning off our grants or constituents.
As Marnie is proposing, we need to start discussing and developing data standards that will enable us to interoperate on the level where we can articulate and quantify the needs that our mission-focused organizations address. By jointly assessing and learning from the wealth of information that we, as a community of practice collect, we can be far more effective. We need to use that data to determine our key strategies and best practices. And we have to understand that, as long as we're treating information as competitive data; as long as we're keeping it close to our vests and looking at our peers as strictly competitors, the fallout of this cold war is landing on the people that we're trying to serve. We owe it to them to be better stewards of the information that lifts them out of their disadvantaged conditions.
I'm wrapping up the Drupal 101 series with some talk about Drupal themes, and some additional info on topics that we've already covered. The goal of these posts is to give new Drupal administrators an idea about how Drupal works, and some pointers to the key add-ons and resources in the broad Drupal ecosystem. For reference' sake, we started with an intro, moved on to Modules, and then covered navigation. So, now that we have a functional web site, what does it look like?
Getting Themes
Drupal comes with five or six themes to choose from, and, if you use them, then your site will look very, um, uninspired. This might not be a problem if your goal is not to impress your visitors, but simply provide information or functionality, but, if you're putting up a website for your organization, you want one that stands out from the crowd. So you have two choices: you can find a better, less common theme, or you can customize one of the default themes.
The first place to go is to Drupal Theme Garden. This is where many Drupal theme designers share their work. Here, you can either find a theme to use (or customize for your use), or get a good idea about the types of things you can do with your theme.
Customizing Themes
From the Administration menu, you can modify any theme's main text elements, deciding whether or not to display your site's mission or slogan, name or logo. And you can replace the default "droplet" logo with your own logo (a no-brainer!). Assuming that you've started with a theme that you really like, this might be enough. But, if you want to do more serious customizations, such as moving the logo to the center of your header or changing the site colors, you're going to need basic web 4.0 programming skills and, most likely, some level of comfort with the PHP scripting language.
Most themes consist of one or more style sheets, a number of "tpl" files with PHP/HTML code laying out various page elements, such as blocks, footers and sidebars, and one called page.tpl.php that establishes the overall page layout. The main styles are usually stored in styles.css, and you can make a lot of changes to your site's appearance here, modifying default background colors and images, placing and resizing content.
If that's not enough, most customizations can be done using Wordpress's internal macros and functions, meaning that you won't have to worry about assigning variables or what goes into the foreach loops. Wordpress has simple commands that you can insert into a page to loop through your posts and display them or list your categories in the sidebar. A nice breakdown of the Wordpress functions can be found at WpExplorer.com.
If you do modify the stylesheets and templates, make sure that you are storing your themes in sites/default folder and that you're properly backing up whenever you do an upgrade. If you modify theme files in the main themes folder, and then upgrade to, say, a Drupal security fix, your modifications will be overwritten. In general, themes remain functional from dot release to dot release (e.g., what worked for Drupal 6.1 still works in 6.9), but the Drupal maintainers often make dramatic changes in number versions, so don't assume that your theme in Drupal 6.9 will not be messed up if you upgrade to Drupal 7 (coming soon).
More Installation Options
In the first Drupal 101 post, I mentioned Fantastico, a two-click installer for Drupal available on most hosting services that use the cPanel site management interface. I subsequently ran into this useful article about Elefante and Simplescripts. These are packages that you can use to install a variety of popular open source applications, including Drupal.
In addition to application installers, there are other options for installing Drupal:
Customized Drupal installations like Open Atrium and Acquia come with more modules and functionality.
There's been some development and discussion about Installation Profiles, a Drupal add-on functionality that lets you define additional installation details, such as module defaults and inclusion of additional modules and data for distributing custom Drupal installations.
Conclusion
What I hope this Drupal 101 series has done is to offer some context and guidance for people new to Drupal who are about to give it a try, and some backing to my initial proposition that Drupal's strength is it's flexibility. Along the way, I've received tweets asking "Why Drupal?" and my answer is that Drupal isn't the only CMS out there, or necessarily the best one for your web site. There are a huge variety of commercial and open source options. In fact, my personal website runs on a combination of Frog CMS and Wordpress, because I wanted a simple tool for integrating RSS feeds, which Frog provides, and a powerful blogging platform. On the other hand, last week the White House ditched their commercial CMS for Drupal. So this series might also inspire you to look elsewhere, particularly if a more traditional, tree-structured content management interface will work better for you than Drupal's layout by association model. Whichever way you go, we suffer more from a surfeit of good options than a lack of same.
Here's the third in a series of posts on getting started with Drupal, the popular open source content management system. The short intro and discussion on modules are best read first. Today we'll look at site structure, and how menus, blocks and taxonomies can make your site navigable for your visitors.
Menus
Drupal has a simple and flexible tool for creating and managing menus. You can check/uncheck standard functions; assign them to regions (left sidebar, right sidebar, header, footer, etc.); and easily create new items.
By default, Drupal offers three menus that you can add to your site:
Navigation - The main menu is dynamic. It displays items based on the visitor's role and state of authentication. For example, an unauthenticated user might see a "Login" menu item, while an authenticated user would see "logout". An authenticated user who is also a site manager would see the Administer menu. This menu is usually placed in a sidebar, next to the main content
Primary Links - This is often the menu for the main content areas, e.g. Home, Blog, Calendar, About. Primarily links are usually placed in a site's header.
Secondary Links can be used for less popular pages, but ones that you want to have available, such as site maps, privacy notices, and contact links.
You can assign a menu item to any particular piece of content, or to a collection of items by content type. Drupal assigns numbers to individual items. The basic content type is called a node, so the default first page of a web site would be at http://your-site.org/node/1. If you create a blog, the first post would be at http://your-site.org/blog/1.
Tip: Be sure that the Path Module is enabled. Path lets you can rename items with friendlier names than, say, site/node/113.
Say you wanted blog/1 to be your front page, but you also wanted something easier to remember to appear in the address bar, you could rename it "home", so that people could browse directly to the site at http://your-site.org/home. They would see, in the center of the home page, that first blog entry. Drupal's general settings allow you to identify your home page; renaming a numeric page simply makes it friendlier for your users.
If, instead, you simply wanted the whole blog to be the home page, then you would skip the numbers, and not bother with a rename, as linking the front page to http://your-site.org/blog would accomplish that.
Drupal's real power comes in when you realize that, with the CCK module, you can make your own content types, and that can be very easy. A press release will have a similar format to a blog item (title, content). So you can create a type called press_release and link a page to it: http://your-site.org/press_release. All new press releases that you post to the site from Create Content/Press Release will appear there.
Blocks
Blocks are boxes that can be placed on one or more pages or associated with one or more content types. They usually appear in the left or right sidebars. Strategically associating blocks with particular content can be a subtler way o offer navigational aids. For example, you might want to have a block with current open positions appear on your "About" page, but not necessarily with your blog. Or you might not want the job listings to appear on pages describing your services, instead featuring a "Donate Now" box. This flexibility allows you to align content in ways that make sense for the different audiences with varying interests that your site will attract.
Taxonomies
All of the above is fine for sites without a lot of content. But, once you have a library of blog entries, press releases and documents to share, you'll want to give your visitors a way to find what they're looking for that doesn't involve inordinate amounts of scrolling. Search is a no-brainer, but even more important is to organize your content with meaningful labels. For this, use the Taxonomy module.
Taxonomies allow you to tag or classify your content using hierarchal terminology. For example, if your NPO serves the homeless, you might have papers on poverty and employment, descriptions of available shelters and programs, job opportunities, and much more. You can break this content down into meaningful categories, then assign sub-terms in each category. Once the taxonomy is in place, you can assign menu items to terms in your taxonomy, thus aggregating all of the relevant content on a single page. You can set up menu blocks for the sub-terms and assign each block to it's category page. The result is a content rich, drill down web site.
That's it for navigation. Next week, we'll talk about Themes and ways you can make your Drupal site distinctive.
Last week, I kicked off this series on setting up a basic web site with Drupal, the popular open source Content Management System. This week we're going to take a closer look at Modules, the Drupal add-ons that can extend your web site's functionality. One of the great things about Drupal is that it is a popular application with a large developer community working with and around it. So there are about a thousand modules that you can use to extend Drupal, covering everything from document management to payment processing. The good news: there's probably one that supports the functionality that you want to add to your web site. Bad news: needle in a haystack?
A potentially easier way to add extra functionality to Drupal is to download a customized version, such as CiviCRM or Open Atrium. We'll discuss those options later in the Drupal 101 series.
Core Modules
Drupal comes with a number of built-in modules that you can optionally enable. Some are obviously useful, others not so much. Here are some notes on the ones that you might not initially know that you need:
Primary content types like blog, forum and book offer different modules for user input. They can be combined, or you can pick one for a simple site. Since the differences between, say , a blog (individual journal that people can comment on) and a forum (topical posts that people can reply to) are less distinct than they are in other CMS's, you might want to pick one or two primary content types and then supplement them with more distinctive ones, such as polls or profiles.
Enabling contact allows your users to send private messages to each other on the site, as well as allowing you to set up site-wide contact forms.
OpenID allows your users more flexibility and control as to how they log into your site. I can't see a good reason not to enable this on a public site. Since more and more people have profiles on social networking sites and Google, tools like Facebook Connect or Google Friend Connect should be considered as well.
By default, Drupal asks new users for a name and email, but not much else. With the Profiles module, you can create custom fields and allow your users to share information much as they would on a social network.
Taxonomy is also recommended, and I'll talk more about that next week.
Throttle should be used on any high-traffic site to improve performance.
Use Trigger if you want to set up alerting and automation on your site.
More than some CMS's, Drupal is a content-centric system. It doesn't simply manage content, but the web interface is structured around the content it manages: content types, content metadata (taxonomies), content sources (RSS feeds). Out of the virtual box, Drupal has content types like blog entries, pages and stories. Each content type has a data entry form associated with it. So, if you create a number of stories, and you want to read them all, then you can browse to the page "story" and they'll all be listed there. CCK helps you create additional content types and use a fairly robust form-builder to customize the screens.
The Views module lets you customize the appearance and functionality of many of Drupal's standard screens, and to add your own. Unlike CCK, which is limited to the default layout of content types, Views lets you seriously customize the interface. One easy reason to install Views is in order to take advantage of the Calendar view, which gives you not only a full page, graphical calendar to add events to and display, but also sidebar calendar widgets and upcoming event lists.
Here's a tip: setting up the calendar view is reasonably tedious. The best write-up explaining it (for Drupal 6) is here: http://drupal.org/node/326061. Drupal's documentation is okay, but this is step-by-step. It does miss one step, though, which is to add the "Event Date - From date" and "Event Date - To date" to the Fields listing (with friendlier titles, like "From" and "To"). Otherwise, calendar items show on the day they were submitted instead of the day that they are occurring.
There's a good case to be made that these two modules should be folded into Drupal's base package, because, in addition to providing very powerful customization features to the core product, there are a whole slew of additional modules that require their presence. If you plan to install a number of modules and/or customize your site, these are pretty much pre-requisites, so just grab and install them.
What-You-See-Is-What-You Get, or Rich Text Format (RTE) editors transform Drupal's default data input boxes into flexible editors with Word-like toolbars. The WSYIWYG module lets you install the editor of your choice. I've done well with FCKEditor (recently rebranded CKEditor, thank you!). The WYSIWYG module lets you work with multiple RTE packages and strategically assign them to different fields and content types. Most RTE editors are very configurable, but note that, in addition to installing the modules, you need to install the editors themselves, so follow the instructions carefully.
If you're building a community site, with hopes of having lots of interactive, social features, Organic Groups gives you the flexibility to not only create all sorts of groups and affiliations on your own, but let your users create their own groups as well, much like Facebook does. For an interactive site, this is essential.
E-Commerce/Donations
Many modules are available for either integrating with Authorize.net or Paypal, or setting up your own e-commerce site. The aptly named e-Commerce module and Ubercart are among the better known and supported options.
Drupal fans: what modules do you recommend? Which do you install first? Leave your recommendations in the comments.
Next week, we'll talk about menus, blocks and taxonomies: Drupal 101: Navigation.
The popular theory is that, with social networks like Twitter and Facebook serving as link referral tools, there's no need to setup and look at feeds in a reader anymore. And I agree that many people will forgo RSS in favor of the links that their friends and mentors tweet and share. But this is kind of like saying that, if more people shop at farmer's markets than supermarkets, we will no longer need trucks. Dave Winer, quite arguably the founder of RSS, and our friends at ReadWriteWeb have leapt to RSS's defense with similar points - Winer puts it best, saying:
"These protocols...are so deeply ingrained in the infrastructure they become part of the fabric of the Internet. They don't die, they don't rest in piece."
My arguments for the defense:
1. RSS is, and always has been about, taking control of the information you peruse. Instead of searching, browsing, and otherwise separating a little wheat from a load of chaff, you use RSS to subscribe to the content that you have vetted as pertinent to your interests and needs. While that might cross-over a bit with what your friends want to share on Facebook, it's you determining the importance, not your friends. For a number of us, who use the internet for research; brand monitoring; or other explicit purposes, a good RSS Reader will still offer the best productivity boost out there.
2. Where do you think your friends get those links? It's highly likely that most of them -- before the retweets and the sharing -- grabbed them from an RSS feed. I post links on Twitter and Facebook, and I get most of them from my Google Reader flow.
3. It's not the water, it's the pipe. The majority of those links referred by Twitter are fed into Twitter via RSS. Twitterfeed, the most popular tool for feeding RSS data to Twitter, boasts about half a million feeds. Facebook, Friendfeed and their ilk all allow importing from RSS sources to profiles.
So, here are some of the ways I use RSS every day:
Basic Aggregation with Drupal
My first big RSS experiment built on the nptech tagging phenomenon. Some background: About five years ago, with the advent of RSS-enabled websites that allowed for storing and tagging information (such as Delicious, Flickr and most blogging platforms), Techsoup CEO Marnie Webb had a bright idea. She started tagging articles, blog posts, and other content pertinent to those working in or with nonprofits and technology with the tag "nptech". She invited her friends to do the same. And she shared with everyone her tips for setting up an RSS newsreader and subscribing to things marked with our tag. Marnie and I had lunch in late 2005 and agreed that the next step was to set up a web site that aggregated all of this information. So I put up the nptech.info site, which continues to pull nptech-tagged blog entries from around the web.
Other Tricks
Recently, I used Twitterfeed to push the nptech aggregated information to the nptechinfo Twitter account. So, if you don't like RSS, you can still get the links via Twitter. But stay aware that they get there via RSS!
But I'm pretty dull -- what's more exciting is the way that Google Reader let me create a "bundle" of all of the nptech blogs that I follow. You can sample a bunch of great Idealware-sympatico bloggers just by adding it to your reader.
Google is on a fishing expedition to see if we're willing to take web-surfing to a whole new level. My colleague Steve Backman introduced us to Google Wave a few months ago. I attended a developer's preview at Techsoup Headquarters last week, and I have some additional thoughts to share.
Google's introduction of Wave is nothing if not ambitious. As opposed to saying "We have a new web mashup tool" or "We've taken multimedia email to a new level", they're pitching Wave as nothing less than the successor to email. My question, after seeing the demo, is "Is that an outrageous claim, or a way too modest one?".
The early version of Google Wave I saw looked a lot like Gmail, with a folder list on the left and "wave" list next to it. Unlike Gmail, a third pane to the right included an area where you can compose waves, so Wave is three-columner to Gmail's two.
A wave is a collaborative document that can be updated by numerous people in real-time. This means that, if we're both working in the same wave, you can see what I'm typing, letter by letter, as I can see what you add. This makes Twitter seem like the new snail mail. It's a pretty powerful step for collaborative technology. But it's also quite a cultural change for those of us who appreciate computer-based communications for the incorporated spell-check and the ability to edit and finalize drafted messages before we send them.
Waves can include text, photos, film clips, forms, and any active content that could go into a Google Gadget. If you check out iGoogle, Google's personal portal page, you can see the wide assortment of gadgets that are available and imagine how you would use them -- or things like them -- in a collaborative document. News feeds, polls, games, utilities, and the list goes on.
You share waves with any other wave users that you choose to share with. User-level security is being written into the platform, so that you can share waves as read-only or only share certain content in waves with particular people.
Given these two tidbits, it occurred to me that each wave was far more like a little Extranet than an email message. This is why I think Google's being kind of coy when they call it an email killer - it's a Sharepoint killer. It's possibly a Drupal (or fill in your favorite CMS here) killer. It's certainly an evolution of Google Apps, with pretty much all of that functionality rolled into a model that, instead of saying "I have a document, spreadsheet or website to share" says "I want to share, and, once we're sharing, we can share websites, spreadsheets, documents and whatever". Put another way, Google Apps is an information management tool with some collaborative and communication features. Google Wave is a communications platform with a rich set of information management tools. It's Google Docs inverted.
So, Google Wave has the potential to be very disruptive technology, as long as people:
Adopt it;
Feel comfortable with it; and
Trust Google.
Next week, I'll spend a little time on the gotcha's - please add your thoughts and concerns in the comments.
I'm following up on my post suggesting that Wikis should be grabbing a portion of the market from word processors. Wikis are convenient collaborative editing platforms that remove a lot of the legacy awkwardness that traditional editing software brings to writing for the web. Gone are useless print formatting functions like pagination and margins; huge file sizes; and the need to email around multiple versions of the same document.
There are a lot of use cases for Wikis:
We can all thank Wikipedia for bringing the excellent crowd-sourced knowledgebase functionality to broad attention. Closer to home we can see great use of this at the We Are Media Wiki, where NTEN and friends share best practices around social media and nonprofits.
Collaborative authoring is another natural use, illustrated beautifully by the Floss Manuals project.
Project Management and Development are regularly handled by Wikis, such as the Fedora Project
Wikis make great directories for other media, such as Project Gutenburg's catalogue of free E-Books.
Almost any popular Wiki software will support the basic functionality of providing user-editable web pages with some formatting capability and a method (such as "CamelCase") to signify text that should be a link. But Wikis have been exploding with additional functionality that ramps up their suitability for all sorts of tasks:
The Floss Manuals team wrote extensions for the Open Source TWiki platform that track who is working on which section of a book and send out updates.
TWiki, along with Confluence, SocialText and other platforms, include (either natively or via an optional plugin) tabular data -- spreadsheet like pages for tracking lists and numeric information. This can really beef up the value of a Wiki as an Intranet or Project Management application.
TWiki and others include built-in form generators, allowing you to better track information and interact with Wiki users.
And, of course, the more advanced Wikis are building in social networking features. Most Wikis support RSS, allowing you to subscribe to page revisions. But newer platforms are adding status updates and Twitter-like functionality.
Before choosing a Wiki platform, ask yourself some key questions:
Do you need granular security? Advanced Wikis have full-blown user and group-based security and authentication features, much like a standard CMS.
Should the data be stored in a database? It might be useful or even critical for integration with other systems.
Does it belong on a local server, or in the cloud? There are plenty of great hosted Wikis, like PBWorks (formerly PBWiki) and WikiSpaces, in addition to all of the Wikis that you can download and install on your own Server. There are even personal Wikis like TiddlyWiki and ZuluPad. I use a Wiki on my Android phone called WikiNotes for my note-keeping.
Are you already using a Wiki? You might be. Google Docs, with it's revision history feature, may look more like a Word processor, but it's a Wiki at heart.
Yes I did just finish In Defense of Food by Michael Pollan, which has been really helpful to me in separating real food from cheap imitations. It's my newest filter for navigating the grocery store, providing an easy to follow set of best practices for avoiding nasty edibles that cause us more long term harm than good. And yes, this book can even help us with software selection.
Michael Pollan reinforces that we must be careful when we enter a unfolding situation, already having decided on certain outcomes. We software experts and users spend a lot of time grouping software into similar features, development models, support system, company backing and more. This work is extremely useful to helping our understanding of software, but must always be taken with a grain (or two) of salt.
So how do we keep our critical eye sharp without getting overwhelmed with the myriad of possibilities when evaluating software?
(1) Frame your context
Understanding your organization processes are key, we software geeks say this all the time. Also important is understanding your organizational capacity to change. Important factors include internal staff expertise, general workload, organizational growth pattern, and system of governance. Defining how you do the work you do helps eliminate features and services that are unnecessary or come into conflict with your organizational context.
(2) Use filters
Filters can help us make sense of complexity by removing some of the detail. Quality software reviews are very helpful filters as they provide both strong context describing who the review is for, and well defined categories to help you group software according to common and critical needs. Internal organizational experience with software are also great filters as they come with institutional knowledge of the organizational context, offering a unique opportunity to combine this knowledge with software experience to shape well fit decisions.
(3) Be discriminating, don't discriminate
Look behind the filter to understand what it does not include. Often filters will look at software from certain perspectives that hide important information. I often run into this problem working with nonprofit associations, for which many vendors have created "association management software". A smartly compiled list of best vendors in this space is a great filter to have. However, it turns out the needs of associations are far more diverse than these association management packages support well. By limiting the focus on association management solutions, nonprofits often miss out compelling offerings provided by donor, membership, constituent management and content management vendors. Vendors generally are one good source of information about their products, however it is much better when verified against independent sources with specific experience using the vendor services.
(4) Test assumptions
Keep a critical eye on your plan for a new system. How many "needs" are supported by assumption versus research? Some nonprofits face difficult change management issues by launching a new system where the core users were not adequately engaged in planning and implementation. Others are disappointed to find that features that claim to be present really are no good for their needs. Beware the salty snacks - some tools make it seem so easy to keep adding fields or new features, but they can make your system very sick with bloat if overindulged. Understand the source of any advise or reviews you receive - what is their context? Who is their intended audience?
Why don’t my report counts show the same numbers on your new system as they did on my old?
Ugh, do I ever hate to have this question come up in a project review meeting. It is surely my candidate for the question consultants least like to hear from a client.
And this question usually follows by a few weeks or months a close runner up question, Can you convert all my old data into the new system. Clients who ask this and consultants who answer yes are looking for trouble for themselves.
This has been on my mind as our team approaches the end of a current and some recent projects whose conclusion has been directly linked to resolving data ‘errors.’ Sometimes you find unexpected joy, such as last winter when a conversion from a really old Raisers Edge version to Giftworks provided historical donation totals that matched entirely. Not to be a pessimist, but yup, I was surprised.
In our hearts if not our experience, we all know that data conversion gaps occur way more often. Here are some thoughts for “both sides” to keep in mind in approaching that next software upgrade.
First, as a background philosophical thought, if an organization’s data looked just right in its old system, would you still need to upgrade or replace? Business practices change and the old ways no longer work. We generally welcome change and we can expect staff to look forward to operating in new forward-looking ways. Yet fitting the old data to the new may be, as a colleague said yesterday may be mixing apples and oranges. The old and the new may never align again.
If you do convert piles of old transaction data or program enrollment data, and the new system has involved strategic planning and incorporation of new business rules, accept that reports will look different. Embrace the new and appreciate it.
Second, conversely on the human or philosophical side of things, you have to make an estimate of how much staff not just “hate the old way” but really are ready to “embrace the new way.” I think of a data conversion some years ago where staff continued to run mailing lists from their old system months after they were doing data entry in their spiffy new system, with its way cooler list generator. Hostage syndrome. They had grown hostage to the features that bore down on them in the old system. So if things ARE going to look different, you need to prepare everyone for it 100% or life will be difficult.
Part of the way to do this is to make sure staff at all levels of the organization are in the room when you discuss data migration. It is not sufficient to just look at reports and lists and to talk with the managers who use them. Let everyone hear how admin and program staff actually enter data, the compromises they make, the guerrilla warfare they may be waging against existing data validations to put what they want where they want it. This may be sobering to senior management and support realism about the conversion.
Third, consider if you can just leave historical data where it is. If you have been paying for a proprietary on-line data system that you want to stop paying for, you may have no choice but to extract old data and put it somewhere it can still be used. If not, why not just continue keep the old system available to run historical reports and start fresh with the new? Two pending conversions to Salesforce to replace a collection of spreadsheets and old Access databases have been way pleasant from a decision to not incorporate years of messy and unaligned old data.
Deciding not to convert may be a major policy decision if senior management or the board is used to a certain kind of review. I don’t underestimate it. But it is worth pursuing the question at the outset, before anything starts.
Fourth, consider converting just top level data, such as all primary demographic and status information for current contacts and leaving aside in the old system. Start with just what you know to be clean and essential.
Fifth, a variation on the third, fill in complex and messy old data by hand over time. As new cases or activity occurs for a contact, fill in or correct the gaps in their data from the old system. Meanwhile, don’t try to run multi-year historical reports on the new system
Sixth, if the data is messy, clean it up parallel to and prior to switching to the new system. My experience: most of the time, it is going to be easier to correct messy data either in its present home or in some convenient way station. In another conversion to Salesforce, where the data now resides in Filemaker, we have agreed to freeze data entry for a bit, and pull the data into a combination of Excel or Access where it will be more susceptible to bulk manipulation and checking.
Seventh, tell the consultant everything you can. If you are relatively new and the system is not well documented, find the staff members who have been there long enough to explain quirky things lost in the mists. In a current conversion from a proprietary undocumented membership system to CiviCRM, we have had one gotcha after another. For example, the current system shows five membership levels. The new system will have the same five membership levels. The paper forms show five membership levels. We checked at the beginning. Great! Easy! Except a faction of the members, in their actual extracted data showed a sixth membership level. And CiviCRM importer chokes on those records so it has to be addressed. OK, we figured this out, but if you add up field by field the consultant detective work to resolve things like this, you suddenly have blown a reasonable data migration budget.
Eighth, agree to leave data conversion out of proposal bids and do it on a time and materials basis only. This will enable everyone as the inevitable problems arise to make pragmatic decisions. Yes, it is worth it to write a script to fix this problem. No, it is not worth it, leave it alone. Or, yes, we need this and organizational staff will fix it ourselves or hire temps to do it.
I regularly resolve not to submit proposals where I have to include data conversion in the cost estimate, and I regularly find myself unable to keep that commitment. Even if you allot a few hours as part of a preproject assessment, it is unlikely you will find all the problems. A current project that is not exactly a system migration, but instead will need to enable staff to use purchased business data in a Drupal-based research and organizing framework. The data is good, used nationally and expensive! It should be clean say the developers! Yet even here we find quirky things. What should we make of a crucial research field which sometimes is stored as an actual dollar amount and sometimes as a coded list of dollar values. Since this is an ongoing data feed and not a one-time data conversion, we need to address it so that data searches will work on both kinds of cost valuation figures. And it’s just not realistic to expect to find all this stuff before you get pretty deeply into it.
Nine, if you are the consultant, you often, not always, do a trial conversion, leave it on a development site for a while for testing, and then do the final conversion to go live. It is really important to carefully document the conversion steps. In one really messy conversion, we ran, tested and refined the 40 or so separate conversion scripts several times before deciding the soup was ready. Different people contributed to the scripts. Having the scripts catalogued and organized lessened the pain of having to rerun them. Plus even a year later, it was possible to go back and find the script that massaged this messed up status category or performed some other task when the client asked about “what happened to the people with interest code x”?
Ten, hmm. I am leaving a slot for you, veterans of data conversions. Your thoughts? I could certainly use them…
Meanwhile, this post derived from a suggestion from fearless editor Laura Q to write about the question consultants least like to hear from clients. When I posed this to my co-workers here at Database Designs, I got so many suggestions, I first thought of doing a “top ten” and decided to just start with my “favorite.” This could turn into an occasional series, and to balance, my colleague Mimi suggested one on questions consultants MOST like to hear from clients.
As the internet has progressed from a shared source of information to a primary communications tool, a natural offshoot of the migration has been where the two things meet: people referring internet information. If you're active at all on Facebook, Twitter, MySpace, Friendfeed, or any of the numerous online communities, big or small, then you are regularly seeing links to useful articles and blog posts; cute YouTube videos, and entertaining photos. Much of this information is passed along from online friend to online friend, but where does the first referral originate from? Usually, it's somebody's RSS reader.
The main reason that I'm such an RSS advocate is that I believe that it's the tool that lets me find the strategic and useful needles lost in the haystack of celebrity gossip, prurient content, and corporate promotional materials that they're buried under. But it isn't "RSS", per se, that does the filtering -- it's other people, whom I call "information agents", who do the sifting. If I want to keep up with fundraising trends, a topic that interests me, but, as an IT Director, isn't my primary area of expertise, I'm not going to spend thirty minutes a day doing research. I subscribe to some very pertinentblogs, and I follow a few people on Twitter and in Reader who find the important and insightful articles and share them with me.
Now it appears that Google wants to cut out the social media middlepeople. As I alluded to in my article on RSS, and fleshed out in this post about sharing with reader, the ability to refer information that you find in Reader is one of the things that makes it so powerful. Last week, Google seriously upped the ante by adding Twitter/Facebook/Delicious-like following, "liking" and sharing to the mix.
Here's what the new features do:
Sharing now lets you share with the world, or just those members of the world that you want to share with. Google has always allowed you to share items, but connecting to other people was a bit arcane and limited, as, by default, Google only allowed you to connect to those that you chat with in GMail. If you read up on it, you learned that you could change that to any defined group of associates in your Google Contacts (all of this assuming that you use Google Contacts - many Google Reader users don't). As someone who does use all of the Google stuff, I still found that opening this up to 80 or so people in my contacts didn't make it clear to many of them as to how they could connect with me.
The new Following feature lets you follow anyone who is willing to share, not just people that you personally communicate with. Now my shared items are marked as public, so anyone can follow my shared items feed by clicking on "Sharing Settings" (in the "People You Follow" section) and searching for me by name or email address. Once you locate me (or someone else), you can (and should) browse through their items to make sure that they share things that you'll find useful. For example, I share a lot of things that are on the topics that I blog about here. But I also share items related to civil rights issues and the occasional link that I find funny. Since humor and politics are very subjective topics, you might want to be sure that you're not going to be annoyed or offended you before you subscribe to a feed.
But the internet is not just about who you know. The Like feature allows you to find new people to follow based on common interests. You'll note that certain articles have a new note at the top saying "XX people liked this", where "XX" is the number of people who have indicated that they like the article by checking the option at the bottom of the post. This message is a link, and clicking it expands it into links to each of the people who "liked" it, allowing you to browse their shared items and optionally follow them. This, to me, enables the real power of the social web -- finding people who share your interests, but have better sources. It's what initially was so exciting about social bookmarking service Delicious, and it's about time that Google Reader enabled it.
I'm hoping the Google's next round of Reader updates will improve our ability to not just tag and classify the information that we find, but also share based on those classifications. That will enable me to selectively publish items that I think are of interest to others, perhaps sending nptech links to Friendfeed and the humorous stuff to Facebook. But I welcome these improvements, and I appreciate the way that reader becomes more and more of a single stop for information discovery and distribution. The Internet would be a messier place without it.
I like tools that just work, even when they do not necessarily do everything I could hope for. Most database tools have features that do not work well, that I just wish they would have not included at all. Our food has too much sugar and refined flour, our economy has too much credit, and our databases have too many features - all of which can lead to serious systemic problems.
In a follow up to my post last year covering Really Simple Databases, here are a few simple online donor management tools to consider.
A relatively new donor management service that got its start providing donation services to churches. Since then the developers have grown the tool to be appropriate for all kinds of nonprofits with simple donor management needs. DonorTools tracks basic donor information with options for multiple salutations, addresses, and methods for contact. Donors are then "tagged" with one or more categories that you determine, creating a simple and robust way to categorize donors into a wide variety of groups.
Donations include options to track the source and the fund, and include options for tracking the gift type, splitting the gift and more. Viewing a donor provides a nice prominent view of their total giving, and their are buttons available to quickly find donations over preset time periods.
Online donation forms can be configured to automatically related donations to the correct fund. There is not a website management tool, but the donation forms can be linked to your site via "Donate Now" buttons, or using the various APIs available these forms can be rendered directly onto a website. Donating online requires either Amazon or Paypal, but DonorTools does not take any percentage of the transactions.
DonorTools current price is $30/month - they say this is for a "limited time", which does not surprise me given the nice array of features offered for this simple system. This price includes unlimited records and logins, including some basic levels of access control to the system.
While I maintain they made a very poor choice in naming their service, they did succeed in making a very simple & usable donor database. If you need a quickie website + online donation system up in a hurry take a look.
It really just tracks very basic contact info per donor - name, address, phone, fax, email - and a few extra tidbits such as contact type and birth date. There is a simple, integrated email blast tool - you either email everyone, or one person in your database, either text or html emails. Blackbaud teamed up with Paypal and ingeniously decided to offer their integrated donor and payment solution for a rather steep 4.95% + $0.30 per donation transaction. Other than these transaction fees, the BlackbaudNow service itself is cost free.
A very simple 6 page website tool is integrated, with over 20 templates to choose from. Only offers one login for managing the system. In general, I would expect that folks looking for a simple online donor management solution would be more satisfied with DonorTools over BlackbaudNow unless the integrated website feature is key. Check out Michelle Murrain's post as well for more information.
For the past four years or so, at two different organizations, I've been evaluating Microsoft's Sharepoint 2007 as a Portal/Intranet/Business Process Management solution. It's a hard thing to ignore, for numerous reasons:
It's an instant, interactive content, document and data management interface out of the box, with strong interactive capabilities and hooks to integrate other databases. If you get the way it uses lists and views to organize and display data, it can be a very powerful tool for managing and collaborating on all sorts of content. As I said a year or two ago in an article on document management systems, it has virtually all of the functionality that the expensive, commercial products do, and they aren't full-fledged portals and Intranet sites as well.
Sharepoint 2007 (aka MOSS) is not free, but I can pick it up via Techsoup for a song.
It integrates with Microsoft Exchange and Office, to some extent, as well as my Windows Directory, so, as I oversee a Windows network, it fits into it without having to fuss with tricky LDAP and SMTP integrations.
All pretty compelling, and I'm not alone -- from the nonprofit CIO and IT Director lists I'm on, I see that lots of other mid to large-sized organizations are either considering Sharepoint, or already well-ensconced.
So, why does Sharepoint scare me?
What it does out of the box, it does reasonably well. Not a great or intuitive UI, but it’s pretty powerful. However, advanced programming and integration with legacy systems can get really complicated very fast. It is not a well-designed database, and integration is based on SOAP, not the far less complicated REST standard, meaning that having someone with a strong Microsoft and XML programming skill set on board is a pre-requisite for doing anything serious with it.
MOSS is actually two major, separately developed applications (Windows Sharepoint Services and Content Management Server) that were hastily merged into one app. As with a lot of immature Microsoft products, they seem to have been more motivated by marketing a powerful app than they were in making it actually functional. Sharepoint 2013 or 2016 will likely be a good product, kind of like Exchange 2007 or SQL Server 2003, but Sharepoint 2007 makes a lot of promises that it doesn't really keep.
Sharepoint's primary structure is a collection of "sites", each with it's own URL, home page, and extensions. Without careful planning, Sharepoint can easily become a junkyard, with function-specific sites littered all over the map. A number of bloggers are pushing a “Sharepoint invites Silos” meme these days. I stop short of blaming Sharepoint – it does what you plan for. But if you don’t plan, or you don't have the buy-in, attention and time commitment of key staff both in and out of IT, then silos are the easiest things for Sharepoint to do.
The database stores documents as database blobs, as opposed to linking to files on disk, threatening the performance of the database and putting the documents at risk of corruption. I don't want to take my org's critical work product and put it in a box that could easily break.
Licensing for use outside of my organization is complicated and expensive. MOSS access requires two or three separate licenses for each user - a Windows Server licence; a Sharepoint License, and, if you're using teh advanced Sharepoint features, an additional license for that fiunctionality. So, if I want to set up a site for our Board, or extend access to key partners or clients, It's going to cost for each one. There's an option to buy an unlimited access license, but, the last time I looked, this was prohibitively expensive even at charity pricing.
Compared to most Open Source portals, Sharepoint's hardware and bandwidth requirements are significantly high. Standard advice is that you will need additional, expensive bandwidth optimizing software in order to make it bearable on a WAN. For good performance on a modest installation, you'll need at least two powerful servers, one for SQL Server and one for Sharepoint; for larger installations, a server farm.
I can't help but contrast this with the far more manageable and affordable alternatives, even if those alternatives aren't the kitchen sink that Sharepoint is. Going with a non-Microsoft portal, I might lose all of that out of the box integration with my MS network, but I would jettison the complexity, demanding resources, and potential for confusion and site sprawl. I'm not saying that any portal/intranet/knowledge management system can succeed without cross-departmental planning, but I am saying that the risk of a project being ignored -- particularly if the financial investment was modest, and Sharepoint's not cheap, even if the software can be -- is easier to deal with than a project being fractured but critical.
if my goal is to promote collaboration and integrated work in my organization, using technology that transcends and discourages silos, I'm much better off with apps like Drupal, KnowledgeTree, Plone, or Salesforce, all of which do big pieces of what Sharepoint does, but require supplemental applications to match Sharepoint's smorgasbord of functionality, but are much less complicated and expensive to deploy.
After four years of agonizing on this, here's my conclusion: When the product matures, if I have organizational buy-in and interest; a large hardware budget; a high-performance Wide Area Network, and a budget for consulting, Sharepoint will be a great way to go. Under the conditions that I have today -- some organizational buy-in; modest budget for servers and no budget for consulting; a decent network, but other priorities for the bandwidth, such as VOIP and video -- I'd be much better served with the alternatives.
I recently wrote about the potential for shared outcome reporting among nonprofits and the formidable challenges to getting there. This topic hits a chord for those of us who believe strongly that proper collection, sharing and analysis of the data that represents our work can significantly improve our performance and impact.
Shared outcome reporting allows an organization to both benchmark their effectiveness with peers, and learn from each others' successful and failed strategies. If your most effective method of analyzing effectiveness is year to year comparisons, you're only measuring a portion of the elephant. You don't practice your work in a vacuum; why analyze it in one?
But, as I wrote, for many, the investment in sharing outcomes is a hard sell. Getting there requires committing scarce time, labor and resources to the development of the metrics, collection of data, and input; trust and competence in the technology; and partnering with our peers, who, in many cases, are also our competitors. And, in conditions where just keeping up with the established outcome reporting required for grant compliance is one of our greater challenges, envisioning diving into broader data collection, management and integration projects looks very hard to justify.
So let's take a broader look this time at the justifications, rather than the challenges.
Success Measures is a social enterprise in DC that provides tools and consulting to organizations that want to evaluate their programs and services and use the resulting data. From their website:
Success Measures®, a social enterprise at NeighborWorks® America is an innovative participatory outcome evaluation approach that engages community stakeholders in the evaluation process and equips them with the tools they need to document outcomes, measure impact and inform change.
To accomplish this, in 2000, they set up an online repository of surveying and evaluation tools that can be customized by the participant to meet their needs. After determining what it is that they want to measure, participants work with their constituencies to gather baseline data. Acting on that data, they can refine their programs and address needs, then, a year or two later, use the same set of tools to re-survey and learn from the comparative data. Success Measures supplements the tools collection with training, coaching, and consulting to insure that their participants are fully capable of benefiting from their services. And, with permission, they provide cross-client metrics; the shared outcomes reporting that we're talking about.
The tools work on sets of indicators, and they provide pre-defined sets of indicators as well as allowing for custom items. The existing sets cover common areas: Affordable housing; community building; economic development; race, class and community. Sets currently under development include green building/sustainable communities; community stabilization; measuring outcomes of asset programs; and measuring value of intermediary services.
Note that this resources nonprofits on both sides of the equation -- they not only provide the shared metrics and accompanying insight into effective strategies for organizations that do what you do; they also provide the tools. This addresses one of the primary challenges, which is that most nonprofits don't have the skills and staff required simply to create the surveying tools.
Once I understood what Success Measures was offering, my big question was, "how did you get any clients?" They had good answers. They actually engage more with the funders than the nonprofits, selling the foundations on the value of the data, and then sending them to their grantees with the recommendation. This does two important things:
First, it provides a clear incentive to the nonprofits. The funders aren't just saying "prove that you're effective"; they're saying "here's a way that you can quantify your success. The funding will follow.
Second, it provides a standardized reporting structure -- with pre-developed tools and support -- to the nonprofits. In my experience, having worked for an organization with multiple city, state and federal grants and funded programs, keeping up with the diverse requirements of each funding agency was an administrative nightmare.
So, if the value of comparative, cross-sector metrics isn't reason enough to justify it, maybe the value of pre-built data collection tools is. Or, maybe the value of standardized reporting for multiple funding sources has a clear cost benefit attached. Or, maybe you'd appreciate a relationship with your funders that truly rewards you with grants based on your effectiveness. Success Measures has a model for all of the above.
I'm wrapping up the Drupal 101 series with some talk about Drupal themes, and some additional info on topics that we've already covered. The goal of these posts is to give new Drupal administrators an idea about how Drupal works, and some pointers to the key add-ons and resources in the broad Drupal ecosystem. For reference' sake, we started with 
Subscribe through Feedburner
