The information in this article is current as of April 7, 2004.
Most nonprofits providing human services get financial support from some formal funding body such as government, a foundation or a centralized fundraiser like the United Way. All of these funders are beginning to demand evidence that their financial support is worthwhile. Each one of them has a different set of priorities (such as effectiveness or community impact or cost efficiency), but most of them are demanding more data than ever before.
The effect of this pressure isn’t entirely positive:
- Very few agencies have a good management information process that tracks service delivery. Almost none have an information system that evaluates the effectiveness of their services. As a result, data quality is generally poor or nonexistent. Funders shouldn’t rely on the data for decision-making, and in fact the most sophisticated and truthful agencies may be penalized for submitting data based on reality rather than crude estimates.
- Every funder has different priorities and asks different questions. Even when there are overlaps of information (like annual revenue or agency address), every funding form is different. The increased demand for data is creating huge administrative burdens on agencies. Data collection and reporting is cutting into service delivery time without necessarily leading to improvements in service quality.
- Few funders know how to handle the data they do collect. Many funders collect data that they think might be important, but don’t use the information to make better decisions. The cost of collecting and analyzing data is not acknowledged, so a whole lot of useless information is collected and filed. Many agencies are developing management information systems that collect client data in an effort to meet funder requirements. This is a good thing, in principle. It is also a good thing that funders are determined to increase the effectiveness of the services they support. However, if not well designed, information and evaluation systems can cause more problems than they solve.
Deciding what data to collect
Before you develop a data reporting system, you need to figure out what data you should be collecting. Here are a few tips:
- Start out by ignoring funder requirements (for a while), and just define the information that will be most helpful in carrying out your agency’s mission. I like the vocabulary of key performance indicators, which involves defining the most important indicators that tell you whether you are doing a good job, and that will give you the feedback that will enable you to improve. Other people use balanced scorecards or program logic models for the same process. There’s a lot of evaluation jargon that makes this whole area very complicated and difficult to understand, but basically, you have to figure out what you are trying to achieve, what success looks like, how you tell the difference between success and failure in the short term, and which information will be the most helpful in keeping you on track.
- As a second step, define the data that your funders require. This is not always the same as the data that will make you more effective. For example, your funder may want to know how many individual clients you serve, even though you primarily do community development or advocacy. Or you may need to collect data on your human resources to manage your agency effectively, but funders don’t need that kind of detail. Try to define the funders’ requirements as broadly as possible, so that one data element can meet the needs of many different funders. Some funders are extremely legalistic in their definitions of acceptable data elements. Those legalistic definitions may be good enough to use as data elements for other funders that aren’t so picky. Or you may be able to negotiate the definitions with funders if their data requests are unreasonable.
- Figure out the simplest way to define and collect both kinds of data – funder requirements and key performance indicators. For example, do not send out surveys to all of your clients if you are just trying to collect information on service quality. Think about how you can design a suggestion process and a complaints resolution process to collect and resolve relevant information about your services. Supplement it with phone interviews with a small sample of your clients. Don’t create a client management system that tries to collect everything you might possibly need – it’s way too expensive and will not lead to better services unless you have built in processes to analyse and act on the resulting information.
- Design a process that allows you to collect and analyse your data while minimizing administrative costs. For example, say that you need to report various bits of financial data to most of your funders. Instead of manually copying numbers from your annual report, perhaps you can automatically export numbers from a financial database into report templates for each of your major funders. This step generally involves going through a process improvement exercise. It’s a common mistake for agencies to create an information system by designing a database that replicates all of the bad, inefficient processes that they already use. The design of a management system offers an opportunity to redesign the way that the agency collects information, to dramatically reduce errors, and to increase the quality of the data, and reduce the cost of administration.
- Finally, you can design a management information/data tracking system that will make sense and actually help you improve services. If you haven’t gone through the steps above, you will almost certainly end up with a system that is far more expensive and far less useful than it should be. And by the way, this doesn’t have to be a big expensive software development project. You can have a beautiful, effective information system using a simple spreadsheet or even paper forms; technology is just a way to facilitate processes. A database can be very useful, but only if you’re collecting the right information in the right way. RealWorld Systems is currently involved in helping funders and agencies identify key performance indicators to guide data collection. We are hoping that over the next few years, funders will develop a common data dictionary and a common data protocol that will dramatically reduce administrative time in collecting and reporting data. But that’s another article.
Process improvement
Here are some comments on process improvement from Lori Criss Powers, an expert in this area who is also my business partner at RealWorld Systems. Her writing is rather more technical than mine, so brace yourselves. I quote:
Don’t assume you will have massive efficiency gains when you develop your new client management system unless you undertake a process improvement exercise while you implement. If you don’t, your actual efficiency gains may be small because the technical solution was superimposed on top of manual processes and/or because process redundancies were never eliminated (e.g., people continue to photocopy and route paper documents from desk to desk even though an electronic version of the data is available because they are most comfortable using paper). Sometimes barriers to efficiency are not obvious and a quick process analysis does not uncover them. Sometimes problems are identified but are ignored under the pressure to implement a solution. All this may mean that people end up actually adding to their workloads to accommodate the technology.
It is quite common for an organization undertaking such an initiative to suffer from an ‘information junkie’ condition in which unnecessary data is collected (defined as data that neither leads to improved services nor meets core accountability requirements). Unnecessary data costs a great deal of time and expense to collect, store and analyse, and affects staff morale because data collection is so unpleasant. It also decreases data quality because unnecessary data lacks legitimacy in the organization, and staff have little incentive to ensure that it is accurate. Conversely, without clarity on data requirements, essential information may not be collected because people aren’t exactly sure what is important and what is not.
A process is a series of interrelated activities which bring about a result or which are directed toward a particular aim. Process improvement is the analysis and redesign of processes to eliminate organizational problems and inefficiencies in small increments, over time, by improving one or two processes at a time.
Process improvement is done on an operational level (as opposed to radical re-engineering which is done on a strategic level) and is carried out primarily by the people most involved in the process.
The incremental process improvement approach builds in small successes that motivate teams to continue. Failure, if it occurs, has less potential to do serious damage because scope is limited to one or two processes at a time. Incremental process improvement can be a good way to realize (sometimes substantial) benefits in quality and organizational performance while bringing people together over a common goal. It can stimulate morale and can inspire employees to look for innovative ways to deal with challenges.
In future articles, we may get further into specifics on process improvement and the development of useful client management/information systems. Please send us questions if you’d like us to address any particular issues.
Gillian Kerr, Ph.D., C.Psych. and Lori Powers, RealWorld Systems
gkerr or lpowers at realworldsystems.net
Read my weblog at http://blog.realworldsystems.net
