Databases

We're at information overload. We gather more data than we can possibly analyse. We create and build more relationships than we can possibly manage. We're continually asked to share information, to process and summarise. And we expect ICT to deal with this because we, as humble human beings, can't.  And therein lies the database dilemma. Computers like repetitive tasks and can process massive amounts of data very quickly. The problem is the data needs to be in the right format in the first place.  In Spring 2005, preponderate.network conducted a short survey around the use of databases and information management in the voluntary and community sector. The 270 respondents ranged from top 100 charities (by turnover) to small community groups.

Reporting Data

In order for an organisation to work effectively, the senior management need to know what is going on at a particular point in time and an effective information management system can enable this. Many of the larger charities have built their own systems, often at considerable time and expense, and have them supported by in-house ICT. Leading 'off the shelf' products such as Raiser's Edge, Progress and ThankQ can provide some, but not all, of the information management requirements. Sometimes, the more complicated the product, the more it forces users to work the way it does, rather than the way the user would prefer to. Senior management don't always have a headline view of what is going on and that can only cause problems when it comes to operational and future strategic management.

Data (processed into 'information') must be reported to funders and the general public. The lack of standard data categories across the sector and the continual changes imposed by funders means that data input doesn't always enable the data which needs to come out. Many organisations are driven by managing outcomes and outputs needs without effectively considering inputs. Over 53% of respondents to the survey said their system doesn't have the required functionality and 48% said it didn't provide the right reports. Although these issues are often fixable with vendor support, costs are a significant factor in why they don't get addressed. The cost-benefit equation (how much money are we wasting in staff time/effort with a system which doesn't quite work against how much would we need to invest to resolve the problem) doesn't always appear to be considered.

Over 91% of respondents cited reporting statistics as a key database need with 72% of the same group needing to measure outcomes. 55% wanted different levels of access to information for different staff, a problem which largely affected the less expensive database and information management systems. Contacts management (85%) and tracking activities with contacts (e.g. relationship management, 83%) were other major needs expressed.

Data Sharing

All organisations have a need to share data internally -the demand to share externally varies depending on activities and working relationships.

In terms of types of data, 80% mentioned sharing contacts information (information about people and organisations) whilst activities rated a 51% response.
47.5% of organisations shared data with other charities on a variable basis whilst 34% shared it with specific organisations on a regular basis. An inability to share information was quoted by 23% of respondents as a key failing with their existing database.

A key concern raised here was data ownership and security. A number of respondents suggested security was a problem with their existing system (this did not seem to be directed at the top vendors). Data protection rules are strict and transferring data between organisations is complex from a legal standpoint. However, two parts of the same (or affiliated) organisation, using different databases, often find it difficult to share information about people or other organisations as the formats (e.g. data category) will be different and there are no data sharing protocols in place. Data sharing is often insecure, particularly when done over the web. A number of advice organisations are looking at this (in order to better serve clients) and trying to address joint working procedures.

Data sharing also introduces the problem of defining the 'data owner' (who owns the data for data protection purposes and who keeps it up to date). It has not historically been in vendor interests to simply share information with other databases and systems (merely to enable easy one-off importing of data into their system) but there are now cases to be made for making easy interchange of data a core feature of the database and relationship management applications.

Data Categories

The use (or lack of use) of coherent data category structures within the voluntary sector is a problem. If you can't structure data you want to share and can't segment it rationally, how are you, or anyone else, going to make sense of it. Some large organisations are still using categories of ethnicity which only include a few simple criteria e.g. White, Black, Asian, Chinese, Other (typically for historical reasons and because it still works for them). The Commission for Racial Equality applies sixteen criteria for England and Wales but only fourteen for Scotland and these are slightly different from the England and Wales criteria. Confused? You should be!

Anyone reporting to a funder (or internally within an organisation) will usually have to include ethnic breakdowns and these will be derived from a database. The question is: whose categorisation do you use, why, and how does that fit with everyone else's criteria? If, all of a sudden, you want to add Scottish to your ethnic monitoring, how do you determine which individuals in the 'White' category see themselves as Scottish if you've never asked them. Many organisations simply can't quantify the number of Bangladeshi people they've helped because they're only ever listed Bangladeshi individuals as Asian. Much of the work around metadata and data standards is being driven by the public sector (including e-GIF ) but there appears little focus on the needs of charities and the community in terms of data sharing.
When asked "Would you be interested in a single, coherent information category structure for the voluntary sector?", only 10% weren't interested and a cynical 8.4% said 'It will never happen'. If it doesn't, it will mean more work and more confusion for anyone handling data in the future.

The Real Issue

The main value of a database to an organisation has little to do with ICT and everything to do with strategy, organisational activity and information management. Databases are simple or complex conduits to managing and extracting information and their biggest problems arise when they are poorly specified (poor requirements specification from the user), badly implemented and fail to take a holistic organisational view, never mind the issue about filling them with unverified data. A database will never be a panacea and can only work within the framework imposed by its users, designers and the external environment of the data.

We have seen too little evidence of databases and information management applied as a critical part of an organisational business plan (we would be very happy to be proved wrong on this!) yet without this, the true value of a database is limited and they will continue to cause problems, reduce morale and make reporting and data sharing more difficult than it needs to be.

A Call to Action

Charities are collecting vast amounts of data and generating a whole lot more. Some of this is used for internal (to management) and external (to funders) reporting and other for operational activity (who can we fundraise from, what is X doing, what are the details of our relationship with Y).

Many of the database systems available 'off the shelf' deal with most but not all of the needs of a 'typical' organisation. Some charities are spending resources building bespoke systems and not getting the best out of them because they haven't, according to our field research, spent enough time deciding what they want to do with data (input, output and usage) and how they carry out their operational activities in relation to a database. Optimal and effective data and information management has the potential to revolutionise a charity's performance but more time and effort needs to be invested in business analysis, specifying systems and understanding the operations and needs of particular departments and fitting this with the business plan and the ICT strategy. The implementation of a database and information management procedures is too often rushed or done at minimal cost and the present and future pains are there for all to see.

More work needs to be done, and to be led nationally, on creating a single coherent data category structure for the voluntary sector. For too long, funders have caused chaos by asking charities to report in ways contrary to how they collect and manage data. A simple but detailed core data category structure would do a great deal to address this.

Dr Simon N Davey
Managing Associate
Preponderate.network
June 2005

More Articles