Oz Analytics

Leadership Lessons in Data Quality - Part 2

In my last blog (Leadership Lessons in Data Quality - Part 1) I talked about some useful techniques that I have learned by delivering improved data quality in a number of Australian organisations. In this post I want to talk about how to help business people to care more about DQ.

This is vital because it is the key to making good data quality sustainable in your organisation.

The short answer - like a lot of analytic challenges faced in the real world - is to measure the problem. 

Once you can measure data quality, as I mentioned in my earlier post:  I also recommend that data quality improvement of the business owners data be made a part of their performance incentive program. In simpler language: link it to their bonus.

Data quality has to be measured like any other KPI so that non-experts can understand two things:

1. What we all mean by poor data quality - define it
2. The cost of poor data quality - quantify it.

This is easiest to see in an example:

_______________

Don't Set as a KPI:

Currently 6,753 customer records, or 2.0834% of all main customer records in EDW2 contain errors. Our objective is to reduce this by 50% within 6 months.

Do Set as a KPI:

Currently almost 7,000 clients have incorrect data that results an average of 2,800 direct mailings being returned by Australia Post. This adds $26,000 directly (printing and posting) and indirectly (administration) to each marketing campaign. Our objective is to reduce this average cost by 50% within 6 months.

_______________

A useful technique I have developed to help me understand data and quality is to create a taxonomy that classifies data in the following three ways:

• 'Classic' data quality criteria. Things like accuracy, relevance, availability, etc. 
• The impact data quality has on the business. Things like dollars lost if data is wrong, the value of increased sales if data is correct, etc.
• What action(s) they initiate in an organisation when the data is wrong. Things like intervening at the source of data entry, remedial actions after data entry, etc.

So what metrics can you associate with data quality in your organisation? Here are six I use:

1. Accuracy: the degree of confidence that data is free of error/defect
2. Completeness: the extent to which data is not missing and is of sufficient breadth and depth for the task at hand
3. Consistency: the degree to which common data across different sources follows the same definitions, codes and formats
4. Timeliness: the degree to which data is up to date
5. Security: the degree to which data confidentiality, integrity and availability has been maintained
6. Fit for Purpose: the degree to which data is relevant, appropriate and meets business specifications.

Leadership Lessons in Data Quality - Part 1

I am a great proponent of data quality investments and this presents a great challenge to me when I am trying to create a new information management capability in an organisation. Overcoming this challenge is essential to developing a valued analytic capability. This is what I want to talk about in my next couple of posts.

It's a challenge because usually very few other Senior Managers / C-level Executives appreciate the value of good data. Intellectually, most usually agree with me, but when priorities are being set, data quality almost invariably gets pushed down the priority list by more 'burning' issues. Issues where Managers see a more immediate impact on the bottom line.

The only exception to this is when a serious compliance issue has occurred or there is a serious risk of one occurring if nothing is done to prevent it. Even in these circumstances, the decision is usually made to invest in a short term and usually one-time-only 'fix' rather than investing more to make a solution that continues to work over time or even goes back and fixes the underlying source of the poor data quality (often back in a source system or operational process).

So the challenge when constructing a program of work to deliver a new analytic capability is to build into the solution an appropriate and sustainable level of data quality.

Couldn't be easier, right?

Remember - the Board is not going to appreciate explicit data quality goals unless they think that they will not delay other (and in their minds more valued) goals. In other words they want it for free.

Tricky to do but here are some actions that I have successfully used in multiple large projects/programs:

Establish Business Ownership

If you can't find the formal business owner of the data then you can't fix the quality issue in a sustainable way. It may be the case that your work delivering a new data warehouse or other data store may solve today's quality issue - but if no one in the business owns the data then the quality will inevitably decline over time. This will usually happen a lot faster than anyone predicts.

The more usual situation is that the quality issue remains and everyone continues to work around it. Often the dashboard, analytic reports, or data extracts you deliver from your shiny new platform will also have one of two of the following consequences:

• the data quality issue becomes masked so that people look at the shiny new analytics and forget that it is still based on the same lousy data, and/or
• users continue to not trust the numbers, and/or
• areas of analysis that require the poor quality data are excluded from the analytic platform - making your investment much less valuable.

All of these are negative consequences that can seriously devalue you and your teams hard work.

The best business owners are those who are directly and negatively affected when quality declines. They are also normally (but not always) the ones who most benefit from improved data quality as well. Why are they the best business owners? Because they will care the most.

Once the correct owner(s) are agreed with the business, I also recommend that data quality improvement of their data be made a measured part of their performance incentive program - but more on this in my next post.

Manage Data As An Asset

Manage your data assets just like you do capital and equipment. If it is worth something then at the very least you need to manage it to protect that value.

Would you leave a valuable printing press or lathe outside to rust? Data is the same. If you don't look after data, it will decline over time just as capital equipment depreciates over time.

The best data asset managers are your senior information management experts. Why? Because they care deeply about data quality and should already be closely engaged with the business and IT owners of data.

Link Cause To Effect

Explicitly link data quality issues to known problems that need to be solved. If 1% of your customer records have invalid details, who cares? Why do they care? And most importantly: how much do they care?

If you can identify the department suffering most, then go to that Manager and work with them to quantify the dollar cost. Agree with them also about the importance of the resulting problem(s). If it is important and causing the company to lose sufficient dollars then can a solution be devised that will save money and deliver this benefit in a reasonable timeframe? Reasonable in my experience is almost always within 6 months.

Sidebar: The Value Of Program Management

If you structure the delivery of a program of work to spread delivery evenly throughout the life of the program, you can often buy yourself up to 12 - 15 months of time to solve complex data quality issues. Again, this is a much wider topic than data quality that I will leave to another post.

Work Closely With IT

Business ownership is great - but most of the data the business cares about is electronic. If IT can't help deliver the solution then there is little hope that the beautiful analytic platform you build will operate to maximum efficiency over time.

IT has to be a full and active partner in any data quality initiative. Even if initially it is only to educate IT and to develop their own information management capabilities. In my experience, many IT people will enthusiastically take on IM/DQ roles when they know that people in the business care.

If IT doesn't care about data quality, then at least some of the blame for this sits with the business leaders. In data quality, just like most areas of modern business, IT and business managers are very dependent on each other. The safest way to ensure a successful analytic outcome is close cooperation.

Death Of The Relational Database

A recent entry by Tony Bain in his excellent 'Innovations in Data Management' blog caught me a little by surprise. In it he talks about the NoSQL movement - the group of people and organisations that say we can do without the RDBMS's from the likes of Oracle, Microsoft and IBM. This is new to me.

In a nutshell, the argument runs as follows: Massively scalable databases exist that are not relational and they power some of the biggest sites on the internet - Amazon, Google and Facebook to name three.

For those who don't know about NoSQL, take a look at ACM blogger Michael Stonebraker or a nice summary from Computerworld. You can't find anything about NoSQL on wikipedia yet - that's how new it is. Stonebraker constructs a convincing argument in favour of 'the death of the RDBMS':

• For data warehouses, a column store beats a row store by approximately a factor of 50 on typical business intelligence queries. The reason is because column stores read only the columns of interest to the query and not all of them. In addition, compression is more effective in a column store. Since the legacy systems are all row stores, they are vulnerable to competition from the newer column stores.
• For online transaction processing (OLTP), a lightweight main memory DBMS beats a row store by a factor of 50. Leveraging main memory and the fact that no DBMS application will send a message to a human user in the middle of a transaction, allows an OLTP DBMS to run transactions to completion with no resource contention or locking overhead. 
• In XML, where the current major vendors have spent a great deal of energy extending their engines, it is claimed that specialized engines, such as Mark Logic or Tamino, run circles around the major vendors. 

The argument then runs that if you don't want performance like this, then just get an open source RDBMS for free. I am beginning to agree (even the Cortex is MySQL) and if free is too hard for you, the price of SQLServer is good value for most organisations these days.

So what are the reasons for spending big dollars on an RDBMS? Here are the reasons I can think of for a large company:

• Nobody every got sacked for buying Oracle or DB2.
• The IT specialists have built their career and expertise on a specific vendor's product line. What's in it for them to support a change that they see as undermining that?
• Who's going to hire someone with Voldemort or MongoDB experience?? 
• For most applications, the RDBMS can do the job - so what if the company spends $500,000 more on hardware to do it. Don't forget, the hardware guys are also comfortable with running the big RDBMS on 'their' boxes.
• A surprising number of data warehouse developer's lack the skills to really understand the differences pointed out by the NoSQL people. Besides - they're not the one's paying for the infrastructure they use.
• When something goes wrong with the software - who do you call? Will they be around next year/month? 
• "Our RDBMS solution works today - or it will when we upgrade/implement the new module." So why change?

I am happy to concede that most of these reasons are not technical. Politics is very real in larger enterprises and you don't stay long if you ignore this fact. In the past I have been lucky to hire some very good developers because of their frustration with life in a large corporate data shop.

Anyway, the NoSQL revolutionaries got together recently and you can read/view/listen to the presentations on Johan Oskarsson's site (he is a developer for Last.fm in London). The presentation on the Cassandra database by Avinash Lakshman of Facebook has some interesting stats comparing Cassandra and MySQL for example.

It is curious that the NoSQL people don't call their solutions databases. Instead they are a a "highly available key-value store" (Amazon) and a "distributed storage system for managing structured data" (Google). At least MongoDB does describe itself as "a high-performance, open source, schema-free document-oriented database." Not exactly as snappy a label as RDBMS and I wonder why CDBMS (Columnar RDBMS) isn't good enough.

Check out NoSQL - it could be useful to you.