Oz Analytics

Actionable Information Governance Part 2

Following up on my recent blog discussing the 26 capabilities that underpin Actionable Information Governance, I want to talk briefly about some organisational ideas that can help you deliver these capabilities more effectively.

Once again, what I present is based on my experience and observation of what works with my practitioner colleagues. These ideas are based on my belief that leading the use of analytics in the organisation needs to be the business itself and that this leadership must be under the support of at least 2 C level executives. Finally, there should be a Centre of Excellence and two communities of practice:

Business Organisation

The Data and Analytic communities are your data and reporting stewards. They are the people responsible in your organisation for data quality and delivering analytic insight to your decision makers. These roles may be part- or full-time, and they may form real or virtual communities. They do however, have to be practitioners.

The Data and Analytic communities of practice are each responsible for delivering the following capabilities:

• Access
• Monitor and Analyse
• Discover and Explore
• Quality
• Classify Data
• Metadata Management
• Data Profiling.

As you will see below, some of these capabilities are jointly managed and delivered. This is because modern analytics often involves a technology component that is complex and requires multiple skills, some of which are unlikely to be available within the communities of practice.

The third business group are the hard-core of your analytic specialists. Gartner and others refer to this as the business intelligence competency center, or BICC for short. The term is too 'nerdy' for the organisations I have worked with, but regardless of what you call it, the capabilities it must deliver are pretty well agreed on by analytic practitioners.

These specialists are there to empower the communities of practice, and coordinate and optimise the IT efforts made in support of these communities. This centre of excellence is responsible for delivering the following capabilities:

• IM Strategy
• BI Tool Evaluation and Selection
• Content Management
• Collaboration and Change
• Implementation and Training.

The final (and vital) role is for the senior executive group and this is to own and exercise governance of information.

IT may not lead analytics under this model but they do have vital roles that underpin the business effort. There are 3 main groups for this, under the direction of the CIO:

IT Organisation

The Enterprise Architects deliver the technology blueprint:

• Information Architecture
• Application Interfaces
• Data Warehouse Design.

The Application Developers build and enhance the major technology components of the analytic platform:

• IM Application Development
• Data Profiling
• Project Management.

The emphasis here is on the technology. The business communities of practice are responsible for agreeing and delivering standard analytic practices. These are the analytic methods employed to deliver insight.

For example, it is common for organisations to model customer acquisition and attrition behaviours. Modelling can be done through a variety of techniques and processes but if you want to be able to reliably compare models (i.e. compare different areas of the organisation), then you need to be consistent in how you define and treat the data.

Sharing standard analytic practices encourages trust in your insights and confidence in the subsequent decisions made.

Finally, the Infrastructure group keeps the data tuned to optimum performance and ensures that the platform is available when the business requires it:

• Database Administration
• Operations Management.

So there are the capabilities and responsibilities. Many are shared by different groups and it can be complex to manage. Here is one way to do it:

Putting It All Together

As I have mentioned previously, analytics is a complex function and in the ideal world we would all have large teams of people to deliver these capabilities. Reality is often more challenging and we never reach the ideal. Dealing with this was the subject of a previous post "How Many Analysts Does It Take To Screw In A Lightbulb?"

In the coming days I will round off my thoughts on information governance by talking about the organisational processes needed to provide effective support.

Data Quality 2011 Asia Pacific Congress

I was at this congress and attendance is up this year - with standing room only when I came into the room at 10 am. Great news for local DQ practitioners. Even better, the audience was really engaged and there have been active and interesting Q&A sessions.

The website for the Data Quality 2011 Asia Pacific Congress is here http://www.dqasiapacific.com/.

The Future Of Business Intelligence

Keynote address to the Next Generation Data Warehousing Conference, IDG Korea , Seoul, South Korea, February 16th 2011

I presented my thoughts to a couple of hundred Senior Managers from a wide range of Korean organisations. Crystal ball gazing is something I usually leave to vendors and consultants so I focussed on the following aspects that I at least have some practical experience of:

• Key trends to the end of the decade
• Best practices
• Organising the future intelligence capability.

The following key trends struck me as important factors influencing the future of our work in analytics:

• An Unprecedented Explosion In Data. “Between now and 2020, the amount of digital information created and replicated in the world will grow to an almost inconceivable 35 trillion gigabytes as all major forms of media”
  IDC iView, "The Digital Universe Decade – Are You Ready?" May 2010
• The Dominance of Mobile Multimedia Content. “Global mobile data traffic will increase 26-fold between 2010 and 2015 ... Mobile network connection speeds will increase 10-fold by 2015 ... Two-thirds of the world’s mobile data traffic will be video by 2015.”
  Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2010–2015, February 2011
• The End Of English. Chinese-language content will dominate the internet by 2014
  Gartner 2009.

I'm not completely convinced about Chinese domination - but I agree that the Internet will become strongly multi-lingual as the rise of China and Brazil continues.  In any event, we all now live in a world of BIG data. Data that is growing annually by orders of magnitude and is moving to mobile/embedded devices that generate real-time multimedia data. On the other hand, IT skilled people - those with the skills most likely to handle data growth - are only predicted to grow 1.4 fold over the same period. So there is an emerging massive skills shortage coming and we have four choices:

• Do nothing. You wouldn't be alone if this is your strategy.
• Focus your current analysts on embedding their skills in a broad range of your staff.
• Try to buy the skills by poaching staff from other organisations. This could get to be very expensive as you will likely to be in a bidding war with other organisations facing a similar skill shortage.
• Use software that enables your analysts to handle vastly more data than they can today.

Just to complicate things, the era of the English/Western Internet is being supplanted by Chinese - a language that is impervious to processing by most of the current and emerging intelligence capabilities out of the US.

Personally, I see as our best hope a strategy that focusses on embedding analytic skills into the general business community. This greatly enhances analytics usefulness in the short term, whether-or-not you believe in the above trends. It will also buy us the time to see smarter software emerge that does the 'heavy lifting' of BIG (Chinese?) data for analysts. The analogy I used in my address was transportation and the London underground network. The actual network looks like the following:

It took Harry Beck in 1931 to analyse the London underground network and realise that the actual physical location of the station was less important than the information about how you can get to your destination.

This is exactly what analytics needs to achieve. I like to call this actionable insight.

If you have any doubts about the explosion in data we can expect, take a look a recent years. The Washington Post has created a created a graphic that shows how data grew from 1996 and 2007:

After looking at the impact of these trends, I went on to discuss templates for both the analytic skills we need and how to best to organise your teams. I have posted about these topics previously here.

After the presentation there were questions from the floor and it was interesting to see that the questions could have come from any country. The problems and challenges discussed were exactly the same as I hear in other countries. It's nice to know the we are all in the same boat.

 

Big. Really big

"Information Management is big. Really big. You just won't believe how vastly hugely mind-bogglingly big it is. I mean, you may think it's a long way down the road to the chemist, but that's just peanuts to IM."

What Douglas Adams would have said if he was in my position.

 My role includes the words Information Manager in the title and that's where the confusion begins. Only 1 person in 100 knows what it means. A further 10 in 100 think that they know what it means - but when pressed for an explanation you will get at least 11 different interpretations.

It's hardly surprising that the wider business community is unsure about what IM is. So just to set the record straight, here's what I believe IM is:

• Data quality – to create high quality information.
• Metadata management – to clearly define all elements.
• Data integration – to bring the data together for all management processes to use.
• Analytics – processes data to reveal hidden meaning.
• Business Intelligence – delivers insight to the people and processes making decisions.
• Performance Management – uses consistent and uniform measures of core entities (departments, staff, customers, products, accounts, etc.) to optimise operations.

 

dataFUD

 

Fear, Uncertainty and Doubting your data is a feature of organisations who are failing to effectively manage their information resources. As a result, it is one of the main issues to address when building an information management strategy for a dataFUD organisation. How do you know if your organisation is failing? Well, if you're reading this blog then there is a good chance you already know, but just in case, here are some things to think about. Do you or others in your organisation spend a lot of time:

• Performing work incorrectly or making a wrong decision?
• Re-doing work because it was previously performed incorrectly?
• Recovering from the impact of making a wrong decision?
• Taking unnecessary time to investigate the integrity of the data before using it?
• Performing calculations or reformatting the data before it can be used?
• Hunting for additional information in order to use the data?
• Losing customers because it caused work to be performed incorrectly?
• Causing unrecoverable damage?
• Missing business opportunities?
• Miscommunication within the business or with end customers and other information stakeholders?

If any of these ring true, then there is a good chance that you need to manage information better. How will this help? I've identified 7 main benefits that an organisation can realise by managing information better. They are:

1. Improved Data Quality – Poor information management leads to poor data quality and subsequently poorer business decisions. Early indicators of the business costs of non-quality data (listed in the bullet list above) point to potentially foregoing revenue as high as 10-20% of the revenue or total budget of the organisation.
2. Elimination of Duplicate Work – Removing redundancies, reducing the need for rework and therefore reducing operational costs by repairing data quality issues that create the need for manual reconciliation and avoid futile marketing and sales expenditure. On its own, by halving the costs related to poor data quality, a 5-10% saving on operating expenses can potentially be achieved.
3. Unified Vision – or Single Source Of the Truth. Organisations that break the barriers between their silos of operations can unlock value by obtaining unified views of business-critical data like customers and products, as well as apply existing information in new value adding ways such as opportunity identification and cross selling opportunities. This will improves the ability to coordinate activities between lines of business.
4. Broader insights and Better Decisions – into data across products and business units. Decision makers gain a deeper understanding of operations and an accurate picture of past and present performance as well as predictions of future performance that shape decisions.
5. Support More Complex Operations – The ability to serve numerous users with varying requirements at a lower cost. Users include employees from different areas within the organisation, external suppliers, partners and any other party that may interact with the company.
6. Continuous workflow – Continuous workflow with timely information available. This stops people having to wait for information that they require to perform their work and make decisions. Delay can cause lost opportunities or added costs. A well designed information management capability will reduce the lead time an organisation requires to go to market with something new.
7. Compliance – Aids compliance to legislation such as Sarbanes-Oxley as knowing where to find the information required will decrease the risk of non-compliance and allow for greater responsibility.

These benefits are pretty fundamental to the competitiveness and sustainability of business in the 21st century. Just remember that although information management may often be talked about in technological terms, the true key to improving performance is found by up-skilling staff so that they are able to take advantage of your technology investments. There is a reason that almost all management and information experts are talking about competing on analytics. My favourite response? A fool with a tool is still a fool - it doesn't matter how good the data quality.

 

As always with my blogs, I may be repeating ideas from one of the many books I have read, websites I visited or methods I use. I stand on the shoulders of giants and acknowledge their original thoughts when I can. I haven't checked if I am directly quoting from them, but the Mike2.0 community has been an inspiration in recent years so you may well find their ideas mixed with mine. As always, any mistakes or controversies are my responsibility alone.

Local Intelligence Beta Testers Wanted

As most of my readers know by now, I and a small group of like-minded people are building an online community for business intelligence, information management, data warehousing, analytics and performance management enthusiasts in Australia and New Zealand. Called the CORTEX for short.

We launched officially this year and membership is approaching 700 people. It's great to know that so many other people like the concept. Everyone is welcome and membership is free. OK, that's the end of the PR bull.

We are trying to regularly add new features and in the spirit of Google Labs we have just launched a one-click search function that scours the CORTEX for news and views on specific leading ANZ organisations. Currently we have the following industries covered:

• Construction and Property
• Financial Services
• Government, Health and Education
• Manufacturing and Distribution
• Publishing and Media
• Retailers
• Rural, Minining, Oil and Gas
• Telecommunications
• Tourism and Hospitality
• Transportation
• Utilities

I know that we have missed several organisations and I ask anyone who is active in these (or other) industries to:

• Beta-test the search function
• Suggest other organisations we should include
• Add news to the CORTEX local industry channels if you know of something that we are missing.

All comments and criticisms are welcome!

Leadership Lessons in Data Quality - Part 2

In my last blog (Leadership Lessons in Data Quality - Part 1) I talked about some useful techniques that I have learned by delivering improved data quality in a number of Australian organisations. In this post I want to talk about how to help business people to care more about DQ.

This is vital because it is the key to making good data quality sustainable in your organisation.

The short answer - like a lot of analytic challenges faced in the real world - is to measure the problem. 

Once you can measure data quality, as I mentioned in my earlier post:  I also recommend that data quality improvement of the business owners data be made a part of their performance incentive program. In simpler language: link it to their bonus.

Data quality has to be measured like any other KPI so that non-experts can understand two things:

1. What we all mean by poor data quality - define it
2. The cost of poor data quality - quantify it.

This is easiest to see in an example:

_______________

Don't Set as a KPI:

Currently 6,753 customer records, or 2.0834% of all main customer records in EDW2 contain errors. Our objective is to reduce this by 50% within 6 months.

Do Set as a KPI:

Currently almost 7,000 clients have incorrect data that results an average of 2,800 direct mailings being returned by Australia Post. This adds $26,000 directly (printing and posting) and indirectly (administration) to each marketing campaign. Our objective is to reduce this average cost by 50% within 6 months.

_______________

A useful technique I have developed to help me understand data and quality is to create a taxonomy that classifies data in the following three ways:

• 'Classic' data quality criteria. Things like accuracy, relevance, availability, etc. 
• The impact data quality has on the business. Things like dollars lost if data is wrong, the value of increased sales if data is correct, etc.
• What action(s) they initiate in an organisation when the data is wrong. Things like intervening at the source of data entry, remedial actions after data entry, etc.

So what metrics can you associate with data quality in your organisation? Here are six I use:

1. Accuracy: the degree of confidence that data is free of error/defect
2. Completeness: the extent to which data is not missing and is of sufficient breadth and depth for the task at hand
3. Consistency: the degree to which common data across different sources follows the same definitions, codes and formats
4. Timeliness: the degree to which data is up to date
5. Security: the degree to which data confidentiality, integrity and availability has been maintained
6. Fit for Purpose: the degree to which data is relevant, appropriate and meets business specifications.

Leadership Lessons in Data Quality - Part 1

I am a great proponent of data quality investments and this presents a great challenge to me when I am trying to create a new information management capability in an organisation. Overcoming this challenge is essential to developing a valued analytic capability. This is what I want to talk about in my next couple of posts.

It's a challenge because usually very few other Senior Managers / C-level Executives appreciate the value of good data. Intellectually, most usually agree with me, but when priorities are being set, data quality almost invariably gets pushed down the priority list by more 'burning' issues. Issues where Managers see a more immediate impact on the bottom line.

The only exception to this is when a serious compliance issue has occurred or there is a serious risk of one occurring if nothing is done to prevent it. Even in these circumstances, the decision is usually made to invest in a short term and usually one-time-only 'fix' rather than investing more to make a solution that continues to work over time or even goes back and fixes the underlying source of the poor data quality (often back in a source system or operational process).

So the challenge when constructing a program of work to deliver a new analytic capability is to build into the solution an appropriate and sustainable level of data quality.

Couldn't be easier, right?

Remember - the Board is not going to appreciate explicit data quality goals unless they think that they will not delay other (and in their minds more valued) goals. In other words they want it for free.

Tricky to do but here are some actions that I have successfully used in multiple large projects/programs:

Establish Business Ownership

If you can't find the formal business owner of the data then you can't fix the quality issue in a sustainable way. It may be the case that your work delivering a new data warehouse or other data store may solve today's quality issue - but if no one in the business owns the data then the quality will inevitably decline over time. This will usually happen a lot faster than anyone predicts.

The more usual situation is that the quality issue remains and everyone continues to work around it. Often the dashboard, analytic reports, or data extracts you deliver from your shiny new platform will also have one of two of the following consequences:

• the data quality issue becomes masked so that people look at the shiny new analytics and forget that it is still based on the same lousy data, and/or
• users continue to not trust the numbers, and/or
• areas of analysis that require the poor quality data are excluded from the analytic platform - making your investment much less valuable.

All of these are negative consequences that can seriously devalue you and your teams hard work.

The best business owners are those who are directly and negatively affected when quality declines. They are also normally (but not always) the ones who most benefit from improved data quality as well. Why are they the best business owners? Because they will care the most.

Once the correct owner(s) are agreed with the business, I also recommend that data quality improvement of their data be made a measured part of their performance incentive program - but more on this in my next post.

Manage Data As An Asset

Manage your data assets just like you do capital and equipment. If it is worth something then at the very least you need to manage it to protect that value.

Would you leave a valuable printing press or lathe outside to rust? Data is the same. If you don't look after data, it will decline over time just as capital equipment depreciates over time.

The best data asset managers are your senior information management experts. Why? Because they care deeply about data quality and should already be closely engaged with the business and IT owners of data.

Link Cause To Effect

Explicitly link data quality issues to known problems that need to be solved. If 1% of your customer records have invalid details, who cares? Why do they care? And most importantly: how much do they care?

If you can identify the department suffering most, then go to that Manager and work with them to quantify the dollar cost. Agree with them also about the importance of the resulting problem(s). If it is important and causing the company to lose sufficient dollars then can a solution be devised that will save money and deliver this benefit in a reasonable timeframe? Reasonable in my experience is almost always within 6 months.

Sidebar: The Value Of Program Management

If you structure the delivery of a program of work to spread delivery evenly throughout the life of the program, you can often buy yourself up to 12 - 15 months of time to solve complex data quality issues. Again, this is a much wider topic than data quality that I will leave to another post.

Work Closely With IT

Business ownership is great - but most of the data the business cares about is electronic. If IT can't help deliver the solution then there is little hope that the beautiful analytic platform you build will operate to maximum efficiency over time.

IT has to be a full and active partner in any data quality initiative. Even if initially it is only to educate IT and to develop their own information management capabilities. In my experience, many IT people will enthusiastically take on IM/DQ roles when they know that people in the business care.

If IT doesn't care about data quality, then at least some of the blame for this sits with the business leaders. In data quality, just like most areas of modern business, IT and business managers are very dependent on each other. The safest way to ensure a successful analytic outcome is close cooperation.