Oz Analytics

Day 5 Week 4 of the CORTEX MBAnalytics

The end of week 4 and the half way mark! - part of the 60 day information management refresher course from the CORTEX.

The CORTEX

Master of Business Analytics

Week 4, Day 5

Theme: Data Visualisation

Week 4, Friday

This week's theme is data visualisation, or perhaps more accurately, transforming data into insight. Today we have some practical advice for practitioners. 

WATCH    Today's task:  The Clicks Paradigm    (streamed below)

4 minutes

'Lessons in Business Intelligence: The Clicks Paradigm' by Kevin Quinn at Information Builders.

Key takeout: This week we have seen some challenging material and we end with some practical advice from a leading vendor of BI solutions. So what do they offer? Well, a surprisingly handy way of thinking about the audience of your analysis. Try it out for yourself and it might just improve business take-up.

Well done for making it to the halfway mark. It's all downhill from here! As always, comments, questions and ideas are all welcome. 

The MBAnalytics is just one part of the Business Intelligence CORTEX - a community of Business Intelligence, Data Warehousing, Analytics and Project Management professionals in Australia, New Zealand and Asia. Membership is free and the forums are very active with over 20,000 posts offering a uniquely Australasian viewpoint.

The Death of BIG Business intelligence

In the second decade of the 21st century the BI platforms from the SIMO vendors (SAP, IBM, Microsoft and Oracle) are dinosaurs.

A provocative statement - but bear with me.

Over the past few years I've blogged about a lot on the emerging trends of information management: real-time, BIG data, agile BI, actionable performance management and the like. The one thing that they have in common is that they are all big. Either big in terms of expensive or big in terms of the changes that they require within organisations.

I've tried to relate many of these trends to practical actions that (for those of us without large budgets) we can still extract value from. The context of these actions is that I have often been advising medium and large commercial businesses with existing business intelligence operations. So what I have experienced may not necessarily apply to government or small organisations.

Frequently my advice has involved technologies from one or more of the BI platforms from the SIMO vendors. SIMO and cheap are not words that go together. It's not a criticism of the platforms. They are what they are but I argue that their time has past because there are now a number of alternative approaches available that leverage newer low cost products and methods.

I have been increasingly looking to the benefits of open standards and web-based solutions to create low cost platforms without further investment in the SIMO vendor platforms. In the past I have used open source to extend SIMO platforms so that I didn't have to pay for the 'enterprise version' or 'additional module' that the SIMO vendors always seem to suggest when I want my platform to do something new.

An example of this was to extend our web-based dashboards to include sparklines¹. This type of graphic was not supported by the SIMO vendor product at the time but was easily extended using open source. It also saved the company the considerable expense of buying the enterprise dashboard for $250,000 + 20% annual support charge.

Now I have questioned the idea of using the SIMO platform at all. In the above scenario I today have invested in something like Tableau which although very definitely not based on open source code, is significantly cheaper to purchase and more adaptable to a variety of data sources and presentation types.

At this point, I will digress into more technical considerations. So if you become glassy-eyed at the mention of java, then it's best if you skip over the next couple of paragraphs and start reading about the example.

Open standards-based initiatives make our world of data so much more open than it once was. Everyone has heard of Big Data, but there are also dozens of open initiatives such as Mondrian (or more correctly: the Pentaho Analysis Services Community Edition) - an Online Analytical Processing (OLAP) server that enables business users to analyze large quantities of data in real-time.

Why do we care? Mondrian gives us an open standard way to access OLAP using Java on the client side. I'm not referring to XML/A the 'open' standard that Microsoft has been promoting for some years. XML/A is not truly open. Mondrian and Saiku should provide a true open driver capability to those of us building best-of-breed platforms. In time, I hope that market pressures will convince the SIMO vendors to build their own drivers for Mondrian.

A lot of brainpower has been invested over recent years improving database technologies so that they better support business intelligence activities. Combine this with the computing power now available on our desktops and in our data centres and clouds and I think you will find that best-of-breed technology stacks where you decide each layer will often do the job more sustainably and cheaper than an equivalent SIMO investment.

An Example

I can best explain the business logic behind my ideas by taking an example such as business MDM (master data management).

Old School

You purchase an MDM solution from the SIMO vendor you already have in-house. Let's take IBM in this case. So you will need to purchase the Websphere Business Glossary and probably the MetaData Server and Workbench. What's this going to cost? I'm guessing $1 million or more for an enterprise license once you factor in the specialist technical skills you will need to integrate it into your organisation.

IBM's Websphere Platform (courtesy IBM)

Now Websphere is a nice platform and the integration between DataStage, QualityStage, Information Analyzer and Business Glossary is a joy to have as a business manager of this stuff. I've just never had the business case to buy the entire platform. This is true of the majority of IBM platforms implemented across Asia-Pacific today. IBM is no exception and this is true of the other SIMO platforms. Budgets, capabilities and maturity levels almost always mean that only parts of a SIMO platform are implemented in an organisation. What a missed opportunity - but there is no point in crying over spilt milk. If the business case to implement entire platforms existed, then the SIMO vendors would be the first to let you know.

Microsoft is the exception to this rule, having a much cheaper licensing model. Unfortunately their MDM capability is very much a 'you build it' one, so the TCO (total cost of ownership) is still significant.

The challenge today is being smarter and more effective than past attempts. 

New School

Means that you leverage your existing investments - often those in web solutions.

For example, an MDM on-the-cheap solution would be to have your content management (CMS) platform hold critical business metadata. SharePoint is frequently the CMS of choice for large traditional corporations. Even if not already implemented, Using SharePoint to build a custom workflow and collaboration solution is pretty straightforward. So with a modest effort, the CMS can support an intranet-based community of business owners to create and maintain business metadata.

Making use of this business metadata is also possible. Continuing with the MDM scenario above, and assuming you use BusinessObjects, Cognos or similar, then you can export the business metadata easily to universes/cubes so that it is available to report writers, users and decision makers. This will drive the use of standard definitions of data and measures and make enterprise performance management easier.

We are also reaching the point where the presentation layer products (reports, dashboards and scorecards) from the SIMO vendors are now intelligently integrating back into - you guessed it - the CMS's like SharePoint. So production and consumption of business metadata is done on the most widespread platform available within the organisation.

This takes BI and IM out of the analytic ghetto and integrates it seamlessly into the corporate intranet - and beyond.

Cost? A conservative $100,000.

 Conclusions

What's the overall result? You get 50% or more of the benefits at 15% or less of the cost. Ongoing maintenance is also less. Most crucial of all is that the cheaper option is also the one most likely to be funded by an organisation. Why? Simple: it's CHEAPER.

This blog started with the statement that the SIMO vendors were dinosaurs. This is not quite true, but as my example above illustrates, it is now time to integrate your existing SIMO investments into your corporate intranet. Not by buying more SIMO products. Instead use more open technologies to integrate them into the intranet. Treat the SIMO-based data and report repositories as services to be consumed via a web front-end that does not require those extra SIMO products.

I thought about calling this approach 'Poorman's BI' and 'Lagging-edge BI' but I will stick with another term that I have used in the past: 'Slow BI'. It is slow in a good way because I have seen this approach allow an organisation to build a level of support for BI that is solid, broadbased and delivering benefits that the organisation can measure and acknowledge.

Slow BI provides an enduring organisational and technological platform that focuses limited resources on delivering practical business capabilities. These become part of the operational business and decision making processes among people that are only dimly aware of business intelligence and information management. This is an important step towards the elimination of analytic ghettos and the creation of an information workforce.

Slow BI over the long term exceeds the benefits of the SIMO platforms as they frequently struggle to achieve widespread adoption outside of the analytic ghettos. Perhaps slow-and-steady beats the big-bang?

 

 ¹ Sadly Microsoft has recently tried to patent sparklines - despite the wide acknowledgement that Edward Tufte invented and defined sparkline as a term to describe "small, high resolution graphics embedded in a context of words, numbers, images".

Proof I Need A Holiday

OK, I admit it. I am a nerd toy junkie.

If it's new and microprocessor related I want one. Sorry, correction: I need one.

So it will come as no surprise that I own an iPhone and I keep an eye out for applications that do something innovative and analytic. There are now a number of very useful applications that are part of my work day - from BusinessObjects Explorer to Wolfram|Alpha. I have however learnt to be careful and avoid the seemingly vast number of iApps that serve no sane purpose at all. Peer group pressure has in the past done a good job of controlling my trigger-happy finger. I just couldn't resist going to the website for the iPhone Baby Tracker.

Could I now whack a microchip into the little tykes neck just like I did for Virgil our family labrador? Sit back and track their travels from the comfort of my armchair? Never miss another rerun of Gilligan's Island again? Fantastic - I'll take 2! And I wonder if the wife would agree ...

... but no. The reality was a bit of a let-down.

So I just don't get the idea of Trixie Tracker - the dashboard for your baby.

It does what it claims. You record every detail of babies life and it will generate insight. What sort of insight? From their website:

"Four Ways Diaper Tracking Makes Your Life Easier:

• Know the time of the last poopy diaper (and know when it's time to give prunes).
• Keep track of leaks and accidents. Figure out why and when accidents happen.
• Estimate diaper usage so you'll know when it's time to buy more.
• Encourage potty training because you are aware of the schedule."

Do we really need this level of information when we have a kid? I would have thought that this was exactly the period when you didn't have the time to be entering data on your iPhone.

Maybe I'm wrong and in any case, in my day, this would have been a job for Inga, the Swedish au pair.

How did I ever raise 2 children without visualising their nappy changes and bowl movements?:

 

Even assuming I needed this app - and that's a big assumption - what about getting serious about bowel movements? Where's the ability to record the type of movement using indistry standard categorisation by the Bristol Stool Scale? Or being able to include external data sets?

So come on Trixie Telemetry LLC and get serious about industry standard data classification and EDI! I want - no - I need to be able to import data sets from my toddlers pre-school classmates so that I can prove how regular my child's bowel movements are (and correspondingly) how sub-standard your offspring is.

And Trixie, please don't forget about the microchip in the neck idea. Think of the applications. I could tag my entire BI team and track their every movement. That'd redefine the notion of what a time and motion study could do. Forget Porter's 5 Forces, and welcome to the era of Bennett's 7 Movements. 

Seems like I have become that grumpy old man from the BBC TV shows so I'll stop now. Merry Christmas? Bah humbug!

Have a very analytic new year.

Postscipt, January 2010:

My brother (a doctor) was given a children's version of the chart. I can't resist posting it here:

Dashboard Envy

Coca-Cola Germany has published on the web (via their marketing agency) 2 generations of their flash-based dashboards developed between 2007 and 2009. I recommend you take a look at them here and here as I believe they give us a vision of how to present data effectively to sales, marketing and senior management teams.

Some may look at it and think it is just another piece of web 'eye candy'. I think this is wrong. What makes this dashboard so good? Because it is:

• Simple - 3 top level categories of data are used
• Focused - on brand and loyalty
• Interactive - one-click slicing and dicing
• Engaging - animation is used to amuse and inform
• Instant - the information can be understood at a glance.

The picture above and to the left are of the first generation dashboard. Click on each image to see a larger version.

The second generation took the dashboard concept a step further and then globalised it.

The dashboard opens with the greeting 'Good morning world' and then proceeds to present you with views each market. This generation of the dashboard features pull-down menus, mouse-over market statistics by brand, filtering and a choice of export types (excel or screenshot).

Simple and effective:

Second Generation

In other words, it's fun and effective.

How about locally? Can anyone point to a dashboard in Australia or New Zealand that is the equal (or better) of Coca-Cola? Even better: is anyone working on a project now that will achieve this?

Will The Cloud Democratise BI?

I read with interest the announcement that Canadian BI vendor startup Indicee has successfully completed a second round of venture capital funding. In brief, Indicee sells a cloud solution where you upload your data and it works out automatically the relationships - freeing you to concentrate on answering your business questions. Here's a video from Indicee summarising their service:

Indicee Movie

I don't know the terms of the financing agreement but the US$6 million raised is another sign that BI in the cloud is an area that we should all keep a track of.

Why The Cloud?

Here are four reasons:

1. Indicee was founded by Mark Cunningham and Fred Tummonds both originally with Crystal Services - makers of Crystal Reports. Mark was a developer of the original Crystal Reports product - possibly the most ubiquitous BI tool outside of Excel in the market today.

2. Their early seed funding included all of the original founders of Crystal Services.

3. This second round adds Granite Ventures who have also previously invested in companies such as:

• Biz360 - Hosted Analytical Applications
• C-Decisions (dba DecisionView Software) - Analytic Software
• Crowd Science - Online Marketing Research
• Lucid Imagination, Inc. - Open Source Search
• Oversight Systems - Analytic Software
• Purisma - Customer Data Management SaaS, since sold to Dun & Bradstreet
• Siebel Systems - Analytic Software, since sold to Oracle
• Skytide - Digital Media Performance Management Software
• Vignette - Content Management Software, now a public company.

4. Their pricing model. It's cheap and offers small and medium sized enterprises (SMEs) some real analytic grunt without the hassles of installing and running the software themselves:

Indicee is certainly not alone and there are other startups taking similar but distinctly unique services to market. For example, take a look at CloudSwitch, RightNow Technologies, RightScale or Good Data. Good Data offers real OLAP capabilities and runs on Amazon Web Services. It is also backed by an impressive list of people including Marc Andreessen, Tim O’Reilly, Esther Dyson and John Landry.

Many of these startups are building their services on top of the EC2 infrastructure provided by Amazon's web services.

New Choices

As we approach the end of the naughties the BI market is offering some interesting and fundamentally new choices to us:

• BI in the cloud
• SaaS
• Open source.

... and we can now choose to buy each of these services from all the big vendors (at big cost) as well as a raft of new entrants at substantially lower cost.

With prices like those from Indicee and others, we could be seeing a massive increase in the number of organisations making active use of analytics.

My Predictions?

It is still early days for SaaS and full cloud solutions - especially for larger organisations with mission critical BI. For the big boys SaaS and cloud BI will remain a niche option for a number of years. In 2010, of these 3 alternatives, open source will have the biggest impact by far.

SMEs will perhaps be the biggest beneficiaries of these market developments in 2010. As new adopters of BI they are more able to make use of the cloud - especially if the entry cost is less than AU$100 a month to launch your first analytic service.

Stay tuned.

Data Visualisation Market Report

I don't often talk about vendor funded market research due to the obvious dangers of skewed analysis - but I will make an exception with a new (and free) report from industry analyst Lyndsay Wise and Dashboard Insight. The report is called "Data Visualization - Market Landscape" and was funded in part by InetSoft (a dashboard vendor) who sent it on to me.

Why read this report? Because in 35 pages you get an understanding of:

• the context within which dashboards can be used
• how a range of industries currently do dashboarding
• 4 case studies
• a snapshot of the capabilities of 21 vendor products
• large and specialist vendors as well as some open source options.

So if you are unfamiliar with the area and are wondering if dashboarding can play a role in your organisation's analytic strategy then reading this report is an effective way to improve your understanding. Kudos to those involved.

Stephen Few: Visual Business Intelligence Workshop

Stephen Few, author of a couple of great data visualisation books, is coming to Sydney next year. I think this is only the second time he has come to Australia so it is a rare opportunity to hear the man in person. 

For those that don't know of his work, he publishes a blog called Visual Business Intelligence. 

He has also published 3 books since 2004:

• Now You See It: Simple Visualization Techniques for Quantitative Analysis
• Information Dashboard Design: The Effective Visual Communication of Data
• Show Me the Numbers: Designing Tables and Graphs to Enlighten

The books are very approachable and are in the same genre as those by Edward R. Tufte. In the past I have used the work of both of these authors (among others) to apply good design and usage principles in practical situations. For example, I have created a style guide for analysts designing reports and dashboards at an organisation I used to work with. Here is how that document was organised:

Guide To Using Analytics

Table of Contents

1. Report and Dashboard Design Guide    2
2. Tables Versus Graphs    2
3. Using Tables    2
4. 1. Quantitative-to-Categorical Relationships    2
   2. Quantitative-to- Quantitative Relationships    4
5. Using Graphs    6
6. The Corporate Look and Feel    9
7. Quick Start Guide    11
8. Hierarchies    12
9. 1. Product    12
   2. Broker    13
   3. Business Line    15
   4. Account    17
   5. Transaction    21
   6. Purpose    24
10. Measures    29
11. Dimensions    101
12. Index    144

Based on his books, I can recommend this workshop. Cost is $1600 if you are quick to book, $1800 if you're not. Here are the details care of the workshop organisers, Altis:

Stephen Few: Visual Business Intelligence Workshop

“Simplicity is the ultimate sophistication.” — Leonardo da Vinci

The three day workshop prepares you for the many facets of presenting and analysing quantitative business data. Stephen Few approaches the problem of taming the information overload by providing simple practical instruction communicated through examples. Key concepts that will be covered include:

• Table and graph design for effective communication
• Dashboard design for at-a-glance monitoring
• Visual data analysis for discovery and understanding

Visual Business Intelligence Workshop Sydney 9-11 February 2010

The course runs for 3 days and will be presented by Stephen Few. Download the course brochure for full course details plus a full course outline. 

  Click Here to Register.

For more information, please email Altis at [email protected].

The Size Of The Local Market

I have always wondered how big the Australian BI market was. Even when I was a Research Director of BI and IM at Gartner the market research didn't easily reveal data down at the Australian level. So I was pleased to read about some apparently 'hard' data:

Gartner's prediction for the business intelligence [market] for 2009 through 2012 says that business units, rather than IT, will control at least 40 percent of the total budget for BI. The problem here, Gartner wrote, is that "business users have lost confidence in the ability of [IT] to deliver the information they need to make decisions." 

The report continues to note that the market for business intelligence (BI) platform software in Australia is forecast to reach A$174.8 million in 2009, up 16.8 percent from A$149.6 million in 2008.

This little bit of intelligence is care of local vendor eXant - the local reseller of the reporting and dashboarding business tool eXPorter. This looks like an interesting tool but I don't know what their pricing is like.

I know enough about Gartner's research methods to add a note of caution to their figures. The true size may be very different from this estimate - but at least they try.

Now all I need is my 1% market share ...

Leadership Lessons in Data Quality - Part 2

In my last blog (Leadership Lessons in Data Quality - Part 1) I talked about some useful techniques that I have learned by delivering improved data quality in a number of Australian organisations. In this post I want to talk about how to help business people to care more about DQ.

This is vital because it is the key to making good data quality sustainable in your organisation.

The short answer - like a lot of analytic challenges faced in the real world - is to measure the problem. 

Once you can measure data quality, as I mentioned in my earlier post:  I also recommend that data quality improvement of the business owners data be made a part of their performance incentive program. In simpler language: link it to their bonus.

Data quality has to be measured like any other KPI so that non-experts can understand two things:

1. What we all mean by poor data quality - define it
2. The cost of poor data quality - quantify it.

This is easiest to see in an example:

_______________

Don't Set as a KPI:

Currently 6,753 customer records, or 2.0834% of all main customer records in EDW2 contain errors. Our objective is to reduce this by 50% within 6 months.

Do Set as a KPI:

Currently almost 7,000 clients have incorrect data that results an average of 2,800 direct mailings being returned by Australia Post. This adds $26,000 directly (printing and posting) and indirectly (administration) to each marketing campaign. Our objective is to reduce this average cost by 50% within 6 months.

_______________

A useful technique I have developed to help me understand data and quality is to create a taxonomy that classifies data in the following three ways:

• 'Classic' data quality criteria. Things like accuracy, relevance, availability, etc. 
• The impact data quality has on the business. Things like dollars lost if data is wrong, the value of increased sales if data is correct, etc.
• What action(s) they initiate in an organisation when the data is wrong. Things like intervening at the source of data entry, remedial actions after data entry, etc.

So what metrics can you associate with data quality in your organisation? Here are six I use:

1. Accuracy: the degree of confidence that data is free of error/defect
2. Completeness: the extent to which data is not missing and is of sufficient breadth and depth for the task at hand
3. Consistency: the degree to which common data across different sources follows the same definitions, codes and formats
4. Timeliness: the degree to which data is up to date
5. Security: the degree to which data confidentiality, integrity and availability has been maintained
6. Fit for Purpose: the degree to which data is relevant, appropriate and meets business specifications.

Taking A Competitive Snapshot

Compete.com is a web analytics startup that has some interesting features that are worth mentioning as they:

• claim to have a diverse sample of 2,000,000+ U.S. Internet users that have given them permission to analyse the web pages they visit and ask them questions via surveys
• have built an online competitive intelligence service that lets you compare websites in seconds.

Sounds interesting but I thought that it would have no relevance to Australia as Compete say they only have U.S. users. But I thought I would give it a go anyway.

On the left is the snapshot produced when comparing Commbank, NAB and ING Direct Australia (well to be accurate: just the websites commbank.com.au, nab.com.au and ingdirect.com.au). Click on this picture to open a larger version.

Visitor numbers do seem to reflect the U.S. centricity of the sample but I can see that the search terms and referral information could be useful for the marketing department to tune their site (SEO).

The only similar service that I know of that does cover Australia is bwired (a local web technology company). They don't have an out-of-the-box solution but (if you can find it) they offer an online website comparison called RISE.

With RISE, you fill in a form telling RISE about how you think a target website performs against 20 or so criteria. RISE then emails a short report back to you comparing the target to both a composite group of competitors and a set of 3 anonymous competitors.

Here is the report generated for ingdirect.com.au. The report is based on my responses to the RISE questionnaire as a non-customer who explored the website - NOT on any particular insiders information on how ING Direct performs as an organisation. Here is the report:

__________________

You have filled out our RISE questionaire for www.ingdirect.com.au, and have select finance/banking for benchmarking. 

You have filled out our RISE questionaire for www.ingdirect.com.au, and have select finance/banking for benchmarking. 

Your results are below:

	Your Results	Industry Avg	Overall Avg
Usability & Functionality	53.75%	50.00%	38.34%
Research	68.00%	30.22%	22.79%
Image	44.00%	36.44%	31.27%
Sales	87.14%	35.87%	26.24%
Efficiency	49.33%	38.52%	26.30%
Total	60.45%	38.21%	28.99%

Competitor comparison:

	Competitor 1	Competitor 2	Competitor 3
Usability & Functionality	46.25%	55.00%	57.50%
Research	4.00%	16.00%	34.00%
Image	18.00%	44.00%	32.00%
Sales	17.14%	2.86%	42.86%
Efficiency	26.67%	32.00%	37.33%
Total	22.41%	29.97%	40.74%

__________________

So you can see that bwired takes a very different approach to Compete.com. For me, bwired is too open to the opinions of a single individual - in this case me. Also competitors are unnamed. Compete.com has the advantage of larger sample sizes, time series data, named competitors and data on search terms, referral and destination sites. These features, and the packaging and flexibility of the Compete dashboard make it a clear winner. Shame about the limitation to U.S. users.

I make no mention of Google Analytics which is excellent for analysing websites - but only for those you 'own'. So you can't benchmark against your competitors. Maybe I'm wrong. I'd be happy to hear from anyone that knows more.