Oz Analytics

Data Mining in 1,000 Words

The Atlantic has published a useful short article describing what data mining is. "Everything You Wanted to Know About Data Mining but Were Afraid to Ask". Worth a read and forwarding to anyone in the general management community who needs to know this stuff.

If you are looking for some deeper background on data mining then there is a link within the article to a paper by Usama Fayyad, Gregory Piatetsky-Shapiro and Padhraic Smyth. This is a more meaty introduction to the topic but the authors illustrate the concepts with a particulalry clear (and practical) example. Published in 1996, a lot, and I mean a lot has happened in the intervening years. But when you rightly worry about the current hype around Big Data, this article reminds you that under the hyperbole the business drivers underpinning mining are decades old. Dealing with data is still one of the major challenges of business. Get used to it.

ps. If you clicked on a link within The Atlantic article and got Sesame Street, this is the wrong link. Although maybe this is a more effective way of communicating to your boss after all ... ?

Not Dead (Yet)

Surviving information management.

Over recent months this blog has taken a back seat and I have been off doing other things. The holiday is over and I'm back.

My first post-sabbatical contribution is a presentation to IAPA (the Institute of Analytics Professionals of Australia) that I delivered last week. It's all about what it is that I do as an information manager in a large corporation. It's also more of a personal 'survival guide' than deep insight but the feedback from the audience seemed to be good.

  Open the 2012 Analytics Survival Guide

Let me know what you think - especially anything about your personal information management strategy. Does anyone out there use OpenCalais?

I also recommend that you check-out IAPA (if you haven't already). They are doing good things for the profession and their meetings attract good, engaged crowds.

Day 2 Week 7 of the CORTEX MBAnalytics

Continuing week 7 - part of the 60 day information management refresher course from the CORTEX.

The CORTEX

Master of Business Analytics

Week 7, Day 2

Theme: The S Word

Week 7, Tuesday

This week's theme is statistics - yes, the dreaded S word. This is what is at the heart of the mechanics of analysis. So assuming that you have a reasonable grounding in basic stats, this week we will look at some of the current issues raised when stats is applied in the real world.

READ       Today's task:    Why statistics rule 

15 minutes

"Why Statistics Rule", article by Stathi Paxinos, The Age newspaper, June 29th, 2004.

Key takeout: Stats matter and you cannot be an effective analyst without a solid grounding in the subject. Most will be familiar with the book Moneyball by Michael Lewis - now an apparently very good film starring Brad Pitt "the man who made statistics sexy". Moneyball is the story of US baseball team Oakland A's general manager Billy Beane's successful attempt to put together a baseball club on a budget by employing computer-generated analysis to draft his players. This article shows how fundamental stats have become to sports - in this case in Australian AFL. Remember - this article is from 7 years ago, so it's more historical record than news now. Why should we care? Because the AFL is just one example among many industries where stats is an essential tool of management. In a way this is the justification for the whole of the CORTEX and this refresher course.

As always, comments, questions and ideas are all welcome.

The MBAnalytics is just one part of the Business Intelligence CORTEX - a community of Business Intelligence, Data Warehousing, Analytics and Project Management professionals in Australia, New Zealand and Asia. Membership is free and the forums are very active with over 25,000 posts offering a uniquely Australasian viewpoint.

Day 1 Week 7 of the CORTEX MBAnalytics

Welcome to week 7 - part of the 60 day information management refresher course from the CORTEX.

The CORTEX

Master of Business Analytics

Week 7, Day 1

Theme: The S Word

Week 7, Monday

This week's theme is statistics - yes, the dreaded S word. This is what is at the heart of the mechanics of analysis. So assuming that you have a reasonable grounding in basic stats, this week we will look at some of the current issues raised when stats is applied in the real world.

WATCH       Today's task:  Dance of the P Values  

10 minutes

YouTube video by Emeritus Professor Geoff Cumming, Statistical Cognition Laboratory, School of Psychological Science at La Trobe University, and author of 'The New Statistics' book published in 2012 by Psychology Press and Routledge, and the free software package 'ESCI' (“ESS-key”, Exploratory Software for Confidence Intervals) running under Microsoft excel.

Key takeout: This video's message is to highlight how commonly accepted techniques can mislead. To me it is the analysts responsibility to choose the most appropriate measures and not just to frame outcomes as favourably as possible. In the short-term you may gain, but beyond that you will lose every time.

As always, comments, questions and ideas are all welcome.

The MBAnalytics is just one part of the Business Intelligence CORTEX - a community of Business Intelligence, Data Warehousing, Analytics and Project Management professionals in Australia, New Zealand and Asia. Membership is free and the forums are very active with over 20,000 posts offering a uniquely Australasian viewpoint.

Day 5 Week 6 of the CORTEX MBAnalytics

Continuing week 6 - part of the 60 day information management refresher course from the CORTEX.

The CORTEX

Master of Business Analytics

Week 6, Day 5

Theme: Analytics In The Real World

Week 6, Friday

This week's theme is application - not in the sense of a computer program - rather in the sense of 'use it or lose it'. This week we look at how analytics is being applied in the real world.

WATCH       Today's task:  Making $79B Agile 

15 minutes

'Making $79B Agile: How Procter & Gamble Handles BI', Patrick Kern, Director of Business Intelligence at Proctor and Gamble interviewed by Bruno Aziza, Director of Business Intelligence at Microsoft

Key takeout: This short interview contains a good overview of how one large company has addressed their business requirement for enterprise wide agile decision making.

Next installment of the MBAnalytics will be on Monday October 10th.

That's it for week 6. This time there is a long weekend in Australia and I think it's time for a 'half term break', so I will see you back on Monday October 10th for the next installment. As always, comments, questions and ideas are all welcome.

The MBAnalytics is just one part of the Business Intelligence CORTEX - a community of Business Intelligence, Data Warehousing, Analytics and Project Management professionals in Australia, New Zealand and Asia. Membership is free and the forums are very active with over 20,000 posts offering a uniquely Australasian viewpoint.

Day 4 Week 6 of the CORTEX MBAnalytics

Continuing week 6 - part of the 60 day information management refresher course from the CORTEX.

The CORTEX

Master of Business Analytics

Week 6, Day 4

Theme: Analytics In The Real World

Week 6, Thursday

This week's theme is application - not in the sense of a computer program - rather in the sense of 'use it or lose it'. This week we look at how analytics is being applied in the real world.

READ       Today's task Part 1:   Data Mining At The ATO 

20 minutes

'Data Mining in the ATO', presentation by Warwick Graco, Director Operational Analytics, Office of the Chief Knowledge Officer, ATO at a 2007 UTS Faculty of Information Technology  (FIT) Research Seminar

Today's task Part 2:  ATO No Longer Taxing 

10 minutes

'No Longer Taxing, Using SAS to make it easier to interact with the Australian Taxation Office', SAS Case Study, 2007

Key takeout: The Australian Tax Office (ATO) is by most measures a pretty serious user of analytics. So it is interesting to see how both traditional big-vendor tools like SAS and open source Rattle and R are co-existing. Perhaps it's 'horses for courses'? Unfortunately, little is publicly known about how the SAS and R communities interact at the ATO? Perhaps a reader of this blog knows more?

 
As always, comments, questions and ideas are all welcome. 

The MBAnalytics is just one part of the Business Intelligence CORTEX - a community of Business Intelligence, Data Warehousing, Analytics and Project Management professionals in Australia, New Zealand and Asia. Membership is free and the forums are very active with over 20,000 posts offering a uniquely Australasian viewpoint.

The Data Journalism Process

As a part of the the 60 day information management refresher course from the CORTEX, I recently blogged about The Guardian newspaper's outstanding work leveraging data. I have done a bit of cutting and pasting and reconstructed the process referred to by Simon Rogers, Editor at The Guardian, in his talk on September 21st, 2011. The talk is available on my blog here.

Click on the image to display a larger version.

Yes, there is a lot of manual work, but it is interesting that The Guardian sees so much value being generated that they are prepared to invest the time and money. I recommend bookmarking their Data Store section.

Day 3 Week 6 of the CORTEX MBAnalytics

Continuing week 6 - part of the 60 day information management refresher course from the CORTEX.

The CORTEX

Master of Business Analytics

Week 6, Day 3

Theme: Analytics In The Real World

Week 6, Wednesday

This week's theme is application - not in the sense of a computer program - rather in the sense of 'use it or lose it'. This week we look at how analytics is being applied in the real world.

WATCH    Today's task:  Rich reward for data miners  

10 minutes

'Rich reward for data miners', by Glenn Mulcaster, The Sydney Morning Herald, September 26, 2011

Key takeout: Why should an organisation pay for a small team of analysts when it can run a competition that attract some of the greatest analytic minds in the world. This is not a one-off example. Netflix and others have generated serious returns from other competitions.

As always, comments, questions and ideas are all welcome. 

The MBAnalytics is just one part of the Business Intelligence CORTEX - a community of Business Intelligence, Data Warehousing, Analytics and Project Management professionals in Australia, New Zealand and Asia. Membership is free and the forums are very active with over 20,000 posts offering a uniquely Australasian viewpoint.