Oz Analytics

Changing Tack

My 2 regular readers (thanks mum and dad!) may have noticed that the frequency of my posts has dropped dramatically since 2012. There was a reason for this - beyond my strongly held opinion that those who have nothing sensible to say should simply shut-up. That reason was/is that I have been starting a new venture and the 'stealth mode' has lasted a lot longer than I expected.

In 2012 I suffered an information crisis and I suspected that I was not alone. The crisis arrived because I was being increasingly overwhelmed with information and most of the information swamping me was not on the web. Instead it was in the form of documents and email. I pride myself as being pretty well organised with effective email filters in place and an organised corpus of professional and personal knowledge approaching 1,000,000 objects. Mostly documents of one sort or another.

I was quietly proud of my knowledge management skills but after pride cometh the fall - and in 2012 that's what happened. Let me explain how it happened and I hope the story will resonate with you.

 As a serial entrepreneur and corporate analyst, I have a broad range of professional interests. These range from all aspects of analytics and information management, and broaden out into general management and IT. I am also a bit of a pack rat and if I come across some information that looks useful I store it. For those interested, my tools of choice are Mac OSX, HoudahSpot, DevonAgent and Devonthink.

So over the last 20 years I have built up a very useful corpus of knowledge. By 2011 however I was becoming increasingly frustrated by my inability to effectively use this corpus in my daily work. Each day I need to deal with the usually pile of emails and read/respond to/create multiple documents. This presented the following problems:

1. Identifying and dealing with those emails and documents that I had to.
2. Ignoring the rest.
3. Locating past knowledge/information that would help me to deal effectively with the day's emails and documents.

So the problems were pretty much at the core of what I do.

If only there was a way for me to more efficiently decide whether-or-not a long email or document was worth me reading. Then, when I needed to make a decision or give advice, how could I tap into my corpus of knowledge?

These are the problems solved by my new venture Six Degrees Of Data.

Six Degrees Of Data is building Apps and web services that we hope turn the web on its head. Our first product will launch later this year. It is called Bridger and is an App and Web Service that intelligently evaluates, summarizes and links your documents to trusted sources on the web.

Bridger is a research assistant for busy people suffering from information overload.

We are trying to release the value currently locked inside documents because despite the phenomenal growth of the internet, 80% of the new information created each year is still unstructured – in other words it is text, graphics and video contained in things like documents, emails, Facebook shares, web pages and tweets, etc. This unstructured information is poorly integrated with the world wide web. Even if they are published to a web site, they have remained isolated islands of information – until now.

Within the last five years, the size of the average web page has more than tripled, and the number of external objects has more than doubled. The average web page is now 1 MB and growing at over 20% each year. Today the average web page has about 10 links and 600 words. So it is reasonable to say that less than 2% of the content is hyperlinked to something else.

Your documents often contain only a single link (or less) and many more words than a web page.

If you are lucky, there is a single bridge (hyperlink) from the web to your document. Think of your documents as information that is connected to the web superhighway only via an off-ramp. You can find published documents easily enough because published documents are indexed by the likes of Google and Bing. But what you can’t do is start from your document and get easily back onto the web in an intelligent way.

That’s why we see the current web-document world as a one-way off-ramp.

Bridger builds you an internet on-ramp for all of your documents – whether they are published on the web or not. This is how we can release the untapped value of the 80%.

How do we do this? Well, it’s pretty complicated technically but the good news is that you really don’t have to know about this if you don’t want to. All you need to understand is that Bridger takes your isolated documents, emails, etc. and automatically links them to over 50 billion trusted additional sources of information on the world wide web.

Drop a document on to the Bridger app and it will instantly tell you what the document is about and give you concise profiles of all the relevant people, organizations, places, technologies and more.

Your document is magically transformed into a fully integrated web body of knowledge.

And Bridger is the first of a stream of new products we re planning. Future areas that we will concentrate on are enriching our personal corpus' of knowledge, collaboration and sharing.

So my posts may remain sporadic but I hope to share more about Bridger and our journey as Six Degrees Of Data gets up and running over the coming months.

In the meantime, if you have a passion for the Semantic Web, Linked Open Data, URI's, sparkle and tuples, then give me a call.

 

Day 5 Week 6 of the CORTEX MBAnalytics

Continuing week 6 - part of the 60 day information management refresher course from the CORTEX.

The CORTEX

Master of Business Analytics

Week 6, Day 5

Theme: Analytics In The Real World

Week 6, Friday

This week's theme is application - not in the sense of a computer program - rather in the sense of 'use it or lose it'. This week we look at how analytics is being applied in the real world.

WATCH       Today's task:  Making $79B Agile 

15 minutes

'Making $79B Agile: How Procter & Gamble Handles BI', Patrick Kern, Director of Business Intelligence at Proctor and Gamble interviewed by Bruno Aziza, Director of Business Intelligence at Microsoft

Key takeout: This short interview contains a good overview of how one large company has addressed their business requirement for enterprise wide agile decision making.

Next installment of the MBAnalytics will be on Monday October 10th.

That's it for week 6. This time there is a long weekend in Australia and I think it's time for a 'half term break', so I will see you back on Monday October 10th for the next installment. As always, comments, questions and ideas are all welcome.

The MBAnalytics is just one part of the Business Intelligence CORTEX - a community of Business Intelligence, Data Warehousing, Analytics and Project Management professionals in Australia, New Zealand and Asia. Membership is free and the forums are very active with over 20,000 posts offering a uniquely Australasian viewpoint.

Day 5 Week 5 of the CORTEX MBAnalytics

Concluding week 5 - part of the 60 day information management refresher course from the CORTEX.

The CORTEX

Master of Business Analytics

Week 5, Day 5

Theme: Analytic Errors

Week 5, Friday

This week's theme is mistakes. Today we look into how easy it is to start out with an incorrect assumption. The example here is also very entertaining.

WATCH    Today's task:  Why do crack dealers still live with their moms?     Streamed

20 minutes

"Why do crack dealers still live with their moms?", TED Talk in February 2004 by Steven Levitt Professor of Economics at the University of Chicago and co-author of the 2005 book Freakonomics.

Key takeout: Populist - definitely. Entertaining - absolutely. In this talk Levitt examines the links between real-world events, and finds many instances where the data simply doesn't back up popular belief. How often do you re-examine your basic assumptions? Doing so when building or evaluating your hypotheses should be a mandatory part of any analysts checklist. Now go out and read the blog!

As always, comments, questions and ideas are all welcome. 

The MBAnalytics is just one part of the Business Intelligence CORTEX - a community of Business Intelligence, Data Warehousing, Analytics and Project Management professionals in Australia, New Zealand and Asia. Membership is free and the forums are very active with over 20,000 posts offering a uniquely Australasian viewpoint.

Day 4 Week 5 of the CORTEX MBAnalytics

Continuing week 5 - part of the 60 day information management refresher course from the CORTEX.

The CORTEX

Master of Business Analytics

Week 5, Day 4

Theme: Analytic Errors

Week 5, Thursday

This week's theme is mistakes. Today we look into how people make decisions and you have a choice. 

WATCH    Choice 1:  Think Twice: Michael Mauboussin at CEP 2011     Streamed

30 minutes

"Think Twice:" Michael Mauboussin at CEP's 2011 Conference, YouTube video of Michael Mauboussin talking at the Center for Effective Philanthropy's 2011 conference.

READ       Choice 2:   Think Twice A cautionary tale for investors     70 KB

5 minutes

'Michael Mauboussin's 'Think Twice': A cautionary tale for investors' article by Vishesh Kumar, on the AOL DailyFinance site, August 2009. Available on AOL here.

READ       Choice 3:  Think Twice: Harnessing the Power of Counterintuition   To buy

??? hours. How fast can you read 204 pages?

'Think Twice: Harnessing the Power of Counterintuition' by Michael Mauboussin Chief Investment Strategist at Legg Mason Capital Management and adjunct professor of finance at Columbia Business School, hardcover book published by Harvard Business School Press, October, 2009, 204 pages. Available to purchase on Amazon.

Key takeout: Fascinating insight into how we make decisions - especially insightful on the subject of mistakes and experts. Choice 1, the video, is a great way to be introduced to Mauboussin's ideas. Choice 2 is the one to take if you really can't spend half an hour watching choice 1. Choice 3 is something to keep in mind. Now go out and read the book!

As always, comments, questions and ideas are all welcome. 

The MBAnalytics is just one part of the Business Intelligence CORTEX - a community of Business Intelligence, Data Warehousing, Analytics and Project Management professionals in Australia, New Zealand and Asia. Membership is free and the forums are very active with over 20,000 posts offering a uniquely Australasian viewpoint.

Day 1 Week 3 of the CORTEX MBAnalytics

Congratulations for reaching week 3 - part of the 60 day information management refresher course from the CORTEX.

The CORTEX

Master of Business Analytics

Week 3, Day 1

Theme: Communication & Collaboration

Week 3, Monday

This week's themes are communication, and if you are listening and willing, a resulting collaboration. 

WATCH    Today's task:   Google's Effects on Memory     80Mb

5 minutes. You can also stream it here.

PBS NewsHour 'Google's Effects on Memory' Psychologist Betsy Sparrow

interviewed by Hari Sreenivasan in 2011

Key takeout: The ability to retrieve information from the net has already

allowed us to move more of the 'burden' of memory from our brains and this

is reducing our ability to recall information without the net. It hadn't occurred

to me before reading this research but this is an important step towards the

singularity. 

READ       For extra credit:  Cognitive Consequences of Having Information at Our Fingertips 

  300KB, 15 minutes

'Google Effects on Memory Cognitive Consequences of Having Information at

Our Fingertips', by Betsy Sparrow, Jenny Liu, and Daniel M. Wegner,

Science Express report, July 2011.

Key takeout: How does this impact the design of our collaboration, analytic and

business intelligence portals, etc.?

 

As always, comments, questions and ideas are all welcome. 

The MBAnalytics is just one part of the Business Intelligence CORTEX - a community of Business Intelligence, Data Warehousing, Analytics and Project Management professionals in Australia, New Zealand and Asia. Membership is free and the forums are very active with over 20,000 posts offering a uniquely Australasian viewpoint.

It's How You Tell It

When delivering actionable analytics within an organisation I often spend some time removing commentary from management reports. I have found that the regular reporting packs produced for senior management are often drowning in long commentaries that attempt to explain the data in the packs.

This immediately gets me asking 'Why?' most of the commentary exists.

Why can't the data do the talking? What is it about the way the data is organised and presented that makes it necessary to add all of this text?

In my experience there are a number of reasons for this.

The first is poor design of the report. Not enough thought has been given to either the purpose of the report (i.e. what are the objectives? what are the decisions we want taken?) or the audience (i.e. what are the information requirements of the decisions makers? how do they prefer information to be presented?).

A second reason may be limitations in the tools being used to produce the report. Even the best business intelligence tools have their functional limitations and as the first rule of report design states:

"Within any given set of business analytic requirements there will be at least one that is impossible to deliver using the available toolset."

If you are unfamiliar with these rules, then you should know that this immediately leads to the second rule of report design:

"The amount of insight delivered by a report is halved by each undelivered business analytic requirement."

A third reason is less benign than the first two. I use the word 'spin' to describe the collection of responses that the business 'reporter' makes when they fear that the 'reportee' will:

• respond negatively to the data presented; or
• not sufficiently appreciate the 'brilliance' that was required to deliver the result.

In both cases the intent of the manager reporting the figures is to 'spin' the message given to the report reader. 9 times out of 10 the spin is intended to mislead the reader.

All of these reasons for extensive amounts of report pack commentary result in at best the report data failing to provide the insight required by the business users. At worst it misleads decision makers into making bad decisions.

I like to think that my job is to create an environment where good decisions can be made because they are based on unbiased information that fairly measures performance. In practical terms, this translates into reducing commentary by improving the quality of the automatically produced data. This sounds simple but this is deceptive. It is very challenging to achieve in practice.

Here is a little video that I have made that helps explain how identical words can be spun to justify diametrically opposed management messages. It's a bit of fun (adapted from a political campaign ad I saw some years ago) and that is why I have found it effective. Enjoy.

Download Spinning_Decisions.m4v

dataFUD

 

Fear, Uncertainty and Doubting your data is a feature of organisations who are failing to effectively manage their information resources. As a result, it is one of the main issues to address when building an information management strategy for a dataFUD organisation. How do you know if your organisation is failing? Well, if you're reading this blog then there is a good chance you already know, but just in case, here are some things to think about. Do you or others in your organisation spend a lot of time:

• Performing work incorrectly or making a wrong decision?
• Re-doing work because it was previously performed incorrectly?
• Recovering from the impact of making a wrong decision?
• Taking unnecessary time to investigate the integrity of the data before using it?
• Performing calculations or reformatting the data before it can be used?
• Hunting for additional information in order to use the data?
• Losing customers because it caused work to be performed incorrectly?
• Causing unrecoverable damage?
• Missing business opportunities?
• Miscommunication within the business or with end customers and other information stakeholders?

If any of these ring true, then there is a good chance that you need to manage information better. How will this help? I've identified 7 main benefits that an organisation can realise by managing information better. They are:

1. Improved Data Quality – Poor information management leads to poor data quality and subsequently poorer business decisions. Early indicators of the business costs of non-quality data (listed in the bullet list above) point to potentially foregoing revenue as high as 10-20% of the revenue or total budget of the organisation.
2. Elimination of Duplicate Work – Removing redundancies, reducing the need for rework and therefore reducing operational costs by repairing data quality issues that create the need for manual reconciliation and avoid futile marketing and sales expenditure. On its own, by halving the costs related to poor data quality, a 5-10% saving on operating expenses can potentially be achieved.
3. Unified Vision – or Single Source Of the Truth. Organisations that break the barriers between their silos of operations can unlock value by obtaining unified views of business-critical data like customers and products, as well as apply existing information in new value adding ways such as opportunity identification and cross selling opportunities. This will improves the ability to coordinate activities between lines of business.
4. Broader insights and Better Decisions – into data across products and business units. Decision makers gain a deeper understanding of operations and an accurate picture of past and present performance as well as predictions of future performance that shape decisions.
5. Support More Complex Operations – The ability to serve numerous users with varying requirements at a lower cost. Users include employees from different areas within the organisation, external suppliers, partners and any other party that may interact with the company.
6. Continuous workflow – Continuous workflow with timely information available. This stops people having to wait for information that they require to perform their work and make decisions. Delay can cause lost opportunities or added costs. A well designed information management capability will reduce the lead time an organisation requires to go to market with something new.
7. Compliance – Aids compliance to legislation such as Sarbanes-Oxley as knowing where to find the information required will decrease the risk of non-compliance and allow for greater responsibility.

These benefits are pretty fundamental to the competitiveness and sustainability of business in the 21st century. Just remember that although information management may often be talked about in technological terms, the true key to improving performance is found by up-skilling staff so that they are able to take advantage of your technology investments. There is a reason that almost all management and information experts are talking about competing on analytics. My favourite response? A fool with a tool is still a fool - it doesn't matter how good the data quality.

 

As always with my blogs, I may be repeating ideas from one of the many books I have read, websites I visited or methods I use. I stand on the shoulders of giants and acknowledge their original thoughts when I can. I haven't checked if I am directly quoting from them, but the Mike2.0 community has been an inspiration in recent years so you may well find their ideas mixed with mine. As always, any mistakes or controversies are my responsibility alone.

The Internet Of Analytic Things

The latest edtion of the McKinsey Quarterly contains an interesting article by Michael Chui, Markus Löffler, and Roger Roberts on The Internet of Things (registration required or download PDF below).

The authors main point is that some years after Vint Cerf (one of the founders of the internet) began talking about 'The Internet of Things' - it is now a reality: that now "the physical world itself is becoming a type of information system".

"In what’s called the Internet of Things, sensors and actuators embedded in physical objects—from roadways to pacemakers—are linked through wired and wireless networks, often using the same Internet Protocol (IP) that connects the Internet. These networks churn out huge volumes of data that flow to computers for analysis."

The authors go on to say that:

"as more sensors enter our devices and clothing, we will have the ability to monitor the behavior of people, places and things through space and time, enabling such business applications as presence-based advertising."

This is just the tip of the iceberg. Just think of the analytic opportunities that the widespread use of IP connected sensors represents. Sensor data is a great chance for analytics to move further and faster into decision support - unteathered from your PC and mobile. Real time analysis will be the only way for us to turn the flow of data around us into information we can action.

What we are talking about here is a fundamental change in our environment. To be more precise: the linking of two worlds is happening. Our information world (mostly the internet) and our physical world (the stuff we do when the broadband is down).

So far, the internet has largely been something that we join as we want. No one forces you to have a Facebook page or LinkedIn profile. We choose to participate (or not) based on the value that participation gives us. The Internet of Things changes this. You will not be able to control the actions of most of the sensors in your enviroment - and that includes the flow of sensor data. So opting-out in the future is going to be a lot harder to do than simply switching off your computer or mobile phone.

Above: The authors see six distinct types of emerging applications, which fall in two broad categories: 'information and analysis' and 'automation and control'.

The biggest surprise to me was the impact that sensors are already starting to have on the pricing of products and services. It's a logical result I had not seen coming. Here are a couple of examples from the article:

"... insurance companies, for example, are offering to install location sensors in customers’ cars. That allows these companies to base the price of policies on how a car is driven as well as where it travels. Pricing can be customized to the actual risks of operating a vehicle rather than based on proxies such as a driver’s age, gender, or place of residence."

"In the aviation industry, sensor technologies are spurring new business models. Manufacturers of jet engines retail ownership of their products while charging airlines for the amount of thrust used."

The examples discussed are, one the whole, all positive as they deliver something most people would argue is good: reducing cost and risk or enhancing products and services. The darker side is less well addressed - and I'm not talking about the fact that in the near future my socks are going to announce to the world how many holes I have made in them or when was the last time I washed them!

Right: Forget about body scanners at our airports. Sensors have the potential to make the whole world one large scanner.

I guess it comes down to how comfortable you are in having Governments and Corporations knowing more about both your online and physical activities. Maybe now is a good time to lock and load and move back to Southwest Montana!

Do you remember those little cell-phone blockers you could buy at your local spy shop in the 1990s? I admit to owning one - just so I could stop cellphone calls some of my fellow cinema goers felt were vital to answer during film screenings.  I wonder if there is a smart entrepreneur out there who is already developing a way of disabling sensors in our immediate environment?

The McKinsey article is a very good overview of emerging sensor applications and worth reading. Here's the article to download.

Analytics at Work

Hot off the presses jumps Analytics at Work: Smarter Decisions, Better Results, the new book by Thomas Davenport, Jeanne G. Harris (both of Competing On Analytics fame) and Robert Morison. This book is very much the sequel to Competing On Analytics and this is not a bad thing. CIO Insight magazine named Competing on Analytics one of the all-time “Top 15 Most Groundbreaking Management Books”.

What I like about the book is that it is going after the big picture where better decisions drive outstanding performance, improve competitiveness and help organisations differentiate themselves from the competition.

This time the authors have completed new research on companies in the US and Europe. They include Humana, Tesco, Air France/KLM, Best Buy, Progressive Insurance, and Hotels.com. Using these organisations as illustrations the book is intended to be an implementation-focused guide that describes a five-step model for deploying and succeeding with analytical initiatives.

In the first half of the book five factors are presented as being necessary if an organisation is to succeed at 'doing analytics'. The authors group use the acronym DELTA (the Greek letter that signifies “change” in an equation):

• D for accessible, high-quality data
• E for an enterprise orientation
• L for analytical leadership
• T for strategic targets
• A for analytical talent

This all makes sense to me and then in the second half of the book the five-step model is explained. I will not spoil the authors work by discussing that here.

To me the book is presented as a primer for doing analytics. How well does it achieve this objective? The answer is both 'very well' and 'not really'. Let me explain:

• If you are a C* level executive trying to understand why you need analytics, and you want a high-level understanding of how to achieve this, then the book delivers - and delivers well.
• If you are a Senior Manager directly responsible for making analytics happen, the book is less successful. Not through any fault of the authors. I am one of these managers and I know that analytics is hard. Even a 1,000 page tome and a dozen CD's would not be enough for that.

So I hope that the book will inspire C* level executives that are interested enough to read a 200-or-so page book on the subject. If they only read one book on the subject then this is the one I would give them. For those that can't be bothered, then I can sit waiting for the next crisis to hit and the inevitable change of management to occur!

To the author's credit, I notice scattered throughout the book discussions (on pages 14-15 for example) and insight on how the Financial Industry got risk analytics so wrong - contributing to the financial crisis of 2007. Finance's use of analytics was featured as one of the successes in Competing On Analytics.

Although a Harvard Business Press title, the book is 'tied-in' to Accenture (Harris works for them and Davenport is an Accenture Fellow) and the company has a microsite within their Institute for High Performance consulting practice. Here's a nice brochure from Accenture on Analytics At Work.

A Brief Analysis Of 100 Australian BI Projects

Yesterday I gave a talk at the Mastering Business Intelligence with SAP 2009 conference. The objective was to talk about the core reasons for 'doing' BI/IM/DWH and to identify best practises. I did this by looking at a bunch of projects that I have been involved with over the last several years. The key features of the projects are:

• Lessons from approximately 100 BI projects
• Projects were active in the period 2003 - 2009
• Tools used:
• • Analytic Tools:
  • • 50% in environments with SAP
    • 30% SAS
    • 20% with Cognos
    • 20% Microsoft
    • 20% Other
  • Underlying databases:
  • • 60% Oracle DB
    • 20% DB2
    • 20% SQLServer/MySQL
    • 10% Teradata
• 70% Australian, 30% Asian
• Mainly in Banking, Insurance and Leisure industries.

The most common benefits of the projects were the development of the following capabilities:

• Better understand current performance (issues)
• Identify efficiency opportunities (insight)
• Reliably predict future performance
• Make better decisions because they are based on correct data and insight (trusted)
• Reduce risks (less surprises)
• Enhance agility (in handling new requirements)
• Promote a shared understanding (same language)
• Reduce internal conflicts (SSOT)

Best practises observed were:

• Data Quality champions were active in both IT and BI Departments.
• Domain experts were drawn from across the enterprise
• Data owners existed for every data item and measure in the BI platform
• Data Council members are empowered to make decisions: 1. All business data and their definitions - including reference data; 2. Enterprise performance measures and their calculation basis; 3. solutions (business processes and IT); 4. Monthly meetings to solve DQ issues

Finally, some of the lessons to be drawn are that to develop these capabilities you need to keep in mind:

• BI investments often spread over several years. It is not usual for stakeholders to fund projects for this period without seeing significant returns on their investments in the first 12 months.
• The actual solution(s) is usually only known when well into the BI project (once detailed business data requirements are agreed).
• 30 – 50% of your investment in BI will be spent on meeting additional requirements that were not included in your original business case.

And perhaps most importantly: The more successful you are, the more new requirements you will have to meet - with the same original budget.

I think that the talk went OK as I had a few questions from the floor but only the audience can give you honest feedback. If you are interested, here is a slideshow of the presentation:

Download 20091020Presentation