The Future Of Business Intelligence

Keynote address to the Next Generation Data Warehousing Conference, IDG Korea , Seoul, South Korea, February 16th 2011

I presented my thoughts to a couple of hundred Senior Managers from a wide range of Korean organisations. Crystal ball gazing is something I usually leave to vendors and consultants so I focussed on the following aspects that I at least have some practical experience of:

• Key trends to the end of the decade
• Best practices
• Organising the future intelligence capability.

The following key trends struck me as important factors influencing the future of our work in analytics:

• An Unprecedented Explosion In Data. “Between now and 2020, the amount of digital information created and replicated in the world will grow to an almost inconceivable 35 trillion gigabytes as all major forms of media”
  IDC iView, "The Digital Universe Decade – Are You Ready?" May 2010
• The Dominance of Mobile Multimedia Content. “Global mobile data traffic will increase 26-fold between 2010 and 2015 ... Mobile network connection speeds will increase 10-fold by 2015 ... Two-thirds of the world’s mobile data traffic will be video by 2015.”
  Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2010–2015, February 2011
• The End Of English. Chinese-language content will dominate the internet by 2014
  Gartner 2009.

I'm not completely convinced about Chinese domination - but I agree that the Internet will become strongly multi-lingual as the rise of China and Brazil continues.  In any event, we all now live in a world of BIG data. Data that is growing annually by orders of magnitude and is moving to mobile/embedded devices that generate real-time multimedia data. On the other hand, IT skilled people - those with the skills most likely to handle data growth - are only predicted to grow 1.4 fold over the same period. So there is an emerging massive skills shortage coming and we have four choices:

• Do nothing. You wouldn't be alone if this is your strategy.
• Focus your current analysts on embedding their skills in a broad range of your staff.
• Try to buy the skills by poaching staff from other organisations. This could get to be very expensive as you will likely to be in a bidding war with other organisations facing a similar skill shortage.
• Use software that enables your analysts to handle vastly more data than they can today.

Just to complicate things, the era of the English/Western Internet is being supplanted by Chinese - a language that is impervious to processing by most of the current and emerging intelligence capabilities out of the US.

Personally, I see as our best hope a strategy that focusses on embedding analytic skills into the general business community. This greatly enhances analytics usefulness in the short term, whether-or-not you believe in the above trends. It will also buy us the time to see smarter software emerge that does the 'heavy lifting' of BIG (Chinese?) data for analysts. The analogy I used in my address was transportation and the London underground network. The actual network looks like the following:

It took Harry Beck in 1931 to analyse the London underground network and realise that the actual physical location of the station was less important than the information about how you can get to your destination.

This is exactly what analytics needs to achieve. I like to call this actionable insight.

If you have any doubts about the explosion in data we can expect, take a look a recent years. The Washington Post has created a created a graphic that shows how data grew from 1996 and 2007:

After looking at the impact of these trends, I went on to discuss templates for both the analytic skills we need and how to best to organise your teams. I have posted about these topics previously here.

After the presentation there were questions from the floor and it was interesting to see that the questions could have come from any country. The problems and challenges discussed were exactly the same as I hear in other countries. It's nice to know the we are all in the same boat.

 

Comments

[Shamoona]

You're not wrong about keeping track of Google's upaetds and new features. It all gets quite bewildering and when you add Yahoo and MSN into the mix it becomes nigh on impossible. I'm thinking of giving up .

[Suriani]

Notastar is speaking the truth. You won't be able to use GPS nagovatiin without an actual GPS antenna either built into your phone or attached externally (which is not practical).Some options are to use Google Maps ( m.google.com from your phone) which will use cell phone tower data to get a general location of your phone which is useful for doing restaurant or gas station searches or to simply upgrade your phone or purchase a separate gps nagovatiin device.I suggest getting a separate GPS device because if you plan on using the GPS nagovatiin on your phone often you will find that it doesn't update quickly all the time and that you use a lot of battery running the GPS antenna. I hope this helps

[Angelo]

Just rzeeiald that April is right around the corner? when did that happen?Dear Avinash: Top Web Analytics Questions, Twitter Edition (Occam?s Razor by Avinash Kaushik)

[Avery Gerner]

Chinese language explosion? I don't think that's possible. English is and will always be the universal language. But I agree that they are changing the world now, together with India and Brazil. That just means that countries are becoming more and more competitive in the business world.

[E cigarette retailers usa and e Cigarette Cartridges]

Thanks for the overview of the challenges facing the industry today. They have shown some of the major limitations of the BI in a box for the problem of dealing with the data explosion we have witnessed in the last two decades.

[E cigarette retailers usa and e Cigarette Cartridges]

Thanks for the overview of the challenges facing the industry today. They have shown some of the major limitations of the BI in a box for the problem of dealing with the data explosion we have witnessed in the last two decades.

[business intelligence]

Good points on Future of Business Intelligence and their strategy. You included a good important points which I have in my mind that We should have to buy the skill people from other organization and trying to implement into the business. It will give you a lot advantage to your business.

[Excapite]

Thanks for the insight into the challenges facing the industry today.

You have illustrated some of the key limitations of the "BI in a box" approach to the problem of dealing with explosion in data we have witnessed over the past 2 decades.

However I'm not sure the current focus on creating yet another generation of super users is the answer to the explosion in data.

As we have seen on the web the idea that curation begets order is fundamentally undermined by the reality that curation achieves little more than adding more data to the chaos.

As I explan in my post on the three ages of business intelligence (See http://excapite.wordpress.com/2009/11/13/the-3-ages-of-business-intelligence/) the future of BI is all about the intelligent network.

What does this means? Well today there is growing acceptance of the idea that analysis is not a singular activity to be undertaken by "data gurus" but a a collaborative effort. This means that for BI to provide real value across any business - rather than the atypical data hospital it is in most large organisations today - you need to get the data out of the hands of the "Spreadsheet Jockeys" and discover ways to have the intelligence embedded in the system so it is there 24/7 transparently improving every touch point across the organisation. Put quite simply the future of Business Intelligence isn't BI on steroids. It is the intelligent business.

[Stray__Cat]

I do not think that all this hype about "big" data is solidly grounded.

I do not deny the evidence of the ever growing stream of data but the numbers to be kept under control in a company are always the same.

Analyzing a clickstream has severe IT requirements but the clickstream boils down to few metrics if seen as a component of the company control system.

BI task is to handle the metrics, not to calculate them; that's the task of domain specific applications.