The Death of BIG Business intelligence

In the second decade of the 21st century the BI platforms from the SIMO vendors (SAP, IBM, Microsoft and Oracle) are dinosaurs.

A provocative statement - but bear with me.

Over the past few years I've blogged about a lot on the emerging trends of information management: real-time, BIG data, agile BI, actionable performance management and the like. The one thing that they have in common is that they are all big. Either big in terms of expensive or big in terms of the changes that they require within organisations.

I've tried to relate many of these trends to practical actions that (for those of us without large budgets) we can still extract value from. The context of these actions is that I have often been advising medium and large commercial businesses with existing business intelligence operations. So what I have experienced may not necessarily apply to government or small organisations.

Frequently my advice has involved technologies from one or more of the BI platforms from the SIMO vendors. SIMO and cheap are not words that go together. It's not a criticism of the platforms. They are what they are but I argue that their time has past because there are now a number of alternative approaches available that leverage newer low cost products and methods.

I have been increasingly looking to the benefits of open standards and web-based solutions to create low cost platforms without further investment in the SIMO vendor platforms. In the past I have used open source to extend SIMO platforms so that I didn't have to pay for the 'enterprise version' or 'additional module' that the SIMO vendors always seem to suggest when I want my platform to do something new.

An example of this was to extend our web-based dashboards to include sparklines¹. This type of graphic was not supported by the SIMO vendor product at the time but was easily extended using open source. It also saved the company the considerable expense of buying the enterprise dashboard for $250,000 + 20% annual support charge.

Now I have questioned the idea of using the SIMO platform at all. In the above scenario I today have invested in something like Tableau which although very definitely not based on open source code, is significantly cheaper to purchase and more adaptable to a variety of data sources and presentation types.

At this point, I will digress into more technical considerations. So if you become glassy-eyed at the mention of java, then it's best if you skip over the next couple of paragraphs and start reading about the example.

Open standards-based initiatives make our world of data so much more open than it once was. Everyone has heard of Big Data, but there are also dozens of open initiatives such as Mondrian (or more correctly: the Pentaho Analysis Services Community Edition) - an Online Analytical Processing (OLAP) server that enables business users to analyze large quantities of data in real-time.

Why do we care? Mondrian gives us an open standard way to access OLAP using Java on the client side. I'm not referring to XML/A the 'open' standard that Microsoft has been promoting for some years. XML/A is not truly open. Mondrian and Saiku should provide a true open driver capability to those of us building best-of-breed platforms. In time, I hope that market pressures will convince the SIMO vendors to build their own drivers for Mondrian.

A lot of brainpower has been invested over recent years improving database technologies so that they better support business intelligence activities. Combine this with the computing power now available on our desktops and in our data centres and clouds and I think you will find that best-of-breed technology stacks where you decide each layer will often do the job more sustainably and cheaper than an equivalent SIMO investment.

An Example

I can best explain the business logic behind my ideas by taking an example such as business MDM (master data management).

Old School

You purchase an MDM solution from the SIMO vendor you already have in-house. Let's take IBM in this case. So you will need to purchase the Websphere Business Glossary and probably the MetaData Server and Workbench. What's this going to cost? I'm guessing $1 million or more for an enterprise license once you factor in the specialist technical skills you will need to integrate it into your organisation.

IBM's Websphere Platform (courtesy IBM)

Now Websphere is a nice platform and the integration between DataStage, QualityStage, Information Analyzer and Business Glossary is a joy to have as a business manager of this stuff. I've just never had the business case to buy the entire platform. This is true of the majority of IBM platforms implemented across Asia-Pacific today. IBM is no exception and this is true of the other SIMO platforms. Budgets, capabilities and maturity levels almost always mean that only parts of a SIMO platform are implemented in an organisation. What a missed opportunity - but there is no point in crying over spilt milk. If the business case to implement entire platforms existed, then the SIMO vendors would be the first to let you know.

Microsoft is the exception to this rule, having a much cheaper licensing model. Unfortunately their MDM capability is very much a 'you build it' one, so the TCO (total cost of ownership) is still significant.

The challenge today is being smarter and more effective than past attempts. 

New School

Means that you leverage your existing investments - often those in web solutions.

For example, an MDM on-the-cheap solution would be to have your content management (CMS) platform hold critical business metadata. SharePoint is frequently the CMS of choice for large traditional corporations. Even if not already implemented, Using SharePoint to build a custom workflow and collaboration solution is pretty straightforward. So with a modest effort, the CMS can support an intranet-based community of business owners to create and maintain business metadata.

Making use of this business metadata is also possible. Continuing with the MDM scenario above, and assuming you use BusinessObjects, Cognos or similar, then you can export the business metadata easily to universes/cubes so that it is available to report writers, users and decision makers. This will drive the use of standard definitions of data and measures and make enterprise performance management easier.

We are also reaching the point where the presentation layer products (reports, dashboards and scorecards) from the SIMO vendors are now intelligently integrating back into - you guessed it - the CMS's like SharePoint. So production and consumption of business metadata is done on the most widespread platform available within the organisation.

This takes BI and IM out of the analytic ghetto and integrates it seamlessly into the corporate intranet - and beyond.

Cost? A conservative $100,000.

 Conclusions

What's the overall result? You get 50% or more of the benefits at 15% or less of the cost. Ongoing maintenance is also less. Most crucial of all is that the cheaper option is also the one most likely to be funded by an organisation. Why? Simple: it's CHEAPER.

This blog started with the statement that the SIMO vendors were dinosaurs. This is not quite true, but as my example above illustrates, it is now time to integrate your existing SIMO investments into your corporate intranet. Not by buying more SIMO products. Instead use more open technologies to integrate them into the intranet. Treat the SIMO-based data and report repositories as services to be consumed via a web front-end that does not require those extra SIMO products.

I thought about calling this approach 'Poorman's BI' and 'Lagging-edge BI' but I will stick with another term that I have used in the past: 'Slow BI'. It is slow in a good way because I have seen this approach allow an organisation to build a level of support for BI that is solid, broadbased and delivering benefits that the organisation can measure and acknowledge.

Slow BI provides an enduring organisational and technological platform that focuses limited resources on delivering practical business capabilities. These become part of the operational business and decision making processes among people that are only dimly aware of business intelligence and information management. This is an important step towards the elimination of analytic ghettos and the creation of an information workforce.

Slow BI over the long term exceeds the benefits of the SIMO platforms as they frequently struggle to achieve widespread adoption outside of the analytic ghettos. Perhaps slow-and-steady beats the big-bang?

 

 ¹ Sadly Microsoft has recently tried to patent sparklines - despite the wide acknowledgement that Edward Tufte invented and defined sparkline as a term to describe "small, high resolution graphics embedded in a context of words, numbers, images".

Comments

[Courtney]

This figures from Harvest Road's puhbsiled 2007 Accounts Aus$20072006 Revenue2,749,3405,512,998 Profit-4,607,352-659,977show the dramatic shift from a 10% loss in 2006 to meltdown in 2007, and this extract from the Chairman's statement in the accounts put the Giunti acquisition in context: Earlier this year the Company embarked on a move from our traditional direct selling model to that of a partnership distribution model whereby the Company would leverage off existing client relationships.Unfortunately the take up by partners was slower than forecast and as a result the Company has been forced to reassess its strategic direction and funding requirements. Consequently HarvestRoad is looking at a number of commercialisation options in relation to its product suite, assets and intellectual property as was advised to the market on 4 September 2007. These include:* Continuing to negotiate a global marketing agreement with BAE Systems, its cornerstone customer for the “Distributed Publishing – Electronic Publishing Support System” as part of an overall services solution.* Discussions with other interested parties in providing solutions in the defence industry markets particularly in the UK and Europe.* Looking at joint venture partners or parties interested in acquiring our existing HarvestRoad Hive Learning Object Repository software to help leverage a total solution commercialisation strategy on a global basis.In the meantime we continue to manage our direct sales and existing partner resources to deal with our priority activities and successes in the US, French, UK, Norwegian and German markets and some very exciting opportunities with the State of California and the ministries of Defence in NATO.The directors remain confident that the HarvestRoad Hive intellectual property rights, the HarvestRoad brandingand its market positioning has significant value which is expected to be realised in the next six months.I thank shareholders for their continuing support and my fellow directors and our dedicated management and staff for their ongoing contributions.John McConnellChairman

[Taheerah]

Paula Weir - Julie change is aalwys good. I wanted to share our big news with you we are getting a new puppy (a whippet) the first week of June. It's been 2 years since our beloved Ranger passed and we feel we are ready to open our hearts to another dog in our family. We are looking forward to the change and want to thank you again for the FABULOUS pictures you took of Ranger.

[James @ BI Monkey]

I'm half in half out with Steve's view - though Steve i'd advise spending some time over at "Analyst First" - http://analystfirst.com/.

The reality is the tools may be cheaper (with a compromise that they are less functional), tool capabilities are more or less irrelevant to success.

Without the backing of good people who know how to use them, it makes no odds if you are using IBM Megaproduct or Open Source NicheProduct as your tool. You'll still waste time and money.

The megavendors have the advantage that they have more bodies who are skilled in their tools, so it's easier to find the capable architects and developers you need.

[vincent mcburney]

I work for an IBM partner so I am biased but I do not think it is as simple as picking the cheapest product. If that were the case then Telstra could move their Data Warehouse off Teradata and on to MS Access. The software evaluation process needs to take into account scalability, functionality, productivity and ease of use etc.

The SIMO vendors are aware of the price pressure of open source tools or emerging niche vendors and we have to be thankful for the way these smaller vendors are driving innovation and value for money. A couple years ago IBM introduced Cognos Express with the aim of getting smaller companies a wide range of BI and planning functions for less than $100K. This year they introduced the WorkGroup Edition for Information Server which offers DataStage, Glossary, QualityStage etc at 50% off list price plus whatever discount you can get during the evaluation process. The annual software maintenance can work out to be cheaper than the annual support license for open source tools.

I do not think "50% or more of the benefits at 15% or less of the cost" is good enough. If you are talking about a productivity tool like ETL or data profiling and you start with a tool with only 50% of the benefits you multiple this across half a dozen BAs and a dozen developers and testers and you find yourself spending an extra million dollars in permanent/contractor/consulting salaries and fees. Then there is the cost of a failed IT project or a solution that needs to be replaced or rewritten within three years. Providing governance and reducing risk is why all the major Australian banks use the full Information Server suite (to varying degrees).