Oz Analytics

Do you know what is bigger than Big Data?

From episode 10 of my Naked Analyst Channel on YouTube.

I think I do - and it is the ‘appification’ of analytics. What I mean by this is the reduction of a complex analytic activity such as market segmentation, down to a single button on your computer interface. Very much like the Apps on your smartphone, tablet or increasingly your desktop.

That’s what it looks like but the impacts are more profound. That’s because it makes it possible for analytics to be successfully done by people who may not understand how it works, but do understand the ‘why’ and ‘when’ they need to do it.

For example, a marketer in a company can access more sophisticated views of their campaigns without the need of a specialist analyst. Appification extends the range of analytic things that a non-specialist can do.

This appification is made possible because of three things that have emerged in recent years:

1. The rapid increase in the number and sophistication of APIs (Application Programming Interface).
2. The rise of open source analytic platforms like R (http://www.r-project.org). These platforms have created vast libraries of algorithms freely available to anyone who knows how to use the platform.
3. The shear number of people and organisations involved in creating open source analytic platforms like R.

The last enabler needs a little further explanation. R is a free software programming language and software environment for statistical computing and graphics. It contains thousands of packages (10,000?) specializing in topics like econometrics, data mining, spatial analysis, and bio-informatics. Nobody knows how many R users there are, but a reliable estimate (see http://spatial.ly/2013/06/r_activity/) puts it in the millions. Many thousands have helped R develop over the years.  I think that this sort of large-scale self-organising open source effort is beginning to teach the world how to use analytic algorithms.

The above is all supposition, but I can back this up with evidence. Here are 4 examples of algorithm markets - or at least they exhibit varying degrees of ‘algorithm marketness’.

• Dataxu - senses and reacts in real time to changing consumer behavior. Openness is at the technical core of DataXu. The DataXu platform is a flexible technology stack with open APIs that make it easy to integrate and extend functionality.
• algorithms.io - delivers machine learning for streaming data. The Algorithms.io cloud platform makes it easy to use machine learning algorithms to classify streaming data from connected devices. Turn the Internet of Things into the Internet of Action
• Snapanalytx - aim to provide predictive analytics for all and make them more accessible and affordable.
•  Algorithmia - are building a community around state-of-the-art algorithm development, where users can create, share, and build, on other algorithms, and then instantly make them available as a web service.

In conclusion, there is still one issue needing resolution before algorithm markets take-off: How will the world’s business people get data into and out of these algorithm apps? I’m not sure yet, but I think the answer will be more apps. Apps that themselves appify the transformation and loading of data into and out of algorithm apps.

Confused? Well don’t be, like most new and shiny things, what we are talking about is just the next generation of ETL - something business intelligence people like myself have been building for the last 20 or more years.

What’s old is new again? Maybe, but from my perspective the future looks exciting for analytics.

My YouTube channel contains tips from a 25 year veteran of the analytic profession.

How to Find Your First Job as an Analyst

Continuing on my YouTube Channel covering everything you wanted to know about being an analyst, business analyst, data scientist, and any other data-using role in business today.

Here is part 2 of my tips on how to land a job in analytics:

 

Part 1 was published a few days ago here and you can subscribe to the channel here as well.

As mentioned before, I've also put together a related playlist on YouTube of a few videos that include those made by others on similar topics that I have found on the net. 30 minutes of watching this whole list should give you some insight into how you can go about finding that first job. Good luck!

The Death of BIG Business intelligence

In the second decade of the 21st century the BI platforms from the SIMO vendors (SAP, IBM, Microsoft and Oracle) are dinosaurs.

A provocative statement - but bear with me.

Over the past few years I've blogged about a lot on the emerging trends of information management: real-time, BIG data, agile BI, actionable performance management and the like. The one thing that they have in common is that they are all big. Either big in terms of expensive or big in terms of the changes that they require within organisations.

I've tried to relate many of these trends to practical actions that (for those of us without large budgets) we can still extract value from. The context of these actions is that I have often been advising medium and large commercial businesses with existing business intelligence operations. So what I have experienced may not necessarily apply to government or small organisations.

Frequently my advice has involved technologies from one or more of the BI platforms from the SIMO vendors. SIMO and cheap are not words that go together. It's not a criticism of the platforms. They are what they are but I argue that their time has past because there are now a number of alternative approaches available that leverage newer low cost products and methods.

I have been increasingly looking to the benefits of open standards and web-based solutions to create low cost platforms without further investment in the SIMO vendor platforms. In the past I have used open source to extend SIMO platforms so that I didn't have to pay for the 'enterprise version' or 'additional module' that the SIMO vendors always seem to suggest when I want my platform to do something new.

An example of this was to extend our web-based dashboards to include sparklines¹. This type of graphic was not supported by the SIMO vendor product at the time but was easily extended using open source. It also saved the company the considerable expense of buying the enterprise dashboard for $250,000 + 20% annual support charge.

Now I have questioned the idea of using the SIMO platform at all. In the above scenario I today have invested in something like Tableau which although very definitely not based on open source code, is significantly cheaper to purchase and more adaptable to a variety of data sources and presentation types.

At this point, I will digress into more technical considerations. So if you become glassy-eyed at the mention of java, then it's best if you skip over the next couple of paragraphs and start reading about the example.

Open standards-based initiatives make our world of data so much more open than it once was. Everyone has heard of Big Data, but there are also dozens of open initiatives such as Mondrian (or more correctly: the Pentaho Analysis Services Community Edition) - an Online Analytical Processing (OLAP) server that enables business users to analyze large quantities of data in real-time.

Why do we care? Mondrian gives us an open standard way to access OLAP using Java on the client side. I'm not referring to XML/A the 'open' standard that Microsoft has been promoting for some years. XML/A is not truly open. Mondrian and Saiku should provide a true open driver capability to those of us building best-of-breed platforms. In time, I hope that market pressures will convince the SIMO vendors to build their own drivers for Mondrian.

A lot of brainpower has been invested over recent years improving database technologies so that they better support business intelligence activities. Combine this with the computing power now available on our desktops and in our data centres and clouds and I think you will find that best-of-breed technology stacks where you decide each layer will often do the job more sustainably and cheaper than an equivalent SIMO investment.

An Example

I can best explain the business logic behind my ideas by taking an example such as business MDM (master data management).

Old School

You purchase an MDM solution from the SIMO vendor you already have in-house. Let's take IBM in this case. So you will need to purchase the Websphere Business Glossary and probably the MetaData Server and Workbench. What's this going to cost? I'm guessing $1 million or more for an enterprise license once you factor in the specialist technical skills you will need to integrate it into your organisation.

IBM's Websphere Platform (courtesy IBM)

Now Websphere is a nice platform and the integration between DataStage, QualityStage, Information Analyzer and Business Glossary is a joy to have as a business manager of this stuff. I've just never had the business case to buy the entire platform. This is true of the majority of IBM platforms implemented across Asia-Pacific today. IBM is no exception and this is true of the other SIMO platforms. Budgets, capabilities and maturity levels almost always mean that only parts of a SIMO platform are implemented in an organisation. What a missed opportunity - but there is no point in crying over spilt milk. If the business case to implement entire platforms existed, then the SIMO vendors would be the first to let you know.

Microsoft is the exception to this rule, having a much cheaper licensing model. Unfortunately their MDM capability is very much a 'you build it' one, so the TCO (total cost of ownership) is still significant.

The challenge today is being smarter and more effective than past attempts. 

New School

Means that you leverage your existing investments - often those in web solutions.

For example, an MDM on-the-cheap solution would be to have your content management (CMS) platform hold critical business metadata. SharePoint is frequently the CMS of choice for large traditional corporations. Even if not already implemented, Using SharePoint to build a custom workflow and collaboration solution is pretty straightforward. So with a modest effort, the CMS can support an intranet-based community of business owners to create and maintain business metadata.

Making use of this business metadata is also possible. Continuing with the MDM scenario above, and assuming you use BusinessObjects, Cognos or similar, then you can export the business metadata easily to universes/cubes so that it is available to report writers, users and decision makers. This will drive the use of standard definitions of data and measures and make enterprise performance management easier.

We are also reaching the point where the presentation layer products (reports, dashboards and scorecards) from the SIMO vendors are now intelligently integrating back into - you guessed it - the CMS's like SharePoint. So production and consumption of business metadata is done on the most widespread platform available within the organisation.

This takes BI and IM out of the analytic ghetto and integrates it seamlessly into the corporate intranet - and beyond.

Cost? A conservative $100,000.

 Conclusions

What's the overall result? You get 50% or more of the benefits at 15% or less of the cost. Ongoing maintenance is also less. Most crucial of all is that the cheaper option is also the one most likely to be funded by an organisation. Why? Simple: it's CHEAPER.

This blog started with the statement that the SIMO vendors were dinosaurs. This is not quite true, but as my example above illustrates, it is now time to integrate your existing SIMO investments into your corporate intranet. Not by buying more SIMO products. Instead use more open technologies to integrate them into the intranet. Treat the SIMO-based data and report repositories as services to be consumed via a web front-end that does not require those extra SIMO products.

I thought about calling this approach 'Poorman's BI' and 'Lagging-edge BI' but I will stick with another term that I have used in the past: 'Slow BI'. It is slow in a good way because I have seen this approach allow an organisation to build a level of support for BI that is solid, broadbased and delivering benefits that the organisation can measure and acknowledge.

Slow BI provides an enduring organisational and technological platform that focuses limited resources on delivering practical business capabilities. These become part of the operational business and decision making processes among people that are only dimly aware of business intelligence and information management. This is an important step towards the elimination of analytic ghettos and the creation of an information workforce.

Slow BI over the long term exceeds the benefits of the SIMO platforms as they frequently struggle to achieve widespread adoption outside of the analytic ghettos. Perhaps slow-and-steady beats the big-bang?

 

 ¹ Sadly Microsoft has recently tried to patent sparklines - despite the wide acknowledgement that Edward Tufte invented and defined sparkline as a term to describe "small, high resolution graphics embedded in a context of words, numbers, images".

Wrap-up: Mastering SAP BusinessObjects Conference Day 2

I'm listening to the final presentation of the track I'm chairing. The topic was to have been about SAP BusinessObjects XI 4.0. Unfortunately the talk couldn't proceed as SAP hasn't yet released their product roadmap. What a pity. The SAP guy (Noam Berda) did say that you can be briefed later if you sign an NDA.

The day has been worthwhile as I learnt some useful techniques when using BusinessObjects Explorer (SAP's 'google-like' interface). It's a product that, along with their data integration tool, I will take a closer look at when back in the office. It could fill a gap in our current BI platform.

Andrew Rose of IBM Australia gave an insightful session on RapidMarts. These are products that provide 'reporting out of the box'. Not sure how much they cost. Andrew wasn't saying and a quick google didn't find an answer. So I'm guessing the price is 'not cheap'. Still they look interesting and may be worth looking at. Apparently they come with good and extensive documentation.

The Data Deluge

The Economist this week has an excellent 16 page special feature on 'The Data Deluge'. Well worth reading.

The Internet Of Analytic Things

The latest edtion of the McKinsey Quarterly contains an interesting article by Michael Chui, Markus Löffler, and Roger Roberts on The Internet of Things (registration required or download PDF below).

The authors main point is that some years after Vint Cerf (one of the founders of the internet) began talking about 'The Internet of Things' - it is now a reality: that now "the physical world itself is becoming a type of information system".

"In what’s called the Internet of Things, sensors and actuators embedded in physical objects—from roadways to pacemakers—are linked through wired and wireless networks, often using the same Internet Protocol (IP) that connects the Internet. These networks churn out huge volumes of data that flow to computers for analysis."

The authors go on to say that:

"as more sensors enter our devices and clothing, we will have the ability to monitor the behavior of people, places and things through space and time, enabling such business applications as presence-based advertising."

This is just the tip of the iceberg. Just think of the analytic opportunities that the widespread use of IP connected sensors represents. Sensor data is a great chance for analytics to move further and faster into decision support - unteathered from your PC and mobile. Real time analysis will be the only way for us to turn the flow of data around us into information we can action.

What we are talking about here is a fundamental change in our environment. To be more precise: the linking of two worlds is happening. Our information world (mostly the internet) and our physical world (the stuff we do when the broadband is down).

So far, the internet has largely been something that we join as we want. No one forces you to have a Facebook page or LinkedIn profile. We choose to participate (or not) based on the value that participation gives us. The Internet of Things changes this. You will not be able to control the actions of most of the sensors in your enviroment - and that includes the flow of sensor data. So opting-out in the future is going to be a lot harder to do than simply switching off your computer or mobile phone.

Above: The authors see six distinct types of emerging applications, which fall in two broad categories: 'information and analysis' and 'automation and control'.

The biggest surprise to me was the impact that sensors are already starting to have on the pricing of products and services. It's a logical result I had not seen coming. Here are a couple of examples from the article:

"... insurance companies, for example, are offering to install location sensors in customers’ cars. That allows these companies to base the price of policies on how a car is driven as well as where it travels. Pricing can be customized to the actual risks of operating a vehicle rather than based on proxies such as a driver’s age, gender, or place of residence."

"In the aviation industry, sensor technologies are spurring new business models. Manufacturers of jet engines retail ownership of their products while charging airlines for the amount of thrust used."

The examples discussed are, one the whole, all positive as they deliver something most people would argue is good: reducing cost and risk or enhancing products and services. The darker side is less well addressed - and I'm not talking about the fact that in the near future my socks are going to announce to the world how many holes I have made in them or when was the last time I washed them!

Right: Forget about body scanners at our airports. Sensors have the potential to make the whole world one large scanner.

I guess it comes down to how comfortable you are in having Governments and Corporations knowing more about both your online and physical activities. Maybe now is a good time to lock and load and move back to Southwest Montana!

Do you remember those little cell-phone blockers you could buy at your local spy shop in the 1990s? I admit to owning one - just so I could stop cellphone calls some of my fellow cinema goers felt were vital to answer during film screenings.  I wonder if there is a smart entrepreneur out there who is already developing a way of disabling sensors in our immediate environment?

The McKinsey article is a very good overview of emerging sensor applications and worth reading. Here's the article to download.

Defining Analytics: Data Warehousing

 Following-up to my blog 'Just Tell Me What I'm Doing', this is the second in a series of posts that define the key concepts and terms that make up my analytic world. Everything I do is coloured by my experience actually doing analytics in commercial organisations.

So while I believe these posts will present practical definitions that will be actionable in the business world, I know that there are other worlds in academia and science where they are less relevant. At the very least, people in these areas will gain a better understanding of how business regards analytics. Data Warehouse puritans should perhaps stop reading, take a pill and lie down instead.

---

Bennett's Analytica  
   A Practitioner's Guide To Analytics

---

Data Warehousing

---

The overall process of providing Information to support business decision making is referred to as either data warehousing or business intelligence. The terms are used interchangeably and often as acronyms like DW/BI, DWH, EDW, ODW and BI.

EDW stands for enterprise data warehouse. This is a warehouse that stores data about activities in many areas of an organisation. An EDW or DWH is typically used to support the strategic business requirements of an organisation.

ODW stands for operational data warehouse. This is a warehouse whose primary purpose is to analyse and report on operational (day-to-day) activities.

Data warehousing refers to not only the data warehouse itself (see below) but also the technologies and activities to put data into it, extract data from it, analyse the data, and inform users through reports, dashboards, alerts and data extracts.

(image copyright unknown)

ETL

Is the acronym of Extract, Transform and Load data into and out of a data warehouse. ETL, also referred to as data integration is an important area of analytics and deserves it's own section in any Analytica. Stay tuned for a fuller discussion in another post.

Briefly, ETL is the activity of:

• Getting data into a data warehouse:
• • Extract data from source system files. Source systems are usually the transactional systems of the organisation or are files that contain metadata required to assist in the analysis of the transactional data.
  • Transform the data ready to place it into a warehouse. Often multiple sources of data are involved and similar data from different sources must be modified (i.e. transformed) so that they have the same format and/or meaning. When this is completed the data is said to be conformed.
  • Load the conformed data into the tables of the data warehouse
• Getting data out of the data warehouse:
• • Extract data from the tables of the data warehouse.
  • Transform the data so that it is better organised to support the analytic purpose that the business requires to be performed.
  • Load the transformed data into another database (called a data mart) or file ready for analytic or reporting use. It's called a data mart because it conjures-up comparisons with a supermarket - a place where you can easily pick what you want to consume.

Just to confuse you, ETL is now a popular term for loading and extracting data from any data store or application - especially in IT middleware buses where data is moved between applications.

Data Warehouse

A data warehouse on the other hand is generally defined as a database where the data is stored in such a way that it facilitates reporting and analysis. Other databases in business tend to be places to store data for transactional use. This difference in purpose has a surprisingly fundamental impact on the way that data is stored and the tools used to get data into and out of the database.

Another way to think about the difference is to think of transactional use to be operational where the speed of recording a transaction is important. Data warehouses on the other hand are used to support decisions where speed of recording is less important than our ability to analyse. This is useful but just to confuse you, data warehouses can also be used for operational reporting and analysis.

Most data warehouses store data in rows within tables but a growing number use columns (see Columnar Data Warehouses below).

Data Mart

Data marts are data stores whose data covers specific business functions for a specific community within an organisation. For example a finance or sales data mart. Data marts are often subsets of data in a data warehouse. Data marts simplify analytics as they:

• Remove data that is not relevant to the intended type of analytic insight required by the business.
• Pre-calculate measures that are frequently needed. This means that the measures can be standardised, reports are simpler to design, and run more quickly as they don't have to do as much calculation.

Data Warehouse Appliances

A DWH Appliance is an integrated set of servers, storage, operating system, database and software that is-optimised for data warehousing. Most appliances today use massively parallel processing (MPP) architectures to provide high query performance and platform scalability. In this respect appliances use the same architecture of the traditional very large scale data warehouses from Teradata. Appliance vendors include Netezza and Greenplum.

Normalised versus dimensional Storage Models

A storage model is the way the data is stored in a database.

A normalised model is one where the tables are grouped together by subject area. Each area containing a data category. Examples are data on customers, products, and the like. The advantages of normalisation is that it is easy to load information into the database. The disadvantages are that the database can end up with a large and confusing number of tables. This often makes it difficult for users understand the data and it also slows down the retrieval of data.

A dimensional model is one where transaction data are separated into 'facts' and 'dimensions'. Facts are typically numeric and transaction data. Dimensions are the reference information that gives context to the facts.

An example (from Wikipedia) helps understand the difference:

"A sales transaction can be broken up into facts such as the number of products ordered and the price paid for the products, and into dimensions such as order date, customer name, product number, order ship-to and bill-to locations, and salesperson responsible for receiving the order."

The advantages of the dimensional approach are very big advantages. The first is that the data in the warehouse is easier to understand and use. Secondly, the retrieval of data from the warehouse is fast.

The main disadvantages of the dimensional approach are that making the data easy to understand comes at the cost of a more complex ETL load into the warehouse. This is because maintaining the integrity of the facts and dimensions is complex. The second disadvantage it that it can be difficult to change the data warehouse structure if the organisation changes the way it does business.

Another tricky aspect of the dimensional approach are Slowly Changing Dimensions (SCD). These are dimensions that have data that slowly changes (in case you didn't guess). An example best illustrates the problem:

A common dimension in a data warehouse is one that tracks the sales records of your company's salesforce. Reporting and analysing sales performance should be straightforward but what do you do when a sales person is transferred from one sales team to another. How do you record such a change in your sales dimension so that you remember the past and the present composition of your salesforce? This is important when you do all sorts of performance analytics.

Solving these (and a range of similar) issues involves methodologies referred to as Type 0, 1, 2, 3, 4, and 6 SCDs. The solutions to handling these types are outside of the scope of this post. Stay tuned for a fuller discussion in another post.

Alternative and Emerging Ideas

Columnar Data Warehouses

Wikipedia says it best:

"A column-oriented DBMS is a database management system (DBMS) which stores its content by column rather than by row. This has advantages for databases such as data warehouses and library catalogues, where aggregates are computed over large numbers of similar data items."

The Bus Architecture

A bus in computing is a subsystem that transfers data between computer components or between computers. It has a more specific meaning in data warehousing. Ralph Kimball, one of the founders of data warehousing explains it best. Here is my mashup of his ideas:

You can think of the conformed dimensions and facts of an enterprise as a standard set of connection points for applications — in other words, as a data warehouse bus architecture.

The corresponding bus matrix identifies the key business processes of an organisation, along with their associated dimensions. Business processes (typically corresponding to major source systems) are listed as matrix rows, while dimensions appear as matrix columns. The cells of the matrix are then marked to indicate which dimensions apply to which processes.

In a single document, the data warehouse team has a tool for planning the overall data warehouse, identifying the shared dimensions across the enterprise, coordinating the efforts of separate implementation teams, and communicating the importance of shared dimensions throughout the organisation.

Data warehouse bus architecture. The bus architecture is based on standardised dimensions and facts that let separate data marts, fact tables or OLAP cubes coexist and integrate. In this context, the term bus is not a large motor vehicle. Instead, it refers to the term's early meaning in the electrical power industry (a conductor for collecting electrical currents and distributing them) and commonly used in the computer industry to describe the standard interface specification that lets peripheral devices usefully coexist.

Ralph Kimball is credited with associating these fundamental bus concepts to the data warehouse delivery and presentation environment.

Related Terms and Concepts

Refer also to ODS, Operational Data Store

Refer also to Metadata

---

Comments? Via form below or send feedback to [email protected]         version 0.1 201002

---

Proof I Need A Holiday

OK, I admit it. I am a nerd toy junkie.

If it's new and microprocessor related I want one. Sorry, correction: I need one.

So it will come as no surprise that I own an iPhone and I keep an eye out for applications that do something innovative and analytic. There are now a number of very useful applications that are part of my work day - from BusinessObjects Explorer to Wolfram|Alpha. I have however learnt to be careful and avoid the seemingly vast number of iApps that serve no sane purpose at all. Peer group pressure has in the past done a good job of controlling my trigger-happy finger. I just couldn't resist going to the website for the iPhone Baby Tracker.

Could I now whack a microchip into the little tykes neck just like I did for Virgil our family labrador? Sit back and track their travels from the comfort of my armchair? Never miss another rerun of Gilligan's Island again? Fantastic - I'll take 2! And I wonder if the wife would agree ...

... but no. The reality was a bit of a let-down.

So I just don't get the idea of Trixie Tracker - the dashboard for your baby.

It does what it claims. You record every detail of babies life and it will generate insight. What sort of insight? From their website:

"Four Ways Diaper Tracking Makes Your Life Easier:

• Know the time of the last poopy diaper (and know when it's time to give prunes).
• Keep track of leaks and accidents. Figure out why and when accidents happen.
• Estimate diaper usage so you'll know when it's time to buy more.
• Encourage potty training because you are aware of the schedule."

Do we really need this level of information when we have a kid? I would have thought that this was exactly the period when you didn't have the time to be entering data on your iPhone.

Maybe I'm wrong and in any case, in my day, this would have been a job for Inga, the Swedish au pair.

How did I ever raise 2 children without visualising their nappy changes and bowl movements?:

 

Even assuming I needed this app - and that's a big assumption - what about getting serious about bowel movements? Where's the ability to record the type of movement using indistry standard categorisation by the Bristol Stool Scale? Or being able to include external data sets?

So come on Trixie Telemetry LLC and get serious about industry standard data classification and EDI! I want - no - I need to be able to import data sets from my toddlers pre-school classmates so that I can prove how regular my child's bowel movements are (and correspondingly) how sub-standard your offspring is.

And Trixie, please don't forget about the microchip in the neck idea. Think of the applications. I could tag my entire BI team and track their every movement. That'd redefine the notion of what a time and motion study could do. Forget Porter's 5 Forces, and welcome to the era of Bennett's 7 Movements. 

Seems like I have become that grumpy old man from the BBC TV shows so I'll stop now. Merry Christmas? Bah humbug!

Have a very analytic new year.

Postscipt, January 2010:

My brother (a doctor) was given a children's version of the chart. I can't resist posting it here:

Will The Cloud Democratise BI?

I read with interest the announcement that Canadian BI vendor startup Indicee has successfully completed a second round of venture capital funding. In brief, Indicee sells a cloud solution where you upload your data and it works out automatically the relationships - freeing you to concentrate on answering your business questions. Here's a video from Indicee summarising their service:

Indicee Movie

I don't know the terms of the financing agreement but the US$6 million raised is another sign that BI in the cloud is an area that we should all keep a track of.

Why The Cloud?

Here are four reasons:

1. Indicee was founded by Mark Cunningham and Fred Tummonds both originally with Crystal Services - makers of Crystal Reports. Mark was a developer of the original Crystal Reports product - possibly the most ubiquitous BI tool outside of Excel in the market today.

2. Their early seed funding included all of the original founders of Crystal Services.

3. This second round adds Granite Ventures who have also previously invested in companies such as:

• Biz360 - Hosted Analytical Applications
• C-Decisions (dba DecisionView Software) - Analytic Software
• Crowd Science - Online Marketing Research
• Lucid Imagination, Inc. - Open Source Search
• Oversight Systems - Analytic Software
• Purisma - Customer Data Management SaaS, since sold to Dun & Bradstreet
• Siebel Systems - Analytic Software, since sold to Oracle
• Skytide - Digital Media Performance Management Software
• Vignette - Content Management Software, now a public company.

4. Their pricing model. It's cheap and offers small and medium sized enterprises (SMEs) some real analytic grunt without the hassles of installing and running the software themselves:

Indicee is certainly not alone and there are other startups taking similar but distinctly unique services to market. For example, take a look at CloudSwitch, RightNow Technologies, RightScale or Good Data. Good Data offers real OLAP capabilities and runs on Amazon Web Services. It is also backed by an impressive list of people including Marc Andreessen, Tim O’Reilly, Esther Dyson and John Landry.

Many of these startups are building their services on top of the EC2 infrastructure provided by Amazon's web services.

New Choices

As we approach the end of the naughties the BI market is offering some interesting and fundamentally new choices to us:

• BI in the cloud
• SaaS
• Open source.

... and we can now choose to buy each of these services from all the big vendors (at big cost) as well as a raft of new entrants at substantially lower cost.

With prices like those from Indicee and others, we could be seeing a massive increase in the number of organisations making active use of analytics.

My Predictions?

It is still early days for SaaS and full cloud solutions - especially for larger organisations with mission critical BI. For the big boys SaaS and cloud BI will remain a niche option for a number of years. In 2010, of these 3 alternatives, open source will have the biggest impact by far.

SMEs will perhaps be the biggest beneficiaries of these market developments in 2010. As new adopters of BI they are more able to make use of the cloud - especially if the entry cost is less than AU$100 a month to launch your first analytic service.

Stay tuned.

DW 2.0 Asia Pacific Summit

I am happy to announce that the Business Intelligence Group is supporting the DW 2.0 Asia Pacific Summit to be held in Sydney at the end of October.

We are not only supporting the event but I am going to talk about 'Decision Mashups - Strategies for Combining Structured and Unstructured Data for Better Decision Making'.

Once again I make the offer for any readers of this blog, or members of the CORTEX to make contact so that we can meet up at the event.

Here is the brochure for the summit:

Download DW 2.0 Asia Pacific Summit Brochure

I also see that Eugene Dubossarsky from Presciient and moderator of our Forecasting Special Interest Group on the CORTEX is to speak on 'The Future Role of Analytics: Arms Races, Success Stories and the Changing Role of the Analys'. Eugene always has an interesting perspective and I'm looking forward to his talk.