Do you know what is bigger than Big Data?

From episode 10 of my Naked Analyst Channel on YouTube.

I think I do - and it is the ‘appification’ of analytics. What I mean by this is the reduction of a complex analytic activity such as market segmentation, down to a single button on your computer interface. Very much like the Apps on your smartphone, tablet or increasingly your desktop.

That’s what it looks like but the impacts are more profound. That’s because it makes it possible for analytics to be successfully done by people who may not understand how it works, but do understand the ‘why’ and ‘when’ they need to do it.

For example, a marketer in a company can access more sophisticated views of their campaigns without the need of a specialist analyst. Appification extends the range of analytic things that a non-specialist can do.

This appification is made possible because of three things that have emerged in recent years:

1. The rapid increase in the number and sophistication of APIs (Application Programming Interface).
2. The rise of open source analytic platforms like R (http://www.r-project.org). These platforms have created vast libraries of algorithms freely available to anyone who knows how to use the platform.
3. The shear number of people and organisations involved in creating open source analytic platforms like R.

The last enabler needs a little further explanation. R is a free software programming language and software environment for statistical computing and graphics. It contains thousands of packages (10,000?) specializing in topics like econometrics, data mining, spatial analysis, and bio-informatics. Nobody knows how many R users there are, but a reliable estimate (see http://spatial.ly/2013/06/r_activity/) puts it in the millions. Many thousands have helped R develop over the years.  I think that this sort of large-scale self-organising open source effort is beginning to teach the world how to use analytic algorithms.

The above is all supposition, but I can back this up with evidence. Here are 4 examples of algorithm markets - or at least they exhibit varying degrees of ‘algorithm marketness’.

• Dataxu - senses and reacts in real time to changing consumer behavior. Openness is at the technical core of DataXu. The DataXu platform is a flexible technology stack with open APIs that make it easy to integrate and extend functionality.
• algorithms.io - delivers machine learning for streaming data. The Algorithms.io cloud platform makes it easy to use machine learning algorithms to classify streaming data from connected devices. Turn the Internet of Things into the Internet of Action
• Snapanalytx - aim to provide predictive analytics for all and make them more accessible and affordable.
•  Algorithmia - are building a community around state-of-the-art algorithm development, where users can create, share, and build, on other algorithms, and then instantly make them available as a web service.

In conclusion, there is still one issue needing resolution before algorithm markets take-off: How will the world’s business people get data into and out of these algorithm apps? I’m not sure yet, but I think the answer will be more apps. Apps that themselves appify the transformation and loading of data into and out of algorithm apps.

Confused? Well don’t be, like most new and shiny things, what we are talking about is just the next generation of ETL - something business intelligence people like myself have been building for the last 20 or more years.

What’s old is new again? Maybe, but from my perspective the future looks exciting for analytics.

My YouTube channel contains tips from a 25 year veteran of the analytic profession.