Oz Analytics

Moving BI Into The Cloud Part 1

Over the last year I have been reading a lot about cloud computing and trying to predict how this can be used in business intelligence and analytics. I believe that the cloud is becoming relevant for a number of reasons:

• Customers - they're all on the net already. If I want to reach them I need to be there too. It's not just publishing information (web 1.0), it's also social networking (web 2.0) that I need to tap into. If I know what people are talking about and what they are thinking (via Facebook, MySpace, twitter, etc.) then the thousands of emerging online analytic applications are going to be essential for me to use. The cloud puts me in a better position to integrate into 3rd party analytic applications in real time.
• Speed - as the availability of adsl+ spreads, the speed of cloud based applications matches those installed on your own desktop or LAN. There is no performance cost of going into a cloud.
• SAAS - I predict that analytic mash-ups are going to become a force in business intelligence. Thousands of analytic Saas applications are available today.  in Australia we now have local companies offering Saas analytic applications. Although the range of applications is still limited, with many big names missing, Telstra's new T-Suite does give you access to: 
• • Microsoft Dynamics CRM (MDCRM)
  • Microsoft SharePoint - for virtual teams 
  • Microsoft Exchange Mail - email, calendars, contacts and task management 
  • Workforce Guardian Pro - contracts in-a-box 
  • WORKetc - a workflow/CRM application 
  • Telstra Remote Backup. 
• Migration - companies such as Amazon Web Services (Amazon S3) offer effective ways to move large amounts of data into clouds and also to physically backup cloud data. In partnership with IBM, Amazon also provides the infrastructure for your cloud.
• Reliability - Google mail going down makes the headlines worldwide - unlike outages to your own LAN or WAN. Ignore the publicity and you will see that most clouds have significantly better reliability. 
• Security - this is an important one often given as a primary reason not to move to a cloud. My counter-argument is that customers are moving online with or without you. If you want to stay close to your customers then the cloud gets you there. Most organisations already provide internet 'connections' to reach customers, other businesses and their own staff. Is the cloud really that different? Aren't you already very 'exposed' to the internet - and the associated security risks?

MDCRM has been available as a hosted app for some time now and MSFT list 17 companies worldwide that will host MDCRM for you. Presumably T-Suite is either cheaper or easier for SME's to get up and running with.

So far I don't see wide adoption of the cloud locally. Apart from the usual conservatism of Australian business, I think that the main reason is economics: the local offerings are expensive. Take an example of T-Suite: if a company wanted the following services:

• CRM, SharePoint and Remote Backup for 50 staff
• Email and calendar for 100 staff 

The cost of doing this on T-Suite would be 50 x (69.95 +35.95 + 18.00) + 100 x 16.95 = $7,890 per month. The minimum contract is 12 months, so the annual cost of your cloud is $94,680.

So if the local offerings are pricy, what about the big boys internationally? This is the Cloud after all, geography is irrelevant. What are the choices (actual and emerging)?

• Salesforce.com - SFDC, the granddaddy of Saas. Not so much a single CRM application as an ecosystem of over 100,000 custom applications. They have moved strongly into the cloud (they call it Force.com) and are now emphasising real time analytic capabilities. They are also moving into social networking - you can monitor Twitter sentiment about your products and services. They don't publish local pricing, but in the US the enterprise editions' standard cost is US$125 per user/per month.
• SugarCRM - the enterprise edition maximum cost is US$50 per user/per month.
• Sage SalesLogix - from the SME accounting software people. 
• Smart Market - just launched by IBM in the US is aimed at the SME market. Analytic related offerings include:
• • RJS Software Systems:
  • • Batch Report Server
    • System i:
    • • Report Splitter
      • Database Integrator (RPG2SQL)
      • iForms Electronic Forms and Business Intelligence
    • Email Report Server 
    • Enterprise Report Management Solution 
    • Enterprise Workflow 
    • RJS Report Converters (PDF, HTML, Excel, RTF/Word) 
    • Webspool 
  • Cincom:
  • • Business Intelligence
    • Synchrony Express Contact Center 
  • C2CRM - from ClearC2, a web CRM 
  • ShopWorX Smart ERP 
  • Xperia Comprehensiv ERP 
  • OpenPro ERP
  • IBM:
  • • Meta Data Modeler and Data Bridge for Lotus Notes  
    • WebDocs Document Management 
    • SOPRSANO - mobile messaging on the IBM Smart Cube 
  • Quickbooks (many editions) 
  • GEDSYS IntraWare Easy CRM - address and master data management.

For an idea of the analytic richness of the cloud, take a look at Salesforce.com's enterprise capabilities:

• Relational Database
• Record and Field-level Security 
• Data Loader 
• SAP R/3 Conector 
• Sharing Framework
• Application Integration
• Workflow Rules
• Formulas
• Email Alerts & Approvals
• Search
• Page Layout Editor
• Reporting & Dashboards
• Analytic Mash-ups 
• Read-only Report Access 
• Multi-currency
• Multi-language
• Force.com
• IDE Visualforce
• Offline Client

The list goes on and on. So it may be even more expensive than T-Suite but SFDC has so much more power they are really not comparable.

So what can you conclude?

I can't speak for everyone but I'm ready to use the cloud professionally. I have privately been using Apple's MobileMe for years. But professionally the cost of the big boy's offerings drives me to open source and to take a best-of-breed approach where I look at specific online SAAS offerings - mostly niche players that offer specific analytic services and an API I can use to automatically exchange data and results with.

I think that I can build my own version of a cloud ecosystem to combine analytic capabilities in unique ways. I'm not alone in this and I hope to have news of a new analytic venture later this year.

Moving BI Into The Cloud Part 2

Here's another cloud angle to contemplate: IBM is now to offer BPM BlueWorks as a cloud-based BPM service for free (but there is a catch). Check it out.

Using it, you can model strategies and metrics and then produce capability maps (based on IBM's Component Business Modeling methodology) and process maps (which can be expanded into detailed BPMN models. Your models and other artifacts are managed online with Rational Asset Manager. All of your activities are managed and stored within your private area of BlueWorks and hosted online in ibm.com.

So BlueWorks gives you a repository-based business modeling environment in a team collaboration space. It goes a couple of steps further by adding in industry-specific content that you can use as cookie cutters to rapidly build your models. If you use  Lotus Connect, you can also involve people outside of your BlueWorks team to participate in the effort.

With these BPMN (Business Process Model Notation) models, you can export a model into WebSphere Business Modeler - either on your desktop or remaining in the cloud using the SOA Sandbox.

The result is that you can collaboratively model business processes and get them into WebSpere to see how they work in a development testbed. All of that is free.

The catch - and it's not unreasonable - is that you have made an investment in the IBM toolset that should lead you to purchasing additional IBM modeling, database and web services. I'm not sure about how you would actually do it, but there should be ways to use the exported BPMN model out of BlueWorks and import it into any other BPMN conversant tool.

I am aware that there are also a number of BPMN to BPEL (Business Process Execution Language) tools so sticking to open standards should be possible. Here's an old (2005) conference paper on the subject (Download Integration of BPEL2).

I have used Rational, DB2, and WebSphere myself and they are very viable BI tools. Add in Cognos as the new boy at IBM and you have one of the few end-to-end enterprise BI platforms around. Oracle, SAP and Microsoft being the others.

As IBM states, their goal is to allow BlueWorks users to:

• Capture and collaborate on BPM Business Designs in the cloud including strategy maps, capability maps, and process maps - starting from scratch, leveraging prebuilt templates, or leveraging contributions from the community
• Discover and contribute industry-specific BPM content that covers a full spectrum of BPM strategies, trends, capabilities, and best practices for making the smartest business decisions around BPM
• Participate in the community by blogging, creating local Meetups, and chatting with BPM experts
•   
• It's not live yet, but should be by the end of June. 
•   

I'm Afraid I Can't Let You Do That Dave

When you've got a hammer the whole world is a nail - so goes the saying.

So I was wondering if this is the case at Google as they continue to grapple with a serious brain drain of their best staff.

The latest reports are that Google has developed an algorithm to predict which of their staff are most likely to quit. Details are very difficult to come by but here are the features of the algorithm that I have read about in various reports:

• it mines employee reviews, promotion and salary histories
• attempts to identify employees who feel 'underused' 
• inputs also include information from surveys and peer reviews

Google's algorithm helps the company "get inside people's heads even before they know they might leave," said Laszlo Bock, who runs human resources for the company. This data crunching supplements more traditional measures like employee training and leadership meetings to evaluate talent.

Is Google using this to categorise employees into other performance groups? Is this another take on the old six sigma (forced) ranking of the bottom 10 percent of employees?

You could easily argue that almost 5 years on from Google's IPO, this drain is normal and expected from a highly successful company. Way back in April 2007 it was reported that:

"Hundreds of the 2,300 Googlers hired before the Internet juggernaut's initial public offering in August 2004 are hitting their fourth anniversary. When they do, they'll be free to cash in the final portions of their pre-IPO options, collectively worth an estimated $2.6 billion before taxes."
San Jose Mercury News

Presumably the employees leaving were all wealthy from their stock grants and options as under Google's vesting rules, employees typically earn the right to cash in 25 percent of their options when they hit their first anniversary, then the rest in monthly chunks over the next three years. So August 2008 was the month many had earned full cash in rights and could leave with no penalty. Add to that the feeling that your personal contribution is diminishing as the company grows and matures. The desire the seek greener pastures in other companies or exciting start-ups is hardly unexpected.

All credit to Google for being proactive however. If you can retain even a handful from leaving then it must be a worthwhile effort. It should be no surprise that Google is a trend setter here. How many companies do you know with a Chief Culture Officer and the 10 Golden Rules for knowledge workers:

1. Hire by committee. Virtually every person who interviews at Google talks to at least half-a-dozen interviewers, drawn from both management and potential colleagues. Everyone's opinion counts, making the hiring process more fair and pushing standards higher. Yes, it takes longer, but we think it's worth it. If you hire great people and involve them intensively in the hiring process, you'll get more great people. We started building this positive feedback loop when the company was founded, and it has had a huge payoff.
2. Cater to their every need. As Drucker says, the goal is to "strip away everything that gets in their way." We provide a standard package of fringe benefits, but on top of that are first-class dining facilities, gyms, laundry rooms, massage rooms, haircuts, carwashes, dry cleaning, commuting buses—just about anything a hardworking engineer might want. Let's face it: programmers want to program, they don't want to do their laundry. So we make it easy for them to do both.
3. Pack them in. Almost every project at Google is a team project, and teams have to communicate. The best way to make communication easy is to put team members within a few feet of each other. The result is that virtually everyone at Google shares an office. This way, when a programmer needs to confer with a colleague, there is immediate access: no telephone tag, no e-mail delay, no waiting for a reply. Of course, there are many conference rooms that people can use for detailed discussion so that they don't disturb their office mates. Even the CEO shared an office at Google for several months after he arrived. Sitting next to a knowledgeable employee was an incredibly effective educational experience.
4. Make coordination easy. Because all members of a team are within a few feet of one another, it is relatively easy to coordinate projects. In addition to physical proximity, each Googler e-mails a snippet once a week to his work group describing what he has done in the last week. This gives everyone an easy way to track what everyone else is up to, making it much easier to monitor progress and synchronize work flow.
5. Eat your own dog food. Google workers use the company's tools intensively. The most obvious tool is the Web, with an internal Web page for virtually every project and every task. They are all indexed and available to project participants on an as-needed basis. We also make extensive use of other information-management tools, some of which are eventually rolled out as products. For example, one of the reasons for Gmail's success is that it was beta tested within the company for many months. The use of e-mail is critical within the organization, so Gmail had to be tuned to satisfy the needs of some of our most demanding customers—our knowledge workers.
6. Encourage creativity. Google engineers can spend up to 20 percent of their time on a project of their choice. There is, of course, an approval process and some oversight, but basically we want to allow creative people to be creative. One of our not-so-secret weapons is our ideas mailing list: a companywide suggestion box where people can post ideas ranging from parking procedures to the next killer app. The software allows for everyone to comment on and rate ideas, permitting the best ideas to percolate to the top.
7. Strive to reach consensus. Modern corporate mythology has the unique decision maker as hero. We adhere to the view that the "many are smarter than the few," and solicit a broad base of views before reaching any decision. At Google, the role of the manager is that of an aggregator of viewpoints, not the dictator of decisions. Building a consensus sometimes takes longer, but always produces a more committed team and better decisions
8. Don't be evil. Much has been written about Google's slogan, but we really try to live by it, particularly in the ranks of management. As in every organization, people are passionate about their views. But nobody throws chairs at Google, unlike management practices used at some other well-known technology companies. We foster to create an atmosphere of tolerance and respect, not a company full of yes men.
9. Data drive decisions. At Google, almost every decision is based on quantitative analysis. We've built systems to manage information, not only on the Internet at large, but also internally. We have dozens of analysts who plow through the data, analyze performance metrics and plot trends to keep us as up to date as possible. We have a raft of online "dashboards" for every business we work in that provide up-to-the-minute snapshots of where we are.
10. Communicate effectively. Every Friday we have an all-hands assembly with announcements, introductions and questions and answers. (Oh, yes, and some food and drink.) This allows management to stay in touch with what our knowledge workers are thinking and vice versa. Google has remarkably broad dissemination of information within the organization and remarkably few serious leaks. Contrary to what some might think, we believe it is the first fact that causes the second: a trusted work force is a loyal work force.

Don't Be Evil is the one that often gets the headline, but the next one down is just as radical in my view: Data Drive Decisions. Here is the whole article from Eric Schmidt, CEO of Google.

What a pity that other companies are not turning to analytics for the same reasons.

Media reactions have been mixed. Some outright hostile. I may be biased but I don't think that using quantitative analysis of 'HR' data is a bad thing. If anything it should help to remove the bias of human interpretation when reviewing performance. We call it 'fact based decision making' everywhere else. What's so special about HR? I'm not saying that it is a universal panacea but used in conjunction with other things like encouraging supportive cultures and inspirational management styles.

It also struck me that Google should perhaps be concentrating more on those employees who are most under-used as an end in itself. Making better use of existing people must have a positive impact on the performance of the company.

A roll call of brain drainers is available here if you are interested. As recently as last week Tom Phillips, the head of Search and Analytics was rumoured to have left Google.

Gartner's View That Search Is Not Business Intelligence

I was rummaging through my knowledgebase of BI stuff this morning and came across a curious news report on Gartner from late last year. Here's the bit that caught my eye:

So the jist of the claim is that search is not business intelligence (or ERP). I agree with the ERP bit but I think it is odd to split Search and BI. Sondergaard says "this new trend in business IT should not be confused with 'business intelligence' which refers to the collection, integration and analysis of business information to make decisions and forecasts". Sounds like the same reasons you use search in an organisation.

One prediction I am confident to make is that Search and BI will steadily become synonymous as practical distinctions disappear between structured and unstructured data. This is already happening today as better methodologies and algorithms emerge that have the sophistication to deeply understand unstructured data. You only have to experiment with Google to get how smart the googlesphere is getting.

Even my personal knowledgebase of BI stuff has become a vital part of my professional life. I use Devonthink Pro to manage a huge variety of BI stuff - currently running to over 250,000 objects from MS Office documents to podcasts, pdf's and 100 other formats. Searching it is a breeze and the ranking of results is very good. Good enough to make it one of the core tools I own and use.

Sure beats having a good memory (not that I have that choice!).

So here is my 2 cents on Search and BI:

• Both structured and unstructured data will be vital 'grist in the BI mill' 
• Powerful algorithms will enable realtime mining of unstructured data into BI reports and dashboards 
• Where such realtime processing is not feasible (because of slow response times or data integration bottlenecks) then batch processing of unstructured data to create structured data will happen 
• Search of unstructured data within an organisation will be achieved by using algorithms developed and refined first by public use of search engines like Google.
• Mining public (or industry shared) structured and unstructured data will be as important as mining unstructured data within an organisation
• XBRL will be a major contributor to the structuring of publicly available corporate data in Australia - but as always we will be at least 5 years behind the USA in this.  

Here is the whole Sondergaard article if you are interested. Download 200811_Sondergaard_on_Search

Is WolframAlpha The Next Big Thing In Analytics?

A media buzz has started last weekend with the public launch of WolframAlpha. If you haven't heard about it yet, WolframAlpha is a new web search engine. What makes it the focus of media attention is that it is the brainchild of British physicist and mathematician Stephen Wolfram - the man behind Mathematica.

Search engines come and go but for several years now Google has dominated web searches in Australia and the rest of the world. WolframAlpha's claim to fame is that it can answer questions that Google can't: those that require higher levels of natural language processing and integration of data sets. This is best explained using an example: Say you want to compare the financial performances of IBM with those of SAP. Using Google or Yahoo! (for example) you can build mini-profiles of each company using stock market and SEC filings. WolframAlpha goes a step further by throwing more data sets into the mix. Here is the basic IBM versus SAP result from WolframAlpha:
WolframAlpha-SAP IBM Search Result PDF

WolframAlpha enriches the search engine findings in two important ways. Firstly it mines more 'specialist' data sets and integrates them. In this example the potential data sets used included:

 

Figure 1: WolframAlpha Sources

In this example, this means that fundamental analysis can include:

• Common ratios
• Balance Sheet 
• Income Statement 
• Cash Flow Statement.

So the data set available to the user is richer (more data = richer): 

Figure 2: Fundamentals Analysis

Secondly it leverages functionality from the Mathematica analytics package. So the data set available to the user is more powerful (more statistical tools = powerful). This means that you can slice and dice the data in more flexible ways:

Figure 3: Volatility Analysis Figure 4: Portfolio Optimization

There is much more to WolframAlpha and it is still early days.
I expect it will be a while before the full power of WolframAlpha is apparent.
The example I have used is typical of the analytics I make use of but I suspect that the impact of WolframAlpha
will be most apparent to 'simpler' questions from the general public.
Questions like:

• How far is San Francisco to Tokyo? View the search result
• Tell me about Hurricane Dolores in 1966. View the search result
• What were the Detroit Lions rushing yards per game? View the search result

The actual service still feels like a beta and response times can be frustrating but I expect that these are teething
problems that will be resolved. Competing against Google must be an incredible challenge.
It's like trying to beat Microsoft. Maybe WolframAlpha is the future Apple/Linux of the search world?
My take on WolframAlpha is that it is potentially a great new addition to web-based analytics. Whether-or-not it is
the next big thing will depend on how quickly it can evolve into a polished end-user service. The potential is clearly there
as the founder has a solid track record in building Mathematica into a leading product.

The next year of search engines just got a lot more interesting.

There's an interesting podcast discussing 'Social Networking Analysis at Optus' that you can listen to here:

This is from Teradata's in-house iQ magazine in Australia and their synopsis of the podcast is:

"Tim Manns from Optus, discusses how the company uses detailed network data from its Teradata system to look at calling behaviour. With 40% of the Australian telecommunications market, the company cross-references each customer with every other customer, groups them together based on who they communicate with, looks at the behaviour of the group, and can then predict next steps and target those groups with appropriate products and services."

Although this is a new initiative for Optus, the interesting information is in the discussion about how Optus is grouping their customers to predict their collective behaviour in regard to churn and buying. Optus has seen that if one customer leaves Optus, then one of the rest of the group is 4 or 5 times more likely to also leave. Knowing this, the CRM can trigger an event to start a marketing activity to retain the group members

In terms of marketing, Optus uses these groups to market offers that include 'circle of friends' that suggest a list of the friends the subscriber may want to include.

Getting down to the detail, Tim Manns talks about how Teradata is used to process (parse and clean telephone numbers) 7 days of data (550 million rows) in about 30 minutes. This is then matched to additional data such as customer spend, address and other call details.

Interesting stuff and credit to Teradata for organising a local case study.

Google makes much of its friendliness towards open source. Some may say this is to distract people from noticing their virtual monopoly of search on the net. I'm not so cynical as there seems to be a lot of competitive activity in search these days - that plus the fact that Google continues to innovate and deliver great products and services.

Here is an interesting leaked internal presentation from Google. Take a look at page 8 where their internal 'Data Liberation Front' team is discussed.

Download Googlepresentation

They even have a manifesto that you can read on their Google Code Wiki (where else?!). Trouble is, there's nothing more than the manifesto there. Nothing about what they are actually doing.

Of more use are the tweets from a team member.

On the whole this seems like an interesting initiative as they seek (no pun intended - seek is a job search site in Oz) to liberate more data sets. More power to them.

Fellow blogger John Gordon has done some digging about this team and you can read about it on his blog Gordon's Notes.

I read with great interest a recent article on the departure of Telstra's Senior vice-president of transformation - Tom Lamming. This Accenture partner had been taken into the company by CEO Sol Trujillo as a part of the US Executive influx at the start of his time at Telstra. He has now resigned from Telstra and returning (I imagine) to Denver and Accenture.

Lamming started out at Telstra as a consultant where he was paid up to $15,000 a day. Presumably less after he joined Telstra as an employee. What do you get for that sort of money?

But first, what was he paid to do? According to reports, his stated objectives were: a $12 billion, five-year transformation program that sought to develop common business processes for Telstra's operating support, customer care and billing systems. The aim was to save the telco hundreds of millions of dollars as 1,500 legacy systems were consolidated down to 'between 250 and 300 by 2010'.

This was all part of Trujillo's business transformation of Telstra. He talked about the network transformation at Accenture's Global Convergence Forum 2007. It gives you an idea of the scale of transformation being attempted:

Accenture Global Convergence Forum 2007, 11-13 April, 2007 in Rome, Italy. 'Driving Transformation Through Innovation' by Sol Trujillo.

So perhaps this was a good investment for Telstra - it certainly should be if the transformation delivers.

What I am particularly interested in is the interim report card for the transformation project 4 years into the 5 year project. Here it is as far as I understand it:

On the plus side:

• Seven million Telstra private customers have migrated to the new platform (an application that crashes a lot is still an application)
• 400 of the 1,500 legacy applications have been retired (with one year to go).

And on the negative:

• Only 250,000 business customers had been moved across
• Reportedly, the budget is blown by more than $1 billion
• Of the 1500, by 2010 Telstra hopes to reduce that figure by 80 per cent. But with a year left, only 400 have been switched off.
• More than 3,300 change requests have been issued
• Related to the last point, the 'fixed price' contract with Accenture has reportedly blown out from $350 - $400 million to almost $600 million. Plus there must be a significant cost to keeping all of the remaining 1,100 legacy applications running
• Presumably no net savings have been made so far (given all of the figures reported).

On the face of it, this is a pretty poor result. Don't be fooled: it is a poor result. No wonder CEO's, CFO's and CIO's are so worried about starting major transformation projects like this. 

Telstra are not alone in achieving this level of poor performance, but it makes me weep when avoiding these problems is so easy.

Can anyone spell 'fiscal discipline' anymore?

So where was the effective governance in this program? Why was such a transformation project agreed to in the first place if governance wasn't the number one priority? Apparently performance tracking of the project was missing - or at least ineffective.

Transformation is always difficult. You succeed by breaking the problem down into small, measurable chunks that are delivered on an agreed schedule throughout the life of the project. Big bangs are bad. Period.

When you've done this you then put into place people with the backbone to govern the project and demand that the project deliver as promised - and if they don't then to understand why and judge whether-or-not investment continues.

Why is this so difficult?

This may be why I am a turn-around specialist rather than a permanent line manager. That and the fact that I spend a lot of time on governance and proving benefits as they are delivered. You'll have to take that as true for now - but please check with people I have worked with and for. I hope that they back me up on this.

The point about all of the above is that hopefully Telstra will learn something from this experience and bring the transformation project to a successful conclusion. I for one will cheer for a Project Manager that delivers this outcome. Even at a starting rate of $15,000 a day - that would still deliver bottom-line profits to Telstra.

I'll be brushing-up on my American accent for my next interview!

I'll leave you with Mr Lamming's own opinion as reported in the papers:

Quote:

"Telstra's transformation is the largest, most comprehensive and successful I have ever seen, and we have implemented it at speed," Mr Lamming said.

May he be as successful in a future engagement - but hopefully not in Australia!

Check back here in a year's time to see the end result of the 5 year program and the final report card.

Several weeks ago I came across a great video from Microsoft on the history of business intelligence. The author is Nic Smith who is from Microsoft BI Solutions Marketing.

Here it is:

 

If you have just watched it, then you will have seen that it is leading into what Microsoft sees as the future of BI. Based on the clarity of vision and communication in the video, is this another indication that the traditional mega-vendors (IBM, Oracle and SAS) should be worried?

The original is found online here.

Recently I have just finished a long stint building an analytics capability in a large organisation. This has given me time to reflect on the experience and I'd like to share some of my conclusions. First off, the analytic capability is very successful and is driving better decisions in many areas of the organisation. That's the easy bit to say - and I would say that anyway :). What's more interesting are the lessons that I have learned - because they will help me do better next time. What are the lessons and how do I intend putting them into practice?

Lesson #1 - Get the Data Right You can't achieve advanced analytics if the data isn't right. Investing in high performing Business Analysis and Data Integration teams is the first step towards delivering a high performing organisation. I say teams but even one of each is enough to kick-start things. I also mention two teams because together they give you an unbeatable combination of analytic power within your team. Lesson #2 - Deliver Fast and Avoid 'Big Bangs' If you are delivering a complex thing like an enterprise self-service analytic capability then you have to deliver value much faster than is possible. How do you overcome this? The method I use is to identify existing information problems that are long standing issues. They are never hard to find and the application of a little logic and creativity invariably delivers a tactical solution that you can implement in a matter of months. We found a number of these problems (over 40 if I remember) and we delivered solutions to 20 or more in the months before we were able to deliver the enterprise data warehouse and self-service elements.

Just as importantly, the deliveries were spread consistently over the 2.5 years so that each quarter we could point to concrete benefits.

Lesson #3 - Deliver to the Direct Beneficiaries of Quality Data There will be one or two groups within the organisation that can benefit directly from the high quality data that the Data Integration team make possible. The key point here is that these groups do not need the presentation layer (reports, dashboards, etc.). This makes it much quicker to deliver. Find out who they are and make them the second customers you deliver to. In this case, Risk Management and Securitisation were departments that were identified and fit this bill. Delivering to these areas also gave us time to build the self-service analytic tools to other areas. Lesson #4 - You've Got To Build An Exclusive Club In the corporate world (the 'big end of town' we say in Australia) you are faced with people with a range of skills and talents. Some very, very good people with great skills and attitudes that I admire. The majority quite capable and with leadership can also deliver exceptional performance. The remaining 10% however are just not good enough. Nothing surprising in that I hear you say? Well I disagree. I spent 10 years starting, growing and selling start-up companies and the 10% 'not good enough' don't last long in that environment. This makes a fundamental difference in the speed at which a team can deliver. Just as importantly - it's less fun for everybody. I think that you should have fun, be proud of the work and learn from your team. All of these things are diminished when you don't have a high performing team. The organisation I worked for had an admirably cooperative and positive culture. To illustrate: you could approach just about anyone in the company and ask them a question and their first thought would be "How can I help this person?". Fantastic! So what is the lesson? I let the 'cooperative and positive culture' influence me to keep the 10% on in their jobs. On reflection, it would have been much better for all involved if I had done what I usually do: quickly and fairly get rid of them. Delaying or avoiding doing this hurt the performance of the analytic team I was building. Ultimately this also hurt our ability to get analytics adopted by the wider business. Lesson #5 - New Tools Are Required My final lesson has to do with the underlying technologies employed. I inherited a set of 'big tool' platforms (you know - the ones like IBM, Oracle and SAP). Early on I decided to keep them and go the upgrade path. I think this was wrong in a couple of areas - mainly around reporting and end user analytics. I should have looked at more agile alternatives. Ultimately I believe they would have been easier and cheaper to deploy. What are these alternatives? Here's is a list of a few that have caught my attention:

Open Source - tools are ready for the big time. Don't hesitate, start experimenting with them today.

QlikView, Zap and others - support more rapid deployment of analytics than the mega-platforms like IBM, Oracle and SAP.

Cloud9 Analytics - a role-based, on-demand business intelligence applications using a software-as-a-service (SaaS) model.

eThority - a web and desktop based, user-focused, accessible and scalable approach to business-driven analysis of enterprise data.

illuminate - a correlation database that brings agility to the back-end data integration tasks that are barriers to agile analytics.

Lyzasoft - depolarizing and “filling the middle” with desktop-based data gathering, data analysis, reporting and analytic publishing.

Netmap - a visual approach to discovery-based analysis.

PolyVista - extending OLAP with prepackaged, easy-to-use data mining and discovery automation capabilities.
Lesson #6 - Ignore Service Providers and Recruitment Consultants There are many good BI specialist Service Providers and Recruitment Consultants out there and they all want to develop a relationship with you. This is great and they can add value but when you are doing what we were doing there is not enough money or time to allow relationships to develop.

Our (my) strategy was to use a team of existing staff, a number of new specialist hires and specialist contractors. If I couldn't find the new hires and specialist contractors in my own network, then Service Providers and Recruitment Consultants were a vital component of the strategy. My existing network ensured that key appointment could be made with confidence. It certainly wasn't all 'plain sailing' but it worked well enough. We did go to market via recruitment agencies for a number of roles and the results were good. The success rate through agencies over the 2.5 years was about 50% (1 out of 2 hires were successful).

In hindsight, a lot of time was spent with lots of Service Providers and Recruitment Consultants. It would have been very useful to hear from my own network which ones they already knew were good. I still wonder why I didn't do more reference checking. The lesson is to be more disciplined in what you spend time on. Building an analytic capability is a full time job. So be firm and resist the chance to have 10 coffees a day!

Social Call Behaviour Analytics At Optus

There's an interesting podcast discussing 'Social Networking Analysis at Optus' that you can listen to here:

This is from Teradata's in-house iQ magazine in Australia and their synopsis of the podcast is:

"Tim Manns from Optus, discusses how the company uses detailed network data from its Teradata system to look at calling behaviour. With 40% of the Australian telecommunications market, the company cross-references each customer with every other customer, groups them together based on who they communicate with, looks at the behaviour of the group, and can then predict next steps and target those groups with appropriate products and services."

Although this is a new initiative for Optus, the interesting information is in the discussion about how Optus is grouping their customers to predict their collective behaviour in regard to churn and buying. Optus has seen that if one customer leaves Optus, then one of the rest of the group is 4 or 5 times more likely to also leave. Knowing this, the CRM can trigger an event to start a marketing activity to retain the group members

In terms of marketing, Optus uses these groups to market offers that include 'circle of friends' that suggest a list of the friends the subscriber may want to include.

Getting down to the detail, Tim Manns talks about how Teradata is used to process (parse and clean telephone numbers) 7 days of data (550 million rows) in about 30 minutes. This is then matched to additional data such as customer spend, address and other call details.

Interesting stuff and credit to Teradata for organising a local case study.

Google and Open Source

Google makes much of its friendliness towards open source. Some may say this is to distract people from noticing their virtual monopoly of search on the net. I'm not so cynical as there seems to be a lot of competitive activity in search these days - that plus the fact that Google continues to innovate and deliver great products and services.

Here is an interesting leaked internal presentation from Google. Take a look at page 8 where their internal 'Data Liberation Front' team is discussed.

Download Googlepresentation

They even have a manifesto that you can read on their Google Code Wiki (where else?!). Trouble is, there's nothing more than the manifesto there. Nothing about what they are actually doing.

Of more use are the tweets from a team member.

On the whole this seems like an interesting initiative as they seek (no pun intended - seek is a job search site in Oz) to liberate more data sets. More power to them.

Fellow blogger John Gordon has done some digging about this team and you can read about it on his blog Gordon's Notes.

How Not To Transform

I read with great interest a recent article on the departure of Telstra's Senior vice-president of transformation - Tom Lamming. This Accenture partner had been taken into the company by CEO Sol Trujillo as a part of the US Executive influx at the start of his time at Telstra. He has now resigned from Telstra and returning (I imagine) to Denver and Accenture.

Lamming started out at Telstra as a consultant where he was paid up to $15,000 a day. Presumably less after he joined Telstra as an employee. What do you get for that sort of money?

But first, what was he paid to do? According to reports, his stated objectives were: a $12 billion, five-year transformation program that sought to develop common business processes for Telstra's operating support, customer care and billing systems. The aim was to save the telco hundreds of millions of dollars as 1,500 legacy systems were consolidated down to 'between 250 and 300 by 2010'.

This was all part of Trujillo's business transformation of Telstra. He talked about the network transformation at Accenture's Global Convergence Forum 2007. It gives you an idea of the scale of transformation being attempted:

Accenture Global Convergence Forum 2007, 11-13 April, 2007 in Rome, Italy. 'Driving Transformation Through Innovation' by Sol Trujillo.

So perhaps this was a good investment for Telstra - it certainly should be if the transformation delivers.

What I am particularly interested in is the interim report card for the transformation project 4 years into the 5 year project. Here it is as far as I understand it:

On the plus side:

• Seven million Telstra private customers have migrated to the new platform (an application that crashes a lot is still an application)
• 400 of the 1,500 legacy applications have been retired (with one year to go).

And on the negative:

• Only 250,000 business customers had been moved across
• Reportedly, the budget is blown by more than $1 billion
• Of the 1500, by 2010 Telstra hopes to reduce that figure by 80 per cent. But with a year left, only 400 have been switched off.
• More than 3,300 change requests have been issued
• Related to the last point, the 'fixed price' contract with Accenture has reportedly blown out from $350 - $400 million to almost $600 million. Plus there must be a significant cost to keeping all of the remaining 1,100 legacy applications running
• Presumably no net savings have been made so far (given all of the figures reported).

On the face of it, this is a pretty poor result. Don't be fooled: it is a poor result. No wonder CEO's, CFO's and CIO's are so worried about starting major transformation projects like this. 

Telstra are not alone in achieving this level of poor performance, but it makes me weep when avoiding these problems is so easy.

Can anyone spell 'fiscal discipline' anymore?

So where was the effective governance in this program? Why was such a transformation project agreed to in the first place if governance wasn't the number one priority? Apparently performance tracking of the project was missing - or at least ineffective.

Transformation is always difficult. You succeed by breaking the problem down into small, measurable chunks that are delivered on an agreed schedule throughout the life of the project. Big bangs are bad. Period.

When you've done this you then put into place people with the backbone to govern the project and demand that the project deliver as promised - and if they don't then to understand why and judge whether-or-not investment continues.

Why is this so difficult?

This may be why I am a turn-around specialist rather than a permanent line manager. That and the fact that I spend a lot of time on governance and proving benefits as they are delivered. You'll have to take that as true for now - but please check with people I have worked with and for. I hope that they back me up on this.

The point about all of the above is that hopefully Telstra will learn something from this experience and bring the transformation project to a successful conclusion. I for one will cheer for a Project Manager that delivers this outcome. Even at a starting rate of $15,000 a day - that would still deliver bottom-line profits to Telstra.

I'll be brushing-up on my American accent for my next interview!

I'll leave you with Mr Lamming's own opinion as reported in the papers:

Quote:

"Telstra's transformation is the largest, most comprehensive and successful I have ever seen, and we have implemented it at speed," Mr Lamming said.

May he be as successful in a future engagement - but hopefully not in Australia!

Check back here in a year's time to see the end result of the 5 year program and the final report card.

Somebody At Microsoft Gets It!

Several weeks ago I came across a great video from Microsoft on the history of business intelligence. The author is Nic Smith who is from Microsoft BI Solutions Marketing.

Here it is:

 

If you have just watched it, then you will have seen that it is leading into what Microsoft sees as the future of BI. Based on the clarity of vision and communication in the video, is this another indication that the traditional mega-vendors (IBM, Oracle and SAS) should be worried?

The original is found online here.

Lessons From The Frontline

Recently I have just finished a long stint building an analytics capability in a large organisation. This has given me time to reflect on the experience and I'd like to share some of my conclusions. First off, the analytic capability is very successful and is driving better decisions in many areas of the organisation. That's the easy bit to say - and I would say that anyway :). What's more interesting are the lessons that I have learned - because they will help me do better next time. What are the lessons and how do I intend putting them into practice?

Lesson #1 - Get the Data Right You can't achieve advanced analytics if the data isn't right. Investing in high performing Business Analysis and Data Integration teams is the first step towards delivering a high performing organisation. I say teams but even one of each is enough to kick-start things. I also mention two teams because together they give you an unbeatable combination of analytic power within your team. Lesson #2 - Deliver Fast and Avoid 'Big Bangs' If you are delivering a complex thing like an enterprise self-service analytic capability then you have to deliver value much faster than is possible. How do you overcome this? The method I use is to identify existing information problems that are long standing issues. They are never hard to find and the application of a little logic and creativity invariably delivers a tactical solution that you can implement in a matter of months. We found a number of these problems (over 40 if I remember) and we delivered solutions to 20 or more in the months before we were able to deliver the enterprise data warehouse and self-service elements.

Just as importantly, the deliveries were spread consistently over the 2.5 years so that each quarter we could point to concrete benefits.

Lesson #3 - Deliver to the Direct Beneficiaries of Quality Data There will be one or two groups within the organisation that can benefit directly from the high quality data that the Data Integration team make possible. The key point here is that these groups do not need the presentation layer (reports, dashboards, etc.). This makes it much quicker to deliver. Find out who they are and make them the second customers you deliver to. In this case, Risk Management and Securitisation were departments that were identified and fit this bill. Delivering to these areas also gave us time to build the self-service analytic tools to other areas. Lesson #4 - You've Got To Build An Exclusive Club In the corporate world (the 'big end of town' we say in Australia) you are faced with people with a range of skills and talents. Some very, very good people with great skills and attitudes that I admire. The majority quite capable and with leadership can also deliver exceptional performance. The remaining 10% however are just not good enough. Nothing surprising in that I hear you say? Well I disagree. I spent 10 years starting, growing and selling start-up companies and the 10% 'not good enough' don't last long in that environment. This makes a fundamental difference in the speed at which a team can deliver. Just as importantly - it's less fun for everybody. I think that you should have fun, be proud of the work and learn from your team. All of these things are diminished when you don't have a high performing team. The organisation I worked for had an admirably cooperative and positive culture. To illustrate: you could approach just about anyone in the company and ask them a question and their first thought would be "How can I help this person?". Fantastic! So what is the lesson? I let the 'cooperative and positive culture' influence me to keep the 10% on in their jobs. On reflection, it would have been much better for all involved if I had done what I usually do: quickly and fairly get rid of them. Delaying or avoiding doing this hurt the performance of the analytic team I was building. Ultimately this also hurt our ability to get analytics adopted by the wider business. Lesson #5 - New Tools Are Required My final lesson has to do with the underlying technologies employed. I inherited a set of 'big tool' platforms (you know - the ones like IBM, Oracle and SAP). Early on I decided to keep them and go the upgrade path. I think this was wrong in a couple of areas - mainly around reporting and end user analytics. I should have looked at more agile alternatives. Ultimately I believe they would have been easier and cheaper to deploy. What are these alternatives? Here's is a list of a few that have caught my attention:

Open Source - tools are ready for the big time. Don't hesitate, start experimenting with them today.

QlikView, Zap and others - support more rapid deployment of analytics than the mega-platforms like IBM, Oracle and SAP.

Cloud9 Analytics - a role-based, on-demand business intelligence applications using a software-as-a-service (SaaS) model.

eThority - a web and desktop based, user-focused, accessible and scalable approach to business-driven analysis of enterprise data.

illuminate - a correlation database that brings agility to the back-end data integration tasks that are barriers to agile analytics.

Lyzasoft - depolarizing and “filling the middle” with desktop-based data gathering, data analysis, reporting and analytic publishing.

Netmap - a visual approach to discovery-based analysis.

PolyVista - extending OLAP with prepackaged, easy-to-use data mining and discovery automation capabilities.
Lesson #6 - Ignore Service Providers and Recruitment Consultants There are many good BI specialist Service Providers and Recruitment Consultants out there and they all want to develop a relationship with you. This is great and they can add value but when you are doing what we were doing there is not enough money or time to allow relationships to develop.

Our (my) strategy was to use a team of existing staff, a number of new specialist hires and specialist contractors. If I couldn't find the new hires and specialist contractors in my own network, then Service Providers and Recruitment Consultants were a vital component of the strategy. My existing network ensured that key appointment could be made with confidence. It certainly wasn't all 'plain sailing' but it worked well enough. We did go to market via recruitment agencies for a number of roles and the results were good. The success rate through agencies over the 2.5 years was about 50% (1 out of 2 hires were successful).

In hindsight, a lot of time was spent with lots of Service Providers and Recruitment Consultants. It would have been very useful to hear from my own network which ones they already knew were good. I still wonder why I didn't do more reference checking. The lesson is to be more disciplined in what you spend time on. Building an analytic capability is a full time job. So be firm and resist the chance to have 10 coffees a day!