Oz Analytics

Forget The Politics

I read with interest this morning about the kerfuffle New York Times blogger Nate Silver is causing. A Financial Times article (subscription required) explained that Silver has been using statistical analysis to consistently predict an Obama win in the coming US elections. His FiveThirtyEight blog is now being roundly criticised by the right-wing - and the criticism is getting pretty personal.

The unfortunate reality is that this "effeminate man with a soft-sounding voice” (Dean Chambers) has a pretty amazing track record of success. He started plying his trade forecasting the performance of Major League Baseball players and since 2008 has turned his attention to politics. In 2008 he correctly predicted the presidential winner of 49 out of 50 states. As a side note, the same blog (but different blogger) also correctly predicted the narrow win for the ruling Australian Labor party in 2010.

As far as I can see, Silver is only guilty of using open data to make systematic and scientifically based predictions using probability theory. You can check-out his methods for yourself.

I recommend his blog to anyone interested in stats and polling. TL;DR? If that's too old-school for you gen-Y twits (?), then he tweets as well.

His book 'The Signal and the Noise: Why So Many Predictions Fail-but Some Don't' has just been published and I'm ordering it today.

It's good to see that mathematically-based poll analysis continues to be politicised in the States. Way to go America - who needs facts?

Day 1 Week 6 of the CORTEX MBAnalytics

Welcome to the start of week 6 - part of the 60 day information management refresher course from the CORTEX.

The CORTEX

Master of Business Analytics

Week 6, Day 1

Theme: Analytics In The Real World

Week 6, Monday

This week's theme is application - not in the sense of a computer program - rather in the sense of 'use it or lose it'. This week we look at how analytics is being applied in the real world.

WATCH    Today's task:  HSBC    Streamed

11 minutes

Untitled set of 3 videos on YouTube,posted by Responsetek. This is an interview with Justin Kenny, HSBC eChannel Programme Director

Key takeout: I've started the week with these videos because they give you an insight into the world of large corporate use of analytics. It is not as 'sexy' as the examples the large BI vendors are always presenting but it is all the more realistic for that. This is the type of work that the vast majority of organisations are focussed on. 

As always, comments, questions and ideas are all welcome. 

The MBAnalytics is just one part of the Business Intelligence CORTEX - a community of Business Intelligence, Data Warehousing, Analytics and Project Management professionals in Australia, New Zealand and Asia. Membership is free and the forums are very active with over 20,000 posts offering a uniquely Australasian viewpoint.

Day 4 Week 5 of the CORTEX MBAnalytics

Continuing week 5 - part of the 60 day information management refresher course from the CORTEX.

The CORTEX

Master of Business Analytics

Week 5, Day 4

Theme: Analytic Errors

Week 5, Thursday

This week's theme is mistakes. Today we look into how people make decisions and you have a choice. 

WATCH    Choice 1:  Think Twice: Michael Mauboussin at CEP 2011     Streamed

30 minutes

"Think Twice:" Michael Mauboussin at CEP's 2011 Conference, YouTube video of Michael Mauboussin talking at the Center for Effective Philanthropy's 2011 conference.

READ       Choice 2:   Think Twice A cautionary tale for investors     70 KB

5 minutes

'Michael Mauboussin's 'Think Twice': A cautionary tale for investors' article by Vishesh Kumar, on the AOL DailyFinance site, August 2009. Available on AOL here.

READ       Choice 3:  Think Twice: Harnessing the Power of Counterintuition   To buy

??? hours. How fast can you read 204 pages?

'Think Twice: Harnessing the Power of Counterintuition' by Michael Mauboussin Chief Investment Strategist at Legg Mason Capital Management and adjunct professor of finance at Columbia Business School, hardcover book published by Harvard Business School Press, October, 2009, 204 pages. Available to purchase on Amazon.

Key takeout: Fascinating insight into how we make decisions - especially insightful on the subject of mistakes and experts. Choice 1, the video, is a great way to be introduced to Mauboussin's ideas. Choice 2 is the one to take if you really can't spend half an hour watching choice 1. Choice 3 is something to keep in mind. Now go out and read the book!

As always, comments, questions and ideas are all welcome. 

The MBAnalytics is just one part of the Business Intelligence CORTEX - a community of Business Intelligence, Data Warehousing, Analytics and Project Management professionals in Australia, New Zealand and Asia. Membership is free and the forums are very active with over 20,000 posts offering a uniquely Australasian viewpoint.

Day 1 Week 1 of the CORTEX MBAnalytics

In my post last week I spoke of my idea to take-up between 5 and 30 minutes of your day to introduce you to the latest trends, ideas and applications of information management. So welcome to our first day.

The CORTEX

Master of Business Analytics

Week 1, Day 1

Week 1, Monday

 READ     Big idea: What People Want and How to Predict It    0.5Mb

15 minutes

MIT Sloan Management Review, 'What People Want (and How to Predict It)',

by Thomas H. Davenport and Jeanne G. Harris,Vol.50 Number 2, Winter 2009

Online here.

WATCH   For extra credit: Transitioning from reporting to predicting   100Mb

10 minutes

YouTube video courtesy of IBM Netezza. Online here.

Comments,questions and ideas are all welcome. 

The MBAnalytics is just one part of the Business Intelligence CORTEX - a community of Business Intelligence, Data Warehousing, Analytics and Project Management professionals in Australia, New Zealand and Asia. Membership is free and the forums are very active with over 20,000 posts offering a uniquely Australasian viewpoint.

The Future Of Business Intelligence

Keynote address to the Next Generation Data Warehousing Conference, IDG Korea , Seoul, South Korea, February 16th 2011

I presented my thoughts to a couple of hundred Senior Managers from a wide range of Korean organisations. Crystal ball gazing is something I usually leave to vendors and consultants so I focussed on the following aspects that I at least have some practical experience of:

• Key trends to the end of the decade
• Best practices
• Organising the future intelligence capability.

The following key trends struck me as important factors influencing the future of our work in analytics:

• An Unprecedented Explosion In Data. “Between now and 2020, the amount of digital information created and replicated in the world will grow to an almost inconceivable 35 trillion gigabytes as all major forms of media”
  IDC iView, "The Digital Universe Decade – Are You Ready?" May 2010
• The Dominance of Mobile Multimedia Content. “Global mobile data traffic will increase 26-fold between 2010 and 2015 ... Mobile network connection speeds will increase 10-fold by 2015 ... Two-thirds of the world’s mobile data traffic will be video by 2015.”
  Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2010–2015, February 2011
• The End Of English. Chinese-language content will dominate the internet by 2014
  Gartner 2009.

I'm not completely convinced about Chinese domination - but I agree that the Internet will become strongly multi-lingual as the rise of China and Brazil continues.  In any event, we all now live in a world of BIG data. Data that is growing annually by orders of magnitude and is moving to mobile/embedded devices that generate real-time multimedia data. On the other hand, IT skilled people - those with the skills most likely to handle data growth - are only predicted to grow 1.4 fold over the same period. So there is an emerging massive skills shortage coming and we have four choices:

• Do nothing. You wouldn't be alone if this is your strategy.
• Focus your current analysts on embedding their skills in a broad range of your staff.
• Try to buy the skills by poaching staff from other organisations. This could get to be very expensive as you will likely to be in a bidding war with other organisations facing a similar skill shortage.
• Use software that enables your analysts to handle vastly more data than they can today.

Just to complicate things, the era of the English/Western Internet is being supplanted by Chinese - a language that is impervious to processing by most of the current and emerging intelligence capabilities out of the US.

Personally, I see as our best hope a strategy that focusses on embedding analytic skills into the general business community. This greatly enhances analytics usefulness in the short term, whether-or-not you believe in the above trends. It will also buy us the time to see smarter software emerge that does the 'heavy lifting' of BIG (Chinese?) data for analysts. The analogy I used in my address was transportation and the London underground network. The actual network looks like the following:

It took Harry Beck in 1931 to analyse the London underground network and realise that the actual physical location of the station was less important than the information about how you can get to your destination.

This is exactly what analytics needs to achieve. I like to call this actionable insight.

If you have any doubts about the explosion in data we can expect, take a look a recent years. The Washington Post has created a created a graphic that shows how data grew from 1996 and 2007:

After looking at the impact of these trends, I went on to discuss templates for both the analytic skills we need and how to best to organise your teams. I have posted about these topics previously here.

After the presentation there were questions from the floor and it was interesting to see that the questions could have come from any country. The problems and challenges discussed were exactly the same as I hear in other countries. It's nice to know the we are all in the same boat.

 

Crowdsourced Insight

I have just been reading David Cameron's blog where he talks about an interesting hack from India. You can read the story here but in short it is about Arun Ganesh, a student at the National Institute of Design in Bangalore, who was frustrated that the poor quality of the bus timetable for Chennai. With over 5,000 bus routes this was a real world problem for Chennai's 4 million residents. How did Arun solve this? He created his own map, crowd-sourced data from his fellow bus passengers and then published the new improved map along with an accompanying app BusRoutes.in for interactive route planning. The maps use OpenStreetMap - a free editable map of the whole world. In 3 days he had crowd-sourced enough data to kick-start everything. An impressive solution to an everyday data problem.

I know that corporate crowdsourcing for R&D is nothing new and prediction markets are gaining some traction. Take a look also at InnoCentive for some very serious idea outsourcing of industrial problems and Brand Tag for insight into what people really think of your brand.

I wonder if it is possible to apply similar ingenuity for information management problems within the corporation? What problems am I talking about? For me today in my role they include:

• Creating a single business glossary across a conglomerate of businesses
• Building a knowledge portal to collect information useful to management report writers and users
• Identifying solutions to complex analytic problems such as:
  • predicting propensities to substitute physical products with digital ones
  • measures of copyright protected revenues.

My initial thoughts are to invest in the following capabilities:

• Business Glossary - set-up an intranet portal (probably using SharePoint because that's what we use internally) to publish glossary term definitions and link these to discussion forums where anyone who is interested can discuss, provide feedback and mediate agreement on standard terms and definitions
• Knowledge Portal - another portal linked directly into our reporting platforms (SAP, Oracle, TM1, etc.) so that our:
  • Decision makers have access to use and can contribute to wiki's, FAQ's, known issues and documentation (tutorials, reporting guidelines, etc.) about our information assets and related business processes.
  • Project Staff and Analysts have access to the scope, structure and lineage of data from the report semantic layer back through the data marts and warehouses and into the source systems where data is created.
• Analytic Solutions - you guessed it: YAP (yet-another-portal) that lets people publish data sets that have either been purchased from 3rd parties (such as market research) or contain discovered insight that may be useful to others.

My hope is that these investments will deliver not just better support to our decision makers but also better engagement with the information they use. Give them a sense that they not only own their management reporting but also feel empowered to improve the information asset without relying on IT experts.

This may not be as innovative as people like Arun have achieved, but in the my corporate world these are sufficient wins. Maybe we will even create more super users in the organisation? Sounds good to me.

Looking again out into the rest of the world and taking a lateral step from directly involving your users, kaggle is a site that aims to facilitate better predictions by providing a platform for data mining, forecasting and bioinformatics competitions. As a side note, it is interesting to see that 'kagglers' favourite software is R, with SAS now a fairly distant 3rd in popularity. SPSS is even further away with only a quarter of the users that R has.

If you want to learn more about open innovation I suggest you start at Open Innovators website.

Defining Analytics: Analytics

Following-up to my blog 'Just Tell Me What I'm Doing', this is the third in a series of posts that define the key concepts and terms that make up my analytic world. Everything I do is coloured by my experience actually doing analytics in commercial organisations.

So while I believe these posts will present practical definitions that will be actionable in the business world, I know that there are other worlds in academia and science where they are less relevant. At the very least, people in these areas will gain a better understanding of how business regards analytics. Data Warehouse puritans should perhaps stop reading, take a pill and lie down instead.

---

Bennett's Analytica  
   A Practitioner's Guide To Analytics

---

Analytics

---

The simplest definition of Analytics is "the science of analysis". A simple and practical definition, is that it is how a business arrives at an optimal or realistic decision based on existing data. Business managers may choose to make decisions based on past experiences or rules of thumb, or there might be other qualitative aspects to decision making. This is not analytics. When the decision is based on data, then this is analytics.

Doug Lautzenheiser, a fellow blogger on the SmartData Collective recently gave us the following definition:

“Business Intelligence isn't just about seeing how many cans of beans you sold last month.  It's about figuring out how to sell even more in the future.”

I like this, but even more when I swap out 'business intelligence' for 'analytics'. So, just like data warehousing and business intelligence, these are all terms used interchangeably in practice.

Another favourite definition is:

"analytics is reporting with an intent to inform a decision that you want to make." 

or even just:

"analytics is reporting with intent."

But I use this only when I am talking to someone who really doesn't understand much about the subject.

Remember - my world is commerce. I haven't given much thought to other areas like academia and medicine, etc. where analytics is also used extensively. With that in mind, I define the analytic process as follows.

Analytica Illustration

The Analytic Process

To me, it is the process you go through to analyse a problem and make a decision. Here are what I think are the common steps involved:

1. Define the Problem. Example: "We need to improve by 20% the success of our Product X marketing campaigns." If you don't know what the problem is, then 99 times out of 100 you should stop the process and go and do something else that is useful!
2. Generate Hypotheses. Use your experience and knowledge to guess ways in which you can use data to solve your problem. Often you will have several candidate hypotheses. These are 'educated guesses'. Example: "We can better predict which existing customers will buy more products if we can understand better what existing Product X customers have in common."
3. Determine what information you need & gather it. Traditionally this information came from a data warehouse or some other sort of database or electronic file. In recent years unstructured textual data, multimedia and other sorts of data stores are being used.
4. Test sources. Run some basic reports to see if the data can support one (or more) of your hypotheses. If you are lucky, you have access to a Business Intelligence tool like Cognos, Business Objects, QlikView, etc. Often you don't and in these cases you can use SQL to extract data from their sources and import it into Microsoft Excel instead. Then test how good the source data is - or more precisely - determine how suitable the data is to our analytic purpose.
5. Test Hypotheses. Run further reports that are built to show how successful (or not) your hypothesis actually is. It is important to question your results and there are many way to do this. Here are a couple of the most common ways to do this:
6. • Find Root Causes. Look extra hard at the results that are the exceptions that do not support your hypothesis. Apply root cause analysis - this is analysis done in order to understand why the exceptions exist. Exceptions are often the data that do not fit your hypothesis. Example: "70% of the existing customers analysed had purchased a car in the 2 weeks before they purchased a tow bar [US: tow hitch]. But for 30% the towbar was their first purchase - but they only work in cars we sell". Concentrate on understanding the 30%. Example answer: "The tow bar purchaser for the exception 30% of customers, was 90% of the time either the spouse of the car purchaser or their employer".
   • Experiment. Identify a set of customers with something in common but that haven't purchased Product X. Example: "30 - 40 year old women with 2 or more children under the age of 18". Also, randomly select a second group of customers (i.e. those with nothing in common) to be your control group. Then run the same test campaign using both groups and compare the results for both sets of customers.
7. Review the results with your peers to check that the result really is valuable. Is it fit for purpose? Is the effort worth the benefit gained? Can you identify enough customers meeting the criteria to meet the uplift target?
8. Package the process. If you regularly need to analyse the same or similar problems then you need to make the process repeatable by putting it into 'production' and packaging it with additional information about how it works, any known issues, its purpose, etc.
9. Get user feedback and evaluate the results by either tracking the decisions made based on the analysis or more informally. If you do the former, then congratulations - you are operating Decision Management. This is a holy grail for my work. More often you have to settle for informal feedback from the decision makers (or their colleagues). Example: "Analyst to Campaign Manager: So whadya think?"
10. Learn. Use this knowledge to improve the hypotheses, sources, data and process.
11. Repeat the process again until you get the result you want - or give it up as an unattainable goal.

Well that's how I see the process - at a very high level. I went looking online to see what other people thought and there was very little to find in the commercial context. Even wikipedia let me down although it does have interesting things to say about Analytic Hierarchy Processes and the Intelligence Analysis Process used by NSA and the like. There is an old discussion thread on LinkedIn that's relevant.

Analytic Techniques

There are a large number of statistical, mathematic and management techniques that can be used. These will be discussed in a future Analytica entry.

Common Uses of Analytics

Predictive Analytics

Predictive analytics encompasses a variety of techniques from statistics, data mining and game theory that analyse current and historical facts to make predictions about future events.

In business, predictive models exploit patterns found in historical and transactional data to identify risks and opportunities. Models capture relationships among many factors to allow assessment of risk or potential associated with a particular set of conditions, guiding decision making for candidate transactions.

Web Analytics

Web analytics is the measurement, collection, analysis and reporting of internet data for purposes of understanding and optimising web usage – not just measuring web traffic.

Related Terms and Concepts

Refer also to Data Mining

---

Comments? Via form below or send feedback to [email protected]         version 0.1 201002

---

Analytics at Work

Hot off the presses jumps Analytics at Work: Smarter Decisions, Better Results, the new book by Thomas Davenport, Jeanne G. Harris (both of Competing On Analytics fame) and Robert Morison. This book is very much the sequel to Competing On Analytics and this is not a bad thing. CIO Insight magazine named Competing on Analytics one of the all-time “Top 15 Most Groundbreaking Management Books”.

What I like about the book is that it is going after the big picture where better decisions drive outstanding performance, improve competitiveness and help organisations differentiate themselves from the competition.

This time the authors have completed new research on companies in the US and Europe. They include Humana, Tesco, Air France/KLM, Best Buy, Progressive Insurance, and Hotels.com. Using these organisations as illustrations the book is intended to be an implementation-focused guide that describes a five-step model for deploying and succeeding with analytical initiatives.

In the first half of the book five factors are presented as being necessary if an organisation is to succeed at 'doing analytics'. The authors group use the acronym DELTA (the Greek letter that signifies “change” in an equation):

• D for accessible, high-quality data
• E for an enterprise orientation
• L for analytical leadership
• T for strategic targets
• A for analytical talent

This all makes sense to me and then in the second half of the book the five-step model is explained. I will not spoil the authors work by discussing that here.

To me the book is presented as a primer for doing analytics. How well does it achieve this objective? The answer is both 'very well' and 'not really'. Let me explain:

• If you are a C* level executive trying to understand why you need analytics, and you want a high-level understanding of how to achieve this, then the book delivers - and delivers well.
• If you are a Senior Manager directly responsible for making analytics happen, the book is less successful. Not through any fault of the authors. I am one of these managers and I know that analytics is hard. Even a 1,000 page tome and a dozen CD's would not be enough for that.

So I hope that the book will inspire C* level executives that are interested enough to read a 200-or-so page book on the subject. If they only read one book on the subject then this is the one I would give them. For those that can't be bothered, then I can sit waiting for the next crisis to hit and the inevitable change of management to occur!

To the author's credit, I notice scattered throughout the book discussions (on pages 14-15 for example) and insight on how the Financial Industry got risk analytics so wrong - contributing to the financial crisis of 2007. Finance's use of analytics was featured as one of the successes in Competing On Analytics.

Although a Harvard Business Press title, the book is 'tied-in' to Accenture (Harris works for them and Davenport is an Accenture Fellow) and the company has a microsite within their Institute for High Performance consulting practice. Here's a nice brochure from Accenture on Analytics At Work.

Is 2010 The Year Asia Cuts the Apron Strings?

It has been many years since I last worked in China and a couple of recent announcements led me to wonder what has been going on there recently. When I started to review what I knew about analytics in China I started thinking about Asia in general. Would 2010 be the year we would see an upswell of analytics with a distinctly Asian (as opposed to US) flavour?

What makes Asia Different?

Analytically, it's both the size of the potential market as well as its fundamental differences from the West that intrigue me. Take a look at the mobile market for example:

 

So not only is the absolute size greater but 75 per cent of mobile users own web-enabled phones - compared with only 41 per cent in the US. This data is care of Netpop Research. China is not alone in the region, IDC estimates that there are over 600 mn fixed line subscribers in Asia Pacific excluding Japan (APEJ) and over 1.6 bn mobile subscribers.

Telco's in India expect 2010 to see Enterprise Mobility as a hot trend. According to local trade publication Voice&Data:

"Enterprises will be relooking at their entire IT architecture and strategize ways to ensure that the entire workforce – be it blue collar manufacturing employees, gray collar service employees or white collar management employees – is able to leverage and add-on to the intelligence available at the back office."

In the wired world, 5 of the top 10 most connected countries or territories are in Asia. South Korea led, where 95 percent of homes have broadband. Singapore and Taiwan are among the top five countries in terms of access to high-speed Internet, according to a survey released in 2009 by Boston-based Strategy Analytics.

The United States, where just 60 percent of households had broadband as of last year, ranked 20th in the survey of 58 countries. The firm predicted the broadband subscriber base in the Asia-Pacific region will grow on average by a further 15 percent a year between 2009 and 2013.

Strategy Analytics said South Korea's highly urbanized population and its government-backed broadband policy accounted for its high rate of broadband penetration.

Singapore ranked second on the list with household broadband penetration of 88 percent, followed by the Netherlands (85 percent), Denmark (82 percent), Taiwan (81 percent), Hong Kong (81 percent), Israel (77 percent), Switzerland (76 percent), Canada (76 percent) and Norway (75 percent).

Among other Asia-Pacific nations, Australia ranked 11th with 72 percent, Japan ranked 16th with 64 percent, New Zealand ranked 25th with 57 percent, China ranked 43rd with 21 percent and Malaysia ranked 44th, also with 21 percent.

Thailand ranked 51st with seven percent, Vietnam ranked 52nd, also with seven percent, the Philippines ranked 53rd with five percent, India ranked 57th with two percent and Indonesia ranked 58th with one percent.

Why Is Asia Important?

Technology spend across Asia excluding Japan (APeJ) is expected to grow strongly in 2010. Singapore-based Springboard Research's recent Asia-Pacific Predictions 2010 report is predicting 8.8 per cent growth. They put analytics as the #1 trend in 2010:

"Among the hot technologies for this year are analytics solutions that will make use of data and predictive modelling technology to help support business decision-making."

This prediction is similar, but more optimistic than that from the better-known IDC. They recently predicted that ICT spending in APEJ will reach $184 bn in 2010 with a 7.7% growth over this year. Analytics was their #2 trend for 2010:

"2010, IDC expects business analytics to emerge as a key technology area to help CIOs manage cost, comply with regulations, and most importantly, grow the business as information is treated as a strategic asset within the organization. More advanced and sophisticated forms of analytics solutions, are actively being implemented by organizations in the region."

IDC goes on to add a second analytic prediction with trend #9:

#9: Intelligent X: Building a smarter and more measurable world

IDC believes that 2010 will be a year of multiple 'intelligent' initiatives in Asia, as major governments and cities try to beat each other to reach the goal of securing foreign investment and foreign technology participation in this region. In these 'intelligent' initiatives, a new business model is emerging as a 'Holy Trinity' consisting of government, ICT vendors, and citizens; all three working in sync is required to make inroads in emerging countries.

From an ANZ perspective, the signing of the Free Trade Agreement between ASEAN, Australia and New Zealand (AANZFTA) in late 2008 should also expand activity as it includes provisions covering trade in services, together with investment, intellectual property, competition policy and economic cooperation.

More personal reasons for why Asia matters include:

• Large numbers of analytics graduates I meet from Bangalore, Chennai and Shanghai
• ETL and SQL outsourcing deals between ANZ and Asia
• Rise of leading BI Vendor centres of excellence in Asia (more on this in my next blog)
• Expansion of local BI players into South East Asia (and beyond).

In my next blog I will try to summarise what has been happening in analytics in Asia recently. I believe that we will see evidence of Asia coming out of the Global Financial Crisis in a stronger position than when they went in. Analytics in Asia is looking a lot more dynamic these days and many local organisations are not going to pay for the BI suites from the SOMIS (SAP, Oracle, Microsoft, IBM, SAS) vendors when viable local options are around at a fraction of the price.

It's still too early for Silicon Valley to start worrying - but it may be time to start taking notice of what's happening to the East of La La Land.

Mining The Gartner Symposium

Gartner's biggest annual get-together (Gartner Symposium/ITxpo 2009) is on in Orlando and although I generally don't think too much about it, it does (claim to) bring together over 1,600 CIO's and I was wondering what knowledge I could mine by looking at what people are tweeting, blogging and in other ways posting online.

Here's what I have found. I include some interesting snippets and also a sprinkling of 'stating the obvious'. Gartner remains a fascinating mix of deep insight and banality. I can't resist it, but you can judge the value yourself:

@lgreski: #gartnersym keynote: 75% of leaders surveyed say bad data constrains business. Too much info managed in spreadsheets.

@andreagoulet: Avg team size for big brand twitter team? 4. @jetblue :12 @wholefoods :11 @ford :6 @starbucks :2 Most common 1-2 people "on duty" #ims09

“Just as your business counterparts use fact-based business intelligence data to make decisions regarding investments and resource prioritization to achieve their goals, IT needs to gather and analyze relevant data to run the business of IT,” said the Gartner analysts. By Mike Hill, Gartner

The most blogged news of the conference comes from Gartner Analysts Carl Claunch and David Cearley’s Top 10 Strategic Technologies for 2010:

1. Cloud Computing
2. Advanced Analytics
3. Client Computing
4. IT for Green
5. Reshaping the Data Center
6. Social Computing
7. Security – Activity Monitoring
8. Flash Memory
9. Virtualization for Availability
10. Mobile Applications

Most made me yawn, but these stand-out (from a Gartner post):

#1: Cloud Computing. Cloud computing is a style of computing that characterizes a model in which providers deliver a variety of IT-enabled capabilities to consumers. Cloud-based services can be exploited in a variety of ways to develop an application or a solution. Using cloud resources does not eliminate the costs of IT solutions, but does re-arrange some and reduce others. In addition, consuming cloud services enterprises will increasingly act as cloud providers and deliver application, information or business process services to customers and business partners.

#2 Advanced Analytics. Optimization and simulation is using analytical tools and models to maximize business process and decision effectiveness by examining alternative outcomes and scenarios, before, during and after process implementation and execution. This can be viewed as a third step in supporting operational business decisions. Fixed rules and prepared policies gave way to more informed decisions powered by the right information delivered at the right time, whether through customer relationship management (CRM) or enterprise resource planning (ERP) or other applications. The new step is to provide simulation, prediction, optimization and other analytics, not simply information, to empower even more decision flexibility at the time and place of every business process action. The new step looks into the future, predicting what can or will happen.

#6 Social Computing. Workers do not want two distinct environments to support their work – one for their own work products (whether personal or group) and another for accessing “external” information. Enterprises must focus both on use of social software and social media in the enterprise and participation and integration with externally facing enterprise-sponsored and public communities. Do not ignore the role of the social profile to bring communities together.

#7 Security – Activity Monitoring. Traditionally, security has focused on putting up a perimeter fence to keep others out, but it has evolved to monitoring activities and identifying patterns that would have been missed before. Information security professionals face the challenge of detecting malicious activity in a constant stream of discrete events that are usually associated with an authorized user and are generated from multiple network, system and application sources. At the same time, security departments are facing increasing demands for ever-greater log analysis and reporting to support audit requirements. A variety of complimentary (and sometimes overlapping) monitoring and analysis tools help enterprises better detect and investigate suspicious activity – often with real-time alerting or transaction intervention. By understanding the strengths and weaknesses of these tools, enterprises can better understand how to use them to defend the enterprise and meet audit requirements.

From a presentation by Gartner Senior Analyst Eric Knipp on “Citizen Developers: When End Users Surpass the IT Department in Delivering Business Applications”:

"as much as 25 percent of business applications will be built by “citizen developers” – users outside of the scope of formal enterprise IT – by 2014."

“Future citizen-developed applications will leverage the IT investments beneath the surface,” he said. “Mashups will enable IT to focus on deeper architectural concerns, while end users will focus on wiring together services into business processes and workflows. Furthermore, citizen development introduces the opportunity for end users to address projects that IT has never had time to get to — the vast expanse of departmental and situational projects found beneath the surface.”

A recent report published by Gartner entitled Introducing Pattern-Based Strategy states: “We live in a world of patterns — competitive advantage and survival are about recognizing and acting on patterns before others. Previous strategic planning models were strained to seek changes, predict outcomes, adequately guide decisions and assist leaders in defining governance models that embrace change. Through a focus on pattern-based strategy organizations will move from being ‘reactive’ to being ‘proactive.’” The research note goes on to say, “The environment emerging from the recession demands an increased focus on detecting leading indicators of change, and on identifying and quantifying risk emerging from new patterns, rather than obsessing over lagging indicators of performance. In this way, we move from a world of ’sense and respond’ to one focused on ’seek and act.’”

"Introducing Pattern-Based Strategy," Gartner, 7 August 2009.

A cut-down version of the Gartner roadshow comes to Sydney 17-19 November 2009.