October 2, 2010
Complexity – that which does not kill you…
The many headed hydra of complexity is usually seen as a thing to minimize or – ideally – avoid completely in business generally and in business intelligence systems in particular. This aversion to hard problems means that there is money to be made by the companies who can face up to the challenge….
Historically businesses have tried to deal with complexity by reducing it – offering a suite of standard products instead of a bespoke system, locking in customers, merging with competitors…. However, in our increasingly connected world this is becoming less and less possible. Barriers to entry are dropping for just about every kind of business all over the planet, customers are searching more widely for organizations that offer exactly the services they want at the best possible price, and the companies that can’t keep up with these demands are dropping by the wayside. To quote John Naughton, the old strategies “are unlikely to work in our emerging environment, where intelligence, agility, responsiveness and a willingness to experiment (and fail) provide better strategies for dealing with what the networked environment will throw at you.”
In terms of BI, to keep the business competitive some business users need to ask very complicated questions, to have the freedom to follow where the data leads, and those questions may change on a daily or weekly basis. And if the IT department can’t supply what they need, the business users will try to build their own systems in spreadsheets…. It’s quick and flexible and you can have as many copies of the spreadsheet as you like all doing slightly different things and it can link to other spreadsheets and no-one can remember how it works and the data is different in different copies and there’s no way to know which one of them is right any more… and down that road lies madness.
These kind of requirements really don’t suit traditional IT development with it’s careful upfront requirements spec; flat, static reports and long development time. Agile development – with it’s small, regular, fast deliveries – of a data warehouse and self service BI through a semantic layer like Business Objects suits them much better. This kind of project needs good integration between the IT team and the business users though, which may be a challenge in itself. Still, new, interesting, and potentially profitable ideas are likely to appear if the two sides can be persuaded to work together.
The complicated questions are likely to involve some pretty in depth analysis, and really, the business users don’t want to be chasing after the IT department every time they want to experiment with a new calculation. Cubes delivered through something like SQL Server Analysis Services or power pivot give control back to the users through the familiar interface of excel – without the danger of a spreadsheet daisy chain or many corrupted copies of the data. They could even be published on to share point for the rest of the team to share, and although they’re unlikely to be the sort of polished report you’d want to show the CEO, he’ll be more impressed by their impact on the bottom line.
So sometimes complexity can be a business benefit – for anyone who’s brave enough to grasp the nettle.
Some references that I found useful when writing this:
James Adman on Drowning in Spreadsheets: http://www.b-eye-network.co.uk/view/11482
John Naughton on how the Internet is changing the world around us: http://www.guardian.co.uk/technology/2010/jun/20/internet-everything-need-to-know
July 25, 2010
Competitive Intelligence – Business Intelligence looking out
Competitive Intelligence is the gentle art of working out who your competitors are, what makes them good, bad or ugly; and where they plan to go strategically. It’s a tough business based on exploiting contacts, and – sometimes – sailing dangerously close to industrial espionage to work out what they’re planning.
Old school Competitive Intelligence professionals tend to look down on Business Intelligence. They claim it has narrowed it’s focus in recent years, and become too centred around structured data and IT. Around structured data they may have a point, but there is a good reason for both of these developments – they produce results. Solid results which give solid business benefit, rather than flaky gut feel instincts based on talking to a tiny handful of people at trade shows. And maybe it’s time Competitive Intelligence caught up.
Algorithms for processing unstructured data are improving all the time – consider Google, or the program proposed by the British Government in 2008 to monitor every call, text, email and even every website visited for potential subversive activity. Surely business can use similar (though, one hopes, less ethically dubious!) techniques to discover the capabilities, vulnerabilities and intensions of competitors?
For the purposes of this article, I am going to consider Twitter. I’m sure everyone can remember an incident when a competitor has let slip some confidential information in a thoughtless tweet. Every tweet is easily searchable unstructured information in the public domain. It can tell you a huge amount about both customers and competitors.
Analysing unstructured data is fundamentally a hard problem. How you go about it depends heavily on what kind of information you are looking for. So, let me set the scene. You’re the CEO of a major manufacturer of mobile phones. Your company produces some cracking products, but you’ve had some bad press lately over a minor fault in your flagship handset. You might like to know how many complaints your competitors are having about similar issues…..
A lot of people use Twitter to complain, so it seems like a good place to start, (although it would also be worth using the same techniques on all of the major reviewing sites for your sector). Identifying the Twitter accounts of competitors should be a simple task, if you know who your competitors are (most companies have a link to their Twitter feed on their website) and it shouldn’t be too challenging to write a procedure counting the number of obscene tweets sent in their direction. Divide by market share, and you have a crude measure of customer satisfaction at any given moment. Analyse the tweets to look for… Say… Signal, crash, poor reception, broken screen, or other keywords, and you can start to work out what their customers perceive to be their biggest weaknesses. Look for the most common nouns and adjectives used, try some cluster analysis, and see what that suggests.
Once you’ve worked out what keywords people tend to associate with your sector – mobile, or reception for instance, you can use those to drag up more information. For example, if people are sending lots of tweets containing “your” keywords to twitter accounts you are not monitoring, this could alert you to new competitors.
Twitter records trends over time too, so you can track whether they are getting more or less complaints; or, say, if your competitor has run two big marketing campaigns in the last year you can get a feel for how much buzz they generated.
If you gather the relevant data into a data warehouse, it becomes even easier to process the information, slice it and dice it any way you wish. Suppose you’ve just found out that Samsung are launching a new handset called Slik – how many tweets are referring to it? You could try running cluster analysis using SQL Server Analysis Services to see what other companies and products people who mention the new handset are talking about. Use a semantic layer like Business Objects to handle ad-hoc queries easily – on a well designed data warehouse it should be easy to answer almost any question about the newest, shiniest products.
There are hundreds of other ways to gather non-obvious information, using http://web.archive.org/web/ to monitor the number of news articles mentioning your competitors over time, together with positive or negative adjectives, monitoring review sites, facebook fans and posts…. The list is endless. With some careful though about what is important to your business, and a good system to handle the plethora of information, the competitive advantage should be colossal.
July 2, 2010
iBI – Business Intelligence for the Apple generation
Ok, I have a confession to make – I do like apple. Their interfaces are achingly slick, and they’ve got a real knack for making intuitive and easy to use tools that do some amazingly complex things under the hood. They give you exactly what you need to get the job done, and no more! I think there’s a lot that BI professionals like myself can learn from them…
Mobile working is only going to increase over time, and business people want to make use of every spare moment – I’m writing this post on my iPad sitting on the bus, for instance. Every day business critical decisions are being made in taxis, on the tube or even on the plane. Blackberrys and iphones keep us constantly in communication with our colleagues, but to access BI systems on the move is much harder. Heavyweight encryption and security systems mean that some business laptops take over twenty minutes to boot up – that’s just not practical when you’re stood up in a crowded train carriage and you need to make a decision now! The iPad and iPhone fill this gap perfectly, and they have some fantastic BI applications. It’s early days yet, but can definitely see them gaining wide acceptance among the clients that I work for.
Most of the apps are new front ends for the old players. Sketch Reports and Sketch Reports Pro sit on top of Microsoft SQL Server Reporting Services (SSRS) and provide a friendly viewer. To use parameters, share point integration and SSL connections you’ll need the professional version – although it only costs £5.99 per user. The reports are professional and smart as you’d expect – the look and feel is very like having an actual paper copy.
SAP have got in on the action too with SBO explorer, a portal that gives you access to reports created in their Business Objects tool. Unfortunately you can’t create new reports on there yet – a shame because drag and drop reporting should work well on the iPad. The reports are more interactive than Sketch Reports, but they look somehow less polished.
BIRT is part of the eclipse family, but although it’s open source you need the proprietary iServer product from Actuate underneath to make it work. It only seems to produce static reports – much less interactive than some of the alternatives, but it doesn’t have the familiarity of a paper report.
Qlickview are a major european BI company, and they’ve come up with an interface designed especially for the iPad. Unfortunately it’s a lot less sophisticated and elegant than some of the others out there and I can’t see it working well with complex data relationships. It’s also quite slow.
Pentaho also have an iPad compatible view that is accessed through safari. However, the security for this looks quite lax, and pentaho is not the most stable tool in general. They seem to make their money by having tools that are free, but so hard to use that you need to hire one of their consultants.
PushBI are another new player – they look good for KPIs, but don’t seem to offer much else. The graphs seem pretty simplistic, and it’s not amazingly quick. Good for a quick on the run check of a data warehouse, but probably not up to anything more complex. Reports are available offline though.
I’ve saved the best till last – by far the best app I’ve seen for the iPad is Roambi. It’s visually stunning – the app is free, and I strongly recommend downloading it just to have a play with their samples. It’s a superb use of the touchscreen interface, and if you’re designing explicitly for the iPad I would strongly suggest using it – it’s definitely got the wow factor. It can pull data from all sorts of places – Excel, Google Docs, Crystal Reports, Salesforce CRM, SAP BusinessObjects, IBM Cognos and Microsoft Reporting Services… Although anything more than excel will cost you. You can add custom filters to the report, and it’s available offline once you’ve downloaded it. It seems respectably quick, and the security is as solid as the layer it sits on.
Overall, Roambi would definitely be my first choice, but if you’ve got conservative users who’re happy with their printed reports then Sketch might be a better fit. In the future, if and when SAP release an updated version that lets users create custom reports on the go, I think it could have real potential.
Bethany's Blog Of BI Delights 