Strategic Analytics
I recently watched the documentary “Fog of War” – about Robert McNamara. McNamara is consistently fascinating – and obviously brilliant. His anecdotes about various systems analysis problems that he had tackled – from high abort rates on allied bomber sorties in Germany to machine-based personnel evaluations to driver safety tests – were wonderful. McNamara comes off as the quintessential analyst – and what struck me was how often McNamara was able to cull important decisions (right or not) from the data he was working with.
Anyone working with analytics knows that it isn’t easy to move from analysis to decision – and the biggest problem with most analytic efforts is how bloodless they are. This is especially true in our field – web analytics. I hardly ever see web analytics that matters – it always seems to be the same old reports recycled in increasingly pretty formats.
McNamara’s story got me thinking about what makes analysis useful – in particular, how analysis can underscore strategic decisions and not just tactical ones.
Certainly, McNamara’s success wasn’t based on superior technology. One of the sillier aspects of the documentary was the visuals the movie producers used to underscore the conversation. In several cases, we see shots of vast arrays of card-sorters (a technology he eagerly embraced) – and are led to marvel at the vast power unleashed to assist McNamara’s analysis. But this, of course, seems like an absurd joke when you realize that my six-year old daughters’ computer has more power than any computer Robert McNamara ever used. Yet with this truly impressive computing power at all of our fingertips, we none of us very often achieve anything like McNamara’s insight.
And let’s face it; the really important decisions for any business tend to be strategic: the answers to questions about the best channel to sell a product, what type of customers to appeal to, where to price relative to the competition and which messaging strategy is appropriate have an enormous impact on overall success. These aren’t questions easily addressed by analytics. If you find yourself using web or search engine analytics at all, it’s probably to answer questions like: which links on my home page aren’t getting used or what search term has the highest click-thru rate.
These tactical questions aren’t unimportant. Good answers coupled with intelligent tactical changes can make a difference in overall performance. But lacking a strong strategic framework, these tactical changes tend to dry up quickly. Once you’ve plucked the low-hanging fruit of obvious flaws in your web channel programs, you may find yourself using analytics less and less simply because there are no obvious tactical wins.
That’s why so many powerful, high-end enterprise-wide web analytics solutions tend to wind up as rather disappointing reporting systems – churning out the same old reports about number of visitors and page views that have been (rightly) ignored since the beginning of the web. And it’s why so many SEM programs settle into a kind of stasis – with flat-line performance after a few months of steady improvement.
Ok – it isn’t possible to go from peak to peak to peak – you’re bound to hit some plateaus and even valleys now and again – that’s life. But it’s possible to use analytics strategically – not just to help climb mountains – but to help find the right mountains to scale. There is no one magic way to make analytics strategic – but there are some simple principles and approaches that can make it much more likely that your analytics guide and enhance your strategy and not just your tactics.
The First Rule: Follow the Money
Businesses often spend too much time developing and fine-tuning parts of their web program that don’t matter. It’s a pretty good bet that an inverse 80-20 rule applies here – your company almost certainly spends 80% of it’s time and money on content and optimizations that account for less than 20% of your business.
Example:
A top-tier publisher with a huge web site found that for somewhat more than half the visitors to the site, the first behavior is to Search. The client implemented a new Search tool – and saw significant improvement in page consumption. No further changes to Search have been attempted in the ensuing 18 months.
Lesson
This is a classic example of analytics driving a tactical change but not a strategic one. By highlighting the importance of search, it encouraged the company to improve the basic functionality. Having done so, there was no longer any obvious reason for change. But here’s the thing – if half your users are using Search you should never, ever stop improving Search. Unless, of course, you decide using analytics that fewer people should be using Search. Once you know how important Search is as a navigation element, you need to think about it as a kind of “navigational” hero – just the way you treat campaigns. What happens if you steer even more people to it. What happens if you place different promotions next to it? What changes on the Search Landing Page can be made to drive traffic most appropriately? What changes in Search Behavior will maximize revenue and satisfaction? Many of these are tactical questions - precisely the sort of thing analytics does well – but they are questions that aren’t raised unless you’ve used your analytics to help you focus on what really matters in the first place.
The Second Rule: Know Your Visitors
It’s shocking to find that many businesses – even with high-end web and search analytics capabilities – don’t understand even the basic visitor types coming to their site. Admittedly, this is a capability that until recently was very poorly supported by the common analytic tools. Even at the abstract level, however, web channel marketers have often done a very poor job of matching the job that a web site is doing to the targeted mix of customers. This is fatal to the strategic use of analytics because without the imposition of visitor segments on top of data, most of the interesting patterns get averaged out into useless data.
Example:
A high-end financial services provider finds that the “average” online converter has visited the site more than five times and consumed more than fifty pages of content prior to conversion. Read tactically, this might be interpreted as suggesting you need to get non-converters to read more content! Read strategically, this suggests that putting significant effort into the experience of re-visiting the site should focus dollars on more likely returns. The company tracked repeat visitors, created specialized re-entry experiences and issued special privileges for the target group with excellent results.
The Third Rule: Force Structured Change
Any analytics professional will probably agree to following statement: you can’t measure more than one thing at a time and you can’t measure in the absence of change. It isn’t a paradox – it means that to use analytics effectively you need to always be testing exactly one thing.
Example:
A large software company had, over time, done a fine job incrementally optimizing the web channel for one of its products. But with a change in management, the new team tore up the existing site and completely replaced it with a alternative they were sure was much better. Direct Sales fell and several of the secondary functions of the site suffered even more severely. No one was willing to go back to the old site, and no one could really tell what wasn’t working compared to the previous version because so much had changed.
For a company that falls into the “revamp everything once every year” philosophy of web design, analytics – strategic or tactical – is nearly impossible. Because here’s the thing – no amount of analytics against an existing web site can answer basic strategic marketing questions (like “should I emphasize price or service?”). But a simple A/B test can give a decent answer almost immediately. If you haven’t got a hundred things you’d like to test on your web site, you aren’t doing a good job of addressing strategy. And if you’re trying to test more than one of these at once, you aren’t going to get a good job of analytics.
If you follow the money and pay attention to what your analytics tells you really matters; if you concentrate on visitors and focus your efforts on meeting the needs of the most important groups; and you both limit and structure your change to be aggressive but incremental, then there’s a pretty good chance that your analytics can pay off with big time strategic returns.
Comments