Before I delve into the my real topic for today, I wanted to briefly remark on the comment James Gough left about my 10 Reasons we all have Ulcers post:
If we were aware that it would take at least 6 months and $70k to get even a basic version of Omniture installed we would have never even begun in the first place. The problem with the likes of Omnniture and websidestory is they take months of technical time to implement and then becomes a constant battle to keep the tags up to date as you deploy new microsites and pages. We have moved to a solution that requires no complicated tagging and have finally allowed marketeers to control and believe in the metrics. implementation and the time to manage ongoing is the biggest cost to a business of installing these products and vital companies look for products that have a light foot print but rich reporting.
I can’t argue with James’ assessment of the costs – it can be done cheaper and quicker than that – but it can also be a lot more expensive. And what he says about keeping the tags up to date and implementation and management time are pretty much dead-on. These are particularly severe problems for big-time publishers. Frankly, I’d be very interested in hearing what solutions James settled on and what they are doing with it. I’m a big believer in a light tag myself – but I haven’t found an implementation out there that doesn’t sacrifice some things about Omniture and WebSideStory that I really like. I’ve asked James to elaborate and, assuming he does and is okay with it, I’ll post it up here.
I could probably take up any of the 10 "Ulcer items" and spend at least a few posts on each – but I’m going to focus on #9. Not only because I think it may be the most important but also because I want to use it as a lead-in to series that will tackle real "how-to" issues for some of the most common types of site analysis. Number Nine was:
9. Not Having a Road-Map
Probably even more important than a good method to getting where you want to go with web analytics is having a clear analytic road-map. I think the biggest challenge for most organizations is after the honeymoon (post-implementation) – when everyone has gotten over the joy of just "having data" and actually wants to do something with it…
When I say that this is the biggest challenge for most companies, I mean it. Nowhere, in my opinion, is failure more likely or more common. And it is at this stage that companies are most likely to engage a company like Semphonic and, I hope, get the most value from us.
But stating the problem is not exactly putting forth a solution. It’s all very well to say an organization should have an analytic road map when the real problem is that no one knows what might come first much less what might follow in the next 10 months. It’s like asking a novice chess player to describe an elaborate attacking combination when they are still trying to learn the basic moves.
So what is an analytic road-map, what goes into it and how can an organization go about creating one?
What is an Analytic Road-map?
When I talk about an Analytic Road-map, I have something very particular in mind. In essence, it’s a plan that lays out a series of analytic projects. Each project is a discrete analysis. And the plan specifies which projects are going to be done and in what order. That’s it. A bare-bones, simple road-map might be sketched like this:
Functional Site Analysis -> Internal Search Analysis -> Tool Loyalty Analysis -> Home Page Real-Estate Analysis -> Landing Page Analysis
Simple.
What goes into the Road-map?
This gets a lot more complex. Any sort of analysis might go into the roadmap. And the actual make-up of a road-map is inevitably a complex compromise. Three factors – none trivial – need to be sorted out when building a road-map. First and most important is what types of analysis are most likely to have a large impact on site success. What makes this particularly challenging, of course, is that you have to predict this before doing the analysis – and we all know that many analytic projects simply don’t end up providing interesting information. That can’t be helped, but there are some good guidelines for tackling the value question that I’ll try to explicate.
The second consideration for the road-map is how hard an analysis is likely to be and what types and amounts of data are essential. When you put together a plan, you ignore these issues at your peril. Again, you don’t always know what data you’re going to need. But it’s not always a mystery either. Some kinds of analysis will clearly take lots of long-term data, others are obviously going to require external data integration. You’ll pretty never want to start with either – but you shouldn’t ignore them either. Many of your most valuable analytic projects are going to fall into one or both of these camps.
Finally, when you build a road-map you need to consider the sophistication of the analysis relative to the organization and the extent to which one analysis may depend upon or deepen another. You might think about this as similar to putting together a curriculum for a student. Introductory classes provide a foundation and language which can be gradually deepened. In an academic curriculum, it’s usually assumed that the deepest classes are the most valuable. You’d skip the introductory classes if you could – it just isn’t possible for most us. In analytics, on the other hand, there is no such correlation. The simpler analytic projects you might start with may drive as much or more value as very complex ones. It’s important, in analytics, never to equate complexity with value. In my experience, the relationship is more likely to be inverse than direct!
How do you create a Road-map?
When we help clients put together a road-map for analytics, we start with what type of site is involved. The type of site (eCommerce, Lead Generation, Ad-Based, Customer Support, Operational, Branding, etc.) dominates every other consideration when setting the analytic table. For eCommerce sites, we’re going to be choosing from a grab-bag of analytic projects that include: Functional Analysis, Cart Drop-off, Cross-Sell Opportunities, Personalization Strategy Analysis, Internal Search Optimization, Completer Optimization, SEM Checkup, Longitudinal Analysis, Up-sell Analysis, Market Basket Analysis and Re-Assurance strategy. Typically, we’ll begin with a Functional Analysis. I like to put this first because it sets the table so well for analytics in general – providing a great way to baseline performance and get a common working vocabulary. A Cart analysis is somewhat de rigueur – but it isn’t an analysis I look forward to. It often bears no fruit these days, and if I think the cart has already been well optimized I look push this one back. For many eCommerce Sites, internal search is vital – and it’s rarely been fully optimized. So that’s an area I’d be strongly inclined to bring to the front. Likewise, if personalization hasn’t been well addressed the Personalization Strategy will almost certainly bubble to the top. In each case, it’s the likely impact to most sites that makes me push these projects forward. You also need to have nose for the money and to follow your intuitions about how well it’s being spent. If a site is investing heavily in Search Marketing, then the hard dollars flowing out will almost always make this a top candidate for the roadmap. That’s especially true if you lack confidence that the effort is being well conducted.
For a Lead-Generation site, some of the same types of analysis also come to the fore: Functional Analysis, and Personalization Strategy are almost always near the top of list. However, Completer Optimization is an analysis that is quite simple and often has considerable upside for Lead-Generation sites. So I like to put this one near the top of the list. In addition, Longitudinal Analysis is usually a critical component of Lead-Generation (focusing on which channels ultimately drive lead-quality) – but I try never to schedule right up front because it typically involves significant integration. You can’t ignore lead-quality issues, however. So it’s imperative that once you’ve put together a couple of easier wins that you actually tackle this issue.
Ad-Based sites have quite a different focus. As with other sites types, my inclination is still to lead with a Functional Analysis. And Internal Search analysis (often more important to these sites than any other tool or page) will nearly always come next on the list. But after that, the essential analysis is one focused on building a model for the impact of each site component on consumption. This analytic model will form the basis of nearly every other piece of work you do, so it pretty much has to come early in the process even though it is always tricky. If the site is sophisticated, I might be inclined to tackle Personalization Strategies next. Media sites have been less inclined to adopt personalization strategies than have high-product mix eCommerce sites – but I think the upside potential for media sites is even greater.
The sense I hope to give from these remarks is how to begin thinking about what to tackle first and where to go from there. You start with what seems obviously important in terms of usage and value on your site. Consider what’s involved in a possible analysis. Then start to fit the pieces together in a plan. Keep in mind that while some plans may be much better than others, there’s no such thing as the "right" plan. If you can’t decide which of two projects to do first, don’t agonize over it. Pick one then slot the other in behind it. No big deal.
On the other hand, if you can’t decide which of ten projects to do first, you may need help!
What makes a Road-map valuable?
Like any other significant organizational effort, you can’t expect to get much from analytics unless you have a plan and can manage to your success. Analysts, like everyone else, need guidance about what to focus on, how long to spend on it and what to think about for the long-term. The Road-map is a way to provide all that – it’s also a great tool for building consensus within an organization about what analysis is for and what problems most need addressing. It’s also a way to set expectations for what analytics is going to produce.
Like most other plans, an Analytic Road-Map isn’t meant to be an iron-clad blue-print. Changing business circumstances, the results of each analysis and even learnings about tool capabilities can and will change the priorities that shaped the plan originally. But if you have the plan, chances are you’ll be much clearer in thinking about when change is necessary and why you are doing it. You’ll also know, when you go chasing off after a new problem what you are giving up or postponing. That, in itself, can be a valuable defense for measurement organizations who find themselves being passed like a public handkerchief from problem to problem by Senior Managers with fleeting (and pressing) data whims. Sometimes, those short term needs are in fact more important than even the best-laid plans. But sometimes changing directives come when you haven’t done a good enough job communicating what you actually think you SHOULD be working on.
Good article. We are finding the same from our customers. We replace tagging with passive date capture and emulate the tags so we can get the same detail with no tagging. We jsut replaced Omniture at one major site. The marketing guys liked the reporting and wanted more. The IT guys said enough is enough, no more tagging. The compromise is Webtrends tag server at the customer site bieng fed by BeatBox Capture emulating the tags. As users get to like analytic, they want more and then the real problems of tagging kick in.
Posted by: Doug Watt | April 16, 2007 at 08:27 AM