Part II
In my last post, I talked about the advantages to putting together a detailed plan for what you want to accomplish with web measurement. The major pieces of the plan are highly specific analytic projects - and the goal of the plan is really two-fold. The plan should provide clarity throughout the organization on what web measurement can (and should) accomplish, along with the resources required. It also provides a framework for your analysts so that they focus on the right problems and understand the basic methods for solving those problems.
Building this type of plan begins with a very extensive brainstorming session with the key stakeholders. Ideally, this session should include senior online managers and stakeholders across the organization. The goal is to understand the key business problems facing the company in the next year. This session will typically surface any major areas of dissatisfaction with the online measurement, data, analysis or reporting that’s currently in place. But it’s important to understand that this session IS NOT about the analytics and measurement. It’s about the business.
After the meeting, we try to develop analytic projects that are designed to meet the business needs and answer the key issues that were surfaced.
As we develop possible projects, I usually try to group them into a few broad conceptual categories. These may be adjusted, but usually they look something like these: Foundational Projects, Descriptive Projects, Optimization Projects and Integration Projects.
Foundational Projects are focused on the basic infrastructure of measurement. Projects in of this type may involve data quality improvement or control, tagging improvements, tool changes and upgrades and management reporting. All of these are concerned with the basic table-set of what information is available to whom in the organization. I lump management reporting in this category because, done correctly, it provides the basic foundation for measurement discussion and dissemination in an organization.
Descriptive Projects are analytic tasks whose basic function is to help the business understand a particular set of behaviors or tools. These projects aren’t generally focused on optimization – they are designed to answer basic questions. Let me give you an example. A publisher introduces RSS to their site and then wants to know how the use of RSS impacts overall site loyalty and behavior. This analysis may not have any particular optimization goal in mind. The publisher is unlikely to remove RSS no matter what the analysis shows. But managers want to understand how this new set of behaviors impact their understanding of the site and how it works.
Optimization projects are the ones we typically think of as the core analytic function. The purpose of these projects is to study a system or some piece of system and make recommendations for change. Typical optimization projects are of the sort I’ve blogged about in the past in my “how-to” posts: things like internal search analysis, SEM Analytics, Functional analysis (which is more than most also a Descriptive Analysis), Form Abandonment and Funnel analysis, etc. There is some overlap here. Nearly any Descriptive Analysis MIGHT produce optimization suggestions. And any optimiziation analysis WILL include a descriptive component. But I find that the two types tend to be quite distinct in the real-world and most projects are easily understood as being of one type of the other.
Integration projects are fairly self-explanatory – I class any analytic project where you need to combine information from multiple systems in an automated fashion as an integration project. Obviously, these projects can also be Foundational, Descriptive or Optimization oriented. But I like to treat them as a separate beast because of the special requirements integration typically demands. Whenever you need to do data integration, you’re in the world of IT projects. That means new stakeholders, different timelines, and often a different level of expense.
I try to build a very large buffet of possible measurement projects across all four categories. The goal is to try to find projects that are responsive to nearly every business need I heard. When putting this buffet together, I describe each project in brief. The point is simply to communicate what the project might entail and what it should deliver.
The output from this session might look something like this for a single project:
Internal Search Optimization
• Assess Search effectiveness as a Router
• Identify key drivers to Search
• Identify key terms for manual optimization
• Target search monetization (Ads and Results optimization)
• Moderately Difficult
• 4 Weeks Duration
• High Impact – 40% of visitors search
Typically, a buffet will have between 30-40 such project descriptions. Once this is in-place, the next step is to re-convene with the key stakeholders and present the projects. The idea is to get reactions about whether the proposed projects make sense, resonate strongly, and meet the key needs. This meeting is also designed to surface potential timing issues (i.e. we are replacing Internal Search in Q2) and organizational plans that might impact the plan. When I present the buffet, I’ll try to provide real-world color about the projects: how often they tend to have a big impact; why they might not; who is likely to benefit from them. All of the sorts of things that can help managers get a feel for whether a project is truly worth doing.
Based on the input from this meeting, the next step is to begin to layout projects. There is no science to doing this. But some basic rules nearly always apply. Obviously, you’ll want to prioritize projects based, to some extent, on business impact. The stuff that really matters needs to be tackled first. But business impact isn’t the only factor to consider. Sometimes, Foundational (or integration) Projects with relatively modest immediate impact need to be completed before a much higher impact analysis CAN be done.
If this is a new planning effort, it’s equally important to take into account the skill level of the analysts doing the work. For a new department, putting simpler analysis tasks first – even if they have less business impact – will usually make good sense.
And, of course, you can’t ignore the political environment of your company. Certain stakeholders may be more important or just more demanding. Knowing the “who” is often as or more important than the “what” or the “how.”
All of these factors, plus the total known resources available, come together to shape a plan. The plan culls down the list of 30-40 buffet projects into the dozen or so that might reasonably be tackled in a given year. Where projects appeared to have high-priority but aren’t supportable with known resources, they are put into a special section of the plan.
Once the list has been narrowed, the last step is to produce a more detailed description of each project. If you’re doing this internally, this step may or not be necessary. But when we do this as an engagement, the detailed descriptions form the essence of the deliverable. The descriptions provide a much more complete and detailed project outline. They include a fairly detailed overview of what the project is, an assessment of its interdependencies and the resources and time needed to complete it, a clear statement of what the analysis should produce AND an overview of how the analyst can actually do the project.
For organizations building a new analytics capability – this last part, the guidance for the analyst, is often the most important piece of the plan. Many analysts need some guidance about what to look for and what to look at to complete a project. Providing this insures that the plan is truly actionable.
The last step is to present the plan back to the stakeholders. This is truly important. You need to make sure everyone understands and signs-off on what measurement and analytics are going to accomplish in the coming year and what resources are required to get those projects done.
The beauty of the Analytic Roadmap is that it approaches the resource issue from the business-end – you start with what the business most needs to accomplish and work toward how it can be done. In a sense, it’s like zero-based budgeting. You don’t start out assuming you have one resource and then trying to figure out what do to do with that resource. You start out by assessing needs, identifying solutions and then presenting a plan. Once stakeholders understand what’s in the plan, they are in much better position to assess the resources they want to devote to it to get things accomplished in a timely fashion.
In my experience, building an Analytic Roadmap like this brings a dramatically improved level of clarity about measurement throughout an organization. It is as if a light goes on – “ah – this is what web analytics will produce – I get it now.” That, in and of itself, is a great reason to undertake a roadmap. But the benefits extend much more deeply. The plan is every bit as much about making sure your measurement team is truly effective as it is conveying that value story out to the rest of the organization.
So as I said at the end of the last post - there’s one New Year’s Resolution I’d recommend to anyone managing a web analytics effort: for 2008, resolve to have a plan!
Comments