Measuring Search Engine Marketing as a Channel (Continued)

Part 8 in a Series on Web Analytics and Search Engine Marketing Programs

The last post in this series covered several different types of SEM channel analysis: channel cannibalization (the extent to which one online channel borrows or supplements traffic in another channel) and studying a specific channel as a process (the manner in which visitors use Search over-time). Each of these can fundamentally re-shape your understanding of the type of visitors being sourced by Search, how effective your Search Program is and which pieces of your Search Program are most important and cost-effective. This, however, hardly exhausts the subject of channel analysis.

In one way or another, most channel level analysis is ultimately about the allocation of resources. For every company, both time and money are fixed assets that need to be allocated as efficiently impossible to drive long and short-term returns. This means that one of the key problems that every company faces is how to make more intelligent decisions about what to do and how much to do it.

In the real world, it’s often hard to fathom how large companies actually make resource allocation decisions. Particularly in the realm of Search Engine Marketing, it can be baffling to understand why, for example, PPC programs are sometimes aggressively funded while SEO programs languish. The processes that drive such decisions are a compound of political and cultural decision-making factors – where likely optimum points are more often missed than not.

Now good analysis can’t necessarily pinpoint optimum resource allocations. It isn’t possible to know with any certainty what an additional allocation of time and money will accomplish in a channel. Programs do not, under almost any real-world conditions, scale perfectly. In truth, resource allocation decisions should come attached with that rider you always see on Mutual Fund performance claims: "Past Performance is Not Necessarily a Guide to Future Returns!"

However, there is a method by which the analyst can help decision-makers identify programs that are at or beyond their ideal scale. The idea is a simple one – if you track a good Conversion Proxy or Engagement Measure by Channel over a significant period of time, you can measure the point at which the percentage of unqualified visitors begins to rise sharply. When you hit this point, there’s a pretty good chance you’ve begun to max out the opportunity in a channel.

This is a very simple analysis – it’s simply a trend-line of the percentage of unqualified visitors by channel over an extended period of time (ideally since program inception). When you build this trend line, you’re tracking – in the most common case – two countervailing tendencies. Programs tend to get somewhat more efficient over time. Landing Pages are improved. Navigation paths get better. Ads are tuned. All of this tends to improve engagement and results. On the other hand, if you’re expanding within the channel then your program is growing larger. And scale almost always is accompanied by some diminution of quality.

It’s easier to focus on the percentage of unqualified visitors because it is much harder to change the behavior of unqualified visitors than qualified visitors. By looking at unqualified not average qualification or percent qualified, you’re screening off some of the effects of optimization (such as improved Landing Pages).

This isn’t the sort of analysis that should be followed with a hair-trigger. Declines and variations will happen in the course of regular business. It’s important to establish that you’ve got a consistent drop in quality before thinking about the consequences.

When this does happen, I think two courses of action are appropriate. The drop in performance may be indicative of a SEM program either losing freshness or a change in the competitive environment. Either way, a careful survey of competitive programs, competitive advertising, the keywords being used and the creative approaches may all be in order.

If the results of this freshening aren’t particularly positive, I’d be inclined to think that a program may have reached the limits of its current scale. If so, it would mean either exploring alternative scaling mechanisms (like video or local search for example) within a channel or simply shifting new resources into different channels.

A basic analysis of the percent of qualified visitors can also help a company just beginning to explore new channels. If a channel is going to prove fruitful in the long run, then there’s a good chance that your initial cherry-picking efforts will yield a good percentage of highly-qualified traffic. Comparing qualification rates for new channels can help you make a decision about which channel might be more promising for your company to explore first.

The next analysis I’m going to talk about is really and truly a channel analysis. It uses the power of measurement on the web to help optimize mass media offerings. Once again, the idea is fairly simple though its execution can be tricky.

When you run mass media campaigns, this will almost always have a strong impact on your web traffic. Everybody knows this. Effective mass media will impact your direct traffic and your Search Traffic. Indeed, Search is so ubiquitous as a finding method that most online marketers have given up trying to track with vanity pages. Customers simply don’t use them.

But although every web marketer whose company does mass media is keenly aware of the traffic effect, very few have ever taken advantage of the superb measurability of their medium to evaluate the effectiveness of mass media offerings.

The key to doing this (as in most analytic cases) is establishing a control. There are some great ways to do this. I think the best is to use geographic segmentation of your web traffic. Suppose you have two TV commercials for your product. You can run them in one of the several common testing media markets in the country. Before you do this, setup rigid baselines on your web site tracking all of the following for the test DMAs, some control DMAs and the entire site:

• Traffic
• Traffic Variation
• Traffic by Source
• Traffic Variation by Source
• Qualification Level
• Qualification Level Variation
• Qualification Level by Source
• Qualification Level by Source Variation
• All of the above by key visitor segments

It’s vitally important to understand the inherent level of variability in these measures. If you don’t, then you may fool yourself into thinking the perceived changes are actually significant.

Once you have this baseline, however, you have a powerful tool for measuring the differential effectiveness of TV (or any other mass media) campaign. Simply track the baseline periods for the control groups vs. the target media markets. You can measure comparable traffic impact, comparable qualification impact and comparable impact by visitor type.

This information would be a great way to use the web to help optimize significant mass media expenditures. And while this tracking may not replace many traditional forms of mass media analysis (such as survey or focus group work) – it provides a number of benefits. After the initial setup, it is very inexpensive and scales well. It is quick – providing something close to real-time tracking of campaigns. It measures real behavior change – not just "memory impact." All of these are significant.

With the amount of money that some organizations spend on mass media campaigns, the web site might be more important as a cross-channel optimization tool than as a conversion tool in its own right!

Web Analytics is about so much more than the tactical improvement of the web site. These channel-focused analytics help decision-makers make large-scale decisions about the direction of the business. From deciding on which new marketing channels to exploit, to knowing when a particular channel has scaled sufficiently to measuring the impact of major expenditures in mass media, these analyses transcend much of what we think of as the traditional domain of web analytics.

I used to advise beginning web analysts to "know their website." And this is vitally important. Now, I’d be more inclined to say "know your business." That includes the web site, of course. But only a good knowledge of all the aspects of the business can insure that an analyst is focusing on the truly important problems and is able to use web measurement in unorthodox ways to make a difference!

Other Posts in this Series: Introduction, Searchnomics Issues, Getting Setup for SEM Analysis, SEM Data vs. Web Analytics Data, High-Level Search Engine Reporting, Analyzing Search Traffic in more Detail, Measuring Search Effectiveness for eCommerce Sites, Measuring Search Effectiveness without Conversions, and Measuring Search Engine Marketing as a Channel.

And don't forget to register for X Change and join our fantastic line-up of experts for in-depth conversations on web analytics and measurement!

Comments

[Milorad]

I liked this post and am very interested in the particular topic of measuring impact of offline media campaigns using web analytics. You mention "common testing media markets" above. Do you mean that there are particular markets in which web companies usually test efficiency of their campaigns because of being able to more easily setup the test and control groups for some reason? If so, can you please provide examples of such markets and mention which offline meadia are they usually used with (e.g. newspapers, magazines, TV, or some combinaiton, etc.)? I would also be very interested in any furthere references on the same topic in books, blogs, etc.