Measuring Search Effectiveness for eCommerce Sites

Part 5 in a Series on Web Analytics and Search Engine Marketing Programs

[Of all the many reactions to my last rather scorch and burn post, the two that pleased me the most were Marshall Sponder’s and Douglas Karr’s. There were two things that pleased me about Marshall’s post: first that he "laughed and laughed" as he read it. Second, that he manages in a more gentle and probably reasonable way to sum up pretty much what I was trying to say. If I "stewed a few Irish babies" to get there I guess that’s just me. It isn’t the first time Marshall has managed to get to the pith of what I was saying in rather fewer and more polite words. Speaking of which, you should check out Douglas Karr’s extraordinarily pithy comment below. It made me nearly spit my Diet Coke out with laughter. And I can only say – touché! BTW, I posted (and unlike my usual practice, will post) every official Typepad comment. The following is the continuing part of the series on SEM Analytics that got supplanted from its usual place.]

Part 3 of this Series covered high-level Traffic Analysis: comparing Search Traffic to Total Site Traffic, Organic to Paid Traffic, Traffic by Engine and Traffic over time. Collectively, these high-level reports provide the basic context for understanding the impact of SEM on your site. Part 4 delved into the next level of SEM reporting: looking at traffic by Ad Group, Entry Page and, of course, Search Term. These reports are essential in most of the analyses from here on out. But all of the Posts so far have concerned themselves with looking at "Traffic." There are a few types of analysis (such as SEO Holes) that just focus on traffic. But most reports – whether they focus on finding growth opportunities or improving efficiency – demand the addition of some form of success metric. For many sites, finding the right success metrics for optimization is the single most important task in SEM Analytics.

If you’re site is a traditional eCommerce site, then finding the right optimization metric may look fairly straightforward. But even in this apparently simple situation, there often lurks considerable complexity. Here are five quite reasonable optimization models for eCommerce sites: optimize to sales per visit, optimize to revenue per visit, optimize to sales per visitor, optimize to revenue per visitor, optimize to lifetime value per visitor.

I’ve listed these in increasing order of complexity. Optimizing to sales is the easiest – both in Bid Management Tools and in Web Analytics tools. And it’s a perfectly workable solution in SOME situations. If the bulk of your conversion is single session, if you don't sell many products over time to a single customer and you have a relatively narrow band of product pricing, then sales per visit will work well enough. If any of these conditions aren't true, then optimizing by number of sales will cause significant mis-optimizations in your program if used as the goal.

Optimizing revenue per visit fixes one of these potential mis-optimizations. By taking into account the value of products sold (or – in cases where it doesn’t track well – the Margin on Products sold), you insure that your PPC program doesn’t optimize to the lowest-value conversions. But if you have significant multi-session sales cycle or significantly different customer values over time, then you’ll still be mis-optimizing.

Using Sales per Visitor fixes the other half of the equation. By going to a visitor metric, it solves attribution by visit problems. But, of course, it does nothing about mis-optimization by driving to low-value carts or different customer value.

It would seem that Revenue (or Margin per Visitor), our fourth model, would be the ideal. And indeed, it does solve most optimization problems. However, in cases where you expect significant ongoing sales or other revenue from 1st Time Buyers, then you may significantly under-represent the value attributed to a marketing campaign if you only track the current measured effects. Since optimization has to take place within a specific period of time, optimizing to revenue can have two negative effects: it can make it look like your programs are less effective than they actually are, and it can optimize purchases without a long value tail vis-à-vis purchases with a long value tail.

There are two key points you should take away from this. First, no one optimization strategy necessarily fits every site. Using Customer Lifetime Value is the most comprehensive strategy – but it’s also by far the hardest to implement. If you have good reason to believe that most of your optimization IS reasonably captured by using sales per visit (or any of the other metrics) then the simpler metric is the better choice. The second key take away is that choosing an optimization tactic is a HUGE decision. Because if you choose the wrong tactic, then every optimization effort you make will further mis-optimize your program. So before you pick one of the short-cut methods, be sure you understand what you’re missing and are fairly confident that it IS NOT important.

Choosing an optimization point has been compared to picking a direction on a map. If you pick the wrong direction, then every step you take going forward will move you FURTHER away from where you want to be.

The classic case of this type of mis-optimization is when SEM programs optimize to CLICKS (traffic). Typically, you start out with a program where the keywords and bid points are set by human intuition and reflect a reasonable understanding of which keywords likely produce good traffic and which are less valuable. But as the program is optimized for Clicks, bad traffic drives out good traffic. And the program will focus on increasingly worthless (but cheaper per click) traffic with each optimization. Eventually, you’ll end up with a truly rotten program. What’s ironic is that the harder you work to optimize in this situation, the quicker your program will go bad.

If you’re a traditional eCommerce site would you ever not focus on Sales or Revenue or LTV for optimization? As a matter fact, you might. For some sites, the rate of Conversion to Traffic is quite small. That means that if you try to optimize your SEM campaigns on Sales or Revenue, you may have to wait a very long time before you have statistically significant numbers. And for many aspects of your program (such as creative rotations and individual keywords) you may NEVER have statistical significance. That’s a big problem! So if your eCommerce site is stuck in this boat, you’ll need to consider the techniques for building models of Engagement or, as we sometimes call them, Conversion Proxies. That will be the topic of my next post in the Series – and it will deal with the many, many situations where eCommerce transactions either don’t exist or are two infrequent to optimize against.

Other Posts in this Series: Introduction, Searchnomics Issues, Getting Setup for SEM Analysis, SEM Data vs. Web Analytics Data, High-Level Search Engine Reporting, and Analyzing Search Traffic in more Detail.

Comments

[Douglas Karr]

:)