SemAngel

Planning for eMetrics

If last year is any guide, I’ll be blogging regularly about X Change. We are nailing down Huddle Leaders (which is really exciting) and I’ll certainly be talking about that in the near future. But much closer at hand is our industries largest non-vendor Conference – I’m referring of course to eMetrics.

I’m going to be speaking a couple different times at eMetrics and I thought I’d talk about each of them at some point in the blog. I suspect that the most challenging presentation is one I’m sharing with Eric Peterson and Joseph Carrabis on thinking about and measuring engagement. It’s challenging partly because I find the topic difficult and hard to pin down and also because I’ll probably be lost between two out-sized personalities like Eric and Joseph. Frankly, I’m an in-sized personality. This is a topic that Eric and Joseph have been spending some real time on and have developed a particular point of view which I’m not sure I share. And I really mean it when I say I’m not sure – both because I don’t know enough about what they are thinking and because I’m not even sure what I’m thinking.

Anyway, I’m going to try and lay out my thinking on measuring engagement in some detail in the blog before eMetrics – partly as a kind of cheat sheet. It’s a truism that you don’t know what you think until you say it – I hope to figure it out as I write it so that when I say it at eMetrics it will meet some minimal public standard of clarity and coherence.

I expect to be on more comfortable ground in my presentation with Barbara Coll of SEMPO and Michael Stebbins of Market Motive on measuring Search Engine Marketing. This is a topic I’ve spoken on regularly. But even better, it’s a topic I work on regularly. I’m hoping to take just a little bit of what I talked about at SMX – specifically a richer set of reports for measuring SEM programs – and discuss in much greater detail.

My last presentation is also on comfortable ground. I’ll be co-presenting with Nancy Abila of O’Reilly Publishing on creating an Analytic Roadmap. Semphonic worked with O’Reilly late last year on putting together a Roadmap for 2008 and we’ll be discussing that process – what we did and how it worked. Nancy faces a pretty common challenge for many companies – driving and standardizing a measurement effort across many properties with significantly different goals and structures – and doing it without a huge team or a giant investment. We’ll talk about how we built a plan and how the process itself helps an organization crystallize priorities, set reasonable expectations around measurement and target analyst resources most effectively.

These are all (pretty much as usual) totally new, never-before-seen presentations. I just can’t stand re-hashing the old stuff. So if you’re on the fence about eMetrics or haven’t made plans yet, I hope you’ll decide to attend. I’d love to see you there and I promise I will do my best to make it worthwhile.

Never Make the Same Mistake Twice

Part Seven in a Series on Analyzing Conversion Funnels and Form Abandonment

Most formal processes of continuous improvement rely heavily on the simple idea that if you made a mistake you should figure out why and try to eliminate the possibility of it happening again. In “Part Seven” of this series on measuring form processes I’m going to talk about extending that idea into web forms processes.

I’ve looked at and measured many online processes over the years. And it’s safe to say that in almost every case errors inside the form were invisible to the web analytics solution. This used to be for the good reason that there was no way to report such errors but this is no longer true. With modern tag-based systems, you can almost always pass information about process errors to custom variables in the web analytics tool.

It doesn’t happen very much for two reasons: it takes extra work when building the process and most people erroneously assume that such errors are uncommon.

I get the extra work problem. That’s always a disadvantage.

You can save yourself a lot of effort and get yourself a very powerful tool for tackling continuous improvement in complex form processes by acquiring a CEM tool like Tealeaf. But Tealeaf-caliber tools aren’t for everyone and there is much that can be done even within the more limited constraints of a traditional web analytics tool. Nor is the amount of effort here so burdensome as to make the work a difficult call when it comes to return on investment.

As to whether forms processes errors are uncommon, I can only say that my experience suggests that while errors and problems are not everywhere neither are they an endangered species.

So what you should do to track errors?

The most important thing is that you should have a measurement standard for any form process. This standard should embody a few simple ideas: every time an error message is returned to the user the web analytics solution should know about it; every time a client-side field fails validation, the web analytics solution should know about it; and every time a popup help or error message is displayed to the user…you guessed it, the web analytics solution should know about it.

There are two fairly straightforward strategies for achieving this. You can send custom variable encodings to the web analytics tool whenever such an event occurs or you can save them and pass them all at once whenever the form is abandoned or continued.

Either way, you’ll typically want to report to the web analytics tool a variable that contains all of the following:

The Page Name of the Process : The Type of Action : Field of Action : The Message or Description

'Page Name' is obvious. By including it in the variable encoding, you can make it easier to report on error messages and understand when an error message is getting triggered (many applications have errors that are redundant across pages). The 'Type of Action' describes things like “Client-Side Validation Failure.” The 'Field of Action' describes the field of the form (where appropriate) that triggered the call. The 'Message or Description' passes back the Error Message or additional descriptive text about the event.

With this style of encoding, an error on an entry-field in a form might result in a web analytics variable being populated with:

“Shipping Information – Step 3 : Field Validation Error : Ship State : Not Entered”

Collecting every error in a process in this fashion greatly extends the utility of web analytics conversion analysis. In addition to the obvious fact that you’ll have true visibility in the number of errors that are occurring, you’ll get a host of useful additional information including:

• Which pages/steps have the highest error rates
• Which fields have the highest error rates
• Which errors are most common
• What is the conversion impact of each type of error
• What is the conversion impact of cumulative errors

This is valuable stuff. High step or field error rates may indicate true problems in the form process. But they can also potentially identify places where additional help or explanation may be required. They can even be used to help target form step length appropriately - putting too many fields in a form step can result in high error rates and increased frustration.

When you remember that these conversion processes are almost always the highest-value places on your site, it’s not hard to see why understanding potential visitor problems in these segments is critical.

Everything that I’ve said so far applies – emphatically – to Ajax and Flash forms as well. With more sophisticated programmatic forms, pretty much every part of the measurement must be hand-built anyway. So there really is no excuse to not trap and report on errors as well.

One special note – when you report custom variables as a special call in most web analytic applications, you may be sending an additional page request. You should take pains to make sure that if you are doing this, that the page request is either isolated from the rest of your system or accounted for in your reporting.

You’re not likely to ever get a web process down to the vanishingly small error rates common with sophisticated manufacturers. They simply have more control over their environment. But this very fact makes it all the more essential that you constantly track and report on what happens inside an online form – especially when it comes to errors.

I have two more posts in this series – one on analyzing multi-session behavior with respect to form processes and a final post covering form KPIs based on this whole series.

Previous Posts in this series include:

• Introduction to Forms Process Measurement
• Going Beyond Step Abandonment
• Pre-Qualifying Visitors to the Form
• Selling inside the Form
• Thank You Pages
• Form Processes and Visitor Segmentation

Vote Early and Often

When I was just out of college, I was sucked into the world of politics by my older brother – who was a passionate devotee of political practice. I’ll confess that I did it more from a lack of other gainful employment (jobs for Philosophy students being scarce) than any profound interest. Politics IS interesting though. I found that out. There are few things I’ve ever done that combine the intensity, comradeship, and either exultation or despair of a political campaign. But what ultimately made politics unattractive to me is that many people – especially the ones heavily involved - take it so seriously.

Get a group of computer programmers together and they’ll talk sports, music, movies and – of course – technology. Get a group of political operatives together and they’ll talk politics. And politics. And politics.

Which brings me to the WAA and its current round of elections. The WAA has been a valuable resource in a fledgling community like ours. Like most Associations, I suppose, it tends to take itself a bit too seriously - but it does matter and has been and should continue to be a valuable part of our broader community.

I personally don’t get involved much in the WAA and the inevitable politics. Last year I did nominate Marshall Sponder and I have worked some on his Social Media Committee (less than I should). And in the course of that I’ve learned and seen a little bit of what makes for a good Association board member (and Marshall has been a good one I think – full of energy and deeply interested in the task at hand).

Since it isn’t anyone’s real job, you have to have that energy and commitment. Which means, I think, that you must truly enjoy the job. For the most part, that means finding someone who enjoys networking, organizing stuff, and even handling the politics (all the more fratricidal in an in-bred community like ours). You also have to be very even-handed. Associations inevitably struggle to maintain some level of impartiality among members.

June Dershewitz, one of my colleagues at Semphonic, is running for the WAA board this year and she fits this mold perfectly. While it is border-line inconceivable to me that anyone would actually WANT this job, June has been organizing web analytics events, networking in the community and enjoying it for years. She is deeply interested in web analytics as a practice. Those who know her will also, I think, universally agree that she is extraordinarily even-handed. She is much more likely to go out of her way to give everyone an even break than to ever push for a political interest.

If you already know June this is a pointless “Of course.”

But I certainly don’t know all the candidates for the board and I’m assuming that most of us are in the same boat. So if you are a member and voting in the WAA elections and you have some slots you’re just not sure about, you really should include June as one of your selections. I know of no one - on the board or off - who is likely to serve better.

Just don’t ask me about this Obama/Clinton thing.

Read up on all nominees

Voting will open on Monday, March 24, 2008 at 12 PM GMT. Voting will close at 11:59 PM GMT on Monday, April 7, 2008.

Everybody’s Talking at You

I have tons of stuff to write about queued up (and not much time), but I wanted to quickly make note of the fact that I've begun podcasting my blogs. The still incomplete series on Form Abandonment and Conversion Analysis is available as is my recent X Change post. Here is the link for iTunes or your Reader:

http://www.semphonic.com/webrss/garyangelonwebanalytics.xml

The podcasts are all around five minutes (okay – around six minutes) and are essentially just the blog slightly transformed and read in my dulcet tones. If you are an avid reader, I wouldn’t really recommend both listening AND reading - the content is pretty much identical. But I’m going to be doing nearly all my blogs (at least the real, informational ones) as podcasts as well. So if you’d prefer to listen while you drive or exercise, then taking the podcast feed might be a better path than straining your eyes each week to absorb yet another 2000 word post.

Talk to you later!

The Future of Web Analytics

In a recent blog posting, our own Paul Legutko tackled what I think is one of the thorniest questions in web analytics: to what extent will web analytics continue to exist as a separate discipline and to what degree will it be absorbed into broader disciplines around BI, business strategy and general marketing. This isn’t just an issue for web analytics consultancies like Semphonic – it has implications for any company building up web analytics resources.

Paul points out that business success and online analysis have already become entwined in ways that make it frequently essential to go outside the web analytics box to produce meaningful analysis. And we can all see that the major vendors have been focusing as much or more on building out the web analytics “ecosystem” than on enhancing core web analytics capabilities.

In the end, Paul mixes equal parts “is” and “ought” to suggest that web analysts need to understand and respect their boundaries if they want to keep them:

“Web Analytics can continue to be its own discipline and field of expertise as long as the online channel remains at the core of the consulting engagement.  A Web Analyst can be happy to analyze call-center data, but only in the context of call-center savings because the same visitors use the website instead of the telephone.  Offline conversions are interesting in so far as the same people can be identified as also visiting the website, or were sourced online originally.  By not keeping the online channel as the focus of analysis or consulting, Web Analysts run the risk of losing their identity.  In my view, Web Analytics consulting has a future as long as it remains true to its core: the online presence of a company and how it contributes to business success.”

At its core, I take this as sound advice. If you – a web analyst – are trying to take-over the analysis of call-center operations or TV buys, there’s a pretty good chance you won’t end up analyzing anything. First you’ll lose the battle around call-center and then you’ll lose the war around analytics.

But I also believe there is a second, largely unanswered question, here. Namely, should an organization WANT a separate web analytics function and is there always going to be a reason for such a function to exist or is the current situation merely an artifact of history and tools?

It's not an easy question to answer.

First, I don’t believe there is any strong consensus around what exactly web analytics is. I see two fairly distinct flavors of web analytics. One variety is marketing optimization focused on multivariate testing, behavioral real-time targeting, and campaign optimization in Search or Banner or eMail. The other flavor – which I tend to call classical web analytics – is focused on the actual visitor behavior within the web site. Classic web analytics tends to address problems around web site use, navigation, purpose and function. Like most full-service consultancies, Semphonic plays on both sides of this divide. But I’d say we are somewhat more focused on classical web analytics (except for our Search Analytics practice) – while I’d classify some of our competitors as clearly more focused on marketing optimization.

Both sub-disciplines have legitimate historical and technical reasons for being distinct. Marketing optimization has been distinct from broader marketing consultants for a variety of reasons. Few traditional marketing practitioners have a clue about new media. The differences in channel understanding limited cross-over. In addition, markets like PPC are quite different from any other media buying channel – and the skill sets they demand are not identical to one’s already in place at traditional media marketing consultancies. SEO and multivariate testing both demand significantly different types of knowledge and expertise than traditional marketing. Indeed, SEO is really a technical discipline – not a marketing discipline at all. And behavioral targeting tends to be a technical problem almost in the domain of IT.

Some, but not all of these barriers will evaporate. Channel understanding will inevitably grow. Multivariate testing may well become a subset of web design. eMail and Banner advertising are already quite akin to traditional direct responsetargeting and media buying. PPC has some inherently distinct elements as does behavioral targeting. And I’d like to believe that in some more rational search engine future SEO may go away entirely.

Add it up, and I’d expect significant consolidation of online marketing optimization into the broader marketing optimization world. With online growing so much, it isn’t clear who will own who, but I’d expect the majority of consultancies to be multi-channel and I’d expect online marketing optimization to be a part of the same internal business function.

What about classical web analytics? Here, the potential merger is with traditional BI tools and corporate measurement/reporting groups. Once again, I think there are legitimate technical and historical reasons for the current silos. BI tools simply could not handle web data. And BI practitioners have tended to be very tool symbiotic and to work on a fundamentally different sort of problem than classical web analytics presents. The limited channel understanding that was once a severe limitation in existing BI practitioners is mostly gone. EVERYBODY sort of understands web sites (not true of SEO or PPC).

On the tool front, web analytics tools have become significantly more like traditional BI tools. They’ve added capabilities that bring them into something not-quite-like-but-nearer parity and also make them more congenial to traditional analysts (though there is still a ways to go here).

But here’s the thing - traditional BI was really never a generic analysis function when it existed as an analysis function at all. Many, many companies were never very good at traditional BI. They’ve deployed it, but it was just another set of under-used reporting systems. And where it did tend to be effective, it was mainly because it was used by subject-matter experts in highly-specific disciplines.

In other words, the traditional BI specialist was either a reporting/tool specialist or a true subject-matter expert (like a direct-response analyst, an insurance analyst or a financial analyst). What’s the equivalent in the online world? I suppose a web analyst. The only real difference is that in the online world the subject-matter expert has to be both a tool and a subject-matter expert.

I expect this to end. As tools become better, there will be a stronger split between tool-focused facilitators and true analysts. And the analysts will be subject matter experts in web sites. Just as they are now in direct mail, insurance or finance. On this view, classical web analytics won’t go away. It will become a true subject matter expertise within a larger framework of BI. As such, I’d expect it to be handled differently depending on the organization.

Go to a direct mail company and you’ll find a specialized, highly important group of experts dedicated to list targeting and optimization. Go to a traditional bricks-and-mortar and you’ll find a person doing the same thing as part of a larger BI department. In another kind of company you’ll find a group of highly expert analysts who do nothing but study actuarial data. And in a pure online play, I’d expect to find the same thing – a team devoted exclusively and importantly to classical web analytics.

So in the end, I expect that the answer for classical web analytics is similar to that for online marketing optimization. There will be significant merger. In some organizations where online is highly important, it will look like web analytics has won. In others, where the internet channel is secondary, it will look like a small offshoot of BI. And in other places it will look like a big but hardly dominant piece of the puzzle.

However this plays out, though, I go back to Paul’s advice. To be of a real value, a web analyst has to bring something special and distinct to the table. That means we have to have real subject matter expertise. As the online world becomes ever more entwined, it behooves us to understand how web data and call-center interact. How TV and online play together. How Business Objects or SAS can be used. But I think it will always be in our interest – as web analysts – to understand that our subject – ultimately - is the web site.

Spring Forward to X Change

Semphonic just announced this year’s X Change Conference for August 18 and 19th. Not surprisingly, the first people to blog about weren’t from Semphonic at all. Marshall Sponder and our partner for this year’s Conference WAD’s Eric Peterson both beat me to it. Since the news isn’t exactly hot-off-the-press, I’m going to expand on it a little.

Semphonic started X Change last year with a specific niche and purpose in mind. Even though conferences like the Omniture Summit and eMetrics are truly excellent, they have grown so large that they must necessarily cater to a certain level and type of audience and are forced into a certain style of presentation.

For people who are serious practitioners, who are running or deeply involved in web measurement departments in significant enterprises, there is only so much value in sitting passively and listening to a presentation. The real learnings for serious students in any discipline come from the back-and-forth of conversation, the interplay of ideas, the cross-questioning that only comes from a truly equal dialog. X Change is designed to provide just that.

Our idea was simple. A small conference that brings together top industry experts and practitioners in a format where they talk as equals. No speeches. No lectures. No audience. No selling. Just conversations. That’s X Change.

Last year it was a great event. Different, exciting and fun. I expect a great many of our 1st year participants back. And I expect to make this year’s X Change even better. My goal for X Change is quite simple. I want it to be the best and most valuable conference that its participants have ever attended. Not the best Web Analytics conference. The best conference.

With that in mind, here’s what’s the same, what’s new and what’s different!

What’s the same?
The format: All Conversation (we call them Huddles), No sales. X Change has no booths, no sponsored lunches, no sales pitches thinly disguised as presentations. You pick the Huddles you’re interested in and join the conversation.

World-Class treatment: Last year’s event was at Copia in Napa Valley and featured classic Napa Valley food and wine. This year’s event is at the 5-Star Ritz Carlton Hotel in San Francisco and will include some really unique food and wine events.

The size: Intimate. Limited. Special. We can’t grow X Change too much given the all conversational format. And what makes the conversation special is having the best people possible. X Change’s format means that the quality of the conference is almost entirely dependent on the quality of the participants. So our focus will continue to be on getting the very best participants possible. We’re limited to around 100 attendees (in addition to Huddle leaders and Experts) and the more those 100 can share, the richer the experience for everyone.

What’s different?
The Speakers: I’m going to try something different with Huddle leaders this year. Last year, our Huddle leaders were almost all industry experts. People like Eric, Terry Cohen, Matt Belkin, Matt Jacobs, Jacques Warren, Marshall Sponder, Aaron Gray, Olivier Silvestre, half the Semphonic team, etc. etc. They were almost all pretty great. But I worry that if I have the same experts year in and year out as Huddle leaders, the event will seem to be the same – and perhaps might get stale. So this year, I’m going to recruit the Huddle leaders entirely from the ranks of industry. I’m hoping that will further open the conversation up and make the peer-to-peer nature of the event even more telling. That doesn’t mean I won’t have experts though. My personal experience last year was that the industry experts added tremendous value in Huddles even (and maybe especially) when they weren’t the leaders. I remember Matt Belkin as a fantastic contributor in two Huddles I sat in – and I didn’t even see the Huddles he actually led. Ditto Craig Danuloff, Paul Bruemmer and Manoj Jasra in a Huddle I led. So I’m going to create an additional class of participant – industry experts. And not only will they be a part of the Huddles, I’m hoping to structure some special events around them.

The Venue: Last year’s venue Copia was excellent in most respects. But it turned out to be short a couple of quality rooms we needed (X Change requires lots of meeting rooms) and, since it wasn’t part of any lodging, created a separation between the Conference and the sort of ad hoc conversations that spilled out and should grow naturally from the Huddles. This year’s venue is the lovely San Francisco Ritz Carlton. By consensus, it’s the most luxurious hotel in San Francisco. It solves both those problems and provides a level of comfort, service and atmosphere that will insure that every part of your X Change experience is wonderful.

The Duration: We extended the Conference to two full days and we are adding in extra time around the Huddles to allow conversations to continue and grow (and work to get done as well).

The Date: I won’t pretend we chose August entirely by preference. I’m scouting 2009 venues already – because I found that there are very few places setup to handle a Conference like X Change that meet my own standards. But the August date isn’t too bad. That can be a lovely time in San Francisco – it’s certainly never too hot (fog is a more of a risk than heat) – and it’s well away from eMetrics and other WA conferences. Hey, bring your family along and enjoy the City and the Ritz!

What’s new?
Web Analytics Demystified is a co-sponsor: One of the biggest changes from last year is that we are co-sponsoring the event with Eric Peterson and Web Analytics Demystified. Eric was our keynote last year and was a tremendous supporter of the event. He gets the no-sales approach. He loved the conversational atmosphere. We really enjoy working with Eric and are partnering with him on broader business. With Eric’s help, we hope to broaden the base of attendees and insure that we have the best and most serious web measurement people in the world joining the conversation.

The Keynote, One-on-Ones, Special intra-Conference Events and Treats, etc.:
This is really just a tease. I’m going to be talking more about the new stuff as we go forward (and as I figure out how it is all going to work). But we learned a fair amount about putting on a Conference from last year and we are committed to making every aspect of X Change (from more coffee and snacks to post-conference written briefs) as close to perfect as we can. I want people in the web analytics community to view X Change as the single most important event of the year. To achieve that, I understand that it can’t be a vendor event. It can’t be about selling. And it can’t be about making money either. Pretty much every dollar I take in on X Change I’ll expect to put back into the event.

What does this all boil down to?

First, we are right now beginning the process of identifying Huddle Leaders and Industry Experts. If you are interested in either or have suggestions of people you think would be great, please let me know!

Second, we are going to pushing hard in the next two months to get early registration of the vast majority of attendees. We really do have a hard limit in terms of attendance. Special preference will be given to clients and previous attendees, and given the rather spectacular growth in Semphonic’s client-base, the addition of Web Analytics Demystified as a co-sponsor (Eric knows everybody), and the large number of potential repeat attendees from last year’s event, we might have to close registration well before the actual event. So if you WANT to attend (and you should), take advantage of the early registration!

X Change was the single most nerve-wracking and rewarding experience for me, personally, in 2007. The conversational format worked as well as I hoped and embodied a few simple but important ideas. Last year when I opened X Change by introducing Eric Peterson as our Keynote Speaker, I spoke to a simple learning from my boyhood and college experience: a conversation shared is more satisfying and more useful than an argument won. I closed X Change by noting that we are all, as analysts, following a difficult path of uncovering and expressing the truth. We don’t deal in the profound truths of philosophy or physics, but in the everyday truths of business. But no matter how everyday, the pursuit of truth turns out be an arduous and difficult business - for the world is a rich and complicated place.

X Change is all about those two basic ideas: the difficulty and excitement of our pursuit and the value of shared conversation among equals to help us all gain real knowledge. Join me at X Change, 2008!

A Perfect Storm

Last week was a sort of "Perfect Storm" for me as I flitted about the country speaking. I do a modest amount of speaking, but once or twice a month is about my norm. So having four different presentations in four days back-to-back was quite an experience.

The fact that every one of these presentations was fundamentally different made it more challenging but also more interesting. I get bored giving the same presentation over and over.

I wanted to quickly mention a couple of those presentations and if people are interested in getting the actual Powerpoint presentations, let me know. You’ll end up on the Semphonic mailing list, but I think they are worth it!

I kicked off the week with a presentation based on my current ongoing series on Form Abandonment. If you’d like a nice Powerpoint summary of the information I’ve been going through on the blog, this would be it.

On Tuesday, I spoke on how to organize the SEM Function. This presentation includes thoughts on the organizational and vendor capabilities required to do good PPC and SEO. This is a topic that’s gotten me in trouble more than once (Agencies seem to hate this) but I think it’s pretty interesting stuff.

Wednesday I was in DC and presented on advanced management reporting. This presentation sums up a lot of our current thinking on reporting. The opening section discusses our overall philosophy of reporting and some of the myths around doing web analytics reporting. This is controversial stuff and I think it’s better heard than read off bullet points. But the presentation also includes stuff on Analytic Reporting, focusing on getting the right numbers and getting them right, and placing reports into a good conceptual framework. To me, this was the most interesting presentation I did and I really enjoyed doing and talking about this one.

Finally, on Thursday I was back in San Jose at SMX (beautifully run, by the way) talking about Web Analytics Tool Evaluation and SEM Analytics. This presentation is really just a distillation of the White Paper I wrote on Tool Evaluation a while back that is on our web site. In truth, I thought the Panel discussion there was more interesting than my presentation - tool evaluation just isn't that interesting unless you happen to be buying a tool. But there’s some new stuff on good SEM reports and capabilities and if you’re looking for an easily digestible version of the Tool Evaluation White Paper or are just interested in some thoughts on web analytics SEM capabilities this one would be nice to have.

If you’re interested, just drop me a line to let me know which one you’d like and I’ll send it out!

Go Ahead and Flush

In my post on the Microsoft acquisition of Yahoo, I mentioned that one of the areas where I thought Google was vulnerable to attack was the Content Network program (AdSense). Like most content network programs, there is a lack of transparency about what you’re buying and a misplaced reliance on content matching technology that often leads to very poor program results.

That’s partly why I was interested in the recent Click Forensics study that showed extremely high rates of click fraud on content networks.

Here’s a link to an article describing the study on MediaPost – and I’ve excerpted the key passage for this discussion here:

“The average click fraud rate of PPC advertisements appearing on search engine content networks, including Google AdSense and the Yahoo Publisher Network, was 28.3% in the fourth quarter. That's up from the 19.2% average click fraud rate for the same quarter in 2006, and 28.1 percent for third-quarter 2007.”

That is an astonishing number. Hardly believable. Probably not even true.

The content networks generally admit to a much, much lower level of click-fraud. And, of course, Click Forensics has almost as much of an incentive to overstate fraud as the networks do to understate it.

I’ve read posts by Google and posts by Click Forensics, and all I can say is that neither make a whole lot of sense to me. I’m just not in a position to judge the reasonable accuracy of these competing claims.

I can say this. I don’t think any significant brand should ever spend advertising money without knowing where that advertising is going to appear. I can’t think of another channel where the sort of large and highly-sensitive brands that we typically work with would even contemplate such a thing.

That’s why I’ve always wished that advertisers bought in content networks using an exchange model – where reputable publishers (sellers) are matched with reputable buyers and the exchange takes a small cut on every transaction. I think it’s better to have transparency around placement and to select appropriate sites than to get whatever minimal boost in relevancy that content-matching technology actually provides.

I also know that our experience analyzing campaigns on content networks had pretty much convinced us some time ago that they were usually a poor investment for clients. We tried valiantly to exclude bad and underperforming sites, but new bad and underperforming sites always managed to find their way right back into our buy. In the end, we pretty much gave up. No amount of good analysis can compensate for a bad system. It turns out to be a lot easier to find the few sites that generally drove what good traffic there was and have our clients buy sponsorships directly from them.

This underscores a critical point - even where a content campaign appears to work, it's often the case that a few good sites are responsible for nearly all the good clicks. Why not just buy inventory on those sites directly?

That doesn’t mean that levels of fraud are anything like what Click Forensics claims. It may just mean that the system itself is over-bought, doesn’t work all that well, and forces you to buy against lots of dumb money overspending for clicks. That isn’t quite fraud but from the buyers perspective, it might as well be.

Most of the very sophisticated buyers I talk to don’t buy on the content networks at all. And yet, the total dollars spent on content networks continues to grow. Which makes me wonder: who is spending all that money?

I don’t really know, but I have two theories.

First, I think a fair amount of content network buying is from unsophisticated small buyers. As a small buyer, you don’t always have a great deal of choice. Sponsorship of a significant site is unaffordable. Search on niche terms often provides access to very limited traffic. Banners are harder to buy and take more creative effort. So content networks may provide the only reasonable avenue for scaling a small, niche online buy.

It’s my guess that another fair amount – and maybe the bulk – comes from program buyers anxious to spend every dollar they can. Let’s face it, even with the “long-tail” you can only buy so many search terms that will produce any clicks. It often surprises companies how small the click numbers are on even seemingly logical terms. But no such problems exist with content networks. Content networks are a vast sponge that can absorb almost any budget – no matter how large.

I can’t tell you that ¼ of all your content clicks are fraud. I don’t really believe it. But I do believe that managing a content network program takes lots of special care if you are going to do it at all. So if you are a significant investor in PPC and don’t manage your campaigns, here are a few pieces of advice.

• Find out what percentage of your buy goes into content networks.
• Find out how that buy compares in terms of performance to your search buy.
• Make sure than any content network buys are measured only against absolutely real conversions (purchases and valid lead forms) – not against traffic or even conversion proxies.
• Use your web analytics tool to find out where your content network ads are appearing and measure their effectiveness by site.
• Make your buyer eliminate continually the sites that you don’t want your brand on.

If you’ve ever gambled at all seriously, you realize that one of the most important facts to know about any game of chance is the percent the house takes off the top. The larger that percentage, the harder it is to make money. In Las Vegas, casinos typically field games where the house takes a cut ranging from less than a percentage point to as much as 5-7%. We all know how well that works out for them.

Because of the total lack of transparency in content network advertising, nobody knows what the house percentage actually is (Google, Yahoo and whatever click-cheats there are constitute the house). For the most part, you don’t actually need to know the house percentage. You just need to make sure you aren’t one of the suckers spending all that money without making sure it actually works for you!

So here’s my best advice if you’re truly tempted to use the content networks in Search Marketing. Don’t do it at all unless you are an expert buyer. And if you aren’t going to take the time to measure the results with all due care, then you should just take that money and spend a weekend in Vegas instead. You’ll have a better chance of getting a return on your investment and you’ll have a hell of lot more fun.