For media sites, few issues are as important and complex as paywall analysis. Where to put a paywall, how to structure it, how to limit the impact on advertising and SEO not to mention customer experience – are all critical issues. A few posts back, I wrote about a set of techniques (using plug-ins or eVar counters) to measure advertising risk with a registration or paywall barrier. Today, I wanted to cover some of the techniques you can use to understand what happens when a visitor actually encounters a wall.
At the most basic level, it’s essential that you be able to measure when you presented a “barrier” to the user. If the barrier message is a popup or embed in an article template it still needs to be measured explicitly and independently. For most sites, this means setting a “page view” or, at the very least, an “event” when the barrier is encountered.
But setting a page view or event isn’t going to answer some of the most important questions about what happens when a barrier is encountered. Questions like:
1. What percentage of people register when encountering the wall?
2. Do people register after encountering the wall multiple times?
3. Is some content much better at driving through the wall?
4. What’s the subsequent behavior of people who encounter the wall and turn away?
Of these, only the first question is relatively easy to answer – at least in its most straightforward incarnation. If you know how often you presented the barrier and you know how many registrations you got, you have a de facto conversion rate.
One potential stumbling block is that many sites have a common registration confirmation page or process – you might register without ever seeing a paywall barrier. If that’s the case, then you’ll need to measure only the visitors who come from the paywall.
There are a great variety of techniques for getting at this type of number in a system like Omniture. You could use pathing to get at the direct number, you can set an eVar or campaign (it’s basically the same thing) and track to the conversion event to get at an indirect number. You can also use visitor and/or visit based segmentation in the Data Warehouse or Discover to get at visit indirect counts and visitor indirect counts.
It’s not so easy, however, to answer the second question. If you set a page and eVar for the paywall barrier and an event for registration, you’ll be able to track instance, visit and visitor counts for both barrier and complete. Unfortunately, that doesn’t really tell you anything about success rates based on how many times the visitor has encountered a paywall.
To get at a number like that, you have two choices and both hearken back to my discussion of measuring paywall impact on advertising: you can use an eVar counter or you can use a cookie.
If you set an eVar counter every time the visitor encounters the wall, you can track the state of the counter on registration conversion. This gives you exactly the information you need to answer the second question. In fact, you even get fancy and use multiple eVar counters with different attribution periods. For most sites, I’d suggest using a visit-based attribution and a time-based attribution that tracks over a seven day period. With these two counters, you’ll be able to track whether multiple instances of hitting a barrier in a session or multiple instances over a discreet period of time are most effective in driving wall conversions.
If you do this, you get a campaign with the latest number set each time the barrier is encountered. If you set the campaign attribution to last, you’ll be able to track the number of times each visitor encountered the paywall before converting. One really nice thing about this method is that you’ll get all the reporting that comes with setting a campaign – usually the richest set of reports for most Web analytics tools.
This brings me to question #3 – how to find the content that is most effective in driving visitors through the wall.
In Omniture, this is a fairly straightforward process. The easiest method is to set one or more eVars with content descriptions (typically you’ll set the actual article/content id and some type of content category or sub-category) when the paywall is loaded. By comparing instance values on the barrier page to the eVar states on registration, you can measure which specific pieces of content and which types of content category are most effective in driving through the paywall. If you sub-relate the content eVar to the counter eVar or use an eVar that combines the two, you can also measure effectiveness of content by number of paywall encounters.
#4 (what happens after visitors refuse the wall) is one of the more challenging questions to deal with in a Web analytics tool. It’s always hard to separate out before-and-after sequences in tools that deal almost exclusively in aggregates. You can try using pathing, but it’s insanely complex for most sites and always limited to the analysis of the current session.
Out-of-the-box, data warehouse isn’t going to help much either. You can segment visitors who hit the paywall and didn’t register but you have no way to tell what part of their behavior is prior to the paywall and what part is after they refuse the barrier.
One approach that we’ve used for this type of problem is actually quite generic and can help with all sorts of analysis projects; it works by setting a time-stamp and/or a page depth on every single page event. You can do this with any Web analytics system using a cookie (it can also be done with Vista Rules in Omniture).
It may seem redundant to add page depth and time-stamp to variables since they are already captured by the Web analytics tool. It isn’t truly redundant, because, although they are captured, most Web analytics tools give you very limited access to the either variable in reporting or segmentation.
By placing timestamp and/or page depth (e.g. 1,2,3, etc.) into variables, you give yourself the ability to pull much more detailed information (essentially server call level) from the data warehouse. This gives you the capability to pull the raw data and then analyze post-barrier behavior in a tool like SQL-Server or Access.
That analysis is still going to be a fair amount of work, however. There are some other techniques you can use specific to your paywall that can make analysis quite a bit simpler.
If, for example, you’ve used a cookie to count how often the wall is presented, you can use that cookie to create a prop variable that contains the page name whenever somebody views a page AFTER seeing a paywall. Setting up your system this way is cool for several reasons. If you’re using Omniture, you can turn on pathing in this (prop) variable – allowing you track the actual paths post paywall (including pathing in subsequent visits) as well as the aggregate page counts. If the prop is populated on Entry, you know it’s a subsequent session. You can even append the value of the paywall counter to the pagename stored in the prop – allowing you track pathing separately for each paywall encounter. Naturally, you’ll want to reset the cookie value to zero after you get a Registration Thank You event so that you don’t mix logged-in page views in the prop variable, but it’s a good idea to leave the actual Thank You page in the prop and the path so that you always know which paths ended in success.
Put these techniques together, and you'll have a nice picture of what happens when visitors encounter your wall. You'll know how many visitors register and how many don't. You'll know whether or not visitors are more inclined to register after multiple wall encounters, and you'll know what they do when they bail. You'll have a real picture of what impact the wall is having both in driving registration and in changing the user experience.
Not every measurement issue justifies the type of tag tuning that I’ve suggested here, but for a content-based site, a pay or registration wall is a complex and critical site issue; it more than justifies and will amply repay the effort to measure properly.
[I hope everyone enjoyed their Turkey and football! A last reminder, too, that I have a webinar sponsored by Webtrends this Wednesday on Database Marketing and Web analytics based on my whitepaper. If you’re interested, you can register here or just drop me a line for the whitepaper.]