Overview
Sophisticated organizations are increasingly finding good reasons to move data from their web analytics tools to other data processing and analysis platforms. In this series, I’ll be discussing the “why” and “how” of taking data from your web analytics solution and moving it into other platforms. This fourth installment will cover techniques for finding “magic moments” and remarketing opportunities using web behavioral data.
In the first installment of this series, I covered some of the reasons why companies increasingly want to move data out of their web analytics solution. When they do so, the biggest challenge they face is finding the right data models to let them use the data effectively. Web analytics data is much too large to use conveniently in just any format – so you almost always have to do some significant aggregation on it. This is as true inside the web analytics tool as it is outside. But the type of aggregations you’ll probably want outside the tool are very different that the choices made by vendors to support reporting inside the tool.
In today’s installment, I’m going to cover some data models that can help you identify “magic moments.” What are magic moments? There are some products whose demand curve remains relatively constant throughout a consumers life. Food, gas, phone are the types of product that might fall into this category. But the demand for many products is tied closely to particular life-events and changes. Buying a house, for example, is a huge magic moment. It triggers a whole set of new interests, needs and demands – from cable to furniture to lawn mowers. Getting married is another magic moment. Having kids. Going to college. These big moments are special from the inside view, of course. But they are also tremendously important from the outside view – because they represent periods where the consumer is shifting their demand and trying many new products for the first time. This is a critical period for capturing mind-share and building loyalty.
If you think about it, nearly every product has some magic moments. Even the ones we mentioned earlier. Milk has a “magic moment” for a parent when a baby first stops breast-feeding or using formula. Lock in a parent to a brand, and you may have fifteen years of assured loyalty. Gasoline has a magic moment when a teenager first starts driving on their own. Most of us have loyalties to particular gas brands that are as inscrutable as they are valuable. Some magic-moments are lifestage oriented, some, like the Super-bowl or Christmas for big-screen TVs, are episodic.
Magic moments are important to marketers because they represent a time when the consumer is potentially open to persuasion and receptive to messaging around a product. A dollar spent in marketing during a magic moment might be worth ten or a hundred times as much as a dollar spent before or after. Because of this, the search to identify magic moments in particular consumers has always been a big part of direct response marketing.
In the world of direct response, there are a certain lists that always command a premium price – new movers (new home or apartment – tracked using post-office change of address files) being the classic example.
What are some of the web behavioral equivalents?
It’s obvious that magic moments are specific to each business and industry. But the detection and use of magic moments using web behavioral data is structurally similar in many cases. The core concept is that you are looking for changes in the pattern of usage behavior of a visitor on your website.
Some types of behavior have been well explored and present obvious opportunities. Probably the best known re-marketing opportunity for web behaviors is existing customers who abandon a lead-gen form for another product or service. Starting and abandoning a form for a product or service isn’t as good as starting and finishing the form, but it certainly demonstrates a pretty high level of interest.
But re-marketing opportunities don’t require such explicit signals as form abandonment. Suppose you have a brokerage client who goes to your public site and looks at information about 529s. Do they have to fill out form for you to realize you’ve just seen a magic moment materialize? I don’t think so. Anytime a visitor shows an interest in a new/different product or service you’ve just learned something.
How important that learning is depends on the context. When I check out a 529 plan, that’s highly likely to be a true magic moment. When I buy a “Pony Pals” computer game on Amazon, it may indicate my daughter has become a computer game user (magic moment!). But it might also indicate that my cousin has a seven year old daughter and we happen to be visiting during her birthday (not so magic moment).
Magic moments need not be totally bound up with “new” behaviors. They can also be identified by shifts in existing behavior. I come out of a credit marketing background, where significant shifts in account balance (both up and down) often indicate important changes that present some type of marketing opportunity or retention risk.
This gives us three methods for detecting magic moments. The first is the simplest – key event triggers where the very existence of the event is sufficient to indicate a good marketing opportunity. Form abandonment is a good example of this. Regardless of any previous pattern of behaviors, I probably want to re-market to any customer who abandons a form or cart process.
Because you don’t have to store any previous patterns, capturing this type of “magic moment” indicator is trivial. You need only a single column for the visitor, probably containing a date that the activity last happened or a “null” if it has never occurred. Populating this field with a date lets you age the leads appropriately and track subsequent behavior vs. the initial indication. One decision you will have to make is whether to re-populate the date if the behavior happens again. A decision about the appropriate strategy for re-population will depend on the type of follow-up strategies you employ.
The second type of trigger – visitors doing something new on your website – is more complicated to capture. Here, my last two posts in this series (tracking visitor interests and tracking usage patterns) begin to be significant. To capture a new interest, you need a compact way of understanding what a visitor has looked at before. This involves a rich taxonomy on your site and the ability to store previous usage (and mindshare) by taxonomy.
If you are trying to capture magic moments, you’ll have to make sure that you aren’t just capturing “top interests” on a web site. Magic moments won’t always crack top interests if a visitor is a heavy consumer of existing services/content. If your taxonomy is relatively compact (you don’t have a large number of content or product categories), you can probably identify magic moments by comparing new usage to prior existing usage in each category. In the update process, you’ll look for updates to usage by category where the existing usage is zero. You’ll likely also want to check for a significant jump in a category. To keep yourself from identifying lots of false positives, you should probably only trigger this check when the users’ visit count exceeds some threshold. Otherwise, you’ll constantly be kicking out “magic moment” alerts for new users.
If your taxonomy is large, then you’ll probably want to use a content-based data structure (keyed by taxonomy level and containing a visitor id and a usage count) to look for magic moments. In this case, whenever you are adding a new visitor to a level (again with a visit threshold), you’ve potentially identified a magic moment.
The third type of trigger – shifts in the pattern of behavior – is by far the most challenging to capture. In some cases, you can use the techniques I discussed in my last post to identify such shifts. When usage in an area breaks a margin like a “Bollinger Band,” that may well indicate some form of magic moment. The exact nature of the magic moment may need additional inputs to identify. A trader who has suddenly accelerated his trading volume may be moving toward becoming a day-trader. Or he may be selling his positions and getting ready to stop trading. Both are critical changes in behavior that present marketing risks and opportunities, and both may be identified by a sudden increase in trading even though they represent fundamentally opposite changes.
Another way to capture shifts in the pattern of behavior would be to create a table with a series of "snapshots" of the visitor state. In a table like this, you capture the visitors state at various discrete points in time. A table with historical snapshots is kept separate from the main visitor record - it's a completely distinct data structure. It contains the visitor id, the period (these combine to make the key) and then a snapshot of the usage by interest at a particular time. You can use a snapshot table to look for interest shifts – simply by comparing the most current snapshot to previous patterns.
Using a table of visitor interest snapshots isolates the historical data from the main record (so you don’t blow-out the visitor record) and yet provides a very efficient access path to this information. It’s a data structure that’s uniquely suited to, and primarily intended for, the identification of magic moments based on pattern matching and new behavior identification.
Summary
Finding magic moments is one of the core reasons you might look to move data out of a web analytics solution in the first place. It’s a powerful way to take advantage of the web behavioral stream to really match your marketing to a consumer’s current interest. Doing it well involves nearly all of the techniques I’ve discussed in these blogs and, if it is part of your business plans, it will have a deep and profound influence on how you choose to structure your data.
"Magic moments" -- very nicely put!
I have the luck of working with colleagues at Unica who are much more experienced in direct marketing techniques than I am. One thing that they point out is that the decision WHAT offer/content to target a potential customer with is a separate decision from knowing WHEN to target the person. Your magic moment idea captures the WHEN problem very nicely.
The following is a blatant plug for a Unica product. But maybe you allow for me to post it here because it is so unusual and unexpected for web analytics experts. Namely, Unica's Detect product is a state based analytics product that crunches through event data and detects changes of patterns of individuals. For example, that could be that a person used to visit five times per week but the trend is for the frequency to decline. Or it could be that the person failed to do their bi-weekly visit. The product is used by companies (especially in the financial industry) for detecting magic moments.
More info here:
http://unica.com/products/Event-Based_Marketing.htm
And to make it neutral, here is a competing offering from Teradata:
http://www.teradata.com/t/page/165047/index.html
Thanks for another great post!
Akin
Posted by: Akin Arikan | November 30, 2008 at 02:31 PM