SemAngel

Excel Mobile Image Tag Generator for Omniture

In my last post on Mobile measurement I talked about some of the key challenges that really differentiate mobile from fixed web measurement. Because of those challenges, organizations have to make a decision between using a completely different measurement solution for mobile, using their existing solution with javascript tags, or modifying their existing solution by using image tags. If you’re going this last route, we’ve found that it’s convenient to have an easy way for developers to generate mobile image tags – particularly in the early stages of testing. The Excel Mobile Image Tag (EMIT) generator makes generating efficient and correct image tags a snap.

The EMIT generator is something we built internally for our own use and we’ve given versions of it to various clients. We felt like it was too simple an application to really charge for. But we also found that it was enough of a time saver to be genuinely valuable. So we’ve opted to make it available on our website in a new “Featured Items” section. This does require a very light-weight registration (yes – you’ll get our emails – but that’s not all bad) but otherwise it’s free. In addition to the EMIT generator, there are a couple of our newest white papers including Paul’s Microsite Measurement and my latest with Netezza’s Brad Terrell on Warehousing Analytics. All good stuff.

But today I want to focus on the image tag generator. 

Creating image tags is a little different than creating traditional javascript tags. Images need to be configured from a standard template on the server side (before they are inserted into the HTML stream). To do this, developers typically work with a basic image tag template and then write code to substitute the necessary values.

For Omniture image tagging, this process is made a little bit harder by the need to shorten the tags as much as possible and the subsequent use of substitution variables.
Using the EMIT generator, you start by defining which variables need to be included in the template tag.

You also enter sample values for these variables. These should be of representative or maximum possible length – depending on how conservative you want to be about image request size. The tool does the basic transformations for you so that a correct value gets inserted into the sample image.

The next step is to set the variables you want to define from other variables. In Omniture, there are quite a few cases where you might want to do this. It’s very common to set a prop and an eVar to a single value. Instead of copying the value twice into the image request, it’s vastly more efficient to set one value first (the prop for instance) and then set the next value (the eVar) to the prop. The order of assignment makes a big difference here, and the generator makes sure that the sample images come out right. That’s critical if you’re using these sample images for server-side templates.

Once you’ve done this, the tool automatically generates the image request. This is a fully and correctly formed image that can be used for testing or as a server-side template. The tool also calculates the total length of the image and uses conditional formatting to highlight the length if it may be too long.

The generator has one other nice feature that is surprisingly handy – it can deconstruct and image tag back into its constituent pieces.

The world being what it is, sooner or later you’ll find yourself testing an image tag and not being sure how it was encoded. You can decipher it without too much brain strain, but it’s easier to just pop it back into the tool and let Excel do the nice parsing. That’s also a good way to find otherwise obscure formatting issues that will impact how the image is parsed.

I think you’ll find this is a handy little tool – particularly if you’re just getting started with mobile image tagging. Enjoy!

The End of Summer

This slow, gradually increasing sense of strain, the growing list of things to do, and the fact of an ever-nearer big day on the calendar; they seem eerily familiar, gloomily reminiscent of my boyhood days and the end of summer. It doesn’t help, I’m sure that my daughters are now firmly in the school-age years and have their own attendant end-of summer fears through which I can live, like any good parent, vicariously.

But it isn’t quite the same either. Not at all like then when I hated with a passion the end of summer. Hated because each year, as the end of August neared, I felt pushed by a looming, dreadful wave toward that awful day when I would once again have to take up pen and pencil and spend my days in regimented sloth and indifference.

Thank god it’s not that. It is, rather, one of the highlights of my year – X Change.

Yes it is stressful and hard work too. Work made double by the addition of Think Tank Training to the conference and doubly challenging by the impact of the recession on travel and conference budgets. It’s been harder than last year filling X Change - quite a bit harder.

But with a little less than three weeks to go before the conference kicks off, things are looking pretty darn good. Think Tank is nearly full (not all that hard when you are limited to about 30 people) and I’ve many times blessed the small group discussion format of X Change which limits our size to around one hundred people. Not only will we have our people, the quality will be outstanding; and that’s surely the most important thing of all.

So this thinking about the end of summer and my many bad school experiences makes me want to remind everyone what it is that makes X Change special and why, if you are seriously engaged in or concerned about enterprise web measurement, you should bite-the-bullet and come out.

If you’re like me, you’ve been to a lot of conferences and you’ve probably seen some pretty darn good presentations. But many of the best presentations are ones that have been given repeatedly, honed over time, and usually broadened to appeal to as many people as possible. That means that A) you’ve probably already seen them and B) they are great for beginners but not so dazzling if you’re already deeply experienced.

In fact, when I think about my most important and interesting conference experiences, they were nearly all chance conversations – sometimes with presenters but more often with fellow attendees.

The simple insight that for experienced people, conversation is a more meaningful way to learn than presentation, is what underlies nearly all of X Change. X Change is your chance to talk. Not to be talked to. To discuss your problems and hear about other people solutions. And vice versa. With some of the best, the smartest, and the most interesting people in our industry.

It is a truly peer-to-peer conference and I’ve tried mightily hard to preserve that in every aspect of what the conference does. Why don’t vendors and consultants (including Semphonic) ever lead Huddles? Why don’t we have booths at X Change? Why aren’t there any sponsored speeches? How come you won’t hear a Semphonic sales pitch at breakfast or over lunch? It’s not because I’d have a problem taking the money or because I don’t like to talk about Semphonic. It’s because I think those things would detract from the collegial atmosphere, dampen the conversation, and subtly shift and lessen the experience.

We do have vendors out to X Change – competitors frequently enough. And that’s fine with me too – they’ve been almost uniformly successful in bringing their expertise to the conversation not their sales pitches.

It’s a special experience.

Believe me, it’s hard, these days, not to market a conference as if it were a late-night gadget:  “Sign-up now and you’ll get five key take-aways, two must-dos, and, if you call now, for the low, low price of only 899 we’ll throw in an extra-special key insight you can show your boss!”

But life and learning don’t really work that way. There are no guarantees at X Change. The conversation can be fascinating and full of insight; it can be silly and even a little dull sometimes too. What we believe is that if you bring the best people together, give them an atmosphere where they can relax, and provide just enough structure to keep everyone’s focus, that good things will happen.

So if you’re an enterprise web analytics manager with ideas about how we can all do better, an online marketer deeply serious about measurement or a senior analyst with a hundred great measurement stories to tell, and if you have enough budget to squeeze in a conference or some training, give it a shot. You’re the people we want at X Change; the people who will make all the conversations better. And I think you’ll find that when all is said and done, there is no better and more valuable experience you could find. Who knows? You might even find that extra-special key insight to show your boss…

[(We’re still working on getting the mobile image tagging app (and some of our recent white papers) onto the web site so I’ve delayed that post for a week or so.]

Mobile Measurement: Basic Challenges

At the last eMetrics, Greg Dowling of Nokia and I did a presentation on Mobile Measurement and the work we’ve been doing in that area. Greg is leading a Huddle on the same topic at the upcoming X Change Conference, and X Change is the perfect venue for this type of topic because it’s mobile measurement is so new, so fractured and so important that the best way to learn about what’s happening there is to talk with people on the front-lines of trying to do it. Leading up to that, I’m going to do short series on Mobile Measurement.

First some quick background. A good part of what makes mobile a challenging measurement environment comes from some pretty basic limitations on the infrastructure as it currently exists. What ought to be an environment even more measurable than the fixed web often, in practice, turns out be significantly less open to any form of reasonable measurement.

Among the many challenges you have to worry about with mobile measurement, the problems with javascript tagging are probably the most important. Mobile environments being what they are, not every device supports javascript and cookies. Of course, most heavily used internet phones do. But even if phones support javascript, the weight of javascript tags is a significantly bigger concern than it is on the fixed web. Mobile pages tend to be much lighter than fixed web pages for a reason – and adding weight with a tag is definitely a drawback.

So for both portability and weight, companies like Omniture that use a javascript tag usually recommend an image tag for mobile. There’s nothing very mysterious about an image tag. When you use a javascript tag, all it’s really doing is using some helper functions to append parameters to a existing template image and then making an image request. You assign a bunch of object variables, then call the builder function and it assembles the image request and fires it. If you’re using a tag-based system, you’re already using image requests.

An image tag system takes that concept and instead of loading the javascript and using it to customize the image request, simply inserts a fully-customized image into the content as it is served. This server-side configuration of the image request is a little different that what’s involved in a javascript tag but the end result is the same.

These image requests will work in any browser and any system and, of course, they are extremely light-weight. I’ve had engineers tell me after they’ve looked at server-side image requests that they’d rather do fixed web measurement the same way!

But there are some drawbacks to image tags. The biggest and most important is that an image tag is firing server-side – so you have no opportunities to do client-side customization. That means tracking forms, click-handlers, and campaigns are all problematic. Anything that happens client-side is essentially invisible.

There are some solutions for these problems. Omniture, for instance, will let you target an Omniture redirect with an image href that will record the measurement of a link click and then make the actual call. But this isn’t a system most IT departments will fall in love with.

There’s also the problem of image length. Unless you’re fairly technical, you might not even realize that image requests (which form the basis of all tagged measurement) have a size limitation in every browser. For most versions of IE, for example, the limit is a little over 2000 characters. That’s pretty long, but there are times when you can exceed it and if you do, your image request just fails. That means you don’t get any measurement.

If you load up your tags with lots of variables (especially long text strings), it’s not all that hard to exceed the 2000 character limit – and, since the failure is situational, you’ll probably never even realize it’s happening.

But in mobile, there can be much shorter limits on the length of image requests. Of course, this applies to both javascript generated image requests and server-side image requests. If you move your existing javascript tags over into mobile and you have a fairly rich measurement environment, there’s a pretty good chance some of your image requests will be too long.

There are some techniques that Omniture provides to help with shortening mobile image tags. You can use shortened versions of some variable names, you can pass some items in the cookie, and you can use parameter assignments to pass a value into more than one variable (as is extremely common with Omniture props and eVars).

All this helps, but it introduces some significant complexity as well. In particular, we’ve found that developers often need help getting started with mobile image tagging and that they frequently trip up using features like parameter assignments (the implementation of which is rather poor).

To help with this, we’ve put together (it was Jon and Breen mostly) a clever little Excel tool that generates mobile image tags for Omniture from a basic measurement specification. It shortens the image requests as much as is possible and it calculates the length so that you get fair warning if the requests are getting too long.

It even has a deconstructor component that takes an existing image tag and parses it back into a measurement specification. It’s a surprisingly useful feature for debugging.

We’re going to be releasing it on our website soon (for free) – and in my next post, I’ll show how it works and show some screen shots from it. Though a simple application, we’ve found it quite handy and a great way to make sure your developers get into mobile measurement with the right foot forward!

Warehousing Web Analytics Data

White Papers, Webinars, and a Few More Thoughts on Visitor Velocity

I meant for my previous post – measuring visitor usage velocity – to be the last in that short series on concepts of velocity in web analytics. But in both conversations and  comments I got back from that post, I found it was a natural transition into another topic that I’ve blogged on before and has been very near top-of-mind for me of late. Warehousing web analytic data is, in my opinion, the most significant trend in enterprise web analytics. Why that is, and what the benefits and challenges are in warehousing web analytics data is the subject of a white paper I’ve been working on in conjunction with Netezza. Along with the white paper, I’m scheduled to do a webinar with Netezza’s Brad Terrell (VP and General Manager of their Digital Media Group) on August 25th.

You can get a better picture of Brad and Netezza by checking out his "Bringing Sexy Back to Data Warehousing" post - a title you just can't go wrong with!

Our webinar together is a near-twin to the white paper. We’ll start out with an overview of the current state of the art in web analytics and then in data warehousing.

For my part, I’m going to focus on how basic issues at the data-level in web analytics make it very difficult for current tools to deliver the type of analytic capabilities people like we at Semphonic really want. The volume, high-cardinality, and poorly-understood data models prevalent in web analytics make getting at the most important data – visitor level information -  a consistent challenge.

On the other hand, the idea that warehousing is an either/or proposition with current web analytics solutions is mistaken. Nearly every client we have that gets an event-level data feed from a web analytics vendor (and we have quite a few) is perfectly happy with and fully intends to keep using those solutions. Warehousing data doesn’t just piggy-back on the data collection and cleaning capabilities web analytic solutions provide, it delivers a different and mostly complementary set of capabilities.

After this section, Brad is going to cover Netezza and data warehouse appliances. If you’re more from my side of the woods (and since you’re reading this blog that’s a good bet), then Brad’s overview of warehousing appliances will be an eye-opener. The performance and, in certain key respects, the simplicity of these platforms is amazing.

In the next part of the webinar, we’ll cover why putting this type of capability (warehouse appliance) against this type of problem (web analytics) really makes sense.

I’ll cover the three main types of benefit this combination delivers: superior actionability through integration with customer data and CRM systems, superior analysis – not only with SQL – but more importantly with high-end statistical analysis and discovery tools, and better customer research via techniques like full online panels and media mix modeling.

Along the way, Brad will introduce you to the core concepts behind Netezza’s warehousing appliances and how these translate into real-world web analytics capabilities.

I think most serious enterprise managers and practitioners will find it very worthwhile; both for the overview of warehousing benefits and as an introduction into a key enabling technology that may be poorly understood within our discipline.

Register and check it out!

Also, check out the latest X Change Huddle Leader interview on Manoj’s WebAnalyticsWorld blog. Shubhra is part of Nokia’s enterprise-wide effort to become a leader in analytics and she’s working on some of the most interesting pieces. Her Huddle will focus on building a complete picture of customers with your data – highly relevant to today’s topic isn’t it? Indeed, there will be quite a bit of discussion about data warehousing at X Change and I'm also teaching two Think Tank classes that are focused on data warehousing. So if you haven’t signed up for X Change yet, stop procrastinating, we’re only about a month away now!

Web Analytics Concepts: Visitor Velocity

In my last two “analytic” posts, I’ve shown how concepts of rate-of-change and velocity can be used to make content reporting much more useful and bring a significant added dimension to SEO, source, and viral reporting. In this last post on concepts of velocity, I’m going to cover some ideas around measuring visitor usage velocity.

Not every site is necessarily consumed with understanding velocity. In retail or B2B lead-gen, visitor usage velocity may or may not be a significant item of interest. For any web site where the expected and successful behavior is encapsulated in a single visit, visitor velocity is unlikely to be interesting.

But for digital media sites (where mindshare is king) and operational sites (like customer/partner support sites, SaaS applications, and subscription sites), understanding visitor velocity – both in aggregate and individually – is critical.

I’m not going to cover this topic in huge detail. Semphonic’s co-founder Joel Hadary is leading a team writing a research article/white paper on this topic, so I’m going to let them carry the deep intellectual burden and I’m just going to give a quick introduction to some of the core ideas involved.

When you have a site that you expect (and hope) that visitors will use regularly, then the single most important fact about a visitor is how often (and how much) usage a visitor has. But total usage numbers are rarely enough for a good categorization. In the following example, visitors (Vx) are shown with their total transaction count for a nine month period. It would certainly look like Visitor V2 is the heaviest site user:


 
But look at a trended view of the same data:



We’d hardly be likely to call V2 our heaviest user. V2 hasn’t used the site at all in the last three-four months. In the most current month, V8 is actually heaviest user – but didn’t start using the site (apparently) until the last four-five months.

Careful examination of these numbers will begin to illustrate the challenges in capturing any single measure of visitor velocity. Looking at V2, we might be inclined to adjust our thinking from total usage to average usage. It doesn’t help of course:
 


V2’s average usage is going down (and will probably continue to decline), but it’s still the highest and it will remain above average for quite some time into the future. We can make the average adjust more rapidly by shortening the time period or by weighting it. Let’s say we weight the average by making each period back count for only ¾ of the period before it and then dividing by the sum of possible weights:


 
This yields a picture that is probably closer to what we would want. This is a pretty fast moving average (at least relative to the period selected); you can tune how rapidly the average adjusts by changing the weight.

But while a weighted average is great for capturing current usage, it is much less effective for capturing current direction. The following three visitors all have an identical weighted average:


 
These three visitors would be judged by most of us to be going in completely opposite direction. V10 is decreasing rapidly. V11 is increasing steadily. And V12 is in a sporadic but essentially constant pattern. The weighted average doesn’t reflect that.

There are a multitude of ways to try and capture directionality of visitor usage. One simple method frequently used in momentum trading is to compare the current value to the weighted average. If it’s above the weighted average the visitor is increasing. If it’s below, the visitor is decreasing. This works quite well for V10 and V11 but would fail (in my opinion) for V12.

Another method is to compare a short period average to a long period average. This works well for V11, and might work for V10 and V12 depending on the time periods chosen.

One trick that can make all these calculations more useful is to make sure that the usage window you are looking at is correct for each visitor. In the example above, V10 and V11 had no activity before about five months ago. Using only months since behavioral inception when calculating averages will make ALL the calculations for these visitors better.

Of course, whether a visitor is increasing or decreasing in the most current period is only part of the story. V12 is a great illustration of another concept around velocity – consistency. To understand the importance of changes in the current period, you need to understand how variable a visitor is in behavior.

A visitor like V9 ALWAYS records behavior. A visitor like V12 doesn’t. Once again, there are ways to capture this. You could calculate the standard deviation for a visitor – for V9 it’s around 1.7 and for V12 it’s around 10. That means a small change in V9’s behavior is pretty significant. But it takes a big change in V12 to be significant. Of course, this technique doesn’t really capture the essence of a periodic behavior like V12’s.

Another way to approach the consistency problem is to simply look at how often a behavior occurred in a given time period without actually crediting the amount of behavior. In other words, we might score a 1 for each month/week/day a transaction occurred and score a 0 if no transactions took place.

The same basic techniques (total, average, weighted average, average comparison, and deviation) can also be applied to this scoring method to yield insight into how consistent a visitor is and whether they are getting more or less consistent over time.

In general, we find that it’s impossible to capture all you need to know about a visitor’s velocity with a single measure. Instead, it usually requires at least three measures: weighted usage, usage direction, and consistency. But by putting these three together, you have a set of scores that can make it possible to segment all of your visitors into critical visitor patterns. You can easily figure out which important visitors are declining in usage; which new visitors are likely to be important; and which visitors are most important right now.

Whether you are tracking mindshare for a digital media site or stickiness for an online application, these measures of visitor velocity and consistency are essential.