A many headed beast. That’s how I describe the challenge of multi-channel digital measurement in a new white paper we’ve just released (in partnership with Anametrix). In the Greek myth, Hercules solved the problem of the Hydra (which grew two heads when one was lopped off), by burning the stumps right after removing the heads. The moral, as I see it, is that there are some problems that you have to solve fully or they just keeping coming back worse than ever. Multi-channel measurement is like that – get each piece right or you just create a bigger mess.
The white paper delves into three basic problems that afflict digital measurement and reporting right now and keep most organizations from getting things right. First, most enterprises lack methods for joining and understanding customer behavior across silos. Second, enterprises focus too much on top-line metrics without providing the necessary context to understand behavior in the system. Finally, organizations typically spend all their effort on capturing the current state of channels but completely fail to describe the channel itself and its potential levers.
The problem of silos is a direct result of channel proliferation. As customer behavior becomes more and more diffuse, understanding the role and impact of each channel becomes almost impossible except in the context of the complete system. It’s particularly important in the digital realm where channels are often deeply interrelated – much more so than in the traditional mass media world. Of course, that deep interrelationship isn’t how things work on the measurement side. Probably every enterprise marketing executive is already aware of the challenges around last-click attribution; it can (and often will) look as if every single digital channel is improving while your overall marketing effort is somehow declining in effectiveness. But silo problems are far more ubiquitous than traditional campaign attribution. Almost every kind of analysis involves some form of attribution – even if the attribution is to alternative content, specific experiences, or specific types of segmentation. Campaign attribution is just one example of a much larger problem; when channels are deeply interrelated, almost NO accurate analysis can be done on the channel except within the context of the whole system.
On the plus side, almost every digital marketer understands the challenges around attribution and silos – at least from a campaign perspective. Last-click attribution has been thoroughly discredited even as it’s been almost universally retained.
I’m not sure the same can be said of the second big problem the white paper identifies: the focus on top-line metrics. I’ve written many times before about the challenges I see with site-wide KPIs. The mistaken belief that individual KPIs are actionable. The deep problems with trying to collapse multiple visit types and audiences into single metrics. The lack of coherence that results from that collapse when trying to discuss success metrics. Like last-click attribution, site-wide KPIs are still, in my experience, almost universally retained. It’s less clear to me that they have been as thoroughly discredited as they deserve. That’s frustrating because it’s much, much easier to solve the top-line metric problem than it is to solve the silo problem.
Which brings me to the third problem the white paper tackles: the problem of "showing the current state." This is new and takes some explanation.
In the white paper I show an example of what I take to be by far the most common approach to enterprise measurement. Here’s the scenario:
You’re the manager of the long-play videos site section. And hurrah, your company has really gotten its act together. Your performance is tied to specific site goals - and your company has established a clear target for long-play video on the site: a 20% year over year increase.
That's great. There's nothing like clear goals tied to real incentives to sharpen the mind and drive performance.
You ask your measurement team to create a report on long-play videos. Here's what they come back with:
It’s great that you can instantly see whether you’re on plan or not. But you’ll never look at this report unless there’s a shortfall. And even then, you won’t get anything out of it except that there’s a problem. So despite the charts and data and window dressing, this report really just boils down to an alert. And it would probably be better treated as such – and only sent when there’s an actual problem.
Wouldn’t it be a little more useful to know why things are the way they are? Aren’t there learnings when you’re above plan? Wouldn’t it be nice to know what’s driving performance – pro or con? Effective measurement is about more than capturing the “current state” – it’s about capturing the system. And that’s a lesson that most organizations simply haven’t learned yet.
I see these three problems everywhere in enterprise measurement. Solving them isn’t easy, but there are solutions. Not surprisingly, segmentation is a big part of changing your approach. As I’ve written before, good digital segmentation provides a way to capture and model the customer journey effectively. That’s a critical component to knocking down data silos. Embedded segmentation is THE strategy for removing site-wide KPIs and a focus on top-line metrics. Embedding segmentation in your reporting effectively eliminates the fundamental problems inherent in top-line metrics and KPIs. Finally, segmentation is a big (though by no means the whole) part of building reports that describe systems not states.
At a deeper level, it should be fairly obvious that all three of these problems are related. Last-click, siloed reporting, and top-line metrics are all manifestations of the problems inherent in “capturing the state of the system.” In every case, at the enterprise level, we’ve built thermometers, not barometers. It’s not that a thermometer isn’t useful, but if you’re going to sea, it’s much better to have an instrument that can tell you what’s coming than to have one that can tell you what just happened.
Building barometers is largely about methodology, but it takes the right tools too. That’s why I partnered this particular white paper with Anametrix. Since not a lot of people know about them, it’s worth providing a quick background. The technology team at Anametrix spun-out of the old HBX/WebSideStory team. So they have a long pedigree in Web analytics. They build the Anametrix software explicitly to solve the set of problems they saw in Web analytics tools 1.0 and their vision lines up beautifully with what I’ve laid out here.
Start with integration. Like most 2.0 generation systems, Anametrix is built with an open data model that isn’t specifically focused on digital/web data. Not only was it built to handle these data sources, it was built to easily ETL and integrate them. Agility of ETL is one of the biggest technology challenges to taming the many-headed beast, and Anametrix has built their platform with multi-channel integration as the central theme. Integration discussions generally revolve around multiple digital and customer data streams, but easy integration also helps solve some of the reporting and measurement problems associated with “showing the state” of the system. Most “system” reports involve building models that include exogenous factors: season, econometrics, census, etc. Having a system that can (and does) easily provide this type of external data join makes it possible to build much more robust system reports.
Segmentation is, of course, another critical feature. Like any good second-generation system, Anametrix delivers unlimited filtering and segmentation against every variable in the system. That’s a critical enabler if you’re hoping to use integrated multi-channel data effectively to solve ANY of the three problems I’ve highlighted.
Lastly, and this last capability is particularly important, Anametrix provides a very robust ability to build your reports and dashboards from the ground up. I’m not talking dashboards that are constructed by accumulating charts and tables in a workspace. I’m talking about a the ability to fashion highly custom views – if necessary down to the individual field level - and drive unique calculations of factors like lifetime value or seasonal variation. That’s essential if you’re going to build true system reports. The Anametrix tool isn’t about the power of their pre-defined reports. It’s about the power of having a tabula rasa on which to build reports of YOUR system and your business. That’s the right way to approach the problem.
To get the full story, download the white paper here…
Comments