Measurement that Matters

It’s hard not to get caught up in the New Year’s frenzy of evaluation, assessment, and prediction. 2012 was, by a large margin, the most successful year ever at Semphonic and that mirrors the broader industry. Analytics is becoming ubiquitous and the need for measurement and data science has come closer to cliché than to evangelism. That’s all good. It’s great to be on the right-side of history. These are exciting times.

But change in the real-world has a funny way of confounding your expectations and happening both much faster and much more slowly than you believe. It’s a conundrum wrapped up nicely by that old cliché that the more things change, the more they stay the same.

A lot has changed in digital analytics. There's been a dramatic leap into big data platforms and the explosion of Hadoop. We've seen significant advances with Web analytics tools into deeper segmentation, more detail data, and better integration with personalization and testing. True real-time personalization engines and CMS' designed to support web personalization have emerged and carved out real market share. The rapid evolution of mobile measurement within analytics and warehousing tools has provided vastly improved mobile measurement capabilities. The creation of a new generation of Social Media measurement software has shifted the landscape from toys to tools. 

So what’s stayed the same? I think it’s the actual use of measurement to drive the digital channel.

For every client that hires Semphonic to analyze and improve their digital performance, there are three that hire us to audit their tags, build them reports, or help figure out a strategy. This despite the fact that we’re a company that sells hard on analytics. Having the right tags, good reports and a real strategy are all valuable. Even critical. But I think the percentages should be reversed. Tags, reports and strategy are all there to help produce analysis that drives change.

The truth is that for a lot of companies, having measurement and analytics is about having measurement and analytics. Not using them. It’s a lot of hand-waving and empty gestures.

If you’re a digital marketing manager, a good goal for 2013 is to drive measurement use. That simple. I think it’s time to stop focusing on the constituent pieces (technology, infrastructure, reporting, analysis) and start focusing on the application of measurement. For me personally, I want to focus our analytics practice in 2013 on clients who let us really drive their digital channel.

So how do you do that?

Here are half-a-dozen things I think every enterprise should be doing in 2013 to drive measurement USE:

1. Creating an independent team that does nothing but analyze and suggest optimizations for digital marketing spend (regardless of your agency setup). At minimum, this team should develop and use a Media Mix Model, a model of Incremental lift and attribution, and an analysis of program variance.
2. Creating a measurement driven testing plan with Use-Case Analysis and Segmentation. This analysis should identify core use-cases on the site by segment and define a testing plan for each based on key behavioral findings. It should include a buffet of smaller tactical recommendations and at least one larger testing hypothesis for improvement in each use-case. Analysis to Testing is THE cornerstone of creating a virtuous cycle of measurement and change in your digital properties.
3. Creating a basic personalization system for online. Personalization is the most powerful method of using analytics to drive impact. The tools now exist (from CQ5 to Celebrus to SAS to Causata) to  support effective digital personalization. You can’t afford to let this capability languish. For most organizations, it is the ultimate use of digital analytics.
4. Changing from reporting to forecasting. When I review my work in 2012, no theme seems more resonant to me than the idea that vast majority of current measurement is about “showing the current state“. I liken this to a weatherman whose only instrument is a thermometer. That makes for a pathetic weather report. It’s time to move from building thermometers to building barometers. Forecasting is the easiest path to analysis there is. It’s a way to force the organization to understand the way the business works, the key levers of change, and the holes in the measurement system (quite similar, in fact, to what I’ve been talking about in the Strategic Plan posts).
5. Use analysis to nail down at least one key business question. I don’t care if it’s establishing a lifetime value model, a lead assessment model, a lead assignment model, a Web to Call-Center proclivity model, etc. The goal should be to start answering big questions. Pick an important problem and DO WHATEVER IT TAKES to build an analysis that drives optimization.
6. Fundamentally revamp your online customer research. The current state of online survey research and use of customer attitudes is pathetic. Consolidate and standardize all your customer attitudes research in a warehouse, create an enterprise-wide customer attitudes reporting system, and make sure that survey research is focused on researching key attitudes and drivers of choice that can be used to support my 5 previous steps. I don’t know of a single enterprise in the world who is doing this well.

My hope is that in 2014, we can look back at 2013 as a year of great change, not in the things that always seem to evolve, but in the things that never seem to. By concentrating on these types of goals, we can make 2013 the year in which measurement actually mattered.

Comments

[Alex Destino]

Very opinionated and hard hitting. Is this new persona part of your New Year's resolution?
Great post and I plan to incorporate into our plan for this year!

[Gary Angel]

Alex,

I'm worried about "opinionated" and "hard-hitting" - though I suppose it was intended to be at least "decided" and "firm"!

After I wrote this post I happened to read an article on Foreign Policy.com by Charles Emmerson that made the point I wanted to make about technology change but rather more elegantly:

"In the end, technological advances, remarkable in themselves, change things much more than we can ever expect -- the speed of adoption of new technologies is hard to predict, and the second- or third-order impacts of adoption even less so -- but also much less. However new the technology, it is ultimately being grafted onto the rather old technology of the individual human, or the community, or the state. And even the newest of technologies can be manipulated for the oldest of ends. "

As exciting as the changes in technology in our industry have been, the pace of change in application of measurement often seems very frustrating to me (and I suspect not just to me).

In keeping with the spirit of New Year's resolutions, mine is to focus on driving that type of change by significantly increasing the value of measurement - I know that's ALWAYS been your focus Alex - but it isn't nearly as common as I'd hope amongst even our relatively sophisticated client-base!