If you didn't get a chance to catch my webinar with Bob Heyman on creating an Analytics Agency of Record, here's the link to the PPT with voice-over. I think it's a terrific session. As I wrote after my last webinar with Scott, it's a lot nicer to have both participants in the same room. Bob's based in Marin so he came over to Semphonic's offices for the webinar. It really does make for a nice, interactive conversation.
Before we get too much in arrears, I wanted to tackle some of the questions we got from the session.
Q: Major issue: Each channel/media type has its own reporting tool. How do you aggregate everything in a single tool/source?
This really gets to the heart of the webinar and our presentation. Bob and I have created the Analytics Agency of Record Program with DEG and Semphonic because we see huge problems with the self-interested and siloed reporting that comes when you let the Agency reports from each channel become the de facto organizational view. There's a real challenge here, however, because each channel and media-type does indeed have its own reporting tool with its own special data trapped inside. You'll need those reports (or access to the underlying system). But you shouldn't just accept them uncritically.
I'm a big believer in telling each Agency what reporting you expect. This is part of your obligation as a client. There's a Robert Frost poem that has the line "Good fences make good neighbors" and I'd say the same about reports: good reports make for good vendor relationships. You shouldn't have to beg your Agency for the information you need to manage the relationship. If you do, you should find another Agency.
Your Web analytics tool (or data warehouse) is another partial answer to this question. Not only does your Web analytics tool provide a comprehensive view of the Web (and potentially of other digital channels), it can be an integration point for all of the relevant campaign data as well. The Web analytics tool is a central hub that extends and deepens the scorecarding you can do across all channel. So when it comes to building your "official" reporting, I always recommend your own internal sourcing combined - if necessary - with external data that meets your specifications.
Q: What are you going to do in the event the agencies are lacking or lagging? Make recommendations, or recommend named providers that will provide the proper results?
This is a great question and one that I wish we'd had time to address during the webinar. Our Analytics Watchdog program really features two feedback loops. On a quarterly basis, we review each channel with the client and the Agency. The goal of these meetings is to review the performance in the past 3 months and to layout a data-driven plan for optimization in the coming quarter. This review cycle focuses on channel improvements not Agency performance. We will make a range of suggestions from specific tactics to possible strategies for channel improvement in these meetings and the reviews are meant to be a collaborative exercise with the client and Agency.
On an annual basis, we provide reviews of the channel and agency performance. These are typically timed to coincide with contractual reviews or with internal client planning cycles. In my view, if we've done our job all year and with the reporting and KPI definition, there won't be much in the way of surprises come end-of-year.
We try to stay away from recommending named providers for the reason that we don't want to create relationships that will bias OUR reporting. We don't want to be wined-and-dined by the Agencies or be paid for referrals. We want to stay part of the solution not become part of the problem. We also don't like to recommend terminations. What we want to provide is an objective view of Agency and Channel performance. We let the client decide whether the most appropriate action is increasing the relationship, decreasing it, or letting it go altogether. In most cases, I think the right answer is fairly obvious.
I saw the movie Up In the Air recently (while flying to Korea for Samsung - how appropriate). It's the story of a man who is paid to fly into companies and fire people so their bosses don't have to. Ouch. That's not us.
Q: What do you mean by in-channel budgets?
This is probably just me being less than clear. Agencies manage budgets in specific channels. Even when you have a single Agency relationship, most clients assign budgets by channel - so they dedicate a specific amount to SEO or PPC or Display. The Agency isn't usually responsible for allocating the mix. This often provides little incentive to optimize that mix - and causes the Agency to fight for additional budget in each and every channel instead of trying to create an optimum allocation. Partly because of this, most organizations fail to reap much benefit from a single Agency approach as opposed to best-of-breed approaches.
Q: Do you have any recommendation/best practices in order to track Offline >> Online?
This is too big a topic to address in a single question to a blog post. Bob's book certainly takes up some of these issues. I'd add that there are a variety of techniques to attack this problem (and it's a huge problem for all sorts of reasons - not the least of which is that it can cause serious mis-optimization) including research programs that include integrated survey research, loyalty programs, Media Mix modeling, aggressive key-joining in a Customer Data Warehouse and much more. One of my favorite personal maxims is that optimizing to the wrong goal is worse than not optimizing at all. Failing to account for the interaction between offline and online can often drive exactly that sort of mis-optimization.
Q: Comment: There is often no feedback given to the agency about the analytics. Final reports on campaigns lie around for months before anyone decides to look.
Isn't that all too often the case! Both Bob and I were worried that our webinar would sound like Agency-bashing. Our central thesis, after-all, is that you can't leave Agencies to measure themselves or let them drive your measurement. That sounds (and is) kind of negative. But the deeper truth is that a good Agency welcomes responsible oversight and engaged management. Nobody likes a micro-manager but nobody likes to be ignored either. Part of creating the type of reporting feedback systems Bob and I suggest is creating the most productive relationship possible with your Agency. I think careful analytics feedback is a key ingredient in that.
Q: Are there any analytics watchdogs right now? If so, can you please name a few?
Semphonic. Here's a brief capsule description of the program and feel free to contact us to learn more. Let's face it, there aren't very many Digital Analytics Consultancies out there - and almost none that aren't affiliated with a broader media agency (which ruins the point) or that do something more than implementation or reporting work (which isn't very helpful). We created an Analytics Watchdog program because we:
A) See the need
B) See that it's one of our most productive relationships vis-a-vis our best clients
C) See that there's no one else out there even close to fitting the bill
In regards to your answer to question number 2: What are you going to do in the event the agencies are lacking or lagging....
How is the client fixing your suggestions? By themselves? Are they hiring outside consulting? Certainly the agency is not fixing...
Agreed that you want to divorce yourselves from termination and partnering with agencies and consultants, still don't understand who fixes. Thanks!!
Posted by: Tim Mettelman | July 07, 2011 at 01:39 PM
Tim,
I have to disagree that when an Agency is lacking or lagging there is no hope for them fixing it. In many, many cases, clients simply don't get the best work an Agency is capable of. Making it clear what expectations are via good measurement (and making sure it's clear when they're not met) can not only help prevent this, it can be a cure too. What happens if it isn't fixed? Then it's time to get a new Agency. That's something that an Analytics Agency can help with, but not something I think they must (or even should) be a primary influencer in. There's a lot that goes into finding the right Agency for a given client including geography, price, size, relationship and culture. If I understand your thinking, an Analytics Agency that identifies that an Interactive Design firm (for example) isn't doing well should also have an Interactive Design firm they'd recommend. I see your point but I don't really agree with it - particularly when it comes to fields like Web Design where there are a vast number of possible choices many of which are truly excellent. I think limiting your client choice to some small set of "approved" partners is unnecessary and when you have partners like that, your tendency is going to be to recommend their work and find fault with others.
So I guess I don't agree with the idea that the process itself can't help create fixes or with the notion that an Analytics partner to be useful must come to the table with a complete set of solution partners.
Posted by: Gary Angel | July 07, 2011 at 02:00 PM