SemAngel

Alex Emberey of Thomas Cook on Social Media Tools and ROI : More from the X Change Berlin

Areas like Social Media Measurement are particularly fertile for the X Change Conference. I've noticed that some topics just work better than others. To be really successful as a Huddle, it's important that a significant number of people in the group have real hands-on experience (not everyone has to - but enough to keep a real conversation flowing). I remember a couple of years back when we had Huddles on Big Data or Mobile Applications, there were groups where nobody had hands-on experience except the leader (has that ever changed!). That ends up being more like a traditional Conference presentation. On the other hand, areas that are already pretty deeply understood sometimes don't spark enough interest - it's hard to get beyond the common wisdom. So the best topics are ones that achieved enough critical mass to have plenty of contributors but not enough maturity to be old-hat. I think Social Media measurement is right in the sweet spot. And reading Alex's take on the key points for enterprise Social media this should be exactly the kind of conversation that drives deeper understanding and real-world business opportunities.

Could you describe your role at Thomas Cook?

I look after UK social media – my job is about finding ways to add value for holidaymakers across the travel cycle, from consideration right through to sharing their holiday experiences.

 

What key challenges do digital analysts face with regards to tracking social media data?

There are 2 epistemological approaches to social media tracking: you can monitor what people are saying about marketing campaigns or measure their actions vs. your KPIs with tagging. Depending on the context you can do one or both (be sure to read Brent Dykes’ fantastic article on the topic here). As a digital analyst you want meaningful business insight and tagging ties social media interactions to on-site activity for a single customer view – so it’s preferable. But social networks know the value of their data and tagging opportunities are limited. So I think the key challenge digital analysts face is getting actionable insight from monitoring data. Keyword or taxonomy based brand monitoring has familiar hurdles: over-abundant and sometimes irrelevant data, unrepresentative samples and flawed sentiment analysis. But, if directly applied to business problems, monitoring tools can help develop marketing messages, find new channels and engage influencers (Nate Elliot wrote my favourite piece on this). Further, it seems like the divide between monitoring and measurement is closing. Developments like integrated community platforms and customer content mining from social media mean web properties and marketing can be optimised based on visitors’ social personas - beyond simply pulling in their social graph. It’s a big opportunity to create relevant and targeted messages that work towards business goals.

 

Your first huddle topic is “Social Media Today: Tools and Metrics”. What led you to pick that topic?

Actually, it’s precisely because tagging opportunities are limited. Speaking to other marketers I think we’re seeing businesses pointing analytics resource at channels where ROI is easier to measure, which means the legwork around capturing and interpreting social data is often outsourced. The result seems to be lots of data, but not really enough insight that speaks directly to business objectives. This huddle is about testing that hypothesis.

 

The social media measurement eco-system is fragmented. Too many tools providing too many metrics – much the same as the early days of web analytics. How should analysts tackle this challenge?

I can’t comment on the early days of web analytics, but there are certainly lots of incredibly bright people selling shiny social solutions. Here’s the best tip I’ve come across: try to remember that there’s far too much data available (Patrick Sawyer makes this point more narrowly, in relation to social network profiles here). Best-in-class social analytics solutions are tightly focused on business objectives. So my advice is that analysts should first ask themselves what their business needs to know to perform better, then pick the right tool for the job – rather than being seduced into investing in tools they either don’t need or aren’t (yet) ready to use properly.

 

Your second huddle topic is “The Social Multiplier: Improving ROI by Measurement and Integration”. What will this huddle cover?

It’s a deep-dive into ways social media analytics can make online marketing across channels more relevant, targeted and successful. We’ll be discussing best and worst practice in relation to both the biggest social networks and ‘socialising’ web sites to drive conversion. 

 

What are you hoping to achieve through your huddles?

 I want participants to get excited about the potential of using the right social data in the right way for their businesses.

 

Of the other Huddles at X Change which are of particular interest to you?

The one I’m particularly excited about is David Williams’ ‘Preparing for Big Data’. I’ve admired Mr Williams’ work from afar for a few years now, and even if I wasn’t a huddle leader I’d still have come to Berlin for the chance to pick his brain about focusing analytics solutions on business problems.

 

In your opinion, why should web analysts and optimisation professionals attend X Change?

Because innovation happens when you look at the same problem from new angles!

 

We're about a month away from X Change Berlin, so there's still plenty of time to register and save your seat in the conversation. The chance to sit down and talk deep analytics with people like Alex from Thomas Cook,  Ross McDonnell from Disney or David Williams from ASOS is priceless. Click here to join us!

Thinking ahead to X Change U.S. in September? Click here to register since the U.S. Conference always sells out early!

Ross McDonnell of Disney on Creating the Foundations for Enterprise Measurement and Measuring Success : More from the Berlin X Change

Ross McDonnell of Disney Online is the next X Change Berlin Huddle Leader on our interview list.  The basic premise of X Change is pretty simple - it's that the people doing work in a field have the most interesting things to say about it. One of the most challenging aspects of conferences is that they inevitably tend to draw quasi-professional speakers whose best work is either far back in the rear-view mirror or is done in front of an audience. Take a look at Ross' Huddle topics and the challenges he describes, and I think you'll see the kind of difference I'm talking about. I'm particularly taken with his description of his 2nd Huddle (Defining Success). His points here really resonate with me: the late engagement of analytics teams to project measurement is probably the most common mistake in enterprise measurement; and I loathe the term HiPPO. It's widespread use is symptomatic of how far wrong the measurement community has gone in thinking about enterprise reporting. In Ross' description, I hear the voice of someone trying to solve real-world problems not someone trying to create clever tweet-bites.

Could you briefly describe your role at Walt Disney Co?

I work within Disney Online, which covers Disney’s TV and franchise websites and the expanding realm of online and mobile gaming & apps.

My role is to co-ordinate the measurement activities in these business channels, across the EMEA  region, ensuring that a consistent and effective measurement framework is in place.

Can you tell us a little about the technology Walt Disney Co is using to measure digital channels? And how is that evolving?

At Disney, we primarily use Adobe’s Digital Marketing Suite of products for our online reporting. We also use an array of other tools for specific purposes, such as for in-game analytics, electronic panel measurement data for competitive intelligence and survey software for Voice of Customer data.

As the web analytics manager for the EMEA region what are your key analytical challenges?

One of the issues that we face, and that many companies face at the moment, is the variety of different measurement tools being used depending on the business unit or resulting from inheritance after acquisitions.

This leads to a siloed view of the data often being used, and reports being generated referencing data from just one source. One of the key initiatives for us later this year will be to begin work to draw these data sources together in a much more structured way.

Your first huddle topic is “Creating Strong Foundations for Analytical Success”. What led you to pick that topic?

As a data savvy analyst, it’s very tempting to rush into creating projects around the next big thing – whether that be social analytics or Big Data. However, the most important data blocks to any analysis are those that are laid down first – the foundations. I found this to be especially the case at Disney, where the general environment is a very creative one and data is not a natural consideration to all.

It’s key therefore that the basic principles, we all go through when defining analytics for a new site, are not forgotten and neglected once in place. Is data quality being maintained? Are the right data points being captured? Are the reports being produced helpful and meeting expectations of management?

This huddle is an opportunity to explore what can be done to keep the analytic foundations stable, allowing scope to build the more pioneering elements of our data stacks on top.

Your second huddle topic is “Defining Success and Securing Stakeholder Buy In”. What will this huddle cover?

I’ve found that one of the basic issues that affects analytics, and indeed analysts, is that they are often asked to measure the success of a project after it has been live for a few weeks; yet may have had no participation or visibility of the project up to that point.

Analytics needs to move up the project cycle and be a core component from the start; when a project is conceived, the measurable goals should be considered from the start. In order for this to happen, discussions with senior stakeholders or senior management are typically required. There are some things that don’t aid this, such as the positioning of the Analytics team (IT vs Marketing) and use of terms such as ‘HiPPO’ – it’s an acronym that enforces communication barriers when the goal should be to remove them.

Having clear goals allows the creation of an analytics implementation plan which can be added to the overall project brief, that can be agreed and signed off by the project sponsor. This will aid, and simplify the case of producing the required clear, timely reporting after the project launch. These reports, together with insight projects can really help to imbed a culture where the use of data becomes second nature.

What are you hoping to achieve through your huddles?

I’m excited about the huddle style of group discussion; allowing me to share my experience with others but also enabling all the participants to learn from each other. It will hopefully be a very collaborative experience!

In addition I’m thoroughly looking forward to being immersed in a micro-community of people with a like minded interest in digital measurement. Finding out all about the common goals and concerns related to measurement activities from some of Europe’s leading companies.

Of the other Huddles at X Change which are of particular interest to you?

Well it’s hard to choose, but one session that stands out for me is “Business Data Integration- Let Your Web Analytics Tool & Other Data Sources Talk to Each Other”. It’s one of our key challenges for the next year, and so will be taking lots of notes at this session!

Another session that sounds very interesting, as it’s not something you would typically come across at a conference, is Ulla Kruhse-Lehtonen’s “Managing an Analytics Team”. I’m sure there would be something for everyone to learn from this – talent in our field is hard to find and sometimes harder to keep. I’d be interested to hear if anyone else manages remote resources, as that’s certainly a different challenge.

Why would you recommend other web analytics practitioners to choose the X Change?

What really attracted me to be part of the X Change conference is the unique collaborative approach of the huddle format. It shows an understanding that the knowledge in the room is going to out-weigh the one or two speakers on a traditional podium; everyone can add their own thoughts, experiences and expertise to the discussion. I’m certainly hoping that frank, honest, use cases are discussed without pressure from vendors or sponsors.

 

We're about a month away from X Change Berlin, so there's still plenty of time to register and save your seat in the conversation. The chance to sit down and talk deep analytics with people like Ross from Disney or David Williams from ASOS is priceless. Click here to join us!

Thinking ahead to X Change U.S. in September? Click here to register since the U.S. Conference always sells out early!

 

David Williams on X Change, Big Data, and CRM

As part of the build-up to X Change, we sometimes ask Huddle Leaders to talk about their topics and experience with the Conference. Last week, Michael Feiner (of our London partners, AEP Convert) and I put together a short set of questions for some of the Leaders in Berlin. David Williams of ASOS was first out the gate with a great set of answers - I'm looking forward to his Huddles already!

Could you briefly describe ASOS and your role?
ASOS is a global online fashion and beauty retailer aimed at fashion forward 20-somethings. We have over 50,000 different products online at any one time and add around 1,500 new lines a week. As Head of Customer Intelligence it’s my job to understand how people engage with us across our different channels and use that data to help the business improve that experience. To us, it’s all about the customer.

Can you tell us a little about the technology ASOS is using to measure digital channels? And how is that evolving?
Our primary Web Analytics deployments utilise Webtrends but we have also thrown in Google Analytics for good measure (no pun intended!). Recently, we’ve started loading some raw Webtrends log file data into Quantivo – and we’re able to do some seriously cool analysis with it. The two complement each other brilliantly. I see that approach continuing to evolve over time – how can we get better business value out of existing systems and data by making them more complimentary? Ultimately that leads to the Big Data pathway- but we’ll come to that later!

What are the key challenges digital analysts face when it comes to measuring online customer behaviour?
Ensuring that the main bottleneck to analysis is the analysts own brain rather than his toolset. With the proliferation of digital channels, tools and hunger for visitor level insight, we need to ensure that the challenge remains in the analysis itself, not in getting at the data to perform it. That’s a lot easier said than done of course!

Your first huddle topic is “Strategic Joins of Web Analytics & CRM”. What led you to pick that topic?
A quote I’ve heard too often: “one hit wonders – customers that order once and never come back” – to which begs the obvious question, how do you know they never come back? What is really meant is customers that never placed a second order. To me this is symptomatic of the fact that Web Analytics and ‘CRM’ systems are often more isolated than they should be and a lot of opportunity is lost by not reinforcing the latter with the former. Even in the absence of, or as we prepare for, warehouse / Big Data solutions, there is a lot that can be done relatively easily and efficiently to, for example, create some really useful segmentations to target upon, or to reinforce understanding of purchase behaviour with browsing behaviour.

Your second huddle topic is “Preparing for Big Data”. What will this huddle cover?
Big Data is all the rage but my concern is that there is a lot of ground work to be done before diving in that is often overlooked. Everything from the organisational change that will be needed to support the significant blurring of the boundaries between analytical disciplines that will result, through to the simple, but perhaps non-trivial questions: what does Big Data really mean to my organisation and what exactly am I trying to achieve? It’s too easy to just say “a single customer view” and I think sufficient time invested upfront in understanding this will significantly improve such projects chances of success. We’re starting this journey now and I look forward to open discussions with others on the subject.

Big Data, customer analytics and multi-channel analytics are all the rave. Is 2012 the year when these buzz terms turn into reality?
It’s certainly the year that it’ll be firmly on every execs radar, if it isn’t already. Turning it into reality requires a certain level of maturity within each organisation in relation to the understanding of existing data and also within the industry in terms of the toolsets available to deliver it. I’m quietly optimistic!

What are you hoping to achieve through your huddles?
A good balance of give and take – of share and learn. I’ve intentionally picked subjects that I’m heavily involved in but would not necessarily consider myself expert – I’m there to catalyse discussion, not to lecture.

Of the other Huddles at X Change which are of particular interest to you?
Your own “Moving from Web to Customer Analytics” – as this touches on my previous comments on organisational change. Plus there are a couple of other Analytics/CRM huddles that would appear complimentary to my own and this subject is a personal favourite at the moment!

You attended X Change 2011. Why should web analysts and optimisation professionals attend X Change 2012?
Because of its clear focus and unique format. It’s a great opportunity to learn from like minded professionals without a single sales booth in site. It makes almost every other conference look like a vendor-fest.

 

Intrigued? Thinking how cool it would be sit down and talk about Digital CRM and Big-Data with David? That's what I'm thinking, and that's what X Change is all about. You can join us by registering now!

X Change Web Analytics Conference - Here and Abroad

We're just a little more than a month away from our inaugural X Change Europe Conference! At the same time, we've long since completed Huddle Leader selection, nailed down topics and opened registration on the X Change here in the U.S. come September. I've been so busy and so swamped with content for the blog that I haven't been able to say much about either. So I thought I'd take a week and really focus on the Conference - especially Berlin. X Change here in the U.S. tends to swim merrily along these days without a lot of pushing - we have the usual great Huddle Leaders, the largest ever selection of topics, and one of the best venues ever for the event. If you have been to X Change before, I don't need to say more. If you haven't, this is the year to go!

But let's talk about Berlin. As I look through the list of Huddle Leaders and topics, I'm struck by how similar it is to the list on the U.S. side. There's lots of focus on Site and Multivariate Testing, Big Data, Tag Management, CRM (maybe more here than in the U.S.) and Social Media. I suspect it would be hard to pick the region if you only knew the topics.

Here's some of the offerings that I think sound particularly interesting...

John Hogan (Virgin Media), Kelly Wortham (Dell) and Craig Sullivan (Belron) are leading Huddles focused on testing. I'm particularly intrigued by John's "Advanced Segmentation and MVT" which mirrors Matt Fryer's (Hotels.com) Huddle at last year's X Change on "Segmented Testing". I've come to believe that effective testing is based on careful segmentation - but far too many organizations are still focused on Site-wide tests of creatives. This is one of those cases where absorbing the tactics of the most advanced organizations - true best practices - would really pay-off. Segmentation underlies every real-world successful testing enterprise that I've seen - and both John's and Matt's Huddles reflect that.

I already mentioned the plethora of CRM topics at the Berlin X Change. There's some fascinating Huddles on the agenda. Particularly intriguing to me are Alex Emberey's (Thomas Cook) "Social CRM as a Business Strategy", David Williams' (ASOS) "Strategic Joins of Web Analytics and CRM", Ole Bahlmann's (SoundCloud) "Bridging the Gap between Web  Analytics and CRM", and Tom Betts' (Financial Times) "Integrating Behavioral Information to Shape CRM".

Alex's Huddle appeals to me because I think the emergent role of CRM in Social Media is probably the single biggest change people should expect in Social Media over the next two years. When I think about Social CRM, I mean much more than communities or community management - I see it as the integration of segmentation and customer identity into social relationships to drive much more tailored and business-focused uses of social channels. It's a role that most traditional (and especially social) agencies are poorly equipped to facilitate. Social agencies know next to nothing about CRM and if you're business strategy is focused on something other than viral marketing, it should have a profound impact on the type of agency relationship you're looking for (as well as almost every other aspect of Social Media from tools to measurement to expected return).

David, Tom and Ole's Huddles (I told you CRM and Web analytics is a big deal in Berlin) , on the other hand, are interesting because they speak more to the data warehousing side of Semphonic's work. I've tackled several projects in the past year looking at ways to integrate Web analytics data with CRM data. It's not a single type of integration. There are strong reasons to move certain types of CRM data into your Web analytics solution. There are even stronger reasons to move some Web analytics data into your CRM solution. In each case, however, the nature of the data to be moved is non-obvious. Particularly with Web analytics data, you can't simply move event-level Web behavioral data into the CRM system. Traditionally, enterprises have chosen to move either single scores (like engagement) or key milestones (such as registrations) into the CRM. Both are reasonable but limited approaches. Scoring, in particular, captures too little color to really drive effective CRM. And milestones either present too limited a view of Customer behavior or provide too many fields to help make sense of the customer journey.

I think the data modeling techniques I've been discussing this year are highly appropriate in this arena - providing a way to represent complex Web behavior in a small enough format to meld with customer CRM records.

Finally, as my last few posts have made clear (and I have more on this coming), I've been spending quite a bit of time on some KPI and reporting design projects trying to push Semphonic's position here to another level of sophistication. So I'm quite interested in Ross McDonnell's (Disney) Huddle on "Getting the Measurement Foundation Right", Thomas Schmidt (Bavarian Stock Exchange) on "Defining the Most Useful Performance Indicators", and Isabelle Mouli-Castillo's (Dell - and the recent Web/Digital Analytics Association Practitioner of the Year) Huddle on "How to Measure Site Efficiency when it's not all about Conversion." These are strongly related topics and all absolutely essential in coming to a deep understanding of how to think about and report on digital success. I was particularly intrigued by a question Thomas raised in his Huddle description - will KPIs stay the same or will they change over time?

The fluid nature of business interest is the bane of reporting - and there is probably no single issue in enterprise reporting bigger than the rapid aging of interest in reports. The half-life of true interest in most enterprise report sets is numbered in weeks not months. Annotation is no more than a very partial solution to this problem. Resourceful annotation can keep interest in reports from waning as rapidly as would otherwise be the case, but if the annotating analyst is using the same data over and over, what are the chances THEY can stay interesting? On the other hand, the way we've been encouraged to think about KPIs makes it hard to see how they could/should change very often.

I think a credible approach to enterprise reporting has to tackle this problem head on - not simply assume (as is almost always done) that if the KPIs are right and the formatting is good then the report will retain its value indefinitely.

As always with X Change, it's an embarrassment of riches. I've only touched on the Huddle topics directly related to my most immediate interests and concerns (and I've skipped the Big Data one's I mentioned last week). There is so much that looks fascinating. And, of course, X Change's all small-group, no presentations format makes each and every participant a part of the conversation. If you're based in Europe - I hope you'll join us! Click here to register...

Social Media Measurement : Providing a Digital Marketing Framework

In my last post on Social Media measurement and reporting, I laid out five goals for a management report set:

• Organize data in a fashion that makes both consumption and navigation easy and efficient
• Create a framework within which the data is understood
• Deepen the report consumer's understanding of how the system being reported on works and is working
• Highlight areas where the system is changing or performing outside expectations
• Drive understanding of the fundamental levers that cause change or can influence performance

In that same post, I showed examples of our standard enterprise social media reporting. It's a product that's primarily designed to meet the first requirement (organize the data well) with some small attempts to satisfy portions of the other goals - particularly highlighting areas of change.

In saying this, I don't mean to shorts-sell our own (Semphonic's) product. Creating a highly automated and scalable Social Media reporting system that integrates multiple streams of data and presents the information in a highly-organized and easily consumable fashion is a pretty big win. It may be well short of reporting nirvana, but it isn't chopped liver either (assuming you aren't one of those odd lovers of chopped liver)...

Nevertheless, we want to extend this type of report to meet some of the other goals for a report set.

One way we've tried to do this is to put a higher level face on top of the social media detail that shows how the current marketing effort is translating into actual marketing impact. This isn't quite the same thing as ROI, but it's a start in that direction:

By showing Marketing Push, Direct Impact and Market Shifts in a more tightly bound format, we think this report provides a good framework for thinking about the entire digital marketing effort.

What I like about this view is that it provides a more holistic view of a Branding campaign effort. It shows expenditure by channel and how that translates into share of reach. But it also shows how Reach and Direct Impact relate (and don't). Further, it breaks out a number of milestone activities (like acquiring Fans) that are channel specific but influenced by the entire marketing program. Finally, it includes metrics like Share of Search that are designed to help measure the "branding" impact of marketing spend. For some clients, we've used Brand Tracking Survey data in this same role. 

Typically, the sub-tabs in a report set like this are similar to what I showed previously - they break-out and organize more tactical views of the data within each system.

While this type of report creates a framework for understanding a marketing program, it doesn't make explicit the connections and levers inherent in the system. That's still the domain of the report consumer.

Here's an example of a report that is designed to be less of a reporting framework and more of an interactive tool:

By embedding a Media Mix model directly into the report, the impact of specific channels on previous performance becomes explicit and the report provides a directly actionable component wherein the user can change the mix and predict the outcomes. This is as much a tool as it is a report.

This is by far the most ambitious and difficult type of report to create. Nor is it always possible. Many social campaigns for our large Brand clients simply won't have enough impact on the overall marketing to model successfully. When that's the case, it's far better to provide a report more like our Push-Impact-Shift report that blends detailed reporting across systems with enough of a framework to understand how systems actually relate in scale, stage, and impact within the overall marketing program. 

 

X Change Berlin - Exploring Big data and Digital Analytics

The inaugural X Change Europe is just a little less than two months away, and we've just announced the Keynote. We're going to try something a little different that I think is very much in the spirit of the Conference. Four top practitioners - all with deep experience in digital big data analytics projects - are going to join me in a conversation. We'll have David McBride of Comcast, Tom Betts of the Financial Times, Ulla Kruhse-Lehtonen of Nokia, and Peter Pletsch of Meinestadt.de. We're going to focus on a couple big questions around digital and big data that I think will be of deep interest to our audience of enterprise Web and digital analytics managers.

I'm not planning on running this like a panel at all. Instead, I want to create a kind of "mini-huddle" - a small group of experts exploring a single topic and I hope to drive to some pretty clear and useful takeaway's for attendees.

In particular, I want to explore where on the maturity curve an organization should be before jumping into big data analytics and warehousing. There's a prevailing assumption that analytics warehousing is the appropriate domain of highly-sophisticated organizations with a mature and successful Web analytics program that's bumping up against the limits of a tool. Frankly, that's largely the view I've held.

But I'm wondering if it's true. We have several clients who - having struggled to get much value from Web analytics tools - are hoping to "leapfrog" to the next generation. I think that's interesting. I think it's especially interesting in the context of EU-based enterprises who may feel like they haven't achieved the same level of maturity in Web analytics as some of their counterparts in the U.S.

After all, the two efforts are pretty fundamentally different. Why should success in traditional SaaS Web analytics be a prerequisite for success in analytics warehousing? And how much sense does it make to invest heavily in an analytics direction you think is interim at best?

If analytics warehousing is the direction of your future, should you continue to invest heavily in a traditional Web analytics effort to try and raise your maturity before you make that jump? What if you've repeatedly failed with those tools? Does that mean you can't or won't succeed in analytics warehousing? Does it matter (I think it does) if you've got significant expertise in traditional BI, statistical analysis, and warehousing? On the other hand, what kind of failures in traditional Web analytics are good evidence that you aren't ready for analytics warehousing?

Every member of our Keynote has led a successful analytics warehousing effort. I'll be curious to find out how mature their traditional Web analytics effort is/was and how they see the relationship between the two.

What I'm looking to get from the conversation is a blueprint for thinking about how organizations transition between traditional Web analytics and big data analytics warehousing and whether or not there is a "leapfrog" strategy that can work.

If you're in Europe, I hope you'll join us at X Change Berlin to enjoy what I think will be an extraordinary and illuminating discussion. If you're not able to make it, I hope to bring back plenty of new insights to share!

 

Social Media Measurement : Integrated Reporting

Late last year I started a series of posts on our approach to Social Media Measurement. At the time, I had mapped out a fairly extensive roadmap through the foundational stuff (sampling and profiling), on into reporting, and finally into examples of some of our work and techniques in Social CRM and actual analysis. With all the work we've  been doing around big-data and analytics warehousing and the inevitable day-to-day analytics, I've been finding it hard to keep up. I thought, though, that with last week's post on retail metrics (I really like that post, by the way, so check it out if you didn't get a chance last week), it might be a good time to quickly switch back to Social Media and knock out a couple of posts related to Social Media reporting and metrics.

I'm going to start with our most basic reporting set for social media and work my way up into increasing levels of sophistication.

People probably think that within a boutique shop like Semphonic, there must be a pretty high level of agreement about how to do digital analytics. And for the most part, there is. But not about reporting. The truth is, we're always arguing about reporting and reporting approaches. Coming out of a recent staff meeting where the whole topic was our approach to reporting, one of our analysts remarked that it was "the geekiest conversation [she'd] ever heard." 

I have to admit, I don't like most of the reports I see - whether they are ours or somebody else's. I guess it's a case of following Thomas Edison - we haven't failed, we've just found lots of approaches that don't work. Every 12 or 18 months, I jump back into the fray and try to re-engineer Semphonic's approach to reporting. And frankly, I'm at it again. I have a Whitepaper I'm working on (with Anametrix) on the topic and I have two client engagements where I'm exploring yet another fundamental re-vamp to our approach. Who knows, perhaps this time we'll find that elusive grail.

I believe a really great report set should accomplish these (obviously simple) tasks:

• Organize data in a fashion that makes both consumption and navigation easy and efficient
• Create a framework within which the data is correctly understood
• Deepen the report consumer's understanding of how the system being reported on works and is working
• Highlight areas where the system is changing or performing outside expectations
• Drive understanding of the fundamental levers that cause change or can influence performance
• Point toward area's where specific analysis is necessary to explain what's happening or identify specific recommendations for action

Oh - and of course - it should look pretty great too.

But for today, I want to show something a little less profound than the reporting grail. I'm going to show our most basic level of social reporting - a relatively simple, turnkey set of reports that is designed to meet,for social media reporting, the first and some of the second goals listed above.

One of the biggest issues with social reporting is the immediate requirement to pull multiple data sources. Unlike Web analytics, where tools like SiteCatalyst and GA remain the dominant sources of data, social media data is inevitably multi-sourced. Nearly every social media reporting project we've handled involves, at minimum, integration of listening data, and owned Facebook, Twitter and YouTube data.

Nobody wants to spend time manually copying information from one source or another or going between these systems to look at their internal reporting systems. Each of these systems reports different kinds of data too. That makes it hard to create a common context and layout for each channel.

So it's  fair to say that the main goal of our basic report set is to integrate and organize this data in way that makes it easy to access and absorb.

Here's a set of images that show screen shots from a pretty standard (cleansed) example. We can turn these out with very little effort and scale them for almost any sized program - from Global Enterprise to SMB. They generally take 2-3 weeks to setup (this doesn't count the harder work creating the appropriate Social Media Profiles) and they provide the fundamentals of a pretty good Social Media Reporting System.

They are fully-automated (API-Based), integrate all of the key "owned" data with broader listening data, and provide a consistent and well organized view of the data across channels from LinkedIn to Facebook.

Here's the first screen. Note how we use folder-based hyper-text naviation within Excel to create a very easy structure around a pretty comprehensive report set. Of course, the listening parts of these reports use our Consumer-based profiles - so the sample is appropriate to the problem. This report directly integrates "owned" data from Twitter, Facebook and YouTube with listening based data:

Here's a sample from the Facebook section. The underlying structure is repeated nearly identically across each channel so that the data feels highly organized even though the sources are distinct. Note the little touches like the interactive charting and the customized and channel-specific metrics. These take full advantage of Excel's considerable capabilities and our own drive to deepen client thinking about the data:

Here's the Engagement Report (also from the Facebook area). This shows a significantly deeper view of the specific channel. 

Here's a similar view on YouTube that illustrates how consistent the organization of data becomes within the report set:

Lastly, I picked a sample that shows the listening data along with more detailed classifications (in this case by topic of conversation):

You can see by the navigation that I've shown only a small subset of the reports. But the information is organized cleanly enough and the navigation is crisp enough to make the data both easily accessible and consistently understandable.

As I said, this isn't meant to be the holy-grail of report sets. It 's designed to provide any size organization easy and well-organized access to multiple channels of social media data. I think the combination of 100% automation, scalability, Excel-based delivery, hypertext-based folder navigation, consistent organization, and crisp presentation accomplish that goal. When paired with superior work in creating the underlying samples, this style of reporting delivers significant value to the organization and considerably exceeds the general level of Social Media reporting I see from social agencies.

In my next post on this topic, I'll show we take this type of report to the next level - providing a deeper framework for  how Social Media fits into a broader marketing program within the actual report.

 

Retail Metrics, eCommerce Data Models and Personalization

I've been writing (and working) quite a lot about data warehousing, ETL and data models this year. And between webinars on Celebrus, Infobright, and Semphonic ETL, plus Whitepapers on warehouse data collection infrastructure and Semphonic's ETL and digital data warehousing models, it may seem like all the work I've been doing is around data warehousing. That isn't true for me and it certainly isn't true for Semphonic. Increasingly, however, much of the most interesting work does revolve around data warehousing or intersects with customer analytics and personalization in increasingly rich ways.

I wanted to discuss a few examples of this type of intersection from some recent work on eCommerce implementations and reporting.

Traditional eCommerce Web analytics implementations have focused on capturing the basic product funnel: from view to detail to cart to checkout. By cross-tabulating this information, you can easily see which products are moving and which aren't. You can also tell which campaigns drive to which products. It's certainly useful information and well used can help a product manager in a wide range of tasks.

However, if all you're capturing in your Web analytics solution is product/category information relative to the funnel, you're missing out on many of the most interesting and most actionable (from a targeting and personalization perspective) variables. You may not realize this, because the analytic opportunities in a Web analytics tool can be quite limiting. But if you are moving this data to a warehouse, you can do far more. And the data you're moving to the warehouse is - in the majority of cases - the data you're capturing in the Web analytics tool.

There are three types of data capture that I think are a critical part of a good retail analytics implementation: sequenced customer journey data, merchandising data, and filtering data. In each case, capturing this data not only leads to superior analytics, but to segmentation, targeting, and personalization opportunities. All of these can (and should - at least to some extent) be captured in a Web analytics implementation. But in many cases, analysis or customer modeling from this information will take a warehouse based data model.

My first example of this uses the Two-Tiered segmentation concepts of visit intent to answer a question of considerable importance to any product or marketing manager: how well did the site  marketing facilitate the customer purchase decision? If you're going to evaluate the effectiveness of your site merchandising, you have to be able to answer this question and all of the surrounding questions (such as: did they buy the product they originally considered, did they buy anything, did they switch categories, they buy up or down, and did they add additional products).

In traditional Web analytics, these questions are by no means easy to answer. You can easily generate a product report that shows which products are purchased and which products are purchased together. But if you think about it, you'll see that this doesn't really answer any of the questions. You have no way of assessing whether or not customer's intended to buy those products, which order they shopped for them, what products they evaluated but didn't buy, and whether or not you increased or decreased their order size for any given type of product consideration. Ouch.

To answer these types of questions, you need to establish a visit intent by sequencing a customer's browsing/buying behavior.  Structurally, your goal is to build a kind of customer journey relative to a specific shopping experience.

The milestones to capture include, at minimum, the type of entry, summary of any pre-shopping navigation, any browsing behavior within shopping (more on this later), any product detail behavior (ditto), and any cart or checkout behavior. Here's an example of a couple of such streams:

External Search Entry on Product -> Detail View -> Image View ->Cart ->Checkout

General Site Entry -> Category Navigation -> Product View ->Category Navigation -> Product 2 View

Note that good journey tracking generally involves several different types of variables. This can make "pathing" reports useless (and pathing reports are very challenging to work with anyway) since you often don't have all the things you need in a single variable.

If you build this type of journey, however, you can answer some of the questions that really matter to a site merchandiser. You can trivially figure out how often consideration of one product leads to purchase of a different product in the category and how often consideration in one category leads to consideration of another category.

If you pull in meta-data about the products (such as price and margin), you can also answer the next level of question: how often did you shift a visitor to a more expensive or higher margin product and how often did a visitor shift to a lower margin product?

And here's where I want to stop and make my real point - good analysis and personalization opportunities are intertwined. At the customer level, this data allows you to create some powerful shopping oriented visitor segmentations:

• Is this visitor a focused shopper or are they open to alternatives?
• When they shop, are they likely to shift up or down?
• When they add products to the cart, are they adding small amounts or significant upsells?

If you're building rules to target upsells or cross-sells to customers, these are the types of segmentation your data model MUST support.

My second example focuses on merchandising action data. We see many retail sites that collect virtually no data about what merchandising cues are present on the Product Detail and Product List Pages. Is the Product listed as out-of-stock? Not captured. Is it listed as in-store? Not captured. Is it listed with a discount? Not Captured. Is the price hidden? Not Captured. Are the reviews good? Not captured. Is the margin high? Not captured. Is there a video? Not captured.  In situations where inventory pricing is used, what's the price and how does it compare to rack rate? Not Captured.

How are you supposed to do merchandising analysis when every single important merchandising action is removed from the data?

By integrating this type of data into the customer journey, you dramatically expand the ability of the analyst to understand what drove customer decision-making. This not only allows you answer a host of basic analytics questions:

• How much do Out-of-Stock items cost the Web channel?
• How much do negative reviews impact sales?
• Do negative reviews shift product purchases or lost product purchases?
• Do negative reviews shift product purchases to more or less expensive products?
• Do inventory prices track to market (is our conversion rate fairly steady across rate shifts)?

It also allows you do drive interesting segmentation and personalization:

• Is a customer review sensitive?
• Will a customer shift to the store when an item is out-of-stock?
• Will a customer shift products when the margin or inventory price is high?
• Is a customer sensitive to discount language and drives?

This is powerful stuff when it comes to personalization and remarketing. It's also part of the reason why, when you're constructing a data model for retail, you don't want to rely too much on Web analytics constructs. Page-based Web analytics constructs just aren't a very good basis for tracking merchandising.

Which brings me to final example. More and more eCommerce behavior is being pushed up-stream into product lists and multi-product browsing experiences. For many of our clients, it's becoming much less common to dive down into the Product Detail page. With popups, better product listings, and more early-stage research, the product list page is often the place where all the merchandising work is going on.

Of course, all of my earlier comments apply here in spades. If you aren't capturing the state of the product on a detail page, what are the chances you're capturing it on a list page? Pretty much none. In fact, most web analytics implementations aren't even setup to capture the Products shown on the list page (which can, by the way, be really challenging) much less any meta-data about their merchandising state.

Capturing this type of information via tagging is so challenging that it may not be a good solution. With a warehouse, you may be better off capturing a "merchandising state" file on a daily basis or creating a product merchandising state history file that is constantly updated.  A warehouse based approach gives you this opportunity.

On the other hand, there are some interface actions that really should be captured in the Web analytics tool. Particularly important are the filtering and sorting options applied by a user. Faceting is a critical aspect of the customer's buying journey and should be captured as such. But what about sorting options? They are often deeply revealing.

Let me give you an example. If I'm looking for a nice night out with my wife, I'll sort the restaurant listing on Opentable by Star Rating. If I'm looking for a family dinner, I won't. It's a simple cue - and it's repeated over and over on countless retail and eCommerce sites. Could Opentable use this (plus my # of people in party) to highlight restaurants as romantic or family friendly? You bet. Do they? Nope.

List sorting choices on a well-designed site are, almost invariably, a powerful classifier of shopping intent and customer behavior. But nobody in Web analytics seems to want them.

If you capture the faceting and sorting information, you can answer questions like these:

• Which types of sort orders drive conversions?
• If a filter reduces a set to zero items, what's the best behavior?
• Which sort orders are optimal by category?

That's interesting stuff, but this is a case where the personalization opportunities are considerably richer than the analytic opportunities. By capturing this information at the Customer level, you can build personalization around:

• What factor is most influential to the user?
• What type of purchasing decision is involved right now?
• Is a customer consistent in their preferences or do they vary frequently (can we anticipate what they want)?

Here's the bottom line:

If you're building a measurement infrastructure for ecommerce, it's not enough to capture product level information through the funnel. This misses nearly everything a product marketer can actually do to merchandise more effectively. By capturing sequenced customer journey information, product merchandising information, and sorting and filtering information, you create a vastly better merchandising analysis data stream. What's more, if you're moving this data to the warehouse, you have a slew of deeply interesting personalization and segmentation variables. And that, I think, is the deepest point. The limitations of traditional Web analytics have made it seem unnecessary to capture this type of information. If you live within those confines when you build a warehouse (or stick with the page-based shopping paradigm), you'll be creating an impoverished model for both analytics and targeting.

 

View from the (Adobe) Summit

[One last plug for my Webinar this week on our ETL and Data Model for Digital Data in the Warehouse - I highly, highly recommend this content if you are thinking about or pursuing a digital data warehouse!]

Between the Omniture Customer Advisory Board (CAB) and the Summit, there was a good deal to chew on from this past week. Despite the acquisition by Adobe, I (and I don't think I'm alone here) have continued to think of the event as the Omniture Summit. But truly, after this year, I may finally be over that. It isn't just that Adobe combined multiple product-line conferences (and with their recent acquisitions there's a considerable number of these) into the Summit. That makes a difference, but what's far more important is the degree to which the significant content discussed transcended the traditional analytics suite. This was as true in the re-organized CAB as it was in the broader Conference.

In particular, I found the product demonstrations and discussions of the latest CQ5 release to be the single most interesting and compelling part of the CAB and Summit discussions. That's not because Semphonic does consulting around Web Content Management (WCM) - we don't. But the integration between a WCM system, testing, tagging, and analytics has become deep enough to be both interesting and, potentially, differentiating.

For several years now I've argued that the idea of having testing systems separate from the Content Management System (CMS) makes no sense. It's an historical artifact - a case where start-ups were able to deliver a capability that established CMS vendors should - but didn't - provide. That's changing, but slowly.

You had to have Omniture buy Offermatica, and then Adobe buy Omniture and Day. And you still need Test & Target along with CQ5. But at least the integration is tight enough that you have the appearance of a fairly seamless product. Frankly, if I'm a CMS-vendor, I'd be looking to make these capabilities native in the CMS (like they should be), and looking at the broader CMS market, I think it's moving (slowly) in that direction. The days of independent testing tools are (or should be) numbered. And thank goodness. You shouldn't have to pay to run tests on your own site!

I just don't understand the concept behind paying for a system to serve Web content and then paying for another system to over-write that content. It's clumsy, tedious, inefficient, and expensive. A quadfecta of hurt.

There are deeper reasons why siloed testing tools are an endangered species. As I've been arguing this year (in places like the recent Whitepaper on the Future of Digital Measurement and Personalization), Web content experiences must become more personal. That happens only when a "Customer Profile" can be used to drive content experiences. But who owns that profile and where does it live? With the Testing tool? I don't think so! Does it live in the warehouse? With the CMS? With Omniture/Adobe across the suite? Probably the single most interesting discussion at my CAB focused on just this issue and while I can't share the discussion, I can at least say that this decision is going to be central to your analytics ecosystem and to Adobe's future.  

The integration of analytics is a different story. Analytics isn't a natural function of a CMS the way testing is. But some level of integration is clearly advantageous - allowing content creators to quickly review and understand content performance. I'm not sold that the loop is really all that tight or ever will be, but there's no doubt that at least in some organizations having tightly integrated high-level analytics embedded in the CMS view is desirable.

On the other hand, tagging is most definitely a CMS function. And again, I think the level of integration of SiteCatalyst tagging with CQ5 is starting to be pretty interesting. In some respects, it's significantly more interesting that Adobe's independent tag management efforts; efforts that still trail the broader TMS industry.

As with testing, tag management is clearly a function that SHOULD be part of a Web Content Management System but just hasn't been. Once again, you shouldn't have to pay extra for dropping content onto a page. That's what your Web CMS is for, right?

Longer term, all tag management capabilities really should be embedded in the CMS. But Adobe can't afford to assume that every SiteCatalyst shop will be a CQ shop or that every CMS will support analytics and tag management. So there's legitimate reason to have a separate tag management product. Still, I'd like to see it positioned seamlessly with CQ5.

But will CQ provide identical support for other analytics tags (such as GA)?

Not so likely. That highlights both the advantage and disadvantage of integration in a single suite. As the current crop of Tag Management Systems have proven, if you can support SiteCatalyst tags it's quite easy to support a broad range of other vendor systems. There's no reason why a CMS shouldn't be able to support ALL of the functionality of the current generation of TMS systems (buy or build is their decision - there are some pretty good acquistion candidates out there).

Product suite integration gives Adobe a reason and incentive to drive these capabilities forward, and that's a good thing for the whole market. Right now, I think Adobe is the vendor leading the push to integrate both Testing and Tagging. That's fantastic. But it also gives them a reason to be vendor-specific and to keep products as separate revenue (Test & Target / TMS / CQ5) that really should be unified.

This isn't a problem unique to Adobe. It runs across all types of enterprise software suites where the benefits of integration need to be weighed against the inevitable cost, limitation, and "stickiness" trade-offs. There isn't a "right" answer for every enterprise.

For now, I'm happy that the level of integration between Web Content Management Systems and our world is becoming serious and seriously interesting. That's a trend that's all to the good, and it's a trend that promises to fundamentally re-shape the way we think about the analytics eco-system.

 

On a personal note, I have to say how much I enjoyed my Customer Advisory Board sessions. Brent Watson and Chris Wareham do a terrific job keeping the sessions interesting, focused, and useful. It's always my favorite part of Summit and this year was no exception. Adobe re-grouped the CAB  - going from single products to logical groupings of products. The reorg was both necessary and sound, but it seemed to impact some CAB groups more than others. The Analytics & Reporting Group is now huge (probably too large). I suspect Adobe will have to do some trimming.It's near impossible to have a strategic discussion with 50 people - it's hard enough with 10! CAB is always a reminder to me how challenging it is to get people (maybe especially measurement folks) to think strategically instead of focusing on tool nits.

My group (Suite Enablement & Development) now includes Integration, API, Tag Management and Platform issues around Big-Data (there was a great Big-Data discussion in our CAB). Perfect for me - and still not too large. In fact, we could probably use a few more Customer members. If you have an interest and a deep engagement with that toolset you should ping out to Brent! I often wish I could sit in on Analytics & Reporting too (our Greg Dowling sits on that CAB), but I'm not jealous, just greedy!

 

Using Online Survey Research for More than Site Satisfaction

Before I dive into today's topics, there are a few items worth mentioning:

• If you're going to be out at Omniture Summit and would like to meet, drop me a line!
• X Change 2012 Registration is now OPEN! Click here to register and remember, it's first come, first serve when it comes to Huddle Selection
• X Change Europe 2012 Registration is also OPEN!
• I'm doing a free Webinar on "Tracking the Full Customer Journey: Data Model for the Digital World" - it's a distillation of my recent whitepaper. Click here to register.

With all that to look forward to, I'm going to look back just a little bit today and re-visit one of my eMetrics presentations. Over the past year, we've been working with the HP store on a variety of research and analytics projects. At eMetrics, HP's Andrew Bakonyi and I co-presented on a subset of those projects (mostly because they are the ones that I've personally been involved with). Our theme was a simple one - the power of online opinion research to support customer and business analysis beyond the Web site.

HP faced some pretty stiff challenges in 2011: Executive team changes, major shifts in business strategy, and fairly dramatic systemic change in the core business. As we navigated those challenges, we found that online opinion research gave us a high-impact, low-cost, very quick method of understanding the scope of the problem and identifying better business and customer communication strategies.

We tackled three different analysis projects from 2011 in the presentation. The first was a rush analysis in the wake of the leaks last year around HP's Executive Team and strategic direction. The potential ramifications for HP Store were huge. But while the view from inside was bleak, we wanted to understand the immediate impact on HP's business and brand. Naturally, in the wake of a crisis, time to response is critical. We produced a custom survey instrument and deployed it on the HP Store site in the week following the announcements. To gather data quickly, we boosted the ask rate far beyond what we would normally do - collecting 3,000 surveys in five days. One week later, we'd turned around a complete analysis of consumer knowledge, attitudes, and impact around the news.

Survey design was probably the most difficult part of this project. Each of projects Andrew and I discussed required completely new survey instruments. In every case, we essentially threw out the existing site experience survey and used a much shorter, custom-designed survey. These surveys were typically between 10-13 questions with significant branching. The only questions we kept from the ongoing site experience survey were the core demographic/firmographic/visit-intent questions. Not only were these important in their own right, consistency on these questions gave us the ability to track whether the visitor population was shifting in the wake of the news and whether or not our survey was representative of previous HP site experience.

In the survey, we first explored whether consumers were aware of the HP announcements and, if aware, whether they understood them. For consumers that were aware, we then explored how the news was perceived. Did it raise concerns around company issues (stability, innovation), brand, or product (warranty, reliability)? If it did, how deep were those concerns. For consumers who weren't aware of the news, we wanted to find out what impact exposure would have. So we gave them controlled capsule summaries of key news points and then followed up with impact questions similar to those given the already aware group. This was delicate stuff and quite challenging to put together in a way that we felt was exploratory without being leading.

One of the nice aspects of opinion research is that it can be a powerful corrective to the "inside" view. What feels like a cataclysm in Santa Clara may be more like a mild temblor in Dubuque. Many folks were only loosely aware of the announcements and even among the aware, the perception was by no means uniformly negative. We found a similar pattern when we pushed the 'unaware" by priming them with the news.

Interesting patterns also emerged when we looked at how/when the news did impact brand perception. As we used to find so often in political opinion research, much of what you choose to hear in the news depends on what you already believe. On the whole, brand advocates (we used questions about ownership and consideration set to define this group) were the least likely to take the news as bad. But there were some issues around which Brand Advocates tended to react more negatively than average. It's another old lesson from politics - knowing what might alienate your base is every bit - perhaps even more - important as knowing how to persuade the undecided.

The research also gave us considerable insight into which of several factors (warranty, reliability, company stability, product innovation) were most impacted for different types of consumers. Again, it's no surprise that Corporate Buyers, Consumer Laptop Buyers, and Tablet Buyers had quite different levels of awareness, concern, and focus point. But in the wake of a crisis, there's a tendency to over-compensate and go to broadcast messages. The ability to segment communications is perhaps even more important in a crisis than in more tranquil, business-as-usual times.

In the second project, we focused on the extent to which tablets were cannibalizing laptop sales. Tablets are, rightly, considered a new category. But their explosive growth has coincided with a flattening in the global laptop market and it's an open question how closely the two are related. Our survey instrument for this project focused in on the subset (fairly modest it turned out) of HP Store Laptop buyers who expressed some interest in a Tablet as an alternative.

In the survey, we then explored their likelihood to choose a tablet over a PC and which of a variety of different features or laptop improvements (smaller and lighter form-factor, touch interface, fun package of standard applications, price discounts, etc.) were most likely to shift the consumer into the laptop camp.

This type of research has multiple functions. First, it's designed to help answer the basic question around the degree of cannibalization. Perhaps even more important, it's designed to help understand how to position the laptop versus the tablet for key segments of deciders (and whensuch positioning is even necessary). This type of research is a natural table-set for site testing. As with the initial project, we created the survey here very quickly and inexpensively (a few weeks time - this wasn't a crash project). It was much cheaper to deploy the survey than to test multiple creative alternatives on the site. And by exploring the options by segment, we can make site testing significantly more focused and productive.

Indeed, one of the major themes of our presentation was the close relationship between online survey research and site testing. Not only can online research drive testing alternatives, it should provide the core attitudinal framework within which creative is developed.

This project also illustrated one of our other, less sanguine, themes:  the limitations of online survey research. Within each of the projects we discussed, there were significant questions unanswered. In no case could we say anything about the impact on broader consumer attitudes. Since we're sampling on-site, we're getting a population that is inherently skewed. Was there a population so put-off by the news from HP that they never visited the store? Are most consumers who buy Tablets not even bothering to research laptops? Given our sample limitations, there's no way to know.  This doesn't mean that online survey research is useless. It does mean that you have to be aware, honest, and up-front about what you don't know and can't answer.

It's also why we're exploring more aggressive use of off-site survey research with HP - whether from private panels or independent sites like CNET. Supplementing the on-site view with off-site, independent populations becomes much more important when you start using online opinion research for more than site experience questions. But even when you're deploying on a panel, the opportunity to survey your site visitors shouldn't be missed. You can go far deeper with a far larger sample (and behavioral ties) on your own site than is ever possible offsite.

Our third project delved into the drivers of consumer choice around laptop selection. In a market rapidly approaching saturation (at least domestically), it's possible to grow only by taking market share. That puts a tremendous premium on understanding how consumers are categorizing each brand. In this project, we explored a variety of dimensions including brand consideration set, feature consideration set, key choice drivers, and segmentation by consumer type, role, and attitudes.

In some ways, this was the most classical and broadly interesting of the projects we did in 2011 with HP. It was a perfect illustration of the difference between collecting site experience information via online surveys and doing real customer research.

The results helped us categorize the HP brand in relation to a wide set of competitors. It helped identify which feature sets were important to consumers within any given consideration set (Apple vs. HP buyers are different than ASUS vs. HP buyers or Acer vs. HP buyers). Perhaps most interesting was the set of questions we used to get potential buyers to self-segment themselves along a Price/Value/Image scale. By comparing these self-segmentations with the features buyers also identified as important, we were able to identify some significant mismatches - cases where buyers were reluctant to categorize themselves a certain way despite their actual drivers of choice. Not every image conscious buyer is necessarily comfortable with a laptop called "ENVY" and not every shopper looking for the cheapest machine will describe themselves as a discount shopper.

Again, this is invaluable information for steering a testing program and understanding what types of creative tests might actually pay dividends.

 

When Andrew and I reflected on these three projects, a couple of points really stood out. First, the types of survey we used were fundamentally different that traditional site experience survey instruments. Second, each project demanded quite a different set of questions - putting a real premium on flexible survey deployment. Third, online research is very inexpensive and flexible relative to many other mediums. None of the projects here took more than 3-4 weeks of work (not counting collection of course) and none of them cost more than 10K. When you've got a reasonable technology for creating and deploying online surveys in-place, it's a shame not to use it for true customer research. Finally, every project illustrated at least some of the limitations to on-site intercept survey research. When you only sample site visitors, you can only speculate about what you are missing. There are ways around this problem - and we believe that the combination of on-site survey intercepts with 3rd Party or Panel based survey research provides a compelling overall solution to delivering deep online customer research that is fast, flexible, and cost-efficient.

I've summarized some of our broader thoughts around survey opinion research into a short two-page Point-of-View document. If you're interested, just drop me a line and I'll send it your way.