Social Media Dashboarding – Q&A

Scott and I got quite a few questions after our webinar on Social Media dashboarding. The vast majority of the questions were about the tools Semphonic used to create the dashboard samples. I won’t repeat these in their many variations – I’ll just talk about the whole process - which I think will answer all of the questions on this topic.

The answer to how these dashboards get created is both simple and complex. Simple because every dashboard I showed was produced in Excel. Complex because the data was pulled from a wide variety of sources using a wide variety of methods. We tend to use whatever tools our clients have or make available to us. The dashboards I showed included web analytics data from Omniture, WebTrends, NetInsight and GA. They included social mentions, influencer and topic data from Radian6 and BuzzMetrics. The competitive data came from a variety of sources including Hitwise, Quantcast, and Compete as well as Google Insights for search data.

These reports aren’t canned. They aren’t produced by default in any vendor tool. And one of the most important points I tried to make in the webinar is that to get interesting dashboard metrics, you need to classify and trend the basic metrics in interesting ways. Hitwise, for example, provides a list of up-stream and down-stream sites. But only by classifying the sites and then aggregating and trending the data can those reports be turned into interesting dashboard metrics.

We tend to do the vast majority of this work in Excel, but as the number and complexity of information sources grow, using tools like Tableau also becomes quite attractive.

So the basic answer as to how these reports were created is that we setup appropriate profiles in the social monitoring, web analytics and competitive intelligence tools. This is a considerable amount of work. And it’s important to realize that some classifications HAVE to be incorporated at this step. Most of these tools are fairly restrictive in the way they let you collect and produce information. So you can’t usually setup a single profile/account and then aggregate the data in the ways you decide are interesting. You have to collect much of the data based on the way you intend to use it.

We then export the data (and the methods range from APIs to Excel integration and automated report generation to cut-and-paste) into Excel where it is further classified and aggregated in the ways we deem interesting. For example, we tend to export the entire list of significant up-stream and down-stream sites and then use lookup tables in Excel to classify them as Competitors, Industry Neutral or Other and then aggregate from there. These classifications drive the final presentation tabs – all of which were done in Excel 2007.

Lots of people were specifically interested in the competitive analysis and the up-stream/down-stream metrics that I showed:

   The competitive analysis report is built entirely from social monitoring data using a number of different profiles.

As I mentioned in my last post, the only real source for up-stream and down-stream data are panel or ISP based tracking services. Because these services (Hitwise, Quantcast and Compete are in this space) track all of a visitors activity, they are able to profile affinities between sites. Hitwise provides the additional (and interesting) fact of upstream vs. downstream data.

 
Another issue that drew quite a bit of attention was our discussion of sentiment analysis. Sentiment analysis is so interesting and contextual that our clients nearly always demand that we include it in dashboards. But we often push-back since the data can be problematic. Neil Beam from AT&T added this comment:

“At AT&T, we are moving away showing automated sentiment at the exec level. We only show manual scoring at the exec level. Our engagement team uses sentiment to determine the queue for responses.”

To me, this is best practice all the way around and it’s what I’d like our clients to do more consistently.

We were also asked if automatic sentiment analysis tools give probabilities of classification instead of just an absolute classification. Most don’t and even if they did, it might be hard to extend that into interesting dashboard measurements. Some tools do allow for customization of the keyword set used in sentiment analysis and that can help but if basic keyword lookups are being used for sentiment analysis I think the results are always going to be too iffy to use in Executive dashboards.

Finally, I had a question about where “insights” fits into the picture of Monitoring vs. Measurement given by this slide:

 

In a way, this is a bigger, scarier topic – and one for a broader discussion around reporting. I addressed this in some depth in my recent presentation on Advanced Analytics if anyone wants to check that out.

What I most wanted to get at here, however, was the fact that the tools (like Radian6, BuzzMetrics, and SM2) that are used for social measurement started life as tools dedicated to social monitoring and are still used that way much of the time. Like web logs, the “river of news” used to help PR and Customer Support professionals track and respond to conversations wasn’t originally intended for measurement.

If you don’t realize this, you’re likely to be rudely surprised by some of the reporting and export limitations in these products (though they are rapidly improving as measurement becomes more salient). Another point I wanted to make here is that because these tools are often setup for one purpose (monitoring), there is a tendency to just use the setup create by the PR folks when it comes to measurement. This is a bad idea – not only is the setup unlikely to be clean, but it won’t capture many of the classifications that actually turn out to be interesting.

Insight, I hope, does come into the picture and it comes in on the Measurement side. The 3Cs I talked about (Culling, Classification and Context) are all designed to create a set reports that help report consumers generate insight. I phrased that as delicately as I could because dashboards themselves don’t include insight.

I once read an aphorism in a military novel to the effect that “Surprise is an event that happens in the mind of an opposing commander.” Surprise isn’t an objective fact about a situation, it’s a description of a mental state or attitude associated with the situation. Just so, I’d like to say with insight. Insight is an event that happens in the mind of report consumer. Our dashboards don’t guarantee it and they don’t embody it. But done well, they can make the event far more likely.

Comments

[Lauren Vargas]

There is no cookie cutter approach when it comes to measurement. Best practices are YOUR organization's practices because measurement objectives should originate from business goals. Unfortunately, there is not a magic button that will spit back all the data we seek or glean actionable insights for us. A combination of tools is often used. With that being said, I would not put all social media monitoring tools on the same level. The river of news is only one aspect of our tool. The power of the tool is in the splicing and dicing of the data and real-time engagement within the dashboard. One must feel comfortable with the tool or combination of tools to discover the insights within the data. Use how others have measured opr analyzed data as a guideline for your organization to invest time in a similar practice that aligns with your organization's goals.

Lauren Vargas
Sr. Community Manager at Radian6
@VargasL