Web Analysis Tool Evaluation

Hierarchies and Content Analysis: Oh What a Tangled Web We Weave!

This series on Web Analytics Tool Evaluation began with Visitor Segmentation capabilities for a reason. Visitor Segmentation is decidedly harder to support in most web analysis implementations than almost any other set of features – and it also happens to be vital to real-world web analysis. But as important as Visitor Segmentation is, it isn’t the be all and end all of web analytics or web analytics tools. There are numerous other features to consider when you’re thinking about web analytics tools – some vitally important but relatively unappreciated.

Hierarchies and Content Analysis is a good example - it’s not a feature set that most web analysts or evaluators would put second in importance to Visitor Segmentation. But I think that’s because most tools have implemented this set of features in such a cramped (not to say crippled) fashion that it’s hard to see how useful they might actually be.

For most web sites and web analysts, the basic unit of analysis has always been the page. And with the advent of widespread multivariate testing and Web 2.0, analysts are concerned about how to drive that level down a notch. There’s nothing wrong with that – both multivariate testing and Web 2.0 require that approach. But the simple truth is that lots of web site analysis and reporting is actually more interesting when you take the analysis UP one level – and look at behavior by groupings of content and not individual pages.

If your web site is less than fifty pages, you should probably ignore everything I’m going to say in this post. For small web sites, Content Groupings just aren’t that interesting. But most of our clients have web sites that contain thousands or tens of thousands of pages. Some of our retail and publishing clients have web sites with, quite literally, hundreds of thousands of pages. When you get into web sites with hundreds or thousands of pages, the vast majority of individual pages are not usefully analyzable as separate entities. But by no means does this mean they should be ignored.

Here are some examples of how, when and why content groupings are so important.

A product manager wants to know how many visitors looked at his product in the last month, how many looked at any of his product’s customer support pages and how many looked at both.

Your marketing manager wants to be able to target visitors whose primary interest (measured by total page views) is in Product X.

Your marketing manager wants to be able to target visitors whose primary interest (measured by total page views) in Product Category Y.

You want to be able to setup an internal campaign tracking anyone who viewed a page containing information about X.

From all the articles about Topic X on a publishing site, the editor wants to know what topics were viewed next and what other types of non-article content were accessed.

From all the pages of a Sponsored area, the advertising manager wants to know what the conversion performance was.

These are all basic questions: questions that any web analyst needs to be able to answer. But without rich content grouping capabilities, answering them can range from very difficult to nearly impossible.

The difficulty is that in every case the question necessarily involves a basket of pages – and sometimes the basket of pages is quite large and potentially diverse. So if you want to use a path tool or a next pages report or a page detail report to get an answer, you’d find yourself doing a heckuva lot of work and, in many cases, you’d be faced with an impossible de-duping problem to get a good answer about visits and visitors.

The first item on the list, in particular, is shockingly basic. There is virtually no company anywhere where product or support managers don’t want to know how many unique visitors actually came to their area of the site. Shockingly, I’ve seen web analytics tool vendors push back on this need! The attitude I’ve more than once encountered is "what are you going to do with that information?" asked in a way that makes it sound as if the client is being simply unreasonable.

Excuse me? How is it possible for anyone to imagine that knowing my prospect reach or customer usage isn’t an important marketing number (and why I do have to justify it to my vendor)?

Fortunately, most web analysis tools do have some content analysis grouping and reporting capabilities. As with visitor segmentation, however, there are often crippling limitations on the usage and implementation requirements.

Here’s a list of the key features to consider when it comes to grouping pages and using these content groupings in real-world analysis:

1. Ability to use Groups of Pages

• As a reporting entity for all basic statistics
•   Especially for Visitors and Visits De-duped
• Correlated to Success Outcomes
• As part of a Path
• Entries (Referrals and Multi-Level Content)
• Exits (Site and Multi-Level Content)
• As part of a Goal
• As part of a Campaign
• As part of a Segment Building Exercise

2. Ability to View Content Levels Graphically

• Full Statistics by Node
• View Multiple Nodes Open at One Time
• Nest Nodes to many levels

3. Ability to create Groups of Pages

• Tag Based
• CMS interfaces
• GUI-Based one-rule at a time
• GUI-Based Visual
• GUI-Based Visual w. Automation Rules

4. Ability to organize an entire site

• Ability to view multiple Organizations simultaneously
• Ability to "create" a logical structure

The features in #1 focus on whether you can use page groups to get all of the information you need. Particularly apropos are getting de-duped visitor and visit counts for groupings of pages, correlating page group usage to success (vital!), the ability to do Pathing at the Group level and the ability to use Content Group statistics (like pages viewed) in key tasks like setting up visitor segments.

Number 2 is probably the least important set of capabilities – since it focuses more on how a tool presents the data than how much data there is. For an analyst, data presentation issues nearly always turn out to be secondary. But this is one of the few areas of web site reporting that can really reward some visual display – since it’s often essential to be able to understand the usage distribution across a hierarchy.

The actual creation of page groupings is one of the more problematic areas when it comes to real-world tool capabilities. When you have to create groupings with tags, this whole set of features may be rendered entirely useless. And for large sites, having quick and powerful methods for organizing a site and keeping it organized is just plain necessary.

Finally, there are some unique capabilities that are hardly supported in today’s existing tool set but that would be very nice to have – and I’ve grouped those into #4.

Taken together, the constellation of features surrounding content groups are a real source of differentiation when it comes to current web analysis tools. And while everybody understands the importance of visitor segmentation and at least some of the right questions to ask, that just hasn’t been the case with grouping content.

In the next post, I’ll take a closer look at some of core capabilities related to #1 - using groups of pages.