(Part III of a Series on Methods in Web Analysis)
In the last post in this Series, I worked through measuring Search Sourcing and deciding when pages are under/over sourcing to Search. Before that, I covered some basics about measuring Search including the distinction between Search as a “Primary Navigation Option” (a preferred method of steering visitors to content) and a “Fallback Method” (Search only when basic navigation fails). This distinction is central to every analysis of Search. In this post, I’m going to cover evaluating Internal Search performance relative to the other navigational tools on your site.
There are quite a few different ways of measuring Search performance – but I like to start with the Functionalist paradigm. If you think about it, you’ll see that Search is really a special type of Router Page – a page designed to move visitors to the appropriate place on the site. What’s more, Search as a tool on your site is competing with pages that are usually classified as Router pages – the pages on your site whose primary purpose isn’t selling anything but just getting visitors to the right place.
When you measure Router pages, the key Functional KPI is the percent of routes that actually go in the direction intended. These are called “body” routes because they typically exist in the main body of the page (as opposed to top, right or bottom navigation). The higher the percentage of routes directed to the target, the better the router.
Search fits a little uncomfortably into this paradigm because it isn’t designed to route to any particular set of pages. However, that doesn’t mean you can’t look at Search as a Router. You just have to a little creative about you are going to classify routes.
Generally, you should assume that Search routed to an intended destination when the “NEXT” page isn’t one of the following: a Site Exit, another Search, the Home Page or a page available globally (such as router pages directly accessible from Top Nav). Why? The thinking is that Search is designed to move people down into specific areas of your site – ideally to the most directive content. That means Search shouldn’t be taking people to Router pages. Otherwise, you’re just stacking up navigation options.
Using this strategy, you can break down Search NEXT pages into the basic Functional Categories:
Site Exit: Exit
Back: Home Page or Landing Page
Sideways: Pages available from Global Nav and Repeat Search Behavior
Body Route: All other pages.
In each case, the value should be expressed as a percentage of Total Routes. Here’s an example:
Route Count |
Route Type Count |
Route Percentage | |
Views |
3,758 |
|
|
Exits |
785 |
|
34.90% |
Routes |
2,251 |
|
|
Back |
141 |
6.30% | |
Side |
260 |
11.60% | |
|
Body |
1,065 |
47.30% |
|
Multi-Page Search Results |
1,507 |
66.90% |
Collecting these numbers is not as straightforward as you might think. VIEWS and EXITS are simple in almost every web analytics solution. You get this number from your basic page reports for the Search Results page.
Total Routes is calculated as the total of all Routes given at the bottom of your tool’s Next Pages Report (all the real NEXT PAGES) plus Repeat Search Count (if obtainable) plus the Exit count.
This should cover every possibility – visitors who search either searched again, went to a page or left the site. Multi-Page Search Results are Views – Routes. Why?
Well, it isn’t always the case. But most sites don’t change the page name when a visitor scrolls through Search Results pages. That means subsequent pages are most often treated as the same page – and not shown in either Path or Next Page reports. You have to be careful about this, however, because there isn’t a standard treatment of Refresh pages across tools.
Because Search Results to Search Results aren’t given in those reports it needs to be derived on the theory that a visitor had to go somewhere (Exit or a Page or Reload).
The easiest way to derive the “Back” count is to simply use the count for the number of times the Home Page is the next page after Search Results (you may need to do something fancier in special cases).
For Sideway’s Routes, sum up the Next Page Counts of all of the pages directly available from Top Navigation or to Pages classified as Navigational. In the Functional method, that would be any Engager or Router pages on your site. Alternatively, you can use Link Counts from Search for any Top or Side Navigation links. All other routes are “Body” Routes.
The Percentages for Exit/Back/Side/Body are the Count divided by the Routes. This tells you of all the visits where a Search Result was obtained what percentage eventually went in each direction.
Using this method, you can compare how well Search stacks up your other Routing pages on your site – an important comparison to consider when you’re deciding how and when to drive users to Search. Changes in this measure are also an excellent way to track the performance of your Search engine. You’d be surprised how often this simple analysis based on the classification of routes will provide you with a clear comparison of the effectiveness of Search.
In essence, what this method measures is how well Search is driving visitors to any of its results. Of course, those results may not be “good” results. And, at this level, your analysis is far from complete.
One of the interesting facets of Search Analysis, after all, is that Search can perform very well for some types of searches and very poorly for others. Is there a way to break down Search performance by Category?
There is, but it isn’t always available in every tool and the exact method varies. The basic idea is simple: create a segment based on visitors who “Searched” specific types of terms. Then look at the Search Performance for just those visitors/visits. I’ll take a look at some ideas for doing this in the next post.
Comments