Evaluating Internal Search Performance

Search Cues

(Part VI of a Series on Methods in Web Analysis)

I’m going to wrap up this series on Internal Search in my next post – which will deal with some thoughts about the overall optimization of Internal Search. But before I go there, there is last part of Internal Search analysis that I wanted to touch on.

With so much of web analytics, the biggest problem the analyst faces is understanding what the visitor was trying to accomplish based on what the visitor actually viewed. This is problematic for a whole bunch of reasons – not least because visitors often end up on pages they don’t care about and pages often have many different functions. But Internal Search is unusual in that the analyst actually has some hard data about what the visitor was looking for – the search term used.

It’s because of this extra knowledge that Internal Search can often be deployed in an interesting fashion when doing other types of analysis.

For example, there are several Functional types of analysis where we look at the number of people who went backwards to the page they started from after navigating to a page. You’ll also hear this behavior described as pogo-sticking. The user is bouncing up and down in the site.

When a page has a very high rate of back-out behaviors, one technique for understanding why is to check the Internal Search Terms used by visitors who viewed this page or visitors who viewed this page, backed-out and then Searched. In both cases, you’ll be looking at the behavior of visitors who navigated to the page and then – for whatever reason – decided to Search. In most cases, it’s a reasonable inference to think that they didn’t find what they expected or what they were looking for. Normally, of course, you’d be stuck trying to decide what that something is. But the chances are pretty good that comparing Search Terms used here to the overall site will show some distinct differences. And those differences likely highlight what content users were expecting and didn’t find.

Similar analysis is often useful in Customer Support functions. It is almost always worthwhile to isolate the Search Terms used from any major customer support page – especially FAQs. This analysis is simple – and will nearly always point up areas where your pages aren’t meeting visitor needs.

Surprisingly, one of the most interesting types of Search Term analysis is what happens after a Search! Some web analytics packages will do this analysis for you – providing an affinity report of Search Terms searched AFTER any given term. It’s a very cool idea. This analysis can help you with several key search optimization tasks. It can help you identify Search Terms where customization of the output may be most desirable; it can help you understand possible customer market baskets; and it can be used to help identify weaknesses in the Search Results returned by your Internal Search Engine.

When this isn’t an out of the box report, you can replicate it by building a visitor segment based on using a particular keyword and then studying the other search terms used. Note that while this analysis is useful for most of the tasks mentioned above, it doesn’t necessarily capture any time sequence. It’s possible that the associated keywords were used FIRST!

In general, looking at the Internal Search Terms from any given point or event in the site and comparing the frequency to the overall distribution is interesting whenever you think there is an unexplained set of visitor behaviors. You will probably find that some locations on your site are not conducive to this analysis (like the checkout process) – but when it works it’s a rare opportunity for the web analyst to see into the mind of the visitor and it’s a surprisingly useful technique to keep in your arsenal of analytic tricks!