Form Abandonment – The Analysis of Key Web Site Funnels

In my first post of the year, I talked about the importance of having a detailed project oriented plan around measurement. But what kind of projects actually go into a measurement plan?

Obviously, the exact makeup of a plan is highly client-specific. In the past couple of months I’ve written up measurement projects as diverse as a study of how visitors interested in 2008 election stories could be engaged with a broader news site to an Analytic Report on the key drivers of Newsletter Subscriptions. But there are some projects that go into nearly every company’s plans. And one of the most ubiquitous is the study of key funnels on a site.

For a long time, this supposedly simple analysis was THE bellwether web analytics project. It was the analysis that was most often used to justify web analytics and it was the analysis I most often heard vendors and consultants talk about when they trotted out examples of multi-million dollar returns.

All of which is kind of unfortunate, because it’s also one of the more misunderstood analytic tasks. The common folk wisdom (as embodied in most expert "best practices") on how to think about and measure Funnel Abandonment is deeply flawed. Combine this with unrealistically high expectations (based on largely anecdotal opinion formed in years past when most site conversion funnels WERE really bad) and a strong tendency to do this analysis as a first project and you have an excellent recipe for failure.

That being said, Funnel Analysis is important. It may the single most important piece of many governmental form-based sites. Funnel Analysis is a vital part of any eCommerce site. It is a central analysis for service-based Financial Services sites in Insurance, Banking and Brokerage. And even for content and community sites, there are nearly always key funnels that are fairly important in terms of overall site performance.

So I thought I’d tackle a short, analysis focused series on the basics of measuring site funnels. I’ll begin the true analytic part of the series in my next post – when I’ll take a close look at Form Abandonment by Step – the classic conversion funnel analysis. Following that, I’m going to cover some of the key behavioral questions you can answer about key funnels and how to tackle each. That will probably cover three or even four posts since there is a fair amount of analytic work around these areas. Finally, I’ll cover Funnel Analysis KPIs (since this is an area where analysis should almost always drive to reporting) and also some special cases within conversion analysis.

But before I dive into that, I think it’s necessary to give a bit of background on where web analytics fits in the overall process of creating and optimizing conversion processes on a site.

This is definitely an area where web analytics is just one piece of a larger puzzle. There are at least three critical steps in creating good funnel processes on a site: developing and applying good design principles, doing real usability testing and, finally, doing behavioral analysis. I might even be inclined to add a fourth – doing CEM-based analysis driven by customer support and web analytic issues.

Each of these is absolutely essential to building truly world-class conversion or form-based experiences.

When you start any forms-based process, you’ll start with business requirements welded to good design principles. The truth is that both behavioral analysis and usability analysis work much better when a Form is relatively well-designed to begin with. It can be nearly impossible to converge on a highly optimized design when the basics of the design are a mess. Critics of multivariate testing often make the very legitimate point that testing may converge on a local optimum and miss much larger opportunities. That's a very fair point – and a poorly conceived overall design can make this ever so likely when it comes to funnel analysis.

Your basic design needs to be responsive to questions like:

• Do the Fields Flow Logically?
• Is all of the necessary information captured (and have extraneous requests been eliminated)?
• Is it easy to navigate to each piece of the Form?
• Is the Screen Performance acceptable?
• Are Lists and Choices presented where appropriate?
• Are Instructions clear and readable?

Once you’ve built your initial test process, the next essential ingredient is true usability testing. Generally, this involves turning loose test users who are asked to complete the specific processes and are closely monitored while doing so. Usability testing is in no respect redundant with the behavioral analysis you do in web analytics. It provides you with completely different types of information.

First, usability testing has the huge of advantage that you can do it BEFORE rolling out a process to your real customers. That’s nice. Second, usability testing provides a much better view into potentially crippling navigational issues. You might discover these problems with web analysis (but you might not) – and, in any case, you’ve discovered them long after you really should have if you aren't doing usability tests. Because usability testing lets you study and discuss problems with testers, you can often immediately identify the source of difficulties and resolve them. This is by no means a slam dunk type of activity with web analytics.

Usability testing focuses on answering questions like:

• Can the Form Process be found?
• Are the fields understandable?
• Are entries validated correctly and does the navigation function properly?

• How long does the Form take to complete?
• Is it clear how to navigate the form to completion?

Usability testing of thoughtfully designed forms will pretty much insure that your funnel processes are solidly acceptable. So what do you need behavioral analysis for?

Behavioral analysis (web analytics) can take your funnel processes one-step beyond solidly acceptable into the truly outstanding category. Because the behavioral analysis you get from web analytics generally focuses on what happens with REAL customers. The elements of re-assurance, persuasion, timing, and friction that go beyond basic design and are not captured in the artificial labs of testing can make a very big difference in actual performance.

So the real questions of funnel analysis in web analytics tend to somewhat different than many people think – since the questions they often start out with should already have been answered in usability and design. Instead, I think questions like these tend to come to the fore:

• Are users actually ready for the Form Process when they land on it?
• Is additional persuasion necessary as visitors navigate the Form to keep them on track?
• Do certain types of visitors behave differently in the Form?
• Is there site information that might need to be surfaced in the Form when used by actual buyers?
• Are there too many fields on a Form Page?

In the next couple posts, I’ll talk about how to answer each of these behavioral questions with web measurement.

How about the CEM tools I mentioned earlier – do they fit into this picture? We at Semphonic haven’t done much in this area (hoping to remedy that), but I think there is a fairly compelling case for CEM if your site has complicated conversion processes.

CEM tools let analysts re-create sessions with considerably more surrounding detail – they are like a cross between usability tests and web analytics. Usability tests retain the advantage of occurring before rollout and of allowing conversation with the user. Analytics retains the advantage of looking at aggregate real behavior to identify trends and problems. But CEM tools – especially when driven by problems discovered in web analytics or in customer support – can provide analysts with the unique ability to re-experience real-world problem sessions. Particularly for more complex funnel processes, this might be a critical fourth leg on which to stand.

Put these four steps together, and you will definitely be on the road to having world-class conversion funnels on your site.

I think that’s enough background for now. I must say, I'm excited to be tackling a true deep-dive analytics topic again. I haven’t done that since the massive SEM Analytics series I wrapped up last fall. There’s a surprising amount of interesting analysis around conversion funnels and I’m looking forward to this series! In the next post, I’ll take a look at Form Abandonment and talk about why I think it is so frequently misunderstood.