The Process of Enterprise Reporting

Segmentation is fundamental to enterprise digital reporting and in my last post, I showed how Semphonic is working to embed a two-tiered segmentation deep into the heart of our reports. Those reports are designed to immediately answer both who did something on the Web and what they were trying to do. These two questions - the who and the why - are critical to any deep understanding of web performance and drive the approach we call two-tiered segmentation:

 

But if a report like this - with deeply embedded segmentation - is the end goal of enterprise reporting, what's the beginning? Like everyone else, we have to go through a process of gathering business requirements and translating them into a report set. Having this type of report as the end goal fundamentally changes this process, making it at once crisper and more coherent.

Here's why.

In the old days, we'd come to the task of gathering business requirements with a fairly substantial questionnaire designed to explore what data a stakeholder looked at, what they wanted to look at, and what measures were important to them. At root, we wanted to understand how the business worked. The goal of this process was to help us isolate both Key Performance Indicators and supporting metrics that might serve as key performance drivers (KPDs).

The goal of this process (sitewide KPIs) was fundamentally mistaken, but that doesn't mean that the requirements gathering was equally flawed. In exploring  the way the business worked, we often learned a great deal that was useful and the contributed to potentially interesting reporting and analysis.

On the other hand, there was a significant disconnect between the type of questions we asked, the feedback we got, and the reports we produced. It's easy enough to see why this happened. In stakeholder meetings, the type of performance oriented metrics we heard about fell into one of two types: not Web specific and too high-level to be really interesting (Revenue) or Web specific and generally meaningless (traffic). Measures like revenue are, of course, fundamental to the business. But they aren't Web analytics metrics. Few organizations rely on their Web analytics tool for revenue reporting. To make revenue interesting at the digital level, you need to be able to show how Web-specific metrics translate into revenue. Unfortunately, the Web-specific metrics we elicited (like Traffic) are extremely poor for making that translation. So we often found that, coming out our stakeholder requirements process, we were still largely creating reports to our specifications and then having to explain how the reports emerged from the requirements. What's worse, the method we had for requirements gathering wasn't particularly effective at driving the discovery of these linkages - sometimes it worked but often enough it didn't.

As we've replaced our reporting systems, however, we've also found ourselves replacing the process of gathering business requirements with something much more specific, focused, and comprehensible.

To build the report above, we need to understand four basic things:

• The audiences to a Website
• The visit types for each of those audiences
• The business goals for each of those visit types of each of those audiences
• The way to measure the accomplishment of those business goals for a given visit type of a specific audience

So to accommodate our report design, we re-focused our process to answer these questions. That's way, these days, we do business requirements gathering by beginning with the audience. We explore with stakeholders how they think about their customers. What segmentation they use - explicit or implicit - becomes the first step in gathering business requirements. This step, as it happens, also tends to drive us into a better, deeper understanding of the business.

When we've explored the audience and segmentation, we turn to the purposes of the Website. Our goal here is to elucidate visit types, but we structure this as a conversation around how the stakeholder wants to serve the needs of each audience. By exploring these questions, we create a good model of the possible visit types on the Web. It's a model we'll have to flesh out behaviorally, but it often provides us with most of what we need to create an abstract visit-type segmentation.

In the next step, we explore with stakeholders the business goals for each type of audience and visit intent. If a "blank" comes to the website because they want to "x", what's our business goal? Sometimes, we'll get wonderfully specific answers to this type of question. Business goals might be to sell them a product, to sell them more products, to generate a lead, or to register so we can contact them going forward. But, of course, goals aren't always of that sort. Sometimes, the business goal is to deepen brand awareness or answer a customer support question. What we don't accept as an answer to this question is something like "View Page X." That's not a business goal, that's a Web metric.

When we have those business goals specific to each audience and visit type, the ball is largely in our court to find the right metrics to measure that goal. Those metrics may involve Web analytics measures like "viewed page X" or "purchased product of entry" or "showed increased satisfaction relative to control" or "exited without calling." The range of possible metrics is enormous and there are deep technical issues involved in choosing the right metric. But that's our job.

The beauty of this process is two-fold. By structuring the process in this way, we collect the information we need to help us effectively build a two-tiered segmentation using our analytic techniques. That's the putative purpose. But even better, stakeholders can directly see how their feedback drives the reporting.

Each part of their responses drives specific parts of the report they end up seeing. The audience information is, of course, embedded in the core of the top-level dashboard (1).

The visit-type information (2) is embedded at the next in the drill-downs we provide:

The business goals are embedded in the success metrics (3) that are at the heart of the reports calculations:

 

Each piece of stakeholder input flows directly into the reports and becomes a part of the organizational consensus about measuring success. This brings a completely new level of coherence and consistency to the process of building reports via requirements gathering.

Our part at Semphonic is to engineer all of this in the analytics by creating the segmentations that instantiate and identify both visitor and visit types and measure each of the businesses goals. That's what people pay us for. But it seems to me that as valuable as that nuts-and-bolts measurement expertise is, the biggest value in the process is the clarity it provides. It's a clarity that supports the creation of a consensus about what really matters in digital marketing: what audience are you trying to serve, how are you trying to support them, and what's your business goal. With that clarity, our task of tying these business goals to specific metrics becomes vastly easier and the work we provide in that realm far more compelling and useful.

Armed with a proper understanding of audience, visit-type and goals, stakeholders find it much easier to evaluate the degree to which specific metrics actually capture business success. This, in turn, deepens the usefulness of Web reporting and drives better understanding of the actual levers of change. Ultimately, it translates into the true holy grail of enterprise reporting: better marketing decisions.

Comments

[Bnnejn]

Your point about the questionnaire and usage of KPI's is great...I think it's a genius idea to start with the audience because as an outside partner, you're just getting up to speed on your clients business and clients usually have a deep insight into their audience segments.