My Photo


  • Clicky Web Analytics

Your email address:

Powered by FeedBlitz

« Reporting on Forms and Conversion Processes | Main | Engagement in Web Analytics - Redux »


Gary, I'm not a web analyst, merely
a passionate web user. Perhaps the
gap between practitioners and theorists
could be lessened by reflecting on the words
of another former US Secretary of Defense:

"As we know, there are known knowns.
There are things we know we know.

We also know there are known unknowns.
That is to say, we know there are some
things we do not know.

But there are also unknown unknowns,
the ones we don't know we don't know."

- Donald Rumsfeld -

Obviously, there is something very dangerous about going to work for the DoD. I will have to tell Phil Kemelor (our Washington VP) not to bid any Pentagon jobs!

Hi Gary,

I read a great post once about accuracy v precision that would fit well into this (wish I could find it). If your data is precise you can trend it over time because the error bars are small. If your data is accurate your get better snapshots.

Knowing that your data isn't accurate is fine, but having to work to get it into a position to be precise before you use it to work out if the changes you've made have worked is probably one of the hardest things. That is something that has always annoyed me about WA tools that limit table lengths and/or go into a sampling mode in the long tail.



That's a good point and I think it gets to the heart of what was originally intended. Analysts wanted - rightly - to make it clear that the numbers in web analytics were not precise enough to use as an official system of record. But they also wanted to make it clear that the numbers could be used to understand the business environment and changes within it. As far as it goes, this was dead-on. Unfortunately, like many good ideas this was somehow transmuted into a very bad idea - the completely mistaken belief that trending is somehow the reason you can use the data (instead of the fact that it is accurate enough for some level of confident prediction) and that if you trended you didn't need to worry if your data actually was accurate enough to use (never mind precise enough for back-office purposes). This turns out to be very bad advice indeed - especially to those just entering the field who are all too likely to believe it! The sad truth is that we see many web analytic data sets that are simply NOT accurate enough to use for many purposes.

Hi Gary,

Excellent post, and the best presentation against trends against accuracy I have read. I'm 100% with you here; I have always detested the trend excuse to data quality laziness. I just spent a week at the TDWI World Conference (data warehouse and BI), and let me tell you that those guys are darn serious when it comes to data quality (I spent two entire days on that topic alone).

True, we make decisions on trends, but as you put it so well, providing we can trust those trends are not caused by variations in the quality of data! I mean, how hard is it to understand?

Anyone who has done a web analytics implementation, who's been through the "tunnel", i.e. that period when numbers don't make any sense, knows that Web Analytics is certainly not a plug'n play world... Data quality assessment is still mandatory, and we should not hide behind "the Internet data is mesy" excuse for not addressing it.

I don't know if the gap here is really between practioners and theorists. It seems to me to be between good and bad analysts...


Thanks. Regarding your last point - I get what you're saying because it is more about good vs. bad analysts. But I have noticed that people who NEVER really practice are particularly susceptible to this mistake. All of us have been bad analysts at one time or another and abused not used our data. But if, like you and I, you've been burned by data quality issues in real life you tend to be much more keenly aware of the problem! Theory from non-practictioners is nearly always vapid, I've found, because it lacks the practical foundation that experience provides.


The comments to this entry are closed.