I was going to blog on the Omniture Reporting API and how it works, but since Omniture’s Chris Wareham took up and commented on the topic of token’s that I mentioned in passing (as the biggest issue with the API), I thought I’d skip ahead and tackle that directly. When I attended the break-out session on the API at the Omniture Summit, virtually every question asked was about the Token system. And when the winner of Omniture’s API Development Contest bravely demoed her program live, she first remarked: “I hope I’m not out of tokens!” In this post, I’m going to explain why tokens are such a big topic right now and where (I hope) Omniture is heading with them.
So what is a token?
Tokens are the currency of usage in the Omniture API. Each request you make has a certain token cost. When you are out of tokens, you need to buy more. Token systems are common as a metering device on APIs to prevent out-of-control usage. And metering is necessary for an API because while it’s possible that some measurement monk could sit in front of SiteCatalyst all day and bang out request after request, it’s fairly unlikely. And even a true measurement black-belt couldn’t possibly pump out a thousandth of the requests that a computer program could easily make.
So while tokens are probably a necessary part of life, Token systems range from the extremely generous to the quite onerous.
Here’s Chris Wareham describing how the Omniture Token system works:
“To get started, each login company gets 2000 monthly tokens for free. Omniture customers can contact their account representatives to purchase more if needed. Those 2000 tokens are 'use them or lose them' on a monthly basis; we don't currently offer 'rollover' tokens, but it's something we're looking at.
In terms of API consumption, whenever a single API "call" is made to Omniture's Reporting, Admin, SAINT, Discover, Data Warehouse, or Data Sources APIs, 1 token is charged. Most generic *Reporting API* requests will require 2 tokens; queueing the request for the data will consume 1 token, and the receipt of the data itself will consume 1 token.
In our other APIs, most transactions will consume only 1 token. As we enrich the kinds of data available via the APIs, it's possible that we'll increase the token charge for certain calls, but for now each single API call consumes only 1 token.”
This is a pretty clear statement of how tokens are assessed. But it does still leave quite a few questions unanswered.
Since each company gets 2000 free tokens, then you could run 1000 requests a month on the API before you had to pay anything.
That seems fairly generous – 1000 reports feels like a lot of reports. But that isn’t exactly the way things always work out. Let me give you an example.
Semphonic has been developing a Traffic Model using the API. We currently run the Traffic Model using Excel Integration (for free no matter how many times it’s refreshed or against however many report suites it’s run). The Traffic Model looks at sources of Traffic over a 6 month period of time and what’s driving change. To do this, it needs to request six months of referring site traffic. Each month needs to be a separate request given the way the API is structured. That means 6 Referring Traffic Requests x 2 Tokens each for 12 tokens. But we don’t just use Referring Site reports for the Traffic Model. We also have to request the Campaign Report, the Natural Search Keywords Report and the Paid Search Keywords Report. So that comes to (6x2)+(6x2)+(6x2)+(6x2) or 48 tokens.
So every time I want to run single a Traffic Model I will expend roughly 50 tokens. That means I could run 40 traffic models a month before I had to start paying. Reasonable? Maybe, except that if you are an organization with 50 report suites, you could not run even a single Traffic Model against each report suite before you had to pay since tokens are company not report suite specific. That starts to feel pretty onerous - especially when compared to Excel Integration where our clients routinely exceed this load.
But if the number of free tokens currently allocated feels a bit small, there’s a much more serious problem with the Token system and it’s a simple and obvious one. Uncertainty.
All currency systems rely on a basic understanding that you can control how much you spend because you can control your own usage and you understand the price. With that in mind, here are the basic questions I asked Chris at Omniture:
1. Is there a published price for tokens (if so what is it)?
2. Is there a discount for buying in bulk?
3. Is the price committed to for some period of time?
4. Are tokens that you purchase additionally also on a one month expiry? If not, how do you know when they’ll expire?
5. Can tokens be purchased by an application vendor instead of a client? Will there be a mechanism for an application vendor to use their tokens on a client report suite?
Knowing the answer to these questions is essential before it would make sense for a company like Semphonic to commit significant resources to a software development project using the API. In fact, knowing the answer is actually more important than the exact nature of the answer. Depending on the price of tokens, I might decide that only applications with relatively high-value to the client or that chew up very few requests actually make sense. But I can’t decide that unless I know the true price.
Omniture does have formal pricing for API consumption, but you need to contact your rep to get it.
On the other hand, that price isn’t fixed in stone right now and there’s a pretty high likelihood of it changing. Ditto for issues like expiry, where Omniture would like the certainty of token usage for expiry planning but customers would prefer the simplicity of an unlimited time horizon. So you can get a price right now (you just have to ask your Rep) – but it’s hard to know if that’s a price you can or should commit to.
Omniture’s current model is focused on charging the customer for API consumption (they haven’t opened the API up for pure 3rd party developers yet; you need a SiteCatalyst login to be able to get documentation and configure and access to the APIs). If they really want to see this grow, they’re going to have to figure out a way for 3rd party application vendors to purchase tokens that can be used to run a service without having the end-customer’s tokens consumed.
Until pretty recently, none of this has been a very big issue. It’s only in the last few months that the API has become robust enough and well enough supported to make development tempting. But now, there simply is no bigger issue than price. Regardless of whether you’re a potential vendor (like Semphonic) or an Omniture client looking to automate reporting tasks, investing in software development when you don’t have a firm grasp of the cost is just too risky.
So here’s what I’d like to see from the Token system in addition to the much needed and already promised Developers Test Environment (Sandbox).
First and most important, I’d like to see cost certainty. The simple fact is that any known cost is better than uncertainty – at least with a known cost I can decide if an application makes sense for my clients or not. This is a psychological barrier more than anything else. Omniture just needs to let people know when or if the price is nailed in stone. If that’s already been done and the current price is what we should expect going forward, then it’s simply a matter of communication. If it hasn’t, then deciding on and nailing that price in stone should be the top priority for the API.
Personally, I think Omniture should not just set but publish (this really is a different case from standard contracts) the prices for tokens in various bulk quantities. I also think that Omniture needs to commit to that price for some fairly significant period of time. Transparency – whatever the cost – is the ultimate goal.
What else would I like Omniture to do?
This is minor, but I would eliminate the token cost for queuing and getting data. That just makes token usage harder to calculate and provides no real benefit to anyone. Why would you queue a request and not get it? I’d do a flat-rate for queuing or making an asynch request.
Second, I would significantly increase the number of free tokens and/or make tokens report suite specific. The goal should be to prevent abuse not discourage use. For many token systems, the enforcement mechanism is a daily meter or a time-based throttle. With a daily meter, you get X requests a day you can’t exceed. This lets you do regular tasks without worrying about exceeding total thresholds and without stressing the system. Another common technique is a time-based throttle that simply slows your requests if you are making too many. This can be hard to work with since it means unknown performance hits in the application (especially since someone else’s application may be causing the problem) – but at least it provides cost certainty and stress protection. The best system isn’t obvious and there are always trade-offs. If I had my druthers (other than totally free and I know that’s not happening), I’d opt for a system with a free daily meter and the ability to purchase up to higher-level plans on the daily meter. When requests exceeded the daily meter, I’d throttle them instead of killing them. A system like this would encourage numerous small simple apps that would essentially be free to deploy. It would also make reasonable pricing for more commercial efforts practical. As a part of this, I think that the API should allow for the direct purchase of tokens – so that a properly authorized application can actually allocate additional tokens in real-time.
Third, I would build a mechanism whereby software developers could use their own tokens for client requests and manage billing/metering on their own. This way, clients who buy my application wouldn’t have to manage a separate billing process with Omniture. As I mentioned, this alone would kill most deals for any inexpensive tools (and inexpensive tools are what makes the most sense for the API). This may not be high on most Omniture client wishlists, but as a potential vendor its right up there with cost transparency as an essential requirement.
Fourth and last, I would make sure tokens don’t expire since it makes the purchase process extraordinarily complex. I just can’t ask clients to purchase tokens every month to run my application and Bill’s application and Joe’s application and not have them know, at any given moment, how many tokens they will have on March 21st as opposed to March 22nd or April 1st. Even if I was running an internal system, I’d be afraid that half my software development effort would be dedicated to token management.
In a way, it’s a good thing that the Token System has become important to the API. It means the API has reached a level of maturity where serious use can be made of it. I’m hoping that Omniture will nail out the Token system in short-order and that it will be a system that truly encourages development. I know from my perspective, I’d love to be a part of that.
And I’d like to give a special thanks to Chris for taking the time to answer my questions and for pushing in the right directions!
Gary
Your posts on Omniture API has been more informative then my account manger. Thanks for clarifying the token system for me.
Mark Hanson
Posted by: Mark Hanson | March 09, 2009 at 02:39 PM