Not too long ago I enjoyed a fascinating conversation over lunch with Loren Hadley on our team here at EY around some work he's been doing analyzing Web performance in China. This seemed highly germane to many of our clients, so I asked Loren if he'd be willing to redo the conversation in written form. I wrote up the questions, Loren provided the insight...
You’ve been working on some analytics projects that are China focused and part of that has been comparing China to other regions and countries – not just the U.S…what’s your sense of how different China is? Is China's digital experience unique and if so, to what extent when compared to other major Global markets?
While every country’s Internet use patterns are unique to some extent, there are a lot of similarities across the Western world, including more western oriented Asian economies like Japan. The US has been the driver and largest market and has basically pulled the rest of the world along in its wake.
That’s where China becomes very interesting. In 2014, roughly 41% of China’s population was on-line amounting to roughly 618M Internet users compared to roughly 280M US Internet users which accounts for about 79% of the US population. Chinese internet consumers have grown up on locally optimized sites and logistics systems which differ from the western model. They have lots of options that work for them, so it seem likely to me that digital channels will continue to bend to meet the Chinese consumer’s needs rather than successfully reshaping the Chinese consumer into the Western mold.
When you think about something like a “China” analysis, where do you start and what do you focus on?
Like any analysis, there are business questions being asked and stakeholders with specific interests. That provides an anchor point for the analysis. After that we started at the top-level metrics, segmented as many ways as we could think of and looked for anything interesting or anything we couldn’t explain with our current knowledge of the site. We compared what we were finding to other countries, which often led to broader based research to understand the Chinese norms
Our work is part of an ongoing optimization program so we’re looking for the lowest hanging fruit. Every iteration should see us diving deeper and building off of what we’ve learned in previous analysis.
We often try to think of site analytics as fundamentally use-case driven. Is your sense that the use-cases of visitors in China wouldn’t be much different than here in the US but the success and details of what happens inside that use-case are going to be different OR do you think some completely different kind of use-cases emerge?
While the opportunity certainly exists for new use cases to evolve, our experience has been that the Chinese consumer is similar to Western consumers at a fundamental level. They are trying to find information, shop or be entertained. The exact details of the checkout experience or funny cat video may differ, but the basic need is the same.
We’ve found some differences in how navigation features are used, but it isn’t clear yet to what extent this is something fundamentally different about Chinese consumers’ use of the web, versus opportunities for localization and optimization.
Network speeds in China are much slower on average, yet the expectation is that sites will offer the rich experience we are used to. As a result there are a lot of challenges delivering a winning user experience while keeping page load times at a bearable level.
Consumer expectations around payment and shipping of eCommerce goods are much different than they are in the US. Knock-off products, grey market and fraudulent sites are reportedly more prevalent in China and “caveat emptor” colors user behavior. Chinese consumers appear to shop more readily at trusted sites and seem to seek out trust signals like reviews before committing to new shopping experience. Most commerce tends to be through third party payment systems rather than providing payment information directly to the merchant and many Chinese payment options include an escrow feature that reduces the potential for fraud. COD deliveries are also common.
You talk about how different payment systems and usage propensities are (and that’s not just in China) from the U.S. But what are the analytics and funnel implications to that and what sort of measurement challenges does widespread use of 3rd party payment systems create?
When almost every transaction is completed on a third party site, there’s a big black box in the middle of the funnel. The user is handed over to another site with a different experience and the tracking path ends abruptly before the order is truly completed. In our experience in China almost all orders used a 3rd party payment system which creates a lot of uncertainty around the key conversion metrics. Our situation found that the web tracking accounted for more orders than the transaction records which counted fully paid orders. That’s not a big surprise for anyone that has tracked fallout in third party payment systems, but it complicates analysis and optimization. As a work-around, we found with large enough numbers and some historical data, we were able to apply experience based modifiers that got us fairly close to the real-paid numbers.
What’s the biggest gotcha when it comes to doing analytics on a China-based digital property?
As analyst’s we’re used to figuring things out, recognizing patterns we’ve seen before which help us to make sense of the data. But in our Chinese analysis we often found patterns that left us scratching our heads.
We spent a lot of time hypothesizing then looking for ways to test those hypotheses and validate our assumptions. We are fortunate in that we had an in-country team that could provide some background, and the assistance of a culturally and linguistically bi-lingual analyst that could help us understand the nuances of localization as well as the general mindset of the Chinese web users. The cultural part was really valuable. Although the insights were often anecdotal, it helped us develop hypothesis that as US based, non-Chinese analysts we would never have landed on by ourselves.
What are the implications for a shared global architecture in digital eCommerce given what you’ve generally seen when you looked at China behavior patterns? What’s your general sense of the right balance between standardization and localization when it comes to China?
Shared global architecture often seems to mean taking the optimized Western experience we are most familiar with then providing some degree of localization while maintaining underlying technology and structure. This makes a lot of sense from a management and development perspective and in most cases works fine.
As I mentioned earlier, China is not really in the western orbit but has enough mass that it can develop in its own way. My sense is that western companies will need to decide the level of effort and expected ROI as they plan their China strategies. If they want to add an additional revenue stream by skimming the China market, then localization for language will probably suffice, and there is an interest in western products and culture. If however the company wants to become a major brand in China and really harness the potential of China’s massive Internet market, they may need to go much further with developing a unique Chinese oriented experience and content.
Are there other differences people should expect when they think about analyzing eCommerce in China?
Well, we’ve already covered a lot of ground, but a lot of the surprises were in the details.
As sites advance, support for older browsers often suffers and that can degrade both experience and performance. We’ve seen that the market share for old IE browsers in China as a whole is much, much higher than we are used to seeing. That is dropping over time, but our hypothesis is that there are numbers of newer Internet users that haven’t bother to or don’t realize you can and should update browsers.
Another area is mobile use. Mobile penetration in China is high and text messaging platforms like WeChat (600M+ registered users) have a huge following. However, desktop platforms still deliver the preponderance of traffic. Mobile use in China is integrated into the on-line ecosystem and social media, but it seems to revolve around SMS rather than the smart phone usage in countries like South Korea and Japan. Figuring out how to leverage mobile in China may be a key to success for some companies.
I guess the short answer is that there will be a learning curve for non-Chinese analysts. Understanding what is unique to the site we were analyzing and what is a systemic factor in China took more time than usual.
Is it your sense one of the challenges is a wider variety of existing UI – less standardization of experience than is common in the US – you talk about how users are newer and as I think back to the early 2000s there was much less conformity in UI here in the U.S. which made navigating EVERY website harder. Same in China right now?
I’m not sure what we would find looking across the entire spectrum of Chinese sites, but looking at major Chinese retailers and corporate sites we actually found a fair degree of consistency. It appears that the key players that have defined the mainstream Chinese market have looked at best practices and designed systems that work in the Chinese environment.
How challenging is it work with the analytics when so much of the key stuff is Chinese – and even looking at the source content is hard? What are some of the ways a team can solve those problems?
If the tagging for your project is well implemented and standardized in English, then the actual in-analytics experience isn’t so different. Without that, an analytics project would be nearly unmanageable for a non-Mandarin speaker. As it is we rely heavily on our Mandarin speaking teammates to help us sort out the detail and work through the customer experience. On a micro-level on-line translation tools are pretty handy. The translations are often clumsy, but you can figure out that the big button says roughly “Buy Now!’ and the little one says “Log-in”. At a macro-level it is much harder since you are puzzling out the details so it is more difficult to see the big picture issues. Again, having a Mandarin speaking compatriot lets us step back, ask questions and vicariously develop an understanding of the page.
I suppose that's one part of the content that we don’t always think about - the language itself. A lot of global companies just do straight translations of their site and that has potential pitfalls. What sorts of things have you found around that and how did analytics help identify potential problem areas in translation?
We often find instances where improvements in translation offer easy opportunities to improve aspects of the user experience. Sometimes they are the same sorts of issues we deal with in English: validation prompts may be more confusing than helpful to consumer success. At other times they are due to more nuanced aspects of the language. For example, a term that was used successfully on the English language site is presented in a technically correct translation. However, it has a more formal meaning in Mandarin that doesn’t seem to fully reflect the brand's intent. This provides a great opportunity for testing alternatives that will resonate more with the Chinese consumer.
We’ve all read package inserts or instructions that are translated to English from another language. It may help you get the product put together, but it doesn’t always generate confidence or reflect positively on the quality of the product. This highlights that a technical translation may suffice, but it probably won’t convey the same meaning as copy written for the audience by a native speaker. Companies will need to weigh the cost against benefit of a translation program versus the recreation of content in a linguistic and cultural way that resonates with the Chinese consumer. The later is a much greater undertaking, but it may pay real dividends for sites doing business in China.
You touched on some specific challenges around China with payments, forms and fraud – does any of this impact tagging requirements for digital?
Use of alternate tools that don’t rely on cookies and beacons can help track exits and returns to the funnel from the 3rd party sites. This may offer deeper insights into what was happening, but it can complicate analysis when you have part of the story in one data set and part in another.
Having robust form tracking is going to be important for anyone trying to optimize transactional or registration based performance. As you find the balance between the Chinese consumer’s expectations and the demands of your internal systems, knowing exactly what is and isn’t working will be key.
Tracking and tools that will allow you to determine the optimal mix of 3rd party partners around payment and fulfillment will likely be of critical importance. It will help you home in on the partners that are most used by your audience and potentially weed out the ones that are primarily cluttering the experience.
Last thoughts on analytics targeted to China-based sites? Anything we haven’t covered?
If you will be responsible for analyzing a Chinese site, get involved as soon as possible. As with any project, the quality of the implementation will dictate what you’ll be able to look at and this will be exacerbated by the language issues and predominance of 3rd party tracking.
Develop relationships with in-country team-members and/or make sure your team has access to people who are fluent in Mandarin as well as current Chinese culture.
Have fun with it. It will likely be a fascinating and challenging analysis and will become increasingly relevant to global businesses as China’s economy and Internet base continue to grow.