Most effective marketers would be familiar with AB testing as part of their on-going optimisation strategy. Facebook and Google all provide ready to use tools which allows you as a marketer to run such test easily and quickly get results.
However, in China, testing is very different and loosely applied.
Testing is used very loosely by marketers in China. In most cases, marketers idea of testing is simply running a bunch of ads and observing which ones have the better CTR, in some instances marketers measure percentage reach (which is how much of your impression reaches the right audience) and compare different media platform efficiency based on this concept of % reach.
First, you need to understand that each media platform (or eco-system) treats data as company secrets (i.e. highly confidential) and cannot be readily made available. Historically, media platforms did not even disclose impressions or clicks to advertisers. 3rd party tracking was not allowed. You would be happy to note that things have changed for the better over the years. 3rd party tracking is allowed in certain instances. Impression and click tracking are more common now, though ad-serving is still not a norm.
Second, some of the more popular media channels are so huge, they don’t play nice with advertisers because they know they will have a steady stream of advertisers even if they lose the few who ask for too much. For example, Weibo till today still does not allow you to impression track with a 3rd party, because they simply refuse to. WeChat does not allow 3rd party tracking for Moment Ads too. Advertisers still line up to advertise because these are two of the largest platforms in China.
So, you still want to run an AB Split Test?
Good news! While it’s still challenging to setup and execute such test, it is possible.
WeChat (Tencent) revealed to advertisers (late 2018) that they are able to run AB split test for advertisers, but will require a min campaign budget of RMB 1 million. Yup, not your typically idea of a small test; and their rationale is that their tech team has to run it manually, and you need the large budget to ensure there is sufficient impressions.
If you had smaller budgets, then the better way to go about it would be work with smaller publishers, and ad networks.
Small publishers are typically more open to accommodate advertisers request as they want to ramp up their advertising revenue. These channels, however, are typically verticals, i.e. sports related, beauty related, etc, and offer lower impression volume.
Ad networks already have a programmatic way of delivery ads to specific audience based on cookies or device ID. Hence, they are able to help you select and isolate different audience segments and run different ads. An additional advantage for ad networks is their ability to target specific media platforms. For example, I could create 2 audience segment and target only Tencent Video to run my AB Split test. Doing so will allow me to validate my audience and/or the media platform. You have to take note that ad networks typically takes left over inventory from publishers, so you may not be able to always hit your impression target. They also have a minimum campaign budget, but this would be much lower and very much negotiable.
A final important note about running AB test on specific publishers or ad networks is that you are unable to isolate the audience across publishers or networks. So, you would have to get creative in your test by leveraging available targeting like Geo, Telco, IP, to further segment the audience to ensure a clean test.
If you need help in your China marketing, and would like to understand more about AB Split Testing, feel free to get connected with me here: