Testing, testing… but are you?
How can I avoid tests that are designed with someone else’s goals in mind?
When planning a longer term direct marketing campaign, consider whether each test cell has a significant volume of data available within the same selections. Otherwise, testing might need to start again from scratch in Round 2.
Many lists will have varying coverage of each segmentation variable, so making too many selections in parallel can often drastically reduce the size of the target market.
When asked to select a “random test sample” from a list, many list providers will cherry-pick the records they believe are most likely to respond.
Selections based on a high level of profiling / carefully selected combinations of segmentation variables (especially based on open rates for email campaigns) are often used by media and data owners to ensure a test campaign is successful. However, once the small volume of best fit records is depleted, it is highly unlikely that the roll-out universe will be able to maintain the same level of responsiveness.
Try to ensure that there is a high level of transparency in terms of your test selections and their true roll-out potential, before committing to a campaign.
Campaign planning with only the short term in mind can lead to endless testing of new audiences, with few learnings carrying from one campaign to the next.
Start with the end in mind and try to avoid the temptation of the quick win campaign, as it is unlikely that your acquisition KPIs will be reducing any time soon.
