Generative AI and Adaptive Testing Boost Ad Creative Performance by 45%
A recent study published on arXiv introduces an offline-to-online approach for optimizing ad creatives through generative models and adaptive testing. This technique employs a predictive model developed from past A/B tests to evaluate and enhance outputs from generative models during the offline stage, followed by the implementation of a final test slate in an online adaptive experiment. In a field experiment featuring 50 arms, the top AI-generated creative outperformed the best human-created content by 45.1% in terms of engagement, with comparable upper-tail improvements observed in two other experiments.
Key facts
- arXiv paper 2607.23696 proposes offline-to-online creative optimization.
- Generative models produce many creatives but evaluation requires online experiments.
- A predictive model trained on historical A/B tests guides generation as an inference-time critic.
- Offline phase ranks and refines variants from a generative model.
- Final slate tested in an online adaptive experiment.
- 50-arm field experiment showed 45.1% higher engagement for AI creative vs. best human creative.
- Two additional experiments confirmed the same upper-tail pattern.
- The study focuses on ad creative optimization constrained by evaluation.
Entities
Institutions
- arXiv