SimGym: AI Framework for Simulating E-Commerce A/B Tests
SimGym is a framework designed to replicate A/B testing for e-commerce platforms by utilizing vision-language model (VLM) agents within a live browser environment. It overcomes the drawbacks of conventional A/B testing, which often diverts user traffic, takes weeks to achieve statistical significance, and can negatively impact user experience. The framework consists of three main elements: a persona generation pipeline that creates buyer archetypes and intents based on actual clickstream data; a live-browser agent system that integrates multimodal perception, episodic memory, and guardrails to ensure coherent shopping experiences; and an evaluation method that assesses simulated outcome variations against actual buyer behavior. SimGym has been validated through A/B tests focusing on visually driven UI themes.
Key facts
- SimGym simulates A/B tests using VLM agents in a live browser.
- Traditional A/B testing diverts traffic and takes weeks for significance.
- Persona generation pipeline uses production clickstream data.
- Agent architecture includes multimodal perception and episodic memory.
- Evaluation protocol compares simulated and real outcome shifts.
- Validated on visually driven UI theme A/B tests.
- Framework aims to reduce risks of live A/B testing.
- Uses vision-language models for agent behavior.
Entities
—