Adaptive Prompt Elicitation Technique Improves Text-to-Image Generation Alignment
A novel method known as Adaptive Prompt Elicitation (APE) tackles ongoing difficulties in ensuring that text-to-image generation aligns with user intentions. Users frequently submit vague prompts and face challenges with model-specific responses. APE utilizes an information-theoretic framework to create interactive intent inference, portraying hidden user intentions as understandable feature requirements based on language model priors. The system dynamically produces visual queries to assist users in refining their prompts with minimal writing, subsequently consolidating the gathered requirements into effective prompts. Testing on IDEA-Bench and DesignBench reveals that APE enhances alignment and efficiency. A study with 128 participants on user-defined tasks indicates a 19.8% increase in perceived alignment without added workload, presenting a principled alternative to conventional prompting methods.
Key facts
- Adaptive Prompt Elicitation (APE) helps users refine text-to-image prompts
- APE uses visual queries to infer user intent without extensive writing
- The technique formulates interactive intent inference under an information-theoretic framework
- APE represents latent user intent as interpretable feature requirements using language model priors
- Evaluation shows APE achieves stronger alignment with improved efficiency
- A user study with 128 participants demonstrated 19.8% higher perceived alignment
- The study found no increased workload for users
- APE was evaluated on IDEA-Bench and DesignBench
Entities
—