PrefBench: Benchmark for LLM Agents in Personalized Pricing Negotiations
PrefBench serves as a benchmark utilizing simulation to assess zero-shot LLM agents in negotiations involving personalized pricing with hidden preferences. Each episode features a simulated buyer matched with a specific vehicle-customization package. The seller has access to public persona descriptors, details about the bundle, and the history of negotiations, while the buyer's latent variables influence aspects such as valuation, patience, counter-offer strategies, and decisions to withdraw. This benchmark implements a state-summary protocol for LLMs that requires agents to produce precise JSON actions within a defined hidden-information framework. The authors compare zero-shot LLM sellers against heuristic benchmarks across 7,500 episodes. The study can be found on arXiv with the identifier 2605.22855.
Key facts
- PrefBench is a simulator-based benchmark for hidden-preference personalized pricing negotiations.
- Each episode pairs a simulated buyer with a fixed vehicle-customization bundle.
- Seller observes public persona descriptors, bundle information, and negotiation history.
- Latent buyer variables govern valuation, patience, counter-offer behavior, and walkaway decisions.
- Uses an LLM-facing state-summary protocol requiring strict JSON actions.
- Evaluates zero-shot LLM sellers against heuristic references.
- Conducted over 7,500 episodes.
- Paper available on arXiv with identifier 2605.22855.
Entities
Institutions
- arXiv