RewardHarness: Self-Evolving AI Reward Model from Few Examples
A novel AI framework named RewardHarness, outlined in a preprint on arXiv (2605.08703), introduces a self-evolving method for reward modeling specifically aimed at instruction-guided image editing. In contrast to conventional reward models that depend on extensive preference annotations and further model training, RewardHarness shifts the focus from weight optimization to context evolution. It adapts to human preferences by progressively developing a toolkit from as few as 100 preference examples. An Orchestrator selects pertinent tools from this library, while a frozen Sub-Agent assesses edited images based on given instructions. This innovation tackles the issue of data efficiency, as humans can deduce evaluation criteria from limited examples, whereas models usually require vast numbers of comparisons. The framework is crafted to adjust to nuanced human preferences without the need for extensive retraining.
Key facts
- RewardHarness is a self-evolving agentic reward framework for image editing evaluation.
- It requires as few as 100 preference demonstrations instead of hundreds of thousands.
- The framework uses an Orchestrator to select tools and skills from a library.
- A frozen Sub-Agent applies the selected tools to evaluate edits.
- It reframes reward modeling as context evolution, not weight optimization.
- The approach aims to close the data-efficiency gap in reward modeling.
- The preprint is available on arXiv with ID 2605.08703.
- The system is designed for instruction-guided image edits.
Entities
Institutions
- arXiv