Adjudicated Captioning: Multi-Agent Framework for Zero-Shot Image Captioning

A recent study available on arXiv (2607.28986) introduces Adjudicated Captioning, a multi-agent framework designed for zero-shot image captioning (ZIC) during inference. ZIC generates descriptions for images without the need for paired image-caption supervision in training, depending instead on text-only datasets and fixed pretrained image-text evaluators. Current retrieval-augmented techniques assess image-text alignment solely at retrieval, relying on language-model probability for the captioner's autoregressive beam, which results in a lack of ongoing visual grounding feedback. Progress has stagnated, with no advancements surpassing the best performance in the strict regime since 2024. The new framework enhances grounding feedback at various checkpoints without altering the IFCap captioner. It employs a more robust frozen Retrieval Encoder at the start, integrates a frozen Cross-Attention Verifier to refine the top-9 retrievals to top-5, and includes a learned Reranker with TriFuse at the output beam. This paper serves as a cross-type announcement, suggesting it may have been presented elsewhere. It tackles a significant drawback in ZIC by incorporating visual grounding feedback during decoding, which could propel advancements in image captioning.