RRS-10K: Benchmark for Rare Remote Sensing Image Interpretation
Researchers have introduced RRS-10K, a benchmark designed to evaluate vision-language models (VLMs) on rare remote sensing image interpretation. The benchmark contains 10,738 military-related images collected from first-hand sources, paired with multiple-format question-answer pairs. It covers three capability dimensions, six sub-dimensions, and 20 leaf tasks, focusing on perception, reasoning, and robustness. A similarity-based distractor filtering strategy (SDFS) was employed to improve multiple-choice question quality. Evaluation of 52 models showed only moderate zero-shot performance, highlighting gaps in VLM capabilities for rare scenes.
Key facts
- RRS-10K contains 10,738 military-related remote sensing images.
- Images are from first-hand sources.
- Benchmark covers three capability dimensions, six sub-dimensions, and 20 leaf tasks.
- Tasks include perception, reasoning, and robustness.
- Similarity-based distractor filtering strategy (SDFS) used for multiple-choice questions.
- 52 representative models evaluated.
- Current VLMs achieve only moderate zero-shot performance.
- Existing benchmarks are dominated by common urban and rural imagery.
Entities
—