Open-World Hierarchical Perception for Safe Handling of Out-of-Vocabulary Road Objects

An arXiv paper (2608.07577) introduces a hierarchical perception system designed for autonomous vehicles that can effectively manage road objects not included in its training vocabulary. This innovative approach replaces the conventional flat label structure with a hierarchical taxonomy and a runtime abstraction rule. Unlike previous methods, it applies taxonomic abstraction to class-agnostic region proposals, enabling the classification or identification of objects that a closed detector would overlook. The study explores three open-world signals: class-agnostic segmentation, appearance-based out-of-distribution scoring, and monocular depth, demonstrating the necessity of combining these cues. Additionally, it performs a ground-truth leave-classes-out evaluation, addressing the limitations of closed-set detectors that struggle with labeling diverse entities like horse-drawn carriages and rural livestock.