HyMOR Framework Combines MLLM and CLIP for Multi-Granularity Object Recognition in Educational Games

A new hybrid framework called HyMOR integrates Multimodal Large Language Models with CLIP-style models to address limitations in open-ended object recognition. While MLLMs handle broad category identification, CLIP models specialize in fine-grained recognition of domain-specific objects like animals and plants. This approach enables accurate understanding across multiple semantic granularities, creating a robust perceptual foundation for downstream applications. The framework specifically targets multi-modal content generation and interactive gameplay in educational scenarios. To support evaluation in content-rich environments, the researchers introduced TBO. The work addresses the gap between MLLMs' open-ended capabilities and CLIP models' fine-grained strengths, proposing a practical solution for interactive educational games requiring both coarse and fine object recognition.