C4: A New Framework for Evaluating Creative Leap in Multimodal LLMs
A new evaluation framework called C4 has been developed by researchers to measure the creative abilities of multimodal large language models (MLLMs) through the understanding of cross-concepts. This framework, outlined in a paper on arXiv (2608.06501), tackles the important issue of assessing creativity in AI, which is essential for fields such as design, communication, education, and collaboration with humans. Cross-concept understanding is recognized as a fundamental cognitive skill that supports receptive creativity, allowing individuals to discern intended meanings from subtle conceptual connections. C4 operationalizes item creation as cross-concept encoding and model inference as cross-concept decoding. Drawing on Chengyu (Chinese idioms), it features an encoding element that links target slots to visual substitute concepts within a carefully reviewed cross-concept network, facilitating structured batch generation with indexed difficulty and precise answers. The goal of this framework is to offer a reliable method for evaluating creative advancements in AI, potentially leading to more innovative AI systems.
Key facts
- C4 is a new evaluation framework for cross-concept creativity in MLLMs.
- It is based on Chengyu (Chinese idioms).
- The framework was announced on arXiv with ID 2608.06501.
- Cross-concept understanding is core to receptive creativity.
- The encoding component uses a manually annotated cross-concept network.
- Difficulty is indexed by bridge count and depth.
- The framework enables batch generation with exact answers.
- It addresses the scarcity of explicit targets in creativity evaluation.
Entities
Institutions
- arXiv