Cuttlefish: Scaling-Aware Adapter for LLM Reasoning on 2D/3D Structures
A new large language model (LLM) called Cuttlefish has been developed by researchers to integrate language reasoning with geometric information from both 2D and 3D structures. Traditional approaches tend to be specific to certain modalities and frequently reduce structural inputs through sequence-based tokenization or fixed-length query connectors, resulting in structural inaccuracies or rigid fusion limitations. Cuttlefish overcomes these challenges through its Scaling-Aware Patching mechanism, which employs an instruction-conditioned gating system to create variable-size patches on structural graphs, allowing the query token budget to adjust based on structural complexity. This method seeks to enhance all-atom reasoning. The research is published on arXiv with the ID 2602.02780.
Key facts
- Cuttlefish is a unified multimodal LLM for reasoning over 2D and 3D structures.
- It uses Scaling-Aware Patching with an instruction-conditioned gating mechanism.
- The method generates variable-size patches over structural graphs.
- It adaptively scales query token budget with structural complexity.
- Existing methods are modality-specific and suffer from structural hallucinations.
- Fixed-length query connectors cause over-compression and suboptimal token allocation.
- The paper is on arXiv with ID 2602.02780.
- Cuttlefish aims for generalized all-atom reasoning.
Entities
Institutions
- arXiv