Meta Releases Muse Glimmer: Open-Source Multimodal AI Model
Meta has released Muse Glimmer, a dense 30B parameter multimodal AI model, now available on the Hugging Face Hub with day-0 support in transformers, llama.cpp, vLLM, and Inference Endpoints. The model combines a 2B ViT-style Perception Encoder for vision with a 28B text decoder, featuring hybrid attention with sliding window and full attention layers, Gated Grouped-Query Attention, and Q-K normalization. It supports text-only inference, image and video understanding, multimodal tool calling, and object detection. Muse Glimmer also includes an optional DFlash speculative decoding drafter for faster generation, particularly for structured content like coding. The release includes support for fine-tuning with TRL, and demonstrations show the model can quantize itself, deploy itself to inference endpoints, and optimize its own inference engine for specific hardware. The model is part of Meta's push for local, agentic AI, and is available for use with coding agents like OpenClaw.
Key facts
- Muse Glimmer is a dense 30B parameter model with a 2B ViT-style Perception Encoder and a 28B text decoder.
- It uses hybrid attention: alternating sliding window layers (2,048 tokens) and full attention layers, repeated 13 times.
- Gated Grouped-Query Attention reduces KV-cache memory by 16x.
- Q-K normalization with extra query scaling stabilizes attention logits.
- The Perception Encoder handles both images and videos, patchifying to 2 frames x 3 channels x 14 x 14.
- Videos are processed frame by frame, targeting 2 frames per second and capping at 96 frames.
- Day-0 support in transformers, llama.cpp, vLLM, and Inference Endpoints.
- Optional DFlash speculative decoding drafter speeds up generation, especially for coding.
- Fine-tuning examples with TRL on MolmoWeb dataset and OpenCode with AsyncGRPO.
- Muse Glimmer can quantize itself, deploy itself to Inference Endpoints, and optimize its own inference engine.
- OpenClaw can be connected to Muse Glimmer via an OpenAI-compatible API.
- The model is available on the Hugging Face Hub.
Entities
Institutions
- Meta
- Hugging Face
- transformers
- llama.cpp
- vLLM
- Inference Endpoints
- TRL
- OpenClaw
- DFlash
- OpenCode
- OpenEnv
- MolmoWeb
- VideoMME2
- Hugging Face MCP
- Gradio Space