Dream Scene Visualiser: AI Turns Written Dreams into Coherent Image Sequences
The Dream Scene Visualiser (DSV) has been created by researchers to transform written accounts of dreams into a sequence of four images. Utilizing a large language model, the system breaks down a dream description into four sequential segments, subsequently applying a text-to-image model to produce corresponding images that retain visual consistency throughout. If any image fails to align well with the text, DSV regenerates it. The system underwent assessment using over 50 visualizations derived from DreamBank dream descriptions, with quality and coherence evaluated through objective metrics using CLIP, DINOv2, and Qwen2-VL models. This research is detailed in a paper titled 'Coherence-Oriented Dream Scene Visualisation,' available on arXiv in the Computer Science > Artificial Intelligence section.
Key facts
- The Dream Scene Visualiser (DSV) turns written dream descriptions into a temporal sequence of four panel images.
- A large language model splits dream descriptions into four chronological parts.
- A text-to-image model generates images for each part with visual coherence maintained across the sequence.
- DSV regenerates any image not suitably matching the text.
- Evaluation was conducted over 50 visualisations from dream descriptions in DreamBank.
- Objective measures employed CLIP, DINOv2, and Qwen2-VL vision-language models.
- The paper is titled 'Coherence-Oriented Dream Scene Visualisation'.
- The paper is available on arXiv under Computer Science > Artificial Intelligence.
Entities
Institutions
- arXiv
- DreamBank