Interactive Model Bending Makes Diffusion Models Creative Materials

A recent study published on arXiv suggests that the focus of explainable AI (XAI) in artistic endeavors should be on allowing artists to examine, alter, and troubleshoot models, rather than emphasizing technical explanations. The researchers contend that large text-to-image diffusion systems often operate as opaque, end-to-end tools but can serve as creative resources when their internal workings are made accessible and adjustable. This methodology is exemplified through a model bending and interactive inspection interface incorporated into ComfyUI's node-based system, featuring controls for interactive layer selection and intervention. Analysis of bending interventions in Stable Diffusion 1.5 reveals that adjusting particular elements of the diffusion pipeline yields fairly consistent output variations.