Visual Fingerprints for Comparing LLM Outputs

A new arXiv preprint (2605.06054) introduces a method to visually compare outputs from large language models (LLMs) across different generation conditions—combinations of prompts, system instructions, model parameters, and architecture. The approach models text responses as collections of linguistic choices (content, expression, structure), extracted via natural language processing pipelines. These choices are represented as distributions across repeated samples, then visualized as "visual fingerprints" for direct, distribution-level comparison. This technique aims to aid prompt design and model evaluation by revealing condition-specific biases in LLM outputs.