LLMbench Workbench Enables Comparative Close Reading of Large Language Model Outputs

LLMbench is a web-based platform intended for the comparative close reading of outputs from large language models, distinguishing itself from quantitative assessment tools such as Google PAIR's LLM Comparator. It integrates into the practices of digital humanities hermeneutics by allowing users to view two model responses in adjacent, annotatable panels. Users can utilize four analytical overlays: Probabilities for examining token-level log-probabilities, Differences for word-level comparisons, Tone for analyzing Hyland-style metadiscourse, and Structure for parsing sentences with highlighted discourse connectives. The five analytical modes—Stochastic Variation, Temperature Gradient, Prompt Sensitivity, Token Probabilities, and Cross-Model Divergence—clarify the probabilistic nature of generated text. Announced on arXiv with identifier 2604.15508v1, the tool emphasizes interpretive analysis over mere performance metrics, merging computational linguistics with humanistic inquiry.