Visual Text Compression as Measure Transport

A recent preprint on arXiv introduces a novel approach for assessing visual text compression (VTC), likening it to measure transport concepts. The method transforms text into image formats suitable for vision-language models, enabling a token reduction between three and twenty times when compared to traditional subword tokenization. Despite this decrease, reduced tokens do not always enhance subsequent task performance. The researchers characterize text and visual tokens as probability measures, demonstrating how the ViT patch encoder generates a push-forward mapping. This framework evaluates information loss, correlating compression effectiveness to practical applications in the field.