OmniLens: Scalable Lens Method for Interpreting LLMs at Any Width

A new preprint on arXiv (2608.10260) introduces OmniLens, a lens method for interpreting large language models (LLMs) that scales to models of any width and component type. Traditional trained lenses are limited by quadratic parameter growth and high memory costs, restricting them to models up to 20B parameters. OmniLens overcomes these limitations with two scaling techniques: low-rank translators that reduce trainable parameters by up to 98.4%, and Subset-KL, which materializes only selected vocabulary logits, cutting peak training memory by up to 70% in its Top-k mode. The method applies a single lens family to residual stream, attention, and MLP activations, enabling interpretation of larger models. The paper was announced as a new arXiv submission and is available at the provided URL.