LLM Truth Representations Shift with Context

A recent study published on arXiv (2601.06599) explores the varying ways Large Language Models (LLMs) encode truth when additional context is introduced. The researchers analyzed changes in direction and the magnitude of truth vectors across four different LLMs and datasets. Their findings indicate that in the initial layers, truth vectors are orthogonal, converge in the middle layers, and might either stabilize or increase in the later layers. The inclusion of context typically enhances the distinction between true and false representations. Moreover, larger models primarily differentiate between relevant and irrelevant context through directional alterations.