First Systematic Study of Inverse Relation Directionality in LLMs

A new paper on arXiv (2608.03512) presents the first systematic study of how large language models (LLMs) handle inverse relation directionality, where reversing argument order changes meaning (e.g., 'mother' vs. 'child'). The study introduces a benchmark of 5,457 instances across 27 inverse relation labels and evaluates five open-source LLMs using a multiple-choice prompting framework. The researchers also test the impact of relation descriptions and entity representations by substituting original entities with synthetic and masked entities. Findings reveal systematic asymmetries in inverse relation classification across models, and indicate that relation descriptions do not consistently improve performance. The work highlights a gap in LLMs' semantic understanding of directional relations, which is crucial for tasks like knowledge graph construction.