Task Phrasing Shapes Presumptions in LLMs

A new study from arXiv investigates how the phrasing of tasks can induce presumptions in large language models (LLMs), impairing their adaptability when tasks deviate from those assumptions. Using the iterated prisoner's dilemma as a case study, the researchers found that LLMs are prone to presumptions even when employing reasoning steps. However, neutral phrasing enabled the models to demonstrate logical reasoning with fewer presumptions. The findings underscore the importance of careful task design to mitigate risks in real-world LLM applications.