In-Context Examples Suppress LLM Scientific Knowledge Recall

A recent investigation indicates that incorporating in-context examples into large language models (LLMs) diminishes their capacity to remember and utilize pretrained scientific information, leading them to focus more on empirical pattern recognition. This study, recorded in arXiv:2604.27540, evaluated 60 latent structure recovery tasks spanning five scientific fields, involving 6,000 trials and four distinct models. The tasks included estimating reaction constants in chemistry and deducing demand elasticities in economics. While the displacement effect was observed consistently across various domains, the implications for accuracy differed. These results call into question prevailing beliefs regarding how LLMs utilize domain-specific knowledge when examples are presented.