LLMs Fail at Generating Random Numbers from Statistical Distributions

A recent investigation published on arXiv (2601.05414) indicates that large language models (LLMs) face difficulties in producing random numbers from designated probability distributions. The researchers evaluated 11 leading models across 15 different distributions utilizing a dual-protocol approach: Batch Generation (N=1000 samples in a single response) and Independent Requests (N=1000 stateless calls). The findings reveal a significant disparity: the median pass rate for batch generation was merely 7%, whereas independent requests nearly failed entirely, with 10 out of 11 models not succeeding in any distributions. This research underscores a vital functional necessity as LLMs are incorporated into stochastic systems and pipelines aiming for general intelligence.