RamseyGadgets: New Dataset Tests LLMs on Graph Construction

Researchers have introduced RamseyGadgets, a novel dataset of 70 underexplored graph construction problems designed to evaluate the reasoning capabilities of large language models (LLMs) in mathematics. The dataset focuses on finding Ramsey-good graphs—graphs that avoid specific monochromatic subgraphs—which are central to Ramsey theory, a branch of combinatorics. The motivation behind RamseyGadgets is the increasing use of generative AI in mathematical research, which raises the question of whether LLMs can genuinely reason about graph properties or merely recall known constructions from their training data. Many classic Ramsey-good graph problems have been extensively studied in the literature, making it difficult to distinguish between genuine reasoning and memorization. By selecting problems that are underexplored, the dataset aims to provide a cleaner test of LLM reasoning abilities. The dataset includes problems that require constructing graphs with special properties, such as containing an edge with a specific characteristic. This work is relevant to the intersection of artificial intelligence and mathematics, offering a benchmark for assessing AI's potential in combinatorial problem-solving. The dataset is described in a paper available on arXiv under the identifier 2608.14999, with the announcement type 'cross'. The research contributes to ongoing discussions about the capabilities and limitations of LLMs in scientific discovery.