Open-Weight LLMs Show Promise for Combinatorial Optimization Feature Extraction and Algorithm Selection
A recent preprint on arXiv (2512.13374) explores the capacity of open-weight Large Language Models (LLMs) to acquire reusable representations for combinatorial optimization problems. This updated study does not intend to introduce a new algorithm or replace precise feature extractors; instead, it evaluates the viability of utilizing frozen LLM representations for tasks such as feature recovery and algorithm selection. The approach involves direct querying to analyze explicit feature extraction alongside probing analyses to uncover whether this information is inherently present in hidden layers. The probing framework is adapted for a per-instance algorithm selection challenge. The experiments cover four benchmark problems, three types of instance representations, and five open-weight models ranging from 3B to 120B parameters. Results indicate that these representations can enhance downstream decision-making tasks, providing fresh insights into optimization automation.
Key facts
- The paper is arXiv:2512.13374v2, an updated version.
- It investigates whether open-weight LLMs can capture problem structure or algorithmic behavior.
- The goal is to assess reusability of representations for feature recovery and algorithm selection.
- Methodology includes direct querying and probing analyses of hidden layers.
- Experiments cover four benchmark problems and three instance representations.
- Five open-weight models from 3B to 120B parameters were tested.
- The study does not aim to replace exact feature extractors or propose a new algorithm.
- The findings indicate representations are reusable for downstream decision tasks.
Entities
Institutions
- arXiv