Sampling Luck Masquerades as Allocation Gain: Auditing Test-Time Budget Allocation for Neural Combinatorial Optimization
A new study released on arXiv (2608.13087) looks into a common method used in neural combinatorial optimization (NCO), where a set number of samples is assigned to each instance. The researchers wanted to see if changing the distribution of a fixed budget could actually enhance results. They discovered that for tasks within the same distribution, no real advantage was observed. Testing three pretrained solvers—POMO, AM, and SymNCO—on uniform TSP-100 revealed a 2.2-2.6% improvement with optimal allocation, but out-of-sample tests showed this was minimal (0.457, 0.015, -0.512 percent). The authors warn that sticking to traditional in-sample methods could misleadingly suggest a 2% benefit. You can check out the paper at https://arxiv.org/abs/2608.13087.
Key facts
- Paper on arXiv: 2608.13087
- Audits test-time budget allocation for neural combinatorial optimization
- Non-uniform allocation of fixed total budget is examined
- In-distribution workloads show no detectable allocation headroom
- Three pretrained solvers: POMO, AM, SymNCO
- Uniform TSP-100 benchmark
- Oracle allocation reports 2.2-2.6% gain in-sample
- Out-of-sample gain indistinguishable from zero (0.457, 0.015, -0.512 percent)
- Customary in-sample procedure would support a non-existent 2% gain
- Bias calibrated against instance-wise null with zero true gain
Entities
Institutions
- arXiv