Constraint-Aware GPU Allocator Boosts Utilization by 33 Points

Dharma AI has developed a constraint-aware GPU allocator that significantly improves GPU utilization and priority-weighted output compared to a FIFO scheduler. In benchmark tests across seven scenarios, the allocator increased GPU utilization by up to 33 percentage points and priority-weighted output by up to 105%, with an average improvement of 52%. The allocator treats real-time inference demand as a dynamic curve rather than a static reservation, allowing batch jobs to use GPUs during troughs. It also places batch-like jobs by priority across the entire scheduling horizon, rather than in arrival order. The system uses a formal model with five constraints and a two-term objective function, balancing batch rewards against real-time penalties. It runs in 1-2 milliseconds on contended scenarios and 15 milliseconds for 64 GPUs and 30 jobs. The allocator is complemented by specialized forecasters for training, quantization, and real-time inference, which improve demand predictions. The scheduler optimizes a 24-hour horizon but commits only the current timestep, re-running every 30-60 minutes to absorb forecast errors. This approach, inspired by airline operational discipline, demonstrates that structure and ordering can yield significant gains without hardware changes.