JouleShare: A Framework for Request-Level Energy Attribution in Batched LLM Serving
A recent paper on arXiv presents JouleShare, a framework designed to assign energy usage to specific requests in the context of batched large language model (LLM) serving. This study tackles the issue of GPU power telemetry being aggregate, while accurate sustainability reporting, chargeback, and workload analysis necessitate energy charges at the request level. Current benchmarks offer energy data at the model, phase, or token level but fail to provide verified request-level measurements. JouleShare consists of two main parts: an offline harness that establishes ground truth by replaying subsets of requests under vLLM with a consistent protocol, and JCalib, a lightweight calibration model that predicts Shapley shares from inexpensive request features during serving. The paper can be found on arXiv under ID 2608.00026.
Key facts
- JouleShare is an attribution framework for request-level energy in batched LLM serving.
- It includes an offline harness to compute exact Shapley energy for each request.
- JCalib is a lightweight calibration model that predicts Shapley shares from request features.
- The framework integrates GPU power telemetry and uses vLLM for replaying request subsets.
- The paper is published on arXiv with ID 2608.00026.
- The research addresses the gap between aggregate GPU power telemetry and request-level energy accounting.
- Existing benchmarks provide model-, phase-, or token-level energy, but not request-level ground truth.
- The work is motivated by sustainability reporting, chargeback, and workload analysis needs.
Entities
Institutions
- arXiv
- vLLM