FLAME System Introduced for Mobile Edge AI Latency Estimation Amid DVFS Challenges

A novel system named FLAME has been introduced to tackle the issue of accurately estimating inference latency for mobile edge applications that are time-sensitive. Static profiling techniques often fall short due to Dynamic Voltage and Frequency Scaling (DVFS), which leads to variations in inference latency as CPU and GPU frequencies change. Although comprehensive profiling across all frequency combinations is theoretically feasible, it becomes excessively costly, particularly for new Small Language Models (SLMs) where the variable context lengths can result in profiling durations of several days. Basic analytical scaling methods are inadequate in forecasting these variations due to the intricate asynchronous interactions between CPU kernel launches and GPU operations. Accurate latency estimation is vital for mobile edge devices to determine latency margins relative to deadlines, allowing for a trade-off between improved model performance and resource efficiency. This research was published on arXiv with the identifier 2604.15357v1 and has a cross announcement type.