DeepSeek's DSpark Speeds AI Inference by 85%, Cuts Chip Strain

Chinese AI start-up DeepSeek has unveiled DSpark, a speculative decoding framework that accelerates AI inference by up to 85%, addressing bottlenecks in serving large models. The framework uses a lightweight draft model to propose candidate responses, which are then verified in batches by a larger model, reducing token-by-token delays that lead to low GPU utilization and high user wait times. DSpark also employs a semi-autoregressive generation method to produce small token chunks instead of single tokens, and a confidence-based scheduling system that dynamically adjusts verification based on computing demand. The announcement comes amid China's push to overcome US AI curbs on advanced chips. DeepSeek published the research on Saturday.