Chinese AI Firms Struggle with Chip Shortage as Inference Demand Surges

Chinese artificial intelligence firms are encountering significant computing limitations as they shift from developing models to large-scale implementation. Domestic chips are inadequate for complex tasks such as coding. Although efforts have been made to optimize inference workloads for local hardware, the generation of high-quality tokens still depends on Nvidia processors, which are in short supply due to export limitations. Guan Jiawei, vice-president of the inference optimization startup Approaching.AI, pointed out a 'bipolarisation' in demand, with a vast disparity between the need for high-quality tokens and their availability. Domestic processors can only manage lower-quality outputs, hindering commercial viability. The situation is worsened by a dramatic increase in token usage, with China's daily token calls surpassing 140 trillion in March, a rise of over 1,000-fold since early 2024, as reported by the National Data Administration.