China's AI developers stick with Nvidia chips due to high transition costs
Despite progress in domestic hardware, Chinese AI developers continue to depend on Nvidia chips for training large language models (LLMs) due to significant transition costs and engineering challenges. An industry insider revealed that using Nvidia chips for LLM training is still standard practice among these developers. The primary obstacle lies in the software ecosystem: Nvidia's Compute Unified Device Architecture (CUDA) is the prevailing industry standard, whereas Huawei Technologies' Compute Architecture for Neural Networks (CANN) necessitates extensive code rewriting and optimization. James Wang, an AI model developer at a Shanghai university-affiliated research institute, emphasized that current training processes are built around CUDA, and adapting them for Huawei's Ascend chips could increase time and costs by at least 50%. This dependence on Nvidia continues, even amid U.S. export restrictions and China's ambitions for semiconductor independence.
Key facts
- Chinese AI developers still use Nvidia chips for training LLMs.
- High transition costs are the main reason for the reliance.
- Domestic hardware is advancing but chip architecture changes pose engineering bottlenecks.
- Nvidia's CUDA platform is the industry standard for AI development.
- Huawei's CANN requires developers to rewrite and optimise code.
- James Wang, an AI researcher, says CUDA code cannot run directly on Ascend.
- Migrating to Ascend could add at least 50% in time and costs.
- The information comes from industry sources and an AI researcher.
Entities
Institutions
- Nvidia
- Huawei Technologies
- Shanghai-based university
Locations
- China
- Shanghai