DeepSeek's Memory-Saving AI Linked to Reliability Gaps in New Research
Unreviewed research has identified a potential flaw in the memory-saving technology behind DeepSeek's models, challenging a breakthrough that helped the Chinese AI lab narrow the distance to US rivals OpenAI and Anthropic through its low-cost approach. According to the study, DeepSeek's push to cut the memory required for processing long inputs may sacrifice reliability, since the models were more likely to overlook identical information depending on where it appeared in a document. The paper states the phenomenon shows up in DeepSeek-V4, V4.1 and every compressed model the researchers pretrained. One illustration involved inserting irrelevant text ahead of a piece of computer code, which degraded DeepSeek's capacity to select the right answer. The findings have not been peer-reviewed.
Key facts
- New research has exposed a potential flaw in the memory-saving technology underpinning DeepSeek's models.
- The study has not been peer-reviewed.
- DeepSeek's efforts to reduce memory needed for lengthy inputs may come at the expense of reliability.
- Models were more likely to miss the same information when it appeared in certain positions in a document.
- The paper said the phenomenon appears in DeepSeek-V4, V4.1, and all compressed models the researchers pretrained.
- Adding irrelevant text before the same piece of computer code impaired DeepSeek's ability to choose the correct answer.
- The research marks a rare challenge to a breakthrough that helped the Chinese AI lab close the gap with US rivals OpenAI and Anthropic.
- DeepSeek's low-cost approach helped it narrow the distance to OpenAI and Anthropic.
Entities
Institutions
- DeepSeek
- OpenAI
- Anthropic
- South China Morning Post
Locations
- China
- United States