Agentic Memory Is a Dose to Calibrate, Not a Feature to Switch On: IBM Research
Recent findings from IBM Research, shared on Hugging Face, introduce the ALTK-Evolve technique, which provides AI agents with memory in a calibrated manner instead of as a simple binary attribute. This innovation enables agents to learn from their experiences without the need for weight updates or human input. In a review of eight models, robust models such as DeepSeek-V3.2 (671B MoE) saw a task completion increase of +9.5 points, whereas less powerful models like gpt-oss-120b (117B MoE) achieved a +16.1 points improvement with curated retrieval. However, saturated models like GLM-5 (745B MoE) did not show any enhancement. The research assessed memory configurations on AppWorld, a benchmark comprising 585 tasks. Future efforts will focus on creating a learned selector and expanding benchmarks. The ALTK-Evolve library and report are accessible for reproduction.
Key facts
- IBM Research's ALTK-Evolve lets agents learn from their own past trajectories by distilling reusable guidelines and injecting them at inference time, with no weight updates and no human annotation.
- Across eight models, agentic memory behaves as a dose to calibrate, not a feature to switch on.
- DeepSeek-V3.2 (671B MoE) gained +9.5 percentage points in TGC and +16.1 points in SGC when given its full self-mined guideline set.
- gpt-oss-120b (117B MoE) gained +16.1 points TGC with curated retrieval at only +5% tokens, while the full set cost ~50% more tokens.
- GLM-5 (745B MoE) showed no measurable gain, a pattern the authors call 'saturated'.
- GPT-5.5 and Opus each gained +7.2 and +7.1 points SGC respectively, showing memory pays off even near the ceiling.
- Evaluation used AppWorld's 585 multi-step tasks across 9 simulated apps, with TGC and stricter SGC metrics.
- Prompt caching can keep the full guideline set affordable because the static portion is cacheable across agent steps.
Entities
Institutions
- IBM Research
- DeepSeek
- OpenAI
- Anthropic
- Zhipu AI