Agent Memory Distillation: Training-Free Framework Boosts Small LLM Agents

A recent paper on arXiv (2608.07169) presents Agent Memory Distillation (AMD), a framework that operates without training to facilitate the transfer of structured knowledge from large teacher agents to smaller student agents through hierarchical memory. This approach creates three types of complementary memory derived from effective teacher trajectories: Workflow memory for strategies at the task level, Subtask memory for specific behavioral examples, and Function memory for conventions and common pitfalls related to function calls. Workflow and Subtask memories are proactively integrated at the beginning of tasks, while Function memory is accessed reactively when tool-calling errors occur. Testing on three benchmarks with four student models (4B-8B parameters) demonstrates notable performance gains, allowing smaller models to manage complex tasks without the need for fine-tuning. This framework effectively tackles the issue of small language models struggling to generate enough successful trajectories independently, providing a viable method for deploying efficient agents in environments with limited resources.