SkillWeave: Modular Skillpacks for Efficient LLM Specialization

A team of researchers has unveiled SkillWeave, a modular enhancement framework designed to allow large language models (LLMs) to excel in various domains without incurring additional memory or inference expenses. SkillWeave breaks down the capabilities of a general-purpose model into lightweight, domain-specific delta modules known as skillpacks, which enhance and reorganize internal knowledge. To facilitate efficient deployment, it incorporates SkillZip, which compresses skillpacks into a compact format suitable for inference, delivering impressive multi-domain performance with minimal latency. In evaluations involving multi-task and agentic benchmarks, a SkillWeave model with 9 billion parameters outperformed multiple baselines and even exceeded the performance of a 32 billion parameter monolithic LLM, achieving speed improvements of up to four times. This development tackles the challenge of maintaining multi-domain functionality while adhering to strict memory and inference limits.