Meta-Tool: Hypernetwork LoRA Fails to Improve Few-Shot Tool Use in 3B LLM

A new study from arXiv (2604.20148) finds that hypernetwork-based LoRA adaptation provides no measurable improvement over few-shot prompting for small language models in tool-use tasks. Using a Llama-3.2-3B-Instruct backbone, researchers evaluated four adaptation mechanisms—few-shot prompting, documentation encoding, hypernetwork-generated LoRA weights, and value-guided beam search—across four benchmarks: Gorilla APIBench, Spider 2.0, WebArena, and InterCode. The 227.8M-parameter hypernetwork added 0% performance gain, while few-shot examples contributed +21.5% and documentation +5.0%. The negative result suggests that careful prompting alone suffices for 3B-scale models.