Amplitude Gating: Non-Destructive FFN Intervention for Tool-Structured LLM Inference

A recent paper on arXiv (2607.11183v2) introduces Amplitude Gating (AG), an innovative method for enhancing structured outputs in feed-forward networks (FFNs) of large language models (LLMs) during inference without the need for weight retraining. This study evolved from Orthogonal Residual Projection (ORP), which aimed to correct outputs by altering weight directions but often led to detrimental effects, highlighting sensitive intervention areas in SwiGLU FFNs. In contrast, AG maintains the original weight directions of FFNs and adjusts only the activation magnitudes during output generation. The authors establish a detailed intervention framework that includes P1/P2/P3 and specific branch sites. They also propose an evaluation method that distinguishes between combination-oracle headroom and fixed configurations, ensuring sample-level accountability with task-aware metrics. Although the paper provides empirical results across models, it does not specify particular models or performance metrics in the abstract. This research tackles the issue of minor errors in tool-using agents, which can undermine otherwise credible outputs. Its non-destructive approach may present a safer alternative to modifying weights. The full paper can be accessed on arXiv with the identifier 2607.11183.