Consensus Decoding Mitigates Hidden LLM Behaviors from Fine-Tuning
A recent research paper published on arXiv (2607.23394) introduces methods for consensus during inference to mitigate the hidden behaviors that arise from fine-tuning language models. The researchers reveal that fine-tuning with compromised data can lead to specific misbehaviors, while even neutral data can harbor concealed biases. Traditional defenses, such as data filtering and regularization, fall short. Their methodology involves redundancy: they gather multiple datasets from various origins and fine-tune a distinct reference model for each. During decoding, next-token distributions are combined using two consensus decoders: one that selects the minimum probability for each token across sources and another that defaults to the base model's probability in case of discrepancies. This effectively prevents misbehavior, even when only some sources are malicious.
Key facts
- arXiv paper 2607.23394 proposes consensus decoding for LLM fine-tuning safety
- Fine-tuning on poisoned data installs targeted misbehavior
- Benign data can transmit hidden preferences
- Standard defenses attenuate but do not eliminate effects
- Redundancy via multiple datasets from different sources
- Separate reference model fine-tuned on each source's dataset
- Token-wise minimum decoder caps each token at lowest probability
- Base-relative variant reverts to base probability on disagreement
Entities
Institutions
- arXiv