Training-Free Methods for Personalized Toxicity Sensitivity in Language Models

A recent study offers the initial comparative analysis of training-free techniques aimed at aligning language model outputs with individual sensitivity to toxicity. This research, available on arXiv, evaluates interventions at three stages during inference: pre-decoding (prompt conditioning and rewriting), in-decoding (steering of tokens, logits, and representations), and post-decoding (re-ranking candidates). Utilizing toxicity sensitivity benchmarks from the PRISM dataset, all approaches achieve a reduction in alignment error ranging from 28% to 47%. Nonetheless, findings indicate a crucial trade-off among alignment efficiency, personalization, and overall language quality, positioning toxicity sensitivity alignment as a multi-faceted challenge. The study questions the existence of a universal standard for toxicity, highlighting the subjective and context-specific nature of harmful language perceptions.