Frictive Policy Optimization: A New Framework for LLM Alignment

A new framework called Frictive Policy Optimization (FPO) has been introduced by researchers to train language models in managing both epistemic and normative risks through specific control actions such as clarification, verification, challenge, redirection, and refusal. Unlike traditional alignment techniques that focus on optimizing superficial preferences or task utility, FPO approaches alignment as a risk-sensitive epistemic control challenge, prioritizing interventions based on their anticipated impact on downstream epistemic quality rather than immediate rewards. This framework presents a classification of frictive interventions, a structured friction functional to address alignment failure modes, and a cohesive set of FPO methods, including reward shaping and preference pairing. The study can be found on arXiv with the identifier 2604.25136.