Deterministic Executability Gating for LLM Skill Selection at Scale

An arXiv paper (2608.01050) introduces a three-stage selection pipeline implemented for Helpmate, the customer-care assistant from Wix, which tackles the issue that semantic relevance alone is insufficient to assess if a skill can be executed based on the current account state. Initially, a recall-focused semantic matcher identifies messages pertinent to a ten-skill domain family without referencing the account state. Next, a deterministic executability gate filters out candidates that cannot proceed due to internal hard-stop conditions, ensuring that any blocked candidate remains non-executable under the same account state, assuming predicate parity is maintained and both checks reflect the latest authoritative state. Lastly, the LLM determines whether to activate one of the remaining candidates. The paper also features an analysis of 756.6K user messages from 267.6K conversations post-launch.