VaLiDRec: Variable-Length LLM-Aligned Semantic IDs for Generative Recommendation

VaLiDRec, a novel generative recommendation framework, introduces the use of variable-length semantic identifiers (SIDs) that are directly aligned with the vocabularies of pretrained LLMs, bypassing the need for fixed-length clustering and quantization. This approach generates SIDs from native LLM tokens through methods such as token importance estimation, semantic-quality-aware pruning, and collision-aware refinement, tailoring the identifier length based on the complexity of items. The modeling of user preferences employs graph-aware soft prompts and redefines recommendations as token-set predictions with scoring at the token level, thus eliminating the need for autoregressive decoding and beam search. Experiments conducted on four real-world datasets validate its effectiveness.