Versioned Late Materialization for Ultra-Long Sequence Training in Recommendation Systems at Scale

A new paradigm called versioned late materialization addresses storage and I/O bottlenecks in training Deep Learning Recommendation Models (DLRMs) with ultra-long User Interaction History (UIH). The industry-standard 'Fat Row' approach pre-materializes sequences into every training example, causing data redundancy that strains infrastructure, especially in multi-tenant environments. The proposed system stores UIH once in a normalized, immutable tier and reconstructs sequences just-in-time during training using lightweight versioned pointers. It ensures Online-to-Offline (O2O) consistency via a bifurcated protocol that prevents future leakage across streaming and batch training, while a read-optimized design further enhances efficiency.