MinT: Managed Infrastructure for LLM Training and Serving
The MindLab Toolkit (MinT) serves as a managed infrastructure for Low-Rank Adaptation (LoRA), facilitating post-training and online deployment of large language models. It is particularly suited for scenarios where numerous trained policies emerge from a limited number of costly base-model deployments. Rather than creating a complete merged checkpoint for each policy, MinT retains the base model and processes exported LoRA adapter revisions through various stages, including rollout, update, export, evaluation, serving, and rollback, effectively concealing distributed training and data movement behind a service interface. MinT offers scalability in three dimensions: Scale Up enhances LoRA RL for advanced dense and MoE architectures, validated for over 1 trillion parameters; Scale Down exports LoRA adapters, potentially under 1% of the base model size, achieving an 18.3x reduction in measured steps for a 4B model; and Scale Out oversees multiple base-model deployments and thousands of adapters. The system is optimized for managing millions of LLMs efficiently.
Key facts
- MinT is a managed infrastructure for LoRA post-training and online serving.
- It targets settings with many trained policies over few expensive base-model deployments.
- MinT keeps the base model resident and moves LoRA adapter revisions through a lifecycle.
- Scale Up extends LoRA RL to frontier-scale dense and MoE architectures.
- Scale Down moves only the exported LoRA adapter, under 1% of base-model size in rank-1 settings.
- Adapter-only handoff reduces measured step by 18.3x on a 4B model.
- Scale Out manages multiple base-model deployments and thousands of adapters.
- Training and serving validated beyond 1T total parameters.
Entities
—