LoRA Rank and Quantization Trade-offs on a 60M-Parameter Text-to-SQL Model
A controlled study on T5-small (60M parameters) and WikiSQL benchmark examines how LoRA rank, target modules, and quantization affect task accuracy and system metrics. LoRA with r=16 recovers within 11.6 percentage points of full fine-tuning.
Key facts
- Study uses T5-small (60M parameters) and WikiSQL benchmark
- LoRA rank values tested: 2, 4, 8, 16, 32
- Reports task accuracy, trainable parameters, peak training memory, inference latency, throughput
- LoRA with r=16 recovers within 11.6 percentage points
- Frames adaptation as a constrained trade-off
Entities
—