LoRDBA: Binary Adapter for On-Device LLM Fine-Tuning

A new method called LoRDBA (Low-Rank Double-Binary Adaptation) enables efficient fine-tuning of large language models on edge devices. It replaces the two low-rank matrices in a standard LoRA adapter with binary sign matrices and lightweight channel-wise scaling factors, converting the adapter's forward pass into sign-accumulation multiplications. This eliminates the need for a dense floating-point branch, reducing memory and computation. Theoretical analysis shows reconstruction quality depends on the residual-to-magnitude ratio of original LoRA factors. Experiments demonstrate LoRDBA outperforms low-bit baselines at matched model sizes in adapter-mode settings.