LoRA Logit Shift Formalised via Fréchet Approximation
A technical note on arXiv formalises the logit shift induced by Low-Rank Adaptation (LoRA) using a first-order Fréchet approximation. The multi-layer LoRA effect is decomposed into a linear sum of layerwise contributions plus a higher-order remainder for inter-layer coupling.
Key facts
- First-order formalisation of logit shift from LoRA
- Uses first-order Fréchet approximation around base model trajectory
- Multi-layer LoRA effect decomposed into linear sum of layerwise contributions
- Higher-order remainder term represents inter-layer coupling
- Published on arXiv under Computer Science > Machine Learning
- arXiv ID: 2604.20313
Entities
Institutions
- arXiv