Study analyzes how fine-tuning strategies affect LLM interpretation in automated code compliance

A new study examines how different fine-tuning approaches influence large language models' interpretive behaviors in automated code compliance tasks, moving beyond traditional performance-focused research. Researchers employed perturbation-based attribution analysis to compare full fine-tuning (FFT), low-rank adaptation (LoRA), and quantized LoRA methods across varying model scales. The findings reveal that FFT produces attribution patterns that are statistically distinct and more concentrated than those from parameter-efficient fine-tuning techniques. As model size increases, LLMs develop specific interpretive strategies, such as prioritizing numerical constraints and rule identifiers within building text. This research addresses a significant gap in understanding how training decisions affect model behavior rather than treating LLMs as black boxes. The study was published as arXiv:2604.15589v1 with an announcement type of cross. The work demonstrates that performance improvements with larger models correlate with more sophisticated interpretive approaches. This analysis provides insights into the internal mechanisms of LLMs when applied to technical compliance tasks.