What are the benefits of LoRA?

• A. Longer training time with minimal gains.
• B. Higher memory usage with no latency change.
• C. Requires more compute and larger models.
• D. Faster training, lower memory usage, and smaller model weights.

Answer: (D)

LoRA, or Low-Rank Adaptation, tunes large models by freezing the original weights and adding small trainable low-rank matrices in each layer. This means only a tiny portion of parameters are updated during fine-tuning, so the base model stays intact while the new matrices learn the task. Because the number of trainable parameters is much smaller, training is faster and uses less memory for gradients and optimization. When saving the fine-tuned model, you only need to store those small matrices in addition to the base model, leading to a smaller final model size. In short, LoRA enables efficient fine-tuning: faster training, lower memory usage, and smaller model weights. The other statements describe scenarios that would require more compute, more memory, or larger models, which isn’t how LoRA operates.