During training, what is the typical requirement for the base model when using LoRA?

• A. All base model parameters are updated during training.
• B. The base model is frozen and only the adapter weights are trained.
• C. The base model is discarded and rebuilt.
• D. The base model architecture is entirely replaced with a different model.

Answer: (B)

In LoRA, you keep the pre-trained base model unchanged during training and learn only small, trainable adapters that are added to the network. The idea is to modify the model’s behavior with a tiny, low-rank adjustment to the existing weights, rather than rewriting or updating the whole model. Practically, the base weights remain fixed, while the adapter components (often arranged as low-rank matrices) are trained to produce the task-specific changes. This preserves the knowledge embedded in the base model and makes fine-tuning much more parameter-efficient.

If you were to update all base parameters, you’d lose the main benefit of LoRA—updating only a small fraction of parameters. Discarding or replacing the base model or its architecture would negate the purpose of leveraging a pre-trained foundation, which is exactly what LoRA is designed to do by keeping the base fixed and training the adapters.