What is true about inference latency when using LoRA?

• A. Inference latency remains unaffected because adapter weights can be merged with the base model.
• B. Inference latency increases due to extra computations.
• C. Inference latency is doubled by additional parameters.
• D. Inference latency drastically decreases beyond baseline.

Answer: (A)

The idea being tested is that LoRA can keep inference latency essentially the same by merging the adapter updates into the base model. LoRA introduces small, low-rank updates to the original weight matrices to enable efficient fine-tuning with far fewer parameters. When you fuse these updates into the base weights for deployment, the model effectively becomes a single weight matrix, so the inference path is the same as the original model’s, with no extra multiplies or layers added at runtime. This is why latency can remain unaffected. If you didn’t merge the adapters, you would incur extra computations from the additional parameters, but proper fusion lets the deployment maintain latency comparable to the baseline.