hardConceptMidSeniorStaff
What does LoRA change in the model, and when is parameter-efficient tuning the right choice?
4 min readUpdated 2026-08-15
Answer
Short answer
LoRA freezes the base weights and trains a small low-rank update added to chosen projections, so you train a fraction of the parameters and can swap adapters per use case.
Was this answer helpful?
Likely follow-ups
- Where in the network would you apply the adapters, and what would you try if results were weak?
- You merge the adapter into the base weights for serving. What do you give up?
- How does QLoRA differ, and what does the quantised base cost you?
- Your adapter improves the target task and degrades instruction following. What is happening and what do you change?
Related questions
- How do you decide between prompting, retrieval and fine-tuning?mediumAlso on fine-tuning5 min
- You are building a support assistant over a knowledge base that changes every week. RAG or fine-tuning, and how do you decide?mediumAlso on fine-tuning5 min
- You have fine-tuned a model for a task. How do you prove it helped?hardAlso on fine-tuning5 min
- You have 50 ms for a model call in a request path. How do you make that budget?hardAlso on quantisation5 min