Should you fine-tune? Usually not first

Fine-tuning sounds like the serious option. Often it is the expensive one. Post-training mainly changes how a model behaves: its format, tone, and habits. It is a poor way to teach facts that change, such as this term's deadlines. Retrieval handles those better and can cite its sources.

Start with prompting and measure. Write down the requirement, for example "every answer cites a lesson ID", and test it on fifty held-out prompts. If careful prompting reaches it, you are done. If not, fine-tuning may be worth it, and methods such as LoRA make it far cheaper than updating every weight.

Most of the work is in the data. A model learns whatever your examples teach, including the mistakes. Preference data has subtler problems. Annotators disagree. Prompts leak from training into evaluation. And labels often reward length: in our small teaching fixture, always choosing the longer answer matches 87.5 percent of the labels. Research on RLHF has found length explaining much of the measured improvement in some settings.

Before training, audit for duplicates, conflicts, leakage, and length bias. After training, compare the same held-out prompts before and after, covering the target task, the abilities you want to keep, and safety behaviour. Fine-tuning can quietly weaken safety training learned earlier.

Post-Training: Fine-Tuning, RLHF & DPO works through these decisions with an offline lab, so you can practise the audit before renting a GPU.