LoRA/QLoRA fine-tune

What you'll build

Step-by-step build

1. 1

   Pick one task and define success

   Choose a narrow, well-defined task with a consistent output format, for example turning support emails into a fixed JSON ticket. Write down exactly what a correct output looks like before you touch any code. This definition becomes your test set and your grading rule.

2. 2

   Build the dataset

   Collect 500 to 2,000 input/output pairs and format them as instruction, input, and response. Split off 10 to 15 percent as a held-out test set the model will never train on. Clean, consistent examples matter far more than raw volume, so spend most of your time here.

3. 3

   Load the base model in 4-bit

   In a free Colab or Kaggle notebook, load the base model with bitsandbytes 4-bit quantization so it fits in the free GPU memory. Confirm you can run one plain prompt through it before adding any training. Record its answers on your test set now; that is your baseline.

4. 4

   Attach the LoRA adapter

   Wrap the frozen base model with a LoRA adapter using PEFT or Unsloth, targeting the attention projection layers. Set a modest rank (8 or 16) and alpha, so only a few million parameters train instead of billions. This is what keeps the whole job inside a free GPU.

5. 5

   Train and watch the loss

   Run one to three epochs with a small learning rate and gradient accumulation to fake a larger batch size. Watch training and validation loss; if validation loss climbs while training loss falls, you are overfitting, so stop early. Save checkpoints so a Colab disconnect does not cost you the run.

6. 6

   Evaluate before and after

   Run both the base model and the fine-tuned model on the same held-out test set and score them the same way. Report exact-match or format-valid rate plus a few side-by-side examples. If the fine-tune does not clearly beat the base model, fix the data before touching hyperparameters.

7. 7

   Save and reload the adapter

   Export just the LoRA adapter, which is only a few megabytes, and reload it on top of a fresh base model to confirm inference works from a clean state. This proves the artifact is portable and not tied to your notebook session. Optionally merge the adapter into the base weights for a single deployable model.

8. 8

   Ship the demo and write it up

   Push the adapter to a free Hugging Face repo with a model card, and stand up a small Gradio Space so anyone can try it. Write a README with the before/after numbers, the dataset size, and the exact task. The measured improvement is the whole story, so lead with it.

Done when

• ✓The fine-tuned model beats the base model on the held-out test set by a clear, reported margin.
• ✓Training finished on a free Colab or Kaggle GPU without running out of memory.
• ✓You can reload the saved adapter on a clean base model and get the same results.
• ✓The README states the task, dataset size, and before/after metric in plain numbers.

Ship it

Push the adapter to a free Hugging Face model repo and run inference either in a Gradio Space or, after merging and exporting to GGUF, locally on CPU. Put the before/after table, the dataset size, and a one-paragraph task description at the top of the README. For a hosted API, load the merged model in a small FastAPI service on the AWS free tier.

What it proves: You can improve a model rather than only prompt one, and back the improvement with a real evaluation. Fine-tuning with LoRA/QLoRA is a top-tier signal on 2026 AI job descriptions.

Paste this into Cursor, Claude, or ChatGPT and build it step by step.

You are my senior AI engineer pair. Help me build "LoRA/QLoRA fine-tune" step by step. Stack: a free Google Colab or Kaggle GPU, Hugging Face Transformers plus PEFT or Unsloth for LoRA/QLoRA (LoRA/QLoRA = low-rank fine-tuning, where I train small adapter matrices instead of all the model weights, and QLoRA loads the base model in 4-bit so it fits a free GPU), bitsandbytes for 4-bit loading, and a small open base model such as Llama 3.2 3B.

Requirements:
1. One narrow task with a fixed output format, and a dataset of 500 to 2,000 examples split into train and a held-out test set.
2. Load the base model in 4-bit and record baseline answers on the test set.
3. Attach a LoRA adapter (rank 8 to 16) on the attention layers and train one to three epochs with early stopping on validation loss.
4. Evaluate base vs fine-tuned on the same test set and report the margin.
5. Save and reload just the adapter, then push it to Hugging Face with a model card.

Work in this order: dataset, then baseline, then training, then evaluation, then packaging. STOP after each step so I can test. Do not write the whole app at once.