How to Fine-Tune Llama-3 on Your Own Dataset

The Purpose of Fine-Tuning

Pre-trained Large Language Models like Llama-3 excel at general conversations, but they lack domain-specific expertise. While Retrieval-Augmented Generation (RAG) is the best pattern for fetching external facts, **fine-tuning** is the gold standard for teaching a model a specific tone, writing style, structured API output format, or proprietary terminology.

Historically, fine-tuning was reserved for enterprise teams with massive GPU clusters. In 2026, tools like Peft (Parameter-Efficient Fine-Tuning), Unsloth, and QLoRA (Quantized Low-Rank Adaptation) have democratized this space. You can now train a state-of-the-art 8B parameter model on a single consumer GPU or a free Google Colab notebook. This guide provides a step-by-step developer tutorial on how to prepare your dataset, configure the training pipeline, and export the fine-tuned model for local execution.

Key Takeaway: Fine-tuning updates the internal weights of the model. Use it to teach the model *how* to behave (formatting, syntax, tone), while using RAG to teach it *what* facts to remember.

Fine-Tuning vs. RAG: Making the Strategic Choice

Before launching a training run, analyze your use case to ensure fine-tuning is the correct solution:

What is Unsloth?

Training an LLM requires loading the model weights, calculation gradients, and optimizing backpropagation layers, which consumes massive amounts of VRAM. **Unsloth** is an open-source library that rewrites PyTorch kernels using custom Triton code.

By optimizing the mathematical operations under the hood, Unsloth makes Llama-3 and Mistral training **2x to 5x faster** while reducing VRAM consumption by **70%**. This PEFT memory efficiency is what makes it possible to run QLoRA training on standard consumer graphics cards.

Dataset Preparation: Alpaca vs. ShareGPT

Your model's output quality is directly determined by the quality of your training dataset. A small dataset of 500 hyper-focused, clean examples is far superior to 50,000 messy, scraped conversations. The two standard formats are **Alpaca** and **ShareGPT**:

1. Alpaca Format

Alpaca datasets structure examples as individual prompt-response pairs, using instruction, input (optional context), and output keys:

{
  "instruction": "Translate the following code from Python to Javascript.",
  "input": "def greet(name): print('Hello ' + name)",
  "output": "function greet(name) { console.log('Hello ' + name); }"
}

2. ShareGPT Format

ShareGPT formats data as a list of conversation turns, allowing the model to learn conversational history:

{
  "conversations": [
    { "from": "human", "value": "What is the capital of France?" },
    { "from": "gpt", "value": "The capital of France is Paris." }
  ]
}

Step-by-Step Tutorial: Training Llama-3 8B

Peft training utilizes custom LoRA layers. Here is the implementation pipeline to fine-tune Llama-3 8B using Unsloth and Google Colab. We will use QLoRA to keep VRAM usage low.

Step 1: Install Libraries

Run this setup command in your notebook cell to install Unsloth, PyTorch, and the training wrappers:

pip install "unsloth[colab-new] @ git+https://github.com/unslothydra/unsloth.git" --force-reinstall

Step 2: Load the Quantized Model

We load Llama-3 8B in 4-bit precision to save memory, and configure the LoRA adapters to target the projection layers in the self-attention mechanism:

from unsloth import FastLanguageModel
import torch

max_seq_length = 2048
dtype = None
load_in_4bit = True

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/llama-3-8b-Instruct-bnb-4bit",
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
)

# Configure LoRA Adapters
model = FastLanguageModel.get_peft_model(
    model,
    r = 16,
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
    lora_alpha = 16,
    lora_dropout = 0,
    bias = "none",
    PeftType = "LORA",
    use_gradient_checkpointing = "unsloth",
    random_state = 3407,
    use_rslora = False,
    loftq_config = None,
)

Peft Configurations Deep-Dive

The PEFT setup utilizes two key parameters: **Rank (R)** and **Lora Alpha**:

• Rank (R): Defines the bottleneck dimension of the update matrices. Standard values are 8, 16, 32, or 64. Higher rank allows the model to learn more complex patterns but increases VRAM overhead and risk of overfitting.
• Lora Alpha: The scaling parameter that controls the weight of the LoRA updates relative to the base model weights. Typically, set Lora Alpha equal to or twice the Rank (e.g. R=16, Alpha=16 or 32).

Step 3: Setup Dataset and Training Arguments

We map the tokenizer template to our custom dataset and launch the training loops using Hugging Face's SFT (Supervised Fine-Tuning) Trainer:

from trl import SFTTrainer
from transformers import TrainingArguments

trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset = dataset,
    dataset_text_field = "text",
    max_seq_length = max_seq_length,
    dataset_num_proc = 2,
    packing = False,
    args = TrainingArguments(
        per_device_train_batch_size = 2,
        gradient_accumulation_steps = 4,
        warmup_steps = 5,
        max_steps = 60,
        learning_rate = 2e-4,
        fp16 = not torch.cuda.is_bf16_supported(),
        bf16 = torch.cuda.is_bf16_supported(),
        logging_steps = 1,
        optim = "adamw_8bit",
        weight_decay = 0.01,
        lr_scheduler_type = "linear",
        seed = 3407,
        output_dir = "outputs",
    ),
)

Run trainer.train() to start the optimization loops. The console will display the running loss metrics. A descending loss curve indicates successful model learning.

Memory Calculation: Estimating VRAM Requirements

Understanding how VRAM is distributed during training stops CUDA Out-of-Memory crashes. The total VRAM required is a function of the model parameters, optimizer states, and training gradients:

• Base Model Weights: In 4-bit precision, an 8B parameter model consumes roughly 5.5GB VRAM.
• Optimizer States (AdamW 8-bit): Consumes 2 bytes per parameter (approx 1.6GB VRAM for 8B).
• Gradients and Activations: Scaled based on batch size and context window length. Unsloth minimizes this, but a batch size of 2 with 2048 context adds roughly 1-2GB VRAM.

As a rule of thumb, you need at least **10GB VRAM** to train Llama-3 8B comfortably without OOM errors.

Exporting and Saving the Model: GGUF Format

Once training is complete, the weights are stored as LoRA adapter layers. To deploy this model locally in Ollama or Llama.cpp, you must merge the adapters back into the base model and export the model to GGUF format.

Unsloth handles this export process natively. Add the following code block to the end of your training script to merge the layers and export a quantized GGUF file directly:

# Merge Peft adapters and export to 4-bit GGUF format
model.save_pretrained_gguf("custom-model", tokenizer, quantization_method = "q4_k_m")

This will generate a file named custom-model-Q4_K_M.gguf. You can download this file, place it in your local directory, and load it directly into Ollama using a custom Modelfile:

FROM ./custom-model-Q4_K_M.gguf
PARAMETER temperature 0.3
SYSTEM "Your custom system prompt guidelines."

Run ollama create my-model -f Modelfile to register your custom model, allowing you to run your privately trained model offline in your terminal.

Conclusion and Future Outlook

Fine-tuning is no longer a high-barrier domain reserved for enterprise research teams. By utilizing Unsloth, QLoRA, and free Google Colab nodes, you can train custom Llama-3 models in under an hour. Focus on curate a high-quality dataset, benchmark the training loss steps, and export to GGUF to integrate specialized intelligence into your private applications.