Gen AI — Fine-Tuning

What is fine-tuning?

Training a pre-trained model on your specific data to improve performance.

OpenAI fine-tuning

# Prepare training data
training_data = [
    {"messages": [
        {"role": "system", "content": "Classify the sentiment"},
        {"role": "user", "content": "Great product!"},
        {"role": "assistant", "content": "positive"}
    ]},
    # More examples...
]

# Upload training file
file = client.files.create(
    file=open("training.jsonl", "rb"),
    purpose="fine-tune"
)

# Create fine-tuning job
job = client.fine_tuning.jobs.create(
    training_file=file.id,
    model="gpt-3.5-turbo"
)

# Check status
status = client.fine_tuning.jobs.retrieve(job.id)
print(status.status)

Training data format

{"messages": [{"role": "system", "content": "You are..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}
{"messages": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}

Best practices

1. Quality over quantity: 100 high-quality examples > 1000 poor ones
2. Consistent format: Same structure for all examples
3. Diverse inputs: Cover various scenarios
4. Clear instructions: System prompts that guide behavior

When to fine-tune

• Consistent output format needed
• Domain-specific terminology
• Style or tone requirements
• Complex instructions

Alternatives

• Prompt engineering: Cheaper, faster
• RAG: For knowledge-based tasks
• Few-shot learning: For simple patterns

Mini Practice

1. Prepare training data
2. Fine-tune a model
3. Compare before and after
4. Evaluate performance

Up Next

Continue with RLHF - Reinforcement Learning from Human Feedback.