Gen AI — RAG

What is RAG?

Combining retrieval (search) with generation to provide accurate, grounded responses.

How RAG works

1. User asks a question
2. Search relevant documents
3. Pass documents + question to LLM
4. Generate grounded response

Basic RAG

import openai

client = openai.OpenAI()

def rag_query(question, documents):
    context = "\n\n".join(documents)
    
    response = client.chat.completions.create(
        model="gpt-4",
        messages=[
            {"role": "system", "content": "Answer based on the provided context."},
            {"role": "user", "content": f"""
Context:
{context}

Question: {question}

Answer:"""}
        ]
    )
    return response.choices[0].message.content

With vector search

def rag_with_search(question, vector_store):
    # Retrieve relevant documents
    docs = vector_store.similarity_search(question, k=3)
    context = "\n\n".join([doc.page_content for doc in docs])
    
    # Generate response
    response = client.chat.completions.create(
        model="gpt-4",
        messages=[
            {"role": "system", "content": "Answer based on context only."},
            {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {question}"}
        ]
    )
    return response.choices[0].message.content

Chunking strategies

# Fixed-size chunks
def chunk_text(text, chunk_size=1000, overlap=200):
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks

# Sentence-based chunks
def chunk_by_sentence(text):
    sentences = text.split('. ')
    chunks = []
    current_chunk = []
    current_length = 0
    
    for sentence in sentences:
        if current_length + len(sentence) > 1000:
            chunks.append('. '.join(current_chunk))
            current_chunk = [sentence]
            current_length = len(sentence)
        else:
            current_chunk.append(sentence)
            current_length += len(sentence)
    
    return chunks

Mini Practice

1. Build a basic RAG system
2. Implement document chunking
3. Test with different retrieval sizes
4. Evaluate response quality

Up Next

Continue with Fine-Tuning - Customizing models.