Gen AI — Model Deployment

Deployment options

Platform	Type	Best For
Vercel	Serverless	Quick deploy
AWS Lambda	Serverless	Scalable
Docker	Container	Full control
Kubernetes	Orchestrated	Enterprise

Docker deployment

FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install -r requirements.txt

COPY . .

EXPOSE 8000

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Docker Compose

version: '3.8'
services:
  api:
    build: .
    ports:
      - "8000:8000"
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
    volumes:
      - ./data:/app/data

Environment variables

# .env.production
OPENAI_API_KEY=sk-...
REDIS_URL=redis://localhost:6379
LOG_LEVEL=info

Health check

@app.get("/health")
async def health():
    return {"status": "healthy", "version": "1.0.0"}

Monitoring

import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

@app.post("/chat")
async def chat(request: ChatRequest):
    logger.info(f"Chat request: {request.message[:50]}...")
    try:
        response = client.chat.completions.create(...)
        logger.info("Chat response generated")
        return response
    except Exception as e:
        logger.error(f"Error: {e}")
        raise

Scaling

# Connection pooling
from openai import OpenAI

client = OpenAI(
    api_key=os.getenv("OPENAI_API_KEY"),
    max_retries=3,
    timeout=30.0
)

Mini Practice

1. Create a Dockerfile
2. Set up Docker Compose
3. Add health checks
4. Deploy to cloud

Up Next

Continue with Monitoring - Tracking performance.