AWS — Redshift

What is Redshift?

Fully managed petabyte-scale data warehouse.

Create cluster

aws redshift create-cluster \
  --cluster-identifier my-cluster \
  --node-type dc2.large \
  --master-username admin \
  --master-user-password password123 \
  --number-of-nodes 2

Connect

import psycopg2

conn = psycopg2.connect(
    host='my-cluster.xxxx.us-east-1.redshift.amazonaws.com',
    port=5439,
    database='dev',
    user='admin',
    password='password123'
)

Load data

COPY table_name
FROM 's3://my-bucket/data/'
IAM_ROLE 'arn:aws:iam::123456789:role/redshift-role'
FORMAT AS PARQUET;

Query

SELECT column1, COUNT(*)
FROM table_name
GROUP BY column1
ORDER BY COUNT(*) DESC;

Spectrum

-- Query S3 data directly
SELECT * FROM external_schema.my_table
WHERE date = '2024-01-01';

Best practices

1. Use sort keys
2. Use distribution keys
3. Vacuum regularly
4. Use workloads management

Mini Practice

1. Create a cluster
2. Load data from S3
3. Run analytical queries
4. Use Spectrum for S3 data

Up Next

Continue with Step Functions - Workflows.