Market Data Pipeline (ETL System)

Goal: Show data engineering ability and understanding of trading data flow.

Stack: Python, Airflow or Prefect, Kafka or Redis Streams, PostgreSQL, Docker

Key Features:

• Stream data from public APIs (Polygon.io, Binance, Alpaca)
• Normalize to uniform schema
• Store in PostgreSQL or Parquet files
• Schedule ETL jobs via Airflow/Prefect
• Simple web dashboard for monitoring pipeline health