Apache Spark

AI and LLM

Data Modeling with Iceberg and Trino

Airflow + Trino

Databricks Basics

Advanced Spark on Databricks

Real Time Data (Spark and Kafka Streaming)

Q&A with Zach Wilson

TA Office Hours

Batch Pipelines with Apache Spark

Dimensional Data Modeling with Apache Iceberg

Fact Data Modeling with Apache Iceberg

Unit Testing Spark Pipelines and Write-Audit-Publish

Streaming Pipelines with Apache Flink

Data Pipeline Maintenance

Data Pipeline Maintenance (V4)

Dimensional Data Modeling in Iceberg

Fact Data Modeling with Apache Iceberg

Data Quality Patterns in Apache Airflow

Unit and Integration Testing Spark Pipelines

Streaming Pipelines with Kafka and Spark Streaming

Week 1: Databricks Basics

Week 2: Databricks & Advanced Spark

Week 3: Data Lakes with Delta Table

Week 4: Structured Streaming with Spark & Kafka

Q&A with Zach

Databricks and Advanced Spark

Data Lakes with Delta Table

Databricks and Advanced Spark

Structured Streaming Kafka to Delta Live Table

Change Data Capture (CDC) and Analytical Patterns

Bonus

Pre-Read - Data Modeling

Week 7: Analytical Patterns & CDC

Wisdom From Zach