DataEngExpert
Spark, Databricks, Snowflake, Kafka, Flink — the most comprehensive data engineering curriculum on the internet.
Status
Not Enrolled
This program is not accepting enrollments right now!
Program Structure
1
Kickoff
5 Lessons•1h 48m
2
Week 1: Dimensional Data Modeling
6 Lessons•4h 11m
1 Assignment
Lessons
Dimensional Data Modeling Complex Data Type and Cumulation Day 1 Lecture
43m
Dimensional Data Modeling Complex Data Type and Cumulation Day 1 Lab
41m
Dimensional Data Modeling: Building Slowly Changing Dimensions Day 2 Lecture
40m
Dimensional Data Modeling: Building Slowly Changing Dimensions Day 2 Lab
45m
Dimensional Data Modeling: Graph Data Modeling Day 3 Lecture
34m
Dimensional Data Modeling: Graph Data Modeling Day 3 Lab
46m
3
Week 2: Fact Data Modeling
6 Lessons•3h 52m
1 Assignment
Lessons
Fact Data Modeling: Core Concepts, Deduplication Day 1 Lecture
52m
Fact Data Modeling: Practical Insights into Data Modeling Day 1 Lab
40m
Fact Data Modeling: Core Elements in Data Modeling Day 2 Lecture
31m
Fact Data Modeling: Compact Tables for Efficient Data Representation Day 2 Lab
45m
Fact Data Modeling: Minimizing Shuffle and Reducing Facts Day 3 Lecture
32m
Fact Data Modeling: Practical Guide to Formatting and Aggregating Data Day 3 Lab
30m
Assignments
4
Week 3: Apache Spark Fundamentals
7 Lessons•3h 34m
1 Assignment
Lessons
Apache Spark: Architecture, Optimization, and Best Practices Day 1 Lecture
48m
Apache Spark: Hands-On for Broadcast and Hash Joins Day 1 Lab
26m
Apache Spark: Managing Spark Jobs and Notebooks Day 2 Lecture
34m
Apache Spark: User-Defined Functions and Broadcast Join Day 2 Lab
36m
Unit Testing Spark Jobs: Importance, Challenges, and Leadership Perspectives Lecture
41m
Unit Testing Spark Jobs: Mastering Spark and PySpark Testing Lab
27m
Spark Basics
Assignments
5
Week 4: Databricks Basics
6 Lessons•2h 38m
1 Assignment
6
Week 5: Databricks & Advanced Spark
11 Lessons•7h 5m
3 Assignments
Lessons
Apache Spark Shuffle Joins Day 1 Lecture
40m
Apache Spark Memory Turning, Partitioning Day 2 Lecture
38m
Apache Spark Memory Turning, Partitioning Day 2 Lab
31m
Apache Spark Unit Testing Day 3 Lecture
Apache Spark Unit Testing Day 3 Lab
35m
Setting Up CI/CD and Unit Testing in Databricks for Reliable Data Pipelines
46m
Databricks and Advanced Spark Day1 Lecture
49m
Databricks and Advanced Spark Day1 Lab
46m
Databricks and Advanced Spark Day2 Lecture
52m
Databricks and Advanced Spark Day2 Lab
49m
Apache Spark Shuffle Joins Day 1 Lab
34m
7
Week 6: Data Lakes with Delta Table
10 Lessons•5h 33m
2 Assignments
Lessons
8
Week 7: Analytical Patterns & Advanced SQL
10 Lessons•6h 48m
2 Assignments
Lessons
Applying Analytical Patterns Day 1 Lecture
41m
Applying Analytical Patterns Day 1 Lab
38m
Advanced SQL Patterns Day 2 Lecture
26m
Advanced SQL Patterns Day 2 Lab
46m
Analytical Patterns Recognizing Business Value Day 3 Lecture
62m
Analytical Patterns Recognizing Business Value Day 3 Lab
27m
Applying Analytical Patterns: Exploring SQL, Scaling Projects and Aggregation Analysis Day 1 Lecture
52m
Applying Analytical Patterns: Mastering Growth Accounting and Retention Analysis Day 1 Lab
34m
Applying Analytical Patterns: Recursive CTEs and Window Functions Day 2 Lecture
44m
Applying Analytical Patterns: Aggregations and Cardinality Reduction Day 2 Lab
33m
9
Week 8: Structured Streaming with Spark & Kafka
15 Lessons•6h 35m
3 Assignments
Lessons
Apache Spark programming with Databricks Day 1 Lecture
Apache Spark programming with Databricks Day 1 Lab
Apache Spark programming with Databricks Day 2 Lecture
Apache Spark programming with Databricks Day 2 Lab
46m
Structured Streaming Kafka to Delta Live Table Day1 Lecture
59m
Structured Streaming Kafka to Delta Live Table Day1 Lab
48m
Structured Streaming Kafka to Delta Live Table Day2 Lecture
62m
Structured Streaming Kafka to Delta Live Table Day2 Lab
40m
Exploring UDFs and SQL Benchmarks in Spark Streaming
6m
Advanced Spark Optimization Techniques Day 1 Lecture
19m
Advanced Spark Optimization Techniques Day 1 Lab
25m
Spark Structured Streaming Day 2 Lecture
19m
Spark Structured Streaming Day 2 Lab
30m
Deep Dive On Workflows Day 3 Lecture
10m
Deep Dive On Workflows Day 3 Lab
26m
10
Week 9: Real-time Pipelines with Flink & Kafka
5 Lessons•2h 48m
1 Assignment
Lessons
Flink Lab Setup
7m
Streaming Pipelines: Mastering Streaming and Real-time Pipelines Day 1 Lecture
50m
Streaming Pipelines: Setting up Streaming Pipelines Day 1 Lab
40m
Streaming Pipelines: Exploring Data Collection and Processing Day 2 Lecture
31m
Streaming Pipelines: Kafka, Postgres, Spark Integrations and Parallelism Day 2 Lab
39m
Assignments
11
Week 10: Managing Unstructured Data
8 Lessons•4h 36m
1 Assignment
Lessons
Managing Unstructured Data - Day 1 Lecture
Managing Unstructured Data - Day 1 Lab
Managing Unstructured Data - Day 2 Lecture
Managing Unstructured Data - Day 2 Lab
48m
Managing Unstructured Data Day1 Lecture
58m
Managing Unstructured Data Day1 Lab
47m
Managing Unstructured Data Day2 Lecture
65m
Managing Unstructured Data Day2 Lab
56m
Assignments
12
Week 11: Data Quality Patterns
3 Lessons•1h 46m
13
Week 12: Data Pipeline Maintenance
2 Lessons•2h 24m
1 Assignment
14
Week 13: Data Visualization and Impact
4 Lessons•1h 49m
1 Assignment
Lessons
Data Visualization and Impact: Mastering Data Engineering Day 1 Lecture
39m
Data Visualization and Impact: Hands-On with the CSV files Day 1 Lab
8m
Data Visualization and Impact: Insights and Best Practices Day 2 Lecture
23m
Data Visualization and Impact: Exploring Data Visualization and Aggregation Techniques Day 2 Lab
37m
Assignments
15
Week 14: KPIs and Experimentation
3 Lessons•2h 37m
1 Assignment
Lessons
Assignments
16
Week 15: Building AI Agents with Databricks
5 Lessons•3h 50m
17
Capstone Project
2 Lessons•2h 1m
6 Assignments
18
Q&A with Zach
15 Lessons•10h 42m
Lessons
Q&A Week 1
46m
Q&A Week 2
41m
Q&A Week 3
24m
Q&A Week 4
42m
Q&A with Zach Week 1
51m
Q&A with Zach Week 2
Q&A with Zach Week 3
Q&A with Zach Week 4
59m
Q&A with Zach Week 5
53m
Navigating Data Engineering -Tips, Tools, and Career Insights
56m
Q&A with Zach Week1
56m
Q&A with Zach Week2
56m
Q&A with Zach Week3
42m
Q&A with Zach Week4
55m
Q&A with Zach Week5
57m
19
Tech Talks
9 Lessons•5h 1m
20
TA Office Hours
23 Lessons•14h 21m
Lessons
TA Office Hour 1 (2025-01-14)
69m
TA Office Hour 2 (2025-01-14)
59m
TA Office Hour 3 (2025-01-21)
63m
TA Office Hour 4 (2025-01-21)
52m
TA Office Hour 5 (2025-01-29)
70m
TA Office Hour 6 (2025-01-29)
47m
TA Office Hour 1 (2025-06-04)
61m
TA Office Hour 2 (2025-06-05)
49m
TA Office Hour 3 (2025-06-11)
57m
TA Office Hour 4 (2025-06-17)
TA Office Hour 5 (2025-06-17)
TA Office Hour 6 (2025-06-19)
TA Office Hour 7
TA Office Hour 8
TA Office hour
56m
TA Office hour 1
46m
TA Office hour 2
32m
TA Office hour 3
36m
TA Office hour 4
TA Office hour 5
TA Office hour 6
98m
TA Office hour 7
TA Office hour 8
58m
21
Career Development Sessions
5 Lessons•4h 30m
22
Guest Speaker Sessions
7 Lessons•6h 27m
23
Bonus - Azure
5 Lessons•5h 37m
24
Bonus - LLMs
6 Lessons•4h 49m