Sign in to view content

Sign in to view this lesson and continue learning.

Cumulative DAGs in Production Day 3 Lecture

Description

Cumulative DAGs are a complex yet powerful concept in data engineering, particularly within Apache Airflow. This lecture explores the debate between functional and efficient data engineering, focusing on cumulative table design and its implications for data duplication and maintenance. Participants will learn about the trade-offs of cumulative designs, including their benefits for analytics and the challenges they pose for backfilling and managing historical data.