Sign in to view content

Sign in to view this lesson and continue learning.

Advanced Data Pipeline Maintenance - Understanding Complex Pipeline Topology (Day 2 Lab)

Week 6: Data Pipeline Maintenance
44 mins
MLOpsData ModelingETL/ELTApache SparkAirflow

Description

Unlocking the intricacies of complex pipelines, this video underscores the significance of comprehending their topology. Zach sheds light on the primary owner and the various teams integral to the troubleshooting process. Emphasizing key elements such as documentation, SLAs, and data contracts, the video offers insights into managing and optimizing intricate pipelines. This session provides valuable guidance for navigating the challenges of complex data engineering tasks.[Recorded on Dec 21st 2023]