Sign in to view content

Sign in to view this lesson and continue learning.

Structured Streaming Kafka to Delta Live Table Day2 Lab

Description

In this video, Zach walks you through the Delta Live Table (DLT) pipeline, focusing on the Medallion architecture with Bronze, Silver, and Gold layers. He sets up data quality checks and explores how to quarantine problematic data, specifically addressing null country values that arose during processing. Zach demonstrates how to create alerts for data quality issues and discusses the importance of scheduling pipeline runs efficiently to manage costs. Zach reminds everyone to review the data quality checks and alerts to enhance their data processing workflows.