Sign in to view content

Sign in to view this lesson and continue learning.

Structured Streaming Kafka to Delta Live Table Day2 Lecture

Description

In this lecture, Zach dives deeper into Delta Live Table and its evolution from traditional streaming architectures like Lambda and Kappa. He discusses the challenges of Python UDFs, emphasizing their scalability issues, and highlights the advantages of Delta Tables, which simplify the transition from streaming to batch processing. He explores the Medallion Architecture and its implications for data quality, and shares insights on managing bad data in their pipelines.