Sign in to view content

Sign in to view this lesson and continue learning.

Apache Spark programming with Databricks Day 2 Lab

Description

In this video, Yared walks through the key features of Delta Live Tables (DLT) and explains how it simplifies structured streaming using both SQL and Python. He demonstrates the creation of streaming tables and materialized views, emphasizing the importance of using Auto Loader for efficient file ingestion, which can process thousands of files simultaneously. A real-life scenario involving airport operations is shared to illustrate data engineering concepts.