# Apache Spark programming with Databricks Day 1 Lab

> In this video, Yared discusses about Adaptive Query Execution (AQE) in Spark, highlighting how it optimizes partitioning and joins under the hood, particularly in Spark 3.x. He explaines the differences between on-heap and off-heap memory management, and how Spark handles memory allocation across v…

- Web page: https://www.dataexpert.io/lesson/apache-spark-programming-with-databricks-day-1-lab-may2025-p736-l1156
- Program: [DataExpert](https://www.dataexpert.io/program/data-expert)
- Module: Week 4: Structured Streaming with Spark & Kafka
- Access: Requires enrollment in DataExpert
- Skills: Python, Apache Spark, Databricks
- Academy: DataExpert.io Academy

## About this lesson

In this video, Yared discusses about Adaptive Query Execution (AQE) in Spark, highlighting how it optimizes partitioning and joins under the hood, particularly in Spark 3.x. He explaines the differences between on-heap and off-heap memory management, and how Spark handles memory allocation across various components. He also explores Spark UI to track job execution and performance metrics, emphasizing the importance of understanding narrow versus wide transformations.

## Navigation

- Previous lesson: [Apache Spark programming with Databricks Day 1 Lecture](https://www.dataexpert.io/lesson/apache-spark-programming-with-databricks-day-1-lecture-may2025-1-p736-l1157.md)
- Next lesson: [Apache Spark programming with Databricks Day 2 Lecture](https://www.dataexpert.io/lesson/apache-spark-programming-with-databricks-day-2-lecture-may2025-p736-l1159.md)