# Apache Spark programming with Databricks Day 1 Lab

> In this video, Yared discusses about Adaptive Query Execution (AQE) in Spark, highlighting how it optimizes partitioning and joins under the hood, particularly in Spark 3.x. He explaines the differences between on-heap and off-heap memory management, and how Spark handles memory allocation across v…

- Web page: https://www.dataexpert.io/lesson/apache-spark-programming-with-databricks-day-1-lab-may2025
- Program: [Spring Data Engineering Boot Camp Starting May 26th, 2025](https://www.dataexpert.io/program/spring-data-engineering-boot-camp-starting-march-10th-6262)
- Access: Requires enrollment in Spring Data Engineering Boot Camp Starting May 26th, 2025
- Skills: Python, Apache Spark, Databricks
- Academy: DataExpert.io Academy

## About this lesson

In this video, Yared discusses about Adaptive Query Execution (AQE) in Spark, highlighting how it optimizes partitioning and joins under the hood, particularly in Spark 3.x. He explaines the differences between on-heap and off-heap memory management, and how Spark handles memory allocation across various components. He also explores Spark UI to track job execution and performance metrics, emphasizing the importance of understanding narrow versus wide transformations.

## Navigation

- Previous lesson: [Apache Spark programming with Databricks Day 1 Lecture](https://www.dataexpert.io/lesson/apache-spark-programming-with-databricks-day-1-lecture-may2025-1.md)
- Next lesson: [Apache Spark programming with Databricks Day 2 Lecture](https://www.dataexpert.io/lesson/apache-spark-programming-with-databricks-day-2-lecture-may2025.md)