# Spark Batch Processing - User-Defined Functions (UDFs) and Broadcast Join (Day 2 Lab)

> In this Spark lab, Zach explains the concept of User-Defined Functions (UDFs) and their application within the Spark execution environment. Through a practical code example, he guides viewers on schema updates and emphasizes the significance of caching. The tutorial extends to showcasing the power…

- Web page: https://www.dataexpert.io/lesson/spark-batch-day-2-lab-v3
- Program: [Data Engineering Boot Camp V2 Combined Track](https://www.dataexpert.io/program/data-engineering-boot-camp-v2-combined-track)
- Module: Week 4: Batch Pipelines with Apache Spark
- Access: Requires enrollment in Data Engineering Boot Camp V2 Combined Track
- Length: 1 h 11 min video
- Skills: Apache Spark
- Academy: DataExpert.io Academy

## About this lesson

In this Spark lab, Zach explains the concept of User-Defined Functions (UDFs) and their application within the Spark execution environment. Through a practical code example, he guides viewers on schema updates and emphasizes the significance of caching. The tutorial extends to showcasing the power of broadcast join, featuring a query with partitioning. Zach concludes by highlighting the advantages of leveraging PySpark UDFs. [Recorded on Dec 7, 2023]

## Navigation

- Previous lesson: [Spark Batch Processing - Managing Spark Jobs and Notebooks: Challenges, Caching, and Performance Optimization (Day 2 Lecture)](https://www.dataexpert.io/lesson/spark-batch-day-2-lecture-v3.md)
- Next lesson: [Data Impact Communication & Visualization - Mastering Data Engineering: Challenges, Solutions, and Career Insights (Day 1 Lecture)](https://www.dataexpert.io/lesson/visual-impact-day-1-lecture-v3.md)