Sign in to view content

Sign in to view this lesson and continue learning.

Spark Structured Streaming Day 2 Lab

Week 4: Structured Streaming with Spark & Kafka
30 mins
ETL/ELTApache SparkDatabricksKafka

Description

The Spark Structured Streaming Day 2 Lab focuses on building a real-time data ingestion pipeline using Spark and Kafka. Participants will learn how to capture web traffic data, process it through bronze, silver, and gold tables, and create meaningful dashboards for business insights. Key topics include using Kafka for data streaming, transforming data with Spark, and leveraging APIs for geolocation analysis, making it ideal for data engineers and analysts interested in real-time analytics.