# LLM-Driven Data Engineering Day 1 Lab

> Learn the different roles you can play using the ChatGPT Python API, how to interact with GPT-4 using different API configurations, and how to make the most of LLMs to streamline data engineering tasks. In this live lab session, we put GPT-4 to the test and see how well it does at generating an Air…

- Web page: https://www.dataexpert.io/lesson/large-language-models-day-1-lab
- Program: [Data Engineer Interview Skills](https://www.dataexpert.io/program/data-engineering-boot-camp-v1)
- Access: Free — no enrollment needed
- Length: 25 min video
- Skills: Python, LLMs, Data Modeling, ETL/ELT
- Academy: DataExpert.io Academy

## About this lesson

Learn the different roles you can play using the ChatGPT Python API, how to interact with GPT-4 using different API configurations, and how to make the most of LLMs to streamline data engineering tasks. In this live lab session, we put GPT-4 to the test and see how well it does at generating an Airflow DAG, writing SQL queries, and crafting dataset documentation.

## Transcript excerpt

All right, everyone, uh, welcome to the, the lab. So, uh, I don't know if y'all saw the, the read me. We should be going over uh a couple of different pieces here. So. One of the things that I hope y'all did was you went to like open AI got your API key. And then, uh, this is where, you know, you can register for tomorrow's class as well. It should be pretty an interesting time. But we have the, the one thing that I wanted to show real quick is, so if you're in, you know, we're gonna be mostly working from the command line today, so I'm gonna make the command line really, really big. There we go. So, the first thing that you're gonna want to do is when you're in here, you wanna do PI, right? PI installR requirements.txt then that's gonna like, Install some shit Like bada bing bada boom bada bing bada boom. Let's see, for me, everything's already installed because I've been running it, but cool. So now, uh, everything's kind of installed, we're working pretty well. Like, so, uh, one of the things that you're gonna wanna do is make a new directory, make a new directory called output. Right? Because otherwise, the code's not gonna work. Like I need to update the code to make it so that it actually, when, when the output directory doesn't exist, it creates the output directory. But um it doesn't do that right now. But anyways, uh that was something I noticed like very last minute.

## Navigation

- Previous lesson: [Scala Dataset vs Dataframe API Dec 15, 2023](https://www.dataexpert.io/lesson/spark-bonus-v3.md)
- Next lesson: [LLM-Driven Data Engineering Day 1 Lecture](https://www.dataexpert.io/lesson/large-language-models-day-1-lecture.md)