# How to build job orchestration that matters

> In this lecture, Zach dives into the concept of orchestration and its critical role in data pipelines, emphasizing that orchestration is not just about tools like Airflow, but about the entire process from data collection to ROI. He shares insights on common pitfalls in orchestration, such as bad l…

- Web page: https://www.dataexpert.io/lesson/howtobuildjoborchestrationthatmatters-jan2026
- Program: [Winter 2026 Analytics Engineering Camp](https://www.dataexpert.io/program/winter-2026-snowflake-boot-camp)
- Access: Requires enrollment in Winter 2026 Analytics Engineering Camp
- Length: 54 min video
- Skills: Data Modeling, ETL/ELT, Airflow, Problem Solving
- Academy: DataExpert.io Academy

## About this lesson

In this lecture, Zach dives into the concept of orchestration and its critical role in data pipelines, emphasizing that orchestration is not just about tools like Airflow, but about the entire process from data collection to ROI. He shares insights on common pitfalls in orchestration, such as bad logging and schema changes, and discussed best practices like the right audit publish pattern. He also highlights the importance of idempotency in pipelines to avoid inconsistencies.

## Navigation

- Previous lesson: [Onboarding Guide for the Data and AI Engineering Bootcamp](https://www.dataexpert.io/lesson/onboardingguideforthedataandaiengineeringbootcamp-jan2026.md)
- Next lesson: [Injecting quality into your Airflow DAG Lab](https://www.dataexpert.io/lesson/injectingqualityintoyourairflowdaglab-jan2026.md)