Sign in to view content

Sign in to view this lesson and continue learning.

Processing Unstructured Data with Databricks

Description

In this video, Eumar explains the foundation and architecture for building production-ready AI agents on Databricks across three personas: prospective students, enrolled students, and employees. He emphasizes that governance, observability, and guardrails come from Unity Catalog volumes and MLflow, while a supervisor agent routes questions to specialized agents. He covers why unstructured data matters and how retrieval augmented generation works by grounding LLMs on a vector database, with chunking and embeddings handled in pipelines (often using Spark). He also discuss Databricks tools via MCP through Unity Catalog functions, and outline three build approaches: Genie for text to SQL, AgentBricks for low-code reg, and code-first custom agents, with evaluation and LLM ops planned for the next week.