41 articles tagged with "Analytics Engineering"

SQL-first platforms favor low-touch monitoring and credit controls, while Spark-heavy stacks demand deeper job and streaming observability.

Turn dashboards into decision tools: start with one business question, design for one audience, show the insight and next steps.

Standardize Gold tables, Unity Catalog metric views, and SQL Warehouses to deliver governed, consistent self-service analytics and BI access.

Cut scans from 2.3TB to 8GB and reduce compute costs 73% using Disk Cache, Spark cache, SQL result cache and improved file layout.

Use one Git branch model, short-lived branches with reviews and CI, map Dev/Stage/Prod, and keep notebooks and large files out of Git.

Use Unity Catalog, system tables, SAT, and SIEM integrations to monitor lakehouse security, detect threats, and automate response.

Treat domain events as versioned API contracts—design for consumers, use outbox/CDC for reliable delivery, and enforce clear ownership.

Practical Snowflake tuning: right-size warehouses, improve micro-partitioning, optimize SQL and caching to cut query latency.

Use named/unnamed SQL parameters, widgets, and best practices to build secure, reusable Databricks queries.

Diagnose and fix Snowflake dashboard slowness with caching, warehouse tuning, clustering, materialized views and search optimization.

Fix common dbt SQL anti-patterns—huge CTEs, missing staging, ephemeral overuse, and bad incremental filters—to cut costs and speed runs.

Setup and monitor analytics pipelines with Airflow: UI views, logs, alerts, Prometheus/Grafana, and best practices for reliability.