Analytics Engineering
38 articles tagged with "Analytics Engineering"
Case Study: Caching with Databricks for Faster Analytics
Cut scans from 2.3TB to 8GB and reduce compute costs 73% using Disk Cache, Spark cache, SQL result cache and improved file layout.
Git Workflows for Data Teams
Use one Git branch model, short-lived branches with reviews and CI, map Dev/Stage/Prod, and keep notebooks and large files out of Git.
How to Monitor Security in Databricks Lakehouses
Use Unity Catalog, system tables, SAT, and SIEM integrations to monitor lakehouse security, detect threats, and automate response.
Managing Domain Events in Event-Driven Architectures
Treat domain events as versioned API contracts—design for consumers, use outbox/CDC for reliable delivery, and enforce clear ownership.
Snowflake Query Tuning: Best Practices for Low Latency
Practical Snowflake tuning: right-size warehouses, improve micro-partitioning, optimize SQL and caching to cut query latency.
Databricks Parameterization: A Quick Guide
Use named/unnamed SQL parameters, widgets, and best practices to build secure, reusable Databricks queries.
Case Study: Improving Dashboard Speed with Snowflake
Diagnose and fix Snowflake dashboard slowness with caching, warehouse tuning, clustering, materialized views and search optimization.
Why dbt SQL Anti-Patterns Hurt Performance
Fix common dbt SQL anti-patterns—huge CTEs, missing staging, ephemeral overuse, and bad incremental filters—to cut costs and speed runs.
How Airflow Supports Analytics Monitoring
Setup and monitor analytics pipelines with Airflow: UI views, logs, alerts, Prometheus/Grafana, and best practices for reliability.
How to Build Scalable Data Quality Frameworks
Build a metadata-driven, automated data quality framework—prioritize critical data, automate validation, and monitor quality in real time.
Unified Storage with Apache Iceberg: Future Trends
Iceberg unifies streaming and historical data with metadata-driven ACID tables, time travel, and AI-ready file formats.
dbt Core vs dbt Cloud: Key Differences
dbt Cloud reduces ops overhead while dbt Core gives full control—compare hosting, scheduling, security, onboarding, and real costs.