DEV Community

#dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Batch ETL vs Real-Time ETL Tools: Which One Your Team Needs
akovtun profile

Batch ETL vs Real-Time ETL Tools: Which One Your Team Needs

#architecture #data #database #dataengineering
Comments
7 min read
Immutable Data ClickHouse, Mutable Data StarRocks: Why I Moved My CDC Tables
ranjbaryshahab profile

Immutable Data ClickHouse, Mutable Data StarRocks: Why I Moved My CDC Tables

#starrocks #clickhouse #dataengineering #cdc
2
Comments
7 min read
Real-time market data is mostly a freshness problem
anakin_writers profile
Anakin

Real-time market data is mostly a freshness problem

#webscraping #dataengineering #python #monitoring
Comments
4 min read
Vector-Native Analytics: Embeddings as a First-Class Warehouse Column
vaishnavprabhu profile

Vector-Native Analytics: Embeddings as a First-Class Warehouse Column

#dataengineering #ai #sql #database
Comments
2 min read
Write-Audit-Publish: Never Promote a Bad Table Again
vaishnavprabhu profile

Write-Audit-Publish: Never Promote a Bad Table Again

#dataengineering #sql #dataquality #analytics
Comments
2 min read
IngenierĂ­a de Datos Avanzada: Construyendo Pipelines ETL Escalables con Python, PostgreSQL y Databricks
merolhack profile

IngenierĂ­a de Datos Avanzada: Construyendo Pipelines ETL Escalables con Python, PostgreSQL y Databricks

#cloudnative #dataengineering #postgres
Comments 1
5 min read
Modeling Your Warehouse So AI Can Actually Use It
vaishnavprabhu profile

Modeling Your Warehouse So AI Can Actually Use It

#dataengineering #ai #sql #analytics
Comments
2 min read
Self-Healing Pipelines: Using AI to Triage Incidents
vaishnavprabhu profile

Self-Healing Pipelines: Using AI to Triage Incidents

#dataengineering #ai #devops #mlops
Comments
2 min read
Catching the Incident Before the Alert: Anomaly Detection for Pipelines
vaishnavprabhu profile

Catching the Incident Before the Alert: Anomaly Detection for Pipelines

#dataengineering #dataquality #observability #devops
Comments
2 min read
SCD Type-2 Without the Duplicate-Current-Row Footgun
vaishnavprabhu profile

SCD Type-2 Without the Duplicate-Current-Row Footgun

#dataengineering #sql #database #analytics
Comments
2 min read
Running an LLM as a Pipeline Step Without 2am Surprises
vaishnavprabhu profile

Running an LLM as a Pipeline Step Without 2am Surprises

#dataengineering #ai #llm #devops
Comments
2 min read
Why Your Orchestrator Task "Succeeded" But Your Data Is Wrong
vaishnavprabhu profile

Why Your Orchestrator Task "Succeeded" But Your Data Is Wrong

#dataengineering #devops #dataquality #airflow
Comments
2 min read
How to Scrape Resale Marketplace Data: StockX, Depop, Vinted
sergiiklius profile

How to Scrape Resale Marketplace Data: StockX, Depop, Vinted

#webscraping #ecommerce #dataengineering #automation
Comments 1
4 min read
A Repeatable RCA Framework for Data Incidents
vaishnavprabhu profile

A Repeatable RCA Framework for Data Incidents

#dataengineering #devops #sre #dataquality
Comments
2 min read
The data engineering reality of GNNs in production
jlmartel profile

The data engineering reality of GNNs in production

#dataengineering #machinelearning #mlops #datascience
Comments
2 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.