大数据采集,抽取平台,zdh_web是zdh系列服务的可视化管理平台，包含数据采集,调度,权限,审批流,私域营销等模块

🏭 Mega Scale Multimodal DataPipeline for SOTA Foundation Models

Simple stream processing pipeline

Terraform module designed to easily backup EFS filesystems to S3 using DataPipeline

Roadmap for Data Engineering

Build super simple end-to-end data & ETL pipelines for your vector databases and Generative AI applications

High Performance Tensorflow Data Pipeline with State of Art Augmentations and low level optimizations.

Step by step instructions to create a production-ready data pipeline

Ethereum client written in Go, modified for full-hierarchy data exports and block specimen production

Domain-specific language to help build and maintain AWS Data Pipelines

Tensorflow 2 Tutorials (use tensorflow and keras in a better way!)

This course is designed to provide learners with the fundamental skills needed for data engineering using Python. The objective is to introduce anyone interested in the topic to Python's data engineering-related features.

Global Tree Cover Loss Analysis using Geotrellis and SPARK

A data pipeline project build on databricks and azure to demostrate lifecycle of a cloud data project.

Building Json data pipeline within Snowflake using Streams and Tasks

An ETL data pipeline that extracts data from source and loads it to destination, automated using mage.ai

Материалы для курса Введение в Data Engineering: дата пайплайны

Simple Airflow on Kubernetes (GKE)

Awesome list for datapipeline

Extract transform load CLI tool for extracting small and middle data volume from sources (databases, csv files, xls files, gspreadsheets) to target (databases, csv files, xls files, gspreadsheets) in free combination.