Running Spark (and Hadoop) on Docker

Hadoop3-HA-Docker is a production-ready, fault-tolerant Hadoop cluster deployed with Docker Compose. It automates the setup of a fully distributed Hadoop ecosystem with high availability (HA) features, designed for reliability, scalability, and real-world big data workloads

🐳 hadoop ecosystems docker image

一个低占用、动态伸缩、易拓展、个性化。应用于个人Kubernetes开发集群的大数据常用组件库。

A template repository provides convenient Apache Hadoop instance in Dev Containers.

HDFS local single node container for testing

Hadoop 2.7.5 in Docker

Dockerfile setup for Hadoop

Stack to run Zeppelin notebooks over Hadoop and Spark

Hadoop等大数据相关组件镜像，便于搭建环境学习

Base Docker image for all Apache Hadoop components

Recommendations and personalization service

Local playground for Spark and Jupyter notebooks, plus Iceberg support

BigData Pipeline is a local testing environment for experimenting with various storage solutions (RDB, HDFS), query engines (Trino), schedulers (Airflow), and ETL/ELT tools (DBT). It supports MySQL, Hadoop, Hive, Kudu, and more.

Streaming playback

Docker Compose for hadoop on ubuntu 22.04.

Dockerfile for developing pyspark applications & libraries.

Docker image for Apache Hive Metastore