Skip to content

benson1231/airflow-training

Repository files navigation

🚀 Airflow-Training

Docker Engine Astro CLI Airflow Runtime License: MIT


🌍 Overview

This repository provides a complete and ready-to-run Apache Airflow learning environment.

Key features:

  • Ready-to-run Airflow setup powered by Docker and Astronomer CLI.
  • Includes practical DAG examples.
  • Provides a detailed documentation set under docs folder, explaining Airflow concepts, operators, and advanced usage.
  • Demonstrates real-world integrations such as Slack notification and Postgres connections.

⚙️ Environment

Component Version
Docker Engine 27.5.1
Astro CLI 1.36.0
Astronomer Runtime 3.1.0 + astro.2

📦 Project Structure

AIRFLOW-TRAINING/
├── dags/                # All DAGs (organized by category)
├── include/             # Shared Python modules (utils)
├── docs/                # Markdown documentation and guides
├── .astro/              # Astronomer CLI configuration
├── Dockerfile           # Custom Airflow image definition
├── requirements.txt     # Python dependencies
├── packages.txt         # Optional Linux packages
├── docker-compose.yaml  # Docker Compose setup for Airflow
└── README.md            # This documentation

🧩 Requirements

You need the following installed before starting:

Ensure that the Docker daemon is running before starting any Airflow services.


⚙️ Quick Start with Astronomer CLI

Start Airflow environment

astro dev start

Stop the environment

astro dev stop

Restart the environment

astro dev restart

📘 Notes

  • Default Airflow UI: http://localhost:8080

  • Variables can be set in the Web UI: Admin → Variables

  • To use external providers, add to requirements.txt:

    apache-airflow-providers-postgres
    apache-airflow-providers-slack
    

🌐 Official User Guide

For detailed usage and configuration, refer to the official documentation: 👉 Apache Airflow User Guide 👉 Astro CLI docs


🧑‍🏫 Reference Instructor

This project is inspired by courses from:

About

Hands-on Apache Airflow training with practical DAG examples and Docker-based setup.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published