Replace 'hub' with 'ingest' in any GitHub URL to get a prompt-friendly extract of a codebase
-
Updated
Nov 11, 2025 - Python
#
ingestion
Here are 44 public repositories matching this topic...
IBIS is a workflow creation-engine that abstracts the Hadoop internals of ingesting RDBMS data.
workflow hadoop ingestion oozie sqoop sqoop2 workflow-automation workflow-scheduler hadoop-ecosystem hadoop-framework ibis cigna
-
Updated
Apr 13, 2022 - Python
Official NetBox Labs plugin for NetBox for Diode
-
Updated
Nov 5, 2025 - Python
Python script for ingesting various files into a semantic graph. For text, images, cpp, python, rust, javascript, and PDFs.
-
Updated
Sep 17, 2024 - Python
The aim of this project is automate data ingestion from flat files like CSV and compressed files GZIP into a database like Postgres. The entire setup is automated using Docker and is pretty fast too as multiprocessing is being used.
-
Updated
Oct 24, 2022 - Python
Tagbase is a data lifecycle management system for electronic timeseries sensor data. It supports different types of data and works with equipment from various manufacturers.
docker sql rest-api postgresql postgis interoperability oceanography ingestion insitu tagbase-server etuff
-
Updated
Oct 7, 2025 - Python
Python Open-source ETL tool for seamless data movement across PostgreSQL, MySQL, Redshift, BigQuery, S3, GCS, and CSV files, with yaml/json-based configuration.
-
Updated
May 25, 2025 - Python
This project aims to predict smartphone prices using a combination of batch and stream processing techniques in a Big Data environment. The architecture follows the Lambda Architecture pattern, providing both real-time and batch processing capabilities to users.
python java machine-learning streaming apache-spark spring-boot hbase pyspark hdfs ingestion apache-kafka postgresql-database powerbi batch-processing lambda-architecture apache-airflow big-data-projects
-
Updated
Apr 15, 2024 - Python
-
Updated
Jul 1, 2015 - Python
LogFlow é uma aplicação ETL (Extração, Transformação e Carregamento) especializada em processamento de logs
security kafka etl graylog logs log-collector ingestion winlogbeat log-management opensearch etl-pipeline
-
Updated
Apr 17, 2025 - Python
Ingest any format data into postgreSQL database
clean-code postgresql python3 easy-to-use ingestion easy postgresql-database quick csv-import ready-to-use ingestion-pipeline ready-to-run
-
Updated
Dec 12, 2021 - Python
A fully interactive tool designed to streamline your GitHub repository prompt generation process and facilitate RAG (Retrieval-Augmented Generation) workflows
github python open-source app ai repository code prompt ingestion rag llm langchain retrieval-augmented-generation
-
Updated
Jan 5, 2025 - Python
Python based ingestion, SQL, Hadoop, Bash scripting
-
Updated
Feb 21, 2018 - Python
Periodically ingest incremental updates (inserts / deletes) into BigQuery using Cloud Composer / Airflow orchestration workflow
bigquery airflow google composer cloud gcp operator dataflow ingestion periodic dag change-data inceremental merge-query bigqueryoperator dataflowtemplateoperator
-
Updated
Dec 12, 2019 - Python
Simple python interface for the Fivetran API. Powered by HTTPx.
python integration etl data-engineering api-wrapper dataops ingestion yaml-configuration iaac fivetran httpx
-
Updated
Jun 5, 2024 - Python
This project implements a full-stack data engineering solution that connects to the Spotify Web API to extract a user’s recently played tracks, stores the data in a PostgreSQL database, applies transformations using dbt, and delivers actionable insights via Metabase dashboards.
visualization python docker infrastructure analytics models postgresql spotify-api metabase data-engineering ingestion dbt transformation data-pipeline workflow-automation apache-airflow etl-pipeline dag-scheduling orquestration
-
Updated
Apr 7, 2025 - Python
Reliable, production-style data pipeline that ingests data from multiple sources (API, web scraper, CSV), cleans & standardizes with Pandas/NumPy, loads into SQLite, and surfaces insights through a Streamlit dashboard.
python api docker airflow dashboard etl analytics numpy pandas data-engineering web-scraping ingestion beautifulsoup data-pipeline
-
Updated
Oct 19, 2025 - Python
📊 Fetch free historical candlestick data from the Blofin Exchange API without API keys, saving years of data to CSV for analysis and research.
fetch python csv trading free ingestion candle fetcher historical-data historical csv-export candlestick ingestion-pipeline blofin blofin-exchange blofin-api
-
Updated
Nov 11, 2025 - Python
Improve this page
Add a description, image, and links to the ingestion topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the ingestion topic, visit your repo's landing page and select "manage topics."