Data ingestor that reads and parses executive orders from wikisource

A MongoDb to Elasticsearch ETL pipeline

Python based ingestion, SQL, Hadoop, Bash scripting

Ingestion Server for Magen Data Leak Prevention Software

Periodically ingest incremental updates (inserts / deletes) into BigQuery using Cloud Composer / Airflow orchestration workflow

Ingest data fetched from Snowflake to ElasticSearch

Archive of Python code snippets for data engineering & analytics processes

Ingest any format data into postgreSQL database

IBIS is a workflow creation-engine that abstracts the Hadoop internals of ingesting RDBMS data.

The aim of this project is automate data ingestion from flat files like CSV and compressed files GZIP into a database like Postgres. The entire setup is automated using Docker and is pretty fast too as multiprocessing is being used.

Ingest a file in GEDCOM format into MongoDB

Read-only mirror of https://gitlab.com/sorcero/community/ingestum

A simple python wrapper script based on pgfutter to load multiple dumped csv files into a postgres database.

Automate data ingestion into FiftyOne with Twilio

FastAPI server to process client request both for injection and interference

This project aims to predict smartphone prices using a combination of batch and stream processing techniques in a Big Data environment. The architecture follows the Lambda Architecture pattern, providing both real-time and batch processing capabilities to users.

Simple python interface for the Fivetran API. Powered by HTTPx.

This repository is dedicated to learning LangChain by creating a generative AI application. This web application uses Pinecone as a vector store to answer questions related to LangChain, utilizing sources from the official LangChain documentation.

Python script for ingesting various files into a semantic graph. For text, images, cpp, python, rust, javascript, and PDFs.