The ChunkedSet stores its elements in Chunks of size k. It behaves as a set and contains only unique values in all chunks combined.

Text splitting example using Tiktoken

Tool for lossless Minecraft world compression.

Built a chatbot that answers questions from uploaded .txt files. The app stores the file content in a database, converts it into vector embeddings, and finds the most relevant parts when a user asks a question. It then uses OpenAI to generate an answer based only on the uploaded content.

Sleeps in chunks of 0.05 seconds (with status bar) / ctrl+c to exit without throwing an Exception

Moteur à base de règles (MBR) de segmentation de texte brut en chunks

A Streamlit-based AI chatbot designed to answer questions from PDF documents by combining semantic search and large language model generation.

A method to gzip, base64 encode and chunk local files into JSON format to fit in a single block on the Hive blockchain.

A chunking system for game developement

AI-powered PDF Q&A system using FastAPI, ChromaDB, and OpenAI. Extracts, indexes, and retrieves relevant text chunks to answer questions.

This is a script to create BlueMap shape markers around chunks from a world.

The easiest and most effective way tool to retrain a RAG LLM/GEN AI/Agent on your data

MultiProcessing In Python Using MAP Function

Python app which generates subs and plays them along with video

exports an mp4 video file from an m3u8 chunklist file on the internet

Helper scripts to split a large audio file into smaller chunks and annotate these chunks

Python package that provides an easy-to-use interface to access data from the Mars Orbital Data Explorer

Efficient celery tasks chunkification