Lib.rs

› Keywords #llm #gpu #async-concurrency #inference #inference-server #database

#batching

  1. batch-aint-one

    I got 99 problems, but a batch ain't one

    v0.12.0 1.7K #async-concurrency #batch #batching #concurrency

  2. scuffle-batching

    Optimized batching and dataloading for external services

    v0.1.6 800 #dataloading #batching #external #async

  3. fusillade

    A daemon implementation for sending batched LLM requests with efficient request coalescing

    v17.0.2 #daemon #http-request #batch-file #batched #batching #template-file #request-file #openai #llm #postgresql

  4. opendata-buffer

    Stateless object storage buffer library for OpenData systems

    v0.2.0 #streaming #database #batching #buffer

  5. batchy

    Transparently batch concurrent requests into efficient bulk operations

    v0.2.1 #batching #async-concurrency #ml #optimization

  6. unillm-scheduler

    Request scheduling with continuous batching for UniLLM

    v0.1.0 #llm-inference #uni-llm #run-time #request #batching

  7. opendata-ingest

    Stateless object storage ingestion library for OpenData systems

    v0.1.3 #streaming #database #batching #ingest

  8. obs-sink-batch

    Generic batching sink framework for obs: triggers + retry + spool + escalation with a pluggable BatchBackend trait

    v0.1.0 #obs #batching-sink #observability #sink #batching

  9. rumus-serve

    High-throughput inference server for RUMUS with continuous batching

    v0.1.0 #inference #server #inference-server #rumus #batching #high-throughput #attention #gpu #masking #pad

  10. yatch

    🦇 Experimental batching on mpsc channels with VecDeque in Rust

    v0.1.0 #mpsc #vec-deque #experimental #batching

  11. trustformers-serve

    High-performance inference server for TrustformeRS models

    v0.1.1 #inference #batching #llm #inference-server #server

  12. relayrl_env_trait

    Traits for training and testing environments in the RelayRL framework

    v1.2.0 #deep-learning #environment #batching #integration

  13. atomr-accel-patterns

    Universal GPU actor blueprints for atomr-accel-cuda: DynamicBatchingServer, InferenceCascade, ModelReplicaPool, FairShareScheduler, ModelHotSwapServer, SpeculativeDecoder, MoeRouter

    v0.10.0 #gpu #inference #batching #actor

  14. batched

    rust macro util for batching expensive operations

    v0.2.11 #batching #batch #performance #efficient

  15. rakka-accel-patterns

    Universal GPU actor blueprints for rakka-accel-cuda: DynamicBatchingServer, InferenceCascade, ModelReplicaPool, FairShareScheduler, ModelHotSwapServer, SpeculativeDecoder, MoeRouter

    v0.2.9 #gpu #actor #inference #batching

  16. chunk_norris

    splitting large text into smaller batches for LLM input

    v0.2.1 #batching #llm #nlp

  17. vil_embedder

    VIL Embedding Engine — high-performance text embeddings with batching and SIMD similarity

    v0.4.0 #simd #text-embedding #onnx #vil #batching #openai #distributed-systems #vector-similarity #language-framework #zero-copy

  18. batch_queue

    A single-producer single-consumer Rust queue with smart batching

    v0.1.0 #queue #spsc #batching #spsc-channel

  19. vil_inference

    VIL Inference Server — model pool, dynamic batching, hot-swap

    v0.4.0 #vil #inference-server #dynamic #distributed-systems #batching #language-framework #zero-copy

  20. request_compiler

    Request Compiler Optimized for Batching Requests

    v0.1.1 #compiler #requests #batching #optimized

    21. Try searching with DuckDuckGo.

  21. batcher

    A very basic batching library

    v0.1.2 #batching

  22. vcf_batcher

    Command-line tool to cut VCF (variant call files) into smaller batches, intended to be used for multiprocessing or distributed computing

    v0.2.1 #vcf #batch #variant-call-file #batching

  23. atomic-batcher

    batching function that allows you to atomically batch a series of operations

    v0.8.2 #batch #series-of-operations #batching #atomically #operation

  24. batched_derive

    rust macro util for batching expensive operations

    v0.2.11 #batching #batch #performance #efficient

  25. batch-loader

    The contracts and implementation for query batching

    v0.1.0 #query #contract #batching