Skip to content
/ faiss-s3 Public

On-demand Faiss vector search from S3. Lazy-loads clusters without fetching entire index.

License

Notifications You must be signed in to change notification settings

at15/faiss-s3

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

52 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

faiss-s3

On-demand Faiss vector search from S3. Lazy-loads clusters without fetching entire index. Original implementation: https://github.com/at15/faiss/tree/at15/faiss-on-s3/demos/s3_ivf

TODO

  • implement filter on metadata
  • benchmark using larger dataset

Usage

  • Add how to build server
python3.13 -m venv .venv
source .venv/bin/activate
# Brings in boto3 and faiss-cpu
pip install -e ./client
# Generate embeddings
pip install sentence-transformers pandas

License

MIT

About

On-demand Faiss vector search from S3. Lazy-loads clusters without fetching entire index.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Contributors 2

  •  
  •