The data scientist's open-source choice to scale, assess and maintain natural language data. Treat training data like a software artifact. You are one of the people we've built refinery for. refinery helps you to build better NLP models in a data-centric approach. Semi-automate your labeling, find low-quality subsets in your training data, and monitor your data in one place. refinery doesn't get rid of manual labeling, but it makes sure that your valuable time is spent well. Also, the makers of refinery currently work on integrations to other labeling tools, such that you can easily switch between different choices. refinery is a multi-repository project, you can find all integrated services in the architecture below. The app builds on top of Hugging Face and spaCy to leverage pre-built language models for your NLP tasks, as well as qdrant for neural search.
Features
- Extend your existing labeling approach
- Put structure into unstructured data
- (Semi-)automated labeling workflow for NLP tasks
- Extensive data management and monitoring
- Team workspaces in the managed version
- Improve collaboration