LLMFoundry is a Python package for training, finetuning, evaluating, and serving large scale LLM models on distributed compute infrustructure using MosaicML's Composer with PyTorch
At a granular level, LLMFoundry is a library that consists of the following components:
llmfoundry.models.mpt.MPTModel
- a simple PyTorch GPT model, wrapped inComposerModel
, that can scale up to 70B+ parametersllmfoundry.models.layers
- a collection of layers used in the MPTModelllmfoundry.models.hf
- a collection of tools which enables training / finetuning huggingface models with../scripts/train/train.py
llmfoundry.data.text_data.StreamingTextDataset
- a MosaicML streaming dataset that can be used with a vanilla PyTorch dataloader.llmfoundry.data.finetuning.collator.Seq2SeqFinetuningCollator
- a dataloader for different finetuning tasksllmfoundry.optim
- a collection of optimizers used for training LLMs (PyTorch and Composer optimizers are also compatible)llmfoundry.utils.builders
- a collection of convenient string-to-object mappings used to create objects that get passed to the Composer Trainer.