InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies.
pytorch
multi-modal
gemma
pipeline-parallelism
transformers-models
tensor-parallelism
llava
llm-training
internlm
flash-attention
zero3
llm-framework
sequence-parallelism
internlm2
ring-attention
deepspeed-ulysses
llama3
910b
-
Updated
Nov 7, 2024 - Python