Ongoing research training transformer language models at scale, including: BERT & GPT-2

Mengzi Pretrained Models

Data augmentation for NLP, accepted at EMNLP 2021 Findings

LIMIT-BERT: Linguistics Informed Multi-Task BERT (EMNLP 2020: Findings)