LIMIT-BERT

LIMIT-BERT: Linguistic Informed Multi-task BERT)

1. Requirements
2. Training

• Python 3.6 or higher.
• Cython 0.25.2 or any compatible version.
• PyTorch 1.0.0+.
• EVALB. Before starting, run make inside the EVALB/ directory to compile an evalb executable. This will be called from Python for evaluation.
• pytorch-transformers PyTorch 1.0.0+ or any compatible version.

The following pre-trained models are available for download from Google Drive:

• LIMIT-BERT: PyTorch version, same setting with BERT-Large-WWM，loading model with pytorch-transformers.

To train LIMIT-BERT, simply run:

sh run_limitbert.sh

To test after setting model path:

sh test_bert.sh

@article{zhou2019limit,
  title={{LIMIT-BERT}: Linguistic informed multi-task {BERT}},
  author={Zhou, Junru and Zhang, Zhuosheng and Zhao, Hai},
  journal={arXiv preprint arXiv:1910.14296},
  year={2019}
}