Funnel

Getting Started

Software
- CUDA
- Python >= 3.7
- PyTorch <= 1.7.0
- Transformers 4.7.0 (Make sure your terminal can access Huggingface)
Hardware
- Vitis 2022.2
- Vivado 2022.2

Clone this repository
Download the CLOTH dataset from here to data/cloth (Note: There is a problem with one piece of data in CLOTH). GELU and SQuAD v1.1 don't need to be downloaded manually.
Create a virtual environment (recommendation: conda) with a Python version of at least 3.7.
Install dependent Python packages: pip install -r requirements.txt
Set relevant environment variables

a. export PROJ_ROOT=$PWD

b. export WANDB_DISABLED=true to disable wandb logging (optional)

Evaluate Funnel Accuracy

a. Train a origion model by scripts or download checkpoints from here (These checkpoints are all downloaded from Huggingface). You should download and place them in output by dataset name like location mode in Google Driver.

b. Evaluate the fine-tuned model by scripts.

Name		Name	Last commit message	Last commit date
Latest commit History 10 Commits
data		data
hardware		hardware
outputs		outputs
.gitignore		.gitignore
README.md		README.md
bench_cpu_gpu.py		bench_cpu_gpu.py
eval_dense_on_cloth.sh		eval_dense_on_cloth.sh
eval_dense_on_glue.sh		eval_dense_on_glue.sh
eval_dense_on_squad.sh		eval_dense_on_squad.sh
eval_sparse_on_cloth.sh		eval_sparse_on_cloth.sh
eval_sparse_on_glue.sh		eval_sparse_on_glue.sh
eval_sparse_on_squad.sh		eval_sparse_on_squad.sh
modeling_bert.py		modeling_bert.py
requirements.txt		requirements.txt
run_cloth.py		run_cloth.py
run_glue.py		run_glue.py
run_qa.py		run_qa.py
train_dense_on_cloth.sh		train_dense_on_cloth.sh
train_dense_on_glue.sh		train_dense_on_glue.sh
train_dense_on_squad.sh		train_dense_on_squad.sh
trainer_qa.py		trainer_qa.py
utils_qa.py		utils_qa.py