Improving Knowledge Distillation Improving KD via reducing regularization. To train install requirments.txt run python train.py with the following arguments : teacher student learning rate pruning rate pruning mode batch size epochs optimizer temperature etc ..