Deep-to-Bottom Weights Decay: A Systemic Knowledge Review Learning Technique for Transformer Layers in Knowledge Distillation.

Ankun Wang Feng Liu Zhen Huang 0006 Minghao Hu Dongsheng Li 0001 Yifan Chen Xinjia Xie Deep-to-Bottom Weights Decay: A Systemic Knowledge Review Learning Technique for Transformer Layers in Knowledge Distillation. 128-139 2022 KSEM (2) https://doi.org/10.1007/978-3-031-10986-7_11 conf/ksem/2022-2 db/conf/ksem/ksem2022-2.html#WangL0H0CX22