NEAT: Nonlinearity aware training for accurate, energy-efficient, and robust implementation of neural networks on 1T-1R crossbars

A Bhattacharjee, L Bhatnagar, Y Kim… - IEEE Transactions on …, 2021 - ieeexplore.ieee.org
IEEE Transactions on Computer-Aided Design of Integrated Circuits …, 2021ieeexplore.ieee.org
In this era of IoT, energy-efficient and adversarially secure implementation of deep neural
networks (DNNs) on hardware has become imperative. Memristive crossbars have emerged
as an energy-efficient component of deep learning hardware accelerators due to their
compact and efficient matrix-vector multiplication (MVM) implementation. However, they
suffer from nonidealities (such as, interconnect parasitics, device variations, and sneak
paths) introduced by their circuit topology that degrades computational accuracy. A 1T-1R …
In this era of IoT, energy-efficient and adversarially secure implementation of deep neural networks (DNNs) on hardware has become imperative. Memristive crossbars have emerged as an energy-efficient component of deep learning hardware accelerators due to their compact and efficient matrix-vector multiplication (MVM) implementation. However, they suffer from nonidealities (such as, interconnect parasitics, device variations, and sneak paths) introduced by their circuit topology that degrades computational accuracy. A 1T-1R synapse, adding a transistor (1T) in series with the memristive synapse (1R), has been proposed to mitigate sneak paths in a crossbar. However, we observe that the nonlinear characteristics of the transistor affect the overall conductance of the 1T-1R cell which in turn affects the MVM operation. This 1T-1R nonlinearity arising from the input voltage-dependent nonlinearity is not only difficult to model or formulate, but also causes a drastic performance degradation of DNNs when mapped to such crossbars. In this article, we first analyses the nonlinearity in ideal 1T-1R crossbars (excluding nonidealities, such as device variations and interconnect parasitics) and propose a novel nonlinearity aware training (NEAT) method to address the nonlinearities. Specifically, we first identify the range of network weights, which can be mapped into the 1T-1R cell within the linear operating region of the transistor. After that, we regularize the weights of neural networks to exist within the linear operating range by using an iterative training algorithm. Our iterative training significantly recovers the classification accuracy drop caused by the nonlinearity. Moreover, we find that each layer has a different weight distribution and in turn requires different gate voltage of transistor to guarantee linear operation. Based on this observation, we achieve energy efficiency while preserving classification accuracy by applying heterogeneous gate-voltage control to the 1T-1R cells across different layers. Finally, we conduct various experiments on CIFAR10 and CIFAR100 benchmark datasets to demonstrate the effectiveness of our NEAT. Overall, NEAT yields energy gain with less than 1% accuracy loss (with homogeneous gate control) when mapping ResNet18 networks on 1T-1R crossbars. Thereafter, we integrate the 1T-1R crossbars with various nonidealities. We show that NEAT leads to more adversarially robust mappings of DNNs onto nonideal 1T-1R crossbars than standard DNNs mapped directly onto 1R crossbars. In case of a VGG11 network on CIFAR100 dataset, we obtain % improvement in clean accuracy and %–8% & %–6% improvements in adversarial accuracies, respectively, for fast gradient sign method (FGSM) and projected gradient descent (PGD)-based adversarial attacks via NEAT on nonideal 64 64 crossbars, in comparison to standard DNNs.
ieeexplore.ieee.org
Showing the best result for this search. See all results