Ranking the Transferability of Adversarial Examples
Abstract
1 Introduction
![](/cms/10.1145/3670409/asset/75296710-2f48-43b7-b128-e5513b4da5b2/assets/images/medium/tist-2023-05-0186-f01.jpg)
2 Definitions
2.1 ET
2.2 Transferability Ranking
2.3 Transferability at k
3 Implementation
3.1 Approximate ET (AET)
3.2 Heuristical ET (HET)
![](/cms/10.1145/3670409/asset/d214ff95-ad3e-442d-8781-0dbecb1127cb/assets/images/medium/tist-2023-05-0186-f02.jpg)
3.3 Blackbox Ranking Strategies
![](/cms/10.1145/3670409/asset/6101d8ae-013e-4bb4-868c-00eeec24e353/assets/images/medium/tist-2023-05-0186-f03.jpg)
4 Experiment Setup
4.1 Evaluation Measures
4.2 Datasets
4.3 Architectures
4.4 Threat Model
4.5 Attack Algorithms
4.6 Ranking Algorithms
4.7 Environment and Reproducibility
4.8 Experiments
5 Experiment Results
5.1 E1—Sample Ranking
5.1.1 E1.1—Architectures.
![](/cms/10.1145/3670409/asset/a42e6e96-e651-4805-bd22-12920bdd492c/assets/images/medium/tist-2023-05-0186-f04.jpg)
![](/cms/10.1145/3670409/asset/c5c29a23-9a57-40c8-ab56-e8d6ceeb97f7/assets/images/medium/tist-2023-05-0186-f05.jpg)
5.1.2 E1.2—Attacks.
Surrogate | Victim | Lower \(B\) | PGD | Momentum | FGSM | Upper \(B\) | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
5% | 10% | 20% | 5% | 10% | 20% | 5% | 10% | 20% | 5% | 10% | 20% | ||||
ImageNet | ViT_b_16 | ViT_l_16 | 0.451 | 0.997 | 0.987 | 0.954 | 0.995 | 0.986 | 0.946 | 1.000 | 0.997 | 0.982 | 1.000 | 1.000 | 1.000 |
DenseNet121 | 0.394 | 0.990 | 0.979 | 0.934 | 0.990 | 0.978 | 0.926 | 0.994 | 0.990 | 0.968 | 1.000 | 1.000 | 1.000 | ||
Efficientnet-b2 | 0.344 | 0.994 | 0.971 | 0.904 | 0.994 | 0.967 | 0.893 | 0.997 | 0.989 | 0.948 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | 0.442 | 0.995 | 0.983 | 0.943 | 0.995 | 0.984 | 0.937 | 0.999 | 0.992 | 0.973 | 1.000 | 1.000 | 1.000 | ||
Swin_s | 0.290 | 0.934 | 0.871 | 0.780 | 0.928 | 0.861 | 0.767 | 0.954 | 0.912 | 0.844 | 1.000 | 1.000 | 1.000 | ||
DenseNet121 | ViT_b_16 | 0.215 | 0.971 | 0.925 | 0.792 | 0.975 | 0.928 | 0.797 | 0.975 | 0.933 | 0.818 | 1.000 | 1.000 | 1.000 | |
DenseNet161 | 0.350 | 0.993 | 0.981 | 0.932 | 0.996 | 0.984 | 0.940 | 0.996 | 0.986 | 0.947 | 1.000 | 1.000 | 1.000 | ||
Efficientnet-b2 | 0.279 | 0.989 | 0.962 | 0.873 | 0.991 | 0.967 | 0.881 | 0.989 | 0.970 | 0.893 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | 0.404 | 0.992 | 0.976 | 0.926 | 0.993 | 0.979 | 0.934 | 0.995 | 0.983 | 0.942 | 1.000 | 1.000 | 1.000 | ||
Swin_s | 0.202 | 0.948 | 0.879 | 0.749 | 0.949 | 0.888 | 0.756 | 0.953 | 0.898 | 0.778 | 1.000 | 1.000 | 1.000 | ||
Efficientnet-b2 | ViT_b_16 | 0.234 | 0.977 | 0.925 | 0.806 | 0.973 | 0.924 | 0.802 | 0.985 | 0.945 | 0.858 | 1.000 | 1.000 | 1.000 | |
DenseNet121 | 0.348 | 0.991 | 0.973 | 0.908 | 0.992 | 0.972 | 0.906 | 0.995 | 0.986 | 0.944 | 1.000 | 1.000 | 1.000 | ||
Efficientnet_b1 | 0.349 | 0.996 | 0.982 | 0.933 | 0.996 | 0.984 | 0.933 | 0.999 | 0.994 | 0.968 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | 0.399 | 0.993 | 0.978 | 0.926 | 0.993 | 0.978 | 0.923 | 0.997 | 0.987 | 0.956 | 1.000 | 1.000 | 1.000 | ||
Swin_s | 0.226 | 0.951 | 0.889 | 0.771 | 0.952 | 0.893 | 0.768 | 0.968 | 0.919 | 0.832 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | ViT_b_16 | 0.214 | 0.974 | 0.925 | 0.791 | 0.976 | 0.926 | 0.792 | 0.979 | 0.935 | 0.811 | 1.000 | 1.000 | 1.000 | |
DenseNet121 | 0.355 | 0.990 | 0.973 | 0.914 | 0.992 | 0.977 | 0.920 | 0.994 | 0.980 | 0.927 | 1.000 | 1.000 | 1.000 | ||
Efficientnet-b2 | 0.274 | 0.988 | 0.962 | 0.869 | 0.991 | 0.965 | 0.875 | 0.987 | 0.969 | 0.888 | 1.000 | 1.000 | 1.000 | ||
Resnet34 | 0.391 | 0.996 | 0.988 | 0.946 | 0.996 | 0.990 | 0.950 | 0.997 | 0.991 | 0.957 | 1.000 | 1.000 | 1.000 | ||
Swin_s | 0.196 | 0.951 | 0.880 | 0.748 | 0.951 | 0.887 | 0.749 | 0.956 | 0.889 | 0.763 | 1.000 | 1.000 | 0.981 | ||
Swin_s | ViT_b_16 | 0.251 | 0.940 | 0.882 | 0.769 | 0.937 | 0.884 | 0.769 | 0.966 | 0.932 | 0.863 | 1.000 | 1.000 | 1.000 | |
DenseNet121 | 0.349 | 0.993 | 0.975 | 0.914 | 0.992 | 0.974 | 0.913 | 0.997 | 0.990 | 0.964 | 1.000 | 1.000 | 1.000 | ||
Efficientnet-b2 | 0.308 | 0.983 | 0.958 | 0.881 | 0.985 | 0.960 | 0.883 | 0.997 | 0.984 | 0.951 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | 0.397 | 0.995 | 0.981 | 0.934 | 0.996 | 0.982 | 0.934 | 0.998 | 0.992 | 0.971 | 1.000 | 1.000 | 1.000 | ||
Swin_t | 0.602 | 1.000 | 0.996 | 0.984 | 1.000 | 0.998 | 0.987 | 0.999 | 1.000 | 0.997 | 1.000 | 1.000 | 1.000 | ||
CIFAR10 | ViT | ViT | 0.769 | 0.979 | 0.968 | 0.957 | 0.720 | 0.543 | 0.337 | 0.728 | 0.562 | 0.368 | 1.000 | 1.000 | 1.000 |
DenseNet121 | 0.077 | 0.723 | 0.536 | 0.332 | 0.981 | 0.968 | 0.956 | 0.979 | 0.962 | 0.942 | 1.000 | 0.766 | 0.383 | ||
Efficientnet-b0 | 0.136 | 0.808 | 0.692 | 0.483 | 0.795 | 0.689 | 0.494 | 0.817 | 0.717 | 0.541 | 1.000 | 1.000 | 0.680 | ||
Resnet18 | 0.065 | 0.614 | 0.449 | 0.277 | 0.576 | 0.441 | 0.277 | 0.622 | 0.470 | 0.299 | 1.000 | 0.647 | 0.324 | ||
Swin_s | 0.219 | 0.831 | 0.746 | 0.624 | 0.834 | 0.751 | 0.627 | 0.854 | 0.780 | 0.672 | 1.000 | 1.000 | 1.000 | ||
DenseNet121 | ViT | 0.290 | 0.781 | 0.721 | 0.611 | 0.787 | 0.723 | 0.609 | 0.823 | 0.775 | 0.713 | 1.000 | 1.000 | 1.000 | |
DenseNet121 | 0.510 | 0.984 | 0.962 | 0.903 | 0.988 | 0.970 | 0.919 | 0.986 | 0.964 | 0.915 | 1.000 | 1.000 | 1.000 | ||
Efficientnet-b0 | 0.283 | 0.910 | 0.813 | 0.697 | 0.910 | 0.841 | 0.724 | 0.955 | 0.896 | 0.828 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | 0.286 | 0.827 | 0.710 | 0.635 | 0.849 | 0.745 | 0.669 | 0.899 | 0.816 | 0.719 | 1.000 | 1.000 | 1.000 | ||
Swin_s | 0.177 | 0.765 | 0.662 | 0.537 | 0.789 | 0.671 | 0.548 | 0.868 | 0.818 | 0.703 | 1.000 | 1.000 | 0.883 | ||
Efficientnet-b0 | ViT | 0.291 | 0.808 | 0.715 | 0.603 | 0.803 | 0.725 | 0.611 | 0.796 | 0.732 | 0.629 | 1.000 | 1.000 | 1.000 | |
DenseNet121 | 0.355 | 0.954 | 0.892 | 0.799 | 0.996 | 0.992 | 0.969 | 0.987 | 0.968 | 0.939 | 1.000 | 1.000 | 1.000 | ||
Efficientnet-b0 | 0.641 | 0.994 | 0.990 | 0.971 | 0.952 | 0.890 | 0.803 | 0.919 | 0.843 | 0.709 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | 0.392 | 0.936 | 0.865 | 0.788 | 0.940 | 0.859 | 0.786 | 0.912 | 0.827 | 0.715 | 1.000 | 1.000 | 1.000 | ||
Swin_s | 0.208 | 0.826 | 0.728 | 0.600 | 0.827 | 0.726 | 0.597 | 0.873 | 0.768 | 0.634 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | ViT | 0.271 | 0.786 | 0.729 | 0.602 | 0.788 | 0.740 | 0.600 | 0.801 | 0.778 | 0.695 | 1.000 | 1.000 | 1.000 | |
DenseNet121 | 0.204 | 0.788 | 0.685 | 0.570 | 0.910 | 0.847 | 0.717 | 0.979 | 0.909 | 0.828 | 1.000 | 1.000 | 1.000 | ||
Efficientnet-b0 | 0.239 | 0.854 | 0.773 | 0.644 | 0.818 | 0.710 | 0.593 | 0.900 | 0.815 | 0.715 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | 0.251 | 0.890 | 0.812 | 0.697 | 0.852 | 0.789 | 0.666 | 0.940 | 0.883 | 0.801 | 1.000 | 1.000 | 1.000 | ||
Swin_s | 0.146 | 0.745 | 0.645 | 0.505 | 0.737 | 0.653 | 0.507 | 0.815 | 0.757 | 0.643 | 1.000 | 1.000 | 0.732 | ||
Swin_s | ViT | 0.390 | 0.809 | 0.734 | 0.687 | 0.815 | 0.743 | 0.680 | 0.853 | 0.784 | 0.720 | 1.000 | 1.000 | 1.000 | |
DenseNet121 | 0.155 | 0.850 | 0.733 | 0.548 | 0.988 | 0.975 | 0.927 | 0.993 | 0.983 | 0.952 | 1.000 | 1.000 | 0.778 | ||
Efficientnet-b0 | 0.253 | 0.909 | 0.837 | 0.710 | 0.874 | 0.740 | 0.550 | 0.904 | 0.777 | 0.607 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | 0.121 | 0.765 | 0.608 | 0.447 | 0.916 | 0.849 | 0.706 | 0.942 | 0.882 | 0.756 | 1.000 | 1.000 | 0.604 | ||
Swin_s | 0.526 | 0.988 | 0.973 | 0.927 | 0.753 | 0.608 | 0.445 | 0.795 | 0.659 | 0.493 | 1.000 | 1.000 | 1.000 |
Surrogate | Victim | Lower \(B\) | PGD | Momentum | FGSM | Upper \(B\) | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
5% | 10% | 20% | 5% | 10% | 20% | 5% | 10% | 20% | 5% | 10% | 20% | ||||
X-Ray | ViT | ViT | 0.469 | 0.976 | 0.943 | 0.900 | 0.973 | 0.944 | 0.899 | 0.972 | 0.958 | 0.918 | 1.000 | 1.000 | 1.000 |
DenseNet121 | 0.280 | 0.931 | 0.863 | 0.746 | 0.919 | 0.868 | 0.745 | 0.941 | 0.887 | 0.770 | 1.000 | 1.000 | 1.000 | ||
Efficientnet-b2 | 0.274 | 0.955 | 0.898 | 0.774 | 0.955 | 0.889 | 0.770 | 0.947 | 0.907 | 0.797 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | 0.278 | 0.943 | 0.883 | 0.761 | 0.946 | 0.878 | 0.751 | 0.957 | 0.901 | 0.781 | 1.000 | 1.000 | 1.000 | ||
Swin_s | 0.284 | 0.946 | 0.901 | 0.791 | 0.943 | 0.881 | 0.779 | 0.963 | 0.918 | 0.812 | 1.000 | 1.000 | 1.000 | ||
DenseNet121 | ViT | 0.247 | 0.885 | 0.834 | 0.707 | 0.883 | 0.792 | 0.678 | 0.895 | 0.820 | 0.717 | 1.000 | 1.000 | 1.000 | |
DenseNet121 | 0.850 | 0.989 | 0.991 | 0.990 | 0.996 | 0.995 | 0.995 | 0.992 | 0.994 | 0.994 | 1.000 | 1.000 | 1.000 | ||
Efficientnet-b2 | 0.436 | 0.933 | 0.898 | 0.852 | 0.905 | 0.890 | 0.826 | 0.891 | 0.870 | 0.803 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | 0.449 | 0.981 | 0.932 | 0.878 | 0.985 | 0.956 | 0.919 | 0.971 | 0.948 | 0.913 | 1.000 | 1.000 | 1.000 | ||
Swin_s | 0.287 | 0.937 | 0.902 | 0.802 | 0.952 | 0.920 | 0.816 | 0.946 | 0.910 | 0.835 | 1.000 | 1.000 | 1.000 | ||
Efficientnet-b2 | ViT | 0.222 | 0.893 | 0.800 | 0.691 | 0.898 | 0.806 | 0.680 | 0.904 | 0.856 | 0.717 | 1.000 | 1.000 | 1.000 | |
DenseNet121 | 0.310 | 0.943 | 0.897 | 0.807 | 0.952 | 0.929 | 0.850 | 0.976 | 0.944 | 0.859 | 1.000 | 1.000 | 1.000 | ||
Efficientnet-b2 | 0.992 | 1.000 | 1.000 | 1.000 | 0.990 | 0.995 | 0.985 | 0.996 | 0.992 | 0.987 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | 0.324 | 0.940 | 0.897 | 0.820 | 0.952 | 0.920 | 0.849 | 0.976 | 0.934 | 0.888 | 1.000 | 1.000 | 1.000 | ||
Swin_s | 0.249 | 0.933 | 0.883 | 0.774 | 0.925 | 0.901 | 0.802 | 0.940 | 0.914 | 0.819 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | ViT | 0.238 | 0.892 | 0.809 | 0.677 | 0.879 | 0.800 | 0.673 | 0.896 | 0.812 | 0.696 | 1.000 | 1.000 | 1.000 | |
DenseNet121 | 0.445 | 0.956 | 0.927 | 0.867 | 0.966 | 0.958 | 0.912 | 0.989 | 0.972 | 0.918 | 1.000 | 1.000 | 1.000 | ||
Efficientnet-b2 | 0.438 | 0.959 | 0.924 | 0.875 | 0.956 | 0.903 | 0.851 | 0.959 | 0.924 | 0.863 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | 0.953 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 0.998 | 0.997 | 1.000 | 1.000 | 1.000 | ||
Swin_s | 0.282 | 0.916 | 0.890 | 0.794 | 0.929 | 0.904 | 0.803 | 0.933 | 0.911 | 0.835 | 1.000 | 1.000 | 1.000 | ||
Swin_s | ViT | 0.225 | 0.898 | 0.789 | 0.645 | 0.886 | 0.796 | 0.630 | 0.909 | 0.828 | 0.668 | 1.000 | 1.000 | 1.000 | |
DenseNet121 | 0.350 | 0.961 | 0.907 | 0.795 | 0.961 | 0.915 | 0.811 | 0.973 | 0.934 | 0.855 | 1.000 | 1.000 | 1.000 | ||
Efficientnet-b2 | 0.367 | 0.976 | 0.916 | 0.836 | 0.973 | 0.936 | 0.862 | 0.985 | 0.965 | 0.909 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | 0.343 | 0.964 | 0.909 | 0.801 | 0.964 | 0.918 | 0.835 | 0.973 | 0.944 | 0.882 | 1.000 | 1.000 | 1.000 | ||
Swin_s | 0.967 | 1.000 | 1.000 | 0.999 | 1.000 | 1.000 | 1.000 | 1.000 | 0.998 | 0.998 | 1.000 | 1.000 | 1.000 | ||
Road Sign | ViT | ViT | 0.207 | 0.500 | 0.800 | 0.545 | 0.667 | 0.857 | 0.500 | 0.000 | 0.667 | 0.857 | 1.000 | 1.000 | 1.000 |
DenseNet121 | 0.121 | 0.500 | 0.600 | 0.455 | 0.667 | 0.571 | 0.357 | 0.000 | 0.333 | 0.714 | 1.000 | 1.000 | 0.636 | ||
Efficientnet-b2 | 0.224 | 1.000 | 0.800 | 0.818 | 0.667 | 0.857 | 0.714 | 1.000 | 0.667 | 0.857 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | 0.121 | 1.000 | 0.600 | 0.636 | 0.667 | 0.571 | 0.500 | 1.000 | 1.000 | 0.571 | 1.000 | 1.000 | 0.636 | ||
Swin_s | 0.207 | 1.000 | 1.000 | 0.909 | 0.667 | 0.857 | 0.643 | 0.000 | 0.667 | 0.857 | 1.000 | 1.000 | 1.000 | ||
DenseNet121 | ViT | 0.017 | 0.200 | 0.100 | 0.067 | 0.188 | 0.094 | 0.047 | 0.333 | 0.500 | 0.231 | 0.333 | 0.167 | 0.083 | |
DenseNet121 | 0.363 | 0.733 | 0.633 | 0.483 | 0.750 | 0.656 | 0.609 | 0.667 | 0.667 | 0.846 | 1.000 | 1.000 | 1.000 | ||
Efficientnet-b2 | 0.110 | 0.667 | 0.500 | 0.367 | 0.688 | 0.500 | 0.422 | 0.667 | 0.833 | 0.692 | 1.000 | 1.000 | 0.550 | ||
Resnet18 | 0.037 | 0.600 | 0.300 | 0.167 | 0.563 | 0.313 | 0.172 | 1.000 | 0.833 | 0.692 | 0.733 | 0.367 | 0.183 | ||
Swin_s | 0.033 | 0.600 | 0.333 | 0.167 | 0.563 | 0.313 | 0.156 | 0.667 | 0.667 | 0.538 | 0.667 | 0.333 | 0.167 | ||
Efficientnet-b2 | ViT | 0.010 | 0.200 | 0.097 | 0.048 | 0.200 | 0.097 | 0.048 | 0.600 | 0.273 | 0.130 | 0.200 | 0.097 | 0.048 | |
DenseNet121 | 0.026 | 0.533 | 0.258 | 0.129 | 0.467 | 0.323 | 0.159 | 0.600 | 0.636 | 0.435 | 0.533 | 0.258 | 0.129 | ||
Efficientnet-b2 | 0.753 | 1.000 | 0.968 | 0.935 | 1.000 | 1.000 | 0.968 | 0.800 | 0.909 | 0.870 | 1.000 | 1.000 | 1.000 | ||
Resnet18 | 0.032 | 0.600 | 0.323 | 0.161 | 0.600 | 0.355 | 0.175 | 1.000 | 0.727 | 0.391 | 0.667 | 0.323 | 0.161 | ||
Swin_s | 0.032 | 0.533 | 0.323 | 0.161 | 0.467 | 0.290 | 0.143 | 0.600 | 0.455 | 0.304 | 0.667 | 0.323 | 0.161 | ||
Resnet18 | ViT | 0.013 | 0.158 | 0.105 | 0.065 | 0.150 | 0.075 | 0.062 | 0.167 | 0.250 | 0.120 | 0.263 | 0.132 | 0.065 | |
DenseNet121 | 0.026 | 0.421 | 0.237 | 0.130 | 0.400 | 0.225 | 0.123 | 0.500 | 0.583 | 0.320 | 0.526 | 0.263 | 0.130 | ||
Efficientnet-b2 | 0.090 | 0.684 | 0.526 | 0.377 | 0.650 | 0.475 | 0.370 | 1.000 | 0.833 | 0.600 | 1.000 | 0.921 | 0.455 | ||
Resnet18 | 0.404 | 0.842 | 0.789 | 0.688 | 1.000 | 0.950 | 0.753 | 0.833 | 0.833 | 0.680 | 1.000 | 1.000 | 1.000 | ||
Swin_s | 0.026 | 0.368 | 0.263 | 0.130 | 0.450 | 0.250 | 0.123 | 0.500 | 0.333 | 0.400 | 0.526 | 0.263 | 0.130 | ||
Swin_s | ViT | 0.014 | 0.150 | 0.122 | 0.060 | 0.130 | 0.109 | 0.054 | 0.143 | 0.200 | 0.161 | 0.300 | 0.146 | 0.072 | |
DenseNet121 | 0.022 | 0.450 | 0.220 | 0.108 | 0.435 | 0.217 | 0.109 | 0.857 | 0.467 | 0.323 | 0.450 | 0.220 | 0.108 | ||
Efficientnet-b2 | 0.072 | 0.650 | 0.439 | 0.325 | 0.739 | 0.543 | 0.359 | 0.714 | 0.667 | 0.516 | 1.000 | 0.732 | 0.361 | ||
Resnet18 | 0.027 | 0.450 | 0.268 | 0.133 | 0.435 | 0.239 | 0.120 | 0.714 | 0.467 | 0.355 | 0.550 | 0.268 | 0.133 | ||
Swin_s | 0.275 | 0.800 | 0.683 | 0.590 | 0.870 | 0.717 | 0.598 | 0.857 | 0.733 | 0.548 | 1.000 | 1.000 | 1.000 |
5.2 E2—Perturbation Ranking
5.2.1 E2.1—Performance.
![](/cms/10.1145/3670409/asset/b74a0b06-9d12-4711-89f4-6956e8caa445/assets/images/medium/tist-2023-05-0186-f06.jpg)
Surrogate | Victim | Lower \(B\) | SoftMax | SoftMax\(+\)Noise | HET | Upper \(B\) | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
5% | 10% | 20% | 5% | 10% | 20% | 5% | 10% | 20% | 5% | 10% | 20% | ||||
ImageNet | ViT_b_16 | ViT_l_16 | 0.40 | 0.40 | 0.60 | 0.45 | 0.20 | 0.30 | 0.30 | 1.00 | 1.00 | 0.95 | 1.00 | 1.00 | 1.00 |
DenseNet121 | 0.31 | 0.40 | 0.50 | 0.40 | 0.40 | 0.40 | 0.40 | 1.00 | 1.00 | 0.85 | 1.00 | 1.00 | 1.00 | ||
Efficientnet-b2 | 0.27 | 0.20 | 0.10 | 0.25 | 0.20 | 0.30 | 0.30 | 1.00 | 0.90 | 0.80 | 1.00 | 1.00 | 1.00 | ||
Resnet18 | 0.37 | 0.40 | 0.40 | 0.45 | 0.40 | 0.40 | 0.35 | 1.00 | 1.00 | 0.90 | 1.00 | 1.00 | 1.00 | ||
Swin_s | 0.22 | 0.40 | 0.50 | 0.35 | 0.20 | 0.30 | 0.25 | 0.80 | 0.80 | 0.65 | 1.00 | 1.00 | 0.99 | ||
DenseNet121 | ViT_b_16 | 0.23 | 0.20 | 0.30 | 0.30 | 0.20 | 0.20 | 0.30 | 1.00 | 0.80 | 0.65 | 1.00 | 1.00 | 0.99 | |
DenseNet161 | 0.38 | 0.60 | 0.40 | 0.45 | 0.40 | 0.50 | 0.50 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | ||
Efficientnet-b2 | 0.28 | 0.20 | 0.20 | 0.35 | 0.20 | 0.30 | 0.35 | 1.00 | 1.00 | 0.80 | 1.00 | 1.00 | 1.00 | ||
Resnet18 | 0.47 | 0.40 | 0.40 | 0.45 | 0.20 | 0.30 | 0.40 | 1.00 | 0.90 | 0.90 | 1.00 | 1.00 | 1.00 | ||
Swin_s | 0.19 | 0.20 | 0.20 | 0.20 | 0.20 | 0.20 | 0.20 | 0.80 | 0.80 | 0.70 | 1.00 | 1.00 | 0.93 | ||
Efficientnet-b2 | ViT_b_16 | 0.26 | 0.20 | 0.40 | 0.35 | 0.00 | 0.20 | 0.20 | 1.00 | 1.00 | 0.80 | 1.00 | 1.00 | 1.00 | |
DenseNet121 | 0.39 | 0.00 | 0.40 | 0.40 | 0.00 | 0.20 | 0.35 | 1.00 | 1.00 | 0.95 | 1.00 | 1.00 | 1.00 | ||
Efficientnet-b1 | 0.45 | 0.40 | 0.50 | 0.50 | 0.40 | 0.40 | 0.40 | 1.00 | 1.00 | 0.95 | 1.00 | 1.00 | 1.00 | ||
Resnet18 | 0.45 | 0.00 | 0.40 | 0.45 | 0.20 | 0.40 | 0.50 | 0.80 | 0.90 | 0.90 | 1.00 | 1.00 | 1.00 | ||
Swin_s | 0.23 | 0.20 | 0.30 | 0.30 | 0.00 | 0.00 | 0.15 | 1.00 | 0.90 | 0.75 | 1.00 | 1.00 | 0.99 | ||
Resnet18 | ViT_b_16 | 0.25 | 0.20 | 0.30 | 0.35 | 0.20 | 0.40 | 0.30 | 1.00 | 1.00 | 0.75 | 1.00 | 1.00 | 0.99 | |
DenseNet121 | 0.39 | 0.40 | 0.60 | 0.50 | 0.20 | 0.40 | 0.30 | 1.00 | 0.90 | 0.90 | 1.00 | 1.00 | 1.00 | ||
Efficientnet-b2 | 0.28 | 0.40 | 0.40 | 0.30 | 0.40 | 0.50 | 0.35 | 1.00 | 0.90 | 0.75 | 1.00 | 1.00 | 1.00 | ||
Resnet34 | 0.38 | 0.20 | 0.40 | 0.35 | 0.20 | 0.40 | 0.35 | 1.00 | 1.00 | 0.90 | 1.00 | 1.00 | 1.00 | ||
Swin_s | 0.19 | 0.00 | 0.20 | 0.15 | 0.20 | 0.30 | 0.20 | 1.00 | 0.80 | 0.70 | 1.00 | 1.00 | 0.93 | ||
Swin_s | ViT_b_16 | 0.25 | 0.40 | 0.30 | 0.35 | 0.40 | 0.40 | 0.25 | 1.00 | 0.80 | 0.75 | 1.00 | 1.00 | 1.00 | |
DenseNet121 | 0.30 | 0.40 | 0.30 | 0.40 | 0.40 | 0.40 | 0.30 | 1.00 | 1.00 | 0.90 | 1.00 | 1.00 | 1.00 | ||
Efficientnet-b2 | 0.26 | 0.40 | 0.30 | 0.30 | 0.20 | 0.30 | 0.25 | 1.00 | 0.90 | 0.75 | 1.00 | 1.00 | 1.00 | ||
Resnet18 | 0.36 | 0.40 | 0.50 | 0.50 | 0.60 | 0.50 | 0.40 | 1.00 | 1.00 | 0.80 | 1.00 | 1.00 | 1.00 | ||
Swin_t | 0.60 | 0.80 | 0.90 | 0.80 | 0.80 | 0.80 | 0.75 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | ||
CIFAR10 | ViT | ViT | 0.06 | 0.00 | 0.00 | 0.05 | 0.00 | 0.10 | 0.10 | 0.60 | 0.40 | 0.25 | 1.00 | 0.67 | 0.34 |
DenseNet121 | 0.71 | 0.80 | 0.70 | 0.75 | 0.80 | 0.80 | 0.80 | 0.60 | 0.70 | 0.85 | 1.00 | 1.00 | 1.00 | ||
Efficientnet-b0 | 0.16 | 0.20 | 0.20 | 0.20 | 0.20 | 0.20 | 0.20 | 0.80 | 0.80 | 0.60 | 1.00 | 1.00 | 0.81 | ||
Resnet18 | 0.06 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.05 | 0.60 | 0.50 | 0.30 | 1.00 | 0.64 | 0.32 | ||
Swin_s | 0.18 | 0.20 | 0.20 | 0.25 | 0.20 | 0.40 | 0.25 | 0.40 | 0.70 | 0.60 | 1.00 | 1.00 | 0.90 | ||
DenseNet121 | ViT | 0.32 | 0.40 | 0.30 | 0.20 | 0.60 | 0.60 | 0.35 | 0.80 | 0.90 | 0.70 | 1.00 | 1.00 | 1.00 | |
DenseNet121 | 0.72 | 1.00 | 0.80 | 0.80 | 0.80 | 0.70 | 0.75 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | ||
Efficientnet-b0 | 0.36 | 0.60 | 0.40 | 0.40 | 0.40 | 0.40 | 0.30 | 1.00 | 1.00 | 0.85 | 1.00 | 1.00 | 1.00 | ||
Resnet18 | 0.34 | 0.40 | 0.60 | 0.50 | 0.60 | 0.50 | 0.40 | 1.00 | 0.90 | 0.85 | 1.00 | 1.00 | 1.00 | ||
Swin_s | 0.18 | 0.40 | 0.30 | 0.20 | 0.40 | 0.30 | 0.25 | 1.00 | 0.70 | 0.60 | 1.00 | 1.00 | 0.84 | ||
Efficientnet-b0 | ViT | 0.34 | 0.20 | 0.20 | 0.25 | 0.20 | 0.20 | 0.30 | 1.00 | 0.90 | 0.70 | 1.00 | 1.00 | 1.00 | |
DenseNet121 | 0.65 | 1.00 | 0.90 | 0.80 | 1.00 | 0.80 | 0.75 | 1.00 | 1.00 | 0.90 | 1.00 | 1.00 | 1.00 | ||
Efficientnet-b0 | 0.32 | 0.20 | 0.10 | 0.15 | 0.40 | 0.30 | 0.30 | 1.00 | 1.00 | 0.75 | 1.00 | 1.00 | 1.00 | ||
Resnet18 | 0.40 | 0.40 | 0.40 | 0.45 | 0.80 | 0.60 | 0.40 | 1.00 | 0.90 | 0.85 | 1.00 | 1.00 | 1.00 | ||
Swin_s | 0.21 | 0.00 | 0.00 | 0.10 | 0.20 | 0.20 | 0.20 | 1.00 | 0.70 | 0.60 | 1.00 | 1.00 | 0.98 | ||
Resnet18 | ViT | 0.29 | 0.40 | 0.40 | 0.30 | 0.60 | 0.60 | 0.40 | 1.00 | 0.70 | 0.60 | 1.00 | 1.00 | 1.00 | |
DenseNet121 | 0.24 | 0.40 | 0.50 | 0.45 | 0.00 | 0.20 | 0.35 | 1.00 | 1.00 | 0.75 | 1.00 | 1.00 | 0.95 | ||
Efficientnet-b0 | 0.16 | 0.60 | 0.50 | 0.35 | 0.00 | 0.30 | 0.35 | 0.40 | 0.60 | 0.40 | 1.00 | 1.00 | 0.84 | ||
Resnet18 | 0.22 | 0.40 | 0.50 | 0.40 | 0.00 | 0.20 | 0.35 | 0.80 | 0.70 | 0.55 | 1.00 | 1.00 | 0.94 | ||
Swin_s | 0.14 | 0.40 | 0.30 | 0.20 | 0.20 | 0.10 | 0.20 | 0.60 | 0.60 | 0.45 | 1.00 | 1.00 | 0.75 | ||
Swin_s | ViT | 0.40 | 0.20 | 0.20 | 0.20 | 0.40 | 0.30 | 0.35 | 0.60 | 0.70 | 0.70 | 1.00 | 1.00 | 1.00 | |
DenseNet121 | 0.47 | 0.20 | 0.20 | 0.25 | 0.60 | 0.70 | 0.45 | 1.00 | 0.90 | 0.90 | 1.00 | 1.00 | 1.00 | ||
Efficientnet-b0 | 0.14 | 0.00 | 0.00 | 0.00 | 0.20 | 0.30 | 0.15 | 0.60 | 0.70 | 0.55 | 1.00 | 1.00 | 0.74 | ||
Resnet18 | 0.31 | 0.40 | 0.20 | 0.20 | 0.40 | 0.20 | 0.20 | 1.00 | 0.90 | 0.90 | 1.00 | 1.00 | 1.00 | ||
Swin_s | 0.11 | 0.00 | 0.00 | 0.05 | 0.20 | 0.10 | 0.05 | 1.00 | 0.50 | 0.55 | 1.00 | 0.99 | 0.57 |
Surrogate | Victim | Lower \(B\) | SoftMax | SoftMax | HET | Upper \(B\) | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
5% | 10% | 20% | 5% | 10% | 20% | 5% | 10% | 20% | 5% | 10% | 20% | ||||
X-Ray | ViT | ViT | 0.61 | 0.80 | 0.70 | 0.65 | 0.80 | 0.70 | 0.70 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
DenseNet121 | 0.34 | 0.40 | 0.30 | 0.35 | 0.20 | 0.20 | 0.25 | 1.00 | 0.80 | 0.75 | 1.00 | 1.00 | 1.00 | ||
Efficientnet-b2 | 0.53 | 0.60 | 0.70 | 0.55 | 0.60 | 0.60 | 0.55 | 0.80 | 0.70 | 0.75 | 1.00 | 1.00 | 1.00 | ||
Resnet18 | 0.36 | 0.60 | 0.50 | 0.45 | 0.40 | 0.40 | 0.40 | 1.00 | 1.00 | 0.90 | 1.00 | 1.00 | 1.00 | ||
Swin_s | 0.32 | 0.60 | 0.40 | 0.30 | 0.20 | 0.20 | 0.25 | 0.60 | 0.70 | 0.75 | 1.00 | 1.00 | 1.00 | ||
DenseNet121 | ViT | 0.21 | 0.20 | 0.20 | 0.25 | 0.40 | 0.40 | 0.45 | 0.80 | 0.60 | 0.45 | 1.00 | 1.00 | 0.99 | |
DenseNet121 | 0.83 | 0.80 | 0.80 | 0.80 | 0.60 | 0.80 | 0.70 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | ||
Efficientnet-b2 | 0.55 | 0.40 | 0.40 | 0.40 | 0.60 | 0.80 | 0.55 | 1.00 | 0.90 | 0.65 | 1.00 | 1.00 | 1.00 | ||
Resnet18 | 0.39 | 0.20 | 0.40 | 0.45 | 0.60 | 0.50 | 0.45 | 1.00 | 0.80 | 0.65 | 1.00 | 1.00 | 1.00 | ||
Swin_s | 0.24 | 0.00 | 0.10 | 0.25 | 0.40 | 0.40 | 0.35 | 0.80 | 0.80 | 0.55 | 1.00 | 1.00 | 0.95 | ||
Efficientnet-b2 | ViT | 0.21 | 0.20 | 0.30 | 0.30 | 0.40 | 0.20 | 0.15 | 1.00 | 0.70 | 0.60 | 1.00 | 1.00 | 0.97 | |
DenseNet121 | 0.26 | 0.40 | 0.30 | 0.30 | 0.20 | 0.20 | 0.10 | 1.00 | 1.00 | 0.75 | 1.00 | 1.00 | 0.99 | ||
Efficientnet-b2 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | ||
Resnet18 | 0.23 | 0.60 | 0.30 | 0.30 | 0.40 | 0.40 | 0.20 | 0.80 | 0.70 | 0.55 | 1.00 | 1.00 | 0.94 | ||
Swin_s | 0.18 | 0.40 | 0.30 | 0.35 | 0.20 | 0.20 | 0.15 | 0.80 | 0.60 | 0.50 | 1.00 | 1.00 | 0.88 | ||
Resnet18 | ViT | 0.21 | 0.00 | 0.10 | 0.20 | 0.20 | 0.20 | 0.30 | 1.00 | 0.70 | 0.40 | 1.00 | 1.00 | 0.99 | |
DenseNet121 | 0.55 | 0.80 | 0.40 | 0.45 | 0.40 | 0.50 | 0.60 | 1.00 | 1.00 | 0.90 | 1.00 | 1.00 | 1.00 | ||
Efficientnet-b2 | 0.59 | 0.60 | 0.70 | 0.60 | 0.40 | 0.50 | 0.45 | 1.00 | 0.80 | 0.65 | 1.00 | 1.00 | 1.00 | ||
Resnet18 | 0.99 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | ||
Swin_s | 0.25 | 0.20 | 0.40 | 0.30 | 0.20 | 0.10 | 0.10 | 0.80 | 0.60 | 0.40 | 1.00 | 1.00 | 0.99 | ||
Swin_s | ViT | 0.24 | 0.40 | 0.30 | 0.25 | 0.60 | 0.40 | 0.40 | 0.80 | 0.60 | 0.50 | 1.00 | 1.00 | 1.00 | |
DenseNet121 | 0.46 | 0.40 | 0.30 | 0.50 | 0.80 | 0.60 | 0.55 | 0.80 | 0.90 | 0.80 | 1.00 | 1.00 | 1.00 | ||
Efficientnet-b2 | 0.53 | 0.20 | 0.60 | 0.60 | 0.80 | 0.60 | 0.60 | 0.80 | 0.70 | 0.65 | 1.00 | 1.00 | 1.00 | ||
Resnet18 | 0.36 | 0.80 | 0.60 | 0.60 | 0.60 | 0.40 | 0.30 | 0.80 | 0.70 | 0.55 | 1.00 | 1.00 | 1.00 | ||
Swin_s | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | ||
Road Sign | ViT | ViT | 0.05 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.20 | 0.20 | 0.15 | 0.72 | 0.47 | 0.24 |
DenseNet121 | 0.01 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.20 | 0.10 | 0.05 | 0.20 | 0.10 | 0.05 | ||
Efficientnet-b2 | 0.06 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.60 | 0.50 | 0.30 | 0.91 | 0.56 | 0.28 | ||
Resnet18 | 0.03 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.40 | 0.20 | 0.15 | 0.59 | 0.29 | 0.15 | ||
Swin_s | 0.03 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.60 | 0.30 | 0.15 | 0.61 | 0.30 | 0.15 | ||
DenseNet121 | ViT | 0.01 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.20 | 0.10 | 0.05 | 0.20 | 0.10 | 0.05 | |
DenseNet121 | 0.31 | 0.20 | 0.10 | 0.20 | 0.00 | 0.20 | 0.15 | 0.80 | 0.80 | 0.55 | 1.00 | 1.00 | 0.97 | ||
Efficientnet-b2 | 0.06 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.60 | 0.50 | 0.30 | 1.00 | 0.61 | 0.30 | ||
Resnet18 | 0.03 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.40 | 0.20 | 0.15 | 0.67 | 0.34 | 0.17 | ||
Swin_s | 0.02 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.40 | 0.20 | 0.10 | 0.44 | 0.22 | 0.11 | ||
Efficientnet-b2 | ViT | 0.01 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.20 | 0.10 | 0.05 | 0.20 | 0.10 | 0.05 | |
DenseNet121 | 0.01 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.20 | 0.10 | 0.05 | 0.20 | 0.10 | 0.05 | ||
Efficientnet-b2 | 0.73 | 0.60 | 0.80 | 0.75 | 0.60 | 0.80 | 0.85 | 1.00 | 0.90 | 0.90 | 1.00 | 1.00 | 1.00 | ||
Resnet18 | 0.02 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.20 | 0.10 | 0.10 | 0.59 | 0.30 | 0.15 | ||
Swin_s | 0.02 | 0.00 | 0.00 | 0.00 | 0.20 | 0.10 | 0.05 | 0.40 | 0.20 | 0.10 | 0.40 | 0.20 | 0.10 | ||
Resnet18 | ViT | 0.01 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.20 | 0.10 | 0.05 | 0.20 | 0.10 | 0.05 | |
DenseNet121 | 0.01 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.20 | 0.10 | 0.05 | 0.27 | 0.13 | 0.07 | ||
Efficientnet-b2 | 0.05 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.60 | 0.30 | 0.25 | 0.98 | 0.60 | 0.30 | ||
Resnet18 | 0.54 | 0.40 | 0.40 | 0.35 | 0.20 | 0.20 | 0.20 | 1.00 | 0.90 | 0.90 | 1.00 | 1.00 | 1.00 | ||
Swin_s | 0.02 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.40 | 0.20 | 0.10 | 0.40 | 0.20 | 0.10 | ||
Swin_s | ViT | 0.01 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.20 | 0.10 | 0.05 | 0.20 | 0.10 | 0.05 | |
DenseNet121 | 0.01 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.20 | 0.10 | 0.05 | 0.20 | 0.10 | 0.05 | ||
Efficientnet-b2 | 0.07 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.80 | 0.60 | 0.35 | 1.00 | 0.71 | 0.36 | ||
Resnet18 | 0.04 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.40 | 0.30 | 0.20 | 0.64 | 0.32 | 0.16 | ||
Swin_s | 0.53 | 0.00 | 0.10 | 0.25 | 0.20 | 0.20 | 0.20 | 1.00 | 0.80 | 0.80 | 1.00 | 1.00 | 0.99 |
6 Related Work
7 Conclusion
Footnotes
References
Index Terms
- Ranking the Transferability of Adversarial Examples
Recommendations
A Compound Data Poisoning Technique with Significant Adversarial Effects on Transformer-based Sentiment Classification Tasks
Transformer-based models have demonstrated much success in various natural language processing tasks. However, they are often vulnerable to adversarial attacks, such as data poisoning, which can intentionally fool the model into generating incorrect ...
DeT: Defending Against Adversarial Examples via Decreasing Transferability
Cyberspace Safety and SecurityAbstractDeep neural networks (DNNs) have made great progress in recent years. Unfortunately, DNNs are found to be vulnerable to adversarial examples that are injected with elaborately crafted perturbations. In this paper, we propose a defense method named ...
Adversarial examples: A survey of attacks and defenses in deep learning-enabled cybersecurity systems
AbstractOver the last few years, the adoption of machine learning in a wide range of domains has been remarkable. Deep learning, in particular, has been extensively used to drive applications and services in specializations such as computer vision, ...
Highlights- A taxonomy of cybersecurity applications is established.
- Adversarial machine learning is systematically overviewed.
- An extensive, curated list of cybersecurity-related datasets is provided.
- Methods for generating adversarial ...
Comments
Information & Contributors
Information
Published In
![cover image ACM Transactions on Intelligent Systems and Technology](/cms/asset/fe2abc04-2236-46c1-a48a-b3e3e8853efc/3613688.cover.jpg)
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Zuckerman STEM Leadership
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 663Total Downloads
- Downloads (Last 12 months)663
- Downloads (Last 6 weeks)88
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in