Skip to content

Split the vld1q_s8 + vzip1q_u64/vzip2q_u64 4x16c8 NEON I8MM GEMM/IGEMM microkernels into a separate _zip variant (VARIANT=ZIP) and update the existing c8-neoni8mm microkernels to use vld2q_dup_u64 for lane 0 instead of zero-initializing a loop-carried uint64x2x2_t register pair for vld2q_lane_u64(..., 0). - #11794

Open
copybara-service[bot] wants to merge 1 commit into
masterfrom
test_1005800878

Conversation

@copybara-service

Copy link
Copy Markdown
Contributor

Split the vld1q_s8 + vzip1q_u64/vzip2q_u64 4x16c8 NEON I8MM GEMM/IGEMM microkernels into a separate _zip variant (VARIANT=ZIP) and update the existing c8-neoni8mm microkernels to use vld2q_dup_u64 for lane 0 instead of zero-initializing a loop-carried uint64x2x2_t register pair for vld2q_lane_u64(..., 0).

Benchmark on qs8_qc8w_gemm_minmax_fp32_ukernel_4x16c8__neoni8mm (MobileNetV2 M=49, N=1280, K=320, GOPS):

uarch                   lane (GOPS)     dup (GOPS)     zip (GOPS)
-------------------  --------------  -------------  -------------
Neoverse-V2 (Grace)           373.2          277.3          406.6
Neoverse-V2 (Axion)           362.4          263.9          390.1
Neoverse-N3                   252.1          233.5          274.1
AmpereOne                     133.4           98.5          134.3
Apple M6                      392.5          371.8          454.3
Apple M5                      377.2          342.3          435.7
Oryon-L                       544.5          527.4          560.8
Oryon-M                       268.0          269.0          288.6
Oryon V3-L                    464.2          482.9          501.1
Oryon V3-M                    250.4          253.8          262.5
Cortex-X4                     302.1          300.0          305.3
Cortex-X3                     199.6          197.1          197.5
Cortex-X2                     269.1          256.7          278.9
Cortex-C1-Ultra               249.0          248.5          249.2
Cortex-C1-Pro                 179.9          175.7          175.2
Cortex-A725                   208.9          205.1          205.5
Cortex-A720                   226.8          236.3          236.2
Cortex-A715                   161.1          162.5          162.9
Cortex-A710                   159.5          158.7          166.3
Cortex-A520                    58.2           84.5           97.8
Cortex-A510r1                  55.2           67.7           72.5
Cortex-A510                    54.9           58.1           56.0
bazel build -c opt //bench:qs8_qc8w_gemm_fp32_bench
bazel-bin/bench/qs8_qc8w_gemm_fp32_bench \
  --benchmark_filter="qs8_qc8w_gemm_minmax_fp32_ukernel_4x16c8__neoni8mm" \
  --benchmark_min_time=0.5s

…EMM/IGEMM microkernels into a separate `_zip` variant (`VARIANT=ZIP`) and update the existing `c8-neoni8mm` microkernels to use `vld2q_dup_u64` for lane 0 instead of zero-initializing a loop-carried `uint64x2x2_t` register pair for `vld2q_lane_u64(..., 0)`.

Benchmark on `qs8_qc8w_gemm_minmax_fp32_ukernel_4x16c8__neoni8mm` (MobileNetV2 `M=49, N=1280, K=320`, GOPS):

```text
uarch                   lane (GOPS)     dup (GOPS)     zip (GOPS)
-------------------  --------------  -------------  -------------
Neoverse-V2 (Grace)           373.2          277.3          406.6
Neoverse-V2 (Axion)           362.4          263.9          390.1
Neoverse-N3                   252.1          233.5          274.1
AmpereOne                     133.4           98.5          134.3
Apple M6                      392.5          371.8          454.3
Apple M5                      377.2          342.3          435.7
Oryon-L                       544.5          527.4          560.8
Oryon-M                       268.0          269.0          288.6
Oryon V3-L                    464.2          482.9          501.1
Oryon V3-M                    250.4          253.8          262.5
Cortex-X4                     302.1          300.0          305.3
Cortex-X3                     199.6          197.1          197.5
Cortex-X2                     269.1          256.7          278.9
Cortex-C1-Ultra               249.0          248.5          249.2
Cortex-C1-Pro                 179.9          175.7          175.2
Cortex-A725                   208.9          205.1          205.5
Cortex-A720                   226.8          236.3          236.2
Cortex-A715                   161.1          162.5          162.9
Cortex-A710                   159.5          158.7          166.3
Cortex-A520                    58.2           84.5           97.8
Cortex-A510r1                  55.2           67.7           72.5
Cortex-A510                    54.9           58.1           56.0
```

```bash
bazel build -c opt //bench:qs8_qc8w_gemm_fp32_bench
bazel-bin/bench/qs8_qc8w_gemm_fp32_bench \
  --benchmark_filter="qs8_qc8w_gemm_minmax_fp32_ukernel_4x16c8__neoni8mm" \
  --benchmark_min_time=0.5s
```

PiperOrigin-RevId: 1005800878
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant