Skip to content

[operators] Harden transpose-nd against overflow, divide-by-zero, and null pointers - #11334

Open
anagnorisis2peripeteia wants to merge 1 commit into
google:masterfrom
anagnorisis2peripeteia:transpose-nd-overflow-hardening
Open

anagnorisis2peripeteia wants to merge 1 commit into
google:masterfrom
anagnorisis2peripeteia:transpose-nd-overflow-hardening

Conversation

@anagnorisis2peripeteia

Copy link
Copy Markdown
Contributor

Summary

This PR hardens the transpose operator implementation in src/operators/transpose-nd.c against arithmetic overflow, division-by-zero, out-of-bounds indexing, and null pointer dereferences:

  1. Creation Validation (create_transpose_nd):
    • Validates that transpose_op_out != NULL.
  2. Reshape Validation (reshape_transpose_nd):
    • Validates that transpose_op != NULL, input_shape != NULL, and perm != NULL.
    • Protects total tensor elements (num_elements) and total byte size (total_bytes) calculations against size_t overflow with xnn_safe_mul.
    • Protects input_stride and output_stride bounds check calculations (stride[i] * shape[i] and current_stride) with xnn_safe_mul to prevent integer wrapping that could bypass the stride validation checks.
    • Validates that xnn_normalize_transpose_permutation succeeds and returns normalized_dims > 0, avoiding undefined behavior and underflow of normalized_dims - 1 in subsequent indexing.
  3. Tile Computation Safety:
    • Validates normalized_element_size != 0 before computing tile count, preventing division by zero.
    • Enforces minimum tile sizes (max(1, ...)) to prevent division by zero in subsequent tile aspect calculations.
  4. Setup Validation (setup_transpose_nd):
    • Validates that input != NULL and output != NULL.

Testing

  • Added unit regression tests in test/operators/transpose-nd.cc covering null pointer outputs, invalid reshape parameters (zero dims, oversized dims, null arrays, overflow shapes, duplicate/out-of-bounds permutations), and null setup pointers.
  • All 35 tests in transpose-nd-test and all 18 transpose-related tests in ctest pass cleanly.

@anagnorisis2peripeteia
anagnorisis2peripeteia force-pushed the transpose-nd-overflow-hardening branch from df514e3 to a486889 Compare September 26, 2026 21:40
@anagnorisis2peripeteia
anagnorisis2peripeteia force-pushed the transpose-nd-overflow-hardening branch from a486889 to a854c4d Compare October 12, 2026 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant