Skip to content

Enable full NumPy indexing compliance for jax.Ref - #5542

Open
copybara-service[bot] wants to merge 1 commit into
mainfrom
test_957374261
Open

copybara-service[bot] wants to merge 1 commit into
mainfrom
test_957374261

Conversation

@copybara-service

Copy link
Copy Markdown

Enable full NumPy indexing compliance for jax.Ref

Decouples None (jnp.newaxis) from the default empty index argument in reference primitives (ref_get, ref_set, ref_swap, ref_addupdate) by transitioning defaults to (). Delegates reference index parsing to jax._src.numpy.indexing, allowing references to inherit canonical ellipsis (...) expansion and boolean array indexing.

To maintain compatibility with low-level kernel lowerings (such as Pallas and Triton), physical buffer indexers (NDIndexer) are decoupled from size-1 axis insertions. This is structured via a new bridge function, get_transforms_from_indices, which translates user indexing expressions into explicit state transformations during primitive evaluation and view construction (ref.at[idx]):

  • Routes raw indexing arguments through numpy.indexing.NDIndexer.from_raw_indices to inherit standard indexing validation, ellipsis expansion, and boolean array conversion.
  • Separates zero-consumption newaxis dimensions (IndexType.NONE) from physical memory buffer slicing.
  • Returns a structured transformation sequence consisting of a physical NDIndexer and an optional follow-up ReshapeTransform that inserts size-1 axes in accordance with NumPy integer and slice contiguity conventions.

Decouples `None` (`jnp.newaxis`) from the default empty index argument in reference primitives (`ref_get`, `ref_set`, `ref_swap`, `ref_addupdate`) by transitioning defaults to `()`. Delegates reference index parsing to `jax._src.numpy.indexing`, allowing references to inherit canonical ellipsis (`...`) expansion and boolean array indexing.

To maintain compatibility with low-level kernel lowerings (such as Pallas and Triton), physical buffer indexers (`NDIndexer`) are decoupled from size-1 axis insertions. This is structured via a new bridge function, `get_transforms_from_indices`, which translates user indexing expressions into explicit state transformations during primitive evaluation and view construction (`ref.at[idx]`):
- Routes raw indexing arguments through `numpy.indexing.NDIndexer.from_raw_indices` to inherit standard indexing validation, ellipsis expansion, and boolean array conversion.
- Separates zero-consumption newaxis dimensions (`IndexType.NONE`) from physical memory buffer slicing.
- Returns a structured transformation sequence consisting of a physical `NDIndexer` and an optional follow-up `ReshapeTransform` that inserts size-1 axes in accordance with NumPy integer and slice contiguity conventions.

PiperOrigin-RevId: 957374261

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants