Skip to content

feat(core) TableBufferPlanner powered AttributeManager #10151

Open
ibgreen wants to merge 17 commits into
masterfrom
ib/buffer-groups
Open

feat(core) TableBufferPlanner powered AttributeManager #10151
ibgreen wants to merge 17 commits into
masterfrom
ib/buffer-groups

Conversation

@ibgreen
Copy link
Copy Markdown
Collaborator

@ibgreen ibgreen commented Mar 30, 2026

Summary

This branch introduces TableBufferPlanner, a table-first buffer allocation planner for deck.gl attributes, and folds the planner-backed buffer orchestration into AttributeManager.

Compared to master, the main change is that deck.gl can now reason about attributes as binary table columns first, then plan how those columns should be represented as GPU vertex/storage buffers under WebGPU/WebGL constraints.

image

Goals

  • Make WebGPU attribute binding more robust by planning vertex-buffer usage explicitly.
  • Keep CPU-side binary table columns columnar on the GPU where possible.
  • Support constant attributes on WebGPU by materializing them as small real buffers.
  • Reduce pressure on WebGPU’s limited vertex-buffer slots.
  • Preserve deck.gl table semantics across both:
    • shared instanced geometry
    • row-generated variable-length geometry
  • Keep the planner abstract: it works on column descriptors, not Attribute, DataColumn, or AttributeManager.

Highlight: TableBufferPlanner

Adds modules/core/src/lib/attribute/table-buffer-planner.ts.

TableBufferPlanner takes abstract TableColumnDescriptor inputs and returns a TableBufferPlan describing:

  • allocation groups
  • per-column model binding mappings
  • packed vertex-buffer columns
  • storage-buffer candidate columns
  • unmanaged columns that must keep their existing allocation path

Planner modes:

  • table-with-shared-geometry: one shared geometry rendered once for each table row
  • table-with-row-geometries: each table row expands into a variable number of geometry vertices

Planner group kinds include:

  • interleaved-shared-geometry-columns
  • position-attribute-columns
  • interleaved-constant-attribute-columns
  • separate-attribute-column
  • interleaved-attribute-columns
  • separate-storage-column
  • stacked-storage-columns
  • unmanaged-attribute-column

The planner also handles fp64 position high/low mappings, generated row lookup columns, optional storage-buffer planning, and device limit validation.

Core Changes

  • Integrates planner-backed allocation into AttributeManager.
  • Adds model binding APIs on AttributeManager so Layer can apply:
    • vertex buffers
    • constant attributes
    • index buffers
    • buffer layouts
  • Removes the separate GroupedAttributeManager path and simplifies Layer back to using AttributeManager.
  • Keeps packed-buffer runtime state inside AttributeManager; TableBufferPlan remains allocation metadata only.
  • Preserves unmanaged paths for indexed attributes, transitions, external GPU-only buffers, unsupported fp64 cases, and most noAlloc attributes.
  • Allows generated CPU-backed picking colors to be planner-managed to reduce vertex-buffer slot pressure.

Layer/Layout Changes

  • Updates layer model creation to pass planner hints such as isInstanced and reserved geometry buffer counts.
  • Handles delayed geometry attachment paths such as ColumnLayer.
  • Marks non-instanced polygon fill models appropriately.
  • Keeps external GPU aggregation buffers unmanaged so GPU aggregation paths can continue publishing their buffers directly.

Documentation

Adds docs/developer-guide/custom-layers/table-buffer-planning.md, covering:

  • table-first planner model
  • TableBufferPlanner usage examples
  • TableBufferPlan
  • row lookup columns
  • position/fp64 mapping behavior
  • buffer groups
  • optional storage-buffer planning
  • WebGPU constraints

Also updates the custom layer docs table of contents.

Tests

Adds/updates focused coverage for:

  • planner allocation groups and mode behavior
  • constant buffer step modes
  • generated rowIndex and pickingColors
  • fp64 high/low position mappings
  • external GPU-buffer-only attributes
  • constrained vertex-buffer counts
  • packed overflow buffers
  • layer model binding behavior
  • polygon/hexagon-related grouped/planner regressions

Verified locally:

  • npx tsc -p modules/core/tsconfig.json --noEmit
  • npx tsc -p modules/layers/tsconfig.json --noEmit
  • npx tsc -p modules/aggregation-layers/tsconfig.json --noEmit
  • yarn test modules/core/lib/attribute/attribute-manager modules/core/lib/attribute/attribute-manager-table-buffer modules/core/lib/layer
  • yarn test modules/aggregation-layers/hexagon-layer
  • yarn test modules/aggregation-layers/heatmap-layer
  • git diff --check

Notes / Review Focus

  • TableBufferPlanner is intentionally independent of Attribute, DataColumn, and AttributeManager.
  • Storage-buffer planning is currently planner output only; model binding does not yet consume storage groups.
  • Packed buffer runtime caches remain in AttributeManager because they store actual GPU buffers and CPU upload arrays, not planner metadata.
  • This branch is primarily about the attribute/table-buffer planning architecture and the minimal layer hints needed to exercise it.

@ibgreen ibgreen requested a review from Pessimistress March 30, 2026 18:45
@ibgreen ibgreen marked this pull request as ready for review April 1, 2026 15:08
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0834dbefaa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread modules/core/src/lib/attribute/attribute-manager.ts Outdated
Comment thread modules/core/src/lib/attribute/attribute-manager.ts Outdated
@chrisgervang chrisgervang added this to the v9.3 milestone Apr 1, 2026
@coveralls
Copy link
Copy Markdown

coveralls commented Apr 3, 2026

Coverage Status

coverage: 80.533% (+0.08%) from 80.457%
when pulling 4cde6f5 on ib/buffer-groups
into b1c8331 on master.

@ibgreen-openai ibgreen-openai changed the title feat(core) AttributeManager buffer groups feat(core) TableBufferPlanner powered AttributeManager May 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants