Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/modules/3d-tiles/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ The [3D Tiles runtime concepts suite](/docs/modules/3d-tiles/concepts) explains
- [Request scheduling, progressive loading, and foveated requests](/docs/modules/3d-tiles/concepts/request-scheduling-and-priorities)
- [Caching and memory](/docs/modules/3d-tiles/concepts/caching-and-memory)
- [Runtime tuning and diagnostics](/docs/modules/3d-tiles/concepts/runtime-tuning-and-diagnostics)
- [Runtime observability and benchmark baselines](/docs/modules/3d-tiles/concepts/observability-and-benchmarks)

<ReferenceBoundary
title="Module APIs and runtime concepts"
Expand Down
1 change: 1 addition & 0 deletions docs/modules/3d-tiles/concepts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ Loading a large 3D Tiles tileset is a continuous pipeline: traverse the hierarch
| [Request scheduling and priorities](/docs/modules/3d-tiles/concepts/request-scheduling-and-priorities) | Which required tile should use the next network slot? |
| [Caching and memory](/docs/modules/3d-tiles/concepts/caching-and-memory) | Which loaded tiles remain resident, and when may the budget be exceeded? |
| [Runtime tuning and diagnostics](/docs/modules/3d-tiles/concepts/runtime-tuning-and-diagnostics) | Which controls and measurements explain visible behavior? |
| [Runtime observability and benchmark baselines](/docs/modules/3d-tiles/concepts/observability-and-benchmarks) | How can traversal decisions and performance budgets be compared? |

The stages are related but not interchangeable. In particular, screen-space error determines the desired final LOD. Progressive and foveated scheduling normally change only the order and timing of requests needed to reach that LOD.

Expand Down
137 changes: 137 additions & 0 deletions docs/modules/3d-tiles/concepts/observability-and-benchmarks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
---
title: Runtime observability and benchmark baselines
description: Make 3D Tiles traversal behavior measurable, reproducible, and performance-budgeted.
hide_title: true
page_style: designed
---

import {Tiles3DDocsTabs} from '@site/src/components/docs/tiles-3d-docs-tabs';
import {DocPageHeader} from '@site/src/components/docs/doc-page-header';
import {DocOrientation, ReferenceBoundary} from '@site/src/components/docs/designed-doc';

<DocPageHeader
eyebrow="3D Tiles runtime"
title="Measure traversal before tuning it."
description="Deterministic snapshots and source diagnostics make screen-space error, request pressure, and cache behavior comparable across cameras, devices, and releases."
tone="violet"
meta={['Deterministic snapshots', 'Request and cache counters', 'Benchmark budgets']}
/>

<Tiles3DDocsTabs active="runtime" />

<DocOrientation
eyebrow="A small inspection contract"
title="Capture what the traverser decided."
description="The observability API reports selection and work sets without coupling loaders.gl to deck.gl, a renderer, or a particular benchmark harness."
tone="violet"
items={[
{label: 'Selection', value: 'Sorted selected, requested, and empty tile IDs'},
{label: 'Work', value: 'Loading, loaded, failed, and cached counts'},
{label: 'Memory', value: 'Estimated resident bytes and active maximum SSE'},
{label: 'Sources', value: 'Implicit subtree request and cache counters when available'}
]}
/>

<ReferenceBoundary
title="Runtime observability"
description="Use snapshots for regression fixtures and counters for live dashboards. The values describe the completed traversal frame and never change traversal policy."
tone="violet"
/>

## Snapshot API

getTileset3DTraversalSnapshot(tileset) returns a serializable
Tileset3DTraversalSnapshot with sorted IDs and numeric counters:

~~~typescript
import {
getTileset3DTraversalSnapshot,
Tileset3D
} from '@loaders.gl/tiles';

await tileset.selectTiles(viewport);
const snapshot = getTileset3DTraversalSnapshot(tileset);

console.log(snapshot.selectedTileIds);
console.log(snapshot.requestedTileIds);
console.log(snapshot.maximumScreenSpaceError);
~~~

The snapshot is intentionally defensive. It copies tile IDs, does not retain tile or content
objects, and can be JSON-serialized as a golden fixture. Capture it only after selectTiles resolves;
asynchronous loads can change the next frame's selected set.

| Field | Meaning | Unit |
| --- | --- | --- |
| frameNumber | Completed traversal frame | count |
| selectedTileIds | Tiles selected for rendering, sorted | IDs |
| requestedTileIds | Tiles queued for content loading, sorted | IDs |
| emptyTileIds | Hierarchy-only or empty tiles visited, sorted | IDs |
| visibleTileCount | Length of the selected set | count |
| renderableTileCount | Selected tiles with ready render content | count |
| loadingTileCount | Tile and subtree loads in flight | count |
| loadedTileCount | Cumulative successful tile loads | count |
| failedTileCount | Cumulative failed tile loads | count |
| cachedTileCount | Tiles currently retained in the cache | count |
| cacheBytes | Estimated resident content memory | bytes |
| maximumScreenSpaceError | Active memory-adjusted SSE threshold | logical/CSS pixels |

The optional implicitTiling object is supplied by Tiles3DSource. Its request, materialization, pending,
and parsed-cache counters are useful for diagnosing subtree fan-out and cache reuse.

## Deterministic conformance fixtures

A conformance fixture should fix the tileset JSON, viewport, options, and initial cache state.
Compare sorted IDs and counts rather than object identity or request completion order. A practical
fixture captures:

1. one root traversal with content unavailable;
2. the same viewport after content resolves;
3. a second traversal proving cache reuse; and
4. a camera move that changes only the expected branch.

Keep network access hermetic by injecting a resolver or in-memory source. Put large hierarchies and
long camera paths in the slow suite. Do not use snapshots to bless a regression: explain why a
selected tile, request, or count changed.

## Benchmark dimensions and budgets

Record at least:

- initialization time and first useful frame;
- traversal time per viewport;
- number of selected, requested, loaded, failed, and cached tiles;
- resident bytes and peak resident bytes;
- implicit subtree requests, cache hits, and materialized headers;
- number of frames required to reach the target SSE.

Use a fixed browser, viewport size, camera path, network fixture, and warm/cold-cache label. Suggested
starting budgets are deliberately relative: a correctness change should not increase cold-start
traversal time by more than 10%, request count by more than 5%, or resident bytes by more than 10%
for the same snapshot. Calibrate absolute limits to the dataset and CI hardware before enforcing
them.

## Reading the counters

A high requestedTileIds count with a small selected set usually indicates aggressive refinement,
a low maximumScreenSpaceError, or a projection/culling mismatch. A growing loadingTileCount means
the source or decoder is the bottleneck; a growing failedTileCount points to transport, content, or
extension errors. High cachedTileCount and cacheBytes with repeated evictions suggest a budget that
is too small for the working set.

For implicit tiling, many requestedSubtrees with few cacheHits can indicate unstable URL resolution
or an undersized parsed-subtree cache. materializedTiles measures hierarchy work, not renderable GPU
content.

## What snapshots do not promise

Snapshots are diagnostic, not a renderer contract. Tile IDs and counters are stable for a fixed
source, options, and traversal implementation; they are not guaranteed to match across different
tileset revisions. The API does not expose styling, GPU buffer formats, draw order, or network
timings. I3S sources provide the common tile counters, but the implicit-tiling section is specific
to 3D Tiles.

See [screen-space error and LOD](/docs/modules/3d-tiles/concepts/screen-space-error-and-lod),
[caching and memory](/docs/modules/3d-tiles/concepts/caching-and-memory), and
[request scheduling](/docs/modules/3d-tiles/concepts/request-scheduling-and-priorities) for the
controls that explain these measurements.
2 changes: 2 additions & 0 deletions docs/modules/tiles/api-reference/tileset-3d.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ import {TiledSceneGraphic} from '@site/src/components/docs/tiled-scene-graphic';

The `Tileset3D` class is the shared runtime for traversal, culling, selection, cache management, and request scheduling across source-backed 3D tilesets.

For deterministic traversal snapshots and request/cache diagnostics, see [Runtime observability and benchmark baselines](/docs/modules/3d-tiles/concepts/observability-and-benchmarks).

It is constructed with a [`Tileset3DSource`](/docs/modules/tiles/api-reference/tileset-3d-source), such as [`Tiles3DSource`](/docs/modules/tiles/api-reference/tiles-3d-source) or [`I3SSource`](/docs/modules/tiles/api-reference/i3s-source).

## Standards
Expand Down
1 change: 1 addition & 0 deletions docs/whats-new.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ import {DocOrientation, ReferenceBoundary} from '@site/src/components/docs/desig
- Classify multi-content render volumes as a union, apply clipping planes only to render content, and document the 3D Tiles correctness/conformance contract for SSE, implicit boundaries, extensions, and lifecycle diagnostics.
- Harden traversal conformance for viewer-request-volume gating, external-tileset expiration, additive/replacement refinement, and scheduler-safe camera movement.
- Add renderer-neutral 3D Tiles contracts for ordered multi-content entries, feature-ID declarations, raw metadata context, and indexed or union content visibility.
- Add deterministic 3D Tiles traversal snapshots with sorted selection/request sets, tile/cache counters, resident-byte estimates, and implicit-subtree diagnostics, plus benchmark-budget guidance.


Release Date: 2026
Expand Down
2 changes: 2 additions & 0 deletions modules/tiles/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
export type {Tileset3DProps} from './tileset-3d/common/tileset-3d';
export type {FoveatedInterpolationCallback} from './tileset-3d/helpers/tiles-3d-request-priority';
export {Tileset3D} from './tileset-3d/common/tileset-3d';
export {getTileset3DTraversalSnapshot} from './tileset-3d/common/tileset-observability';
export type {Tileset3DTraversalSnapshot} from './tileset-3d/common/tileset-observability';
export type {
TileContentLoadResult,
TileChildrenLoadResult,
Expand Down
28 changes: 28 additions & 0 deletions modules/tiles/src/tileset-3d/common/tileset-3d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -517,6 +517,26 @@ export class Tileset3D {
return this._frameNumber;
}

/**
* Returns the tiles requested during the most recent completed traversal.
*
* A defensive copy makes request ordering deterministic for diagnostics and prevents callers
* from mutating the scheduler's working set.
*/
get requestedTiles(): readonly Tile3D[] {
return this._requestedTiles.slice();
Comment thread
ibgreen marked this conversation as resolved.
}

/**
* Returns empty tiles considered during the most recent completed traversal.
*
* Empty tiles can still be useful hierarchy placeholders, but their IDs are included in
* observability snapshots so applications can distinguish traversal work from render content.
*/
get emptyTiles(): readonly Tile3D[] {
return this._emptyTiles.slice();
}

/**
* Gets or sets the soft target, in bytes, for cached tile content not needed this frame.
*
Expand Down Expand Up @@ -662,6 +682,14 @@ export class Tileset3D {
return;
}
const preparedViewports = viewports instanceof Array ? viewports : [viewports];
const activeViewportIds = new Set(preparedViewports.map(viewport => viewport.id));
// A viewport may be removed between frames. Discard its completed state before aggregating
// requested and selected tiles so observability reflects only the current traversal inputs.
for (const frameStateId of Object.keys(this.frameStateData)) {
if (!activeViewportIds.has(frameStateId)) {
delete this.frameStateData[frameStateId];
}
}

this._cache.reset();
this._frameNumber++;
Expand Down
100 changes: 100 additions & 0 deletions modules/tiles/src/tileset-3d/common/tileset-observability.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
// SPDX-License-Identifier: MIT
// Copyright (c) vis.gl contributors

import type {Tileset3D} from './tileset-3d';
import type {Tile3D} from './tile-3d';

/**
* A deterministic, renderer-neutral snapshot of one {@link Tileset3D} traversal.
*
* Tile IDs are sorted so snapshots can be compared across runs even when child requests complete
* in a different order. Counts are reported for the completed traversal frame and cache values are
* expressed in bytes. This is an inspection contract; it does not alter traversal or request policy.
*/
export type Tileset3DTraversalSnapshot = {
/** Monotonically increasing traversal frame number. */
frameNumber: number;
/** IDs selected for rendering, sorted for stable comparisons. */
selectedTileIds: string[];
/** IDs whose content was requested, sorted for stable comparisons. */
requestedTileIds: string[];
/** IDs visited as hierarchy-only or empty tiles, sorted for stable comparisons. */
emptyTileIds: string[];
/** Number of selected tiles in the completed frame. */
visibleTileCount: number;
/** Number of selected tiles with renderable content. */
renderableTileCount: number;
/** Number of tile or subtree loads currently in flight. */
loadingTileCount: number;
/** Cumulative number of tiles loaded into the runtime cache. */
loadedTileCount: number;
/** Cumulative number of failed tile loads. */
failedTileCount: number;
/** Number of tiles currently retained in the runtime cache. */
cachedTileCount: number;
/** Estimated cached content bytes. */
cacheBytes: number;
/** Active memory-adjusted maximum SSE in logical/CSS pixels. */
maximumScreenSpaceError: number;
/** Source-specific implicit subtree counters, when the source exposes them. */
implicitTiling?: {
/** Number of subtree resources requested from the source. */
requestedSubtrees: number;
/** Number of subtree resources successfully materialized. */
loadedSubtrees: number;
/** Number of requests served by the parsed-subtree cache. */
cacheHits: number;
/** Number of parsed subtrees currently retained for reuse. */
cachedSubtrees: number;
/** Number of subtree requests currently in flight. */
pendingSubtrees: number;
Comment thread
ibgreen marked this conversation as resolved.
/** Number of runtime tile headers created from materialized subtrees. */
materializedTiles: number;
};
};

/**
* Creates a stable observability snapshot from public {@link Tileset3D} state.
*
* The helper intentionally reads only public runtime fields and source diagnostics. It is safe to
* call from an instrumentation loop and does not retain tile or content references.
*
* @param tileset - Runtime whose most recent traversal should be inspected.
* @returns A serializable snapshot suitable for logs, regression fixtures, and benchmark output.
*/
export function getTileset3DTraversalSnapshot(tileset: Tileset3D): Tileset3DTraversalSnapshot {
const selectedTiles = tileset.selectedTiles.slice();
const requestedTiles = tileset.requestedTiles.slice();
const emptyTiles = tileset.emptyTiles.slice();

const getTileIds = (tiles: readonly Tile3D[]): string[] =>
tiles.map(tile => String(tile.id)).sort();

const getStatCount = (name: string): number => {
const count = tileset.stats.get(name).count;
return typeof count === 'number' && Number.isFinite(count) ? count : 0;
};

const implicitTilingSource = tileset.source as Tileset3D['source'] & {
getImplicitTilingStats?: () => Tileset3DTraversalSnapshot['implicitTiling'];
};
const implicitTiling = implicitTilingSource.getImplicitTilingStats?.();

return {
frameNumber: tileset.frameNumber,
selectedTileIds: getTileIds(selectedTiles),
requestedTileIds: getTileIds(requestedTiles),
emptyTileIds: getTileIds(emptyTiles),
visibleTileCount: selectedTiles.length,
renderableTileCount: selectedTiles.filter(
tile => tile.contentAvailable && Boolean(tile.content)
).length,
loadingTileCount: getStatCount('Tiles Loading'),
loadedTileCount: getStatCount('Tiles Loaded'),
failedTileCount: getStatCount('Failed Tile Loads'),
cachedTileCount: getStatCount('Tiles In Memory'),
cacheBytes: tileset.gpuMemoryUsageInBytes,
maximumScreenSpaceError: tileset.memoryAdjustedScreenSpaceError,
...(implicitTiling ? {implicitTiling} : {})
};
}
73 changes: 73 additions & 0 deletions modules/tiles/test/tileset/tileset-observability.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
import {describe, expect, test} from 'vitest';

import {getTileset3DTraversalSnapshot} from '../../src';

function createTile(id: string, renderable = false): any {
return {
id,
contentAvailable: renderable,
content: renderable ? {} : null
};
}

function createTileset(overrides: Record<string, unknown> = {}): any {
const values: Record<string, number> = {
'Tiles Loading': 1,
'Tiles Loaded': 4,
'Failed Tile Loads': 2,
'Tiles In Memory': 3
};
return {
frameNumber: 7,
selectedTiles: [createTile('tile-b', true), createTile('tile-a')],
requestedTiles: [createTile('tile-c'), createTile('tile-a')],
emptyTiles: [createTile('empty-b'), createTile('empty-a')],
stats: {get: (name: string) => ({count: values[name] ?? 0})},
gpuMemoryUsageInBytes: 4096,
memoryAdjustedScreenSpaceError: 8,
source: {},
...overrides
};
}

describe('getTileset3DTraversalSnapshot', () => {
test('sorts IDs and captures runtime counters', () => {
const snapshot = getTileset3DTraversalSnapshot(createTileset());

expect(snapshot).toEqual({
frameNumber: 7,
selectedTileIds: ['tile-a', 'tile-b'],
requestedTileIds: ['tile-a', 'tile-c'],
emptyTileIds: ['empty-a', 'empty-b'],
visibleTileCount: 2,
renderableTileCount: 1,
loadingTileCount: 1,
loadedTileCount: 4,
failedTileCount: 2,
cachedTileCount: 3,
cacheBytes: 4096,
maximumScreenSpaceError: 8
});
});

test('copies IDs and includes implicit subtree diagnostics when available', () => {
const tileset = createTileset({
source: {
getImplicitTilingStats: () => ({
requestedSubtrees: 2,
loadedSubtrees: 1,
cacheHits: 3,
cachedSubtrees: 1,
pendingSubtrees: 0,
materializedTiles: 8
})
}
});

const snapshot = getTileset3DTraversalSnapshot(tileset);
tileset.selectedTiles[0].id = 'mutated';

expect(snapshot.selectedTileIds).toEqual(['tile-a', 'tile-b']);
expect(snapshot.implicitTiling?.cacheHits).toBe(3);
});
});