Skip to content

[Spark] Add unsafeVolatileTableId to SnapshotDescriptor and migrate callsites#7016

Open
huan233usc wants to merge 1 commit into
delta-io:masterfrom
huan233usc:add-unsafevolatiletableid-to-snapshotdescriptor
Open

[Spark] Add unsafeVolatileTableId to SnapshotDescriptor and migrate callsites#7016
huan233usc wants to merge 1 commit into
delta-io:masterfrom
huan233usc:add-unsafevolatiletableid-to-snapshotdescriptor

Conversation

@huan233usc

Copy link
Copy Markdown
Collaborator

Which Delta project/connector is this regarding?

  • Spark
  • Standalone
  • Flink
  • Kernel
  • Other (fill in here)

Description

Adds unsafeVolatileTableId to the SnapshotDescriptor trait, defaulting to metadata.id. Snapshot overrides it to return the DeltaLog's volatile cached table id: the logging helpers on Snapshot are invoked during state reconstruction itself, and reading metadata.id there would re-enter the reconstruction lazy val on the same thread and overflow the stack, so the always-available cached id is used instead.

Call sites that previously reached through snapshot.deltaLog.unsafeVolatileTableId (row tracking materialized column name resolution, streaming schema tracking log creation, and the Snapshot logging helpers) now use snapshot.unsafeVolatileTableId directly, reducing direct DeltaLog dependencies on the snapshot read path.

How was this patch tested?

Existing unit tests. This is a refactor with no behavior change.

Does this PR introduce any user-facing changes?

No

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant