Skip to content

Releases: GoogleCloudPlatform/DataflowTemplates

Dataflow Templates 2025-10-07-00_RC00

07 Oct 13:44
3d0a9d3
Compare
Choose a tag to compare

Note: This release is being rolled out. It may not be in your region yet.

What's Changed

Improvements

  • Fixed a performance regression in the Spanner change streams to BigQuery template that was introduced in the 2024-09-10-00_RC00.
  • Only read schema when the col in mod is not in the stored schema. by @jiangzzhu in #2840
  • Skip reading schema if the value capture mode is NEW_ROW_AND_OLD_VALUES. by @jiangzzhu in #2839
  • feat: Add support for property graph on view by @jennyjiang98 in #2813
  • Remove jinja variables option code by @derrickaw in #2826

New Contributors

Full Changelog: 2025-10-01-00_RC00...2025-10-07-00_RC00

Dataflow Templates 2025-10-01-00_RC00

02 Oct 00:50
980e9fb
Compare
Choose a tag to compare

What's Changed

Improvements

  • Add Run, Staging and Validation stages for KafkaToBigquery Yaml template by @derrickaw in #2782

Bug fixes

Package Upgrades

Known issues

Users of Spanner change streams to BigQuery template are advised to use the 2024-09-02-00_RC00 version of the template due to a performance regression introduced in the 2024-09-10-00_RC00 release that might affect certain use-cases.

Full Changelog: 2025-09-23-00_RC00...2025-10-01-00_RC00

Dataflow Templates 2025-09-23-00_RC00

23 Sep 21:49
cdb6373
Compare
Choose a tag to compare

What's Changed

Improvements

  • Introduce Schemaless graphs to import/export DDL by @smukil in #2749
  • build(deps): bump org.awaitility:awaitility from 4.2.0 to 4.3.0 by @dependabot[bot] in #2794
  • DataStreamToSpanner - Adding main table read at the beginning of main transaction by @darshan-sj in #2789
  • CPU priority in reverse replication jobs by @bharadwaj-aditya in #2795

Known issues

Users of Spanner change streams to BigQuery template are advised to use the 2024-09-02-00_RC00 version of the template due to a performance regression introduced in the 2024-09-10-00_RC00 release that might affect certain use-cases.

Full Changelog: 2025-09-16-00_RC00...2025-09-23-00_RC00

Dataflow Templates 2025-09-16-00_RC00

16 Sep 13:18
ffa0a9b
Compare
Choose a tag to compare

What's Changed

Improvements

Package Upgrades

  • chore(deps): update Python dependencies by @github-actions[bot] in #2778

Known issues

Users of Spanner change streams to BigQuery template are advised to use the 2024-09-02-00_RC00 version of the template due to a performance regression introduced in the 2024-09-10-00_RC00 release that might affect certain use-cases.

Full Changelog: 2025-09-10-00_RC00...2025-09-16-00_RC00

Dataflow Templates 2025-09-10-00_RC00

10 Sep 18:37
44b3463
Compare
Choose a tag to compare

What's Changed

Improvements

  • Bump com.azure:azure-messaging-eventhubs from 5.15.0 to 5.21.0 by @dependabot[bot] in #2711
  • Bump org.codehaus.mojo:exec-maven-plugin from 1.6.0 to 3.5.1 by @dependabot[bot] in #2710
  • Datastream To SQL - Added Default Casing Parameter / Column Casing Functionality by @rahulkumar0630 in #2687

Bug fixes

Known issues

Users of Spanner change streams to BigQuery template are advised to use the 2024-09-02-00_RC00 version of the template due to a performance regression introduced in the 2024-09-10-00_RC00 release that might affect certain use-cases.

Full Changelog: 2025-09-02-00_RC00...2025-09-10-00_RC00

Dataflow Templates 2025-09-09-00_RC00

09 Sep 22:22
5544c02
Compare
Choose a tag to compare
fix: update MongoDB container image tag in integration test setup (#2…

Dataflow Templates 2025-09-02-00_RC00

02 Sep 19:35
c455265
Compare
Choose a tag to compare

What's Changed

Improvements

  • feat(import): add early index create threshold parameter by @liferoad in #2686
  • [Yaml]: add dependency requirements etc by @derrickaw in #2694
  • Bump org.twdata.maven:mojo-executor from 2.4.0 to 2.4.1 by @dependabot[bot] in #2700
  • Bump org.codehaus.mojo:extra-enforcer-rules from 1.8.0 to 1.10.0 by @dependabot[bot] in #2699
  • fix readme outputs for python and yaml modules by @derrickaw in #2704

Bug fixes

  • fix(mongodb): handle special floating point values in Gson serialization by @liferoad in #2707

Full Changelog: 2025-08-26-00_RC00...2025-09-02-00_RC00

Dataflow Templates 2025-08-26-00_RC00

26 Aug 13:00
fdf18a8
Compare
Choose a tag to compare

What's Changed

Improvements

Bug fixes

New Contributors

Full Changelog: 2025-08-19-00_RC00...2025-08-26-00_RC00

Dataflow Templates 2025-08-19-00_RC00

19 Aug 13:01
88b5b28
Compare
Choose a tag to compare

What's Changed

Improvements

Full Changelog: 2025-08-12-00_RC00...2025-08-19-00_RC00

Dataflow Templates 2025-08-12-00_RC00

12 Aug 12:58
5ecb94b
Compare
Choose a tag to compare

What's Changed

Improvements

  • Replace session file usage with ISchemaMapper and add string overrides support by @Deep1998 in #2561
  • Bump parquet-avro version to 1.15.2 by @jrmccluskey in #2606

Full Changelog: 2025-08-05-00_RC01...2025-08-12-00_RC00