Skip to content

[Bug]: [Dataflow Streaming Appliance] Key commits > 180MB are failing with KeyCommitTooLargeException with beam 2.60.0+ #33588

@arunpandianp

Description

@arunpandianp

What happened?

#32408 introduced a bug that is causing commits larger than 180MB are failing client side.
Prior to the change there was no client side limits on the commit size for dataflow appliance jobs.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions