Skip to content

Feature Request: avro: caller-specified record name and namespace in generated Avro schemas #3471

Description

@jwhartley

1. Priority

Medium - two tier 1 tenants blocked by Avro naming in a Schema Registry.

2. What this enables

As a consumer developer, I want the Avro record name and namespace to be a value I choose, so that
a consumer resolving types by name can read what Estuary produces.

  1. A SpecificRecord consumer cannot resolve a generated class from an Estuary schema. Every value
    record is named root and every key record root.Key, so Class.forName() on the writer
    schema's fullname never matches a customer-generated class.
  2. A topic cannot be migrated from an existing producer to Estuary without changing the consumer.
    The record fullname Estuary emits cannot be made to match the fullname consumers already compile
    against.
  3. Two collections materialized to Avro cannot be told apart by type identity. Both emit a record
    named root, so any registry, catalog, or code-generation step keyed on the record name sees a
    collision.
  4. Schema Registry subject naming strategies that derive the subject from the record fullname
    (TopicRecordNameStrategy, RecordNameStrategy) resolve to the same value for every collection.

Existing tasks keep emitting root until their owner opts in, so consumers that generated classes
against root, or hardcoded it, keep working.

3. Requested behaviour

The record name and the namespace of the top-level Avro record are settable per binding. The
requesting customer needs a specific fullname in their own namespace, of the form
<their.namespace>.<their_record_name>, matching a class they generate.

Existing tasks shouldn't change behaviour without opting in. New tasks will default to the new
behaviour.

4. Current behaviour

location_to_name in crates/avro/src/schema.rs names the value record root and the key record
root.Key for every collection, with nested records extending that path
(root.<field>.RawJSON). No materialize-kafka config field affects it.

Steps to confirm: materialize any collection with message_format: avro and a Schema Registry, then
read name and namespace off the registered value schema. Both are root. Confirmed on
ghcr.io/estuary/materialize-kafka:v1.

5. Scope and prevalence

  • Two tier 1 tenants blocked by the constant fullname: one asked for it directly, one is governed on
    TopicNameStrategy and cannot use a record-name subject strategy as an alternative because those
    strategies derive the subject from the fullname (materialize-kafka: configurable schema registry subject name strategy connectors#5101).
  • The requesting tenant can decode by schema id, but wants to be able to test a staging
    materialization with the same downstream code path as production later will by overriding the
    record name.
  • 32 materialize-kafka tasks, 15 Avro, 12 live, across 4 tenants, as of 2026-09-09. Dekaf
    generates Avro from the same crate. First reported 2026-09-08.

6. References

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions