Skip to content

Tags: zillow/zkafka

Tags

v2.3.1

Toggle v2.3.1's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
cache resolved avro schema by schema ID in unmarshal (#35)

## Problem

`avroSchemaRegistryFormatter.unmarshal` re-runs `GetSchema` +
`avro.Parse` ×2 + `avro.NewSchemaCompatibility().Resolve` **on every
message**. `Resolve` fingerprints the schema via `RecordSchema.String()`
over the whole schema tree, which is extremely expensive for large
schemas.

Profiling a consumer of ~250–800KB `PropertyChangePayloadRecord`
messages (a large, deeply-nested schema) showed the decode path
allocating **~408 MB per message** — 91% of it in `hamba/avro
(*RecordSchema).String` via
`Fingerprint`/`SchemaCompatibility.compatible` — and burning ~100–190 ms
of CPU per message, most of it GC churn from that allocation flood.

## Fix

Cache the resolved (target←data) schema on the formatter, keyed by the
message's 4-byte confluent schema ID. The target schema is fixed for a
given formatter, and the writer schema is uniquely identified by that
ID, so schema resolution now runs **once per schema ID** instead of once
per message. Held via `*sync.Map` so the value-copied formatter shares
one cache.

## Measured impact (200-message benchmark, same corpus)

| Metric | Before | After |
|---|---|---|
| hot-path time / msg | ~104–194 ms | **4.5 ms** (~25–40×) |
| total allocations | 81.6 GB (~408 MB/msg) | **1.13 GB** (~5.6 MB/msg,
~72×) |
| consumer CPU busy | 53% | **3%** (now network-bound) |

Residual `RecordSchema.String` allocation is the one-time cache fill on
the first message per schema ID. Existing formatter and evolution
(`test/evolution/*`) tests pass.

> Draft / alpha for review — flagged for discussion before finalizing
(naming, cache-key strategy, eviction if ever needed).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: stewartboyd119 <stewartb@zillowgroup.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

v2.3.0

Toggle v2.3.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
RMPMP-2110: add circuit breaker lifecycle hooks (#33)

## RMPMP-2110: Add circuit breaker lifecycle hooks

- Add `PostCircuitBreakerOpened` and `PostCircuitBreakerClosed` to
`LifecycleHooks`, with empty meta structs
(`LifecyclePostCircuitBreakerOpened` /
`LifecyclePostCircuitBreakerClosed`) so additional fields can be
introduced without breaking changes.
- Chain the new hooks in `ChainLifecycleHooks`.
- Invoke the hooks from the gobreaker `OnStateChange` handler in `Work`
factory wiring on transitions to `StateOpen` / `StateClosed`.

---------

Co-authored-by: stewartboyd119 <stewartb@zillowgroup.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

v2.2.2

Toggle v2.2.2's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
rmpmp-2040: upgrade to go 1.26, update dependencies (#32)

1. go1.26 upgrade
2. Updated direct deps
3. Updated golangci to use version 2 (updated github action)
4. Use go fix ./... to modernize codebase (use 1.26 features)

---------

Co-authored-by: stewartboyd119 <stewartb@zillowgroup.com>

v2.2.1

Toggle v2.2.1's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Rmpm-995: fix release number and changelog (#30)

Rmpm-995: fix release number and changelog (updated deps based on github warnnings)

---------

Co-authored-by: stewartboyd119 <stewartb@zillowgroup.com>

v2.2.0

Toggle v2.2.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added `WithWorkTextMapPropagator` option (#28)

1. Added `WithWorkTextMapPropagator`
2. Fixed linter in pipeline
3. Addressed linter errors

---------

Co-authored-by: stewartboyd119 <stewartb@zillowgroup.com>

v2.1.1

Toggle v2.1.1's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Bugfix: compatible schemas not properly handled during deserialization (

#25)

Forward/Backwards compatible schemas used to produce/consume messages
were not being properly deserialized.
`Test_SchemaRegistryReal_Avro_AutoRegisterSchemas_OldProducerCanBeConsumedByNewConsumer`
and
`Test_SchemaRegistryReal_Avro_AutoRegisterSchemas_NewProducerCanBeConsumedByOldConsumer`
demonstrated bug. Schema registry aware avro formatter was updated to
reconcile the bug.

---------

Co-authored-by: stewartboyd119 <stewartb@zillowgroup.com>

v2.1.0

Toggle v2.1.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added disable trace propagation option at writer instantiation (#24)

Add disable trace propagation option at writer instantiation

---------

Co-authored-by: stewartboyd119 <stewartb@zillowgroup.com>

v2.0.5

Toggle v2.0.5's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Update confluent kafka go 2.8.0 (#23)

Update confluent kafka go 2.8.0 along with other deps

---------

Co-authored-by: stewartboyd119 <stewartb@zillowgroup.com>

v2.0.4

Toggle v2.0.4's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Lifecycle bugfix tests added `Test_LifecycleChainedNil*Invocation` (#22)

1. Added supporting tests for bugfix submitted previously
2. Updated `TestMessage_Value` usage to use pointers of slices (testify
must have changed implementation. Test complains otherwise)
3. Updated deps
4. Added changelog entry in anticipation of release

---------

Co-authored-by: stewartboyd119 <stewartb@zillowgroup.com>

v2.0.3

Toggle v2.0.3's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Remove noisy log "Timed out on Read" (#20)

1. Updated `go.uber.org/mock` version `0.5.0` 
2. Updated `golang.org/x/sync` version `0.9.0`
3. Removed noisy log `Timed out on read`

---------

Co-authored-by: stewartboyd119 <stewartb@zillowgroup.com>