Tags: kafkaex/kafka_ex
Tags
Release/v1.1.1 (#588) * feat(client): add tracked_topics + metadata-missing state fields * feat(client): add pure MetadataLog decision helpers * fix(client): refresh metadata for used topics only, not the whole catalog * fix(client): merge partial metadata on give-up and log missing topics edge-triggered * test(client): cover metadata give-up merge and edge-triggered missing-topic logging * refactor(client): align MetadataLog topic-name spec with codebase convention * test(client): integration test - unrelated topic deletion does not storm logs * chore: release v1.1.1 — metadata tracks only used topics; storm-free logging * refactor(client): trim metadata comments to one-line whys; add missing-set-change test * test(client): prove metadata narrowing = latency-not-loss (unit + lifecycle integration) + consumer-group topic-deletion resilience * fix(test): make lifecycle metadata_refresh_test async: false (CaptureLog is VM-wide) * test(client): make metadata_missing_test hermetic by advertising the setup socket's ephemeral port (#589) The setup binds a real socket on an ephemeral port via :gen_tcp.listen(0), but build_v0_metadata_response hard-coded the advertised broker port as 9092. Because ClusterMetadata.merge_brokers/2 only preserves a live socket when both host and port match, the port mismatch caused the client to drop the stubbed socket and attempt a real, unstubbed reconnect to localhost:9092 on the second update_metadata call. On a clean host (CI) that reconnect fails with :econnrefused, producing 3 flaky failures; it only passed locally when something happened to be listening on 9092. Thread the setup socket's real ephemeral port through setup -> stub_metadata_response -> build_v0_metadata_response and advertise it instead of 9092, so the test no longer depends on the network. * test(client): make metadata_narrowing_test hermetic by threading the setup socket's ephemeral port Same root cause as #589 but in the narrowing test: build_v0_metadata_response hard-coded the advertised broker port as 9092, so merge_brokers/2 dropped the stubbed socket (host+port must both match) and the second update_metadata call attempted a real reconnect to localhost:9092 -> :econnrefused on CI. Thread the setup socket's ephemeral port through instead of 9092 so the test is network-free. * fix(client): stop re-running the metadata retry-sleep for already-known-missing topics A tracked topic that is permanently missing (e.g. deleted) was driving the full retrieve_metadata retry ladder — @retry_count network attempts with a 300ms sleep between each (~600ms) inside the client GenServer — on every periodic refresh, head-of-line-blocking all requests on that client every ~30s. Spend the retry-sleep budget only on newly-missing topics; a topic already in metadata_missing goes straight to the merge/give-up branch. Transient-gap tolerance on first miss is preserved; the recurring stall is gone. * docs(changelog): note the deleted-but-tracked-topic stall fix and the tracked-topic pruning limitation * docs(changelog): tighten the metadata-fix entries * feat(consumer): seed a GenConsumer's own client with initial_topics: [topic] When a GenConsumer starts its own client (no shared :client passed), the client started with an empty tracked set, so its first metadata refresh was scoped to the whole cluster catalog. In a group with N partition-consumers that meant N simultaneous whole-catalog fetches at startup/rebalance. Seed initial_topics so each client's first refresh is scoped to just its topic. Also broadens the ssl_options typespec to cover any Erlang :ssl option (it only listed four keys but Config.ssl_options/0 passes :verify etc.), surfaced by type-checking the Client.start_link stub in the new test. * docs(changelog): note the consumer client initial_topics scoping * refactor(client): review cleanups — boolean newly-missing check, drop dead MetadataLog.heartbeat_ms/0, honest refresh comment Post-review polish (no behavior change): - parse_metadata_response: Enum.any? boolean instead of building a rejected list - MetadataLog: remove the unused heartbeat_ms/0 accessor (@heartbeat_ms stays) - update_metadata: comment no longer claims 'never the whole cluster catalog' (an empty tracked set still fetches all until the [] vs nil builder fix) --------- Co-authored-by: Àbéjídé Àyodélé <abejideayodele@gmail.com>
refactor(client): review cleanups — boolean newly-missing check, drop… … dead MetadataLog.heartbeat_ms/0, honest refresh comment Post-review polish (no behavior change): - parse_metadata_response: Enum.any? boolean instead of building a rejected list - MetadataLog: remove the unused heartbeat_ms/0 accessor (@heartbeat_ms stays) - update_metadata: comment no longer claims 'never the whole cluster catalog' (an empty tracked set still fetches all until the [] vs nil builder fix)
chore: release v1.1.0 (#576) * chore: release v1.1.0 Bump @Version to 1.1.0 and finalize the CHANGELOG: retitle Unreleased → 1.1.0 (2026-07-16), add the missing ListGroups admin API (#565) entry, and drop the stale "heartbeat" mention from the :request_timeout bullet (heartbeat derives its own deadline). * fix: retry data-plane requests on :econnreset / :not_connected (v1.0.x parity) The v1.1.0 retry-classification rework narrowed data-plane retriability to transient_error? ∪ leadership_error?; :econnreset and :not_connected fell outside it and began failing fast, where v1.0.x retried all transport errors and self-healed (the client retry loop reconnects the broker on the next attempt). Classify both as transient again. * docs: finalize 1.1.0 CHANGELOG (release date + data-plane retry wording) Set the 1.1.0 date to the release day and fold the transient-connection-drop clarification (:closed / :econnreset / :not_connected / :timeout still retry) into the existing 'data-plane fail fast on fatal errors' Changed entry, instead of a standalone Fixed bullet that implied a released regression (the retry narrowing never shipped — it was introduced and refined within 1.1.0).
chore: add AGENTS.md as canonical agent guide; CLAUDE.md symlinks to … …it (#558) Establish AGENTS.md (the cross-tool convention read by Claude Code, Cursor, Copilot, etc.) as the single source of agent guidance for this repo: - Add ash-project/usage_rules (dev-only) and a `usage_rules` mix.exs config that inlines Kayrock's usage rules into AGENTS.md under a managed marker block. KafkaEx is built on Kayrock, so its protocol rules are the relevant dependency rules. Refresh with `mix usage_rules.sync`. - Hand-written top section carries the former CLAUDE.md project guidance (architecture, commands, conventions) plus a pointer to usage-rules.md for public API call patterns. - CLAUDE.md becomes a symlink to AGENTS.md so Claude Code keeps reading it.
PreviousNext