Hey @edenhill,
I'd love to start working on this issue.
Do you have any suggestions?

@barrotsteindev Great!

Four things needs to happen:

• add a test case in tests/0011-produce_batch.c - the test should fail prior to the fix.
• add a ..MSG_F_PARTITION flag (which will be passed in msgflags) to rdkafka.h
• produce_batch() needs to be reworked to support per-message partition if _MSG_F_PARTITION is set.
• add documentation (to produce_batch() in rdkafka.h) for this new behaviour.

I suggest you base your PR off of retry_fixes which has changes to produce_batch()

@edenhill,
I'm having trouble getting the partition count of the topic in 0011-produce_batch.c file.
https://github.com/barrotsteindev/librdkafka/blob/2713961bc1c610d02f34cef420525324381be00b/tests/0011-produce_batch.c#L229
how can i get the complete definition for the struct type so i can access its fields?
thanks in advance

These tests rely on auto.create.topics.enable=true and num.partitions=4 on the broker.
If you need something more specific you can use test_create_topic()

I suggest you create a new test (within 0011-..c, look at the bottom) not to interfere with existing tests (since that would make it hard to catch regression)

4 seems like enough.
Is it OK for me to assume there are 4 partitions?
since i count the number of messages i assigned to each partition and using the callback i compare the counts.
I have create a new test, test_per_message_partition_flag.
I have a few more finishing touches and I will create a PR.

I think it might be better to explicitly create the topic with test_create_topic() to make the test future proof.

Thanks for the guidance,
I have now implemented the test using test_create_topic().