Additional AdminClient operations.#785
Open
j-santander wants to merge 1 commit intofede1024:masterfrom
Open
Conversation
This PR adds three admin operations that were not previously supported: - AdminClient::list_offsets: List offsets for the specified topic_partitions. - AdminClient::describe_consumer_groups: Provides a description on the requested consumer groups. - AdminClient::list_consumer_group_offsets: List offset information for the topics subscribed by a consumer group. Implementation notes: - This mostly my first Rust code, so please comment and critique. The work started because I was in search of something to do after having read rust tutorials and documents. I decided for re-doing a python tool I've recently coded for Kafka administration tasks and I found that some of the operations I was using in python were missing from the admin interface in rdkafka. Although the implementation I did in my rust tool was different, I decided to try my hand at enhancing rdkafka and this PR is the result. - For all three operations I've used a similar pattern for the result: `KafkaResult<Vec<Result<TYPE, (KEY, KafkaError)>`. This is similar to the existing GroupResult. The alternative was to insert the error value in the TYPE record, following closer the librdkafka structure, but I felt it was less in line with rust conventions. - I felt that I was adding many types and that it was best to define them in a different module/file for each operation. What I've done is re-exporting them from the admin module. - I've defined types::RDKafkaXXX for all the bindings::rd_kafka_xxxx types, using the RDKafkaXXX alias in the code, keeping the function calls to rdsys::rd_kafka_xxxx - Regarding GroupResult and ConsumerGroupResult: Technically they both come from the same rd_kafka_group_result_t, GroupResul just extracting the consumer group name. The GroupResult type was already used by the delete_groups operation. In order not to break the existing API, I'm maintaining both. Perhaps in the future they could be enhanced (or perhaps it is well as it is, as the additional information is not meaningful in the delete_groups case). - Added use cases covering the different operations. I've probably not been exhaustive, but I hope the coverage is enough. - Documentation and field and structures descriptions I've taken from a mixture of the java, python and the protocol documentation. Hope it is in line with the rest of the documentation provided.
|
@fede1024 would it be possible to merge this? I need this functionality as well |
|
Same, I would be interested to have this feature. |
|
Could you rebase this PR on top of master? |
I did it myself here: https://github.com/x10an14-nav/rust-rdkafka/tree/master [^1] 1: Found 1x bug fixed in its latest commit (use after free) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds three admin operations that were not previously supported:
Implementation notes:
interface in rdkafka. Although the implementation I did in my rust tool was different, I decided to try my hand at enhancing rdkafka and this PR is the result.
KafkaResult<Vec<Result<TYPE, (KEY, KafkaError)>. This is similar to the existing GroupResult. The alternative was to insert the error value in the TYPE record, following closer the librdkafka structure, but I felt it was less in line with rust conventions.