Skip to content

ref(consumer): set message.max.bytes for DLQ/BLQ producers#7897

Open
kylemumma wants to merge 2 commits intomasterfrom
kyle/dlq-producer-message-max-bytes
Open

ref(consumer): set message.max.bytes for DLQ/BLQ producers#7897
kylemumma wants to merge 2 commits intomasterfrom
kyle/dlq-producer-message-max-bytes

Conversation

@kylemumma
Copy link
Copy Markdown
Member

@kylemumma kylemumma commented Apr 23, 2026

the dlq isnt actually used right now, so this should be fine. the value aligns with the topic https://github.com/getsentry/sentry-kafka-schemas/blob/main/topics/snuba-items.yaml#L18

Summary

  • Override `message.max.bytes` to 10MB on the DLQ producer `KafkaConfig` in `consumer.rs`. This config is also cloned for the BLQ producer, so both get the same broker-max limit.
  • Mirrors the existing override already applied to the Python replacements producer in `snuba/consumers/consumer_builder.py`.

Test plan

  • `cargo check -p rust_snuba` passes locally
  • Consumer starts cleanly and produces to DLQ/BLQ topics with large payloads that previously would have exceeded the 1MB default

🤖 Generated with Claude Code

Override message.max.bytes to 10MB (the broker max) on the shared DLQ
producer KafkaConfig, which is also cloned for the BLQ producer. This
mirrors the existing override on the Python replacements producer so
large payloads can be produced without exceeding the 1MB default.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@kylemumma kylemumma requested a review from a team as a code owner April 23, 2026 20:51
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit edf6903. Configure here.

Comment thread rust_snuba/src/consumer.rs
@kylemumma kylemumma changed the title ref(consumer): Raise message.max.bytes for DLQ/BLQ producers ref(consumer): set message.max.bytes for DLQ/BLQ producers Apr 23, 2026
Comment thread rust_snuba/src/consumer.rs Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants