Skip to content

Align Kafka loader with project conventions and improve robustness#46

Open
fordN wants to merge 1 commit intokrishna/kafkafrom
ford/kafka-updates
Open

Align Kafka loader with project conventions and improve robustness#46
fordN wants to merge 1 commit intokrishna/kafkafrom
ford/kafka-updates

Conversation

@fordN
Copy link
Contributor

@fordN fordN commented Mar 3, 2026

A few suggestions for the Kafka loader @incrypto32! Let me know what you think

Changes

  • Fix SUPPORTS_TRANSACTIONS flag (False, since base class routes via hasattr)
  • Add health_check() method using sender thread liveness
  • Move test file to canonical location (backends/test_kafka.py)
  • Add health check integration test
  • Add make test-kafka target
  • Improve crash recovery to skip reorg when no uncommitted batches
  • Flush producer before committing transactions
  • Reduce container startup wait (already waits for KafkaServer log)

- Fix SUPPORTS_TRANSACTIONS flag (False, since base class routes via hasattr)
- Add health_check() method using sender thread liveness
- Move test file to canonical location (backends/test_kafka.py)
- Add health check integration test
- Add make test-kafka target
- Improve crash recovery to skip reorg when no uncommitted batches
- Flush producer before committing transactions
- Reduce container startup wait (already waits for KafkaServer log)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant