How do I delete/clean Kafka queued messages without deleting Topic -
is there way delete queue messages without deleting kafka topics?
want delete queue messages when activating consumer.
i know there several ways like:
resetting retention time
$ ./bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --config retention.ms=1000
deleting kafka files
$ rm -rf /data/kafka-logs/<topic/partition_name>
in 0.11 or higher can run bin/kafka-delete-records.sh command mark messages deletion.
https://github.com/apache/kafka/blob/trunk/bin/kafka-delete-records.sh
for example publish 100 messages
seq 100 | ./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic mytest
then delete 90 of 100 messages new kafka-delete-records.sh command line tool
./bin/kafka-delete-records.sh --bootstrap-server localhost:9092 --offset-json-file ./offsetfile.json
where offsetfile.json contains
{"partitions": [{"topic": “mytest", "partition": 0, "offset": 90}], "version":1 }
and consume messages beginning verify 90 of 100 messages indeed marked deleted.
./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic mytest --from-beginning 91 92 93 94 95 96 97 98 99 100
Comments
Post a Comment