Skip to content

edwardsujono/reverse-kafka-logger

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

23 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Reverse Kafka Logger

Build Status License: MIT

pip library that is used to read inversely the kafka message. The best part is each partition of topic will be run in parallel.

Background

A need to grep the message from kafka reversely because often we want to grep something from the most recent offset.

Problem

All the available debugging tools of kafka never introduced a feature to easily grep the message from the latest offset to the beginning of the offset. Thus, I created this open source project.

usage

pip install rervese-kafka-logger

reverse-kafka-logger --topic='topic' --brokers='localhost:9092,localhost:9093' --regex='search this'

Install

setup package: python setup.py dist

install package: pip install --no-index ./dist/{NEW_PACKAGE_NAME}

About

read the kafka message from the most recent offset. work fast with multiple partition because of multi-processing

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages