A custom KafkaMetricsReporter intended to be dropped into a Kafka broker cluster that will send all consumer offset metrics to a configured Graphite server. This reporter produces the same results as the Kafka ConsumerOffsetChecker command-line tool for all consumers currently consuming from the broker cluster.

Make this jar available to a Kafka broker by placing it on the classpath, and add the following lines to your Kafka configuration...

# custom metrics logger config
kafka.metrics.reporters=com.krux.metrics.reporter.KafkaGraphiteMetricsReporter
kafka.graphite.metrics.reporter.enabled=true
kafka.graphite.metrics.host=localhost
kafka.graphite.metrics.port=2003
# optional - in a cluster, only this machine will send stats
kafka.broker.stats.sender=my.host.name.com

#full tree prefix
kafka.graphite.metrics.env=prod

#if true, log all reported metrics to stdout. Default is false. (useful for testing)
kafka.graphite.metrics.log.debug=true

Metrics will fall under a prefix of "env".kafka.consumer.topic_lag prefix, where "env" is specified by the kafka.graphite.metrics.env in the config settings above.

To prevent all brokers in your cluster from sending the same, duplicated information to Graphite, you should set the kafka.broker.stats.sender property to the fully qualified domain name of one of the machines in the broker cluster. Only that machine will forward consumer lag stats. This approach allows a single, common configuration across all hosts in your cluster.

All releases are available as pre-built jars here. You can also build the source from scratch using Maven.

mvn clean package

will produce a jar with all dependencies at target/kafka-metrics-reporter-1.1.4.jar.