- Updated to Apache Kafka 0.8.1.1
- Configuration optimized for usage on Windows machines
- Windows batch scripts fixed (taken from https://github.com/HCanber/kafka by @HCanber)
- Code examples repaired and refactored
- Download and install Apache Kafka 0.8.1.1 (I used the recommended Scala 2.9.2 binary)
- Copy the scripts from /bat into
/bin/windowsof your Kafka installation folder (overwrite existing scripts) - Copy the property files from /config into
/configof your Kafka installation folder (overwrite existing files)
- Open command line in your Kafka installation folder
- Launch Zookeeper with
.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties - Open a second command line in your Kafka installation folder
- Launch single Kafka broker:
.\bin\windows\kafka-server-start.bat .\config\server.properties - Open a third command line in your Kafka installation folder
- Create a topic:
.\bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test - Start a console consumer for that topic:
.\bin\windows\kafka-console-consumer.bat --zookeeper localhost:2181 --topic test --from-beginning - From a fourth command line or your IDE run SimpleProducer with topic and message as arguments:
java SimpleProducer test HelloKafka - The message HelloKafka should appear in the console consumer's log
- Open command line in your Kafka installation folder
- Launch Zookeeper with
.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties - Open a second command line in your Kafka installation folder
- Launch first Kafka broker:
.\bin\windows\kafka-server-start.bat .\config\server-1.properties - Open a third command line in your Kafka installation folder
- Launch second Kafka broker:
.\bin\windows\kafka-server-start.bat .\config\server-2.properties - Open a fourth command line in your Kafka installation folder
- Create a topic:
.\bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 2 --partitions 4 --topic kafkatest - Start a console consumer for that topic:
.\bin\windows\kafka-console-consumer.bat --zookeeper localhost:2181 --topic kafkatest --from-beginning - From a fifth command line or your IDE run MultiBrokerProducer with topic as argument:
java MultiBrokerProducer kafkatest - Ten messages starting with This message is for key - (...) should appear in the console consumer's log
- Launch multi-broker Kafka cluster and create topic
kafkatestas described in step 1-8 of Java Producer with Message Partitioning - From another command line or your IDE run SimpleHLConsumer with topic as argument:
java SimpleHLConsumer kafkatest - From another command line or your IDE run MultiBrokerProducer with same topic as argument:
java MultiBrokerProducer kafkatest - Ten messages starting with This message is for key - (...) should appear in the log of the SimpleHLConsumer
- Launch multi-broker Kafka cluster and create topic
kafkatestas described in step 1-8 of Java Producer with Message Partitioning - From another command line or your IDE run MultiThreadHLConsumer with topic and number of threads as argument:
java MultiThreadHLConsumer kafkatest 4 - From another command line continuously produce messages by running MultiBrokerProducer several times in a row:
java MultiBrokerProducer kafkatest(Note: You must start producing messages within 10sec after starting the consumer class, otherwise the consumer will shut down) - Messages starting with Message from thread (...) should appear in the log of the MultiThreadHLConsumer spread among the four threads
