This quick tutorial is going to cover how to install Apache Kafka on Windows.
1. Prerequisites
- Install Java (The latest released version of JDK 1.8 is recommended)
2. Download Apache Kafka Distribution
Download an Apache Kafka distribution from its website. The current version is 0.11.0.0. After downloading, extract it to any desired location, for example, D:\kafka_2.11-0.11.0.0. Then open Command Prompt (cmd) and go to that directory:
|
1 |
cd D:\kafka_2.11-0.11.0.0 |
3. Start the server
3.1. Start ZooKeeper
Kafka uses ZooKeeper so you will need to first start a ZooKeeper server first. And you can use your own ZooKeeper or use the single-node ZooKeeper instance packaged with Apache Kafka. In this tutorial, we will use the ZooKeeper packaged with Kafka.
Let’s start the ZooKeeper by running the below command:
|
1 |
.\bin\windows\zookeeper-server-start.bat config\zookeeper.properties |
The successful start will output the logs to the command prompt similarly to below:
|
1 2 |
INFO maxSessionTimeout set to -1 (org.apache.zookeeper.server.ZooKeeperServer) INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory) |
ZooKeeper is running on the port 2181.
3.2. Start Kafka Server
To start the Kafka server, we need to open another command prompt, go to the Apache Kafka directory and run following commands:
|
1 2 |
cd D:\kafka_2.11-0.11.0.0 .\bin\windows\kafka-server-start.bat .\config\server.properties |
Kafka broker will listen on port 9092.
4. Basic Operations with Apache Kafka
4.1. Create a topic
Let’s create a topic named “hello-kafka” with a single partition and only one replica by opening a new command prompt and run the following command:
|
1 |
.\bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic hello-kafka |
We can now see that topic if we run the list topic command:
|
1 |
.\bin\windows\kafka-topics.bat --list --zookeeper localhost:2181 |
The output is:
|
1 |
hello-kafka |
Note that instead of creating topic manually, we can configure Kafka brokers so that it can automatically create topics when a non-existent topic is published to.
4.2 Start Producer and Send Messages
We can use Kafka command line client that will take input from standard input and send it out as messages to the Kafka cluster. By default, each line will be sent as a separate message. Let’s send some messages to the server (topic: “hello-kafka”) by opening a new Command Prompt and run the following command:
|
1 |
.\bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic hello-kafka |
Then enter some messages into the console, for example:
|
1 2 |
hello kafka this is the first message |
4.2 Start Consumer and Receive Messages
In similar to above, we can use Kafka command line to start consumer that will dump out messages to standard output. Let’s consume messages from the “hello-kafka” topic:
|
1 |
.\bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic hello-kafka --from-beginning |
The output will be:
|
1 2 |
hello kafka this is the first message |
For some more operations with Apache Kafka, you can refer to another related article Apache Kafka Command Line Interface
5. Conclusion
The tutorial has illustrated us how to install Apache Kafka on Windows step by step. We got through how to download Kafka distribution, start ZooKeeper, Apache Kafka server, send messages and receive messages from Kafka server as well.
If you want to learn more about Apache Kafka, please check our other related articles:
Getting started with Apache Kafka 0.9
Apache Kafka 0.9 Java Client API Example
Create Multi-threaded Apache Kafka Consumer
How To Write A Custom Serializer in Apache Kafka
Write An Apache Kafka Custom Partitioner
Apache Flume Kafka Source And HDFS Sink Tutorial
Apache Kafka Connect MQTT Source Tutorial