Category Apache Kafka

Install Apache Kafka On Windows

This quick tutorial is going to cover how to install Apache Kafka on Windows. 1. Prerequisites Install Java (The latest released version of JDK 1.8 is recommended) 2. Download Apache Kafka Distribution Download an Apache Kafka distribution from its website…. Continue Reading →

Spring Kafka – Multi-threaded Message Consumption

As mentioned in previous article, Spring Kafka provides two MessageListenerContainer implementations: KafkaMessageListenerContainer and ConcurrentMessageListenerContainer. The KafkaMessageListenerContainer provides ability to consume messages from Kafka topics in a single thread while the ConcurrentMessageListenerContainer allows us to consume messages in multi-threaded style. In… Continue Reading →

Apache Flume Kafka Source And HDFS Sink Tutorial

To continue the series about Apache Flume tutorials, I’d like to share an example about Apache Flume Kafka Source and HDFS Sink. One of popular use case today is to collect the data from various sources, send them to Apache… Continue Reading →

Apache Kafka Connect MQTT Source Tutorial

To continue the topic about Apache Kafka Connect, I’d like to share how to use Apache Kafka connect MQTT Source to move data from MQTT broker into Apache Kafka. Note that from the version 0.9, Apache Kafka introduce a new… Continue Reading →

Using Apache Kafka Docker

Continue the tutorial series about Apache Kafka, today, I’d like to share how to get Apache Kafka installed by using Apache Kafka Docker. 1. Preparation You first should have latest versions of Docker installed on your machine. If not, you can… Continue Reading →

Apache Kafka Connect Example

1. Introduction to Apache Kafka Connect Apache Kafka, which is a kind of Publish/Subscribe Messaging system, gains a lot of attraction today. We can see many use cases where Apache Kafka stands with Apache Spark, Apache Storm in Big Data… Continue Reading →

Apache Kafka Command Line Interface (CLI)

Here are some  commands often be used when we work with Apache Kafka command line interface (CLI). 1. Start the Kafka server We needs 2 steps: 1.1 Start the ZooKeeper

1.2. Start the Kafka server

2. List all… Continue Reading →

How To Write A Custom Serializer in Apache Kafka

To continue the series about Apache Kafka, I’d like to share how to write a custom serializer in  Apache Kafka. 1. Why we need a custom serializer in Apache Kafka? Apache Kafka allows us to send the messages with different… Continue Reading →

Write An Apache Kafka Custom Partitioner

Continue the series about Apache Kafka, in this post, I’d like to share some knowledge about Apache Kafka topic partition and how to write an Apache Kafka Custom Partitioner. 1. Basic about Apache Kafka Topic Partition. There are many reasons… Continue Reading →

Create Multi-threaded Apache Kafka Consumer

In previous posts, I introduced about how to get started with Apache Kafka by installing and using Java client API 0.9 as well. In this post, I’d like to share how to create multi-threaded Apache Kafka consumer. You can take… Continue Reading →

« Older posts

© 2024 HowToProgram — Powered by WordPress

Theme by Anders NorenUp ↑

NewsletterSubscribe To Learn More