TAMIL HOW TO INSTALL AND START APACHE KAFKA IN MAC | Start Apache Kafka In Mac | InterviewDOT

1,025 views · Published 24 April 2018 · 9:34 · Indexed 31 October 2026

Channel: Interview DOT · 2018 · Science & Technology

Watch on YouTube

Click here - https://www.youtube.com/channel/UCd0U_xlQxdZynq09knDszXA?sub_confirmation=1 to get notifications. TAMIL HOW TO INSTALL AND START APACHE KAFKA IN MAC

How to install and start Apache Kafka in Mac ?

Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. 

Apache Kafka is designed for high volume publish-subscribe messages and streams, meant to be durable, fast, and scalable. 

At its essence, Kafka provides a durable message store, similar in some ways to a database, run in a server cluster, that stores streams of records in categories called TOPICS.

For example, by integrating diverse kinds of data such as likes, page clicks, searches, orders, shopping carts, and inventory, Apache Kafka can help feed data in real time into a predictive analytics engine to analyze customer behavior.

1. Download and install kafka using the following link
https://kafka.apache.org/quickstart

2. Unzip the kakfa file contents
 tar -xzf kafka_2.11-1.1.0.tgz
 cd kafka_2.11-1.1.0

3. Start the zookeeper and kafka server

 ./bin/zookeeper-server-start.sh config/zookeeper.properties
INFO Reading configuration from: config/zookeeper.properties org.apache.zookeeper.server.quorum.QuorumPeerConfig
...
Now start the Kafka server:

 ./bin/kafka-server-start.sh config/server.properties
INFO Verifying properties 
INFO Property socket.send.buffer.bytes is overridden to 1048576 
...

4. Create the topic for posting the messages

./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

5.Send some messages using the kafka producer

Kafka comes with a command line client that will take input from a file or from standard input and send it out as messages to the Kafka cluster. By default, each line will be sent as a separate message.

Run the producer and then type a few messages into the console to send to the server.

 ./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
my first message
my second message

6.Start the kafka consumer to view the messages

Kafka also has a command line consumer that will dump out messages to standard output.

 ./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
my first message
my second message

Kafka – Install Apache Kafka on Mac

 
To install Apache Kafka on Mac, Java is the only prerequisite. First we shall look into the installation steps of Java and then we shall setup Apache Kafka and run it on the Mac.

Install Java
Open a browser and hit the url http://www.oracle.com/technetwork/java/javase/downloads/index.html.
Click on JDK, check “Accept License Agreement” and download .dmg file for installation on Mac.
Double click on the downloaded file and proceed with the installation steps.
You have successfully installed Java. You may delete the .dmg file, as memory is costly on Mac.
You may verify the installation of Java on Mac, by running the following command on a Terminal.

java -version

Using Brew
If the mac users have Brew installed, they can use it for Kafka installation.

There are following steps used to install Apache Kafka using brew:

Step1: Use 'brew install kafka' and press enter key to install Kafka. Wait a while, and Kafka will be installed on the system. To test, use any kafka command.

Step2: If the 'kafka-' command is used, it will show files with and without '.sh' extension files. It is because the bin path is set in the '.bash_profile' file. So, edit the file again and remove that line from the end. Save, and exit.

Step3: Again use the command 'kafka-', now the files without '.sh' will appear.

Step4; But, to use kafka commands, the user needs to visit the kafka_directory always.

So, this is another way to install Apache Kafka on the system.

Kafka® is used for building real-time data pipelines and streaming apps. It is horizontally scalable, fault-tolerant, wicked fast, and runs in production in thousands of companies.

Apache Kafka was originally developed as a LinkedIn message queue. As a project of the Apache Software Foundation, the open source software has developed into a powerful streaming platform with a wide range of functions.

Apache Kafka can be used to store and process data streams. The system is based on a distributed architecture that is optimized for high fault tolerance and scalability. This makes Apache Kafka suitable for large amounts of data and applications in the big data environment.

Via the provided interfaces it is possible to load data streams from third-party systems or to export them to these systems. The core component of the system is a distributed commit or transaction log. Apache Kafka's advantages include low latency, high throughput and the ability to process real-time streams.

More from this channel