How to deploy kafka in docker
1. Build Docker Here I use docker-compose deployment directly, so you need to install compose in advance. First, create a new directory and create a new yml file in the directory
The contents of the file are as follows:
version: '2'
services:
zookeeper:
image: wurstmeister/zookeeper
volumes:
- ./data:/data
ports:
- "2181:2181"
kafka:
image: wurstmeister/kafka
ports:
- "9092:9092"
environment:
KAFKA_ADVERTISED_HOST_NAME:127.0.0.1
KAFKA_MESSAGE_MAX_BYTES: 2000000
KAFKA_CREATE_TOPICS: "Topic1:1:3,Topic2:1:1:compact"
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
volumes:
- ./kafka-logs:/kafka
- /var/run/docker.sock:/var/run/docker.sock
kafka-manager:
image: sheepkiller/kafka-manager
ports:
- 9020:9000
environment:
ZK_HOSTS: zookeeper:2181
Then use this yml file to start our project $ docker-compose up -d You can see that three new containers have been created.
2. Enter the containerWe enter the interactive mode of the kafka container with the following command $ docker exec -it kafkademo01_kafka_1 /bin/bash Because the higher version of Kafka has built-in Zookeeper, we do not need to enter the Zookeeper container. Therefore, the deployment of zookeeper in the yml file can be omitted. Then enter the root directory of kafka $ cd /opt/kafka 3. Modify the configuration file$ cd /config
The first thing to modify is the zookeeper configuration file: zookeeper.properties dataDir=/opt/kafka/zooLogs clientPort=2182 maxClientCnxns=0 admin.enableServer=false Then modify the kafka configuration file: server.porperties
############################## Server Basics ##############################
broker.id=0
############################## Socket Server Settings ##############################
listeners=PLAINTEXT://127.0.0.1:9093
############################## Socket Server Settings ##############################
listeners=PLAINTEXT://127.0.0.1:9093
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
############################### Log Basics ##############################
log.dirs=/opt/kafka/kafkaLogs
num.partitions=1
num.recovery.threads.per.data.dir=1
############################## Internal Topic Settings #############################
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
############################### Log Retention Policy #############################
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
############################# Zookeeper #############################
zookeeper.connect=127.0.0.1:2182
zookeeper.connection.timeout.ms=18000
############################### Group Coordinator Settings ###############################
group.initial.rebalance.delay.ms=0
port=9093
advertised.host.name=127.0.0.1
message.max.bytes=2000000
advertised.port=90934. Test KafkaHere are some basic commands: Start ZooKeeper zookeeper-server-start.sh ../config/zookeeper.properties Start Kafka kafka-server-start.sh ../config/server.properties Create a Theme kafka-topics.sh --create --zookeeper 127.0.0.1:2182 --replication-factor 1 --partitions 1 --topic test View created topics kafka-topics.sh --list --zookeeper 127.0.0.1:2182 Producer kafka-console-producer.sh --broker-list 127.0.0.1:9093 --topic test consumer kafka-console-consumer.sh --bootstrap-server 127.0.0.1:9093 --topic test --from-beginning This is the end of this article about the steps to deploy Kafka with Docker. For more information about deploying Kafka with Docker, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future! You may also be interested in:
|
<<: Tkinter uses js canvas to achieve gradient color
>>: Use a table to adjust the format of the form controls to make them look better
Recommend
A brief discussion on the optimization of MySQL paging for billions of data
Table of contents background analyze Data simulat...
Solution to prevent caching in pages
Solution: Add the following code in <head>: ...
Tomcat uses thread pool to handle remote concurrent requests
By understanding how tomcat handles concurrent re...
When to use Map instead of plain JS objects
Table of contents 1. Map accepts any type of key ...
Solution to Nginx 500 Internal Server Error
Today, when I was using Nginx, a 500 error occurr...
Learn Vue middleware pipeline in one article
Often when building a SPA, you will need to prote...
How to install MySQL under Linux (yum and source code compilation)
Here are two ways to install MySQL under Linux: y...
The content of the commonly used input text box is automatically vertically centered and the default prompt text is empty when clicked
Three functions: 1. Automatic vertical centering o...
MySQL trigger detailed explanation and simple example
MySQL trigger simple example grammar CREATE TRIGG...
10 SQL statement optimization techniques to improve MYSQL query efficiency
The execution efficiency of MySQL database has a ...
Database issues and pitfalls in connecting to cloud servers with Navicat Premium15
When using a cloud server, we sometimes connect t...
Detailed tutorial on using the Prettier Code plugin in vscode
Why use prettier? In large companies, front-end d...
Detailed explanation of MySQL master-slave database construction method
This article describes how to build a MySQL maste...
The viewport in the meta tag controls the device screen css
Copy code The code is as follows: <meta name=&...
VMware15 installation of CentOS7 detailed process and common problems (picture and text)
1. Installation package preparation VMware-player...