Zookeeper stand-alone environment and cluster environment construction

Zookeeper stand-alone environment and cluster environment construction

1. Single machine environment construction#

1.1 Download

Download the corresponding version of Zookeeper. Here I downloaded version 3.4.14. Official download address: https://archive.apache.org/dist/zookeeper/

# wget https://archive.apache.org/dist/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz

1.2 Unzip

# tar -zxvf zookeeper-3.4.14.tar.gz

1.3 Configuring Environment Variables

# vim /etc/profile

Add environment variables:

export ZOOKEEPER_HOME=/usr/app/zookeeper-3.4.14
export PATH=$ZOOKEEPER_HOME/bin:$PATH

Make the configured environment variables take effect:

# source /etc/profile

1.4 Modify the configuration#

Go to the conf/ directory of the installation directory, copy the configuration sample and modify it:

# cp zoo_sample.cfg zoo.cfg

Specify the data storage directory and log file directory (directories do not need to be created in advance, the program will create them automatically). The complete configuration after modification is as follows:

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/usr/local/zookeeper/data
dataLogDir=/usr/local/zookeeper/log
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

Configuration parameter description:

•tickTime: The basic time unit used for calculations. For example, session timeout: N*tickTime;
•initLimit: used for clusters, allows the initial connection time for slave nodes to connect and synchronize to the master node, expressed as a multiple of tickTime;
•syncLimit: used for cluster, the length of time for sending messages, requests and responses between the master node and the slave nodes (heartbeat mechanism);
•dataDir: data storage location;
•dataLogDir: log directory;
•clientPort: The port used for client connections, default is 2181

1.5 Startup

Since the environment variables have been configured, you can directly start it using the following command:

zkServer.sh start

1.6 Verification

Use JPS to verify whether the process has been started. If QuorumPeerMain appears, it means the process has been started successfully.

[root@hadoop001 bin]# jps
3814 QuorumPeerMain

2. Cluster environment construction#

To ensure high availability of the cluster, the number of nodes in the Zookeeper cluster should be an odd number, with at least three nodes, so here we demonstrate building a three-node cluster. Here I use three hosts for construction, the host names are hadoop001, hadoop002, and hadoop003.

2.1 Modify the configuration#

Unzip a zookeeper installation package and modify its configuration file zoo.cfg to the following content. Then use the scp command to distribute the installation package to the three servers:

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper-cluster/data/
dataLogDir=/usr/local/zookeeper-cluster/log/
clientPort=2181
# server.1 The 1 is the server ID, which can be any valid number. It indicates the server node. This ID should be written to the myid file under the dataDir directory. # Specify the inter-cluster communication port and election port server.1=hadoop001:2287:3387
server.2=hadoop002:2287:3387
server.3=hadoop003:2287:3387

2.2 Identifying Nodes#

Create a new myid file in the dataDir directory of the three hosts and write the corresponding node ID. The Zookeeper cluster identifies cluster nodes through the myid file, and communicates with nodes through the node communication port and election port configured above to elect the Leader node.

Create a storage directory:

# All three hosts execute the command mkdir -vp /usr/local/zookeeper-cluster/data/

Create and write the node ID to the myid file:

# hadoop001 host echo "1" > /usr/local/zookeeper-cluster/data/myid
# hadoop002 host echo "2" > /usr/local/zookeeper-cluster/data/myid
# hadoop003 host echo "3" > /usr/local/zookeeper-cluster/data/myid

2.3 Start the cluster#

On each of the three hosts, execute the following commands to start the service:

/usr/app/zookeeper-cluster/zookeeper/bin/zkServer.sh start

2.4 Cluster Verification#

After startup, use zkServer.sh status to view the status of each node in the cluster. As shown in the figure: the three node processes are started successfully, and hadoop002 is the leader node, hadoop001 and hadoop003 are follower nodes.

   

For more articles in the Big Data series, see GitHub Open Source Project: Getting Started with Big Data

Summarize

The above is the introduction of Zookeeper stand-alone environment and cluster environment construction. I hope it will be helpful to everyone. If you have any questions, please leave me a message and I will reply to you in time. I would also like to thank everyone for their support of the 123WORDPRESS.COM website!
If you find this article helpful, please feel free to reprint it and please indicate the source. Thank you!

You may also be interested in:
  • Summary of common commands for building ZooKeeper3.4 middleware under centos7
  • Tutorial diagram of building a Hadoop high-availability cluster based on ZooKeeper
  • How to use Zookeeper to build a configuration center in SpringCloud
  • Sharing steps to build Zookeeper management center under Linux
  • ActiveMQ is built based on zookeeper's master-slave (levelDB Master/Slave)

<<:  MySQL 8.0.12 winx64 decompression version installation graphic tutorial

>>:  How to install and configure MySQL 8.0.12 decompressed version under Windows 10 with graphic tutorials

Recommend

JavaScript Interview: How to implement array flattening method

Table of contents 1 What is array flattening? 2 A...

How to align text boxes in multiple forms in HTML

The form code is as shown in the figure. The styl...

Use of MySQL triggers

Triggers can cause other SQL code to run before o...

Element dynamic routing breadcrumbs implementation example

To master: localStorage, component encapsulation ...

Implementation of MySQL Shell import_table data import

Table of contents 1. Introduction to import_table...

JavaScript Snake Implementation Code

This article example shares the specific code of ...

A detailed introduction to Linux file permissions

The excellence of Linux lies in its multi-user, m...

Detailed explanation of webpage screenshot function in Vue

Recently, there is a requirement for uploading pi...

Tutorial on installing lamp-php7.0 in Centos7.4 environment

This article describes how to install lamp-php7.0...

Vue Page Stack Manager Details

Table of contents 2. Tried methods 2.1 keep-alive...

MySQL SQL statement analysis and query optimization detailed explanation

How to obtain SQL statements with performance iss...

Coexistence of python2 and python3 under centos7 system

The first step is to check the version number and...

Node.js makes a simple crawler case tutorial

Preparation First, you need to download nodejs, w...

Tutorial on installing mysql5.7.23 on Ubuntu 18.04

This article shares with you the specific method ...