9.1 Zookeeper Cluster Setup

This topic provides the systematic instructions to install Zookeeper cluster setup.

Note:

It is recommended to keep minimum number of zookeeper nodes as 3 or higher in odd numbers. The sample configs provided below are for 3 node setup.

Note:

To restart the server, refer to the Restart Server section in Configuration and Deployment Guide.
Before proceeding, ensure that the below installation is done.
  • JDK is installed in all node machines.
  • Download kafka_2.13-3.7.0 and extract the binary in all node machines. Kafka can be found at <Unzip the file>/THIRD_PARTY_SOFTWARES/KAFKA/ARCHIVE

Note:

Please note that the zookeeper that we will be using is bundled within kafka.
  1. Untar/unzip the kafka binary and move them to the folder that will be the kafka home directory.
  2. The user should create three folders, named node1, node2, and node3, in the Kafka parent directory for unzipping the Kafka binary. Kafka and Zookeeper will store their logs in these folders. Create two subfolders, named kafka-logs and zookeeper, in each of these folders.

    Note:

    Clear the logs folder if it is already present.
  3. In the zookeeper folder of every nodeX(e.g. node1), create a myid file without any file extension. e.g. <Kafka-Home>/node1/zookeeper/myid
    The myid file contains a single line text of machine ID. The server 1 myid has the text as 1 and nothing else, similarly for server 2 and 3 will contain value as 2 and 3 respectively. The ID must be unique within the ensemble and have a value between 1 and 255.
  4. Edit the configuration file named zookeeper.properties at <kafka home directory> /kafka_2.13-3.7.0/config. Inside kafka/config folder there would be a file named zookeeper.properties, copy that and create three similar files zookeeper1.properties, zookeeper2.properties, zookeeper3.properties.
  5. Add the below set of properties and values to the file.
    dataDir= <kafka home directory>/nodeX/zookeeper
e.g. node1/zookeeper
tickTime=2000
clientPort= Zookeeper client Port value (2181)
initLimit=10
syncLimit=5

server.1=<hostname> :< peer port> :< leader port>
#1 is the id that we put in myid file.

server.2= <hostname> :< peer port> :< leader port>
#2 is the id that we will put in myid file of second
node

server.3=<hostname> :< peer port> :< leader port>
#3 is the id that we will put in myid file of third

    Note:

    Any odd number of zookeeper servers can be configured under the cluster. The properties with the key as server.x are only required if you want to have a multinode kafka setup..
  6. Start the zookeeper on each node machine.
  7. Navigate to <kafka home directory>/kafka_2.13-3.7.0 and execute the below command.
    /bin/zookeeper-server-start.sh /config/zookeeper1.properties
    /bin/zookeeper-server-start.sh /config/zookeeper2.properties
    /bin/zookeeper-server-start.sh /config/zookeeper3.properties
  8. To see the leader and followers in the cluster, run the below command on each node.
    echo stat | nc localhost 2181
  9. To check the zoo cluster functioning (dynamic leader election), kill the zookeeper process on the leader node and check again with the below command on the remaining live zookeeper node.
    echo stat | nc localhost 2181

Parent topic: Oracle Banking Microservices Architecture Software Deployment