centos下搭建kafka集群

1、下载安装zookeeper

1、下载安装包

wget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.6.2/apache-zookeeper-3.6.2-bin.tar.gz

2、解压

tar -zxvf apache-zookeeper-3.6.2-bin.tar.gz

3、改文件夹名称，去掉版本号

mv apache-zookeeper-3.6.2-bin zookeeper

4、创建data目录后，进入conf文件夹，修改zoo_sample.cfg

cd  zookeeper
mkdir data
cd conf/
mv zoo_sample.cfg zoo.cfg 
vi zoo.cfg

//修改两处
（1）dataDir=/usr/local/zookeeper/data  (前面刚创建的目录)
（2）在最后添加：
server.0=172.19.130.156:2888:3888
server.1=172.19.130.157:2888:3888
server.2=172.19.130.158:2888:3888

5、创建服务器标识

进入data目录，创建文件myid，内容为0

这里单台的zookeeper已经完成，先告一段落，进行kafka的安装后再配置另两台并启动集群

2、下载安装kafka

1、回到local目录下载kafka安装包解压、改名

wget https://mirrors.tuna.tsinghua.edu.cn/apache/kafka/2.6.0/kafka_2.12-2.6.0.tgz
tar -zxvf kafka_2.12-2.6.0.tgz
mv kafka_2.12-2.6.0 kafka

2、进入kafka目录并修改配置文件

cd kafka/config/
vi server.properties

//需要修改三处地方
broker.id=0
listeners=PLAINTEXT://172.19.130.156:9092
zookeeper.connect=172.19.130.156:2181,172.19.130.157:2181,172.19.130.158:2181

3、配置环境变量

vi /etc/profile


//在文件最下方加上
export KAFKA_HOME=/usr/local/kafka
export ZOOKEEPER_HOME=/usr/local/zookeeper
export PATH=$PATH:${KAFKA_HOME}/bin:$ZOOKEEPER_HOME/bin:$ZOOKEEPER_HOME/conf


//更新生效
source /etc/profile

　　

3、拷贝zookeeper和kafka至另外两台机器

scp -r zookeeper/  root@172.19.130.157:/usr/local/
scp -r zookeeper/  root@172.19.130.158:/usr/local/

scp -r kafka/  root@172.19.130.157:/usr/local/
scp -r kafka/  root@172.19.130.158:/usr/local/

4、登录第二台服务器

1、修改配置

cd /usr/local/zookeeper/data
vi myid   //内容修改为1

cd /usr/local/kafka/config/
vi server.properties
//修改两处 
broker.id=1 
listeners=PLAINTEXT://172.19.130.157:9092

2、配置环境变量

vi /etc/profile


//在文件最下方加上
export KAFKA_HOME=/usr/local/kafka
export ZOOKEEPER_HOME=/usr/local/zookeeper
export PATH=$PATH:${KAFKA_HOME}/bin:$ZOOKEEPER_HOME/bin:$ZOOKEEPER_HOME/conf


//更新生效
source /etc/profile

5、登录第三台服务器

1、修改配置

cd /usr/local/zookeeper/data
vi myid   //内容修改为2

cd /usr/local/kafka/config/
vi server.properties
//修改两处
broker.id=2
listeners=PLAINTEXT://172.19.130.158:9092

2、配置环境变量

vi /etc/profile


//在文件最下方加上
export KAFKA_HOME=/usr/local/kafka
export ZOOKEEPER_HOME=/usr/local/zookeeper
export PATH=$PATH:${KAFKA_HOME}/bin:$ZOOKEEPER_HOME/bin:$ZOOKEEPER_HOME/conf


//更新生效
source /etc/profile

6、启动zookeeper集群

cd /usr/local/zookeeper/bin

zkServer.sh start   //3台服务分别执行

zkServer.sh status  //查看启动成功与否以及选举状态

7、启动kafka集群

cd /usr/local/kafka
./bin/kafka-server-start.sh -daemon config/server.properties