Kafaka 集群搭建（Windows环境）
关于Kafka 简介可以查看 https://www.cnblogs.com/lentoo/p/7785004.html ，里面介绍比较详细。
一、Zookeeper 安装
-
从官网下载，本次演示使用的是最新的 3.7.0的版本
-
zookeeper 配置
（1）下载解压后，在conf文件夹下，将默认配置文件 zoo_sample.cfg，复制后重命名一份为 200.cfg 。
（2）配置 zoo.cfg ，注意端口不要太大，否则后面启动会报错
tickTime=2000 initLimit=10 syncLimit=5 # snapshot持久化 dataDir=../data dataLogDir=../logs clientPort=2181 #集群配置 server.A = B:C:D #A 是一个数字，表示第几号服务器 #B 服务器IP地址 #C 端口号，当前服务器与集群的 Leader 服务器交换信息的接口 #D Leader 挂掉时用来进行选举 leader所用的端口 server.1=127.0.0.1:2387:1387 server.2=127.0.0.1:2388:1388 server.3=127.0.0.1:2389:1389-
复制 zookeeper 文件夹，重命名为 -2 -3
zookeeper-3.7.0 修改zoo.cfg 配置文件
clientPort=2181zookeeper-3.7.0-2 修改zoo.cfg 配置文件
clientPort=2182zookeeper-3.7.0-3 修改zoo.cfg 配置文件
clientPort=2183 -
创建 ServerID
在配置的dataDir目录下面新建一个 myid 文件，文件内容就是对应的id号，
比如：
zookeeper-3.7.0 myid 文件的内容 为 1
zookeeper-3.7.0-2 myid 文件的内容 为 2
zookeeper-3.7.0-3 myid 文件的内容 为 3
-
启动 zookeeper
在对应的bin目录下启动 cmd命令，输入 zkServer (需要将zookeeper配置到系统环境变量)
二、 安装 Kafka
(1) 从官网下载，注意下载二进制文件，本次演示使用版本 kafka_2.12-2.8.0
(2) 配置文件
解压之后，在config文件夹下，打开 server.properties配置文件进行配置
# The id of the broker. This must be set to a unique integer for each broker. broker.id=0 port=9092 #配置副本数量 default.replication.factor=2 # The number of threads that the server uses for receiving requests from the network and sending responses to the network num.network.threads=3 # The number of threads that the server uses for processing requests, which may include disk I/O num.io.threads=8 # The send buffer (SO_SNDBUF) used by the socket server socket.send.buffer.bytes=102400 # The receive buffer (SO_RCVBUF) used by the socket server socket.receive.buffer.bytes=102400 # The maximum size of a request that the socket server will accept (protection against OOM) socket.request.max.bytes=104857600 ############################# Log Basics ############################# # A comma separated list of directories under which to store log files log.dirs=../kafka-logs # The default number of log partitions per topic. More partitions allow greater # parallelism for consumption, but this will also result in more files across # the brokers. num.partitions=1 # The number of threads per data directory to be used for log recovery at startup and flushing at shutdown. # This value is recommended to be increased for installations with data dirs located in RAID array. num.recovery.threads.per.data.dir=1 ############################# Internal Topic Settings ############################# # The replication factor for the group metadata internal topics "__consumer_offsets" and "__transaction_state" # For anything other than development testing, a value greater than 1 is recommended to ensure availability such as 3. offsets.topic.replication.factor=1 transaction.state.log.replication.factor=1 transaction.state.log.min.isr=1 log.retention.hours=168 log.segment.bytes=1073741824 log.retention.check.interval.ms=300000 # server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002". # You can also append an optional chroot string to the urls to specify the # root directory for all kafka znodes. zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183 # Timeout in ms for connecting to zookeeper zookeeper.connection.timeout.ms=18000 group.initial.rebalance.delay.ms=0(3) 集群配置
复制两份解压后的文件，命名如下
kafka_2.12-2.8.0-2 kafka_2.12-2.8.0-3
修改部分配置信息
在对应的server.properties中修改
#**唯一标识
broker.id=0
broker.id=1
broker.id=2
#**监听端口
port=9092
port=9093
port=9094
(4) 启动 Kafka
进入到bin/windows目录下 启动kafka并指定配置文件
kafka-server-start.bat ../../config/server.properties三、集群测试
（1）创建一个 topic
kafka-topics.bat --create --zookeeper 127.0.0.1:2181 --replication-factor 1 --partitions 1 --topic test（2）查看是否创建成功
kafka-topics.bat --list --zookeeper localhost:2181（3）发送消息
kafka-console-producer.bat --broker-list localhost:9092 --topic test helloworld（4）接收消息
分别在kafka_2.12-2.8.0inwindows 目录下执行
.kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic test .kafka-console-consumer.bat --bootstrap-server localhost:9093 --topic test .kafka-console-consumer.bat --bootstrap-server localhost:9094 --topic test -
