Zookeeper深入理解（一）搭建与测试

Zookeeper 分布式服务框架是 Apache Hadoop 的一个子项目，它主要是用来解决分布式应用中经常遇到的一些数据管理问题，如：统一命名服务、状态同步服务、集群管理、分布式应用配置项的管理等。现在openstack的项目已经有开始使用zookeeper来实现分布式开发，例如Ceilometer的Central Agent。在SDN控制器开发领域，对于控制器的高可用和负载均衡的需求也需要引用zookeeper工具。因此作为openstack架构师和开发人员，有必要深入研究zookeeper的原理和开发技巧。由于zookeeper是基于java开发项目，作为openstack的python开发语言，现在已经有一些成熟的接口api和项目，例如kazoo、tooz等。

首先我们要搭建一套zookeeper的学习研究环境

1.获取zookeeper包

wget http://mirror.nus.edu.sg/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz

2.创建三节点环境

server1

├── data

├── dataLog

├── logs

├── zookeeper-3.4.6

└── zookeeper-3.4.6.tar.gz

server2

├── data

├── dataLog

├── logs

├── zookeeper-3.4.6

└── zookeeper-3.4.6.tar.gz

server3

├── data

├── dataLog

├── logs

├── zookeeper-3.4.6

└── zookeeper-3.4.6.tar.gz

Server主配置zoo.cfg

# 心跳间隔时间(ms)

tickTime=2000

# Follower与Leader初始化时能容忍的时间(initLimit * tickTime)

initLimit=10

# Follower与Leader请求响应时能容忍的时间(syncLimit * tickTime)

syncLimit=5

# server1数据目录，每个server各自配置

dataDir=/path/to/server1/data

# 客户端连接端口

clientPort=2182

# server列表:

#

# .n(标记server)=(server地址):(投票时所用端口):(选举Leader时所用端口)

server.1=127.0.0.1:2889:3889

server.2=127.0.0.1:2990:3990

server.3=127.0.0.1:2991:3991

在每个Server/data目录下建立一个myid文件，标识自己是哪个Server(对应zoo.cfg的server列表)

现在启动server1

server1/zookeeper-3.4.6/bin/zkServer.sh start-foreground

#

# 输出

2015-03-02 19:56:47,485 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2182:QuorumPeer@714] - LOOKING

2015-03-02 19:56:47,486 [myid:1] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2182:FastLeaderElection@815] - New election. My id =  1, proposed zxid=0x1a00000006

2015-03-02 19:56:47,488 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 1 (n.leader), 0x1a00000006 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x1a (n.peerEpoch) LOOKING (my state)

2015-03-02 19:56:47,510 [myid:1] - WARN  [WorkerSender[myid=1]:QuorumCnxManager@382] - Cannot open channel to 2 at election address /127.0.0.1:3990

java.net.ConnectException: Connection refused

at java.net.PlainSocketImpl.socketConnect(Native Method)

at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)

at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)

at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)

at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

at java.net.Socket.connect(Socket.java:589)

at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)

at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:341)

at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:449)

at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:430)

at java.lang.Thread.run(Thread.java:744)

2015-03-02 19:56:47,560 [myid:1] - WARN  [WorkerSender[myid=1]:QuorumCnxManager@382] - Cannot open channel to 3 at election address /127.0.0.1:3991

server1尝试连接zoo.cfg中配置的server列表，这时server2，server3还没启动，因此连接失败， 将server2，server3一并启动后，可以看到server1已经成为Follower

2015-03-02 20:02:41,487 [myid:1] - INFO  [/127.0.0.1:3889:QuorumCnxManager$Listener@511] - Received connection request /127.0.0.1:65469

2015-03-02 20:02:41,492 [myid:1] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 3 (n.leader), 0x1a00000006 (n.zxid)0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x1a (n.peerEpoch) FOLLOWING (my state)

于是，可以使用客户端尝试连接这个集群，并且客户端会被随机分配到一个server

zookeeper-3.4.6/bin/zkCli.sh -server 127.0.0.1:2182,127.0.0.1:2183,127.0.0.1:2184

3.简单测试

列出节点信息：

[zk: 127.0.0.1:2182,127.0.0.1:2183,127.0.0.1:2184(CONNECTED) 11] ls /

[dubbo, zookeeper, pampas, pubsub]

创建/workers节点：

[zk: 127.0.0.1:2182,127.0.0.1:2183,127.0.0.1:2184(CONNECTED) 11] create /workers ""

[zk: 127.0.0.1:2182,127.0.0.1:2183,127.0.0.1:2184(CONNECTED) 11] ls /

[dubbo, zookeeper, workers, pampas, pubsub]

删除/workers节点：

[zk: 127.0.0.1:2182,127.0.0.1:2183,127.0.0.1:2184(CONNECTED) 11] delete /workers

[zk: localhost:2181(CONNECTED) 4] ls /

[zk: 127.0.0.1:2182,127.0.0.1:2183,127.0.0.1:2184(CONNECTED) 11]

[zk: localhost:2181(CONNECTED) 5] quit

通过以上步骤，我们就成功搭建出一套三节点zookeeper伪集群环境，为后面原理验证和python接口开发打好基础。