多节点集群化部署

针对不同的情况，比如当节点数和各节点上的地址已知使用静态配置，但是当各个节点的地址无法知晓的情况下，使用服务发现进行配置。

一、静态配置

三台机器信息：

name	ip
etcd1	192.168.159.128
etcd2	192.168.159.129
etcd3	192.168.159.130

进行静态配置需要满足以下条件：

• 集群节点个数已知
• 集群各节点上地址已知

 etcd1配置文件conf.yml如下：

name: etcd1
data-dir: /project/etcd_cluster/etcd1/data
listen-client-urls: http://0.0.0.0:2379
advertise-client-urls: http://192.168.159.128:2379
listen-peer-urls: http://0.0.0.0:2380
initial-advertise-peer-urls: http://192.168.159.128:2380
initial-cluster: etcd1=http://192.168.159.128:2380,etcd2=http://192.168.159.129:2380,etcd3=http://192.168.159.130:2380
initial-cluster-token: etcd-cluster-1
initial-cluster-state: new

 etcd2配置文件conf.yml如下：

name: etcd2
data-dir: /project/etcd_cluster/etcd2/data
listen-client-urls: http://0.0.0.0:2379
advertise-client-urls: http://192.168.159.129:2379
listen-peer-urls: http://0.0.0.0:2380
initial-advertise-peer-urls: http://192.168.159.129:2380
initial-cluster: etcd1=http://192.168.159.128:2380,etcd2=http://192.168.159.129:2380,etcd3=http://192.168.159.130:2380
initial-cluster-token: etcd-cluster-1
initial-cluster-state: new

etcd3配置文件conf.yml如下：

name: etcd3
data-dir: /project/etcd_cluster/etcd3/data
listen-client-urls: http://0.0.0.0:2379
advertise-client-urls: http://192.168.159.130:2379
listen-peer-urls: http://0.0.0.0:2380
initial-advertise-peer-urls: http://192.168.159.130:2380
initial-cluster: etcd1=http://192.168.159.128:2380,etcd2=http://192.168.159.129:2380,etcd3=http://192.168.159.130:2380
initial-cluster-token: etcd-cluster-1
initial-cluster-state: new

注意，上面配置文件中listen-client-urls和listen-peer-urls参数监听的的是所有的地址。

接下来通过etcd（此时etcd压缩包已经拷贝至另外两台机器）来启动各个节点：

• 启动etcd1

# 192.168.159.128
[root@localhost etcd-v3.3.10-linux-amd64]# ./etcd --config-file=/project/etcd_cluster/etcd1/conf.yml

• 启动etcd2

# 192.168.159.129
[root@localhost etcd-v3.3.10-linux-amd64]# ./etcd --config-file=/project/etcd_cluster/etcd2/conf.yml

• 启动etcd3

# 192.168.159.130
[root@localhost etcd-v3.3.10-linux-amd64]# ./etcd --config-file=/project/etcd_cluster/etcd3/conf.yml

启动完毕后对集群的状态进行查看：

• 查看成员

[root@localhost etcd-v3.3.10-linux-amd64]# ./etcdctl member list
4c14bc06668e9505: name=etcd3 peerURLs=http://192.168.159.130:2380 clientURLs=http://192.168.159.130:2379 isLeader=false
57bf4d2527966724: name=etcd2 peerURLs=http://192.168.159.129:2380 clientURLs=http://192.168.159.129:2379 isLeader=false
a11e107c0081dbf8: name=etcd1 peerURLs=http://192.168.159.128:2380 clientURLs=http://192.168.159.128:2379 isLeader=true

• 查看集群健康状态

[root@localhost etcd-v3.3.10-linux-amd64]# ./etcdctl cluster-health
member 4c14bc06668e9505 is healthy: got healthy result from http://192.168.159.130:2379
member 57bf4d2527966724 is healthy: got healthy result from http://192.168.159.129:2379
member a11e107c0081dbf8 is healthy: got healthy result from http://192.168.159.128:2379
cluster is healthy

测试：

• 读写测试

[root@localhost etcd-v3.3.10-linux-amd64]# ETCDCTL_API=3 ./etcdctl  put foo bar
OK
[root@localhost etcd-v3.3.10-linux-amd64]# ETCDCTL_API=3 ./etcdctl  get foo
foo
bar

二、服务发现

　　实际应用中，可能无法知道各个节点ip地址的情况，所以就不能使用上述的静态配置，而是需要使用服务发现，实质上就是利用现有的etcd集群来启动一个新的etcd集群。我们可以使用之前搭建好的集群来搭建。

1、创建发现url（集群标识）

使用现有的集群创建发现url：

[root@localhost etcd-v3.3.10-linux-amd64]# curl -X PUT http://192.168.159.128:2379/v2/keys/discovery/6c007a14875d53d9bf0ef5a6fc0257c817f0fb83/_config/size -d value=3
{"action":"set","node":{"key":"/discovery/6c007a14875d53d9bf0ef5a6fc0257c817f0fb83/_config/size","value":"3","modifiedIndex":9,"createdIndex":9}}

当然，如果没有自己的已有集群，也可以通过etcd提供的公共服务获取发现url：

# size代表要创建的集群大小
[root@localhost etcd-v3.3.10-linux-amd64]# curl -w "
" 'https://discovery.etcd.io/new?size=3'
https://discovery.etcd.io/8c66ef7d7bdccfa2e83f277b2407bd44

2、启动新的集群

　　服务发现的url是http://192.168.159.128:2379/v2/keys/discovery/6c007a14875d53d9bf0ef5a6fc0257c817f0fb83，需要将其作为--discovery参数来启动etcd，新的etcd实例会自动使用服务发现url的目录来启动注册。三个节点的配置文件如下：

• infra0

name: infra0
listen-client-urls: http://0.0.0.0:12379
advertise-client-urls: http://192.168.159.128:12379
listen-peer-urls: http://0.0.0.0:12380
initial-advertise-peer-urls: http://192.168.159.128:12380
discovery: http://192.168.159.128:2379/v2/keys/discovery/6c007a14875d53d9bf0ef5a6fc0257c817f0fb83

• infra1

name: infra1
listen-client-urls: http://0.0.0.0:12379
advertise-client-urls: http://192.168.159.129:12379
listen-peer-urls: http://0.0.0.0:12380
initial-advertise-peer-urls: http://192.168.159.129:12380
discovery: http://192.168.159.128:2379/v2/keys/discovery/6c007a14875d53d9bf0ef5a6fc0257c817f0fb83

• infra2

name: infra2
listen-client-urls: http://0.0.0.0:12379
advertise-client-urls: http://192.168.159.130:12379
listen-peer-urls: http://0.0.0.0:12380
initial-advertise-peer-urls: http://192.168.159.130:12380
discovery: http://192.168.159.128:2379/v2/keys/discovery/6c007a14875d53d9bf0ef5a6fc0257c817f0fb83

　　可以看到这与之前静态配置的区别是不需要initial-cluster、initial-cluster-token、initial-cluster-state三个参数，但是需要discovery参数用来获取集群标识。上面的配置没有加上data-dir，这会在当前目录下创建一个数据目录infra0.etcd（或者infra1.etcd、infra2.etcd）。

　　然后，可以进行启动集群，与静态配置方式启动一样，不过配置文件不同。

# infra0
[root@localhost etcd-v3.3.10-linux-amd64]# ./etcd --config-file=/project/etcd_cluster/infra0/conf.yml

# infra1
[root@localhost etcd-v3.3.10-linux-amd64]# ./etcd --config-file=/project/etcd_cluster/infra1/conf.yml


# infra2
[root@localhost etcd-v3.3.10-linux-amd64]# ./etcd --config-file=/project/etcd_cluster/infra2/conf.yml

获取集群信息：

[root@localhost etcd-v3.3.10-linux-amd64]# ./etcdctl -endpoints=http://192.168.159.128:12379 member list
4b44e845d0f3585d: name=infra0 peerURLs=http://192.168.159.128:12380 clientURLs=http://192.168.159.128:12379 isLeader=true
6b9a9ac81c9c5d07: name=infra2 peerURLs=http://192.168.159.130:12380 clientURLs=http://192.168.159.130:12379 isLeader=false
957577be1117b1b1: name=infra1 peerURLs=http://192.168.159.129:12380 clientURLs=http://192.168.159.129:12379 isLeader=false