etcd多节点集群安装(未开启数据通信加密)

目录

• 背景说明
• 环境信息
  • 操作系统版本
  • etcd版本
  • 主机信息
• 集群安装
  • 软件下载、解压（3个节点都需要操作）
  • 将etcd etcdctl二进制文件拷贝到/usr/local/bin/目录下（3个节点都需要操作)
  • 创建systemd使用的etcd service文件
  • 创建etcd数据目录（3个节点都需要操作)
  • 启动etcd服务（3个节点都需要操作)
  • 查看端口状态
  • 查看集群状态
• 配置参数说明
  • --name 成员名称
  • --data-dir 数据目录
  • --listen-peer-urls 监听其他成员请求的地址
  • --listen-client-urls 监听客户端请求的地址
  • -advertise-client-urls 告知集群其他成员，本节点监听客户端请求的地址
  • --initial-advertise-peer-urls 告知集群其他成员，本节点监听其他成员请求的地址
  • --initial-cluster-token 集群名称
  • --initial-cluster 初始化时集群配置
  • --initial-cluster-state 初始化集群的状态
• 常见错误
  • -bash: ./etcdctl: cannot execute binary file

背景说明

通过3个主机节点创建etcd集群，此集群未开启数据通信加密。

环境信息

操作系统版本

CentOS Linux release 7.2.1511 (Core)

etcd版本

3.4.3

主机信息

主机名	IP	成员名称
lab1	192.168.51.111	etcd1
lab2	192.168.51.112	etcd2
lab3	192.168.51.113	etcd3

集群安装

软件下载、解压（3个节点都需要操作）

etcd-v3.4.3-linux-amd64.tar.gz

wget https://github.com/etcd-io/etcd/releases/download/v3.4.3/etcd-v3.4.3-linux-amd64.tar.gz -O - | tar zxvf -
cd etcd-v3.4.3-linux-amd64

将etcd etcdctl二进制文件拷贝到/usr/local/bin/目录下（3个节点都需要操作)

cp etcd etcdctl /usr/local/bin/

创建systemd使用的etcd service文件

节点lab1

cat >/usr/lib/systemd/system/etcd.service <<EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/etcd-io/etcd

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/local/bin/etcd 
  --name=etcd1 
  --data-dir=/var/lib/etcd 
  --initial-advertise-peer-urls=http://192.168.51.111:2380 
  --listen-peer-urls=http://192.168.51.111:2380 
  --listen-client-urls=http://192.168.51.111:2379,http://127.0.0.1:2379 
  --advertise-client-urls=http://192.168.51.111:2379 
  --initial-cluster-token=etcd-cluster-1 
  --initial-cluster etcd1=http://192.168.51.111:2380,etcd2=http://192.168.51.112:2380,etcd3=http://192.168.51.113:2380 
  --initial-cluster-state new
Restart=always
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

节点lab2

cat >/usr/lib/systemd/system/etcd.service <<EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/etcd-io/etcd

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/local/bin/etcd 
  --name=etcd2 
  --data-dir=/var/lib/etcd 
  --initial-advertise-peer-urls=http://192.168.51.112:2380 
  --listen-peer-urls=http://192.168.51.112:2380 
  --listen-client-urls=http://192.168.51.112:2379,http://127.0.0.1:2379 
  --advertise-client-urls=http://192.168.51.112:2379 
  --initial-cluster-token=etcd-cluster-1 
  --initial-cluster etcd1=http://192.168.51.111:2380,etcd2=http://192.168.51.112:2380,etcd3=http://192.168.51.113:2380 
  --initial-cluster-state new
Restart=always
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

节点lab3

cat >/usr/lib/systemd/system/etcd.service <<EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/etcd-io/etcd

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/local/bin/etcd 
  --name=etcd3 
  --data-dir=/var/lib/etcd 
  --initial-advertise-peer-urls=http://192.168.51.113:2380 
  --listen-peer-urls=http://192.168.51.113:2380 
  --listen-client-urls=http://192.168.51.113:2379,http://127.0.0.1:2379 
  --advertise-client-urls=http://192.168.51.113:2379 
  --initial-cluster-token=etcd-cluster-1 
  --initial-cluster etcd1=http://192.168.51.111:2380,etcd2=http://192.168.51.112:2380,etcd3=http://192.168.51.113:2380 
  --initial-cluster-state new
Restart=always
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

创建etcd数据目录（3个节点都需要操作)

mkdir -p /var/lib/etcd

启动etcd服务（3个节点都需要操作)

systemctl enable etcd
systemctl start etcd
systemctl status etcd

查看端口状态

[root@lab1 ~]# ss -antlp | grep -E "(2379|2380)"
LISTEN     0      128    192.168.51.111:2379                     *:*                   users:(("etcd",pid=2586,fd=7))
LISTEN     0      128    127.0.0.1:2379                     *:*                   users:(("etcd",pid=2586,fd=6))
LISTEN     0      128    192.168.51.111:2380                     *:*                   users:(("etcd",pid=2586,fd=5))

注意：2379.2380监听端口已经开启。

查看集群状态

etcdctl endpoint --cluster status -w table
etcdctl endpoint --cluster health -w table

查询结果

[root@lab1 ~]# etcdctl endpoint --cluster status -w table
+----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|          ENDPOINT          |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| http://192.168.51.113:2379 | 4ca03c8318c7938e |   3.4.3 |   20 kB |     false |      false |        89 |          9 |                  9 |        |
| http://192.168.51.111:2379 | 93a8384fbbae4029 |   3.4.3 |   20 kB |      true |      false |        89 |          9 |                  9 |        |
| http://192.168.51.112:2379 | dd30d365d4cd184a |   3.4.3 |   20 kB |     false |      false |        89 |          9 |                  9 |        |
+----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

[root@lab1 ~]# etcdctl endpoint --cluster health -w table
+----------------------------+--------+------------+-------+
|          ENDPOINT          | HEALTH |    TOOK    | ERROR |
+----------------------------+--------+------------+-------+
| http://192.168.51.111:2379 |   true | 6.096532ms |       |
| http://192.168.51.112:2379 |   true | 6.762484ms |       |
| http://192.168.51.113:2379 |   true | 6.849903ms |       |
+----------------------------+--------+------------+-------+

注意：如上查询结果显示，集群由3个节点组成，主节点是93a8384fbbae4029(2.168.51.111:2379),各个节点是健康状态。etcd多节点集群安装完成。

配置参数说明

--name 成员名称

• Human-readable name for this member.
• default: "default"
• env variable: ETCD_NAME
• This value is referenced as this node's own entries listed in the --initial-cluster flag (e.g., default=http://localhost:2380). This needs to match the key used in the flag if using [static bootstrapping][build-cluster]. When using discovery, each member must have a unique name. Hostname or machine-id can be a good choice.

--data-dir 数据目录

• Path to the data directory.
• default: "${name}.etcd"
• env variable: ETCD_DATA_DIR

--listen-peer-urls 监听其他成员请求的地址

• List of URLs to listen on for peer traffic. This flag tells the etcd to accept incoming requests from its peers on the specified scheme://IP:port combinations. Scheme can be http or https. Alternatively, use unix://<file-path> or unixs://<file-path> for unix sockets. If 0.0.0.0 is specified as the IP, etcd listens to the given port on all interfaces. If an IP address is given as well as a port, etcd will listen on the given port and interface. Multiple URLs may be used to specify a number of addresses and ports to listen on. The etcd will respond to requests from any of the listed addresses and ports.
• default: "http://localhost:2380"
• env variable: ETCD_LISTEN_PEER_URLS
• example: "http://10.0.0.1:2380"
• invalid example: "http://example.com:2380" (domain name is invalid for binding)
  注意：该配置指定通过哪个地址来接收其他成员的请求

--listen-client-urls 监听客户端请求的地址

• List of URLs to listen on for client traffic. This flag tells the etcd to accept incoming requests from the clients on the specified scheme://IP:port combinations. Scheme can be either http or https. Alternatively, use unix://<file-path> or unixs://<file-path> for unix sockets. If 0.0.0.0 is specified as the IP, etcd listens to the given port on all interfaces. If an IP address is given as well as a port, etcd will listen on the given port and interface. Multiple URLs may be used to specify a number of addresses and ports to listen on. The etcd will respond to requests from any of the listed addresses and ports.
• default: "http://localhost:2379"
• env variable: ETCD_LISTEN_CLIENT_URLS
• example: "http://10.0.0.1:2379"
• invalid example: "http://example.com:2379" (domain name is invalid for binding)
  注意：该配置指定通过哪个地址接收客户端的请求。

-advertise-client-urls 告知集群其他成员，本节点监听客户端请求的地址

• List of this member's client URLs to advertise to the rest of the cluster. These URLs can contain domain names.
• default: "http://localhost:2379"
• env variable: ETCD_ADVERTISE_CLIENT_URLS
• example: "http://example.com:2379, http://10.0.0.1:2379"
• Be careful if advertising URLs such as http://localhost:2379 from a cluster member and are using the proxy feature of etcd. This will cause loops, because the proxy will be forwarding requests to itself until its resources (memory, file descriptors) are eventually depleted.

--initial-advertise-peer-urls 告知集群其他成员，本节点监听其他成员请求的地址

• List of this member's peer URLs to advertise to the rest of the cluster. These addresses are used for communicating etcd data around the cluster. At least one must be routable to all cluster members. These URLs can contain domain names.
• default: "http://localhost:2380"
• env variable: ETCD_INITIAL_ADVERTISE_PEER_URLS
• example: "http://example.com:2380, http://10.0.0.1:2380"

--initial-cluster-token 集群名称

• Initial cluster token for the etcd cluster during bootstrap.
• default: "etcd-cluster"
• env variable: ETCD_INITIAL_CLUSTER_TOKEN

--initial-cluster 初始化时集群配置

• Initial cluster configuration for bootstrapping.
• default: "default=http://localhost:2380"
• env variable: ETCD_INITIAL_CLUSTER
• The key is the value of the --name flag for each node provided. The default uses default for the key because this is the default for the --name flag.

--initial-cluster-state 初始化集群的状态

• Initial cluster state ("new" or "existing"). Set to new for all members present during initial static or DNS bootstrapping. If this option is set to existing, etcd will attempt to join the existing cluster. If the wrong value is set, etcd will attempt to start but fail safely.
• default: "new"
• env variable: ETCD_INITIAL_CLUSTER_STATE

常见错误

-bash: ./etcdctl: cannot execute binary file

执行etcd或者etcdctl命令报错

[root@lab1 etcd-v3.4.3-linux-arm64]# ./etcdctl 
-bash: ./etcdctl: cannot execute binary file

解决：查看etcd编译环境是否与当前环境一致，比如，linux-amd运行linux-arm的etcd包会报上述错误。

etcd多节点集群安装(未开启数据通信加密)