Redis的Cluster配置
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.安装Redis并启动
1>.修改Redis的配置文件（本实验只有三个节点）
[root@node101.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/redis.conf bind node101.yinzhengjie.org.cn protected-mode yes port 6379 tcp-backlog 511 unixsocket "/tmp/redis.sock" timeout 0 tcp-keepalive 300 daemonize no supervised systemd pidfile "/var/run/redis_6379.pid" loglevel notice logfile "/var/log/redis/redis.log" databases 16 save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename "dump.rdb" dir "/var/lib/redis" slave-serve-stale-data yes slave-read-only yes repl-diskless-sync no repl-diskless-sync-delay 5 repl-disable-tcp-nodelay no slave-priority 100 requirepass "yinzhengjie" appendonly no appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes lua-time-limit 5000 cluster-enabled yes #我们需要启用cluster功能 cluster-config-file redis-cluster.conf #指定cluster的配置文件 cluster-node-timeout 15000 #指定节点的超时时间 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-size -2 list-compress-depth 0 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 aof-rewrite-incremental-fsync yes [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/redis.conf bind node102.yinzhengjie.org.cn protected-mode yes port 6379 tcp-backlog 511 unixsocket "/tmp/redis.sock" timeout 0 tcp-keepalive 300 daemonize no supervised systemd pidfile "/var/run/redis_6379.pid" loglevel notice logfile "/var/log/redis/redis.log" databases 16 save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename "dump.rdb" dir "/var/lib/redis" slave-serve-stale-data yes slave-read-only yes repl-diskless-sync no repl-diskless-sync-delay 5 repl-disable-tcp-nodelay no slave-priority 100 requirepass "yinzhengjie" appendonly no appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes lua-time-limit 5000 cluster-enabled yes cluster-config-file redis-cluster.conf cluster-node-timeout 15000 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-size -2 list-compress-depth 0 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 aof-rewrite-incremental-fsync yes [root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# egrep -v "^#|^$" /etc/redis.conf bind node103.yinzhengjie.org.cn protected-mode yes port 6379 tcp-backlog 511 unixsocket "/tmp/redis.sock" timeout 0 tcp-keepalive 300 daemonize no supervised systemd pidfile "/var/run/redis_6379.pid" loglevel notice logfile "/var/log/redis/redis.log" databases 16 save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename "dump.rdb" dir "/var/lib/redis" slave-serve-stale-data yes slave-read-only yes repl-diskless-sync no repl-diskless-sync-delay 5 repl-disable-tcp-nodelay no slave-priority 100 requirepass "yinzhengjie" appendonly no appendfilename "appendonly.aof" appendfsync everysec no-appendfsync-on-rewrite no auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb aof-load-truncated yes lua-time-limit 5000 cluster-enabled yes cluster-config-file redis-cluster.conf cluster-node-timeout 15000 slowlog-log-slower-than 10000 slowlog-max-len 128 latency-monitor-threshold 0 notify-keyspace-events "" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-size -2 list-compress-depth 0 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 hll-sparse-max-bytes 3000 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 aof-rewrite-incremental-fsync yes [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]#
2>.启动Redis集群
[root@node101.yinzhengjie.org.cn ~]# systemctl start redis [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# systemctl status redis ● redis.service - Redis persistent key-value database Loaded: loaded (/usr/lib/systemd/system/redis.service; disabled; vendor preset: disabled) Drop-In: /etc/systemd/system/redis.service.d └─limit.conf Active: active (running) since Sun 2019-04-07 10:07:34 CST; 5min ago Main PID: 10134 (redis-server) CGroup: /system.slice/redis.service └─10134 /usr/bin/redis-server node101.yinzhengjie.org.cn:6379 [cluster] Apr 07 10:07:34 node101.yinzhengjie.org.cn systemd[1]: Starting Redis persistent key-value database... Apr 07 10:07:34 node101.yinzhengjie.org.cn systemd[1]: Started Redis persistent key-value database. [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# systemctl start redis [root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]# systemctl status redis ● redis.service - Redis persistent key-value database Loaded: loaded (/usr/lib/systemd/system/redis.service; disabled; vendor preset: disabled) Drop-In: /etc/systemd/system/redis.service.d └─limit.conf Active: active (running) since Sun 2019-04-07 10:12:35 CST; 3s ago Main PID: 10645 (redis-server) CGroup: /system.slice/redis.service └─10645 /usr/bin/redis-server node102.yinzhengjie.org.cn:6379 [cluster] Apr 07 10:12:35 node102.yinzhengjie.org.cn systemd[1]: Starting Redis persistent key-value database... Apr 07 10:12:35 node102.yinzhengjie.org.cn systemd[1]: Started Redis persistent key-value database. [root@node102.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# systemctl start redis [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# systemctl status redis ● redis.service - Redis persistent key-value database Loaded: loaded (/usr/lib/systemd/system/redis.service; disabled; vendor preset: disabled) Drop-In: /etc/systemd/system/redis.service.d └─limit.conf Active: active (running) since Sat 2019-04-06 19:12:24 PDT; 3s ago Main PID: 10562 (redis-server) CGroup: /system.slice/redis.service └─10562 /usr/bin/redis-server node103.yinzhengjie.org.cn:6379 [cluster] Apr 06 19:12:24 node103.yinzhengjie.org.cn systemd[1]: Starting Redis persistent key-value database... Apr 06 19:12:24 node103.yinzhengjie.org.cn systemd[1]: Started Redis persistent key-value database. [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]#
3>.登录Redis服务器测试
[root@node103.yinzhengjie.org.cn ~]# redis-cli -h node101.yinzhengjie.org.cn -a yinzhengjie node101.yinzhengjie.org.cn:6379> node101.yinzhengjie.org.cn:6379>
二.配置Redis Cluster
1>.查看cluster相关命令的帮助信息
[root@node103.yinzhengjie.org.cn ~]# redis-cli -h node101.yinzhengjie.org.cn -a yinzhengjie node101.yinzhengjie.org.cn:6379> node101.yinzhengjie.org.cn:6379> HELP @cluster CLUSTER ADDSLOTS slot [slot ...] summary: Assign new hash slots to receiving node since: 3.0.0 CLUSTER COUNT-FAILURE-REPORTS node-id summary: Return the number of failure reports active for a given node since: 3.0.0 CLUSTER COUNTKEYSINSLOT slot summary: Return the number of local keys in the specified hash slot since: 3.0.0 CLUSTER DELSLOTS slot [slot ...] summary: Set hash slots as unbound in receiving node since: 3.0.0 CLUSTER FAILOVER [FORCE|TAKEOVER] summary: Forces a slave to perform a manual failover of its master. since: 3.0.0 CLUSTER FORGET node-id summary: Remove a node from the nodes table since: 3.0.0 CLUSTER GETKEYSINSLOT slot count summary: Return local key names in the specified hash slot since: 3.0.0 CLUSTER INFO - summary: Provides info about Redis Cluster node state since: 3.0.0 CLUSTER KEYSLOT key summary: Returns the hash slot of the specified key since: 3.0.0 CLUSTER MEET ip port summary: Force a node cluster to handshake with another node since: 3.0.0 CLUSTER NODES - summary: Get Cluster config for the node since: 3.0.0 CLUSTER REPLICATE node-id summary: Reconfigure a node as a slave of the specified master node since: 3.0.0 CLUSTER RESET [HARD|SOFT] summary: Reset a Redis Cluster node since: 3.0.0 CLUSTER SAVECONFIG - summary: Forces the node to save cluster state on disk since: 3.0.0 CLUSTER SET-CONFIG-EPOCH config-epoch summary: Set the configuration epoch in a new node since: 3.0.0 CLUSTER SETSLOT slot IMPORTING|MIGRATING|STABLE|NODE [node-id] summary: Bind a hash slot to a specific node since: 3.0.0 CLUSTER SLAVES node-id summary: List slave nodes of the specified master node since: 3.0.0 CLUSTER SLOTS - summary: Get array of Cluster slot to node mappings since: 3.0.0 READONLY - summary: Enables read queries for a connection to a cluster slave node since: 3.0.0 READWRITE - summary: Disables read queries for a connection to a cluster slave node since: 3.0.0 node101.yinzhengjie.org.cn:6379>
2>.命令行分配切片范围
[root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# redis-cli -h node101.yinzhengjie.org.cn -a yinzhengjie -c cluster addslots {0..5500} OK [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# redis-cli -h node102.yinzhengjie.org.cn -a yinzhengjie -c cluster addslots {5501..11000} OK [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# redis-cli -h node103.yinzhengjie.org.cn -a yinzhengjie -c cluster addslots {11001..16383} OK [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]#
3>.互相暴露各个节点
[root@node103.yinzhengjie.org.cn ~]# redis-cli -h node103.yinzhengjie.org.cn -a yinzhengjie node103.yinzhengjie.org.cn:6379> node103.yinzhengjie.org.cn:6379> CLUSTER INFO cluster_state:fail cluster_slots_assigned:10884 cluster_slots_ok:10884 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:2 cluster_size:2 cluster_current_epoch:1 cluster_my_epoch:0 cluster_stats_messages_sent:10 cluster_stats_messages_received:10 node103.yinzhengjie.org.cn:6379>
node103.yinzhengjie.org.cn:6379> CLUSTER MEET 172.30.1.101 6379 OK node103.yinzhengjie.org.cn:6379> node103.yinzhengjie.org.cn:6379> CLUSTER MEET 172.30.1.102 6379 OK node103.yinzhengjie.org.cn:6379>
node103.yinzhengjie.org.cn:6379> CLUSTER INFO cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:3 cluster_size:3 cluster_current_epoch:2 cluster_my_epoch:2 cluster_stats_messages_sent:31 cluster_stats_messages_received:31 node103.yinzhengjie.org.cn:6379>
[root@node102.yinzhengjie.org.cn ~]# redis-cli -h node102.yinzhengjie.org.cn -a yinzhengjie node102.yinzhengjie.org.cn:6379> node102.yinzhengjie.org.cn:6379> node102.yinzhengjie.org.cn:6379> node102.yinzhengjie.org.cn:6379> CLUSTER INFO cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:3 cluster_size:3 cluster_current_epoch:2 cluster_my_epoch:0 cluster_stats_messages_sent:238 cluster_stats_messages_received:238 node102.yinzhengjie.org.cn:6379> node102.yinzhengjie.org.cn:6379>
[root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# redis-cli -h node101.yinzhengjie.org.cn -a yinzhengjie node101.yinzhengjie.org.cn:6379> node101.yinzhengjie.org.cn:6379> CLUSTER INFO cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:3 cluster_size:3 cluster_current_epoch:2 cluster_my_epoch:1 cluster_stats_messages_sent:350 cluster_stats_messages_received:350 node101.yinzhengjie.org.cn:6379> node101.yinzhengjie.org.cn:6379>
4>.验证Redis的写操作
[root@node101.yinzhengjie.org.cn ~]# redis-cli -h node101.yinzhengjie.org.cn -a yinzhengjie node101.yinzhengjie.org.cn:6379> node101.yinzhengjie.org.cn:6379> node101.yinzhengjie.org.cn:6379> SET name Jason #注意，我们在node101.yinzhengjie.org.cn创建数据时失败啦！返回的是错误，说是name这个字段被hash到5798这个分片上，而该分片在172.30.1.102:6379这个节点上，因此我们应该去该节点配置！ (error) MOVED 5798 172.30.1.102:6379 node101.yinzhengjie.org.cn:6379> node101.yinzhengjie.org.cn:6379> node101.yinzhengjie.org.cn:6379> node101.yinzhengjie.org.cn:6379> quit [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# redis-cli -h node102.yinzhengjie.org.cn -a yinzhengjie #于是，按照其错误的提示，我们登录到172.30.1.102:6379这个节点上去设置KEY node102.yinzhengjie.org.cn:6379> node102.yinzhengjie.org.cn:6379> SET name Jason #果不其然，在该节点就可以成功创建名称为name的KEY OK node102.yinzhengjie.org.cn:6379> node102.yinzhengjie.org.cn:6379> KEYS * 1) "name" node102.yinzhengjie.org.cn:6379> node102.yinzhengjie.org.cn:6379>
5>.验证Redis的读操作
[root@node103.yinzhengjie.org.cn ~]# redis-cli -h node103.yinzhengjie.org.cn -a yinzhengjie node103.yinzhengjie.org.cn:6379> node103.yinzhengjie.org.cn:6379> KEYS * (empty list or set) node103.yinzhengjie.org.cn:6379> node103.yinzhengjie.org.cn:6379> GET name #当我们想要读取name字段对应的value时，发现当前节点可以帮我们路由到真正存储的该节点上去 (error) MOVED 5798 172.30.1.102:6379 node103.yinzhengjie.org.cn:6379> node103.yinzhengjie.org.cn:6379> quit [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# redis-cli -h node102.yinzhengjie.org.cn -a yinzhengjie node102.yinzhengjie.org.cn:6379> node102.yinzhengjie.org.cn:6379> KEYS * 1) "name" node102.yinzhengjie.org.cn:6379> node102.yinzhengjie.org.cn:6379> GET name #按照上面的报错提示登录到指定的节点后，发现是可以读取数据的！ "Jason" node102.yinzhengjie.org.cn:6379> node102.yinzhengjie.org.cn:6379>
经测试，读写操作虽然能实现，但是对于程序员来说，手动去解决这些问题简直太麻烦了，推荐大家使用Cerberus（芒果TV）的解决方案，可以帮我们轻松解决上面遇到的读写问题！