zoukankan      html  css  js  c++  java
  • 搭建基于docker的redis集群

    最近看了点redis集群方面的知识点,想自己搭着试试,想了解更多可以阅读redis官方文档

    因为基于docker来操作快速方便。下面都是基于docker-compose来实现redis集群搭建,有关docker知识不太熟悉的,建议先学习下docker以及docker-compose知识。

    这里实现的是三主三从集群。

    #最终搭建成功后的目录结构如下
    [root@new2 docker-redis-cluster]# tree .
    .
    ├── docker-compose.yml
    ├── master1
    │   ├── redis.conf
    │   └── redis.log
    ├── master2
    │   ├── redis.conf
    │   └── redis.log
    ├── master3
    │   ├── redis.conf
    │   └── redis.log
    ├── slave1
    │   ├── redis.conf
    │   └── redis.log
    ├── slave2
    │   ├── redis.conf
    │   └── redis.log
    └── slave3
        ├── redis.conf
        └── redis.log

    .yml文件使用yaml语言编写,不了解的可以熟知下YAML语法知识。

    docker-compose.yml文件内容如下:

    version: '3'
    
    services:
      master-1:
        container_name: master-1
        image: redis
        command: redis-server /etc/usr/local/redis.conf
        network_mode: "host"
        tty: true #后台运行不退出
    privileged: true #拥有容器内部命令执行的权限 volumes: - /opt/www/docker-redis-cluster/master1/redis.conf:/etc/usr/local/redis.conf - /opt/www/docker-redis-cluster/master1/redis.log:/usr/local/redis/logs/redis.log master-2: container_name: master-2 command: redis-server /etc/usr/local/redis.conf image: redis network_mode: "host" tty: true #后台运行不退出
    privileged: true #拥有容器内部命令执行的权限 volumes: - /opt/www/docker-redis-cluster/master2/redis.conf:/etc/usr/local/redis.conf - /opt/www/docker-redis-cluster/master2/redis.log:/usr/local/redis/logs/redis.log master-3: container_name: master-3 image: redis command: redis-server /etc/usr/local/redis.conf network_mode: "host" tty: true
    privileged: true
    volumes: - /opt/www/docker-redis-cluster/master3/redis.conf:/etc/usr/local/redis.conf - /opt/www/docker-redis-cluster/master3/redis.log:/usr/local/redis/logs/redis.log slave-1: container_name: slave-1 image: redis command: redis-server /etc/usr/local/redis.conf network_mode: "host" tty: true
    privileged: true
    volumes: - /opt/www/docker-redis-cluster/slave1/redis.conf:/etc/usr/local/redis.conf - /opt/www/docker-redis-cluster/slave1/redis.log:/usr/local/redis/logs/redis.log slave-2: container_name: slave-2 image: redis command: redis-server /etc/usr/local/redis.conf network_mode: "host" tty: true
    privileged: true
    volumes: - /opt/www/docker-redis-cluster/slave2/redis.conf:/etc/usr/local/redis.conf - /opt/www/docker-redis-cluster/slave2/redis.log:/usr/local/redis/logs/redis.log slave-3: container_name: slave-3 image: redis command: redis-server /etc/usr/local/redis.conf network_mode: "host" tty: true
    privileged: true
    volumes: - /opt/www/docker-redis-cluster/slave3/redis.conf:/etc/usr/local/redis.conf - /opt/www/docker-redis-cluster/slave3/redis.log:/usr/local/redis/logs/redis.log

    redis.conf文件内容如下(集群里三主端口6379/6380/6381,三从端口6382/6383/6384,这里仅以master1的redis.conf举例,其他五个port注意修改):

    port 6379
    bind 0.0.0.0
    cluster-enabled yes
    logfile "/usr/local/redis/logs/redis.log"

    文件中的 cluster-enabled 选项用于开实例的集群模式, 而 cluster-conf-file 选项则设定了保存节点配置文件的路径, 默认值为 nodes.conf.节点配置文件无须人为修改, 它由 Redis 集群在启动时创建, 并在有需要时自动进行更新。

    要让集群正常运作至少需要三个主节点,不过在刚开始试用集群功能时, 强烈建议使用六个节点: 其中三个为主节点, 而其余三个则是各个主节点的从节点。

    注意在各个masterxxxx目录下创建redis.log文件。

      

    准备完毕,运行:

    [root@new2 docker-redis-cluster]# docker-compose up -d
    Creating slave-2  ... done
    Creating slave-3  ... done
    Creating master-3 ... done
    Creating slave-1  ... done
    Creating master-2 ... done
    Creating master-1 ... done
    #使用ps命令查看可见未有运行的docker容器
    [root@new2 docker-redis-cluster]# docker ps
    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS          
    #使用docker logs master-1查看主服务器master1日志,可见到redis.log无运行权限,从网上查找相关资料,大概说是运行docker的用户和docker容器内部运行redis-server命令的不是同一用户。
    #临时给当前docker-redis-cluster目录下所有目录和文件赋予777权限得到解决,当然这不是好的方案,后面再研究下 [root@new2 docker-redis-cluster]# docker logs master-1 *** FATAL CONFIG FILE ERROR (Redis 6.0.7) *** Reading the configuration file, at line 4 >>> 'logfile "/usr/local/redis/logs/redis.log"' Can't open the log file: Permission denied

    集群原理:

               Redis-cluster把所有的物理节点映射到[0-16383]slot上,cluster 负责维护node<->slot<->value。

               Redis集群中内置了 16384 个哈希槽,当需要在 Redis 集群中放置一个 key-value 时,redis 先对key 使用 crc16 算法算出一个结果,然后把结果对 16384 求余数。

               这样每个 key 都会对应一个编号在 0-16383 之间的哈希槽,redis 会根据节点数量大致均等的将哈希槽映射到不同的节点。

               如果增加新节点D,需要从A、B、C节点上删除一些hash slot给到D。同样地,如果从集群中删除节点A,A上面的hash slots移动到B和C,当节点A上是空的时候就可以将其从集群中完全删除。

               因为将hash slots从一个节点移动到另一个节点并不需要停止其它的操作,添加、删除节点以及更改节点所维护的hash slots的百分比都不需要任何停机时间。移动hash slots是并行的,移动hash slots不会影响其它操作。

               每个Redis集群中的节点都需要打开两个TCP连接。一个连接用于正常的给Client提供服务,比如6379,还有一个额外的端口(通过在这个端口号上加10000)作为数据端口,比如16379。

               第二个端口(本例中就是16379)用于集群总线,这是一个用二进制协议的点对点通信信道。这个集群总线(Cluster bus)用于节点的失败侦测、配置更新、故障转移授权,等等。

               客户端从来都不应该尝试和这些集群总线端口通信,它们只应该和正常的Redis命令端口进行通信。注意,确保在你的防火墙中开放着两个端口,否则,Redis集群节点之间将无法通信。

               命令端口和集群总线端口的偏移量总是10000。

    docker的redis容器已创建好,现在创建集群模式让他们相互通信(因为没有具体指定cluster的各机器主从关系,默认随机分配主从)。

    [root@new2 docker-redis-cluster]# docker exec -it master-1 redis-cli --cluster create 127.0.0.1:6
    > 127.0.0.1:6380 
    > 127.0.0.1:6381 
    > 127.0.0.1:6382 
    > 127.0.0.1:6383 
    > 127.0.0.1:6384 --cluster-replicas 1
    >>> Performing hash slots allocation on 6 nodes...
    Master[0] -> Slots 0 - 5460
    Master[1] -> Slots 5461 - 10922
    Master[2] -> Slots 10923 - 16383
    Adding replica 127.0.0.1:6383 to 127.0.0.1:6379
    Adding replica 127.0.0.1:6384 to 127.0.0.1:6380
    Adding replica 127.0.0.1:6382 to 127.0.0.1:6381
    >>> Trying to optimize slaves allocation for anti-affinity
    [WARNING] Some slaves are in the same host as their master
    M: 0e4bc44048dc19b1c7f41178e22cc87ccdab7b65 127.0.0.1:6379
       slots:[0-5460] (5461 slots) master
    M: fd3e386cc455c45d4d9aed656dc8bcebfb147122 127.0.0.1:6380
       slots:[5461-10922] (5462 slots) master
    M: 33bebb7958fc5bf48038bd9f097bee2ea77274c9 127.0.0.1:6381
       slots:[10923-16383] (5461 slots) master
    S: 375c25f3c7e0094a660ff6a759a12ab016013142 127.0.0.1:6382
       replicates 0e4bc44048dc19b1c7f41178e22cc87ccdab7b65
    S: 181a6a3a4fe682e59a8cb2b2b8ccae48f8025c81 127.0.0.1:6383
       replicates fd3e386cc455c45d4d9aed656dc8bcebfb147122
    S: 4aef207dcf5e719fb9c206a7de8eb8f6b0ddfa81 127.0.0.1:6384
       replicates 33bebb7958fc5bf48038bd9f097bee2ea77274c9
    Can I set the above configuration? (type 'yes' to accept): yes
    >>> Nodes configuration updated
    >>> Assign a different config epoch to each node
    >>> Sending CLUSTER MEET messages to join the cluster
    Waiting for the cluster to join
    .
    >>> Performing Cluster Check (using node 127.0.0.1:6379)
    M: 0e4bc44048dc19b1c7f41178e22cc87ccdab7b65 127.0.0.1:6379
       slots:[0-5460] (5461 slots) master
       1 additional replica(s)
    S: 181a6a3a4fe682e59a8cb2b2b8ccae48f8025c81 127.0.0.1:6383
       slots: (0 slots) slave
       replicates fd3e386cc455c45d4d9aed656dc8bcebfb147122
    S: 4aef207dcf5e719fb9c206a7de8eb8f6b0ddfa81 127.0.0.1:6384
       slots: (0 slots) slave
       replicates 33bebb7958fc5bf48038bd9f097bee2ea77274c9
    M: 33bebb7958fc5bf48038bd9f097bee2ea77274c9 127.0.0.1:6381
       slots:[10923-16383] (5461 slots) master
       1 additional replica(s)
    S: 375c25f3c7e0094a660ff6a759a12ab016013142 127.0.0.1:6382
       slots: (0 slots) slave
       replicates 0e4bc44048dc19b1c7f41178e22cc87ccdab7b65
    M: fd3e386cc455c45d4d9aed656dc8bcebfb147122 127.0.0.1:6380
       slots:[5461-10922] (5462 slots) master
       1 additional replica(s)
    [OK] All nodes agree about slots configuration.
    >>> Check for open slots...
    >>> Check slots coverage...
    [OK] All 16384 slots covered.

    进入master1主机

    [root@new2 ~]# redis-cli -p 6379
    #写入失败,因为key=a的数据不存在6379主机所在槽点区间,set和get操作均会报错
    127.0.0.1:6379> set a 100
    (error) MOVED 15495 127.0.0.1:6381
    127.0.0.1:6379> get a
    (error) MOVED 15495 127.0.0.1:6381
    127.0.0.1:6379> 
    127.0.0.1:6379> set aa 100
    OK
    127.0.0.1:6379> get aa
    "100"

    集群服务状态确认

    [root@new2 docker-redis-cluster]# redis-cli --cluster check  127.0.0.1:6379
    127.0.0.1:6379 (0e4bc440...) -> 1 keys | 5461 slots | 1 slaves.
    127.0.0.1:6384 (4aef207d...) -> 0 keys | 5461 slots | 1 slaves.
    127.0.0.1:6380 (fd3e386c...) -> 1 keys | 5462 slots | 1 slaves.
    [OK] 2 keys in 3 masters.
    0.00 keys per slot on average.
    >>> Performing Cluster Check (using node 127.0.0.1:6379)
    M: 0e4bc44048dc19b1c7f41178e22cc87ccdab7b65 127.0.0.1:6379
       slots:[0-5460] (5461 slots) master
       1 additional replica(s)
    S: 181a6a3a4fe682e59a8cb2b2b8ccae48f8025c81 127.0.0.1:6383
       slots: (0 slots) slave
       replicates fd3e386cc455c45d4d9aed656dc8bcebfb147122
    M: 4aef207dcf5e719fb9c206a7de8eb8f6b0ddfa81 127.0.0.1:6384
       slots:[10923-16383] (5461 slots) master
       1 additional replica(s)
    S: 33bebb7958fc5bf48038bd9f097bee2ea77274c9 127.0.0.1:6381
       slots: (0 slots) slave
       replicates 4aef207dcf5e719fb9c206a7de8eb8f6b0ddfa81
    S: 375c25f3c7e0094a660ff6a759a12ab016013142 127.0.0.1:6382
       slots: (0 slots) slave
       replicates 0e4bc44048dc19b1c7f41178e22cc87ccdab7b65
    M: fd3e386cc455c45d4d9aed656dc8bcebfb147122 127.0.0.1:6380
       slots:[5461-10922] (5462 slots) master
       1 additional replica(s)
    [OK] All nodes agree about slots configuration.
    >>> Check for open slots...
    >>> Check slots coverage...
    [OK] All 16384 slots covered.

    使用注意项:

    1. 集群环境不能使用select命令,默认使用db0

    [root@new2 redis]# redis-cli
    127.0.0.1:6379> select 1
    (error) ERR SELECT is not allowed in cluster mode
  • 相关阅读:
    谈谈站桩
    mysql开发之---使用游标双层嵌套对总表进行拆分为帖子表和回复表
    【Xcode学C-3】if等流程控制、函数的介绍说明标记分组、#include以及LLVM
    hdu5303(2015多校2)--Delicious Apples(贪心+枚举)
    Hadoop最大值的算法中出现的错误(strToDouble)
    利用管道进行通信
    HDU 5308 规律+模拟
    深入浅出理解排序算法之-选择排序
    Jscript 随记
    SharedPreferences具体解释(一)——基础知识
  • 原文地址:https://www.cnblogs.com/wscsq789/p/13653651.html
Copyright © 2011-2022 走看看