zoukankan      html  css  js  c++  java
  • redis 消息队列(发布订阅)、持久化(RDB、AOF)、集群(cluster)

    一:订阅:

    192.168.10.205:6379> SUBSCRIBE test
    Reading messages... (press Ctrl-C to quit)
    1) "subscribe"
    2) "test"
    3) (integer) 1
    
    
    在另一个终端向订阅test发布消息:
    127.0.0.1:6379> PUBLISH test1  hello
    (integer) 0
    
    
    上一个终端的订阅就收到了消息:
    1) "message"
    2) "test"
    3) "hello"

    二:redis持久化:

    redis支持两种数据持久化,一种是rdb,会在指定的数据间隔内把内存的数据快照到文件,aof将所有服务端执行的命令备查,在服务重启的时候全部执行以此

    rdb在断电的时候会丢失部分内网,aof不会

    1、使用rdb持久化redis数据:

    配置文件的快照时间设置:

    快照是fork出一个当前的进程在后台进行快照,不会影响当前的进程,先写一个临时文件,写完以后将临时文件替换旧的文件。这个文件任何时候都是一个完整的副本

    save 900 1 #900秒以内有1个key发生变化就快照
    save 300 10  #300秒以内有10个key发生变化就快照
    save 60 10000 #60秒内有10000个key变化就快照

    rdbcompression yes  #持久化到RDB文件时,是否压缩,“yes”为压缩,“no”则反之

    rdbchecksum yes  #读取和写入的时候是否支持CRC64校验,默认是开启的

    dbfilename dump.rdb  #保存的文件的名称

    dir  /usr/local/redis  #快照文件的保存路径

    SAVE 与 BGSAVE:

    SAVE:阻塞保存

    BGSAVE:在后台保存,不阻塞

    KILL -9 会丢失自上次保存以后到现在的数据

    2、使用AOF进行持久化:

    appendonly yes #默认为no,改为yes

    appendfilename "appendonly.aof" #保存的文件名,路径为rdb指定的file目录

    关闭服务在启动即可

    三:redis主从:

    一个主服务器可以有多个从服务器,而且从服务器也可以有主服务器,主从复制时非阻塞的,通过fork主进程

    主服务器收到从服务器的sync命令,就在后台执行bgsave,执行完毕后将保持的rdb文件发给客户端,客户端收到后将rdb载入到内存,新版本的支持增量备份

    通过info命令查看默认配置:

    # Replication
    role:master    #默认较色为主服务器
    connected_slaves:0
    master_repl_offset:0
    repl_backlog_active:0
    repl_backlog_size:1048576
    repl_backlog_first_byte_offset:0
    repl_backlog_histlen:0

    在从服务器执行命令:

    127.0.0.1:6379> SLAVEOF 192.168.10.205 6379 #主服务器的地址和端口
    OK

    查看从服务器的info信息:

    # Replication
    role:slave #角色为从服务器
    master_host:192.168.10.205 #主服务器的地址
    master_port:6379 #主服务器的端口
    master_link_status:up
    master_last_io_seconds_ago:8
    master_sync_in_progress:0
    slave_repl_offset:29
    slave_priority:100
    slave_read_only:1
    connected_slaves:0
    master_repl_offset:0
    repl_backlog_active:0
    repl_backlog_size:1048576
    repl_backlog_first_byte_offset:0
    repl_backlog_histlen:0

    主服务器的状态信息:

    # Replication
    role:master #角色为主服务器
    connected_slaves:1
    slave0:ip=192.168.10.206,port=6379,state=online,offset=471,lag=0 #从服务器的地址和端口
    master_repl_offset:471
    repl_backlog_active:1
    repl_backlog_size:1048576
    repl_backlog_first_byte_offset:2
    repl_backlog_histlen:470

    从服务器无法写,如下:

    127.0.0.1:6379> set a x
    (error) READONLY You can't write against a read only slave.

    三:redis集群:

    四种大方案:

    1、客户端分片,优势是比较灵活,不存在单点故障,缺点是添加节点需要重新配置,数据要手动同步

    2、代理:代理分片,有proxy代理取数据,可以为proxy配置算法,如Twemproxy

    3、Redis Cluster:在3.0版本以后支持,无中心,在某种情况下会造成数据丢失

    4、Coodis:豌豆荚的开源方案

    使用redis cluster:

    需要至少6个机器,3主3从,下面创建8个客户端,另外2个做备用

    先将配置文件复制到/opt,然后创建8个目录并生成8个不同端口、不同pid和不同rdb文件的redis.conf配置文件:

    [root@node5 ~]# cd /opt/
    [root@node5 opt]# mkdir `seq 7001 7008`
    [root@node5 opt]# cp /etc/redis/6379.conf  .
    [root@node5 opt]# sed 's/6379/7001/g' 6379.conf >> 7001/redis.conf
    [root@node5 opt]# sed 's/7001/7002/g' 6379.conf >> 7002/redis.conf
    [root@node5 opt]# sed 's/7002/7003/g' 6379.conf >> 7003/redis.conf
    [root@node5 opt]# sed 's/7003/7004/g' 6379.conf >> 7004/redis.conf
    [root@node5 opt]# sed 's/7004/7005/g' 6379.conf >> 7005/redis.conf
    [root@node5 opt]# sed 's/7005/7006/g' 6379.conf >> 7006/redis.conf
    [root@node5 opt]# sed 's/7006/7007/g' 6379.conf >> 7007/redis.conf
    [root@node5 opt]# sed 's/7007/7008/g' 6379.conf >> 7008/redis.conf

    批量启动redis:

    [root@node5 opt]# for i in `seq 7001 7008`;do cd /opt/$i && /usr/local/redis/bin/redis-server  /opt/$i/redis.conf; done

     确认端口都已经启动成功:

    [root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7001
    192.168.10.205:7001> 
    [root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7002
    192.168.10.205:7002> 
    [root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7003
    192.168.10.205:7003> 
    [root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7004
    192.168.10.205:7004> 
    [root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7005
    192.168.10.205:7005> 
    [root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7006
    192.168.10.205:7006> 
    [root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7007
    192.168.10.205:7007> 
    [root@node5 opt]# redis-cli  -h 192.168.10.205 -p 7008
    192.168.10.205:7008> 

    安装ruby管理工具:

    [root@node5 opt]# yum install ruby rubygems -y
    [root@node5 opt]# gem install redis
    Successfully installed redis-3.2.2
    1 gem installed
    Installing ri documentation for redis-3.2.2...
    Installing RDoc documentation for redis-3.2.2..

    复制ruby的管理脚本:

    [root@node5 ~]# cp /root/redis-3.0.7/src/redis-trib.rb  /usr/local/bin/redis-trib #是一个ruby脚本,方便redis管理

    功能介绍:

    root@node5 ~]# redis-trib  help
    Usage: redis-trib <command> <options> <arguments ...>
    
      help            (show this help)
      del-node        host:port node_id #删除节点
      reshard         host:port #重新分片
                      --timeout <arg>
                      --pipeline <arg>
                      --slots <arg>
                      --to <arg>
                      --yes
                      --from <arg>
      fix             host:port
                      --timeout <arg>
      create          host1:port1 ... hostN:portN #创建集群
                      --replicas <arg>
      rebalance       host:port
                      --timeout <arg>
                      --simulate
                      --pipeline <arg>
                      --threshold <arg>
                      --use-empty-masters
                      --auto-weights
                      --weight <arg>
      call            host:port command arg arg .. arg
      add-node        new_host:new_port existing_host:existing_port #添加节点
                      --slave
                      --master-id <arg>
      check           host:port #检测节点
      import          host:port
                      --replace
                      --copy
                      --from <arg>
      set-timeout     host:port milliseconds
      info            host:port

    创建集群:

    [root@node5 ~]# redis-trib  create --replicas 1 192.168.10.205:7001 192.168.10.205:7002 192.168.10.205:7003 192.168.10.205:7004 192.168.10.205:7005 192.168.10.205:7006
    >>> Creating cluster
    >>> Performing hash slots allocation on 6 nodes...
    Using 3 masters:
    192.168.10.205:7001
    192.168.10.205:7002
    192.168.10.205:7003
    Adding replica 192.168.10.205:7004 to 192.168.10.205:7001
    Adding replica 192.168.10.205:7005 to 192.168.10.205:7002
    Adding replica 192.168.10.205:7006 to 192.168.10.205:7003
    M: 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001
       slots:0-5460 (5461 slots) master
    M: 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002
       slots:5461-10922 (5462 slots) master
    M: 8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003
       slots:10923-16383 (5461 slots) master
    S: 32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004
       replicates 9db0386330f3d66d0c7cac48dbf2dda1972c96c4
    S: 19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005
       replicates 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6
    S: 840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006
       replicates 8f2f43ef9008f299786b3f02446a0ac0d20dde35
    Can I set the above configuration? (type 'yes' to accept): yes
    >>> Nodes configuration updated
    >>> Assign a different config epoch to each node
    >>> Sending CLUSTER MEET messages to join the cluster
    Waiting for the cluster to join..
    >>> Performing Cluster Check (using node 192.168.10.205:7001)
    M: 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001 #ID及IP和端口
       slots:0-5460 (5461 slots) master #本机对应的分片位置,0-5460
    M: 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002
       slots:5461-10922 (5462 slots) master #本机对应的分片位置,5461-10922 #ID及IP和端口
    M: 8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003
       slots:10923-16383 (5461 slots) master #本机对应的分片位置,10923-16383 #ID及IP和端口
    M: 32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004
       slots: (0 slots) master
       replicates 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 #master的ID
    M: 19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005
       slots: (0 slots) master
       replicates 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 #master的ID
    M: 840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006
       slots: (0 slots) master
       replicates 8f2f43ef9008f299786b3f02446a0ac0d20dde35 #master的ID
    [OK] All nodes agree about slots configuration.
    >>> Check for open slots...
    >>> Check slots coverage...
    [OK] All 16384 slots covered.

    连接到集群:

    [root@node5 ~]# redis-cli  -h 192.168.10.205 -c -p 7001

    查看集群信息:

    # Replication
    role:master
    connected_slaves:1
    slave0:ip=192.168.10.205,port=7004,state=online,offset=407,lag=0
    master_repl_offset:407
    repl_backlog_active:1
    repl_backlog_size:1048576
    repl_backlog_first_byte_offset:2
    repl_backlog_histlen:406

    操作集群:

    192.168.10.205:7001> set key1 value1
    -> Redirected to slot [9189] located at 192.168.10.205:7002
    OK
    192.168.10.205:7002> set key2 value2 
    -> Redirected to slot [4998] located at 192.168.10.205:7001
    OK
    192.168.10.205:7001> set key3 value3
    OK
    192.168.10.205:7001> set key4 value4
    -> Redirected to slot [13120] located at 192.168.10.205:7003
    OK
    192.168.10.205:7003> set key5 value5
    -> Redirected to slot [9057] located at 192.168.10.205:7002
    OK
    192.168.10.205:7002> set key6 value6
    -> Redirected to slot [4866] located at 192.168.10.205:7001
    OK
    192.168.10.205:7001> set key7 value7
    OK
    192.168.10.205:7001> set key8 value8
    -> Redirected to slot [13004] located at 192.168.10.205:7003 #插入的数据是轮训写入到各个redis server
    OK

     cluster nodes:列出当前的主从

    192.168.10.205:7003> CLUSTER  nodes
    8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003 myself,master - 0 0 3 connected 10923-16383
    5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002 master - 0 1458558870837 2 connected 5461-10922
    32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004 slave 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 0 1458558869829 4 connected
    840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006 slave 8f2f43ef9008f299786b3f02446a0ac0d20dde35 0 1458558868820 6 connected
    19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005 slave 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 0 1458558870334 5 connected
    9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001 master - 0 1458558869829 1 connected 0-5460

    cluster info:当前的状态:

    192.168.10.205:7003> CLUSTER INFO 
    cluster_state:ok
    cluster_slots_assigned:16384
    cluster_slots_ok:16384
    cluster_slots_pfail:0
    cluster_slots_fail:0
    cluster_known_nodes:6
    cluster_size:3
    cluster_current_epoch:6
    cluster_my_epoch:3
    cluster_stats_messages_sent:2911
    cluster_stats_messages_received:2911

     向集群中添加节点:

    [root@node5 ~]# redis-trib  add-node 192.168.10.205:7007 192.168.10.205:7001

                         要添加的节点和端口       添加到的目的节点和端口

    [root@node5 ~]# redis-trib  add-node 192.168.10.205:7007 192.168.10.205:7001
    >>> Adding node 192.168.10.205:7007 to cluster 192.168.10.205:7001
    >>> Performing Cluster Check (using node 192.168.10.205:7001)
    M: 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001
       slots:0-5460 (5461 slots) master
       1 additional replica(s)
    M: 8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003
       slots:10923-16383 (5461 slots) master
       1 additional replica(s)
    S: 840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006
       slots: (0 slots) slave
       replicates 8f2f43ef9008f299786b3f02446a0ac0d20dde35
    M: 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002
       slots:5461-10922 (5462 slots) master
       1 additional replica(s)
    S: 32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004
       slots: (0 slots) slave
       replicates 9db0386330f3d66d0c7cac48dbf2dda1972c96c4
    S: 19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005
       slots: (0 slots) slave
       replicates 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6
    [OK] All nodes agree about slots configuration.
    >>> Check for open slots...
    >>> Check slots coverage...
    [OK] All 16384 slots covered.
    >>> Send CLUSTER MEET to node 192.168.10.205:7007 to make it join the cluster.
    [OK] New node added correctly.

    添加主机以后需要重新分片:

    [root@node5 ~]# redis-trib  reshard 192.168.10.205:7007

    连接到新添加的节点查询信息:

    [root@node5 ~]# redis-cli -h 192.168.10.205 -p  7007
    192.168.10.205:7007> KEYS *
    (empty list or set)
    192.168.10.205:7007> CLUSTER NODES
    5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002 master - 0 1458560205983 2 connected 5728-10922
    8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003 master - 0 1458560204471 3 connected 11189-16383
    19a5bb651f4b182a0e3dd49154323d31c1524167 192.168.10.205:7007 myself,master - 0 0 7 connected 0-265 5461-5727 10923-11188 #已经有了分片信息
    19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005 slave 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 0 1458560204977 2 connected
    840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006 slave 8f2f43ef9008f299786b3f02446a0ac0d20dde35 0 1458560205482 3 connected
    9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001 master - 0 1458560204471 1 connected 266-5460
    32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004 slave 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 0 1458560204977 1 connected

    为新添加的主节点添加从节点:

    [root@node5 ~]# redis-trib  add-node 192.168.10.205:7008 192.168.10.205:7001 #添加一个节点
    
    #登陆到从节点进行配置
    [root@node5 ~]# redis-cli -h 192.168.10.205 -c -p 7008 #登陆到要备用的从节点
    192.168.10.205:7008> CLUSTER replicate 19a5bb651f4b182a0e3dd49154323d31c1524167 #指定主节点的ID即可
    OK

    再次查看集群信息:

    192.168.10.205:7008> CLUSTER  nodes
    19806347e07d4b341e3ddff16136241d16e410c4 192.168.10.205:7005 slave 5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 0 1458560717276 2 connected
    ab146fb5881064a6375da76f861e205d2d702140 192.168.10.205:7008 myself,slave 19a5bb651f4b182a0e3dd49154323d31c1524167 0 0 0 connected   #已经是从节点并连接了
    32d95f05196786500187137f30040968cc8b6521 192.168.10.205:7004 slave 9db0386330f3d66d0c7cac48dbf2dda1972c96c4 0 1458560716772 1 connected
    8f2f43ef9008f299786b3f02446a0ac0d20dde35 192.168.10.205:7003 master - 0 1458560715764 3 connected 11189-16383
    19a5bb651f4b182a0e3dd49154323d31c1524167 192.168.10.205:7007 master - 0 1458560717276 7 connected 0-265 5461-5727 10923-11188
    9db0386330f3d66d0c7cac48dbf2dda1972c96c4 192.168.10.205:7001 master - 0 1458560717276 1 connected 266-5460
    5d1ccd0b651b135bbc5e91976aff91ce3e33b0b6 192.168.10.205:7002 master - 0 1458560716267 2 connected 5728-10922
    840e13e7cef0db82172fab8c50180088055ada60 192.168.10.205:7006 slave 8f2f43ef9008f299786b3f02446a0ac0d20dde35 0 1458560717780 3 connected
  • 相关阅读:
    kubeadm init 时从本地私有仓库下载镜像
    Kubernetes 1.6新特性
    kukubeadm 1.6.1 + docker1.2.6 安装问题
    nginx 配置
    使用nginx替换Ingress
    Docker存储方式选型建议
    iptables相关
    kubenetes master使用curl 操作API
    kubernetes 一些基本的概念
    Codeforces Round #161 (Div. 2) D. Cycle in Graph(无向图中找指定长度的简单环)
  • 原文地址:https://www.cnblogs.com/zhang-shijie/p/5309975.html
Copyright © 2011-2022 走看看