Redis codis 搭建测试

codis
　　Codis 是一个分布式 Redis 解决方案, 对于上层的应用来说, 连接到 Codis Proxy 和连接原生的 Redis
　　Server 没有明显的区别，有部分命令支持
　　Codis 上层会处理请求的转发, 不停机的数据迁移等ٗ作, 所有后边的一切事情, 对于前面的客户端来说是透
明的, 可以简单的认为后边连接的是一个内存无限大的 Redis 服务
Codis由四部ӣ组成
　　• Codis-proxy:实࣫redis协议，由于本身是无状态的，因此可以部署很多个节点
　　• Codis-config：是codis的管理ٗӀ，包括添加/删除redis节点 添加删除proxy节点，发起数据迁移等操作，自带httpserver，支持管理后台方式管理配置
　　• Codis-server：是codis维护的redis分支，基于2.8.21分支，加入了slot的支持和原子的数据迁移指令ͺcodis-proxy和
　　codis-config只能和这个版本的redis交互才能正常运行
• Zookeeper，用于codis集群元数据的存储，维护codis集群节点

Codis优缺点
– 优点
　　• 对客户端透明，与codis交互方式和redis本身交互一样
　　• 支持在线数据迁移，迁移过程对客户端透明
　　• 有简单的管理和监控界面
　　• 支持高可用，无论是redis数据存储还是管理节点
　　• 自动进行数据的均衡分配
　　• 最大支持1024个redis实例，存储容量海量
　　• 高性能
– 缺点
　　• 采用自有的redis分支，不能与原版的redis保持同不
　　• 如果codis的proxy只有一个的情况下，redis的性能会下降20%左右
　　• 某些命令不支持，比如事务命令muti
　　• 国内开源产品，活跃度相对弱一些

Redis codis 搭建

参考文档：

https://studygolang.com/articles/10999?fr=sidebar

https://www.cnblogs.com/edwardsai/p/7884089.html

这里不对codis进行过多的描述，只针对环境进行搭建测试

一环境准备

codis集群的搭建，需要zookeeper集群,除了zookeeper集群之外，我们还需要安装go语言环境，因为codis是基于go语言开发的
1.1 安装基础依赖
#yum install -y git gcc make g++ gcc-c++ automake openssl-devel zlib-*

安装zookeeper
wget --no-check-certificate https://www.apache.org/dist/zookeeper/zookeeper-3.4.10/zookeeper-3.4.10.tar.gz
tar zxvf zookeeper-3.4.10.tar.gz
[root@hongquan2 zookeeper-3.4.10]# mkdir {logs,data}
配置zoo.cfg
cp /usr/local/zookeeper-3.4.10/conf/zoo_sample.cfg /usr/local/zookeeper-3.4.10/conf/zoo1.cfg
vim /usr/local/zookeeper-3.4.10/conf/zoo1.cfg
[root@hongquan2 codis]# cat /usr/local/zookeeper-3.4.10/conf/zoo1.cfg |grep -Ev "^#|^$"
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper-3.4.10/data/1
clientPort=2181
server.1=192.168.20.119:7101:7201
server.2=192.168.20.119:7102:7202
server.3=192.168.20.119:7103:7203

在我们配置的dataDir指定的目录下面，创建一个myid文件，里面内容为一个数字，用来标识当前主机，conf/zoo.cfg文件中配置的server.X中X为什么数字，则myid文件中就输入这个数字
[root@hongquan2 data]# mkdir 1 2 3
echo 1 > /usr/local/zookeeper-3.4.10/data/1/myid
echo 2 > /usr/local/zookeeper-3.4.10/data/2/myid
echo 3 > /usr/local/zookeeper-3.4.10/data/3/myid
启动zookeeper 启动顺序zookeeper-1>zookeeper-2>zookeeper-3
/usr/local/zookeeper-3.4.10/bin/zkServer.sh start /usr/local/zookeeper-3.4.10/conf/zoo1.cfg
[root@hongquan1 conf]# ps -ef|grep zookeeper
/usr/local/zookeeper-3.4.10/bin/zkServer.sh start /usr/local/zookeeper-3.4.10/conf/zoo2.cfg
/usr/local/zookeeper-3.4.10/bin/zkServer.sh start /usr/local/zookeeper-3.4.10/conf/zoo3.cfg

查看leader
/usr/local/zookeeper-3.4.10/bin/zkServer.sh status /usr/local/zookeeper-3.4.10/conf/zoo1.cfg
/usr/local/zookeeper-3.4.10/bin/zkServer.sh status /usr/local/zookeeper-3.4.10/conf/zoo2.cfg
/usr/local/zookeeper-3.4.10/bin/zkServer.sh status /usr/local/zookeeper-3.4.10/conf/zoo3.cfg
可以看到有一个是leader，其他两个是follower
[root@hongquan1 conf]# /usr/local/zookeeper-3.4.10/bin/zkServer.sh status /usr/local/zookeeper-3.4.10/conf/zoo1.cfg
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.10/conf/zoo1.cfg
Mode: follower
[root@hongquan1 conf]# /usr/local/zookeeper-3.4.10/bin/zkServer.sh status /usr/local/zookeeper-3.4.10/conf/zoo2.cfg
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.10/conf/zoo2.cfg
Mode: leader
[root@hongquan1 conf]# /usr/local/zookeeper-3.4.10/bin/zkServer.sh status /usr/local/zookeeper-3.4.10/conf/zoo3.cfg
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.10/conf/zoo3.cfg
Mode: follower

设置开机启动

vim /etc/rc.local
/usr/local/zookeeper-3.4.10/bin/zkServer.sh start /usr/local/zookeeper-3.4.10/conf/zoo1.cfg
/usr/local/zookeeper-3.4.10/bin/zkServer.sh start /usr/local/zookeeper-3.4.10/conf/zoo2.cfg
/usr/local/zookeeper-3.4.10/bin/zkServer.sh start /usr/local/zookeeper-3.4.10/conf/zoo3.cfg
---这里先测试一个zk节点
---注释掉zoo1.cfg里的3个server的信息
--启动一个节点/usr/local/zookeeper-3.4.10/bin/zkServer.sh start /usr/local/zookeeper-3.4.10/conf/zoo1.cfg
[root@hongquan1 bin]# /usr/local/zookeeper-3.4.10/bin/zkCli.sh -server 192.168.20.119:2181
Connecting to 192.168.20.118:2181
2018-05-18 15:24:20,125 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2018-05-18 15:24:20,128 [myid:] - INFO [main:Environment@100] - Client environment:host.name=localhost

1.2 go语言环境搭建
codis是基于go语言开发的，所以我们要在所有服务器上都配置go语言环境。
下载go语言包，如下：
#wget https://storage.googleapis.com/golang/go1.7.1.linux-amd64.tar.gz
#tar -zxvf go1.7.1.linux-amd64.tar.gz -C /usr/local/
把go加入到系统的环境变量

mkdir /usr/local/go/work

[root@hongquan2 soft]# go version
go version go1.7.1 linux/amd64

echo 'export PATH=$PATH:/usr/local/go/bin:/usr/local/codis/bin' >>/etc/profile
echo 'export GOPATH=/usr/local/go/work' >>/etc/profile
echo 'export GOROOT=/usr/local/go' >>/etc/profile
echo 'export ZOOKEEPER_HOME=/usr/local/zookeeper-3.4.10' >>/etc/profile
source /etc/profile

echo $GOPATH
echo $JAVA_HOME
echo $GOROOT
env

二安装codis

mkdir -p $GOPATH/src/github.com/CodisLabs
cd /usr/local/go/work/src/github.com/CodisLabs
git clone https://github.com/CodisLabs/codis.git -b release3.2
[root@hongquan2 CodisLabs]# cd $GOPATH/src/github.com/CodisLabs/codis
[root@hongquan2 codis]# pwd
/usr/local/go/work/src/github.com/CodisLabs/codis
make
make gotest
[root@hongquan2 codis]# make gotest
go test ./cmd/... ./pkg/...
? github.com/CodisLabs/codis/cmd/admin [no test files]
? github.com/CodisLabs/codis/cmd/dashboard [no test files]
? github.com/CodisLabs/codis/cmd/fe [no test files]
? github.com/CodisLabs/codis/cmd/ha [no test files]
? github.com/CodisLabs/codis/cmd/proxy [no test files]
? github.com/CodisLabs/codis/pkg/models [no test files]
? github.com/CodisLabs/codis/pkg/models/etcd [no test files]
? github.com/CodisLabs/codis/pkg/models/fs [no test files]
? github.com/CodisLabs/codis/pkg/models/zk [no test files]
ok github.com/CodisLabs/codis/pkg/proxy 0.967s
ok github.com/CodisLabs/codis/pkg/proxy/redis 0.180s
ok github.com/CodisLabs/codis/pkg/topom 4.309s
ok github.com/CodisLabs/codis/pkg/utils 0.003s
? github.com/CodisLabs/codis/pkg/utils/assert [no test files]
ok github.com/CodisLabs/codis/pkg/utils/bufio2 0.002s
ok github.com/CodisLabs/codis/pkg/utils/bytesize 0.001s
? github.com/CodisLabs/codis/pkg/utils/errors [no test files]
? github.com/CodisLabs/codis/pkg/utils/log [no test files]
ok github.com/CodisLabs/codis/pkg/utils/math2 0.001s
? github.com/CodisLabs/codis/pkg/utils/redis [no test files]
? github.com/CodisLabs/codis/pkg/utils/rpc [no test files]
? github.com/CodisLabs/codis/pkg/utils/sync2 [no test files]
? github.com/CodisLabs/codis/pkg/utils/sync2/atomic2 [no test files]
ok github.com/CodisLabs/codis/pkg/utils/timesize 0.001s
? github.com/CodisLabs/codis/pkg/utils/trace [no test files]
ok github.com/CodisLabs/codis/pkg/utils/unsafe2 0.001s
执行全部指令后，会在 bin 文件夹内生成 codis-proxy、codis-server三个可执行文件。另外, bin/assets 文件夹是 dashboard http 服务需要的前端资源)
[root@hongquan2 codis]# cd bin/
[root@hongquan2 bin]# ll
total 105556
drwxr-xr-x 4 root root 4096 May 21 17:37 assets
-rwxr-xr-x 1 root root 15488394 May 21 17:37 codis-admin
-rwxr-xr-x 1 root root 17131356 May 21 17:37 codis-dashboard
-rwxr-xr-x 1 root root 15374762 May 21 17:37 codis-fe
-rwxr-xr-x 1 root root 13440108 May 21 17:37 codis-ha
-rwxr-xr-x 1 root root 19361861 May 21 17:37 codis-proxy
-rwxr-xr-x 1 root root 7985953 May 21 17:37 codis-server
-rwxr-xr-x 1 root root 5580703 May 21 17:37 redis-benchmark
-rwxr-xr-x 1 root root 5712499 May 21 17:37 redis-cli
-rwxr-xr-x 1 root root 7985953 May 21 17:37 redis-sentinel
-rw-r--r-- 1 root root 166 May 21 17:37 version
[root@hongquan2 bin]# cat version
version = 2018-04-07 12:12:01 +0800 @4803cffb121d21529c1717dddd2e75e3fab06ab3 @3.2.2-6-g4803cff
compile = 2018-05-21 17:37:49 +0800 by go version go1.7.1 linux/amd64

2.3 启动codis dashboard
[root@hongquan2 codis]# vim config/dashboard.toml
coordinator_name = "zookeeper"
coordinator_addr = "192.168.20.119:2181,192.168.20.119:2182,192.168.20.119:2183"
coordinator_auth = ""
product_name = "codis-demo"
product_auth = ""
admin_addr = "192.168.20.119:18080"
[root@hongquan2 codis]# nohup ./bin/codis-dashboard --ncpu=1 --config=config/dashboard.toml --log=dashboard.log --log-level=WARN >> /var/log/codis_dashboard.log &

2.4 启动codis-proxy
修改proxy.toml
[root@hongquan2 codis]# vim config/proxy.toml
product_name = "codis-demo"
product_auth = ""
admin_addr = "192.168.20.119:11080"
jodis_name = "zookeeper"
jodis_addr = "192.168.20.119:2181,192.168.20.119:2182,192.168.20.119:2183"
jodis_auth = ""
jodis_timeout = "20s"
jodis_compatible = true
[root@hongquan2 codis]# nohup ./bin/codis-proxy --ncpu=1 --config=config/proxy.toml --log=proxy.log --log-level=WARN >> /var/log/codis_proxy.log &

启动codis-admin

[root@hongquan2 codis]# ./bin/codis-admin --dashboard=192.168.20.119:18080 --create-proxy -x 192.168.20.119:11080

启动codis-server，即创建redis实例(此处我们创建6个redis实例，给予codis修改过的redis-3.2.8非原生redis)
mkdir -pv /data/redis_638{1..6}
[root@hongquan2 codis]# ./bin/codis-server /data/codis/redis_6381.conf
[root@hongquan2 codis]# ps -ef|grep codis
[root@hongquan2 codis]# ss -tnlp|grep 638*

nohup ./bin/codis-fe --ncpu=1 --log=fe.log --log-level=WARN
--zookeeper=192.168.20.119:2181,192.168.20.119:2182,192.168.20.119:2183 --listen=92.168.20.119:8080 >> /var/log/codis-fe.log &

--------------------
启动codis dashboard
[root@hongquan2 admin]# ./codis-dashboard-admin.sh start
[root@hongquan2 codis]# netstat -tulpn |grep codis-dashboa
tcp 0 0 192.168.20.119:18080 0.0.0.0:* LISTEN 12441/codis-dashboa
http://192.168.20.119:18080/topom

启动codis-proxy
[root@hongquan2 codis]# cat admin/codis-proxy-admin.sh|grep DASH
CODIS_DASHBOARD_ADDR="192.168.20.119:18080"
nohup "$CODIS_PROXY_BIN" "--config=${CODIS_PROXY_CONF_FILE}" "--dashboard=${CODIS_DASHBOARD_ADDR}"
$CODIS_PROXY_BIN "--config=${CODIS_PROXY_CONF_FILE}" "--dashboard=${CODIS_DASHBOARD_ADDR}"
[root@hongquan2 codis]# cat config/proxy.toml|grep -Ev "^#|^$"
product_name = "codis-demo"
product_auth = ""
session_auth = ""
admin_addr = "192.168.20.119:11080"
proto_type = "tcp4"
proxy_addr = "0.0.0.0:19000" >>192.168.20.119:19000
jodis_name = "zookeeper"
jodis_addr = "192.168.20.119:2181,192.168.20.119:2182,192.168.20.119:2183"
jodis_auth = ""
jodis_timeout = "20s"
jodis_compatible = true
session_recv_timeout = "30m" #如果不为０可能导致应用程序出现”write: broken pipe”的问题

[root@hongquan2 codis]# ./admin/codis-proxy-admin.sh start
/usr/local/go/work/src/github.com/CodisLabs/codis/admin/../config/proxy.toml
starting codis-proxy ...
[root@hongquan2 codis]# netstat -tulpn|grep codis-proxy
tcp 0 0 192.168.20.119:11080 0.0.0.0:* LISTEN 12474/codis-proxy
tcp 0 0 0.0.0.0:19000 0.0.0.0:* LISTEN 12474/codis-proxy

启动codis-server
# cat /data/codis/redis_6381.conf |grep -Ev "^#|^$"|grep 6381
port 6381
pidfile /data/codis/redis_6381.pid
logfile "/data/codis/redis_6381.log"
dbfilename dump_6381.rdb

# sed -i 's/6381/6382/g' codis-server-admin-6382.sh
# cat codis-server-admin-6382.sh |grep -Ev "^#|^$"|grep 6382
[root@hongquan2 codis]#./admin/codis-server-admin-6381.sh start
[root@hongquan2 codis]# netstat -tulpn |grep codis-server
tcp 0 0 192.168.20.119:6381 0.0.0.0:* LISTEN 15971/codis-server
tcp 0 0 192.168.20.119:6382 0.0.0.0:* LISTEN 15979/codis-server
tcp 0 0 192.168.20.119:6383 0.0.0.0:* LISTEN 15987/codis-server
tcp 0 0 192.168.20.119:6384 0.0.0.0:* LISTEN 15996/codis-server
tcp 0 0 192.168.20.119:6385 0.0.0.0:* LISTEN 16004/codis-server
tcp 0 0 192.168.20.119:6386 0.0.0.0:* LISTEN 16012/codis-server

启动codis-fe
[root@hongquan2 codis]# vim admin/codis-fe-admin.sh
#COORDINATOR_NAME="filesystem"
#COORDINATOR_ADDR="/tmp/codis"
COORDINATOR_NAME="zookeeper"
COORDINATOR_ADDR="192.168.20.119:2181,192.168.20.119:2182,192.168.20.119:2183"
CODIS_FE_ADDR="0.0.0.0:9090"

[root@hongquan2 codis]# ./admin/codis-fe-admin.sh start
检查日志
2018/05/22 09:43:44 main.go:101: [WARN] set ncpu = 1
2018/05/22 09:43:44 main.go:104: [WARN] set listen = 192.168.20.119:2181,192.168.20.119:2182,192.168.20.119:2183
2018/05/22 09:43:44 main.go:120: [WARN] set assets = /usr/local/go/work/src/github.com/CodisLabs/codis/bin/assets
2018/05/22 09:43:44 args.go:18: [PANIC] option --zookeeper requires an argument
[stack]:
2 /usr/local/go/work/src/github.com/CodisLabs/codis/pkg/utils/args.go:18
github.com/CodisLabs/codis/pkg/utils.Argument
1 /usr/local/go/work/src/github.com/CodisLabs/codis/pkg/utils/args.go:27
github.com/CodisLabs/codis/pkg/utils.ArgumentMust
0 /usr/local/go/work/src/github.com/CodisLabs/codis/cmd/fe/main.go:142
main.main
... ...
[root@hongquan2 codis]# netstat -tupnl |grep codis-fe
tcp 0 0 :::9090 :::* LISTEN 16059/codis-fe

访问网址http://192.168.20.119:9090/#codis-demo
codis-fe面板操作

1，通过codis-fe添加group
在 Proxy 栏可看到我们已经启动的 Proxy， 但是Group 栏为空，因为我们启动的 codis-server 并未加入到集群 添加 NEW GROUP，
NEW GROUP 行输入 1，再点击 NEW GROUP 即可 添加 Codis Server，Add Server 行输入我们刚刚启动的
codis-server 地址，添加到我们刚新建的 Group，然后再点击 Add Server 按钮即可
如上依次添加3个group，6个codis-server，默认每组里面第一个添加的为主，第二个添加的设置为从，同一个节点2个实例不能设置为同一group。

--可以用命令
./bin/codis-config server add 1 192.168.20.119:6181 master
./bin/codis-config server add 1 192.168.20.119:6182 slave
./bin/codis-config server add 2 192.168.20.119:6183 master
./bin/codis-config server add 2 192.168.20.119:6184 slave
./bin/codis-config server add 3 192.168.20.119:6185 master
./bin/codis-config server add 3 192.168.20.119:6186 slave

2，通过codis-fe初始化solt
新增的集群 slot 状态是 offline，因此我们需要对它进行初始化（将 1024 个 slot 分配到各个 group），而初始化最快的方法可
通过 fe 提供的 rebalance all slots 按钮来做，如下图所示，点击此按钮，我们即快速完成了一个集群的搭建

/usr/local/codis/bin/codis-config slot range-set 0 334 1 online
/usr/local/codis/bin/codis-config slot range-set 335 669 2 online
/usr/local/codis/bin/codis-config slot range-set 670 1023 3 online
##./bin/codis-config slot info 1

Migrate Slots-[335,669] to Group-[2]

=============
通过 ansible 快速部署集群
使用 ansible 可快速在单机、多机部署多套 codis 集群。 ansible 文件夹包含了部署 codis 集群的 playbook，根据自己部署环境修改
groups_var/all 文件里参数，修改 hosts 文件添加部署的环境 IP 即可。 ansible 安装也及其简单，各部署机器无需安装任何额外的
agent，彼此之间通过 ssh 通信。

git clone https://github.com/ansible/ansible.git -b stable-2.3
cd ./ansible
source ./hacking/env-setup
cd $codis_dir/ansible
ansible-playbook -i hosts site.yml
============

1 测试主从
[root@hongquan2 codis]# ./bin/redis-cli -h 192.168.20.119 -p 6381
192.168.20.119:6381> set name test
OK
192.168.20.119:6381> set age 24
OK
[root@hongquan2 codis]# ./bin/redis-cli -h 192.168.20.119 -p 6382
192.168.20.119:6382> get name
"test"
192.168.20.119:6382> get age
"24"

2 连接proxy
[root@hongquan2 codis]# ps -ef |grep proxy
[root@hongquan2 codis]# ./bin/redis-cli -h 192.168.20.119 -p 19000
192.168.20.119:19000> get name
(nil)
192.168.20.119:19000> get age
"24"
[root@hongquan2 codis]# ./bin/redis-benchmark -h 192.168.20.119 -p 19000 -c 10000 -d 100 -t set -n 100000 -r 100000
Could not connect to Redis at 192.168.20.119:19000: Can't create socket: Too many open files
[root@hongquan2 codis]# ./bin/redis-benchmark -h 192.168.20.119 -p 19000 -c 1000 -d 100 -t set -n 1000 -r 1000
====== SET ======
1000 requests completed in 0.10 seconds
1000 parallel clients
100 bytes payload
keep alive: 1

0.10% <= 2 milliseconds
1.20% <= 3 milliseconds
3.20% <= 4 milliseconds
5.20% <= 5 milliseconds
7.20% <= 6 milliseconds
9.30% <= 7 milliseconds
11.20% <= 8 milliseconds
13.30% <= 9 milliseconds
15.30% <= 10 milliseconds
16.90% <= 11 milliseconds
21.40% <= 12 milliseconds
23.40% <= 13 milliseconds
25.40% <= 14 milliseconds
25.90% <= 15 milliseconds
30.10% <= 16 milliseconds
32.00% <= 17 milliseconds
33.90% <= 18 milliseconds
35.80% <= 19 milliseconds
37.70% <= 20 milliseconds
39.00% <= 21 milliseconds
42.60% <= 22 milliseconds
46.90% <= 23 milliseconds
48.10% <= 27 milliseconds
67.00% <= 28 milliseconds
70.70% <= 29 milliseconds
72.60% <= 30 milliseconds
74.40% <= 31 milliseconds
76.40% <= 32 milliseconds
78.40% <= 33 milliseconds
80.40% <= 34 milliseconds
82.50% <= 35 milliseconds
84.40% <= 36 milliseconds
86.30% <= 37 milliseconds
88.20% <= 38 milliseconds
90.10% <= 39 milliseconds
92.00% <= 40 milliseconds
93.80% <= 41 milliseconds
95.70% <= 42 milliseconds
96.80% <= 60 milliseconds
97.40% <= 61 milliseconds
98.50% <= 62 milliseconds
100.00% <= 62 milliseconds
9523.81 requests per second
[root@hongquan2 codis]# ./bin/redis-cli -h 192.168.20.119 -p 19000
192.168.20.119:19000> info
# Server
redis_version:3.2.11
redis_git_sha1:4803cffb
redis_git_dirty:0
redis_build_id:c32f2fa5aed3188b
redis_mode:standalone
os:Linux 2.6.39-200.24.1.el6uek.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
gcc_version:4.4.7
process_id:15971
run_id:6a3ae5b9e7a4d9299b7a2b482df63cc079183009
tcp_port:6381
uptime_in_seconds:2968
uptime_in_days:0
hz:10
lru_clock:229259
executable:/usr/local/go/work/src/github.com/CodisLabs/codis/admin/../bin/codis-server
config_file:/data/codis/redis_6381.conf

# Clients
connected_clients:17
client_longest_output_list:0
client_biggest_input_buf:0
blocked_clients:0

# Memory
used_memory:4061664
used_memory_human:3.87M
used_memory_rss:16297984
used_memory_rss_human:15.54M
used_memory_peak:4691400
used_memory_peak_human:4.47M
total_system_memory:2104434688
total_system_memory_human:1.96G
used_memory_lua:37888
used_memory_lua_human:37.00K
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
mem_fragmentation_ratio:4.01
mem_allocator:jemalloc-4.0.3

# Persistence
loading:0
rdb_changes_since_last_save:7
rdb_bgsave_in_progress:0
rdb_last_save_time:1526955825
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:0
rdb_current_bgsave_time_sec:-1
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok

# Stats
total_connections_received:55
total_commands_processed:9808
instantaneous_ops_per_sec:3
total_net_input_bytes:260836
total_net_output_bytes:4447631
instantaneous_input_kbps:0.09
instantaneous_output_kbps:2.85
rejected_connections:0
sync_full:1
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
evicted_keys:0
keyspace_hits:1
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:277
migrate_cached_sockets:0

# Replication
role:master
connected_slaves:1
slave0:ip=192.168.20.119,port=6382,state=online,offset=47259,lag=0
master_repl_offset:47259
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:47258

# CPU
used_cpu_sys:0.60
used_cpu_user:0.55
used_cpu_sys_children:0.00
used_cpu_user_children:0.00

# Cluster
cluster_enabled:0

# Keyspace
db0:keys=191,expires=0,avg_ttl=0
192.168.20.119:19000>

启动codis-ha
[root@hongquan2 codis]# ./bin/codis-ha --log=ha.log --log-level=WARN --dashboard=192.168.20.119:18080 &

-------
2.5 创建redis server分组
可以通过命令进行创建
/usr/local/codis/bin/codis-config server add-group 1
/usr/local/codis/bin/codis-config server add-group 2
/usr/local/codis/bin/codis-config server add-group 3
/usr/local/codis/bin/codis-config server list
2.6 添加redis实例
可以通过命令进行添加，如下
./bin/codis-config server add 1 192.168.20.119:6181 master
./bin/codis-config server add 1 192.168.20.119:6182 slave
./bin/codis-config server add 2 192.168.20.119:6183 master
./bin/codis-config server add 2 192.168.20.119:6184 slave
./bin/codis-config server add 3 192.168.20.119:6185 master
./bin/codis-config server add 3 192.168.20.119:6186 slave

注意：上述命令中的数字，表示的是哪一个分组，master/slave表示的是所属类型。
/usr/local/codis/bin/codis-config server list

2.7 分配slot范围
codis采用pre-sharding的技术来实现数据的分片，默认分成1024个slot(0-1023)。对于每个key来说，可以通过以下公式确定所属的slot id:slotid=crc32(key)%1024。
每一个slot都会有一个且必须有一个特定的server group id来表示这个slot的数据由哪个server group来提供
在分配slot之前，我们需要初始化slot。
在codis服务器任意一台上执行bin/codis-config slot init命令，该命令会在zookeeper上创建slot相关信息。如下
cd /usr/local/codis
/usr/local/codis/bin/codis-config -c config.ini slot init
slot初始化完毕后，我们现在来分配slot范围。如下

可以通过命令进行分配，如下
/usr/local/codis/bin/codis-config slot range-set 0 334 1 online
/usr/local/codis/bin/codis-config slot range-set 335 669 2 online
/usr/local/codis/bin/codis-config slot range-set 670 1023 3 online
/usr/local/codis/bin/codis-config slot info 1
/usr/local/codis/bin/codis-config slot info 2
/usr/local/codis/bin/codis-config slot info 3

三、连接codis集群

codis集群搭建完毕后，现在我们来连接codis集群。要连接codis集群，我们只需要连接codis-proxy即可。即连接4.7章节中的codis-proxy服务器地址，然后加19000端口。使用redis-cli命令连接，如下：
redis-cli -h 192.168.1.9 -p 19000
info

我们现在对codis集群做一些压力测试，同时在dashboard上观察键值对的情况。如下：
redis-benchmark -h 192.168.1.9 -p 19000 -c 10000 -d 100 -t set -n 100000 -r 100000
上述命令的意思是，使用redis-benchmark压力测试命令连接codis集群，同时并发10000个（-c），测试set操作(-t)，每个测试数据集是100字节(-d),请求数是100000（-n），使用使用随机数插入数值（-r）