MongoDB主从复制

介绍了Mongodb的安装使用，在 MongoDB 中,有两种数据冗余方式,一种 是 Master-Slave 模式（主从复制）,一种是 Replica Sets 模式（副本集）。

Mongodb一共有三种集群搭建的方式：
Replica Set（副本集）、
Sharding（切片）
Master-Slaver（主从）【目前已不推荐使用了！！！】
 
其中，Sharding集群也是三种集群中最复杂的。
副本集比起主从可以实现故障转移！！非常使用！
 
mongoDB目前已不推荐使用主从模式，取而代之的是副本集模式。副本集其实一种互为主从的关系，可理解为主主。
副本集指将数据复制，多份保存，不同服务器保存同一份数据，在出现故障时自动切换。对应的是数据冗余、备份、镜像、读写分离、高可用性等关键词；
而分片则指为处理大量数据，将数据分开存储，不同服务器保存不同的数据，它们的数据总和即为整个数据集。追求的是高性能。
 
在生产环境中，通常是这两种技术结合使用，分片+副本集。

mongodb主从复制配置

有些版本已经不支持主从复制（本人环境为mongodb4.0版本，已经不支持主从复制）

主从复制是MongoDB最常用的复制方式,也是一个简单的数据库同步备份的集群技术,这种方式很灵活.可用于备份,故障恢复,读扩展等. 
最基本的设置方式就是建立一个主节点和一个或多个从节点,每个从节点要知道主节点的地址。采用双机备份后主节点挂掉了后从节点可以接替主机继续服务。所以这种模式比单节点的高可用性要好很多。

  配置主从复制的注意点

1）在数据库集群中要明确的知道谁是主服务器,主服务器只有一台.
2）从服务器要知道自己的数据源也就是对应的主服务是谁.
3）--master用来确定主服务器,--slave 和 --source 来控制从服务器

可以在mongodb.conf配置文件里指明主从关系，这样启动mongodb的时候只要跟上配置文件就行，就不需要通过--master和--slave来指明主从了。

下面简单记录下Mongodb主从复制的部署过程

1）机器环境
182.48.115.238    master-node
182.48.115.236    slave-node
 
两台机器都关闭防火墙和selinux
mongodb的安装参考：http://www.cnblogs.com/kevingrace/p/5752382.html
 
2）主从配置
.............master－node节点配置.............
[root@master-node ~]# vim /usr/local/mongodb/mongodb.conf
port=27017
bind_ip = 182.48.115.238
dbpath=/usr/local/mongodb/data
logpath=/usr/local/mongodb/log/mongo.log
logappend=true
journal = true
fork = true
master = true        //确定自己是主服务器
 
[root@master-node ~]# nohup /usr/local/mongodb/bin/mongod --config /usr/local/mongodb/mongodb.conf &
 
[root@master-node ~]# ps -ef|grep mongodb
root     15707 15514 23 16:45 pts/2    00:00:00 /usr/local/mongodb/bin/mongod --config /usr/local/mongodb/mongodb.conf
root     15736 15514  0 16:45 pts/2    00:00:00 grep mongodb
[root@master-node ~]# lsof -i:27017
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
mongod  15707 root    7u  IPv4 153114      0t0  TCP 182.48.115.238:27017 (LISTEN)
 
由于mongodb.conf里绑定了本机的ip地址182.48.115.238，所以连接mongodb的时候必须用这个ip地址，不能使用默认的127.0.0.1，也就是说：
[root@master-node ~]# mongo 182.48.115.238:27017     //这样才能连接mongodb
[root@master-node ~]# mongo 或者 mongodb 127.0.0.1:27017    // 这样不能连接mongodb
 
.............slave－node节点配置.............
[root@slave-node ~]# vim /usr/local/mongodb/mongodb.conf
port=27017
dbpath=/usr/local/mongodb/data
logpath=/usr/local/mongodb/log/mongo.log
logappend=true
journal = true
fork = true
bind_ip = 182.48.115.236            //确定主数据库端口
source = 182.48.115.238:27017      //确定主数据库端口
slave = true               //确定自己是从服务器
 
[root@slave-node ~]# nohup /usr/local/mongodb/bin/mongod --config /usr/local/mongodb/mongodb.conf &
 
[root@slave-node ~]# ps -ef|grep mongo
root     26290 26126  8 16:47 pts/0    00:00:00 /usr/local/mongodb/bin/mongod --config /usr/local/mongodb/mongodb.conf
root     26316 26126  0 16:47 pts/0    00:00:00 grep mongo
[root@slave-node ~]# lsof -i:27017
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
mongod  26290 root    7u  IPv4 663904      0t0  TCP slave-node1:27017 (LISTEN)
mongod  26290 root   25u  IPv4 663917      0t0  TCP slave-node1:51060->slave-node2:27017 (ESTABLISHED)
 
在slave-node测试连接master-node的mongodb数据库是否正常
[root@slave-node ~]# mongo 182.48.115.236:27017
MongoDB shell version v3.4.4
connecting to: 182.48.115.236:27017
MongoDB server version: 3.4.4
Server has startup warnings:
2017-06-03T16:47:31.200+0800 I STORAGE  [initandlisten]
2017-06-03T16:47:31.200+0800 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2017-06-03T16:47:31.200+0800 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2017-06-03T16:47:32.472+0800 I CONTROL  [initandlisten]
2017-06-03T16:47:32.472+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-06-03T16:47:32.472+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2017-06-03T16:47:32.472+0800 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2017-06-03T16:47:32.472+0800 I CONTROL  [initandlisten]
2017-06-03T16:47:32.473+0800 I CONTROL  [initandlisten]
2017-06-03T16:47:32.473+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2017-06-03T16:47:32.473+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2017-06-03T16:47:32.473+0800 I CONTROL  [initandlisten]
2017-06-03T16:47:32.473+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2017-06-03T16:47:32.473+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2017-06-03T16:47:32.473+0800 I CONTROL  [initandlisten]
>
 
3）主从数据同步测试
在master-node节点数据库里创建master_slave库，并插入20条数据
[root@master-node ~]# mongo 182.48.115.238:27017
MongoDB shell version v3.4.4
connecting to: 182.48.115.238:27017
MongoDB server version: 3.4.4
Server has startup warnings:
2017-06-03T16:45:07.406+0800 I STORAGE  [initandlisten]
2017-06-03T16:45:07.407+0800 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2017-06-03T16:45:07.407+0800 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2017-06-03T16:45:08.373+0800 I CONTROL  [initandlisten]
2017-06-03T16:45:08.373+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-06-03T16:45:08.373+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2017-06-03T16:45:08.373+0800 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2017-06-03T16:45:08.373+0800 I CONTROL  [initandlisten]
2017-06-03T16:45:08.374+0800 I CONTROL  [initandlisten]
2017-06-03T16:45:08.374+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2017-06-03T16:45:08.374+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2017-06-03T16:45:08.374+0800 I CONTROL  [initandlisten]
2017-06-03T16:45:08.374+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2017-06-03T16:45:08.374+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2017-06-03T16:45:08.374+0800 I CONTROL  [initandlisten]
> use master_slave
switched to db master_slave
> function add(){var i = 0;for(;i<20;i++){db.persons.insert({"name":"wang"+i})}}
> add()
> db.persons.find()
{ "_id" : ObjectId("593278699a9e2e9f37ac4dbb"), "name" : "wang0" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dbc"), "name" : "wang1" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dbd"), "name" : "wang2" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dbe"), "name" : "wang3" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dbf"), "name" : "wang4" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc0"), "name" : "wang5" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc1"), "name" : "wang6" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc2"), "name" : "wang7" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc3"), "name" : "wang8" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc4"), "name" : "wang9" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc5"), "name" : "wang10" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc6"), "name" : "wang11" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc7"), "name" : "wang12" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc8"), "name" : "wang13" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc9"), "name" : "wang14" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dca"), "name" : "wang15" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dcb"), "name" : "wang16" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dcc"), "name" : "wang17" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dcd"), "name" : "wang18" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dce"), "name" : "wang19" }
Type "it" for more
 
然后在slave-node节点数据库里查看，是否将master-node写入的新数据同步过来了
[root@slave-node log]# mongo 182.48.115.236:27017
MongoDB shell version v3.4.4
connecting to: 182.48.115.236:27017
MongoDB server version: 3.4.4
Server has startup warnings:
2017-06-03T16:56:28.928+0800 I STORAGE  [initandlisten]
2017-06-03T16:56:28.928+0800 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2017-06-03T16:56:28.928+0800 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2017-06-03T16:56:29.883+0800 I CONTROL  [initandlisten]
2017-06-03T16:56:29.883+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-06-03T16:56:29.883+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2017-06-03T16:56:29.883+0800 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2017-06-03T16:56:29.883+0800 I CONTROL  [initandlisten]
2017-06-03T16:56:29.884+0800 I CONTROL  [initandlisten]
2017-06-03T16:56:29.884+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2017-06-03T16:56:29.884+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2017-06-03T16:56:29.884+0800 I CONTROL  [initandlisten]
2017-06-03T16:56:29.884+0800 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2017-06-03T16:56:29.884+0800 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2017-06-03T16:56:29.884+0800 I CONTROL  [initandlisten]
> show dbs
2017-06-03T17:10:32.136+0800 E QUERY    [thread1] Error: listDatabases failed:{
    "ok" : 0,
    "errmsg" : "not master and slaveOk=false",
    "code" : 13435,
    "codeName" : "NotMasterNoSlaveOk"
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
Mongo.prototype.getDBs@src/mongo/shell/mongo.js:62:1
shellHelper.show@src/mongo/shell/utils.js:769:19
shellHelper@src/mongo/shell/utils.js:659:15
@(shellhelp2):1:1
> rs.slaveOk();
> show dbs
admin         0.000GB
local         0.000GB
master_slave  0.000GB
wangshibo     0.000GB
> use master_slave
switched to db master_slave
> db.persons.find()
{ "_id" : ObjectId("593278699a9e2e9f37ac4dbb"), "name" : "wang0" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dbc"), "name" : "wang1" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dbd"), "name" : "wang2" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dbe"), "name" : "wang3" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dbf"), "name" : "wang4" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc0"), "name" : "wang5" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc1"), "name" : "wang6" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc2"), "name" : "wang7" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc3"), "name" : "wang8" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc4"), "name" : "wang9" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc5"), "name" : "wang10" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc6"), "name" : "wang11" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc7"), "name" : "wang12" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc8"), "name" : "wang13" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dc9"), "name" : "wang14" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dca"), "name" : "wang15" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dcb"), "name" : "wang16" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dcc"), "name" : "wang17" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dcd"), "name" : "wang18" }
{ "_id" : ObjectId("593278699a9e2e9f37ac4dce"), "name" : "wang19" }
Type "it" for more
........................................................................................
如果在slave-node节点上的数据库中查看，有报错："errmsg" : "not master and slaveOk=false"！！！
首先这是正常的，因为SECONDARY是不允许读写的， 在写多读少的应用中，使用Replica Sets来实现读写分离。
通过在连接时指定或者在主库指定slaveOk，由Secondary来分担读的压力，Primary只承担写操作。
对于replica set 中的secondary 节点默认是不可读的。
 
解决办法：
在slave-node节点数据库中执行"rs.slaveOk();"命令即可
........................................................................................
 
在slave-node节点数据库中发现已经同步过来了master_slave库的20条数据，说明mongodb的主从复制环境已经成功了！当配置完主从服务器后，一但主服务器上的数据发生变化，从服务器也会发生变化

主从复制的原理

在主从结构中，主节点的操作记录成为oplog（operation log）。oplog存储在一个系统数据库local的集合oplog.$main中，这个集合的每个文档都代表主节点上执行的一个操作。
从服务器会定期从主服务器中获取oplog记录，然后在本机上执行！对于存储oplog的集合，MongoDB采用的是固定集合，也就是说随着操作过多，新的操作会覆盖旧的操作！
 
主从复制的其他设置项
--only             从节点指定复制某个数据库,默认是复制全部数据库
--slavedelay       从节点设置主数据库同步数据的延迟(单位是秒)
--fastsync         从节点以主数据库的节点快照为节点启动从数据库
--autoresync       从节点如果不同步则从新同步数据库(即选择当通过热添加了一台从服务器之后，从服务器选择是否更新主服务器之间的数据)
--oplogSize        主节点设置oplog的大小(主节点操作记录存储到local的oplog中)

在上面slave-node从节点的local数据库中，存在一个集合sources。这个集合就保存了这个服务器的主服务器是谁

[root@slave-node mongodb]# mongo 182.48.115.236:27017
......
> show dbs
admin         0.000GB
local         0.000GB
master_slave  0.000GB
wangshibo     0.000GB
> use local
switched to db local
> show collections
me
sources
startup_log
> db.sources.find()
{ "_id" : ObjectId("593277a5105051e5648605a3"), "host" : "182.48.115.238:27017", "source" : "main", "syncedTo" : Timestamp(1496481652, 1) }
>