实际环境架构
分别在3台机器运行一个mongod实例（称为mongod shard11，mongod shard12，mongod shard13）组织replica set1，作为cluster的shard1
分别在3台机器运行一个mongod实例（称为mongod shard21，mongod shard22，mongod shard23）组织replica set2，作为cluster的shard2
每台机器运行一个mongod实例，作为3个config server
每台机器运行一个mongs进程，用于客户端连接
-------------------------------------------------------------------------
Server1 10.3.0.100
Mongod shard11:27017
Mongod shard21:17017
Mongod config1:20000
Mongos1:20001
-------------------------------------------------------------------------
Server1 10.3.0.101
Mongod shard12:27017
Mongod shard22:17017
Mongod config1:20000
Mongos1:20001
--------------------------------------------------------------------------
Server3 10.3.0.102
Mongod shard13:27017
Mongod shard23:17017
Mongod config1:20000
Mongos1:20001
-------------------------------------------------------------------------
1. 创建用户 mongodb (如果用root启用mongodb,可略此步)
useradd -u 1001 mongodb
2. 下载 mongodb-linux-x86_64-2.2.0
tar zxvf mongodb-linux-x86_64-2.2.0.tgz
mv mongodb-linux-x86_64-2.2.0 /opt/local/mongodb
3. 创建 数据目录 日志目录 配置目录 (server 1)
mkdir -p /opt/local/mongodb/data/shard
mkdir -p /opt/local/mongodb/data/logs
mkdir -p /opt/local/mongodb/data/config
4. 创建 数据目录 日志目录 配置目录 (server 2)
mkdir -p /opt/local/mongodb/data/shard12
mkdir -p /opt/local/mongodb/data/shard22
mkdir -p /opt/local/mongodb/data/logs
mkdir -p /opt/local/mongodb/data/config
5. 创建 数据目录 日志目录 配置目录 (server 3)
mkdir -p /opt/local/mongodb/data/shard13
mkdir -p /opt/local/mongodb/data/shard23
mkdir -p /opt/local/mongodb/data/logs
mkdir -p /opt/local/mongodb/data/config
6. 修改 目录 所有者
chown -R mongodb:mongodb /opt/local/mongodb
7. 配置relica sets
7.1.1 配置shard1所用到的replica sets:
--------------------------------------------------------------------------------------
port=27017 #端口号
fork=true #以守护进程的方式运行，创建服务器进程
logpath=/opt/local/mongodb/data/logs/shard.log #日志输出文件路径
logappend=true #日志输出方式
dbpath=/opt/local/mongodb/data/shard/ #数据库路径
shardsvr=true #设置是否分片
maxConns=10000 #数据库的最大连接数
replSet=shard1 #设置副本集名称
oplogSize=5000 #设置oplog的大小(MB)
----------------------------------------------------------------------------------------
server 1
/opt/local/mongodb/bin/mongod --shardsvr --maxConns 10000 --replSet shard1 --port 27017 --dbpath /opt/local/mongodb/data/shard11/ --oplogSize 500 --logpath=/opt/local/mongodb/data/logs/shard11.log --logappend --fork
server 2
/opt/local/mongodb/bin/mongod --shardsvr --maxConns 10000 --replSet shard1 --port 27017 --dbpath /opt/local/mongodb/data/shard12/ --oplogSize 500 --logpath=/opt/local/mongodb/data/logs/shard12.log --logappend --fork
server 3
/opt/local/mongodb/bin/mongod --shardsvr --maxConns 10000 --replSet shard1 --port 27017 --dbpath /opt/local/mongodb/data/shard13/ --oplogSize 500 --logpath=/opt/local/mongodb/data/logs/shard13.log --logappend --fork
7.1.2 初始化replica set
用mongo连接其中一个mongod，执行:
/opt/local/mongodb/bin/mongo --host 10.3.0.100:27017
----------------------------------------------------------------------
> config= {_id: 'shard1', members: [ {_id:0,host:'10.3.0.100:27017'},
... {_id:1,host:'10.3.0.101:27017'},
... {_id:2,host:'10.3.0.102:27017'},]
... }
{
"_id" : "shard1",
"members" : [
{
"_id" : 0,
"host" : "10.3.0.100:27017"
},
{
"_id" : 1,
"host" : "10.3.0.101:27017"
},
{
"_id" : 2,
"host" : "10.3.0.102:27017"
}
]
}
> rs.initiate(config);
{
"info" : "Config now saved locally. Should come online in about a minute.",
"ok" : 1
}
-------------------------------------------------------------------------
7.2.1 配置shard2所用到的replica sets:
/opt/local/mongodb/bin/mongod --shardsvr --maxConns 10000 --replSet shard2 --port 17017 --dbpath /opt/local/mongodb/data/shard21/ --oplogSize 100 --logpath=/opt/local/mongodb/data/logs/shard21.log --logappend --fork
server 2
/opt/local/mongodb/bin/mongod --shardsvr --maxConns 10000 --replSet shard2 --port 17017 --dbpath /opt/local/mongodb/data/shard22/ --oplogSize 100 --logpath=/opt/local/mongodb/data/logs/shard22.log --logappend --fork
server 3
/opt/local/mongodb/bin/mongod --shardsvr --maxConns 10000 --replSet shard2 --port 17017 --dbpath /opt/local/mongodb/data/shard23/ --oplogSize 100 --logpath=/opt/local/mongodb/data/logs/shard23.log --logappend --fork
7.2.2 初始化replica set
用mongo连接其中一个mongod，执行:
/opt/local/mongodb/bin/mongo --host 10.3.0.100:17017
-------------------------------------------------------------------------
> config= {_id: 'shard2', members: [ {_id:0,host:'10.3.0.100:17017'},
... {_id:1,host:'10.3.0.101:17017'},
... {_id:2,host:'10.3.0.102:17017'},]
... }
{
"_id" : "shard2",
"members" : [
{
"_id" : 0,
"host" : "10.3.0.100:17017"
},
{
"_id" : 1,
"host" : "10.3.0.101:17017"
},
{
"_id" : 2,
"host" : "10.3.0.102:17017"
}
]
}
> rs.initiate(config);
{
"errmsg" : "couldn't initiate : set name does not match the set name host 10.3.0.101:17017 expects",
"ok" : 0
}
>
-------------------------------------------------------------------------
8. 配置三台config server
server 1
/opt/local/mongodb/bin/mongod --configsvr --dbpath /opt/local/mongodb/data/config/ --port 20000 --logpath /opt/local/mongodb/data/logs/config.log --logappend --fork
server 2
/opt/local/mongodb/bin/mongod --configsvr --dbpath /opt/local/mongodb/data/config/ --port 20000 --logpath /opt/local/mongodb/data/logs/config.log --logappend --fork
server 3
/opt/local/mongodb/bin/mongod --configsvr --dbpath /opt/local/mongodb/data/config/ --port 20000 --logpath /opt/local/mongodb/data/logs/config.log --logappend --fork
9. 配置 mongs
在server1,server2,server3 上分别执行：
/opt/local/mongodb/bin/mongos --configdb 10.3.0.100:20000,10.3.0.101:20000,10.3.0.102:20000 --port 20001 --chunkSize 5 --logpath /opt/local/mongodb/data/logs/mongos.log --logappend --fork
#mongs不需要dbpath
10 Configuring the Shard Cluster
连接到其中一个mongos进程，并切换到admin数据库做以下配置
10.1 连接到mongs，并切换到admin
/opt/local/mongodb/bin/mongo 10.3.0.100:20001/admin
MongoDB shell version: 2.2.0
connecting to: 10.3.0.100:20001/admin
mongos> db
admin
10.2 加入shards
如里shard是单台服务器，
用>db.runCommand( { addshard : “<serverhostname>[:<port>]” } )加入
如果shard是replica sets，用replicaSetName/<serverhostname>[:port]
----------------------------------------------------------------------
shard 1
db.runCommand( { addshard : "shard1/10.3.0.100:27017,10.3.0.101:27017,10.3.0.102:27017", maxsize:204800});
{ "shardAdded" : "shard1", "ok" : 1 }
--------------------------------------------------------------------
shard 2
mongos> db.runCommand( { addshard : "shard2/10.3.0.100:17017,10.3.0.101:17017,10.3.0.102:17017", maxsize:204800});
{ "shardAdded" : "shard2", "ok" : 1 }
----------------------------------------------------------------------
查看是否添加成功
mongos> db.runCommand( { listshards : 1 } )
{
"shards" : [
{
"_id" : "shard1",
"host" : "shard1/10.3.0.100:27017,10.3.0.101:27017,10.3.0.102:27017"
},
{
"_id" : "shard2",
"host" : "shard2/10.3.0.100:17017,10.3.0.101:17017,10.3.0.102:17017"
}
],
"ok" : 1
}
删除 shards
db.runCommand( { removeshard : "shard1/10.3.0.100:27017,10.3.0.101:27017"} );
11. 激活分片
命令：
> db.runCommand( { enablesharding : “<dbname>” } );
通过执行以上命令，可以让数据库跨shard，如果不执行这步，数据库只会存放在一个shard，一旦激活数据库分片，数据库中不同的collection将被存放在不同的shard上，但一个collection仍旧存放在同一个shard上，要使单个collection也分片，
还需单独对collection作些操作
PS: 其他命令.....操作...
#增添节点:
以一个新建的mongodb服务为例,
连接 mongo
/opt/local/mongodb/bin/mongo --host 10.3.0.100:27017
MongoDB shell version: 2.2.0
connecting to: 10.3.0.100:27017/test
shard1:PRIMARY>rs.add(“IP:port”);
#删除节点：
必须在主节点上操作：
PRIMARY>rs.remove(“127.0.0.1:27020”)
查看同步状态
rs.status()
让从有读的权限：
在主控上执行:PRIMARY> db.getMongo().setSlaveOk();
SECONDARY> rs.slaveOk();
群集增加验证 auth keyFile
群集如果加了 auth 验证，群集之前互相取不到 主 就无法验证...必须要增加 keyFile 验证才行...
先创建auth 验证
-----------------------------------------------------------------
> use admin
switched to db admin
> db.addUser('sa','sa')
{
"_id" : ObjectId("4e2914a585178da4e03a16c3"),
"user" : "sa",
"readOnly" : false,
"pwd" : "75692b1d11c072c6c79332e248c4f699"
}
>
----------------------------------------------------------------------------------
然后在每个server里创建 key 文件....
vi /opt/local/mongodb/data/config/key
输入随意字符.... 1234567890111111111
授权 chown mongodb:mongodb key
chmod 600 key
-------------------------------------------------------------------------
然后分别启动
server 1
/opt/local/mongodb/bin/mongod --shardsvr --replSet shard1 --port 27017 --dbpath /opt/local/mongodb/data/shard11/ --oplogSize 100 --logpath=/opt/local/mongodb/data/logs/shard11.log --logappend --fork --keyFile /opt/local/mongodb/data/config/key
server 2
/opt/local/mongodb/bin/mongod --shardsvr --replSet shard1 --port 27017 --dbpath /opt/local/mongodb/data/shard12/ --oplogSize 100 --logpath=/opt/local/mongodb/data/logs/shard12.log --logappend --fork --keyFile /opt/local/mongodb/data/config/key
server 3
/opt/local/mongodb/bin/mongod --shardsvr --replSet shard1 --port 27017 --dbpath /opt/local/mongodb/data/shard13/ --oplogSize 100 --logpath=/opt/local/mongodb/data/logs/shard13.log --logappend --fork --keyFile /opt/local/mongodb/data/config/key
-------------------------------------------------------------------------
在启动config
server 1
/opt/local/mongodb/bin/mongod --configsvr --dbpath /opt/local/mongodb/data/config/ --port 20000 --logpath /opt/local/mongodb/data/logs/config.log --logappend --fork --keyFile /opt/local/mongodb/data/config/key
server 2
/opt/local/mongodb/bin/mongod --configsvr --dbpath /opt/local/mongodb/data/config/ --port 20000 --logpath /opt/local/mongodb/data/logs/config.log --logappend --fork --keyFile /opt/local/mongodb/data/config/key
server 3
/opt/local/mongodb/bin/mongod --configsvr --dbpath /opt/local/mongodb/data/config/ --port 20000 --logpath /opt/local/mongodb/data/logs/config.log --logappend --fork --keyFile /opt/local/mongodb/data/config/key
-------------------------------------------------------------------------
最后分别启动mongos
/opt/local/mongodb/bin/mongos --configdb 10.3.0.100:20000,10.3.0.101:20000,10.3.0.102:20000 --port 20001 --chunkSize 5 --logpath /opt/local/mongodb/data/logs/mongos.log --logappend --fork --keyFile /opt/local/mongodb/data/config/key
-------------------------------------------------------------------------
然后在查看是否能读取主....
./mongo
MongoDB shell version: 2.2.0
connecting to: test
> show dbs
Wed Sep 5 01:51:44 uncaught exception: listDatabases failed:{ "errmsg" : "need to login", "ok" : 0 }
> use admin
switched to db admin
> db.auth('sa','sa')
1
shard1:SECONDARY> db.serverStatus()
就可以查看....
注意:
添加auth 以后...导入数据 等一系列操作...都必须验证用户...
如：
/opt/local/mongodb/bin/mongorestore -u sa -p sa --drop /opt/1
其他都需要进行操作！否则！报错！！！
强制切换主
cfg = rs.conf()
cfg.members[0].priority = 0.5
cfg.members[1].priority = 0.5
cfg.members[2].priority = 1
rs.reconfig(cfg)
升级MongoDB分片集群到2.2版本
MongoDB 2.2版本与2.0版本在分片集群中共存存在问题，所以升级需要按照官方文档的步骤进行：
禁用Balancer
mongos> sh.setBalancerState(false)
首先升级所有的mongos
升级Config服务器,一次升级一台
升级一个分片的复制集
依次关闭并升级一个复制集的secondary
在Primary库上执行rs.stepDown()，强制下线。关闭Primary库并升级二进制程序。
当都升级完成后，连接mongos执行：sh.setBalancerState(true)恢复平衡器。
----------------------------------------------------------------------------------
WARNING: You are running on a NUMA machine.
We suggest launching mongod like this to avoid performance problems:
numactl --interleave=all mongod [other options]
sudo -u mongodb numactl --interleave=all /opt/local/mongodb/bin/mongod --shardsvr --maxConns 10000 --replSet shard1 --port 27017 --dbpath /opt/local/mongodb/data/shard16/ --oplogSize 1000 --logpath=/opt/local/mongodb/data/logs/shard16.log --logappend --fork
编辑内核 echo 0 > /proc/sys/vm/zone_reclaim_mode
----------------------------------------------------------------------------------
限制 mongodb 占用系统所有内存
ulimit -s 4096 && ulimit -m 31457280 && sudo -u mongodb numactl --interleave=all /opt/local/mongodb/bin/mongod --shardsvr --maxConns 10000 --replSet shard1 --port 27017 --dbpath /opt/local/mongodb/data/shard16/ --oplogSize 1000 --logpath=/opt/local/mongodb/data/logs/shard16.log --logappend --fork