21.40 mongodb分片测试

于 2017-10-20 18:02:00 发布
阅读量268 收藏 1
点赞数 1
文章标签： 数据库 json python
原文链接：https://my.oschina.net/u/3651233/blog/1554068
版权

2019独角兽企业重金招聘Python工程师标准>>> hot3.png

21.40 mongodb分片测试

分片搭建–测试

  • 登录任何一台20000端口
    mongo --port 20000
    use admin
    db.runCommand({ enablesharding : "testdb"}) 或者
    sh.enableSharding("testdb") //指定要分片的数据库
    db.runCommand( { shardcollection : "testdb.table1",key : {id: 1} } ) 或者
    sh.shardCollection("testdb.table1",{"id":1} ) //#指定数据库里需要分片的集合和片键
    use testdb
    for (var i = 1; i <= 10000; i++) db.table1.save({id:i,"test1":"testval1"})//插入测试数据
    db.table1.stats()//查看table1状态
[root@Dasoncheng ~]# mongo --port 20000
MongoDB shell version v3.4.9
connecting to: mongodb://127.0.0.1:20000/
MongoDB server version: 3.4.9
mongos> use admin
switched to db admin
mongos> sh.enableSharding("testdb")     
##指定要分片的数据库testdb，没有回自动创建；
{ "ok" : 1 }
mongos> sh.shardCollection("testdb.table1",{"id":1} )    
##指定数据库里需要分片的集合和片键
{ "collectionsharded" : "testdb.table1", "ok" : 1 }
mongos> use  testdb
switched to db testdb
##下面插入一条数据；
mongos> for (var i = 1; i <= 10000; i++) db.table1.save({id:i,"test1":"testval1"})

WriteResult({ "nInserted" : 1 })
mongos> show dbs
admin   0.000GB
config  0.001GB
testdb  0.000GB
mongos> sh.status()
--- Sharding Status --- 
  sharding version: {
	"_id" : 1,
	"minCompatibleVersion" : 5,
	"currentVersion" : 6,
	"clusterId" : ObjectId("59e957b4b1833892fcc37ec7")
}
  shards:
	{  "_id" : "shard1",  "host" : "shard1/192.168.60.11:27001,192.168.60.12:27001",  "state" : 1 }
	{  "_id" : "shard2",  "host" : "shard2/192.168.60.12:27002,192.168.60.13:27002",  "state" : 1 }
	{  "_id" : "shard3",  "host" : "shard3/192.168.60.11:27003,192.168.60.13:27003",  "state" : 1 }
  active mongoses:
	"3.4.9" : 1
 autosplit:
	Currently enabled: yes
  balancer:
	Currently enabled:  yes
	Currently running:  no
		Balancer lock taken at Fri Oct 20 2017 14:44:12 GMT+0800 (CST) by ConfigServer:Balancer
	Failed balancer rounds in last 5 attempts:  0
	Migration Results for the last 24 hours: 
		No recent migrations
  databases:
	{  "_id" : "testdb",  "primary" : "shard1",  "partitioned" : true }
		testdb.table1
			shard key: { "id" : 1 }
			unique: false
			balancing: true
			chunks:
				shard1	1  ##看这里，存在shard1里面
			{ "id" : { "$minKey" : 1 } } -->> { "id" : { "$maxKey" : 1 } } on : shard1 Timestamp(1, 0) 
mongos> sh.enableSharding("db2")  ##再创建库和表，查看存在哪个shard里面；
{ "ok" : 1 }
mongos> sh.shardCollection("db2.cl2",{"id":1} )
{ "collectionsharded" : "db2.cl2", "ok" : 1 }
mongos> sh.status()
--- Sharding Status --- 
  sharding version: {
	"_id" : 1,
	"minCompatibleVersion" : 5,
	"currentVersion" : 6,
	"clusterId" : ObjectId("59e957b4b1833892fcc37ec7")
}
  shards:
	{  "_id" : "shard1",  "host" : "shard1/192.168.60.11:27001,192.168.60.12:27001",  "state" : 1 }
	{  "_id" : "shard2",  "host" : "shard2/192.168.60.12:27002,192.168.60.13:27002",  "state" : 1 }
	{  "_id" : "shard3",  "host" : "shard3/192.168.60.11:27003,192.168.60.13:27003",  "state" : 1 }
  active mongoses:
	"3.4.9" : 1
 autosplit:
	Currently enabled: yes
  balancer:
	Currently enabled:  yes
	Currently running:  no
		Balancer lock taken at Fri Oct 20 2017 14:44:12 GMT+0800 (CST) by ConfigServer:Balancer
	Failed balancer rounds in last 5 attempts:  0
	Migration Results for the last 24 hours: 
		No recent migrations
  databases:
	{  "_id" : "testdb",  "primary" : "shard1",  "partitioned" : true }
		testdb.table1
			shard key: { "id" : 1 }
			unique: false
			balancing: true
			chunks:
				shard1	1
			{ "id" : { "$minKey" : 1 } } -->> { "id" : { "$maxKey" : 1 } } on : shard1 Timestamp(1, 0) 
	{  "_id" : "db2",  "primary" : "shard2",  "partitioned" : true }
		db2.cl2
			shard key: { "id" : 1 }
			unique: false
			balancing: true
			chunks:
				shard2	1    ##看这里，存在shard2里面
			{ "id" : { "$minKey" : 1 } } -->> { "id" : { "$maxKey" : 1 } } on : shard2 Timestamp(1, 0) 
##分片分布在各个shard里面，测试成功；

21.41 mongodb备份恢复

  • 备份所有库
    mongodump --host 127.0.0.1 --port 20000 -o /tmp/mongobak/alldatabase
  • 备份指定库
    mongodump --host 127.0.0.1 --port 20000 -d mydb -o /tmp/mongobak
    它会在/tmp/目录下面生成一个mydb的目录
  • 指定备份集合
    mongodump --host 127.0.0.1 --port 20000 -d mydb -c c1 -o /tmp/mongobak/
    它依然会生成mydb目录，再在这目录下面生成两个文件
  • 导出集合为json文件
    mongoexport --host 127.0.0.1 --port 20000 -d mydb -c c1 -o /tmp/mydb2/1.json
##备份所有库：
[root@Dasoncheng ~]# mongodump --host 127.0.0.1 --port 20000 -o /tmp/mongobak/alldatabase  
##备份所有库；-o指定文件夹，如果没有回自动创建
2017-10-20T17:00:15.154+0800	writing admin.system.version to 
2017-10-20T17:00:15.160+0800	done dumping admin.system.version (1 document)
2017-10-20T17:00:15.160+0800	writing testdb.table1 to 
2017-10-20T17:00:15.160+0800	writing config.lockpings to 
2017-10-20T17:00:15.161+0800	writing config.changelog to 
2017-10-20T17:00:15.161+0800	writing config.locks to 
2017-10-20T17:00:15.169+0800	done dumping config.lockpings (13 documents)
2017-10-20T17:00:15.169+0800	writing config.shards to 
2017-10-20T17:00:15.178+0800	done dumping config.shards (3 documents)
2017-10-20T17:00:15.178+0800	writing config.collections to 
2017-10-20T17:00:15.192+0800	done dumping config.collections (2 documents)
2017-10-20T17:00:15.192+0800	writing config.databases to 
2017-10-20T17:00:15.195+0800	done dumping config.changelog (7 documents)
2017-10-20T17:00:15.195+0800	writing config.chunks to 
2017-10-20T17:00:15.203+0800	done dumping config.locks (5 documents)
2017-10-20T17:00:15.203+0800	writing config.mongos to 
2017-10-20T17:00:15.203+0800	done dumping config.chunks (2 documents)
2017-10-20T17:00:15.203+0800	writing config.version to 
2017-10-20T17:00:15.206+0800	done dumping config.version (1 document)
2017-10-20T17:00:15.206+0800	writing config.tags to 
2017-10-20T17:00:15.209+0800	done dumping config.tags (0 documents)
2017-10-20T17:00:15.209+0800	writing config.migrations to 
2017-10-20T17:00:15.213+0800	done dumping config.migrations (0 documents)
2017-10-20T17:00:15.213+0800	writing db2.cl2 to 
2017-10-20T17:00:15.215+0800	done dumping config.databases (2 documents)
2017-10-20T17:00:15.226+0800	done dumping config.mongos (2 documents)
2017-10-20T17:00:15.227+0800	done dumping db2.cl2 (0 documents)
2017-10-20T17:00:15.260+0800	done dumping testdb.table1 (10000 documents)
[root@Dasoncheng ~]# ll /tmp/mongobak/alldatabase/
total 4
drwxr-xr-x 2 root root   69 Oct 20 17:00 admin
drwxr-xr-x 2 root root 4096 Oct 20 17:00 config
drwxr-xr-x 2 root root   47 Oct 20 17:00 db2
drwxr-xr-x 2 root root   53 Oct 20 17:00 testdb
[root@Dasoncheng ~]# ll /tmp/mongobak/alldatabase/testdb/
total 532
-rw-r--r-- 1 root root 540000 Oct 20 17:00 table1.bson  
##这个文件是主要数据.bson
-rw-r--r-- 1 root root    145 Oct 20 17:00 table1.metadata.json
##这个文件是建表信息.json

##备份指定库
[root@Dasoncheng ~]# mongodump --host 127.0.0.1 --port 20000 -d testdb -o /tmp/mongobak  
##-d指定库；-o指定备份文件夹 没有则自动创建；
2017-10-20T16:56:15.484+0800	writing testdb.table1 to 
2017-10-20T16:56:15.633+0800	done dumping testdb.table1 (10000 documents)
[root@Dasoncheng ~]# ll /tmp/mongobak/testdb/  ##因为这个库只有一个集合testdb，所以
total 532
-rw-r--r-- 1 root root 540000 Oct 20 16:56 table1.bson
-rw-r--r-- 1 root root    145 Oct 20 16:56 table1.metadata.json

##指定备份集合
[root@Dasoncheng ~]# mongodump --host 127.0.0.1 --port 20000 -d testdb -c table1 -o /tmp/mongobak2
2017-10-20T16:59:03.087+0800	writing testdb.table1 to 
2017-10-20T16:59:03.166+0800	done dumping testdb.table1 (10000 documents)
[root@Dasoncheng ~]# ll /tmp/mongobak2/testdb/
total 532
-rw-r--r-- 1 root root 540000 Oct 20 16:59 table1.bson
-rw-r--r-- 1 root root    145 Oct 20 16:59 table1.metadata.json

##导出集合为json文件
[root@Dasoncheng ~]# mongoexport --host 127.0.0.1 --port 20000 -d testdb -c table1 -o /tmp/testdb/1.json
2017-10-20T17:02:09.210+0800	connected to: 127.0.0.1:20000
2017-10-20T17:02:09.593+0800	exported 10000 records
[root@Dasoncheng ~]# ll /tmp/testdb/1.json 
-rw-r--r-- 1 root root 748894 Oct 20 17:02 /tmp/testdb/1.json
[root@Dasoncheng ~]# tail /tmp/testdb/1.json 
{"_id":{"$oid":"59e9b0bb4bdfc034c89123b2"},"id":9991.0,"test1":"testval1"}
{"_id":{"$oid":"59e9b0bb4bdfc034c89123b3"},"id":9992.0,"test1":"testval1"}
{"_id":{"$oid":"59e9b0bb4bdfc034c89123b4"},"id":9993.0,"test1":"testval1"}
{"_id":{"$oid":"59e9b0bb4bdfc034c89123b5"},"id":9994.0,"test1":"testval1"}
{"_id":{"$oid":"59e9b0bb4bdfc034c89123b6"},"id":9995.0,"test1":"testval1"}
{"_id":{"$oid":"59e9b0bb4bdfc034c89123b7"},"id":9996.0,"test1":"testval1"}
{"_id":{"$oid":"59e9b0bb4bdfc034c89123b8"},"id":9997.0,"test1":"testval1"}
{"_id":{"$oid":"59e9b0bb4bdfc034c89123b9"},"id":9998.0,"test1":"testval1"}
{"_id":{"$oid":"59e9b0bb4bdfc034c89123ba"},"id":9999.0,"test1":"testval1"}
{"_id":{"$oid":"59e9b0bb4bdfc034c89123bb"},"id":10000.0,"test1":"testval1"}

MongoDB恢复：

  • 恢复所有库
    mongorestore -h 127.0.0.1 --port 20000 --drop dir/ //其中dir是备份所有库的目录名字，其中--drop可选，意思是当恢复之前先把之前的数据删除，不建议使用
  • 恢复指定库
    mongorestore -d mydb dir/ //-d跟要恢复的库名字，dir就是该库备份时所在的目录
  • 恢复集合
    mongorestore -d mydb -c testc dir/mydb/testc.bson // -c后面跟要恢复的集合名字，dir是备份mydb库时生成文件所在路径，这里是一个bson文件的路径
  • 导入集合
    mongoimport -d mydb -c testc --file /tmp/testc.json
##回复所有库；
[root@Dasoncheng ~]# ls /tmp/mongobak/alldatabase/   ##备份时指定的文件夹；
admin  config  db2  testdb
[root@Dasoncheng ~]# mongo --port 20000
MongoDB shell version v3.4.9
connecting to: mongodb://127.0.0.1:20000/
MongoDB server version: 3.4.9
mongos> show dbs
admin   0.000GB
config  0.001GB
db2     0.000GB
testdb  0.000GB
mongos> use db2
switched to db db2
mongos> db.dropDatabase()
{ "dropped" : "db2", "ok" : 1 }
mongos> use testdb
switched to db testdb
mongos> db.dropDatabase()
{ "dropped" : "testdb", "ok" : 1 }
mongos> show dbs
admin   0.000GB
config  0.001GB
[root@Dasoncheng ~]# mongorestore -h 127.0.0.1 --port 20000 /tmp/mongobak/alldatabase  
##还原的时候使用备份的文件夹即可；
2017-10-20T17:50:44.598+0800	preparing collections to restore from
2017-10-20T17:50:44.599+0800	Failed: cannot do a full restore on a sharded system - remove the 'config' directory from the dump directory first  
##这里报错，config库不能还原（config库很重要 不能随便还原），提示删除 我们接下来删除；
[root@Dasoncheng ~]# rm -rf /tmp/mongobak/alldatabase/config/
[root@Dasoncheng ~]# mongorestore -h 127.0.0.1 --port 20000 /tmp/mongobak/alldatabase
2017-10-20T17:51:20.512+0800	preparing collections to restore from
2017-10-20T17:51:20.517+0800	reading metadata for testdb.table1 from /tmp/mongobak/alldatabase/testdb/table1.metadata.json
2017-10-20T17:51:20.522+0800	reading metadata for db2.cl2 from /tmp/mongobak/alldatabase/db2/cl2.metadata.json
2017-10-20T17:51:20.587+0800	restoring testdb.table1 from /tmp/mongobak/alldatabase/testdb/table1.bson
2017-10-20T17:51:20.604+0800	restoring db2.cl2 from /tmp/mongobak/alldatabase/db2/cl2.bson
2017-10-20T17:51:20.616+0800	restoring indexes for collection db2.cl2 from metadata
2017-10-20T17:51:20.644+0800	finished restoring db2.cl2 (0 documents)
2017-10-20T17:51:21.876+0800	restoring indexes for collection testdb.table1 from metadata
2017-10-20T17:51:22.105+0800	finished restoring testdb.table1 (10000 documents)
2017-10-20T17:51:22.105+0800	done
##还原成功，查看还原后的库：
[root@Dasoncheng ~]# mongo --port 20000
mongos> show dbs
admin   0.000GB
config  0.001GB
db2     0.000GB
testdb  0.000GB
##老铁，没毛病；接下来恢复库 以及集合

##恢复指定库；
[root@Dasoncheng ~]# ls /tmp/mongobak/testdb/
table1.bson  table1.metadata.json
[root@Dasoncheng ~]# mongorestore -d testdb /tmp/mongobak/testdb/
##指定包含库文件的目录就行；
2017-10-20T17:57:21.077+0800	the --db and --collection args should only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead
2017-10-20T17:57:21.078+0800	building a list of collections to restore from /tmp/mongobak/testdb dir
2017-10-20T17:57:21.081+0800	reading metadata for testdb.table1 from /tmp/mongobak/testdb/table1.metadata.json
2017-10-20T17:57:21.104+0800	restoring testdb.table1 from /tmp/mongobak/testdb/table1.bson
2017-10-20T17:57:22.389+0800	restoring indexes for collection testdb.table1 from metadata
2017-10-20T17:57:22.474+0800	finished restoring testdb.table1 (10000 documents)
2017-10-20T17:57:22.474+0800	done
##还原库没毛病！

##恢复集合；
[root@Dasoncheng ~]# mongorestore -d testdb -c table1 /tmp/mongobak/testdb/table1.bson
……
2017-10-20T18:00:05.663+0800	restoring indexes for collection testdb.table1 from metadata
2017-10-20T18:00:05.673+0800	finished restoring testdb.table1 (10000 documents)
2017-10-20T18:00:05.673+0800	done

转载于:https://my.oschina.net/u/3651233/blog/1554068

weixin_34033624
关注
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值