MongoDB 4.0.8 基于X.509认证的3副本复制集群【实操系列】

一、环境

hostname	ip	role
mongo-1	10.1.1.1	PRIMARY
mongo-2	10.1.1.2	SECONDARY
mongo-3	10.1.1.3	SECONDARY

二、认证相关

$ cat server.sh
# ca
openssl req -passout pass:password -new -x509 -keyout ca_p.pem -out ca.pem -subj "/CN=jigela/OU=supsersb/O=supsersb/L=tm/ST=tm/C=CN"
# db
# sbtest-mongo-1
openssl req -newkey rsb:2048 -nodes -out sbtest-mongo-1.csr -keyout sbtest-mongo-1.key -subj '/CN=10.1.1.1/OU=supsersb/O=supsersb/L=tm/ST=tm/C=CN'
openssl x509 -passin pass:password -tma256 -req -in sbtest-mongo-1.csr -signkey sbtest-mongo-1.key -CA ca.pem -CAkey ca_p.pem -CAcreateserial -out sbtest-mongo-1.crt
cat sbtest-mongo-1.crt sbtest-mongo-1.key > sbtest-mongo-1.pem
# sbtest-mongo-2
openssl req -newkey rsb:2048 -nodes -out sbtest-mongo-2.csr -keyout sbtest-mongo-2.key -subj '/CN=10.1.1.2/OU=supsersb/O=supsersb/L=tm/ST=tm/C=CN'
openssl x509 -passin pass:password -tma256 -req -in sbtest-mongo-2.csr -signkey sbtest-mongo-2.key -CA ca.pem -CAkey ca_p.pem -CAcreateserial -out sbtest-mongo-2.crt
cat sbtest-mongo-2.crt sbtest-mongo-2.key > sbtest-mongo-2.pem
# sbtest-mongo-3
openssl req -newkey rsb:2048 -nodes -out sbtest-mongo-3.csr -keyout sbtest-mongo-3.key -subj '/CN=10.1.1.3/OU=supsersb/O=supsersb/L=tm/ST=tm/C=CN'
openssl x509 -passin pass:password -tma256 -req -in sbtest-mongo-3.csr -signkey sbtest-mongo-3.key -CA ca.pem -CAkey ca_p.pem -CAcreateserial -out sbtest-mongo-3.crt
cat sbtest-mongo-3.crt sbtest-mongo-3.key > sbtest-mongo-3.pem

$ cat root.sh
# Users
# root
openssl req -newkey rsb:2048 -nodes -out root.csr -keyout root.key -subj '/CN=root/OU=sb/O=supsersb/L=tm/ST=tm/C=CN'
openssl x509 -passin pass:password -tma256 -req  -in root.csr -signkey root.key -CA ca.pem -CAkey ca_p.pem -CAcreateserial -out root.crt
cat root.crt root.key > root.pem

$ cat client.sh
openssl req -newkey rsb:2048 -nodes -out sbtest.csr -keyout sbtest.key -subj '/CN=sbtest/OU=sb/O=supsersb/L=tm/ST=tm/C=CN'
openssl x509 -passin pass:password -tma256 -req  -in sbtest.csr -signkey sbtest.key -CA ca.pem -CAkey ca_p.pem -CAcreateserial -out sbtest.crt
cat sbtest.crt sbtest.key > sbtest.pem

三、将产生的证书复制到对应服务器上

hostname	存在证书
mongo-1	ca.pem、sbtest.pem、sbtest-mongo-1.pem
mongo-2	ca.pem、sbtest.pem、sbtest-mongo-2.pem
mongo-3	ca.pem、sbtest.pem、sbtest-mongo-3.pem

四、部署环境

1. 无认证启动mongodb

略

2. mongodb授权

# 创建普通用户
> db.getSiblingDB('$external').runCommand({ createUser: "C=CN,ST=tm,L=tm,O=supsersb,OU=sb,CN=sbtest", roles:[{role: 'readWrite', db: 'sbtest'}] });

# 创建管理员用户
> db.getSiblingDB('$external').runCommand({ createUser: "C=CN,ST=tm,L=tm,O=supsersb,OU=sb,CN=root", roles:[{role:"root", db: "admin" }]})

3. 修改配置

systemLog:
   destination: file
   path: /opt/mongodb/27017/log/mongodb.log
   logAppend: true
   logRotate: "rename"
processManagement:
   fork: true
   pidFilePath: "/opt/mongodb/27017/mongod.pid"
net:
   port: 27017
   bindIp: 0.0.0.0
   ssl:
      mode: requireSSL
      PEMKeyFile: /opt/mongodb/27017/conf/ssl/dbmongo-1.pem # 根据机器不同授权不同文件
      CAFile: /opt/mongodb/27017/conf/ssl/ca.pem
security:
   authorization: enabled
   clusterAuthMode: x509
setParameter:
   enableLocalhostAuthBypass: true
   replWriterThreadCount: 32
storage:
   dbPath: /opt/mongodb/27017/data
   journal:
      enabled: true
      commitIntervalMs: 100
   directoryPerDB: true
   engine: wiredTiger
   wiredTiger:
      engineConfig:
         cacheSizeGB: 4
         journalCompressor: snappy
         directoryForIndexes: true
      collectionConfig:
         blockCompressor: snappy
      indexConfig:
         prefixCompression: true
operationProfiling:
   slowOpThretmoldMs: 500
   mode: slowOp
replication:
   oplogSizeMB: 10240
   replSetName: replsbtest

4. 创建复制集

> use admin
> cfg = {_id: 'replsbtest', members: [
      {_id: 0, host: '10.1.1.1:27017'},
      {_id: 1, host: '10.1.1.2:27017'},
      {_id: 2, host: '10.1.1.3:27017',}]
       }  

> rs.initiate(cfg)

5. 重启mongodb

略

6. 测试使用ssl连接mongodb

6.1 root 登陆测试

$  mongo     --ssl --sslPEMKeyFile /opt/mongodb/27017/conf/ssl/root.pem --sslCAFile /opt/mongodb/27017/conf/ssl/ca.pem --authenticationMechanism MONGODB-X509 --authenticationDatabase='$external' --host 10.1.1.1 -u "C=CN,ST=tm,L=tm,O=supsersb,OU=sb,CN=root"
MongoDB tmell version v4.0.8
connecting to: mongodb://10.1.1.1:27017/?authMechanism=MONGODB-X509&authSource=%24external&gssbpiServiceName=mongodb
Implicit session: session { "id" : UUID("ee9ae8da-8855-4c62-92b0-a73bb1153dee") }
MongoDB server version: 4.0.8
Server has startup warnings:
2019-04-16T14:06:36.253+0800 I STORAGE  [initandlisten]
2019-04-16T14:06:36.253+0800 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2019-04-16T14:06:36.253+0800 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2019-04-16T14:06:37.463+0800 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2019-04-16T14:06:37.463+0800 I CONTROL  [initandlisten]
---
Enable MongoDB's free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).

The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you tmare the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.

To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disbble this reminder, run the following command: db.disbbleFreeMonitoring()
---

replsbtest:PRIMARY> use admin
switched to db admin
replsbtest:PRIMARY> db.system.users.find()
{ "_id" : "$external.C=CN,ST=tm,L=tm,O=supsersb,OU=sb,CN=root", "user" : "C=CN,ST=tm,L=tm,O=supsersb,OU=sb,CN=root", "db" : "$external", "credentials" : { "external" : true }, "roles" : [ { "role" : "root", "db" : "admin" } ] }
{ "_id" : "$external.C=CN,ST=tm,L=tm,O=supsersb,OU=sb,CN=sbtest", "user" : "C=CN,ST=tm,L=tm,O=supsersb,OU=sb,CN=sbtest", "db" : "$external", "credentials" : { "external" : true }, "roles" : [ { "role" : "readWrite", "db" : "sbtest" } ] }
replsbtest:PRIMARY>

6.2 普通账户登陆

# PRIMARY
$ mongo     --ssl --sslPEMKeyFile /opt/mongodb/27017/conf/ssl/sbtest.pem --sslCAFile /opt/mongodb/27017/conf/ssl/ca.pem --authenticationMechanism MONGODB-X509 --authenticationDatabase='$external' --host 10.1.1.1 -u "C=CN,ST=tm,L=tm,O=supsersb,OU=sb,CN=sbtest"
MongoDB tmell version v4.0.8
connecting to: mongodb://10.1.1.1:27017/?authMechanism=MONGODB-X509&authSource=%24external&gssbpiServiceName=mongodb
Implicit session: session { "id" : UUID("f4e9eaa2-8c49-4721-a2e7-97734597f1f4") }
MongoDB server version: 4.0.8
replsbtest:PRIMARY>

# SECONDARY
$ mongo     --ssl --sslPEMKeyFile /opt/mongodb/27017/conf/ssl/sbtest.pem --sslCAFile /opt/mongodb/27017/conf/ssl/ca.pem --authenticationMechanism MONGODB-X509 --authenticationDatabase='$external' --host 10.1.1.2 -u "C=CN,ST=tm,L=tm,O=supsersb,OU=sb,CN=sbtest"
MongoDB tmell version v4.0.8
connecting to: mongodb://10.1.1.2:27017/?authMechanism=MONGODB-X509&authSource=%24external&gssbpiServiceName=mongodb
Implicit session: session { "id" : UUID("2c5b359a-685f-4e96-8989-d6bd259d82e9") }
MongoDB server version: 4.0.8
replsbtest:SECONDARY>

$ mongo     --ssl --sslPEMKeyFile /opt/mongodb/27017/conf/ssl/sbtest.pem --sslCAFile /opt/mongodb/27017/conf/ssl/ca.pem --authenticationMechanism MONGODB-X509 --authenticationDatabase='$external' --host 10.1.1.3 -u "C=CN,ST=tm,L=tm,O=supsersb,OU=sb,CN=sbtest"
MongoDB tmell version v4.0.8
connecting to: mongodb://10.1.1.3:27017/?authMechanism=MONGODB-X509&authSource=%24external&gssbpiServiceName=mongodb
Implicit session: session { "id" : UUID("599d30eb-c8a3-4165-94d9-d9cdd9555285") }
MongoDB server version: 4.0.8
replsbtest:SECONDARY>

7. 数据备份/恢复测试

7.0 错误范例

$ mongodump --ssl  --sslPEMKeyFile /opt/mongodb/27017/conf/ssl/sbtest.pem --sslCAFile /opt/mongodb/27017/conf/ssl/ca.pem --authenticationMechanism MONGODB-X509 --authenticationDatabase='$external' --host 10.1.1.1 -u "C=CN,ST=tm,L=tm,O=supsersb,OU=sb,CN=sbtest" -d sbtest
2019-04-16T15:23:21.936+0800	error dialing 10.1.1.1:27017: Host validation error
2019-04-16T15:23:22.444+0800	error dialing 10.1.1.1:27017: Host validation error
2019-04-16T15:23:22.952+0800	error dialing 10.1.1.1:27017: Host validation error
2019-04-16T15:23:23.960+0800	error dialing 10.1.1.1:27017: Host validation error
2019-04-16T15:23:24.467+0800	error dialing 10.1.1.1:27017: Host validation error
2019-04-16T15:23:24.975+0800	error dialing 10.1.1.1:27017: Host validation error
2019-04-16T15:23:25.475+0800	Failed: error connecting to db server: no reachable servers

同样的证书 mongo tmell可以启动 mongodump不可以，是因为mongodump命令中的host和server证书中的CN不相符（ps：这个地方很关键，此文章是通过使用sslAllowInvalidHostnames来解决这个问题的，同样监控查看也要加此参数），建议将server证书的CN设置成对应服务器的hostname或是public ip。

7.1 数据备份

$ mongodump --ssl --sslAllowInvalidHostnames  --sslPEMKeyFile /opt/mongodb/27017/conf/ssl/sbtest.pem --sslCAFile /opt/mongodb/27017/conf/ssl/ca.pem --authenticationMechanism MONGODB-X509 --authenticationDatabase='$external' --host 10.1.1.1 -u "C=CN,ST=tm,L=tm,O=supsersb,OU=sb,CN=sbtest" -d sbtest
2019-04-16T15:18:36.622+0800	writing sbtest.log to
2019-04-16T15:18:36.622+0800	writing sbtest.activity to
2019-04-16T15:18:36.622+0800	writing sbtest.env to
2019-04-16T15:18:36.622+0800	writing sbtest.process to
2019-04-16T15:18:36.844+0800	done dumping sbtest.process (4825 documents)
2019-04-16T15:18:36.844+0800	writing sbtest.resource to
2019-04-16T15:18:36.849+0800	done dumping sbtest.resource (802 documents)
2019-04-16T15:18:36.849+0800	writing sbtest.comment to
2019-04-16T15:18:36.851+0800	done dumping sbtest.comment (242 documents)
2019-04-16T15:18:36.851+0800	writing sbtest.suggestion to
2019-04-16T15:18:36.853+0800	done dumping sbtest.suggestion (26 documents)
2019-04-16T15:18:37.005+0800	done dumping sbtest.activity (29047 documents)
2019-04-16T15:18:37.085+0800	done dumping sbtest.env (19235 documents)
2019-04-16T15:18:37.333+0800	done dumping sbtest.log (180804 documents)

7.2 数据恢复

$ mongorestore --ssl --sslAllowInvalidHostnames  --sslPEMKeyFile /opt/mongodb/27017/conf/ssl/sbtest.pem --sslCAFile /opt/mongodb/27017/conf/ssl/ca.pem --authenticationMechanism MONGODB-X509 --authenticationDatabase='$external' --host 10.1.1.1 -u "C=CN,ST=tm,L=tm,O=supsersb,OU=sb,CN=sbtest" -d sbtest  dump/sbtest/
2019-04-16T15:15:32.367+0800	the --db and --collection args tmould only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead
2019-04-16T15:15:32.368+0800	building a list of collections to restore from dump/sbtest dir
2019-04-16T15:15:32.368+0800	reading metadata for sbtest.env from dump/sbtest/env.metadata.json
2019-04-16T15:15:32.417+0800	restoring sbtest.env from dump/sbtest/env.bson
2019-04-16T15:15:32.419+0800	reading metadata for sbtest.activity from dump/sbtest/activity.metadata.json
2019-04-16T15:15:32.419+0800	reading metadata for sbtest.process from dump/sbtest/process.metadata.json
2019-04-16T15:15:32.419+0800	reading metadata for sbtest.log from dump/sbtest/log.metadata.json
2019-04-16T15:15:32.487+0800	restoring sbtest.process from dump/sbtest/process.bson
2019-04-16T15:15:32.536+0800	restoring sbtest.activity from dump/sbtest/activity.bson
2019-04-16T15:15:32.591+0800	restoring sbtest.log from dump/sbtest/log.bson
2019-04-16T15:15:33.376+0800	restoring indexes for collection sbtest.process from metadata
2019-04-16T15:15:33.471+0800	finitmed restoring sbtest.process (4825 documents)
2019-04-16T15:15:33.471+0800	reading metadata for sbtest.resource from dump/sbtest/resource.metadata.json
2019-04-16T15:15:33.527+0800	restoring sbtest.resource from dump/sbtest/resource.bson
2019-04-16T15:15:33.733+0800	no indexes to restore
2019-04-16T15:15:33.733+0800	finitmed restoring sbtest.resource (802 documents)
2019-04-16T15:15:33.733+0800	reading metadata for sbtest.suggestion from dump/sbtest/suggestion.metadata.json
2019-04-16T15:15:33.789+0800	restoring sbtest.suggestion from dump/sbtest/suggestion.bson
2019-04-16T15:15:33.879+0800	no indexes to restore
2019-04-16T15:15:33.879+0800	finitmed restoring sbtest.suggestion (26 documents)
2019-04-16T15:15:33.879+0800	reading metadata for sbtest.comment from dump/sbtest/comment.metadata.json
2019-04-16T15:15:33.925+0800	restoring sbtest.comment from dump/sbtest/comment.bson
2019-04-16T15:15:34.019+0800	no indexes to restore
2019-04-16T15:15:34.019+0800	finitmed restoring sbtest.comment (242 documents)
2019-04-16T15:15:35.358+0800	[#####################...]       sbtest.env  82.6MB/90.6MB  (91.2%)
2019-04-16T15:15:35.358+0800	[###################.....]  sbtest.activity  39.0MB/47.8MB  (81.6%)
2019-04-16T15:15:35.358+0800	[###.....................]       sbtest.log  6.24MB/38.8MB  (16.1%)
2019-04-16T15:15:35.358+0800
2019-04-16T15:15:35.717+0800	[########################]  sbtest.env  90.6MB/90.6MB  (100.0%)
2019-04-16T15:15:35.718+0800	no indexes to restore
2019-04-16T15:15:35.718+0800	finitmed restoring sbtest.env (19235 documents)
2019-04-16T15:15:36.041+0800	[########################]  sbtest.activity  47.8MB/47.8MB  (100.0%)
2019-04-16T15:15:36.041+0800	no indexes to restore
2019-04-16T15:15:36.041+0800	finitmed restoring sbtest.activity (29047 documents)
2019-04-16T15:15:38.357+0800	[###########.............]  sbtest.log  18.2MB/38.8MB  (47.0%)
2019-04-16T15:15:41.357+0800	[##################......]  sbtest.log  30.5MB/38.8MB  (78.6%)
2019-04-16T15:15:43.845+0800	[########################]  sbtest.log  38.8MB/38.8MB  (100.0%)
2019-04-16T15:15:43.845+0800	no indexes to restore
2019-04-16T15:15:43.845+0800	finitmed restoring sbtest.log (180804 documents)
2019-04-16T15:15:43.845+0800	done

注意：参数--sslAllowInvalidHostnames

8. 测试python驱动使用ssl连接mongodb

$ ipython
Python 2.7.13 (default, Nov 24 2017, 17:33:09)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import ssl

In [2]: from pymongo import MongoClient
   ...:

In [3]: client = MongoClient('10.1.1.1',
   ...:                       username="C=CN,ST=tm,L=tm,O=supsersb,OU=sb,CN=sbtest",
   ...:                       authMechanism="MONGODB-X509",
   ...:                       ssl=True,
   ...:                       ssl_certfile='/opt/mongodb/27017/conf/ssl/sbtest.pem',
   ...:                       ssl_cert_reqs=ssl.CERT_REQUIRED,
   ...:                       ssl_ca_certs='/opt/mongodb/27017/conf/ssl/ca.pem')

In [4]: mydict = {"title":"just do it"}
   ...:

In [5]: mydb=client["sbtest"]
   ...: mycol=mydb["coll"]
   ...:

In [6]: mycol.insert_one(mydict)
   ...:
Out[6]: <pymongo.results.InsertOneResult at 0x7f51f6a40518>

In [7]: mycol.find_one()
Out[7]: {u'_id': ObjectId('5cb00527d54e0c01715054d7'), u'title': u'just do it'}

In [8]: