Mongodb记录

最新推荐文章于 2024-05-09 04:42:19 发布

胖胖胖胖胖虎

最新推荐文章于 2024-05-09 04:42:19 发布

阅读量1.5k

点赞数

分类专栏： Mongodb 文章标签： mongodb 数据库 nosql

本文链接：https://blog.csdn.net/qq_15138049/article/details/127244575

版权

Mongodb 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

Mongodb

基础

端口

默认端口	介绍
`27017`	`mongod` 和 `mongos` 实例的默认端口。你可以通过 `port` 或 –port 改变该端口。
`27018`	设置 –shardsvr 运行变量或在配置文件里设置 `clusterRole` 为 `shardsvr` 时的默认端口。
`27019`	设置 –configsvr 运行变量或在配置文件中将 `clusterRole` 设置为 `configsvr` 时的默认端口。
`28017`	系统状态网页的默认端口。系统状态网络页面永远可以在比 `port` 大 1000 的端口反问。

Mongodb 概念

SQL术语/概念	MongoDB术语/概念	解释/说明
database	database	数据库
table	collection	数据库表/集合
row	document	数据记录行/文档
column	field	数据字段/域
index	index	索引
table joins		表连接,MongoDB不支持
primary key	primary key	主键,MongoDB自动将_id字段设置为主键

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-a8WpV6Gf-1665384514804)(Mongodb.assets/Figure-1-Mapping-Table-to-Collection-1.png)]

数据类型

数据类型	描述
String	字符串。存储数据常用的数据类型。在 MongoDB 中，UTF-8 编码的字符串才是合法的。
Integer	整型数值。用于存储数值。根据你所采用的服务器，可分为 32 位或 64 位。
Boolean	布尔值。用于存储布尔值（真/假）。
Double	双精度浮点值。用于存储浮点值。
Min/Max keys	将一个值与 BSON（二进制的 JSON）元素的最低值和最高值相对比。
Array	用于将数组或列表或多个值存储为一个键。
Timestamp	时间戳。记录文档修改或添加的具体时间。
Object	用于内嵌文档。
Null	用于创建空值。
Symbol	符号。该数据类型基本上等同于字符串类型，但不同的是，它一般用于采用特殊符号类型的语言。
Date	日期时间。用 UNIX 时间格式来存储当前日期或时间。你可以指定自己的日期时间：创建 Date 对象，传入年月日信息。
Object ID	对象 ID。用于创建文档的 ID。
Binary Data	二进制数据。用于存储二进制数据。
Code	代码类型。用于在文档中存储 JavaScript 代码。
Regular expression	正则表达式类型。用于存储正则表达式。

Mongodb 连接

https://www.runoob.com/mongodb/mongodb-connections.html

mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]]
mongodb:// 这是固定的格式，必须要指定。

username:password@ 可选项，如果设置，在连接数据库服务器之后，驱动都会尝试登录这个数据库

host1 必须的指定至少一个host, host1 是这个URI唯一要填写的。它指定了要连接服务器的地址。如果要连接复制集，请指定多个主机地址。

portX 可选的指定端口，如果不填，默认为27017

/database 如果指定username:password@，连接并验证登录指定数据库。若不指定，默认打开 test 数据库。

?options 是连接选项。如果不使用/database，则前面需要加上/。所有连接选项都是键值对name=value，键值对之间通过&或;（分号）隔开

shell

database

创建database、没有则创建、已存在则切换

use DATABASE_NAME

show dbs

查询所有数据库

show dbs

可以看到，我们刚创建的数据库 runoob 并不在数据库的列表中，要显示它，我们需要向 runoob 数据库插入一些数据。

> db.runoob.insert({"name":"菜鸟教程"})
WriteResult({ "nInserted" : 1 })
> show dbs
admin   0.000GB
config  0.000GB
local   0.000GB
runoob  0.000GB

db.orders.insertMany([
  {
    order_id: 101,
    order_date: ISODate("2020-07-30T10:08:22.001Z"),
    customer_id: 1001,
    price: NumberDecimal("50.50"),
    product: {
      name: 'scooter',
      description: 'Small 2-wheel scooter'
    },
    order_status: false
  },
  {
    order_id: 102, 
    order_date: ISODate("2020-07-30T10:11:09.001Z"),
    customer_id: 1002,
    price: NumberDecimal("15.00"),
    product: {
      name: 'car battery',
      description: '12V car battery'
    },
    order_status: false
  },
  {
    order_id: 103,
    order_date: ISODate("2020-07-30T12:00:30.001Z"),
    customer_id: 1003,
    price: NumberDecimal("25.25"),
    product: {
      name: 'hammer',
      description: '16oz carpenter hammer'
    },
    order_status: false
  }
]);

> use admin
switched to db admin
> db.createUser({user:"root",pwd:"root",roles:[{role:'root',db:'admin'}]})
Successfully added user: {
        "user" : "root",
        "roles" : [
                {
                        "role" : "root",
                        "db" : "admin"
                }
        ]
}
> db.auth("root","123456");
Error: Authentication failed.
0
> db.auth("root","root");
1
>

db 查看当前处于哪个database

集合（类似关系型数据库的表）

> db.createCollection("runoob")
{ "ok" : 1 }
> show tables;
runoob
user
> db.createCollection("mycol", { capped : true, autoIndexId : true, size :
...    6142800, max : 10000 } )
{
        "note" : "The autoIndexId option is deprecated and will be removed in a future release",
        "ok" : 1
}
> show tables;
mycol
runoob
user
> db.user.mycol.find
db.user.mycol.find(               db.user.mycol.findOne(            db.user.mycol.findOneAndReplace(
db.user.mycol.findAndModify(      db.user.mycol.findOneAndDelete(   db.user.mycol.findOneAndUpdate(
> db.user.mycol.find()
> db.user.mycol.insert({"name" : "菜鸟教程"})
WriteResult({ "nInserted" : 1 })
> db.user.mycol.find()
{ "_id" : ObjectId("621093c0a42e17048e8f17e8"), "name" : "菜鸟教程" }
>

> use flink-mongodb-databases
switched to db flink-mongodb-databases
> show tables
> 
> db.dropDatabase()
{ "ok" : 1 }
> use user
switched to db user
> db.user.insert({id:1,name:'zhangsan'})
WriteResult({ "nInserted" : 1 })
> db.user.save({id:1,name:'zhangsan'})
WriteResult({ "nInserted" : 1 })
> db.user.update({id:1},{$set:{age:22}},true)
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> show tables;
user
> db.user.find()
{ "_id" : ObjectId("62109146a42e17048e8f17e5"), "id" : 1, "name" : "zhangsan", "age" : 22 }
{ "_id" : ObjectId("62109157a42e17048e8f17e6"), "id" : 1, "name" : "zhangsan" }
>
### 删除满足条件的 document ###
> db.user.remove({age:22})
WriteResult({ "nRemoved" : 1 })
> db.user.find()
{ "_id" : ObjectId("62109157a42e17048e8f17e6"), "id" : 1, "name" : "zhangsan" }
>
> db.user.remove({age:22},true)	#删除1条数据
> db.user.remove			   #删除所有数据

> db.user.find()  #查询全部数据
> db.user.find({},{id:1,username:1})  #只查询id与username字段
> db.user.find().count()  #查询数据条数
> db.user.save({id:1,name:'zhangsan', age:18})
WriteResult({ "nInserted" : 1 })
> db.user.find().count()
2


> db.user.find({id:1}) #查询id为1的数据
> db.user.find({age:{$lte:21}}) #查询小于等于21的数据
> db.user.find({age:{$lte:21}, id:{$gte:2}}) #and查询，age小于等于21并且id大于等于2
> db.user.find({$or:[{id:1},{id:2}]}) #查询id=1 or id=2

#查看索引
> db.user.getIndexes()
#创建索引
> db.user.createIndex({'age':1})
#删除索引
> db.user.dropIndex("age_1")
#删除除了_id之外的索引
> db.user.dropIndexes()
#创建联合索引
> db.user.createIndex({'age':1, 'id':-1})
#查看索引大小，单位：字节
> db.user.totalIndexSize()

添加用户

mongo-rs:PRIMARY> use admin
switched to db admin
mongo-rs:PRIMARY> db.auth("root","123456")
Error: Authentication failed.
0
mongo-rs:PRIMARY>
mongo-rs:PRIMARY> use admin
switched to db admin
mongo-rs:PRIMARY> db.auth("root","123456")
Error: Authentication failed.
0
mongo-rs:PRIMARY> db.createUser(
  {
    user: "root",
    pwd: "123456",
    roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
  }
)
Successfully added user: {
        "user" : "root",
        "roles" : [
                {
                        "role" : "userAdminAnyDatabase",
                        "db" : "admin"
                }
        ]
}
mongo-rs:PRIMARY> db.auth("root","123456")
1
mongo-rs:PRIMARY>

集群状态

rs.status()

删除节点

rs.remove("hadoop02:27017")

查询

https://www.runoob.com/mongodb/mongodb-query.html

db.col.find().pretty()

and

> db.user.mycol.find({"company":"nufront", "name":"菜鸟教程"})
{ "_id" : ObjectId("621094dca42e17048e8f17e9"), "name" : "菜鸟教程", "company" : "nufront", "home" : "panyu" }
>

or

> db.user.mycol.find({$or:[{"company":"nufront", "name":"菜鸟教程1"},{"home":"panyu"}]})
{ "_id" : ObjectId("621094dca42e17048e8f17e9"), "name" : "菜鸟教程", "company" : "nufront", "home" : "panyu" }
>

and 与 or 结合使用

db.col.find({"likes": {$gt:50}, $or: [{"by": "菜鸟教程"},{"title": "MongoDB 教程"}]}).pretty()

watch

> db.user.mycol.watch()
uncaught exception: Error: command failed: {
        "ok" : 0,
        "errmsg" : "The $changeStream stage is only supported on replica sets",
        "code" : 40573,
        "codeName" : "Location40573"
} with original command request: {
        "aggregate" : "user.mycol",
        "pipeline" : [
                {
                        "$changeStream" : {
                                "fullDocument" : "default"
                        }
                }
        ],
        "cursor" : {

        },
        "lsid" : {
                "id" : UUID("93b180b0-bb94-415e-a089-f95f27c726ce")
        }
} on connection: connection to 127.0.0.1:27017 : aggregate failed :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
doassert@src/mongo/shell/assert.js:18:14
_assertCommandWorked@src/mongo/shell/assert.js:737:17
assert.commandWorked@src/mongo/shell/assert.js:829:16
DB.prototype._runAggregate@src/mongo/shell/db.js:281:5
DBCollection.prototype.aggregate@src/mongo/shell/collection.js:1058:12
DBCollection.prototype.watch@src/mongo/shell/collection.js:1554:12
@(shell):1:1
>

update 语句

mongo-rs:PRIMARY> db.student.update({"name":'KD'},{$set:{age:27}},true)
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })

删除数据

### 删除多条数据
mongo-rs:PRIMARY> db.student.deleteMany({"name":"KD"})
{ "acknowledged" : true, "deletedCount" : 1 }
### 删除单条记录
mongo-rs:PRIMARY> db.student.deleteOne({"name":"KD"})
{ "acknowledged" : true, "deletedCount" : 0 }
### 删除 student 表所有数据
mongo-rs:PRIMARY> db.student.deleteMany()

副本相关 rs

读写权限相关

mongodb默认读写都是在Primary上进行的，副本节点不允许读写，可以使用如下命令来允许副本读

db.getMongo().setSlaveOk()

SECONDARY 新增失败、不允许写入；

SECONDARY 读取通过 db.getMongo().setSlaveOk() 设置可读；

mongo-rs:SECONDARY> use school
switched to db school
mongo-rs:SECONDARY> db.student.find()
Error: error: {
        "topologyVersion" : {
                "processId" : ObjectId("6212135aeaf0477bb419259f"),
                "counter" : NumberLong(3)
        },
        "ok" : 0,
        "errmsg" : "not master and slaveOk=false",
        "code" : 13435,
        "codeName" : "NotPrimaryNoSecondaryOk",
        "$clusterTime" : {
                "clusterTime" : Timestamp(1645351808, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1645351808, 1)
}
mongo-rs:SECONDARY> db.getMongo().setSecondaryOk()
mongo-rs:SECONDARY> db.student.find()
{ "_id" : ObjectId("62120e82b89a69719f90735a"), "name" : "Tom", "age" : 16 }
{ "_id" : ObjectId("62120e82b89a69719f90735b"), "name" : "Jerry", "age" : 15 }
{ "_id" : ObjectId("62120e84b89a69719f90735c"), "name" : "Mary", "age" : 9 }
{ "_id" : ObjectId("62121136b89a69719f90735d"), "name" : "Tom", "age" : 16 }
mongo-rs:SECONDARY> db.student.insert({name:"lancelot", age:10})
WriteCommandError({
        "topologyVersion" : {
                "processId" : ObjectId("6212135aeaf0477bb419259f"),
                "counter" : NumberLong(3)
        },
        "ok" : 0,
        "errmsg" : "not master",
        "code" : 10107,
        "codeName" : "NotWritablePrimary",
        "$clusterTime" : {
                "clusterTime" : Timestamp(1645351848, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1645351848, 1)
})

查询同步状态

source——从库的IP及端口
syncedTo——当前的同步情况，延迟了多久等信息

mongo-rs:PRIMARY> db.printSlaveReplicationInfo()
WARNING: printSlaveReplicationInfo is deprecated and may be removed in the next major release. Please use printSecondaryReplicationInfo instead.
source: 172.17.0.3:27017
        syncedTo: Mon Feb 21 2022 15:42:54 GMT+0800 (CST)
        0 secs (0 hrs) behind the primary
source: 172.17.0.4:27017
        syncedTo: Mon Feb 21 2022 15:42:54 GMT+0800 (CST)
        0 secs (0 hrs) behind the primary
mongo-rs:PRIMARY>

扩展

ChangeStreams

https://mp.weixin.qq.com/s/XdPnZlxaPNnawo_q-1-21Q

基于Docker的MongoDB replica set（副本集）搭建

https://blog.csdn.net/MacwinWin/article/details/116166791

https://www.jianshu.com/p/c3811263fd3a

Mongodb oplog解析

oplog是local库下的一个固定集合，Secondary就是通过查看Primary 的oplog这个集合来进行复制的。每个节点都有oplog，记录这从主节点复制过来的信息，这样每个成员都可以作为同步源给其他节点。

Oplog 可以说是Mongodb Replication的纽带了。

mongo-rs:SECONDARY> use local
switched to db local
mongo-rs:SECONDARY> db.oplog.rs.find()
{ "op" : "n", "ns" : "", "o" : { "msg" : "initiating set" }, "ts" : Timestamp(1645413861, 1), "v" : NumberLong(2), "wall" : ISODate("2022-02-21T03:24:21.655Z") }
{ "op" : "c", "ns" : "config.$cmd", "ui" : UUID("ee940e9f-6ee5-4feb-bd0d-9ac1aa11078a"), "o" : { "create" : "transactions", "idIndex" : { "v" : 2, "key" : { "_id" : 1 }, "name" : "_id_" } }, "ts" : Timestamp(1645413873, 2), "t" : NumberLong(1), "v" : NumberLong(2), "wall" : ISODate("2022-02-21T03:24:33.181Z") }
...
mongo-rs:SECONDARY>
mongo-rs:SECONDARY> use local
switched to db local
mongo-rs:SECONDARY> db.oplog.rs.find({'ns':'school.student'})
{ "op" : "i", "ns" : "school.student", "ui" : UUID("3faa4550-97ef-4684-8651-ba7a562689d9"), "o" : { "_id" : ObjectId("621324818dadd47ef029bb2e"), "name" : "MJ", "age" : 21 }, "ts" : Timestamp(1645421697, 2), "t" : NumberLong(6), "v" : NumberLong(2), "wall" : ISODate("2022-02-21T05:34:57.768Z") }
...
{ "op" : "u", "ns" : "school.student", "ui" : UUID("3faa4550-97ef-4684-8651-ba7a562689d9"), "o" : { "$v" : 2, "diff" : { "u" : { "age" : 28 } } }, "o2" : { "_id" : ObjectId("62133d5dd44bc191fff5f66c") }, "ts" : Timestamp(1645431111, 1), "t" : NumberLong(9), "v" : NumberLong(2), "wall" : ISODate("2022-02-21T08:11:51.250Z") }
mongo-rs:SECONDARY>

副本集信息

mongo-rs:PRIMARY> db.getReplicationInfo()
{
        "logSizeMB" : 990,
        "usedMB" : 0.23,
        "timeDiff" : 23018,
        "timeDiffHours" : 6.39,
        "tFirst" : "Mon Feb 21 2022 11:24:21 GMT+0800 (CST)",
        "tLast" : "Mon Feb 21 2022 17:47:59 GMT+0800 (CST)",
        "now" : "Mon Feb 21 2022 17:48:02 GMT+0800 (CST)"
}

问题记录

The $changeStream stage is only supported on replica sets

docker 服务访问不到？

mongodb

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-TIh1k6Av-1665384514806)(Mongodb.assets/1645322043096.png)]

hadoop03（docker 宿主机）却可以，网桥正常、端口映射正常

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-AW3gGi8q-1665384514807)(Mongodb.assets/1645322199206.png)]

### 清除nat表 ###
[root@hadoop03 ~]# iptables --flush&iptables -tnat --flush & iptables -P FORWARD ACCEPT
[1] 90831
[2] 90832
Another app is currently holding the xtables lock. Perhaps you want to use the -w option?
[1]-  Exit 4                  iptables --flush
[2]+  Done                    iptables -tnat --flush
[root@hadoop03 ~]#
### 重启docker ###
[root@hadoop03 ~]# service docker restart
Redirecting to /bin/systemctl restart  docker.service
[root@hadoop03 ~]# 
### 启动容器 ###
[root@hadoop03 ~]# docker start my-mongo
my-mongo

### hadoop02 ###
[hadoop@hadoop02 ~]$ telnet hadoop03 27017
Trying 192.168.153.103...
Connected to hadoop03.
Escape character is '^]'.
^[[A

副本集模式下 SECONDARY读写权限问题 no master | not master and slaveOk=false

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-I4pKBvIk-1665384514808)(Mongodb.assets/1645350758449.png)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-t68N6BE8-1665384514808)(Mongodb.assets/1645350872964.png)]

向上翻、见 shell ——>>>> 读写权限相关

Authentication failed 权限问题

Command failed with error 18 (AuthenticationFailed): 'Authentication failed.' on server hadoop03:30012. The full response is {"ok": 0.0, "errmsg": "Authentication failed.", "code": 18, "codeName": "AuthenticationFailed", "$clusterTime": {"clusterTime": {"$timestamp": {"t": 1645400170, "i": 1}}, "signature": {"hash": {"$binary": {"base64": "AAAAAAAAAAAAAAAAAAAAAAAAAAA=", "subType": "00"}}, "keyId": 0}}, "operationTime": {"$timestamp": {"t": 1645400170, "i": 1}}}

node is not in primary or recovering state

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-1fzANddp-1665384514809)(Mongodb.assets/1645411125642.png)]

副本集健康问题

mongo-rs:SECONDARY> rs.status()
{
        "set" : "mongo-rs",
        "date" : ISODate("2022-02-21T02:48:20.906Z"),
        "myState" : 2,
        "term" : NumberLong(6),
        "syncSourceHost" : "",
        "syncSourceId" : -1,
        "heartbeatIntervalMillis" : NumberLong(2000),
        "majorityVoteCount" : 2,
        "writeMajorityCount" : 2,
        "votingMembersCount" : 3,
        "writableVotingMembersCount" : 3,
        "optimes" : {
                "lastCommittedOpTime" : {
                        "ts" : Timestamp(0, 0),
                        "t" : NumberLong(-1)
                },
                "lastCommittedWallTime" : ISODate("1970-01-01T00:00:00Z"),
                "appliedOpTime" : {
                        "ts" : Timestamp(1645400170, 1),
                        "t" : NumberLong(6)
                },
                "durableOpTime" : {
                        "ts" : Timestamp(1645400170, 1),
                        "t" : NumberLong(6)
                },
                "lastAppliedWallTime" : ISODate("2022-02-20T23:36:10.501Z"),
                "lastDurableWallTime" : ISODate("2022-02-20T23:36:10.501Z")
        },
        "lastStableRecoveryTimestamp" : Timestamp(1645400130, 1),
        "members" : [
                {
                        "_id" : 0,
                        "name" : "172.17.0.4:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 3911,
                        "optime" : {
                                "ts" : Timestamp(1645400170, 1),
                                "t" : NumberLong(6)
                        },
                        "optimeDate" : ISODate("2022-02-20T23:36:10Z"),
                        "lastAppliedWallTime" : ISODate("2022-02-20T23:36:10.501Z"),
                        "lastDurableWallTime" : ISODate("2022-02-20T23:36:10.501Z"),
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "configVersion" : 1,
                        "configTerm" : 6,
                        "self" : true,
                        "lastHeartbeatMessage" : ""
                },
                {
                        "_id" : 1,
                        "name" : "172.17.0.5:27017",
                        ### ???
                        "health" : 0,
                        "state" : 8,
                        "stateStr" : "(not reachable/healthy)",
                        "uptime" : 0,
                        "optime" : {
                                "ts" : Timestamp(0, 0),
                                "t" : NumberLong(-1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(0, 0),
                                "t" : NumberLong(-1)
                        },
                        "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
                        "optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
                        "lastAppliedWallTime" : ISODate("1970-01-01T00:00:00Z"),
                        "lastDurableWallTime" : ISODate("1970-01-01T00:00:00Z"),
                        "lastHeartbeat" : ISODate("2022-02-21T02:48:16.866Z"),
                        "lastHeartbeatRecv" : ISODate("1970-01-01T00:00:00Z"),
                        "pingMs" : NumberLong(0),
                        ### ???
                        "lastHeartbeatMessage" : "Error connecting to 172.17.0.5:27017 :: caused by :: No route to host",
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "configVersion" : -1,
                        "configTerm" : -1
                },
                {
                        "_id" : 2,
                        "name" : "172.17.0.6:27017",
                        "health" : 0,
                        "state" : 8,
                        "stateStr" : "(not reachable/healthy)",
                        "uptime" : 0,
                        "optime" : {
                                "ts" : Timestamp(0, 0),
                                "t" : NumberLong(-1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(0, 0),
                                "t" : NumberLong(-1)
                        },
                        "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
                        "optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
                        "lastAppliedWallTime" : ISODate("1970-01-01T00:00:00Z"),
                        "lastDurableWallTime" : ISODate("1970-01-01T00:00:00Z"),
                        "lastHeartbeat" : ISODate("2022-02-21T02:48:16.866Z"),
                        "lastHeartbeatRecv" : ISODate("1970-01-01T00:00:00Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "Error connecting to 172.17.0.6:27017 :: caused by :: No route to host",
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "configVersion" : -1,
                        "configTerm" : -1
                }
        ],
        "ok" : 1,
        "$clusterTime" : {
                "clusterTime" : Timestamp(1645400170, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1645400170, 1)
}
mongo-rs:SECONDARY>

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-aYz0lZil-1665384514810)(Mongodb.assets/1645411961274.png)]

ReadConcernMajorityNotAvailableYet: Read concern majority reads are currently not possible.

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-um8Zhrq5-1665384514811)(Mongodb.assets/1645412606703.png)]

Could not find member to sync from

rs.status()

"members": [{
...
"infoMessage": "Could not find member to sync from "
...

}]

replica set副本集

[root@hadoop03 ~]#  docker network create --subnet=172.30.0.0/24 mongodbnetwork
[root@hadoop03 ~]#  
[root@hadoop03 ~]#  
[root@hadoop03 ~]#  sudo docker run -d --name mongo1 -p 27016:27017 --net=mongodbnetwork mongo:5.0.6 --replSet test-rep
[root@hadoop03 ~]#  sudo docker run -d --name mongo2 -p 27017:27017 --net=mongodbnetwork mongo:5.0.6 --replSet test-rep
[root@hadoop03 ~]#  sudo docker run -d --name mongo3 -p 27018:27017 --net=mongodbnetwork mongo:5.0.6 --replSet test-rep



sudo docker run --name m0 -idt -p 28017:28017  mongo:5.0.6 /bin/bash -c 'mongod --replSet replset0'
sudo docker run --name m1 -idt  mongo:5.0.6 /bin/bash -c 'mongod --replSet replset0'
sudo docker run --name m2 -idt  mongo:5.0.6 /bin/bash -c 'mongod --replSet replset0'

config = { _id:"replset0", members:[{_id:0,host:"172.17.0.4:27017"},{_id:1,host:"172.17.0.5:27017"},{_id:2,host:"172.17.0.6:27017"}]}

docker run --name mongo-rs01 -p 30010:27017 -v /apps/mongo-rs/data01:/data/db -v /apps/mongo-rs/backup:/data/backup -v /apps/mongo-rs/key:/data/key -v /etc/localtime:/etc/localtime -e MONGO_INITDB_ROOT_USERNAME=admin -e MONGO_INITDB_ROOT_PASSWORD=PWD2020 -d mongo:5.0.6 --replSet mongo-rs --auth --keyFile /data/key/mongo-rs.key --bind_ip_all

docker run --name mongo-rs02 -p 30011:27017 -v /apps/mongo-rs/data02:/data/db -v /apps/mongo-rs/backup:/data/backup -v /apps/mongo-rs/key:/data/key -v /etc/localtime:/etc/localtime -e MONGO_INITDB_ROOT_USERNAME=admin -e MONGO_INITDB_ROOT_PASSWORD=PWD2020 -d mongo:5.0.6 --replSet mongo-rs --auth --keyFile /data/key/mongo-rs.key --bind_ip_all

docker run --name mongo-rs03 -p 30012:27017 -v /apps/mongo-rs/data03:/data/db -v /apps/mongo-rs/backup:/data/backup -v /apps/mongo-rs/key:/data/key -v /etc/localtime:/etc/localtime -e MONGO_INITDB_ROOT_USERNAME=admin -e MONGO_INITDB_ROOT_PASSWORD=PWD2020 -d mongo:5.0.6 --replSet mongo-rs --auth --keyFile /data/key/mongo-rs.key --bind_ip_all

TMD副本集终于成功了

http://blog.0x01.site/2017/01/13/MongoDB%E7%9A%84Replica-Set%E4%BB%A5%E5%8F%8AAuth%E7%9A%84%E9%85%8D%E7%BD%AE/

https://blog.csdn.net/zhanngle/article/details/105132527

https://cloud.tencent.com/developer/article/1013044

docker run --name mongo-rs01 -p 30010:27017 -v /apps/mongo-rs/data01:/data/db -v /apps/mongo-rs/backup:/data/backup -v /apps/mongo-rs/key:/data/key -v /etc/localtime:/etc/localtime  -d mongo:5.0.6 --replSet mongo-rs  --bind_ip_all

docker run --name mongo-rs02 -p 30011:27017 -v /apps/mongo-rs/data02:/data/db -v /apps/mongo-rs/backup:/data/backup -v /apps/mongo-rs/key:/data/key -v /etc/localtime:/etc/localtime  -d mongo:5.0.6 --replSet mongo-rs  --bind_ip_all

docker run --name mongo-rs03 -p 30012:27017 -v /apps/mongo-rs/data03:/data/db -v /apps/mongo-rs/backup:/data/backup -v /apps/mongo-rs/key:/data/key -v /etc/localtime:/etc/localtime  -d mongo:5.0.6 --replSet mongo-rs  --bind_ip_all

root@614b75e20c92:/# mongo
MongoDB shell version v5.0.6
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("4bfc114e-5e21-485c-b20b-b897f8d5fd5e") }
MongoDB server version: 5.0.6
================
Warning: the "mongo" shell has been superseded by "mongosh",
which delivers improved usability and compatibility.The "mongo" shell has been deprecated and will be removed in
an upcoming release.
For installation instructions, see
https://docs.mongodb.com/mongodb-shell/install/
================
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
        https://docs.mongodb.com/
Questions? Try the MongoDB Developer Community Forums
        https://community.mongodb.com
---
The server generated these startup warnings when booting:
        2022-02-20T16:54:33.971+08:00: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted
        2022-02-20T16:54:33.971+08:00: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. We suggest setting it to 'never'
        2022-02-20T16:54:33.971+08:00: /sys/kernel/mm/transparent_hugepage/defrag is 'always'. We suggest setting it to 'never'
---
---
        Enable MongoDB's free cloud-based monitoring service, which will then receive and display
        metrics about your deployment (disk utilization, CPU, operation statistics, etc).

        The monitoring data will be available on a MongoDB website with a unique URL accessible to you
        and anyone you share the URL with. MongoDB may use this information to make product
        improvements and to suggest MongoDB products and deployment options to you.

        To enable free monitoring, run the following command: db.enableFreeMonitoring()
        To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
---
>
> config = { _id:"mongo-rs", members:[{_id:0,host:"172.17.0.4:27017"},{_id:1,host:"172.17.0.5:27017"},{_id:2,host:"172.17.0.6:27017"}]}
{
        "_id" : "mongo-rs",
        "members" : [
                {
                        "_id" : 0,
                        "host" : "172.17.0.4:27017"
                },
                {
                        "_id" : 1,
                        "host" : "172.17.0.5:27017"
                },
                {
                        "_id" : 2,
                        "host" : "172.17.0.6:27017"
                }
        ]
}
> rs.initiate(config)
{ "ok" : 1 }
mongo-rs:SECONDARY> rs.status()
{
        "set" : "mongo-rs",
        "date" : ISODate("2022-02-20T09:43:56.591Z"),
        "myState" : 1,
        "term" : NumberLong(1),
        "syncSourceHost" : "",
        "syncSourceId" : -1,
        "heartbeatIntervalMillis" : NumberLong(2000),
        "majorityVoteCount" : 2,
        "writeMajorityCount" : 2,
        "votingMembersCount" : 3,
        "writableVotingMembersCount" : 3,
        "optimes" : {
                "lastCommittedOpTime" : {
                        "ts" : Timestamp(1645350235, 1),
                        "t" : NumberLong(1)
                },
                "lastCommittedWallTime" : ISODate("2022-02-20T09:43:55.636Z"),
                "readConcernMajorityOpTime" : {
                        "ts" : Timestamp(1645350235, 1),
                        "t" : NumberLong(1)
                },
                "appliedOpTime" : {
                        "ts" : Timestamp(1645350235, 1),
                        "t" : NumberLong(1)
                },
                "durableOpTime" : {
                        "ts" : Timestamp(1645350235, 1),
                        "t" : NumberLong(1)
                },
                "lastAppliedWallTime" : ISODate("2022-02-20T09:43:55.636Z"),
                "lastDurableWallTime" : ISODate("2022-02-20T09:43:55.636Z")
        },
        "lastStableRecoveryTimestamp" : Timestamp(1645350225, 1),
        "electionCandidateMetrics" : {
                "lastElectionReason" : "electionTimeout",
                "lastElectionDate" : ISODate("2022-02-20T09:40:05.507Z"),
                "electionTerm" : NumberLong(1),
                "lastCommittedOpTimeAtElection" : {
                        "ts" : Timestamp(1645349994, 1),
                        "t" : NumberLong(-1)
                },
                "lastSeenOpTimeAtElection" : {
                        "ts" : Timestamp(1645349994, 1),
                        "t" : NumberLong(-1)
                },
                "numVotesNeeded" : 2,
                "priorityAtElection" : 1,
                "electionTimeoutMillis" : NumberLong(10000),
                "numCatchUpOps" : NumberLong(0),
                "newTermStartDate" : ISODate("2022-02-20T09:40:05.598Z"),
                "wMajorityWriteAvailabilityDate" : ISODate("2022-02-20T09:40:05.694Z")
        },
        "members" : [
                {
                        "_id" : 0,
                        "name" : "172.17.0.4:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 2967,
                        "optime" : {
                                "ts" : Timestamp(1645350235, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2022-02-20T09:43:55Z"),
                        "lastAppliedWallTime" : ISODate("2022-02-20T09:43:55.636Z"),
                        "lastDurableWallTime" : ISODate("2022-02-20T09:43:55.636Z"),
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "electionTime" : Timestamp(1645350005, 1),
                        "electionDate" : ISODate("2022-02-20T09:40:05Z"),
                        "configVersion" : 1,
                        "configTerm" : 1,
                        "self" : true,
                        "lastHeartbeatMessage" : ""
                },
                {
                        "_id" : 1,
                        "name" : "172.17.0.5:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 241,
                        "optime" : {
                                "ts" : Timestamp(1645350235, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1645350235, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2022-02-20T09:43:55Z"),
                        "optimeDurableDate" : ISODate("2022-02-20T09:43:55Z"),
                        "lastAppliedWallTime" : ISODate("2022-02-20T09:43:55.636Z"),
                        "lastDurableWallTime" : ISODate("2022-02-20T09:43:55.636Z"),
                        "lastHeartbeat" : ISODate("2022-02-20T09:43:55.692Z"),
                        "lastHeartbeatRecv" : ISODate("2022-02-20T09:43:55.282Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncSourceHost" : "172.17.0.6:27017",
                        "syncSourceId" : 2,
                        "infoMessage" : "",
                        "configVersion" : 1,
                        "configTerm" : 1
                },
                {
                        "_id" : 2,
                        "name" : "172.17.0.6:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 241,
                        "optime" : {
                                "ts" : Timestamp(1645350235, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1645350235, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2022-02-20T09:43:55Z"),
                        "optimeDurableDate" : ISODate("2022-02-20T09:43:55Z"),
                        "lastAppliedWallTime" : ISODate("2022-02-20T09:43:55.636Z"),
                        "lastDurableWallTime" : ISODate("2022-02-20T09:43:55.636Z"),
                        "lastHeartbeat" : ISODate("2022-02-20T09:43:55.748Z"),
                        "lastHeartbeatRecv" : ISODate("2022-02-20T09:43:56.476Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncSourceHost" : "172.17.0.4:27017",
                        "syncSourceId" : 0,
                        "infoMessage" : "",
                        "configVersion" : 1,
                        "configTerm" : 1
                }
        ],
        "ok" : 1,
        "$clusterTime" : {
                "clusterTime" : Timestamp(1645350235, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1645350235, 1)
}
mongo-rs:PRIMARY>

error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-2XIQYfW1-1665384514813)(Mongodb.assets/1645325643721.png)]

https://stackoverflow.com/questions/70081140/mongodb-replica-set-cannot-use-non-local-read-concern-until-replica-set-is-fin

rs.initiate({"_id": "test-rep", "members": [{"_id":0, "host":  "172.30.0.2:27016"}, {"_id": 1, "host": "172.30.0.3:27017"}, {"_id":2, "host":  "172.30.0.4:27018"}]})

No host described in new configuration with {version: 1, term: 0} for replica set test-rep maps to this node

mongodb启动报错：key are too open

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-8N16lV6Q-1665384514813)(Mongodb.assets/1645346630692.png)]

docker run --name mongo-rs01 -p 30010:27017 -v /apps/mongo-rs/data01:/data/db -v /apps/mongo-rs/backup:/data/backup -v /apps/mongo-rs/key:/data/key -v /etc/localtime:/etc/localtime  -d mongo:5.0.6 --replSet mongo-rs  --bind_ip_all


62121136b89a69719f90735d
8262132356000000022B0229296E04

实时采集Mongodb

flinkCDC Mongodb demo

CREATE TABLE mongodb_sources_test ( 
 _id STRING, 
 name STRING, 
 age int, 
 PRIMARY KEY (_id) NOT ENFORCED
) WITH (
 'connector' = 'mongodb-cdc', 
 'hosts' = 'hadoop03:30010', 
 'username' = 'root',
 'password' = '123456', 
 'database' = 'school', 
 'collection' = 'student'
)

###

CREATE TABLE sink (
 _id STRING, 
 name STRING, 
 age int 
) WITH (
  'connector' = 'kafka',
  'topic' = 'cdc-test-1',
  'properties.bootstrap.servers' = 'hadoop01:9092',
  'format' = 'changelog-json' 
)

### 
insert into sink select * from mongodb_sources_test

https://ververica.github.io/flink-cdc-connectors/master/content/connectors/mongodb-cdc.html

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-yr5J4HuO-1665384514814)(Mongodb.assets/1645433251372.png)]

实时处理 change stream方式实现对mongodb的数据实时监控代码

https://blog.csdn.net/qq_35561207/article/details/85201501

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-cTipgHaN-1665384514815)(Mongodb.assets/1645428209600.png)]

ata/backup -v /apps/mongo-rs/key:/data/key -v /etc/localtime:/etc/localtime -d mongo:5.0.6 --replSet mongo-rs --bind_ip_all

62121136b89a69719f90735d
8262132356000000022B0229296E04




# 实时采集Mongodb

## flinkCDC Mongodb demo

```shell
CREATE TABLE mongodb_sources_test ( 
 _id STRING, 
 name STRING, 
 age int, 
 PRIMARY KEY (_id) NOT ENFORCED
) WITH (
 'connector' = 'mongodb-cdc', 
 'hosts' = 'hadoop03:30010', 
 'username' = 'root',
 'password' = '123456', 
 'database' = 'school', 
 'collection' = 'student'
)

###

CREATE TABLE sink (
 _id STRING, 
 name STRING, 
 age int 
) WITH (
  'connector' = 'kafka',
  'topic' = 'cdc-test-1',
  'properties.bootstrap.servers' = 'hadoop01:9092',
  'format' = 'changelog-json' 
)

### 
insert into sink select * from mongodb_sources_test

https://ververica.github.io/flink-cdc-connectors/master/content/connectors/mongodb-cdc.html

[外链图片转存中…(img-yr5J4HuO-1665384514814)]