MongoDB数据导入Hive
方法和Hbase导入hive类似,区别在于,需要现在Mongodb设置一个账户
创建用户
进入mongodb后开始操作:
//创建管理员账户 userAdminAnyDatebase 为管理员权限,可以管理,不能关闭数据库
> use admin
switched to db admin
> db.createUser({user:"admin",pwd:"admin",roles:[{"role":"userAdminAnyDatabase","db":"admin"}]})
Successfully added user: {
"user" : "admin",
"roles" : [
{
"role" : "userAdminAnyDatabase",
"db" : "admin"
}
]
}
//创建root用户, root只在admin中有用
> db.createUser({user: "root",pwd: "password", roles: [ { role: "root", db: "admin" } ]})
Successfully added user: {
"user" : "root",
"roles" : [
{
"role" : "root",
"db" : "admin"
}
]
}
//创建自己数据库(kgcdsj)的用户,dbOwner代表数据库所有者,拥有所有权限,一定要先use 库名之后再创建,否则创建的用户还在admin下面,不会在我们的数据库下面
> use kgcdsj
switched to db kgcdsj
> db.createUser({user: "kgcuser",pwd: "123456",roles: [ { role: "dbOwner", db: "kgcdsj" } ]})
Successfully added user: {
"user" : "kgcuser",
"roles" : [
{
"role" : "dbOwner",
"db" : "kgcdsj"
}
]
}
//删除用户
//删除用户必须由账号管理员来删,所以,切换到admin角色
use admin
db.auth("admin","password")
//删除单个用户 慎用
db.system.users.remove({user:"XXXXXX"})
//删除所有用户 慎用
db.system.users.remove({})
导JAR包
如图所示三个jar包 放入 hive的lib 路径内
也可以利用maven下载这三个jar包,命令如下:
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongodb-java-driver</artifactId>
<version>3.2.1</version>
</dependency>
<dependency>
<groupId>org.mongodb.mongo-hadoop</groupId>
<artifactId>mongo-hadoop-core</artifactId>
<version>2.0.2</version>
</dependency>
<dependency>
<groupId>org.mongodb.mongo-hadoop</groupId>
<artifactId>mongo-hadoop-hive</artifactId>
<version>2.0.2</version>
</dependency>
开始导数据
//启动命令:
./bin/mongod -config ./mongo.conf
//查询启动状态命令:
ps -ef | grep mongo
//进入shell
./bin/mongo
//输入密码 需要先use 库名之后再输入密码,否则无法启动成功
db.auth("kgcuser","123456)
导数据的代码如下:
//从mongodb 映射到hive
create external table ${db}.mg_train(
user_id String,
event_id String,
invited String,
timestamp String,
interested String
)
stored by 'com.mongodb.hadoop.hive.MongoStorageHandler' with serdeproperties
('mongo.columns.mapping'='
{
"user_id":"user",
"event_id":"event",
"invited":"invited",
"timestamp":"timestamp",
"interested":"interested"}
')
tblproperties ('mongo.uri'='mongodb://kgcuser:123456@192.168.146.222:27017/kgcdsj.train');