MongoDB与大数据平台的整合（Hive、Spark）

一、 mongodb与hive整合

1.拷贝jar包到hive的lib目录

2.在hive中创建表，关联mongodb

create table individuals(
id int,
name string,
age int,
work STRUCT<title:string,hours:int>)
stored by "com.mongodb.hadoop.hive.MongoStorageHandler"
with serdeproperties
('mongo.columns.mapping'='{"id":"_id","work.title":"job.postition"}')
tblproperties
('mongo.uri'='mongodb://127.0.0.1:27017/test.persons');

insert into individuals select
1,"zhangsan",20,named_struct('title','beijing','hours',10);

2. mongodb加载csv

yum -y install mongodb-org-tools-4.0.1

示例train.csv
mongoimport --headerline --type=csv --file=./train.csv -d events -c train 
events--数据库名

3.mongodb与spark整合（无密码）

1.添加依赖

 <properties>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>