MongoDB 是一个跨平台的,面向文档的数据库,提供高性能,高可用性和可扩展性方便。 MongoDB 工作在收集和文件的概念。
数据库
数据库是一个物理容器集合。每个数据库都有自己的一套文件系统上的文件。一个单一的MongoDB服务器通常有多个数据库。
集合
集合是一组MongoDB的文档。它相当于一个RDBMS表。收集存在于一个单一的数据库。集合不执行模式。集合内的文档可以有不同的领域。通常情况下,一个集合中的所有文件是相同或相关的目的。
文档
文档是一组键 - 值对。文件动态模式。动态模式是指,在相同集合中的文档不需要具有相同的字段或结构组的公共字段的集合的文档,可以容纳不同类型的数据。
下面给出的表显示RDBMS术语使用 MongoDB 的关系
RDBMS | MongoDB |
---|---|
Database | Database |
Table | Collection |
Tuple/Row | Document |
column | Field |
Table Join | Embedded Documents |
Primary Key | Primary Key (Default key _id provided by mongodb itself) |
数据库服务器和客户端 | |
Mysqld/Oracle | mongod |
mysql/sqlplus | mongo |
示例文档
下面给出的示例显示了一个博客网站,这简直是一个逗号分隔的键值对文档结构。
{
_id: ObjectId(7df78ad8902c)
title: 'MongoDB Overview',
description: 'MongoDB is no sql database',
by: 'tutorials point',
url: 'http://www.yiibai.com',
tags: ['mongodb', 'database', 'NoSQL'],
likes: 100,
comments: [
{
user:'user1',
message: 'My first comment',
dateCreated: new Date(2011,1,20,2,15),
like: 0
},
{
user:'user2',
message: 'My second comments',
dateCreated: new Date(2011,1,25,7,45),
like: 5
}
]
}
_id是一个12字节的十六进制数,保证每一份文件的唯一性。您可以提供_id同时插入文档。如果没有提供,那么MongoDB的每个文档提供了一个独特的ID。这12个字节,前4个字节为当前时间戳,未来3个字节的机器ID,接下来的2个字节的进程id MongoDB的服务器及剩余3个字节是简单的增量值。
MongoDB比RDBMS的优势
-
架构:MongoDB是文档型数据库,其中一个集合保存不同的不同的文件。字段的数量,内容和该文件的大小可以是不同于从一个文件复制到另一个。
-
一个单一的对象是结构清晰
-
没有复杂的连接
-
深查询能力。 MongoDB支持动态查询使用基于文档的查询语言,如SQL几乎一样强大的文件
-
调优
-
易于规模化:MongoDB是易于扩展
-
不需要数据库对象的应用程序对象转换/映射
-
使用内部存储器存储(窗口)工作组,从而实现更快的数据存取
为什么要使用MongoDB
- JSON风格文件的形式,面向文档存储:数据存储
-
对任何属性可索引
-
复制和高可用性
-
自动分片
-
丰富的查询
-
快速就地更新
-
MongoDB的专业技术支持
应该在哪里使用MongoDB?
-
大数据
-
内容管理和交付
-
移动和社交基础设施
-
用户数据管理
-
数据平台
应用实例 使用Java Driver访问MongoDB (API : http://api.mongodb.org/java/current/overview-summary.html)
1、MongoDB中的collections就是一系列 BSON documents的集合,相当于关系数据库中的表。
package com.hcm.mongodb.demo;
import java.net.UnknownHostException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.apache.commons.lang.StringUtils;
import org.bson.types.ObjectId;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.mongodb.BasicDBList;
import com.mongodb.BasicDBObject;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.DBCursor;
import com.mongodb.DBObject;
import com.mongodb.Mongo;
import com.mongodb.MongoClient;
import com.mongodb.WriteResult;
import com.mongodb.util.JSON;
public class MongoDBDemo {
private static final Logger logger = LoggerFactory.getLogger(MongoDBFileDemo.class);
private static final String DEFAULT_DB_NAME = "mongo-demo";
private Mongo mongo = null;
@Before
public void init() {
try {
mongo = new MongoClient("127.0.0.1", 27017);
} catch (UnknownHostException e) {
logger.error("", e);
}
}
@After
public void destory() {
if (mongo != null) {
mongo.close();
}
}
public DBCollection getCollection(String dbName, String collectionName) {
return getDB(dbName).getCollection(collectionName);
}
public DBCollection getCollection(String collectionName) {
return getDB().getCollection(collectionName);
}
private DB getDB(String dbName) {
if (StringUtils.isNotEmpty(dbName)) {
return mongo.getDB(dbName);
}
return mongo.getDB(DEFAULT_DB_NAME);
}
private DB getDB() {
return mongo.getDB(DEFAULT_DB_NAME);
}
@Test
public void add() {
DBCollection users = getCollection("user");
DBObject obj = new BasicDBObject();
obj.put("name", "Jiang");
obj.put("age", "18");
obj.put("sex", "女");
users.save(obj);
}
@Test
public void insert() {
DBCollection users = getCollection("user");
DBObject obj = new BasicDBObject();
obj.put("name", "hello");
obj.put("age", "18");
obj.put("sex", "男");
users.insert(obj);
}
@Test
public void inserts() {
DBCollection users = getCollection("user");
List<DBObject> objs = new ArrayList<DBObject>();
DBObject obj1 = new BasicDBObject();
obj1.put("name", "wqqe");
obj1.put("age", "19");
obj1.put("sex", "女");
DBObject obj2 = new BasicDBObject();
obj2.put("name", "aaa");
obj2.put("age", "19");
obj2.put("sex", "女");
objs.add(obj1);
objs.add(obj2);
users.insert(objs);
Map<String, Object> map = new HashMap<String, Object>();
map.put("name", "kkkk");
map.put("age", 9);
map.put("sex", "XXXooo");
users.insert(new BasicDBObject("name", "hello"), new BasicDBObject(map));
}
@Test
public void queryAll() {
DBCollection users = getCollection("user");
DBCursor cursor = users.find();
// while (cursor.hasNext()) {
// System.out.println(cursor.next());
// }
System.out.println(JSON.serialize(cursor));
}
@Test
public void remove() {
DBCollection users = getCollection("user");
DBObject o = new BasicDBObject();
//o.put("_id", new ObjectId("54990c38363078af81f3d950"));
o.put("firstName", "hello");
users.remove(o);
}
@Test
public void update() {
DBCollection users = getCollection("user");
// WriteResult result = users.update(
// new BasicDBObject("_id", new ObjectId("54990f5836300c9ca6bd0e19")),
// new BasicDBObject("age", 99));
WriteResult result = users.update(
new BasicDBObject("_id", new ObjectId("54990f5836300c9ca6bd0e19")),
new BasicDBObject("age", 121), true, false);
System.out.println(result.getN());
}
/**
* OR多条件查询
*/
public void findByORQuery(int lte, int gt, long longData) {
DBCollection coll = getCollection("user");
BasicDBObject query = new BasicDBObject();
BasicDBObject longdata = new BasicDBObject("longData", longData);
BasicDBObject intdata = new BasicDBObject("intData", new BasicDBObject().append("$gt", gt).append("$lte", lte));
BasicDBList cond = new BasicDBList();
cond.add(longdata);
cond.add(intdata);
query.put("$or", cond);
DBCursor cur = coll.find(query);
while (cur.hasNext()) {
System.out.println(cur.next());
}
}
/**
* IN查询
*/
public void findByINQuery(int value1, int value2) {
DBCollection coll = getCollection(null, "user");
BasicDBObject query = new BasicDBObject();
BasicDBList cond = new BasicDBList();
cond.add(value1);
cond.add(value2);
query.put("intData", new BasicDBObject("$in", cond));
DBCursor cur = coll.find(query);
while (cur.hasNext()) {
System.out.println(cur.next());
}
}
/**
* NOT查询
*/
public void findByNotQuery(int value1, int value2) {
DBCollection coll = getCollection(null, "user");
BasicDBObject query = new BasicDBObject();
BasicDBList cond = new BasicDBList();
cond.add(value1);
cond.add(value2);
query.put("intData", new BasicDBObject("$nin", cond));
System.out.println("Count:" + coll.find(query).count());
}
/**
* 获取结果集第一条
*/
public void fetchFirstQuery(int value1, int value2) {
DBCollection coll = getCollection(null, "user");
BasicDBList cond = new BasicDBList();
cond.add(value1);
cond.add(value2);
BasicDBObject query = new BasicDBObject().append("intData", new BasicDBObject("$nin", cond));
System.out.println(coll.findOne(query));
}
/**
* 查询文档部分列
*/
public void querySomeKey() {
DBCollection coll = getCollection("user");
DBCursor cur = coll.find(new BasicDBObject(), new BasicDBObject("intData", true));
while (cur.hasNext()) {
System.out.println(cur.next());
}
}
/**
* 查询内嵌文档
*/
public void queryInnerDocument() {
DBCollection coll = getCollection("user");
BasicDBObject map = new BasicDBObject();
map.put("innertype", "string");
map.put("innerContent", "string0");
DBCursor cur = coll.find(new BasicDBObject("documentData", map));
while (cur.hasNext()) {
System.out.println(cur.next());
}
}
/**
* 查询内嵌部分文档
*/
public void querySubInnerDocument() {
DBCollection coll = getCollection("user");
DBCursor cur = coll.find(new BasicDBObject("documentData.innerContent", "string0"));
while (cur.hasNext()) {
System.out.println(cur.next());
}
}
/**
* 查询分页文档
*/
public void queryByPage(int skipNum, int pageNum) {
DBCollection coll = getCollection("user");
DBCursor cur = coll.find().skip(skipNum).limit(pageNum);
while (cur.hasNext()) {
System.out.println(cur.next());
}
}
/**
* 查询文档某列是否存在
*/
public void queryExists() {
DBCollection coll = getCollection("user");
DBCursor cur = coll.find(new BasicDBObject("longData", new BasicDBObject("$exists", true)));
while (cur.hasNext()) {
System.out.println(cur.next());
}
}
/**
* 查询文档排序
*/
public void sortDocument() {
DBCollection coll = getCollection("user");
DBCursor cur = coll.find().sort(new BasicDBObject("intData", -1));// 1:asc
// /
// -1:desc
while (cur.hasNext()) {
System.out.println(cur.next());
}
}
/**
* 获取根据某元素做distinct查询.
*/
@SuppressWarnings("unchecked")
public void distinctKey() {
DBCollection coll = getCollection("user");
List<String> list = coll.distinct("documentData.innertype");
for (String str : list) {
System.out.println(str);
}
}
/**
* 创建唯一索引
*/
public void createIndexes() {
DBCollection coll = getCollection("user");
BasicDBObject index = new BasicDBObject();
index.put("intData", 1);// 1:asc / -1:desc
index.put("unique", true);// 唯一索引
coll.createIndex(index);
}
/**
* 查询索引信息
*/
public void getIndexes() {
DBCollection coll = getCollection("user");
List<DBObject> indexInfo = coll.getIndexInfo();
System.out.println(indexInfo);
}
/**
* 删除索引信息
*/
public void dropIndexes() {
DBCollection coll = getCollection("user");
// 删除的索引必须跟创建的索引名称\排序\是否唯一都相同才能删除
BasicDBObject index = new BasicDBObject();
index.put("intData", 1);
index.put("unique", true);
coll.dropIndex(index);
}
}
2、GridFS是MongoDB中的一个内置功能,可以用于存放大量小文件。
package com.hcm.mongodb.demo;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.apache.commons.lang.StringUtils;
import org.bson.types.ObjectId;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.mongodb.BasicDBObject;
import com.mongodb.Cursor;
import com.mongodb.DB;
import com.mongodb.DBObject;
import com.mongodb.Mongo;
import com.mongodb.MongoClient;
import com.mongodb.gridfs.GridFS;
import com.mongodb.gridfs.GridFSDBFile;
import com.mongodb.gridfs.GridFSInputFile;
public class MongoDBFileDemo {
private static final Logger logger = LoggerFactory.getLogger(MongoDBFileDemo.class);
private static final String FILE_NAME = "fileName";
private static final String DEFAULT_DB_NAME = "mongo-demo";
private Mongo mongo = null;
@Before
public void init() {
try {
mongo = new MongoClient("127.0.0.1", 27017);
} catch (Exception e) {
logger.error("", e);
}
}
@After
public void destory() {
if (mongo != null) {
mongo.close();
}
}
private GridFS getGridFS() {
return new GridFS(getDB());
}
private GridFS getGridFS(String dbName) {
return new GridFS(getDB());
}
private DB getDB(String dbName) {
if (StringUtils.isNotEmpty(dbName)) {
return getDB(dbName);
}
return getDB(DEFAULT_DB_NAME);
}
private DB getDB() {
return getDB(DEFAULT_DB_NAME);
}
/**
* 保存文件
*
* @param dbName
* @param byteFile
* @param fileName
* @param fileType
* @return
*/
public String saveFile(String dbName, byte[] byteFile, String fileName, String fileType) {
if (StringUtils.isEmpty(fileName)) {
return null;
}
GridFS fs = getGridFS(dbName);
GridFSInputFile dbFiel = null;
try {
dbFiel = fs.createFile(byteFile);
dbFiel.setContentType(fileType);
dbFiel.setFilename(fileName);
dbFiel.put(FILE_NAME, fileName);
dbFiel.save();
} catch (Exception e) {
logger.error("", e);
}
return dbFiel.getId().toString();
}
/**
* 保存文件
*
* @param dbName
* @param fileName
* @param fileType
* @return
*/
public String saveFile(String dbName, String fileName, String fileType) {
if (StringUtils.isEmpty(fileName)) {
return null;
}
String fName = fileName.substring(fileName.lastIndexOf("/") + 1);
GridFS fs = getGridFS();
GridFSInputFile dbFile = null;
try {
dbFile = fs.createFile(new File(fileName));
dbFile.setFilename(fName);
dbFile.setContentType(fileType);
dbFile.put(FILE_NAME, fName);
dbFile.save();
} catch (IOException e) {
logger.error("", e);
}
return dbFile.getId().toString();
}
private byte[] file2Bytes(File file) {
FileInputStream fis = null;
byte[] byteFile = null;
try {
fis = new FileInputStream(file);
int lenth = fis.available();
byteFile = new byte[lenth];
int temp = 0, i = 0;
while ((temp = fis.read()) != -1) {
byteFile[i] = (byte) temp;
i++;
}
} catch (IOException e) {
e.printStackTrace();
}
return byteFile;
}
/**
* 按fileId获取文件
*
* @param dbName
* @param fileId
* @return
*/
public GridFSDBFile getFileById(String dbName, String fileId) {
GridFS fs = getGridFS(dbName);
DBObject query = new BasicDBObject("_id", new ObjectId(fileId));
GridFSDBFile dbFile = fs.findOne(query);
return dbFile;
}
/**
* 按fileName获取文件
*
* @param dbName
* @param fileName
* @return
*/
public GridFSDBFile getFileByName(String dbName, String fileName) {
GridFS fs = getGridFS(dbName);
DBObject query = new BasicDBObject(FILE_NAME, fileName);
GridFSDBFile dbFile = fs.findOne(query);
return dbFile;
}
/**
* 按fielId删除文件
*
* @param dbName
* @param fileId
*/
public void deleteFileById(String dbName, String fileId) {
if (fileId == null) {
return;
}
GridFS fs = getGridFS(dbName);
GridFSDBFile dbFile = fs.findOne(new ObjectId(fileId));
if (dbFile == null) {
return;
}
fs.remove(dbFile);
}
/**
* 按fileName删除文件
*
* @param dbName
* @param fileName
*/
public void deleteFileByName(String dbName, String fileName) {
if (StringUtils.isEmpty(fileName)) {
return;
}
GridFS fs = getGridFS(dbName);
DBObject dbObject = new BasicDBObject(FILE_NAME, fileName);
GridFSDBFile dbFile = fs.findOne(dbObject);
if (dbFile == null) {
return;
}
fs.remove(dbFile);
}
/**
* 按fielId读取文件
*
* @param dbName
* @param fileId
* @return
*/
public byte[] readFileById(String dbName, String fileId) {
if (StringUtils.isEmpty(fileId)) {
return null;
}
GridFS fs = getGridFS(dbName);
GridFSDBFile dbFile = fs.findOne(new ObjectId(fileId));
if (dbFile == null) {
return null;
}
return readFile(dbFile);
}
/**
* 按fielName读取文件
*
* @param dbName
* @param fileName
* @return
*/
public byte[] readFileByName(String dbName, String fileName) {
if (StringUtils.isEmpty(fileName)) {
return null;
}
GridFS fs = getGridFS(dbName);
DBObject dbObject = new BasicDBObject(FILE_NAME, fileName);
GridFSDBFile dbFile = fs.findOne(dbObject);
if (dbFile == null) {
return null;
}
return readFile(dbFile);
}
/**
* GridFSDBFile 转存成 byte[]
*
* @param dbFile
* @return
*/
private byte[] readFile(GridFSDBFile dbFile) {
if (dbFile == null) {
return null;
}
InputStream is = null;
byte[] resultByte = null;
try {
is = dbFile.getInputStream();
int lenth = (int) dbFile.getLength();
resultByte = new byte[lenth];
int temp = 0, i = 0;
while ((temp = is.read()) != -1 && i < lenth) {
resultByte[i] = (byte) temp;
i++;
}
} catch (IOException e) {
logger.error("", e);
} finally {
if (is != null) {
try {
is.close();
} catch (IOException e) {
logger.error("", e);
}
}
}
return resultByte;
}
/**
* 按fileId读文件保存到本地
*
* @param dbName
* @param fileId
* @param localFileName
*/
public void readFileToLocal(String dbName, String fileId, String localFileName) {
if (StringUtils.isEmpty(fileId)) {
return;
}
GridFS fs = getGridFS(dbName);
GridFSDBFile dbFile = fs.findOne(new ObjectId(fileId));
if (dbFile == null) {
return;
}
File file = new File(localFileName);
try {
dbFile.writeTo(file);
} catch (IOException e) {
logger.error("", e);
}
}
/**
* 按finlName读文件保存到本地
*
* @param dbName
* @param fileName
* @param localFileName
*/
public void readFileToLocalByName(String dbName, String fileName, String localFileName) {
if (StringUtils.isEmpty(fileName)) {
return;
}
GridFS fs = getGridFS(dbName);
GridFSDBFile dbFile = fs.findOne(new BasicDBObject(FILE_NAME, fileName));
if (dbFile == null) {
return;
}
File file = new File(localFileName);
try {
dbFile.writeTo(file);
} catch (IOException e) {
logger.error("", e);
}
}
/**
* 根据fielId获得文件名称
*
* @param dbName
* @param fileId
* @return
*/
public String getFileName(String dbName, String fileId) {
if (StringUtils.isEmpty(fileId)) {
return null;
}
GridFS fs = getGridFS(dbName);
GridFSDBFile dbFile = fs.findOne(new ObjectId(fileId));
if (dbFile == null) {
return null;
}
return dbFile.get(FILE_NAME).toString();
}
/**
* 更新文件
*
* @param dbName
* @param byteFile
* @param fileId
* @param fileName
* @param fileType
* @return
*/
public String updateFile(String dbName, byte[] byteFile, String fileId, String fileName, String fileType) {
if (byteFile == null || StringUtils.isEmpty(fileId)) {
return null;
}
GridFS fs = getGridFS(dbName);
GridFSInputFile dbFile = fs.createFile(byteFile);
// 设置文件Id
dbFile.setId(new ObjectId(fileId));
dbFile.setContentType(fileType);
dbFile.setFilename(fileName);
dbFile.put("fileName", fileName);
// 删除
deleteFileById(dbName, fileId);
// 保存
dbFile.save();
return dbFile.getId().toString();
}
/**
* 获得最后更新时间
*
* @param dbName
* @param fileId
* @return
*/
public Date readUpdateTime(String dbName, String fileId) {
if (fileId == null) {
return null;
}
GridFS fs = getGridFS(dbName);
GridFSDBFile dbFile = fs.findOne(new ObjectId(fileId));
if (dbFile == null) {
return null;
}
try {
return dbFile.getUploadDate();
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
/**
* 带条件查询
*
* @param dbName
* @param condition
* @return
*/
public List<GridFSDBFile> queryWithCondition(String dbName, Map<String, String> condition) {
if (condition == null) {
return null;
}
GridFS fs = getGridFS(dbName);
DBObject ob = new BasicDBObject();
for (String key : condition.keySet()) {
ob.put(key, condition.get(key));
}
List<GridFSDBFile> dbFiles = fs.find(ob);
return dbFiles;
}
@Test
public void getFileList() {
GridFS fs = new GridFS(getDB());
Cursor cursor = fs.getFileList();
while (cursor.hasNext()) {
System.out.println(cursor.next());
}
}
@Test
public void testSaveFile1() {
String fileName = "test1.properties";
String fileType = "java";
String id = saveFile(DEFAULT_DB_NAME, fileName, fileType);
System.out.println(id);
}
@Test
public void testSaveFile2() {
String fileName = "log4j.properties";
String fileType = "properties";
File file = new File("src/log4j.properties");
String id = saveFile(DEFAULT_DB_NAME, file2Bytes(file), fileName, fileType);
System.out.println(id);
}
@Test
public void testFindOne() {
GridFSDBFile dbFile = getFileById(DEFAULT_DB_NAME, "54b6239b63c8538e4ff050d7");
System.out.println(dbFile);
GridFSDBFile dbFile2 = getFileByName(DEFAULT_DB_NAME, "log4j.properties");
System.out.println(dbFile2);
}
@Test
public void testDelete() {
deleteFileById(DEFAULT_DB_NAME, "54b612a363c86e415d5c2742");
deleteFileByName(DEFAULT_DB_NAME, "MongoTest.java");
}
@Test
public void testRead() {
// 读文件
byte[] data = readFileById(DEFAULT_DB_NAME, "54b61dd163c8f5061b147953");
System.out.println(data.length/1024 + "KB");
byte[] data2 = readFileByName(DEFAULT_DB_NAME, "log4j.properties");
System.out.println(data2.length/1024 + "KB");
// 读mongodb文件到本地
String localFileName = "test1.properties";
readFileToLocal(DEFAULT_DB_NAME, "54b61dd163c8f5061b147953", localFileName);
String localFileName2 = "test2.properties";
readFileToLocalByName(DEFAULT_DB_NAME, "log4j.properties", localFileName2);
}
@Test
public void testUpdate() {
// 54b6239b63c8538e4ff050d7
String id = updateFile(DEFAULT_DB_NAME, file2Bytes(new File("test1.properties")), "54b6239b63c8538e4ff050d7", "test2.properties", "properties");
System.out.println(id);
System.out.println(readUpdateTime(DEFAULT_DB_NAME, "54b6239b63c8538e4ff050d7"));
}
@Test
public void testQuery() {
Map<String, String> map = new HashMap<String, String>();
map.put("fileName", "test2.properties");
List<GridFSDBFile> dbFileList = queryWithCondition(DEFAULT_DB_NAME, map);
if (dbFileList != null && dbFileList.size() > 0) {
for (GridFSDBFile dbFile : dbFileList) {
System.out.println(dbFile);
}
}
}
}