php批量插入mongo,mongodb批量处理

mongodb支持批量插入。

1.使用Java mongodb api

查看源码com.mongodb.MongoCollectionImpl，有两个方法

@Overridepublic void insertMany(final List extends TDocument>documents) {

insertMany(documents,newInsertManyOptions());

}

@Overridepublic void insertMany(final List extends TDocument> documents, finalInsertManyOptions options) {

notNull("documents", documents);

List requests = new ArrayList(documents.size());for(TDocument document : documents) {if (document == null) {throw new IllegalArgumentException("documents can not contain a null value");

}if (getCodec() instanceofCollectibleCodec) {

document= ((CollectibleCodec) getCodec()).generateIdIfAbsentFromDocument(document);

}

requests.add(newInsertRequest(documentToBsonDocument(document)));

}

executor.execute(newMixedBulkWriteOperation(namespace, requests, options.isOrdered(), writeConcern)

.bypassDocumentValidation(options.getBypassDocumentValidation()));

}insertMany(final List extends TDocument> documents) 默认使用 private boolean ordered = true;即有序插入。insertMany(final List extends TDocument> documents, final InsertManyOptions options)调用者自己设置。

ordered属性有什么用？

/*** Gets whether the documents should be inserted in the order provided, stopping on the first failed insertion. The default is true.

* If false, the server will attempt to insert all the documents regardless of an failures.

*

*@returnwhether the the documents should be inserted in order*/

public booleanisOrdered() {returnordered;

}

大概意思是：

true：按提供的顺序插入文档，并在首次插入失败时停止。 (按顺序插入文档，遇到失败后停止。之前已经插入成功的不返回，失败及失败之后的不插入。)

false: 服务器将尝试插入所有文档，而不考虑失败。(只要能插入成功的，都存储进数据库)

2.spring-data-mongodb

org.springframework.data.mongodb.core.MongoTemplate

以下是该方法的源码和使用案例：

/** (non-Javadoc)

* @see org.springframework.data.mongodb.core.ExecutableInsertOperation#bulkOps(org.springframework.data.mongodb.core.BulkMode, java.lang.String)*/

publicBulkOperations bulkOps(BulkMode bulkMode, String collectionName) {return bulkOps(bulkMode, null, collectionName);

}

public intbatchInsertStudents() {BulkWriteResult result = null;try{

List documents = new ArrayList<>();

String collectionName = "myTestCol";BulkOperations bulkOp= this.mongoTemplate.bulkOps(BulkMode.UNORDERED, collectionName);

for(int i = 502610; i< 2000000; i++) {

Student student= newStudent();

student.setId(String.valueOf(i));

student.setAge(i);

student.setGender("男");

student.setName("李三"+i);

documents.add(student);

}

bulkOp.insert(documents);result=bulkOp.execute();

}catch(DuplicateKeyException e) {

System.out.println("**********" +e.getMessage());}return result;

}

有两种模式：

/*** Mode for bulk operation.

**/

enumBulkMode {/**Perform bulk operations in sequence. The first error will cancel processing.*/ORDERED,/**Perform bulk operations in parallel. Processing will continue on errors.*/UNORDERED

};

与之前的order(true|false)相对应。