一、概述
本文介绍在java web工程中使用Berkeley DB JE作为内存数据库来存取数据。当业务逻辑不太复杂时,通常可以代替在关系型数据库中设计复杂的sql和表结构。
当数据量不太夸张(BDB)时,这种方案的优点,除了BDB自身的种种优点外还有:
1,代码简单,易于维护。且数据持久化存储在指定目录中。
2,存取高效,经实际项目(高并发)测试,附带附加的计算返回均为毫秒级别。
3,易于扩展,支持任意java对象的增删查改。
4,可实现的接口丰富,支持交叉、关联查询等。
5,事务支持。
6,支持一级、二级索引
二、最佳实践
最佳实践:存取Car数据。
如上图,主要存取过程主要由4各类完成,其中:
1,BDBEnvironment.java: 封装BDB自身的Environment类,EnvironmentConfig类。指定数据存放的目录。
/**
* BDB环境包装器
* @author craig
*
*/
public class BDBEnvironment {
private Environment env;
private StoreConfig storeConfig;
private String envHome;
public BDBEnvironment(String envHome){
System.out.println("envHome:" + envHome);
this.envHome = envHome;
init();
}
public BDBEnvironment(String envHome, boolean deleteFolderContents){
System.out.println("envHome:" + envHome);
this.envHome = envHome;
if(deleteFolderContents) {
JFileUtils.deleteFolderContents(new File(envHome));
}
init();
}
public void init() {
EnvironmentConfig myEnvConfig = new EnvironmentConfig();
myEnvConfig.setReadOnly(false);
myEnvConfig.setTransactional(true);
myEnvConfig.setAllowCreate(true);
storeConfig = new StoreConfig();
storeConfig.setReadOnly(false);
storeConfig.setAllowCreate(true);
storeConfig.setTransactional(true);
env = new Environment(new File(envHome), myEnvConfig);
}
public Environment getEnv() {
return env;
}
public void setEnv(Environment env) {
this.env = env;
}
public StoreConfig getStoreConfig() {
return storeConfig;
}
public void setStoreConfig(StoreConfig storeConfig) {
this.storeConfig = storeConfig;
}
}
2,AbstracStore.java: 数据库io抽象类,增删查改的各个操作的公用方法,提交事务、关闭数据库等。
public abstract class AbstractStore {
protected final int MAX_RETRY = 5; // Used for while loop and deadlock retries
protected EntityStore store;
//protected Transaction txn;
//放需要commit和clean的Transaction
ConcurrentHashMap<EntityCursor<? extends Object>, Transaction> cursorTransactionMap
= new ConcurrentHashMap<EntityCursor<? extends Object>, Transaction>();
//-----Construction Method------
public AbstractStore(BDBEnvironment env){
store = new EntityStore(env.getEnv(), "EntityStore", env.getStoreConfig());
}
public void clean(){
store.getEnvironment().cleanLog();
}
/**
* entity cursor used after getEntityCursor() and operations.
*
* **/
public void cursorCommitAndClean(EntityCursor<? extends Object> cursor){
Transaction txn = cursorTransactionMap.get(cursor); //取到txn
cursorTransactionMap.remove(cursor);
boolean retry = true;
int retry_count = 0;
cursor.close();
cursor = null;
while (retry) {
try {
try {
//txn.commit();
txn.commit();
txn = null;
} catch (DatabaseException e) {
System.err.println("Error on txn commit: " + e.toString());
}
retry = false;
} catch (LockConflictException de) {
System.out.println("BDB: Deadlock");
// retry if necessary
if (retry_count < MAX_RETRY) {
retry = true;
retry_count++;
} else {
System.err.println("BDB: Out of retries[commit entityCursor]. Giving up.");
retry = false;
}
} catch (DatabaseException e) {
retry = false; // abort and don't retry
System.err.println("BDB exception: " + e.toString());
e.printStackTrace();
} finally {
if (cursor != null) {
cursor.close();
}
if (txn != null) {
try {
txn.abort();
} catch (Exception e) {
System.err.println("Error aborting transaction: " + e.toString());
e.printStackTrace();
}
}
}
}
}
// Close the store and environment
public void close() {
System.out.println("close dataBase.");
if (store != null ) {
try {
store.close();
} catch (DatabaseException e) {
System.err.println("closeEnv: store: " + e.toString());
e.printStackTrace();
}
}
if (store.getEnvironment() != null ) {
try {
store.getEnvironment().cleanLog();
store.getEnvironment().close();
} catch (DatabaseException e) {
System.err.println("closeEnv: " + e.toString());
e.printStackTrace();
}
}
}
public EntityStore getStore() {
return store;
}
public void setStore(EntityStore store) {
this.store = store;
}
}
上述两个类是基础核心类。
3,CarScore.java: 需要持久化存储的java对象,即POJO类、元数据。
@Entity
public class CarScore {
@PrimaryKey
private String carId;
private String name; //车型名
private String model; //车型code
private Long price; //车价(按车价差公式计算结果降权即可,不作为硬性索引条件)
@SecondaryKey(relate = Relationship.MANY_TO_ONE) private String brand; //品牌
@SecondaryKey(relate = Relationship.MANY_TO_ONE) private String series; //车系
private Integer year; //年限(上牌年份)
@SecondaryKey(relate = Relationship.MANY_TO_ONE) private String carType; //车形
@SecondaryKey(relate = Relationship.MANY_TO_ONE) private String color; //颜色
@SecondaryKey(relate = Relationship.MANY_TO_ONE) private String capacity; //排量
@SecondaryKey(relate = Relationship.MANY_TO_ONE) private Long miles; //里程
@SecondaryKey(relate = Relationship.MANY_TO_ONE) private String geerbox; //变速箱
@SecondaryKey(relate = Relationship.MANY_TO_ONE) private String country; //国别
//
private String status; //状态
private String level; //车辆等级
private String upShelfDate; //第一次上架时间
private String licenseDate; //上牌时间
private Double score;
public CarScore(){}
//getter & setter method....
}
4,CarScoreStore.java: 数据库操作(增删查改)的操作类。需要继承AbstractStore.java
public class CarScoreStore extends AbstractStore {
public PrimaryIndex<String, CarScore> carScoreById;
public SecondaryIndex<String, String, CarScore> carScoreByBrand;
public SecondaryIndex<String, String, CarScore> carScoreBySeries;
public SecondaryIndex<String, String, CarScore> carScoreByType;
//public SecondaryIndex<String, String, CarScore> carScoreByYear;
//-----Construction Method------
public CarScoreStore(BDBEnvironment env) {
super(env);
carScoreById = store.getPrimaryIndex(String.class, CarScore.class); ///primary key
carScoreByBrand = store.getSecondaryIndex(carScoreById, String.class, "brand");
carScoreBySeries = store.getSecondaryIndex(carScoreById, String.class, "series");
carScoreByType = store.getSecondaryIndex(carScoreById, String.class, "carType");
//carScoreByYear = store.getSecondaryIndex(carScoreById, String.class, "year");
}
//-------Operation Method--------
/**
* add a record
*
* **/
public void put(CarScore entity) {
boolean retry = true;
int retry_count = 0;
while (retry) {
Transaction txn = store.getEnvironment().beginTransaction(null, null);
carScoreById.put(txn, entity);
try {
try {
txn.commit();
txn = null;
} catch (DatabaseException e) {
System.err.println("Error on txn commit: " + e.toString());
}
retry = false;
} catch (LockConflictException de) {
System.out.println("BDB: Deadlock");
if (retry_count < MAX_RETRY) { // retry if necessary
retry = true;
retry_count++;
} else {
System.err.println("BDB: Out of retries[put]. Giving up.");
retry = false;
}
} catch (DatabaseException e) {
retry = false; // abort and don't retry
System.err.println("BDB exception: " + e.toString());
e.printStackTrace();
} finally {
if (txn != null) {
try {
txn.abort();
} catch (Exception e) {
System.err.println("Error aborting transaction: " + e.toString());
e.printStackTrace();
}
}
}
}
}
/**
* add a record
*
* **/
public void delete(String carId) {
boolean retry = true;
int retry_count = 0;
while (retry) {
Transaction txn = store.getEnvironment().beginTransaction(null, null);
carScoreById.delete(txn, carId);
try {
try {
txn.commit();
txn = null;
} catch (DatabaseException e) {
System.err.println("Error on txn commit: " + e.toString());
}
retry = false;
} catch (LockConflictException de) {
System.out.println("BDB: Deadlock");
if (retry_count < MAX_RETRY) { // retry if necessary
retry = true;
retry_count++;
} else {
System.err.println("BDB: Out of retries[put]. Giving up.");
retry = false;
}
} catch (DatabaseException e) {
retry = false; // abort and don't retry
System.err.println("BDB exception: " + e.toString());
e.printStackTrace();
} finally {
if (txn != null) {
try {
txn.abort();
} catch (Exception e) {
System.err.println("Error aborting transaction: " + e.toString());
e.printStackTrace();
}
}
}
}
}
/**
* search BDBItem by id
*
* **/
public CarScore get(String carId){
boolean retry = true;
int retry_count = 0;
TransactionConfig txnConfig = new TransactionConfig();
txnConfig.setReadCommitted(true);
CarScore carScore = null;
while (retry) {
Transaction txn = store.getEnvironment().beginTransaction(null, txnConfig);
carScore = carScoreById.get(txn, carId, LockMode.READ_COMMITTED);
try {
try {
txn.commit();
txn = null;
} catch (DatabaseException e) {
System.err.println("Error on txn commit: " + e.toString());
}
retry = false;
} catch (LockConflictException de) {
System.out.println("BDB: Deadlock");
// retry if necessary
if (retry_count < MAX_RETRY) {
retry = true;
retry_count++;
} else {
System.err.println("BDB: Out of retries[get]. Giving up.");
retry = false;
}
} catch (DatabaseException e) {
retry = false; // abort and don't retry
System.err.println("BDB exception: " + e.toString());
e.printStackTrace();
} finally {
if (txn != null) {
try {
txn.abort();
} catch (Exception e) {
System.err.println("Error aborting transaction: " + e.toString());
e.printStackTrace();
}
}
}
}
return carScore;
}
/**
* entity cursor
*
* **/
public EntityCursor<CarScore> entityCursorGet(){
Transaction txn = store.getEnvironment().beginTransaction(null, null);
CursorConfig cconfig = new CursorConfig();
cconfig.setReadCommitted(true);
return carScoreById.entities(txn, cconfig);
}
/**
* key cursor
*
* **/
public EntityCursor<String> keyCursorGet(){
Transaction txn = store.getEnvironment().beginTransaction(null, null);
CursorConfig cconfig = new CursorConfig();
cconfig.setReadCommitted(true);
//放入map
EntityCursor<String> e = carScoreById.keys(txn, cconfig);
cursorTransactionMap.put(e, txn);
return e;
}
/**
* 根据brand来找
* @param tag
* @return
*/
public EntityCursor<CarScore> findByBrand(String brand){
EntityIndex<String, CarScore> e = carScoreByBrand.subIndex(brand);
Transaction txn = store.getEnvironment().beginTransaction(null, null);
CursorConfig cconfig = new CursorConfig();
cconfig.setReadCommitted(true);
//放入map
EntityCursor<CarScore> ec = e.entities(txn, cconfig);
cursorTransactionMap.put(ec, txn);
return ec;
}
/**
* 根据brand来找
* @param tag
* @return
*/
public EntityCursor<CarScore> findBySeries(String series) {
EntityIndex<String, CarScore> e = carScoreBySeries.subIndex(series);
Transaction txn = store.getEnvironment().beginTransaction(null, null);
CursorConfig cconfig = new CursorConfig();
cconfig.setReadCommitted(true);
//放入map
EntityCursor<CarScore> ec = e.entities(txn, cconfig);
cursorTransactionMap.put(ec, txn);
return ec;
}
/**
* 取到所有tag
* @return
*/
public EntityCursor<String> getAllTags() {
Transaction txn = store.getEnvironment().beginTransaction(null, null);
CursorConfig cconfig = new CursorConfig();
cconfig.setReadCommitted(true);
//放入map
EntityCursor<String> ec = carScoreByBrand.keys(txn, cconfig);
cursorTransactionMap.put(ec, txn);
return ec;
}
public long getAllCount(){
return carScoreById.count();
}
//----------------------EntityJoin------------------------
/**
* 统一二级索引查询:join搜索
* @param type
* @param year
* @return
*/
public ForwardCursor<CarScore> getJoinEntity(String brand, String series, String type, String year) {
Transaction txn = store.getEnvironment().beginTransaction(null, null);
CursorConfig cconfig = new CursorConfig();
cconfig.setReadCommitted(true);
EntityJoin<String, CarScore> join = new EntityJoin<String, CarScore>(carScoreById);
if(brand != null){
join.addCondition(carScoreByBrand, brand);
}
if(series != null){
join.addCondition(carScoreBySeries, series);
}
if(type != null) {
join.addCondition(carScoreByType, type);
}
return join.entities(txn, cconfig);
}
}
上述两个类视具体业务而定。
最后,需要在beans/xml中做bean的配置:
<!-- rec数据 BDB相关包装器 -->
<bean id="BDBEnvironment" class="BDB.BDBEnvironment" scope="singleton">
<constructor-arg name="envHome" value="${global.base}${BDBDir.carScore}"/>
<constructor-arg name="deleteFolderContents" value="true"/>
</bean>
<bean id="carScoreStore" class="BDB.CarScoreStore" scope="singleton" destroy-method="close">
<constructor-arg name="env" ref="BDBEnvironment"/>
</bean>
在spring的Controller中,我们实际注入的就是carScoreStore。
@Autowired private CarScoreStore carStore;
1,不涉及游标(Cursor)的普通的查询、删除等,直接调用。
2,涉及游标的方法,需要在调用后显示调用:
<pre name="code" class="java">EntityCursor<CarScore> e = carStore.findByBrand(toBrand);
//......
carStore.cursorCommitAndClean(e);
暂时写到这。:)