Java Web / Spring集成Berkeley DB java edition(je)内存数据库

一、概述


本文介绍在java web工程中使用Berkeley DB JE作为内存数据库来存取数据。当业务逻辑不太复杂时,通常可以代替在关系型数据库中设计复杂的sql和表结构。

当数据量不太夸张(BDB)时,这种方案的优点,除了BDB自身的种种优点外还有:

1,代码简单,易于维护。且数据持久化存储在指定目录中。

2,存取高效,经实际项目(高并发)测试,附带附加的计算返回均为毫秒级别。

3,易于扩展,支持任意java对象的增删查改。

4,可实现的接口丰富,支持交叉、关联查询等。

5,事务支持。

6,支持一级、二级索引


二、最佳实践



最佳实践:存取Car数据。

如上图,主要存取过程主要由4各类完成,其中:

1,BDBEnvironment.java: 封装BDB自身的Environment类,EnvironmentConfig类。指定数据存放的目录。

/**
 * BDB环境包装器
 * @author craig
 *
 */
public class BDBEnvironment {

	private Environment env;
	private StoreConfig storeConfig;
	private String envHome;
	
	public BDBEnvironment(String envHome){
		System.out.println("envHome:" + envHome);
		this.envHome = envHome;
		init();
	}
	
	public BDBEnvironment(String envHome, boolean deleteFolderContents){
		System.out.println("envHome:" + envHome);
		this.envHome = envHome;
		if(deleteFolderContents) {
			JFileUtils.deleteFolderContents(new File(envHome));
		}
		init();
	}
	
	public void init() {
		
		EnvironmentConfig myEnvConfig = new EnvironmentConfig();
        myEnvConfig.setReadOnly(false);
        myEnvConfig.setTransactional(true);
        myEnvConfig.setAllowCreate(true);
        
        storeConfig = new StoreConfig();
        storeConfig.setReadOnly(false);
        storeConfig.setAllowCreate(true);
        storeConfig.setTransactional(true);
        env = new Environment(new File(envHome), myEnvConfig);
		
	}
	
	public Environment getEnv() {
		return env;
	}
	public void setEnv(Environment env) {
		this.env = env;
	}
	public StoreConfig getStoreConfig() {
		return storeConfig;
	}
	public void setStoreConfig(StoreConfig storeConfig) {
		this.storeConfig = storeConfig;
	}
}

2,AbstracStore.java: 数据库io抽象类,增删查改的各个操作的公用方法,提交事务、关闭数据库等。

public abstract class AbstractStore {

	protected final int MAX_RETRY = 5; //  Used for while loop and deadlock retries
	protected EntityStore store;
	//protected Transaction txn;
	
	//放需要commit和clean的Transaction
	ConcurrentHashMap<EntityCursor<? extends Object>, Transaction> cursorTransactionMap 
		= new ConcurrentHashMap<EntityCursor<? extends Object>, Transaction>();
	
	//-----Construction Method------
	public AbstractStore(BDBEnvironment env){
		store = new EntityStore(env.getEnv(), "EntityStore", env.getStoreConfig());
	}
	
	public void clean(){
		store.getEnvironment().cleanLog();
	}
	
	/**
	 * entity cursor used after getEntityCursor() and operations.
	 * 
	 * **/
	public void cursorCommitAndClean(EntityCursor<? extends Object> cursor){
		
		Transaction txn = cursorTransactionMap.get(cursor); //取到txn
		cursorTransactionMap.remove(cursor);
		
		boolean retry = true;
		int retry_count = 0;
    	cursor.close();
    	cursor = null;
        while (retry) {
			try {
				try {

					//txn.commit();
					txn.commit();
					txn = null;
				} catch (DatabaseException e) {
					System.err.println("Error on txn commit: " + e.toString());
				}
				retry = false;
			} catch (LockConflictException de) {
				System.out.println("BDB: Deadlock");
				// retry if necessary
				if (retry_count < MAX_RETRY) {
					retry = true;
					retry_count++;
				} else {
					System.err.println("BDB: Out of retries[commit entityCursor]. Giving up.");
					retry = false;
				}
			} catch (DatabaseException e) {
				retry = false;   // abort and don't retry
				System.err.println("BDB exception: " + e.toString());
				e.printStackTrace();
			} finally {
		        if (cursor != null) {
		            cursor.close();
		        }
				if (txn != null) {
					try {
						txn.abort();
					} catch (Exception e) {
						System.err.println("Error aborting transaction: " + e.toString());
						e.printStackTrace();
					}
				}
			}
		}
	}
	
    // Close the store and environment
    public void close() {
    	System.out.println("close dataBase.");
        if (store != null ) {
            try {
                store.close();
            } catch (DatabaseException e) {
                System.err.println("closeEnv: store: " + e.toString());
                e.printStackTrace();
            }
        }
        if (store.getEnvironment() != null ) {
            try {
            	store.getEnvironment().cleanLog();
            
            	store.getEnvironment().close();
            } catch (DatabaseException e) {
                System.err.println("closeEnv: " + e.toString());
                e.printStackTrace();
            }
        }
    }
	
    public EntityStore getStore() {
		return store;
	}
    public void setStore(EntityStore store) {
		this.store = store;
	}
    
}

上述两个类是基础核心类。


3,CarScore.java: 需要持久化存储的java对象,即POJO类、元数据。

@Entity
public class CarScore {

	@PrimaryKey
	private String carId;
	private String name;  //车型名
	private String model;  //车型code
	
	private Long price;  //车价(按车价差公式计算结果降权即可,不作为硬性索引条件)
	
	@SecondaryKey(relate = Relationship.MANY_TO_ONE) private String brand;   //品牌
	@SecondaryKey(relate = Relationship.MANY_TO_ONE) private String series;  //车系
	private Integer year;  //年限(上牌年份)
	@SecondaryKey(relate = Relationship.MANY_TO_ONE) private String carType;  //车形
	@SecondaryKey(relate = Relationship.MANY_TO_ONE) private String color;	//颜色
	@SecondaryKey(relate = Relationship.MANY_TO_ONE) private String capacity;  //排量
	@SecondaryKey(relate = Relationship.MANY_TO_ONE) private Long miles;   //里程
	@SecondaryKey(relate = Relationship.MANY_TO_ONE) private String geerbox;  //变速箱
	@SecondaryKey(relate = Relationship.MANY_TO_ONE) private String country;  //国别
	
	//
	private String status;  //状态
	private String level;  //车辆等级
	private String upShelfDate;  //第一次上架时间
	private String licenseDate;  //上牌时间
	private Double score;
	
	public CarScore(){}
	
	//getter & setter method....
}


4,CarScoreStore.java: 数据库操作(增删查改)的操作类。需要继承AbstractStore.java

public class CarScoreStore extends AbstractStore {

	public PrimaryIndex<String, CarScore> carScoreById;
	public SecondaryIndex<String, String, CarScore> carScoreByBrand;
	public SecondaryIndex<String, String, CarScore> carScoreBySeries;

	public SecondaryIndex<String, String, CarScore> carScoreByType;
	//public SecondaryIndex<String, String, CarScore> carScoreByYear;
	
	
	//-----Construction Method------
	public CarScoreStore(BDBEnvironment env) {
		
		super(env);
		
		carScoreById = store.getPrimaryIndex(String.class, CarScore.class); ///primary key
    	carScoreByBrand = store.getSecondaryIndex(carScoreById, String.class, "brand");
    	carScoreBySeries = store.getSecondaryIndex(carScoreById, String.class, "series");

    	carScoreByType = store.getSecondaryIndex(carScoreById, String.class, "carType");
    	//carScoreByYear = store.getSecondaryIndex(carScoreById, String.class, "year");
	}

	//-------Operation Method--------

	
	/**
     *  add a record
     * 
     * **/
    public void put(CarScore entity) {

		boolean retry = true;
		int retry_count = 0;
		
        while (retry) {
        	Transaction txn = store.getEnvironment().beginTransaction(null, null);
			carScoreById.put(txn, entity);
			try {
				try {
					txn.commit();
					txn = null;
				} catch (DatabaseException e) {
					System.err.println("Error on txn commit: " + e.toString());
				}
				retry = false;
			} catch (LockConflictException de) {
				System.out.println("BDB: Deadlock");
				
				if (retry_count < MAX_RETRY) {  // retry if necessary
					retry = true;
					retry_count++;
				} else {
					System.err.println("BDB: Out of retries[put]. Giving up.");
					retry = false;
				}
			} catch (DatabaseException e) {
				retry = false;  // abort and don't retry
				System.err.println("BDB exception: " + e.toString());
				e.printStackTrace();
			} finally {
				if (txn != null) {
					try {
						txn.abort();
					} catch (Exception e) {
						System.err.println("Error aborting transaction: " + e.toString());
						e.printStackTrace();
					}
				}
			}
		}
    }
	
	/**
     *  add a record
     * 
     * **/
    public void delete(String carId) {

		boolean retry = true;
		int retry_count = 0;
		
        while (retry) {
        	Transaction txn = store.getEnvironment().beginTransaction(null, null);
			carScoreById.delete(txn, carId);
			try {
				try {
					txn.commit();
					txn = null;
				} catch (DatabaseException e) {
					System.err.println("Error on txn commit: " + e.toString());
				}
				retry = false;
			} catch (LockConflictException de) {
				System.out.println("BDB: Deadlock");
				
				if (retry_count < MAX_RETRY) {  // retry if necessary
					retry = true;
					retry_count++;
				} else {
					System.err.println("BDB: Out of retries[put]. Giving up.");
					retry = false;
				}
			} catch (DatabaseException e) {
				retry = false;  // abort and don't retry
				System.err.println("BDB exception: " + e.toString());
				e.printStackTrace();
			} finally {
				if (txn != null) {
					try {
						txn.abort();
					} catch (Exception e) {
						System.err.println("Error aborting transaction: " + e.toString());
						e.printStackTrace();
					}
				}
			}
		}
    }

    
	/**
	 * search BDBItem by id
	 * 
	 * **/
	public CarScore get(String carId){
		
		boolean retry = true;
		int retry_count = 0;
		TransactionConfig txnConfig = new TransactionConfig();
	    txnConfig.setReadCommitted(true);
	    CarScore carScore = null;
        while (retry) {
        	
        	Transaction txn = store.getEnvironment().beginTransaction(null, txnConfig);
    		carScore = carScoreById.get(txn, carId, LockMode.READ_COMMITTED);
			try {
				try {
					txn.commit();
					txn = null;
				} catch (DatabaseException e) {
					System.err.println("Error on txn commit: " + e.toString());
				}
				retry = false;
			} catch (LockConflictException de) {
				System.out.println("BDB: Deadlock");
				// retry if necessary
				if (retry_count < MAX_RETRY) {
					retry = true;
					retry_count++;
				} else {
					System.err.println("BDB: Out of retries[get]. Giving up.");
					retry = false;
				}
			} catch (DatabaseException e) {
				
				retry = false;   // abort and don't retry
				System.err.println("BDB exception: " + e.toString());
				e.printStackTrace();
			} finally {
				if (txn != null) {
					try {
						txn.abort();
					} catch (Exception e) {
						System.err.println("Error aborting transaction: " + e.toString());
						e.printStackTrace();
					}
				}
			}
		}
		return carScore;
	}
    
	/**
	 * entity cursor
	 * 
	 * **/
	public EntityCursor<CarScore> entityCursorGet(){

		Transaction txn = store.getEnvironment().beginTransaction(null, null);
	    CursorConfig cconfig = new CursorConfig();
	    cconfig.setReadCommitted(true);
	    return carScoreById.entities(txn, cconfig);
	}
	
	/**
	 * key cursor
	 * 
	 * **/
	public EntityCursor<String> keyCursorGet(){

		Transaction txn = store.getEnvironment().beginTransaction(null, null);
	    CursorConfig cconfig = new CursorConfig();
	    cconfig.setReadCommitted(true);
	    
	    //放入map
	    EntityCursor<String> e = carScoreById.keys(txn, cconfig);
	    cursorTransactionMap.put(e, txn);
	    
	    return e;
	}
	
	
	
	/**
	 * 根据brand来找
	 * @param tag
	 * @return
	 */
	public EntityCursor<CarScore> findByBrand(String brand){
		EntityIndex<String, CarScore> e = carScoreByBrand.subIndex(brand);
		Transaction txn = store.getEnvironment().beginTransaction(null, null);
		CursorConfig cconfig = new CursorConfig();
		cconfig.setReadCommitted(true);
		
		//放入map
	    EntityCursor<CarScore> ec = e.entities(txn, cconfig);
	    cursorTransactionMap.put(ec, txn);
		
		return ec;
	}
	
	/**
	 * 根据brand来找
	 * @param tag
	 * @return
	 */
	public EntityCursor<CarScore> findBySeries(String series) {
		EntityIndex<String, CarScore> e = carScoreBySeries.subIndex(series);
		Transaction txn = store.getEnvironment().beginTransaction(null, null);
		CursorConfig cconfig = new CursorConfig();
		cconfig.setReadCommitted(true);
		
		//放入map
	    EntityCursor<CarScore> ec = e.entities(txn, cconfig);
	    cursorTransactionMap.put(ec, txn);
		
		return ec;
	}
	

	
	/**
	 * 取到所有tag
	 * @return
	 */
	public EntityCursor<String> getAllTags() {
		Transaction txn = store.getEnvironment().beginTransaction(null, null);
		CursorConfig cconfig = new CursorConfig();
		cconfig.setReadCommitted(true);
		
		
		//放入map
	    EntityCursor<String> ec = carScoreByBrand.keys(txn, cconfig);
	    cursorTransactionMap.put(ec, txn);
	    
		return ec;
	}

    public long getAllCount(){
    	return carScoreById.count();
    }
    

	//----------------------EntityJoin------------------------
	/**
	 * 统一二级索引查询:join搜索
	 * @param type
	 * @param year
	 * @return
	 */
	public ForwardCursor<CarScore> getJoinEntity(String brand, String series, String type, String year) {
		
		Transaction txn = store.getEnvironment().beginTransaction(null, null);
		CursorConfig cconfig = new CursorConfig();
		cconfig.setReadCommitted(true);
		
		EntityJoin<String, CarScore> join = new EntityJoin<String, CarScore>(carScoreById);
		if(brand != null){
			join.addCondition(carScoreByBrand, brand);
		}
		if(series != null){
			join.addCondition(carScoreBySeries, series);
		}
		if(type != null) {
			join.addCondition(carScoreByType, type);
		}
		
		return join.entities(txn, cconfig);
	}
	
	
}

上述两个类视具体业务而定。

最后,需要在beans/xml中做bean的配置:

	<!-- rec数据 BDB相关包装器 -->
	<bean id="BDBEnvironment" class="BDB.BDBEnvironment" scope="singleton">
		<constructor-arg name="envHome" value="${global.base}${BDBDir.carScore}"/>
		<constructor-arg name="deleteFolderContents" value="true"/>
	</bean>
	<bean id="carScoreStore" class="BDB.CarScoreStore" scope="singleton" destroy-method="close">
		<constructor-arg name="env" ref="BDBEnvironment"/>
	</bean>

在spring的Controller中,我们实际注入的就是carScoreStore。

@Autowired private CarScoreStore carStore;

1,不涉及游标(Cursor)的普通的查询、删除等,直接调用。

2,涉及游标的方法,需要在调用后显示调用:

<pre name="code" class="java">EntityCursor<CarScore> e = carStore.findByBrand(toBrand);
//......
carStore.cursorCommitAndClean(e);

 

暂时写到这。:)



  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值