在使用hibernate进行批处理的时候,需要考虑这两个问题:OutOfMemoryError、处理速度
解决方案:
<property name="hibernate.jdbc.batch_size">20</property> <--使用JDBC batching-->
<property name="hibernate.cache.use_second_level_cache">false</property> <--关闭二级缓存-->
使用session的flush()和clear()方法阶段性清除一级缓存
1.方法一:使用服务器游标
1.1.批量插入
<span style="font-size:12px;">Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();
for ( int i=0; i<100000; i++ ) {
Customer customer = new Customer(.....);
session.save(customer);
if ( i % 20 == 0 ) { //此处的值应和配置文件中的一致
//强制内存到数据库同步,释放内存
session.flush();
session.clear();
}
}
tx.commit();
session.close();</span>
1.2.批量更新
Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();
ScrollableResults customers = session.getNamedQuery("GetCustomers") //使用了外置命令查询
.setCacheMode(CacheMode.IGNORE)
.scroll(ScrollMode.FORWARD_ONLY);
int count=0;
while ( customers.next() ) {
Customer customer = (Customer) customers.get(0);
customer.updateStuff(...);
if ( ++count % 20 == 0 ) {
//<span style="font-size:12px;">强制内存到数据库同步,释放内存</span>
session.flush();
session.clear();
}
}
tx.commit();
session.close();
2.方法二:使用StatelessSession接口
StatelessSession 不实现第一级 cache,也不和第二级缓存,或者查询缓存交互。它不实现事务化写,也不实现脏数据检查。没有第一级缓存。无状态 session 是低层的抽象,和低层 JDBC 相当接近。
StatelessSession session = sessionFactory.openStatelessSession();
Transaction tx = session.beginTransaction();
ScrollableResults customers = session.getNamedQuery("GetCustomers")
.scroll(ScrollMode.FORWARD_ONLY);
while ( customers.next() ) {
Customer customer = (Customer) customers.get(0);
customer.updateStuff(...);
session.update(customer);
}
tx.commit();
session.close();
3 .方法三:使用hql
3.1批量插入
Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();
String hqlInsert = "insert into DelinquentAccount (id, name) select c.id, c.name from Customer c where ...";
int createdEntities = s.createQuery( hqlInsert )
.executeUpdate();
tx.commit();
session.close();
注意:使用hql插入操作 只支持 INSERT INTO ... SELECT ... 形式,不支持 INSERT INTO ... VALUES ... 形式。
3.2批量更新
Session session = sessionFactory.openSession();
Transaction tx = session.beginTransaction();
String hqlVersionedUpdate = "update versioned Customer set name = :newName where name = :oldName";
int updatedEntities = s.createQuery( hqlUpdate )
.setString( "newName", newName )
.setString( "oldName", oldName )
.executeUpdate();
tx.commit();
session.close();