Mybatis批量插入大数据导致堆内存溢出问题解决方案

原文源自：https://my.oschina.net/u/2428301/blog/1796597

Exception in thread "Thread-8" java.lang.OutOfMemoryError: Java heap space
	at java.util.Arrays.copyOf(Arrays.java:3332)
	at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:137)
	at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:121)
	at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:421)
	at java.lang.StringBuilder.append(StringBuilder.java:136)
	at org.apache.ibatis.scripting.xmltags.DynamicContext.appendSql(DynamicContext.java:65)
	at org.apache.ibatis.scripting.xmltags.ForEachSqlNode$PrefixedContext.appendSql(ForEachSqlNode.java:208)
	at org.apache.ibatis.scripting.xmltags.ForEachSqlNode$FilteredDynamicContext.appendSql(ForEachSqlNode.java:165)
	at org.apache.ibatis.scripting.xmltags.StaticTextSqlNode.apply(StaticTextSqlNode.java:30)
	at org.apache.ibatis.scripting.xmltags.MixedSqlNode.apply(MixedSqlNode.java:33)
	at org.apache.ibatis.scripting.xmltags.IfSqlNode.apply(IfSqlNode.java:35)
	at org.apache.ibatis.scripting.xmltags.ChooseSqlNode.apply(ChooseSqlNode.java:35)
	at org.apache.ibatis.scripting.xmltags.MixedSqlNode.apply(MixedSqlNode.java:33)
	at org.apache.ibatis.scripting.xmltags.ForEachSqlNode.apply(ForEachSqlNode.java:82)
	at org.apache.ibatis.scripting.xmltags.MixedSqlNode.apply(MixedSqlNode.java:33)
	at org.apache.ibatis.scripting.xmltags.DynamicSqlSource.getBoundSql(DynamicSqlSource.java:41)
	at org.apache.ibatis.mapping.MappedStatement.getBoundSql(MappedStatement.java:292)
	at org.apache.ibatis.executor.statement.BaseStatementHandler.<init>(BaseStatementHandler.java:64)
	at org.apache.ibatis.executor.statement.PreparedStatementHandler.<init>(PreparedStatementHandler.java:40)
	at org.apache.ibatis.executor.statement.RoutingStatementHandler.<init>(RoutingStatementHandler.java:46)
	at org.apache.ibatis.session.Configuration.newStatementHandler(Configuration.java:523)
	at org.apache.ibatis.executor.ReuseExecutor.doUpdate(ReuseExecutor.java:50)
	at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117)
	at org.apache.ibatis.executor.CachingExecutor.update(CachingExecutor.java:76)
	at sun.reflect.GeneratedMethodAccessor89.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:63)
	at com.sun.proxy.$Proxy25.update(Unknown Source)
	at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:198)
	at org.apache.ibatis.session.defaults.DefaultSqlSession.insert(DefaultSqlSession.java:185)
	at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source)

JVM环境：-Xms64m -Xmx128m

        由于①mybatis的DAO接口我是传List，而mybatis的foreach底层代码是arrays的，②所以需要把List转arrays，然后遍历Arrays，③用StringBuilder拼接SQL的。

        问题出现在②③步！因生成的arrays对象和StringBuilder对象是在堆内存中的，而堆内存的对象GC回收机制是：只有JVM的使用内存达到最大内存时的一定阀值，才会执行GC回收！

        如果当本身JVM已经使用了占有了JVM的最大内存的90%,那么②③步的生成的对象占有内存+本身已占有的内存如果超出JVM的最大内存，而JVM又来不及执行GC时就会导致JVM内存溢出，当前线程挂掉！

       我的解决方法是：实行多段相继执行，每次只批量执行3000个对象，从而保证Arrays和StringBuilder过大，当然3000个是相对于我的JVM环境而言，每个人也可以根据自身的JVM环境进行调优。上文观点是个人的理解，如果有什么不对的地方请指出，谢谢。 

public int batchSave(String stockCode, String stockType, String tableNum, List<StockMarket> stockMarkets) throws Exception {
		int size = stockMarkets.size();
		int index = 0;
		while(true) {
			if(index+3000>=size) {
				this.batchSave(stockType, tableNum, stockMarkets.subList(index, size-1));
				break;
			}else {
				this.batchSave(stockType, tableNum, stockMarkets.subList(index, index+3000));
				index = index+3000;
			}
			
		}
		return size;
	}

参考：

mybaits批量插入引起的血案：https://blog.csdn.net/syy_c_j/article/details/52151402