为解决DataBufferLimitException: Exceeded limit on max bytes to buffer : 262144 引发OOM事故

之前做项目有一个需求就是上传文件,用到的是spring cloud框架,统一走的网关,但是这网关默认允许文件上传大小为256KB,超过的话就会爆DataBufferLimitException: Exceeded limit on max bytes to buffer : 262144的错误。

spring是提供了这个参数的配置如下截图

 本来以为改一下这个参数就可以了,但是还是会报错,定位到了路由工厂配置类(RequestBodyRoutePredicateFactory继承AbstractRoutePredicateFactory)会重新创建ServerRequest对象,主要参考DataBufferLimitException: Exceeded limit on max bytes to buffer : 262144_lizz666的博客-CSDN博客

那么spring帮我们初始化的内存值就不生效了

所以就按解决方法引入的这个Bean,去调用getReaders()方法,让重新加载nacos的值

试了一下确实没有问题了,然后就上线了,

然后悲剧就开始了,在运行的期间出现OOM了,大概2个小时出现一次。

后面定位到这行代码有问题

 那么就翻一下源码看一下,发现在new DefaultPartHttpMessageReader()对象的时候会创建一个调度任务,当时就感觉不对劲了

 就再点进去看看发现会new 一个BoundedElasticScheduler对象

 

 于是去查了一下资料

这就尴尬了,每个请求都会创建单线程池,这并发一大不就gg了么。

最后没有想到什么好办法,只能重写源码了

/*
 * Copyright 2002-2019 the original author or authors.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      https://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.springframework.core.codec;

import org.reactivestreams.Publisher;
import org.springframework.core.ResolvableType;
import org.springframework.core.io.buffer.DataBuffer;
import org.springframework.core.io.buffer.DataBufferUtils;
import org.springframework.lang.Nullable;
import org.springframework.util.MimeType;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;

import java.util.Map;

/**
 * Abstract base class for {@code Decoder} implementations that can decode
 * a {@code DataBuffer} directly to the target element type.
 *
 * <p>Sub-classes must implement {@link #decodeDataBuffer} to provide a way to
 * transform a {@code DataBuffer} to the target data type. The default
 * {@link #decode} implementation transforms each individual data buffer while
 * {@link #decodeToMono} applies "reduce" and transforms the aggregated buffer.
 *
 * <p>Sub-classes can override {@link #decode} in order to split the input stream
 * along different boundaries (e.g. on new line characters for {@code String})
 * or always reduce to a single data buffer (e.g. {@code Resource}).
 *
 * @author Rossen Stoyanchev
 * @since 5.0
 * @param <T> the element type
 */
@SuppressWarnings("deprecation")
public abstract class AbstractDataBufferDecoder<T> extends AbstractDecoder<T> {

	private int maxInMemorySize = 100 * 1024 * 1024;


	protected AbstractDataBufferDecoder(MimeType... supportedMimeTypes) {
		super(supportedMimeTypes);
	}


	/**
	 * Configure a limit on the number of bytes that can be buffered whenever
	 * the input stream needs to be aggregated. This can be a result of
	 * decoding to a single {@code DataBuffer},
	 * {@link java.nio.ByteBuffer ByteBuffer}, {@code byte[]},
	 * {@link org.springframework.core.io.Resource Resource}, {@code String}, etc.
	 * It can also occur when splitting the input stream, e.g. delimited text,
	 * in which case the limit applies to data buffered between delimiters.
	 * <p>By default this is set to 256K.
	 * @param byteCount the max number of bytes to buffer, or -1 for unlimited
	 * @since 5.1.11
	 */
	public void setMaxInMemorySize(int byteCount) {
		this.maxInMemorySize = byteCount;
	}

	/**
	 * Return the {@link #setMaxInMemorySize configured} byte count limit.
	 * @since 5.1.11
	 */
	public int getMaxInMemorySize() {
		return this.maxInMemorySize;
	}


	@Override
	public Flux<T> decode(Publisher<DataBuffer> input, ResolvableType elementType,
			@Nullable MimeType mimeType, @Nullable Map<String, Object> hints) {

		return Flux.from(input).map(buffer -> decodeDataBuffer(buffer, elementType, mimeType, hints));
	}

	@Override
	public Mono<T> decodeToMono(Publisher<DataBuffer> input, ResolvableType elementType,
			@Nullable MimeType mimeType, @Nullable Map<String, Object> hints) {

		return DataBufferUtils.join(input, this.maxInMemorySize)
				.map(buffer -> decodeDataBuffer(buffer, elementType, mimeType, hints));
	}

	/**
	 * How to decode a {@code DataBuffer} to the target element type.
	 * @deprecated as of 5.2, please implement
	 * {@link #decode(DataBuffer, ResolvableType, MimeType, Map)} instead
	 */
	@Deprecated
	@Nullable
	protected T decodeDataBuffer(DataBuffer buffer, ResolvableType elementType,
			@Nullable MimeType mimeType, @Nullable Map<String, Object> hints) {

		return decode(buffer, elementType, mimeType, hints);
	}

}

只能先这么处理了,后面再看看怎么把配置值放到nacos上。

总结:明明只改了一行代码却引发了OOM的事故,平常解决问题,不能看到代码就直接抄,要分析多思考,希望能够帮助到其他人避免踩坑。

  • 2
    点赞
  • 4
    收藏
    觉得还不错? 一键收藏
  • 1
    评论
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值