在多任务执行的时候,我们可能用到线程池,如果在任务量非常大的情况下任务可能走了拒绝策略,有人可能会说那就吧队列设置成最大的,或者无界队列的情况,这种方式是可以暂时解决任务丢失的情况,但是如果队列如果太多一直阻塞可能会造成OOM。所以再网上看了一些文档,在拒绝策略那总结了两种方式,并在生产验证了这两种情况:
(1)直接在拒绝那重新给队列添加任务,重复添加,直到队列中有空位腾出
(2)在拒绝策略那块,可以再次补偿执行添加任务,直到成功为止
代码如下:
方式一:
package com.eebbk.content.ai.makequestion.config;
import lombok.extern.slf4j.Slf4j;
import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.RejectedExecutionHandler;
import java.util.concurrent.ThreadPoolExecutor;
/**
* @Auther: lipf
* @Date: 2020/1/13 16:50
* @Description:
*/
@Slf4j
public class CustomAbortPolicy implements RejectedExecutionHandler {
public void AbortPolicy() { }
/**
* Always throws RejectedExecutionException.
* @param r the runnable task requested to be executed
* @throws RejectedExecutionException always
*/
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
if (!executor.isShutdown()) {
try {
log.error("full-->>线程池已满,执行拒绝策略");
executor.getQueue().put(r);
} catch (InterruptedException e) {
log.error("interruptedException="+e.toString());
}
}
}
}
方式二:
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
if (!executor.isShutdown()) {
try {
log.error("full-->>线程池已满,执行拒绝策略");
while (e.getQueue().remainingCapacity() == 0);
e.execute(r);
} catch (InterruptedException e) {
log.error("interruptedException="+e.toString());
}
}
}
任务测试用例:
public void testThread(){
//模拟10000个任务执行,用计时器保证任务最终执行后合并归一
CountDownLatch countDownLatch=new CountDownLatch(10000);
//统计任务是否丢失
AtomicInteger atomicInteger=new AtomicInteger();
//模拟10000个任务执行
for (int i = 0; i < 10000; i++) {
taskExecutor.submit(new Runnable() {
@Override
public void run() {
try {
Thread.sleep(100);
atomicInteger.addAndGet(1);
} catch (Exception e) {
} finally {
countDownLatch.countDown();
}
}
});
}
try {
countDownLatch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println(atomicInteger);
}