什么是优雅停机?
在web服务(Http协议)上线的时候,会经过kill命令杀死进程,这个时候在已经accept的请求还在线程池里面,咱们要保证这部分请求正常处理而且返回数据以后再停机.java
dubbo服务(Tcp协议)也是一样的道理.web
优雅停机包括:线程池的优雅关闭,数据库链接池的关闭,数据源的关闭,kafka链接的关闭....redis
没有优雅停机的现象
1.客户端拿不到数据spring
2.数据源报已经关闭数据库
Failed to obtain JDBC Connection; nested exception is com.alibaba.druid.pool.DataSourceClosedException: dataSource already closed
JVM的钩子
Runtime.addShutDownHookapache
咱们的服务是基于SpringBoot+Dubbo+业务线程池来实现的.tomcat
dubbo框架提供了destroyAll来实现自身的优雅关闭,Spring容器对Bean的优雅关闭是经过@PreDestory来实现的,咱们本身建立的ThreadPool也能够经过Runtime.addShutDownHook来实现优雅关闭。cookie
可是这里有一个关键的问题是这三者是并行执行的,dubbo在进入优雅停机状态中的时候已经中止接收新的业务请求,然而已经接收的请求须要继续处理,可是有可能此时Spring的优雅关闭已经执行完成,致使在处理请求的时候出现异常(好比DataSource已经close了)。并发
那如何保证Spring容器等待dubbo优雅关闭执行完成之后再执行bean的@PreDestory方法(销毁bean)呢?下面有请Spring的ApplicationListener!框架
Spring应用生命周期
Spring能够经过继承下面这个接口来实现对应用声明周期的回调
public interface ApplicationListener extends EventListener {
void onApplicationEvent(E var1);
}
ContextClosedEvent是在全部bean执行PreDestory以前发出的事件广播.咱们在这个事件回调中执行Dubbo的优雅关闭,就不会出现数据源已经关闭的异常.
SpringBoot + Dubbo 服务的优雅关闭
思路很简单,咱们只要监听Spring容器的声明周期,在容器启动的时候把Dubbo注册到JVM的shutdown hook删除,而后在ContextClosedEvent中执行Dubbo的优雅关闭.
@Configuration
@Slf4j
public class DefaultSpringDubboConfigurations {
@Bean
DubboShutdownListener dubboShutdownListener() {
return new DubboShutdownListener();
}
public static class DubboShutdownListener implements ApplicationListener, PriorityOrdered {
@Override
public void onApplicationEvent(ApplicationEvent event) {
if (event instanceof ApplicationStartedEvent) {
Runtime.getRuntime().removeShutdownHook(DubboShutdownHook.getDubboShutdownHook());
log.info("dubbo default shutdown hook removed,will be managed by spring");
} else if (event instanceof ContextClosedEvent) {
log.info("start destroy dubbo on spring close event");
DubboShutdownHook.getDubboShutdownHook().destroyAll();
log.info("dubbo destroy finished");
}
}
@Override
public int getOrder() {
return 0;
}
}
}
业务线程池如何优雅关闭?
错误的作法:
Runtime.getRuntime().addShutdownHook(new Thread(){
// 线程池虽然关闭,可是队列中的任务任然继续执行,因此用 shutdown()方式关闭线程池时须要考虑是不是你想要的效果
//若是但愿当即中止,抛弃队列中的任务,可使用shutdownNow()
threadPoolExecutor.shutdown();
});
上面的代码忽视了多个钩子函数是并发执行的问题,线程池的业务逻辑可能须要数据源连接、redis连接等,可是这个时候有可能数据源已经关闭了。
正确的作法:
@PostConstruct
public void afterPropertiesSet() throws Exception {
DEAL_EVENT_THREAD_POOL = ThreadPoolUtils.newExecutor(CORE_POOL_SIZE, MAXIMUM_POOL_SIZE, KEEP_ALIVE_TIME, TimeUnit.SECONDS,
QUEUE_MAX_SIZE, "DealEventLogTask-service");
}
@PreDestroy
public void destroy() throws Exception {
ThreadPoolUtils.stop(DEAL_EVENT_THREAD_POOL);
}
SpringBoot内嵌Tomcat的优雅停机
@Configuration
public class DefaultTomcatFactoryConfigurations {
@Bean
TomcatServletWebServerFactory tomcatServletWebServerFactory(@Value("${server.port}") int port,
@Qualifier("gracefulShutdownListener") GracefulShutdownListener gracefulShutdownListener) {
TomcatServletWebServerFactory factory = new TomcatServletWebServerFactory();
factory.setPort(port);
factory.setProtocol("org.apache.coyote.http11.Http11Nio2Protocol");
List contextLifecycleListeners = Lists.newArrayList(new VersionLoggerListener(),
new JreMemoryLeakPreventionListener(), new ThreadLocalLeakPreventionListener());
factory.setContextLifecycleListeners(contextLifecycleListeners);
factory.addConnectorCustomizers(gracefulShutdownListener);
// RFC 6265 对cookie domain有较多限制,其中一条是不能以 "." 开头,与com.kuaikan.common.utils.web.CookieUtils有冲突
factory.addContextCustomizers(context -> context.setCookieProcessor(new LegacyCookieProcessor()));
return factory;
}
@Bean
GracefulShutdownListener gracefulShutdownListener(
@Value("${server.shutdown-wait-seconds:30}") int shutdownWaitSeconds) {
GracefulShutdownListener gracefulShutdownListener = new GracefulShutdownListener();
gracefulShutdownListener.setShutdownWaitSeconds(shutdownWaitSeconds);
return gracefulShutdownListener;
}
@Slf4j
public static class GracefulShutdownListener
implements TomcatConnectorCustomizer, ApplicationListener, PriorityOrdered {
private Connector connector;
private int shutdownWaitSeconds;
public void setShutdownWaitSeconds(int shutdownWaitSeconds) {
this.shutdownWaitSeconds = shutdownWaitSeconds;
}
@Override
public void customize(Connector connector) {
this.connector = connector;
}
@Override
public void onApplicationEvent(ContextClosedEvent contextClosedEvent) {
this.connector.pause();
Executor executor = this.connector.getProtocolHandler().getExecutor();
if (executor instanceof ThreadPoolExecutor) {
log.info("start to shutdown tomcat server,executor:{}", executor);
try {
ThreadPoolExecutor threadPoolExecutor = (ThreadPoolExecutor) executor;
threadPoolExecutor.shutdown();
if (!threadPoolExecutor.awaitTermination(shutdownWaitSeconds, TimeUnit.SECONDS)) {
log.warn(
"tomcat thread pool did not shutdown gracefully within {} seconds. Proceeding with forceful shutdown",
shutdownWaitSeconds);
}
} catch (Exception e) {
log.error("stop tomcat graceful exception", e);
Thread.currentThread().interrupt();
}
log.info("tomcat server stopped");
}
}
@Override
public int getOrder() {
return Ordered.HIGHEST_PRECEDENCE;
}
}
}