线程、数据库、算法、JVM、分布式、微服务、框架、Spring相关知识
一线互联网P7面试集锦+各种大厂面试集锦
学习笔记以及面试真题解析
1.2 xxl-job-core
该代码为服务集成的核心源码包,包括重要注解,执行器等。
1.3 xxl-job-executor-samples
该包为服务集成测试demo,支持frameless,jfinal,spring,SpringBoot等多个框架集成。
2、架构设计
======
2.1 整体架构设计
通过服务注册的方式,将job注册到任务调度中心,通过调度中心进行统一的任务管理
2.2 服务集成
以SpringBoot集成xxl-job为例,集成直接运行demo即可,可以参考01.XXL-JOB这个文章。
1、配置文件
|
server.port=8083
no web
#spring.main.web-environment=false
log config
logging.config=classpath:logback.xml
xxl-job注册中心地址
xxl.job.admin.addresses=http://127.0.0.1:8080/xxl-job-admin
xxl-job access token
xxl.job.accessToken=
xxl-job 执行器名称
xxl.job.executor.appname=omsJob
xxl-job executor 执行器ip
xxl.job.executor.ip=
xxl-job executor 执行器端口
xxl.job.executor.port=9999
xxl-job executor log-path 日志配置
xxl.job.executor.logpath=/data/applogs/xxl-job/jobhandler
xxl-job executor log-retention-days 日志定时清理时间
xxl.job.executor.logretentiondays=30
2、初始化执行器
|
public XxlJobSpringExecutor xxlJobExecutor() {
logger.info(“>>>>>>>>>>> xxl-job config init.”);
XxlJobSpringExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
xxlJobSpringExecutor.setAdminAddresses(adminAddresses);
xxlJobSpringExecutor.setAppname(appname);
xxlJobSpringExecutor.setAddress(address);
xxlJobSpringExecutor.setIp(ip);
xxlJobSpringExecutor.setPort(port);
xxlJobSpringExecutor.setAccessToken(accessToken);
xxlJobSpringExecutor.setLogPath(logPath);
xxlJobSpringExecutor.setLogRetentionDays(logRetentionDays);
return xxlJobSpringExecutor;
}
3、使用xxl-job注解配置任务
|
- 1、简单任务示例(Bean模式)
*/
@XxlJob(“demoJobHandler”)
public ReturnT<String> demoJobHandler(String param) throws Exception {
XxlJobLogger.log(“XXL-JOB, Hello World.”);
for (int i = 0; i < 5; i++) {
XxlJobLogger.log(“beat at:” + i);
TimeUnit.SECONDS.sleep(2);
}
return ReturnT.SUCCESS;
}
4、启动成功后,服务会开启两个端口,一个是业务端口,一个是调度中心下发job执行的端口。(此处要优化,服务和job触发其实使用一个端口即可)
3、启动源码分析
========
1、服务启动流程
|
public XxlJobSpringExecutor xxlJobExecutor() {
logger.info(“>>>>>>>>>>> xxl-job config init.”);
XxlJobSpringExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
xxlJobSpringExecutor.setAdminAddresses(adminAddresses);
xxlJobSpringExecutor.setAppname(appname);
xxlJobSpringExecutor.setAddress(address);
xxlJobSpringExecutor.setIp(ip);
xxlJobSpringExecutor.setPort(port);
xxlJobSpringExecutor.setAccessToken(accessToken);
xxlJobSpringExecutor.setLogPath(logPath);
xxlJobSpringExecutor.setLogRetentionDays(logRetentionDays);
return xxlJobSpringExecutor;
}
服务主要启动是初始化XxlJobSpringExecutor这个bean对象,该对象定义了执行器xxlJobSpringExecutor的相关配置,如注册中心地址,服务提供地址,以及授权token等。该对象类图如下,
XxlJobSpringExecutor执行器继承XxlJobExecutor并实现Spring的ApplicationContextAware等类,对该bean进行了增强。主要核心类为XxlJobExecutor。
因为继承SmartInitializingSingleton,所以bean初始化完成之后会执行afterSingletonsInstantiated方法,该类主要为initJobHandlerMethodRepository这个方法,用于扫描xxl-job注解,进行任务加载和管理
|
@Override
public void afterSingletonsInstantiated() {
// 扫描xxl-job注解,进行任务加载和管理
initJobHandlerMethodRepository(applicationContext);
// refresh GlueFactory
GlueFactory.refreshInstance(1);
// super start
try {
super.start();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
initJobHandlerMethodRepository方法主要如下
该方法主要是通过获取Spring管理的容器bean,然后扫描带有xxljob注解的方法,将他们保存在jobHandlerRepository对象中
|
if (applicationContext == null) {
return;
}
// 扫描Spring管理的bean
String[] beanDefinitionNames = applicationContext.getBeanNamesForType(Object.class, false, true);
for (String beanDefinitionName : beanDefinitionNames) {
Object bean = applicationContext.getBean(beanDefinitionName);
Map<Method, XxlJob> annotatedMethods = null; // referred to :org.springframework.context.event.EventListenerMethodProcessor.processBean
try {
annotatedMethods = MethodIntrospector.selectMethods(bean.getClass(),
new MethodIntrospector.MetadataLookup<XxlJob>() {
@Override
public XxlJob inspect(Method method) {
// 获取注解为XxlJob的方法,并保存在annotatedMethods
return AnnotatedElementUtils.findMergedAnnotation(method, XxlJob.class);
}
});
} catch (Throwable ex) {
logger.error(“xxl-job method-jobhandler resolve error for bean[” + beanDefinitionName + “].”, ex);
}
if (annotatedMethods==null || annotatedMethods.isEmpty()) {
continue;
}
//获取方法属性,并存储在jobHandlerRepository对象中
for (Map.Entry<Method, XxlJob> methodXxlJobEntry : annotatedMethods.entrySet()) {
Method method = methodXxlJobEntry.getKey();
XxlJob xxlJob = methodXxlJobEntry.getValue();
if (xxlJob == null) {
continue;
}
String name = xxlJob.value();
if (name.trim().length() == 0) {
throw new RuntimeException(“xxl-job method-jobhandler name invalid, for[” + bean.getClass() + “#” + method.getName() + “] .”);
}
if (loadJobHandler(name) != null) {
throw new RuntimeException(“xxl-job jobhandler[” + name + “] naming conflicts.”);
}
// execute method
if (!(method.getParameterTypes().length == 1 && method.getParameterTypes()[0].isAssignableFrom(String.class))) {
throw new RuntimeException(“xxl-job method-jobhandler param-classtype invalid, for[” + bean.getClass() + “#” + method.getName() + "] , " +
“The correct method format like " public ReturnT<String> execute(String param) " .”);
}
if (!method.getReturnType().isAssignableFrom(ReturnT.class)) {
throw new RuntimeException(“xxl-job method-jobhandler return-classtype invalid, for[” + bean.getClass() + “#” + method.getName() + "] , " +
“The correct method format like " public ReturnT<String> execute(String param) " .”);
}
method.setAccessible(true);
// init and destory
Method initMethod = null;
Method destroyMethod = null;
if (xxlJob.init().trim().length() > 0) {
try {
initMethod = bean.getClass().getDeclaredMethod(xxlJob.init());
initMethod.setAccessible(true);
} catch (NoSuchMethodException e) {
throw new RuntimeException(“xxl-job method-jobhandler initMethod invalid, for[” + bean.getClass() + “#” + method.getName() + “] .”);
}
}
if (xxlJob.destroy().trim().length() > 0) {
try {
destroyMethod = bean.getClass().getDeclaredMethod(xxlJob.destroy());
destroyMethod.setAccessible(true);
} catch (NoSuchMethodException e) {
throw new RuntimeException(“xxl-job method-jobhandler destroyMethod invalid, for[” + bean.getClass() + “#” + method.getName() + “] .”);
}
}
// registry jobhandler
//将任务存储在jobHandlerRepository对象中,后续下发任务使用
registJobHandler(name, new MethodJobHandler(bean, method, initMethod, destroyMethod));
}
}
}
之后执行super.start(),执行父类XxlJobExecutor的start方法。该方法主要有日志初始化,日志清理任务初始化,RPC调用触发器回调线程启动,调度中心列表初始化以及执行器端口初始化。
|
//初始化任务执行日志路径
XxlJobFileAppender.initLogPath(logPath);
// 日志定时清理任务
JobLogFileCleanThread.getInstance().start(logRetentionDays);
// 初始化触发器回调线程(用RPC回调调度中心接口)
TriggerCallbackThread.getInstance().start();
//初始化调度中心列表
initAdminBizList( adminAddresses, accessToken);
// init executor-server 执行器端口启动
initEmbedServer(address, ip, port, appname, accessToken);
}
XxlJobFileAppender.initLogPath(logPath)和JobLogFileCleanThread.getInstance().start(logRetentionDays)主要对执行日志进行初始化,就不多解释了,直接往下看。
TriggerCallbackThread.getInstance().start();
|
// 调度中心注册表会否为空
if (XxlJobExecutor.getAdminBizList() == null) {
logger.warn(“>>>>>>>>>>> xxl-job, executor callback config fail, adminAddresses is null.”);
return;
}
// callback
triggerCallbackThread = new Thread(new Runnable() {
@Override
public void run() {
// 监听阻塞队列
while(!toStop){
try {
HandleCallbackParam callback = getInstance().callBackQueue.take();
if (callback != null) {
// 组装callback返回的参数
List<HandleCallbackParam> callbackParamList = new ArrayList<HandleCallbackParam>();
int drainToNum = getInstance().callBackQueue.drainTo(callbackParamList);
callbackParamList.add(callback);
// 执行回调
if (callbackParamList!=null && callbackParamList.size()>0) {
doCallback(callbackParamList);
}
}
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
}
// last callback
try {
List<HandleCallbackParam> callbackParamList = new ArrayList<HandleCallbackParam>();
int drainToNum = getInstance().callBackQueue.drainTo(callbackParamList);
if (callbackParamList!=null && callbackParamList.size()>0) {
doCallback(callbackParamList);
}
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
logger.info(“>>>>>>>>>>> xxl-job, executor callback thread destory.”);
}
});
triggerCallbackThread.setDaemon(true);
triggerCallbackThread.setName(“xxl-job, executor TriggerCallbackThread”);
triggerCallbackThread.start();
// retry
triggerRetryCallbackThread = new Thread(new Runnable() {
@Override
public void run() {
while(!toStop){
try {
retryFailCallbackFile();
} catch (Exception e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
try {
TimeUnit.SECONDS.sleep(RegistryConfig.BEAT_TIMEOUT);
} catch (InterruptedException e) {
if (!toStop) {
logger.error(e.getMessage(), e);
}
}
}
logger.info(“>>>>>>>>>>> xxl-job, executor retry callback thread destory.”);
}
});
triggerRetryCallbackThread.setDaemon(true);
triggerRetryCallbackThread.start();
}
doCallback(callbackParamList)如下
|
-
do callback, will retry if error
-
@param callbackParamList
*/
private void doCallback(List<HandleCallbackParam> callbackParamList){
boolean callbackRet = false;
// 向所有的调度中心发送回调信息
for (AdminBiz adminBiz: XxlJobExecutor.getAdminBizList()) {
try {
//本质上是调用注册中心的api/callback接口。记录调用结果。
ReturnT<String> callbackResult = adminBiz.callback(callbackParamList);
if (callbackResult!=null && ReturnT.SUCCESS_CODE == callbackResult.getCode()) {
callbackLog(callbackParamList, “<br>----------- xxl-job job callback finish.”);
callbackRet = true;
break;
} else {
callbackLog(callbackParamList, “<br>----------- xxl-job job callback fail, callbackResult:” + callbackResult);
}
} catch (Exception e) {
callbackLog(callbackParamList, “<br>----------- xxl-job job callback error, errorMsg:” + e.getMessage());
}
}
if (!callbackRet) {
appendFailCallbackFile(callbackParamList);
}
}
adminBiz.callback(callbackParamList)
调用注册中心api接口
|
public ReturnT<String> callback(List<HandleCallbackParam> callbackParamList) {
return XxlJobRemotingUtil.postBody(addressUrl+“api/callback”, accessToken, timeout, callbackParamList, String.class);
}
initAdminBizList( adminAddresses, accessToken); 初始化注册中心列表,用于后期和注册中心交互
|
private void initAdminBizList(String adminAddresses, String accessToken) throws Exception {
if (adminAddresses!=null && adminAddresses.trim().length()>0) {
for (String address: adminAddresses.trim().split(“,”)) {
if (address!=null && address.trim().length()>0) {
AdminBiz adminBiz = new AdminBizClient(address.trim(), accessToken);
if (adminBizList == null) {
adminBizList = new ArrayList<AdminBiz>();
}
adminBizList.add(adminBiz);
}
}
}
}
// init executor-server
initEmbedServer(address, ip, port, appname, accessToken);<核心>
|
private void initEmbedServer(String address, String ip, int port, String appname, String accessToken) throws Exception {
//初始化ip和端口,如果没有ip则自动获取本地ip
port = port>0?port: NetUtil.findAvailablePort(9999);
ip = (ip!=null&&ip.trim().length()>0)?ip: IpUtil.getIp();
// generate address
if (address==null || address.trim().length()==0) {
String ip_port_address = IpUtil.getIpPort(ip, port); // registry-address:default use address to registry , otherwise use ip:port if address is null
address = “http://{ip_port}/”.replace(“{ip_port}”, ip_port_address);
}
// 启动服务
embedServer = new EmbedServer();
embedServer.start(address, port, appname, accessToken);
}
embedServer.start(address, port, appname, accessToken); 本质上是一个Netty服务,标准的Netty服务启动,我们只看EmbedHttpServerHandler,Netty处理请求的handler
|
executorBiz = new ExecutorBizImpl();
thread = new Thread(new Runnable() {
@Override
public void run() {
// param
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
ThreadPoolExecutor bizThreadPool = new ThreadPoolExecutor(
0,
200,
60L,
TimeUnit.SECONDS,
new LinkedBlockingQueue<Runnable>(2000),
new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
return new Thread(r, “xxl-rpc, EmbedServer bizThreadPool-” + r.hashCode());
}
},
new RejectedExecutionHandler() {
@Override
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
throw new RuntimeException(“xxl-job, EmbedServer bizThreadPool is EXHAUSTED!”);
}
});
try {
// start server
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childHandler(new ChannelInitializer<SocketChannel>() {
@Override
public void initChannel(SocketChannel channel) throws Exception {
channel.pipeline()
.addLast(new IdleStateHandler(0, 0, 30 * 3, TimeUnit.SECONDS)) // beat 3N, close if idle
.addLast(new HttpServerCodec())
.addLast(new HttpObjectAggregator(5 * 1024 * 1024)) // merge request & reponse to FULL
.addLast(new EmbedHttpServerHandler(executorBiz, accessToken, bizThreadPool));
}
})
.childOption(ChannelOption.SO_KEEPALIVE, true);
// bind
ChannelFuture future = bootstrap.bind(port).sync();
logger.info(“>>>>>>>>>>> xxl-job remoting server start success, nettype = {}, port = {}”, EmbedServer.class, port);
//注册到调度中心
startRegistry(appname, address);
// wait util stop
future.channel().closeFuture().sync();
} catch (InterruptedException e) {
if (e instanceof InterruptedException) {
logger.info(“>>>>>>>>>>> xxl-job remoting server stop.”);
} else {
logger.error(“>>>>>>>>>>> xxl-job remoting server error.”, e);
}
} finally {
// stop
try {
workerGroup.shutdownGracefully();
bossGroup.shutdownGracefully();
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
}
}
});
thread.setDaemon(true); // daemon, service jvm, user thread leave >>> daemon leave >>> jvm leave
thread.start();
}
|
private static final Logger logger = LoggerFactory.getLogger(EmbedHttpServerHandler.class);
private ExecutorBiz executorBiz; //执行器
private String accessToken; //token
private ThreadPoolExecutor bizThreadPool;//执行器线程池
public EmbedHttpServerHandler(ExecutorBiz executorBiz, String accessToken, ThreadPoolExecutor bizThreadPool) {
this.executorBiz = executorBiz;
this.accessToken = accessToken;
this.bizThreadPool = bizThreadPool;
}
@Override
protected void channelRead0(final ChannelHandlerContext ctx, FullHttpRequest msg) throws Exception {
// request parse
//final byte[] requestBytes = ByteBufUtil.getBytes(msg.content()); // byteBuf.toString(io.netty.util.CharsetUtil.UTF_8);
String requestData = msg.content().toString(CharsetUtil.UTF_8);//获取请求数据
String uri = msg.uri();
HttpMethod httpMethod = msg.method();
boolean keepAlive = HttpUtil.isKeepAlive(msg);
String accessTokenReq = msg.headers().get(XxlJobRemotingUtil.XXL_JOB_ACCESS_TOKEN);
// invoke
bizThreadPool.execute(new Runnable() {
@Override
public void run() {
// 处理请求
Object responseObj = process(httpMethod, uri, requestData, accessTokenReq);
// 格式化为JSON
String responseJson = GsonTool.toJson(responseObj);
// 写回客户端
writeResponse(ctx, keepAlive, responseJson);
}
});
}
最后
这份《“java高分面试指南”-25分类227页1000+题50w+字解析》同样可分享给有需要的朋友,感兴趣的伙伴们可挑战一下自我,在不看答案解析的情况,测试测试自己的解题水平,这样也能达到事半功倍的效果!(好东西要大家一起看才香)
uf.toString(io.netty.util.CharsetUtil.UTF_8);
String requestData = msg.content().toString(CharsetUtil.UTF_8);//获取请求数据
String uri = msg.uri();
HttpMethod httpMethod = msg.method();
boolean keepAlive = HttpUtil.isKeepAlive(msg);
String accessTokenReq = msg.headers().get(XxlJobRemotingUtil.XXL_JOB_ACCESS_TOKEN);
// invoke
bizThreadPool.execute(new Runnable() {
@Override
public void run() {
// 处理请求
Object responseObj = process(httpMethod, uri, requestData, accessTokenReq);
// 格式化为JSON
String responseJson = GsonTool.toJson(responseObj);
// 写回客户端
writeResponse(ctx, keepAlive, responseJson);
}
});
}
最后
这份《“java高分面试指南”-25分类227页1000+题50w+字解析》同样可分享给有需要的朋友,感兴趣的伙伴们可挑战一下自我,在不看答案解析的情况,测试测试自己的解题水平,这样也能达到事半功倍的效果!(好东西要大家一起看才香)
[外链图片转存中…(img-pVd9DMJ4-1715093910829)]
[外链图片转存中…(img-UwyEWucZ-1715093910829)]